Theory of Operation

We designed and implemented our provisioning engine from the ground up to solve problems associated with cloud provisioning. We hope you enjoy reading about the "ity"s as much as we have spent time and effort solving them!

Portability

One of the most deceptive marketing tactics for infrastructure provisioning systems is to claim that it is a multi-cloud solution. The truth is almost always quite different. Cloud Provisioning systems before Tuono might be considered "multi-cloud capable". The cloud-specific rules and API calls are exposed through those systems without a filter, and the onus falls upon the consumer to spend the time to learn and use them properly. The end result is that the consumer ends up locked into one cloud provider because it becomes too expensive to rewrite scripts and learn new rules, even though they used a system that claimed to be "multi-cloud".

We have developed the first cloud provider provisioning abstraction layer. You can describe the resources you need in a cloud-agnostic, easy-to-use text language and then provision those resources to any supported cloud provider, all without making any changes to your infrastructure definition.

For example, this blueprint can be used to create a virtual network and subnet in either AWS or Azure:

location:
  region:
    preferred:
      country: USA
      area: northwest
  folder:
    overview-example:
      region: preferred

networking:
  network:
    overview-network:
      folder: overview-example
      range: 10.0.0.0/16
  subnet:
    overview-subnet:
      network: overview-network
      range: 10.0.0.0/24

Identity (The Source of Truth)

We firmly believe that the only source of truth can be the venue itself. At any point in time if we need to know if something exists or is configured in a particular way, we ask the venue about it. We never make provisioning decisions based on cached or stale state.

Security

Often times, provisioning requires secrets that are incredibly sensitive. Keeping that information safe is of paramount importance to our customers and to us. We allocate a dedicated Vault to each and every one of our paid subscribers. This is a fully independent vault instance running in our infrastructure. We handle your secrets in a way that ensures the window of opportunity where we have access to the sensitive content is extremely narrow.

Integrity (Fail Early)

We avoid late failures that leave infrastructure in an indeterminate state. Our venue simulation layer makes internal changes to venue models as if they had been executed. We code the venue provisioning rules into our venue object layer and then test it in many ways to guarantee it is behaving just like the venue itself. Here are some examples of cases where we will fail early and will not even start provisioning:

• A subnet is removed from a blueprint but it is still being used by other resources.

• A subnet has a range that falls outside the network range.

• A virtual machine has a name that is longer than the venue supports.

In these cases (and many more not shown here), where the venue would eventually reject the provisioning request, we detect it early so that we do not even start making changes.

Reliability (Test Everything)

We know from experience that untested code is broken code. Testing is built into our DNA. We perform isolation unit testing, modular integration testing, component level interface testing, and holistic system testing. How do you know when you are done testing? The truth is, you are never done testing software. By measuring branch coverage, you can gauge how much you purposefully (or accidentally) test, and where you need to focus on adding more tests to make your product more reliable:

Component	on	had branch coverage of...
AWS Adapter	August 20, 2020	88.26%
Azure Adapter	August 20, 2020	86.88%
Provisioning Engine	August 20, 2020	97.84%
Vault Management	August 20 2020	100.00%

Reproducability (Idempotence)

Idempotence is a fancy word that basically says, “performing the same operation multiple times produces the same result.” In our case, this means applying the same blueprint to the same venue more than once will produce the same result. If something does not need to change then we will not change it.

Idempotence is really important, because unnecessary changes lead to excessive cost or unintended service interruptions. We took one part of Idempotence and cooked it together with one part of "Test Everything" and the end result is a testing pattern that we apply liberally at all layers of our solution.

For every case of a create, update, or delete of an object, we test:

• Preview (simulate) the action, and verify that changes would occur.

• Preview (simulate) the action again, and verify no changes would occur.

• Perform that action live, and verify that changes occur.

• Perform that action live again, and verify that no changes occur.

• Preview (simulate) the action again, and verify no changes would occur.

The last entry may appear to be a "cut & paste error", but it is crucial to ensuring that our preview (simulation) layer is accurate with respect to the source of truth.