VMware Cloud on AWS - Part 2 - Automated Deployment with Terraform

Written By Ralf Frankemölle

Welcome back to my next post on Securefever! If you missed our manual deployment guide for VMC on AWS, you can catch up here:
https://www.securefever.com/vmc-aws-manual-deployment

It took a while for the second post because I’ve had the great chance to write a few articles for the renowned German IT magazine “IT Administrator”, develop a new VMware exam, followed by my Take 3 at VMware.

What is “Take 3”? Great question!
The "Take 3" program allows eligible employees to take a break from their current role and explore a different role or project within the company for a duration of roughly three months. The intention behind the program is to foster a culture of continuous learning, enable career development, and encourage cross-functional collaboration within the company.

Read more on that in the dedicated blog here:
https://www.securefever.com/blog/off-topic-take-3

As promised in the previous blog, this post focuses on automating our VMC on AWS deployment using Terraform. Why? Because automation is king in the cloud! It promotes repeatability, scalability, and defining our infrastructure as code, ensures consistency while reducing human error.

I am very much still learning Terraform, so let me know about suggestions to improve the code, structure, etc. Also, since this blog was in the making for quite a while, some screenshots might show outdated versions of providers.

Let's get started!

Why Terraform?

Terraform is a popular Infrastructure as Code (IaC) tool, which allows users to define and provision infrastructure using a declarative configuration language (HCL, Hashicorp Configuration Language). Its interoperability with both AWS and VMC makes it an ideal choice. 

Prerequisites

- Familiarity with Terraform. If you're new, check out Terraform's Getting Started Guide (https://learn.hashicorp.com/terraform/getting-started/install.html).
- Terraform installed on your machine.
- Necessary permissions and credentials set up for both AWS and VMC.

Steps to Deploy VMC on AWS using Terraform

1. Setting Up Terraform

I am using MacOS, so I used brew to install terraform on my machine:

brew install terraform

Once that is done, we can check the installed terraform version:

“terraform version” output

2. Terraform Files

For the deployment I have set up multiple files.
Splitting terraform deployments into multiple files makes it easier to troubleshoot and also, I wanted to start early with good habits.

Next to main.tf I created “provider.tf”, “variables.tf”, “version.tf”, “sddc.tf” and “vmc_vpc.tf” and “terraform.tfvars” in my project.
Let’s review the different files and what they are used for.

main.tf and module sources

the main.tf file

The main.tf file is the primary entry point for Terraform configurations. It contains the core resources and infrastructure definitions that Terraform will manage. In big projects, main.tf might be split into multiple files. For our use case the single main.tf file is fine.

The first block defines where the Terraform state file will be stored. In our case, it’s on the “local backend”. That means the state will be kept in my local file system, under “../../phase1.tfstate”.

The second block contains a “module” definition. A module in Terraform is like a reusable function for infrastructure code. It groups related resources to better organize our code. Each module has a source showing where its content is found (source = “..”/vmc_vpc”).
This module is used to configure the AWS VPC that we use for our VMC on AWS deployments.
It holds configuration for the region, VPC CIDR and subnet CIDRs, all based on variables found in the module source file:

vmc_vpc.tf - the file configuring the AWS VPC module

In this file we define the resources that will be configured in main.tf.
It starts with some variable definitions. That way we can re-use the structure for different deployment by changing the contents of “variables.tf”.

Next, we define an “aws_vpc” and three “aws_subnet” resources.
These reference our connected VPC and the connected VPC subnets.

The module source file pulls the relevant content from “variables.tf”:

image of variables.tf - this is where the module files get their variables content

In the last block we create another module, but this time for the VMC SDDC configuration. The logic remains the same, though this time the variable contents are found in the source “../sddc”:

the module for the sddc creation

This module source also pulls content from variables.tf and configures a few more options. As this is a demo environment again, we are using the single node i3 deployment. That is why we have “host_instance_type” set to “I3_METAL” and the “sddc_type” to “1NODE”.

provider.tf

In provider.tf we specify and configure the Terraform providers we use in our project (AWS, VMC). It centralizes provider configuration, to make our Terraform project more organized.

content of provider.tf

versions.tf

The `versions.tf` file in a Terraform deployment is typically used to specify the versions of Terraform itself and the providers. This ensures consistency and compatibility across different machines – who doesn’t love a good old “works on my machine” error. Our “versions.tf” looks like this:

In the future this will be updated to constrain the allowed versions of the used providers.

Tokens, Authentication, Privileges

We have seen all the different configurations that we have Terraform do. However, we did not specify credentials or similar anywhere. So how do we actually “log in” and create these resources? The last file we need to look at is “terraform.tfvars”. This file contains all the secrets and token information that Terraform uses to operate with the configured providers. I do not recommend storing them as plain text on your machine, but I have not yet explored secrets management any further.
So, for this time, this must be good enough. This is sample content of “terraform.tfvars” in my current project:

example output of terraform.tfvars - the file I currently store secrets in

3. Initializing Terraform

Now that we have explored the different parts of our Terraform deployment, we can initialize Terraform with running “terraform init” in the folder with the config files:

rfrankemolle@rfrankemolFV7TR main % terraform init

Initializing the backend...
Initializing modules...

Initializing provider plugins...
- Reusing previous version of terraform-providers/vmc from the dependency lock file
- Reusing previous version of vmware/nsxt from the dependency lock file
- Reusing previous version of hashicorp/aws from the dependency lock file
- Using previously-installed vmware/nsxt v3.3.0
- Using previously-installed hashicorp/aws v4.58.0
- Using previously-installed terraform-providers/vmc v1.13.0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

The command is used to perform several tasks like install provider plugins, setting up modules, initialise the backend (the location where state files are stored) and set up the working directory to run other commands like “plan” or “apply”.

4. Plan and Apply

After initializing, we run “terraform plan”. Terraform will output what it is going to deploy / change in the infrastructure:

rfrankemolle@rfrankemolFV7TR main % terraform plan
module.vmc_vpc.data.aws_availability_zones.az: Reading...
module.vmc_vpc.data.aws_availability_zones.az: Read complete after 0s [id=eu-west-2]
module.sddc.data.vmc_connected_accounts.my_accounts: Reading...
module.sddc.data.vmc_connected_accounts.my_accounts: Read complete after 0s [id=42686b6b-163d-3465-a953-09b3da081d31]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # module.sddc.vmc_sddc.vmc_sddc1 will be created
  + resource "vmc_sddc" "vmc_sddc1" {
      + account_link_state       = (known after apply)
      + availability_zones       = (known after apply)
      + cloud_password           = (known after apply)
      + cloud_username           = (known after apply)
      + cluster_id               = (known after apply)
      + cluster_info             = (known after apply)
      + created                  = (known after apply)
      + delay_account_link       = false
      + deployment_type          = "SingleAZ"
      + edrs_policy_type         = (known after apply)
      + enable_edrs              = (known after apply)
      + host_instance_type       = "I3_METAL"
      + id                       = (known after apply)
      + intranet_mtu_uplink      = 1500
      + max_hosts                = (known after apply)
      + min_hosts                = (known after apply)
      + nsxt_cloudadmin          = (known after apply)
      + nsxt_cloudadmin_password = (known after apply)
      + nsxt_cloudaudit          = (known after apply)
      + nsxt_cloudaudit_password = (known after apply)
      + nsxt_private_ip          = (known after apply)
      + nsxt_private_url         = (known after apply)
      + nsxt_reverse_proxy_url   = (known after apply)
      + nsxt_ui                  = (known after apply)
      + num_host                 = 1
      + org_id                   = (known after apply)
      + provider_type            = "AWS"
      + region                   = "EU_WEST_2"
      + sddc_access_state        = (known after apply)
      + sddc_name                = "rfrankemolle_tf_test"
      + sddc_size                = (known after apply)
      + sddc_state               = (known after apply)
      + sddc_type                = "1NODE"
      + size                     = "medium"
      + skip_creating_vxlan      = false
      + sso_domain               = "vmc.local"
      + updated                  = (known after apply)
      + updated_by_user_id       = (known after apply)
      + updated_by_user_name     = (known after apply)
      + user_id                  = (known after apply)
      + user_name                = (known after apply)
      + vc_url                   = (known after apply)
      + version                  = (known after apply)
      + vpc_cidr                 = "10.20.0.0/16"
      + vxlan_subnet             = "10.100.100.0/24"

      + account_link_sddc_config {
          + connected_account_id = "42686b6b-163d-3465-a953-09b3da081d31"
          + customer_subnet_ids  = (known after apply)
        }

      + timeouts {
          + create = "300m"
          + delete = "180m"
          + update = "300m"
        }
    }

  # module.vmc_vpc.aws_subnet.con_vpc_subnet1 will be created
  + resource "aws_subnet" "con_vpc_subnet1" {
      + arn                                            = (known after apply)
      + assign_ipv6_address_on_creation                = false
      + availability_zone                              = "eu-west-2a"
      + availability_zone_id                           = (known after apply)
      + cidr_block                                     = "10.10.0.64/26"
      + enable_dns64                                   = false
      + enable_resource_name_dns_a_record_on_launch    = false
      + enable_resource_name_dns_aaaa_record_on_launch = false
      + id                                             = (known after apply)
      + ipv6_cidr_block_association_id                 = (known after apply)
      + ipv6_native                                    = false
      + map_public_ip_on_launch                        = true
      + owner_id                                       = (known after apply)
      + private_dns_hostname_type_on_launch            = (known after apply)
      + tags                                           = {
          + "Name" = "rf_connected_vpc_subnet1"
        }
      + tags_all                                       = {
          + "Name" = "rf_connected_vpc_subnet1"
        }
      + vpc_id                                         = (known after apply)
    }

  # module.vmc_vpc.aws_subnet.con_vpc_subnet2 will be created
  + resource "aws_subnet" "con_vpc_subnet2" {
      + arn                                            = (known after apply)
      + assign_ipv6_address_on_creation                = false
      + availability_zone                              = "eu-west-2b"
      + availability_zone_id                           = (known after apply)
      + cidr_block                                     = "10.10.0.128/26"
      + enable_dns64                                   = false
      + enable_resource_name_dns_a_record_on_launch    = false
      + enable_resource_name_dns_aaaa_record_on_launch = false
      + id                                             = (known after apply)
      + ipv6_cidr_block_association_id                 = (known after apply)
      + ipv6_native                                    = false
      + map_public_ip_on_launch                        = true
      + owner_id                                       = (known after apply)
      + private_dns_hostname_type_on_launch            = (known after apply)
      + tags                                           = {
          + "Name" = "rf_connected_vpc_subnet2"
        }
      + tags_all                                       = {
          + "Name" = "rf_connected_vpc_subnet2"
        }
      + vpc_id                                         = (known after apply)
    }

  # module.vmc_vpc.aws_subnet.con_vpc_subnet3 will be created
  + resource "aws_subnet" "con_vpc_subnet3" {
      + arn                                            = (known after apply)
      + assign_ipv6_address_on_creation                = false
      + availability_zone                              = "eu-west-2c"
      + availability_zone_id                           = (known after apply)
      + cidr_block                                     = "10.10.0.192/26"
      + enable_dns64                                   = false
      + enable_resource_name_dns_a_record_on_launch    = false
      + enable_resource_name_dns_aaaa_record_on_launch = false
      + id                                             = (known after apply)
      + ipv6_cidr_block_association_id                 = (known after apply)
      + ipv6_native                                    = false
      + map_public_ip_on_launch                        = true
      + owner_id                                       = (known after apply)
      + private_dns_hostname_type_on_launch            = (known after apply)
      + tags                                           = {
          + "Name" = "rf_connected_vpc_subnet3"
        }
      + tags_all                                       = {
          + "Name" = "rf_connected_vpc_subnet3"
        }
      + vpc_id                                         = (known after apply)
    }

  # module.vmc_vpc.aws_vpc.con_vpc will be created
  + resource "aws_vpc" "con_vpc" {
      + arn                                  = (known after apply)
      + cidr_block                           = "10.10.0.0/16"
      + default_network_acl_id               = (known after apply)
      + default_route_table_id               = (known after apply)
      + default_security_group_id            = (known after apply)
      + dhcp_options_id                      = (known after apply)
      + enable_classiclink                   = (known after apply)
      + enable_classiclink_dns_support       = (known after apply)
      + enable_dns_hostnames                 = true
      + enable_dns_support                   = true
      + enable_network_address_usage_metrics = (known after apply)
      + id                                   = (known after apply)
      + instance_tenancy                     = "default"
      + ipv6_association_id                  = (known after apply)
      + ipv6_cidr_block                      = (known after apply)
      + ipv6_cidr_block_network_border_group = (known after apply)
      + main_route_table_id                  = (known after apply)
      + owner_id                             = (known after apply)
      + tags                                 = {
          + "Name" = "rf_connected_vpc"
        }
      + tags_all                             = {
          + "Name" = "rf_connected_vpc"
        }
    }

Plan: 5 to add, 0 to change, 0 to destroy.

If we are happy with the output, we run “terraform apply”. This will start the deployment by terraform:

rfrankemolle@rfrankemolFV7TR main % terraform apply
module.vmc_vpc.data.aws_availability_zones.az: Reading...
module.vmc_vpc.data.aws_availability_zones.az: Read complete after 1s [id=eu-west-2]
module.sddc.data.vmc_connected_accounts.my_accounts: Reading...
module.sddc.data.vmc_connected_accounts.my_accounts: Read complete after 1s [id=42686b6b-163d-3465-a953-09b3da081d31]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # module.sddc.vmc_sddc.vmc_sddc1 will be created
  + resource "vmc_sddc" "vmc_sddc1" {
      + account_link_state       = (known after apply)
      + availability_zones       = (known after apply)
      + cloud_password           = (known after apply)
      + cloud_username           = (known after apply)
      + cluster_id               = (known after apply)
      + cluster_info             = (known after apply)
      + created                  = (known after apply)
      + delay_account_link       = false
      + deployment_type          = "SingleAZ"
      + edrs_policy_type         = (known after apply)
      + enable_edrs              = (known after apply)
      + host_instance_type       = "I3_METAL"
      + id                       = (known after apply)
      + intranet_mtu_uplink      = 1500
      + max_hosts                = (known after apply)
      + min_hosts                = (known after apply)
      + nsxt_cloudadmin          = (known after apply)
      + nsxt_cloudadmin_password = (known after apply)
      + nsxt_cloudaudit          = (known after apply)
      + nsxt_cloudaudit_password = (known after apply)
      + nsxt_private_ip          = (known after apply)
      + nsxt_private_url         = (known after apply)
      + nsxt_reverse_proxy_url   = (known after apply)
      + nsxt_ui                  = (known after apply)
      + num_host                 = 1
      + org_id                   = (known after apply)
      + provider_type            = "AWS"
      + region                   = "EU_WEST_2"
      + sddc_access_state        = (known after apply)
      + sddc_name                = "rfrankemolle_tf_test"
      + sddc_size                = (known after apply)
      + sddc_state               = (known after apply)
      + sddc_type                = "1NODE"
      + size                     = "medium"
      + skip_creating_vxlan      = false
      + sso_domain               = "vmc.local"
      + updated                  = (known after apply)
      + updated_by_user_id       = (known after apply)
      + updated_by_user_name     = (known after apply)
      + user_id                  = (known after apply)
      + user_name                = (known after apply)
      + vc_url                   = (known after apply)
      + version                  = (known after apply)
      + vpc_cidr                 = "10.20.0.0/16"
      + vxlan_subnet             = "10.100.100.0/24"

      + account_link_sddc_config {
          + connected_account_id = "42686b6b-163d-3465-a953-09b3da081d31"
          + customer_subnet_ids  = (known after apply)
        }

      + timeouts {
          + create = "300m"
          + delete = "180m"
          + update = "300m"
        }
    }

  # module.vmc_vpc.aws_subnet.con_vpc_subnet1 will be created
  + resource "aws_subnet" "con_vpc_subnet1" {
      + arn                                            = (known after apply)
      + assign_ipv6_address_on_creation                = false
      + availability_zone                              = "eu-west-2a"
      + availability_zone_id                           = (known after apply)
      + cidr_block                                     = "10.10.0.64/26"
      + enable_dns64                                   = false
      + enable_resource_name_dns_a_record_on_launch    = false
      + enable_resource_name_dns_aaaa_record_on_launch = false
      + id                                             = (known after apply)
      + ipv6_cidr_block_association_id                 = (known after apply)
      + ipv6_native                                    = false
      + map_public_ip_on_launch                        = true
      + owner_id                                       = (known after apply)
      + private_dns_hostname_type_on_launch            = (known after apply)
      + tags                                           = {
          + "Name" = "rf_connected_vpc_subnet1"
        }
      + tags_all                                       = {
          + "Name" = "rf_connected_vpc_subnet1"
        }
      + vpc_id                                         = (known after apply)
    }

  # module.vmc_vpc.aws_subnet.con_vpc_subnet2 will be created
  + resource "aws_subnet" "con_vpc_subnet2" {
      + arn                                            = (known after apply)
      + assign_ipv6_address_on_creation                = false
      + availability_zone                              = "eu-west-2b"
      + availability_zone_id                           = (known after apply)
      + cidr_block                                     = "10.10.0.128/26"
      + enable_dns64                                   = false
      + enable_resource_name_dns_a_record_on_launch    = false
      + enable_resource_name_dns_aaaa_record_on_launch = false
      + id                                             = (known after apply)
      + ipv6_cidr_block_association_id                 = (known after apply)
      + ipv6_native                                    = false
      + map_public_ip_on_launch                        = true
      + owner_id                                       = (known after apply)
      + private_dns_hostname_type_on_launch            = (known after apply)
      + tags                                           = {
          + "Name" = "rf_connected_vpc_subnet2"
        }
      + tags_all                                       = {
          + "Name" = "rf_connected_vpc_subnet2"
        }
      + vpc_id                                         = (known after apply)
    }

  # module.vmc_vpc.aws_subnet.con_vpc_subnet3 will be created
  + resource "aws_subnet" "con_vpc_subnet3" {
      + arn                                            = (known after apply)
      + assign_ipv6_address_on_creation                = false
      + availability_zone                              = "eu-west-2c"
      + availability_zone_id                           = (known after apply)
      + cidr_block                                     = "10.10.0.192/26"
      + enable_dns64                                   = false
      + enable_resource_name_dns_a_record_on_launch    = false
      + enable_resource_name_dns_aaaa_record_on_launch = false
      + id                                             = (known after apply)
      + ipv6_cidr_block_association_id                 = (known after apply)
      + ipv6_native                                    = false
      + map_public_ip_on_launch                        = true
      + owner_id                                       = (known after apply)
      + private_dns_hostname_type_on_launch            = (known after apply)
      + tags                                           = {
          + "Name" = "rf_connected_vpc_subnet3"
        }
      + tags_all                                       = {
          + "Name" = "rf_connected_vpc_subnet3"
        }
      + vpc_id                                         = (known after apply)
    }

  # module.vmc_vpc.aws_vpc.con_vpc will be created
  + resource "aws_vpc" "con_vpc" {
      + arn                                  = (known after apply)
      + cidr_block                           = "10.10.0.0/16"
      + default_network_acl_id               = (known after apply)
      + default_route_table_id               = (known after apply)
      + default_security_group_id            = (known after apply)
      + dhcp_options_id                      = (known after apply)
      + enable_classiclink                   = (known after apply)
      + enable_classiclink_dns_support       = (known after apply)
      + enable_dns_hostnames                 = true
      + enable_dns_support                   = true
      + enable_network_address_usage_metrics = (known after apply)
      + id                                   = (known after apply)
      + instance_tenancy                     = "default"
      + ipv6_association_id                  = (known after apply)
      + ipv6_cidr_block                      = (known after apply)
      + ipv6_cidr_block_network_border_group = (known after apply)
      + main_route_table_id                  = (known after apply)
      + owner_id                             = (known after apply)
      + tags                                 = {
          + "Name" = "rf_connected_vpc"
        }
      + tags_all                             = {
          + "Name" = "rf_connected_vpc"
        }
    }

Plan: 5 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

Accept to continue by typing “yes” and watch the magic happen!

5. Patience is a virtue

The process will take a while, as Terraform provisions resources in both AWS and VMware Cloud. Once completed, Terraform will provide an output with a summary.

module.sddc.vmc_sddc.vmc_sddc1: Still creating... [1h46m41s elapsed]
module.sddc.vmc_sddc.vmc_sddc1: Creation complete after 1h46m44s [id=d87b5f5b-6299-4627-839a-c0d83aa57167]

Apply complete! Resources: 5 added, 0 changed, 0 destroyed.

Benefits of Using Terraform for VMC on AWS Deployment

 Let’s quickly review the benefits of using Terraform (and IaC in general) for deployments in the cloud.

  1. Consistency: No manual errors. Your infrastructure is now version-controlled and can be deployed repeatedly with the same configurations.

  2. Scalability: Need to deploy multiple SDDCs? Just tweak your Terraform scripts.

  3. Transparency: Your entire infrastructure is visible as code. This is great for team collaboration, auditing and makes for easier configuration reviews.

Conclusion

Automating VMC on AWS deployment using Terraform does not necessarily speed up the process but ensures our infrastructure is consistent and scalable. As we evolve our cloud strategy, using tools like Terraform will be key to staying on top of things.

In my next blogs, I will likely delve deeper into different aspects of VMware Cloud. Stay tuned, and feel free to leave any comments or questions below! 

Links: 

https://www.securefever.com/vmc-aws-manual-deployment
https://learn.hashicorp.com/terraform/getting-started/install.html
https://registry.terraform.io/providers/vmware/vmc/latest/docs

Ralf Frankemölle
Previous

How DPU`s accelerate VMware ESXi with NSX - a deeper look to the data path!

Next

Off topic: Take 3