Introduction

Terraform is one of the most popular IAC tools out there. Although, it is quite simple to grasp and use, the complexity rises quickly once you introduce multi-account and multi-region deployments.

This post is inspired by one such problem I had to tackle recently.

terraform-workspace

Problem

How to create and manage multi-account and multi-region AWS resources using Terraform?

A lot of organizations now seggregate their AWS accounts on the basis of environments (development, test, production etc.). At the same time, due to DR considerations, the production workloads also need to span across regions. This results in a multi-dimensional array of combinations between accounts and regions.

To top it all, the cicd tool (most often) runs in a separate dedicated account (say, operator).

Solution

Terraform workspaces to the rescue! Well, almost.

Terraform workspaces have been available for quite sometime now, and don’t need much of an introduction. A small clarification here is in order though. When I mention workspaces, I am referring to the cli workspace and NOT the Terraform cloud workspace.

Workspaces make it fairly easy to deploy the same configuration over and over again by simply overriding the variable values. This keeps the configurations minimal, but, at the same time provides enough flexibility to create multiple environments that are isolated from each other.

In this particular case, I use the same names for the workspaces as the environments - development and production. The benefit of this will become clear when you look at the configurations and cicd commands.

As you already know, to target different AWS accounts and regions, the provider block is what we need to focus on. This is the configuration which dictates which account and region Terraform runs the API commands against.

Since Terraform does not support dynamic provider blocks (yet!), it is clear a bunch of explicit provider blocks are necessary to interact with different accounts and regions.

Some optimization is still possible though since the provider block does support specifying variables. Following is an example providers.tf file which enables working with multiple accounts and regions. Explanation follows after this snippet.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

provider "aws" {
    region = "eu-west-1"
}

provider "aws" {
    region = "eu-west-1"
    alias = "euw1"
    assume_role {
        role_arn = var.role_arn[terraform.workspace]
    }
}

provider "aws" {
    region = "us-west-1"
    alias = "usw1"
    assume_role {
        role_arn = var.role_arn[terraform.workspace]
    }
}

The first provider block is the default one, and is configured partially. AWS credentials are provided for it via environment variables. This is the provider which allows Terraform to use an S3 bucket, and Dynamodb table in the cicd tool account (operator) for state files and locking respectively.

Next, 2 additional provider blocks are configured, each pointing to a specific region. Notice the alias being set for these providers, as this is necessary to use them later on in the configuration. Also note that the role_arn is not specified in these, rather it is being derived from a variable role_arn of type map(string). This variable is essentially where the magic happens. Depending on the current workspace, it provides the role_arn to Terraform from either the development account or production account. This can be easily extended to include more accounts and workspaces.

Looking at the auto loading tfvars file below, should make it clearer. Note that these roles are pre-existing, and created as part of the cicd setup.

1
2
3
4

role_arn = {
    development = "arn:aws:iam::123456789012:role/TFRole"
    production  = "arn:aws:iam::123456789013:role/TFRole"
}

With the above configs, we are now ready to create and manage resources across AWS accounts and regions.

Below is a dummy file which can be quickly used to test (provided ofcourse, your target accounts/regions have a vpc named MyVPC) the configuration. The alias I mentioned above comes in handy here when declaring the data block. You can also use alias for resources and modules. More details here

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

variable "role_arn" {
  description = "Map of role_arn to use in each account"
}

data "aws_vpc" "eu" {
  provider = aws.euw1
  filter {
    name   = "tag:Name"
    values = ["MyVPC"]
  }
}

data "aws_vpc" "us" {
  provider = aws.usw1
  filter {
    name   = "tag:Name"
    values = ["MyVPC"]
  }
}

output "eu" {
    value = data.aws_vpc.eu.arn
}

output "us" {
    value = data.aws_vpc.us.arn
}

Finally, in the cicd pipeline, it’s very simple to target the specific environment using workspaces. The following few cli commands should do it. I only show this for the development environment, but I am sure, you get the drift!

1
2
3
4
5
6
7

$ export WORKSPACE=development
$ terraform select workspace $WORKSPACE  || terraform new workspace $WORKSPACE

# This commands loads an additional variable file named development.tfvars
# Any environment specific customization can be put into this file
$ terraform plan -input=false -var-file $WORKSPACE.tfvars
$ terraform apply -input=false -auto-approve -var-file $WORKSPACE.tfvars

The visualization of the above setup would probably look something like the diagram included at the begining of this post. Have a look at it now, it would probably make more sense.

Conclusion

This is a simple pattern to manage resources with Terraform across AWS accounts and regions. It can be extended further to cater to environment specific customizations etc. This should atleast give you some ideas and help in getting started.

If you use a different simpler pattern, I would love to hear more about it. Please feel free to leave a link in the comments below.

References (1)

  1. Configuration#alias Multiple Provider Configurations