Mar 13, 2021

Iterations in Terraform Part 1: meta-arguments

In this post I’m going to cover two extremely useful meta-arguments in Terraform - count and for_each, used to create resources iteratively. I’ll go through their benefits, drawbacks, and demonstrate a comprehensive example of how to use them.

Theory

Let’s start with a reviewing some theoretic review. Meta-arguments are the kind of arguments that can be used in all resources and modules regardless of what they do. There are five meta-arguments at the moment with count and for_each being the most often used.

count

The oldest way to create resources iteratively is to use meta-argument count. It takes a number and creates that exact number of copies of the resources. On each iteration it exposes an object count with a single argument index inside of the resource block. By accessing count.index you can get an index number of current iteration. You would use this index to somehow ensure that resource arguments which have to be unique - are unique. Some examples of that would be using count.index interpolation in your strings or as a way to pick an item from some list.

There are multiple ways you can use count in your code:

  1. The most primitive one is to create identical resources with slightly different names:

    resource "google_service_account" "service_accounts" {
  count        = length(3)
  account_id   = "${var.account_id}-${count.index}"
  display_name = var.display_name
}

  2. More useful scenario is to reference an item in the list of config objects. This way each of the resources you create can have fully customizable config. Please also note that using length function acts as a toggle in a way. By providing an empty list you disable resource creation.

    variable "service_accounts" {
  type    = list(map(string))
  default = [
    {account_id = "one-sa", display_name = "Custom SA"},
    {account_id = "two-sa", display_name = "Yet another SA"},
  ]
}

resource "google_service_account" "service_accounts" {
  count        = length(var.service_accounts)
  account_id   = var.service_accounts[count.index].account_id
  display_name = var.service_accounts[count.index].display_name
}

  3. Sometimes you are in a situations when you need to create resource only if certain conditions are met(eg certain parameter is set to true). So it can act as a toggle if combined with conditional expression:

    resource "google_service_account" "service_account" {
  count        = var.create_service_account == true ? 1 : 0
  account_id   = var.service_account_id
  display_name = var.service_account_display_name
}

  4. You can also combine toggle feature with creation from the list of configs:

    resource "google_service_account" "service_accounts" {
  count        = var.create_service_accounts == true ? length(var.service_accounts) : 0
  account_id   = var.service_accounts[count.index].account_id
  display_name = var.service_accounts[count.index].display_name
}

As you can see it is very easy to use and quite powerful too. Unfortunately there is one particularly nasty side effect. Resources created using count are referenced in the state by their list index. If you delete a single item in the middle of the underlying list, all the items to the right of it will have their index recalculated, causing recreation of all those resources. Nonetheless it still can be used in scenarios when underlying list is not supposed to change frequently or items will be deleted only from the end of the list(scaling up/down).

If you want to reference an argument of a resource created with count in other resources/outputs, you will have to use a list index or splat expression.

One item:

resource "google_service_account" "service_accounts" {
  count        = length(3)
  account_id   = "${var.account_id}-${count.index}"
  display_name = var.display_name
}

output "service_account_email" {
  value = google_service_account.service_accounts[0].email
}

All items:

# ...
output "service_account_emails" {
  value = google_service_account.service_accounts[*].email
}

for_each

This meta-argument is relatively new. Originally for_each was introduced in Terraform 0.12.0 as a way of creating “dynamic” resource blocks (blocks you can use within the same resource multiple times) but since Terraform 0.12.6 it’s usage was extended as a count alternative for resource creation. On each iteration it exposes each object with two arguments - key and value, which will have map element key and value correspondingly. Although map is the main resource type to use with for_each you can also use it with sets (or lists converted to sets with toset function).

When used with maps, for_each is not a subject to the side effect that count has. Due to the fact that every item in the map has a unique key and resources created with for_each are referenced in the state by the key name of a corresponding map element, deleting an item from the map will not affect other map elements resulting in more predictable “plan”.

The main use case scenario is simple creation of multiple resources, but there is also a way you can implement a toggle feature.

  1. Creating multiple resources with fully customizable configs. Please also note that using empty map works as toggle by itself just like the empty list did in count = length(....

    variable "service_accounts" {
  type    = map(map(string))
  default = {
    "one-sa" = {
      display_name = "Custom SA"
      description  = "A custom SA"
      },
    "two-sa" = {
      display_name = "Yet another SA"
      description  = "A custom SA"
    },
  }
}

resource "google_service_account" "service_account" {
  for_each     = var.service_accounts
  account_id   = each.key
  display_name = each.value.display_name
  description  = each.value.description
}

  2. Create single resources with a toggle. Looks ugly and does exactly the same thing as count toggle. I would not recommend using it but for the sake of completeness:

    resource "google_service_account" "service_accounts" {
  for_each     = var.create_service_accounts == true ? map(var.account_id, var.display_name) : {}
  account_id   = each.key
  display_name = each.value
}

  3. Create multiple resources with a toggle:

    resource "google_service_account" "service_accounts" {
  for_each     = var.create_service_accounts == true ? var.service_accounts : {}
  account_id   = each.key
  display_name = each.value.display_name
  description  = each.value.description
}

When it comes to referencing resource values in the outputs, If you want to reference an argument of a resource created with for_each, you would need to address each resource by the key it was created with:

output "service_account_email" {
  value = google_service_account.service_accounts["one-sa"].email
}

Splat expressions will not work with maps but you can use list and map expressions to achieve the same result:

output "service_account_email" {
  value = [
    for k, v in google_service_account.service_accounts: v.email
  ]
}

which would produce following output:

service_account_email = [
  "one-sa@.....",
  "two-sa@....."
]

or

output "service_account_email" {
  value = {
    for k, v in google_service_account.service_accounts:
      k => v.email
  }
}

which would produce following output:

service_account_email = {
  one-sa = "one-sa@.....",
  two-sa = "two-sa@....."
}

There is one thing you should be aware of when using for_each. It will not be able to calculate the plan if the keys of the underlying map are composed from the outputs of any other resource. Consider the following snippet:

locals {
  numbers = ["one", "two", "three"]
  the_map = {
    for number in local.numbers:
      "${number}-${random_string.random.result}" => "some text"
  }
}

resource "random_string" "random" {
  length           = 2
  special          = false
}

resource "local_file" "foo" {
  for_each = local.the_map
  content  = each.value
  filename = each.key
}

Even though it is clear to the human reader that the number of elements in the map does not depend on the result of random_string resource, terraform will not be able to create the plan. So try to avoid such scenarios.

Error: Invalid for_each argument

  on main.tf line 13, in resource "local_file" "foo":
  13:     for_each = local.the_map

The "for_each" value depends on resource attributes that cannot be determined
until apply, so Terraform cannot predict how many instances will be created.
To work around this, use the -target argument to first apply only the
resources that the for_each depends on.

Practice

Now as a practical example I will assemble some primitive certificate creation tool. It is not production-grade but might come handy for experiments in the home lab. The code will allow to either generate the CA or import the external one. I will be using TLS and Local providers with most of the aforementioned ways of using count and for_each.

Let’s start with the variable declaration. I’m going to use two composite variables, one for CA configuration and one for all leaf certificates.

cat variables.tf

variable "root_ca" {
  description = "Root CA Specification"
  type = object({
    algorithm     = string
    rsa_bits      = number
    ecdsa_curve   = string
    external_cert = string
    external_key  = string
    subject       = map(string)
  })
  default = null
}

variable "certificates" {
  description = "Leaf Certs Specification"
  default     = {}
}

You have probably noticed one significant difference between the root_ca variable and the certificates. First one has an object type declared while second has none. While certificates is a map of objects, keeping its type unset allows me to have “optional parameters” in the config. Omitting such parameters will not be interpreted by terraform as an error and will make the tfvars file significantly shorter in some cases. Terraform is smart enough to deduce variable type automatically during the plan/apply. If I were using strictly defined type I would have to set all key/value pairs declared in the object type. Even empty ones would have to be explicitly set to null.

Next I’ll put some values to have a better perspective on the possible way of consuming them:

cat terraform.tfvars

root_ca = {
  algorithm     = "RSA"
  rsa_bits      = 4096
  ecdsa_curve   = null
  external_cert = null
  external_key  = null
  subject = {
    CN = "Root CA"
    O  = "Example Org"
  }
}
certificates = {
  "example.com" = {
    algorithm = "RSA"
    rsa_bits  = 2048
    subject = {
      CN = "Example Leaf Cert"
      O  = "Example Org"
    }
  }
  "website.com" = {
    algorithm = "ECDSA"
    subject = {
      CN = "Some Website"
      O  = "Example Org"
    }
  }
  "anotherone.com" = {
    subject = {
      CN = "Leaf Cert"
      O  = "Example Org"
    }
  }
}

As you can see, the keys of certificates variable are domain names which make them unique to a certain degree. The “website.com” value has only algorithm set and “anotherone.com” omits even that. In both cases missing values will be calculated automatically according to the sane defaults I will put in my main.tf. At the same time omitting root_ca.ecdsa_curve would be an error.

Finally the heart of this tool, the resources.

cat main.tf

# the locals block is prefect for reusable expressions, it has a toggle which
# defines if CA cert and key must be generated or imported from file
locals {
  use_external_cert = lookup(var.root_ca, "external_key", null) != null && lookup(var.root_ca, "external_cert", null) != null
}
# next two data sources import the external cert and key file only it toggle
# is true
data "local_file" "ca-key" {
  count    = local.use_external_cert == true ? 1 : 0
  filename = var.root_ca.external_key
}

data "local_file" "ca-cert" {
  count    = local.use_external_cert == true ? 1 : 0
  filename = var.root_ca.external_cert
}

# Root CA Config #
# same logic goes into the self-signed CA config, but the toggle is reversed
resource "tls_private_key" "root" {
  count       = local.use_external_cert == true ? 0 : 1
  algorithm   = var.root_ca.algorithm
  rsa_bits    = var.root_ca.algorithm == "RSA" ? var.root_ca.rsa_bits : null
  ecdsa_curve = var.root_ca.algorithm == "ECDSA" ? var.root_ca.ecdsa_curve : null
}

# notice that in subject block we cant reference CN and O as we did with with
# algorithm so usilg `lookup` function is desirable
resource "tls_self_signed_cert" "root" {
  count           = local.use_external_cert == true ? 0 : 1
  key_algorithm   = tls_private_key.root[0].algorithm
  private_key_pem = tls_private_key.root[0].private_key_pem

  subject {
    common_name  = lookup(var.root_ca.subject, "CN")
    organization = lookup(var.root_ca.subject, "O")
  }

  validity_period_hours = 87600 # 10 years

  allowed_uses = [
    "crl_signing",
    "cert_signing"
  ]
  is_ca_certificate = true
}

# Leaf Certs Config #
# leaf config is not as strict as root ca, because there was no explicit type
# declaration. Again the `lookup` function saves the day providing defaults for
# all omitted values
resource "tls_private_key" "leaf" {
  for_each    = var.certificates
  algorithm   = lookup(each.value, "algorithm", "RSA")
  rsa_bits    = lookup(each.value, "algorithm", "RSA") == "RSA" ? lookup(each.value, "rsa_bits", 2048) : null
  ecdsa_curve = lookup(each.value, "algorithm", "RSA") == "ECDSA" ? lookup(each.value, "ecdsa_curve", "P224") : null
}

resource "tls_cert_request" "leaf" {
  for_each        = var.certificates
  key_algorithm   = tls_private_key.leaf[each.key].algorithm
  private_key_pem = tls_private_key.leaf[each.key].private_key_pem

  subject {
    common_name  = lookup(each.value.subject, "CN")
    organization = lookup(each.value.subject, "O")
  }

  dns_names = [each.key]
}

# another resource with local toggle use, this time to either get
# the generated CA or the external one
resource "tls_locally_signed_cert" "leaf" {
  for_each           = var.certificates
  cert_request_pem   = tls_cert_request.leaf[each.key].cert_request_pem
  ca_key_algorithm   = var.root_ca.algorithm
  ca_private_key_pem = local.use_external_cert == true ? data.local_file.ca-key[0].content : tls_private_key.root[0].private_key_pem
  ca_cert_pem        = local.use_external_cert == true ? data.local_file.ca-cert[0].content : tls_self_signed_cert.root[0].cert_pem

  validity_period_hours = 8760 # 1 year

  allowed_uses = [
    "key_encipherment",
    "digital_signature",
    "server_auth",
  ]
}

# File Outputs #
# and finally file outputs
# nothing new here, same tricks
resource "local_file" "ca-cert" {
  count    = local.use_external_cert == true ? 0 : 1
  content  = tls_self_signed_cert.root[0].cert_pem
  filename = "ca.pem"
}

resource "local_file" "ca-key" {
  count    = local.use_external_cert == true ? 0 : 1
  content  = tls_private_key.root[0].private_key_pem
  filename = "ca-key.pem"
}

resource "local_file" "leaf-certs" {
  for_each = var.certificates
  content  = tls_locally_signed_cert.leaf[each.key].cert_pem
  filename = "${each.key}-cert.pem"
}

resource "local_file" "leaf-keys" {
  for_each = var.certificates
  content  = tls_private_key.leaf[each.key].private_key_pem
  filename = "${each.key}-key.pem"
}

Now run it and you will get the CA certificate, all leaf certificates and corresponding private keys:

terraform init
terraform plan -out main.tfplan
terraform apply main.tfplan
ls -l | grep pem

Output:

-rwxr-xr-x 1 alex alex  1554 Mar  9 23:52 anotherone.com-cert.pem
-rwxr-xr-x 1 alex alex  1679 Mar  9 23:52 anotherone.com-key.pem
-rwxr-xr-x 1 alex alex  3243 Mar  9 23:52 ca-key.pem
-rwxr-xr-x 1 alex alex  1834 Mar  9 23:52 ca.pem
-rwxr-xr-x 1 alex alex  1558 Mar  9 23:52 example.com-cert.pem
-rwxr-xr-x 1 alex alex  1675 Mar  9 23:52 example.com-key.pem
-rwxr-xr-x 1 alex alex  1265 Mar  9 23:52 website.com-cert.pem
-rwxr-xr-x 1 alex alex   207 Mar  9 23:52 website.com-key.pem

To test the script with imported CA you can delete the state and all leaf certs and then replace the root_ca if your terraform.tfvars with the following content:

root_ca = {
  algorithm     = "RSA"
  rsa_bits      = 4096
  ecdsa_curve   = null
  external_cert = "ca.pem"
  external_key  = "ca-key.pem"
  subject = {
    CN = "Root CA"
    O  = "Example Org"
  }
}

and then repeat the Terraform routine

rm -f terraform.tfstate*
ls | grep .com | xargs rm -f
terraform plan -out main.tfplan
terraform apply main.tfplan
ls -l | grep pem

Output:

-rwxr-xr-x 1 alex alex  1554 Mar 10 21:27 anotherone.com-cert.pem
-rwxr-xr-x 1 alex alex  1679 Mar 10 21:27 anotherone.com-key.pem
-rwxr-xr-x 1 alex alex  3243 Mar 10 21:25 ca-key.pem
-rwxr-xr-x 1 alex alex  1834 Mar 10 21:25 ca.pem
-rwxr-xr-x 1 alex alex  1562 Mar 10 21:27 example.com-cert.pem
-rwxr-xr-x 1 alex alex  1675 Mar 10 21:27 example.com-key.pem
-rwxr-xr-x 1 alex alex  1265 Mar 10 21:27 website.com-cert.pem
-rwxr-xr-x 1 alex alex   207 Mar 10 21:27 website.com-key.pem

Conclusion

I hope this article shed some light on the potential and the ways of using the iteration mechanisms in Terraform. In part two I am going to cover the list and map expressions for building robust data structures that can be consumed by the count and for_each meta-arguments. Combining those you will be able to write the code to support “self-serve” infrastructure deployment processes and create complex customizable resource hierarchies.

Stay tuned.