Terraform patterns: loops
Create multiple resources with a loop
If you want to create multiple instances of, say, an Azure resource group, you can add a for_each argument. The for_each argument accepts a map or a set, and creates an instance for each item in that map or set.
So you can create a map of key value pairs (aka a dictionary) and use it to define multiple resource groups:
1resource "azurerm_resource_group" "rg" {
2 for_each = {
3 projectx-dev-we = "westeurope"
4 projectx-dev-us = "eastus"
5 }
6
7 name = "${each.key}-rg"
8 location = each.value
9 tags = var.tags
10}
Sure enough you can also refactor the map as a variable
1variable "groups" {
2 default = {
3 projectx-prod-we = "westeurope"
4 projectx-prod-us = "eastus"
5 }
6}
7
8resource "azurerm_resource_group" "otherrg" {
9 for_each = var.groups
10
11 name = "${each.key}-rg"
12 location = each.value
13 tags = var.tags
14}
Reference the resource group
What if you want to create a storage account in each of the created resource groups? You could reference the resource group that has been created in the previous step as follows:
1resource "azurerm_storage_account" "storage" {
2 name = "projectxprodwestorage"
3 account_replication_type = "LRS"
4 account_tier = "Standard"
5 location = azurerm_resource_group.otherrg["projectx-prod-we"].location
6 resource_group_name = azurerm_resource_group.otherrg["projectx-prod-we"].name
7}
A more complex example
A map has just some keys and values. They can contain many things of just one type. But what if I want to use strings, lists and so on to create my new resource, in a for-each loop? For example, I have a vnet and it contains multiple subnets. So there is a one-to-many relationship. Here is its definition.
The subnet has a name (of type string) and address_prefixes (of type list). Objects to the rescue! Objects contain a specific set of things of many types, and they have name whih we can refer to.
First, let's create the vnet:
1resource "azurerm_virtual_network" "vnet" {
2 address_space = ["172.16.0.0/16"]
3 location = azurerm_resource_group.otherrg["projectx-prod-we"].location
4 name = "${var.prefix}-vnet"
5 resource_group_name = azurerm_resource_group.otherrg["projectx-prod-we"].name
6}
Let's now define the subnets.
Use a map of objects
As the variable type we could use a map of objects. The key is the name of the subnet instance, and the value is a complex object with the subnet properties. The map has the following definition:
1variable "subnets" {
2 type = map(object({
3 address_prefixes = list(string)
4 service_endpoints = list(string)
5 }))
6}
We can then define the default value of the variable:
1variable "subnets" {
2 type = map(object({
3 address_prefixes = list(string)
4 service_endpoints = list(string)
5 }))
6 default = {
7 "db-subnet" = {
8 address_prefixes = ["172.16.1.0/24"]
9 service_endpoints = ["Microsoft.AzureCosmosDB","Microsoft.Sql"]
10 },
11 "generic-subnet" = {
12 address_prefixes = ["172.16.2.0/24"]
13 service_endpoints = ["Microsoft.Storage","Microsoft.KeyVault"]
14 }
15 }
16}
Finally, we can create the subnets as follows:
1resource "azurerm_subnet" "subnet" {
2 for_each = var.subnets
3 name = "${var.prefix}-${each.key}"
4 resource_group_name = azurerm_resource_group.otherrg["projectx-prod-we"].name
5 virtual_network_name = azurerm_virtual_network.vnet.name
6 address_prefixes = each.value.address_prefixes
7 service_endpoints = each.value.service_endpoints
8}
Use a list of objects
We can also use a list of objects, but then we can only use name and one extra property.
1variable "subnets_list" {
2 description = "Required. A map of string, object with the subnet definition"
3 type = list(object({
4 name = string
5 address_prefixes = list(string)
6 service_endpoints = list(string)
7 }))
8 default = [
9 {
10 name : "firewall_subnet"
11 address_prefixes : ["10.12.0.0/24"]
12 service_endpoints : ["Microsoft.Sql"]
13 },
14 {
15 name : "jumpbox_subnet"
16 address_prefixes : ["10.12.1.0/24"]
17 service_endpoints : ["Microsoft.Sql"]
18 }
19 ]
20}
The for_each meta-argument accepts a map or a set of strings, so we need to translate the list to a map. We say: we set the key of the map to the unique subnet.name, and the value is the complete subnet object.
1resource "azurerm_subnet" "subnet" {
2 for_each = { for subnet in var.subnets : subnet.name => subnet }
3 name = "${var.prefix}-${each.key}"
4 resource_group_name = azurerm_resource_group.otherrg["projectx-prod-we"].name
5 virtual_network_name = azurerm_virtual_network.vnet.name
6 address_prefixes = each.value.address_prefixes
7 service_endpoints = each.value.service_endpoints
8}
Take aways
- Terraform can give you headaches.
- For_each accepts a map. A map is a dictionary, with a key and a value. The value can be a complex property like an an object or a list.
- When using a list, we need to transform the list to a map by setting a key and a value with the arrow notation. We can set the value to a complex object.
Complete example
1terraform {
2 required_providers {
3 azurerm = {
4 source = "hashicorp/azurerm"
5 version = "=3.2.0"
6 }
7 }
8}
9
10# Configure the Microsoft Azure Provider
11provider "azurerm" {
12 features {}
13}
14
15variable "prefix" {
16 default = "headache-dev"
17}
18
19variable "network_portion" {
20 default = "10.14"
21}
22
23//using locals, not variables, because Terraform does not support variables in variables (variable nesting)
24locals {
25 common_tags = {
26 environment = "sratch"
27 creation_date = formatdate("YYYY-MM-01", timestamp())
28 }
29 subnets-map = {
30 "db-subnet" = {
31 address_prefixes = ["${var.network_portion}.1.0/24"]
32 service_endpoints = ["Microsoft.AzureCosmosDB", "Microsoft.Sql"]
33 }
34 "generic-subnet" = {
35 address_prefixes = ["${var.network_portion}.2.0/24"]
36 service_endpoints = ["Microsoft.Storage", "Microsoft.KeyVault"]
37 }
38 }
39 subnets-list = [
40 {
41 name = "fw-subnet"
42 address_prefixes = ["${var.network_portion}.3.0/24"]
43 service_endpoints = ["Microsoft.AzureCosmosDB", "Microsoft.Sql"]
44 },
45 {
46 name = "vm-subnet"
47 address_prefixes = ["${var.network_portion}.4.0/24"]
48 service_endpoints = ["Microsoft.Storage", "Microsoft.KeyVault"]
49 }
50 ]
51}
52
53resource "azurerm_resource_group" "vnetgroup" {
54 location = "westeurope"
55 name = "${var.prefix}-rg"
56}
57
58resource "azurerm_virtual_network" "vnet" {
59 address_space = ["${var.network_portion}.0.0/16"]
60 location = "westeurope"
61 name = "${var.prefix}-vnet"
62 resource_group_name = azurerm_resource_group.vnetgroup.name
63}
64
65resource "azurerm_subnet" "subnet" {
66 for_each = { for subnet in local.subnets-list : subnet.name => subnet }
67 name = "${var.prefix}-${each.key}"
68 resource_group_name = azurerm_resource_group.vnetgroup.name
69 virtual_network_name = azurerm_virtual_network.vnet.name
70 address_prefixes = each.value.address_prefixes
71 service_endpoints = each.value.service_endpoints
72
73}
74
75resource "azurerm_subnet" "subnet2" {
76 for_each = local.subnets-map
77 name = "${var.prefix}-${each.key}"
78 resource_group_name = azurerm_resource_group.vnetgroup.name
79 virtual_network_name = azurerm_virtual_network.vnet.name
80 address_prefixes = each.value.address_prefixes
81 service_endpoints = each.value.service_endpoints
82}
Other musings
What I don't like in the above examples, is that my subnet object definition is incomplete. There is no resource_group_name or location, because we use the values of the vnet definition for that. Can't we add those properties to our object definition?
That would lead to a whole lot of repetition, because we can't use variables in variables. Terraform will fail when using nested variables (can not interpolate variables when using a datastructure as a variable). And optional properties in an object are not (yet?) supported.
What we could do to solve this is to create a local variable. Locals support variable nesting.
Anyway I will stick to using the root value.
In next posts we will discuss modules and maybe also dynamics.
https://stackoverflow.com/questions/58594506/how-to-for-each-through-a-listobjects-in-terraform-0-12 https://www.reddit.com/r/Terraform/comments/hyqago/difference_between_maps_and_objects/ https://www.terraform.io/language/meta-arguments/for_each