Terraform - iterate over nested map

Question

I am trying to create IAM binding for Bigquery dataset using the resource - google_bigquery_dataset_iam_binding. The requirement is I read the parameters in this resource (dataset_id, role, members) using a variable of the following structure -

  bq_iam_role_bindings = {
    "member1" = {
      "dataset1" : ["role1","role2", "role5"],
      "dataset2" : ["role3","role2"],
    },
    "member2" = {
      "dataset3" : ["role1","role4"],
      "dataset2" : ["role5"],
    } 
  }

So, I need to loop over this variable and get the roles assigned on a dataset for each member. Here total resources created would be eight (for each member, each dataset and each role). I am new to terraform and understand only how to apply simple for loop over a map and for_each loop in a resource. Want to understand how is it possible what I am trying to do.

This is the nearest what I have found - Map within a map in terraform variables where I can read the value in a nested map but I need to extract key also in my case. Can anyone help here please.

Solution

You could re-organize it into more for_each friendly list of objects and store it in a local helper_list.

For example:

variable "bq_iam_role_bindings" {

  default = {
    "member1" = {
      "dataset1" : ["role1","role2", "role5"],
      "dataset2" : ["role3","role2"],
    },
    "member2" = {
      "dataset3" : ["role1","role4"],
      "dataset2" : ["role5"],
    } 
  }
}

locals {

  helper_list = flatten([for member, value in var.bq_iam_role_bindings:
                 flatten([for dataset, roles in value: 
                           [for role in roles:
                            {"member" = member
                            "dataset" = dataset
                            "role" = role}
                         ]])
                   ])
}

which will result in helper_list in the form of:

[
  {
    "dataset" = "dataset1"
    "member" = "member1"
    "role" = "role1"
  },
  {
    "dataset" = "dataset1"
    "member" = "member1"
    "role" = "role2"
  },
  {
    "dataset" = "dataset1"
    "member" = "member1"
    "role" = "role5"
  },
  {
    "dataset" = "dataset2"
    "member" = "member1"
    "role" = "role3"
  },
  {
    "dataset" = "dataset2"
    "member" = "member1"
    "role" = "role2"
  },
  {
    "dataset" = "dataset2"
    "member" = "member2"
    "role" = "role5"
  },
  {
    "dataset" = "dataset3"
    "member" = "member2"
    "role" = "role1"
  },
  {
    "dataset" = "dataset3"
    "member" = "member2"
    "role" = "role4"
  },
]

The above form is much easier to work with for_each, e.g.:

resource "google_bigquery_dataset_iam_binding" "reader" {

  for_each =  { for idx, record in local.helper_list : idx => record }

  dataset_id = each.value.dataset
  role       = each.value.role

  members = [
    each.value.member
  ]
}

Comments

[luk2302]

Does this answer your question? Use a map of lists of maps in resource creation

[luk2302]

You need to iterate over the map "manually", flatten the result and then use that result in for_each.

[gkb_1696]

But how will I refer 3 parameters in for_each - member, dataset_id and role

[luk2302]

you will end up iterating over a map and the key of the map is a unique combination of all those three parameters (e.g. "something|some_data|some_role") while the value will be a map of structure {member="something" dataset_id="some_data" role="some_role"}

[gkb_1696]

This makes it so simple. Exactly how I was trying to get it..Thank you so much!!

[red888]

because this a list and not a set though if the order of any "dataset" lists change terraform will try to destroy and recreate all elements right?