Merge Maps in Golang

Question

I need to merge multiple maps map1 = [ id: id_1 val: val_1 ], map2 = [ id: id_2 val: val_2 ] and map3 = [id: id_1, val: val_3] such that the result map should be merged on the id values:

result_map = [id: id_1 val: {val_1, val_3}, id: id_2 var: {val_2}} ]

The code I've tried:

var a = make(map[string]interface{})
for _, m := range data {
    for _, n := range data {
        if m["id"] == n["id"] {
            for l, k := range n {
                c[l] = k
            }
        }
    }
}

Is there a way this can be done? Am using Golang 1.7

Thanks

Simple merge

Yes, they can be merged, but since in the result map there may be multiple values associated to the same key, the value type should be a slice, such as map[string][]string.

To do the merge, simply range over the maps to be merged, and append each value from the source maps to the slice associated with the same key in the result map.

One thing to look out for is that once you do the append, you have to assign back the result slice to the same key in the result map.

This is a simple implementation:

func merge(ms ...map[string]string) map[string][]string {
    res := map[string][]string{}
    for _, m := range ms {
        for k, v := range m {
            res[k] = append(res[k], v)
        }
    }
    return res
}

This merge() function has a variadic parameter, which means you may pass any number of maps to it.

Note that you don't need to initialize the slices in the target map, as indexing a map with a key that is not yet in it will result in the zero value of its type (which is nil for slices), and you may append to a nil slice, the builtin append() function takes care of (re-)allocations.

Testing it:

m1 := map[string]string{"id_1": "val_1"}
m2 := map[string]string{"id_2": "val_2"}
m3 := map[string]string{"id_1": "val_3"}

res := merge(m1, m2, m3)
fmt.Println(res)

Output (try it on the Go Playground):

map[id_1:[val_1 val_3] id_2:[val_2]]

Avoiding duplicates

Note that the above merge() will not filter out duplicates, meaning if the same "id_1": "val_1" pair is contained in multiple input maps, it will be listed multiple times in the target like "id_1": ["val_1", "val_1", "val_x"]. To filter out such duplicates (to only list it once in the target), we have to check this before doing the append (and if we've encountered it before, skip the append).

This is how it could be done:

func merge(ms ...map[string]string) map[string][]string {
    res := map[string][]string{}
    for _, m := range ms {
    srcMap:
        for k, v := range m {
            // Check if (k,v) was added before:
            for _, v2 := range res[k] {
                if v == v2 {
                    continue srcMap
                }
            }
            res[k] = append(res[k], v)
        }
    }
    return res
}

Testing it:

m1 := map[string]string{"id_1": "val_1"}
m2 := map[string]string{"id_2": "val_2", "id_1": "val_1"}
m3 := map[string]string{"id_1": "val_3"}

res := merge(m1, m2, m3)
fmt.Println(res)

Output (try it on the Go Playground):

map[id_1:[val_1 val_3] id_2:[val_2]]

We can see that "id_1": "val_1" was included both in m1 and m2, yet the value "val_1" is only listed once in in the slice associated with "id_1" key in the target map.

Comments

[CallMeLoki]

it doesnt work with duplicated values in slice try using github.com/fatih/set for that

[icza]

@danicheeta Yes, duplicate values will be listed multiple times. This may or may not be what the asker wants, he didn't say anything about this.

[icza]

@user3809560 Do you want to handle duplicates? I mean if the same "id_1": "val_1" pair is contained in multiple input maps, is it OK that it will be listed multiple times in the target like "id_1": ["val_1", "val_x", "val_1"]? Or you want to filter out these duplicates?

[user3809560]

@icza, you have used a for loop within srcMap. What control structure is it. Is it supported in 1.7

[icza]

@user3809560 Yes, it's supported by Go 1.7 (but you really should drop Go 1.7 and go with Go 1.10.2). There are 3 loops in the 2nd version. The most outer loops over the slice of input maps (ms), the 2nd loop loops over an input map (m), and the most inner loop loops over a slice value of the result map to check for a duplicate.

[The Fool]

Wouldn't it make more sense to override the old value when the same key is encountered again, instead of dropping the new value? I would expect the last seen occurrence to persist in a merge operation.