Merging dictionary value lists in python

Question

I'm trying to merge three dictionaries, which all have the same keys, and either lists of values, or single values.

one={'a': [1, 2], 'c': [5, 6], 'b': [3, 4]}
two={'a': [2.4, 3.4], 'c': [5.6, 7.6], 'b': [3.5, 4.5]}
three={'a': 1.2, 'c': 3.4, 'b': 2.3}

What I need is for all the items in the values to be added to one list.

result={'a': [1, 2, 2.4, 3.4, 1.2], 'c': [5, 6, 5.6, 7.6, 2.3], 'b': [3, 4, 3.5, 4.5, 3.4]}

I have tried several things, but most put the values into nested lists. E.g.

out=dict((k, [one[k], two.get(k), three.get(k)]) for k in one)
{'a': [[1, 2], [2.4, 3.4], 1.2], 'c': [[5, 6], [5.6, 7.6], 3.4], 'b': [[3, 4], [3.5, 4.5], 2.3]}

I tried updating it by looping through the values:

out.update((k, [x for x in v]) for k,v in out.iteritems())

but the results was exactly the same. I have tried to simply add the lists, but because the third dictionary has only a float, I couldn't do it.

check=dict((k, [one[k]+two[k]+three[k]]) for k in one)

So I tried to first add the lists in values of one and two, and then append the value of three. Adding the lists worked well, but then when I tried to append the float from the third dictionary, suddenly the whole value went to 'None'

check=dict((k, [one[k]+two[k]]) for k in one)
{'a': [[1, 2, 2.4, 3.4]], 'c': [[5, 6, 5.6, 7.6]], 'b': [[3, 4, 3.5, 4.5]]}
new=dict((k, v.append(three[k])) for k,v in check.items())
{'a': None, 'c': None, 'b': None}

Solution 1

As a one-liner, with a dictionary comprehension:

new = {key: value + two[key] + [three[key]] for key, value in one.iteritems()}

This creates new lists, concatenating the list from one with the corresponding list from two, putting the single value in three into a temporary list to make concatenating easier.

Or with a for loop updating one in-place:

for key, value in one.iteritems():
    value.extend(two[key])
    value.append(three[key])

This uses list.extend() to update original list in-place with the list from two, and list.append() to add the single value from three.

Where you went wrong:

• your first attempt creates a new list with the values from one, two and three nested within rather than concatenating the existing lists. Your attempt to clean that up just copied those nested lists across.

• Your second attempt didn't work because the value in three is not a list so could not be concatenated. I created a new list just for that one value.

• Your last attempt should not have used list.append() in a generator expression, because you store the return value of that method, which is always None (its change is stored in v directly and the list doesn't need returning again).

Demo of the first approach:

>>> one={'a': [1, 2], 'c': [5, 6], 'b': [3, 4]}
>>> two={'a': [2.4, 3.4], 'c': [5.6, 7.6], 'b': [3.5, 4.5]}
>>> three={'a': 1.2, 'c': 3.4, 'b': 2.3}
>>> {key: value + two[key] + [three[key]] for key, value in one.iteritems()}
{'a': [1, 2, 2.4, 3.4, 1.2], 'c': [5, 6, 5.6, 7.6, 3.4], 'b': [3, 4, 3.5, 4.5, 2.3]}

Solution 2

Arbitrary dictionary number and keys

The issues with your attempt are covered by @MartijnPieters' solution.

For a generalised solution, consider using itertools.chain to chain multiple dictionaries. You can also use a defaultdict for the more general case where you do not find the same keys in each dictionary.

from collections import defaultdict
from itertools import chain
from operator import methodcaller

# dictionaries with non-equal keys, values all lists for simplicity
one = {'a': [1, 2], 'c': [5, 6], 'b': [3, 4], 'e': [6.2]}
two = {'a': [2.4, 3.4], 'c': [5.6, 7.6], 'b': [3.5, 4.5], 'f': [1.3]}
three = {'a': [1.2], 'c': [3.4], 'b': [2.3], 'e': [3.1]}

# initialise defaultdict of lists
dd = defaultdict(list)

# iterate dictionary items
dict_items = map(methodcaller('items'), (one, two, three))
for k, v in chain.from_iterable(dict_items):
    dd[k].extend(v)

print(dd)

# defaultdict(list,
#             {'a': [1, 2, 2.4, 3.4, 1.2],
#              'b': [3, 4, 3.5, 4.5, 2.3],
#              'c': [5, 6, 5.6, 7.6, 3.4],
#              'e': [6.2, 3.1],
#              'f': [1.3]})

Note defaultdict is a subclass of dict so there's generally no need to convert the result to a regular dict.

Solution 3

A robust solution. =)

def FullMergeDict(D1, D2):
  for key, value in D1.items():
    if key in D2:
      if type(value) is dict:
        FullMergeDict(D1[key], D2[key])
      else:
        if type(value) in (int, float, str):
          D1[key] = [value]
        if type(D2[key]) is list:
          D1[key].extend(D2[key])
        else:
          D1[key].append(D2[key])
  for key, value in D2.items():
    if key not in D1:
      D1[key] = value

if __name__ == '__main__':
  X = {
    'a': 'aaa',
    'c': [1,3,5,7],
    'd': 100,
    'e': {'k': 1, 'p': 'aa','t': [-1,-2]},
    'f': {'j':1}
  }
  Y = {
    'b': 'bbb',
    'd': 200,
    'e': {'k': 2, 'p': 'bb','o': [-4]},
    'c': [2,4,6],
    'g': {'v':2}
  }
  FullMergeDict(X, Y)
  exit(0)

Result:

  X = {
    'a': 'aaa',
    'b': 'bbb',
    'c': [1, 3, 5, 7, 2, 4, 6],
    'd': [100, 200],
    'e': {'k': [1, 2], 'o': [-4], 'p': ['aa', 'bb'], 't': [-1, -2]},
    'f': {'j': 1},
    'g': {'v': 2}}

Solution 4

If you have different keys and different types of values in dictionaries you can use the following approach:

from collections import defaultdict, Iterable

dct1 = {'a': [1, 2]}
dct2 = {'a': [3], 'b': [5, 6]}
dct3 = {'a': 4, 'c': 7}

result = defaultdict(list)
for dct in [dct1, dct2, dct3]:
    for k, v in dct.items():
        if isinstance(v, Iterable):
            result[k].extend(v)
        else:
            result[k].append(v)

print(result)
# defaultdict(<class 'list'>, {'a': [1, 2, 3, 4], 'b': [5, 6], 'c': [7]}) 

Solution 5

One line solution (also handles for keys present in only one dict):

{ key:one.get(key,[])+two.get(key,[]) for key in set(list(one.keys())+list(two.keys())) }

Example 1:

one = {'a': [1, 2], 'c': [5, 6], 'b': [3, 4]}
two = {'a': [2.4, 3.4], 'c': [5.6, 7.6], 'd': [3.5, 4.5]}
{ key:one.get(key,[])+two.get(key,[]) for key in set(list(one.keys())+list(two.keys())) }

Output:

{'a': [1, 2, 2.4, 3.4], 'b': [3, 4], 'c': [5, 6, 5.6, 7.6], 'd': [3.5, 4.5]}

Example 2:

x={1:['a','b','c']}
y={1:['d','e','f'],2:['g']}
{ key:x.get(key,[])+y.get(key,[]) for key in set(list(x.keys())+list(y.keys())) }

Output:

{1: ['a', 'b', 'c', 'd', 'e', 'f'], 2: ['g']}

View more solutions