How to sum dict elements

Question

In Python, I have list of dicts:

dict1 = [{'a':2, 'b':3},{'a':3, 'b':4}]

I want one final dict that will contain the sum of all dicts. I.e the result will be: {'a':5, 'b':7}

N.B: every dict in the list will contain same number of key, value pairs.

Solution 1

A little ugly, but a one-liner:

dictf = reduce(lambda x, y: dict((k, v + y[k]) for k, v in x.iteritems()), dict1)

Solution 2

You can use the collections.Counter

counter = collections.Counter()
for d in dict1: 
    counter.update(d)

Or, if you prefer oneliners:

functools.reduce(operator.add, map(collections.Counter, dict1))

Solution 3

Leveraging sum() should get better performance when adding more than a few dicts

>>> dict1 = [{'a':2, 'b':3},{'a':3, 'b':4}]
>>> from operator import itemgetter
>>> {k:sum(map(itemgetter(k), dict1)) for k in dict1[0]}        # Python2.7+
{'a': 5, 'b': 7}
>>> dict((k,sum(map(itemgetter(k), dict1))) for k in dict1[0])  # Python2.6
{'a': 5, 'b': 7}

adding Stephan's suggestion

>>> {k: sum(d[k] for d in dict1) for k in dict1[0]}            # Python2.7+
{'a': 5, 'b': 7}
>>> dict((k, sum(d[k] for d in dict1)) for k in dict1[0])      # Python2.6
{'a': 5, 'b': 7}

I think Stephan's version of the Python2.7 code reads really nicely

Solution 4

This might help:

def sum_dict(d1, d2):
    for key, value in d1.items():
        d1[key] = value + d2.get(key, 0)
    return d1

>>> dict1 = [{'a':2, 'b':3},{'a':3, 'b':4}]
>>> reduce(sum_dict, dict1)
{'a': 5, 'b': 7}

Solution 5

I was interested in the performance of the proposed Counter, reduce and sum methods for large lists. Maybe someone else is interested in this as well. You can have a look here: https://gist.github.com/torstenrudolf/277e98df296f23ff921c

I tested the three methods for this list of dictionaries:

dictList = [{'a': x, 'b': 2*x, 'c': x**2} for x in xrange(10000)]

the sum method showed the best performance, followed by reduce and Counter was the slowest. The time showed below is in seconds.

In [34]: test(dictList)
Out[34]: 
{'counter': 0.01955194902420044,
 'reduce': 0.006518083095550537,
 'sum': 0.0018319153785705566}

But this is dependent on the number of elements in the dictionaries. the sum method will slow down faster than the reduce.

l = [{y: x*y for y in xrange(100)} for x in xrange(10000)]

In [37]: test(l, num=100)
Out[37]: 
{'counter': 0.2401433277130127,
 'reduce': 0.11110662937164306,
 'sum': 0.2256883692741394}

View more solutions

Comments

[Nazmul Hasan]

actually i have a object list and this dictionary is an object property, have any solution? :(

[Nazmul Hasan]

[ob1, ob2, ob3].. each object has a property data ob1.data it returns a dict {'a':2, 'b':3} like that

[carl]

dictf = reduce(lambda x, y: dict((k, v + y.data[k]) for k, v in x.data.iteritems()), dict1

[Nazmul Hasan]

my code working by this solution ..ok...reduce(lambda x, y: dict((k, v + y.get_local_expenses()[k] if not isinstance(y.get_local_expenses()[k], dict) else 0) for k, v in x.get_local_expenses().iteritems()), glc)

[John La Rooy]

You can simplify the first for loop to final={}.fromkeys(dict1[0],0). or is that what "readable" is getting at? :)

[John Machin]

@nazmul hasan: Will you understand that in 6 months time? You have written get_local_expenses() 3 times -- is that necessary? What is glc? Have you read @paxdiablo's answer?

[paxdiablo]

I could simplify the whole thing into carl's answer but that would mean (1) I may as well delete my answer; and (2) I wouldn't be able to read it next month when I found I needed a small change :-) I should mention that I use Python for teaching (my brand of Python rather than carl's brand). It really is a good language for teaching the basics (sequence, iteration, selection) to kids but, if you're going to whack them over the head with lambdas and such, you may as well teach them F# or Haskell.

[John La Rooy]

or sum(map(collections.Counter, dict1),Counter()). But I am not sure about the relative performance of the functional versions creating all those Counters()

[carl]

Perhaps the only good thing about my answer is that it utilizes some cool concepts: reduce, lambda, generator expressions, and iterators.

[Nazmul Hasan]

@John Machin: glc is a list of object get_local_expense() is a property of each object and return a dictionary dictionary value may be another dict or str value or deciamal value.....i have read the @paxdiablo's ans and i solved it before him myself but i like one-liner solution :)

[kriss]

@paxdiablo: for readability, you can fully remove the init loop, just replace + d[k] with + res.get(k, 0)

[paxdiablo]

That's a good one, @kriss, and still very readable but I think you meant replace final[k] + d[k] with final.get(k,0) + d[k] - it's the final dictionary that I was requiring a default value from if the key didn't exist - I know it esists for d.

[stephan]

Is there any reason you use map and itemgetter instead of list comprehension in the inner loop (i.e. dict((k, sum(d[k] for d in dict1)) for k in dict1[0]))?

[Duncan]

This answer demonstrates the golden rule of Python programming: if it comes included with Python don't reinvent the wheel. One point: the final result counter is an instance of a subclass of dict, if the OP wants a plain dict he might add a final counter = dict(counter).

[John La Rooy]

@stephan, It used to be faster..seems to be about the same speed now. I'll add it to my answer

[kriss]

@paxdiablo: oups! Yes, your are absolutely right, I inverted dictionnaries.

[kriss]

@paxdiablo: you may even go one step further (but I wonder for readibality, see by yourself). Replace the inner loop with final = dict((k, v + final.get(k, 0)) for k, v in d.iteritems())

[stephan]

Thanks, I didn't know that. +1

[paxdiablo]

Now it's just starting to look like "Είναι ένα, ένα χιλιόμετρο μακρινός από εδώ" as in useful, but not overly comprehensible (unless you're steeped in the language).

[ygbr]

A nice addition to this version that also checks the dict types making sure we can do mathematical operations on top of them: {k: sum(d[k] if type(d[k]) in (int, float) else 0 for d in dict1) for k in dict1[0]}

[roob]

If the dicts do not all have the same keys, the first solution will output the result with only the keys shared between all dicts, while the second one-liner solution will output the result with all keys (treating missing keys as value 0)