What's the best way to initialize a dict of dicts in Python?

Question

A lot of times in Perl, I'll do something like this:

$myhash{foo}{bar}{baz} = 1

How would I translate this to Python? So far I have:

if not 'foo' in myhash:
    myhash['foo'] = {}
if not 'bar' in myhash['foo']:
    myhash['foo']['bar'] = {}
myhash['foo']['bar']['baz'] = 1

Is there a better way?

Solution 1

class AutoVivification(dict):
    """Implementation of perl's autovivification feature."""
    def __getitem__(self, item):
        try:
            return dict.__getitem__(self, item)
        except KeyError:
            value = self[item] = type(self)()
            return value

Testing:

a = AutoVivification()

a[1][2][3] = 4
a[1][3][3] = 5
a[1][2]['test'] = 6

print a

Output:

{1: {2: {'test': 6, 3: 4}, 3: {3: 5}}}

Solution 2

If the amount of nesting you need is fixed, collections.defaultdict is wonderful.

e.g. nesting two deep:

myhash = collections.defaultdict(dict)
myhash[1][2] = 3
myhash[1][3] = 13
myhash[2][4] = 9

If you want to go another level of nesting, you'll need to do something like:

myhash = collections.defaultdict(lambda : collections.defaultdict(dict))
myhash[1][2][3] = 4
myhash[1][3][3] = 5
myhash[1][2]['test'] = 6

edit: MizardX points out that we can get full genericity with a simple function:

import collections
def makehash():
    return collections.defaultdict(makehash)

Now we can do:

myhash = makehash()
myhash[1][2] = 4
myhash[1][3] = 8
myhash[2][5][8] = 17
# etc

Solution 3

Is there a reason it needs to be a dict of dicts? If there's no compelling reason for that particular structure, you could simply index the dict with a tuple:

mydict = {('foo', 'bar', 'baz'):1} # Initializes dict with a key/value pair
mydict[('foo', 'bar', 'baz')]      # Returns 1

mydict[('foo', 'unbar')] = 2       # Sets a value for a new key

The parentheses are required if you initialize the dict with a tuple key, but you can omit them when setting/getting values using []:

mydict = {}                        # Initialized the dict
mydict['foo', 'bar', 'baz'] = 1    # Sets a value
mydict['foo', 'bar', 'baz']        # Returns 1

Solution 4

I guess the literal translation would be:

 mydict = {'foo' : { 'bar' : { 'baz':1}}}

Calling:

 >>> mydict['foo']['bar']['baz']

gives you 1.

That looks a little gross to me, though.

(I'm no perl guy, though, so I'm guessing at what your perl does)

View more solutions

Comments

[mike]

That only works at initialization time, though, right?

[Kiv]

Can you be clear on when you can omit the parentheses? Is it because the comma is the tuple operator, and the parentheses only needed if we have ambiguous grouping?

[sedavidw]

or def makehash(): return collections.defaultdict(makehash); myhash = makehash()

[Sheep]

I've got no problems with "traditional" recursive functions, but there's something about that that I find unintuitive. Odd. Anyway, thanks!

[wheaties]

Thanks for this. That lambda: defaultdict() is what I needed.

[Londerson Araújo]

Unfortunately that recursive makehash approach does not handle the following a = makehash(); a['foo'] += 1; as I understand it's because the default_factory is not specified here. Do you know how to overcome this issue?

[Londerson Araújo]

Is it possible to extend it so it supports the following behavior: a[1][2][3] += some_value. So if the key did not exist in advance, then the a[1][2][3] would be initialized with the default value of the type(some_value)?

[Dave Rawks]

This function has the side effect that any attempts to get a non-existent key also creates the key. Typically you would only want to auto create a key if you were at the same time setting a key or subkey.

[Wexxor]

This just saved my bacon on a data conversion project that was overdue before I started it. I needed to do some time coalescing of log messages based on sec,msec tags; I really wanted to use hash-hash-list. tagtable = collections.defaultdict(lambda : collections.defaultdict(list)) really hit the spot.

[Cees Timmerman]

@Dana, as opposed to adding new values to mydict during runtime.

[PascalVKooten]

Is there also a way to make the assignment variable? So that given var = [1,2,3], I could do like a[var] = 1, which would expand to a[1][2][3] = 1?

[PascalVKooten]

var = [1,2] would then allow a[var] = 1 to be a[1][2] = 1

[nosklo]

@Dualinity that already works if you use normal dicts and tuples: d = {}; k = (1, 2, 3); d[k] = 1 then you can use d[1,2,3]. If you want to use d[1][2][3] (why?) you'd have to modify the recipe above, ask another question. remember tat flat is better than nested.

[PascalVKooten]

Because I want to have a structure with

[PonyEars]

makehash() is awesome. But is there a pythonic way to accomplish the same thing with a default value as well? Difficult, because the default value is a defaultdict()?

[timfeirg]

very handy it is when initializing, but it seems there's no easy way to work with this dict since there's lambda function involved, e.g. print myhash will get a bunch of <function <lambda> at ...> sort of thing.

[eric.frederich]

Another solution would be... from collections import defaultdict followed by AutoVivification = lambda: defaultdict(AutoVivification) This one seems more simple. Subclassing dict though has the side effect of the dictionary being easier to read when printed though. Source: en.wikipedia.org/wiki/Autovivification#Python

[Daniel Pérez Rada]

@AntonDaneyko to auto initialize a[1][2][3] += some_value you can overwrite the method def __iadd__(self, some_value): return some_value

[ChaimG]

This may actually be faster than nested dictionaries because there is one lookup instead of three.

[diegopso]

@timfeirg you can use json lib to overcome this: import json; print json.dumps(myhash).

[rubmz]

Yes for initializing well defined structures which are then written to json? A very common case/practice...