lambda in python can iterate dict?

Question

I have an interview recently. The interviewer asked me the ways to iterate dict in python. I said all the ways use for statement. But he told me that how about lambda?

I feel confused very much and I consider lambda as an anonymity function, but how it iterates a dict? some code like this:

new_dict = sorted(old_dict.items(), lambda x: x[1]) # sorted by value in dict

But in this code, the lambda is used as a function to provide the compared key. What do you think this question?

Solution 1

You don't iterate with lambda. There are following ways to iterate an iterable object in Python:

1. for statement (your answer)
2. Comprehension, including list [x for x in y], dictionary {key: value for key, value in x} and set {x for x in y}
3. Generator expression: (x for x in y)
4. Pass to function that will iterate it (map, all, itertools module)
5. Manually call next function until StopIteration happens.

Note: 3 will not iterate it unless you iterate over that generator later. In case of 4 it depends on function.

For iterating specific collections like dict or list there can be more techniques like while col: remove element or with index slicing tricks.

Now lambda comes into the picture. You can use lambdas in some of those functions, for example: map(lambda x: x*2, [1, 2, 3]). But lambda here has nothing to do with iteration process itself, you can pass a regular function map(func, [1, 2, 3]).

Solution 2

You can iterate dict using lambda like this:

d = {'a': 1, 'b': 2}
values = map(lambda key: d[key], d.keys())

Solution 3

Dictionary iteration using lambda

dct = {1: '1', 2 : '2'}

Iterating over Dictionary using lambda:

map(lambda x : str(x[0]) + x[1], dct.iteritems())

here x[0] is the key and x[1] is the value

Result : ['11', '22']

Filtering on Dictionary using lambda:

filter(lambda x : x[0] > 1, dct.iteritems())

Result : [(2, '2')]

Solution 4

lambda itself doesn't iterate anything. As you thought, it just defines an anonymous function - aside from the syntactic rule about only being able to have an expression, a lambda does nothing more than a similar function made using def. The code inside the lambda might iterate something, but only in the same ways as any other function might use (provided they are expressions, and so valid inside a lambda).

In the example you mention using sorted, the key function is called on each element of the list being sorted - but it is sorted itself that does this, and which does the iteration. When you provide a key function, sorted does something broadly similar to this:

def sorted(seq, key):
    decorated = [(key(elem), i) for i, elem in enumerate(seq)]
    # Sort using the normal tuple lexicographic comparisons
    decorated.sort()
    return [seq[i] for _,i in decorated]

As you can see, sorted does the iteration here, not the lambda. Indeed, there is no reason why the key has to be a lambda - any function (or any callable) will do as far as sorted is concerned.

---

At the lowest level, there is only really one way to iterate a dict (or, indeed, any other iterable) in Python, which is to use the iterator protocol. This is what the for loop does behind the scenes, and you could also use a while statement like this:

it = iter(my_iterable)
while True:
    try:
        val = next(it)
    except StopIteration:
        # Run else clause of for loop
        break
    else:
        # Run for loop body

The comments in this aren't strictly part of the iterator protocol, they are instead part of the for loop (but having at least a loop body in there is mostly the point of iterating in the first place).

Other functions and syntax that consume iterables (such as list, set and dict comprehensions, generator expressions or builtins like sum, sorted or max) all use this protocol, by either:

• Using a Python for loop,
• Doing something like the above while loop (especially for modules written in C),
• Delegating to another function or piece of syntax that uses one of these

---

A class can be made so that its instances become iterable in either of two ways:

• Provide the iterator protocol directly. You need a method called __iter__ (called by iter), which returns an iterator. That iterator has a method called __next__ (just next in Python 2) which is called by next and returns the value at the iterator's current location and advances it (or raises StopIteration if it is already at the end); or
• Implement part of the sequence protocol (which means behaving like a list or tuple). For forward iteration, it is sufficient to define __getitem__ in such a way that doing my_sequence[0], my_sequence[1], up until my_sequence[n-1] (where n is the number of items in the sequence), and higher indexes raise an error. You usually want to define __len__ as well, which is used when you do len(my_sequence).

Solution 5

Using a plain lambda to iterate anything in Python sounds very wrong. Certainly the most Pythonic method to iterate sequences and collections is to use list comprehensions and generator expressions like @Andrey presented.

If the interviewer was leaning on the more theoretical/Computer Sciencey answers, it is worth noting that using lambdas to iterate is quite possible, although I must stress that this is not Pythonic nor useful at any context other than academic exercises:

# the legendary Y combinator makes it possible
# to let nameless functions recurse using an indirection
Y = lambda f: (lambda x: x(x))(lambda y: f(lambda *args: y(y)(*args)))
# our iterator lambda
it = lambda f: lambda Lst: (Lst[0], f(Lst[1:])) if Lst else None
# see it in action:
Y(it)([1,2,3])
=> (1, (2, (3, None)))

View more solutions

Comments

[Marcus Müller]

without a lot more context, we can't tell you whether your interviewer had any idea of what he asked.

[jonrsharpe]

Also, note that using map and sorted and other iterating functions (enumerate, etc.) will also use for, somewhere under the hood.

[Andrey]

@jonrsharpe usually those functions are written in C and strictly speaking don't use Python for.

[wim]

Very strange question from the interviewer

[jonrsharpe]

@Andrey this is true (for CPython), but it's not clear whether that was a relevant distinction.

[chyoo CHENG]

@MarcusMüller his word means that lambda is a way to iterate dict.

[Andrey]

@jonrsharpe it just makes your statement not universally true and considering that CPython is by far most popular implementation it makes it practically mostly false.

[jonrsharpe]

@Andrey note that I (carefully) didn't say "Python for", just for. It's pointless getting into that level of detail when what we're really discussing is what the OP thought they interviewer was asking (which either or both of them may not really have understood). My point was that there is some form of for loop (and therefore O(n) operation) going on somewhere, so calling that distinct is somewhat meaningless.

[Andrey]

@chyooCHENG it doesn't make too much sense. Either he worded it differently or worded it poorly or s/he doesn't understand lambdas themself.

[Andrey]

@jonrsharpe you can iterate over iterable with while or even with recursion. My point is that if you want to be precise you should not make such assumptions on implementations of functions.

[jonrsharpe]

@Andrey that's a good point; in many ways, of course, a Python for loop is just a while that breaks on StopIteration.

[chyoo CHENG]

@Andrey I think so. He said he is good at php ... so ...

[jonrsharpe]

Strictly, map is iterating over the dictionary

[Andrey]

You better use function without side effects in map.

[jonrsharpe]

If you're going to separate comprehensions from generator expressions, you should include dictionary and set comprehensions. And note that generator expressions/comprehensions do include for as part of their syntax.

[Andrey]

@jonrsharpe they include for but OP explicitly said "for statement" so he didn't just said "for keyword". I will include dict/set comprehensions but it is not about separating comprehensions from generator expressions. Those two are principally different in nature while dict/set ones are of same nature as list.

[wim]

There are more ways than just 4. For example, you could use while d: d.popitem()

[mike3996]

You may be able to iterate using lambdas if you write a Y combinator. Pythonic? God no.

[Andrey]

@wim I meant how to iterate iterable, yeah for specific collections there are more ways

[Namita Maharanwar]

{key: value for key, value in x} not working! I tried {key: value for key, value in dict1} for interating dict1 = {'a': 1, 'b': 2}. I get "ValueError: need more than 1 value to unpack".

[mike3996]

@NamitaMaharanwar: if you iterate through a dict like that, you only iterate through the keys, not key-value-tuples. Iterate dict.iteritems() instead.

[JGFMK]

values =d.values() !!

[Borislav Markov]

AttributeError: 'dict' object has no attribute 'iteritems'