Loop over two generator together

Question

I have two generators say A() and B(). I want to iterate over both the generators together. Something like:

for a,b in A(),B():    # I know this is wrong
    #do processing on a and b

One way is to store the results of both the functions in lists and then loop over the merged list. Something like this:

resA = [a for a in A()]
resB = [b for b in B()]
for a,b in zip(resA, resB):
    #do stuff

If you are wondering, then yes both the functions yield equal number of value.

But I can't use this approach because A()/B() returns so many values. Storing them in a list would exhaust the memory, that's why I am using generators.

Is there any way to loop over both the generators at once?

Solution 1

You were almost there. In Python 3, just pass the generators to zip():

for a, b in zip(A(), B()):

zip() takes any iterable, not just lists. It will consume the generators one by one.

In Python 2, use itertools.izip():

from itertools import izip

for a, b in izip(A(), B()):

As an aside, turning a generator into a list is as simple as list(generator); no need to use a list comprehension there.

Solution 2

Sounds like you want itertools.izip:

from itertools import izip

for a, b in izip(A(), B()):

From the docs:

Like zip() except that it returns an iterator instead of a list.

So this way you never create a list, either of A(), B() or the izip().

Note that in Python 3's basic zip is like Python 2.x's itertools.izip.

Comments

[Ciasto piekarz]

in this case both the lists has to have equal number of items, and if second list has more items it will not be printed

[jonrsharpe]

@san to be specific, whichever argument is longer will be truncated to the length of the shortest. If you need to process pairs, that is necessary. If you want to get as many items as possible, there is [i]zip_longest, although if one of your arguments is e.g. an infinitely long generator you will have a problem.