Which is the most efficient way to iterate through a list in python?

Question

Say I have a list of items:

x = [1, 2, 3, 4, 5]

I need to perform some functions for each of these items. In a certain case, I need to return the index of an item.

Which is the best and most efficient way?

for item in list:
    ....

or

for i in range(len(list)):
    ....

Solution 1

for item in list:

its obviously the one with fewer function calls.

If you want to get the index of items as you go use enumerate like this

for pos, item in enumerate(collection):

Solution 2

def loop_1(data):
    for i in range(len(data)):
        print(data[i])


def looper_2(data):
    for val in data:
        print(val)

Checking with dis gives us the following bytecode for loop_1:

 12       0 SETUP_LOOP              40 (to 43)
          3 LOAD_GLOBAL              0 (range)
          6 LOAD_GLOBAL              1 (len)
          9 LOAD_FAST                0 (data)
         12 CALL_FUNCTION            1
         15 CALL_FUNCTION            1
         18 GET_ITER            
    >>   19 FOR_ITER                20 (to 42)
         22 STORE_FAST               1 (i)

13       25 LOAD_GLOBAL              2 (print)
         28 LOAD_FAST                0 (data)
         31 LOAD_FAST                1 (i)
         34 BINARY_SUBSCR       
         35 CALL_FUNCTION            1
         38 POP_TOP             
         39 JUMP_ABSOLUTE           19
    >>   42 POP_BLOCK           
    >>   43 LOAD_CONST               0 (None)
         46 RETURN_VALUE        

The bytecode for loop_2 looks like this:

17        0 SETUP_LOOP              24 (to 27)
          3 LOAD_FAST                0 (data)
          6 GET_ITER            
    >>    7 FOR_ITER                16 (to 26)
         10 STORE_FAST               1 (val)

18       13 LOAD_GLOBAL              0 (print)
         16 LOAD_FAST                1 (val)
         19 CALL_FUNCTION            1
         22 POP_TOP             
         23 JUMP_ABSOLUTE            7
    >>   26 POP_BLOCK           
    >>   27 LOAD_CONST               0 (None)
         30 RETURN_VALUE

The second version is obviously better.

Solution 3

Another possible solution would be to use numpy which would be very efficient, for large lists perhaps even more efficient than a list comprehension or a for loop.

import numpy as np

a = np.arange(5.0)   # a --> array([0., 1., 2., 3., 4.])

# numpy operates on arrays element by element 
#
b =3.*a              # b --> array([0., 3., 6., 9., 12.])

This is a pretty simple operation but you can get more complex using an array as simply an argument in a formula. For large arrays this can be much faster than a list comprehension and it makes the code cleaner and easier to read (no need to create a function to map in a list comprehension). You can also use indexing and slicing to tailor what you want to do:

If you want to have access to the actual index positions use ndenumerate

# b is as above
for i, x in np.ndenumerate(b):
    print i, x

The output of this for loop is:

(0,) 0.0 
(1,) 3.0 
(2,) 6.0 
(3,) 9.0 
(4,) 12.0 

NOTE: the index returned as a tuple by numpy to handle additional dimensions. Here we only have a single dimension so you'd have to unpack the tuple to get the index of the element.

Solution 4

Obviously for i in range(len(list)): will be slower - in python 2, it's equivalent to this:

list2 = range(len(list))

for i in list2:
    ...

If that were faster, then this would be even faster, right?

list2 = range(len(list))
list3 = range(len(list2))
list4 = range(len(list3))

for i in list4:
    ...

View more solutions