Execute statement every N iterations in Python

Question

I have a very long loop, and I would like to check the status every N iterations, in my specific case I have a loop of 10 million elements and I want to print a short report every millionth iteration.

So, currently I am doing just (n is the iteration counter):

if (n % 1000000==0):
    print('Progress report...')

but I am worried I am slowing down the process by computing the modulus at each iteration, as one iteration lasts just few milliseconds.

Is there a better way to do this? Or shouldn't I worry at all about the modulus operation?

Solution 1

How about keeping a counter and resetting it to zero when you reach the wanted number? Adding and checking equality is faster than modulo.

printcounter = 0

# Whatever a while loop is in Python
while (...):   
    ...
    if (printcounter == 1000000):
        print('Progress report...')
        printcounter = 0
    ...
    printcounter += 1

Although it's quite possible that the compiler is doing some sort of optimization like this for you already... but this may give you some peace of mind.

Solution 2

Sup, dawg? Contact me if you need additional comment/explanation:

1. Human-language declarations for x and n:

let x be the number of iterations that have been examined at a given time. let n be the multiple of iterations upon which your code will execute.

2. What we're doing:

The first code block (Block A) uses only one variable, x (defined above), and uses 5 (an integer) rather than the variable n (defined above).

The second code block (Block B) uses both of the variables (x and n) that are defined above. The integer, 5, will be replaced by the variable, n. So, Block B literally performs an action at each nth iteration.

Our goal is to do something every xth iteration and every 5th/nth iteration. We are going through 100 iterations.

m. Easy-to-understand code:

Block A, minimal variables:

for x in 100:
    #what to do every time (100 times): replace this line with your every-iteration functions.
    if x % 5 == 0:
        #what to do every 5th time: replace this line with your nth-iteration functions.

Block B, generalization.

n = 5
for x in 100:
    #what to do every time (100 times): replace this line with your every-iteration functions.
    if x % n == 0:
        #what to do every 5th time: replace this line with your nth-iteration functions.

Please, let me know if you have any issues because I haven't had time to test it after writing it here.

3. Exercises

1. If you've done this properly, see if you can use it with the turtle.Pen() and turtle.forward() function.
2. See if you can use this program with the turtle.circle() function.
3. Check out the reading (seen below) to attempt to improve the programs from exercise 1 and 2.

About modulo and other basic operators: https://docs.python.org/2/library/stdtypes.html http://www.tutorialspoint.com/python/python_basic_operators.htm

About turtle: https://docs.python.org/2/library/turtle.html https://michael0x2a.com/blog/turtle-examples

Solution 3

Is it really slowing down? You have to try and see for yourself. It won't be much of a slowdown, but if we're talking about nanoseconds it may be considerable. Alternatively you can convert one 10 million loop to two smaller loops:

m = 1000000
for i in range(10):
    for i in range(m):
        // do sth
    print("Progress report")

Solution 4

It's difficult to know how your system will optimize your code without testing.

You could simplify the relational part by realizing that zero is evaluated as false.

if(not N % 10000000)
   do stuff

Solution 5

Something like that ? :

for n in xrange(1000000,11000000,1000000):
    for i in xrange(n-1000000,n):
        x = 10/2
    print 'Progress at '+str(i)

result

Progress at 999999
Progress at 1999999
Progress at 2999999
Progress at 3999999
Progress at 4999999
Progress at 5999999
Progress at 6999999
Progress at 7999999
Progress at 8999999
Progress at 9999999

.

EDIT

Better:

for n in xrange(0,10000000,1000000):
    for i in xrange(n,n+1000000):
        x = 10/2
    print 'Progress at '+str(i)

And inspired from pajton:

m = 1000000
for n in xrange(0,10*m,m):
    for i in xrange(n,n+m):
        x = 10/2
    print 'Progress at '+str(i+1)

I prefer this that I find more immediately readable than the pajton's solution. It keeps the display of a value depending of i

View more solutions

Comments

[Andrea Zonca]

thanks, this looks like the best solution, but I understand I should really profile the code to have a definite answer.

[Andrea Zonca]

thanks, good solution, this is better if the total length is a multiple of the report length, AndrewKS's answer is more general.

[15ee8f99-57ff-4f92-890c-b56153]

This. The answer is "don't ever optimize without first establishing that there's really a need".

[PanDe]

This is something that is very easy to think, only if you know you can do something like this :-) Great Answer, helped me twice.