Two independent async loops in Python

Question

What would be a good approach to execute two asynchronous loops running in parallel in Python, using async/await?

I've thought about something like the code below, but can't wrap my head around how to use async/await/EventLoop in this particular case.

import asyncio

my_list = []

def notify():
    length = len(my_list)
    print("List has changed!", length)

async def append_task():
    while True:
        time.sleep(1)
        await my_list.append(random.random())
        notify()

async def pop_task():
    while True:
        time.sleep(1.8)
        await my_list.pop()
        notify()

loop = asyncio.get_event_loop()
loop.create_task(append_task())
loop.create_task(pop_task())
loop.run_forever()

Expected output:

$ python prog.py
List has changed! 1 # after 1sec
List has changed! 0 # after 1.8sec
List has changed! 1 # after 2sec
List has changed! 2 # after 3sec
List has changed! 1 # after 3.6sec
List has changed! 2 # after 4sec
List has changed! 3 # after 5sec
List has changed! 2 # after 5.4sec

Solution 1

this works fine:

note: you wanted to await fast non-io bound operations (list.append and list.pop that are not even coroutines); what you can do is awaitasyncio.sleep(...) (which is a coroutine and yield control back to the caller):

import asyncio
import random

my_list = []


def notify():
    length = len(my_list)
    print("List has changed!", length)

async def append_task():
    while True:
        await asyncio.sleep(1)
        my_list.append(random.random())
        notify()

async def pop_task():
    while True:
        await asyncio.sleep(1.8)
        my_list.pop()
        notify()


loop = asyncio.get_event_loop()
cors = asyncio.wait([append_task(), pop_task()])
loop.run_until_complete(cors)

time.sleep itself is blocking and does not play nicely with await.

Solution 2

List objects do not have awaitable operations, nor do they need to as there is no I/O or other delay you could handle asynchronously.

You also want to use asyncio.sleep(), not time.sleep(); the latter blocks.

The following works just fine; I added in a timestamp in the notify to show this off better:

from datetime import datetime
# ...

start = datetime.now()
def notify():
    length = len(my_list)
    print("t: {:.1f}, List has changed! {}".format(
        (datetime.now() - start).total_seconds(), length))

async def append_task():
    while True:
        await asyncio.sleep(1)
        my_list.append(random.random())
        notify()

async def pop_task():
    while True:
        await asyncio.sleep(1.8)
        my_list.pop()
        notify()

Note that we use await on the asyncio.sleep() call; that provides a point where your coroutine (cooperative routine) yields control to another routine.

This produces:

$ python asyncio_demo.py
t: 1.0, List has changed! 1
t: 1.8, List has changed! 0
t: 2.0, List has changed! 1
t: 3.0, List has changed! 2
t: 3.6, List has changed! 1
t: 4.0, List has changed! 2
t: 5.0, List has changed! 3
t: 5.4, List has changed! 2
t: 6.0, List has changed! 3

Comments

[Fab]

could you explain what to do if I do have a blocking function (such as time.sleep())? All examples use this asyncio.sleep(), but I have actual long computations (or i/o bound stuff), and it's unclear to me how to wrap it such that this example would work. Thanks a lot.

[hiro protagonist]

@Fab: asyncio will not help with cpu-bound functions. and if your i/o-bound stuff is not built on asyncio it will not work either (that is why time.sleep() does not work, but asyncio.sleep() does.) you need to call coroutines that yield the program flow back to the caller to get a benefit from the async loop.

[Fab]

Thanks a lot, hiro. That clears things up considerably. I've been looking in the wrong place then!

[rbonallo]

Asyncio not available on 2.7.

[hiro protagonist]

@rbonallo that is true. but as the question is about asyncio it is about python 3. or what is your point?

[rbonallo]

No point, just to save people spending 5 mins pip installing, googling error and realizing it is sadly not available on 2.7.

[hiro protagonist]

@rbonallo ah, ok! thanks for clarifying! will edit the question and add the python 3 tag. that may make it a bit more explicit.