What does asyncio.create_task() do?

Question

What does asyncio.create_task() do? I have looked at the docs and can't seem to understand it. A bit of code that confuses me is this:

import asyncio

async def counter_loop(x, n):
    for i in range(1, n + 1):
        print(f"Counter {x}: {i}")
        await asyncio.sleep(0.5)
    return f"Finished {x} in {n}"

async def main():
    slow_task = asyncio.create_task(counter_loop("Slow", 4))
    fast_coro = counter_loop("Fast", 2)

    print("Awaiting Fast")
    fast_val = await fast_coro
    print("Finished Fast")

    print("Awaiting Slow")
    slow_val = await slow_task
    print("Finished Slow")

    print(f"{fast_val}, {slow_val}")

asyncio.run(main())

This gives the following output:

001 | Awaiting Fast
002 | Counter Fast: 1
003 | Counter Slow: 1
004 | Counter Fast: 2
005 | Counter Slow: 2
006 | Finished Fast
007 | Awaiting Slow
008 | Counter Slow: 3
009 | Counter Slow: 4
010 | Finished Slow
011 | Finished Fast in 2, Finished Slow in 4

I don't understand quite how this is working.

1. Shouldn't the slow_task not be able to run until the completion of the fast_coro because it was never used in an asyncio.gather method?
2. Why do we have to await slow_task?
3. Why is Awaiting Slow printed after the coroutine appears to have started?
4. What really is a task? I know that what gather is doing is scheduling a task. And create_task supposedly creates a task.

An in-depth answer would be greatly appreciated. Thanks!

It also might be worth mentioning that I know very little about Futures.

Solution

What does asyncio.create_task() do?

It submits the coroutine to run "in the background", i.e. concurrently with the current task and all other tasks, switching between them at await points. It returns an awaitable handle called a "task" which you can also use to cancel the execution of the coroutine.

It's one of the central primitives of asyncio, the asyncio equivalent of starting a thread. (In the same analogy, awaiting the task with await is the equivalent of joining a thread.)

Shouldn't the slow_task not be able to run until the completion of the fast_coro

No, because you explicitly used create_task to start slow_task in the background. Had you written something like:

    slow_coro = counter_loop("Slow", 4)
    fast_coro = counter_loop("Fast", 2)
    fast_val = await fast_coro

...indeed slow_coro would not run because no one would have yet submitted it to the event loop. But create_task does exactly that: submit it to the event loop for execution concurrently with other tasks, the point of switching being any await.

because it was never used in an asyncio.gather method?

asyncio.gather is not the only way to achieve concurrency in asyncio. It's just a utility function that makes it easier to wait for a number of coroutines to all complete, and submit them to the event loop at the same time. create_task does just the submitting, it should have probably been called start_coroutine or something like that.

Why do we have to await slow_task?

We don't have to, it just serves to wait for both coroutines to finish cleanly. The code could have also awaited asyncio.sleep() or something like that. Returning from main() (and the event loop) immediately with some tasks still pending would have worked as well, but it would have printed a warning message indicating a possible bug. Awaiting (or canceling) the task before stopping the event loop is just cleaner.

What really is a task?

It's an asyncio construct that tracks execution of a coroutine in a concrete event loop. When you call create_task, you submit a coroutine for execution and receive back a handle. You can await this handle when you actually need the result, or you can never await it, if you don't care about the result. This handle is the task, and it inherits from Future, which makes it awaitable and also provides the lower-level callback-based interface, such as add_done_callback.

Comments

[Pynchia]

As the log shows, a task is executed immediately

[BeastCoder]

Thanks! I have just two follow up questions.

[BeastCoder]

1. What would happen if you did asyncio.gather(list_of_tasks)? 2. Does create_task return the results of whatever is running? Or would the only way to get the return statements be to use await and hope that it hadn't finished in the background? 3. Gather also converts all coroutines into tasks, right? But, it waits for them to finish.

[user4815162342]

@BeastCoder asyncio.gather(*list_of_tasks) would wait for the tasks to finish and return the result of their respective coroutines. create_task returns a task, which you can await to get the result of whatever is (was) running. If it had already finished, don't worry, it'll keep the result anyway. Yes, gather calls create_task if needed for your convenience - this is because it supports awaitables other than coroutines.

[BeastCoder]

Ah, thanks for clearing that up. And, gather behind the scenes is basically creating these tasks and then awaiting them?

[user4815162342]

@BeastCoder Pretty much, yes. Something like tasks = [asyncio.create_task(c) for c in list_of_coros]; results = [await t for t in tasks] would be a good first approximation of what gather does. The implementation is much more complex, however, because gather is very careful about how it propagates exceptions, how it responds to cancelation, as well as various other concerns.

[Arnab Mukherjee]

One question: when you said create tasks executes coroutines in parallel or concurrent execution? Can you elaborate on it. Does it execute in separate thread ?

[user4815162342]

@ArnabMukherjee It executes in the same thread, so I guess it would be concurrent rather than truly parallel.

[Arnab Mukherjee]

@user4815162342 so what ever tasks we wrap with create_task it will executed in background. We can await and check the results if we want. What is the role of event loop in the tasks ?

[user4815162342]

@ArnabMukherjee There is no real "background", they run when you await something, and they are ready to run. These comments should be posted as a separate question; see also this one.

[Arnab Mukherjee]

@user4815162342 i will draft a question as you said, thanks for sharing knowledge.

[gratis224]

Running this code, "counter fast: 1" always appears to be printed first. Is there a deterministic reason for this? If there's a task already submitted by create_task and then an await coroutine call, is the await coroutine guaranteed to run first and if so why? ..or this is just an implementation-dependent determinism

[user4815162342]

@aleks224 If you are referring to the code exactly as in the question, there is. In general asyncio doesn't guarantee the order in which runnable tasks are executed, but here the code awaits a coroutine, not a task. The code in a directly awaited coroutine is immediately executed (without yielding to the event loop) up to the first suspension. Since there is no suspension between await fast_coro and the first print, it will always before anything that comes from another task.

[user4815162342]

@aleks224 Immediately executing awaited coroutine up to a suspension is guaranteed, although you might find it hard to find chapter and verse in the docs. Basically it follows from await being both specified and implemented in terms of yield from, and this is how yield from must behave to satisfy the refactoring principle of PEP 380.

[Chiefir]

@user4815162342 should we always call .create_task() before calling .gether() this task?

[user4815162342]

@Chiefir No, in fact it should never be necessary because asyncio.gather() creates the task for you.