COROUTINE_SUSPENDED and suspendCoroutineOrReturn in Kotlin

Question

The idea of coroutines in kotlin was to abstract the notion of suspension and callbacks and write simple sequential code. You never need to worry if the coroutine is suspended or not, similar to threads.

What is the purpose of suspendCoroutineOrReturn and COROUTINE_SUSPENDED and in what case would you use them?

Solution

The suspendCoroutineOrReturn and COROUTINE_SUSPENDED intrinsics were introduced very recently in 1.1 to address the particular stack overflow problem. Here is an example:

fun problem() = async { 
  repeat(10_000) { 
    await(work()) 
  } 
}

Where await simply waits for completion:

suspend fun <T> await(f: CompletableFuture<T>, c: Continuation<T>): Unit { 
  f.whenComplete { value, exception ->              // <- await$lambda
    if (exception != null) c.resumeWithException(exception) else 
                           c.resume(value)
  }
}

Let's have a look at the case when work doesn't really suspend, but returns the result immediately (for example, cached). The state machine, which is what coroutines are compiled into in Kotlin, is going to make the following calls: problem$stateMachine, await, CompletableFuture.whenComplete, await$lambda, ContinuationImpl.resume, problem$stateMachine, await, ...

In essence, nothing is ever suspended and the state machine invokes itself within the same execution thread again and again, which ends up with StackOverflowError.

A suggested solution is to allow await return a special token (COROUTINE_SUSPENDED) to distinguish whether the coroutine actually did suspend or not, so that the state machine could avoid stack overflow. Next, suspendCoroutineOrReturn is there to control coroutine execution. Here is its declaration:

public inline suspend fun <T> suspendCoroutineOrReturn(crossinline block: (Continuation<T>) -> Any?): T

Note that it receives a block that is provided with a continuation. Basically it is a way to access the Continuation instance, which is normally hidden away and appears only during the compilation. The block is also allowed to return any value or COROUTINE_SUSPENDED.

Since this all looks rather complicated, Kotlin tries to hide it away and recommends to use just suspendCoroutine function, which internally does all the stuff mentioned above for you. Here's the correct await implementation which avoids StackOverflowError (side note: await is shipped in Kotlin lib, and it's actually an extension function, but it's not that important for this discussion)

suspend fun <T> await(f: CompletableFuture<T>): T = 
  suspendCoroutine { c -> 
    f.whenComplete { value, exception -> 
      if (exception != null) c.resumeWithException(exception) else
                             c.resume(value)
  }
} 

But if you ever want to take over fine-graned control over coroutine continuation, you should call suspendCoroutineOrReturn and return COROUTINE_SUSPENDED whenever an external call is made.