Unit testing coroutines runBlockingTest: This job has not completed yet

Question

Please find below a function using a coroutine to replace callback :

override suspend fun signUp(authentication: Authentication): AuthenticationError {
    return suspendCancellableCoroutine {
        auth.createUserWithEmailAndPassword(authentication.email, authentication.password)
            .addOnCompleteListener(activityLifeCycleService.getActivity()) { task ->
                if (task.isSuccessful) {
                    it.resume(AuthenticationError.SignUpSuccess)
                } else {
                    Log.w(this.javaClass.name, "createUserWithEmail:failure", task.exception)
                    it.resume(AuthenticationError.SignUpFail)
                }
            }
    }
}

Now I would like to unit testing this function. I am using Mockk :

  @Test
  fun `signup() must be delegated to createUserWithEmailAndPassword()`() = runBlockingTest {

      val listener = slot<OnCompleteListener<AuthResult>>()
      val authentication = mockk<Authentication> {
        every { email } returns "email"
        every { password } returns "pswd"
      }
      val task = mockk<Task<AuthResult>> {
        every { isSuccessful } returns true
      }

      every { auth.createUserWithEmailAndPassword("email", "pswd") } returns
          mockk {
            every { addOnCompleteListener(activity, capture(listener)) } returns mockk()
          }

    service.signUp(authentication)

      listener.captured.onComplete(task)
    }

Unfortunately this test failed due to the following exception : java.lang.IllegalStateException: This job has not completed yet

I tried to replace runBlockingTest with runBlocking but the test seems to wait in an infinite loop.

Can someone help me with this UT please?

Thanks in advance

Solution 1

As can be seen in this post:

This exception usually means that some coroutines from your tests were scheduled outside the test scope (more specifically the test dispatcher).

Instead of performing this:

private val networkContext: CoroutineContext = TestCoroutineDispatcher()

private val sut = Foo(
  networkContext,
  someInteractor
)

fun `some test`() = runBlockingTest() {
  // given
  ...

  // when
  sut.foo()

  // then
  ...
}

Create a test scope passing test dispatcher:

private val testDispatcher = TestCoroutineDispatcher()
private val testScope = TestCoroutineScope(testDispatcher)
private val networkContext: CoroutineContext = testDispatcher

private val sut = Foo(
  networkContext,
  someInteractor
)

Then in test perform testScope.runBlockingTest

fun `some test`() = testScope.runBlockingTest {
  ...
}

See also Craig Russell's "Unit Testing Coroutine Suspend Functions using TestCoroutineDispatcher"

Solution 2

This is not an official solution, so use it at your own risk.

This is similar to what @azizbekian posted, but instead of calling runBlocking, you call launch. As this is using TestCoroutineDispatcher, any tasks scheduled to be run without delay are immediately executed. This might not be suitable if you have several tasks running asynchronously.

It might not be suitable for every case but I hope that it helps for simple cases.

You can also follow up on this issue here:

• https://github.com/Kotlin/kotlinx.coroutines/issues/1204

If you know how to solve this using the already existing runBlockingTest and runBlocking, please be so kind and share with the community.

class MyTest {
    private val dispatcher = TestCoroutineDispatcher()
    private val testScope = TestCoroutineScope(dispatcher)

    @Test
    fun myTest {
       val apiService = mockk<ApiService>()
       val repository = MyRepository(apiService)
       
       testScope.launch {
            repository.someSuspendedFunction()
       }
       
       verify { apiService.expectedFunctionToBeCalled() }
    }
}

Solution 3

In case of Flow testing:

• Don't use flow.collect directly inside runBlockingTest. It should be wrapped in launch
• Don't forget to cancel TestCoroutineScope in the end of a test. It will stop a Flow collecting.

Example:

class CoroutinesPlayground {

    private val job = Job()
    private val testDispatcher = StandardTestDispatcher()
    private val testScope = TestScope(job + testDispatcher)

    @Test
    fun `play with coroutines here`() = testScope.runBlockingTest {

        val flow = MutableSharedFlow<Int>()

        launch {
            flow.collect { value ->
                println("Value: $value")
            }
        }

        launch {
            repeat(10) { value ->
                flow.emit(value)
                delay(1000)
            }
            job.cancel()
        }
    }
}

Solution 4

There is an open issue for this problem: https://github.com/Kotlin/kotlinx.coroutines/issues/1204

The solution is to use the CoroutineScope intead of the TestCoroutinScope until the issue is resolved, you can do by replacing

@Test
fun `signup() must be delegated to createUserWithEmailAndPassword()`() = 
runBlockingTest {

with

@Test
fun `signup() must be delegated to createUserWithEmailAndPassword()`() = 
runBlocking {

Solution 5

According to my understanding, this exception occurs when you are using a different dispatcher in your code inside the runBlockingTest { } block with the one that started runBlockingTest { }.

So in order to avoid this, you first have to make sure you inject Dispatchers in your code, instead of hardcoding it throughout your app. If you haven't done it, there's nowhere to begin because you cannot assign a test dispatcher to your test codes.

Then, in your BaseUnitTest, you should have something like this:

@get:Rule
val coroutineRule = CoroutineTestRule()

@ExperimentalCoroutinesApi
class CoroutineTestRule(
    val testDispatcher: TestCoroutineDispatcher = TestCoroutineDispatcher()
) : TestWatcher() {

    override fun finished(description: Description?) {
        super.finished(description)
        Dispatchers.setMain(testDispatcher)
    }

    override fun starting(description: Description?) {
        super.starting(description)
        Dispatchers.resetMain()
        testDispatcher.cleanupTestCoroutines()
    }
}

Next step really depends on how you do Depedency Injection. The main point is to make sure your test codes are using coroutineRule.testDispatcher after the injection.

Finally, call runBlockingTest { } from this testDispatcher:

@Test
fun `This should pass`() = coroutineRule.testDispatcher.runBlockingTest {
    //Your test code where dispatcher is injected
}

View more solutions

Comments

[Ravi]

You can replace runBlockingTest on runBlocking or try to use workarounds described here: github.com/Kotlin/kotlinx.coroutines/issues/1204

[theapache64]

Is this solution supposed to work with non-android projects also ? (JVM + Coroutines + JUnit)

[azizbekian]

@theapache64, haven't tried that personally, but cannot see why that shouldn't work. Have you stumbled on obstacle in a non-android project?

[theapache64]

Yeah. I've tried and failed. Logically,it should work. but it isn't.

[azizbekian]

"Failed" is a bit broad term, what exactly didn't work. Maybe some error message?

[theapache64]

It says This job has not completed yet, before implementing the solution, and it says the same after implementing the solution. (but it's working with Android)

[azizbekian]

Hmmmm, sounds interesting. Not sure how I can help there, sorry. But please drop a message here if you find out what was the issue.

[azizbekian]

Sorry to hear that, but this was the approach that worked for me, that's why I posted it here. Obviously, that's not working for some specific cases (or working for some specific cases, not sure on this one).

[Akbolat SSS]

Who said that this is liveData? You better check what is inside of this lib, before asking such comments

[Akbolat SSS]

This Rule is for managing Executor( plain Java stuff)

[Akbolat SSS]

Now it's not liveDataRule :P

[Bob Liberatore]

You really shouldn't replace runBlocking with Launch. The tests can give false positives if the test finishes before the coroutine does. RunBlocking exists specifically to address this.

[Heitor Colangelo]

Hey, @RobertLiberatore, in this case, the testScope is using TestCoroutineDispatcher as a dispatcher, and using this makes any task scheduled to be run without delay be executed immediately. I'm not using runBlocking and runBlockingTest because these just don't work for the case. If you have suggestions on how to make them work for this case please share them with the community.

[Bob Liberatore]

Per Android's best practices with coroutines: "Use the TestCoroutineDispatcher's runBlockingTest in the test body to wait for all coroutines that use that dispatcher to complete... As all the coroutines created by the classes under test use the same TestCoroutineDispatcher, and the test body waits for them to execute using runBlockingTest, your tests become deterministic and won't suffer from race conditions." Maybe you're willing to ride without a helmet, but I personally consider it bad advice.

[Heitor Colangelo]

I don't think you got the point, or I wasn't clear. I'm aware that it is not the best way and I never said the opposite, but runBlocking or runBlockingTest just doesn't work for this case. I would love to keep with the best practices, that's why I asked you to share an alternative solution. Using the same analogy, if the helmet that they provide for you to ride is too small for your head, so you have no choice but to go without it. I'm looking forward to your solution for this issue following the best practices.

[Bob Liberatore]

If you want to improve your answer, I suggest offering some sort of deeper explanation as to why runBlockingTest, runBlocking, and CoroutineTestDispatcher.runBockingTest don't work for OP's case, making your answer acceptable.

[Bob Liberatore]

Final note: TestCoroutineDispatcher skips over coroutine delays, but it does not guarantee that tests finish deterministically, let alone "immediately".

[Heitor Colangelo]

I added a couple of more explanations in my answer. You are assuming that not go to the ride is a possible solution, then the choice is easy. I'm looking forward to your solution for this issue following the best practices.

[Bob Liberatore]

This is still a bad answer for the question. I've had my say.

[Bob Liberatore]

Also, for anyone reading this: Azizbekian's answer follows best practices, will work if you implement it correctly, and the test will be robust (i.e. will still be correct even if you change the working code under test).

[Chris.Jenkins]

This is the first useful answer for testing flows that actually helped. Thank you!

[ThaiPD]

Don't use flow.collect directly inside runBlockingTest. It should be wrapped in launch <--- Could you explain why?

[Mikhail Sharin]

@ThaiPD otherwise, the collect will suspend entire method and you won't be able to cancel it in the end.

[konnovdev]

The question is about coroutines, not livedata

[Atif Afridi]

TestCoroutineDispatcher is deprecated.