Confused about testCompile and androidTestCompile in Android Gradle

Question

I'm new to testing world and even more to Android testing world. While doing research on Robolectric that aids with tests on android one thing confuses me the most. Sometimes on the web I see people using testCompile keyword in dependencies of the gradle build script when referencing Robolectric while others use androidTestCompile. Certainly both can't be valid?

Can somebody explain the difference between the both and which of these should be the one used when using Robolectric?

Solution 1

Simply testCompile is the configuration for unit tests (those located in src/test) and androidTestCompile is used for the test api (that located in src/androidTest). Since you are intending to write unit tests, you should use testCompile.

Update: The main distinction between the two is the test sourceset runs in a regular Java JVM, whereas the androidTest sourceset tests run on an Android device (or an emulator).

Solution 2

To answer your question - Use testCompile for robolectric

why, because robolectric runs on the JVM mocking all the android device behaviour.

testCompile and androidTestCompile are "by convention" android folders which gradle uses while running tasks provided by android plugin.

androidTestDebug picks tests from androidTest folder, testDebug picks tests from test folder,

Again these are only by convention folders you can give source sets for these configurations

Note: espresso is such an awesome library try to move away from robolectric :)

Solution 3

//unit testing

testCompile 'junit:junit:4.12'

The above code is a dependency of JUnit 4 in build.gradle file in android studio. You see that it has testCompile, beacuse JUnit runs on JVM and does not require a device or emulator to run. That also means that JUnit tests will not require the application context to run and if they require we would need to "MOCK" them.

//Insturmented Unit Testing

androidTestCompile('com.android.support.test:runner:0.5', {
        exclude group: 'com.android.support', module: 'support-annotations'
    })

Now we see androidTestCompile here, because this time we intend to use the device or emulator for tests, that is Instrumentation testing. For beter clarification I would suggest to read from developer.android.com

Comments

[Lucas]

Thanks. That's what I figured at first, but if that's the case why some reference robolectric dependence with testCompile and some with androidTestCompile. Isn't that a library that helps writing integration tests? If so then shouldn't it be referenced with androidTestCompile? However even the official guide of robolectric directs to use testCompile... Sorry but it's just too confusing to me at this point as you can see.

[Mark Vieira]

The naming conventions are a little strange. Basically, if you are writing unit tests (tests that will not be run on device) then they would exist in 'src/test' and therefore their dependencies belong in the testCompile configuration. Dependencies added to the androidTestCompile configuration will only be available to the source in 'src/androidTest', which is actually built into an APK and deployed on a device.

[Lucas]

Thanks for pointing me to some direction. It did not answer all my questions but it helped me during my research. Just to clarify what you've said, the Unit Test's are not only the ones in test folder (by default). Ironicly google calls sometimes the tests located in androidTest also as Unit Test's. Depends ofcourse on the purpose of particular test, but still adds to the confussion.

[Mark Vieira]

This is mainly semantics, so I wouldn't get hung up on them. Many test written with Roboelectric are arguably integration tests, and not unit tests anyway. That being said, the main distinction between the two is that 'src/test' runs on the developers machine in a standard JVM and 'src/androidTest' is packaged in an APK and run on an actual device (or emulator).

[Lucas]

Thanks a lot. Yeah, the fact one is run on JVM and other in emulator or device is the actual distinction. Can you tell me while we're at it what changed when google added "experimental junit" with release of AS 1.1.0? I'm asking because even before then there was "test" config and "androidTest" config, you could add dependencies to both, the gradle built tasks for both integration tests as well as unit tests (compileDebugTestJava) which you could run with"test" task. Could you explain what Really changed in that regard with the introduction of experimental unit tests to how it was prior?

[Mark Vieira]

I think the support for 'src/test' that you saw before was simply that which was available via the standard Gradle Java plugin. Therefore, there was no support for build flavors or types. Now the Android plugin has full support for unit tests, to include per-variant unit test source sets.

[Eugen Martynov]

I would edit answer a bit since androidTestCompile was\is used with Robolectric also. The purpose is to allow write code of Robolectric tests in Android Studio. It was workaround before android gradle plugin version 1.1

[Ambi]

How do I add "androidTestCompile" statement in Android.mk file? Can you please help me with this?