Android unit test case automation: Robolectric library vs Android Testing framework

Question

Wondering which one is the better choice to write unit test cases for android apps and libraries: Using Robolectric library or sticking with Android Testing framework. I want to run test suite at commandline and want it be independent of need of configuring emulator or letting a device attached with build machine. Does anyone of you run a comparative analysis on both of these or something better? Your experiences will be great help me to decide on the better solution.

Solution 1

I use a tiered system, where I prefer earlier tiers where possible:

1. Pure unit tests. I try to make as much code as possible fully independent of Android APIs, and then use "pure" unit tests which can run on any JVM. These tests are the fastest, and it helps keep code that has no need to be Android-specific portable.
2. Robolectric-supported unit tests. Where my code has only small dependencies on Android APIs, that can be satisfied by Robolectric shadows, I test it with Robolectric. There is a little more setup time for Robolectric compared to pure tests, but it's still faster than starting/running on an emulator.
3. Android framework tests. Where Robolectric doesn't cut it - either because the shadows don't exist, or because I'm heavily using Android APIs (and therefore want to test against the Real Thing) - I write test that run on the emulator/device with the default framework.

The point of the tiers is to keep things as simple as possible, which keeps the full suite faster and helps promote cleaner code.

Solution 2

I worked on both, what i found is :-

1) Robolectric do not support API 19, it's mention in its document - http://robolectric.org/eclipse-quick-start/. It's a great disadvantage of it.

2) Robolectric run on JVM not on DVM. So we can not detect that on that particular time GPS is enable in device or not etc. We can only pass our pre-decided value for it.

3) Code writing in Robolectric is complex than junit specially for fragment there are lot of complexity and issues.

4) Robolectric needs external jar and configuration and for junit test we do not need any external library.

5) Robolectric is faster because it runs on JVM but this have disadvantage too, we can not see UI on our device, what screen code is executing.

For Android, i like jUnit test.

Comments

[Suragch]

This question is pretty old. As I read the documentation now, it seems like the Android testing framework is the way to go. These are not command line, but for those just getting started, here are some examples.

[bianca]

Do you create multiple dedicated test projects tom implement tiers?

[Jason Sankey]

I've used just two projects: one for both types of unit tests (pure and Robotium), the other for framework tests.

[Jason Sankey]

I haven't been using Robotium in my most recent setups, but mainly because it didn't suit the app I was working on (which had a largely OpenGL UI).

[Jayesh Bhoot]

@JasonSankey, where do you test the pure unit tests? Through Roboelectric or on emulator? The latter will only increase the execution time again, and I am looking forward to decrease time for "pure" tests.

[Jason Sankey]

The pure unit tests are those that don't need Robolectric or an emulator, because they don't touch Android APIs. They run on a regular JVM with stock Java APIs, and are thus very fast.

[Jason Sankey]

Yes Adam, I use JUnit 4, although you could use any framework you're comfortable with for these tests.