Google Espresso or Robotium

Question

I have to use Automated UI test tool and I am confused between using Robotium vs Google Espresso.

What are the major differences between the two? Are there features that exist in one but not the other?

Solution 1

Full disclosure: I am one of Espresso's authors.

Both Espresso and Robotium are instrumentation-based frameworks, meaning they use Android Instrumentation to inspect and interact with Activities under test.

At Google, we started out by using Robotium because it was more convenient than stock instrumentation (hats off to Robotium developers for making it so). However, it did not satisfy our need for a framework that made writing reliable tests easy for developers.

The major advances in Espresso over Robotium:

1. Synchronization. By default, instrumentation test logic runs on a different (instrumentation) thread than UI operations (processed on the UI thread). Without synchronization of test operations with UI updates, the tests will be prone to flakiness - i.e. will fail randomly because of timing issues. Most test authors ignore this fact, some add sleeps/retry mechanisms and even fewer implement more sophisticated thread safety code. None of these are ideal. Espresso takes care of thread safety by seamlessly synchronizing test actions and assertions with the UI of the application under test. Robotium attempts to address this with sleep/retry mechanisms, which are not only unreliable, but also cause tests to run slower than necessary.

2. API. Espresso has a small, well-defined and predictable API, which is open to customization. You tell the framework how to locate a UI element using standard hamcrest matchers and then instruct it to either perform an action or check an assertion on the target element. You can contrast this with Robotium's API, where the test author is expected to choose from 30+ click methods. Further, Robotium exposes dangerous methods like getCurrentActivity (what does current mean anyway?) and getView, which allow you to operate on objects outside of the main thread (see the point above).

3. Clear failure information. Espresso strives to provide rich debugging information when a failure happens. Further, you can customize the way failures are handled by Espresso with your own failure handler. I haven't tried it in a while, but prior versions of Robotium suffered from inconsistent failure handling (e.g. the clickOnView method would swallow SecurityExceptions).

Contrary to a previous answer, Espresso is supported on all API versions with significant number of users (see: http://developer.android.com/about/dashboards/index.html). It works on some of the older versions, but testing on those would be a waste of resources. Speaking about testing... Espresso is tested on every change by a comprehensive test suite (with over 95% coverage) as well as the majority of android applications developed by Google.

Solution 2

Espresso is much faster than Robotium, but only works on some SDK versions.

So if you want a test that works on all devices, go for Roboitum. If not, go for espresso, and don't forget you will be a beta tester for still some time.

Comments

[Snicolas]

An upvote would help me to create a synonym for this tag.. ;)

[nbe_42]

Hi ! Thank you for your answer, Does Espresso offer the possibility to test several applications in the same test case? I have to test my application that calls an activity from another application (my other app that share the same sharedUserId) and then retrieves the result. I can't do that with Robotium, but maybe with Espresso ? :-)

[ValeraZakharov]

No - you cannot interact with UI outside of your process with Espresso. This is a limitation of the instrumentation framework.

[Evin1_]

Link above changed, this is the new one: google.github.io/android-testing-support-library/docs/espres‌​so/…

[Naresh Gunda]

@ValeraZakharov:: Hii...!! As you said, espresso will take care of UI Thread Synchronization and no need to put Sleeps. But in my case, i've written few testcases and all the testcases are working in my local machine(With one sleep per TestSuite as the start). But almost 99% testcases are getting failed when i run with local/server jenkins. I've disabled all the animations in the jenkins emulator. Most of the time i’m getting AppNotIdleException. Im unable to figure out the root cause. Can you please help me out.

[Rakib]

@Radu Have done it. Your comment is null and without proper explanation seems like a foolhardy attempt to get attention.