Automated testing for REST Api

Question

I would like to write an automated testing suite for a REST API. As we complete new services, we'd like to check to make sure all the previously created services are working as expected. Any suggestions on the best tools to use to accomplish this? I know tools like Apigee exist that allow you to test 1 service at a time, but we'd like for a way to test all services with the click of a button.

Solution 1

At my work we have recently put together a couple of test suites written in Java to test some RESTful APIs we built. Our Services could invoke other RESTful APIs they depend on. We split it into two suites.

---

• Suite 1 - Testing each service in isolation
  • Mock any peer services the API depends on using restito. Other alternatives include rest-driver, wiremock and betamax.
  • Tests the service we are testing and the mocks all run in a single JVM
  • Launches the service in Jetty

I would definitely recommend doing this. It has worked really well for us. The main advantages are:

• Peer services are mocked, so you needn't perform any complicated data setup. Before each test you simply use restito to define how you want peer services to behave, just like you would with classes in unit tests with Mockito.
• You can ask the mocked peer services if they were called. You can't do these asserts as easily with real peer services.
• The suite is super fast as mocked services serve pre-canned in-memory responses. So we can get good coverage without the suite taking an age to run.
• The suite is reliable and repeatable as its isolated in it's own JVM, so no need to worry about other suites/people mucking about with an shared environment at the same time the suite is running and causing tests to fail.

---

• Suite 2 - Full End to End
  • Suite runs against a full environment deployed across several machines
  • API deployed on Tomcat in environment
  • Peer services are real 'as live' full deployments

This suite requires us to do data set up in peer services which means tests generally take more time to write. As much as possible we use REST clients to do data set up in peer services.

Tests in this suite usually take longer to write, so we put most of our coverage in Suite 1. That being said there is still clear value in this suite as our mocks in Suite 1 may not be behaving quite like the real services.

---

Solution 2

Frisby is a REST API testing framework built on node.js and Jasmine that makes testing API endpoints easy, fast, and fun. http://frisbyjs.com

Example:

var frisby = require('../lib/frisby');

var URL = 'http://localhost:3000/';
var URL_AUTH = 'http://username:password@localhost:3000/';

frisby.globalSetup({ // globalSetup is for ALL requests
  request: {
    headers: { 'X-Auth-Token': 'fa8426a0-8eaf-4d22-8e13-7c1b16a9370c' }
  }
});

frisby.create('GET user johndoe')
  .get(URL + '/users/3.json')
  .expectStatus(200)
  .expectJSONTypes({
    id: Number,
    username: String,
    is_admin: Boolean
  })
  .expectJSON({
    id: 3,
    username: 'johndoe',
    is_admin: false
  })
  // 'afterJSON' automatically parses response body as JSON and passes it as an argument
  .afterJSON(function(user) {
    // You can use any normal jasmine-style assertions here
    expect(1+1).toEqual(2);

    // Use data from previous result in next test
    frisby.create('Update user')
      .put(URL_AUTH + '/users/' + user.id + '.json', {tags: ['jasmine', 'bdd']})
      .expectStatus(200)
    .toss();
  })
.toss();

Solution 3

I collaborated with one of my coworkers to start the PyRestTest framework for this reason: https://github.com/svanoort/pyresttest

Although you can work with the tests in Python, the normal test format is in YAML.

Sample test suite for a basic REST app -- verifies that APIs respond correctly, checking HTTP status codes, though you can make it examine response bodies as well:

---
- config:
    - testset: "Tests using test app"

- test: # create entity
    - name: "Basic get"
    - url: "/api/person/"
- test: # create entity
    - name: "Get single person"
    - url: "/api/person/1/"
- test: # create entity
    - name: "Get single person"
    - url: "/api/person/1/"
    - method: 'DELETE'
- test: # create entity by PUT
    - name: "Create/update person"
    - url: "/api/person/1/"
    - method: "PUT"
    - body: '{"first_name": "Gaius","id": 1,"last_name": "Baltar","login": "gbaltar"}'
    - headers: {'Content-Type': 'application/json'}
- test: # create entity by POST
    - name: "Create person"
    - url: "/api/person/"
    - method: "POST"
    - body: '{"first_name": "Willim","last_name": "Adama","login": "theadmiral"}'
    - headers: {Content-Type: application/json}

Solution 4

I used SOAP UI for functional and automated testing. SOAP UI allows you to run the tests on the click of a button. There is also a spring controllers testing page created by Ted Young. I used this article to create Rest unit tests in our application.

Solution 5

Runscope is a cloud based service that can monitor Web APIs using a set of tests. Tests can be , scheduled and/or run via parameterized web hooks. Tests can also be executed from data centers around the world to ensure response times are acceptable to global client base.

The free tier of Runscope supports up to 10K requests per month.

Disclaimer: I am a developer advocate for Runscope.

View more solutions

Comments

[Pankaj Jangid]

You could give vREST a try. It has both, unit testing and mocks.

[vins]

JMeter is the best tool for REST API testing - Adding this comment for people who are looking for some detailed steps to test a REST API using JMeter. testautomationguru.com/how-to-test-rest-api-using-jmeter

[Piyush Chordia]

Nothing beats FRISBY - Just the perfect and the most Powerful tool for REST API testing

[Kevin M]

JMeter is overkill, not to mention has a horrible UI, for just basic functional testing of a REST api. It's meant for performance/load testing.

[Joxi]

JMeter is focused more on load testing, maybe you should check 12 Great Web Service Testing Tools to find the best option. Some of the tools from this list, for example SOAPUI or HttpMaster, have a pretty decent automation support for REST API endpoints.

[Pratik Singhal]

I have developed a simple framework for basic use cases when testing REST endpoints, you could give it a try. (github.com/ps06756/Rest-Test)

[Peter Thomas]

Besides REST and SOAP, Karate is one of the few frameworks that is good for testing GraphQL as well: github.com/intuit/karate

[lnaie]

Related to Azure REST API testing, here is an useful process around Postman: blogs.msdn.microsoft.com/visualstudioalmrangers/2017/09/24/…

[Piyush Chordia]

I voted this article, I used it in my daily work and now its all frisbies being tossed around. I've shared about the same on my blog: cubicrace.com/2016/04/frisby-rest-api-automation-framework.h‌​tml

[Karthic.K]

automationextra.blogspot.in/2015/03/…

[Ernst Robert]

Frisby is dead github.com/vlucas/frisby/issues/347

[Kasper Holdum]

Frisby is easy to get started with and powerful enough to test even advanced API flows. Sadly the report output leaves a lot to be desired. The error messages when the API is failing is almost to the point of being useless. Which is strange given that the output when you manually run the tests are quite good. It's a shame.

[agfe2]

With authentication , you add below to the each test refered from github.com/svanoort/pyresttest/blob/master/quickstart.md with below headers; - auth_username: "foobar" - auth_password: "secret" - expected_status: [200]

[S.Huston]

In case someone stumbles on this the way I did, frisby does not seem to be dead yet as a comment noted above. The git has recent updates. github.com/vlucas/frisby

[Rao]

Link seems to be broken.

[everdream]

Updated link: theodoreyoung.wordpress.com/2011/02/14/…