How to get Slurpable data from REST client in Groovy?

Question

I have code that looks like this:

def client = new groovyx.net.http.RESTClient('myRestFulURL')

def json = client.get(contentType: JSON) 
net.sf.json.JSON jsonData = json.data as net.sf.json.JSON   
def slurper = new JsonSlurper().parseText(jsonData)

However, it doesn't work! :( The code above gives an error in parseText because the json elements are not quoted. The overriding issue is that the "data" is coming back as a Map, not as real Json. Not shown, but my first attempt, I just passed the parseText(json.data) which gives an error about not being able to parse a HashMap.

So my question is: how do I get JSON returned from the RESTClient to be parsed by JsonSlurper?

Solution 1

The RESTClient class automatically parses the content and it doesn't seem possible to keep it from doing so.

However, if you use HTTPBuilder you can overload the behavior. You want to get the information back as text, but if you only set the contentType as TEXT, it won't work, since the HTTPBuilder uses the contentType parameter of the HTTPBuilder.get() method to determine both the Accept HTTP Header to send, as well was the parsing to do on the object which is returned. In this case, you need application/json in the Accept header, but you want the parsing for TEXT (that is, no parsing).

The way you get around that is to set the Accept header on the HTTPBuilder object before calling get() on it. That overrides the header that would otherwise be set on it. The below code runs for me.

@Grab(group='org.codehaus.groovy.modules.http-builder', module='http-builder', version='0.6')
import static groovyx.net.http.ContentType.TEXT

def client = new groovyx.net.http.HTTPBuilder('myRestFulURL')
client.setHeaders(Accept: 'application/json')

def json = client.get(contentType: TEXT)
def slurper = new groovy.json.JsonSlurper().parse(json)

Solution 2

The type of response from RESTClient will depend on the version of :

org.codehaus.groovy.modules.http-builder:http-builder

For example, with version 0.5.2, i was getting a net.sf.json.JSONObject back.

In version 0.7.1, it now returns a HashMap as per the question's observations.

When it's a map, you can simply access the JSON data using the normal map operations :

def jsonMap = restClientResponse.getData() def user = jsonMap.get("user") ....

Solution 3

Solution posted by jesseplymale workes for me, too.

HttpBuilder has dependencies to some appache libs, so to avoid to add this dependencies to your project, you can take this solution without making use of HttpBuilder:

def jsonSlurperRequest(urlString) {
    def url = new URL(urlString)
    def connection = (HttpURLConnection)url.openConnection()
    connection.setRequestMethod("GET")
    connection.setRequestProperty("Accept", "application/json")
    connection.setRequestProperty("User-Agent", "Mozilla/5.0")
    new JsonSlurper().parse(connection.getInputStream())
}

Comments

[JoeG]

This only works perfectly! Thanks! I will say that when I converted I missed the change to "parse" from "parseText". The "json" variable above is really a StringReader, so "parse" is the required method.

[JoeG]

However, if you need to support going through a Proxy Server, you will be far better off with HttpBuilder....

[Marc]

Looks like a nice workaround, but isn't anyone curious as to why RESTClient returns a hashmap when the response is JSON ??? I'm getting the same thing and I'd like to understand why it's returning a hashmap, and how to convert this to JSON (if this is not a bug)

[Marc]

I understand now, I will post another solution

[MarkHu]

It seems kind of contradictory/pessimistic to set the request header client.setHeaders(Accept: 'application/json') but then override the response header def json = client.get(contentType: TEXT) eh? Well-behaved servers should respect the "Accept" header, and mis-behaved ones will need an override. And yes, @Marc, it is a common bungle to conflate JSON and Maps.