Pulling multiple values from JSON response using RegEx Extractor

Question

I'm testing a web service that returns JSON responses and I'd like to pull multiple values from the response. A typical response would contain multiple values in a list. For example:

{
"name":"@favorites",
"description":"Collection of my favorite places",
"list_id":4894636,
}

A response would contain many sections like the above example.

What I'd like to do in Jmeter is go through the JSON response and pull each section outlined above in a manner that I can tie the returned name and description as one entry to iterate over.

What I've been able to do thus far is return the name value with regular expression extractor ("name":"(.+?)") using the template $1$. I'd like to pull both name and description but can't seem to get it to work. I've tried using a regex "name":"(.+?)","description":"(.+?)" with a template of $1$$2$ without any success.

Does anyone know how I might pull multiple values using regex in this example?

Solution 1

It may be worth to use BeanShell scripting to process JSON response.

So if you need to get ALL the "name/description" pairs from response (for each section) you can do the following:
1. extract all the "name/description" pairs from response in loop;
2. save extracted pairs in csv-file in handy format;
3. read saved pairs from csv-file later in code - using CSV Data Set Config in loop, e.g.

JSON response processing can be implemented using BeanShell scripting (~ java) + any json-processing library (e.g. json-rpc-1.0):
- either in BeanShell Sampler or in BeanShell PostProcessor;
- all the required beanshell libs are currently provided in default jmeter delivery;
- to use json-processing library place jar into JMETER_HOME/lib folder.

Schematically it will look like:

1. in case of BeanShell PostProcessor:

   Thread Group
       . . .
       YOUR HTTP Request
           BeanShell PostProcessor    // added as child
       . . .
   

2. in case of BeanShell Sampler:

   Thread Group
       . . .
       YOUR HTTP Request
       BeanShell Sampler          // added separate sampler - after your
       . . .
   

In this case there is no difference which one use.

You can either put the code itself into the sampler body ("Script" field) or store in external file, as shown below.

Sampler code:

import java.io.*;
import java.util.*;
import org.json.*;
import org.apache.jmeter.samplers.SampleResult;

ArrayList nodeRefs = new ArrayList();
ArrayList fileNames = new ArrayList();

String extractedList = "extracted.csv";
StringBuilder contents = new StringBuilder();

try
{
    if (ctx.getPreviousResult().getResponseDataAsString().equals("")) {
        Failure = true;
        FailureMessage = "ERROR: Response is EMPTY.";
        throw new Exception("ERROR: Response is EMPTY.");
    } else {
        if ((ResponseCode != null) && (ResponseCode.equals("200") == true)) {
            SampleResult result = ctx.getPreviousResult();    
            JSONObject response = new JSONObject(result.getResponseDataAsString());

            FileOutputStream fos = new FileOutputStream(System.getProperty("user.dir") + File.separator + extractedList);

            if (response.has("items")) {
                JSONArray items = response.getJSONArray("items");

                if (items.length() != 0) {
                    for (int i = 0; i < items.length(); i++) {
                        String name = items.getJSONObject(i).getString("name");
                        String description = items.getJSONObject(i).getString("description");
                        int list_id = items.getJSONObject(i).getInt("list_id");

                        if (i != 0) {
                            contents.append("\n");
                        }

                        contents.append(name).append(",").append(description).append(",").append(list_id);
                        System.out.println("\t " + name + "\t\t" + description + "\t\t" + list_id);
                    }
                }                                       
            }

            byte [] buffer = contents.toString().getBytes();    

            fos.write(buffer);
            fos.close();
        } else {
            Failure = true;
            FailureMessage = "Failed to extract from JSON response.";
        }
    }
}
catch (Exception ex) {
    IsSuccess = false;
    log.error(ex.getMessage());
    System.err.println(ex.getMessage());
}
catch (Throwable thex) {
    System.err.println(thex.getMessage());
}

As well a set of links on this:

• JSON in JMeter
• Processing JSON Responses with JMeter and the BSF Post Processor

---

Upd. on 08.2017:

At the moment JMeter has set of built-in components (merged from 3rd party projects) to handle JSON without scripting:

• JSON Path Extractor (contributed from ATLANTBH jmeter-components project);
• JSON Extractor (contributed from UBIK Load Pack since JMeter 3.0) - see answer below.

Solution 2

You can just add (?s) to the regex to avoid line breaks.

E.g: (?s)"name":"(.+?)","description":"(.+?)"

It works for me on assertions.

Solution 3

I am assuming that JMeter uses Java-based regular expressions... This could mean no named capturing groups. Apparently, Java7 now supports them, but that doesn't necessarily mean JMeter would. For JSON that looks like this:

{
"name":"@favorites",
"description":"Collection of my favorite places",
"list_id":4894636,
}

{
"name":"@AnotherThing",
"description":"Something to fill space",
"list_id":0048265,
}

{
"name":"@SomethingElse",
"description":"Something else as an example",
"list_id":9283641,
}

...this expression:

\{\s*"name":"((?:\\"|[^"])*)",\s*"description":"((?:\\"|[^"])*)",(?:\\}|[^}])*}

...should match 3 times, capturing the "name" value into the first capturing group, and the "description" into the second capturing group, similar to the following:

1                 2
---------------   ---------------------------------------
@favorites        Collection of my favorite places
@AnotherThing     Something to fill space
@SomethingElse    Something else as an example

Importantly, this expression supports quote escaping in the value portion (and really even in the identifier name portion as well, so that the Javascript string I said, "What is your name?"! will be stored in JSON as AND parsed correctly as I said, \"What is your name?\"!