Convert JSON object with duplicate keys to JSON array

Question

I have a JSON string that I get from a database which contains repeated keys. I want to remove the repeated keys by combining their values into an array.

For example

Input

{
"a":"b",
"c":"d",
"c":"e",
"f":"g"
}

Output

{
"a":"b",
"c":["d","e"],
"f":"g"
}

The actual data is a large file that may be nested. I will not know ahead of time what or how many pairs there are.

I need to use Java for this. org.json throws an exception because of the repeated keys, gson can parse the string but each repeated key overwrites the last one. I need to keep all the data.

If possible, I'd like to do this without editing any library code

Solution 1

As of today the org.json library version 20170516 provides accumulate() method that stores the duplicate key entries into JSONArray

JSONObject jsonObject = new JSONObject();
jsonObject.accumulate("a", "b");
jsonObject.accumulate("c", "d");
jsonObject.accumulate("c", "e");
jsonObject.accumulate("f", "g");
System.out.println(jsonObject);

Output:

{  
    "a":"b",  
    "c":["d","e"],  
    "f":"g"  
}

Solution 2

I want to remove the repeated keys by combining their values into an array.

Think other than JSON parsing library. It's very simple Java Program using String.split() method that convert Json String into Map<String, List<String>> without using any library.

Sample code:

String jsonString = ...
// remove enclosing braces and double quotes
jsonString = jsonString.substring(2, jsonString.length() - 2);

Map<String, List<String>> map = new HashMap<String, List<String>>();
for (String values : jsonString.split("\",\"")) {
    String[] keyValue = values.split("\":\"");
    String key = keyValue[0];
    String value = keyValue[1];

    if (!map.containsKey(key)) {
        map.put(key, new ArrayList<String>());
    }
    map.get(key).add(value);
}

output:

{
  "f": ["g"],
  "c": ["d","e"],
  "a": ["b"]
}

Solution 3

In order to accomplish what you want, you need to create some sort of custom class since JSON cannot technically have 2 values at one key. Below is an example:

public class SomeClass {

Map<String, List<Object>> values = new HashMap<String, List<Object>>();

public void add(String key, Object o) {
    List<Object> value = new ArrayList<Object>();
    if (values.containsKey(key)) {
        value = values.get(key);
    }
    value.add(o);
    values.put(key, value);
}

public JSONObject toJson() throws JSONException {
    JSONObject json = new JSONObject();
    JSONArray tempArray = null;

    for (Entry<String, List<Object>> en : values.entrySet()) {
        tempArray = new JSONArray();
        for (Object o : en.getValue()) {
            tempArray.add(o);
        }
        json.put(en.getKey(), tempArray);
    }

    return json;
}
}

You can then retrieve the values from the database, call the .add(String key, Object o) function with the column name from the database, and the value (as the Object param). Then call .toJson() when you are finished.

Solution 4

Thanks to Mike Elofson and Braj for helping me in the right direction. I only wanted to have the keys with multiple values become arrays so I had to modify the code a bit. Eventually I want it to work for nested JSON as well, as it currently assumes it is flat. However, the following code works for what I need it for at the moment.

public static String repeatedKeysToArrays(String jsonIn) throws JSONException 
{
    //This assumes that the json is flat
    String jsonString = jsonIn.substring(2, jsonIn.length() - 2);

    JSONObject obj = new JSONObject();
    for (String values : jsonString.split("\",\"")) {
        String[] keyValue = values.split("\":\"");
        String key = keyValue[0];
        String value = ""; 
        if (keyValue.length>1) value = keyValue[1];

        if (!obj.has(key)) {
            obj.put(key, value);
        } else {
            Object Oold = obj.get(key);
            ArrayList<String> newlist = new ArrayList<String>();

            //Try to cast as JSONArray. Otherwise, assume it is a String
            if (Oold.getClass().equals(JSONArray.class)) {
                JSONArray old = (JSONArray)Oold;
                //Build replacement value
                for (int i=0; i<old.length(); i++) {
                    newlist.add( old.getString(i) );
                }
            }
            else if (Oold.getClass().equals(String.class)) newlist = new ArrayList<String>(Arrays.asList(new String[] {(String)Oold})); 
            newlist.add(value);

            JSONArray newarr = new JSONArray( newlist );
            obj.put(key,newarr);                
        }
    }
    return obj.toString();
}

View more solutions

Comments

[adamF]

This is much too hard coded to work. The example I gave was just to illustrate the problem. My actual data is a large nested file and I don't know ahead of time what or how many fields there are. Even if I abstracted some of it away, I'd still need a way to parse through the json string reliably.

[Braj]

Make it generic. There can be any type of values in JSON.

[Mike Elofson]

Make it generic using a map. Read in the key-value pairs one at a time, check if the key exists in a map, if it does, add the value to a list at that key, if it doesn't, create a new list with that value and add it at that key. Then, in toJSON, iterate through each Entry

[Mike Elofson]

I have updated my answer showing how to make it generic using a map.

[adamF]

awesome! Don't need it anymore, but just what I was looking for long ago :)

[Kainix]

Thought so, because the question was 3yrs old, but hey it can be useful to others :)