how to parse a JSON String with jq (or other alternatives)?

Question

I'm trying to get jq to parse a JSON structure like:

{
  "a" : 1,
  "b" : 2,
  "c" : "{\"id\":\"9ee ...\",\"parent\":\"abc...\"}\n"
}

That is, an element in the JSON is a string with escaped json.

So, I have something along the lines of $ jq [.c] myFile.json | jq [.id]

But that crashes with jq: error: Cannot index string with string

This is because the output of .c is a string, not more JSON. How do I get jq to parse this string?

My initial solution is to use sed to replace all the escape chars (\":\", \",\" and \") but that's messy, I assume there's a way built into jq to do this?

Thanks!

edit: Also, the jq version available here is:

$ jq --version
jq version 1.3

I guess I could update it if required.

Solution 1

jq has the fromjson builtin for this:

jq '.c | fromjson | .id' myFile.json

fromjson was added in version 1.4.

Solution 2

You can use the raw output (-r) that will unescape characters:

jq -r .c myfile.json | jq .id

ADDENDUM: This has the advantage that it works in jq 1.3 and up; indeed, it should work in every version of jq that has the -r option.

Solution 3

Motivation: you want to parse JSON string - you want to escape a JSON object that's wrapped with quotes and represented as a String buffer, and convert it to a valid JSON object. For example:

some JSON unescaped string :

"{\"name\":\"John Doe\",\"position\":\"developer\"}"

the expected result ( a JSON object ):

{"name":"John Doe","position":"developer"}

Solution: In order to escape a JSON string and convert it into a valid JSON object use the sed tool in command line and use regex expressions to remove/replace specific characters:

cat current_json.txt | sed -e 's/\\\"/\"/g' -e 's/^.//g' -e 's/.$//g'

s/\\\"/\"/g replacing all backslashes and quotes ( \" ) into quotes only (")

s/^.//g replacing the first character in the stream to none character

s/.$//g replacing the last character in the stream to none character

Comments

[k0pernikus]

This question also helps if you are looking for: "How to unescape json string using jq?"

[Colin Grogan]

Thank you. This works. I'll accept this answer, as it's more 'idiomaitic' I feel. Cheers.

[vbence]

@ColinGrogan please do.

[Casimir et Hippolyte]

@ColinGrogan: I don't see any reason to change the accepted answer since you clearly wrote in your question that you used the version 1.3 of jq in which the fromjson feature isn't available. In other words, even if this answer is interesting, it doesn't answer the question.

[Itération 122442]

Is it possible to use this but on an entire json file (not specifying the .id property)?

[Admin]

@FlorianCastelain yes, either omit it or use dot: jq 'fromjson | .' myfile, where myfile contains "{\"key\":1, \"word\":\"cat\"}"

[Noah Gary]

-r is much easier than the accepted answer. You got my upvote. I use jq to view json from my clipboard. So when I run into this I do: pbpaste | jq -r

[Noah Gary]

bonus: if you want to escape quotes you can use sed: pbpaste | sed -e 's/\\\"/\"/g'. this helps when you have escaped json that contains escaped json. Take out the first level of escaped quotes with sed and then jq can remove the last level. e.g pbpaste | sed -e 's/\\\"/\"/g' | jq -r

[Noah Gary]

I had a similar problem. Used the same sed command verbatum. Bravo! Pipe that result to jq -r and you can process escaped json inside your json.

[Charles Duffy]

Boo, hiss re: trying to parse JSON with sed. Any tool that tries to match just a few of the most common escapings is going to miss others (for example, you're handling " but not \t)