Unix command-line JSON parser?
Solution 1
You can use this command-line parser (which you could put into a bash alias if you like), using modules built into the Perl core:
perl -MData::Dumper -MJSON::PP=from_json -ne'print Dumper(from_json($_))'
Solution 2
I prefer python -m json.tool
which seems to be available per default on most *nix operating systems per default.
$ echo '{"foo":1, "bar":2}' | python -m json.tool
{
"bar": 2,
"foo": 1
}
Note: Depending on your version of python, all keys might get sorted alphabetically, which can or can not be a good thing. With python 2 it was the default to sort the keys, while in python 3.5+ they are no longer sorted automatically, but you have the option to sort by key explicitly:
$ echo '{"foo":1, "bar":2}' | python3 -m json.tool --sort-keys
{
"bar": 2,
"foo": 1
}
Solution 3
If you're looking for a portable C compiled tool:
https://stedolan.github.io/jq/
From the website:
jq is like sed for JSON data - you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text.
jq can mangle the data format that you have into the one that you want with very little effort, and the program to do so is often shorter and simpler than you’d expect.
Tutorial: https://stedolan.github.io/jq/tutorial/
Manual: https://stedolan.github.io/jq/manual/
Download: https://stedolan.github.io/jq/download/
Solution 4
I have created a module specifically designed for command-line JSON manipulation:
https://github.com/ddopson/underscore-cli
- FLEXIBLE - THE "swiss-army-knife" tool for processing JSON data - can be used as a simple pretty-printer, or as a full-powered Javascript command-line
- POWERFUL - Exposes the full power and functionality of underscore.js (plus underscore.string)
- SIMPLE - Makes it simple to write JS one-liners similar to using "perl -pe"
- CHAINED - Multiple command invokations can be chained together to create a data processing pipeline
- MULTI-FORMAT - Rich support for input / output formats - pretty-printing, strict JSON, etc [coming soon]
- DOCUMENTED - Excellent command-line documentation with multiple examples for every command
It allows you to do powerful things really easily:
cat earthporn.json | underscore select '.data .title'
# [ 'Fjaðrárgljúfur canyon, Iceland [OC] [683x1024]',
# 'New town, Edinburgh, Scotland [4320 x 3240]',
# 'Sunrise in Bryce Canyon, UT [1120x700] [OC]',
# ...
# 'Kariega Game Reserve, South Africa [3584x2688]',
# 'Valle de la Luna, Chile [OS] [1024x683]',
# 'Frosted trees after a snowstorm in Laax, Switzerland [OC] [1072x712]' ]
cat earthporn.json | underscore select '.data .title' | underscore count
# 25
underscore map --data '[1, 2, 3, 4]' 'value+1'
# prints: [ 2, 3, 4, 5 ]
underscore map --data '{"a": [1, 4], "b": [2, 8]}' '_.max(value)'
# [ 4, 8 ]
echo '{"foo":1, "bar":2}' | underscore map -q 'console.log("key = ", key)'
# key = foo
# key = bar
underscore pluck --data "[{name : 'moe', age : 40}, {name : 'larry', age : 50}, {name : 'curly', age : 60}]" name
# [ 'moe', 'larry', 'curly' ]
underscore keys --data '{name : "larry", age : 50}'
# [ 'name', 'age' ]
underscore reduce --data '[1, 2, 3, 4]' 'total+value'
# 10
And it has one of the best "smart-whitespace" JSON formatters available:
If you have any feature requests, comment on this post or add an issue in github. I'd be glad to prioritize features that are needed by members of the community.
Solution 5
Checkout TickTick.
It's a true Bash JSON parser.
#!/bin/bash
. /path/to/ticktick.sh
# File
DATA=`cat data.json`
# cURL
#DATA=`curl http://foobar3000.com/echo/request.json`
tickParse "$DATA"
echo ``pathname``
echo ``headers["user-agent"]``
Jé Queue
Updated on August 07, 2020Comments
-
Jé Queue over 3 years
Can anyone recommend a Unix (choose your flavor) JSON parser that could be used to introspect values from a JSON response in a pipeline?
-
Landon Kuhn about 12 yearsI am confused by the output of this. The output includes fat arrows (=>) between keys and values. This isn't JSON.
-
Jé Queue about 12 yearsGotta love shell-level tools :)
-
Ether about 12 years@landon: no, the input is JSON, and the output is a native Perl data structure, which you can then go on to further manipulate if needed. The point of this one-liner is it produces data that is much easier to read.
-
activout.se almost 12 yearsEasy to install, on Ubuntu: sudo apt-get install python-pip && sudo pip install jsonpipe
-
scorpiodawg over 11 yearsUnderrated answer. This is a nice command-line alternative if the goal is to validate a given JSON file as containing valid JSON.
-
Camilo Martin over 10 yearsAwesome! But, is it possible to run console commands on JSON data? For example: given a JSON file with an URL array,
wget
every URL. -
Dave Dopson over 10 years@CamiloMartin - the easiest way to do that is to print out the URLs, one URL per line, and then run that through xargs or GNU parallel.
-
FrozenCow over 10 yearsBest answer in here imo. No heavy dependencies, small, powerful, good documentation and a breeze to try it out. Thanks a lot for suggesting this!
-
gitaarik over 10 years@divideandconquer.se Sorry but you install this tool using npm with
npm install json
. -
user227666 about 10 yearsHow can I install
jkid
in mac? -
user227666 about 10 years@DaveDopson Can I use
underscore
for parsing nested json having nested objects and arrays? -
Dave Dopson about 10 years@user227666 - sure. JSON supports nesting many levels of objects. Or you might mean JSON that has a string which encodes further JSON. Which also works, but requires just a bit of munging.
-
Colin Su almost 10 yearsthis answer didn't describe how to inspect values of specified key.
-
muhqu almost 10 years@ColinSu but that was also not the original question.
json.tool
is just a short hand to pretty print json. If you need to extract/manipulate json data in a shell script, I would usejq
which is pure awesome at what is does... -
Colin Su almost 10 years@muhqu yeah, I know, I use
json.tool
ten times daily. I think I misread the meaning of "introspec" in the question, thank for your pointing out. -
Colin Su almost 10 yearsBTW, I use
jq
too, it's awesome! -
ekta over 9 years@DaveDopson Does underscore support "contains" a "pattern", ie. for a specific "key", the possible set of (case-insenstitive) values ? I tried "jq" with match, but it doesn't work. Also posted my complete use-case here - stackoverflow.com/questions/25463196/…
-
glerYbo over 9 yearsExample usage on OSX:
brew install jshon
,cat *.json | jshon
-
Brad over 9 years@rednaw Unfortunately, the NPM package
json
seems to be taken over by a completely different package now. -
Devy over 7 yearsIMMO this is an incorrect answer because python's
json.tool
only does two things: validate and pretty-print json. It does NOT introspect values in the json likejq
does. -
Pablo Bianchi over 6 yearsOn Ubuntu/Debian you can just
apt install jq
. -
Georgy Vladimirov over 5 yearsIf you want a JSON output, you can use this Perl one-liner:
perl -e "use JSON; print to_json( decode_json(<>), { pretty => 1 } )"
-
Jé Queue over 4 yearsI asked this many-a-moon ago, and have learned to love
jq
. -
Aditya Sriram over 3 yearsFor Python 3.5+, the keys are not sorted automatically but rather follow the same order as the input. One needs to use the
--sort-keys
option to have the keys sorted. -
muhqu over 3 years@AdityaSriram good to know! …will add this info to the answer.