How can I filter out a specific part of a JSON response in ash shell

Question

I'm using wget with -qO to a specific updater URL for OpenELEC which provides the following JSON response, similar to the one below.

{"data":{"update":"OpenELEC-RPi2.arm-5.0.8.tar","folder":"releases","host":"","MD5":""}}

I only need the full filename of the tar and nothing else.

What is the best way of filtering out all the other non-needed information?

Solution 1

jq would do:

$ jq ".data.update" <<< '{"data":{"update":"OpenELEC-RPi2.arm-5.0.8.tar","folder":"releases","host":"","MD5":""}}'
"OpenELEC-RPi2.arm-5.0.8.tar"

or with -r:

jq -r ".data.update" <<< '{"data":{"update":"OpenELEC-RPi2.arm-5.0.8.tar","folder":"releases","host":"","MD5":""}}'
OpenELEC-RPi2.arm-5.0.8.tar

to get the (raw) string without quotes.

Solution 2

Using sed, pipe the output of wget to

sed -n 's/^.*{"update":"\([^"]*\)".*$/\1/p'

This matches the {"update": part of the returned string and then extracts the contents in double quotes following the matched part.

Solution 3

I would have to say - use perl. I'm not 100% sure if that's installed on your OS, but there's quite a good chance it is - it's a pretty common base install component.

#!/usr/bin/env perl

use strict;
use warnings;

use JSON;
print decode_json ( <DATA> ) -> {data} -> {update}; 

__DATA__
{"data":{"update":"OpenELEC-RPi2.arm-5.0.8.tar","folder":"releases","host":"","MD5":""}}

This would be a 'one liner' by:

perl -MJSON -0ne 'print decode_json($_)->{data}->{update}' jsonfile

The advantage of doing it this way is that you're using a JSON parser, so it'll work if your json file contains:

{"data":{"update":"OpenELEC-RPi2.arm-5.0.8.tar","folder":"releases","host":"","MD5":""}}

or

{
   "data" : {
      "host" : "",
      "MD5" : "",
      "update" : "OpenELEC-RPi2.arm-5.0.8.tar",
      "folder" : "releases"
   }
}

Or some other perfectly valid reformatting of JSON. This is why using a parser is really the ideal way of doing this, because JSON officially doesn't care where your linefeeds go, and allows nesting - and that tends to make regular expressions and line based parsing brittle.

Solution 4

Assuming that you store the JSON in a file called jsonfile, then:

awk -F "update" '{print $2}' jsonfile | awk -F ":" '{print $2}' | awk -F "," '{ gsub(/"/,"" ); print $1}'

Will give as result OpenELEC-RPi2.arm-5.0.8.tar.

Keep in mind that it's not an JSON parser, so this will work if you have always the same structure as the example.

Solution 5

As FloHimself said, you should really use a real JSON parser. There are many edge cases that are not easily covered by a simple text processor.

With that caveat:

sed 's/.*"update":"\([^"]\+\)".*/\1/'

View more solutions

Comments

[rosi97]

Because OpenELEC is JeOS, it doesn't have jq and I the root file system is read only, so its difficult to add additional packages.

[FloHimself]

Well, then jq isn't really a option here...

[rosi97]

Unfortunately not. But useful for other projects.

[rosi97]

I agree, jq would be the normal usage case, but the setup of OpenELEC isn't extensible in a usual fashion as other Linux/Unix distros.

[rosi97]

Indeed. Its not ideal using tools not geared towards JSON parsing, but I am limited with the whole JeOS setup, but the JSON response format shouldn't change.

[rosi97]

I'm marking this as the answer as its the correct way to work with JSON data as sed and such has limitations. I was able to compile jq 1.4 with the OpenELEC build system as a static binary.

[rosi97]

Another note OpenELEC doesn't like <<< as its BusyBox, not full bash, instead I echo the JSON response and then piped it to jq to grab the part I wanted.

[rosi97]

Absolutely. Just pointing out from the perspective of OpenELEC that the <<< substitution would fail. If the JSON was in a file you could also use jq ".data.update" < jsonfile.json which is pretty clean.

[rosi97]

For OpenELEC users: I submitted a PR to the unofficial addons repo and jq is now available as a console only addon github.com/OpenELEC/unofficial-addons/tree/master/addons/too‌​ls/…