editing a JSON file with sed

Question

I need to edit a JSON file, using sed, to add some data in to the file. The JSON is as follows:

{
  'name':
  // more attributes that are already filled in 
}

I have written this sed command to try and do it:

sed "s/(\'name\':)/\1\"hello\"\,/g" /path/to/file.json

However, I keep getting this error:

sed: \1 not defined in the RE

The expected results are:

{
  'name': "hello",
  // more attributes here, left untouched
}

I know this is a bad way of doing it, but I don't think I am able to use tools such as jq because the file will be edited on a server and I cannot install jq on the server. If anyone has a better solution i'd be very interested to hear it. Thank you!

Solution 1

Use python, it's so much easier and on most POSIX systems:

#!/usr/bin/python
import json
jsonObject = json.load(filename)
jsonObject['name'] = "hello"
json.dump(filename, jsonObject)

And run it:

python parse_json.py

EDIT: To use this in a bash setting, use a here document:

#!/bin/sh

# ...

# set the name variable to something
name=VALUE

# ...

# and use it in a here document
parse_json_script=$(mktemp parse_json.XXXX.py)
cat > $parse_json_script << SCRIPT
#!/usr/bin/env python
import json
jsonObject = json.load(filename)
# You can use bash variables in here documents.
# the line below uses the bash variable `name` and inserts it into the python script
jsonObject['name'] = "$name"
print(jsonObject['name'])
json.dump(filename, jsonObject)
SCRIPT
python $parse_json_script && rm $parse_json_script

The foregoing script uses a here document to create a python script on the fly and run it to alter the JSON, then removes the parse json script afterwards.

Personally, though, I prefer to use the sh module of python and just write the script in python:

#!/usr/bin/python
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals 
import sh

# now you can call any command as if it were a python function
sh.ls("-lah") # `ls -lah`, but maybe you use python's `os` module instead
sh.rsync("-avHAX","foo","bar")

Between python's sh (command execution), os (file manipulation), shutil (recursive file manipulation) and re (regex) modules, I find bash's functionality may be generally completely replaced.

Solution 2

As you stated, sed is not the right tool here, instead, use a proper JSON parser :

INPUT json

$ cat json
{
  "name": "foobar"
}

using jq :

$ jq '.name |= "qux"' json | tee json 

(the latest with tee work with small files)

or using perl :

perl -i -MJSON -0ne '
    my $DS = decode_json $_;
    $DS->{name} = "qux";
    print encode_json $DS
' json

remove the -i switch if you want to test the command without editing the file in place

OUTPUT json

$ cat json
{
  "name": "qux"
}

Solution 3

You have the backslashes in the wrong places. ' does not need to be escaped, but the parentheses do. This works:

sed "s/\('name':\)/\1\"hello\"\,/g" /path/to/file.json

Note: I'm assuming that file.json is not a json file but a specially crafted template, otherwise this substitution wouldn't make any sense (the result would not be JSON). Working on JSON files with sed is quite certainly a bad idea, so if you're trying to modify JSON rather than generating it, take a look at jq or so instead.

Solution 4

sed "s/'name':/& \"hello\",/" File

Almost similar to Wintermutes's answer, but here we are avoiding the (grouping) and therefore instead of \1 we use & (pattern matched). Also, global substitution (g) is not required if only one occurence of 'name': in each line. I'm not sure if sed is the right tool for the job though. I do not know JSON yet. :-)

View more solutions