How to parse JSON with shell scripting on Linux?

Question

I have a Json output from which I need to extract a few parameters in linux.

This is the json output:

{
  "OwnerId":"121456789127",
  "ReservationId":"r-48465168",
  "Groups":[

  ],
  "Instances":[
    {
      "Monitoring":{
        "State":"disabled"
      },
      "PublicDnsName":null,
      "RootDeviceType":"ebs",
      "State":{
        "Code":16,
        "Name":"running"
      },
      "EbsOptimized":false,
      "LaunchTime":"2014-03-19T09:16:56.000Z",
      "PrivateIpAddress":"10.250.171.248",
      "ProductCodes":[
        {
          "ProductCodeId":"aacglxeowvn5hy8sznltowyqe",
          "ProductCodeType":"marketplace"
        }
      ],
      "VpcId":"vpc-86bab0e4",
      "StateTransitionReason":null,
      "InstanceId":"i-1234576",
      "ImageId":"ami-b7f6c5de",
      "PrivateDnsName":"ip-10-120-134-248.ec2.internal",
      "KeyName":"Test_Virginia",
      "SecurityGroups":[
        {
          "GroupName":"Test",
          "GroupId":"sg-12345b"
        }
      ],
      "ClientToken":"VYeFw1395220615808",
      "SubnetId":"subnet-12345314",
      "InstanceType":"t1.micro",
      "NetworkInterfaces":[
        {
          "Status":"in-use",
          "SourceDestCheck":true,
          "VpcId":"vpc-123456e4",
          "Description":"Primary network interface",
          "NetworkInterfaceId":"eni-3619f31d",
          "PrivateIpAddresses":[
            {
              "Primary":true,
              "PrivateIpAddress":"10.120.134.248"
            }
          ],
          "Attachment":{
            "Status":"attached",
            "DeviceIndex":0,
            "DeleteOnTermination":true,
            "AttachmentId":"eni-attach-9210dee8",
            "AttachTime":"2014-03-19T09:16:56.000Z"
          },
          "Groups":[
            {
              "GroupName":"Test",
              "GroupId":"sg-123456cb"
            }
          ],
          "SubnetId":"subnet-31236514",
          "OwnerId":"109030037527",
          "PrivateIpAddress":"10.120.134.248"
        }
      ],
      "SourceDestCheck":true,
      "Placement":{
        "Tenancy":"default",
        "GroupName":null,
        "AvailabilityZone":"us-east-1c"
      },
      "Hypervisor":"xen",
      "BlockDeviceMappings":[
        {
          "DeviceName":"/dev/sda",
          "Ebs":{
            "Status":"attached",
            "DeleteOnTermination":false,
            "VolumeId":"vol-37ff097b",
            "AttachTime":"2014-03-19T09:17:00.000Z"
          }
        }
      ],
      "Architecture":"x86_64",
      "KernelId":"aki-88aa75e1",
      "RootDeviceName":"/dev/sda1",
      "VirtualizationType":"paravirtual",
      "Tags":[
        {
          "Value":"Server for testing RDS feature in us-east-1c AZ",
          "Key":"Description"
        },
        {
          "Value":"RDS_Machine (us-east-1c)",
          "Key":"Name"
        },
          {
          "Value":"1234",
          "Key":"Cost.centre"
        },
        {
          "Value":"Jyoti Bhanot",
          "Key":"Owner"
        }
      ],
      "AmiLaunchIndex":0
    }
  ]
}

Expected output :

 Instance id         Name                           cost centre             Owner
    i-1234576          RDS_Machine (us-east-1c)        1234                   Jyoti Bhanot

I want to write a file that contains headings like instance id, tag like name, cost centre, owner. Below that, certain values from the json output. The output here given is just a example.

How can I do that using sed and awk?

Any lead is appreciated.

Thanks

Solution

Here's a sample using jsawk. Reference: Parsing JSON with Unix tools

Setup:

First download jsawk from https://github.com/micha/jsawk:

$ curl -L http://github.com/micha/jsawk/raw/master/jsawk > jsawk
$ chmod 755 jsawk && mv jsawk ~/bin/

You might want to install js-devel first before you can use jsawk. I'm using Fedora, so what I did was:

$ sudo yum install js-devel

The test:

I copied your JSON output sample to a text file. Called it sample.json. Here a sample to get a value from your JSON output sample:

$ jsawk 'return this.Instances[0].Monitoring.State' < sample.json
disabled
$ jsawk 'return this.Instances[0].VpcId' < sample.json
vpc-86bab0e4

For JSON data from a URL, you can use curl http://someserver.com/data.json instead of cat:

$ curl http://someserver.com/data.json | jsawk 'return this.Instances[0].VpcId'
vpc-86bab0e4

You can use these commands in your bash file to generate a new file that contains strings / text that you wanted. You can read more about jsawk from the GitHub link that I provided here.

Is this what you were looking for?

Comments

[Muhammad Baja Aksha]

Looks like this is a duplicate from: stackoverflow.com/questions/22316714/… and stackoverflow.com/questions/1955505/…

[Muhammad Baja Aksha]

as Rahul R Dhobi said, you might want to see this: stackoverflow.com/questions/1955505/… for your specific question. jsawk might be your answer.

[Muhammad Baja Aksha]

please see my edited answer. I won't be able to write you a shell script to generate the file that you asked, but you should get how to write it yourself from here. let me know if it works.

[icedwater]

You could write a script that calls this one for each of the 100s of servers.

[Muhammad Baja Aksha]

Refer to this: A quick guide to writing scripts using the bash shell on how to write a shell script. Try it first on your own, then ask or search again for help.