Loop through json using jq to get multiple value

Question

Here is volumes.json :

{
"Volumes": [
    {
        "AvailabilityZone": "us-east-1a",
        "Tags": [
            {
                "Value": "vol-rescue-system",
                "Key": "Name"
            }
        ],
        "VolumeId": "vol-00112233",
    },
    {
        "AvailabilityZone": "us-east-1a",
        "Tags": [
            {
                "Value": "vol-rescue-swap",
                "Key": "Name"
            }
        ],
        "VolumeId": "vol-00112234",
    },
    {
        "AvailabilityZone": "us-east-1a",
        "Tags": [
            {
                "Value": "vol-rescue-storage",
                "Key": "Name"
            }
        ],
        "VolumeId": "vol-00112235",
    }
]
}

I need to get both the value of VolumeId and Tags.Value to be used as the input to invoke another command. It is easy to get a single value from the json array, but I am not able to extract multiple value from it and pass it to another bash command.

I can get a single value using this:

cat volumes.json |jq -r '.Volumes[].VolumeId' |while read v; do another_bash_command $v; done

but I am not able to get multiple value cause this is wrong:

 cat volumes.json |jq -r '.Volumes[].VolumeId, .Volumes[].Tags[].Value' |while read v w; do another_bash_command $v $w; done

as it will then loop 6 times of the outcome instead of 3.

And, how do I pass those multiple json value in the loop to a bash array so I can use the value in a better way ? Like VolumeId-> $arr[0][0], Tags.Value-> $arr[0][1], AvailabilityZone-> $arr[0][2]...etc. I have searched through SO and the jq docs, and tried readarray, but still not able to find out the solution :( Thanks for any help given.

Solution 1

It seems to me that you want to output the two values (VolumeId and Tags[].Value) on the same line?

If that's the case, then a simple string concatenation should be enough:

$ jq -r '.Volumes[] | .VolumeId + " " + .Tags[].Value' volumes.json
vol-00112233 vol-rescue-system
vol-00112234 vol-rescue-swap
vol-00112235 vol-rescue-storage

The above can then be used in a pipeline with while-read:

$ cat my_script
jq -r '.Volumes[] | .VolumeId + " " + .Tags[].Value' volumes.json \
| while IFS= read -r volumeId tagValue; do
  other_command "$volumeId" "$tagValue"
done

You should note that if there is more than one element in Tags the result will reflect that. This can however be avoided by referring the first element in Tags: .Tags[0].Value

Solution 2

As @andlrc observed, you may need to decide what you really want in the event that any Tags array has more or less than one element. Assuming you want Tags[0] in all cases, I would recommend considering the use of @tsv as follows:

jq -r '.Volumes[] | [.VolumeId, .Tags[0].Value] | @tsv' volumes.json

This would be especially appropriate if any of the .VolumeId or .Tags[0].Value values contained spaces, tabs, newlines, etc. The point is that @tsv will handle these in a standard way, so that handling the pair of values can be done in a standard way as well. E.g. using awk, you could read in the pair with awk -F\\t; using bash, IFS=$'\t', etc.

Solution 3

I know the question is about how to get the information using jq but its also possible to directly get the expected parameter directly from aws cli using the --query flag.

aws ec2 describe-volumes --query "Volumes[].[VolumeId, Tags[].Value]" --output text

Comments

[Oliver]

jamespath (used by --query) is awsome for that, pitty there are things you can't do with it that you can do with jq (see eg news.ycombinator.com/item?id=16400320, comment by justin_oaks on Feb 17, 2018).