bash JSON with jq - how to check for empty JSON members?

10,537

Solution 1

Let's go over this from the top so you understand what's happening.

~$ err=$(jq ".errMsg" $input)
~$ echo $err
""

This output indicates that err contains "" (it may also have some spaces before and after the ""). This is not the same thing as an empty string. It's literally two double-quote characters.

~$ if [ $err = "" ]; then echo EMPTY; else echo CONTENT; fi 
CONTENT

I excpected the EMPTY as $err just showed "".

As mentioned earlier, $err is not empty, it contains two double-quotes. On the other hand, in the Bash statement [ ... = "" ] the "" does really mean an empty value.

The Bash statement [ $err = "" ] is problematic, because it will result in a syntax error if the value of $err is empty, or if it contains whitespace (assuming you haven't customized IFS).

The correct way to write this statement to enclose $err in double-quotes:

if [ "$err" = "" ]; then echo EMPTY; else echo CONTENT; fi

The same goes for the $input variable in your first command, it should be written as:

err=$(jq ".errMsg" "$input")

If you want to check for empty .errMsg value in the object, you have at least two options.

Option 1: check that the value is "".

if [ "$err" = '""' ]; then echo EMPTY; else echo CONTENT; fi

Option 2: make jq output in raw mode, so it doesn't include the enclosing double-quotes around string values.

err=$(jq -r ".errMsg" "$input")
if [ "$err" = "" ]; then echo EMPTY; else echo CONTENT; fi

I recommend this latter option, so that the value of err will be the real value. The enclosing double-quotes seem just noise for your purpose.

Solution 2

You can read your entire structure into a native bash associative array as follows:

declare -A data=( )
while IFS=$'\t' read -r key value; do
  data[$key]=$value
done < <(jq -r 'to_entries[] | [.key, .value] | @tsv' "$input")

From there, you have several options: You can check items individually...

[[ ${data[errMsg]} ]] || echo "ERROR: errMsg is not set!"

...or you can loop over all of them:

for key in "${!data[@]}"; do
    [[ ${data[$key]} ]] || echo "ERROR: ${key} is not set!"
done
Share:
10,537
onouv
Author by

onouv

Updated on June 09, 2022

Comments

  • onouv
    onouv over 1 year

    I want to detect an error depending on an error message JSON field being empty or non-empty . I am using jq to slice out the member, but then have massive trouble evaluating it in my bash "if".

    I have a json file referenced by my variable $input which does have an errMsg member, but as an empty string (note that my CLI prompt is ~$) :

    ~$ echo $(jq "." $input)
    { "projectCode": "145", "fullCode": "1", "errMsg": "" }
    

    retrieving the errMsg field in this case properly yields an empty string:

    ~$ err=$(jq ".errMsg" $input)
    ~$ echo $err
    ""
    

    Problem : testing that variable seems impossible :

    ~$ if [ $err = "" ]; then echo EMPTY; else echo CONTENT; fi 
    CONTENT
    

    I excpected the EMPTY as $err just showed "". Checking other way around:

    ~$ if [ $err != "" ]; then echo EMPTY; else echo CONTENT; fi
    EMPTY 
    

    so $err is apparently not "", even though echo $err yields "" ? How can that be ? What bash feature am I missing here ?

    Anyway, lets try that with a non-empty json (after changing the $input content):

    ~$ echo $(jq "." $input)
    { "projectCode": "145", "fullCode": "1", "errMsg": "not empty" }
    ~$ echo $err
    "not empty"
    
    ~$ if [ $err != "" ]; then echo EMPTY; else echo CONTENT; fi
    bash: syntax error near unexpected token `then'
    

    okay, then masking like this :

    ~$ if [ "$err" != "" ]; then echo EMPTY; else echo CONTENT; fi
    bash: syntax error near unexpected token `then'
    

    I guess it finds a second token after "not" and doesnt like it. After much research i find this truly unreadable solution that seems to work:

    ~$ val=$(sed "s/^\(\"\)\(.*\)\1\$/\2/g" <<<"$err" | sed -e 's/\s.*$//')
    ~$ if [ "$val" = "" ]; then echo "NO ERROR"; else echo "ERROR"; fi
    NO ERROR
    

    it also works for non-empty errMsg members. Please, there MUST be a simple and elegant solution to this.

    • Charles Duffy
      Charles Duffy about 6 years
      [ $err = "" ] isn't safe bash code; nothing to do with jq. If err is empty, then it runs [ = "" ], which isn't valid syntax. You need [ "$err" = "" ].
    • Charles Duffy
      Charles Duffy about 6 years
      As for if [ "$err" != "" ]; then echo EMPTY; else echo CONTENT; fi -- it really, truly is not a syntax error (though the != means there's a logic error). See it running correctly at ideone.com/CblHNK
    • Charles Duffy
      Charles Duffy about 6 years
      ...if you could provide a minimal reproducible example in the form of code other people could copy-and-paste to reproduce that particular bug (ideally, a reproducer tested to work on ideone or another unbiased 3rd-party sandbox), I'd be very interested to see it.
  • onouv
    onouv about 6 years
    thanks, that jq -r is just what was needed. Short, concise, works. No more sed needed.