Sanitize JSON for sending with Curl

bash curl json

12,412

Using the jo utility, the sanitized JSON document could be constructed using

data=$( jo Attribute="$USERINPUT" )

You would then use -d "$data" with curl to pass this to the end point.

Old (but still valid) answer using jq instead of jo:

Using jq:

USERINPUT=$'a e""R<*&\04\n\thello!\''

This string has a couple of double quotes, an EOT character, a newline, a tab and a single quote, along with some ordinary text.

data="$( jq --null-input --compact-output --arg str "$USERINPUT" '{"Attribute": $str}' )"

This builds a JSON object containing the user data as the value for the lone Attribute field.

The same thing using short options:

data="$( jq -nc --arg str "$USERINPUT" '{"Attribute": $str}' )"

From this we get

{"Attribute":"ae\"\"R<*&\u0004\n\thello!'"}

as the value in $data.

This can now be used in your call to curl:

RESP="$( curl --connect-timeout "10" -s \
           -H "Content-Type: application/json" \
           -X POST -d "$data" \
           "$ENDPOINT" | jq -r '.key' )"

12,412

Admin

Updated on September 18, 2022

Comments

Admin over 1 year
I have to send a POST request to some service with JSON payload, and it includes some user input. That input variable needs to be JSON-encoded to prevent injection attacks.

Code example that sends requests and parses response JSON to RESP variable:
```
RESP=`curl --connect-timeout "10" -s -H "Content-Type: application/json" \
        -X POST -d '{ "Attribute": '"'$USERINPUT'" }',\
        $ENDPOINT | $JQ -r '.key'`
```
How to sanitize, or JSON encode, $USERINPUT before creating JSON payload?
Admin almost 5 years

not gonna lie, this is pretty fing gnar gnar..there might not be a better way with jq, but all we want to do is sanitize a JSON field individually, ideally we just want some thing like echo "$gnar_str" | jq --sanitize
Alexander Mills almost 5 years

See my request here: github.com/stedolan/jq/issues/1918
Kusalananda almost 5 years

@MrCholo What's wrong with jq -nc --arg str "$gnar_str" '$str'? BTW, if you use echo on unsanitised variable data, you are already in trouble.
Admin almost 5 years

just look at this gist: gist.github.com/ORESoftware/bc74a07153193d7926ceda12cc4d0c6f
Kusalananda almost 5 years

@MrCholo Um... The data is extracted with jq -r, which returns the raw data, not the encoded data, so it's not strange that they get the same thing back. Also, they needlessly create a key-value pair, and then extract its value. See my previous comment above.
Admin almost 5 years

yes I think it needs to be echo '$gnar_str' | jq -R not echo '$gnar_str' | jq -r
Admin almost 5 years

there is nothing wrong with jq -nc --arg str "$gnar_str" '$str' , I didn't know that was possible, your answer would be ideal if it showed that too, since that is the simpler case.
Kusalananda almost 5 years

@MrCholo It has very little to do with the question though, as that's not what the user is wanting to do.
Admin almost 5 years

I disagree, it's exactly what the user wants to do - instead of your solution -X POST -d "$data" it should just be -X POST -d "{"Attribute":"$data"}, where $data is cleaned up with the simpler cleanup I used. I mean normally youre 150% spot on, but I have to disagree here.
Admin almost 5 years

I added an answer according to what I said above, I did not test it though: unix.stackexchange.com/a/523170/287203