How to write routine to escape double quotes in a JSON string

bash shell json

5,960

Solution 1

Presumably you only want to escape " unless it preceded by \.

echo ' bad \"  string"' | sed -E 's/([^\]|^)"/\1\\"/g'

Explanation

This will match ", but only if it's preceded by [^\], which is "any character except \" (or the start of the line ^). However, since this new character will be replaced itself, we need to capture it in a capturing group (), then replace it again with the match \1. In this example, I've used extended regular expressions with -E for simplicity.

Solution 2

$ echo ' bad \"  string"' | perl -pe 's/(?<!\\)"/\\\"/g'
 bad \"  string\"
$ echo ' bad \"  string" """""""""' | perl -pe 's/(?<!\\)"/\\\"/g'
 bad \"  string\" \"\"\"\"\"\"\"\"\"
$ echo ' bad \"  string" """"""""" \"' | perl -pe 's/(?<!\\)"/\\\"/g'
 bad \"  string\" \"\"\"\"\"\"\"\"\" \"

using negative lookbehind, you can achieve this. https://www.regular-expressions.info/lookaround.html

5,960

Alexander Mills

Updated on September 18, 2022

Comments

Alexander Mills over 1 year
I have to be efficient so I can't use tools like jq which loads up a big binary executable. I just want to escape double quotes in a string so it's safe for JSON. This isn't good enough:
```
echo ' bad \"  string"' | sed 's/"/\\"/g' 
```
because it will escape double quotes that are already escaped. Is there a way to replace double quotes only if they are not already escaped?
- Inian almost 5 years
  
  What environment is that you are in that considers a simpler executable jq to be bad? I would suggest using jq anyway for its myriad of functionalies
- Alexander Mills almost 5 years
  
  I only need one piece of functionality and it's for another library so I don't need to load a big executable
- Inian almost 5 years
  
  It doesn't make sense if you don't wanna use jq and use another third party tool instead
- Stéphane Chazelas almost 5 years
  
  How about the other characters that would also need to be encoded for json (like control characters)?
- Stéphane Chazelas almost 5 years
  
  How is sed any better than jq? On my system, sed is more than 5 times the size of jq. When you include the size of the libraries their are linked to, jq is slightly bigger, but that's mostly down to libm which will already be in memory anyway. jq takes 60% more time to start than GNU sed on my system though.
- Kusalananda almost 5 years
  
  Is your application restricted by the speed at which external utilities can be launched?
- roaima almost 5 years
  
  Or maybe it's an embedded system? Details would help steer the choice of answers.
Alexander Mills almost 5 years

upvoted, but yeah unfortunately it won't work in cases like this stackoverflow.com/questions/56471705/…, I need to add more logic to it
Alexander Mills almost 5 years

I need to escape unescaped escapes or whatever lol
Alexander Mills almost 5 years

There's probably a few edge cases, basically need to produce a string that can surrounded by double quotes and not have any double quotes in it that aren't escaped, but if something is already escaped cannot double escape it.
Sparhawk almost 5 years

@AlexanderMills The "however JSON works" is what confuses me. If you could give explicit examples in your question, that would help us answer you!