Python - Converting a string with escape characters to json
Solution 1
if \"
can occur in your string you have to escape \
and "
with
import json
resp = '{"from_hostname": {"value": "mysite.edu\\\"", "value2": 0, "value3": 1}}'
print(json.loads(resp))
it prints
{u'from_hostname': {u'value3': 1, u'value2': 0, u'value': u'mysite.edu"'}}
is this the right interpration of your question?
Solution 2
The problem is that the backslash character is escaping the double quote in the Python string, but it is not actually present in the string. Printing the string demonstrates this:
>>> print '{"from_hostname": {"value": "mysite.edu\"", "value2": 0, "value3": 1}}'
'{"from_hostname": {"value": "mysite.edu"", "value2": 0, "value3": 1}}'
This shows that the backslash is not in the string. So the double quote must be escaped for the string to be a valid JSON string, which means that the backslash must be present in the string. You can do that by escaping the backslash itself with another backslash, i.e. \\
:
>>> print '{"from_hostname": {"value": "mysite.edu\\"", "value2": 0, "value3": 1}}'
{"from_hostname": {"value": "mysite.edu\"", "value2": 0, "value3": 1}}
and json.loads()
now works:
>>> json.loads('{"from_hostname": {"value": "mysite.edu\\"", "value2": 0, "value3": 1}}')
{u'from_hostname': {u'value3': 1, u'value2': 0, u'value': u'mysite.edu"'}}
Or you could use a raw string:
>>> json.loads(r'{"from_hostname": {"value": "mysite.edu\"", "value2": 0, "value3": 1}}')
{u'from_hostname': {u'value3': 1, u'value2': 0, u'value': u'mysite.edu"'}}
However, json.loads()
fails on JSON strings that you extracted from the log file which strongly suggests that the problem is there. You should post the extraction code in your question so that can be checked.
gixxer
Updated on June 11, 2022Comments
-
gixxer almost 2 years
JSON objects are printed into my syslog file. I need to extract the string from the log and convert it into JSON. I don't have any problems extracting the string between '
{
' and '}
', but certain strings have an escape character in them, and this is causingjson.loads
to failHere is the problem:
>>> import json >>> resp = '{"from_hostname": {"value": "mysite.edu\"", "value2": 0, "value3": 1}}' >>> json.loads(resp) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads return _default_decoder.decode(s) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 365, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 381, in raw_decode obj, end = self.scan_once(s, idx) ValueError: Expecting , delimiter: line 1 column 41 (char 40) >>> resp[40] '"' >>> resp[41] ',' >>> resp[39] '"' >>>
When
json
sees the\"
I guess it only sees"
and assumes that the string is over, and it throws the delimiter error.I tried replacing
\"
with\\"
but that doesn't seem to work.NOTE: The
\"
can occur at the beginning or end or in the middle of the string.How do I get this working?
-
gixxer over 8 yearshi, yes, however I had problem appending extra escape characters to the string. What ever the number of escape characters I add, I still didn't see the \ appended in the string.
-
gixxer over 8 yearsyes, this was exactly what I started doing, and I was stuck at adding the extra \ to the string. I tried str.replace('\"', '\\"'), and some other variants but couldn't add the additional \.. Can you please help, i know this is basics but for some reason couldn't get past this.
-
mhawke over 8 yearsCan you post the code that extracts the JSON strings from the log file and a sample of the file? If the file contains valid JSON, you shouldn't need to perform any substitutions.