How do I serialize a Python dictionary into a string, and then back to a dictionary?
Solution 1
It depends on what you're wanting to use it for. If you're just trying to save it, you should use pickle
(or, if you’re using CPython 2.x, cPickle
, which is faster).
>>> import pickle
>>> pickle.dumps({'foo': 'bar'})
b'\x80\x03}q\x00X\x03\x00\x00\x00fooq\x01X\x03\x00\x00\x00barq\x02s.'
>>> pickle.loads(_)
{'foo': 'bar'}
If you want it to be readable, you could use json
:
>>> import json
>>> json.dumps({'foo': 'bar'})
'{"foo": "bar"}'
>>> json.loads(_)
{'foo': 'bar'}
json
is, however, very limited in what it will support, while pickle
can be used for arbitrary objects (if it doesn't work automatically, the class can define __getstate__
to specify precisely how it should be pickled).
>>> pickle.dumps(object())
b'\x80\x03cbuiltins\nobject\nq\x00)\x81q\x01.'
>>> json.dumps(object())
Traceback (most recent call last):
...
TypeError: <object object at 0x7fa0348230c0> is not JSON serializable
Solution 2
Pickle is great but I think it's worth mentioning literal_eval
from the ast
module for an even lighter weight solution if you're only serializing basic python types. It's basically a "safe" version of the notorious eval
function that only allows evaluation of basic python types as opposed to any valid python code.
Example:
>>> d = {}
>>> d[0] = range(10)
>>> d['1'] = {}
>>> d['1'][0] = range(10)
>>> d['1'][1] = 'hello'
>>> data_string = str(d)
>>> print data_string
{0: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], '1': {0: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], 1: 'hello'}}
>>> from ast import literal_eval
>>> d == literal_eval(data_string)
True
One benefit is that the serialized data is just python code, so it's very human friendly. Compare it to what you would get with pickle.dumps
:
>>> import pickle
>>> print pickle.dumps(d)
(dp0
I0
(lp1
I0
aI1
aI2
aI3
aI4
aI5
aI6
aI7
aI8
aI9
asS'1'
p2
(dp3
I0
(lp4
I0
aI1
aI2
aI3
aI4
aI5
aI6
aI7
aI8
aI9
asI1
S'hello'
p5
ss.
The downside is that as soon as the the data includes a type that is not supported by literal_ast
you'll have to transition to something else like pickling.
Solution 3
Use Python's json module, or simplejson if you don't have python 2.6 or higher.
Solution 4
If you fully trust the string and don't care about python injection attacks then this is very simple solution:
d = { 'method' : "eval", 'safe' : False, 'guarantees' : None }
s = str(d)
d2 = eval(s)
for k in d2:
print k+"="+d2[k]
If you're more safety conscious then ast.literal_eval
is a better bet.
Solution 5
One thing json
cannot do is dict
indexed with numerals. The following snippet
import json
dictionary = dict({0:0, 1:5, 2:10})
serialized = json.dumps(dictionary)
unpacked = json.loads(serialized)
print(unpacked[0])
will throw
KeyError: 0
Because keys are converted to strings. cPickle
preserves the numeric type and the unpacked dict
can be used right away.
Related videos on Youtube
TIMEX
Updated on July 05, 2022Comments
-
TIMEX almost 2 years
How do I serialize a Python dictionary into a string, and then back to a dictionary? The dictionary will have lists and other dictionaries inside it.
-
Gabe over 13 yearsAre you familiar with
pickle
? -
Joachim Wagner about 7 yearsa module that is part of the Python Standard Library
-
-
nosklo over 13 years+1: json is way better than pickle and can be used in the same way:
json.dumps(mydict)
andjson.loads(mystring)
-
Dan D. over 13 yearsbut json can only do strings, numbers, lists, and dictionaries while pickle can do any python type but json is far more portable then pickle for the types it can do
-
Chris Morgan over 13 yearsI wish I knew what on earth that -1 was for.
-
Piotr Dobrogost over 9 yearsI guess this -1 might be for not mentioning security problems inherent in pickling. See stackoverflow.com/questions/10282175/attacking-pythons-pickle
-
Evan Pu about 8 yearshonestly this is the method I use all the time. thanks for sharing the safety tip. I use repr instead of str if the dictionary contains custom made objects that can be initialized by the repr string
-
Jason Heo over 7 yearsWhen you use
json.dumps()
, take care of some types (False
,True
, andNone
) because they are not compatible withjson
-
Jean-François Fabre over 7 yearsYou should use
ast.literal_eval
by default.eval
has zero added values and a big security issue. -
Ori over 6 yearsIt is worth mentioning that the cPickle part of the answer is not relevant for python 3.x. See here for the official explanation. In short, the accelerated C version of a package should be the default choice for any python module, and, if not available, the module itself falls back to the python implementation. This encapsulates the implementation from the user. Quote:
In Python 3.0... Users should always import the standard version, which attempts to import the accelerated version and falls back to the pure Python version.
-
ArtuX about 4 yearsBad things happen because persons honestly thought there is no security concerns in their particular peace of code, so they can just happly
eval
away. I'm just disgusted every time, someone promote this culture of sloppiness. Just usejson.dumps
andjson.loads
(or any other non-eval
solution), there is no real reason not to -
ArtuX about 4 years"Warning The pickle module is not secure. Only unpickle data you trust." - docs
-
Giovanni Faglia almost 3 yearsliteral_ast seem to me unable to encode instances of user defined classes, even the simplest one; the solution mentioned by @georg (namely pyYAML) do that and produces human readable serializations. From pypi.org project home (June 2021):
YAML is a data serialization format designed for human readability and interaction with scripting languages. PyYAML is a YAML parser and emitter for Python.
-
Giovanni Faglia almost 3 yearsSee pypi.org/project/PyYAML. Consider vulnerabilities though (eg: ibm.com/support/pages/…)
-
georg almost 3 yearsthx for the new link, I updated my answer accordingly.