Usage of pickle.dump in Python

Question

I'm trying to learn how to use the pickle module in Python:

import pickle
x = 123
f = open('data.txt','w')
pickle.dump(x,f)

Here's what I get:

Traceback (most recent call last):
  File "D:\python\test.py", line 5, in <module>
    pickle.dump(x,f)
TypeError: must be str, not bytes

However, this code works just fine:

import pickle
dump = pickle.dump(123)
print(dump)

What am I doing wrong?

Solution 1

The problem is that you're opening the file in text mode. You need to use binary here:

>>> f = open('data.txt','w')
>>> pickle.dump(123,f)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: must be str, not bytes
>>> 
>>> f = open('data.txt','wb')
>>> pickle.dump(123,f)
>>> 

Solution 2

The write method for file-like objects, only accept a single string argument. The dumps method in the pickle module automatically casts arguments as strings, whereas the the dump method will write a pickled representation of the object to the open file. Since 123 is not a string it throws the TypeError error.

This is acknowledged in pickle.dump documentation.

Comments

[Edwin]

Just something worth mentioning: for all file modes in Python, you should add a 'b' to the end of the string, even if it's only 'rb', since it's platform independent.

[Admin]

@Edwin: It doesn't do platform-specific newline transliterations, but in 3.x there's an important different: Files opened in binary mode handle only bytes, files opened in text mode handle only text ("unicode"). While binary mode certainly makes sense for binary data, and text mode may not apply in some other cases, your suggestion is far too general. If you'd just going to decode the bytes to text, opening in binary mode is usually nonsense. It's easier and more robust to let Python handle that, and it's better at guessing encodings than we are at hardcoding encodings.

[Edwin]

@delnan Ah, ok; I just remembered reading that somewhere on the python.org site.

[Sergey]

This I understand, but why doesn't the first piece of code (the one with pickle.dump) work?

[garnertb]

Because the write method on the file object only accepts a single string argument.

[DSM]

@SrinivasReddyThatiparthy: were you using Python 3? Things changed.

[DSM]

@SrinivasReddyThatiparthy: no, I mean that it would have worked in the past, and only stopped working in Python 3.