pickle.load Not Working

Question

I got a file that contains a data structure with test results from a Windows user. He created this file using the pickle.dump command. On Ubuntu, I tried to load this test results with the following program:

import pickle
import my_module

f = open('results', 'r')
print pickle.load(f)
f.close()

But I get an error inside pickle module that no module named "my_module".

May the problem be due to corruption in the file, or maybe moving from Widows to Linux is the couse?

Solution 1

You should open the pickled file in binary mode, especially if you are using pickle on different platforms. See this and this questions for an explanation.

Solution 2

The problem lies in pickle's way of handling newline characters. Some of the line feed characters cripple module names in dumped / loaded data.

Storing and loading files in binary mode may help, but I was having trouble with them too. After a long time reading docs and searching I found that pickle handles several different "protocols" for storing data and due to backward compatibility it uses the oldest one: protocol 0 - the original ASCII protocol.

User can select modern protocol by specifing the protocol keyword while storing data in dump file, something like this:

pickle.dump(someObj, open("dumpFile.dmp", 'wb'), protocol=2)

or, by choosing the highest protocol available (currently 2)

pickle.dump(someObj, open("dumpFile.dmp", 'wb'), protocol=pickle.HIGHEST_PROTOCOL)

Protocol version is stored in dump file, so Load() function handles it automaticaly.

Regards