Writing a set to an output file in python

Question

I usually use json for lists, but it doesn't work for sets. Is there a similar function to write a set into an output file,f? Something like this, but for sets:

f=open('kos.txt','w')
json.dump(list, f)
f.close()

Solution 1

json is not a python-specific format. It knows about lists and dictionaries, but not sets or tuples.

But if you want to persist a pure python dataset you could use string conversion.

with open('kos.txt','w') as f:
   f.write(str({1,3,(3,5)}))  # set of numbers & a tuple

then read it back again using ast.literal_eval

import ast
with open('kos.txt','r') as f:
   my_set = ast.literal_eval(f.read())

this also works for lists of sets, nested lists with sets inside... as long as the data can be evaluated literally and no sets are empty (a known limitation of literal_eval). So basically serializing (almost) any python basic object structure with str can be parsed back with it.

For the empty set case there's a kludge to apply since set() cannot be parsed back.

import ast
with open('kos.txt','r') as f:
   ser = f.read()
my_set = set() if ser == str(set()) else ast.literal_eval(ser)

You could also have used the pickle module, but it creates binary data, so more "opaque", and there's also a way to use json: How to JSON serialize sets?. But for your needs, I would stick to str/ast.literal_eval

Solution 2

Using ast.literal_eval(f.read()) will give error ValueError: malformed node or string, if we write empty set in file. I think, pickle would be better to use.

If set is empty, this will give no error.

import pickle

s = set()

##To save in file
with open('kos.txt','wb') as f:
   pickle.dump(s, f)

##To read it again from file
with open('kos.txt','rb') as f:
   my_set = pickle.load(f)

Comments

[piyush-balwani]

This solution will give ValueError: malformed node or string: at ast.literal_eval(f.read()), if there is empty set.

[Jean-François Fabre]

str(set()) yields set() so yes, it doesn't work with an empty set. Some hack would be possible, though, checking for set() and replacing by an empty set

[Jean-François Fabre]

I already mentionned pickle in my solution. The only interesting feature is the empty set case, that can be emulated (discussed here: bugs.python.org/issue24663). otherwise pickle drawback is ... all the rest (impossible to edit, binary format...)