How can I save a list of dictionaries to a file?

Question

I have a list of dictionaries. Occasionally, I want to change and save one of these dictionaries so that the new message is utilized if the script is restarted. Right now, I make that change by modifying the script and rerunning it. I'd like to pull this out of the script and put the list of dictionaries into a configuration file of some kind.

I've found answers on how to write a list to a file, but this assumes that it is a flat list. How can I do it with a list of dictionaries?

My list looks like this:

logic_steps = [
    {
        'pattern': "asdfghjkl",
        'message': "This is not possible"
    },
    {
        'pattern': "anotherpatterntomatch",
        'message': "The parameter provided application is invalid"
    },
    {
        'pattern': "athirdpatterntomatch",
        'message': "Expected value for debugging"
    },
]

Solution 1

provided that the object only contains objects that JSON can handle (lists, tuples, strings, dicts, numbers, None, True and False), you can dump it as json.dump:

import json
with open('outputfile', 'w') as fout:
    json.dump(your_list_of_dict, fout)

Solution 2

if you want each dictionary in one line:

 import json
 output_file = open(dest_file, 'w', encoding='utf-8')
 for dic in dic_list:
    json.dump(dic, output_file) 
    output_file.write("\n")

Solution 3

Just for completeness I add also the json.dumps() method:

with open('outputfile_2', 'w') as file:
    file.write(json.dumps(logic_steps, indent=4))

Have a look here for the difference between json.dump() and json.dumps()

Solution 4

The way you will have to follow to write a dict to a file is kind different from the post you have mentioned.

First, you need serialize the object and than you persist it. These are fancy names for "write python objects to a file".

Python has 3 serialization modules included by default that you can use to achieve your objective. They are: pickle, shelve and json. Each one has its own characteristics and the one you have to use is the one which is more suitable to your project. You should check each module documentation to get more on it.

If your data will be only be accessed by python code, you can use shelve, here is an example:

import shelve

my_dict = {"foo":"bar"}

# file to be used
shelf = shelve.open("filename.shlf")

# serializing
shelf["my_dict"] = my_dict

shelf.close() # you must close the shelve file!!!

To retrieve the data you can do:

import shelve

shelf = shelve.open("filename.shlf") # the same filename that you used before, please
my_dict = shelf["my_dict"]
shelf.close()

See that you can treat the shelve object almost the same way you do with a dict.

View more solutions

Comments

[Mona Jalal]

I get the error TypeError: -0.69429028 is not JSON serializable

[Eliza R]

what is dest_file?

[Cyrus]

As I was importing my data from the file I needed to add fout.write("data = ") before the json.dump

[zwep]

@ElizaR dest_file is the location of the destination file. This can be something like /home/user/file on linux