How to append data to YAML file

Question

I have a file *.yaml with contents as below:

bugs_tree:
  bug_1:
    html_arch: filepath
    moved_by: user1
    moved_date: '2018-01-30'
    sfx_id: '1'

I want to add a new child element to this file under the node [bugs_tree] I have tried to do this as below:

if __name__ == "__main__":
    new_yaml_data_dict = {
        'bug_2': {
            'sfx_id': '2', 
            'moved_by': 'user2', 
            'moved_date': '2018-01-30', 
            'html_arch': 'filepath'
        }
    }

    with open('bugs.yaml','r') as yamlfile:
        cur_yaml = yaml.load(yamlfile)
        cur_yaml.extend(new_yaml_data_dict)
        print(cur_yaml)

Then file should looks that:

bugs_tree:
  bug_1:
    html_arch: filepath
    moved_by: username
    moved_date: '2018-01-30'
    sfx_id: '1234'
  bug_2:
    html_arch: filepath
    moved_by: user2
    moved_date: '2018-01-30'
    sfx_id: '2'

When I'm trying to perform .append() OR .extend() OR .insert() then getting error

cur_yaml.extend(new_yaml_data_dict)
AttributeError: 'dict' object has no attribute 'extend'

Solution 1

If you want to update the file, a read isn't enough. You need to also write again against the file. Something like this would work:

with open('bugs.yaml','r') as yamlfile:
    cur_yaml = yaml.safe_load(yamlfile) # Note the safe_load
    cur_yaml['bugs_tree'].update(new_yaml_data_dict)

if cur_yaml:
    with open('bugs.yaml','w') as yamlfile:
        yaml.safe_dump(cur_yaml, yamlfile) # Also note the safe_dump

I didn't test this, but he idea is that you use a read to read the file and write to write to the file. Use safe_load and safe_dump like Anthon said:

"There is absolutely no need to use load(), which is documented to be unsafe. Use safe_load() instead"

Solution 2

You need to use update

cur_yaml.update(new_yaml_data_dict)

Resulting code

with open('bugs.yaml','r') as yamlfile:
        cur_yaml = yaml.load(yamlfile)
        cur_yaml.update(new_yaml_data_dict)
        print(cur_yaml)

with open('bugs.yaml','w') as yamlfile:
        yaml.safe_dump(cur_yaml, yamlfile) # Also note the safe_dump

Solution 3

Not sure if this will fit everyone's use cases, but I find you can just... append to the file IF it just holds a top level list.

One motivation for doing it this is that it just made sense. Another was that I am skeptical about having to reload and parse the whole yaml file everytime. What I wanted to do was to use Django middleware to log incoming requests to debug a bug I was having with multiple page loads in development and that's fairly time-critical.

If I had to do what the OP wanted, I'd think about leaving the bugs in their own file and compose the contents of bugs_tree from it.

import os
import yaml
def write(new_yaml_data_dict):

    if not os.path.isfile("bugs.yaml"):

        with open("bugs.yaml", "a") as fo:
            fo.write("---\n")

    #the leading spaces and indent=4 are key here!
    sdump = "  " + yaml.dump(
                new_yaml_data_dict
                ,indent=4
                )

    with open("bugs.yaml", "a") as fo:
        fo.write(sdump)

new_yaml_data_dict = {
        'bug_1': {
            'sfx_id': '1', 
            'moved_by': 'user2', 
            'moved_date': '2018-01-20', 
            'html_arch': 'filepath'
        }
    }
write(new_yaml_data_dict)
new_yaml_data_dict = {
        'bug_2': {
            'sfx_id': '2', 
            'moved_by': 'user2', 
            'moved_date': '2018-01-30', 
            'html_arch': 'filepath'
        }
    }
write(new_yaml_data_dict)

which results in

---
  bug_1:
    html_arch: filepath
    moved_by: user2
    moved_date: '2018-01-20'
    sfx_id: '1'
  bug_2:
    html_arch: filepath
    moved_by: user2
    moved_date: '2018-01-30'
    sfx_id: '2'

Comments

[Qex]

Are you sure about .update? AttributeError: 'list' object has no attribute 'update'

[Arpit Solanki]

Your traceback says that you are applying extend on dict which is not possible. list always use extend and dict uses update. So whichever is your data structure you can use accordingly. @Qex

[Nico Schlömer]

This is missing the write step.

[David Beauchemin]

As of 2022, the load function is deprecated without specifying the Loader due to security reason. One can set Loader=yaml.SafeLoader for an equivalent solution proposed here.