python append dictionary to list

Question

According to this post, I need to use .copy() on a dictionary, if I want to reference a dictionary which gets updated in a loop (instead of always referencing the same dictionary). However, in my code example below this doesn't seem to work:

main.py:

import collections
import json

nodes_list = ['donald', 'daisy', 'mickey', 'minnie']
edges_list = [('donald', 'daisy', '3'), ('mickey', 'minnie', '3'), ('daisy', 'minnie', '2')]
node_dict, edge_dict = collections.defaultdict(dict), collections.defaultdict(dict)
ultimate_list = []


for n in nodes_list:
    node_dict["data"]["id"] = str(n)
    ultimate_list.append(node_dict.copy())

for e in edges_list:
    edge_dict["data"]["id"] = str(e[2])
    edge_dict["data"]["source"] = e[0]
    edge_dict["data"]["target"] = e[1]
    ultimate_list.append(edge_dict.copy())

print(json.dumps(ultimate_list, indent=2))

As a result here I get the following:

[
  {
    "data": {
      "id": "minnie"
    }
  },
  {
    "data": {
      "id": "minnie"
    }
  },
  {
    "data": {
      "id": "minnie"
    }
  },
  {
    "data": {
      "id": "minnie"
    }
  },
  {
    "data": {
      "target": "minnie",
      "id": "2",
      "source": "daysi"
    }
  },
  {
    "data": {
      "target": "minnie",
      "id": "2",
      "source": "daysi"
    }
  },
  {
    "data": {
      "target": "minnie",
      "id": "2",
      "source": "daysi"
    }
  }
]

Whereas I would actually expect to get this:

[
  {
    "data": {
      "id": "donald"
    }
  },
  {
    "data": {
      "id": "daisy"
    }
  },
  {
    "data": {
      "id": "mickey"
    }
  },
  {
    "data": {
      "id": "minnie"
    }
  },
  {
    "data": {
      "target": "donald",
      "id": "3",
      "source": "daysi"
    }
  },
  {
    "data": {
      "target": "mickey",
      "id": "3",
      "source": "minnie"
    }
  },
  {
    "data": {
      "target": "minnie",
      "id": "2",
      "source": "daysi"
    }
  }
]

Can anyone please tell me what I'm doing wrong here?

Solution 1

dict.copy only makes a shallow copy of the dict, the nested dictionaries are never copied, you need deep copies to have those copied over too.

However, you can simply define each new dict at each iteration of the loop and append the new dict at that iteration instead:

for n in nodes_list:
    node_dict = collections.defaultdict(dict) # create new instance of data structure
    node_dict["data"]["id"] = str(n)
    ultimate_list.append(node_dict)

Same applies to the edge_dict:

for e in edges_list:
    edge_dict = collections.defaultdict(dict)
    ...
    ultimate_list.append(edge_dict)

Solution 2

Use copy.deepcopy(your_dict): deepcopy.

Solution 3

I see a few things. According to your desired results your edge_list is a bit off.

Change:

('daisy', 'minnie', '2')

To:

('minnie', 'daisy', '2')

To create the data the way you would like in your desired output we can do this with a more basic approach to dicts.

If you are trying to match the desired results in your question then you are calling the wrong index in your for e in edges_list function.

It should be:

"target" : e[0]
"id" : str(e[2])
"source" : e[1]

First I removed

node_dict, edge_dict = collections.defaultdict(dict), collections.defaultdict(dict)

as its not needed for my method.

Next I changed how you are defining the data.

Instead of using pre-defined dictionaries we can just append the results of each set of data to the ultimate_list directly. This shortens the code and is a bit easier to set up.

for n in nodes_list:
    ultimate_list.append({"data" : {"id" : str(n)}})

for e in edges_list:
    ultimate_list.append({"data" : {"target" : e[0], "id" : str(e[2]), "source" : e[1]}})

print(json.dumps(ultimate_list, indent=2))

So the following code:

import collections
import json

nodes_list = ['donald', 'daisy', 'mickey', 'minnie']
edges_list = [('donald', 'daisy', '3'), ('mickey', 'minnie', '3'), ('minnie', 'daisy', '2')]
ultimate_list = []

for n in nodes_list:
    ultimate_list.append({"data" : {"id" : str(n)}})

for e in edges_list:
    ultimate_list.append({"data" : {"target" : e[0], "id" : str(e[2]), "source" : e[1]}})

print(json.dumps(ultimate_list, indent=2))

Should result in:

[
  {
    "data": {
      "id": "donald"
    }
  },
  {
    "data": {
      "id": "daisy"
    }
  },
  {
    "data": {
      "id": "mickey"
    }
  },
  {
    "data": {
      "id": "minnie"
    }
  },
  {
    "data": {
      "target": "donald",
      "id": "3",
      "source": "daisy"
    }
  },
  {
    "data": {
      "target": "mickey",
      "id": "3",
      "source": "minnie"
    }
  },
  {
    "data": {
      "target": "minnie",
      "id": "2",
      "source": "daisy"
    }
  }
]

Comments

[Chris_Rands]

ultimate_list sounds pretty special

[dliv]

Works now. Thank you very much!

[dliv]

Thank you for the more elegant solution!