How to merge two json file in Python?

python json merge

18,835

Solution 1

The error you are facing is that your data1 upon loading from your file is going to be a list. You are trying to call the dictionary update on it, which is incorrect.

To help provide an example of your error, look at this:

>>> [].update('stuff')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'list' object has no attribute 'update'

I tried calling update on my list. There is no update method on the list. If I do this for a dictionary, however:

>>> d = {'a': 1}
>>> d.update({'b': 2})
>>> d
{'b': 2, 'a': 1}

As you can see, the update method exists, and works fine.

So, what you want to do is iterate over one of the list of dictionaries, and just update along the way. Since you are confident in your data being sequential and of equal length, this can be simply done by:

with open('f1.txt') as f1, open('f2.txt') as f2:
    first_list = json.load(f1)
    second_list = json.load(f2)

for i, v in enumerate(first_list):
    second_list[i].update(v)

So, what is happening here is that by using the enumerate, you will have an incremental count representing the index of your list to easily access each dictionary. With this, simply update the dictionary of each dict in second_list with the dictionary you get from each iteration of first_list

Solution 2

To merge dictionaries that are at the same index in two lists:

merged = [{**d1, **d2} for d1, d2 in zip(data1, data2)]

18,835

Author by

Hossein

Msc student of software engineering / interested in java/Python

Updated on June 11, 2022

Comments

Hossein almost 2 years

I'm trying to do a python script that merge 2 json files for example:

First file: test.json

[
  {
    "user_id": "273631610",
    "item_id": "0829482",
    "rating": "7"
  },
  {
    "user_id": "40688800",
    "item_id": "1343092",
    "rating": "8"
  },
  {
    "user_id": "395680865",
    "item_id": "1453405",
    "rating": "8"
  }
]

Second file: test_userz.json

[
{"contributors": null, "truncated": false, "text": "", "in_reply_to_status_id": null, "id": 421040281296052225, "favorite_count": 0, "source": "<a href=\"http://itunes.apple.com/us/app/imdb-movies-tv/id342792525?mt=8&uo=4\" rel=\"nofollow\">IMDb Movies & TV on iOS</a>", "retweeted": false, "coordinates": null, "entities": {"symbols": [], "user_mentions": [], "hashtags": [{"indices": [23, 28], "text": "IMDb"}], "urls": [{"url": "5E1fLOxB", "indices": [29, 51], "expanded_url": "http://www.imdb.com/title/tt0829482", "display_url": "imdb.com/title/tt0829482"}]}, "in_reply_to_screen_name": null, "id_str": "421040281296052225", "retweet_count": 0, "in_reply_to_user_id": null, "favorited": false, "user": {"follow_request_sent": false, "profile_use_background_image": true, "id": 273631610, "verified": false, "profile_text_color": "333333", "profile_image_url_https": "https://pbs.twimg.com/profile_images/378800000723050633/0fadb894b240dd426eb7b850dd8378d6_normal.jpeg", "profile_sidebar_fill_color": "DDEEF6", "is_translator": false, "geo_enabled": true, "entities": {"description": {"urls": []}}, "followers_count": 330, "protected": false, "location": "southend on sea", "default_profile_image": false, "id_str": "273631610", "lang": "en", "utc_offset": null, "statuses_count": 897, "description": "", "friends_count": 973, "profile_link_color": "0084B4", "profile_image_url": "http://pbs.twimg.com/profile_images/378800000723050633/0fadb894b240dd426eb7b850dd8378d6_normal.jpeg", "notifications": false, "profile_background_image_url_https": "https://abs.twimg.com/images/themes/theme1/bg.png", "profile_background_color": "C0DEED", "profile_background_image_url": "http://abs.twimg.com/images/themes/theme1/bg.png", "name": "Ben Morris", "is_translation_enabled": false, "profile_background_tile": false, "favourites_count": 12, "screen_name": "benyere", "url": null, "created_at": "Mon Mar 28 21:31:52 +0000 2011", "contributors_enabled": false, "time_zone": null, "profile_sidebar_border_color": "C0DEED", "default_profile": true, "following": false, "listed_count": 0}, "geo": null, "in_reply_to_user_id_str": null, "possibly_sensitive": false, "lang": "en", "created_at": "Wed Jan 08 22:06:40 +0000 2014", "in_reply_to_status_id_str": null, "place": null},
{"contributors": null, "truncated": false, "text": "", "in_reply_to_status_id": null, "id": 421040870931320833, "favorite_count": 0, "source": "<a href=\"http://www.apple.com\" rel=\"nofollow\">iOS</a>", "retweeted": false, "coordinates": null, "entities": {"symbols": [], "user_mentions": [], "hashtags": [{"indices": [31, 36], "text": "IMDb"}], "urls": [{"url": "dTFeexXDk", "indices": [37, 59], "expanded_url": "http://www.imdb.com/title/tt1343092", "display_url": "imdb.com/title/tt1343092"}]}, "in_reply_to_screen_name": null, "id_str": "421040870931320833", "retweet_count": 0, "in_reply_to_user_id": null, "favorited": false, "user": {"follow_request_sent": false, "profile_use_background_image": false, "id": 40688800, "verified": false, "profile_text_color": "333333", "profile_image_url_https": "https://pbs.twimg.com/profile_images/378800000807375392/2ee9bbee70e09ac097f8415794877e7c_normal.jpeg", "profile_sidebar_fill_color": "FFFFFF", "is_translator": false, "geo_enabled": true, "entities": {"description": {"urls": []}}, "followers_count": 230, "protected": false, "location": "Alexandria", "default_profile_image": false, "id_str": "40688800", "lang": "en", "utc_offset": 7200, "statuses_count": 8914, "description": "#TedxAlexandriaU head. resp of #emshi_fe_7artak in #alex Day dreamer. #medstudent by accident. I speak sarcasm.", "friends_count": 722, "profile_link_color": "0099CC", "profile_image_url": "http://pbs.twimg.com/profile_images/378800000807375392/2ee9bbee70e09ac097f8415794877e7c_normal.jpeg", "notifications": false, "profile_background_image_url_https": "https://si0.twimg.com/profile_background_images/778453690/27087117198cdf1d18bb86a36140de62.jpeg", "profile_background_color": "FFF04D", "profile_banner_url": "https://pbs.twimg.com/profile_banners/40688800/1378766815", "profile_background_image_url": "http://a0.twimg.com/profile_background_images/778453690/27087117198cdf1d18bb86a36140de62.jpeg", "name": "NOUR", "is_translation_enabled": false, "profile_background_tile": true, "favourites_count": 324, "screen_name": "Nour_Ayman", "url": null, "created_at": "Sun May 17 16:38:59 +0000 2009", "contributors_enabled": false, "time_zone": "Cairo", "profile_sidebar_border_color": "FFFFFF", "default_profile": false, "following": false, "listed_count": 2}, "geo": null, "in_reply_to_user_id_str": null, "possibly_sensitive": false, "lang": "en", "created_at": "Wed Jan 08 22:09:00 +0000 2014", "in_reply_to_status_id_str": null, "place": null},
{"contributors": null, "truncated": false, "text": "", "in_reply_to_status_id": null, "id": 421041098287755264, "favorite_count": 0, "source": "<a href=\"http://www.apple.com\" rel=\"nofollow\">iOS</a>", "retweeted": false, "coordinates": null, "entities": {"symbols": [], "user_mentions": [], "hashtags": [{"indices": [34, 39], "text": "IMDb"}], "urls": [{"url": "I2HqieyDnn", "indices": [40, 62], "expanded_url": "http://www.imdb.com/title/tt1453405", "display_url": "imdb.com/title/tt1453405"}]}, "in_reply_to_screen_name": null, "id_str": "421041098287755264", "retweet_count": 0, "in_reply_to_user_id": null, "favorited": false, "user": {"follow_request_sent": false, "profile_use_background_image": true, "id": 395680865, "verified": false, "profile_text_color": "333333", "profile_image_url_https": "https://pbs.twimg.com/profile_images/3130997221/4a981c8594f3c9ec127542ae40bac4ce_normal.jpeg", "profile_sidebar_fill_color": "DDEEF6", "is_translator": false, "geo_enabled": false, "entities": {"description": {"urls": []}}, "followers_count": 9, "protected": false, "location": "Denmark", "default_profile_image": false, "id_str": "395680865", "lang": "en", "utc_offset": 3600, "statuses_count": 1557, "description": "On basketball, film and other really cool things.", "friends_count": 27, "profile_link_color": "0084B4", "profile_image_url": "http://pbs.twimg.com/profile_images/3130997221/4a981c8594f3c9ec127542ae40bac4ce_normal.jpeg", "notifications": false, "profile_background_image_url_https": "https://abs.twimg.com/images/themes/theme1/bg.png", "profile_background_color": "C0DEED", "profile_background_image_url": "http://abs.twimg.com/images/themes/theme1/bg.png", "name": "Kristian Meller", "is_translation_enabled": false, "profile_background_tile": false, "favourites_count": 5, "screen_name": "kristianmeller", "url": null, "created_at": "Sat Oct 22 02:18:30 +0000 2011", "contributors_enabled": false, "time_zone": "Berlin", "profile_sidebar_border_color": "C0DEED", "default_profile": true, "following": false, "listed_count": 0}, "geo": null, "in_reply_to_user_id_str": null, "possibly_sensitive": false, "lang": "en", "created_at": "Wed Jan 08 22:09:55 +0000 2014", "in_reply_to_status_id_str": null, "place": null}
]

I would like to merge these into a single array like this:

[
{"user_id":"273631610","item_id":"0829482","rating":"7","contributors":null,"truncated":false,"text":"","in_reply_to_status_id":null,"id":421040281296052225,"favorite_count":0,"source":"<a href=\"http:\/\/itunes.apple.com\/us\/app\/imdb-movies-tv\/id342792525?mt=8&uo=4\" rel=\"nofollow\">IMDb Movies & TV on iOS<\/a>","retweeted":false,"coordinates":null,"entities":{"symbols":[],"user_mentions":[],"hashtags":[{"indices":[23,28],"text":"IMDb"}],"urls":[{"url":"/bk5E1fLOxB","indices":[29,51],"expanded_url":"http:\/\/www.imdb.com\/title\/tt0829482","display_url":"imdb.com\/title\/tt0829482"}]},"in_reply_to_screen_name":null,"id_str":"421040281296052225","retweet_count":0,"in_reply_to_user_id":null,"favorited":false,"user":{"follow_request_sent":false,"profile_use_background_image":true,"id":273631610,"verified":false,"profile_text_color":"333333","profile_image_url_https":"https:\/\/pbs.twimg.com\/profile_images\/378800000723050633\/0fadb894b240dd426eb7b850dd8378d6_normal.jpeg","profile_sidebar_fill_color":"DDEEF6","is_translator":false,"geo_enabled":true,"entities":{"description":{"urls":[]}},"followers_count":330,"protected":false,"location":"southend on sea","default_profile_image":false,"id_str":"273631610","lang":"en","utc_offset":null,"statuses_count":897,"description":"","friends_count":973,"profile_link_color":"0084B4","profile_image_url":"http:\/\/pbs.twimg.com\/profile_images\/378800000723050633\/0fadb894b240dd426eb7b850dd8378d6_normal.jpeg","notifications":false,"profile_background_image_url_https":"https:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png","profile_background_color":"C0DEED","profile_background_image_url":"http:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png","name":"Ben Morris","is_translation_enabled":false,"profile_background_tile":false,"favourites_count":12,"screen_name":"benyere","url":null,"created_at":"Mon Mar 28 21:31:52 +0000 2011","contributors_enabled":false,"time_zone":null,"profile_sidebar_border_color":"C0DEED","default_profile":true,"following":false,"listed_count":0},"geo":null,"in_reply_to_user_id_str":null,"possibly_sensitive":false,"lang":"en","created_at":"Wed Jan 08 22:06:40 +0000 2014","in_reply_to_status_id_str":null,"place":null},
{"user_id":"40688800","item_id":"1343092","rating":"8","contributors":null,"truncated":false,"text":"","in_reply_to_status_id":null,"id":421040281296052225,"favorite_count":0,"source":"<a href=\"http:\/\/itunes.apple.com\/us\/app\/imdb-movies-tv\/id342792525?mt=8&uo=4\" rel=\"nofollow\">IMDb Movies & TV on iOS<\/a>","retweeted":false,"coordinates":null,"entities":{"symbols":[],"user_mentions":[],"hashtags":[{"indices":[23,28],"text":"IMDb"}],"urls":[{"url":"/bk5E1fLOxB","indices":[29,51],"expanded_url":"http:\/\/www.imdb.com\/title\/tt0829482","display_url":"imdb.com\/title\/tt0829482"}]},"in_reply_to_screen_name":null,"id_str":"421040281296052225","retweet_count":0,"in_reply_to_user_id":null,"favorited":false,"user":{"follow_request_sent":false,"profile_use_background_image":true,"id":273631610,"verified":false,"profile_text_color":"333333","profile_image_url_https":"https:\/\/pbs.twimg.com\/profile_images\/378800000723050633\/0fadb894b240dd426eb7b850dd8378d6_normal.jpeg","profile_sidebar_fill_color":"DDEEF6","is_translator":false,"geo_enabled":true,"entities":{"description":{"urls":[]}},"followers_count":330,"protected":false,"location":"southend on sea","default_profile_image":false,"id_str":"273631610","lang":"en","utc_offset":null,"statuses_count":897,"description":"","friends_count":973,"profile_link_color":"0084B4","profile_image_url":"http:\/\/pbs.twimg.com\/profile_images\/378800000723050633\/0fadb894b240dd426eb7b850dd8378d6_normal.jpeg","notifications":false,"profile_background_image_url_https":"https:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png","profile_background_color":"C0DEED","profile_background_image_url":"http:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png","name":"Ben Morris","is_translation_enabled":false,"profile_background_tile":false,"favourites_count":12,"screen_name":"benyere","url":null,"created_at":"Mon Mar 28 21:31:52 +0000 2011","contributors_enabled":false,"time_zone":null,"profile_sidebar_border_color":"C0DEED","default_profile":true,"following":false,"listed_count":0},"geo":null,"in_reply_to_user_id_str":null,"possibly_sensitive":false,"lang":"en","created_at":"Wed Jan 08 22:06:40 +0000 2014","in_reply_to_status_id_str":null,"place":null},
{"user_id":"3956800865","item_id":"1453405","rating":"8","contributors":null,"truncated":false,"text":"","in_reply_to_status_id":null,"id":421040281296052225,"favorite_count":0,"source":"<a href=\"http:\/\/itunes.apple.com\/us\/app\/imdb-movies-tv\/id342792525?mt=8&uo=4\" rel=\"nofollow\">IMDb Movies & TV on iOS<\/a>","retweeted":false,"coordinates":null,"entities":{"symbols":[],"user_mentions":[],"hashtags":[{"indices":[23,28],"text":"IMDb"}],"urls":[{"url":"\/bk5E1fLOxB","indices":[29,51],"expanded_url":"http:\/\/www.imdb.com\/title\/tt0829482","display_url":"imdb.com\/title\/tt0829482"}]},"in_reply_to_screen_name":null,"id_str":"421040281296052225","retweet_count":0,"in_reply_to_user_id":null,"favorited":false,"user":{"follow_request_sent":false,"profile_use_background_image":true,"id":273631610,"verified":false,"profile_text_color":"333333","profile_image_url_https":"https:\/\/pbs.twimg.com\/profile_images\/378800000723050633\/0fadb894b240dd426eb7b850dd8378d6_normal.jpeg","profile_sidebar_fill_color":"DDEEF6","is_translator":false,"geo_enabled":true,"entities":{"description":{"urls":[]}},"followers_count":330,"protected":false,"location":"southend on sea","default_profile_image":false,"id_str":"273631610","lang":"en","utc_offset":null,"statuses_count":897,"description":"","friends_count":973,"profile_link_color":"0084B4","profile_image_url":"http:\/\/pbs.twimg.com\/profile_images\/378800000723050633\/0fadb894b240dd426eb7b850dd8378d6_normal.jpeg","notifications":false,"profile_background_image_url_https":"https:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png","profile_background_color":"C0DEED","profile_background_image_url":"http:\/\/abs.twimg.com\/images\/themes\/theme1\/bg.png","name":"Ben Morris","is_translation_enabled":false,"profile_background_tile":false,"favourites_count":12,"screen_name":"benyere","url":null,"created_at":"Mon Mar 28 21:31:52 +0000 2011","contributors_enabled":false,"time_zone":null,"profile_sidebar_border_color":"C0DEED","default_profile":true,"following":false,"listed_count":0},"geo":null,"in_reply_to_user_id_str":null,"possibly_sensitive":false,"lang":"en","created_at":"Wed Jan 08 22:06:40 +0000 2014","in_reply_to_status_id_str":null,"place":null}
]

To do that I did the following code:

import json

with open("test.json") as fin1:
    data1 = json.load(fin1)
with open("test_userz.json") as fin2:
    data2 = json.load(fin2)
data1.update(data2)
with open("merged.json", "w") as fout:
    json.dump(data1, fout)

but in runtime i see this error :

Traceback (most recent call last):
File "json/merge2jsons.py", line 7, in <module>
data1.update(data2)
AttributeError: 'list' object has no attribute 'update'

How can i do this merge ?

idjaw almost 7 years

What is your criteria for merging the two? Are you supposed to match on anything between them to determine which ones should be merged?
jordanm almost 7 years

Your code does not match your stacktrace. Your stacktrace is an error in the line data1.load(data2), which doesn't exist in your included code sample.
Hossein almost 7 years

@jordanm edited , that was update not load
Hossein almost 7 years

@idjaw yes these two json is relaition to each other by sequential

wwii almost 7 years

Are you able to load OP's test_userz.json?
idjaw almost 7 years

I copy pasted the structures in to files and had no problems.
wwii almost 7 years

Thnx, tried that and it worked - I was using loads on the string and load on an io.StringIO and it didn't like either, go figure.
idjaw almost 7 years

@wwii load and loads catches me off guard a lot too. :) It's one of those