JSON object must be str, not 'bytes'

Question

Using Python 3.5.1, I pulled in a text file where each line is in JSON form: {"a":"windows", "b":"stairs"...}

import json
path = 'folder/data.txt'
records=[json.loads(line) for line in open(path,'rb')]

But I received the error:

the JSON object must be str, not 'bytes'

I have no problem printing the first line of file, so I am reassured that the file path is correct.

Solution 1

Open the file in text mode, not binary mode (possibly explicitly passing encoding='utf-8' to override the system default, since JSON is usually stored as UTF-8). The json module only takes str input; reading from a file opened in binary mode returns bytes objects:

# Using with statement just for good form; in this case it would work the
# same on CPython, but on other interpreters or different CPython use cases,
# it's easy to screw something up; use with statements all the time to build good habits
with open(path, encoding='utf-8') as f:
    records=[json.loads(line) for line in f]

Solution 2

Try: records=[json.loads(line.decode()) for line in open(path,'rb')]

Solution 3

You do not want to specify "rb", since binary representation of the file isn't going to be readable by the JSON module. You likely want "utf-8" encoding and "read". EDIT: I originally had said both of these are defaults, but it was brought to my attention that many OS's have different default encoding and that Python uses the system setting as default in open(). I would therefore recommend explicitly providing the encoding setting as "utf-8".

json supports loading from an open file with "json.load" instead of "json.loads", which loads from a string, so we can skip the read-in-as-text and go right to JSON. I don't think you'll want to "loads" individual lines, since this likely won't be valid JSON.

import json
# open has __enter__ and __exit__ functions, so we can call it as a guard
# using "with" syntax and it'll close when the scope ends
with open(r".\myjson.json", encoding="utf-8") as fh:
    # load() is a convenience function to help us avoid iterating lines
    # on our own. It calls loads() on the whole doc and returns an obj
    json_obj = json.load(fh)
print (json_obj)