Python writing binary

Question

I use python 3 I tried to write binary to file I use r+b.

for bit in binary:
    fileout.write(bit)

where binary is a list that contain numbers. How do I write this to file in binary?

The end file have to look like b' x07\x08\x07\

Thanks

Solution 1

When you open a file in binary mode, then you are essentially working with the bytes type. So when you write to the file, you need to pass a bytes object, and when you read from it, you get a bytes object. In contrast, when opening the file in text mode, you are working with str objects.

So, writing “binary” is really writing a bytes string:

with open(fileName, 'br+') as f:
    f.write(b'\x07\x08\x07')

If you have actual integers you want to write as binary, you can use the bytes function to convert a sequence of integers into a bytes object:

>>> lst = [7, 8, 7]
>>> bytes(lst)
b'\x07\x08\x07'

Combining this, you can write a sequence of integers as a bytes object into a file opened in binary mode.

---

As Hyperboreus pointed out in the comments, bytes will only accept a sequence of numbers that actually fit in a byte, i.e. numbers between 0 and 255. If you want to store arbitrary (positive) integers in the way they are, without having to bother about knowing their exact size (which is required for struct), then you can easily write a helper function which splits those numbers up into separate bytes:

def splitNumber (num):
    lst = []
    while num > 0:
        lst.append(num & 0xFF)
        num >>= 8
    return lst[::-1]

bytes(splitNumber(12345678901234567890))
# b'\xabT\xa9\x8c\xeb\x1f\n\xd2'

So if you have a list of numbers, you can easily iterate over them and write each into the file; if you want to extract the numbers individually later you probably want to add something that keeps track of which individual bytes belong to which numbers.

with open(fileName, 'br+') as f:
    for number in numbers:
        f.write(bytes(splitNumber(number)))

Solution 2

where binary is a list that contain numbers

A number can have one thousand and one different binary representations (endianess, width, 1-complement, 2-complement, floats of different precision, etc). So first you have to decide in which representation you want to store your numbers. Then you can use the struct module to do so.

For example the byte sequence 0x3480 can be interpreted as 32820 (little-endian unsigned short), or -32716 (little-endian signed short) or 13440 (big-endian short).

Small example:

#! /usr/bin/python3

import struct

binary = [1234, 5678, -9012, -3456]
with open('out.bin', 'wb') as f:
    for b in binary:
        f.write(struct.pack('h', b)) #or whatever format you need

with open('out.bin', 'rb') as f:
    content = f.read()
    for b in content:
        print(b)
    print(struct.unpack('hhhh', content)) #same format as above

prints

210
4
46
22
204
220
128
242
(1234, 5678, -9012, -3456)

Comments

[wim]

+1 but maybe mention open and bytes behave differently on 2.x

[poke]

@wim OP said they’re using Python 3; added the tag to the question now too. (Guess we both did that at the same time ^^)

[wim]

yes, I just noticed and tagged it at the same instant! now I have an edit in the history with nothing actually changed

[poke]

@Hyperboreus Would appreciate if you would remove your downvote after my update.

[Hyperboreus]

@poke How can OP read a file created with your algorithm? How can he know whether an occurrence of the separator byte sequence is actually a separator or a chance occurrence generated by encoding a number with splitNumber (which uses all 256 possible values)? Let's say separator is \r\n, how can he read and write the list [10, 13, 10, 1]?

[poke]

@Hyperboreus None of that was part of the question, it’s all just overinterpretation on your behalf.

[Hyperboreus]

@poke Maybe, I understood the question was about writing a list of numbers to a file, but obviously you interpreted the answer more adequately than me as yours has been accepted. Nevertheless I see no point in providing an algorithm to write positives integers to a file, if the resulting file cannot be read without ambiguities.

[poke]

@Hyperboreus Well, from the title and the example output OP mentioned, I understood the question simply as how to write binary, which is why I explained that. I merely mentioned the conversion from a number list using bytes as a side note given that OP apparently has the numbers in a list. The output highly suggest that these numbers represent bytes though. My following edits were just triggered by your downvote for the limitations of using bytes to convert the number list, but to me do not really seem relevant at all to the question.

[Nande]

shouldnt you use "w+b" on the first open? (notice the w)

[poke]

@Nande w+ truncates the file, so that might not be desireable. r+ opens the file for writing too.

[Nande]

@poke , i just tested it and with python 2.7 when the file doesnt exists calling open with "r+" will raise an exception. i think the correct way to use it is with "rw+" but that raises an exception too

[poke]

@Nande That behavior for r+ is intended. Not all open modes will create the file if it doesn’t exist. That doesn’t make r+ less valid though ;) rw+ isn’t a valid mode btw., see the docs for more information on the open modes.

[lmiguelvargasf]

When the file does not exist, you should you use 'bw+' instead of 'br+'