Using Python How can I read the bits in a byte?

Question

I have a file where the first byte contains encoded information. In Matlab I can read the byte bit by bit with var = fread(file, 8, 'ubit1'), and then retrieve each bit by var(1), var(2), etc.

Is there any equivalent bit reader in python?

Solution 1

The smallest unit you'll be able to work with is a byte. To work at the bit level you need to use bitwise operators.

x = 3
#Check if the 1st bit is set:
x&1 != 0
#Returns True

#Check if the 2nd bit is set:
x&2 != 0
#Returns True

#Check if the 3rd bit is set:
x&4 != 0
#Returns False

Solution 2

With numpy it is easy like this:

Bytes = numpy.fromfile(filename, dtype = "uint8")
Bits = numpy.unpackbits(Bytes)

More info here:
http://docs.scipy.org/doc/numpy/reference/generated/numpy.fromfile.html

Solution 3

You won't be able to read each bit one by one - you have to read it byte by byte. You can easily extract the bits out, though:

f = open("myfile", 'rb')
# read one byte
byte = f.read(1)
# convert the byte to an integer representation
byte = ord(byte)
# now convert to string of 1s and 0s
byte = bin(byte)[2:].rjust(8, '0')
# now byte contains a string with 0s and 1s
for bit in byte:
    print bit

Solution 4

Joining some of the previous answers I would use:

[int(i) for i in "{0:08b}".format(byte)]

For each byte read from the file. The results for an 0x88 byte example is:

>>> [int(i) for i in "{0:08b}".format(0x88)]
[1, 0, 0, 0, 1, 0, 0, 0]

You can assign it to a variable and work as per your initial request. The "{0.08}" is to guarantee the full byte length

Solution 5

To read a byte from a file: bytestring = open(filename, 'rb').read(1). Note: the file is opened in the binary mode.

To get bits, convert the bytestring into an integer: byte = bytestring[0] (Python 3) or byte = ord(bytestring[0]) (Python 2) and extract the desired bit: (byte >> i) & 1:

>>> for i in range(8): (b'a'[0] >> i) & 1
... 
1
0
0
0
0
1
1
0
>>> bin(b'a'[0])
'0b1100001'

View more solutions

Comments

[Bruno Brant]

Do you mind adding more info, since the OP clearly seems like a beginner?

[David]

Sure I'm coming from a matlab background and can't find a 'ubit1' typecode for python. I've used the following: f=open('filename','rb') var=f.read(1) which returns var as the hex value string '\x04' how do i get the binary representation of the string?

[David]

Tried it and for the example where byte='\0x04' the code above returns '0b'

[David]

Thanks your code now gives byte=100 which is the correct base 2 representation of ord('\0x04')=4 but shouldn't the byte read be '00000100'

[Daniel G]

Sure, I'll add that really quickly (the problem is that it truncates leading zeros).

[David]

I realize I can pad the bits to get the representation once i have the binary value but it just seems odd that I can't read the bits directly.

[Daniel G]

This is a limitation of Python - it reads the entire byte at once. Then, when you call bin(), it gives you the smallest possible representation of that number in binary (that is, with the fewest possible bits, rather than using any standard like 8 or 32 bits). If you want all eight bits of each byte, you need to pad it again after calling bin().

[David]

Got it! thanks for the detail in your comment I looked at the bit shift operators but couldn't see how it worked for this. Your answer helps clarify the bitwise operators and the approach. Thanks

[David]

Daniel, I appreciate your help. Thanks for the quick and straight forward solution

[apraetor]

Thank you for this answer. I foolishly never thought of thinking about it like that; I'm still too stuck thinking about things in base 10. This makes so much sense, though.

[vac]

you have to install bitarray module first: pip install bitarray

[deadbabykitten]

one thing i'd like to point out about this approach is that if it is an extremely large file it could lead to hitting memory limits. just stuff to think about