How can I unpack binary hex formatted data in Python?
Solution 1
There's an easy way to do this with the binascii
module:
>>> import binascii
>>> print binascii.hexlify("ABCZ")
'4142435a'
>>> print binascii.unhexlify("4142435a")
'ABCZ'
Unless I'm misunderstanding something about the nibble ordering (high-nibble first is the default… anything different is insane), that should be perfectly sufficient!
Furthermore, Python's hashlib.md5
objects have a hexdigest()
method to automatically convert the MD5 digest to an ASCII hex string, so that this method isn't even necessary for MD5 digests. Hope that helps.
Solution 2
There's no corresponding "hex nibble" code for struct.pack, so you'll either need to manually pack into bytes first, like:
hex_string = 'abcdef12'
hexdigits = [int(x, 16) for x in hex_string]
data = ''.join(struct.pack('B', (high <<4) + low)
for high, low in zip(hexdigits[::2], hexdigits[1::2]))
Or better, you can just use the hex codec. ie.
>>> data = hex_string.decode('hex')
>>> data
'\xab\xcd\xef\x12'
To unpack, you can encode the result back to hex similarly
>>> data.encode('hex')
'abcdef12'
However, note that for your example, there's probably no need to take the round-trip through a hex representation at all when encoding. Just use the md5 binary digest directly. ie.
>>> x = md5.md5('some string')
>>> x.digest()
'Z\xc7I\xfb\xee\xc96\x07\xfc(\xd6f\xbe\x85\xe7:'
This is equivalent to your pack()ed representation. To get the hex representation, use the same unpack method above:
>>> x.digest().decode('hex')
'acbd18db4cc2f85cedef654fccc4a4d8'
>>> x.hexdigest()
'acbd18db4cc2f85cedef654fccc4a4d8'
[Edit]: Updated to use better method (hex codec)
Solution 3
In Python you use the struct module for this.
>>> from struct import *
>>> pack('hhl', 1, 2, 3)
'\x00\x01\x00\x02\x00\x00\x00\x03'
>>> unpack('hhl', '\x00\x01\x00\x02\x00\x00\x00\x03')
(1, 2, 3)
>>> calcsize('hhl')
8
HTH
davidmytton
Co-founder of Console (the best tools for developers). Researching sustainable computing at Imperial College London & Uptime Institute. Previously Co-founder, Server Density (acquired by StackPath).
Updated on May 08, 2020Comments
-
davidmytton almost 4 years
Using the PHP pack() function, I have converted a string into a binary hex representation:
$string = md5(time); // 32 character length $packed = pack('H*', $string);
The H* formatting means "Hex string, high nibble first".
To unpack this in PHP, I would simply use the unpack() function with the H* format flag.
How would I unpack this data in Python?
-
Brian over 15 yearsNote : "h" means something different in struct than "nibble encoded as hex" - it refers to a 16 bit integer.
-
Leandro López over 14 yearsIn the first version, is there anything special to import to use the group statement?