How to translate from a hexdigest to a digest and vice-versa?

Question

I want to store hashes as binary (64 bytes). But for any type of API (web service) I would want to pass them around as strings. hashlib.hexdigest() will give me a string, and hashlib.digest() will give me the binary. But if, for example, I read in the binary version from disk, how would I convert it to a string? And if I read in the string from a web service, how would I convert it to binary?

Solution 1

You could start with the string version to pass around and display:

>>> import hashlib
>>> string_version = hashlib.md5(b'hello world').hexdigest()

Convert it to binary to write it to disk:

>>> save_as_binary = string_version.encode('utf-8')
>>> print(save_as_binary)
b'5eb63bbbe01eeed093cb22bb8f5acdc3'

When reading it back from disk, convert it back to a string:

>>> back_to_string = save_as_binary.decode('utf-8')
>>> print(back_to_string)
5eb63bbbe01eeed093cb22bb8f5acdc3

Solution 2

You might want to look into binascii module, specifically hexlify and unhexlify functions.

Solution 3

In 2.x you can use str.decode('hex') and str.encode('hex') to convert between raw bytes and a hex string. In 3.x you need to use the binascii module.

Comments

[Ben]

To clarify: hashlib.md5(b'hello world').hexdigest().decode('hex') == hashlib.md5(b'hello world').digest()

[anirudha sonar]

@Ben Thanks alot . This has saved lot of my time. I am working on aws s3 and was trying to figure out how does ETag conversion happens from string->binary->string .. There are lot of answers online but nothing was working for me. But when I tried your answer, it worked just fine. So thank you alotttttt.

[Johannes Overmann]

The statement string_version.encode('utf-8') does not deliver the binary interpretation of the hexdigest. It just delivers a binary string of the hex string. save_as_binary is not the same as digest() which was asked for.

[young_souvlaki]

Or you can use the built-in bytes.fromhex().