Convert little endian string to integer

Question

I have read samples out of a wave file using the wave module, but it gives the samples as a string, it's out of wave so it's little endian (for example, \x00).

What is the easiest way to convert this into a python integer, or a numpy.int16 type? (It will eventually become a numpy.int16, so going directly there is fine).

Code needs to work on little endian and big endian processors.

Solution 1

The struct module converts packed data to Python values, and vice-versa.

>>> import struct
>>> struct.unpack("<h", "\x00\x05")
(1280,)
>>> struct.unpack("<h", "\x00\x06")
(1536,)
>>> struct.unpack("<h", "\x01\x06")
(1537,)

"h" means a short int, or 16-bit int. "<" means use little-endian.

Solution 2

struct is fine if you have to convert one or a small number of 2-byte strings to integers, but array and numpy itself are better options. Specifically, numpy.fromstring (called with the appropriate dtype argument) can directly convert the bytes from your string to an array of (whatever that dtype is). (If numpy.little_endian is false, you'll then have to swap the bytes -- see here for more discussion, but basically you'll want to call the byteswap method on the array object you just built with fromstring).

Solution 3

Kevin Burke's answer to this question works great when your binary string represents a single short integer, but if your string holds binary data representing multiple integers, you will need to add an additional 'h' for each additional integer that the string represents.

For Python 2

Convert Little Endian String that represents 2 integers

import struct
iValues = struct.unpack("<hh", "\x00\x04\x01\x05")
print(iValues)

Output: (1024, 1281)

Convert Little Endian String that represents 3 integers

import struct
iValues = struct.unpack("<hhh", "\x00\x04\x01\x05\x03\x04")
print(iValues)

Output: (1024, 1281, 1027)

Obviously, it's not realistic to always guess how many "h" characters are needed, so:

import struct

# A string that holds some unknown quantity of integers in binary form
strBinary_Values = "\x00\x04\x01\x05\x03\x04"

# Calculate the number of integers that are represented by binary string data
iQty_of_Values = len(strBinary_Values)/2

# Produce the string of required "h" values
h = "h" * int(iQty_of_Values)

iValues = struct.unpack("<"+h, strBinary_Values)
print(iValues)

Output: (1024, 1281, 1027)

For Python 3

import struct

# A string that holds some unknown quantity of integers in binary form
strBinary_Values = "\x00\x04\x01\x05\x03\x04"

# Calculate the number of integers that are represented by binary string data
iQty_of_Values = len(strBinary_Values)/2

# Produce the string of required "h" values
h = "h" * int(iQty_of_Values)

iValues = struct.unpack("<"+h, bytes(strBinary_Values, "utf8"))
print(iValues)

Output: (1024, 1281, 1027)

Comments

[Jeffrey Aylesworth]

This is really good to know as well, I'm going to go with the struct solution though, so that I don't need to worry about correcting endianess manually.

[Alex Martelli]

If you're in no hurry, or have very few data points, that's fine. Otherwise, you can codify endianness as a string as part of the dtype (I don't know the details offhand, though).

[Sir l33tname]

For python3 you need to convert your string first to bytes struct.unpack("<h", bytes("\x00\x06", "utf8"))