How to make a fixed-size byte variable in Python

Question

Let's say, I have a string (Unicode if it matters) variable which is less than 100 bytes. I want to create another variable with exactly 100 byte in size which includes this string and is padded with zero or whatever. How would I do it in Python 3?

Solution 1

For assembling packets to go over the network, or for assembling byte-perfect binary files, I suggest using the struct module.

• struct — Interpret bytes as packed binary data

Just for the string, you might not need struct, but as soon as you start also packing binary values, struct will make your life much easier.

Depending on your needs, you might be better off with an off-the-shelf network serialization library, such as Protocol Buffers; or you might even just use JSON for the wire format.

• Protocol Buffer Basics: Python
• PyMOTW - JavaScript Object Notation Serializer

Solution 2

Something like this should work:

st = "具有"
by = bytes(st, "utf-8")
by += b"0" * (100 - len(by))
print(by)
# b'\xe5\x85\xb7\xe6\x9c\x890000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000'

Obligatory addendum since your original post seems to conflate strings with the length of their encoded byte representation: Python unicode explanation

Solution 3

To pad with null bytes you can do it the way they do it in the stdlib base64 module.

some_data = b'foosdsfkl\x05'
null_padded = some_data + bytes(100 - len(some_data))

Solution 4

Here's a roundabout way of doing it:

>>> import sys
>>> a = "a"
>>> sys.getsizeof(a)
22
>>> a = "aa"
>>> sys.getsizeof(a)
23
>>> a = "aaa"
>>> sys.getsizeof(a)
24

So following this, an ASCII string of 100 bytes will need to be 79 characters long

>>> a = "".join(["a" for i in range(79)])
>>> len(a)
79
>>> sys.getsizeof(a)
100

This approach above is a fairly simple way of "calibrating" strings to figure out their lengths. You could automate a script to pad a string out to the appropriate memory size to account for other encodings.

def padder(strng):
    TARGETSIZE = 100
    padChar = "0"

    curSize = sys.getsizeof(strng)

    if curSize <= TARGETSIZE:
        for i in range(TARGETSIZE - curSize):
            strng = padChar + strng

        return strng
    else:
        return strng  # Not sure if you need to handle strings that start longer than your target, but you can do that here

View more solutions