base64.encodestring failing in python 3

Question

The following piece of code runs successfully on a python 2 machine:

base64_str = base64.encodestring('%s:%s' % (username,password)).replace('\n', '')

I am trying to port it over to Python 3 but when I do so I encounter the following error:

>>> a = base64.encodestring('{0}:{1}'.format(username,password)).replace('\n','')
Traceback (most recent call last):
  File "/auto/pysw/cel55/python/3.4.1/lib/python3.4/base64.py", line 519, in _input_type_check
    m = memoryview(s)
TypeError: memoryview: str object does not have the buffer interface

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/auto/pysw/cel55/python/3.4.1/lib/python3.4/base64.py", line 548, in encodestring
    return encodebytes(s)
  File "/auto/pysw/cel55/python/3.4.1/lib/python3.4/base64.py", line 536, in encodebytes
    _input_type_check(s)
  File "/auto/pysw/cel55/python/3.4.1/lib/python3.4/base64.py", line 522, in _input_type_check
    raise TypeError(msg) from err
TypeError: expected bytes-like object, not str

I tried searching examples for encodestring usage but not able to find a good document. Am I missing something obvious? I am running this on RHEL 2.6.18-371.11.1.el5

Solution 1

You can encode() the string (to convert it to byte string) , before passing it into base64.encodestring . Example -

base64_str = base64.encodestring(('%s:%s' % (username,password)).encode()).decode().strip()

Solution 2

To expand on Anand's answer (which is quite correct), Python 2 made little distinction between "Here's a string which I want to treat like text" and "Here's a string which I want to treat like a sequence of 8-bit byte values". Python 3 firmly distinguishes the two, and doesn't let you mix them up: the former is the str type, and the latter is the bytes type.

When you Base64 encode a string, you're not actually treating the string as text, you're treating it as a series of 8-bit byte values. That's why you're getting an error from base64.encodestring() in Python 3: because that is an operation that deals with the string's characters as 8-bit bytes, and so you should pass it a paramter of type bytes rather than a parameter of type str.

Therefore, to convert your str object into a bytes object, you have to call its encode() method to turn it into a set of 8-bit byte values, in whatever Unicode encoding you have chosen to use. (Which should be UTF-8 unless you have a very specific reason to choose something else, but that's another topic).

Solution 3

In Python 3 encodestring docs says:

def encodestring(s): """Legacy alias of encodebytes().""" import warnings warnings.warn("encodestring() is a deprecated alias, use encodebytes()", DeprecationWarning, 2) return encodebytes(s)

Here is working code for Python 3.5.1, it also shows how to url encode:

def _encodeBase64(consumer_key, consumer_secret):
    """
     :type consumer_key: str
     :type consumer_secret: str
     :rtype str
    """
    # 1. URL encode the consumer key and the consumer secret according to RFC 1738.
    dummy_param_name = 'bla'
    key_url_encoded = urllib.parse.urlencode({dummy_param_name: consumer_key})[len(dummy_param_name) + 1:]
    secret_url_encoded = urllib.parse.urlencode({dummy_param_name: consumer_secret})[len(dummy_param_name) + 1:]

    # 2. Concatenate the encoded consumer key, a colon character “:”, and the encoded consumer secret into a single string.
    credentials = '{}:{}'.format(key_url_encoded, secret_url_encoded)

    # 3. Base64 encode the string from the previous step.
    bytes_base64_encoded_credentials = base64.encodebytes(credentials.encode('utf-8'))

    return bytes_base64_encoded_credentials.decode('utf-8').replace('\n', '')

(I am sure it could be more concise, I am new to Python...)

Also see: http://pythoncentral.io/encoding-and-decoding-strings-in-python-3-x/

Comments

[Vinay Pai]

Hi Anand, Thanks for your response. When I tried what you suggested, I got the following error: >>> base64_str = base64.encodestring(('%s:%s' % (username,password)).encode()).replace('\n', '') Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: expected bytes, bytearray or buffer compatible object Please advise.

[Anand S Kumar]

I added an decode() call in between (added in the answer) , encodestring returns a byte string, we cannot directly call replace() on it.

[MrValdez]

I changed replace('\n', '') to strip() to make it more readable.

[wisecrack]

how to replace \n if they are in the middle of the string,the strip() doesn't work