How to decode base64 url in python?

Question

For Facebook fbml Apps Facebook is sending in a signed_request parameter explained here:

http://developers.facebook.com/docs/authentication/canvas

They have given the php version of decoding this signed request:

http://pastie.org/1054154

How to do the same in python?

I tried base64 module but I am getting Incorrect padding error:

>>> base64.urlsafe_b64decode("eyJhbGdvcml0aG0iOiJITUFDLVNIQTI1NiIsImV4cGlyZXMiOjEyNzk3NDYwMDAsIm9hdXRoX3Rva2VuIjoiMjk1NjY2Njk1MDY0fDIuRXpwem5IRVhZWkJVZmhGQ2l4ZzYzUV9fLjM2MDAuMTI3OTc0NjAwMC0xMDAwMDA0ODMyNzI5MjN8LXJ6U1pnRVBJTktaYnJnX1VNUUNhRzlNdEY4LiIsInVzZXJfaWQiOiIxMDAwMDA0ODMyNzI5MjMifQ")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/base64.py", line 112, in urlsafe_b64decode
    return b64decode(s, '-_')
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/base64.py", line 76, in b64decode
    raise TypeError(msg)
TypeError: Incorrect padding

Solution 1

Apparently you missed the last two characters when copying the original base64-encoded string. Suffix the input string with two is-equal (=) signs and it will be decoded correctly.

Solution 2

try

s = 'iEPX-SQWIR3p67lj_0zigSWTKHg'
base64.urlsafe_b64decode(s + '=' * (4 - len(s) % 4))

as it is written here

Solution 3

I have shared a code snippet for parsing signed_request parameter in a python based facebook canvas application at http://sunilarora.org/parsing-signedrequest-parameter-in-python-bas:

import base64
import hashlib
import hmac
import simplejson as json

def base64_url_decode(inp):
    padding_factor = (4 - len(inp) % 4) % 4
    inp += "="*padding_factor 
    return base64.b64decode(unicode(inp).translate(dict(zip(map(ord, u'-_'), u'+/'))))

def parse_signed_request(signed_request, secret):

    l = signed_request.split('.', 2)
    encoded_sig = l[0]
    payload = l[1]

    sig = base64_url_decode(encoded_sig)
    data = json.loads(base64_url_decode(payload))

    if data.get('algorithm').upper() != 'HMAC-SHA256':
        log.error('Unknown algorithm')
        return None
    else:
        expected_sig = hmac.new(secret, msg=payload, digestmod=hashlib.sha256).digest()

    if sig != expected_sig:
        return None
    else:
        log.debug('valid signed request received..')
return data

Solution 4

Alternative to @dae.eklen's solution, you can append === to it:

s = 'iEPX-SQWIR3p67lj_0zigSWTKHg'
base64.urlsafe_b64decode(s + '===')

This works because Python only complains about missing padding, but not extra padding.

Solution 5

Surprising, but currently accepted answer is not exactly correct. Like some other answers stated, it's something called base64url encoding, and it's a part of RFC7515.

Basically, they replaced '+' and '/' chars by '-' and '_' respectively; and additionally removed any trailing '=' chars, because you can always tell how many chars you're missing, just by looking at the encoded string length.

Here's illustrative example from RFC7515 in C#:

 static string base64urlencode(byte [] arg)
 {
   string s = Convert.ToBase64String(arg); // Regular base64 encoder
   s = s.Split('=')[0]; // Remove any trailing '='s
   s = s.Replace('+', '-'); // 62nd char of encoding
   s = s.Replace('/', '_'); // 63rd char of encoding
   return s;
 }

 static byte [] base64urldecode(string arg)
 {
   string s = arg;
   s = s.Replace('-', '+'); // 62nd char of encoding
   s = s.Replace('_', '/'); // 63rd char of encoding
   switch (s.Length % 4) // Pad with trailing '='s
   {
     case 0: break; // No pad chars in this case
     case 2: s += "=="; break; // Two pad chars
     case 3: s += "="; break; // One pad char
     default: throw new System.Exception(
       "Illegal base64url string!");
   }
   return Convert.FromBase64String(s); // Standard base64 decoder
 }

View more solutions

Comments

[kevin]

thanks i tried base64 , but i am getting this error: pastie.org/1054201

[user1066101]

Please actually post the smallest code that shows the error and the actual error. Most of us don't have the patiences to follow links all over the place.

[Geordie]

Note: If you're by chance using Azure blob URLs returned from a search, you need to strip out the trailing '0' from the encoded URL stackoverflow.com/questions/44338134/…

[kevin]

Geert, thanks for this. but that is exactly the code that i got from facebook and it did not have = at the end. is this expected?

[Geert]

This is not to be expected I would say. However, you can verify the length of the base64 input by checking the length of it: the length must always be a multiple of 4 bytes (this is actually the reason why the decoder threw an error). If it's not, you can add is-equal signs until it is and then the string will be decoded correctly.

[Nas Banov]

Seems = padding is not always required in all variants: en.wikipedia.org/wiki/Base64

[Nas Banov]

PS. seems like the python base64url implementation is broken - if i read wiki correct, string does not have to be padded for base64url!

[Geert]

RFC 3548 & RFC 4648 both state that "...implementations MUST include appropriate pad characters at the end of encoded data unless the specification referring to this document explicitly states otherwise." That's probably why Python's base64 does not accept strings that are not correctly padded.

[sax]

The solution by dae.eklen does the same and is more elegant. (base64.urlsafe_b64decode(s + '=' * (4 - len(s) % 4)))

[sax]

Make sure that the string s you work with is instance of str - unicode would fail with error. If that is your case use str(s) function for conversion.

[Dmitry Yudin]

for me helped to add "====" to my 32 baseencoded string

[dgel]

Thanks. That's a pretty short code snippet- it would be great to see it included in this answer.

[gdvalderrama]

Ok, this makes sense and works with your example. I'm confused though, I have a string of a length multiple of 4 with no padding returning the Incorrect padding error, when I had an =, I still get the same problem, but if I had at least == it works. What's up with that?

[Maarten Bodewes]

Warning this answer is wrong; it forgets about the - and _ characters that replaces the + and / characters.

[Maarten Bodewes]

@DmitryYudin That's a horrible way of hacking the code; base 64 uses either no padding (if the number of bytes is dividable by 3), a double == (for # bytes dividable by 3 + 1) or single = (if the bytes is dividable by 3 + 2). Note that this talks about base 64, not base 32 at all.

[Franklin Yu]

@NasBanov Python standard library didn’t implement base64url. It only implements RFC 3548.

[Franklin Yu]

This feature was proposed to standard library.

[CMartins]

Please edit your answer adding some explanation/documentation.

[Scott Johnson]

Even with the URL-safe variant, you still need to adjust the padding on the input before the operation will work.

[zabop]

Link is broken.