Access AWS API Gateway with IAM roles from Python

Question

I have an AWS API Gateway that I would like to secure using IAM Roles .

I am looking for a package to help me accessing it using Python. I am trying to avoid implementing the entire Version 4 Signing Process. I am sure there must be some library I can use.

I looked into aws-requests-auth but it requires a "aws_service" to generate the signature. I looked also to boto3 but I am not able to find any way to just add authentication headers to a general request.

Solution 1

You can use aws-requests-auth to generate the signature for your request to API Gateway with execute-api as the service name.

import requests
from aws_requests_auth.aws_auth import AWSRequestsAuth

auth = AWSRequestsAuth(aws_access_key='YOURKEY',
                       aws_secret_access_key='YOURSECRET',
                       aws_host='restapiid.execute-api.us-east-1.amazonaws.com',
                       aws_region='us-east-1',
                       aws_service='execute-api')

headers = {'params': 'ABC'}
response = requests.get('https://restapiid.execute-api.us-east-1.amazonaws.com/stage/resource_path',
                        auth=auth, headers=headers)

Solution 2

Just to build on Ka Hou Ieong's response, there is one other thing which tripped me up. I was using aws-requests-auth==0.3.0, and in using requests.get(url, auth=auth) I was still getting a 403.

;TLDR;: My URL had a querystring and it looks like aws-requests-auth doesn't or probably cannot make sure the querystring parameters are sorted in ascending order and %-encoded.

==> So once I changed my url querystring to be ordered and %-encoded, I got 200.

Details: I turned on API Gateway logging and I was getting

In [46]: resp = requests.get(url, auth=auth)

In [47]: resp.text
Out[47]: u'{"message":"The request signature we calculated
 does not match the signature you provided. Check your AWS Secret Access Key
 and signing method. Consult the service documentation for details.... 

(the new lines and truncation(...) above is mine)

Per the Amazon Canonical Request for Signature Version 4 documentation,

To construct the canonical query string, complete the following steps:

Sort the parameter names by character code point in ascending order. For example, a parameter name that begins with the uppercase letter F precedes a parameter name that begins with a lowercase letter b.

URI-encode each parameter name and value according to the following rules:

a. Do not URI-encode any of the unreserved characters that RFC 3986 defines: A-Z, a-z, 0-9, hyphen ( - ), underscore ( _ ), period ( . ), and tilde ( ~ ).

b. Percent-encode all other characters with %XY, where X and Y are hexadecimal characters (0-9 and uppercase A-F). For example, the space character must be encoded as %20 (not using '+', as some encoding schemes do) and extended UTF-8 characters must be in the form %XY%ZA%BC.

That canonical querystring is used in generating the Authorization Signature, and AWS applies the same rules when calculating the Signature Version 4 sig. Bottom line, I think of course aws-requests-auth Auth of course cannot change your url, you have to.

Solution 3

If you want to make a call using the IAM role, you should use BotoAWSRequestsAuth from aws-requests-auth:

import requests
from aws_requests_auth.boto_utils import BotoAWSRequestsAuth
auth = BotoAWSRequestsAuth(
    aws_host="API_ID.execute-api.us-east-1.amazonaws.com",
    aws_region="us-east-1",
    aws_service="execute-api"
)
response = requests.get("https://API_ID.execute-api.us-east-1.amazonaws.com/STAGE/RESOURCE", auth=auth)

This will use botocore to retrieve a key and secret from the AWS metadata service rather than you needing to pass them yourself.

Thanks to Ka Hou Leong for the suggestion of the aws-requests-auth library.

Solution 4

One more decision is using requests_aws4auth package:

import requests
from requests_aws4auth import AWS4Auth

session = requests.Session()
session.auth = AWS4Auth(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, 'us-east-1', 'execute-api')

response = requests.get('https://restapiid.execute-api.us-east-1.amazonaws.com/stage/resource_path', auth=auth)

Also, you need to add execution permissions for your IAM user, like these (all API endpoints in the region can be invoked):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "execute-api:Invoke"
      ],
      "Resource": [
        "arn:aws:execute-api:us-east-1:*:*"
      ]
    }
  ]
}

More permissions examples can be found by link

View more solutions

Comments

[FelixEnescu]

Thank, works like a charm. After setting Custom Domain Names I replaced aws_host and get URL with my own domain name.

[Travis Bear]

Where do you normally go to find out that execute-api is the correct service name?

[Serhii Kushchenko]

Currently, it works, very convenient. If you don;t have the botocore module, execute pip install boto3 --upgrade.

[peoplespete]

@TravisBear, looks like this link will help: docs.aws.amazon.com/service-authorization/latest/reference/…‌​. Each service in the navigation on the left, when selected will show a 'service prefix' which is what you would use for this.