Cannot use Requests-Module on AWS Lambda
Solution 1
I finally solved the problem: The structure in my zip file was broken. It is important that the python script and the packed dependencies (as folders) are in the root of the zip file. This solved my problem.
It's a bit depressing if you find such easy errors after hours of try and failure.
Solution 2
EDIT: On Oct-21-2019 Botocore removed the vendored version of requests: https://github.com/boto/botocore/pull/1829.
EDIT 2: (March 10, 2020): The deprecation date for the Lambda service to bundle the requests module in the AWS SDK is now January 30, 2021. https://aws.amazon.com/blogs/compute/upcoming-changes-to-the-python-sdk-in-aws-lambda/
To use requests module, you can simply import requests
from botocore.vendored
. For example:
from botocore.vendored import requests
def lambda_handler(event, context):
response = requests.get("https://httpbin.org/get", timeout=10)
print(response.json())
you can see this gist to know more modules that can be imported directly in AWS lambda.
Solution 3
If you're working with Python on AWS Lambda, and need to use requests, you better use urllib3, it is currently supported on AWS Lambda and you can import it directly, check the example on urllib3 site.
import urllib3
http = urllib3.PoolManager()
r = http.request('GET', 'http://httpbin.org/robots.txt')
r.data
# b'User-agent: *\nDisallow: /deny\n'
r.status
# 200
Solution 4
I believe you have lambda_function.py
on the Lambda console. You need to first create the Lambda function deployment package, and then use the console to upload the package.
- You create a directory, for example
project-dir
on your system (locally) - create
lambda_function.py
inproject-dir
, copy the content oflambda_function.py
from lambda console and paste it inproject-dir/lambda_function.py
pip install requests -t /path/to/project-dir
- Zip the content of the
project-dir
directory, which is your deployment package (Zip the directory content, not the directory)
Go to the Lambda console, select upload zip file in code entry type and upload your deployment package. Import requests should work without any error.
Solution 5
With this command download the folder package
pip install requests -t .
Run this command on your local machine, then zip your working directory, then upload to aws.
Related videos on Youtube
codepleb
#frontend #pwa #angular #vuejs #nodejs #expressjs #playwright #cryptocurrency #blockchain #dag
Updated on July 05, 2022Comments
-
codepleb almost 2 years
I need to do a rest-call within a python script, that runs once per day. I can't pack the "requests" package into my python-package using the AWS Lambdas. I get the error: "Unable to import module 'lambda_function': No module named lambda_function"
I broke it down to the hello_world predefined script. I can pack it into a zip and upload it. Everything works. As soon as I put "import requests" into the file, I get this error.
Here is what I already did:
- The permissions of the zip and the project folder (including subfolders) are set to `chmod 777`. So permissions shouldn't be a problem.
- The script itself is within the root folder. When you open the zip file, you directly see it.
- I installed the requests package into the root-folder of the project using `sudo pip install requests -t PATH_TO_ROOT_FOLDER`
The naming of everything looks like this:
- zip-file: lambda_function.zip
- py-file: lambda_function.py
- handler method: lambda_handler(event, context)
- handler-definition in the "webconfig: lambda_function.lambda_handler
The file I want to run in the end looks like this:
import requests import json def lambda_handler(event, context): url = 'xxx.elasticbeanstalk.com/users/login' headers = {"content-type": "application/json", "Authorization": "Basic Zxxxxxxxxx3NjxxZxxxxzcw==" } response = requests.put(url, headers=headers, verify=False) return 'hello lambda_handler'
I'm glad for ANY kind of help. I already used multiple hours on this issue.
-
Leon over 7 yearsIn your question, the position of the
-t
option in thepip install
command is incorrect - it must bepip install requests -t PATH_TO_ROOT_FOLDER
. Did you just mistype it, or this is how you really ran it? -
codepleb over 7 years@Leon: Right. Just checked the history and I did it like you mention it here. :) So that didn't cause the problem.
-
Leon over 7 yearsWhat version of Python do you use locally?
-
codepleb over 7 years@Leon: 2.7.12 (15chars)
-
Klaus D. over 7 yearsHave you seen the guide at docs.aws.amazon.com/lambda/latest/dg/… ?
-
codepleb over 7 years@KlausD. Yes I followed those steps
-
Leon over 7 yearsDoes the contents of your zip file suggest that the
requests
package was correctly added to it? -
codepleb over 7 years@Leon I just saw that the ZIP file did not contain the right structure. The files weren't in the "root" like I wrote in my question. Wtf man, such a dumb mistake. Thank you all! :)
-
kafka over 7 yearsDoes your script return json then? I'm trying to do similar, but keep getting 'not json serialisable'. I don't have 'import json' however... code works fine locally without 'import json' so not sure why it doesn't in Lambda.
-
codepleb over 7 years@kafka You may need to import the json library. Not sure though, but AWS uses python 2.7 and not 3.x. My script doesn't return json.
-
kafka over 7 yearsyeah python 2.7 here. Going to attempt with explicit reference to json library. Currently re-packaging.
-
kafka over 7 yearsOK no joy here, in your code you simply return 'hello lambda_helper', I want to return the entire JSON response from the API Get though.... no worries if you don't know I might do a new question
-
codepleb over 7 yearsI return status codes. But yeah, I'm not much into python. It was barely enough to write that script.
-
John Paul Hayes over 6 yearsMay I suggest that the answer goes one step further to clarify a crucial point? What does packed dependances mean? In this case, when you are zipping the packed contents you should do something like: zip package.zip -r * whilst in the directory containing the handler script and the dependances
-
The Unknown Dev about 6 yearsThank you. I've been driving myself insane trying to use
urllib2
(not as user friendly asrequests
) -
softmarshmallow over 5 yearsLittle more details please
-
Joshka over 5 yearsIs there a concern that these might go away in later versions? " While these vendored dependencies are still in the botocore package they should not be used as they will be removed in the future" botocore.amazonaws.com/v1/documentation/api/latest/…
-
codepleb about 5 yearsSorry guys, this question is so old, I cannot give a reliable answer anymore. That project I was working on is long gone. Packed dependencies meant dependencies as folders, that usually are in the root folder, but I can't give you any more on this sadly.
-
Jason Wheeler about 5 yearsdon't think this applies to AWS Lambda
-
Neeraj Gupta about 5 yearsWow. saved me from increasing my bundle size and keeping my lambda to few lines
-
SMDC almost 5 yearsThis should be the accepted answer. Helped people like me to save a lot of time.
-
BoomShaka over 4 yearsFYI: I get a deprecation warning if i use this solution.
-
cvb over 4 yearsI would not use this method anymore. The official repo for botocore has a warning saying that botocore.vendored requests will be removed in the future.
-
Michael Brant over 4 yearsSo if this is being deprecated, what do we use now?
-
Sathed over 4 years@MichaelBrant, you'll need to upload the requests library as part of your function. docs.aws.amazon.com/lambda/latest/dg/…
-
Penumbra over 4 yearsA note - this no longer seems to work.
AttributeError: module 'botocore.vendored.requests' has no attribute 'get'
-
Hansang over 4 yearsyou can use
pip install requests -t .
(that's a dot after the -t) from within the directory containing your lambda_function.py rather than typing out the full project dir path. (form qarly_blue's answer). Bonus: Also works on windows -
Hansang over 4 yearsthis answer comined with the one below by pramod munemanik together is currently the best answer
-
Tula almost 4 yearsIs there any way without zipping and without using vendor modules?
-
davegallant almost 4 years@Penumbra this is because you are likely using Python 3.8.
-
CLAbeel almost 3 yearsThis is the best answer.
-
Andras Deak -- Слава Україні over 2 yearsThis is just this 2017 answer rephrased.