aws - "Unable to import module 'process': /var/task/numpy/core/multiarray.so: invalid ELF header"

Question

Using aws lambda I am receiving the following error when the script is run with the numpy module:

Unable to import module 'process': /var/task/numpy/core/multiarray.so: invalid ELF header

Is this problem related to numpy itself or numpy specifically on aws lambda. What is an invalid ELF header?

Edit:

I believe this is related to native code execution, as stated in this answer "invalid ELF header" when using the nodejs "ref" module on AWS Lambda

Solution

The problem has to do with the multiarray.so file which was complied on my local computer's architecture. Spin up an ec2 instance and create your virtualenv with the necessary dependencies. This will cause it to compile with the correct architecture as used by Aws lambda. Then download your virtualenv from the ec2 instance and use that for lambda.

Comments

[ZJS]

So the solution was to start with a fresh EC2 instance, create virtual env, install numpy, and then zip up site-packages? Or did you zip up even more than that?

[Michael]

Yes, I also added the compiled files which can be found in /lib64.

[zeta]

I just wanted to add that the EC2 instance to spin should be with name similar to (AMI name: amzn-ami-hvm-2016.03.3.x86_64-gp2) based on docs.aws.amazon.com/lambda/latest/dg/…. In case you don't know which one to create.