Unzipped size must be smaller than 262144000 bytes - AWS Lambda Error

Question

I have tried to upload my application using servless/lambda function AWS, but i got this issue:

An error occurred: AppLambdaFunction - Unzipped size must be smaller than 262144000 bytes (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: 8ea0d887-5743-4db1-96cd-6c5efa57b081).

What is the best way to resolve it?

Look my dependencies:

  "dependencies": {
    "ethereumjs-tx": "^1.3.7",
    "aws-sdk": "^2.4.52",
    "body-parser": "^1.18.3",
    "compression": "^1.7.4",
    "consign": "^0.1.6",
    "cors": "^2.8.5",
    "express": "^4.16.4",
    "helmet": "^3.16.0",
    "moment": "^2.24.0",
    "openzeppelin-solidity": "^2.3.0",
    "serverless": "^1.48.2",
    "serverless-http": "^1.9.1",
    "serverless-offline": "^4.9.4",
    "truffle": "^5.1.9",
    "truffle-hdwallet-provider": "^1.0.17",
    "web3": "^1.2.5-rc.0"
  },

Serverless.yml:

provider:
  name: aws
  runtime: nodejs8.10
  stage: v1
  region: us-east-1
  timeout: 30
  memorySize: 512
  package:
    excludeDevDependencies: true
    exclude:
      - .git/**
      - .vscode/**        
      - venv/**

functions:
  app:  
    handler: handler.run
    events:
      - http:
          path: /
          method: ANY
          cors: true
      - http:
          path: /{proxy+}
          method: ANY
          cors: true

plugins:
  - serverless-offline  

Solution 1

Use the directive exclude at your serverless.yml file. In case of Python, I've been used it as follows:

package:
  exclude:
    - node_modules/**
    - venv/**

The build process will exclude them from the build before sending to AWS.

Tip I got in this issue at Github. The documentation for this directive is detailed here.

Solution 2

You can use module bundlers to package the code.

Using module bundlers such as webpack

You can consider using plugins like serverless-webpack. The serverless-webpack plugin is using webpack to build the project and it will only include the bare minimum files required to run your application. It will not include the entire node_modules directory. so that your deployment package will be smaller.

a note about using of Lambda layers

Like others mentioned, you can use the layers and move some of the libraries and code to the layer. Layers are mainly used to share code between functions. The unzipped deployed package including layers cannot exceed 250MB.

hope this helps.

References:

https://github.com/serverless-heaven/serverless-webpack

https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html#configuration-layers-path

Comments

[gandaliter]

I don't think you can have a package over 250MB, no matter where you put it. Serverless is probably uploading it to S3 first anyway.

[debug]

That is the limit, you're right. 250MB is a lot. They must be pulling in a lot of extra stuff.

[Michel Fernandes]

Worked here and saved my night! :)

[nbs]

If 250MB is the limit & if I required to use machine learning libs like xgboost, what is the way out? The zipped version of xgboost is about 150 MB and unable create a layer that has only this library

[jccampos]

@NBhat you should look for another alternative if you can't fit everything in a lambda, some limits are soft (you can request an increase by talking to AWS) while others (like deployment file size) are not docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.h‌​tml

[nbs]

Refer to this comment; I'm using a combination of both: aws layers (for scipy numpy pandas that's within limit) & extracting xgboost lib into /tmp/pkg/ directory along with adding /tmp/pkg to sys.path. I was successful with this approach.

[Liran Brimer]

package.exclude is soon to be deprecated and patterns should be used instead according to the docs

[0xD1x0n]

how do my packages get loaded then...?

[Abhishek Gautam]

use requirements.txt along with custom: pythonRequirements: dockerizePip: true zip: true slim: true