IP Address exhaustion for lambdas in VPC

Question

I am currently part of a group that is dealing with the repercussions of deploying 300+ AWS Lambda functions running Node.js in place of a more traditional REST API. I recently implemented an application that runs on EC2 instances in a VPC on a private subnet. Thus, the Lambdas themselves, which will be communicating heavily with this application, are also placed into private subnets in the same VPC. The EC2 instances run in 6 private /24 subnets (one for each availability zone in our region). The lambdas have 6 /20 private subnets that they are configured to run in. As I understand it, each invocation of a lambda will allocate an IP address in one of those subnets, which gives me about 24,564 IP addresses before we run out.

My questions are:

1) does a Lambda occupy an IP address only for the period that it is actually running and then the IP address is released? More succinctly, are we going to run out IP addresses fairly quickly as we're expecting an uptick in invocations very soon? I know I can just allocate more of the address space in the VPC to the lambdas, but will this be a problem we are continuously chasing?

2) How can I see in the AWS console or elsewhere how many IP addresses are being occupied by lambdas?

NOTE: I am a junior DevOps engineer so forgive me if something is unclear and feel free to correct me

Solution

This answer is now out of date. See this blog post or other AWS documentation for up to date information.

---

This page says how many ENIs are required when using Lambda inside a VPC - this is per subnet

If your Lambda function accesses a VPC, you must make sure that your VPC has sufficient ENI capacity to support the scale requirements of your Lambda function. You can use the following formula to approximately determine the ENI requirements.

Projected peak concurrent executions * (Memory in GB / 3GB)

Let's say you have 10 concurrent executions of a lambda with 1GB allocated, that is 10 x 1 / 3 = 3.33 or 4 ENIs / private IPs per subnet. If you have 100 concurrent executions with 2GB allocated it's 80 ENIs / private IPs per subnet. I don't think there's any description of how often the ENIs are created and deleted, and that implementation detail could change over time.

You can look at ENIs in the console or the ENI API.

Lambda has to have permissions to create and delete ENIs, as per this question.

Comments

[ceejayoz]

See aws.amazon.com/blogs/compute/container-reuse-in-lambda - containers (and the IPs they've reserved) should get reused if you're seeing steady, constant usage. Should be readily testable - just have them log their IP and invoke them in a way that mirrors your usage pattern for a while, then review the logs.

[Joseph Ruffino]

This answer is out of date as of August 2020: aws.amazon.com/blogs/compute/… The simplified (NAT) ENI mapping currently used decreases ENI usage: "Every unique security group:subnet combination across functions in your account requires a distinct network interface. If a combination is shared across multiple functions in your account, we reuse the same network interface across functions."

[Tim]

Yeah, I agree. It's not practical to keep the hundreds or thousands of answers I've supplied up to date, they're correct at a point in time but technology changes.

[assiegee]

Sorry for my bad english comprehension.. Does the new update means that # IP address in the subnet is NO longer the limiting factor for number of lambda can be executed at the same time?