max concurrent connection to amazon load balancer

Question

My testing shows that amazon load balancer rest connection with its instance when it has about 10k concurrent connections into it. Is that a limit of Amazon load balancer? If not, is there a setting for it? I need to support upto 1M concurrent connections for my testing.

Thanks,

Sean Nguyen

Solution

The ELB should scale way beyond that, but you need to be testing from multiple test clients that appear to come from unique source IPs. This will cause multiple ELB instances to spawn multiple instances behind the scenes (this can be detected by DNS lookups). This is explained in the whitepaper that Rightscale published: http://blog.rightscale.com/2010/04/01/benchmarking-load-balancers-in-the-cloud/

Note that it takes a little while for ELB resources to scale out, so tests need to run for 20 minutes or more.

You also need to be sure that you have enough resources behind the load balancer. EC2 instances (as shown in the white paper mentioned above) seem to hit a throughput limit of around 100k packets per second which limits the number of concurrent connections that can be served (bear in mind the overhead of TCP and HTTP). You will need a lot of instances to be able to cope with 1M concurrent connections, and I'm not sure at what point you will hit the limit of ELB; in RightScale's test they only hit 19k.

Also you need to be clear about exactly what you mean by 1M concurrent connections, do you mean total keep-alive connections (assuming keep-alive enabled), or do you mean 1M transactions per second?

Comments

[Sean Nguyen]

No I mean 1M sockets open at the same time.

[James Little]

Do you have the ELB in TCP or HTTP mode?

[Sean Nguyen]

it is in http mode. And I am using jmeter trying to create 20k keep-alive connections to amazon server.