Cause of slow download speed on a particular EC2 instance?

Question

I have a networking issue I'm trying to solve. I have two EC2 instances, same zone, same type.

On one of the two EC2 instances (the 'bad' instance), the download speed is really poor (200k/s), while on the other (the 'good' instance), the download speed is fine, comfortable at 30M/s +). To clarify, I'm talking about downloading files to the EC2 instance while ssh'd into the server, e.g running wget with a large file. I've tried different files, including S3 objects and a large linux ISO from elsewhere.

Running ethtool eth0 only returns 'Link detected: yes' for both.

When running ifconfig, both return the same for most part, aside from how the good instance shows no error packets yet the bad instance shows many, with the figure increasing by about 10 a second, even when I'm not actively running a download:

UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
RX packets:168372370 errors:5075643 dropped:0 overruns:0 frame:0
TX packets:122116480 errors:0 dropped:0 overruns:0 carrier:0

Both servers are configured the same, at least were supposed to be. How can I go about diagnosing the cause for the slow download speed? Is there anything particular to EC2 instances that could cause this?

Having trouble knowing where to start. Thanks for any help!

Solution

If your instance is EBS-backed then stop the instance, wait a minute or two, and then restart it. That will cause the instance to be started up on different hardware and may resolve your issue.

Comments

[Admin]

Have you tried cloning the "bad" one to see if launching a new copy of it works? Might be a hardware thing.

[James]

Thanks for the response KayakJim. Can you recommend ways in which I could identify if the hardware is bad without needing to restart an instance and see if it fixes the issue? Would like to know what to look out for in future.

[daemonofchaos]

Unfortunately Amazon does not provide that capability as they abstract the hardware to the VM so it all looks the same regardless of what it actually is. The basis for my suggestion comes from having a similar problem happen to me. We had 5 instances all the same AMI, type, and zone but one took over 2 minutes just to connect via SSH while the others were near instant. We spent 3 days trying everything and then I posted on the AWS forums about it. An AWS employee responded with what I suggested. We did it and everything worked fine after that.

[James]

Simply stopping and starting the instance did the track - speed back to being what it should be. Thanks for the help!

[daemonofchaos]

Glad to hear you got it resolved. :)