What's best practice for communication between Amazon EC2 instances?

Question

I've been setting up Amazon EC2 instances for an upcoming project. They are all micro instances, running Ubuntu Server 64bit. Here's what I've setup so far:

• Web Server -- Apache
• Database Server -- MySQL
• Development Server -- Apache & MySQL
• File Server -- SVN & Bacula (backups are done to S3 buckets)

Currently, there's only one Web Server, but eventually there will be more.

My first question is, what is the best, most secure way for Amazon EC2 instances to communicate between each other? Currently I'm using SSH, is that the best method?

According to Amazon, instances communicating between themselves using their Elastic IP addresses will be charged data transfer fees. However, instances communicating using their Private IP addresses can do so for free. Unfortunately, it appears Private IPs change if the instance is stopped and re-started.

So that's my second question, how do you make use of Amazon instances' Private IPs if they're not static?

I know that the instances probably won't be stopped and started very frequently, but still, if the IP address is in various config files, it would be a pain to have to go through them all and change it.

I'm primarily concerned about the Web servers, which will need access to the Database server and the File server, which will need access to all the instances when performing backups.

Note: I've never used Bacula before and I don't have it setup yet, but I'm assuming it will need the IP addresses of the clients to back them up.

Solution 1

Check out Eric Hammond's article explaining how to use Elastic IP addresses even from within EC2. This method does NOT result in any bandwidth charges because resolving the Elastic IP address (by name) from within EC2 returns the Private IP address.

http://alestic.com/2009/06/ec2-elastic-ip-internal

For more options, I have an article examining a few alternatives:

http://shlomoswidler.com/2010/06/track-changes-to-your-dynamic-cloud-services-automatically.html

Solution 2

Deploy your EC2 instance into an AWS Virtual Private Cloud (VPC). When you configure your VPC you will assign CIDR to all the EC2 instances in the VPC and the internal IP will be static.

Solution 3

• Most secure communication method

SSH is a very good method for transferring data between different servers but if you're looking for something like a permanent connection (to a database for example) you can use any kind of encrypted tunneling software like stunnel

• Non static private IPs

Since there's no way to have static private IPs you can use some kind of automatic server deployment for this, there's several tools like mcollective, capistrano or func that'll allow you to register your new instance to a central designated server and generate actions on multiple machines based on that

Comments

[Andras Balázs Lajtha]

+1 -- I would love elastic private IPs. Also take note that I think you can't communicate between regions over private IPs.

[Eric Hammond]

I think you'll find that a single m1.small is going to give you much better performance than multiple t1.micro. Then, upgrade to c1.medium. Then, start launching multiple c1.medium or use an even larger instance type. Note, however, that m1.small and c1.medium only support 32-bit, not 64-bit.

[ks78]

I had originally posted the same question on StackOverflow. There someone suggested using a DNS server, so if the Private IPs change it wouldn't matter because they'd be using DNS-provided names to communicate. What's your opinion of that solution?

[lynxman]

It's not a bad solution if you're able to keep a dynamic DNS zone up to date, although since you would need to use that as well as a primary DNS server to all your instances it would also give you a single point of failure, that's why I suggested mcollective or capistrano instead since that would also give you the advantage of executing complex operations across your nodes

[ks78]

Thanks. I just wondered what your opinion of that solution would be. I'll look into mcollective and capistrano.

[ks78]

Thanks for your response. I'm thinking more and more that dynamic DNS is the way to go, although I've never setup a DNS before. Do you know of any sites that offer a detailed walkthrough of how to do it?

[ceejayoz]

This is by far the best approach.

[icyitscold]

sorry to revive an old thread but are private IPs still non-static on ec2? That's not what I've been witnessing on a brand new ec2 setup - the network interfaces stick around after I stop servers and private IP addresses haven't changed on me yet. I also seem to be able to choose secondary private IP addresses if needed.