What are the default security groups created when I set up AWS EB for the first time?

Question

I'm puzzled by the role played by several groups that seem to have been added automatically to my list of AWS security groups, connected in what I gather is the default configuration, and wonder how they work (and what about them it is safe to change). Specifically there are three that are mysterious:

• launch-wizard-1 which has an inbound rule SSH, TCP, 22, 0.0.0.0/0.
• default described as "default VPC security group" which has an inbound rule for all traffic and all ports that uses itself as a source.
• default_elb_... described as "ELB created security group used when no security group is specified during ELB creation - modifications could impact traffic to future ELBs" which has an inbound rule allowing HTTP from all IP addresses

The first two do not appear to be connected to any other security groups, while the latter is the source for a for an inbound HTTP rule in each of the security groups for my Elastic Beanstalk environments.

What do these do three groups do? Can I change them? Or change connections to them?

For example, the latter rule seems to have the effect of allowing HTTP traffic from anywhere to all of my EB environments. Can I change this rule to limit IPs (to to all environments)? Can I "un hook" the rule as a source from a given EB environment (e.g. replacing it as a source with a range of IPs)?

Solution

Looks like you've got a handle on what a security group is: a stateful firewall that is applied to EC2 instances.

When you manually launch an EC2 VM from the web console, AWS will provide you with the option of reusing an existing security group or creating a new one. When you create a new one, the default rule is SSH (port 22) and a default security group name of "launch-wizard-#".

Unfortunately, since a security group can be used by multiple EC2 instances, they are not cleaned up when you delete a VM. So if you deleted the VM that launch-wizard-1 was created with, it does not delete the security group.

Onto the "default security group for VPC". When you create your VPC, a default security group is created alongside with it. When EC2 instances are launched into a VPC subnet, they will have the default security group assigned to them if another is not specified. (http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_SecurityGroups.html#DefaultSecurityGroup).

So what does that rule mean that allows it to talk to itself? By default, all inbound traffic is denied by a security group. This 'talk to itself' inbound rule indicates that if two VMs both have this rule assigned to them, they will be allowed to communicate with one another on all ports. Should you use this default group? No. Create unique security groups that exercise the rule of least privilege (only open the ports you need to the instances that need them).

Unfortunately, I do not have much elastic beanstalk experience, so this is where my answer turns to assumptions. In the little that I have played with beanstalk, I recall that it created auxiliary resources in your account. This appears to the be the case with your Elastic Load Balancer (ELB). As the description indicates, when Elastic Beanstalk needs to launch a new load balancer, the load balancer will use this default group unless you specify another. I believe that this link documents how you would do this (http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.managing.elb.html).

In all cases, I would recommend against using the default security groups in favor of individual firewall rules unique to that instance's security needs.

Can you change or delete these?

• launch-wizard-1: Yes, you can delete or modify this group. Since you mentioned he is unused, go ahead and nuke him.
• default: VPC is finicky about some of the default resources that it creates. I tested it on my account and I cannot delete it. You can of course modify it, but I'd recommend instead just not using it.
• default_elb: If I remember properly, elastic beanstalk uses cloudformation to create additional resources, such as an ELB security group. You can modify this security group, but it will create inconsistencies between the cloudformation definition and reality. For your specific question, you can change the range of allowable IPs, but if you're writing rules on a private IP you won't be able to cross environments if the environments are deployed to separate VPCs.

Comments

[orome]

I think I understand SGs in general a bit less than it might seem from my question. But this is very helpful. I think I get the launch-wizard-1 group (essentially a leftover from something I did earlier and now don't use: I must have deleted the EC2 instance that it was created with, right?

[orome]

But starting with default I have some questions. Even though I don't see this group assigned to anything ("not connected to any other security groups" in my question) is it in fact associated with the VPC itself and, everything in it (which is probably why it can't be deleted)? If that's right than I don't think "not using it" is an option, is it?

[scubadev]

You are right about the launch wizard. The default group is owned by the vpc, but is optionally assigned to EC2 instances. When you create new instances you can assign any security group you'd like. If you don't select a security group then the default one will be used.

[orome]

But if I don't choose one, a launch-wizard group is assigned (named that if I don't rename it); not default. Right? Unless I pick it, default seems never to be assigned to anything. (When EB creates an instance, it uses the group for the environment, I think). Is default always there for everything in a VPC, no matter what? Or is it for the VPC as a whole, for attaching to other things (only, usefully, in other VPCs) so they can communicate with the VPC?

[scubadev]

The docs for Elastic Beanstalk (EB) & security groups (SG) are subpar, I had to create an environment to observe what was happening with SGs. Here's what is going on as far as I can see: 1. Your EB environment is created in a VPC. Here you have the opportunity to select a SG (let's call it A -- in your case this is most likely the VPC "default"). 2. Your EB env creates another SG specific to the tier you selected (web tier would allow :80 & :22) 3. Your EB env creates another SG for your ELB (default_elb). When instances are created for your EB environment, SG-A & SG-B are assigned.

[orome]

If I eb create, I get an environment with a single security group, awseb-e-... ("SecurityGroup for ElasticBeanstalk environment") — which I think corresponds to your 2 — that is assigned as the only security group to each instance in that environment. Neither default nor default_elb are assigned to anything.

[scubadev]

Did you select for your new (and old) elastic beanstalk environments to be part of a vpc? If not, that would better explain what you're describing. On your new environment, I wouldn't expect default or default_elb to be used. Beanstalk created awseb-e-... for you based on the environment type (web-server would receive a new security group with :80 & :22 open). default exists as part of your vpc, but is not being used because instances are not being deployed to it. Did you deploy a load balancer with your environment? If you did, I suspect eb would create another awseb-e-... for it.

[orome]

Everything — all my instances and all my security groups — list a VPC ID. I think I understand the awseb-e-... group. But that leaves me with the original question: what's going on with default and default-elb? And what does it mean that everything lists a VPC ID.

[scubadev]

default is easy to answer. By having a VPC, you must have a default VPC security group, regardless of if you ever use it. When you create an EB environment in a vpc, you can select the default group as an additional security group to be assigned (in addition to the awseb-e... that is created). The effective firewall rules on your instance would be the union of the awseb-e-... rules AND the secondary security group you select.

[scubadev]

default_elb is more difficult, as I'm not as familiar with it. When you launch an ELB into a VPC, you're given the option to select a security group. When launching into EC2 classic, you're not allowed to select a security group. The default_elb may be used in the case that you're deploying an ELB to EC2 classic. It may also be used when creating load balancers within EB. I'd recommend testing creating an ELB to EC2 classic and as part of an EB environment to assess whether default_elb is used in these cases, or if a new awseb-e... security group is generated.

[orome]

For default, that sounds consistent with what I see: and for console created EB environments it just looks like the default isn't assigned automatically. What remains confusing to me about default are the rules, especially inbound. With the default settings, default seems to be a useless rule to give to anything in the VPC, since the source is itself. It seems more like something you'd assign to a different VPC so that that VPC can communicate with the first VPC. Is that not right?

[scubadev]

The default group is more open then I'd like, but conceptually what they're doing makes sense. If you deploy two instances to a VPC without this rule applied, they would not be able to communicate with one another on any port. If you were to assign the default security to group to both of these instances, it would allow them to speak over any port. The key thing to remember about security groups is that all traffic is denied unless it is explicitly allowed. Self referential rules like this are useful for something like a memory cluster, where they need to sync with one another over port N.

[orome]

I think I'm getting closer now (bear with me): What's throwing me off I think is what it means to have a security group as a source in a rule. The default rule has this (in fact it has itself as the source), and the groups associated with EC2 instances have it (they each have a rule with default-elb... as a source. I think I understand how this works with default-elb..; but clearly I don't fully understand what it means to have a group as a source in general, because I'm still puzzled by the self reference in default.

[orome]

Obviously assigning a group to some instance lets that instance receive incoming traffic specified by inbound rules, and send outgoing traffic via the outbound rules. But —maybe – here's the crux: outgoing traffic from an instance comes from the rules attached to it. Is that right? So if my instances can send out to all IPs and can receive on HTTP/80 I could curl or wget over the internet between them, but nothing else (once I'm outside the groups no longer identify traffic, of course). But if each has group X that can receive from X, they can talk to each other inside the VPC. Right?

[scubadev]

Typically you write firewall rules to allow incoming traffic from a.b.c.d./r over port X. In an auto scaled cloud, you may not know the IP address of a box before it is created -- making it difficult to write rules with source IP addresses. Substituting a source security group in place of an IP address allows you to say "I accept incoming traffic from any instance that is assigned this SG". Concrete example, let's say you have SG-WS for a web server and SG-DB for a DB. You could add an incoming rule to SG-DB to allow traffic from any instance assigned SG-WS, regardless of the server's IP..

[orome]

So (almost there): adding a security group to an instance is (a) equivalent to "putting it in" or "making it a part of" a security group; (b) any traffic from an instance comes from the security groups it is a part of; and, (c) all security groups and instances are part of a VPC, so that if I had a second VPC and wanted it to I (i) could not give any of its instances rules from the first, but (ii) could allow it to receive traffic from the first by specifying a rule in the first as a source.

[scubadev]

A. You are adding that security group's set of rules to that instance. B. It comes from that instance private ip. Writing rules against a security group that Amazon provides (they handle the private ip mapping). C.i. yes C.ii. Not in practice. The security group as a source trick only works for private ip addresses. Two vpcs cannot communicate with each other using private ips, you need a publicly accessible ip address.

[orome]

I'm one level of abstraction up in my questions (which I think is what's missing, pervasively, from the AWS documentation) — A: Yes, but those two phrases mean the same thing right (they're used in the docs and all over the Web); B: Yes, but it boils down to (in terms of what is allowed) coming from the groups; i.e., if instance I is part of group G and instance J has a group with a rule allowing incoming from G then J can receive from I.

[scubadev]

Correct. On B, I wanted to make the distinction that this only allows traffic to and from private ip addresses. It will not allow all traffic, only that directed to a 10.b.c.d ip address. It will not work if vm a attempts to communicate with b using B's public ip.

[orome]

So (this is the last thing, I promise): I think I finally get the trick of setting up a security group with itself as a source. If, for example, I had some EC2 instances that were public facing web servers and an RDS in my VPC (but not public) that they used, I could simply create a security group with a suitable rule (e.g., MYSQL/TCP/3306) with itself as the source, and put all the EC2 instances and the RDS in the group — like circling the boxes for the EC2s and the RDS on a diagram.

[scubadev]

You could do that. Better practice would be two groups, one for the Web server and one for db. Db group allows incoming traffic from Web server. Web server group allows incoming from anywhere on 80/443

[orome]

Actually the Web (1) server group needs to allow traffic from the group associate with an ELB; and (2) if I want to control traffic to the Web server instances I need to replace (not edit) the group used by the ELB. Right?

[scubadev]

Yes, with the addition of a load balancer you should 3 total security groups. ELB accepts traffic from 0.0.0.0/0. Web Server accepts traffic from ELB on :80. DB accepts traffic from web server on :3306.

[orome]

I have related follow ups about ELB traffic and about the best way to restrict Web access to EB environments through an ELB that you might be able to help with.