How to create variable number of EC2 instance resources in Cloudformation template?
Solution 1
The AWS::EC2::Instance Resource doesn't support the MinCount/MaxCount parameters of the underlying RunInstances API, so it's not possible to create a variable number of EC2 instances by passing Parameters to a single copy of this Resource.
To create a variable number of EC2 instance resources in CloudFormation template according to a template Parameter, and without deploying an Auto Scaling Group instead, there are two options:
1. Conditions
You can use Conditions to create a variable number of AWS::EC2::Instance Resources depending on the Parameter.
It's a little verbose (because you have to use Fn::Equals), but it works.
Here's a working example that allows the user to specify up to a maximum of 5 instances:
Description: Create a variable number of EC2 instance resources.
Parameters:
InstanceCount:
Description: Number of EC2 instances (must be between 1 and 5).
Type: Number
Default: 1
MinValue: 1
MaxValue: 5
ConstraintDescription: Must be a number between 1 and 5.
ImageId:
Description: Image ID to launch EC2 instances.
Type: AWS::EC2::Image::Id
# amzn-ami-hvm-2016.09.1.20161221-x86_64-gp2
Default: ami-9be6f38c
InstanceType:
Description: Instance type to launch EC2 instances.
Type: String
Default: m3.medium
AllowedValues: [ m3.medium, m3.large, m3.xlarge, m3.2xlarge ]
Conditions:
Launch1: !Equals [1, 1]
Launch2: !Not [!Equals [1, !Ref InstanceCount]]
Launch3: !And
- !Not [!Equals [1, !Ref InstanceCount]]
- !Not [!Equals [2, !Ref InstanceCount]]
Launch4: !Or
- !Equals [4, !Ref InstanceCount]
- !Equals [5, !Ref InstanceCount]
Launch5: !Equals [5, !Ref InstanceCount]
Resources:
Instance1:
Condition: Launch1
Type: AWS::EC2::Instance
Properties:
ImageId: !Ref ImageId
InstanceType: !Ref InstanceType
Instance2:
Condition: Launch2
Type: AWS::EC2::Instance
Properties:
ImageId: !Ref ImageId
InstanceType: !Ref InstanceType
Instance3:
Condition: Launch3
Type: AWS::EC2::Instance
Properties:
ImageId: !Ref ImageId
InstanceType: !Ref InstanceType
Instance4:
Condition: Launch4
Type: AWS::EC2::Instance
Properties:
ImageId: !Ref ImageId
InstanceType: !Ref InstanceType
Instance5:
Condition: Launch5
Type: AWS::EC2::Instance
Properties:
ImageId: !Ref ImageId
InstanceType: !Ref InstanceType
1a. Template preprocessor with Conditions
As a variation on the above, you can use a template preprocessor like Ruby's Erb to generate the above template based on a specified maximum, making your source code more compact and eliminating duplication:
<%max = 10-%>
Description: Create a variable number of EC2 instance resources.
Parameters:
InstanceCount:
Description: Number of EC2 instances (must be between 1 and <%=max%>).
Type: Number
Default: 1
MinValue: 1
MaxValue: <%=max%>
ConstraintDescription: Must be a number between 1 and <%=max%>.
ImageId:
Description: Image ID to launch EC2 instances.
Type: AWS::EC2::Image::Id
# amzn-ami-hvm-2016.09.1.20161221-x86_64-gp2
Default: ami-9be6f38c
InstanceType:
Description: Instance type to launch EC2 instances.
Type: String
Default: m3.medium
AllowedValues: [ m3.medium, m3.large, m3.xlarge, m3.2xlarge ]
Conditions:
Launch1: !Equals [1, 1]
Launch2: !Not [!Equals [1, !Ref InstanceCount]]
<%(3..max-1).each do |x|
low = (max-1)/(x-1) <= 1-%>
Launch<%=x%>: !<%=low ? 'Or' : 'And'%>
<% (1..max).each do |i|
if low && i >= x-%>
- !Equals [<%=i%>, !Ref InstanceCount]
<% elsif !low && i < x-%>
- !Not [!Equals [<%=i%>, !Ref InstanceCount]]
<% end
end
end-%>
Launch<%=max%>: !Equals [<%=max%>, !Ref InstanceCount]
Resources:
<%(1..max).each do |x|-%>
Instance<%=x%>:
Condition: Launch<%=x%>
Type: AWS::EC2::Instance
Properties:
ImageId: !Ref ImageId
InstanceType: !Ref InstanceType
<%end-%>
To process the above source into a CloudFormation-compatible template, run:
ruby -rerb -e "puts ERB.new(ARGF.read, nil, '-').result" < template.yml > template-out.yml
For convenience, here is a gist with the generated output YAML for 10 variable EC2 instances.
2. Custom Resource
An alternate approach is to implement a Custom Resource that calls the RunInstances/TerminateInstances APIs directly:
Description: Create a variable number of EC2 instance resources.
Parameters:
InstanceCount:
Description: Number of EC2 instances (must be between 1 and 10).
Type: Number
Default: 1
MinValue: 1
MaxValue: 10
ConstraintDescription: Must be a number between 1 and 10.
ImageId:
Description: Image ID to launch EC2 instances.
Type: AWS::EC2::Image::Id
# amzn-ami-hvm-2016.09.1.20161221-x86_64-gp2
Default: ami-9be6f38c
InstanceType:
Description: Instance type to launch EC2 instances.
Type: String
Default: m3.medium
AllowedValues: [ m3.medium, m3.large, m3.xlarge, m3.2xlarge ]
Resources:
EC2Instances:
Type: Custom::EC2Instances
Properties:
ServiceToken: !GetAtt EC2InstancesFunction.Arn
ImageId: !Ref ImageId
InstanceType: !Ref InstanceType
MinCount: !Ref InstanceCount
MaxCount: !Ref InstanceCount
EC2InstancesFunction:
Type: AWS::Lambda::Function
Properties:
Handler: index.handler
Role: !GetAtt LambdaExecutionRole.Arn
Code:
ZipFile: !Sub |
var response = require('cfn-response');
var AWS = require('aws-sdk');
exports.handler = function(event, context) {
var physicalId = event.PhysicalResourceId || 'none';
function success(data) {
return response.send(event, context, response.SUCCESS, data, physicalId);
}
function failed(e) {
return response.send(event, context, response.FAILED, e, physicalId);
}
var ec2 = new AWS.EC2();
var instances;
if (event.RequestType == 'Create') {
var launchParams = event.ResourceProperties;
delete launchParams.ServiceToken;
ec2.runInstances(launchParams).promise().then((data)=> {
instances = data.Instances.map((data)=> data.InstanceId);
physicalId = instances.join(':');
return ec2.waitFor('instanceRunning', {InstanceIds: instances}).promise();
}).then((data)=> success({Instances: instances})
).catch((e)=> failed(e));
} else if (event.RequestType == 'Delete') {
if (physicalId == 'none') {return success({});}
var deleteParams = {InstanceIds: physicalId.split(':')};
ec2.terminateInstances(deleteParams).promise().then((data)=>
ec2.waitFor('instanceTerminated', deleteParams).promise()
).then((data)=>success({})
).catch((e)=>failed(e));
} else {
return failed({Error: "In-place updates not supported."});
}
};
Runtime: nodejs4.3
Timeout: 300
LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal: {Service: [lambda.amazonaws.com]}
Action: ['sts:AssumeRole']
Path: /
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
Policies:
- PolicyName: EC2Policy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- 'ec2:RunInstances'
- 'ec2:DescribeInstances'
- 'ec2:DescribeInstanceStatus'
- 'ec2:TerminateInstances'
Resource: ['*']
Outputs:
Instances:
Value: !Join [',', !GetAtt EC2Instances.Instances]
Solution 2
I think what the original poster was after is something like:
"Parameters" : {
"InstanceCount" : {
"Description" : "Number of instances to start",
"Type" : "String"
},
...
"MyAutoScalingGroup" : {
"Type" : "AWS::AutoScaling::AutoScalingGroup",
"Properties" : {
"AvailabilityZones" : {"Fn::GetAZs" : ""},
"LaunchConfigurationName" : { "Ref" : "MyLaunchConfiguration" },
"MinSize" : "1",
"MaxSize" : "2",
"DesiredCapacity" : **{ "Ref" : "InstanceCount" }**,
}
},
...in other words, insert the number of initial instances (the capacity) from a parameter.
Solution 3
Short answer is : you can't. You can't get the exact same result (N identical EC2 instances, not tied by an auto scaling group).
Launching several instances alike from the console is not like creating an auto scaling group with N instances as desired capacity. It's just a useful shortcut you have, instead of having to go N times through the same EC2 creation process. It's called "a reservation" (no relation to reserved instance). Auto scaling groups are a different beast (even though you end up with N identical EC2 instances).
You can either:
- duplicate (yuk) the EC2 resource in the template
- use a nested template, which will do the EC2 creation itself, and call it N times from your master stack, feeding it each time with the same parameters
Problem is, the number of EC2 instances will not be dynamic, it cannot be a parameter.
- use a frontend to CloudFormation templates, like troposphere, which allows you to write the EC2 description inside a function, and call the function N times (my choice now). At the end, you've got a CloudFormation template which does the job, but you've written the EC2 creation code only once. It's not a real CloudFormation parameter, but at the end of the day, you get your dynamical number of EC2.
Solution 4
Meanwhile there are lots of AWS CloudFormation Sample Templates available, and several include launching multiple instances, albeit usually demonstrating other features in parallel; for example, the AutoScalingKeepAtNSample.template creates a load balanced, Auto Scaled sample website and is configured to start 2 EC2 instances for this purpose as per this template excerpt:
"WebServerGroup": {
"Type": "AWS::AutoScaling::AutoScalingGroup",
"Properties": {
"AvailabilityZones": {
"Fn::GetAZs": ""
},
"LaunchConfigurationName": {
"Ref": "LaunchConfig"
},
"MinSize": "2",
"MaxSize": "2",
"LoadBalancerNames": [
{
"Ref": "ElasticLoadBalancer"
}
]
}
},
There are more advanced/complete samples available as well, e.g. the Drupal template for a Highly Available Web Server with Multi-AZ Amazon RDS database instance and using S3 for storing file content, which is currently configured to allow 1-5 web server instances talking to a Multi-AZ MySQL Amazon RDS database instance and running behind an Elastic Load Balancer, which orchestrates the web server instances via Auto Scaling.
Solution 5
Use the Ref function.
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-ref.html
User-defined variables are defined in the "Parameters" section of the config file. In the "Resources" section of the config file you can fill in values using references to these parameters.
{
"AWSTemplateFormatVersion": "2010-09-09",
...
"Parameters": {
"MinNumInstances": {
"Type": "Number",
"Description": "Minimum number of instances to run.",
"Default": "1",
"ConstraintDescription": "Must be an integer less than MaxNumInstances."
},
"MaxNumInstances": {
"Type": "Number",
"Description": "Maximum number of instances to run.",
"Default": "5",
"ConstraintDescription": "Must be an integer greater than MinNumInstances."
},
"DesiredNumInstances": {
"Type": "Number",
"Description": "Number of instances that need to be running before creation is marked as complete in CloudFormation management console.",
"Default": "1",
"ConstraintDescription": "Must be an integer in the range specified by MinNumInstances..MaxNumInstances."
}
},
"Resources": {
"MyAutoScalingGroup": {
"Type": "AWS::AutoScaling::AutoScalingGroup",
"Properties": {
...
"MinSize": { "Ref": "MinNumInstances" },
"MaxSize": { "Ref": "MaxNumInstances" },
"DesiredCapacity": { "Ref": "DesiredNumInstances" },
...
},
},
...
},
...
}
In the example above { "Ref": ... } is used to fill values into the template. In this case we're providing integers as values for "MinSize" and "MaxSize".
nivertech
Updated on July 09, 2022Comments
-
nivertech almost 2 years
How to create variable number of EC2 instance resources in Cloudformation template, according to a template parameter?
The EC2 API and management tools allow launching multiple instances of the same AMI, but I can't find how to do this using Cloudformation.