How to configure knife and EC2 to create a new instance from the command line?

Question

I am playing with Amazon EC2 to create instance. I was reading here: here in the knife documenation for EC2, that I would need to set:

# EC2:
knife[:aws_access_key_id]     = "Your AWS Access Key"
knife[:aws_secret_access_key] = "Your AWS Secret Access Key"

Now, when I try to set the security credentials from the AWS console in my knife.rb, I get a

ERROR: You did not provide a valid 'AWS SSH Key Id' value.

And, when I create a new "keypair" I get some .pem file, which confuses me too.

How do I setup my knife and EC2 to create a new node?

Solution 1

I'm assuming that you are receiving this error while creating a new node and not when you try to set the security credentials in your knife.rb.

To be precise, the error should come up when you initiate:

knife ec2 server create -I <ami-id>

The error

ERROR: You did not provide a valid 'AWS SSH Key Id' value.

is due to a missing SSH Keypair in your knife ec2 server create command. You need to specify an SSH Keypair whenever you launch an Amazon EC2 instance. This SSH public key is added to a newly launched instance to enable a passwordless SSH login. Keypairs can either be created or imported. Keypairs (for US-East region) can be found at EC2 -> Keypairs section of your Amazon AWS console.

Once you pick an SSH Keypair, you could specify it's name in your knife ec2 create command as:

knife ec2 server create -I <ami-id> -S <your-ssh-keypair-name>

Note: This will launch an EC2 instance out of the specified ami-id and with the SSH keypair. Knife will proceed towards knife bootstrap once the instance is launched. To do so, you need to make sure the SSH keypair's private key is added to your ssh-agent session before initiating a knife ec2 server create. You might also want to specify your Chef server URL with --server-url option of knife ec2 server create.

Solution 2

If you want to store the settings in the knife.rb file:

knife[:aws_ssh_key_id] = 'pemfilename'

pemfilename should be the pem file name without the .pem extension and it has to be located in: ~/.ssh/ Also, it has to be chmodded 600: (chmod 600 ~/.ssh/pemfile.pem)

So if your pem file is ~/.ssh/mypem.pem you need to set:

knife[:aws_ssh_key_id] = 'mypem'

Then the access key and the secret accees keys:

knife[:aws_access_key_id] = 'AAAAAAAAAAAAAAAAAAAA'
knife[:aws_secret_access_key] = 'UYUYW/IUYITYUIGGUGIUYGKJHGHG'

After you save the file, you can run:

knife ec2 server create --node-name mynodename.mydomain.com --run-list "role[myrole]"

Solution 3

As rhetonik mentioned above - you need to specify keypair name to be used for a new server. One option is to pass it directly using -S switch to knife ec2 command, the other option is to add something like that to your knife.rb configuration file:

knife[:aws_ssh_key_id] = "myawshosts"

Solution 4

In addition to the other answers. The aws_ssh_key_id must be one of the 'key pair names' registered with your AWS account. This usually corresponds with your ssh key file name, but it can differ.

You can see these aws ec2 'Key pair name' values in the EC2 'Network & Security -> Key Pairs' screen.

If you are using AWS CLI, you can also look at the "KeyName" values when running aws ec2 describe-key-pairs

View more solutions

Comments

[Guus L]

Could you confirm if you get an error when you try to set the security credentials in knife.rb or when you execute a knife ec2 create command? In case of the latter, could you update your question with the exact command that was executed?

[Jason Floyd]

Keypairs are used for ssh authentication, which is different than your aws access key and secret key. You can find those under the Security Credentials for your account, but I recommend you create an IAM user and grant it specific permissions instead of using the root login. The IAM user will have their own set of access & secret keys to use.

[poseid]

very interesting to create an IAM user. I am just trying, thanks

[poseid]

Hmm.. now I get: ERROR: Fog::Compute::AWS::NotFound: The key pair '/Users/pmu/.chef/new_test.pem' does not exist

[poseid]

Also, the format for the AWS keys in the knife file was really knife[:...] = 'key' for those who are confused by the knife format rules too

[jmreicha]

This worked for me. Just a caveat to add - [:aws_secret_access_key] is located at the top right of the console under username -> security credentials.