AWS Error Message: InvalidInstanceID.NotFound

Question

I'm trying to start a Amazon EC2 cloud machine with [startInstance][2] method using aws-sdk in Java. My code is as follows.

public String startInstance(String instanceId) throws Exception {
    List<String> instanceIds = new ArrayList<String>();
    instanceIds.add(instanceId);

    StartInstancesRequest startRequest = new StartInstancesRequest(
            instanceIds);
    startRequest.setRequestCredentials(getCredentials());

    StartInstancesResult startResult = ec2.startInstances(startRequest);
    List<InstanceStateChange> stateChangeList = startResult
            .getStartingInstances();
    log.trace("Starting instance '{}':", instanceId);

    // Wait for the instance to be started
    return waitForTransitionCompletion(stateChangeList, "running",
            instanceId);
}

When I run the above code, i'm getting the following AWS error:

Status Code: 400, AWS Request ID: e1bd4795-a609-44d1-9e80-43611e80006b, AWS Erro
r Code: InvalidInstanceID.NotFound, AWS Error Message: The instance ID 'i-2b97ac
2f' does not exist
        at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpCli
ent.java:538)
        at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.ja
va:283)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:168
)
        at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.jav
a:5208)
        at com.amazonaws.services.ec2.AmazonEC2Client.startInstances(AmazonEC2Cl
ient.java:2426)

Solution 1

AWS Error Message: The instance ID 'i-2b97ac2f' does not exist

You'll have to take the AWS response for granted here, i.e. the instance does not exist ;)

But seriously: Presumably you have already verified that you are actually running an instance with this ID in your account? Then this is most likely caused by targeting the wrong API endpoint, insofar an instance ID is only valid within a specific region (if not specified, the region defaults to 'us-east-1', see below).

In this case you need to specify the actual instance region via the setEndpoint() method of the AmazonEC2Client object within the apparently global ec2 variable before calling startInstances().

There are some examples regarding Using Regions with the AWS SDKs and all currently available AWS regional endpoint URLs are listed in Regions and Endpoints, specifically the Amazon Elastic Compute Cloud (EC2) defaults to 'us-east-1':

If you just specify the general endpoint (ec2.amazonaws.com), Amazon EC2 directs your request to the us-east-1 endpoint.

Solution 2

We run a service (Qubole) that frequently spawns and then tags (and in some cases terminates) AWS instances immediately.

We have found that Amazon will, every once in a while, claim an instanceid as invalid - even though it has just created it. Retrying a few times with some sleep time thrown in usually solves the problem. Even a total retry interval of 15s proved insufficient in rare cases.

This experience comes from the useast region. We do not make api calls to different regions - so that is not an explanation. More likely - this is the infamous eventual consistency at work - where AWS is unable to provide read-after-write consistency for these api calls.

Solution 3

I am using the AWS ruby api and I noticed the same issue when creating an AMI image and its status is pending when I look in the AWS console but after a while the image is available for use.

Here is my script

image = ec2.images.create(:name => image_name, :instance_id => ami_id, :description => desc)

sleep 5 while image.state != :available

I sleep for about 5 sec for image to be in available but I get the error saying that the "AWS Error Message: InvalidInstanceID.NotFound". During my testing this is fine but most of the time this seems to be failing during continuous integration builds.

Comments

[Martijn]

I encountered the same problem twice in the eu-west-1.