"The specified key does not exist" S3 error for really existing object in the bucket

Question

In our application we have to download on backend by AWS Java SDK uploaded user's image for some processing operations(resize,crop,etc.) Sometimes, we get the following error:

com.amazonaws.services.s3.model.AmazonS3Exception: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey;

but this key and the object which was saved by this path exist. I know that in AWS developer guide this behavior is expected:

However, information about the changes might not immediately replicate across Amazon S3 and you might observe the following behaviors: A process writes a new object to Amazon S3 and immediately attempts to read it. Until the change is fully propagated, Amazon S3 might report "key does not exist."

but how can I process this error in my code? I tried to wait some milliseconds,I tried to retry download this object - and all my attempts are failed.

         try
            {
                Download download = s3TransferManager
                        .download(new GetObjectRequest(bucketName, key), new File(tempUrl));
                download.waitForCompletion();
            }
            catch (AmazonS3Exception amazonS3Exception)
            {
                Thread.sleep(1000);
                //retry 3 time.... }

I would be glad to hear any advice how to download existing file in that case. Thank you!

Solution 1

The answer really depends on the region you are using. From S3 FAQ (here http://aws.amazon.com/s3/faqs/):

Q: What data consistency model does Amazon S3 employ?

Amazon S3 buckets in all Regions provide read-after-write consistency for PUTS of new objects and eventual consistency for overwrite PUTS and DELETES. Amazon S3 buckets in the US Standard Region only provide read-after-write consistency when accessed through the Northern Virginia endpoint (s3-external-1.amazonaws.com).

If using US Standard and don't specify an endpoint you might experience large delays in edge cases between the put and when the object is available (anecdotally I've observed delays that are measured in hours). The pattern to follow is to execute the put and after that to spin and wait for the object.

The immediate fix is to use the Virginia endpoint (per FAQ) is case of US Standard or to move away from US standard and use another region (for example US-West-2). All other regions have read-after-write so the object will become available once the put is completed.

Solution 2

If your image processing is a background job (async job), you can use S3 Event Notification. So, whenever your image are has just put on a bucket, S3 can trigger a SNS notification/SQS or call AWS Lambda function.