AmazonS3ClientBuilder.defaultClient() fails to account for region?

Question

The Amazon Java SDK has marked the Constructors for AmazonS3Client deprecated in favor of some AmazonS3ClientBuilder.defaultClient(). Following the recommendation, though, does not result in an AmazonS3 client that works the same. In particular, the client has somehow failed to account for Region. If you run the tests below, the thisFails test demonstrates the problem.

public class S3HelperTest {
  @Test
  public void thisWorks() throws Exception {
    AmazonS3 s3Client = new AmazonS3Client();  // this call is deprecated
    s3Client.setS3ClientOptions(S3ClientOptions.builder().setPathStyleAccess(true).build());
    assertNotNull(s3Client);
  }

  @Test
  public void thisFails() throws Exception {
    AmazonS3 s3Client = AmazonS3ClientBuilder.defaultClient();
    /*
     * The following line throws like com.amazonaws.SdkClientException:
     * Unable to find a region via the region provider chain. Must provide an explicit region in the builder or
     * setup environment to supply a region.
     */
    s3Client.setS3ClientOptions(S3ClientOptions.builder().setPathStyleAccess(true).build());
  }
}

com.amazonaws.SdkClientException: Unable to find a region via the region provider chain. Must provide an explicit region in the builder or setup environment to supply a region.
    at com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:371)
    at com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:337)
    at com.amazonaws.client.builder.AwsSyncClientBuilder.build(AwsSyncClientBuilder.java:46)
    at com.amazonaws.services.s3.AmazonS3ClientBuilder.defaultClient(AmazonS3ClientBuilder.java:54)
    at com.climate.tenderfoot.service.S3HelperTest.thisFails(S3HelperTest.java:21)
    ...

Is this an AWS SDK Bug? Is there some "region default provider chain" or some mechanism to derive the region from the Environment and set it into the client? It seems really weak that the method to replace the deprecation doesn't result in the same capability.

Solution

Looks like a region is required for the builder. Probably this thread is related (I would use .withRegion(Regions.US_EAST_1) though in the 3rd line):

To emulate the previous behavior (no region configured), you'll need to also enable "forced global bucket access" in the client builder:

AmazonS3 client = 
        AmazonS3ClientBuilder.standard()
                             .withRegion("us-east-1") // The first region to try your request against
                             .withForceGlobalBucketAccess(true) // If a bucket is in a different region, try again in the correct region
                             .build();

This will suppress the exception you received and automatically retry the request under the region in the exception. It is made explicit in the builder so you are aware of this cross-region behavior. Note: The SDK will cache the bucket region after the first failure, so that every request against this bucket doesn't have to happen twice.

---

Also, from the AWS documentation if you want to use AmazonS3ClientBuilder.defaultClient(); then you need to have ~/.aws/credentials and ~/.aws/config files

~/.aws/credentials contents:

[default]
aws_access_key_id = your_id
aws_secret_access_key = your_key

~/.aws/config contents:

[default]
region = us-west-1

---

From the same AWS documentation page, if you don't want to hardcode the region/credentials, you can have it as environment variables in your Linux machine the usual way:

export AWS_ACCESS_KEY_ID=your_access_key_id
export AWS_SECRET_ACCESS_KEY=your_secret_access_key
export AWS_REGION=your_aws_region

Comments

[Bob Kuhar]

This answer is mostly correct, or pointed me in the right direction at least. If I just create that ~/.aws/config file and get the default region set up in there, then AmazonS3ClientBuilder.standard().withPathStyleAccessEnabled(‌​true) .build(); gets the job done

[Bob Kuhar]

Defining AWS_REGION in the environment gets it done too.

[Jakob]

Defining AWS_REGION in the environment does not work on windows it seems.