AWS S3 Java SDK - Download file help

Question

The code below only works for downloading text files from a bucket in S3. This does not work for an image. Is there an easier way to manage downloads/types using the AWS SDK? The example included in the documentation does not make it apparent. Thanks!

AWSCredentials myCredentials = new BasicAWSCredentials(
       String.valueOf(Constants.act), String.valueOf(Constants.sk)); 
AmazonS3Client s3Client = new AmazonS3Client(myCredentials);        
S3Object object = s3Client.getObject(new GetObjectRequest("bucket", "file"));

BufferedReader reader = new BufferedReader(new InputStreamReader(
       object.getObjectContent()));
File file = new File("localFilename");      
Writer writer = new OutputStreamWriter(new FileOutputStream(file));

while (true) {          
     String line = reader.readLine();           
     if (line == null)
          break;            

     writer.write(line + "\n");
}

writer.close();

Solution 1

Instead of Reader and Writer classes you should be using InputStream and OutputStream classes:

InputStream reader = new BufferedInputStream(
   object.getObjectContent());
File file = new File("localFilename");      
OutputStream writer = new BufferedOutputStream(new FileOutputStream(file));

int read = -1;

while ( ( read = reader.read() ) != -1 ) {
    writer.write(read);
}

writer.flush();
writer.close();
reader.close();

Solution 2

Though the code written in Mauricio's answer will work - and his point about streams is of course correct - Amazon offers a quicker way to save files in their SDK. I don't know if it wasn't available in 2011 or not, but it is now.

AmazonS3Client s3Client = new AmazonS3Client(myCredentials);

File localFile = new File("localFilename");

ObjectMetadata object = s3Client.getObject(new GetObjectRequest("bucket", "s3FileName"), localFile);

Solution 3

Eyals answer gets you half way there but its not all that clear so I will clarify.

AmazonS3Client s3Client = new AmazonS3Client(myCredentials);

//This is where the downloaded file will be saved
File localFile = new File("localFilename");

//This returns an ObjectMetadata file but you don't have to use this if you don't want 
s3Client.getObject(new GetObjectRequest(bucketName, id.getId()), localFile);

//Now your file will have your image saved 
boolean success = localFile.exists() && localFile.canRead(); 

Solution 4

There is even much simpler way to get this. I used below snippet. Got reference from http://docs.ceph.com/docs/mimic/radosgw/s3/java/

AmazonS3 s3client = AmazonS3ClientBuilder.standard()
                .withCredentials(new AWSStaticCredentialsProvider(credentials)).withRegion(Regions.US_EAST_1).build();

s3client.getObject(
                new GetObjectRequest("nomad-prop-pics", "Documents/1.pdf"),
                new File("D:\\Eka-Contract-Physicals-Dev\\Contracts-Physicals\\utility-service\\downlods\\1.pdf")
    );

Solution 5

Use java.nio.file.Files to copy S3Object to local file.

public File getFile(String fileName) throws Exception {
    if (StringUtils.isEmpty(fileName)) {
        throw new Exception("file name can not be empty");
    }
    S3Object s3Object = amazonS3.getObject("bucketname", fileName);
    if (s3Object == null) {
        throw new Exception("Object not found");
    }
    File file = new File("you file path");
    Files.copy(s3Object.getObjectContent(), file.toPath());
    return file;
}

View more solutions

Comments

[dvs]

Needs a minor edit: The object contents are saved to the file, but the returned object is ObjectMetadata, not S3Object. See docs.amazonwebservices.com/AWSJavaSDK/latest/javadoc/com/…

[Eyal]

@azdev Actually, the code sample uses a different version of getObject which does return a S3Object.

[Shikha Shah]

How come I cannot see the S3Client.getObject method in my code?..I want to achieve the same but there is no such method associated to my s3Client object.Am i missing something?

[Eyal]

@ShikhaShah The getObject method which returns a S3Object is from the AmazonS3 interface which is implemented by AmazonS3Client. Perhaps your IDE doesn't show the methods inherited from the interface?

[Shikha Shah]

But I am able to download the file and save it via AmazonS3Client only...but cant see this particular method which expedite the saving process..m still confused!!

[Eyal]

What version of the SDK are you using? I have a Maven dependency to aws-java-sdk, version 1.3.12.

[Teraiya Mayur]

what you have used for id.getId()) ?

[MartinS]

The id.getId() is the key you used to store the object in the bucket. I just use my id for the key as I know its unique. To summarise, you create an empty file, do a getRequest telling the aws api where the new empty file is and the aws api handles writing the object to the file you specified. So convenient.

[saurabheights]

@ShawnVader : Please can you clear why you are validating file. I though S3 api does a hash check. Thank you.

[MartinS]

@saurabheights You are right the above example of checking the file may return incorrect results as it is acceptable for the localFile in the above case to exist. Perhaps a better way is to ensure that the ObjectMetadata which returns is not null. Docs say > Returns All S3 object metadata for the specified object. Returns null if constraints were specified but not met. So if it has metadata then you can be confident that it's valid content in the file because looking through the AWS source I can see it validates the file integrity and will retry once before failing.

[saurabheights]

@ShawnVader: Edited the answer, please can you review it once it is peer reviewed.

[R-JANA]

This works.. Programs listed by aws documentation does not work.This worked very well. Thank you for posting

[Kramer]

Where does inputStream come from? I think you mean either s3Object.close or s3Object.getObjectContent.close(). Regardless, nio is a good suggestion.