Uploading PIL image object to Amazon s3 python

Question

I want to get an image from the web and upload it to Amazon s3. While doing so, I would like to check the image dimensions. I have the following code in Python 3:

from PIL import Image
import requests

# Get response
response = requests.get(url, stream= True)

# Open image
im = Image.open(response.raw)

# Get size
size = im.size

# Upload image to s3
S3.Client.upload_fileobj(
    im, # This is what i am trying to upload
    AWS_BUCKET_NAME,
    key,
    ExtraArgs={
        'ACL': 'public-read'
    }
)

The problem is that the PIL image object does not support read. I get the following error when i try to upload the PIL Image object im.

ValueError: Fileobj must implement read

It works when I just try to upload the 'response.raw', but I need to get the image dimensions. How can I change the PIL image object to a file-like-object? Is there an easier way to get the dimensions while still being able to upload the image to s3?

So the question is; how do I upload an image to s3 after getting the dimensions of an image?

Solution 1

Instead of calling read() to get the file contents back, you 'save' the file to either a real file object or an file like object in memory. Then call getValue() on it.

Here is an example function you could pass file content into, print out the height and width, then return the file data back in a format that an AWS clients put_object function will accept as the Body parameter.

from PIL import Image
import io

def modify_image(image, format):
    pil_image = Image.open(image)

    # Prints out (1280, 960) 
    print(pil_image.size)

    in_mem_file = io.BytesIO()

    # format here would be something like "JPEG". See below link for more info.
    pil_image.save(in_mem_file, format=format)
    return in_mem_file.getvalue()

There are also separate width and height attributes here: http://pillow.readthedocs.io/en/3.4.x/reference/Image.html#attributes

See more about the file formats here http://pillow.readthedocs.io/en/3.4.x/handbook/image-file-formats.html

Note: Example uses Python 3.6.1

Solution 2

You need to use a file-like object but you should not call getValue() contrary to the accepted answer. Using the following snippet, you can then upload the image to S3 using in_mem_file when calling upload_fileobj:

from PIL import Image
import io

# Open image
pil_image = Image.open(response.raw)

# Save the image to an in-memory file
in_mem_file = io.BytesIO()
pil_image.save(in_mem_file, format=pil_image.format)
in_mem_file.seek(0)

# Upload image to s3
client_s3.upload_fileobj(
    in_mem_file, # This is what i am trying to upload
    AWS_BUCKET_NAME,
    key,
    ExtraArgs={
        'ACL': 'public-read'
    }
)

The .seek(0) part is needed to rewind the file-like object if you see that the uploaded file is 0kB.

Comments

[john]

This does not change anything. The image opens fine. The problem occurs when i want to upload the image to s3.

[amarynets]

Did you try to load f into aws? Use f instead im

[john]

Yes, I tried that. Well first of all, response does not have an attribute called read, I am assuming you mean response.raw.read(). When I try to do that I upload an empty file_obj, because f has been emptied, when it is opened with Image.open().

[amarynets]

How about trying content attr from a response?

[john]

It still just uploads an empty picture. If i remove the Image.open(f) part all together it works fine.

[alphazwest]

Just a nag for the next guy: format=format should probably be format=pil_image.format

[Samuel Prevost]

This is plain useless, the requests doc states that If you set stream to True when making a request, Requests cannot release the connection back to the pool unless you consume all the data (i.e. calling reponse.content) or call Response.close. This can lead to inefficiency with connections. so by calling response.content you're defeating the purpose of asking for a stream in the first place as it means that you try reading the whole thing.

[hax]

This is the accurate answer. Thank you for providing this. I was trying the accepted answer and was kind of lost on what I did wrong. This really helped.

[Serhii Kushchenko]

'ACL': 'public-read' - requires s3:PutObjectAcl permission. Also, after manipulations with the file, the pil_image.format data may be lost. So it's good to save it somewhere immediately after opening the file.