S3 - Anonymous Upload - Key prefix

Question

I am trying to understand exactly how to setup a bucket that is generally private but allows anonymous uploads with restrictions. The specific criteria are:

• The bucket is mostly private and requires my key/secret to add/remove/update/list files.
• There is a "directory" (i.e. key prefix) called "incoming" that will allow anonymous users to upload content to but not list.
• The bucket has a one day expiration on all content. As a bonus I would like the "incoming" directory to have a 30 minute expiration although if that is not possible a one day expiration for the whole bucket will do.
• Files with the "incoming" prefix would be limited in size per object.
• I might want to also limit objects with the "incoming" prefix to only certain content types.

Questions I have are:

1. Would it be better to simply create two buckets. One for my incoming files and one for my own personal processing and storage?
2. What would the code look like that for a file to be uploaded into the incoming directory. Ideally I would like to avoid a dependency on a S3 library and just use HTTP calls. Bonus points if you can show me the right direction on this in Ruby. :)

The expiration seems settable via the S3 Management Console but is only limited to 1 day as the smallest expiration. Can I put a decimal in that field? Permissions seem to apply to an entire bucket instead of just a prefix. This is making me think I just need two buckets. If I keep with one bucket I think I need to create an IAM policy and apply that to the bucket but it is beyond my limited knowledge of S3 and I want to ensure I don't leave a hole in the permissions that allow people to do more than I want them to.

I have found lots of documentation on doing anonymous uploads to S3 via a HTTP form post. I could adapt that into code but I am wondering since I am in application code (and not a HTTP form post) is there an easier way?

Solution 1

What you describe can be implemented within one bucket. You can allow anonymous access to specific folder via bucket policy, check examples or use AWS Policy Generator. In your case it could look something like this:

{
    "Version": "2008-10-17",
    "Id": "Policy1346097257207",
    "Statement": [
        {
            "Sid": "Allow anonymous upload to /incoming",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::[your_bucket]/incoming/*"
        }
    ]
}

It is also possible to upload files to your bucket anonymously using a simple html form:

<form action="http://[your_bucket].s3.amazonaws.com/" method="post" enctype="multipart/form-data">
    <input type="hidden" name="acl" value="public-read" />
    Name: <input type="text" name="key" value="incoming/[filename]" /><br/>
    File: <input type="file" name="file" /> <br />
    <input type="submit" name="submit" value="Upload" />
</form>​

S3 browser based uploads are described here in detail.

Solution 2

I recently spent a bit of time figuring out the ins and outs of anonymous uploads to S3, and came across this question as well. I wrote about the solution that worked for ME in some length at:

https://gist.github.com/jareware/d7a817a08e9eae51a7ea

Basically you can achieve what you want to, except that authenticated requests for management won't work (or at least I'm not aware of a solution).

I know this is an older question but just documenting it here in case it helps someone else.

Comments

[rb512]

for 2, have you considered configuring s3 with paperclip of carrierwave?

[Eric Anderson]

Thanks for the suggestion. Yes, I have used those before although I think they generally all operate on an authenticated basis (not anonymous). There might be some add-ons to do anonymous but I am really just hoping since S3 is a REST interface and I am allowing anonymous uploads that a simple HTTP request without any 3rd party library could do the trick. But trying to parse the Amazon documentation has been more of a chore than I hoped it would be so would love if someone could point me in the right direction.

[rb512]

Didn't quite get it. By anonymous do you mean you want to be able to upload files without providing your s3 credentials? I don't think that's possible.

[Eric Anderson]

Yes. I want to set the permissions so that "Everybody" can write but only to a specific prefix. I have seen where this is possible via a HTML form post using some special hidden fields. But I didn't know if there was something easier if I am not doing it via a HTML form but instead able to just make a HTTP request.

[Eric Anderson]

I haven't checked this all to see if it works like I want but the bounty is ending soon and I didn't want you to not get the points if you did have it all right. Looks good and will do some testing soon. Thanks!

[Dave Gregory]

Note that anonymously-uploaded objects are owned by "anonymous," which means: 1) anonymous access is granted to them and 2) the bucket-owning account does not have access to them (so IAM policies might not apply). Resolutions at aws.amazon.com/premiumsupport/knowledge-center/…