S3: make a public folder private again?

Question

How do you make an AWS S3 public folder private again?

I was testing out some staging data, so I made the entire folder public within a bucket. I'd like to restrict its access again. So how do I make the folder private again?

Solution 1

From what I understand, the 'Make public' option in the managment console recursively adds a public grant for every object 'in' the directory. You can see this by right-clicking on one file, then click on 'Properties'. You then need to click on 'Permissions' and there should be a line:

 Grantee:  Everyone  [x] open/download  [] view permissions   [] edit permission.

If you upload a new file within this directory it won't have this public access set and therefore be private.

You need to remove public read permission one by one, either manually if you only have a few keys or by using a script.

I wrote a small script in Python with the 'boto' module to recursively remove the 'public read' attribute of all keys in a S3 folder:

#!/usr/bin/env python
#remove public read right for all keys within a directory

#usage: remove_public.py bucketName folderName

import sys
import boto3

BUCKET = sys.argv[1]
PATH = sys.argv[2]
s3client = boto3.client("s3")
paginator = s3client.get_paginator('list_objects_v2')
page_iterator = paginator.paginate(Bucket=BUCKET, Prefix=PATH)
for page in page_iterator:
    keys = page['Contents']
    for k in keys:
        response = s3client.put_object_acl(
                        ACL='private',
                        Bucket=BUCKET,
                        Key=k['Key']
                    )

I tested it in a folder with (only) 2 objects and it worked. If you have lots of keys it may take some time to complete and a parallel approach might be necessary.

Solution 2

The accepted answer works well - seems to set ACLs recursively on a given s3 path too. However, this can also be done more easily by a third-party tool called s3cmd - we use it heavily at my company and it seems to be fairly popular within the AWS community.

For example, suppose you had this kind of s3 bucket and dir structure: s3://mybucket.com/topleveldir/scripts/bootstrap/tmp/. Now suppose you had marked the entire scripts "directory" as public using the Amazon S3 console.

Now to make the entire scripts "directory-tree" recursively (i.e. including subdirectories and their files) private again:

s3cmd setacl --acl-private --recursive s3://mybucket.com/topleveldir/scripts/

It's also easy to make the scripts "directory-tree" recursively public again if you want:

s3cmd setacl --acl-public --recursive s3://mybucket.com/topleveldir/scripts/

You can also choose to set the permission/ACL only on a given s3 "directory" (i.e. non-recursively) by simply omitting --recursive in the above commands.

For s3cmd to work, you first have to provide your AWS access and secret keys to s3cmd via s3cmd --configure (see http://s3tools.org/s3cmd for more details).

Solution 3

For AWS CLI, it is fairly straight forward.

If the object is: s3://<bucket-name>/file.txt

For single object:

aws s3api put-object-acl --acl private --bucket <bucket-name> --key file.txt

For all objects in the bucket (bash one-liner):

aws s3 ls --recursive s3://<bucket-name> | cut -d' ' -f5- | awk '{print $NF}' | while read line; do
    echo "$line"
    aws s3api put-object-acl --acl private --bucket <bucket-name> --key "$line"
done

Solution 4

From the AWS S3 bucket listing (The AWS S3 UI), you can modify individual file's permissions after making either one file public manually or by making the whole folder content public (To clarify, I'm referring to a folder inside a bucket). To revert the public attribute back to private, you click on the file, then go to permissions and click in the radial button under "EVERYONE" heading. You get a second floating window where you can uncheck the *read object" attribute. Don't forget to save the change. If you try to access the link, you should get the typical "Access Denied" message. I have attached two screenshots. The first one shows the folder listing. Clicking the file and following the aforementioned procedure should show you the second screenshot, which shows the 4 steps. Notice that to modify multiple files, one would need to use the scripts as proposed in previous posts. -Kf

---

Solution 5

I actually used Amazon's UI following this guide http://aws.amazon.com/articles/5050/

View more solutions

Comments

[GoodGets]

You've answered my question, so I've accepted your answer. However, this sucks on Amazon's part. We have to write scripts to make things private again? Just terrible. ascobol, thank you for your help

[sumit]

Am using s3cmd 1.6.1 and this suggestion will not work if you already have 'private' on a folder. In our case we had public and private and setting to private via s3cmd, doesn't remove 'public'. Leaving a note here for other users. Here's the output with --verbose - "already Private, skipping"

[Joe Famme]

The instructions @ aws.amazon.com/articles/5050 appear to be out-of-date...from 2011. I do not have an option for going into "Properties" on my Bucket. The Python script mights be the only way now :(

[metanerd]

how would this look with regexy stuff like s3cmd --access_key $AWS_ACCESS_KEY_ID --secret_key $AWS_SECRET_ACCESS_KEY setacl --acl-private -r s3://$S3_BUCKET_NAME/*.map ?

[Luca Bezerra]

Your answer is great, but I kinda missed the explanation of what each tool is doing in the script, so that people don't just go blindly executing it. Also, there's an issue with filenames that contain spaces in it. I've posted a new answer with the changes below, but all credit still goes to you :)

[Admin]

I was able to get to the Permissions in a non-intuitive way. I browsed to the folder containing the object I cared about and selected the check box next to its name. This caused a pop-up box to slide in from the right. On this page there is a section called Permissions. I noticed that name was a link, and when I clicked it I got to the Permissions tab for the object.

[Mojtaba]

This tool is amazing. Thanks for sharing your knowledge

[he77kat_]

OP's question was regarding making a single folder private again, not the entire bucket

[star]

aws already provided another solution, please check aws.amazon.com/blogs/aws/…