Download S3 Files with Boto
Solution 1
The problem is that you are downloading to a local directory that doesn't exist (media/user1
). You need to either:
- Create the directory on the local machine first
- Just use the filename rather than a full path
- Use the full path, but replace slashes (
/
) with another character -- this will ensure uniqueness of filename without having to create directories
The last option could be achieved via:
k.get_contents_to_filename(str(k.key).replace('/', '_'))
See also: Boto3 to download all files from a S3 Bucket
Solution 2
Downloading files using boto3 is very simple, configure your AWS credentials at system level before using this code.
client = boto3.client('s3')
// if your bucket name is mybucket and the file path is test/abc.txt
// then the Bucket='mybucket' Prefix='test'
resp = client.list_objects_v2(Bucket="<your bucket name>", Prefix="<prefix of the s3 folder>")
for obj in resp['Contents']:
key = obj['Key']
//to read s3 file contents as String
response = client.get_object(Bucket="<your bucket name>",
Key=key)
print(response['Body'].read().decode('utf-8'))
//to download the file to local
client.download_file('<your bucket name>', key, key.replace('test',''))
replace is to locate the file in your local with s3 file name, if you don't replace it will try to save as 'test/abc.txt'.
Comments
-
pepper5319 almost 2 years
I am trying to set up an app where users can download their files stored in an S3 Bucket. I am able to set up my bucket, and get the correct file, but it won't download, giving me the this error:
No such file or directory: 'media/user_1/imageName.jpg'
Any idea why? This seems like a relatively easy problem, but I can't quite seem to get it. I can delete an image properly, so it is able to identify the correct image.Here's my views.py
def download(request, project_id=None): conn = S3Connection('AWS_BUCKET_KEY', 'AWS_SECRET_KEY') b = Bucket(conn, 'BUCKET_NAME') k = Key(b) instance = get_object_or_404(Project, id=project_id) k.key = 'media/'+str(instance.image) k.get_contents_to_filename(str(k.key)) return redirect("/dashboard/")
-
pepper5319 almost 8 yearsCould you possibly give me a simple example. I relatively new to Boto and AWS. If not, that's totally fine.
-
Roshan over 7 yearsThis answer should have more up vote. simplest solution
-
jkdev over 7 yearsYou can use
os.makedirs()
to create a directory, including its parent directories if needed. -
user1717828 about 5 yearsThis was very helpful, but the C-style comments in Python code are unnerving.