Get only file names from s3 bucket folder
18,091
Solution 1
You should use list_object_v2 which gives you the list from the defined prefix used.
... snippet ...
filenames = []
get_filenames(s3):
result = s3.list_objects_v2(Bucket=bucket, Prefix=prefix)
for item in result['Contents']:
files = item['Key']
print(files)
filenames.append(files) #optional if you have more filefolders to got through.
return filenames
get_filenames(my_bucketfolder)
Solution 2
Here you go.
import boto3
bucket = "Sample_Bucket"
folder = "Sample_Folder"
s3 = boto3.resource("s3")
s3_bucket = s3.Bucket(bucket)
files_in_s3 = [f.key.split(folder + "/")[1] for f in s3_bucket.objects.filter(Prefix=folder).all()]
Solution 3
for myself, i made a function that you might find helpful:
import boto3
s3_client = boto3.client('s3')
def list_objects_without_response_metadata(**kwargs):
ContinuationToken = None
while True:
if ContinuationToken:
kwargs["ContinuationToken"] = ContinuationToken
res = s3_client.list_objects_v2(**kwargs)
for obj in res["Contents"]:
yield obj
ContinuationToken = res.get("NextContinuationToken", None)
if not ContinuationToken:
break
file_names = [obj["Key"] for obj in list_objects_without_response_metadata(Bucket='Sample_Bucket', Prefix='Sample_Folder')]
Author by
TeeKay
Updated on June 22, 2022Comments
-
TeeKay almost 2 years
I have a s3 bucket named 'Sample_Bucket' in which there is a folder called 'Sample_Folder'. I need to get only the names of all the files in the folder 'Sample_Folder'.
I am using the following code to do so -
import boto3 s3 = boto3.resource('s3', region_name='us-east-1', verify=False) bucket = s3.Bucket('Sample_Bucket') for files in bucket.objects.filter(Prefix='Sample_Folder): print(files)
The variable files contain object variables which has the filename as key.
s3.ObjectSummary(bucket_name='Sample-Bucket', key='Sample_Folder/Sample_File.txt')
But I need only the filename. How do I extract that? Or is there any other way to do it?