How to list the files in S3 subdirectory using Python

Question

I'm trying to list the files under sub-directory in S3 but I'm not able to list the files name:

import boto
from boto.s3.connection import S3Connection
access=''
secret=''
conn=S3Connection(access,secret)
bucket1=conn.get_bucket('bucket-name')
prefix='sub -directory -path'
print bucket1.list(prefix) 
files_list=bucket1.list(prefix,delimiter='/') 
print files_list
for files in files_list:
  print files.name

Can you please help me to resolve this issue.

Solution 1

Your code can be fixed by adding a / at the end of the Prefix.

The modern equivalent using boto3 would be:

import boto3
s3 = boto3.resource('s3')

## Bucket to use
bucket = s3.Bucket('my-bucket')

## List objects within a given prefix
for obj in bucket.objects.filter(Delimiter='/', Prefix='fruit/'):
    print(obj.key)

Output:

fruit/apple.txt
fruit/banana.txt

Rather than using the S3 client, this code uses the S3 object provided by boto3, which makes some code simpler.

Solution 2

You can do this by using boto3. Listing out all the files.

import boto3
s3 = boto3.resource('s3')
bucket = s3.Bucket('bucket-name')
objs = list(bucket.objects.filter(Prefix='sub -directory -path'))
for i in range(0, len(objs)):
    print(objs[i].key)

This piece of code will print all the files with path present in the sub-directory