AWS CLI for searching a file in s3 bucket

Question

I want to search for a file name abc.zip in s3 buckets and there are nearly 60 buckets and each buckets have 2 to 3 levels subdirectories or folders .I tried to perform search using AWS CLI commands and below are the commands which i tried but even though the file is existing in the bucket.The results are not being displayed for the file.

aws s3api list-objects --bucket bucketname --region ca-central-1 \
    --recursive --query "Contents[?contains(Key, 'abc.zip')]"

aws s3 ls --summarize --human-readable --recursive bucketname \
    --region ca-central-1 | egrep 'abc.zip'

For all the above commands execution i dont see the filename in command line and when i manually check the bucket the file exists. Is there any way i can find the file.

Solution 1

Hmm. I used your command from #1 without "--recursive" because this throws Unknown options: --recursive. The file I was searching for is on the second level of the bucket and it was found. --region is also not used.

My guess is you are using some old version of AWS client or pointing to an incorrect bucket. My working command:

aws s3api list-objects --bucket XXXXX --query "Contents[?contains(Key, 'animate.css')]"

[
{
    "LastModified": "2015-06-14T23:29:03.000Z",
    "ETag": "\"e5612f9c5bc799b8b129e9200574dfd2\"",
    "StorageClass": "STANDARD",
    "Key": "css/animate.css",
    "Owner": {
        "DisplayName": "XXXX",
        "ID": "XXXX"
    },
    "Size": 78032
}
]

If you decide to upgrade your CLI client: https://github.com/aws/aws-cli/tree/master Current version is awscli-1.15.77 which you may check by aws --version.

Solution 2

I tried in the following way

aws s3 ls s3://Bucket1/folder1/2019/ --recursive |grep filename.csv

This outputs the actual path where the file exists

2019-04-05 01:18:35     111111 folder1/2019/03/20/filename.csv

Hope this helps!

Solution 3

I know this is ancient, but I found a way to do this without piping text to grep...

aws s3api list-objects-v2 --bucket myBucket --prefix 'myFolder' \
--query "Contents[*]|[?ends_with(Key,'jpg')].[Key]"

Comments

[jacoor]

@aish-mahesh if the answer works for you could you make it accepted answer? Also please share information on how you solved your issue.

[Aish Mahesh]

Thanks for the response it worked for me after upgrading the version and executing the same command and as you said --region was not required. Thanks for the response.

[Gonza]

Ancient or not, you helped a man under desperation. Thanks