Only copy files from a particular date from s3 storage
7,805
That's because in the second aws s3 you use sync. Try cp instead. Also you can merge the "grep" and "awk" together.
$ aws s3 ls s3://cve-etherwan/ --recursive | awk '/^2018-11-06/{system("aws s3 cp s3://cve-etherwan/$4 .") }'
Related videos on Youtube
01 : 50
Unix & Linux: Only copy files from a particular date from s3 storage (2 Solutions!!)
15 : 13
AWS Lambda function | Copy files from one S3 bucket to another S3 bucket as soon as uploaded
09 : 31
AWS Tutorial: How to Copy Files From Your S3 Bucket to EC2 Instance
09 : 59
How To Copy (CP) AWS S3 Files Between Buckets
08 : 21
How to Restore Files From Different Storage Class in AWS S3
Author by
stdcerr
Coding mostly on embedded systems (24/7/365 uptime) to make a living at daytime, coding on a variety of systems (x86, arm) for fun at nighttime, preferences: Linux, C & C++. Always interested in learning other ways/better ways to solve problems!
Updated on September 18, 2022Comments
-
stdcerr almost 2 years
I would like to only copy files from S3 that are from today out of a certain bucket with 100s of files. I tried the following:
$ aws s3 ls s3://cve-etherwan/ --recursive --region=us-west-2 | grep 2018-11-06 | awk '{system("aws s3 sync s3://cve-etherwan/$4 . --region=us-west-2") }'but it doesn't quite work, I also get files from other dates. How do I do this correctly?