Copy files that are created today without FIND command and SFTP to another server

Question

I have a directory with 'n' number of files in it. I am trying to write a shell script which will list only the files created on the particular day the script is running, and will use sftp to move the files listed for the day to another server. I don't want to use find.

I tried using

ls -l  --time-style=+%D | grep $(date +%D) | grep -v '^d'

to list the files that are created for today. How do I copy the listed files to another directory, so that I can sftp the directory to another server. I tried the below command, but no use

ls -l  --time-style=+%D | grep $(date +%D) | grep -v '^d' > /home/oracle/SABARISH/logs/files/

And how do I sftp using shell script, I have tried sftp in command line, but is different when used in script.

Solution 1

You can do something like this:

files=$( ls -l --time-style=+%D | grep $(date +%D) | grep -v '^d' | awk '{print $NF}' ) ; for f in $files ; do cp -rf $f /home/oracle/SABARISH/logs/files/ ; done ; sftp {user}@{host}:{remote_dir} <<< 'put /home/oracle/SABARISH/logs/files/*'

or similarly:

for f in $(ls -l --time-style=+%D | grep $(date +%D) | grep -v '^d' | awk '{print $NF}') ; cp -rf $f /home/oracle/SABARISH/logs/files/ ; done ; sftp {user}@{host}:{remote_dir} <<< 'put /home/oracle/SABARISH/logs/files/*'

Awk is used to extract file names.

#!/bin/bash
FILES=$(ls -l test1_*.txt --time-style=+%D | grep ${DATE} | grep -v '^d' | awk '{p rint $NF}' )

if [ -n "${FILES}" ]
then
    for f in ${FILES}
    do
        cp -prf ${f} ${DESTINATION}
    done
    sftp ${USER}@${HOST}:${CD} <<< 'put /home/oracle/SABARISH/logs/sftp/*'
else
    echo "NO FILES TO MOVE"
fi

Solution 2

I recommend to use zsh shell for this job:

cp *(m-1) /home/oracle/SABARISH/logs/files/

where (m-1) is so called glob qualifier.

In this case we select all (*) files modified (m) within (-) last (1) day.

Comments

[infixed]

No time to actually figure out the details for you, but you can use tar --after-date=DATE to send a tar over stdout. pipe that through ssh, and untar on the other side

[Michael]

It is working awesome now. Any idea, how to SFTP that directory to another server in the same script?

[tuzion]

You can add sftp {user}@{host}:{remote_dir} <<< $'put /home/oracle/SABARISH/logs/files/* to the end of command. See here for more info on single line sftp: stackoverflow.com/questions/16721891/…

[Michael]

FILES=$(ls -l test1_*.txt --time-style=+%D | grep ${DATE} | grep -v '^d' | awk '{p rint $NF}' ) if [ -n "${FILES}" ] then for f in ${FILES} do cp -prf ${f} ${DESTINATION} done sftp ${USER}@${HOST}:${CD} <<< $'put /home/oracle/SABARISH/logs/sftp/*' else echo "NO FILES TO MOVE" fi I have implemented the above code, but after that <<< it is showing in red color, is there anyway to fix it. But script is working fine.

[tuzion]

Please use a script when you have many commands to make it easier for yourself and others to read and maintain!

[Michael]

That was a script block though. I just wanted to learn what <<< does!