Transfer files using SFTP and either PHP or shell/terminal script

Question

I need to write a script that is run as a cron job every night which transfers some report files via sftp to another server.

The report files are created every night using another cron in the format 'support_[date].csv' & 'download_[date].csv'.

I'm wondering if you had any pointers on how to do the following:

1. Find the 2 files created on latest [date]
2. Copy these files to another server using SFTP

I've tried several PHP scripts utilising the ssh2 extension, but to no avail. Is there a way to do it using a shell script? It's not something I am hugely familiar with to be honest (hence going down the PHP route initially)

This was one of my PHP scripts which didn't work:

$src = 'test.csv';

$filename = 'test.csv';
$dest = '/destination_directory_on_server/'.$filename;

$connection = ssh2_connect('example.com', 22);
ssh2_auth_password($connection, 'username', 'password');

// Create SFTP session
$sftp = ssh2_sftp($connection);

$sftpStream = fopen('ssh2.sftp://'.$sftp.$dest, 'w');

try {
       
            if (!$sftpStream) {
                throw new Exception("Could not open remote file: $dest<br>");
            }
           
            $data_to_send = file_get_contents($src);
           
            if ($data_to_send === false) {
                throw new Exception("Could not open local file: $src.<br>");
            }
           
            if (fwrite($sftpStream, $data_to_send) === false) {
                throw new Exception("Could not send data from file: $src.<br>");
            } else {
                //Upload was successful, post-upload actions go here...
            }
           
            fclose($sftpStream);
                           
        } catch (Exception $e) {
           
            //error_log('Exception: ' . $e->getMessage());
           echo 'Exception: ' . $e->getMessage();
            if($sftpStream) {fclose($sftpStream);}
            
        }

This were the error messages I got:

Warning: fopen() [function.fopen]: URL file-access is disabled in the server configuration in /path_to_script/sftp-test.php on line 17

Warning: fopen(ssh2.sftp://Resource id

3/destination_directory_on_server/test.csv)

[function.fopen]: failed to open stream: no suitable wrapper could be found in /path_to_script/sftp-test.php on line 17 Exception: Could not open remote file: /destination_directory_on_server/test.csv

Solution 1

Muchos kudos to ghostdog74! Managed to get this working, but with sftp.

First I managed to set up key authentication, then partly using ghostdog74's script I did this and it worked perfectly!

cd /directorywithfilesin
latest_download=$(ls -1tr download* | tail -1)
latest_support=$(ls -1tr support* | tail -1)
sftp [email protected] <<EOF
cd /dir_to_copy_to
put $latest_download
put $latest_support
EOF

Thanks!

Solution 2

using the terminal to find latest date of your file, you can use ls -1tr . Then use scp (not sftp) to copy/transfer files over example,

#!/bin/bash
latest_download=$(ls -1tr download*csv | tail -1)
latest_support=$(ls -1tr support*csv | tail -1)
scp $latest_download [email protected]:somedir  # syntax from memory, check man page for correct syntax
scp $latest_support [email protected]:somedir

check the man page of scp for usage