How to copy a file to a remote server in Python using SCP or SSH?
Solution 1
You can call the scp
bash command (it copies files over SSH) with subprocess.run
:
import subprocess
subprocess.run(["scp", FILE, "USER@SERVER:PATH"])
#e.g. subprocess.run(["scp", "foo.bar", "[email protected]:/path/to/foo.bar"])
If you're creating the file that you want to send in the same Python program, you'll want to call subprocess.run
command outside the with
block you're using to open the file (or call .close()
on the file first if you're not using a with
block), so you know it's flushed to disk from Python.
You need to generate (on the source machine) and install (on the destination machine) an ssh key beforehand so that the scp automatically gets authenticated with your public ssh key (in other words, so your script doesn't ask for a password).
Solution 2
To do this in Python (i.e. not wrapping scp through subprocess.Popen or similar) with the Paramiko library, you would do something like this:
import os
import paramiko
ssh = paramiko.SSHClient()
ssh.load_host_keys(os.path.expanduser(os.path.join("~", ".ssh", "known_hosts")))
ssh.connect(server, username=username, password=password)
sftp = ssh.open_sftp()
sftp.put(localpath, remotepath)
sftp.close()
ssh.close()
(You would probably want to deal with unknown hosts, errors, creating any directories necessary, and so on).
Solution 3
You'd probably use the subprocess module. Something like this:
import subprocess
p = subprocess.Popen(["scp", myfile, destination])
sts = os.waitpid(p.pid, 0)
Where destination
is probably of the form user@remotehost:remotepath
. Thanks to
@Charles Duffy for pointing out the weakness in my original answer, which used a single string argument to specify the scp operation shell=True
- that wouldn't handle whitespace in paths.
The module documentation has examples of error checking that you may want to perform in conjunction with this operation.
Ensure that you've set up proper credentials so that you can perform an unattended, passwordless scp between the machines. There is a stackoverflow question for this already.
Solution 4
Reached the same problem, but instead of "hacking" or emulating command line:
Found this answer here.
from paramiko import SSHClient
from scp import SCPClient
ssh = SSHClient()
ssh.load_system_host_keys()
ssh.connect('example.com')
with SCPClient(ssh.get_transport()) as scp:
scp.put('test.txt', 'test2.txt')
scp.get('test2.txt')
Solution 5
You can do something like this, to handle the host key checking as well
import os
os.system("sshpass -p password scp -o StrictHostKeyChecking=no local_file_path username@hostname:remote_path")
M07
Updated on October 29, 2021Comments
-
M07 over 2 years
I have a text file on my local machine that is generated by a daily Python script run in cron.
I would like to add a bit of code to have that file sent securely to my server over SSH.
-
Charles Duffy over 15 yearsUsing subprocess.Popen is the Right Thing. Passing it a string rather than an array (and using shell=True) is the Wrong Thing, as it means filenames with spaces don't work correctly.
-
searchingpruegel over 14 yearsparamiko has a nice sftp.put(self, localpath, remotepath, callback=None) function too, so you don't have to open write, and close each file.
-
db42 over 12 yearsinstead of "sts = os.waitpid(p.pid, 0)", we can use "sts = p.wait()".
-
Drahkar over 11 yearsI would note that SFTP is not the same thing as SCP.
-
whyoz over 11 years@Drahkar the question asks for the file to be sent over SSH. That's what this does.
-
doda over 11 yearsNote: to make this work out of the box add
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
after instantiatingssh
. -
Tom over 10 years@doda only if you don't want to use the known_hosts file
-
László Papp over 10 yearsAny idea for an "scp" based solution? SFTP does not work with our server.
-
László Papp over 10 yearsis there an execution free way of it with direct python API for this protocol?
-
jfs about 10 years@CharlesDuffy:
subprocess.check_call(['scp', srcfile, dest])
could be used since Python 2.5 instead ofrc = Popen(..).wait(); if rc != 0: raise ..
-
jfs about 10 years
subprocess.check_call(['scp', myfile, destination])
could be used instead since Python 2.5 (2006) -
László Papp about 10 years@J.F.Sebastian: yep, I checked it out in December. My upvotes prove that, at least. :) Thanks for the follow-up though.
-
orezvani over 9 yearsadding ssh key is not a good solution when it is not needed. For example, if you need a one time communication, you don't set up ssh key due to security reasons.
-
Aysennoussi over 9 yearsGuys, is it safe to use the server password ? is there a possibility to use the Private Key ?
-
Joel B about 9 yearsIt's certainly possible to use a private key: gist.github.com/batok/2352501
-
Chuck over 6 yearsNote - this relies on the scp package. As of Dec 2017, the latest update to that package was in 2015 and the version number is 0.1.x. Not sure I would want to add this dependency.
-
Adriano_Pinaffo almost 6 yearsso, it´s 2018 now and in may they released version 0.11.0. So, it seems SCPClient is not dead after all?
-
WinEunuuchs2Unix over 3 yearsInteresting that this answer has almost three-fold more votes than accepted answer.
-
volkit almost 3 yearsthe scp module is regulary updated (last was in June 2021) and has 380 stars on github. this is a legit solution.
-
Can H. Tartanoglu over 2 years@WinEunuuchs2Unix why?
-
WinEunuuchs2Unix over 2 years@CanH.Tartanoglu I guess it's called individualism. I find something interesting that you do not. You can find something interesting that I do not.
-
Can H. Tartanoglu over 2 years@WinEunuuchs2Unix I was just curious why you found it interesting. You don't have to defend yourself, everyone is entitled to an opinion. I just wanted to know because I didn't find it interesting. Cheers.