How to run the sftp command with a password from Bash script?

Question

I need to transfer a log file to a remote host using sftp from a Linux host. I have been provided credentials for the same from my operations group. However, since I don't have control over other host, I cannot generate and share RSA keys with the other host.

So is there a way to run the sftp command (with the username/password provided) from inside the Bash script through a cron job?

I found a similar Stack Overflow question, Specify password to sftp in a Bash script, but there was no satisfactory answer to my problem.

Solution 1

You have a few options other than using public key authentication:

1. Use keychain
2. Use sshpass (less secured but probably that meets your requirement)
3. Use expect (least secured and more coding needed)

If you decide to give sshpass a chance here is a working script snippet to do so:

export SSHPASS=your-password-here
sshpass -e sftp -oBatchMode=no -b - sftp-user@remote-host << !
   cd incoming
   put your-log-file.log
   bye
!

Solution 2

Another way would be to use lftp:

lftp sftp://user:password@host  -e "put local-file.name; bye"

The disadvantage of this method is that other users on the computer can read the password from tools like ps and that the password can become part of your shell history.

A more secure alternative which is available since LFTP 4.5.0 is setting the LFTP_PASSWORD environment variable and executing lftp with --env-password. Here's a full example:

export LFTP_PASSWORD="just_an_example"
lftp --env-password sftp://user@host  -e "put local-file.name; bye"

# Destroy password after use
export LFTP_PASSWORD=""

LFTP also includes a cool mirroring feature (can include delete after confirmed transfer --Remove-source-files):

lftp -e 'mirror -R /local/log/path/ /remote/path/' --env-password -u user sftp.foo.com

Solution 3

EXPECT is a great program to use.

On Ubuntu install it with:

sudo apt-get install expect

On a CentOS Machine install it with:

yum install expect

Lets say you want to make a connection to a sftp server and then upload a local file from your local machine to the remote sftp server

#!/usr/bin/expect

spawn sftp [email protected]
expect "password:"
send "yourpasswordhere\n"
expect "sftp>"
send "cd logdirectory\n"
expect "sftp>"
send "put /var/log/file.log\n"
expect "sftp>"
send "exit\n"
interact

This opens a sftp connection with your password to the server.

Then it goes to the directory where you want to upload your file, in this case "logdirectory"

This uploads a log file from the local directory found at /var/log/ with the files name being file.log to the "logdirectory" on the remote server

Solution 4

You can use lftp interactively in a shell script so the password not saved in .bash_history or similar by doing the following:

vi test_script.sh

Add the following to your file:

#!/bin/sh
HOST=<yourhostname>
USER=<someusername>
PASSWD=<yourpasswd>

cd <base directory for your put file>

lftp<<END_SCRIPT
open sftp://$HOST
user $USER $PASSWD
put local-file.name
bye
END_SCRIPT

And write/quit the vi editor after you edit the host, user, pass, and directory for your put file typing :wq .Then make your script executable chmod +x test_script.sh and execute it ./test_script.sh.

Solution 5

You can override by enabling Password less authentication. But you should install keys (pub, priv) before going for that.

Execute the following commands at local server.

Local $> ssh-keygen -t rsa 

Press ENTER for all options prompted. No values need to be typed.

Local $> cd .ssh
Local $> scp .ssh/id_rsa.pub user@targetmachine:
Prompts for pwd$>  ENTERPASSWORD

Connect to remote server using the following command

Local $> ssh user@targetmachine
Prompts for pwd$> ENTERPASSWORD

Execute the following commands at remote server

Remote $> mkdir .ssh
Remote $> chmod 700 .ssh
Remote $> cat id_rsa.pub >> .ssh/authorized_keys
Remote $> chmod 600 .ssh/authorized_keys
Remote $> exit

Execute the following command at local server to test password-less authentication. It should be connected without password.

$> ssh user@targetmachine

View more solutions

Comments

[anubhava]

For using it on Mac: stackoverflow.com/questions/9102557/…

[mtk]

Hi, I have similar problem, and I tried to use expect as other alternatives are not present on the host. With expect I am getting this error.. Permission denied (publickey,keyboard-interactive). . Please can you let me know, how to connect to a sftp host with passwd from a bash script.

[mtk]

Please see, have asked the question here.. stackoverflow.com/q/12508206/1135954

[anubhava]

Thanks for your answer, I had expect as one of my options for achieving this task.

[Jens]

No need to export, I think. SSHPASS=password sshpass -e ... should do.

[rezizter]

Thank you. I found your answer through a search, used it and built a script successfully with expect. Just posted the script for poterity

[anubhava]

@Jens: Agreed that export is not really required here.

[anubhava]

Thank you so much for you answer, I have already upvoted it :)

[anubhava]

+1 for you good suggestion. However this will require you to pass the password on command line and anybody on the system doing ps command would be able to see your password.

[Cloud Artisans]

I was able to do a one-liner to dowload a log file: sshpass -p "my_password" sftp -oPort=9999 user@host:dir/file.log

[anubhava]

@gorus: One-liner is very cool however I didn't want to use password on command line because of enhanced snooping risks.

[anubhava]

+1 for suggesting an alternative but can we avoid supplying password on command line?

[anubhava]

@Dru: Yes export SSHPASS=your-password-here goes into env of that user but how will ps -ef reveal env of a different user?

[Balaji Natarajan]

To overwrite existing file, add "set xfer:clobber on" after lftp<<END_SCRIPT

[anubhava]

+1 for your answer but question was about doing it without public/private keys.

[SriniV]

oops.. Just saw the restrictions on key sharing :)

[SriniV]

Out of curiosity will this command lftp -p ${port} -u ${login_id},${password} ${ip_number} when invoked from a shell script print them some where? How did you solve apart from sshpass?

[anubhava]

I don't have lftp and rely on sshpass from Mac and Linux both.

[richardkmiller]

-oBatchMode=no was the crucial missing piece for me.

[anubhava]

What did you use and what didn't work. Feel free to create a new question since this is very old thread.

[Teja]

Hi Anubhava - Here is my question. stackoverflow.com/questions/24540156/…

[Teja]

I am working on IBM AIX and trying to get a file from remote server using sftp command...

[Teja]

Hi Anubhava - Can you help me out regarding this script... I tried using except but it doesn't work. Here is my code I have written at below link. stackoverflow.com/questions/24540156/…

[Teja]

Hi Anubhava - Any luck?

[Teja]

Hi Anubhava - I tried all the three options but none of them worked for me....Can you please help me out... thank you...

[anubhava]

What error did you get while trying sshpass command? I use it every day on my system.

[anubhava]

+1 but I think you state it clearly what the problems are with this approach.

[Dejell]

I don't have root user on that machine to install new software. I am looking for a way to do it without installing new software

[anubhava]

I have installed it without root access but discussing that will be diverging too much from this question.

[Nikolay Tsenkov]

Pure gold. Thanks! :)

[spackmat]

Hi, sshpass is a great idea, thanks, works fine! But now, I get a mail with all SFTP-commands in the here script area every time this is run via crontab. So does anyone know how to suppress this output to the console?

[anubhava]

You can add > ~/file.out 2>&1 to redirect the output and stderr to a file.

[Aaron Digulla]

Downvoted for showing the password to anyone on the same computer

[obaranovsky]

You can create the script file for lftp, this way you will not have to provide the password in the command line. Create a file put-script: open sftp://user:password@host; put local-file.name; exit Than run lftp -f put-script This way you do not have to have the username and password in a command line and can set up restrictive permissions to your script file.

[e2a]

@anubhava very thanks for this answer. I used it with my bash script file that is start via crontab -e. I change the exclamation point "!" to "EOF" because crontab doens't recognied this character in script

[Admin]

@obaranovsky That is a good idea. I wonder if I were to add this to my environment variable as alias or a function, would it be secure and possible to hide it from other users?

[Admin]

I'm using this to enter the passphrase for ssh private key. It is handier to put same public key on all servers and just rotate the private key passphrase periodically.

[rao]

Tip: If you are setting this up for additional remote servers, copy the same id_rsa.pub to the remote servers. The public key id_rsa.pub is generated once on the local server and shared with all remote hosts with which password less authentication needs to be set up.

[code-8]

I tried that, I got unknown option -- - usage: sftp [-1246aCfpqrv] [-B buffer_size] [-b batchfile] [-c cipher] [-D sftp_server_path] [-F ssh_config] [-i identity_file] [-l limit] [-o ssh_option] [-P port] [-R num_requests] [-S program] [-s subsystem | sftp_server] host sftp [user@]host[:file ...] sftp [user@]host[:dir[/]] sftp -b batchfile [user@]host

[qris]

@ihue probably because you need Techia SSH, not OpenSSH

[MeanEYE]

Far easier to do lftp than mess around with sftp and sshpass. Good answer.

[Honza Zidek]

@AaronDigulla Upvoted because it is exactly what the OP asked for - "a way to run the sftp command (with the username / password provided) from inside the Bash script".

[Martin Prikryl]

The question is about using a password authentication, not a public key authentication.

[Rodrigo]

Yes, you are right, but can be useful for other ones looking for alternatives for their problems. Because the issue is related.

[Paxi1337]

May I ask what exactly does the ! do?

[anubhava]

! can be any marker string. You can use EOF or EOD as well.

[access_granted]

Just a note, it only ran for us on Solaris if we used: export LFTP_PASSWORD=${our_password}, would not work without the export clause.

[user1764359]

isn't this the opposite direction? this shows how to copy a file from the remote server to the local, when the question is to upload from local to remote.

[Yvan]

@AaronDigulla I've changed the script to reflect your thoughts. Thanks for pointing this out!

[Dan Garthwaite]

I've forgotten how often I used to use lftp [every day] and this answer completely solved an issue a client had. Also respects here docs for commands. $ lftp sftp://[email protected] << EOF ls cd foo get bar EOF