"Wrapper" ssh script freezes when executing remote command

Question

I am running OSX Mountain Lion and the remote server is CentOS 6. I have several scripts on my local machine that I want to run on the remote server and to that end, I thought to write one script that connects via ssh that takes a the a second script to execute as a parameter. This way, I can write bash scripts naturally without having to worry about ssh and all the concerns that come with that, and the same scripts would also be able to be repurposed on my local machine. For instance,

exec_remote.sh

!#/bin/bash
ssh -t -t jeff@remoteserver 'bash -s' < $1

make_dir.sh (note the sudo)

!#/bin/bash
sudo mkdir -p /some/path/to/a/new/location

And the idea being to run as

./exec_remote.sh make_dir.sh

I've run into many problems, most of which I have been able overcome. However, now when I execute the scripts as above, the terminal hangs for several minutes, the remote directory is not created and there is no error message.

Moreover, since I haven't called logout, I still have the ssh terminal open and all subsequent commands (i.e. ls) are completely frozen. I let an ls run for an 70 minutes before breaking it. Note: I dont actually care about running commands afterwards; I merely include this as a symptom of what is wrong.

How do I correct this? Or better yet, is there a better way to approach this problem?

EDIT

If I update my make_dir.sh script to create a directory that doesnt require sudo, the script executes perfectly normally, as expected. Is there a known issue with running sudo commands remotely? Perhaps just with CentOS?

Solution 1

Try running the script remotely in the background redirecting std out and std err to a log file. In order for your shell to return (i.e. not "get locked for a long time while script runs") you will need to use nohup as well as redirect all pipes: StdOut, StdIn, StdErr. For example:

ssh jeff@remoteserver 'nohup make_dir.sh >> /tmp/log_file 2>&1 </dev/null &'

The command above will log you into the remote server and run make_dir.sh in the background while redirecting StdOut and StdErr to /tmp/log_file. StdIn is redirected from /dev/null. Otherwise, ssh will hang until the script completes.

You mentioned the script takes a long while to finish. running the script in this manner should allow you to login again and examine what is going on further.

Solution 2

Your way of executing ssh has a fundamental flaw, which is: you only have one stdin. Since you're reading the script using <, the stdin is "filled" with the script content. And merged with your input (in this case, your password). Which cannot work.

So the only way I see for this to work is: you have to copy the script to the remote server and then start a ssh command which just executes that script. So something along those lines:

#!/bin/bash
USERHOST=jeff@remoteserver

ssh -o ControlMaster=yes -o ControlPersist=3600 -o ControlPath=/tmp/ssh-master-$USER-%r@%h:%p -Nf $USERHOST
trap "ssh -o ControlPath=/tmp/ssh-master-$USER-%r@%h:%p $USERHOST -O exit" exit

TEXEC=$(ssh $USERHOST mktemp)
scp $1 ${USERHOST}:${TEXEC}
ssh -t -t $USERHOST $TEXEC

This script opens a shareable connection to the remote server (so you only have to enter you SSH password once), then copies your shell script over and finally executes it using the normal ssh command, which allows it to allocate a TTY, and thus allows your password to be entered for sudo in a (relatively) secure way. At the end of the wrapper script, the shared connection is closed via the exit trap.

Solution 3

Just do

ssh -t -t jeff@remoteserver $1

to run the command (make sure it's executable) then exit.

bash -s with leave the bash session open.

You probably want $* instead of $1 so you can pass arguments to the script you are calling

Comments

[parkydr]

Are those the actual scripts? !# should be #! What do you see if you change the first line of make_dir.sh to #!/bin/bash -x

[Jeff]

Sorry, no. My scripts though very similar, did have the correct order for #!. Adding the -x flag did not appear to have any effect.

[parkydr]

Could you see the sudo line being called?

[Jeff]

Yes. The sudo line is called. That is where it hangs for about 5 minute. Afterwards, there is no output and the prompt returns but not directory was created.

[slm]

How are you providing your password to sudo on the remote system? Are you using a technique similar to this one? askubuntu.com/questions/155791/…

[Jeff]

No, I was typing it in when prompted. I'll give this a shot.

[John Siu]

Did you try running all scripts(eg. make_dir.sh) on the remote machine manually? Maybe they need some debugging.

[Jeff]

All scripts are on my local machine. If I copy the scripts over to the server and manually ssh into the server, the scripts work perfectly. I have editted my post to clarify.

[Shane Hsu]

It sounds more like a problem with sudo, but I could always do that with CentOS.

[Shane Hsu]

Have you took a look at the target machine's system logs? Maybe an error is present.

[Jeff]

Thanks parkdyr, but that only solves the session remaining open which wasnt really the issue I was concerned with. I am more concerned with the long execution time and the fact that no directory was created.

[Stefan Seidel]

cat $1 | ssh jeff@remoteserver "bash " is the same as ssh jeff@remoteserver "bash " < $1 except for the UUOC (useless use of cat).