Run a nohup command over SSH, then disconnect

Question

I want to execute a script, start.sh on a remote server which runs this:

nohup node server.js &

Naively, I call SSH like this:

ssh myserver <<EOF
./start.sh &
EOF

This starts the script, but leaves the session connected. I want to follow this step with other commands in a script, so that's no good.

How can I SSH to the remote machine, launch a nohup command into the background, then disconnect? I suppose I could put the SSH process itself into the background, but that doesn't seem right.

Solution 1

You have already found the right way, here document.

NOTE: you can put the ssh (client) into background by placing a & at the end, but you will not see the output. If you really want to do this, redirect the stdout/stderr to a file in case you need to check the response from the remote host.

Basically you can do it in either way:

Directly run the command{,s}

ssh user@host "nohup command1 > /dev/null 2>&1 &; nohup command2; command3"

OR

ssh user@host "$(nohup command1 > /dev/null 2>&1 &) && nohup command2 >> /path/to/log 2>&1 &"

NOTE: && requires the first command to return 0 before executing the second

Use Here document

ssh user@host << EOF
nohup command1 > /dev/null 2>&1 &
nohup command2 >> /path/to/command2.log 2>&1 &
......
EOF

The above 3 options should work for you.

In addition, take a look at the answer here: https://askubuntu.com/a/348921/70270

Solution 2

ssh node "nohup sleep 10 &"

is not running in daemon mode, keeping your ssh session connected. Ssh session will return in 10 seconds, despite you used nohup.

Reason is that remote stdout and stderr still connected to your session. It keeps ssh session alive, nohup does not help.

This:

ssh node "nohup sleep 10 1>/dev/null 2>/dev/null &"

returns immediately. It does start remote process with nohup and exits ssh session immediately.

Solution 3

Why not just tmux or screen and be done with it? For example:

$ tmux new -s SessionNameHere
$ nohup /path/to/your/script.sh

This is practical if it's something that will continuously loop or take a while to finish. You can disconnect from the session and it will remain active.

Solution 4

Shorter form:

ssh host "(command 1; command 2; ...) &>/dev/null &"

It appears, that bash itself performs detaching from terminal, so no nohup needed. I run Ubuntu 14.04 x86_64, bash 4.3.11.

Solution 5

And last but not least, remember not to allocate a tty for your session.
If you do that, it will not work.

So if you got any -t or -tt or -ttt and so on in your command. Thats the reason why it is not working for you.

So instead of

ssh -tt host "(command 1; command 2; ...) &>/dev/null &"

Use

ssh host "(command 1; command 2; ...) &>/dev/null &"

View more solutions

Comments

[Saptronic]

Thanks - so the bit I was missing was really the > /dev/null.

[Terry Wang]

Not really, > /dev/null 2>&1 is to discard any stdout/stderr output by redirecting to the black hole lol

[Alexis Wilke]

Note that I would suggest using >> instead of just > to write to log files. Otherwise, you'll only get the errors from the very last run.

[Luke]

Is it correct that if I simply start a command as nohup command2 >> /path/to/command2.log 2>&1 & in a terminal and then close the terminal that the process is lost?

[Terry Wang]

@Luke No, nohup makes the process immune to hang up signal (e.g. issued when you close the terminal session). It'll be running in the background until terminated.

[Saptronic]

Doesn't really fit what I'm asking for, which is a single command that can be run on my laptop, which will connect, fork a command, then disconnect.

[David Foerster]

This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post - you can always comment on your own posts, and once you have sufficient reputation you will be able to comment on any post. - From Review

[Saptronic]

This doesn't seem to work. nohup; just outputs a complaint about missing arguments.

[Alek_A]

@DavidFoerster, no Idea why, if command 1; command 2; ... represents script OP want to run. OP can also use special case of this answer: ssh myserver ./start.sh &>/dev/null &

[tsionyx]

@TerryWang the second example will not work with ampersand followed by double ampersand, see unix.stackexchange.com/q/67006/86716

[Terry Wang]

@tsionyx thank you for pointing this out, thumb up! Will update the original post.

[CKM]

@TerryWang. What does > /dev/null 2>&1 do here?

[Terry Wang]

@chandresh redirect the output to blackhole including stderr

[fiorentinoing]

the EOF version ever wins when ampersands are involved

[Lee Meador]

This was exactly our problem. We only had a 1>file.log. The ssh on the one machine was stuck. The process on the 'node' machine was running. When we added a 2>&1 just before the closing &, that let the ssh disconnect.

[Davor Cubranic]

This answer explains the key issue: that you need to redirect both stdout and stderr otherwise SSH will kill the command.

[Sebastiaan]

Thank you for clarifying the actual problem: the need to redirect stdout & stderr.

[fullStackChris]

Just as an even further clarification, nohup isn't even needed when redirecting the stdout and stderr.