Why ssh fails from crontab but succedes when executed from a command line?

Question

I have a bash script that does ssh to a remote machine and executes a command there, like:

ssh -nxv user@remotehost echo "hello world"

When I execute the command from a command line it works fine, but it fails when is being executed as a part of crontab (errorcode=255 - cannot establish SSH connection). Details:

...
Waiting for server public key.
Received server public key and host key.
Host 'remotehost' is known and matches the XXX host key.
...
Remote: Your host key cannot be verified: unknown or invalid host key.
Server refused our host key.
Trying XXX authentication with key '...'
Server refused our key.
...

When executing locally I'm acting as a root, crontab works as root as well. Executing 'id' from crontab and command line gives exactly the same result:

$ id
> uid=0(root) gid=0(root) groups=0(root),...

I do ssh from some local machine to the machine running crond. I have ssh key and credentials to ssh to crond machine and any other machine that the scripts connects to.

PS. Please do not ask/complain/comment that executing anything as root is bad/wrong/etc - it is not the purpose of this question.

Solution 1

I am guessing that normally when you ssh from your local machine to the machine running crond, your private key is loaded in ssh-agent and forwarded over the connection. So when you execute the command from the command line, it finds your private key in ssh-agent and uses it to log in to the remote machine.

When crond executes the command, it does not have access to ssh-agent, so cannot use your private key.

You will have to create a new private key for root on the machine running crond, and copy the public part of it to the appropriate authorized_keys file on the remote machine that you want crond to log in to.

Solution 2

keychain

solves this in a painless way. It's in the repos for Debian/Ubuntu:

sudo apt-get install keychain

and perhaps for many other distros (it looks like it originated from Gentoo).

This program will start an ssh-agent if none is running, and provide shell scripts that can be sourced and connect the current shell to this particular ssh-agent.

For bash, with a private key named id_rsa, add the following to your .profile:

keychain --nogui id_rsa

This will start an ssh-agent and add the id_rsa key on the first login after reboot. If the key is passphrase-protected, it will also ask for the passphrase. No need to use unprotected keys anymore! For subsequent logins, it will recognize the agent and not ask for a passphrase again.

Also, add the following as a last line of your .bashrc:

. ~/.keychain/$HOSTNAME-sh

This will let the shell know where to reach the SSH agent managed by keychain. Make sure that .bashrc is sourced from .profile.

However, it seems that cron jobs still don't see this. As a remedy, include the line above in the crontab, just before your actual command:

* * * * * . ~/.keychain/$HOSTNAME-sh; your-actual-command

Solution 3

Don't expose your SSH keys without passphrase. Use ssh-cron instead, which allows you to schedule tasks using SSH agents.

Solution 4

So I had a similar problem. I came here and saw various answers but with some experimentation here is how I got it work with sshkeys with passphrase, ssh-agent and cron.

First off, my ssh setup uses the following script in my bash init script.

# JFD Added this for ssh
SSH_ENV=$HOME/.ssh/environment

    # start the ssh-agent
    function start_agent {
        echo "Initializing new SSH agent..."
        # spawn ssh-agent
        /usr/bin/ssh-agent | sed 's/^echo/#echo/' > "${SSH_ENV}"
        echo succeeded
        chmod 600 "${SSH_ENV}"
        . "${SSH_ENV}" > /dev/null
        /usr/bin/ssh-add
    }


    if [ -f "${SSH_ENV}" ]; then
         . "${SSH_ENV}" > /dev/null
         ps -ef | grep ${SSH_AGENT_PID} | grep ssh-agent$ > /dev/null || {
            start_agent;
        }
   else
        start_agent;
   fi

When I login, I enter my passphrase once and then from then on it will use ssh-agent to authenticate me automatically.

The ssh-agent details are kept in .ssh/environment. Here is what that script will look like:

SSH_AUTH_SOCK=/tmp/ssh-v3Tbd2Hjw3n9/agent.2089; export SSH_AUTH_SOCK;
SSH_AGENT_PID=2091; export SSH_AGENT_PID;
#echo Agent pid 2091;

Regarding cron, you can setup a job as a regular user in various ways. If you run crontab -e as root user it will setup a root user cron. If you run as crontab -u davis -e it will add a cron job as userid davis. Likewise, if you run as user davis and do crontab -e it will create a cron job which runs as userid davis. This can be verified with the following entry:

30 *  *   *   *     /usr/bin/whoami

This will mail the result of whoami every 30 minutes to user davis. (I did a crontabe -e as user davis.)

If you try to see what keys are used as user davis, do this:

36 *  *   *   *     /usr/bin/ssh-add -l

It will fail, the log sent by mail will say

To: [email protected]
Subject: Cron <davis@hostyyy> /usr/bin/ssh-add -l

Could not open a connection to your authentication agent.

The solution is to source the env script for ssh-agent above. Here is the resulting cron entry:

55 10  *   *   *     . /home/davis/.ssh/environment; /home/davis/bin/domythingwhichusesgit.sh

This will run the script at 10:55. Notice the leading . in the script. It says to run this script in my environment similar to what is in the .bash init script.

View more solutions