Unison - Automatically Sync Computers

Question

Is there a way two sync multiple computers (i.e. particular directories in the home folder) automatically using Unison? If not, is there an alternative that can do this? I'm not completely against opening Unison all the time and doing the sync manually, but when it opens you must select a profile, which corresponds to just one directory, so you have to sync each individually. Add to that, the IP address is dynamic, and therefore has to be edited. This is quite tedious, and it seems there should be a simpler way. Am I missing something? Is there a way to simplify this process (possibly a bash script)?

Solution 1

My suggestion would be to solve the problem of the dynamic IP (most routers have the concept of address reservation, that you can use to "fix" the IP of your computer). After that, you can use the command line version of unison:

unison -auto -batch profilename 

and you can add it to your login/logout scripts or as a cron job.

If you still need to change the ip, you can use sed in a script:

1. create a template file in ~/.unison, newprof.tmpl, like this:

   root = /home/romano/
   root = ssh://romano@MYIPHERE//home/romano/
   
   path = education
   path = research
   

2. create the following script, call it sync_newprof:

   #!/bin/bash
   # 
   cd $HOME/.unison
   cat newprof.tmpl | sed s/MYIPHERE/$1 >! newprof.prf
   unison -auto -batch profilename
   

   basically, we create a profile file changing the MYIPHERE word with the script first argument

3. now you can call it as

   sync_newprof 1.1.1.1 
   

   and it will create the profile with the given IP and sync.

Solution 2

On a local network you have two machines on which you want to securely and automatically synchronize /home/<username>/Desktop every 10 minutes: "Host" which runs your OpenSSH Server and "Client" which connects over SSH to "Host".

Overview of Solution*

• Setup an OpenSSH Server on Host
• Install Unison on the Client and Host
• Create Script 1 so that cron can use <username>'s RSA key
• Add two lines to <username>'s crontab, and
• Set Script 1 to run on startup.

Setup OpenSSH Server on Host

See Setup SFTP Server to setup an OpenSSH server that listens for SSH connections as <username> on Host, uses a Static IP, listens on a non-standard port, refuses password-based connections, and allows RSA key connections if Host possesses the RSA public_key.

Install Unison on Client and Host

On both machines, run (unison-gtk is GUI if you want it):

sudo apt-get install unison unison-gtk

Create One Script

cron does not share the same ENVIRONMENT as <username> and thus will fail to find the Client RSA key needed to connect to Host. Script 1 provides that ENVIRONMENT to <username>'s crontab file.

Script 1 also generates Script 2 which is executed by <username>'s crontab file every 10 minutes and runs Unison.

• CAVEAT! I am not a scripting expert and adapted this [1]

Script 1 (/home/<username>/Documents/Scripts/set_ssh_env_for_cron_unison.sh)

#!/bin/sh
SSH_ENV=$HOME/.ssh/environment_for_cron    ## file to be generated
SSH_ENV2=$HOME/Documents/Scripts/unison_sync.sh    ## Script 2 to be generated
function start_agent {
   echo "Initialising new SSH agent..." 
   env | grep SSH | sed 's/SSH_AUTH_SOCK=//' > ${SSH_ENV}   ## send output of env | grep SSH minus "SSH_AUTH_SOCK=" so cron can get output from SSH_ENV
     ## create a new script with three lines, line 3 is unison command
   echo "#!/bin/sh" > ${SSH_ENV2}   ## line 1
   env | grep SSH >> ${SSH_ENV2}    ## line 2
   echo "/usr/bin/unison /home/<username>/Desktop ssh://<Host's local IP address>:<Host's listening SSH Port>//home/<username>/Desktop -batch -debug update+ -ignore 'Name *.gvfs' -ignore 'Name .cache*' -ignore 'Name [Cc]ache*' -ignore 'Name .thumbnails*' -ignore 'Name [Tt]rash*' -ignore 'Name *.backup*' -ignore 'Name *~' -ignore 'Name .dropbox*' -ignore 'Name /proc/*' -ignore 'Name /sys/*' -ignore 'Name /dev/*' -ignore 'Name /run/*' -ignore 'Name /root/Ubuntu One/*'" >> ${SSH_ENV2}  ## line 3
   echo succeeded
   chmod 644 ${SSH_ENV} ## change permissions so cron can read SSH_ENV
   chmod 775 ${SSH_ENV2}    ## change permissions so cron can execute SSH_ENV2
   . ${SSH_ENV} > /dev/null
   ssh-add id_rsa       ## unlocks your RSA key and loads it into the ssh-agent which holds keys in memory throughout login session
}
if [ -f "${SSH_ENV}" ]; then
   . ${SSH_ENV} > /dev/null
   ps -ef | grep ${SSH_AGENT_PID} | grep ssh-agent$ > /dev/null || {
     start_agent;
   }
else
   start_agent;
fi

Edit <username>'s crontab to execute synchronization every 10 minutes

crontab -e

Now append the following lines to <username>'s crontab

SSH_AUTH_SOCK=/home/<username>/.ssh/environment_for_cron
*/10 * * * * /home/<username>/Documents/Scripts/unison_sync.sh

• Make sure there is a blank new line following the two lines above in the new crontab!

Exit with Ctrl+x; y for "YES"; Enter for confirmation. Check the crontab file:

crontab -l

Set Script 1 to run on Startup

Use Dash > Startup Applications (or Applications > System Tools > Preferences on Gnome) to add a new startup program. Where it says "Command:", enter:

/home/<username>/Documents/Scripts/set_ssh_env_for_cron_unison.sh

Congratulations!

If all went well, you will enter the password to Client's RSA key upon Client login and you will not have to enter the password again throughout the login session. Every 10 minutes, Client will then automatically negotiate a passwordless SSH connection with Host and Unison will synchronize /home/<username>/Desktop and //HOST//home/<username>/Desktop. Your laptop and desktop may now be joined in holy synchromony.

*Details:

My final setup communicates as <username> (not root) using a non-standard SSH port to “Host” at a static IP. Password Authentication is disabled on “Host” (requiring a pre-authenticated RSA key), while my RSA key on “Client” requires a password for use (once per login session). Thus, an attacker needs to guess the correct port, RSA key, and password to create a shell as <username> with “Host.”

After this one password entry per login session, “Client” and “Host” synchronize themselves automatically every 10 minutes. This was the method I found would work using <username>'s RSA key and cron's ENVIRONMENT.**

CAVEAT! Of course, if one or both machines are offline, shutdown, etc., they will not synchronize!

**Please don't hesitate to let us know if there is a simpler method using an RSA key belonging to root on Client that does not require a password for use while Host simultaneously refuses password-based SSH connections.

Comments

[Nateowami]

Just discovered that on a local network you can use hostname.local, rather than the actual IP address. That really helps simplify the process, but I'd still be nice to automate it.

[Nateowami]

Well, that stopped working, and I now learn I can't explain why it worked at all in the first place. It's as tedious as ever, and I've resorted to editing the IP address in the Unison config files.

[Nateowami]

Now I can hardly believe what I didn't use to know... To configure hostnames, edit /etc/hosts (you can find lots of guides on how to do this). Additionally, if you want to use the .local method, which updates its map of IP's to host names automatically, checkout this for an explanation of how it works.

[Nateowami]

Thanks so much, and sorry for taking so long to accept your answer! The IP problem was fixed when I started using a router, and I finally got SSH keys setup. Then I wrote a script to automate it: gist.github.com/Nateowami/722edc74c15ee25c0e9c.