Simple one-way synchronisation of user password list between servers

Question

Using a RedHat-derivative distro (CentOS), I'd like to keep the list of regular users (UID over 500), and group (and shadow files) pushed to a backup server.
The sync is only one-way, from the main server to the backup server.

I don't really want to have to deal with LDAP or NIS.
All I need is a simple script that can be run nightly to keep the backup server updated.
The main server can SSH into the backup system.

Any suggestion?

Edit:
Thanks for the suggestions so far but I think I didn't make myself clear enough.
I'm only looking at synchronising normal users whose UID is on or above 500.
System/service users (with UID below 500) may be different on both system.
So you can't just sync the whole files I'm afraid.

Solution 1

Well, I thought there was something existing I could use without having to roll my own solution, but I had to do something quick.

Below is a script that will do just what I needed.

Instructions

For it to work, just change the few config variables for the minimum and maximum UID to be considered as a normal user and the remote host name or IP address.

You must have setup the remote server to accept incoming SSH sessions from the local server's root user without having to enter a password.
Commander Keen hinted on how it's done in his answer on this page but you can also refer to password-less SSH login for detailed instructions.

How it works

What the script does is copy each of the remote passwd, group, shadow, gshadow files from the remote server to a temporary location on the lcoal server.
Then it strips these temp files from all "normal" users, keeping only the references to the system users.

The next step is going through each of the local versions of passwd, group, shadow, gshadow and appending just the "normal" users to their corresponding temp files, then uploading each of them back to the remote server to replace the old one.

Warning

Before you attempt anything, make sure you make a copy of your passwd, group, shadow, gshadow on both the local and remote servers.

Security

File ownership and attributes are preserved.
Temporary files are saved in /tmp and deleted, whether the sync was successful or not.
The local server must have password-less root access to the backup (but not the other way around). This is necessary so we can get the user accounts config files (which are otherwise restricted).

The Code

This is a first-attempt and it's a bit messy (not beautiful code) but it does the job pretty well and someone else may find it useful.

It's a Perl script that only has a dependency on the Net::SCP module to copy files securely between servers.

#!/usr/bin/perl -w
use Net::SCP qw(scp);
use strict;

use constant TRUE  => (1==1);
use constant FALSE => (1==0);

#--------------------------------------------------------
# Configuration
# Modify as needed
#--------------------------------------------------------
my $remoteHost = '10.13.113.2';  # email backup server
my $minUID     = 500;
my $maxUID     = 30000;
my $minGID     = 500;
my $maxGID     = 30000;

#--------------------------------------------------------
# Internal variables, normally not to be modified.
#--------------------------------------------------------
my $systemConfigDir = '/etc';
my $tmpDir = $ENV{TMPDIR} || $ENV{TMP} || $ENV{TEMP} || '/tmp';

#--------------------------------------------------------
#  Main
#--------------------------------------------------------
# STEP 1
# Get the remote files to /tmp and
# clean them of their normal users
ProcessFiles('remote');

# STEP 2
# Append the local normal users to the temp files
# and then send them back to the remote
ProcessFiles('local');

#--------------------------------------------------------
# ProcessFiles sub does one of two things:
# - if the passed argument is 'remote', then fetch each
#   user account file from the remote server, then remove
#   all normal users from each file, only keeping the
#   system users.
# - if the passed argument is 'local', then appends all
#   normal local users to the previously fetched and
#   cleaned-up files, then copies them back to the remote.
#--------------------------------------------------------
sub ProcessFiles {
        my $which = shift;
        my $tmpfile;
        my %username = ();
        my %usergroup = ();
        my %userUID = ();
        my %userGID = ();
        my @info;
        foreach my $f ('passwd','group','shadow','gshadow') {
                my $tmpfile = "$tmpDir/$f.REMOTE";
                if ($which eq 'remote') {
                        # Fetch the remote file
                        unlink $tmpfile if -e $tmpfile;
                        scp("$remoteHost:$systemConfigDir/$f", $tmpfile)
                                or die ("Could not get '$f' from '$remoteHost'");
                }
                # Glob the file content
                open CONFIGFILE, (($which eq 'remote') ? $tmpfile : "$systemConfigDir/$f");
                my @lines = <CONFIGFILE>;
                close CONFIGFILE;
                # Open the temp file, either truncating it or in append mode
                open TMPFILE,  (($which eq 'remote') ? ">$tmpfile" : ">>$tmpfile" )
                        or die "Could not open '$tmpfile' for processing";
                foreach my $line (@lines) {
                         # Skip comments, although they should be illegal in these files
                        next if $f =~ /^\s*#/;
                        @info = (split ':', $line);
                        if ($f eq 'passwd') {
                                my $uid = $info[2];
                                my $isnormaluser = ($uid > $minUID) && ($uid < $maxUID);
                                next if (($which eq 'remote') ? $isnormaluser : !$isnormaluser);
                                $username{$info[0]} = TRUE;
                                $userUID{$uid} = TRUE;
                                $userGID{$info[3]} = TRUE;
                        } elsif ($f eq 'group') {
                                my $gid = $info[2];
                                my $isnormalgroup = ($gid > $minGID) && ($gid < $maxGID);
                                next if (($which eq 'remote') ? $isnormalgroup : !$isnormalgroup);
                                $usergroup{$info[0]} = TRUE;
                        } elsif ($f eq 'shadow') {
                                next if !exists $username{$info[0]};
                        } else {
                                next if !exists $usergroup{$info[0]};
                        }
                        # Any line that reaches this point is valid
                        print TMPFILE $line;
                }
                close TMPFILE;
                if ($which eq 'local') {
                        # send the file back
                        scp($tmpfile, "$remoteHost:$systemConfigDir/$f") or
                                die ("Could not send '$f' to '$remoteHost'");
                        unlink $tmpfile;
                }
        }
}

#--------------------------------------------------------
# Make sure we cleanup the temp files when we exit
#--------------------------------------------------------
END {
        my $tmpfile;
        foreach my $f ('passwd','group','shadow','gshadow') {
                $tmpfile = "$tmpDir/$f.REMOTE";
                unlink $tmpfile if -e $tmpfile;
        }
}

Update 21MAY2010: updated code to improve sync of group ID

Solution 2

You can use awk to extract users/groups with IDs of 500 or greater. I have also taken the liberty of excluding user id 65534, which is often reserved for the "nobody" user (depending on distro; no clue if CentOS does so):

awk -F: '($3>=500) && ($3!=65534)' /etc/passwd > passwd.new
awk -F: '($3>=500) && ($3!=65534)' /etc/group > group.new
awk -F: '($3>=500) && ($3!=65534) {print $1}' /etc/passwd | grep -f - /etc/shadow > shadow.new

Then use rsync, scp, or your file transmission method of choice to copy the files to your backup system. These files can then be appended to the end of a 'clean' passwd, group or shadow file when you need to restore them (ie: default system users/groups only, to prevent unintentional duplications of ID/username).

cat passwd.new >> /etc/passwd
cat group.new >> /etc/group
cat shadow.new >> /etc/shadow

Solution 3

NIS/NIS+ were invented for this exact reason.

But they're kind of ugly and centralized (LDAP/Kerberos/SMB/etc.) authentication is a much much better idea if you can do it. To setup NIS/NIS+ you will need:

Packages:

yp-tools ypbind ypserv portmap

and an /etc/yp.conf with something like:

domain example.org server nis.example.org
ypserver nis.example.org

and then in /etc/sysconfig/network:

NISDOMAIN=example.org

And I got lazy, here's a good howto: http://www.wains.be/index.php/2007/02/28/setting-up-nis-under-centos-4/ that will walk you through it.

Personally for backup I'd just backup the entire /etc/ directory and be done with it. It's only a few megs at most.

Solution 4

use cppw and cpgr:

CPPW(8)                                                                                                                                                      

NAME
       cppw, cpgr - copy with locking the given file to the 
       password or group file

SYNOPSIS<br>
       cppw [-h] [-s] password_file
       cpgr [-h] [-s] group_file

DESCRIPTION
       cppw  and  cpgr will copy, with locking, the given file to
       /etc/passwd and /etc/group, respectively.  With the -s flag, 
       they will copy the shadow versions of those files, 
       /etc/shadow and /etc/gshadow, respectively.

       With the -h flag, the commands display a short help message
       and exit silently.

SEE ALSO
       vipw(8), vigr(8), group(5), passwd(5), shadow(5), gshadow(5)

AUTHOR
       cppw and cpgr were written by Stephen Frost, based on vipw 
       and vigr written by Guy Maor.

Solution 5

There are many ways and solutions here, but to answer the original question there are three steps:

1. Create a password-less SSH-key on the server:

   ssh-keygen -b 4096

2. Copy .ssh/id_rsa.pub to .ssh/authorized__keys2 on the client:

   scp ~/.ssh/id_rsa.pub client:.ssh/authorized_keys2 

3. Add something like this to your /etc/crontab (or edit with crontab -e):

   0 0 * * * scp /etc/{passwd,shadow,group} root@backupbox:/var/mybackupdir 

View more solutions

Comments

[Renaud Bompuis]

Thanks but I don't want to sync the whole file because the UID of system users isn't the same on both system, so I can't just replace the passwd/shadow/etc files from one system to the other. Only normal users should be synchronized.

[Renaud Bompuis]

+1 for mentioning the password-less SSH-Key.

[Renaud Bompuis]

+1 for a good attempt at a practical solution.

[Renaud Bompuis]

+1 for mentioning a better -albeit more complex to setup and maintain- way of doing what I was looking for. Maybe it will help others.

[Renaud Bompuis]

+1 for showing me something new. Unfortunately, it's not close to a solution, and it's not available on RedHat systems? ;-(

[Noumenon]

I'm not sure if you skipped gshadow for a reason, but this works: awk -F: '($3>=500) && ($3!=65534) {print $1}' /etc/passwd | grep -f - /etc/gshadow > gshadow.new.

[Noumenon]

Tip for when there are existing users: delete the duplicated ones and renumber the overlapping ones, then chown -R existingUser:existingGroup on all folders they used to own to set them to their new id.

[Z4-]

I like this better than the accepted answer, I don't see why it's more involved than maintaining perl scripts. Here's a tutorial for debian: server-world.info/en/note?os=Debian_8&p=nis Updated tutorial link for Kurt's answer: blog.wains.be/2007/2007-02-28-setting-up-nis-under-centos-4.‌​md