How to set up disk cloning with dd, netcat and ssh tunnel?

Question

I would like to copy stuff in bulk (reimage disk using dd) with netcat from host A to B via ssh encrypted channel on Linux.

What commands should I type on both ends?

Solution 1

Copying from source to target where target has sshd running:

• dd if=/dev/sda | gzip | ssh root@target 'gzip -d | dd of=/dev/sda'

Copying from source to target via sshd_host when target is not running sshd.

• Target: nc -l -p 62222 | dd of=/dev/sda bs=$((16 * 1024 * 1024))
• Source: ssh -L 62222:target:62222 sshd_host &

• Source: dd if=/dev/sda | nc -w 3 localhost 62222

  dd - if= is the source, of= is the destination, bs= is the block size. Different block sizes may improve performance. 16 is usuually a fairly reasonable starting point. You can also use count= to indicate how many blocks to copy.

  nc - -p indicates the port to use for services. -l is used to start a service. -w sets up the time to wait for data in the pipline before quiting.

  ssh - -L sets up the tunnel on the remote host. The format of the argument is, local_port:target_host:target_port. Your local program (nc) connects to the local_port, this connection is tunneled and connected to target_port on the target_host.

The options defined are just the ones used for this. Look at the man pages for more details.

A few notes:

1. If you are doing this over anything but a LAN, I'd suggest compressing the datastream with gzip or compress. Bzip2 would work too but it takes a bit more CPU time. The first one has an example of that usage.
2. Its better if the source partition is not mounted or is mounted read-only. If not you will need to fsck the destination image.
3. Unless one of the machines has netcat but not ssh, netcat isn't really needed here. That case would look like:

source machine dd -> nc -> ssh -> ssh tunnel -> sshd server -> nc on target -> dd

4. dd works best if the source and targets are the same size. If not the target must be the bigger of the 2.
5. If you are using ext2/3 or xfs, dump (or xfsdump) and restore may be a better option. It wont handle the boot sector but it works when the target and source are different sizes.

Solution 2

Host A is the one to image, host B is the one the image will be stored on:

root@A# dd if=/dev/sda | ssh root@B "dd of=/some/file"

Restoring to disk would just swap those two.

Solution 3

netcat is not needed.

on src machine run:

dd if=/dev/sdX bs=1M | ssh root@dstMachine " dd of=/dev/sdY bs=1M"

i assume none of partitions on sdX and sdY are mounted. you can boot both boxes with knoppix or other similar live distro.

dd - takes data from if [ if not provided - takes it from stdin ], sends data to of [ if not provided - data is sent to stdout ]. bs - block size... will speed things out.

ssh - executes command provided in quotes on remote box, all data pumped to stdin of ssh will be tunneled to remote machine and porovided as stdin to command executed there.

Solution 4

If you want use netcat without ssh. I presume that is the fastest way and not the secure one, you can copy and restore the whole disk like this:
On computer A with IP 192.168.0.1

cat /dev/hdb | nc -p 9000

On computer B

nc -l 192.168.0.1 9000 > /dev/hdb

Remember that according to man nc the -l option is:

  -l  Used to specify that nc should listen for an incoming connection rather than initiate a connection to a remote host.  It is an error to use this option in conjunction with the -p, -s, or -z options.

Solution 5

So long as the filesystems are both unmounted, dd works well.

(from server1) dd if=/dev/sda bs=32k | ssh <server2> dd of=/dev/sda bs=32k

You'll need hostkey authentication setup ahead of time or else the password prompt will cause the copy to fail.

Doing this on a mounted volume will produce poor results.

View more solutions

Comments

[danny]

thanks. are quotes necessary?

[pQd]

@Evgeny it'll be ok if disk is mounted as read only. otherwise - dont do it... your copy will be inconsistent.

[Mark Stosberg]

If you are copying a partition, don't you want the target partition to be unmounted? More detail on this option would be helpful.