Can I automate tar's multi-volume-feature?

Question

Ok, so I've just had a read through this page after a way to improve my current backup solution on my Debian server. Tar seems to be offering a quite nice multi-volume feature, although when I try it out, it asks me to Prepare volume #X for ‘mybackup.tar.gz’ and hit return:.

How should I automate this as I would like to take usage of this feature in an automated CRON script where no one is there to push return and enter whatever is rquired by the multi-volume prompt.

Is using split the only way?

Solution

Here is a solution:

printf 'n file-%02d.tar\n' {2..100} | 
    tar -ML 716800 -cf file-01.tar Documents/ 2>/dev/null

where 100 is a number greater or equal to the number of volumes.

Edit

Setting a big number should not be a problem, though I tend to not take a ridiculous one.

An alternative could be a "next volume" script, that you can set with the -F option,

tar -ML 716800 -F './myscript file' -cf file.tar Documents/ 2>/dev/null

then in ./myscript put

#!/bin/bash

prefix="$1"
n=1
while [[ -e "$prefix-$n.tar" ]]; do
  ((n++))
done
mv "$prefix.tar" "$prefix-$n.tar"
echo "$prefix-$n.tar"

It will be executed at each volume end, and will move file.tar to the appropriate fileNNN.tar. For the last volume the script will not be executed, so the last volume name stay file.tar.

Edit 2

I ended up with the following elaborated solution.
Here are two script, one for the creation and the other for the extraction:

#!/bin/bash
# CREATION SCRIPT

# save on file the initial volume number
echo 1 >number

# multi-volume archive creation
tar -ML 100000 -F './tar-multi-volume-script c file' -cf file.tar Documents2/ 2>&-

# execute the "change-volume" script a last time
./tar-multi-volume-script c file

and

#!/bin/bash
# EXTRACTION SCRIPT

# save on file the initial volume number
echo 1 >number

# execute the "change-volume" script a first time
./tar-multi-volume-script x file

# multi-volume archive extraction
tar -M -F './tar-multi-volume-script x file' -xf file.tar 2>&-

# remove a spurious file
rm file.tar

where ./tar-multi-volume-script is given by

#!/bin/bash
# TAR INVOKED SCRIPT

mode="$1"
prefix="$2"
n=$(<number)

case $mode in
  c) mv "$prefix.tar"    "$prefix-$n.tar" ;;
  x) cp "$prefix-$n.tar" "$prefix.tar"    ;;
esac

echo $((n+1)) >number

Obviously you have to change many bits here and there to adapt to your situation and to be sure it would work in cron, that is always a little challenge.

Comments

[Industrial]

I like that, but then I need to know the max number of volumes per directory in beforehand? I mean, what happens when I exceed 716m in 100 directories? Can I just set max to a ridiculously high number like 919191 and leave it?

[enzotib]

@Industrial: see edit

[Industrial]

Thanks a lot for this great answer, Enzotib! How would you suggest restoring/extracting using this "next-volume" script?

[enzotib]

@Industrial: here an almost complete solution

[SamB]

You seem to have forgotten to mention the --new-volume-script flag?

[jasonwryan]

Please don't just post links; include some context so that, should the other site go down, there is sufficient information here to make sense of your answer.

[Nemo]

Thank you. How to make the external command/next volume script launch a command in a subshell, without letting tar wait for that new subshell to be done?

[Nemo]

Self-answer: use screen -m -d, stupid: superuser.com/a/172052/283120 Try with $ cat next.sh #!/bin/bash screen -d -m sh -c "sleep 10s; date > /tmp/test; exit" $ tar cf test.tar -ML 10k -F "./next.sh" testinput

[Matthieu]

From the tar man page you linked, you could use $TAR_ARCHIVE-$TAR_VOLUME in your next-volume-script.