tar files from another directory without copying them

Question

This should be very simple for experts in here.

I have a Linux account with user disk quota of only 6GB. There is some data-folder in /opt with about 3GB of data with when compressed comes up 1GB. There are some symbolic links within data-folder. Usually for taking a work backup I used to do the following.

cp -rvfL /opt/data-folder ~/
cd ~/
tar -zcvf data-folder.tar.gz data-folder
rm -rvf data-folder

Now I have already consumed 2.5GB in my user disk quota, I have just enough space to copy files to my home, but not to tar it again.

Is there any way to tar and copy the folder with in a single command, say by using a pipe.

P.S. I cannot tar in /opt because I have only read access to /opt.

Solution

You want to create a tar file away from the place the files you need to tar reside?

There are many ways to do this.

If it is to be created locally (= on the same machine) :

tar czvf /path/to/destination/newfile.tar.gz  ./SOURCEDIR_OR_FILES

You can add additionnal files or directories to tar at the end of that command.

If it is to be created remotely (ie, you want to create the tar file on a remote host from the one containing the data to be tared):

tar czvf - ./SOURCEDIR_OR_FILES | ssh user@host 'cat > newfile.tar.gz'

The later version is very versatile. For example you can also "duplicate" a directory + subdirs using the same technique:

Duplicate a directory+subdirs to another local directory:

tar cf - ./SOURCEDIR_OR_FILES | ( cd LOCAL_DEST_DIR && tar xvf - )

Duplicate a directory+subdirs to another remote directory:

tar cvf - ./SOURCEDIR_OR_FILES | ssh user@host 'cd REMOTE_DEST_DIR && tar xf - '

Drop the 'v' if you don't need it to display files as they are tar-ed (or untarred): it will then go much faster, but won't say much unless there is an error.

I use "./..." for the source to force tar to store it as a RELATIVE path. In some cases you'll want to add additionnal path information:

For example to tar the crontab files, including the one in /etc, you could do:

cd / ; tar czf all_crons.tgz  ./etc/*cron*  ./var/spool/cron

I use on purpose the relative path: some OLD versions of tar may be dangerous and extract files with their original GLOBAL path, meaning you could do : cd /safedir ; tar xvf sometar and have the files with global names overwrite files at their original path, which is OUTSIDE of /safedir and not underneath it! Very dangerous, and still possible as there are old production servers out there. Better to be used to use relative paths all the time, even if you use a more recent tar.

Comments

[Kamath]

Ohh I should have mentioned it, I already tried tar -zcvfh ~/data-folder.tar.gz /opt/data-folder. I am getting into some kind of circular archive problem. I keeps copying some files and never ends.

[Kamath]

Looks like there is a circular symbolic link in a sub folder of data.

[Remon]

@AUZKamath You can simultaneously tar and remove the files from your home directory use tar -rf <tarfilename> <filename> to add to existing tar file.

[Olivier Dulac]

another not about differences between tars : on some you may need to use -cvf instead of cvf, others won't recognise the - style. And z used to mean "compress with 'compress'", but on 'recent' tar it means "compress with gzip". You can know which compression is used on a file with: file zefile . And then rename it accordingly (if your tar use compress, it should better be renamed .tar.Z not .tar.gz)