Gzip file but excluding some directories|files and also append current date

Question

I'm looking for a way to create a .tar.gz file from a directory but I have some doubts. I'm know that the command for create the file is:

tar -zcvf /path/to/compressed/file.tar.gz /path/to/compress 

And then I will get a file.tar.gz but this keep the path to the original sources. So this is what I'm looking for:

• Should be compress as .tar.gz
• If it's possible shouldn't keep the path to the compressed directory I mean I only needs the compress directory and its content .If I uncompress the file under Windows for example I'll get /path[folder]/to[folder]/compress[folder+content] and I just want the latest

• If it's possible can I ommit some directories? Example I like to compress app/ folder and its content is:

  drwxrwxr-x  6 ubuntu ubuntu  4096 Feb 24 14:48 ./
  drwxr-xr-x 10 ubuntu ubuntu  4096 Feb 26 22:33 ../
  -rw-rw-r--  1 ubuntu ubuntu   141 Jan 29 06:07 AppCache.php
  -rw-rw-r--  1 ubuntu ubuntu  2040 Feb 24 14:48 AppKernel.php
  -rw-rw-r--  1 ubuntu ubuntu   267 Jan 29 06:07 autoload.php
  -rw-r--r--  1 root   root   94101 Feb 19 21:09 bootstrap.php.cache
  drwxrwxrwx  4 ubuntu ubuntu  4096 Feb 25 16:44 cache/
  -rw-rw-r--  1 ubuntu ubuntu  3958 Feb 24 14:48 check.php
  drwxrwxr-x  2 ubuntu ubuntu  4096 Feb 24 14:48 config/
  -rwxrwxr-x  1 ubuntu ubuntu   867 Jan 29 06:07 console*
  -rw-rw-r--  1 ubuntu ubuntu  6148 Jan 29 06:07 .DS_Store
  -rw-rw-r--  1 ubuntu ubuntu   143 Jan 29 06:07 .htaccess
  drwxrwxrwx  2 ubuntu ubuntu  4096 Feb 24 14:48 logs/
  -rw-rw-r--  1 ubuntu ubuntu  1118 Jan 29 06:07 phpunit.xml.dist
  drwxrwxr-x  5 ubuntu ubuntu  4096 Jan 29 06:07 Resources/
  -rw-rw-r--  1 ubuntu ubuntu 30404 Feb 24 14:48 SymfonyRequirements.php
  

  But I want to leave out cache/, logs/ directories and bootstrap.php.cache file, how? Is that possible?

• I need to append the current date (DD/MM/YYYY-H:M:S) to the file name, how?

Can I get some advise or help on this? I'm planning to add this to a Bash script so it will works as a bash script and not as a command line

Update: testing inside the script

Following @Ariel suggestion I have added this line to a bash script:

read -e -p "Enter the directory to compress: " -i "/var/www/html/" directory
read -e -p "Enter the filename: " filename
FILENAME = "$filename-`date +%d-%m-%Y-%X`.tgz"

cd "$directory"
tar -zcvf /home/$FILENAME --exclude cache --exclude logs --exclude bootstrap.php.cache --exclude composer.lock --exclude vendor

But I'm getting this error:

Enter the directory to compress: /var/www/html/lynxoft/apps/checkengine/
Enter the filename: checkengine
/usr/local/bin/script-task: line 109: FILENAME: command not found
tar: Cowardly refusing to create an empty archive

Why FILENAME is treat as a command and not as a var as docs says?

Update 2: still compressing the whole path

I have fixed some issues thanks to users comments and lines on the script looks likes this:

read -e -p "Enter the directory to compress: " -i "/var/www/html/" directory
read -e -p "Enter the filename: " filename
FILENAME="$filename"-$(date +%d-%m-%Y-%T).tgz

cd "$directory/.."
tar -zcvf /home/$FILENAME "$directory" --exclude="*cache*" --exclude=logs --exclude=bootstrap.php.cache --exclude=composer.lock --exclude=vendor --exclude=.git

The only remaining issue is that compressed file still having the whole path and not just the end directory. For example:

Enter the directory to compress: /var/www/html/lynxoft/apps/checkengine/
Enter the filename: file

That result in file-28-02-2015-17:44:32.tgz but content inside the compressed file still having the whole path /var/www/html/lynxoft/apps/checkengine/, why?

Solution 1

Just cd to the folder from where you want the tarball tress structure to start, and use relative paths! So for instance:

# Note, one folder above
cd /path/to/compress/..
# Tell tar to compress the relative path    
tar -zcvf /path/to/compressed/file.tar.gz compress

When you uncompress it, it will create the compress folder in the current working directory.

You can exclude files or folders using the --exclude <PATTERN> option, and add a date in the filename with date +FORMAT, for instance:

FILENAME="file-`date +%d-%m-%Y-%T`.tgz"
tar -zcvf /path/to/compressed/$FILENAME --exclude cache --exclude logs --exclude bootstrap.php.cache compress

Solution 2

With gnu tar you could use:

--transform=EXPRESSION, --xform=EXPRESSION
      Use sed replace EXPRESSION to transform file names.

which means --xform='s|path/to/||' would remove path/to/ from the path;
and

--exclude=PATTERN
      Exclude files matching PATTERN, a glob(3)-style wildcard pattern.

so you could run:

tar -zcvf /path/to/compressed/"$(date +%d-%m-%Y-%T)"-testdir.tar.gz \
--transform='s|^path/to/||' /path/to/compress \
--exclude='cache' --exclude='logs' --exclude='bootstrap.php.cache'

Note there's no leading / in the expression used with xform. A more generic approach:

--xform='s|^([^/]*/){2}||x'

That would strip the first two elements of the path (replace 2 with another number - as needed).

Comments

[Costas]

tar -zcvf "/path/to/compressed/$(date +%d/%m/%Y-%X).tar.gz" -C /path/to/compress/ ./ --exclude="*cache*" --exclude=logs (read man tar)

[Kurtcebe Eroglu]

@Costas I've tried your solution also but get this tar (child): /home/checkengine(date +%d/%m/%Y-%X).tar.gz: Cannot open: No such file or directory error on console (also tried from the script) why?

[don_crissti]

You prolly forgot the $ before (date...). Also, @Costas - are you trying to create a file with /s in its name ?

[Ariel]

@ReynierPM you should not leave spaces in the shell variable declarations! Use exactly FILENAME="..."

[Kurtcebe Eroglu]

@Ariel fixed that one but still getting this: tar: Cowardly refusing to create an empty archive

[Ariel]

@ReynierPM it means that tar is not finding a single file to pack in the path you gave it... check your folders/paths. Actually, you forgot the path to "tar" at the very end of that command line: either add a . (a single dot, meaning everything in the current folder), a *, some filename or some subfolder you want to pack.

[Kurtcebe Eroglu]

@Ariel thanks I have fixed that one too but one still, the related to path inside the compressed file can you take a look and point me in the right direction on this one as well?

[Ariel]

@ReynierPM your tar command still contains an absolute path: you use tar -zcvf /home/$FILENAME "$directory" ... Change that to tar -zcvf /home/$FILENAME . ... (seriously, a single dot, means "everything here")

[Ariel]

Well, the dots are not very happily chosen, and also better quote the paths, sorry: tar -zcvf "/home/$FILENAME" . --exclude="*cache*" <ETC>

[Ariel]

Actually i missed you point with the '.' You use cd $directory/.. so you should add: FOLDER=$(basename "$direcory") and then tar -zcvf "/home/$FILENAME" "$FOLDER" --exclude="*cache*" <ETC>

[Ariel]

You may use backticks instead of $(), i can't get them here inside code blocks. The command basename gives you the last part of the $directory path, so the forlder you want to compress as a relative path.

[Kurtcebe Eroglu]

can you take a look to my latest edit?

[don_crissti]

Note that %X = locale's time representation; on some systems that means 11:43:11 AM. So quote your variables (and preferably use %T).

[Kurtcebe Eroglu]

@don_crissti did you see my latest edit? I have changed the FILENAME to FILENAME = "$filename"-$(date +%d-%m-%Y-%T).tgz but still getting the same issue /usr/local/bin/script-task: line 109: FILENAME: command not found tar: Cowardly refusing to create an empty archive why? Any advice?

[Ariel]

@don_crissti good point, i fixed my answer with that improvement now, thanks

[Kurtcebe Eroglu]

What would be the output of this one? The same but using another approach? If so valid too thanks

[don_crissti]

@ReynierPM - the output is as per your question. The difference when using --xform is that you don't have to cd to the parent directory & use relative paths. Only available with gnu tar afaik.

[don_crissti]

@ReynierPM - wrong quoting. It's tar -zcvf /path/to/compressed/"$FILENAME" ... but then your FILENAME="$filename"-$(date +%d-%m-%Y-%T).tgz is wrong too, it has to be FILENAME="$filename-$(date +%d-%m-%Y-%T).tgz"... or if you want to include the date in the argument passed to tar see my answer.