How to use 7z to archive all the files and directories (including hidden ones) in a directory?

Question

Because of specifics of my archiving needs I am not comfortable with solid tar.gz archives and use 7z instead.

I use the following command to do this:

7z a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=off ~/my/folder.7z ~/my/folder/*

To create an archive of everything inside ~/my/folder/ as the ~/my/folder.7z file.

But ~/my/folder/.hiddenFolder doesm't get into the archive then. How to fix this? Isn't * supposed to return all the files and folders?

Solution 1

If you want the contents of a single directory, an easy method is to change to it first:

cd ~/my/folder
7z a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=off ~/my/folder.7z .

What you saw is that * expands to the list of names of files that don't begin with a .. That's the documented behavior, and it's the main reason why files whose name begins with a . are said to be hidden (the other is that ls doesn't show them by default).

There's no really convenient portable way to list all files in a directory. You can use

~/my/folder/..?* ~/my/folder/.[!.]* ~/my/folder/*

but if there is no file matching one of the patterns then the pattern will remain unexpanded. In bash, you can set the dotglob option to avoid treating a leading . specially (. and .. are still excluded from the matches):

shopt -s dotglob
7z a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=off ~/my/folder.7z ~/my/folder/*

In ksh, or in bash if you set the extglob option (or in zsh if you set the ksh_glob option), you can write a pattern that matches all files except . and ..:

7z a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=off ~/my/folder.7z ~/my/folder/@(..?*|.[!.]*|*)

In zsh, there's a simpler way of saying that . must not be treated specially in a pattern:

7z a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=off ~/my/folder.7z ~/my/folder/*(D)

Solution 2

TL;DR

7z a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=off ~/my/folder.7z ~/my/folder/.

More examples

Example directory structure

test1
├── .hidden
└── normal.txt

0 directories, 2 files

Try following commands

1. Root folder with all its contents.

   7za a test1_a.7z ~/test1/
   

   gives

       Date      Time    Attr         Size   Compressed  Name
   ------------------- ----- ------------ ------------  ------------------------
   2017-08-06 09:23:51 D....            0            0  test1
   2017-08-06 09:23:44 ....A            0            0  test1/.hidden
   2017-08-06 09:23:51 ....A            0            0  test1/normal.txt
   ------------------- ----- ------------ ------------  ------------------------
   2017-08-06 09:23:51                  0            0  2 files, 1 folders
   

2. No root folder and no hidden files

   7za a test1_b.7z ~/test1/*
   

   gives

      Date      Time    Attr         Size   Compressed  Name
   ------------------- ----- ------------ ------------  ------------------------
   2017-08-06 09:23:51 ....A            0            0  normal.txt
   ------------------- ----- ------------ ------------  ------------------------
   2017-08-06 09:23:51                  0            0  1 files
   

3. No root folder but hidden files are included (it's what we usually want)

   7za a test1_c.7z ~/test1/.
   

   gives

      Date      Time    Attr         Size   Compressed  Name
   ------------------- ----- ------------ ------------  ------------------------
   2017-08-06 09:23:44 ....A            0            0  .hidden
   2017-08-06 09:23:51 ....A            0            0  normal.txt
   ------------------- ----- ------------ ------------  ------------------------
   2017-08-06 09:23:51                  0            0  2 files
   

Solution 3

No, * is not supposed to return all files. It returns only visible ones.

The easier solution is:

cd ~/my/folder
7z a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=off ~/my/folder.7z .

Comments

[enzotib]

You are using bash path expansion by last *, try removing it, 7z should archive the whole directory.

[Klaws]

Had a similar problem, solved it by placing single quotes around the path. That way, the * is handled by 7z (and not by bash).

[will]

Yes just tested this and it is much tidier than using shopt.

[Marcin]

Best answer here. Thank you.