Tar archiving that takes input from a list of files

linux unix archive tar

179,998

Solution 1

Yes:

tar -cvf allfiles.tar -T mylist.txt

Solution 2

Assuming GNU tar (as this is Linux), the -T or --files-from option is what you want.

Solution 3

You can also pipe in the file names which might be useful:

find /path/to/files -name \*.txt | tar -cvf allfiles.tar -T -

Solution 4

Some versions of tar, for example, the default versions on HP-UX (I tested 11.11 and 11.31), do not include a command line option to specify a file list, so a decent work-around is to do this:

tar cvf allfiles.tar $(cat mylist.txt)

Solution 5

On Solaris, you can use the option -I to read the filenames that you would normally state on the command line from a file. In contrast to the command line, this can create tar archives with hundreds of thousands of files (just did that).

So the example would read

tar -cvf allfiles.tar -I mylist.txt

View more solutions

179,998

Danf

Updated on February 02, 2020

Comments

Danf over 4 years
I have a file that contain list of files I want to archive with tar. Let's call it mylist.txt

It contains:
```
file1.txt
file2.txt
...
file10.txt
```
Is there a way I can issue TAR command that takes mylist.txt as input? Something like
```
tar -cvf allfiles.tar -[someoption?] mylist.txt
```
So that it is similar as if I issue this command:
```
tar -cvf allfiles.tar file1.txt file2.txt file10.txt 
```
- Ogre Psalm33 over 10 years
  
  The tar man page is extremely unhelpful for this option (at least on RedHat 5.4 thru 6.3): "-T: get names to extract or create from file F". "Extract or create" sounds like it applies to taking files out of the tar archive, but not putting them in. The -X exclude option survives from the old Unix tar command, but apparently -I (include) did not!
- sam boosalis about 5 years
  
  There are few man pages with an EXAMPLES section, despite it being a standard section. See <unix.stackexchange.com/questions/306189/…>.
drue about 10 years

This option also exists in "bsdtar 3.1.2" on FreeBSD 10.
Stphane over 8 years

What if the .txt files list is really huge ? Should one use xarg command with tar -r.. instead of tar -c.. ?
woot over 8 years

@Stphane Hmm, I don't think the length of the list matters much for this method. In fact, I would imagine this method is better than xargs as xargs will rerun tar over and over to append data, but I haven't really tested the methods side by side.
David C. Rankin almost 8 years

UUoC (unnecessary use of cat), simply $(<mylist.txt).
Andre Holzner over 6 years

note that this may exceed the maximum length of the command line if mylist.txt is large
Mike D about 6 years

in that case tar -T <(<mylist.txt). redundant like this answer.
Stphane almost 6 years

I wish I could put comments in mylist.txt .. is there any workaround using some tar option from inside mylist.txt ?
Matt G over 5 years

@Stphane that's simple, using the --exclude flag allows this. Assuming your comment lines start with a '#', a command such as the following would ignore / exclude any attempted file operations on lines containing cmments, i.e. your command can look like this: tar -cvf allfiles.tar --exclude='^#' -T mylist.txt. Tar reports an error, but when you check your tar archive, there are no errors, and all files from the list are inside your archive.
Nasri Najib almost 5 years

Pointing out that this command is for linux variant and for those on sunOS or other variants, do check out the other answers below. E.g. For sunOS, two alternatives I have tested: tar -cvf file.tar -I list.txt and tar -cvf file.tar $(cat list.txt)
Richard Gomes over 4 years

When a pipe is employed, which is the case here, the operating system creates streams on both sides of the pipe and synchronizes production and consumption of data. The list of files could be infinite. You could tar/gz the entire internet using a Raspberry Pi Zero, given that you have enough storage on the end of the pipe.
Roland about 4 years

-L mylist.txt on AIX
drevicko almost 4 years

@Stpihane if you did eg find /path/to -name '*.txt' -exec tar -cvf all.tar {} \+ you'd be in trouble, yes, as find would execute the tar command multiple times with successive subsets of the files. Only the last subset would be in the archive, as it would be created anew for each tar execution.
Omari Victor Omosa over 3 years

How do you tar and not include full path?
Stalinko over 3 years

My one-liner to use with pigz: find /path/to/files -name '*.log*' | tar -cvf - -T - | pigz -9 > logs.tgz
William almost 3 years

You may also want to add a "--verbatim-files-from" option (before the -T), otherwise items in the list file which start with a dash (like -h or --dereference) will be treated as tar commands.
grenix over 2 years

Just wonder if this would work 'cat mylist.txt | tr '\n' '\0' | xargs -0L1 tar rf allfiles.tar' to avoid exceeding command line length. With 'tar rf' files are appended to archive. -0L1 stands for taking only one line from the file. -0L2, -0L3, ... should also be possible
Maske over 2 years

In debian based the option is -T, --files-from=FILE.