Is it possible to make zcat output text even if it's uncompressed?

Question

The problem is I have some database dumps which are either compressed or in plain text. There is no difference in file extension etc. Using zcat on uncompressed files produces an error instead of the output.

Is there maybe another cat sort of tool that is smart enough to detect what type of input it gets?

Solution 1

Just add the -f option.

$ echo foo | tee file | gzip > file.gz
$ zcat file file.gz
gzip: file: not in gzip format
foo
$ zcat -f file file.gz
foo
foo

(use gzip -dcf instead of zcat -f if your zcat is not the GNU (or GNU-emulated like in modern BSDs) one and only knows about .Z files).

Solution 2

One portable, simple suggestion would be to use zgrep instead of zcat, and just use a search pattern that matches every line.

zgrep $ some-file

Unlike zcat, zgrep will happily handle uncompressed files. From man zgrep:

zgrep - search possibly compressed files for a regular expression

Solution 3

With GNU gzip you can do zcat file 2> /dev/null || cat file. This is not POSIX-standard, and does not work on BSD gzip, you really should fix your system so that all gzipped files have the .gz extension (of course plain text files may have any extension, including .gz).

Solution 4

To add my conclusion from the comments as an answer, I think the best most compatible way is to use

gzip -cdf [ name ... ]

This is also how zless and zgrep do it internally.

View more solutions

Comments

[Seth]

What file extension are they? Is there any way I could get some examples to play around with?

[mikeserv]

Just use zcat on only the compressed dumps.

[rsk82]

Yea, but I don't know which ones are compressed without manually checking, some of them are *.gz some not and that is the problem. May I rephrase the question, how to check if file is gzipped ? and use that information in next command ?

[Seth]

@rsk82 But you just said they all have the same extension.. So you mean they all have .gz but only some of those are actually compressed? The others are just plain text?

[mikeserv]

Well - that sounds like your problem. Whatever system you've setup that provides that kind of output needs revising. In the meantime, GNU grep can be instructed what to do if it encounters a binary type file - and that might make a good filter for the cleanup.

[mikeserv]

You should add an || { echo "$filename" ; cat $file ; } >&2 to cat to keep the out streams separate so the asker can more easily clean up the mess. I mean - well, I hope that's clear enough.... But this is a good answer.

[mikeserv]

What do you mean by portable? If you mean that it can be installed and used on any system then isn't is as portable as any other?

[godlygeek]

I mean that zcat and zgrep are normally packaged together, so this ought to work anywhere where zcat was available to begin with. And it's agnostic to the particular shell being used - ought to work fine in bash, zsh, and even csh or Solaris's non-POSIX bourne /bin/sh.

[mikeserv]

Oh, cool. I noticed I had both - but I didn't know they came together. Thanks. You've got my vote.

[Stéphane Chazelas]

zgrep is a script that wraps around gzip and grep. The reason it works with uncompressed data is because it passes the -f option to gzip

[Stéphane Chazelas]

Also note that as far as portability goes, zcat was first only dealing with .Z files. GNU came up with gzip and the gz format later and its gzip/zcat handles both .Z and .gz files. You'll probably still find commercial Unices where zcat knows nothing about gz files.

[ajkal]

One correction: you need the c parameter for gzip to get the output to stdout: gzip -cdf. But zcat -f works also fine in Mac OS if you use stdin: zcat -f < file.