Using grep with pipe and ampersand to filter errors from find

Question

I am using cygwin to find a file on the cygdrive.

However I need to suppress the permission denied messages (otherwise the results get hidden in the error messages). The following command works:

find -name 'myfile.*' |& grep -v "Permission denied"

I don't understand why the ampersand needs to be put into this command, would have expected this to work but it doesn't.

find -name 'myfile.*' | grep -v "Permission denied"

Please explain the meaning of the ampersand.

Solution 1

In Unix-like systems, there are two output paths that if left unmodified will send output to your screen. Standard error (or stderr) is the one that captures most failures and error conditions.

To pass the permission denied message in the stderr to the same output stream as "regular output" you must combine the two. In your example, in order for your grep -v to properly operate on it, you combine stdout (standard output) and stderr with the arcane syntax you see.

From GNU Bash manual section 3.2.2 Pipelines:

If ‘|&’ is used, command1’s standard error, in addition to its standard output, is connected to command2’s standard input through the pipe; it is shorthand for 2>&1 |. This implicit redirection of the standard error to the standard output is performed after any redirections specified by the command.

Also, as geirha points out, if you want to just get rid of stderr output, you would want to do something like

find -name 'myfile.*' 2> /dev/null

or perhaps

find -name 'myfile.*' 2> /tmp/errorlog

And note that if you have strings of commands, such as a find passing its output to xargs you would need to put the entire pipeline of commands in parentheses to capture the output from all components of the command. E.g.,

(find | egrep ^[RS].[0-9]+/.svg] | xargs head -1 )  2> /dev/null

If you left out the parentheses, and did this instead --

find | egrep ^[RS].[0-9]+/.svg] | xargs head -1 2> /dev/null

you would still see permission denied errors from the find or egrep, but stderr would be redirected for xargs.

As you've seen, you would likely throw away the stderr only after viewing its contents during a test run.

Note that with GNU find and as far as I can tell, any POSIX-compliant find, the -print option is implicit. You can still supply it explicitly if you like.

Solution 2

Error messages are written to stderr, not stdout, but | pipes only stdout.

You probably want |&, which pipes stderr as well as stdout.

Solution 3

If you want to ignore the error messages, just redirect stderr to /dev/null.

find . -name 'myfile.*' -print 2>/dev/null

Also, consider reading http://mywiki.wooledge.org/UsingFind.

Comments

[Luke Exton]

Seems to be only bash4+ wiki.bash-hackers.org/bash4#redirection

[Eliah Kagan]

@LukeExton Yes. In other shells, 2>&1 | may be used in place of |& (i.e., one can explicitly redirect stderr to stdout and then pipe that to the next command in the pipeline).

[user1133275]

2> >( grep filter ) is also handy