How do I grep multiple patterns from a pipe

shell grep regular-expression quoting

20,492

Solution 1

Try:

pip3 list | grep -Ei 'foo|bar|baz'

Here is a real life example from my Arch server:

pip3 list | grep -Ei 'ufw|set'
setuptools 40.0.0 
ufw        0.35

OS and grep info:

uname -a
Linux archlinux 4.16.6-1-ARCH #1 SMP PREEMPT Mon Apr 30 12:30:03 UTC 2018 x86_64 GNU/Linux

grep --version
grep (GNU grep) 3.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Mike Haertel and others, see <http://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.

Solution 2

The reason

grep -ei foo -ei bar -ei baz

does not work is because the semantics for the -e option is -e PATTERN, as in

grep -i -e foo -e bar -e baz

... which is what the command should have looked like. The -i option (for case insensitive matching) will only need to be specified once and will affect all patterns.

With -ei foo you ask grep to look for the pattern i in the file foo.

The "broken pipe" error comes from pip3 trying to write to the end of a dead pipe. The pipe is dead because grep could not find the files foo, bar or baz, and terminated (with three "file not found" errors). The traceback is from pip3 which is a Python program (so it tells you exactly where in the Python code the fault happened on its side).

20,492

bit

Updated on September 18, 2022

Comments

bit over 1 year
I want to find three patterns in a list. I tried typing
```
$ pip3 list | grep -ei foo -ei bar -ei baz
```
but the shell throws a broken pipe error and a large Traceback.

How do I grep for multiple patterns passed from a list that is piped to grep?
- smw almost 6 years
  
  That should probably be grep -ie foo -e bar -e baz, no? the expressions should follow the -e
- bit almost 6 years
  
  @steeldriver can you clarify? I'm not sure what you mean.
- smw almost 6 years
  
  Try pip3 list | grep -ei - you're asking grep to find all lines matching i
- bit almost 6 years
  
  Your 1st comment worked but your 2nd didn't. Very strange, I get another broken pipe error and a very large Traceback for grep -ei but grep -ie works like the other answers below.
- smw almost 6 years
  
  That was my point - the order of the i and e matters (because i is just a switch, whereas e expects an argument)
- bit almost 6 years
  
  Ahh now its clear why -e must be last and also why -i is needed only once, while -e is needed for every pattern. Good answer.
multithr3at3d almost 6 years

Note that egrep is deprecated in favor of grep -E, for what it's worth.
jesse_b almost 6 years

@multithr3at3d: Depends on the system. Solaris for example does not have a grep -E and therefore your only option is egrep. Parenthesis are not required though.
maulinglawns almost 6 years

@multithr3at3d Umm... Ok. That was news to me. I have used egrep forever, and still do.
multithr3at3d almost 6 years

@Jesse_b you're right, my comment is in reference to GNU grep. @maulinglawns it still works for historical compatibility
bit almost 6 years

can confirm egrep and grep -E both work.
terdon almost 6 years

@Jesse_b really? Could that be a particularly old version of Solaris, or is that the norm? -E is specified by POSIX and even the POSIX specs suggest that egrep is historical.
jesse_b almost 6 years

@terdon: At least up to v9
bit almost 6 years

@maulinglawns is it important that the patterns are protected by single quotations or would double quotations suffice?
bit almost 6 years

Really good explanation, thanks. Related question: if -ei foo is asking grep to look for the pattern i in the file foo then why doesn't it just return a pattern or file not found error, instead of filling up my screen with a Traceback for a broken pipe error.
Kusalananda almost 6 years

@MyWrathAcademia Use single quotes for static strings. only use double quotes if you need the shell to expand anything in the strings, like in -e "something$myvar", where you may want it to substitute the value of $myvar. If the pattern contains any $ that shouldn't be expanded by the shell, these need to be escaped as \$ if you're using double quotes.
maulinglawns almost 6 years

Thank you @terdon and @multithr3at3d for enlightening me on grep -E vs egrep. I will try to refrain from the latter in the future, but it will take some getting used to!
bit almost 6 years

thanks @Kusalananda I will keep that in mind, I thought may be it was needed to prevent the | from expanding.
Kusalananda almost 6 years

@MyWrathAcademia grep should have given you a "file not found" error (look at the top, before the traceback). Python programs often give tracebacks on errors. Note that it's pip3 that gives you the traceback, not grep. It does that because you've asked to write to a pipe that nobody is listening at the other end of (grep already died because it couldn't open the files you told it to look in).
Kusalananda almost 6 years

@MyWrathAcademia The pipe symbol is not special in any way inside a quoted string of any type.
bit almost 6 years

Your right, I didn't see the grep: foo: No such file or directory error before because it was lost in the sea of information, I can't see the others, may be they are there somewhere. Thanks for imparting the knowledge. One more thing, is this Traceback saved anywhere so I know if I have to delete it or is it just passed to stdoutput only?
Kusalananda almost 6 years

@MyWrathAcademia The traceback from pip3 is sent to its standard error stream. It is not saved anywhere on file by default.