Grep for a string in file without using pipe

Question

I want to grep for a word in a file in the last n lines without using the pipe.

grep <string> filename

enables to search the filename for a string. But, I want to search for a string in the last N lines of the file. Any command to search for that without using the pipe?

Solution 1

If your shell supports it (zsh, bash, some implementations of ksh), you could utilise process substitution

grep <pattern> <(tail -n5 yourfile.txt)

Where -n5 means get the five last lines.

Similarly,

grep <pattern> <(head -n5 yourfile.txt)

would search through the 5 first lines of yourfile.txt.

Explanation

Simply speaking, the substituted process pretends to be a file, which is what grep is expecting. One advantage with process substitution is that you can feed output from multiple commands as input for other commands, like diff in this example.

diff -y <(brew leaves) <(brew list)

This gets rid of the pipe (|) character, but each substitution is in fact creating a pipe¹.

---

_{¹Note that with ksh93 on Linux at least, | does not use a pipe but a socket pair while process substitution does use a pipe (as it's not possible to open a socket):}

$ ksh93 -c 'readlink <(:)'
pipe:[620224]
$ ksh93 -c ': | readlink /proc/self/fd/0'
socket:[621301]

Solution 2

n=$some_num
{   head -n"$(($(wc -l <in)-n))" >/dev/null
    grep 'match your string'
}   <in

Unfortunately this requires reading the file entirely through w/ wc to get a line-count because it's not clear otherwise how many lines are in the file or how large $n is. That aside, this should be a very performant solution provided <in is a regular, lseek()able file.

So first we get our line count and subtract $n from it. head reads in that many lines from stdin and writes the results to /dev/null. What remains afterward are $n-count lines of input on stdin and just your grep and your pattern.

Technically this does cheat - there is a pipe in the command substitution for wc. It is my hope that you can overlook that.

By the way, another way to do this could look like:

{   grep "-m$n" 'some pattern near yours' >/dev/null
    grep 'your pattern'
}   <in

...with a GNU grep. If you can grep $n occurrences of another pattern which will get you in the neighborhood of your targeted pattern, then you might truly do it without a pipe at all.

I tried to stick w/ grep, but here is a sed solution anyway. The pipes below are just for input - and grep is not involved at all excpet to prepend line numbers so you can see which numbers they are. All of that is only for the example case. You can use the sed script alone with a named file or stdin of any kind and set $pat and $n appropriately and it will work.

I actually just rewrote this because I didn't like not being able to anchor the matches. This is a little slower - not noticeably, and it is still very fast, but for each buffered tail line it trims all of trailing pattern space and isolates the first line in the buffer. In this way all of the normal anchor expressions work as expected.

pat=man n=40
man man   |
grep -n ''|
sed -e:B -e'${/^\n/D'  \
    -eh  -e's/\n.*//'  \
         -e"/$pat/p;x" \
    -e\} -e'$D;N;$bB'  \
         -e"$n,$ D;bB"

---

648:       /etc/man_db.conf
649:              man-db configuration file.
651:       /usr/share/man
652:              A global manual page hierarchy.
654:       /usr/share/man/index.(bt|db|dir|pag)
657:       /var/cache/man/index.(bt|db|dir|pag)
661:       apropos(1), groff(1), less(1), manpath(1),  nroff(1),  troff(1),  whatis(1),
662:       zsoelim(1),  setlocale(3),  manpath(5),  ascii(7),  latin1(7),  man(7), cat-
663:       man(8), mandb(8), the man-db package manual, FSSTND
680:       developing and maintaining man-db.

Here is another example, but on a file:

pat=. n=15
seq 100 >nums
sed -e:B -e'${/^\n/D'  \
    -eh  -e's/\n.*//'  \
         -e"/$pat/p;x" \
    -e\} -e'$D;N;$bB'  \
<nums    -e"$n,$ D;bB"

---

86
87
88
89
90
91
92
93
94
95
96
97
98
99
100

Solution 3

Why do you want to avoid pipe?

If you really want to avoid pipe, then you will have to run two commands:

tail -N filename > filename.tmp
grep "string" filename.tmp

(when N is the last number of lines)

Solution 4

You can do that with awk and a little bit of help:

$ N=8
$ awk -v start_line="$(( $(wc -l < alphabet) - N + 1 ))" 'NR>=start_line  &&  /e/' alphabet
sierra
whiskey
yankee
$

finds all lines containing e in the last 8 lines of the phonetic alphabet.  This has the drawback that it reads the entire input file twice.

View more solutions

Comments

[dave_thompson_085]

awk doesn't need to read twice as long as the last N lines can be held in memory -- or in this case, the selected lines from the last N: awk -vN=whatever '{delete x[NR-N]} /pattern/{x[NR]=$0} END{for(i=NR-N;++i<=NR;){if(i in x)print x[i]}}' file. (If the total number of lines can exceed 9e15 you need to add some %'s and wraparound, but at present you can't even have a file much larger than 1e12 characters.)

[Scott - Слава Україні]

@dave_thompson_085: Clever!  That's different from everything else on this page; I suggest that you post it as an answer.  (But do make it a complete answer to the question, including the string matching requirement.)

[Stéphane Chazelas]

Note that $(...) uses a pipe.