Remove line containing certain string and the following line

Question

I use this

cat foo.txt | sed '/bar/d'

to remove lines containing the string bar in the file.

I would like however to remove those lines and the line directly after it. Preferably in sed, awk or other tool that's available in MinGW32.

It's a kind of reverse of what I can get in grep with -A and -B to print matching lines as well as lines before/after the matched line.

Is there any easy way to achieve it?

Solution 1

If you have GNU sed (so non-embedded Linux or Cygwin):

sed '/bar/,+1 d'

If you have bar on two consecutive lines, this will delete the second line without analyzing it. For example, if you have a 3-line file bar/bar/foo, the foo line will stay.

Solution 2

If bar may occur on consecutive lines, you could do:

awk '/bar/{n=2}; n {n--; next}; 1' < infile > outfile

which can be adapted to delete more than 2 lines by changing the 2 above with the number of lines to delete including the matching one.

If not, it's easily done in sed with @MichaelRollins' solution or:

sed '/bar/,/^/d' < infile > outfile

Solution 3

I am not fluent in sed, but it is easy to do so in awk:

awk '/bar/{getline;next} 1' foo.txt 

The awk script reads: for a line containing bar, get the next line (getline), then skip all subsequent processing (next). The 1 pattern at the end prints the remaining lines.

Update

As pointed out in the comment, the above solution did not work with consecutive bar. Here is a revised solution, which takes it into consideration:

awk '/bar/ {while (/bar/ && getline>0) ; next} 1' foo.txt 

We now keep reading to skip all the /bar/ lines.

Solution 4

You will want to make use of sed's scripting capabilities to accomplish this.

$ sed -e '/bar/ { 
 $!N
 d
 }' sample1.txt

Sample data:

$ cat sample1.txt 
foo
bar
biz
baz
buz

The "N" command appends the next line of input into the pattern space. This combined with the line from the pattern match (/bar/) will be the lines that you wish to delete. You can then delete normally with the "d" command.

Solution 5

If any line immediately following a match should be removed then your sed program will have to consider consecutive matches. In other words, if you remove a line following a match which also matches, then probably you should remove the line following that as well.

It is implemented simply enough - but you have to look-behind a little.

printf %s\\n     0 match 2 match match \
                 5 6 match match match \
                 10 11 12 match 14 15  |
sed -ne'x;/match/!{g;//!p;}'

---

0
6
11
12
15

It works by swapping hold and pattern spaces for each line read in - so the last line can be compared to the current each time. So when sed reads a line it exchanges the contents of its buffers - and the previous line is then the contents of its edit buffer, while the current line is put in hold space.

So sed checks the previous line for a match to match, and if its ! not found the two expressions in the { function } are run. sed will get the hold space by overwriting the pattern space - which means the current line is then in both the hold and pattern spaces - and then it will // check it for a match to its most recently compiled regular expression - match - and if it does not match it is printed.

This means a line is only printed if it does not match and the immediately previous line does not match. It also foregoes any unnecessary swaps for sequences of matches.

If you wanted a version that could drop an arbitrary number of lines occurring after a match it would need a little more work:

printf %s\\n    1 2 3 4 match  \
                match match 8  \
                9 10 11 12 13  \
                14 match match \
                17 18 19 20 21 |
sed -net -e'/match/{h;n;//h;//!H;G;s/\n/&/5;D;}' -ep

...replace the 5 with the number of lines (including the matched line) that you would like to remove...

---

1
2
3
4
12
13
14
21

View more solutions

Comments

[jakub.g]

Just for information: I'm analyzing logs in which entries are two-liners. So I want to find an entry matching the pattern and remove it as well as the next line. Hence I don't need to handle consecutive match lines, but thanks anyway for the completeness of your answers!

[jw013]

To replicate grep -A 100%, you also need to handle any number of consecutive bar lines correctly (by removing the whole block and 1 line after).

[jakub.g]

+1 for the length :) In my particular example I don't have consecutive bars so this one is super easy to remember.

[jakub.g]

The other plus in the AWK solution is that I can replace /bar/ with /bar|baz|whatever/. In sed that syntax doesn't seem to work.

[minorcaseDev]

@jakub.g: with GNU sed: sed -e '/bar/{N;d}' sample1.txt

[AJP]

sed '/bar/d' if you just want to "Remove line containing certain string" and not the next.

[Pandya]

If I want to remove all the lines after math then?

[Pandya]

@Gilles Thanks that worked, but I might have done some mistake and asked the question.

[Tom Saleeba]

To break down the command: the first part /bar/,+1/ is a sed address (gnu.org/software/sed/manual/sed.html#sed-addresses) that says start at the regexp /bar/, then a , as a separator between start/end addresses, and end +1 lines (from the start). Then the d is the command (delete)

[A. K.]

I actually wanted to delete the matching line so I just replaced +1 with +0 and it worked! like: sed '/bar/,+0 d'

[Gilles 'SO- stop being evil']

@A.K. If you just want to delete the matching line, it's even simpler: sed '/bar/d'

[Victor Yarema]

@jakub.g , I have GNU sed (v4.4 now). Not sure about the others. What I know is that it uses "basic" regular expression syntax by default this is why your example didn't work. To achieve what you want you can either put a backslash in front of each vertical line or you can ask sed to use "extended" regular expressions. More information here: gnu.org/software/sed/manual/html_node/… . Please note that this is applicable to grep as well. Here's my own working example: echo $'0a\n1b\n2c' | sed '/0a\|1b/d'.