Delete range of lines above pattern with sed (or awk)

Question

I have the following code that will remove lines with the pattern banana and 2 lines after it:

sed '/banana/I,+2 d' file

So far, so good! But I need it to remove 2 lines before banana, but I can't get it with a “minus sign” or whatever (similar to what grep -v -B2 banana file should do but doesn't):

teresaejunior@localhost ~ > LC_ALL=C sed '-2,/banana/I d' file
sed: invalid option -- '2'
teresaejunior@localhost ~ > LC_ALL=C sed '/banana/I,-2 d' file
sed: -e expression #1, char 16: unexpected `,'
teresaejunior@localhost ~ > LC_ALL=C sed '/banana/I,2- d' file
sed: -e expression #1, char 17: unknown command: `-'

Solution 1

Sed doesn't backtrack: once it's processed a line, it's done. So “find a line and print the previous N lines” isn't going to work as is, unlike “find a line and print the next N lines” which is easy to graft on.

If the file isn't too long, since you seem to be ok with GNU extensions, you can use tac to reverse the lines of the file.

tac | sed '/banana/I,+2 d' | tac

Another angle of attack is to maintain a sliding window in a tool like awk. Adapting from Is there any alternative to grep's -A -B -C switches (to print few lines before and after )? (warning: minimally tested):

#!/bin/sh
{ "exec" "awk" "-f" "$0" "$@"; } # -*-awk-*-
# The array h contains the history of lines that are eligible for being "before" lines.
# The variable skip contains the number of lines to skip.
skip { --skip }
match($0, pattern) { skip = before + after }
NR > before && !skip { print NR h[NR-before] }
{ delete h[NR-before]; h[NR] = $0 }
END { if (!skip) {for (i=NR-before+1; i<=NR; i++) print h[i]} }

Usage: /path/to/script -v pattern='banana' -v before=2

Solution 2

This is pretty easy with ex or vim -e

    vim -e - $file <<@@@
g/banana/.-2,.d
wq
@@@

The expression reads: for every line containing banana in the range from the current line -2 to the current line, delete.

What's cool is that the range can also contain backwards and forwards searches, for example this will delete all sections of the file starting with a line containing apple and ending with a line containing orange and containing a line with banana:

    vim -e - $file <<@@@
g/banana/?apple?,/orange/d
wq
@@@

Also note that up to ten vim/ex commands can be submitted using the inline command option "-c". See the man page.

vim -e -c 'g/banana/.-2,.d' -c 'wq' $yourfilename

and

ex -c 'g/banana/?apple?,/orange/d' -c 'wq' $yourfilename 

Solution 3

You can do this fairly simply with sed:

printf %s\\n    1 2 3 4match 5match 6 \
                7match 8 9 10 11match |
sed -e'1N;$!N;/\n.*match/!P;D'

I don't know why anyone would say otherwise, but to find a line and print previous lines sed incorporates the built-in Print primitive which writes only up to the first \newline character in pattern space. The complementary Delete primitive removes that same segment of pattern space before recursively recycling the script with what remains. And to round it off, there is a primitive for appending the Next input line to pattern space following an inserted \newline character.

So that one line of sed should be all you need. You just replace match with whatever your regexp is and you're golden. That should be a very fast solution as well.

Note also that it will correctly count a match immediately preceding another match as both a trigger to quiet output for the previous two lines and quiet its print as well:

---

1
7match
8
11match

In order for it to work for an arbitrary number of lines, all you need to do is get a lead.

So:

    printf %s\\n     1 2 3 4 5 6 7match     \
                     8match 9match 10match  \
                     11match 12 13 14 15 16 \
                     17 18 19 20match       |
    sed -e:b -e'$!{N;2,5bb' -e\} -e'/\n.*match/!P;D'

---

1
11match
12
13
14
20match

...deletes the 5 lines preceding any match.

Solution 4

Using the "sliding window" in perl:

perl -ne 'push @lines, $_;
          splice @lines, 0, 3 if /banana/;
          print shift @lines if @lines > 2
          }{ print @lines;'

Solution 5

Using man 1 ed:

str='
1
2
3
banana
4
5
6
banana
8
9
10
'

# using Bash
cat <<-'EOF' | ed -s <(echo "$str")  | sed -e '1{/^$/d;}' -e '2{/^$/d;}'
H
0i


.
,g/banana/km\
'm-2,'md
,p
q
EOF

View more solutions

Comments

[manatwork]

The easiest is to load all the data into an array, skip the undesired lines then output what remains: awk '{l[m=NR]=$0}/banana/{for(i=NR-2;i<=NR;i++)delete l[i]}END{for(i=1;i<=m;i++)if(i in l)print l[i]}'. This is not efficient, so this is just a hint, not a solution.

[angus]

Just do tac file | sed ... | tac. :P

[Rizwan Khan]

@angus I didn't think about it ;)

[Rohit Kapur]

you could have done sed '/banana/,+2d' file that will also work

[Luv2code]

If you're open to using awk, it's pretty simple: awk 'tolower($0)~/bandana/{print prev[!idx];print prev[idx]} {idx=!idx;prev[idx]=$0}' filein Since this is a comment and not an answer (there are already other answers), I won't go into too much detail, but the crux of it is you always have the previous two records in prev[0] and prev[1], the "freshest" depending on which iteration but always in prev[idx], so when you print, you print in !idx then idx order. Regardless, alternate idx and put the current record in prev[idx].

[lav]

A good read with many examples: sed.sourceforge.net/grabbag/tutorials/sedfaq.txt

[jw013]

sed can do sliding windows too, but the resulting script is typically so unreadable that it's easier to just use awk.

[Peter.O]

@Gilles.. The awk script is not quite right; as-is it prints blank lines and misses the last lines. This seems to fix it, but it may not be ideal or right itself: if (NR-before in h) { print...; delete...; } ... and in the END section: for (i in h) print h[i] ... Also, the awk script prints the matching line, but the tac/sec version does not; but the question is a bit ambiguous on this.. The "original" awk script, to which you provided a link, works fine.. I like it... I'm not sure how the above 'mod' affects the print after lines...

[Gilles 'SO- stop being evil']

@Peter.O Thanks, the awk script should be better now. And it took me less than 6–8 years!

[soMuchToLearnAndShare]

what does -e:b here do? I was trying to delete 4 lines of stuff off my xml file which has <tag> something I want to match; next line something i want to match; </tag> and i could not get it working

[Timo]

Regarding your line sed -e'1N;$!N;/\n.*match/!P;D', if I want to have just one line above I would remove the 1N. The !N must remain because of the \n?