Delete lines that come after a line with a specific pattern in Shell

Question

I'm trying to delete all lines comes after a specific pattern in files.

I have many files, which all have the same structure:

Example:

file1

line 1
...
line x "here there is a specific pattern"
...
EOF

file n

line 1
...
line x "here there is a specific pattern"
...
EOF

I tried to get a simple solution, but since I have many files, I proceed with a long way :p

The pattern appears one time in each file.

So, I got all lines number which contains this pattern, and save in one file.

this is my code:

count=$(ls -f path_to_folder/*.txt | wc -l)
echo "Number of txt file : $count"
###


    for ((i=1;i < $count+1 ;i++))

    {

    vt=$(grep -n PATTERN $i.txt | cut -d : -f 1)
    echo $vt >> PATTERN_line.txt

    }

Every line in PATTERN_line.txt contains the line number, in each file, where the pattern exists.

Now, I'm trying to use those numbers to delete all lines that come after the pattern to the file end.

This means I need to keep the file from the head to the patten line which must be included.

I appreciate your help

Solution 1

This is very trivial with text processing utilities. For example, using sed:

sed '1,/pattern/!d' file

Meaning, match every line from the first one to the one with pattern and delete all the non-matched lines. So, replace pattern with your pattern. If it contains /, you need to escape those characters. For example, if the pattern is pattern-with/character:

sed '1,/pattern-with\/character/!d' file

To actually edit files (rather than print the edited stream to stdout), you can use the -i flag:

sed -i '1,/pattern/!d' file

You can make a backup of the original file by adding an extension for the old file to -i. Take care here - you must not include a space before the extension.

sed -i.backup '1,/pattern/!d' file

sed takes multiple filename arguments. For example, to act on all the non-hidden files in the current directory you could use:

sed -i '1,/pattern/!d' *

Solution 2

Another solution, using awk:

awk '/specific-pattern/{stop=1} stop==0{print}' < input_file >> output_file

While the variable stop is 0 (which is is, by default), awk will print the current line. However, if the current line matches the regular expression /specific-pattern/, then stop will be set to 1. This makes stop==0 untrue, so awk will no longer execute the print statement.

Input is read from input_file and appended to output_file.

If you want to keep the line with the pattern, reverse the two parts of the awk script.

Solution 3

Thank you @Zanna

I found this solution :

for ((i=1;i < $count+1 ;i++)) 

sed -n '/PATTERN/q;p' $i.txt > file_out$i.txt

Thank you

Comments

[Katu]

Have a look at this and if the answers don't solve your problem, please edit it explaining why stackoverflow.com/questions/5227295/…

[Zanna]

@Katu since this cannot be closed as a duplicate of that question, which is on another SE site, there is not much benefit in OP editing this question to show how it differs from that one.

[MGM]

@Katu , yes I got the solution from this question

[Katu]

@Zanna What is the correct thing to do in this situation? As you said, can't be a cross-site duplicate. Should I copy and paste the accepted answer in stackoverflow as an answer to this question with a reference link?

[Zanna]

@Katu posting the link in a comment is enough. You can answer this question here however you wish! You can post content from the answers there blockquoted with appropriate attribution if you think it would be useful to do that.

[Zanna]

I would expect that to delete the line with the pattern, but I thought you wanted to keep that line...

[Zanna]

grep has a -q flag (see man grep) and checking exit status is exactly what if does (see help if) so you can write things like if grep -q "$2"; then ...

[Robin Winslow]

How can we delete the matched line as well?

[Zanna]

@RobinWinslow that can be done with sed -n '/pattern/q;p'

[Nathan Wallace]

sed '/pattern/q' works just the same as sed '1,/pattern/!d', right? Also, sed -n '/pattern/q:p' isnt' working for me (GNU sed 4.8). What's the :p part supposed to do? Edit Oh, I see from @mgm you probably meant ;p, right? I see that sed -n '/pattern/q;p' deletes all lines before the matched pattern.

[Zanna]

@NathanWallace yes I meant ; and not : (I edited my comment to fix it). Not sure about '/pattern/q' but, it does seem to do the same thing (I have never really learned to use q properly). I must do some sed practice...

[Nathan Wallace]

Your answers and comments here have taught me a lot! The q command says "when you get a match, stop processing" the file, which is why it works for the original question. Your solution if you want to exclude the matching line works, I think, because the -n silences normal output, but the p puts it back for every line that doesn't match the pattern