Deleting Lines Containing a Specified String in Different Files

Question

Suppose there are two files in web/ named foo.php and bar.php. The 1st line of foo.php is "sdajgeSTRINGdsad" and the 10th line of bar.php is "gdfhu98324STRING". The task is to first locate these two files, then delete 1st line of foo.php and 10th line of bar.php given that the specified string is "STRING".

Solution 1

sed can do that :

sed -i.bak '/STRING/d' web/*

Solution 2

To find a file containing a string, use grep. If you want to find which files contain STRING in web/ and all subdirectories:

grep -R "STRING" web/

To edit a file and remove lines containing a string, there are many options but here are two common ones. Since you want to operate on entire lines this is really easy :)

First you could use grep with the -v argument to reverse the search and hide the matched lines from the output instead of showing them. For your first file, let's do this using sponge. This will allow us to read your file, filter it, then write it back to the same file all in one motion:

grep -v "sdajgeSTRINGdsad" foo.php | sponge foo.php

For your second example we'll do the same thing with grep, but if you don't have sponge you can write it to a different file, then move that file back to the original name:

grep -v "gdfhu98324STRING" bar.php  > bar.php.cleaned
mv bar.php{.cleaned,}

The second way would be to use sed with the d line delete command. For your first example we will use the in-place file edit with the option to create a .bk backup file:

sed -i.bk '/sdajgeSTRINGdsad/d'  foo.php

Pretty much the same thing could be accomplished by copying the file yourself then using sed to filter the backup and over-write the original:

cp bar.php bar.php.bk
sed '/gdfhu98324STRING/d' bar.php.bk > bar.php

Now to combine these two tasks! Let's make a function so you can do this repeatedly with different strings:

function nuke_string () {
    string="$1"
    path="$2"
    grep -R -l "$string" "$path" | while IFS= read -r file; do
        sed -i.bk "/$string/d" "$file"
    done
}

Once you create that function, you could use it like this:

nuke_string "STRING" web/

Any files containing STRING in web/ would then have those lines removed and a backup copy would be made of the original version. You can create a function like this in your current shell or you can write it to your .bashrc file and have it always available.

Solution 3

Just make use of grep -l together with -R! Here's a complete one-liner:

grep -Rl "STRING" web/ | xargs sed -i "/STRING/d"

Comments

[Rommel Castro]

Thank you, Caleb. I think I have to apologize for I have not made myself clear. My problem is that I don't know which files (and how many of them) contain a specified string, which is exactly "STRING" but not "gdfhu98324STRING" or "sdajgeSTRINGdsad". So the method has to first locate or find those files that contain the string, then modify them.

[tcoolspy]

@ZonghengYang: Ya sorry I missed the big about finding the files until I went to edit your question to clean it up. See my edited answer.

[Rommel Castro]

Thanks, tuxce. How ever it does not work as expected. Should I remove the "*" at the end in order to recursively search all files and subfolders under web?

[tcoolspy]

@ZonghengYang: To recursivly search with this method you will have to change your shell globing method so you can use **, a glob that matches recursively. First run this: shopt -s extglob, then use web/**.

[tcoolspy]

This will create backup files of every file matching the glob even if a match not found and no change is made to the file! If you want be more judicious you will have to do the file and edit operations in separate steps.

[tuxce]

Try: grep -Zlr STRING web | xargs -0 sed -i.bak '/STRING/d' (Edit: I didn't see previous comment, extglob saves 2 process)

[tcoolspy]

@tuxce: It saves two processes and makes a mess of the directory and wastes a lot of disk space. Note that sed makes backups even if it doesn't change anything in a file. Your grep | xargs sed is a better solution, and does basically what I wrote a function for.

[tcoolspy]

find is not an appropriate tool to find files based on their contents, which is what the OP is asking here. It works best to identify files by their properties.

[mouviciel]

I didn't mean to use find for filtering by contents, but by file extension. I assumed that what interests OP is locating php files, that are more likely to contain STRING, then apply the sed command for deleting matching lines.

[Gilles 'SO- stop being evil']

With the usual dangers of xargs, i.e. it won't work if any of the file names contains whitespace or \'".

[rozcietrzewiacz]

@Gilles, True, but it should not be a problem in a sane directory tree as I understand web/ should be.

[Vicky Dev]

Well for more safer side, you should also include file-type check, so it is removed only from those type of files, example '.csv'