Using grep and sed to find and replace a string

Question

I am using the following to search a directory recursively for specific string and replace it with another:

grep -rl oldstr path | xargs sed -i 's/oldstr/newstr/g'

This works okay. The only problem is that if the string doesn't exist then sed fails because it doesn't get any arguments. This is a problem for me since i'm running this automatically with ANT and the build fails since sed fails.

Is there a way to make it fail-proof in case the string is not found?

I'm interested in a one line simple solution I can use (not necessarily with grep or sed but with common unix commands like these).

Solution 1

You can use find and -exec directly into sed rather than first locating oldstr with grep. It's maybe a bit less efficient, but that might not be important. This way, the sed replacement is executed over all files listed by find, but if oldstr isn't there it obviously won't operate on it.

find /path -type f -exec sed -i 's/oldstr/newstr/g' {} \;

Solution 2

Your solution is ok. only try it in this way:

files=$(grep -rl oldstr path) && echo $files | xargs sed....

so execute the xargs only when grep return 0, e.g. when found the string in some files.

Solution 3

Standard xargs has no good way to do it; you're better off using find -exec as someone else suggested, or wrap the sed in a script which does nothing if there are no arguments. GNU xargs has the --no-run-if-empty option, and BSD / OS X xargs has the -L option which looks like it should do something similar.

Solution 4

I have taken Vlad's idea and changed it a little bit. Instead of

grep -rl oldstr path | xargs sed -i 's/oldstr/newstr/g' /dev/null

Which yields

sed: couldn't edit /dev/null: not a regular file

I'm doing in 3 different connections to the remote server

touch deleteme
grep -rl oldstr path | xargs sed -i 's/oldstr/newstr/g' ./deleteme
rm deleteme

Although this is less elegant and requires 2 more connections to the server (maybe there's a way to do it all in one line) it does the job efficiently as well

Solution 5

My use case was I wanted to replace foo:/Drive_Letter with foo:/bar/baz/xyz In my case I was able to do it with the following code. I was in the same directory location where there were bulk of files.

find . -name "*.library" -print0 | xargs -0 sed -i '' -e 's/foo:\/Drive_Letter:/foo:\/bar\/baz\/xyz/g'

hope that helped.

View more solutions

Comments

[Michael]

thanks, i tried to use --no-run-if-empty but it still returns nonzero code (returns 1) and that would also trigger a build fail for me. how generic and common is find command ?

[geekosaur]

find -exec goes back to 7th Research Edition UNIX; it should work anywhere that has find installed.

[geekosaur]

@Vlad: It is, because you're running a separate sed for each file instead of letting xargs batch them. That said, unless you're talking about several thousand tiny files, you're unlikely to notice a difference.

[Admin]

@geekosaur: Oh, right.. I haven't thought about a number of execs shell should make. Good point!

[Michael]

thanks, i tried your solution. it did solve my return value problem, now it returns 0 when it doesn't find anything. the problem is that it returns : sed: no input files even though i have files with the oldstring in the directory. know why ?

[Michael]

i'm using this in directory tree with possibly thousand of files to look in (although the amount of files that actually need change is not big) is that an issue ?

[pbarill]

My problem with this: the timestamp of every file will get reset to "right now" as if they were all "touched", even if no edits occurred in the file.

[tripleee]

+1 I'm surprised the xargs -r fix is not mentioned in more answers. For this limited use case, it's simple and sufficient.

[workdreamer]

And this script save my life!

[Leopoldo Sanczyk]

That is a nice underestimated solution! This is what I needed to execute sed over a list of files output of grep, and avoid the error: "can't read .... No such file or directory". Thanks for your answer!

[Jerry Miller]

I've seen this "Hello world" example of using find all over the place, but nothing that constrains it from running sed on every file, including binaries. I have yet to find a working example with pipes, and I've tried a number of ways to hack the simplistic version of very limited utility.

[Michael Berkowski]

@JerryMiller you can of course limit find to specific filename patterns, which is what I often do: find -name "*.c" -o -name "*.h" -type f -exec sed... would modify all the .c and .h files for example... To exclude binary files, it can get more complicated: unix.stackexchange.com/questions/46276/…

[trogne]

But if you want a solution with grep and sed : find . -name "find_files" -exec grep -l "text_to_find" {} \; -exec sed -i 's/change_text/to_this_text/g' {} \; The grep -l option returns a list of files, instead of the text found.

[jww]

Agree with @Leopoldo. This looks like the most Posixy and portable (for OS like Solaris) and the one with the least side effects (not touching every file mtime, etc). files=$(grep -rl oldstr path | cut -f 1 -d ':' | sort | uniq) should build a smaller list where the files are listed once.

[oz19]

Worked for me. Made me save quite much time. Thanks!