Unix command to replace all instances of a string in every file in a folder

Question

I have a folder "model". In it, I need to replace all instances of the term "Test_Dbv3" to "TestDbv3". There are multiple files with names like test_host.hbm.xml, test_host2.hbm.xml, testHost.java, testHost2.java and so on. Is there any way I can possibly do this using a Unix command or a script in any language?

I'm working on RHEL5.

Solution 1

sed in in-place mode along with find should probably work:

find . -type f -exec sed -e 's/Test_Dbv3/TestDbv3/g' -i.bak '{}' +

The aptly named find command finds files. Here, we're finding files in the current working directory (.) that are files (-type f). Using these files, we're going to -exec a command: sed. + indicates the end of the command and that we'd like to replace {} with as many files as the operating system will allow.

sed will go file-by-file, line-by-line, executing commands we specify. The command we're giving it is s/Test_Dbv3/TestDbv3/g, which translates to “substitute matches of the regular expression Test_Dbv3 with the text TestDbv3, allowing multiple substitutions per line”. The -i.bak means to replace the original file with the result, saving the unmodified version with the filename suffixed with .bak.

Solution 2

s/_//g is your regex assuming you want all _ gone; otherwise I need to guess how to specify your regex:

• For example s/^(Test|test)_/$1/g to replace test_ with test and Test_ with Test if they are at the beginning of a line.

• Or s/^(test)_/$1/gi will additionally work for all TEST_, tEsT_, etc.

• If you decide to need completely case insensitive matching that is only available for the for perl -pi -e 's/.../.../gi' or GNU sed or more but not the sed command (not even variables like $1 are, are they?)

• If there are also filenames starting like Test2_ or 1EXPERIMENT_ and more words you may would use s/^([A-Za-z0-9]{3,10})_/$1/g to match every combination of letters and numbers of length 3 to 10 chars, not just the Test or test you mentioned.

• For even more specific regex search for "regex cheatsheet" and just don't wonder when single tools like sed or grep don't support everything should you even decide to use them.

• Should you also ever need a command to only rename files in a folder, but not edit their content you may try rename 's/search/relace/' folder/* (not matching subdirectories) or rename search replace folder/* (depending on version of rename).

Comments

[jaypal singh]

+1 Great explaination.

[Stephane Chazelas]

That's the FreeBSD sed syntax. With GNU sed (as found on RHEL5), because the argument to -i is optional, it has to follow the option. So it should be -i.bak (-i.back will also work with FreeBSD sed).

[icktoofay]

@sch: Thanks; I vaguely remember that there was an incompatibility there that had gotten me before. Would you, by chance, know a portable way to specify no backup? I know how to do it with the BSD way (-i '') and the GNU way (-i), but I can't think of a way that would work on both.

[Jonathan Leffler]

Not my down-vote, but since the question is asking to change Test_Dbv3 to TestDbv3, it is reasonable to guess that there are other occurrences of a simple underscore that should not be replaced (especially in Java source, for example), and guessing what's requested is not really necessary. The i modifier for sed substitute commands is not standard (I'd not seen it before, but I haven't studied the GNU sed manual for every nuance since I usually work on other systems too).

[Jonas]

@Jonathan Leffler Thanks but no, the other answer isnt solving the question since it just matches one of the 5 example terms while s/_//g is matching all five that we know and in the most simple way, but of course i included more regex guessing that it may be too broad (you are guessing saying that in the same way i did), since the question just doesnt specify more but probably requires a more specific searchterm. Also as said above GNU sed does support i and sed is just one in many alternatives here, not even usually available on RHEL5. sed was suggested wrongfully here by the first answer.

[Jonathan Leffler]

The question isn't about the file names; it is about the contents of the files...the file names are simply examples of the files that might be found in the directory. Which, come to think of it, means that your rename suggestions are also off the mark, I think.

[Jonas]

@Jonthan Leffler I just mentioned that in the end, since i imagined gran_profaci may also need to do this at some point since his general question seemed like he didnt work with Regex command lines and was much about filenames and folders. Else at least it may be useful as of the search terms this question already is defined by. I understand an extensive answer is rather good than bad. - At times i just wonder about a "hostile use of votes on stackoverflow; it doesnt always seem to be reflection of greater competence.

[Stephane Chazelas]

@icktoofay, you can use perl (where the -i comes from): perl -pi -e ... and then it works on all Unices that ship with perl (most noawdays).

[Jonathan Leffler]

Comprehensive answers don't always gather the most points. Believe me, I'm an expert on that! See Users who post long answers and set the threshold at 5000. The points per character ratings are not good for me!

[doubleDown]

@icktoofay, I remember seeing somewhere that OSX's sed uses -i'' to specify no backup, and OSX's sed is probably the same as BSD sed.

[doubleDown]

In general, you should answer based on what the asker says he needs/wants to achieve, and not what you assume he needs. At least try not dedicate most of your answer to be about imagined scenarios beyond what the question asks.

[Stephane Chazelas]

@doubleDown, -i'' is the same as -i. OSX's -i sed option just like FreeBSD's require an argument, so you have to write it -i '', that is pass an empty argument after the one containing "-i". sed -i '' doesn't work in the GNU sed, because since it's a separate argument, it's taken as a non-option argument.