sed beginner: changing all occurrences in a folder

Question

I need to do a regex find and replace on all the files in a folder (and its subfolders). What would be the linux shell command to do that?

For example, I want to run this over all the files and overwrite the old file with the new, replaced text.

sed 's/old text/new text/g' 

Solution 1

There is no way to do it using only sed. You'll need to use at least the find utility together:

find . -type f -exec sed -i.bak "s/foo/bar/g" {} \;

This command will create a .bak file for each changed file.

Notes:

• The -i argument for sed command is a GNU extension, so, if you are running this command with the BSD's sed you will need to redirect the output to a new file then rename it.
• The find utility does not implement the -exec argument in old UNIX boxes, so, you will need to use a | xargs instead.

Solution 2

I prefer to use find | xargs cmd over find -exec because it's easier to remember.

This example globally replaces "foo" with "bar" in .txt files at or below your current directory:

find . -type f -name "*.txt" -print0 | xargs -0 sed -i "s/foo/bar/g"

The -print0 and -0 options can be left out if your filenames do not contain funky characters such as spaces.

Solution 3

For portability, I don't rely on features of sed that are specific to linux or BSD. Instead I use the overwrite script from Kernighan and Pike's book on the Unix Programming Environment.

The command is then

find /the/folder -type f -exec overwrite '{}' sed 's/old/new/g' {} ';'

And the overwrite script (which I use all over the place) is

#!/bin/sh
# overwrite:  copy standard input to output after EOF
# (final version)

# set -x

case $# in
0|1)        echo 'Usage: overwrite file cmd [args]' 1>&2; exit 2
esac

file=$1; shift
new=/tmp/$$.new; old=/tmp/$$.old
trap 'rm -f $new; exit 1' 1 2 15    # clean up files

if "$@" >$new               # collect input
then
    cp $file $old   # save original file
    trap 'trap "" 1 2 15; cp $old $file     # ignore signals
          rm -f $new $old; exit 1' 1 2 15   # during restore
    cp $new $file
else
    echo "overwrite: $1 failed, $file unchanged" 1>&2
    exit 1
fi
rm -f $new $old

The idea is that it overwrites a file only if a command succeeds. Useful in find and also where you would not want to use

sed 's/old/new/g' file > file  # THIS CODE DOES NOT WORK

because the shell truncates the file before sed can read it.

Solution 4

Might I suggest (after backing up your files):

find /the/folder -type f -exec sed -ibak 's/old/new/g' {} ';'

Solution 5

Example: inline replace {AutoStart} with 1 for all of the ini files under the /app/config/ folder and its child folders:

sed -i 's/{AutoStart}/1/g' /app/config/**/*.ini

View more solutions

Comments

[Jakub Kukul]

If you're on OSX, try find . -type f -name "*.txt" -print0 | xargs -0 sed -i '' "s/foo/bar/g" (note providing an empty string to the -i argument).

[Kunal Parekh]

this is not related to the question asked. The question does not mention anything about same file / folder name pattern. Please avoid such answers

[Andriy Makukha]

What is \; for?

[wolf4]

We need to tell to find where the command of the argument -exec ends with a ”;”. But the shell uses the same symbol (;) as a shell command separator, so, we need to escape the ”;” from the shell to pass it to the find’s -exec argument.

[Kyle]

It's worth noting that -i by itself does not create a backup file, and is what causes sed to perform the operation on the file in place.

[somenickname]

What is {} for ?

[wolf4]

The {} will be replaced by each filename found by find and \; tells to find that the command that he needs to execute finish at this point.

[Dropout]

Please explain your answer.

[SherylHohman]

While this code may resolve the OP's issue, it's better to include an explanation on how your code addresses the OP's issue. This way, future visitors can learn from your post, & apply it to their own code. SO is not a coding service, but a resource for knowledge. High quality, complete answers reinforce this idea, and are more likely to be upvoted. These features, plus the requirement that all posts be self-contained, are some strengths of SO as a platform that differentiates us from forums. You can edit to add additional info &/or to supplement your explanations with source documentation.

[forzagreen]

On MacOS, run sed -i.bak instead of sed -i. I think as mentioned by @JakubKukul , sed -i '' also works.

[user176692]

sed -i.bak 's/REPLACE_THIS/WITH_THIS/' * seems to work on any file in the current folder. Is this acceptable if subfolder replacements are not needed?

[user176692]

sed -i.bak 's/REPLACE_THIS/WITH_THIS/g' * (Edit to prior comment)

[armourbear]

this wont resolve the issue because it only list current working directory and does not list sub folders

[Arcsector]

This doesn't replace it, it just prints out the replacement - if you want to replace it, add the inline flag: sed -i 's/{AutoStart}/1/g' /app/config/**/*.ini