How to report "sed" in-place changes

Question

When using sed to replace strings in-place, is there a way to make it report the changes it does (without relying on a diff of old and new files)?

For instance, how can I change the command line

find . -type f | xargs sed -i 's/abc/def/g'

so I can see the changes that are made on the fly?

Solution 1

You could use sed's w flag with either /dev/stderr, /dev/tty, /dev/fd/2 if supported on your system. E.g. with an input file like:

foo first
second: missing
third: foo
none here

running

sed -i '/foo/{
s//bar/g
w /dev/stdout
}' file

outputs:

bar first
third: bar

though file content was changed to:

bar first
second: missing
third: bar
none here

So in your case, running:

find . -type f -printf '\n%p:\n' -exec sed -i '/foo/{
s//bar/g
w /dev/fd/2
}' {} \;

will edit the files in-place and output:

./file1:
bar stuff
more bar

./file2:

./file3:
bar first
third: bar

You could also print something like original line >>> modified line e.g.:

find . -type f -printf '\n%p:\n' -exec sed -i '/foo/{
h
s//bar/g
H
x
s/\n/ >>> /
w /dev/fd/2
x
}' {} \;

edits the files in-place and outputs:

./file1:
foo stuff >>> bar stuff
more foo >>> more bar

./file2:

./file3:
foo first >>> bar first
third: foo >>> third: bar

Solution 2

You could do it in two passes using the print action on the first pass with:

find . -type f | xargs sed --quiet 's/abc/def/gp'

where --quiet makes sed not show every line and the p suffix shows only lines where the substitution has matched.

This has the limitation that sed will not show which files are being changed which of course could be fixed with some additional complexity.

Solution 3

I don't think that's possible, but a workaround might be to use perl instead:

find . -type f | xargs perl -i -ne 's/abc/def/ && print STDERR' 

This will print the altered lines to standard error. For example:

$ cat foo
fooabcbar
$ find . -type f | xargs perl -i -ne 's/abc/def/ && print STDERR' 
foodefbar

You can also make this slightly more complex, printing the line number, file name, original line and changed line:

$ find . -type f | 
   xargs perl -i -ne '$was=$_; chomp($was);
                      s/abc/def/ && print STDERR "$ARGV($.): $was : $_"' 
./foo(1): fooabcbar : foodefbar

Solution 4

It is possible using the w flag which writes the current pattern to a file. So by adding it to the substitute command we can report successive substitutions to a file and print it after the job is done. I also like to colorize the replaced string with grep.

sed -i -e "s/From/To/gw /tmp/sed.done" file_name
grep --color -e "To" /tmp/sed.done

Note, that there must be only one space between the w and its file name.

This is even better than diff, since diffing might also show changes even if they were not made by sed.

Solution 5

I like the @terdon solution - perl is good for this.

Here's my tweaked version that:

• won't try to alter files that dont have have the matching string
• will backup the before version of the files that change (create a .bak version)
• will list every file/lineno changed, and show the OLD and NEW versions of that line indented and underneath for easy reading

code

find /tmp/test -type f ! -name "*.bak" -exec grep -l '/opt/gridmon' {} \; | xargs -L1 perl -ni'.bak' -e'$old=$_; s/\/opt\/gridmon/~/g && print STDERR "$ARGV($.):\n\tOLD:$old\tNEW:$_"'

example output

/tmp/test/test4.cfg(13):
    OLD:    ENVFILE /opt/gridmon/server/etc/gridmonserver.cfg
    NEW:    ENVFILE ~/server/etc/gridmonserver.cfg
/tmp/test/test4.cfg(24):
    OLD:    ENVFILE /opt/gridmon/server/etc/gridmonserver.cfg
    NEW:    ENVFILE ~/server/etc/gridmonserver.cfg

View more solutions

Comments

[ricab]

It doesn't quite do what I wanted since you need two passes, but I'm voting up because I just learned this p thing and it's very useful. Especially if, as you said, the files were also printed. Something like for x in `find . -type f`; do echo ///File $x: ; sed --quiet 's/abc/def/gp' $x; done

[Jeff Hewitt]

+1 You can also use $ARGV for the name of the file being operated on.

[terdon]

@JosephR. good point, added.

[ricab]

Thanks, that's probably helpful (didn't test it myself). I am not selecting as it doesn't use sed, so it doesn't exactly answer the question.

[s.khan]

Just tested it out and sed only writes lines it substituded to sed.done. The lines with "To" in the original file are therefore not printed to sed.done, thus when you grep "To" sed.done you will only see the lines that are changed by sed. You won't see the original line in the file before it was substituted, if that's what you're aiming at ...

[alhelal]

@msw I have many sed expression, I don't want to write /gp for each one. How to set it globaly?