Redirect output from sed 's/c/d/' myFile to myFile

Question

I am using sed in a script to do a replace and I want to have the replaced file overwrite the file. Normally I think that you would use this:

% sed -i 's/cat/dog/' manipulate
sed: illegal option -- i

However as you can see my sed does not have that command.

I tried this:

% sed 's/cat/dog/' manipulate > manipulate

But this just turns manipulate into an empty file (makes sense).

This works:

% sed 's/cat/dog/' manipulate > tmp; mv tmp manipulate

But I was wondering if there was a standard way to redirect output into the same file that input was taken from.

Solution 1

I commonly use the 3rd way, but with an important change:

$ sed 's/cat/dog/' manipulate > tmp && mv tmp manipulate

I.e. change ; to && so the move only happens if sed is successful; otherwise you'll lose your original file as soon as you make a typo in your sed syntax.

Note! For those reading the title and missing the OP's constraint "my sed doesn't support -i": For most people, sed will support -i, so the best way to do this is:

$ sed -i 's/cat/dog/' manipulate

Solution 2

Yes, -i is also supported in FreeBSD/MacOSX sed, but needs the empty string as an argument to edit a file in-place.

sed -i "" 's/old/new/g' file   # FreeBSD sed

Solution 3

If you don't want to move copies around, you could use ed:

ed file.txt <<EOF
%s/cat/dog/
wq
EOF

Solution 4

Kernighan and Pike in The Art of Unix Programming discuss this issue. Their solution is to write a script called overwrite, which allows one to do such things.

The usage is: overwrite file cmd file.

# overwrite: copy standard input to output after EOF

opath=$PATH
PATH=/bin:/usr/bin

case $# in
0|1)   echo 'Usage: overwrite file cmd [args]' 1>&2; exit 2
esac

file=$1; shift
new=/tmp/overwr1.$$; old=/tmp/overwr2.$$
trap 'rm -f $new $old; exit 1' 1 2 15  # clean up

if PATH=$opath "$@" >$new
then
       cp $file $old           # save original
       trap '' 1 2 15          # wr are commmitted
       cp $new $file
else
       echo "overwrite: $1 failed, $file unchanged" 1>&2
       exit 1
fi
rm -f $new $old

Once you have the above script in your $PATH, you can do:

overwrite manipulate sed 's/cat/dog/' manipulate

To make your life easier, you can use replace script from the same book:

# replace: replace  str1 in files with str2 in place
PATH=/bin:/usr/bin

case $# in
    0|2) echo 'Usage: replace str1 str2 files' 1>&2; exit 1
esac

left="$1"; right="$2"; shift; shift

for i
do
    overwrite $i sed "s@$left@$right@g" $i
done

Having replace in your $PATH too will allow you to say:

replace cat dog manipulate

Solution 5

You can use sponge from the moreutils.

sed "s/cat/dog/" manipulate | sponge manipulate

View more solutions

Comments

[Anders]

Or in this case simply download a new version of sed, compliant with -i option, from www.sunfreeware.com ;)

[Arkku]

+1, the overwrite script is very useful (after a few changes according to personal preferences)!

[Arkku]

@Anders: Replacing the system sed with a non-standard alternative, however free it may be, may not be a viable option, nor does it answer the question of how to do it with available/standard tools.

[Anders]

The POSIX standards - which Solaris follows - only dictates a minimum set of option for sed to be compliant, so GNU/Sed or whatever you like to call is actually compliant with what you call "standard tools".

[Alok Singhal]

@Anders, It is compliant with POSIX, but if you use a non-POSIX option, then of course you are not relying on standard tools. One can of course install GNU sed on a system, but then one might not be able to.

[sixtyfootersdude]

does ed support all of the regular expressions that sed does? Specifically ranges?

[amertune]

ed supports all of the regular expressions that sed does, including ranges. ed - works on files sed - works on streams The other main difference is that an ed command, by default will only work on the current line of a file (which is why I edited my example and added the %, to make the command run on all lines), while sed commands, by default, run against all lines.

[Isaac]

-i is also supported in FreeBSD/MacOSX sed

[Gilles 'SO- stop being evil']

While technically correct and a cute trick, this is dangerous. If your power fails just after while sed is executing, you'll end up with missing data.

[romaninsh]

you can also use "ex", which is alias to "vim". It's very feature-rich.

[bartimar]

-i don't need empty string. Value of the parameter is backup extension to use, when it's empty string - no backup will be done. On GNU sed the default value for -i is empty string.

[arkascha]

The OP clearly stated that the -i flag is considered "illegal" by his version of sed. So it cannot be used...

[JB.]

Just link elsewhere before you remove and it'll be fine.

[Jianxin Gao]

It's my life saver

[acgbox]

for me, this is the best answer