Modify a file without creating another file

Question

Is there a way to modify a file without writing the contents to another file, without sed and awk?

For example:

$ cat test.txt
aaa
aaa
bbb
ccc
ddd

Replacing using sed with -i option, sed -i 's/aaa/NNN/g' test.txt will produce the following:

NNN
NNN
bbb
ccc
ddd

How to do that without awk and sed?

Solution 1

You can use a vi script:

$ vi test.txt -c '%s/aaa/NNN/ | wq'
$ cat test.txt
NNN
NNN
bbb
ccc
ddd

You're simply automating what would normally be entered when using vi in command mode (accessed using Esc: usually):

% - carry out the following command on every line:

s/aaa/NNN/ - subtitute aaa with NNN

| - command delimiter

w - write changes to file

q - quit

Solution 2

Using sponge:

#!/bin/bash

pattern='aaa'
replacement='NNN'

while read -r line
do                                                                              
  printf '%s\n' "${line//$pattern/$replacement}"
done < "${1}"

Call with:

./script.sh test.txt | sponge test.txt

Solution 3

With ed, the line editor:

ed -s test.txt <<< $',s/pattern/replace/g\nw\nq'

or

ed -s test.txt <<IN
,s/pattern/replace/g
w
q
IN

or

printf '%s\n' ,s/pattern/replace/g w q | ed -s test.txt

Solution 4

If you are using bash or ksh, you can use pattern substitution for shell variables. Note however, that basic shell globs are less powerful and extended shell globs have some features that sed doesn't and vice versa. For more details, see 'Parameter Expansion' in man 1 bash:

t=$(< test.txt); printf '%s\n' "${t//aaa/NNN}" >test.txt

Extended shell globs are disabled by default, so you may need to explicitly enable them:

shopt -s extglob

Solution 5

You can also use perl

perl -pi -e 's/aaa/bbb/g' file.txt

This will give the output you desire.

You can also backup your original file automatically using i.bak instead of i. This will create a backup named file.txt.bak.

View more solutions

Comments

[Sebastian]

by the way, sed -i internally writes to a temporary file and then moves it into the place of the original file. The option title --in-place is a little misleading.

[bnikhil]

The vi and sponge solutions also create temporary files to do their work...

[Abradolf_linclr]

My intention to post this question is that atleast the user should not manually create a new file and write the contents. Thank you for simple answers and comments.

[garethTheRed]

yours too ;-) Never heard of sponge before.

[Sebastian]

I found it here: tools.suckless.org/sbase

[Abradolf_linclr]

seems very neat and simple.

[nyuszika7h]

That script suffers from multiple issues. (1) read line should be read -r line. (2) The $() in the if is redundant. I'm surprised if that even works. (3) echo suffers from a lot of portability issues even within bash. Use printf '%s\n' "${var}" instead. (4) $1 should be wrapped in double quotes: "$1".

[nyuszika7h]

Also, if it's a bash script, you can just do grep -q "$pattern" <<< "$line".

[nyuszika7h]

I think you forgot to define pattern and replacement, too.

[nyuszika7h]

I submitted an edit with some improvements. While it may not exactly be suitable for an edit, I'm hoping to save you some work.

[nyuszika7h]

I can't see any globs there.

[Sebastian]

thanks @nyuszika7h for suggesting multiple improvements. The script was a quick (sloppy) submission. If there are issues left, please let me know - I'm always keen on improving. E.g. I didn't know that echo has any issues. I'll check that.

[Abradolf_linclr]

sponge command seems to do similar function as redirection operator but changing contents in the same file. I could not find it in my linux distribution.

[Sebastian]

It's not a standard tool, but I think it deserves more publicity. See the link above , for download and source.

[Franki]

@nyuszika7h true, although it is possible if you want to, like so: printf '%s\n' "${t//+(a|d)/NNN}" if you wanted to replace all runs of 'a's or 'd's with 'NNN'

[deltab]

sponge is packaged in moreutils; Debian, Ubuntu and other distributions have it as an optional package in their repositories.

[Abradolf_linclr]

I was using your script for replacing tab with comma. vi test.txt -c '%s/\t/,/**g** | wq' did well for multiple columns. Without g, it was only replacing the first instance. Thank you.

[don_crissti]

@mikeserv - wrt ed & temp file creation, it all depends on the implementation; see the posts here by bakunin and alister.

[Wildcard]

Is there some advantage to using ed over using ex?

[don_crissti]

@Wildcard - I've never had any interest in ex (and as a result, I've never used it - same for vi). I only know that on some exotic setups you may find ed but not ex. Other than that I'm afraid I can't answer your question. Someone like Stéphane - who knows the ins and outs of both - would be the right guy to ask.

[Wildcard]

Both are specified in POSIX, and ex is more flexible. But thanks for explaining. :)