How do I use file names in a sed command?

Question

I have a number of .conf files that are identical and in the same directory, with the exception of having different file names. In each uniquely named .conf file, I would like to replace a set of characters in the file with the name of the file. For example:

Currently in all files:

datafname = example.nex
ofprefix = best.example

Ideal output:

Filename: 25.conf

datafname = 25.nex
ofprefix = best.25

Filename: 26.conf

datafname = 26.nex
ofprefix = best.26

I thought that I could use sed to run through all these files to find and replace the string of text with the file name using something like:

sed -i conf 's/example/echo $f/g' *

but this is not working properly. Would anyone happen to have a suggestion on how to do this?

Solution 1

You can do:

for f in *.conf; do
    base=$(basename "$f" '.conf') # gives '25' from '25.conf'
    sed -i.before "s/example/$base/g" "$f"
done

When using the -i switch to sed, you must be absolutely sure that your sed command works because -i changes the files in-place. This means the generated output will overwrite the input file. If your replacement command (s/…/…/) is wrong, you may end up with empty files and no backups. Hence, I used -i.before which will leave a *.before file with the original content.

Solution 2

You can use a for loop to iterate over the files. Use parameter expansion to remove the file extension. Use double quotes around the expression, otherwise the variable wouldn't be expanded.

#! /bin/bash
for f in *.conf ; do
    b=${f%.conf}
    sed -i "s/example/$b/" "$f"
done

Solution 3

You can do the task without a loop using GNU parallel :

parallel sed -i.old s/example/{.}/ {} ::: *.conf

This is especially useful if you have a lot of files to edit, as parallel runs one job per CPU core.

• -i.old means: edit the file in-place and make a backup adding the .old extension to the original filename (remove .old if you don't want a backup, but remember you don't have a backup then)
• s/example/{.}/g means substitute example with the input filename without its extension ({.}) and do it globally (= to every occurence)
• {} is replaced with the input filename
• ::: separates the command to run from the arguments to pass to it
• *.conf matches every .conf file in the current directory

Solution 4

With awk, which has a FILENAME variable automatically set to the filename (and, if GNU awk, with edits in-place as well):

$ for i in {20..26}; do printf "%s\n" "datafname = example.nex" "ofprefix = best.example" > $i.conf; done
$ gawk -i inplace 'FNR == 1 {split(FILENAME, file, ".")} {gsub("example", file[1])} 1' *.conf
$ cat 25.conf
datafname = 25.nex
ofprefix = best.25

• FNR == 1 {split(FILENAME, file, ".")}: on the first line of each file, split the filename on . and store it in the file array
• {gsub("example", file[1])} 1: for all lines, replace example with the first element of the file array and print.

View more solutions

Comments

[terdon]

Yes, this is 100% on topic @foolsparadise. PerlDuck, please familiarize yourself with the site's scope before suggesting otherwise.

[dessert]

@OleTange You're probably right, I'm not quite sure about the correct terminology and just described the feature like in man parallel now. Thank you!

[kaitosenpai]

Elegant solution. Initially I worried whether it would break if the filenames contain space, but you got that covered, too.

[dessert]

@OleTange I? No, parallel covered it! It's just an amazing piece of software if you ask me, useful as a replacement to almost every for loop and xargs call.

[dessert]

@OleTange Oh wait, I just realized you may have already known that. :D Thank you very much for making things so easy, --will-cite!