Passing wildcard * to a (bash?) script

Question

I want to to remove all color codes which look like '@n', '@R' etc, from a moderately large size collection of text files.

So in a file called 'remove_cc', I wrote the following:

sed -ie 's/@r//g' $1
sed -ie 's/@g//g' $1
sed -ie 's/@y//g' $1
sed -ie 's/@b//g' $1
sed -ie 's/@m//g' $1
sed -ie 's/@c//g' $1
sed -ie 's/@n//g' $1
sed -ie 's/@R//g' $1
sed -ie 's/@G//g' $1
sed -ie 's/@Y//g' $1
sed -ie 's/@B//g' $1
sed -ie 's/@M//g' $1
sed -ie 's/@C//g' $1
sed -ie 's/@N//g' $1
sed -ie 's/@W//g' $1
sed -ie 's/@K//g' $1

Anyways, the script works fantastically if I use it like: ./remove_cc file.txt

But if I type ./remove_cc *.txt in a folder with many txt files that I want to run the script on, it fails to do anything. ./remove_cc * is equally ineffective.

Is there a way to fix this so it works?

Solution 1

When you type ./remove_cc *, the shell changes it to ./remove_cc file1.txt file2.txt file3.txt etc, and runs your script that way. Your script is only looking at $1, the first parameter (file1.txt).

The most general way to do this is to loop over each parameter in turn. "$@" expands to the list of all the parameters, and we can use for to loop over them.

for n in "$@" ; do
    sed -ie 's/@r//g' "$n"
    ...
done

In this particular case, since sed will take multiple filenames, you can simplify:

sed -ie 's/@r//g' "$@"

Note that paying attention to quotes is important when writing shell scripts. Without the quotes, for example, the script would not work on a file named My Text File.txt.

Solution 2

just use

 for x in *
 do
     ./remove_cc $x
 done

By the way, you can combine all sed in one line

sed -i \
    -e 's/@r//g' -e 's/@g//g' -e 's/@y//g' -e 's/@b//g' \
    -e 's/@m//g' -e 's/@c//g' -e 's/@n//g' -e 's/@R//g' \
    -e 's/@G//g' -e 's/@Y//g' -e 's/@B//g' -e 's/@M//g' \
    -e 's/@C//g' -e 's/@N//g' -e 's/@W//g' -e 's/@K//g' \
    $1

you can also join sed commands with semicolons:

sed -i -e 's/@r//g;s/@g//g;s/@y//g' $1

or just use character list

sed -i -e s/@[rgbcmyRGBCMY]//g $1

or

 sed -i -e s/@[rgbcmyRGBCMY]//g *

(be sure to check I have put all the letter)

Solution 3

Use "$@" instead of $1. Or quote the wildcard:

$ bash -c 'echo $1' 'some command' '*'
file1 file2
$ bash -c 'echo "$@"' 'some command' *
file1 file2

Solution 4

If you do not wish to change your script you can also use xargs (considering you have it or it's available on your platform) by running it like this:

ls *.txt | xargs -L1 ./remove_cc

xargs is a utility that takes stdin and transforms it to command parameters on a command to be run. It has the -L flag that limits the amount of input lines to be used in a single execution.

View more solutions

Comments

[peterph]

Just a short note: -i is not specified by POSIX, so you may encounter seds that won't understand it.

[Angel Todorov]

don't parse ls