Recursively rename files using find and sed

Question

I want to go through a bunch of directories and rename all files that end in _test.rb to end in _spec.rb instead. It's something I've never quite figured out how to do with bash so this time I thought I'd put some effort in to get it nailed. I've so far come up short though, my best effort is:

find spec -name "*_test.rb" -exec echo mv {} `echo {} | sed s/test/spec/` \;

NB: there's an extra echo after exec so that the command is printed instead of run while I'm testing it.

When I run it the output for each matched filename is:

mv original original

i.e. the substitution by sed has been lost. What's the trick?

Solution 1

This happens because sed receives the string {} as input, as can be verified with:

find . -exec echo `echo "{}" | sed 's/./foo/g'` \;

which prints foofoo for each file in the directory, recursively. The reason for this behavior is that the pipeline is executed once, by the shell, when it expands the entire command.

There is no way of quoting the sed pipeline in such a way that find will execute it for every file, since find doesn't execute commands via the shell and has no notion of pipelines or backquotes. The GNU findutils manual explains how to perform a similar task by putting the pipeline in a separate shell script:

#!/bin/sh
echo "$1" | sed 's/_test.rb$/_spec.rb/'

(There may be some perverse way of using sh -c and a ton of quotes to do all this in one command, but I'm not going to try.)

Solution 2

To solve it in a way most close to the original problem would be probably using xargs "args per command line" option:

find . -name "*_test.rb" | sed -e "p;s/test/spec/" | xargs -n2 mv

It finds the files in the current working directory recursively, echoes the original file name (p) and then a modified name (s/test/spec/) and feeds it all to mv in pairs (xargs -n2). Beware that in this case the path itself shouldn't contain a string test.

Solution 3

you might want to consider other way like

for file in $(find . -name "*_test.rb")
do 
  echo mv $file `echo $file | sed s/_test.rb$/_spec.rb/`
done

Solution 4

I find this one shorter

find . -name '*_test.rb' -exec bash -c 'echo mv $0 ${0/test.rb/spec.rb}' {} \;

Solution 5

You mention that you are using bash as your shell, in which case you don't actually need find and sed to achieve the batch renaming you're after...

Assuming you are using bash as your shell:

$ echo $SHELL
/bin/bash
$ _

... and assuming you have enabled the so-called globstar shell option:

$ shopt -p globstar
shopt -s globstar
$ _

... and finally assuming you have installed the rename utility (found in the util-linux-ng package)

$ which rename
/usr/bin/rename
$ _

... then you can achieve the batch renaming in a bash one-liner as follows:

$ rename _test _spec **/*_test.rb

(the globstar shell option will ensure that bash finds all matching *_test.rb files, no matter how deeply they are nested in the directory hierarchy... use help shopt to find out how to set the option)

View more solutions

Comments

[opsb]

That does look like a good way to do it. I'm really looking to crack the one liner though, to improve my knowledge more than anything else.

[Damodharan R]

Ah..i am not aware of a way to use sed other than putting the logic in a shell script and call that in exec. didnt see the requirement to use sed initially

[opsb]

For those wondering about the perverse usage of sh -c here it is: find spec -name "*_test.rb" -exec sh -c 'echo mv "$1" "$(echo "$1" | sed s/test.rb\$/spec.rb/)"' _ {} \;

[agtb]

Hi, I think '_test.rb" should be '_test.rb' (double quote to single quote). Can I ask why you're using the underscore to push the argument you want to position $1 when it seems to me that find . -name '*_test.rb' -exec bash -c 'echo mv $0 ${0/test.rb/spec.rb}' {} \; works? As would find . -name '*_test.rb' -exec bash -c 'echo mv $1 ${1/test.rb/spec.rb}' iAmArgumentZero {} \;

[csg]

Thanks for your suggestions, fixed

[agtb]

Thanks for clearing that up - I only commented because I spent a while pondering the meaning of _ thinking it was maybe some trick use of $_ ('_' is pretty hard to search for in docs!)

[Ali]

this does not work (the sed one) as explained by the accepted answer.

[Wayne Conrad]

@Ali, It does work--I tested it myself when I wrote the answer. @larsman's explanation does not apply to for i in... ; do ... ; done, which executes commands via the shell and does understand backtick.

[Bretticus]

for file in $(find . -name "*_test.rb"); do echo mv $file echo $file | sed s/_test.rb$/_spec.rb/; done is a one-liner, is it not?

[mikeserv]

How does the sed work without escaping () if you don't set the -r option?

[onitake]

This will not work if you have filenames with spaces. for will split them into separate words. You can make it work by instructing the for loop to split only on newlines. See cyberciti.biz/tips/handling-filenames-with-spaces-in-bash.ht‌​ml for examples.

[cde]

Unfortunately this has white space issues. So using with folders that have spaces in the name will break it at xargs (confirm with -p for verbose/interactive mode)

[ShellFish]

I agree with @onitake, although I would prefer to use the -exec option from find.

[iRaS]

@opsb what the heck is that _ for? great solution - but i like ramtam answer more :)

[Sven M.]

Cheers! Saved me a lot of headaches. For the sake of completeness this is how I pipe it to a script: find . -name "file" -exec sh /path/to/script.sh {} \;

[Michele Dall'Agata]

That's exactly what I was looking for. Too bad for the white space issue (I didn't test it, though). But for my current needs it's perfect. I'd suggest to test it first with "echo" instead of "mv" as parameter in "xargs".

[Evan Purkhiser]

If you need to deal with whitespace in paths and you're using GNU sed >= 4.2.2 then you can use the -z option along with finds -print0 and xargs -0: find -name '*._test.rb' -print0 | sed -ze "p;s/test/spec/" | xargs -0 -n2 mv

[Miguel A. Baldi Hörlle]

Best solution. So much faster than find -exec. Thank you

[Vinay Vissh]

Thanks a lot! It helped me in easily removing trailing .gz from all file names recursively. while IFS= read -r file; do mv $file ${file%.gz}; done < <(find -type f -name "*.gz")

[fedorqui]

@CasualCoder nice to read that :) Note you can directly say find .... -exec mv .... Also, be careful with $file since it will fail if it contains spaces. Better use quotes "$file".

[Casey]

This won't work, if there are multiple test folders in one path. sed will only rename first one and mv command will fail on No such file or directory error.