linux wildcard usage in cp and mv

Question

I am composing a script to process 20 files. All of them located in different directories. I have partial file name.

1. In log directory, File1_Date_time.err change to File1__Date_time_orig.err
2. cd ../scripts/
3. sh File.sh

File1 directory is /data/data1directory/Sample_File1/logs/File1_Data_time.err
File2 directory is /data/data2directory/Sample_File2/logs/File2_Data_time.err
.....

My script looks like this. (runrunrun.sh)

#!/bin/bash
INPUT=$1
mv /data/*/Sample_*/logs/*_Data_time.err /data/*/Sample_*/logs/*_Data_time_orig.err
cp /data/*/Sample_*/scripts/*.sh /data/*/Sample_*/scripts/*_orig.sh
sh /data/*/Sample_*/scripts/*_orig.sh

When running it, I tried.
./runrunrun.sh File1
. runrunrun.sh File1
sh runrunrun.sh File1

mv: cannot move /data/data1directory/Sample_File1/logs/File1_Data_time.err /data/*/Sample_*/logs/*_Data_time_orig.err: No such file or directory cp also got similar feedback

Am I doing it correct?

Thanks!

Solution 1

Let's talk about how wildcards work for a minute.

cp *.txt foo

doesn't actually invoke cp with an argument *.txt, if any files matching that glob exist. Instead, it runs something like this:

cp a.txt b.txt c.txt foo

Similarly, something like

mv *.txt *.old

...can't possibly know what to do, because when it's invoked, what it sees is:

mv a.txt b.txt c.txt *.old

or, worse, if you already have a file named z.old, it'll see:

mv a.txt b.txt c.txt z.old

---

Thus, you need to use different tools. Consider:

# REPLACES: mv /data/*/Sample_*/logs/*_Data_time.err /data/*/Sample_*/logs/*_Data_time_orig.err
for f in /data/*/Sample_*/logs/*_Data_time.err; do
  mv "$f" "${f%_Data_time.err}_Data_time_orig.err"
done

# REPLACES: cp /data/*/Sample_*/scripts/*.sh /data/*/Sample_*/scripts/*_orig.sh
for f in /data/*/Sample_*/scripts/*.sh; do
  cp "$f" "${f%.sh}_orig.sh"
done

# REPLACES: sh /data/*/Sample_*/scripts/*_orig.sh
for f in /data/*/Sample_*/scripts/*_orig.sh; do
  if [[ -e "$f" ]]; then
    # honor the script's shebang and let it choose an interpreter to use
    "$f"
  else
    # script is not executable, assume POSIX sh (not bash, ksh, etc)
    sh "$f"
  fi
done

This uses a parameter expansion to strip off the tail end of the old name before adding the new name.

Solution 2

The find command can be used quite concisely in simple cases where you want to perform operations on wildcard (or more complex) filename matches. The technique below can be committed to memory ... almost !

This works by letting the find command run another command on each filename it finds. You can dry-run this example using echo instead of/in front of mv .

If we wanted to move all files in the current directory with name beginning 'report', to another parallel directory called 'reports' :

find . -name "report*.*" -exec mv '{}' ../reports/ \;

The wildcard string must be in quotes, the {} marking the filename that was 'found' must be in quotes, and the final semicolon must be escaped - all due to Bash/shell treatment of those characters.

Look at the man page for find for more uses: https://linux.die.net/man/1/find

Solution 3

This is what I use, very fast to write and remember

For copying:

ls -1 *.txt | xargs -L1 -I{} cp {} {}.old 

For moving:

ls -1 *.txt | xargs -L1 -I{} mv {} {}.old 

the explanation:

xargs(1) -L1 -I{}  

build and execute command lines from standard input

-L max-lines
       Use  at most max-lines nonblank input lines per command line.  Trailing blanks cause an input line
       to be logically continued on the next input line.  Implies -x.

-I replace-str
       Replace occurrences of replace-str in the initial-arguments with names read from  standard  input.
       Also,  unquoted  blanks  do  not  terminate  input  items;  instead  the  separator is the newline
       character.  Implies -x and -L 1.

Basically with -L1 it sends the arguments on by one, and -I{} makes {} the placeholder

Solution 4

using awk and sed

Filter using ls, use awk to build your move command for each record and the sed 'e' in the end will execute the command.

ls  *.avro  | awk ' { print "mv "$1" aa_"$1 }' | sed 'e'

First try without the sed -- once you know your command is showing correctly add the sed.

View more solutions