Linux: Run command on a batch of files, with matching output files

Question

A simple task: given a directory of files and a command that takes one file as input and produces one file as output, I want to process all the files in a directory at once and output them with corresponding matching names (with new extensions).

So if the command would normally be:

convert -input sourceFile.ext -output destFile.out

I want to process a folder with

file1.ext, file2.ext, etc.

and produce in the same directories the files

file1.out, file2.out, etc.

Is there any way to do this in terminal without writing a bash script? I'm not very familiar with scripting, so any simple solution would be appreciated.

Solution 1

Without using basename:

for file in *.ext; do convert -input "$file" -output "${file/%ext/out}"; done

See http://www.gnu.org/software/bash/manual/bashref.html#Shell-Parameter-Expansion

Solution 2

Look into the find command

Be sure to cd to your input directory.

cd input_directory
find . -iname "*.ext" -exec convert -input {} -output {}.out \;

This results in .out being appended to the name of your input file. Getting your stated output file I haven't figured out yet.

It's always wise to try out what find does before running the -exec that changes things.

cd input_directory    
find  .  -iname "*.ext"  -type f  -exec ls -l {} \;

will do a sort of "dry run".

• -exec what command to run
• {} what file was found; one at a time
• \; need to use this to end an -exec

---

My limited experimentation indicates you do not need to quote {}

$ find . -type f -exec ls -bl {} \;  
-rw-r--r--. 1 me me 0 Oct 20 19:03 ./a\ b\ c.txt  
-rw-r--r--. 1 me me 0 Oct 20 19:03 ./abc.txt  

$ ls -bl
total 0
-rw-r--r--. 1 me 0 Oct 20 19:03 a\ b\ c.txt
-rw-r--r--. 1 me 0 Oct 20 19:03 abc.txt
me ~/a $`

---

I have a text file, help.txt, that contains hints to all my hard to remember bash commands. I have it tied to a simple script to print out the file .., h

Here is a list of my find commands:

# searches for all files on the system for the string you fill in between the ""
sudo find  /  -type f -exec grep -il "" {} \;
# search for all files starting with python.
find / -iname 'python*'
# search for the file type .jpeg and sort the list by date
find  ~  -iname "*.jpeg" -type f  -exec ls -l  {} \; 2>/dev/null | sort -r -k 8,8 -k 6,7M
# so I can remember the or syntax. 
find  ~  \( -iname "*.jpeg" -o -iname "*.png"  \) -type f  -exec ls -l  {} \;

Solution 3

Try this (it's all one line):

find /path/to/folder -maxdepth 1 -iname \*.ext -exec sh -c 'convert -input "{}" -output "/path/to/folder/$(basename "{}" .ext).out"' \;

Here's the breakdown:

find /path/to/folder -maxdepth 1 -iname \*.ext

searches for all files ending with .ext in /path/to/folder but not in the subfolders. The -exec portion tells the find command what to do with each file that it finds. Each occurence of {} is then replaced with the file. The pattern is

find /path/to/folder -iname \*.ext -exec echo "found this file: {}" \;

As I said, the {} expands to the name of the file found. The \; marks the end of the -exec parameter. The sh -c is used in this case because otherwise the command $(basename {} .ext) would be expanded by the shell prematurely. So we enclose it in single quotes to prevent expansion and pass it to another instance of the shell. The {} will be expanded by find, and the rest of the command will be passed verbatim to sh. The command

$(basename "filename.ext" .ext).out

will strip the extension from "filename.ext" and append the .out extension.

EDIT:

I just realized how unapproachable that command must look. Here is the most bare-bones version of it.

find . -name \*.ext -exec sh -c 'convert -input {} -output $(basename {} .ext).out' \;

It almost fits without scrolling this time. :)

Solution 4

It depends what you mean by “writing a bash script”.  The following is a bash script, but you can just type it into the terminal, if you want; you don’t have to put it into a file:

cddirectory_where_the files_are
for x in *.ext
do
        convert -input "$x" -output $(basename "$x" .ext).out
done

This will operate only on *.ext files in the directory that you cd to.  If you want to search subdirectories also, find is the best solution; e.g., use nispio’s answer but leave out the -maxdepth 1.

---

Edit: Hennes's comment on nispio’s answer applies here, too; you could say

for x in /path/to/folder/*.ext
do
        convert -input "$x" -output $(basename "$x" .ext).out
done

View more solutions

Comments

[historystamp]

nispio: You do "{}" to worry about blanks? Perhaps you need a / before $(basname ... ? I'm too new to add a comment to your post :(

[nispio]

Yes. the quotes are mostly to avoid problems with spaces.

[nispio]

This is definitely easier than trying to remember all of the syntax that goes along with my answer. You can even turn it into a "one-liner " by adding some semi-colons: for x in *.ext; do convert -input "$x" -output $(basename "$x" .ext).out; done

[ganesh]

If you need to do this often an do not want to type the command every time, then save it to a file. Add #!/usr/bin/env bash as the first line and set the script to executable with chmod +x scriptname.

[ganesh]

If the command (convert in this case) supports working with a path then you might not even need to cd into the target directory. Just use find /path/to/folder rather than find ..

[ganesh]

True. But even the one-liners seem to be to long and or to complex to type in regularly. So you might as well save the command in a file to copy and paste it. And if you do that, then why not make that file executable. Add to that that the OP did not just mention specifically that he was not looking for a bash script, but that he is not very familiar with scripting. In my mind that does not conflict with a very simple, well explained script.

[Scott - Слава Україні]

That’s peculiar; “Is there any way to do this in terminal without writing a bash script?” seems fairly specific to me.

[Besi]

I had to do the following to get the extension: ... -output "${file%.*}.wav"; rm "$file"; done. Also I did a `rm "$file" since after converting I don't need the input file anylonger