Shell script to delete multiple files

Question

So, I have a file which contains a list of hundreds of files that need to be deleted from various directories. I figured the easiest method would be to create a batch file but I'm not having great success.

#/bin/bash
rm "/home/e-smith/files/users/bill/Maildir/cur/1392373930.28512.comp01:2,S"
rm "/home/e-smith/files/users/ted/Maildir/cur/1420726198.17690.comp01:2,S"

But when I run the script with

./myfile.txt

I get

rm: cannot remove '/home/e-smith/files/users/bill/Maildir/cur/1392373930.28512.comp01:2,S\r': no such file or directory
rm: cannot remove '/home/e-smith/files/users/ted/Maildir/cur/1420726198.17690.comp01:2,S\r': no such file or directory

I don't understand where the \r is coming from, I'm guessing it's a carriage return but it appends itself to the command.

What is the correct method/syntax for an exercise like this?

Solution 1

This is due to the line-endings that Windows uses (\r\n).

You can remove thes via vi:

$ vi myfile.txt
<ESC>
:set fileformat=unix
:wq

Solution 2

If you have a list of hundreds of filenames and you want to delete them, there is no reason to write separate rm commands for each of them. Assuming your file names saved in a file called list.txt, one per line, and that they do not contain newline characters, you could just do

while read -r file; do rm "$file"; done < list.txt 

Explanation

• The while read variable; do something; done < file construct will read each line from a file and save it as variable (in the example above, the variable's name is file). The -r is needed to allow for file names containing things like \r or \t. With the -r they will be treated literally while without it they will be expanded to a carriage return or a tab respectively.
• rm "$file" : this will remove each file in the list as it is read.

Or, you can use bash's mapfile builtin which lets you save the lines of a file in an array:

mapfile -t filelist < list.txt && rm "${filelist[@]}"

Solution 3

Thanks for the responses, the technical reason for this is probably in the file encoding somewhere but to fix I simply created my file on the CentOS box

$> touch myfile.txt
$> nano myfile.txt

Added the "!" in to my bash script (thanks @jimmij)

#!/bin/bash

And saved, I then used notepad++ on windows to add the files I wanted to remove as per my initial syntax and bingo.

Thanks for the suggestions @Marko Frelih, @jimmij

Solution 4

xargs will do, 'something', with a list of somethings...

cat file_containing_list_of_files_to_delete.txt | xargs rm

make a file

touch file1

look in this file called foo

cat foo

file1
file2
file3

delete the files mentioned in file foo.

cat foo | xargs rm

rm: cannot remove `file2': No such file or directory
rm: cannot remove `file3': No such file or directory

file1 gone now.

ls file1
ls: cannot access file1: No such file or directory

View more solutions

Comments

[KernelPanic]

Did you edit script in windows?

[pickles]

I created the script in windows, i've since opened in nano and resaved the document but it made no difference.

[KernelPanic]

@user9129, try to open it with vi and get rid of unwanted hidden chars

[pickles]

No unexpected characters in vi, saved and re-run but no success.

[KernelPanic]

@user9129, did you issue command :set list inside vi to show all characters?

[pickles]

I didn't but with the setting enabled the only characters besides my rm lines are the carriage return lines for each file that needs to be removed. Strange, historically i've run this script before without a problem...

[jimmij]

Apart of carriage return you also have wrong shebang, namely missing !, it should be #!/bin/bash.

[KernelPanic]

@user9129, try that command and you will se some characters in the script, remove them. I had same problem 10 mins ago.

[pickles]

I think that might be OS specific, Ubuntu for instance requires the ! but CentOS raises a "bad intepreter" error when it is included in the bash script.

[jimmij]

Perhaps your bash is not in /bin/bash but in other place, try type bash.

[pickles]

result = bash is /bin/bash

[jimmij]

So most probably you have also carriage return in the first line. Shebang is no OS specific, it must be in the form #!..., and when ! is missing your script runs in the current shell (whatever it is) and the lines which start from # are simply interpreted as a comments, so ignored.

[jimmij]

You may run command dos2unix if it is present on the system.

[terdon]

For future reference, there is no need to run touch. nano myfile.txt will create the file if it doesn't exist. Also, next time, you can run sed -i 's/\r//g' myfile.txt to remove the carriage returns.

[pickles]

very nice, I've saved this for the next time I carry out this task.