Bash globbing that matches all files except those with a specific extension, that works on filenames that include dot characters

Question

I'm doing some stuff with audio files, most but not all of which are mp3 files. Now I want to run some commands on only the files which are not mp3 files, or only those which don't have a .mp3 extension.

I consider myself pretty good at regular expressions, but not so much at file globbing, which is subtly different in unexpected ways.

I looked around and learned from other SO & SE answers that Bash has "extended globbing" that allows me to do this:

file ../foo/bar/*.!(mp3)

But some of my filenames have dots in them besides the one forming the filename extension:

../foo/bar/Naked_Scientists_Show_19.10.15.mp3
../foo/bar/YWCS_ep504-111519-pt1_5ej4_41cc9320.mp3_42827d48daefaa81ec09202e67fa8461_24419113.mp3
../foo/bar/eLife_Podcast_19.09.26.mp3
../foo/bar/gdn.sci.080428.bg.science_weekly.mp3

It seems the glob matches from the first dot onward, rather than from the last dot. I looked at the documentation but it seems they are far less powerful than regexes. But I didn't really grok everything as I don't spend that much time on *nix shells.

Have I missed some way that I can still do this with Bash globbing? If not, a way to achieve the same thing with find or some other tool would still be worth knowing.

Solution 1

*.!(mp3) matches on foo.bar.mp3 because that's foo. followed by bar.mp3 which is not mp3.

You want !(*.mp3) here, which matches anything that doesn't end in .mp3.

If you want to match files whose name contains at least one . (other than a leading one which would make them a hidden file) but don't end in .mp3, you could do !(*.mp3|!(*.*)).

Solution 2

There's also the GLOBIGNORE variable:

The GLOBIGNORE shell variable may be used to restrict the set of file names matching a pattern. If GLOBIGNORE is set, each matching file name that also matches one of the patterns in GLOBIGNORE is removed from the list of matches. If the nocaseglob option is set, the matching against the patterns in GLOBIGNORE is performed without regard to case. The filenames . and .. are always ignored when GLOBIGNORE is set and not null. However, setting GLOBIGNORE to a non-null value has the effect of enabling the dotglob shell option, so all other filenames beginning with a ‘.’ will match. To get the old behavior of ignoring filenames beginning with a ‘.’, make ‘.*’ one of the patterns in GLOBIGNORE. The dotglob option is disabled when GLOBIGNORE is unset.

$ touch a.b.mp3 .c.mp3 foo.tgz .bar
$ echo *
a.b.mp3 foo.tgz
$ shopt -s extglob; echo *.!(mp3)
a.b.mp3 foo.tgz
$ GLOBIGNORE='*.mp3'; echo *
.bar foo.tgz

Solution 3

the simplest way I can think of:

find ${path-to-folder} -type f  | grep -vE ".*\.mp3$"

You find all files in a folder and pipe it to a reverse extended grep:

Find all files in a location

find ${path-to-folder} -type f 

Reverse grep and use a regex to filter by extension

grep -vE ".*\.mp3$"

Grep flags:

• -v reverse grep

• -E extended grep (use of regex)

Regex explained:

• .* means from 0 to whatever char count, any possible char

• \. looks for the actual dot

• mp3 for the string that forms the extension

• $ stands for EOL

Solution 4

Possibly not ideal for your situation but I was wondering how about making a function in pure Bash and then passing the extension as an arg, which can be handy if you want to add it in your bashrc and you want to do the same thing in the future, like the following

So, basically what we are doing is the regex check for your desired extension.

for file in *; do
  if [[ $file =~ \.mp3$ ]]; then
    echo $file
  fi
done

Implementing it in a function: This would output all the files with specific extension in your current directory.

getFileByExt(){
local extension="$1"
if (( $# == 1 )); then
for file in *; do
  if [[ $file =~ \.$extension$ ]]; then
    echo $file
  fi
done
else
  echo "Usage: ${FUNCNAME[0]} <extension>" 
## If run from bashrc, return the name of function
fi
}

Now, for example to get all the files with mp3 extension, it would be as simple as doing:

getFileByExt mp3

And to get files without specefic extension: This would output all the files that do not have specific extension in your current directory.

getFileWithoutExt(){
local extension="$1"
if (( $# == 1 )); then
for file in *; do
  if [[ ! $file =~ \.$extension$ ]]; then
    echo $file
  fi
done
else
  echo "Usage: ${FUNCNAME[0]} <extension>"
fi
}

Similarly to get files without mp3 extension just simply do:

getFileWithoutExt mp3

For more info:

https://stackoverflow.com/questions/407184/how-to-check-the-extension-of-a-filename-in-a-bash-script

https://stackoverflow.com/questions/18278990/easiest-way-to-check-for-file-extension-in-bash/18280350

View more solutions

Comments

[Stéphane Chazelas]

Or just grep -v '\.mp3$' (no need for -E as there's no extended operator there), no need for .* if you're not using the -x option. That assumes the file paths don't contain newline characters though.