Test whether a glob has any matches in Bash

Question

If I want to check for the existence of a single file, I can test for it using test -e filename or [ -e filename ].

Supposing I have a glob and I want to know whether any files exist whose names match the glob. The glob can match 0 files (in which case I need to do nothing), or it can match 1 or more files (in which case I need to do something). How can I test whether a glob has any matches? (I don't care how many matches there are, and it would be best if I could do this with one if statement and no loops (simply because I find that most readable).

(test -e glob* fails if the glob matches more than one file.)

Solution 1

Bash-specific solution:

compgen -G "<glob-pattern>"

Escape the pattern or it'll get pre-expanded into matches.

Exit status is:

• 1 for no-match,
• 0 for 'one or more matches'

stdout is a list of files matching the glob. I think this is the best option in terms of conciseness and minimizing potential side effects.

Example:

if compgen -G "/tmp/someFiles*" > /dev/null; then
    echo "Some files exist."
fi

Solution 2

The nullglob shell option is indeed a bashism.

To avoid the need for a tedious save and restore of the nullglob state, I'd only set it inside the subshell that expands the glob:

if test -n "$(shopt -s nullglob; echo glob*)"
then
    echo found
else
    echo not found
fi

For better portability and more flexible globbing, use find:

if test -n "$(find . -maxdepth 1 -name 'glob*' -print -quit)"
then
    echo found
else
    echo not found
fi

Explicit -print -quit actions are used for find instead of the default implicit -print action so that find will quit as soon as it finds the first file matching the search criteria. Where lots of files match, this should run much faster than echo glob* or ls glob* and it also avoids the possibility of overstuffing the expanded command line (some shells have a 4K length limit).

If find feels like overkill and the number of files likely to match is small, use stat:

if stat -t glob* >/dev/null 2>&1
then
    echo found
else
    echo not found
fi

Solution 3

I like

exists() {
    [ -e "$1" ]
}
if exists glob*; then
    echo found
else
    echo not found
fi

This is both readable and efficient (unless there are a huge number of files).
The main drawback is that it's much more subtle than it looks, and I sometimes feel compelled to add a long comment.
If there's a match, "glob*" is expanded by the shell and all the matches are passed to exists(), which checks the first one and ignores the rest.
If there's no match, "glob*" is passed to exists() and found not to exist there either.

Edit: there may be a false positive, see comment

Solution 4

#!/usr/bin/env bash
# If it is set, then an unmatched glob is swept away entirely -- 
# replaced with a set of zero words -- 
# instead of remaining in place as a single word.
shopt -s nullglob
M=(*px)
if [ "${#M[*]}" -ge 1 ]; then
    echo "${#M[*]} matches."
else
    echo "No such files."
fi

Solution 5

If you have globfail set you can use this crazy (which you really should not)

shopt -s failglob # exit if * does not match 
( : * ) && echo 0 || echo 1

or

q=( * ) && echo 0 || echo 1

View more solutions

Comments

[Brian Chrisman]

I suspect my answer below is 'clearly correct' in a way that all the others kind of hack-around. It's a one-line shell-builtin that's been around forever and appears to be 'the intended tool for this particular job'. I'm concerned that users will mistakenly reference the accepted answer here. Anybody please feel free to correct me and I'll withdraw my comment here, I'm more than happy to be wrong and learn from it. If the difference didn't appear so drastic, I wouldn't raise this issue.

[Ken Bloom]

My favorite solutions to this question are the find command which works in any shell (even non-Bourne shells) but requires GNU find, and the compgen command which is clearly a Bashism. Too bad I can't accept both answers.

[Chris Johnsen]

To avoid a possible false “no matches” set nullglob instead of checking to see if a single result equals the pattern itself. Some patterns can match names that are exactly equal to the pattern itself (e.g. a*b; but not e.g. a?b or [a]).

[Ken Bloom]

I suppose this fails on the highly unlikely chance that there's actually a file named like the glob. (e.g. somebody ran touch '*py'), but this does point me in another good direction.

[Ken Bloom]

I like this one as the most general version.

[Ken Bloom]

find seems to be exactly correct. It has no corner cases, since the shell isn't doing expansion (and passing an unexpanded glob to some other command), it's portable between shells (though apparently not all of the options you use are specified by POSIX), and it's faster than ls -d glob* (the previous accepted answer) becasue it stops when it reaches the first match.

[Stephane Chazelas]

It may return a false positive if the glob is something like *.[cC] (there may be not c or C file, but a file called *.[cC]) or false negative if the first file expanded from that is for instance a symlink to an unexistent file or to a file in a directory you don't have access to (you way want to add a || [ -L "$1" ]).

[Ryan C. Thompson]

That would not work correctly if there was a file called *py.

[yegle]

If there's no file end with py, `echo *py` will be evaluate to *py.

[Ryan C. Thompson]

Yes, but it will also do so if there is a single file called *py, which is the wrong result.

[yegle]

Correct me if I'm wrong, but if there's no file that matches *py, your script will echo "Glob matched"?

[Ryan C. Thompson]

No, that's incorrect. I've added an explanation of how my answer works.

[yegle]

Tompson, set nullglob will not enable the option, but shopt -s nullglob will.

[Ryan C. Thompson]

Huh, you're right. I posted this a while ago, but I thought I tested it. Thanks for correcting me.

[Calimo]

Note that this answer may require a shopt -u failglob as these options seem to conflict somehow.

[flabdablet]

This works exactly like the version I already presented using stat; not sure how find is "easier" than stat.

[flabdablet]

Be aware that &> redirection is a bashism, and will quietly do the wrong thing in other shells.

[We Are All Monica]

The find solution will match a filename with no glob characters as well. In this case, that's what I wanted. Just something to be aware of though.

[Peter Cordes]

That still doesn't fix the false-positive on filenames that have glob special-characters in them, like Stephane Chazelas points out for Dan Bloch's answer. (unless you monkey with nullglob).

[Peter Cordes]

And also shortest. If you are only expecting one match, you can use "$M" as a shorthand for "${M[0]}". Otherwise, well you already have the glob expansion in an array variable, so you're gtg for passing it to other things as a list, instead of making them re-expand the glob.

[Stephane Chazelas]

You shoud avoid -o and -a in test/[. For instance, here, it fails if $1 is = with most implementations. Use [ -e "$1" ] || [ -L "$1" ] instead.

[Tobia]

Nice. You can test M more quickly (less bytes and without spawning a [ process) with if [[ $M ]]; then ...

[Brian Chrisman]

A fantastic use of a noop failing. Never should be used... but really beautiful. :)

[drwatsoncode]

This seems to be better than flabdablet's find answer because it accepts paths in the glob and it is more terse (doesn't require -maxdepth etc). It also seems better than his stat answer because it doesn't continue to do the extra stating on each additional glob match. I'd appreciate if anyone could contribute corner cases where this doesn't work.

[drwatsoncode]

After futher consideration, I would add -maxdepth 0 because it allows more flexibility in adding conditions. e.g. assume I want to restrict the result to matching files only. I might try find $glob -type f -quit , but that would return true if the glob did NOT match a file, but did match a directory that contained a file (even recursively). In contrast find $glob -maxdepth 0 -type f -quit would only return true if the glob itself matched at least one file. Note that maxdepth does not prevent the glob from having a directory component. ( FYI 2> is sufficient. no need for &>)

[Diomidis Spinellis]

Note that compgen is a bash-specific built-in command and is not part of the POSIX standard Unix shell specified built-in commands. pubs.opengroup.org/onlinepubs/9699919799 pubs.opengroup.org/onlinepubs/9699919799/utilities/… Therefore, avoid using it in scripts where portability to other shells is a concern.

[Ian Kelling]

Works except if the file is actually named 'glob*'.

[Dewi Morgan]

It seems to me that a similar effect without bash builtins would be to use any other command which acts on a glob and fails if no files matched, such as ls: if ls /tmp/*Files 2>&1 >/dev/null; then echo exists; fi - maybe useful for code golf? Fails if there's a file named the same as the glob, which the glob shouldn't have matched, but if that's the case you probably have bigger problems.

[Jesin]

Since when is stat "Bash only"?

[flabdablet]

Since somebody else decided to edit my answer to make it say that, apparently.

[DKebler]

does work for passing in glob as variable - gives "too many arguments" error when there is more than one match. "$(echo $GLOB)" is not returning a single string or at least it's not interpreted as single single thus the too many arguments error

[Ken Bloom]

unix.stackexchange.com/questions/275637/… discusses how to replace the -maxdepth option for a POSIX find.

[flabdablet]

The point of using find in the first place is to avoid having the shell generate and sort a potentially huge list of glob matches; find -name ... -quit will match at most one filename. If a script relies on passing a shell-generated list of glob matches to find, invoking find achieves nothing but unnecessary process-startup overhead. Simply testing the resulting list directly for non-emptiness will be quicker and clearer.

[user4581301]

@DewiMorgan This is simpler: if ls /tmp/*Files &> /dev/null; then echo exists; fi

[sondra.kinsey]

find is usually the resilient and portable answer when scripting, but for me, detecting a glob match is usually to prepare for using the glob, so I want all the existing shopt glob options and stick to the shell itself so I get the same results.

[Brian Chrisman]

yeah, quote it or the filename wildcard will be pre-expanded. compgen "dir/*.ext"

[Toby Speight]

@johnraff, yes, I normally assume nounset is active. Also, it might be (slightly) cheaper to test the string is non-empty than to check for a file's presence. Unlikely though, given that we've just performed a glob, meaning the directory contents should be fresh in the OS's cache.

[user1934428]

@DKebler : it should be interpreted as single string, because it is wrapped in double-quotes.

[Masoud Rahimi]

For a better answer try to add some explanation to your code.

[Thomas Praxl]

Interesting. Shellcheck reports that globbing only works with -e, when there are 0 or 1 matches. It doesn't work for multiple matches, because that would become [ -e file1 file2 ] and this would fail. Also see github.com/koalaman/shellcheck/wiki/SC2144 for the rationale and suggested solutions.

[flabdablet]

You can put the shopt inside the parens. That way it only affects the test: (shopt -s failglob; : *) 2>/dev/null && echo exists

[tripleee]

The duplicate stackoverflow.com/questions/6363441/… has a number of other non-Bash solutions, many of them horrible.

[z2s8]

Doesn't work if the glob contains {}, for example ./dir/*.{ext1,ext2}, while bash otherwise can expand it

[Brian Chrisman]

might try extglob... ./dir/*.(ext1|ext2)

[Charles Duffy]

In this case, though, it's not even good code. If you have a file named glob -a foo = bar, you'll get a false result even though it should be true; and it's extremely inefficient.

[Charles Duffy]

This will fail if the nullglob shell option is set, and it's always unnecessarily slow (as $(...) involves forking off a new copy of the shell).

[Dr. Jan-Philip Gehrcke]

Added a new answer for extended globs: stackoverflow.com/a/65217274/145400. About Doesn't work if the glob contains {}: Brace expansion is not working well with the compgen approach because brace expansion is not actually globbing! It's a separate mechanism. Maybe use extended globbing instead of brace expansion, as I've tried to show in my answer :-).

[teknopaul]

This does not test for files, it tests for directories that match glob*

[tripleee]

As always, don't use ls in scripts.

[not2savvy]

The find solution did not work when there's a path in the pattern, but stat did.

[flabdablet]

The difference is because the stat solution has the shell in charge of expanding the glob, but the find solution passes it unexpanded to find, whose -name option is specifically designed to match against the filename (last pathname component) only. There are also options (-path, -ipath, -regex, others) for matching whole pathnames against a given pattern. None of these work quite the same way that shell globbing does; see the find manual for details.

[John Vincent]

@DewiMorgan: your suggestion of: if ls /tmp/*Files 2>&1 >/dev/null; then echo exists; fi is bascially a good one, but the redirection won't work as written, you want: if ls /tmp/*Files >/dev/null 2>&1 ; then echo exists; fi

[Dewi Morgan]

@JohnVincent Good catch! Though Clay Bridges' suggestion of &> is cleaner than my suggestion anyway, at least where supported.