How do I get a list of directories in bash and then expand them as command line parameters?

Question

I'm writing a bash script which needs to, for one step, get a list of directories (variable) in a target directory (which may also contain files), and then expand them out as parameters to a python script.

Example:

/stuff/a dir/
/stuff/b other/
/stuff/c/

And I need to, within a bash script, call:

script.py "a dir/" "b other/" "c/"

or alternately, escaped spaces:

script.py a\ dir/ b\ other/ c/

I need the script to be called exactly once for directory 'stuff'.

Is there a straightforward way to do this kind of thing? I've been googling around and the best I've managed to figure out requires me to know how many directories there are.

Solution 1

This is a job for find.

find /stuff -type d -exec script.py {} +

When you use -exec the curly braces {} are replaced with the names of the matching files, and + indicates the end of the command (in case you want to tell find to take additional actions). This is the ideal way to execute a command using find as it will handle file names with unusual characters (such as whitespace) correctly.

find is quite flexible, especially if you have the GNU version typically bundled with Linux distros.

# Don't recurse into subdirectories.
find /stuff -maxdepth 1 -type d -exec script.py {} +

# Pass in a/, b/, c/ instead of /stuff/a/, /stuff/b/, /stuff/c/.
find /stuff -type d -printf '%P\0' | xargs -0 script.py

In the second example notice the careful use of \0 and xargs -0 to use the NUL character to delimit file names. It might seem odd but this allows the command to work even if you do something really weird like use newlines \n in your directory names.

---

Alternatively, you could do this using only shell builtins. I don't recommend this, but for educational value, here's how:

# Start with an empty array.
DIRS=()

# For each file in /stuff/...
for FILE in /stuff/*; do
    # If the file is a directory add it to the array. ("&&" is shorthand for
    # if/then.)
    [[ -d $FILE ]] && DIRS+=("$FILE")

    # (Normally variable expansions should have double quotes to preserve
    # whitespace; thanks to bash magic we don't them inside double brackets.
    # [[ ]] has special parsing rules.)
done

# Pass directories to script. The `"${array[@]}"` syntax is an unfortunately
# verbose way of expanding an array into separate strings. The double quotes
# and the `[@]` ensure that whitespace is preserved correctly.
script.py "${DIRS[@]}"

Solution 2

A simpler solution that does not create a new process (as find does) is:

for f in stuff/*; do
  if [ -d "$f" ]; then
     ./script.py "$f"
  fi
done

Solution 3

You can use the find command and tell it to only print out the directories with -type d. Your command will look like this:

script.py $(find /stuff/* -type d)

If you're worried about spaces and other special characters, you can do this:

script.py $(find /stuff/* -type d | while read line; do echo "\"$line"\"; done)

Solution 4

find /stuff/* -type d -maxdepth 1 -print0 | xargs -0 script.py

This will find all the directories under /stuff, but not recursively and pass them to script.py and make sure they are passed correctly even if there are spaces in the directory names.

View more solutions

Comments

[Paul V]

Keep in mind that find will recurse into directories. You need -maxdepth 1. Also, -exec script.py {} + could possibly call script multiple times, if there are many directories. It's unclear from the question if that's ok.

[Charles Randall]

Is there a way to wrap the individual outputs in quotes so that it properly handles spaces in the directory names?

[Charles Randall]

This almost works, except that it also lists /stuff in the output, which would cause me problems.

[Charles Randall]

Slightly off topic, but why would you not recommend just doing it with shell builtins?

[Steve Prentice]

Updated based on your comment.

[Peter.O]

@Charles Randall: Re 'builtin' vs 'not-builtin' .. find has some particularly useful features for dealing with files; moreso than the "builtins' eg. when dealing with space-embedded filenames (see -print0 in man find), and the ability to exec.. for more info about the how/what/why of builtins, see: unix.stackexchange.com/questions/11454/…

[John Kugelman]

@Charles I didn't mean to imply that the bash-only script is bad. It's perfectly good code and has no flaws that I know of. It's just more verbose than using find, that's all.

[Sam Daniel]

This recurses into the subdir. Using -mindepth 1 together with maxdepth 1 without the * prints only the immediate directories.