How to store a path built with wildcards and containing with spaces into a variable

Question

Here's the situation (I'm on a Mac, OS X El Capitan):

# This works:

$ cd /Applications/Adobe\ Illustrator*/Cool\ Extras.localized/en_US/Templates/;

# These do not work:

$ INSTALL_DIR=/Applications/Adobe\ Illustrator*/Cool\ Extras.localized/en_US/Templates;
$ cd $INSTALL_DIR
# Moves me here: /Applications/Adobe

$ cd "$INSTALL_DIR"
-bash: cd: /Applications/Adobe Illustrator*/Cool Extras.localized/en_US/Templates: No such file or directory

$ cd "${INSTALL_DIR}"
-bash: cd: /Applications/Adobe Illustrator*/Cool Extras.localized/en_US/Templates: No such file or directory

My goal is to use $INSTALL_DIR in tar like so:

$ tar -xz $SOURCE_ZIP --strip-components 1 -C $INSTALL_DIR "*.ait";

Unfortunately, the -C (changing to destination directory) doesn't like the spaces in $INSTALL_DIR; if I use quotes, I can't get the * to work.

Is there an elegant way to handle this scenario?

Solution 1

When the * is not quoted the shell expands the argument list before running the command. It passes the expand argument list to the program.

When the * appears in a quoted string it is not expanded by the shell before being passed to the program.

Try expanding the path, assigning it to another variable, and then quoting the second variable when passing it as an argument.

Solution 2

The problem isn't tar, it's in your shell code. The argument of the -C option is supposed to be a path, not a wildcard pattern. Notice that you had exactly the same problem with the cd command.

You stored a wildcard pattern in the INSTALL_DIR variable. When you write -C $INSTALL_DIR, this applies the “split+glob” operator: take the value of the variable, split it at whitespace, and interpret each word as a wildcard pattern which is then expanded. Here the value of the variable is /Applications/Adobe Illustrator*/Cool Extras.localized/en_US/Templates, which is split into three words /Applications/Adobe, Illustrator*/Cool, Extras.localized/en_US/Templates; the middle word contains a wildcard (*), so it's interpreted as a wildcard pattern, but since it doesn't match any file, the pattern is left as is. This makes the argument of the -C option the string /Applications/Adobe, and then there are two more arguments to the tar command: Illustrator*/Cool and Extras.localized/en_US/Templates.

If you use double quotes, then "$INSTALL_DIR" is simply the value of the variable INSTALL_DIR. With the * still in it, since it was never expanded at any point.

As a rule of thumb, wildcards are expanded in contexts where multiple words are expected. After all, in general, wildcard patterns match multiple files. The right-hand side of an assignment suppresses wildcard expansion, because the result is expected to be a single string. To get a list instead, assign to an array variable instead of a string variable:

INSTALL_DIRS=(/Applications/Adobe\ Illustrator*/Cool\ Extras.localized/en_US/Templates)

There could potentially be multiple array elements, if you have multiple versions of Illustrator installed. Let's take the last element.

INSTALL_DIR=${INSTALL_DIRS[$((${#INSTALL_DIRS}-1))]}

Now INSTALL_DIR is a path to an existing file (assuming that the wildcard did match). You can use it normally (i.e. you can expand it inside double quotes).

tar -xz "$SOURCE_ZIP" --strip-components 1 -C "$INSTALL_DIR" "*.ait"

Comments

[pfnuesel]

Is there actually an * in your folder name or do I misunderstand your question?

[mhulse]

Oh, sorry, the actual folder name that I'm trying to target is: Adobe Illustrator CC 2015. When adobe apps upgrade, they change the CC and year in the path name; I just wanted to have my script be more future-compatible by not caring about the version name (CC) or the year (2015). Thanks for asking and reading my question!

[mhulse]

Ahhh, interesting. Trying that now and I'll let you know how it goes. Thanks!

[Chad Clark]

Let us know if it works. By the way in case it ever comes up DOS always passes the * without expanding. Each Windows application needs code to expand wildcards.

[mhulse]

Ah, this works: INSTALL_DIR=$(cd /Applications/Adobe\ Illustrator*/Cool\ Extras.localized/en_US/Templates; pwd);. Then in my tar line, I wrap the variables in quotes (e.g. "$INSTALL_DIR"). Thanks for tips and help Chad, I really appreciate it!

[mhulse]

Out of curiosity, is there a better way to expand the path?

[mikeserv]

@mhulse - cd /Applications/"Adobe Illustrator"*/"Cool Extras.localized/en_US/Templates" && cd - && tar ... | tar ... -"C$OLDPWD" ...

[mhulse]

Oooh, nice one @mikeserv; nice use of cd - too! +1 :)

[mhulse]

Cool! Thanks so much for the detailed answer Giles! Very helpful. Also, +100 karma points for the INSTALL_DIR=${INSTALL_DIRS[-1]} tip! After I posted this question, I realized that * could target the wrong Adobe folder if multiple installed ... I was not sure how best to tackle this. Using -1 is a great solution! Simple and clean. Thank you so much for taking the time to help me (and future readers) out! :)

[mhulse]

When I put two folders on my desktop (foo 1 and foo 2) and run: INSTALL_DIRS=(~/Desktop/foo*), then run: INSTALL_DIR=${INSTALL_DIRS[-1]}, I get this output: -bash: INSTALL_DIRS: bad array subscript. The output of echo $INSTALL_DIRS is this: /Volumes/hdd/Users/user/Desktop/foo 1. I hate to bug you with more question(s), but can you see what I'm doing wrong here?

[Gilles 'SO- stop being evil']

@mhulse You aren't doing anything wrong. I'd forgotten that OSX ships a very old version of bash that doesn't support negative array indices to mean counting from the end. I've fixed my answer.

[mhulse]

Ah, OSX shipping with an old version of bash would explain things. Thank you so much Gilles, I really appreciate it!!!