Why does parameter expansion with spaces without quotes work inside double brackets "[[" but not inside single brackets "["?

Question

I'm confused with using single or double brackets. Look at this code:

dir="/home/mazimi/VirtualBox VMs"

if [[ -d ${dir} ]]; then
    echo "yep"
fi

It works perfectly although the string contains a space. But when I change it to single bracket:

dir="/home/mazimi/VirtualBox VMs"

if [ -d ${dir} ]; then
    echo "yep"
fi

It says:

./script.sh: line 5: [: /home/mazimi/VirtualBox: binary operator expected

When I change it to:

dir="/home/mazimi/VirtualBox VMs"

if [ -d "${dir}" ]; then
    echo "yep"
fi

It works fine. Can someone explain what is happening? When should I assign double quotes around variables like "${var}" to prevent problems caused by spaces?

Solution 1

The single bracket [ is actually an alias for the test command, it's not syntax.

One of the downsides (of many) of the single bracket is that if one or more of the operands it is trying to evaluate return an empty string, it will complain that it was expecting two operands (binary). This is why you see people do [ x$foo = x$blah ], the x guarantees that the operand will never evaluate to an empty string.

The double bracket [[ ]], on the other hand, is syntax and is much more capable than [ ]. As you found out, it does not have the "missing operand" issue and it also allows for more C-like syntax with >, <, >=, <=, !=, ==, &&, || operators.

My recommendation is the following: If your interpreter is #!/bin/bash, then always use [[ ]]

It is important to note that [[ ]] is not supported by all POSIX shells, however many shells do support it such as zsh and ksh in addition to bash

Solution 2

The [ command is an ordinary command. Although most shells provide it as a built-in for efficiency, it obeys the shell's normal syntactic rules. [ is exactly equivalent to test, except that [ requires a ] as its last argument and test doesn't.

The double brackets [[ … ]] are special syntax. They were introduced in ksh (several years after [) because [ can be troublesome to use correctly and [[ allows some new nice additions that use shell special characters. For example, you can write

[[ $x = foo && $y = bar ]]

because the entire conditional expression is parsed by the shell, whereas [ $x = foo && $y = bar ] would first be split into two commands [ $x = foo and $y = bar ] separated by the && operator. Similarly double brackets enable things like the pattern matching syntax, e.g. [[ $x == a* ]] to test whether the value of x starts with a; in single brackets this would expand a* to the list of files whose names starts with a in the current directory. Double brackets were first introduced in ksh and are only available in ksh, bash and zsh.

Inside single brackets, you need to use double quotes around variable substitutions, like in most other places, because they're just arguments to a command (which happens to be the [ command). Inside double brackets, you don't need double quotes, because the shell doesn't do word splitting or globbing: it's parsing a conditional expression, not a command.

An exception though is [[ $var1 = "$var2" ]] where you need the quotes if you want to do a byte-to-byte string comparison, otherwise, $var2 would be a pattern to match $var1 against.

One thing you can't do with [[ … ]] is use a variable as an operator. For example, this is perfectly legal (but rarely useful):

if [ -n "$reverse_sort" ]; then op=-gt; else op=-lt; fi
…
if [ "$x" "$op" "$y" ]; then …

In your example

dir="/home/mazimi/VirtualBox VMs"
if [ -d ${dir} ]; then …

the command inside the if is [ with the 4 arguments -d, /home/mazimi/VirtualBox, VMs and ]. The shell parses -d /home/mazimi/VirtualBox and then doesn't know what to do with VMs. You would need to prevent word splitting on ${dir} to get a well-formed command.

Generally speaking, always use double quotes around variable and command substitutions unless you know you want to perform word splitting and globbing on the result. The main places where it's safe not to use the double quotes are:

• in an assignment: foo=$bar (but note that you do need the double quotes in export "foo=$bar" or in array assignments like array=("$a" "$b"));
• in a case statement: case $foo in …;
• inside double brackets except on the right hand side of the = or == operator (unless you do want pattern matching): [[ $x = "$y" ]].

In all of these, it's correct to use double quotes, so you might as well skip the advanced rules and use the quotes all the time.

Solution 3

When should I assign double quotes around variables like "${var}" to prevent problems caused by spaces?

Implicit in this question is

Why isn’t ${variable_name} good enough?

${variable_name} doesn’t mean what you think it does …

… if you think it has anything to do with problems caused by spaces or “glob” (filename pattern) characters (in variable values).  ${variable_name} is good for this:

$ bar=foo
$ bard=Shakespeare
$ echo $bard
Shakespeare
$ echo ${bar}d
food

and nothing else! ¹  ${variable_name} doesn’t do any good unless you’re immediately following it with a character that could be part of a variable name: a letter (A-Z or a-z), an underscore (_), or a digit (0-9).  And even then, you can work around it:

$ echo "$bar"d
food

I’m not trying to discourage its use — echo "${bar}d" is probably the best solution here — but to discourage people from relying on braces instead of quotes, or applying braces instinctively and then asking, “Now, do I need quotes, also?”  You should always use quotes unless you have a good reason not to, and you’re sure you know what you’re doing.
_________________
¹   Except, of course, for the fact that the fancier forms of parameter expansion, for example, ${parameter:-[word]}, ${parameter%[word]}, ${#parameter} and ${parameter^^}, build on the ${parameter} syntax.  Also, you need to use ${10}, ${11}, etc., to reference the 10th, 11th, etc., positional parameters — quotes won’t help you with that.

Solution 4

For handling spaces and blank|special characters in variables you should always surround them with double quotes. Setting a proper IFS is a good practice too.

Recommended: http://www.dwheeler.com/essays/filenames-in-shell.html

View more solutions

Comments

[jw013]

Bash FAQ 31

[Ciro Santilli Путлер Капут 六四事]

A more generic vs question: unix.stackexchange.com/questions/306111/…

[Stéphane Chazelas]

The empty string (and many other) problem is solved by using quotes. The "x" is to cover another kind of problems: where operands may be taken as operators. Like when $foo is ! or ( or -n... That problem is not meant to be a problem with POSIX shells where the number of arguments (beside [ and ]) is not greater than four.

[Gilles 'SO- stop being evil']

@StephaneChazelas A mistake of mine. It turns out that [ is indeed from System III in 1981, I thought it was older.

[Stéphane Chazelas]

To clarify, [ x$foo = x$blah ] is just as plain wrong as [ $foo = $bar ]. [ "$foo" = "$bar" ] is correct in any POSIX compliant shell, [ "x$foo" = "x$bar" ] would work in any Bourne-like shell, but the x is not for cases where you have empty strings but where $foo may be ! or -n...

[ssola]

Interesting semantics in this answer. I'm not sure what you mean by it's *not* syntax. Of course, [ isn't precisely an alias for test. If it were, then test would accept a closing square bracket. test -n foo ]. But test does not, and [ requires one. [ is identical to test in all other respects, but that's not how alias works. Bash describes [ and test as shell builtins, but [[ as a shell keyword.

[Evan]

Well, in bash [ is a shell builtin, but /usr/bin/[ is also an executable. It is traditionally a link to /usr/bin/test, but in modern gnu coreutils is a separate binary. The traditional version of test examines argv[0] to see if it is invoked as [ and then looks for the matching ].

[Student]

My bash does not work with >= and <=

[Nate T]

@kojiro Agreed, and regardless the test command is syntax as well. Aside from literals, (and I know I'll somehow regret stating this part) It is all bash syntax OP if you read this, maybe change to 'operator'? Sry to nit-pick, I know it was likely a figure of speech, but the Q/A is fairly popular now, and many in the linux realm get their start with coding via Ubuntu /Bash or similar, so this may be an introduction to the term for some.