bash: how to pass command line arguments containing special characters

Question

I've written myself a linux program program that needs a regular expression as input.

I want to call the program in the bash shell and pass that regular expression as a command line argument to the program (there are also other command line arguments). A typical regular expression looks like

[abc]\_[x|y]

Unfortunately the characters [, ], and | are special characters in bash. Thus, calling

program [abc]\_[x|y] anotheragument

doesn't work. Is there a way to pass the expression by using some sort of escape characters or quotation marks etc.?

(Calling program "[abc]\_[x|y] anotheragument" isn't working either, because it interprets the two arguments as one.)

Solution 1

You can either:

1. Escape each single special symbol with a backslash (as in \[abc\]_\[x\|y\]) or
2. Double-quote the entire argument (as in "[abc]_[x|y]").

EDIT: As some have pointed out, double-quoting does not prevent variable expansion nor command substitution. Therefore if your regex contains something that can be interpreted by bash as one of those, use single quotes instead.

Solution 2

Use single quotes. Single quotes ensure that none of the characters are interpreted.

$ printf %s 'spaces  are  not  interpreted away
neither are new lines
nor variable names $TESTING
nor square brackets [TESTING]
nor pipe characters or redirection symbols | > <
nor the semicolon ;
nor backslashes \a \b \c \\
the only thing that does not work is the single quote itself
'

There are two solutions if you need to embed a single quote:

$ printf '%s\n' '[ Don'"'"'t worry, be happy! ]'
[ Don't worry, be happy! ]
$ printf '%s\n' '[ Don'\''t worry, be happy! ]'
[ Don't worry, be happy! ]

Solution 3

Per man bash

There are three quoting mechanisms: the escape character, single quotes, and double quotes.

A non-quoted backslash (\) is the escape character. It preserves the literal value of the next character that follows, with the exception of <newline>. If a \<newline> pair appears, and the backslash is not itself quoted, the \<newline> is treated as a line continuation (that is, it is removed from the input stream and effectively ignored).

Enclosing characters in single quotes preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.

Enclosing characters in double quotes preserves the literal value of all characters within the quotes, with the exception of $, `, \, and, when history expansion is enabled, !. The characters $ and ` retain their special meaning within double quotes. The backslash retains its special meaning only when followed by one of the following characters: $, `, ", \, or <newline>. A double quote may be quoted within double quotes by preceding it with a backslash. If enabled, history expansion will be performed unless an ! appearing in double quotes is escaped using a backslash. The backslash preceding the ! is not removed.

The special parameters * and @ have special meaning when in double quotes (see PARAMETERS below).

Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard. Backslash escape sequences, if present, are decoded as follows:

       \a     alert (bell)
       \b     backspace
       \e
       \E     an escape character
       \f     form feed
       \n     new line
       \r     carriage return
       \t     horizontal tab
       \v     vertical tab
       \\     backslash
       \'     single quote
       \"     double quote
       \nnn   the eight-bit character whose value is the octal value nnn
              (one to three digits)
       \xHH   the eight-bit character whose value is the hexadecimal value HH
              (one or two hex digits)
       \uHHHH the Unicode (ISO/IEC 10646) character whose value is
              the hexadecimal value HHHH (one to four hex digits)
       \UHHHHHHHH
              the Unicode (ISO/IEC 10646) character whose value is
              the hexadecimal value HHHHHHHH (one to eight hex digits)
       \cx    a control-x character

The expanded result is single-quoted, as if the dollar sign had not been present.

A double-quoted string preceded by a dollar sign ($"string") will cause the string to be translated according to the current locale. If the current locale is C or POSIX, the dollar sign is ignored. If the string is translated and replaced, the replacement is double-quoted.

Solution 4

You can use a backslash ( \ ) in front of special characters to escape them like so:

john@awesome:~ # echo \&
&

Solution 5

Although it might not be useful as a regex, some character sequences may be interpreted as Bash variable names. To prevent this from occurring and avoid having them expanded, use single quotes instead of double quotes:

program '[abc]_[x|y]' anotherargument

Quote each argument separately (if they need quoting) so they are interpreted as independent arguments. You can also use arrays in some cases:

param_array=('[abc]_[x|y]' anotherargument)    # create an array
param_array+=(yetanother)     # append another element to the array
program "${param_array[@]}"   # use the array elements as arguments to program

View more solutions

Comments

[jpalecek]

This is one of the most correct answers here (I appreciate especially the instruction for replacing ' with '"'"'), however, it is still not correct. [ IS a special character to shell, it is used in wildcards when doing path-expansion (which shell does for everything unquoted).

[Phil P]

It's special in some contexts, such as variable subscripting or for globbing, but you can still type foo=a[b] and then echo $foo and see that the string did not need quoting. You're right, I was too brief.

[Flimm]

In bash, double-quoting does not bypass expanding variables "$HOME" or parameters "${USER:-root}", command substitution in either form "$(date)" or "`date`", arithmetic expansion "$((1 + 2))", history expansion "!!" or backslash escaping "\\". Use single quotes instead. See the man page of the bash manual, the section titled "Quoting".

[Daniel Labonté]

If you are unlucky, there is a file ab in the current directory, and then foo will contain ab rather than a[b]. Quote your square brackets, people.

[Phil P]

(For clarity: I do quote (as the original answer made clear, where I was pushing for quoting), and this is a side-derailment which I'm addressing). This assertion surprised me, so I tested it. It's not true in zsh or bash, but is true in BSD /bin/sh. This is against POSIX and is non-standard behavior, so you'll need to quote to handle it. In zsh, you can setopt glob_assign to enable this behavior too, so quoting is the safest answer.

[tgkprog]

Can also Use a file to get the args and in the commandline give path to file