What is the exact meaning of IFS=$'\n'?

Question

If the following example, which sets the IFS environment variable to a line feed character...

IFS=$'\n'

• What does the dollar sign mean exactly?
• What does it do in this specific case?
• Where can I read more on this specific usage (Google doesn't allow special characters in searches and I don't know what to look for otherwise)?

I know what the IFS environment variable is, and what the \n character is (line feed), but why not just use the following form: IFS="\n" (which does not work)?

For example, if I want to loop through every line of a file and want to use a for loop, I could do this:

for line in (< /path/to/file); do
    echo "Line: $line"
done

However, this won't work right unless IFS is set to a line feed character. To get it to work, I'd have to do this:

OLDIFS=$IFS
IFS=$'\n'
for line in (< /path/to/file); do
    echo "Line: $line"
done
IFS=$OLDIFS

Note: I don't need another way for doing the same thing, I know many other already... I'm only curious about that $'\n' and wondered if anyone could give me an explanation on it.

Solution 1

Normally bash doesn't interpret escape sequences in string literals. So if you write \n or "\n" or '\n', that's not a linebreak - it's the letter n (in the first case) or a backslash followed by the letter n (in the other two cases).

$'somestring' is a syntax for string literals with escape sequences. So unlike '\n', $'\n' actually is a linebreak.

Solution 2

Just to give the construct its official name: strings of the form $'...' are called ANSI C-quoted strings.

That is, as in [ANSI] C strings, backlash escape sequences are recognized and expanded to their literal equivalent (see below for the complete list of supported escape sequences).

After this expansion, $'...' strings behave the same way as '...' strings - i.e., they're treated as literals NOT subject to any [further] shell expansions.

For instance, $'\n' expands to a literal newline character - which is something a regular bash string literal (whether '...' or "...") cannot do.^[1]

Another interesting feature is that ANSI C-quoted strings can escape ' (single quotes) as \', which, '...' (regular single-quoted strings) cannot:

echo $'Honey, I\'m home' # OK; this cannot be done with '...'

List of supported escape sequences:

Backslash escape sequences, if present, are decoded as follows:

\a alert (bell)

\b backspace

\e \E an escape character (not ANSI C)

\f form feed

\n newline

\r carriage return

\t horizontal tab

\v vertical tab

\ backslash

\' single quote

\" double quote

\nnn the eight-bit character whose value is the octal value nnn (one to three digits)

\xHH the eight-bit character whose value is the hexadecimal value HH (one or two hex digits)

\uHHHH the Unicode (ISO/IEC 10646) character whose value is the hexadecimal value HHHH (one to four hex digits)

\UHHHHHHHH the Unicode (ISO/IEC 10646) character whose value is the hexadecimal value HHHHHHHH (one to eight hex digits)

\cx a control-x character

The expanded result is single-quoted, as if the dollar sign had not been present.

---

^{[1] You can, however, embed actual newlines in '...' and "..." strings; i.e., you can define strings that span multiple lines.}

Solution 3

From http://www.linuxtopia.org/online_books/bash_guide_for_beginners/sect_03_03.html:

Words in the form "$'STRING'" are treated in a special way. The word expands to a string, with backslash-escaped characters replaced as specified by the ANSI-C standard. Backslash escape sequences can be found in the Bash documentation.found

I guess it's forcing the script to escape the line feed to the proper ANSI-C standard.

Solution 4

Re recovering the default IFS- this OLDIFS=$IFS is not necessary. Run new IFS in subshell to avoid overriding the default IFS:

ar=(123 321); ( IFS=$'\n'; echo ${ar[*]} )

Besides I don't really believe you recover the old IFS fully. You should double quote it to avoid line breaking such as OLDIFS="$IFS".

Solution 5

ANSI C-quoted strings is a key point. Thanks to @mklement0 .

You can test ANSI C-quoted strings with command od.

echo -n $'\n' | od -c
echo -n '\n' | od -c
echo -n $"\n" | od -c
echo -n "\n" | od -c

Outputs:

0000000  \n  
0000001

0000000   \   n   
0000002

0000000   \   n   
0000002

0000000   \   n   
0000002

You can know the meaning clearly by the outputs.

View more solutions

Comments

[sepp2k]

This has nothing to do with variables. $'FOO' (unlike $FOO which this was question was not about) is a string literal. If you execute echo $'VAR', you'll see that it prints the string VAR, not test.

[Roman Cheplyaka]

Not exactly so -- \n is just an (escaped) letter n. You are right that '\n' and "\n" are backlash followed by n.

[Richard Hansen]

Note that $'\n' is bash specific -- it won't work in a POSIX shell (/bin/sh). To get the same effect in a POSIX-compliant manner, you can type IFS=', then hit return to type an actual newline character, then type the closing '

[Vineet]

IFS=$(echo -e '\n') should also do it in a POSIX-compatible way.

[Digital Trauma]

@Vineet - it gave me pause to dispute an upvoted comment. While this is Posix-correct, it doesn't work - The command substitution operators in bash remove all trailing newline characters. See this for more detail.

[jeberle]

this is a really useful technique. i just used it for a cleaner shell join op: args=$(IFS='&'; echo "$*"). restoring IFS to $' \t\n' in a Bourne shell friendly manner is no mean feat.

[nvd]

Make sure that the script starts with "!/bin/bash" and not "!/bin/sh". Not relevant but may be a pitfall.

[Ciro Santilli OurBigBook.com]

@DigitalTrauma I think it's not even POSIX: -e is not defined, and \n without -e works as an XSI extension: pubs.opengroup.org/onlinepubs/9699919799/utilities/… . printf '\n' rocks ;)

[mklement0]

Re Besides I don't really believe you recover the old IFS fully: word splitting is not performed on the RHS of variable assignments (but quote removal is), so OLDIFS=$IFS and OLDIFS="$IFS" behave the same way.

[Braden Best]

A rather unexpected side effect of using IFS='\n' (without the dollar sign) is that your user name gets truncated if it has an n in it (This bug is how I got to this thread). echo $USER yields brade instead of braden in my case.

[anishjp]

@Richard Hansen: It is not working in my bash. Has this changed in newer versions of bash? I am using 4.2.46(2)-release

[Andrey Kaipov]

Here is an actual POSIX solution: nl="$(printf '\nx')"; nl="${nl%x}"; IFS="$nl". By printing an extra character with our newline, the newline isn't lost under command substitution. Then we just use parameter expansion to remove the extra character.