Adding newline characters to unix shell variables

Question

I have a variable in a shell script in which I'd like to format the data. The variable stores new data during every iteration of a loop. Each time the new data is stored, I'd like to insert a new line character. Here is how I'm trying to store the data into the variable.

VARIABLE="$VARIABLE '\n' SomeData"

Unfortunately, the output includes the literal '\n' Any help would be appreciative.

Solution 1

Try $'\n':

VAR=a
VAR="$VAR"$'\n'b
echo "$VAR"

gives me

a
b

Solution 2

A common technique is:

nl='
'
VARIABLE="PreviousData"
VARIABLE="$VARIABLE${nl}SomeData"

echo "$VARIABLE"
PreviousData
SomeData

Also common, to prevent inadvertently having your string start with a newline:

VARIABLE="$VARIABLE${VARIABLE:+$nl}SomeData"

(The expression ${VARIABLE:+$nl} will expand to a newline if and only if VARIABLE is set and non-empty.)

Solution 3

VAR="one"
VAR="$VAR.\n.two"
echo -e $VAR

Output:

one.
.two

Solution 4

Other than $'\n' you can use printf also like this:

VARIABLE="Foo Bar"
VARIABLE=$(printf "${VARIABLE}\nSomeData")
echo "$VARIABLE"

OUTPUT:

Foo Bar
SomeData

Solution 5

It's a lot simpler than you think:

VARIABLE="$VARIABLE
SomeData"

View more solutions

Comments

[William Pursell]

Why the downvote? This is absolutely the canonical way to inject newlines into a variable.

[jordanm]

The reason is is common is likely because people don't know about $'\n'. edit: I had originally downvoted but decided to just stick with a comment instead.

[William Pursell]

@jordanm Or more likely because people are aware that \n only works in a limited number of shells, while this technique will work in all shells.

[Jonathan Leffler]

Recommend enclosing the variables in quotes. While bash may be OK, most shells will not be OK.

[James P.]

Thanks vmpstr. I may be mistaken, but I believe VAR=$VAR$'\n'b must be surrounded by double quotes. Which results in a literal '\n'.

[James P.]

Thanks Alex. I think when you echo here, the output is on multiple lines. But I think the variable is still be stored on one line? I'm not sure.

[James P.]

Doesn't the entire value need to be double quoted in order to append SomeData to VARIABLES Such as VAR="$VARIABLE${nl}SomeData" In this answer, there are two separate variables (VAR and VARIABLE I'm trying to append VARIABLE to VARIABLE

[William Pursell]

@James The double quotes are not strictly necessary, but if VARIABLE contains whitespace or shell special characters it is probably advisable to use them. The VAR in the answer should read VARIABLE; my initial response was sloppy.

[Alex]

You're correct, -e makes echo treat escaped characters as special characters, without -e you would see entire value on one line with "\n" in the middle

[James P.]

Unfortunately I'm storing data in VAR and then sending the contents of VAR to an email utility. So the value can't be echoed, it must be formatted within the var itself. Sorry for the confusion.

[James P.]

Thank you anubhava. But like the answer below, the output of echo is different than how the values are being stored in VARIABLE. The contents of VARIABLE are passed to an email util. I am not echoing the data to command line. Sorry for the confusion.

[James P.]

Thanks William. I've discovered that the output of echo is different than how the data is actually stored in VARIABLE. I need to take the contents of VARIABLE and pass it to an email util. Which means I wont be able to use echo. Any thoughts?

[Gordon Davisson]

You can switch between different kinds of quotes in the same "word": VAR="$VAR"$'\n'"some data" has $VAR in double-quotes, \n in $'' so it'll get interpreted as a newline, and then some data back in double-quotes. As long as there's no space between them, they'll all be concatenated together and the result assigned to VAR.

[vmpstr]

Thanks, I added double quotes around $VAR so that the existing newlines are preserved

[James P.]

The output of echo is different than how the values are being stored in VARIABLE. The contents of VARIABLE are passed to an email util. I am not echoing the data to command line. Sorry for the confusion.

[James P.]

I got it! Turns out I had some spacing issues. Thanks to everyone who cared to comment, especially Gordon Davisson. Here was my issue: Original Code: VAR="$VAR"$'\n'"Audit Source: $B Audit Path: $C" The spaces for formatting was messing things up. New Code:VAR="$VAR"$'\n'"Audit Source:$B"$'\t'"Audit Path:$C" No spaces here! Anyone know why the spacing causes issues? Even though the Original Code has everything surrounded by "?

[patrick]

Side note: all the solutions here (with the exeption of printf) won't work smoothly if your first character pair has a hash sign in it, c.f. this markdown header FOO="# Markdown Title #" would get split into two lines. I'm curious if there's another way.

[ack]

Please note in your answer that this only works with Bash (and maybe some other shells that I don't know of). This solution is not generic, not everyone uses Bash.