Insert linefeed in sed (Mac OS X)
Solution 1
Your sed version apparently does not support \n
in RHS (right-hand side of substitution). You should read THE SED FAQ maintained by Eric Pement to choose one of possible solutions. I suggest trying first inserting literal newline character.
Below is the quote from it.
4.1. How do I insert a newline into the RHS of a substitution?
Several versions of sed permit \n
to be typed directly into the RHS, which is then converted to a newline on output: ssed, gsed302a+, gsed103 (with the -x
switch), sed15+, sedmod, and UnixDOS sed. The easiest solution is to use one of these versions.
For other versions of sed, try one of the following:
(a) If typing the sed script from a Bourne shell, use one backslash \
if the script uses 'single quotes' or two backslashes \\
if the script requires "double quotes". In the example below, note that the leading >
on the 2nd line is generated by the shell to prompt the user for more input. The user types in slash, single-quote, and then ENTER to terminate the command:
[sh-prompt]$ echo twolines | sed 's/two/& new\
>/'
two new
lines
[bash-prompt]$
(b) Use a script file with one backslash \
in the script, immediately followed by a newline. This will embed a newline into the "replace" portion. Example:
sed -f newline.sed files
# newline.sed
s/twolines/two new\
lines/g
Some versions of sed may not need the trailing backslash. If so, remove it.
(c) Insert an unused character and pipe the output through tr:
echo twolines | sed 's/two/& new=/' | tr "=" "\n" # produces
two new
lines
(d) Use the G
command:
G appends a newline, plus the contents of the hold space to the end of the pattern space. If the hold space is empty, a newline is appended anyway. The newline is stored in the pattern space as \n
where it can be addressed by grouping \(...\)
and moved in the RHS. Thus, to change the "twolines" example used earlier, the following script will work:
sed '/twolines/{G;s/\(two\)\(lines\)\(\n\)/\1\3\2/;}'
(e) Inserting full lines, not breaking lines up:
If one is not changing lines but only inserting complete lines before or after a pattern, the procedure is much easier. Use the i
(insert) or a
(append) command, making the alterations by an external script. To insert This line is new
BEFORE each line matching a regex:
/RE/i This line is new # HHsed, sedmod, gsed 3.02a
/RE/{x;s/$/This line is new/;G;} # other seds
The two examples above are intended as "one-line" commands entered from the console. If using a sed script, i\
immediately followed by a literal newline will work on all versions of sed. Furthermore, the command s/$/This line is new/
will only work if the hold space is already empty (which it is by default).
To append This line is new
AFTER each line matching a regex:
/RE/a This line is new # HHsed, sedmod, gsed 3.02a
/RE/{G;s/$/This line is new/;} # other seds
To append 2 blank lines after each line matching a regex:
/RE/{G;G;} # assumes the hold space is empty
To replace each line matching a regex with 5 blank lines:
/RE/{s/.*//;G;G;G;G;} # assumes the hold space is empty
(f) Use the y///
command if possible:
On some Unix versions of sed (not GNU sed!), though the s///
command won't accept \n
in the RHS, the y///
command does. If your Unix sed supports it, a newline after aaa
can be inserted this way (which is not portable to GNU sed or other seds):
s/aaa/&~/; y/~/\n/; # assuming no other '~' is on the line!
Solution 2
Here's a single-line solution that works with any POSIX-compatible sed
(including the FreeBSD version on macOS), assuming your shell is bash
or ksh
or zsh
:
sed 's/\(1234\)/\'$'\n''\1/g' <<<'test1234foo123bar1234'
Note that you could use a single ANSI C-quoted string as the entire sed
script, sed $'...' <<<
, but that would necessitate \
-escaping all \
instances (doubling them), which is quite cumbersome and hinders readability, as evidenced by @tovk's answer).
-
$'\n'
represents a newline and is an instance of ANSI C quoting, which allows you to create strings with control-character escape sequences. - The above splices the ANSI C-quoted string into the
sed
script as follows:- The script is simply broken into 2 single-quoted strings, with the ANSI C-quoted string stuck between the two halves:
-
's/\(1234\)/\'
is the 1st half - note that it ends in\
, so as to escape the newline that will be inserted as the next char. (this escaping is necessary to mark the newline as part of the replacement string rather than being interpreted as the end of the command). -
$'\n'
is the ANSI C-quoted representation of a newline character, which the shell expands to an actual newline before passing the script tosed
. -
'\1/g'
is the 2nd half.
Note that this solution works analogously for other control characters, such as $'\t'
to represent a tab character.
Background info:
- The POSIX
sed
specification: http://man.cx/sed-
BSD
sed
(also used on macOS) stays close to this spec, while GNUsed
offers many extensions.
-
BSD
- A summary of the differences between GNU
sed
and BSDsed
can be found at https://stackoverflow.com/a/24276470/45375
Solution 3
The solaris version of sed
I could convince to work this way (in bash
):
echo test1234foo123bar1234 | sed 's/\(1234\)/\
\1/g'
(you have to put the line break directly after the backslash).
In csh
I had to put one more backslash:
echo test1234foo123bar1234 | sed 's/\(1234\)/\\
\1/g'
The Gnu version of sed
simply worked using \n
:
echo test1234foo123bar1234 | sed 's/\(1234\)/\n\1/g'
Solution 4
Perl provides a richer "extended" regex syntax which is useful here:
perl -p -e 's/(?=1234)/\n/g'
means "substitute a newline for the zero-width match following the pattern 1234". This avoids having to capture and repeat part the expression with backreferences.
Solution 5
Get a GNU sed.
$ brew install gnu-sed
Then your command will work as expected:
$ gsed "s/\(1234\)/\n\1/g" input.txt
test
1234foo123bar
1234
nb: you may get GNU sed thanks to mac ports too.
Related videos on Youtube
Comments
-
Tyilo over 3 years
How do I insert a newline in the replacement part of sed?
This code isn't working:
sed "s/\(1234\)/\n\1/g" input.txt > output.txt
where input.txt is:
test1234foo123bar1234
and output.txt should be:
test 1234foo123bar 1234
but insted I get this:
testn1234foo123barn1234
NOTE:
This question is specifically about the Mac OS X version of "sed", and the community has noted that it behaves differently than, say, Linux versions.
-
PhilR almost 13 yearsDid you typo the example? I can't reproduce this "testn..." output; it looks like the \n is wrong in the right-hand-side of the s///
-
rid almost 13 yearsI can. It shows "testn" for me as well. Probably each system's sed interprets this in its own way.
-
PhilR almost 13 years@rdineiu, I think the problem is the way the backslash is passed with the quotes. I assume Tylio is using bash (from the question-tag); what do you get if you type
set -x; echo "n \n \\n"
? Specifically, are the backslashes preserved? -
rid almost 13 yearsI use bash as well. I get
n \n \n
. I tried replacing that\n
with everything I could think of, including\\n
,\<CR><CR>
,\<^V><CR>
,\\<^V><CR>
. The only thing that worked was an explicit\n\n\r
from anecho
. -
Tyilo almost 13 years@Phil outputs: + echo 'n \n \n' n \n \n
-
PhilR almost 13 yearsOK, elsewhere you've said that this is sed on a Mac. I'd go with @prezemoc's FAQ entry or the perl snippet I gave.
-
-
Tyilo almost 13 yearsSorry missed the g part, but the problem is displaying the linefeed
-
Tyilo almost 13 yearsAnd output is still: testn1234foo123barn1234
-
Fredrik Pihl almost 13 years@Tylio Works for me, as can be seen in my answer. I'm using GNU sed version 4.2.1 on Ubuntu 10.04. What system are you on?
-
rid almost 13 years@Tyilo I'm also on a Mac. Must be a Mac specific thing then. See my answer or Phil's, if you have no problem using perl instead of sed (perl's regexps are way more advanced and work the same way on all systems).
-
Tyilo almost 13 yearsSorry, it works in terminal output, but when saving it to a file
-
rid almost 13 yearsUse sed "s/(1234)/\`echo -e '\n\r'`\1/g" input.txt > output.txt to save to a file. It will work the same way.
-
Tyilo almost 13 yearsBut i don't want the carriage return, only line feed
-
PhilR almost 13 yearsIntroducing CR (\r) characters isn't going to help. Superficially it might force a break on the terminal, but anything else which handles the data isn't going to expect, or deal with, LFCR line endings. In fact, nothing uses LFCR line endings other than the Acorn BBC, according to Wikipedia :)
-
rid almost 13 years@Phil, indeed, you're right. @Tyilo, Check bmk's answer. The part with \\<ENTER>\1/g should work correctly on the Mac. If you do want portability though, the perl solution is the best.
-
andrewdski almost 13 yearsThis is the way I have always done it. I just tested bash on Mac OS X 10.6, and the first example works.
-
rid almost 13 yearsThe first example, with a single \, doesn't work on my Mac OS X 10.6. The second one, with \\, does.
-
przemoc almost 13 years@Tyilo While I am happy that I was helpful, it's good to avoid executing other tools if it can be done w/o them, thus I suggest you to try all other ways too and then choose the best one suiting you.
-
Tyilo almost 13 yearsOops, the command with double backslashes is working for me, thank you
-
przemoc almost 13 years@Tyilo This is (a), so sorry, but I am not sure whether you tried them all, because you're contradicting yourself. Have you tried (f) for example?
-
noogrub over 11 yearsFirst example works great on Mac OSX Lion. Thanks for a SIMPLE answer.
-
starlocke over 10 yearsThis helped a lot in a Mac OS X server's shell. Instead of sed, perl becomes the "go to" tool for doing regex replace involving "insert a line return" work~
-
mklement0 almost 10 yearsAs of at least GNU sed version 4.2.1, (f) works there, too. In fact, the
y
function with\n
handling is part of POSIX: man.cx/sed -
LOAS over 9 yearsIf you have your sed expression in double quotes, then you also need double escaping \\ in bash (in osx)
-
mklement0 about 9 years
-
mko over 8 yearsThis is brilliant, and it also work with sh that comes with Freebsd
-
Ian Will over 7 yearsTo make this work with double-quotes on OSX I did this:
sed "s/\(1234\)/\\"$'\n'"\1/g" <<<'test1234foo123bar1234'
Effectively means double-escaping the"
. -
mochadwi over 4 yearsWell, this should be the accepted answer for macOS users. With
gnu-sed
I'm able to run myscript_file
as well.