Newlines in sed on Mac OS X
Solution 1
You can brew install gnu-sed
and replace calls to sed
with gsed
.
To use it as sed
instead of gsed
, brew helpfully prints the following instructions after you install:
GNU "sed" has been installed as "gsed".
If you need to use it as "sed", you can add a "gnubin" directory
to your PATH from your bashrc like:
PATH="/usr/local/opt/gnu-sed/libexec/gnubin:$PATH"
That is, append the following line to your ~/.bashrc or ~/.zshrc:
export PATH="/usr/local/opt/gnu-sed/libexec/gnubin:$PATH"
Solution 2
These would also work:
echo 'foo bar' | sed 's/ /\
/g'
echo 'foo bar' | sed $'s/ /\\\n/g'
lf=$'\n'; echo 'foo bar' | sed "s/ /\\$lf/g"
OS X's sed doesn't interpret \n
in the replace pattern, but you can use a literal linefeed preceded by a line continuation character. The shell replaces $'\n'
with a literal linefeed before the sed command is run.
Solution 3
The workaround you found passes a single argument string to sed -e
.
That argument ends up being a string in the familiar sed s/ / /g
format.
That string is created in two parts, one after the other.
The first part is quoted in '...'
form.
The second part is quoted in $'...'
form.
The 's/ /\'
part gets the single-quotes stripped off, but otherwise passes through to sed just as it looks on the command-line. That is, the backslash isn't eaten by bash, it's passed to sed.
The $'\n/g'
part gets the dollar sign and the single-quotes stripped off, and the \n
gets converted to a newline character.
All together, the argument becomes
s/ /\newline/g
[That was fun. Took a while to unwrap that. +1 for an interesting question.]
Solution 4
The expression $'...'
is a bash
-ism which produces ...
with the standard escape sequences expanded. Th \'
before it just means a backslash followed by the end of the quoted section, the resulting string is s/ /\
. (Yes, you can switch quoting in the middle of a string; it doesn't end the string.)
POSIX standard sed
only accepts \n
as part of a search pattern. OS X uses the FreeBSD sed
, which is strictly POSIX compliant; GNU, as usual, adds extra stuff and then Linux users all think that is some kind of "standard" (maybe I'd be more impressed if either of them had a standards process).
Solution 5
There's a very easy to visually see what's happening. Simply echo the string!
echo 's/$/\'$'\n/g'
results in
s/$/\
/g
which is equivalent to s/$/\
newline/g
If you didn't have the extra \
before the newline
, the shell would interpret the newline
as the end of the command prematurely.
Related videos on Youtube
Comments
-
Ivan Xiao over 1 year
I find that
\n
doesn't work in sed under Mac OS X. Specifically, say I want to break the words separated by a single space into lines:# input foo bar
I use,
echo "foo bar" | sed 's/ /\n/'
But the result is stupid, the
\n
is not escaped!foonbar
After I consulted to google, I found a workaround:
echo 'foo bar' | sed -e 's/ /\'$'\n/g'
After reading the article, I still cannot understand what
\'$'\n/g'
means. Can some one explain it to me, or if there is any other way to do it? Thanks!-
Admin almost 13 yearsthis would probably work too:
echo "foo bar" | tr ' ' '\n'
-
Admin almost 13 yearsThanks for the advice. But currently I just use the above case as an example, I do need to know how to escape a
\n
. -
Admin about 7 yearsPossible duplicate: stackoverflow.com/questions/21621722/…
-
Admin over 3 yearsMaybe you could also use Perl instead of sed which would make it simpler. See my answer here for details. Basically, it would be
echo "foo bar" | perl -pe 's/ /\n/
orperl -pe 's/ +/\n/g
to replace all spaces or groups of spaces by a new line. -
Admin over 3 yearsI just noticed a script running on Big Sur escaped the newline just fine, but the same script on Catalina showed your output... not a newline. I think Apple updated it to address this issue.
-
-
Ivan Xiao almost 13 yearsNow I understand the
$'...'
part. But... what iss/ /\
? What do you mean byswitch quoting
? -
Clément almost 9 yearsUsing the same software on all platform is way easier (at last to me) than dealing with every specificities of the Mac version. Beware, the option is now
--with-default-names
and not--default-names
. However, this option did not worked on my installation, so I had to put aalias gsed=sed
in my~/.profile
to make it work. -
octosquidopus over 8 yearsThis is an ugly workaround. It doesn't explain why
sed
on OS X behaves the way it does and makes the incorrect assumption thatgnu-sed
is more correct. Don't be a GNU-addict and stick to POSIX standards to avoid problems in the long run. -
wolfhammer over 8 yearsWhy does
sed $'s/ /\\\n/g'
work, but notsed $'s/\\\n/ /g'
? -
rascio about 7 yearsIt is how Apple should make its OS works by default. Why use the name of a program and then make it works differently? I waste an hour because of this...
-
wisbucky over 6 years@alec You can't use
sed
even in linux/unix to remove newlines because its parsed/split at each newline. If you run this in linux/unix, it won't do anything either:echo -e 'foo\nbar' | sed 's/\n//'
-
iAdjunct almost 6 years@rascio - because OS X is BSD-like and Linux is Linux-like.
-
Duncan Bayne over 5 years@rascio I think you'll find that GNU sed is the newcomer; BSD sed dates back to 1979 (see en.wikipedia.org/wiki/Version_7_Unix).
-
Robin Keskisarkka about 5 yearsThis is the preferred answer in most cases I think. If you get a compile script in your lap you don't want to spend time modifying it just to make it work with OSX. Instead, simply do
brew install gnu-sed
,alias sed='gsed'
,sh compile.sh
, and remove the alias when you're done (unalias sed
). -
Robin Keskisarkka about 5 yearsCorrection: For the alias to work correctly inside the script, the command should be
source compile.sh
-
k1eran over 4 years
brew install gnu-sed --with-default-names
gives meError: invalid option: --with-default-names
-
grepit over 2 yearsthis should have been the accepted answer