Removing Carriage return on Mac OS X using sed

Question

On Linux for removing carriage return we can execute:

sed -i 's/\r//g' <file>

But the same will not work on Mac OS X. Need to prepend $ like:

sed -i $'s/\r//' <file>

And "g" is also not needed.

Why is this so?

Solution 1

It is because sed available on OSX doesn't recognize \r as a special character unlike the sed on Linux does.

You can either use it the way you're using:

sed -i.bak $'s/\r//' file

OR this:

sed -i.bak "s/$(printf '\r')//" file

OR else you can use tr on OSX:

tr -d '\r' < file

Solution 2

Another portable and flexible solution is:

sed -i.bak $'s/\x0D//' file

Because the return character has the ASCII code 0D. \x substitutions work for all POSIX releases of sed, and you can match any other troublesome character by looking up the ASCII code. To find other valid substitutions, execute man re_format or view an ASCII Table.

The final /g is needed for Linux because the carriage return (\r) does not end the line. Many Windows "plain text" editors (e.g. Notepad) end each line with both carriage return and new line (\r\n), but Macs with OS 9 (ca. 2001) or earlier ended each line of a "plain text" file with a single \r. If you're cleaning up a Windows file, the /g is not needed on any *X system. If you're on macOS, the /g isn't needed, either, because macOS recognizes the single \r as a line ending.

(A Linux system reading an old Mac file will think that all the text is on one very long line and only convert the first \r. If you're on a Linux system and need to preserve the line breaks from an old Mac file,

sed -i.bak $'s/\x0D/\x0A/g' file

converts each \r to \n.)

Comments

[NeronLeVelu]

(supposition, i have no Mac) shell interpretation/file management. Sed work line by line and normaly the line is ended by CR, not taken by sed. Linux and mac version does not consider the same way the end of the line. g is needed if more than 1 substitution but this is always the last char by default.

[Gaurav Fotedar]

But replacing normal text works just fine without any need for "$"..... so if \r is not considered as special, even then $ should not be required??

[anubhava]

That is true only \n and \r need prefix of $

[Avram]

would someone explain what the $ prefix does here? I understand $ in most unix shells to expand a shell variable. I don't understand why it has the effect of getting a \r to be processed as a carriage return by sed.

[anubhava]

That is bash syntax to deal with backslash-escaped characters. Check man bash and search for \$'string'