Find and remove DOS line endings on Ubuntu

Question

I have found that many of my files have DOS line endings. In VI they look like this: "^M". I don't want to modify files that don't have these DOS line endings. How do I do this using a bash script? Thanks!

EV

Solution 1

grep -URl ^M . | xargs fromdos

grep gets you a list of all files under the current directory that have DOS line endings.

-U makes grep consider line endings instead of stripping them away by default

-R makes it recursive

-l makes it list only the filenames and not the matching lines

then you're piping that list into the converter command (which is fromdos on ubuntu, dos2unix where i come from).

NOTE: don't actually type ^M. instead, you'll need to press <Ctrl-V> then <Ctrl-M> to insert the ^M character and make grep understand what you're going for. or, you could type in $'\r' in place of ^M (but i think that may only work for bash...).

Solution 2

One way using GNU coreutils:

< file.txt tr -d '\r'

Solution 3

On ubuntu, you use the fromdos utility

fromdos test.txt

The above example would take a MS-DOS or Microsoft Windows file or other file with different line separators and format the file with new line separators to be read in Linux and Unix.

Solution 4

Many options are there..you can try with any of these.. http://www.theunixschool.com/2011/03/different-ways-to-delete-m-character-in.html

Solution 5

cat origin_file.txt | sed "s/^M//" > dest_file.txt

You have to do the same thing mentioned above, ctl-V then ctl-M to get that character. This is preferable for me because it is portable across many platforms and keeps it simple within bash.

on ubuntu I also find this works:

cat origin_file.txt | sed "s/\r//" > dest_file.txt

View more solutions

Comments

[Dalinaum]

grep -URl ^M . | xargs fromdos ?

[Apalala]

The Ubuntu package, for installation, is called "tofrodos".

[bdesham]

On tcsh, and probably on csh too, you can get the same effect with grep -URl "\r" . | xargs fromdos.

[nullrevolution]

if you need it to work for files with spaces in their names, try grep -URl ^M . | xargs -I{} dos2unix "{}" instead.