Does Python csv writer always use DOS end-of-line characters?

Question

I realize that the csv library in Python always generates DOS end-of-line characters. Even if I use the 'wb' mode, even if I use Linux.

import csv

f = open('output.txt', 'wb');
writer = csv.writer(f)
writer.writerow([2,3,4]);
f.close()

The above code always uses '\r\n' as the end of line separator. How can I make it use use '\n' only?

Solution 1

You can give your writer instance a custom lineterminator argument in the constructor:

writer = csv.writer(f, lineterminator="\n")

Solution 2

As Niklas answered, the lineterminator argument lets you choose your line endings. Rather than hard coding it to '\n', make it platform independent by using your platform's line separator: os.linesep. Also, make sure to specify newline='' for Python 3 (see this comment).

import csv
import os

with open('output.csv', 'w', newline='') as f:
    writer = csv.writer(f, lineterminator=os.linesep)
    writer.writerow([2, 3, 4])

Old solution (Python 2 only)

In Python 2, use 'wb' mode (see the docs).

import csv
import os

with open('output.csv', 'wb') as f:
    writer = csv.writer(f, lineterminator=os.linesep)
    writer.writerow([2, 3, 4])

For others who find this post, don't miss the 'wb' if you're still using Python 2. (In Python 3, this problem is handled by Python). You won't notice a problem if you're missing it on some platforms like GNU/Linux, but it is important to open the file in binary mode on platforms where that matters, like Windows. Otherwise, the csv file can end up with line endings like \r\r\n. If you use the 'wb' and os.linesep, your line endings should be correct on all platforms.

Comments

[user1479571]

can lineterminator be given as combination of two characters ?

[Ciprian Tomoiagă]

so even if you make the writer "platform independent", your script is still platform dependent because you need to open it in binary, but only where necessary ? If I try wb on linux I get an error for writer.writeheader()

[Don Kirkby]

I just tried it on Ubuntu 16.04 using both writer and DictWriter with wb, @CiprianTomoiaga, and it works fine for me. I suggest you post a new question with a code sample and the error message, then add a comment here with a link to your question.

[maxschlepzig]

@CiprianTomoiaga This doesn't work with Python 3. The CSV module assumes that it can write normal strings and this isn't the case anymore if you open the file with 'wb' (you would get a TypError). Thus, you have to open the file like this: f = open('output.csv', 'w', newline='') when you use Python 3. See also the Python 3 CSV documentation.

[LarsH]

@user1479571 If you follow the link to the documentation for lineterminator, it says that it is a string, and the default value is '\r\n'. So yes.

[LarsH]

Consider too that the main purpose of CSV in general is as an interchange format that works between dissimilar systems. So while using lineterminator=os.linesep will produce CSV that's suited to the system on which this script runs, that also means the CSV produced is different depending on where the script runs; neither of which is necessarily right for the system on which the CSV is consumed. So "platform independent" for the consumer might mean sticking to a single standard rather than varying with the OS of the producer.