Write each row of pandas dataframe into a new text file - pythonic way

Question

I was trying to google up if there's a way to parse a pandas dataframe row wise and write the contents of each row into a new text file. My dataframe consists of a single column called Reviews.

I'm looking to do some sentiment analysis on movie reviews and that I need each review to be in a separate text file. Can somebody help me here.

Solution 1

I've written something like this and it works. anyways thanks for your inputs guys

for index, row in p.iterrows():
    if i > len(p):
       break
    else:
       f = open(str(i)+'.txt', 'w')
       f.write(row[0])
       f.close()
       i+=1

where p is a dataframe.

Solution 2

It's still inefficient, but since it's required here's one possible solution.

import pandas as pd
from io import StringIO

data="""
column1 column2
c1 c2
c3 c4
c5 c6
"""

df = pd.read_csv(StringIO(data), delimiter='\s+')

i=0
for row in df.values:
    filename = 'testdir/review{}.csv'.format(i)
    row.tofile(filename, sep=",", format="%s")
    i+=1

This will take the values as an array and write the data to a csv file named review0.csv, review1.csv... Another solution is to use pd.to_csv within the loop and specify the chunk

Comments

[Leb]

That's going to be very inefficient, what's the purpose for that?

[Wolf]

Just to perform classification. My requirement is in that way

[R Nar]

make a file name variable that changes every time you write a new line then open that filename with the w parameter

[Wolf]

Can you please suggest the format to write data from dataframe to text file ? @RNar I've been wondering on that for quite a while. Does to_csv work for this ?

[R Nar]

i wouldnt suggest it, no. because you want to write a new file for each row, iterate through the rows then just have something like f = open(filename, 'w') then f.write(row) kind of thing. just make sure to change filename each time.

[Laurie]

For anyone else receiving the Unicode Error: change f = open(str(i)+'.txt', 'w'), to f = open(str(i)+'.txt', 'w', encoding='utf-8')