Delete specific columns from csv file in python3

Question

Need to delete some specific column and rows(by index) of multiple csv files, without creating new files.

For the code below, it is giving output with new blank rows after each row.

import csv

with open('file.csv') as fd:
   reader = csv.reader(fd)
   valid_rows = [row for idx, row in enumerate(reader) if idx != 0]

with open('file.csv', 'w') as out:
   csv.writer(out).writerows(valid_rows)

What is the simpler way to do this(might be by other python libraries)?

Solution

Since you wish not to generate any new csv files and would like the data to perform operations, I would suggest you to make use of Pandas Framework. Make use of drop function in this framework.

Consider the following example:

Sample.csv:

col1,col2,col3,col4
1,2,3,4
5,6,7,8
9,10,11,12
13,14,15,16
17,18,19,20

Code:

import pandas as pd
df = pd.read_csv('./Sample.csv')

To delete columns:

df.drop('col3', axis = 1, inplace = True)

df contents:

   col1     col2    col4
0   1   2   4
1   5   6   8
2   9   10  12
3   13  14  16
4   17  18  20

To delete rows:

df.drop(df.index[[1,4]], inplace = True)

df contents:

   col1     col2    col4
0   1   2   4
2   9   10  12
3   13  14  16

Finally to save the edited csv file:

df.to_csv('new_sample.csv', index = False)

Comments

[Mahesh Karia]

can you also provide sample csv?

[wp78de]

Possible duplicate of Deleting columns in a CSV with python

[Shaila Zaman]

sample csv

[Shaila Zaman]

@wp78de Need to delete column/row from source file, without creating new file.

[wp78de]

But you write to file - it does not matter if it is the same or a new file.

[Shaila Zaman]

Hi, this solution is for reading data only. I need to make the changes(deleting rows& columns) on the csv files also.

[Prasad]

@ShailaZaman, make use of inplace option for that. I have edited the code. Check it out

[Prasad]

Please can you elaborate where df is becoming None? The code I have given is a working example. (Check if you are not deleting all columns or rows by mistake)

[Shaila Zaman]

That was a mistake by me. It is working. Thank you so much. Could you please edit your writing to csv line to the following line? df.to_csv('sample.csv', index=False) It was saving the csv with index of df.