Remove all quotes within values in Pandas

Question

I want to remove all double quotes within all columns and all values in a dataframe. So if I have a value such as

potatoes are "great"

I want to return

potatoes are great

DataFrame.replace() lets me do this if I know the entire value I'm changing, but is there a way to remove individual characters?

Solution 1

You can do this on each Series/column using str.replace:

In [11]: s = pd.Series(['potatoes are "great"', 'they are'])

In [12]: s
Out[12]: 
0    potatoes are "great"
1                they are
dtype: object

In [13]: s.str.replace('"', '')
Out[13]: 
0    potatoes are great
1              they are
dtype: object

I would be wary of doing this across the entire DataFrame, because it will also change columns of non-strings to strings, however you could iterate over each column:

for i, col in enumerate(df.columns):
    df.iloc[:, i] = df.iloc[:, i].str.replace('"', '')

If you were sure every item was a string, you could use applymap:

df.applymap(lambda x: x.replace('"', ''))

Solution 2

use DataFrame.apply() and Series.str.replace():

import numpy as np
import pandas as pd
import random

a = np.array(["".join(random.sample('abcde"', 3)) for i in range(100)]).reshape(10, 10)
df = pd.DataFrame(a)
df.apply(lambda s:s.str.replace('"', ""))

If just string columns:

df.ix[:,df.dtypes==object].apply(lambda s:s.str.replace('"', ""))

Comments

[Charles]

Why? How? Code dumps are not answers. Please edit your answer to explain what this code does and how it solves the problem.

[Noah]

might better link to pandas.pydata.org/pandas-docs/stable/… (with wonderful 0.13 additions!)

[Andy Hayden]

@Noah and (one) more in 0.13.1 - get_dummies :)

[NumenorForLife]

is there an easy way to make this inplace?

[mdmjsh]

you can get still use applymap if you put a condition for handling the non string values as part of the lambda: df.applymap(lambda x: x.replace('"', '') if (isinstance(x, str)) else x)