Correct way to set value on a slice in pandas

Question

I have a pandas dataframe: data. it has columns ["name", 'A', 'B']

What I want to do (and works) is:

d2 = data[data['name'] == 'fred'] #This gives me multiple rows
d2['A'] = 0

This will set the column A on the fred rows to 0. I've also done:

indexes = d2.index
data['A'][indexes] = 0

However, both give me the same warning:

/Users/brianp/work/cyan/venv/lib/python2.7/site-packages/pandas/core/indexing.py:128: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy

How does pandas WANT me to do this?

Solution

This is a very common warning from pandas. It means you are writing in a copy slice, not the original data so it might not apply to the original columns due to confusing chained assignment. Please read this post. It has detailed discussion on this SettingWithCopyWarning. In your case I think you can try

data.loc[data['name'] == 'fred', 'A'] = 0

Comments

[tnknepp]

I was about to post the same thing. A logical "one-liner" is better than unnecessary lines.

[Andreas Hsieh]

You're right. Thanks for reminding me.

[user48956]

Isn't there a straight forward way to return a new df with the columns edited, and leave the original unchanged?

[Calvin Ku]

A lot of people say this is the correct way, and this is the way I go with as well. However, sometimes I get the warning anyway saying I'm setting values on a copy and advices me to use .loc when I'm already using. Anyone experienced the same thing?

[Bryan Goggin]

@CalvinKu, yes! I am getting the same warning when doing what it is asking me to do! IMO, this is ambiguous behavior and should go down as a bug, but the Pandas people are tired of hearing about it, so I have little confidence it will be addressed... Such a shame... especially coming from R.

[Calvin Ku]

It's interesting that sometimes I get this and it won't go away no matter how I refactor it. But then when I run the same code again some time later the warning is gone. I'm guessing the implementation of this part of pandas isn't very robust so you see false positives like this once in a while. But what bugs me is it seems like it doesn't happen to some people so they are convinced that it's your code that is wrong...haha

[branwen85]

@CalvinKu, yep, I get the same warning using this solution.

[JakeCowton]

@user48956 (very late to the party, but) you can do data[:].loc[data['name'] == 'fred', 'A'] = 0 to get a copy.

[Viktoriya Malyasova]

@CalvinKu, this happens when the dataframe you're assigning to is a view of another dataframe. E.g consider the code: {a = pd.DataFrame({'x':[1],'y':[1]}); b = a[['x']]; b.loc[:,'x'] = 0 }. Here you will get a settingwithcopy warning to inform you that you have changed the values of b, but not a.