pandas: subtracting two columns and saving result as an absolute

Question

I have the code where I have a csv file opened in pandas and a new one I'm creating. There's a row I need to create "two last lines commented out" of an absolute value of subtracting two rows. I've tried a number of ideas in my head all bring an error.

import pandas as pd
import numpy as np

df = pd.read_csv(filename_read)
ids = df['id']

oosDF = pd.DataFrame()
oosDF['id'] = ids
oosDF['pred'] = pred
oosDF['y'] = df['target']
#oosDF['diff'] = oosdF['pred'] - oosDF['y']
#oosDF['diff'] = oosDF.abs()

Solution 1

I think you need for new DataFrame by subset (columns names in double []) and then get abs value of difference of columns:

oosDF = df[['id','pred', 'target']].replace(columns={'target':'y'})
oosDF['diff'] = (oosDF['pred'] - oosDF['y']).abs()

Solution 2

In your first commented line, you have oosdF instead of oosDF.

In your second commented line, you're setting the column to be abs() applied to the whole dataframe. That should be oosDF['diff'].abs()

Hope this helps!

Comments

[Sam B.]

pred is made from an ML prediction so is not previously in df but I might go with adding it to df first. The error i get is NameError: name 'oosdF' is not defined

[Sam B.]

oosDF['diff'] = (oosdF['pred'] - oosDF['y']).abs()

[jezrael]

It is really weird, if change oosDF to df1 it should working.

[Sam B.]

looks like i have a typo let me try that first