Pandas: How to drop multiple columns with nan as col name?

14,894

Solution 1

In [218]: df = df.loc[:, df.columns.notna()]

In [219]: df
Out[219]:
      x   y
0  this NaN
1  that NaN
2  this NaN
3  that NaN
4  this NaN

Solution 2

You can try

df.columns = df.columns.fillna('to_drop')
df.drop('to_drop', axis = 1, inplace = True)
Share:
14,894

Related videos on Youtube

RDJ
Author by

RDJ

Updated on July 09, 2022

Comments

  • RDJ
    RDJ almost 2 years

    As per the title here's a reproducible example:

    raw_data = {'x': ['this', 'that', 'this', 'that', 'this'], 
                np.nan: [np.nan, np.nan, np.nan, np.nan, np.nan], 
                'y': [np.nan, np.nan, np.nan, np.nan, np.nan],
                np.nan: [np.nan, np.nan, np.nan, np.nan, np.nan]}
    
    df = pd.DataFrame(raw_data, columns = ['x', np.nan, 'y', np.nan])
    df
    
       x     NaN  y    NaN
    0  this  NaN  NaN  NaN
    1  that  NaN  NaN  NaN
    2  this  NaN  NaN  NaN
    3  that  NaN  NaN  NaN
    4  this  NaN  NaN  NaN
    

    Aim is to drop only the columns with nan as the col name (so keep column y). dropna() doesn't work as it conditions on the nan values in the column, not nan as the col name.

    df.drop(np.nan, axis=1, inplace=True) works if there's a single column in the data with nan as the col name, but not with multiple columns with nan as the col name, as in my data.

    So how to drop multiple columns where the col name is nan?