Pandas Plot with Index causes 'KeyError [] not in index'

Question

I am very new to the Pandas concept in Python. Usually plots are not a problem. However, I am now confronted with a dataframe that contains an index. Somehow nothing is working anymore.

What I want to achieve: Create a subplot for every column [Plant1,Plant2,Plant3] against one specific colum [Trafo1].

Here is my code:

import numpy as np
import datetime
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
import pandas as pd
import os

# Create the sample data

plant1 = {'Date' : pd.date_range('1/1/2011', periods=10, freq='D'),
     'Plant' : pd.Series(["Plant1"]*10),
     'Output' : pd.Series(abs(np.random.randn(10)))}

plant2 = {'Date' : pd.date_range('1/3/2011', periods=10, freq='D'),
     'Plant' : pd.Series(["Plant2"]*10),
     'Output' : pd.Series(abs(np.random.randn(10)))}

plant3 = {'Date' : pd.date_range('1/5/2011', periods=10, freq='D'),
     'Plant' : pd.Series(["Plant3"]*10),
     'Output' : pd.Series(abs(np.random.randn(10)))}     

trafo1 = {'Date' : pd.date_range('1/5/2011', periods=10, freq='D'),
     'Plant' : pd.Series(["Trafo1"]*10),
     'Output' : pd.Series(abs(np.random.randn(10)))}          

trafo2 = {'Date' : pd.date_range('1/5/2011', periods=10, freq='D'),
     'Plant' : pd.Series(["Trafo2"]*10),
     'Output' : pd.Series(abs(np.random.randn(10)))}          



df_plant_1 = pd.DataFrame(plant1)
df_plant_2 = pd.DataFrame(plant2)
df_plant_3 = pd.DataFrame(plant3)
df_trafo_1 = pd.DataFrame(trafo1)
df_trafo_2 = pd.DataFrame(trafo2)


sample = pd.concat([df_plant_1,df_plant_2,df_plant_3,df_trafo_1,df_trafo_2])
test = pd.pivot_table(sample, index='Date', columns='Plant', values='Output')
test = test.fillna(method='pad')                            
test = test.fillna(method='bfill')     



# Draw the plots

matplotlib.style.use('ggplot')

cols = len(test.columns) - 1

fig, axes = plt.subplots(nrows=cols/2, ncols=2, figsize=(12, 4))
for column in test.iloc[:,:-1]:
    test.plot(x=test[column], y=test['Trafo1'], title=column)
    plt.gca().set_aspect('equal', adjustable='box')
    plt.show()

Resulting in the following error output:

runfile('C:/..../untitled12.py', wdir='C:/...')


Traceback (most recent call last):

  File "<ipython-input-206-1acb55933d7f>", line 1, in <module>
    runfile('C:/Users/bjl/untitled12.py', wdir='C:/Users/bjl')

  File "C:\Anaconda2\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 699, in runfile
    execfile(filename, namespace)

  File "C:\Anaconda2\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 74, in execfile
    exec(compile(scripttext, filename, 'exec'), glob, loc)

  File "C:/Users/bjl/untitled12.py", line 52, in <module>
    test.plot(x=test[column], y=test['Trafo1'], title=column)

  File "C:\Anaconda2\lib\site-packages\pandas\tools\plotting.py", line 3671, in __call__
    sort_columns=sort_columns, **kwds)

  File "C:\Anaconda2\lib\site-packages\pandas\tools\plotting.py", line 2556, in plot_frame
    **kwds)

  File "C:\Anaconda2\lib\site-packages\pandas\tools\plotting.py", line 2370, in _plot
    series = data[y].copy()  # Don't modify

  File "C:\Anaconda2\lib\site-packages\pandas\core\frame.py", line 1963, in __getitem__
    return self._getitem_array(key)

  File "C:\Anaconda2\lib\site-packages\pandas\core\frame.py", line 2007, in _getitem_array
    indexer = self.ix._convert_to_indexer(key, axis=1)

  File "C:\Anaconda2\lib\site-packages\pandas\core\indexing.py", line 1150, in _convert_to_indexer
    raise KeyError('%s not in index' % objarr[mask])

KeyError: '[ 1.20311253  1.20311253  1.20311253  1.20311253  1.20311253  0.32765014\n  1.65686117  2.58118029  0.58903059  0.13907876  0.59270297  0.27072611\n  0.50167366  1.0310578 ] not in index'

I don't understand what the problem with the Index appears to be. I was not able to find any help online as all examples work without index.

I really appreciate your help. The error is a cryptic to newcomers.

Solution

According to the docs you are supposed to give the column names, not the columns themselves when plotting this way. So replacing:

test.plot(x=test[column], y=test['Trafo1'], title=column)

with

test.plot(x=column, y='Trafo1', title=column)

should solve this error.

EDIT: As for the subplotting, to get it in the right subplot you have to specify the axis you want your plot to end up. You could so in the following manner:

for i, column in enumerate(test.iloc[:, :-2]):
    j = i // 2
    k = i % 2
    test.plot(x=column, y='Trafo1', title=column, ax=axes[j][k])

Now do the operation you wanted on all axes (Although it really screws stuff up)

for ax in axes.reshape(4):
    ax.set_aspect('equal', adjustable='box')

Show plots :)

plt.show()

Comments

[lammy]

I should probably book a vacation :(

[PdevG]

@lammy happens to the best of us ;). I fixed the subplotting aswell in my next edit, but I haven't figured out yet how to do the set_aspect thingy on the right axis. Shouldn't be that hard from here on though!

[lammy]

If you could do this I will book your spot as well :) I am currently trying to create subplots with an iteration over the dataframe as described. No success yet. One of those days... ;)

[PdevG]

All done :) The set_aspect thingy is really ugly though, so I'm assuming this was not exactly what you wanted. Where are we going for our vacation?