Get feature importance from GridSearchCV

24,667

Solution 1

Got it. It goes something like this :

optimized_GBM.best_estimator_.feature_importance()

if you happen ran this through a Pipeline and receive object has no attribute 'feature_importance' try optimized_GBM.best_estimator_.named_steps["step_name"].feature_importances_

where step_name is the corresponding name in your pipeline

Solution 2

This one works

optimized_GBM.best_estimator_.feature_importances_

Solution 3

That depends on what model you have selected. If you choose a SVM you wont be having feature importance parameter, but in decision trees you will get it

Share:
24,667
Nick M
Author by

Nick M

If I do this I get a badge here. If I do this I get a badge here. If I do this I get a badge here. If I do this I get a badge here. If I do this I get a badge here. If I do this I get a badge here. If I do this I get a badge here.

Updated on May 27, 2020

Comments

  • Nick M
    Nick M over 3 years

    Is there a way to get feature importance from a sklearn's GridSearchCV?

    For example :

    from sklearn.model_selection import GridSearchCV
    print("starting grid search ......")
    optimized_GBM = GridSearchCV(LGBMRegressor(),
                                 params,
                                 cv=3,
                                 n_jobs=-1)
    # 
    optimized_GBM.fit(tr, yvar)
    preds2 = optimized_GBM.predict(te)
    

    Is there a way I can access feature importance ?

    Maybe something like

    optimized_GBM.feature_importances_
    
  • limitless
    limitless over 5 years
    Did you maybe got object has no attribute 'feature_importance' error?
  • Nick M
    Nick M over 5 years
    No,I did not get any error. This worked for me. I was using python 3.6. However, this was in Jan so the function call might have changed as suggested by other answer.
  • 00schneider
    00schneider about 4 years
    In addition: If you are using a pipeline, .i.e. your estimator is a pipeline object, you have to add the pipeline step name: optimized_GBM.best_estimator_.named_steps["step_name"].featu‌​re_importances_
  • Jeremy K.
    Jeremy K. almost 3 years
    @aptha-gowda is there a way to also extract the feature names? i.e. the name of the variable?
  • BND
    BND over 2 years
    @00schneider if I do PCA then fit a model in my pipeline, how do I recover the importance of the original variables in the model.