How do you resolve 'hidden imports not found!' warnings in pyinstaller for scipy?

14,988

You need to go into the hook-scipy.py (or create one) and have it look like this:

from PyInstaller.utils.hooks import collect_submodules
from PyInstaller.utils.hooks import collect_data_files
hiddenimports = collect_submodules('scipy')

datas = collect_data_files('scipy')

then go into the hook-sklearn.metrics.cluster.py file and modify it to look like this:

from PyInstaller.utils.hooks import collect_data_files

hiddenimports = ['sklearn.utils.sparsetools._graph_validation',
                 'sklearn.utils.sparsetools._graph_tools',
                 'sklearn.utils.lgamma',
                 'sklearn.utils.weight_vector']

datas = collect_data_files('sklearn')

I do not know if this part is necessary but I also created a hook-sklearn.py file that looks like this:

from PyInstaller.utils.hooks import collect_submodules
hiddenimports = collect_submodules('sklearn')

In the cmd I used pyinstaller test.py -F to create one file.

Then it should work:

enter image description here

Share:
14,988

Related videos on Youtube

Luke
Author by

Luke

Ex-.NET developer who converted to The Python. Currently work with energy analytics and lots of messy building sensor data :)

Updated on June 04, 2022

Comments

  • Luke
    Luke over 1 year

    I'm working on using pyinstaller to create an .exe for a python program that uses pandas and sklearn. The pyinstaller process completes and produces the dist folder with the executable as expected. However, when I run the .exe I get module import errors related to sklearn and scipy.

    I created a test script (test.py) to test imports, which only imports pandas and sklearn and then prints a success message:

    import time
    import pandas as pd
    import sklearn
    
    def main():
      print('hello world!')
      time.sleep(5)
    
    
    if __name__ == '__main__':
      main()
    

    I'm aware of pyinstaller hooks and I was able to resolve the pandas errors by adding a hook to the pyinstaller hooks directory. I added similar hooks for sklearn and scipy it looks like they're running, but in the pyinstaller output I'm getting warnings that 'Hidden import "sklearn.utils.sparsetools._graph_validation" not found!' and similar one for '._graph_tools'.

    Here's the hook for scipy (hook-scipy.py):

    print('loading custome hook for scipy')
    
    from PyInstaller.utils.hooks import collect_submodules
    hiddenimports = collect_submodules('scipy') 
    

    Here's a snapshot of the warnings generated from running pyinstaller

    Here's a snapshot of the error when running test.exe

    I'm working in a virtual environment where pyinstaller, pandas, sklearn, scipy and all dependencies are installed (at least I can get the regular test.py script running in this venv). Using PyInstaller 3.3.1, Python 3.6.4 on Windows 10.10.0.

    Any help is appreciated!

  • Luke
    Luke over 5 years
    Thanks for your reply Chris. I checked in the hooks folder and there's already a file in the hooks dir called hook-sklearn.metrics.cluster.py: # Tested on Windows 7 64bit with scikit-learn 0.17 and Python 2.7 hiddenimports = ['sklearn.utils.sparsetools._graph_validation', 'sklearn.utils.sparsetools._graph_tools', 'sklearn.utils.lgamma', 'sklearn.utils.weight_vector']' I also added a print statement in this hook, which verifies that it's being called. It appears directly before the warnings now.
  • Luke
    Luke over 5 years
    It worked! I did have hook files in place for the ones you mention above, but the problem was I only had the hiddenimports statements. Once I included the 'datas' line it worked as you outline. I'll also mention I ran pyinstaller in the onedir mode but I'll try it in single file mode too. Edit: also I still get the warnings shown above even though it works- I guess they are just warnings and not errors? Thanks very much for your help!
  • It_is_Chris
    It_is_Chris over 5 years
    @LukeWalsh I'm pretty sure the Warning Hidden import ... not found is satisfied in the hook-sklearn.py file. I am glad it worked for you. Let me know if you run into any other issues.
  • schrodingerscat11
    schrodingerscat11 over 2 years
    Where is the hooks folder located? I am not sure where to place these files. Thanks.
  • It_is_Chris
    It_is_Chris over 2 years
    @schrodingerscat11 using Anaconda on windows: C:\Users\user_name\AppData\Local\Programs\Python\Python38\Li‌​b\site-packages\PyIn‌​staller\hooks or on a Mac: /Users/user_name/opt/anaconda3/pkgs/pyinstaller-3.6-py38ha44‌​1bb4_5/lib/python3.8‌​/site-packages/PyIns‌​taller/hooks but you need to make sure the version are correct. Probably easiest to search your computer for the pyinstaller folder.