Pyinstaller numpy "Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll"

Question

I'm new with python apps. I'm trying to build my python GUI app with pyinstaller. My app depends on the following packages: PyQt4, numpy, pyqtgraph, h5py. I'm working with WinPython-32bit-3.4.4.1.

I build the app with this command:

pyinstaller --hidden-import=h5py.defs --hidden-import=h5py.utils --hidden-import=h5py.h5ac --hidden-import=h5py._proxy VOGE.py

I launch my app with the exe file in the dist directory created by pyinstaller and it seems work fine until the program call numpy and crash with this error:

Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll

The mkl_intel_thread.dll is not present in the software directory; but with the file copied in the root dir of the program I got the same error

Thanks for your help

Solution 1

I created a hook-numpy.py to deal with this problem:

from PyInstaller import log as logging 
from PyInstaller import compat
from os import listdir

libdir = compat.base_prefix + "/lib"
mkllib = filter(lambda x : x.startswith('libmkl_'), listdir(libdir))
if mkllib <> []: 
   logger = logging.getLogger(__name__)
   logger.info("MKL installed as part of numpy, importing that!")
   binaries = map(lambda l: (libdir + "/" + l, ''), mkllib)

In my case, conda is installing the mkl libraries to speed up numpy and scipy.

Solution 2

I had the same issue using Pyinstaller and Numpy. By default pyinstaller seems to not take into account numpy binaries so you have to specify it manually. You can add the files editing the ".spec" file "binaries" variable, but that will only work for your current program. If you want it working for all the programs your freeze you should make a "hook" and save it in C:\Python3*\Lib\site-packages\PyInstaller\hooks.

I had to adapt LeonidR's code to make the numpy-hook working. I rewrited it using a more modern, pythonic approach using list comprehensions:

from PyInstaller import log as logging 
from PyInstaller import compat
from os import listdir

mkldir = compat.base_prefix + "/Lib/site-packages/numpy/core" 
logger = logging.getLogger(__name__)
logger.info("MKL installed as part of numpy, importing that!")
binaries = [(mkldir + "/" + mkl, '') for mkl in listdir(mkldir) if mkl.startswith('mkl_')] 

"Binaries" is a list of tuples. The second item of the tuple corresponds to the folder where you want to place the 'dlls'. In this case is empty so it copies them directly in the main folder where your '.exe' is.

Solution 3

I just ran into the same problem. As a workaround I copied the DLL's manually, as described in https://stackoverflow.com/a/34893933/4089081

I'm trying to find a better solution though.

Solution 4

Just tested with Anaconda Navigator 2.1.0, pyinstaller 3.6, python 3.8.8, Windows 10. As documented above, pyinstaller doesn't know to install mkl_intel_thread.1.dll so there's a fatal error when you run the new program saying the DLL can't be found.

It works to tell pyinstaller either on the command line or in the .spec script to install from a specific location. You almost certainly have the DLL if you've got conda installed numpy. Pyinstaller is fussy about the specifications so here are two examples:

1. .spec script:

   binaries=[('~\\anaconda3\\pkgs\\mkl-2021.4.0-haa95532_640\\Library\\bin\\mkl_intel_thread.1.dll', '.')]
   

2. command line:

   pyinstaller  --add-binary '~\anaconda3\pkgs\mkl-2021.4.0-haa95532_640\Library\bin\mkl_intel_thread.1.dll;.' yourmodule.py
   

Note the single quotes on the command line and the ;. because this is really a tuple.

Solution 5

I just update numpy+mkl to the latest version, you can download numpy+mkl from here

View more solutions

Comments

[f_ciriolo]

Thanks. Your fix has solved my problem. But now I realized that when my app call function related to pyqtgraph it crash with the same error: Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll

[LeonidR]

Is this a different app? it is crashing with the same error?

[f_ciriolo]

No, the same app include also pyqtgraph. Now when I execute the the exe genereted by pyinstaller it runs well until a function that use pyqtgraph is called after that it crash with the Intel MKL FATAL ERROR. Before your fix the crash occurs erlier

[LeonidR]

Perplexing, my intuition is that you're using different Python package management schemes for pyqtgraph and numpy. I'd recommend that you build the one-dir app and then run it with a trace tracking which libraries get loaded.

[f_ciriolo]

I'm building with one-dir (default). Sorry but I'm new in python what do you exactly mean with "run it with a trace tracking"? Have I to track the library loading on the executable or on the original .py?

[muazfaiz]

Can you please elaborate it ? I am facing the same problem but I don't understand that If I create hook.py, and save it to C:\Python3*\Lib\site-packages\PyInstaller\hooks how will I call it in my code so that it reads file as well ?

[Raj]

This almost worked for me but as others noted that libiomp5md.dll is also needed, I modified the last line to: binaries = [(mkldir + "/" + mkl, '') for mkl in listdir(mkldir) if mkl.startswith('mkl_') or mkl.startswith('libio')]

[Regi Mathew]

This is my solution too.

[Guimoute]

Oh, this is why my .exe suddenly weighted 200MB less when I switched computers. It doesn't find the binaries properly on this one.