Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation
Solution 1
Tensorflow 2.1+
What's going on?
With the new Tensorflow 2.1 release, the default tensorflow
pip package contains both CPU and GPU versions of TF. In previous TF versions, not finding the CUDA libraries would emit an error and raise an exception, while now the library dynamically searches for the correct CUDA version and, if it doesn't find it, emits the warning (The W in the beginning stands for warnings, errors have an E (or F for fatal errors) and falls back to CPU-only mode. In fact, this is also written in the log as an info message right after the warning (do note that if you have a higher minimum log level that the default, you might not see info messages). The full log is (emphasis mine):
2020-01-20 12:27:44.554767: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found
2020-01-20 12:27:44.554964: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Should I worry? How do I fix it?
If you don't have a CUDA-enabled GPU on your machine, or if you don't care about not having GPU acceleration, no need to worry. If, on the other hand, you installed tensorflow and wanted GPU acceleration, check your CUDA installation (TF 2.1 requires CUDA 10.1, not 10.2 or 10.0).
If you just want to get rid of the warning, you can adapt TF's logging level to suppress warnings, but that might be overkill, as it will silence all warnings.
Tensorflow 1.X or 2.0:
Your CUDA setup is broken, ensure you have the correct version installed.
Solution 2
To install the prerequisites for GPU support in TensorFlow 2.1:
- Install your latest GPU drivers.
- Install CUDA 10.1.
- If the CUDA installer reports "you are installing an older driver version", you may wish to choose a custom installation and deselect some components. Indeed, note that software bundled with CUDA including GeForce Experience, PhysX, a Display Driver, and Visual Studio integration are not required by TensorFlow.
- Also note that TensorFlow requires a specific version of the CUDA Toolkit unless you build from source; for TensorFlow 2.1 and 2.2, this is currently version 10.1.
- Install cuDNN.
- Download cuDNN v7.6.4 for CUDA 10.1. This will require you to sign up to the NVIDIA Developer Program.
- Unzip to a suitable location and add the bin directory to your PATH.
- Install tensorflow by
pip install tensorflow
. - You may need to restart your PC.
Solution 3
TensorFlow 2.3.0 works fine with CUDA 11. But you have to install tf-nightly-gpu (after you installed tensorflow and CUDA 11): https://pypi.org/project/tf-nightly-gpu/
Try:
pip install tf-nightly-gpu
Afterwards you'll get the message in your console:
I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_110.dll
Solution 4
I solved this another way. First of all I installed cuda 10.1 toolkit from this link.
Where I selected installer type: exe(local) (for windows) and installed 10.1 in custom mode (without visual studio integration, NVIDIA PhysX because previously I installed CUDA 10.2 so required dependencies were installed automatically)
After installation, From the Following Path
(C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin
)
, in my case, I copied cudart64_101.dll
file and pasted in
(C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\bin
).
Then importing Tensorflow worked smoothly.
Solution 5
In my case the tensorflow install was looking for cudart64_101.dll
The 101 part of cudart64_101 is the Cuda version - here 101 = 10.1
I had downloaded 11.x, so the version of cudart64 on my system was cudart64_110.dll
This is the wrong file!! cudart64_101.dll ≠ cudart64_110.dll
Solution
Download Cuda 10.1 from https://developer.nvidia.com/
Install (mine crashes with NSight Visual Studio Integration, so I switched that off)
When the install has finished you should have a Cuda 10.1 folder, and in the bin the dll the system was complaining about being missing
Check that the path to the 10.1 bin folder is registered as a system environmental variable, so it will be checked when loading the library
You may need a reboot if the path is not picked up by the system straight away
GPhilo
Just another young engineer fighting his way through C++, CUDA and Python
Updated on October 29, 2021Comments
-
GPhilo over 2 years
I just installed the latest version of Tensorflow via
pip install tensorflow
and whenever I run a program, I get the log message:W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found
Is this bad? How do I fix the error?
-
GPhilo about 4 yearsThis is a terrible idea, CUDA minor versions (i.e., 10.1, 10.2, etc) are not compatible with each other. Linking to 10.2 "as if" it was 10.1 will cause random crashes (that's assuming the DLL loads at all). Besides, this implies you installed cuda 10.2 on your machine, which for a CPU-only installation is not necessary
-
Dan Marinescu about 4 yearsof course it is not necessary for cpu only. as far as 10.2 being completely different than 10.1 and random crashes, i would recommend you learn about dynamic linking and exported symbols
-
Dan Marinescu about 4 yearsthe dll loads and works just fine (for your information)
-
Dan Marinescu about 4 yearsbut end users should install prescribed dynamic libraries, especially if they are not sure what this is all about
-
GPhilo about 4 years"of course it's not necessary for CPU-only": read again the question title.
-
SaeedM almost 4 yearsI've had the same problem when using Conda env. The solution was the same: copied the Dlls mentioned in the warning from "...\Anaconda\Envs\<myEnv>\Library\bin" to the "...\Anaconda3\Library\bin" The required Dlls should be in a path that os knows about it.
-
GPhilo almost 4 yearsJust a note: Adding the folder
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin
toPATH
would've worked as well (and is the recommended way to go, to avoid messing up with CUDA installations) -
Dustin Andrews almost 4 yearsIf you installed CUDA 10.1 for example you probably already have the DLL in "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin\cudart64_101.dll". I'm personally averse to downloading DLLs from secondary sources.
-
Mir-Ismaili over 3 yearsThanks. But I add another point: If you installed python from Microsoft Store, you may still see this issue, even if you've done the required steps! See my answer here.
-
Mir-Ismaili over 3 yearsThanks. But I add another point: If you installed python from Microsoft Store, you may still see this issue, even if you've done the required steps! See my answer here.
-
Mark over 3 yearsThank you! I thought I was going to have to roll back to 10.1.
-
t3chb0t over 3 years6. Restart your IDE after modifying the PATH variable.
-
Maf over 3 yearsWhere is located the cuda
bin
directory? -
Michael Hoffmann over 3 yearsRecommending that people install an executable from an unofficial source is not a good solution.
-
Hagbard over 3 yearsThank you! "pip install tf-nightly" did the trick for me.
-
Mitch over 3 yearsThank you, copying the dll file there worked great: tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
-
Upulie Han over 3 yearsdeveloper.nvidia.com/… Better use the NVIDIA site for downloading the .dll file.
-
Upulie Han over 3 yearsThank you! This was my only solution after trying for more than 15 hours.
-
quarkz over 3 yearsFor me, the directory location is C:\Users\<user>\.conda\envs\<env name>\Library\bin
-
Nerxis about 3 yearsNote: I was a bit confused that after installing the latest GPU drivers,
nvidia-smi
showed meCUDA Version: 11.2
(CUDA 11 is needed for TF 2.4) but the real installed and used CUDA version by my TF was 10.1. So never skip the CUDA installation step and always ensure your PATH is correctly set up. -
BSalita about 3 yearsHad to do four steps: 1) Install
tf-nightly-gpu
, 2) install latest nvidia cuda toolkit (11.2+) using custom install but don't overwrite newer drivers (unselect), 3) Restart terminal. 4) Reinstall Nvidia driver if above steps borked it. e.g. nvidia-smi results inFailed to initialize NVML: GPU access blocked by the operating system
. -
JoseOrtiz3 about 3 yearsConfirmed that a reboot was needed in my case after installing CUDA 11.0 to get tensorflow to find it (cudart64_xyz.DLL requires CUDA version xy.z, in my case, xyz=110, not 101)
-
Christian Vincenzo Traina about 3 yearsIn March 2021, tensorflow new version is stated to be compatible with cuda 11, and I don't feel fine at using a nightly tensorflow-gpu version, it sounds like a hack. Nonetheless, this was the only solution working
-
Beolap almost 3 yearsWow, restarting my pc could have saved me 2h. Thanks :)
-
pete over 2 yearsI uninstalled every component of CUDA and still got the same error as if CUDA still existed
-
GPhilo over 2 years@pete the error appears if it can't find the right CUDA version. Since you uninstalled it, of course it won't find it and print the error... That's the normal behaviour.
-
GPhilo almost 2 years@brainslugs83 CUDA minor versions are not interchangeable, that's a limitation coming from CUDA itself