After building TensorFlow from source, seeing libcudart.so and libcudnn errors

Question

I'm building TensorFlow from source code. The build appears to succeed; however, when my TensorFlow program invokes import tensorflow, one or both of the following errors appear:

• ImportError: libcudart.so.8.0: cannot open shared object file: No such file or directory
• ImportError: libcudnn.5: cannot open shared object file: No such file or directory

Solution 1

First, for the following error:

ImportError: libcudart.so.8.0: cannot open shared object file: No such file or directory

make sure your LD_LIBRARY_PATH includes your lib64 directory in whichever path you installed your cuda package in. You can do this by adding an export line in your .bashrc. For Omar, it looked like the following:

I fixed this just adding the cuda path to my .bashrc

export LD_LIBRARY_PATH=/usr/local/cuda/lib64/

---

For me, I had to do Omar's line and also: export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64/ because I have two directories involving cuda (probably not the best).

---

Second, are you sure you installed cuDNN? Note that this is different from the regular cuda package. You will need to register, then download and install the package from the following page: https://developer.nvidia.com/cudnn

---

Third, I had this same problem:

ImportError: libcudnn.5: cannot open shared object file: No such file or directory

It turns out there is no libcudnn.5 in my /usr/local/cuda/lib64 or /usr/local/cuda-8.0/lib64 directories. However, I do have a libcudnn.so.6.* file. To solve the problem, I created a soft link:

ln -s libcudnn.so.6.* libcudnn.so.5

in my /usr/local/cuda/lib64 directory. Now everything works for me. Your directory might be different if you already had cuDNN, and your libcudnn.so.6.* might be a different version, so check that.

Solution 2

I came across the same issue

In [1]: import tensorflow
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
/usr/local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow.py in <module>()
     40     sys.setdlopenflags(_default_dlopen_flags | ctypes.RTLD_GLOBAL)
---> 41   from tensorflow.python.pywrap_tensorflow_internal import *
     42   from tensorflow.python.pywrap_tensorflow_internal import __version__

/usr/local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow_internal.py in <module>()
     27             return _mod
---> 28     _pywrap_tensorflow_internal = swig_import_helper()
     29     del swig_import_helper

/usr/local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow_internal.py in swig_import_helper()
     23             try:
---> 24                 _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
     25             finally:

/usr/local/lib/python3.5/imp.py in load_module(name, file, filename, details)
    241         else:
--> 242             return load_dynamic(name, filename, file)
    243     elif type_ == PKG_DIRECTORY:

/usr/local/lib/python3.5/imp.py in load_dynamic(name, path, file)
    341             name=name, loader=loader, origin=path)
--> 342         return _load(spec)
    343

ImportError: libcudnn.so.5: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
<ipython-input-1-a649b509054f> in <module>()
----> 1 import tensorflow

/usr/local/lib/python3.5/site-packages/tensorflow/__init__.py in <module>()
     22
     23 # pylint: disable=wildcard-import
---> 24 from tensorflow.python import *
     25 # pylint: enable=wildcard-import
     26

/usr/local/lib/python3.5/site-packages/tensorflow/python/__init__.py in <module>()
     49 import numpy as np
     50
---> 51 from tensorflow.python import pywrap_tensorflow
     52
     53 # Protocol buffers

/usr/local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow.py in <module>()
     50 for some common reasons and solutions.  Include the entire stack trace
     51 above this error message when asking for help.""" % traceback.format_exc()
---> 52   raise ImportError(msg)
     53
     54 # pylint: enable=wildcard-import,g-import-not-at-top,unused-import,line-too-long

ImportError: Traceback (most recent call last):
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow.py", line 41, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
  File "/usr/local/lib/python3.5/imp.py", line 242, in load_module
    return load_dynamic(name, filename, file)
  File "/usr/local/lib/python3.5/imp.py", line 342, in load_dynamic
    return _load(spec)
ImportError: libcudnn.so.5: cannot open shared object file: No such file or directory


Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/install_sources#common_installation_problems

for some common reasons and solutions.  Include the entire stack trace
above this error message when asking for help.

I have installed cudnn 6.0 while it needs libcudnn.so.5, apparently it couldn't find libcudnn.so.5. It seems that your tensorflow needs cudnn 5.x, so install cudnn 5.x

Make sure you have already installed cuda 8.0 and exported the PATH and LD_LIBRARY_PATH

To install cudnn 5.x, try the following commands

Extract tgz files

$ tar -zxvf cudnn-8.0-linux-x64-v5.1.tgz

Check the files

$ cd cuda/lib64/
$ ls -l
total 150908
lrwxrwxrwx 1 doom doom       13 Nov  7  2016 libcudnn.so -> libcudnn.so.5
lrwxrwxrwx 1 doom doom       18 Nov  7  2016 libcudnn.so.5 -> libcudnn.so.5.1.10
-rwxr-xr-x 1 doom doom 84163560 Nov  7  2016 libcudnn.so.5.1.10
-rw-r--r-- 1 doom doom 70364814 Nov  7  2016 libcudnn_static.a

Here you will see 2 symbolic link files, and just copy libcudnn.so.5.1.10 and libcudnn_static.a to /usr/local/cuda/lib64

Make symbolic link files

$ cd /usr/local/cuda/lib64/
$ sudo ln -s libcudnn.so.5.1.10 libcudnn.so.5
$ sudo ln -s libcudnn.so.5 libcudnn.so
$ ls -l libcudnn*
lrwxrwxrwx 1 root root       13 May 24 09:24 libcudnn.so -> libcudnn.so.5
lrwxrwxrwx 1 root root       18 May 24 09:24 libcudnn.so.5 -> libcudnn.so.5.1.10
-rwxr-xr-x 1 root root 84163560 May 24 09:23 libcudnn.so.5.1.10
-rw-r--r-- 1 root root 70364814 May 24 09:23 libcudnn_static.a

Copy cudnn.h in include directory to /usr/local/cuda/include

$ sudo cp cudnn.h /usr/local/cuda/include/

Hope it will help you!

Solution 3

I fixed this just adding the cuda path to my .bashrc

export LD_LIBRARY_PATH=/usr/local/cuda/lib64/

Just have in mind that first you need to go to nvidia Deep Learning page, register and download cuDNN, extract and copy the files from include and lib64 folders into your cuda installation.

Solution 4

I have seen a similar error (bottom of this post), but complaining about libcudnn.so.6 instead of libcudart.so.8.0 (see a note below).

Solution:

1. Download 'cuDNN v6.0 Library for Linux':
   • Go to https://developer.nvidia.com/rdp/cudnn-download
   • Click on 'Download cuDNN v6.0 (April 27, 2017), for CUDA 8.0'
   • Then click on 'cuDNN v6.0 Library for Linux'. A file (named 'cudnn-8.0-linux-x64-v6.0.tgz') will be downloaded.
2. Follow the instructions of Alexander Yau above to install the cuDNN v6.0 library.

Note:

the Tensorflow installation instructions (as of 20/Aug/2017) require installing cuDNN v5.1, but my Tensorflow installation (following the instructions for installing in a virtualenv) required cuDNN v6.x (as indicated by the error). I don't know if it is a mistake on my side or a Tensorflow documentation one. Nevertheless, above solution worked for me.

Encountered error:

In [1]: import tensorflow as tf
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-41389fad42b5> in <module>()
----> 1 import tensorflow as tf

/home/haseeb/.virtualenvs/attention_transformer/local/lib/python2.7/site-packages/tensorflow/__init__.py in <module>()
     22 
     23 # pylint: disable=wildcard-import
---> 24 from tensorflow.python import *
     25 # pylint: enable=wildcard-import
     26 

/home/haseeb/.virtualenvs/attention_transformer/local/lib/python2.7/site-packages/tensorflow/python/__init__.py in <module>()
     47 import numpy as np
     48 
---> 49 from tensorflow.python import pywrap_tensorflow
     50 
     51 # Protocol buffers

/home/haseeb/.virtualenvs/attention_transformer/local/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py in <module>()
     50 for some common reasons and solutions.  Include the entire stack trace
     51 above this error message when asking for help.""" % traceback.format_exc()
---> 52   raise ImportError(msg)
     53 
     54 # pylint: enable=wildcard-import,g-import-not-at-top,unused-import,line-too-long

ImportError: Traceback (most recent call last):
  File "/home/haseeb/.virtualenvs/attention_transformer/local/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 41, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/home/haseeb/.virtualenvs/attention_transformer/local/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/home/haseeb/.virtualenvs/attention_transformer/local/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
ImportError: libcudnn.so.6: cannot open shared object file: No such file or directory


Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/install_sources#common_installation_problems

for some common reasons and solutions.  Include the entire stack trace
above this error message when asking for help.

Solution 5

As of now tensor-flow supports cuda-9.0

Do following things. Hope it helps :

$ sudo apt-get install cuda-9.0
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-9.0/lib64

Download cuDNN for 9.0 (You need to register before downloading) https://developer.nvidia.com/rdp/form/cudnn-download-survey

$ sudo dpkg -i libcudnn7_7.1.2.21-1+cuda9.0_amd64.deb

Close all terminal and open new

$ source activate tensorflow
$ python
>> import tensorflow as tf

You should not get any error after this.

View more solutions

Comments

[Cosmo]

Hi, I'm using Ubuntu 14.04 but I can't get access to any cuDNN debian package from https://developer.nvidia.com/rdp/cudnn-download. Each link to a Ubuntu deb package echos a 403 error and says "Page Not Found". Have you ever met this?

[rayryeng]

@cosmozhang probably temporary. I also use 14.04 and I was able to download it. Just did it yesterday.

[WaterRocket8236]

What if cudnn.h is not there in include?

[GoingMyWay]

@BhabaniMohapatra, how did you download the cudnn-8.0-linux-x64-v5.1.tgz?

[WaterRocket8236]

@AlexanderYau I downloaded the file from nvidia website. I already registered as a member. I dint noticed that include folder was there after extracting it. I made it work. Thank you.

[WY Hsu]

Work for me on Ubuntu 16.04 + Gtx 660 + CUDA 8.0, @AlexanderYau thx

[Victor Silva]

Works gor me, except I have to use the command export LD_LIBRARY_PATH=/usr/local/lib:${LD_LIBRARY_PATH} to change the environment variable.

[kRazzy R]

I'm facing same issue. I was trying to run the gpu code from here : github.com/Franck-Dernoncourt/NeuroNER

[Mr_and_Mrs_D]

It was a mistake in docs - see: github.com/tensorflow/tensorflow/issues/12416