How to use numpy with OpenBLAS instead of Atlas in Ubuntu?

Question

I have looked for an easy way to install/compile Numpy with OpenBLAS but didn't find an easy answer. All the documentation I have seen takes too much knowledge as granted for someone like me who is not used to compile software. There are two packages in Ubuntu related to OpenBLAS : libopenblas-base and libopenblas-dev. Once they are installed, what should I do to install Numpy again with them? Thanks!

Note that when these OpenBLAS packages are installed, Numpy doesn't work anymore: it can't be imported: ImportError: /usr/lib/liblapack.so.3gf: undefined symbol: ATL_chemv. The problem occurs as well when installing Theano with their website instructions for Ubuntu.

This was noticed here already.

Solution 1

Run sudo update-alternatives --all and set liblapack.so.3gf to /usr/lib/lapack/liblapack.so.3gf

Solution 2

To add to the accepted answer (of using update-alternatives), the reason for this is because OpenBlas is not compatible with the Atlas version of Lapack. For each of the Blas and Lapack versions:

1. Default Blas + Default Lapack => OK
2. OpenBlas + Default Lapack => OK
3. Atlas-Blas + Default Lapack => OK
4. Atlas-Blas + Atlas-Lapack => OK
5. OpenBlas + Atlas-Lapack => ERROR! (The following case here.)

This is from both personal experience (with the exact same issue) and realizing why such a combination wasn't mentioned in this comparison blog.

By the way, you can just find the necessary files in /etc/alternatives/, usually with a filename starting with lib*. For each one do sudo update-alternatives --config <filename>. For example, do to following:

• sudo update-alternatives --config libblas.so
• sudo update-alternatives --config libblas.so.3

to change the Blas version.

Solution 3

Consider using EasyBuild (http://hpcugent.github.io/easybuild/), an open-source framework for building and installing software.

It allows you to (very easily) build and install (scientific) software with various compiler, and using different BLAS libraries (ATLAS, OpenBLAS, ACML, Intel MKL, ...).

Once you install EasyBuild (pro tip: use the bootstrapping procedure described at https://github.com/hpcugent/easybuild/wiki/Bootstrapping-EasyBuild), it boils down to running a single command, something like:

eb numpy-1.6.2-goolf-1.4.10-Python-2.7.3.eb -ldr

This will first build and install of full compiler toolchain (goolf: GCC+OpenBLAS+OpenMPI+LAPACK+FFTW), and subsequently build Python and numpy with that toolchain. And all that while you're getting lunch. ;-)

Disclaimer: I'm one of the EasyBuild developers.

Comments

[seberg]

Since I don't have a step to step instructions: You may have to run sudo update-alternatives --all and set the setting for that (you may not have to do this if you did not install Atlas). You also must install liblapack3gf I believe. I personally had some problems with other packages when doing that (shogun), but probably that doesn't matter for you. And run "import numpy; numpy.test()" before and after you change things...

[PierreE]

It actually worked! : liblipack3gf was already installed, and using update-alternatives --all and choose the alternative /usr/lib/lapack/liblapack.so.3gf for liblapack.so.3gf

[user239558]

If you just uninstall all the atlas packages, then this error goes away.

[Henry Thornton]

Does eb build latest releases eg. python 2.7.5, numpy 1.7.1 etc. ?

[Kenneth Hoste]

See github.com/hpcugent/easybuild-easyconfigs/tree/master/easybu‌​ild/… and github.com/hpcugent/easybuild-easyconfigs/tree/master/easybu‌​ild/… for easyconfig files in the latest release. EasyBuild v1.5 (release planned end of this week), will add easyconfig files for Python 2.7.5 (see github.com/hpcugent/easybuild-easyconfigs/tree/develop/…), but usually installing new versions of already supported is easy: just bump the version number in the easyconfig file and give it to eb.

[Luke Yeager]

Great answer. I solved my issue by installing liblapack3 in addition to libopenblas-base and libatlas3-base.