How read Common Data Format (CDF) in Python

Question

I need to read CDF file with using python. I have found libraries but i did not understand how use it. For example at this(Python lib), i need to download CDF lib, i don't know where to download. There is download page for CDF but it seems irrelevant with this library.

Solution 1

The answer by @miraculixx is correct, but it assumes that you have already installed the CDF C Library.

Here's an easy to follow guide if you didn't even know what the CDF file format was before you found this question on SO.

1. Download the latest version of the CDF C Library:

You can find the latest stable release at this link. Grab the source code using wget, and extract it. Note: the following will create a directory in the current folder ./ if you want to download the code in a different path make sure you change the code below.

wget -r -l1 -np -nd -nc http://cdaweb.gsfc.nasa.gov/pub/software/cdf/dist/latest-release/linux/ -A cdf*-dist-all.tar.gz
tar xf cdf*-dist-all.tar.gz -C ./
cd cdf*dist

2. Install all the dependencies:

SpacePy and the CDF Library have several dependencies (as pointed out by @Michal Dyzma). You can install them all using conda or pip, and apt.

pip install numpy scipy h5py matplotlib networkx
apt install build-essential gfortran libncurses5-dev

3. Compile the C Library:

You should have downloaded a README.install file that contains a lot more details on this step than I'll provide. The two cents are that you want to check which compile variables are required/optional for your system and needs.

make all.help

I will be building the distribution for Linux using the GNU C compiler. I am not interested in the FORTRAN interface, and my operating system supports shareable libraries. I want to install the Curses-based toolkit programs that allow to use the command-line based interactive CDF tools (that's why we installed libncurses5-dev dependency in step 2). As a result this is the final make command:

make OS=linux ENV=gnu CURSES=yes FORTRAN=no UCOPTIONS=-O2 SHARED=yes -j4 all
make install #no sudo

Installation should run smooth and add all the files in the ./bin, ./include, and ./lib sub-directories.

4. Set the environment variables:

There should be a file in ./bin called definitions.B that does this automatically for you, make it executable with chmod+x and add the following line to your ~/.bashrc (Note: 1) I'm assuming you installed the library at the path $HOME/Libraries/; 2) There is a space after the .):

. $HOME/Libraries/cdf/cdf36_3-dist/bin/definitions.B

IMPORTANT NOTE: The file above has a bug at line 68, instead of appending to environment variable LD_LIBRARY_PATH it overwrites it. The fix is easy, replace line 68 with the following:

export LD_LIBRARY_PATH=$HOME/Libraries/cdf/cdf36_3-dist/lib:$LD_LIBRARY_PATH

If for some reason definitions.B is not there, simply add the following:

export CDF_BASE=$HOME/Libraries/cdf/cdf36_3-dist
export CDF_INC=$CDF_BASE/include
export CDF_LIB=$CDF_BASE/lib
export CDF_BIN=$CDF_BASE/bin
export LD_LIBRARY_PATH=$CDF_BASE/lib:$LD_LIBRARY_PATH

5. You're all set, go and do good:

Assuming you installed spacepy with pip the following should work out of the box:

from spacepy import pycdf
cdf = pycdf.CDF('/path/to/file.cdf')
print(cdf)

Solution 2

If you have Python's package tool, pip installed, you can get the spacepy cdf library as follows:

$ pip install git+https://github.com/spacepy/spacepy.git

Note this will install a lot of dependencies, including numpy and scipy. These can be a bit difficult to install from scratch. You might want to install a ready-made package first, e.g. anaconda. Once that's done, just use the above command and spacepy should install like a breeze.

Once the installation of spacepy was successful, according to this example it should work something like this:

from spacepy import pycdf
cdf = pycdf.CDF('/path/to/file.cdf')
print(cdf)

Solution 3

While ago I had same problem. I assume, you work on Windows...

According to Spacepy documentation you need several dependencies to use its's cdf module.

First of all it SpacePy officially supports only 32-bit python version, therefore you have to have python in 323bit.

Second, it requires NASA CDF library installed in you system (also 32 bit version). You can grab it from here.

Third proceed with Spacepy dependencies:

• numpy
• scipy
• matplotlib
• h5py
• networkx
• ffnet

Most of them are part of Anaconda bundle. If they are not and you have to install them simply pip install <package name>.

If you have problems with compiling from the sources, I advice to go to Christoph Gohlke web site and grab pre-built binaries for Windows matching your python version. http://www.lfd.uci.edu/~gohlke/pythonlibs/

This should get you going with Spacepy CDF module.

You can also try another approach. Download CDF-to-netCDF converter from NASA page and run it on your CDF file.

Python has nice netCDF module, which can be installed from GitHub, or python repo. In this case you also need several dependencies like HDF5, netCDF-4, numpy, cython.

Once you have netCDF file, you can access it with netCDF module or scipy.io module.