Convert NetCDF file to CSV or text using Python

Question

I'm trying to convert a netCDF file to either a CSV or text file using Python. I have read this post but I am still missing a step (I'm new to Python). It's a dataset including latitude, longitude, time and precipitation data.

This is my code so far:

import netCDF4
import pandas as pd

precip_nc_file = 'file_path'
nc = netCDF4.Dataset(precip_nc_file, mode='r')

nc.variables.keys()

lat = nc.variables['lat'][:]
lon = nc.variables['lon'][:]
time_var = nc.variables['time']
dtime = netCDF4.num2date(time_var[:],time_var.units)
precip = nc.variables['precip'][:]

I am not sure how to proceed from here, though I understand it's a matter of creating a dataframe with pandas.

Solution 1

I think pandas.Series should work for you to create a CSV with time, lat,lon,precip.

import netCDF4
import pandas as pd

precip_nc_file = 'file_path'
nc = netCDF4.Dataset(precip_nc_file, mode='r')

nc.variables.keys()

lat = nc.variables['lat'][:]
lon = nc.variables['lon'][:]
time_var = nc.variables['time']
dtime = netCDF4.num2date(time_var[:],time_var.units)
precip = nc.variables['precip'][:]

# a pandas.Series designed for time series of a 2D lat,lon grid
precip_ts = pd.Series(precip, index=dtime) 

precip_ts.to_csv('precip.csv',index=True, header=True)

Solution 2

import xarray as xr

nc = xr.open_dataset('file_path')
nc.precip.to_dataframe().to_csv('precip.csv')

Solution 3

Depending on your requirements, you may be able to use Numpy's savetxt method:

import numpy as np

np.savetxt('lat.csv', lat, delimiter=',')
np.savetxt('lon.csv', lon, delimiter=',')
np.savetxt('precip.csv', precip, delimiter=',')

This will output the data without any headings or index column, however.

If you do need those features, you can construct a DataFrame and save it as CSV as follows:

df_lat = pd.DataFrame(data=lat, index=dtime)
df_lat.to_csv('lat.csv')

# and the same for `lon` and `precip`.

Note: here, I assume that the date/time index runs along the first dimension of the data.

Comments

[aliki43]

Thank you!! This was perfect

[aliki43]

Thanks! Unfortunately this didn't work - I decided to just extract all the latitudes and longitudes I was using in my other dataset, and looped over that to get the time series of each place. Like in the link I provided above. Time consuming, but it works!

[Eric Bridger]

You're welcome. You should accept the answer for future readers.

[denis_lor]

Can you provide more clarity of why you wrote this code compared to the question being asked? How does this code give an answer to the question?

[Robert Davy]

The popular answer didn't work for me. I am not sure why (maybe something I did wrong?). It seems that xarray library provides a solution in fewer lines of code. This alternative may save time for some people, as it did for me.

[Robert Davy]

Regarding the other answer which didn't work. I tried it on a standard NOAA mslp netCDF file, esrl.noaa.gov/psd/thredds/fileServer/Datasets/ncep.reanalysi‌​s2/…, and obtained the following error at the 2nd last line:

[Robert Davy]

>>> # a pandas.Series designed for time series of a 2D lat,lon grid ... precip_ts = pd.Series(precip, index=dtime) Traceback (most recent call last): File "<stdin>", line 2, in <module> File "C:\Users\dav500\AppData\Local\Continuum\anaconda3\lib\site-‌​packages\pandas\core‌​\series.py", line 262, in init raise_cast_failure=True) File "C:\Users\dav500\AppData\Local\Continuum\anaconda3\lib\site-‌​packages\pandas\core‌​\internals\construct‌​ion.py", line 658, in sanitize_array raise Exception('Data must be 1-dimensional') Exception: Data must be 1-dimensional

[Jane Kathambi]

Thank you so much this worked for me so perfectly! I have struggled for days but you saved me!