Why is my export to netcdf command leading to a __truediv__ error?

[msaharia]

Code Sample, a copy-pastable example if possible

import numpy as np
import xarray as xr
import glob, os

NCDIR = './output/out/'
finalfile = 'summaout.nc'

outfilelist = glob.glob((NCDIR+'/*{}*.nc').format('basin_*timestep'))

ds=xr.open_mfdataset(outfilelist, concat_dim='hru')

replace = ds['pptrate']
runoff = ds['averageInstantRunoff'].values
runoff = np.squeeze(runoffdata, axis=2)
runoff = runoff.transpose()
replace.values = runoff

ncconvert = ds.drop('averageInstantRunoff')
runoffarray = xr.DataArray(runoff, dims=['time','hru'])
ds['averageInstantRunoff'] = runoffarray
ds.to_netcdf('test.nc')

Problem description

This is ds just before export.

<xarray.Dataset>
Dimensions:               (hru: 17, time: 233)
Coordinates:
  * hru                   (hru) int64 9 17 11 8 3 2 6 4 7 12 1 13 10 16 15 5 14
  * time                  (time) datetime64[ns] 2010-01-01 ... 2010-01-30
Data variables:
    pptrate               (time, hru) float64 9.241e-05 9.241e-05 ... 2.717e-09
    hruId                 (hru) int64 dask.array<shape=(17,), chunksize=(1,)>
    averageInstantRunoff  (time, hru) float64 9.241e-05 9.241e-05 ... 2.717e-09
    nSnow                 (time, hru) int32 dask.array<shape=(233, 17), chunksize=(233, 1)>
    nSoil                 (time, hru) int32 dask.array<shape=(233, 17), chunksize=(233, 1)>
    nLayers               (time, hru) int32 dask.array<shape=(233, 17), chunksize=(233, 1)>

I get this error message:

TypeError: cannot perform __truediv__ with this index type: <class 'pandas.core.indexes.datetimes.DatetimeIndex'>

Expected Output

netCDF

Output of xr.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.6.4.final.0 python-bits: 64 OS: Linux OS-release: 3.12.62-60.64.8-default machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8

xarray: 0.11.0
pandas: 0.21.0
numpy: 1.13.3
scipy: 1.0.0
netCDF4: 1.4.0
h5netcdf: 0.5.0
h5py: 2.7.1
Nio: None
zarr: None
cftime: 1.0.0
PseudonetCDF: None
rasterio: None
iris: None
bottleneck: 1.2.1
cyordereddict: None
dask: 0.19.3
distributed: 1.23.3
matplotlib: 2.1.2
cartopy: 0.15.1
seaborn: 0.9.0
setuptools: 36.7.2
pip: 18.1
conda: 4.5.11
pytest: 3.2.5
IPython: 6.2.1
sphinx: 1.6.5

[shoyer]

Thanks for the report. Could you please share the full error trace-back?

[msaharia]

@shoyer Thank you! Here it is:

Traceback (most recent call last):
  File "./concat.py", line 38, in <module>
    ncconcat.to_netcdf(os.path.join(ncdir, finalfile), 'w')
  File "/glade/u/home/manab/anaconda3/lib/python3.6/site-packages/xarray/core/dataset.py", line 1222, in to_netcdf
    compute=compute)
  File "/glade/u/home/manab/anaconda3/lib/python3.6/site-packages/xarray/backends/api.py", line 718, in to_netcdf
    unlimited_dims=unlimited_dims)
  File "/glade/u/home/manab/anaconda3/lib/python3.6/site-packages/xarray/backends/api.py", line 761, in dump_to_store
    unlimited_dims=unlimited_dims)
  File "/glade/u/home/manab/anaconda3/lib/python3.6/site-packages/xarray/backends/common.py", line 261, in store
    variables, attributes = self.encode(variables, attributes)
  File "/glade/u/home/manab/anaconda3/lib/python3.6/site-packages/xarray/backends/common.py", line 347, in encode
    variables, attributes = cf_encoder(variables, attributes)
  File "/glade/u/home/manab/anaconda3/lib/python3.6/site-packages/xarray/conventions.py", line 573, in cf_encoder
    for k, v in iteritems(variables))
  File "/glade/u/home/manab/anaconda3/lib/python3.6/site-packages/xarray/conventions.py", line 573, in <genexpr>
    for k, v in iteritems(variables))
  File "/glade/u/home/manab/anaconda3/lib/python3.6/site-packages/xarray/conventions.py", line 235, in encode_cf_variable
    var = coder.encode(var, name=name)
  File "/glade/u/home/manab/anaconda3/lib/python3.6/site-packages/xarray/coding/times.py", line 395, in encode
    encoding.pop('calendar', None))
  File "/glade/u/home/manab/anaconda3/lib/python3.6/site-packages/xarray/coding/times.py", line 365, in encode_cf_datetime
    num = (pd.DatetimeIndex(dates.ravel()) - ref_date) / time_delta
  File "/glade/u/home/manab/anaconda3/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 3885, in invalid_op
    "{typ}".format(name=name, typ=type(self)))
TypeError: cannot perform __truediv__ with this index type: <class 'pandas.core.indexes.datetimes.DatetimeIndex'>

[shoyer]

can you share ds.info() right before you save it?

[msaharia]

Here it is:

xarray.Dataset {
dimensions:
	hru = 17 ;
	time = 233 ;

variables:
	int64 hru(hru) ;
	datetime64[ns] time(time) ;
	float64 pptrate(time, hru) ;
		pptrate:long_name = precipitation rate (instant) ;
		pptrate:units = kg m-2 s-1 ;
	int64 hruId(hru) ;
		hruId:long_name = ID defining the hydrologic response unit (instant) ;
		hruId:units = - ;
	float64 averageInstantRunoff(time, hru) ;
	int32 nSnow(time, hru) ;
		nSnow:long_name = number of snow layers (instant) ;
		nSnow:units = - ;
	int32 nSoil(time, hru) ;
		nSoil:long_name = number of soil layers (instant) ;
		nSoil:units = - ;
	int32 nLayers(time, hru) ;
		nLayers:long_name = total number of layers (instant) ;
		nLayers:units = - ;

// global attributes:
	:summaVersion = v2.0.0 ;
	:buildTime = Mon Mar 26 17:11:28 MDT 2018 ;
	:gitBranch = develop-0-gcbbec4a ;
	:gitHash = cbbec4a639446a285366339f0a2a9680c7df1ed4 ;
	:soilStress = NoahType ;
	:stomResist = BallBerry ;
	:num_method = itertive ;
	:fDerivMeth = analytic ;
	:LAI_method = monTable ;
	:f_Richards = mixdform ;
	:groundwatr = bigBuckt ;
	:hc_profile = constant ;
	:bcUpprTdyn = nrg_flux ;
	:bcLowrTdyn = zeroFlux ;
	:bcUpprSoiH = liq_flux ;
	:bcLowrSoiH = drainage ;
	:veg_traits = CM_QJRMS1988 ;
	:canopyEmis = difTrans ;
	:snowIncept = lightSnow ;
	:windPrfile = logBelowCanopy ;
	:astability = louisinv ;
	:canopySrad = BeersLaw ;
	:alb_method = varDecay ;
	:snowLayers = CLM_2010 ;
	:compaction = anderson ;
	:thCondSnow = jrdn1991 ;
	:thCondSoil = funcSoilWet ;
	:spatial_gw = localColumn ;
	:subRouting = timeDlay ;
	:notPopulatedYet = notPopulatedYet ;
}None

[shoyer]

I'm not entirely sure, but it may be that installing newer versions of numpy or pandas would fix this issue.

[msaharia]

Arrgh. You're right. Updated Pandas and Numpy and the error went away. Thanks a lot!

[shoyer]

This is definitely still a bug (we shouldn't randomly fail with older but supported versions of our dependencies), but it will probably be relatively low priority given there's a simple work-around...