NARR: Subsetting and OPeNDAP
Contents
NARR: Subsetting and OPeNDAP#
About the Dataset#
This tutorial uses data from NOAA’s North American Regional Reanalysis (NARR)
The North American Regional Reanalysis (NARR) is a model produced by the National Centers for Environmental Prediction (NCEP) that generates reanalyzed data for temperature, wind, moisture, soil, and dozens of other parameters. The NARR model assimilates a large amount of observational data from a variety of sources to produce a long-term picture of weather over North America.
For this recipe, we will access the data via OPeNDAP, a widely-used API for remote access of environmental data over HTTP. A key point is that, since we use using OPeNDAP, there are no input files to download / cache. We open the data directly from the remote server.
The data we will use are catalogged here (3D data on pressure levels): https://psl.noaa.gov/thredds/catalog/Datasets/NARR/pressure/catalog.html
Let’s peek at one file. Xarray should automatically do the right thing with the OPeNDAP url. But just to be safe, we can pass the option, engine='netcdf4', which is needed to open OPeNDAP links correctly. (We will need this again later when writing our recipe.)
import xarray as xr
url = "https://psl.noaa.gov/thredds/dodsC/Datasets/NARR/pressure/air.197901.nc"
ds = xr.open_dataset(url, engine='netcdf4')
ds
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
<xarray.Dataset>
Dimensions: (time: 248, level: 29, y: 277, x: 349)
Coordinates:
* time (time) datetime64[ns] 1979-01-01 ... 1979-01-31T21:00:00
* level (level) float32 1e+03 975.0 950.0 ... 150.0 125.0 100.0
* y (y) float32 0.0 3.246e+04 ... 8.927e+06 8.96e+06
* x (x) float32 0.0 3.246e+04 ... 1.126e+07 1.13e+07
lat (y, x) float32 1.0 1.104 1.208 ... 46.93 46.64 46.35
lon (y, x) float32 -145.5 -145.3 -145.1 ... -2.644 -2.57
Data variables:
Lambert_Conformal int32 -2147483647
air (time, level, y, x) float32 ...
Attributes: (12/17)
Conventions: CF-1.2
centerlat: 50.0
centerlon: -107.0
comments:
institution: National Centers for Environmental Predi...
latcorners: [ 1.000001 0.897945 46.3544 46.63433 ]
... ...
history: created Fri Jan 8 02:35:07 MST 2016 by ...
dataset_title: NCEP North American Regional Reanalysis ...
references: https://www.esrl.noaa.gov/psd/data/gridd...
source: http://www.emc.ncep.noaa.gov/mmb/rreanl/...
References:
DODS_EXTRA.Unlimited_Dimension: timeThis is just one file.
But it’s a very big file (several GB)!
We will want to use the subsetting capability of XarrayZarrRecipe.
To figure out how to subset, let’s examine the _ChunkSizes attribute.
This attribute part of the NetCDF4 files that the OPeNDAP server is serving.
We want to make sure our subsetting is compatible with the internal chunking of the files.
ds.air._ChunkSizes
array([ 1, 1, 277, 349], dtype=int32)
This tells us that we can subset in the time or level dimensions, but problably should avoid subsetting in the x and y dimensions.
Define File Pattern#
We are now ready to define the FilePattern for the recipe. There is one file per month. So we start with a function like this:
def format_function(time):
return f"https://psl.noaa.gov/thredds/dodsC/Datasets/NARR/pressure/air.{time}.nc"
To keep things short and simple for this tutorial, we will just use one file, and subset it into many chunks.
But we could easily add more months to build up the entire dataset.
Since each file is monthly, and the number of days per months varies, we cannot set nitems_per_input in the ConcatDim.
Note
It’s important that we specify file_type="opendap" when creating a FilePattern with OPeNDAP URLs.
OPeNDAP is actually an API, so there are no files to download.
from pangeo_forge_recipes.patterns import FilePattern, ConcatDim, MergeDim
time_dim = ConcatDim("time", ["197901"])
pattern = FilePattern(format_function, time_dim, file_type="opendap")
pattern
<FilePattern {'time': 1}>
Define the Recipe#
This tutorial introduces a new option we have not seen before: subset_inpus={"time": 30}. This tells the XarrayZarrRecipe to break each input file into 30 distinct chunks along the time dimension.
Some other points to consider when defining this recipe.
Since the inputs are OPeNDAP, we want to use
file_type="opendap".Since we have not specified
nitems_per_inputin theFilePattern, we need to explicitly set the target chunks.
from pangeo_forge_recipes.recipes import XarrayZarrRecipe
recipe = XarrayZarrRecipe(
pattern,
subset_inputs={"time": 30},
target_chunks={"time": 1},
)
recipe
XarrayZarrRecipe(file_pattern=<FilePattern {'time': 1}>, storage_config=StorageConfig(target=FSSpecTarget(fs=<fsspec.implementations.local.LocalFileSystem object at 0x7f82c8703ee0>, root_path='/var/folders/tt/4f941hdn0zq549zdwhcgg98c0000gn/T/tmpiwh5o7jj/N53XYGfA'), cache=CacheFSSpecTarget(fs=<fsspec.implementations.local.LocalFileSystem object at 0x7f82c8703ee0>, root_path='/var/folders/tt/4f941hdn0zq549zdwhcgg98c0000gn/T/tmpiwh5o7jj/N6C32mGA'), metadata=MetadataTarget(fs=<fsspec.implementations.local.LocalFileSystem object at 0x7f82c8703ee0>, root_path='/var/folders/tt/4f941hdn0zq549zdwhcgg98c0000gn/T/tmpiwh5o7jj/uj3HYDU8')), inputs_per_chunk=1, target_chunks={'time': 1}, cache_inputs=False, copy_input_to_local_file=False, consolidate_zarr=True, consolidate_dimension_coordinates=True, xarray_open_kwargs={'engine': 'netcdf4'}, xarray_concat_kwargs={}, delete_input_encoding=True, process_input=None, process_chunk=None, lock_timeout=None, subset_inputs={'time': 30}, open_input_with_kerchunk=False)
Run the Recipe#
# logging will display some interesting information about our recipe during execution
from pangeo_forge_recipes.recipes import setup_logging
setup_logging()
Here we just use the simple python function executor.
recipe.to_function()()
[05/10/22 17:48:40] INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
INFO Caching metadata for input xarray_zarr.py:167 'Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
//pangeo-forge-recipes/pangeo_forge_recipes/recipes/xarray_zarr.py:115: RuntimeWarning: Failed to open Zarr store with consolidated metadata, falling back to try reading non-consolidated metadata. This is typically much slower for opening a dataset. To silence this warning, consider:
1. Consolidating metadata in this existing store with zarr.consolidate_metadata().
2. Explicitly setting consolidated=False, to avoid trying to read consolidate metadata, or
3. Explicitly setting consolidated=True, to raise an error in this case instead of falling back to try reading non-consolidated metadata.
return xr.open_zarr(target.get_mapper())
INFO Creating a new dataset in target xarray_zarr.py:511
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=0, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:48:41] INFO Subsetting input according to time-0 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=0, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
[05/10/22 17:48:44] INFO Storing dataset in /var/folders/tt/4f941hdn0z xarray_zarr.py:553 q549zdwhcgg98c0000gn/T/tmpiwh5o7jj/N53XYGfA
INFO Expanding target concat dim 'time' to size xarray_zarr.py:569 248
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=0, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:48:45] INFO Subsetting input according to time-0 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=0, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=0, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(0, 8, None),)
[05/10/22 17:49:02] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=0, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(0, 8, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=1, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:49:03] INFO Subsetting input according to time-1 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=1, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=1, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(8, 16, None),)
[05/10/22 17:49:29] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=1, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(8, 16, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=2, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:49:30] INFO Subsetting input according to time-2 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=2, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=2, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(16, 24, None),)
[05/10/22 17:49:34] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=2, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(16, 24, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=3, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:49:35] INFO Subsetting input according to time-3 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=3, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=3, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(24, 32, None),)
[05/10/22 17:49:41] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=3, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(24, 32, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=4, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:49:42] INFO Subsetting input according to time-4 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=4, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=4, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(32, 40, None),)
[05/10/22 17:49:46] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=4, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(32, 40, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=5, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:49:47] INFO Subsetting input according to time-5 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=5, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=5, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(40, 48, None),)
[05/10/22 17:49:52] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=5, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(40, 48, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=6, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
INFO Subsetting input according to time-6 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=6, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=6, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(48, 56, None),)
[05/10/22 17:49:57] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=6, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(48, 56, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=7, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:49:58] INFO Subsetting input according to time-7 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=7, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=7, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(56, 64, None),)
[05/10/22 17:50:16] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=7, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(56, 64, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=8, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:50:17] INFO Subsetting input according to time-8 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=8, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=8, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(64, 72, None),)
[05/10/22 17:50:50] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=8, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(64, 72, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=9, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:50:51] INFO Subsetting input according to time-9 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=9, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=9, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(72, 80, None),)
[05/10/22 17:50:56] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=9, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(72, 80, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
[05/10/22 17:50:57] INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=10, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
INFO Subsetting input according to time-10 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=10, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=10, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(80, 88, None),)
[05/10/22 17:51:15] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=10, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(80, 88, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=11, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:51:16] INFO Subsetting input according to time-11 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=11, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=11, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(88, 96, None),)
[05/10/22 17:51:34] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=11, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(88, 96, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=12, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
INFO Subsetting input according to time-12 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=12, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=12, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(96, 104, None),)
[05/10/22 17:51:59] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=12, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(96, 104, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=13, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
INFO Subsetting input according to time-13 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=13, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
[05/10/22 17:52:00] INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=13, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(104, 112, None),)
[05/10/22 17:52:17] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=13, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(104, 112, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=14, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:52:18] INFO Subsetting input according to time-14 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=14, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=14, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(112, 120, None),)
[05/10/22 17:52:36] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=14, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(112, 120, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=15, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
INFO Subsetting input according to time-15 xarray_zarr.py:402
[05/10/22 17:52:37] INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=15, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=15, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(120, 128, None),)
[05/10/22 17:52:54] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=15, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(120, 128, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=16, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:52:55] INFO Subsetting input according to time-16 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=16, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=16, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(128, 136, None),)
[05/10/22 17:53:13] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=16, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(128, 136, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=17, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
INFO Subsetting input according to time-17 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=17, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=17, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(136, 144, None),)
[05/10/22 17:53:31] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=17, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(136, 144, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=18, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:53:32] INFO Subsetting input according to time-18 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=18, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=18, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(144, 152, None),)
[05/10/22 17:53:37] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=18, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(144, 152, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=19, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:53:38] INFO Subsetting input according to time-19 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=19, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=19, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(152, 160, None),)
[05/10/22 17:53:56] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=19, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(152, 160, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=20, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:53:57] INFO Subsetting input according to time-20 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=20, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=20, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(160, 168, None),)
[05/10/22 17:54:02] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=20, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(160, 168, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=21, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
INFO Subsetting input according to time-21 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=21, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=21, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(168, 176, None),)
[05/10/22 17:54:07] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=21, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(168, 176, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=22, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:54:08] INFO Subsetting input according to time-22 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=22, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=22, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(176, 184, None),)
[05/10/22 17:54:12] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=22, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(176, 184, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=23, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:54:13] INFO Subsetting input according to time-23 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=23, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=23, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(184, 192, None),)
[05/10/22 17:54:20] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=23, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(184, 192, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=24, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:54:21] INFO Subsetting input according to time-24 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=24, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=24, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(192, 200, None),)
[05/10/22 17:54:41] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=24, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(192, 200, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=25, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:54:42] INFO Subsetting input according to time-25 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=25, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=25, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(200, 208, None),)
[05/10/22 17:54:47] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=25, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(200, 208, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=26, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
INFO Subsetting input according to time-26 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=26, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=26, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(208, 216, None),)
[05/10/22 17:55:04] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=26, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(208, 216, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
[05/10/22 17:55:05] INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=27, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
INFO Subsetting input according to time-27 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=27, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=27, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(216, 224, None),)
[05/10/22 17:55:23] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=27, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(216, 224, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=28, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:55:24] INFO Subsetting input according to time-28 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=28, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=28, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(224, 232, None),)
[05/10/22 17:55:41] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=28, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(224, 232, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
INFO Opening inputs for chunk xarray_zarr.py:390 Index({DimIndex(name='time', index=29, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:253 Index({DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}): 'https://psl.noaa.gov/thredds/dodsC/Da tasets/NARR/pressure/air.197901.nc'
/envs/pangeo-forge-recipes/lib/python3.8/site-packages/xarray/conventions.py:516: SerializationWarning: variable 'air' has multiple fill values {9.96921e+36, -9.96921e+36}, decoding all values to NaN.
new_vars[k] = decode_cf_variable(
[05/10/22 17:55:42] INFO Subsetting input according to time-29 xarray_zarr.py:402
INFO Combining inputs for chunk xarray_zarr.py:408 'Index({DimIndex(name='time', index=29, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=29, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(232, 248, None),)
[05/10/22 17:56:12] INFO Storing variable air chunk xarray_zarr.py:632 Index({DimIndex(name='time', index=29, sequence_len=30, operation=<CombineOp.SUBSET: 3>), DimIndex(name='time', index=0, sequence_len=1, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(232, 248, None), slice(None, None, None), slice(None, None, None), slice(None, None, None))
[05/10/22 17:56:13] INFO Consolidating dimension coordinate arrays xarray_zarr.py:649
INFO Consolidating Zarr metadata xarray_zarr.py:673
Check The Outputs#
ds_target = xr.open_zarr(recipe.target_mapper)
ds_target
<xarray.Dataset>
Dimensions: (time: 248, level: 29, y: 277, x: 349)
Coordinates:
lat (y, x) float32 dask.array<chunksize=(277, 349), meta=np.ndarray>
* level (level) float32 1e+03 975.0 950.0 ... 150.0 125.0 100.0
lon (y, x) float32 dask.array<chunksize=(277, 349), meta=np.ndarray>
* time (time) datetime64[ns] 1979-01-01 ... 1979-01-31T21:00:00
* x (x) float32 0.0 3.246e+04 ... 1.126e+07 1.13e+07
* y (y) float32 0.0 3.246e+04 ... 8.927e+06 8.96e+06
Data variables:
Lambert_Conformal int32 ...
air (time, level, y, x) float32 dask.array<chunksize=(1, 29, 277, 349), meta=np.ndarray>
Attributes: (12/17)
Conventions: CF-1.2
DODS_EXTRA.Unlimited_Dimension: time
References:
centerlat: 50.0
centerlon: -107.0
comments:
... ...
platform: Model
references: https://www.esrl.noaa.gov/psd/data/gridd...
source: http://www.emc.ncep.noaa.gov/mmb/rreanl/...
standardpar1: 50.0
standardpar2: 50.000001
title: 8x Daily NARRds_target.air.isel(level=0).mean("time").plot()
<matplotlib.collections.QuadMesh at 0x7f82e90ba370>
