Complex NetCDF to Zarr Recipe: TerraClimate
Contents
Complex NetCDF to Zarr Recipe: TerraClimate#
About the Dataset#
From http://www.climatologylab.org/terraclimate.html:
TerraClimate is a dataset of monthly climate and climatic water balance for global terrestrial surfaces from 1958-2019. These data provide important inputs for ecological and hydrological studies at global scales that require high spatial resolution and time-varying data. All data have monthly temporal resolution and a ~4-km (1/24th degree) spatial resolution. The data cover the period from 1958-2019. We plan to update these data periodically (annually).
What makes it tricky#
This is an advanced example that illustrates the following concepts
Multiple variables in different files: There is one file per year for a dozen different variables.
Complex Preprocessing: We want to apply different preprocessing depending on the variable. This example shows how.
Inconsistent size of data in input files: This means we have to scan each input file and cache its metadata before we can start writing the target.
This recipe requires a new storage target, a metadata_cache. In this example, this is just another directory. You could hypothetically use a database or other key/value store for this.
from pangeo_forge_recipes.recipes import XarrayZarrRecipe
from pangeo_forge_recipes.patterns import FilePattern, ConcatDim, MergeDim
import xarray as xr
Define Filename Pattern#
To keep this example smaller, we just use two years instead of the whole record.
target_chunks = {"lat": 1024, "lon": 1024, "time": 12}
# only do two years to keep the example small; it's still big!
years = list(range(1958, 1960))
variables = [
"aet",
"def",
"pet",
"ppt",
"q",
"soil",
"srad",
"swe",
"tmax",
"tmin",
"vap",
"ws",
"vpd",
"PDSI",
]
def make_filename(variable, time):
return f"http://thredds.northwestknowledge.net:8080/thredds/fileServer/TERRACLIMATE_ALL/data/TerraClimate_{variable}_{time}.nc"
pattern = FilePattern(
make_filename,
ConcatDim(name="time", keys=years),
MergeDim(name="variable", keys=variables)
)
pattern
<FilePattern {'time': 2, 'variable': 14}>
Check out the pattern:
for key, filename in pattern.items():
break
key, filename
(frozenset({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>),
DimIndex(name='variable', index=0, sequence_len=14, operation=<CombineOp.MERGE: 1>)}),
'http://thredds.northwestknowledge.net:8080/thredds/fileServer/TERRACLIMATE_ALL/data/TerraClimate_aet_1958.nc')
Define Preprocessing Functions#
These functions apply masks for each variable to remove invalid data.
rename_vars = {'PDSI': 'pdsi'}
mask_opts = {
"PDSI": ("lt", 10),
"aet": ("lt", 32767),
"def": ("lt", 32767),
"pet": ("lt", 32767),
"ppt": ("lt", 32767),
"ppt_station_influence": None,
"q": ("lt", 2147483647),
"soil": ("lt", 32767),
"srad": ("lt", 32767),
"swe": ("lt", 10000),
"tmax": ("lt", 200),
"tmax_station_influence": None,
"tmin": ("lt", 200),
"tmin_station_influence": None,
"vap": ("lt", 300),
"vap_station_influence": None,
"vpd": ("lt", 300),
"ws": ("lt", 200),
}
def apply_mask(key, da):
"""helper function to mask DataArrays based on a threshold value"""
if mask_opts.get(key, None):
op, val = mask_opts[key]
if op == "lt":
da = da.where(da < val)
elif op == "neq":
da = da.where(da != val)
return da
def preproc(ds):
"""custom preprocessing function for terraclimate data"""
rename = {}
station_influence = ds.get("station_influence", None)
if station_influence is not None:
ds = ds.drop_vars("station_influence")
var = list(ds.data_vars)[0]
if var in rename_vars:
rename[var] = rename_vars[var]
if "day" in ds.coords:
rename["day"] = "time"
if station_influence is not None:
ds[f"{var}_station_influence"] = station_influence
with xr.set_options(keep_attrs=True):
ds[var] = apply_mask(var, ds[var])
if rename:
ds = ds.rename(rename)
return ds
Define Recipe#
We are now ready to define the recipe. We also specify the desired chunks of the target dataset.
A key property of this recipe is nitems_per_input=None, which triggers caching of input metadata.
chunks = {"lat": 1024, "lon": 1024, "time": 12}
recipe = XarrayZarrRecipe(
file_pattern=pattern,
target_chunks=target_chunks,
process_chunk=preproc
)
recipe
XarrayZarrRecipe(file_pattern=<FilePattern {'time': 2, 'variable': 14}>, storage_config=StorageConfig(target=FSSpecTarget(fs=<fsspec.implementations.local.LocalFileSystem object at 0x179047400>, root_path='/var/folders/tt/4f941hdn0zq549zdwhcgg98c0000gn/T/tmpquyof0ta/zhCkRV0Z'), cache=CacheFSSpecTarget(fs=<fsspec.implementations.local.LocalFileSystem object at 0x179047400>, root_path='/var/folders/tt/4f941hdn0zq549zdwhcgg98c0000gn/T/tmpquyof0ta/TUgR0g5v'), metadata=MetadataTarget(fs=<fsspec.implementations.local.LocalFileSystem object at 0x179047400>, root_path='/var/folders/tt/4f941hdn0zq549zdwhcgg98c0000gn/T/tmpquyof0ta/CpxAUJnB')), inputs_per_chunk=1, target_chunks={'lat': 1024, 'lon': 1024, 'time': 12}, cache_inputs=True, copy_input_to_local_file=False, consolidate_zarr=True, consolidate_dimension_coordinates=True, xarray_open_kwargs={}, xarray_concat_kwargs={}, delete_input_encoding=True, process_input=None, process_chunk=<function preproc at 0x1790443a0>, lock_timeout=None, subset_inputs={}, open_input_with_fsspec_reference=False)
Execute with Prefect#
This produces A LOT of output because we turn on logging.
# logging will display some interesting information about our recipe during execution
from pangeo_forge_recipes.recipes import setup_logging
setup_logging()
flow = recipe.to_prefect()
flow.run()
[2022-03-10 11:19:13-0800] INFO - prefect.FlowRunner | Beginning Flow run for 'pangeo-forge-recipe'
[2022-03-10 11:19:13-0800] INFO - prefect.TaskRunner | Task 'cache_input': Starting task run...
[2022-03-10 11:19:13-0800] INFO - prefect.TaskRunner | Task 'cache_input': Finished task run for task with final state: 'Mapped'
[2022-03-10 11:19:13-0800] INFO - prefect.TaskRunner | Task 'cache_input[0]': Starting task run...
[03/10/22 11:19:13] INFO Caching input 'Index({DimIndex(name='time', xarray_zarr.py:149 index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=0, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_aet_1958.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_aet_1958.nc' to cache
[03/10/22 11:19:19] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=0, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_aet_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_aet_1958.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=0, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
[2022-03-10 11:19:19-0800] INFO - prefect.TaskRunner | Task 'cache_input[0]': Finished task run for task with final state: 'Success'
[2022-03-10 11:19:19-0800] INFO - prefect.TaskRunner | Task 'cache_input[1]': Starting task run...
INFO Caching input 'Index({DimIndex(name='time', xarray_zarr.py:149 index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=1, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_def_1958.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_def_1958.nc' to cache
[03/10/22 11:19:24] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=1, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_def_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_def_1958.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=1, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
[2022-03-10 11:19:24-0800] INFO - prefect.TaskRunner | Task 'cache_input[1]': Finished task run for task with final state: 'Success'
[2022-03-10 11:19:24-0800] INFO - prefect.TaskRunner | Task 'cache_input[2]': Starting task run...
INFO Caching input 'Index({DimIndex(name='time', xarray_zarr.py:149 index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=2, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_pet_1958.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_pet_1958.nc' to cache
[03/10/22 11:19:30] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=2, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_pet_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_pet_1958.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=2, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
[2022-03-10 11:19:30-0800] INFO - prefect.TaskRunner | Task 'cache_input[2]': Finished task run for task with final state: 'Success'
[2022-03-10 11:19:30-0800] INFO - prefect.TaskRunner | Task 'cache_input[3]': Starting task run...
INFO Caching input 'Index({DimIndex(name='time', xarray_zarr.py:149 index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=3, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_ppt_1958.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_ppt_1958.nc' to cache
[03/10/22 11:19:37] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=3, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_ppt_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_ppt_1958.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=3, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
[2022-03-10 11:19:37-0800] INFO - prefect.TaskRunner | Task 'cache_input[3]': Finished task run for task with final state: 'Success'
[2022-03-10 11:19:37-0800] INFO - prefect.TaskRunner | Task 'cache_input[4]': Starting task run...
INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=4, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_q_1958.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_q_1958.nc' to cache
[03/10/22 11:19:41] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=4, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_q_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_q_1958.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=4, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:19:41-0800] INFO - prefect.TaskRunner | Task 'cache_input[4]': Finished task run for task with final state: 'Success'
[2022-03-10 11:19:41-0800] INFO - prefect.TaskRunner | Task 'cache_input[5]': Starting task run...
INFO Caching input 'Index({DimIndex(name='time', xarray_zarr.py:149 index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=5, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_soil_1958.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_soil_1958.nc' to cache
[03/10/22 11:19:48] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=5, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_soil_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_soil_1958.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=5, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
[2022-03-10 11:19:48-0800] INFO - prefect.TaskRunner | Task 'cache_input[5]': Finished task run for task with final state: 'Success'
[2022-03-10 11:19:48-0800] INFO - prefect.TaskRunner | Task 'cache_input[6]': Starting task run...
INFO Caching input 'Index({DimIndex(name='time', xarray_zarr.py:149 index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=6, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_srad_1958.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_srad_1958.nc' to cache
[03/10/22 11:19:56] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=6, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_srad_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_srad_1958.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=6, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
[2022-03-10 11:19:56-0800] INFO - prefect.TaskRunner | Task 'cache_input[6]': Finished task run for task with final state: 'Success'
[2022-03-10 11:19:56-0800] INFO - prefect.TaskRunner | Task 'cache_input[7]': Starting task run...
INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=7, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_swe_1958.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_swe_1958.nc' to cache
[03/10/22 11:20:02] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=7, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_swe_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_swe_1958.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=7, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:20:02-0800] INFO - prefect.TaskRunner | Task 'cache_input[7]': Finished task run for task with final state: 'Success'
[2022-03-10 11:20:02-0800] INFO - prefect.TaskRunner | Task 'cache_input[8]': Starting task run...
INFO Caching input 'Index({DimIndex(name='time', xarray_zarr.py:149 index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=8, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_tmax_1958.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_tmax_1958.nc' to cache
[03/10/22 11:20:14] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=8, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_tmax_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_tmax_1958.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=8, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
[2022-03-10 11:20:14-0800] INFO - prefect.TaskRunner | Task 'cache_input[8]': Finished task run for task with final state: 'Success'
[2022-03-10 11:20:14-0800] INFO - prefect.TaskRunner | Task 'cache_input[9]': Starting task run...
INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=9, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_tmin_1958.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_tmin_1958.nc' to cache
[03/10/22 11:20:25] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=9, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_tmin_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_tmin_1958.nc' from cache
[03/10/22 11:20:26] INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=9, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:20:26-0800] INFO - prefect.TaskRunner | Task 'cache_input[9]': Finished task run for task with final state: 'Success'
[2022-03-10 11:20:26-0800] INFO - prefect.TaskRunner | Task 'cache_input[10]': Starting task run...
INFO Caching input 'Index({DimIndex(name='time', xarray_zarr.py:149 index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=10, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_vap_1958.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_vap_1958.nc' to cache
[03/10/22 11:20:33] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=10, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_vap_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_vap_1958.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=10, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
[2022-03-10 11:20:33-0800] INFO - prefect.TaskRunner | Task 'cache_input[10]': Finished task run for task with final state: 'Success'
[2022-03-10 11:20:33-0800] INFO - prefect.TaskRunner | Task 'cache_input[11]': Starting task run...
INFO Caching input 'Index({DimIndex(name='time', xarray_zarr.py:149 index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=11, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_ws_1958.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_ws_1958.nc' to cache
[03/10/22 11:20:40] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=11, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_ws_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_ws_1958.nc' from cache
[03/10/22 11:20:41] INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=11, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
[2022-03-10 11:20:41-0800] INFO - prefect.TaskRunner | Task 'cache_input[11]': Finished task run for task with final state: 'Success'
[2022-03-10 11:20:41-0800] INFO - prefect.TaskRunner | Task 'cache_input[12]': Starting task run...
INFO Caching input 'Index({DimIndex(name='time', xarray_zarr.py:149 index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=12, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_vpd_1958.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_vpd_1958.nc' to cache
[03/10/22 11:20:45] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=12, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_vpd_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_vpd_1958.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=12, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
[2022-03-10 11:20:45-0800] INFO - prefect.TaskRunner | Task 'cache_input[12]': Finished task run for task with final state: 'Success'
[2022-03-10 11:20:45-0800] INFO - prefect.TaskRunner | Task 'cache_input[13]': Starting task run...
INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=13, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_PDSI_1958.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_PDSI_1958.nc' to cache
[03/10/22 11:20:53] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=13, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_PDSI_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_PDSI_1958.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=13, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:20:53-0800] INFO - prefect.TaskRunner | Task 'cache_input[13]': Finished task run for task with final state: 'Success'
[2022-03-10 11:20:53-0800] INFO - prefect.TaskRunner | Task 'cache_input[14]': Starting task run...
INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=0, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_aet_1959.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_aet_1959.nc' to cache
[03/10/22 11:20:58] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=0, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_aet_1959.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_aet_1959.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=0, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:20:58-0800] INFO - prefect.TaskRunner | Task 'cache_input[14]': Finished task run for task with final state: 'Success'
[2022-03-10 11:20:58-0800] INFO - prefect.TaskRunner | Task 'cache_input[15]': Starting task run...
INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=1, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_def_1959.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_def_1959.nc' to cache
[03/10/22 11:21:03] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=1, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_def_1959.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_def_1959.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=1, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:21:03-0800] INFO - prefect.TaskRunner | Task 'cache_input[15]': Finished task run for task with final state: 'Success'
[2022-03-10 11:21:03-0800] INFO - prefect.TaskRunner | Task 'cache_input[16]': Starting task run...
INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=2, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_pet_1959.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_pet_1959.nc' to cache
[03/10/22 11:21:09] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=2, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_pet_1959.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_pet_1959.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=2, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:21:09-0800] INFO - prefect.TaskRunner | Task 'cache_input[16]': Finished task run for task with final state: 'Success'
[2022-03-10 11:21:10-0800] INFO - prefect.TaskRunner | Task 'cache_input[17]': Starting task run...
[03/10/22 11:21:10] INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=3, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_ppt_1959.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_ppt_1959.nc' to cache
[03/10/22 11:21:17] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=3, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_ppt_1959.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_ppt_1959.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=3, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:21:17-0800] INFO - prefect.TaskRunner | Task 'cache_input[17]': Finished task run for task with final state: 'Success'
[2022-03-10 11:21:17-0800] INFO - prefect.TaskRunner | Task 'cache_input[18]': Starting task run...
INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=4, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_q_1959.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_q_1959.nc' to cache
[03/10/22 11:21:20] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=4, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_q_1959.nc'
[03/10/22 11:21:21] INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_q_1959.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=4, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:21:21-0800] INFO - prefect.TaskRunner | Task 'cache_input[18]': Finished task run for task with final state: 'Success'
[2022-03-10 11:21:21-0800] INFO - prefect.TaskRunner | Task 'cache_input[19]': Starting task run...
INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=5, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_soil_1959.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_soil_1959.nc' to cache
[03/10/22 11:21:27] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=5, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_soil_1959.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_soil_1959.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=5, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:21:27-0800] INFO - prefect.TaskRunner | Task 'cache_input[19]': Finished task run for task with final state: 'Success'
[2022-03-10 11:21:27-0800] INFO - prefect.TaskRunner | Task 'cache_input[20]': Starting task run...
INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=6, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_srad_1959.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_srad_1959.nc' to cache
[03/10/22 11:21:36] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=6, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_srad_1959.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_srad_1959.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=6, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:21:36-0800] INFO - prefect.TaskRunner | Task 'cache_input[20]': Finished task run for task with final state: 'Success'
[2022-03-10 11:21:36-0800] INFO - prefect.TaskRunner | Task 'cache_input[21]': Starting task run...
INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=7, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_swe_1959.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_swe_1959.nc' to cache
[03/10/22 11:21:42] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=7, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_swe_1959.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_swe_1959.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=7, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:21:42-0800] INFO - prefect.TaskRunner | Task 'cache_input[21]': Finished task run for task with final state: 'Success'
[2022-03-10 11:21:42-0800] INFO - prefect.TaskRunner | Task 'cache_input[22]': Starting task run...
INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=8, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_tmax_1959.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_tmax_1959.nc' to cache
[03/10/22 11:21:53] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=8, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_tmax_1959.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_tmax_1959.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=8, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:21:53-0800] INFO - prefect.TaskRunner | Task 'cache_input[22]': Finished task run for task with final state: 'Success'
[2022-03-10 11:21:53-0800] INFO - prefect.TaskRunner | Task 'cache_input[23]': Starting task run...
INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=9, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_tmin_1959.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_tmin_1959.nc' to cache
[03/10/22 11:22:05] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=9, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_tmin_1959.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_tmin_1959.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=9, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:22:05-0800] INFO - prefect.TaskRunner | Task 'cache_input[23]': Finished task run for task with final state: 'Success'
[2022-03-10 11:22:05-0800] INFO - prefect.TaskRunner | Task 'cache_input[24]': Starting task run...
INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=10, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_vap_1959.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_vap_1959.nc' to cache
[03/10/22 11:22:13] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=10, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_vap_1959.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_vap_1959.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=10, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:22:13-0800] INFO - prefect.TaskRunner | Task 'cache_input[24]': Finished task run for task with final state: 'Success'
[2022-03-10 11:22:13-0800] INFO - prefect.TaskRunner | Task 'cache_input[25]': Starting task run...
INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=11, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_ws_1959.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_ws_1959.nc' to cache
[03/10/22 11:22:21] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=11, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_ws_1959.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_ws_1959.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=11, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:22:21-0800] INFO - prefect.TaskRunner | Task 'cache_input[25]': Finished task run for task with final state: 'Success'
[2022-03-10 11:22:21-0800] INFO - prefect.TaskRunner | Task 'cache_input[26]': Starting task run...
INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=12, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_vpd_1959.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_vpd_1959.nc' to cache
[03/10/22 11:22:24] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=12, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_vpd_1959.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_vpd_1959.nc' from cache
[03/10/22 11:22:25] INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=12, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:22:25-0800] INFO - prefect.TaskRunner | Task 'cache_input[26]': Finished task run for task with final state: 'Success'
[2022-03-10 11:22:25-0800] INFO - prefect.TaskRunner | Task 'cache_input[27]': Starting task run...
INFO Caching input xarray_zarr.py:149 'Index({DimIndex(name='variable', index=13, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Caching file 'http://thredds.northwestknowledge.n storage.py:154 et:8080/thredds/fileServer/TERRACLIMATE_ALL/data/ TerraClimate_PDSI_1959.nc'
INFO Copying remote file 'http://thredds.northwestknow storage.py:165 ledge.net:8080/thredds/fileServer/TERRACLIMATE_AL L/data/TerraClimate_PDSI_1959.nc' to cache
[03/10/22 11:22:33] INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=13, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_PDSI_1959.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_PDSI_1959.nc' from cache
INFO Caching metadata for input xarray_zarr.py:163 'Index({DimIndex(name='variable', index=13, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=1, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
[2022-03-10 11:22:33-0800] INFO - prefect.TaskRunner | Task 'cache_input[27]': Finished task run for task with final state: 'Success'
[2022-03-10 11:22:33-0800] INFO - prefect.TaskRunner | Task 'prepare_target': Starting task run...
/Users//Dropbox/pangeo/pangeo-forge-recipes/pangeo_forge_recipes/recipes/xarray_zarr.py:111: RuntimeWarning: Failed to open Zarr store with consolidated metadata, falling back to try reading non-consolidated metadata. This is typically much slower for opening a dataset. To silence this warning, consider:
1. Consolidating metadata in this existing store with zarr.consolidate_metadata().
2. Explicitly setting consolidated=False, to avoid trying to read consolidate metadata, or
3. Explicitly setting consolidated=True, to raise an error in this case instead of falling back to try reading non-consolidated metadata.
return xr.open_zarr(target.get_mapper())
INFO Creating a new dataset in target xarray_zarr.py:504
INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=0, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=0, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_aet_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_aet_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=0, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing dataset in /var/folders/tt/4f941hdn0z xarray_zarr.py:546 q549zdwhcgg98c0000gn/T/tmpquyof0ta/zhCkRV0Z
[03/10/22 11:22:34] INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=1, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=1, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_def_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_def_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=1, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing dataset in /var/folders/tt/4f941hdn0z xarray_zarr.py:546 q549zdwhcgg98c0000gn/T/tmpquyof0ta/zhCkRV0Z
INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=2, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=2, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_pet_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_pet_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=2, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing dataset in /var/folders/tt/4f941hdn0z xarray_zarr.py:546 q549zdwhcgg98c0000gn/T/tmpquyof0ta/zhCkRV0Z
INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=3, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=3, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_ppt_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_ppt_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=3, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing dataset in /var/folders/tt/4f941hdn0z xarray_zarr.py:546 q549zdwhcgg98c0000gn/T/tmpquyof0ta/zhCkRV0Z
INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='variable', index=4, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=4, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_q_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_q_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='variable', index=4, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing dataset in /var/folders/tt/4f941hdn0z xarray_zarr.py:546 q549zdwhcgg98c0000gn/T/tmpquyof0ta/zhCkRV0Z
INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=5, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=5, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_soil_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_soil_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=5, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing dataset in /var/folders/tt/4f941hdn0z xarray_zarr.py:546 q549zdwhcgg98c0000gn/T/tmpquyof0ta/zhCkRV0Z
INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=6, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=6, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_srad_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_srad_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=6, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing dataset in /var/folders/tt/4f941hdn0z xarray_zarr.py:546 q549zdwhcgg98c0000gn/T/tmpquyof0ta/zhCkRV0Z
INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='variable', index=7, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=7, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_swe_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_swe_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='variable', index=7, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing dataset in /var/folders/tt/4f941hdn0z xarray_zarr.py:546 q549zdwhcgg98c0000gn/T/tmpquyof0ta/zhCkRV0Z
INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=8, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=8, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_tmax_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_tmax_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=8, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing dataset in /var/folders/tt/4f941hdn0z xarray_zarr.py:546 q549zdwhcgg98c0000gn/T/tmpquyof0ta/zhCkRV0Z
INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='variable', index=9, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=9, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_tmin_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_tmin_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='variable', index=9, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing dataset in /var/folders/tt/4f941hdn0z xarray_zarr.py:546 q549zdwhcgg98c0000gn/T/tmpquyof0ta/zhCkRV0Z
INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=10, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=10, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_vap_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_vap_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=10, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing dataset in /var/folders/tt/4f941hdn0z xarray_zarr.py:546 q549zdwhcgg98c0000gn/T/tmpquyof0ta/zhCkRV0Z
INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=11, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=11, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_ws_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_ws_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=11, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing dataset in /var/folders/tt/4f941hdn0z xarray_zarr.py:546 q549zdwhcgg98c0000gn/T/tmpquyof0ta/zhCkRV0Z
INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=12, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=12, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_vpd_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_vpd_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=12, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing dataset in /var/folders/tt/4f941hdn0z xarray_zarr.py:546 q549zdwhcgg98c0000gn/T/tmpquyof0ta/zhCkRV0Z
INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='variable', index=13, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=13, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_PDSI_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_PDSI_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='variable', index=13, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing dataset in /var/folders/tt/4f941hdn0z xarray_zarr.py:546 q549zdwhcgg98c0000gn/T/tmpquyof0ta/zhCkRV0Z
INFO Expanding target concat dim 'time' to size 24 xarray_zarr.py:560
[2022-03-10 11:22:35-0800] INFO - prefect.TaskRunner | Task 'prepare_target': Finished task run for task with final state: 'Success'
[2022-03-10 11:22:35-0800] INFO - prefect.TaskRunner | Task 'store_chunk': Starting task run...
[2022-03-10 11:22:35-0800] INFO - prefect.TaskRunner | Task 'store_chunk': Finished task run for task with final state: 'Mapped'
[2022-03-10 11:22:35-0800] INFO - prefect.TaskRunner | Task 'store_chunk[0]': Starting task run...
[03/10/22 11:22:35] INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=0, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=0, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_aet_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_aet_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=0, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing variable time chunk xarray_zarr.py:623 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=0, sequence_len=14, operation=<CombineOp.MERGE: 1>)}) to Zarr region (slice(0, 12, None),)
WARNING Variable aet of 1791590400 bytes is 3.58 xarray_zarr.py:602 times larger than specified maximum variable array size of 500000000 bytes. Consider re-instantiating recipe with `subset_inputs = {"time": 4}`. If `len(ds["time"])` < 4, substitute "time" for any name in ds["aet"].dims with length >= 4 or consider subsetting along multiple dimensions. Setting PANGEO_FORGE_MAX_MEMORY env variable changes the variable array size which will trigger this warning.
[03/10/22 11:22:42] INFO Storing variable aet chunk xarray_zarr.py:623 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=0, sequence_len=14, operation=<CombineOp.MERGE: 1>)}) to Zarr region (slice(0, 12, None), slice(None, None, None), slice(None, None, None))
[2022-03-10 11:22:43-0800] INFO - prefect.TaskRunner | Task 'store_chunk[0]': Finished task run for task with final state: 'Success'
[2022-03-10 11:22:43-0800] INFO - prefect.TaskRunner | Task 'store_chunk[1]': Starting task run...
[03/10/22 11:22:43] INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=1, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=1, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_def_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_def_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=1, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing variable time chunk xarray_zarr.py:623 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=1, sequence_len=14, operation=<CombineOp.MERGE: 1>)}) to Zarr region (slice(0, 12, None),)
WARNING Variable def of 1791590400 bytes is 3.58 xarray_zarr.py:602 times larger than specified maximum variable array size of 500000000 bytes. Consider re-instantiating recipe with `subset_inputs = {"time": 4}`. If `len(ds["time"])` < 4, substitute "time" for any name in ds["def"].dims with length >= 4 or consider subsetting along multiple dimensions. Setting PANGEO_FORGE_MAX_MEMORY env variable changes the variable array size which will trigger this warning.
[03/10/22 11:22:50] INFO Storing variable def chunk xarray_zarr.py:623 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=1, sequence_len=14, operation=<CombineOp.MERGE: 1>)}) to Zarr region (slice(0, 12, None), slice(None, None, None), slice(None, None, None))
[2022-03-10 11:22:51-0800] INFO - prefect.TaskRunner | Task 'store_chunk[1]': Finished task run for task with final state: 'Success'
[2022-03-10 11:22:51-0800] INFO - prefect.TaskRunner | Task 'store_chunk[2]': Starting task run...
[03/10/22 11:22:51] INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=2, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=2, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_pet_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_pet_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=2, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing variable time chunk xarray_zarr.py:623 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=2, sequence_len=14, operation=<CombineOp.MERGE: 1>)}) to Zarr region (slice(0, 12, None),)
WARNING Variable pet of 1791590400 bytes is 3.58 xarray_zarr.py:602 times larger than specified maximum variable array size of 500000000 bytes. Consider re-instantiating recipe with `subset_inputs = {"time": 4}`. If `len(ds["time"])` < 4, substitute "time" for any name in ds["pet"].dims with length >= 4 or consider subsetting along multiple dimensions. Setting PANGEO_FORGE_MAX_MEMORY env variable changes the variable array size which will trigger this warning.
[03/10/22 11:22:59] INFO Storing variable pet chunk xarray_zarr.py:623 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=2, sequence_len=14, operation=<CombineOp.MERGE: 1>)}) to Zarr region (slice(0, 12, None), slice(None, None, None), slice(None, None, None))
[2022-03-10 11:22:59-0800] INFO - prefect.TaskRunner | Task 'store_chunk[2]': Finished task run for task with final state: 'Success'
[2022-03-10 11:22:59-0800] INFO - prefect.TaskRunner | Task 'store_chunk[3]': Starting task run...
INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=3, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=3, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_ppt_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_ppt_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=3, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing variable time chunk xarray_zarr.py:623 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=3, sequence_len=14, operation=<CombineOp.MERGE: 1>)}) to Zarr region (slice(0, 12, None),)
WARNING Variable ppt of 3583180800 bytes is 7.17 xarray_zarr.py:602 times larger than specified maximum variable array size of 500000000 bytes. Consider re-instantiating recipe with `subset_inputs = {"time": 8}`. If `len(ds["time"])` < 8, substitute "time" for any name in ds["ppt"].dims with length >= 8 or consider subsetting along multiple dimensions. Setting PANGEO_FORGE_MAX_MEMORY env variable changes the variable array size which will trigger this warning.
[03/10/22 11:23:20] INFO Storing variable ppt chunk xarray_zarr.py:623 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=3, sequence_len=14, operation=<CombineOp.MERGE: 1>)}) to Zarr region (slice(0, 12, None), slice(None, None, None), slice(None, None, None))
[2022-03-10 11:23:21-0800] INFO - prefect.TaskRunner | Task 'store_chunk[3]': Finished task run for task with final state: 'Success'
[2022-03-10 11:23:21-0800] INFO - prefect.TaskRunner | Task 'store_chunk[4]': Starting task run...
[03/10/22 11:23:21] INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='variable', index=4, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=4, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_q_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_q_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='variable', index=4, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:623 Index({DimIndex(name='variable', index=4, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(0, 12, None),)
WARNING Variable q of 3583180800 bytes is 7.17 times xarray_zarr.py:602 larger than specified maximum variable array size of 500000000 bytes. Consider re-instantiating recipe with `subset_inputs = {"time": 8}`. If `len(ds["time"])` < 8, substitute "time" for any name in ds["q"].dims with length >= 8 or consider subsetting along multiple dimensions. Setting PANGEO_FORGE_MAX_MEMORY env variable changes the variable array size which will trigger this warning.
[03/10/22 11:23:40] INFO Storing variable q chunk xarray_zarr.py:623 Index({DimIndex(name='variable', index=4, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(0, 12, None), slice(None, None, None), slice(None, None, None))
[2022-03-10 11:23:42-0800] INFO - prefect.TaskRunner | Task 'store_chunk[4]': Finished task run for task with final state: 'Success'
[2022-03-10 11:23:42-0800] INFO - prefect.TaskRunner | Task 'store_chunk[5]': Starting task run...
[03/10/22 11:23:42] INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=5, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=5, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_soil_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_soil_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=5, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing variable time chunk xarray_zarr.py:623 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=5, sequence_len=14, operation=<CombineOp.MERGE: 1>)}) to Zarr region (slice(0, 12, None),)
WARNING Variable soil of 1791590400 bytes is 3.58 xarray_zarr.py:602 times larger than specified maximum variable array size of 500000000 bytes. Consider re-instantiating recipe with `subset_inputs = {"time": 4}`. If `len(ds["time"])` < 4, substitute "time" for any name in ds["soil"].dims with length >= 4 or consider subsetting along multiple dimensions. Setting PANGEO_FORGE_MAX_MEMORY env variable changes the variable array size which will trigger this warning.
[03/10/22 11:23:48] INFO Storing variable soil chunk xarray_zarr.py:623 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=5, sequence_len=14, operation=<CombineOp.MERGE: 1>)}) to Zarr region (slice(0, 12, None), slice(None, None, None), slice(None, None, None))
[2022-03-10 11:23:49-0800] INFO - prefect.TaskRunner | Task 'store_chunk[5]': Finished task run for task with final state: 'Success'
[2022-03-10 11:23:49-0800] INFO - prefect.TaskRunner | Task 'store_chunk[6]': Starting task run...
[03/10/22 11:23:49] INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=6, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=6, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_srad_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_srad_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=6, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing variable time chunk xarray_zarr.py:623 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=6, sequence_len=14, operation=<CombineOp.MERGE: 1>)}) to Zarr region (slice(0, 12, None),)
WARNING Variable srad of 1791590400 bytes is 3.58 xarray_zarr.py:602 times larger than specified maximum variable array size of 500000000 bytes. Consider re-instantiating recipe with `subset_inputs = {"time": 4}`. If `len(ds["time"])` < 4, substitute "time" for any name in ds["srad"].dims with length >= 4 or consider subsetting along multiple dimensions. Setting PANGEO_FORGE_MAX_MEMORY env variable changes the variable array size which will trigger this warning.
[03/10/22 11:23:56] INFO Storing variable srad chunk xarray_zarr.py:623 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=6, sequence_len=14, operation=<CombineOp.MERGE: 1>)}) to Zarr region (slice(0, 12, None), slice(None, None, None), slice(None, None, None))
[2022-03-10 11:23:56-0800] INFO - prefect.TaskRunner | Task 'store_chunk[6]': Finished task run for task with final state: 'Success'
[2022-03-10 11:23:56-0800] INFO - prefect.TaskRunner | Task 'store_chunk[7]': Starting task run...
INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='variable', index=7, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=7, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_swe_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_swe_1958.nc' from cache
[03/10/22 11:23:57] INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='variable', index=7, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:623 Index({DimIndex(name='variable', index=7, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(0, 12, None),)
WARNING Variable swe of 3583180800 bytes is 7.17 xarray_zarr.py:602 times larger than specified maximum variable array size of 500000000 bytes. Consider re-instantiating recipe with `subset_inputs = {"time": 8}`. If `len(ds["time"])` < 8, substitute "time" for any name in ds["swe"].dims with length >= 8 or consider subsetting along multiple dimensions. Setting PANGEO_FORGE_MAX_MEMORY env variable changes the variable array size which will trigger this warning.
[03/10/22 11:24:16] INFO Storing variable swe chunk xarray_zarr.py:623 Index({DimIndex(name='variable', index=7, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(0, 12, None), slice(None, None, None), slice(None, None, None))
[2022-03-10 11:24:18-0800] INFO - prefect.TaskRunner | Task 'store_chunk[7]': Finished task run for task with final state: 'Success'
[2022-03-10 11:24:18-0800] INFO - prefect.TaskRunner | Task 'store_chunk[8]': Starting task run...
[03/10/22 11:24:18] INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=8, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=8, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_tmax_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_tmax_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=8, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing variable time chunk xarray_zarr.py:623 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=8, sequence_len=14, operation=<CombineOp.MERGE: 1>)}) to Zarr region (slice(0, 12, None),)
WARNING Variable tmax of 1791590400 bytes is 3.58 xarray_zarr.py:602 times larger than specified maximum variable array size of 500000000 bytes. Consider re-instantiating recipe with `subset_inputs = {"time": 4}`. If `len(ds["time"])` < 4, substitute "time" for any name in ds["tmax"].dims with length >= 4 or consider subsetting along multiple dimensions. Setting PANGEO_FORGE_MAX_MEMORY env variable changes the variable array size which will trigger this warning.
[03/10/22 11:24:25] INFO Storing variable tmax chunk xarray_zarr.py:623 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=8, sequence_len=14, operation=<CombineOp.MERGE: 1>)}) to Zarr region (slice(0, 12, None), slice(None, None, None), slice(None, None, None))
[2022-03-10 11:24:25-0800] INFO - prefect.TaskRunner | Task 'store_chunk[8]': Finished task run for task with final state: 'Success'
[2022-03-10 11:24:25-0800] INFO - prefect.TaskRunner | Task 'store_chunk[9]': Starting task run...
INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='variable', index=9, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='variable', index=9, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_tmin_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_tmin_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='variable', index=9, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
INFO Storing variable time chunk xarray_zarr.py:623 Index({DimIndex(name='variable', index=9, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(0, 12, None),)
WARNING Variable tmin of 1791590400 bytes is 3.58 xarray_zarr.py:602 times larger than specified maximum variable array size of 500000000 bytes. Consider re-instantiating recipe with `subset_inputs = {"time": 4}`. If `len(ds["time"])` < 4, substitute "time" for any name in ds["tmin"].dims with length >= 4 or consider subsetting along multiple dimensions. Setting PANGEO_FORGE_MAX_MEMORY env variable changes the variable array size which will trigger this warning.
[03/10/22 11:24:33] INFO Storing variable tmin chunk xarray_zarr.py:623 Index({DimIndex(name='variable', index=9, sequence_len=14, operation=<CombineOp.MERGE: 1>), DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)}) to Zarr region (slice(0, 12, None), slice(None, None, None), slice(None, None, None))
[2022-03-10 11:24:33-0800] INFO - prefect.TaskRunner | Task 'store_chunk[9]': Finished task run for task with final state: 'Success'
[2022-03-10 11:24:33-0800] INFO - prefect.TaskRunner | Task 'store_chunk[10]': Starting task run...
INFO Opening inputs for chunk xarray_zarr.py:386 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=10, sequence_len=14, operation=<CombineOp.MERGE: 1>)})
INFO Opening input with Xarray xarray_zarr.py:249 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=10, sequence_len=14, operation=<CombineOp.MERGE: 1>)}): 'http://thredds.northwestknowledge.net :8080/thredds/fileServer/TERRACLIMATE_ALL/dat a/TerraClimate_vap_1958.nc'
INFO Opening 'http://thredds.northwestknowledge.net:80 storage.py:260 80/thredds/fileServer/TERRACLIMATE_ALL/data/Terra Climate_vap_1958.nc' from cache
INFO Combining inputs for chunk xarray_zarr.py:404 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=10, sequence_len=14, operation=<CombineOp.MERGE: 1>)})'
INFO Storing variable time chunk xarray_zarr.py:623 Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>), DimIndex(name='variable', index=10, sequence_len=14, operation=<CombineOp.MERGE: 1>)}) to Zarr region (slice(0, 12, None),)
WARNING Variable vap of 1791590400 bytes is 3.58 xarray_zarr.py