Release Notes#

v0.9.4 - 2023-01-06#

  • Fixed bug in 0.9.3 which broke all recipe object imports if scipy (an optional dependency) was missing from environment. GH PR 466

v0.9.3 - 2023-01-03#

Warning

Yanked due to presence of a bug. Please use 0.9.4 instead.

  • Bugfix to allow opening of zarr files. This fix allows using Zarr stores as an input source for recipes. GH PR 462

  • Add netcdf3 support for opening source files with kerchunk. Resolves a long-standing issue wherein netcdf3 source files could not be loaded lazily, which effectively blocked the use of large netcdf3 files as recipe sources. GH PR 383

  • Fix zarr reference bug GH PR 455

  • Add dataset_type class attribute for recipe classes GH PR 437

v0.9.2 - 2022-10-26#

Contributors to this release#

(GitHub contributors page for this release)

@alxmrs | @andersy005 | @dependabot | @derekocallaghan | @martindurant | @pre-commit-ci | @rabernat | @yuvipanda

v0.9.1 - 2022-09-08#

  • Persist Pangeo Forge execution context metadata in target datasets. This information, which includes the pangeo-forge-recipes version as well as recipe and input hashes, attaches execution provenance to the dataset itself. GH PR 359

  • File pattern support for .tiffs. GH PR 393

  • Improved HDFReferenceRecipe by passing target_options to the MultiZarrToZarr class.

  • Typo fixes, documentation updates, and project health improvements. GH PR 364, GH PR 365, GH PR 366, GH PR 388, GH PR 396, GH PR 394, GH PR 398, GH PR 407.

  • Fixed XarrayZarrRecipe’s finalize_target() stage by using bulk delete APIs for consolidating Zarr coordinates.

v0.9 - 2022-05-11#

  • Breaking changes: Deprecated XarrayZarrRecipe manual stage methods. Manual execution can be performed with any of the executors described in Recipe Execution. Also deprecated FilePattern(..., is_opendap=True) kwarg, which is superseded by FilePattern(..., file_type="opendap"). GH PR 362

  • Added serialization module along with BaseRecipe.sha256 and FilePattern.sha256 methods. Collectively, this provides for generation of deterministic hashes for both recipe and file pattern instances. Checking these hashes against those from a prior version of the recipe can be used to determine whether or not a particular recipe instance in a Python module (which may contain arbitrary numbers of recipe instances) has changed since the last time the instances in that module were executed. The file pattern hashes are based on a blockchain built cumulatively from all of the index:filepath pairs yielded by the pattern’s self.items() method. As such, in cases where a new pattern is intended to append to an existing dataset which was built from a prior version of that pattern, the pattern hash can be used to determine the index from which to begin appending. This is demonstrated in the tests. GH PR 349

  • Created new Prefect executor which wraps the Dask executor in a single Task. This should mitigate problems related to large numbers of Prefect Tasks (GH issue 347). See Recipe Execution for details.

  • Implemented feature to cap cached filename lengths at 255 bytes on local filesystems, to accomodate the POSIX filename length limit. Cached filename lengths are not truncated on any other filesystem. GH PR 353

v0.8.3 - 2022-04-19#

  • Added .file_type attribute to pangeo_forge_recipes.patterns.FilePattern. This attribute will eventually supercede .is_opendap, which will be deprecated in 0.9.0. Until then, FilePattern(..., is_opendap=True) is supported as equivalent to FilePattern(..., file_type="opendap"). GH PR 322

v0.8.2 - 2022-02-23#

  • Removed click from dependencies and removed cli entrypoint.

v0.8.1 - 2022-02-23#

  • Fixed dependency issue with pip installation.

  • Fixed bug where recipes would fail if the target chunks exceeded the full array length. GH issue 279

v0.8.0 - 2022-02-17#

v0.7.0 - 2022-02-14 ❤️#

v0.6.1 - 2021-10-25#

  • Major internal refactor of executors. GH PR 219. Began deprecation cycle for recipe methods (e.g. recipe.prepare_target()) in favor of module functions.

  • Addition of open_input_with_fsspec_reference option on pangeo_forge_recipes.recipes.XarrayZarrRecipe, permitting the bypassing of h5py when opening inputs. GH PR 218

v0.6.0 - 2021-09-02#

v0.5.0 - 2021-07-11#

v0.4.0 - 2021-06-25#

  • Fixed issue with recipe serialilzation causing high memory usage of Dask schedulers and workers when executing recipes with Prefect or Dask GH PR 160.

  • Added new methods .to_dask(), to_prefect(), and .to_function() for converting a recipe to one of the Dask, Prefect, or Python execution plans. The previous method, recpie.to_pipelines() is now deprecated.

v0.3.4 - 2021-05-25#

v0.3.3 - 2021-05-10#

Many feature enhancements. Non-backwards compatible changes to core API. Package renamed from pangeo_forge to pangeo_forge_recipes.

There were problems with packaging for the 0.3.0-0.3.2 releases.

v0.2.0 - 2021-04-26#

First release since major Spring 2021 overhaul. This release depends on Xarray v0.17.1, which has not yet been released as of the date of this release.

v0.1.0 - 2020-10-22#

First release.