gridstats 2.0.0__tar.gz

gridstats-2.0.0/AUTHORS.rst

@@ -0,0 +1,13 @@
+=======
+Credits
+=======
+
+Development Lead
+----------------
+
+* Oceanum Developers <developers@oceanum.science>
+
+Contributors
+------------
+
+None yet. Why not be the first?

gridstats-2.0.0/CONTRIBUTING.rst

@@ -0,0 +1,128 @@
+.. highlight:: shell
+
+============
+Contributing
+============
+
+Contributions are welcome, and they are greatly appreciated! Every little bit
+helps, and credit will always be given.
+
+You can contribute in many ways:
+
+Types of Contributions
+----------------------
+
+Report Bugs
+~~~~~~~~~~~
+
+Report bugs at https://github.com/oceanum/gridstats/issues.
+
+If you are reporting a bug, please include:
+
+* Your operating system name and version.
+* Any details about your local setup that might be helpful in troubleshooting.
+* Detailed steps to reproduce the bug.
+
+Fix Bugs
+~~~~~~~~
+
+Look through the GitHub issues for bugs. Anything tagged with "bug" and "help
+wanted" is open to whoever wants to implement it.
+
+Implement Features
+~~~~~~~~~~~~~~~~~~
+
+Look through the GitHub issues for features. Anything tagged with "enhancement"
+and "help wanted" is open to whoever wants to implement it.
+
+Write Documentation
+~~~~~~~~~~~~~~~~~~~
+
+gridstats could always use more documentation, whether as part of the
+official gridstats docs, in docstrings, or even on the web in blog posts,
+articles, and such.
+
+Submit Feedback
+~~~~~~~~~~~~~~~
+
+The best way to send feedback is to file an issue at https://github.com/oceanum/gridstats/issues.
+
+If you are proposing a feature:
+
+* Explain in detail how it would work.
+* Keep the scope as narrow as possible, to make it easier to implement.
+* Remember that this is a volunteer-driven project, and that contributions
+  are welcome :)
+
+Get Started!
+------------
+
+Ready to contribute? Here's how to set up `gridstats` for local development.
+
+1. Fork the `gridstats` repo on GitHub.
+2. Clone your fork locally::
+
+    $ git clone git@github.com:your_name_here/gridstats.git
+
+3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development::
+
+    $ mkvirtualenv gridstats
+    $ cd gridstats/
+    $ python setup.py develop
+
+4. Create a branch for local development::
+
+    $ git checkout -b name-of-your-bugfix-or-feature
+
+   Now you can make your changes locally.
+
+5. When you're done making changes, check that your changes pass flake8 and the
+   tests, including testing other Python versions with tox::
+
+    $ flake8 gridstats tests
+    $ python setup.py test or py.test
+    $ tox
+
+   To get flake8 and tox, just pip install them into your virtualenv.
+
+6. Commit your changes and push your branch to GitHub::
+
+    $ git add .
+    $ git commit -m "Your detailed description of your changes."
+    $ git push origin name-of-your-bugfix-or-feature
+
+7. Submit a pull request through the GitHub website.
+
+Pull Request Guidelines
+-----------------------
+
+Before you submit a pull request, check that it meets these guidelines:
+
+1. The pull request should include tests.
+2. If the pull request adds functionality, the docs should be updated. Put
+   your new functionality into a function with a docstring, and add the
+   feature to the list in README.rst.
+3. The pull request should work for Python 2.7, 3.4, 3.5 and 3.6, and for PyPy. Check
+   https://travis-ci.org/oceanum/gridstats/pull_requests
+   and make sure that the tests pass for all supported Python versions.
+
+Tips
+----
+
+To run a subset of tests::
+
+$ py.test tests.test_stats
+
+
+Deploying
+---------
+
+A reminder for the maintainers on how to deploy.
+Make sure all your changes are committed (including an entry in HISTORY.rst).
+Then run::
+
+$ bumpversion patch # possible: major / minor / patch
+$ git push
+$ git push --tags
+
+Travis will then deploy to PyPI if tests pass.

gridstats-2.0.0/HISTORY.rst

@@ -0,0 +1,8 @@
+=======
+History
+=======
+
+0.1.0 (2019-08-09)
+------------------
+
+* First release on PyPI.

gridstats-2.0.0/MANIFEST.in

@@ -0,0 +1,12 @@
+include AUTHORS.rst
+include CONTRIBUTING.rst
+include HISTORY.rst
+include LICENSE
+include README.rst
+include gridstats/attributes.yml
+
+recursive-include tests *
+recursive-exclude * __pycache__
+recursive-exclude * *.py[co]
+
+recursive-include docs *.rst conf.py Makefile make.bat *.jpg *.png *.gif

gridstats-2.0.0/PKG-INFO

@@ -0,0 +1,192 @@
+Metadata-Version: 2.4
+Name: gridstats
+Version: 2.0.0
+Summary: Oceanum library for computing gridded statistics on oceanographic datasets
+Author-email: Oceanum Developers <developers@oceanum.science>
+License: MIT
+Project-URL: Homepage, https://github.com/oceanum/gridstats
+Project-URL: Repository, https://github.com/oceanum/gridstats
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: Natural Language :: English
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Atmospheric Science
+Classifier: Topic :: Scientific/Engineering :: Oceanography
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: AUTHORS.rst
+Requires-Dist: bottleneck
+Requires-Dist: dask
+Requires-Dist: distributed
+Requires-Dist: flox
+Requires-Dist: netcdf4
+Requires-Dist: pydantic>=2
+Requires-Dist: pyyaml
+Requires-Dist: scipy
+Requires-Dist: typer
+Requires-Dist: xarray
+Requires-Dist: zarr<3
+Provides-Extra: extra
+Requires-Dist: gcsfs; extra == "extra"
+Requires-Dist: intake; extra == "extra"
+Requires-Dist: intake-forecast; extra == "extra"
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Provides-Extra: docs
+Requires-Dist: mkdocs>=1.5; extra == "docs"
+Requires-Dist: mkdocs-material>=9; extra == "docs"
+Requires-Dist: mkdocstrings[python]>=0.24; extra == "docs"
+Dynamic: license-file
+
+# gridstats
+
+**gridstats** is an Oceanum library for computing gridded statistics over large oceanographic and climate datasets. Pipelines are defined in YAML and run as a single CLI command. Computation is lazy and out-of-core via [xarray](https://xarray.dev) and [dask](https://dask.org), so datasets of arbitrary size are handled without loading them into memory.
+
+## Features
+
+- **YAML-driven pipelines** — source, operations, and output are all declared in one config file
+- **Out-of-core** — processes arbitrarily large grids lazily; spatial `tiles:` keeps peak memory bounded
+- **Rich stat library** — aggregations, quantiles, exceedance, return period values, directional stats, distributions, and more
+- **Multiple output formats** — NetCDF or Zarr
+- **CF-compliant** — output variables are automatically annotated with standard names, units, and long names
+- **Extensible** — register custom stat functions and loaders via decorator or package entry point
+
+## Installation
+
+Requires Python ≥ 3.10.
+
+```bash
+pip install gridstats
+```
+
+For loading data from an [intake](https://intake.readthedocs.io) catalog:
+
+```bash
+pip install "gridstats[intake]"
+```
+
+## Quick start
+
+### 1. Write a config file
+
+```yaml
+# stats.yml
+source:
+  type: xarray
+  urlpath: /data/hindcast/waves.zarr
+  engine: zarr
+  sel:
+    time: {start: "2000-01-01", stop: "2020-12-31"}
+    latitude: {start: -50, stop: -30}
+    longitude: {start: 160, stop: 180}
+
+output:
+  outfile: ./wave_stats.zarr
+
+calls:
+  - func: mean
+    dim: time
+    data_vars: [hs, tp]
+
+  - func: quantile
+    dim: time
+    data_vars: [hs]
+    q: [0.5, 0.90, 0.95, 0.99]
+
+  - func: rpv
+    dim: time
+    data_vars: [hs]
+    return_periods: [10, 50, 100]
+    distribution: gumbel_r
+```
+
+### 2. Run it
+
+```bash
+gridstats run stats.yml
+```
+
+The output dataset will contain variables like `hs_mean`, `tp_mean`, `hs_quantile`, and `hs_rpv`, each with CF-standard attributes.
+
+### 3. Use the result
+
+```python
+import xarray as xr
+
+ds = xr.open_zarr("wave_stats.zarr")
+print(ds)
+```
+
+## Available stat functions
+
+| Function | Description |
+|---|---|
+| `mean`, `max`, `min`, `std`, `count` | Basic aggregations |
+| `quantile` | Quantiles at arbitrary probability levels |
+| `pcount` | Count of non-NaN values per grid cell |
+| `exceedance` / `nonexceedance` | Probability of exceeding a threshold |
+| `range_probability` | Probability of a value falling in a range |
+| `rpv` | Return period values via extreme value fitting |
+| `distribution2` / `distribution3` | 2- and 3-parameter distribution fitting |
+| `statdir` | Directional statistics (sector-binned) |
+| `hmo` | Significant wave height from spectral moments |
+| `winpow` | Wind power density |
+
+All calls accept a `group:` key (`month`, `season`, `hour`, …) to compute statistics per calendar period.
+
+## Grouping and spatial tiling
+
+```yaml
+calls:
+  # Monthly mean
+  - func: mean
+    dim: time
+    data_vars: [hs]
+    group: month
+
+  # Quantile with spatial tiling to control memory
+  - func: quantile
+    dim: time
+    data_vars: [hs]
+    q: [0.95, 0.99]
+    chunks: {time: -1, latitude: 50, longitude: 50}
+    tiles: {latitude: 10, longitude: 10}
+```
+
+## Plugin system
+
+Register a custom stat function in your own package:
+
+```python
+from gridstats.registry import register_stat
+import xarray as xr
+
+@register_stat("my_stat")
+def my_stat(data: xr.Dataset, *, dim: str = "time", **kwargs) -> xr.Dataset:
+    ...
+```
+
+Or declare it as a package entry point so it is discovered automatically:
+
+```toml
+[project.entry-points."gridstats.stats"]
+my_stat = "my_package.stats:my_stat"
+```
+
+## CLI
+
+```
+Usage: gridstats [OPTIONS] COMMAND [ARGS]...
+
+Commands:
+  run         Run a stats pipeline from a YAML configuration file.
+  list-stats  List all registered stat functions.
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

gridstats-2.0.0/README.md

@@ -0,0 +1,148 @@
+# gridstats
+
+**gridstats** is an Oceanum library for computing gridded statistics over large oceanographic and climate datasets. Pipelines are defined in YAML and run as a single CLI command. Computation is lazy and out-of-core via [xarray](https://xarray.dev) and [dask](https://dask.org), so datasets of arbitrary size are handled without loading them into memory.
+
+## Features
+
+- **YAML-driven pipelines** — source, operations, and output are all declared in one config file
+- **Out-of-core** — processes arbitrarily large grids lazily; spatial `tiles:` keeps peak memory bounded
+- **Rich stat library** — aggregations, quantiles, exceedance, return period values, directional stats, distributions, and more
+- **Multiple output formats** — NetCDF or Zarr
+- **CF-compliant** — output variables are automatically annotated with standard names, units, and long names
+- **Extensible** — register custom stat functions and loaders via decorator or package entry point
+
+## Installation
+
+Requires Python ≥ 3.10.
+
+```bash
+pip install gridstats
+```
+
+For loading data from an [intake](https://intake.readthedocs.io) catalog:
+
+```bash
+pip install "gridstats[intake]"
+```
+
+## Quick start
+
+### 1. Write a config file
+
+```yaml
+# stats.yml
+source:
+  type: xarray
+  urlpath: /data/hindcast/waves.zarr
+  engine: zarr
+  sel:
+    time: {start: "2000-01-01", stop: "2020-12-31"}
+    latitude: {start: -50, stop: -30}
+    longitude: {start: 160, stop: 180}
+
+output:
+  outfile: ./wave_stats.zarr
+
+calls:
+  - func: mean
+    dim: time
+    data_vars: [hs, tp]
+
+  - func: quantile
+    dim: time
+    data_vars: [hs]
+    q: [0.5, 0.90, 0.95, 0.99]
+
+  - func: rpv
+    dim: time
+    data_vars: [hs]
+    return_periods: [10, 50, 100]
+    distribution: gumbel_r
+```
+
+### 2. Run it
+
+```bash
+gridstats run stats.yml
+```
+
+The output dataset will contain variables like `hs_mean`, `tp_mean`, `hs_quantile`, and `hs_rpv`, each with CF-standard attributes.
+
+### 3. Use the result
+
+```python
+import xarray as xr
+
+ds = xr.open_zarr("wave_stats.zarr")
+print(ds)
+```
+
+## Available stat functions
+
+| Function | Description |
+|---|---|
+| `mean`, `max`, `min`, `std`, `count` | Basic aggregations |
+| `quantile` | Quantiles at arbitrary probability levels |
+| `pcount` | Count of non-NaN values per grid cell |
+| `exceedance` / `nonexceedance` | Probability of exceeding a threshold |
+| `range_probability` | Probability of a value falling in a range |
+| `rpv` | Return period values via extreme value fitting |
+| `distribution2` / `distribution3` | 2- and 3-parameter distribution fitting |
+| `statdir` | Directional statistics (sector-binned) |
+| `hmo` | Significant wave height from spectral moments |
+| `winpow` | Wind power density |
+
+All calls accept a `group:` key (`month`, `season`, `hour`, …) to compute statistics per calendar period.
+
+## Grouping and spatial tiling
+
+```yaml
+calls:
+  # Monthly mean
+  - func: mean
+    dim: time
+    data_vars: [hs]
+    group: month
+
+  # Quantile with spatial tiling to control memory
+  - func: quantile
+    dim: time
+    data_vars: [hs]
+    q: [0.95, 0.99]
+    chunks: {time: -1, latitude: 50, longitude: 50}
+    tiles: {latitude: 10, longitude: 10}
+```
+
+## Plugin system
+
+Register a custom stat function in your own package:
+
+```python
+from gridstats.registry import register_stat
+import xarray as xr
+
+@register_stat("my_stat")
+def my_stat(data: xr.Dataset, *, dim: str = "time", **kwargs) -> xr.Dataset:
+    ...
+```
+
+Or declare it as a package entry point so it is discovered automatically:
+
+```toml
+[project.entry-points."gridstats.stats"]
+my_stat = "my_package.stats:my_stat"
+```
+
+## CLI
+
+```
+Usage: gridstats [OPTIONS] COMMAND [ARGS]...
+
+Commands:
+  run         Run a stats pipeline from a YAML configuration file.
+  list-stats  List all registered stat functions.
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

gridstats-2.0.0/docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SPHINXOPTS    =
+SPHINXBUILD   = python -msphinx
+SPHINXPROJ    = gridstats
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

gridstats-2.0.0/docs/authors.rst

@@ -0,0 +1 @@
+.. include:: ../AUTHORS.rst

gridstats-2.0.0/docs/conf.py

@@ -0,0 +1,161 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# gridstats documentation build configuration file, created by
+# sphinx-quickstart on Fri Jun  9 13:47:02 2017.
+#
+# This file is execfile()d with the current directory set to its
+# containing dir.
+#
+# Note that not all possible configuration values are present in this
+# autogenerated file.
+#
+# All configuration values have a default; values that are commented out
+# serve to show the default.
+
+# If extensions (or modules to document with autodoc) are in another
+# directory, add these directories to sys.path here. If the directory is
+# relative to the documentation root, use os.path.abspath to make it
+# absolute, like shown here.
+#
+import os
+import sys
+
+sys.path.insert(0, os.path.abspath(".."))
+
+import gridstats
+
+# -- General configuration ---------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+#
+# needs_sphinx = '1.0'
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
+extensions = ["sphinx.ext.autodoc", "sphinx.ext.viewcode"]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ["_templates"]
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+#
+# source_suffix = ['.rst', '.md']
+source_suffix = ".rst"
+
+# The master toctree document.
+master_doc = "index"
+
+# General information about the project.
+project = u"gridstats"
+copyright = u"2019, Oceanum Developers"
+author = u"Oceanum Developers"
+
+# The version info for the project you're documenting, acts as replacement
+# for |version| and |release|, also used in various other places throughout
+# the built documents.
+#
+# The short X.Y version.
+version = gridstats.__version__
+# The full version, including alpha/beta/rc tags.
+release = gridstats.__version__
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#
+# This is also used if you do content translation via gettext catalogs.
+# Usually you set "language" from the command line for these cases.
+language = None
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This patterns also effect to html_static_path and html_extra_path
+exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = "sphinx"
+
+# If true, `todo` and `todoList` produce output, else they produce nothing.
+todo_include_todos = False
+
+
+# -- Options for HTML output -------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = "alabaster"
+
+# Theme options are theme-specific and customize the look and feel of a
+# theme further.  For a list of options available for each theme, see the
+# documentation.
+#
+# html_theme_options = {}
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ["_static"]
+
+
+# -- Options for HTMLHelp output ---------------------------------------
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = "statsdoc"
+
+
+# -- Options for LaTeX output ------------------------------------------
+
+latex_elements = {
+    # The paper size ('letterpaper' or 'a4paper').
+    #
+    # 'papersize': 'letterpaper',
+    # The font size ('10pt', '11pt' or '12pt').
+    #
+    # 'pointsize': '10pt',
+    # Additional stuff for the LaTeX preamble.
+    #
+    # 'preamble': '',
+    # Latex figure (float) alignment
+    #
+    # 'figure_align': 'htbp',
+}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title, author, documentclass
+# [howto, manual, or own class]).
+latex_documents = [
+    (
+        master_doc,
+        "gridstats.tex",
+        u"gridstats Documentation",
+        u"Oceanum Developers",
+        "manual",
+    )
+]
+
+
+# -- Options for manual page output ------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [(master_doc, "gridstats", u"gridstats Documentation", [author], 1)]
+
+
+# -- Options for Texinfo output ----------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+#  dir menu entry, description, category)
+texinfo_documents = [
+    (
+        master_doc,
+        "gridstats",
+        u"gridstats Documentation",
+        author,
+        "gridstats",
+        "One line description of project.",
+        "Miscellaneous",
+    )
+]

gridstats-2.0.0/docs/contributing.rst

@@ -0,0 +1 @@
+.. include:: ../CONTRIBUTING.rst

gridstats-2.0.0/docs/history.rst

@@ -0,0 +1 @@
+.. include:: ../HISTORY.rst