mxalign 0.1.0__tar.gz

mxalign-0.1.0/.github/dependabot.yaml

@@ -0,0 +1,22 @@
+version: 2
+updates:
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "monthly"
+    cooldown:
+      default-days: 7
+  - package-ecosystem: "pre-commit"
+    directory: "/"
+    schedule:
+      interval: "monthly"
+  - package-ecosystem: "uv"
+    directory: "/"
+    schedule:
+      interval: "monthly"
+    cooldown:
+      default-days: 7
+    groups:
+      uv.lock-patches:
+        update-types:
+        - patch

mxalign-0.1.0/.github/workflows/linting.yaml

@@ -0,0 +1,16 @@
+name: Linting Checks
+
+on:
+  pull_request:
+  push:
+    branches: [main, develop]
+  workflow_dispatch:
+
+jobs:
+  pre-commit:
+    name: Run Linters
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v6
+    - uses: actions/setup-python@v6
+    - uses: pre-commit/action@v3.0.1

mxalign-0.1.0/.gitignore

@@ -0,0 +1,10 @@
+# Python-generated files
+__pycache__/
+*.py[oc]
+build/
+dist/
+wheels/
+*.egg-info
+
+# Virtual environments
+.venv

mxalign-0.1.0/.pre-commit-config.yaml

@@ -0,0 +1,26 @@
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v6.0.0
+    hooks:
+      - id: trailing-whitespace
+      - id: end-of-file-fixer
+      - id: check-yaml
+      - id: check-json
+      - id: check-toml
+      - id: check-added-large-files
+      - id: check-merge-conflict
+      - id: mixed-line-ending
+      - id: debug-statements
+
+  - repo: https://github.com/codespell-project/codespell
+    rev: v2.4.2
+    hooks:
+    - id: codespell
+      args: [--ignore-words-list=COO, --ignore-regex=^\s*"image\/png":\s.*]
+
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.15.6
+    hooks:
+      - id: ruff-check
+        args: [--fix]
+      - id: ruff-format

mxalign-0.1.0/.python-version

@@ -0,0 +1 @@
+3.12

mxalign-0.1.0/CHANGELOG.md

@@ -0,0 +1,20 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.0](https://github.com/mlwp-tools/mxalign/releases/tag/v0.1.0)
+
+First release of `mxalign`, an xarray-based package for alignment of meteorological datasets, with the following functionality and configuration:
+
+- Execution of the verification tooling pipeline.
+- Core alignment capabilities (handling of space, time, and NaNs).
+- Base data loaders, including support for Anemoi datasets and inference.
+- Validation functionality for datasets.
+- Implementations for interpolations (e.g., Delaunay, xarray).
+- Accessors (space, time), transformations, and properties management tools.
+- Introductory Jupyter notebook (`examples/introduction.ipynb`) demonstrating interactive usage.
+- Integration of `uv.lock` dependency locking to ensure specific versions are used in testing, allowing for safe and reliable releases.
+- Dependabot configuration with a strategy where PRs for flexible requirements will only be merged once tests confirm that `mxalign` works correctly with the newer upstream packages.

mxalign-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 rmi-mlwp
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

mxalign-0.1.0/PKG-INFO

@@ -0,0 +1,136 @@
+Metadata-Version: 2.4
+Name: mxalign
+Version: 0.1.0
+Summary: Add your description here
+Author-email: Michiel Van Ginderachter <michiel.vanginderachter@meteo.be>
+License-File: LICENSE
+Requires-Python: >=3.12
+Requires-Dist: bokeh>=3.8.2
+Requires-Dist: cartopy>=0.25.0
+Requires-Dist: dask>=2026.1.2
+Requires-Dist: distributed>=2026.1.2
+Requires-Dist: earthkit-data>=0.19.0
+Requires-Dist: h5netcdf>=1.8.1
+Requires-Dist: h5py>=3.15.1
+Requires-Dist: netcdf4>=1.7.4
+Requires-Dist: pyyaml>=6.0.3
+Requires-Dist: scipy>=1.17.0
+Requires-Dist: xarray>=2026.1.0
+Requires-Dist: zarr<3.0
+Provides-Extra: earthkit
+Requires-Dist: earthkit-meteo>=0.6.1; extra == 'earthkit'
+Provides-Extra: jobqueue
+Requires-Dist: dask-jobqueue>=0.9.0; extra == 'jobqueue'
+Provides-Extra: verification
+Requires-Dist: xskillscore>=0.0.29; extra == 'verification'
+Description-Content-Type: text/markdown
+
+# Meteo-xAlign
+
+**An xarray based package for alignment of meteorological datasets**
+
+## What is this?
+
+`mxalign` is an `xarray`-based package designed for the alignment and verification of meteorological datasets. It standardizes operations across datasets by attaching properties along three main axes:
+- **Space:** Grid or point-based data
+- **Time:** Forecasts, observations, or climatology
+- **Uncertainty:** Deterministic, ensemble, or quantile forecasts
+
+Currently, `mxalign` also acts as a full execution engine. It can load datasets (e.g., Anemoi inference outputs, observation datasets), apply transformations, align datasets in both space and time to match a reference, safely broadcast NaNs, and execute verification metrics on scaled Dask clusters (Local or Slurm).
+
+> ⚠️ **Roadmap & Future Architecture Changes (planned for v0.2.0):**
+> Currently, `mxalign` handles both alignment and the execution of the verification tooling pipeline, including loading and validation. In the upcoming `v0.2.0` release, this architecture will be refactored:
+> - **Loading** will be split out into [`mlwp-data-loaders`](https://github.com/mlwp-tools/mlwp-data-loaders).
+> - **Validation** of loaded `xr.Dataset`s will be moved to [`mlwp-data-specs`](https://github.com/mlwp-tools/mlwp-data-specs) (which will contain the requirements for each of the dataset traits and the validation logic).
+> - **Execution** of the full verification pipeline (loading, transformations, alignment, and verification) from configuration files may be moved to a separate package in future releases.
+> - **Tests** will be added to `mxalign` (building on test datasets already integrated into `mlwp-data-loaders`) that ensure that all alignment operations work correctly (Testing notebook execution inside `mxalign` is explicitly excluded from the current roadmap).
+
+## Python API
+
+`mxalign` provides building blocks for manual alignment, transformations, and interpolations of `xarray` datasets. This is ideal for interactive use in Jupyter notebooks or custom Python scripts.
+
+```python
+import xarray as xr
+from mxalign import load, align_space, align_time, transform
+
+# Load datasets (using registered loaders)
+ds_obs = load(name="observations_loader", files=["obs.nc"])
+ds_fcst = load(name="anemoi_inference", files=["forecast.nc"])
+
+# Align the forecast spatially to match the observation reference
+ds_fcst_aligned_space = align_space(ds_fcst, reference=ds_obs, method="interpolation")
+
+# Align datasets temporally
+datasets = {"obs": ds_obs, "fcst": ds_fcst_aligned_space}
+aligned_datasets = align_time(datasets, method="intersection")
+```
+
+For a more comprehensive interactive example, check out the [introductory notebook](./examples/introduction.ipynb).
+
+## Executing via a Configuration
+
+For full verification pipeline execution, `mxalign` uses a YAML configuration file. This allows you to declaratively define how datasets are loaded, transformed, aligned, and verified.
+
+### Configuration Contents
+
+The configuration file is divided into several main sections:
+
+```yaml
+datasets:
+  # Define datasets to load, specifying the loader, files, and variables
+  obs_data:
+    loader: observations_loader
+    files: ["obs.nc"]
+  fcst_data:
+    loader: anemoi_inference
+    files: ["forecast.nc"]
+
+transformations:
+  # Apply transformations to loaded datasets
+
+alignment:
+  # Define reference dataset and alignment methods (space, time, NaN broadcasting)
+  reference: obs_data
+  time:
+    method: intersection
+
+verification:
+  # Specify the reference dataset and the metrics to calculate
+  reference: obs_data
+  metrics:
+    # define metrics here
+```
+
+### Running from the Command Line
+
+The CLI uses Dask to distribute the workload and supports both local execution and execution on Slurm-managed HPC clusters.
+
+**Local Execution**
+Run the pipeline on a local Dask cluster:
+```bash
+mxalign local path/to/config.yaml --n_workers 4 --threads_per_worker 1
+```
+
+**Slurm Execution**
+Run the pipeline on a Slurm cluster:
+```bash
+mxalign slurm path/to/config.yaml --account your_account --queue your_queue --cores 8 --memory 64GB
+```
+
+### Running from Python
+
+You can also execute the entire configuration-driven pipeline directly from Python using the `Runner` class.
+
+```python
+from mxalign.runner import Runner
+
+# Initialize the runner with a YAML config file or a dictionary
+runner = Runner("path/to/config.yaml")
+
+# Execute the pipeline: loads, transforms, aligns, and verifies the datasets
+runner.run()
+
+# The resulting aligned datasets and computed metrics are accessible via:
+aligned_datasets = runner.datasets
+metrics = runner.metrics
+```

mxalign-0.1.0/README.md

@@ -0,0 +1,109 @@
+# Meteo-xAlign
+
+**An xarray based package for alignment of meteorological datasets**
+
+## What is this?
+
+`mxalign` is an `xarray`-based package designed for the alignment and verification of meteorological datasets. It standardizes operations across datasets by attaching properties along three main axes:
+- **Space:** Grid or point-based data
+- **Time:** Forecasts, observations, or climatology
+- **Uncertainty:** Deterministic, ensemble, or quantile forecasts
+
+Currently, `mxalign` also acts as a full execution engine. It can load datasets (e.g., Anemoi inference outputs, observation datasets), apply transformations, align datasets in both space and time to match a reference, safely broadcast NaNs, and execute verification metrics on scaled Dask clusters (Local or Slurm).
+
+> ⚠️ **Roadmap & Future Architecture Changes (planned for v0.2.0):**
+> Currently, `mxalign` handles both alignment and the execution of the verification tooling pipeline, including loading and validation. In the upcoming `v0.2.0` release, this architecture will be refactored:
+> - **Loading** will be split out into [`mlwp-data-loaders`](https://github.com/mlwp-tools/mlwp-data-loaders).
+> - **Validation** of loaded `xr.Dataset`s will be moved to [`mlwp-data-specs`](https://github.com/mlwp-tools/mlwp-data-specs) (which will contain the requirements for each of the dataset traits and the validation logic).
+> - **Execution** of the full verification pipeline (loading, transformations, alignment, and verification) from configuration files may be moved to a separate package in future releases.
+> - **Tests** will be added to `mxalign` (building on test datasets already integrated into `mlwp-data-loaders`) that ensure that all alignment operations work correctly (Testing notebook execution inside `mxalign` is explicitly excluded from the current roadmap).
+
+## Python API
+
+`mxalign` provides building blocks for manual alignment, transformations, and interpolations of `xarray` datasets. This is ideal for interactive use in Jupyter notebooks or custom Python scripts.
+
+```python
+import xarray as xr
+from mxalign import load, align_space, align_time, transform
+
+# Load datasets (using registered loaders)
+ds_obs = load(name="observations_loader", files=["obs.nc"])
+ds_fcst = load(name="anemoi_inference", files=["forecast.nc"])
+
+# Align the forecast spatially to match the observation reference
+ds_fcst_aligned_space = align_space(ds_fcst, reference=ds_obs, method="interpolation")
+
+# Align datasets temporally
+datasets = {"obs": ds_obs, "fcst": ds_fcst_aligned_space}
+aligned_datasets = align_time(datasets, method="intersection")
+```
+
+For a more comprehensive interactive example, check out the [introductory notebook](./examples/introduction.ipynb).
+
+## Executing via a Configuration
+
+For full verification pipeline execution, `mxalign` uses a YAML configuration file. This allows you to declaratively define how datasets are loaded, transformed, aligned, and verified.
+
+### Configuration Contents
+
+The configuration file is divided into several main sections:
+
+```yaml
+datasets:
+  # Define datasets to load, specifying the loader, files, and variables
+  obs_data:
+    loader: observations_loader
+    files: ["obs.nc"]
+  fcst_data:
+    loader: anemoi_inference
+    files: ["forecast.nc"]
+
+transformations:
+  # Apply transformations to loaded datasets
+
+alignment:
+  # Define reference dataset and alignment methods (space, time, NaN broadcasting)
+  reference: obs_data
+  time:
+    method: intersection
+
+verification:
+  # Specify the reference dataset and the metrics to calculate
+  reference: obs_data
+  metrics:
+    # define metrics here
+```
+
+### Running from the Command Line
+
+The CLI uses Dask to distribute the workload and supports both local execution and execution on Slurm-managed HPC clusters.
+
+**Local Execution**
+Run the pipeline on a local Dask cluster:
+```bash
+mxalign local path/to/config.yaml --n_workers 4 --threads_per_worker 1
+```
+
+**Slurm Execution**
+Run the pipeline on a Slurm cluster:
+```bash
+mxalign slurm path/to/config.yaml --account your_account --queue your_queue --cores 8 --memory 64GB
+```
+
+### Running from Python
+
+You can also execute the entire configuration-driven pipeline directly from Python using the `Runner` class.
+
+```python
+from mxalign.runner import Runner
+
+# Initialize the runner with a YAML config file or a dictionary
+runner = Runner("path/to/config.yaml")
+
+# Execute the pipeline: loads, transforms, aligns, and verifies the datasets
+runner.run()
+
+# The resulting aligned datasets and computed metrics are accessible via:
+aligned_datasets = runner.datasets
+metrics = runner.metrics
+```

mxalign-0.1.0/doc/adr/ADR-001_seperate-validation-and-loading.md

@@ -0,0 +1,39 @@
+---
+# These are optional metadata elements. Feel free to remove any of them.
+status: accepted
+date: 2026-03-25
+decision-makers: Denby L. Van Ginderachter M.
+informed: Francois B., Buurman S.
+---
+
+# Separation of concerns between dataset-loading, -validation and -alignment
+
+## Context
+
+Currently `mxalign` implements dataset loading functionality, validation functionality (checking if the loaded dataset has all the correct metadata and) and alignment functionality. However, dataset-loading and -validation fall outside the main scope of the `mxalign` package.
+
+<!-- This is an optional element. Feel free to remove. -->
+## Decision Drivers
+
+* Provide clarity on what the package does by separating the different concerns
+* Provide a clear interface for users who want to bring their own dataset
+* Provide flexibility for users
+* Ease of maintenance by decoupling concerns
+
+## Considered Options
+
+1. Split out both dataset validation `mlwp-data-specs` and dataset loading `mlwp-data-loaders`
+2. Split out dataset-validation and loading but keep those two together
+
+## Decision Outcome
+
+Chosen option 1., because it allows for most flexibility and provides a clear entry point for users who want to bring their own loader.
+
+<!-- This is an optional element. Feel free to remove. -->
+### Consequences
+* Users can now validate their dataset with CLI
+* Simplification of the loader structure; only functions no classes
+* `mxalign` is now only responsible for the alignment tasks
+
+## More Information
+Currently the interface between dataset loaded with an `mlwp-data-loaders` loader and `mxalign` is not defined. Ideally `mxalign` should know the traits of the dataset to correctly align dataset. How do we inform `mxalign` on the traits? See [ADR-002](./ADR-002_mxalign-loader-interface.md) for possible options.

mxalign-0.1.0/doc/adr/ADR-002_mxalign-loader-interface.md

@@ -0,0 +1,74 @@
+---
+# These are optional metadata elements. Feel free to remove any of them.
+status: "{proposed | rejected | accepted | deprecated | … | superseded by ADR-0123"
+date: {YYYY-MM-DD when the decision was last updated}
+decision-makers: {list everyone involved in the decision}
+consulted: {list everyone whose opinions are sought (typically subject-matter experts); and with whom there is a two-way communication}
+informed: {list everyone who is kept up-to-date on progress; and with whom there is a one-way communication}
+---
+
+# {short title, representative of solved problem and found solution}
+
+## Context and Problem Statement
+
+{Describe the context and problem statement, e.g., in free form using two to three sentences or in the form of an illustrative story. You may want to articulate the problem in form of a question and add links to collaboration boards or issue management systems.}
+
+<!-- This is an optional element. Feel free to remove. -->
+## Decision Drivers
+
+* {decision driver 1, e.g., a force, facing concern, …}
+* {decision driver 2, e.g., a force, facing concern, …}
+* … <!-- numbers of drivers can vary -->
+
+## Considered Options
+
+* {title of option 1}
+* {title of option 2}
+* {title of option 3}
+* … <!-- numbers of options can vary -->
+
+## Decision Outcome
+
+Chosen option: "{title of option 1}", because {justification. e.g., only option, which meets k.o. criterion decision driver | which resolves force {force} | … | comes out best (see below)}.
+
+<!-- This is an optional element. Feel free to remove. -->
+### Consequences
+
+* Good, because {positive consequence, e.g., improvement of one or more desired qualities, …}
+* Bad, because {negative consequence, e.g., compromising one or more desired qualities, …}
+* … <!-- numbers of consequences can vary -->
+
+<!-- This is an optional element. Feel free to remove. -->
+### Confirmation
+
+{Describe how the implementation of/compliance with the ADR can/will be confirmed. Are the design that was decided for and its implementation in line with the decision made? E.g., a design/code review or a test with a library such as ArchUnit can help validate this. Not that although we classify this element as optional, it is included in many ADRs.}
+
+<!-- This is an optional element. Feel free to remove. -->
+## Pros and Cons of the Options
+
+### {title of option 1}
+
+<!-- This is an optional element. Feel free to remove. -->
+{example | description | pointer to more information | …}
+
+* Good, because {argument a}
+* Good, because {argument b}
+<!-- use "neutral" if the given argument weights neither for good nor bad -->
+* Neutral, because {argument c}
+* Bad, because {argument d}
+* … <!-- numbers of pros and cons can vary -->
+
+### {title of other option}
+
+{example | description | pointer to more information | …}
+
+* Good, because {argument a}
+* Good, because {argument b}
+* Neutral, because {argument c}
+* Bad, because {argument d}
+* …
+
+<!-- This is an optional element. Feel free to remove. -->
+## More Information
+
+{You might want to provide additional evidence/confidence for the decision outcome here and/or document the team agreement on the decision and/or define when/how this decision the decision should be realized and if/when it should be re-visited. Links to other decisions and resources might appear here as well.}

mxalign-0.1.0/doc/adr/template.md

@@ -0,0 +1,74 @@
+---
+# These are optional metadata elements. Feel free to remove any of them.
+status: "{proposed | rejected | accepted | deprecated | … | superseded by ADR-0123"
+date: {YYYY-MM-DD when the decision was last updated}
+decision-makers: {list everyone involved in the decision}
+consulted: {list everyone whose opinions are sought (typically subject-matter experts); and with whom there is a two-way communication}
+informed: {list everyone who is kept up-to-date on progress; and with whom there is a one-way communication}
+---
+
+# {short title, representative of solved problem and found solution}
+
+## Context and Problem Statement
+
+{Describe the context and problem statement, e.g., in free form using two to three sentences or in the form of an illustrative story. You may want to articulate the problem in form of a question and add links to collaboration boards or issue management systems.}
+
+<!-- This is an optional element. Feel free to remove. -->
+## Decision Drivers
+
+* {decision driver 1, e.g., a force, facing concern, …}
+* {decision driver 2, e.g., a force, facing concern, …}
+* … <!-- numbers of drivers can vary -->
+
+## Considered Options
+
+* {title of option 1}
+* {title of option 2}
+* {title of option 3}
+* … <!-- numbers of options can vary -->
+
+## Decision Outcome
+
+Chosen option: "{title of option 1}", because {justification. e.g., only option, which meets k.o. criterion decision driver | which resolves force {force} | … | comes out best (see below)}.
+
+<!-- This is an optional element. Feel free to remove. -->
+### Consequences
+
+* Good, because {positive consequence, e.g., improvement of one or more desired qualities, …}
+* Bad, because {negative consequence, e.g., compromising one or more desired qualities, …}
+* … <!-- numbers of consequences can vary -->
+
+<!-- This is an optional element. Feel free to remove. -->
+### Confirmation
+
+{Describe how the implementation of/compliance with the ADR can/will be confirmed. Are the design that was decided for and its implementation in line with the decision made? E.g., a design/code review or a test with a library such as ArchUnit can help validate this. Not that although we classify this element as optional, it is included in many ADRs.}
+
+<!-- This is an optional element. Feel free to remove. -->
+## Pros and Cons of the Options
+
+### {title of option 1}
+
+<!-- This is an optional element. Feel free to remove. -->
+{example | description | pointer to more information | …}
+
+* Good, because {argument a}
+* Good, because {argument b}
+<!-- use "neutral" if the given argument weights neither for good nor bad -->
+* Neutral, because {argument c}
+* Bad, because {argument d}
+* … <!-- numbers of pros and cons can vary -->
+
+### {title of other option}
+
+{example | description | pointer to more information | …}
+
+* Good, because {argument a}
+* Good, because {argument b}
+* Neutral, because {argument c}
+* Bad, because {argument d}
+* …
+
+<!-- This is an optional element. Feel free to remove. -->
+## More Information
+
+{You might want to provide additional evidence/confidence for the decision outcome here and/or document the team agreement on the decision and/or define when/how this decision the decision should be realized and if/when it should be re-visited. Links to other decisions and resources might appear here as well.}