aggmetrics 0.1.0.dev0__tar.gz

aggmetrics-0.1.0.dev0/LICENSE

@@ -0,0 +1,29 @@
+BSD 3-Clause License
+
+Copyright (c) 2026, Ronaldo C. Prati
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its contributors
+   may be used to endorse or promote products derived from this software without
+   specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

aggmetrics-0.1.0.dev0/PKG-INFO

@@ -0,0 +1,203 @@
+Metadata-Version: 2.4
+Name: aggmetrics
+Version: 0.1.0.dev0
+Summary: Aggregating classification performance measures for machine learning: indicator matrices and aggregation operators
+Author-email: "Ronaldo C. Prati" <ronaldo.prati@ufabc.edu.br>
+License: BSD-3-Clause
+Project-URL: Homepage, https://github.com/rcprati/aggmetrics
+Project-URL: Repository, https://github.com/rcprati/aggmetrics
+Project-URL: Documentation, https://aggmetrics.readthedocs.io
+Project-URL: Issues, https://github.com/rcprati/aggmetrics/issues
+Keywords: machine learning,classification,evaluation,performance measures,metrics,multilabel,multiclass,class imbalance,confusion matrix
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: BSD License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.20
+Provides-Extra: sklearn
+Requires-Dist: scikit-learn>=1.0; extra == "sklearn"
+Provides-Extra: frame
+Requires-Dist: pandas>=1.3; extra == "frame"
+Provides-Extra: test
+Requires-Dist: pytest>=7.0; extra == "test"
+Requires-Dist: scikit-learn>=1.0; extra == "test"
+Requires-Dist: pandas>=1.3; extra == "test"
+Provides-Extra: docs
+Requires-Dist: sphinx>=5.0; extra == "docs"
+Requires-Dist: furo; extra == "docs"
+Provides-Extra: all
+Requires-Dist: scikit-learn>=1.0; extra == "all"
+Requires-Dist: pandas>=1.3; extra == "all"
+Dynamic: license-file
+
+# aggmetrics
+
+**Aggregating classification performance measures for machine learning: indicator matrices and aggregation operators.**
+
+Any binary performance measure expressible in terms of the four confusion-matrix
+counts is extended to multiclass, multilabel, ordinal, cost-sensitive and
+soft-label settings by substituting one of a small set of aggregation operators.
+Micro-, macro- and exemplar-averaging (and pooling across cross-validation folds)
+become instances of a single construction rather than separate conventions.
+
+The package also reports when a measure is being used outside the range where it
+behaves as commonly assumed — which is the part that is hard to get from a
+formula library.
+
+```python
+import numpy as np
+from aggmetrics import IndicatorMatrix, diagnose
+
+y_true = np.array([0, 1, 2, 2, 1, 0])
+y_pred = np.array([0, 2, 2, 1, 1, 1])
+im = IndicatorMatrix(y_true, y_pred, task='multiclass')
+
+im.compute('f1', 'macro')          # 0.5222
+im.compute('f1', 'micro')          # 0.5   -- equals accuracy on single-label data
+im.compute('mcc', 'macro')         # macro-MCC: no standard implementation exists
+im.compute('informedness', 'micro')
+im.accuracy()                      # 0.5
+
+im.report()                        # DataFrame: measures x operators
+```
+
+## Installation
+
+```bash
+pip install aggmetrics                # core, requires only numpy
+pip install "aggmetrics[all]"         # adds scikit-learn and pandas support
+```
+
+## Why not just use `sklearn.metrics`?
+
+`sklearn` exposes four `average` values for a fixed list of measures. The
+correspondence is exact — `'micro'`, `'macro'`, `'weighted'` and `'samples'` are
+Σ₁, Σₘ with uniform weights, Σₘ with prevalence weights and Σₙ respectively — but
+the combinations available are a subset of those the construction generates, and
+nothing in the library tells you when a combination is misleading.
+
+Three things this package adds:
+
+**Any measure under any operator.** Macro-MCC, exemplar-Jaccard, micro-informedness,
+weighted G-mean. 21 binary measures × 4 operators, without special-casing.
+
+**Structural properties as first-class data.** Each measure records whether it is
+*aggregation-decomposable*, whether it is skew-invariant, and whether its value at
+`tp = 0` depends on `tn`. These determine which identities apply.
+
+```python
+from aggmetrics import get_measure, list_measures
+
+list_measures(decomposable=True)       # micro == denominator-weighted macro
+get_measure('mcc').decomposable        # False
+get_measure('recall').bayes_threshold()  # 0.0 -- maximised by predicting all positive
+get_measure('mcc').attainable_floor(10)  # -0.111, not -1
+```
+
+**The generative claim, as an operation.** Supply a binary formula; the properties
+that decide which identities apply are *derived*, not declared:
+
+```python
+from aggmetrics import register_measure, infer_properties
+
+def prevalence_threshold(tp, fp, fn, tn):
+    r, s = tp / (tp + fn), tn / (tn + fp)
+    return (np.sqrt(r * (1 - s)) + s - 1) / (r + s - 1)
+
+m = register_measure('prevalence_threshold', prevalence_threshold)
+m.decomposable          # False  -- inferred by fitting <a,v>/<b,v> and validating
+m.skew_invariant        # True   -- a function of recall and specificity
+m.bayes_threshold()     # None   -- no closed form outside the decomposable class
+
+im.compute('prevalence_threshold', 'exemplar')   # usable immediately
+```
+
+Inference is checked against the 21 measures whose properties are known
+analytically — `verify_inference()` — and all 21 agree, with recovered coefficients
+accurate to 4.2e-15. The check paid for itself: it found two errors in our own hand
+annotations (error rate and FPR both retain a `tn` dependence at `tp = 0`, so both
+*flatter* a wholly wrong classifier, reaching 2/m and 1/(m-1) respectively).
+
+**Diagnostics.** `diagnose()` flags the traps:
+
+```python
+y = np.arange(500) % 10
+yhat = (y + 1) % 10                    # every single prediction is wrong
+im = IndicatorMatrix(y, yhat, task='multiclass')
+
+im.compute('label_accuracy', 'micro')  # 0.800  (!)
+im.compute('mcc', 'micro')             # -0.111 (!)
+im.accuracy()                          # 0.000
+
+for f in diagnose(im):
+    print(f)
+# [warning] inflated_floor: 'label_accuracy' cannot fall below 0.8000 on this
+#           10-class one-hot problem even if every prediction is wrong ...
+# [note]    micro_collapse: micro-precision, micro-recall and micro-F1 are all
+#           equal to accuracy ...
+# [warning] degenerate_optimum: 'recall' is maximised at threshold 0.000 ...
+```
+
+Checks currently implemented: pooled value outside the per-class range;
+`tn`-dependent measures on one-hot multiclass data; micro-collapse on single-label
+data; undefined per-class entries and the size of the convention effect; measures
+whose optimal decision rule is not the Bayes rule for 0/1 loss.
+
+## Features
+
+- **21 binary measures**, resolvable by name or alias (`iou` → `jaccard`, `dice` → `f1`),
+  plus `register_measure` for your own, with properties inferred.
+- **Four operators**: `micro`, `macro`, `weighted`, `exemplar`.
+- **Explicit undefined-value conventions**: `exclude` (default), `zero`
+  (`sklearn`'s behaviour), `one`, `nan`. The choice is a parameter, not a default
+  buried in the implementation.
+- **Soft ground truth** via t-norms, with `marginal_defect()` showing that only the
+  product t-norm preserves the marginals.
+- **Cost matrices**, recovering MAE and MSE as special cases.
+- **Observation masks** for partial labels, semi-supervised evaluation and abstention.
+- **scikit-learn scorers** via `make_scorer`, usable in `cross_val_score` and
+  `GridSearchCV`.
+- **AI Agent Skills included** (`.agents/skills/aggmetrics-guide/`), providing machine-readable
+  runbooks, axiomatic invariants, and diagnostic protocols for AI coding assistants.
+
+## The test suite verifies the theory
+
+Each result of the companion paper is an executable test, so a change that breaks
+a theorem breaks a test:
+
+```bash
+python -m pytest tests/test_theory.py -v
+```
+
+| Test | Result verified |
+|---|---|
+| `test_theorem1_holds_for_decomposable` | micro = denominator-weighted macro, 15 measures × 3 samples |
+| `test_theorem1_fails_for_non_decomposable` | the identity has no extension beyond that class |
+| `test_theorem2_product_is_uniquely_marginally_consistent` | only the product t-norm preserves marginals |
+| `test_theorem5_micro_collapse` | micro-P = micro-R = micro-F₁ = accuracy, single-label |
+| `test_proposition5_thresholds` | Bayes threshold `(β+δ)/(α+β+γ+δ)` |
+| `test_proposition7_floors` | attainable floors under one-hot encoding |
+| `test_proposition2_recall_prior_free_specificity_not` | one-vs-rest skew behaviour |
+| `test_corollary5_gap_is_normalised_covariance` | the micro–macro gap identity |
+| `test_theorem3_mae_and_mse_as_cost_matrices` | ordinal losses as cost matrices |
+| `test_inference_reproduces_every_annotation` | inference recovers all 21 analytic annotations |
+
+138 tests, no skips.
+
+## Examples
+
+```bash
+python examples/quickstart.py
+```
+
+## Citing
+
+Please cite both the software and the companion paper; see `CITATION.cff`.
+
+## License
+
+BSD 3-Clause.

aggmetrics-0.1.0.dev0/README.md

@@ -0,0 +1,167 @@
+# aggmetrics
+
+**Aggregating classification performance measures for machine learning: indicator matrices and aggregation operators.**
+
+Any binary performance measure expressible in terms of the four confusion-matrix
+counts is extended to multiclass, multilabel, ordinal, cost-sensitive and
+soft-label settings by substituting one of a small set of aggregation operators.
+Micro-, macro- and exemplar-averaging (and pooling across cross-validation folds)
+become instances of a single construction rather than separate conventions.
+
+The package also reports when a measure is being used outside the range where it
+behaves as commonly assumed — which is the part that is hard to get from a
+formula library.
+
+```python
+import numpy as np
+from aggmetrics import IndicatorMatrix, diagnose
+
+y_true = np.array([0, 1, 2, 2, 1, 0])
+y_pred = np.array([0, 2, 2, 1, 1, 1])
+im = IndicatorMatrix(y_true, y_pred, task='multiclass')
+
+im.compute('f1', 'macro')          # 0.5222
+im.compute('f1', 'micro')          # 0.5   -- equals accuracy on single-label data
+im.compute('mcc', 'macro')         # macro-MCC: no standard implementation exists
+im.compute('informedness', 'micro')
+im.accuracy()                      # 0.5
+
+im.report()                        # DataFrame: measures x operators
+```
+
+## Installation
+
+```bash
+pip install aggmetrics                # core, requires only numpy
+pip install "aggmetrics[all]"         # adds scikit-learn and pandas support
+```
+
+## Why not just use `sklearn.metrics`?
+
+`sklearn` exposes four `average` values for a fixed list of measures. The
+correspondence is exact — `'micro'`, `'macro'`, `'weighted'` and `'samples'` are
+Σ₁, Σₘ with uniform weights, Σₘ with prevalence weights and Σₙ respectively — but
+the combinations available are a subset of those the construction generates, and
+nothing in the library tells you when a combination is misleading.
+
+Three things this package adds:
+
+**Any measure under any operator.** Macro-MCC, exemplar-Jaccard, micro-informedness,
+weighted G-mean. 21 binary measures × 4 operators, without special-casing.
+
+**Structural properties as first-class data.** Each measure records whether it is
+*aggregation-decomposable*, whether it is skew-invariant, and whether its value at
+`tp = 0` depends on `tn`. These determine which identities apply.
+
+```python
+from aggmetrics import get_measure, list_measures
+
+list_measures(decomposable=True)       # micro == denominator-weighted macro
+get_measure('mcc').decomposable        # False
+get_measure('recall').bayes_threshold()  # 0.0 -- maximised by predicting all positive
+get_measure('mcc').attainable_floor(10)  # -0.111, not -1
+```
+
+**The generative claim, as an operation.** Supply a binary formula; the properties
+that decide which identities apply are *derived*, not declared:
+
+```python
+from aggmetrics import register_measure, infer_properties
+
+def prevalence_threshold(tp, fp, fn, tn):
+    r, s = tp / (tp + fn), tn / (tn + fp)
+    return (np.sqrt(r * (1 - s)) + s - 1) / (r + s - 1)
+
+m = register_measure('prevalence_threshold', prevalence_threshold)
+m.decomposable          # False  -- inferred by fitting <a,v>/<b,v> and validating
+m.skew_invariant        # True   -- a function of recall and specificity
+m.bayes_threshold()     # None   -- no closed form outside the decomposable class
+
+im.compute('prevalence_threshold', 'exemplar')   # usable immediately
+```
+
+Inference is checked against the 21 measures whose properties are known
+analytically — `verify_inference()` — and all 21 agree, with recovered coefficients
+accurate to 4.2e-15. The check paid for itself: it found two errors in our own hand
+annotations (error rate and FPR both retain a `tn` dependence at `tp = 0`, so both
+*flatter* a wholly wrong classifier, reaching 2/m and 1/(m-1) respectively).
+
+**Diagnostics.** `diagnose()` flags the traps:
+
+```python
+y = np.arange(500) % 10
+yhat = (y + 1) % 10                    # every single prediction is wrong
+im = IndicatorMatrix(y, yhat, task='multiclass')
+
+im.compute('label_accuracy', 'micro')  # 0.800  (!)
+im.compute('mcc', 'micro')             # -0.111 (!)
+im.accuracy()                          # 0.000
+
+for f in diagnose(im):
+    print(f)
+# [warning] inflated_floor: 'label_accuracy' cannot fall below 0.8000 on this
+#           10-class one-hot problem even if every prediction is wrong ...
+# [note]    micro_collapse: micro-precision, micro-recall and micro-F1 are all
+#           equal to accuracy ...
+# [warning] degenerate_optimum: 'recall' is maximised at threshold 0.000 ...
+```
+
+Checks currently implemented: pooled value outside the per-class range;
+`tn`-dependent measures on one-hot multiclass data; micro-collapse on single-label
+data; undefined per-class entries and the size of the convention effect; measures
+whose optimal decision rule is not the Bayes rule for 0/1 loss.
+
+## Features
+
+- **21 binary measures**, resolvable by name or alias (`iou` → `jaccard`, `dice` → `f1`),
+  plus `register_measure` for your own, with properties inferred.
+- **Four operators**: `micro`, `macro`, `weighted`, `exemplar`.
+- **Explicit undefined-value conventions**: `exclude` (default), `zero`
+  (`sklearn`'s behaviour), `one`, `nan`. The choice is a parameter, not a default
+  buried in the implementation.
+- **Soft ground truth** via t-norms, with `marginal_defect()` showing that only the
+  product t-norm preserves the marginals.
+- **Cost matrices**, recovering MAE and MSE as special cases.
+- **Observation masks** for partial labels, semi-supervised evaluation and abstention.
+- **scikit-learn scorers** via `make_scorer`, usable in `cross_val_score` and
+  `GridSearchCV`.
+- **AI Agent Skills included** (`.agents/skills/aggmetrics-guide/`), providing machine-readable
+  runbooks, axiomatic invariants, and diagnostic protocols for AI coding assistants.
+
+## The test suite verifies the theory
+
+Each result of the companion paper is an executable test, so a change that breaks
+a theorem breaks a test:
+
+```bash
+python -m pytest tests/test_theory.py -v
+```
+
+| Test | Result verified |
+|---|---|
+| `test_theorem1_holds_for_decomposable` | micro = denominator-weighted macro, 15 measures × 3 samples |
+| `test_theorem1_fails_for_non_decomposable` | the identity has no extension beyond that class |
+| `test_theorem2_product_is_uniquely_marginally_consistent` | only the product t-norm preserves marginals |
+| `test_theorem5_micro_collapse` | micro-P = micro-R = micro-F₁ = accuracy, single-label |
+| `test_proposition5_thresholds` | Bayes threshold `(β+δ)/(α+β+γ+δ)` |
+| `test_proposition7_floors` | attainable floors under one-hot encoding |
+| `test_proposition2_recall_prior_free_specificity_not` | one-vs-rest skew behaviour |
+| `test_corollary5_gap_is_normalised_covariance` | the micro–macro gap identity |
+| `test_theorem3_mae_and_mse_as_cost_matrices` | ordinal losses as cost matrices |
+| `test_inference_reproduces_every_annotation` | inference recovers all 21 analytic annotations |
+
+138 tests, no skips.
+
+## Examples
+
+```bash
+python examples/quickstart.py
+```
+
+## Citing
+
+Please cite both the software and the companion paper; see `CITATION.cff`.
+
+## License
+
+BSD 3-Clause.

aggmetrics-0.1.0.dev0/aggmetrics/__init__.py

@@ -0,0 +1,74 @@
+# Copyright (c) 2026, Ronaldo C. Prati
+# SPDX-License-Identifier: BSD-3-Clause
+"""
+aggmetrics --- classification evaluation via indicator matrices.
+
+Any binary performance measure expressible in terms of the four confusion-matrix
+counts is extended to multiclass, multilabel, ordinal, cost-sensitive and
+soft-label settings by substituting one of three aggregation operators, so that
+micro-, macro- and exemplar-averaging are three instances of one construction
+rather than three separate conventions.
+
+Quick start
+-----------
+>>> import numpy as np
+>>> from aggmetrics import IndicatorMatrix
+>>> y_true = np.array([0, 1, 2, 2, 1, 0])
+>>> y_pred = np.array([0, 2, 2, 1, 1, 1])
+>>> im = IndicatorMatrix(y_true, y_pred, task='multiclass')
+>>> round(im.compute('f1', 'macro'), 4)
+0.5222
+>>> round(im.compute('f1', 'micro'), 4)
+0.5
+>>> round(im.accuracy(), 4)
+0.5
+
+Micro-F1 equals accuracy on single-label data, so reporting both adds nothing;
+macro-F1 differs because it weights the three classes equally.
+
+The package also reports when a measure is being used outside the range where it
+behaves as commonly assumed:
+
+>>> from aggmetrics import diagnose
+>>> codes = sorted({f.code for f in diagnose(im)})
+>>> 'micro_collapse' in codes
+True
+"""
+from __future__ import annotations
+
+__version__ = "0.1.0.dev0"
+__author__ = "Ronaldo C. Prati"
+__license__ = "BSD-3-Clause"
+
+from .aggregation import (aggregate, cells, fold_aggregate, sigma_1, sigma_k,
+                          sigma_m, sigma_n)
+from .core import IndicatorMatrix, pareto_dominance
+from .diagnostics import (Finding, MeasureWarning, check_hull, check_ovr_skew_drift,
+                          check_redundancy, diagnose, micro_macro_gap,
+                          uniform_error_deviation)
+from .inference import (InferredProperties, infer_properties, register_measure,
+                        verify_inference)
+from .measures import MEASURES, BinaryMeasure, get_measure, list_measures
+from .sklearn_compat import make_scorer, score, sklearn_average
+from .tnorms import TNORMS, get_tnorm, marginal_defect, soft_cells
+
+__all__ = [
+    "__version__",
+    # core
+    "IndicatorMatrix", "pareto_dominance",
+    # measures
+    "BinaryMeasure", "MEASURES", "get_measure", "list_measures",
+    # operators
+    "aggregate", "cells", "fold_aggregate", "sigma_1", "sigma_m", "sigma_n", "sigma_k",
+    # soft labels
+    "TNORMS", "get_tnorm", "marginal_defect", "soft_cells",
+    # diagnostics
+    "Finding", "MeasureWarning", "check_hull", "check_ovr_skew_drift",
+    "check_redundancy", "diagnose", "micro_macro_gap", "uniform_error_deviation",
+    # inference
+    "register_measure", "infer_properties", "verify_inference",
+    "InferredProperties",
+    # scikit-learn
+    "make_scorer", "score", "sklearn_average",
+]
+

aggmetrics-0.1.0.dev0/aggmetrics/aggregation.py

@@ -0,0 +1,210 @@
+# Copyright (c) 2026, Ronaldo C. Prati
+# SPDX-License-Identifier: BSD-3-Clause
+"""
+The three aggregation operators, plus the conventions they require.
+
+Given indicator matrices :math:`Y, \\hat{Y} \\in \\{0,1\\}^{n \\times m}`, the four
+cell matrices are formed entrywise and then reduced in one of three ways:
+
+===============  ==========================  ====================================
+Operator         Reduction                   Conventional name
+===============  ==========================  ====================================
+``sigma_1``      over all entries            micro
+``sigma_m``      column-wise, then averaged  macro (or weighted)
+``sigma_n``      row-wise, then averaged     exemplar (``average='samples'``)
+===============  ==========================  ====================================
+
+Column- and row-wise aggregation divide before averaging, so an individual entry
+can be undefined when its denominator vanishes. Because the resolution changes the
+reported value, it is an explicit parameter rather than an implicit default. The
+package defaults to ``'exclude'``, which averages over the defined entries only.
+"""
+from __future__ import annotations
+
+from typing import Literal, Optional, Tuple
+
+import numpy as np
+
+from .measures import BinaryMeasure, get_measure
+
+__all__ = ["cells", "sigma_1", "sigma_m", "sigma_n", "sigma_k",
+           "aggregate", "fold_aggregate",
+           "UNDEFINED_CONVENTIONS", "Aggregation"]
+
+Aggregation = Literal["micro", "macro", "weighted", "exemplar"]
+UNDEFINED_CONVENTIONS = ("exclude", "zero", "one", "nan")
+
+
+def _as2d(a) -> np.ndarray:
+    a = np.asarray(a)
+    return a.reshape(-1, 1) if a.ndim == 1 else a
+
+
+def cells(Y, P, mask=None) -> Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:
+    """Entrywise TP, FP, FN, TN matrices.
+
+    Works for crisp indicator matrices and, unchanged, for soft ones in
+    ``[0, 1]`` when they have already been combined by a t-norm (see
+    :mod:`aggmetrics.tnorms`).
+
+    ``mask`` is an optional observation mask: entries where ``mask == 0`` are
+    treated as unobserved and contribute nothing, which covers partial labels,
+    semi-supervised evaluation and masked-out abstentions.
+    """
+    Y, P = _as2d(Y).astype(float), _as2d(P).astype(float)
+    if Y.shape != P.shape:
+        raise ValueError(f"shape mismatch: Y is {Y.shape}, predictions are {P.shape}")
+    TP, FP, FN, TN = Y * P, (1 - Y) * P, Y * (1 - P), (1 - Y) * (1 - P)
+    if mask is not None:
+        M = _as2d(mask).astype(float)
+        if M.shape != Y.shape:
+            raise ValueError(f"mask shape {M.shape} does not match {Y.shape}")
+        TP, FP, FN, TN = TP * M, FP * M, FN * M, TN * M
+    return TP, FP, FN, TN
+
+
+def sigma_1(X: np.ndarray) -> float:
+    """Global aggregation: sum over all entries."""
+    return float(np.sum(X))
+
+
+def sigma_m(X: np.ndarray) -> np.ndarray:
+    """Column-wise aggregation: one value per class or label."""
+    return np.asarray(X).sum(axis=0)
+
+
+def sigma_n(X: np.ndarray) -> np.ndarray:
+    """Row-wise aggregation: one value per example."""
+    return np.asarray(X).sum(axis=1)
+
+
+def sigma_k(cell_list: Sequence[Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]]
+            ) -> Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]:
+    """Fold-wise aggregation across K cross-validation splits.
+
+    Sums the four cell matrices (TP, FP, FN, TN) across multiple folds.
+    """
+    if not cell_list:
+        raise ValueError("cell_list must not be empty")
+    TP = sum(c[0] for c in cell_list)
+    FP = sum(c[1] for c in cell_list)
+    FN = sum(c[2] for c in cell_list)
+    TN = sum(c[3] for c in cell_list)
+    return TP, FP, FN, TN
+
+
+def _combine(values: np.ndarray, convention: str,
+             weights: Optional[np.ndarray] = None) -> float:
+    values = np.asarray(values, dtype=float)
+    defined = ~np.isnan(values)
+    if convention == "nan":
+        return float(np.nan) if not defined.all() else float(np.average(
+            values, weights=weights))
+    if convention == "exclude":
+        if not defined.any():
+            return float("nan")
+        w = None if weights is None else np.asarray(weights, float)[defined]
+        return float(np.average(values[defined], weights=w))
+    if convention in ("zero", "one"):
+        filled = np.where(defined, values, 0.0 if convention == "zero" else 1.0)
+        return float(np.average(filled, weights=weights))
+    raise ValueError(f"convention must be one of {UNDEFINED_CONVENTIONS}, "
+                     f"got {convention!r}")
+
+
+def aggregate(Y, P, measure: str | BinaryMeasure, how: Aggregation = "micro",
+              *, convention: str = "exclude", mask=None,
+              return_parts: bool = False):
+    """Apply ``measure`` to ``(Y, P)`` under the operator ``how``.
+
+    Parameters
+    ----------
+    how
+        ``'micro'`` pools all entries before applying the measure; ``'macro'``
+        applies it per class and averages; ``'weighted'`` applies it per class and
+        averages with weights proportional to the class prevalence; ``'exemplar'``
+        applies it per example and averages.
+    convention
+        Treatment of undefined per-class or per-example entries. ``'exclude'``
+        averages over defined entries only. ``'zero'`` reproduces the behaviour of
+        ``sklearn``'s ``zero_division=0``.
+    return_parts
+        If True, also return the vector of per-class or per-example values, which
+        is what the diagnostics use to check whether a pooled value lies inside
+        their convex hull.
+    """
+    meas = get_measure(measure)
+    TP, FP, FN, TN = cells(Y, P, mask=mask)
+
+    if how == "micro":
+        value = meas(sigma_1(TP), sigma_1(FP), sigma_1(FN), sigma_1(TN))
+        return (value, None) if return_parts else value
+
+    if how in ("macro", "weighted"):
+        red, axis = sigma_m, 0
+    elif how == "exemplar":
+        red, axis = sigma_n, 1
+    else:
+        raise ValueError("how must be 'micro', 'macro', 'weighted' or 'exemplar'")
+
+    tp, fp, fn, tn = red(TP), red(FP), red(FN), red(TN)
+    parts = np.array([meas(*c) for c in zip(tp, fp, fn, tn)], dtype=float)
+
+    weights = None
+    if how == "weighted":
+        weights = tp + fn                      # class prevalence
+        if float(np.sum(weights)) <= 0:
+            weights = None
+
+    value = _combine(parts, convention, weights)
+    return (value, parts) if return_parts else value
+
+
+def fold_aggregate(im_list: Sequence, measure: str | BinaryMeasure,
+                   how: Aggregation = "macro",
+                   pooling: Literal["pooled", "averaged"] = "pooled",
+                   *, convention: str = "exclude") -> float:
+    """Aggregate a performance measure across K cross-validation folds.
+
+    Parameters
+    ----------
+    im_list
+        Sequence of :class:`~aggmetrics.core.IndicatorMatrix` instances, one per fold.
+    measure
+        The performance measure to compute.
+    how
+        Intra-fold aggregation operator: ``'micro'``, ``'macro'``, or ``'exemplar'``.
+    pooling
+        ``'pooled'`` applies the $\\Upsigma_{\\mathrm{k}}$ reduction over fold cell matrices
+        prior to computing the non-linear ratio; ``'averaged'`` computes the measure
+        separately in each fold and returns the sample-weighted mean.
+    convention
+        Convention for undefined entries (``'exclude'``, ``'zero'``, etc.).
+    """
+    if not im_list:
+        raise ValueError("im_list must not be empty")
+    meas = get_measure(measure)
+
+    if pooling == "averaged":
+        vals = [im.compute(meas, how, convention=convention) for im in im_list]
+        weights = [im.n for im in im_list]
+        return float(np.average(vals, weights=weights))
+
+    if pooling == "pooled":
+        cell_tuples = [im.cells for im in im_list]
+        TP, FP, FN, TN = sigma_k(cell_tuples)
+        if how == "micro":
+            return float(meas(sigma_1(TP), sigma_1(FP), sigma_1(FN), sigma_1(TN)))
+        if how in ("macro", "weighted"):
+            tp, fp, fn, tn = sigma_m(TP), sigma_m(FP), sigma_m(FN), sigma_m(TN)
+            parts = np.array([meas(*c) for c in zip(tp, fp, fn, tn)], dtype=float)
+            weights = (tp + fn) if how == "weighted" else None
+            return _combine(parts, convention, weights)
+        if how == "exemplar":
+            tp, fp, fn, tn = sigma_n(TP), sigma_n(FP), sigma_n(FN), sigma_n(TN)
+            parts = np.array([meas(*c) for c in zip(tp, fp, fn, tn)], dtype=float)
+            return _combine(parts, convention)
+        raise ValueError("how must be 'micro', 'macro', 'weighted' or 'exemplar'")
+
+    raise ValueError("pooling must be 'pooled' or 'averaged'")
+