eval-toolkit 0.40.0__tar.gz → 0.42.0__tar.gz

{eval_toolkit-0.40.0 → eval_toolkit-0.42.0}/CHANGELOG.md

@@ -7,6 +7,136 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.42.0] — 2026-05-19 — fit_isotonic_binary completes 4-calibrator family (closes #44)
+
+Final element of the binary scalar-prob calibrator family started by
+`fit_temperature_binary` (v0.35.0). All four now uniformly return
+`(params, apply)`:
+
+| Function | Params | Shipped |
+|---|---|---|
+| `fit_temperature_binary` | `(T,)` — single float | v0.35.0 |
+| `fit_isotonic_binary`    | `None` — non-parametric | **v0.42.0** |
+| `fit_platt_binary`       | `(a, b)` | v0.40.0 |
+| `fit_beta_binary`        | `(a, b, c)` | v0.40.0 |
+
+Consumer code can now iterate the family with a single shape, used
+to distinguish parametric from non-parametric via
+`if params is not None`:
+
+```text
+CALIBRATORS = {
+    "temperature": fit_temperature_binary,
+    "isotonic":    fit_isotonic_binary,
+    "platt":       fit_platt_binary,
+    "beta":        fit_beta_binary,
+}
+for name, fit_fn in CALIBRATORS.items():
+    params, apply = fit_fn(y_val, p_val)
+    calibrated = apply(p_test)
+    if params is not None:
+        manifest.record(f"{name}_params", params)
+```
+
+This matches the consumer's calibration-battery pattern in
+`prompt-injection-detection-prototype` (their ADR-056 supersedes
+ADR-023 to adopt the canonical `(params, apply)` shape across the
+full 4-calibrator audit battery).
+
+### Added
+
+- **`eval_toolkit.fit_isotonic_binary(y_true, y_score) -> (None,
+  apply)`** — thin wrapper over `fit_isotonic_calibrator`. The
+  `None` in the params slot encodes "non-parametric" (isotonic
+  regression is a monotone step function, no scalar params to log).
+- 6 new unit tests in `tests/test_calibration_binary_adapters.py`
+  including a 4-calibrator family-iteration integration test that
+  verifies the `None`-vs-tuple convention.
+
+### Protocol stability
+
+Additive only. No Tier-2 Protocol shape edits. v0.42 is minor 3 of
+consecutive-without-Protocol-changes (v0.40 + v0.41 + v0.42). Gate 2
+stays MET.
+
+## [0.41.0] — 2026-05-18 — Croissant end-to-end (closes #42, v1.0 Gate 4 MET)
+
+Closes v1.0 readiness Gate 4 — "Croissant interop verified end-to-end."
+`HFDatasetsLoader.describe()` now fetches per-file `sha256` hashes
+from HF Hub and exposes them in `distribution[].sha256`. The
+integration test (`tests/test_croissant_e2e.py`) downloads a real
+parquet shard from `stanfordnlp/sst2` and verifies the bytes hash
+bit-exactly to the value `describe()` reports.
+
+### Added
+
+- **`HFDatasetsLoader.describe()` Croissant + tree-API enrichment.**
+  When `fetch_remote_metadata=True` (default), the loader fetches from
+  two HF Hub endpoints:
+  - `/api/datasets/{repo}/croissant` — JSON-LD metadata (name,
+    description, license, citeAs, schema).
+  - `/api/datasets/{repo}/tree/refs%2Fconvert%2Fparquet?recursive=true`
+    — per-file `sha256` (read from each file's `lfs.oid` field — the
+    git-LFS content hash, equal to `sha256sum` of the raw bytes).
+  Caller-provided fields (`name=`, `cite_as=`, etc.) win over
+  Croissant fetches; Croissant fills only gaps. Network failures
+  degrade gracefully (warning emitted; sha256 empty as in pre-v0.41).
+- **`fetch_remote_metadata: bool = True`** constructor field on
+  `HFDatasetsLoader`. Set `False` for offline / unit-test paths.
+- **`tests/test_croissant_e2e.py`** — 5 integration tests against
+  live HF Hub:
+  1. `describe()` returns real `sha256:<64-hex>` per shard.
+  2. **Bit-exact verification**: download shard from `contentUrl`,
+     hash bytes, assert equals `describe()`'s sha256. This is the
+     literal v1.0 Gate 4 check.
+  3. Croissant metadata enriches name/citeAs/license/description.
+  4. Caller overrides win over remote.
+  5. `fetch_remote_metadata=False` preserves v0.40 behavior.
+  All pass against `stanfordnlp/sst2` (~3 MB train shard).
+- **New `integration` pytest marker** for network-dependent tests.
+  Excluded from `make coverage` (PR CI); runs explicitly via
+  `pytest -m integration`.
+
+### Why dual-sourced
+
+HF Hub's Croissant emitter currently fills `distribution[].sha256`
+with a placeholder URL pointing at MLCommons Croissant spec issue
+[#80](https://github.com/mlcommons/croissant/issues/80) ("In
+<Download>, check SHA256 or MD5"), which is **open**. The Croissant
+spec doesn't yet require per-file checksums from emitters; HF Hub is
+honest and punts the field. The authoritative hash IS available via
+HF Hub's tree API: `lfs.oid` is precisely sha256 of the file content
+(verified bit-exact via `sha256sum`).
+
+When MLCommons #80 resolves and HF Hub starts populating Croissant
+`sha256` with real values (which will equal the existing `lfs.oid`),
+the loader's source switches in ~5 LOC. Same downstream contract.
+
+### Documentation
+
+- `docs/source/methodology/reproducibility.md` §"Croissant
+  interoperability": replaces v0.7-era "subset" framing with the
+  end-to-end-verified narrative + dual-source rationale.
+- `docs/source/roadmap.md` §"v1.0.0 path":
+  - **Gate 2 (Protocol stability) ✅ MET** — v0.41 = minor 2 of 2
+    without Protocol shape edits (v0.40 fit_*_binary additives +
+    v0.41 HFDatasetsLoader enrichment leave Tier-2 Protocols
+    untouched).
+  - **Gate 4 (Croissant end-to-end) ✅ MET** — with dual-source caveat
+    documented; one-line migration path when MLCommons #80 resolves.
+
+### v1.0 readiness state after v0.41.0
+
+- Gate 1 (real consumer ≥1 review cycle on v0.7+): partial — consumer
+  pinned to v0.34.0; needs bump + cycle. **External**.
+- Gate 2 ✅ MET (v0.41 is minor 2 of 2 stable).
+- Gate 3 (methodology peer review): not met — needs external reader.
+  **External**.
+- Gate 4 ✅ MET — see this release.
+
+Two of four gates closed in-repo. The remaining two require external
+coordination (consumer review cycle, methodology peer reviewer).
+
 ## [0.40.0] — 2026-05-18 — fit_platt_binary + fit_beta_binary (closes #43)
 
 Completes the binary scalar-prob calibrator family started in v0.35.0

{eval_toolkit-0.40.0 → eval_toolkit-0.42.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: eval-toolkit
-Version: 0.40.0
+Version: 0.42.0
 Summary: Reusable evaluation contracts for binary classification: metrics, bootstrap CIs, calibration, artifacts, and evidence gates.
 Project-URL: Homepage, https://github.com/brandon-behring/eval-toolkit
 Project-URL: Documentation, https://brandon-behring.github.io/eval-toolkit/

{eval_toolkit-0.40.0 → eval_toolkit-0.42.0}/pyproject.toml

@@ -188,6 +188,7 @@ markers = [
     "slow: Tests > 2s (bootstrap-t studentized, multi-seed K-fold). Opt out with `pytest -m 'not slow'`.",
     "monte_carlo: Monte Carlo calibration suite (~14 min). Skipped in PR CI; runs only in the nightly-mc workflow via `-m monte_carlo`.",
     "benchmark: pytest-benchmark perf-regression tests on math kernels. Skipped in PR CI; runs in the nightly-benchmarks workflow via `-m benchmark`. Per v0.29.0 plan Tier γ #1.",
+    "integration: Network-dependent integration tests (HF Hub API, Croissant endpoints, etc.). Excluded from PR CI to avoid network-flake; runs in nightly. Opt in via `-m integration`. Added v0.41.0 (#42 Croissant Gate 4).",
 ]
 
 [tool.coverage.run]

{eval_toolkit-0.40.0 → eval_toolkit-0.42.0}/src/eval_toolkit/__init__.py

@@ -85,6 +85,7 @@ _EXPORTS: dict[str, str] = {
     "bayes_optimal_threshold": "eval_toolkit.calibration",
     "fit_beta_binary": "eval_toolkit.calibration",
     "fit_beta_calibrator": "eval_toolkit.calibration",
+    "fit_isotonic_binary": "eval_toolkit.calibration",
     "fit_isotonic_calibrator": "eval_toolkit.calibration",
     "fit_platt_binary": "eval_toolkit.calibration",
     "fit_platt_calibrator": "eval_toolkit.calibration",

{eval_toolkit-0.40.0 → eval_toolkit-0.42.0}/src/eval_toolkit/_version.py

@@ -2,4 +2,4 @@
 
 __all__ = ["__version__"]
 
-__version__ = "0.40.0"
+__version__ = "0.42.0"

{eval_toolkit-0.40.0 → eval_toolkit-0.42.0}/src/eval_toolkit/calibration.py

@@ -55,6 +55,7 @@ __all__ = [
     "bayes_optimal_threshold",
     "fit_beta_binary",
     "fit_beta_calibrator",
+    "fit_isotonic_binary",
     "fit_isotonic_calibrator",
     "fit_platt_binary",
     "fit_platt_calibrator",
@@ -1294,6 +1295,83 @@ def fit_beta_binary(
     return (a, b, c), apply
 
 
+def fit_isotonic_binary(
+    y_true: np.ndarray, y_score: np.ndarray
+) -> tuple[None, Callable[[np.ndarray], np.ndarray]]:
+    r"""Binary-probability adapter for :func:`fit_isotonic_calibrator`.
+
+    Mirror of :func:`fit_temperature_binary` / :func:`fit_platt_binary`
+    / :func:`fit_beta_binary`: returns ``(None, apply)``. Isotonic
+    regression is non-parametric — there are no introspectable scalar
+    parameters to log alongside the apply callable — so the params
+    slot is :obj:`None`.
+
+    The ``None``-in-params slot makes "non-parametric" unambiguous
+    while preserving the canonical ``(params_tuple, apply)`` shape
+    shared by the four binary scalar-prob calibrators. Consumer code
+    can iterate over the full family with one idiom:
+
+    .. code-block:: text
+
+        CALIBRATORS = {
+            "temperature": fit_temperature_binary,
+            "isotonic":    fit_isotonic_binary,
+            "platt":       fit_platt_binary,
+            "beta":        fit_beta_binary,
+        }
+        for name, fit_fn in CALIBRATORS.items():
+            params, apply = fit_fn(y_val, p_val)
+            calibrated = apply(p_test)
+            if params is not None:
+                manifest.record(f"{name}_params", params)
+
+    Added v0.42.0 (closes #44) to complete the binary scalar-prob
+    calibrator family started by ``fit_temperature_binary`` (v0.35.0).
+
+    Parameters
+    ----------
+    y_true : np.ndarray, shape (n,)
+        Binary validation labels in ``{0, 1}``.
+    y_score : np.ndarray, shape (n,)
+        Validation predicted probabilities of class 1, in [0, 1].
+
+    Returns
+    -------
+    tuple
+        ``(None, apply)`` — ``None`` in the params slot (isotonic is
+        non-parametric); ``apply`` maps probabilities through the
+        fitted monotonic step function.
+
+    Raises
+    ------
+    ValueError
+        On shape mismatch, empty input, non-finite scores, or
+        single-class ``y_true`` (propagated from
+        :func:`fit_isotonic_calibrator`).
+
+    Examples
+    --------
+    >>> import numpy as np
+    >>> rng = np.random.default_rng(0)
+    >>> n = 500
+    >>> y_val = rng.binomial(1, 0.3, size=n).astype(int)
+    >>> p_val = np.clip(y_val * 0.6 + rng.normal(0, 0.2, n), 0.01, 0.99)
+    >>> params, apply = fit_isotonic_binary(y_val, p_val)
+    >>> params is None
+    True
+    >>> apply(np.array([0.1, 0.5, 0.9])).shape == (3,)
+    True
+
+    See Also
+    --------
+    fit_isotonic_calibrator : underlying non-parametric fitter.
+    fit_temperature_binary : 1-parameter sibling.
+    fit_platt_binary : 2-parameter sibling.
+    fit_beta_binary : 3-parameter sibling.
+    """
+    return None, fit_isotonic_calibrator(y_true, y_score)
+
+
 def fit_temperature_oracle(
     y_true: np.ndarray, y_score: np.ndarray
 ) -> tuple[float, Callable[[np.ndarray], np.ndarray]]:

{eval_toolkit-0.40.0 → eval_toolkit-0.42.0}/src/eval_toolkit/loaders.py

@@ -29,7 +29,9 @@ References
 from __future__ import annotations
 
 import glob as _glob
+import json as _json
 import logging
+import urllib.request as _urlrequest
 from collections.abc import Mapping, Sequence
 from dataclasses import dataclass
 from pathlib import Path
@@ -38,6 +40,24 @@ from typing import TYPE_CHECKING, Any, Protocol, cast, runtime_checkable
 from eval_toolkit.harness import EvalSlice
 from eval_toolkit.provenance import file_sha256
 
+_HF_HUB_BASE = "https://huggingface.co"
+_HF_FETCH_TIMEOUT_SEC = 15
+
+
+def _hf_get_json(path: str) -> Any:
+    """GET ``https://huggingface.co{path}`` and return parsed JSON.
+
+    Stdlib-only (no ``requests`` / ``huggingface_hub`` dep). Raises
+    ``OSError`` (urllib error) or ``ValueError`` (JSON decode) on
+    failure — callers catch both. The 15-second timeout caps any one
+    fetch so CI doesn't hang on a slow HF Hub.
+    """
+    url = f"{_HF_HUB_BASE}{path}"
+    req = _urlrequest.Request(url, headers={"User-Agent": "eval-toolkit"})
+    with _urlrequest.urlopen(req, timeout=_HF_FETCH_TIMEOUT_SEC) as resp:
+        return _json.loads(resp.read().decode("utf-8"))
+
+
 _logger = logging.getLogger(__name__)
 
 if TYPE_CHECKING:
@@ -367,10 +387,22 @@ class HFDatasetsLoader:
     raised at :meth:`load_splits` time with a clear install hint. This is
     intentional — eval-toolkit's core deps are numpy / scipy / sklearn only.
 
+    Since v0.41.0, :meth:`describe` enriches its output with per-file
+    ``sha256`` hashes fetched from the HF Hub tree API (the ``lfs.oid``
+    field), plus Croissant metadata fetched from HF Hub's Croissant
+    endpoint. The dual-source design is documented in
+    ``methodology/reproducibility.md`` §"Croissant interoperability";
+    in short: HF Hub's Croissant emitter currently punts the
+    ``distribution[].sha256`` field per MLCommons Croissant issue #80
+    (open), so we read the authoritative sha256 from the tree API's
+    ``lfs.oid`` instead. When #80 resolves and HF Hub starts populating
+    Croissant ``sha256`` with real values, the implementation collapses
+    to a single source.
+
     Parameters
     ----------
     repo_id : str
-        HuggingFace dataset repo, e.g. ``"deepset/prompt-injections"``.
+        HuggingFace dataset repo, e.g. ``"stanfordnlp/sst2"``.
     splits : sequence of str or None, optional
         Subset of HF splits to load. ``None`` = every split the repo defines.
     feature_col : str, optional
@@ -381,7 +413,12 @@ class HFDatasetsLoader:
     config_name : str or None, optional
         HF dataset config name (some datasets have multiple configs).
     name, description, cite_as, license, url : str, optional
-        Croissant metadata fields.
+        Croissant metadata overrides. If empty, :meth:`describe` will
+        fall back to fetching from HF Hub's Croissant endpoint.
+    fetch_remote_metadata : bool, optional
+        If ``True`` (default), :meth:`describe` fetches Croissant + tree
+        metadata from HF Hub. Set ``False`` to disable network calls
+        (useful for offline / unit testing).
     """
 
     repo_id: str
@@ -395,6 +432,7 @@ class HFDatasetsLoader:
     cite_as: str = ""
     license: str = ""
     url: str = ""
+    fetch_remote_metadata: bool = True
 
     def _load_dataset(self) -> Mapping[str, Any]:
         """Soft-import ``datasets`` and return the loaded DatasetDict.
@@ -447,20 +485,118 @@ class HFDatasetsLoader:
         return out
 
     def describe(self) -> dict[str, object]:
-        """Croissant-subset metadata pointing at the HF repo (no file hashes — HF caches)."""
-        return {
-            "name": self.name or self.repo_id,
-            "description": self.description,
-            "citeAs": self.cite_as,
-            "license": self.license,
-            "url": self.url or f"https://huggingface.co/datasets/{self.repo_id}",
-            "distribution": [
+        """Croissant-compatible metadata + per-file sha256 from HF Hub.
+
+        When ``fetch_remote_metadata=True`` (default), enriches the
+        baseline metadata with two HF Hub API fetches:
+
+        - **Croissant endpoint** (``/api/datasets/{repo}/croissant``) —
+          provides ``name``, ``description``, ``citeAs``, ``license``,
+          ``url`` defaults when the loader's fields are empty.
+        - **Tree API** (``/api/datasets/{repo}/tree/...?recursive=true``) —
+          provides per-file ``sha256`` (from ``lfs.oid``) and
+          ``contentSize`` for each parquet shard under the
+          ``refs/convert/parquet`` branch.
+
+        Network failures degrade gracefully (warning emitted; sha256
+        empty as in pre-v0.41 behavior). See class docstring for the
+        dual-source rationale (MLCommons Croissant issue #80).
+        """
+        remote_meta: dict[str, object] = {}
+        distribution: list[dict[str, object]] = []
+        if self.fetch_remote_metadata:
+            remote_meta = self._fetch_croissant_metadata_safe()
+            distribution = self._fetch_tree_distribution_safe()
+
+        # Caller-provided fields win; Croissant fills gaps.
+        def _pick(local: str, key: str) -> str:
+            if local:
+                return local
+            val = remote_meta.get(key)
+            return val if isinstance(val, str) else ""
+
+        if not distribution:
+            distribution = [
                 {
                     "name": f"hf:{self.repo_id}",
                     "contentUrl": f"https://huggingface.co/datasets/{self.repo_id}",
-                    "sha256": "",  # HF cache hash not exposed via the public API
+                    "sha256": "",
                     "contentSize": 0,
                 }
-            ],
+            ]
+
+        return {
+            "name": _pick(self.name, "name") or self.repo_id,
+            "description": _pick(self.description, "description"),
+            "citeAs": _pick(self.cite_as, "citeAs"),
+            "license": _pick(self.license, "license"),
+            "url": self.url or f"https://huggingface.co/datasets/{self.repo_id}",
+            "distribution": distribution,
             "config_name": self.config_name,
         }
+
+    def _fetch_croissant_metadata_safe(self) -> dict[str, object]:
+        """Fetch HF Hub Croissant JSON-LD; return empty dict on any failure."""
+        try:
+            data = _hf_get_json(f"/api/datasets/{self.repo_id}/croissant")
+            return data if isinstance(data, dict) else {}
+        except (OSError, ValueError) as exc:  # urllib.URLError, JSONDecodeError, etc.
+            _logger.warning(
+                "HFDatasetsLoader %s: Croissant fetch failed (%s); proceeding without",
+                self.repo_id,
+                exc,
+            )
+            return {}
+
+    def _fetch_tree_distribution_safe(self) -> list[dict[str, object]]:
+        """Fetch HF Hub tree API for the parquet-convert branch; return ``cr:FileObject`` entries.
+
+        Each entry carries ``sha256`` (from ``lfs.oid`` — the git-LFS
+        content hash, equal to ``sha256sum`` of the file content) and
+        ``contentSize`` (from the tree response's ``size`` field).
+
+        Falls back to an empty list on any failure — callers should
+        treat empty distribution as "no remote provenance available."
+        """
+        # HF stores native parquet (or auto-converts) under
+        # refs/convert/parquet; that's the canonical hash target.
+        path = f"/api/datasets/{self.repo_id}/tree/refs%2Fconvert%2Fparquet?recursive=true"
+        try:
+            entries = _hf_get_json(path)
+        except (OSError, ValueError) as exc:
+            _logger.warning(
+                "HFDatasetsLoader %s: tree-API fetch failed (%s); sha256 unavailable",
+                self.repo_id,
+                exc,
+            )
+            return []
+        if not isinstance(entries, list):
+            return []
+        out: list[dict[str, object]] = []
+        for entry in entries:
+            if not isinstance(entry, dict):
+                continue
+            if entry.get("type") != "file":
+                continue
+            path_val = entry.get("path", "")
+            if not isinstance(path_val, str) or not path_val.endswith(".parquet"):
+                continue
+            lfs = entry.get("lfs")
+            sha = ""
+            if isinstance(lfs, dict):
+                oid = lfs.get("oid")
+                if isinstance(oid, str) and len(oid) == 64:  # sha256 hex
+                    sha = f"sha256:{oid}"
+            size = entry.get("size", 0)
+            out.append(
+                {
+                    "name": path_val,
+                    "contentUrl": (
+                        f"https://huggingface.co/datasets/{self.repo_id}"
+                        f"/resolve/refs%2Fconvert%2Fparquet/{path_val}"
+                    ),
+                    "sha256": sha,
+                    "contentSize": int(size) if isinstance(size, (int, float)) else 0,
+                }
+            )
+        return out

{eval_toolkit-0.40.0 → eval_toolkit-0.42.0}/tests/golden/public_api/snapshot.json

@@ -136,6 +136,7 @@
     "file_sha256",
     "fit_beta_binary",
     "fit_beta_calibrator",
+    "fit_isotonic_binary",
     "fit_isotonic_calibrator",
     "fit_operating_points",
     "fit_platt_binary",
@@ -553,7 +554,7 @@
       ],
       "doc_first_line": "Load a HuggingFace ``datasets`` repo as ``{split: EvalSlice}``.",
       "kind": "class",
-      "signature": "(repo_id: 'str', splits: 'Sequence[str] | None' = None, feature_col: 'str' = 'text', label_col: 'str' = 'label', strata_col: 'str | None' = None, config_name: 'str | None' = None, name: 'str' = '', description: 'str' = '', cite_as: 'str' = '', license: 'str' = '', url: 'str' = '') -> None"
+      "signature": "(repo_id: 'str', splits: 'Sequence[str] | None' = None, feature_col: 'str' = 'text', label_col: 'str' = 'label', strata_col: 'str | None' = None, config_name: 'str | None' = None, name: 'str' = '', description: 'str' = '', cite_as: 'str' = '', license: 'str' = '', url: 'str' = '', fetch_remote_metadata: 'bool' = True) -> None"
     },
     "HoldoutSplitter": {
       "bases": [
@@ -1036,7 +1037,7 @@
       "doc_first_line": "str(object='') -> str",
       "kind": "value",
       "type": "str",
-      "value": "'0.40.0'"
+      "value": "'0.42.0'"
     },
     "apply_operating_points": {
       "doc_first_line": "Apply fitted thresholds to a mixed-class or single-class target slice.",
@@ -1208,6 +1209,11 @@
       "kind": "function",
       "signature": "(y_true: 'np.ndarray', y_score: 'np.ndarray') -> 'Callable[[np.ndarray], np.ndarray]'"
     },
+    "fit_isotonic_binary": {
+      "doc_first_line": "Binary-probability adapter for :func:`fit_isotonic_calibrator`.",
+      "kind": "function",
+      "signature": "(y_true: 'np.ndarray', y_score: 'np.ndarray') -> 'tuple[None, Callable[[np.ndarray], np.ndarray]]'"
+    },
     "fit_isotonic_calibrator": {
       "doc_first_line": "Niculescu-Mizil & Caruana 2005 [#nm05]_ isotonic regression.",
       "kind": "function",

{eval_toolkit-0.40.0 → eval_toolkit-0.42.0}/tests/test_calibration_binary_adapters.py

@@ -12,6 +12,8 @@ import pytest
 from eval_toolkit import (
     fit_beta_binary,
     fit_beta_calibrator,
+    fit_isotonic_binary,
+    fit_isotonic_calibrator,
     fit_platt_binary,
     fit_platt_calibrator,
     fit_temperature_binary,
@@ -160,6 +162,66 @@ def test_beta_binary_apply_rejects_non_finite_test_scores(
         apply(np.array([0.5, np.nan, 0.8]))
 
 
+# ---------- fit_isotonic_binary ----------
+
+
+@pytest.mark.unit
+def test_isotonic_binary_returns_none_params_and_apply(
+    synthetic_binary_data: tuple[np.ndarray, np.ndarray],
+) -> None:
+    """Isotonic is non-parametric → params slot is None."""
+    y, p = synthetic_binary_data
+    params, apply = fit_isotonic_binary(y, p)
+    assert params is None
+    assert callable(apply)
+
+
+@pytest.mark.unit
+def test_isotonic_binary_apply_returns_same_shape(
+    synthetic_binary_data: tuple[np.ndarray, np.ndarray],
+) -> None:
+    y, p = synthetic_binary_data
+    _, apply = fit_isotonic_binary(y, p)
+    test = np.array([0.1, 0.5, 0.9])
+    out = apply(test)
+    assert out.shape == test.shape
+    assert (out >= 0.0).all() and (out <= 1.0).all()
+
+
+@pytest.mark.unit
+def test_isotonic_binary_apply_matches_underlying_calibrator(
+    synthetic_binary_data: tuple[np.ndarray, np.ndarray],
+) -> None:
+    """fit_isotonic_binary apply should match fit_isotonic_calibrator output."""
+    y, p = synthetic_binary_data
+    _, apply = fit_isotonic_binary(y, p)
+    canonical_apply = fit_isotonic_calibrator(y, p)
+    test = np.linspace(0.05, 0.95, 20)
+    np.testing.assert_allclose(apply(test), canonical_apply(test))
+
+
+@pytest.mark.unit
+def test_isotonic_binary_rejects_single_class() -> None:
+    y_single = np.zeros(50, dtype=int)
+    p = np.random.default_rng(0).uniform(0.0, 1.0, 50)
+    with pytest.raises(ValueError):
+        fit_isotonic_binary(y_single, p)
+
+
+@pytest.mark.unit
+def test_isotonic_binary_monotone(
+    synthetic_binary_data: tuple[np.ndarray, np.ndarray],
+) -> None:
+    """Isotonic regression is monotone non-decreasing in the score."""
+    y, p = synthetic_binary_data
+    _, apply = fit_isotonic_binary(y, p)
+    test = np.linspace(0.05, 0.95, 50)
+    out = apply(test)
+    # Allow tiny numerical noise but enforce non-decreasing trend
+    deltas = np.diff(out)
+    assert (deltas >= -1e-12).all(), "isotonic should be non-decreasing"
+
+
 # ---------- consistency across the calibrator family ----------
 
 
@@ -167,14 +229,15 @@ def test_beta_binary_apply_rejects_non_finite_test_scores(
 def test_all_four_binary_adapters_have_consistent_shape(
     synthetic_binary_data: tuple[np.ndarray, np.ndarray],
 ) -> None:
-    """temperature_binary, platt_binary, beta_binary all return ``(params, apply)``.
+    """temperature, isotonic, platt, beta all return ``(params, apply)``.
 
     Documents the audit-battery contract for the 4-calibrator pattern.
     """
     y, p = synthetic_binary_data
-    # All return (params, apply); apply is a callable taking (n,) -> (n,)
+    # All return (params, apply); apply is a callable taking (n,) -> (n,).
     for name, fitter in [
         ("temperature", fit_temperature_binary),
+        ("isotonic", fit_isotonic_binary),
         ("platt", fit_platt_binary),
         ("beta", fit_beta_binary),
     ]:
@@ -185,6 +248,38 @@ def test_all_four_binary_adapters_have_consistent_shape(
         assert (out >= 0.0).all() and (out <= 1.0).all(), f"{name}: output not in [0,1]"
 
 
+@pytest.mark.unit
+def test_consumer_idiom_iterating_all_four_calibrators(
+    synthetic_binary_data: tuple[np.ndarray, np.ndarray],
+) -> None:
+    """End-to-end consumer idiom: iterate the 4-element family with one shape.
+
+    Matches the calibration-battery pattern in
+    ``prompt-injection-detection-prototype/src/eval/calibration_battery.py``
+    (ADR-056). The ``params is not None`` check distinguishes parametric
+    from non-parametric in a single conditional.
+    """
+    y, p = synthetic_binary_data
+    fitters = {
+        "temperature": fit_temperature_binary,
+        "isotonic": fit_isotonic_binary,
+        "platt": fit_platt_binary,
+        "beta": fit_beta_binary,
+    }
+    p_test = np.linspace(0.05, 0.95, 30)
+    recorded_params: dict[str, object] = {}
+    calibrated: dict[str, np.ndarray] = {}
+    for name, fit_fn in fitters.items():
+        params, apply = fit_fn(y, p)
+        calibrated[name] = apply(p_test)
+        if params is not None:
+            recorded_params[name] = params
+    # Three of four have inspectable params; isotonic is None.
+    assert set(recorded_params.keys()) == {"temperature", "platt", "beta"}
+    # All four produced calibrated outputs of matching shape.
+    assert all(out.shape == p_test.shape for out in calibrated.values())
+
+
 @pytest.mark.unit
 def test_platt_binary_params_are_pair(
     synthetic_binary_data: tuple[np.ndarray, np.ndarray],