eval-toolkit 0.38.0__tar.gz → 0.40.0__tar.gz

{eval_toolkit-0.38.0 → eval_toolkit-0.40.0}/CHANGELOG.md

@@ -7,6 +7,104 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.40.0] — 2026-05-18 — fit_platt_binary + fit_beta_binary (closes #43)
+
+Completes the binary scalar-prob calibrator family started in v0.35.0
+with `fit_temperature_binary` (#28). Consumers running the canonical
+4-calibrator audit battery (temperature + isotonic + Platt + Beta) now
+have library-first primitives for all four, eliminating ~100 LOC of
+hand-roll across the calibration battery pattern.
+
+### Added
+
+- **`eval_toolkit.fit_platt_binary(y_true, y_score)`** — binary-
+  probability adapter for `fit_platt_calibrator`. Returns
+  `((a, b), apply)` where `(a, b)` are the fitted Platt slope and
+  intercept and `apply: (n,) -> (n,)` is the scalar-in / scalar-out
+  callable that maps probabilities through `σ(a·s + b)`. Trivially
+  wraps `fit_platt_calibrator`; exposes `(a, b)` explicitly so
+  consumers can log the parameters in a `RunManifest` without
+  introspecting the `PlattFit` dataclass.
+- **`eval_toolkit.fit_beta_binary(y_true, y_score)`** — 3-parameter
+  Beta calibration per Kull et al. 2017. Returns `((a, b, c), apply)`
+  where `(a, b, c)` are the coefficients on `log(s)`, `log(1-s)`, and
+  the intercept respectively. Implemented directly (rather than
+  wrapping `fit_beta_calibrator` which returns only the callable) so
+  the fitted coefficients are captured alongside the apply function.
+- Both functions: 5 doctest examples each (run in CI via
+  `.doctest-modules`) + 15 unit tests covering shape contracts, param
+  exposure, parity with the underlying calibrators, single-class
+  rejection, and a family-consistency test verifying all 3 binary
+  adapters (temperature, Platt, Beta) satisfy the same
+  `(params_tuple, apply)` shape.
+
+### Why mirror `fit_temperature_binary`'s tuple shape
+
+#43's body proposed `Callable[[NDArray], NDArray]` as the return type
+(just the apply). We landed on `tuple[params, apply]` instead —
+matches the existing v0.35.0 contract and gives consumers the params
+they need for audit-battery `RunManifest` entries. Consumers wanting
+just the apply can use `_, apply = fit_platt_binary(...)`.
+
+### Protocol stability
+
+No Protocol shape changes. Per v0.39.0's roadmap Gate 2 update, v1.0
+requires ≥2 minors without Protocol edits — v0.40.0 is minor 1 of 2
+toward that target.
+
+## [0.39.0] — 2026-05-18 — consumer-feedback batch (closes #39, #40, #41)
+
+Three issues lifted from `prompt-injection-detection-prototype v1.0.0`
+consumer-side workarounds into upstream toolkit primitives + docs.
+All small scope; the consumer had filed each with a "Rationale for
+upstream landing" section per the *"Working around a library
+limitation without filing an upstream issue"* anti-pattern discipline.
+
+### Added
+
+- **`eval_toolkit.metrics.is_metric_defined_for_slice`** + module-level
+  constant **`SINGLE_CLASS_INCOMPATIBLE_METRICS`** (`frozenset({"auroc",
+  "auprc"})`). AUROC and AUPRC are mathematically undefined on
+  single-class slices — ranking metrics require both classes. The
+  primitive takes ``is_single_class: bool`` (caller computes from the
+  slice's class distribution) and returns whether the metric is
+  defined. Use at the per-cell layer to filter `(metric, slice)`
+  combinations BEFORE bootstrap so degenerate values never pollute
+  downstream artifacts. Default incompatible set overridable per call.
+  Closes #39.
+
+### Changed
+
+- **`LeakageCheck.name` Protocol member redeclared as `@property`**
+  (was `name: str` class-level annotation). Strictly contract-
+  tightening: existing consumers who *read* `check.name` keep working;
+  this fixes the `mypy --strict` rejection of
+  `list[LeakageCheck] = [CrossSplitLeakageCheck()]` that surfaced when
+  consumers built lists of frozen-dataclass leakage checks. Runtime
+  `isinstance(check, LeakageCheck)` was always fine — only mypy strict
+  was unhappy with read-only-vs-settable Protocol semantics (PEP 544).
+  Closes #40.
+
+### Documentation
+
+- **`docs/source/methodology/parallelism.md` §"Memory model:
+  worker-copy semantics"** — new section documenting joblib loky's
+  pickle-based worker IPC and its memory implications for DataFrame-
+  bearing specs (`n_jobs × spec_size` resident memory; the bug that
+  OOM-killed a 128-core BCa bootstrap sweep on prompt-injection-
+  detection-prototype's marginal-CI regen). Worked example of the
+  file-path-shared-state pattern + recommended ceiling formula
+  (`min(N, available_RAM_GB / spec_size_GB)`). Cross-referenced from
+  the `parallel_map` docstring in `_parallel.py`. Closes #41.
+
+### Why bundled
+
+Three small consumer-feedback items from the same downstream project;
+ship as one minor rather than three patches. Pattern matches the
+v0.37.0 "TokenizationLeakageCheck + per-module floors" bundling —
+small consumer-driven fixes consolidated to keep release-prep
+overhead per item low.
+
 ## [0.38.0] — 2026-05-18 — executable examples (myst-nb migration)
 
 Docs-only minor. Migrates the 14 walkthrough pages in

{eval_toolkit-0.38.0 → eval_toolkit-0.40.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: eval-toolkit
-Version: 0.38.0
+Version: 0.40.0
 Summary: Reusable evaluation contracts for binary classification: metrics, bootstrap CIs, calibration, artifacts, and evidence gates.
 Project-URL: Homepage, https://github.com/brandon-behring/eval-toolkit
 Project-URL: Documentation, https://brandon-behring.github.io/eval-toolkit/

{eval_toolkit-0.38.0 → eval_toolkit-0.40.0}/src/eval_toolkit/__init__.py

@@ -83,8 +83,10 @@ _EXPORTS: dict[str, str] = {
     "DEFAULT_STRATEGY": "eval_toolkit.calibration",
     "CostMatrix": "eval_toolkit.calibration",
     "bayes_optimal_threshold": "eval_toolkit.calibration",
+    "fit_beta_binary": "eval_toolkit.calibration",
     "fit_beta_calibrator": "eval_toolkit.calibration",
     "fit_isotonic_calibrator": "eval_toolkit.calibration",
+    "fit_platt_binary": "eval_toolkit.calibration",
     "fit_platt_calibrator": "eval_toolkit.calibration",
     "fit_temperature": "eval_toolkit.calibration",
     "fit_temperature_binary": "eval_toolkit.calibration",
@@ -164,6 +166,7 @@ _EXPORTS: dict[str, str] = {
     "write_manifest": "eval_toolkit.manifest",
     # --- metrics ---
     "DEFAULT_ASSUMED_PRIORS": "eval_toolkit.metrics",
+    "SINGLE_CLASS_INCOMPATIBLE_METRICS": "eval_toolkit.metrics",
     "ThresholdResult": "eval_toolkit.metrics",
     "brier_decomposition": "eval_toolkit.metrics",
     "brier_score": "eval_toolkit.metrics",
@@ -173,6 +176,7 @@ _EXPORTS: dict[str, str] = {
     "expected_calibration_error_l2": "eval_toolkit.metrics",
     "expected_calibration_error_l2_debiased": "eval_toolkit.metrics",
     "headline_metrics": "eval_toolkit.metrics",
+    "is_metric_defined_for_slice": "eval_toolkit.metrics",
     "metrics_at_threshold": "eval_toolkit.metrics",
     "pr_auc": "eval_toolkit.metrics",
     "precision_at_prior": "eval_toolkit.metrics",

{eval_toolkit-0.38.0 → eval_toolkit-0.40.0}/src/eval_toolkit/_parallel.py

@@ -48,6 +48,19 @@ def parallel_map[T, R](
       state (use ``np.random.SeedSequence(seed).spawn(n)`` for resample
       loops so ``n_jobs > 1`` produces identical results to ``n_jobs == 1``
       for the same seed).
+    - **Worker-copy memory model.** Each loky worker receives a full copy
+      of every argument bound at ``delayed(fn)(item)`` call time —
+      loky's worker IPC is pickle-based, so caller objects duplicate
+      rather than alias. For DataFrame-bearing specs this means
+      ``n_jobs × spec_size`` resident memory; a 128-core sweep over a
+      30 GB-per-cell BCa bootstrap will OOM-kill before any cell
+      finishes. Pass DataFrames via file-path indirection (parquet
+      path + reload inside ``fn``) for memory-bounded parallelism, or
+      cap ``n_jobs`` at ``min(N, available_RAM_GB / spec_size_GB)``.
+      NumPy arrays >~1 MB transparently use joblib's memmap fast path
+      and do NOT duplicate. See
+      ``methodology/parallelism.md#memory-model-worker-copy-semantics``
+      for the shared-state pattern with a worked example.
 
     Parameters
     ----------

{eval_toolkit-0.38.0 → eval_toolkit-0.40.0}/src/eval_toolkit/_version.py

@@ -2,4 +2,4 @@
 
 __all__ = ["__version__"]
 
-__version__ = "0.38.0"
+__version__ = "0.40.0"

{eval_toolkit-0.38.0 → eval_toolkit-0.40.0}/src/eval_toolkit/calibration.py

@@ -53,8 +53,10 @@ __all__ = [
     "CostMatrix",
     "PlattFit",
     "bayes_optimal_threshold",
+    "fit_beta_binary",
     "fit_beta_calibrator",
     "fit_isotonic_calibrator",
+    "fit_platt_binary",
     "fit_platt_calibrator",
     "fit_temperature",
     "fit_temperature_binary",
@@ -1135,6 +1137,163 @@ def fit_temperature_binary(
     return t_optimal, apply
 
 
+def fit_platt_binary(
+    y_true: np.ndarray, y_score: np.ndarray
+) -> tuple[tuple[float, float], Callable[[np.ndarray], np.ndarray]]:
+    r"""Binary-probability adapter for :func:`fit_platt_calibrator`.
+
+    Mirror of :func:`fit_temperature_binary`: returns ``((a, b), apply)``
+    where ``(a, b)`` are the fitted Platt slope and intercept and
+    ``apply: (n,) -> (n,)`` is a scalar-in / scalar-out callable that
+    maps any input probability array through :math:`\sigma(a \cdot s + b)`.
+
+    Trivially wraps :func:`fit_platt_calibrator` — added in v0.40.0 to
+    provide naming consistency with :func:`fit_temperature_binary` and
+    :func:`fit_beta_binary` for the 4-calibrator audit-battery pattern
+    (temperature, isotonic, Platt, Beta). The ``(a, b)`` tuple is exposed
+    explicitly so consumers can log it in a :class:`RunManifest` without
+    introspecting the :class:`PlattFit` dataclass.
+
+    Parameters
+    ----------
+    y_true : np.ndarray, shape (n,)
+        Binary validation labels in {0, 1}.
+    y_score : np.ndarray, shape (n,)
+        Validation predicted probabilities of class 1, in [0, 1]. Values
+        at the extremes are clipped to ``[1e-7, 1 - 1e-7]`` by the
+        underlying Platt fitter.
+
+    Returns
+    -------
+    tuple
+        ``((a, b), apply)`` — ``a`` is the Platt slope, ``b`` is the
+        intercept, ``apply`` maps probabilities through the fitted
+        sigmoid.
+
+    Raises
+    ------
+    ValueError
+        On shape mismatch, empty input, non-finite scores, or single-class
+        ``y_true`` (propagated from :func:`fit_platt_calibrator`).
+    RuntimeError
+        If the L-BFGS-B optimizer fails to converge.
+
+    Examples
+    --------
+    >>> import numpy as np
+    >>> rng = np.random.default_rng(0)
+    >>> n = 500
+    >>> y_val = rng.binomial(1, 0.3, size=n).astype(int)
+    >>> p_val = np.clip(y_val * 0.6 + rng.normal(0, 0.2, n), 0.01, 0.99)
+    >>> (a, b), apply = fit_platt_binary(y_val, p_val)
+    >>> isinstance(a, float) and isinstance(b, float)
+    True
+    >>> apply(np.array([0.1, 0.5, 0.9])).shape == (3,)
+    True
+
+    See Also
+    --------
+    fit_platt_calibrator : underlying canonical Platt fitter (returns ``PlattFit``).
+    fit_temperature_binary : 1-parameter sibling.
+    fit_beta_binary : 3-parameter sibling.
+    """
+    fit = fit_platt_calibrator(y_true, y_score)
+    return (fit.a, fit.b), fit.transform
+
+
+def fit_beta_binary(
+    y_true: np.ndarray, y_score: np.ndarray
+) -> tuple[tuple[float, float, float], Callable[[np.ndarray], np.ndarray]]:
+    r"""Binary-probability adapter for :func:`fit_beta_calibrator` per Kull et al. 2017.
+
+    Mirror of :func:`fit_temperature_binary`: returns
+    ``((a, b, c), apply)`` where ``(a, b, c)`` are the 3-parameter Beta
+    calibration coefficients (``a`` on :math:`\log s`, ``b`` on
+    :math:`\log(1-s)`, ``c`` intercept) and ``apply`` is the scalar-in /
+    scalar-out callable.
+
+    Added in v0.40.0 for naming consistency with
+    :func:`fit_temperature_binary` and :func:`fit_platt_binary` in the
+    4-calibrator audit-battery pattern. Implemented directly (rather
+    than wrapping :func:`fit_beta_calibrator`) so the fitted ``(a, b, c)``
+    parameters can be exposed alongside the apply callable — the
+    existing :func:`fit_beta_calibrator` returns only the callable.
+
+    Beta calibration is a strict generalization of Platt scaling
+    (Kull 2017 §3); it empirically dominates Platt on most real-world
+    classifiers at the cost of one extra parameter. Unlike Platt and
+    isotonic, the Beta sigmoid is *not* monotone in the raw score.
+
+    Parameters
+    ----------
+    y_true : np.ndarray, shape (n,)
+        Binary validation labels in {0, 1}.
+    y_score : np.ndarray, shape (n,)
+        Validation predicted probabilities of class 1, in (0, 1). Values
+        at the extremes are clipped to ``[1e-7, 1 - 1e-7]`` so the
+        log-link is finite.
+
+    Returns
+    -------
+    tuple
+        ``((a, b, c), apply)`` — ``(a, b, c)`` are the Beta calibration
+        coefficients; ``apply`` maps probabilities through
+        :math:`\sigma(a \log s + b \log(1-s) + c)`.
+
+    Raises
+    ------
+    ValueError
+        On shape mismatch, empty input, non-finite scores, or single-class
+        ``y_true``.
+
+    Examples
+    --------
+    >>> import numpy as np
+    >>> rng = np.random.default_rng(0)
+    >>> n = 500
+    >>> y_val = rng.binomial(1, 0.3, size=n).astype(int)
+    >>> p_val = np.clip(y_val * 0.6 + rng.normal(0, 0.2, n), 0.01, 0.99)
+    >>> (a, b, c), apply = fit_beta_binary(y_val, p_val)
+    >>> all(isinstance(x, float) for x in (a, b, c))
+    True
+    >>> apply(np.array([0.1, 0.5, 0.9])).shape == (3,)
+    True
+
+    See Also
+    --------
+    fit_beta_calibrator : underlying Beta fitter (returns only the callable).
+    fit_temperature_binary : 1-parameter sibling.
+    fit_platt_binary : 2-parameter sibling.
+
+    References
+    ----------
+    Kull, M., Filho, T. S., & Flach, P. "Beta calibration: a well-founded
+    and easily implemented improvement on logistic calibration for binary
+    classifiers." AISTATS 2017. arXiv:1607.06770.
+    """
+    from sklearn.linear_model import LogisticRegression  # noqa: PLC0415
+
+    y_true_arr, y_score_arr = _validate_calibrator_inputs(y_true, y_score)
+    s_clipped = np.clip(y_score_arr, _SCORE_CLIP_LO, _SCORE_CLIP_HI)
+    features = np.column_stack([np.log(s_clipped), np.log(1.0 - s_clipped)])
+    lr = LogisticRegression(C=1e9, solver="lbfgs", max_iter=2000)
+    lr.fit(features, y_true_arr)
+    a = float(lr.coef_[0][0])
+    b = float(lr.coef_[0][1])
+    c = float(lr.intercept_[0])
+
+    def apply(scores: np.ndarray) -> np.ndarray:
+        arr = np.asarray(scores, dtype=float).ravel()
+        if not np.isfinite(arr).all():
+            raise ValueError("scores contains NaN or inf")
+        clipped = np.clip(arr, _SCORE_CLIP_LO, _SCORE_CLIP_HI)
+        z = a * np.log(clipped) + b * np.log(1.0 - clipped) + c
+        out: np.ndarray = (1.0 / (1.0 + np.exp(-z))).astype(float)
+        return out
+
+    return (a, b, c), apply
+
+
 def fit_temperature_oracle(
     y_true: np.ndarray, y_score: np.ndarray
 ) -> tuple[float, Callable[[np.ndarray], np.ndarray]]:

{eval_toolkit-0.38.0 → eval_toolkit-0.40.0}/src/eval_toolkit/leakage.py

@@ -218,9 +218,16 @@ class LeakageCheck(Protocol):
     name : str
         Stable identifier for the check, written into the
         :class:`LeakageFinding` and used for downstream filtering.
+        Read-only (a ``@property`` on the Protocol) so the natural
+        ``@dataclass(frozen=True)`` implementation pattern is
+        compatible with ``mypy --strict``: a frozen attribute satisfies
+        a read-only Protocol member but not a settable one (PEP 544).
     """
 
-    name: str
+    @property
+    def name(self) -> str:  # pragma: no cover
+        """Stable check identifier (read-only)."""
+        ...
 
     def validate(self, splits: Mapping[str, EvalSlice]) -> LeakageFinding:  # pragma: no cover
         """Validate the splits and return one :class:`LeakageFinding`."""

{eval_toolkit-0.38.0 → eval_toolkit-0.40.0}/src/eval_toolkit/metrics.py

@@ -8,7 +8,7 @@ JSON-serializable values.
 
 from __future__ import annotations
 
-from collections.abc import Mapping
+from collections.abc import Collection, Mapping
 from dataclasses import dataclass
 from typing import TYPE_CHECKING, Final, Literal, overload
 
@@ -95,6 +95,7 @@ def _empty_strategy_guard(
 
 __all__ = [
     "DEFAULT_ASSUMED_PRIORS",
+    "SINGLE_CLASS_INCOMPATIBLE_METRICS",
     "ThresholdResult",
     "brier_decomposition",
     "brier_score",
@@ -104,6 +105,7 @@ __all__ = [
     "expected_calibration_error_l2",
     "expected_calibration_error_l2_debiased",
     "headline_metrics",
+    "is_metric_defined_for_slice",
     "metrics_at_threshold",
     "pr_auc",
     "precision_at_prior",
@@ -115,6 +117,88 @@ __all__ = [
     "stratified_recall",
 ]
 
+SINGLE_CLASS_INCOMPATIBLE_METRICS: Final[frozenset[str]] = frozenset({"auroc", "auprc"})
+"""Metrics that are mathematically undefined on single-class slices.
+
+AUROC and AUPRC are *ranking* metrics — both require at least one
+positive AND one negative example to produce a well-defined score.
+sklearn's implementations return degenerate ``1.0`` or ``0.0`` (or
+warn) when called on single-class input; those values entering
+bootstrap/CI artifacts produce misleading downstream evidence.
+
+Overridable per call to :func:`is_metric_defined_for_slice` via the
+``incompatible_metrics`` kwarg. Surfaced by v0.39.0 / closes #39.
+"""
+
+
+def is_metric_defined_for_slice(
+    metric_name: str,
+    *,
+    is_single_class: bool,
+    incompatible_metrics: Collection[str] = SINGLE_CLASS_INCOMPATIBLE_METRICS,
+) -> bool:
+    """Return ``True`` iff a metric is well-defined for the given class distribution.
+
+    Use at the per-cell layer to filter ``(metric, slice)`` combinations
+    BEFORE bootstrap so degenerate values never pollute the downstream
+    artifacts (CI bands, MDE estimates, marginal-effect summaries).
+
+    Parameters
+    ----------
+    metric_name : str
+        Metric identifier. Case-insensitive. Common values:
+        ``"auroc"``, ``"auprc"``, ``"ece"``, ``"brier"``, ``"recall"``,
+        ``"precision"``, ``"recall_at_fpr"``.
+    is_single_class : bool
+        Whether the slice has only one observed class (all-positive
+        or all-negative ``y_true``). Caller computes this from the
+        slice's label vector — the function does not inspect data.
+    incompatible_metrics : Collection[str], optional
+        Metric names that are undefined on single-class slices. Default
+        :data:`SINGLE_CLASS_INCOMPATIBLE_METRICS` (``{"auroc", "auprc"}``).
+        Override to widen (e.g., add ``"recall_at_fpr"`` if your threshold
+        selector also requires both classes) or narrow per call. Compared
+        case-insensitively.
+
+    Returns
+    -------
+    bool
+        ``True`` if the metric is well-defined for the slice's class
+        distribution; ``False`` if it would be degenerate.
+
+    Examples
+    --------
+    >>> is_metric_defined_for_slice("auroc", is_single_class=True)
+    False
+    >>> is_metric_defined_for_slice("auroc", is_single_class=False)
+    True
+    >>> is_metric_defined_for_slice("ece", is_single_class=True)
+    True
+    >>> is_metric_defined_for_slice("AUROC", is_single_class=True)
+    False
+    >>> is_metric_defined_for_slice(
+    ...     "recall_at_fpr",
+    ...     is_single_class=True,
+    ...     incompatible_metrics=frozenset({"auroc", "auprc", "recall_at_fpr"}),
+    ... )
+    False
+
+    Notes
+    -----
+    The function takes ``is_single_class`` rather than inspecting
+    ``y_true`` directly to keep the primitive cheap (no array
+    materialisation) and composable with caller-side slice taxonomies
+    where the single-class property is known a priori from slice
+    semantics (e.g., a benign-only or attack-only split).
+
+    See Also
+    --------
+    SINGLE_CLASS_INCOMPATIBLE_METRICS : the default incompatibility set.
+    """
+    if not is_single_class:
+        return True
+    return metric_name.lower() not in {m.lower() for m in incompatible_metrics}
+
 
 @dataclass(frozen=True, slots=True)
 class ThresholdResult:

{eval_toolkit-0.38.0 → eval_toolkit-0.40.0}/tests/golden/public_api/snapshot.json

@@ -75,6 +75,7 @@
     "RecallAtFprResult",
     "RunManifest",
     "RunResult",
+    "SINGLE_CLASS_INCOMPATIBLE_METRICS",
     "Scorer",
     "SimilarityAuditFinding",
     "SimilarityAuditReport",
@@ -133,9 +134,11 @@
     "fdr_bh_correct",
     "figure_metadata",
     "file_sha256",
+    "fit_beta_binary",
     "fit_beta_calibrator",
     "fit_isotonic_calibrator",
     "fit_operating_points",
+    "fit_platt_binary",
     "fit_platt_calibrator",
     "fit_temperature",
     "fit_temperature_binary",
@@ -144,6 +147,7 @@
     "frozen_config",
     "headline_metrics",
     "headline_present_gate",
+    "is_metric_defined_for_slice",
     "iter_folds_with_pool",
     "load_prediction_arrays",
     "low_fpr_feasibility_gate",
@@ -827,6 +831,12 @@
       "kind": "class",
       "signature": "(run_id: 'str', git_sha: 'str | None', config: 'dict[str, object]', by_slice: 'dict[str, dict[str, object]]' = <factory>, by_fold: 'dict[str, RunResult]' = <factory>, fold_summary: 'dict[str, dict[str, object]]' = <factory>, claim_report: 'dict[str, object]' = <factory>, prediction_artifacts: 'list[dict[str, object]]' = <factory>, evidence_axes: 'list[dict[str, object]]' = <factory>, pairing_metadata: 'dict[str, object]' = <factory>, aggregate_evidence: 'dict[str, object]' = <factory>, threshold_policy: 'dict[str, object]' = <factory>, schema_version: 'str' = 'v1') -> None"
     },
+    "SINGLE_CLASS_INCOMPATIBLE_METRICS": {
+      "doc_first_line": "Build an immutable unordered collection of unique elements.",
+      "kind": "value",
+      "type": "frozenset",
+      "value": "['auprc', 'auroc']"
+    },
     "Scorer": {
       "bases": [
         "Protocol"
@@ -1026,7 +1036,7 @@
       "doc_first_line": "str(object='') -> str",
       "kind": "value",
       "type": "str",
-      "value": "'0.38.0'"
+      "value": "'0.40.0'"
     },
     "apply_operating_points": {
       "doc_first_line": "Apply fitted thresholds to a mixed-class or single-class target slice.",
@@ -1188,6 +1198,11 @@
       "kind": "function",
       "signature": "(path: 'Path | str', *, strict: 'bool' = False) -> 'str | None'"
     },
+    "fit_beta_binary": {
+      "doc_first_line": "Binary-probability adapter for :func:`fit_beta_calibrator` per Kull et al. 2017.",
+      "kind": "function",
+      "signature": "(y_true: 'np.ndarray', y_score: 'np.ndarray') -> 'tuple[tuple[float, float, float], Callable[[np.ndarray], np.ndarray]]'"
+    },
     "fit_beta_calibrator": {
       "doc_first_line": "Beta calibration per Kull et al. 2017 [#kull]_.",
       "kind": "function",
@@ -1203,6 +1218,11 @@
       "kind": "function",
       "signature": "(y_true: 'np.ndarray', y_score: 'np.ndarray', selectors: 'Sequence[ThresholdSelector]', *, fitted_on_slice: 'str' = '', scorer_name: 'str' = '') -> 'dict[str, FittedOperatingPoint]'"
     },
+    "fit_platt_binary": {
+      "doc_first_line": "Binary-probability adapter for :func:`fit_platt_calibrator`.",
+      "kind": "function",
+      "signature": "(y_true: 'np.ndarray', y_score: 'np.ndarray') -> 'tuple[tuple[float, float], Callable[[np.ndarray], np.ndarray]]'"
+    },
     "fit_platt_calibrator": {
       "doc_first_line": "Platt 1999 [#platt]_ sigmoid scaling with Lin 2007 [#lin]_ Laplace-smoothed targets.",
       "kind": "function",
@@ -1243,6 +1263,11 @@
       "kind": "function",
       "signature": "(path: 'str' = 'headline', *, severity: 'GateSeverity' = 'error') -> 'EvidenceGate'"
     },
+    "is_metric_defined_for_slice": {
+      "doc_first_line": "Return ``True`` iff a metric is well-defined for the given class distribution.",
+      "kind": "function",
+      "signature": "(metric_name: 'str', *, is_single_class: 'bool', incompatible_metrics: 'Collection[str]' = frozenset({'auprc', 'auroc'})) -> 'bool'"
+    },
     "iter_folds_with_pool": {
       "doc_first_line": "Compose a :class:`Splitter` with a :class:`PoolBuilder`.",
       "kind": "function",