eval-toolkit 0.46.1__tar.gz → 0.48.0__tar.gz

{eval_toolkit-0.46.1 → eval_toolkit-0.48.0}/CHANGELOG.md

@@ -5,6 +5,230 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.48.0] — 2026-05-22 — Polish + audit-driven tightening before v1.0 (Round 7 follow-on + cross-API consistency + doc-execution gates)
+
+Third + final BREAKING minor of the staggered v0.45 → v0.46 → v0.46.1 → v0.47
+→ v0.48 → v1.0 release sequence (plan
+``~/.claude/plans/evaluate-all-the-work-twinkly-kite.md``, Step 4). Migration
+guide: ``docs/source/migration/v0.48.md``.
+
+Closes:
+
+- Round 7 audit STOP-GATE per Decision Y.2 (Codex R7-F1/F2/F3 + 6 Gemini
+  observations; see ``docs/source/audit_findings.md`` for the per-finding
+  ledger).
+- Audit-as-seed extensions surfaced during plan refinement: full
+  module-docstring sweep across ``src/eval_toolkit/``; expanded
+  ``.doctest-modules`` from 11 → 21 modules; comprehensive cross-API
+  shape-validation consistency sweep.
+- Round 5 §5E-prep packet-drift fixes (7 methodology documentation
+  corrections).
+
+After v0.48 observes ≥1 consumer cycle, the Round 8 audit STOP-GATE
+opens before ``v1.0.0`` tag.
+
+### BREAKING
+
+- **``BootstrapCI.to_dict()`` + ``PairedBootstrapCI.to_dict()`` schema
+  rewrite** (§5B). Pre-v0.48 hard-coded a ``"ci_95"`` key regardless of
+  the actual ``confidence`` field — the key contradicted the data.
+  v0.48 schema is self-describing:
+
+    Before: ``{"point_estimate": p, "ci_95": [l, h], "confidence": 0.95, ...}``
+    After:  ``{"point": p, "low": l, "high": h, "confidence": 0.95, ...}``
+
+  Migration: ``d["point_estimate"]`` → ``d["point"]``; ``d["ci_95"]``
+  → ``(d["low"], d["high"])``. Same rewrite for ``PairedBootstrapCI``.
+- **``sweep()`` schema grows by 1 column** (§5I, Decision R7-B option C).
+  New ``strategy_id`` column inserted between ``text_id`` and ``variant``
+  carries the canonical per-row identifier built from configured
+  kwargs. Callers indexing by column position must re-check offsets.
+- **``sweep()`` rejects duplicate ``strategy_id``** (§5I). Mirrors
+  R6-B's duplicate ``MetricSpec.name`` rejection in ``scorecard()``.
+- **``sweep()`` validates scorer output shape** (§5J, Decision R7-C).
+  Wrong-shape arrays from ``Scorer.predict_proba`` raise contextual
+  ``ValueError`` at the boundary. Pre-v0.48: silent truncation
+  (overlong), ``IndexError`` (short), or ``TypeError`` (matrix-shaped).
+- **``paired_bootstrap_op_point_diff()`` rejects ``val_y is test_y``**
+  (§5E-prep). The two-level bootstrap assumes disjoint val + test
+  partitions; passing the same array causes ~63.2% silent overlap.
+
+### Added
+
+- **``make pre-push``** Makefile target (§5L) running all 3 doc-
+  execution surfaces — Sybil-collected ``.md`` fences, MyST-NB example
+  notebooks, and in-source ``>>>`` docstring examples. Closes the
+  v0.47 Sub-PR 7 incident class.
+- **``nb_execution_raise_on_error = True``** in ``docs/source/conf.py``
+  (§5H, Decision R7-A). Docs CI now fails on notebook execution errors.
+- **``.doctest-modules`` expanded** from 11 → 21 modules (§5M).
+
+### Changed
+
+- **Cross-API shape-validation consistency** (§5N). Every public-API
+  surface with array inputs now validates shape + raises ``ValueError``
+  with context (rather than leaking low-level numpy/sklearn errors).
+- **Standardized ``ImportError`` messages** across lazy-extras (§5C).
+  Canonical template: ``"<feature> requires <pkg>. Install with: pip
+  install eval-toolkit[<extra>]"``.
+- **Pin-exact-key-set regression-guards** (§5A) for every dict-returning
+  metrics function. Audit revealed no drift; the tests pin existing
+  key sets so future drift fails CI loud.
+- **Docs polish** (§5K + §5E-prep): ``SynonymSubstitution`` whitelist
+  ``Notes``; ``Scorecard.to_pandas()`` dtype coercion ``Notes``;
+  ``CostSensitiveSelector`` calibrated-prior ``Warning``; ``cv_clt_ci``
+  docstring per Bayle et al. (2020) Theorem 3.1; ``methodology/parallelism.md``
+  post-v0.36 state; ``methodology/testing.md`` reference-equivalence-gap
+  framing; ``methodology/calibration.md`` 4-binary-adapter family;
+  ``methodology/bootstrap.md`` disjoint-split example; DeLong docs
+  aligned to shipped state (Decision U).
+
+### Fixed
+
+- **R7-F1**: 6 MyST-NB example notebooks (``docs/source/examples/*.md``)
+  migrated to v0.47 API; 4 module-level docstrings rewritten; 5
+  drifted ``docs/source/api/*.md`` autosummary lists corrected;
+  8 missing ``api/*.md`` pages created; roadmap "Sybil-validated
+  examples" wording corrected (§5G).
+- **ADR 0001** (flat-module layout) + **ADR 0003** (stability contract
+  + Gate 3 methodology) finalized for v1.0 (§5E + §5F).
+- **schemas.md** + **methodology/claims.md** + **getting-started.md**:
+  ``BootstrapCI`` schema references updated for the §5B rewrite.
+
+## [0.47.0] — 2026-05-21 — Sweep unification + TextTransform + advanced-6 + cleanup + Round 6 follow-on
+
+Second BREAKING minor of the staggered v0.45 → v0.46 → v0.46.1 → v0.47 →
+v0.48 → v1.0 release sequence (plan
+``~/.claude/plans/evaluate-all-the-work-twinkly-kite.md``, Step 3).
+
+Closes:
+
+- The v0.43 CHANGELOG forward-look re: advanced-6 character-injection
+  techniques (Decision Q11→11.3 — "12-technique suite + new sweep API
+  in one migration step").
+- Round 6 audit follow-on items per Decision R6-E (R6-A docstring,
+  R6-B duplicate name guard, R6-C to_pandas schema, R6-D Protocol
+  method-shape drift guard, R6-F5 narrow except, R6-F6 plan/roadmap
+  refresh, R6-H make_spec_name helper) — see ``docs/source/audit_findings.md``
+  for the per-finding ledger.
+
+### Removed (BREAKING)
+
+- **Top-level scalar metric names** (``eval_toolkit.pr_auc``,
+  ``eval_toolkit.roc_auc``, ``eval_toolkit.brier_score``, all 5
+  ``expected_calibration_*`` variants) — the v0.46 ``__getattr__``
+  deprecation shim has been deleted. These names now raise
+  ``AttributeError`` at the top-level. Migration: use ``scorecard(...)``
+  with ``metric_specs`` (primary) OR import from the
+  ``eval_toolkit.metrics`` submodule (internal API per ADR 0002).
+  (Decision L; plan §4D.)
+- **Module-level ``adversarial.sweep`` + ``preprocessing.sweep``** —
+  consolidated into the top-level :func:`sweep` (Decision D + plan §4C).
+  Parity tests in Sub-PR 4 of this release proved 1:1 output equivalence
+  on the neutral subset.
+- **``adversarial.character_injection`` + ``preprocessing.spotlighting``
+  ``SimpleNamespace`` shortcuts** — removed (Decision N + plan §4E).
+  The 12 adversarial dataclasses + the 3 preprocessing variants + the
+  underlying functional API are the only public paths.
+- **``adversarial.CharacterInjectionStrategy``** per-module Protocol —
+  removed. The top-level :class:`TextTransform` Protocol (Decision K)
+  is the single canonical contract; all 12 character-injection
+  dataclasses + 3 preprocessing variants satisfy it structurally.
+
+### Added
+
+- **``TextTransform`` Protocol** (top-level; ``eval_toolkit.protocols`` module).
+  Decision K + Audit R5-F3 (Codex Round 5): unifies the "name + transform(text)"
+  shape across preprocessing (defence) and adversarial (attack) strategies so
+  the v0.47 top-level :func:`sweep` (next sub-PR) can mix them in one call. The
+  9th strict Tier-2 Protocol per ADR 0003.
+- **Advanced 6 character-injection techniques** (plan §4F, Decision Q11→11.3) —
+  closes the v0.43.0 CHANGELOG forward-look that referenced these as "scheduled
+  for v0.43.1" (a version that never shipped). Each satisfies the top-level
+  :class:`TextTransform` Protocol structurally; all are frozen + ``slots=True``
+  dataclasses with deterministic behaviour under their ``seed`` kwarg where
+  applicable:
+
+    - :class:`BidiRTLInjection` — wrap input in ``U+202E … U+202C``
+      RIGHT-TO-LEFT OVERRIDE block.
+    - :class:`TagStrippingInjection` — strip HTML/XML-like ``<…>`` tags
+      (idempotent).
+    - :class:`SynonymSubstitution` — replace whitelisted prompt-injection-
+      relevant function words / verbs with semantic-preserving synonyms.
+    - :class:`TokenSplitting` — insert a single space inside long enough
+      words; forces subword tokenizers to re-segment.
+    - :class:`UnicodeNormalization` — NFC / NFD / NFKC / NFKD; default NFKC
+      folds compatibility chars (e.g., fullwidth ``ＡＢＣ`` → ``ABC``).
+    - :class:`InvisibleCharsInjection` — sample from the 5-element invisible-
+      code-point set (ZWSP, ZWNJ, ZWJ, word joiner, BOM) — distinct from the
+      single-codepoint :class:`ZeroWidthSpaceInjection`.
+
+  Also exported: ``ADVANCED_TECHNIQUES`` (6-tuple) and ``ALL_TECHNIQUES``
+  (12-tuple = core 6 + advanced 6).
+- **Top-level :func:`sweep`** — single ``TextTransform`` enumeration entry
+  point (Decision K + Decision D + Audit R5-F3). Replaces the per-module
+  ``adversarial.sweep`` + ``preprocessing.sweep`` (those are removed in a
+  subsequent sub-PR of this release). New contract:
+
+    - ``sweep(strategies, texts)`` → neutral DataFrame with ``text_id`` /
+      ``variant`` / ``transformed_text`` columns. Pure text-transform
+      enumeration; defence + attack strategies compose freely.
+    - ``sweep(..., scorer=...)`` → also emits ``original_score`` /
+      ``transformed_score`` columns (single batched scorer call per
+      strategy, not per-row).
+    - ``sweep(..., scorer=..., attack_threshold=t)`` → also emits ``asr``
+      (per-row attack-success flag). Explicit threshold REQUIRED to
+      materialize ``asr``; no magic ``threshold=0.5`` default.
+      ``attack_threshold`` without ``scorer`` raises ``ValueError``.
+
+  Parity tests against the existing module-level sweeps ship in this
+  sub-PR (``tests/test_sweep.py``) and prove the v0.47 consolidation
+  produces identical transformed-text rows for the 6 core character-
+  injection techniques + the 3 spotlighting variants.
+- **3 preprocessing dataclasses** (``DelimitVariant``, ``DatamarkVariant``,
+  ``EncodeVariant``) in :mod:`eval_toolkit.preprocessing`. Frozen +
+  ``slots=True`` thin wrappers over the existing :func:`delimit` /
+  :func:`datamark` / :func:`encode` functions. Closes Audit R5-F3
+  (Codex Round 5) — prior to this commit, ``preprocessing.__all__`` exported
+  only functions, so the "concrete classes satisfy ``TextTransform``
+  structurally" claim only held on the adversarial side. Now both sides
+  share the dataclass-strategy shape.
+- ``metric_specs.make_spec_name(prefix, **kwargs)`` canonicalization helper
+  for custom parameterized :class:`MetricSpec` implementations. Alphabetized
+  kwargs joined by underscore — same convention the v0.46 ECE factory uses.
+  Lands in ``metric_specs.__all__`` only; **not** top-level ``__all__`` per
+  Decision R6-H. (Closes Round 6 Gemini R6-F4.)
+
+### Changed (Round 6 follow-on)
+
+- ``scorecard()`` now raises ``ValueError`` when two :class:`MetricSpec`
+  instances in the ``metrics`` list share a ``name``. Forces caller
+  disambiguation; the ``Mapping[str, MetricResult]`` contract never silently
+  drops a cell. Error message reports both indices. (Decision R6-B; closes
+  Round 6 Codex R6-F3.)
+- ``scorecard(seed=None)`` docstring rewritten to document the deterministic-
+  by-default contract (``None`` is treated as ``seed=0``). No behavior
+  change; v0.46 documented the wrong contract. (Decision R6-A; closes Round 6
+  Codex R6-F4 + Gemini R6-F1.)
+- ``_evaluate_spec()`` exception catches narrowed: ``MemoryError``,
+  ``RecursionError``, ``KeyboardInterrupt``, and ``SystemExit`` now propagate
+  out of ``scorecard()`` instead of being captured as a ``status="error"``
+  cell. Per-cell isolation remains for ordinary application errors.
+  (Decision R6-F5; closes Round 6 Gemini R6-F5.)
+- ``Scorecard.to_pandas()`` MultiIndex schema extended with two new inner-
+  field columns: ``n_resamples`` (int / NaN sentinel) and ``method``
+  (string / ``""`` sentinel). The DataFrame view is now lossless against
+  :meth:`BootstrapCI.to_dict` — trace provenance (resample count + CI
+  method) no longer drops at the DataFrame boundary. Callers indexing the
+  MultiIndex by name keep working; callers indexing by position must
+  re-check column offsets. (Decision R6-C; closes Round 6 Gemini R6-F3.)
+- ``tests/test_public_api.py`` drift guard now captures method signatures
+  for ``typing.Protocol`` classes in ``__all__`` (a ``protocol_methods``
+  sub-entry in the snapshot). Together with a Tier-2 coverage test, this
+  actually enforces the strict method-shape stability ADR 0003 promises
+  for the 9 Tier-2 Protocols. (Decision R6-D; closes Round 6 Codex R6-F5.)
+  Public-API golden regenerated alongside this change.
+
 ## [0.46.1] — 2026-05-21 — Round 6 hotfix: ECE strategy validation + deprecation warning content
 
 Hotfix release per **Decision Q** (data correctness regression + time-sensitive

{eval_toolkit-0.46.1 → eval_toolkit-0.48.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: eval-toolkit
-Version: 0.46.1
+Version: 0.48.0
 Summary: Reusable evaluation contracts for binary classification: metrics, bootstrap CIs, calibration, artifacts, and evidence gates.
 Project-URL: Homepage, https://github.com/brandon-behring/eval-toolkit
 Project-URL: Documentation, https://brandon-behring.github.io/eval-toolkit/
@@ -215,7 +215,7 @@ pip install "eval-toolkit[all]"                 # everything
 
 ```python
 import numpy as np
-from eval_toolkit import pr_auc, roc_auc, expected_calibration_error
+from eval_toolkit.metrics import pr_auc, roc_auc, expected_calibration_error
 
 rng = np.random.default_rng(42)
 y = rng.integers(0, 2, size=200)
@@ -230,7 +230,8 @@ print(f"ECE (10 bins): {expected_calibration_error(y, s, n_bins=10):.3f}")
 ### Bootstrap confidence intervals
 
 ```python
-from eval_toolkit import bootstrap_ci, paired_bootstrap_diff, pr_auc
+from eval_toolkit import bootstrap_ci, paired_bootstrap_diff
+from eval_toolkit.metrics import pr_auc
 
 ci = bootstrap_ci(y, s, pr_auc, n_resamples=1000, seed=42)
 print(f"PR-AUC: {ci.point_estimate:.3f}  95% CI: [{ci.ci_low:.3f}, {ci.ci_high:.3f}]")
@@ -244,8 +245,10 @@ print(f"Δ PR-AUC: {diff.delta:.3f}  overlaps zero: {diff.overlaps_zero}")
 ### Temperature scaling (Guo et al. 2017)
 
 ```python
+import numpy as np
 from eval_toolkit import fit_temperature
 
+rng = np.random.default_rng(42)
 logits = rng.normal(size=(500, 2))
 labels = (logits[:, 1] > logits[:, 0]).astype(int)
 result = fit_temperature(logits, labels)

{eval_toolkit-0.46.1 → eval_toolkit-0.48.0}/README.md

@@ -132,7 +132,7 @@ pip install "eval-toolkit[all]"                 # everything
 
 ```python
 import numpy as np
-from eval_toolkit import pr_auc, roc_auc, expected_calibration_error
+from eval_toolkit.metrics import pr_auc, roc_auc, expected_calibration_error
 
 rng = np.random.default_rng(42)
 y = rng.integers(0, 2, size=200)
@@ -147,7 +147,8 @@ print(f"ECE (10 bins): {expected_calibration_error(y, s, n_bins=10):.3f}")
 ### Bootstrap confidence intervals
 
 ```python
-from eval_toolkit import bootstrap_ci, paired_bootstrap_diff, pr_auc
+from eval_toolkit import bootstrap_ci, paired_bootstrap_diff
+from eval_toolkit.metrics import pr_auc
 
 ci = bootstrap_ci(y, s, pr_auc, n_resamples=1000, seed=42)
 print(f"PR-AUC: {ci.point_estimate:.3f}  95% CI: [{ci.ci_low:.3f}, {ci.ci_high:.3f}]")
@@ -161,8 +162,10 @@ print(f"Δ PR-AUC: {diff.delta:.3f}  overlaps zero: {diff.overlaps_zero}")
 ### Temperature scaling (Guo et al. 2017)
 
 ```python
+import numpy as np
 from eval_toolkit import fit_temperature
 
+rng = np.random.default_rng(42)
 logits = rng.normal(size=(500, 2))
 labels = (logits[:, 1] > logits[:, 0]).astype(int)
 result = fit_temperature(logits, labels)

{eval_toolkit-0.46.1 → eval_toolkit-0.48.0}/src/eval_toolkit/__init__.py

@@ -1,9 +1,12 @@
 """eval-toolkit — reusable evaluation contracts for binary classification.
 
-Public API remains available from ``eval_toolkit`` and from submodules:
+The v1.0 primary metric surface is :func:`~eval_toolkit.scorecard` plus the
+:mod:`~eval_toolkit.metric_specs` namespace (ADR 0002). Submodule paths
+remain available for scalar primitives and adapter authors:
 
-    from eval_toolkit import pr_auc, bootstrap_ci, BootstrapCI
-    from eval_toolkit.metrics import pr_auc
+    from eval_toolkit import scorecard, metric_specs as ms
+    from eval_toolkit import bootstrap_ci, BootstrapCI
+    from eval_toolkit.metrics import pr_auc  # internal API, ADR 0002
 
 The package root uses lazy exports so importing ``eval_toolkit`` does not
 eagerly import optional-heavy modules such as plotting, loaders, or harnesses.
@@ -31,22 +34,36 @@ _logging.getLogger("eval_toolkit").addHandler(_logging.NullHandler())
 # tests/golden/public_api/ reads dict keys + values, not comments.
 _EXPORTS: dict[str, str] = {
     # --- adversarial ---
+    "ADVANCED_TECHNIQUES": "eval_toolkit.adversarial",
+    "ALL_TECHNIQUES": "eval_toolkit.adversarial",
+    "BidiRTLInjection": "eval_toolkit.adversarial",
     "CORE_TECHNIQUES": "eval_toolkit.adversarial",
     "CaseRandomization": "eval_toolkit.adversarial",
-    "CharacterInjectionStrategy": "eval_toolkit.adversarial",
     "DiacriticInjection": "eval_toolkit.adversarial",
     "HomoglyphSubstitution": "eval_toolkit.adversarial",
+    "InvisibleCharsInjection": "eval_toolkit.adversarial",
     "PunctuationInjection": "eval_toolkit.adversarial",
+    "SynonymSubstitution": "eval_toolkit.adversarial",
+    "TagStrippingInjection": "eval_toolkit.adversarial",
+    "TokenSplitting": "eval_toolkit.adversarial",
+    "UnicodeNormalization": "eval_toolkit.adversarial",
     "WhitespaceInjection": "eval_toolkit.adversarial",
     "ZeroWidthSpaceInjection": "eval_toolkit.adversarial",
-    "character_injection": "eval_toolkit.adversarial",
+    # CharacterInjectionStrategy + character_injection SimpleNamespace
+    # removed at v0.47 (Decision N + plan §4E). TextTransform Protocol +
+    # the 12 concrete dataclasses are now the only public path.
     # --- losses ---
     "RecallAtLowFPR": "eval_toolkit.losses",
     # --- preprocessing ---
+    # `spotlighting` SimpleNamespace removed at v0.47 (Decision N + plan §4E).
+    # The 3 Variant dataclasses + the underlying functional API are the
+    # only public path.
+    "DatamarkVariant": "eval_toolkit.preprocessing",
+    "DelimitVariant": "eval_toolkit.preprocessing",
+    "EncodeVariant": "eval_toolkit.preprocessing",
     "datamark": "eval_toolkit.preprocessing",
     "delimit": "eval_toolkit.preprocessing",
     "encode": "eval_toolkit.preprocessing",
-    "spotlighting": "eval_toolkit.preprocessing",
     # --- probes ---
     "ActivationDeltaProbe": "eval_toolkit.probes",
     "ActivationExtractor": "eval_toolkit.probes",
@@ -193,12 +210,15 @@ _EXPORTS: dict[str, str] = {
     "SINGLE_CLASS_INCOMPATIBLE_METRICS": "eval_toolkit.metrics",
     "ThresholdResult": "eval_toolkit.metrics",
     "brier_decomposition": "eval_toolkit.metrics",
-    # `brier_score`, `pr_auc`, `roc_auc`, and the 5 ECE variants removed from
-    # `_EXPORTS` at v0.46 (Decision L). They remain reachable at the top
-    # level via the `__getattr__` deprecation branch (emits
-    # `DeprecationWarning`; branch removed at v0.47) and via the metrics
-    # submodule (`from eval_toolkit.metrics import pr_auc` — internal API
-    # per ADR 0002, not part of the v1.0 stability contract).
+    # `brier_score`, `pr_auc`, `roc_auc`, and the 5 ECE variants were removed
+    # from `_EXPORTS` at v0.46 (Decision L); the v0.46 `__getattr__`
+    # deprecation branch that kept them reachable with `DeprecationWarning`
+    # was removed at v0.47. They now raise `AttributeError` at the top level.
+    # The metrics submodule (`from eval_toolkit.metrics import pr_auc`)
+    # remains the only stable import path for scalar primitives — internal
+    # API per ADR 0002, not part of the v1.0 stability contract. The
+    # `scorecard()` + `metric_specs` surface is the primary path going
+    # forward (`metric_specs.pr_auc`, `metric_specs.roc_auc`, etc.).
     "headline_metrics": "eval_toolkit.metrics",
     "is_metric_defined_for_slice": "eval_toolkit.metrics",
     "metrics_at_threshold": "eval_toolkit.metrics",
@@ -247,6 +267,7 @@ _EXPORTS: dict[str, str] = {
     "PredictionReader": "eval_toolkit.protocols",
     "Scorer": "eval_toolkit.protocols",
     "SliceAwareScorer": "eval_toolkit.protocols",
+    "TextTransform": "eval_toolkit.protocols",
     "Versioned": "eval_toolkit.protocols",
     # --- seeds ---
     "set_global_seeds": "eval_toolkit.seeds",
@@ -298,61 +319,28 @@ _EXPORTS: dict[str, str] = {
     "MetricSpec": "eval_toolkit._scorecard",
     "Scorecard": "eval_toolkit._scorecard",
     "scorecard": "eval_toolkit._scorecard",
+    # --- sweep (top-level v0.47 unification — Decision K + Decision D) ---
+    "sweep": "eval_toolkit._sweep",
 }
 
 __all__ = ["__version__", *_EXPORTS.keys()]
 
 
-# ── BEGIN TRANSITIONAL DEPRECATION BRANCH (Decision L; REMOVE AT v0.47) ──
-# At v0.46 the scalar metric functions left the top-level `_EXPORTS` map (above)
-# in favor of the `scorecard()` surface (Decision A). To give the consumer one
-# release of overlap before the hard removal at v0.47, the names below remain
-# reachable via the package-level `__getattr__` (which delegates to the
-# `eval_toolkit.metrics` submodule) but emit a `DeprecationWarning` on first
-# lookup pointing at the new API.
-#
-# WHY THIS IS A BRANCH, NOT A REPLACEMENT (Audit F4 — Round 5):
-# `__getattr__` below is the load-bearing lazy export resolver for every name
-# in `_EXPORTS`. The deprecation branch is a discrete `if name in
-# _DEPRECATED_SCALARS` check ABOVE the resolver — the resolver's existing
-# behavior for non-deprecated names is unchanged. At v0.47 we delete this
-# transitional block and the resolver continues to work for every remaining
-# `_EXPORTS` entry.
-_DEPRECATED_SCALARS: frozenset[str] = frozenset(
-    {
-        "pr_auc",
-        "roc_auc",
-        "brier_score",
-        "expected_calibration_error",
-        "expected_calibration_error_debiased",
-        "expected_calibration_error_equal_mass",
-        "expected_calibration_error_l2",
-        "expected_calibration_error_l2_debiased",
-    }
-)
-# ── END TRANSITIONAL DEPRECATION (Decision L; REMOVE AT v0.47) ──
-
-
 def __getattr__(name: str) -> Any:
-    """Resolve public symbols lazily."""
+    """Resolve public symbols lazily.
+
+    v0.47 cleanup (Decision L): the BEGIN/END TRANSITIONAL DEPRECATION
+    BRANCH that v0.46 inserted in front of the resolver — together with the
+    ``_DEPRECATED_SCALARS`` frozenset and the ``_deprecation_warning_for``
+    helper — has been removed. The lazy resolver below is the v0.46 base
+    behavior; with the transitional block gone, deprecated v0.45 scalar names
+    (``pr_auc``, ``roc_auc``, ``brier_score``, the 5 ``expected_calibration_*``
+    variants) now raise :class:`AttributeError` cleanly. Submodule-level
+    access (e.g., ``from eval_toolkit.metrics import pr_auc``) is unaffected
+    per Decision C / ADR 0002.
+    """
     if name == "__version__":
         return __version__
-    # ── BEGIN TRANSITIONAL DEPRECATION BRANCH (Decision L; REMOVE AT v0.47) ──
-    if name in _DEPRECATED_SCALARS:
-        import warnings
-
-        warnings.warn(
-            _deprecation_warning_for(name),
-            DeprecationWarning,
-            stacklevel=2,
-        )
-        module = import_module("eval_toolkit.metrics")
-        value = getattr(module, name)
-        # Do NOT cache in globals() — repeated lookups should keep re-warning
-        # (one warning per call site, modulo Python's default
-        # DeprecationWarning de-duplication).
-        return value
-    # ── END TRANSITIONAL DEPRECATION (Decision L; REMOVE AT v0.47) ──
     module_name = _EXPORTS.get(name)
     if module_name is None:
         raise AttributeError(f"module 'eval_toolkit' has no attribute {name!r}")
@@ -362,113 +350,6 @@ def __getattr__(name: str) -> Any:
     return value
 
 
-# ── BEGIN TRANSITIONAL DEPRECATION HELPER (Decision L; REMOVE AT v0.47) ──
-#
-# Per Round 6 audit (Codex R6-F2 + Gemini R6-F2; Decisions R6-F + R6-G):
-# - For deprecated scalars with a first-party `metric_specs` equivalent, the
-#   warning emits an EXECUTABLE scorecard snippet (factory expression + the
-#   correct encoded scorecard key, not the factory call string).
-# - For the 3 ECE variants without a `metric_specs` equivalent
-#   (expected_calibration_error_debiased / _l2 / _l2_debiased), the warning
-#   instead points at the submodule path per Decision R6-G — no first-party
-#   replacement is shipped at v0.47.
-# - ECE `n_bins=10` preserves the pre-v0.46 default (verified at
-#   `metrics.py:730-734`) — Decision R6-F. A migration note explains that
-#   the v0.46+ `metric_specs.ece()` factory defaults to `n_bins=15` (matching
-#   Hines et al.) and how to opt in.
-_FirstParty = tuple[str, str]  # (factory_expression, scorecard_key)
-"""Type alias for a deprecated-scalar that has a metric_specs replacement.
-
-The factory expression is what the user types after ``metric_specs.``; the
-scorecard key is the literal string that indexes ``Scorecard``.
-"""
-
-
-_FIRST_PARTY_REPLACEMENTS: dict[str, _FirstParty] = {
-    "pr_auc": ("pr_auc", "pr_auc"),
-    "roc_auc": ("roc_auc", "roc_auc"),
-    "brier_score": ("brier", "brier"),
-    # ECE variants: use n_bins=10 (pre-v0.46 default per Decision R6-F).
-    # The migration note in the warning text explains how to switch to
-    # n_bins=15 if the user wants the v0.46+ metric_specs.ece() default.
-    "expected_calibration_error": (
-        "ece(n_bins=10)",
-        "ece_n_bins_10_strategy_uniform",
-    ),
-    "expected_calibration_error_equal_mass": (
-        'ece(n_bins=10, strategy="quantile")',
-        "ece_n_bins_10_strategy_quantile",
-    ),
-}
-"""Names that have a first-party metric_specs replacement at v0.46.
-
-The 3 ECE variants NOT in this map (_debiased, _l2, _l2_debiased) get the
-submodule-path warning template instead (Decision R6-G).
-"""
-
-
-def _deprecation_warning_for(name: str) -> str:
-    """Render the DeprecationWarning message for a deprecated scalar name.
-
-    Branches on whether ``name`` has a first-party `metric_specs` replacement
-    (Decision R6-G):
-
-    - First-party (5 names): scorecard snippet with the correct encoded key
-      (Decision R6-F).
-    - Submodule-only (3 ECE variants): point at the submodule path per
-      Decision R6-G.
-
-    The first-party variants for ECE include a migration note explaining the
-    new ``metric_specs.ece()`` factory default of ``n_bins=15`` so users can
-    opt in to the new convention; the snippet itself uses ``n_bins=10`` for
-    bit-identical pre-v0.46 math (Decision R6-F).
-
-    Parameters
-    ----------
-    name : str
-        A name in ``_DEPRECATED_SCALARS``.
-
-    Returns
-    -------
-    str
-        The warning message, ready to pass to ``warnings.warn``.
-    """
-    first_party = _FIRST_PARTY_REPLACEMENTS.get(name)
-    if first_party is not None:
-        factory_expr, scorecard_key = first_party
-        msg = (
-            f"eval_toolkit.{name} is deprecated and will be removed in v0.47. "
-            f"For the same math, use:\n"
-            f"    scorecard(y, s, metrics=[metric_specs.{factory_expr}])"
-            f'["{scorecard_key}"].value\n'
-            f"Or import from the eval_toolkit.metrics submodule directly "
-            f"(internal API per ADR 0002 — stable across v1.x, subject to "
-            f"refactor in major versions)."
-        )
-        # ECE-specific migration note about the n_bins default change.
-        if name.startswith("expected_calibration_error"):
-            msg += (
-                "\nNote: the v0.46+ metric_specs.ece() factory defaults to "
-                "n_bins=15 (matching Hines et al.); the n_bins=10 in this "
-                "snippet preserves the pre-v0.46 math. Pass n_bins=15 to use "
-                "the new convention."
-            )
-        return msg
-    # Decision R6-G: 3 ECE variants without first-party replacements →
-    # submodule path only.
-    return (
-        f"eval_toolkit.{name} is deprecated and will be removed in v0.47. "
-        f"This variant is NOT in v0.46+ metric_specs. Use:\n"
-        f"    from eval_toolkit.metrics import {name}\n"
-        f"(internal API per ADR 0002 — stable across v1.x, subject to "
-        f"refactor in major versions). Or contribute the variant to "
-        f"metric_specs if you use it regularly."
-    )
-
-
-# ── END TRANSITIONAL DEPRECATION HELPER (Decision L; REMOVE AT v0.47) ──
-
-
 def __dir__() -> list[str]:
     """Expose lazy public symbols to introspection."""
     return sorted(__all__)