closed-loop-default-detection 0.3.0__tar.gz → 0.4.0__tar.gz

{closed_loop_default_detection-0.3.0 → closed_loop_default_detection-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: closed-loop-default-detection
-Version: 0.3.0
+Version: 0.4.0
 Summary: Closed loop that measures selective-labels default detection on synthetic SMB lending cohorts and finds the PD model's operating frontier.
 Author: Hossain Pazooki
 License-Expression: MIT
@@ -26,6 +26,7 @@ Requires-Dist: matplotlib>=3.8
 Provides-Extra: dev
 Requires-Dist: pytest>=8.0; extra == "dev"
 Requires-Dist: pytest-cov>=5; extra == "dev"
+Requires-Dist: build>=1; extra == "dev"
 Provides-Extra: docs
 Requires-Dist: sphinx>=7; extra == "docs"
 Requires-Dist: furo; extra == "docs"
@@ -48,134 +49,14 @@ hides labels the way real approval policies do, and grades every correction agai
 - **Deterministic** — byte-identical per seed, scikit-learn-only, no services or GPUs.
 - **Pluggable** — correction levers (IPW, retrain, exploration, reject inference) are classes;
   add yours by subclassing `Corrector`.
-- **Honest by construction** — every number below recomputes from committed CSVs; limits are
-  reported, not smoothed over.
-
-## The result it produces
-
-The loop escalates selection severity until correction fails and reports the **operating
-frontier** — the last severity at which declined-cohort calibration still holds (target
-ECE ≤ 0.10). From the committed runs (`artifacts/loop_frontier*.csv`, seed 42):
-
-| Selection severity | 0.0 | 0.2 | 0.4 | 0.6 |
-|---|---|---|---|---|
-| Naive declined ECE (flat world) | 0.021 | 0.045 | 0.108 | 0.161 |
-| **IPW-corrected** (flat world) | 0.020 | 0.038 | **0.086 ✓** | **0.154 ✗** |
-| **IPW-corrected** (SCM world) | 0.036 | 0.038 | **0.097 ✓** | **0.244 ✗** |
-
-On **this seed** both worlds land the frontier at severity 0.4, and the counterfactual
-deliverable breaks at the same boundary: across 25 seeds, g-computation cuts
-strong-propagation counterfactual MAE from 0.099 to 0.086 (−13.5%, positive on 24/25 seeds,
-Wilcoxon p = 1.5e-7) *inside* the frontier — and collapses to a negligible +0.0017 at full
-severity, where **no deployable advantage is claimed**. One cause explains both: selection
-through an **unobserved confounder**, which backdoor adjustment and IPW cannot fix. That
-single measured limit — not an unverifiable score — is the deliverable.
-
-### The frontier is a distribution, not a point (v2)
-
-A frontier quoted from one seed is a figure published without an error bar. `v0.2.0` runs the
-loop across the full 25-seed set (`artifacts/frontier_sweep.csv`) — and **seed 42 turns out to
-sit at the optimistic end**:
-
-| World | min | median | max | seeds at 0.4 | seeds at 0.2 |
-|---|---|---|---|---|---|
-| flat | 0.2 | **0.4** | 0.4 | 15/25 | 10/25 |
-| SCM | 0.2 | **0.2** | 0.4 | 11/25 | 14/25 |
-
-In the SCM world the *majority of seeds fail one step earlier than the headline*: the median
-frontier is **0.2**, and seed 42's 0.4 is a minority outcome (11/25). The honest statement is
-that the operating frontier is **0.2–0.4 depending on the draw**, not a clean 0.4 — the
-single-seed table above is a valid instance of it, not its center. Nothing about the
-mechanism changes (the unobserved confounder still explains the failure); what changes is how
-precisely the boundary can be quoted.
-
-*Caveat on the sweep:* loop seed `s` consumes generator seeds `s..s+7`, and the 25-seed set has
-gaps smaller than 8, so some runs share feature draws. No two runs duplicate a cohort, but the
-25 rows are not fully independent — the set is kept for comparability with the counterfactual
-sweep that uses it.
-
-Reproduce the headline from committed evidence: `python scripts/paired_significance.py`.
-The full independent assessment (methodology, all numbers, what didn't hold) is the
-accompanying article, [`docs/assessment.md`](docs/assessment.md) — a **dated snapshot**, written
-against the single-seed frontier and not retro-fitted with the distribution above.
-
-## Pricing the frontier (v2)
-
-Calibration says *whether* the model is wrong on the declines; it does not say **what being
-wrong costs**. v2 adds an expected-maximum-profit (EMP) reporting axis — computed from the
-same in-process scores the loop already measures, and **never** a loop-control input (ECE
-still decides pass/fail and the frontier). Two variants run side by side:
-
-| Variant | Parameters | What it answers |
-|---|---|---|
-| `empc` — literature EMPC | Verbraken et al. (2014) prior: `p0=0.55, p1=0.10, ROI=0.2644` | benchmark-comparable: what the standard measure says |
-| `emp_h` — harness-derived | this harness's own economics + **planted** default timing | what the loan structure actually pays |
-
-**The two disagree — and the disagreement is the finding.** In the SCM world they move in
-opposite directions as selection severity rises (from `artifacts/loop_frontier_scm.csv`,
-declined subpopulation, seed 42):
-
-| Selection severity | 0.0 | 0.2 | 0.4 | 0.6 |
-|---|---|---|---|---|
-| Literature `empc` (naive) | 0.0217 | 0.0331 | 0.0416 | **0.0420 ↑** |
-| Harness `emp_h` (naive) | 0.0387 | 0.0297 | 0.0139 | **0.0024 ↓** |
-
-The convenience prior **misprices this loan structure**: it assumes `ROI = 0.2644` where a
-60-day daily-ACH loan actually returns `0.0875` (3% origination fee + 5.75% term interest) —
-a **3.0× overstatement** — and puts **55% of defaults at full recovery where the harness plants
-1.2%** (prior mean λ = 0.275 vs harness 0.419). Priced honestly, the profit on the declined
-pool collapses to near-zero at the frontier; priced by the literature prior, it appears to
-*grow*. Same model, same scores, opposite conclusion.
-
-**Reading caveats — the boundary of what this measures:**
-
-- **Raw EMP moves with world hardness.** A riskier declined pool changes EMP even for a
-  perfect model, so a cross-severity EMP delta is *not* pure model signal. Read the two
-  variants against each other at fixed severity, not the trend in isolation.
-- **`emp_h` rests on unfitted timing.** `days_to_default` is planted but independent of
-  features and risk given default — spec-shaped, never validated against real recovery data.
-  So `emp_h` is a **verified experiment, not a verified result**: the arithmetic is exact, the
-  timing distribution is an assumption.
-- **Post-term defaults are imputed.** ~22.5% of planted defaults land at day 90, past the
-  60-day term, and the generator does not model their payment history; they are priced at the
-  cohort's mean in-term loss fraction (a stated convention, not measured truth).
-- **`emp_h` is SCM-only.** The flat generator plants no timing, so its `emp_h` columns are
-  empty by design; only `empc` is reported there.
-
-Exploration now carries a price tag too. From `artifacts/exploration_frontier.csv` (SCM,
-seed 42, 10% budget, severity 0.6): the lever buys **157 labels for $439,578** — about
-**$2,800 per label**, of which 56 are realized defaults. That is the cost of *identification*,
-in dollars a lender can see, on the only lever that buys it rather than reweighting what is
-already identified.
-
-## Pricing the feedback loop (v3)
-
-`v0.3.0` prices the **dynamic** regime, where the model's own approvals create its next
-training set: three arms (treatment = retrain each generation; **frozen** = generation-0
-model deployed forever; **prior** = prior-policy funding), 25 spaced seeds × 3 severities ×
-2 exploration rates × 12 generations = **450 runs**
-(`artifacts/feedback_profit_sweep.csv`), with three pre-registered hypotheses tested as
-within-seed paired contrasts on realized funded-book P&L at severity 0.4, Holm-corrected.
-Every number below recomputes via `python scripts/feedback_sweep_stats.py`. All of it
-consumes **planted, risk-unlinked default timing** — a verified *experiment*, not a
-verified *result*.
-
-| Hypothesis (severity 0.4, gens 1–11) | Median paired deficit | Sign | Verdict |
-|---|---|---|---|
-| H1 feedback accumulation costs money (treatment − frozen, ε=0) | **−0.0028** | 24/25 | **confirmed** (p_holm 2.3e-06, clears the 0.0018 noise floor) |
-| H2 the policy switch costs money (frozen − prior, ε=0) | +0.0119 | 0/25 | **not confirmed — measured opposite in direction** |
-| H3 exploration buys profit back (treatment ε=.05 − ε=0) | −0.0024 | 0/25 | **not confirmed** |
-
-The decomposition v3 exists for came back one-for-three: letting the loop close on its own
-labels measurably costs the book (H1), but at these severities the *switch* to model-based
-funding is profitable relative to the prior policy (H2's predicted sign was wrong — reported
-as measured, not softened), and 5% exploration does not pay for itself in-window (H3; the
-policy-book-only secondary H3a is negative too). The **H4 integrity control** — frozen-arm
-paired difference must equal the explored slice's own P&L *exactly* — holds at ≤1e-10 across
-all 900 checked rows, and the answer to the v2-banked question is re-verified at 25 seeds:
-the declined-ECE alarm fires at generation 1–2 in every ε=0 run (severity 0.4: 25/25 at
-generation 1). Full mechanics in [`docs/how-it-works.md`](docs/how-it-works.md); gates in
-[`docs/validation.md`](docs/validation.md).
+- **Honest by construction** — every results figure registered in the doc-number gate
+  recomputes from a committed CSV in CI, and every artifact is pinned by content hash; the
+  results and their limits are in [docs/results.md](docs/results.md).
+
+> **What this is — and is not.** CLDD is a *validation harness* graded against planted
+> synthetic ground truth; it is **not a credit-modeling toolkit**. The correction levers it
+> ships (IPW, retrain, exploration, reject inference) exist to be *measured against a known
+> answer* — nothing in this repo is validated for making decisions on real borrower data.
 
 ## Install
 
@@ -277,17 +158,18 @@ python scripts/paired_significance.py         # recompute the headline stat from
 
 ## Validation
 
-`pytest` — 149 tests, all synthetic, no real data needed. CI runs a pinned-repro job (exact
-pins), a cross-version/OS compat matrix, and a strict docs build. Six float-sensitive tests
-reproduce only under the pins in `requirements-dev.txt`; the optional marginal-fidelity gate
-compares the SCM against a **private** real dataset via `CLDD_DATA_DIR` and is the only thing
-that needs it. Details, reproducibility, and troubleshooting:
-[docs/validation.md](docs/validation.md).
+`pytest`, all synthetic, no real data needed. CI runs a pinned-repro job (exact pins), a
+cross-version/OS compat matrix, and a strict docs build; a doc-number gate recomputes every
+registered figure in the docs from a committed artifact. A few float-sensitive tests reproduce
+only under the pins in `requirements-dev.txt`; the optional marginal-fidelity gate compares
+the SCM against a **private** real dataset via `CLDD_DATA_DIR` and is the only thing that
+needs it. Details, reproducibility, and troubleshooting: [docs/validation.md](docs/validation.md).
 
 ## Documentation
 
 | Where | What |
 |---|---|
+| [docs/results.md](docs/results.md) | **every measured result** — the frontier and its distribution, the v4 intervention, the two pre-registered experiments, the two pricing layers |
 | [docs/quickstart.md](docs/quickstart.md) | run the loop, the counterfactual eval, the fidelity report |
 | [docs/how-it-works.md](docs/how-it-works.md) | loop mechanics, diagnostics, feedback simulation, repo map |
 | [docs/configuration.md](docs/configuration.md) | every knob (`config.py`) and the one env var |
@@ -299,12 +181,8 @@ Build locally: `pip install -e ".[docs]" && sphinx-build -b html -W docs docs/_b
 
 ## Status
 
-`0.2.0` **alpha** on [PyPI](https://pypi.org/project/closed-loop-default-detection/),
-changelog in [CHANGELOG.md](CHANGELOG.md).
-
-Shipped in 0.1.0: the loop, both synthetic worlds, all correction levers, the fidelity gate,
-the sklearn estimator, CI on three gates. Added in 0.2.0: `cldd.emp` (both EMP variants),
-priced exploration, the EMP-optimal cutoff, and the 25-seed frontier sweep.
+**Alpha**, on [PyPI](https://pypi.org/project/closed-loop-default-detection/); version
+history in [CHANGELOG.md](CHANGELOG.md).
 
 CLDD began as a validation harness for the Intuit TechWeek SMB Underwriting Challenge; it is
 not a submission and does not alter challenge files.
@@ -319,7 +197,7 @@ Metadata in [`CITATION.cff`](CITATION.cff) (GitHub's "Cite this repository" read
   title   = {{closed-loop-default-detection}: measuring selective-labels default
              detection and the PD model's operating frontier},
   year    = {2026},
-  version = {0.2.0},
+  version = {0.4.0},
   license = {MIT},
   url     = {https://github.com/hossainpazooki/closed-loop-default-detection}
 }

closed_loop_default_detection-0.4.0/README.md

@@ -0,0 +1,169 @@
+# CLDD — closed-loop default detection
+
+[![CI](https://github.com/hossainpazooki/closed-loop-default-detection/actions/workflows/ci.yml/badge.svg)](https://github.com/hossainpazooki/closed-loop-default-detection/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+[![Docs: Sphinx](https://img.shields.io/badge/docs-sphinx-blue.svg)](docs/)
+[![Python ≥3.10](https://img.shields.io/badge/python-%E2%89%A53.10-blue.svg)](pyproject.toml)
+
+**Stress-test a probability-of-default (PD) model under *selective labels* — and get the
+severity at which it breaks.** Real lending data only labels the loans a prior underwriter
+approved, so you cannot measure calibration on the applicants you *declined* — exactly where a
+new model must still be right. CLDD builds synthetic lending worlds with planted ground truth,
+hides labels the way real approval policies do, and grades every correction against that truth.
+
+- **Deterministic** — byte-identical per seed, scikit-learn-only, no services or GPUs.
+- **Pluggable** — correction levers (IPW, retrain, exploration, reject inference) are classes;
+  add yours by subclassing `Corrector`.
+- **Honest by construction** — every results figure registered in the doc-number gate
+  recomputes from a committed CSV in CI, and every artifact is pinned by content hash; the
+  results and their limits are in [docs/results.md](docs/results.md).
+
+> **What this is — and is not.** CLDD is a *validation harness* graded against planted
+> synthetic ground truth; it is **not a credit-modeling toolkit**. The correction levers it
+> ships (IPW, retrain, exploration, reject inference) exist to be *measured against a known
+> answer* — nothing in this repo is validated for making decisions on real borrower data.
+
+## Install
+
+```bash
+pip install closed-loop-default-detection
+```
+
+The import name is **`cldd`**. For development (tests, docs, the committed evidence),
+install from source:
+
+```bash
+git clone https://github.com/hossainpazooki/closed-loop-default-detection.git
+cd closed-loop-default-detection
+pip install -e ".[dev]"
+```
+
+Python ≥ 3.10; dependencies are ranges (`numpy>=2.0`, `pandas>=2.2`, `scikit-learn>=1.6`,
+`scipy>=1.11`, `matplotlib>=3.8`) so `cldd` sits alongside your stack. Exact pins for
+float-exact reproduction: [`requirements-dev.txt`](requirements-dev.txt)
+([details](docs/validation.md)).
+
+## 60-second tour
+
+```python
+from cldd import SelectiveLabelsLoop
+
+result = SelectiveLabelsLoop(improve_mode="both").run()   # "reweight" | "retrain" | "both"
+print("Operating frontier:", result.frontier_severity)
+for r in result.rounds:
+    print(r.selection_severity, r.naive.declined_ece, r.passed)
+```
+
+```mermaid
+flowchart TD
+    A["<b>1. Generate</b><br/>synthetic cohort at a given selection severity<br/>plant true default, then hide it via the approval policy"]
+    B["<b>2. Measure</b><br/>train the PD model on approved rows only,<br/>score it against planted truth on the declined subpopulation"]
+    C["<b>3. Improve</b><br/>apply a correction lever:<br/>IPW reweight &middot; disjoint retrain &middot; exploration"]
+    D{"Corrected declined-cohort<br/>ECE &le; target?"}
+    E["<b>Operating frontier</b><br/>report the highest severity<br/>that still passes"]
+
+    A --> B --> C --> D
+    D -->|"yes &mdash; raise the severity"| A
+    D -->|"no &mdash; stop"| E
+```
+
+A runnable end-to-end demo (classic + custom-lever paths) is
+[`examples/quickstart.py`](examples/quickstart.py). Full mechanics, diagnostics, and the
+feedback simulation: [docs/how-it-works.md](docs/how-it-works.md).
+
+> **Scope.** CLDD is a synthetic **validation harness**, not a production pipeline: retraining
+> and feedback are seeded simulations inside the harness; it never acts on live data or real
+> lending decisions.
+
+## What's in the box
+
+Everything is importable from top-level `cldd` (full reference: the [Sphinx docs](docs/)):
+
+| Import | What it is |
+|---|---|
+| `SelectiveLabelsLoop` | the closed loop; `.run()` → `LoopResult` (frontier + per-round metrics) |
+| `Corrector` + `NaiveCorrector`, `IPWReweightCorrector`, `DisjointRetrainCorrector`, `ExplorationCorrector` | the lever ABC and the four built-ins |
+| `ReclassificationCorrector`, `AugmentationCorrector`, `FuzzyAugmentationCorrector`, `ParcellingCorrector` | four classic reject-inference methods, graded against planted truth ([honest results](docs/reject_inference.md)) |
+| `SyntheticBorrowerGenerator`, `StructuralBorrowerGenerator` | the flat and fitted-SCM synthetic worlds |
+| `run_counterfactual_eval`, `GComputationEstimator` | counterfactual validator (g-computation vs naive conditioning) |
+| `FeedbackLoop` | model-in-the-loop selective-labels simulation |
+| `empc_literature`, `emp_harness` | the two EMP variants — price the frontier in profit (reporting only, never loop control) |
+| `positivity_diagnostics` | observable regime/drift alarm — needs **no** declined-row labels |
+| `CalibratedPDClassifier` | the calibrated PD detector as a scikit-learn estimator |
+| `cldd.fidelity.run_fidelity_gate` | SCM-vs-real **marginal**-fidelity gate (univariate marginals only) |
+
+**Add a lever** by subclassing `Corrector` (`name`, `control_priority`, `apply`) and passing
+`correctors=[NaiveCorrector(), MyCorrector()]` — the legacy `improve_mode` API is unchanged
+and byte-identical. Contract details: [CONTRIBUTING.md](CONTRIBUTING.md).
+
+**Use the detector from sklearn tooling** — `CalibratedPDClassifier` is a thin, tested wrapper
+(binary-only; NaN features OK; the full `check_estimator` battery passes with zero failed
+checks on scikit-learn 1.7.2–1.9.0; probabilities byte-identical to the research API):
+
+```python
+from sklearn.model_selection import cross_val_score
+from cldd import CalibratedPDClassifier
+
+scores = cross_val_score(CalibratedPDClassifier(random_state=42), X, y, scoring="neg_brier_score")
+```
+
+## Command-line drivers
+
+Each driver runs without install (adds `src/` to the path) and writes to `artifacts/`:
+
+```bash
+python scripts/run_loop.py                    # the closed loop → frontier table + plot (--generator scm for the SCM world)
+python scripts/run_frontier_sweep.py --quick  # frontier distribution across the 25-seed set (drop --quick for all)
+python scripts/run_seed_sweep.py --quick      # counterfactual certification (drop --quick for all seeds)
+python scripts/run_reject_inference.py        # reject-inference levers vs the frontier
+python scripts/run_exploration_sweep.py       # frontier vs exploration budget
+python scripts/run_feedback.py                # model-in-the-loop feedback simulation
+python scripts/paired_significance.py         # recompute the headline stat from committed CSVs
+```
+
+## Validation
+
+`pytest`, all synthetic, no real data needed. CI runs a pinned-repro job (exact pins), a
+cross-version/OS compat matrix, and a strict docs build; a doc-number gate recomputes every
+registered figure in the docs from a committed artifact. A few float-sensitive tests reproduce
+only under the pins in `requirements-dev.txt`; the optional marginal-fidelity gate compares
+the SCM against a **private** real dataset via `CLDD_DATA_DIR` and is the only thing that
+needs it. Details, reproducibility, and troubleshooting: [docs/validation.md](docs/validation.md).
+
+## Documentation
+
+| Where | What |
+|---|---|
+| [docs/results.md](docs/results.md) | **every measured result** — the frontier and its distribution, the v4 intervention, the two pre-registered experiments, the two pricing layers |
+| [docs/quickstart.md](docs/quickstart.md) | run the loop, the counterfactual eval, the fidelity report |
+| [docs/how-it-works.md](docs/how-it-works.md) | loop mechanics, diagnostics, feedback simulation, repo map |
+| [docs/configuration.md](docs/configuration.md) | every knob (`config.py`) and the one env var |
+| [docs/validation.md](docs/validation.md) | tests, gates, reproducibility, troubleshooting |
+| [docs/reject_inference.md](docs/reject_inference.md) | the four RI methods and their honest (modest) results |
+| [`docs/assessment.md`](docs/assessment.md) | **the accompanying article** — independent results & methodology assessment (dated snapshot) |
+
+Build locally: `pip install -e ".[docs]" && sphinx-build -b html -W docs docs/_build/html`.
+
+## Status
+
+**Alpha**, on [PyPI](https://pypi.org/project/closed-loop-default-detection/); version
+history in [CHANGELOG.md](CHANGELOG.md).
+
+CLDD began as a validation harness for the Intuit TechWeek SMB Underwriting Challenge; it is
+not a submission and does not alter challenge files.
+
+## Citation
+
+Metadata in [`CITATION.cff`](CITATION.cff) (GitHub's "Cite this repository" reads it):
+
+```bibtex
+@software{pazooki_cldd_2026,
+  author  = {Pazooki, Hossain},
+  title   = {{closed-loop-default-detection}: measuring selective-labels default
+             detection and the PD model's operating frontier},
+  year    = {2026},
+  version = {0.4.0},
+  license = {MIT},
+  url     = {https://github.com/hossainpazooki/closed-loop-default-detection}
+}
+```

{closed_loop_default_detection-0.3.0 → closed_loop_default_detection-0.4.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "closed-loop-default-detection"
-version = "0.3.0"
+version = "0.4.0"
 description = "Closed loop that measures selective-labels default detection on synthetic SMB lending cohorts and finds the PD model's operating frontier."
 readme = "README.md"
 requires-python = ">=3.10"
@@ -45,7 +45,7 @@ dependencies = [
 ]
 
 [project.optional-dependencies]
-dev = ["pytest>=8.0", "pytest-cov>=5"]
+dev = ["pytest>=8.0", "pytest-cov>=5", "build>=1"]
 docs = ["sphinx>=7", "furo", "myst-parser>=2"]
 
 [project.urls]
@@ -70,7 +70,7 @@ markers = [
 # Coverage is opt-in (run `pytest --cov=cldd`); it is not wired into addopts so a
 # plain `pytest` stays lightweight. The private-data fidelity path (cldd/fidelity.py)
 # is only exercised when the real dataset is present, so its data-loading branch is
-# expected to be uncovered on public CI by design — see README.
+# expected to be uncovered on public CI by design — see docs/validation.md.
 [tool.coverage.run]
 source = ["cldd"]
 branch = true

{closed_loop_default_detection-0.3.0 → closed_loop_default_detection-0.4.0}/src/cldd/__init__.py

@@ -11,7 +11,7 @@ from importlib.metadata import PackageNotFoundError, version as _pkg_version
 try:  # pyproject `version` is the single source of truth (read from installed metadata)
     __version__ = _pkg_version("closed-loop-default-detection")
 except PackageNotFoundError:  # running from a source tree with no install
-    __version__ = "0.2.0"
+    __version__ = "0.4.0"
 
 from .correctors import (
     CorrectionOutcome,
@@ -57,6 +57,7 @@ from .scm import (
     dag_children,
     dag_parents,
 )
+from .selection import SegmentedSelection, knockout_rates
 from .synthetic import SyntheticBorrowerGenerator
 
 __all__ = [
@@ -105,4 +106,6 @@ __all__ = [
     "GComputationEstimator",
     "run_counterfactual_eval",
     "generate_queries",
+    "SegmentedSelection",
+    "knockout_rates",
 ]

{closed_loop_default_detection-0.3.0 → closed_loop_default_detection-0.4.0}/src/cldd/counterfactual.py

@@ -632,6 +632,7 @@ def run_counterfactual_eval(
     n_query_applicants: int = 300,
     seed: int = config.RANDOM_SEED,
     generator: StructuralBorrowerGenerator | None = None,
+    unobserved_strength: float | None = None,
 ) -> CounterfactualResult:
     """Build a cohort, generate Deliverable-C-style queries, grade the estimators.
 
@@ -644,13 +645,27 @@ def run_counterfactual_eval(
     Both deployable estimators are fit on the APPROVED rows only (selective-labels
     regime), so their conditionals are distorted by selection — exactly the
     confound g-computation's backdoor adjustment is meant to repair.
+
+    ``unobserved_strength`` (v4 surface knob) is forwarded to the internal
+    ``StructuralBorrowerGenerator`` construction only when ``generator`` is None;
+    the effective value is echoed in ``meta``. ``None`` (the default) is
+    byte-identical to the pre-knob path.
     """
+    if generator is not None and unobserved_strength is not None:
+        raise ValueError(
+            "unobserved_strength is forwarded to the internal generator "
+            "construction only; pass it OR a pre-built generator, not both "
+            "(silently ignoring it would let a driver drift from the eval)"
+        )
     if generator is None:
-        generator = StructuralBorrowerGenerator(
+        gen_kwargs: dict = dict(
             n_applicants=n_applicants,
             selection_severity=selection_severity,
             seed=seed,
         )
+        if unobserved_strength is not None:
+            gen_kwargs["unobserved_strength"] = unobserved_strength
+        generator = StructuralBorrowerGenerator(**gen_kwargs)
     cohort = generator.generate_cohort()
     state = cohort["scm_state"]
     features = cohort["features"]
@@ -763,5 +778,6 @@ def run_counterfactual_eval(
             "n_applicants": n_applicants,
             "seed": seed,
             "approval_rate": float(approved.mean()),
+            "unobserved_strength": float(generator.unobserved_strength),
         },
     )

{closed_loop_default_detection-0.3.0 → closed_loop_default_detection-0.4.0}/src/cldd/emp.py

@@ -1,7 +1,7 @@
 """Expected-maximum-profit (EMP) reporting layer — the v2 measurement axis.
 
 Prices the frontier the loop already measures; **never** a loop-control input
-(spec: ``docs/superpowers/specs/2026-07-13-cldd-v2-emp-design.md``). Two
+(spec: ``2026-07-13-cldd-v2-emp-design.md``). Two
 variants, side by side:
 
 * :func:`empc_literature` — the Verbraken/Bravo/Weber/Baesens EMPC closed form
@@ -18,7 +18,7 @@ of mean per-loan exposure** (exposure = ``requested_amount``), so the two
 columns are directly comparable. Both consume ``scores`` as a ranking only and
 are invariant to strictly monotone transforms.
 
-Reading caveats (documented in README + CSV column notes):
+Reading caveats (documented in docs/results.md + CSV column notes):
 
 * Planted default timing is **marginally spec-shaped but conditionally
   cosmetic** — independent of features/risk given default status — so harness

{closed_loop_default_detection-0.3.0 → closed_loop_default_detection-0.4.0}/src/cldd/feedback.py

@@ -23,7 +23,7 @@ correction (IPW, retrain) is undefined on the declined side.
   labeled-propensity weights — the identification-buying lever from the static
   loop, now acting as a *stabilizer* of the feedback dynamics.
 
-v3 (docs/superpowers/specs/2026-07-14-cldd-v3-design.md) adds two additive flags,
+v3 (spec 2026-07-14-cldd-v3-design.md) adds two additive flags,
 byte-exact at their defaults, that isolate the trajectory's causes into three
 arms: ``retrain=False`` freezes the generation-0 model forever (isolates
 feedback accumulation — the "frozen" arm), and ``policy_mode="prior"`` funds

{closed_loop_default_detection-0.3.0 → closed_loop_default_detection-0.4.0}/src/cldd/fidelity.py

@@ -53,19 +53,14 @@ from .scm import StructuralBorrowerGenerator
 # Configuration
 # --------------------------------------------------------------------------- #
 
-#: Historical author-machine dataset location, kept only as a last-resort fallback
-#: so the maintainer's local ``check_fidelity`` run needs no configuration.
-_FALLBACK_DATA_DIR = Path(
-    "C:/Users/hossa/dev/intuit-techweek-hackathon/"
-    "intuit-techweek-nyc-hackathon-2026/dataset"
-)
-
-#: Default location of the real Intuit dataset. The dataset is **private and not
+#: Default location of the real Intuit dataset, read from the ``CLDD_DATA_DIR``
+#: environment variable; ``None`` when it is unset. The dataset is **private and not
 #: shipped with the package**, so the fidelity gate is runnable only where that
-#: data is available. Point the gate at your copy with the ``CLDD_DATA_DIR``
-#: environment variable (the portable knob) or by passing ``data_dir=`` explicitly;
-#: absent both, this falls back to the maintainer's historical path.
-DEFAULT_DATA_DIR = Path(os.environ.get("CLDD_DATA_DIR", _FALLBACK_DATA_DIR))
+#: data is available. ``CLDD_DATA_DIR`` is the only data-location knob; ``data_dir=``
+#: on the loaders overrides it per call.
+DEFAULT_DATA_DIR: Path | None = (
+    Path(os.environ["CLDD_DATA_DIR"]) if "CLDD_DATA_DIR" in os.environ else None
+)
 
 #: Continuous features compared on p1/p50/p99 (modeled in the SCM marginals).
 CONTINUOUS_FEATURES = (
@@ -246,15 +241,29 @@ class FidelityReport:
 # --------------------------------------------------------------------------- #
 
 
-def load_real(data_dir: str | Path = DEFAULT_DATA_DIR, split: str = "train") -> pd.DataFrame:
+def _resolve_data_dir(data_dir: str | Path | None) -> Path:
+    if data_dir is None:
+        data_dir = DEFAULT_DATA_DIR
+    if data_dir is None:
+        raise FileNotFoundError(
+            "real Intuit dataset location is not set. The dataset is private and is "
+            "not shipped with cldd; set the CLDD_DATA_DIR environment variable (or "
+            "pass data_dir=) to the directory containing train.csv."
+        )
+    return Path(data_dir)
+
+
+def load_real(data_dir: str | Path | None = None, split: str = "train") -> pd.DataFrame:
     """Load a split of the real Intuit dataset as a DataFrame.
 
     Parameters
     ----------
-    data_dir : path to the dataset directory (defaults to :data:`DEFAULT_DATA_DIR`).
+    data_dir : path to the dataset directory; ``None`` means :data:`DEFAULT_DATA_DIR`
+        (the ``CLDD_DATA_DIR`` environment variable), and if that is unset too a
+        ``FileNotFoundError`` names the variable.
     split : one of ``"train"``, ``"validation"``, ``"test"`` (file ``<split>.csv``).
     """
-    path = Path(data_dir) / f"{split}.csv"
+    path = _resolve_data_dir(data_dir) / f"{split}.csv"
     if not path.exists():
         raise FileNotFoundError(
             f"real Intuit dataset not found: {path}. The dataset is private and is "
@@ -411,7 +420,7 @@ def compute_fidelity(
 
 
 def run_fidelity_gate(
-    data_dir: str | Path = DEFAULT_DATA_DIR,
+    data_dir: str | Path | None = None,
     split: str = "train",
     n_applicants: int = 12000,
     seed: int = 42,
@@ -426,5 +435,5 @@ def run_fidelity_gate(
         n_applicants=n_applicants, seed=seed, generator=generator, **generator_kwargs
     )
     report = compute_fidelity(real, cohort, strict_categoricals=strict_categoricals)
-    report.data_dir = str(data_dir)
+    report.data_dir = str(data_dir if data_dir is not None else DEFAULT_DATA_DIR)
     return report

{closed_loop_default_detection-0.3.0 → closed_loop_default_detection-0.4.0}/src/cldd/loop.py

@@ -92,13 +92,18 @@ def make_generator(
     seed: int,
     n_applicants: int,
     approval_rate: float,
+    **extra,
 ):
     """Fresh single-shot cohort generator (shared by this loop and ``feedback``).
 
     A fresh instance per cohort is required for determinism — both generator
     classes consume their own ``Generator(PCG64(seed))`` on ``generate_cohort``,
     so reusing an instance would silently shift streams. The SCM path always
-    sets ``independent_selection_noise=True`` (see ``GENERATORS`` note).
+    sets ``independent_selection_noise=True`` (see ``GENERATORS`` note). Any
+    ``**extra`` kwargs are forwarded verbatim to the generator constructor (v4:
+    the surface driver passes ``unobserved_strength``); the scm path's
+    ``independent_selection_noise=True`` cannot be overridden this way — a
+    duplicate keyword raises ``TypeError``.
     """
     if generator not in GENERATORS:
         raise ValueError(f"generator must be one of {GENERATORS}; got {generator!r}")
@@ -118,12 +123,14 @@ def make_generator(
             approval_rate=approval_rate,
             seed=seed,
             independent_selection_noise=True,
+            **extra,
         )
     return SyntheticBorrowerGenerator(
         n_applicants=n_applicants,
         selection_severity=severity,
         approval_rate=approval_rate,
         seed=seed,
+        **extra,
     )
 
 
@@ -203,6 +210,7 @@ class SelectiveLabelsLoop:
         generator: str = "flat",
         exploration_rate: float = 0.0,
         correctors: list[Corrector] | None = None,
+        generator_kwargs: dict | None = None,
     ) -> None:
         if improve_mode not in IMPROVE_MODES:
             raise ValueError(f"improve_mode must be one of {IMPROVE_MODES}; got {improve_mode!r}")
@@ -211,6 +219,9 @@ class SelectiveLabelsLoop:
         if not 0.0 <= exploration_rate < 1.0:
             raise ValueError(f"exploration_rate must be in [0, 1); got {exploration_rate!r}")
         self.generator = generator
+        # Extra constructor kwargs forwarded to make_generator (v4 surface knob).
+        # None (the default) is byte-identical to the pre-knob path.
+        self.generator_kwargs = generator_kwargs
         self.exploration_rate = exploration_rate
         self.target_declined_ece = target_declined_ece
         self.start_severity = start_severity
@@ -265,6 +276,7 @@ class SelectiveLabelsLoop:
             seed=seed,
             n_applicants=self.n_applicants,
             approval_rate=self.approval_rate,
+            **(self.generator_kwargs or {}),
         )
 
     def _generate(self, severity: float, iteration: int) -> dict: