closed-loop-default-detection 0.1.0__tar.gz → 0.3.0__tar.gz

{closed_loop_default_detection-0.1.0 → closed_loop_default_detection-0.3.0}/PKG-INFO

@@ -1,8 +1,8 @@
 Metadata-Version: 2.4
 Name: closed-loop-default-detection
-Version: 0.1.0
-Summary: CLUE-style closed loop that measures selective-labels default detection on synthetic SMB lending cohorts and finds the PD model's operating frontier.
-Author-email: Hossain Pazooki <hossain@pazooki.com>
+Version: 0.3.0
+Summary: Closed loop that measures selective-labels default detection on synthetic SMB lending cohorts and finds the PD model's operating frontier.
+Author: Hossain Pazooki
 License-Expression: MIT
 Project-URL: Homepage, https://github.com/hossainpazooki/closed-loop-default-detection
 Project-URL: Repository, https://github.com/hossainpazooki/closed-loop-default-detection
@@ -55,7 +55,7 @@ hides labels the way real approval policies do, and grades every correction agai
 
 The loop escalates selection severity until correction fails and reports the **operating
 frontier** — the last severity at which declined-cohort calibration still holds (target
-ECE ≤ 0.10). From the committed runs (`artifacts/clue_frontier*.csv`, seed 42):
+ECE ≤ 0.10). From the committed runs (`artifacts/loop_frontier*.csv`, seed 42):
 
 | Selection severity | 0.0 | 0.2 | 0.4 | 0.6 |
 |---|---|---|---|---|
@@ -63,17 +63,119 @@ ECE ≤ 0.10). From the committed runs (`artifacts/clue_frontier*.csv`, seed 42)
 | **IPW-corrected** (flat world) | 0.020 | 0.038 | **0.086 ✓** | **0.154 ✗** |
 | **IPW-corrected** (SCM world) | 0.036 | 0.038 | **0.097 ✓** | **0.244 ✗** |
 
-Both worlds land the frontier at **severity 0.4**, and the counterfactual deliverable breaks at
-the same boundary: across 25 seeds, g-computation cuts strong-propagation counterfactual MAE
-from 0.099 to 0.086 (−13.5%, positive on 24/25 seeds, Wilcoxon p = 1.5e-7) *inside* the
-frontier — and collapses to a negligible +0.0017 at full severity, where **no deployable
-advantage is claimed**. One cause explains both: selection through an **unobserved
-confounder**, which backdoor adjustment and IPW cannot fix. That single measured limit — not
-an unverifiable score — is the deliverable.
+On **this seed** both worlds land the frontier at severity 0.4, and the counterfactual
+deliverable breaks at the same boundary: across 25 seeds, g-computation cuts
+strong-propagation counterfactual MAE from 0.099 to 0.086 (−13.5%, positive on 24/25 seeds,
+Wilcoxon p = 1.5e-7) *inside* the frontier — and collapses to a negligible +0.0017 at full
+severity, where **no deployable advantage is claimed**. One cause explains both: selection
+through an **unobserved confounder**, which backdoor adjustment and IPW cannot fix. That
+single measured limit — not an unverifiable score — is the deliverable.
+
+### The frontier is a distribution, not a point (v2)
+
+A frontier quoted from one seed is a figure published without an error bar. `v0.2.0` runs the
+loop across the full 25-seed set (`artifacts/frontier_sweep.csv`) — and **seed 42 turns out to
+sit at the optimistic end**:
+
+| World | min | median | max | seeds at 0.4 | seeds at 0.2 |
+|---|---|---|---|---|---|
+| flat | 0.2 | **0.4** | 0.4 | 15/25 | 10/25 |
+| SCM | 0.2 | **0.2** | 0.4 | 11/25 | 14/25 |
+
+In the SCM world the *majority of seeds fail one step earlier than the headline*: the median
+frontier is **0.2**, and seed 42's 0.4 is a minority outcome (11/25). The honest statement is
+that the operating frontier is **0.2–0.4 depending on the draw**, not a clean 0.4 — the
+single-seed table above is a valid instance of it, not its center. Nothing about the
+mechanism changes (the unobserved confounder still explains the failure); what changes is how
+precisely the boundary can be quoted.
+
+*Caveat on the sweep:* loop seed `s` consumes generator seeds `s..s+7`, and the 25-seed set has
+gaps smaller than 8, so some runs share feature draws. No two runs duplicate a cohort, but the
+25 rows are not fully independent — the set is kept for comparability with the counterfactual
+sweep that uses it.
 
 Reproduce the headline from committed evidence: `python scripts/paired_significance.py`.
 The full independent assessment (methodology, all numbers, what didn't hold) is the
-accompanying article, [`FABLE.md`](FABLE.md).
+accompanying article, [`docs/assessment.md`](docs/assessment.md) — a **dated snapshot**, written
+against the single-seed frontier and not retro-fitted with the distribution above.
+
+## Pricing the frontier (v2)
+
+Calibration says *whether* the model is wrong on the declines; it does not say **what being
+wrong costs**. v2 adds an expected-maximum-profit (EMP) reporting axis — computed from the
+same in-process scores the loop already measures, and **never** a loop-control input (ECE
+still decides pass/fail and the frontier). Two variants run side by side:
+
+| Variant | Parameters | What it answers |
+|---|---|---|
+| `empc` — literature EMPC | Verbraken et al. (2014) prior: `p0=0.55, p1=0.10, ROI=0.2644` | benchmark-comparable: what the standard measure says |
+| `emp_h` — harness-derived | this harness's own economics + **planted** default timing | what the loan structure actually pays |
+
+**The two disagree — and the disagreement is the finding.** In the SCM world they move in
+opposite directions as selection severity rises (from `artifacts/loop_frontier_scm.csv`,
+declined subpopulation, seed 42):
+
+| Selection severity | 0.0 | 0.2 | 0.4 | 0.6 |
+|---|---|---|---|---|
+| Literature `empc` (naive) | 0.0217 | 0.0331 | 0.0416 | **0.0420 ↑** |
+| Harness `emp_h` (naive) | 0.0387 | 0.0297 | 0.0139 | **0.0024 ↓** |
+
+The convenience prior **misprices this loan structure**: it assumes `ROI = 0.2644` where a
+60-day daily-ACH loan actually returns `0.0875` (3% origination fee + 5.75% term interest) —
+a **3.0× overstatement** — and puts **55% of defaults at full recovery where the harness plants
+1.2%** (prior mean λ = 0.275 vs harness 0.419). Priced honestly, the profit on the declined
+pool collapses to near-zero at the frontier; priced by the literature prior, it appears to
+*grow*. Same model, same scores, opposite conclusion.
+
+**Reading caveats — the boundary of what this measures:**
+
+- **Raw EMP moves with world hardness.** A riskier declined pool changes EMP even for a
+  perfect model, so a cross-severity EMP delta is *not* pure model signal. Read the two
+  variants against each other at fixed severity, not the trend in isolation.
+- **`emp_h` rests on unfitted timing.** `days_to_default` is planted but independent of
+  features and risk given default — spec-shaped, never validated against real recovery data.
+  So `emp_h` is a **verified experiment, not a verified result**: the arithmetic is exact, the
+  timing distribution is an assumption.
+- **Post-term defaults are imputed.** ~22.5% of planted defaults land at day 90, past the
+  60-day term, and the generator does not model their payment history; they are priced at the
+  cohort's mean in-term loss fraction (a stated convention, not measured truth).
+- **`emp_h` is SCM-only.** The flat generator plants no timing, so its `emp_h` columns are
+  empty by design; only `empc` is reported there.
+
+Exploration now carries a price tag too. From `artifacts/exploration_frontier.csv` (SCM,
+seed 42, 10% budget, severity 0.6): the lever buys **157 labels for $439,578** — about
+**$2,800 per label**, of which 56 are realized defaults. That is the cost of *identification*,
+in dollars a lender can see, on the only lever that buys it rather than reweighting what is
+already identified.
+
+## Pricing the feedback loop (v3)
+
+`v0.3.0` prices the **dynamic** regime, where the model's own approvals create its next
+training set: three arms (treatment = retrain each generation; **frozen** = generation-0
+model deployed forever; **prior** = prior-policy funding), 25 spaced seeds × 3 severities ×
+2 exploration rates × 12 generations = **450 runs**
+(`artifacts/feedback_profit_sweep.csv`), with three pre-registered hypotheses tested as
+within-seed paired contrasts on realized funded-book P&L at severity 0.4, Holm-corrected.
+Every number below recomputes via `python scripts/feedback_sweep_stats.py`. All of it
+consumes **planted, risk-unlinked default timing** — a verified *experiment*, not a
+verified *result*.
+
+| Hypothesis (severity 0.4, gens 1–11) | Median paired deficit | Sign | Verdict |
+|---|---|---|---|
+| H1 feedback accumulation costs money (treatment − frozen, ε=0) | **−0.0028** | 24/25 | **confirmed** (p_holm 2.3e-06, clears the 0.0018 noise floor) |
+| H2 the policy switch costs money (frozen − prior, ε=0) | +0.0119 | 0/25 | **not confirmed — measured opposite in direction** |
+| H3 exploration buys profit back (treatment ε=.05 − ε=0) | −0.0024 | 0/25 | **not confirmed** |
+
+The decomposition v3 exists for came back one-for-three: letting the loop close on its own
+labels measurably costs the book (H1), but at these severities the *switch* to model-based
+funding is profitable relative to the prior policy (H2's predicted sign was wrong — reported
+as measured, not softened), and 5% exploration does not pay for itself in-window (H3; the
+policy-book-only secondary H3a is negative too). The **H4 integrity control** — frozen-arm
+paired difference must equal the explored slice's own P&L *exactly* — holds at ≤1e-10 across
+all 900 checked rows, and the answer to the v2-banked question is re-verified at 25 seeds:
+the declined-ECE alarm fires at generation 1–2 in every ε=0 run (severity 0.4: 25/25 at
+generation 1). Full mechanics in [`docs/how-it-works.md`](docs/how-it-works.md); gates in
+[`docs/validation.md`](docs/validation.md).
 
 ## Install
 
@@ -139,6 +241,7 @@ Everything is importable from top-level `cldd` (full reference: the [Sphinx docs
 | `SyntheticBorrowerGenerator`, `StructuralBorrowerGenerator` | the flat and fitted-SCM synthetic worlds |
 | `run_counterfactual_eval`, `GComputationEstimator` | counterfactual validator (g-computation vs naive conditioning) |
 | `FeedbackLoop` | model-in-the-loop selective-labels simulation |
+| `empc_literature`, `emp_harness` | the two EMP variants — price the frontier in profit (reporting only, never loop control) |
 | `positivity_diagnostics` | observable regime/drift alarm — needs **no** declined-row labels |
 | `CalibratedPDClassifier` | the calibrated PD detector as a scikit-learn estimator |
 | `cldd.fidelity.run_fidelity_gate` | SCM-vs-real **marginal**-fidelity gate (univariate marginals only) |
@@ -163,7 +266,8 @@ scores = cross_val_score(CalibratedPDClassifier(random_state=42), X, y, scoring=
 Each driver runs without install (adds `src/` to the path) and writes to `artifacts/`:
 
 ```bash
-python scripts/run_clue.py                    # the closed loop → frontier table + plot (--generator scm for the SCM world)
+python scripts/run_loop.py                    # the closed loop → frontier table + plot (--generator scm for the SCM world)
+python scripts/run_frontier_sweep.py --quick  # frontier distribution across the 25-seed set (drop --quick for all)
 python scripts/run_seed_sweep.py --quick      # counterfactual certification (drop --quick for all seeds)
 python scripts/run_reject_inference.py        # reject-inference levers vs the frontier
 python scripts/run_exploration_sweep.py       # frontier vs exploration budget
@@ -173,7 +277,7 @@ python scripts/paired_significance.py         # recompute the headline stat from
 
 ## Validation
 
-`pytest` — 123 tests, all synthetic, no real data needed. CI runs a pinned-repro job (exact
+`pytest` — 149 tests, all synthetic, no real data needed. CI runs a pinned-repro job (exact
 pins), a cross-version/OS compat matrix, and a strict docs build. Six float-sensitive tests
 reproduce only under the pins in `requirements-dev.txt`; the optional marginal-fidelity gate
 compares the SCM against a **private** real dataset via `CLDD_DATA_DIR` and is the only thing
@@ -189,17 +293,21 @@ that needs it. Details, reproducibility, and troubleshooting:
 | [docs/configuration.md](docs/configuration.md) | every knob (`config.py`) and the one env var |
 | [docs/validation.md](docs/validation.md) | tests, gates, reproducibility, troubleshooting |
 | [docs/reject_inference.md](docs/reject_inference.md) | the four RI methods and their honest (modest) results |
-| [`FABLE.md`](FABLE.md) | **the accompanying article** — independent results & methodology assessment |
+| [`docs/assessment.md`](docs/assessment.md) | **the accompanying article** — independent results & methodology assessment (dated snapshot) |
 
 Build locally: `pip install -e ".[docs]" && sphinx-build -b html -W docs docs/_build/html`.
 
 ## Status
 
-`0.1.0` **alpha** on [PyPI](https://pypi.org/project/closed-loop-default-detection/),
-changelog in [CHANGELOG.md](CHANGELOG.md). Shipped: the loop, both worlds, all levers, the
-fidelity gate, the sklearn estimator, CI on three gates. CLDD began as a validation harness
-for the Intuit TechWeek SMB Underwriting Challenge; it is not a submission and does not
-alter challenge files.
+`0.2.0` **alpha** on [PyPI](https://pypi.org/project/closed-loop-default-detection/),
+changelog in [CHANGELOG.md](CHANGELOG.md).
+
+Shipped in 0.1.0: the loop, both synthetic worlds, all correction levers, the fidelity gate,
+the sklearn estimator, CI on three gates. Added in 0.2.0: `cldd.emp` (both EMP variants),
+priced exploration, the EMP-optimal cutoff, and the 25-seed frontier sweep.
+
+CLDD began as a validation harness for the Intuit TechWeek SMB Underwriting Challenge; it is
+not a submission and does not alter challenge files.
 
 ## Citation
 
@@ -211,7 +319,7 @@ Metadata in [`CITATION.cff`](CITATION.cff) (GitHub's "Cite this repository" read
   title   = {{closed-loop-default-detection}: measuring selective-labels default
              detection and the PD model's operating frontier},
   year    = {2026},
-  version = {0.1.0},
+  version = {0.2.0},
   license = {MIT},
   url     = {https://github.com/hossainpazooki/closed-loop-default-detection}
 }

closed_loop_default_detection-0.3.0/README.md

@@ -0,0 +1,292 @@
+# CLDD — closed-loop default detection
+
+[![CI](https://github.com/hossainpazooki/closed-loop-default-detection/actions/workflows/ci.yml/badge.svg)](https://github.com/hossainpazooki/closed-loop-default-detection/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+[![Docs: Sphinx](https://img.shields.io/badge/docs-sphinx-blue.svg)](docs/)
+[![Python ≥3.10](https://img.shields.io/badge/python-%E2%89%A53.10-blue.svg)](pyproject.toml)
+
+**Stress-test a probability-of-default (PD) model under *selective labels* — and get the
+severity at which it breaks.** Real lending data only labels the loans a prior underwriter
+approved, so you cannot measure calibration on the applicants you *declined* — exactly where a
+new model must still be right. CLDD builds synthetic lending worlds with planted ground truth,
+hides labels the way real approval policies do, and grades every correction against that truth.
+
+- **Deterministic** — byte-identical per seed, scikit-learn-only, no services or GPUs.
+- **Pluggable** — correction levers (IPW, retrain, exploration, reject inference) are classes;
+  add yours by subclassing `Corrector`.
+- **Honest by construction** — every number below recomputes from committed CSVs; limits are
+  reported, not smoothed over.
+
+## The result it produces
+
+The loop escalates selection severity until correction fails and reports the **operating
+frontier** — the last severity at which declined-cohort calibration still holds (target
+ECE ≤ 0.10). From the committed runs (`artifacts/loop_frontier*.csv`, seed 42):
+
+| Selection severity | 0.0 | 0.2 | 0.4 | 0.6 |
+|---|---|---|---|---|
+| Naive declined ECE (flat world) | 0.021 | 0.045 | 0.108 | 0.161 |
+| **IPW-corrected** (flat world) | 0.020 | 0.038 | **0.086 ✓** | **0.154 ✗** |
+| **IPW-corrected** (SCM world) | 0.036 | 0.038 | **0.097 ✓** | **0.244 ✗** |
+
+On **this seed** both worlds land the frontier at severity 0.4, and the counterfactual
+deliverable breaks at the same boundary: across 25 seeds, g-computation cuts
+strong-propagation counterfactual MAE from 0.099 to 0.086 (−13.5%, positive on 24/25 seeds,
+Wilcoxon p = 1.5e-7) *inside* the frontier — and collapses to a negligible +0.0017 at full
+severity, where **no deployable advantage is claimed**. One cause explains both: selection
+through an **unobserved confounder**, which backdoor adjustment and IPW cannot fix. That
+single measured limit — not an unverifiable score — is the deliverable.
+
+### The frontier is a distribution, not a point (v2)
+
+A frontier quoted from one seed is a figure published without an error bar. `v0.2.0` runs the
+loop across the full 25-seed set (`artifacts/frontier_sweep.csv`) — and **seed 42 turns out to
+sit at the optimistic end**:
+
+| World | min | median | max | seeds at 0.4 | seeds at 0.2 |
+|---|---|---|---|---|---|
+| flat | 0.2 | **0.4** | 0.4 | 15/25 | 10/25 |
+| SCM | 0.2 | **0.2** | 0.4 | 11/25 | 14/25 |
+
+In the SCM world the *majority of seeds fail one step earlier than the headline*: the median
+frontier is **0.2**, and seed 42's 0.4 is a minority outcome (11/25). The honest statement is
+that the operating frontier is **0.2–0.4 depending on the draw**, not a clean 0.4 — the
+single-seed table above is a valid instance of it, not its center. Nothing about the
+mechanism changes (the unobserved confounder still explains the failure); what changes is how
+precisely the boundary can be quoted.
+
+*Caveat on the sweep:* loop seed `s` consumes generator seeds `s..s+7`, and the 25-seed set has
+gaps smaller than 8, so some runs share feature draws. No two runs duplicate a cohort, but the
+25 rows are not fully independent — the set is kept for comparability with the counterfactual
+sweep that uses it.
+
+Reproduce the headline from committed evidence: `python scripts/paired_significance.py`.
+The full independent assessment (methodology, all numbers, what didn't hold) is the
+accompanying article, [`docs/assessment.md`](docs/assessment.md) — a **dated snapshot**, written
+against the single-seed frontier and not retro-fitted with the distribution above.
+
+## Pricing the frontier (v2)
+
+Calibration says *whether* the model is wrong on the declines; it does not say **what being
+wrong costs**. v2 adds an expected-maximum-profit (EMP) reporting axis — computed from the
+same in-process scores the loop already measures, and **never** a loop-control input (ECE
+still decides pass/fail and the frontier). Two variants run side by side:
+
+| Variant | Parameters | What it answers |
+|---|---|---|
+| `empc` — literature EMPC | Verbraken et al. (2014) prior: `p0=0.55, p1=0.10, ROI=0.2644` | benchmark-comparable: what the standard measure says |
+| `emp_h` — harness-derived | this harness's own economics + **planted** default timing | what the loan structure actually pays |
+
+**The two disagree — and the disagreement is the finding.** In the SCM world they move in
+opposite directions as selection severity rises (from `artifacts/loop_frontier_scm.csv`,
+declined subpopulation, seed 42):
+
+| Selection severity | 0.0 | 0.2 | 0.4 | 0.6 |
+|---|---|---|---|---|
+| Literature `empc` (naive) | 0.0217 | 0.0331 | 0.0416 | **0.0420 ↑** |
+| Harness `emp_h` (naive) | 0.0387 | 0.0297 | 0.0139 | **0.0024 ↓** |
+
+The convenience prior **misprices this loan structure**: it assumes `ROI = 0.2644` where a
+60-day daily-ACH loan actually returns `0.0875` (3% origination fee + 5.75% term interest) —
+a **3.0× overstatement** — and puts **55% of defaults at full recovery where the harness plants
+1.2%** (prior mean λ = 0.275 vs harness 0.419). Priced honestly, the profit on the declined
+pool collapses to near-zero at the frontier; priced by the literature prior, it appears to
+*grow*. Same model, same scores, opposite conclusion.
+
+**Reading caveats — the boundary of what this measures:**
+
+- **Raw EMP moves with world hardness.** A riskier declined pool changes EMP even for a
+  perfect model, so a cross-severity EMP delta is *not* pure model signal. Read the two
+  variants against each other at fixed severity, not the trend in isolation.
+- **`emp_h` rests on unfitted timing.** `days_to_default` is planted but independent of
+  features and risk given default — spec-shaped, never validated against real recovery data.
+  So `emp_h` is a **verified experiment, not a verified result**: the arithmetic is exact, the
+  timing distribution is an assumption.
+- **Post-term defaults are imputed.** ~22.5% of planted defaults land at day 90, past the
+  60-day term, and the generator does not model their payment history; they are priced at the
+  cohort's mean in-term loss fraction (a stated convention, not measured truth).
+- **`emp_h` is SCM-only.** The flat generator plants no timing, so its `emp_h` columns are
+  empty by design; only `empc` is reported there.
+
+Exploration now carries a price tag too. From `artifacts/exploration_frontier.csv` (SCM,
+seed 42, 10% budget, severity 0.6): the lever buys **157 labels for $439,578** — about
+**$2,800 per label**, of which 56 are realized defaults. That is the cost of *identification*,
+in dollars a lender can see, on the only lever that buys it rather than reweighting what is
+already identified.
+
+## Pricing the feedback loop (v3)
+
+`v0.3.0` prices the **dynamic** regime, where the model's own approvals create its next
+training set: three arms (treatment = retrain each generation; **frozen** = generation-0
+model deployed forever; **prior** = prior-policy funding), 25 spaced seeds × 3 severities ×
+2 exploration rates × 12 generations = **450 runs**
+(`artifacts/feedback_profit_sweep.csv`), with three pre-registered hypotheses tested as
+within-seed paired contrasts on realized funded-book P&L at severity 0.4, Holm-corrected.
+Every number below recomputes via `python scripts/feedback_sweep_stats.py`. All of it
+consumes **planted, risk-unlinked default timing** — a verified *experiment*, not a
+verified *result*.
+
+| Hypothesis (severity 0.4, gens 1–11) | Median paired deficit | Sign | Verdict |
+|---|---|---|---|
+| H1 feedback accumulation costs money (treatment − frozen, ε=0) | **−0.0028** | 24/25 | **confirmed** (p_holm 2.3e-06, clears the 0.0018 noise floor) |
+| H2 the policy switch costs money (frozen − prior, ε=0) | +0.0119 | 0/25 | **not confirmed — measured opposite in direction** |
+| H3 exploration buys profit back (treatment ε=.05 − ε=0) | −0.0024 | 0/25 | **not confirmed** |
+
+The decomposition v3 exists for came back one-for-three: letting the loop close on its own
+labels measurably costs the book (H1), but at these severities the *switch* to model-based
+funding is profitable relative to the prior policy (H2's predicted sign was wrong — reported
+as measured, not softened), and 5% exploration does not pay for itself in-window (H3; the
+policy-book-only secondary H3a is negative too). The **H4 integrity control** — frozen-arm
+paired difference must equal the explored slice's own P&L *exactly* — holds at ≤1e-10 across
+all 900 checked rows, and the answer to the v2-banked question is re-verified at 25 seeds:
+the declined-ECE alarm fires at generation 1–2 in every ε=0 run (severity 0.4: 25/25 at
+generation 1). Full mechanics in [`docs/how-it-works.md`](docs/how-it-works.md); gates in
+[`docs/validation.md`](docs/validation.md).
+
+## Install
+
+```bash
+pip install closed-loop-default-detection
+```
+
+The import name is **`cldd`**. For development (tests, docs, the committed evidence),
+install from source:
+
+```bash
+git clone https://github.com/hossainpazooki/closed-loop-default-detection.git
+cd closed-loop-default-detection
+pip install -e ".[dev]"
+```
+
+Python ≥ 3.10; dependencies are ranges (`numpy>=2.0`, `pandas>=2.2`, `scikit-learn>=1.6`,
+`scipy>=1.11`, `matplotlib>=3.8`) so `cldd` sits alongside your stack. Exact pins for
+float-exact reproduction: [`requirements-dev.txt`](requirements-dev.txt)
+([details](docs/validation.md)).
+
+## 60-second tour
+
+```python
+from cldd import SelectiveLabelsLoop
+
+result = SelectiveLabelsLoop(improve_mode="both").run()   # "reweight" | "retrain" | "both"
+print("Operating frontier:", result.frontier_severity)
+for r in result.rounds:
+    print(r.selection_severity, r.naive.declined_ece, r.passed)
+```
+
+```mermaid
+flowchart TD
+    A["<b>1. Generate</b><br/>synthetic cohort at a given selection severity<br/>plant true default, then hide it via the approval policy"]
+    B["<b>2. Measure</b><br/>train the PD model on approved rows only,<br/>score it against planted truth on the declined subpopulation"]
+    C["<b>3. Improve</b><br/>apply a correction lever:<br/>IPW reweight &middot; disjoint retrain &middot; exploration"]
+    D{"Corrected declined-cohort<br/>ECE &le; target?"}
+    E["<b>Operating frontier</b><br/>report the highest severity<br/>that still passes"]
+
+    A --> B --> C --> D
+    D -->|"yes &mdash; raise the severity"| A
+    D -->|"no &mdash; stop"| E
+```
+
+A runnable end-to-end demo (classic + custom-lever paths) is
+[`examples/quickstart.py`](examples/quickstart.py). Full mechanics, diagnostics, and the
+feedback simulation: [docs/how-it-works.md](docs/how-it-works.md).
+
+> **Scope.** CLDD is a synthetic **validation harness**, not a production pipeline: retraining
+> and feedback are seeded simulations inside the harness; it never acts on live data or real
+> lending decisions.
+
+## What's in the box
+
+Everything is importable from top-level `cldd` (full reference: the [Sphinx docs](docs/)):
+
+| Import | What it is |
+|---|---|
+| `SelectiveLabelsLoop` | the closed loop; `.run()` → `LoopResult` (frontier + per-round metrics) |
+| `Corrector` + `NaiveCorrector`, `IPWReweightCorrector`, `DisjointRetrainCorrector`, `ExplorationCorrector` | the lever ABC and the four built-ins |
+| `ReclassificationCorrector`, `AugmentationCorrector`, `FuzzyAugmentationCorrector`, `ParcellingCorrector` | four classic reject-inference methods, graded against planted truth ([honest results](docs/reject_inference.md)) |
+| `SyntheticBorrowerGenerator`, `StructuralBorrowerGenerator` | the flat and fitted-SCM synthetic worlds |
+| `run_counterfactual_eval`, `GComputationEstimator` | counterfactual validator (g-computation vs naive conditioning) |
+| `FeedbackLoop` | model-in-the-loop selective-labels simulation |
+| `empc_literature`, `emp_harness` | the two EMP variants — price the frontier in profit (reporting only, never loop control) |
+| `positivity_diagnostics` | observable regime/drift alarm — needs **no** declined-row labels |
+| `CalibratedPDClassifier` | the calibrated PD detector as a scikit-learn estimator |
+| `cldd.fidelity.run_fidelity_gate` | SCM-vs-real **marginal**-fidelity gate (univariate marginals only) |
+
+**Add a lever** by subclassing `Corrector` (`name`, `control_priority`, `apply`) and passing
+`correctors=[NaiveCorrector(), MyCorrector()]` — the legacy `improve_mode` API is unchanged
+and byte-identical. Contract details: [CONTRIBUTING.md](CONTRIBUTING.md).
+
+**Use the detector from sklearn tooling** — `CalibratedPDClassifier` is a thin, tested wrapper
+(binary-only; NaN features OK; the full `check_estimator` battery passes with zero failed
+checks on scikit-learn 1.7.2–1.9.0; probabilities byte-identical to the research API):
+
+```python
+from sklearn.model_selection import cross_val_score
+from cldd import CalibratedPDClassifier
+
+scores = cross_val_score(CalibratedPDClassifier(random_state=42), X, y, scoring="neg_brier_score")
+```
+
+## Command-line drivers
+
+Each driver runs without install (adds `src/` to the path) and writes to `artifacts/`:
+
+```bash
+python scripts/run_loop.py                    # the closed loop → frontier table + plot (--generator scm for the SCM world)
+python scripts/run_frontier_sweep.py --quick  # frontier distribution across the 25-seed set (drop --quick for all)
+python scripts/run_seed_sweep.py --quick      # counterfactual certification (drop --quick for all seeds)
+python scripts/run_reject_inference.py        # reject-inference levers vs the frontier
+python scripts/run_exploration_sweep.py       # frontier vs exploration budget
+python scripts/run_feedback.py                # model-in-the-loop feedback simulation
+python scripts/paired_significance.py         # recompute the headline stat from committed CSVs
+```
+
+## Validation
+
+`pytest` — 149 tests, all synthetic, no real data needed. CI runs a pinned-repro job (exact
+pins), a cross-version/OS compat matrix, and a strict docs build. Six float-sensitive tests
+reproduce only under the pins in `requirements-dev.txt`; the optional marginal-fidelity gate
+compares the SCM against a **private** real dataset via `CLDD_DATA_DIR` and is the only thing
+that needs it. Details, reproducibility, and troubleshooting:
+[docs/validation.md](docs/validation.md).
+
+## Documentation
+
+| Where | What |
+|---|---|
+| [docs/quickstart.md](docs/quickstart.md) | run the loop, the counterfactual eval, the fidelity report |
+| [docs/how-it-works.md](docs/how-it-works.md) | loop mechanics, diagnostics, feedback simulation, repo map |
+| [docs/configuration.md](docs/configuration.md) | every knob (`config.py`) and the one env var |
+| [docs/validation.md](docs/validation.md) | tests, gates, reproducibility, troubleshooting |
+| [docs/reject_inference.md](docs/reject_inference.md) | the four RI methods and their honest (modest) results |
+| [`docs/assessment.md`](docs/assessment.md) | **the accompanying article** — independent results & methodology assessment (dated snapshot) |
+
+Build locally: `pip install -e ".[docs]" && sphinx-build -b html -W docs docs/_build/html`.
+
+## Status
+
+`0.2.0` **alpha** on [PyPI](https://pypi.org/project/closed-loop-default-detection/),
+changelog in [CHANGELOG.md](CHANGELOG.md).
+
+Shipped in 0.1.0: the loop, both synthetic worlds, all correction levers, the fidelity gate,
+the sklearn estimator, CI on three gates. Added in 0.2.0: `cldd.emp` (both EMP variants),
+priced exploration, the EMP-optimal cutoff, and the 25-seed frontier sweep.
+
+CLDD began as a validation harness for the Intuit TechWeek SMB Underwriting Challenge; it is
+not a submission and does not alter challenge files.
+
+## Citation
+
+Metadata in [`CITATION.cff`](CITATION.cff) (GitHub's "Cite this repository" reads it):
+
+```bibtex
+@software{pazooki_cldd_2026,
+  author  = {Pazooki, Hossain},
+  title   = {{closed-loop-default-detection}: measuring selective-labels default
+             detection and the PD model's operating frontier},
+  year    = {2026},
+  version = {0.2.0},
+  license = {MIT},
+  url     = {https://github.com/hossainpazooki/closed-loop-default-detection}
+}
+```

{closed_loop_default_detection-0.1.0 → closed_loop_default_detection-0.3.0}/pyproject.toml

@@ -4,13 +4,13 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "closed-loop-default-detection"
-version = "0.1.0"
-description = "CLUE-style closed loop that measures selective-labels default detection on synthetic SMB lending cohorts and finds the PD model's operating frontier."
+version = "0.3.0"
+description = "Closed loop that measures selective-labels default detection on synthetic SMB lending cohorts and finds the PD model's operating frontier."
 readme = "README.md"
 requires-python = ">=3.10"
 license = "MIT"
 license-files = ["LICENSE"]
-authors = [{ name = "Hossain Pazooki", email = "hossain@pazooki.com" }]
+authors = [{ name = "Hossain Pazooki" }]
 keywords = [
     "selective-labels",
     "probability-of-default",

{closed_loop_default_detection-0.1.0 → closed_loop_default_detection-0.3.0}/src/cldd/__init__.py

@@ -1,7 +1,7 @@
-"""Closed-loop default-rate detection — a CLUE-style harness for selective labels.
+"""Closed-loop default-rate detection — a selective-labels validation harness.
 
 generate (``synthetic``) -> measure (``eval_default``) -> improve/frontier
-(``loop``). See README.md for the CLUE mapping and how to run.
+(``loop``). See README.md for how to run.
 """
 
 from __future__ import annotations
@@ -11,7 +11,7 @@ from importlib.metadata import PackageNotFoundError, version as _pkg_version
 try:  # pyproject `version` is the single source of truth (read from installed metadata)
     __version__ = _pkg_version("closed-loop-default-detection")
 except PackageNotFoundError:  # running from a source tree with no install
-    __version__ = "0.1.0"
+    __version__ = "0.2.0"
 
 from .correctors import (
     CorrectionOutcome,
@@ -36,6 +36,13 @@ from .reject_inference import (
     ReclassificationCorrector,
     RejectInferenceCorrector,
 )
+from .emp import (
+    EMPCParams,
+    EMPResult,
+    LoanEconomics,
+    emp_harness,
+    empc_literature,
+)
 from .eval_default import PdDetectionResult, fit_observed_model, score_pd_detection
 from .model_pd import CalibratedPDClassifier, CalibratedPDModel
 from .feedback import FeedbackLoop, FeedbackResult, GenerationResult
@@ -87,6 +94,11 @@ __all__ = [
     "PdDetectionResult",
     "fit_observed_model",
     "score_pd_detection",
+    "EMPCParams",
+    "EMPResult",
+    "LoanEconomics",
+    "empc_literature",
+    "emp_harness",
     "CalibratedPDModel",
     "CalibratedPDClassifier",
     "CounterfactualResult",

{closed_loop_default_detection-0.1.0 → closed_loop_default_detection-0.3.0}/src/cldd/config.py

@@ -24,8 +24,7 @@ RANDOM_SEED = 42
 
 # Disjoint train-cohort offset for the retrain lever. Large enough that train
 # seeds (RANDOM_SEED + TRAIN_SEED_OFFSET + iteration) never collide with measure
-# seeds (RANDOM_SEED + iteration). This is the no-leakage discipline ported from
-# upstream-label-correction/clue/loop.py.
+# seeds (RANDOM_SEED + iteration).
 TRAIN_SEED_OFFSET = 1000
 
 # --------------------------------------------------------------------------- #
@@ -59,6 +58,19 @@ TARGET_DECLINED_ECE = 0.10
 
 DEFAULT_N_APPLICANTS = 4000
 
+# --------------------------------------------------------------------------- #
+# EMP reporting layer (cldd.emp) — v2 measurement axis, never loop control
+# --------------------------------------------------------------------------- #
+# Literature EMPC prior. Verified against the primary source before entry
+# (2026-07-13 spike): Verbraken, Bravo, Weber & Baesens (2014), EJOR
+# 238(2):505-513 — h(lambda) = p0*delta(0) + p1*delta(1) + (1-p0-p1)*U(0,1),
+# constant ROI, computed over the ROC convex hull (paper Eqs. 13/15). The
+# defaults are the benchmark-standard CRAN EMP::empCreditScoring values, so
+# empc columns are comparable to the published literature.
+EMPC_P0 = 0.55       # P(lambda = 0): full recovery
+EMPC_P1 = 0.10       # P(lambda = 1): total loss
+EMPC_ROI = 0.2644    # per-loan return on investment, fraction of principal
+
 # --------------------------------------------------------------------------- #
 # Observable positivity-diagnostic thresholds (cldd.diagnostics)
 # --------------------------------------------------------------------------- #