deup 0.1.1__tar.gz → 0.3.0__tar.gz

{deup-0.1.1 → deup-0.3.0}/CHANGELOG.md

@@ -1,5 +1,37 @@
 # Changelog
 
+## [Unreleased]
+
+## [0.3.0] — 2026-06-05
+
+### Added
+
+- **Reliability diagnostics** (`deup.diagnostics`): `AggregationReliability` /
+  `should_trust_aggregate` (Finding 1) and pluggable `HealthIndex` (Finding 2).
+- **Domain presets** (`deup.domains`):
+  - `CrossSectionalDEUP` — finance flagship: `PurgedWalkForward`, rank
+    residualization, vol/breadth/regime g-features, `HealthIndex`, multi-horizon
+    targets, panel DataFrame API.
+  - `TabularDEUP` — KFold + raw X + Mahalanobis density.
+  - `VisionDEUP` — embedding → density + variance for OOD classification.
+- Docs: `reliability.md`, `domains.md`, API pages.
+
+### Changed
+
+- PyPI release bundles P5–P10 features (feature builders through domain presets).
+
+## [0.2.0] — 2026-06-05
+
+### Added
+
+- **`DEUPClassifier`** — classification with log-loss / Brier OOF errors + `predict_proba`
+- **`DEUPRanker`** — cross-sectional ranking; `loss="rank"`, `PurgedWalkForward` default,
+  rank-geometry residualization ON by default (Finding 3)
+- **`acquire(pool, k)`** — active-learning hook (top-k by epistemic uncertainty)
+- Refactored **`DEUPRegressor`** onto `ErrorEstimator` + optional `features` /
+  `aleatoric` / `decompose`
+- Docs updated for all three estimators and `acquire`
+
 ## [0.1.1] — 2026-06-04
 
 First release published to PyPI.

{deup-0.1.1 → deup-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: deup
-Version: 0.1.1
+Version: 0.3.0
 Summary: Direct Epistemic Uncertainty Prediction (DEUP) for any scikit-learn model, with first-class time-series support.
 Project-URL: Homepage, https://github.com/ursinasanderink/deup
 Project-URL: Repository, https://github.com/ursinasanderink/deup
@@ -67,12 +67,13 @@ model.fit(X_train, y_train)
 pred, unc = model.predict(X_test, return_uncertainty=True)
 ```
 
-For time-series / cross-sectional data, pass a leakage-safe splitter:
+For time-series / cross-sectional finance panels:
 
 ```python
-from deup.splitters import PurgedWalkForward
+from deup.domains.finance import CrossSectionalDEUP
 
-model = DEUPRegressor(base_model=my_model, cv=PurgedWalkForward(embargo=5))
+model = CrossSectionalDEUP(horizon=20).fit(panel_df)
+pred, unc = model.predict(test_df, return_uncertainty=True)
 ```
 
 ## Install
@@ -80,6 +81,7 @@ model = DEUPRegressor(base_model=my_model, cv=PurgedWalkForward(embargo=5))
 ```bash
 pip install deup            # core (numpy + scikit-learn)
 pip install "deup[gbm]"     # + LightGBM error predictor
+pip install "deup[finance]" # + pandas (CrossSectionalDEUP)
 pip install "deup[docs]"    # + MkDocs site locally
 ```
 
@@ -96,11 +98,10 @@ than ensemble disagreement or a conformal residual baseline — see [BENCHMARKS.
 
 ## Status / roadmap
 
-**v0.1 (released):** `DEUPRegressor`, OOF collector, splitters, full loss registry
-(squared / Brier / pinball / rank), target transforms (log / asinh), benchmark, docs.
+**v0.3 (current):** everything in v0.2 plus aggregation-reliability diagnostics
+(Findings 1–2), domain presets (`CrossSectionalDEUP`, `TabularDEUP`, `VisionDEUP`).
 
-**v0.2:** `DEUPClassifier` / `DEUPRanker`, conformal intervals, aleatoric decomposition,
-density/GP features, aggregation-reliability diagnostics.
+**Next:** thesis parity migration (P11), full benchmark suite with N-sweep (P12).
 
 ## Citing
 

{deup-0.1.1 → deup-0.3.0}/README.md

@@ -26,12 +26,13 @@ model.fit(X_train, y_train)
 pred, unc = model.predict(X_test, return_uncertainty=True)
 ```
 
-For time-series / cross-sectional data, pass a leakage-safe splitter:
+For time-series / cross-sectional finance panels:
 
 ```python
-from deup.splitters import PurgedWalkForward
+from deup.domains.finance import CrossSectionalDEUP
 
-model = DEUPRegressor(base_model=my_model, cv=PurgedWalkForward(embargo=5))
+model = CrossSectionalDEUP(horizon=20).fit(panel_df)
+pred, unc = model.predict(test_df, return_uncertainty=True)
 ```
 
 ## Install
@@ -39,6 +40,7 @@ model = DEUPRegressor(base_model=my_model, cv=PurgedWalkForward(embargo=5))
 ```bash
 pip install deup            # core (numpy + scikit-learn)
 pip install "deup[gbm]"     # + LightGBM error predictor
+pip install "deup[finance]" # + pandas (CrossSectionalDEUP)
 pip install "deup[docs]"    # + MkDocs site locally
 ```
 
@@ -55,11 +57,10 @@ than ensemble disagreement or a conformal residual baseline — see [BENCHMARKS.
 
 ## Status / roadmap
 
-**v0.1 (released):** `DEUPRegressor`, OOF collector, splitters, full loss registry
-(squared / Brier / pinball / rank), target transforms (log / asinh), benchmark, docs.
+**v0.3 (current):** everything in v0.2 plus aggregation-reliability diagnostics
+(Findings 1–2), domain presets (`CrossSectionalDEUP`, `TabularDEUP`, `VisionDEUP`).
 
-**v0.2:** `DEUPClassifier` / `DEUPRanker`, conformal intervals, aleatoric decomposition,
-density/GP features, aggregation-reliability diagnostics.
+**Next:** thesis parity migration (P11), full benchmark suite with N-sweep (P12).
 
 ## Citing
 

deup-0.3.0/docs/api/calibration.md

@@ -0,0 +1,7 @@
+# API: Calibration
+
+::: deup.calibration.conformal.UncertaintyCalibrator
+
+::: deup.calibration.conformal.ConformalResult
+
+::: deup.calibration.conformal.deup_normalizer

deup-0.3.0/docs/api/decomposition.md

@@ -0,0 +1,17 @@
+::: deup.core.error_estimator.ErrorEstimator
+
+::: deup.core.aleatoric.Homoscedastic
+
+::: deup.core.aleatoric.Heteroscedastic
+
+::: deup.core.aleatoric.Quantile
+
+::: deup.core.decompose.decompose_epistemic
+
+::: deup.core.decompose.RankResidualizer
+
+::: deup.core.decompose.coupling_retention_report
+
+::: deup.core.decompose.density_kill_criterion
+
+::: deup.core.decompose.partial_correlation

deup-0.3.0/docs/api/diagnostics.md

@@ -0,0 +1,23 @@
+# API: Diagnostics
+
+## Aggregation reliability
+
+::: deup.diagnostics.aggregation.AggregationReliability
+
+::: deup.diagnostics.aggregation.AggregationVerdict
+
+::: deup.diagnostics.aggregation.effective_sample_size
+
+::: deup.diagnostics.aggregation.should_trust_aggregate
+
+## Composite health index
+
+::: deup.diagnostics.health.HealthIndex
+
+::: deup.diagnostics.health.HealthReport
+
+::: deup.diagnostics.health.realized_efficacy
+
+::: deup.diagnostics.health.drift_psi
+
+::: deup.diagnostics.health.model_disagreement

deup-0.3.0/docs/api/domains.md

@@ -0,0 +1,23 @@
+# API: Domain presets
+
+## Finance
+
+::: deup.domains.finance.CrossSectionalDEUP
+
+::: deup.domains.finance.enrich_panel
+
+::: deup.domains.finance.FINANCE_G_FEATURES
+
+## Tabular
+
+::: deup.domains.tabular.TabularDEUP
+
+::: deup.domains.tabular.tabular_feature_pipeline
+
+## Vision
+
+::: deup.domains.vision.VisionDEUP
+
+::: deup.domains.vision.EmbeddingUncertaintyFeatures
+
+::: deup.domains.vision.IdentityEmbedding

deup-0.3.0/docs/api/estimators.md

@@ -0,0 +1,7 @@
+# API: Estimators
+
+::: deup.estimators.DEUPRegressor
+
+::: deup.estimators.DEUPClassifier
+
+::: deup.estimators.DEUPRanker

deup-0.3.0/docs/api/features.md

@@ -0,0 +1,13 @@
+::: deup.core.features.FeaturePipeline
+
+::: deup.core.features.RawFeatures
+
+::: deup.core.features.DensityFeature
+
+::: deup.core.features.VarianceFeature
+
+::: deup.core.features.DistanceToTrain
+
+::: deup.core.features.SeenBit
+
+::: deup.core.features.ResidualMagnitude

deup-0.3.0/docs/calibration.md

@@ -0,0 +1,89 @@
+# Conformal calibration
+
+DEUP's `predict_epistemic` returns an *uncalibrated* score: higher means "less
+trustworthy", but not a probability. **Split-conformal calibration** turns it into
+prediction intervals with finite-sample, distribution-free marginal coverage
+$P(y \in [\hat{y}^-, \hat{y}^+]) \ge 1 - \alpha$ — using the DEUP signal as the
+interval's *width*.
+
+## How it works
+
+On a **held-out** calibration set, compute normalized residuals
+$r_i = |y_i - f(x_i)| / g(x_i)$ and take their $(1-\alpha)$ empirical quantile $q$.
+The interval at a new point is
+
+$$
+[\,f(x) - q\,g(x),\;\; f(x) + q\,g(x)\,].
+$$
+
+Intervals are **narrow where $g$ is small** (confident) and wide where $g$ is large —
+locally adaptive coverage, unlike a constant-width baseline.
+
+## Usage
+
+```python
+from deup import DEUPRegressor
+
+model = DEUPRegressor(base_model=my_model).fit(X_train, y_train)
+
+# calibrate on a separate held-out split (NOT the training data)
+model.calibrate(X_cal, y_cal, method="normalized", alpha=0.1)
+
+interval = model.predict_interval(X_test)
+interval.lower, interval.upper, interval.width
+```
+
+!!! warning "Use held-out data"
+    Coverage guarantees require the calibration set to be unseen by both the base model
+    $f$ and the error model $g$. Don't calibrate on training rows.
+
+## Methods
+
+| `method` | Score | Use when |
+|---|---|---|
+| `normalized` (default) | $\lvert y-f(x)\rvert / g(x)$ | locally adaptive intervals |
+| `mondrian` | per-group quantile | group/regime-conditional coverage |
+| `cqr` | conformalized quantile regression | you already have quantile models |
+
+```python
+# Mondrian: group-conditional coverage (e.g. per regime)
+model.calibrate(X_cal, y_cal, method="mondrian", alpha=0.1, groups=regime_cal)
+interval = model.predict_interval(X_test, groups=regime_test)
+```
+
+The standalone `UncertaintyCalibrator` works with raw arrays (any base model):
+
+```python
+from deup.calibration import UncertaintyCalibrator
+
+cal = UncertaintyCalibrator(method="normalized", alpha=0.1)
+cal.fit(y_cal, y_pred_cal, uncertainty_cal)
+interval = cal.predict_interval(y_pred_test, uncertainty_test)
+```
+
+## MAPIE interop
+
+`deup` is **complementary** to [MAPIE](https://mapie.readthedocs.io/): MAPIE supplies
+mature conformal machinery, DEUP supplies a high-quality per-point scale $g(x)$. Expose
+the DEUP scale as a normalizer:
+
+```python
+from deup.calibration import deup_normalizer
+
+normalizer = deup_normalizer(model)   # .predict(X) == model.predict_epistemic(X)
+scale = normalizer.predict(X_cal)     # feed into MAPIE as a residual scale
+```
+
+See [`examples/mapie_interop.py`](https://github.com/ursinasanderink/deup/blob/main/examples/mapie_interop.py)
+for a runnable script.
+
+## Coverage guarantee
+
+Split conformal gives the finite-sample bound (Lei et al., 2018)
+
+$$
+1 - \alpha \;\le\; P(y \in \hat{C}(x)) \;\le\; 1 - \alpha + \tfrac{1}{n_{\text{cal}}+1},
+$$
+
+so intervals may *slightly over-cover*; this is correct, not a bug. `deup`'s test suite
+checks empirical coverage within tolerance on i.i.d. and purged time-split fixtures.

deup-0.3.0/docs/decomposition.md

@@ -0,0 +1,98 @@
+# Decomposition & rank residualization
+
+This page covers the v0.2 components that turn the raw error estimate $g(x)$ into a
+reported epistemic signal: the error estimator, aleatoric estimators, the
+$\hat{e} = \max(0, g - a)$ decomposition, and cross-sectional rank-geometry
+residualization. See [Theory](theory.md) for the underlying math.
+
+## ErrorEstimator
+
+`ErrorEstimator` is the reusable DEUP error model $g$ — feature pipeline +
+target transform + non-negativity, fit on out-of-fold errors.
+
+```python
+from deup.core import ErrorEstimator
+from deup.core.features import DensityFeature, FeaturePipeline, RawFeatures
+from deup.core.oof import OOFErrorCollector
+from sklearn.ensemble import RandomForestRegressor
+from sklearn.model_selection import KFold
+
+oof = OOFErrorCollector(
+    RandomForestRegressor(), cv=KFold(5), loss="squared"
+).fit_collect(X, y)
+
+g = ErrorEstimator(
+    features=FeaturePipeline([("raw", RawFeatures()), ("density", DensityFeature())]),
+    target_transform="log",
+).fit(X[oof.indices], oof.errors)
+
+error_estimate = g.predict(X_new)   # >= 0
+```
+
+## Aleatoric estimators $a(x)$
+
+Model-agnostic estimates of the irreducible noise floor $A(x) = \mathrm{Var}(Y\mid X=x)$
+(variance scale, matching a squared-error target).
+
+| Estimator | $a(x)$ | When |
+|---|---|---|
+| `Homoscedastic` | constant $\sigma^2$ | noise ~ constant across $\mathcal{X}$ |
+| `Heteroscedastic` | local k-NN label variance | input-dependent noise |
+| `Quantile` | $((q_{hi}-q_{lo})/z)^2$ from quantile regression | skewed / tail noise |
+
+```python
+from deup.core import Heteroscedastic
+
+a = Heteroscedastic(k=20).fit(X, y).predict(X_new)
+```
+
+## Decomposition
+
+```python
+from deup.core import decompose_epistemic
+
+e_hat = decompose_epistemic(error_estimate, a)   # max(0, g - a)
+# a=None -> conservative proxy e_hat = g (the v0.1 default)
+```
+
+$\hat{e}$ is always non-negative.
+
+## Rank-geometry residualization (Finding 3)
+
+For cross-sectional rankers, $g$ and the loss target can be partly **mechanical rank
+geometry** rather than genuine error. `RankResidualizer` fits an isotonic map from the
+within-group rank of $|score|$ to the signal and subtracts it, leaving the part *not*
+explained by rank geometry.
+
+```python
+from deup.core import RankResidualizer, coupling_retention_report
+
+# decouple g from rank geometry, per date
+res = RankResidualizer().fit(g_values, abs_score, groups=dates)
+g_decoupled = res.transform(g_values, abs_score, groups=dates)
+
+# diagnostics: coupling before/after + loss-association retention
+report = coupling_retention_report(g_values, score, loss, groups=dates)
+print(report.coupling_before, report.coupling_after, report.retention)
+```
+
+!!! note "Thesis finding"
+    Residualization decoupled the signal (per-date $\rho(\hat{e}, |score|)$:
+    $0.616 \to 0.317$) while **retaining ~92.5%** of the loss association. This is
+    **off by default** and **on in `DEUPRanker`** (P7).
+
+## Density kill criterion (Finding 3 corollary)
+
+Density features can be an **informative null** in homogeneous universes. The kill
+criterion drops them when their gain importance is negligible **and** they barely move
+the loss partial-correlation.
+
+```python
+from deup.core import density_kill_criterion
+
+decision = density_kill_criterion(gain_importance=1e-5, delta_partial_corr=0.001)
+print(decision.keep, decision.reason)   # False, "killed: ..."
+```
+
+Use `partial_correlation(a, b, control)` to compute the $\Delta$ partial-correlation
+with vs without the density feature.

deup-0.3.0/docs/domains.md

@@ -0,0 +1,57 @@
+# Domain presets
+
+The core library is domain-agnostic; these modules are **thin presets** that wire the
+right splitter, features, and diagnostics for common workflows. They do not duplicate
+OOF collection or error-estimator logic — see ``ARCHITECTURE.md``.
+
+## Cross-sectional finance (flagship)
+
+```python
+import pandas as pd
+from deup.domains.finance import CrossSectionalDEUP
+
+# long-format panel: one row per (date, asset)
+panel = pd.read_parquet("signals.parquet")  # columns: date, score, vol_20d, ...
+
+model = CrossSectionalDEUP(horizon=20, cv=5, embargo=1).fit(panel)
+model.calibrate(cal_panel, alpha=0.1)
+
+pred, unc = model.predict(test_panel, return_uncertainty=True)
+health = model.health_report(test_panel)   # per-date context gating (Finding 2)
+health.gate                                # bool per date: trust / trade?
+```
+
+Defaults wired in:
+
+| Setting | Value |
+|---|---|
+| Estimator | :class:`~deup.estimators.DEUPRanker` |
+| CV | :class:`~deup.splitters.PurgedWalkForward` + embargo |
+| Rank geometry | residualization **ON** (Finding 3) |
+| g-features | vol / breadth / regime preset columns when present |
+| Context health | :class:`~deup.diagnostics.HealthIndex` |
+
+Requires ``pip install "deup[finance]"`` (pandas).
+
+## Generic tabular
+
+```python
+from deup.domains.tabular import TabularDEUP
+
+model = TabularDEUP(task="regression", cv=5).fit(X, y)
+unc = model.predict_epistemic(X_test)
+```
+
+Wires ``KFold`` + raw ``X`` + Mahalanobis density features for ``g``.
+
+## Vision / OOD classification
+
+```python
+from deup.domains.vision import VisionDEUP
+
+model = VisionDEUP(cv=5).fit(images, labels)   # tensors OK — auto-flattened
+unc = model.predict_epistemic(images)
+```
+
+Wires embedding → density + variance features for ``g`` (CIFAR-style high-N path).
+Pass a custom ``embedding=`` transformer or callable for CNN embeddings.

deup-0.3.0/docs/features.md

@@ -0,0 +1,119 @@
+# Feature builders for $g(x)$
+
+The error predictor $g$ in DEUP can use **stationarizing features**
+$\phi_{z^N}(x)$ beyond raw inputs (Lahlou *et al.*, 2023, Sec. 3.2). Each builder
+is a scikit-learn `TransformerMixin` that **fits on training data only** — the same
+leakage discipline as `OOFErrorCollector` (Finding 4).
+
+See [Theory](theory.md) for the mathematical definitions.
+
+## Quick example
+
+```python
+import numpy as np
+from sklearn.ensemble import RandomForestRegressor
+
+from deup.core.features import (
+    DensityFeature,
+    DistanceToTrain,
+    FeaturePipeline,
+    RawFeatures,
+    SeenBit,
+)
+
+pipe = FeaturePipeline([
+    ("raw", RawFeatures()),
+    ("density", DensityFeature(method="mahalanobis")),
+    ("dist", DistanceToTrain(k=5)),
+    ("seen", SeenBit(atol=1e-8)),
+])
+
+X_train = np.random.default_rng(0).normal(size=(500, 8))
+X_test = np.random.default_rng(1).normal(size=(50, 8))
+
+phi_train = pipe.fit_transform(X_train)
+phi_test = pipe.transform(X_test)
+print(phi_train.shape, phi_test.shape)  # (500, 8+1+1+1), (50, ...)
+```
+
+## Builders
+
+| Class | Output | Methods / notes |
+|---|---|---|
+| `RawFeatures` | $x$ | passthrough |
+| `DensityFeature` | $\log \hat{q}(x)$ column | `mahalanobis`, `knn`, `kde`; `flow` requires `[torch]` |
+| `VarianceFeature` | $\log \hat{V}(x)$ column | `ensemble` (bootstrap); `gp` requires `[torch]` |
+| `DistanceToTrain` | $k$-th NN distance | default `k=5` |
+| `SeenBit` | $s \in \{0,1\}$ | exact / `atol` duplicate detection |
+| `ResidualMagnitude` | kNN-smoothed $\|y-f(x)\|$ | needs `estimator` + `y` at `fit` |
+
+### DensityFeature
+
+```python
+# Diagonal Gaussian — matches thesis GaussianDensity.log_prob (Lee et al. 2018)
+DensityFeature(method="mahalanobis")
+
+# k-NN distance proxy: log q ≈ -log(d_k + ε)
+DensityFeature(method="knn", k=5)
+
+# sklearn KernelDensity
+DensityFeature(method="kde", bandwidth=1.0)
+```
+
+!!! warning "Finding 3"
+    Density can be **informative null** in homogeneous tabular panels. Ablate with
+    `FeaturePipeline` column importances or drop if $\Delta\rho < 0.005$.
+
+### VarianceFeature (ensemble)
+
+Fits `n_estimators` bootstrap replicas of a base model and returns
+$\log(\mathrm{Var}_j f_j(x) + \varepsilon)$.
+
+```python
+VarianceFeature(
+    method="ensemble",
+    estimator=RandomForestRegressor(n_estimators=50, random_state=0),
+    n_estimators=10,
+)
+```
+
+### ResidualMagnitude
+
+At `fit(X, y)` stores training residuals $|y - f(x)|$. At `transform(X)` returns
+the mean residual magnitude among $k$ nearest training neighbors — a local error prior
+when $y$ is unavailable at inference.
+
+```python
+ResidualMagnitude(
+    estimator=RandomForestRegressor(),
+    k=5,
+).fit(X_train, y_train)
+```
+
+## FeaturePipeline
+
+`FeaturePipeline` horizontally stacks named builders (FeatureUnion-style). Names appear
+in `get_feature_names_out()`.
+
+```python
+from deup.core.features import FeaturePipeline, VarianceFeature, SeenBit
+
+pipe = FeaturePipeline([
+    ("var", VarianceFeature(method="ensemble")),
+    ("seen", SeenBit()),
+])
+```
+
+## Torch-dependent methods
+
+`DensityFeature(method="flow")` and `VarianceFeature(method="gp")` require
+`pip install "deup[torch]"`. Without torch, construction raises `ImportError` with an
+install hint; the module still imports cleanly on a torch-free install.
+
+## v0.1 vs v0.2
+
+**v0.1 (this release):** feature builders + pipeline are available as primitives.
+`DEUPRegressor` still trains $g$ on raw $X$ by default.
+
+**v0.2 (P6):** `ErrorEstimator` wires `FeaturePipeline` into the DEUP training loop
+with target transforms and non-negativity clipping.