causalscreen 0.2.0__tar.gz

causalscreen-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Rahul Kumar Mandal
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

causalscreen-0.2.0/MANIFEST.in

@@ -0,0 +1,3 @@
+recursive-include tests *.py
+include LICENSE
+recursive-include examples *.py

causalscreen-0.2.0/PKG-INFO

@@ -0,0 +1,127 @@
+Metadata-Version: 2.4
+Name: causalscreen
+Version: 0.2.0
+Summary: Residual-correlation causal screening with iterative peeling, localized modeling, and drift early-warning
+Author-email: Rahul Kumar Mandal <rahulkm.jobs@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/rahulkm3/causalscreen
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy
+Requires-Dist: scipy
+Requires-Dist: scikit-learn
+Dynamic: license-file
+
+# causalscreen
+
+Residual-correlation causal screening with iterative peeling, causal-factor
+partitioning with inference-time local models, and a missing-causal-power
+drift alarm.
+
+Companion code for: **R. K. Mandal, "Residual-Correlation Causal Screening with
+Localized Supervised Modeling: A Practical Framework for Confounder
+Identification, Error Reduction, and Drift Early-Warning"** (working paper,
+2026; research 2022–present). Paper PDF in `/paper` · Preprint: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7135078
+
+## What it does
+
+Supervised models fail in deployment because they lean on **confounded
+proxies** — features whose correlation with the target is inherited from other
+variables. `causalscreen`:
+
+1. **Screens** every candidate feature via the two-regression residual
+   construction (Frisch–Waugh–Lovell partial correlation): regress `y` on the
+   other features, regress `x_j` on the other features, correlate the
+   residuals. High residual correlation = genuine independent effect.
+2. **Peels** iteratively: each discovered factor's contribution is stripped
+   out of the response before the next round, so its influence cannot re-enter
+   the screen through the variables it confounds.
+3. **Ranks** features by causal power and reports the **missing causal
+   power** — the share of response variance the discovered factors cannot
+   explain — a deployable early-warning statistic for data/model drift.
+4. **Exploits** the ranking: `PartitionedRegressor` partitions data on top
+   causal factors and trains one small local model per query at inference
+   time (interpolation), falling back to the global model out of support
+   (extrapolation) — so the error reduction of local modeling never costs
+   out-of-sample generalization.
+
+## Quick result (synthetic ground truth, `examples/demo.py`)
+
+| feature | naive \|corr\| with y | causal screen verdict |
+|---|---|---|
+| x1 (true cause) | 0.92 | **ranked** (power 0.78) |
+| x2 (confounded proxy of x1) | 0.87 | **rejected** |
+| x3 (noise) | 0.01 | rejected |
+| x4 (true cause, weak) | 0.33 | **ranked** (power 0.83) |
+
+Missing causal power: 5.1% — the exact noise floor of the generating process.
+Partitioned local modeling: **99.0% out-of-sample MSE reduction** vs. a global
+model on regime-structured data.
+
+## Install & use
+
+```bash
+pip install -e .
+python examples/demo.py
+pytest tests/
+```
+
+```python
+from causalscreen import CausalScreen, PartitionedRegressor
+res = CausalScreen(alpha=0.05).fit(X, y, feature_names)
+res.ranking            # features in decreasing causal power
+res.causal_power       # partial correlation at selection round
+res.missing_causal_power  # drift-alarm statistic
+
+pr = PartitionedRegressor(partition_features=res.ranking[:2], min_cell=50)
+pr.fit(X, y, feature_names)
+pr.predict(X_new)      # local model in-support, global model out-of-support
+```
+
+## Assumptions (stated plainly)
+
+Stationary cross-sectional data · causal sufficiency (all relevant variables
+observed) · no reverse causation · features measured before the response.
+The screen addresses the *confounding* component of causal analysis; it does
+not prove causality in full generality. See paper §3, §7.
+
+## Deployed results
+
+Layered over Random Forest, OLS and Prophet models in commercial deployments
+(credit risk, segmentation, forecasting), the framework reduced out-of-sample
+error by 65–85%. Client data is not releasable; this repo's synthetic
+benchmarks are the reproducible counterpart. DoWhy/EconML comparison suite:
+in progress.
+
+## Performance (v0.2.0)
+
+Screening uses a single precision-matrix inversion per round (FWL-equivalent,
+with an exact residual-regression fallback for ill-conditioned designs) —
+~77x faster than the reference implementation at n=5000, p=40. Partitioned
+prediction indexes cells at fit time and caches local models — ~409x faster
+on batch prediction (20k train / 5k queries). Equivalence to the reference
+implementation is enforced in the test suite to ~1e-8 (38 tests; reference
+implementation vendored in `tests/`).
+
+## License
+
+MIT
+
+## Pipeline
+
+```mermaid
+flowchart LR
+    A[Training data] --> B["CausalScreen<br/>residual-correlation test<br/>+ iterative peeling"]
+    B -->|"ranked causal factors<br/>(decreasing causal power)"| C["Partition data on<br/>top causal factors<br/>(strength x significance rule)"]
+    C --> G1[(Subgroup 1)]
+    C --> G2[(Subgroup 2)]
+    C --> Gn[(Subgroup n)]
+    Q[Query point] --> R{inside a<br/>subgroup?}
+    R -->|"yes (interpolation)"| L["Train ONE local model<br/>in that subgroup only<br/>(Optimisation 2)"]
+    R -->|"no (extrapolation)"| F["Global model fallback<br/>(retains significance &<br/>generalization)"]
+    G1 -.-> L
+    G2 -.-> L
+    Gn -.-> L
+    B --> M["Missing causal power<br/>= drift early-warning alarm"]
+```

causalscreen-0.2.0/README.md

@@ -0,0 +1,112 @@
+# causalscreen
+
+Residual-correlation causal screening with iterative peeling, causal-factor
+partitioning with inference-time local models, and a missing-causal-power
+drift alarm.
+
+Companion code for: **R. K. Mandal, "Residual-Correlation Causal Screening with
+Localized Supervised Modeling: A Practical Framework for Confounder
+Identification, Error Reduction, and Drift Early-Warning"** (working paper,
+2026; research 2022–present). Paper PDF in `/paper` · Preprint: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7135078
+
+## What it does
+
+Supervised models fail in deployment because they lean on **confounded
+proxies** — features whose correlation with the target is inherited from other
+variables. `causalscreen`:
+
+1. **Screens** every candidate feature via the two-regression residual
+   construction (Frisch–Waugh–Lovell partial correlation): regress `y` on the
+   other features, regress `x_j` on the other features, correlate the
+   residuals. High residual correlation = genuine independent effect.
+2. **Peels** iteratively: each discovered factor's contribution is stripped
+   out of the response before the next round, so its influence cannot re-enter
+   the screen through the variables it confounds.
+3. **Ranks** features by causal power and reports the **missing causal
+   power** — the share of response variance the discovered factors cannot
+   explain — a deployable early-warning statistic for data/model drift.
+4. **Exploits** the ranking: `PartitionedRegressor` partitions data on top
+   causal factors and trains one small local model per query at inference
+   time (interpolation), falling back to the global model out of support
+   (extrapolation) — so the error reduction of local modeling never costs
+   out-of-sample generalization.
+
+## Quick result (synthetic ground truth, `examples/demo.py`)
+
+| feature | naive \|corr\| with y | causal screen verdict |
+|---|---|---|
+| x1 (true cause) | 0.92 | **ranked** (power 0.78) |
+| x2 (confounded proxy of x1) | 0.87 | **rejected** |
+| x3 (noise) | 0.01 | rejected |
+| x4 (true cause, weak) | 0.33 | **ranked** (power 0.83) |
+
+Missing causal power: 5.1% — the exact noise floor of the generating process.
+Partitioned local modeling: **99.0% out-of-sample MSE reduction** vs. a global
+model on regime-structured data.
+
+## Install & use
+
+```bash
+pip install -e .
+python examples/demo.py
+pytest tests/
+```
+
+```python
+from causalscreen import CausalScreen, PartitionedRegressor
+res = CausalScreen(alpha=0.05).fit(X, y, feature_names)
+res.ranking            # features in decreasing causal power
+res.causal_power       # partial correlation at selection round
+res.missing_causal_power  # drift-alarm statistic
+
+pr = PartitionedRegressor(partition_features=res.ranking[:2], min_cell=50)
+pr.fit(X, y, feature_names)
+pr.predict(X_new)      # local model in-support, global model out-of-support
+```
+
+## Assumptions (stated plainly)
+
+Stationary cross-sectional data · causal sufficiency (all relevant variables
+observed) · no reverse causation · features measured before the response.
+The screen addresses the *confounding* component of causal analysis; it does
+not prove causality in full generality. See paper §3, §7.
+
+## Deployed results
+
+Layered over Random Forest, OLS and Prophet models in commercial deployments
+(credit risk, segmentation, forecasting), the framework reduced out-of-sample
+error by 65–85%. Client data is not releasable; this repo's synthetic
+benchmarks are the reproducible counterpart. DoWhy/EconML comparison suite:
+in progress.
+
+## Performance (v0.2.0)
+
+Screening uses a single precision-matrix inversion per round (FWL-equivalent,
+with an exact residual-regression fallback for ill-conditioned designs) —
+~77x faster than the reference implementation at n=5000, p=40. Partitioned
+prediction indexes cells at fit time and caches local models — ~409x faster
+on batch prediction (20k train / 5k queries). Equivalence to the reference
+implementation is enforced in the test suite to ~1e-8 (38 tests; reference
+implementation vendored in `tests/`).
+
+## License
+
+MIT
+
+## Pipeline
+
+```mermaid
+flowchart LR
+    A[Training data] --> B["CausalScreen<br/>residual-correlation test<br/>+ iterative peeling"]
+    B -->|"ranked causal factors<br/>(decreasing causal power)"| C["Partition data on<br/>top causal factors<br/>(strength x significance rule)"]
+    C --> G1[(Subgroup 1)]
+    C --> G2[(Subgroup 2)]
+    C --> Gn[(Subgroup n)]
+    Q[Query point] --> R{inside a<br/>subgroup?}
+    R -->|"yes (interpolation)"| L["Train ONE local model<br/>in that subgroup only<br/>(Optimisation 2)"]
+    R -->|"no (extrapolation)"| F["Global model fallback<br/>(retains significance &<br/>generalization)"]
+    G1 -.-> L
+    G2 -.-> L
+    Gn -.-> L
+    B --> M["Missing causal power<br/>= drift early-warning alarm"]
+```

causalscreen-0.2.0/causalscreen/__init__.py

@@ -0,0 +1,4 @@
+from .screening import CausalScreen, ScreenResult
+from .local import PartitionedRegressor
+__version__ = "0.1.0"
+__all__ = ["CausalScreen", "ScreenResult", "PartitionedRegressor"]

causalscreen-0.2.0/causalscreen/local.py

@@ -0,0 +1,95 @@
+"""Causal-factor partitioning with inference-time localized training.
+
+Optimized implementation: cells are indexed ONCE at fit time (dict of
+cell-key -> row indices), local models are fitted at most once per cell
+and cached, and batch prediction groups queries by cell so each cell's
+model predicts its queries in a single vectorized call.  Semantics are
+identical to the per-query construction: a query gets a local model iff
+its exact cell holds >= min_cell training rows, else the global model.
+"""
+from __future__ import annotations
+
+import numpy as np
+from sklearn.base import clone
+from sklearn.linear_model import LinearRegression
+
+
+class PartitionedRegressor:
+    """Route each query to its causal-factor cell; train ONE local model there
+    (interpolation). Out-of-support queries fall back to the global model
+    (extrapolation), preserving generalization -- see paper Sec. 5.1.
+
+    Note: cell membership is exact-match on the partition features, so
+    partition features should be discrete/categorical.  Continuous partition
+    features will rarely match and queries will fall back to the global model.
+    """
+
+    def __init__(self, base_estimator=None, partition_features=None, min_cell: int = 30):
+        if min_cell < 2:
+            raise ValueError("min_cell must be >= 2")
+        self.base = base_estimator if base_estimator is not None else LinearRegression()
+        self.partition_features = partition_features or []
+        self.min_cell = min_cell
+
+    def fit(self, X, y, feature_names):
+        self.names_ = list(feature_names)
+        self.X_, self.y_ = np.asarray(X, float), np.asarray(y, float).ravel()
+        if self.X_.ndim != 2 or self.X_.shape[0] != self.y_.shape[0]:
+            raise ValueError("X must be 2-D with rows matching y")
+        if not np.isfinite(self.X_).all() or not np.isfinite(self.y_).all():
+            raise ValueError("X or y contains NaN or inf")
+        missing = [f for f in self.partition_features if f not in self.names_]
+        if missing:
+            raise ValueError(f"partition features not in feature_names: {missing}")
+        self.idx_ = [self.names_.index(f) for f in self.partition_features]
+        self.global_ = clone(self.base).fit(self.X_, self.y_)
+        # Index cells once: cell key -> training-row indices.
+        self._cells: dict[tuple, np.ndarray] = {}
+        if self.idx_:
+            keys = self.X_[:, self.idx_]
+            order = np.lexsort(keys.T[::-1])
+            sorted_keys = keys[order]
+            boundaries = np.flatnonzero(
+                np.r_[True, (sorted_keys[1:] != sorted_keys[:-1]).any(axis=1)]
+            )
+            for s, e in zip(boundaries, np.r_[boundaries[1:], len(order)]):
+                if e - s >= self.min_cell:
+                    self._cells[tuple(sorted_keys[s])] = order[s:e]
+        self._models: dict[tuple, object] = {}   # lazy per-cell model cache
+        return self
+
+    def _cell_model(self, key: tuple):
+        """Fit (once) and return the cached local model for a qualifying cell."""
+        mdl = self._models.get(key)
+        if mdl is None:
+            rows = self._cells[key]
+            mdl = clone(self.base).fit(self.X_[rows], self.y_[rows])
+            self._models[key] = mdl
+        return mdl
+
+    def predict_one(self, xq) -> float:
+        xq = np.asarray(xq, float)
+        if self.idx_:
+            key = tuple(xq[self.idx_])
+            if key in self._cells:
+                return float(self._cell_model(key).predict(xq.reshape(1, -1))[0])
+        return float(self.global_.predict(xq.reshape(1, -1))[0])
+
+    def predict(self, Xq) -> np.ndarray:
+        Xq = np.asarray(Xq, float)
+        if Xq.ndim == 1:
+            Xq = Xq.reshape(1, -1)
+        out = np.empty(len(Xq), float)
+        if not self.idx_:
+            return self.global_.predict(Xq).astype(float)
+        # Group queries by cell so each model predicts once, vectorized.
+        keys = [tuple(row) for row in Xq[:, self.idx_]]
+        groups: dict[tuple, list[int]] = {}
+        fallback: list[int] = []
+        for i, key in enumerate(keys):
+            (groups.setdefault(key, []) if key in self._cells else fallback).append(i)
+        for key, idxs in groups.items():
+            out[idxs] = self._cell_model(key).predict(Xq[idxs])
+        if fallback:
+            out[fallback] = self.global_.predict(Xq[fallback])
+        return out

causalscreen-0.2.0/causalscreen/screening.py

@@ -0,0 +1,159 @@
+"""Residual-correlation causal screening with iterative peeling.
+
+Implements the two-model residual construction (Frisch-Waugh-Lovell partial
+correlation) with an iterative peeling schedule, as described in:
+Mandal, R.K. (2026). "Residual-Correlation Causal Screening with Localized
+Supervised Modeling" (working paper).
+
+Optimized implementation: each screening round computes ALL candidate
+partial correlations from a single precision (inverse-covariance) matrix
+of the remaining design -- O(m^3) per round -- instead of 2m separate
+least-squares solves -- O(m * n * m^2).  Results are numerically identical
+to the residual construction (FWL theorem); a per-candidate residual
+fallback guards ill-conditioned designs.
+"""
+from __future__ import annotations
+
+import numpy as np
+from dataclasses import dataclass, field
+from scipy import stats
+
+_EPS = 1e-12
+
+
+@dataclass
+class ScreenResult:
+    ranking: list = field(default_factory=list)       # feature names, decreasing causal power
+    causal_power: dict = field(default_factory=dict)  # name -> partial correlation at selection round
+    p_value: dict = field(default_factory=dict)       # name -> p-value at selection round
+    missing_causal_power: float = np.nan              # share of response variance unexplained by ranked factors
+
+
+def _validate(X: np.ndarray, y: np.ndarray) -> None:
+    if X.ndim != 2:
+        raise ValueError(f"X must be 2-D, got shape {X.shape}")
+    if y.ndim != 1:
+        raise ValueError(f"y must be 1-D after ravel, got shape {y.shape}")
+    if X.shape[0] != y.shape[0]:
+        raise ValueError(f"X and y have mismatched rows: {X.shape[0]} vs {y.shape[0]}")
+    if X.shape[0] < 3:
+        raise ValueError("need at least 3 samples")
+    if not np.isfinite(X).all():
+        raise ValueError("X contains NaN or inf")
+    if not np.isfinite(y).all():
+        raise ValueError("y contains NaN or inf")
+
+
+def _residuals(y: np.ndarray, X: np.ndarray) -> np.ndarray:
+    """OLS residuals of y on X (with intercept)."""
+    if X.shape[1] == 0:
+        return y - y.mean()
+    A = np.column_stack([np.ones(len(X)), X])
+    beta, *_ = np.linalg.lstsq(A, y, rcond=None)
+    return y - A @ beta
+
+
+def _partial_corr(y: np.ndarray, xj: np.ndarray, Xrest: np.ndarray) -> tuple[float, float]:
+    """Reference path: corr(resid(y|Xrest), resid(xj|Xrest)) and its p-value."""
+    ey = _residuals(y, Xrest)
+    ej = _residuals(xj, Xrest)
+    if ey.std() < _EPS or ej.std() < _EPS:
+        return 0.0, 1.0
+    r = float(np.corrcoef(ey, ej)[0, 1])
+    return r, _pval(r, len(y), Xrest.shape[1])
+
+
+def _pval(r: float, n: int, k: int) -> float:
+    df = max(n - k - 2, 1)
+    t = r * np.sqrt(df / max(_EPS, 1.0 - r * r))
+    return float(2 * stats.t.sf(abs(t), df))
+
+
+def _round_partial_corrs(y: np.ndarray, Xr: np.ndarray) -> np.ndarray | None:
+    """Partial corr of y with each column of Xr, given the other columns.
+
+    Single precision-matrix computation for the whole round:
+        Z = [Xr, y] centered;  P = inv(cov(Z));
+        pcorr(y, x_j | rest) = -P[j, m] / sqrt(P[j, j] * P[m, m]).
+    Returns None if the design is too ill-conditioned to trust (caller
+    falls back to the per-candidate residual construction).
+    """
+    n, m = Xr.shape
+    Z = np.column_stack([Xr, y])
+    Z = Z - Z.mean(axis=0)
+    sd = Z.std(axis=0)
+    if (sd < _EPS).any():
+        return None                      # constant column: use reference path
+    Z /= sd                              # correlation scale for conditioning
+    C = (Z.T @ Z) / n
+    # reciprocal condition number guard
+    if np.linalg.cond(C) > 1e10:
+        return None
+    P = np.linalg.inv(C)
+    d = np.sqrt(np.diag(P))
+    r = -P[:m, m] / (d[:m] * d[m])
+    if not np.isfinite(r).all() or (np.abs(r) > 1.0 + 1e-8).any():
+        return None
+    return np.clip(r, -1.0, 1.0)
+
+
+class CausalScreen:
+    """Iterative residual-correlation screen ('peeling').
+
+    Each round computes, for every remaining candidate j, the partial
+    correlation of X_j with y given all other remaining candidates; the
+    strongest significant candidate is recorded and REMOVED from all
+    subsequent conditioning sets (so a discovered factor's influence cannot
+    re-enter through the variables it confounds).
+    """
+
+    def __init__(self, alpha: float = 0.05, max_factors: int | None = None):
+        if not 0.0 < alpha <= 1.0:
+            raise ValueError("alpha must be in (0, 1]")
+        self.alpha = alpha
+        self.max_factors = max_factors
+
+    def fit(self, X, y, feature_names=None) -> ScreenResult:
+        X = np.asarray(X, dtype=float)
+        y = np.asarray(y, dtype=float).ravel()
+        _validate(X, y)
+        n, p = X.shape
+        names = list(feature_names) if feature_names is not None else [f"x{j}" for j in range(p)]
+        if len(names) != p:
+            raise ValueError(f"feature_names has {len(names)} entries for {p} columns")
+        remaining = list(range(p))
+        res = ScreenResult()
+        budget = self.max_factors or p
+        y_cur = y.copy()          # response, progressively peeled of discovered factors
+        for _ in range(budget):
+            if not remaining:
+                break
+            m = len(remaining)
+            k = m - 1                       # conditioning-set size per candidate
+            rs = _round_partial_corrs(y_cur, X[:, remaining])
+            if rs is None:
+                # ill-conditioned round: exact per-candidate reference path
+                best = None
+                for j in remaining:
+                    rest = [t for t in remaining if t != j]
+                    r, pv = _partial_corr(y_cur, X[:, j], X[:, rest])
+                    if best is None or abs(r) > abs(best[1]):
+                        best = (j, r, pv)
+                j, r, pv = best
+            else:
+                i = int(np.argmax(np.abs(rs)))
+                j, r = remaining[i], float(rs[i])
+                pv = _pval(r, n, k)
+            if pv > self.alpha:
+                break
+            res.ranking.append(names[j])
+            res.causal_power[names[j]] = r
+            res.p_value[names[j]] = pv
+            remaining.remove(j)
+            # PEEL: strip the discovered factor's contribution out of the
+            # response so its effect cannot re-enter the screen through
+            # variables it confounds (paper Sec. 4.3).
+            y_cur = _residuals(y_cur, X[:, [j]])
+        res.missing_causal_power = float(np.var(y_cur) / np.var(y)) if np.var(y) > 0 else np.nan
+        self.result_ = res
+        return res

causalscreen-0.2.0/causalscreen.egg-info/PKG-INFO

@@ -0,0 +1,127 @@
+Metadata-Version: 2.4
+Name: causalscreen
+Version: 0.2.0
+Summary: Residual-correlation causal screening with iterative peeling, localized modeling, and drift early-warning
+Author-email: Rahul Kumar Mandal <rahulkm.jobs@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/rahulkm3/causalscreen
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy
+Requires-Dist: scipy
+Requires-Dist: scikit-learn
+Dynamic: license-file
+
+# causalscreen
+
+Residual-correlation causal screening with iterative peeling, causal-factor
+partitioning with inference-time local models, and a missing-causal-power
+drift alarm.
+
+Companion code for: **R. K. Mandal, "Residual-Correlation Causal Screening with
+Localized Supervised Modeling: A Practical Framework for Confounder
+Identification, Error Reduction, and Drift Early-Warning"** (working paper,
+2026; research 2022–present). Paper PDF in `/paper` · Preprint: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7135078
+
+## What it does
+
+Supervised models fail in deployment because they lean on **confounded
+proxies** — features whose correlation with the target is inherited from other
+variables. `causalscreen`:
+
+1. **Screens** every candidate feature via the two-regression residual
+   construction (Frisch–Waugh–Lovell partial correlation): regress `y` on the
+   other features, regress `x_j` on the other features, correlate the
+   residuals. High residual correlation = genuine independent effect.
+2. **Peels** iteratively: each discovered factor's contribution is stripped
+   out of the response before the next round, so its influence cannot re-enter
+   the screen through the variables it confounds.
+3. **Ranks** features by causal power and reports the **missing causal
+   power** — the share of response variance the discovered factors cannot
+   explain — a deployable early-warning statistic for data/model drift.
+4. **Exploits** the ranking: `PartitionedRegressor` partitions data on top
+   causal factors and trains one small local model per query at inference
+   time (interpolation), falling back to the global model out of support
+   (extrapolation) — so the error reduction of local modeling never costs
+   out-of-sample generalization.
+
+## Quick result (synthetic ground truth, `examples/demo.py`)
+
+| feature | naive \|corr\| with y | causal screen verdict |
+|---|---|---|
+| x1 (true cause) | 0.92 | **ranked** (power 0.78) |
+| x2 (confounded proxy of x1) | 0.87 | **rejected** |
+| x3 (noise) | 0.01 | rejected |
+| x4 (true cause, weak) | 0.33 | **ranked** (power 0.83) |
+
+Missing causal power: 5.1% — the exact noise floor of the generating process.
+Partitioned local modeling: **99.0% out-of-sample MSE reduction** vs. a global
+model on regime-structured data.
+
+## Install & use
+
+```bash
+pip install -e .
+python examples/demo.py
+pytest tests/
+```
+
+```python
+from causalscreen import CausalScreen, PartitionedRegressor
+res = CausalScreen(alpha=0.05).fit(X, y, feature_names)
+res.ranking            # features in decreasing causal power
+res.causal_power       # partial correlation at selection round
+res.missing_causal_power  # drift-alarm statistic
+
+pr = PartitionedRegressor(partition_features=res.ranking[:2], min_cell=50)
+pr.fit(X, y, feature_names)
+pr.predict(X_new)      # local model in-support, global model out-of-support
+```
+
+## Assumptions (stated plainly)
+
+Stationary cross-sectional data · causal sufficiency (all relevant variables
+observed) · no reverse causation · features measured before the response.
+The screen addresses the *confounding* component of causal analysis; it does
+not prove causality in full generality. See paper §3, §7.
+
+## Deployed results
+
+Layered over Random Forest, OLS and Prophet models in commercial deployments
+(credit risk, segmentation, forecasting), the framework reduced out-of-sample
+error by 65–85%. Client data is not releasable; this repo's synthetic
+benchmarks are the reproducible counterpart. DoWhy/EconML comparison suite:
+in progress.
+
+## Performance (v0.2.0)
+
+Screening uses a single precision-matrix inversion per round (FWL-equivalent,
+with an exact residual-regression fallback for ill-conditioned designs) —
+~77x faster than the reference implementation at n=5000, p=40. Partitioned
+prediction indexes cells at fit time and caches local models — ~409x faster
+on batch prediction (20k train / 5k queries). Equivalence to the reference
+implementation is enforced in the test suite to ~1e-8 (38 tests; reference
+implementation vendored in `tests/`).
+
+## License
+
+MIT
+
+## Pipeline
+
+```mermaid
+flowchart LR
+    A[Training data] --> B["CausalScreen<br/>residual-correlation test<br/>+ iterative peeling"]
+    B -->|"ranked causal factors<br/>(decreasing causal power)"| C["Partition data on<br/>top causal factors<br/>(strength x significance rule)"]
+    C --> G1[(Subgroup 1)]
+    C --> G2[(Subgroup 2)]
+    C --> Gn[(Subgroup n)]
+    Q[Query point] --> R{inside a<br/>subgroup?}
+    R -->|"yes (interpolation)"| L["Train ONE local model<br/>in that subgroup only<br/>(Optimisation 2)"]
+    R -->|"no (extrapolation)"| F["Global model fallback<br/>(retains significance &<br/>generalization)"]
+    G1 -.-> L
+    G2 -.-> L
+    Gn -.-> L
+    B --> M["Missing causal power<br/>= drift early-warning alarm"]
+```

causalscreen-0.2.0/causalscreen.egg-info/SOURCES.txt

@@ -0,0 +1,17 @@
+LICENSE
+MANIFEST.in
+README.md
+pyproject.toml
+causalscreen/__init__.py
+causalscreen/local.py
+causalscreen/screening.py
+causalscreen.egg-info/PKG-INFO
+causalscreen.egg-info/SOURCES.txt
+causalscreen.egg-info/dependency_links.txt
+causalscreen.egg-info/requires.txt
+causalscreen.egg-info/top_level.txt
+examples/demo.py
+tests/reference_impl_local.py
+tests/reference_impl_screening.py
+tests/test_optimized.py
+tests/test_screening.py

causalscreen-0.2.0/causalscreen.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

causalscreen-0.2.0/causalscreen.egg-info/requires.txt

@@ -0,0 +1,3 @@
+numpy
+scipy
+scikit-learn

causalscreen-0.2.0/causalscreen.egg-info/top_level.txt

@@ -0,0 +1 @@
+causalscreen

causalscreen-0.2.0/examples/demo.py

@@ -0,0 +1,39 @@
+"""Demo: confounded-proxy rejection + partitioned local modeling error reduction.
+Run: python examples/demo.py
+"""
+import numpy as np
+from causalscreen import CausalScreen, PartitionedRegressor
+
+rng = np.random.default_rng(7)
+n = 3000
+
+# --- Part 1: screening on synthetic ground truth ---
+x1 = rng.normal(size=n)                       # true cause (strong)
+x2 = 0.9 * x1 + 0.3 * rng.normal(size=n)      # confounded proxy of x1
+x3 = rng.normal(size=n)                       # noise
+x4 = rng.normal(size=n)                       # true cause (weak)
+y = 2.0 * x1 + 0.7 * x4 + 0.5 * rng.normal(size=n)
+X = np.column_stack([x1, x2, x3, x4]); names = ["x1", "x2", "x3", "x4"]
+
+res = CausalScreen().fit(X, y, names)
+print("== Screening ==")
+print(f"naive |corr| with y : " + ", ".join(f"{nm}={abs(np.corrcoef(X[:,j],y)[0,1]):.2f}" for j, nm in enumerate(names)))
+print(f"causal ranking      : {res.ranking}")
+print(f"causal powers       : " + ", ".join(f"{k}={v:.2f}" for k, v in res.causal_power.items()))
+print(f"missing causal power: {res.missing_causal_power:.1%}  (noise floor of the DGP is ~5%)")
+
+# --- Part 2: partition on a causal categorical factor; local vs global model ---
+g = rng.integers(0, 5, n).astype(float)       # causal regime variable
+xr = rng.normal(size=n)
+slopes = np.array([2.0, -1.0, 0.5, 3.0, -2.5])
+yr = slopes[g.astype(int)] * xr + 0.2 * rng.normal(size=n)
+Xr = np.column_stack([g, xr])
+tr, te = slice(0, 2500), slice(2500, None)
+
+pr = PartitionedRegressor(partition_features=["g"], min_cell=50)   # linear base model
+pr.fit(Xr[tr], yr[tr], ["g", "x"])
+mse_local = np.mean((pr.predict(Xr[te]) - yr[te]) ** 2)
+mse_global = np.mean((pr.global_.predict(Xr[te]) - yr[te]) ** 2)
+print("\n== Partitioned local modeling (out-of-sample) ==")
+print(f"global model MSE : {mse_global:.4f}")
+print(f"local  models MSE : {mse_local:.4f}   (reduction: {1 - mse_local/mse_global:.1%})")

causalscreen-0.2.0/pyproject.toml

@@ -0,0 +1,19 @@
+[build-system]
+requires = ["setuptools>=61"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "causalscreen"
+version = "0.2.0"
+description = "Residual-correlation causal screening with iterative peeling, localized modeling, and drift early-warning"
+authors = [{name = "Rahul Kumar Mandal", email = "rahulkm.jobs@gmail.com"}]
+readme = "README.md"
+license = {text = "MIT"}
+requires-python = ">=3.9"
+dependencies = ["numpy", "scipy", "scikit-learn"]
+
+[project.urls]
+Homepage = "https://github.com/rahulkm3/causalscreen"
+
+[tool.setuptools.packages.find]
+include = ["causalscreen*"]

causalscreen-0.2.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

causalscreen-0.2.0/tests/reference_impl_local.py

@@ -0,0 +1,42 @@
+"""Reference per-query local regressor. Test oracle only."""
+import numpy as np
+from sklearn.base import clone
+from sklearn.linear_model import LinearRegression
+
+
+class PartitionedRegressor:
+    """Route each query to its causal-factor cell; train ONE local model there
+    (interpolation). Out-of-support queries fall back to the global model
+    (extrapolation), preserving generalization -- see paper Sec. 5.1.
+    """
+
+    def __init__(self, base_estimator=None, partition_features=None, min_cell=30):
+        self.base = base_estimator if base_estimator is not None else LinearRegression()
+        self.partition_features = partition_features or []
+        self.min_cell = min_cell
+
+    def fit(self, X, y, feature_names):
+        self.names_ = list(feature_names)
+        self.X_, self.y_ = np.asarray(X, float), np.asarray(y, float).ravel()
+        self.idx_ = [self.names_.index(f) for f in self.partition_features]
+        self.global_ = clone(self.base).fit(self.X_, self.y_)
+        return self
+
+    def _cell_mask(self, xq):
+        mask = np.ones(len(self.X_), bool)
+        for i in self.idx_:
+            mask &= (self.X_[:, i] == xq[i])
+            if mask.sum() < self.min_cell:      # stop before losing significance
+                return None
+        return mask
+
+    def predict_one(self, xq):
+        xq = np.asarray(xq, float)
+        mask = self._cell_mask(xq) if self.idx_ else None
+        if mask is not None and mask.sum() >= self.min_cell:
+            local = clone(self.base).fit(self.X_[mask], self.y_[mask])
+            return float(local.predict(xq.reshape(1, -1))[0])
+        return float(self.global_.predict(xq.reshape(1, -1))[0])
+
+    def predict(self, Xq):
+        return np.array([self.predict_one(x) for x in np.asarray(Xq, float)])

causalscreen-0.2.0/tests/reference_impl_screening.py

@@ -0,0 +1,89 @@
+"""Reference implementation (naive residual construction). Test oracle only.
+
+Implements the two-model residual construction (Frisch-Waugh-Lovell partial
+correlation) with an iterative peeling schedule, as described in:
+Mandal, R.K. (2026). "Residual-Correlation Causal Screening with Localized
+Supervised Modeling" (working paper).
+"""
+import numpy as np
+from dataclasses import dataclass, field
+from scipy import stats
+
+
+@dataclass
+class ScreenResult:
+    ranking: list = field(default_factory=list)       # feature names, decreasing causal power
+    causal_power: dict = field(default_factory=dict)  # name -> partial correlation at selection round
+    p_value: dict = field(default_factory=dict)       # name -> p-value at selection round
+    missing_causal_power: float = np.nan              # share of response variance unexplained by ranked factors
+
+
+def _residuals(y, X):
+    """OLS residuals of y on X (with intercept)."""
+    if X.shape[1] == 0:
+        return y - y.mean()
+    A = np.column_stack([np.ones(len(X)), X])
+    beta, *_ = np.linalg.lstsq(A, y, rcond=None)
+    return y - A @ beta
+
+
+def _partial_corr(y, xj, Xrest):
+    """corr(resid(y|Xrest), resid(xj|Xrest)) and its p-value."""
+    ey = _residuals(y, Xrest)
+    ej = _residuals(xj, Xrest)
+    if ey.std() < 1e-12 or ej.std() < 1e-12:
+        return 0.0, 1.0
+    r = float(np.corrcoef(ey, ej)[0, 1])
+    n, k = len(y), Xrest.shape[1]
+    df = max(n - k - 2, 1)
+    t = r * np.sqrt(df / max(1e-12, 1 - r * r))
+    p = 2 * stats.t.sf(abs(t), df)
+    return r, float(p)
+
+
+class CausalScreen:
+    """Iterative residual-correlation screen ('peeling').
+
+    Each round computes, for every remaining candidate j, the partial
+    correlation of X_j with y given all other remaining candidates; the
+    strongest significant candidate is recorded and REMOVED from all
+    subsequent conditioning sets (so a discovered factor's influence cannot
+    re-enter through the variables it confounds).
+    """
+
+    def __init__(self, alpha=0.05, max_factors=None):
+        self.alpha = alpha
+        self.max_factors = max_factors
+
+    def fit(self, X, y, feature_names=None):
+        X = np.asarray(X, dtype=float)
+        y = np.asarray(y, dtype=float).ravel()
+        p = X.shape[1]
+        names = list(feature_names) if feature_names is not None else [f"x{j}" for j in range(p)]
+        remaining = list(range(p))
+        res = ScreenResult()
+        budget = self.max_factors or p
+        y_cur = y.copy()          # response, progressively peeled of discovered factors
+        for _ in range(budget):
+            if not remaining:
+                break
+            best = None
+            for j in remaining:
+                rest = [k for k in remaining if k != j]
+                r, pv = _partial_corr(y_cur, X[:, j], X[:, rest])
+                if best is None or abs(r) > abs(best[1]):
+                    best = (j, r, pv)
+            j, r, pv = best
+            if pv > self.alpha:
+                break
+            res.ranking.append(names[j])
+            res.causal_power[names[j]] = r
+            res.p_value[names[j]] = pv
+            remaining.remove(j)
+            # PEEL: strip the discovered factor's contribution out of the
+            # response so its effect cannot re-enter the screen through
+            # variables it confounds (paper Sec. 4.3).
+            y_cur = _residuals(y_cur, X[:, [j]])
+        res.missing_causal_power = float(np.var(y_cur) / np.var(y)) if np.var(y) > 0 else np.nan
+        self.result_ = res
+        return res

causalscreen-0.2.0/tests/test_optimized.py

@@ -0,0 +1,152 @@
+"""Robustness suite: equivalence with the reference residual construction,
+edge cases, batch/loop consistency, and input validation."""
+import sys, pathlib
+import numpy as np
+import pytest
+
+import importlib.util
+
+_HERE = pathlib.Path(__file__).parent
+
+def _load(name, path):
+    spec = importlib.util.spec_from_file_location(name, str(path))
+    mod = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(mod)
+    return mod
+
+# Reference implementations: the naive residual (FWL) construction and the
+# per-query local regressor, kept as the mathematical spec the optimized
+# code must reproduce exactly.
+ref_screen = _load("reference_impl_screening", _HERE / "reference_impl_screening.py")
+ref_local = _load("reference_impl_local", _HERE / "reference_impl_local.py")
+
+from causalscreen import CausalScreen, PartitionedRegressor
+from causalscreen.screening import _round_partial_corrs, _partial_corr
+
+
+# ---------- screening: numerical equivalence with reference ----------
+
+@pytest.mark.parametrize("seed", range(6))
+@pytest.mark.parametrize("n,p", [(200, 5), (500, 10), (120, 8)])
+def test_round_pcorrs_match_residual_construction(seed, n, p):
+    rng = np.random.default_rng(seed)
+    A = rng.normal(size=(p, p))
+    X = rng.normal(size=(n, p)) @ A          # correlated design
+    y = X @ rng.normal(size=p) + rng.normal(size=n)
+    rs = _round_partial_corrs(y, X)
+    assert rs is not None
+    for j in range(p):
+        rest = np.delete(np.arange(p), j)
+        r_ref, _ = _partial_corr(y, X[:, j], X[:, rest])
+        assert rs[j] == pytest.approx(r_ref, abs=1e-8)
+
+@pytest.mark.parametrize("seed", range(5))
+def test_full_screen_matches_reference(seed):
+    rng = np.random.default_rng(seed)
+    n, p = 800, 7
+    X = rng.normal(size=(n, p))
+    X[:, 1] = 0.8 * X[:, 0] + 0.3 * rng.normal(size=n)     # proxy
+    y = 1.5 * X[:, 0] - 0.9 * X[:, 4] + 0.5 * rng.normal(size=n)
+    names = [f"f{j}" for j in range(p)]
+    new = CausalScreen().fit(X, y, names)
+    old = ref_screen.CausalScreen().fit(X, y, names)
+    assert new.ranking == old.ranking
+    for f in new.ranking:
+        assert new.causal_power[f] == pytest.approx(old.causal_power[f], abs=1e-7)
+        assert new.p_value[f] == pytest.approx(old.p_value[f], rel=1e-4, abs=1e-12)
+    assert new.missing_causal_power == pytest.approx(old.missing_causal_power, abs=1e-9)
+
+def test_constant_feature_falls_back_and_matches_reference():
+    rng = np.random.default_rng(3)
+    n = 300
+    X = rng.normal(size=(n, 4)); X[:, 2] = 5.0             # constant column
+    y = 2 * X[:, 0] + rng.normal(size=n)
+    names = list("abcd")
+    new = CausalScreen().fit(X, y, names)
+    old = ref_screen.CausalScreen().fit(X, y, names)
+    assert new.ranking == old.ranking
+    assert "c" not in new.ranking
+
+def test_perfectly_collinear_features_do_not_crash():
+    rng = np.random.default_rng(4)
+    n = 300
+    x = rng.normal(size=n)
+    X = np.column_stack([x, 2 * x, rng.normal(size=n)])    # exact collinearity
+    y = x + 0.1 * rng.normal(size=n)
+    res = CausalScreen().fit(X, y, ["a", "a2", "b"])
+    assert isinstance(res.ranking, list)                    # no crash; sane output
+    assert np.isfinite(res.missing_causal_power)
+
+def test_max_factors_budget_respected():
+    rng = np.random.default_rng(5)
+    X = rng.normal(size=(500, 6))
+    y = X @ np.ones(6) + 0.1 * rng.normal(size=500)
+    res = CausalScreen(max_factors=2).fit(X, y)
+    assert len(res.ranking) <= 2
+
+def test_screen_input_validation():
+    with pytest.raises(ValueError):
+        CausalScreen(alpha=0.0)
+    with pytest.raises(ValueError):
+        CausalScreen().fit(np.ones((5, 2)), np.ones(4))
+    Xbad = np.ones((10, 2)); Xbad[0, 0] = np.nan
+    with pytest.raises(ValueError):
+        CausalScreen().fit(Xbad, np.ones(10))
+    with pytest.raises(ValueError):
+        CausalScreen().fit(np.random.default_rng(0).normal(size=(10, 3)),
+                           np.zeros(10), ["a", "b"])       # name-count mismatch
+
+# ---------- partitioned regressor: equivalence + batching ----------
+
+def _cells_data(seed=1, n=2500):
+    rng = np.random.default_rng(seed)
+    g = rng.integers(0, 5, n).astype(float)
+    h = rng.integers(0, 3, n).astype(float)
+    x = rng.normal(size=n)
+    y = (g - 2) * x + 0.5 * h + 0.2 * rng.normal(size=n)
+    return np.column_stack([g, h, x]), y, ["g", "h", "x"]
+
+def test_predictions_match_reference_implementation():
+    X, y, names = _cells_data()
+    Xq = X[:300]
+    new = PartitionedRegressor(partition_features=["g", "h"], min_cell=40).fit(X, y, names)
+    old = ref_local.PartitionedRegressor(partition_features=["g", "h"], min_cell=40).fit(X, y, names)
+    np.testing.assert_allclose(new.predict(Xq), old.predict(Xq), atol=1e-8)
+
+def test_batch_predict_equals_loop_predict():
+    X, y, names = _cells_data(seed=2)
+    pr = PartitionedRegressor(partition_features=["g"], min_cell=40).fit(X, y, names)
+    Xq = np.vstack([X[:100], [[99.0, 0.0, 0.3]]])          # incl. unseen cell
+    loop = np.array([pr.predict_one(x) for x in Xq])
+    np.testing.assert_allclose(pr.predict(Xq), loop, atol=1e-10)
+
+def test_out_of_support_uses_global_model():
+    X, y, names = _cells_data(seed=3)
+    pr = PartitionedRegressor(partition_features=["g"], min_cell=40).fit(X, y, names)
+    xq = np.array([42.0, 1.0, 0.5])                        # cell never seen
+    assert pr.predict_one(xq) == pytest.approx(float(pr.global_.predict(xq.reshape(1, -1))[0]))
+
+def test_small_cell_falls_back_to_global():
+    X, y, names = _cells_data(seed=4)
+    pr = PartitionedRegressor(partition_features=["g"], min_cell=10**6).fit(X, y, names)
+    np.testing.assert_allclose(pr.predict(X[:50]), pr.global_.predict(X[:50]), atol=1e-10)
+
+def test_no_partition_features_is_pure_global():
+    X, y, names = _cells_data(seed=5)
+    pr = PartitionedRegressor().fit(X, y, names)
+    np.testing.assert_allclose(pr.predict(X[:50]), pr.global_.predict(X[:50]), atol=1e-10)
+
+def test_local_models_cached_one_fit_per_cell():
+    X, y, names = _cells_data(seed=6)
+    pr = PartitionedRegressor(partition_features=["g"], min_cell=40).fit(X, y, names)
+    pr.predict(X[:500]); n_models = len(pr._models)
+    pr.predict(X[:500])                                     # second pass: no refits
+    assert len(pr._models) == n_models
+    assert n_models <= len(pr._cells)
+
+def test_partition_validation():
+    X, y, names = _cells_data(seed=7)
+    with pytest.raises(ValueError):
+        PartitionedRegressor(partition_features=["nope"]).fit(X, y, names)
+    with pytest.raises(ValueError):
+        PartitionedRegressor(min_cell=1)

causalscreen-0.2.0/tests/test_screening.py

@@ -0,0 +1,47 @@
+import numpy as np
+from causalscreen import CausalScreen, PartitionedRegressor
+
+def _synth(n=4000, seed=0):
+    """Ground truth: x1 true cause; x2 = confounded proxy of x1 (no own effect);
+    x3 pure noise; x4 weaker true cause."""
+    rng = np.random.default_rng(seed)
+    x1 = rng.normal(size=n)
+    x2 = 0.9 * x1 + 0.3 * rng.normal(size=n)   # rides on x1
+    x3 = rng.normal(size=n)
+    x4 = rng.normal(size=n)
+    y = 2.0 * x1 + 0.7 * x4 + 0.5 * rng.normal(size=n)
+    X = np.column_stack([x1, x2, x3, x4])
+    return X, y
+
+def test_ranking_recovers_true_causes():
+    X, y = _synth()
+    res = CausalScreen().fit(X, y, ["x1", "x2", "x3", "x4"])
+    assert set(res.ranking[:2]) == {"x1", "x4"}   # both true causes found
+    assert "x2" not in res.ranking                  # confounded proxy rejected
+    assert "x3" not in res.ranking                  # noise rejected
+
+def test_raw_corr_vs_causal_power_exposes_proxy():
+    X, y = _synth()
+    raw2 = abs(np.corrcoef(X[:, 1], y)[0, 1])
+    res = CausalScreen().fit(X, y, ["x1", "x2", "x3", "x4"])
+    assert raw2 > 0.8                      # proxy looks great naively...
+    assert "x2" not in res.ranking          # ...but is rejected by the screen
+
+def test_missing_causal_power_bounds():
+    X, y = _synth()
+    res = CausalScreen().fit(X, y, ["x1", "x2", "x3", "x4"])
+    assert 0.0 <= res.missing_causal_power < 0.2
+
+def test_partitioned_regressor_runs_and_beats_nothing():
+    rng = np.random.default_rng(1)
+    n = 3000
+    g = rng.integers(0, 4, n).astype(float)      # categorical causal factor
+    x = rng.normal(size=n)
+    slopes = np.array([2.0, -1.0, 0.5, 3.0])
+    y = slopes[g.astype(int)] * x + 0.2 * rng.normal(size=n)
+    X = np.column_stack([g, x])
+    pr = PartitionedRegressor(partition_features=["g"], min_cell=50).fit(X, y, ["g", "x"])
+    Xq = X[:200]
+    err_local = np.mean((pr.predict(Xq) - y[:200]) ** 2)
+    err_global = np.mean((pr.global_.predict(Xq) - y[:200]) ** 2)
+    assert err_local < 0.25 * err_global    # local models slash error in-cell