squeeze-kernel 0.5.0__tar.gz → 0.6.0__tar.gz

{squeeze_kernel-0.5.0 → squeeze_kernel-0.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: squeeze-kernel
-Version: 0.5.0
+Version: 0.6.0
 Summary: Streaming, PSD-by-construction covariance estimator with Fisher-kernel weighting and adaptive shrinkage
 Keywords: covariance,correlation,ewma,kernel,risk,streaming
 Author: Robert Kende
@@ -116,7 +116,7 @@ kappa = SqueezeKernelEstimator.calibrate_kappa(burn_in_returns, target_weight=0.
 
 ## Advanced options
 
-**Scale-free correlation memory** (`corr_half_lives=(43, 173, 693)`, `corr_theta=0.25`): replaces the single correlation timescale with a positive combination of EWMAs on a geometric half-life ladder — each scale normalized and adaptively shrunk against its own effective sample size, then the covariances blended with weights ∝ half-life^`corr_theta`. By Bernstein's theorem this approximates the power-law memory of financial correlations (the streaming analogue of HAR); positive weights keep it PSD by construction, and `None` (default) reproduces the published single-scale estimator exactly. This is the paper's **headline configuration**: on the S&P 500 benchmark it leads every tested method at every universe size (held-out one-step NLL −3.9 at n=100, up to −18 at n=300 before the cluster target), and the 90% model confidence set collapses to it alone. Cost is O(K·n²) per update. Composes with `shrinkage_target="cluster"`; mutually exclusive with `lambda_corr_fast`.
+**Adaptive scale-free correlation memory** (`corr_half_lives=(43, 173, 693)`, `corr_theta=0.25`): replaces the single correlation timescale with a positive combination of EWMAs on a geometric half-life ladder — each scale normalized and adaptively shrunk against its own effective sample size, then the covariances blended with weights resting at the prior ∝ half-life^`corr_theta`. By Bernstein's theorem the ladder approximates the power-law memory of financial correlations (the streaming analogue of HAR). For ladders of two or more rungs the blend weights are gated by a sequential surprise detector: a two-sided Page CUSUM on the studentized fast-vs-slow per-rung predictive-score drift (threshold set by Siegmund's average-run-length approximation at ~2 years, no tuned parameters) tilts the weights toward the fast or slow end of the ladder when one side accumulates statistically forced evidence, decaying back at the fastest rung's half-life. Weights equal the prior on all non-alarmed days, the blend stays convex, so PSD holds by construction; `None` (default) reproduces the published single-scale estimator exactly. This is the paper's **headline configuration**: on the S&P 500 benchmark it leads every tested method at every universe size (held-out one-step NLL −4.6 vs single-scale at n=100, −11.1 at n=300 before the cluster target), the 90% model confidence set collapses to it alone, and the detector's margin is confirmed out-of-time on an external industry panel (+0.53 NLL/day, p=1×10⁻⁴). Cost is O(K·n²) per update plus one Cholesky per rung per day for the detector scores. Composes with `shrinkage_target="cluster"`; mutually exclusive with `lambda_corr_fast`.
 
 ```python
 est = SqueezeKernelEstimator(n_assets=100, corr_half_lives=(43, 173, 693), corr_theta=0.25)
@@ -144,6 +144,15 @@ est = SqueezeKernelEstimator(n_assets=100, vol_anchor_phi=0.995)
 est = SqueezeKernelEstimator(n_assets=300, shrinkage_target="cluster")
 ```
 
+**New-listing usability gate** (`min_obs=60`): on an expanding universe, an asset's forecast rows are dominated by its single-observation variance initialization for its first weeks of life and are unusable for scoring or portfolio construction (measured ≈ +2,800 NLL/day on days whose scored set included such assets, on a 42-instrument multi-asset panel). `min_obs` gates nothing inside the estimator — states warm normally, all outputs are unchanged — it exposes a `usable_mask` property marking assets with at least `min_obs` finite observations, so deployments subset with it:
+
+```python
+est = SqueezeKernelEstimator(n_assets=42, min_obs=60)
+# ... update loop ...
+m = est.usable_mask
+cov_usable = est.get_cov()[np.ix_(m, m)]
+```
+
 **Alternative kernels**: pass `kernel_fn=kernel_exponential` (with `kernel_kwargs={"gamma": ...}`) or `kernel_chi2_cdf`, or any callable `(d2, *, n_observed, **kw) -> float` mapping to `[0, 1)`. The PSD guarantee holds for any such kernel.
 
 ## How it works

{squeeze_kernel-0.5.0 → squeeze_kernel-0.6.0}/README.md

@@ -86,7 +86,7 @@ kappa = SqueezeKernelEstimator.calibrate_kappa(burn_in_returns, target_weight=0.
 
 ## Advanced options
 
-**Scale-free correlation memory** (`corr_half_lives=(43, 173, 693)`, `corr_theta=0.25`): replaces the single correlation timescale with a positive combination of EWMAs on a geometric half-life ladder — each scale normalized and adaptively shrunk against its own effective sample size, then the covariances blended with weights ∝ half-life^`corr_theta`. By Bernstein's theorem this approximates the power-law memory of financial correlations (the streaming analogue of HAR); positive weights keep it PSD by construction, and `None` (default) reproduces the published single-scale estimator exactly. This is the paper's **headline configuration**: on the S&P 500 benchmark it leads every tested method at every universe size (held-out one-step NLL −3.9 at n=100, up to −18 at n=300 before the cluster target), and the 90% model confidence set collapses to it alone. Cost is O(K·n²) per update. Composes with `shrinkage_target="cluster"`; mutually exclusive with `lambda_corr_fast`.
+**Adaptive scale-free correlation memory** (`corr_half_lives=(43, 173, 693)`, `corr_theta=0.25`): replaces the single correlation timescale with a positive combination of EWMAs on a geometric half-life ladder — each scale normalized and adaptively shrunk against its own effective sample size, then the covariances blended with weights resting at the prior ∝ half-life^`corr_theta`. By Bernstein's theorem the ladder approximates the power-law memory of financial correlations (the streaming analogue of HAR). For ladders of two or more rungs the blend weights are gated by a sequential surprise detector: a two-sided Page CUSUM on the studentized fast-vs-slow per-rung predictive-score drift (threshold set by Siegmund's average-run-length approximation at ~2 years, no tuned parameters) tilts the weights toward the fast or slow end of the ladder when one side accumulates statistically forced evidence, decaying back at the fastest rung's half-life. Weights equal the prior on all non-alarmed days, the blend stays convex, so PSD holds by construction; `None` (default) reproduces the published single-scale estimator exactly. This is the paper's **headline configuration**: on the S&P 500 benchmark it leads every tested method at every universe size (held-out one-step NLL −4.6 vs single-scale at n=100, −11.1 at n=300 before the cluster target), the 90% model confidence set collapses to it alone, and the detector's margin is confirmed out-of-time on an external industry panel (+0.53 NLL/day, p=1×10⁻⁴). Cost is O(K·n²) per update plus one Cholesky per rung per day for the detector scores. Composes with `shrinkage_target="cluster"`; mutually exclusive with `lambda_corr_fast`.
 
 ```python
 est = SqueezeKernelEstimator(n_assets=100, corr_half_lives=(43, 173, 693), corr_theta=0.25)
@@ -114,6 +114,15 @@ est = SqueezeKernelEstimator(n_assets=100, vol_anchor_phi=0.995)
 est = SqueezeKernelEstimator(n_assets=300, shrinkage_target="cluster")
 ```
 
+**New-listing usability gate** (`min_obs=60`): on an expanding universe, an asset's forecast rows are dominated by its single-observation variance initialization for its first weeks of life and are unusable for scoring or portfolio construction (measured ≈ +2,800 NLL/day on days whose scored set included such assets, on a 42-instrument multi-asset panel). `min_obs` gates nothing inside the estimator — states warm normally, all outputs are unchanged — it exposes a `usable_mask` property marking assets with at least `min_obs` finite observations, so deployments subset with it:
+
+```python
+est = SqueezeKernelEstimator(n_assets=42, min_obs=60)
+# ... update loop ...
+m = est.usable_mask
+cov_usable = est.get_cov()[np.ix_(m, m)]
+```
+
 **Alternative kernels**: pass `kernel_fn=kernel_exponential` (with `kernel_kwargs={"gamma": ...}`) or `kernel_chi2_cdf`, or any callable `(d2, *, n_observed, **kw) -> float` mapping to `[0, 1)`. The PSD guarantee holds for any such kernel.
 
 ## How it works

{squeeze_kernel-0.5.0 → squeeze_kernel-0.6.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "uv_build"
 
 [project]
 name = "squeeze-kernel"
-version = "0.5.0"
+version = "0.6.0"
 description = "Streaming, PSD-by-construction covariance estimator with Fisher-kernel weighting and adaptive shrinkage"
 readme = "README.md"
 license = "MIT"

{squeeze_kernel-0.5.0 → squeeze_kernel-0.6.0}/src/squeeze_kernel/__init__.py

@@ -34,4 +34,4 @@ __all__ = [
     "kernel_chi2_cdf",
 ]
 
-__version__ = "0.5.0"
+__version__ = "0.6.0"

{squeeze_kernel-0.5.0 → squeeze_kernel-0.6.0}/src/squeeze_kernel/estimator.py

@@ -6,6 +6,15 @@ from typing import TYPE_CHECKING
 
 import numpy as np
 
+try:
+    # SciPy's direct LAPACK bindings factorise an SPD matrix ~4x faster than
+    # the NumPy slogdet+solve route (one dpotrf vs two LU factorisations) and
+    # compute the identical quantities; the detector falls back to the
+    # NumPy-only path when SciPy is absent.
+    from scipy.linalg import cho_factor as _cho_factor, cho_solve as _cho_solve
+except ImportError:  # pragma: no cover
+    _cho_factor = _cho_solve = None
+
 from squeeze_kernel.kernels import (
     KernelFn, kernel_fisher, calibrate_kappa, extract_d2_series,
 )
@@ -113,27 +122,63 @@ class SqueezeKernelEstimator:
         benchmark: +0.14 (negligible) at n=100, −4.2 at n=200, −25.0 at
         n=300.  Recommended when n approaches the effective sample size.
     corr_half_lives : sequence of float, optional
-        Scale-free correlation memory.  When set, the single correlation
-        timescale is replaced by a positive combination of EWMAs on the
-        given geometric half-life ladder (in trading days, e.g.
-        ``(43, 173, 693)``): each scale is normalised and adaptively
-        shrunk against its own effective sample size, and the resulting
-        covariances are blended with weights ∝ half-life\\ :sup:`corr_theta`.
-        By Bernstein's theorem this approximates the power-law memory of
-        financial correlations (the streaming analogue of HAR).  PSD by
-        construction (positive combination of PSD matrices).  ``None``
-        (default) is the published single-scale estimator, bit-for-bit;
-        a one-element ladder reduces to a single-scale estimator at that
-        half-life.  Mutually exclusive with ``lambda_corr_fast``; composes
-        with ``shrinkage_target='cluster'``.  Cost is O(K·n²) per update.
-        Held-out one-step NLL on the S&P-500 benchmark improves at every
-        dimension (−3.9 at n=100, up to −18 at n=300 before the cluster
-        target); the $90\\%$ model confidence set collapses to this
-        configuration alone.  Recommended default: the base-centred ladder
-        ``(43, 173, 693)`` with ``corr_theta=0.25``.
+        Scale-free correlation memory with surprise-gated adaptive blend
+        weights.  When set, the single correlation timescale is replaced
+        by a positive combination of EWMAs on the given geometric
+        half-life ladder (in trading days, e.g. ``(43, 173, 693)``): each
+        scale is normalised and adaptively shrunk against its own
+        effective sample size, and the resulting covariances are blended
+        with weights resting at the prior ∝ half-life\\ :sup:`corr_theta`.
+        By Bernstein's theorem the ladder approximates the power-law
+        memory of financial correlations (the streaming analogue of HAR).
+        For ladders of two or more rungs the blend weights are gated by a
+        sequential surprise detector: a two-sided Page CUSUM on the
+        studentised fast-vs-slow per-rung predictive-score drift
+        (reference drift 0.5, threshold 4.9721 = Siegmund average-run-
+        length ~2 years); an alarm applies a half-magnitude tilt of the
+        theta-prior toward the inverse-horizon vector (fast alarms,
+        w ~ 1/h) or the square-root-horizon vector (slow alarms,
+        w ~ h^0.5), decaying at the fastest rung's half-life.  Weights
+        equal the prior on all non-alarmed days and the blend stays
+        convex, so PSD is structural throughout.  The detector adds five
+        scalars of state and one Cholesky per rung per update for the
+        scores; it is validated across S&P panels (held-out +0.5-0.6
+        NLL/day over the detector-off blend), an external industry panel
+        incl. an out-of-time seal (+0.53/day, p=1e-4), a multi-asset
+        futures panel, and synthetic regime/null suites, with a
+        sign-stable one-at-a-time sensitivity sweep over all structural
+        constants.  ``None`` (default) is the published single-scale
+        estimator, bit-for-bit; a one-element ladder reduces to a
+        single-scale estimator at that half-life (no detector).
+        Mutually exclusive with ``lambda_corr_fast``; composes with
+        ``shrinkage_target='cluster'``.  Cost is O(K·n²) per update plus
+        the detector's per-rung score factorisations.  Held-out one-step
+        NLL on the S&P-500 benchmark improves on the single-scale
+        estimator at every dimension (−4.6 at n=100, −11.1 at n=300
+        before the cluster target); the $90\\%$ model confidence set
+        collapses to this configuration alone.  Recommended default: the
+        base-centred ladder ``(43, 173, 693)`` with ``corr_theta=0.25``.
     corr_theta : float
-        Long-memory exponent controlling the ladder weights (default
-        0.25).  Only used when ``corr_half_lives`` is set.
+        Long-memory exponent controlling the resting blend weights
+        (default 0.25).  Only used when ``corr_half_lives`` is set.
+    min_obs : int or None
+        Usability gate for newly listed assets.  When set, the property
+        ``usable_mask`` marks an asset usable only once it has delivered
+        at least ``min_obs`` finite observations.  The gate is purely
+        diagnostic: state evolution and ``get_cov``/``get_corr`` are
+        unchanged (the states keep warming during the gated window, so an
+        asset is fully warm when the gate lifts).  Motivation: on an
+        expanding multi-asset universe, forecast rows for assets in their
+        first ~100 observations are dominated by the single-observation
+        variance initialisation and are unusable for scoring or portfolio
+        construction (measured ≈ +2,800 NLL/day on days whose scored set
+        included such assets).  Deployment recipe::
+
+            m = est.usable_mask
+            cov_usable = est.get_cov()[np.ix_(m, m)]
+
+        Recommended ``min_obs`` ≈ 60–100 for daily data.  ``None``
+        (default) disables the gate (``usable_mask`` is all-True).
 
     Examples
     --------
@@ -167,6 +212,7 @@ class SqueezeKernelEstimator:
         shrinkage_target: str = "equicorrelation",
         corr_half_lives: "Sequence[float] | None" = None,
         corr_theta: float = 0.25,
+        min_obs: int | None = None,
     ):
         self.n_assets = n_assets
         self.lambda_vol = lambda_vol
@@ -192,6 +238,10 @@ class SqueezeKernelEstimator:
         if shrinkage_target not in ("equicorrelation", "cluster"):
             raise ValueError("shrinkage_target must be 'equicorrelation' or 'cluster'.")
         self.shrinkage_target = shrinkage_target
+        if min_obs is not None and (not isinstance(min_obs, int) or min_obs < 1):
+            raise ValueError("min_obs must be a positive integer or None.")
+        self.min_obs = min_obs
+        self._obs_count = np.zeros(n_assets, dtype=np.int64)
 
         # Scale-free correlation memory (opt-in): replace the single correlation
         # timescale by a positive combination of EWMAs on a geometric half-life
@@ -201,8 +251,9 @@ class SqueezeKernelEstimator:
         self.corr_theta = corr_theta
         self._corr_lam: np.ndarray | None = None
         self._corr_w: np.ndarray | None = None
-        self._M_list: list[np.ndarray] | None = None
+        self._Q_list: list[np.ndarray] | None = None
         self._S_list: list[float] | None = None
+        self._adaptive = False
         if corr_half_lives is not None:
             hl = np.asarray(corr_half_lives, dtype=np.float64)
             if hl.ndim != 1 or hl.size < 1 or np.any(hl <= 0.0):
@@ -218,8 +269,23 @@ class SqueezeKernelEstimator:
             self._corr_lam = 2.0 ** (-1.0 / hl)
             w = hl ** corr_theta
             self._corr_w = w / w.sum()
-            self._M_list = [np.eye(n_assets, dtype=np.float64) * epsilon for _ in hl]
+            self._Q_list = [np.eye(n_assets, dtype=np.float64) for _ in hl]
             self._S_list = [float(epsilon) for _ in hl]
+            # Surprise-gated blend weights (integral for K >= 2): two-sided
+            # Page CUSUM on the studentised fast-vs-slow per-rung
+            # predictive-score drift; drift 0.5, threshold from Siegmund's
+            # ARL approximation at ARL0 = 504 trading days (~2 years).
+            self._adaptive = hl.size >= 2
+            if self._adaptive:
+                self._aw_pi_fast = (1.0 / hl) / (1.0 / hl).sum()
+                self._aw_pi_slow = hl ** 0.5 / (hl ** 0.5).sum()
+                self._aw_lam_tilt = 2.0 ** (-1.0 / float(hl.min()))
+                self._aw_gamma_scale = 2.0 ** (-1.0 / float(np.median(hl)))
+                self._aw_drift, self._aw_b, self._aw_snap = 0.5, 4.9721088583, 0.5
+                self._aw_gp = self._aw_gm = 0.0
+                self._aw_tilt = 0.0
+                self._aw_scale = 1.0
+                self._aw_prev_sig: list[np.ndarray] | None = None
 
         # Resolve shrinkage
         if isinstance(shrinkage, str):
@@ -231,21 +297,33 @@ class SqueezeKernelEstimator:
         self._kernel_fn, self._kernel_kwargs = _resolve_kernel(kappa, kernel_fn, kernel_kwargs)
         self.kappa = self._kernel_kwargs.get("kappa") if self._kernel_fn is kernel_fisher else None
 
-        # State
+        # State. The correlation memory is stored NORMALISED: Q_t = M_t / S_t
+        # with the recursion Q_t = (1 - eta_t) Q_{t-1} + eta_t z_t z_t',
+        # eta_t = w_t / S_t after S_t <- lam S_{t-1} + w_t. This is
+        # algebraically identical to the raw-mass form (M init eps*I, S init
+        # eps => Q init I), keeps the matrix state well scaled, and makes the
+        # PSD convex-combination recursion explicit.
         self._var_t: np.ndarray | None = None
         self._var_init: np.ndarray | None = None
         self._var_anchor: np.ndarray | None = None
-        self._M_t = np.eye(n_assets, dtype=np.float64) * epsilon
+        self._vol_t: np.ndarray | None = None
+        # No single-scale state is allocated in ladder mode.
+        self._Q_t = (np.eye(n_assets, dtype=np.float64)
+                     if self._corr_lam is None else None)
         self._S_t = float(epsilon)
         self._cov: np.ndarray | None = None
         self._corr: np.ndarray | None = None
         self._last_weight: float = 0.0
+        # Extraction (normalise + shrink + vol application) is deferred until
+        # get_cov()/get_corr(); _dirty marks state newer than _cov/_corr.
+        self._dirty = False
 
         # Cached scratch buffers reused per ``update()`` to avoid per-step
         # allocator churn. These are intentionally module-private and
         # never escape the estimator.
         self._scratch_outer = np.empty((n_assets, n_assets), dtype=np.float64)
         self._scratch_corr = np.empty((n_assets, n_assets), dtype=np.float64)
+        self._scratch_had = np.empty((n_assets, n_assets), dtype=np.float64)
         self._n_off = float(n_assets * (n_assets - 1)) if n_assets > 1 else 1.0
 
     # ── Public API ────────────────────────────────────────────────────────
@@ -270,6 +348,63 @@ class SqueezeKernelEstimator:
             raise ValueError(f"Expected shape ({n},), got {r_t.shape}.")
 
         finite = np.isfinite(r_t)
+        self._obs_count[finite] += 1
+
+        # ── Adaptive-weight detector: score r_t under yesterday's per-rung
+        # forecasts, then advance the CUSUM (weights used below therefore
+        # reflect information through r_t only — causal). ──
+        if (self._adaptive and self._aw_prev_sig is not None
+                and finite.any()):
+            oidx = np.flatnonzero(finite)
+            r_o = r_t[oidx]
+            ell = np.empty(len(self._aw_prev_sig))
+            ok = True
+            for k, sig in enumerate(self._aw_prev_sig):
+                sub = sig[np.ix_(oidx, oidx)]
+                sub = (sub + sub.T) * 0.5
+                if _cho_factor is not None:
+                    # One Cholesky per rung: logdet from the factor's
+                    # diagonal, quadratic form via triangular solves.
+                    try:
+                        cf = _cho_factor(sub, lower=True, check_finite=False)
+                    except np.linalg.LinAlgError:
+                        ok = False           # degenerate day (e.g. a fresh
+                        break                # listing): no clean score
+                    logdet = 2.0 * np.log(np.diagonal(cf[0])).sum()
+                    quad = float(r_o @ _cho_solve(cf, r_o, check_finite=False))
+                else:
+                    sign, logdet = np.linalg.slogdet(sub)
+                    if sign <= 0:
+                        ok = False           # degenerate day (e.g. a fresh
+                        break                # listing): no clean score
+                    try:
+                        quad = float(r_o @ np.linalg.solve(sub, r_o))
+                    except np.linalg.LinAlgError:
+                        ok = False
+                        break
+                ell[k] = -0.5 * (oidx.size * np.log(2 * np.pi) + logdet + quad)
+            if not ok:
+                self._aw_tilt *= self._aw_lam_tilt
+                ell = None
+        else:
+            ell = None
+        if ell is not None:
+            dd = ell - ell.mean()
+            rms = float(np.sqrt((dd @ dd) / ell.size))
+            self._aw_scale = (self._aw_gamma_scale * self._aw_scale
+                              + (1.0 - self._aw_gamma_scale) * rms)
+            zc = np.clip(dd / (self._aw_scale + 1e-12), -3.0, 3.0)
+            k_fast = int(np.argmin(self.corr_half_lives))
+            k_slow = int(np.argmax(self.corr_half_lives))
+            zfs = float(zc[k_fast] - zc[k_slow])
+            self._aw_gp = max(0.0, self._aw_gp + zfs - self._aw_drift)
+            self._aw_gm = max(0.0, self._aw_gm - zfs - self._aw_drift)
+            if self._aw_gp > self._aw_b:
+                self._aw_tilt, self._aw_gp = self._aw_snap, 0.0
+            elif self._aw_gm > self._aw_b:
+                self._aw_tilt, self._aw_gm = -self._aw_snap, 0.0
+            else:
+                self._aw_tilt *= self._aw_lam_tilt
 
         # ── Volatility update ──
         if self._var_t is None:
@@ -317,10 +452,13 @@ class SqueezeKernelEstimator:
         if n_obs > 0:
             z_t[finite] = r_t[finite] / (vol_t[finite] + eps)
             d2 = float(z_t[finite] @ z_t[finite]) / n_obs
-            if self.weight_statistic == "mahalanobis" and self._corr is not None:
+            if self.weight_statistic == "mahalanobis" and self._vol_t is not None:
                 # Score-exact surprise against the estimator's own previous
                 # correlation; falls back to the marginal d² on the first
-                # step or a (rare) singular observed submatrix.
+                # step or a (rare) singular observed submatrix. Extraction is
+                # lazy, so bring _corr up to the t-1 state first.
+                if self._dirty:
+                    self._materialize()
                 try:
                     c_sub = self._corr[np.ix_(finite, finite)]
                     d2 = float(z_t[finite] @ np.linalg.solve(c_sub, z_t[finite])) / n_obs
@@ -334,58 +472,63 @@ class SqueezeKernelEstimator:
         if self.impute_missing and 0 < n_obs < n:
             self._impute(z_t, finite)
 
+        add = w_t > 0.0 and n_obs > 0
+        if add:
+            # np.multiply.outer with out= avoids the temporary that
+            # np.outer otherwise allocates each step. zz' is computed once
+            # and shared by every rung.
+            np.multiply.outer(z_t, z_t, out=self._scratch_outer)
         if self._corr_lam is None:
-            # ── Single-scale correlation EWMA (published path, unchanged) ──
+            # ── Single-scale correlation EWMA (published path) ──
             lam_c = self.lambda_corr
             if self.lambda_corr_fast is not None:
                 # Score-driven memory: stress days (w_t → 1) shorten the memory
                 # toward lambda_corr_fast; calm days keep the slow decay.
                 lam_c = self.lambda_corr + (self.lambda_corr_fast - self.lambda_corr) * w_t
             self._S_t = lam_c * self._S_t + w_t
-            self._M_t *= lam_c
-            if w_t > 0.0 and n_obs > 0:
-                # np.multiply.outer with out= avoids the temporary that
-                # np.outer otherwise allocates each step.
-                np.multiply.outer(z_t, z_t, out=self._scratch_outer)
-                self._M_t += w_t * self._scratch_outer
-            self._cov, self._corr = self._extract(vol_t)
+            if add:
+                # Q <- (1 - eta) Q + eta zz'. With w_t = 0 both S and M decay
+                # by lam_c, so Q is unchanged — no matrix work at all.
+                eta = w_t / self._S_t
+                self._Q_t *= 1.0 - eta
+                self._scratch_outer *= eta
+                self._Q_t += self._scratch_outer
         else:
             # ── Scale-free ladder (Mode A) ──
-            # Update K correlation accumulators on the geometric half-life
-            # ladder; normalise and adaptively shrink each against its OWN
-            # effective sample size, then blend the per-scale covariances.
-            add = w_t > 0.0 and n_obs > 0
-            if add:
-                np.multiply.outer(z_t, z_t, out=self._scratch_outer)
-            cov = np.zeros((n, n), dtype=np.float64)
+            # Update K normalised correlation states on the geometric
+            # half-life ladder; extraction (normalise + shrink + blend)
+            # happens lazily in _materialize().
             s_eff = 0.0
             for k in range(self._corr_lam.size):
                 self._S_list[k] = self._corr_lam[k] * self._S_list[k] + w_t
-                self._M_list[k] *= self._corr_lam[k]
                 if add:
-                    self._M_list[k] += w_t * self._scratch_outer
-                cov_k, _ = self._extract(vol_t, self._M_list[k], self._S_list[k])
-                cov += self._corr_w[k] * cov_k
+                    eta = w_t / self._S_list[k]
+                    self._Q_list[k] *= 1.0 - eta
+                    np.multiply(self._scratch_outer, eta, out=self._scratch_had)
+                    self._Q_list[k] += self._scratch_had
                 s_eff += self._corr_w[k] * self._S_list[k]
-            cov = 0.5 * (cov + cov.T)
-            self._cov = cov
             self._S_t = s_eff                        # blended effective size (for the property)
-            d = np.sqrt(np.maximum(np.diagonal(cov), eps))
-            self._corr = cov / np.outer(d, d)
-            np.fill_diagonal(self._corr, 1.0)
+        self._vol_t = vol_t
+        self._dirty = True
+        if self._adaptive:
+            self._materialize_adaptive()
         self._last_weight = w_t
         return w_t
 
     def get_cov(self) -> np.ndarray:
         """Return the current covariance matrix estimate (n x n)."""
-        if self._cov is None:
+        if self._vol_t is None:
             raise RuntimeError("Call update() at least once before get_cov().")
+        if self._dirty:
+            self._materialize()
         return self._cov.copy()
 
     def get_corr(self) -> np.ndarray:
         """Return the current correlation matrix estimate (n x n)."""
-        if self._corr is None:
+        if self._vol_t is None:
             raise RuntimeError("Call update() at least once before get_corr().")
+        if self._dirty:
+            self._materialize()
         return self._corr.copy()
 
     @property
@@ -393,6 +536,17 @@ class SqueezeKernelEstimator:
         """Kernel weight assigned to the most recent observation."""
         return self._last_weight
 
+    @property
+    def usable_mask(self) -> np.ndarray:
+        """Boolean mask of assets with at least ``min_obs`` observations.
+
+        All-True when ``min_obs`` is None. Purely diagnostic — estimates
+        are not affected; subset the outputs with it (see class docstring).
+        """
+        if self.min_obs is None:
+            return np.ones(self.n_assets, dtype=bool)
+        return self._obs_count >= self.min_obs
+
     @property
     def effective_sample_size(self) -> float:
         """Kernel-weighted effective sample size S_t."""
@@ -432,9 +586,13 @@ class SqueezeKernelEstimator:
     # ── Private helpers ───────────────────────────────────────────────────
 
     def _impute(self, z_t: np.ndarray, finite: np.ndarray) -> None:
+        if self._Q_t is None:
+            # Ladder mode: imputation reads the single-scale state, which
+            # was never updated on this path — historically a silent no-op
+            # (all correlations below threshold); keep it an explicit one.
+            return
         eps = self.epsilon
-        denom = max(self._S_t, eps)
-        sigma_z = self._M_t / denom
+        sigma_z = self._Q_t
         diag_z = np.diag(sigma_z)
         inv_diag = 1.0 / np.sqrt(np.maximum(diag_z, eps))
         missing = ~finite
@@ -453,24 +611,22 @@ class SqueezeKernelEstimator:
             if den > 0.0:
                 z_t[i] = num / den
 
-    def _extract(self, vol_t: np.ndarray, M_t=None, S_in=None) -> tuple[np.ndarray, np.ndarray]:
+    def _shrunk_corr_from_Q(self, Q: np.ndarray, S_t: float) -> np.ndarray:
+        """Normalise one Q state to a correlation and shrink it in place.
+
+        Returns ``self._scratch_corr`` — valid only until the next call.
+        """
         eps = self.epsilon
-        M_t = self._M_t if M_t is None else M_t
-        S_t = max(self._S_t if S_in is None else S_in, eps)
         n = self.n_assets
+        S_t = max(S_t, eps)
 
-        # Normalised standardised covariance matrix sigma_z = M_t / S_t.
-        # Compute correlations directly into the cached scratch buffer to
-        # avoid two intermediate allocations (sigma_z and corr).
-        diag_z = np.diagonal(M_t).copy()
-        diag_z /= S_t                                # in-place
+        # corr_ij = Q_ij * inv_diag_i * inv_diag_j; the scalar S_t cancels
+        # in the normalisation, so Q needs no rescaling pass.
+        diag_z = np.diagonal(Q).copy()
         inv_diag = 1.0 / np.sqrt(np.maximum(diag_z, eps))
-        # corr_ij = (M_ij / S_t) * inv_diag_i * inv_diag_j; this writes
-        # the rescaled outer-product into _scratch_corr in one pass.
         np.multiply.outer(inv_diag, inv_diag, out=self._scratch_corr)
         corr = self._scratch_corr
-        corr *= M_t                                  # in-place; corr = sigma_z * outer(inv_diag,inv_diag)
-        corr *= (1.0 / S_t)                          # absorb the M_t / S_t scale
+        corr *= Q                                    # in-place
         np.fill_diagonal(corr, np.where(diag_z > eps, 1.0, 0.0))
 
         # Adaptive shrinkage: blend toward the equicorrelation target
@@ -484,6 +640,9 @@ class SqueezeKernelEstimator:
             # Off-diagonal mean: O(n^2) sum, no mask allocation.
             rho_bar = (corr.sum() - corr.trace()) / self._n_off
             if self.shrinkage_target == "equicorrelation" or rho_bar <= 0.0:
+                # rho_bar <= 0 also covers the q = meanoff(C o C) = 0 corner:
+                # C o C has nonnegative entries, so q = 0 forces C = I and
+                # hence rho_bar = 0 — the equicorrelation fallback applies.
                 corr *= (1.0 - alpha)
                 corr += alpha * rho_bar
                 np.fill_diagonal(corr, 1.0)
@@ -495,27 +654,82 @@ class SqueezeKernelEstimator:
                 # level-matched so the target carries the same average
                 # correlation mass as the equicorrelation target.  As
                 # alpha -> 0 this reduces exactly to the published estimator.
-                had = corr * corr                       # Hadamard square, O(n^2)
+                had = self._scratch_had
+                np.multiply(corr, corr, out=had)        # Hadamard square, O(n^2)
                 mean_off = (had.sum() - np.trace(had)) / self._n_off
                 gamma = min(1.0, rho_bar / max(mean_off, eps))
                 corr *= (1.0 - alpha)
                 corr += (alpha * (1.0 - alpha)) * rho_bar
-                corr += (alpha * alpha * gamma) * had
+                had *= alpha * alpha * gamma
+                corr += had
                 np.fill_diagonal(corr, 1.0)
+        return corr
 
-        # Build covariance from corr and vol_t.  We need an output array
-        # that the caller can keep, so allocate one cov here (cannot reuse
-        # corr buffer because both are returned).
-        cov = corr * np.outer(vol_t, vol_t)
-        # Defensive symmetrisation against floating-point asymmetry.
-        cov += cov.T
-        cov *= 0.5
-        # Return a copy of corr so callers see a stable snapshot even if
-        # the next update() overwrites the scratch buffer.
-        corr_out = corr.copy()
-        corr_out += corr_out.T
-        corr_out *= 0.5
-        return cov, corr_out
+    def _materialize_adaptive(self) -> None:
+        """Adaptive-weight extraction: build per-rung shrunk covariances
+        (kept for the next update's detector scores), blend with the
+        CUSUM-tilted weights, derive _cov/_corr. Runs eagerly."""
+        eps = self.epsilon
+        vol_t = self._vol_t
+        vv = np.multiply.outer(vol_t, vol_t)
+        sig_k = []
+        for k in range(self._corr_lam.size):
+            corr_k = self._shrunk_corr_from_Q(self._Q_list[k], self._S_list[k])
+            sig_k.append(corr_k * vv)
+        self._aw_prev_sig = sig_k
+        t = self._aw_tilt
+        pi = self._corr_w
+        if t >= 0:
+            w = (1.0 - t) * pi + t * self._aw_pi_fast
+        else:
+            w = (1.0 + t) * pi + (-t) * self._aw_pi_slow
+        cov = w[0] * sig_k[0]
+        for k in range(1, len(sig_k)):
+            cov = cov + w[k] * sig_k[k]
+        cov = (cov + cov.T) * 0.5
+        self._cov = cov
+        d = np.sqrt(np.maximum(np.diagonal(cov), eps))
+        self._corr = cov / np.outer(d, d)
+        np.fill_diagonal(self._corr, 1.0)
+        self._dirty = False
+
+    def _materialize(self) -> None:
+        """Extract _cov/_corr from the current state (lazy, on demand)."""
+        eps = self.epsilon
+        vol_t = self._vol_t
+        if self._corr_lam is None:
+            # ── Single scale: shrunk correlation IS the correlation output ──
+            corr = self._shrunk_corr_from_Q(self._Q_t, self._S_t)
+            np.multiply.outer(vol_t, vol_t, out=self._scratch_outer)
+            cov = corr * self._scratch_outer
+            cov += cov.T
+            cov *= 0.5
+            corr_out = corr.copy()
+            corr_out += corr_out.T
+            corr_out *= 0.5
+            self._cov, self._corr = cov, corr_out
+        else:
+            # ── Ladder: blend per-rung shrunk correlations, then apply the
+            # (shared) volatilities once — algebraically identical to
+            # blending per-rung covariances, K-1 fewer O(n^2) passes.
+            mix = np.zeros((self.n_assets, self.n_assets), dtype=np.float64)
+            for k in range(self._corr_lam.size):
+                corr_k = self._shrunk_corr_from_Q(self._Q_list[k], self._S_list[k])
+                corr_k *= self._corr_w[k]
+                mix += corr_k
+            cov = mix
+            np.multiply.outer(vol_t, vol_t, out=self._scratch_outer)
+            cov *= self._scratch_outer
+            cov += cov.T
+            cov *= 0.5
+            self._cov = cov
+            # Correlation is re-derived from the blended covariance (not the
+            # blended correlation mix) to keep the published dead-asset
+            # semantics: rows of never-observed assets renormalise to zero.
+            d = np.sqrt(np.maximum(np.diagonal(cov), eps))
+            self._corr = cov / np.outer(d, d)
+            np.fill_diagonal(self._corr, 1.0)
+        self._dirty = False
 
 
 # ── Kernel resolution ─────────────────────────────────────────────────────────