squeeze-kernel 0.2.0__tar.gz

squeeze_kernel-0.2.0/PKG-INFO

@@ -0,0 +1,164 @@
+Metadata-Version: 2.4
+Name: squeeze-kernel
+Version: 0.2.0
+Summary: Streaming, PSD-by-construction covariance estimator with Fisher-kernel weighting and adaptive shrinkage
+Keywords: covariance,correlation,ewma,kernel,risk,streaming
+Author: Robert Kende
+License-Expression: MIT
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: Topic :: Scientific/Engineering :: Mathematics
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Operating System :: OS Independent
+Requires-Dist: numpy>=1.24
+Requires-Dist: pytest>=7.0 ; extra == 'dev'
+Requires-Dist: ruff>=0.7 ; extra == 'dev'
+Requires-Dist: scipy>=1.11 ; extra == 'dev'
+Requires-Dist: scipy>=1.11 ; extra == 'full'
+Requires-Python: >=3.10
+Project-URL: Homepage, https://github.com/r0k3/squeeze-kernel
+Project-URL: Repository, https://github.com/r0k3/squeeze-kernel
+Project-URL: Issues, https://github.com/r0k3/squeeze-kernel/issues
+Provides-Extra: dev
+Provides-Extra: full
+Description-Content-Type: text/markdown
+
+# Squeeze Kernel Covariance Estimator
+
+[![CI](https://github.com/r0k3/squeeze-kernel/actions/workflows/ci.yml/badge.svg)](https://github.com/r0k3/squeeze-kernel/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/squeeze-kernel.svg)](https://pypi.org/project/squeeze-kernel/)
+[![Python](https://img.shields.io/pypi/pyversions/squeeze-kernel.svg)](https://pypi.org/project/squeeze-kernel/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
+
+A **streaming covariance estimator for panels of daily financial returns**. One `O(n²)` update per day, positive semi-definite **by construction** at every step, missing values handled **natively**, and defaults that require no tuning. Only dependency: NumPy.
+
+Reference: *"The Squeeze Kernel Covariance Estimator: Dual-Timescale Tracking with Adaptive Shrinkage"* (Kende, 2026).
+
+## Why
+
+Rolling-window estimators (Ledoit–Wolf, nonlinear shrinkage, RMT denoising) refit over a fixed window each day and cannot adapt within it; multivariate GARCH (DCC) adapts but needs multi-stage estimation and a fragile news coefficient. The Squeeze Kernel is a single streaming recursion that:
+
+- **is PSD at every step, structurally** — never needs eigenvalue clipping, nearest-PSD projection, or a solver;
+- **adapts fastest exactly when it matters** — a Fisher-information kernel up-weights high-dispersion (stress) days, when correlation regimes actually move;
+- **regularises itself** — an adaptive equicorrelation shrinkage activates automatically as the asset count approaches the effective sample size, with a provable condition-number bound;
+- **ingests missing values natively** — listings, delistings, and halts enter as `NaN`; no imputation or complete-case subsetting;
+- **is fast** — a full 30-year daily pass takes ~0.75 s at n=100 and ~3.4 s at n=300 (single-threaded), 30–40× faster than rolling-window baselines at scale.
+
+On a 30-year S&P 500 panel (n=100, ~7,600 out-of-sample days) it statistically ties DCC on one-step density forecasts and beats EWMA, Ledoit–Wolf, OAS, nonlinear shrinkage, RMT denoising, and the Gerber statistic — and it is the only method in the 90% model confidence set together with DCC. At n=300 it leads every competitor that remains statistically viable.
+
+## Installation
+
+```bash
+pip install squeeze-kernel          # NumPy only
+pip install "squeeze-kernel[full]"  # + SciPy (kappa calibration, chi² kernel)
+```
+
+## Quickstart
+
+```python
+import numpy as np
+from squeeze_kernel import SqueezeKernelEstimator
+
+# daily_returns: array of shape (T, n) — may contain NaN for missing assets
+est = SqueezeKernelEstimator(n_assets=daily_returns.shape[1])
+
+for r_t in daily_returns:          # stream one day at a time
+    est.update(r_t)
+
+cov  = est.get_cov()               # (n, n) covariance, PSD by construction
+corr = est.get_corr()              # (n, n) correlation
+```
+
+That is the whole API for most uses. The defaults (`lambda_vol=0.98`, `lambda_corr=0.996`, `kappa=0.25`) are the paper-recommended settings for daily returns, selected by time-series cross-validation and robust across a 50× parameter sweep — deploy them as-is.
+
+Batch mode, if you prefer the full path in one call:
+
+```python
+from squeeze_kernel import estimate_squeeze_cov
+
+cov_path, corr_path, weights = estimate_squeeze_cov(daily_returns, with_weights=True)
+# cov_path: (T, n, n) — the estimate after each day
+```
+
+A complete runnable walkthrough (streaming, missing data, batch) is in [`examples/quickstart.py`](examples/quickstart.py).
+
+## Missing values
+
+Pass `NaN` for any asset not observed on a given day — nothing else to do:
+
+```python
+r_t = np.array([0.004, np.nan, -0.011])   # asset 2 not trading today
+est.update(r_t)                            # PSD preserved, no imputation
+```
+
+## Parameters
+
+| Parameter | Default | Meaning |
+|---|---|---|
+| `lambda_vol` | `0.98` | volatility EWMA decay (half-life ≈ 34 days) |
+| `lambda_corr` | `0.996` | correlation EWMA decay (half-life ≈ 173 days, T_eff ≈ 250) |
+| `kappa` | `0.25` | Fisher kernel saturation; higher = stronger calm-day filtering |
+| `shrinkage` | `"auto"` | adaptive equicorrelation shrinkage (`"none"` or a float to override) |
+| `shrinkage_delta` | `0.10` | concentration threshold at which shrinkage activates |
+
+Useful read-only state after each `update()`: `est.weight` (last kernel weight), `est.effective_sample_size` (kernel-weighted T_eff), `est.shrinkage_intensity` (current α).
+
+To recalibrate `kappa` for a different asset class (requires the `full` extra):
+
+```python
+kappa = SqueezeKernelEstimator.calibrate_kappa(burn_in_returns, target_weight=0.6)
+```
+
+## Advanced options
+
+**Score-exact weighting** (`weight_statistic="mahalanobis"`, use with `kappa=1.0`): drives the kernel with the Mahalanobis surprise `z'C⁻¹z/N` against the estimator's own correlation instead of the marginal dispersion. Improves accuracy in the moderate-concentration regime — use only when `n / T_eff ≲ 0.5` (e.g. n ≤ 100 at the default `lambda_corr`); at higher concentration the estimated inverse degrades it and the default is strictly better.
+
+```python
+est = SqueezeKernelEstimator(n_assets=100, kappa=1.0, weight_statistic="mahalanobis")
+```
+
+**Score-driven memory** (`lambda_corr_fast=0.99`): lets stress days also *shorten* the correlation memory (decay slides from `lambda_corr` toward `lambda_corr_fast` as the kernel weight rises). Do **not** combine with the Mahalanobis option — they act on the same channel and the combination degrades accuracy.
+
+**Alternative kernels**: pass `kernel_fn=kernel_exponential` (with `kernel_kwargs={"gamma": ...}`) or `kernel_chi2_cdf`, or any callable `(d2, *, n_observed, **kw) -> float` mapping to `[0, 1)`. The PSD guarantee holds for any such kernel.
+
+## How it works
+
+Three mechanisms in one recursion:
+
+1. **Dual-timescale EWMA** — fast per-asset volatility (`lambda_vol`) is separated from slow correlation dynamics (`lambda_corr`), so variance shocks don't contaminate the correlation estimate.
+2. **Fisher kernel weighting** — each day's standardized outer product enters with weight `w = d²/(d² + kappa)`, where `d²` is the mean squared standardized return: calm days contribute little, dispersion shocks contribute fully.
+3. **Adaptive equicorrelation shrinkage** — `alpha = min(1, max(0, n/(2·S) − delta))` blends toward an equicorrelation target using the estimator's own kernel-weighted sample size `S`; it is a no-op at low dimension and provides provably bounded conditioning at high dimension.
+
+The complete update is a natural-gradient step on the Gaussian log-likelihood, with the kernel weight acting as an adaptive Riemannian learning rate (paper, Appendix B).
+
+## Development
+
+```bash
+uv sync --extra full --extra dev
+uv run python -m pytest        # test suite
+uv run python -m ruff check .  # lint
+uv build                       # build sdist + wheel
+```
+
+Releases: publishing a GitHub release from a `v*` tag triggers the [publish workflow](.github/workflows/publish.yml), which builds and uploads to PyPI via trusted publishing.
+
+## Citation
+
+```bibtex
+@article{kende2026squeeze,
+  title  = {The Squeeze Kernel Covariance Estimator: Dual-Timescale Tracking with Adaptive Shrinkage},
+  author = {Kende, Robert},
+  year   = {2026}
+}
+```
+
+See also [`CITATION.cff`](CITATION.cff).
+
+## License
+
+MIT

squeeze_kernel-0.2.0/README.md

@@ -0,0 +1,134 @@
+# Squeeze Kernel Covariance Estimator
+
+[![CI](https://github.com/r0k3/squeeze-kernel/actions/workflows/ci.yml/badge.svg)](https://github.com/r0k3/squeeze-kernel/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/squeeze-kernel.svg)](https://pypi.org/project/squeeze-kernel/)
+[![Python](https://img.shields.io/pypi/pyversions/squeeze-kernel.svg)](https://pypi.org/project/squeeze-kernel/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
+
+A **streaming covariance estimator for panels of daily financial returns**. One `O(n²)` update per day, positive semi-definite **by construction** at every step, missing values handled **natively**, and defaults that require no tuning. Only dependency: NumPy.
+
+Reference: *"The Squeeze Kernel Covariance Estimator: Dual-Timescale Tracking with Adaptive Shrinkage"* (Kende, 2026).
+
+## Why
+
+Rolling-window estimators (Ledoit–Wolf, nonlinear shrinkage, RMT denoising) refit over a fixed window each day and cannot adapt within it; multivariate GARCH (DCC) adapts but needs multi-stage estimation and a fragile news coefficient. The Squeeze Kernel is a single streaming recursion that:
+
+- **is PSD at every step, structurally** — never needs eigenvalue clipping, nearest-PSD projection, or a solver;
+- **adapts fastest exactly when it matters** — a Fisher-information kernel up-weights high-dispersion (stress) days, when correlation regimes actually move;
+- **regularises itself** — an adaptive equicorrelation shrinkage activates automatically as the asset count approaches the effective sample size, with a provable condition-number bound;
+- **ingests missing values natively** — listings, delistings, and halts enter as `NaN`; no imputation or complete-case subsetting;
+- **is fast** — a full 30-year daily pass takes ~0.75 s at n=100 and ~3.4 s at n=300 (single-threaded), 30–40× faster than rolling-window baselines at scale.
+
+On a 30-year S&P 500 panel (n=100, ~7,600 out-of-sample days) it statistically ties DCC on one-step density forecasts and beats EWMA, Ledoit–Wolf, OAS, nonlinear shrinkage, RMT denoising, and the Gerber statistic — and it is the only method in the 90% model confidence set together with DCC. At n=300 it leads every competitor that remains statistically viable.
+
+## Installation
+
+```bash
+pip install squeeze-kernel          # NumPy only
+pip install "squeeze-kernel[full]"  # + SciPy (kappa calibration, chi² kernel)
+```
+
+## Quickstart
+
+```python
+import numpy as np
+from squeeze_kernel import SqueezeKernelEstimator
+
+# daily_returns: array of shape (T, n) — may contain NaN for missing assets
+est = SqueezeKernelEstimator(n_assets=daily_returns.shape[1])
+
+for r_t in daily_returns:          # stream one day at a time
+    est.update(r_t)
+
+cov  = est.get_cov()               # (n, n) covariance, PSD by construction
+corr = est.get_corr()              # (n, n) correlation
+```
+
+That is the whole API for most uses. The defaults (`lambda_vol=0.98`, `lambda_corr=0.996`, `kappa=0.25`) are the paper-recommended settings for daily returns, selected by time-series cross-validation and robust across a 50× parameter sweep — deploy them as-is.
+
+Batch mode, if you prefer the full path in one call:
+
+```python
+from squeeze_kernel import estimate_squeeze_cov
+
+cov_path, corr_path, weights = estimate_squeeze_cov(daily_returns, with_weights=True)
+# cov_path: (T, n, n) — the estimate after each day
+```
+
+A complete runnable walkthrough (streaming, missing data, batch) is in [`examples/quickstart.py`](examples/quickstart.py).
+
+## Missing values
+
+Pass `NaN` for any asset not observed on a given day — nothing else to do:
+
+```python
+r_t = np.array([0.004, np.nan, -0.011])   # asset 2 not trading today
+est.update(r_t)                            # PSD preserved, no imputation
+```
+
+## Parameters
+
+| Parameter | Default | Meaning |
+|---|---|---|
+| `lambda_vol` | `0.98` | volatility EWMA decay (half-life ≈ 34 days) |
+| `lambda_corr` | `0.996` | correlation EWMA decay (half-life ≈ 173 days, T_eff ≈ 250) |
+| `kappa` | `0.25` | Fisher kernel saturation; higher = stronger calm-day filtering |
+| `shrinkage` | `"auto"` | adaptive equicorrelation shrinkage (`"none"` or a float to override) |
+| `shrinkage_delta` | `0.10` | concentration threshold at which shrinkage activates |
+
+Useful read-only state after each `update()`: `est.weight` (last kernel weight), `est.effective_sample_size` (kernel-weighted T_eff), `est.shrinkage_intensity` (current α).
+
+To recalibrate `kappa` for a different asset class (requires the `full` extra):
+
+```python
+kappa = SqueezeKernelEstimator.calibrate_kappa(burn_in_returns, target_weight=0.6)
+```
+
+## Advanced options
+
+**Score-exact weighting** (`weight_statistic="mahalanobis"`, use with `kappa=1.0`): drives the kernel with the Mahalanobis surprise `z'C⁻¹z/N` against the estimator's own correlation instead of the marginal dispersion. Improves accuracy in the moderate-concentration regime — use only when `n / T_eff ≲ 0.5` (e.g. n ≤ 100 at the default `lambda_corr`); at higher concentration the estimated inverse degrades it and the default is strictly better.
+
+```python
+est = SqueezeKernelEstimator(n_assets=100, kappa=1.0, weight_statistic="mahalanobis")
+```
+
+**Score-driven memory** (`lambda_corr_fast=0.99`): lets stress days also *shorten* the correlation memory (decay slides from `lambda_corr` toward `lambda_corr_fast` as the kernel weight rises). Do **not** combine with the Mahalanobis option — they act on the same channel and the combination degrades accuracy.
+
+**Alternative kernels**: pass `kernel_fn=kernel_exponential` (with `kernel_kwargs={"gamma": ...}`) or `kernel_chi2_cdf`, or any callable `(d2, *, n_observed, **kw) -> float` mapping to `[0, 1)`. The PSD guarantee holds for any such kernel.
+
+## How it works
+
+Three mechanisms in one recursion:
+
+1. **Dual-timescale EWMA** — fast per-asset volatility (`lambda_vol`) is separated from slow correlation dynamics (`lambda_corr`), so variance shocks don't contaminate the correlation estimate.
+2. **Fisher kernel weighting** — each day's standardized outer product enters with weight `w = d²/(d² + kappa)`, where `d²` is the mean squared standardized return: calm days contribute little, dispersion shocks contribute fully.
+3. **Adaptive equicorrelation shrinkage** — `alpha = min(1, max(0, n/(2·S) − delta))` blends toward an equicorrelation target using the estimator's own kernel-weighted sample size `S`; it is a no-op at low dimension and provides provably bounded conditioning at high dimension.
+
+The complete update is a natural-gradient step on the Gaussian log-likelihood, with the kernel weight acting as an adaptive Riemannian learning rate (paper, Appendix B).
+
+## Development
+
+```bash
+uv sync --extra full --extra dev
+uv run python -m pytest        # test suite
+uv run python -m ruff check .  # lint
+uv build                       # build sdist + wheel
+```
+
+Releases: publishing a GitHub release from a `v*` tag triggers the [publish workflow](.github/workflows/publish.yml), which builds and uploads to PyPI via trusted publishing.
+
+## Citation
+
+```bibtex
+@article{kende2026squeeze,
+  title  = {The Squeeze Kernel Covariance Estimator: Dual-Timescale Tracking with Adaptive Shrinkage},
+  author = {Kende, Robert},
+  year   = {2026}
+}
+```
+
+See also [`CITATION.cff`](CITATION.cff).
+
+## License
+
+MIT

squeeze_kernel-0.2.0/pyproject.toml

@@ -0,0 +1,38 @@
+[build-system]
+requires = ["uv_build>=0.10.6,<0.11.0"]
+build-backend = "uv_build"
+
+[project]
+name = "squeeze-kernel"
+version = "0.2.0"
+description = "Streaming, PSD-by-construction covariance estimator with Fisher-kernel weighting and adaptive shrinkage"
+readme = "README.md"
+license = "MIT"
+requires-python = ">=3.10"
+authors = [{ name = "Robert Kende" }]
+keywords = ["covariance", "correlation", "ewma", "kernel", "risk", "streaming"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Science/Research",
+    "Topic :: Scientific/Engineering :: Mathematics",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Operating System :: OS Independent",
+]
+dependencies = ["numpy>=1.24"]
+
+[project.urls]
+Homepage = "https://github.com/r0k3/squeeze-kernel"
+Repository = "https://github.com/r0k3/squeeze-kernel"
+Issues = "https://github.com/r0k3/squeeze-kernel/issues"
+
+[project.optional-dependencies]
+full = ["scipy>=1.11"]
+dev = ["pytest>=7.0", "ruff>=0.7", "scipy>=1.11"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

squeeze_kernel-0.2.0/src/squeeze_kernel/__init__.py

@@ -0,0 +1,37 @@
+"""
+Squeeze Kernel Covariance Estimator
+====================================
+
+Streaming, PSD-by-construction covariance estimator with adaptive
+equicorrelation shrinkage and pluggable kernel weighting.
+
+Quick start::
+
+    import numpy as np
+    from squeeze_kernel import SqueezeKernelEstimator
+
+    returns = np.random.default_rng(42).normal(0.0, 0.01, size=(250, 3))
+
+    # Defaults (lambda_vol=0.98, lambda_corr=0.996, kappa=0.25) are the
+    # paper-recommended settings for panels of daily financial returns.
+    est = SqueezeKernelEstimator(n_assets=3)
+    for r_t in returns:
+        est.update(r_t)
+
+    cov = est.get_cov()
+    corr = est.get_corr()
+"""
+
+from squeeze_kernel.estimator import SqueezeKernelEstimator
+from squeeze_kernel.kernels import kernel_fisher, kernel_exponential, kernel_chi2_cdf
+from squeeze_kernel.batch import estimate_squeeze_cov
+
+__all__ = [
+    "SqueezeKernelEstimator",
+    "estimate_squeeze_cov",
+    "kernel_fisher",
+    "kernel_exponential",
+    "kernel_chi2_cdf",
+]
+
+__version__ = "0.2.0"

squeeze_kernel-0.2.0/src/squeeze_kernel/batch.py

@@ -0,0 +1,81 @@
+"""Batch estimation over an entire returns panel."""
+
+from __future__ import annotations
+
+import numpy as np
+
+from squeeze_kernel.estimator import SqueezeKernelEstimator
+from squeeze_kernel.kernels import KernelFn
+
+
+def estimate_squeeze_cov(
+    returns,
+    *,
+    lambda_vol: float = 0.98,
+    lambda_corr: float = 0.996,
+    kappa: float | None = None,
+    kernel_fn: KernelFn | None = None,
+    kernel_kwargs: dict[str, object] | None = None,
+    epsilon: float = 1e-8,
+    shrinkage: str | float = "auto",
+    shrinkage_delta: float = 0.10,
+    impute_missing: bool = False,
+    impute_threshold: float = 0.6,
+    with_corr: bool = True,
+    with_weights: bool = False,
+) -> tuple[np.ndarray, np.ndarray | None, np.ndarray | None]:
+    """Estimate streaming covariance over an entire returns panel.
+
+    Parameters
+    ----------
+    returns : array-like, shape (T, n)
+        2D return matrix.  May contain NaN for missing observations.
+    kappa : float, optional
+        Saturation parameter for the default Fisher kernel.
+    kernel_fn : callable, optional
+        Custom kernel ``(d2, *, n_observed, **kw) -> float``.
+    kernel_kwargs : dict, optional
+        Extra keyword arguments forwarded to ``kernel_fn``.
+    with_corr : bool
+        If True, also return the correlation tensor.
+    with_weights : bool
+        If True, also return per-timestamp kernel weights.
+
+    Returns
+    -------
+    cov : ndarray, shape (T, n, n)
+    corr : ndarray or None, shape (T, n, n)
+    weights : ndarray or None, shape (T,)
+    """
+    values = np.asarray(returns, dtype=np.float64)
+    if values.ndim != 2:
+        raise ValueError(f"Expected 2D returns, got shape {values.shape}.")
+    t_total, n_assets = values.shape
+
+    est = SqueezeKernelEstimator(
+        n_assets,
+        lambda_vol=lambda_vol,
+        lambda_corr=lambda_corr,
+        kappa=kappa,
+        kernel_fn=kernel_fn,
+        kernel_kwargs=kernel_kwargs,
+        epsilon=epsilon,
+        shrinkage=shrinkage,
+        shrinkage_delta=shrinkage_delta,
+        impute_missing=impute_missing,
+        impute_threshold=impute_threshold,
+    )
+
+    cov = np.empty((t_total, n_assets, n_assets), dtype=np.float64)
+    corr = np.empty_like(cov) if with_corr else None
+    weights = np.empty(t_total, dtype=np.float64) if with_weights else None
+
+    for t in range(t_total):
+        w_t = est.update(values[t])
+        cov[t] = est.get_cov()
+        if corr is not None:
+            corr[t] = est.get_corr()
+        if weights is not None:
+            weights[t] = w_t
+
+    return cov, corr, weights

squeeze_kernel-0.2.0/src/squeeze_kernel/estimator.py

@@ -0,0 +1,380 @@
+"""Core streaming Squeeze Kernel covariance estimator."""
+
+from __future__ import annotations
+
+import numpy as np
+
+from squeeze_kernel.kernels import (
+    KernelFn, kernel_fisher, calibrate_kappa, extract_d2_series,
+)
+
+
+class SqueezeKernelEstimator:
+    """Streaming robust covariance estimator with pluggable kernel weighting.
+
+    The estimator is positive semi-definite by construction at every time step,
+    handles missing observations natively, and separates volatility and
+    correlation dynamics through dual-timescale EWMAs.  An adaptive
+    equicorrelation shrinkage rule automatically calibrates regularisation
+    to the concentration ratio n / T_eff.
+
+    Parameters
+    ----------
+    n_assets : int
+        Number of assets.
+    lambda_vol : float
+        Decay factor for per-asset volatility EWMA (default 0.98,
+        half-life ≈ 34 trading days).
+    lambda_corr : float
+        Decay factor for correlation EWMA (default 0.996, half-life
+        ≈ 173 trading days, effective sample size ≈ 250).
+    kappa : float, optional
+        Saturation parameter for the default Fisher kernel (default 0.25).
+        The defaults for ``lambda_vol``, ``lambda_corr`` and ``kappa`` are
+        the values recommended in the paper for panels of daily financial
+        returns; they were selected by time-series cross-validation and are
+        insensitive to moderate perturbation.
+    kernel_fn : callable, optional
+        Custom kernel ``(d2, *, n_observed, **kw) -> float``.
+        If omitted, the estimator uses ``kernel_fisher``.
+    kernel_kwargs : dict, optional
+        Extra keyword arguments forwarded to ``kernel_fn``.
+        Use this to configure alternative kernels such as
+        ``kernel_exponential(gamma=...)``.
+    epsilon : float
+        Numerical floor (default 1e-8).
+    shrinkage : str or float
+        ``'auto'`` (default) for adaptive shrinkage,
+        ``0`` or ``'none'`` to disable, or a float in [0, 1] for fixed intensity.
+    shrinkage_delta : float
+        Threshold for adaptive shrinkage (default 0.10).
+    impute_missing : bool
+        If True, impute missing standardized returns from correlated assets.
+    impute_threshold : float
+        Minimum |correlation| for imputation donors (default 0.6).
+    weight_statistic : str
+        Statistic fed to the kernel.  ``'marginal'`` (default) uses the mean
+        squared standardized return d² = z'z/N — the published estimator.
+        ``'mahalanobis'`` uses the score-exact surprise z'C⁻¹z/N measured
+        against the estimator's own previous correlation matrix (one linear
+        solve per update).  With κ ≈ 1 (the parameter-free default, since
+        E[z'C⁻¹z/N] = 1 under a correct C) this improved one-step NLL by
+        ≈2.5 points on the S&P-500 n=100 benchmark, holdout-confirmed.
+        IMPORTANT — regime-dependent: the advantage inverts in the
+        high-concentration regime (≈12 NLL worse at n=200 and ≈61 worse at
+        n=300 with T_eff = 250), because the estimated inverse inflates the
+        statistic as n approaches T_eff, saturating the kernel and weakening
+        the adaptive shrinkage.  Use only when n/T_eff ≲ 0.5; the marginal
+        default is the concentration-robust choice at every dimension.
+    lambda_corr_fast : float or None
+        If set, enables score-driven memory: the correlation decay becomes
+        λ_t = lambda_corr + (lambda_corr_fast − lambda_corr)·w_t, shortening
+        the memory on high-weight (stress) days.  PSD is preserved.  Pass
+        ``None`` (default) for the published constant-λ behaviour.
+        Do not combine with ``weight_statistic='mahalanobis'`` — the two
+        mechanisms act on the same reactivity channel and their combination
+        degraded out-of-sample accuracy in testing.
+
+    Examples
+    --------
+    >>> import numpy as np
+    >>> returns = np.random.default_rng(42).normal(0.0, 0.01, size=(250, 3))
+    >>> est = SqueezeKernelEstimator(n_assets=3)
+    >>> for r_t in returns:
+    ...     est.update(r_t)
+    >>> cov = est.get_cov()
+    >>> corr = est.get_corr()
+    """
+
+    def __init__(
+        self,
+        n_assets: int,
+        *,
+        lambda_vol: float = 0.98,
+        lambda_corr: float = 0.996,
+        kappa: float | None = None,
+        kernel_fn: KernelFn | None = None,
+        kernel_kwargs: dict[str, object] | None = None,
+        epsilon: float = 1e-8,
+        shrinkage: str | float = "auto",
+        shrinkage_delta: float = 0.10,
+        impute_missing: bool = False,
+        impute_threshold: float = 0.6,
+        weight_statistic: str = "marginal",
+        lambda_corr_fast: float | None = None,
+    ):
+        self.n_assets = n_assets
+        self.lambda_vol = lambda_vol
+        self.lambda_corr = lambda_corr
+        self.epsilon = epsilon
+        self.impute_missing = impute_missing
+        self.impute_threshold = impute_threshold
+        self.shrinkage_delta = shrinkage_delta
+
+        # Opt-in extensions (defaults preserve the published estimator exactly)
+        if weight_statistic not in ("marginal", "mahalanobis"):
+            raise ValueError("weight_statistic must be 'marginal' or 'mahalanobis'.")
+        self.weight_statistic = weight_statistic
+        if lambda_corr_fast is not None and not (0.0 < lambda_corr_fast < 1.0):
+            raise ValueError("lambda_corr_fast must be in (0, 1).")
+        self.lambda_corr_fast = lambda_corr_fast
+
+        # Resolve shrinkage
+        if isinstance(shrinkage, str):
+            self._shrinkage_alpha = -1.0 if shrinkage == "auto" else 0.0
+        else:
+            self._shrinkage_alpha = float(shrinkage)
+
+        # Resolve kernel
+        self._kernel_fn, self._kernel_kwargs = _resolve_kernel(kappa, kernel_fn, kernel_kwargs)
+        self.kappa = self._kernel_kwargs.get("kappa") if self._kernel_fn is kernel_fisher else None
+
+        # State
+        self._var_t: np.ndarray | None = None
+        self._var_init: np.ndarray | None = None
+        self._M_t = np.eye(n_assets, dtype=np.float64) * epsilon
+        self._S_t = float(epsilon)
+        self._cov: np.ndarray | None = None
+        self._corr: np.ndarray | None = None
+        self._last_weight: float = 0.0
+
+        # Cached scratch buffers reused per ``update()`` to avoid per-step
+        # allocator churn. These are intentionally module-private and
+        # never escape the estimator.
+        self._scratch_outer = np.empty((n_assets, n_assets), dtype=np.float64)
+        self._scratch_corr = np.empty((n_assets, n_assets), dtype=np.float64)
+        self._n_off = float(n_assets * (n_assets - 1)) if n_assets > 1 else 1.0
+
+    # ── Public API ────────────────────────────────────────────────────────
+
+    def update(self, r_t) -> float:
+        """Process one return vector and update the covariance estimate.
+
+        Parameters
+        ----------
+        r_t : array-like, shape (n_assets,)
+            Return vector.  May contain NaN for missing assets.
+
+        Returns
+        -------
+        float
+            Kernel weight w_t assigned to this observation.
+        """
+        r_t = np.asarray(r_t, dtype=np.float64)
+        n = self.n_assets
+        eps = self.epsilon
+        if r_t.shape != (n,):
+            raise ValueError(f"Expected shape ({n},), got {r_t.shape}.")
+
+        finite = np.isfinite(r_t)
+
+        # ── Volatility update ──
+        if self._var_t is None:
+            self._var_t = np.zeros(n, dtype=np.float64)
+            self._var_init = np.zeros(n, dtype=bool)
+
+        first = finite & ~self._var_init
+        repeat = finite & self._var_init
+        if np.any(first):
+            self._var_t[first] = r_t[first] ** 2 + eps
+            self._var_init[first] = True
+        if np.any(repeat):
+            self._var_t[repeat] = (
+                self.lambda_vol * self._var_t[repeat]
+                + (1.0 - self.lambda_vol) * r_t[repeat] ** 2
+            )
+
+        vol_t = np.zeros(n, dtype=np.float64)
+        vol_t[self._var_init] = np.sqrt(self._var_t[self._var_init])
+
+        # ── Standardized returns ──
+        z_t = np.zeros(n, dtype=np.float64)
+        n_obs = int(finite.sum())
+        if n_obs > 0:
+            z_t[finite] = r_t[finite] / (vol_t[finite] + eps)
+            d2 = float(z_t[finite] @ z_t[finite]) / n_obs
+            if self.weight_statistic == "mahalanobis" and self._corr is not None:
+                # Score-exact surprise against the estimator's own previous
+                # correlation; falls back to the marginal d² on the first
+                # step or a (rare) singular observed submatrix.
+                try:
+                    c_sub = self._corr[np.ix_(finite, finite)]
+                    d2 = float(z_t[finite] @ np.linalg.solve(c_sub, z_t[finite])) / n_obs
+                except np.linalg.LinAlgError:
+                    pass
+            w_t = self._kernel_fn(d2, n_observed=n_obs, **self._kernel_kwargs)
+        else:
+            w_t = 0.0
+
+        # ── Imputation ──
+        if self.impute_missing and 0 < n_obs < n:
+            self._impute(z_t, finite)
+
+        # ── Correlation EWMA (in-place to avoid per-step allocations) ──
+        lam_c = self.lambda_corr
+        if self.lambda_corr_fast is not None:
+            # Score-driven memory: stress days (w_t → 1) shorten the memory
+            # toward lambda_corr_fast; calm days keep the slow decay.
+            lam_c = self.lambda_corr + (self.lambda_corr_fast - self.lambda_corr) * w_t
+        self._S_t = lam_c * self._S_t + w_t
+        self._M_t *= lam_c
+        if w_t > 0.0 and n_obs > 0:
+            # np.multiply.outer with out= avoids the temporary that
+            # np.outer otherwise allocates each step.
+            np.multiply.outer(z_t, z_t, out=self._scratch_outer)
+            self._M_t += w_t * self._scratch_outer
+
+        # ── Extract covariance ──
+        self._cov, self._corr = self._extract(vol_t)
+        self._last_weight = w_t
+        return w_t
+
+    def get_cov(self) -> np.ndarray:
+        """Return the current covariance matrix estimate (n x n)."""
+        if self._cov is None:
+            raise RuntimeError("Call update() at least once before get_cov().")
+        return self._cov.copy()
+
+    def get_corr(self) -> np.ndarray:
+        """Return the current correlation matrix estimate (n x n)."""
+        if self._corr is None:
+            raise RuntimeError("Call update() at least once before get_corr().")
+        return self._corr.copy()
+
+    @property
+    def weight(self) -> float:
+        """Kernel weight assigned to the most recent observation."""
+        return self._last_weight
+
+    @property
+    def effective_sample_size(self) -> float:
+        """Kernel-weighted effective sample size S_t."""
+        return self._S_t
+
+    @property
+    def shrinkage_intensity(self) -> float:
+        """Current adaptive shrinkage intensity alpha_t."""
+        if self._shrinkage_alpha >= 0:
+            return self._shrinkage_alpha
+        n = self.n_assets
+        return max(0.0, min(1.0, n / (2.0 * max(self._S_t, self.epsilon)) - self.shrinkage_delta))
+
+    @staticmethod
+    def calibrate_kappa(
+        returns, target_weight: float = 0.5, lambda_vol: float = 0.98,
+    ) -> float:
+        """Calibrate κ from burn-in data so E[w_t] ≈ target_weight.
+
+        Parameters
+        ----------
+        returns : array-like, shape (T, n)
+            Burn-in return data.
+        target_weight : float
+            Target average kernel weight in (0, 1).
+        lambda_vol : float
+            Volatility decay factor used for standardization.
+
+        Returns
+        -------
+        float
+            Calibrated κ value.
+        """
+        d2 = extract_d2_series(returns, lambda_vol=lambda_vol)
+        return calibrate_kappa(d2, target_weight)
+
+    # ── Private helpers ───────────────────────────────────────────────────
+
+    def _impute(self, z_t: np.ndarray, finite: np.ndarray) -> None:
+        eps = self.epsilon
+        denom = max(self._S_t, eps)
+        sigma_z = self._M_t / denom
+        diag_z = np.diag(sigma_z)
+        inv_diag = 1.0 / np.sqrt(np.maximum(diag_z, eps))
+        missing = ~finite
+        for i in range(self.n_assets):
+            if not missing[i]:
+                continue
+            num = den = 0.0
+            for j in range(self.n_assets):
+                if not finite[j]:
+                    continue
+                c_ij = sigma_z[i, j] * inv_diag[i] * inv_diag[j]
+                if abs(c_ij) < self.impute_threshold:
+                    continue
+                num += c_ij * z_t[j]
+                den += abs(c_ij)
+            if den > 0.0:
+                z_t[i] = num / den
+
+    def _extract(self, vol_t: np.ndarray) -> tuple[np.ndarray, np.ndarray]:
+        eps = self.epsilon
+        S_t = max(self._S_t, eps)
+        n = self.n_assets
+
+        # Normalised standardised covariance matrix sigma_z = M_t / S_t.
+        # Compute correlations directly into the cached scratch buffer to
+        # avoid two intermediate allocations (sigma_z and corr).
+        diag_z = np.diagonal(self._M_t).copy()
+        diag_z /= S_t                                # in-place
+        inv_diag = 1.0 / np.sqrt(np.maximum(diag_z, eps))
+        # corr_ij = (M_ij / S_t) * inv_diag_i * inv_diag_j; this writes
+        # the rescaled outer-product into _scratch_corr in one pass.
+        np.multiply.outer(inv_diag, inv_diag, out=self._scratch_corr)
+        corr = self._scratch_corr
+        corr *= self._M_t                            # in-place; corr = sigma_z * outer(inv_diag,inv_diag)
+        corr *= (1.0 / S_t)                          # absorb the M_t / S_t scale
+        np.fill_diagonal(corr, np.where(diag_z > eps, 1.0, 0.0))
+
+        # Adaptive shrinkage: blend toward the equicorrelation target
+        # T = (1 - rho_bar) I + rho_bar 11'.  We avoid materialising T by
+        # blending the off-diagonal toward rho_bar in place and resetting
+        # the diagonal to 1.
+        alpha = self._shrinkage_alpha
+        if alpha < 0:
+            alpha = max(0.0, min(1.0, n / (2.0 * S_t) - self.shrinkage_delta))
+        if alpha > 0.0 and n > 1:
+            # Off-diagonal mean: O(n^2) sum, no mask allocation.
+            rho_bar = (corr.sum() - corr.trace()) / self._n_off
+            corr *= (1.0 - alpha)
+            corr += alpha * rho_bar
+            np.fill_diagonal(corr, 1.0)
+
+        # Build covariance from corr and vol_t.  We need an output array
+        # that the caller can keep, so allocate one cov here (cannot reuse
+        # corr buffer because both are returned).
+        cov = corr * np.outer(vol_t, vol_t)
+        # Defensive symmetrisation against floating-point asymmetry.
+        cov += cov.T
+        cov *= 0.5
+        # Return a copy of corr so callers see a stable snapshot even if
+        # the next update() overwrites the scratch buffer.
+        corr_out = corr.copy()
+        corr_out += corr_out.T
+        corr_out *= 0.5
+        return cov, corr_out
+
+
+# ── Kernel resolution ─────────────────────────────────────────────────────────
+
+def _resolve_kernel(
+    kappa: float | None,
+    kernel_fn: KernelFn | None,
+    kernel_kwargs: dict[str, object] | None,
+) -> tuple[KernelFn, dict[str, object]]:
+    kw = dict(kernel_kwargs) if kernel_kwargs else {}
+
+    if kernel_fn is not None:
+        if kappa is not None:
+            raise ValueError(
+                "Pass kernel-specific parameters via kernel_kwargs when kernel_fn is set."
+            )
+        return kernel_fn, kw
+
+    if "kappa" in kw and kappa is not None:
+        raise ValueError("Pass kappa either as a top-level argument or in kernel_kwargs, not both.")
+
+    resolved_kappa = float(kw.get("kappa", 0.25 if kappa is None else kappa))
+    if resolved_kappa <= 0.0:
+        raise ValueError("kappa must be > 0.")
+    kw["kappa"] = resolved_kappa
+    return kernel_fisher, kw

squeeze_kernel-0.2.0/src/squeeze_kernel/kernels.py

@@ -0,0 +1,139 @@
+"""Built-in kernel weight functions for the Squeeze Kernel estimator."""
+
+from __future__ import annotations
+
+import math
+from typing import Callable
+
+KernelFn = Callable[..., float]
+
+
+def kernel_fisher(d2: float, /, *, kappa: float, **kw) -> float:
+    """Fisher information saturation kernel: w = d² / (d² + κ).
+
+    Motivated by the signal-to-noise structure of the Gaussian score.
+    Default kernel for the Squeeze Kernel estimator.
+    """
+    if kappa <= 0.0:
+        raise ValueError("kappa must be > 0.")
+    return d2 / (d2 + kappa)
+
+
+def kernel_exponential(d2: float, /, *, gamma: float, **kw) -> float:
+    """Rayleigh survival (exponential) kernel: w = 1 − exp(−d²/(2γ²))."""
+    if gamma <= 0.0:
+        raise ValueError("gamma must be > 0.")
+    return 1.0 - math.exp(-d2 / (2.0 * gamma * gamma))
+
+
+def kernel_chi2_cdf(d2: float, /, *, n_observed: int, **kw) -> float:
+    """Chi-squared CDF kernel: w = F_χ²_N(N·d²).  Requires scipy."""
+    try:
+        from scipy.stats import chi2
+    except ImportError as exc:
+        raise ImportError(
+            "kernel_chi2_cdf() requires SciPy. Install the optional "
+            "'squeeze-kernel[full]' dependencies."
+        ) from exc
+    return float(chi2.cdf(n_observed * d2, df=n_observed))
+
+
+def calibrate_kappa(d2_samples, target_mean_weight: float = 0.5) -> float:
+    """Find κ such that E[d²/(d²+κ)] ≈ target_mean_weight on burn-in data.
+
+    This implements the closed-form initializer from Proposition 1 of the paper,
+    refined by one-dimensional root finding.
+
+    Parameters
+    ----------
+    d2_samples : array-like
+        Observed d̄²_t values from a burn-in window (use ``extract_d2_series``).
+    target_mean_weight : float
+        Target average kernel weight in (0, 1).
+
+    Returns
+    -------
+    float
+        Calibrated κ value.
+    """
+    import numpy as np
+    try:
+        from scipy.optimize import brentq
+    except ImportError as exc:
+        raise ImportError(
+            "calibrate_kappa() requires SciPy. Install the optional "
+            "'squeeze-kernel[full]' dependencies."
+        ) from exc
+
+    d2 = np.asarray(d2_samples, dtype=np.float64)
+    d2 = d2[np.isfinite(d2) & (d2 > 0)]
+    if len(d2) < 10:
+        raise ValueError("Need at least 10 finite positive d² samples for calibration.")
+
+    mu = float(d2.mean())
+    # Closed-form initializer (Proposition 1): κ₀ = μ·(1/w₀ − 1)
+    kappa_init = mu * (1.0 / target_mean_weight - 1.0)
+
+    # Refine via root finding
+    def residual(kappa):
+        return float(np.mean(d2 / (d2 + kappa))) - target_mean_weight
+
+    lo = max(kappa_init * 0.01, 1e-6)
+    hi = kappa_init * 100.0
+    # Ensure bracket
+    while residual(lo) < 0:
+        lo *= 0.1
+    while residual(hi) > 0:
+        hi *= 10.0
+
+    return float(brentq(residual, lo, hi, xtol=1e-8))
+
+
+def extract_d2_series(
+    returns, lambda_vol: float = 0.98, epsilon: float = 1e-8,
+):
+    """Extract the d̄²_t series from a returns panel for κ calibration.
+
+    Parameters
+    ----------
+    returns : array-like, shape (T, n)
+        Daily return matrix.
+    lambda_vol : float
+        Volatility decay factor.
+
+    Returns
+    -------
+    ndarray, shape (T,)
+        Average squared standardized return at each timestep.
+    """
+    import numpy as np
+
+    x = np.asarray(returns, dtype=np.float64)
+    t_total, n = x.shape
+    var_t = np.zeros(n, dtype=np.float64)
+    var_init = np.zeros(n, dtype=bool)
+    d2_out = np.empty(t_total, dtype=np.float64)
+    one_minus_lv = 1.0 - lambda_vol
+
+    for t in range(t_total):
+        rt = x[t]
+        finite = np.isfinite(rt)
+        first = finite & ~var_init
+        repeat = finite & var_init
+
+        if np.any(first):
+            var_t[first] = rt[first] ** 2 + epsilon
+            var_init[first] = True
+        if np.any(repeat):
+            var_t[repeat] = lambda_vol * var_t[repeat] + one_minus_lv * rt[repeat] ** 2
+
+        vol = np.sqrt(var_t)
+        obs = finite & var_init
+        n_obs = int(obs.sum())
+        if n_obs > 0:
+            z = rt[obs] / (vol[obs] + epsilon)
+            d2_out[t] = float(z @ z) / n_obs
+        else:
+            d2_out[t] = float("nan")
+
+    return d2_out