squeeze-kernel 0.7.0__tar.gz → 0.7.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: squeeze-kernel
3
- Version: 0.7.0
3
+ Version: 0.7.1
4
4
  Summary: Streaming, PSD-by-construction covariance estimator with Fisher-kernel weighting and adaptive shrinkage
5
5
  Keywords: covariance,correlation,ewma,kernel,risk,streaming
6
6
  Author: Robert Kende
@@ -36,13 +36,13 @@ Description-Content-Type: text/markdown
36
36
  [![Python](https://img.shields.io/pypi/pyversions/squeeze-kernel.svg)](https://pypi.org/project/squeeze-kernel/)
37
37
  [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
38
38
 
39
- A **streaming covariance estimator for panels of daily financial returns**. One `O(n²)` update per day, positive semi-definite **by construction** at every step, missing values handled **natively**, and defaults that require no tuning. Only dependency: NumPy.
39
+ A **streaming covariance estimator for panels of financial returns** that learns fastest on the days that matter. One `O(n²)` update per period, positive semi-definite **by construction** at every step, missing values handled **natively**, and defaults that require no tuning. Only dependency: NumPy.
40
40
 
41
- Reference: *"The Squeeze Kernel Covariance Estimator: Dual-Timescale Tracking with Adaptive Shrinkage"* (Kende, 2026) — [SSRN abstract 6455918](https://ssrn.com/abstract=6455918).
41
+ References: *"The Squeeze Kernel Covariance Estimator: Dual-Timescale Tracking with Adaptive Shrinkage"* (Kende, 2026) — [SSRN abstract 6455918](https://ssrn.com/abstract=6455918) — and its companion *"Cluster-Respecting Shrinkage for Streaming Covariance Estimation"* (Kende, 2026).
42
42
 
43
43
  ## Why
44
44
 
45
- Rolling-window estimators (Ledoit–Wolf, nonlinear shrinkage, RMT denoising) refit over a fixed window each day and cannot adapt within it; multivariate GARCH (DCC) adapts but needs multi-stage estimation and a fragile news coefficient. The Squeeze Kernel is a single streaming recursion that:
45
+ Every standard covariance estimator treats all trading days as equally informative. Markets don't work that way: **correlations reveal themselves when markets move; calm days are mostly noise.** The Squeeze Kernel weighs each day by the information it actually carries — quiet days barely count, dispersion shocks pass through in full — and runs volatility and correlation on separate clocks, so vol spikes never contaminate the correlation estimate. The result is a single streaming recursion that:
46
46
 
47
47
  - **is PSD at every step, structurally** — never needs eigenvalue clipping, nearest-PSD projection, or a solver;
48
48
  - **adapts fastest exactly when it matters** — a Fisher-information kernel up-weights high-dispersion (stress) days, when correlation regimes actually move;
@@ -50,7 +50,37 @@ Rolling-window estimators (Ledoit–Wolf, nonlinear shrinkage, RMT denoising) re
50
50
  - **ingests missing values natively** — listings, delistings, and halts enter as `NaN`; no imputation or complete-case subsetting;
51
51
  - **is fast** — a full 30-year daily pass takes ~0.75 s at n=100 and ~3.4 s at n=300 (single-threaded), 30–40× faster than rolling-window baselines at scale.
52
52
 
53
- On a 30-year S&P 500 panel (n=100, ~7,600 out-of-sample days) it statistically ties DCC on one-step density forecasts and beats EWMA, Ledoit–Wolf, OAS, nonlinear shrinkage, RMT denoising, and the Gerber statistic — and it is the only method in the 90% model confidence set together with DCC. At n=300 it leads every competitor that remains statistically viable.
53
+ **The scoreboard.** On a 30-year S&P 500 panel (~7,600 out-of-sample days) the default single-scale estimator beats EWMA, Ledoit–Wolf, OAS, nonlinear shrinkage, RMT denoising, and the Gerber statistic on one-step density forecasts, and statistically ties DCC — the only two methods in the 90% model confidence set. The headline configuration (multi-scale correlation memory, `corr_half_lives=(43, 173, 693)`) goes further: it **leads every tested method at every universe size and the 90% model confidence set collapses to it alone**, with the margin confirmed out-of-time on an external industry panel. For large equity universes, the cluster shrinkage target (`shrinkage_target="cluster"`) adds a further large gain exactly where shrinkage binds: 25 NLL points at n=300.
54
+
55
+ Against the alternatives:
56
+
57
+ - **RiskMetrics / EWMA** — same one-recursion simplicity, but two clocks and information weighting: better forecasts at zero extra operational cost.
58
+ - **Ledoit–Wolf, nonlinear shrinkage, RMT denoising** — static snapshots refit from scratch on a rolling window each day; the Squeeze Kernel is genuinely dynamic, more accurate on the benchmark, and 30–40× faster at scale.
59
+ - **DCC-GARCH** — matched (single-scale) or beaten (multi-scale) without multi-stage likelihood fitting or the fragile news-impact coefficient; at n=300 DCC needs a multi-year warm-up before its forecasts stabilise and still trails by ~25 NLL.
60
+ - **Gerber statistic** — the Squeeze Kernel is a PSD-by-construction generalization of the same robust-comovement idea: no nearest-PSD repair step, and it wins the head-to-head.
61
+
62
+ ## See the difference
63
+
64
+ Take a passive strategy any allocator would recognize: a long-only minimum-variance portfolio of 300 liquid US stocks, scaled to a 15% volatility target, rebalanced once a month, with 5 bps trading costs. Run it twice on identical data. The only thing that changes between the two runs is the covariance matrix that picks the weights and sets the exposure.
65
+
66
+ <picture>
67
+ <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/r0k3/squeeze-kernel/main/examples/figures/vol_targeted_portfolio_dark.png">
68
+ <img alt="Vol-targeted long-only minimum-variance portfolio on 300 US equities: Squeeze Kernel vs Ledoit-Wolf equity curves, drawdown, realized volatility, and risk/return profile" src="https://raw.githubusercontent.com/r0k3/squeeze-kernel/main/examples/figures/vol_targeted_portfolio.png">
69
+ </picture>
70
+
71
+ | Method | CAGR | Vol | Sharpe | MaxDD | Calmar | Vol-target RMSE |
72
+ |---|---|---|---|---|---|---|
73
+ | **Squeeze Kernel** | **13.1%** | **13.0%** | **1.00** | **-31.2%** | **0.42** | **6.47%** |
74
+ | Ledoit-Wolf (252d) | 11.7% | 14.6% | 0.80 | -38.7% | 0.30 | 7.51% |
75
+
76
+ You can regenerate this example from the repo alone. The returns panel ships as a parquet (daily returns for 300 US stocks, sourced from Yahoo Finance), and the script prints the table and redraws the figure:
77
+
78
+ ```bash
79
+ pip install squeeze-kernel pandas pyarrow scikit-learn matplotlib
80
+ python examples/vol_targeted_portfolio.py # ~2 minutes
81
+ ```
82
+
83
+ The full protocol and data notes live in [`examples/vol_targeted_portfolio.py`](examples/vol_targeted_portfolio.py) and [`examples/data/build_equity_panel.py`](examples/data/build_equity_panel.py).
54
84
 
55
85
  ## Installation
56
86
 
@@ -117,44 +147,27 @@ kappa = SqueezeKernelEstimator.calibrate_kappa(burn_in_returns, target_weight=0.
117
147
 
118
148
  ## Advanced options
119
149
 
120
- **Adaptive scale-free correlation memory** (`corr_half_lives=(43, 173, 693)`, `corr_theta=0.25`): replaces the single correlation timescale with a positive combination of EWMAs on a geometric half-life ladder — each scale normalized and adaptively shrunk against its own effective sample size, then the covariances blended with weights resting at the prior ∝ half-life^`corr_theta`. By Bernstein's theorem the ladder approximates the power-law memory of financial correlations (the streaming analogue of HAR). For ladders of two or more rungs the blend weights are gated by a sequential surprise detector: a two-sided Page CUSUM on the studentized fast-vs-slow per-rung predictive-score drift (threshold set by Siegmund's average-run-length approximation at ~2 years, no tuned parameters) tilts the weights toward the fast or slow end of the ladder when one side accumulates statistically forced evidence, decaying back at the fastest rung's half-life. Weights equal the prior on all non-alarmed days, the blend stays convex, so PSD holds by construction; `None` (default) reproduces the published single-scale estimator exactly. This is the paper's **headline configuration**: on the S&P 500 benchmark it leads every tested method at every universe size (held-out one-step NLL −4.6 vs single-scale at n=100, −11.1 at n=300 before the cluster target), the 90% model confidence set collapses to it alone, and the detector's margin is confirmed out-of-time on an external industry panel (+0.53 NLL/day, p=1×10⁻⁴). Cost is O(K·n²) per update plus one Cholesky per rung per day for the detector scores. Composes with `shrinkage_target="cluster"`; mutually exclusive with `lambda_corr_fast`.
150
+ All options are off by default; the defaults reproduce the published estimator exactly. Full derivations and benchmark tables are in the papers.
121
151
 
122
- ```python
123
- est = SqueezeKernelEstimator(n_assets=100, corr_half_lives=(43, 173, 693), corr_theta=0.25)
124
- # maximal variant at high dimension:
125
- est = SqueezeKernelEstimator(n_assets=300, corr_half_lives=(43, 173, 693), shrinkage_target="cluster")
126
- ```
152
+ **Multi-scale correlation memory** (`corr_half_lives=(43, 173, 693)`): replaces the single correlation timescale with a ladder of EWMAs whose blend a sequential surprise detector tilts toward fast or slow memory as the evidence demands. This is the papers' headline configuration: it leads every tested method at every universe size, and the blend stays convex so PSD still holds by construction.
127
153
 
128
- **Score-exact weighting** (`weight_statistic="mahalanobis"`, use with `kappa=1.0`): drives the kernel with the Mahalanobis surprise `z'C⁻¹z/N` against the estimator's own correlation instead of the marginal dispersion. Improves accuracy in the moderate-concentration regime — use only when `n / T_eff ≲ 0.5` (e.g. n ≤ 100 at the default `lambda_corr`); at higher concentration the estimated inverse degrades it and the default is strictly better.
154
+ **Cluster shrinkage target** (`shrinkage_target="cluster"`): shrinks toward a target that respects the correlation matrix's own block structure instead of a single equicorrelation, with no clustering algorithm and zero added parameters. The best choice for large equity universes: worth 4 held-out NLL points at n=200 and 25 at n=300 on the benchmark.
129
155
 
130
156
  ```python
131
- est = SqueezeKernelEstimator(n_assets=100, kappa=1.0, weight_statistic="mahalanobis")
157
+ # recommended setup for a large equity universe:
158
+ est = SqueezeKernelEstimator(n_assets=300, corr_half_lives=(43, 173, 693),
159
+ shrinkage_target="cluster")
132
160
  ```
133
161
 
134
- **Score-driven memory** (`lambda_corr_fast=0.99`): lets stress days also *shorten* the correlation memory (decay slides from `lambda_corr` toward `lambda_corr_fast` as the kernel weight rises). Do **not** combine with the Mahalanobis option — they act on the same channel and the combination degrades accuracy.
135
-
136
- **OU volatility anchor** (`vol_anchor_phi=0.995`): mean-reverts each asset's variance prediction toward a slow per-asset anchor (a ~1000-day EWMA of squared returns) before the daily update — a two-timescale, component-style volatility structure. One global parameter with a clean interpretation (deviation half-life ≈ ln 2/(1−φ) days; φ=0.995 ≈ 139 d). On the S&P 500 n=100 benchmark this improved held-out one-step NLL by 3.3 points (4.3 at φ=0.99) and five-step NLL by 3.9 (5.0), with no degradation at n=300. `None` (default) or φ=1 reproduces the published estimator exactly.
162
+ **OU volatility anchor** (`vol_anchor_phi=0.995`): mean-reverts each asset's variance forecast toward a slow per-asset anchor, giving a two-timescale volatility structure with a single parameter (deviation half-life ≈ ln 2/(1−φ) days). Worth 3–5 NLL points on the benchmark.
137
163
 
138
- ```python
139
- est = SqueezeKernelEstimator(n_assets=100, vol_anchor_phi=0.995)
140
- ```
164
+ **Score-exact weighting** (`weight_statistic="mahalanobis"`, use with `kappa=1.0`): drives the kernel with the Mahalanobis surprise against the estimator's own correlation. Use it only in the moderate-concentration regime (`n / T_eff ≲ 0.5`).
141
165
 
142
- **Cluster shrinkage target** (`shrinkage_target="cluster"`): generalizes the equicorrelation shrinkage target to respect the correlation matrix's own block/cluster structure — with **no clustering algorithm**. The target morphs with the shrinkage intensity, T = (1−α)·T_equi + α·[(1−γ)I + γ·(C∘C)], where C∘C is the Hadamard square of the current correlation (positive semi-definite by the Schur product theorem; entries are pairwise shared-variance fractions) and γ is level-matched automatically. Zero added parameters, still O(n²), and as α→0 it reduces exactly to the default estimator. Held-out one-step NLL on the S&P 500 benchmark: ±0.1 at n=100, **−4.2 at n=200, −25.0 at n=300** — recommended whenever the universe size approaches the effective sample size.
166
+ **Score-driven memory** (`lambda_corr_fast=0.99`): lets stress days also shorten the correlation memory. Do not combine with the Mahalanobis option.
143
167
 
144
- ```python
145
- est = SqueezeKernelEstimator(n_assets=300, shrinkage_target="cluster")
146
- ```
168
+ **New-listing usability gate** (`min_obs=60`): exposes a `usable_mask` property marking assets with at least `min_obs` observations, so deployments can exclude cold starts from scoring and optimization; the estimates themselves are unchanged.
147
169
 
148
- **New-listing usability gate** (`min_obs=60`): on an expanding universe, an asset's forecast rows are dominated by its single-observation variance initialization for its first weeks of life and are unusable for scoring or portfolio construction (measured ≈ +2,800 NLL/day on days whose scored set included such assets, on a 42-instrument multi-asset panel). `min_obs` gates nothing inside the estimator — states warm normally, all outputs are unchanged — it exposes a `usable_mask` property marking assets with at least `min_obs` finite observations, so deployments subset with it:
149
-
150
- ```python
151
- est = SqueezeKernelEstimator(n_assets=42, min_obs=60)
152
- # ... update loop ...
153
- m = est.usable_mask
154
- cov_usable = est.get_cov()[np.ix_(m, m)]
155
- ```
156
-
157
- **Alternative kernels**: pass `kernel_fn=kernel_exponential` (with `kernel_kwargs={"gamma": ...}`) or `kernel_chi2_cdf`, or any callable `(d2, *, n_observed, **kw) -> float` mapping to `[0, 1)`. The PSD guarantee holds for any such kernel.
170
+ **Alternative kernels**: pass `kernel_fn=kernel_exponential` or `kernel_chi2_cdf`, or any callable mapping to `[0, 1)`; the PSD guarantee holds for any such kernel.
158
171
 
159
172
  ## How it works
160
173
 
@@ -176,8 +189,6 @@ uv run mypy # strict type check (src/squeeze_kernel)
176
189
  uv build # build sdist + wheel
177
190
  ```
178
191
 
179
- Releases: publishing a GitHub release from a `v*` tag triggers the [publish workflow](.github/workflows/publish.yml), which builds and uploads to PyPI via trusted publishing.
180
-
181
192
  ## Citation
182
193
 
183
194
  ```bibtex
@@ -0,0 +1,176 @@
1
+ # Squeeze Kernel Covariance Estimator
2
+
3
+ [![CI](https://github.com/r0k3/squeeze-kernel/actions/workflows/ci.yml/badge.svg)](https://github.com/r0k3/squeeze-kernel/actions/workflows/ci.yml)
4
+ [![PyPI](https://img.shields.io/pypi/v/squeeze-kernel.svg)](https://pypi.org/project/squeeze-kernel/)
5
+ [![Python](https://img.shields.io/pypi/pyversions/squeeze-kernel.svg)](https://pypi.org/project/squeeze-kernel/)
6
+ [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
7
+
8
+ A **streaming covariance estimator for panels of financial returns** that learns fastest on the days that matter. One `O(n²)` update per period, positive semi-definite **by construction** at every step, missing values handled **natively**, and defaults that require no tuning. Only dependency: NumPy.
9
+
10
+ References: *"The Squeeze Kernel Covariance Estimator: Dual-Timescale Tracking with Adaptive Shrinkage"* (Kende, 2026) — [SSRN abstract 6455918](https://ssrn.com/abstract=6455918) — and its companion *"Cluster-Respecting Shrinkage for Streaming Covariance Estimation"* (Kende, 2026).
11
+
12
+ ## Why
13
+
14
+ Every standard covariance estimator treats all trading days as equally informative. Markets don't work that way: **correlations reveal themselves when markets move; calm days are mostly noise.** The Squeeze Kernel weighs each day by the information it actually carries — quiet days barely count, dispersion shocks pass through in full — and runs volatility and correlation on separate clocks, so vol spikes never contaminate the correlation estimate. The result is a single streaming recursion that:
15
+
16
+ - **is PSD at every step, structurally** — never needs eigenvalue clipping, nearest-PSD projection, or a solver;
17
+ - **adapts fastest exactly when it matters** — a Fisher-information kernel up-weights high-dispersion (stress) days, when correlation regimes actually move;
18
+ - **regularises itself** — an adaptive equicorrelation shrinkage activates automatically as the asset count approaches the effective sample size, with a provable condition-number bound;
19
+ - **ingests missing values natively** — listings, delistings, and halts enter as `NaN`; no imputation or complete-case subsetting;
20
+ - **is fast** — a full 30-year daily pass takes ~0.75 s at n=100 and ~3.4 s at n=300 (single-threaded), 30–40× faster than rolling-window baselines at scale.
21
+
22
+ **The scoreboard.** On a 30-year S&P 500 panel (~7,600 out-of-sample days) the default single-scale estimator beats EWMA, Ledoit–Wolf, OAS, nonlinear shrinkage, RMT denoising, and the Gerber statistic on one-step density forecasts, and statistically ties DCC — the only two methods in the 90% model confidence set. The headline configuration (multi-scale correlation memory, `corr_half_lives=(43, 173, 693)`) goes further: it **leads every tested method at every universe size and the 90% model confidence set collapses to it alone**, with the margin confirmed out-of-time on an external industry panel. For large equity universes, the cluster shrinkage target (`shrinkage_target="cluster"`) adds a further large gain exactly where shrinkage binds: 25 NLL points at n=300.
23
+
24
+ Against the alternatives:
25
+
26
+ - **RiskMetrics / EWMA** — same one-recursion simplicity, but two clocks and information weighting: better forecasts at zero extra operational cost.
27
+ - **Ledoit–Wolf, nonlinear shrinkage, RMT denoising** — static snapshots refit from scratch on a rolling window each day; the Squeeze Kernel is genuinely dynamic, more accurate on the benchmark, and 30–40× faster at scale.
28
+ - **DCC-GARCH** — matched (single-scale) or beaten (multi-scale) without multi-stage likelihood fitting or the fragile news-impact coefficient; at n=300 DCC needs a multi-year warm-up before its forecasts stabilise and still trails by ~25 NLL.
29
+ - **Gerber statistic** — the Squeeze Kernel is a PSD-by-construction generalization of the same robust-comovement idea: no nearest-PSD repair step, and it wins the head-to-head.
30
+
31
+ ## See the difference
32
+
33
+ Take a passive strategy any allocator would recognize: a long-only minimum-variance portfolio of 300 liquid US stocks, scaled to a 15% volatility target, rebalanced once a month, with 5 bps trading costs. Run it twice on identical data. The only thing that changes between the two runs is the covariance matrix that picks the weights and sets the exposure.
34
+
35
+ <picture>
36
+ <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/r0k3/squeeze-kernel/main/examples/figures/vol_targeted_portfolio_dark.png">
37
+ <img alt="Vol-targeted long-only minimum-variance portfolio on 300 US equities: Squeeze Kernel vs Ledoit-Wolf equity curves, drawdown, realized volatility, and risk/return profile" src="https://raw.githubusercontent.com/r0k3/squeeze-kernel/main/examples/figures/vol_targeted_portfolio.png">
38
+ </picture>
39
+
40
+ | Method | CAGR | Vol | Sharpe | MaxDD | Calmar | Vol-target RMSE |
41
+ |---|---|---|---|---|---|---|
42
+ | **Squeeze Kernel** | **13.1%** | **13.0%** | **1.00** | **-31.2%** | **0.42** | **6.47%** |
43
+ | Ledoit-Wolf (252d) | 11.7% | 14.6% | 0.80 | -38.7% | 0.30 | 7.51% |
44
+
45
+ You can regenerate this example from the repo alone. The returns panel ships as a parquet (daily returns for 300 US stocks, sourced from Yahoo Finance), and the script prints the table and redraws the figure:
46
+
47
+ ```bash
48
+ pip install squeeze-kernel pandas pyarrow scikit-learn matplotlib
49
+ python examples/vol_targeted_portfolio.py # ~2 minutes
50
+ ```
51
+
52
+ The full protocol and data notes live in [`examples/vol_targeted_portfolio.py`](examples/vol_targeted_portfolio.py) and [`examples/data/build_equity_panel.py`](examples/data/build_equity_panel.py).
53
+
54
+ ## Installation
55
+
56
+ ```bash
57
+ pip install squeeze-kernel # NumPy only
58
+ pip install "squeeze-kernel[full]" # + SciPy (kappa calibration, chi² kernel)
59
+ ```
60
+
61
+ ## Quickstart
62
+
63
+ ```python
64
+ import numpy as np
65
+ from squeeze_kernel import SqueezeKernelEstimator
66
+
67
+ # daily_returns: array of shape (T, n) — may contain NaN for missing assets
68
+ est = SqueezeKernelEstimator(n_assets=daily_returns.shape[1])
69
+
70
+ for r_t in daily_returns: # stream one day at a time
71
+ est.update(r_t)
72
+
73
+ cov = est.get_cov() # (n, n) covariance, PSD by construction
74
+ corr = est.get_corr() # (n, n) correlation
75
+ ```
76
+
77
+ That is the whole API for most uses. The defaults (`lambda_vol=0.98`, `lambda_corr=0.996`, `kappa=0.25`) are the paper-recommended settings for daily returns, selected by time-series cross-validation and robust across a 50× parameter sweep — deploy them as-is.
78
+
79
+ Batch mode, if you prefer the full path in one call:
80
+
81
+ ```python
82
+ from squeeze_kernel import estimate_squeeze_cov
83
+
84
+ cov_path, corr_path, weights = estimate_squeeze_cov(daily_returns, with_weights=True)
85
+ # cov_path: (T, n, n) — the estimate after each day
86
+ ```
87
+
88
+ A complete runnable walkthrough (streaming, missing data, batch) is in [`examples/quickstart.py`](examples/quickstart.py).
89
+
90
+ ## Missing values
91
+
92
+ Pass `NaN` for any asset not observed on a given day — nothing else to do:
93
+
94
+ ```python
95
+ r_t = np.array([0.004, np.nan, -0.011]) # asset 2 not trading today
96
+ est.update(r_t) # PSD preserved, no imputation
97
+ ```
98
+
99
+ ## Parameters
100
+
101
+ | Parameter | Default | Meaning |
102
+ |---|---|---|
103
+ | `lambda_vol` | `0.98` | volatility EWMA decay (half-life ≈ 34 days) |
104
+ | `lambda_corr` | `0.996` | correlation EWMA decay (half-life ≈ 173 days, T_eff ≈ 250) |
105
+ | `kappa` | `0.25` | Fisher kernel saturation; higher = stronger calm-day filtering |
106
+ | `shrinkage` | `"auto"` | adaptive equicorrelation shrinkage (`"none"` or a float to override) |
107
+ | `shrinkage_delta` | `0.10` | concentration threshold at which shrinkage activates |
108
+
109
+ Useful read-only state after each `update()`: `est.weight` (last kernel weight), `est.effective_sample_size` (kernel-weighted T_eff), `est.shrinkage_intensity` (current α).
110
+
111
+ To recalibrate `kappa` for a different asset class (requires the `full` extra):
112
+
113
+ ```python
114
+ kappa = SqueezeKernelEstimator.calibrate_kappa(burn_in_returns, target_weight=0.6)
115
+ ```
116
+
117
+ ## Advanced options
118
+
119
+ All options are off by default; the defaults reproduce the published estimator exactly. Full derivations and benchmark tables are in the papers.
120
+
121
+ **Multi-scale correlation memory** (`corr_half_lives=(43, 173, 693)`): replaces the single correlation timescale with a ladder of EWMAs whose blend a sequential surprise detector tilts toward fast or slow memory as the evidence demands. This is the papers' headline configuration: it leads every tested method at every universe size, and the blend stays convex so PSD still holds by construction.
122
+
123
+ **Cluster shrinkage target** (`shrinkage_target="cluster"`): shrinks toward a target that respects the correlation matrix's own block structure instead of a single equicorrelation, with no clustering algorithm and zero added parameters. The best choice for large equity universes: worth 4 held-out NLL points at n=200 and 25 at n=300 on the benchmark.
124
+
125
+ ```python
126
+ # recommended setup for a large equity universe:
127
+ est = SqueezeKernelEstimator(n_assets=300, corr_half_lives=(43, 173, 693),
128
+ shrinkage_target="cluster")
129
+ ```
130
+
131
+ **OU volatility anchor** (`vol_anchor_phi=0.995`): mean-reverts each asset's variance forecast toward a slow per-asset anchor, giving a two-timescale volatility structure with a single parameter (deviation half-life ≈ ln 2/(1−φ) days). Worth 3–5 NLL points on the benchmark.
132
+
133
+ **Score-exact weighting** (`weight_statistic="mahalanobis"`, use with `kappa=1.0`): drives the kernel with the Mahalanobis surprise against the estimator's own correlation. Use it only in the moderate-concentration regime (`n / T_eff ≲ 0.5`).
134
+
135
+ **Score-driven memory** (`lambda_corr_fast=0.99`): lets stress days also shorten the correlation memory. Do not combine with the Mahalanobis option.
136
+
137
+ **New-listing usability gate** (`min_obs=60`): exposes a `usable_mask` property marking assets with at least `min_obs` observations, so deployments can exclude cold starts from scoring and optimization; the estimates themselves are unchanged.
138
+
139
+ **Alternative kernels**: pass `kernel_fn=kernel_exponential` or `kernel_chi2_cdf`, or any callable mapping to `[0, 1)`; the PSD guarantee holds for any such kernel.
140
+
141
+ ## How it works
142
+
143
+ Three mechanisms in one recursion:
144
+
145
+ 1. **Dual-timescale EWMA** — fast per-asset volatility (`lambda_vol`) is separated from slow correlation dynamics (`lambda_corr`), so variance shocks don't contaminate the correlation estimate.
146
+ 2. **Fisher kernel weighting** — each day's standardized outer product enters with weight `w = d²/(d² + kappa)`, where `d²` is the mean squared standardized return: calm days contribute little, dispersion shocks contribute fully.
147
+ 3. **Adaptive equicorrelation shrinkage** — `alpha = min(1, max(0, n/(2·S) − delta))` blends toward an equicorrelation target using the estimator's own kernel-weighted sample size `S`; it is a no-op at low dimension and provides provably bounded conditioning at high dimension.
148
+
149
+ The complete update is a natural-gradient step on the Gaussian log-likelihood, with the kernel weight acting as an adaptive Riemannian learning rate (paper, Appendix B).
150
+
151
+ ## Development
152
+
153
+ ```bash
154
+ uv sync --extra full --extra dev
155
+ uv run python -m pytest # test suite
156
+ uv run python -m ruff check . # lint
157
+ uv run mypy # strict type check (src/squeeze_kernel)
158
+ uv build # build sdist + wheel
159
+ ```
160
+
161
+ ## Citation
162
+
163
+ ```bibtex
164
+ @article{kende2026squeeze,
165
+ title = {The Squeeze Kernel Covariance Estimator: Dual-Timescale Tracking with Adaptive Shrinkage},
166
+ author = {Kende, Robert},
167
+ year = {2026},
168
+ note = {Available at SSRN: \url{https://ssrn.com/abstract=6455918}}
169
+ }
170
+ ```
171
+
172
+ See also [`CITATION.cff`](CITATION.cff).
173
+
174
+ ## License
175
+
176
+ MIT
@@ -4,7 +4,7 @@ build-backend = "uv_build"
4
4
 
5
5
  [project]
6
6
  name = "squeeze-kernel"
7
- version = "0.7.0"
7
+ version = "0.7.1"
8
8
  description = "Streaming, PSD-by-construction covariance estimator with Fisher-kernel weighting and adaptive shrinkage"
9
9
  readme = "README.md"
10
10
  license = "MIT"
@@ -4,7 +4,7 @@ build-backend = "uv_build"
4
4
 
5
5
  [project]
6
6
  name = "squeeze-kernel"
7
- version = "0.7.0"
7
+ version = "0.7.1"
8
8
  description = "Streaming, PSD-by-construction covariance estimator with Fisher-kernel weighting and adaptive shrinkage"
9
9
  readme = "README.md"
10
10
  license = "MIT"
@@ -34,4 +34,4 @@ __all__ = [
34
34
  "kernel_chi2_cdf",
35
35
  ]
36
36
 
37
- __version__ = "0.7.0"
37
+ __version__ = "0.7.1"
@@ -1,165 +0,0 @@
1
- # Squeeze Kernel Covariance Estimator
2
-
3
- [![CI](https://github.com/r0k3/squeeze-kernel/actions/workflows/ci.yml/badge.svg)](https://github.com/r0k3/squeeze-kernel/actions/workflows/ci.yml)
4
- [![PyPI](https://img.shields.io/pypi/v/squeeze-kernel.svg)](https://pypi.org/project/squeeze-kernel/)
5
- [![Python](https://img.shields.io/pypi/pyversions/squeeze-kernel.svg)](https://pypi.org/project/squeeze-kernel/)
6
- [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
7
-
8
- A **streaming covariance estimator for panels of daily financial returns**. One `O(n²)` update per day, positive semi-definite **by construction** at every step, missing values handled **natively**, and defaults that require no tuning. Only dependency: NumPy.
9
-
10
- Reference: *"The Squeeze Kernel Covariance Estimator: Dual-Timescale Tracking with Adaptive Shrinkage"* (Kende, 2026) — [SSRN abstract 6455918](https://ssrn.com/abstract=6455918).
11
-
12
- ## Why
13
-
14
- Rolling-window estimators (Ledoit–Wolf, nonlinear shrinkage, RMT denoising) refit over a fixed window each day and cannot adapt within it; multivariate GARCH (DCC) adapts but needs multi-stage estimation and a fragile news coefficient. The Squeeze Kernel is a single streaming recursion that:
15
-
16
- - **is PSD at every step, structurally** — never needs eigenvalue clipping, nearest-PSD projection, or a solver;
17
- - **adapts fastest exactly when it matters** — a Fisher-information kernel up-weights high-dispersion (stress) days, when correlation regimes actually move;
18
- - **regularises itself** — an adaptive equicorrelation shrinkage activates automatically as the asset count approaches the effective sample size, with a provable condition-number bound;
19
- - **ingests missing values natively** — listings, delistings, and halts enter as `NaN`; no imputation or complete-case subsetting;
20
- - **is fast** — a full 30-year daily pass takes ~0.75 s at n=100 and ~3.4 s at n=300 (single-threaded), 30–40× faster than rolling-window baselines at scale.
21
-
22
- On a 30-year S&P 500 panel (n=100, ~7,600 out-of-sample days) it statistically ties DCC on one-step density forecasts and beats EWMA, Ledoit–Wolf, OAS, nonlinear shrinkage, RMT denoising, and the Gerber statistic — and it is the only method in the 90% model confidence set together with DCC. At n=300 it leads every competitor that remains statistically viable.
23
-
24
- ## Installation
25
-
26
- ```bash
27
- pip install squeeze-kernel # NumPy only
28
- pip install "squeeze-kernel[full]" # + SciPy (kappa calibration, chi² kernel)
29
- ```
30
-
31
- ## Quickstart
32
-
33
- ```python
34
- import numpy as np
35
- from squeeze_kernel import SqueezeKernelEstimator
36
-
37
- # daily_returns: array of shape (T, n) — may contain NaN for missing assets
38
- est = SqueezeKernelEstimator(n_assets=daily_returns.shape[1])
39
-
40
- for r_t in daily_returns: # stream one day at a time
41
- est.update(r_t)
42
-
43
- cov = est.get_cov() # (n, n) covariance, PSD by construction
44
- corr = est.get_corr() # (n, n) correlation
45
- ```
46
-
47
- That is the whole API for most uses. The defaults (`lambda_vol=0.98`, `lambda_corr=0.996`, `kappa=0.25`) are the paper-recommended settings for daily returns, selected by time-series cross-validation and robust across a 50× parameter sweep — deploy them as-is.
48
-
49
- Batch mode, if you prefer the full path in one call:
50
-
51
- ```python
52
- from squeeze_kernel import estimate_squeeze_cov
53
-
54
- cov_path, corr_path, weights = estimate_squeeze_cov(daily_returns, with_weights=True)
55
- # cov_path: (T, n, n) — the estimate after each day
56
- ```
57
-
58
- A complete runnable walkthrough (streaming, missing data, batch) is in [`examples/quickstart.py`](examples/quickstart.py).
59
-
60
- ## Missing values
61
-
62
- Pass `NaN` for any asset not observed on a given day — nothing else to do:
63
-
64
- ```python
65
- r_t = np.array([0.004, np.nan, -0.011]) # asset 2 not trading today
66
- est.update(r_t) # PSD preserved, no imputation
67
- ```
68
-
69
- ## Parameters
70
-
71
- | Parameter | Default | Meaning |
72
- |---|---|---|
73
- | `lambda_vol` | `0.98` | volatility EWMA decay (half-life ≈ 34 days) |
74
- | `lambda_corr` | `0.996` | correlation EWMA decay (half-life ≈ 173 days, T_eff ≈ 250) |
75
- | `kappa` | `0.25` | Fisher kernel saturation; higher = stronger calm-day filtering |
76
- | `shrinkage` | `"auto"` | adaptive equicorrelation shrinkage (`"none"` or a float to override) |
77
- | `shrinkage_delta` | `0.10` | concentration threshold at which shrinkage activates |
78
-
79
- Useful read-only state after each `update()`: `est.weight` (last kernel weight), `est.effective_sample_size` (kernel-weighted T_eff), `est.shrinkage_intensity` (current α).
80
-
81
- To recalibrate `kappa` for a different asset class (requires the `full` extra):
82
-
83
- ```python
84
- kappa = SqueezeKernelEstimator.calibrate_kappa(burn_in_returns, target_weight=0.6)
85
- ```
86
-
87
- ## Advanced options
88
-
89
- **Adaptive scale-free correlation memory** (`corr_half_lives=(43, 173, 693)`, `corr_theta=0.25`): replaces the single correlation timescale with a positive combination of EWMAs on a geometric half-life ladder — each scale normalized and adaptively shrunk against its own effective sample size, then the covariances blended with weights resting at the prior ∝ half-life^`corr_theta`. By Bernstein's theorem the ladder approximates the power-law memory of financial correlations (the streaming analogue of HAR). For ladders of two or more rungs the blend weights are gated by a sequential surprise detector: a two-sided Page CUSUM on the studentized fast-vs-slow per-rung predictive-score drift (threshold set by Siegmund's average-run-length approximation at ~2 years, no tuned parameters) tilts the weights toward the fast or slow end of the ladder when one side accumulates statistically forced evidence, decaying back at the fastest rung's half-life. Weights equal the prior on all non-alarmed days, the blend stays convex, so PSD holds by construction; `None` (default) reproduces the published single-scale estimator exactly. This is the paper's **headline configuration**: on the S&P 500 benchmark it leads every tested method at every universe size (held-out one-step NLL −4.6 vs single-scale at n=100, −11.1 at n=300 before the cluster target), the 90% model confidence set collapses to it alone, and the detector's margin is confirmed out-of-time on an external industry panel (+0.53 NLL/day, p=1×10⁻⁴). Cost is O(K·n²) per update plus one Cholesky per rung per day for the detector scores. Composes with `shrinkage_target="cluster"`; mutually exclusive with `lambda_corr_fast`.
90
-
91
- ```python
92
- est = SqueezeKernelEstimator(n_assets=100, corr_half_lives=(43, 173, 693), corr_theta=0.25)
93
- # maximal variant at high dimension:
94
- est = SqueezeKernelEstimator(n_assets=300, corr_half_lives=(43, 173, 693), shrinkage_target="cluster")
95
- ```
96
-
97
- **Score-exact weighting** (`weight_statistic="mahalanobis"`, use with `kappa=1.0`): drives the kernel with the Mahalanobis surprise `z'C⁻¹z/N` against the estimator's own correlation instead of the marginal dispersion. Improves accuracy in the moderate-concentration regime — use only when `n / T_eff ≲ 0.5` (e.g. n ≤ 100 at the default `lambda_corr`); at higher concentration the estimated inverse degrades it and the default is strictly better.
98
-
99
- ```python
100
- est = SqueezeKernelEstimator(n_assets=100, kappa=1.0, weight_statistic="mahalanobis")
101
- ```
102
-
103
- **Score-driven memory** (`lambda_corr_fast=0.99`): lets stress days also *shorten* the correlation memory (decay slides from `lambda_corr` toward `lambda_corr_fast` as the kernel weight rises). Do **not** combine with the Mahalanobis option — they act on the same channel and the combination degrades accuracy.
104
-
105
- **OU volatility anchor** (`vol_anchor_phi=0.995`): mean-reverts each asset's variance prediction toward a slow per-asset anchor (a ~1000-day EWMA of squared returns) before the daily update — a two-timescale, component-style volatility structure. One global parameter with a clean interpretation (deviation half-life ≈ ln 2/(1−φ) days; φ=0.995 ≈ 139 d). On the S&P 500 n=100 benchmark this improved held-out one-step NLL by 3.3 points (4.3 at φ=0.99) and five-step NLL by 3.9 (5.0), with no degradation at n=300. `None` (default) or φ=1 reproduces the published estimator exactly.
106
-
107
- ```python
108
- est = SqueezeKernelEstimator(n_assets=100, vol_anchor_phi=0.995)
109
- ```
110
-
111
- **Cluster shrinkage target** (`shrinkage_target="cluster"`): generalizes the equicorrelation shrinkage target to respect the correlation matrix's own block/cluster structure — with **no clustering algorithm**. The target morphs with the shrinkage intensity, T = (1−α)·T_equi + α·[(1−γ)I + γ·(C∘C)], where C∘C is the Hadamard square of the current correlation (positive semi-definite by the Schur product theorem; entries are pairwise shared-variance fractions) and γ is level-matched automatically. Zero added parameters, still O(n²), and as α→0 it reduces exactly to the default estimator. Held-out one-step NLL on the S&P 500 benchmark: ±0.1 at n=100, **−4.2 at n=200, −25.0 at n=300** — recommended whenever the universe size approaches the effective sample size.
112
-
113
- ```python
114
- est = SqueezeKernelEstimator(n_assets=300, shrinkage_target="cluster")
115
- ```
116
-
117
- **New-listing usability gate** (`min_obs=60`): on an expanding universe, an asset's forecast rows are dominated by its single-observation variance initialization for its first weeks of life and are unusable for scoring or portfolio construction (measured ≈ +2,800 NLL/day on days whose scored set included such assets, on a 42-instrument multi-asset panel). `min_obs` gates nothing inside the estimator — states warm normally, all outputs are unchanged — it exposes a `usable_mask` property marking assets with at least `min_obs` finite observations, so deployments subset with it:
118
-
119
- ```python
120
- est = SqueezeKernelEstimator(n_assets=42, min_obs=60)
121
- # ... update loop ...
122
- m = est.usable_mask
123
- cov_usable = est.get_cov()[np.ix_(m, m)]
124
- ```
125
-
126
- **Alternative kernels**: pass `kernel_fn=kernel_exponential` (with `kernel_kwargs={"gamma": ...}`) or `kernel_chi2_cdf`, or any callable `(d2, *, n_observed, **kw) -> float` mapping to `[0, 1)`. The PSD guarantee holds for any such kernel.
127
-
128
- ## How it works
129
-
130
- Three mechanisms in one recursion:
131
-
132
- 1. **Dual-timescale EWMA** — fast per-asset volatility (`lambda_vol`) is separated from slow correlation dynamics (`lambda_corr`), so variance shocks don't contaminate the correlation estimate.
133
- 2. **Fisher kernel weighting** — each day's standardized outer product enters with weight `w = d²/(d² + kappa)`, where `d²` is the mean squared standardized return: calm days contribute little, dispersion shocks contribute fully.
134
- 3. **Adaptive equicorrelation shrinkage** — `alpha = min(1, max(0, n/(2·S) − delta))` blends toward an equicorrelation target using the estimator's own kernel-weighted sample size `S`; it is a no-op at low dimension and provides provably bounded conditioning at high dimension.
135
-
136
- The complete update is a natural-gradient step on the Gaussian log-likelihood, with the kernel weight acting as an adaptive Riemannian learning rate (paper, Appendix B).
137
-
138
- ## Development
139
-
140
- ```bash
141
- uv sync --extra full --extra dev
142
- uv run python -m pytest # test suite
143
- uv run python -m ruff check . # lint
144
- uv run mypy # strict type check (src/squeeze_kernel)
145
- uv build # build sdist + wheel
146
- ```
147
-
148
- Releases: publishing a GitHub release from a `v*` tag triggers the [publish workflow](.github/workflows/publish.yml), which builds and uploads to PyPI via trusted publishing.
149
-
150
- ## Citation
151
-
152
- ```bibtex
153
- @article{kende2026squeeze,
154
- title = {The Squeeze Kernel Covariance Estimator: Dual-Timescale Tracking with Adaptive Shrinkage},
155
- author = {Kende, Robert},
156
- year = {2026},
157
- note = {Available at SSRN: \url{https://ssrn.com/abstract=6455918}}
158
- }
159
- ```
160
-
161
- See also [`CITATION.cff`](CITATION.cff).
162
-
163
- ## License
164
-
165
- MIT