squeeze-kernel 0.7.0__tar.gz → 0.7.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {squeeze_kernel-0.7.0 → squeeze_kernel-0.7.1}/PKG-INFO +46 -35
- squeeze_kernel-0.7.1/README.md +176 -0
- {squeeze_kernel-0.7.0 → squeeze_kernel-0.7.1}/pyproject.toml +1 -1
- {squeeze_kernel-0.7.0 → squeeze_kernel-0.7.1}/pyproject.toml.orig +1 -1
- {squeeze_kernel-0.7.0 → squeeze_kernel-0.7.1}/src/squeeze_kernel/__init__.py +1 -1
- squeeze_kernel-0.7.0/README.md +0 -165
- {squeeze_kernel-0.7.0 → squeeze_kernel-0.7.1}/src/squeeze_kernel/batch.py +0 -0
- {squeeze_kernel-0.7.0 → squeeze_kernel-0.7.1}/src/squeeze_kernel/estimator.py +0 -0
- {squeeze_kernel-0.7.0 → squeeze_kernel-0.7.1}/src/squeeze_kernel/kernels.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: squeeze-kernel
|
|
3
|
-
Version: 0.7.
|
|
3
|
+
Version: 0.7.1
|
|
4
4
|
Summary: Streaming, PSD-by-construction covariance estimator with Fisher-kernel weighting and adaptive shrinkage
|
|
5
5
|
Keywords: covariance,correlation,ewma,kernel,risk,streaming
|
|
6
6
|
Author: Robert Kende
|
|
@@ -36,13 +36,13 @@ Description-Content-Type: text/markdown
|
|
|
36
36
|
[](https://pypi.org/project/squeeze-kernel/)
|
|
37
37
|
[](LICENSE)
|
|
38
38
|
|
|
39
|
-
A **streaming covariance estimator for panels of
|
|
39
|
+
A **streaming covariance estimator for panels of financial returns** that learns fastest on the days that matter. One `O(n²)` update per period, positive semi-definite **by construction** at every step, missing values handled **natively**, and defaults that require no tuning. Only dependency: NumPy.
|
|
40
40
|
|
|
41
|
-
|
|
41
|
+
References: *"The Squeeze Kernel Covariance Estimator: Dual-Timescale Tracking with Adaptive Shrinkage"* (Kende, 2026) — [SSRN abstract 6455918](https://ssrn.com/abstract=6455918) — and its companion *"Cluster-Respecting Shrinkage for Streaming Covariance Estimation"* (Kende, 2026).
|
|
42
42
|
|
|
43
43
|
## Why
|
|
44
44
|
|
|
45
|
-
|
|
45
|
+
Every standard covariance estimator treats all trading days as equally informative. Markets don't work that way: **correlations reveal themselves when markets move; calm days are mostly noise.** The Squeeze Kernel weighs each day by the information it actually carries — quiet days barely count, dispersion shocks pass through in full — and runs volatility and correlation on separate clocks, so vol spikes never contaminate the correlation estimate. The result is a single streaming recursion that:
|
|
46
46
|
|
|
47
47
|
- **is PSD at every step, structurally** — never needs eigenvalue clipping, nearest-PSD projection, or a solver;
|
|
48
48
|
- **adapts fastest exactly when it matters** — a Fisher-information kernel up-weights high-dispersion (stress) days, when correlation regimes actually move;
|
|
@@ -50,7 +50,37 @@ Rolling-window estimators (Ledoit–Wolf, nonlinear shrinkage, RMT denoising) re
|
|
|
50
50
|
- **ingests missing values natively** — listings, delistings, and halts enter as `NaN`; no imputation or complete-case subsetting;
|
|
51
51
|
- **is fast** — a full 30-year daily pass takes ~0.75 s at n=100 and ~3.4 s at n=300 (single-threaded), 30–40× faster than rolling-window baselines at scale.
|
|
52
52
|
|
|
53
|
-
On a 30-year S&P 500 panel (
|
|
53
|
+
**The scoreboard.** On a 30-year S&P 500 panel (~7,600 out-of-sample days) the default single-scale estimator beats EWMA, Ledoit–Wolf, OAS, nonlinear shrinkage, RMT denoising, and the Gerber statistic on one-step density forecasts, and statistically ties DCC — the only two methods in the 90% model confidence set. The headline configuration (multi-scale correlation memory, `corr_half_lives=(43, 173, 693)`) goes further: it **leads every tested method at every universe size and the 90% model confidence set collapses to it alone**, with the margin confirmed out-of-time on an external industry panel. For large equity universes, the cluster shrinkage target (`shrinkage_target="cluster"`) adds a further large gain exactly where shrinkage binds: 25 NLL points at n=300.
|
|
54
|
+
|
|
55
|
+
Against the alternatives:
|
|
56
|
+
|
|
57
|
+
- **RiskMetrics / EWMA** — same one-recursion simplicity, but two clocks and information weighting: better forecasts at zero extra operational cost.
|
|
58
|
+
- **Ledoit–Wolf, nonlinear shrinkage, RMT denoising** — static snapshots refit from scratch on a rolling window each day; the Squeeze Kernel is genuinely dynamic, more accurate on the benchmark, and 30–40× faster at scale.
|
|
59
|
+
- **DCC-GARCH** — matched (single-scale) or beaten (multi-scale) without multi-stage likelihood fitting or the fragile news-impact coefficient; at n=300 DCC needs a multi-year warm-up before its forecasts stabilise and still trails by ~25 NLL.
|
|
60
|
+
- **Gerber statistic** — the Squeeze Kernel is a PSD-by-construction generalization of the same robust-comovement idea: no nearest-PSD repair step, and it wins the head-to-head.
|
|
61
|
+
|
|
62
|
+
## See the difference
|
|
63
|
+
|
|
64
|
+
Take a passive strategy any allocator would recognize: a long-only minimum-variance portfolio of 300 liquid US stocks, scaled to a 15% volatility target, rebalanced once a month, with 5 bps trading costs. Run it twice on identical data. The only thing that changes between the two runs is the covariance matrix that picks the weights and sets the exposure.
|
|
65
|
+
|
|
66
|
+
<picture>
|
|
67
|
+
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/r0k3/squeeze-kernel/main/examples/figures/vol_targeted_portfolio_dark.png">
|
|
68
|
+
<img alt="Vol-targeted long-only minimum-variance portfolio on 300 US equities: Squeeze Kernel vs Ledoit-Wolf equity curves, drawdown, realized volatility, and risk/return profile" src="https://raw.githubusercontent.com/r0k3/squeeze-kernel/main/examples/figures/vol_targeted_portfolio.png">
|
|
69
|
+
</picture>
|
|
70
|
+
|
|
71
|
+
| Method | CAGR | Vol | Sharpe | MaxDD | Calmar | Vol-target RMSE |
|
|
72
|
+
|---|---|---|---|---|---|---|
|
|
73
|
+
| **Squeeze Kernel** | **13.1%** | **13.0%** | **1.00** | **-31.2%** | **0.42** | **6.47%** |
|
|
74
|
+
| Ledoit-Wolf (252d) | 11.7% | 14.6% | 0.80 | -38.7% | 0.30 | 7.51% |
|
|
75
|
+
|
|
76
|
+
You can regenerate this example from the repo alone. The returns panel ships as a parquet (daily returns for 300 US stocks, sourced from Yahoo Finance), and the script prints the table and redraws the figure:
|
|
77
|
+
|
|
78
|
+
```bash
|
|
79
|
+
pip install squeeze-kernel pandas pyarrow scikit-learn matplotlib
|
|
80
|
+
python examples/vol_targeted_portfolio.py # ~2 minutes
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
The full protocol and data notes live in [`examples/vol_targeted_portfolio.py`](examples/vol_targeted_portfolio.py) and [`examples/data/build_equity_panel.py`](examples/data/build_equity_panel.py).
|
|
54
84
|
|
|
55
85
|
## Installation
|
|
56
86
|
|
|
@@ -117,44 +147,27 @@ kappa = SqueezeKernelEstimator.calibrate_kappa(burn_in_returns, target_weight=0.
|
|
|
117
147
|
|
|
118
148
|
## Advanced options
|
|
119
149
|
|
|
120
|
-
|
|
150
|
+
All options are off by default; the defaults reproduce the published estimator exactly. Full derivations and benchmark tables are in the papers.
|
|
121
151
|
|
|
122
|
-
|
|
123
|
-
est = SqueezeKernelEstimator(n_assets=100, corr_half_lives=(43, 173, 693), corr_theta=0.25)
|
|
124
|
-
# maximal variant at high dimension:
|
|
125
|
-
est = SqueezeKernelEstimator(n_assets=300, corr_half_lives=(43, 173, 693), shrinkage_target="cluster")
|
|
126
|
-
```
|
|
152
|
+
**Multi-scale correlation memory** (`corr_half_lives=(43, 173, 693)`): replaces the single correlation timescale with a ladder of EWMAs whose blend a sequential surprise detector tilts toward fast or slow memory as the evidence demands. This is the papers' headline configuration: it leads every tested method at every universe size, and the blend stays convex so PSD still holds by construction.
|
|
127
153
|
|
|
128
|
-
**
|
|
154
|
+
**Cluster shrinkage target** (`shrinkage_target="cluster"`): shrinks toward a target that respects the correlation matrix's own block structure instead of a single equicorrelation, with no clustering algorithm and zero added parameters. The best choice for large equity universes: worth 4 held-out NLL points at n=200 and 25 at n=300 on the benchmark.
|
|
129
155
|
|
|
130
156
|
```python
|
|
131
|
-
|
|
157
|
+
# recommended setup for a large equity universe:
|
|
158
|
+
est = SqueezeKernelEstimator(n_assets=300, corr_half_lives=(43, 173, 693),
|
|
159
|
+
shrinkage_target="cluster")
|
|
132
160
|
```
|
|
133
161
|
|
|
134
|
-
**
|
|
135
|
-
|
|
136
|
-
**OU volatility anchor** (`vol_anchor_phi=0.995`): mean-reverts each asset's variance prediction toward a slow per-asset anchor (a ~1000-day EWMA of squared returns) before the daily update — a two-timescale, component-style volatility structure. One global parameter with a clean interpretation (deviation half-life ≈ ln 2/(1−φ) days; φ=0.995 ≈ 139 d). On the S&P 500 n=100 benchmark this improved held-out one-step NLL by 3.3 points (4.3 at φ=0.99) and five-step NLL by 3.9 (5.0), with no degradation at n=300. `None` (default) or φ=1 reproduces the published estimator exactly.
|
|
162
|
+
**OU volatility anchor** (`vol_anchor_phi=0.995`): mean-reverts each asset's variance forecast toward a slow per-asset anchor, giving a two-timescale volatility structure with a single parameter (deviation half-life ≈ ln 2/(1−φ) days). Worth 3–5 NLL points on the benchmark.
|
|
137
163
|
|
|
138
|
-
|
|
139
|
-
est = SqueezeKernelEstimator(n_assets=100, vol_anchor_phi=0.995)
|
|
140
|
-
```
|
|
164
|
+
**Score-exact weighting** (`weight_statistic="mahalanobis"`, use with `kappa=1.0`): drives the kernel with the Mahalanobis surprise against the estimator's own correlation. Use it only in the moderate-concentration regime (`n / T_eff ≲ 0.5`).
|
|
141
165
|
|
|
142
|
-
**
|
|
166
|
+
**Score-driven memory** (`lambda_corr_fast=0.99`): lets stress days also shorten the correlation memory. Do not combine with the Mahalanobis option.
|
|
143
167
|
|
|
144
|
-
|
|
145
|
-
est = SqueezeKernelEstimator(n_assets=300, shrinkage_target="cluster")
|
|
146
|
-
```
|
|
168
|
+
**New-listing usability gate** (`min_obs=60`): exposes a `usable_mask` property marking assets with at least `min_obs` observations, so deployments can exclude cold starts from scoring and optimization; the estimates themselves are unchanged.
|
|
147
169
|
|
|
148
|
-
**
|
|
149
|
-
|
|
150
|
-
```python
|
|
151
|
-
est = SqueezeKernelEstimator(n_assets=42, min_obs=60)
|
|
152
|
-
# ... update loop ...
|
|
153
|
-
m = est.usable_mask
|
|
154
|
-
cov_usable = est.get_cov()[np.ix_(m, m)]
|
|
155
|
-
```
|
|
156
|
-
|
|
157
|
-
**Alternative kernels**: pass `kernel_fn=kernel_exponential` (with `kernel_kwargs={"gamma": ...}`) or `kernel_chi2_cdf`, or any callable `(d2, *, n_observed, **kw) -> float` mapping to `[0, 1)`. The PSD guarantee holds for any such kernel.
|
|
170
|
+
**Alternative kernels**: pass `kernel_fn=kernel_exponential` or `kernel_chi2_cdf`, or any callable mapping to `[0, 1)`; the PSD guarantee holds for any such kernel.
|
|
158
171
|
|
|
159
172
|
## How it works
|
|
160
173
|
|
|
@@ -176,8 +189,6 @@ uv run mypy # strict type check (src/squeeze_kernel)
|
|
|
176
189
|
uv build # build sdist + wheel
|
|
177
190
|
```
|
|
178
191
|
|
|
179
|
-
Releases: publishing a GitHub release from a `v*` tag triggers the [publish workflow](.github/workflows/publish.yml), which builds and uploads to PyPI via trusted publishing.
|
|
180
|
-
|
|
181
192
|
## Citation
|
|
182
193
|
|
|
183
194
|
```bibtex
|
|
@@ -0,0 +1,176 @@
|
|
|
1
|
+
# Squeeze Kernel Covariance Estimator
|
|
2
|
+
|
|
3
|
+
[](https://github.com/r0k3/squeeze-kernel/actions/workflows/ci.yml)
|
|
4
|
+
[](https://pypi.org/project/squeeze-kernel/)
|
|
5
|
+
[](https://pypi.org/project/squeeze-kernel/)
|
|
6
|
+
[](LICENSE)
|
|
7
|
+
|
|
8
|
+
A **streaming covariance estimator for panels of financial returns** that learns fastest on the days that matter. One `O(n²)` update per period, positive semi-definite **by construction** at every step, missing values handled **natively**, and defaults that require no tuning. Only dependency: NumPy.
|
|
9
|
+
|
|
10
|
+
References: *"The Squeeze Kernel Covariance Estimator: Dual-Timescale Tracking with Adaptive Shrinkage"* (Kende, 2026) — [SSRN abstract 6455918](https://ssrn.com/abstract=6455918) — and its companion *"Cluster-Respecting Shrinkage for Streaming Covariance Estimation"* (Kende, 2026).
|
|
11
|
+
|
|
12
|
+
## Why
|
|
13
|
+
|
|
14
|
+
Every standard covariance estimator treats all trading days as equally informative. Markets don't work that way: **correlations reveal themselves when markets move; calm days are mostly noise.** The Squeeze Kernel weighs each day by the information it actually carries — quiet days barely count, dispersion shocks pass through in full — and runs volatility and correlation on separate clocks, so vol spikes never contaminate the correlation estimate. The result is a single streaming recursion that:
|
|
15
|
+
|
|
16
|
+
- **is PSD at every step, structurally** — never needs eigenvalue clipping, nearest-PSD projection, or a solver;
|
|
17
|
+
- **adapts fastest exactly when it matters** — a Fisher-information kernel up-weights high-dispersion (stress) days, when correlation regimes actually move;
|
|
18
|
+
- **regularises itself** — an adaptive equicorrelation shrinkage activates automatically as the asset count approaches the effective sample size, with a provable condition-number bound;
|
|
19
|
+
- **ingests missing values natively** — listings, delistings, and halts enter as `NaN`; no imputation or complete-case subsetting;
|
|
20
|
+
- **is fast** — a full 30-year daily pass takes ~0.75 s at n=100 and ~3.4 s at n=300 (single-threaded), 30–40× faster than rolling-window baselines at scale.
|
|
21
|
+
|
|
22
|
+
**The scoreboard.** On a 30-year S&P 500 panel (~7,600 out-of-sample days) the default single-scale estimator beats EWMA, Ledoit–Wolf, OAS, nonlinear shrinkage, RMT denoising, and the Gerber statistic on one-step density forecasts, and statistically ties DCC — the only two methods in the 90% model confidence set. The headline configuration (multi-scale correlation memory, `corr_half_lives=(43, 173, 693)`) goes further: it **leads every tested method at every universe size and the 90% model confidence set collapses to it alone**, with the margin confirmed out-of-time on an external industry panel. For large equity universes, the cluster shrinkage target (`shrinkage_target="cluster"`) adds a further large gain exactly where shrinkage binds: 25 NLL points at n=300.
|
|
23
|
+
|
|
24
|
+
Against the alternatives:
|
|
25
|
+
|
|
26
|
+
- **RiskMetrics / EWMA** — same one-recursion simplicity, but two clocks and information weighting: better forecasts at zero extra operational cost.
|
|
27
|
+
- **Ledoit–Wolf, nonlinear shrinkage, RMT denoising** — static snapshots refit from scratch on a rolling window each day; the Squeeze Kernel is genuinely dynamic, more accurate on the benchmark, and 30–40× faster at scale.
|
|
28
|
+
- **DCC-GARCH** — matched (single-scale) or beaten (multi-scale) without multi-stage likelihood fitting or the fragile news-impact coefficient; at n=300 DCC needs a multi-year warm-up before its forecasts stabilise and still trails by ~25 NLL.
|
|
29
|
+
- **Gerber statistic** — the Squeeze Kernel is a PSD-by-construction generalization of the same robust-comovement idea: no nearest-PSD repair step, and it wins the head-to-head.
|
|
30
|
+
|
|
31
|
+
## See the difference
|
|
32
|
+
|
|
33
|
+
Take a passive strategy any allocator would recognize: a long-only minimum-variance portfolio of 300 liquid US stocks, scaled to a 15% volatility target, rebalanced once a month, with 5 bps trading costs. Run it twice on identical data. The only thing that changes between the two runs is the covariance matrix that picks the weights and sets the exposure.
|
|
34
|
+
|
|
35
|
+
<picture>
|
|
36
|
+
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/r0k3/squeeze-kernel/main/examples/figures/vol_targeted_portfolio_dark.png">
|
|
37
|
+
<img alt="Vol-targeted long-only minimum-variance portfolio on 300 US equities: Squeeze Kernel vs Ledoit-Wolf equity curves, drawdown, realized volatility, and risk/return profile" src="https://raw.githubusercontent.com/r0k3/squeeze-kernel/main/examples/figures/vol_targeted_portfolio.png">
|
|
38
|
+
</picture>
|
|
39
|
+
|
|
40
|
+
| Method | CAGR | Vol | Sharpe | MaxDD | Calmar | Vol-target RMSE |
|
|
41
|
+
|---|---|---|---|---|---|---|
|
|
42
|
+
| **Squeeze Kernel** | **13.1%** | **13.0%** | **1.00** | **-31.2%** | **0.42** | **6.47%** |
|
|
43
|
+
| Ledoit-Wolf (252d) | 11.7% | 14.6% | 0.80 | -38.7% | 0.30 | 7.51% |
|
|
44
|
+
|
|
45
|
+
You can regenerate this example from the repo alone. The returns panel ships as a parquet (daily returns for 300 US stocks, sourced from Yahoo Finance), and the script prints the table and redraws the figure:
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
pip install squeeze-kernel pandas pyarrow scikit-learn matplotlib
|
|
49
|
+
python examples/vol_targeted_portfolio.py # ~2 minutes
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
The full protocol and data notes live in [`examples/vol_targeted_portfolio.py`](examples/vol_targeted_portfolio.py) and [`examples/data/build_equity_panel.py`](examples/data/build_equity_panel.py).
|
|
53
|
+
|
|
54
|
+
## Installation
|
|
55
|
+
|
|
56
|
+
```bash
|
|
57
|
+
pip install squeeze-kernel # NumPy only
|
|
58
|
+
pip install "squeeze-kernel[full]" # + SciPy (kappa calibration, chi² kernel)
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
## Quickstart
|
|
62
|
+
|
|
63
|
+
```python
|
|
64
|
+
import numpy as np
|
|
65
|
+
from squeeze_kernel import SqueezeKernelEstimator
|
|
66
|
+
|
|
67
|
+
# daily_returns: array of shape (T, n) — may contain NaN for missing assets
|
|
68
|
+
est = SqueezeKernelEstimator(n_assets=daily_returns.shape[1])
|
|
69
|
+
|
|
70
|
+
for r_t in daily_returns: # stream one day at a time
|
|
71
|
+
est.update(r_t)
|
|
72
|
+
|
|
73
|
+
cov = est.get_cov() # (n, n) covariance, PSD by construction
|
|
74
|
+
corr = est.get_corr() # (n, n) correlation
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
That is the whole API for most uses. The defaults (`lambda_vol=0.98`, `lambda_corr=0.996`, `kappa=0.25`) are the paper-recommended settings for daily returns, selected by time-series cross-validation and robust across a 50× parameter sweep — deploy them as-is.
|
|
78
|
+
|
|
79
|
+
Batch mode, if you prefer the full path in one call:
|
|
80
|
+
|
|
81
|
+
```python
|
|
82
|
+
from squeeze_kernel import estimate_squeeze_cov
|
|
83
|
+
|
|
84
|
+
cov_path, corr_path, weights = estimate_squeeze_cov(daily_returns, with_weights=True)
|
|
85
|
+
# cov_path: (T, n, n) — the estimate after each day
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
A complete runnable walkthrough (streaming, missing data, batch) is in [`examples/quickstart.py`](examples/quickstart.py).
|
|
89
|
+
|
|
90
|
+
## Missing values
|
|
91
|
+
|
|
92
|
+
Pass `NaN` for any asset not observed on a given day — nothing else to do:
|
|
93
|
+
|
|
94
|
+
```python
|
|
95
|
+
r_t = np.array([0.004, np.nan, -0.011]) # asset 2 not trading today
|
|
96
|
+
est.update(r_t) # PSD preserved, no imputation
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
## Parameters
|
|
100
|
+
|
|
101
|
+
| Parameter | Default | Meaning |
|
|
102
|
+
|---|---|---|
|
|
103
|
+
| `lambda_vol` | `0.98` | volatility EWMA decay (half-life ≈ 34 days) |
|
|
104
|
+
| `lambda_corr` | `0.996` | correlation EWMA decay (half-life ≈ 173 days, T_eff ≈ 250) |
|
|
105
|
+
| `kappa` | `0.25` | Fisher kernel saturation; higher = stronger calm-day filtering |
|
|
106
|
+
| `shrinkage` | `"auto"` | adaptive equicorrelation shrinkage (`"none"` or a float to override) |
|
|
107
|
+
| `shrinkage_delta` | `0.10` | concentration threshold at which shrinkage activates |
|
|
108
|
+
|
|
109
|
+
Useful read-only state after each `update()`: `est.weight` (last kernel weight), `est.effective_sample_size` (kernel-weighted T_eff), `est.shrinkage_intensity` (current α).
|
|
110
|
+
|
|
111
|
+
To recalibrate `kappa` for a different asset class (requires the `full` extra):
|
|
112
|
+
|
|
113
|
+
```python
|
|
114
|
+
kappa = SqueezeKernelEstimator.calibrate_kappa(burn_in_returns, target_weight=0.6)
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
## Advanced options
|
|
118
|
+
|
|
119
|
+
All options are off by default; the defaults reproduce the published estimator exactly. Full derivations and benchmark tables are in the papers.
|
|
120
|
+
|
|
121
|
+
**Multi-scale correlation memory** (`corr_half_lives=(43, 173, 693)`): replaces the single correlation timescale with a ladder of EWMAs whose blend a sequential surprise detector tilts toward fast or slow memory as the evidence demands. This is the papers' headline configuration: it leads every tested method at every universe size, and the blend stays convex so PSD still holds by construction.
|
|
122
|
+
|
|
123
|
+
**Cluster shrinkage target** (`shrinkage_target="cluster"`): shrinks toward a target that respects the correlation matrix's own block structure instead of a single equicorrelation, with no clustering algorithm and zero added parameters. The best choice for large equity universes: worth 4 held-out NLL points at n=200 and 25 at n=300 on the benchmark.
|
|
124
|
+
|
|
125
|
+
```python
|
|
126
|
+
# recommended setup for a large equity universe:
|
|
127
|
+
est = SqueezeKernelEstimator(n_assets=300, corr_half_lives=(43, 173, 693),
|
|
128
|
+
shrinkage_target="cluster")
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
**OU volatility anchor** (`vol_anchor_phi=0.995`): mean-reverts each asset's variance forecast toward a slow per-asset anchor, giving a two-timescale volatility structure with a single parameter (deviation half-life ≈ ln 2/(1−φ) days). Worth 3–5 NLL points on the benchmark.
|
|
132
|
+
|
|
133
|
+
**Score-exact weighting** (`weight_statistic="mahalanobis"`, use with `kappa=1.0`): drives the kernel with the Mahalanobis surprise against the estimator's own correlation. Use it only in the moderate-concentration regime (`n / T_eff ≲ 0.5`).
|
|
134
|
+
|
|
135
|
+
**Score-driven memory** (`lambda_corr_fast=0.99`): lets stress days also shorten the correlation memory. Do not combine with the Mahalanobis option.
|
|
136
|
+
|
|
137
|
+
**New-listing usability gate** (`min_obs=60`): exposes a `usable_mask` property marking assets with at least `min_obs` observations, so deployments can exclude cold starts from scoring and optimization; the estimates themselves are unchanged.
|
|
138
|
+
|
|
139
|
+
**Alternative kernels**: pass `kernel_fn=kernel_exponential` or `kernel_chi2_cdf`, or any callable mapping to `[0, 1)`; the PSD guarantee holds for any such kernel.
|
|
140
|
+
|
|
141
|
+
## How it works
|
|
142
|
+
|
|
143
|
+
Three mechanisms in one recursion:
|
|
144
|
+
|
|
145
|
+
1. **Dual-timescale EWMA** — fast per-asset volatility (`lambda_vol`) is separated from slow correlation dynamics (`lambda_corr`), so variance shocks don't contaminate the correlation estimate.
|
|
146
|
+
2. **Fisher kernel weighting** — each day's standardized outer product enters with weight `w = d²/(d² + kappa)`, where `d²` is the mean squared standardized return: calm days contribute little, dispersion shocks contribute fully.
|
|
147
|
+
3. **Adaptive equicorrelation shrinkage** — `alpha = min(1, max(0, n/(2·S) − delta))` blends toward an equicorrelation target using the estimator's own kernel-weighted sample size `S`; it is a no-op at low dimension and provides provably bounded conditioning at high dimension.
|
|
148
|
+
|
|
149
|
+
The complete update is a natural-gradient step on the Gaussian log-likelihood, with the kernel weight acting as an adaptive Riemannian learning rate (paper, Appendix B).
|
|
150
|
+
|
|
151
|
+
## Development
|
|
152
|
+
|
|
153
|
+
```bash
|
|
154
|
+
uv sync --extra full --extra dev
|
|
155
|
+
uv run python -m pytest # test suite
|
|
156
|
+
uv run python -m ruff check . # lint
|
|
157
|
+
uv run mypy # strict type check (src/squeeze_kernel)
|
|
158
|
+
uv build # build sdist + wheel
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
## Citation
|
|
162
|
+
|
|
163
|
+
```bibtex
|
|
164
|
+
@article{kende2026squeeze,
|
|
165
|
+
title = {The Squeeze Kernel Covariance Estimator: Dual-Timescale Tracking with Adaptive Shrinkage},
|
|
166
|
+
author = {Kende, Robert},
|
|
167
|
+
year = {2026},
|
|
168
|
+
note = {Available at SSRN: \url{https://ssrn.com/abstract=6455918}}
|
|
169
|
+
}
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
See also [`CITATION.cff`](CITATION.cff).
|
|
173
|
+
|
|
174
|
+
## License
|
|
175
|
+
|
|
176
|
+
MIT
|
|
@@ -4,7 +4,7 @@ build-backend = "uv_build"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "squeeze-kernel"
|
|
7
|
-
version = "0.7.
|
|
7
|
+
version = "0.7.1"
|
|
8
8
|
description = "Streaming, PSD-by-construction covariance estimator with Fisher-kernel weighting and adaptive shrinkage"
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
license = "MIT"
|
|
@@ -4,7 +4,7 @@ build-backend = "uv_build"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "squeeze-kernel"
|
|
7
|
-
version = "0.7.
|
|
7
|
+
version = "0.7.1"
|
|
8
8
|
description = "Streaming, PSD-by-construction covariance estimator with Fisher-kernel weighting and adaptive shrinkage"
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
license = "MIT"
|
squeeze_kernel-0.7.0/README.md
DELETED
|
@@ -1,165 +0,0 @@
|
|
|
1
|
-
# Squeeze Kernel Covariance Estimator
|
|
2
|
-
|
|
3
|
-
[](https://github.com/r0k3/squeeze-kernel/actions/workflows/ci.yml)
|
|
4
|
-
[](https://pypi.org/project/squeeze-kernel/)
|
|
5
|
-
[](https://pypi.org/project/squeeze-kernel/)
|
|
6
|
-
[](LICENSE)
|
|
7
|
-
|
|
8
|
-
A **streaming covariance estimator for panels of daily financial returns**. One `O(n²)` update per day, positive semi-definite **by construction** at every step, missing values handled **natively**, and defaults that require no tuning. Only dependency: NumPy.
|
|
9
|
-
|
|
10
|
-
Reference: *"The Squeeze Kernel Covariance Estimator: Dual-Timescale Tracking with Adaptive Shrinkage"* (Kende, 2026) — [SSRN abstract 6455918](https://ssrn.com/abstract=6455918).
|
|
11
|
-
|
|
12
|
-
## Why
|
|
13
|
-
|
|
14
|
-
Rolling-window estimators (Ledoit–Wolf, nonlinear shrinkage, RMT denoising) refit over a fixed window each day and cannot adapt within it; multivariate GARCH (DCC) adapts but needs multi-stage estimation and a fragile news coefficient. The Squeeze Kernel is a single streaming recursion that:
|
|
15
|
-
|
|
16
|
-
- **is PSD at every step, structurally** — never needs eigenvalue clipping, nearest-PSD projection, or a solver;
|
|
17
|
-
- **adapts fastest exactly when it matters** — a Fisher-information kernel up-weights high-dispersion (stress) days, when correlation regimes actually move;
|
|
18
|
-
- **regularises itself** — an adaptive equicorrelation shrinkage activates automatically as the asset count approaches the effective sample size, with a provable condition-number bound;
|
|
19
|
-
- **ingests missing values natively** — listings, delistings, and halts enter as `NaN`; no imputation or complete-case subsetting;
|
|
20
|
-
- **is fast** — a full 30-year daily pass takes ~0.75 s at n=100 and ~3.4 s at n=300 (single-threaded), 30–40× faster than rolling-window baselines at scale.
|
|
21
|
-
|
|
22
|
-
On a 30-year S&P 500 panel (n=100, ~7,600 out-of-sample days) it statistically ties DCC on one-step density forecasts and beats EWMA, Ledoit–Wolf, OAS, nonlinear shrinkage, RMT denoising, and the Gerber statistic — and it is the only method in the 90% model confidence set together with DCC. At n=300 it leads every competitor that remains statistically viable.
|
|
23
|
-
|
|
24
|
-
## Installation
|
|
25
|
-
|
|
26
|
-
```bash
|
|
27
|
-
pip install squeeze-kernel # NumPy only
|
|
28
|
-
pip install "squeeze-kernel[full]" # + SciPy (kappa calibration, chi² kernel)
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
## Quickstart
|
|
32
|
-
|
|
33
|
-
```python
|
|
34
|
-
import numpy as np
|
|
35
|
-
from squeeze_kernel import SqueezeKernelEstimator
|
|
36
|
-
|
|
37
|
-
# daily_returns: array of shape (T, n) — may contain NaN for missing assets
|
|
38
|
-
est = SqueezeKernelEstimator(n_assets=daily_returns.shape[1])
|
|
39
|
-
|
|
40
|
-
for r_t in daily_returns: # stream one day at a time
|
|
41
|
-
est.update(r_t)
|
|
42
|
-
|
|
43
|
-
cov = est.get_cov() # (n, n) covariance, PSD by construction
|
|
44
|
-
corr = est.get_corr() # (n, n) correlation
|
|
45
|
-
```
|
|
46
|
-
|
|
47
|
-
That is the whole API for most uses. The defaults (`lambda_vol=0.98`, `lambda_corr=0.996`, `kappa=0.25`) are the paper-recommended settings for daily returns, selected by time-series cross-validation and robust across a 50× parameter sweep — deploy them as-is.
|
|
48
|
-
|
|
49
|
-
Batch mode, if you prefer the full path in one call:
|
|
50
|
-
|
|
51
|
-
```python
|
|
52
|
-
from squeeze_kernel import estimate_squeeze_cov
|
|
53
|
-
|
|
54
|
-
cov_path, corr_path, weights = estimate_squeeze_cov(daily_returns, with_weights=True)
|
|
55
|
-
# cov_path: (T, n, n) — the estimate after each day
|
|
56
|
-
```
|
|
57
|
-
|
|
58
|
-
A complete runnable walkthrough (streaming, missing data, batch) is in [`examples/quickstart.py`](examples/quickstart.py).
|
|
59
|
-
|
|
60
|
-
## Missing values
|
|
61
|
-
|
|
62
|
-
Pass `NaN` for any asset not observed on a given day — nothing else to do:
|
|
63
|
-
|
|
64
|
-
```python
|
|
65
|
-
r_t = np.array([0.004, np.nan, -0.011]) # asset 2 not trading today
|
|
66
|
-
est.update(r_t) # PSD preserved, no imputation
|
|
67
|
-
```
|
|
68
|
-
|
|
69
|
-
## Parameters
|
|
70
|
-
|
|
71
|
-
| Parameter | Default | Meaning |
|
|
72
|
-
|---|---|---|
|
|
73
|
-
| `lambda_vol` | `0.98` | volatility EWMA decay (half-life ≈ 34 days) |
|
|
74
|
-
| `lambda_corr` | `0.996` | correlation EWMA decay (half-life ≈ 173 days, T_eff ≈ 250) |
|
|
75
|
-
| `kappa` | `0.25` | Fisher kernel saturation; higher = stronger calm-day filtering |
|
|
76
|
-
| `shrinkage` | `"auto"` | adaptive equicorrelation shrinkage (`"none"` or a float to override) |
|
|
77
|
-
| `shrinkage_delta` | `0.10` | concentration threshold at which shrinkage activates |
|
|
78
|
-
|
|
79
|
-
Useful read-only state after each `update()`: `est.weight` (last kernel weight), `est.effective_sample_size` (kernel-weighted T_eff), `est.shrinkage_intensity` (current α).
|
|
80
|
-
|
|
81
|
-
To recalibrate `kappa` for a different asset class (requires the `full` extra):
|
|
82
|
-
|
|
83
|
-
```python
|
|
84
|
-
kappa = SqueezeKernelEstimator.calibrate_kappa(burn_in_returns, target_weight=0.6)
|
|
85
|
-
```
|
|
86
|
-
|
|
87
|
-
## Advanced options
|
|
88
|
-
|
|
89
|
-
**Adaptive scale-free correlation memory** (`corr_half_lives=(43, 173, 693)`, `corr_theta=0.25`): replaces the single correlation timescale with a positive combination of EWMAs on a geometric half-life ladder — each scale normalized and adaptively shrunk against its own effective sample size, then the covariances blended with weights resting at the prior ∝ half-life^`corr_theta`. By Bernstein's theorem the ladder approximates the power-law memory of financial correlations (the streaming analogue of HAR). For ladders of two or more rungs the blend weights are gated by a sequential surprise detector: a two-sided Page CUSUM on the studentized fast-vs-slow per-rung predictive-score drift (threshold set by Siegmund's average-run-length approximation at ~2 years, no tuned parameters) tilts the weights toward the fast or slow end of the ladder when one side accumulates statistically forced evidence, decaying back at the fastest rung's half-life. Weights equal the prior on all non-alarmed days, the blend stays convex, so PSD holds by construction; `None` (default) reproduces the published single-scale estimator exactly. This is the paper's **headline configuration**: on the S&P 500 benchmark it leads every tested method at every universe size (held-out one-step NLL −4.6 vs single-scale at n=100, −11.1 at n=300 before the cluster target), the 90% model confidence set collapses to it alone, and the detector's margin is confirmed out-of-time on an external industry panel (+0.53 NLL/day, p=1×10⁻⁴). Cost is O(K·n²) per update plus one Cholesky per rung per day for the detector scores. Composes with `shrinkage_target="cluster"`; mutually exclusive with `lambda_corr_fast`.
|
|
90
|
-
|
|
91
|
-
```python
|
|
92
|
-
est = SqueezeKernelEstimator(n_assets=100, corr_half_lives=(43, 173, 693), corr_theta=0.25)
|
|
93
|
-
# maximal variant at high dimension:
|
|
94
|
-
est = SqueezeKernelEstimator(n_assets=300, corr_half_lives=(43, 173, 693), shrinkage_target="cluster")
|
|
95
|
-
```
|
|
96
|
-
|
|
97
|
-
**Score-exact weighting** (`weight_statistic="mahalanobis"`, use with `kappa=1.0`): drives the kernel with the Mahalanobis surprise `z'C⁻¹z/N` against the estimator's own correlation instead of the marginal dispersion. Improves accuracy in the moderate-concentration regime — use only when `n / T_eff ≲ 0.5` (e.g. n ≤ 100 at the default `lambda_corr`); at higher concentration the estimated inverse degrades it and the default is strictly better.
|
|
98
|
-
|
|
99
|
-
```python
|
|
100
|
-
est = SqueezeKernelEstimator(n_assets=100, kappa=1.0, weight_statistic="mahalanobis")
|
|
101
|
-
```
|
|
102
|
-
|
|
103
|
-
**Score-driven memory** (`lambda_corr_fast=0.99`): lets stress days also *shorten* the correlation memory (decay slides from `lambda_corr` toward `lambda_corr_fast` as the kernel weight rises). Do **not** combine with the Mahalanobis option — they act on the same channel and the combination degrades accuracy.
|
|
104
|
-
|
|
105
|
-
**OU volatility anchor** (`vol_anchor_phi=0.995`): mean-reverts each asset's variance prediction toward a slow per-asset anchor (a ~1000-day EWMA of squared returns) before the daily update — a two-timescale, component-style volatility structure. One global parameter with a clean interpretation (deviation half-life ≈ ln 2/(1−φ) days; φ=0.995 ≈ 139 d). On the S&P 500 n=100 benchmark this improved held-out one-step NLL by 3.3 points (4.3 at φ=0.99) and five-step NLL by 3.9 (5.0), with no degradation at n=300. `None` (default) or φ=1 reproduces the published estimator exactly.
|
|
106
|
-
|
|
107
|
-
```python
|
|
108
|
-
est = SqueezeKernelEstimator(n_assets=100, vol_anchor_phi=0.995)
|
|
109
|
-
```
|
|
110
|
-
|
|
111
|
-
**Cluster shrinkage target** (`shrinkage_target="cluster"`): generalizes the equicorrelation shrinkage target to respect the correlation matrix's own block/cluster structure — with **no clustering algorithm**. The target morphs with the shrinkage intensity, T = (1−α)·T_equi + α·[(1−γ)I + γ·(C∘C)], where C∘C is the Hadamard square of the current correlation (positive semi-definite by the Schur product theorem; entries are pairwise shared-variance fractions) and γ is level-matched automatically. Zero added parameters, still O(n²), and as α→0 it reduces exactly to the default estimator. Held-out one-step NLL on the S&P 500 benchmark: ±0.1 at n=100, **−4.2 at n=200, −25.0 at n=300** — recommended whenever the universe size approaches the effective sample size.
|
|
112
|
-
|
|
113
|
-
```python
|
|
114
|
-
est = SqueezeKernelEstimator(n_assets=300, shrinkage_target="cluster")
|
|
115
|
-
```
|
|
116
|
-
|
|
117
|
-
**New-listing usability gate** (`min_obs=60`): on an expanding universe, an asset's forecast rows are dominated by its single-observation variance initialization for its first weeks of life and are unusable for scoring or portfolio construction (measured ≈ +2,800 NLL/day on days whose scored set included such assets, on a 42-instrument multi-asset panel). `min_obs` gates nothing inside the estimator — states warm normally, all outputs are unchanged — it exposes a `usable_mask` property marking assets with at least `min_obs` finite observations, so deployments subset with it:
|
|
118
|
-
|
|
119
|
-
```python
|
|
120
|
-
est = SqueezeKernelEstimator(n_assets=42, min_obs=60)
|
|
121
|
-
# ... update loop ...
|
|
122
|
-
m = est.usable_mask
|
|
123
|
-
cov_usable = est.get_cov()[np.ix_(m, m)]
|
|
124
|
-
```
|
|
125
|
-
|
|
126
|
-
**Alternative kernels**: pass `kernel_fn=kernel_exponential` (with `kernel_kwargs={"gamma": ...}`) or `kernel_chi2_cdf`, or any callable `(d2, *, n_observed, **kw) -> float` mapping to `[0, 1)`. The PSD guarantee holds for any such kernel.
|
|
127
|
-
|
|
128
|
-
## How it works
|
|
129
|
-
|
|
130
|
-
Three mechanisms in one recursion:
|
|
131
|
-
|
|
132
|
-
1. **Dual-timescale EWMA** — fast per-asset volatility (`lambda_vol`) is separated from slow correlation dynamics (`lambda_corr`), so variance shocks don't contaminate the correlation estimate.
|
|
133
|
-
2. **Fisher kernel weighting** — each day's standardized outer product enters with weight `w = d²/(d² + kappa)`, where `d²` is the mean squared standardized return: calm days contribute little, dispersion shocks contribute fully.
|
|
134
|
-
3. **Adaptive equicorrelation shrinkage** — `alpha = min(1, max(0, n/(2·S) − delta))` blends toward an equicorrelation target using the estimator's own kernel-weighted sample size `S`; it is a no-op at low dimension and provides provably bounded conditioning at high dimension.
|
|
135
|
-
|
|
136
|
-
The complete update is a natural-gradient step on the Gaussian log-likelihood, with the kernel weight acting as an adaptive Riemannian learning rate (paper, Appendix B).
|
|
137
|
-
|
|
138
|
-
## Development
|
|
139
|
-
|
|
140
|
-
```bash
|
|
141
|
-
uv sync --extra full --extra dev
|
|
142
|
-
uv run python -m pytest # test suite
|
|
143
|
-
uv run python -m ruff check . # lint
|
|
144
|
-
uv run mypy # strict type check (src/squeeze_kernel)
|
|
145
|
-
uv build # build sdist + wheel
|
|
146
|
-
```
|
|
147
|
-
|
|
148
|
-
Releases: publishing a GitHub release from a `v*` tag triggers the [publish workflow](.github/workflows/publish.yml), which builds and uploads to PyPI via trusted publishing.
|
|
149
|
-
|
|
150
|
-
## Citation
|
|
151
|
-
|
|
152
|
-
```bibtex
|
|
153
|
-
@article{kende2026squeeze,
|
|
154
|
-
title = {The Squeeze Kernel Covariance Estimator: Dual-Timescale Tracking with Adaptive Shrinkage},
|
|
155
|
-
author = {Kende, Robert},
|
|
156
|
-
year = {2026},
|
|
157
|
-
note = {Available at SSRN: \url{https://ssrn.com/abstract=6455918}}
|
|
158
|
-
}
|
|
159
|
-
```
|
|
160
|
-
|
|
161
|
-
See also [`CITATION.cff`](CITATION.cff).
|
|
162
|
-
|
|
163
|
-
## License
|
|
164
|
-
|
|
165
|
-
MIT
|
|
File without changes
|
|
File without changes
|
|
File without changes
|