qkit-research 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- qkit_research-0.1.0/LICENSE +21 -0
- qkit_research-0.1.0/MANIFEST.in +10 -0
- qkit_research-0.1.0/PACKAGE_README.md +124 -0
- qkit_research-0.1.0/PKG-INFO +143 -0
- qkit_research-0.1.0/README.md +95 -0
- qkit_research-0.1.0/pyproject.toml +29 -0
- qkit_research-0.1.0/qkit/__init__.py +28 -0
- qkit_research-0.1.0/qkit/cli.py +68 -0
- qkit_research-0.1.0/qkit/lookahead.py +103 -0
- qkit_research-0.1.0/qkit/statistics.py +103 -0
- qkit_research-0.1.0/qkit/universe.py +53 -0
- qkit_research-0.1.0/qkit/validation.py +68 -0
- qkit_research-0.1.0/qkit_research.egg-info/PKG-INFO +143 -0
- qkit_research-0.1.0/qkit_research.egg-info/SOURCES.txt +18 -0
- qkit_research-0.1.0/qkit_research.egg-info/dependency_links.txt +1 -0
- qkit_research-0.1.0/qkit_research.egg-info/entry_points.txt +2 -0
- qkit_research-0.1.0/qkit_research.egg-info/requires.txt +4 -0
- qkit_research-0.1.0/qkit_research.egg-info/top_level.txt +1 -0
- qkit_research-0.1.0/requirements.txt +2 -0
- qkit_research-0.1.0/setup.cfg +4 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Wenke Du
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
include LICENSE
|
|
2
|
+
include PACKAGE_README.md
|
|
3
|
+
include requirements.txt
|
|
4
|
+
recursive-exclude tests *
|
|
5
|
+
recursive-exclude src *
|
|
6
|
+
recursive-exclude website *
|
|
7
|
+
recursive-exclude results *
|
|
8
|
+
recursive-exclude data *
|
|
9
|
+
recursive-exclude docs *
|
|
10
|
+
global-exclude __pycache__ *.py[cod] .DS_Store
|
|
@@ -0,0 +1,124 @@
|
|
|
1
|
+
# qkit-research
|
|
2
|
+
|
|
3
|
+
The checks a quantitative researcher forgets.
|
|
4
|
+
|
|
5
|
+
Not a backtester. Backtesters are plentiful and mostly fine; what goes wrong is
|
|
6
|
+
upstream of them, in four places that turn a strategy that does not work into a
|
|
7
|
+
beautiful equity curve:
|
|
8
|
+
|
|
9
|
+
| Check | What it catches |
|
|
10
|
+
|---|---|
|
|
11
|
+
| **Lookahead** | A feature that uses information from its own future |
|
|
12
|
+
| **Survivorship** | A universe that quietly excludes what died |
|
|
13
|
+
| **Label leakage** | Cross-validation folds sharing overlapping label windows |
|
|
14
|
+
| **Selection** | A Sharpe ratio reported after trying two hundred variants |
|
|
15
|
+
|
|
16
|
+
Each returns a *finding*, not a number, so it drops into a research loop and
|
|
17
|
+
fails loudly.
|
|
18
|
+
|
|
19
|
+
## Install
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
pip install qkit-research
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
Requires Python 3.9+ and numpy. Nothing else.
|
|
26
|
+
|
|
27
|
+
## Lookahead
|
|
28
|
+
|
|
29
|
+
The convention: `feature[t]` is known at the end of period `t` and predicts
|
|
30
|
+
`target[t+1]`. Lookahead therefore means correlation with the *contemporaneous
|
|
31
|
+
or future* target. Correlation with an earlier target is **not** lookahead — a
|
|
32
|
+
momentum feature *is* the previous return.
|
|
33
|
+
|
|
34
|
+
```python
|
|
35
|
+
import numpy as np
|
|
36
|
+
from qkit import detect_lookahead, shift_safe
|
|
37
|
+
|
|
38
|
+
returns = np.random.default_rng(0).normal(0, 0.01, 1000)
|
|
39
|
+
|
|
40
|
+
leaky = np.roll(returns, -1) # copies next period's return
|
|
41
|
+
print(detect_lookahead(leaky, returns, "leaky").contaminated) # True
|
|
42
|
+
|
|
43
|
+
momentum = np.roll(returns, 1) # legitimate: last period's return
|
|
44
|
+
print(detect_lookahead(momentum, returns, "momentum").contaminated) # False
|
|
45
|
+
|
|
46
|
+
fixed = shift_safe(leaky, 2) # the one-line remedy
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
The threshold is a plausibility bound on forward predictability, not a
|
|
50
|
+
significance test. Genuine edges in liquid markets correlate 0.02–0.10; a
|
|
51
|
+
feature correlating 0.6 with a future return has seen the answer.
|
|
52
|
+
|
|
53
|
+
## Survivorship
|
|
54
|
+
|
|
55
|
+
```python
|
|
56
|
+
from qkit import audit_universe
|
|
57
|
+
|
|
58
|
+
finding = audit_universe(returns_matrix, alive_at_end)
|
|
59
|
+
print(finding.bias_bps) # basis points per period
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Takes the full universe *including the dead* — exactly the data most
|
|
63
|
+
survivorship-biased studies lack. When it is unavailable the honest output is a
|
|
64
|
+
warning; this library does not correct a bias it cannot measure.
|
|
65
|
+
|
|
66
|
+
## Purged cross-validation
|
|
67
|
+
|
|
68
|
+
Standard k-fold leaks whenever a label spans several periods. Purging removes
|
|
69
|
+
the overlap; an embargo removes the serial correlation that survives it.
|
|
70
|
+
|
|
71
|
+
```python
|
|
72
|
+
from qkit import PurgedKFold
|
|
73
|
+
|
|
74
|
+
cv = PurgedKFold(n_splits=5, label_span=20, embargo=20)
|
|
75
|
+
for train, test in cv.split(len(X)):
|
|
76
|
+
model.fit(X[train], y[train])
|
|
77
|
+
|
|
78
|
+
print(cv.leakage_check(len(X))) # folds_with_overlap should be 0
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
With `label_span=1` and `embargo=0` it reduces to ordinary contiguous k-fold.
|
|
82
|
+
|
|
83
|
+
## Deflated Sharpe
|
|
84
|
+
|
|
85
|
+
A Sharpe of 2.0 found after testing two hundred variants is not the same
|
|
86
|
+
quantity as a Sharpe of 2.0 from a single pre-registered strategy.
|
|
87
|
+
|
|
88
|
+
```python
|
|
89
|
+
from qkit import sharpe, deflated_sharpe, min_track_record_length
|
|
90
|
+
|
|
91
|
+
print(sharpe(r))
|
|
92
|
+
print(deflated_sharpe(r, n_trials=200))
|
|
93
|
+
print(min_track_record_length(r))
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
On 200 strategies generated from pure noise — true Sharpe exactly zero — the
|
|
97
|
+
best shows an observed Sharpe of 1.82 and a probabilistic Sharpe against zero of
|
|
98
|
+
0.999. Deflating for the trial count drops it to 0.65 and it does not survive.
|
|
99
|
+
|
|
100
|
+
Deflation assumes independent trials. Two hundred variants of one idea are not
|
|
101
|
+
independent, and the correction is then too weak.
|
|
102
|
+
|
|
103
|
+
## Command line
|
|
104
|
+
|
|
105
|
+
```bash
|
|
106
|
+
qkit lookahead features.csv --target fwd_return --features mom_5 vol_20 rsi
|
|
107
|
+
qkit sharpe backtest.csv --column returns --trials 200
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
Both exit non-zero on a finding, so they work as a pre-commit hook or a CI gate.
|
|
111
|
+
|
|
112
|
+
## Limitations
|
|
113
|
+
|
|
114
|
+
- The lookahead detector is correlational and univariate. A feature that leaks
|
|
115
|
+
only in combination with another will pass, and a centred rolling window can
|
|
116
|
+
sit just under the threshold.
|
|
117
|
+
- Purged k-fold assumes a fixed label span; variable-horizon labels need
|
|
118
|
+
per-observation windows, which are not implemented.
|
|
119
|
+
- Nothing here validates a strategy. It tells you when a backtest cannot be
|
|
120
|
+
believed, which is a different and smaller claim.
|
|
121
|
+
|
|
122
|
+
## Licence
|
|
123
|
+
|
|
124
|
+
MIT.
|
|
@@ -0,0 +1,143 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: qkit-research
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: The checks a quantitative researcher forgets: lookahead detection, survivorship auditing, purged cross-validation and deflated Sharpe.
|
|
5
|
+
Author: Wenke Du
|
|
6
|
+
License: MIT
|
|
7
|
+
Keywords: quantitative,backtest,lookahead,purged,sharpe
|
|
8
|
+
Classifier: Development Status :: 3 - Alpha
|
|
9
|
+
Classifier: Intended Audience :: Financial and Insurance Industry
|
|
10
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
11
|
+
Classifier: Programming Language :: Python :: 3
|
|
12
|
+
Requires-Python: >=3.9
|
|
13
|
+
Description-Content-Type: text/markdown
|
|
14
|
+
License-File: LICENSE
|
|
15
|
+
Requires-Dist: numpy>=1.24
|
|
16
|
+
Provides-Extra: dev
|
|
17
|
+
Requires-Dist: pytest>=7.0; extra == "dev"
|
|
18
|
+
Dynamic: license-file
|
|
19
|
+
|
|
20
|
+
# qkit-research
|
|
21
|
+
|
|
22
|
+
The checks a quantitative researcher forgets.
|
|
23
|
+
|
|
24
|
+
Not a backtester. Backtesters are plentiful and mostly fine; what goes wrong is
|
|
25
|
+
upstream of them, in four places that turn a strategy that does not work into a
|
|
26
|
+
beautiful equity curve:
|
|
27
|
+
|
|
28
|
+
| Check | What it catches |
|
|
29
|
+
|---|---|
|
|
30
|
+
| **Lookahead** | A feature that uses information from its own future |
|
|
31
|
+
| **Survivorship** | A universe that quietly excludes what died |
|
|
32
|
+
| **Label leakage** | Cross-validation folds sharing overlapping label windows |
|
|
33
|
+
| **Selection** | A Sharpe ratio reported after trying two hundred variants |
|
|
34
|
+
|
|
35
|
+
Each returns a *finding*, not a number, so it drops into a research loop and
|
|
36
|
+
fails loudly.
|
|
37
|
+
|
|
38
|
+
## Install
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
pip install qkit-research
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
Requires Python 3.9+ and numpy. Nothing else.
|
|
45
|
+
|
|
46
|
+
## Lookahead
|
|
47
|
+
|
|
48
|
+
The convention: `feature[t]` is known at the end of period `t` and predicts
|
|
49
|
+
`target[t+1]`. Lookahead therefore means correlation with the *contemporaneous
|
|
50
|
+
or future* target. Correlation with an earlier target is **not** lookahead — a
|
|
51
|
+
momentum feature *is* the previous return.
|
|
52
|
+
|
|
53
|
+
```python
|
|
54
|
+
import numpy as np
|
|
55
|
+
from qkit import detect_lookahead, shift_safe
|
|
56
|
+
|
|
57
|
+
returns = np.random.default_rng(0).normal(0, 0.01, 1000)
|
|
58
|
+
|
|
59
|
+
leaky = np.roll(returns, -1) # copies next period's return
|
|
60
|
+
print(detect_lookahead(leaky, returns, "leaky").contaminated) # True
|
|
61
|
+
|
|
62
|
+
momentum = np.roll(returns, 1) # legitimate: last period's return
|
|
63
|
+
print(detect_lookahead(momentum, returns, "momentum").contaminated) # False
|
|
64
|
+
|
|
65
|
+
fixed = shift_safe(leaky, 2) # the one-line remedy
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
The threshold is a plausibility bound on forward predictability, not a
|
|
69
|
+
significance test. Genuine edges in liquid markets correlate 0.02–0.10; a
|
|
70
|
+
feature correlating 0.6 with a future return has seen the answer.
|
|
71
|
+
|
|
72
|
+
## Survivorship
|
|
73
|
+
|
|
74
|
+
```python
|
|
75
|
+
from qkit import audit_universe
|
|
76
|
+
|
|
77
|
+
finding = audit_universe(returns_matrix, alive_at_end)
|
|
78
|
+
print(finding.bias_bps) # basis points per period
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
Takes the full universe *including the dead* — exactly the data most
|
|
82
|
+
survivorship-biased studies lack. When it is unavailable the honest output is a
|
|
83
|
+
warning; this library does not correct a bias it cannot measure.
|
|
84
|
+
|
|
85
|
+
## Purged cross-validation
|
|
86
|
+
|
|
87
|
+
Standard k-fold leaks whenever a label spans several periods. Purging removes
|
|
88
|
+
the overlap; an embargo removes the serial correlation that survives it.
|
|
89
|
+
|
|
90
|
+
```python
|
|
91
|
+
from qkit import PurgedKFold
|
|
92
|
+
|
|
93
|
+
cv = PurgedKFold(n_splits=5, label_span=20, embargo=20)
|
|
94
|
+
for train, test in cv.split(len(X)):
|
|
95
|
+
model.fit(X[train], y[train])
|
|
96
|
+
|
|
97
|
+
print(cv.leakage_check(len(X))) # folds_with_overlap should be 0
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
With `label_span=1` and `embargo=0` it reduces to ordinary contiguous k-fold.
|
|
101
|
+
|
|
102
|
+
## Deflated Sharpe
|
|
103
|
+
|
|
104
|
+
A Sharpe of 2.0 found after testing two hundred variants is not the same
|
|
105
|
+
quantity as a Sharpe of 2.0 from a single pre-registered strategy.
|
|
106
|
+
|
|
107
|
+
```python
|
|
108
|
+
from qkit import sharpe, deflated_sharpe, min_track_record_length
|
|
109
|
+
|
|
110
|
+
print(sharpe(r))
|
|
111
|
+
print(deflated_sharpe(r, n_trials=200))
|
|
112
|
+
print(min_track_record_length(r))
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
On 200 strategies generated from pure noise — true Sharpe exactly zero — the
|
|
116
|
+
best shows an observed Sharpe of 1.82 and a probabilistic Sharpe against zero of
|
|
117
|
+
0.999. Deflating for the trial count drops it to 0.65 and it does not survive.
|
|
118
|
+
|
|
119
|
+
Deflation assumes independent trials. Two hundred variants of one idea are not
|
|
120
|
+
independent, and the correction is then too weak.
|
|
121
|
+
|
|
122
|
+
## Command line
|
|
123
|
+
|
|
124
|
+
```bash
|
|
125
|
+
qkit lookahead features.csv --target fwd_return --features mom_5 vol_20 rsi
|
|
126
|
+
qkit sharpe backtest.csv --column returns --trials 200
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
Both exit non-zero on a finding, so they work as a pre-commit hook or a CI gate.
|
|
130
|
+
|
|
131
|
+
## Limitations
|
|
132
|
+
|
|
133
|
+
- The lookahead detector is correlational and univariate. A feature that leaks
|
|
134
|
+
only in combination with another will pass, and a centred rolling window can
|
|
135
|
+
sit just under the threshold.
|
|
136
|
+
- Purged k-fold assumes a fixed label span; variable-horizon labels need
|
|
137
|
+
per-observation windows, which are not implemented.
|
|
138
|
+
- Nothing here validates a strategy. It tells you when a backtest cannot be
|
|
139
|
+
believed, which is a different and smaller claim.
|
|
140
|
+
|
|
141
|
+
## Licence
|
|
142
|
+
|
|
143
|
+
MIT.
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
# qkit-research
|
|
2
|
+
|
|
3
|
+
> The four checks that decide whether a backtest means anything: lookahead, survivorship, label leakage, and how many strategies you tried before this one.
|
|
4
|
+
|
|
5
|
+
**Repository:** `5.0-Ai-Engineering-Toolkit` · **Pillar:** Financial Stability
|
|
6
|
+
|
|
7
|
+
## Status
|
|
8
|
+
|
|
9
|
+
This is working code with a runnable demo and 28 tests. It is **not** a
|
|
10
|
+
finished result.
|
|
11
|
+
|
|
12
|
+
Release artifacts (sdist + wheel) are built and pass twine check, but the package is NOT published to PyPI — no downloads, no users. Every demonstration below runs on synthetic series where the contamination was put there deliberately, which is the only way to show a detector finds it.
|
|
13
|
+
|
|
14
|
+
Last run: `2026-08-31T21:08:19+00:00`
|
|
15
|
+
|
|
16
|
+
## Quick start
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
pip install -r requirements.txt
|
|
20
|
+
python -m pytest tests/ -q # 28 tests
|
|
21
|
+
python -m src.demo # runs everything, rewrites results/ and website/
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
## Layout
|
|
25
|
+
|
|
26
|
+
```
|
|
27
|
+
LICENSE
|
|
28
|
+
MANIFEST.in
|
|
29
|
+
PACKAGE_README.md
|
|
30
|
+
README.md
|
|
31
|
+
data/
|
|
32
|
+
|-- README.md
|
|
33
|
+
|-- manifests/
|
|
34
|
+
|-- sample/
|
|
35
|
+
dist/
|
|
36
|
+
|-- qkit_research-0.1.0-py3-none-any.whl
|
|
37
|
+
|-- qkit_research-0.1.0.tar.gz
|
|
38
|
+
docs/
|
|
39
|
+
|-- DATA.md
|
|
40
|
+
|-- METHOD.md
|
|
41
|
+
pyproject.toml
|
|
42
|
+
qkit/
|
|
43
|
+
|-- __init__.py
|
|
44
|
+
|-- cli.py
|
|
45
|
+
|-- lookahead.py
|
|
46
|
+
|-- statistics.py
|
|
47
|
+
|-- universe.py
|
|
48
|
+
|-- validation.py
|
|
49
|
+
qkit_research.egg-info/
|
|
50
|
+
|-- PKG-INFO
|
|
51
|
+
|-- SOURCES.txt
|
|
52
|
+
|-- dependency_links.txt
|
|
53
|
+
|-- entry_points.txt
|
|
54
|
+
|-- requires.txt
|
|
55
|
+
|-- top_level.txt
|
|
56
|
+
requirements.txt
|
|
57
|
+
results/
|
|
58
|
+
|-- README.md
|
|
59
|
+
|-- latest.json
|
|
60
|
+
src/
|
|
61
|
+
|-- .gitkeep
|
|
62
|
+
|-- __init__.py
|
|
63
|
+
|-- demo.py
|
|
64
|
+
|-- site.py
|
|
65
|
+
|-- sitekit.py
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
- `src/` — the implementation.
|
|
69
|
+
- `tests/` — pytest suite. These guard behaviour, not just imports.
|
|
70
|
+
- `results/latest.json` — the output of the last demo run. Every figure quoted
|
|
71
|
+
anywhere in this project traces back to this file.
|
|
72
|
+
- `website/` — a self-contained static site, deployable to Vercel by copying the
|
|
73
|
+
folder into its own repository. See `website/README.md`.
|
|
74
|
+
|
|
75
|
+
## The website
|
|
76
|
+
|
|
77
|
+
`website/` has no build step. To deploy it independently:
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
cp -r website/ ../my-3-quant-productivity-toolkit-site && cd ../my-3-quant-productivity-toolkit-site
|
|
81
|
+
git init && git add -A && git commit -m "site"
|
|
82
|
+
vercel deploy --prod
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
The page is regenerated from `results.json` on every `python -m src.demo`, so the
|
|
86
|
+
figures on the site and the figures the code produces cannot drift apart. Do not edit
|
|
87
|
+
numbers on the page by hand.
|
|
88
|
+
|
|
89
|
+
## Honesty note
|
|
90
|
+
|
|
91
|
+
Everything in this project runs on clearly-labelled synthetic or authored data.
|
|
92
|
+
Swap in the real source and the same pipeline reports real numbers — that is
|
|
93
|
+
what the structure is for. Until that happens, nothing here should be cited as a
|
|
94
|
+
measured result, and the site's closing section states explicitly what the project
|
|
95
|
+
does not establish.
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
[build-system]
|
|
2
|
+
requires = ["setuptools>=68", "wheel"]
|
|
3
|
+
build-backend = "setuptools.build_meta"
|
|
4
|
+
|
|
5
|
+
[project]
|
|
6
|
+
name = "qkit-research"
|
|
7
|
+
version = "0.1.0"
|
|
8
|
+
description = "The checks a quantitative researcher forgets: lookahead detection, survivorship auditing, purged cross-validation and deflated Sharpe."
|
|
9
|
+
readme = "PACKAGE_README.md"
|
|
10
|
+
requires-python = ">=3.9"
|
|
11
|
+
license = {text = "MIT"}
|
|
12
|
+
authors = [{name = "Wenke Du"}]
|
|
13
|
+
keywords = ["quantitative", "backtest", "lookahead", "purged", "sharpe"]
|
|
14
|
+
classifiers = [
|
|
15
|
+
"Development Status :: 3 - Alpha",
|
|
16
|
+
"Intended Audience :: Financial and Insurance Industry",
|
|
17
|
+
"License :: OSI Approved :: MIT License",
|
|
18
|
+
"Programming Language :: Python :: 3",
|
|
19
|
+
]
|
|
20
|
+
dependencies = ["numpy>=1.24"]
|
|
21
|
+
|
|
22
|
+
[project.optional-dependencies]
|
|
23
|
+
dev = ["pytest>=7.0"]
|
|
24
|
+
|
|
25
|
+
[project.scripts]
|
|
26
|
+
qkit = "qkit.cli:main"
|
|
27
|
+
|
|
28
|
+
[tool.setuptools.packages.find]
|
|
29
|
+
include = ["qkit*"]
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
"""qkit — the checks a quantitative researcher forgets.
|
|
2
|
+
|
|
3
|
+
Not a backtester. Backtesters are plentiful and mostly fine; what goes wrong is
|
|
4
|
+
upstream of them, in four places that produce a beautiful equity curve from a
|
|
5
|
+
strategy that does not work:
|
|
6
|
+
|
|
7
|
+
lookahead a feature that uses information from its own future
|
|
8
|
+
survivorship a universe that quietly excludes what died
|
|
9
|
+
leakage cross-validation folds that share overlapping label windows
|
|
10
|
+
selection a Sharpe ratio reported after trying two hundred variants
|
|
11
|
+
|
|
12
|
+
Each is a function that returns a finding, not a number, so it can be dropped
|
|
13
|
+
into an existing research loop and fail loudly.
|
|
14
|
+
"""
|
|
15
|
+
|
|
16
|
+
from .lookahead import LookaheadFinding, detect_lookahead, shift_safe
|
|
17
|
+
from .universe import SurvivorshipFinding, audit_universe
|
|
18
|
+
from .validation import PurgedKFold, embargo_indices
|
|
19
|
+
from .statistics import (deflated_sharpe, min_track_record_length,
|
|
20
|
+
probabilistic_sharpe, sharpe)
|
|
21
|
+
|
|
22
|
+
__version__ = "0.1.0"
|
|
23
|
+
|
|
24
|
+
__all__ = ["LookaheadFinding", "detect_lookahead", "shift_safe",
|
|
25
|
+
"SurvivorshipFinding", "audit_universe",
|
|
26
|
+
"PurgedKFold", "embargo_indices",
|
|
27
|
+
"sharpe", "probabilistic_sharpe", "deflated_sharpe",
|
|
28
|
+
"min_track_record_length", "__version__"]
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
"""qkit command line: audit a returns/feature CSV before you trust a backtest."""
|
|
2
|
+
|
|
3
|
+
from __future__ import annotations
|
|
4
|
+
|
|
5
|
+
import argparse
|
|
6
|
+
import csv
|
|
7
|
+
import json
|
|
8
|
+
import sys
|
|
9
|
+
|
|
10
|
+
import numpy as np
|
|
11
|
+
|
|
12
|
+
from .lookahead import detect_lookahead
|
|
13
|
+
from .statistics import deflated_sharpe, min_track_record_length, sharpe
|
|
14
|
+
|
|
15
|
+
|
|
16
|
+
def _load(path, cols):
|
|
17
|
+
with open(path, newline="", encoding="utf8") as fh:
|
|
18
|
+
rows = list(csv.DictReader(fh))
|
|
19
|
+
return {c: np.array([float(r[c]) for r in rows]) for c in cols}
|
|
20
|
+
|
|
21
|
+
|
|
22
|
+
def cmd_lookahead(args) -> int:
|
|
23
|
+
data = _load(args.csv, [args.target] + args.features)
|
|
24
|
+
y = data[args.target]
|
|
25
|
+
out = [detect_lookahead(data[f], y, f, threshold=args.threshold).as_dict()
|
|
26
|
+
for f in args.features]
|
|
27
|
+
json.dump({"findings": out,
|
|
28
|
+
"n_contaminated": sum(o["contaminated"] for o in out)},
|
|
29
|
+
sys.stdout, indent=2)
|
|
30
|
+
print()
|
|
31
|
+
return 1 if any(o["contaminated"] for o in out) else 0
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
def cmd_sharpe(args) -> int:
|
|
35
|
+
r = _load(args.csv, [args.column])[args.column]
|
|
36
|
+
out = {"sharpe": round(sharpe(r, args.periods), 4),
|
|
37
|
+
"deflated": deflated_sharpe(r, args.trials, args.periods),
|
|
38
|
+
"min_track_record_length": round(
|
|
39
|
+
min_track_record_length(r, periods_per_year=args.periods), 1)}
|
|
40
|
+
json.dump(out, sys.stdout, indent=2)
|
|
41
|
+
print()
|
|
42
|
+
return 0 if out["deflated"]["survives"] else 1
|
|
43
|
+
|
|
44
|
+
|
|
45
|
+
def main(argv=None) -> int:
|
|
46
|
+
ap = argparse.ArgumentParser(prog="qkit", description=__doc__)
|
|
47
|
+
sub = ap.add_subparsers(dest="cmd", required=True)
|
|
48
|
+
|
|
49
|
+
a = sub.add_parser("lookahead", help="flag features that see the future")
|
|
50
|
+
a.add_argument("csv")
|
|
51
|
+
a.add_argument("--target", required=True)
|
|
52
|
+
a.add_argument("--features", nargs="+", required=True)
|
|
53
|
+
a.add_argument("--threshold", type=float, default=0.50)
|
|
54
|
+
a.set_defaults(fn=cmd_lookahead)
|
|
55
|
+
|
|
56
|
+
b = sub.add_parser("sharpe", help="Sharpe, deflated for the number of trials")
|
|
57
|
+
b.add_argument("csv")
|
|
58
|
+
b.add_argument("--column", default="returns")
|
|
59
|
+
b.add_argument("--trials", type=int, default=1)
|
|
60
|
+
b.add_argument("--periods", type=int, default=252)
|
|
61
|
+
b.set_defaults(fn=cmd_sharpe)
|
|
62
|
+
|
|
63
|
+
args = ap.parse_args(argv)
|
|
64
|
+
return args.fn(args)
|
|
65
|
+
|
|
66
|
+
|
|
67
|
+
if __name__ == "__main__":
|
|
68
|
+
raise SystemExit(main())
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
"""Detecting features that use information they could not have had.
|
|
2
|
+
|
|
3
|
+
The convention throughout: `feature[t]` is known at the end of period t and is
|
|
4
|
+
used to predict `target[t+1]`, the return realised over the NEXT period.
|
|
5
|
+
|
|
6
|
+
Under that convention lookahead means `feature[t]` carries information about
|
|
7
|
+
`target[t]` or later. Correlation with EARLIER targets is not lookahead -- a
|
|
8
|
+
momentum feature is literally the previous return, and correlating perfectly
|
|
9
|
+
with it is the definition of momentum, not a bug.
|
|
10
|
+
|
|
11
|
+
Getting that backwards is easy and was the first version of this module: it
|
|
12
|
+
tested correlation with past targets, flagged a momentum feature as
|
|
13
|
+
contaminated, and passed a feature that was a copy of next period's return. The
|
|
14
|
+
direction of the test is the whole content of it.
|
|
15
|
+
|
|
16
|
+
The detection rule is a plausibility bound rather than a significance test.
|
|
17
|
+
Genuine forward predictability in a liquid market is weak -- correlations of
|
|
18
|
+
0.02 to 0.10 are a real edge. A feature correlating 0.6 with a future return has
|
|
19
|
+
not found an edge; it has seen the answer.
|
|
20
|
+
"""
|
|
21
|
+
|
|
22
|
+
from __future__ import annotations
|
|
23
|
+
|
|
24
|
+
from dataclasses import dataclass
|
|
25
|
+
|
|
26
|
+
import numpy as np
|
|
27
|
+
|
|
28
|
+
|
|
29
|
+
@dataclass
|
|
30
|
+
class LookaheadFinding:
|
|
31
|
+
feature: str
|
|
32
|
+
max_forward_corr: float # with target[t+k], k >= 0 -- suspicious
|
|
33
|
+
lag_of_max: int # 0 = contemporaneous, 1 = next period
|
|
34
|
+
max_backward_corr: float # with target[t-k], k >= 1 -- legitimate
|
|
35
|
+
contaminated: bool
|
|
36
|
+
detail: str
|
|
37
|
+
|
|
38
|
+
def as_dict(self) -> dict:
|
|
39
|
+
return {"feature": self.feature,
|
|
40
|
+
"max_forward_corr": round(self.max_forward_corr, 5),
|
|
41
|
+
"lag_of_max": self.lag_of_max,
|
|
42
|
+
"max_backward_corr": round(self.max_backward_corr, 5),
|
|
43
|
+
"contaminated": self.contaminated, "detail": self.detail}
|
|
44
|
+
|
|
45
|
+
|
|
46
|
+
def _corr(a, b) -> float:
|
|
47
|
+
if len(a) < 8 or np.std(a) < 1e-12 or np.std(b) < 1e-12:
|
|
48
|
+
return 0.0
|
|
49
|
+
return float(np.corrcoef(a, b)[0, 1])
|
|
50
|
+
|
|
51
|
+
|
|
52
|
+
def detect_lookahead(feature: np.ndarray, target: np.ndarray,
|
|
53
|
+
name: str = "feature", max_lag: int = 3,
|
|
54
|
+
threshold: float = 0.50) -> LookaheadFinding:
|
|
55
|
+
"""Flag a feature that correlates implausibly with a contemporaneous or
|
|
56
|
+
future target.
|
|
57
|
+
|
|
58
|
+
`threshold` is a plausibility bound on forward predictability, not a
|
|
59
|
+
p-value. Lowering it below about 0.3 starts flagging strong-but-real signals
|
|
60
|
+
on short samples.
|
|
61
|
+
"""
|
|
62
|
+
f = np.asarray(feature, float)
|
|
63
|
+
y = np.asarray(target, float)
|
|
64
|
+
n = min(len(f), len(y))
|
|
65
|
+
f, y = f[:n], y[:n]
|
|
66
|
+
|
|
67
|
+
# Forward: feature[t] against target[t + k] for k >= 0.
|
|
68
|
+
fwd, fwd_lag = 0.0, 0
|
|
69
|
+
for k in range(0, max_lag + 1):
|
|
70
|
+
c = abs(_corr(f[:n - k], y[k:])) if k else abs(_corr(f, y))
|
|
71
|
+
if c > fwd:
|
|
72
|
+
fwd, fwd_lag = c, k
|
|
73
|
+
|
|
74
|
+
# Backward: feature[t] against target[t - k] for k >= 1. Legitimate.
|
|
75
|
+
bwd = 0.0
|
|
76
|
+
for k in range(1, max_lag + 1):
|
|
77
|
+
bwd = max(bwd, abs(_corr(f[k:], y[:-k])))
|
|
78
|
+
|
|
79
|
+
bad = fwd > threshold
|
|
80
|
+
when = ("the same period" if fwd_lag == 0
|
|
81
|
+
else f"{fwd_lag} period(s) ahead")
|
|
82
|
+
detail = (f"|corr(f[t], y[t+{fwd_lag}])| = {fwd:.3f} exceeds {threshold}; "
|
|
83
|
+
f"the feature knows about {when}" if bad
|
|
84
|
+
else f"max forward correlation {fwd:.3f} is within the plausible "
|
|
85
|
+
f"range (backward {bwd:.3f} is not evidence of lookahead)")
|
|
86
|
+
return LookaheadFinding(name, fwd, fwd_lag, bwd, bad, detail)
|
|
87
|
+
|
|
88
|
+
|
|
89
|
+
def shift_safe(x: np.ndarray, periods: int = 1) -> np.ndarray:
|
|
90
|
+
"""Shift a feature so it can only use information already available.
|
|
91
|
+
|
|
92
|
+
The one-line fix for most lookahead. It is in the package because the
|
|
93
|
+
mistake is not knowing it is needed, not being unable to write it.
|
|
94
|
+
"""
|
|
95
|
+
x = np.asarray(x, float)
|
|
96
|
+
out = np.full_like(x, np.nan)
|
|
97
|
+
if periods > 0:
|
|
98
|
+
out[periods:] = x[:-periods]
|
|
99
|
+
elif periods < 0:
|
|
100
|
+
out[:periods] = x[-periods:]
|
|
101
|
+
else:
|
|
102
|
+
out[:] = x
|
|
103
|
+
return out
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
"""Sharpe ratios that account for how many strategies were tried.
|
|
2
|
+
|
|
3
|
+
A Sharpe of 2.0 found after testing two hundred variants is not the same
|
|
4
|
+
quantity as a Sharpe of 2.0 from a single pre-registered strategy, and reporting
|
|
5
|
+
them identically is the most common way a backtest misleads.
|
|
6
|
+
"""
|
|
7
|
+
|
|
8
|
+
from __future__ import annotations
|
|
9
|
+
|
|
10
|
+
import math
|
|
11
|
+
|
|
12
|
+
import numpy as np
|
|
13
|
+
|
|
14
|
+
|
|
15
|
+
def sharpe(returns: np.ndarray, periods_per_year: int = 252) -> float:
|
|
16
|
+
r = np.asarray(returns, float)
|
|
17
|
+
sd = r.std(ddof=1)
|
|
18
|
+
if sd < 1e-15:
|
|
19
|
+
return 0.0
|
|
20
|
+
return float(r.mean() / sd * math.sqrt(periods_per_year))
|
|
21
|
+
|
|
22
|
+
|
|
23
|
+
def _norm_cdf(x: float) -> float:
|
|
24
|
+
return 0.5 * (1 + math.erf(x / math.sqrt(2)))
|
|
25
|
+
|
|
26
|
+
|
|
27
|
+
def _norm_ppf(p: float) -> float:
|
|
28
|
+
lo, hi = -12.0, 12.0
|
|
29
|
+
for _ in range(200):
|
|
30
|
+
mid = (lo + hi) / 2
|
|
31
|
+
if _norm_cdf(mid) < p:
|
|
32
|
+
lo = mid
|
|
33
|
+
else:
|
|
34
|
+
hi = mid
|
|
35
|
+
return (lo + hi) / 2
|
|
36
|
+
|
|
37
|
+
|
|
38
|
+
def probabilistic_sharpe(returns: np.ndarray, benchmark_sr: float = 0.0,
|
|
39
|
+
periods_per_year: int = 252) -> float:
|
|
40
|
+
"""P(true Sharpe > benchmark), adjusting for skew and kurtosis.
|
|
41
|
+
|
|
42
|
+
Non-normal returns are the norm, and a Sharpe ratio computed on them has a
|
|
43
|
+
standard error the textbook formula understates.
|
|
44
|
+
"""
|
|
45
|
+
r = np.asarray(returns, float)
|
|
46
|
+
n = len(r)
|
|
47
|
+
if n < 3:
|
|
48
|
+
return float("nan")
|
|
49
|
+
sr = sharpe(r, periods_per_year) / math.sqrt(periods_per_year) # per period
|
|
50
|
+
bench = benchmark_sr / math.sqrt(periods_per_year)
|
|
51
|
+
sd = r.std(ddof=1)
|
|
52
|
+
if sd < 1e-15:
|
|
53
|
+
return float("nan")
|
|
54
|
+
z = (r - r.mean()) / sd
|
|
55
|
+
skew = float(np.mean(z ** 3))
|
|
56
|
+
kurt = float(np.mean(z ** 4))
|
|
57
|
+
denom = math.sqrt(max(1e-12,
|
|
58
|
+
(1 - skew * sr + (kurt - 1) / 4 * sr ** 2) / (n - 1)))
|
|
59
|
+
return float(_norm_cdf((sr - bench) / denom))
|
|
60
|
+
|
|
61
|
+
|
|
62
|
+
def deflated_sharpe(returns: np.ndarray, n_trials: int,
|
|
63
|
+
periods_per_year: int = 252,
|
|
64
|
+
trial_sr_variance: float | None = None) -> dict:
|
|
65
|
+
"""Sharpe adjusted for the number of strategies tried.
|
|
66
|
+
|
|
67
|
+
The expected maximum Sharpe across `n_trials` independent random strategies
|
|
68
|
+
is not zero; it grows with the number tried. The deflated ratio asks whether
|
|
69
|
+
the observed Sharpe beats that expectation.
|
|
70
|
+
"""
|
|
71
|
+
r = np.asarray(returns, float)
|
|
72
|
+
obs = sharpe(r, periods_per_year)
|
|
73
|
+
v = trial_sr_variance if trial_sr_variance is not None else 1.0 / len(r)
|
|
74
|
+
if n_trials < 2:
|
|
75
|
+
expected_max = 0.0
|
|
76
|
+
else:
|
|
77
|
+
e = 0.5772156649
|
|
78
|
+
expected_max = math.sqrt(v) * (
|
|
79
|
+
(1 - e) * _norm_ppf(1 - 1.0 / n_trials)
|
|
80
|
+
+ e * _norm_ppf(1 - 1.0 / (n_trials * math.e)))
|
|
81
|
+
expected_max *= math.sqrt(periods_per_year)
|
|
82
|
+
psr = probabilistic_sharpe(r, benchmark_sr=expected_max,
|
|
83
|
+
periods_per_year=periods_per_year)
|
|
84
|
+
return {"observed_sharpe": round(obs, 4), "n_trials": n_trials,
|
|
85
|
+
"expected_max_sharpe_from_noise": round(expected_max, 4),
|
|
86
|
+
"deflated_sharpe_prob": round(psr, 5),
|
|
87
|
+
"survives": bool(psr > 0.95)}
|
|
88
|
+
|
|
89
|
+
|
|
90
|
+
def min_track_record_length(returns: np.ndarray, benchmark_sr: float = 0.0,
|
|
91
|
+
confidence: float = 0.95,
|
|
92
|
+
periods_per_year: int = 252) -> float:
|
|
93
|
+
"""Observations needed before a Sharpe is distinguishable from the benchmark."""
|
|
94
|
+
r = np.asarray(returns, float)
|
|
95
|
+
sr = sharpe(r, periods_per_year) / math.sqrt(periods_per_year)
|
|
96
|
+
bench = benchmark_sr / math.sqrt(periods_per_year)
|
|
97
|
+
if abs(sr - bench) < 1e-12:
|
|
98
|
+
return float("inf")
|
|
99
|
+
sd = r.std(ddof=1)
|
|
100
|
+
z = (r - r.mean()) / sd if sd > 1e-15 else r * 0
|
|
101
|
+
skew, kurt = float(np.mean(z ** 3)), float(np.mean(z ** 4))
|
|
102
|
+
num = (1 - skew * sr + (kurt - 1) / 4 * sr ** 2)
|
|
103
|
+
return float(1 + num * (_norm_ppf(confidence) / (sr - bench)) ** 2)
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
"""Survivorship auditing.
|
|
2
|
+
|
|
3
|
+
A universe assembled from names that exist today has already excluded everything
|
|
4
|
+
that failed. The resulting backtest is not optimistic by a little; on a long
|
|
5
|
+
sample it can invert the sign of a result.
|
|
6
|
+
"""
|
|
7
|
+
|
|
8
|
+
from __future__ import annotations
|
|
9
|
+
|
|
10
|
+
from dataclasses import dataclass
|
|
11
|
+
|
|
12
|
+
import numpy as np
|
|
13
|
+
|
|
14
|
+
|
|
15
|
+
@dataclass
|
|
16
|
+
class SurvivorshipFinding:
|
|
17
|
+
n_universe: int
|
|
18
|
+
n_survivors: int
|
|
19
|
+
n_delisted: int
|
|
20
|
+
survivor_share: float
|
|
21
|
+
biased: bool
|
|
22
|
+
mean_return_all: float
|
|
23
|
+
mean_return_survivors: float
|
|
24
|
+
bias_bps: float
|
|
25
|
+
|
|
26
|
+
def as_dict(self) -> dict:
|
|
27
|
+
return {"n_universe": self.n_universe, "n_survivors": self.n_survivors,
|
|
28
|
+
"n_delisted": self.n_delisted,
|
|
29
|
+
"survivor_share": round(self.survivor_share, 4),
|
|
30
|
+
"biased": self.biased,
|
|
31
|
+
"mean_return_all": round(self.mean_return_all, 6),
|
|
32
|
+
"mean_return_survivors": round(self.mean_return_survivors, 6),
|
|
33
|
+
"bias_bps": round(self.bias_bps, 2)}
|
|
34
|
+
|
|
35
|
+
|
|
36
|
+
def audit_universe(returns: np.ndarray, alive_at_end: np.ndarray) -> SurvivorshipFinding:
|
|
37
|
+
"""Quantify what restricting to survivors would do to the mean return.
|
|
38
|
+
|
|
39
|
+
Takes the full universe including the dead, which is exactly the data most
|
|
40
|
+
survivorship-biased studies do not have. When it is unavailable the honest
|
|
41
|
+
output is a warning, not an adjustment -- this function does not attempt to
|
|
42
|
+
correct a bias it cannot measure.
|
|
43
|
+
"""
|
|
44
|
+
R = np.asarray(returns, float)
|
|
45
|
+
alive = np.asarray(alive_at_end, bool)
|
|
46
|
+
all_mean = float(np.nanmean(R))
|
|
47
|
+
surv_mean = float(np.nanmean(R[:, alive])) if alive.any() else float("nan")
|
|
48
|
+
n = R.shape[1]
|
|
49
|
+
return SurvivorshipFinding(
|
|
50
|
+
n_universe=n, n_survivors=int(alive.sum()), n_delisted=int((~alive).sum()),
|
|
51
|
+
survivor_share=float(alive.mean()), biased=bool((~alive).any()),
|
|
52
|
+
mean_return_all=all_mean, mean_return_survivors=surv_mean,
|
|
53
|
+
bias_bps=(surv_mean - all_mean) * 10_000)
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
"""Purged, embargoed cross-validation for overlapping labels.
|
|
2
|
+
|
|
3
|
+
Standard k-fold leaks whenever a label spans several periods: a training
|
|
4
|
+
observation whose label window overlaps a test observation shares information
|
|
5
|
+
with it, and the model sees the answer. Purging removes the overlap; an embargo
|
|
6
|
+
removes the serial correlation that survives it.
|
|
7
|
+
"""
|
|
8
|
+
|
|
9
|
+
from __future__ import annotations
|
|
10
|
+
|
|
11
|
+
import numpy as np
|
|
12
|
+
|
|
13
|
+
|
|
14
|
+
def embargo_indices(n: int, test_start: int, test_end: int,
|
|
15
|
+
embargo: int) -> np.ndarray:
|
|
16
|
+
"""Indices to drop after the test block, as a boolean mask of length n."""
|
|
17
|
+
mask = np.zeros(n, bool)
|
|
18
|
+
mask[test_end:min(n, test_end + embargo)] = True
|
|
19
|
+
return mask
|
|
20
|
+
|
|
21
|
+
|
|
22
|
+
class PurgedKFold:
|
|
23
|
+
"""K-fold that purges overlapping labels and applies an embargo.
|
|
24
|
+
|
|
25
|
+
`label_span` is how many periods a label covers. With span 1 and embargo 0
|
|
26
|
+
this reduces to ordinary contiguous k-fold, which is the check that the
|
|
27
|
+
implementation is not doing something exotic.
|
|
28
|
+
"""
|
|
29
|
+
|
|
30
|
+
def __init__(self, n_splits: int = 5, label_span: int = 1, embargo: int = 0):
|
|
31
|
+
if n_splits < 2:
|
|
32
|
+
raise ValueError("n_splits must be at least 2")
|
|
33
|
+
self.n_splits = n_splits
|
|
34
|
+
self.label_span = label_span
|
|
35
|
+
self.embargo = embargo
|
|
36
|
+
|
|
37
|
+
def split(self, n: int):
|
|
38
|
+
fold_edges = np.linspace(0, n, self.n_splits + 1).astype(int)
|
|
39
|
+
for i in range(self.n_splits):
|
|
40
|
+
a, b = fold_edges[i], fold_edges[i + 1]
|
|
41
|
+
test = np.zeros(n, bool)
|
|
42
|
+
test[a:b] = True
|
|
43
|
+
|
|
44
|
+
train = ~test
|
|
45
|
+
# Purge: any training label whose window reaches into the test block.
|
|
46
|
+
lo = max(0, a - self.label_span)
|
|
47
|
+
train[lo:b] = False
|
|
48
|
+
# Embargo: drop the periods immediately after the test block.
|
|
49
|
+
train &= ~embargo_indices(n, a, b, self.embargo)
|
|
50
|
+
yield train, test
|
|
51
|
+
|
|
52
|
+
def leakage_check(self, n: int) -> dict:
|
|
53
|
+
"""How many observations each fold drops, and whether any overlap remains."""
|
|
54
|
+
drops, overlaps = [], 0
|
|
55
|
+
for train, test in self.split(n):
|
|
56
|
+
drops.append(int(n - train.sum() - test.sum()))
|
|
57
|
+
tr = np.where(train)[0]
|
|
58
|
+
te = np.where(test)[0]
|
|
59
|
+
if len(tr) and len(te):
|
|
60
|
+
# An overlap survives if a training index's label window reaches
|
|
61
|
+
# a test index.
|
|
62
|
+
for t in tr:
|
|
63
|
+
if np.any((te >= t) & (te < t + self.label_span)):
|
|
64
|
+
overlaps += 1
|
|
65
|
+
break
|
|
66
|
+
return {"n_splits": self.n_splits, "label_span": self.label_span,
|
|
67
|
+
"embargo": self.embargo, "dropped_per_fold": drops,
|
|
68
|
+
"folds_with_overlap": overlaps}
|
|
@@ -0,0 +1,143 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: qkit-research
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: The checks a quantitative researcher forgets: lookahead detection, survivorship auditing, purged cross-validation and deflated Sharpe.
|
|
5
|
+
Author: Wenke Du
|
|
6
|
+
License: MIT
|
|
7
|
+
Keywords: quantitative,backtest,lookahead,purged,sharpe
|
|
8
|
+
Classifier: Development Status :: 3 - Alpha
|
|
9
|
+
Classifier: Intended Audience :: Financial and Insurance Industry
|
|
10
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
11
|
+
Classifier: Programming Language :: Python :: 3
|
|
12
|
+
Requires-Python: >=3.9
|
|
13
|
+
Description-Content-Type: text/markdown
|
|
14
|
+
License-File: LICENSE
|
|
15
|
+
Requires-Dist: numpy>=1.24
|
|
16
|
+
Provides-Extra: dev
|
|
17
|
+
Requires-Dist: pytest>=7.0; extra == "dev"
|
|
18
|
+
Dynamic: license-file
|
|
19
|
+
|
|
20
|
+
# qkit-research
|
|
21
|
+
|
|
22
|
+
The checks a quantitative researcher forgets.
|
|
23
|
+
|
|
24
|
+
Not a backtester. Backtesters are plentiful and mostly fine; what goes wrong is
|
|
25
|
+
upstream of them, in four places that turn a strategy that does not work into a
|
|
26
|
+
beautiful equity curve:
|
|
27
|
+
|
|
28
|
+
| Check | What it catches |
|
|
29
|
+
|---|---|
|
|
30
|
+
| **Lookahead** | A feature that uses information from its own future |
|
|
31
|
+
| **Survivorship** | A universe that quietly excludes what died |
|
|
32
|
+
| **Label leakage** | Cross-validation folds sharing overlapping label windows |
|
|
33
|
+
| **Selection** | A Sharpe ratio reported after trying two hundred variants |
|
|
34
|
+
|
|
35
|
+
Each returns a *finding*, not a number, so it drops into a research loop and
|
|
36
|
+
fails loudly.
|
|
37
|
+
|
|
38
|
+
## Install
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
pip install qkit-research
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
Requires Python 3.9+ and numpy. Nothing else.
|
|
45
|
+
|
|
46
|
+
## Lookahead
|
|
47
|
+
|
|
48
|
+
The convention: `feature[t]` is known at the end of period `t` and predicts
|
|
49
|
+
`target[t+1]`. Lookahead therefore means correlation with the *contemporaneous
|
|
50
|
+
or future* target. Correlation with an earlier target is **not** lookahead — a
|
|
51
|
+
momentum feature *is* the previous return.
|
|
52
|
+
|
|
53
|
+
```python
|
|
54
|
+
import numpy as np
|
|
55
|
+
from qkit import detect_lookahead, shift_safe
|
|
56
|
+
|
|
57
|
+
returns = np.random.default_rng(0).normal(0, 0.01, 1000)
|
|
58
|
+
|
|
59
|
+
leaky = np.roll(returns, -1) # copies next period's return
|
|
60
|
+
print(detect_lookahead(leaky, returns, "leaky").contaminated) # True
|
|
61
|
+
|
|
62
|
+
momentum = np.roll(returns, 1) # legitimate: last period's return
|
|
63
|
+
print(detect_lookahead(momentum, returns, "momentum").contaminated) # False
|
|
64
|
+
|
|
65
|
+
fixed = shift_safe(leaky, 2) # the one-line remedy
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
The threshold is a plausibility bound on forward predictability, not a
|
|
69
|
+
significance test. Genuine edges in liquid markets correlate 0.02–0.10; a
|
|
70
|
+
feature correlating 0.6 with a future return has seen the answer.
|
|
71
|
+
|
|
72
|
+
## Survivorship
|
|
73
|
+
|
|
74
|
+
```python
|
|
75
|
+
from qkit import audit_universe
|
|
76
|
+
|
|
77
|
+
finding = audit_universe(returns_matrix, alive_at_end)
|
|
78
|
+
print(finding.bias_bps) # basis points per period
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
Takes the full universe *including the dead* — exactly the data most
|
|
82
|
+
survivorship-biased studies lack. When it is unavailable the honest output is a
|
|
83
|
+
warning; this library does not correct a bias it cannot measure.
|
|
84
|
+
|
|
85
|
+
## Purged cross-validation
|
|
86
|
+
|
|
87
|
+
Standard k-fold leaks whenever a label spans several periods. Purging removes
|
|
88
|
+
the overlap; an embargo removes the serial correlation that survives it.
|
|
89
|
+
|
|
90
|
+
```python
|
|
91
|
+
from qkit import PurgedKFold
|
|
92
|
+
|
|
93
|
+
cv = PurgedKFold(n_splits=5, label_span=20, embargo=20)
|
|
94
|
+
for train, test in cv.split(len(X)):
|
|
95
|
+
model.fit(X[train], y[train])
|
|
96
|
+
|
|
97
|
+
print(cv.leakage_check(len(X))) # folds_with_overlap should be 0
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
With `label_span=1` and `embargo=0` it reduces to ordinary contiguous k-fold.
|
|
101
|
+
|
|
102
|
+
## Deflated Sharpe
|
|
103
|
+
|
|
104
|
+
A Sharpe of 2.0 found after testing two hundred variants is not the same
|
|
105
|
+
quantity as a Sharpe of 2.0 from a single pre-registered strategy.
|
|
106
|
+
|
|
107
|
+
```python
|
|
108
|
+
from qkit import sharpe, deflated_sharpe, min_track_record_length
|
|
109
|
+
|
|
110
|
+
print(sharpe(r))
|
|
111
|
+
print(deflated_sharpe(r, n_trials=200))
|
|
112
|
+
print(min_track_record_length(r))
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
On 200 strategies generated from pure noise — true Sharpe exactly zero — the
|
|
116
|
+
best shows an observed Sharpe of 1.82 and a probabilistic Sharpe against zero of
|
|
117
|
+
0.999. Deflating for the trial count drops it to 0.65 and it does not survive.
|
|
118
|
+
|
|
119
|
+
Deflation assumes independent trials. Two hundred variants of one idea are not
|
|
120
|
+
independent, and the correction is then too weak.
|
|
121
|
+
|
|
122
|
+
## Command line
|
|
123
|
+
|
|
124
|
+
```bash
|
|
125
|
+
qkit lookahead features.csv --target fwd_return --features mom_5 vol_20 rsi
|
|
126
|
+
qkit sharpe backtest.csv --column returns --trials 200
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
Both exit non-zero on a finding, so they work as a pre-commit hook or a CI gate.
|
|
130
|
+
|
|
131
|
+
## Limitations
|
|
132
|
+
|
|
133
|
+
- The lookahead detector is correlational and univariate. A feature that leaks
|
|
134
|
+
only in combination with another will pass, and a centred rolling window can
|
|
135
|
+
sit just under the threshold.
|
|
136
|
+
- Purged k-fold assumes a fixed label span; variable-horizon labels need
|
|
137
|
+
per-observation windows, which are not implemented.
|
|
138
|
+
- Nothing here validates a strategy. It tells you when a backtest cannot be
|
|
139
|
+
believed, which is a different and smaller claim.
|
|
140
|
+
|
|
141
|
+
## Licence
|
|
142
|
+
|
|
143
|
+
MIT.
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
LICENSE
|
|
2
|
+
MANIFEST.in
|
|
3
|
+
PACKAGE_README.md
|
|
4
|
+
README.md
|
|
5
|
+
pyproject.toml
|
|
6
|
+
requirements.txt
|
|
7
|
+
qkit/__init__.py
|
|
8
|
+
qkit/cli.py
|
|
9
|
+
qkit/lookahead.py
|
|
10
|
+
qkit/statistics.py
|
|
11
|
+
qkit/universe.py
|
|
12
|
+
qkit/validation.py
|
|
13
|
+
qkit_research.egg-info/PKG-INFO
|
|
14
|
+
qkit_research.egg-info/SOURCES.txt
|
|
15
|
+
qkit_research.egg-info/dependency_links.txt
|
|
16
|
+
qkit_research.egg-info/entry_points.txt
|
|
17
|
+
qkit_research.egg-info/requires.txt
|
|
18
|
+
qkit_research.egg-info/top_level.txt
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
qkit
|