qkit-research 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Wenke Du
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,10 @@
1
+ include LICENSE
2
+ include PACKAGE_README.md
3
+ include requirements.txt
4
+ recursive-exclude tests *
5
+ recursive-exclude src *
6
+ recursive-exclude website *
7
+ recursive-exclude results *
8
+ recursive-exclude data *
9
+ recursive-exclude docs *
10
+ global-exclude __pycache__ *.py[cod] .DS_Store
@@ -0,0 +1,124 @@
1
+ # qkit-research
2
+
3
+ The checks a quantitative researcher forgets.
4
+
5
+ Not a backtester. Backtesters are plentiful and mostly fine; what goes wrong is
6
+ upstream of them, in four places that turn a strategy that does not work into a
7
+ beautiful equity curve:
8
+
9
+ | Check | What it catches |
10
+ |---|---|
11
+ | **Lookahead** | A feature that uses information from its own future |
12
+ | **Survivorship** | A universe that quietly excludes what died |
13
+ | **Label leakage** | Cross-validation folds sharing overlapping label windows |
14
+ | **Selection** | A Sharpe ratio reported after trying two hundred variants |
15
+
16
+ Each returns a *finding*, not a number, so it drops into a research loop and
17
+ fails loudly.
18
+
19
+ ## Install
20
+
21
+ ```bash
22
+ pip install qkit-research
23
+ ```
24
+
25
+ Requires Python 3.9+ and numpy. Nothing else.
26
+
27
+ ## Lookahead
28
+
29
+ The convention: `feature[t]` is known at the end of period `t` and predicts
30
+ `target[t+1]`. Lookahead therefore means correlation with the *contemporaneous
31
+ or future* target. Correlation with an earlier target is **not** lookahead — a
32
+ momentum feature *is* the previous return.
33
+
34
+ ```python
35
+ import numpy as np
36
+ from qkit import detect_lookahead, shift_safe
37
+
38
+ returns = np.random.default_rng(0).normal(0, 0.01, 1000)
39
+
40
+ leaky = np.roll(returns, -1) # copies next period's return
41
+ print(detect_lookahead(leaky, returns, "leaky").contaminated) # True
42
+
43
+ momentum = np.roll(returns, 1) # legitimate: last period's return
44
+ print(detect_lookahead(momentum, returns, "momentum").contaminated) # False
45
+
46
+ fixed = shift_safe(leaky, 2) # the one-line remedy
47
+ ```
48
+
49
+ The threshold is a plausibility bound on forward predictability, not a
50
+ significance test. Genuine edges in liquid markets correlate 0.02–0.10; a
51
+ feature correlating 0.6 with a future return has seen the answer.
52
+
53
+ ## Survivorship
54
+
55
+ ```python
56
+ from qkit import audit_universe
57
+
58
+ finding = audit_universe(returns_matrix, alive_at_end)
59
+ print(finding.bias_bps) # basis points per period
60
+ ```
61
+
62
+ Takes the full universe *including the dead* — exactly the data most
63
+ survivorship-biased studies lack. When it is unavailable the honest output is a
64
+ warning; this library does not correct a bias it cannot measure.
65
+
66
+ ## Purged cross-validation
67
+
68
+ Standard k-fold leaks whenever a label spans several periods. Purging removes
69
+ the overlap; an embargo removes the serial correlation that survives it.
70
+
71
+ ```python
72
+ from qkit import PurgedKFold
73
+
74
+ cv = PurgedKFold(n_splits=5, label_span=20, embargo=20)
75
+ for train, test in cv.split(len(X)):
76
+ model.fit(X[train], y[train])
77
+
78
+ print(cv.leakage_check(len(X))) # folds_with_overlap should be 0
79
+ ```
80
+
81
+ With `label_span=1` and `embargo=0` it reduces to ordinary contiguous k-fold.
82
+
83
+ ## Deflated Sharpe
84
+
85
+ A Sharpe of 2.0 found after testing two hundred variants is not the same
86
+ quantity as a Sharpe of 2.0 from a single pre-registered strategy.
87
+
88
+ ```python
89
+ from qkit import sharpe, deflated_sharpe, min_track_record_length
90
+
91
+ print(sharpe(r))
92
+ print(deflated_sharpe(r, n_trials=200))
93
+ print(min_track_record_length(r))
94
+ ```
95
+
96
+ On 200 strategies generated from pure noise — true Sharpe exactly zero — the
97
+ best shows an observed Sharpe of 1.82 and a probabilistic Sharpe against zero of
98
+ 0.999. Deflating for the trial count drops it to 0.65 and it does not survive.
99
+
100
+ Deflation assumes independent trials. Two hundred variants of one idea are not
101
+ independent, and the correction is then too weak.
102
+
103
+ ## Command line
104
+
105
+ ```bash
106
+ qkit lookahead features.csv --target fwd_return --features mom_5 vol_20 rsi
107
+ qkit sharpe backtest.csv --column returns --trials 200
108
+ ```
109
+
110
+ Both exit non-zero on a finding, so they work as a pre-commit hook or a CI gate.
111
+
112
+ ## Limitations
113
+
114
+ - The lookahead detector is correlational and univariate. A feature that leaks
115
+ only in combination with another will pass, and a centred rolling window can
116
+ sit just under the threshold.
117
+ - Purged k-fold assumes a fixed label span; variable-horizon labels need
118
+ per-observation windows, which are not implemented.
119
+ - Nothing here validates a strategy. It tells you when a backtest cannot be
120
+ believed, which is a different and smaller claim.
121
+
122
+ ## Licence
123
+
124
+ MIT.
@@ -0,0 +1,143 @@
1
+ Metadata-Version: 2.4
2
+ Name: qkit-research
3
+ Version: 0.1.0
4
+ Summary: The checks a quantitative researcher forgets: lookahead detection, survivorship auditing, purged cross-validation and deflated Sharpe.
5
+ Author: Wenke Du
6
+ License: MIT
7
+ Keywords: quantitative,backtest,lookahead,purged,sharpe
8
+ Classifier: Development Status :: 3 - Alpha
9
+ Classifier: Intended Audience :: Financial and Insurance Industry
10
+ Classifier: License :: OSI Approved :: MIT License
11
+ Classifier: Programming Language :: Python :: 3
12
+ Requires-Python: >=3.9
13
+ Description-Content-Type: text/markdown
14
+ License-File: LICENSE
15
+ Requires-Dist: numpy>=1.24
16
+ Provides-Extra: dev
17
+ Requires-Dist: pytest>=7.0; extra == "dev"
18
+ Dynamic: license-file
19
+
20
+ # qkit-research
21
+
22
+ The checks a quantitative researcher forgets.
23
+
24
+ Not a backtester. Backtesters are plentiful and mostly fine; what goes wrong is
25
+ upstream of them, in four places that turn a strategy that does not work into a
26
+ beautiful equity curve:
27
+
28
+ | Check | What it catches |
29
+ |---|---|
30
+ | **Lookahead** | A feature that uses information from its own future |
31
+ | **Survivorship** | A universe that quietly excludes what died |
32
+ | **Label leakage** | Cross-validation folds sharing overlapping label windows |
33
+ | **Selection** | A Sharpe ratio reported after trying two hundred variants |
34
+
35
+ Each returns a *finding*, not a number, so it drops into a research loop and
36
+ fails loudly.
37
+
38
+ ## Install
39
+
40
+ ```bash
41
+ pip install qkit-research
42
+ ```
43
+
44
+ Requires Python 3.9+ and numpy. Nothing else.
45
+
46
+ ## Lookahead
47
+
48
+ The convention: `feature[t]` is known at the end of period `t` and predicts
49
+ `target[t+1]`. Lookahead therefore means correlation with the *contemporaneous
50
+ or future* target. Correlation with an earlier target is **not** lookahead — a
51
+ momentum feature *is* the previous return.
52
+
53
+ ```python
54
+ import numpy as np
55
+ from qkit import detect_lookahead, shift_safe
56
+
57
+ returns = np.random.default_rng(0).normal(0, 0.01, 1000)
58
+
59
+ leaky = np.roll(returns, -1) # copies next period's return
60
+ print(detect_lookahead(leaky, returns, "leaky").contaminated) # True
61
+
62
+ momentum = np.roll(returns, 1) # legitimate: last period's return
63
+ print(detect_lookahead(momentum, returns, "momentum").contaminated) # False
64
+
65
+ fixed = shift_safe(leaky, 2) # the one-line remedy
66
+ ```
67
+
68
+ The threshold is a plausibility bound on forward predictability, not a
69
+ significance test. Genuine edges in liquid markets correlate 0.02–0.10; a
70
+ feature correlating 0.6 with a future return has seen the answer.
71
+
72
+ ## Survivorship
73
+
74
+ ```python
75
+ from qkit import audit_universe
76
+
77
+ finding = audit_universe(returns_matrix, alive_at_end)
78
+ print(finding.bias_bps) # basis points per period
79
+ ```
80
+
81
+ Takes the full universe *including the dead* — exactly the data most
82
+ survivorship-biased studies lack. When it is unavailable the honest output is a
83
+ warning; this library does not correct a bias it cannot measure.
84
+
85
+ ## Purged cross-validation
86
+
87
+ Standard k-fold leaks whenever a label spans several periods. Purging removes
88
+ the overlap; an embargo removes the serial correlation that survives it.
89
+
90
+ ```python
91
+ from qkit import PurgedKFold
92
+
93
+ cv = PurgedKFold(n_splits=5, label_span=20, embargo=20)
94
+ for train, test in cv.split(len(X)):
95
+ model.fit(X[train], y[train])
96
+
97
+ print(cv.leakage_check(len(X))) # folds_with_overlap should be 0
98
+ ```
99
+
100
+ With `label_span=1` and `embargo=0` it reduces to ordinary contiguous k-fold.
101
+
102
+ ## Deflated Sharpe
103
+
104
+ A Sharpe of 2.0 found after testing two hundred variants is not the same
105
+ quantity as a Sharpe of 2.0 from a single pre-registered strategy.
106
+
107
+ ```python
108
+ from qkit import sharpe, deflated_sharpe, min_track_record_length
109
+
110
+ print(sharpe(r))
111
+ print(deflated_sharpe(r, n_trials=200))
112
+ print(min_track_record_length(r))
113
+ ```
114
+
115
+ On 200 strategies generated from pure noise — true Sharpe exactly zero — the
116
+ best shows an observed Sharpe of 1.82 and a probabilistic Sharpe against zero of
117
+ 0.999. Deflating for the trial count drops it to 0.65 and it does not survive.
118
+
119
+ Deflation assumes independent trials. Two hundred variants of one idea are not
120
+ independent, and the correction is then too weak.
121
+
122
+ ## Command line
123
+
124
+ ```bash
125
+ qkit lookahead features.csv --target fwd_return --features mom_5 vol_20 rsi
126
+ qkit sharpe backtest.csv --column returns --trials 200
127
+ ```
128
+
129
+ Both exit non-zero on a finding, so they work as a pre-commit hook or a CI gate.
130
+
131
+ ## Limitations
132
+
133
+ - The lookahead detector is correlational and univariate. A feature that leaks
134
+ only in combination with another will pass, and a centred rolling window can
135
+ sit just under the threshold.
136
+ - Purged k-fold assumes a fixed label span; variable-horizon labels need
137
+ per-observation windows, which are not implemented.
138
+ - Nothing here validates a strategy. It tells you when a backtest cannot be
139
+ believed, which is a different and smaller claim.
140
+
141
+ ## Licence
142
+
143
+ MIT.
@@ -0,0 +1,95 @@
1
+ # qkit-research
2
+
3
+ > The four checks that decide whether a backtest means anything: lookahead, survivorship, label leakage, and how many strategies you tried before this one.
4
+
5
+ **Repository:** `5.0-Ai-Engineering-Toolkit` · **Pillar:** Financial Stability
6
+
7
+ ## Status
8
+
9
+ This is working code with a runnable demo and 28 tests. It is **not** a
10
+ finished result.
11
+
12
+ Release artifacts (sdist + wheel) are built and pass twine check, but the package is NOT published to PyPI — no downloads, no users. Every demonstration below runs on synthetic series where the contamination was put there deliberately, which is the only way to show a detector finds it.
13
+
14
+ Last run: `2026-08-31T21:08:19+00:00`
15
+
16
+ ## Quick start
17
+
18
+ ```bash
19
+ pip install -r requirements.txt
20
+ python -m pytest tests/ -q # 28 tests
21
+ python -m src.demo # runs everything, rewrites results/ and website/
22
+ ```
23
+
24
+ ## Layout
25
+
26
+ ```
27
+ LICENSE
28
+ MANIFEST.in
29
+ PACKAGE_README.md
30
+ README.md
31
+ data/
32
+ |-- README.md
33
+ |-- manifests/
34
+ |-- sample/
35
+ dist/
36
+ |-- qkit_research-0.1.0-py3-none-any.whl
37
+ |-- qkit_research-0.1.0.tar.gz
38
+ docs/
39
+ |-- DATA.md
40
+ |-- METHOD.md
41
+ pyproject.toml
42
+ qkit/
43
+ |-- __init__.py
44
+ |-- cli.py
45
+ |-- lookahead.py
46
+ |-- statistics.py
47
+ |-- universe.py
48
+ |-- validation.py
49
+ qkit_research.egg-info/
50
+ |-- PKG-INFO
51
+ |-- SOURCES.txt
52
+ |-- dependency_links.txt
53
+ |-- entry_points.txt
54
+ |-- requires.txt
55
+ |-- top_level.txt
56
+ requirements.txt
57
+ results/
58
+ |-- README.md
59
+ |-- latest.json
60
+ src/
61
+ |-- .gitkeep
62
+ |-- __init__.py
63
+ |-- demo.py
64
+ |-- site.py
65
+ |-- sitekit.py
66
+ ```
67
+
68
+ - `src/` — the implementation.
69
+ - `tests/` — pytest suite. These guard behaviour, not just imports.
70
+ - `results/latest.json` — the output of the last demo run. Every figure quoted
71
+ anywhere in this project traces back to this file.
72
+ - `website/` — a self-contained static site, deployable to Vercel by copying the
73
+ folder into its own repository. See `website/README.md`.
74
+
75
+ ## The website
76
+
77
+ `website/` has no build step. To deploy it independently:
78
+
79
+ ```bash
80
+ cp -r website/ ../my-3-quant-productivity-toolkit-site && cd ../my-3-quant-productivity-toolkit-site
81
+ git init && git add -A && git commit -m "site"
82
+ vercel deploy --prod
83
+ ```
84
+
85
+ The page is regenerated from `results.json` on every `python -m src.demo`, so the
86
+ figures on the site and the figures the code produces cannot drift apart. Do not edit
87
+ numbers on the page by hand.
88
+
89
+ ## Honesty note
90
+
91
+ Everything in this project runs on clearly-labelled synthetic or authored data.
92
+ Swap in the real source and the same pipeline reports real numbers — that is
93
+ what the structure is for. Until that happens, nothing here should be cited as a
94
+ measured result, and the site's closing section states explicitly what the project
95
+ does not establish.
@@ -0,0 +1,29 @@
1
+ [build-system]
2
+ requires = ["setuptools>=68", "wheel"]
3
+ build-backend = "setuptools.build_meta"
4
+
5
+ [project]
6
+ name = "qkit-research"
7
+ version = "0.1.0"
8
+ description = "The checks a quantitative researcher forgets: lookahead detection, survivorship auditing, purged cross-validation and deflated Sharpe."
9
+ readme = "PACKAGE_README.md"
10
+ requires-python = ">=3.9"
11
+ license = {text = "MIT"}
12
+ authors = [{name = "Wenke Du"}]
13
+ keywords = ["quantitative", "backtest", "lookahead", "purged", "sharpe"]
14
+ classifiers = [
15
+ "Development Status :: 3 - Alpha",
16
+ "Intended Audience :: Financial and Insurance Industry",
17
+ "License :: OSI Approved :: MIT License",
18
+ "Programming Language :: Python :: 3",
19
+ ]
20
+ dependencies = ["numpy>=1.24"]
21
+
22
+ [project.optional-dependencies]
23
+ dev = ["pytest>=7.0"]
24
+
25
+ [project.scripts]
26
+ qkit = "qkit.cli:main"
27
+
28
+ [tool.setuptools.packages.find]
29
+ include = ["qkit*"]
@@ -0,0 +1,28 @@
1
+ """qkit — the checks a quantitative researcher forgets.
2
+
3
+ Not a backtester. Backtesters are plentiful and mostly fine; what goes wrong is
4
+ upstream of them, in four places that produce a beautiful equity curve from a
5
+ strategy that does not work:
6
+
7
+ lookahead a feature that uses information from its own future
8
+ survivorship a universe that quietly excludes what died
9
+ leakage cross-validation folds that share overlapping label windows
10
+ selection a Sharpe ratio reported after trying two hundred variants
11
+
12
+ Each is a function that returns a finding, not a number, so it can be dropped
13
+ into an existing research loop and fail loudly.
14
+ """
15
+
16
+ from .lookahead import LookaheadFinding, detect_lookahead, shift_safe
17
+ from .universe import SurvivorshipFinding, audit_universe
18
+ from .validation import PurgedKFold, embargo_indices
19
+ from .statistics import (deflated_sharpe, min_track_record_length,
20
+ probabilistic_sharpe, sharpe)
21
+
22
+ __version__ = "0.1.0"
23
+
24
+ __all__ = ["LookaheadFinding", "detect_lookahead", "shift_safe",
25
+ "SurvivorshipFinding", "audit_universe",
26
+ "PurgedKFold", "embargo_indices",
27
+ "sharpe", "probabilistic_sharpe", "deflated_sharpe",
28
+ "min_track_record_length", "__version__"]
@@ -0,0 +1,68 @@
1
+ """qkit command line: audit a returns/feature CSV before you trust a backtest."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import argparse
6
+ import csv
7
+ import json
8
+ import sys
9
+
10
+ import numpy as np
11
+
12
+ from .lookahead import detect_lookahead
13
+ from .statistics import deflated_sharpe, min_track_record_length, sharpe
14
+
15
+
16
+ def _load(path, cols):
17
+ with open(path, newline="", encoding="utf8") as fh:
18
+ rows = list(csv.DictReader(fh))
19
+ return {c: np.array([float(r[c]) for r in rows]) for c in cols}
20
+
21
+
22
+ def cmd_lookahead(args) -> int:
23
+ data = _load(args.csv, [args.target] + args.features)
24
+ y = data[args.target]
25
+ out = [detect_lookahead(data[f], y, f, threshold=args.threshold).as_dict()
26
+ for f in args.features]
27
+ json.dump({"findings": out,
28
+ "n_contaminated": sum(o["contaminated"] for o in out)},
29
+ sys.stdout, indent=2)
30
+ print()
31
+ return 1 if any(o["contaminated"] for o in out) else 0
32
+
33
+
34
+ def cmd_sharpe(args) -> int:
35
+ r = _load(args.csv, [args.column])[args.column]
36
+ out = {"sharpe": round(sharpe(r, args.periods), 4),
37
+ "deflated": deflated_sharpe(r, args.trials, args.periods),
38
+ "min_track_record_length": round(
39
+ min_track_record_length(r, periods_per_year=args.periods), 1)}
40
+ json.dump(out, sys.stdout, indent=2)
41
+ print()
42
+ return 0 if out["deflated"]["survives"] else 1
43
+
44
+
45
+ def main(argv=None) -> int:
46
+ ap = argparse.ArgumentParser(prog="qkit", description=__doc__)
47
+ sub = ap.add_subparsers(dest="cmd", required=True)
48
+
49
+ a = sub.add_parser("lookahead", help="flag features that see the future")
50
+ a.add_argument("csv")
51
+ a.add_argument("--target", required=True)
52
+ a.add_argument("--features", nargs="+", required=True)
53
+ a.add_argument("--threshold", type=float, default=0.50)
54
+ a.set_defaults(fn=cmd_lookahead)
55
+
56
+ b = sub.add_parser("sharpe", help="Sharpe, deflated for the number of trials")
57
+ b.add_argument("csv")
58
+ b.add_argument("--column", default="returns")
59
+ b.add_argument("--trials", type=int, default=1)
60
+ b.add_argument("--periods", type=int, default=252)
61
+ b.set_defaults(fn=cmd_sharpe)
62
+
63
+ args = ap.parse_args(argv)
64
+ return args.fn(args)
65
+
66
+
67
+ if __name__ == "__main__":
68
+ raise SystemExit(main())
@@ -0,0 +1,103 @@
1
+ """Detecting features that use information they could not have had.
2
+
3
+ The convention throughout: `feature[t]` is known at the end of period t and is
4
+ used to predict `target[t+1]`, the return realised over the NEXT period.
5
+
6
+ Under that convention lookahead means `feature[t]` carries information about
7
+ `target[t]` or later. Correlation with EARLIER targets is not lookahead -- a
8
+ momentum feature is literally the previous return, and correlating perfectly
9
+ with it is the definition of momentum, not a bug.
10
+
11
+ Getting that backwards is easy and was the first version of this module: it
12
+ tested correlation with past targets, flagged a momentum feature as
13
+ contaminated, and passed a feature that was a copy of next period's return. The
14
+ direction of the test is the whole content of it.
15
+
16
+ The detection rule is a plausibility bound rather than a significance test.
17
+ Genuine forward predictability in a liquid market is weak -- correlations of
18
+ 0.02 to 0.10 are a real edge. A feature correlating 0.6 with a future return has
19
+ not found an edge; it has seen the answer.
20
+ """
21
+
22
+ from __future__ import annotations
23
+
24
+ from dataclasses import dataclass
25
+
26
+ import numpy as np
27
+
28
+
29
+ @dataclass
30
+ class LookaheadFinding:
31
+ feature: str
32
+ max_forward_corr: float # with target[t+k], k >= 0 -- suspicious
33
+ lag_of_max: int # 0 = contemporaneous, 1 = next period
34
+ max_backward_corr: float # with target[t-k], k >= 1 -- legitimate
35
+ contaminated: bool
36
+ detail: str
37
+
38
+ def as_dict(self) -> dict:
39
+ return {"feature": self.feature,
40
+ "max_forward_corr": round(self.max_forward_corr, 5),
41
+ "lag_of_max": self.lag_of_max,
42
+ "max_backward_corr": round(self.max_backward_corr, 5),
43
+ "contaminated": self.contaminated, "detail": self.detail}
44
+
45
+
46
+ def _corr(a, b) -> float:
47
+ if len(a) < 8 or np.std(a) < 1e-12 or np.std(b) < 1e-12:
48
+ return 0.0
49
+ return float(np.corrcoef(a, b)[0, 1])
50
+
51
+
52
+ def detect_lookahead(feature: np.ndarray, target: np.ndarray,
53
+ name: str = "feature", max_lag: int = 3,
54
+ threshold: float = 0.50) -> LookaheadFinding:
55
+ """Flag a feature that correlates implausibly with a contemporaneous or
56
+ future target.
57
+
58
+ `threshold` is a plausibility bound on forward predictability, not a
59
+ p-value. Lowering it below about 0.3 starts flagging strong-but-real signals
60
+ on short samples.
61
+ """
62
+ f = np.asarray(feature, float)
63
+ y = np.asarray(target, float)
64
+ n = min(len(f), len(y))
65
+ f, y = f[:n], y[:n]
66
+
67
+ # Forward: feature[t] against target[t + k] for k >= 0.
68
+ fwd, fwd_lag = 0.0, 0
69
+ for k in range(0, max_lag + 1):
70
+ c = abs(_corr(f[:n - k], y[k:])) if k else abs(_corr(f, y))
71
+ if c > fwd:
72
+ fwd, fwd_lag = c, k
73
+
74
+ # Backward: feature[t] against target[t - k] for k >= 1. Legitimate.
75
+ bwd = 0.0
76
+ for k in range(1, max_lag + 1):
77
+ bwd = max(bwd, abs(_corr(f[k:], y[:-k])))
78
+
79
+ bad = fwd > threshold
80
+ when = ("the same period" if fwd_lag == 0
81
+ else f"{fwd_lag} period(s) ahead")
82
+ detail = (f"|corr(f[t], y[t+{fwd_lag}])| = {fwd:.3f} exceeds {threshold}; "
83
+ f"the feature knows about {when}" if bad
84
+ else f"max forward correlation {fwd:.3f} is within the plausible "
85
+ f"range (backward {bwd:.3f} is not evidence of lookahead)")
86
+ return LookaheadFinding(name, fwd, fwd_lag, bwd, bad, detail)
87
+
88
+
89
+ def shift_safe(x: np.ndarray, periods: int = 1) -> np.ndarray:
90
+ """Shift a feature so it can only use information already available.
91
+
92
+ The one-line fix for most lookahead. It is in the package because the
93
+ mistake is not knowing it is needed, not being unable to write it.
94
+ """
95
+ x = np.asarray(x, float)
96
+ out = np.full_like(x, np.nan)
97
+ if periods > 0:
98
+ out[periods:] = x[:-periods]
99
+ elif periods < 0:
100
+ out[:periods] = x[-periods:]
101
+ else:
102
+ out[:] = x
103
+ return out
@@ -0,0 +1,103 @@
1
+ """Sharpe ratios that account for how many strategies were tried.
2
+
3
+ A Sharpe of 2.0 found after testing two hundred variants is not the same
4
+ quantity as a Sharpe of 2.0 from a single pre-registered strategy, and reporting
5
+ them identically is the most common way a backtest misleads.
6
+ """
7
+
8
+ from __future__ import annotations
9
+
10
+ import math
11
+
12
+ import numpy as np
13
+
14
+
15
+ def sharpe(returns: np.ndarray, periods_per_year: int = 252) -> float:
16
+ r = np.asarray(returns, float)
17
+ sd = r.std(ddof=1)
18
+ if sd < 1e-15:
19
+ return 0.0
20
+ return float(r.mean() / sd * math.sqrt(periods_per_year))
21
+
22
+
23
+ def _norm_cdf(x: float) -> float:
24
+ return 0.5 * (1 + math.erf(x / math.sqrt(2)))
25
+
26
+
27
+ def _norm_ppf(p: float) -> float:
28
+ lo, hi = -12.0, 12.0
29
+ for _ in range(200):
30
+ mid = (lo + hi) / 2
31
+ if _norm_cdf(mid) < p:
32
+ lo = mid
33
+ else:
34
+ hi = mid
35
+ return (lo + hi) / 2
36
+
37
+
38
+ def probabilistic_sharpe(returns: np.ndarray, benchmark_sr: float = 0.0,
39
+ periods_per_year: int = 252) -> float:
40
+ """P(true Sharpe > benchmark), adjusting for skew and kurtosis.
41
+
42
+ Non-normal returns are the norm, and a Sharpe ratio computed on them has a
43
+ standard error the textbook formula understates.
44
+ """
45
+ r = np.asarray(returns, float)
46
+ n = len(r)
47
+ if n < 3:
48
+ return float("nan")
49
+ sr = sharpe(r, periods_per_year) / math.sqrt(periods_per_year) # per period
50
+ bench = benchmark_sr / math.sqrt(periods_per_year)
51
+ sd = r.std(ddof=1)
52
+ if sd < 1e-15:
53
+ return float("nan")
54
+ z = (r - r.mean()) / sd
55
+ skew = float(np.mean(z ** 3))
56
+ kurt = float(np.mean(z ** 4))
57
+ denom = math.sqrt(max(1e-12,
58
+ (1 - skew * sr + (kurt - 1) / 4 * sr ** 2) / (n - 1)))
59
+ return float(_norm_cdf((sr - bench) / denom))
60
+
61
+
62
+ def deflated_sharpe(returns: np.ndarray, n_trials: int,
63
+ periods_per_year: int = 252,
64
+ trial_sr_variance: float | None = None) -> dict:
65
+ """Sharpe adjusted for the number of strategies tried.
66
+
67
+ The expected maximum Sharpe across `n_trials` independent random strategies
68
+ is not zero; it grows with the number tried. The deflated ratio asks whether
69
+ the observed Sharpe beats that expectation.
70
+ """
71
+ r = np.asarray(returns, float)
72
+ obs = sharpe(r, periods_per_year)
73
+ v = trial_sr_variance if trial_sr_variance is not None else 1.0 / len(r)
74
+ if n_trials < 2:
75
+ expected_max = 0.0
76
+ else:
77
+ e = 0.5772156649
78
+ expected_max = math.sqrt(v) * (
79
+ (1 - e) * _norm_ppf(1 - 1.0 / n_trials)
80
+ + e * _norm_ppf(1 - 1.0 / (n_trials * math.e)))
81
+ expected_max *= math.sqrt(periods_per_year)
82
+ psr = probabilistic_sharpe(r, benchmark_sr=expected_max,
83
+ periods_per_year=periods_per_year)
84
+ return {"observed_sharpe": round(obs, 4), "n_trials": n_trials,
85
+ "expected_max_sharpe_from_noise": round(expected_max, 4),
86
+ "deflated_sharpe_prob": round(psr, 5),
87
+ "survives": bool(psr > 0.95)}
88
+
89
+
90
+ def min_track_record_length(returns: np.ndarray, benchmark_sr: float = 0.0,
91
+ confidence: float = 0.95,
92
+ periods_per_year: int = 252) -> float:
93
+ """Observations needed before a Sharpe is distinguishable from the benchmark."""
94
+ r = np.asarray(returns, float)
95
+ sr = sharpe(r, periods_per_year) / math.sqrt(periods_per_year)
96
+ bench = benchmark_sr / math.sqrt(periods_per_year)
97
+ if abs(sr - bench) < 1e-12:
98
+ return float("inf")
99
+ sd = r.std(ddof=1)
100
+ z = (r - r.mean()) / sd if sd > 1e-15 else r * 0
101
+ skew, kurt = float(np.mean(z ** 3)), float(np.mean(z ** 4))
102
+ num = (1 - skew * sr + (kurt - 1) / 4 * sr ** 2)
103
+ return float(1 + num * (_norm_ppf(confidence) / (sr - bench)) ** 2)
@@ -0,0 +1,53 @@
1
+ """Survivorship auditing.
2
+
3
+ A universe assembled from names that exist today has already excluded everything
4
+ that failed. The resulting backtest is not optimistic by a little; on a long
5
+ sample it can invert the sign of a result.
6
+ """
7
+
8
+ from __future__ import annotations
9
+
10
+ from dataclasses import dataclass
11
+
12
+ import numpy as np
13
+
14
+
15
+ @dataclass
16
+ class SurvivorshipFinding:
17
+ n_universe: int
18
+ n_survivors: int
19
+ n_delisted: int
20
+ survivor_share: float
21
+ biased: bool
22
+ mean_return_all: float
23
+ mean_return_survivors: float
24
+ bias_bps: float
25
+
26
+ def as_dict(self) -> dict:
27
+ return {"n_universe": self.n_universe, "n_survivors": self.n_survivors,
28
+ "n_delisted": self.n_delisted,
29
+ "survivor_share": round(self.survivor_share, 4),
30
+ "biased": self.biased,
31
+ "mean_return_all": round(self.mean_return_all, 6),
32
+ "mean_return_survivors": round(self.mean_return_survivors, 6),
33
+ "bias_bps": round(self.bias_bps, 2)}
34
+
35
+
36
+ def audit_universe(returns: np.ndarray, alive_at_end: np.ndarray) -> SurvivorshipFinding:
37
+ """Quantify what restricting to survivors would do to the mean return.
38
+
39
+ Takes the full universe including the dead, which is exactly the data most
40
+ survivorship-biased studies do not have. When it is unavailable the honest
41
+ output is a warning, not an adjustment -- this function does not attempt to
42
+ correct a bias it cannot measure.
43
+ """
44
+ R = np.asarray(returns, float)
45
+ alive = np.asarray(alive_at_end, bool)
46
+ all_mean = float(np.nanmean(R))
47
+ surv_mean = float(np.nanmean(R[:, alive])) if alive.any() else float("nan")
48
+ n = R.shape[1]
49
+ return SurvivorshipFinding(
50
+ n_universe=n, n_survivors=int(alive.sum()), n_delisted=int((~alive).sum()),
51
+ survivor_share=float(alive.mean()), biased=bool((~alive).any()),
52
+ mean_return_all=all_mean, mean_return_survivors=surv_mean,
53
+ bias_bps=(surv_mean - all_mean) * 10_000)
@@ -0,0 +1,68 @@
1
+ """Purged, embargoed cross-validation for overlapping labels.
2
+
3
+ Standard k-fold leaks whenever a label spans several periods: a training
4
+ observation whose label window overlaps a test observation shares information
5
+ with it, and the model sees the answer. Purging removes the overlap; an embargo
6
+ removes the serial correlation that survives it.
7
+ """
8
+
9
+ from __future__ import annotations
10
+
11
+ import numpy as np
12
+
13
+
14
+ def embargo_indices(n: int, test_start: int, test_end: int,
15
+ embargo: int) -> np.ndarray:
16
+ """Indices to drop after the test block, as a boolean mask of length n."""
17
+ mask = np.zeros(n, bool)
18
+ mask[test_end:min(n, test_end + embargo)] = True
19
+ return mask
20
+
21
+
22
+ class PurgedKFold:
23
+ """K-fold that purges overlapping labels and applies an embargo.
24
+
25
+ `label_span` is how many periods a label covers. With span 1 and embargo 0
26
+ this reduces to ordinary contiguous k-fold, which is the check that the
27
+ implementation is not doing something exotic.
28
+ """
29
+
30
+ def __init__(self, n_splits: int = 5, label_span: int = 1, embargo: int = 0):
31
+ if n_splits < 2:
32
+ raise ValueError("n_splits must be at least 2")
33
+ self.n_splits = n_splits
34
+ self.label_span = label_span
35
+ self.embargo = embargo
36
+
37
+ def split(self, n: int):
38
+ fold_edges = np.linspace(0, n, self.n_splits + 1).astype(int)
39
+ for i in range(self.n_splits):
40
+ a, b = fold_edges[i], fold_edges[i + 1]
41
+ test = np.zeros(n, bool)
42
+ test[a:b] = True
43
+
44
+ train = ~test
45
+ # Purge: any training label whose window reaches into the test block.
46
+ lo = max(0, a - self.label_span)
47
+ train[lo:b] = False
48
+ # Embargo: drop the periods immediately after the test block.
49
+ train &= ~embargo_indices(n, a, b, self.embargo)
50
+ yield train, test
51
+
52
+ def leakage_check(self, n: int) -> dict:
53
+ """How many observations each fold drops, and whether any overlap remains."""
54
+ drops, overlaps = [], 0
55
+ for train, test in self.split(n):
56
+ drops.append(int(n - train.sum() - test.sum()))
57
+ tr = np.where(train)[0]
58
+ te = np.where(test)[0]
59
+ if len(tr) and len(te):
60
+ # An overlap survives if a training index's label window reaches
61
+ # a test index.
62
+ for t in tr:
63
+ if np.any((te >= t) & (te < t + self.label_span)):
64
+ overlaps += 1
65
+ break
66
+ return {"n_splits": self.n_splits, "label_span": self.label_span,
67
+ "embargo": self.embargo, "dropped_per_fold": drops,
68
+ "folds_with_overlap": overlaps}
@@ -0,0 +1,143 @@
1
+ Metadata-Version: 2.4
2
+ Name: qkit-research
3
+ Version: 0.1.0
4
+ Summary: The checks a quantitative researcher forgets: lookahead detection, survivorship auditing, purged cross-validation and deflated Sharpe.
5
+ Author: Wenke Du
6
+ License: MIT
7
+ Keywords: quantitative,backtest,lookahead,purged,sharpe
8
+ Classifier: Development Status :: 3 - Alpha
9
+ Classifier: Intended Audience :: Financial and Insurance Industry
10
+ Classifier: License :: OSI Approved :: MIT License
11
+ Classifier: Programming Language :: Python :: 3
12
+ Requires-Python: >=3.9
13
+ Description-Content-Type: text/markdown
14
+ License-File: LICENSE
15
+ Requires-Dist: numpy>=1.24
16
+ Provides-Extra: dev
17
+ Requires-Dist: pytest>=7.0; extra == "dev"
18
+ Dynamic: license-file
19
+
20
+ # qkit-research
21
+
22
+ The checks a quantitative researcher forgets.
23
+
24
+ Not a backtester. Backtesters are plentiful and mostly fine; what goes wrong is
25
+ upstream of them, in four places that turn a strategy that does not work into a
26
+ beautiful equity curve:
27
+
28
+ | Check | What it catches |
29
+ |---|---|
30
+ | **Lookahead** | A feature that uses information from its own future |
31
+ | **Survivorship** | A universe that quietly excludes what died |
32
+ | **Label leakage** | Cross-validation folds sharing overlapping label windows |
33
+ | **Selection** | A Sharpe ratio reported after trying two hundred variants |
34
+
35
+ Each returns a *finding*, not a number, so it drops into a research loop and
36
+ fails loudly.
37
+
38
+ ## Install
39
+
40
+ ```bash
41
+ pip install qkit-research
42
+ ```
43
+
44
+ Requires Python 3.9+ and numpy. Nothing else.
45
+
46
+ ## Lookahead
47
+
48
+ The convention: `feature[t]` is known at the end of period `t` and predicts
49
+ `target[t+1]`. Lookahead therefore means correlation with the *contemporaneous
50
+ or future* target. Correlation with an earlier target is **not** lookahead — a
51
+ momentum feature *is* the previous return.
52
+
53
+ ```python
54
+ import numpy as np
55
+ from qkit import detect_lookahead, shift_safe
56
+
57
+ returns = np.random.default_rng(0).normal(0, 0.01, 1000)
58
+
59
+ leaky = np.roll(returns, -1) # copies next period's return
60
+ print(detect_lookahead(leaky, returns, "leaky").contaminated) # True
61
+
62
+ momentum = np.roll(returns, 1) # legitimate: last period's return
63
+ print(detect_lookahead(momentum, returns, "momentum").contaminated) # False
64
+
65
+ fixed = shift_safe(leaky, 2) # the one-line remedy
66
+ ```
67
+
68
+ The threshold is a plausibility bound on forward predictability, not a
69
+ significance test. Genuine edges in liquid markets correlate 0.02–0.10; a
70
+ feature correlating 0.6 with a future return has seen the answer.
71
+
72
+ ## Survivorship
73
+
74
+ ```python
75
+ from qkit import audit_universe
76
+
77
+ finding = audit_universe(returns_matrix, alive_at_end)
78
+ print(finding.bias_bps) # basis points per period
79
+ ```
80
+
81
+ Takes the full universe *including the dead* — exactly the data most
82
+ survivorship-biased studies lack. When it is unavailable the honest output is a
83
+ warning; this library does not correct a bias it cannot measure.
84
+
85
+ ## Purged cross-validation
86
+
87
+ Standard k-fold leaks whenever a label spans several periods. Purging removes
88
+ the overlap; an embargo removes the serial correlation that survives it.
89
+
90
+ ```python
91
+ from qkit import PurgedKFold
92
+
93
+ cv = PurgedKFold(n_splits=5, label_span=20, embargo=20)
94
+ for train, test in cv.split(len(X)):
95
+ model.fit(X[train], y[train])
96
+
97
+ print(cv.leakage_check(len(X))) # folds_with_overlap should be 0
98
+ ```
99
+
100
+ With `label_span=1` and `embargo=0` it reduces to ordinary contiguous k-fold.
101
+
102
+ ## Deflated Sharpe
103
+
104
+ A Sharpe of 2.0 found after testing two hundred variants is not the same
105
+ quantity as a Sharpe of 2.0 from a single pre-registered strategy.
106
+
107
+ ```python
108
+ from qkit import sharpe, deflated_sharpe, min_track_record_length
109
+
110
+ print(sharpe(r))
111
+ print(deflated_sharpe(r, n_trials=200))
112
+ print(min_track_record_length(r))
113
+ ```
114
+
115
+ On 200 strategies generated from pure noise — true Sharpe exactly zero — the
116
+ best shows an observed Sharpe of 1.82 and a probabilistic Sharpe against zero of
117
+ 0.999. Deflating for the trial count drops it to 0.65 and it does not survive.
118
+
119
+ Deflation assumes independent trials. Two hundred variants of one idea are not
120
+ independent, and the correction is then too weak.
121
+
122
+ ## Command line
123
+
124
+ ```bash
125
+ qkit lookahead features.csv --target fwd_return --features mom_5 vol_20 rsi
126
+ qkit sharpe backtest.csv --column returns --trials 200
127
+ ```
128
+
129
+ Both exit non-zero on a finding, so they work as a pre-commit hook or a CI gate.
130
+
131
+ ## Limitations
132
+
133
+ - The lookahead detector is correlational and univariate. A feature that leaks
134
+ only in combination with another will pass, and a centred rolling window can
135
+ sit just under the threshold.
136
+ - Purged k-fold assumes a fixed label span; variable-horizon labels need
137
+ per-observation windows, which are not implemented.
138
+ - Nothing here validates a strategy. It tells you when a backtest cannot be
139
+ believed, which is a different and smaller claim.
140
+
141
+ ## Licence
142
+
143
+ MIT.
@@ -0,0 +1,18 @@
1
+ LICENSE
2
+ MANIFEST.in
3
+ PACKAGE_README.md
4
+ README.md
5
+ pyproject.toml
6
+ requirements.txt
7
+ qkit/__init__.py
8
+ qkit/cli.py
9
+ qkit/lookahead.py
10
+ qkit/statistics.py
11
+ qkit/universe.py
12
+ qkit/validation.py
13
+ qkit_research.egg-info/PKG-INFO
14
+ qkit_research.egg-info/SOURCES.txt
15
+ qkit_research.egg-info/dependency_links.txt
16
+ qkit_research.egg-info/entry_points.txt
17
+ qkit_research.egg-info/requires.txt
18
+ qkit_research.egg-info/top_level.txt
@@ -0,0 +1,2 @@
1
+ [console_scripts]
2
+ qkit = qkit.cli:main
@@ -0,0 +1,4 @@
1
+ numpy>=1.24
2
+
3
+ [dev]
4
+ pytest>=7.0
@@ -0,0 +1,2 @@
1
+ numpy>=1.24
2
+ pytest>=7.0
@@ -0,0 +1,4 @@
1
+ [egg_info]
2
+ tag_build =
3
+ tag_date = 0
4
+