permutalpha 0.9.1__tar.gz

permutalpha-0.9.1/CHANGELOG.md

@@ -0,0 +1,180 @@
+# Changelog
+
+Changes that can alter what a test measures are marked **(affects results)**. A frozen card records the lab
+version it was frozen with, and its gate report shows both versions if they differ.
+
+## 0.9.1
+- Published on PyPI as **permutalpha** (`pip install permutalpha`; the import name and the `oql` command are
+  unchanged, and `permutalpha` works as a command too). Extras: `[trees]`, `[paper]`, `[all]`.
+- `oql demo [DIR]`: a ready lab on synthetic markets with two frozen cards, so a fresh install shows a PASS and a
+  FAIL in minutes.
+
+## 0.9.0: bulletproofing
+- **(affects results)** No position without a trustworthy volatility estimate. A flat or stale stretch (half of
+  the last 20 moves exactly zero: a pegged currency, a frozen feed) had a tiny measured volatility, and dividing by it
+  put the market at the 10x leverage cap. Such bars now get no position; normal markets are unaffected.
+- **Frozen data means the data at the freeze.** The freeze records each dataset's last timestamp and a fingerprint
+  of its rows up to then. Gate and holdout read only those rows, so the daily collector appending new data no longer
+  blocks a frozen card, while any change to an existing row (a vendor revision, an edit) is still refused. Cards
+  frozen before 0.9 keep the old whole-file check.
+- **Tests that cannot work are refused at the freeze:** n_perm too small to reach p < alpha, alpha above 0.2,
+  negative costs, a development share outside 30-95%, a fill delay outside [0, 1), unknown rules or gates, a
+  dataset listed twice, an experiment.py that does not load.
+- **Tamper-evident ledger.** Every row carries a hash chained to the row before; `oql ledger verify` reports any
+  edited, removed or reordered row, and `oql ledger head` prints the latest hash to publish (e.g. in a commit
+  message). An existing ledger is sealed as it stands the first time a row is added.
+- **Safe with several processes.** Board changes happen under a lab lock; a gate or holdout records its result on a
+  fresh copy of the board, so cards created or edited while a long gate ran are no longer overwritten; two gate
+  runs of the same card cannot overlap; ledger appends cannot fork the chain.
+- New test suite `tests/test_robust.py`: no look-ahead across every rule, side and gate setting; engine invariants on
+  messy markets (gaps, frozen feeds, fat tails); costs only ever reduce returns; the gate is deterministic across
+  process counts; frozen-data and validation behaviour; ledger tampering; concurrency.
+- Continuous integration (`.github/workflows/tests.yml`): the full suite on Python 3.10 with pandas 2 and Python
+  3.12 with pandas 3, on every push.
+- Removed deprecated `Timestamp.utcnow` calls (gone in a future pandas); live volatility ignores zero prices.
+
+## 0.8.0
+- **Third null model: block bootstrap** (`nulls=["block"]`). The history is cut into 20-bar blocks (random grid
+  offset) and the blocks are shuffled, one order for all markets: volatility clustering, short-term persistence and
+  cross-market co-movement survive inside each block; longer patterns are broken.
+- Calibration (docs/CALIBRATION.md): false alarms 3% and 2% on no-edge markets; it catches 88% of planted trend
+  edges against 96-98% for the other two nulls, because 20-bar blocks keep part of each ~67-bar trend. It is the
+  strictest test for trend strategies and only credits structure longer than a block.
+
+## 0.7.1
+- **Holdout power analysis.** A holdout FAIL only means something if the holdout could have seen the edge. Every
+  holdout now writes `holdout.md` with: the Sharpe's 95% interval, the bar it had to beat, the smallest true Sharpe
+  it detects 80% of the time, the chance it would have passed if the edge were as strong as in development, and a
+  test of whether the edge shrank. `oql card holdout-power ID` writes the same report for a holdout already run
+  (it recomputes the shuffles for the frozen finalist; the recorded verdict, ledger and stage are unchanged).
+
+## 0.7.0
+- **Project 11, large search** (Projects tab): the lab's search loop at growing sizes, up to about 3,000 distinct
+  strategies from dense grids, on a synthetic book with a known truth. For each size it compares a naive t-test of
+  the winner, the Deflated Sharpe Ratio, and the gate. On a no-edge book the naive test calls a ~700-strategy winner
+  significant (p 0.03) while the gate does not (p 0.55); on a planted edge the gate passes at every size.
+- Power study (docs/CALIBRATION.md): with about 11 years of data, edges whose best strategy shows a Sharpe near 0.4
+  are caught 8-15% of the time; edges near Sharpe 1 over decades pass reliably.
+- Calibration can run the large-search space (`--kinds big1000-noise`).
+
+## 0.6.2
+- `oql data pull-long-history-excess` **(fixes a data flaw found by card 0003)**: the long histories as returns a
+  financed position actually earns, under `fred-xs/`. Treasuries earn yield minus the 3-month T-bill rate (the
+  0.6.1 bond indexes credited the full yield, so a leveraged low-volatility bond looked like a riskless trend);
+  currencies earn the spot move plus the interest-rate difference (OECD 3-month rates, used one month late), and
+  start when that rate is known. Oil is left out: its tradable futures return cannot be built from the spot price.
+  The 0.6.1 `fred-bars/` datasets are kept unchanged, because card 0003 was frozen on them.
+
+## 0.6.1
+- `oql data pull-long-history`: free FRED daily series turned into tradable bars under `fred-bars/`: 12 currencies
+  (from 1971, as the foreign currency's price in dollars), constant-maturity Treasuries as total-return indexes
+  (from 1962; -duration x yield change + carry) and WTI oil (from 1986). Close-only, so open = previous close.
+  Decades of history give tests the statistical power that 2010-onward data cannot.
+- Calibration can run power studies on weaker planted trends (`--kinds drift0.0001,...`).
+
+## 0.6.0
+- **Second null model** (`Experiment(nulls=["bars", "vol_matched"])`; `write_experiment(nulls=...)`). The
+  volatility-matched null divides each bar by the market's volatility at its time, shuffles the standardized bars,
+  and multiplies each by the volatility of the slot it lands in: calm periods stay calm and crises stay at their
+  dates, and only the order of the moves is destroyed. With several nulls declared, the gate and the holdout PASS
+  only if p < alpha under every one; the report shows each. Default stays `["bars"]`, so existing cards are unchanged.
+- **Calibration study** (`python -m oql.calibration`, docs/CALIBRATION.md): 300 full gates on synthetic books with a
+  known truth. Both nulls keep false positives at or below 5% (3-4%; 1% under strong volatility clustering) and pass
+  96-98% of real edges. Under strong clustering the bar shuffle is too strict (p-values pile up near 1); the
+  volatility-matched null is better calibrated there.
+
+## 0.5.2
+- **Rebalance band** (opt-in, `Experiment(rebalance_band=0.25)`; `write_experiment(rebalance_band=...)` for the
+  assistant; "Resizing" in the gate check). A position keeps its size until the volatility-based target has moved more
+  than the band away, then jumps to the target, so a strategy stops paying to resize every bar while staying within
+  the band of its risk target. Paper trading puts the catalog's history in front of the live bars so its sizes match
+  the backtest's. On synthetic hourly data a 25% band cut the cost bill of a slow rule (250-hour momentum) by about
+  30% and of a fast one (20-hour) by about 8%: for fast rules most of the cost is direction changes, not resizing.
+  Experiments without it are unchanged.
+
+## 0.5.1
+- The gate report counts efficiency studies and gate checks on the card's datasets made before its freeze
+  ("Efficiency-lab explorations on these datasets before the freeze"). They shape which ideas get proposed, so they
+  belong next to the result. Recorded at the freeze; cards frozen earlier say "not recorded".
+- A project whose data is not in the catalog says what is missing and how to add it, instead of failing.
+- `oql serve` on a port already in use says so (it was reported as a network error).
+- No more "All-NaN slice" warning when a feature has no values yet early in a study.
+
+## 0.5.0
+- **Efficiency lab** (`oql/efficiency.py`, `oql efficiency study`, the dashboard's Efficiency tab, the assistant's
+  `efficiency_study` tool). Is the next day's (hourly bars) or hour's (minute bars) efficiency ratio predictable?
+  Walk-forward ridge (or boosted trees) on development data only, from price features plus optional calendar,
+  funding, perp-spot basis, VIX and yield slope, CFTC positioning and Pi order-book groups, all joined as of the
+  period start. The order-book data from the micro study's lock date on stays sealed. Each run first checks itself on
+  a synthetic market with planted efficiency regimes and on a random walk, and is counted in the ledger.
+- **Efficiency gates in the search** (`oql/gates.py`). An experiment can declare `gates=["none", "er_trailing",
+  "er_forecast"]`; the search may then switch any strategy on only in the top (or bottom) third or half of trailing
+  efficiency or of the efficiency forecast. Gates are computed from the bars inside the search, so the gate test
+  reruns them on every shuffled copy. Switching pays costs. Paper trading computes the gate the same way, with the
+  catalog's history in front of the live journal. Experiments without `gates` are unchanged.
+- **Gate check** (`oql efficiency gates`, the Efficiency tab): all 21 gate settings on one strategy, P&L by forecast
+  third, and a simulation of how strong a forecast would need to be. Counted in the ledger as 21 explorations.
+- New rule `orb` (opening-range breakout on intraday bars), available by name; not in the default search.
+- Assistant tools: `explore` and `permutation_check` take an optional gate; `write_experiment` takes `gates` and
+  `gate_features` and counts gate settings in the search size; new `efficiency_study`.
+- Ledger counts (command line, dashboard, assistant) add up the settings a row stands for, so a gate check counts 21.
+- Charts: ranked horizontal bars for long category names.
+
+## 0.4.0
+- **Live dashboard.** `oql serve` can run any computation in the background and stream its results to the page as they
+  arrive (`oql/live.py`, Server-Sent Events); charts redraw as the work happens, with hover readouts and a validated
+  colour palette for light and dark mode (`oql/web/live.js`).
+- **Projects tab.** The ten quant mini-projects (tail index, intraday seasonality, overfitting detector, momentum
+  crashes, factor neutrality, American put, autodiff Greeks, implied distribution, Hawkes trades, news diffusion) run
+  live on the lab's catalog data (`oql/projects/`). Each runs its known-answer checks first and shows them passing or
+  failing; the four projects with published real-data results reproduce them.
+- **Live market tab.** Public exchange data (Kraken, Coinbase, Binance.US; no account) polled by the server: price,
+  the rolling one-hour efficiency ratio, realized volatility and the bid-ask spread, updating on their own
+  (`oql/market.py`).
+- New data commands: `oql data pull-french` (Fama-French factor tables) and `oql data import-projects DIR` (the
+  projects' SPY option chain, licensed and kept local, and a day of Binance.US trades).
+- The ledger tab counts tests (gate and holdout runs) separately from explorations.
+- Packaging includes the `oql.projects` package and the page script, so a regular `pip install` works.
+
+## 0.3.4
+Found in the first dry run with an AI assistant connected over MCP:
+- `write_experiment` and `explore` now say that every strategy trades all of an experiment's datasets together as one
+  equal-risk portfolio. The assistant had counted 4 lookbacks on 2 coins as 8 strategies.
+- Editing a card after asking for review marks the review summary as out of date, on the card and on the dashboard,
+  until the assistant sends a fresh one.
+- The gate report explains a result worse than every shuffled run: the signal pointed the wrong way more often than
+  chance, and testing the opposite idea would be a new card and a new counted test.
+
+## 0.3.3
+- `oql data check` compares prices with a relative tolerance. Prices stored as decimals round slightly, and at large
+  levels (Nikkei futures in the tens of thousands) that rounding was being reported as broken bars.
+- Large one-bar moves are split in two: spikes that reverse on the next bar (the mark of a bad price) and moves that
+  stick (real events, or an unadjusted futures roll). Each warning names a date to look at, as does the gap warning.
+  This does not change any test result.
+
+## 0.3.2
+- **(affects results)** A trade is now a change of direction (enter, exit or flip). Positions are resized every
+  bar to hold volatility steady; those resizes are still charged as costs, but they were also being counted as
+  trades, so every strategy "traded" every bar and the minimum-trades rule never rejected anything. The minimum
+  is now 1 direction change per year per market (`min_trades_per_year=1.0`): it rejects strategies that almost
+  never change position, and keeps slow trend systems such as 12-month momentum on daily data.
+- Exploration (`explore`, `permutation_check`) now uses the costs and fill delay declared in the card's
+  `experiment.py`, so an AI assistant sees what the gate will see. Results report `cost_bps` and
+  `direction_changes_per_year`.
+- `oql card freeze` records the lab version; the gate report shows the version at the freeze and at the gate.
+- `oql ledger` separates tests (gate, holdout) from explorations in its summary.
+- `oql data check` warns about bars that are not on the dataset's time grid (found in real Binance data: 43 hourly
+  bars after the February 2018 outage opened 28 minutes past the hour).
+- The code repository ignores lab state (`cards/`, `board.json`, `LEDGER.csv`): run a lab in its own folder.
+- `demo/check_parity.py` checks that the live demo page computes the same numbers as this code.
+
+## 0.3.1
+- Binance futures files start with a header row; the parser now reads them, and number columns stored as text
+  are converted before saving.
+
+## 0.3.0
+- Phase 5: `oql mcp`, an MCP server for AI research assistants. Every exploration is counted and reported in the
+  gate report.
+
+## 0.2.0
+- Data catalog, validation gate, research board, dashboard (`oql serve`), collector, paper trading.

permutalpha-0.9.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 David DeConti
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

permutalpha-0.9.1/MANIFEST.in

@@ -0,0 +1,7 @@
+include README.md LICENSE NOTICE.md CHANGELOG.md requirements.txt
+recursive-include docs *.md *.json
+recursive-include tests *.py
+recursive-include examples *.py
+recursive-include deploy *
+include demo/check_parity.py
+global-exclude *.pyc __pycache__

permutalpha-0.9.1/NOTICE.md

@@ -0,0 +1,9 @@
+# Credits
+
+- **Bar permutation** (`oql/permute.py`, `permute_ohlc`) is adapted from
+  [neurotrader888/mcpt](https://github.com/neurotrader888/mcpt) `bar_permute.py`, MIT License, (c) 2025 neurotrader888,
+  by way of the author's earlier project mcpt-lab.
+- **Deflated Sharpe ratio**: Bailey, D. H. and López de Prado, M. (2014), "The Deflated Sharpe Ratio".
+- **Pipeline shape**: inspired by QuantConnect's Research Pipeline. No QuantConnect code is used.
+- Data from FRED (Federal Reserve Bank of St. Louis), Binance public data, the U.S. Bureau of Labor Statistics and the
+  Federal Reserve are public. Databento data is licensed and is never included in this repository.

permutalpha-0.9.1/PKG-INFO

@@ -0,0 +1,187 @@
+Metadata-Version: 2.4
+Name: permutalpha
+Version: 0.9.1
+Summary: A trading-research pipeline where every result must beat the same search on shuffled markets: pre-registration, permutation gates, one-shot holdouts, a tamper-evident ledger, and an AI assistant that can explore but never test
+Author: David DeConti
+License: MIT
+Project-URL: Homepage, https://github.com/dnd8639-byte/permutalpha-quant
+Project-URL: Source, https://github.com/dnd8639-byte/permutalpha-quant/tree/main/lab
+Project-URL: Documentation, https://github.com/dnd8639-byte/permutalpha-quant/tree/main/lab/docs
+Project-URL: Changelog, https://github.com/dnd8639-byte/permutalpha-quant/blob/main/lab/CHANGELOG.md
+Project-URL: Live demo, https://dnd8639-byte.github.io/open-quant-lab/
+Keywords: quantitative finance,backtesting,permutation test,overfitting,pre-registration,trading,research,mcp,ai agents
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: Intended Audience :: Financial and Insurance Industry
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Office/Business :: Financial :: Investment
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+License-File: NOTICE.md
+Requires-Dist: numpy>=1.24
+Requires-Dist: pandas>=2.0
+Requires-Dist: pyarrow>=12
+Requires-Dist: certifi
+Provides-Extra: trees
+Requires-Dist: scikit-learn>=1.2; extra == "trees"
+Provides-Extra: paper
+Requires-Dist: ccxt>=4; extra == "paper"
+Provides-Extra: all
+Requires-Dist: scikit-learn>=1.2; extra == "all"
+Requires-Dist: ccxt>=4; extra == "all"
+Dynamic: license-file
+
+# permutalpha (Open Quant Lab)
+
+```bash
+pip install permutalpha
+oql demo && OQL_LAB=oql-demo OQL_CATALOG=oql-demo/catalog oql serve
+```
+
+A fully open research pipeline for trading strategies: a data catalog, a backtest engine with costs and delays, and a validation gate that the strategy has to pass before anything else happens. The rules of every test are enforced by code, not by good intentions.
+
+Designed and directed by David DeConti, with much of the code written with an AI assistant (Claude). MIT licence. Inspired by the shape of QuantConnect's research pipeline (Ideas → Research → Backtest → Paper → Live), rebuilt so that no part of it is paid, and so that a strategy cannot be promoted by trying things until one looks good.
+
+**Live demo:** the validation gate runs in your browser at [dnd8639-byte.github.io/open-quant-lab](https://dnd8639-byte.github.io/open-quant-lab/).
+It is a JavaScript port of the engine, rules and search; `python demo/check_parity.py` confirms that it computes the same
+numbers as this code.
+
+```
+idea ──> research ──> preregistered ──> gate ──> holdout ──> paper
+  explore freely   rules fingerprinted   whole-search      one-shot     Raspberry Pi
+                   (SHA-256)             permutation test  (locked)     paper trader
+                                         FAIL ─> archive   FAIL ─> archive
+```
+
+## What makes it different
+
+| Rule | How the code enforces it |
+|---|---|
+| Decide the test before seeing the result | `oql card freeze` fingerprints `PREREGISTRATION.md`, `experiment.py` and every dataset (SHA-256). The gate and the holdout refuse to run if any of them changed. |
+| Judge the best of many tries as the best of many tries | The gate reruns the **entire search**, every round, on shuffled copies of the data. p = how often luck alone finds something as good. |
+| Count every attempt | Every gate and holdout run goes into `LEDGER.csv`, pass or fail. Changing a tested idea means `oql card revise`: a new card in the same family, and the family's trial count keeps adding up. |
+| Look at the holdout once | `oql card holdout` writes its lock file before it computes anything. A second run is refused, even if someone edits `board.json`. |
+| No skipping | Cards move one stage at a time; a failed gate or holdout sends the card to the archive. |
+| Licensed data never leaks | Every dataset carries a licence tag (`public`, `own`, `licensed`); the whole catalog is git-ignored. |
+
+The lab checks itself: on fake markets with a planted, real edge the gate passes (p = 0.025, the smallest possible with 39 permutations, in 5 of 5 draws); on pure noise it fails (p between 0.30 and 0.85 in 5 of 5 draws). Run `python examples/quickstart.py` to watch both happen.
+
+## Install
+
+```bash
+pip install permutalpha              # numpy, pandas, pyarrow
+pip install "permutalpha[all]"       # + scikit-learn (tree forecasts) and ccxt (live paper trading)
+oql demo                             # a lab on synthetic markets, two cards frozen and ready for their gate
+OQL_LAB=oql-demo OQL_CATALOG=oql-demo/catalog oql serve
+```
+
+Run both gates from the Research board: the planted edge passes and the pure noise fails. From source:
+
+```bash
+git clone https://github.com/dnd8639-byte/permutalpha-quant && cd permutalpha-quant/lab
+pip install -e . && python -m unittest discover tests
+```
+
+## An AI research assistant that cannot cheat
+
+`oql mcp` serves the lab to any Model Context Protocol client (Claude Desktop, Claude Code, or a local model). The
+assistant can read the board and data, propose ideas, write pre-registrations and experiments from validated fields
+(never code), explore development data and run efficiency studies. It cannot freeze a card, run the gate or the
+holdout, or deploy: those stay with a person. Every exploration it makes is logged and printed next to the final
+result. Setup: [docs/AGENTS.md](https://github.com/dnd8639-byte/permutalpha-quant/blob/main/lab/docs/AGENTS.md).
+
+## Use
+
+```bash
+oql data pull-fred DGS10 CPIAUCSL                 # free, no key
+oql data pull-binance BTCUSDT --interval 1h --start 2018-01
+oql data import-databento ../state/v3/contracts --roots ES,CL,GC   # licensed: stays local
+oql data pull-french                              # Fama-French factors, for the projects
+oql data import-projects ../quant-projects/data   # the projects' option chain (licensed) and trades
+oql data list
+oql data check
+
+oql card new "Time-series momentum on CME futures" --mechanism "Slow-moving capital and hedging demand make trends persist."
+#   edit cards/0001-.../PREREGISTRATION.md and experiment.py
+oql card start 1
+oql card freeze 1           # then commit and push: the commit time proves the rules came first
+oql card gate 1             # search + whole-search permutation test -> cards/0001-.../gate.md
+oql card holdout 1          # one shot
+oql board --html board.html
+oql ledger
+oql serve                   # the dashboard: live gate runs, the permutation lab, data, ledger
+oql efficiency study binance/BTCUSDT/1h --groups funding,basis,macro   # is efficiency predictable? (dev data, logged)
+oql efficiency gates binance/BTCUSDT/1h "momentum lookback=20"          # does an efficiency gate help this strategy?
+oql paper deploy 1 --feed binance/BTCUSDT/1h=binanceus:BTC/USDT:1h   # after a holdout PASS
+oql collect init && oql collect run                                  # keep free data current (Pi)
+```
+
+## The dashboard
+
+`oql serve` opens a local web app (nothing leaves your machine):
+
+- **Research board**: every card by stage. Run a preregistered card's gate from the page and watch the shuffled
+  reruns arrive one by one, the null histogram fill in, and the p-value settle.
+- **Permutation lab**: pick a market and a strategy; the lab shuffles the market in front of you and builds the
+  distribution of luck one permutation at a time. Switch to "best of the whole search" to see selection bias.
+  Uses only the first 70% of each dataset, so it never touches a holdout.
+- **Data** and **Ledger**: the catalog with quality checks, and every test ever run.
+- **Projects**: ten quant mini-projects (tail risk, seasonality, overfitting, momentum crashes, factor neutrality,
+  American options, autodiff Greeks, implied distributions, Hawkes processes, news diffusion) running live on the
+  catalog's data. Each runs its known-answer checks first; charts fill in as the work happens.
+- **Live market**: price, the rolling one-hour efficiency ratio, volatility and the spread from public exchange data
+  (Kraken, Coinbase, Binance.US), updating every few seconds. No account or key.
+- **Efficiency**: can the next day's (or hour's) efficiency be forecast from price, calendar, funding, basis, macro,
+  positioning and order-book data, walk-forward on development data? And does switching a strategy on and off by
+  efficiency help it, and how strong would the forecast need to be? Every run checks itself on synthetic markets
+  first and is counted in the ledger. Gates found here are tested for real by declaring them in a card's experiment
+  (`gates=[...]`), where the gate test reruns them on shuffled data. See [docs/EFFICIENCY.md](https://github.com/dnd8639-byte/permutalpha-quant/blob/main/lab/docs/EFFICIENCY.md).
+
+Holdouts are deliberately not runnable from the page: they are one-shot, so they need a typed command.
+
+## Layout
+
+| Path | What it is |
+|---|---|
+| `oql/catalog.py` | Parquet catalog with metadata, licence tags, SHA-256 and quality checks |
+| `oql/sources.py` | FRED, Binance public files, CSV, release calendar, Pi order-book features, Databento front contracts (roll-safe) |
+| `oql/engine.py` | Bar engine: decisions at the close, fills a fraction into the next bar, costs per change, equal-risk sizing |
+| `oql/strategies.py` | Rule library (breakout, moving-average cross, momentum, reversal, opening-range breakout); add your own per experiment |
+| `oql/gates.py` | Efficiency gates a search can add to any strategy (trailing efficiency, or the efficiency forecast) |
+| `oql/efficiency.py` | The efficiency lab: targets, look-ahead-free features, walk-forward forecasts, gate checks |
+| `oql/search.py` | The safe search loop: fixed rounds, elites survive, every strategy logged |
+| `oql/permute.py` | Joint bar permutation across markets (adapted from neurotrader888's MIT-licensed method), plus a volatility-matched null |
+| `oql/calibration.py` | Measures the gate's own false-positive rate and power on synthetic books (docs/CALIBRATION.md) |
+| `oql/gate.py` | The gate and the holdout |
+| `oql/stats.py` | Deflated Sharpe, cost and delay Monte Carlo, long/short split, beta |
+| `oql/board.py` | Cards, stages, enforced moves, reports, the HTML board |
+| `oql/server.py`, `oql/web/` | The `oql serve` dashboard (standard library only) |
+| `oql/live.py`, `oql/web/live.js` | Background jobs streamed to the page; the shared live chart renderer |
+| `oql/projects/` | The ten mini-projects, each with its known-answer checks |
+| `oql/market.py` | Live public exchange data for the Live market tab |
+| `oql/collect.py` | Scheduled, incremental, checksum-verified downloads |
+| `oql/paper.py` | Paper trading: journal, revisions, reconciliation |
+| `oql/agent_tools.py`, `oql/mcp_server.py` | What an AI assistant may do, served over MCP |
+| `oql/ledger.py`, `oql/locks.py` | Every test counted; freeze fingerprints and one-shot locks |
+| `demo/check_parity.py` | Checks that the live demo page computes the same numbers as the Python lab |
+
+## Roadmap
+
+- [x] Phase 1: data catalog and loaders, plus `oql collect` for scheduled downloads on a Raspberry Pi
+- [x] Phase 2: paper trading with the same code as the backtest (`oql paper`), with replay, data and returns
+      reconciliation; a NautilusTrader adapter remains optional
+- [x] Phase 3: validation gate
+- [x] Phase 4: pipeline board (command line + HTML)
+- [x] Phase 5: an MCP server (`oql mcp`) so an AI assistant can propose cards, write pre-registrations and
+      experiments, and explore development data; it cannot freeze, run the gate or holdout, or deploy, and every
+      exploration it makes is counted in the gate report. See [docs/AGENTS.md](https://github.com/dnd8639-byte/permutalpha-quant/blob/main/lab/docs/AGENTS.md)
+
+Building on it: [docs/DEVELOPING.md](https://github.com/dnd8639-byte/permutalpha-quant/blob/main/lab/docs/DEVELOPING.md). Running it on a Pi: [deploy/README.md](https://github.com/dnd8639-byte/permutalpha-quant/blob/main/lab/deploy/README.md).
+See [NOTICE.md](https://github.com/dnd8639-byte/permutalpha-quant/blob/main/lab/NOTICE.md) for credits.

permutalpha-0.9.1/README.md

@@ -0,0 +1,147 @@
+# permutalpha (Open Quant Lab)
+
+```bash
+pip install permutalpha
+oql demo && OQL_LAB=oql-demo OQL_CATALOG=oql-demo/catalog oql serve
+```
+
+A fully open research pipeline for trading strategies: a data catalog, a backtest engine with costs and delays, and a validation gate that the strategy has to pass before anything else happens. The rules of every test are enforced by code, not by good intentions.
+
+Designed and directed by David DeConti, with much of the code written with an AI assistant (Claude). MIT licence. Inspired by the shape of QuantConnect's research pipeline (Ideas → Research → Backtest → Paper → Live), rebuilt so that no part of it is paid, and so that a strategy cannot be promoted by trying things until one looks good.
+
+**Live demo:** the validation gate runs in your browser at [dnd8639-byte.github.io/open-quant-lab](https://dnd8639-byte.github.io/open-quant-lab/).
+It is a JavaScript port of the engine, rules and search; `python demo/check_parity.py` confirms that it computes the same
+numbers as this code.
+
+```
+idea ──> research ──> preregistered ──> gate ──> holdout ──> paper
+  explore freely   rules fingerprinted   whole-search      one-shot     Raspberry Pi
+                   (SHA-256)             permutation test  (locked)     paper trader
+                                         FAIL ─> archive   FAIL ─> archive
+```
+
+## What makes it different
+
+| Rule | How the code enforces it |
+|---|---|
+| Decide the test before seeing the result | `oql card freeze` fingerprints `PREREGISTRATION.md`, `experiment.py` and every dataset (SHA-256). The gate and the holdout refuse to run if any of them changed. |
+| Judge the best of many tries as the best of many tries | The gate reruns the **entire search**, every round, on shuffled copies of the data. p = how often luck alone finds something as good. |
+| Count every attempt | Every gate and holdout run goes into `LEDGER.csv`, pass or fail. Changing a tested idea means `oql card revise`: a new card in the same family, and the family's trial count keeps adding up. |
+| Look at the holdout once | `oql card holdout` writes its lock file before it computes anything. A second run is refused, even if someone edits `board.json`. |
+| No skipping | Cards move one stage at a time; a failed gate or holdout sends the card to the archive. |
+| Licensed data never leaks | Every dataset carries a licence tag (`public`, `own`, `licensed`); the whole catalog is git-ignored. |
+
+The lab checks itself: on fake markets with a planted, real edge the gate passes (p = 0.025, the smallest possible with 39 permutations, in 5 of 5 draws); on pure noise it fails (p between 0.30 and 0.85 in 5 of 5 draws). Run `python examples/quickstart.py` to watch both happen.
+
+## Install
+
+```bash
+pip install permutalpha              # numpy, pandas, pyarrow
+pip install "permutalpha[all]"       # + scikit-learn (tree forecasts) and ccxt (live paper trading)
+oql demo                             # a lab on synthetic markets, two cards frozen and ready for their gate
+OQL_LAB=oql-demo OQL_CATALOG=oql-demo/catalog oql serve
+```
+
+Run both gates from the Research board: the planted edge passes and the pure noise fails. From source:
+
+```bash
+git clone https://github.com/dnd8639-byte/permutalpha-quant && cd permutalpha-quant/lab
+pip install -e . && python -m unittest discover tests
+```
+
+## An AI research assistant that cannot cheat
+
+`oql mcp` serves the lab to any Model Context Protocol client (Claude Desktop, Claude Code, or a local model). The
+assistant can read the board and data, propose ideas, write pre-registrations and experiments from validated fields
+(never code), explore development data and run efficiency studies. It cannot freeze a card, run the gate or the
+holdout, or deploy: those stay with a person. Every exploration it makes is logged and printed next to the final
+result. Setup: [docs/AGENTS.md](https://github.com/dnd8639-byte/permutalpha-quant/blob/main/lab/docs/AGENTS.md).
+
+## Use
+
+```bash
+oql data pull-fred DGS10 CPIAUCSL                 # free, no key
+oql data pull-binance BTCUSDT --interval 1h --start 2018-01
+oql data import-databento ../state/v3/contracts --roots ES,CL,GC   # licensed: stays local
+oql data pull-french                              # Fama-French factors, for the projects
+oql data import-projects ../quant-projects/data   # the projects' option chain (licensed) and trades
+oql data list
+oql data check
+
+oql card new "Time-series momentum on CME futures" --mechanism "Slow-moving capital and hedging demand make trends persist."
+#   edit cards/0001-.../PREREGISTRATION.md and experiment.py
+oql card start 1
+oql card freeze 1           # then commit and push: the commit time proves the rules came first
+oql card gate 1             # search + whole-search permutation test -> cards/0001-.../gate.md
+oql card holdout 1          # one shot
+oql board --html board.html
+oql ledger
+oql serve                   # the dashboard: live gate runs, the permutation lab, data, ledger
+oql efficiency study binance/BTCUSDT/1h --groups funding,basis,macro   # is efficiency predictable? (dev data, logged)
+oql efficiency gates binance/BTCUSDT/1h "momentum lookback=20"          # does an efficiency gate help this strategy?
+oql paper deploy 1 --feed binance/BTCUSDT/1h=binanceus:BTC/USDT:1h   # after a holdout PASS
+oql collect init && oql collect run                                  # keep free data current (Pi)
+```
+
+## The dashboard
+
+`oql serve` opens a local web app (nothing leaves your machine):
+
+- **Research board**: every card by stage. Run a preregistered card's gate from the page and watch the shuffled
+  reruns arrive one by one, the null histogram fill in, and the p-value settle.
+- **Permutation lab**: pick a market and a strategy; the lab shuffles the market in front of you and builds the
+  distribution of luck one permutation at a time. Switch to "best of the whole search" to see selection bias.
+  Uses only the first 70% of each dataset, so it never touches a holdout.
+- **Data** and **Ledger**: the catalog with quality checks, and every test ever run.
+- **Projects**: ten quant mini-projects (tail risk, seasonality, overfitting, momentum crashes, factor neutrality,
+  American options, autodiff Greeks, implied distributions, Hawkes processes, news diffusion) running live on the
+  catalog's data. Each runs its known-answer checks first; charts fill in as the work happens.
+- **Live market**: price, the rolling one-hour efficiency ratio, volatility and the spread from public exchange data
+  (Kraken, Coinbase, Binance.US), updating every few seconds. No account or key.
+- **Efficiency**: can the next day's (or hour's) efficiency be forecast from price, calendar, funding, basis, macro,
+  positioning and order-book data, walk-forward on development data? And does switching a strategy on and off by
+  efficiency help it, and how strong would the forecast need to be? Every run checks itself on synthetic markets
+  first and is counted in the ledger. Gates found here are tested for real by declaring them in a card's experiment
+  (`gates=[...]`), where the gate test reruns them on shuffled data. See [docs/EFFICIENCY.md](https://github.com/dnd8639-byte/permutalpha-quant/blob/main/lab/docs/EFFICIENCY.md).
+
+Holdouts are deliberately not runnable from the page: they are one-shot, so they need a typed command.
+
+## Layout
+
+| Path | What it is |
+|---|---|
+| `oql/catalog.py` | Parquet catalog with metadata, licence tags, SHA-256 and quality checks |
+| `oql/sources.py` | FRED, Binance public files, CSV, release calendar, Pi order-book features, Databento front contracts (roll-safe) |
+| `oql/engine.py` | Bar engine: decisions at the close, fills a fraction into the next bar, costs per change, equal-risk sizing |
+| `oql/strategies.py` | Rule library (breakout, moving-average cross, momentum, reversal, opening-range breakout); add your own per experiment |
+| `oql/gates.py` | Efficiency gates a search can add to any strategy (trailing efficiency, or the efficiency forecast) |
+| `oql/efficiency.py` | The efficiency lab: targets, look-ahead-free features, walk-forward forecasts, gate checks |
+| `oql/search.py` | The safe search loop: fixed rounds, elites survive, every strategy logged |
+| `oql/permute.py` | Joint bar permutation across markets (adapted from neurotrader888's MIT-licensed method), plus a volatility-matched null |
+| `oql/calibration.py` | Measures the gate's own false-positive rate and power on synthetic books (docs/CALIBRATION.md) |
+| `oql/gate.py` | The gate and the holdout |
+| `oql/stats.py` | Deflated Sharpe, cost and delay Monte Carlo, long/short split, beta |
+| `oql/board.py` | Cards, stages, enforced moves, reports, the HTML board |
+| `oql/server.py`, `oql/web/` | The `oql serve` dashboard (standard library only) |
+| `oql/live.py`, `oql/web/live.js` | Background jobs streamed to the page; the shared live chart renderer |
+| `oql/projects/` | The ten mini-projects, each with its known-answer checks |
+| `oql/market.py` | Live public exchange data for the Live market tab |
+| `oql/collect.py` | Scheduled, incremental, checksum-verified downloads |
+| `oql/paper.py` | Paper trading: journal, revisions, reconciliation |
+| `oql/agent_tools.py`, `oql/mcp_server.py` | What an AI assistant may do, served over MCP |
+| `oql/ledger.py`, `oql/locks.py` | Every test counted; freeze fingerprints and one-shot locks |
+| `demo/check_parity.py` | Checks that the live demo page computes the same numbers as the Python lab |
+
+## Roadmap
+
+- [x] Phase 1: data catalog and loaders, plus `oql collect` for scheduled downloads on a Raspberry Pi
+- [x] Phase 2: paper trading with the same code as the backtest (`oql paper`), with replay, data and returns
+      reconciliation; a NautilusTrader adapter remains optional
+- [x] Phase 3: validation gate
+- [x] Phase 4: pipeline board (command line + HTML)
+- [x] Phase 5: an MCP server (`oql mcp`) so an AI assistant can propose cards, write pre-registrations and
+      experiments, and explore development data; it cannot freeze, run the gate or holdout, or deploy, and every
+      exploration it makes is counted in the gate report. See [docs/AGENTS.md](https://github.com/dnd8639-byte/permutalpha-quant/blob/main/lab/docs/AGENTS.md)
+
+Building on it: [docs/DEVELOPING.md](https://github.com/dnd8639-byte/permutalpha-quant/blob/main/lab/docs/DEVELOPING.md). Running it on a Pi: [deploy/README.md](https://github.com/dnd8639-byte/permutalpha-quant/blob/main/lab/deploy/README.md).
+See [NOTICE.md](https://github.com/dnd8639-byte/permutalpha-quant/blob/main/lab/NOTICE.md) for credits.