canli-backtest-mcp 0.2.0 → 0.4.0

package/README.md

@@ -29,6 +29,31 @@ Three mistakes make most agent-run backtests look better than they are:
 | `pit_factor` | A factor (`roa`, `gross_margin`, `operating_margin`, `revenue_growth`, `asset_growth`, `accruals`, or `level:<measure>`) for your tickers, monthly, as known on each date. Writes a signal CSV. |
 | `backtest_signal` | Rank on the signal, long the top quantile (and short the bottom), rebalance weekly, monthly, quarterly or every N days, with `lag_days`, `cost_bps` and an optional `ledger`. |
 
+## Breadth lab, sweeps and labelled data
+
+| tool | what it does |
+|---|---|
+| `price_panel` | A price, dollar-volume and industry-group panel built from a fund's holdings (IVV for the S&P 500), ready for the tools below |
+| `breadth_signals` | The 24 cross-sectional signals the lab builds sleeves from, each with its reference |
+| `breadth_book` | Every signal inside every group of names as a sleeve, traded as one netted account: the book's Sharpe after costs, the netting saved, and its effective independent bets; writes the sleeves as a CSV with their specs |
+| `breadth_sweep` | A grid of book settings run in parallel on every core, or as shards other machines run and merge back (inputs, grid, gaps and duplicates checked); the best Sharpe is set against what the best of that many trials reaches by luck, and with `split_date` the chosen settings are scored after it |
+| `effective_bets` | How many independent bets a set of return series holds (the participation ratio of their correlations), by series and by family |
+| `annotation_packet`, `annotation_agreement`, `annotation_gold` | Blind labelling packets of figures (or of a draft's figures) with guidelines and an answer key; agreement between labellers (Cohen's kappa); and a gold set with its datasheet |
+
+## Size, crises and cross-validation
+
+A backtest that assumes any size fills at the close flatters itself, so:
+
+- **Size.** Give `backtest_signal` a `capital_usd` and a `dollar_volume_file`. An order fills only up to `max_participation` of recent daily volume (10% unless you say otherwise) and pays square-root impact, with volume and volatility read from the days before each trade. The reply shows the Sharpe at $100k to $1B, and no Sharpe for a size the market could not take. Order-book depth is not modelled: it needs paid data.
+- **Crises.** Every reply holds the result against named crises, from the dot-com bust to the 2023 bank run. A crisis its dates do not fully cover is listed as untested, never as passed. `crisis_replay` does this for any returns file.
+- **Cross-validation.** `cpcv` takes every variant's returns and, for each choice of held-out blocks, chooses the best on the rest, with purge and embargo at the edges, then scores it on what was held out. You get a spread of out-of-sample results, not one path. Splits can run as shards on other machines and merge back.
+
+## A record, a league and a paper market
+
+- **Proof ledger.** `proof_filings` writes what companies filed into an append-only file, each line chained to the last and each batch signed, with a line for every figure a later filing changed. `proof_file` adds any file's hash, `proof_verify` checks the whole file and says which line first fails. It shows lines were not altered, not that a figure is right, and no record means only no record. Publish the head somewhere else, or the key holder could rewrite it all. The [published ledger](https://github.com/arhancanli/canlicapital/releases) does that with a public attestation each month.
+- **League.** `league_new`, `league_commit` and `league_score` run a tournament scored after the fact: an entrant signs and commits a hash before a round closes and shows the file afterwards, and anyone with the files can rerun the table. Each record is set against the best luck gives across that many entrants. The top entrants, chosen on earlier rounds only, form a paper book. No prizes, no stakes, nobody's money.
+- **Arena.** `arena_new`, `arena_orders` and `arena_step` are a paper market for agents to trade in, cleared in a call auction, reproducible from a seed. Made-up assets and made-up money: it cannot reach anything real.
+
 ## Quick start
 
 ```bash

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "canli-backtest-mcp",
-  "version": "0.2.0",
+  "version": "0.4.0",
   "description": "MCP server for backtests that refuse lookahead: point-in-time factors from SEC filings, cross-sectional backtests of any signal on your own prices, costs on actual trades, and every variant counted in a trial ledger.",
   "private": false,
   "type": "module",
@@ -21,7 +21,7 @@
   "dependencies": {
     "@modelcontextprotocol/server": "2.3.1",
     "canli-fundamentals-mcp": "^0.7.0",
-    "canli-markets-mcp": "^0.6.0",
+    "canli-markets-mcp": "^0.7.0",
     "canli-validation-mcp": "^0.14.0",
     "zod": "4.6.7"
   },

package/src/arena.mjs

@@ -0,0 +1,150 @@
+// The arena: a paper market for AI agents to trade in. Made-up assets and made-up money, cleared in a
+// uniform-price call auction each step, reproducible from a seed. It is here so agents' behaviour
+// together can be studied: who supplies liquidity, who gets picked off, what a crowd does to a price.
+//
+// It touches nothing real. No real security, token or currency is represented, no order leaves the
+// file, and nothing here can be turned into a real market: running one needs registration as an
+// exchange or as a broker-dealer running an alternative trading system.
+import { existsSync, readFileSync, writeFileSync } from "node:fs";
+import { z } from "zod";
+
+export const ARENA_LIMITS_TEXT = Object.freeze([
+  "Paper only: made-up assets and made-up money. Nothing here represents or can reach a real security, token or currency.",
+  "A simulated market leaves out most of what moves real ones, so behaviour here is a sketch for study and does not predict results in real markets.",
+]);
+
+const file = z.string().min(1).max(1000);
+const assetId = z.string().regex(/^ART[0-9]{1,2}$/, "an arena asset, such as ART1");
+// Names become keys in the state file, so ones every JavaScript object already answers to are refused.
+const agentId = z.string().regex(/^[A-Za-z0-9][A-Za-z0-9_-]{1,39}$/).refine((n) => !(n in {}), "that name is reserved");
+export const arenaNewInput = z.object({
+  arena_file: file.describe("The arena's state file; must not exist yet."),
+  seed: z.number().int().min(0).max(2 ** 31 - 2).optional().describe("The same seed and the same orders give the same market. Default 1."),
+  assets: z.number().int().min(1).max(20).optional().describe("Made-up assets. Default 3."),
+  starting_cash: z.number().positive().max(1e9).optional().describe("Made-up money each agent starts with. Default 100000."),
+  bots: z.number().int().min(0).max(50).optional().describe("Scripted traders that keep the market moving: noise, value and momentum in turn. Default 6."),
+}).strict();
+export const arenaOrdersInput = z.object({
+  arena_file: file,
+  agent: agentId,
+  orders: z.array(z.object({ asset: assetId, side: z.enum(["buy", "sell"]), quantity: z.number().int().positive().max(1e7), limit: z.number().positive().max(1e7) }).strict()).max(50).describe("Replaces this agent's orders for the coming step. No short selling and no borrowing."),
+}).strict();
+export const arenaStepInput = z.object({ arena_file: file, steps: z.number().int().min(1).max(1000).optional().describe("Default 1.") }).strict();
+
+const SHARES_PER_AGENT = 100;
+const load = (f) => { if (!existsSync(f)) throw new RangeError(`${f} is not an arena yet: call arena_new`); const s = JSON.parse(readFileSync(f, "utf8")); if (s.schema !== "canli.arena.v1") throw new RangeError(`${f} is not an arena file`); return s; };
+// Every order in a step comes from an agent in the file, however the file was written.
+const orderLines = (state) => Object.entries(state.orders).filter(([agent]) => Object.hasOwn(state.agents, agent));
+const save = (f, s) => writeFileSync(f, JSON.stringify(s));
+function rng(state) { state.rng = (state.rng * 16807) % 2147483647; return state.rng / 2147483647; }
+const gauss = (s) => Math.sqrt(-2 * Math.log(rng(s))) * Math.cos(2 * Math.PI * rng(s));
+const newAgent = (s) => ({ cash: s.starting_cash, holdings: Object.fromEntries(s.assets.map((x) => [x.id, SHARES_PER_AGENT])), start_worth: s.starting_cash + s.assets.reduce((n, x) => n + SHARES_PER_AGENT * x.last, 0) });
+const round2 = (x) => Math.round(x * 100) / 100;
+
+export function arenaNew(args) {
+  const a = arenaNewInput.parse(args);
+  if (existsSync(a.arena_file)) throw new RangeError(`${a.arena_file} already exists`);
+  const state = { schema: "canli.arena.v1", paper_only: true, step: 0, rng: (a.seed ?? 1) + 1, starting_cash: a.starting_cash ?? 100000, assets: [], agents: {}, orders: {}, history: [] };
+  for (let i = 0; i < (a.assets ?? 3); i += 1) { const value = round2(50 + 100 * rng(state)); state.assets.push({ id: `ART${i + 1}`, value, last: value }); }
+  const kinds = ["noise", "value", "momentum"];
+  state.bots = Array.from({ length: a.bots ?? 6 }, (_, i) => ({ name: `bot-${kinds[i % 3]}-${i}`, kind: kinds[i % 3] }));
+  for (const b of state.bots) state.agents[b.name] = newAgent(state);
+  save(a.arena_file, state);
+  return { arena_file: a.arena_file, assets: state.assets.map((x) => ({ id: x.id, last: x.last })), bots: state.bots.length, plain_reading: "A new paper arena: made-up assets and made-up money. Send orders with arena_orders, then call arena_step.", limits: ARENA_LIMITS_TEXT };
+}
+
+// Refused if the agent could not pay for its buys or deliver its sells even if all of them filled.
+export function checkOrders(state, agent, orders) {
+  const me = Object.hasOwn(state.agents, agent) ? state.agents[agent] : newAgent(state), known = new Set(state.assets.map((x) => x.id));
+  let spend = 0;
+  const selling = {};
+  for (const o of orders) {
+    if (!known.has(o.asset)) throw new RangeError(`no asset ${o.asset} in this arena`);
+    if (o.side === "buy") spend += o.quantity * o.limit; else selling[o.asset] = (selling[o.asset] ?? 0) + o.quantity;
+  }
+  if (spend > me.cash + 1e-9) throw new RangeError(`these buys could cost ${round2(spend)} and ${agent} has ${round2(me.cash)}: no borrowing here`);
+  for (const [k, q] of Object.entries(selling)) if (q > (me.holdings[k] ?? 0)) throw new RangeError(`${agent} holds ${me.holdings[k] ?? 0} ${k} and is selling ${q}: no short selling here`);
+}
+export function arenaOrders(args) {
+  const a = arenaOrdersInput.parse(args), state = load(a.arena_file);
+  if (state.bots.some((b) => b.name === a.agent)) throw new RangeError(`${a.agent} is one of the arena's scripted traders`);
+  checkOrders(state, a.agent, a.orders);
+  if (!Object.hasOwn(state.agents, a.agent)) state.agents[a.agent] = newAgent(state);
+  state.orders[a.agent] = a.orders;
+  save(a.arena_file, state);
+  return { agent: a.agent, orders_waiting: a.orders.length, step: state.step, plain_reading: `${a.orders.length} paper orders wait for step ${state.step + 1}.`, limits: ARENA_LIMITS_TEXT };
+}
+
+// The uniform-price call auction: the price where the most can trade; among those, where the
+// imbalance is smallest, then nearest the last price. Best limits fill first, earliest first at a tie.
+export function clearAuction(orders, last) {
+  const buys = orders.filter((o) => o.side === "buy").sort((a, b) => b.limit - a.limit || a.at - b.at);
+  const sells = orders.filter((o) => o.side === "sell").sort((a, b) => a.limit - b.limit || a.at - b.at);
+  const candidates = [...new Set(orders.map((o) => o.limit))];
+  let best = null;
+  for (const p of candidates) {
+    const demand = buys.filter((o) => o.limit >= p).reduce((n, o) => n + o.quantity, 0), supply = sells.filter((o) => o.limit <= p).reduce((n, o) => n + o.quantity, 0);
+    const volume = Math.min(demand, supply), imbalance = Math.abs(demand - supply), gap = Math.abs(p - last);
+    if (volume > 0 && (!best || volume > best.volume || (volume === best.volume && (imbalance < best.imbalance || (imbalance === best.imbalance && gap < best.gap))))) best = { price: p, volume, imbalance, gap };
+  }
+  if (!best) return { price: null, volume: 0, fills: [] };
+  const fills = [];
+  const take = (list, side) => { let left = best.volume; for (const o of list) { if (left <= 0) break; if (side === "buy" ? o.limit < best.price : o.limit > best.price) continue; const q = Math.min(o.quantity, left); fills.push({ agent: o.agent, side, quantity: q }); left -= q; } };
+  take(buys, "buy");
+  take(sells, "sell");
+  return { price: best.price, volume: best.volume, fills };
+}
+
+function botOrders(state, bot, asset, previous, left) {
+  const me = state.agents[bot.name], qty = 1 + Math.floor(rng(state) * 5), noise = 1 + 0.01 * gauss(state);
+  let side, limit;
+  if (bot.kind === "value") { side = asset.last < asset.value ? "buy" : "sell"; limit = (asset.last + asset.value) / 2; }
+  else if (bot.kind === "momentum") { side = previous !== null && asset.last >= previous ? "buy" : "sell"; limit = asset.last * (side === "buy" ? 1.01 : 0.99); }
+  else { side = rng(state) < 0.5 ? "buy" : "sell"; limit = asset.last * noise; }
+  limit = Math.max(0.01, round2(limit));
+  if (side === "buy") { if (left.cash < qty * limit) return null; left.cash -= qty * limit; }
+  if (side === "sell" && (me.holdings[asset.id] ?? 0) < qty) return null;
+  return { asset: asset.id, side, quantity: qty, limit };
+}
+
+export function stepOnce(state) {
+  state.step += 1;
+  // Buys are held against cash across assets in this step, so no agent can overspend across auctions.
+  const all = [];
+  let at = 0;
+  for (const [agent, orders] of orderLines(state)) for (const o of orders) all.push({ ...o, agent, at: at++ });
+  const previous = state.history.at(-1)?.prices_before ?? {};
+  // What a bot could pay is held across its auctions in a step, as an agent's buys are at arena_orders.
+  for (const bot of state.bots) { const left = { cash: state.agents[bot.name].cash }; for (const asset of state.assets) { const o = botOrders(state, bot, asset, previous[asset.id] ?? null, left); if (o) all.push({ ...o, agent: bot.name, at: at++ }); } }
+  const record = { step: state.step, prices_before: Object.fromEntries(state.assets.map((x) => [x.id, x.last])), auctions: {} };
+  for (const asset of state.assets) {
+    const r = clearAuction(all.filter((o) => o.asset === asset.id), asset.last);
+    record.auctions[asset.id] = { price: r.price, volume: r.volume };
+    if (r.price !== null) {
+      for (const f of r.fills) {
+        const me = state.agents[f.agent], cost = f.quantity * r.price;
+        if (f.side === "buy") { me.cash -= cost; me.holdings[asset.id] = (me.holdings[asset.id] ?? 0) + f.quantity; }
+        else { me.cash += cost; me.holdings[asset.id] -= f.quantity; }
+      }
+      asset.last = r.price;
+    }
+    // The made-up value drifts on its own each step: there is no real company behind it.
+    asset.value = Math.max(1, round2(asset.value * (1 + 0.01 * gauss(state))));
+  }
+  state.orders = {};
+  state.history.push(record);
+  if (state.history.length > 500) state.history.shift();
+}
+
+export function standings(state) {
+  const worth = (me) => me.cash + state.assets.reduce((n, x) => n + (me.holdings[x.id] ?? 0) * x.last, 0);
+  return Object.entries(state.agents).map(([agent, me]) => ({ agent, bot: state.bots.some((b) => b.name === agent), worth: round2(worth(me)), gain: round2(worth(me) - me.start_worth), cash: round2(me.cash), holdings: me.holdings })).sort((a, b) => b.gain - a.gain);
+}
+
+export function arenaStep(args) {
+  const a = arenaStepInput.parse(args), state = load(a.arena_file);
+  for (let i = 0; i < (a.steps ?? 1); i += 1) stepOnce(state);
+  save(a.arena_file, state);
+  const last = state.history.at(-1);
+  return { step: state.step, prices: Object.fromEntries(state.assets.map((x) => [x.id, x.last])), last_auctions: last.auctions, standings: standings(state).slice(0, 25), plain_reading: `Paper step ${state.step} cleared. No real asset or money is involved.`, limits: ARENA_LIMITS_TEXT };
+}

package/src/cpcv.mjs

@@ -0,0 +1,164 @@
+// cpcv: combinatorial purged cross-validation (Lopez de Prado 2018, Advances in Financial Machine
+// Learning, chapter 12) over any set of trial return series. History is cut into N blocks. Every
+// choice of k blocks is a test set; on what remains the best trial is chosen, with days beside the
+// test blocks purged and embargoed so nothing leaks across the edge, and that trial's test returns
+// are kept. The kept pieces stitch into C(N-1, k-1) whole paths, so a search gets a spread of
+// out-of-sample results rather than the one path history happened to take.
+//
+// A split is the unit of work: any machine with the same file can run a shard of them, and a shard
+// carries the hash of the file and settings, so a merge refuses other data, gaps and duplicates.
+import { createHash } from "node:crypto";
+import { readFileSync, writeFileSync } from "node:fs";
+import { z } from "zod";
+
+import { readPanel } from "./panel-io.mjs";
+
+const filePath = z.string().min(1).max(1000);
+export const cpcvInput = z.object({
+  returns_file: filePath.optional().describe("CSV: a date column and one column of daily returns per trial (breadth_book's out_file, say). At least two trials."),
+  groups: z.number().int().min(3).max(30).optional().describe("Blocks history is cut into. Default 10."),
+  test_groups: z.number().int().min(1).max(29).optional().describe("Blocks in each test set. Default 2."),
+  purge_days: z.number().int().min(0).max(500).optional().describe("Training days dropped before each test block: at least the horizon the trials' returns look ahead over. Default 5."),
+  embargo_days: z.number().int().min(0).max(500).optional().describe("Training days dropped after each test block. Default 5."),
+  periods_per_year: z.number().positive().optional().describe("Default 252."),
+  shard: z.object({ index: z.number().int().min(0), of: z.number().int().min(2).max(1000) }).strict().optional().describe("Run only splits s with s mod of = index, to spread the work over machines."),
+  out_file: filePath.optional().describe("Write the shard as JSON here."),
+  merge_files: z.array(filePath).min(2).max(1000).optional().describe("Instead of running: shard files to merge into one result."),
+}).strict();
+
+export const CPCV_LIMITS_TEXT = Object.freeze([
+  "The trial with the highest training Sharpe is the one chosen in each split; another way of choosing would give other paths.",
+  "Paths share days and data, so their spread is not a confidence interval and they are not independent tests.",
+  "This covers the search you supply. Variants tried and left out of the file are not counted, and survivorship in the data behind the trials passes straight through.",
+]);
+const MAX_SPLITS = 5000, MAX_CELLS = 100_000_000;
+
+export function choose(n, k) {
+  let c = 1;
+  for (let i = 1; i <= k; i += 1) c = (c * (n - k + i)) / i;
+  return Math.round(c);
+}
+export function splitsOf(n, k) {
+  const out = [], pick = [];
+  const walk = (from) => {
+    if (pick.length === k) { out.push([...pick]); return; }
+    for (let g = from; g < n; g += 1) { pick.push(g); walk(g + 1); pick.pop(); }
+  };
+  walk(0);
+  return out;
+}
+// Blocks of whole days, as equal as they can be: [first, last] inclusive.
+export function blocksOf(days, n) {
+  return Array.from({ length: n }, (_, g) => [Math.floor((g * days) / n), Math.floor(((g + 1) * days) / n) - 1]);
+}
+// 1 for a day training may use: outside the test blocks and their purge and embargo margins.
+export function trainMask(days, blocks, test, purge, embargo) {
+  const mask = new Uint8Array(days).fill(1);
+  for (const g of test) {
+    const [a, b] = blocks[g];
+    for (let t = Math.max(0, a - purge); t <= Math.min(days - 1, b + embargo); t += 1) mask[t] = 0;
+  }
+  return mask;
+}
+const sharpeOf = (values, scale) => {
+  let n = 0, mean = 0, s = 0;
+  for (const v of values) { if (!Number.isFinite(v)) continue; n += 1; const d = v - mean; mean += d / n; s += d * (v - mean); }
+  return n > 1 && s > 0 ? (mean / Math.sqrt(s / (n - 1))) * scale : null;
+};
+
+// One split: the best trial on the training days, and its returns on each test block.
+export function runSplit(panel, blocks, test, { purge, embargo, scale }) {
+  const days = panel.dates.length, mask = trainMask(days, blocks, test, purge, embargo);
+  let best = -1, bestSharpe = -Infinity;
+  panel.trials.forEach((name, i) => {
+    const r = panel.values[i];
+    let n = 0, mean = 0, s = 0;
+    for (let t = 0; t < days; t += 1) { if (!mask[t] || !Number.isFinite(r[t])) continue; n += 1; const d = r[t] - mean; mean += d / n; s += d * (r[t] - mean); }
+    const sh = n > 1 && s > 0 ? (mean / Math.sqrt(s / (n - 1))) * scale : -Infinity;
+    if (sh > bestSharpe) { bestSharpe = sh; best = i; }
+  });
+  if (best < 0 || !Number.isFinite(bestSharpe)) throw new RangeError(`no trial has enough training days when blocks ${test.join(",")} are the test set; use fewer groups, a shorter purge, or a longer file`);
+  // A chosen trial with no value on a day held nothing that day: it earns zero, and the day is counted.
+  let missing = 0;
+  const pieces = test.map((g) => {
+    const [a, b] = blocks[g], out = [];
+    for (let t = a; t <= b; t += 1) { const v = panel.values[best][t]; if (Number.isFinite(v)) out.push(v); else { out.push(0); missing += 1; } }
+    return [g, out];
+  });
+  return { chosen: best, train_sharpe: bestSharpe, missing, pieces };
+}
+
+// The kept pieces, whole: for path j, block g comes from the j-th split that tested g.
+export function assemble(results, groups) {
+  const seenByGroup = Array.from({ length: groups }, () => []);
+  for (const r of results.slice().sort((x, y) => x.s - y.s)) for (const [g, piece] of r.pieces) seenByGroup[g].push(piece);
+  const paths = seenByGroup[0].length;
+  for (const list of seenByGroup) if (list.length !== paths) throw new RangeError("the splits do not cover every block the same number of times");
+  return Array.from({ length: paths }, (_, j) => seenByGroup.flatMap((list) => list[j]));
+}
+
+function summarize(results, meta) {
+  const scale = Math.sqrt(meta.periods_per_year), sorted = (a) => a.slice().sort((x, y) => x - y), median = (a) => { const s = sorted(a), m = s.length >> 1; return s.length % 2 ? s[m] : (s[m - 1] + s[m]) / 2; };
+  const pathReturns = assemble(results, meta.groups);
+  const path_sharpes = pathReturns.map((p) => sharpeOf(p, scale)).filter((x) => x !== null);
+  const train = results.map((r) => r.train_sharpe);
+  const counts = new Map();
+  for (const r of results) counts.set(r.chosen, (counts.get(r.chosen) ?? 0) + 1);
+  const chosen = [...counts].sort((a, b) => b[1] - a[1]).slice(0, 10).map(([i, n]) => ({ trial: meta.trials[i], splits: n }));
+  const lost = path_sharpes.filter((x) => x <= 0).length;
+  const out = {
+    first_date: meta.first_date, last_date: meta.last_date, trials: meta.trials.length,
+    groups: meta.groups, test_groups: meta.test_groups, purge_days: meta.purge_days, embargo_days: meta.embargo_days,
+    splits: results.length, paths: path_sharpes.length, path_sharpes,
+    out_of_sample_sharpe: { median: median(path_sharpes), lowest: Math.min(...path_sharpes), highest: Math.max(...path_sharpes), share_of_paths_at_or_below_zero: lost / path_sharpes.length },
+    in_sample_sharpe_median: median(train),
+    most_chosen: chosen, distinct_trials_chosen: counts.size,
+    missing_test_days: results.reduce((n, r) => n + r.missing, 0),
+  };
+  out.plain_reading = `The best trial chosen on training days showed a median Sharpe of ${out.in_sample_sharpe_median.toFixed(2)} there. On the days held out it showed ${out.out_of_sample_sharpe.median.toFixed(2)}, from ${out.out_of_sample_sharpe.lowest.toFixed(2)} to ${out.out_of_sample_sharpe.highest.toFixed(2)} across ${out.paths} paths, and ${lost} of the paths lost money. ${counts.size} of ${meta.trials.length} trials were chosen at least once.`;
+  out.limits = CPCV_LIMITS_TEXT;
+  return out;
+}
+
+export function loadTrials(file) {
+  const p = readPanel(file, "returns_file");
+  if (p.tickers.length < 2) throw new RangeError("returns_file needs at least two trials: with one there is nothing to choose between");
+  if (p.dates.length * p.tickers.length > MAX_CELLS) throw new RangeError(`returns_file has more than ${MAX_CELLS} values; cut it down`);
+  return { dates: p.dates, trials: p.tickers, values: p.tickers.map((t) => Float64Array.from(p.values[t], (v) => (v === null ? NaN : v))) };
+}
+
+export function cpcv(args) {
+  const a = cpcvInput.parse(args);
+  if (a.merge_files) {
+    const shards = a.merge_files.map((f) => JSON.parse(readFileSync(f, "utf8")));
+    const id = shards[0].cpcv_id;
+    for (const [i, s] of shards.entries()) if (s.schema !== "canli.cpcv-shard.v1" || s.cpcv_id !== id) throw new RangeError(`${a.merge_files[i]} is not a shard of the same run (file or settings differ)`);
+    const results = shards.flatMap((s) => s.results), seen = new Set();
+    for (const r of results) { if (seen.has(r.s)) throw new RangeError(`split ${r.s} appears in two shards`); seen.add(r.s); }
+    const wanted = shards[0].meta.total_splits, missing = Array.from({ length: wanted }, (_, s) => s).filter((s) => !seen.has(s));
+    if (missing.length) throw new RangeError(`shards are missing splits ${missing.slice(0, 10).join(", ")}${missing.length > 10 ? "..." : ""}`);
+    return { cpcv_id: id, merged_from: shards.length, ...summarize(results, shards[0].meta) };
+  }
+  if (!a.returns_file) throw new RangeError("cpcv needs returns_file (or merge_files).");
+  const groups = a.groups ?? 10, k = a.test_groups ?? 2;
+  if (k >= groups) throw new RangeError("test_groups must be fewer than groups: something has to be left to train on");
+  const total = choose(groups, k);
+  if (total > MAX_SPLITS) throw new RangeError(`${groups} groups taken ${k} at a time is ${total} splits; the most is ${MAX_SPLITS}`);
+  const panel = loadTrials(a.returns_file);
+  if (panel.dates.length < groups * 5) throw new RangeError(`${panel.dates.length} days is too short for ${groups} groups`);
+  const meta = { groups, test_groups: k, purge_days: a.purge_days ?? 5, embargo_days: a.embargo_days ?? 5, periods_per_year: a.periods_per_year ?? 252, trials: panel.trials, first_date: panel.dates[0], last_date: panel.dates.at(-1), total_splits: total };
+  const blocks = blocksOf(panel.dates.length, groups), splits = splitsOf(groups, k), scale = Math.sqrt(meta.periods_per_year);
+  const results = [];
+  splits.forEach((test, s) => {
+    if (a.shard && s % a.shard.of !== a.shard.index) return;
+    results.push({ s, ...runSplit(panel, blocks, test, { purge: meta.purge_days, embargo: meta.embargo_days, scale }) });
+  });
+  if (a.shard) {
+    if (a.shard.index >= a.shard.of) throw new RangeError("shard index must be below shard of");
+    const cpcv_id = createHash("sha256").update(readFileSync(a.returns_file)).update(JSON.stringify(meta)).digest("hex").slice(0, 24);
+    const shard = { schema: "canli.cpcv-shard.v1", cpcv_id, shard: a.shard, meta, results };
+    if (a.out_file) writeFileSync(a.out_file, JSON.stringify(shard));
+    return { cpcv_id, shard: a.shard, ran: results.length, of: total, ...(a.out_file ? { shard_file: a.out_file } : { shard_data: shard }), next: "Run the other shards with the same file and settings, then call cpcv with merge_files." };
+  }
+  return summarize(results, meta);
+}

package/src/engine.mjs

@@ -15,9 +15,65 @@
 export const ENGINE_LIMITS_TEXT = Object.freeze([
   "Prices are the caller's: survivorship, splits, dividends and data errors in them pass straight into the result.",
   "A backtest measures one path of history under these costs; it does not establish that the signal will keep working.",
-  "Costs are a flat rate per unit traded; market impact, borrow fees for shorts and financing are not charged unless included in cost_bps.",
+  "Costs are a flat rate per unit traded; market impact is charged only when capital and dollar volume are given, and borrow fees for shorts and financing are not charged unless included in cost_bps.",
 ]);
 
+// With capital and dollar volume, size meets the market. Average daily dollar volume and daily
+// volatility are read from the days before each trade, never the trade day. An order fills only up
+// to max participation of that volume (the rest stays unfilled and the old position stays open), and
+// pays square-root impact, the same law check_feasibility uses: Y x daily volatility x sqrt(participation)
+// (Toth et al. 2011). A name with too little volume history cannot trade at all: nothing is assumed.
+const LIQUIDITY_DAYS = 20, MIN_LIQUIDITY_DAYS = 10;
+export const FILL_RATIO_TRADABLE = 0.95;
+
+function liquidityAt(prices, volumes, t) {
+  let dollars = 0, days = 0;
+  const rets = [];
+  for (let i = Math.max(1, t - LIQUIDITY_DAYS); i < t; i += 1) {
+    if (finite(volumes[i]) && volumes[i] > 0) { dollars += volumes[i]; days += 1; }
+    const a = prices[i - 1], b = prices[i];
+    if (finite(a) && finite(b) && a > 0) rets.push(b / a - 1);
+  }
+  if (days < MIN_LIQUIDITY_DAYS || rets.length < MIN_LIQUIDITY_DAYS) return null;
+  const mean = rets.reduce((x, y) => x + y, 0) / rets.length;
+  return { adv: dollars / days, vol: Math.sqrt(rets.reduce((x, y) => x + (y - mean) ** 2, 0) / (rets.length - 1)) };
+}
+
+// Whole stretches of market history a strategy can be held against. Dates are peak to trough or the
+// episode's usual bounds; the first eight are the ones stress_worlds drops into its worlds.
+export const CRISES = Object.freeze({
+  dotcom_bust: { from: "2000-03-24", to: "2002-10-09", what: "the 2000-02 dot-com bust" },
+  bank_crisis: { from: "2008-09-02", to: "2009-03-09", what: "the 2008-09 bank crisis" },
+  debt_ceiling_2011: { from: "2011-04-29", to: "2011-10-03", what: "the 2011 US downgrade and euro debt scare" },
+  taper_tantrum: { from: "2013-05-22", to: "2013-09-05", what: "the 2013 taper tantrum in bonds" },
+  oil_collapse: { from: "2014-06-20", to: "2016-02-11", what: "the 2014-16 oil collapse" },
+  dollar_surge: { from: "2014-07-01", to: "2015-03-13", what: "the 2014-15 surge in the dollar" },
+  q4_2018: { from: "2018-09-20", to: "2018-12-24", what: "the late 2018 sell-off" },
+  pandemic_crash: { from: "2020-02-19", to: "2020-03-23", what: "the February-March 2020 crash" },
+  inflation_shock: { from: "2022-01-03", to: "2022-06-16", what: "the 2022 inflation and rate shock" },
+  compute_bust: { from: "2022-01-03", to: "2022-10-14", what: "the 2022 fall in chip and compute stocks" },
+  regional_bank_run: { from: "2023-03-01", to: "2023-03-24", what: "the March 2023 regional bank run" },
+});
+export const CRISIS_DRAWDOWN_LIMIT = 0.5;
+
+// The strategy's own returns inside every crisis its dates fully cover. A crisis the run does not
+// fully cover is listed as untested: not having been there is not the same as having survived it.
+export function crisisReplay({ dates, returns, limit = CRISIS_DRAWDOWN_LIMIT }) {
+  if (!(limit > 0 && limit <= 1)) throw new RangeError("the crisis drawdown limit must be above 0 and at most 1");
+  const tested = [], untested = [];
+  for (const [id, c] of Object.entries(CRISES)) {
+    if (!(dates[0] <= c.from && dates.at(-1) >= c.to)) { untested.push({ id, what: c.what, from: c.from, to: c.to }); continue; }
+    let wealth = 1, peak = 1, drawdown = 0, days = 0, worst = 0;
+    for (let i = 0; i < dates.length; i += 1) {
+      if (dates[i] < c.from || dates[i] > c.to) continue;
+      wealth *= 1 + returns[i]; peak = Math.max(peak, wealth); drawdown = Math.max(drawdown, 1 - wealth / peak); worst = Math.min(worst, returns[i]); days += 1;
+    }
+    tested.push({ id, what: c.what, from: c.from, to: c.to, days, return: wealth - 1, drawdown, worst_day: worst, ruptured: drawdown > limit });
+  }
+  const verdict = !tested.length ? "untested" : tested.some((c) => c.ruptured) ? "ruptured" : "survived_tested";
+  return { verdict, drawdown_limit: limit, tested, untested };
+}
+
 const REBALANCE = Object.freeze({ weekly: "week", monthly: "month", quarterly: "quarter" });
 const periodKey = (date, every) => {
   const [y, m] = date.split("-").map(Number);
@@ -70,12 +126,20 @@ function lastRowOnOrBefore(signalDates, bound) {
   return found;
 }
 
-export function runBacktest({ dates, tickers, prices, signalDates, signals, rebalance = "monthly", lagDays = 1, side = "long_short", quantile = 0.2, costBps = 10, minNames = 5, periodsPerYear = 252 }) {
+export function runBacktest({ dates, tickers, prices, signalDates, signals, rebalance = "monthly", lagDays = 1, side = "long_short", quantile = 0.2, costBps = 10, minNames = 5, periodsPerYear = 252, capital = null, dollarVolumes = null, maxParticipation = 0.1, impactCoefficient = 1 }) {
   checkInputs({ dates, tickers, prices, signalDates, signals });
   if (!Number.isInteger(lagDays) || lagDays < 1) throw new RangeError("lag_days must be a whole number of at least 1: a signal cannot trade on the day it becomes known");
   if (!(quantile > 0 && quantile <= 0.5)) throw new RangeError("quantile must be in (0, 0.5]");
   if (!["long_short", "long_only"].includes(side)) throw new RangeError("side must be long_short or long_only");
   if (!(costBps >= 0)) throw new RangeError("cost_bps cannot be negative");
+  const sized = capital !== null;
+  if (sized) {
+    if (!(capital > 0)) throw new RangeError("capital must be above zero");
+    if (!dollarVolumes) throw new RangeError("capital needs dollar volume: without it nothing says how much the market can take");
+    if (!(maxParticipation > 0 && maxParticipation <= 1)) throw new RangeError("max participation must be above 0 and at most 1");
+    if (!(impactCoefficient >= 0)) throw new RangeError("the impact coefficient cannot be negative");
+  }
+  const liq = { impact: 0, orders: 0, clipped: 0, blocked: 0, wanted: 0, filled: 0, maxFilled: 0, worst: null };
 
   const n = dates.length;
   const rebal = new Set(rebalanceIndices(dates, rebalance, lagDays));
@@ -116,9 +180,26 @@ export function runBacktest({ dates, tickers, prices, signalDates, signals, reba
     const long = ranked.slice(0, k), short = side === "long_short" ? ranked.slice(-k) : [];
     for (const tk of long) target.set(tk, nav / long.length);
     for (const tk of short) target.set(tk, (target.get(tk) ?? 0) - nav / short.length);
-    let traded = 0;
-    for (const tk of new Set([...pos.keys(), ...target.keys()])) traded += Math.abs((target.get(tk) ?? 0) - (pos.get(tk) ?? 0));
-    const cost = traded * costBps / 1e4;
+    let traded = 0, impact = 0;
+    for (const tk of new Set([...pos.keys(), ...target.keys()])) {
+      const held = pos.get(tk) ?? 0, want = (target.get(tk) ?? 0) - held;
+      if (!sized) { traded += Math.abs(want); continue; }
+      if (want === 0) continue;
+      const l = liquidityAt(prices[tk], dollarVolumes[tk] ?? [], t);
+      const wanted = Math.abs(want);
+      liq.orders += 1; liq.wanted += wanted;
+      if (!l) { liq.blocked += 1; target.set(tk, held); continue; }
+      const room = maxParticipation * l.adv / capital;
+      const wantedShare = wanted * capital / l.adv;
+      if (!liq.worst || wantedShare > liq.worst.wanted_participation) liq.worst = { ticker: tk, date: dates[t], wanted_participation: wantedShare, average_daily_dollars: l.adv };
+      const fill = Math.min(wanted, room);
+      if (fill < wanted) { liq.clipped += 1; target.set(tk, held + Math.sign(want) * fill); }
+      const share = fill * capital / l.adv;
+      impact += fill * impactCoefficient * l.vol * Math.sqrt(share);
+      traded += fill; liq.filled += fill; liq.maxFilled = Math.max(liq.maxFilled, share);
+    }
+    liq.impact += impact;
+    const cost = traded * costBps / 1e4 + impact;
     nav -= cost;
     const factor = 1 - cost / (nav + cost);
     if (started && out.returns.length) {
@@ -133,7 +214,14 @@ export function runBacktest({ dates, tickers, prices, signalDates, signals, reba
     started = true;
   }
   if (!rebalances) throw new RangeError("no rebalance had enough names with both a known signal and a price; check the dates overlap and min_names");
-  return { ...out, used, nav, rebalances, turnoverSum, costSum, missingDays, longNames, shortNames, periodsPerYear };
+  const fillRatio = liq.wanted > 0 ? liq.filled / liq.wanted : 1;
+  const liquidity = sized ? {
+    capital_usd: capital, max_participation: maxParticipation, impact_coefficient: impactCoefficient,
+    impact_paid: liq.impact, orders: liq.orders, clipped_orders: liq.clipped, blocked_orders: liq.blocked,
+    fill_ratio: fillRatio, filled_participation_max: liq.maxFilled, worst: liq.worst,
+    verdict: fillRatio >= FILL_RATIO_TRADABLE ? "tradable" : "too_big",
+  } : undefined;
+  return { ...out, used, nav, rebalances, turnoverSum, costSum, missingDays, longNames, shortNames, periodsPerYear, ...(sized ? { liquidity } : {}) };
 }
 
 // Plain statistics of a return series, in the same Sharpe convention as the validators (sample sd).