canli-backtest-mcp 0.3.0 → 0.5.0

package/README.md

@@ -40,6 +40,23 @@ Three mistakes make most agent-run backtests look better than they are:
 | `effective_bets` | How many independent bets a set of return series holds (the participation ratio of their correlations), by series and by family |
 | `annotation_packet`, `annotation_agreement`, `annotation_gold` | Blind labelling packets of figures (or of a draft's figures) with guidelines and an answer key; agreement between labellers (Cohen's kappa); and a gold set with its datasheet |
 
+## Size, crises and cross-validation
+
+A backtest that assumes any size fills at the close flatters itself, so:
+
+- **Size.** Give `backtest_signal` a `capital_usd` and a `dollar_volume_file`. An order fills only up to `max_participation` of recent daily volume (10% unless you say otherwise) and pays square-root impact, with volume and volatility read from the days before each trade. The reply shows the Sharpe at $100k to $1B, and no Sharpe for a size the market could not take. Order-book depth is not modelled: it needs paid data.
+- **Crises.** Every reply holds the result against named crises, from the dot-com bust to the 2023 bank run. A crisis its dates do not fully cover is listed as untested, never as passed. `crisis_replay` does this for any returns file.
+- **Cross-validation.** `cpcv` takes every variant's returns and, for each choice of held-out blocks, chooses the best on the rest, with purge and embargo at the edges, then scores it on what was held out. You get a spread of out-of-sample results, not one path. Splits can run as shards on other machines and merge back.
+
+## A record, a league and a paper market
+
+- **Proof ledger.** `proof_filings` writes what companies filed into an append-only file, each line chained to the last and each batch signed, with a line for every figure a later filing changed. `proof_file` adds any file's hash, `proof_verify` checks the whole file and says which line first fails. It shows lines were not altered, not that a figure is right, and no record means only no record. Publish the head somewhere else, or the key holder could rewrite it all. The [published ledger](https://github.com/arhancanli/canlicapital/releases) does that with a public attestation each month.
+- **League.** `league_new`, `league_commit` and `league_score` run a tournament scored after the fact: an entrant signs and commits a hash before a round closes and shows the file afterwards, and anyone with the files can rerun the table. Each record is set against the best luck gives across that many entrants. The top entrants, chosen on earlier rounds only, form a paper book. No prizes, no stakes, nobody's money.
+- **Arena.** `arena_new`, `arena_orders` and `arena_step` are a paper market for agents to trade in, cleared in a call auction, reproducible from a seed. Made-up assets and made-up money: it cannot reach anything real.
+- **Agent exchange.** `exchange_new`, `exchange_join`, `exchange_view`, `exchange_orders`, `exchange_step` and `exchange_report` are a closed paper market where different AI agents and scripted traders meet in limit order books, with margin rules and forced selling, so a shock can become a cascade. `exchange_run` runs a whole market of scripted traders in one call. Time moves in steps, each agent sees only its own account, and a hash of the tape lets anyone check a run. Made-up assets and money. The rules are in [docs/AGENT_EXCHANGE.md](docs/AGENT_EXCHANGE.md).
+- **Consensus.** `agent_consensus` combines agents' probabilities and counts how many independent views they really hold, from how their errors move together: five models that share their mistakes are not five votes.
+- **Diversifiers.** `find_diversifiers` ranks candidates by the independent bets they add and how they behave on the book's worst days. Return is shown last and never used to rank.
+
 ## Quick start
 
 ```bash

package/docs/AGENT_EXCHANGE.md

@@ -0,0 +1,45 @@
+# The agent exchange protocol
+
+Version 1. How different AI agents trade against each other in a closed paper market, whatever model or host each one runs on. Everything is made-up assets and made-up money. Nothing here can reach a real security, token or currency, and running a real market like this would need registration as an exchange or a broker-dealer.
+
+## What it is for
+
+Seeing what agents do together: who supplies liquidity, who runs when it goes, and how forced selling feeds on itself. It shows how a set of rules can behave. It does not predict any real market.
+
+## The rules
+
+**Time moves in steps.** Nobody wins by being on a faster machine. Every agent acts on the same view of the market, and a step closes only when each joined agent has acted, an empty list counting as passing. Within a step, the order agents act in comes from the seed.
+
+**Each agent sees only its own account.** A view holds the book, recent trades without names, and the agent's own cash, positions and orders. It holds no other agent's positions and no asset's true value.
+
+**An agent acts only as itself.** Joining gives a token, and every later call needs it.
+
+**The book goes by price, then time.** A trade happens at the resting order's price. An order never trades with its own agent: the resting one is cancelled.
+
+**Margin.** Positions may reach `leverage` times equity when opened. An account whose equity falls below `maintenance_margin` of its gross positions is sold down at market, half at a time, into whatever is left in the book. Each sale moves the price, which can put the next account under. That chain is the cascade, and the report counts its rounds.
+
+**Anyone can check a run.** Every trade extends a hash chain, so the same seed, personas, shocks and orders give the same `tape_hash` on any machine.
+
+## The messages
+
+Each is a tool. All take `exchange_file`, the shared state file, and any number of servers on any number of hosts may use one file: writers take turns.
+
+| Tool | Who | What |
+|---|---|---|
+| `exchange_new` | Whoever sets it up | Seed, assets, personas, leverage, margin, and shocks at set steps |
+| `exchange_join` | Each agent | Name and what it is. Returns its token |
+| `exchange_view` | Each agent | The market as this agent may see it |
+| `exchange_orders` | Each agent | `limit`, `market` and `cancel` orders for the coming step |
+| `exchange_step` | Whoever runs the clock | Closes the step once every joined agent has acted |
+| `exchange_report` | Anyone | Each asset's worst fall and recovery, spreads and depth, who gained and lost, forced selling, and the tape hash |
+| `exchange_run` | Anyone | A whole run of scripted personas in one call, for comparing set-ups |
+
+## Personas
+
+Scripted traders fill out the market, each seeing what any agent sees. They are simple on purpose, and their numbers are in `src/exchange.mjs` because they are the assumptions: `market_maker` (quotes both sides and steps back when prices move fast), `value` (trades toward a noisy view of true worth), `momentum` (follows recent moves), `noise` (small random orders), `deleverager` (keeps its risk level, so it sells as volatility rises), `stop_loss` (sells after a fall), `levered_long` (borrows to the limit and holds until the margin rule sells it out).
+
+Different parameters give different cascades. A result here is a question to ask of a real system, not an answer about it.
+
+## Running different models together
+
+`recipes/exchange_agents.mjs` runs each agent through its own server against any OpenAI-compatible endpoint, so one can be Claude and another GPT or a local model.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "canli-backtest-mcp",
-  "version": "0.3.0",
+  "version": "0.5.0",
   "description": "MCP server for backtests that refuse lookahead: point-in-time factors from SEC filings, cross-sectional backtests of any signal on your own prices, costs on actual trades, and every variant counted in a trial ledger.",
   "private": false,
   "type": "module",
@@ -13,6 +13,7 @@
   },
   "files": [
     "src",
+    "docs",
     "README.md"
   ],
   "scripts": {

package/src/arena.mjs

@@ -0,0 +1,150 @@
+// The arena: a paper market for AI agents to trade in. Made-up assets and made-up money, cleared in a
+// uniform-price call auction each step, reproducible from a seed. It is here so agents' behaviour
+// together can be studied: who supplies liquidity, who gets picked off, what a crowd does to a price.
+//
+// It touches nothing real. No real security, token or currency is represented, no order leaves the
+// file, and nothing here can be turned into a real market: running one needs registration as an
+// exchange or as a broker-dealer running an alternative trading system.
+import { existsSync, readFileSync, writeFileSync } from "node:fs";
+import { z } from "zod";
+
+export const ARENA_LIMITS_TEXT = Object.freeze([
+  "Paper only: made-up assets and made-up money. Nothing here represents or can reach a real security, token or currency.",
+  "A simulated market leaves out most of what moves real ones, so behaviour here is a sketch for study and does not predict results in real markets.",
+]);
+
+const file = z.string().min(1).max(1000);
+const assetId = z.string().regex(/^ART[0-9]{1,2}$/, "an arena asset, such as ART1");
+// Names become keys in the state file, so ones every JavaScript object already answers to are refused.
+const agentId = z.string().regex(/^[A-Za-z0-9][A-Za-z0-9_-]{1,39}$/).refine((n) => !(n in {}), "that name is reserved");
+export const arenaNewInput = z.object({
+  arena_file: file.describe("The arena's state file; must not exist yet."),
+  seed: z.number().int().min(0).max(2 ** 31 - 2).optional().describe("The same seed and the same orders give the same market. Default 1."),
+  assets: z.number().int().min(1).max(20).optional().describe("Made-up assets. Default 3."),
+  starting_cash: z.number().positive().max(1e9).optional().describe("Made-up money each agent starts with. Default 100000."),
+  bots: z.number().int().min(0).max(50).optional().describe("Scripted traders that keep the market moving: noise, value and momentum in turn. Default 6."),
+}).strict();
+export const arenaOrdersInput = z.object({
+  arena_file: file,
+  agent: agentId,
+  orders: z.array(z.object({ asset: assetId, side: z.enum(["buy", "sell"]), quantity: z.number().int().positive().max(1e7), limit: z.number().positive().max(1e7) }).strict()).max(50).describe("Replaces this agent's orders for the coming step. No short selling and no borrowing."),
+}).strict();
+export const arenaStepInput = z.object({ arena_file: file, steps: z.number().int().min(1).max(1000).optional().describe("Default 1.") }).strict();
+
+const SHARES_PER_AGENT = 100;
+const load = (f) => { if (!existsSync(f)) throw new RangeError(`${f} is not an arena yet: call arena_new`); let s = null; try { s = JSON.parse(readFileSync(f, "utf8")); } catch { /* a parse error quotes the file, so it stays here */ } if (s?.schema !== "canli.arena.v1") throw new RangeError(`${f} is not an arena file`); return s; };
+// Every order in a step comes from an agent in the file, however the file was written.
+const orderLines = (state) => Object.entries(state.orders).filter(([agent]) => Object.hasOwn(state.agents, agent));
+const save = (f, s) => writeFileSync(f, JSON.stringify(s));
+function rng(state) { state.rng = (state.rng * 16807) % 2147483647; return state.rng / 2147483647; }
+const gauss = (s) => Math.sqrt(-2 * Math.log(rng(s))) * Math.cos(2 * Math.PI * rng(s));
+const newAgent = (s) => ({ cash: s.starting_cash, holdings: Object.fromEntries(s.assets.map((x) => [x.id, SHARES_PER_AGENT])), start_worth: s.starting_cash + s.assets.reduce((n, x) => n + SHARES_PER_AGENT * x.last, 0) });
+const round2 = (x) => Math.round(x * 100) / 100;
+
+export function arenaNew(args) {
+  const a = arenaNewInput.parse(args);
+  if (existsSync(a.arena_file)) throw new RangeError(`${a.arena_file} already exists`);
+  const state = { schema: "canli.arena.v1", paper_only: true, step: 0, rng: (a.seed ?? 1) + 1, starting_cash: a.starting_cash ?? 100000, assets: [], agents: {}, orders: {}, history: [] };
+  for (let i = 0; i < (a.assets ?? 3); i += 1) { const value = round2(50 + 100 * rng(state)); state.assets.push({ id: `ART${i + 1}`, value, last: value }); }
+  const kinds = ["noise", "value", "momentum"];
+  state.bots = Array.from({ length: a.bots ?? 6 }, (_, i) => ({ name: `bot-${kinds[i % 3]}-${i}`, kind: kinds[i % 3] }));
+  for (const b of state.bots) state.agents[b.name] = newAgent(state);
+  save(a.arena_file, state);
+  return { arena_file: a.arena_file, assets: state.assets.map((x) => ({ id: x.id, last: x.last })), bots: state.bots.length, plain_reading: "A new paper arena: made-up assets and made-up money. Send orders with arena_orders, then call arena_step.", limits: ARENA_LIMITS_TEXT };
+}
+
+// Refused if the agent could not pay for its buys or deliver its sells even if all of them filled.
+export function checkOrders(state, agent, orders) {
+  const me = Object.hasOwn(state.agents, agent) ? state.agents[agent] : newAgent(state), known = new Set(state.assets.map((x) => x.id));
+  let spend = 0;
+  const selling = {};
+  for (const o of orders) {
+    if (!known.has(o.asset)) throw new RangeError(`no asset ${o.asset} in this arena`);
+    if (o.side === "buy") spend += o.quantity * o.limit; else selling[o.asset] = (selling[o.asset] ?? 0) + o.quantity;
+  }
+  if (spend > me.cash + 1e-9) throw new RangeError(`these buys could cost ${round2(spend)} and ${agent} has ${round2(me.cash)}: no borrowing here`);
+  for (const [k, q] of Object.entries(selling)) if (q > (me.holdings[k] ?? 0)) throw new RangeError(`${agent} holds ${me.holdings[k] ?? 0} ${k} and is selling ${q}: no short selling here`);
+}
+export function arenaOrders(args) {
+  const a = arenaOrdersInput.parse(args), state = load(a.arena_file);
+  if (state.bots.some((b) => b.name === a.agent)) throw new RangeError(`${a.agent} is one of the arena's scripted traders`);
+  checkOrders(state, a.agent, a.orders);
+  if (!Object.hasOwn(state.agents, a.agent)) state.agents[a.agent] = newAgent(state);
+  state.orders[a.agent] = a.orders;
+  save(a.arena_file, state);
+  return { agent: a.agent, orders_waiting: a.orders.length, step: state.step, plain_reading: `${a.orders.length} paper orders wait for step ${state.step + 1}.`, limits: ARENA_LIMITS_TEXT };
+}
+
+// The uniform-price call auction: the price where the most can trade; among those, where the
+// imbalance is smallest, then nearest the last price. Best limits fill first, earliest first at a tie.
+export function clearAuction(orders, last) {
+  const buys = orders.filter((o) => o.side === "buy").sort((a, b) => b.limit - a.limit || a.at - b.at);
+  const sells = orders.filter((o) => o.side === "sell").sort((a, b) => a.limit - b.limit || a.at - b.at);
+  const candidates = [...new Set(orders.map((o) => o.limit))];
+  let best = null;
+  for (const p of candidates) {
+    const demand = buys.filter((o) => o.limit >= p).reduce((n, o) => n + o.quantity, 0), supply = sells.filter((o) => o.limit <= p).reduce((n, o) => n + o.quantity, 0);
+    const volume = Math.min(demand, supply), imbalance = Math.abs(demand - supply), gap = Math.abs(p - last);
+    if (volume > 0 && (!best || volume > best.volume || (volume === best.volume && (imbalance < best.imbalance || (imbalance === best.imbalance && gap < best.gap))))) best = { price: p, volume, imbalance, gap };
+  }
+  if (!best) return { price: null, volume: 0, fills: [] };
+  const fills = [];
+  const take = (list, side) => { let left = best.volume; for (const o of list) { if (left <= 0) break; if (side === "buy" ? o.limit < best.price : o.limit > best.price) continue; const q = Math.min(o.quantity, left); fills.push({ agent: o.agent, side, quantity: q }); left -= q; } };
+  take(buys, "buy");
+  take(sells, "sell");
+  return { price: best.price, volume: best.volume, fills };
+}
+
+function botOrders(state, bot, asset, previous, left) {
+  const me = state.agents[bot.name], qty = 1 + Math.floor(rng(state) * 5), noise = 1 + 0.01 * gauss(state);
+  let side, limit;
+  if (bot.kind === "value") { side = asset.last < asset.value ? "buy" : "sell"; limit = (asset.last + asset.value) / 2; }
+  else if (bot.kind === "momentum") { side = previous !== null && asset.last >= previous ? "buy" : "sell"; limit = asset.last * (side === "buy" ? 1.01 : 0.99); }
+  else { side = rng(state) < 0.5 ? "buy" : "sell"; limit = asset.last * noise; }
+  limit = Math.max(0.01, round2(limit));
+  if (side === "buy") { if (left.cash < qty * limit) return null; left.cash -= qty * limit; }
+  if (side === "sell" && (me.holdings[asset.id] ?? 0) < qty) return null;
+  return { asset: asset.id, side, quantity: qty, limit };
+}
+
+export function stepOnce(state) {
+  state.step += 1;
+  // Buys are held against cash across assets in this step, so no agent can overspend across auctions.
+  const all = [];
+  let at = 0;
+  for (const [agent, orders] of orderLines(state)) for (const o of orders) all.push({ ...o, agent, at: at++ });
+  const previous = state.history.at(-1)?.prices_before ?? {};
+  // What a bot could pay is held across its auctions in a step, as an agent's buys are at arena_orders.
+  for (const bot of state.bots) { const left = { cash: state.agents[bot.name].cash }; for (const asset of state.assets) { const o = botOrders(state, bot, asset, previous[asset.id] ?? null, left); if (o) all.push({ ...o, agent: bot.name, at: at++ }); } }
+  const record = { step: state.step, prices_before: Object.fromEntries(state.assets.map((x) => [x.id, x.last])), auctions: {} };
+  for (const asset of state.assets) {
+    const r = clearAuction(all.filter((o) => o.asset === asset.id), asset.last);
+    record.auctions[asset.id] = { price: r.price, volume: r.volume };
+    if (r.price !== null) {
+      for (const f of r.fills) {
+        const me = state.agents[f.agent], cost = f.quantity * r.price;
+        if (f.side === "buy") { me.cash -= cost; me.holdings[asset.id] = (me.holdings[asset.id] ?? 0) + f.quantity; }
+        else { me.cash += cost; me.holdings[asset.id] -= f.quantity; }
+      }
+      asset.last = r.price;
+    }
+    // The made-up value drifts on its own each step: there is no real company behind it.
+    asset.value = Math.max(1, round2(asset.value * (1 + 0.01 * gauss(state))));
+  }
+  state.orders = {};
+  state.history.push(record);
+  if (state.history.length > 500) state.history.shift();
+}
+
+export function standings(state) {
+  const worth = (me) => me.cash + state.assets.reduce((n, x) => n + (me.holdings[x.id] ?? 0) * x.last, 0);
+  return Object.entries(state.agents).map(([agent, me]) => ({ agent, bot: state.bots.some((b) => b.name === agent), worth: round2(worth(me)), gain: round2(worth(me) - me.start_worth), cash: round2(me.cash), holdings: me.holdings })).sort((a, b) => b.gain - a.gain);
+}
+
+export function arenaStep(args) {
+  const a = arenaStepInput.parse(args), state = load(a.arena_file);
+  for (let i = 0; i < (a.steps ?? 1); i += 1) stepOnce(state);
+  save(a.arena_file, state);
+  const last = state.history.at(-1);
+  return { step: state.step, prices: Object.fromEntries(state.assets.map((x) => [x.id, x.last])), last_auctions: last.auctions, standings: standings(state).slice(0, 25), plain_reading: `Paper step ${state.step} cleared. No real asset or money is involved.`, limits: ARENA_LIMITS_TEXT };
+}

package/src/consensus.mjs

@@ -0,0 +1,89 @@
+// Consensus among agents, counted honestly. Five models that agree are not five votes if they share
+// their training and their mistakes. So this measures how many independent views a group really holds,
+// from how their errors move together, and weights each by its record. Agreement among copies adds
+// nothing, and the answer says so.
+import { z } from "zod";
+
+const prob = z.number().min(0).max(1);
+export const consensusInput = z.object({
+  agents: z.array(z.object({
+    name: z.string().min(1).max(60),
+    forecasts: z.array(z.object({ question: z.string().min(1).max(200), probability: prob }).strict()).min(1).max(5000),
+  }).strict()).min(2).max(200).describe("Each agent's probability for each question."),
+  outcomes: z.array(z.object({ question: z.string().min(1).max(200), happened: z.boolean() }).strict()).max(5000).optional().describe("Questions already settled. Their forecasts give each agent a record; the rest are combined."),
+}).strict();
+
+const mean = (xs) => xs.reduce((a, b) => a + b, 0) / xs.length;
+const round = (x, d = 3) => (x == null || !Number.isFinite(x) ? null : Number(x.toFixed(d)));
+const clip = (p) => Math.min(0.99, Math.max(0.01, p));
+const logit = (p) => Math.log(clip(p) / (1 - clip(p)));
+const unlogit = (x) => 1 / (1 + Math.exp(-x));
+
+export function correlation(a, b) {
+  const ma = mean(a), mb = mean(b);
+  let up = 0, da = 0, db = 0;
+  for (let i = 0; i < a.length; i += 1) { up += (a[i] - ma) * (b[i] - mb); da += (a[i] - ma) ** 2; db += (b[i] - mb) ** 2; }
+  return da > 1e-12 && db > 1e-12 ? up / Math.sqrt(da * db) : null;
+}
+
+// k for unrelated views, 1 for copies: (sum of eigenvalues)^2 / sum of squared eigenvalues of the
+// correlation matrix. A pair with no measurable spread counts as the same view.
+export function independentViews(series) {
+  const k = series.length;
+  let squares = 0;
+  for (let i = 0; i < k; i += 1) for (let j = 0; j < k; j += 1) { const c = i === j ? 1 : correlation(series[i], series[j]) ?? 1; squares += c * c; }
+  return k * k / squares;
+}
+
+export function agentConsensus(args) {
+  const a = consensusInput.parse(args);
+  const happened = new Map((a.outcomes ?? []).map((o) => [o.question, o.happened ? 1 : 0]));
+  const own = a.agents.map((x) => new Map(x.forecasts.map((f) => [f.question, f.probability])));
+  const allQuestions = [...new Set(a.agents.flatMap((x) => x.forecasts.map((f) => f.question)))];
+  const settled = allQuestions.filter((q) => happened.has(q) && own.every((m) => m.has(q)));
+  const open = allQuestions.filter((q) => !happened.has(q));
+  const base = settled.length ? mean(settled.map((q) => happened.get(q))) : null;
+  const baseBrier = base == null ? null : mean(settled.map((q) => (base - happened.get(q)) ** 2));
+
+  const records = a.agents.map((x, i) => {
+    const errors = settled.map((q) => own[i].get(q) - happened.get(q));
+    const brier = errors.length ? mean(errors.map((e) => e * e)) : null;
+    return { name: x.name, errors, brier };
+  });
+  // Weighted by record once there is one; a Brier score near chance earns little, and with fewer than
+  // ten settled questions everyone counts the same.
+  const useRecord = settled.length >= 10;
+  const raw = records.map((r) => (useRecord ? 1 / Math.max(r.brier, 0.01) : 1)), total = raw.reduce((x, y) => x + y, 0);
+  const weights = raw.map((w) => w / total);
+
+  const basis = settled.length >= 5 ? "their errors on settled questions, after each agent's usual error on that outcome" : "their forecasts, since few questions are settled";
+  const shared = allQuestions.filter((q) => own.every((m) => m.has(q)));
+  // Good forecasters' errors all lean the same way, because they are all right about the same outcomes.
+  // What shows shared thinking is what is left once each agent's usual error on that outcome is removed.
+  const residual = (errors) => {
+    const usual = (hit) => { const xs = errors.filter((_, i) => happened.get(settled[i]) === hit); return xs.length ? mean(xs) : 0; };
+    const yes = usual(1), no = usual(0);
+    return errors.map((e, i) => e - (happened.get(settled[i]) === 1 ? yes : no));
+  };
+  const series = settled.length >= 5 ? records.map((r) => residual(r.errors)) : own.map((m) => shared.map((q) => m.get(q)));
+  const views = series[0]?.length >= 3 ? independentViews(series) : null;
+
+  const rows = open.map((q) => {
+    const have = own.map((m, i) => [m.get(q), weights[i]]).filter(([p]) => p !== undefined);
+    const sum = have.reduce((n, [, w]) => n + w, 0);
+    const combined = unlogit(have.reduce((n, [p, w]) => n + logit(p) * w, 0) / sum);
+    const ps = have.map(([p]) => p);
+    return [q, have.length, round(combined), round(mean(ps)), round(Math.min(...ps)), round(Math.max(...ps))];
+  });
+  const k = a.agents.length;
+  return {
+    agents: k,
+    independent_views: round(views, 1),
+    independent_views_from: views == null ? "too few shared questions to measure" : basis,
+    reading: views == null ? "Too few shared questions to say how independent these agents are." : `These ${k} agents hold about ${round(views, 1)} independent ${views < 1.5 ? "view" : "views"} between them.${views < k / 2 ? " Most of their agreement is shared thinking, so it should not raise confidence the way separate opinions would." : ""}`,
+    records: { columns: ["agent", "settled_questions", "brier_score", "skill_against_base_rate", "weight"], rows: records.map((r, i) => [r.name, settled.length, round(r.brier), r.brier != null && baseBrier ? round(1 - r.brier / baseBrier) : null, round(weights[i])]) },
+    combined: { columns: ["question", "agents", "combined_probability", "plain_average", "lowest", "highest"], rows },
+    meaning: "A Brier score is the mean squared error of a probability (lower is better; always saying 50% scores 0.25). Skill above 0 beats always saying the base rate. The combined probability is a weighted average of log-odds.",
+    limits: ["With few settled questions the records and the count of views are noisy.", "A group's record says how it did on these questions. It does not say it will keep doing so."],
+  };
+}

package/src/cpcv.mjs

@@ -0,0 +1,164 @@
+// cpcv: combinatorial purged cross-validation (Lopez de Prado 2018, Advances in Financial Machine
+// Learning, chapter 12) over any set of trial return series. History is cut into N blocks. Every
+// choice of k blocks is a test set; on what remains the best trial is chosen, with days beside the
+// test blocks purged and embargoed so nothing leaks across the edge, and that trial's test returns
+// are kept. The kept pieces stitch into C(N-1, k-1) whole paths, so a search gets a spread of
+// out-of-sample results rather than the one path history happened to take.
+//
+// A split is the unit of work: any machine with the same file can run a shard of them, and a shard
+// carries the hash of the file and settings, so a merge refuses other data, gaps and duplicates.
+import { createHash } from "node:crypto";
+import { readFileSync, writeFileSync } from "node:fs";
+import { z } from "zod";
+
+import { readPanel } from "./panel-io.mjs";
+
+const filePath = z.string().min(1).max(1000);
+export const cpcvInput = z.object({
+  returns_file: filePath.optional().describe("CSV: a date column and one column of daily returns per trial (breadth_book's out_file, say). At least two trials."),
+  groups: z.number().int().min(3).max(30).optional().describe("Blocks history is cut into. Default 10."),
+  test_groups: z.number().int().min(1).max(29).optional().describe("Blocks in each test set. Default 2."),
+  purge_days: z.number().int().min(0).max(500).optional().describe("Training days dropped before each test block: at least the horizon the trials' returns look ahead over. Default 5."),
+  embargo_days: z.number().int().min(0).max(500).optional().describe("Training days dropped after each test block. Default 5."),
+  periods_per_year: z.number().positive().optional().describe("Default 252."),
+  shard: z.object({ index: z.number().int().min(0), of: z.number().int().min(2).max(1000) }).strict().optional().describe("Run only splits s with s mod of = index, to spread the work over machines."),
+  out_file: filePath.optional().describe("Write the shard as JSON here."),
+  merge_files: z.array(filePath).min(2).max(1000).optional().describe("Instead of running: shard files to merge into one result."),
+}).strict();
+
+export const CPCV_LIMITS_TEXT = Object.freeze([
+  "The trial with the highest training Sharpe is the one chosen in each split; another way of choosing would give other paths.",
+  "Paths share days and data, so their spread is not a confidence interval and they are not independent tests.",
+  "This covers the search you supply. Variants tried and left out of the file are not counted, and survivorship in the data behind the trials passes straight through.",
+]);
+const MAX_SPLITS = 5000, MAX_CELLS = 100_000_000;
+
+export function choose(n, k) {
+  let c = 1;
+  for (let i = 1; i <= k; i += 1) c = (c * (n - k + i)) / i;
+  return Math.round(c);
+}
+export function splitsOf(n, k) {
+  const out = [], pick = [];
+  const walk = (from) => {
+    if (pick.length === k) { out.push([...pick]); return; }
+    for (let g = from; g < n; g += 1) { pick.push(g); walk(g + 1); pick.pop(); }
+  };
+  walk(0);
+  return out;
+}
+// Blocks of whole days, as equal as they can be: [first, last] inclusive.
+export function blocksOf(days, n) {
+  return Array.from({ length: n }, (_, g) => [Math.floor((g * days) / n), Math.floor(((g + 1) * days) / n) - 1]);
+}
+// 1 for a day training may use: outside the test blocks and their purge and embargo margins.
+export function trainMask(days, blocks, test, purge, embargo) {
+  const mask = new Uint8Array(days).fill(1);
+  for (const g of test) {
+    const [a, b] = blocks[g];
+    for (let t = Math.max(0, a - purge); t <= Math.min(days - 1, b + embargo); t += 1) mask[t] = 0;
+  }
+  return mask;
+}
+const sharpeOf = (values, scale) => {
+  let n = 0, mean = 0, s = 0;
+  for (const v of values) { if (!Number.isFinite(v)) continue; n += 1; const d = v - mean; mean += d / n; s += d * (v - mean); }
+  return n > 1 && s > 0 ? (mean / Math.sqrt(s / (n - 1))) * scale : null;
+};
+
+// One split: the best trial on the training days, and its returns on each test block.
+export function runSplit(panel, blocks, test, { purge, embargo, scale }) {
+  const days = panel.dates.length, mask = trainMask(days, blocks, test, purge, embargo);
+  let best = -1, bestSharpe = -Infinity;
+  panel.trials.forEach((name, i) => {
+    const r = panel.values[i];
+    let n = 0, mean = 0, s = 0;
+    for (let t = 0; t < days; t += 1) { if (!mask[t] || !Number.isFinite(r[t])) continue; n += 1; const d = r[t] - mean; mean += d / n; s += d * (r[t] - mean); }
+    const sh = n > 1 && s > 0 ? (mean / Math.sqrt(s / (n - 1))) * scale : -Infinity;
+    if (sh > bestSharpe) { bestSharpe = sh; best = i; }
+  });
+  if (best < 0 || !Number.isFinite(bestSharpe)) throw new RangeError(`no trial has enough training days when blocks ${test.join(",")} are the test set; use fewer groups, a shorter purge, or a longer file`);
+  // A chosen trial with no value on a day held nothing that day: it earns zero, and the day is counted.
+  let missing = 0;
+  const pieces = test.map((g) => {
+    const [a, b] = blocks[g], out = [];
+    for (let t = a; t <= b; t += 1) { const v = panel.values[best][t]; if (Number.isFinite(v)) out.push(v); else { out.push(0); missing += 1; } }
+    return [g, out];
+  });
+  return { chosen: best, train_sharpe: bestSharpe, missing, pieces };
+}
+
+// The kept pieces, whole: for path j, block g comes from the j-th split that tested g.
+export function assemble(results, groups) {
+  const seenByGroup = Array.from({ length: groups }, () => []);
+  for (const r of results.slice().sort((x, y) => x.s - y.s)) for (const [g, piece] of r.pieces) seenByGroup[g].push(piece);
+  const paths = seenByGroup[0].length;
+  for (const list of seenByGroup) if (list.length !== paths) throw new RangeError("the splits do not cover every block the same number of times");
+  return Array.from({ length: paths }, (_, j) => seenByGroup.flatMap((list) => list[j]));
+}
+
+function summarize(results, meta) {
+  const scale = Math.sqrt(meta.periods_per_year), sorted = (a) => a.slice().sort((x, y) => x - y), median = (a) => { const s = sorted(a), m = s.length >> 1; return s.length % 2 ? s[m] : (s[m - 1] + s[m]) / 2; };
+  const pathReturns = assemble(results, meta.groups);
+  const path_sharpes = pathReturns.map((p) => sharpeOf(p, scale)).filter((x) => x !== null);
+  const train = results.map((r) => r.train_sharpe);
+  const counts = new Map();
+  for (const r of results) counts.set(r.chosen, (counts.get(r.chosen) ?? 0) + 1);
+  const chosen = [...counts].sort((a, b) => b[1] - a[1]).slice(0, 10).map(([i, n]) => ({ trial: meta.trials[i], splits: n }));
+  const lost = path_sharpes.filter((x) => x <= 0).length;
+  const out = {
+    first_date: meta.first_date, last_date: meta.last_date, trials: meta.trials.length,
+    groups: meta.groups, test_groups: meta.test_groups, purge_days: meta.purge_days, embargo_days: meta.embargo_days,
+    splits: results.length, paths: path_sharpes.length, path_sharpes,
+    out_of_sample_sharpe: { median: median(path_sharpes), lowest: Math.min(...path_sharpes), highest: Math.max(...path_sharpes), share_of_paths_at_or_below_zero: lost / path_sharpes.length },
+    in_sample_sharpe_median: median(train),
+    most_chosen: chosen, distinct_trials_chosen: counts.size,
+    missing_test_days: results.reduce((n, r) => n + r.missing, 0),
+  };
+  out.plain_reading = `The best trial chosen on training days showed a median Sharpe of ${out.in_sample_sharpe_median.toFixed(2)} there. On the days held out it showed ${out.out_of_sample_sharpe.median.toFixed(2)}, from ${out.out_of_sample_sharpe.lowest.toFixed(2)} to ${out.out_of_sample_sharpe.highest.toFixed(2)} across ${out.paths} paths, and ${lost} of the paths lost money. ${counts.size} of ${meta.trials.length} trials were chosen at least once.`;
+  out.limits = CPCV_LIMITS_TEXT;
+  return out;
+}
+
+export function loadTrials(file) {
+  const p = readPanel(file, "returns_file");
+  if (p.tickers.length < 2) throw new RangeError("returns_file needs at least two trials: with one there is nothing to choose between");
+  if (p.dates.length * p.tickers.length > MAX_CELLS) throw new RangeError(`returns_file has more than ${MAX_CELLS} values; cut it down`);
+  return { dates: p.dates, trials: p.tickers, values: p.tickers.map((t) => Float64Array.from(p.values[t], (v) => (v === null ? NaN : v))) };
+}
+
+export function cpcv(args) {
+  const a = cpcvInput.parse(args);
+  if (a.merge_files) {
+    const shards = a.merge_files.map((f) => JSON.parse(readFileSync(f, "utf8")));
+    const id = shards[0].cpcv_id;
+    for (const [i, s] of shards.entries()) if (s.schema !== "canli.cpcv-shard.v1" || s.cpcv_id !== id) throw new RangeError(`${a.merge_files[i]} is not a shard of the same run (file or settings differ)`);
+    const results = shards.flatMap((s) => s.results), seen = new Set();
+    for (const r of results) { if (seen.has(r.s)) throw new RangeError(`split ${r.s} appears in two shards`); seen.add(r.s); }
+    const wanted = shards[0].meta.total_splits, missing = Array.from({ length: wanted }, (_, s) => s).filter((s) => !seen.has(s));
+    if (missing.length) throw new RangeError(`shards are missing splits ${missing.slice(0, 10).join(", ")}${missing.length > 10 ? "..." : ""}`);
+    return { cpcv_id: id, merged_from: shards.length, ...summarize(results, shards[0].meta) };
+  }
+  if (!a.returns_file) throw new RangeError("cpcv needs returns_file (or merge_files).");
+  const groups = a.groups ?? 10, k = a.test_groups ?? 2;
+  if (k >= groups) throw new RangeError("test_groups must be fewer than groups: something has to be left to train on");
+  const total = choose(groups, k);
+  if (total > MAX_SPLITS) throw new RangeError(`${groups} groups taken ${k} at a time is ${total} splits; the most is ${MAX_SPLITS}`);
+  const panel = loadTrials(a.returns_file);
+  if (panel.dates.length < groups * 5) throw new RangeError(`${panel.dates.length} days is too short for ${groups} groups`);
+  const meta = { groups, test_groups: k, purge_days: a.purge_days ?? 5, embargo_days: a.embargo_days ?? 5, periods_per_year: a.periods_per_year ?? 252, trials: panel.trials, first_date: panel.dates[0], last_date: panel.dates.at(-1), total_splits: total };
+  const blocks = blocksOf(panel.dates.length, groups), splits = splitsOf(groups, k), scale = Math.sqrt(meta.periods_per_year);
+  const results = [];
+  splits.forEach((test, s) => {
+    if (a.shard && s % a.shard.of !== a.shard.index) return;
+    results.push({ s, ...runSplit(panel, blocks, test, { purge: meta.purge_days, embargo: meta.embargo_days, scale }) });
+  });
+  if (a.shard) {
+    if (a.shard.index >= a.shard.of) throw new RangeError("shard index must be below shard of");
+    const cpcv_id = createHash("sha256").update(readFileSync(a.returns_file)).update(JSON.stringify(meta)).digest("hex").slice(0, 24);
+    const shard = { schema: "canli.cpcv-shard.v1", cpcv_id, shard: a.shard, meta, results };
+    if (a.out_file) writeFileSync(a.out_file, JSON.stringify(shard));
+    return { cpcv_id, shard: a.shard, ran: results.length, of: total, ...(a.out_file ? { shard_file: a.out_file } : { shard_data: shard }), next: "Run the other shards with the same file and settings, then call cpcv with merge_files." };
+  }
+  return summarize(results, meta);
+}

package/src/diversifiers.mjs

@@ -0,0 +1,69 @@
+// What to add to a book, by correlation first. Most searches rank by return and check correlation
+// afterwards. This one ranks by how much a candidate adds in independent bets and how it behaves on
+// the book's worst days; return comes last, because a high return found by search is the least
+// trustworthy number there is.
+import { z } from "zod";
+
+import { readPanel } from "./panel-io.mjs";
+
+export const diversifiersInput = z.object({
+  book_file: z.string().min(1).max(1000).describe("A CSV of the book's daily returns: date, then one column per strategy or one for the whole book."),
+  candidates_file: z.string().min(1).max(1000).describe("A CSV of candidates' daily returns: date, then one column each."),
+  limit: z.number().int().min(1).max(100).optional().describe("How many to return; default 10."),
+  min_days: z.number().int().min(60).max(100000).optional().describe("Days a candidate must share with the book; default 252."),
+}).strict();
+
+const round = (x, d = 3) => (x == null || !Number.isFinite(x) ? null : Number(x.toFixed(d)));
+function corr(a, b) {
+  const n = a.length;
+  if (n < 3) return null;
+  let ma = 0, mb = 0;
+  for (let i = 0; i < n; i += 1) { ma += a[i]; mb += b[i]; }
+  ma /= n; mb /= n;
+  let up = 0, da = 0, db = 0;
+  for (let i = 0; i < n; i += 1) { up += (a[i] - ma) * (b[i] - mb); da += (a[i] - ma) ** 2; db += (b[i] - mb) ** 2; }
+  return da > 1e-18 && db > 1e-18 ? up / Math.sqrt(da * db) : null;
+}
+const annualSharpe = (r) => {
+  const m = r.reduce((a, b) => a + b, 0) / r.length, v = r.reduce((a, b) => a + (b - m) ** 2, 0) / (r.length - 1);
+  return v > 0 ? m / Math.sqrt(v) * Math.sqrt(252) : null;
+};
+// k for unrelated series, 1 for copies.
+function bets(series) {
+  const k = series.length;
+  let squares = 0;
+  for (let i = 0; i < k; i += 1) for (let j = 0; j < k; j += 1) { const c = i === j ? 1 : corr(series[i], series[j]) ?? 1; squares += c * c; }
+  return k * k / squares;
+}
+
+export function findDiversifiers(args) {
+  const a = diversifiersInput.parse(args);
+  const book = readPanel(a.book_file, "book_file"), cand = readPanel(a.candidates_file, "candidates_file");
+  const minDays = a.min_days ?? 252;
+  const cDate = new Map(cand.dates.map((d, i) => [d, i]));
+  const shared = book.dates.map((d, i) => [i, cDate.get(d)]).filter(([, j]) => j !== undefined);
+  if (shared.length < minDays) throw new RangeError(`The two files share ${shared.length} days and at least ${minDays} are needed.`);
+  const rows = [];
+  for (const name of cand.tickers) {
+    const pairs = shared.filter(([i, j]) => cand.values[name][j] !== null && book.tickers.every((t) => book.values[t][i] !== null));
+    if (pairs.length < minDays) continue;
+    const bookCols = book.tickers.map((t) => pairs.map(([i]) => book.values[t][i]));
+    const whole = pairs.map((_, n) => bookCols.reduce((s, c) => s + c[n], 0) / bookCols.length);
+    const c = pairs.map(([, j]) => cand.values[name][j]);
+    const worst = [...whole.keys()].sort((x, y) => whole[x] - whole[y]).slice(0, Math.max(20, Math.floor(whole.length / 10)));
+    const onWorst = worst.reduce((s, i) => s + c[i], 0) / worst.length;
+    const before = bets(bookCols), after = bets([...bookCols, c]);
+    rows.push({ name, days: pairs.length, correlation: corr(whole, c), worst_days_correlation: corr(worst.map((i) => whole[i]), worst.map((i) => c[i])), mean_on_worst_days_percent: onWorst * 100, bets_added: after - before, sharpe: annualSharpe(c) });
+  }
+  if (!rows.length) throw new RangeError(`No candidate shares ${minDays} days with the whole book.`);
+  // More independent bets first; among near ties, what it does when the book is at its worst.
+  rows.sort((x, y) => (Math.round((y.bets_added - x.bets_added) * 20) / 20) || ((x.worst_days_correlation ?? 1) - (y.worst_days_correlation ?? 1)));
+  const shown = rows.slice(0, a.limit ?? 10);
+  return {
+    book_series: book.tickers.length, candidates_read: rows.length,
+    columns: ["candidate", "days", "correlation_with_book", "correlation_on_books_worst_days", "mean_on_books_worst_days_percent", "independent_bets_added", "sharpe_alone"],
+    rows: shown.map((r) => [r.name, r.days, round(r.correlation), round(r.worst_days_correlation), round(r.mean_on_worst_days_percent), round(r.bets_added), round(r.sharpe, 2)]),
+    meaning: "independent_bets_added is how much the candidate raises the participation ratio of the correlations (1.0 for something wholly unrelated to everything in the book). The worst days are the book's worst tenth. Sharpe is shown last and is not used to rank.",
+    limits: [`${rows.length} candidates were compared, so the best of them looks better than it is: judge any one with validate_luck_trials at that many trials.`, "Correlations move, and they tend to rise in a crisis. These are from the days in the files."],
+  };
+}