canli-validation-mcp 0.12.0 → 0.13.1

package/README.md

@@ -34,14 +34,16 @@ other stdio client (Cursor, VS Code) run the same `npx` command; see "Claude Des
 
 ## What it does
 
-An MCP (Model Context Protocol) server over canlicapital.com's free, keyed validation API. It
-gives a coding agent fifteen tools: issue a free key, run the nine validators (deflated Sharpe,
-CSCV overfitting, data-snooping tests (Hansen's SPA, White's Reality Check and Romano-Wolf StepM),
-paper-evidence conformance, breadth ceiling, minimum track record length, minimum backtest length,
-haircut Sharpe ratio, luck-equivalent trials),
-audit one backtest with three of them in a single call, fetch a stored receipt, verify a
-receipt's signature offline, read service status, and read a company's reported financial history
-from SEC filings. Every validation result carries, beside the number, the sentences that say what
+An MCP (Model Context Protocol) server over canlicapital.com's free, keyed validation API, with a
+lab that computes on your machine. It gives a coding agent twenty tools: issue a free key, run the
+nine validators (deflated Sharpe, CSCV overfitting, data-snooping tests (Hansen's SPA, White's
+Reality Check and Romano-Wolf StepM), paper-evidence conformance, breadth ceiling, minimum track
+record length, minimum backtest length, haircut Sharpe ratio, luck-equivalent trials), audit one
+backtest with several of them in a single call, verify a receipt's signature offline, read service
+status, read a company's reported financial history from SEC filings, and six lab tools: backtest
+a rule over a parameter grid, summarize a long series, stress-test a strategy's returns, check a
+plan against a broker's limits, check a signal for lookahead, and test a whole research pipeline
+on placebo data. Every validation result carries, beside the number, the sentences that say what
 it does not establish and the receipt that records it, success or error, so the agent cannot see a
 number without its limits.
 
@@ -136,9 +138,9 @@ original SEC response and the record's own boundary sentence: these are accounti
 reported to the SEC, not market prices, returns or a recommendation. Companies and concepts
 outside the current release return an error with the available concepts listed.
 
-## The lab: backtest, summarize, stress, check feasibility
+## The lab: backtest, summarize, stress, feasibility, lookahead, placebos
 
-Four tools compute in the server process itself, on your machine or on the hosted endpoint, with
+Six tools compute in the server process itself, on your machine or on the hosted endpoint, with
 nothing sent to the API and no receipt stored. Each result names the SHA-256 of exactly the numbers
 it was computed from, and states what it does not establish.
 
@@ -174,6 +176,32 @@ it was computed from, and states what it does not establish.
   share of daily volume, square-root market impact (coefficient Y, default 1) with every copy of the
   same strategy counted, and the capital at which costs and impact eat the expected gross return.
   Every broker fact carries the date it was checked and its source.
+- **`check_leakage`** finds lookahead in a signal without seeing its code. A causal signal computed
+  on the first n rows gives exactly the values the full run gives for those rows. `plan` picks the
+  cuts (seeded, between 40% and 95% of the series); you rerun your own code on the rows before each
+  cut, and `compare` checks every column: a value that changed when the later rows were removed
+  used them. On your machine, `columns_file` takes the path of a JSON file `{cuts, columns}` your
+  script wrote (Python's `NaN` reads as missing) instead of thousands of pasted numbers. The result names the pattern: future rows with their horizon (a `shift(-5)` reads as
+  five rows), the full sample (a normalization, rank, scaling or fit over the whole series), or
+  scattered rows (filling gaps backwards). On a fixed zoo of 22 pandas and statsmodels signals
+  (`config/research/leakage-zoo.json`, from `scripts/research/leakage/`), all 11 leaks are found
+  with the right pattern and none of the 11 honest indicators is flagged. A pass covers only the
+  cuts tested: survivorship, vendor revisions or a leak confined to the first rows leave no trace.
+- **`placebo_test`** tests a whole research pipeline at once. `plan` writes your panel (`real.csv`)
+  and 19 placebos to a new folder in the system's temporary directory: the same returns with the
+  periods in a random order, the same order for every asset, so each asset keeps its returns and each
+  day its cross-section, but nothing can be predicted. Run the pipeline unchanged on each, every
+  search, filter and parameter choice included, and `compare` ranks its real result among the
+  placebo results. Every choice the pipeline makes is counted, because it makes them again on every
+  placebo. A pre-registered study chose the reordering (`scripts/research/placebo/v1/`,
+  `config/research/placebo-v1-evaluation.json`): in all 30 combinations of simulated market and
+  pipeline where nothing can be predicted (10,000 tests each; five return shapes, flat,
+  drifting and with assets that differ only in their constant means; SMA, time-series and
+  cross-sectional momentum grids), its false-positive rate at a nominal 5% was 4.6% to 5.5%.
+  On the same markets a t-test on the best rule found an edge 10.9% to 78.9% of the time,
+  and Bonferroni's correction 8.0% to 10.3% wherever the market drifted and 58.5% to
+  59.7% where assets only differed in their means. With 19 placebos the smallest p is 0.05; 99
+  give a finer one.
 
 On your own machine, `prices_file`, `series_file` and `returns_file` take a CSV or JSON path instead
 of the numbers. Only numbers are read, plus an ISO date column when the file has one, so results can
@@ -214,10 +242,10 @@ took three audit questions from 4 of 9 to 8 of 9 answered correctly on gpt-5.4-m
 
 ## Prompts, resources and structured results
 
-Clients that show MCP prompts offer six guided workflows: `validate_backtest` (deflated Sharpe, then
+Clients that show MCP prompts offer eight guided workflows: `validate_backtest` (deflated Sharpe, then
 overfitting, then the track record needed, reported with what each number does not establish),
-`track_record_needed`, `backtest_and_validate`, `stress_my_strategy`, `production_check` and
-`summarize_market_series`. Resources: `canli://limits`, the boundary sentences every result carries;
+`track_record_needed`, `backtest_and_validate`, `stress_my_strategy`, `production_check`,
+`summarize_market_series`, `check_signal_for_lookahead` and `test_pipeline_on_placebos`. Resources: `canli://limits`, the boundary sentences every result carries;
 `canli://sources`, the papers behind each validator and how each is checked against them;
 `canli://strategy-spec`, the JSON Schema of every strategy family and its parameters; and
 `canli://openapi`, the API's OpenAPI 3.1 document. Two resource templates are for writing code
@@ -392,10 +420,11 @@ Every envelope this server returns carries these sentences, verbatim, from the A
   its costs, survivorship, or any lookahead in how the series was built.
 - A deflated Sharpe or overfitting probability above or below any threshold is not admission to
   anything and is not a forecast.
-- The receipt is content-hashed and reproducible from the open-source core it names. It is not
-  signed.
+- The receipt is content-hashed, reproducible from the open-source core it names, and signed with
+  Ed25519 by a key published at https://canlicapital.com/.well-known/canli-receipt-keys.json.
 - Quotas: 1000 validations per key per UTC day, 5 keys per client per UTC day, 1048576 bytes per
-  request, 20000 observations per series, 200 variants per matrix.
+  validation request, 1024 bytes per key revocation request, 20000 observations per series, 200
+  variants per matrix.
 
 Each tool's description also states one of these sentences, so an agent sees the boundary before
 it calls the tool, not only after.
@@ -420,25 +449,25 @@ no key is configured.
 
 | toolset | tools |
 |---|---|
-| `validate` | `get_key`, the eight validators, `audit_backtest` |
+| `validate` | `get_key`, the nine validators, `audit_backtest` |
 | `receipts` | `verify_receipt` |
 | `company` | `company_financial_history` |
 | `status` | `service_status` |
-| `lab` | `backtest_strategy`, `summarize_series`, `stress_test`, `check_feasibility` |
+| `lab` | `backtest_strategy`, `summarize_series`, `stress_test`, `check_feasibility`, `check_leakage`, `placebo_test` |
 
 Measured with `bench/tool_list_tokens.py` (tokenizer: tiktoken `o200k_base`; other models'
 tokenizers give different absolute counts), in the shape an OpenAI-style client sends the list:
 
 | CANLI_TOOLSETS | tools | tokens per turn | of all |
 |---|---|---|---|
-| (default) | 17 | 5,581 | 95% |
-| (default, with CANLI_KEY) | 16 | 5,445 | 93% |
-| `all` | 18 | 5,852 | 100% |
-| `validate` | 11 | 3,708 | 63% |
-| `receipts` | 1 | 195 | 3% |
-| `company` | 1 | 273 | 5% |
-| `status` | 1 | 89 | 2% |
-| `lab` | 4 | 1,595 | 27% |
+| (default) | 19 | 5,978 | 96% |
+| (default, with CANLI_KEY) | 18 | 5,842 | 93% |
+| `all` | 20 | 6,249 | 100% |
+| `validate` | 11 | 3,658 | 59% |
+| `receipts` | 1 | 182 | 3% |
+| `company` | 1 | 273 | 4% |
+| `status` | 1 | 89 | 1% |
+| `lab` | 6 | 2,055 | 33% |
 
 Providers cache a tool list that is identical from turn to turn and bill the cached part at a
 fraction of the price (`test/tool-list-stable.test.mjs` keeps each list byte-stable); a smaller list

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "canli-validation-mcp",
-  "version": "0.12.0",
+  "version": "0.13.1",
   "description": "Check whether a backtest is real: deflated Sharpe, CSCV probability of backtest overfitting, minimum track record and backtest length, haircut Sharpe and luck-equivalent trials, as an MCP server. Local mode, hosted endpoint, signed receipts.",
   "keywords": [
     "mcp",
@@ -38,7 +38,7 @@
     "test:package": "node test/package-smoke.mjs"
   },
   "dependencies": {
-    "@modelcontextprotocol/server": "2.1.0",
+    "@modelcontextprotocol/server": "2.3.1",
     "zod": "4.6.5"
   },
   "mcpName": "io.github.arhancanli/canli-validation-mcp",
@@ -52,7 +52,7 @@
     "url": "https://github.com/arhancanli/canlicapital/issues"
   },
   "devDependencies": {
-    "@modelcontextprotocol/client": "2.1.0",
+    "@modelcontextprotocol/client": "2.3.1",
     "fast-check": "4.10.2"
   }
 }

package/src/lab-schemas.mjs

@@ -1,22 +1,26 @@
 // mcp/src/lab-schemas.mjs
 //
 // Input and output schemas for the lab tools (src/lab.mjs): backtest_strategy, summarize_series,
-// stress_test and check_feasibility. Like the validators' descriptions, each tool description
-// carries one sentence of its own result's boundary language verbatim (test/lab.test.mjs checks
-// it against the sentences the computation attaches), so an agent reads what the result cannot be
-// used to claim before it ever calls the tool. Descriptions stay terse: the tool list is re-sent to
-// the model on every turn.
+// stress_test, check_feasibility, check_leakage and placebo_test. Like the validators'
+// descriptions, each tool description carries one sentence of its own result's boundary language
+// verbatim (test/lab.test.mjs checks it against the sentences the computation attaches), so an
+// agent reads what the result cannot be used to claim before it ever calls the tool. Descriptions
+// stay terse: the tool list is re-sent to the model on every turn.
 import { z } from "zod";
 import { BACKTEST_LIMITS_TEXT, FAMILIES } from "./local/js/backtest-core.js";
 import { FEASIBILITY_LIMITS_TEXT } from "./local/js/feasibility-core.js";
 import { SUMMARY_LIMITS_TEXT } from "./local/js/series-summary-core.js";
 import { STRESS_LIMITS_TEXT } from "./local/js/stress-core.js";
+import { LEAKAGE_LIMITS_TEXT } from "./local/js/leakage-core.js";
+import { PLACEBO_LIMITS_TEXT, PLACEBO_METHODS } from "./local/js/placebo-core.js";
 
 export const LAB_BOUNDARY = Object.freeze({
   backtest_strategy: BACKTEST_LIMITS_TEXT[3],
   summarize_series: SUMMARY_LIMITS_TEXT[2],
   stress_test: STRESS_LIMITS_TEXT[2],
   check_feasibility: FEASIBILITY_LIMITS_TEXT[0],
+  check_leakage: LEAKAGE_LIMITS_TEXT[0],
+  placebo_test: PLACEBO_LIMITS_TEXT[0],
 });
 
 export const LAB_TOOL_DESCRIPTIONS = Object.freeze({
@@ -24,6 +28,8 @@ export const LAB_TOOL_DESCRIPTIONS = Object.freeze({
   summarize_series: `A long price or return series in about 100 words plus fields: growth, risk, dated drawdowns, trend, volatility regime, tails, jumps, stale data. Use instead of reading raw bars. ${LAB_BOUNDARY.summarize_series}`,
   stress_test: `Stress a strategy's returns: bootstrap histories (how often the drawdown limit breaks or Sharpe turns negative) and named crash, volatility, repeat and stuck-position scenarios, with a fragility share. ${LAB_BOUNDARY.stress_test}`,
   check_feasibility: `Check a trading plan before it trades: broker order-rate and minimum-order limits, 2026 US day-trading rules, square-root market impact with crowding, and the capital where costs eat the return. ${LAB_BOUNDARY.check_feasibility}`,
+  check_leakage: `Does a signal look ahead? plan gives cuts; rerun your code on the first cut rows for each, then compare. Rows that changed used later rows; the result names the pattern and horizon. No code is sent. ${LAB_BOUNDARY.check_leakage}`,
+  placebo_test: `Does your pipeline find edges in noise? plan writes placebo files (your data's returns in random orders, so nothing is predictable); run the whole pipeline on real.csv and on each, then compare for a p-value that counts every choice it makes. ${LAB_BOUNDARY.placebo_test}`,
 });
 
 const FAMILY_NAMES = Object.keys(FAMILIES);
@@ -102,6 +108,37 @@ export const feasibilityInput = z
   })
   .strict();
 
+export const leakageInput = z
+  .object({
+    action: z.enum(["plan", "compare"]),
+    observations: z.number().int().optional().describe("plan: rows in the series."),
+    prefixes: z.number().int().optional().describe("plan: cuts, default 5."),
+    seed: z.number().int().optional(),
+    cuts: z.array(z.number().int()).optional().describe("compare: plan's cuts."),
+    // Shapes and lengths are checked by the computation (js/leakage-core.js) with messages that
+    // say what to send; the advertised schema stays small because it is re-sent every turn.
+    columns: z.record(z.string(), z.looseObject({})).optional().describe("compare: {name: {full: [all rows], prefixes: [[first cuts[i] rows], ...]}}, null for missing."),
+    columns_file: z.string().optional().describe("compare, on your machine: a JSON file {cuts, columns} instead."),
+    timestamps: z.array(z.string()).optional(),
+    tolerance: z.number().optional().describe("Default 1e-9."),
+  })
+  .strict();
+
+export const placeboInput = z
+  .object({
+    action: z.enum(["plan", "compare"]),
+    data_file: z.string().optional().describe("plan, on your machine: CSV or JSON, a column per asset, dates optional."),
+    columns: z.record(z.string(), z.array(z.number())).optional().describe("plan: {asset: [values]} instead of data_file."),
+    kind: z.enum(["prices", "returns"]).optional().describe("plan: default prices."),
+    placebos: z.number().int().optional().describe("plan: default 19, up to 199."),
+    method: z.enum(PLACEBO_METHODS).optional(),
+    seed: z.number().int().optional(),
+    real: z.number().optional().describe("compare: the pipeline's result on the real data."),
+    placebo_results: z.array(z.number()).optional().describe("compare: its result on each placebo, in file order."),
+    lower_is_better: z.boolean().optional(),
+  })
+  .strict();
+
 const loose = z.looseObject({}).optional();
 const sentences = z.array(z.string()).optional();
 
@@ -117,6 +154,14 @@ export const stressOutput = z
   .looseObject({ baseline: loose, resampled: loose, scenarios: z.array(z.unknown()).optional(), fragility: loose, plain_reading: z.string().optional(), limits: sentences, source: loose })
   .describe("resampled holds bootstrap percentiles and breach frequencies; scenarios each named stress and its rule; fragility the share of histories breaking the limit or losing.");
 
+export const leakageOutput = z
+  .looseObject({ verdict: z.string().optional(), cuts: z.array(z.number()).optional(), seed: z.number().optional(), columns: loose, flagged_columns: z.array(z.string()).optional(), plain_reading: z.string().optional(), limits: sentences, source: loose })
+  .describe("plan: cuts and seed. compare: verdict over all columns, flagged_columns, and per column its pattern, sentence and per-cut changes.");
+
+export const placeboOutput = z
+  .looseObject({ verdict: z.string().optional(), p_value: z.number().optional(), dir: z.string().optional(), files: z.array(z.string()).optional(), placebo: loose, plain_reading: z.string().optional(), limits: sentences, source: loose })
+  .describe("plan: dir, real.csv and the placebo files, method and seed. compare: verdict, p_value, and what the pipeline found on the placebos.");
+
 export const feasibilityOutput = z
   .looseObject({ verdict: z.string().optional(), checks: z.array(z.unknown()).optional(), to_change: z.array(z.string()).optional(), sources: loose, plain_reading: z.string().optional(), limits: sentences })
   .describe("verdict summarizes the checks; each check has a status (ok, warning, blocking, unknown), its finding and numbers; to_change lists what to fix.");

package/src/lab.mjs

@@ -13,17 +13,23 @@
 //
 // Also here: the code-generation resources (every tool's exact JSON Schemas, the strategy spec,
 // runnable client examples, the API's OpenAPI document) and the lab's guided prompts.
-import { createHash } from "node:crypto";
+import { createHash, randomInt } from "node:crypto";
+import { mkdtempSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
 import { ResourceTemplate } from "@modelcontextprotocol/server";
 import { z } from "zod";
 import { BACKTEST_LIMITS, BACKTEST_LIMITS_TEXT, FAMILIES, runBacktest } from "./local/js/backtest-core.js";
 import { FEASIBILITY_LIMITS_TEXT, checkFeasibility } from "./local/js/feasibility-core.js";
 import { SUMMARY_LIMITS_TEXT, summarizeSeries } from "./local/js/series-summary-core.js";
 import { STRESS_LIMITS_TEXT, stressTest } from "./local/js/stress-core.js";
-import { LAB_TOOL_DESCRIPTIONS, backtestInput, backtestOutput, feasibilityInput, feasibilityOutput, stressInput, stressOutput, summarizeInput, summaryOutput } from "./lab-schemas.mjs";
-import { readSeriesWithDates } from "./series-file.mjs";
+import { LEAKAGE_LIMITS, LEAKAGE_LIMITS_TEXT, checkLeakage, planPrefixes } from "./local/js/leakage-core.js";
+import { PLACEBO_DEFAULT_METHOD, PLACEBO_LIMITS, PLACEBO_LIMITS_TEXT, checkPanel, placeboPValue, placeboPanels } from "./local/js/placebo-core.js";
+import { LAB_TOOL_DESCRIPTIONS, backtestInput, backtestOutput, feasibilityInput, feasibilityOutput, leakageInput, leakageOutput, placeboInput, placeboOutput, stressInput, stressOutput, summarizeInput, summaryOutput } from "./lab-schemas.mjs";
+import { readColumnsFile, readPanelFile, readSeriesWithDates, writePanelFile } from "./series-file.mjs";
+import { leanJsonSchema } from "./schemas.mjs";
 
-export const LAB_TOOLS = Object.freeze(["backtest_strategy", "summarize_series", "stress_test", "check_feasibility"]);
+export const LAB_TOOLS = Object.freeze(["backtest_strategy", "summarize_series", "stress_test", "check_feasibility", "check_leakage", "placebo_test"]);
 
 // Same shape as every other tool's result: the object as text, and as structured content.
 const labText = (value) => ({ content: [{ type: "text", text: JSON.stringify(value) }], structuredContent: value });
@@ -83,16 +89,129 @@ export async function toolCheckFeasibility(session, args) {
   return labText({ ...checkFeasibility(input), limits: FEASIBILITY_LIMITS_TEXT });
 }
 
+// check_leakage: plan picks the cut points (a fresh seed unless one is sent, so the caller does not
+// choose them); compare reads the caller's outputs and reports, per column, what changed and why.
+export async function toolCheckLeakage(session, args) {
+  const input = parse(leakageInput, args, "check_leakage");
+  if (input.action === "plan") {
+    if (input.observations === undefined) throw new Error("check_leakage: plan needs observations, the number of rows in the full series");
+    const seed = input.seed ?? randomInt(1, 2 ** 31);
+    const cuts = planPrefixes(input.observations, { prefixes: input.prefixes ?? LEAKAGE_LIMITS.default_prefixes, seed });
+    return labText({
+      schema: "canli.leakage-plan.v1", observations: input.observations, cuts, seed,
+      plain_reading: `Run your signal code ${cuts.length + 1} times: on all ${input.observations} rows, and on the first ${cuts.join(", ")} rows. Send each output column with action compare and these cuts; prefixes[i] holds the run on the first cuts[i] rows.`,
+      limits: LEAKAGE_LIMITS_TEXT,
+    });
+  }
+  let { cuts, columns, timestamps } = input;
+  if (input.columns_file !== undefined) {
+    if (columns !== undefined) throw new Error("check_leakage: send columns or columns_file, not both");
+    if (session.hosted) throw new Error("check_leakage: the hosted endpoint cannot read files on your machine; send columns as numbers, or run the server locally with npx -y canli-validation-mcp.");
+    const file = readColumnsFile(input.columns_file);
+    if (cuts !== undefined && file.cuts !== undefined && JSON.stringify(cuts) !== JSON.stringify(file.cuts)) throw new Error("check_leakage: the cuts sent differ from the cuts in columns_file");
+    cuts ??= file.cuts;
+    columns = file.columns;
+    timestamps ??= file.timestamps;
+  }
+  if (cuts === undefined || columns === undefined) throw new Error("check_leakage: compare needs cuts (from plan) and columns, sent or in a columns_file");
+  const result = checkLeakage({ cuts, columns, tolerance: input.tolerance, timestamps: labels(timestamps) });
+  // Per cut, only what a reader needs: where the changes start and how large they are.
+  const report = Object.fromEntries(Object.entries(result.columns).map(([name, c]) => [name, {
+    verdict: c.verdict, pattern: c.pattern, sentence: c.sentence,
+    ...(c.horizon_rows !== undefined ? { horizon_rows: c.horizon_rows } : {}),
+    ...(c.share_changed !== undefined ? { share_changed: c.share_changed } : {}),
+    ...(c.first_changed_at !== undefined ? { first_changed_at: c.first_changed_at } : {}),
+    per_cut: c.prefixes.map((p) => ({ cut: p.cut, changed: p.changed, first_changed: p.first_changed, max_abs_difference: p.max_abs_difference })),
+  }]));
+  const flagged = result.flagged_columns;
+  return labText({
+    schema: "canli.leakage.v1", verdict: result.verdict, flagged_columns: flagged, cuts: result.cuts, columns: report,
+    source: { ...(input.columns_file !== undefined ? { columns_file: input.columns_file } : {}), input_sha256: digest({ cuts, columns }) },
+    plain_reading: flagged.length
+      ? `Lookahead in ${flagged.length} of ${Object.keys(report).length} column${Object.keys(report).length === 1 ? "" : "s"} (${flagged.join(", ")}). ${flagged.map((n) => `${n}: ${report[n].sentence}`).join(" ")}`
+      : `No value changed in ${Object.keys(report).length} column${Object.keys(report).length === 1 ? "" : "s"} at ${result.cuts.length} cuts: these prefixes found no lookahead.`,
+    limits: LEAKAGE_LIMITS_TEXT,
+  });
+}
+
+// placebo_test: plan reorders the periods of the caller's panel, the same order for every column,
+// and writes the real panel and each placebo as CSV to a new folder in the system's temporary
+// directory (the hosted endpoint, which writes nothing, returns small placebos inline); compare
+// ranks the pipeline's real result among its placebo results.
+const PLACEBO_INLINE_MAX = 50000;
+const short = (x) => Number(x.toPrecision(4));
+
+export async function toolPlaceboTest(session, args) {
+  const input = parse(placeboInput, args, "placebo_test");
+  if (input.action === "compare") {
+    if (input.real === undefined || input.placebo_results === undefined) throw new Error("placebo_test: compare needs real and placebo_results, from the pipeline's runs on real.csv and the placebo files");
+    const r = placeboPValue(input.real, input.placebo_results, { higherIsBetter: !input.lower_is_better });
+    const beats = r.as_good_as_real === 0;
+    const luck = `On the placebos, where nothing can be predicted, it found a median of ${short(r.placebo.median)} and at best ${short(r.placebo.best)}.`;
+    return labText({
+      schema: "canli.placebo.v1", verdict: r.p_value <= 0.05 ? "beats_placebos" : "within_placebo_range", ...r,
+      source: { input_sha256: digest({ real: input.real, placebo_results: input.placebo_results }) },
+      plain_reading: beats
+        ? `The pipeline's real result, ${short(r.real)}, beat all ${r.placebos} placebos: p = ${short(r.p_value)}, the smallest ${r.placebos} placebos can give${r.placebos < 99 ? "; 99 placebos give a finer p" : ""}. ${luck}`
+        : `${r.as_good_as_real} of ${r.placebos} placebos gave a result at least as good as the real one, ${short(r.real)}: p = ${short(r.p_value)}. ${luck}`,
+      limits: PLACEBO_LIMITS_TEXT,
+    });
+  }
+  if ((input.data_file === undefined) === (input.columns === undefined)) throw new Error("placebo_test: plan needs exactly one of data_file or columns");
+  const kind = input.kind ?? "prices";
+  const method = input.method ?? PLACEBO_DEFAULT_METHOD;
+  const placebos = input.placebos ?? PLACEBO_LIMITS.default_placebos;
+  const seed = input.seed ?? randomInt(1, 2 ** 31);
+  let panel;
+  if (input.data_file !== undefined) {
+    if (session.hosted) throw new Error("placebo_test: the hosted endpoint cannot read files on your machine; send columns as numbers, or run the server locally with npx -y canli-validation-mcp.");
+    panel = readPanelFile(input.data_file);
+  } else {
+    const names = Object.keys(input.columns);
+    panel = { columns: names.map((n) => input.columns[n]), names, dates: null, date_name: null, skipped: [], format: "csv" };
+  }
+  checkPanel(panel.columns, kind);
+  const source = {
+    ...(input.data_file !== undefined ? { data_file: input.data_file, column_positions: panel.positions, ...(panel.date_column ? { date_column_position: panel.date_column } : {}), ...(panel.skipped.length ? { skipped_row_counter_columns: panel.skipped } : {}) } : {}),
+    columns: panel.columns.length, rows: panel.columns[0].length, input_sha256: digest(panel.columns),
+  };
+  const made = placeboPanels(panel.columns, { kind, method, placebos, seed });
+  const plan = { schema: "canli.placebo-plan.v1", kind, method, placebos, seed };
+  if (session.hosted) {
+    if (placebos * panel.columns.length * panel.columns[0].length > PLACEBO_INLINE_MAX) throw new Error(`placebo_test: the hosted endpoint returns at most ${PLACEBO_INLINE_MAX} placebo numbers; send fewer rows or columns, or run the server locally with npx -y canli-validation-mcp, which writes the placebos to files`);
+    const names = panel.names;
+    return labText({ ...plan, placebo_columns: [...made].map((p) => Object.fromEntries(names.map((n, i) => [n, p[i]]))), plain_reading: `Run the pipeline unchanged on the real columns and on each of the ${placebos} placebos, then send its results to compare in this order.`, limits: PLACEBO_LIMITS_TEXT, source });
+  }
+  const dir = mkdtempSync(join(tmpdir(), "canli-placebo-"));
+  const layout = { names: panel.names, dates: panel.dates, dateName: panel.date_name, format: panel.format };
+  const real = `real.${panel.format}`;
+  writePanelFile(join(dir, real), { ...layout, columns: panel.columns });
+  const files = [];
+  for (const columns of made) {
+    const name = `placebo_${String(files.length + 1).padStart(3, "0")}.${panel.format}`;
+    writePanelFile(join(dir, name), { ...layout, columns });
+    files.push(name);
+  }
+  return labText({
+    ...plan, dir, real, files,
+    plain_reading: `Run the pipeline unchanged, every search and choice included, on ${real} and on each of the ${placebos} placebo files in ${dir}, record the one number it reports (such as its best Sharpe), and send them to compare in file order.`,
+    limits: PLACEBO_LIMITS_TEXT, source,
+  });
+}
+
 // The lab computes locally and changes nothing anywhere: read-only, closed world, idempotent.
 const LAB_ANNOTATIONS = { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false };
 
 export function labToolSpecs(session) {
-  const spec = (name, title, input, output, handler) => [name, { title, annotations: { title, ...LAB_ANNOTATIONS }, description: LAB_TOOL_DESCRIPTIONS[name], inputSchema: input, outputSchema: output }, (args) => handler(session, args)];
+  const spec = (name, title, input, output, handler, annotations = LAB_ANNOTATIONS) => [name, { title, annotations: { title, ...annotations }, description: LAB_TOOL_DESCRIPTIONS[name], inputSchema: input, outputSchema: output }, (args) => handler(session, args)];
   return [
     spec("backtest_strategy", "Backtest a rule over a parameter grid, validated", backtestInput, backtestOutput, toolBacktestStrategy),
     spec("summarize_series", "Summarize a price or return series", summarizeInput, summaryOutput, toolSummarizeSeries),
     spec("stress_test", "Stress-test a strategy's returns", stressInput, stressOutput, toolStressTest),
     spec("check_feasibility", "Check a plan against broker limits and impact", feasibilityInput, feasibilityOutput, toolCheckFeasibility),
+    spec("check_leakage", "Check a signal for lookahead", leakageInput, leakageOutput, toolCheckLeakage),
+    // plan writes files, each call to a new temporary folder: not read-only, not idempotent.
+    spec("placebo_test", "Test a research pipeline on placebo data", placeboInput, placeboOutput, toolPlaceboTest, { readOnlyHint: false, destructiveHint: false, idempotentHint: false, openWorldHint: false }),
   ];
 }
 
@@ -121,6 +240,8 @@ export const EXAMPLE_ARGS = Object.freeze({
   summarize_series: { series_file: "prices.csv", series_kind: "prices", name: "My asset" },
   stress_test: { returns_file: "strategy_returns.csv", drawdown_limit: 0.2, paths: 1000, seed: 42 },
   check_feasibility: { broker: "alpaca", asset_class: "us_equity", account: "margin", capital_usd: 250000, orders_per_rebalance: 40, rebalances_per_year: 52, turnover_per_year: 8, adv_usd: 20000000, daily_volatility: 0.02, copies: 1, expected_gross_return: 0.12, spread_and_fees_bps: 3 },
+  check_leakage: { action: "plan", observations: 2520, prefixes: 5 },
+  placebo_test: { action: "plan", data_file: "prices.csv" },
 });
 
 const usesFile = (args) => Object.keys(args).some((k) => k.endsWith("_file"));
@@ -220,7 +341,7 @@ export function registerCodeResources(server, session, catalog) {
     (uri, { tool }) => {
       const entry = catalog[tool];
       if (!entry) throw new Error(`No tool ${tool}; tools are ${names.join(", ")}`);
-      const body = { tool, description: entry.description, input_schema: z.toJSONSchema(entry.inputSchema, { io: "input" }), output_schema: z.toJSONSchema(entry.outputSchema, { io: "output" }) };
+      const body = { tool, description: entry.description, input_schema: leanJsonSchema(z.toJSONSchema(entry.inputSchema, { io: "input" })), output_schema: leanJsonSchema(z.toJSONSchema(entry.outputSchema, { io: "output" })) };
       return { contents: [{ uri: uri.href, mimeType: "application/json", text: JSON.stringify(body, null, 2) }] };
     },
   );
@@ -305,4 +426,26 @@ export function registerLabPrompts(server) {
       "Point out anything in the data check (stale prices, jumps) that should be fixed before the series is backtested.",
     ],
   );
+  prompt(
+    "check_signal_for_lookahead",
+    "Does my signal look ahead?",
+    "Rerun your own signal code on prefixes the server picks, and find any value that depends on later rows.",
+    { rows: z.string().describe("Rows in the full series the signal is computed on") },
+    ({ rows }) => [
+      `Call check_leakage with action plan and observations ${rows}.`,
+      "Run your own signal and feature code once on all rows and once on the first cut rows for each cut it returns, changing nothing else, and send every output column with action compare. On this machine, write them to a JSON file {cuts, columns} and send its path as columns_file instead of pasting numbers.",
+      "For each flagged column, explain the pattern (future rows with their horizon, full-sample statistics, filled gaps), find the line of code that causes it, fix it, and run the check again. Say that a pass covers only these cuts.",
+    ],
+  );
+  prompt(
+    "test_pipeline_on_placebos",
+    "Does my pipeline find edges in noise?",
+    "Run your whole research pipeline on placebo data, where nothing can be predicted, and see whether its real result beats what it finds there.",
+    { data_file: z.string().describe("Path to the CSV or JSON the pipeline reads: a column per asset, dates optional") },
+    ({ data_file }) => [
+      `Call placebo_test with action plan and data_file ${data_file}.`,
+      "Run the research pipeline unchanged, every search, filter and parameter choice included, on real.csv and on each placebo file in the folder it returns, and record the one number the pipeline reports, such as its best Sharpe.",
+      "Call placebo_test with action compare, the real number as real and the placebo numbers in file order as placebo_results. Explain the p-value and what the pipeline finds on data where nothing can be predicted, and say that only this pipeline's search is counted, not pipelines tried before it.",
+    ],
+  );
 }

package/src/local/js/leakage-core.js

@@ -0,0 +1,182 @@
+// =============================================================================
+// leakage-core.js
+// -----------------------------------------------------------------------------
+// A lookahead check that needs no code. A causal signal's value at row t depends only on rows 0
+// to t, so computing it on the first n rows gives exactly the values the full run gives for rows
+// 0 to n - 1. The server picks the cut points; the caller reruns its own code on each prefix and
+// sends every column back; a row whose value changed when the rows after the cut were removed
+// used those rows.
+//
+//   planPrefixes    the cut points (prefix lengths) for a series, seeded so a plan reproduces
+//   compareColumn   for each prefix: rows compared, rows that changed, the first and last changed
+//                   row, the largest change, and how far before the cut the changes start
+//   diagnose        what the pattern of changes says: future rows (a fixed horizon before every
+//                   cut), the full sample (most of every prefix changed: a full-sample
+//                   normalization, rank or fit), or sparse rows (typical of filling gaps backwards)
+//
+// A pass shows only that these prefixes found no lookahead: the caller supplies the columns, and
+// survivorship, vendor revisions or a leak that only shows in rows never cut leave no trace here.
+// =============================================================================
+
+import { makeRandom } from "./selection-risk-core.js";
+
+export const LEAKAGE_LIMITS = Object.freeze({
+  min_observations: 30,
+  max_observations: 100000,
+  min_prefixes: 2,
+  max_prefixes: 20,
+  default_prefixes: 5,
+  max_columns: 20,
+  default_tolerance: 1e-9,
+});
+
+export const LEAKAGE_LIMITS_TEXT = Object.freeze([
+  "A pass covers only these cuts: the columns are the caller's, and survivorship, vendor revisions or a leak no cut reaches leave no trace.",
+  "Cut points fall between 40% and 95% of the series, so a leak confined to its first rows is not tested.",
+]);
+
+/**
+ * Prefix lengths for a series of `observations` rows: `prefixes` distinct cuts between 40% and 95%
+ * of the series, one in each equal slot of that range at a seeded position, sorted.
+ */
+export function planPrefixes(observations, { prefixes = LEAKAGE_LIMITS.default_prefixes, seed = 1 } = {}) {
+  const n = Number(observations);
+  const k = Number(prefixes);
+  if (!Number.isInteger(n) || n < LEAKAGE_LIMITS.min_observations || n > LEAKAGE_LIMITS.max_observations) {
+    throw new RangeError(`observations must be an integer from ${LEAKAGE_LIMITS.min_observations} to ${LEAKAGE_LIMITS.max_observations}`);
+  }
+  if (!Number.isInteger(k) || k < LEAKAGE_LIMITS.min_prefixes || k > LEAKAGE_LIMITS.max_prefixes) {
+    throw new RangeError(`prefixes must be an integer from ${LEAKAGE_LIMITS.min_prefixes} to ${LEAKAGE_LIMITS.max_prefixes}`);
+  }
+  const lo = Math.ceil(0.4 * n);
+  const hi = Math.floor(0.95 * n);
+  if (hi - lo + 1 < k) throw new RangeError(`a series of ${n} rows has room for at most ${hi - lo + 1} cuts`);
+  const next = makeRandom(Number(seed) >>> 0 || 1);
+  const width = (hi - lo + 1) / k;
+  const cuts = new Set();
+  for (let i = 0; i < k; i++) {
+    const start = lo + Math.floor(i * width);
+    const end = Math.max(start, lo + Math.floor((i + 1) * width) - 1);
+    let cut = start + Math.floor(next() * (end - start + 1));
+    while (cuts.has(cut)) cut += 1;
+    cuts.add(cut);
+  }
+  return [...cuts].sort((a, b) => a - b);
+}
+
+const missing = (v) => v === null || v === undefined || (typeof v === "number" && Number.isNaN(v));
+
+function readValue(v, where) {
+  if (missing(v)) return null;
+  const x = typeof v === "number" ? v : typeof v === "string" && v.trim() !== "" ? Number(v) : Number.NaN;
+  if (!Number.isFinite(x)) throw new RangeError(`${where} is ${JSON.stringify(v)}, not a number or null`);
+  return x;
+}
+
+/**
+ * Compares one column's full run with its prefix runs. `prefixes[i]` must hold exactly `cuts[i]`
+ * values: the caller's code run on rows 0 to cuts[i] - 1. Missing values (null) match only missing
+ * values; numbers match within `tolerance` relative to max(1, |a|, |b|).
+ */
+export function compareColumn(full, prefixes, cuts, { tolerance = LEAKAGE_LIMITS.default_tolerance, name = "column" } = {}) {
+  if (!Array.isArray(full)) throw new RangeError(`${name}.full must be an array`);
+  if (!Array.isArray(prefixes) || prefixes.length !== cuts.length) throw new RangeError(`${name}.prefixes must hold ${cuts.length} runs, one per cut`);
+  const f = full.map((v, i) => readValue(v, `${name}.full[${i}]`));
+  return cuts.map((cut, p) => {
+    const run = prefixes[p];
+    if (!Array.isArray(run) || run.length !== cut) {
+      throw new RangeError(`${name}.prefixes[${p}] has ${Array.isArray(run) ? run.length : "no"} values; its cut is ${cut}: send your code's output on exactly the first ${cut} rows`);
+    }
+    if (cut > f.length) throw new RangeError(`${name}: cut ${cut} is beyond the ${f.length} rows of the full run`);
+    let changed = 0, first = -1, last = -1, maxDiff = 0;
+    for (let t = 0; t < cut; t++) {
+      const a = f[t];
+      const b = readValue(run[t], `${name}.prefixes[${p}][${t}]`);
+      let same;
+      if (a === null || b === null) same = a === b;
+      else {
+        const d = Math.abs(a - b);
+        same = d <= tolerance * Math.max(1, Math.abs(a), Math.abs(b));
+        if (!same && d > maxDiff) maxDiff = d;
+      }
+      if (!same) {
+        changed += 1;
+        if (first < 0) first = t;
+        last = t;
+      }
+    }
+    return {
+      cut,
+      compared: cut,
+      changed,
+      share_changed: changed / cut,
+      first_changed: first < 0 ? null : first,
+      last_changed: last < 0 ? null : last,
+      rows_before_cut: first < 0 ? null : cut - first,
+      max_abs_difference: changed ? maxDiff : 0,
+    };
+  });
+}
+
+/**
+ * What a column's changes say. future_rows: in every prefix the changed rows sit in a block that
+ * ends at the cut and is no wider than a tenth of the shortest prefix, so the value at row t uses
+ * rows up to t + horizon. full_sample: most rows of some prefix changed. sparse: anything else.
+ */
+export function diagnose(results) {
+  const touched = results.filter((r) => r.changed > 0);
+  if (!touched.length) {
+    return { verdict: "no_lookahead_found", pattern: "none", sentence: `No value changed in ${results.length} prefixes: these cuts found no lookahead.` };
+  }
+  const shortest = Math.min(...results.map((r) => r.cut));
+  const horizon = Math.max(...touched.map((r) => r.rows_before_cut));
+  const endsAtCut = touched.every((r) => r.last_changed === r.cut - 1 && r.changed === r.rows_before_cut);
+  if (endsAtCut && horizon <= Math.max(1, Math.floor(shortest / 10))) {
+    return {
+      verdict: "lookahead_found",
+      pattern: "future_rows",
+      horizon_rows: horizon,
+      sentence: `Up to ${horizon} row${horizon === 1 ? "" : "s"} before each cut changed when the later rows were removed: the value at row t uses at least the next ${horizon} row${horizon === 1 ? "" : "s"}, as shift(-${horizon}), a centered window or a forward return would.`,
+    };
+  }
+  const widest = Math.max(...touched.map((r) => r.share_changed));
+  if (widest > 0.5) {
+    return {
+      verdict: "lookahead_found",
+      pattern: "full_sample",
+      share_changed: widest,
+      sentence: `Up to ${(widest * 100).toFixed(0)}% of a prefix's rows changed when later rows were removed: values depend on the whole sample, as a full-sample normalization, rank, scaling or model fit would.`,
+    };
+  }
+  const first = Math.min(...touched.map((r) => r.first_changed));
+  return {
+    verdict: "lookahead_found",
+    pattern: "sparse",
+    first_changed: first,
+    sentence: `Scattered rows changed when later rows were removed (the earliest at row ${first}): typical of filling gaps backwards (bfill) or a lookup that reaches forward.`,
+  };
+}
+
+/** The whole check: every column compared at every cut, and one verdict for the set. */
+export function checkLeakage({ cuts, columns, tolerance = LEAKAGE_LIMITS.default_tolerance, timestamps }) {
+  if (!Array.isArray(cuts) || cuts.length < LEAKAGE_LIMITS.min_prefixes || cuts.length > LEAKAGE_LIMITS.max_prefixes) {
+    throw new RangeError(`cuts must list ${LEAKAGE_LIMITS.min_prefixes} to ${LEAKAGE_LIMITS.max_prefixes} prefix lengths, as check_leakage's plan returned them`);
+  }
+  const names = Object.keys(columns ?? {});
+  if (!names.length || names.length > LEAKAGE_LIMITS.max_columns) throw new RangeError(`send 1 to ${LEAKAGE_LIMITS.max_columns} columns`);
+  const out = {};
+  for (const name of names) {
+    const results = compareColumn(columns[name].full, columns[name].prefixes, cuts, { tolerance, name });
+    const d = diagnose(results);
+    const label = (i) => (timestamps && i !== null && i !== undefined && timestamps[i] !== undefined ? timestamps[i] : undefined);
+    out[name] = { ...d, ...(d.pattern === "sparse" && label(d.first_changed) ? { first_changed_at: label(d.first_changed) } : {}), prefixes: results };
+  }
+  const flagged = names.filter((n) => out[n].verdict === "lookahead_found");
+  return {
+    verdict: flagged.length ? "lookahead_found" : "no_lookahead_found",
+    flagged_columns: flagged,
+    cuts,
+    tolerance,
+    columns: out,
+  };
+}

package/src/local/js/placebo-core.js

@@ -0,0 +1,164 @@
+// =============================================================================
+// placebo-core.js
+// -----------------------------------------------------------------------------
+// The pipeline placebo. A research pipeline searches data for a strategy and reports a result,
+// usually the best Sharpe it found. How good a result does the same pipeline find in data where
+// nothing can be predicted? A placebo dataset holds the real returns with their periods reordered,
+// the same reordering for every column: each column keeps its returns (mean, volatility, fat
+// tails) and each period keeps its cross-section, but no period's position says anything about
+// the next. The caller runs its own pipeline on the real data and on every placebo; the rank of the
+// real result among the placebo results is the p-value. The pipeline's code never leaves the
+// caller, and every choice the pipeline makes is counted, because it makes them again on every
+// placebo.
+//
+//   placeboOrder     one reordering of n periods: a permutation, a permutation of blocks, or a
+//                    stationary bootstrap (Politis and Romano 1994)
+//   placeboPanel     a panel of prices or returns, reordered: prices are rebuilt from the
+//                    reordered returns, starting from the real first prices
+//   placeboPanels    K placebos from one seed, one at a time
+//   placeboPValue    the real result's rank among the placebo results, as a p-value
+//
+// When the periods are exchangeable (independent and identically distributed rows), the real data
+// and each permutation of it are equally likely, so the real result's rank is uniform and the
+// permutation p-value is exact. Volatility clustering breaks exchangeability; how far that moves
+// the p-value was measured before a method was chosen (scripts/research/placebo/v1/).
+// =============================================================================
+
+import { makeRandom } from "./selection-risk-core.js";
+import { defaultBlock, stationaryIndices } from "./snooping-core.js";
+
+export const PLACEBO_METHODS = Object.freeze(["permute", "block_permute", "stationary_bootstrap"]);
+// placebo_test's default: the method the pre-registered study selected
+// (config/research/placebo-v1-evaluation.json; a test keeps the two equal).
+export const PLACEBO_DEFAULT_METHOD = "permute";
+
+export const PLACEBO_LIMITS = Object.freeze({
+  min_rows: 30,
+  max_rows: 100000,
+  max_columns: 500,
+  max_cells: 5000000,
+  min_placebos: 19,
+  max_placebos: 199,
+  default_placebos: 19,
+});
+
+export const PLACEBO_LIMITS_TEXT = Object.freeze([
+  "Beating the placebos shows the pipeline finds more in the real order of the returns than in random orders of the same returns; it does not show the edge survives costs, capacity or the future.",
+  "Only the search this pipeline runs is counted: pipelines tried and dropped before it are not.",
+]);
+
+/** A uniform integer in [0, k) from a uniform generator in [0, 1). */
+const below = (next, k) => Math.min(k - 1, Math.floor(next() * k));
+
+/** Fisher-Yates over an Int32Array in place. */
+function shuffle(values, next) {
+  for (let i = values.length - 1; i > 0; i--) {
+    const j = below(next, i + 1);
+    const v = values[i];
+    values[i] = values[j];
+    values[j] = v;
+  }
+  return values;
+}
+
+/**
+ * One reordering of n periods, as the index of the real period each placebo period takes.
+ * permute: every order equally likely. block_permute: the periods cut into consecutive blocks of
+ * `block` (the last may be shorter) and the blocks shuffled, so structure inside a block survives.
+ * stationary_bootstrap: blocks of random length with mean `block`, drawn with replacement.
+ */
+export function placeboOrder(n, { method = PLACEBO_DEFAULT_METHOD, block, next }) {
+  if (!PLACEBO_METHODS.includes(method)) throw new RangeError(`method must be one of ${PLACEBO_METHODS.join(", ")}`);
+  if (method === "permute") return shuffle(Int32Array.from({ length: n }, (_, i) => i), next);
+  const b = block ?? defaultBlock(n);
+  if (!Number.isInteger(b) || b < 1 || b > n) throw new RangeError(`block must be a whole number from 1 to ${n}`);
+  if (method === "stationary_bootstrap") return stationaryIndices(n, b, next);
+  const starts = shuffle(Int32Array.from({ length: Math.ceil(n / b) }, (_, k) => k * b), next);
+  const out = new Int32Array(n);
+  let at = 0;
+  for (const start of starts) for (let t = start; t < Math.min(n, start + b); t++) out[at++] = t;
+  return out;
+}
+
+/**
+ * Checks a panel (columns of equal length) and returns the number of periods the placebo reorders:
+ * the rows of a returns panel, or the returns between rows of a prices panel.
+ */
+export function checkPanel(columns, kind = "prices") {
+  if (kind !== "prices" && kind !== "returns") throw new RangeError('kind must be "prices" or "returns"');
+  if (!Array.isArray(columns) || !columns.length || columns.length > PLACEBO_LIMITS.max_columns) {
+    throw new RangeError(`send 1 to ${PLACEBO_LIMITS.max_columns} columns`);
+  }
+  const rows = columns[0]?.length;
+  if (!Number.isInteger(rows) || rows < PLACEBO_LIMITS.min_rows || rows > PLACEBO_LIMITS.max_rows) {
+    throw new RangeError(`each column needs ${PLACEBO_LIMITS.min_rows} to ${PLACEBO_LIMITS.max_rows} rows`);
+  }
+  if (rows * columns.length > PLACEBO_LIMITS.max_cells) throw new RangeError(`at most ${PLACEBO_LIMITS.max_cells} numbers in all`);
+  columns.forEach((column, c) => {
+    if (!Array.isArray(column) && !ArrayBuffer.isView(column)) throw new RangeError(`column ${c + 1} is not a list of numbers`);
+    if (column.length !== rows) throw new RangeError(`column ${c + 1} has ${column.length} rows; column 1 has ${rows}`);
+    for (let t = 0; t < rows; t++) {
+      const v = column[t];
+      if (typeof v !== "number" || !Number.isFinite(v)) throw new RangeError(`column ${c + 1}, row ${t + 1} is not a finite number: the placebo needs a complete panel`);
+      if (kind === "prices" && v <= 0) throw new RangeError(`column ${c + 1}, row ${t + 1} is not a positive price`);
+    }
+  });
+  return kind === "prices" ? rows - 1 : rows;
+}
+
+/**
+ * A panel reordered by `order`. Returns: placebo row t is real row order[t]. Prices: the placebo
+ * keeps the real first prices, and its return into row t + 1 is the real return into row
+ * order[t] + 1, so a permutation ends every column at its real last price.
+ */
+export function placeboPanel(columns, { kind = "prices", order }) {
+  if (kind === "returns") return columns.map((column) => Array.from(order, (i) => column[i]));
+  return columns.map((column) => {
+    const out = new Array(column.length);
+    out[0] = column[0];
+    for (let t = 1; t < column.length; t++) {
+      const i = order[t - 1] + 1;
+      out[t] = out[t - 1] * (column[i] / column[i - 1]);
+    }
+    return out;
+  });
+}
+
+/** `placebos` placebo panels from one seed, in order, one at a time. */
+export function* placeboPanels(columns, { kind = "prices", method = PLACEBO_DEFAULT_METHOD, block, placebos = PLACEBO_LIMITS.default_placebos, seed = 1 } = {}) {
+  const n = checkPanel(columns, kind);
+  if (!Number.isInteger(placebos) || placebos < PLACEBO_LIMITS.min_placebos || placebos > PLACEBO_LIMITS.max_placebos) {
+    throw new RangeError(`placebos must be a whole number from ${PLACEBO_LIMITS.min_placebos} to ${PLACEBO_LIMITS.max_placebos}`);
+  }
+  const next = makeRandom(Number(seed) >>> 0 || 1);
+  for (let k = 0; k < placebos; k++) yield placeboPanel(columns, { kind, order: placeboOrder(n, { method, block, next }) });
+}
+
+/** The value at nearest rank q (0 < q <= 1) of an ascending list. */
+const nearestRank = (sorted, q) => sorted[Math.max(0, Math.ceil(q * sorted.length) - 1)];
+
+/**
+ * The real result's rank among the placebo results. A placebo result as good as the real one counts
+ * against it, so ties make the p-value larger, never smaller. With K placebos the smallest p-value
+ * is 1 / (K + 1).
+ */
+export function placeboPValue(real, results, { higherIsBetter = true } = {}) {
+  if (typeof real !== "number" || !Number.isFinite(real)) throw new RangeError("real must be a finite number: the pipeline's result on the real data");
+  if (!Array.isArray(results) || results.length < PLACEBO_LIMITS.min_placebos || results.length > PLACEBO_LIMITS.max_placebos) {
+    throw new RangeError(`send ${PLACEBO_LIMITS.min_placebos} to ${PLACEBO_LIMITS.max_placebos} placebo results, one per placebo`);
+  }
+  results.forEach((x, k) => {
+    if (typeof x !== "number" || !Number.isFinite(x)) throw new RangeError(`placebo_results[${k}] is not a finite number: rerun that placebo, or record what the pipeline reports when it finds nothing`);
+  });
+  const atLeast = results.filter((x) => (higherIsBetter ? x >= real : x <= real)).length;
+  const sorted = [...results].sort((a, b) => a - b);
+  const best = higherIsBetter ? sorted[sorted.length - 1] : sorted[0];
+  return {
+    p_value: (1 + atLeast) / (results.length + 1),
+    smallest_possible_p: 1 / (results.length + 1),
+    placebos: results.length,
+    as_good_as_real: atLeast,
+    real,
+    placebo: { min: sorted[0], median: nearestRank(sorted, 0.5), q90: nearestRank(sorted, 0.9), max: sorted[sorted.length - 1], best },
+  };
+}

package/src/schemas.mjs

@@ -7,6 +7,7 @@
 // docs/superpowers/specs/2026-09-05-developer-key-validation-api-design.md section 4 for the
 // validator semantics these shapes mirror.
 import { z } from "zod";
+import { NULL_ZOO_V1 } from "./local/js/null-zoo-v1-sizes.js";
 
 // One description per input field, shared by every shape that uses the field, so a client reads
 // the same meaning, unit and example wherever the field appears. Agents choose arguments from
@@ -331,13 +332,26 @@ export const LIMITS_SENTENCES = Object.freeze({
 export const REGISTRY_LIMITS_CLAUSE = "is not admission to anything and is not a forecast.";
 export const REGISTRY_DESCRIPTION_MAX = 100;
 
+// The headline tests audit_backtest can run, by the names the shipped size table uses. The
+// description reads the chosen ones from that table, so it cannot name a test the audit does not run.
+export const HEADLINE_TEST_NAMES = Object.freeze({
+  reality_check: "White's Reality Check",
+  spa_consistent: "Hansen's SPA",
+  spa_upper: "Hansen's SPA (upper p-value)",
+  luck_trials_lo: "luck trials with Lo's correction",
+});
+const headlineName = (test) => {
+  if (!HEADLINE_TEST_NAMES[test]) throw new Error(`No description name for the headline test ${test}`);
+  return HEADLINE_TEST_NAMES[test];
+};
+
 // One tool description per tool, each stating (in one sentence lifted from the boundary language
 // above) what the tool's result cannot be used to claim, so an agent sees this before it ever
 // calls the tool, not only inside the returned envelope.
 export const TOOL_DESCRIPTIONS = Object.freeze({
   get_key: `Issue a free validation key for this session. Rarely needed: the first validation issues one itself unless CANLI_KEY or local mode is set, and the read tools need none. ${LIMITS_SENTENCES.quotas}`,
   validate_deflated_sharpe: `Deflated Sharpe ratio: the probability (0 to 1) that the selected strategy's Sharpe beats the best that luck gives across the variants tried, with the probabilistic Sharpe and that luck benchmark. Send the seven statistics or a return series. With every variant's returns use validate_overfitting; luck as a trial count, validate_luck_trials; a multiple-testing haircut, validate_haircut_sharpe. ${LIMITS_SENTENCES.notAdmission}`,
-  audit_backtest: `One-call audit of a strategy's returns. Headline: one test whose false-positive rate was measured on nine return shapes (Hansen's SPA with variants). Then deflated Sharpe, minimum track record and, with variants, overfitting and out-of-sample decay, each with its receipt, plus fix_next. Send every variant tried (variants_file) so trials are counted; point returns_file at a CSV instead of pasting. ${LIMITS_SENTENCES.notAdmission}`,
+  audit_backtest: `One-call audit of a strategy's returns. Headline: one test whose false-positive rate was measured on nine return shapes (${headlineName(NULL_ZOO_V1.with_variants.test)} with variants, ${headlineName(NULL_ZOO_V1.single_series.test)} without). Then deflated Sharpe, minimum track record and, with variants, overfitting and out-of-sample decay, each with its receipt, plus fix_next. Send every variant tried (variants_file) so trials are counted; point returns_file at a CSV instead of pasting. ${LIMITS_SENTENCES.notAdmission}`,
   validate_overfitting: `Probability of backtest overfitting (0 to 1) by CSCV: how often the in-sample best variant falls below the out-of-sample median. Needs every variant's returns (periods by variants); with summary statistics only, use validate_deflated_sharpe. ${LIMITS_SENTENCES.notAdmission}`,
   validate_reality_check: `Data-snooping tests on every variant a search tried: Hansen's SPA p-value that the best beat the benchmark only by luck, White's Reality Check, and the variants Romano-Wolf StepM finds better. Send all variants tried, not only the winners. ${LIMITS_SENTENCES.notAdmission}`,
   validate_paper_evidence: `Checks paper-evidence.v0 structure and disclosures. With CANLI_LOCAL=1, record_file reads an export bundle and journal_file verifies signatures, source hashes and recomputed claims. Without a journal, source facts and signatures are unchecked. Local files are never uploaded. ${LIMITS_SENTENCES.scope}`,
@@ -394,3 +408,40 @@ export const statusOutput = z
 export const companyHistoryOutput = z
   .looseObject({ company: loose, claim_boundary: z.string().optional(), source: loose, histories: z.array(z.unknown()).optional(), history: loose, error: refusal })
   .describe("Without a concept, histories lists what is available; with one, history holds the observations newest first as columns and rows, each with its filing. Values are as reported to the SEC.");
+
+// zod writes keywords that constrain nothing a caller sends, and the tool list carrying them is
+// re-sent to the model every turn, so the schemas the server lists leave them out:
+// - JavaScript's safe-integer range (minimum -9007199254740991, maximum 9007199254740991) on every
+//   .int() field, about nine tokens a field;
+// - propertyNames {type: "string"}, which every JSON object key already is;
+// - an empty additionalProperties ({}), which allows anything, as its absence does.
+// A real bound stays, and zod still validates every call. Only subschemas are walked, so a field
+// that happens to be named "minimum" or "additionalProperties" is never touched.
+const SUBSCHEMA_MAPS = ["properties", "patternProperties", "$defs", "dependentSchemas"];
+const SUBSCHEMA_LISTS = ["anyOf", "oneOf", "allOf", "prefixItems"];
+const SUBSCHEMAS = ["items", "additionalProperties", "propertyNames", "contains", "not", "if", "then", "else", "unevaluatedItems", "unevaluatedProperties"];
+const isPlainObject = (v) => v !== null && typeof v === "object" && !Array.isArray(v);
+
+export function leanJsonSchema(schema) {
+  if (!isPlainObject(schema)) return schema;
+  const out = { ...schema };
+  for (const key of SUBSCHEMA_MAPS) {
+    if (isPlainObject(out[key])) out[key] = Object.fromEntries(Object.entries(out[key]).map(([name, sub]) => [name, leanJsonSchema(sub)]));
+  }
+  for (const key of SUBSCHEMA_LISTS) if (Array.isArray(out[key])) out[key] = out[key].map(leanJsonSchema);
+  for (const key of SUBSCHEMAS) if (key in out) out[key] = leanJsonSchema(out[key]);
+  const integer = out.type === "integer" || (Array.isArray(out.type) && out.type.includes("integer"));
+  if (integer && out.minimum === Number.MIN_SAFE_INTEGER) delete out.minimum;
+  if (integer && out.maximum === Number.MAX_SAFE_INTEGER) delete out.maximum;
+  if (isPlainObject(out.propertyNames) && Object.keys(out.propertyNames).length === 1 && out.propertyNames.type === "string") delete out.propertyNames;
+  if (isPlainObject(out.additionalProperties) && Object.keys(out.additionalProperties).length === 0) delete out.additionalProperties;
+  return out;
+}
+
+// The schema a tool is registered with: zod's own validation, and its JSON Schema made lean.
+export function listedSchema(schema) {
+  if (!schema) return schema;
+  const std = schema["~standard"];
+  const lean = (io) => (options) => leanJsonSchema(std.jsonSchema[io](options));
+  return { "~standard": { ...std, jsonSchema: { input: lean("input"), output: lean("output") } } };
+}

package/src/series-file.mjs

@@ -7,15 +7,17 @@
 // Only numbers leave this module. Error messages name rows and columns by position, never by a
 // cell's content or a header's text, so a path pointed at the wrong file cannot echo that file back
 // into the conversation. The hosted endpoint never reads files (see toolAuditBacktest).
-import { closeSync, fstatSync, openSync, readSync } from "node:fs";
+import { closeSync, fstatSync, openSync, readSync, writeFileSync } from "node:fs";
 import { resolve } from "node:path";
 
 export const MAX_SERIES_FILE_BYTES = 5 * 1024 * 1024;
+// check_leakage's columns and placebo_test's panels hold many series at once: a larger cap.
+export const MAX_LARGE_FILE_BYTES = 50 * 1024 * 1024;
 
 // One open file descriptor for the check and the read, so the file checked is the file read: a
 // path swapped between a separate stat and read could otherwise pass the size and type checks as
 // one file and be read as another. The read stops one byte past the cap, whatever the file grows to.
-function readText(path) {
+function readText(path, maxBytes = MAX_SERIES_FILE_BYTES) {
   let fd;
   try {
     fd = openSync(resolve(path), "r");
@@ -25,14 +27,14 @@ function readText(path) {
   try {
     const stat = fstatSync(fd);
     if (!stat.isFile()) throw new Error(`${path}: not a regular file`);
-    if (stat.size > MAX_SERIES_FILE_BYTES) throw new Error(`${path}: larger than ${MAX_SERIES_FILE_BYTES} bytes`);
-    const buffer = Buffer.alloc(MAX_SERIES_FILE_BYTES + 1);
+    if (stat.size > maxBytes) throw new Error(`${path}: larger than ${maxBytes} bytes`);
+    const buffer = Buffer.alloc(maxBytes + 1);
     let length = 0;
     for (;;) {
       const read = readSync(fd, buffer, length, buffer.length - length, null);
       if (read === 0) break;
       length += read;
-      if (length > MAX_SERIES_FILE_BYTES) throw new Error(`${path}: larger than ${MAX_SERIES_FILE_BYTES} bytes`);
+      if (length > maxBytes) throw new Error(`${path}: larger than ${maxBytes} bytes`);
     }
     return buffer.toString("utf8", 0, length);
   } finally {
@@ -178,3 +180,82 @@ export function readMatrixFile(path) {
   if (columns.length < 2) throw new Error(`${path}: needs at least 2 numeric columns, one per variant`);
   return { matrix: table.rows.map((_, i) => columns.map((c) => c.values[i])), skipped };
 }
+
+// check_leakage's columns as the caller's script wrote them: {"columns": {name: {"full": [...],
+// "prefixes": [[...], ...]}}}, with "cuts" and "timestamps" if the script has them. Python's
+// json.dump writes a missing value as NaN, which is not JSON, so NaN reads as null. Every problem
+// is named by position; names leave this module only once every column has the right shape.
+export function readColumnsFile(path) {
+  const text = readText(path, MAX_LARGE_FILE_BYTES).replace(/(?<=[[,:]\s*)NaN(?=\s*[,\]}])/g, "null");
+  let value;
+  try {
+    value = JSON.parse(text);
+  } catch {
+    throw new Error(`${path}: does not parse as JSON (numbers, null for a missing value)`);
+  }
+  const isObject = (v) => v !== null && typeof v === "object" && !Array.isArray(v);
+  if (!isObject(value) || !isObject(value.columns)) throw new Error(`${path}: must be a JSON object {"columns": {name: {"full": [...], "prefixes": [[...], ...]}}}`);
+  Object.entries(value.columns).forEach(([name, column], i) => {
+    if (name.length > 100) throw new Error(`${path}: column ${i + 1} has a name longer than 100 characters`);
+    if (!isObject(column) || !Array.isArray(column.full) || !Array.isArray(column.prefixes) || !column.prefixes.every(Array.isArray)) {
+      throw new Error(`${path}: column ${i + 1} is not {"full": [...], "prefixes": [[...], ...]}`);
+    }
+    [column.full, ...column.prefixes].forEach((run, r) => run.forEach((cell, t) => {
+      if (cell !== null && !(typeof cell === "number" && Number.isFinite(cell))) {
+        throw new Error(`${path}: column ${i + 1}, ${r === 0 ? "full" : `prefixes[${r - 1}]`}[${t}] is not a finite number or null`);
+      }
+    }));
+  });
+  if (value.cuts !== undefined && !(Array.isArray(value.cuts) && value.cuts.every(Number.isInteger))) throw new Error(`${path}: cuts must be a list of whole numbers`);
+  if (value.timestamps !== undefined && !(Array.isArray(value.timestamps) && value.timestamps.every((v) => typeof v === "string" || typeof v === "number"))) {
+    throw new Error(`${path}: timestamps must be a list of dates or numbers`);
+  }
+  return { columns: value.columns, cuts: value.cuts, timestamps: value.timestamps?.map(String) };
+}
+
+// A panel for placebo_test: every numeric column (one per asset) and, when the file has one, its
+// ISO date column. A column with a gap is refused, naming its lines: the placebo reorders whole
+// periods. Header names leave this module only into the placebo files written beside the data,
+// never into an error message.
+export function readPanelFile(path) {
+  const text = readText(path, MAX_LARGE_FILE_BYTES);
+  const table = parseTable(text, path);
+  if (!table.rows.length) throw new Error(`${path}: no data rows`);
+  const { columns, incomplete, skipped } = numericColumns(table);
+  if (incomplete.length) {
+    const c = incomplete[0];
+    throw new Error(`${path}: column ${c.index + 1} holds numbers but is empty or not a number on line${c.gaps.length > 1 ? "s" : ""} ${lineList(c.gaps)}; the placebo reorders whole periods, so fill or remove those rows`);
+  }
+  if (!columns.length) throw new Error(`${path}: no column holds only numbers`);
+  let dateColumn = null;
+  if (typeof table.rows[0][0] !== "number") {
+    for (let j = 0; j < table.rows[0].length && dateColumn === null; j += 1) {
+      if (table.rows.every((row) => ISO_DATE.test(String(row[j]).trim()))) dateColumn = j;
+    }
+  }
+  return {
+    columns: columns.map((c) => c.values),
+    positions: columns.map((c) => c.index + 1),
+    names: table.header ? columns.map((c) => table.header[c.index]) : null,
+    dates: dateColumn === null ? null : table.rows.map((row) => String(row[dateColumn]).trim()),
+    date_name: dateColumn !== null && table.header ? table.header[dateColumn] : null,
+    date_column: dateColumn === null ? null : dateColumn + 1,
+    skipped,
+    format: text.trim().startsWith("[") ? "json" : "csv",
+  };
+}
+
+// Writes a panel in the format it was read from. CSV: a header row when the source had one, then
+// one row per period, its date first when there is one. JSON: a list of numbers for one column, a
+// list of rows for several. Numbers keep every digit. An existing file is never overwritten.
+export function writePanelFile(path, { columns, names, dates, dateName, format = "csv" }) {
+  if (format === "json") {
+    const body = columns.length === 1 ? columns[0] : columns[0].map((_, t) => columns.map((c) => c[t]));
+    writeFileSync(path, `${JSON.stringify(Array.from(body, (v) => (ArrayBuffer.isView(v) ? Array.from(v) : v)))}\n`, { flag: "wx" });
+    return;
+  }
+  const lines = [];
+  if (names) lines.push([...(dates ? [dateName ?? "date"] : []), ...names].join(","));
+  for (let t = 0; t < columns[0].length; t += 1) lines.push([...(dates ? [dates[t]] : []), ...columns.map((c) => String(c[t]))].join(","));
+  writeFileSync(path, `${lines.join("\n")}\n`, { flag: "wx" });
+}

package/src/server.mjs

@@ -22,14 +22,14 @@ import { bestOfTrialsProbability } from "./local/js/luck-core.js";
 import { minimumBacktestLength } from "./local/js/dsr-core.js";
 import { studentTUpper } from "./local/js/student-t.js";
 import { StdioServerTransport } from "@modelcontextprotocol/server/stdio";
-import { breadthInput, trackRecordInput, auditBacktestInput, verifyReceiptToolShape, backtestLengthInput, haircutSharpeInput, luckTrialsInput, auditBacktestToolShape, companyHistoryInput, companyHistoryToolShape, deflatedSharpeInput, deflatedSharpeToolShape, emptyInput, getKeyInput, getReceiptInput, overfittingInput, realityCheckInput, realityCheckToolShape, LIMITS_SENTENCES, paperEvidenceInput, paperEvidenceToolShape, TOOL_DESCRIPTIONS, validationOutput, auditOutput, keyOutput, receiptOutput, verifyReceiptOutput, statusOutput, companyHistoryOutput } from "./schemas.mjs";
+import { breadthInput, trackRecordInput, auditBacktestInput, verifyReceiptToolShape, backtestLengthInput, haircutSharpeInput, luckTrialsInput, auditBacktestToolShape, companyHistoryInput, companyHistoryToolShape, deflatedSharpeInput, deflatedSharpeToolShape, emptyInput, getKeyInput, getReceiptInput, overfittingInput, realityCheckInput, realityCheckToolShape, LIMITS_SENTENCES, paperEvidenceInput, paperEvidenceToolShape, TOOL_DESCRIPTIONS, validationOutput, auditOutput, keyOutput, receiptOutput, verifyReceiptOutput, statusOutput, companyHistoryOutput, listedSchema } from "./schemas.mjs";
 
 export const DEFAULT_BASE = "https://canlicapital.com";
 export const SERVER_NAME = "canlicapital-validation-mcp";
 export const SERVER_VERSION = JSON.parse(readFileSync(new URL("../package.json", import.meta.url), "utf8")).version;
 // Sent once in initialize; clients such as Claude Code put it in the system prompt, so the model
 // knows the first call to make even when tool definitions are deferred. Byte-stable across runs.
-export const SERVER_INSTRUCTIONS = "Checks whether a backtest's result is real. For one strategy's returns, call audit_backtest, pointing returns_file at the backtest's CSV instead of pasting long series. For a Sharpe found by a search, validate_deflated_sharpe needs how many independent variants were tried and how their Sharpes spread; with every variant's returns, validate_overfitting gives the probability of backtest overfitting. To test a rule on prices, backtest_strategy runs a whole parameter grid and validates the best with the count actually run; summarize_series reads a long series in a hundred words; stress_test and check_feasibility ask whether it survives bad markets and a real broker. Set CANLI_LOCAL=1 to compute on this machine with no network. Every result says what it does not establish; quote those limits with the number.";
+export const SERVER_INSTRUCTIONS = "Checks whether a backtest's result is real. For one strategy's returns, call audit_backtest, pointing returns_file at the backtest's CSV instead of pasting long series. For a Sharpe found by a search, validate_deflated_sharpe needs how many independent variants were tried and how their Sharpes spread; with every variant's returns, validate_overfitting gives the probability of backtest overfitting. To test a rule on prices, backtest_strategy runs a whole parameter grid and validates the best with the count actually run; summarize_series reads a long series in a hundred words; stress_test and check_feasibility ask whether it survives bad markets and a real broker. Before trusting a research pipeline, check_leakage finds lookahead in its signals and placebo_test runs it on data where nothing can be predicted. Set CANLI_LOCAL=1 to compute on this machine with no network. Every result says what it does not establish; quote those limits with the number.";
 
 // How the server introduces itself in initialize: a readable title, the page that documents it
 // and its icon, so clients and directories that read serverInfo show more than a package name.
@@ -339,8 +339,9 @@ async function runValidator(session, tool, path, body) {
 const sameJson = (a, b) => JSON.stringify(a) === JSON.stringify(b);
 
 // audit_backtest: one calibrated test first, then the checks behind it.
-//   headline  with every variant's returns, Hansen's SPA (validate_reality_check, with its receipt);
-//             with one series, the single-series test Null Zoo v1 chose. Either way the result quotes
+//   headline  with every variant's returns, the test Null Zoo v1 chose from validate_reality_check's
+//             result (White's Reality Check since 0.12.0), with its receipt; with one series, the
+//             single-series test it chose. Either way the result quotes
 //             that test's measured false-positive rate on returns shaped like these
 //             (local/js/null-zoo-v1-sizes.js) and the worst across the nine families.
 //   checks    the deflated Sharpe (an estimate), the minimum track record and, with variants, CSCV
@@ -881,7 +882,11 @@ export function registerResources(server) {
 
 // Everything the server exposes: the npm package and the hosted endpoint both call this.
 export function registerAll(server, session) {
-  const catalog = registerTools(server, session);
+  // The SDK lists the schemas it is given, so the server registers lean ones (listedSchema), while
+  // registerTools itself hands out the zod schemas: the website's MCP pages read those to check
+  // their example arguments.
+  const listing = { registerTool: (name, config, handler) => server.registerTool(name, { ...config, inputSchema: listedSchema(config.inputSchema), outputSchema: listedSchema(config.outputSchema) }, handler) };
+  const catalog = registerTools(listing, session);
   registerPrompts(server);
   registerLabPrompts(server);
   registerResources(server);