@tangle-network/agent-bench 0.3.6 → 0.3.8

package/src/quant-arena/README.md

@@ -0,0 +1,144 @@
+# QUANT-ARENA — a self-improving trading-strategy lab
+
+A small, fully auditable research loop: an AI strategy author writes candidate strategies, every candidate is screened for look-ahead bias, backtested walk-forward on overlapping in-sample windows against pinned benchmarks, and judged by an acceptance rule whose bar **rises with every strategy tried**.
+Every attempt — accepted, rejected, or killed for leaking — becomes a permanent row in a lab notebook.
+The final two years of data are a locked out-of-sample set that only a separate certification command may touch, once.
+
+Everything is plain TypeScript you can read in an afternoon: the backtester is one file with zero dependencies, the acceptance math is one file, the data is committed CSV.
+
+## 1. Quickstart
+
+```bash
+# from bench/ (needs node >= 20, the `claude` CLI logged in, and `uv` on PATH
+# — scoring runs in a pinned python environment, see "Two engines" below)
+npx tsx src/quant-arena/quant-loop.mts --out /tmp/quant-demo --candidates 2
+```
+
+That runs a full research campaign: 2 strategy authors x 2 candidates each — the committed capture of exactly this command cost $1.65 of model spend across 8 metered calls; backtests are free.
+No API? `--skip-llm-audit` keeps everything but the adversarial code review; the mechanical look-ahead check still runs.
+
+Run the unit tests (backtester hand-computed cases, look-ahead detection, acceptance math, window reproducibility):
+
+```bash
+npx vitest run src/quant-arena
+```
+
+## 2. What happens when you run it
+
+1. **Data loads.** ~8 years of daily bars for 11 tickers (an index `IDX` plus `S01`-`S10`) from `fixtures/data/insample/`. The final 2 years live in `fixtures/data/holdout/` and are **not** loaded — see step 8. The series are synthetic (regime-switching factor model, seeded, regenerable) because the free real-data source we checked licenses personal use only; `fixtures/data/PROVENANCE.md` has the details and how to drop in your own CSVs.
+2. **Evaluation windows are drawn.** 8 overlapping 504-day (~2-year) blocks, block-bootstrap sampled from the in-sample years with a fixed seed — every candidate in the campaign is scored on the same windows, and reruns reproduce bit-identically.
+3. **Benchmarks run.** Three pinned incumbents: buy-and-hold the index, equal-weight monthly rebalance, and a 20/100 moving-average crossover. Their per-window Sharpe ratios define the bar: "best benchmark" is the per-window maximum.
+4. **Strategy authors write code.** Each author is a Claude call with a pinned identity (one plain, one with a quant-researcher system prompt). It gets the strategy contract, the universe summary, the benchmarks' per-window Sharpes, and the current acceptance bar — and must reply with one self-contained TypeScript module exporting `onBar(ctx)`: the harness calls it once per trading day with the bars **up to that day only** (the arrays are physically sliced, so reading the future is structurally impossible), plus the strategy's current holdings and equity, and it answers with target portfolio weights or "hold". Strategies never place orders — a single shared rebalancer (`oms.ts`) turns everyone's target weights into orders under the same sizing rule, LEAN-style `(targetWeight x equity - currentPosition) / price`, long-only. Model spend is metered into a durable cost log (`cost-ledger.jsonl`) with per-call receipts.
+5. **Look-ahead screening, stage 1 (mechanical).** The candidate is re-run on data truncated at several cutoff days. Signals up to each cutoff must be bit-identical to the full-data run — any divergence proves the code read the future, and the candidate is killed with the divergence quoted.
+6. **Look-ahead screening, stage 2 (adversarial).** A second, cheap model reads the source with one job: find look-ahead — indexing past `t`, whole-series statistics feeding per-day decisions, hardcoded dates that smell like memorization. Verdict is JSON; anything but a clean verdict kills the candidate, and an unparseable reply kills it too (the rule fails closed).
+7. **Backtest and verdict.** Survivors are backtested over the whole in-sample period (next-day-open fills, 15 bps one-way costs, no shorting, no leverage) and scored per window. Two engines run: the one-file TypeScript reference engine first, as a fail-closed contract check, then the industry-standard **vectorbt** engine (a persistent python worker in a version-locked environment) produces the official numbers. A parity test suite holds the two engines to agreement on golden fixtures — exact on a no-trade book, within machine precision whenever the book holds cash, and within a documented 0.5% on fully-invested books (the engines differ only in whether fees may be financed by a slightly negative cash balance). The acceptance rule (section 3) decides. Accepted or not, the try is appended to `notebook.jsonl` with its window scores, audit evidence, code hash, and authoring cost.
+8. **Certification, later and by hand.** When you believe a winner, run it once against the untouched final 2 years:
+
+   ```bash
+   npx tsx src/quant-arena/holdout-certify.mts --strategy <path>/strategy.ts --out /tmp/quant-demo
+   ```
+
+   It backtests in-sample + out-of-sample on one axis (so lookbacks are warm), scores only the out-of-sample days against the same three benchmarks, and appends the in-sample vs out-of-sample comparison to the notebook. A second run for the same strategy hash refuses without `--force` — an out-of-sample set answers once; re-rolling it until it agrees turns it into another in-sample set.
+
+## 3. Why the acceptance rule is strict
+
+If you test enough random strategies against the same data, the best one looks brilliant by luck alone.
+Under the assumption of zero skill, the expected best Sharpe among N independent tries grows roughly like sqrt(2 ln N) — try 50 strategies and luck alone buys the winner a substantial edge.
+So the bar a candidate must clear is not fixed: it is `0.10 + 0.15 * sqrt(2 ln N)` of mean excess Sharpe, where N counts **every** candidate ever tried in the campaign, including ones killed for leaking (this is a simplified, auditable version of the Deflated Sharpe Ratio of Bailey & Lopez de Prado, Journal of Portfolio Management, 2014).
+A candidate must ALSO beat the best benchmark in at least 6 of the 8 windows, because one lucky two-year stretch should never carry a decision.
+Every try is a permanent notebook row, so N can never be quietly reset — the price of another shot at the data is a higher bar for everyone after it.
+
+## 4. Reading the notebook
+
+`notebook.jsonl` is append-only JSON lines. Three row types:
+
+- `quant-arena.baselines.v1` — the campaign header: seed, cost assumptions, the 8 windows with dates, and each benchmark's per-window Sharpe.
+- `quant-arena.candidate.v1` — one per try. The fields that matter:
+  - `nTried` — this try's position in the campaign; sets its acceptance bar.
+  - `leakAudit.truncation` / `leakAudit.llm` — both screening verdicts with evidence.
+  - `eval.perWindow` — Sharpe vs best-benchmark Sharpe for each window, with dates.
+  - `verdict` — `accepted`, `rejected-no-edge` (failed the acceptance rule), `rejected-leak`, `rejected-contract` (didn't satisfy the module contract), or `rejected-error`.
+  - `reasons` — the decision spelled out, numbers included.
+- `quant-arena.certification.v1` — the one-shot out-of-sample result, in-sample stats side by side.
+
+A real excerpt from the committed demo campaign (`fixtures/demo-campaign/`): try #4 cleared both look-ahead screens, then lost to the benchmarks in 7 of 8 windows — and after four tries the bar it would have needed had already risen to 0.35 (condensed for width):
+
+```
+{
+  "candidateId": "cand-004-quant-researcher",
+  "proposer": "quant-researcher",
+  "nTried": 4,
+  "leakAudit": {
+    "truncation": {
+      "clean": true
+    },
+    "llm": {
+      "verdict": "clean"
+    }
+  },
+  "eval": {
+    "wins": 1,
+    "requiredWins": 6,
+    "meanExcessSharpe": -0.085,
+    "threshold": 0.35,
+    "perWindow": [
+      {
+        "startDate": "2017-04-06",
+        "endDate": "2019-03-12",
+        "sharpe": 0.94,
+        "bestBaselineSharpe": 1.21,
+        "excess": -0.27
+      },
+      {
+        "startDate": "2018-04-18",
+        "endDate": "2020-03-23",
+        "sharpe": 0.56,
+        "bestBaselineSharpe": 0.61,
+        "excess": -0.05
+      },
+      {
+        "startDate": "2018-08-06",
+        "endDate": "2020-07-09",
+        "sharpe": 0.2,
+        "bestBaselineSharpe": 0.29,
+        "excess": -0.08
+      },
+      "... 5 more windows"
+    ]
+  },
+  "verdict": "rejected-no-edge",
+  "reasons": [
+    "consistency: beat the best baseline in only 1/8 windows (need 6)",
+    "multiplicity: mean excess Sharpe -0.085 < required 0.350 (bar after 4 tried candidates)"
+  ]
+}
+```
+
+## 5. Plugging in your own backtester and data
+
+Scoring goes through one narrow seam: a worker process that takes `{open prices, close prices, target-weight rows, costs, windows}` as JSON lines on stdin and answers `{per-window total return / max drawdown / Sharpe / trade count, full equity curve}` on stdout — see the protocol comment at the top of `python/vbt-worker.py` and the client in `vbt-client.ts`.
+The shipped worker is vectorbt (`Portfolio.from_orders`, target-percent sizing, shared cash, sells before buys), version-locked by `python/pyproject.toml` + `python/uv.lock`; to swap in your own engine, speak the same protocol and keep the fill model (decide at close, fill at next open, bps fees on traded dollars) or re-derive the parity fixtures in `vbt-parity.test.mts` for your model.
+Point the data loader at your own Stooq-format CSVs (one file per ticker, `IDX.csv` as the benchmark asset, an `insample/` and a `holdout/` directory).
+Keep the physical in-sample/out-of-sample split and the once-only certification rule — they are the point, not an implementation detail.
+A third engine is planned but not built: event-driven certification of a winner's order stream through Nautilus Trader (`nautilus-certify.ts` is the named stub).
+
+## Files
+
+| file | what it is |
+| --- | --- |
+| `types.ts` | the strategy contract (v2 `onBar` + the order types), including the no-look-ahead rule |
+| `driver.ts` | the incremental harness: feeds `onBar` day by day with physically truncated history; wraps old batch strategies unchanged |
+| `oms.ts` | the one shared rebalancer: target weights -> orders (strategies never place orders) |
+| `backtest.ts` | the TypeScript reference engine: next-open fills, bps costs, no shorting — zero dependencies; contract prefilter |
+| `vbt-client.ts` + `python/vbt-worker.py` | the official scorer: persistent vectorbt worker, pinned env (`python/uv.lock`), crash-safe request handling |
+| `vbt-parity.test.mts` | the two engines held to agreement on golden fixtures (prints both curves on any disagreement) |
+| `nautilus-certify.ts` | named stub for the planned event-driven certification engine (not implemented) |
+| `windows.ts` | seeded block-bootstrap evaluation windows |
+| `multiplicity.ts` | the rising acceptance bar (documented formula + citation) |
+| `leak-audit.ts` | the mechanical truncation-invariance check |
+| `quant-loop.mts` | the campaign: author -> screen -> backtest -> verdict -> notebook |
+| `holdout-certify.mts` | the once-only out-of-sample certification |
+| `strategies/` | the three pinned benchmarks |
+| `fixtures/data/` | committed daily bars + provenance; `holdout/` is the locked final 2 years |
+| `fixtures/demo-campaign/` | a real captured campaign against the v1 batch contract: notebook, authored strategies, cost receipts |
+| `fixtures/demo-campaign-v2/` | a real captured campaign against the v2 `onBar` contract (1 candidate, honestly rejected: 0/8 windows) |

package/src/quant-arena/backtest.test.mts

@@ -0,0 +1,135 @@
+import { describe, expect, it } from 'vitest'
+import { fillSchedule, runBacktest, statsForRange } from './backtest.ts'
+import type { Bar } from './types.ts'
+
+/** Flat-price bar helper. */
+const bar = (date: string, open: number, close = open): Bar => ({
+  date,
+  open,
+  high: Math.max(open, close),
+  low: Math.min(open, close),
+  close,
+  volume: 1000,
+})
+
+const dates = (n: number): string[] =>
+  Array.from({ length: n }, (_, i) => `2020-01-${String(i + 1).padStart(2, '0')}`)
+
+const flatSeries = (n: number, price: number): Bar[] => dates(n).map((d) => bar(d, price))
+
+const NO_COST = { costBps: 0, slippageBps: 0 }
+
+describe('runBacktest hand-computed toy cases', () => {
+  it('all-cash (no signals) stays at equity 1 with zero trades', () => {
+    const result = runBacktest([flatSeries(5, 100)], [], NO_COST)
+    expect(result.equity).toEqual([1, 1, 1, 1, 1])
+    expect(result.stats.tradeCount).toBe(0)
+    expect(result.stats.totalReturn).toBe(0)
+  })
+
+  it('full weight on a flat ticker loses exactly the round-trip-free fee', () => {
+    // 10bps cost + 5bps slippage on 1.0 traded dollars = 15bps once.
+    const result = runBacktest([flatSeries(5, 100)], [{ t: 0, weights: [1] }], { costBps: 10, slippageBps: 5 })
+    expect(result.stats.tradeCount).toBe(1)
+    expect(result.stats.totalReturn).toBeCloseTo(-0.0015, 12)
+    expect(result.equity[4]).toBeCloseTo(1 - 0.0015, 12)
+  })
+
+  it('captures a known move: buy at open, ride to close', () => {
+    // Day1: open 100 -> close 110 with full weight decided at day0 close.
+    const series = [bar('2020-01-01', 100), bar('2020-01-02', 100, 110), bar('2020-01-03', 110)]
+    const result = runBacktest([series], [{ t: 0, weights: [1] }], NO_COST)
+    expect(result.equity[1]).toBeCloseTo(1.1, 12)
+    expect(result.stats.totalReturn).toBeCloseTo(0.1, 12)
+  })
+
+  it('fills at the NEXT open, not the signal-day close (no free look-ahead)', () => {
+    // Price gaps 100 -> 200 overnight after the signal. A same-close fill would
+    // double money on the gap; a next-open fill must NOT.
+    const series = [bar('2020-01-01', 100, 100), bar('2020-01-02', 200, 200), bar('2020-01-03', 300, 300)]
+    const result = runBacktest([series], [{ t: 0, weights: [1] }], NO_COST)
+    // Bought at 200 (day2 open); close day2 = 200 -> equity 1; day3 300/200 = 1.5.
+    expect(result.equity[1]).toBeCloseTo(1, 12)
+    expect(result.equity[2]).toBeCloseTo(1.5, 12)
+  })
+
+  it('max drawdown on a crafted path: peak 1.2 -> trough 0.9 = 25%', () => {
+    const series = [
+      bar('2020-01-01', 100),
+      bar('2020-01-02', 100, 120),
+      bar('2020-01-03', 120, 90),
+      bar('2020-01-04', 90, 100),
+    ]
+    const result = runBacktest([series], [{ t: 0, weights: [1] }], NO_COST)
+    expect(result.equity).toEqual([1, 1.2, 0.9, 1.0].map((v) => expect.closeTo(v, 12) as unknown as number))
+    expect(result.stats.maxDrawdown).toBeCloseTo((1.2 - 0.9) / 1.2, 12)
+  })
+
+  it('splits weights across two tickers and holds cash for the rest', () => {
+    const a = [bar('2020-01-01', 100), bar('2020-01-02', 100, 110)]
+    const b = [bar('2020-01-01', 50), bar('2020-01-02', 50, 45)]
+    const result = runBacktest([a, b], [{ t: 0, weights: [0.5, 0.25] }], NO_COST)
+    // 0.5 * +10% + 0.25 * -10% + 0.25 cash = 1 + 0.05 - 0.025 = 1.025
+    expect(result.equity[1]).toBeCloseTo(1.025, 12)
+    expect(result.stats.tradeCount).toBe(2)
+  })
+
+  it('is deterministic: identical inputs give identical equity paths', () => {
+    const series = [flatSeries(30, 100).map((b, i) => bar(b.date, 100 + i, 101 + i))]
+    const signals = [
+      { t: 0, weights: [0.7] },
+      { t: 10, weights: [0.2] },
+      { t: 20, weights: [1] },
+    ]
+    const r1 = runBacktest(series as Bar[][], signals, { costBps: 10, slippageBps: 5 })
+    const r2 = runBacktest(series as Bar[][], signals, { costBps: 10, slippageBps: 5 })
+    expect(r1.equity).toEqual(r2.equity)
+    expect(r1.stats).toEqual(r2.stats)
+  })
+
+  it('rejects shorting, leverage, and misaligned universes (fail-closed)', () => {
+    const series = [flatSeries(3, 100)]
+    expect(() => runBacktest(series, [{ t: 0, weights: [-0.1] }], NO_COST)).toThrow(/no shorting/)
+    expect(() => runBacktest(series, [{ t: 0, weights: [1.2] }], NO_COST)).toThrow(/no leverage/)
+    expect(() => runBacktest([flatSeries(3, 100), flatSeries(4, 50)], [], NO_COST)).toThrow(/unaligned/)
+    expect(() => runBacktest(series, [{ t: 5, weights: [1] }], NO_COST)).toThrow(/outside/)
+  })
+
+  it('rebalances ONLY on emitted signals — positions drift in between', () => {
+    // One signal at t=0, then a price runup: the winning position is NOT
+    // trimmed back to its target weight on later days (no hidden churn).
+    const a = [bar('2020-01-01', 100), bar('2020-01-02', 100), bar('2020-01-03', 100, 200), bar('2020-01-04', 200)]
+    const result = runBacktest([a], [{ t: 0, weights: [0.5] }], NO_COST)
+    expect(result.stats.tradeCount).toBe(1)
+    // 0.5 in the ticker doubled -> equity 1.5, still only one fill ever.
+    expect(result.equity[3]).toBeCloseTo(1.5, 12)
+    const schedule = fillSchedule([{ t: 1, weights: [0.5] }], 1, 4)
+    expect(schedule.get(1)).toEqual([0.5])
+    expect(schedule.has(0)).toBe(false)
+  })
+})
+
+describe('statsForRange window slicing', () => {
+  it('scores only the window and carries positions in', () => {
+    // Flat first half, +10% single day in the second half.
+    const series = [
+      bar('2020-01-01', 100),
+      bar('2020-01-02', 100),
+      bar('2020-01-03', 100),
+      bar('2020-01-04', 100, 110),
+      bar('2020-01-05', 110),
+    ]
+    const result = runBacktest([series], [{ t: 0, weights: [1] }], NO_COST)
+    const firstHalf = statsForRange(result, 0, 3)
+    const secondHalf = statsForRange(result, 2, 5)
+    expect(firstHalf.totalReturn).toBeCloseTo(0, 12)
+    expect(secondHalf.totalReturn).toBeCloseTo(0.1, 12)
+    expect(secondHalf.days).toBe(3)
+  })
+
+  it('rejects degenerate ranges', () => {
+    const result = runBacktest([flatSeries(5, 100)], [], NO_COST)
+    expect(() => statsForRange(result, 3, 3)).toThrow(/bad range/)
+    expect(() => statsForRange(result, 0, 99)).toThrow(/bad range/)
+  })
+})

package/src/quant-arena/backtest.ts

@@ -0,0 +1,218 @@
+/**
+ * Daily-bar event-loop backtester. Zero dependencies beyond the Node stdlib
+ * (in fact: zero imports at all except the local types) so a reviewer can
+ * audit the whole execution model in one file.
+ *
+ * Execution model (conservative by construction):
+ *  - A signal decided at the close of day t is FILLED at the open of day t+1.
+ *    Nothing trades on the bar that produced the decision.
+ *  - Every rebalance pays `costBps + slippageBps` (basis points, one-way) on
+ *    the dollars traded, deducted from cash at the fill.
+ *  - No shorting, no leverage: weights >= 0, sum <= 1. Violations THROW —
+ *    a strategy that asks for leverage is a broken candidate, not a clamped one.
+ *  - Cash earns 0. A fully-invested target therefore carries a small negative
+ *    cash balance equal to accumulated fees (fees financed at 0% — this only
+ *    ever understates performance, never flatters it).
+ *
+ * Determinism: pure function of (bars, signals, config). No clock, no RNG.
+ */
+
+import type { Bar, Signal } from './types.ts'
+
+export interface BacktestConfig {
+  /** One-way transaction cost, basis points of traded dollars. */
+  costBps: number
+  /** One-way slippage, basis points of traded dollars. */
+  slippageBps: number
+}
+
+export const TRADING_DAYS_PER_YEAR = 252
+
+export interface RangeStats {
+  /** First/last day index of the scored range (inclusive start, exclusive end). */
+  start: number
+  end: number
+  days: number
+  totalReturn: number
+  maxDrawdown: number
+  /** Annualized Sharpe of daily returns (rf = 0). 0 when volatility is 0. */
+  sharpe: number
+  /** Fills with |trade| > 1e-9 × equity, summed over the range. */
+  tradeCount: number
+  /** Σ traded dollars / equity at each rebalance (two-sided turnover). */
+  turnover: number
+}
+
+export interface BacktestResult {
+  dates: string[]
+  /** Equity at each day's close; equity[0] = 1 (all cash at the first close). */
+  equity: number[]
+  /** dailyReturns[i] = equity[i+1] / equity[i] - 1. */
+  dailyReturns: number[]
+  /** Per-day fill count / turnover fraction (index = fill day). */
+  fillsByDay: number[]
+  turnoverByDay: number[]
+  stats: RangeStats
+}
+
+const WEIGHT_EPS = 1e-9
+
+export function assertAligned(bars: Bar[][]): void {
+  if (bars.length === 0) throw new Error('backtest: empty universe')
+  const T = bars[0]!.length
+  if (T < 2) throw new Error('backtest: need at least 2 days of bars')
+  for (let k = 1; k < bars.length; k++) {
+    if (bars[k]!.length !== T) {
+      throw new Error(`backtest: ticker ${k} has ${bars[k]!.length} bars, ticker 0 has ${T} — unaligned`)
+    }
+    for (let t = 0; t < T; t++) {
+      if (bars[k]![t]!.date !== bars[0]![t]!.date) {
+        throw new Error(`backtest: date mismatch at t=${t}: ${bars[k]![t]!.date} vs ${bars[0]![t]!.date}`)
+      }
+    }
+  }
+}
+
+/** Validate one signal against the contract. Throws on violation. */
+export function assertSignal(signal: Signal, tickers: number, totalDays: number): void {
+  if (!Number.isInteger(signal.t) || signal.t < 0 || signal.t >= totalDays) {
+    throw new Error(`backtest: signal.t=${signal.t} outside [0, ${totalDays})`)
+  }
+  if (signal.weights.length !== tickers) {
+    throw new Error(`backtest: signal at t=${signal.t} has ${signal.weights.length} weights, universe has ${tickers}`)
+  }
+  let sum = 0
+  for (const w of signal.weights) {
+    if (!Number.isFinite(w)) throw new Error(`backtest: non-finite weight at t=${signal.t}`)
+    if (w < -WEIGHT_EPS) throw new Error(`backtest: negative weight ${w} at t=${signal.t} — no shorting`)
+    sum += w
+  }
+  if (sum > 1 + 1e-6) throw new Error(`backtest: weights sum ${sum} > 1 at t=${signal.t} — no leverage`)
+}
+
+/** Map of decision day -> target weights. A rebalance is executed ONLY at the
+ *  open following an emitted signal; between signals, positions drift with
+ *  prices (no silent daily re-targeting, no hidden fee drag). */
+export function fillSchedule(signals: Signal[], tickers: number, totalDays: number): Map<number, number[]> {
+  const sorted = [...signals].sort((a, b) => a.t - b.t)
+  for (let i = 1; i < sorted.length; i++) {
+    if (sorted[i]!.t === sorted[i - 1]!.t) {
+      throw new Error(`backtest: two signals share t=${sorted[i]!.t} — a strategy emits at most one target per day`)
+    }
+  }
+  const schedule = new Map<number, number[]>()
+  for (const s of sorted) {
+    assertSignal(s, tickers, totalDays)
+    schedule.set(s.t, s.weights)
+  }
+  return schedule
+}
+
+export function runBacktest(bars: Bar[][], signals: Signal[], config: BacktestConfig): BacktestResult {
+  assertAligned(bars)
+  const N = bars.length
+  const T = bars[0]!.length
+  const fills = fillSchedule(signals, N, T)
+  const feeRate = (config.costBps + config.slippageBps) / 10_000
+
+  const equity: number[] = new Array(T).fill(0)
+  const fillsByDay: number[] = new Array(T).fill(0)
+  const turnoverByDay: number[] = new Array(T).fill(0)
+  let cash = 1
+  const pos: number[] = new Array(N).fill(0)
+  equity[0] = 1
+
+  for (let t = 1; t < T; t++) {
+    // Overnight: yesterday's close -> today's open.
+    for (let k = 0; k < N; k++) {
+      const prevClose = bars[k]![t - 1]!.close
+      if (prevClose <= 0) throw new Error(`backtest: nonpositive close for ticker ${k} at t=${t - 1}`)
+      pos[k] = pos[k]! * (bars[k]![t]!.open / prevClose)
+    }
+    // Fill the target decided at yesterday's close, at today's open.
+    const target = fills.get(t - 1)
+    if (target !== undefined) {
+      let equityOpen = cash
+      for (let k = 0; k < N; k++) equityOpen += pos[k]!
+      if (equityOpen <= 0) throw new Error(`backtest: equity wiped out at t=${t}`)
+      let traded = 0
+      let fills = 0
+      for (let k = 0; k < N; k++) {
+        const desired = target[k]! * equityOpen
+        const delta = desired - pos[k]!
+        traded += Math.abs(delta)
+        if (Math.abs(delta) > WEIGHT_EPS * equityOpen) fills++
+        cash -= delta
+        pos[k] = desired
+      }
+      const fee = traded * feeRate
+      cash -= fee
+      fillsByDay[t] = fills
+      turnoverByDay[t] = traded / equityOpen
+    }
+    // Intraday: today's open -> today's close.
+    for (let k = 0; k < N; k++) {
+      const open = bars[k]![t]!.open
+      if (open <= 0) throw new Error(`backtest: nonpositive open for ticker ${k} at t=${t}`)
+      pos[k] = pos[k]! * (bars[k]![t]!.close / open)
+    }
+    let eq = cash
+    for (let k = 0; k < N; k++) eq += pos[k]!
+    equity[t] = eq
+  }
+
+  const dailyReturns: number[] = new Array(T - 1)
+  for (let i = 0; i < T - 1; i++) dailyReturns[i] = equity[i + 1]! / equity[i]! - 1
+
+  const result: BacktestResult = {
+    dates: bars[0]!.map((b) => b.date),
+    equity,
+    dailyReturns,
+    fillsByDay,
+    turnoverByDay,
+    stats: undefined as unknown as RangeStats,
+  }
+  result.stats = statsForRange(result, 0, T)
+  return result
+}
+
+/** Score a sub-range [start, end) of an existing backtest — the walk-forward
+ *  window view. Positions carried into the window count; nothing after `end`
+ *  leaks in. */
+export function statsForRange(result: BacktestResult, start: number, end: number): RangeStats {
+  const T = result.equity.length
+  if (!Number.isInteger(start) || !Number.isInteger(end) || start < 0 || end > T || end - start < 2) {
+    throw new Error(`statsForRange: bad range [${start}, ${end}) over ${T} days`)
+  }
+  const eq = result.equity.slice(start, end)
+  const totalReturn = eq[eq.length - 1]! / eq[0]! - 1
+  let peak = eq[0]!
+  let maxDrawdown = 0
+  for (const e of eq) {
+    if (e > peak) peak = e
+    const dd = (peak - e) / peak
+    if (dd > maxDrawdown) maxDrawdown = dd
+  }
+  const returns = result.dailyReturns.slice(start, end - 1)
+  const n = returns.length
+  const mean = returns.reduce((s, r) => s + r, 0) / n
+  const variance = n > 1 ? returns.reduce((s, r) => s + (r - mean) ** 2, 0) / (n - 1) : 0
+  const std = Math.sqrt(variance)
+  const sharpe = std > 0 ? (mean / std) * Math.sqrt(TRADING_DAYS_PER_YEAR) : 0
+  let tradeCount = 0
+  let turnover = 0
+  for (let t = start + 1; t < end; t++) {
+    tradeCount += result.fillsByDay[t]!
+    turnover += result.turnoverByDay[t]!
+  }
+  return {
+    start,
+    end,
+    days: end - start,
+    totalReturn,
+    maxDrawdown,
+    sharpe,
+    tradeCount,
+    turnover,
+  }
+}

package/src/quant-arena/data.test.mts

@@ -0,0 +1,44 @@
+import { describe, expect, it } from 'vitest'
+import { readdir } from 'node:fs/promises'
+import {
+  HOLDOUT_DIR,
+  HOLDOUT_START,
+  IN_SAMPLE_DIR,
+  INDEX_TICKER,
+  loadInSample,
+  parseStooqCsv,
+} from './data.ts'
+
+describe('parseStooqCsv', () => {
+  it('parses the vendored CSV format', () => {
+    const bars = parseStooqCsv('Date,Open,High,Low,Close,Volume\n2020-01-02,10,11,9,10.5,12345\n2020-01-03,10.5,12,10,11,999\n', 'test')
+    expect(bars).toHaveLength(2)
+    expect(bars[0]).toEqual({ date: '2020-01-02', open: 10, high: 11, low: 9, close: 10.5, volume: 12345 })
+  })
+
+  it('fails loud on descending dates, bad prices, and foreign headers', () => {
+    expect(() => parseStooqCsv('Date,Open,High,Low,Close,Volume\n2020-01-03,10,11,9,10,1\n2020-01-02,10,11,9,10,1\n', 't')).toThrow(/ascending/)
+    expect(() => parseStooqCsv('Date,Open,High,Low,Close,Volume\n2020-01-02,0,11,9,10,1\n', 't')).toThrow(/nonpositive/)
+    expect(() => parseStooqCsv('Ticker,Per,Date\nX,D,2020\n', 't')).toThrow(/header/)
+  })
+})
+
+describe('vendored fixture integrity (in-sample side only — the holdout is not read here)', () => {
+  it('in-sample and holdout are physically separate directories with the same tickers', async () => {
+    const inFiles = (await readdir(IN_SAMPLE_DIR)).filter((f) => f.endsWith('.csv')).sort()
+    const outFiles = (await readdir(HOLDOUT_DIR)).filter((f) => f.endsWith('.csv')).sort()
+    expect(inFiles).toEqual(outFiles)
+    expect(inFiles).toContain(`${INDEX_TICKER}.csv`)
+    expect(inFiles.length).toBe(11)
+  })
+
+  it('loadInSample aligns the universe, leads with the index, and never crosses into the holdout era', async () => {
+    const aligned = await loadInSample()
+    expect(aligned.tickers[0]).toBe(INDEX_TICKER)
+    expect(aligned.tickers).toHaveLength(11)
+    expect(aligned.bars).toHaveLength(11)
+    for (const series of aligned.bars) expect(series).toHaveLength(aligned.dates.length)
+    expect(aligned.dates.length).toBeGreaterThan(1800)
+    expect(aligned.dates[aligned.dates.length - 1]! < HOLDOUT_START).toBe(true)
+  })
+})