@coinrithm/mcp-trading 0.7.15 → 0.7.16

package/CHANGELOG.md

@@ -5,11 +5,130 @@ ships two binaries — `coinrithm-mcp` (the MCP server) and `coinrithm-agent` (t
 self-host agent runner) — versioned together. The CoinRithm **API contract** is
 versioned separately (see `openapi.yaml` `info.version`, currently `1.7.0`).
 
-## 0.7.15 (current source, not yet published)
-
-These notes include source changes through 1 October 2026. The frozen
-September 28 draft archives omit later changes and must be replaced with a
-new reviewed artifact set before this complete release is uploaded.
+## 0.7.16 (unpublished)
+
+- Preserve independently timed funding-by-venue context in watchlist and
+  discovered entries, including regular and compact prompts. Keep raw rates,
+  interval normalization and unknown source/next-funding times distinct from
+  settlement funding.
+- Bench reports retain up to two safe route attempts with response source,
+  failure class, actions-string diagnostic and admission categories. Legacy
+  missing evidence remains unknown; error text, latency, prompts and raw model
+  output are excluded. Record explicit truncation for nonstandard providers.
+- Carry open interest, positioning, sampled liquidations, macro proxies,
+  chain TVL, stablecoin supply and sampled order-book depth through the runner
+  with their separate freshness, coverage and provenance limits.
+- Keep news source URLs/categories and every related PM coin in context; add
+  the opt-in per-coin-horizon PM cap. Objective guidance stays within the
+  authored strategy and hard caps. Use the supported MCP SDK 1.31 dependency.
+
+- Bench: the PM baselines now preserve the recorded agent's PM board policy
+  (for house agents: depth 30, churn removed), the original opportunity set.
+  Before, they asked for the uncurated 12-row page a live mechanical agent
+  reads, and a cassette recorded without it (house Mia) reported the market
+  as missing. Originally mechanical recordings keep their uncurated board;
+  live mechanical agents are unchanged. New recordings need only the original
+  pass and mark PM baseline readiness after successful PM reads, rather than
+  assuming that an enabled PM venue guarantees recorded evidence.
+- Bench: two more PM baselines, `baseline:base-rate` (uninformative 50%) and
+  `baseline:random` (seeded 20-80%), next to `baseline:skip` and
+  `baseline:market`. They replay the same recorded market pass, so existing
+  corpora need no re-recording; futures/spot-only corpora still get skip only.
+- Add `coinrithm-agent label --corpus <dir> [--horizon-hours 24] [--overwrite]`:
+  writes `labels/<cassetteId>.json` price bars from candles published after
+  each cassette's asOf (reads only). It waits for the horizon to elapse, uses
+  the finest bar size the cassette's age still allows, keeps existing PM and
+  funding labels, and writes PM settlement only from the public event
+  verdict (settlement-eligible + "settle" + a provider won/lost outcome
+  result; anything else stays unlabelled). Only
+  bars that close by the horizon are stored, and an existing file for
+  another horizon is reported (`horizon_mismatch`), never kept silently.
+- Bench futures scoring walks only bars that open after asOf and close by
+  the horizon end; a horizon off the bar grid stops at the last full bar
+  before it (`horizonFlooredToBar`), so no price after the horizon is used.
+- Add optional per-agent `signals` thresholds (`rsiOversold`, `rsiOverbought`,
+  `strongMovePct`, `leanMovePct`, `minStrength`) and a `data.indicatorRange`
+  data diet (`1D` 5m bars, `1W` 15m, `1M` 1h, `3M` 4h). They change the setup
+  shortlist AND the gate that decides when the model is called, so personality
+  changes what an agent notices. Absent blocks keep the previous constants and
+  the 1D range exactly. Both are validated fail-closed, strict-linted and
+  `$ref`-able; the prompt names the agent's bar size and RSI levels.
+
+- Every watch entry now carries perpetual open interest when the API serves it
+  (`openInterest`: single-side USD over Bybit and OKX, 1h and 24h change,
+  `asOf`, `stale`), read from the existing `/api/agent/market` call. The prompt
+  explains how to read it with price; it is context, never a trade rule. Older
+  APIs without the block leave entries unchanged.
+
+- Add an agent test bench: `coinrithm-agent record` saves the read inputs of
+  dry-run cycles as cassettes (writes refused, no model call), and
+  `coinrithm-agent bench` replays them through the production runner for
+  several agent variants and repeats, next to skip and market-implied
+  baselines. The deterministic report gives reject codes, action overlap,
+  repeat consistency, seeded paired bootstrap CIs, a calibrated A/A null
+  rate, a chronological holdout and optional label scoring. Quotes are
+  synthesized and counted; see "Bench" in docs/agent-runner.md for the limits.
+
+- Add an optional `universe` block (market boundaries): a market-cap rank
+  band, a minimum 24h volume, stablecoins in or out, curated sectors and chain
+  ecosystems in or out, and the sort that defines an opportunity. Each cycle
+  the runner asks `GET /api/agent/universe` for rows inside those boundaries,
+  resolves the top `resolveTop` (max 10) into full `discovered` watch entries
+  and passes the rest as context. Rows are re-checked client-side, so a server
+  can narrow the set but never widen it; the blocklist still wins and every
+  cap applies. A failed screener degrades to the watchlist only. The block
+  can live in its own file via `$ref`. Without it, `universe_scan` keeps the
+  previous top-gainers scan exactly. Needs backend-v2 with the screener.
+
+- Add three optional, runner-enforced prediction-market policy fields, so
+  numbers that strategies stated only in prose become executable. Absent keeps
+  the previous behaviour exactly; malformed values fail closed.
+  - `risk.pmMinEdgeGapPct` (0..100): the forecast must beat the fee-inclusive
+    cost by that share of the room left to 100 (16 means cost 50 needs 58). It
+    never lowers the global minimum edge, and with it set a `pm_open` without
+    `forecastProbability` is rejected (`pm_forecast_required`,
+    `pm_edge_below_gap_rule`). Mechanical benchmarks are exempt.
+  - `risk.pmMaxEdgePoints` (0..100): overconfidence guard; the forecast may
+    beat the fee-inclusive cost by at most that many points, and a forecast
+    becomes mandatory (`pm_edge_overconfident`). This is an optional action
+    constraint; its effect on future performance requires separate evaluation.
+  - `risk.pmMaxOpenPerEvent` (1..50): open bets per event (source + slug),
+    counting held positions and opens accepted this cycle (`pm_event_cap`).
+  - `risk.pmMaxOpenPerCoinHorizon` (1..50): open bets on the same directly
+    linked coin whose events end within 24 h of each other, across different
+    events, counting held positions and opens accepted this cycle
+    (`pm_coin_horizon_cap`; a value outside 1..50 fails closed). Every linked
+    coin is kept. Reads the additive discover `relatedCoins` and
+    `/positions/pm` `relatedCoins` / `eventEndDate`; a missing coin, or an end
+    that is missing, past or over 5 years out, never blocks. A count cap, not
+    a correlation model, over the open positions the feed returns (at most
+    200, newest first).
+  - `risk.pmMinMinutesToClose` (0..10080): reject a market whose known close is
+    nearer than that to the later of `observation.asOf` and the validation
+    clock after the model call (`pm_closes_too_soon`). An unknown or
+    unparseable close never blocks.
+
+- Keep repeated provider 404/410 availability failures out of the generic model
+  failure kill-switch. BYO and self-hosted agents retain their configured model
+  and provider holds without entering a disable/revive loop. Transient failures
+  and invalid decisions still count toward the existing kill-switch.
+- Keep rejected model text out of retained parser errors: JSON syntax errors
+  omit input excerpts, and schema errors omit rejected enum values and unknown
+  field names. Useful field/type diagnostics remain; validation is unchanged.
+- Reject OpenAI-compatible and Anthropic responses that explicitly end at the
+  output token limit, and Anthropic responses stopped at the context-window
+  limit, even if their partial decision text happens to be valid JSON. No action
+  from that incomplete response reaches execution. Preserve reported token usage
+  and the hosted router's existing bounded malformed-response fallback. A
+  truncated direct-provider retry also retains its malformed classification.
+- Classify rejected actions strings with bounded, fixed diagnostic categories.
+  JSON5 is used only for diagnostics; string-valued actions remain rejected.
+  No raw model output is retained by this diagnostic path.
+
+## 0.7.15 (published 2026-10-02)
+
+The verified release includes source changes through 1 October 2026. Its public
+archive and MCP Registry delivery are recorded in the repository publishing log.
 
 - PM board rows carry event settlement context from discovery's optional
   `resolution`. Where `scope` is `per_outcome`, each row also carries its own
@@ -203,9 +322,9 @@ Source changes following the published 0.7.9 release. The TypeScript SDK remains
 - Pin workflow actions and verify release-tool checksums. Add installed-package
   compatibility and restart smoke checks across operating systems and runtimes.
 
-0.7.10 was published and deployed on 2026-09-15. Registry and GitHub downloads
-matched the CI-tested archive; hosted MCP and scheduler deployments finished.
-See the [release record](https://github.com/CoinRithm/coinrithm-agent-trading/releases/tag/mcp-trading-v0.7.10).
+  0.7.10 was published and deployed on 2026-09-15. Registry and GitHub downloads
+  matched the CI-tested archive; hosted MCP and scheduler deployments finished.
+  See the [release record](https://github.com/CoinRithm/coinrithm-agent-trading/releases/tag/mcp-trading-v0.7.10).
 
 ## 0.7.9 - 2026-09-15
 

package/README.md

@@ -44,9 +44,24 @@ This package ships two binaries:
 
 > **Paper trading only** — virtual funds (50,000 mUSD). Not financial advice.
 
-## Version 0.7.15
+## Version 0.7.16 (unpublished)
 
-This release preserves observed-price evidence in public market summaries
+The runner rejects explicitly incomplete provider responses before execution,
+including valid-looking prefixes, and preserves their token usage. Bounded
+shape diagnostics still reject string-valued actions. These guards do not raise
+completion limits, repair model proposals or change customer strategies.
+
+This release also includes configurable universe and signal boundaries, PM
+policies, objective guidance, separately timed market context and the
+`record`/`bench`/`label` evaluation tools. Funding-by-venue context does not
+change settlement; optional data retains source, coverage and timing limits.
+Benchmarks preserve missing outcomes and bounded route diagnostics. They do not
+establish a profitable strategy or a superior model. See the changelog and
+[runner documentation](https://github.com/CoinRithm/coinrithm-agent-trading/blob/main/docs/agent-runner.md).
+
+The preceding published version, 0.7.15, includes the following capabilities.
+
+It preserves observed-price evidence in public market summaries
 (`hasObservedPrice` and bounded `sourceObservation`, including both sides of
 venue comparisons), keeps full public `walletAddress` values in whale summaries
 for `pm_data_whale_wallet` follow-up, adds the opt-in `whale_context` runner
@@ -106,7 +121,7 @@ summaries to the model observation for non-mechanical providers. The read is bes
 availability status; it never expands PM candidates or bypasses risk and write
 gates. Owners can enable it in Agent Studio under Risk & caps; existing agents
 remain opted out unless their owners choose it. It ships in package version
-0.7.15; published MCP `0.7.14` does not include it.
+0.7.15; earlier MCP `0.7.14` does not include it.
 The CoinRithm hosted scheduler runs this same engine for you — see the
 [scheduler README](https://github.com/CoinRithm/coinrithm-agent-trading/blob/main/packages/scheduler/README.md) for the built,
 DB-driven runtime.

package/dist/agent/bench/bench.d.ts

@@ -0,0 +1,59 @@
+import { type ObserveOptions } from "../observe.js";
+import { Provider } from "../providers.js";
+import { AgentSpec, ProposedAction } from "../types.js";
+import { Cassette } from "./cassette.js";
+import type { BenchmarkStrategy } from "../mechanical.js";
+import { LabelFile } from "./labels.js";
+import { BenchReport, CycleRow } from "./report.js";
+export declare const BASELINE_SKIP = "baseline:skip";
+export declare const BASELINE_MARKET = "baseline:market";
+export declare const BASELINE_BASE_RATE = "baseline:base-rate";
+export declare const BASELINE_RANDOM = "baseline:random";
+/**
+ * Mechanical PM baselines, one deterministic run per cassette each: the
+ * market's own probability, an uninformative 50, and a seeded random
+ * forecast (mechanical.ts). They need the recorded PM observation, so a
+ * futures/spot-only corpus has only baseline:skip.
+ */
+export declare const MECHANICAL_BASELINES: ReadonlyArray<readonly [name: string, strategy: BenchmarkStrategy]>;
+export declare const DEFAULT_REPEATS = 3;
+export declare const MAX_REPEATS = 50;
+export declare const DEFAULT_SEED = 20261007;
+export declare const DEFAULT_BOOTSTRAP_RESAMPLES = 2000;
+export interface BenchVariant {
+    name: string;
+    spec: AgentSpec;
+    mergedProse: string;
+    provider: Provider;
+}
+export interface RunBenchOptions {
+    cassettes: Cassette[];
+    variants: BenchVariant[];
+    repeats?: number;
+    labels?: Record<string, LabelFile>;
+    seed?: number;
+    bootstrapResamples?: number;
+    /** Include the skip and market-implied baselines (default true). */
+    baselines?: boolean;
+    log?: (line: string) => void;
+}
+/**
+ * Run `fn` with Date.now() reading `clockMs` plus the real time elapsed since
+ * the call. Restored afterwards even when `fn` throws. `new Date()` with no
+ * argument is NOT shifted (it does not call Date.now); in the runner it only
+ * stamps the whale-context fetchedAt and fallback asOf values.
+ */
+export declare function withReplayClock<T>(clockMs: number, fn: () => Promise<T>): Promise<T>;
+/**
+ * The identity of a move WITHOUT its size, for action-set overlap and repeat
+ * consistency: did the variant do the same thing, not the same amount.
+ */
+export declare function actionKey(action: ProposedAction): string;
+/** Replay one cassette for one variant once, through the real runCycle. */
+export declare function replayCycle(cassette: Cassette, variant: string, repeat: number, spec: AgentSpec, mergedProse: string, provider: Provider, labels?: LabelFile, observeOptions?: ObserveOptions): Promise<CycleRow>;
+/**
+ * Every modelling assumption behind the numbers, written into each report so
+ * a reader never has to guess what a "stop" or a "cost" means here.
+ */
+export declare function benchAssumptions(): Record<string, unknown>;
+export declare function runBench(opts: RunBenchOptions): Promise<BenchReport>;

package/dist/agent/bench/bench.js

@@ -0,0 +1,319 @@
+// The bench: run variants of an agent on the SAME recorded inputs, K times
+// each, next to built-in baselines, and report honest paired statistics.
+//
+// Every replayed cycle goes through the production runCycle in DRY-RUN with a
+// ReplayClient, so prompt building, parsing, ref resolution, capital sizing,
+// validation and every reject code are exactly the shipped code paths. There
+// is no bench-only decision logic to drift from production.
+//
+// Each cycle starts from a fresh run state (the same state recording used),
+// so daily counters, debounce and journal can never leak between cassettes or
+// repeats. The consequence: replay shows how a variant decides on a first
+// look at a recorded market, not how it behaves deep into a running streak.
+//
+// Clock: Date.now() reads the recording's clock plus the real time elapsed in
+// this cycle, so freshness and entry-predicate age checks see the age the data
+// had when recorded (plus the variant's own model latency, as in production).
+import { buildAgentDefinitionSnapshot } from "../definitionSnapshot.js";
+import { clearPmCalibrationCache } from "../observe.js";
+import { runCycle } from "../runner.js";
+import { newState } from "../state.js";
+import { mechanicalBaselineSpec } from "./cassette.js";
+import { FUTURES_SYNTHETIC_FEE_BPS, PM_SYNTHETIC_FEE_RATE_AT_MID, SPOT_SYNTHETIC_FEE_BPS, } from "./costs.js";
+import { scoreAction } from "./labels.js";
+import { BENCH_RUN_ID, SKIP_PROVIDER } from "./recordingClient.js";
+import { ReplayClient } from "./replayClient.js";
+import { buildReport, chronological, } from "./report.js";
+export const BASELINE_SKIP = "baseline:skip";
+export const BASELINE_MARKET = "baseline:market";
+export const BASELINE_BASE_RATE = "baseline:base-rate";
+export const BASELINE_RANDOM = "baseline:random";
+/**
+ * Mechanical PM baselines, one deterministic run per cassette each: the
+ * market's own probability, an uninformative 50, and a seeded random
+ * forecast (mechanical.ts). They need the recorded PM observation, so a
+ * futures/spot-only corpus has only baseline:skip.
+ */
+export const MECHANICAL_BASELINES = [
+    [BASELINE_MARKET, "market-implied"],
+    [BASELINE_BASE_RATE, "base-rate"],
+    [BASELINE_RANDOM, "random"],
+];
+export const DEFAULT_REPEATS = 3;
+export const MAX_REPEATS = 50;
+export const DEFAULT_SEED = 20261007;
+export const DEFAULT_BOOTSTRAP_RESAMPLES = 2000;
+const VARIANT_NAME_RE = /^[A-Za-z0-9][A-Za-z0-9_.-]{0,39}$/;
+/**
+ * Run `fn` with Date.now() reading `clockMs` plus the real time elapsed since
+ * the call. Restored afterwards even when `fn` throws. `new Date()` with no
+ * argument is NOT shifted (it does not call Date.now); in the runner it only
+ * stamps the whale-context fetchedAt and fallback asOf values.
+ */
+export async function withReplayClock(clockMs, fn) {
+    const realNow = Date.now;
+    const offset = clockMs - realNow();
+    Date.now = () => realNow() + offset;
+    try {
+        return await fn();
+    }
+    finally {
+        Date.now = realNow;
+    }
+}
+/**
+ * The identity of a move WITHOUT its size, for action-set overlap and repeat
+ * consistency: did the variant do the same thing, not the same amount.
+ */
+export function actionKey(action) {
+    const lower = (s) => (s ?? "").toLowerCase();
+    const upper = (s) => (s ?? "").toUpperCase();
+    switch (action.type) {
+        case "pm_open":
+            return `pm_open:${lower(action.source)}/${lower(action.slug)}/${action.outcomeExternalMarketId ?? action.ref ?? ""}`;
+        case "futures_open":
+            return `futures_open:${upper(action.symbol)}:${action.side}`;
+        case "futures_close":
+            return `futures_close:${action.positionId}`;
+        case "futures_set_sltp":
+            return `futures_set_sltp:${action.positionId}`;
+        case "spot_order":
+            return `spot_order:${upper(action.symbol)}:${action.side}:${action.orderType}`;
+        case "spot_cancel":
+            return `spot_cancel:${action.orderId}`;
+    }
+}
+function sizeOf(action) {
+    if (action.type === "pm_open")
+        return action.stakeMusd;
+    if (action.type === "futures_open")
+        return action.marginMusd;
+    return undefined;
+}
+function toRow(cassette, variant, repeat, result, runtimeError, client, labels) {
+    const actions = (result?.planned ?? []).map((p) => {
+        const forecast = p.action
+            .forecastProbability;
+        return {
+            key: actionKey(p.action),
+            type: p.action.type,
+            accepted: p.accepted,
+            ...(p.accepted ? {} : { code: p.code ?? "unknown" }),
+            ...(p.quote ? { quote: "synthesized" } : {}),
+            ...(typeof forecast === "number"
+                ? { forecastProbability: forecast }
+                : {}),
+            ...(p.quote?.entryProbability !== undefined
+                ? { entryProbability: p.quote.entryProbability }
+                : {}),
+            ...(sizeOf(p.action) !== undefined ? { sizeMusd: sizeOf(p.action) } : {}),
+            ...(p.accepted
+                ? {
+                    score: scoreAction({
+                        action: p.action,
+                        quote: p.quote,
+                        asOf: cassette.asOf,
+                        labels,
+                    }),
+                }
+                : {}),
+        };
+    });
+    const decisionType = runtimeError
+        ? "runtime_error"
+        : (result?.decisionType ?? (result?.decision === "act" ? "act" : "skip"));
+    return {
+        cassetteId: cassette.id,
+        asOf: cassette.asOf,
+        variant,
+        repeat,
+        decision: result?.decision ?? "skip",
+        decisionType,
+        modelFailed: result?.modelFailed === true,
+        llmCallMade: result?.llmCallMade === true,
+        ...(result?.routeAttempts
+            ? {
+                routeAttempts: result.routeAttempts.slice(0, 2).map((attempt) => ({
+                    provider: attempt.provider,
+                    model: attempt.model,
+                    outcome: attempt.outcome,
+                    ...(attempt.failureClass
+                        ? { failureClass: attempt.failureClass }
+                        : {}),
+                    ...(attempt.status !== undefined ? { status: attempt.status } : {}),
+                    ...(attempt.responseSource
+                        ? { responseSource: attempt.responseSource }
+                        : {}),
+                    ...(attempt.actionsStringDiagnostic
+                        ? { actionsStringDiagnostic: attempt.actionsStringDiagnostic }
+                        : {}),
+                    ...(attempt.admissionReasons
+                        ? { admissionReasons: [...attempt.admissionReasons] }
+                        : {}),
+                })),
+                ...(result.routeAttempts.length > 2
+                    ? { routeAttemptsTruncated: true }
+                    : {}),
+            }
+            : {}),
+        ...(result?.triggerCodes ? { triggerCodes: [...result.triggerCodes] } : {}),
+        ...(result?.skipReason
+            ? { skipReason: result.skipReason.slice(0, 200) }
+            : {}),
+        ...(runtimeError ? { runtimeError: runtimeError.slice(0, 200) } : {}),
+        actions,
+        missingInputs: [...new Set(client.stats.missing)].sort(),
+        synthesizedQuotes: client.stats.synthesizedQuotes.length,
+        refusedWrites: [...client.stats.refusedWrites],
+    };
+}
+/** Replay one cassette for one variant once, through the real runCycle. */
+export async function replayCycle(cassette, variant, repeat, spec, mergedProse, provider, labels, observeOptions) {
+    clearPmCalibrationCache();
+    const client = new ReplayClient(cassette);
+    let result;
+    let runtimeError;
+    try {
+        result = await withReplayClock(cassette.clockMs, () => runCycle({
+            client,
+            provider,
+            spec,
+            mergedProse,
+            state: newState(BENCH_RUN_ID),
+            live: false,
+            ...(observeOptions ? { observeOptions } : {}),
+        }));
+    }
+    catch (error) {
+        // A crash is a result for this variant, never a reason to drop the cycle.
+        runtimeError = error instanceof Error ? error.message : String(error);
+    }
+    return toRow(cassette, variant, repeat, result, runtimeError, client, labels);
+}
+/** The always-skip baseline: no reads, no model, no action, by definition. */
+function skipBaselineRow(cassette) {
+    return {
+        cassetteId: cassette.id,
+        asOf: cassette.asOf,
+        variant: BASELINE_SKIP,
+        repeat: 0,
+        decision: "skip",
+        decisionType: "skip",
+        modelFailed: false,
+        llmCallMade: false,
+        skipReason: "baseline: always skip",
+        actions: [],
+        missingInputs: [],
+        synthesizedQuotes: 0,
+        refusedWrites: [],
+    };
+}
+/**
+ * Every modelling assumption behind the numbers, written into each report so
+ * a reader never has to guess what a "stop" or a "cost" means here.
+ */
+export function benchAssumptions() {
+    return {
+        quotes: "every quote is synthesized from recorded evidence (PM: the discover probability of the chosen outcome; futures/spot: the recorded market price); none comes from the paper engine",
+        entry: "entry at the recorded (synthesized) quote with the paper fee model below; entry latency is not modelled",
+        pmFee: {
+            constant: "PM_SYNTHETIC_FEE_RATE_AT_MID",
+            value: PM_SYNTHETIC_FEE_RATE_AT_MID,
+            model: "fee = rate x 4p(1-p) of the price paid (openapi Paper Execution Realism v1 shape: ~1.8% near 50%, ~0 at the extremes); ask spread and size slippage are not modelled, so PM cost is optimistic by an unmeasured amount",
+        },
+        futuresFee: {
+            constant: "FUTURES_SYNTHETIC_FEE_BPS",
+            value: FUTURES_SYNTHETIC_FEE_BPS,
+            model: "flat taker fee on notional at entry and at exit (the runner's conservative pre-quote estimate); spread and slippage not modelled",
+        },
+        spotFee: {
+            constant: "SPOT_SYNTHETIC_FEE_BPS",
+            value: SPOT_SYNTHETIC_FEE_BPS,
+        },
+        pmEntryFloor: "the synthesized quote does not apply the server floor; the runner validator rejects the same action as pm_entry_below_floor (production: quote_ineligible)",
+        pmOutcome: "settled 0/1 from the label file; cost = stake / net shares (fee-inclusive); Brier uses the forecast the agent sent",
+        futuresOutcome: "an OHLC walk MODEL over label bars that open after asOf; it is not exact fill or stop behaviour",
+        sameBarStopAndTarget: "a bar touching both stop and take-profit is resolved as the STOP (conservative) and counted in labelled.futuresSameBarStopFirst",
+        missingBars: "a missing bar before the exit (or before the horizon end) gives outcome unlabelled_gap: counted, never filled",
+        funding: "included only when the label file carries funding events for the symbol (labelled.futuresFundingIncluded); otherwise excluded",
+        liquidation: "approximated at entry x (1 -/+ 1/leverage); forfeits the margin with no extra fill cost",
+        state: "fresh run state per cycle: no journal, theses, debounce or daily counters carried between cassettes or repeats",
+        clock: "Date.now = recording clock + real elapsed time, so data ages as recorded plus model latency",
+        outcomeCutoff: "asOf is the end of all recording reads; it is not the earlier trades sync cursor",
+        pnlComparisons: "zero means a complete no-action cycle; cycles with missing inputs, runtime errors, or any unscored accepted action are excluded from paired PnL, with exclusion counts shown; all repeats of a cassette must qualify",
+        inference: "per-cassette bootstrap assumes independent observations; overlapping market windows can violate that assumption; repeated A/A calibration does not establish independence or future performance",
+    };
+}
+export async function runBench(opts) {
+    const repeats = opts.repeats ?? DEFAULT_REPEATS;
+    if (!Number.isInteger(repeats) || repeats < 1 || repeats > MAX_REPEATS)
+        throw new Error(`repeats must be an integer from 1 to ${MAX_REPEATS}`);
+    if (opts.variants.length === 0)
+        throw new Error("at least one variant is required");
+    const names = new Set();
+    for (const v of opts.variants) {
+        if (!VARIANT_NAME_RE.test(v.name))
+            throw new Error(`variant name "${v.name}" must be 1-40 letters, digits, "_", "." or "-"`);
+        if (names.has(v.name))
+            throw new Error(`duplicate variant "${v.name}"`);
+        names.add(v.name);
+    }
+    if (opts.cassettes.length === 0)
+        throw new Error("the corpus has no cassettes");
+    const seed = (opts.seed ?? DEFAULT_SEED) >>> 0;
+    const resamples = opts.bootstrapResamples ?? DEFAULT_BOOTSTRAP_RESAMPLES;
+    const baselines = opts.baselines !== false;
+    const log = opts.log ?? (() => { });
+    const cassettes = chronological(opts.cassettes);
+    const rows = [];
+    for (const cassette of cassettes) {
+        const labels = opts.labels?.[cassette.id];
+        for (const v of opts.variants) {
+            for (let r = 0; r < repeats; r++) {
+                const row = await replayCycle(cassette, v.name, r, v.spec, v.mergedProse, v.provider, labels);
+                rows.push(row);
+                log(`${cassette.id} ${v.name}#${r + 1}: ${row.decisionType}, ${row.actions.filter((a) => a.accepted).length}/${row.actions.length} accepted${row.missingInputs.length ? `, ${row.missingInputs.length} missing input(s)` : ""}`);
+            }
+        }
+        if (baselines) {
+            rows.push(skipBaselineRow(cassette));
+            // Deterministic, so one run per cassette each. Use the recorded agent's
+            // board policy, not the replacement mechanical provider's policy.
+            // Originally mechanical recordings keep their uncurated board.
+            if (cassette.marketBaselineRecorded)
+                for (const [name, strategy] of MECHANICAL_BASELINES)
+                    rows.push(await replayCycle(cassette, name, 0, mechanicalBaselineSpec(cassette.spec, strategy), "", SKIP_PROVIDER, labels, {
+                        curatedPmBoard: cassette.spec.model?.provider !== "mechanical",
+                    }));
+        }
+    }
+    const variants = opts.variants.map((v) => ({
+        name: v.name,
+        kind: "variant",
+        definitionHash: buildAgentDefinitionSnapshot(v.spec, v.mergedProse)
+            .definitionHash,
+    }));
+    if (baselines) {
+        variants.push({ name: BASELINE_SKIP, kind: "baseline" });
+        if (cassettes.some((c) => c.marketBaselineRecorded))
+            for (const [name] of MECHANICAL_BASELINES)
+                variants.push({ name, kind: "baseline" });
+    }
+    const specHashes = [...new Set(cassettes.map((c) => c.agentSpecHash))].sort();
+    return buildReport({
+        cassettes,
+        rows,
+        variants,
+        repeats,
+        seed,
+        bootstrapResamples: resamples,
+        assumptions: benchAssumptions(),
+        fidelity: {
+            recordedAgentDefinitions: specHashes,
+            synthesizedQuotes: rows.reduce((s, r) => s + r.synthesizedQuotes, 0),
+            cyclesWithMissingInputs: rows.filter((r) => r.missingInputs.length > 0)
+                .length,
+            refusedWrites: rows.reduce((s, r) => s + r.refusedWrites.length, 0),
+        },
+    });
+}

package/dist/agent/bench/cassette.d.ts

@@ -0,0 +1,97 @@
+import { AgentSpec } from "../types.js";
+import type { BenchmarkStrategy } from "../mechanical.js";
+import { LabelFile } from "./labels.js";
+export declare const CASSETTE_SCHEMA = "coinrithm.bench.cassette.v1";
+export interface RecordedResponse {
+    /** Canonical request key: METHOD + path + sorted query. */
+    key: string;
+    method: string;
+    path: string;
+    query: Record<string, string>;
+    status: number;
+    ok: boolean;
+    /** The body as the client parses it: JSON when it parses, else the text. */
+    data: unknown;
+    /** The recorded request failed in transport (no HTTP response at all). */
+    transportError?: boolean;
+}
+export interface Cassette {
+    schema: typeof CASSETTE_SCHEMA;
+    id: string;
+    recordedAt: string;
+    /** Epoch ms the replay clock starts from (Date.now during replay). */
+    clockMs: number;
+    /** Cutoff after ALL recording reads, including baseline-only inputs. */
+    asOf: string;
+    /** definitionHash of the recorded spec + prose (see definitionSnapshot). */
+    agentSpecHash: string;
+    spec: AgentSpec;
+    /** PM observation available for baselines. Legacy recordings used this
+     *  only to mark an attempted extra mechanical pass; replay still reports
+     *  missing inputs instead of treating the marker as a coverage guarantee. */
+    marketBaselineRecorded: boolean;
+    recordCycle: {
+        decision: string;
+        decisionType?: string;
+        skipReason?: string;
+    };
+    /** Non-GET requests the recorder refused to forward (never sent). */
+    refusedRequests: string[];
+    responses: RecordedResponse[];
+}
+export interface CanonicalRequest {
+    key: string;
+    method: string;
+    path: string;
+    query: Record<string, string>;
+}
+/**
+ * Canonical request identity used for both recording and replay: upper-cased
+ * method, the path relative to the API base path, and the query parameters
+ * sorted by name. The host is deliberately not part of the key, so a cassette
+ * recorded against one base URL replays against the bench's fake one.
+ */
+export declare function canonicalRequest(method: string, url: string, basePath?: string): CanonicalRequest;
+/** The path prefix of an API base URL ("" for a host-root base). */
+export declare function basePathOf(baseUrl: string): string;
+export declare function sha256Hex(text: string): string;
+/** Sorted-key JSON, the same canonicalization the scorecard hash relies on. */
+export declare function canonicalJson(value: unknown): string;
+/**
+ * The spec a recording pass runs with: the agent's own spec with its gate
+ * forced to fire, so the post-gate reads (whale context) are always recorded
+ * and a variant whose gate fires on replay finds them in the cassette.
+ */
+export declare function recordingSpec(spec: AgentSpec): AgentSpec;
+/**
+ * The market-implied baseline: the recorded agent's spec and caps with the
+ * deterministic mechanical "market-implied" strategy as its brain (mechanical
+ * .ts). runBench separately preserves the recorded agent's observation
+ * policy, so replacing the provider never changes the PM opportunity set.
+ */
+export declare function marketBaselineSpec(spec: AgentSpec): AgentSpec;
+/**
+ * The same recorded spec with one of the mechanical benchmark strategies as
+ * its brain. base-rate and random pick the SAME market from the SAME
+ * observation as market-implied (mechanical.ts); all use the original
+ * recording's PM board and never fetch a separate baseline universe.
+ */
+export declare function mechanicalBaselineSpec(spec: AgentSpec, strategy: BenchmarkStrategy): AgentSpec;
+/** Filesystem-safe, content-addressed cassette id: asOf stamp + body hash. */
+export declare function cassetteId(asOf: string, contentHash: string): string;
+export declare function writeCassette(dir: string, cassette: Cassette): string;
+/** Validate a parsed cassette. Throws on a malformed one (fail closed). */
+export declare function parseCassette(value: unknown, source: string): Cassette;
+export interface Corpus {
+    cassettes: Cassette[];
+    labels: Record<string, LabelFile>;
+    /** Top-level .json files that are not cassettes (e.g. a saved report). */
+    ignored: string[];
+}
+/**
+ * Read every cassette at the top level of `dir` plus the optional
+ * `labels/<cassetteId>.json` files. A JSON file with the cassette schema that
+ * fails validation, or a malformed label file, stops the bench: a silently
+ * skipped input would bias the comparison.
+ */
+export declare function readCorpus(dir: string): Corpus;