@tangle-network/agent-eval 0.70.0 → 0.72.0

package/CHANGELOG.md

@@ -4,6 +4,30 @@ All notable changes to `@tangle-network/agent-eval` and its sibling `agent-eval-
 
 ---
 
+## [0.72.0] — 2026-05-31 — cost axis prices unpriced-at-source models (every run carries a real, labeled cost)
+
+A live tax-agent full-loop run (real sandbox, `deepseek-v4-pro`, real tokens) exposed the second root of the cost-ledger split: the sandbox reported `totalCostUsd: 0` despite `17537` input / `622` output tokens — not a stub, not a mis-wired ledger, but a model the **source** can't rate. The cost / Pareto / `tokens_per_dollar` axes blanked even though the substrate's pricing table prices `deepseek` correctly; the table was simply never consulted on the matrix cost projection. A $0 cost on a run that burned real tokens reads as "free," which is the more misleading state.
+
+### Fixed
+
+- **`runProfileMatrix` prices measured tokens when the source reports $0.** Cost precedence is now explicit: **source-billed > token-estimated > none**. When `cell.costUsd === 0` and real output tokens flowed and the model is priced (`isModelPriced`), `buildRunRecord` sets the cost from `estimateCost(in, out, model)` (real published rate × real tokens) and stamps `raw.cost_estimated = 1`. A billed cost is never overridden; a model the table also can't rate stays $0 (no fabrication). The estimate flows into `record.costUsd`, so `byProfile.totalCostUsd`, `integrity.totalCostUsd`, and `tokens_per_dollar` / `cost_per_quality` all populate.
+- **Every cost surface in the matrix result agrees.** The embedded `campaigns[id].aggregates.totalCostUsd` is reconciled to the priced total instead of runCampaign's raw `ctx.cost` ledger (which only sees the source's $0). No more two-`totalCostUsd`-that-disagree in one result.
+- **Honest integrity diagnosis.** `summarizeBackendIntegrity`'s uncosted-records message now names **both** roots — mis-wired ledger OR unpriced-at-source model — and points at `estimateCost` for the latter, instead of asserting the ledger is broken.
+
+Live proof: the same tax case that recorded `$0` now records **`$0.0059453`** (`17537 × 0.0003/1k + 622 × 0.0011/1k`, exact), `cost_estimated: 1`, `uncostedRecords: 0`, verdict `real`. Generalizes to every consumer of `runProfileMatrix`. New regression tests: priced-when-source-zero, billed-takes-precedence, truly-unpriced-stays-$0, campaign-aggregate-reconciled. Full suite (1663) green.
+
+## [0.71.0] — 2026-05-31 — corpus-by-default + multi-dimensional capture (datasets as eval exhaust)
+
+Every matrix run now emits a multi-dimensional, dataset-able record with no side-channel — the groundwork for "datasets gathered for free by running evals."
+
+### Added
+
+- **Multi-dim guardrail projection in `buildRunRecord`.** Each `RunRecord.outcome.raw` carries `cost_usd`, `tokens_input` / `tokens_output` (+ `tokens_cached` when present), `latency_ms`, and the guarded ratios `tokens_per_dollar` / `cost_per_quality`. RAW-ONLY — the composite stays the judge objective (anti-Goodhart); these are tracked + dashboarded + carried into datasets, never optimized.
+- **Corpus-by-default via `corpusText`.** An optional `corpusText(artifact, scenario) => {prompt, completion}` stamps the trajectory text onto each record (the `CorpusRecord` shape), so a run is dataset-able with no side-channel. Fail-soft: a throwing extractor omits the text and keeps the graded record.
+- **`appendToCorpus` / `readCorpus` / `buildDatasetFromCorpus`** (`src/rl/corpus.ts`) — append-only JSONL corpus (deduped by `runId`), with score/split filtering into a train/holdout dataset.
+
+`buildRunRecord` is generic over `<TScenario, TArtifact>`; a `scenarioById` map threads each scenario into the projection.
+
 ## [0.70.0] — 2026-05-31 — error-grounded reflection (the driver targets real failures, not blind rewrites)
 
 Adversarial verification on TWO domains (legal + tax, two worker models) found the same root cause: the gepaDriver's candidates **regressed** the baseline, so the gate correctly held — but nothing improved. The driver was reflecting on per-scenario *scores* only; the judge's `notes` (the "why it failed") were computed but **dropped** before the reflection. So it proposed generic rewrites a capable model already knows, which distract rather than help.

package/dist/adapters/http.js

@@ -95,7 +95,7 @@ async function runDispatchServer(opts) {
         return;
       }
       if (expectedAuth) {
-        const got = req.headers["authorization"];
+        const got = req.headers.authorization;
         if (got !== expectedAuth) {
           res.statusCode = 401;
           res.end("unauthorized");

package/dist/adapters/http.js.map

@@ -1 +1 @@
-{"version":3,"sources":["../../src/adapters/http.ts"],"sourcesContent":["/**\n * # `@tangle-network/agent-eval/adapters/http` — distributed Dispatch over HTTP.\n *\n * Decouples driver and worker. The driver (running `runImprovementLoop` or\n * `runCampaign`) can live anywhere — your VPC, a dev laptop, a cron VM. The\n * workers (running the actual agent) can live anywhere else — different\n * regions, different clouds, different boxes — as long as they speak HTTP.\n *\n * Both sides:\n *\n *   - **`httpDispatch({ url | resolveUrl, ... })`** — client. Returns a\n *     `Dispatch` that POSTs `{ scenario, ctx }` to a worker URL and parses\n *     the artifact back. AbortSignal-aware, retries on idempotent errors,\n *     bounded timeout per call.\n *   - **`runDispatchServer({ dispatch, port, ... })`** — server. Wraps your\n *     local `Dispatch` as an HTTP endpoint. Handles auth, JSON parsing,\n *     error mapping, and cancellation when the client aborts.\n *\n * # Topology examples\n *\n * **Single-worker:** driver on box A, worker on box B. Set\n * `httpDispatch({ url: 'https://box-b/dispatch' })`.\n *\n * **Multi-region:** N workers across regions. Use `httpDispatch({ resolveUrl })`\n * with a function that picks the URL per cell from `ctx.placement`. Combined\n * with `cellPlacement` on `RunCampaignOptions`, the substrate fans cells\n * across geographies in parallel.\n *\n * **Driver-as-a-service:** driver runs as a long-lived process or service\n * (holds optimization state across generations); workers are stateless\n * HTTP services that can scale horizontally per cell.\n */\n\nimport type { Dispatch, DispatchContext, Scenario } from '../contract'\n\n// ── Client ───────────────────────────────────────────────────────────\n\n// eslint-disable-next-line @typescript-eslint/no-unused-vars -- TArtifact is unused\n//  in this options interface but kept as a parameter so callers can write\n//  `HttpDispatchOptions<MyScenario, MyArtifact>` symmetrically with\n//  `Dispatch<MyScenario, MyArtifact>`. Marking it unused at the position\n//  where it bites.\nexport interface HttpDispatchOptions<TScenario extends Scenario, _TArtifact> {\n  /** Static endpoint URL. Mutually exclusive with `resolveUrl`. */\n  url?: string\n  /**\n   * Dynamic per-cell URL resolver. Receives the scenario + the substrate\n   * placement key (from `RunCampaignOptions.cellPlacement`) and returns the\n   * worker URL to invoke. Mutually exclusive with `url`.\n   */\n  resolveUrl?: (input: { scenario: TScenario; placement?: string; cellId: string }) => string\n  /** Bearer token or static auth string set as `Authorization`. */\n  auth?: string | (() => string | Promise<string>)\n  /** Extra headers merged into every request. */\n  headers?: Record<string, string>\n  /** Per-call timeout in ms. Default 5 minutes. */\n  timeoutMs?: number\n  /** How many idempotent retries on 5xx / network errors. Default 2. */\n  retries?: number\n  /** Optional fetch override (auth wrappers, custom agent, mocks). */\n  fetchImpl?: typeof fetch\n}\n\nexport interface HttpDispatchRequestBody<TScenario extends Scenario> {\n  scenario: TScenario\n  cellId: string\n  rep: number\n  generation?: number\n  seed: number\n  placement?: string\n  cycleId?: string\n}\n\nexport interface HttpDispatchResponseBody<TArtifact> {\n  artifact: TArtifact\n}\n\nfunction resolveAuth(auth: HttpDispatchOptions<Scenario, unknown>['auth']): Promise<string | null> {\n  if (!auth) return Promise.resolve(null)\n  if (typeof auth === 'string') return Promise.resolve(auth)\n  return Promise.resolve(auth())\n}\n\n/**\n * Wrap a remote HTTP endpoint as a `Dispatch`. The remote side should run\n * `runDispatchServer` (or any service that speaks the same wire shape).\n *\n * Cancellation: the substrate's per-cell `AbortSignal` is forwarded; the\n * server's `runDispatchServer` translates the resulting `AbortError` into\n * a 499 (client-closed) so the client doesn't retry.\n */\nexport function httpDispatch<TScenario extends Scenario, TArtifact>(\n  opts: HttpDispatchOptions<TScenario, TArtifact>,\n): Dispatch<TScenario, TArtifact> {\n  if (!opts.url && !opts.resolveUrl) {\n    throw new Error('httpDispatch: pass exactly one of `url` or `resolveUrl`.')\n  }\n  if (opts.url && opts.resolveUrl) {\n    throw new Error('httpDispatch: pass exactly one of `url` or `resolveUrl`, not both.')\n  }\n  const timeoutMs = opts.timeoutMs ?? 5 * 60 * 1000\n  const maxRetries = opts.retries ?? 2\n  const f: typeof fetch = opts.fetchImpl ?? ((...args) => fetch(...args))\n\n  return async (scenario, ctx) => {\n    const url =\n      opts.url ?? opts.resolveUrl!({ scenario, placement: ctx.placement, cellId: ctx.cellId })\n    const authValue = await resolveAuth(opts.auth)\n    const body: HttpDispatchRequestBody<TScenario> = {\n      scenario,\n      cellId: ctx.cellId,\n      rep: ctx.rep,\n      generation: ctx.generation,\n      seed: ctx.seed,\n      placement: ctx.placement,\n      cycleId: ctx.cycleId,\n    }\n\n    let lastError: unknown\n    for (let attempt = 0; attempt <= maxRetries; attempt++) {\n      // Compose the request signal: caller's signal OR our timeout.\n      const ourTimeout = AbortSignal.timeout(timeoutMs)\n      const combinedSignal = AbortSignal.any([ctx.signal, ourTimeout])\n      try {\n        const res = await f(url, {\n          method: 'POST',\n          headers: {\n            'Content-Type': 'application/json',\n            ...(authValue\n              ? {\n                  Authorization: authValue.startsWith('Bearer ')\n                    ? authValue\n                    : `Bearer ${authValue}`,\n                }\n              : {}),\n            ...opts.headers,\n          },\n          body: JSON.stringify(body),\n          signal: combinedSignal,\n        })\n        if (!res.ok) {\n          // 4xx is non-retryable (caller error, auth, bad scenario shape).\n          // 5xx / 408 / 429 / 502 / 503 / 504 are retryable.\n          const retryable = res.status >= 500 || res.status === 408 || res.status === 429\n          if (!retryable || attempt === maxRetries) {\n            const text = await res.text().catch(() => '')\n            throw new Error(`httpDispatch ${url} failed (${res.status}): ${text.slice(0, 500)}`)\n          }\n          // exponential backoff with jitter\n          await sleep(2 ** attempt * 200 + Math.random() * 200)\n          continue\n        }\n        const parsed = (await res.json()) as HttpDispatchResponseBody<TArtifact>\n        return parsed.artifact\n      } catch (err) {\n        // Caller-driven abort is terminal — never retry.\n        if (ctx.signal.aborted) throw err\n        lastError = err\n        if (attempt === maxRetries) throw err\n        await sleep(2 ** attempt * 200 + Math.random() * 200)\n      }\n    }\n    throw lastError ?? new Error('httpDispatch exhausted retries')\n  }\n}\n\nfunction sleep(ms: number): Promise<void> {\n  return new Promise((resolve) => {\n    const t = setTimeout(resolve, ms)\n    // Don't keep node process alive purely for backoff sleeps.\n    if (typeof (t as { unref?: () => void }).unref === 'function')\n      (t as { unref: () => void }).unref()\n  })\n}\n\n// ── Server ───────────────────────────────────────────────────────────\n\nexport interface RunDispatchServerOptions<TScenario extends Scenario, TArtifact> {\n  /** The Dispatch this server exposes — what runs when a request lands. */\n  dispatch: Dispatch<TScenario, TArtifact>\n  /** TCP port to bind. */\n  port: number\n  /** Optional bind host; defaults to 0.0.0.0. */\n  host?: string\n  /** Required for any non-test deployment: the bearer token clients must\n   *  send. The substrate refuses to start without auth unless `auth: false`\n   *  is set explicitly (intended ONLY for closed-network/internal testing). */\n  auth: string | false\n  /** Path the server listens on. Default `/dispatch`. */\n  path?: string\n  /**\n   * Per-request handler that wraps `dispatch` with whatever context the\n   * worker side needs to construct a `DispatchContext` — typically the\n   * trace writer, artifact writer, and cost meter. The substrate provides\n   * synthetic-but-typed defaults if not supplied; production deployments\n   * should wire real ones (e.g. ship traces to your OTel collector).\n   */\n  contextFactory?: (\n    req: HttpDispatchRequestBody<TScenario>,\n    signal: AbortSignal,\n  ) => Promise<DispatchContext>\n  /** Optional max payload size for the request body (bytes). Default 10 MB. */\n  maxBodyBytes?: number\n  /** Hook for observability — called on every successful or failed turn. */\n  onRequest?: (event: {\n    cellId: string\n    durationMs: number\n    success: boolean\n    error?: unknown\n  }) => void\n}\n\nexport interface DispatchServerHandle {\n  /** The actual bound port (useful when `port: 0` requests an ephemeral port). */\n  port: number\n  /** Stop accepting new connections and drain existing ones. */\n  close: () => Promise<void>\n}\n\n/**\n * Start an HTTP server exposing a local `Dispatch` over the wire. Pair with\n * `httpDispatch` on the driver side.\n *\n * Wire shape:\n *\n *   POST /dispatch\n *   Authorization: Bearer <token>\n *   Body: HttpDispatchRequestBody\n *   200 OK: HttpDispatchResponseBody\n *   401: missing/invalid auth\n *   408: per-request timeout exceeded\n *   499: client aborted before completion\n *   500: dispatch threw\n *\n * The server is `node:http`-based to keep the runtime dependency surface\n * minimal — works in plain Node, sandbox, or any container.\n */\nexport async function runDispatchServer<TScenario extends Scenario, TArtifact>(\n  opts: RunDispatchServerOptions<TScenario, TArtifact>,\n): Promise<DispatchServerHandle> {\n  if (opts.auth === undefined) {\n    throw new Error(\n      \"runDispatchServer: 'auth' is required (pass a bearer-token string, or `auth: false` explicitly for a closed-network test deployment).\",\n    )\n  }\n  const path = opts.path ?? '/dispatch'\n  const maxBytes = opts.maxBodyBytes ?? 10 * 1024 * 1024\n  const expectedAuth =\n    typeof opts.auth === 'string' ? `Bearer ${opts.auth.replace(/^Bearer\\s+/, '')}` : null\n\n  // Lazy-import node:http so the file is usable from non-Node bundlers\n  // that import the client side only (e.g. an edge driver shipping\n  // httpDispatch alone). Server side is opt-in by calling this function.\n  const { createServer } = await import('node:http')\n\n  const server = createServer(async (req, res) => {\n    const start = Date.now()\n    let cellId = 'unknown'\n    let success = false\n    let errCaught: unknown\n\n    try {\n      if (req.method !== 'POST' || req.url?.split('?')[0] !== path) {\n        res.statusCode = 404\n        res.end('not found')\n        return\n      }\n      if (expectedAuth) {\n        const got = req.headers['authorization']\n        if (got !== expectedAuth) {\n          res.statusCode = 401\n          res.end('unauthorized')\n          return\n        }\n      }\n\n      // Read body up to maxBytes\n      const chunks: Buffer[] = []\n      let totalBytes = 0\n      const aborter = new AbortController()\n      req.on('close', () => {\n        if (!res.writableEnded) aborter.abort()\n      })\n\n      for await (const chunk of req) {\n        const buf = chunk as Buffer\n        totalBytes += buf.length\n        if (totalBytes > maxBytes) {\n          res.statusCode = 413\n          res.end('payload too large')\n          return\n        }\n        chunks.push(buf)\n      }\n\n      const body = JSON.parse(\n        Buffer.concat(chunks).toString('utf8'),\n      ) as HttpDispatchRequestBody<TScenario>\n      cellId = body.cellId\n\n      const ctx: DispatchContext = opts.contextFactory\n        ? await opts.contextFactory(body, aborter.signal)\n        : {\n            cellId: body.cellId,\n            rep: body.rep,\n            generation: body.generation,\n            seed: body.seed,\n            signal: aborter.signal,\n            placement: body.placement,\n            cycleId: body.cycleId,\n            trace: NOOP_TRACE,\n            artifacts: NOOP_ARTIFACTS,\n            cost: NOOP_COST,\n          }\n\n      const artifact = await opts.dispatch(body.scenario, ctx)\n      const responseBody: HttpDispatchResponseBody<TArtifact> = { artifact }\n\n      res.statusCode = 200\n      res.setHeader('content-type', 'application/json')\n      res.end(JSON.stringify(responseBody))\n      success = true\n    } catch (err) {\n      errCaught = err\n      // Client-cancelled — they don't care about the result.\n      if ((err as Error)?.name === 'AbortError') {\n        res.statusCode = 499\n        res.end('client aborted')\n        return\n      }\n      res.statusCode = 500\n      res.setHeader('content-type', 'application/json')\n      res.end(JSON.stringify({ error: err instanceof Error ? err.message : String(err) }))\n    } finally {\n      opts.onRequest?.({\n        cellId,\n        durationMs: Date.now() - start,\n        success,\n        error: errCaught,\n      })\n    }\n  })\n\n  await new Promise<void>((resolve, reject) => {\n    server.once('error', reject)\n    server.listen(opts.port, opts.host ?? '0.0.0.0', () => resolve())\n  })\n\n  const addr = server.address()\n  const boundPort = typeof addr === 'object' && addr ? addr.port : opts.port\n\n  return {\n    port: boundPort,\n    close: () =>\n      new Promise<void>((resolve, reject) => {\n        server.close((err) => (err ? reject(err) : resolve()))\n      }),\n  }\n}\n\n// ── No-op default ctx machinery (worker can replace via contextFactory) ──\n\nconst NOOP_TRACE = {\n  span: () => ({\n    end: () => {},\n    setAttribute: () => {},\n    setStatus: () => {},\n    recordException: () => {},\n    addEvent: () => {},\n  }),\n} as unknown as DispatchContext['trace']\n\nconst NOOP_ARTIFACTS = {\n  write: async () => undefined,\n  read: async () => undefined,\n  list: async () => [],\n} as unknown as DispatchContext['artifacts']\n\nconst NOOP_COST = {\n  record: () => {},\n  total: () => 0,\n} as unknown as DispatchContext['cost']\n"],"mappings":";;;AA6EA,SAAS,YAAY,MAA8E;AACjG,MAAI,CAAC,KAAM,QAAO,QAAQ,QAAQ,IAAI;AACtC,MAAI,OAAO,SAAS,SAAU,QAAO,QAAQ,QAAQ,IAAI;AACzD,SAAO,QAAQ,QAAQ,KAAK,CAAC;AAC/B;AAUO,SAAS,aACd,MACgC;AAChC,MAAI,CAAC,KAAK,OAAO,CAAC,KAAK,YAAY;AACjC,UAAM,IAAI,MAAM,0DAA0D;AAAA,EAC5E;AACA,MAAI,KAAK,OAAO,KAAK,YAAY;AAC/B,UAAM,IAAI,MAAM,oEAAoE;AAAA,EACtF;AACA,QAAM,YAAY,KAAK,aAAa,IAAI,KAAK;AAC7C,QAAM,aAAa,KAAK,WAAW;AACnC,QAAM,IAAkB,KAAK,cAAc,IAAI,SAAS,MAAM,GAAG,IAAI;AAErE,SAAO,OAAO,UAAU,QAAQ;AAC9B,UAAM,MACJ,KAAK,OAAO,KAAK,WAAY,EAAE,UAAU,WAAW,IAAI,WAAW,QAAQ,IAAI,OAAO,CAAC;AACzF,UAAM,YAAY,MAAM,YAAY,KAAK,IAAI;AAC7C,UAAM,OAA2C;AAAA,MAC/C;AAAA,MACA,QAAQ,IAAI;AAAA,MACZ,KAAK,IAAI;AAAA,MACT,YAAY,IAAI;AAAA,MAChB,MAAM,IAAI;AAAA,MACV,WAAW,IAAI;AAAA,MACf,SAAS,IAAI;AAAA,IACf;AAEA,QAAI;AACJ,aAAS,UAAU,GAAG,WAAW,YAAY,WAAW;AAEtD,YAAM,aAAa,YAAY,QAAQ,SAAS;AAChD,YAAM,iBAAiB,YAAY,IAAI,CAAC,IAAI,QAAQ,UAAU,CAAC;AAC/D,UAAI;AACF,cAAM,MAAM,MAAM,EAAE,KAAK;AAAA,UACvB,QAAQ;AAAA,UACR,SAAS;AAAA,YACP,gBAAgB;AAAA,YAChB,GAAI,YACA;AAAA,cACE,eAAe,UAAU,WAAW,SAAS,IACzC,YACA,UAAU,SAAS;AAAA,YACzB,IACA,CAAC;AAAA,YACL,GAAG,KAAK;AAAA,UACV;AAAA,UACA,MAAM,KAAK,UAAU,IAAI;AAAA,UACzB,QAAQ;AAAA,QACV,CAAC;AACD,YAAI,CAAC,IAAI,IAAI;AAGX,gBAAM,YAAY,IAAI,UAAU,OAAO,IAAI,WAAW,OAAO,IAAI,WAAW;AAC5E,cAAI,CAAC,aAAa,YAAY,YAAY;AACxC,kBAAM,OAAO,MAAM,IAAI,KAAK,EAAE,MAAM,MAAM,EAAE;AAC5C,kBAAM,IAAI,MAAM,gBAAgB,GAAG,YAAY,IAAI,MAAM,MAAM,KAAK,MAAM,GAAG,GAAG,CAAC,EAAE;AAAA,UACrF;AAEA,gBAAM,MAAM,KAAK,UAAU,MAAM,KAAK,OAAO,IAAI,GAAG;AACpD;AAAA,QACF;AACA,cAAM,SAAU,MAAM,IAAI,KAAK;AAC/B,eAAO,OAAO;AAAA,MAChB,SAAS,KAAK;AAEZ,YAAI,IAAI,OAAO,QAAS,OAAM;AAC9B,oBAAY;AACZ,YAAI,YAAY,WAAY,OAAM;AAClC,cAAM,MAAM,KAAK,UAAU,MAAM,KAAK,OAAO,IAAI,GAAG;AAAA,MACtD;AAAA,IACF;AACA,UAAM,aAAa,IAAI,MAAM,gCAAgC;AAAA,EAC/D;AACF;AAEA,SAAS,MAAM,IAA2B;AACxC,SAAO,IAAI,QAAQ,CAAC,YAAY;AAC9B,UAAM,IAAI,WAAW,SAAS,EAAE;AAEhC,QAAI,OAAQ,EAA6B,UAAU;AACjD,MAAC,EAA4B,MAAM;AAAA,EACvC,CAAC;AACH;AAgEA,eAAsB,kBACpB,MAC+B;AAC/B,MAAI,KAAK,SAAS,QAAW;AAC3B,UAAM,IAAI;AAAA,MACR;AAAA,IACF;AAAA,EACF;AACA,QAAM,OAAO,KAAK,QAAQ;AAC1B,QAAM,WAAW,KAAK,gBAAgB,KAAK,OAAO;AAClD,QAAM,eACJ,OAAO,KAAK,SAAS,WAAW,UAAU,KAAK,KAAK,QAAQ,cAAc,EAAE,CAAC,KAAK;AAKpF,QAAM,EAAE,aAAa,IAAI,MAAM,OAAO,MAAW;AAEjD,QAAM,SAAS,aAAa,OAAO,KAAK,QAAQ;AAC9C,UAAM,QAAQ,KAAK,IAAI;AACvB,QAAI,SAAS;AACb,QAAI,UAAU;AACd,QAAI;AAEJ,QAAI;AACF,UAAI,IAAI,WAAW,UAAU,IAAI,KAAK,MAAM,GAAG,EAAE,CAAC,MAAM,MAAM;AAC5D,YAAI,aAAa;AACjB,YAAI,IAAI,WAAW;AACnB;AAAA,MACF;AACA,UAAI,cAAc;AAChB,cAAM,MAAM,IAAI,QAAQ,eAAe;AACvC,YAAI,QAAQ,cAAc;AACxB,cAAI,aAAa;AACjB,cAAI,IAAI,cAAc;AACtB;AAAA,QACF;AAAA,MACF;AAGA,YAAM,SAAmB,CAAC;AAC1B,UAAI,aAAa;AACjB,YAAM,UAAU,IAAI,gBAAgB;AACpC,UAAI,GAAG,SAAS,MAAM;AACpB,YAAI,CAAC,IAAI,cAAe,SAAQ,MAAM;AAAA,MACxC,CAAC;AAED,uBAAiB,SAAS,KAAK;AAC7B,cAAM,MAAM;AACZ,sBAAc,IAAI;AAClB,YAAI,aAAa,UAAU;AACzB,cAAI,aAAa;AACjB,cAAI,IAAI,mBAAmB;AAC3B;AAAA,QACF;AACA,eAAO,KAAK,GAAG;AAAA,MACjB;AAEA,YAAM,OAAO,KAAK;AAAA,QAChB,OAAO,OAAO,MAAM,EAAE,SAAS,MAAM;AAAA,MACvC;AACA,eAAS,KAAK;AAEd,YAAM,MAAuB,KAAK,iBAC9B,MAAM,KAAK,eAAe,MAAM,QAAQ,MAAM,IAC9C;AAAA,QACE,QAAQ,KAAK;AAAA,QACb,KAAK,KAAK;AAAA,QACV,YAAY,KAAK;AAAA,QACjB,MAAM,KAAK;AAAA,QACX,QAAQ,QAAQ;AAAA,QAChB,WAAW,KAAK;AAAA,QAChB,SAAS,KAAK;AAAA,QACd,OAAO;AAAA,QACP,WAAW;AAAA,QACX,MAAM;AAAA,MACR;AAEJ,YAAM,WAAW,MAAM,KAAK,SAAS,KAAK,UAAU,GAAG;AACvD,YAAM,eAAoD,EAAE,SAAS;AAErE,UAAI,aAAa;AACjB,UAAI,UAAU,gBAAgB,kBAAkB;AAChD,UAAI,IAAI,KAAK,UAAU,YAAY,CAAC;AACpC,gBAAU;AAAA,IACZ,SAAS,KAAK;AACZ,kBAAY;AAEZ,UAAK,KAAe,SAAS,cAAc;AACzC,YAAI,aAAa;AACjB,YAAI,IAAI,gBAAgB;AACxB;AAAA,MACF;AACA,UAAI,aAAa;AACjB,UAAI,UAAU,gBAAgB,kBAAkB;AAChD,UAAI,IAAI,KAAK,UAAU,EAAE,OAAO,eAAe,QAAQ,IAAI,UAAU,OAAO,GAAG,EAAE,CAAC,CAAC;AAAA,IACrF,UAAE;AACA,WAAK,YAAY;AAAA,QACf;AAAA,QACA,YAAY,KAAK,IAAI,IAAI;AAAA,QACzB;AAAA,QACA,OAAO;AAAA,MACT,CAAC;AAAA,IACH;AAAA,EACF,CAAC;AAED,QAAM,IAAI,QAAc,CAAC,SAAS,WAAW;AAC3C,WAAO,KAAK,SAAS,MAAM;AAC3B,WAAO,OAAO,KAAK,MAAM,KAAK,QAAQ,WAAW,MAAM,QAAQ,CAAC;AAAA,EAClE,CAAC;AAED,QAAM,OAAO,OAAO,QAAQ;AAC5B,QAAM,YAAY,OAAO,SAAS,YAAY,OAAO,KAAK,OAAO,KAAK;AAEtE,SAAO;AAAA,IACL,MAAM;AAAA,IACN,OAAO,MACL,IAAI,QAAc,CAAC,SAAS,WAAW;AACrC,aAAO,MAAM,CAAC,QAAS,MAAM,OAAO,GAAG,IAAI,QAAQ,CAAE;AAAA,IACvD,CAAC;AAAA,EACL;AACF;AAIA,IAAM,aAAa;AAAA,EACjB,MAAM,OAAO;AAAA,IACX,KAAK,MAAM;AAAA,IAAC;AAAA,IACZ,cAAc,MAAM;AAAA,IAAC;AAAA,IACrB,WAAW,MAAM;AAAA,IAAC;AAAA,IAClB,iBAAiB,MAAM;AAAA,IAAC;AAAA,IACxB,UAAU,MAAM;AAAA,IAAC;AAAA,EACnB;AACF;AAEA,IAAM,iBAAiB;AAAA,EACrB,OAAO,YAAY;AAAA,EACnB,MAAM,YAAY;AAAA,EAClB,MAAM,YAAY,CAAC;AACrB;AAEA,IAAM,YAAY;AAAA,EAChB,QAAQ,MAAM;AAAA,EAAC;AAAA,EACf,OAAO,MAAM;AACf;","names":[]}
+{"version":3,"sources":["../../src/adapters/http.ts"],"sourcesContent":["/**\n * # `@tangle-network/agent-eval/adapters/http` — distributed Dispatch over HTTP.\n *\n * Decouples driver and worker. The driver (running `runImprovementLoop` or\n * `runCampaign`) can live anywhere — your VPC, a dev laptop, a cron VM. The\n * workers (running the actual agent) can live anywhere else — different\n * regions, different clouds, different boxes — as long as they speak HTTP.\n *\n * Both sides:\n *\n *   - **`httpDispatch({ url | resolveUrl, ... })`** — client. Returns a\n *     `Dispatch` that POSTs `{ scenario, ctx }` to a worker URL and parses\n *     the artifact back. AbortSignal-aware, retries on idempotent errors,\n *     bounded timeout per call.\n *   - **`runDispatchServer({ dispatch, port, ... })`** — server. Wraps your\n *     local `Dispatch` as an HTTP endpoint. Handles auth, JSON parsing,\n *     error mapping, and cancellation when the client aborts.\n *\n * # Topology examples\n *\n * **Single-worker:** driver on box A, worker on box B. Set\n * `httpDispatch({ url: 'https://box-b/dispatch' })`.\n *\n * **Multi-region:** N workers across regions. Use `httpDispatch({ resolveUrl })`\n * with a function that picks the URL per cell from `ctx.placement`. Combined\n * with `cellPlacement` on `RunCampaignOptions`, the substrate fans cells\n * across geographies in parallel.\n *\n * **Driver-as-a-service:** driver runs as a long-lived process or service\n * (holds optimization state across generations); workers are stateless\n * HTTP services that can scale horizontally per cell.\n */\n\nimport type { Dispatch, DispatchContext, Scenario } from '../contract'\n\n// ── Client ───────────────────────────────────────────────────────────\n\n// eslint-disable-next-line @typescript-eslint/no-unused-vars -- TArtifact is unused\n//  in this options interface but kept as a parameter so callers can write\n//  `HttpDispatchOptions<MyScenario, MyArtifact>` symmetrically with\n//  `Dispatch<MyScenario, MyArtifact>`. Marking it unused at the position\n//  where it bites.\nexport interface HttpDispatchOptions<TScenario extends Scenario, _TArtifact> {\n  /** Static endpoint URL. Mutually exclusive with `resolveUrl`. */\n  url?: string\n  /**\n   * Dynamic per-cell URL resolver. Receives the scenario + the substrate\n   * placement key (from `RunCampaignOptions.cellPlacement`) and returns the\n   * worker URL to invoke. Mutually exclusive with `url`.\n   */\n  resolveUrl?: (input: { scenario: TScenario; placement?: string; cellId: string }) => string\n  /** Bearer token or static auth string set as `Authorization`. */\n  auth?: string | (() => string | Promise<string>)\n  /** Extra headers merged into every request. */\n  headers?: Record<string, string>\n  /** Per-call timeout in ms. Default 5 minutes. */\n  timeoutMs?: number\n  /** How many idempotent retries on 5xx / network errors. Default 2. */\n  retries?: number\n  /** Optional fetch override (auth wrappers, custom agent, mocks). */\n  fetchImpl?: typeof fetch\n}\n\nexport interface HttpDispatchRequestBody<TScenario extends Scenario> {\n  scenario: TScenario\n  cellId: string\n  rep: number\n  generation?: number\n  seed: number\n  placement?: string\n  cycleId?: string\n}\n\nexport interface HttpDispatchResponseBody<TArtifact> {\n  artifact: TArtifact\n}\n\nfunction resolveAuth(auth: HttpDispatchOptions<Scenario, unknown>['auth']): Promise<string | null> {\n  if (!auth) return Promise.resolve(null)\n  if (typeof auth === 'string') return Promise.resolve(auth)\n  return Promise.resolve(auth())\n}\n\n/**\n * Wrap a remote HTTP endpoint as a `Dispatch`. The remote side should run\n * `runDispatchServer` (or any service that speaks the same wire shape).\n *\n * Cancellation: the substrate's per-cell `AbortSignal` is forwarded; the\n * server's `runDispatchServer` translates the resulting `AbortError` into\n * a 499 (client-closed) so the client doesn't retry.\n */\nexport function httpDispatch<TScenario extends Scenario, TArtifact>(\n  opts: HttpDispatchOptions<TScenario, TArtifact>,\n): Dispatch<TScenario, TArtifact> {\n  if (!opts.url && !opts.resolveUrl) {\n    throw new Error('httpDispatch: pass exactly one of `url` or `resolveUrl`.')\n  }\n  if (opts.url && opts.resolveUrl) {\n    throw new Error('httpDispatch: pass exactly one of `url` or `resolveUrl`, not both.')\n  }\n  const timeoutMs = opts.timeoutMs ?? 5 * 60 * 1000\n  const maxRetries = opts.retries ?? 2\n  const f: typeof fetch = opts.fetchImpl ?? ((...args) => fetch(...args))\n\n  return async (scenario, ctx) => {\n    const url =\n      opts.url ?? opts.resolveUrl!({ scenario, placement: ctx.placement, cellId: ctx.cellId })\n    const authValue = await resolveAuth(opts.auth)\n    const body: HttpDispatchRequestBody<TScenario> = {\n      scenario,\n      cellId: ctx.cellId,\n      rep: ctx.rep,\n      generation: ctx.generation,\n      seed: ctx.seed,\n      placement: ctx.placement,\n      cycleId: ctx.cycleId,\n    }\n\n    let lastError: unknown\n    for (let attempt = 0; attempt <= maxRetries; attempt++) {\n      // Compose the request signal: caller's signal OR our timeout.\n      const ourTimeout = AbortSignal.timeout(timeoutMs)\n      const combinedSignal = AbortSignal.any([ctx.signal, ourTimeout])\n      try {\n        const res = await f(url, {\n          method: 'POST',\n          headers: {\n            'Content-Type': 'application/json',\n            ...(authValue\n              ? {\n                  Authorization: authValue.startsWith('Bearer ')\n                    ? authValue\n                    : `Bearer ${authValue}`,\n                }\n              : {}),\n            ...opts.headers,\n          },\n          body: JSON.stringify(body),\n          signal: combinedSignal,\n        })\n        if (!res.ok) {\n          // 4xx is non-retryable (caller error, auth, bad scenario shape).\n          // 5xx / 408 / 429 / 502 / 503 / 504 are retryable.\n          const retryable = res.status >= 500 || res.status === 408 || res.status === 429\n          if (!retryable || attempt === maxRetries) {\n            const text = await res.text().catch(() => '')\n            throw new Error(`httpDispatch ${url} failed (${res.status}): ${text.slice(0, 500)}`)\n          }\n          // exponential backoff with jitter\n          await sleep(2 ** attempt * 200 + Math.random() * 200)\n          continue\n        }\n        const parsed = (await res.json()) as HttpDispatchResponseBody<TArtifact>\n        return parsed.artifact\n      } catch (err) {\n        // Caller-driven abort is terminal — never retry.\n        if (ctx.signal.aborted) throw err\n        lastError = err\n        if (attempt === maxRetries) throw err\n        await sleep(2 ** attempt * 200 + Math.random() * 200)\n      }\n    }\n    throw lastError ?? new Error('httpDispatch exhausted retries')\n  }\n}\n\nfunction sleep(ms: number): Promise<void> {\n  return new Promise((resolve) => {\n    const t = setTimeout(resolve, ms)\n    // Don't keep node process alive purely for backoff sleeps.\n    if (typeof (t as { unref?: () => void }).unref === 'function')\n      (t as { unref: () => void }).unref()\n  })\n}\n\n// ── Server ───────────────────────────────────────────────────────────\n\nexport interface RunDispatchServerOptions<TScenario extends Scenario, TArtifact> {\n  /** The Dispatch this server exposes — what runs when a request lands. */\n  dispatch: Dispatch<TScenario, TArtifact>\n  /** TCP port to bind. */\n  port: number\n  /** Optional bind host; defaults to 0.0.0.0. */\n  host?: string\n  /** Required for any non-test deployment: the bearer token clients must\n   *  send. The substrate refuses to start without auth unless `auth: false`\n   *  is set explicitly (intended ONLY for closed-network/internal testing). */\n  auth: string | false\n  /** Path the server listens on. Default `/dispatch`. */\n  path?: string\n  /**\n   * Per-request handler that wraps `dispatch` with whatever context the\n   * worker side needs to construct a `DispatchContext` — typically the\n   * trace writer, artifact writer, and cost meter. The substrate provides\n   * synthetic-but-typed defaults if not supplied; production deployments\n   * should wire real ones (e.g. ship traces to your OTel collector).\n   */\n  contextFactory?: (\n    req: HttpDispatchRequestBody<TScenario>,\n    signal: AbortSignal,\n  ) => Promise<DispatchContext>\n  /** Optional max payload size for the request body (bytes). Default 10 MB. */\n  maxBodyBytes?: number\n  /** Hook for observability — called on every successful or failed turn. */\n  onRequest?: (event: {\n    cellId: string\n    durationMs: number\n    success: boolean\n    error?: unknown\n  }) => void\n}\n\nexport interface DispatchServerHandle {\n  /** The actual bound port (useful when `port: 0` requests an ephemeral port). */\n  port: number\n  /** Stop accepting new connections and drain existing ones. */\n  close: () => Promise<void>\n}\n\n/**\n * Start an HTTP server exposing a local `Dispatch` over the wire. Pair with\n * `httpDispatch` on the driver side.\n *\n * Wire shape:\n *\n *   POST /dispatch\n *   Authorization: Bearer <token>\n *   Body: HttpDispatchRequestBody\n *   200 OK: HttpDispatchResponseBody\n *   401: missing/invalid auth\n *   408: per-request timeout exceeded\n *   499: client aborted before completion\n *   500: dispatch threw\n *\n * The server is `node:http`-based to keep the runtime dependency surface\n * minimal — works in plain Node, sandbox, or any container.\n */\nexport async function runDispatchServer<TScenario extends Scenario, TArtifact>(\n  opts: RunDispatchServerOptions<TScenario, TArtifact>,\n): Promise<DispatchServerHandle> {\n  if (opts.auth === undefined) {\n    throw new Error(\n      \"runDispatchServer: 'auth' is required (pass a bearer-token string, or `auth: false` explicitly for a closed-network test deployment).\",\n    )\n  }\n  const path = opts.path ?? '/dispatch'\n  const maxBytes = opts.maxBodyBytes ?? 10 * 1024 * 1024\n  const expectedAuth =\n    typeof opts.auth === 'string' ? `Bearer ${opts.auth.replace(/^Bearer\\s+/, '')}` : null\n\n  // Lazy-import node:http so the file is usable from non-Node bundlers\n  // that import the client side only (e.g. an edge driver shipping\n  // httpDispatch alone). Server side is opt-in by calling this function.\n  const { createServer } = await import('node:http')\n\n  const server = createServer(async (req, res) => {\n    const start = Date.now()\n    let cellId = 'unknown'\n    let success = false\n    let errCaught: unknown\n\n    try {\n      if (req.method !== 'POST' || req.url?.split('?')[0] !== path) {\n        res.statusCode = 404\n        res.end('not found')\n        return\n      }\n      if (expectedAuth) {\n        const got = req.headers.authorization\n        if (got !== expectedAuth) {\n          res.statusCode = 401\n          res.end('unauthorized')\n          return\n        }\n      }\n\n      // Read body up to maxBytes\n      const chunks: Buffer[] = []\n      let totalBytes = 0\n      const aborter = new AbortController()\n      req.on('close', () => {\n        if (!res.writableEnded) aborter.abort()\n      })\n\n      for await (const chunk of req) {\n        const buf = chunk as Buffer\n        totalBytes += buf.length\n        if (totalBytes > maxBytes) {\n          res.statusCode = 413\n          res.end('payload too large')\n          return\n        }\n        chunks.push(buf)\n      }\n\n      const body = JSON.parse(\n        Buffer.concat(chunks).toString('utf8'),\n      ) as HttpDispatchRequestBody<TScenario>\n      cellId = body.cellId\n\n      const ctx: DispatchContext = opts.contextFactory\n        ? await opts.contextFactory(body, aborter.signal)\n        : {\n            cellId: body.cellId,\n            rep: body.rep,\n            generation: body.generation,\n            seed: body.seed,\n            signal: aborter.signal,\n            placement: body.placement,\n            cycleId: body.cycleId,\n            trace: NOOP_TRACE,\n            artifacts: NOOP_ARTIFACTS,\n            cost: NOOP_COST,\n          }\n\n      const artifact = await opts.dispatch(body.scenario, ctx)\n      const responseBody: HttpDispatchResponseBody<TArtifact> = { artifact }\n\n      res.statusCode = 200\n      res.setHeader('content-type', 'application/json')\n      res.end(JSON.stringify(responseBody))\n      success = true\n    } catch (err) {\n      errCaught = err\n      // Client-cancelled — they don't care about the result.\n      if ((err as Error)?.name === 'AbortError') {\n        res.statusCode = 499\n        res.end('client aborted')\n        return\n      }\n      res.statusCode = 500\n      res.setHeader('content-type', 'application/json')\n      res.end(JSON.stringify({ error: err instanceof Error ? err.message : String(err) }))\n    } finally {\n      opts.onRequest?.({\n        cellId,\n        durationMs: Date.now() - start,\n        success,\n        error: errCaught,\n      })\n    }\n  })\n\n  await new Promise<void>((resolve, reject) => {\n    server.once('error', reject)\n    server.listen(opts.port, opts.host ?? '0.0.0.0', () => resolve())\n  })\n\n  const addr = server.address()\n  const boundPort = typeof addr === 'object' && addr ? addr.port : opts.port\n\n  return {\n    port: boundPort,\n    close: () =>\n      new Promise<void>((resolve, reject) => {\n        server.close((err) => (err ? reject(err) : resolve()))\n      }),\n  }\n}\n\n// ── No-op default ctx machinery (worker can replace via contextFactory) ──\n\nconst NOOP_TRACE = {\n  span: () => ({\n    end: () => {},\n    setAttribute: () => {},\n    setStatus: () => {},\n    recordException: () => {},\n    addEvent: () => {},\n  }),\n} as unknown as DispatchContext['trace']\n\nconst NOOP_ARTIFACTS = {\n  write: async () => undefined,\n  read: async () => undefined,\n  list: async () => [],\n} as unknown as DispatchContext['artifacts']\n\nconst NOOP_COST = {\n  record: () => {},\n  total: () => 0,\n} as unknown as DispatchContext['cost']\n"],"mappings":";;;AA6EA,SAAS,YAAY,MAA8E;AACjG,MAAI,CAAC,KAAM,QAAO,QAAQ,QAAQ,IAAI;AACtC,MAAI,OAAO,SAAS,SAAU,QAAO,QAAQ,QAAQ,IAAI;AACzD,SAAO,QAAQ,QAAQ,KAAK,CAAC;AAC/B;AAUO,SAAS,aACd,MACgC;AAChC,MAAI,CAAC,KAAK,OAAO,CAAC,KAAK,YAAY;AACjC,UAAM,IAAI,MAAM,0DAA0D;AAAA,EAC5E;AACA,MAAI,KAAK,OAAO,KAAK,YAAY;AAC/B,UAAM,IAAI,MAAM,oEAAoE;AAAA,EACtF;AACA,QAAM,YAAY,KAAK,aAAa,IAAI,KAAK;AAC7C,QAAM,aAAa,KAAK,WAAW;AACnC,QAAM,IAAkB,KAAK,cAAc,IAAI,SAAS,MAAM,GAAG,IAAI;AAErE,SAAO,OAAO,UAAU,QAAQ;AAC9B,UAAM,MACJ,KAAK,OAAO,KAAK,WAAY,EAAE,UAAU,WAAW,IAAI,WAAW,QAAQ,IAAI,OAAO,CAAC;AACzF,UAAM,YAAY,MAAM,YAAY,KAAK,IAAI;AAC7C,UAAM,OAA2C;AAAA,MAC/C;AAAA,MACA,QAAQ,IAAI;AAAA,MACZ,KAAK,IAAI;AAAA,MACT,YAAY,IAAI;AAAA,MAChB,MAAM,IAAI;AAAA,MACV,WAAW,IAAI;AAAA,MACf,SAAS,IAAI;AAAA,IACf;AAEA,QAAI;AACJ,aAAS,UAAU,GAAG,WAAW,YAAY,WAAW;AAEtD,YAAM,aAAa,YAAY,QAAQ,SAAS;AAChD,YAAM,iBAAiB,YAAY,IAAI,CAAC,IAAI,QAAQ,UAAU,CAAC;AAC/D,UAAI;AACF,cAAM,MAAM,MAAM,EAAE,KAAK;AAAA,UACvB,QAAQ;AAAA,UACR,SAAS;AAAA,YACP,gBAAgB;AAAA,YAChB,GAAI,YACA;AAAA,cACE,eAAe,UAAU,WAAW,SAAS,IACzC,YACA,UAAU,SAAS;AAAA,YACzB,IACA,CAAC;AAAA,YACL,GAAG,KAAK;AAAA,UACV;AAAA,UACA,MAAM,KAAK,UAAU,IAAI;AAAA,UACzB,QAAQ;AAAA,QACV,CAAC;AACD,YAAI,CAAC,IAAI,IAAI;AAGX,gBAAM,YAAY,IAAI,UAAU,OAAO,IAAI,WAAW,OAAO,IAAI,WAAW;AAC5E,cAAI,CAAC,aAAa,YAAY,YAAY;AACxC,kBAAM,OAAO,MAAM,IAAI,KAAK,EAAE,MAAM,MAAM,EAAE;AAC5C,kBAAM,IAAI,MAAM,gBAAgB,GAAG,YAAY,IAAI,MAAM,MAAM,KAAK,MAAM,GAAG,GAAG,CAAC,EAAE;AAAA,UACrF;AAEA,gBAAM,MAAM,KAAK,UAAU,MAAM,KAAK,OAAO,IAAI,GAAG;AACpD;AAAA,QACF;AACA,cAAM,SAAU,MAAM,IAAI,KAAK;AAC/B,eAAO,OAAO;AAAA,MAChB,SAAS,KAAK;AAEZ,YAAI,IAAI,OAAO,QAAS,OAAM;AAC9B,oBAAY;AACZ,YAAI,YAAY,WAAY,OAAM;AAClC,cAAM,MAAM,KAAK,UAAU,MAAM,KAAK,OAAO,IAAI,GAAG;AAAA,MACtD;AAAA,IACF;AACA,UAAM,aAAa,IAAI,MAAM,gCAAgC;AAAA,EAC/D;AACF;AAEA,SAAS,MAAM,IAA2B;AACxC,SAAO,IAAI,QAAQ,CAAC,YAAY;AAC9B,UAAM,IAAI,WAAW,SAAS,EAAE;AAEhC,QAAI,OAAQ,EAA6B,UAAU;AACjD,MAAC,EAA4B,MAAM;AAAA,EACvC,CAAC;AACH;AAgEA,eAAsB,kBACpB,MAC+B;AAC/B,MAAI,KAAK,SAAS,QAAW;AAC3B,UAAM,IAAI;AAAA,MACR;AAAA,IACF;AAAA,EACF;AACA,QAAM,OAAO,KAAK,QAAQ;AAC1B,QAAM,WAAW,KAAK,gBAAgB,KAAK,OAAO;AAClD,QAAM,eACJ,OAAO,KAAK,SAAS,WAAW,UAAU,KAAK,KAAK,QAAQ,cAAc,EAAE,CAAC,KAAK;AAKpF,QAAM,EAAE,aAAa,IAAI,MAAM,OAAO,MAAW;AAEjD,QAAM,SAAS,aAAa,OAAO,KAAK,QAAQ;AAC9C,UAAM,QAAQ,KAAK,IAAI;AACvB,QAAI,SAAS;AACb,QAAI,UAAU;AACd,QAAI;AAEJ,QAAI;AACF,UAAI,IAAI,WAAW,UAAU,IAAI,KAAK,MAAM,GAAG,EAAE,CAAC,MAAM,MAAM;AAC5D,YAAI,aAAa;AACjB,YAAI,IAAI,WAAW;AACnB;AAAA,MACF;AACA,UAAI,cAAc;AAChB,cAAM,MAAM,IAAI,QAAQ;AACxB,YAAI,QAAQ,cAAc;AACxB,cAAI,aAAa;AACjB,cAAI,IAAI,cAAc;AACtB;AAAA,QACF;AAAA,MACF;AAGA,YAAM,SAAmB,CAAC;AAC1B,UAAI,aAAa;AACjB,YAAM,UAAU,IAAI,gBAAgB;AACpC,UAAI,GAAG,SAAS,MAAM;AACpB,YAAI,CAAC,IAAI,cAAe,SAAQ,MAAM;AAAA,MACxC,CAAC;AAED,uBAAiB,SAAS,KAAK;AAC7B,cAAM,MAAM;AACZ,sBAAc,IAAI;AAClB,YAAI,aAAa,UAAU;AACzB,cAAI,aAAa;AACjB,cAAI,IAAI,mBAAmB;AAC3B;AAAA,QACF;AACA,eAAO,KAAK,GAAG;AAAA,MACjB;AAEA,YAAM,OAAO,KAAK;AAAA,QAChB,OAAO,OAAO,MAAM,EAAE,SAAS,MAAM;AAAA,MACvC;AACA,eAAS,KAAK;AAEd,YAAM,MAAuB,KAAK,iBAC9B,MAAM,KAAK,eAAe,MAAM,QAAQ,MAAM,IAC9C;AAAA,QACE,QAAQ,KAAK;AAAA,QACb,KAAK,KAAK;AAAA,QACV,YAAY,KAAK;AAAA,QACjB,MAAM,KAAK;AAAA,QACX,QAAQ,QAAQ;AAAA,QAChB,WAAW,KAAK;AAAA,QAChB,SAAS,KAAK;AAAA,QACd,OAAO;AAAA,QACP,WAAW;AAAA,QACX,MAAM;AAAA,MACR;AAEJ,YAAM,WAAW,MAAM,KAAK,SAAS,KAAK,UAAU,GAAG;AACvD,YAAM,eAAoD,EAAE,SAAS;AAErE,UAAI,aAAa;AACjB,UAAI,UAAU,gBAAgB,kBAAkB;AAChD,UAAI,IAAI,KAAK,UAAU,YAAY,CAAC;AACpC,gBAAU;AAAA,IACZ,SAAS,KAAK;AACZ,kBAAY;AAEZ,UAAK,KAAe,SAAS,cAAc;AACzC,YAAI,aAAa;AACjB,YAAI,IAAI,gBAAgB;AACxB;AAAA,MACF;AACA,UAAI,aAAa;AACjB,UAAI,UAAU,gBAAgB,kBAAkB;AAChD,UAAI,IAAI,KAAK,UAAU,EAAE,OAAO,eAAe,QAAQ,IAAI,UAAU,OAAO,GAAG,EAAE,CAAC,CAAC;AAAA,IACrF,UAAE;AACA,WAAK,YAAY;AAAA,QACf;AAAA,QACA,YAAY,KAAK,IAAI,IAAI;AAAA,QACzB;AAAA,QACA,OAAO;AAAA,MACT,CAAC;AAAA,IACH;AAAA,EACF,CAAC;AAED,QAAM,IAAI,QAAc,CAAC,SAAS,WAAW;AAC3C,WAAO,KAAK,SAAS,MAAM;AAC3B,WAAO,OAAO,KAAK,MAAM,KAAK,QAAQ,WAAW,MAAM,QAAQ,CAAC;AAAA,EAClE,CAAC;AAED,QAAM,OAAO,OAAO,QAAQ;AAC5B,QAAM,YAAY,OAAO,SAAS,YAAY,OAAO,KAAK,OAAO,KAAK;AAEtE,SAAO;AAAA,IACL,MAAM;AAAA,IACN,OAAO,MACL,IAAI,QAAc,CAAC,SAAS,WAAW;AACrC,aAAO,MAAM,CAAC,QAAS,MAAM,OAAO,GAAG,IAAI,QAAQ,CAAE;AAAA,IACvD,CAAC;AAAA,EACL;AACF;AAIA,IAAM,aAAa;AAAA,EACjB,MAAM,OAAO;AAAA,IACX,KAAK,MAAM;AAAA,IAAC;AAAA,IACZ,cAAc,MAAM;AAAA,IAAC;AAAA,IACrB,WAAW,MAAM;AAAA,IAAC;AAAA,IAClB,iBAAiB,MAAM;AAAA,IAAC;AAAA,IACxB,UAAU,MAAM;AAAA,IAAC;AAAA,EACnB;AACF;AAEA,IAAM,iBAAiB;AAAA,EACrB,OAAO,YAAY;AAAA,EACnB,MAAM,YAAY;AAAA,EAClB,MAAM,YAAY,CAAC;AACrB;AAEA,IAAM,YAAY;AAAA,EAChB,QAAQ,MAAM;AAAA,EAAC;AAAA,EACf,OAAO,MAAM;AACf;","names":[]}

package/dist/campaign/index.d.ts

@@ -538,6 +538,16 @@ interface RunProfileMatrixOptions<TScenario extends Scenario, TArtifact> {
      *  Default true — catches bad model snapshots and non-finite judge dims at
      *  the boundary instead of letting them poison downstream analysis. */
     validate?: boolean;
+    /** Corpus-by-default: derive the trajectory text (`prompt` + `completion`)
+     *  for each cell from its artifact + scenario. When set, every produced
+     *  record carries `prompt`/`completion` (a `CorpusRecord`) so the run's
+     *  graded trajectories can be appended to the durable RL corpus with no
+     *  side-channel — `appendToCorpus(result.records, path)`. Fail-soft: a
+     *  throwing or undefined-returning extractor just omits the text. */
+    corpusText?: (artifact: TArtifact, scenario: TScenario) => {
+        prompt: string;
+        completion: string;
+    } | undefined;
 }
 interface ProfileSummary {
     profileId: string;

package/dist/campaign/index.js

@@ -7,10 +7,12 @@ import {
   heldoutSignificance,
   pairHoldout,
   runEval
-} from "../chunk-GYELOWB6.js";
+} from "../chunk-UD6EF73X.js";
 import {
-  agentProfileHash
-} from "../chunk-PQV2TKC3.js";
+  agentProfileHash,
+  estimateCost,
+  isModelPriced
+} from "../chunk-SL55X4VN.js";
 import {
   buildLoopProvenanceRecord,
   campaignBreakdown,
@@ -31,14 +33,14 @@ import {
   runOptimization,
   surfaceContentHash,
   surfaceHash
-} from "../chunk-ZZCQQHW7.js";
+} from "../chunk-4QJN7RDX.js";
 import {
   assertRealBackend,
   fsCampaignStorage,
   inMemoryCampaignStorage,
   runCampaign,
   summarizeBackendIntegrity
-} from "../chunk-6XQIEUQ2.js";
+} from "../chunk-ZPSKPT3V.js";
 import "../chunk-YV7J7X5N.js";
 import {
   validateRunRecord
@@ -866,12 +868,29 @@ function buildRunRecord(args) {
     perJudge[judgeName] = { ...js.dimensions };
     for (const [dim, value] of Object.entries(js.dimensions)) {
       raw[`${judgeName}.${dim}`] = value;
-      (dimAccum[dim] ??= []).push(value);
+      dimAccum[dim] ??= [];
+      dimAccum[dim].push(value);
     }
     if (js.notes) notes.push(`${judgeName}: ${js.notes}`);
   }
   const perDimMean = {};
   for (const [dim, values] of Object.entries(dimAccum)) perDimMean[dim] = mean2(values);
+  let costUsd = cell.costUsd;
+  let costEstimated = false;
+  if (costUsd === 0 && cell.tokenUsage.output > 0 && isModelPriced(profile.model)) {
+    costUsd = estimateCost(cell.tokenUsage.input, cell.tokenUsage.output, profile.model);
+    costEstimated = costUsd > 0;
+  }
+  raw.cost_usd = costUsd;
+  raw.cost_estimated = costEstimated ? 1 : 0;
+  raw.tokens_input = cell.tokenUsage.input;
+  raw.tokens_output = cell.tokenUsage.output;
+  if (typeof cell.tokenUsage.cached === "number") raw.tokens_cached = cell.tokenUsage.cached;
+  raw.latency_ms = cell.durationMs;
+  if (costUsd > 0) {
+    raw.tokens_per_dollar = (cell.tokenUsage.input + cell.tokenUsage.output) / costUsd;
+  }
+  if (composite > 0.01) raw.cost_per_quality = costUsd / composite;
   const outcome = splitTag === "holdout" ? { holdoutScore: composite, raw } : { searchScore: composite, raw };
   if (Object.keys(perJudge).length > 0) {
     outcome.judgeScores = {
@@ -881,7 +900,7 @@ function buildRunRecord(args) {
       ...notes.length > 0 ? { notes: notes.join(" | ") } : {}
     };
   }
-  return {
+  const record = {
     runId: `${matrixId}:${profile.id}:${cell.cellId}`,
     experimentId,
     candidateId: profile.id,
@@ -891,13 +910,24 @@ function buildRunRecord(args) {
     configHash,
     commitSha,
     wallMs: cell.durationMs,
-    costUsd: cell.costUsd,
+    costUsd,
     tokenUsage: cell.tokenUsage,
     outcome,
     splitTag,
     scenarioId: cell.scenarioId,
     ...cell.error ? { failureMode: cell.error } : {}
   };
+  if (args.corpusText && args.scenario) {
+    try {
+      const text = args.corpusText(cell.artifact, args.scenario);
+      if (text && typeof text.prompt === "string" && typeof text.completion === "string") {
+        record.prompt = text.prompt;
+        record.completion = text.completion;
+      }
+    } catch {
+    }
+  }
+  return record;
 }
 async function runProfileMatrix(opts) {
   if (opts.profiles.length === 0) throw new ProfileMatrixError("profiles must not be empty");
@@ -909,6 +939,7 @@ async function runProfileMatrix(opts) {
   const profileIds = opts.profiles.map((p) => p.id);
   const experimentId = opts.experimentId ?? `pm_${sha({ profileIds, scenarios: opts.scenarios.map((s) => s.id) }).slice(0, 16)}`;
   const matrixId = `mtx_${sha({ experimentId, profileIds, seed, splitTag }).slice(0, 16)}`;
+  const scenarioById = new Map(opts.scenarios.map((s) => [s.id, s]));
   for (const profile of opts.profiles) {
     const profileHash = agentProfileHash(profile);
     try {
@@ -960,7 +991,6 @@ async function runProfileMatrix(opts) {
       now: opts.now,
       runDir: join2(opts.runDir, sanitize(profile.id))
     });
-    campaigns[profile.id] = campaign;
     const profileRecords = [];
     for (const cell of campaign.cells) {
       const record = buildRunRecord({
@@ -971,19 +1001,26 @@ async function runProfileMatrix(opts) {
         experimentId,
         splitTag,
         commitSha: opts.commitSha,
-        matrixId
+        matrixId,
+        scenario: scenarioById.get(cell.scenarioId),
+        corpusText: opts.corpusText
       });
       if (validate) validateRunRecord(record);
       profileRecords.push(record);
       records.push(record);
     }
+    const pricedTotalCostUsd = profileRecords.reduce((a, r) => a + r.costUsd, 0);
+    campaigns[profile.id] = {
+      ...campaign,
+      aggregates: { ...campaign.aggregates, totalCostUsd: pricedTotalCostUsd }
+    };
     byProfile[profile.id] = {
       profileId: profile.id,
       profileHash,
       model: profile.model,
       records: profileRecords.length,
       meanComposite: mean2(profileRecords.map(compositeOf)),
-      totalCostUsd: profileRecords.reduce((a, r) => a + r.costUsd, 0),
+      totalCostUsd: pricedTotalCostUsd,
       integrity: summarizeBackendIntegrity(profileRecords)
     };
   }