@tangle-network/agent-eval 0.25.0 → 0.27.2

package/CHANGELOG.md

@@ -1,5 +1,150 @@
 # Changelog
 
+## 0.27.2 — 2026-05-17
+
+### Corpus-wide inter-rater agreement primitive
+
+`interRaterReliability(JudgeScore[][])` measures Krippendorff α *within
+a single item* — multiple judges rate the same scenario, how much do
+their scores cluster? That answers "is this one judgement contested?"
+It does not answer "is this judge panel reliable across the whole
+evaluation corpus?" — the question the five product consumers actually
+need before trusting a multi-judge composite over 100+ scenarios.
+
+This release ships the corpus-wide companion. It does not touch the
+existing primitive: the within-item α and the corpus-wide ICC are
+different formulas with different domains of validity.
+
+### Added
+
+- **`corpusInterRaterAgreement(records, opts?)`** (`src/statistics.ts`) —
+  takes a flat list of `{itemId, judgeName, dimension, score}` records.
+  For each dimension, pivots to the [n_items × n_judges] matrix of items
+  every judge rated and delegates to `continuousAgreement` (ICC(2,1) +
+  κ_w + Pearson + Spearman + bootstrap CIs from 0.26.0). An overall
+  pooled mean across dimensions gives one "is the panel reliable on
+  this corpus?" number.
+- **`corpusInterRaterAgreementFromJudgeScores(itemsScores, opts?)`** —
+  adapter for consumers that already hold per-item `JudgeScore[]`
+  arrays (e.g. `ScenarioResult.judgeScores`) and want to skip manual
+  flattening.
+- New exported types: `CorpusScoreRecord`, `CorpusAgreementOptions`,
+  `CorpusAgreementPerDimension`, `CorpusAgreementReport`.
+
+### Fail-loud contract
+
+Per `CLAUDE.md` "no silent fallbacks": the primitive throws
+`ValidationError` on empty input, fewer than 2 judges, fewer than 2
+items rated by every judge on a given dimension, a judge with zero
+items on a dimension (would silently shrink the matrix and corrupt the
+overall metric), duplicate `(itemId, judge, dimension)` records, or any
+non-finite score. There is no quiet-NaN path.
+
+### Consumer contract
+
+`tests/consumer-contract.test.ts` pins both new exports. The 0.27.0
+surface is preserved — no rename, no signature change on the existing
+`interRaterReliability`.
+
+## 0.27.1 — 2026-05-17
+
+### Signal-honesty sweep — substrate
+
+- **`sandbox-harness.ts`** — the timeout-driven `SIGKILL` previously sat
+  inside an empty `} catch {}`. A failed kill would vanish from logs.
+  It now surfaces via `console.warn` with full error context while
+  preserving teardown semantics (the timer already fired; the subprocess
+  is being terminated).
+- **`control-runtime.ts`** — documented the `ControlRunResult.runId:
+  string | null` contract at the type declaration. The 18 sites that
+  coerce `emitter?.runId` to `null` (one per terminal return path) are
+  typed-contract conversions, not silent fallbacks: `null` means "the
+  run executed without a `TraceEmitter` wired and no run record was
+  persisted." Type-level docs end the recurring "is this a bug?" review.
+  (Three sibling `?? null` coercions on the same returns —
+  `actionCostUsd`, `scoreBefore`, `scoreAfter` — are likewise typed-optional
+  span attributes documented at their declaration sites.)
+- **`.gitignore`** — added `data/` (local dev session storage).
+- **`tests/consumer-contract.test.ts`** — pins the runtime symbols that
+  the five product-agent consumers (tax/creative/legal/gtm/agent-builder)
+  import from `@tangle-network/agent-eval`. The full set of types is
+  validated at compile time via the namespace import; runtime classes
+  and functions are exhaustively asserted. Any removal/rename of a
+  load-bearing export now fails this test before shipping.
+
+## 0.27.0 — 2026-05-17
+
+### Substrate reliability — eliminate silent-zero judge corruption
+
+Today's tax + gtm evals shipped composites where the judge LLM silently
+aborted (verbose new prompts streamed past the 60s default timeout) and
+the per-trial score collapsed to `0`. The composite formula then weighted
+that zero into the mean, producing a "−27pp tax regression" that was
+actually a measurement-instrument failure, not a prompt regression.
+
+This release adds three substrate primitives so consumers can stop
+silent-zeroing their own data:
+
+- **`withJudgeRetry(judgeFn, policy)`** — wraps any judge call with retry
+  on transient failures (Abort, Timeout, fetch failed, 429/502/503/504),
+  optional fallback-model rotation, and a typed outcome (`succeeded`,
+  `attempts`, `value`, `error`). Refuses to default to a silent zero.
+- **`aggregateTrialsByMode(trials, { mode })`** — `'exclude-failed'` mode
+  drops trials with `judgeSucceeded === false` from the mean so a failed
+  judge doesn't corrupt the composite. `'strict-fail'` mode refuses the
+  aggregate when any judge failed. `'zero-fill'` preserves legacy.
+- **`discoverPersonas(dir, opts)`** — replaces every consumer's hardcoded
+  `TRAINING_PERSONA_FILES` constant. New personas on disk are picked up
+  automatically; consumers can filter via include/exclude patterns.
+
+Additive to `TrialResult`: `judgeSucceeded?`, `judgeAttempts?`, `judgeError?`
+fields. Existing adapters that don't set these continue to work
+unchanged via `'zero-fill'` mode (default for back-compat).
+
+
+## 0.26.0 — Continuous-value inter-rater agreement (ICC + weighted κ)
+
+The original `calibrateJudge` rounded scores to ints before computing
+Cohen's κ. For fine-grained judges that's lossy — 0.78 vs 0.81 both
+round to "1" and the integer κ pretends they agreed perfectly when they
+actually disagree by 3 percentage points. This release ships principled
+continuous-value agreement metrics so calibration findings become
+quantitative for [0,1]-valued judges.
+
+### Added
+
+- **`continuousAgreement(scores, opts?)`** (`src/judge-calibration.ts`) —
+  inter-rater agreement on continuous scores. Returns:
+    - `weightedKappa` — Cohen's κ_w with quadratic (or linear) weights on
+      raw scores, no quantisation.
+    - `icc` — ICC(2,1), two-way random effects, absolute agreement,
+      single rater (Shrout & Fleiss 1979). The principled reliability
+      coefficient when judges are a random sample of the judge population.
+    - `pearson` / `spearman` — averaged over rater pairs when N ≥ 2 raters.
+    - `ci.icc` / `ci.weightedKappa` — bootstrap percentile 95% CIs
+      (default `n=1000`, seeded for reproducibility).
+  Accepts `scores: number[][]` shaped `[n_items][n_raters]`. Rows with
+  non-finite entries are dropped, not coerced.
+
+- **`calibrateJudgeContinuous(golden, candidate, opts?)`** — drop-in
+  superset of `calibrateJudge`. Preserves every legacy field
+  (`n`, `pearson`, `kappa`, `mae`, `worstItems`) and adds
+  `weightedKappaContinuous`, `icc`, `spearman`, and `ci`. Use this when
+  the judge produces fine-grained [0,1] scores; keep `calibrateJudge`
+  for the original integer-quantised report.
+
+### Why two κ flavours
+
+ICC(2,1) catches systematic bias that Pearson misses. If judge B scores
+2× judge A, Pearson stays ≈ 1 (linear association is perfect) while ICC
+plummets (absolute agreement is poor). The new tests assert this exact
+failure mode so the regression can't sneak back in.
+
+### Unchanged
+
+- `calibrateJudge` keeps its original integer-rounded κ semantics for
+  backwards compatibility. Nothing else moves.
+
 ## 0.25.0 — ProductionLoop primitive: close the eval → prod → eval cycle
 
 This release ships the **orchestration layer** that turns the existing

package/README.md

@@ -88,7 +88,7 @@ await product.storeEvalResult(task.id, result)
 Same loop shape in production, replay, benchmark, and optimization. Swap the
 dependencies behind `observe()` and `act()`, never the eval contract.
 
-## Production loop — close the eval → prod → eval cycle (0.25.0)
+## Production loop — close the eval → prod → eval cycle
 
 Static prompts decay. Yesterday's FTC rule flips today; yesterday's tool quirk
 becomes today's incident. The production agents that win are the ones that
@@ -194,6 +194,7 @@ const next = await analyzeOptimizationResult(campaign, { researcher })
 | Deterministic re-judge / audit | `ReplayCache`, `createReplayFetch` | `/` |
 | Anytime-valid α across rolling looks | `pairedEvalueSequence` | `/reporting` |
 | Judge quality vs gold | `calibrateJudge` (κ, Pearson, MAE, bias probes) | `/` |
+| Continuous inter-rater agreement | `calibrateJudgeContinuous`, `continuousAgreement` (κ_w, ICC(2,1), bootstrap CIs) | `/` |
 | (chosen, rejected) for DPO/KTO/PPO | `extractPreferences` | `/rl` |
 | Verifiable reward signal | `extractVerifiableReward` | `/rl` |
 | Step-level / PRM training data | `extractStepRewards`, `prmTrainingPairs` | `/rl` |
@@ -223,9 +224,8 @@ const next = await analyzeOptimizationResult(campaign, { researcher })
 | `@tangle-network/agent-eval/benchmarks` | benchmark adapter contracts and reference wrappers |
 
 The root export remains available for convenience; new code should prefer
-focused subpaths. Anything under `/rl` should be imported from `/rl` — root
-re-export is retained only for backward compatibility and will be narrowed in
-0.25.
+focused subpaths. Anything under `/rl`, `/pipelines`, `/meta-eval`, `/prm`,
+or `/builder-eval` is only reachable via its subpath.
 
 ## API stability
 
@@ -242,7 +242,7 @@ The `/rl` subpath is the most active surface. See
 [`src/rl/index.ts`](./src/rl/index.ts) for the current stable/experimental
 breakdown.
 
-## Capture integrity (0.21+)
+## Capture integrity
 
 Launch-grade benchmark runs need four things that are easy to forget in glue
 code: (1) raw HTTP capture alongside the structured spans so a reviewer can

package/dist/builder-eval/index.js

@@ -1,7 +1,7 @@
 import {
   SandboxHarness,
   runTestGradedScenario
-} from "../chunk-OWLAAMME.js";
+} from "../chunk-QHF6EQKK.js";
 import {
   judgeSpans
 } from "../chunk-47X6LRCE.js";

package/dist/{chunk-WWYCWKUM.js → chunk-3CKU6VGU.js}

@@ -1,6 +1,6 @@
 import {
   objectiveEval
-} from "./chunk-LSH4MMOZ.js";
+} from "./chunk-NCRFYPS3.js";
 
 // src/knowledge/readiness.ts
 function scoreKnowledgeReadiness(options) {
@@ -193,4 +193,4 @@ export {
   userQuestionsForKnowledgeGaps,
   acquisitionPlansForKnowledgeGaps
 };
-//# sourceMappingURL=chunk-WWYCWKUM.js.map
+//# sourceMappingURL=chunk-3CKU6VGU.js.map

package/dist/{chunk-K2TPS5LB.js → chunk-4U4BKCXK.js}

@@ -1,7 +1,7 @@
 import {
   canonicalize,
   hashJson
-} from "./chunk-4F5DQN55.js";
+} from "./chunk-VSMTAMNK.js";
 import {
   ReplayError
 } from "./chunk-NG236HPC.js";
@@ -566,4 +566,4 @@ export {
   createReplayFetch,
   iterateRawCalls
 };
-//# sourceMappingURL=chunk-K2TPS5LB.js.map
+//# sourceMappingURL=chunk-4U4BKCXK.js.map

package/dist/chunk-4U4BKCXK.js.map

@@ -0,0 +1 @@
+{"version":3,"sources":["../src/trace/store.ts","../src/trace/otel.ts","../src/trace/redact.ts","../src/replay.ts"],"sourcesContent":["/**\n * TraceStore — persistence + query over the TraceSchema v1 corpus.\n *\n * Two implementations ship in the core:\n *   - InMemoryTraceStore: dev + tests, fully in-process\n *   - FileSystemTraceStore: NDJSON append-only files per entity, suitable\n *     for long-running CI jobs; rolled over at 32MB\n *\n * Downstream adapters (DuckDB, Langfuse, R2 parquet) implement this same\n * interface — the rest of the framework is storage-agnostic.\n */\n\nimport type {\n  Artifact,\n  BudgetLedgerEntry,\n  EventKind,\n  Run,\n  RunStatus,\n  Span,\n  SpanKind,\n  TraceEvent,\n} from './schema'\n\nexport interface RunFilter {\n  scenarioId?: string\n  variantId?: string\n  status?: RunStatus\n  since?: number\n  until?: number\n  tag?: { key: string; value: string }\n  parentRunId?: string\n  projectId?: string\n  chatId?: string\n  layer?: import('./schema').RunLayer\n}\n\nexport interface SpanFilter {\n  runId?: string\n  parentSpanId?: string\n  kind?: SpanKind\n  name?: string\n  toolName?: string\n  judgeId?: string\n  since?: number\n  until?: number\n}\n\nexport interface EventFilter {\n  runId?: string\n  spanId?: string\n  kind?: EventKind\n  since?: number\n  until?: number\n}\n\nexport interface TraceStore {\n  appendRun(run: Run): Promise<void>\n  updateRun(runId: string, patch: Partial<Run>): Promise<void>\n  appendSpan(span: Span): Promise<void>\n  updateSpan(spanId: string, patch: Partial<Span>): Promise<void>\n  appendEvent(event: TraceEvent): Promise<void>\n  appendArtifact(artifact: Artifact): Promise<void>\n  appendBudgetEntry(entry: BudgetLedgerEntry): Promise<void>\n\n  getRun(runId: string): Promise<Run | undefined>\n  listRuns(filter?: RunFilter): Promise<Run[]>\n  spans(filter?: SpanFilter): Promise<Span[]>\n  events(filter?: EventFilter): Promise<TraceEvent[]>\n  budget(runId: string): Promise<BudgetLedgerEntry[]>\n  artifacts(runId: string): Promise<Artifact[]>\n}\n\n// ── In-memory ────────────────────────────────────────────────────────\n\nexport class InMemoryTraceStore implements TraceStore {\n  private runs = new Map<string, Run>()\n  private allSpans: Span[] = []\n  private allEvents: TraceEvent[] = []\n  private allArtifacts: Artifact[] = []\n  private allBudget: BudgetLedgerEntry[] = []\n\n  async appendRun(run: Run): Promise<void> {\n    if (this.runs.has(run.runId)) throw new Error(`run ${run.runId} already exists`)\n    this.runs.set(run.runId, { ...run })\n  }\n\n  async updateRun(runId: string, patch: Partial<Run>): Promise<void> {\n    const existing = this.runs.get(runId)\n    if (!existing) throw new Error(`run ${runId} not found`)\n    this.runs.set(runId, { ...existing, ...patch })\n  }\n\n  async appendSpan(span: Span): Promise<void> {\n    this.allSpans.push({ ...span })\n  }\n\n  async updateSpan(spanId: string, patch: Partial<Span>): Promise<void> {\n    const idx = this.allSpans.findIndex((s) => s.spanId === spanId)\n    if (idx < 0) throw new Error(`span ${spanId} not found`)\n    this.allSpans[idx] = { ...this.allSpans[idx], ...patch } as Span\n  }\n\n  async appendEvent(event: TraceEvent): Promise<void> {\n    this.allEvents.push({ ...event })\n  }\n\n  async appendArtifact(artifact: Artifact): Promise<void> {\n    this.allArtifacts.push({ ...artifact })\n  }\n\n  async appendBudgetEntry(entry: BudgetLedgerEntry): Promise<void> {\n    this.allBudget.push({ ...entry })\n  }\n\n  async getRun(runId: string): Promise<Run | undefined> {\n    const r = this.runs.get(runId)\n    return r ? { ...r } : undefined\n  }\n\n  async listRuns(filter: RunFilter = {}): Promise<Run[]> {\n    return [...this.runs.values()].filter((r) => matchesRun(r, filter))\n  }\n\n  async spans(filter: SpanFilter = {}): Promise<Span[]> {\n    return this.allSpans.filter((s) => matchesSpan(s, filter)).map((s) => ({ ...s }))\n  }\n\n  async events(filter: EventFilter = {}): Promise<TraceEvent[]> {\n    return this.allEvents.filter((e) => matchesEvent(e, filter)).map((e) => ({ ...e }))\n  }\n\n  async budget(runId: string): Promise<BudgetLedgerEntry[]> {\n    return this.allBudget.filter((b) => b.runId === runId).map((b) => ({ ...b }))\n  }\n\n  async artifacts(runId: string): Promise<Artifact[]> {\n    return this.allArtifacts.filter((a) => a.runId === runId).map((a) => ({ ...a }))\n  }\n}\n\nfunction matchesRun(r: Run, f: RunFilter): boolean {\n  if (f.scenarioId && r.scenarioId !== f.scenarioId) return false\n  if (f.variantId && r.variantId !== f.variantId) return false\n  if (f.status && r.status !== f.status) return false\n  if (f.since !== undefined && r.startedAt < f.since) return false\n  if (f.until !== undefined && r.startedAt > f.until) return false\n  if (f.tag && r.tags?.[f.tag.key] !== f.tag.value) return false\n  if (f.parentRunId && r.parentRunId !== f.parentRunId) return false\n  if (f.projectId && r.projectId !== f.projectId) return false\n  if (f.chatId && r.chatId !== f.chatId) return false\n  if (f.layer && r.layer !== f.layer) return false\n  return true\n}\n\nfunction matchesSpan(s: Span, f: SpanFilter): boolean {\n  if (f.runId && s.runId !== f.runId) return false\n  if (f.parentSpanId && s.parentSpanId !== f.parentSpanId) return false\n  if (f.kind && s.kind !== f.kind) return false\n  if (f.name && s.name !== f.name) return false\n  if (f.toolName && (s.kind !== 'tool' || s.toolName !== f.toolName)) return false\n  if (f.judgeId && (s.kind !== 'judge' || s.judgeId !== f.judgeId)) return false\n  if (f.since !== undefined && s.startedAt < f.since) return false\n  if (f.until !== undefined && s.startedAt > f.until) return false\n  return true\n}\n\nfunction matchesEvent(e: TraceEvent, f: EventFilter): boolean {\n  if (f.runId && e.runId !== f.runId) return false\n  if (f.spanId && e.spanId !== f.spanId) return false\n  if (f.kind && e.kind !== f.kind) return false\n  if (f.since !== undefined && e.timestamp < f.since) return false\n  if (f.until !== undefined && e.timestamp > f.until) return false\n  return true\n}\n\n// ── Filesystem (NDJSON append-only, one file per entity) ─────────────\n\nexport interface FileSystemTraceStoreOptions {\n  dir: string\n  /** Roll over NDJSON files when they exceed this size in bytes. Default 32 MB. */\n  maxBytes?: number\n}\n\nexport class FileSystemTraceStore implements TraceStore {\n  private dir: string\n  private maxBytes: number\n  /** Lazy in-memory index for queries — populated on first read. */\n  private index?: InMemoryTraceStore\n  private loaded = false\n\n  constructor(options: FileSystemTraceStoreOptions) {\n    this.dir = options.dir\n    this.maxBytes = options.maxBytes ?? 32 * 1024 * 1024\n  }\n\n  private async ensureDir(): Promise<void> {\n    const fs = await import('node:fs/promises')\n    await fs.mkdir(this.dir, { recursive: true })\n  }\n\n  private async append(name: string, record: unknown): Promise<void> {\n    await this.ensureDir()\n    const fs = await import('node:fs/promises')\n    const path = await import('node:path')\n    const active = path.join(this.dir, `${name}.ndjson`)\n    try {\n      const stat = await fs.stat(active)\n      if (stat.size >= this.maxBytes) {\n        const rolled = path.join(this.dir, `${name}.${Date.now()}.ndjson`)\n        await fs.rename(active, rolled)\n      }\n    } catch {\n      /* file doesn't exist yet */\n    }\n    await fs.appendFile(active, `${JSON.stringify(record)}\\n`, 'utf8')\n    // Mirror genuinely-new rows into the lazy index. Update rows (marked\n    // with `_update: true` by updateRun/updateSpan) are applied by those\n    // methods directly via the index's update* APIs — re-inserting them\n    // here triggers a duplicate-id error once the first read populates\n    // the index.\n    if (this.index && !(record as { _update?: boolean })?._update) {\n      void this.insertInto(name, record)\n    }\n  }\n\n  private async insertInto(name: string, record: unknown): Promise<void> {\n    if (!this.index) return\n    switch (name) {\n      case 'runs':\n        await this.index.appendRun(record as Run)\n        break\n      case 'spans':\n        await this.index.appendSpan(record as Span)\n        break\n      case 'events':\n        await this.index.appendEvent(record as TraceEvent)\n        break\n      case 'artifacts':\n        await this.index.appendArtifact(record as Artifact)\n        break\n      case 'budget':\n        await this.index.appendBudgetEntry(record as BudgetLedgerEntry)\n        break\n    }\n  }\n\n  private async load(): Promise<InMemoryTraceStore> {\n    if (this.loaded && this.index) return this.index\n    const fs = await import('node:fs/promises')\n    const path = await import('node:path')\n    const store = new InMemoryTraceStore()\n    try {\n      const entries = await fs.readdir(this.dir)\n      for (const file of entries) {\n        if (!file.endsWith('.ndjson')) continue\n        const full = path.join(this.dir, file)\n        const content = await fs.readFile(full, 'utf8')\n        const base = file.split('.')[0]\n        for (const line of content.split('\\n')) {\n          if (!line.trim()) continue\n          const record = JSON.parse(line)\n          if (base === 'runs') {\n            // Allow re-loading without duplicate error\n            try {\n              await store.appendRun(record)\n            } catch {\n              await store.updateRun(record.runId, record)\n            }\n          } else if (base === 'spans') {\n            // `updateSpan` appends an `_update: true` patch row instead of\n            // rewriting the original span. On reload we must collapse those\n            // patches onto the original span — otherwise a fresh\n            // FileSystemTraceStore reading the same dir reports duplicate\n            // spans (one full, one fragment with no runId/kind/name), which\n            // breaks any downstream consumer that re-opens the store\n            // cross-process (e.g. the OTLP converter).\n            if (record?._update) {\n              try {\n                await store.updateSpan(record.spanId, record)\n              } catch {\n                // Patch row arrived before the original — should not happen\n                // with locked append order, but fall through to append so we\n                // don't lose data.\n                await store.appendSpan(record)\n              }\n            } else {\n              await store.appendSpan(record)\n            }\n          } else if (base === 'events') {\n            await store.appendEvent(record)\n          } else if (base === 'artifacts') {\n            await store.appendArtifact(record)\n          } else if (base === 'budget') {\n            await store.appendBudgetEntry(record)\n          }\n        }\n      }\n    } catch {\n      /* empty dir, first run */\n    }\n    this.index = store\n    this.loaded = true\n    return store\n  }\n\n  async appendRun(run: Run): Promise<void> {\n    await this.append('runs', run)\n  }\n  async updateRun(runId: string, patch: Partial<Run>): Promise<void> {\n    // NDJSON is append-only; record updates as new rows with the same runId —\n    // readers collapse by last-write-wins on load.\n    await this.append('runs', { runId, ...patch, _update: true })\n    if (this.index) await this.index.updateRun(runId, patch)\n  }\n  async appendSpan(span: Span): Promise<void> {\n    await this.append('spans', span)\n  }\n  async updateSpan(spanId: string, patch: Partial<Span>): Promise<void> {\n    await this.append('spans', { spanId, ...patch, _update: true })\n    if (this.index) await this.index.updateSpan(spanId, patch)\n  }\n  async appendEvent(event: TraceEvent): Promise<void> {\n    await this.append('events', event)\n  }\n  async appendArtifact(artifact: Artifact): Promise<void> {\n    await this.append('artifacts', artifact)\n  }\n  async appendBudgetEntry(entry: BudgetLedgerEntry): Promise<void> {\n    await this.append('budget', entry)\n  }\n\n  async getRun(runId: string): Promise<Run | undefined> {\n    return (await this.load()).getRun(runId)\n  }\n  async listRuns(filter?: RunFilter): Promise<Run[]> {\n    return (await this.load()).listRuns(filter)\n  }\n  async spans(filter?: SpanFilter): Promise<Span[]> {\n    return (await this.load()).spans(filter)\n  }\n  async events(filter?: EventFilter): Promise<TraceEvent[]> {\n    return (await this.load()).events(filter)\n  }\n  async budget(runId: string): Promise<BudgetLedgerEntry[]> {\n    return (await this.load()).budget(runId)\n  }\n  async artifacts(runId: string): Promise<Artifact[]> {\n    return (await this.load()).artifacts(runId)\n  }\n}\n","/**\n * OpenTelemetry JSON export — maps TraceSchema v1 to OTLP/JSON so\n * traces render natively in Jaeger / Honeycomb / Langfuse / Grafana.\n *\n * Wire format only. We do NOT depend on the @opentelemetry SDK — that\n * would drag in polyfills incompatible with Workers/Edge. Consumers\n * push the JSON to their collector of choice via HTTP.\n *\n * Reference: OTLP 1.3.2 (ResourceSpans / ScopeSpans / Span).\n */\n\nimport type { Run, Span, TraceEvent } from './schema'\nimport type { TraceStore } from './store'\n\nexport const OTEL_AGENT_EVAL_SCOPE = { name: '@tangle-network/agent-eval', version: '0.3.0' }\n\nexport interface OtlpSpan {\n  traceId: string\n  spanId: string\n  parentSpanId?: string\n  name: string\n  kind: number\n  startTimeUnixNano: string\n  endTimeUnixNano: string\n  attributes: Array<{\n    key: string\n    value: { stringValue?: string; intValue?: string; doubleValue?: number; boolValue?: boolean }\n  }>\n  events?: Array<{ timeUnixNano: string; name: string; attributes?: OtlpSpan['attributes'] }>\n  status?: { code: number; message?: string }\n}\n\nexport interface OtlpResourceSpans {\n  resource: { attributes: OtlpSpan['attributes'] }\n  scopeSpans: Array<{ scope: typeof OTEL_AGENT_EVAL_SCOPE; spans: OtlpSpan[] }>\n}\n\nexport interface OtlpExport {\n  resourceSpans: OtlpResourceSpans[]\n}\n\n/** Export a single run's spans + events in OTLP/JSON. */\nexport async function exportRunAsOtlp(\n  store: TraceStore,\n  runId: string,\n  resourceAttrs: Record<string, string | number | boolean> = {},\n): Promise<OtlpExport> {\n  const run = await store.getRun(runId)\n  if (!run) throw new Error(`run ${runId} not found`)\n  const spans = await store.spans({ runId })\n  const events = await store.events({ runId })\n  const eventsBySpan = new Map<string, TraceEvent[]>()\n  for (const e of events) {\n    if (!e.spanId) continue\n    const arr = eventsBySpan.get(e.spanId) ?? []\n    arr.push(e)\n    eventsBySpan.set(e.spanId, arr)\n  }\n  const traceId = runToTraceId(run)\n  const otlpSpans: OtlpSpan[] = spans.map((s) =>\n    spanToOtlp(s, traceId, eventsBySpan.get(s.spanId) ?? []),\n  )\n  return {\n    resourceSpans: [\n      {\n        resource: {\n          attributes: toAttributes({\n            'service.name': 'agent-eval',\n            'run.id': run.runId,\n            'run.scenario_id': run.scenarioId,\n            'run.variant_id': run.variantId ?? '',\n            'run.dataset_version': run.datasetVersion ?? '',\n            'run.code_sha': run.codeSha ?? '',\n            'run.model_fingerprint': run.modelFingerprint ?? '',\n            ...resourceAttrs,\n          }),\n        },\n        scopeSpans: [{ scope: OTEL_AGENT_EVAL_SCOPE, spans: otlpSpans }],\n      },\n    ],\n  }\n}\n\nfunction spanToOtlp(span: Span, traceId: string, events: TraceEvent[]): OtlpSpan {\n  const endedAt = span.endedAt ?? span.startedAt\n  return {\n    traceId,\n    spanId: padSpanId(span.spanId),\n    parentSpanId: span.parentSpanId ? padSpanId(span.parentSpanId) : undefined,\n    name: span.name,\n    kind: 1, // SPAN_KIND_INTERNAL\n    startTimeUnixNano: msToNs(span.startedAt),\n    endTimeUnixNano: msToNs(endedAt),\n    attributes: toAttributes(flattenSpanAttributes(span)),\n    events: events.map((e) => ({\n      timeUnixNano: msToNs(e.timestamp),\n      name: e.kind,\n      attributes: toAttributes(flattenPayload(e.payload)),\n    })),\n    status: span.status === 'error' ? { code: 2, message: span.error } : { code: 1 },\n  }\n}\n\nfunction flattenSpanAttributes(span: Span): Record<string, string | number | boolean> {\n  const base: Record<string, string | number | boolean> = {\n    'span.kind': span.kind,\n  }\n  if (span.kind === 'llm') {\n    base['llm.model'] = span.model\n    if (span.inputTokens !== undefined) base['llm.input_tokens'] = span.inputTokens\n    if (span.outputTokens !== undefined) base['llm.output_tokens'] = span.outputTokens\n    if (span.costUsd !== undefined) base['llm.cost_usd'] = span.costUsd\n    if (span.finishReason) base['llm.finish_reason'] = span.finishReason\n  } else if (span.kind === 'tool') {\n    base['tool.name'] = span.toolName\n    if (span.latencyMs !== undefined) base['tool.latency_ms'] = span.latencyMs\n  } else if (span.kind === 'retrieval') {\n    base['retrieval.query'] = span.query\n    base['retrieval.hits'] = span.hits.length\n  } else if (span.kind === 'judge') {\n    base['judge.id'] = span.judgeId\n    base['judge.dimension'] = span.dimension\n    base['judge.score'] = span.score\n    base['judge.target_span_id'] = span.targetSpanId\n  } else if (span.kind === 'sandbox') {\n    if (span.image) base['sandbox.image'] = span.image\n    if (span.exitCode !== undefined) base['sandbox.exit_code'] = span.exitCode\n    if (span.testsPassed !== undefined) base['sandbox.tests_passed'] = span.testsPassed\n    if (span.testsTotal !== undefined) base['sandbox.tests_total'] = span.testsTotal\n  }\n  if (span.attributes) {\n    for (const [k, v] of Object.entries(span.attributes)) {\n      if (typeof v === 'string' || typeof v === 'number' || typeof v === 'boolean') base[k] = v\n    }\n  }\n  return base\n}\n\nfunction flattenPayload(\n  payload: Record<string, unknown>,\n): Record<string, string | number | boolean> {\n  const out: Record<string, string | number | boolean> = {}\n  for (const [k, v] of Object.entries(payload)) {\n    if (typeof v === 'string' || typeof v === 'number' || typeof v === 'boolean') out[k] = v\n    else out[k] = JSON.stringify(v)\n  }\n  return out\n}\n\nfunction toAttributes(record: Record<string, string | number | boolean>): OtlpSpan['attributes'] {\n  return Object.entries(record).map(([key, value]) => ({\n    key,\n    value:\n      typeof value === 'number'\n        ? Number.isInteger(value)\n          ? { intValue: value.toString() }\n          : { doubleValue: value }\n        : typeof value === 'boolean'\n          ? { boolValue: value }\n          : { stringValue: value },\n  }))\n}\n\nfunction msToNs(ms: number): string {\n  return (BigInt(Math.floor(ms)) * 1_000_000n).toString()\n}\n\nfunction padSpanId(id: string): string {\n  // OTLP wants 16-hex spanIds. UUIDs are 32-hex; strip dashes and take first 16.\n  const cleaned = id.replace(/-/g, '')\n  return cleaned.slice(0, 16).padEnd(16, '0')\n}\n\nfunction runToTraceId(run: Run): string {\n  // OTLP wants 32-hex traceIds. Use runId directly when it's 32-hex already,\n  // else SHA-ish truncate.\n  const cleaned = run.runId.replace(/-/g, '')\n  return cleaned.slice(0, 32).padEnd(32, '0')\n}\n","/**\n * Redaction — remove PII / secrets from trace payloads before persist.\n *\n * Pre-persistence rules mean raw traces in storage are already scrubbed.\n * Unredacted variants (for debugging / post-mortems) live in a separate\n * storage layer with stricter access controls; this module only covers\n * the default scrub-then-persist path.\n *\n * Rules compose: pass an array of `RedactionRule`, each is applied in\n * order. Strings that match get replaced with a tagged sentinel so the\n * eval framework can count how many redactions happened per run\n * (surfaced via `redaction_applied` events).\n */\n\nexport interface RedactionRule {\n  id: string\n  pattern: RegExp\n  /** Replacement — e.g. '[PII:email]'. Defaults to `[redacted:{id}]`. */\n  replacement?: string\n}\n\nexport interface RedactionReport {\n  redactionCount: number\n  byRule: Record<string, number>\n}\n\n/** OWASP / common-sense defaults — extend per-domain. */\nexport const DEFAULT_REDACTION_RULES: RedactionRule[] = [\n  { id: 'email', pattern: /\\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\\.[A-Z]{2,}\\b/gi },\n  { id: 'ssn', pattern: /\\b\\d{3}-\\d{2}-\\d{4}\\b/g },\n  { id: 'credit-card', pattern: /\\b(?:\\d[ -]*?){13,16}\\b/g },\n  { id: 'phone-us', pattern: /\\b(?:\\+?1[-.\\s]?)?\\(?\\d{3}\\)?[-.\\s]?\\d{3}[-.\\s]?\\d{4}\\b/g },\n  { id: 'ipv4', pattern: /\\b(?:\\d{1,3}\\.){3}\\d{1,3}\\b/g },\n  { id: 'aws-access-key', pattern: /\\bAKIA[0-9A-Z]{16}\\b/g },\n  { id: 'bearer', pattern: /\\bBearer\\s+[A-Za-z0-9._~+/=-]{10,}/gi },\n  { id: 'sk-key', pattern: /\\bsk-[A-Za-z0-9_-]{10,}\\b/g },\n  {\n    id: 'private-key-block',\n    pattern: /-----BEGIN (?:RSA |EC |OPENSSH |DSA )?PRIVATE KEY-----[\\s\\S]*?-----END[^-]*-----/g,\n  },\n]\n\nexport const REDACTION_VERSION = '1.0.0'\n\n/**\n * Redact a single string. Returns the new string and a per-rule count of\n * how many substitutions fired.\n */\nexport function redactString(\n  input: string,\n  rules: RedactionRule[] = DEFAULT_REDACTION_RULES,\n): { output: string; report: RedactionReport } {\n  const byRule: Record<string, number> = {}\n  let redactionCount = 0\n  let output = input\n  for (const rule of rules) {\n    let hits = 0\n    output = output.replace(rule.pattern, () => {\n      hits++\n      return rule.replacement ?? `[redacted:${rule.id}]`\n    })\n    if (hits > 0) {\n      byRule[rule.id] = hits\n      redactionCount += hits\n    }\n  }\n  return { output, report: { redactionCount, byRule } }\n}\n\n/**\n * Walk a JSON-ish value applying `redactString` to every string leaf.\n * Arrays and plain objects are recursed; other types pass through\n * untouched. Circular references throw — traces should be tree-shaped.\n */\nexport function redactValue(\n  value: unknown,\n  rules: RedactionRule[] = DEFAULT_REDACTION_RULES,\n  report: RedactionReport = { redactionCount: 0, byRule: {} },\n): { value: unknown; report: RedactionReport } {\n  if (typeof value === 'string') {\n    const { output, report: r } = redactString(value, rules)\n    report.redactionCount += r.redactionCount\n    for (const [k, v] of Object.entries(r.byRule)) {\n      report.byRule[k] = (report.byRule[k] ?? 0) + v\n    }\n    return { value: output, report }\n  }\n  if (Array.isArray(value)) {\n    return {\n      value: value.map((v) => redactValue(v, rules, report).value),\n      report,\n    }\n  }\n  if (value !== null && typeof value === 'object') {\n    const next: Record<string, unknown> = {}\n    for (const [k, v] of Object.entries(value)) {\n      next[k] = redactValue(v, rules, report).value\n    }\n    return { value: next, report }\n  }\n  return { value, report }\n}\n","/**\n * Replay-from-raw-events — turn every captured campaign run into a\n * re-runnable artifact.\n *\n * `RawProviderSink` captures every provider HTTP envelope; `runEvalCampaign`\n * makes that capture the default. Together they make every past run a\n * complete fingerprint of what happened on the wire — enough to replay\n * the run without burning new LLM cost.\n *\n * Three use cases this primitive enables:\n *\n *   1. **Post-hoc judging** — apply a new judge / rubric / scoring callback\n *      to last week's runs without re-calling any LLM. The cost of trying\n *      a new rubric drops from \"another full sweep\" to a CPU-bound replay.\n *   2. **Determinism audits** — replay the same campaign and verify the\n *      raw responses match byte-for-byte. Any drift is a non-determinism\n *      bug (in the harness, the prompt builder, the sandbox, …).\n *   3. **Free judge calibration** — run two judges on identical responses\n *      and measure inter-judge agreement without doubling LLM spend.\n *\n * The interface is deliberately fetch-shaped. Inject `createReplayFetch`\n * into `LlmClientOptions.fetch` and every `callLlm` transparently reads\n * from the cache instead of calling the network. No new code path through\n * the LLM client is needed; the cache hit is invisible to the runner.\n */\n\nimport { ReplayError } from './errors'\nimport { canonicalize, hashJson } from './pre-registration'\nimport type { RawProviderEvent, RawProviderSink } from './trace/raw-provider-sink'\n\nexport class ReplayCacheMissError extends ReplayError {\n  constructor(\n    public readonly url: string,\n    public readonly requestKey: string,\n    message?: string,\n  ) {\n    super(message ?? `replay cache miss for ${url} (key=${requestKey})`)\n  }\n}\n\nexport interface ReplayCacheEntry {\n  request: RawProviderEvent\n  response: RawProviderEvent\n}\n\nexport interface ReplayCacheStats {\n  total: number\n  byProvider: Record<string, number>\n  byModel: Record<string, number>\n  /** Spans for which we have a request but no response (run aborted mid-call). */\n  orphanRequests: number\n}\n\n/**\n * In-memory deterministic cache of (request → response) keyed on a stable\n * hash of the request body. Built from a `RawProviderSink` containing\n * paired `request` and `response` events from a previous run.\n *\n * The cache is the source of truth for replay; `createReplayFetch` is a\n * thin wrapper that reads from it.\n */\nexport class ReplayCache {\n  private byKey = new Map<string, ReplayCacheEntry>()\n  private orphans = 0\n  private byProvider: Record<string, number> = {}\n  private byModel: Record<string, number> = {}\n\n  /**\n   * Build a cache from a sink's events. The sink must implement `list()`.\n   * Filter by `runId` / `spanId` to scope to a specific replay.\n   */\n  static async fromSink(\n    sink: RawProviderSink,\n    filter: { runId?: string; spanId?: string } = {},\n  ): Promise<ReplayCache> {\n    if (!sink.list) {\n      throw new ReplayError('ReplayCache.fromSink: sink must implement list() to be replayable.')\n    }\n    const events = await sink.list(filter)\n    return ReplayCache.fromEvents(events)\n  }\n\n  /** Build a cache from an in-memory event list. */\n  static async fromEvents(events: RawProviderEvent[]): Promise<ReplayCache> {\n    const cache = new ReplayCache()\n    // Group by (runId, spanId, attemptIndex) so request/response/error pairs are matched.\n    // A cell can have many spans; each retry attempt within a span is a separate group.\n    type GroupKey = string\n    const groups = new Map<GroupKey, { req?: RawProviderEvent; res?: RawProviderEvent }>()\n    for (const e of events) {\n      const k = `${e.runId ?? ''}::${e.spanId ?? ''}::${e.attemptIndex}`\n      const g = groups.get(k) ?? {}\n      if (e.direction === 'request') g.req = e\n      else g.res = e\n      groups.set(k, g)\n    }\n    for (const g of groups.values()) {\n      if (!g.req) continue\n      if (!g.res) {\n        cache.orphans += 1\n        continue\n      }\n      const key = await requestKey(g.req)\n      cache.byKey.set(key, { request: g.req, response: g.res })\n      cache.byProvider[g.req.provider] = (cache.byProvider[g.req.provider] ?? 0) + 1\n      cache.byModel[g.req.model] = (cache.byModel[g.req.model] ?? 0) + 1\n    }\n    return cache\n  }\n\n  /** Number of cacheable (request, response) pairs in the cache. */\n  size(): number {\n    return this.byKey.size\n  }\n\n  stats(): ReplayCacheStats {\n    return {\n      total: this.byKey.size,\n      byProvider: { ...this.byProvider },\n      byModel: { ...this.byModel },\n      orphanRequests: this.orphans,\n    }\n  }\n\n  /** Iterate every cached `(request, response)` pair in insertion order. */\n  *entries(): IterableIterator<ReplayCacheEntry> {\n    for (const entry of this.byKey.values()) yield entry\n  }\n\n  /**\n   * Look up a cached response by hashing the (model, messages, temperature,\n   * maxTokens, response_format) shape. Returns `undefined` on miss; the\n   * caller decides whether to throw, fall back to the network, or skip.\n   */\n  async lookup(requestBody: unknown): Promise<ReplayCacheEntry | undefined> {\n    const key = await keyFromBody(requestBody)\n    return this.byKey.get(key)\n  }\n}\n\nexport interface ReplayFetchOptions {\n  /**\n   * Behaviour on cache miss. Default `'throw'`. `'fallback'` calls the\n   * `fallbackFetch` (typically `globalThis.fetch`) so a partial replay can\n   * still complete; `'fail-closed'` returns a synthetic 599 response so the\n   * call site sees a non-retriable failure.\n   */\n  onMiss?: 'throw' | 'fallback' | 'fail-closed'\n  fallbackFetch?: typeof fetch\n  /** Optional callback fired once per replayed call (for telemetry / counters). */\n  onHit?: (info: { url: string; provider: string; model: string }) => void\n  /** Optional callback fired on cache miss before the `onMiss` policy applies. */\n  onMissNotify?: (info: { url: string; requestBody: unknown }) => void\n}\n\n/**\n * Build a `fetch`-shaped function that serves cached responses out of a\n * `ReplayCache` for any URL ending in `/chat/completions`. Pass through\n * `LlmClientOptions.fetch` and `callLlm` becomes free.\n *\n * Non-`/chat/completions` URLs are passed straight to the fallback fetch\n * (default: `globalThis.fetch`). This matters because non-LLM HTTP work\n * (judge HTTP servers, sandbox callbacks) sometimes flows through the same\n * `fetch` and shouldn't be intercepted.\n */\nexport function createReplayFetch(cache: ReplayCache, opts: ReplayFetchOptions = {}): typeof fetch {\n  const onMiss = opts.onMiss ?? 'throw'\n  const fallback = opts.fallbackFetch ?? globalThis.fetch?.bind(globalThis)\n\n  return (async (input: RequestInfo | URL, init?: RequestInit) => {\n    const url =\n      typeof input === 'string' ? input : input instanceof URL ? input.toString() : input.url\n    if (!/\\/chat\\/completions(?:[?#].*)?$/.test(url)) {\n      if (!fallback)\n        throw new ReplayError(\n          `replay fetch: non-completions URL ${url} but no fallbackFetch configured`,\n        )\n      return fallback(input as RequestInfo, init)\n    }\n    let bodyParsed: unknown\n    if (init?.body && typeof init.body === 'string') {\n      try {\n        bodyParsed = JSON.parse(init.body)\n      } catch {\n        /* raw body, not JSON */\n      }\n    }\n    const hit = bodyParsed === undefined ? undefined : await cache.lookup(bodyParsed)\n    if (hit) {\n      opts.onHit?.({ url, provider: hit.request.provider, model: hit.request.model })\n      const status = hit.response.statusCode ?? 200\n      const headers = new Headers(\n        Object.entries(hit.response.responseHeaders ?? { 'Content-Type': 'application/json' }),\n      )\n      const bodyText =\n        typeof hit.response.responseBody === 'string'\n          ? hit.response.responseBody\n          : JSON.stringify(hit.response.responseBody ?? {})\n      return new Response(bodyText, { status, headers })\n    }\n    opts.onMissNotify?.({ url, requestBody: bodyParsed })\n    if (onMiss === 'throw') {\n      const key = bodyParsed === undefined ? '<unparseable>' : await keyFromBody(bodyParsed)\n      throw new ReplayCacheMissError(url, key)\n    }\n    if (onMiss === 'fail-closed') {\n      return new Response(JSON.stringify({ error: 'replay_cache_miss' }), { status: 599 })\n    }\n    if (!fallback)\n      throw new ReplayError('replay fetch: onMiss=fallback but no fallbackFetch configured')\n    return fallback(input as RequestInfo, init)\n  }) as typeof fetch\n}\n\n/**\n * Convenience iterator over `(request, response)` pairs in a sink — for\n * post-hoc scoring that doesn't need a `fetch` shim. The judge or scorer\n * runs purely in-process over cached LLM outputs.\n */\nexport async function* iterateRawCalls(\n  sink: RawProviderSink,\n  filter: { runId?: string; spanId?: string } = {},\n): AsyncGenerator<ReplayCacheEntry> {\n  if (!sink.list) {\n    throw new ReplayError('iterateRawCalls: sink must implement list().')\n  }\n  const events = await sink.list(filter)\n  const cache = await ReplayCache.fromEvents(events)\n  for (const entry of cache.entries()) yield entry\n}\n\n// ── Hashing ──────────────────────────────────────────────────────────────\n\n/**\n * Canonical request key.\n *\n * `model + messages + temperature + max_tokens|max_completion_tokens +\n * response_format` are the dimensions that affect the response shape.\n * Other fields (timestamp headers, provider-specific metadata) are\n * intentionally excluded so a request hashes the same across re-runs.\n */\nasync function requestKey(event: RawProviderEvent): Promise<string> {\n  return keyFromBody(event.requestBody)\n}\n\nasync function keyFromBody(body: unknown): Promise<string> {\n  if (body == null || typeof body !== 'object') return hashJson({ raw: String(body) })\n  const b = body as Record<string, unknown>\n  const reduced = canonicalize({\n    model: b.model ?? null,\n    messages: b.messages ?? null,\n    temperature: b.temperature ?? null,\n    max_tokens: b.max_tokens ?? null,\n    max_completion_tokens: b.max_completion_tokens ?? null,\n    response_format: b.response_format ?? null,\n  })\n  return hashJson(reduced)\n}\n"],"mappings":";;;;;;;;;AA0EO,IAAM,qBAAN,MAA+C;AAAA,EAC5C,OAAO,oBAAI,IAAiB;AAAA,EAC5B,WAAmB,CAAC;AAAA,EACpB,YAA0B,CAAC;AAAA,EAC3B,eAA2B,CAAC;AAAA,EAC5B,YAAiC,CAAC;AAAA,EAE1C,MAAM,UAAU,KAAyB;AACvC,QAAI,KAAK,KAAK,IAAI,IAAI,KAAK,EAAG,OAAM,IAAI,MAAM,OAAO,IAAI,KAAK,iBAAiB;AAC/E,SAAK,KAAK,IAAI,IAAI,OAAO,EAAE,GAAG,IAAI,CAAC;AAAA,EACrC;AAAA,EAEA,MAAM,UAAU,OAAe,OAAoC;AACjE,UAAM,WAAW,KAAK,KAAK,IAAI,KAAK;AACpC,QAAI,CAAC,SAAU,OAAM,IAAI,MAAM,OAAO,KAAK,YAAY;AACvD,SAAK,KAAK,IAAI,OAAO,EAAE,GAAG,UAAU,GAAG,MAAM,CAAC;AAAA,EAChD;AAAA,EAEA,MAAM,WAAW,MAA2B;AAC1C,SAAK,SAAS,KAAK,EAAE,GAAG,KAAK,CAAC;AAAA,EAChC;AAAA,EAEA,MAAM,WAAW,QAAgB,OAAqC;AACpE,UAAM,MAAM,KAAK,SAAS,UAAU,CAAC,MAAM,EAAE,WAAW,MAAM;AAC9D,QAAI,MAAM,EAAG,OAAM,IAAI,MAAM,QAAQ,MAAM,YAAY;AACvD,SAAK,SAAS,GAAG,IAAI,EAAE,GAAG,KAAK,SAAS,GAAG,GAAG,GAAG,MAAM;AAAA,EACzD;AAAA,EAEA,MAAM,YAAY,OAAkC;AAClD,SAAK,UAAU,KAAK,EAAE,GAAG,MAAM,CAAC;AAAA,EAClC;AAAA,EAEA,MAAM,eAAe,UAAmC;AACtD,SAAK,aAAa,KAAK,EAAE,GAAG,SAAS,CAAC;AAAA,EACxC;AAAA,EAEA,MAAM,kBAAkB,OAAyC;AAC/D,SAAK,UAAU,KAAK,EAAE,GAAG,MAAM,CAAC;AAAA,EAClC;AAAA,EAEA,MAAM,OAAO,OAAyC;AACpD,UAAM,IAAI,KAAK,KAAK,IAAI,KAAK;AAC7B,WAAO,IAAI,EAAE,GAAG,EAAE,IAAI;AAAA,EACxB;AAAA,EAEA,MAAM,SAAS,SAAoB,CAAC,GAAmB;AACrD,WAAO,CAAC,GAAG,KAAK,KAAK,OAAO,CAAC,EAAE,OAAO,CAAC,MAAM,WAAW,GAAG,MAAM,CAAC;AAAA,EACpE;AAAA,EAEA,MAAM,MAAM,SAAqB,CAAC,GAAoB;AACpD,WAAO,KAAK,SAAS,OAAO,CAAC,MAAM,YAAY,GAAG,MAAM,CAAC,EAAE,IAAI,CAAC,OAAO,EAAE,GAAG,EAAE,EAAE;AAAA,EAClF;AAAA,EAEA,MAAM,OAAO,SAAsB,CAAC,GAA0B;AAC5D,WAAO,KAAK,UAAU,OAAO,CAAC,MAAM,aAAa,GAAG,MAAM,CAAC,EAAE,IAAI,CAAC,OAAO,EAAE,GAAG,EAAE,EAAE;AAAA,EACpF;AAAA,EAEA,MAAM,OAAO,OAA6C;AACxD,WAAO,KAAK,UAAU,OAAO,CAAC,MAAM,EAAE,UAAU,KAAK,EAAE,IAAI,CAAC,OAAO,EAAE,GAAG,EAAE,EAAE;AAAA,EAC9E;AAAA,EAEA,MAAM,UAAU,OAAoC;AAClD,WAAO,KAAK,aAAa,OAAO,CAAC,MAAM,EAAE,UAAU,KAAK,EAAE,IAAI,CAAC,OAAO,EAAE,GAAG,EAAE,EAAE;AAAA,EACjF;AACF;AAEA,SAAS,WAAW,GAAQ,GAAuB;AACjD,MAAI,EAAE,cAAc,EAAE,eAAe,EAAE,WAAY,QAAO;AAC1D,MAAI,EAAE,aAAa,EAAE,cAAc,EAAE,UAAW,QAAO;AACvD,MAAI,EAAE,UAAU,EAAE,WAAW,EAAE,OAAQ,QAAO;AAC9C,MAAI,EAAE,UAAU,UAAa,EAAE,YAAY,EAAE,MAAO,QAAO;AAC3D,MAAI,EAAE,UAAU,UAAa,EAAE,YAAY,EAAE,MAAO,QAAO;AAC3D,MAAI,EAAE,OAAO,EAAE,OAAO,EAAE,IAAI,GAAG,MAAM,EAAE,IAAI,MAAO,QAAO;AACzD,MAAI,EAAE,eAAe,EAAE,gBAAgB,EAAE,YAAa,QAAO;AAC7D,MAAI,EAAE,aAAa,EAAE,cAAc,EAAE,UAAW,QAAO;AACvD,MAAI,EAAE,UAAU,EAAE,WAAW,EAAE,OAAQ,QAAO;AAC9C,MAAI,EAAE,SAAS,EAAE,UAAU,EAAE,MAAO,QAAO;AAC3C,SAAO;AACT;AAEA,SAAS,YAAY,GAAS,GAAwB;AACpD,MAAI,EAAE,SAAS,EAAE,UAAU,EAAE,MAAO,QAAO;AAC3C,MAAI,EAAE,gBAAgB,EAAE,iBAAiB,EAAE,aAAc,QAAO;AAChE,MAAI,EAAE,QAAQ,EAAE,SAAS,EAAE,KAAM,QAAO;AACxC,MAAI,EAAE,QAAQ,EAAE,SAAS,EAAE,KAAM,QAAO;AACxC,MAAI,EAAE,aAAa,EAAE,SAAS,UAAU,EAAE,aAAa,EAAE,UAAW,QAAO;AAC3E,MAAI,EAAE,YAAY,EAAE,SAAS,WAAW,EAAE,YAAY,EAAE,SAAU,QAAO;AACzE,MAAI,EAAE,UAAU,UAAa,EAAE,YAAY,EAAE,MAAO,QAAO;AAC3D,MAAI,EAAE,UAAU,UAAa,EAAE,YAAY,EAAE,MAAO,QAAO;AAC3D,SAAO;AACT;AAEA,SAAS,aAAa,GAAe,GAAyB;AAC5D,MAAI,EAAE,SAAS,EAAE,UAAU,EAAE,MAAO,QAAO;AAC3C,MAAI,EAAE,UAAU,EAAE,WAAW,EAAE,OAAQ,QAAO;AAC9C,MAAI,EAAE,QAAQ,EAAE,SAAS,EAAE,KAAM,QAAO;AACxC,MAAI,EAAE,UAAU,UAAa,EAAE,YAAY,EAAE,MAAO,QAAO;AAC3D,MAAI,EAAE,UAAU,UAAa,EAAE,YAAY,EAAE,MAAO,QAAO;AAC3D,SAAO;AACT;AAUO,IAAM,uBAAN,MAAiD;AAAA,EAC9C;AAAA,EACA;AAAA;AAAA,EAEA;AAAA,EACA,SAAS;AAAA,EAEjB,YAAY,SAAsC;AAChD,SAAK,MAAM,QAAQ;AACnB,SAAK,WAAW,QAAQ,YAAY,KAAK,OAAO;AAAA,EAClD;AAAA,EAEA,MAAc,YAA2B;AACvC,UAAM,KAAK,MAAM,OAAO,aAAkB;AAC1C,UAAM,GAAG,MAAM,KAAK,KAAK,EAAE,WAAW,KAAK,CAAC;AAAA,EAC9C;AAAA,EAEA,MAAc,OAAO,MAAc,QAAgC;AACjE,UAAM,KAAK,UAAU;AACrB,UAAM,KAAK,MAAM,OAAO,aAAkB;AAC1C,UAAM,OAAO,MAAM,OAAO,MAAW;AACrC,UAAM,SAAS,KAAK,KAAK,KAAK,KAAK,GAAG,IAAI,SAAS;AACnD,QAAI;AACF,YAAM,OAAO,MAAM,GAAG,KAAK,MAAM;AACjC,UAAI,KAAK,QAAQ,KAAK,UAAU;AAC9B,cAAM,SAAS,KAAK,KAAK,KAAK,KAAK,GAAG,IAAI,IAAI,KAAK,IAAI,CAAC,SAAS;AACjE,cAAM,GAAG,OAAO,QAAQ,MAAM;AAAA,MAChC;AAAA,IACF,QAAQ;AAAA,IAER;AACA,UAAM,GAAG,WAAW,QAAQ,GAAG,KAAK,UAAU,MAAM,CAAC;AAAA,GAAM,MAAM;AAMjE,QAAI,KAAK,SAAS,CAAE,QAAkC,SAAS;AAC7D,WAAK,KAAK,WAAW,MAAM,MAAM;AAAA,IACnC;AAAA,EACF;AAAA,EAEA,MAAc,WAAW,MAAc,QAAgC;AACrE,QAAI,CAAC,KAAK,MAAO;AACjB,YAAQ,MAAM;AAAA,MACZ,KAAK;AACH,cAAM,KAAK,MAAM,UAAU,MAAa;AACxC;AAAA,MACF,KAAK;AACH,cAAM,KAAK,MAAM,WAAW,MAAc;AAC1C;AAAA,MACF,KAAK;AACH,cAAM,KAAK,MAAM,YAAY,MAAoB;AACjD;AAAA,MACF,KAAK;AACH,cAAM,KAAK,MAAM,eAAe,MAAkB;AAClD;AAAA,MACF,KAAK;AACH,cAAM,KAAK,MAAM,kBAAkB,MAA2B;AAC9D;AAAA,IACJ;AAAA,EACF;AAAA,EAEA,MAAc,OAAoC;AAChD,QAAI,KAAK,UAAU,KAAK,MAAO,QAAO,KAAK;AAC3C,UAAM,KAAK,MAAM,OAAO,aAAkB;AAC1C,UAAM,OAAO,MAAM,OAAO,MAAW;AACrC,UAAM,QAAQ,IAAI,mBAAmB;AACrC,QAAI;AACF,YAAM,UAAU,MAAM,GAAG,QAAQ,KAAK,GAAG;AACzC,iBAAW,QAAQ,SAAS;AAC1B,YAAI,CAAC,KAAK,SAAS,SAAS,EAAG;AAC/B,cAAM,OAAO,KAAK,KAAK,KAAK,KAAK,IAAI;AACrC,cAAM,UAAU,MAAM,GAAG,SAAS,MAAM,MAAM;AAC9C,cAAM,OAAO,KAAK,MAAM,GAAG,EAAE,CAAC;AAC9B,mBAAW,QAAQ,QAAQ,MAAM,IAAI,GAAG;AACtC,cAAI,CAAC,KAAK,KAAK,EAAG;AAClB,gBAAM,SAAS,KAAK,MAAM,IAAI;AAC9B,cAAI,SAAS,QAAQ;AAEnB,gBAAI;AACF,oBAAM,MAAM,UAAU,MAAM;AAAA,YAC9B,QAAQ;AACN,oBAAM,MAAM,UAAU,OAAO,OAAO,MAAM;AAAA,YAC5C;AAAA,UACF,WAAW,SAAS,SAAS;AAQ3B,gBAAI,QAAQ,SAAS;AACnB,kBAAI;AACF,sBAAM,MAAM,WAAW,OAAO,QAAQ,MAAM;AAAA,cAC9C,QAAQ;AAIN,sBAAM,MAAM,WAAW,MAAM;AAAA,cAC/B;AAAA,YACF,OAAO;AACL,oBAAM,MAAM,WAAW,MAAM;AAAA,YAC/B;AAAA,UACF,WAAW,SAAS,UAAU;AAC5B,kBAAM,MAAM,YAAY,MAAM;AAAA,UAChC,WAAW,SAAS,aAAa;AAC/B,kBAAM,MAAM,eAAe,MAAM;AAAA,UACnC,WAAW,SAAS,UAAU;AAC5B,kBAAM,MAAM,kBAAkB,MAAM;AAAA,UACtC;AAAA,QACF;AAAA,MACF;AAAA,IACF,QAAQ;AAAA,IAER;AACA,SAAK,QAAQ;AACb,SAAK,SAAS;AACd,WAAO;AAAA,EACT;AAAA,EAEA,MAAM,UAAU,KAAyB;AACvC,UAAM,KAAK,OAAO,QAAQ,GAAG;AAAA,EAC/B;AAAA,EACA,MAAM,UAAU,OAAe,OAAoC;AAGjE,UAAM,KAAK,OAAO,QAAQ,EAAE,OAAO,GAAG,OAAO,SAAS,KAAK,CAAC;AAC5D,QAAI,KAAK,MAAO,OAAM,KAAK,MAAM,UAAU,OAAO,KAAK;AAAA,EACzD;AAAA,EACA,MAAM,WAAW,MAA2B;AAC1C,UAAM,KAAK,OAAO,SAAS,IAAI;AAAA,EACjC;AAAA,EACA,MAAM,WAAW,QAAgB,OAAqC;AACpE,UAAM,KAAK,OAAO,SAAS,EAAE,QAAQ,GAAG,OAAO,SAAS,KAAK,CAAC;AAC9D,QAAI,KAAK,MAAO,OAAM,KAAK,MAAM,WAAW,QAAQ,KAAK;AAAA,EAC3D;AAAA,EACA,MAAM,YAAY,OAAkC;AAClD,UAAM,KAAK,OAAO,UAAU,KAAK;AAAA,EACnC;AAAA,EACA,MAAM,eAAe,UAAmC;AACtD,UAAM,KAAK,OAAO,aAAa,QAAQ;AAAA,EACzC;AAAA,EACA,MAAM,kBAAkB,OAAyC;AAC/D,UAAM,KAAK,OAAO,UAAU,KAAK;AAAA,EACnC;AAAA,EAEA,MAAM,OAAO,OAAyC;AACpD,YAAQ,MAAM,KAAK,KAAK,GAAG,OAAO,KAAK;AAAA,EACzC;AAAA,EACA,MAAM,SAAS,QAAoC;AACjD,YAAQ,MAAM,KAAK,KAAK,GAAG,SAAS,MAAM;AAAA,EAC5C;AAAA,EACA,MAAM,MAAM,QAAsC;AAChD,YAAQ,MAAM,KAAK,KAAK,GAAG,MAAM,MAAM;AAAA,EACzC;AAAA,EACA,MAAM,OAAO,QAA6C;AACxD,YAAQ,MAAM,KAAK,KAAK,GAAG,OAAO,MAAM;AAAA,EAC1C;AAAA,EACA,MAAM,OAAO,OAA6C;AACxD,YAAQ,MAAM,KAAK,KAAK,GAAG,OAAO,KAAK;AAAA,EACzC;AAAA,EACA,MAAM,UAAU,OAAoC;AAClD,YAAQ,MAAM,KAAK,KAAK,GAAG,UAAU,KAAK;AAAA,EAC5C;AACF;;;AC/UO,IAAM,wBAAwB,EAAE,MAAM,8BAA8B,SAAS,QAAQ;AA4B5F,eAAsB,gBACpB,OACA,OACA,gBAA2D,CAAC,GACvC;AACrB,QAAM,MAAM,MAAM,MAAM,OAAO,KAAK;AACpC,MAAI,CAAC,IAAK,OAAM,IAAI,MAAM,OAAO,KAAK,YAAY;AAClD,QAAM,QAAQ,MAAM,MAAM,MAAM,EAAE,MAAM,CAAC;AACzC,QAAM,SAAS,MAAM,MAAM,OAAO,EAAE,MAAM,CAAC;AAC3C,QAAM,eAAe,oBAAI,IAA0B;AACnD,aAAW,KAAK,QAAQ;AACtB,QAAI,CAAC,EAAE,OAAQ;AACf,UAAM,MAAM,aAAa,IAAI,EAAE,MAAM,KAAK,CAAC;AAC3C,QAAI,KAAK,CAAC;AACV,iBAAa,IAAI,EAAE,QAAQ,GAAG;AAAA,EAChC;AACA,QAAM,UAAU,aAAa,GAAG;AAChC,QAAM,YAAwB,MAAM;AAAA,IAAI,CAAC,MACvC,WAAW,GAAG,SAAS,aAAa,IAAI,EAAE,MAAM,KAAK,CAAC,CAAC;AAAA,EACzD;AACA,SAAO;AAAA,IACL,eAAe;AAAA,MACb;AAAA,QACE,UAAU;AAAA,UACR,YAAY,aAAa;AAAA,YACvB,gBAAgB;AAAA,YAChB,UAAU,IAAI;AAAA,YACd,mBAAmB,IAAI;AAAA,YACvB,kBAAkB,IAAI,aAAa;AAAA,YACnC,uBAAuB,IAAI,kBAAkB;AAAA,YAC7C,gBAAgB,IAAI,WAAW;AAAA,YAC/B,yBAAyB,IAAI,oBAAoB;AAAA,YACjD,GAAG;AAAA,UACL,CAAC;AAAA,QACH;AAAA,QACA,YAAY,CAAC,EAAE,OAAO,uBAAuB,OAAO,UAAU,CAAC;AAAA,MACjE;AAAA,IACF;AAAA,EACF;AACF;AAEA,SAAS,WAAW,MAAY,SAAiB,QAAgC;AAC/E,QAAM,UAAU,KAAK,WAAW,KAAK;AACrC,SAAO;AAAA,IACL;AAAA,IACA,QAAQ,UAAU,KAAK,MAAM;AAAA,IAC7B,cAAc,KAAK,eAAe,UAAU,KAAK,YAAY,IAAI;AAAA,IACjE,MAAM,KAAK;AAAA,IACX,MAAM;AAAA;AAAA,IACN,mBAAmB,OAAO,KAAK,SAAS;AAAA,IACxC,iBAAiB,OAAO,OAAO;AAAA,IAC/B,YAAY,aAAa,sBAAsB,IAAI,CAAC;AAAA,IACpD,QAAQ,OAAO,IAAI,CAAC,OAAO;AAAA,MACzB,cAAc,OAAO,EAAE,SAAS;AAAA,MAChC,MAAM,EAAE;AAAA,MACR,YAAY,aAAa,eAAe,EAAE,OAAO,CAAC;AAAA,IACpD,EAAE;AAAA,IACF,QAAQ,KAAK,WAAW,UAAU,EAAE,MAAM,GAAG,SAAS,KAAK,MAAM,IAAI,EAAE,MAAM,EAAE;AAAA,EACjF;AACF;AAEA,SAAS,sBAAsB,MAAuD;AACpF,QAAM,OAAkD;AAAA,IACtD,aAAa,KAAK;AAAA,EACpB;AACA,MAAI,KAAK,SAAS,OAAO;AACvB,SAAK,WAAW,IAAI,KAAK;AACzB,QAAI,KAAK,gBAAgB,OAAW,MAAK,kBAAkB,IAAI,KAAK;AACpE,QAAI,KAAK,iBAAiB,OAAW,MAAK,mBAAmB,IAAI,KAAK;AACtE,QAAI,KAAK,YAAY,OAAW,MAAK,cAAc,IAAI,KAAK;AAC5D,QAAI,KAAK,aAAc,MAAK,mBAAmB,IAAI,KAAK;AAAA,EAC1D,WAAW,KAAK,SAAS,QAAQ;AAC/B,SAAK,WAAW,IAAI,KAAK;AACzB,QAAI,KAAK,cAAc,OAAW,MAAK,iBAAiB,IAAI,KAAK;AAAA,EACnE,WAAW,KAAK,SAAS,aAAa;AACpC,SAAK,iBAAiB,IAAI,KAAK;AAC/B,SAAK,gBAAgB,IAAI,KAAK,KAAK;AAAA,EACrC,WAAW,KAAK,SAAS,SAAS;AAChC,SAAK,UAAU,IAAI,KAAK;AACxB,SAAK,iBAAiB,IAAI,KAAK;AAC/B,SAAK,aAAa,IAAI,KAAK;AAC3B,SAAK,sBAAsB,IAAI,KAAK;AAAA,EACtC,WAAW,KAAK,SAAS,WAAW;AAClC,QAAI,KAAK,MAAO,MAAK,eAAe,IAAI,KAAK;AAC7C,QAAI,KAAK,aAAa,OAAW,MAAK,mBAAmB,IAAI,KAAK;AAClE,QAAI,KAAK,gBAAgB,OAAW,MAAK,sBAAsB,IAAI,KAAK;AACxE,QAAI,KAAK,eAAe,OAAW,MAAK,qBAAqB,IAAI,KAAK;AAAA,EACxE;AACA,MAAI,KAAK,YAAY;AACnB,eAAW,CAAC,GAAG,CAAC,KAAK,OAAO,QAAQ,KAAK,UAAU,GAAG;AACpD,UAAI,OAAO,MAAM,YAAY,OAAO,MAAM,YAAY,OAAO,MAAM,UAAW,MAAK,CAAC,IAAI;AAAA,IAC1F;AAAA,EACF;AACA,SAAO;AACT;AAEA,SAAS,eACP,SAC2C;AAC3C,QAAM,MAAiD,CAAC;AACxD,aAAW,CAAC,GAAG,CAAC,KAAK,OAAO,QAAQ,OAAO,GAAG;AAC5C,QAAI,OAAO,MAAM,YAAY,OAAO,MAAM,YAAY,OAAO,MAAM,UAAW,KAAI,CAAC,IAAI;AAAA,QAClF,KAAI,CAAC,IAAI,KAAK,UAAU,CAAC;AAAA,EAChC;AACA,SAAO;AACT;AAEA,SAAS,aAAa,QAA2E;AAC/F,SAAO,OAAO,QAAQ,MAAM,EAAE,IAAI,CAAC,CAAC,KAAK,KAAK,OAAO;AAAA,IACnD;AAAA,IACA,OACE,OAAO,UAAU,WACb,OAAO,UAAU,KAAK,IACpB,EAAE,UAAU,MAAM,SAAS,EAAE,IAC7B,EAAE,aAAa,MAAM,IACvB,OAAO,UAAU,YACf,EAAE,WAAW,MAAM,IACnB,EAAE,aAAa,MAAM;AAAA,EAC/B,EAAE;AACJ;AAEA,SAAS,OAAO,IAAoB;AAClC,UAAQ,OAAO,KAAK,MAAM,EAAE,CAAC,IAAI,UAAY,SAAS;AACxD;AAEA,SAAS,UAAU,IAAoB;AAErC,QAAM,UAAU,GAAG,QAAQ,MAAM,EAAE;AACnC,SAAO,QAAQ,MAAM,GAAG,EAAE,EAAE,OAAO,IAAI,GAAG;AAC5C;AAEA,SAAS,aAAa,KAAkB;AAGtC,QAAM,UAAU,IAAI,MAAM,QAAQ,MAAM,EAAE;AAC1C,SAAO,QAAQ,MAAM,GAAG,EAAE,EAAE,OAAO,IAAI,GAAG;AAC5C;;;ACvJO,IAAM,0BAA2C;AAAA,EACtD,EAAE,IAAI,SAAS,SAAS,8CAA8C;AAAA,EACtE,EAAE,IAAI,OAAO,SAAS,yBAAyB;AAAA,EAC/C,EAAE,IAAI,eAAe,SAAS,2BAA2B;AAAA,EACzD,EAAE,IAAI,YAAY,SAAS,2DAA2D;AAAA,EACtF,EAAE,IAAI,QAAQ,SAAS,+BAA+B;AAAA,EACtD,EAAE,IAAI,kBAAkB,SAAS,wBAAwB;AAAA,EACzD,EAAE,IAAI,UAAU,SAAS,uCAAuC;AAAA,EAChE,EAAE,IAAI,UAAU,SAAS,6BAA6B;AAAA,EACtD;AAAA,IACE,IAAI;AAAA,IACJ,SAAS;AAAA,EACX;AACF;AAEO,IAAM,oBAAoB;AAM1B,SAAS,aACd,OACA,QAAyB,yBACoB;AAC7C,QAAM,SAAiC,CAAC;AACxC,MAAI,iBAAiB;AACrB,MAAI,SAAS;AACb,aAAW,QAAQ,OAAO;AACxB,QAAI,OAAO;AACX,aAAS,OAAO,QAAQ,KAAK,SAAS,MAAM;AAC1C;AACA,aAAO,KAAK,eAAe,aAAa,KAAK,EAAE;AAAA,IACjD,CAAC;AACD,QAAI,OAAO,GAAG;AACZ,aAAO,KAAK,EAAE,IAAI;AAClB,wBAAkB;AAAA,IACpB;AAAA,EACF;AACA,SAAO,EAAE,QAAQ,QAAQ,EAAE,gBAAgB,OAAO,EAAE;AACtD;AAOO,SAAS,YACd,OACA,QAAyB,yBACzB,SAA0B,EAAE,gBAAgB,GAAG,QAAQ,CAAC,EAAE,GACb;AAC7C,MAAI,OAAO,UAAU,UAAU;AAC7B,UAAM,EAAE,QAAQ,QAAQ,EAAE,IAAI,aAAa,OAAO,KAAK;AACvD,WAAO,kBAAkB,EAAE;AAC3B,eAAW,CAAC,GAAG,CAAC,KAAK,OAAO,QAAQ,EAAE,MAAM,GAAG;AAC7C,aAAO,OAAO,CAAC,KAAK,OAAO,OAAO,CAAC,KAAK,KAAK;AAAA,IAC/C;AACA,WAAO,EAAE,OAAO,QAAQ,OAAO;AAAA,EACjC;AACA,MAAI,MAAM,QAAQ,KAAK,GAAG;AACxB,WAAO;AAAA,MACL,OAAO,MAAM,IAAI,CAAC,MAAM,YAAY,GAAG,OAAO,MAAM,EAAE,KAAK;AAAA,MAC3D;AAAA,IACF;AAAA,EACF;AACA,MAAI,UAAU,QAAQ,OAAO,UAAU,UAAU;AAC/C,UAAM,OAAgC,CAAC;AACvC,eAAW,CAAC,GAAG,CAAC,KAAK,OAAO,QAAQ,KAAK,GAAG;AAC1C,WAAK,CAAC,IAAI,YAAY,GAAG,OAAO,MAAM,EAAE;AAAA,IAC1C;AACA,WAAO,EAAE,OAAO,MAAM,OAAO;AAAA,EAC/B;AACA,SAAO,EAAE,OAAO,OAAO;AACzB;;;ACvEO,IAAM,uBAAN,cAAmC,YAAY;AAAA,EACpD,YACkB,KACAA,aAChB,SACA;AACA,UAAM,WAAW,yBAAyB,GAAG,SAASA,WAAU,GAAG;AAJnD;AACA,sBAAAA;AAAA,EAIlB;AAAA,EALkB;AAAA,EACA;AAKpB;AAuBO,IAAM,cAAN,MAAM,aAAY;AAAA,EACf,QAAQ,oBAAI,IAA8B;AAAA,EAC1C,UAAU;AAAA,EACV,aAAqC,CAAC;AAAA,EACtC,UAAkC,CAAC;AAAA;AAAA;AAAA;AAAA;AAAA,EAM3C,aAAa,SACX,MACA,SAA8C,CAAC,GACzB;AACtB,QAAI,CAAC,KAAK,MAAM;AACd,YAAM,IAAI,YAAY,oEAAoE;AAAA,IAC5F;AACA,UAAM,SAAS,MAAM,KAAK,KAAK,MAAM;AACrC,WAAO,aAAY,WAAW,MAAM;AAAA,EACtC;AAAA;AAAA,EAGA,aAAa,WAAW,QAAkD;AACxE,UAAM,QAAQ,IAAI,aAAY;AAI9B,UAAM,SAAS,oBAAI,IAAkE;AACrF,eAAW,KAAK,QAAQ;AACtB,YAAM,IAAI,GAAG,EAAE,SAAS,EAAE,KAAK,EAAE,UAAU,EAAE,KAAK,EAAE,YAAY;AAChE,YAAM,IAAI,OAAO,IAAI,CAAC,KAAK,CAAC;AAC5B,UAAI,EAAE,cAAc,UAAW,GAAE,MAAM;AAAA,UAClC,GAAE,MAAM;AACb,aAAO,IAAI,GAAG,CAAC;AAAA,IACjB;AACA,eAAW,KAAK,OAAO,OAAO,GAAG;AAC/B,UAAI,CAAC,EAAE,IAAK;AACZ,UAAI,CAAC,EAAE,KAAK;AACV,cAAM,WAAW;AACjB;AAAA,MACF;AACA,YAAM,MAAM,MAAM,WAAW,EAAE,GAAG;AAClC,YAAM,MAAM,IAAI,KAAK,EAAE,SAAS,EAAE,KAAK,UAAU,EAAE,IAAI,CAAC;AACxD,YAAM,WAAW,EAAE,IAAI,QAAQ,KAAK,MAAM,WAAW,EAAE,IAAI,QAAQ,KAAK,KAAK;AAC7E,YAAM,QAAQ,EAAE,IAAI,KAAK,KAAK,MAAM,QAAQ,EAAE,IAAI,KAAK,KAAK,KAAK;AAAA,IACnE;AACA,WAAO;AAAA,EACT;AAAA;AAAA,EAGA,OAAe;AACb,WAAO,KAAK,MAAM;AAAA,EACpB;AAAA,EAEA,QAA0B;AACxB,WAAO;AAAA,MACL,OAAO,KAAK,MAAM;AAAA,MAClB,YAAY,EAAE,GAAG,KAAK,WAAW;AAAA,MACjC,SAAS,EAAE,GAAG,KAAK,QAAQ;AAAA,MAC3B,gBAAgB,KAAK;AAAA,IACvB;AAAA,EACF;AAAA;AAAA,EAGA,CAAC,UAA8C;AAC7C,eAAW,SAAS,KAAK,MAAM,OAAO,EAAG,OAAM;AAAA,EACjD;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOA,MAAM,OAAO,aAA6D;AACxE,UAAM,MAAM,MAAM,YAAY,WAAW;AACzC,WAAO,KAAK,MAAM,IAAI,GAAG;AAAA,EAC3B;AACF;AA2BO,SAAS,kBAAkB,OAAoB,OAA2B,CAAC,GAAiB;AACjG,QAAM,SAAS,KAAK,UAAU;AAC9B,QAAM,WAAW,KAAK,iBAAiB,WAAW,OAAO,KAAK,UAAU;AAExE,UAAQ,OAAO,OAA0B,SAAuB;AAC9D,UAAM,MACJ,OAAO,UAAU,WAAW,QAAQ,iBAAiB,MAAM,MAAM,SAAS,IAAI,MAAM;AACtF,QAAI,CAAC,kCAAkC,KAAK,GAAG,GAAG;AAChD,UAAI,CAAC;AACH,cAAM,IAAI;AAAA,UACR,qCAAqC,GAAG;AAAA,QAC1C;AACF,aAAO,SAAS,OAAsB,IAAI;AAAA,IAC5C;AACA,QAAI;AACJ,QAAI,MAAM,QAAQ,OAAO,KAAK,SAAS,UAAU;AAC/C,UAAI;AACF,qBAAa,KAAK,MAAM,KAAK,IAAI;AAAA,MACnC,QAAQ;AAAA,MAER;AAAA,IACF;AACA,UAAM,MAAM,eAAe,SAAY,SAAY,MAAM,MAAM,OAAO,UAAU;AAChF,QAAI,KAAK;AACP,WAAK,QAAQ,EAAE,KAAK,UAAU,IAAI,QAAQ,UAAU,OAAO,IAAI,QAAQ,MAAM,CAAC;AAC9E,YAAM,SAAS,IAAI,SAAS,cAAc;AAC1C,YAAM,UAAU,IAAI;AAAA,QAClB,OAAO,QAAQ,IAAI,SAAS,mBAAmB,EAAE,gBAAgB,mBAAmB,CAAC;AAAA,MACvF;AACA,YAAM,WACJ,OAAO,IAAI,SAAS,iBAAiB,WACjC,IAAI,SAAS,eACb,KAAK,UAAU,IAAI,SAAS,gBAAgB,CAAC,CAAC;AACpD,aAAO,IAAI,SAAS,UAAU,EAAE,QAAQ,QAAQ,CAAC;AAAA,IACnD;AACA,SAAK,eAAe,EAAE,KAAK,aAAa,WAAW,CAAC;AACpD,QAAI,WAAW,SAAS;AACtB,YAAM,MAAM,eAAe,SAAY,kBAAkB,MAAM,YAAY,UAAU;AACrF,YAAM,IAAI,qBAAqB,KAAK,GAAG;AAAA,IACzC;AACA,QAAI,WAAW,eAAe;AAC5B,aAAO,IAAI,SAAS,KAAK,UAAU,EAAE,OAAO,oBAAoB,CAAC,GAAG,EAAE,QAAQ,IAAI,CAAC;AAAA,IACrF;AACA,QAAI,CAAC;AACH,YAAM,IAAI,YAAY,+DAA+D;AACvF,WAAO,SAAS,OAAsB,IAAI;AAAA,EAC5C;AACF;AAOA,gBAAuB,gBACrB,MACA,SAA8C,CAAC,GACb;AAClC,MAAI,CAAC,KAAK,MAAM;AACd,UAAM,IAAI,YAAY,8CAA8C;AAAA,EACtE;AACA,QAAM,SAAS,MAAM,KAAK,KAAK,MAAM;AACrC,QAAM,QAAQ,MAAM,YAAY,WAAW,MAAM;AACjD,aAAW,SAAS,MAAM,QAAQ,EAAG,OAAM;AAC7C;AAYA,eAAe,WAAW,OAA0C;AAClE,SAAO,YAAY,MAAM,WAAW;AACtC;AAEA,eAAe,YAAY,MAAgC;AACzD,MAAI,QAAQ,QAAQ,OAAO,SAAS,SAAU,QAAO,SAAS,EAAE,KAAK,OAAO,IAAI,EAAE,CAAC;AACnF,QAAM,IAAI;AACV,QAAM,UAAU,aAAa;AAAA,IAC3B,OAAO,EAAE,SAAS;AAAA,IAClB,UAAU,EAAE,YAAY;AAAA,IACxB,aAAa,EAAE,eAAe;AAAA,IAC9B,YAAY,EAAE,cAAc;AAAA,IAC5B,uBAAuB,EAAE,yBAAyB;AAAA,IAClD,iBAAiB,EAAE,mBAAmB;AAAA,EACxC,CAAC;AACD,SAAO,SAAS,OAAO;AACzB;","names":["requestKey"]}

package/dist/{chunk-2A5XJB43.js → chunk-5AKPEK5L.js}

@@ -2,11 +2,11 @@ import {
   cohensD,
   confidenceInterval,
   wilcoxonSignedRank
-} from "./chunk-I4MBDTY5.js";
+} from "./chunk-R5UQJNKC.js";
 import {
   canonicalize,
   hashJson
-} from "./chunk-4F5DQN55.js";
+} from "./chunk-VSMTAMNK.js";
 
 // src/power-analysis.ts
 function requiredSampleSize(opts) {
@@ -1047,4 +1047,4 @@ export {
   RESEARCH_REPORT_HARD_PAIR_FLOOR,
   researchReport
 };
-//# sourceMappingURL=chunk-2A5XJB43.js.map
+//# sourceMappingURL=chunk-5AKPEK5L.js.map

package/dist/chunk-5AKPEK5L.js.map

@@ -0,0 +1 @@
+{"version":3,"sources":["../src/power-analysis.ts","../src/paired-stats.ts","../src/summary-report.ts"],"sourcesContent":["/**\n * Power analysis + multiple-comparison correction.\n *\n * Two jobs:\n *   1. Before running: `requiredSampleSize({ effect, alpha, power })`\n *      returns the N per arm needed to detect a given effect size.\n *   2. After running: `benjaminiHochberg(pValues, fdr)` and\n *      `bonferroni(pValues, alpha)` correct for multiple pairwise tests\n *      so pairwise variant comparisons stay statistically honest.\n *\n * Applying alpha directly across n*(n-1)/2 pairwise tests without\n * correction inflates the false-positive rate when variants ≥ 3 — the\n * BH and Bonferroni helpers prevent that.\n */\n\n/**\n * Required N per arm for a two-sample comparison at target effect size,\n * alpha, and power. Uses the normal-approximation formula:\n *\n *   n = 2 * ( (z_{1-α/2} + z_{1-β}) / d )^2\n *\n * where d is Cohen's d. Returns Infinity for effect ≤ 0.\n */\nexport function requiredSampleSize(opts: {\n  effect: number\n  alpha?: number\n  power?: number\n  twoSided?: boolean\n}): number {\n  const effect = opts.effect\n  if (!Number.isFinite(effect) || effect <= 0) return Infinity\n  const alpha = opts.alpha ?? 0.05\n  const power = opts.power ?? 0.8\n  const twoSided = opts.twoSided ?? true\n  const zAlpha = zQuantile(twoSided ? 1 - alpha / 2 : 1 - alpha)\n  const zBeta = zQuantile(power)\n  const n = 2 * ((zAlpha + zBeta) / effect) ** 2\n  return Math.ceil(n)\n}\n\n/**\n * Minimum detectable paired effect (in standardised units) given a target\n * paired sample size. Closed-form inverse of the paired-t / sign-rank power\n * formula under the normal approximation:\n *\n *   d_min = (z_{1-α/2} + z_β) / sqrt(n_paired)\n *\n * Multiply by `sd(deltas)` to convert to score units. Treat as a lower bound:\n * the Wilcoxon signed-rank test and bootstrap CIs have asymptotic relative\n * efficiency below 1 against the t-test on heavy-tailed distributions, so the\n * true achievable MDE in those regimes is somewhat larger.\n */\nexport function pairedMde(opts: {\n  nPaired: number\n  alpha?: number\n  power?: number\n  twoSided?: boolean\n}): number {\n  if (!Number.isFinite(opts.nPaired) || opts.nPaired <= 0) return Infinity\n  const alpha = opts.alpha ?? 0.05\n  const power = opts.power ?? 0.8\n  const twoSided = opts.twoSided ?? true\n  const zAlpha = zQuantile(twoSided ? 1 - alpha / 2 : 1 - alpha)\n  const zBeta = zQuantile(power)\n  return (zAlpha + zBeta) / Math.sqrt(opts.nPaired)\n}\n\n/** Bonferroni adjustment: multiply every p-value by the number of tests, clamp at 1. */\nexport function bonferroni(\n  pValues: number[],\n  alpha = 0.05,\n): { adjusted: number[]; significant: boolean[] } {\n  const k = pValues.length\n  const adjusted = pValues.map((p) => Math.min(1, p * k))\n  const significant = adjusted.map((p) => p < alpha)\n  return { adjusted, significant }\n}\n\n/**\n * Benjamini–Hochberg false discovery rate. Returns adjusted q-values and\n * significance at the target FDR. Properly handles ties and preserves\n * monotonicity of q-values.\n */\nexport function benjaminiHochberg(\n  pValues: number[],\n  fdr = 0.05,\n): { qValues: number[]; significant: boolean[] } {\n  const n = pValues.length\n  if (n === 0) return { qValues: [], significant: [] }\n  const indexed = pValues.map((p, i) => ({ p, i })).sort((a, b) => a.p - b.p)\n  const q = new Array<number>(n)\n  // Ranks are 1-based; q_i = p_i * n / rank_i\n  let minRight = 1\n  for (let k = n - 1; k >= 0; k--) {\n    const rank = k + 1\n    const entry = indexed[k]!\n    const raw = (entry.p * n) / rank\n    const bounded = Math.min(minRight, raw)\n    minRight = bounded\n    q[entry.i] = Math.min(1, bounded)\n  }\n  const significant = q.map((v) => v < fdr)\n  return { qValues: q, significant }\n}\n\n/** Standard-normal inverse CDF (Acklam approximation). */\nfunction zQuantile(p: number): number {\n  if (p <= 0 || p >= 1) {\n    if (p === 0) return -Infinity\n    if (p === 1) return Infinity\n    return NaN\n  }\n  const a = [\n    -3.969683028665376e1, 2.209460984245205e2, -2.759285104469687e2, 1.38357751867269e2,\n    -3.066479806614716e1, 2.506628277459239,\n  ]\n  const b = [\n    -5.447609879822406e1, 1.615858368580409e2, -1.556989798598866e2, 6.680131188771972e1,\n    -1.328068155288572e1,\n  ]\n  const c = [\n    -7.784894002430293e-3, -3.223964580411365e-1, -2.400758277161838, -2.549732539343734,\n    4.374664141464968, 2.938163982698783,\n  ]\n  const d = [7.784695709041462e-3, 3.224671290700398e-1, 2.445134137142996, 3.754408661907416]\n  const pLow = 0.02425\n  const pHigh = 1 - pLow\n  let q: number\n  let r: number\n  if (p < pLow) {\n    q = Math.sqrt(-2 * Math.log(p))\n    return (\n      (((((c[0]! * q + c[1]!) * q + c[2]!) * q + c[3]!) * q + c[4]!) * q + c[5]!) /\n      ((((d[0]! * q + d[1]!) * q + d[2]!) * q + d[3]!) * q + 1)\n    )\n  }\n  if (p <= pHigh) {\n    q = p - 0.5\n    r = q * q\n    return (\n      ((((((a[0]! * r + a[1]!) * r + a[2]!) * r + a[3]!) * r + a[4]!) * r + a[5]!) * q) /\n      (((((b[0]! * r + b[1]!) * r + b[2]!) * r + b[3]!) * r + b[4]!) * r + 1)\n    )\n  }\n  q = Math.sqrt(-2 * Math.log(1 - p))\n  return (\n    -(((((c[0]! * q + c[1]!) * q + c[2]!) * q + c[3]!) * q + c[4]!) * q + c[5]!) /\n    ((((d[0]! * q + d[1]!) * q + d[2]!) * q + d[3]!) * q + 1)\n  )\n}\n","/**\n * Paper-grade paired statistics for held-out promotion gates.\n *\n * The promotion gate (`HeldOutGate`) needs three things:\n *\n *   1. A bootstrap confidence interval on the per-item paired delta\n *      (`pairedBootstrap`). Median delta is the headline number; the\n *      CI lower bound is what the gate checks against `pairedDeltaThreshold`.\n *   2. A non-parametric significance test on the paired deltas\n *      (`pairedWilcoxon` — re-export of `wilcoxonSignedRank` under the\n *      paper-style name).\n *   3. False-discovery-rate correction across simultaneously-tested\n *      candidate variants (`bhAdjust` — re-export of `benjaminiHochberg`).\n *\n * Why a separate file: every existing primitive lives in `statistics.ts`\n * (general) or `power-analysis.ts` (correction). Paired-bootstrap is\n * paired-only, paper-grade, and load-bearing for the promotion gate.\n * Putting it next to `statistics.ts` would require editing that file;\n * the brief forbids that. New file, new exports, no surface change.\n */\n\nimport { benjaminiHochberg } from './power-analysis'\nimport { wilcoxonSignedRank } from './statistics'\n\nexport interface PairedBootstrapResult {\n  /** Number of paired observations (after dropping unequal lengths is rejected). */\n  n: number\n  /** Median of paired deltas (after − before). */\n  median: number\n  /** Mean of paired deltas. */\n  mean: number\n  /** Lower bound of the bootstrap CI on the median delta. */\n  low: number\n  /** Upper bound of the bootstrap CI on the median delta. */\n  high: number\n  /** Confidence level used (e.g. 0.95). */\n  confidence: number\n  /** Number of bootstrap resamples used. */\n  resamples: number\n}\n\nexport interface PairedBootstrapOptions {\n  /** Confidence level. Default 0.95. */\n  confidence?: number\n  /** Bootstrap resample count. Default 2000. */\n  resamples?: number\n  /** Statistic to bootstrap. Default 'median'. */\n  statistic?: 'median' | 'mean'\n  /** Deterministic seed. If omitted, uses Math.random(). */\n  seed?: number\n}\n\n/**\n * Paired bootstrap on (after - before) deltas. Returns a CI on the\n * chosen statistic (median by default). Pairs are resampled with\n * replacement. The lower bound is what the promotion gate checks: if\n * `low > pairedDeltaThreshold`, the gain is real at the chosen\n * confidence level.\n *\n * Throws on unequal sample sizes — caller must align pairs upstream.\n */\nexport function pairedBootstrap(\n  before: number[],\n  after: number[],\n  opts: PairedBootstrapOptions = {},\n): PairedBootstrapResult {\n  if (before.length !== after.length) {\n    throw new Error(`pairedBootstrap: unequal sample sizes (${before.length} vs ${after.length})`)\n  }\n  const confidence = opts.confidence ?? 0.95\n  const resamples = opts.resamples ?? 2000\n  const statistic = opts.statistic ?? 'median'\n  if (confidence <= 0 || confidence >= 1) {\n    throw new Error(`pairedBootstrap: confidence must be in (0,1), got ${confidence}`)\n  }\n\n  const n = before.length\n  const deltas = before.map((b, i) => after[i]! - b)\n  if (n === 0) {\n    return { n: 0, median: 0, mean: 0, low: 0, high: 0, confidence, resamples }\n  }\n  if (n === 1) {\n    const d = deltas[0]!\n    return { n: 1, median: d, mean: d, low: d, high: d, confidence, resamples }\n  }\n\n  const rng = makeRng(opts.seed)\n  const samples = new Array<number>(resamples)\n  for (let b = 0; b < resamples; b++) {\n    let acc: number[] | null = null\n    if (statistic === 'mean') {\n      let sum = 0\n      for (let k = 0; k < n; k++) {\n        sum += deltas[Math.floor(rng() * n)]!\n      }\n      samples[b] = sum / n\n    } else {\n      acc = new Array<number>(n)\n      for (let k = 0; k < n; k++) {\n        acc[k] = deltas[Math.floor(rng() * n)]!\n      }\n      samples[b] = medianInPlace(acc)\n    }\n  }\n  samples.sort((a, b) => a - b)\n\n  const alpha = 1 - confidence\n  const lowIdx = Math.floor((alpha / 2) * resamples)\n  const highIdx = Math.min(resamples - 1, Math.ceil((1 - alpha / 2) * resamples) - 1)\n\n  return {\n    n,\n    median: medianInPlace([...deltas]),\n    mean: deltas.reduce((s, x) => s + x, 0) / n,\n    low: samples[lowIdx]!,\n    high: samples[Math.max(highIdx, lowIdx)]!,\n    confidence,\n    resamples,\n  }\n}\n\n/**\n * Paper-style alias for `wilcoxonSignedRank`. The signed-rank test on\n * paired deltas is the standard non-parametric significance test for\n * \"candidate beats baseline on matched items.\" Use alongside the\n * bootstrap CI: bootstrap gives effect size, Wilcoxon gives p.\n */\nexport function pairedWilcoxon(before: number[], after: number[]): { w: number; p: number } {\n  return wilcoxonSignedRank(before, after)\n}\n\n/**\n * Paper-style alias for `benjaminiHochberg`. Use to correct p-values\n * across multiple candidate-vs-baseline comparisons run in the same\n * promotion sweep. Returns BH-adjusted q-values and significance at\n * the requested FDR (default 0.05).\n */\nexport function bhAdjust(\n  pValues: number[],\n  fdr = 0.05,\n): { qValues: number[]; significant: boolean[] } {\n  return benjaminiHochberg(pValues, fdr)\n}\n\n// ── Helpers ──────────────────────────────────────────────────────────\n\nfunction medianInPlace(xs: number[]): number {\n  if (xs.length === 0) return 0\n  xs.sort((a, b) => a - b)\n  const mid = Math.floor(xs.length / 2)\n  return xs.length % 2 === 0 ? (xs[mid - 1]! + xs[mid]!) / 2 : xs[mid]!\n}\n\n/**\n * Tiny seedable PRNG (mulberry32). Deterministic given a seed; falls\n * back to Math.random when seed is omitted. Adequate for bootstrap\n * resampling — not cryptographic.\n */\nfunction makeRng(seed: number | undefined): () => number {\n  if (seed === undefined) return Math.random\n  let s = seed | 0 || 0x9e3779b9\n  return () => {\n    s = (s + 0x6d2b79f5) | 0\n    let t = s\n    t = Math.imul(t ^ (t >>> 15), t | 1)\n    t ^= t + Math.imul(t ^ (t >>> 7), t | 61)\n    return ((t ^ (t >>> 14)) >>> 0) / 4294967296\n  }\n}\n","/**\n * Reporting helpers — production summaries and paper-quality figures — sit alongside `reporter.ts` rather\n * than replacing it.\n *\n * Three artefacts:\n *\n *   - `summaryTable`           Markdown table of per-candidate means,\n *                            95% bootstrap CIs, BH-adjusted Wilcoxon\n *                            p-values, and Cohen's d versus a\n *                            comparator candidate.\n *   - `paretoChart`         Abstract spec for a cost vs quality\n *                            scatter, with gate decisions overlaid.\n *                            Returns numbers + labels — caller\n *                            chooses the plotting library.\n *   - `gainHistogram`\n *                            Per-item paired holdout deltas as a\n *                            histogram spec (bins + counts + median +\n *                            CI). Same \"data, not images\" contract.\n *\n * The figure types are PlotSpecs — JSON-friendly, library-agnostic.\n * They aren't React components and they aren't PNGs; they are\n * what you'd hand to vega-lite, plotly, matplotlib, or your own\n * Canvas renderer to draw the actual figure.\n */\n\nimport type { GateDecision } from './held-out-gate'\nimport { pairedBootstrap } from './paired-stats'\nimport type { FailureClusterReport } from './pipelines/failure-cluster'\nimport { benjaminiHochberg, pairedMde } from './power-analysis'\nimport { canonicalize, hashJson } from './pre-registration'\nimport type { RunRecord } from './run-record'\nimport { cohensD, confidenceInterval, wilcoxonSignedRank } from './statistics'\n\n// ── summaryTable ───────────────────────────────────────────────────────\n\nexport interface SummaryTableOptions {\n  /** Comparator candidate id. Wilcoxon + Cohen's d are computed\n   *  versus this candidate. Required for paired stats columns. */\n  comparator?: string\n  /** Which split to read scores from. Default 'holdout'. */\n  split?: 'search' | 'holdout'\n  /** Confidence level for the bootstrap CI on the mean. Default 0.95. */\n  confidence?: number\n  /** FDR for BH adjustment of the comparison p-values. Default 0.05. */\n  fdr?: number\n}\n\nexport interface SummaryTableRow {\n  candidateId: string\n  n: number\n  mean: number\n  ciLow: number\n  ciHigh: number\n  /** BH-adjusted q-value vs comparator. NaN if no comparator. */\n  qValue: number\n  /** Cohen's d vs comparator. NaN if no comparator. */\n  cohensD: number\n}\n\nexport interface SummaryTable {\n  rows: SummaryTableRow[]\n  comparator: string | null\n  split: 'search' | 'holdout'\n  /** Pre-rendered markdown — drop into a paper or PR. */\n  markdown: string\n}\n\n/**\n * Table 1 helper. Buckets runs by `candidateId`, computes mean +\n * bootstrap CI on the chosen split, and (when a comparator is given)\n * BH-adjusted Wilcoxon p + Cohen's d versus that comparator.\n */\nexport function summaryTable(runs: RunRecord[], opts: SummaryTableOptions = {}): SummaryTable {\n  const split = opts.split ?? 'holdout'\n  const confidence = opts.confidence ?? 0.95\n  const fdr = opts.fdr ?? 0.05\n  const comparator = opts.comparator ?? null\n  const scoreField = split === 'holdout' ? 'holdoutScore' : 'searchScore'\n\n  const byCandidate = new Map<string, { runs: RunRecord[]; scores: number[] }>()\n  for (const r of runs) {\n    if (r.splitTag !== split) continue\n    const v = r.outcome[scoreField]\n    if (typeof v !== 'number' || !Number.isFinite(v)) continue\n    const bucket = byCandidate.get(r.candidateId) ?? { runs: [], scores: [] }\n    bucket.runs.push(r)\n    bucket.scores.push(v)\n    byCandidate.set(r.candidateId, bucket)\n  }\n\n  const candidateIds = [...byCandidate.keys()].sort()\n  const compRuns = comparator ? byCandidate.get(comparator) : undefined\n\n  // First pass: per-candidate means + CIs + raw p-values.\n  const tentative: Array<SummaryTableRow & { rawP: number }> = []\n  for (const id of candidateIds) {\n    const bucket = byCandidate.get(id)!\n    const ci = confidenceInterval(bucket.scores, confidence)\n    let rawP = Number.NaN\n    let d = Number.NaN\n    if (comparator && compRuns && id !== comparator) {\n      const paired = pairScoresByKey(bucket.runs, compRuns.runs, scoreField)\n      if (paired.before.length >= 6) {\n        rawP = wilcoxonSignedRank(paired.before, paired.after).p\n      }\n      d = cohensD(compRuns.scores, bucket.scores)\n    }\n    tentative.push({\n      candidateId: id,\n      n: bucket.scores.length,\n      mean: ci.mean,\n      ciLow: ci.lower,\n      ciHigh: ci.upper,\n      qValue: rawP,\n      cohensD: d,\n      rawP,\n    })\n  }\n\n  // BH-adjust across the comparison set (skip NaN rows / the\n  // comparator itself). Adjustment is a no-op when there are 0 or 1\n  // comparators.\n  if (comparator) {\n    const idxs: number[] = []\n    const ps: number[] = []\n    for (let i = 0; i < tentative.length; i++) {\n      const r = tentative[i]!\n      if (r.candidateId === comparator) continue\n      if (!Number.isFinite(r.rawP)) continue\n      idxs.push(i)\n      ps.push(r.rawP)\n    }\n    if (ps.length > 0) {\n      const { qValues } = benjaminiHochberg(ps, fdr)\n      for (let k = 0; k < idxs.length; k++) {\n        tentative[idxs[k]!]!.qValue = qValues[k]!\n      }\n    }\n  }\n\n  const rows = tentative.map(({ rawP: _rawP, ...rest }) => rest)\n  const markdown = renderSummaryTableMarkdown(rows, comparator, split)\n  return { rows, comparator, split, markdown }\n}\n\nfunction pairScoresByKey(\n  candidate: RunRecord[],\n  baseline: RunRecord[],\n  scoreField: 'searchScore' | 'holdoutScore',\n): { before: number[]; after: number[] } {\n  const baseIdx = new Map<string, number>()\n  for (const r of baseline) {\n    const v = r.outcome[scoreField]\n    if (typeof v === 'number' && Number.isFinite(v)) {\n      baseIdx.set(`${r.experimentId}::${r.seed}`, v)\n    }\n  }\n  const before: number[] = []\n  const after: number[] = []\n  for (const r of candidate) {\n    const v = r.outcome[scoreField]\n    if (typeof v !== 'number' || !Number.isFinite(v)) continue\n    const key = `${r.experimentId}::${r.seed}`\n    const b = baseIdx.get(key)\n    if (b === undefined) continue\n    before.push(b)\n    after.push(v)\n  }\n  return { before, after }\n}\n\nfunction renderSummaryTableMarkdown(\n  rows: SummaryTableRow[],\n  comparator: string | null,\n  split: 'search' | 'holdout',\n): string {\n  const lines: string[] = []\n  const cmpLabel = comparator ? ` (vs ${comparator})` : ''\n  lines.push(`Summary Table — ${split} split${cmpLabel}`)\n  lines.push('')\n  lines.push(\"| Candidate | N | Mean | 95% CI | q (BH) | Cohen's d |\")\n  lines.push('|---|---:|---:|---|---:|---:|')\n  for (const r of rows) {\n    const ci = `[${fmt(r.ciLow)}, ${fmt(r.ciHigh)}]`\n    const q = Number.isFinite(r.qValue) ? r.qValue.toFixed(4) : '—'\n    const d = Number.isFinite(r.cohensD) ? r.cohensD.toFixed(3) : '—'\n    lines.push(`| ${r.candidateId} | ${r.n} | ${fmt(r.mean)} | ${ci} | ${q} | ${d} |`)\n  }\n  return lines.join('\\n')\n}\n\n// ── paretoChart ─────────────────────────────────────────────────────\n\nexport interface ParetoPoint {\n  candidateId: string\n  /** Mean USD cost per run on the chosen split. */\n  cost: number\n  /** Mean score on the chosen split. */\n  quality: number\n  /** Number of runs that informed this point. */\n  n: number\n  /** Whether this candidate is on the Pareto frontier — high\n   *  quality, low cost, no dominator. */\n  onFrontier: boolean\n  /** Optional gate verdict for this candidate, if a `GateDecision`\n   *  for it was passed in. */\n  gate?: 'promote' | 'reject_few_runs' | 'reject_negative_delta' | 'reject_overfit_gap' | null\n}\n\nexport interface ParetoFigureSpec {\n  kind: 'pareto-cost-quality'\n  split: 'search' | 'holdout'\n  points: ParetoPoint[]\n  axes: { x: 'costUsd'; y: 'score' }\n}\n\n/**\n * Cost vs quality scatter spec. `gateDecisions` is keyed by\n * candidate id; if present, every point picks up the gate verdict\n * for overlay.\n */\nexport function paretoChart(\n  runs: RunRecord[],\n  opts: {\n    split?: 'search' | 'holdout'\n    gateDecisions?: Record<string, GateDecision>\n  } = {},\n): ParetoFigureSpec {\n  const split = opts.split ?? 'holdout'\n  const scoreField = split === 'holdout' ? 'holdoutScore' : 'searchScore'\n\n  const buckets = new Map<string, { cost: number[]; quality: number[] }>()\n  for (const r of runs) {\n    if (r.splitTag !== split) continue\n    const v = r.outcome[scoreField]\n    if (typeof v !== 'number' || !Number.isFinite(v)) continue\n    const bucket = buckets.get(r.candidateId) ?? { cost: [], quality: [] }\n    bucket.cost.push(r.costUsd)\n    bucket.quality.push(v)\n    buckets.set(r.candidateId, bucket)\n  }\n\n  const points: ParetoPoint[] = []\n  for (const [candidateId, bucket] of buckets.entries()) {\n    points.push({\n      candidateId,\n      cost: avg(bucket.cost),\n      quality: avg(bucket.quality),\n      n: bucket.cost.length,\n      onFrontier: false,\n      gate: opts.gateDecisions?.[candidateId]\n        ? gateLabel(opts.gateDecisions[candidateId]!)\n        : undefined,\n    })\n  }\n\n  // Pareto: minimize cost, maximize quality. A point is dominated if\n  // some other point has lower-or-equal cost AND higher-or-equal\n  // quality with strict inequality somewhere.\n  for (const p of points) {\n    p.onFrontier = !points.some((q) => q !== p && dominates(q, p))\n  }\n\n  return {\n    kind: 'pareto-cost-quality',\n    split,\n    axes: { x: 'costUsd', y: 'score' },\n    points,\n  }\n}\n\nfunction dominates(a: ParetoPoint, b: ParetoPoint): boolean {\n  return a.cost <= b.cost && a.quality >= b.quality && (a.cost < b.cost || a.quality > b.quality)\n}\n\nfunction gateLabel(d: GateDecision): ParetoPoint['gate'] {\n  if (d.promote) return 'promote'\n  if (d.rejectionCode === 'few_runs') return 'reject_few_runs'\n  if (d.rejectionCode === 'negative_delta') return 'reject_negative_delta'\n  if (d.rejectionCode === 'overfit_gap') return 'reject_overfit_gap'\n  return null\n}\n\n// ── gainHistogram ───────────────────────────────────────────\n\nexport interface GainDistributionBin {\n  /** Inclusive lower edge. */\n  lo: number\n  /** Exclusive upper edge (or inclusive if it's the last bin). */\n  hi: number\n  /** Number of pairs whose delta lands in this bin. */\n  count: number\n}\n\nexport interface GainDistributionFigureSpec {\n  kind: 'gain-distribution'\n  candidateId: string\n  comparator: string\n  split: 'search' | 'holdout'\n  /** Number of pairs used. */\n  n: number\n  bins: GainDistributionBin[]\n  median: number\n  ci: { low: number; high: number }\n}\n\nexport interface GainDistributionOptions {\n  /** Number of histogram bins. Default 11 (so the centre is exact at 0). */\n  bins?: number\n  /** Which split to use. Default 'holdout'. */\n  split?: 'search' | 'holdout'\n  /** Confidence level for the CI. Default 0.95. */\n  confidence?: number\n  /** Bootstrap resamples. Default 2000. */\n  resamples?: number\n  /** Deterministic seed. */\n  seed?: number\n}\n\n/**\n * Held-out improvement distribution: per-pair delta (candidate −\n * comparator), histogrammed. Includes the bootstrap CI on the median\n * delta — same primitive the promotion gate uses.\n */\nexport function gainHistogram(\n  runs: RunRecord[],\n  candidateId: string,\n  comparator: string,\n  opts: GainDistributionOptions = {},\n): GainDistributionFigureSpec {\n  const split = opts.split ?? 'holdout'\n  const scoreField = split === 'holdout' ? 'holdoutScore' : 'searchScore'\n  const binCount = opts.bins ?? 11\n  if (binCount < 1) throw new Error('gainHistogram: bins must be ≥ 1')\n\n  const candidate = runs.filter((r) => r.candidateId === candidateId && r.splitTag === split)\n  const baseline = runs.filter((r) => r.candidateId === comparator && r.splitTag === split)\n  // pairScoresByKey returns before=baseline-score, after=candidate-score\n  // for each (experimentId, seed) pair where both sides recorded a\n  // valid score on this split. delta = after - before = candidate - baseline.\n  const { before, after } = pairScoresByKey(candidate, baseline, scoreField)\n  const n = before.length\n\n  if (n === 0) {\n    return {\n      kind: 'gain-distribution',\n      candidateId,\n      comparator,\n      split,\n      n: 0,\n      bins: [],\n      median: 0,\n      ci: { low: 0, high: 0 },\n    }\n  }\n\n  const deltas = before.map((b, i) => after[i]! - b)\n  const sortedDeltas = [...deltas].sort((a, b) => a - b)\n  const median = medianOfSorted(sortedDeltas)\n  const min = sortedDeltas[0]!\n  const max = sortedDeltas[sortedDeltas.length - 1]!\n\n  // Symmetric bins around the wider of (|min|, |max|) so the chart\n  // visually centres on zero without dropping outliers.\n  const bound = Math.max(Math.abs(min), Math.abs(max), 1e-6)\n  const lo = -bound\n  const hi = bound\n  const width = (hi - lo) / binCount\n  const bins: GainDistributionBin[] = []\n  for (let i = 0; i < binCount; i++) {\n    bins.push({ lo: lo + i * width, hi: lo + (i + 1) * width, count: 0 })\n  }\n  for (const d of deltas) {\n    let idx = Math.floor((d - lo) / width)\n    if (idx < 0) idx = 0\n    if (idx >= binCount) idx = binCount - 1\n    bins[idx]!.count += 1\n  }\n\n  const ci = pairedBootstrap(before, after, {\n    confidence: opts.confidence ?? 0.95,\n    resamples: opts.resamples ?? 2000,\n    statistic: 'median',\n    seed: opts.seed,\n  })\n\n  return {\n    kind: 'gain-distribution',\n    candidateId,\n    comparator,\n    split,\n    n,\n    bins,\n    median,\n    ci: { low: ci.low, high: ci.high },\n  }\n}\n\n// ── researchReport ───────────────────────────────────────────────────\n\nexport type ResearchReportDecision =\n  | 'promote'\n  | 'hold'\n  | 'reject'\n  | 'equivalent'\n  | 'needs_more_data'\n\n/**\n * Hard floor below which a paired comparison is treated as uninformative\n * regardless of `minPairs`. Mirrors the lower limit on Wilcoxon signed-rank\n * exact tables; below this the test has no power to separate effect sizes.\n */\nexport const RESEARCH_REPORT_HARD_PAIR_FLOOR = 6\n\nexport interface ResearchReportOptions {\n  /** Human-readable report title. */\n  title?: string\n  /** Comparator candidate id. Required for statistical decision guidance. */\n  comparator?: string\n  /** Which split to use for the primary decision. Default 'holdout'. */\n  split?: 'search' | 'holdout'\n  /** Confidence level used by lower-level report helpers. Default 0.95. */\n  confidence?: number\n  /** FDR threshold for q-values. Default 0.05. */\n  fdr?: number\n  /**\n   * Soft floor on paired observations before issuing a directional\n   * promote / reject. Below this we report `needs_more_data` and surface the\n   * minimum detectable effect at the current N. Default 20 — chosen so the\n   * Wilcoxon signed-rank approximation is reasonable and so the paired\n   * bootstrap CI has non-degenerate coverage. Hard floor is enforced at\n   * `RESEARCH_REPORT_HARD_PAIR_FLOOR` (6) regardless of this value.\n   */\n  minPairs?: number\n  /**\n   * Region of Practical Equivalence on the paired delta. When a candidate's\n   * paired-delta CI is fully contained in `[low, high]`, the decision is\n   * `equivalent` rather than `hold`. Sourced from the domain owner — there is\n   * no statistically-defensible default.\n   */\n  rope?: { low: number; high: number }\n  /**\n   * Power for the minimum detectable effect (MDE) reported on each candidate.\n   * Default 0.8.\n   */\n  mdePower?: number\n  /**\n   * Two-sided alpha for the MDE. Default matches `fdr` so the reported MDE\n   * lines up with the test the report actually runs.\n   */\n  mdeAlpha?: number\n  /** Optional held-out gate decisions keyed by candidate id. */\n  gateDecisions?: Record<string, GateDecision>\n  /** Optional failure clusters from failureClusterView. */\n  failureClusters?: FailureClusterReport\n  /** Build gain histograms for these candidates. Defaults to all non-comparator candidates. */\n  candidateIds?: string[]\n  /** Deterministic bootstrap seed passed to gainHistogram and the posterior helper. */\n  seed?: number\n  /** Report timestamp. Defaults to current time. */\n  generatedAt?: string\n  /**\n   * Hash of a preregistered protocol (e.g. `signManifest({...}).contentHash`).\n   * Embedded verbatim in the report so the analysis can be cited as the\n   * preregistered one rather than a post-hoc fishing expedition.\n   */\n  preregistrationHash?: string\n}\n\nexport interface ResearchReportRecommendation {\n  decision: ResearchReportDecision\n  candidateId: string | null\n  rationale: string[]\n  risks: string[]\n  nextActions: string[]\n}\n\nexport interface ResearchReportCandidate {\n  candidateId: string\n  n: number\n  mean: number\n  ciLow: number\n  ciHigh: number\n  qValue: number\n  cohensD: number\n  meanDeltaVsComparator: number | null\n  pairedN: number\n  medianGain: number | null\n  meanGain: number | null\n  gainCi: { low: number; high: number } | null\n  /**\n   * Bayesian-bootstrap-style posterior summaries on the paired delta. Computed\n   * from the same resamples that produce the gain CI; interpretable as\n   * \"fraction of resamples in which the candidate beats the comparator on\n   * matched pairs.\"\n   */\n  prGreaterThanZero: number | null\n  prInRope: number | null\n  /**\n   * Minimum detectable effect (in score units) at the candidate's paired N,\n   * the configured power, and the configured alpha. Standardised by the\n   * observed paired-delta SD and inverted via `requiredSampleSize`. Reported\n   * for every candidate so a `needs_more_data` verdict is actionable.\n   */\n  mde: number | null\n  onParetoFrontier: boolean\n  gate?: ParetoPoint['gate']\n  decision: ResearchReportDecision\n  decisionReason: string\n}\n\nexport interface ResearchReportMethodology {\n  /**\n   * Plain-language assumptions the report depends on. Read these first when\n   * deciding whether the verdict is load-bearing for a launch decision.\n   */\n  assumptions: string[]\n  /** Tests and estimators the verdict was computed from. */\n  methods: string[]\n  /** Alternatives the author considered and why this report didn't take them. */\n  alternatives: string[]\n  /** Failure modes — when this report should NOT drive a decision. */\n  whenNotToApply: string[]\n  /** Citations for the methodological choices above. */\n  citations: string[]\n}\n\nexport interface ResearchReport {\n  kind: 'agent-eval-research-report'\n  title: string\n  generatedAt: string\n  split: 'search' | 'holdout'\n  comparator: string | null\n  /**\n   * SHA-256 over the canonicalised set of `(runId, candidateId, split)` triples\n   * the report was computed from, plus the comparator and split. Stable across\n   * key insertion order; recomputable by the reader to verify provenance.\n   */\n  runFingerprint: string\n  preregistrationHash: string | null\n  rope: { low: number; high: number } | null\n  executiveSummary: string[]\n  recommendation: ResearchReportRecommendation\n  candidates: ResearchReportCandidate[]\n  summary: SummaryTable\n  charts: {\n    pareto: ParetoFigureSpec\n    gains: GainDistributionFigureSpec[]\n  }\n  methodology: ResearchReportMethodology\n  failureClusters?: FailureClusterReport\n  markdown: string\n  html: string\n}\n\n/**\n * Internal: paired posterior summary on (candidate − comparator) deltas.\n *\n * Returns the bootstrap CI on the median (matching `gainHistogram`) plus\n * Bayesian-flavoured posterior summaries Pr(Δ>0) and Pr(Δ∈ROPE) computed\n * from a Bayesian-bootstrap-flavoured resample distribution on the mean\n * (Rubin 1981 — non-informative bootstrap-prior duality), and the\n * minimum detectable paired effect at the configured power and α.\n *\n * `null` is returned when no paired observations exist; callers must\n * gate on `n` before consuming the bootstrap statistics.\n */\nfunction pairedPosterior(\n  runs: RunRecord[],\n  candidateId: string,\n  comparator: string,\n  opts: {\n    split: 'search' | 'holdout'\n    confidence: number\n    seed?: number\n    rope: { low: number; high: number } | null\n    mdePower: number\n    mdeAlpha: number\n  },\n): {\n  n: number\n  meanDelta: number\n  medianDelta: number\n  sdDelta: number\n  ci: { low: number; high: number }\n  prGreaterThanZero: number\n  prInRope: number | null\n  mde: number\n} | null {\n  const scoreField = opts.split === 'holdout' ? 'holdoutScore' : 'searchScore'\n  const candidate = runs.filter((r) => r.candidateId === candidateId && r.splitTag === opts.split)\n  const baseline = runs.filter((r) => r.candidateId === comparator && r.splitTag === opts.split)\n  const { before, after } = pairScoresByKey(candidate, baseline, scoreField)\n  const n = before.length\n  if (n === 0) return null\n\n  const deltas = before.map((b, i) => after[i]! - b)\n  const meanDelta = deltas.reduce((s, x) => s + x, 0) / n\n  const sortedDeltas = [...deltas].sort((a, b) => a - b)\n  const medianDelta = medianOfSorted(sortedDeltas)\n  const sdDelta = stdev(deltas, meanDelta)\n\n  const ci = pairedBootstrap(before, after, {\n    confidence: opts.confidence,\n    resamples: 2000,\n    statistic: 'median',\n    seed: opts.seed,\n  })\n\n  // Enumerate bootstrap-mean samples to derive posterior summaries on the\n  // mean delta. Same RNG family as `pairedBootstrap` but kept local so we can\n  // examine the full sample distribution rather than just quantiles.\n  const meanSamples = bootstrapMeanSamples(deltas, 2000, opts.seed)\n  const prGreaterThanZero =\n    meanSamples.length === 0 ? 0 : meanSamples.filter((s) => s > 0).length / meanSamples.length\n  const prInRope =\n    opts.rope === null || meanSamples.length === 0\n      ? null\n      : meanSamples.filter((s) => s >= opts.rope!.low && s <= opts.rope!.high).length /\n        meanSamples.length\n\n  const dStandardised = pairedMde({ nPaired: n, alpha: opts.mdeAlpha, power: opts.mdePower })\n  const mde = sdDelta === 0 ? 0 : dStandardised * sdDelta\n\n  return {\n    n,\n    meanDelta,\n    medianDelta,\n    sdDelta,\n    ci: { low: ci.low, high: ci.high },\n    prGreaterThanZero,\n    prInRope,\n    mde,\n  }\n}\n\nfunction bootstrapMeanSamples(deltas: number[], resamples: number, seed?: number): number[] {\n  const n = deltas.length\n  if (n === 0) return []\n  if (n === 1) return new Array<number>(resamples).fill(deltas[0]!)\n  const rng = seedRng(seed)\n  const samples = new Array<number>(resamples)\n  for (let b = 0; b < resamples; b++) {\n    let sum = 0\n    for (let k = 0; k < n; k++) sum += deltas[Math.floor(rng() * n)]!\n    samples[b] = sum / n\n  }\n  return samples\n}\n\nfunction seedRng(seed?: number): () => number {\n  if (seed === undefined) return Math.random\n  let s = seed >>> 0\n  return () => {\n    s = (s + 0x6d2b79f5) >>> 0\n    let t = s\n    t = Math.imul(t ^ (t >>> 15), t | 1)\n    t ^= t + Math.imul(t ^ (t >>> 7), t | 61)\n    return ((t ^ (t >>> 14)) >>> 0) / 4294967296\n  }\n}\n\nfunction stdev(xs: number[], mean: number): number {\n  if (xs.length < 2) return 0\n  let sse = 0\n  for (const x of xs) sse += (x - mean) ** 2\n  return Math.sqrt(sse / (xs.length - 1))\n}\n\n/**\n * Executive research report for CPO / AI-lead / launch-review consumption.\n *\n * Composes:\n *   - `summaryTable`         marginal stats with BH-FDR-adjusted q-values\n *   - `paretoChart`           cost-vs-quality frontier with gate overlay\n *   - `gainHistogram`         per-candidate paired-delta distribution\n *   - paired posterior (this file): bootstrap CI on median, Pr(Δ>0),\n *                              Pr(Δ∈ROPE), MDE at the configured power\n *\n * Decisions are made on paired evidence — never on marginal means alone —\n * and respect any held-out gate decision the caller passes through. The\n * report embeds a SHA-256 fingerprint of the input run set and, optionally,\n * the hash of a preregistered protocol so a downstream reader can verify\n * provenance and that the analysis was the preregistered one.\n *\n * Async because the fingerprint uses Web Crypto via `hashJson`; deterministic\n * for any fixed `runs`, `seed`, and ROPE.\n */\nexport async function researchReport(\n  runs: RunRecord[],\n  opts: ResearchReportOptions = {},\n): Promise<ResearchReport> {\n  const split = opts.split ?? 'holdout'\n  const comparator = opts.comparator ?? null\n  const confidence = opts.confidence ?? 0.95\n  const fdr = opts.fdr ?? 0.05\n  const minPairs = Math.max(opts.minPairs ?? 20, RESEARCH_REPORT_HARD_PAIR_FLOOR)\n  const rope = opts.rope ?? null\n  const mdePower = opts.mdePower ?? 0.8\n  const mdeAlpha = opts.mdeAlpha ?? fdr\n  const title = opts.title ?? 'Agent Evaluation Research Report'\n  const generatedAt = opts.generatedAt ?? new Date().toISOString()\n  const preregistrationHash = opts.preregistrationHash ?? null\n\n  if (rope && !(Number.isFinite(rope.low) && Number.isFinite(rope.high) && rope.low <= rope.high)) {\n    throw new Error(\n      `researchReport: rope must satisfy low ≤ high with finite bounds, got ${JSON.stringify(rope)}`,\n    )\n  }\n\n  const summary = summaryTable(runs, {\n    comparator: comparator ?? undefined,\n    split,\n    confidence,\n    fdr,\n  })\n  const pareto = paretoChart(runs, { split, gateDecisions: opts.gateDecisions })\n  const candidateIds =\n    opts.candidateIds ?? summary.rows.map((r) => r.candidateId).filter((id) => id !== comparator)\n  const gains = comparator\n    ? candidateIds.map((id) =>\n        gainHistogram(runs, id, comparator, {\n          split,\n          confidence,\n          seed: opts.seed,\n        }),\n      )\n    : []\n\n  const gainByCandidate = new Map(gains.map((g) => [g.candidateId, g]))\n  const paretoByCandidate = new Map(pareto.points.map((p) => [p.candidateId, p]))\n  const posteriorByCandidate = new Map<string, ReturnType<typeof pairedPosterior>>()\n  if (comparator) {\n    for (const id of candidateIds) {\n      posteriorByCandidate.set(\n        id,\n        pairedPosterior(runs, id, comparator, {\n          split,\n          confidence,\n          seed: opts.seed,\n          rope,\n          mdePower,\n          mdeAlpha,\n        }),\n      )\n    }\n  }\n\n  const candidates = summary.rows\n    .map((row) => {\n      const gain = gainByCandidate.get(row.candidateId)\n      const point = paretoByCandidate.get(row.candidateId)\n      const posterior = posteriorByCandidate.get(row.candidateId) ?? null\n      const classified = classifyCandidate(row, {\n        comparator,\n        posterior,\n        point,\n        fdr,\n        minPairs,\n        rope,\n      })\n      return {\n        candidateId: row.candidateId,\n        n: row.n,\n        mean: row.mean,\n        ciLow: row.ciLow,\n        ciHigh: row.ciHigh,\n        qValue: row.qValue,\n        cohensD: row.cohensD,\n        meanDeltaVsComparator: posterior ? posterior.meanDelta : null,\n        pairedN: posterior?.n ?? gain?.n ?? 0,\n        medianGain: posterior ? posterior.medianDelta : gain ? gain.median : null,\n        meanGain: posterior ? posterior.meanDelta : null,\n        gainCi: posterior ? posterior.ci : gain ? gain.ci : null,\n        prGreaterThanZero: posterior ? posterior.prGreaterThanZero : null,\n        prInRope: posterior ? posterior.prInRope : null,\n        mde: posterior ? posterior.mde : null,\n        onParetoFrontier: point?.onFrontier ?? false,\n        gate: point?.gate,\n        decision: classified.decision,\n        decisionReason: classified.reason,\n      } satisfies ResearchReportCandidate\n    })\n    .sort((a, b) => {\n      const decisionRank = decisionWeight(b.decision) - decisionWeight(a.decision)\n      if (decisionRank !== 0) return decisionRank\n      return b.mean - a.mean\n    })\n\n  const recommendation = buildRecommendation(candidates, {\n    comparator,\n    failureClusters: opts.failureClusters,\n    rope,\n    minPairs,\n    preregistrationHash,\n  })\n  const executiveSummary = buildExecutiveSummary(candidates, recommendation, {\n    comparator,\n    split,\n    failureClusters: opts.failureClusters,\n    preregistrationHash,\n  })\n  const methodology = buildMethodology({\n    split,\n    comparator,\n    fdr,\n    minPairs,\n    rope,\n    confidence,\n    mdePower,\n    mdeAlpha,\n  })\n\n  const runFingerprint = await hashJson(\n    canonicalize({\n      triples: runs\n        .filter((r) => r.splitTag === split)\n        .map((r) => ({ runId: r.runId, candidateId: r.candidateId, splitTag: r.splitTag }))\n        .sort((a, b) => a.runId.localeCompare(b.runId)),\n      comparator,\n      split,\n    }),\n  )\n\n  const markdown = renderResearchMarkdown({\n    title,\n    generatedAt,\n    split,\n    comparator,\n    rope,\n    runFingerprint,\n    preregistrationHash,\n    executiveSummary,\n    recommendation,\n    candidates,\n    summary,\n    pareto,\n    gains,\n    methodology,\n    failureClusters: opts.failureClusters,\n  })\n  const html = renderResearchHtml(markdown, title)\n\n  return {\n    kind: 'agent-eval-research-report',\n    title,\n    generatedAt,\n    split,\n    comparator,\n    runFingerprint,\n    preregistrationHash,\n    rope,\n    executiveSummary,\n    recommendation,\n    candidates,\n    summary,\n    charts: { pareto, gains },\n    methodology,\n    failureClusters: opts.failureClusters,\n    markdown,\n    html,\n  }\n}\n\nfunction buildMethodology(ctx: {\n  split: 'search' | 'holdout'\n  comparator: string | null\n  fdr: number\n  minPairs: number\n  rope: { low: number; high: number } | null\n  confidence: number\n  mdePower: number\n  mdeAlpha: number\n}): ResearchReportMethodology {\n  const assumptions: string[] = [\n    'Pairs are matched by (experimentId, seed); the candidate and comparator see the same scenarios in the same order.',\n    'Paired deltas are exchangeable conditional on the matched scenario — no mid-run distribution shift.',\n    `Decisions are pre-specified at fdr=${ctx.fdr}, minPairs=${ctx.minPairs}, confidence=${ctx.confidence}; deviating from these post-hoc invalidates the false-discovery control.`,\n  ]\n  if (ctx.rope) {\n    assumptions.push(\n      `The Region of Practical Equivalence ${formatRope(ctx.rope)} is supplied by the domain owner; equivalent verdicts are only meaningful if that range is treated as the standing definition of \"no material difference.\"`,\n    )\n  }\n  if (ctx.comparator === null) {\n    assumptions.push('No comparator was configured; this run is descriptive, not causal.')\n  }\n  const methods: string[] = [\n    \"Marginal scores summarised with BH-FDR-adjusted Wilcoxon signed-rank q-values and Cohen's d via summaryTable.\",\n    'Paired evidence summarised with bootstrap CI on the median delta and Bayesian-bootstrap-style Pr(Δ>0) and Pr(Δ∈ROPE) on the mean delta.',\n    `Minimum detectable effect reported per candidate at α=${ctx.mdeAlpha} (two-sided), power=${ctx.mdePower}, standardised by the observed paired-delta SD.`,\n    'Pareto frontier flagged as a separate axis (cost vs quality); a candidate can be on-frontier without winning the paired test.',\n    'Held-out gate decisions, when supplied, override the statistical verdict in the reject direction.',\n  ]\n  const alternatives: string[] = [\n    'Paired t-test rejected: not robust to the heavy-tailed score distributions common in agent benchmarks.',\n    'Unpaired Mann–Whitney rejected: matched scenarios make pairing free; unpaired throws away that variance reduction.',\n    'Sequential / always-valid inference (e-values, mSPRT) is the right tool for iterative sweeps and is out of scope for this single-look report — preregister and run once, or wrap this report in an alpha-spending schedule.',\n    'Hierarchical Bayesian shrinkage across many candidates is future work; the current ranking uses raw paired statistics.',\n  ]\n  const whenNotToApply: string[] = [\n    `Paired N below ${RESEARCH_REPORT_HARD_PAIR_FLOOR} on any candidate — the bootstrap CI is degenerate.`,\n    'Comparator chosen post-hoc by inspecting the same data; q-values are no longer false-discovery-controlled.',\n    'Scenarios not drawn under a stable preregistered protocol; the report can describe the data but cannot anchor a launch decision.',\n    'Score distributions with mid-run shift (judge model swap, rubric change, infra outage) — pair exchangeability is violated.',\n  ]\n  const citations: string[] = [\n    'Benjamini, Y. & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. JRSS B, 57(1), 289–300.',\n    'Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1(6), 80–83.',\n    'Efron, B. (1979). Bootstrap methods: another look at the jackknife. Annals of Statistics, 7(1), 1–26.',\n    'Rubin, D. B. (1981). The Bayesian bootstrap. Annals of Statistics, 9(1), 130–134.',\n    'Kruschke, J. K. (2018). Rejecting or accepting parameter values in Bayesian estimation. Advances in Methods and Practices in Psychological Science, 1(2), 270–280. (ROPE.)',\n  ]\n  return { assumptions, methods, alternatives, whenNotToApply, citations }\n}\n\nfunction formatRope(rope: { low: number; high: number }): string {\n  return `[${fmt(rope.low)}, ${fmt(rope.high)}]`\n}\n\nfunction classifyCandidate(\n  row: SummaryTableRow,\n  ctx: {\n    comparator: string | null\n    posterior: ReturnType<typeof pairedPosterior> | null\n    point?: ParetoPoint\n    fdr: number\n    minPairs: number\n    rope: { low: number; high: number } | null\n  },\n): { decision: ResearchReportDecision; reason: string } {\n  if (ctx.comparator && row.candidateId === ctx.comparator) {\n    return { decision: 'hold', reason: 'Comparator baseline.' }\n  }\n  if (!ctx.comparator) {\n    return {\n      decision: ctx.point?.onFrontier ? 'hold' : 'needs_more_data',\n      reason:\n        'No comparator configured; report ranks candidates but cannot anchor a promotion call.',\n    }\n  }\n  // Held-out gate is authoritative against — promote requires statistical\n  // evidence even if the gate said `promote` (gate is necessary, not sufficient).\n  if (ctx.point?.gate && ctx.point.gate !== 'promote') {\n    return { decision: 'reject', reason: `Held-out gate returned ${ctx.point.gate}.` }\n  }\n  if (!ctx.posterior || ctx.posterior.n < RESEARCH_REPORT_HARD_PAIR_FLOOR) {\n    return {\n      decision: 'needs_more_data',\n      reason: `Only ${ctx.posterior?.n ?? 0} paired observations; below hard floor of ${RESEARCH_REPORT_HARD_PAIR_FLOOR} for any paired inference.`,\n    }\n  }\n  const ci = ctx.posterior.ci\n  if (ctx.rope && ci.low >= ctx.rope.low && ci.high <= ctx.rope.high) {\n    return {\n      decision: 'equivalent',\n      reason: `Paired-delta CI [${fmt(ci.low)}, ${fmt(ci.high)}] is fully inside ROPE ${formatRope(ctx.rope)}; candidate is practically equivalent to comparator.`,\n    }\n  }\n  const significant = Number.isFinite(row.qValue) && row.qValue <= ctx.fdr\n  const gainPositive = ci.low > 0\n  const gainNegative = ci.high < 0\n  if (gainNegative) {\n    return {\n      decision: 'reject',\n      reason: `Paired-delta CI [${fmt(ci.low)}, ${fmt(ci.high)}] lies entirely below zero.`,\n    }\n  }\n  if (ctx.posterior.n < ctx.minPairs) {\n    return {\n      decision: 'needs_more_data',\n      reason: `Only ${ctx.posterior.n} paired observations; minimum detectable effect at this N is ${fmt(ctx.posterior.mde)} score units (need ≥ ${ctx.minPairs} pairs to issue a directional verdict).`,\n    }\n  }\n  if (significant && gainPositive) {\n    return {\n      decision: 'promote',\n      reason: `BH-adjusted q=${fmt(row.qValue)} ≤ ${ctx.fdr} and paired-delta CI [${fmt(ci.low)}, ${fmt(ci.high)}] excludes zero; Pr(Δ>0)=${fmt(ctx.posterior.prGreaterThanZero)}.`,\n    }\n  }\n  return {\n    decision: 'hold',\n    reason: `Pr(Δ>0)=${fmt(ctx.posterior.prGreaterThanZero)} but CI [${fmt(ci.low)}, ${fmt(ci.high)}] crosses zero; effect not decisive at fdr=${ctx.fdr}.`,\n  }\n}\n\nfunction buildRecommendation(\n  candidates: ResearchReportCandidate[],\n  ctx: {\n    comparator: string | null\n    failureClusters?: FailureClusterReport\n    rope: { low: number; high: number } | null\n    minPairs: number\n    preregistrationHash: string | null\n  },\n): ResearchReportRecommendation {\n  const nonComparator = candidates.filter((c) => c.candidateId !== ctx.comparator)\n  const bestPromote = nonComparator.find((c) => c.decision === 'promote')\n  const bestEquivalent = nonComparator.find((c) => c.decision === 'equivalent')\n  const chosen = bestPromote ?? bestEquivalent ?? nonComparator[0] ?? null\n  const decision: ResearchReportDecision = bestPromote\n    ? 'promote'\n    : nonComparator.some((c) => c.decision === 'needs_more_data')\n      ? 'needs_more_data'\n      : bestEquivalent\n        ? 'equivalent'\n        : nonComparator.some((c) => c.decision === 'hold')\n          ? 'hold'\n          : 'reject'\n\n  const rationale: string[] = []\n  const risks: string[] = []\n  const nextActions: string[] = []\n\n  if (chosen) {\n    rationale.push(`${chosen.candidateId}: ${chosen.decisionReason}`)\n    if (chosen.gainCi) {\n      const probSummary =\n        chosen.prGreaterThanZero !== null ? `, Pr(Δ>0)=${fmt(chosen.prGreaterThanZero)}` : ''\n      rationale.push(\n        `Median paired gain CI: [${fmt(chosen.gainCi.low)}, ${fmt(chosen.gainCi.high)}]${probSummary}.`,\n      )\n    }\n    if (chosen.mde !== null && Number.isFinite(chosen.mde)) {\n      rationale.push(`MDE at current paired N=${chosen.pairedN}: ${fmt(chosen.mde)} score units.`)\n    }\n  }\n  if (!ctx.comparator) {\n    risks.push('No comparator was configured; verdict is descriptive, not causal.')\n    nextActions.push('Re-run with a stable comparator candidate for paired inference.')\n  }\n  if (!ctx.preregistrationHash) {\n    risks.push(\n      'No preregistration hash supplied; readers cannot verify the analysis was specified before data inspection.',\n    )\n    nextActions.push(\n      'Sign a HypothesisManifest before the next sweep and pass `preregistrationHash` so the report cites it.',\n    )\n  }\n  if (ctx.rope === null && nonComparator.length > 0) {\n    risks.push(\n      'No ROPE configured; the report cannot distinguish \"equivalent\" from \"inconclusive\".',\n    )\n    nextActions.push(\n      'Define a domain-specific Region of Practical Equivalence and pass it to lock in the equivalence threshold.',\n    )\n  }\n  const inconclusive = nonComparator.filter((c) => c.decision === 'needs_more_data')\n  if (inconclusive.length > 0) {\n    const worst = inconclusive.reduce((a, b) => (b.pairedN < a.pairedN ? b : a))\n    risks.push(\n      `${inconclusive.length} candidate(s) below soft floor (${ctx.minPairs} pairs); thinnest is ${worst.candidateId} with ${worst.pairedN}.`,\n    )\n    nextActions.push(\n      `Collect at least ${ctx.minPairs - worst.pairedN} more matched holdout runs for ${worst.candidateId}.`,\n    )\n  }\n  const rejected = nonComparator.filter((c) => c.decision === 'reject')\n  if (rejected.length > 0) {\n    risks.push(\n      `${rejected.length} candidate(s) failed the paired test or held-out gate; do not ship those variants.`,\n    )\n  }\n  if (ctx.failureClusters && ctx.failureClusters.clusters.length > 0) {\n    const top = ctx.failureClusters.clusters[0]!\n    risks.push(`Top failure cluster: ${top.failureClass} across ${top.runCount} run(s).`)\n    nextActions.push('Prioritize the largest failure cluster before broad rollout.')\n  }\n  if (decision === 'promote') {\n    nextActions.push('Ship behind the existing promotion gate and monitor canaries.')\n  } else if (decision === 'hold') {\n    nextActions.push('Keep current production candidate while expanding holdout evidence.')\n  } else if (decision === 'equivalent') {\n    nextActions.push(\n      'Either keep the comparator (no quality regression) or promote on cost/latency grounds — equivalence does not justify either; the choice is a product decision, not a stats one.',\n    )\n  } else if (decision === 'reject') {\n    nextActions.push(\n      'Do not promote this sweep; inspect failures and generate a revised candidate.',\n    )\n  }\n\n  return {\n    decision,\n    candidateId: chosen?.candidateId ?? null,\n    rationale,\n    risks,\n    nextActions,\n  }\n}\n\nfunction buildExecutiveSummary(\n  candidates: ResearchReportCandidate[],\n  recommendation: ResearchReportRecommendation,\n  ctx: {\n    comparator: string | null\n    split: 'search' | 'holdout'\n    failureClusters?: FailureClusterReport\n    preregistrationHash: string | null\n  },\n): string[] {\n  const lines: string[] = []\n  const nonComparator = candidates.filter((c) => c.candidateId !== ctx.comparator)\n  lines.push(\n    `Evaluated ${nonComparator.length} candidate(s) on the ${ctx.split} split${ctx.comparator ? ` against ${ctx.comparator}` : ''}.`,\n  )\n  lines.push(\n    `Recommendation: ${recommendation.decision}${recommendation.candidateId ? ` ${recommendation.candidateId}` : ''}.`,\n  )\n  const promoted = nonComparator.filter((c) => c.decision === 'promote').length\n  const held = nonComparator.filter((c) => c.decision === 'hold').length\n  const equivalent = nonComparator.filter((c) => c.decision === 'equivalent').length\n  const rejected = nonComparator.filter((c) => c.decision === 'reject').length\n  const more = nonComparator.filter((c) => c.decision === 'needs_more_data').length\n  lines.push(\n    `Decision mix: ${promoted} promote, ${equivalent} equivalent, ${held} hold, ${rejected} reject, ${more} need more data.`,\n  )\n  const frontier = nonComparator.filter((c) => c.onParetoFrontier).map((c) => c.candidateId)\n  if (frontier.length > 0) lines.push(`Pareto-frontier candidates: ${frontier.join(', ')}.`)\n  if (ctx.failureClusters) {\n    lines.push(\n      `Failure clustering found ${ctx.failureClusters.totalFailures}/${ctx.failureClusters.totalRuns} failed runs across ${ctx.failureClusters.clusters.length} reportable cluster(s).`,\n    )\n  }\n  lines.push(\n    ctx.preregistrationHash\n      ? `Preregistered analysis: ${ctx.preregistrationHash.slice(0, 12)}…`\n      : 'Analysis is post-hoc — no preregistration hash supplied.',\n  )\n  return lines\n}\n\nfunction renderResearchMarkdown(report: {\n  title: string\n  generatedAt: string\n  split: 'search' | 'holdout'\n  comparator: string | null\n  executiveSummary: string[]\n  recommendation: ResearchReportRecommendation\n  candidates: ResearchReportCandidate[]\n  summary: SummaryTable\n  pareto: ParetoFigureSpec\n  gains: GainDistributionFigureSpec[]\n  rope: { low: number; high: number } | null\n  runFingerprint: string\n  preregistrationHash: string | null\n  methodology: ResearchReportMethodology\n  failureClusters?: FailureClusterReport\n}): string {\n  const lines: string[] = []\n  lines.push(`# ${report.title}`)\n  lines.push('')\n  lines.push(`**Generated:** ${report.generatedAt}`)\n  lines.push(`**Primary split:** ${report.split}`)\n  lines.push(`**Comparator:** ${report.comparator ?? 'not configured'}`)\n  lines.push(`**ROPE:** ${report.rope ? formatRope(report.rope) : 'not configured'}`)\n  lines.push(`**Run fingerprint:** \\`${report.runFingerprint}\\``)\n  lines.push(\n    `**Preregistration:** ${report.preregistrationHash ? `\\`${report.preregistrationHash}\\`` : 'none'}`,\n  )\n  lines.push('')\n  lines.push('## Executive Summary')\n  lines.push('')\n  for (const item of report.executiveSummary) lines.push(`- ${item}`)\n  lines.push('')\n  lines.push('## Recommendation')\n  lines.push('')\n  lines.push(`**Decision:** ${report.recommendation.decision}`)\n  lines.push(`**Candidate:** ${report.recommendation.candidateId ?? 'N/A'}`)\n  lines.push('')\n  lines.push('### Rationale')\n  lines.push('')\n  for (const item of report.recommendation.rationale) lines.push(`- ${item}`)\n  lines.push('')\n  lines.push('### Risks')\n  lines.push('')\n  for (const item of report.recommendation.risks.length\n    ? report.recommendation.risks\n    : ['No material report-level risks detected.']) {\n    lines.push(`- ${item}`)\n  }\n  lines.push('')\n  lines.push('### Next Actions')\n  lines.push('')\n  for (const item of report.recommendation.nextActions) lines.push(`- ${item}`)\n  lines.push('')\n  lines.push('## Candidate Decision Table')\n  lines.push('')\n  lines.push(\n    '| Candidate | Decision | Mean | Δ̄ | Pr(Δ>0) | q | d | Paired N | Median Gain CI | MDE | Pareto | Gate |',\n  )\n  lines.push('|---|---|---:|---:|---:|---:|---:|---:|---|---:|---|---|')\n  for (const c of report.candidates) {\n    const delta = c.meanDeltaVsComparator === null ? '-' : signed(c.meanDeltaVsComparator)\n    const prGt = c.prGreaterThanZero === null ? '-' : c.prGreaterThanZero.toFixed(3)\n    const q = Number.isFinite(c.qValue) ? c.qValue.toFixed(4) : '-'\n    const d = Number.isFinite(c.cohensD) ? c.cohensD.toFixed(3) : '-'\n    const gain = c.gainCi ? `[${fmt(c.gainCi.low)}, ${fmt(c.gainCi.high)}]` : '-'\n    const mde = c.mde === null || !Number.isFinite(c.mde) ? '-' : fmt(c.mde)\n    lines.push(\n      `| ${c.candidateId} | ${c.decision} | ${fmt(c.mean)} | ${delta} | ${prGt} | ${q} | ${d} | ${c.pairedN} | ${gain} | ${mde} | ${c.onParetoFrontier ? 'yes' : 'no'} | ${c.gate ?? '-'} |`,\n    )\n  }\n  lines.push('')\n  lines.push('## Statistical Summary')\n  lines.push('')\n  lines.push(report.summary.markdown)\n  lines.push('')\n  lines.push('## Methodology')\n  lines.push('')\n  lines.push('### Assumptions')\n  lines.push('')\n  for (const item of report.methodology.assumptions) lines.push(`- ${item}`)\n  lines.push('')\n  lines.push('### Methods')\n  lines.push('')\n  for (const item of report.methodology.methods) lines.push(`- ${item}`)\n  lines.push('')\n  lines.push('### Alternatives Considered')\n  lines.push('')\n  for (const item of report.methodology.alternatives) lines.push(`- ${item}`)\n  lines.push('')\n  lines.push('### When NOT To Apply')\n  lines.push('')\n  for (const item of report.methodology.whenNotToApply) lines.push(`- ${item}`)\n  lines.push('')\n  lines.push('### Citations')\n  lines.push('')\n  for (const item of report.methodology.citations) lines.push(`- ${item}`)\n  lines.push('')\n  lines.push('## Chart Specs')\n  lines.push('')\n  lines.push(\n    'The report carries JSON chart specs for Pareto cost/quality and paired gain histograms.',\n  )\n  lines.push('')\n  lines.push('```json')\n  lines.push(JSON.stringify({ pareto: report.pareto, gains: report.gains }, null, 2))\n  lines.push('```')\n  if (report.failureClusters) {\n    lines.push('')\n    lines.push('## Failure Clusters')\n    lines.push('')\n    lines.push('| Failure Class | Runs | Scenarios | Tool | Example |')\n    lines.push('|---|---:|---:|---|---|')\n    for (const c of report.failureClusters.clusters.slice(0, 10)) {\n      lines.push(\n        `| ${c.failureClass} | ${c.runCount} | ${c.scenarioIds.length} | ${c.toolName ?? '-'} | ${escapePipes(c.exampleError ?? c.exampleRunId)} |`,\n      )\n    }\n  }\n  return lines.join('\\n')\n}\n\nfunction renderResearchHtml(markdown: string, title: string): string {\n  const body = markdownToHtml(markdown)\n  return [\n    '<!doctype html>',\n    '<html lang=\"en\">',\n    '<head>',\n    '<meta charset=\"utf-8\">',\n    '<meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">',\n    `<title>${escapeHtml(title)}</title>`,\n    '<style>',\n    'body{font-family:Inter,ui-sans-serif,system-ui,-apple-system,BlinkMacSystemFont,\"Segoe UI\",sans-serif;margin:0;color:#172026;background:#f7f8f8;}',\n    'main{max-width:1080px;margin:0 auto;padding:40px 24px 64px;background:#fff;min-height:100vh;}',\n    'h1{font-size:34px;line-height:1.15;margin:0 0 20px;}h2{margin-top:34px;border-top:1px solid #d9dfdf;padding-top:22px;}h3{margin-top:22px;}',\n    'p,li{line-height:1.55;}table{border-collapse:collapse;width:100%;margin:16px 0;font-size:14px;}th,td{border:1px solid #d9dfdf;padding:8px;text-align:left;}th{background:#eef2f2;}',\n    'code,pre{font-family:ui-monospace,SFMono-Regular,Menlo,monospace;}pre{overflow:auto;background:#111827;color:#f9fafb;padding:16px;border-radius:6px;}',\n    '</style>',\n    '</head>',\n    '<body><main>',\n    body,\n    '</main></body></html>',\n  ].join('\\n')\n}\n\nfunction markdownToHtml(markdown: string): string {\n  const lines = markdown.split('\\n')\n  const html: string[] = []\n  let inList = false\n  let inCode = false\n  let code: string[] = []\n  let table: string[] = []\n\n  const flushList = () => {\n    if (inList) {\n      html.push('</ul>')\n      inList = false\n    }\n  }\n  const flushTable = () => {\n    if (table.length === 0) return\n    html.push(renderMarkdownTable(table))\n    table = []\n  }\n\n  for (const line of lines) {\n    if (line.startsWith('```')) {\n      if (inCode) {\n        html.push(`<pre><code>${escapeHtml(code.join('\\n'))}</code></pre>`)\n        code = []\n        inCode = false\n      } else {\n        flushList()\n        flushTable()\n        inCode = true\n      }\n      continue\n    }\n    if (inCode) {\n      code.push(line)\n      continue\n    }\n    if (line.startsWith('|')) {\n      flushList()\n      table.push(line)\n      continue\n    }\n    flushTable()\n    if (line.startsWith('- ')) {\n      if (!inList) {\n        html.push('<ul>')\n        inList = true\n      }\n      html.push(`<li>${inlineMarkdown(line.slice(2))}</li>`)\n      continue\n    }\n    flushList()\n    if (line.startsWith('# ')) html.push(`<h1>${inlineMarkdown(line.slice(2))}</h1>`)\n    else if (line.startsWith('## ')) html.push(`<h2>${inlineMarkdown(line.slice(3))}</h2>`)\n    else if (line.startsWith('### ')) html.push(`<h3>${inlineMarkdown(line.slice(4))}</h3>`)\n    else if (line.trim() === '') html.push('')\n    else html.push(`<p>${inlineMarkdown(line)}</p>`)\n  }\n  flushList()\n  flushTable()\n  return html.join('\\n')\n}\n\nfunction renderMarkdownTable(lines: string[]): string {\n  const rows = lines\n    .filter((line) => !/^\\|[-:\\s|]+\\|$/.test(line))\n    .map((line) =>\n      line\n        .slice(1, -1)\n        .split('|')\n        .map((cell) => inlineMarkdown(cell.trim())),\n    )\n  if (rows.length === 0) return ''\n  const [head, ...body] = rows\n  const th = head!.map((cell) => `<th>${cell}</th>`).join('')\n  const trs = body\n    .map((row) => `<tr>${row.map((cell) => `<td>${cell}</td>`).join('')}</tr>`)\n    .join('\\n')\n  return `<table><thead><tr>${th}</tr></thead><tbody>${trs}</tbody></table>`\n}\n\nfunction inlineMarkdown(s: string): string {\n  return escapeHtml(s).replace(/\\*\\*([^*]+)\\*\\*/g, '<strong>$1</strong>')\n}\n\nfunction escapeHtml(s: string): string {\n  return s\n    .replace(/&/g, '&amp;')\n    .replace(/</g, '&lt;')\n    .replace(/>/g, '&gt;')\n    .replace(/\"/g, '&quot;')\n}\n\nfunction escapePipes(s: string): string {\n  return s.replace(/\\|/g, '\\\\|')\n}\n\nfunction decisionWeight(decision: ResearchReportDecision): number {\n  if (decision === 'promote') return 5\n  if (decision === 'equivalent') return 4\n  if (decision === 'hold') return 3\n  if (decision === 'needs_more_data') return 2\n  return 1\n}\n\nfunction signed(x: number): string {\n  return `${x >= 0 ? '+' : ''}${fmt(x)}`\n}\n\n// ── tiny helpers ─────────────────────────────────────────────────────\n\nfunction avg(xs: number[]): number {\n  if (xs.length === 0) return Number.NaN\n  return xs.reduce((s, x) => s + x, 0) / xs.length\n}\n\nfunction medianOfSorted(sorted: number[]): number {\n  if (sorted.length === 0) return 0\n  const mid = Math.floor(sorted.length / 2)\n  return sorted.length % 2 === 0 ? (sorted[mid - 1]! + sorted[mid]!) / 2 : sorted[mid]!\n}\n\nfunction fmt(x: number): string {\n  if (!Number.isFinite(x)) return String(x)\n  return x.toFixed(4)\n}\n"],"mappings":";;;;;;;;;;;AAuBO,SAAS,mBAAmB,MAKxB;AACT,QAAM,SAAS,KAAK;AACpB,MAAI,CAAC,OAAO,SAAS,MAAM,KAAK,UAAU,EAAG,QAAO;AACpD,QAAM,QAAQ,KAAK,SAAS;AAC5B,QAAM,QAAQ,KAAK,SAAS;AAC5B,QAAM,WAAW,KAAK,YAAY;AAClC,QAAM,SAAS,UAAU,WAAW,IAAI,QAAQ,IAAI,IAAI,KAAK;AAC7D,QAAM,QAAQ,UAAU,KAAK;AAC7B,QAAM,IAAI,MAAM,SAAS,SAAS,WAAW;AAC7C,SAAO,KAAK,KAAK,CAAC;AACpB;AAcO,SAAS,UAAU,MAKf;AACT,MAAI,CAAC,OAAO,SAAS,KAAK,OAAO,KAAK,KAAK,WAAW,EAAG,QAAO;AAChE,QAAM,QAAQ,KAAK,SAAS;AAC5B,QAAM,QAAQ,KAAK,SAAS;AAC5B,QAAM,WAAW,KAAK,YAAY;AAClC,QAAM,SAAS,UAAU,WAAW,IAAI,QAAQ,IAAI,IAAI,KAAK;AAC7D,QAAM,QAAQ,UAAU,KAAK;AAC7B,UAAQ,SAAS,SAAS,KAAK,KAAK,KAAK,OAAO;AAClD;AAGO,SAAS,WACd,SACA,QAAQ,MACwC;AAChD,QAAM,IAAI,QAAQ;AAClB,QAAM,WAAW,QAAQ,IAAI,CAAC,MAAM,KAAK,IAAI,GAAG,IAAI,CAAC,CAAC;AACtD,QAAM,cAAc,SAAS,IAAI,CAAC,MAAM,IAAI,KAAK;AACjD,SAAO,EAAE,UAAU,YAAY;AACjC;AAOO,SAAS,kBACd,SACA,MAAM,MACyC;AAC/C,QAAM,IAAI,QAAQ;AAClB,MAAI,MAAM,EAAG,QAAO,EAAE,SAAS,CAAC,GAAG,aAAa,CAAC,EAAE;AACnD,QAAM,UAAU,QAAQ,IAAI,CAAC,GAAG,OAAO,EAAE,GAAG,EAAE,EAAE,EAAE,KAAK,CAAC,GAAG,MAAM,EAAE,IAAI,EAAE,CAAC;AAC1E,QAAM,IAAI,IAAI,MAAc,CAAC;AAE7B,MAAI,WAAW;AACf,WAAS,IAAI,IAAI,GAAG,KAAK,GAAG,KAAK;AAC/B,UAAM,OAAO,IAAI;AACjB,UAAM,QAAQ,QAAQ,CAAC;AACvB,UAAM,MAAO,MAAM,IAAI,IAAK;AAC5B,UAAM,UAAU,KAAK,IAAI,UAAU,GAAG;AACtC,eAAW;AACX,MAAE,MAAM,CAAC,IAAI,KAAK,IAAI,GAAG,OAAO;AAAA,EAClC;AACA,QAAM,cAAc,EAAE,IAAI,CAAC,MAAM,IAAI,GAAG;AACxC,SAAO,EAAE,SAAS,GAAG,YAAY;AACnC;AAGA,SAAS,UAAU,GAAmB;AACpC,MAAI,KAAK,KAAK,KAAK,GAAG;AACpB,QAAI,MAAM,EAAG,QAAO;AACpB,QAAI,MAAM,EAAG,QAAO;AACpB,WAAO;AAAA,EACT;AACA,QAAM,IAAI;AAAA,IACR;AAAA,IAAsB;AAAA,IAAqB;AAAA,IAAsB;AAAA,IACjE;AAAA,IAAsB;AAAA,EACxB;AACA,QAAM,IAAI;AAAA,IACR;AAAA,IAAsB;AAAA,IAAqB;AAAA,IAAsB;AAAA,IACjE;AAAA,EACF;AACA,QAAM,IAAI;AAAA,IACR;AAAA,IAAuB;AAAA,IAAuB;AAAA,IAAoB;AAAA,IAClE;AAAA,IAAmB;AAAA,EACrB;AACA,QAAM,IAAI,CAAC,sBAAsB,oBAAsB,mBAAmB,iBAAiB;AAC3F,QAAM,OAAO;AACb,QAAM,QAAQ,IAAI;AAClB,MAAI;AACJ,MAAI;AACJ,MAAI,IAAI,MAAM;AACZ,QAAI,KAAK,KAAK,KAAK,KAAK,IAAI,CAAC,CAAC;AAC9B,gBACO,EAAE,CAAC,IAAK,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,SACpE,EAAE,CAAC,IAAK,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,KAAM,IAAI;AAAA,EAE3D;AACA,MAAI,KAAK,OAAO;AACd,QAAI,IAAI;AACR,QAAI,IAAI;AACR,gBACQ,EAAE,CAAC,IAAK,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,KAAM,SAC1E,EAAE,CAAC,IAAK,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,KAAM,IAAI;AAAA,EAEzE;AACA,MAAI,KAAK,KAAK,KAAK,KAAK,IAAI,IAAI,CAAC,CAAC;AAClC,SACE,MAAM,EAAE,CAAC,IAAK,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,SACrE,EAAE,CAAC,IAAK,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,KAAM,IAAI,EAAE,CAAC,KAAM,IAAI;AAE3D;;;ACxFO,SAAS,gBACd,QACA,OACA,OAA+B,CAAC,GACT;AACvB,MAAI,OAAO,WAAW,MAAM,QAAQ;AAClC,UAAM,IAAI,MAAM,0CAA0C,OAAO,MAAM,OAAO,MAAM,MAAM,GAAG;AAAA,EAC/F;AACA,QAAM,aAAa,KAAK,cAAc;AACtC,QAAM,YAAY,KAAK,aAAa;AACpC,QAAM,YAAY,KAAK,aAAa;AACpC,MAAI,cAAc,KAAK,cAAc,GAAG;AACtC,UAAM,IAAI,MAAM,qDAAqD,UAAU,EAAE;AAAA,EACnF;AAEA,QAAM,IAAI,OAAO;AACjB,QAAM,SAAS,OAAO,IAAI,CAAC,GAAG,MAAM,MAAM,CAAC,IAAK,CAAC;AACjD,MAAI,MAAM,GAAG;AACX,WAAO,EAAE,GAAG,GAAG,QAAQ,GAAG,MAAM,GAAG,KAAK,GAAG,MAAM,GAAG,YAAY,UAAU;AAAA,EAC5E;AACA,MAAI,MAAM,GAAG;AACX,UAAM,IAAI,OAAO,CAAC;AAClB,WAAO,EAAE,GAAG,GAAG,QAAQ,GAAG,MAAM,GAAG,KAAK,GAAG,MAAM,GAAG,YAAY,UAAU;AAAA,EAC5E;AAEA,QAAM,MAAM,QAAQ,KAAK,IAAI;AAC7B,QAAM,UAAU,IAAI,MAAc,SAAS;AAC3C,WAAS,IAAI,GAAG,IAAI,WAAW,KAAK;AAClC,QAAI,MAAuB;AAC3B,QAAI,cAAc,QAAQ;AACxB,UAAI,MAAM;AACV,eAAS,IAAI,GAAG,IAAI,GAAG,KAAK;AAC1B,eAAO,OAAO,KAAK,MAAM,IAAI,IAAI,CAAC,CAAC;AAAA,MACrC;AACA,cAAQ,CAAC,IAAI,MAAM;AAAA,IACrB,OAAO;AACL,YAAM,IAAI,MAAc,CAAC;AACzB,eAAS,IAAI,GAAG,IAAI,GAAG,KAAK;AAC1B,YAAI,CAAC,IAAI,OAAO,KAAK,MAAM,IAAI,IAAI,CAAC,CAAC;AAAA,MACvC;AACA,cAAQ,CAAC,IAAI,cAAc,GAAG;AAAA,IAChC;AAAA,EACF;AACA,UAAQ,KAAK,CAAC,GAAG,MAAM,IAAI,CAAC;AAE5B,QAAM,QAAQ,IAAI;AAClB,QAAM,SAAS,KAAK,MAAO,QAAQ,IAAK,SAAS;AACjD,QAAM,UAAU,KAAK,IAAI,YAAY,GAAG,KAAK,MAAM,IAAI,QAAQ,KAAK,SAAS,IAAI,CAAC;AAElF,SAAO;AAAA,IACL;AAAA,IACA,QAAQ,cAAc,CAAC,GAAG,MAAM,CAAC;AAAA,IACjC,MAAM,OAAO,OAAO,CAAC,GAAG,MAAM,IAAI,GAAG,CAAC,IAAI;AAAA,IAC1C,KAAK,QAAQ,MAAM;AAAA,IACnB,MAAM,QAAQ,KAAK,IAAI,SAAS,MAAM,CAAC;AAAA,IACvC;AAAA,IACA;AAAA,EACF;AACF;AAQO,SAAS,eAAe,QAAkB,OAA2C;AAC1F,SAAO,mBAAmB,QAAQ,KAAK;AACzC;AAQO,SAAS,SACd,SACA,MAAM,MACyC;AAC/C,SAAO,kBAAkB,SAAS,GAAG;AACvC;AAIA,SAAS,cAAc,IAAsB;AAC3C,MAAI,GAAG,WAAW,EAAG,QAAO;AAC5B,KAAG,KAAK,CAAC,GAAG,MAAM,IAAI,CAAC;AACvB,QAAM,MAAM,KAAK,MAAM,GAAG,SAAS,CAAC;AACpC,SAAO,GAAG,SAAS,MAAM,KAAK,GAAG,MAAM,CAAC,IAAK,GAAG,GAAG,KAAM,IAAI,GAAG,GAAG;AACrE;AAOA,SAAS,QAAQ,MAAwC;AACvD,MAAI,SAAS,OAAW,QAAO,KAAK;AACpC,MAAI,IAAI,OAAO,KAAK;AACpB,SAAO,MAAM;AACX,QAAK,IAAI,aAAc;AACvB,QAAI,IAAI;AACR,QAAI,KAAK,KAAK,IAAK,MAAM,IAAK,IAAI,CAAC;AACnC,SAAK,IAAI,KAAK,KAAK,IAAK,MAAM,GAAI,IAAI,EAAE;AACxC,aAAS,IAAK,MAAM,QAAS,KAAK;AAAA,EACpC;AACF;;;AChGO,SAAS,aAAa,MAAmB,OAA4B,CAAC,GAAiB;AAC5F,QAAM,QAAQ,KAAK,SAAS;AAC5B,QAAM,aAAa,KAAK,cAAc;AACtC,QAAM,MAAM,KAAK,OAAO;AACxB,QAAM,aAAa,KAAK,cAAc;AACtC,QAAM,aAAa,UAAU,YAAY,iBAAiB;AAE1D,QAAM,cAAc,oBAAI,IAAqD;AAC7E,aAAW,KAAK,MAAM;AACpB,QAAI,EAAE,aAAa,MAAO;AAC1B,UAAM,IAAI,EAAE,QAAQ,UAAU;AAC9B,QAAI,OAAO,MAAM,YAAY,CAAC,OAAO,SAAS,CAAC,EAAG;AAClD,UAAM,SAAS,YAAY,IAAI,EAAE,WAAW,KAAK,EAAE,MAAM,CAAC,GAAG,QAAQ,CAAC,EAAE;AACxE,WAAO,KAAK,KAAK,CAAC;AAClB,WAAO,OAAO,KAAK,CAAC;AACpB,gBAAY,IAAI,EAAE,aAAa,MAAM;AAAA,EACvC;AAEA,QAAM,eAAe,CAAC,GAAG,YAAY,KAAK,CAAC,EAAE,KAAK;AAClD,QAAM,WAAW,aAAa,YAAY,IAAI,UAAU,IAAI;AAG5D,QAAM,YAAuD,CAAC;AAC9D,aAAW,MAAM,cAAc;AAC7B,UAAM,SAAS,YAAY,IAAI,EAAE;AACjC,UAAM,KAAK,mBAAmB,OAAO,QAAQ,UAAU;AACvD,QAAI,OAAO,OAAO;AAClB,QAAI,IAAI,OAAO;AACf,QAAI,cAAc,YAAY,OAAO,YAAY;AAC/C,YAAM,SAAS,gBAAgB,OAAO,MAAM,SAAS,MAAM,UAAU;AACrE,UAAI,OAAO,OAAO,UAAU,GAAG;AAC7B,eAAO,mBAAmB,OAAO,QAAQ,OAAO,KAAK,EAAE;AAAA,MACzD;AACA,UAAI,QAAQ,SAAS,QAAQ,OAAO,MAAM;AAAA,IAC5C;AACA,cAAU,KAAK;AAAA,MACb,aAAa;AAAA,MACb,GAAG,OAAO,OAAO;AAAA,MACjB,MAAM,GAAG;AAAA,MACT,OAAO,GAAG;AAAA,MACV,QAAQ,GAAG;AAAA,MACX,QAAQ;AAAA,MACR,SAAS;AAAA,MACT;AAAA,IACF,CAAC;AAAA,EACH;AAKA,MAAI,YAAY;AACd,UAAM,OAAiB,CAAC;AACxB,UAAM,KAAe,CAAC;AACtB,aAAS,IAAI,GAAG,IAAI,UAAU,QAAQ,KAAK;AACzC,YAAM,IAAI,UAAU,CAAC;AACrB,UAAI,EAAE,gBAAgB,WAAY;AAClC,UAAI,CAAC,OAAO,SAAS,EAAE,IAAI,EAAG;AAC9B,WAAK,KAAK,CAAC;AACX,SAAG,KAAK,EAAE,IAAI;AAAA,IAChB;AACA,QAAI,GAAG,SAAS,GAAG;AACjB,YAAM,EAAE,QAAQ,IAAI,kBAAkB,IAAI,GAAG;AAC7C,eAAS,IAAI,GAAG,IAAI,KAAK,QAAQ,KAAK;AACpC,kBAAU,KAAK,CAAC,CAAE,EAAG,SAAS,QAAQ,CAAC;AAAA,MACzC;AAAA,IACF;AAAA,EACF;AAEA,QAAM,OAAO,UAAU,IAAI,CAAC,EAAE,MAAM,OAAO,GAAG,KAAK,MAAM,IAAI;AAC7D,QAAM,WAAW,2BAA2B,MAAM,YAAY,KAAK;AACnE,SAAO,EAAE,MAAM,YAAY,OAAO,SAAS;AAC7C;AAEA,SAAS,gBACP,WACA,UACA,YACuC;AACvC,QAAM,UAAU,oBAAI,IAAoB;AACxC,aAAW,KAAK,UAAU;AACxB,UAAM,IAAI,EAAE,QAAQ,UAAU;AAC9B,QAAI,OAAO,MAAM,YAAY,OAAO,SAAS,CAAC,GAAG;AAC/C,cAAQ,IAAI,GAAG,EAAE,YAAY,KAAK,EAAE,IAAI,IAAI,CAAC;AAAA,IAC/C;AAAA,EACF;AACA,QAAM,SAAmB,CAAC;AAC1B,QAAM,QAAkB,CAAC;AACzB,aAAW,KAAK,WAAW;AACzB,UAAM,IAAI,EAAE,QAAQ,UAAU;AAC9B,QAAI,OAAO,MAAM,YAAY,CAAC,OAAO,SAAS,CAAC,EAAG;AAClD,UAAM,MAAM,GAAG,EAAE,YAAY,KAAK,EAAE,IAAI;AACxC,UAAM,IAAI,QAAQ,IAAI,GAAG;AACzB,QAAI,MAAM,OAAW;AACrB,WAAO,KAAK,CAAC;AACb,UAAM,KAAK,CAAC;AAAA,EACd;AACA,SAAO,EAAE,QAAQ,MAAM;AACzB;AAEA,SAAS,2BACP,MACA,YACA,OACQ;AACR,QAAM,QAAkB,CAAC;AACzB,QAAM,WAAW,aAAa,QAAQ,UAAU,MAAM;AACtD,QAAM,KAAK,wBAAmB,KAAK,SAAS,QAAQ,EAAE;AACtD,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,wDAAwD;AACnE,QAAM,KAAK,+BAA+B;AAC1C,aAAW,KAAK,MAAM;AACpB,UAAM,KAAK,IAAI,IAAI,EAAE,KAAK,CAAC,KAAK,IAAI,EAAE,MAAM,CAAC;AAC7C,UAAM,IAAI,OAAO,SAAS,EAAE,MAAM,IAAI,EAAE,OAAO,QAAQ,CAAC,IAAI;AAC5D,UAAM,IAAI,OAAO,SAAS,EAAE,OAAO,IAAI,EAAE,QAAQ,QAAQ,CAAC,IAAI;AAC9D,UAAM,KAAK,KAAK,EAAE,WAAW,MAAM,EAAE,CAAC,MAAM,IAAI,EAAE,IAAI,CAAC,MAAM,EAAE,MAAM,CAAC,MAAM,CAAC,IAAI;AAAA,EACnF;AACA,SAAO,MAAM,KAAK,IAAI;AACxB;AAgCO,SAAS,YACd,MACA,OAGI,CAAC,GACa;AAClB,QAAM,QAAQ,KAAK,SAAS;AAC5B,QAAM,aAAa,UAAU,YAAY,iBAAiB;AAE1D,QAAM,UAAU,oBAAI,IAAmD;AACvE,aAAW,KAAK,MAAM;AACpB,QAAI,EAAE,aAAa,MAAO;AAC1B,UAAM,IAAI,EAAE,QAAQ,UAAU;AAC9B,QAAI,OAAO,MAAM,YAAY,CAAC,OAAO,SAAS,CAAC,EAAG;AAClD,UAAM,SAAS,QAAQ,IAAI,EAAE,WAAW,KAAK,EAAE,MAAM,CAAC,GAAG,SAAS,CAAC,EAAE;AACrE,WAAO,KAAK,KAAK,EAAE,OAAO;AAC1B,WAAO,QAAQ,KAAK,CAAC;AACrB,YAAQ,IAAI,EAAE,aAAa,MAAM;AAAA,EACnC;AAEA,QAAM,SAAwB,CAAC;AAC/B,aAAW,CAAC,aAAa,MAAM,KAAK,QAAQ,QAAQ,GAAG;AACrD,WAAO,KAAK;AAAA,MACV;AAAA,MACA,MAAM,IAAI,OAAO,IAAI;AAAA,MACrB,SAAS,IAAI,OAAO,OAAO;AAAA,MAC3B,GAAG,OAAO,KAAK;AAAA,MACf,YAAY;AAAA,MACZ,MAAM,KAAK,gBAAgB,WAAW,IAClC,UAAU,KAAK,cAAc,WAAW,CAAE,IAC1C;AAAA,IACN,CAAC;AAAA,EACH;AAKA,aAAW,KAAK,QAAQ;AACtB,MAAE,aAAa,CAAC,OAAO,KAAK,CAAC,MAAM,MAAM,KAAK,UAAU,GAAG,CAAC,CAAC;AAAA,EAC/D;AAEA,SAAO;AAAA,IACL,MAAM;AAAA,IACN;AAAA,IACA,MAAM,EAAE,GAAG,WAAW,GAAG,QAAQ;AAAA,IACjC;AAAA,EACF;AACF;AAEA,SAAS,UAAU,GAAgB,GAAyB;AAC1D,SAAO,EAAE,QAAQ,EAAE,QAAQ,EAAE,WAAW,EAAE,YAAY,EAAE,OAAO,EAAE,QAAQ,EAAE,UAAU,EAAE;AACzF;AAEA,SAAS,UAAU,GAAsC;AACvD,MAAI,EAAE,QAAS,QAAO;AACtB,MAAI,EAAE,kBAAkB,WAAY,QAAO;AAC3C,MAAI,EAAE,kBAAkB,iBAAkB,QAAO;AACjD,MAAI,EAAE,kBAAkB,cAAe,QAAO;AAC9C,SAAO;AACT;AA2CO,SAAS,cACd,MACA,aACA,YACA,OAAgC,CAAC,GACL;AAC5B,QAAM,QAAQ,KAAK,SAAS;AAC5B,QAAM,aAAa,UAAU,YAAY,iBAAiB;AAC1D,QAAM,WAAW,KAAK,QAAQ;AAC9B,MAAI,WAAW,EAAG,OAAM,IAAI,MAAM,sCAAiC;AAEnE,QAAM,YAAY,KAAK,OAAO,CAAC,MAAM,EAAE,gBAAgB,eAAe,EAAE,aAAa,KAAK;AAC1F,QAAM,WAAW,KAAK,OAAO,CAAC,MAAM,EAAE,gBAAgB,cAAc,EAAE,aAAa,KAAK;AAIxF,QAAM,EAAE,QAAQ,MAAM,IAAI,gBAAgB,WAAW,UAAU,UAAU;AACzE,QAAM,IAAI,OAAO;AAEjB,MAAI,MAAM,GAAG;AACX,WAAO;AAAA,MACL,MAAM;AAAA,MACN;AAAA,MACA;AAAA,MACA;AAAA,MACA,GAAG;AAAA,MACH,MAAM,CAAC;AAAA,MACP,QAAQ;AAAA,MACR,IAAI,EAAE,KAAK,GAAG,MAAM,EAAE;AAAA,IACxB;AAAA,EACF;AAEA,QAAM,SAAS,OAAO,IAAI,CAAC,GAAG,MAAM,MAAM,CAAC,IAAK,CAAC;AACjD,QAAM,eAAe,CAAC,GAAG,MAAM,EAAE,KAAK,CAAC,GAAG,MAAM,IAAI,CAAC;AACrD,QAAM,SAAS,eAAe,YAAY;AAC1C,QAAM,MAAM,aAAa,CAAC;AAC1B,QAAM,MAAM,aAAa,aAAa,SAAS,CAAC;AAIhD,QAAM,QAAQ,KAAK,IAAI,KAAK,IAAI,GAAG,GAAG,KAAK,IAAI,GAAG,GAAG,IAAI;AACzD,QAAM,KAAK,CAAC;AACZ,QAAM,KAAK;AACX,QAAM,SAAS,KAAK,MAAM;AAC1B,QAAM,OAA8B,CAAC;AACrC,WAAS,IAAI,GAAG,IAAI,UAAU,KAAK;AACjC,SAAK,KAAK,EAAE,IAAI,KAAK,IAAI,OAAO,IAAI,MAAM,IAAI,KAAK,OAAO,OAAO,EAAE,CAAC;AAAA,EACtE;AACA,aAAW,KAAK,QAAQ;AACtB,QAAI,MAAM,KAAK,OAAO,IAAI,MAAM,KAAK;AACrC,QAAI,MAAM,EAAG,OAAM;AACnB,QAAI,OAAO,SAAU,OAAM,WAAW;AACtC,SAAK,GAAG,EAAG,SAAS;AAAA,EACtB;AAEA,QAAM,KAAK,gBAAgB,QAAQ,OAAO;AAAA,IACxC,YAAY,KAAK,cAAc;AAAA,IAC/B,WAAW,KAAK,aAAa;AAAA,IAC7B,WAAW;AAAA,IACX,MAAM,KAAK;AAAA,EACb,CAAC;AAED,SAAO;AAAA,IACL,MAAM;AAAA,IACN;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA,IAAI,EAAE,KAAK,GAAG,KAAK,MAAM,GAAG,KAAK;AAAA,EACnC;AACF;AAgBO,IAAM,kCAAkC;AA2J/C,SAAS,gBACP,MACA,aACA,YACA,MAiBO;AACP,QAAM,aAAa,KAAK,UAAU,YAAY,iBAAiB;AAC/D,QAAM,YAAY,KAAK,OAAO,CAAC,MAAM,EAAE,gBAAgB,eAAe,EAAE,aAAa,KAAK,KAAK;AAC/F,QAAM,WAAW,KAAK,OAAO,CAAC,MAAM,EAAE,gBAAgB,cAAc,EAAE,aAAa,KAAK,KAAK;AAC7F,QAAM,EAAE,QAAQ,MAAM,IAAI,gBAAgB,WAAW,UAAU,UAAU;AACzE,QAAM,IAAI,OAAO;AACjB,MAAI,MAAM,EAAG,QAAO;AAEpB,QAAM,SAAS,OAAO,IAAI,CAAC,GAAG,MAAM,MAAM,CAAC,IAAK,CAAC;AACjD,QAAM,YAAY,OAAO,OAAO,CAAC,GAAG,MAAM,IAAI,GAAG,CAAC,IAAI;AACtD,QAAM,eAAe,CAAC,GAAG,MAAM,EAAE,KAAK,CAAC,GAAG,MAAM,IAAI,CAAC;AACrD,QAAM,cAAc,eAAe,YAAY;AAC/C,QAAM,UAAU,MAAM,QAAQ,SAAS;AAEvC,QAAM,KAAK,gBAAgB,QAAQ,OAAO;AAAA,IACxC,YAAY,KAAK;AAAA,IACjB,WAAW;AAAA,IACX,WAAW;AAAA,IACX,MAAM,KAAK;AAAA,EACb,CAAC;AAKD,QAAM,cAAc,qBAAqB,QAAQ,KAAM,KAAK,IAAI;AAChE,QAAM,oBACJ,YAAY,WAAW,IAAI,IAAI,YAAY,OAAO,CAAC,MAAM,IAAI,CAAC,EAAE,SAAS,YAAY;AACvF,QAAM,WACJ,KAAK,SAAS,QAAQ,YAAY,WAAW,IACzC,OACA,YAAY,OAAO,CAAC,MAAM,KAAK,KAAK,KAAM,OAAO,KAAK,KAAK,KAAM,IAAI,EAAE,SACvE,YAAY;AAElB,QAAM,gBAAgB,UAAU,EAAE,SAAS,GAAG,OAAO,KAAK,UAAU,OAAO,KAAK,SAAS,CAAC;AAC1F,QAAM,MAAM,YAAY,IAAI,IAAI,gBAAgB;AAEhD,SAAO;AAAA,IACL;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA,IAAI,EAAE,KAAK,GAAG,KAAK,MAAM,GAAG,KAAK;AAAA,IACjC;AAAA,IACA;AAAA,IACA;AAAA,EACF;AACF;AAEA,SAAS,qBAAqB,QAAkB,WAAmB,MAAyB;AAC1F,QAAM,IAAI,OAAO;AACjB,MAAI,MAAM,EAAG,QAAO,CAAC;AACrB,MAAI,MAAM,EAAG,QAAO,IAAI,MAAc,SAAS,EAAE,KAAK,OAAO,CAAC,CAAE;AAChE,QAAM,MAAM,QAAQ,IAAI;AACxB,QAAM,UAAU,IAAI,MAAc,SAAS;AAC3C,WAAS,IAAI,GAAG,IAAI,WAAW,KAAK;AAClC,QAAI,MAAM;AACV,aAAS,IAAI,GAAG,IAAI,GAAG,IAAK,QAAO,OAAO,KAAK,MAAM,IAAI,IAAI,CAAC,CAAC;AAC/D,YAAQ,CAAC,IAAI,MAAM;AAAA,EACrB;AACA,SAAO;AACT;AAEA,SAAS,QAAQ,MAA6B;AAC5C,MAAI,SAAS,OAAW,QAAO,KAAK;AACpC,MAAI,IAAI,SAAS;AACjB,SAAO,MAAM;AACX,QAAK,IAAI,eAAgB;AACzB,QAAI,IAAI;AACR,QAAI,KAAK,KAAK,IAAK,MAAM,IAAK,IAAI,CAAC;AACnC,SAAK,IAAI,KAAK,KAAK,IAAK,MAAM,GAAI,IAAI,EAAE;AACxC,aAAS,IAAK,MAAM,QAAS,KAAK;AAAA,EACpC;AACF;AAEA,SAAS,MAAM,IAAc,MAAsB;AACjD,MAAI,GAAG,SAAS,EAAG,QAAO;AAC1B,MAAI,MAAM;AACV,aAAW,KAAK,GAAI,SAAQ,IAAI,SAAS;AACzC,SAAO,KAAK,KAAK,OAAO,GAAG,SAAS,EAAE;AACxC;AAqBA,eAAsB,eACpB,MACA,OAA8B,CAAC,GACN;AACzB,QAAM,QAAQ,KAAK,SAAS;AAC5B,QAAM,aAAa,KAAK,cAAc;AACtC,QAAM,aAAa,KAAK,cAAc;AACtC,QAAM,MAAM,KAAK,OAAO;AACxB,QAAM,WAAW,KAAK,IAAI,KAAK,YAAY,IAAI,+BAA+B;AAC9E,QAAM,OAAO,KAAK,QAAQ;AAC1B,QAAM,WAAW,KAAK,YAAY;AAClC,QAAM,WAAW,KAAK,YAAY;AAClC,QAAM,QAAQ,KAAK,SAAS;AAC5B,QAAM,cAAc,KAAK,gBAAe,oBAAI,KAAK,GAAE,YAAY;AAC/D,QAAM,sBAAsB,KAAK,uBAAuB;AAExD,MAAI,QAAQ,EAAE,OAAO,SAAS,KAAK,GAAG,KAAK,OAAO,SAAS,KAAK,IAAI,KAAK,KAAK,OAAO,KAAK,OAAO;AAC/F,UAAM,IAAI;AAAA,MACR,6EAAwE,KAAK,UAAU,IAAI,CAAC;AAAA,IAC9F;AAAA,EACF;AAEA,QAAM,UAAU,aAAa,MAAM;AAAA,IACjC,YAAY,cAAc;AAAA,IAC1B;AAAA,IACA;AAAA,IACA;AAAA,EACF,CAAC;AACD,QAAM,SAAS,YAAY,MAAM,EAAE,OAAO,eAAe,KAAK,cAAc,CAAC;AAC7E,QAAM,eACJ,KAAK,gBAAgB,QAAQ,KAAK,IAAI,CAAC,MAAM,EAAE,WAAW,EAAE,OAAO,CAAC,OAAO,OAAO,UAAU;AAC9F,QAAM,QAAQ,aACV,aAAa;AAAA,IAAI,CAAC,OAChB,cAAc,MAAM,IAAI,YAAY;AAAA,MAClC;AAAA,MACA;AAAA,MACA,MAAM,KAAK;AAAA,IACb,CAAC;AAAA,EACH,IACA,CAAC;AAEL,QAAM,kBAAkB,IAAI,IAAI,MAAM,IAAI,CAAC,MAAM,CAAC,EAAE,aAAa,CAAC,CAAC,CAAC;AACpE,QAAM,oBAAoB,IAAI,IAAI,OAAO,OAAO,IAAI,CAAC,MAAM,CAAC,EAAE,aAAa,CAAC,CAAC,CAAC;AAC9E,QAAM,uBAAuB,oBAAI,IAAgD;AACjF,MAAI,YAAY;AACd,eAAW,MAAM,cAAc;AAC7B,2BAAqB;AAAA,QACnB;AAAA,QACA,gBAAgB,MAAM,IAAI,YAAY;AAAA,UACpC;AAAA,UACA;AAAA,UACA,MAAM,KAAK;AAAA,UACX;AAAA,UACA;AAAA,UACA;AAAA,QACF,CAAC;AAAA,MACH;AAAA,IACF;AAAA,EACF;AAEA,QAAM,aAAa,QAAQ,KACxB,IAAI,CAAC,QAAQ;AACZ,UAAM,OAAO,gBAAgB,IAAI,IAAI,WAAW;AAChD,UAAM,QAAQ,kBAAkB,IAAI,IAAI,WAAW;AACnD,UAAM,YAAY,qBAAqB,IAAI,IAAI,WAAW,KAAK;AAC/D,UAAM,aAAa,kBAAkB,KAAK;AAAA,MACxC;AAAA,MACA;AAAA,MACA;AAAA,MACA;AAAA,MACA;AAAA,MACA;AAAA,IACF,CAAC;AACD,WAAO;AAAA,MACL,aAAa,IAAI;AAAA,MACjB,GAAG,IAAI;AAAA,MACP,MAAM,IAAI;AAAA,MACV,OAAO,IAAI;AAAA,MACX,QAAQ,IAAI;AAAA,MACZ,QAAQ,IAAI;AAAA,MACZ,SAAS,IAAI;AAAA,MACb,uBAAuB,YAAY,UAAU,YAAY;AAAA,MACzD,SAAS,WAAW,KAAK,MAAM,KAAK;AAAA,MACpC,YAAY,YAAY,UAAU,cAAc,OAAO,KAAK,SAAS;AAAA,MACrE,UAAU,YAAY,UAAU,YAAY;AAAA,MAC5C,QAAQ,YAAY,UAAU,KAAK,OAAO,KAAK,KAAK;AAAA,MACpD,mBAAmB,YAAY,UAAU,oBAAoB;AAAA,MAC7D,UAAU,YAAY,UAAU,WAAW;AAAA,MAC3C,KAAK,YAAY,UAAU,MAAM;AAAA,MACjC,kBAAkB,OAAO,cAAc;AAAA,MACvC,MAAM,OAAO;AAAA,MACb,UAAU,WAAW;AAAA,MACrB,gBAAgB,WAAW;AAAA,IAC7B;AAAA,EACF,CAAC,EACA,KAAK,CAAC,GAAG,MAAM;AACd,UAAM,eAAe,eAAe,EAAE,QAAQ,IAAI,eAAe,EAAE,QAAQ;AAC3E,QAAI,iBAAiB,EAAG,QAAO;AAC/B,WAAO,EAAE,OAAO,EAAE;AAAA,EACpB,CAAC;AAEH,QAAM,iBAAiB,oBAAoB,YAAY;AAAA,IACrD;AAAA,IACA,iBAAiB,KAAK;AAAA,IACtB;AAAA,IACA;AAAA,IACA;AAAA,EACF,CAAC;AACD,QAAM,mBAAmB,sBAAsB,YAAY,gBAAgB;AAAA,IACzE;AAAA,IACA;AAAA,IACA,iBAAiB,KAAK;AAAA,IACtB;AAAA,EACF,CAAC;AACD,QAAM,cAAc,iBAAiB;AAAA,IACnC;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,EACF,CAAC;AAED,QAAM,iBAAiB,MAAM;AAAA,IAC3B,aAAa;AAAA,MACX,SAAS,KACN,OAAO,CAAC,MAAM,EAAE,aAAa,KAAK,EAClC,IAAI,CAAC,OAAO,EAAE,OAAO,EAAE,OAAO,aAAa,EAAE,aAAa,UAAU,EAAE,SAAS,EAAE,EACjF,KAAK,CAAC,GAAG,MAAM,EAAE,MAAM,cAAc,EAAE,KAAK,CAAC;AAAA,MAChD;AAAA,MACA;AAAA,IACF,CAAC;AAAA,EACH;AAEA,QAAM,WAAW,uBAAuB;AAAA,IACtC;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA,iBAAiB,KAAK;AAAA,EACxB,CAAC;AACD,QAAM,OAAO,mBAAmB,UAAU,KAAK;AAE/C,SAAO;AAAA,IACL,MAAM;AAAA,IACN;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA,QAAQ,EAAE,QAAQ,MAAM;AAAA,IACxB;AAAA,IACA,iBAAiB,KAAK;AAAA,IACtB;AAAA,IACA;AAAA,EACF;AACF;AAEA,SAAS,iBAAiB,KASI;AAC5B,QAAM,cAAwB;AAAA,IAC5B;AAAA,IACA;AAAA,IACA,sCAAsC,IAAI,GAAG,cAAc,IAAI,QAAQ,gBAAgB,IAAI,UAAU;AAAA,EACvG;AACA,MAAI,IAAI,MAAM;AACZ,gBAAY;AAAA,MACV,uCAAuC,WAAW,IAAI,IAAI,CAAC;AAAA,IAC7D;AAAA,EACF;AACA,MAAI,IAAI,eAAe,MAAM;AAC3B,gBAAY,KAAK,oEAAoE;AAAA,EACvF;AACA,QAAM,UAAoB;AAAA,IACxB;AAAA,IACA;AAAA,IACA,8DAAyD,IAAI,QAAQ,uBAAuB,IAAI,QAAQ;AAAA,IACxG;AAAA,IACA;AAAA,EACF;AACA,QAAM,eAAyB;AAAA,IAC7B;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,EACF;AACA,QAAM,iBAA2B;AAAA,IAC/B,kBAAkB,+BAA+B;AAAA,IACjD;AAAA,IACA;AAAA,IACA;AAAA,EACF;AACA,QAAM,YAAsB;AAAA,IAC1B;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,EACF;AACA,SAAO,EAAE,aAAa,SAAS,cAAc,gBAAgB,UAAU;AACzE;AAEA,SAAS,WAAW,MAA6C;AAC/D,SAAO,IAAI,IAAI,KAAK,GAAG,CAAC,KAAK,IAAI,KAAK,IAAI,CAAC;AAC7C;AAEA,SAAS,kBACP,KACA,KAQsD;AACtD,MAAI,IAAI,cAAc,IAAI,gBAAgB,IAAI,YAAY;AACxD,WAAO,EAAE,UAAU,QAAQ,QAAQ,uBAAuB;AAAA,EAC5D;AACA,MAAI,CAAC,IAAI,YAAY;AACnB,WAAO;AAAA,MACL,UAAU,IAAI,OAAO,aAAa,SAAS;AAAA,MAC3C,QACE;AAAA,IACJ;AAAA,EACF;AAGA,MAAI,IAAI,OAAO,QAAQ,IAAI,MAAM,SAAS,WAAW;AACnD,WAAO,EAAE,UAAU,UAAU,QAAQ,0BAA0B,IAAI,MAAM,IAAI,IAAI;AAAA,EACnF;AACA,MAAI,CAAC,IAAI,aAAa,IAAI,UAAU,IAAI,iCAAiC;AACvE,WAAO;AAAA,MACL,UAAU;AAAA,MACV,QAAQ,QAAQ,IAAI,WAAW,KAAK,CAAC,6CAA6C,+BAA+B;AAAA,IACnH;AAAA,EACF;AACA,QAAM,KAAK,IAAI,UAAU;AACzB,MAAI,IAAI,QAAQ,GAAG,OAAO,IAAI,KAAK,OAAO,GAAG,QAAQ,IAAI,KAAK,MAAM;AAClE,WAAO;AAAA,MACL,UAAU;AAAA,MACV,QAAQ,oBAAoB,IAAI,GAAG,GAAG,CAAC,KAAK,IAAI,GAAG,IAAI,CAAC,0BAA0B,WAAW,IAAI,IAAI,CAAC;AAAA,IACxG;AAAA,EACF;AACA,QAAM,cAAc,OAAO,SAAS,IAAI,MAAM,KAAK,IAAI,UAAU,IAAI;AACrE,QAAM,eAAe,GAAG,MAAM;AAC9B,QAAM,eAAe,GAAG,OAAO;AAC/B,MAAI,cAAc;AAChB,WAAO;AAAA,MACL,UAAU;AAAA,MACV,QAAQ,oBAAoB,IAAI,GAAG,GAAG,CAAC,KAAK,IAAI,GAAG,IAAI,CAAC;AAAA,IAC1D;AAAA,EACF;AACA,MAAI,IAAI,UAAU,IAAI,IAAI,UAAU;AAClC,WAAO;AAAA,MACL,UAAU;AAAA,MACV,QAAQ,QAAQ,IAAI,UAAU,CAAC,gEAAgE,IAAI,IAAI,UAAU,GAAG,CAAC,6BAAwB,IAAI,QAAQ;AAAA,IAC3J;AAAA,EACF;AACA,MAAI,eAAe,cAAc;AAC/B,WAAO;AAAA,MACL,UAAU;AAAA,MACV,QAAQ,iBAAiB,IAAI,IAAI,MAAM,CAAC,WAAM,IAAI,GAAG,yBAAyB,IAAI,GAAG,GAAG,CAAC,KAAK,IAAI,GAAG,IAAI,CAAC,iCAA4B,IAAI,IAAI,UAAU,iBAAiB,CAAC;AAAA,IAC5K;AAAA,EACF;AACA,SAAO;AAAA,IACL,UAAU;AAAA,IACV,QAAQ,gBAAW,IAAI,IAAI,UAAU,iBAAiB,CAAC,YAAY,IAAI,GAAG,GAAG,CAAC,KAAK,IAAI,GAAG,IAAI,CAAC,8CAA8C,IAAI,GAAG;AAAA,EACtJ;AACF;AAEA,SAAS,oBACP,YACA,KAO8B;AAC9B,QAAM,gBAAgB,WAAW,OAAO,CAAC,MAAM,EAAE,gBAAgB,IAAI,UAAU;AAC/E,QAAM,cAAc,cAAc,KAAK,CAAC,MAAM,EAAE,aAAa,SAAS;AACtE,QAAM,iBAAiB,cAAc,KAAK,CAAC,MAAM,EAAE,aAAa,YAAY;AAC5E,QAAM,SAAS,eAAe,kBAAkB,cAAc,CAAC,KAAK;AACpE,QAAM,WAAmC,cACrC,YACA,cAAc,KAAK,CAAC,MAAM,EAAE,aAAa,iBAAiB,IACxD,oBACA,iBACE,eACA,cAAc,KAAK,CAAC,MAAM,EAAE,aAAa,MAAM,IAC7C,SACA;AAEV,QAAM,YAAsB,CAAC;AAC7B,QAAM,QAAkB,CAAC;AACzB,QAAM,cAAwB,CAAC;AAE/B,MAAI,QAAQ;AACV,cAAU,KAAK,GAAG,OAAO,WAAW,KAAK,OAAO,cAAc,EAAE;AAChE,QAAI,OAAO,QAAQ;AACjB,YAAM,cACJ,OAAO,sBAAsB,OAAO,kBAAa,IAAI,OAAO,iBAAiB,CAAC,KAAK;AACrF,gBAAU;AAAA,QACR,2BAA2B,IAAI,OAAO,OAAO,GAAG,CAAC,KAAK,IAAI,OAAO,OAAO,IAAI,CAAC,IAAI,WAAW;AAAA,MAC9F;AAAA,IACF;AACA,QAAI,OAAO,QAAQ,QAAQ,OAAO,SAAS,OAAO,GAAG,GAAG;AACtD,gBAAU,KAAK,2BAA2B,OAAO,OAAO,KAAK,IAAI,OAAO,GAAG,CAAC,eAAe;AAAA,IAC7F;AAAA,EACF;AACA,MAAI,CAAC,IAAI,YAAY;AACnB,UAAM,KAAK,mEAAmE;AAC9E,gBAAY,KAAK,iEAAiE;AAAA,EACpF;AACA,MAAI,CAAC,IAAI,qBAAqB;AAC5B,UAAM;AAAA,MACJ;AAAA,IACF;AACA,gBAAY;AAAA,MACV;AAAA,IACF;AAAA,EACF;AACA,MAAI,IAAI,SAAS,QAAQ,cAAc,SAAS,GAAG;AACjD,UAAM;AAAA,MACJ;AAAA,IACF;AACA,gBAAY;AAAA,MACV;AAAA,IACF;AAAA,EACF;AACA,QAAM,eAAe,cAAc,OAAO,CAAC,MAAM,EAAE,aAAa,iBAAiB;AACjF,MAAI,aAAa,SAAS,GAAG;AAC3B,UAAM,QAAQ,aAAa,OAAO,CAAC,GAAG,MAAO,EAAE,UAAU,EAAE,UAAU,IAAI,CAAE;AAC3E,UAAM;AAAA,MACJ,GAAG,aAAa,MAAM,mCAAmC,IAAI,QAAQ,wBAAwB,MAAM,WAAW,SAAS,MAAM,OAAO;AAAA,IACtI;AACA,gBAAY;AAAA,MACV,oBAAoB,IAAI,WAAW,MAAM,OAAO,kCAAkC,MAAM,WAAW;AAAA,IACrG;AAAA,EACF;AACA,QAAM,WAAW,cAAc,OAAO,CAAC,MAAM,EAAE,aAAa,QAAQ;AACpE,MAAI,SAAS,SAAS,GAAG;AACvB,UAAM;AAAA,MACJ,GAAG,SAAS,MAAM;AAAA,IACpB;AAAA,EACF;AACA,MAAI,IAAI,mBAAmB,IAAI,gBAAgB,SAAS,SAAS,GAAG;AAClE,UAAM,MAAM,IAAI,gBAAgB,SAAS,CAAC;AAC1C,UAAM,KAAK,wBAAwB,IAAI,YAAY,WAAW,IAAI,QAAQ,UAAU;AACpF,gBAAY,KAAK,8DAA8D;AAAA,EACjF;AACA,MAAI,aAAa,WAAW;AAC1B,gBAAY,KAAK,+DAA+D;AAAA,EAClF,WAAW,aAAa,QAAQ;AAC9B,gBAAY,KAAK,qEAAqE;AAAA,EACxF,WAAW,aAAa,cAAc;AACpC,gBAAY;AAAA,MACV;AAAA,IACF;AAAA,EACF,WAAW,aAAa,UAAU;AAChC,gBAAY;AAAA,MACV;AAAA,IACF;AAAA,EACF;AAEA,SAAO;AAAA,IACL;AAAA,IACA,aAAa,QAAQ,eAAe;AAAA,IACpC;AAAA,IACA;AAAA,IACA;AAAA,EACF;AACF;AAEA,SAAS,sBACP,YACA,gBACA,KAMU;AACV,QAAM,QAAkB,CAAC;AACzB,QAAM,gBAAgB,WAAW,OAAO,CAAC,MAAM,EAAE,gBAAgB,IAAI,UAAU;AAC/E,QAAM;AAAA,IACJ,aAAa,cAAc,MAAM,wBAAwB,IAAI,KAAK,SAAS,IAAI,aAAa,YAAY,IAAI,UAAU,KAAK,EAAE;AAAA,EAC/H;AACA,QAAM;AAAA,IACJ,mBAAmB,eAAe,QAAQ,GAAG,eAAe,cAAc,IAAI,eAAe,WAAW,KAAK,EAAE;AAAA,EACjH;AACA,QAAM,WAAW,cAAc,OAAO,CAAC,MAAM,EAAE,aAAa,SAAS,EAAE;AACvE,QAAM,OAAO,cAAc,OAAO,CAAC,MAAM,EAAE,aAAa,MAAM,EAAE;AAChE,QAAM,aAAa,cAAc,OAAO,CAAC,MAAM,EAAE,aAAa,YAAY,EAAE;AAC5E,QAAM,WAAW,cAAc,OAAO,CAAC,MAAM,EAAE,aAAa,QAAQ,EAAE;AACtE,QAAM,OAAO,cAAc,OAAO,CAAC,MAAM,EAAE,aAAa,iBAAiB,EAAE;AAC3E,QAAM;AAAA,IACJ,iBAAiB,QAAQ,aAAa,UAAU,gBAAgB,IAAI,UAAU,QAAQ,YAAY,IAAI;AAAA,EACxG;AACA,QAAM,WAAW,cAAc,OAAO,CAAC,MAAM,EAAE,gBAAgB,EAAE,IAAI,CAAC,MAAM,EAAE,WAAW;AACzF,MAAI,SAAS,SAAS,EAAG,OAAM,KAAK,+BAA+B,SAAS,KAAK,IAAI,CAAC,GAAG;AACzF,MAAI,IAAI,iBAAiB;AACvB,UAAM;AAAA,MACJ,4BAA4B,IAAI,gBAAgB,aAAa,IAAI,IAAI,gBAAgB,SAAS,uBAAuB,IAAI,gBAAgB,SAAS,MAAM;AAAA,IAC1J;AAAA,EACF;AACA,QAAM;AAAA,IACJ,IAAI,sBACA,2BAA2B,IAAI,oBAAoB,MAAM,GAAG,EAAE,CAAC,WAC/D;AAAA,EACN;AACA,SAAO;AACT;AAEA,SAAS,uBAAuB,QAgBrB;AACT,QAAM,QAAkB,CAAC;AACzB,QAAM,KAAK,KAAK,OAAO,KAAK,EAAE;AAC9B,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,kBAAkB,OAAO,WAAW,EAAE;AACjD,QAAM,KAAK,sBAAsB,OAAO,KAAK,EAAE;AAC/C,QAAM,KAAK,mBAAmB,OAAO,cAAc,gBAAgB,EAAE;AACrE,QAAM,KAAK,aAAa,OAAO,OAAO,WAAW,OAAO,IAAI,IAAI,gBAAgB,EAAE;AAClF,QAAM,KAAK,0BAA0B,OAAO,cAAc,IAAI;AAC9D,QAAM;AAAA,IACJ,wBAAwB,OAAO,sBAAsB,KAAK,OAAO,mBAAmB,OAAO,MAAM;AAAA,EACnG;AACA,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,sBAAsB;AACjC,QAAM,KAAK,EAAE;AACb,aAAW,QAAQ,OAAO,iBAAkB,OAAM,KAAK,KAAK,IAAI,EAAE;AAClE,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,mBAAmB;AAC9B,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,iBAAiB,OAAO,eAAe,QAAQ,EAAE;AAC5D,QAAM,KAAK,kBAAkB,OAAO,eAAe,eAAe,KAAK,EAAE;AACzE,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,eAAe;AAC1B,QAAM,KAAK,EAAE;AACb,aAAW,QAAQ,OAAO,eAAe,UAAW,OAAM,KAAK,KAAK,IAAI,EAAE;AAC1E,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,WAAW;AACtB,QAAM,KAAK,EAAE;AACb,aAAW,QAAQ,OAAO,eAAe,MAAM,SAC3C,OAAO,eAAe,QACtB,CAAC,0CAA0C,GAAG;AAChD,UAAM,KAAK,KAAK,IAAI,EAAE;AAAA,EACxB;AACA,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,kBAAkB;AAC7B,QAAM,KAAK,EAAE;AACb,aAAW,QAAQ,OAAO,eAAe,YAAa,OAAM,KAAK,KAAK,IAAI,EAAE;AAC5E,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,6BAA6B;AACxC,QAAM,KAAK,EAAE;AACb,QAAM;AAAA,IACJ;AAAA,EACF;AACA,QAAM,KAAK,0DAA0D;AACrE,aAAW,KAAK,OAAO,YAAY;AACjC,UAAM,QAAQ,EAAE,0BAA0B,OAAO,MAAM,OAAO,EAAE,qBAAqB;AACrF,UAAM,OAAO,EAAE,sBAAsB,OAAO,MAAM,EAAE,kBAAkB,QAAQ,CAAC;AAC/E,UAAM,IAAI,OAAO,SAAS,EAAE,MAAM,IAAI,EAAE,OAAO,QAAQ,CAAC,IAAI;AAC5D,UAAM,IAAI,OAAO,SAAS,EAAE,OAAO,IAAI,EAAE,QAAQ,QAAQ,CAAC,IAAI;AAC9D,UAAM,OAAO,EAAE,SAAS,IAAI,IAAI,EAAE,OAAO,GAAG,CAAC,KAAK,IAAI,EAAE,OAAO,IAAI,CAAC,MAAM;AAC1E,UAAM,MAAM,EAAE,QAAQ,QAAQ,CAAC,OAAO,SAAS,EAAE,GAAG,IAAI,MAAM,IAAI,EAAE,GAAG;AACvE,UAAM;AAAA,MACJ,KAAK,EAAE,WAAW,MAAM,EAAE,QAAQ,MAAM,IAAI,EAAE,IAAI,CAAC,MAAM,KAAK,MAAM,IAAI,MAAM,CAAC,MAAM,CAAC,MAAM,EAAE,OAAO,MAAM,IAAI,MAAM,GAAG,MAAM,EAAE,mBAAmB,QAAQ,IAAI,MAAM,EAAE,QAAQ,GAAG;AAAA,IACpL;AAAA,EACF;AACA,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,wBAAwB;AACnC,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,OAAO,QAAQ,QAAQ;AAClC,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,gBAAgB;AAC3B,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,iBAAiB;AAC5B,QAAM,KAAK,EAAE;AACb,aAAW,QAAQ,OAAO,YAAY,YAAa,OAAM,KAAK,KAAK,IAAI,EAAE;AACzE,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,aAAa;AACxB,QAAM,KAAK,EAAE;AACb,aAAW,QAAQ,OAAO,YAAY,QAAS,OAAM,KAAK,KAAK,IAAI,EAAE;AACrE,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,6BAA6B;AACxC,QAAM,KAAK,EAAE;AACb,aAAW,QAAQ,OAAO,YAAY,aAAc,OAAM,KAAK,KAAK,IAAI,EAAE;AAC1E,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,uBAAuB;AAClC,QAAM,KAAK,EAAE;AACb,aAAW,QAAQ,OAAO,YAAY,eAAgB,OAAM,KAAK,KAAK,IAAI,EAAE;AAC5E,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,eAAe;AAC1B,QAAM,KAAK,EAAE;AACb,aAAW,QAAQ,OAAO,YAAY,UAAW,OAAM,KAAK,KAAK,IAAI,EAAE;AACvE,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,gBAAgB;AAC3B,QAAM,KAAK,EAAE;AACb,QAAM;AAAA,IACJ;AAAA,EACF;AACA,QAAM,KAAK,EAAE;AACb,QAAM,KAAK,SAAS;AACpB,QAAM,KAAK,KAAK,UAAU,EAAE,QAAQ,OAAO,QAAQ,OAAO,OAAO,MAAM,GAAG,MAAM,CAAC,CAAC;AAClF,QAAM,KAAK,KAAK;AAChB,MAAI,OAAO,iBAAiB;AAC1B,UAAM,KAAK,EAAE;AACb,UAAM,KAAK,qBAAqB;AAChC,UAAM,KAAK,EAAE;AACb,UAAM,KAAK,uDAAuD;AAClE,UAAM,KAAK,yBAAyB;AACpC,eAAW,KAAK,OAAO,gBAAgB,SAAS,MAAM,GAAG,EAAE,GAAG;AAC5D,YAAM;AAAA,QACJ,KAAK,EAAE,YAAY,MAAM,EAAE,QAAQ,MAAM,EAAE,YAAY,MAAM,MAAM,EAAE,YAAY,GAAG,MAAM,YAAY,EAAE,gBAAgB,EAAE,YAAY,CAAC;AAAA,MACzI;AAAA,IACF;AAAA,EACF;AACA,SAAO,MAAM,KAAK,IAAI;AACxB;AAEA,SAAS,mBAAmB,UAAkB,OAAuB;AACnE,QAAM,OAAO,eAAe,QAAQ;AACpC,SAAO;AAAA,IACL;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA,UAAU,WAAW,KAAK,CAAC;AAAA,IAC3B;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,IACA;AAAA,EACF,EAAE,KAAK,IAAI;AACb;AAEA,SAAS,eAAe,UAA0B;AAChD,QAAM,QAAQ,SAAS,MAAM,IAAI;AACjC,QAAM,OAAiB,CAAC;AACxB,MAAI,SAAS;AACb,MAAI,SAAS;AACb,MAAI,OAAiB,CAAC;AACtB,MAAI,QAAkB,CAAC;AAEvB,QAAM,YAAY,MAAM;AACtB,QAAI,QAAQ;AACV,WAAK,KAAK,OAAO;AACjB,eAAS;AAAA,IACX;AAAA,EACF;AACA,QAAM,aAAa,MAAM;AACvB,QAAI,MAAM,WAAW,EAAG;AACxB,SAAK,KAAK,oBAAoB,KAAK,CAAC;AACpC,YAAQ,CAAC;AAAA,EACX;AAEA,aAAW,QAAQ,OAAO;AACxB,QAAI,KAAK,WAAW,KAAK,GAAG;AAC1B,UAAI,QAAQ;AACV,aAAK,KAAK,cAAc,WAAW,KAAK,KAAK,IAAI,CAAC,CAAC,eAAe;AAClE,eAAO,CAAC;AACR,iBAAS;AAAA,MACX,OAAO;AACL,kBAAU;AACV,mBAAW;AACX,iBAAS;AAAA,MACX;AACA;AAAA,IACF;AACA,QAAI,QAAQ;AACV,WAAK,KAAK,IAAI;AACd;AAAA,IACF;AACA,QAAI,KAAK,WAAW,GAAG,GAAG;AACxB,gBAAU;AACV,YAAM,KAAK,IAAI;AACf;AAAA,IACF;AACA,eAAW;AACX,QAAI,KAAK,WAAW,IAAI,GAAG;AACzB,UAAI,CAAC,QAAQ;AACX,aAAK,KAAK,MAAM;AAChB,iBAAS;AAAA,MACX;AACA,WAAK,KAAK,OAAO,eAAe,KAAK,MAAM,CAAC,CAAC,CAAC,OAAO;AACrD;AAAA,IACF;AACA,cAAU;AACV,QAAI,KAAK,WAAW,IAAI,EAAG,MAAK,KAAK,OAAO,eAAe,KAAK,MAAM,CAAC,CAAC,CAAC,OAAO;AAAA,aACvE,KAAK,WAAW,KAAK,EAAG,MAAK,KAAK,OAAO,eAAe,KAAK,MAAM,CAAC,CAAC,CAAC,OAAO;AAAA,aAC7E,KAAK,WAAW,MAAM,EAAG,MAAK,KAAK,OAAO,eAAe,KAAK,MAAM,CAAC,CAAC,CAAC,OAAO;AAAA,aAC9E,KAAK,KAAK,MAAM,GAAI,MAAK,KAAK,EAAE;AAAA,QACpC,MAAK,KAAK,MAAM,eAAe,IAAI,CAAC,MAAM;AAAA,EACjD;AACA,YAAU;AACV,aAAW;AACX,SAAO,KAAK,KAAK,IAAI;AACvB;AAEA,SAAS,oBAAoB,OAAyB;AACpD,QAAM,OAAO,MACV,OAAO,CAAC,SAAS,CAAC,iBAAiB,KAAK,IAAI,CAAC,EAC7C;AAAA,IAAI,CAAC,SACJ,KACG,MAAM,GAAG,EAAE,EACX,MAAM,GAAG,EACT,IAAI,CAAC,SAAS,eAAe,KAAK,KAAK,CAAC,CAAC;AAAA,EAC9C;AACF,MAAI,KAAK,WAAW,EAAG,QAAO;AAC9B,QAAM,CAAC,MAAM,GAAG,IAAI,IAAI;AACxB,QAAM,KAAK,KAAM,IAAI,CAAC,SAAS,OAAO,IAAI,OAAO,EAAE,KAAK,EAAE;AAC1D,QAAM,MAAM,KACT,IAAI,CAAC,QAAQ,OAAO,IAAI,IAAI,CAAC,SAAS,OAAO,IAAI,OAAO,EAAE,KAAK,EAAE,CAAC,OAAO,EACzE,KAAK,IAAI;AACZ,SAAO,qBAAqB,EAAE,uBAAuB,GAAG;AAC1D;AAEA,SAAS,eAAe,GAAmB;AACzC,SAAO,WAAW,CAAC,EAAE,QAAQ,oBAAoB,qBAAqB;AACxE;AAEA,SAAS,WAAW,GAAmB;AACrC,SAAO,EACJ,QAAQ,MAAM,OAAO,EACrB,QAAQ,MAAM,MAAM,EACpB,QAAQ,MAAM,MAAM,EACpB,QAAQ,MAAM,QAAQ;AAC3B;AAEA,SAAS,YAAY,GAAmB;AACtC,SAAO,EAAE,QAAQ,OAAO,KAAK;AAC/B;AAEA,SAAS,eAAe,UAA0C;AAChE,MAAI,aAAa,UAAW,QAAO;AACnC,MAAI,aAAa,aAAc,QAAO;AACtC,MAAI,aAAa,OAAQ,QAAO;AAChC,MAAI,aAAa,kBAAmB,QAAO;AAC3C,SAAO;AACT;AAEA,SAAS,OAAO,GAAmB;AACjC,SAAO,GAAG,KAAK,IAAI,MAAM,EAAE,GAAG,IAAI,CAAC,CAAC;AACtC;AAIA,SAAS,IAAI,IAAsB;AACjC,MAAI,GAAG,WAAW,EAAG,QAAO,OAAO;AACnC,SAAO,GAAG,OAAO,CAAC,GAAG,MAAM,IAAI,GAAG,CAAC,IAAI,GAAG;AAC5C;AAEA,SAAS,eAAe,QAA0B;AAChD,MAAI,OAAO,WAAW,EAAG,QAAO;AAChC,QAAM,MAAM,KAAK,MAAM,OAAO,SAAS,CAAC;AACxC,SAAO,OAAO,SAAS,MAAM,KAAK,OAAO,MAAM,CAAC,IAAK,OAAO,GAAG,KAAM,IAAI,OAAO,GAAG;AACrF;AAEA,SAAS,IAAI,GAAmB;AAC9B,MAAI,CAAC,OAAO,SAAS,CAAC,EAAG,QAAO,OAAO,CAAC;AACxC,SAAO,EAAE,QAAQ,CAAC;AACpB;","names":[]}

package/dist/{chunk-RAF443UI.js → chunk-DBIGN5MJ.js}

@@ -1,6 +1,6 @@
 import {
   summaryTable
-} from "./chunk-2A5XJB43.js";
+} from "./chunk-5AKPEK5L.js";
 import {
   VerificationError
 } from "./chunk-NG236HPC.js";
@@ -590,4 +590,4 @@ export {
   judgeReplayGate,
   renderReleaseReport
 };
-//# sourceMappingURL=chunk-RAF443UI.js.map
+//# sourceMappingURL=chunk-DBIGN5MJ.js.map

package/dist/{chunk-JLZQWFV3.js → chunk-K33INZHH.js}

@@ -1,6 +1,6 @@
 import {
   cohensD
-} from "./chunk-I4MBDTY5.js";
+} from "./chunk-R5UQJNKC.js";
 import {
   argHash,
   groupBy,
@@ -615,4 +615,4 @@ export {
   iqr,
   welchsTTest
 };
-//# sourceMappingURL=chunk-JLZQWFV3.js.map
+//# sourceMappingURL=chunk-K33INZHH.js.map