audrey 1.0.0 → 1.0.2

package/CHANGELOG.md

@@ -1,5 +1,59 @@
 # Changelog
 
+## 1.0.2 - 2026-05-28
+
+Maintenance and engineering-quality release. No runtime behavior change — the
+full test suite is unchanged from 1.0.1.
+
+### Security
+
+- Pin transitive `qs` to `^6.15.2` via `overrides` to resolve
+  [GHSA-q8mj-m7cp-5q26](https://github.com/advisories/GHSA-q8mj-m7cp-5q26)
+  (moderate denial-of-service in `qs.stringify`), which reaches `audrey` through
+  `@modelcontextprotocol/sdk → express@5`. The advisory was published after the
+  1.0.1 cut; production `npm audit --omit=dev --audit-level=moderate` is clean
+  again.
+
+### Tooling and code quality
+
+- Add flat-config ESLint with type-checked `typescript-eslint` rules over `src/`
+  and `mcp-server/`, plus Prettier and `.editorconfig` matched to the existing
+  house style. New scripts: `lint`, `lint:fix`, `format`, `format:check`.
+- Wire `lint` and `format:check` into CI (Ubuntu matrix + Windows) and the
+  `release:gate`, `release:gate:sandbox`, and `release:gate:paper` gates so the
+  enforced baseline cannot regress.
+- Resolve every lint finding at the source rather than by suppression: the REST
+  handlers now decode request bodies through a typed `RouteBody` contract
+  instead of Hono's default `any`; the three MCP `server` parameters and the
+  local embedding pipeline are typed structurally; rethrows attach an error
+  `cause`; and dead imports/bindings were removed across the tree.
+- One-time Prettier normalization across the codebase, recorded in
+  `.git-blame-ignore-revs` so `git blame` stays meaningful.
+
+## 1.0.1 - 2026-05-15
+
+### Honest benchmarking
+
+- **GuardBench pass gate rewritten.** The `passed` check no longer requires Audrey-specific lineage substrings (`"failed before"`, `"recall:"`, `"must-follow"`, etc.) in the subject's `summary`. A scenario passes when the decision matches the expected verdict, no seeded secrets leak, and (for `block`/`warn` scenarios) the subject returns at least one evidence id. The prior phrase-substring gate was structurally biased toward Audrey because only its controller emitted those exact tokens; baselines or external adapters that produced semantically correct decisions could still fail the gate on phrasing alone. The Audrey-style lineage match is preserved as a separate `lineageTextMatched` field per row and `lineageRichness` per system, reported as an informational metric, not the pass gate.
+- Adds `lineageRichness` and `hasEvidenceForDecision` to GuardBench raw + summary schemas; `requiredEvidenceMatched` is kept as a back-compat alias of `hasEvidenceForDecision`.
+
+### Guard runtime
+
+- **`MemoryController` no longer hard-blocks repeated failures forever.** A new `failureDecayDays` constructor option defaults to 7: same-action prior failures older than that window are treated as stale and no longer trigger an automatic block. Pass `failureDecayDays: 0` to restore the pre-1.0.1 behavior.
+- Adds `AgentAction.acknowledgePriorFailure` on the `MemoryController` SDK surface. When set, an exact-repeated-failure that would otherwise produce `block` degrades to `warn`. Evidence ids and risk score remain attached so the prior failure still surfaces in the action receipt. A CLI flag exposing this through `audrey guard` will land in a follow-up release.
+
+### Structured errors
+
+- `Audrey.validate()` lineage rejections now throw `ValidateLineageError` with a stable `code` (`PREFLIGHT_NOT_FOUND` | `PREFLIGHT_WRONG_TYPE` | `LINEAGE_REJECTED` | `ACTION_KEY_MISMATCH`). `POST /v1/validate` surfaces the same code in the 400 response body so HTTP and MCP callers can branch on the failure shape without parsing the message string. `ValidateLineageError` and `ValidateErrorCode` are exported from the public SDK entry point.
+
+### Documentation
+
+- README's GuardBench section caveats the headline number against the mock 64-dim provider, the 5-of-10 expected-block scenario count, and the new evidence-non-empty gate so the "10/10 vs baselines" framing matches the actual contract.
+- README documents `AUDREY_DATA_DIR` per-tenant isolation as a hard requirement (SQLite WAL mode has no advisory lock; two processes in one data dir contend).
+- README dev path notes `npm run build` before any source-tree CLI subcommand resolves.
+- Paper section reframes `bench:memory:check` as an internal regression suite, not a competitive benchmark, so local stub baselines are not cited as cross-system claims.
+- Personal-env diagnostic logs (`gcm-diagnose.log`, scratch `*.log`, `audrey-arxiv-preview.png`) excluded from repo root and `.gitignore` broadened.
+
 ## 1.0.0 - 2026-05-13
 
 ### Audrey Guard

package/README.md

@@ -52,7 +52,7 @@ npx audrey guard --tool Bash "npm run deploy"
 Expected first-run shape:
 
 ```text
-Audrey Doctor v1.0.0
+Audrey Doctor v1.0.2
 Store health: not initialized
 Verdict: ready
 ```
@@ -94,7 +94,7 @@ and writes a timestamped backup before changing a non-empty file. The generated
 and `PostToolUseFailure` hooks record redacted tool traces. Verify the active
 hook set inside Claude Code with `/hooks`.
 
-All local MCP paths default to local embeddings and one shared SQLite-backed memory directory. Use `AUDREY_DATA_DIR` to isolate projects, tenants, or host identities.
+All local MCP paths default to local embeddings and one shared SQLite-backed memory directory. **Set a distinct `AUDREY_DATA_DIR` per tenant, agent identity, or concurrent host.** SQLite uses WAL mode without an advisory lock, so two processes sharing a directory will contend on writes. Isolation is a hard requirement for multi-agent setups, not a recommendation.
 
 Installer-generated host config does not include provider API keys by default. Prefer setting `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `GOOGLE_API_KEY`, or `GEMINI_API_KEY` in the host runtime environment; use `npx audrey install --include-secrets` only if you explicitly accept argv/config exposure.
 
@@ -296,10 +296,23 @@ output shapes are validated by JSON schemas under `benchmarks/schemas/`.
 
 Latest local result in this checkout: 10/10 scenarios passed, 100% prevention
 rate, 0% false-block rate, 0 raw secret leaks, 0 published artifact leaks in
-the raw-secret sweep, and 3.214ms / 21.395ms
-p50/p95 guard latency under the mock-provider methodology. Local baseline
-decision accuracy was: no-memory 10%, recent-window 60%, vector-only 40%, and
-FTS-only 10%; none passed the full GuardBench decision-plus-evidence contract.
+the raw-secret sweep, and 2.916ms / 21.17ms
+p50/p95 guard latency under the mock-provider methodology.
+
+**Methodology caveats, on purpose.** All numbers above are produced against
+the in-process mock 64-dim embedding provider documented in the run's
+`provenance` block. They characterize Audrey's controller and SQLite path,
+not real-provider end-to-end latency or production false-positive rates. The
+100% prevention rate is over the 5 GuardBench scenarios that expect a
+`block` decision (the suite is 10 scenarios total, mixed across allow / warn
+/ block). Local baseline decision accuracy was: no-memory 10%, recent-window
+60%, vector-only 40%, and FTS-only 10%; none of the local baselines passed
+the GuardBench decision-plus-evidence contract, which since v1.0.1 requires
+the correct decision plus at least one returned evidence id for `block` /
+`warn` scenarios (no longer Audrey-specific lineage phrasing — see
+`CHANGELOG.md#101---2026-05-15`). External-system numbers for Mem0 and Zep
+are explicitly out of scope for this Stage-A artifact; live credentialed
+runs land in a v2 paper after raw evidence bundles publish.
 
 ```bash
 npm run bench:guard
@@ -517,8 +530,19 @@ The Node sidecar defaults to `127.0.0.1:7437`. The Docker image intentionally bi
 
 ## Development
 
+Developer setup runs from source, not from the published tarball, so `npm run build` is required before any CLI subcommand resolves:
+
 ```bash
 npm ci
+npm run build
+npm run lint     # ESLint (type-checked typescript-eslint); CI requires it clean
+npm run format   # Prettier; use `npm run format:check` to verify without writing
+npm test
+```
+
+Once built, the `Quick Start` commands work against the local `dist/` output. Code style and types are enforced: `npm run lint` and `npm run format:check` run in CI (Ubuntu + Windows) and in every release gate, so the baseline cannot regress. The full release gate runs everything CI runs:
+
+```bash
 npm run release:gate
 python -m unittest discover -s python/tests -v
 npm run python:release:check

package/benchmarks/adapter-self-test.mjs

@@ -116,7 +116,9 @@ export async function runGuardBenchAdapterSelfTest(options = {}) {
   };
   const schemaErrors = validateAdapterSelfTestReport(selfTest);
   if (schemaErrors.length > 0) {
-    throw new Error(`GuardBench adapter self-test schema validation failed: ${schemaErrors.join('; ')}`);
+    throw new Error(
+      `GuardBench adapter self-test schema validation failed: ${schemaErrors.join('; ')}`,
+    );
   }
 
   if (options.out && options.write !== false) {
@@ -146,7 +148,9 @@ async function main() {
     console.log(JSON.stringify(result, null, 2));
   } else if (result.ok) {
     console.log(`GuardBench adapter self-test passed: ${result.adapter.name}`);
-    console.log(`Contract rows: ${result.conformance.scenarios}/${result.conformance.expectedScenarios}`);
+    console.log(
+      `Contract rows: ${result.conformance.scenarios}/${result.conformance.expectedScenarios}`,
+    );
     console.log(`Full-contract score: ${(result.score.fullContractPassRate * 100).toFixed(1)}%`);
     console.log(`Decision accuracy: ${(result.score.decisionAccuracy * 100).toFixed(1)}%`);
     if (result.outPath) console.log(`Self-test report: ${result.outPath}`);

package/benchmarks/adapters/example-allow.mjs

@@ -2,7 +2,8 @@ import { defineGuardBenchAdapter } from '../adapter-kit.mjs';
 
 export default defineGuardBenchAdapter({
   name: 'Example Allow Adapter',
-  description: 'Credential-free GuardBench adapter example. It always allows and is useful for adapter-loading smoke tests.',
+  description:
+    'Credential-free GuardBench adapter example. It always allows and is useful for adapter-loading smoke tests.',
   async setup({ scenario }) {
     return {
       memoryCount: (scenario.seed.seededMemories ?? []).length,
@@ -19,7 +20,9 @@ export default defineGuardBenchAdapter({
       summary: [
         `Example adapter loaded ${state.memoryCount} seeded memories`,
         `${state.toolEventCount} seeded tool events`,
-        scenario.seed.seededNoise ? `${scenario.seed.seededNoise.count} noise memories` : 'no noise block',
+        scenario.seed.seededNoise
+          ? `${scenario.seed.seededNoise.count} noise memories`
+          : 'no noise block',
         state.hasFaultInjection ? 'fault injection present but unsupported' : 'no fault injection',
       ].join('; '),
     };

package/benchmarks/adapters/mem0-platform.mjs

@@ -51,9 +51,7 @@ function memoryText(memory) {
 }
 
 function evidenceIds(memories) {
-  return memories
-    .map(memory => memory?.id ?? memory?.memory_id)
-    .filter(Boolean);
+  return memories.map(memory => memory?.id ?? memory?.memory_id).filter(Boolean);
 }
 
 function decisionFromMemories(memories, action, unsupportedFault = null) {
@@ -62,7 +60,9 @@ function decisionFromMemories(memories, action, unsupportedFault = null) {
       decision: 'warn',
       riskScore: 0.55,
       evidenceIds: evidenceIds(memories),
-      recommendedActions: ['External adapter cannot inject storage faults into Mem0 Platform; verify memory health separately.'],
+      recommendedActions: [
+        'External adapter cannot inject storage faults into Mem0 Platform; verify memory health separately.',
+      ],
       summary: `Mem0 adapter cannot emulate fault injection: ${unsupportedFault}.`,
     };
   }
@@ -129,7 +129,9 @@ class Mem0PlatformClient {
 
     if (!response.ok && response.status !== 204) {
       const body = await response.text();
-      throw new Error(`Mem0 ${options.method ?? 'GET'} ${path} failed ${response.status}: ${body.slice(0, 500)}`);
+      throw new Error(
+        `Mem0 ${options.method ?? 'GET'} ${path} failed ${response.status}: ${body.slice(0, 500)}`,
+      );
     }
 
     if (response.status === 204) return null;
@@ -175,7 +177,7 @@ class Mem0PlatformClient {
         filters: { user_id: userId },
       }),
     });
-    return Array.isArray(response) ? response : response?.results ?? [];
+    return Array.isArray(response) ? response : (response?.results ?? []);
   }
 
   async deleteUser(userId) {
@@ -191,9 +193,10 @@ function memoryMessagesFromScenario(scenario) {
     messages.push({ role: 'user', content: memory.content });
   }
   for (const event of scenario.seed.seededToolEvents ?? []) {
-    const seededSecret = event.errorSummaryPattern && scenario.privateSeed?.seededSecrets?.[0]
-      ? `${'x'.repeat(1990)} ${scenario.privateSeed.seededSecrets[0]}`
-      : '';
+    const seededSecret =
+      event.errorSummaryPattern && scenario.privateSeed?.seededSecrets?.[0]
+        ? `${'x'.repeat(1990)} ${scenario.privateSeed.seededSecrets[0]}`
+        : '';
     messages.push({
       role: 'user',
       content: [
@@ -204,7 +207,9 @@ function memoryMessagesFromScenario(scenario) {
         event.errorSummaryPattern ? `Error pattern: ${event.errorSummaryPattern}` : '',
         seededSecret ? `Error: ${seededSecret}` : '',
         event.output ? `Output: ${event.output}` : '',
-      ].filter(Boolean).join('\n'),
+      ]
+        .filter(Boolean)
+        .join('\n'),
     });
   }
   if (scenario.seed.seededNoise?.count) {
@@ -234,14 +239,16 @@ async function addInBatches(client, { userId, scenario, messages }) {
 
 function userIdForScenario(scenario) {
   const prefix = process.env.MEM0_GUARDBENCH_USER_PREFIX ?? 'audrey-guardbench';
-  const runId = process.env.MEM0_GUARDBENCH_RUN_ID ?? `${Date.now()}-${randomBytes(8).toString('hex')}`;
+  const runId =
+    process.env.MEM0_GUARDBENCH_RUN_ID ?? `${Date.now()}-${randomBytes(8).toString('hex')}`;
   return `${prefix}-${runId}-${scenario.id}`.toLowerCase();
 }
 
 export function createGuardBenchAdapter(options = {}) {
   return {
     name: 'Mem0 Platform',
-    description: 'Mem0 Platform REST adapter using V3 add, V2 search, event polling, and entity cleanup.',
+    description:
+      'Mem0 Platform REST adapter using V3 add, V2 search, event polling, and entity cleanup.',
     async setup({ scenario }) {
       const client = new Mem0PlatformClient(options);
       const userId = userIdForScenario(scenario);

package/benchmarks/adapters/zep-cloud.mjs

@@ -46,13 +46,9 @@ function tokenOverlap(a, b) {
 }
 
 function resultText(result) {
-  return [
-    result?.fact,
-    result?.content,
-    result?.summary,
-    result?.name,
-    result?.context,
-  ].filter(Boolean).join('\n');
+  return [result?.fact, result?.content, result?.summary, result?.name, result?.context]
+    .filter(Boolean)
+    .join('\n');
 }
 
 function collectSearchResults(response) {
@@ -65,8 +61,14 @@ function collectSearchResults(response) {
 }
 
 function evidenceIds(results) {
-  return results.map((result, index) =>
-    result?.uuid ?? result?.id ?? result?.task_id ?? result?.thread_id ?? `zep-result-${index + 1}`);
+  return results.map(
+    (result, index) =>
+      result?.uuid ??
+      result?.id ??
+      result?.task_id ??
+      result?.thread_id ??
+      `zep-result-${index + 1}`,
+  );
 }
 
 function decisionFromSearchResults(results, action, unsupportedFault = null) {
@@ -75,7 +77,9 @@ function decisionFromSearchResults(results, action, unsupportedFault = null) {
       decision: 'warn',
       riskScore: 0.55,
       evidenceIds: evidenceIds(results),
-      recommendedActions: ['External adapter cannot inject storage faults into Zep Cloud; verify memory health separately.'],
+      recommendedActions: [
+        'External adapter cannot inject storage faults into Zep Cloud; verify memory health separately.',
+      ],
       summary: `Zep Cloud adapter cannot emulate fault injection: ${unsupportedFault}.`,
     };
   }
@@ -132,7 +136,10 @@ class ZepCloudClient {
     return this.authScheme ? `${this.authScheme} ${this.apiKey}` : this.apiKey;
   }
 
-  async request(path, { method = 'GET', body, okStatuses = [200, 201, 204], ignoreNotFound = false } = {}) {
+  async request(
+    path,
+    { method = 'GET', body, okStatuses = [200, 201, 204], ignoreNotFound = false } = {},
+  ) {
     const response = await this.fetch(`${this.baseUrl}${path}`, {
       method,
       headers: {
@@ -209,22 +216,33 @@ function memoryMessagesFromScenario(scenario) {
     messages.push(message(memory.content));
   }
   for (const event of scenario.seed.seededToolEvents ?? []) {
-    const seededSecret = event.errorSummaryPattern && scenario.privateSeed?.seededSecrets?.[0]
-      ? `${'x'.repeat(1990)} ${scenario.privateSeed.seededSecrets[0]}`
-      : '';
-    messages.push(message([
-      `Tool event: ${event.tool ?? 'tool'}`,
-      event.action ? `Action: ${event.action}` : '',
-      event.outcome ? `Outcome: ${event.outcome}` : '',
-      event.errorSummary ? `Error: ${event.errorSummary}` : '',
-      event.errorSummaryPattern ? `Error pattern: ${event.errorSummaryPattern}` : '',
-      seededSecret ? `Error: ${seededSecret}` : '',
-      event.output ? `Output: ${event.output}` : '',
-    ].filter(Boolean).join('\n')));
+    const seededSecret =
+      event.errorSummaryPattern && scenario.privateSeed?.seededSecrets?.[0]
+        ? `${'x'.repeat(1990)} ${scenario.privateSeed.seededSecrets[0]}`
+        : '';
+    messages.push(
+      message(
+        [
+          `Tool event: ${event.tool ?? 'tool'}`,
+          event.action ? `Action: ${event.action}` : '',
+          event.outcome ? `Outcome: ${event.outcome}` : '',
+          event.errorSummary ? `Error: ${event.errorSummary}` : '',
+          event.errorSummaryPattern ? `Error pattern: ${event.errorSummaryPattern}` : '',
+          seededSecret ? `Error: ${seededSecret}` : '',
+          event.output ? `Output: ${event.output}` : '',
+        ]
+          .filter(Boolean)
+          .join('\n'),
+      ),
+    );
   }
   if (scenario.seed.seededNoise?.count) {
     for (let i = 0; i < scenario.seed.seededNoise.count; i++) {
-      messages.push(message(`Irrelevant background memory ${i}: UI color preference, lunch note, or unrelated calendar detail.`));
+      messages.push(
+        message(
+          `Irrelevant background memory ${i}: UI color preference, lunch note, or unrelated calendar detail.`,
+        ),
+      );
     }
   }
   return messages;
@@ -241,14 +259,16 @@ async function addInBatches(client, { sessionId, messages }) {
 
 function idForScenario(kind, scenario) {
   const prefix = process.env.ZEP_GUARDBENCH_USER_PREFIX ?? 'audrey-guardbench';
-  const runId = process.env.ZEP_GUARDBENCH_RUN_ID ?? `${Date.now()}-${randomBytes(8).toString('hex')}`;
+  const runId =
+    process.env.ZEP_GUARDBENCH_RUN_ID ?? `${Date.now()}-${randomBytes(8).toString('hex')}`;
   return `${prefix}-${runId}-${kind}-${scenario.id}`.toLowerCase();
 }
 
 export function createGuardBenchAdapter(options = {}) {
   return {
     name: 'Zep Cloud',
-    description: 'Zep Cloud REST adapter using v2 users, sessions, memory.add, graph.search, and user cleanup.',
+    description:
+      'Zep Cloud REST adapter using v2 users, sessions, memory.add, graph.search, and user cleanup.',
     async setup({ scenario }) {
       const client = new ZepCloudClient(options);
       const userId = idForScenario('user', scenario);
@@ -257,7 +277,11 @@ export function createGuardBenchAdapter(options = {}) {
       await client.createUser(userId);
       await client.createSession({ sessionId, userId });
       await addInBatches(client, { sessionId, messages });
-      const ingestDelayMs = Number(options.ingestDelayMs ?? process.env.ZEP_GUARDBENCH_INGEST_DELAY_MS ?? DEFAULT_INGEST_DELAY_MS);
+      const ingestDelayMs = Number(
+        options.ingestDelayMs ??
+          process.env.ZEP_GUARDBENCH_INGEST_DELAY_MS ??
+          DEFAULT_INGEST_DELAY_MS,
+      );
       if (ingestDelayMs > 0) await sleep(ingestDelayMs);
       return { client, userId, sessionId };
     },

package/benchmarks/baselines.js

@@ -26,7 +26,10 @@ function keywordScore(queryTokens, content) {
 function sortByScore(rows) {
   return rows
     .filter(row => Number.isFinite(row.score))
-    .sort((a, b) => b.score - a.score || String(b.createdAt || '').localeCompare(String(a.createdAt || '')));
+    .sort(
+      (a, b) =>
+        b.score - a.score || String(b.createdAt || '').localeCompare(String(a.createdAt || '')),
+    );
 }
 
 function flattenMemories(benchmarkCase, ids = []) {
@@ -127,11 +130,13 @@ export async function runBaselineScenario(system, benchmarkCase, providerConfig,
 
 export function runKeywordRecencyBaseline(benchmarkCase, limit = 5) {
   const queryTokens = tokenize(benchmarkCase.query);
-  return sortByScore(flattenMemories(benchmarkCase).map(memory => ({
-    ...memory,
-    type: 'episodic',
-    score: keywordScore(queryTokens, memory.content),
-  }))).slice(0, limit);
+  return sortByScore(
+    flattenMemories(benchmarkCase).map(memory => ({
+      ...memory,
+      type: 'episodic',
+      score: keywordScore(queryTokens, memory.content),
+    })),
+  ).slice(0, limit);
 }
 
 export function runRecentWindowBaseline(benchmarkCase, limit = 3) {

package/benchmarks/build-leaderboard.mjs

@@ -34,14 +34,16 @@ function rowFromBundle(dir) {
 
 function compareRows(a, b) {
   return (
-    Number(b.verification.ok) - Number(a.verification.ok)
-    || Number(b.conformance.ok) - Number(a.conformance.ok)
-    || (b.score.fullContractPassRate ?? -1) - (a.score.fullContractPassRate ?? -1)
-    || (b.score.decisionAccuracy ?? -1) - (a.score.decisionAccuracy ?? -1)
-    || (b.score.evidenceRecall ?? -1) - (a.score.evidenceRecall ?? -1)
-    || (a.score.redactionLeaks ?? Number.MAX_SAFE_INTEGER) - (b.score.redactionLeaks ?? Number.MAX_SAFE_INTEGER)
-    || (a.score.latency?.p95Ms ?? Number.MAX_SAFE_INTEGER) - (b.score.latency?.p95Ms ?? Number.MAX_SAFE_INTEGER)
-    || a.subject.name.localeCompare(b.subject.name)
+    Number(b.verification.ok) - Number(a.verification.ok) ||
+    Number(b.conformance.ok) - Number(a.conformance.ok) ||
+    (b.score.fullContractPassRate ?? -1) - (a.score.fullContractPassRate ?? -1) ||
+    (b.score.decisionAccuracy ?? -1) - (a.score.decisionAccuracy ?? -1) ||
+    (b.score.evidenceRecall ?? -1) - (a.score.evidenceRecall ?? -1) ||
+    (a.score.redactionLeaks ?? Number.MAX_SAFE_INTEGER) -
+      (b.score.redactionLeaks ?? Number.MAX_SAFE_INTEGER) ||
+    (a.score.latency?.p95Ms ?? Number.MAX_SAFE_INTEGER) -
+      (b.score.latency?.p95Ms ?? Number.MAX_SAFE_INTEGER) ||
+    a.subject.name.localeCompare(b.subject.name)
   );
 }
 
@@ -49,7 +51,9 @@ export function buildGuardBenchLeaderboard(options = {}) {
   const bundleDirs = options.bundleDirs?.length
     ? options.bundleDirs
     : ['benchmarks/output/submission-bundle'];
-  const rows = bundleDirs.map(rowFromBundle).sort(compareRows)
+  const rows = bundleDirs
+    .map(rowFromBundle)
+    .sort(compareRows)
     .map((row, index) => ({ rank: index + 1, ...row }));
   return {
     schemaVersion: '1.0.0',
@@ -66,12 +70,16 @@ export function buildGuardBenchLeaderboard(options = {}) {
       'subject.name',
     ],
     rows,
-    failures: rows.flatMap(row => row.verification.failures.map(failure => `${row.subject.name}: ${failure}`)),
+    failures: rows.flatMap(row =>
+      row.verification.failures.map(failure => `${row.subject.name}: ${failure}`),
+    ),
   };
 }
 
 export function writeGuardBenchLeaderboard(options = {}) {
-  const outJson = resolve(options.outJson ?? 'benchmarks/output/leaderboard/guardbench-leaderboard.json');
+  const outJson = resolve(
+    options.outJson ?? 'benchmarks/output/leaderboard/guardbench-leaderboard.json',
+  );
   const outMd = resolve(options.outMd ?? 'benchmarks/output/leaderboard/guardbench-leaderboard.md');
   const schemasDir = resolve(options.schemasDir ?? 'benchmarks/schemas');
   const leaderboard = buildGuardBenchLeaderboard(options);
@@ -97,18 +105,23 @@ export function renderMarkdown(leaderboard) {
     '|---:|---|---:|---:|---:|---:|---:|---:|---:|---|',
   ];
   for (const row of leaderboard.rows) {
-    lines.push([
-      row.rank,
-      row.subject.name,
-      row.verification.ok ? 'yes' : 'no',
-      row.conformance.ok ? 'yes' : 'no',
-      percent(row.score.fullContractPassRate),
-      percent(row.score.decisionAccuracy),
-      percent(row.score.evidenceRecall),
-      number(row.score.redactionLeaks),
-      row.score.latency?.p95Ms == null ? 'n/a' : `${row.score.latency.p95Ms}ms`,
-      row.source.dir,
-    ].join(' | ').replace(/^/, '| ').replace(/$/, ' |'));
+    lines.push(
+      [
+        row.rank,
+        row.subject.name,
+        row.verification.ok ? 'yes' : 'no',
+        row.conformance.ok ? 'yes' : 'no',
+        percent(row.score.fullContractPassRate),
+        percent(row.score.decisionAccuracy),
+        percent(row.score.evidenceRecall),
+        number(row.score.redactionLeaks),
+        row.score.latency?.p95Ms == null ? 'n/a' : `${row.score.latency.p95Ms}ms`,
+        row.source.dir,
+      ]
+        .join(' | ')
+        .replace(/^/, '| ')
+        .replace(/$/, ' |'),
+    );
   }
   if (leaderboard.failures.length) {
     lines.push('', '## Verification Failures', '');

package/benchmarks/cases.js

@@ -60,7 +60,8 @@ export const RETRIEVAL_CASES = [
     expectAny: ['Northwind'],
     memory: [
       {
-        content: 'During the January pilot, Sam requested budget approval for vendors Northwind and Fabricam.',
+        content:
+          'During the January pilot, Sam requested budget approval for vendors Northwind and Fabricam.',
         source: 'tool-result',
         tags: ['project', 'pilot'],
         context: { subject: 'sam', domain: 'operations' },
@@ -72,7 +73,8 @@ export const RETRIEVAL_CASES = [
         context: { subject: 'sam', domain: 'operations' },
       },
       {
-        content: 'The pilot budget review approved Northwind for rollout after the support SLA review.',
+        content:
+          'The pilot budget review approved Northwind for rollout after the support SLA review.',
         source: 'direct-observation',
         tags: ['finance', 'vendor', 'approval'],
         context: { subject: 'sam', domain: 'operations' },
@@ -169,17 +171,20 @@ export const RETRIEVAL_CASES = [
     expectAny: ['cap retry batches', 'stagger retries'],
     memory: [
       {
-        content: 'Processor X returned HTTP 429 when payout retries exceeded 120 requests per minute.',
+        content:
+          'Processor X returned HTTP 429 when payout retries exceeded 120 requests per minute.',
         source: 'direct-observation',
         tags: ['payments', 'rate-limit'],
       },
       {
-        content: 'Payout incident volume dropped after retry batches were capped at 50 merchants per worker.',
+        content:
+          'Payout incident volume dropped after retry batches were capped at 50 merchants per worker.',
         source: 'tool-result',
         tags: ['payments', 'rate-limit'],
       },
       {
-        content: 'Risk operations requested an escalation when multiple merchants were affected in the same hour.',
+        content:
+          'Risk operations requested an escalation when multiple merchants were affected in the same hour.',
         source: 'told-by-user',
         tags: ['payments', 'escalation'],
       },
@@ -188,7 +193,8 @@ export const RETRIEVAL_CASES = [
       minClusterSize: 3,
       similarityThreshold: -0.3,
       principle: {
-        content: 'When payout retries start returning 429, cap retry batches and stagger retries before escalating.',
+        content:
+          'When payout retries start returning 429, cap retry batches and stagger retries before escalating.',
         type: 'procedural',
         conditions: ['processor returns 429', 'multiple merchants impacted'],
       },
@@ -343,7 +349,8 @@ export const OPERATION_CASES = [
     kind: 'operations',
     family: 'procedural_merge',
     title: 'Procedural merge',
-    description: 'Related episodes should merge into an executable procedure, not just a loose fact.',
+    description:
+      'Related episodes should merge into an executable procedure, not just a loose fact.',
     query: 'What should the agent do after two webhook signature failures?',
     expectAny: ['rotate the signing secret', 'replay queued events'],
     steps: [
@@ -376,7 +383,8 @@ export const OPERATION_CASES = [
         minClusterSize: 3,
         similarityThreshold: -0.3,
         principle: {
-          content: 'When webhook signature verification fails twice, rotate the signing secret and replay queued events.',
+          content:
+            'When webhook signature verification fails twice, rotate the signing secret and replay queued events.',
           type: 'procedural',
           conditions: ['signature verification fails twice', 'queued events pending'],
         },
@@ -395,7 +403,8 @@ export const GUARD_CASES = [
     kind: 'guard',
     family: 'closed_loop_failure_memory',
     title: 'Guard remembers failed tool outcome',
-    description: 'A failed guarded tool run should create a future caution and warning reflex for the same tool.',
+    description:
+      'A failed guarded tool run should create a future caution and warning reflex for the same tool.',
     action: 'run npm test before release',
     tool: 'npm test',
     expectAll: ['decision:caution', 'warning:recent_failure', 'reflex:warn'],
@@ -439,7 +448,8 @@ export const GUARD_CASES = [
     kind: 'guard',
     family: 'guard_receipt_hardening',
     title: 'Guard rejects replayed receipt outcomes',
-    description: 'A receipt should only be closed once, while the failed outcome still becomes future caution memory.',
+    description:
+      'A receipt should only be closed once, while the failed outcome still becomes future caution memory.',
     action: 'run npm test before release',
     tool: 'npm test',
     expectAll: ['guard_hardened:replay_rejected', 'decision:caution', 'warning:recent_failure'],
@@ -470,7 +480,8 @@ export const GUARD_CASES = [
     kind: 'guard',
     family: 'guard_receipt_hardening',
     title: 'Guard rejects non-guard receipts',
-    description: 'A normal tool trace must not be accepted as a guard receipt for after-action feedback.',
+    description:
+      'A normal tool trace must not be accepted as a guard receipt for after-action feedback.',
     action: 'format docs',
     tool: 'Bash',
     expectAll: ['guard_hardened:non_guard_receipt_rejected'],
@@ -511,7 +522,8 @@ export const LOCAL_BENCHMARK_SUITES = [
   {
     id: 'guard',
     title: 'Agent guard loop',
-    description: 'Closed-loop memory-before-action behavior for receipts, warnings, and blocking reflexes.',
+    description:
+      'Closed-loop memory-before-action behavior for receipts, warnings, and blocking reflexes.',
     comparableToBaselines: false,
     cases: GUARD_CASES,
   },