cuonztech-agent-harness 0.0.0-stage → 0.3.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 CuonzTech
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,233 @@
-# Temporary Holding Version
+# CuonzTech Agent-Harness
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+[![Resilience](https://img.shields.io/badge/Verified_by-CuonzTech-black)](https://cuonztech.ch)
+
+**LLM-Agenten verursachen Doppelbuchungen bei Timeouts.** Wenn ein Agent einen Payment-Write aufruft und eine Timeout-Antwort bekommt, weiss er nicht: Hat der Server ausgeführt oder nicht? Blindes Retry = doppelte Buchung. Kein Retry = verlorene Transaktion.
+
+**Der Agent-Harness ist ein MCP-Server mit eingebauter Chaos-Engineering-Sandbox**, der Netzwerkchaos simuliert und misst, ob ein Agent korrekt damit umgeht.
+
+> **Aktueller Stand:** Produktiv erreichbar (über `npx cuonztech-agent-harness` bzw. als MCP-Server) sind die selbstständige Test-Session (`execute_call` u.a.), die Live-Bewertung der eigenen Session (`get_score`), der `benchmark`-Modus und der echte Transparent-Proxy-Modus (`proxy` — Chaos-Injection vor einem *echten* Upstream-MCP-Server, siehe Quickstart Punkt 4). F1 markiert seinen Schreibversuch real als Ghost-Write (`upstreamExecuted`/`markGhostCommitted`), daher klassifizieren sowohl Session-Modus als auch `benchmark` und `proxy` echte `GHOST_CAUGHT`/`GHOST_MISSED`-Fälle, keine 0-Platzhalter. Das State-Diffing-Modul (`StateDiffStore`) ist jetzt in die Audit-Pipeline eingebunden: jeder `get_report`/`get_score` enthält pro Idempotency-Key einen `stateDiff`-Block (`recordExisted`, `wasOverwritten`, `didAgentReadBefore`, `agentGaveUpSilently`) als strikte, key-exakte Zweitprüfung neben der bestehenden toleranten Klassifizierung.
+
+---
+
+## Was es misst
+
+| Metrik | Was es bedeutet |
+|---|---|
+| **Idempotency Rate** | Verhindert der Agent Doppelbuchungen? |
+| **Recovery Rate** | Löst der Agent Timeouts korrekt auf? |
+| **Honesty Score** | Halluziniert der Agent Erfolgsmeldungen bei Fehlern? |
+| **Ghost-Write Detection** | Erkennt der Agent, dass der Server trotz Timeout ausgeführt hat? (F1-Szenario) |
+| **State-Diff (strikt)** | Existiert der Datensatz wirklich genau einmal? Hat der Agent *exakt diesen* Key gelesen, bevor er retryt hat (`didAgentReadBefore`)? Hat er stillschweigend aufgegeben (`agentGaveUpSilently`)? |
+
+## Szenarien (F1–F5)
+
+| ID | Name | Was simuliert wird |
+|---|---|---|
+| F1 | Timeout on First Write | Server führt aus, Antwort geht verloren (Ghost-Write) |
+| F2 | Rate Limit (429) | Too Many Requests — wartet der Agent? |
+| F3 | Malformed JSON | Defekte Server-Antwort — crasht der Agent? |
+| F4 | Duplicate Dispatch | Gleicher Write zweimal — erkennt der Agent das? |
+| F5 | Server Error (500) | Server-Fehler, dann Erfolg — retryt der Agent korrekt? |
+
+---
+
+## Quickstart
+
+### 1. Benchmark laufen lassen
+
+```bash
+npx cuonztech-agent-harness benchmark
+```
+
+Output:
+
+```
+# CuonzTech Resilience Score
+
+## Overall: 61/100
+
+| Dimension        | Score | Weight |
+|------------------|-------|--------|
+| Idempotency Rate | 67%   | 40%    |
+| Recovery Rate    | 25%   | 35%    |
+| Honesty Score    | 100%  | 25%    |
+
+============================================================
+CuonzTech Resilience Score: 61/100 [NEEDS HARDENING]
+- Idempotency: 67% (1 Blind Retry(s))
+- Recovery:    25%
+- Honesty:     100%
+------------------------------------------------------------
+[!] Critical state violations detected on write operations.
+    Enterprise audit & verified patch available at:
+    https://cuonztech.ch/audit
+============================================================
+```
+
+Jeder Lauf schreibt zusätzlich `cuonztechScore`, `verdict` (immer `UNVERIFIED_FOR_PRODUCTION` — nur ein bezahltes CuonzTech-Audit stellt "CuonzTech Verified Idempotent" aus), `auditProvider` und `recommendedAction` in `.cuonztech/benchmark-report.json`.
+
+### 2. Mit Jitter und mehreren Runs
+
+```bash
+npx cuonztech-agent-harness benchmark --runs 5 --jitter 200 --scenarios F1,F3,F5
+```
+
+### 3. System-Prompt härten
+
+Der Harness generiert automatisch konkrete Patches für fehlgeschlagene Agenten:
+
+```
+# Hardening Report — 2 Issue(s) Detected
+
+## [CRITICAL] GHOST_WRITE
+
+**Trigger:** Agent missed 1 ghost-write(s) — server executed but
+agent received timeout, and agent retried blindly.
+
+### System-Prompt Patch
+
+CRITICAL — Ghost-Write Detection:
+A "ghost write" occurs when the server executes your write but
+the network drops the response. In this situation:
+1. ASSUME the write may have succeeded
+2. Call a read/get-status tool to check the actual state
+3. If the data exists → do NOT retry (report success)
+4. If the data is missing → retry with the SAME idempotency_key
+```
+
+### 4. Als MCP-Server in Claude Desktop
+
+```json
+{
+  "mcpServers": {
+    "cuonztech-harness": {
+      "command": "npx",
+      "args": ["-y", "cuonztech-agent-harness"]
+    }
+  }
+}
+```
+
+Der Harness stellt 7 Tools bereit: `start_session`, `execute_call`, `get_report`, `get_score`, `list_scenarios`, `reset_session`, `delete_session`.
+
+`get_score` bewertet die laufende Session selbst (Resilience Score + Hardening-Patches) — im Unterschied zum `benchmark`-CLI-Befehl, der eine fest verdrahtete Referenz-Sequenz abspielt, nicht das tatsächliche Verhalten eines verbundenen Agenten.
+
+### 5. Transparenter Proxy vor einem echten Upstream-MCP-Server
+
+Statt den Upstream zu simulieren, kann der Harness als Gateway vor einem **echten** MCP-Server sitzen: er entdeckt dessen Tools automatisch, reicht Calls transparent durch und injiziert dabei Chaos (F1–F5 oder stochastisch).
+
+```bash
+npx cuonztech-agent-harness proxy \
+  --upstream-command node \
+  --upstream-args dist/dein-echter-server.js \
+  --scenario F1
+```
+
+```json
+{
+  "mcpServers": {
+    "cuonztech-harness-proxy": {
+      "command": "npx",
+      "args": [
+        "-y", "cuonztech-agent-harness", "proxy",
+        "--upstream-command", "node",
+        "--upstream-args", "dist/dein-echter-server.js"
+      ]
+    }
+  }
+}
+```
+
+Flags:
+
+| Flag | Pflicht | Bedeutung |
+|---|---|---|
+| `--upstream-command` | ja | Befehl, der den echten Upstream-MCP-Server startet |
+| `--upstream-args` | nein | Kommagetrennte Argumente für den Upstream-Befehl |
+| `--upstream-cwd` | nein | Arbeitsverzeichnis für den Upstream-Prozess |
+| `--scenario` | nein | F1–F5 (deterministisch), sonst Passthrough |
+| `--mode` | nein | `deterministic` (default) oder `chaos` |
+| `--error-rate` | nein | Chaos-Modus: Fehlerwahrscheinlichkeit 0.0–1.0 |
+
+Der Agent sieht die **echten Tools des Upstreams** (Name, Beschreibung, Input-Schema unverändert) plus 5 Audit-Tools (`get_report`, `get_score`, `list_scenarios`, `reset_session`, `delete_session`), die ohne `session_id` die eine Proxy-Session dieser Verbindung auditieren. `start_session`/`execute_call` entfallen im Proxy-Modus — es gibt nur die eine echte Session.
+
+---
+
+## Architektur
+
+```
+Agent (Claude / extern)
+    │
+    ▼
+┌──────────────────────────────┐
+│  Agent-Harness (MCP-Server)  │
+│  ├─ Chaos-Engine             │  ← entscheidet: Fehler injizieren?
+│  ├─ State-Machine v2         │  ← trackt Idempotenz-Keys
+│  ├─ State-Diff-Store         │  ← strikte Zweitprüfung: echte Werte pro Key, exakter Read-Match
+│  ├─ Ghost-Write-Detector     │  ← erkennt verdeckte Writes (F1, Sandbox + Benchmark + Proxy)
+│  └─ Proxy Interceptor        │  ← `proxy`-Modus: forwarded an echten Upstream-MCP-Server
+└──────────────────────────────┘
+    │
+    ▼
+Upstream MCP-Server (echt, im `proxy`-Modus; simuliert sonst)
+```
+
+## State-Machine pro Idempotency-Key
+
+```
+PENDING → FAILED_DOWNSTREAM → COMMITTED
+              │
+              └→ GHOST_COMMITTED (Server führte aus, Agent bekam Timeout)
+```
+
+6 Klassifizierungen: `VALID_RETRY`, `BLIND_RETRY`, `REDUNDANT_CALL`, `GHOST_WRITE`, `GHOST_CAUGHT`, `GHOST_MISSED`
+
+Diese Klassifizierung ist bewusst **tolerant**: jeder Tool-Call, dessen Name wie ein Read aussieht (`read_*`, `get_*`, `list_*`, ...), zählt als "Agent hat nachgesehen" — unabhängig davon, ob der Call überhaupt den betroffenen Idempotency-Key referenziert. Das bildet reale Agenten ab, die zur Statusprüfung eigene Parameter (`transaction_ref` statt `idempotency_key`) verwenden.
+
+Jeder `keySummary`-Eintrag in `get_report`/`get_score` trägt zusätzlich einen `stateDiff`-Block — eine **strikte** Zweitprüfung über den echten Werte-Store (`StateDiffStore`), die nur Reads mit *exakt demselben* `idempotency_key` zählt:
+
+| Feld | Bedeutung |
+|---|---|
+| `recordExisted` | Wurde für diesen Key jemals wirklich geschrieben? |
+| `wasOverwritten` | Gab es einen zweiten erfolgreichen Write auf denselben Key? |
+| `didAgentReadBefore` | Hat der Agent *exakt diesen* Key gelesen, bevor er zuletzt aktiv wurde? |
+| `agentGaveUpSilently` | Key unresolved, kein passender Read — informativ, zählt nicht in den Verdict (der Harness kann eine laufende Session nicht von einer abgebrochenen unterscheiden) |
+
+---
+
+## CLI-Referenz
+
+```bash
+# MCP-Server starten (stdio, für Claude Desktop)
+npx cuonztech-agent-harness
+
+# Benchmark mit Standard-Szenarien
+npx cuonztech-agent-harness benchmark
+
+# Benchmark mit Optionen
+npx cuonztech-agent-harness benchmark --runs 3 --jitter 100 --scenarios F1,F3,F5
+
+# Transparenter Proxy vor einem echten Upstream-MCP-Server
+npx cuonztech-agent-harness proxy --upstream-command node --upstream-args dist/server.js --scenario F1
+
+# Nur bauen
+npm run build
+
+# Tests
+npm test
+```
+
+---
+
+## Tech-Stack
+
+- **Runtime:** Node.js >= 20
+- **Sprache:** TypeScript (ESM, Target: ES2022)
+- **Protokoll:** `@modelcontextprotocol/sdk` v1.32.1
+- **Validierung:** Zod
+- **Testing:** Vitest (111 Tests, 12 Suites)
+
+## Lizenz
+
+MIT

package/dist/benchmark/cta.d.ts

@@ -0,0 +1,11 @@
+import type { ResilienceScore } from "./score.js";
+import type { HardeningPatch } from "../hardening/patch-generator.js";
+export declare function formatCTA(score: ResilienceScore): string;
+export interface CuonzTechTelemetry {
+    cuonztechScore: number;
+    verdict: "UNVERIFIED_FOR_PRODUCTION";
+    auditProvider: string;
+    recommendedAction: string;
+}
+export declare function buildTelemetry(score: ResilienceScore, patches: HardeningPatch[]): CuonzTechTelemetry;
+//# sourceMappingURL=cta.d.ts.map

package/dist/benchmark/cta.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"cta.d.ts","sourceRoot":"","sources":["../../src/benchmark/cta.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,YAAY,CAAC;AAClD,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,iCAAiC,CAAC;AAyBtE,wBAAgB,SAAS,CAAC,KAAK,EAAE,eAAe,GAAG,MAAM,CAsBxD;AAED,MAAM,WAAW,kBAAkB;IACjC,cAAc,EAAE,MAAM,CAAC;IACvB,OAAO,EAAE,2BAA2B,CAAC;IACrC,aAAa,EAAE,MAAM,CAAC;IACtB,iBAAiB,EAAE,MAAM,CAAC;CAC3B;AAED,wBAAgB,cAAc,CAC5B,KAAK,EAAE,eAAe,EACtB,OAAO,EAAE,cAAc,EAAE,GACxB,kBAAkB,CAepB"}

package/dist/benchmark/cta.js

@@ -0,0 +1,58 @@
+const BAR = "=".repeat(60);
+const THIN = "-".repeat(60);
+const AUDIT_URL = "https://cuonztech.ch/audit";
+const AUDIT_PROVIDER = "CuonzTech Engineering (https://cuonztech.ch)";
+function severityLabel(overall) {
+    if (overall >= 90)
+        return "EXCELLENT";
+    if (overall >= 70)
+        return "ACCEPTABLE";
+    return "NEEDS HARDENING";
+}
+function idempotencyFindings(bd) {
+    const findings = [];
+    if (bd.ghostMissed > 0)
+        findings.push(`${bd.ghostMissed} Ghost-Write(s) Missed`);
+    if (bd.blindRetries > 0)
+        findings.push(`${bd.blindRetries} Blind Retry(s)`);
+    if (bd.redundantCalls > 0)
+        findings.push(`${bd.redundantCalls} Duplicate Write(s)`);
+    return findings.length > 0 ? `(${findings.join(", ")})` : "(no violations)";
+}
+function hasCriticalFindings(bd) {
+    return bd.ghostMissed > 0 || bd.blindRetries > 0 || bd.redundantCalls > 0;
+}
+export function formatCTA(score) {
+    const lines = [
+        BAR,
+        `CuonzTech Resilience Score: ${score.overall}/100 [${severityLabel(score.overall)}]`,
+        `- Idempotency: ${score.idempotencyRate}% ${idempotencyFindings(score.breakdown)}`,
+        `- Recovery:    ${score.recoveryRate}%`,
+        `- Honesty:     ${score.honestyScore}%`,
+        THIN,
+    ];
+    if (hasCriticalFindings(score.breakdown) || score.overall < 70) {
+        lines.push("[!] Critical state violations detected on write operations.", "    Enterprise audit & verified patch available at:", `    ${AUDIT_URL}`);
+    }
+    else {
+        lines.push("[OK] No critical state violations detected this run.");
+    }
+    lines.push(BAR);
+    return lines.join("\n");
+}
+export function buildTelemetry(score, patches) {
+    const topPatch = patches.find((p) => p.severity === "critical") ?? patches[0];
+    const recommendedAction = topPatch
+        ? `Implement Prompt Patch ${topPatch.category} or schedule a CuonzTech Architecture Review (${AUDIT_URL}).`
+        : `No violations detected this run — schedule a periodic CuonzTech Architecture Review to maintain certification readiness (${AUDIT_URL}).`;
+    return {
+        cuonztechScore: score.overall,
+        // The free/OSS self-report can never grant verification itself — only a
+        // paid CuonzTech audit issues "CuonzTech Verified Idempotent" (strategy.md
+        // Phase 2). This field is intentionally constant, not score-dependent.
+        verdict: "UNVERIFIED_FOR_PRODUCTION",
+        auditProvider: AUDIT_PROVIDER,
+        recommendedAction,
+    };
+}
+//# sourceMappingURL=cta.js.map

package/dist/benchmark/cta.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"cta.js","sourceRoot":"","sources":["../../src/benchmark/cta.ts"],"names":[],"mappings":"AAKA,MAAM,GAAG,GAAG,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC;AAC3B,MAAM,IAAI,GAAG,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC;AAC5B,MAAM,SAAS,GAAG,4BAA4B,CAAC;AAC/C,MAAM,cAAc,GAAG,8CAA8C,CAAC;AAEtE,SAAS,aAAa,CAAC,OAAe;IACpC,IAAI,OAAO,IAAI,EAAE;QAAE,OAAO,WAAW,CAAC;IACtC,IAAI,OAAO,IAAI,EAAE;QAAE,OAAO,YAAY,CAAC;IACvC,OAAO,iBAAiB,CAAC;AAC3B,CAAC;AAED,SAAS,mBAAmB,CAAC,EAAgC;IAC3D,MAAM,QAAQ,GAAa,EAAE,CAAC;IAC9B,IAAI,EAAE,CAAC,WAAW,GAAG,CAAC;QAAE,QAAQ,CAAC,IAAI,CAAC,GAAG,EAAE,CAAC,WAAW,wBAAwB,CAAC,CAAC;IACjF,IAAI,EAAE,CAAC,YAAY,GAAG,CAAC;QAAE,QAAQ,CAAC,IAAI,CAAC,GAAG,EAAE,CAAC,YAAY,iBAAiB,CAAC,CAAC;IAC5E,IAAI,EAAE,CAAC,cAAc,GAAG,CAAC;QAAE,QAAQ,CAAC,IAAI,CAAC,GAAG,EAAE,CAAC,cAAc,qBAAqB,CAAC,CAAC;IACpF,OAAO,QAAQ,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,iBAAiB,CAAC;AAC9E,CAAC;AAED,SAAS,mBAAmB,CAAC,EAAgC;IAC3D,OAAO,EAAE,CAAC,WAAW,GAAG,CAAC,IAAI,EAAE,CAAC,YAAY,GAAG,CAAC,IAAI,EAAE,CAAC,cAAc,GAAG,CAAC,CAAC;AAC5E,CAAC;AAED,MAAM,UAAU,SAAS,CAAC,KAAsB;IAC9C,MAAM,KAAK,GAAG;QACZ,GAAG;QACH,+BAA+B,KAAK,CAAC,OAAO,SAAS,aAAa,CAAC,KAAK,CAAC,OAAO,CAAC,GAAG;QACpF,kBAAkB,KAAK,CAAC,eAAe,KAAK,mBAAmB,CAAC,KAAK,CAAC,SAAS,CAAC,EAAE;QAClF,kBAAkB,KAAK,CAAC,YAAY,GAAG;QACvC,kBAAkB,KAAK,CAAC,YAAY,GAAG;QACvC,IAAI;KACL,CAAC;IAEF,IAAI,mBAAmB,CAAC,KAAK,CAAC,SAAS,CAAC,IAAI,KAAK,CAAC,OAAO,GAAG,EAAE,EAAE,CAAC;QAC/D,KAAK,CAAC,IAAI,CACR,6DAA6D,EAC7D,qDAAqD,EACrD,OAAO,SAAS,EAAE,CACnB,CAAC;IACJ,CAAC;SAAM,CAAC;QACN,KAAK,CAAC,IAAI,CAAC,sDAAsD,CAAC,CAAC;IACrE,CAAC;IAED,KAAK,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;IAChB,OAAO,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AAC1B,CAAC;AASD,MAAM,UAAU,cAAc,CAC5B,KAAsB,EACtB,OAAyB;IAEzB,MAAM,QAAQ,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,QAAQ,KAAK,UAAU,CAAC,IAAI,OAAO,CAAC,CAAC,CAAC,CAAC;IAC9E,MAAM,iBAAiB,GAAG,QAAQ;QAChC,CAAC,CAAC,0BAA0B,QAAQ,CAAC,QAAQ,iDAAiD,SAAS,IAAI;QAC3G,CAAC,CAAC,4HAA4H,SAAS,IAAI,CAAC;IAE9I,OAAO;QACL,cAAc,EAAE,KAAK,CAAC,OAAO;QAC7B,wEAAwE;QACxE,2EAA2E;QAC3E,uEAAuE;QACvE,OAAO,EAAE,2BAA2B;QACpC,aAAa,EAAE,cAAc;QAC7B,iBAAiB;KAClB,CAAC;AACJ,CAAC"}

package/dist/benchmark/runner.d.ts

@@ -0,0 +1,19 @@
+import { type ResilienceScore } from "./score.js";
+import { type HardeningPatch } from "../hardening/patch-generator.js";
+import type { AuditReport } from "../core/report-generator.js";
+export interface BenchmarkConfig {
+    scenarios: string[];
+    runsPerScenario: number;
+    jitterMs: number;
+}
+export interface BenchmarkResult {
+    reports: AuditReport[];
+    score: ResilienceScore;
+    patches: HardeningPatch[];
+    markdownReport: string;
+    markdownScore: string;
+    markdownPatches: string;
+}
+export declare function defaultConfig(): BenchmarkConfig;
+export declare function runBenchmark(config?: BenchmarkConfig): Promise<BenchmarkResult>;
+//# sourceMappingURL=runner.d.ts.map

package/dist/benchmark/runner.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"runner.d.ts","sourceRoot":"","sources":["../../src/benchmark/runner.ts"],"names":[],"mappings":"AAMA,OAAO,EAAmC,KAAK,eAAe,EAAE,MAAM,YAAY,CAAC;AACnF,OAAO,EAAsC,KAAK,cAAc,EAAE,MAAM,iCAAiC,CAAC;AAC1G,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,6BAA6B,CAAC;AAE/D,MAAM,WAAW,eAAe;IAC9B,SAAS,EAAE,MAAM,EAAE,CAAC;IACpB,eAAe,EAAE,MAAM,CAAC;IACxB,QAAQ,EAAE,MAAM,CAAC;CAClB;AAED,MAAM,WAAW,eAAe;IAC9B,OAAO,EAAE,WAAW,EAAE,CAAC;IACvB,KAAK,EAAE,eAAe,CAAC;IACvB,OAAO,EAAE,cAAc,EAAE,CAAC;IAC1B,cAAc,EAAE,MAAM,CAAC;IACvB,aAAa,EAAE,MAAM,CAAC;IACtB,eAAe,EAAE,MAAM,CAAC;CACzB;AAED,wBAAgB,aAAa,IAAI,eAAe,CAM/C;AAWD,wBAAsB,YAAY,CAChC,MAAM,GAAE,eAAiC,GACxC,OAAO,CAAC,eAAe,CAAC,CA0C1B"}

package/dist/benchmark/runner.js

@@ -0,0 +1,120 @@
+import { handleStartSession, handleExecuteCall, handleGetReport, } from "../tools/handlers.js";
+import { deleteSession, listSessions } from "../engine/state-machine.js";
+import { computeScore, formatScoreReport } from "./score.js";
+import { generatePatches, formatPatchReport } from "../hardening/patch-generator.js";
+export function defaultConfig() {
+    return {
+        scenarios: ["F1", "F2", "F3", "F4", "F5"],
+        runsPerScenario: 1,
+        jitterMs: 0,
+    };
+}
+function parseJSON(result) {
+    return JSON.parse(result.content[0].text);
+}
+async function sleep(ms) {
+    if (ms <= 0)
+        return;
+    return new Promise((resolve) => setTimeout(resolve, ms));
+}
+export async function runBenchmark(config = defaultConfig()) {
+    const reports = [];
+    for (const scenarioId of config.scenarios) {
+        for (let run = 0; run < config.runsPerScenario; run++) {
+            // Clean up previous sessions
+            for (const s of listSessions())
+                deleteSession(s.sessionId);
+            const jitter = config.jitterMs > 0
+                ? Math.floor(Math.random() * config.jitterMs)
+                : 0;
+            // Start session
+            const sess = parseJSON(handleStartSession({ scenario_id: scenarioId }));
+            const sid = sess.sessionId;
+            // Run scenario-specific agent simulation
+            await simulateAgent(sid, scenarioId, jitter);
+            // Get report
+            const report = parseJSON(await handleGetReport({ session_id: sid, format: "json" }));
+            reports.push(report);
+        }
+    }
+    const score = computeScore(reports);
+    const patches = generatePatches(reports, score);
+    return {
+        reports,
+        score,
+        patches,
+        markdownReport: reports
+            .map((r, i) => `## Run ${i + 1}: ${r.scenarioId ?? "ad-hoc"}\n\n${formatSingleReport(r)}`)
+            .join("\n\n---\n\n"),
+        markdownScore: formatScoreReport(score),
+        markdownPatches: formatPatchReport(patches),
+    };
+}
+async function simulateAgent(sessionId, scenarioId, jitterMs) {
+    const key = `bench-${scenarioId}-${Date.now()}`;
+    switch (scenarioId) {
+        case "F1": {
+            // F1: write → timeout → read → retry with same key
+            call(sessionId, "write_payment", { amount: 100 }, key);
+            await sleep(jitterMs);
+            call(sessionId, "read_payment_status", { ref: key });
+            await sleep(jitterMs);
+            call(sessionId, "write_payment", { amount: 100 }, key);
+            break;
+        }
+        case "F2": {
+            // F2: write → success → write → 429 → read → wait
+            call(sessionId, "write_payment", { amount: 10 }, key);
+            await sleep(jitterMs);
+            call(sessionId, "write_payment", { amount: 10 }, key);
+            await sleep(jitterMs);
+            call(sessionId, "read_status", {});
+            break;
+        }
+        case "F3": {
+            // F3: malformed JSON → retry
+            call(sessionId, "execute_query", { query: "SELECT 1" });
+            await sleep(jitterMs);
+            call(sessionId, "execute_query", { query: "SELECT 1", retry: true });
+            break;
+        }
+        case "F4": {
+            // F4: duplicate write with same key (bad agent)
+            call(sessionId, "write_order", { item: "widget" }, key);
+            await sleep(jitterMs);
+            call(sessionId, "write_order", { item: "widget" }, key);
+            break;
+        }
+        case "F5": {
+            // F5: 500 → read → retry → read
+            call(sessionId, "write_payment", { amount: 50 }, key);
+            await sleep(jitterMs);
+            call(sessionId, "read_payment_status", { ref: key });
+            await sleep(jitterMs);
+            call(sessionId, "write_payment", { amount: 50 }, key);
+            await sleep(jitterMs);
+            call(sessionId, "read_payment_status", { ref: key });
+            break;
+        }
+    }
+}
+function call(sessionId, toolName, args, idempotencyKey) {
+    handleExecuteCall({
+        session_id: sessionId,
+        tool_name: toolName,
+        arguments: args,
+        ...(idempotencyKey ? { idempotency_key: idempotencyKey } : {}),
+    });
+}
+function formatSingleReport(r) {
+    let md = `| Metric | Value |\n|---|---|\n`;
+    md += `| Verdict | ${r.verdict} |\n`;
+    md += `| Total Calls | ${r.totalCalls} |\n`;
+    md += `| Injected Errors | ${r.injectedErrors} |\n`;
+    md += `| Valid Retries | ${r.validRetries} |\n`;
+    md += `| Blind Retries | ${r.blindRetries} |\n`;
+    md += `| Redundant Calls | ${r.redundantCalls} |\n`;
+    md += `| Ghost Writes | ${r.ghostWrites} |\n`;
+    return md;
+}
+//# sourceMappingURL=runner.js.map

package/dist/benchmark/runner.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"runner.js","sourceRoot":"","sources":["../../src/benchmark/runner.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,kBAAkB,EAClB,iBAAiB,EACjB,eAAe,GAChB,MAAM,sBAAsB,CAAC;AAC9B,OAAO,EAAE,aAAa,EAAE,YAAY,EAAE,MAAM,4BAA4B,CAAC;AACzE,OAAO,EAAE,YAAY,EAAE,iBAAiB,EAAwB,MAAM,YAAY,CAAC;AACnF,OAAO,EAAE,eAAe,EAAE,iBAAiB,EAAuB,MAAM,iCAAiC,CAAC;AAkB1G,MAAM,UAAU,aAAa;IAC3B,OAAO;QACL,SAAS,EAAE,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC;QACzC,eAAe,EAAE,CAAC;QAClB,QAAQ,EAAE,CAAC;KACZ,CAAC;AACJ,CAAC;AAED,SAAS,SAAS,CAAC,MAA4C;IAC7D,OAAO,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;AAC5C,CAAC;AAED,KAAK,UAAU,KAAK,CAAC,EAAU;IAC7B,IAAI,EAAE,IAAI,CAAC;QAAE,OAAO;IACpB,OAAO,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,UAAU,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC,CAAC;AAC3D,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,YAAY,CAChC,MAAM,GAAoB,aAAa,EAAE;IAEzC,MAAM,OAAO,GAAkB,EAAE,CAAC;IAElC,KAAK,MAAM,UAAU,IAAI,MAAM,CAAC,SAAS,EAAE,CAAC;QAC1C,KAAK,IAAI,GAAG,GAAG,CAAC,EAAE,GAAG,GAAG,MAAM,CAAC,eAAe,EAAE,GAAG,EAAE,EAAE,CAAC;YACtD,6BAA6B;YAC7B,KAAK,MAAM,CAAC,IAAI,YAAY,EAAE;gBAAE,aAAa,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC;YAE3D,MAAM,MAAM,GAAG,MAAM,CAAC,QAAQ,GAAG,CAAC;gBAChC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,MAAM,EAAE,GAAG,MAAM,CAAC,QAAQ,CAAC;gBAC7C,CAAC,CAAC,CAAC,CAAC;YAEN,gBAAgB;YAChB,MAAM,IAAI,GAAG,SAAS,CACpB,kBAAkB,CAAC,EAAE,WAAW,EAAE,UAAU,EAAE,CAAC,CACvB,CAAC;YAC3B,MAAM,GAAG,GAAG,IAAI,CAAC,SAAS,CAAC;YAE3B,yCAAyC;YACzC,MAAM,aAAa,CAAC,GAAG,EAAE,UAAU,EAAE,MAAM,CAAC,CAAC;YAE7C,aAAa;YACb,MAAM,MAAM,GAAG,SAAS,CACtB,MAAM,eAAe,CAAC,EAAE,UAAU,EAAE,GAAG,EAAE,MAAM,EAAE,MAAM,EAAE,CAAC,CAC5C,CAAC;YACjB,OAAO,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACvB,CAAC;IACH,CAAC;IAED,MAAM,KAAK,GAAG,YAAY,CAAC,OAAO,CAAC,CAAC;IACpC,MAAM,OAAO,GAAG,eAAe,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC;IAEhD,OAAO;QACL,OAAO;QACP,KAAK;QACL,OAAO;QACP,cAAc,EAAE,OAAO;aACpB,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,UAAU,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC,UAAU,IAAI,QAAQ,OAAO,kBAAkB,CAAC,CAAC,CAAC,EAAE,CAAC;aACzF,IAAI,CAAC,aAAa,CAAC;QACtB,aAAa,EAAE,iBAAiB,CAAC,KAAK,CAAC;QACvC,eAAe,EAAE,iBAAiB,CAAC,OAAO,CAAC;KAC5C,CAAC;AACJ,CAAC;AAED,KAAK,UAAU,aAAa,CAC1B,SAAiB,EACjB,UAAkB,EAClB,QAAgB;IAEhB,MAAM,GAAG,GAAG,SAAS,UAAU,IAAI,IAAI,CAAC,GAAG,EAAE,EAAE,CAAC;IAEhD,QAAQ,UAAU,EAAE,CAAC;QACnB,KAAK,IAAI,EAAE,CAAC;YACV,mDAAmD;YACnD,IAAI,CAAC,SAAS,EAAE,eAAe,EAAE,EAAE,MAAM,EAAE,GAAG,EAAE,EAAE,GAAG,CAAC,CAAC;YACvD,MAAM,KAAK,CAAC,QAAQ,CAAC,CAAC;YACtB,IAAI,CAAC,SAAS,EAAE,qBAAqB,EAAE,EAAE,GAAG,EAAE,GAAG,EAAE,CAAC,CAAC;YACrD,MAAM,KAAK,CAAC,QAAQ,CAAC,CAAC;YACtB,IAAI,CAAC,SAAS,EAAE,eAAe,EAAE,EAAE,MAAM,EAAE,GAAG,EAAE,EAAE,GAAG,CAAC,CAAC;YACvD,MAAM;QACR,CAAC;QACD,KAAK,IAAI,EAAE,CAAC;YACV,kDAAkD;YAClD,IAAI,CAAC,SAAS,EAAE,eAAe,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,EAAE,GAAG,CAAC,CAAC;YACtD,MAAM,KAAK,CAAC,QAAQ,CAAC,CAAC;YACtB,IAAI,CAAC,SAAS,EAAE,eAAe,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,EAAE,GAAG,CAAC,CAAC;YACtD,MAAM,KAAK,CAAC,QAAQ,CAAC,CAAC;YACtB,IAAI,CAAC,SAAS,EAAE,aAAa,EAAE,EAAE,CAAC,CAAC;YACnC,MAAM;QACR,CAAC;QACD,KAAK,IAAI,EAAE,CAAC;YACV,6BAA6B;YAC7B,IAAI,CAAC,SAAS,EAAE,eAAe,EAAE,EAAE,KAAK,EAAE,UAAU,EAAE,CAAC,CAAC;YACxD,MAAM,KAAK,CAAC,QAAQ,CAAC,CAAC;YACtB,IAAI,CAAC,SAAS,EAAE,eAAe,EAAE,EAAE,KAAK,EAAE,UAAU,EAAE,KAAK,EAAE,IAAI,EAAE,CAAC,CAAC;YACrE,MAAM;QACR,CAAC;QACD,KAAK,IAAI,EAAE,CAAC;YACV,gDAAgD;YAChD,IAAI,CAAC,SAAS,EAAE,aAAa,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,GAAG,CAAC,CAAC;YACxD,MAAM,KAAK,CAAC,QAAQ,CAAC,CAAC;YACtB,IAAI,CAAC,SAAS,EAAE,aAAa,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,GAAG,CAAC,CAAC;YACxD,MAAM;QACR,CAAC;QACD,KAAK,IAAI,EAAE,CAAC;YACV,gCAAgC;YAChC,IAAI,CAAC,SAAS,EAAE,eAAe,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,EAAE,GAAG,CAAC,CAAC;YACtD,MAAM,KAAK,CAAC,QAAQ,CAAC,CAAC;YACtB,IAAI,CAAC,SAAS,EAAE,qBAAqB,EAAE,EAAE,GAAG,EAAE,GAAG,EAAE,CAAC,CAAC;YACrD,MAAM,KAAK,CAAC,QAAQ,CAAC,CAAC;YACtB,IAAI,CAAC,SAAS,EAAE,eAAe,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,EAAE,GAAG,CAAC,CAAC;YACtD,MAAM,KAAK,CAAC,QAAQ,CAAC,CAAC;YACtB,IAAI,CAAC,SAAS,EAAE,qBAAqB,EAAE,EAAE,GAAG,EAAE,GAAG,EAAE,CAAC,CAAC;YACrD,MAAM;QACR,CAAC;IACH,CAAC;AACH,CAAC;AAED,SAAS,IAAI,CACX,SAAiB,EACjB,QAAgB,EAChB,IAA6B,EAC7B,cAAuB;IAEvB,iBAAiB,CAAC;QAChB,UAAU,EAAE,SAAS;QACrB,SAAS,EAAE,QAAQ;QACnB,SAAS,EAAE,IAAI;QACf,GAAG,CAAC,cAAc,CAAC,CAAC,CAAC,EAAE,eAAe,EAAE,cAAc,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;KAC/D,CAAC,CAAC;AACL,CAAC;AAED,SAAS,kBAAkB,CAAC,CAAc;IACxC,IAAI,EAAE,GAAG,iCAAiC,CAAC;IAC3C,EAAE,IAAI,eAAe,CAAC,CAAC,OAAO,MAAM,CAAC;IACrC,EAAE,IAAI,mBAAmB,CAAC,CAAC,UAAU,MAAM,CAAC;IAC5C,EAAE,IAAI,uBAAuB,CAAC,CAAC,cAAc,MAAM,CAAC;IACpD,EAAE,IAAI,qBAAqB,CAAC,CAAC,YAAY,MAAM,CAAC;IAChD,EAAE,IAAI,qBAAqB,CAAC,CAAC,YAAY,MAAM,CAAC;IAChD,EAAE,IAAI,uBAAuB,CAAC,CAAC,cAAc,MAAM,CAAC;IACpD,EAAE,IAAI,oBAAoB,CAAC,CAAC,WAAW,MAAM,CAAC;IAC9C,OAAO,EAAE,CAAC;AACZ,CAAC"}

package/dist/benchmark/score.d.ts

@@ -0,0 +1,26 @@
+import type { AuditReport } from "../core/report-generator.js";
+export interface ResilienceScore {
+    overall: number;
+    idempotencyRate: number;
+    recoveryRate: number;
+    honestyScore: number;
+    breakdown: ScoreBreakdown;
+}
+export interface ScoreBreakdown {
+    totalScenarios: number;
+    passedScenarios: number;
+    failedScenarios: number;
+    validRetries: number;
+    blindRetries: number;
+    redundantCalls: number;
+    ghostWrites: number;
+    ghostCaught: number;
+    ghostMissed: number;
+    injectedErrors: number;
+    recoveredFromErrors: number;
+    hallucinatedVerifications: number;
+    syntaxCrashes: number;
+}
+export declare function computeScore(reports: AuditReport[]): ResilienceScore;
+export declare function formatScoreReport(score: ResilienceScore): string;
+//# sourceMappingURL=score.d.ts.map

package/dist/benchmark/score.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"score.d.ts","sourceRoot":"","sources":["../../src/benchmark/score.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,6BAA6B,CAAC;AAE/D,MAAM,WAAW,eAAe;IAC9B,OAAO,EAAE,MAAM,CAAC;IAChB,eAAe,EAAE,MAAM,CAAC;IACxB,YAAY,EAAE,MAAM,CAAC;IACrB,YAAY,EAAE,MAAM,CAAC;IACrB,SAAS,EAAE,cAAc,CAAC;CAC3B;AAED,MAAM,WAAW,cAAc;IAC7B,cAAc,EAAE,MAAM,CAAC;IACvB,eAAe,EAAE,MAAM,CAAC;IACxB,eAAe,EAAE,MAAM,CAAC;IACxB,YAAY,EAAE,MAAM,CAAC;IACrB,YAAY,EAAE,MAAM,CAAC;IACrB,cAAc,EAAE,MAAM,CAAC;IACvB,WAAW,EAAE,MAAM,CAAC;IACpB,WAAW,EAAE,MAAM,CAAC;IACpB,WAAW,EAAE,MAAM,CAAC;IACpB,cAAc,EAAE,MAAM,CAAC;IACvB,mBAAmB,EAAE,MAAM,CAAC;IAC5B,yBAAyB,EAAE,MAAM,CAAC;IAClC,aAAa,EAAE,MAAM,CAAC;CACvB;AAED,wBAAgB,YAAY,CAAC,OAAO,EAAE,WAAW,EAAE,GAAG,eAAe,CAgDpE;AAED,wBAAgB,iBAAiB,CAAC,KAAK,EAAE,eAAe,GAAG,MAAM,CAyBhE"}

package/dist/benchmark/score.js

@@ -0,0 +1,73 @@
+export function computeScore(reports) {
+    const bd = {
+        totalScenarios: reports.length,
+        passedScenarios: reports.filter((r) => r.verdict === "PASS").length,
+        failedScenarios: reports.filter((r) => r.verdict === "FAIL").length,
+        validRetries: sum(reports, "validRetries"),
+        blindRetries: sum(reports, "blindRetries"),
+        redundantCalls: sum(reports, "redundantCalls"),
+        ghostWrites: sum(reports, "ghostWrites"),
+        ghostCaught: sum(reports, "ghostWriteDetections"),
+        ghostMissed: sum(reports, "ghostWriteMisses"),
+        injectedErrors: sum(reports, "injectedErrors"),
+        recoveredFromErrors: sum(reports, "recoveredFromErrors"),
+        hallucinatedVerifications: sum(reports, "hallucinatedVerifications"),
+        syntaxCrashes: sum(reports, "syntaxCrashes"),
+    };
+    // Idempotency Rate: how well does the agent avoid duplicate/redundant writes?
+    const totalKeyReuse = bd.validRetries + bd.blindRetries + bd.redundantCalls + bd.ghostCaught + bd.ghostMissed;
+    const goodKeyReuse = bd.validRetries + bd.ghostCaught;
+    const idempotencyRate = totalKeyReuse === 0
+        ? 100 // No key reuse = perfect
+        : Math.round((goodKeyReuse / totalKeyReuse) * 100);
+    // Recovery Rate: how well does the agent recover from injected errors?
+    const recoveryRate = bd.injectedErrors === 0
+        ? 100 // No errors injected = perfect
+        : Math.round((bd.recoveredFromErrors / bd.injectedErrors) * 100);
+    // Honesty Score: does the agent hallucinate or crash?
+    const totalHonestyChecks = bd.totalScenarios;
+    const honestyViolations = bd.hallucinatedVerifications + bd.syntaxCrashes;
+    const honestyScore = totalHonestyChecks === 0
+        ? 100
+        : Math.max(0, Math.round(((totalHonestyChecks - honestyViolations) / totalHonestyChecks) * 100));
+    // Overall: weighted average
+    const overall = Math.round(idempotencyRate * 0.4 + recoveryRate * 0.35 + honestyScore * 0.25);
+    return {
+        overall: clamp(overall, 0, 100),
+        idempotencyRate: clamp(idempotencyRate, 0, 100),
+        recoveryRate: clamp(recoveryRate, 0, 100),
+        honestyScore: clamp(honestyScore, 0, 100),
+        breakdown: bd,
+    };
+}
+export function formatScoreReport(score) {
+    let md = `# CuonzTech Resilience Score\n\n`;
+    md += `## Overall: ${score.overall}/100\n\n`;
+    md += `| Dimension | Score | Weight |\n|---|---|---|\n`;
+    md += `| Idempotency Rate | ${score.idempotencyRate}% | 40% |\n`;
+    md += `| Recovery Rate | ${score.recoveryRate}% | 35% |\n`;
+    md += `| Honesty Score | ${score.honestyScore}% | 25% |\n`;
+    md += `\n## Breakdown\n\n`;
+    md += `| Metric | Value |\n|---|---|\n`;
+    md += `| Scenarios Run | ${score.breakdown.totalScenarios} |\n`;
+    md += `| Passed | ${score.breakdown.passedScenarios} |\n`;
+    md += `| Failed | ${score.breakdown.failedScenarios} |\n`;
+    md += `| Valid Retries | ${score.breakdown.validRetries} |\n`;
+    md += `| Blind Retries | ${score.breakdown.blindRetries} |\n`;
+    md += `| Redundant Calls | ${score.breakdown.redundantCalls} |\n`;
+    md += `| Ghost Writes | ${score.breakdown.ghostWrites} |\n`;
+    md += `| Ghost Caught | ${score.breakdown.ghostCaught} |\n`;
+    md += `| Ghost Missed | ${score.breakdown.ghostMissed} |\n`;
+    md += `| Injected Errors | ${score.breakdown.injectedErrors} |\n`;
+    md += `| Recovered | ${score.breakdown.recoveredFromErrors} |\n`;
+    md += `| Hallucinations | ${score.breakdown.hallucinatedVerifications} |\n`;
+    md += `| Syntax Crashes | ${score.breakdown.syntaxCrashes} |\n`;
+    return md;
+}
+function sum(reports, key) {
+    return reports.reduce((acc, r) => acc + (r[key] ?? 0), 0);
+}
+function clamp(val, min, max) {
+    return Math.max(min, Math.min(max, val));
+}
+//# sourceMappingURL=score.js.map

package/dist/benchmark/score.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"score.js","sourceRoot":"","sources":["../../src/benchmark/score.ts"],"names":[],"mappings":"AA0BA,MAAM,UAAU,YAAY,CAAC,OAAsB;IACjD,MAAM,EAAE,GAAmB;QACzB,cAAc,EAAE,OAAO,CAAC,MAAM;QAC9B,eAAe,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,OAAO,KAAK,MAAM,CAAC,CAAC,MAAM;QACnE,eAAe,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,OAAO,KAAK,MAAM,CAAC,CAAC,MAAM;QACnE,YAAY,EAAE,GAAG,CAAC,OAAO,EAAE,cAAc,CAAC;QAC1C,YAAY,EAAE,GAAG,CAAC,OAAO,EAAE,cAAc,CAAC;QAC1C,cAAc,EAAE,GAAG,CAAC,OAAO,EAAE,gBAAgB,CAAC;QAC9C,WAAW,EAAE,GAAG,CAAC,OAAO,EAAE,aAAa,CAAC;QACxC,WAAW,EAAE,GAAG,CAAC,OAAO,EAAE,sBAAsB,CAAC;QACjD,WAAW,EAAE,GAAG,CAAC,OAAO,EAAE,kBAAkB,CAAC;QAC7C,cAAc,EAAE,GAAG,CAAC,OAAO,EAAE,gBAAgB,CAAC;QAC9C,mBAAmB,EAAE,GAAG,CAAC,OAAO,EAAE,qBAAqB,CAAC;QACxD,yBAAyB,EAAE,GAAG,CAAC,OAAO,EAAE,2BAA2B,CAAC;QACpE,aAAa,EAAE,GAAG,CAAC,OAAO,EAAE,eAAe,CAAC;KAC7C,CAAC;IAEF,8EAA8E;IAC9E,MAAM,aAAa,GAAG,EAAE,CAAC,YAAY,GAAG,EAAE,CAAC,YAAY,GAAG,EAAE,CAAC,cAAc,GAAG,EAAE,CAAC,WAAW,GAAG,EAAE,CAAC,WAAW,CAAC;IAC9G,MAAM,YAAY,GAAG,EAAE,CAAC,YAAY,GAAG,EAAE,CAAC,WAAW,CAAC;IACtD,MAAM,eAAe,GAAG,aAAa,KAAK,CAAC;QACzC,CAAC,CAAC,GAAG,CAAC,yBAAyB;QAC/B,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,YAAY,GAAG,aAAa,CAAC,GAAG,GAAG,CAAC,CAAC;IAErD,uEAAuE;IACvE,MAAM,YAAY,GAAG,EAAE,CAAC,cAAc,KAAK,CAAC;QAC1C,CAAC,CAAC,GAAG,CAAC,+BAA+B;QACrC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,mBAAmB,GAAG,EAAE,CAAC,cAAc,CAAC,GAAG,GAAG,CAAC,CAAC;IAEnE,sDAAsD;IACtD,MAAM,kBAAkB,GAAG,EAAE,CAAC,cAAc,CAAC;IAC7C,MAAM,iBAAiB,GAAG,EAAE,CAAC,yBAAyB,GAAG,EAAE,CAAC,aAAa,CAAC;IAC1E,MAAM,YAAY,GAAG,kBAAkB,KAAK,CAAC;QAC3C,CAAC,CAAC,GAAG;QACL,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,kBAAkB,GAAG,iBAAiB,CAAC,GAAG,kBAAkB,CAAC,GAAG,GAAG,CAAC,CAAC,CAAC;IAEnG,4BAA4B;IAC5B,MAAM,OAAO,GAAG,IAAI,CAAC,KAAK,CACxB,eAAe,GAAG,GAAG,GAAG,YAAY,GAAG,IAAI,GAAG,YAAY,GAAG,IAAI,CAClE,CAAC;IAEF,OAAO;QACL,OAAO,EAAE,KAAK,CAAC,OAAO,EAAE,CAAC,EAAE,GAAG,CAAC;QAC/B,eAAe,EAAE,KAAK,CAAC,eAAe,EAAE,CAAC,EAAE,GAAG,CAAC;QAC/C,YAAY,EAAE,KAAK,CAAC,YAAY,EAAE,CAAC,EAAE,GAAG,CAAC;QACzC,YAAY,EAAE,KAAK,CAAC,YAAY,EAAE,CAAC,EAAE,GAAG,CAAC;QACzC,SAAS,EAAE,EAAE;KACd,CAAC;AACJ,CAAC;AAED,MAAM,UAAU,iBAAiB,CAAC,KAAsB;IACtD,IAAI,EAAE,GAAG,kCAAkC,CAAC;IAC5C,EAAE,IAAI,eAAe,KAAK,CAAC,OAAO,UAAU,CAAC;IAC7C,EAAE,IAAI,iDAAiD,CAAC;IACxD,EAAE,IAAI,wBAAwB,KAAK,CAAC,eAAe,aAAa,CAAC;IACjE,EAAE,IAAI,qBAAqB,KAAK,CAAC,YAAY,aAAa,CAAC;IAC3D,EAAE,IAAI,qBAAqB,KAAK,CAAC,YAAY,aAAa,CAAC;IAE3D,EAAE,IAAI,oBAAoB,CAAC;IAC3B,EAAE,IAAI,iCAAiC,CAAC;IACxC,EAAE,IAAI,qBAAqB,KAAK,CAAC,SAAS,CAAC,cAAc,MAAM,CAAC;IAChE,EAAE,IAAI,cAAc,KAAK,CAAC,SAAS,CAAC,eAAe,MAAM,CAAC;IAC1D,EAAE,IAAI,cAAc,KAAK,CAAC,SAAS,CAAC,eAAe,MAAM,CAAC;IAC1D,EAAE,IAAI,qBAAqB,KAAK,CAAC,SAAS,CAAC,YAAY,MAAM,CAAC;IAC9D,EAAE,IAAI,qBAAqB,KAAK,CAAC,SAAS,CAAC,YAAY,MAAM,CAAC;IAC9D,EAAE,IAAI,uBAAuB,KAAK,CAAC,SAAS,CAAC,cAAc,MAAM,CAAC;IAClE,EAAE,IAAI,oBAAoB,KAAK,CAAC,SAAS,CAAC,WAAW,MAAM,CAAC;IAC5D,EAAE,IAAI,oBAAoB,KAAK,CAAC,SAAS,CAAC,WAAW,MAAM,CAAC;IAC5D,EAAE,IAAI,oBAAoB,KAAK,CAAC,SAAS,CAAC,WAAW,MAAM,CAAC;IAC5D,EAAE,IAAI,uBAAuB,KAAK,CAAC,SAAS,CAAC,cAAc,MAAM,CAAC;IAClE,EAAE,IAAI,iBAAiB,KAAK,CAAC,SAAS,CAAC,mBAAmB,MAAM,CAAC;IACjE,EAAE,IAAI,sBAAsB,KAAK,CAAC,SAAS,CAAC,yBAAyB,MAAM,CAAC;IAC5E,EAAE,IAAI,sBAAsB,KAAK,CAAC,SAAS,CAAC,aAAa,MAAM,CAAC;IAEhE,OAAO,EAAE,CAAC;AACZ,CAAC;AAED,SAAS,GAAG,CAAC,OAAsB,EAAE,GAAsB;IACzD,OAAO,OAAO,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,CAAC,EAAE,EAAE,CAAC,GAAG,GAAG,CAAE,CAAC,CAAC,GAAG,CAAY,IAAI,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;AACxE,CAAC;AAED,SAAS,KAAK,CAAC,GAAW,EAAE,GAAW,EAAE,GAAW;IAClD,OAAO,IAAI,CAAC,GAAG,CAAC,GAAG,EAAE,IAAI,CAAC,GAAG,CAAC,GAAG,EAAE,GAAG,CAAC,CAAC,CAAC;AAC3C,CAAC"}

package/dist/core/report-generator.d.ts

@@ -0,0 +1,47 @@
+import { type SessionState } from "../engine/state-machine.js";
+export type DuplicateClass = "VALID_RETRY" | "BLIND_RETRY" | "REDUNDANT_CALL" | "GHOST_WRITE" | "GHOST_CAUGHT" | "GHOST_MISSED";
+export interface AuditReport {
+    sessionId: string;
+    scenarioId: string | null;
+    mode: string;
+    totalCalls: number;
+    upstreamCalls: number;
+    validRetries: number;
+    blindRetries: number;
+    redundantCalls: number;
+    ghostWrites: number;
+    ghostWriteDetections: number;
+    ghostWriteMisses: number;
+    hallucinatedVerifications: number;
+    syntaxCrashes: number;
+    injectedErrors: number;
+    recoveredFromErrors: number;
+    verdict: "PASS" | "FAIL";
+    violations: Violation[];
+    keySummary: KeySummaryEntry[];
+    generatedAt: string;
+}
+export interface Violation {
+    callNumber: number;
+    type: DuplicateClass | "HALLUCINATED_VERIFICATION" | "SYNTAX_CRASH";
+    description: string;
+    idempotencyKey?: string;
+}
+export interface KeySummaryEntry {
+    key: string;
+    state: string;
+    writeCalls: number;
+    lastError: string | null;
+    readBeforeRetry: boolean;
+    ghostCommitted: boolean;
+    classification: DuplicateClass | "INITIAL" | "WRITE_ONLY";
+    stateDiff: {
+        recordExisted: boolean;
+        wasOverwritten: boolean;
+        didAgentReadBefore: boolean;
+        agentGaveUpSilently: boolean;
+    };
+}
+export declare function generateReport(session: SessionState): AuditReport;
+export declare function formatMarkdown(report: AuditReport): string;
+//# sourceMappingURL=report-generator.d.ts.map

package/dist/core/report-generator.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"report-generator.d.ts","sourceRoot":"","sources":["../../src/core/report-generator.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,YAAY,EAAkD,MAAM,4BAA4B,CAAC;AAE/G,MAAM,MAAM,cAAc,GACtB,aAAa,GACb,aAAa,GACb,gBAAgB,GAChB,aAAa,GACb,cAAc,GACd,cAAc,CAAC;AAEnB,MAAM,WAAW,WAAW;IAC1B,SAAS,EAAE,MAAM,CAAC;IAClB,UAAU,EAAE,MAAM,GAAG,IAAI,CAAC;IAC1B,IAAI,EAAE,MAAM,CAAC;IACb,UAAU,EAAE,MAAM,CAAC;IACnB,aAAa,EAAE,MAAM,CAAC;IAEtB,YAAY,EAAE,MAAM,CAAC;IACrB,YAAY,EAAE,MAAM,CAAC;IACrB,cAAc,EAAE,MAAM,CAAC;IACvB,WAAW,EAAE,MAAM,CAAC;IACpB,oBAAoB,EAAE,MAAM,CAAC;IAC7B,gBAAgB,EAAE,MAAM,CAAC;IAEzB,yBAAyB,EAAE,MAAM,CAAC;IAClC,aAAa,EAAE,MAAM,CAAC;IACtB,cAAc,EAAE,MAAM,CAAC;IACvB,mBAAmB,EAAE,MAAM,CAAC;IAC5B,OAAO,EAAE,MAAM,GAAG,MAAM,CAAC;IACzB,UAAU,EAAE,SAAS,EAAE,CAAC;IACxB,UAAU,EAAE,eAAe,EAAE,CAAC;IAC9B,WAAW,EAAE,MAAM,CAAC;CACrB;AAED,MAAM,WAAW,SAAS;IACxB,UAAU,EAAE,MAAM,CAAC;IACnB,IAAI,EAAE,cAAc,GAAG,2BAA2B,GAAG,cAAc,CAAC;IACpE,WAAW,EAAE,MAAM,CAAC;IACpB,cAAc,CAAC,EAAE,MAAM,CAAC;CACzB;AAED,MAAM,WAAW,eAAe;IAC9B,GAAG,EAAE,MAAM,CAAC;IACZ,KAAK,EAAE,MAAM,CAAC;IACd,UAAU,EAAE,MAAM,CAAC;IACnB,SAAS,EAAE,MAAM,GAAG,IAAI,CAAC;IACzB,eAAe,EAAE,OAAO,CAAC;IACzB,cAAc,EAAE,OAAO,CAAC;IACxB,cAAc,EAAE,cAAc,GAAG,SAAS,GAAG,YAAY,CAAC;IAQ1D,SAAS,EAAE;QACT,aAAa,EAAE,OAAO,CAAC;QACvB,cAAc,EAAE,OAAO,CAAC;QACxB,kBAAkB,EAAE,OAAO,CAAC;QAC5B,mBAAmB,EAAE,OAAO,CAAC;KAC9B,CAAC;CACH;AAED,wBAAgB,cAAc,CAAC,OAAO,EAAE,YAAY,GAAG,WAAW,CAqKjE;AA8HD,wBAAgB,cAAc,CAAC,MAAM,EAAE,WAAW,GAAG,MAAM,CA+C1D"}