@reynsu/reactlens-diagnosis-prompts 0.1.0

package/README.md

@@ -0,0 +1,61 @@
+# @reynsu/reactlens-diagnosis-prompts
+
+Diagnosis prompts + `Diagnosis` Zod schema + pure eval-metrics helpers for the reactlens / nativelens AI failure-diagnosis loop.
+
+Published under `@reynsu/` because the `@reactlens` npm scope is unclaimed. Inside the reactlens monorepo this package is still referenced via local imports; a follow-up swap PR will switch reactlens to consume the published version (mirroring the [`@reynsu/nativelens-event-protocol` migration](https://github.com/reynsu/nativelens-protocol)).
+
+## What's in the box
+
+- **`DIAGNOSE_SYSTEM_PROMPT`** — the system message that primes the agent on its job, the schema, and the confidence calibration.
+- **`CLASSIFY_BUG_RUBRIC`** — the long-form classification rubric (`real-bug` / `test-bug` / `flaky` / `env-issue`), including the test-bug vs flaky disambiguation that traps overconfident classifiers.
+- **`DiagnosisSchema`** + `Diagnosis` type — the canonical structured output. Schema-first; consumers parse agent output with `DiagnosisSchema.safeParse(...)`.
+- **`buildUserMessage(failure)`** — pure user-message builder (title, spec, error fence, component snapshot, truncation). Golden-testable.
+- **`degradedDiagnosis(gitCtx?)`** — fallback when the agent exhausts retries. Deliberately emits `env-issue` / `low` so the operator notices.
+- **`TruthSchema`, `parseTruth`, `compareToTruth`, `aggregateMetrics`** — pure scoring helpers for the eval harness. `falseConfidenceRate` is surfaced explicitly as the primary regression signal.
+
+## What's NOT in the box (kept per-host)
+
+- Agent runner implementations (Anthropic SDK, OpenAI, etc.) — consumers provide their own.
+- Sandboxing logic for the eval pipeline (loading `component.tsx` / `spec.ts` from disk).
+- Git context gathering (host-specific paths + tooling).
+- The eval cases themselves (reactlens ships its under `tests/diagnostic-eval/cases/`; nativelens will ship its own).
+
+## Usage
+
+```ts
+import {
+  DIAGNOSE_SYSTEM_PROMPT,
+  CLASSIFY_BUG_RUBRIC,
+  DiagnosisSchema,
+  buildUserMessage,
+  degradedDiagnosis,
+  type FailedTest,
+  type Diagnosis,
+} from '@reynsu/reactlens-diagnosis-prompts';
+
+const failure: FailedTest = {
+  testId: 'cart-flow:tap-add',
+  testTitle: 'cart shows declined banner',
+  specFile: '/abs/path/cart.spec.ts',
+  errorMessage: 'Timed out waiting for [data-testid="declined-banner"]',
+  componentSnapshot: { /* host-shaped tree */ },
+};
+
+const userMessage = buildUserMessage(failure);
+// Feed [DIAGNOSE_SYSTEM_PROMPT, CLASSIFY_BUG_RUBRIC] as system messages and
+// `userMessage` as the user turn to whichever LLM-runner your host wires up.
+// Parse the agent's final message with DiagnosisSchema.
+```
+
+## Versioning
+
+Pre-1.0: minor bumps may include breaking schema changes; pin to the exact minor in dependents. The prompts themselves change without bumps (they're tuned against the eval set); only `Diagnosis` schema changes are semver-meaningful.
+
+## Development
+
+```bash
+pnpm install
+pnpm test
+pnpm typecheck
+pnpm build
+```

package/dist/diagnosis.d.ts

@@ -0,0 +1,247 @@
+import { z } from 'zod';
+export declare const ClassificationSchema: z.ZodEnum<["real-bug", "test-bug", "flaky", "env-issue"]>;
+export type Classification = z.infer<typeof ClassificationSchema>;
+export declare const ConfidenceSchema: z.ZodEnum<["high", "medium", "low"]>;
+export type Confidence = z.infer<typeof ConfidenceSchema>;
+export declare const PatchEntrySchema: z.ZodObject<{
+    file: z.ZodString;
+    oldStr: z.ZodString;
+    newStr: z.ZodString;
+    rationale: z.ZodString;
+}, "strip", z.ZodTypeAny, {
+    file: string;
+    oldStr: string;
+    newStr: string;
+    rationale: string;
+}, {
+    file: string;
+    oldStr: string;
+    newStr: string;
+    rationale: string;
+}>;
+export type PatchEntry = z.infer<typeof PatchEntrySchema>;
+export declare const GitContextEntrySchema: z.ZodObject<{
+    sha: z.ZodString;
+    author: z.ZodString;
+    date: z.ZodString;
+    message: z.ZodString;
+}, "strip", z.ZodTypeAny, {
+    message: string;
+    sha: string;
+    author: string;
+    date: string;
+}, {
+    message: string;
+    sha: string;
+    author: string;
+    date: string;
+}>;
+export type GitContextEntry = z.infer<typeof GitContextEntrySchema>;
+export declare const GitContextSchema: z.ZodObject<{
+    componentLastChanged: z.ZodOptional<z.ZodObject<{
+        sha: z.ZodString;
+        author: z.ZodString;
+        date: z.ZodString;
+        message: z.ZodString;
+    }, "strip", z.ZodTypeAny, {
+        message: string;
+        sha: string;
+        author: string;
+        date: string;
+    }, {
+        message: string;
+        sha: string;
+        author: string;
+        date: string;
+    }>>;
+    specLastChanged: z.ZodOptional<z.ZodObject<{
+        sha: z.ZodString;
+        author: z.ZodString;
+        date: z.ZodString;
+        message: z.ZodString;
+    }, "strip", z.ZodTypeAny, {
+        message: string;
+        sha: string;
+        author: string;
+        date: string;
+    }, {
+        message: string;
+        sha: string;
+        author: string;
+        date: string;
+    }>>;
+}, "strip", z.ZodTypeAny, {
+    componentLastChanged?: {
+        message: string;
+        sha: string;
+        author: string;
+        date: string;
+    } | undefined;
+    specLastChanged?: {
+        message: string;
+        sha: string;
+        author: string;
+        date: string;
+    } | undefined;
+}, {
+    componentLastChanged?: {
+        message: string;
+        sha: string;
+        author: string;
+        date: string;
+    } | undefined;
+    specLastChanged?: {
+        message: string;
+        sha: string;
+        author: string;
+        date: string;
+    } | undefined;
+}>;
+export type GitContext = z.infer<typeof GitContextSchema>;
+export declare const DiagnosisSchema: z.ZodObject<{
+    classification: z.ZodEnum<["real-bug", "test-bug", "flaky", "env-issue"]>;
+    confidence: z.ZodEnum<["high", "medium", "low"]>;
+    rootCause: z.ZodString;
+    evidence: z.ZodArray<z.ZodString, "many">;
+    suggestedFix: z.ZodString;
+    patch: z.ZodOptional<z.ZodArray<z.ZodObject<{
+        file: z.ZodString;
+        oldStr: z.ZodString;
+        newStr: z.ZodString;
+        rationale: z.ZodString;
+    }, "strip", z.ZodTypeAny, {
+        file: string;
+        oldStr: string;
+        newStr: string;
+        rationale: string;
+    }, {
+        file: string;
+        oldStr: string;
+        newStr: string;
+        rationale: string;
+    }>, "many">>;
+    gitContext: z.ZodOptional<z.ZodObject<{
+        componentLastChanged: z.ZodOptional<z.ZodObject<{
+            sha: z.ZodString;
+            author: z.ZodString;
+            date: z.ZodString;
+            message: z.ZodString;
+        }, "strip", z.ZodTypeAny, {
+            message: string;
+            sha: string;
+            author: string;
+            date: string;
+        }, {
+            message: string;
+            sha: string;
+            author: string;
+            date: string;
+        }>>;
+        specLastChanged: z.ZodOptional<z.ZodObject<{
+            sha: z.ZodString;
+            author: z.ZodString;
+            date: z.ZodString;
+            message: z.ZodString;
+        }, "strip", z.ZodTypeAny, {
+            message: string;
+            sha: string;
+            author: string;
+            date: string;
+        }, {
+            message: string;
+            sha: string;
+            author: string;
+            date: string;
+        }>>;
+    }, "strip", z.ZodTypeAny, {
+        componentLastChanged?: {
+            message: string;
+            sha: string;
+            author: string;
+            date: string;
+        } | undefined;
+        specLastChanged?: {
+            message: string;
+            sha: string;
+            author: string;
+            date: string;
+        } | undefined;
+    }, {
+        componentLastChanged?: {
+            message: string;
+            sha: string;
+            author: string;
+            date: string;
+        } | undefined;
+        specLastChanged?: {
+            message: string;
+            sha: string;
+            author: string;
+            date: string;
+        } | undefined;
+    }>>;
+}, "strip", z.ZodTypeAny, {
+    classification: "real-bug" | "test-bug" | "flaky" | "env-issue";
+    confidence: "high" | "medium" | "low";
+    rootCause: string;
+    evidence: string[];
+    suggestedFix: string;
+    patch?: {
+        file: string;
+        oldStr: string;
+        newStr: string;
+        rationale: string;
+    }[] | undefined;
+    gitContext?: {
+        componentLastChanged?: {
+            message: string;
+            sha: string;
+            author: string;
+            date: string;
+        } | undefined;
+        specLastChanged?: {
+            message: string;
+            sha: string;
+            author: string;
+            date: string;
+        } | undefined;
+    } | undefined;
+}, {
+    classification: "real-bug" | "test-bug" | "flaky" | "env-issue";
+    confidence: "high" | "medium" | "low";
+    rootCause: string;
+    evidence: string[];
+    suggestedFix: string;
+    patch?: {
+        file: string;
+        oldStr: string;
+        newStr: string;
+        rationale: string;
+    }[] | undefined;
+    gitContext?: {
+        componentLastChanged?: {
+            message: string;
+            sha: string;
+            author: string;
+            date: string;
+        } | undefined;
+        specLastChanged?: {
+            message: string;
+            sha: string;
+            author: string;
+            date: string;
+        } | undefined;
+    } | undefined;
+}>;
+export type Diagnosis = z.infer<typeof DiagnosisSchema>;
+export type FailedTest = {
+    testId: string;
+    testTitle: string;
+    specFile: string;
+    errorMessage?: string;
+    componentFile?: string;
+    componentSnapshot?: unknown;
+};
+export declare function buildUserMessage(failure: FailedTest): string;
+export declare function degradedDiagnosis(gitContext?: GitContext): Diagnosis;
+//# sourceMappingURL=diagnosis.d.ts.map

package/dist/diagnosis.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"diagnosis.d.ts","sourceRoot":"","sources":["../src/diagnosis.ts"],"names":[],"mappings":"AAQA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AAExB,eAAO,MAAM,oBAAoB,2DAAyD,CAAC;AAC3F,MAAM,MAAM,cAAc,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,oBAAoB,CAAC,CAAC;AAElE,eAAO,MAAM,gBAAgB,sCAAoC,CAAC;AAClE,MAAM,MAAM,UAAU,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,gBAAgB,CAAC,CAAC;AAE1D,eAAO,MAAM,gBAAgB;;;;;;;;;;;;;;;EAK3B,CAAC;AACH,MAAM,MAAM,UAAU,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,gBAAgB,CAAC,CAAC;AAE1D,eAAO,MAAM,qBAAqB;;;;;;;;;;;;;;;EAKhC,CAAC;AACH,MAAM,MAAM,eAAe,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,qBAAqB,CAAC,CAAC;AAEpE,eAAO,MAAM,gBAAgB;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;EAG3B,CAAC;AACH,MAAM,MAAM,UAAU,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,gBAAgB,CAAC,CAAC;AAE1D,eAAO,MAAM,eAAe;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;EAQ1B,CAAC;AACH,MAAM,MAAM,SAAS,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,eAAe,CAAC,CAAC;AAKxD,MAAM,MAAM,UAAU,GAAG;IACvB,MAAM,EAAE,MAAM,CAAC;IACf,SAAS,EAAE,MAAM,CAAC;IAClB,QAAQ,EAAE,MAAM,CAAC;IACjB,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,iBAAiB,CAAC,EAAE,OAAO,CAAC;CAC7B,CAAC;AAKF,wBAAgB,gBAAgB,CAAC,OAAO,EAAE,UAAU,GAAG,MAAM,CA4B5D;AAKD,wBAAgB,iBAAiB,CAAC,UAAU,GAAE,UAAe,GAAG,SAAS,CASxE"}

package/dist/diagnosis.js

@@ -0,0 +1,82 @@
+// Canonical Diagnosis output shape and the pure helpers that surround it
+// (user-message builder, degraded fallback). Schema-first: `DiagnosisSchema`
+// is the source of truth — `Diagnosis` is `z.infer<typeof DiagnosisSchema>`.
+//
+// Consumers (reactlens, nativelens) provide their own agent runner, supply
+// per-host fields on FailedTest (e.g. a `componentSnapshot` shape that fits
+// their probe), and treat the returned Diagnosis as opaque-but-typed.
+import { z } from 'zod';
+export const ClassificationSchema = z.enum(['real-bug', 'test-bug', 'flaky', 'env-issue']);
+export const ConfidenceSchema = z.enum(['high', 'medium', 'low']);
+export const PatchEntrySchema = z.object({
+    file: z.string(),
+    oldStr: z.string(),
+    newStr: z.string(),
+    rationale: z.string(),
+});
+export const GitContextEntrySchema = z.object({
+    sha: z.string(),
+    author: z.string(),
+    date: z.string(),
+    message: z.string(),
+});
+export const GitContextSchema = z.object({
+    componentLastChanged: GitContextEntrySchema.optional(),
+    specLastChanged: GitContextEntrySchema.optional(),
+});
+export const DiagnosisSchema = z.object({
+    classification: ClassificationSchema,
+    confidence: ConfidenceSchema,
+    rootCause: z.string().min(1),
+    evidence: z.array(z.string()),
+    suggestedFix: z.string(),
+    patch: z.array(PatchEntrySchema).optional(),
+    gitContext: GitContextSchema.optional(),
+});
+// Builds the user-message string fed to the diagnosis agent alongside the
+// system prompts. Pure — produces the exact bytes that go on the wire so
+// the prompt surface can be diffed and golden-tested.
+export function buildUserMessage(failure) {
+    const lines = [];
+    lines.push(`# Failure to diagnose`);
+    lines.push(``);
+    lines.push(`Test: ${failure.testTitle}`);
+    lines.push(`Spec: ${failure.specFile}`);
+    if (failure.componentFile !== undefined)
+        lines.push(`Component (probable): ${failure.componentFile}`);
+    if (failure.errorMessage !== undefined) {
+        lines.push(``);
+        lines.push(`# Error`);
+        lines.push('```');
+        lines.push(failure.errorMessage);
+        lines.push('```');
+    }
+    if (failure.componentSnapshot !== undefined) {
+        lines.push(``);
+        lines.push(`# Component snapshot at failure`);
+        lines.push('```json');
+        const snippet = JSON.stringify(failure.componentSnapshot, null, 2);
+        // Truncate enormous trees to keep token usage sane. 30k chars is the
+        // empirically-tuned ceiling — beyond it the agent's classification
+        // accuracy starts to degrade in eval.
+        lines.push(snippet.length > 30_000 ? snippet.slice(0, 30_000) + '\n…(truncated)' : snippet);
+        lines.push('```');
+    }
+    lines.push(``);
+    lines.push(`Read the spec, the component, and any other context you need. Output a single JSON object matching the Diagnosis schema as the FINAL message.`);
+    return lines.join('\n');
+}
+// Fallback when the agent loop exhausts retries without producing a valid
+// Diagnosis. We DELIBERATELY return `env-issue` / `low` so the operator
+// notices: this is the calibration failure mode Principle 2 forbids.
+export function degradedDiagnosis(gitContext = {}) {
+    return {
+        classification: 'env-issue',
+        confidence: 'low',
+        rootCause: 'diagnosis agent failed to produce a valid output',
+        evidence: ['agent returned non-JSON or schema-mismatched output twice'],
+        suggestedFix: 'rerun the diagnosis with --verbose; if it persists, file an issue with the trace',
+        ...(Object.keys(gitContext).length > 0 ? { gitContext } : {}),
+    };
+}
+//# sourceMappingURL=diagnosis.js.map

package/dist/diagnosis.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"diagnosis.js","sourceRoot":"","sources":["../src/diagnosis.ts"],"names":[],"mappings":"AAAA,yEAAyE;AACzE,6EAA6E;AAC7E,6EAA6E;AAC7E,EAAE;AACF,2EAA2E;AAC3E,4EAA4E;AAC5E,sEAAsE;AAEtE,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AAExB,MAAM,CAAC,MAAM,oBAAoB,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,UAAU,EAAE,UAAU,EAAE,OAAO,EAAE,WAAW,CAAC,CAAC,CAAC;AAG3F,MAAM,CAAC,MAAM,gBAAgB,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,MAAM,EAAE,QAAQ,EAAE,KAAK,CAAC,CAAC,CAAC;AAGlE,MAAM,CAAC,MAAM,gBAAgB,GAAG,CAAC,CAAC,MAAM,CAAC;IACvC,IAAI,EAAE,CAAC,CAAC,MAAM,EAAE;IAChB,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE;IAClB,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE;IAClB,SAAS,EAAE,CAAC,CAAC,MAAM,EAAE;CACtB,CAAC,CAAC;AAGH,MAAM,CAAC,MAAM,qBAAqB,GAAG,CAAC,CAAC,MAAM,CAAC;IAC5C,GAAG,EAAE,CAAC,CAAC,MAAM,EAAE;IACf,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE;IAClB,IAAI,EAAE,CAAC,CAAC,MAAM,EAAE;IAChB,OAAO,EAAE,CAAC,CAAC,MAAM,EAAE;CACpB,CAAC,CAAC;AAGH,MAAM,CAAC,MAAM,gBAAgB,GAAG,CAAC,CAAC,MAAM,CAAC;IACvC,oBAAoB,EAAE,qBAAqB,CAAC,QAAQ,EAAE;IACtD,eAAe,EAAE,qBAAqB,CAAC,QAAQ,EAAE;CAClD,CAAC,CAAC;AAGH,MAAM,CAAC,MAAM,eAAe,GAAG,CAAC,CAAC,MAAM,CAAC;IACtC,cAAc,EAAE,oBAAoB;IACpC,UAAU,EAAE,gBAAgB;IAC5B,SAAS,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC;IAC5B,QAAQ,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC;IAC7B,YAAY,EAAE,CAAC,CAAC,MAAM,EAAE;IACxB,KAAK,EAAE,CAAC,CAAC,KAAK,CAAC,gBAAgB,CAAC,CAAC,QAAQ,EAAE;IAC3C,UAAU,EAAE,gBAAgB,CAAC,QAAQ,EAAE;CACxC,CAAC,CAAC;AAeH,0EAA0E;AAC1E,yEAAyE;AACzE,sDAAsD;AACtD,MAAM,UAAU,gBAAgB,CAAC,OAAmB;IAClD,MAAM,KAAK,GAAa,EAAE,CAAC;IAC3B,KAAK,CAAC,IAAI,CAAC,uBAAuB,CAAC,CAAC;IACpC,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACf,KAAK,CAAC,IAAI,CAAC,SAAS,OAAO,CAAC,SAAS,EAAE,CAAC,CAAC;IACzC,KAAK,CAAC,IAAI,CAAC,SAAS,OAAO,CAAC,QAAQ,EAAE,CAAC,CAAC;IACxC,IAAI,OAAO,CAAC,aAAa,KAAK,SAAS;QAAE,KAAK,CAAC,IAAI,CAAC,yBAAyB,OAAO,CAAC,aAAa,EAAE,CAAC,CAAC;IACtG,IAAI,OAAO,CAAC,YAAY,KAAK,SAAS,EAAE,CAAC;QACvC,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QACf,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;QACtB,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;QAClB,KAAK,CAAC,IAAI,CAAC,OAAO,CAAC,YAAY,CAAC,CAAC;QACjC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;IACpB,CAAC;IACD,IAAI,OAAO,CAAC,iBAAiB,KAAK,SAAS,EAAE,CAAC;QAC5C,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QACf,KAAK,CAAC,IAAI,CAAC,iCAAiC,CAAC,CAAC;QAC9C,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;QACtB,MAAM,OAAO,GAAG,IAAI,CAAC,SAAS,CAAC,OAAO,CAAC,iBAAiB,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC;QACnE,qEAAqE;QACrE,mEAAmE;QACnE,sCAAsC;QACtC,KAAK,CAAC,IAAI,CAAC,OAAO,CAAC,MAAM,GAAG,MAAM,CAAC,CAAC,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,MAAM,CAAC,GAAG,gBAAgB,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC;QAC5F,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;IACpB,CAAC;IACD,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACf,KAAK,CAAC,IAAI,CAAC,+IAA+I,CAAC,CAAC;IAC5J,OAAO,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AAC1B,CAAC;AAED,0EAA0E;AAC1E,wEAAwE;AACxE,qEAAqE;AACrE,MAAM,UAAU,iBAAiB,CAAC,aAAyB,EAAE;IAC3D,OAAO;QACL,cAAc,EAAE,WAAW;QAC3B,UAAU,EAAE,KAAK;QACjB,SAAS,EAAE,kDAAkD;QAC7D,QAAQ,EAAE,CAAC,2DAA2D,CAAC;QACvE,YAAY,EAAE,kFAAkF;QAChG,GAAG,CAAC,MAAM,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,EAAE,UAAU,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;KAC9D,CAAC;AACJ,CAAC"}

package/dist/eval-metrics.d.ts

@@ -0,0 +1,40 @@
+import { z } from 'zod';
+import { type Classification, type Diagnosis } from './diagnosis.js';
+export declare const TruthSchema: z.ZodObject<{
+    expectedClassification: z.ZodEnum<["real-bug", "test-bug", "flaky", "env-issue"]>;
+    minimumConfidence: z.ZodEnum<["high", "medium", "low"]>;
+    category: z.ZodOptional<z.ZodString>;
+    notes: z.ZodOptional<z.ZodString>;
+}, "strip", z.ZodTypeAny, {
+    expectedClassification: "real-bug" | "test-bug" | "flaky" | "env-issue";
+    minimumConfidence: "high" | "medium" | "low";
+    category?: string | undefined;
+    notes?: string | undefined;
+}, {
+    expectedClassification: "real-bug" | "test-bug" | "flaky" | "env-issue";
+    minimumConfidence: "high" | "medium" | "low";
+    category?: string | undefined;
+    notes?: string | undefined;
+}>;
+export type Truth = z.infer<typeof TruthSchema>;
+export type CaseResult = {
+    name: string;
+    expected: Truth;
+    actual: Diagnosis;
+    correct: boolean;
+    falseConfidence: boolean;
+};
+export declare function parseTruth(raw: string): Truth;
+export declare function compareToTruth(actual: Diagnosis, expected: Truth, name: string): CaseResult;
+export type EvalMetrics = {
+    total: number;
+    correct: number;
+    accuracy: number;
+    falseConfidenceRate: number;
+    perClassification: Record<Classification, {
+        total: number;
+        correct: number;
+    }>;
+};
+export declare function aggregateMetrics(results: CaseResult[]): EvalMetrics;
+//# sourceMappingURL=eval-metrics.d.ts.map

package/dist/eval-metrics.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"eval-metrics.d.ts","sourceRoot":"","sources":["../src/eval-metrics.ts"],"names":[],"mappings":"AAQA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AACxB,OAAO,EAA0C,KAAK,cAAc,EAAE,KAAK,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAE7G,eAAO,MAAM,WAAW;;;;;;;;;;;;;;;EAKtB,CAAC;AACH,MAAM,MAAM,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,OAAO,WAAW,CAAC,CAAC;AAEhD,MAAM,MAAM,UAAU,GAAG;IACvB,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,KAAK,CAAC;IAChB,MAAM,EAAE,SAAS,CAAC;IAClB,OAAO,EAAE,OAAO,CAAC;IAIjB,eAAe,EAAE,OAAO,CAAC;CAC1B,CAAC;AAEF,wBAAgB,UAAU,CAAC,GAAG,EAAE,MAAM,GAAG,KAAK,CAE7C;AAED,wBAAgB,cAAc,CAAC,MAAM,EAAE,SAAS,EAAE,QAAQ,EAAE,KAAK,EAAE,IAAI,EAAE,MAAM,GAAG,UAAU,CAS3F;AAED,MAAM,MAAM,WAAW,GAAG;IACxB,KAAK,EAAE,MAAM,CAAC;IACd,OAAO,EAAE,MAAM,CAAC;IAChB,QAAQ,EAAE,MAAM,CAAC;IAEjB,mBAAmB,EAAE,MAAM,CAAC;IAG5B,iBAAiB,EAAE,MAAM,CAAC,cAAc,EAAE;QAAE,KAAK,EAAE,MAAM,CAAC;QAAC,OAAO,EAAE,MAAM,CAAA;KAAE,CAAC,CAAC;CAC/E,CAAC;AAEF,wBAAgB,gBAAgB,CAAC,OAAO,EAAE,UAAU,EAAE,GAAG,WAAW,CAsBnE"}

package/dist/eval-metrics.js

@@ -0,0 +1,53 @@
+// Pure scoring helpers for the diagnostic-eval harness. No fs, no agent —
+// consumers (reactlens, nativelens) drive their own per-host case loading
+// and call into these for grading.
+//
+// `falseConfidenceRate` is surfaced explicitly because Principle 2's
+// implicit cap on it (>=95% high-confidence accuracy) is the most useful
+// regression signal in eval reports.
+import { z } from 'zod';
+import { ClassificationSchema, ConfidenceSchema } from './diagnosis.js';
+export const TruthSchema = z.object({
+    expectedClassification: ClassificationSchema,
+    minimumConfidence: ConfidenceSchema,
+    category: z.string().optional(),
+    notes: z.string().optional(),
+});
+export function parseTruth(raw) {
+    return TruthSchema.parse(JSON.parse(raw));
+}
+export function compareToTruth(actual, expected, name) {
+    const correct = actual.classification === expected.expectedClassification;
+    return {
+        name,
+        expected,
+        actual,
+        correct,
+        falseConfidence: !correct && actual.confidence === 'high',
+    };
+}
+export function aggregateMetrics(results) {
+    const total = results.length;
+    const correct = results.filter((r) => r.correct).length;
+    const falseConfident = results.filter((r) => r.falseConfidence).length;
+    const perClassification = {
+        'real-bug': { total: 0, correct: 0 },
+        'test-bug': { total: 0, correct: 0 },
+        flaky: { total: 0, correct: 0 },
+        'env-issue': { total: 0, correct: 0 },
+    };
+    for (const r of results) {
+        const bucket = perClassification[r.expected.expectedClassification];
+        bucket.total += 1;
+        if (r.correct)
+            bucket.correct += 1;
+    }
+    return {
+        total,
+        correct,
+        accuracy: total === 0 ? 0 : correct / total,
+        falseConfidenceRate: total === 0 ? 0 : falseConfident / total,
+        perClassification,
+    };
+}
+//# sourceMappingURL=eval-metrics.js.map

package/dist/eval-metrics.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"eval-metrics.js","sourceRoot":"","sources":["../src/eval-metrics.ts"],"names":[],"mappings":"AAAA,0EAA0E;AAC1E,0EAA0E;AAC1E,mCAAmC;AACnC,EAAE;AACF,qEAAqE;AACrE,yEAAyE;AACzE,qCAAqC;AAErC,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAC;AACxB,OAAO,EAAE,oBAAoB,EAAE,gBAAgB,EAAuC,MAAM,gBAAgB,CAAC;AAE7G,MAAM,CAAC,MAAM,WAAW,GAAG,CAAC,CAAC,MAAM,CAAC;IAClC,sBAAsB,EAAE,oBAAoB;IAC5C,iBAAiB,EAAE,gBAAgB;IACnC,QAAQ,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE;IAC/B,KAAK,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,QAAQ,EAAE;CAC7B,CAAC,CAAC;AAcH,MAAM,UAAU,UAAU,CAAC,GAAW;IACpC,OAAO,WAAW,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC;AAC5C,CAAC;AAED,MAAM,UAAU,cAAc,CAAC,MAAiB,EAAE,QAAe,EAAE,IAAY;IAC7E,MAAM,OAAO,GAAG,MAAM,CAAC,cAAc,KAAK,QAAQ,CAAC,sBAAsB,CAAC;IAC1E,OAAO;QACL,IAAI;QACJ,QAAQ;QACR,MAAM;QACN,OAAO;QACP,eAAe,EAAE,CAAC,OAAO,IAAI,MAAM,CAAC,UAAU,KAAK,MAAM;KAC1D,CAAC;AACJ,CAAC;AAaD,MAAM,UAAU,gBAAgB,CAAC,OAAqB;IACpD,MAAM,KAAK,GAAG,OAAO,CAAC,MAAM,CAAC;IAC7B,MAAM,OAAO,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,MAAM,CAAC;IACxD,MAAM,cAAc,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,MAAM,CAAC;IACvE,MAAM,iBAAiB,GAAqC;QAC1D,UAAU,EAAE,EAAE,KAAK,EAAE,CAAC,EAAE,OAAO,EAAE,CAAC,EAAE;QACpC,UAAU,EAAE,EAAE,KAAK,EAAE,CAAC,EAAE,OAAO,EAAE,CAAC,EAAE;QACpC,KAAK,EAAE,EAAE,KAAK,EAAE,CAAC,EAAE,OAAO,EAAE,CAAC,EAAE;QAC/B,WAAW,EAAE,EAAE,KAAK,EAAE,CAAC,EAAE,OAAO,EAAE,CAAC,EAAE;KACtC,CAAC;IACF,KAAK,MAAM,CAAC,IAAI,OAAO,EAAE,CAAC;QACxB,MAAM,MAAM,GAAG,iBAAiB,CAAC,CAAC,CAAC,QAAQ,CAAC,sBAAsB,CAAC,CAAC;QACpE,MAAM,CAAC,KAAK,IAAI,CAAC,CAAC;QAClB,IAAI,CAAC,CAAC,OAAO;YAAE,MAAM,CAAC,OAAO,IAAI,CAAC,CAAC;IACrC,CAAC;IACD,OAAO;QACL,KAAK;QACL,OAAO;QACP,QAAQ,EAAE,KAAK,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,OAAO,GAAG,KAAK;QAC3C,mBAAmB,EAAE,KAAK,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,cAAc,GAAG,KAAK;QAC7D,iBAAiB;KAClB,CAAC;AACJ,CAAC"}

package/dist/index.d.ts

@@ -0,0 +1,4 @@
+export { DIAGNOSE_SYSTEM_PROMPT, CLASSIFY_BUG_RUBRIC } from './prompts.js';
+export { ClassificationSchema, ConfidenceSchema, PatchEntrySchema, GitContextEntrySchema, GitContextSchema, DiagnosisSchema, type Classification, type Confidence, type PatchEntry, type GitContextEntry, type GitContext, type Diagnosis, type FailedTest, buildUserMessage, degradedDiagnosis, } from './diagnosis.js';
+export { TruthSchema, type Truth, type CaseResult, type EvalMetrics, parseTruth, compareToTruth, aggregateMetrics, } from './eval-metrics.js';
+//# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAEA,OAAO,EAAE,sBAAsB,EAAE,mBAAmB,EAAE,MAAM,cAAc,CAAC;AAE3E,OAAO,EAEL,oBAAoB,EACpB,gBAAgB,EAChB,gBAAgB,EAChB,qBAAqB,EACrB,gBAAgB,EAChB,eAAe,EAEf,KAAK,cAAc,EACnB,KAAK,UAAU,EACf,KAAK,UAAU,EACf,KAAK,eAAe,EACpB,KAAK,UAAU,EACf,KAAK,SAAS,EACd,KAAK,UAAU,EAEf,gBAAgB,EAChB,iBAAiB,GAClB,MAAM,gBAAgB,CAAC;AAExB,OAAO,EACL,WAAW,EACX,KAAK,KAAK,EACV,KAAK,UAAU,EACf,KAAK,WAAW,EAChB,UAAU,EACV,cAAc,EACd,gBAAgB,GACjB,MAAM,mBAAmB,CAAC"}

package/dist/index.js

@@ -0,0 +1,9 @@
+// Public surface — keep narrow. Consumers should import from the root only.
+export { DIAGNOSE_SYSTEM_PROMPT, CLASSIFY_BUG_RUBRIC } from './prompts.js';
+export { 
+// Schemas
+ClassificationSchema, ConfidenceSchema, PatchEntrySchema, GitContextEntrySchema, GitContextSchema, DiagnosisSchema, 
+// Pure helpers
+buildUserMessage, degradedDiagnosis, } from './diagnosis.js';
+export { TruthSchema, parseTruth, compareToTruth, aggregateMetrics, } from './eval-metrics.js';
+//# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,4EAA4E;AAE5E,OAAO,EAAE,sBAAsB,EAAE,mBAAmB,EAAE,MAAM,cAAc,CAAC;AAE3E,OAAO;AACL,UAAU;AACV,oBAAoB,EACpB,gBAAgB,EAChB,gBAAgB,EAChB,qBAAqB,EACrB,gBAAgB,EAChB,eAAe;AASf,eAAe;AACf,gBAAgB,EAChB,iBAAiB,GAClB,MAAM,gBAAgB,CAAC;AAExB,OAAO,EACL,WAAW,EAIX,UAAU,EACV,cAAc,EACd,gBAAgB,GACjB,MAAM,mBAAmB,CAAC"}

package/dist/prompts.d.ts

@@ -0,0 +1,3 @@
+export declare const DIAGNOSE_SYSTEM_PROMPT = "# reactlens \u2014 diagnose a Playwright failure\n\nYou are the diagnosis agent for reactlens. A Playwright test just failed. You have access to:\n\n- The failing **spec** (`specPath` below) \u2014 read it.\n- The **component** under test (`componentPath`) \u2014 read it. This is the code the user wrote.\n- The **trace summary** \u2014 Playwright's error message + locator chain.\n- The **component snapshot at failure** \u2014 the React tree at the moment the assertion failed, including props and hooks. This is the unique signal you have that other tools don't.\n- Optional **git context** for both files: who changed what most recently, and when.\n\nYou have access to the `Read`, `Glob`, `Grep`, and `Bash` tools to look around. Use them deliberately \u2014 every read costs the user money.\n\n## Your job\n\nOutput a single JSON object matching this schema (plain JSON; no markdown fence):\n\n```json\n{\n  \"classification\": \"real-bug\" | \"test-bug\" | \"flaky\" | \"env-issue\",\n  \"confidence\": \"high\" | \"medium\" | \"low\",\n  \"rootCause\": \"one sentence\",\n  \"evidence\": [\"bullet 1\", \"bullet 2\", ...],\n  \"suggestedFix\": \"human-readable description of what to change\",\n  \"patch\": [{\n    \"file\": \"path\",\n    \"oldStr\": \"exact text to replace\",\n    \"newStr\": \"replacement text\",\n    \"rationale\": \"why this fixes it\"\n  }]\n}\n```\n\n`patch` is OPTIONAL \u2014 only include it when you can produce a concrete edit you're confident about. If you can't, omit it and say what additional information would let you produce one (in `suggestedFix`).\n\n## Classification rubric\n\nSee `classify-bug.md` (you should consult it). Headline rules:\n\n- **`real-bug`** \u2014 code regressed; spec is correct. The behavior the spec asserted was correct yesterday and is wrong today. Best signal: spec hasn't changed, component recently changed.\n- **`test-bug`** \u2014 code is fine; spec is stale or wrong. Best signal: component hasn't changed, spec recently changed; or the spec asserts something the source code never actually does.\n- **`flaky`** \u2014 neither code nor spec changed; the failure is non-deterministic. Best signal: timing-related error (waiting for element, race), no recent commits to either file.\n- **`env-issue`** \u2014 the failure is infrastructure (port conflict, missing env var, browser binary missing). Best signal: error mentions ports, env vars, browser launch, network unreachable.\n\n## Confidence calibration (CRITICAL)\n\nWe measure your accuracy at each confidence level. Do not lie.\n\n- **`high`** \u2014 you have direct evidence in the snapshot AND/OR a recent commit that explains the behavior. Example: \"the snapshot shows `cvv: '12'` \u2014 only 2 chars \u2014 but the component schema requires `min(3)`. The validation error visible in the snapshot is what the spec is asserting against.\"\n- **`medium`** \u2014 strong but indirect signal. Eg \"the spec hasn't changed in 3 weeks, the component changed yesterday \u2014 but I haven't found the actual bug in the diff.\"\n- **`low`** \u2014 informed guess. Use this freely; it's better than a wrong `high`.\n\nIf you cannot produce evidence, use `low`. Never inflate.\n\n## What to read\n\nIn order:\n1. The component snapshot at failure (you'll get this in the user message inline). Note components, props, hook values.\n2. The spec (`Read specPath`).\n3. The component (`Read componentPath`).\n4. Anything the spec or component imports that's relevant (`Read` selectively).\n5. `git log -p` on either file if you need to see what changed recently.\n\nDo not read the entire codebase. Stop reading when you have enough to commit to an answer.\n\n## Output format\n\nFinal message MUST be only the JSON object \u2014 no prose, no markdown, no code fence. Earlier messages can include reasoning. The system parses your last message as JSON.\n";
+export declare const CLASSIFY_BUG_RUBRIC = "# reactlens \u2014 bug classification rubric\n\nThis document defines, with examples, when to choose each classification. Diagnosis agent: consult this when in doubt. Calibration is measured against the eval set in `tests/diagnostic-eval/cases/`.\n\n## `real-bug`\n\n**Definition:** the code regressed and the spec is correctly asserting the prior, correct behavior.\n\n**Strongest signals:**\n- Component file has a recent commit (today, this week) and the spec hasn't changed in much longer.\n- The component snapshot at failure shows props/state inconsistent with the source's invariants \u2014 eg the source says `if (data.length === 0) return Empty` but the snapshot shows `data.length === 0` AND the rendered subtree doesn't include `<Empty />`.\n- The snapshot shows a thrown error in a render path the spec was exercising.\n\n**Examples:**\n\n> Spec: \"checkout shows declined banner for cards starting with 4000\".\n> Component snapshot: status='success' (not 'declined') after submitting a 4000-card.\n> Component diff (last commit): the developer changed the API check from `startsWith('4000')` to `startsWith('5000')`.\n> \u2192 `real-bug`, `high` confidence. Patch: revert the prefix check.\n\n> Spec: \"dashboard shows empty state when there are no orders\".\n> Component snapshot: orders=[] but `<empty />` is not in the tree.\n> Component: the developer added an `early-return null` above the empty check that fires unconditionally.\n> \u2192 `real-bug`, `high`. Patch: remove the early return.\n\n## `test-bug`\n\n**Definition:** the code is correct; the spec is stale, makes a wrong assumption, or asserts something the component doesn't do.\n\n**Strongest signals:**\n- Spec file has a recent commit and the component hasn't changed.\n- The spec asserts a `data-testid` or `text` that doesn't exist anywhere in the component source.\n- The spec assumes a route, a label, or an order-of-operations that contradicts the source.\n- The snapshot shows the component is in the state the test wanted \u2014 but the assertion is wrong.\n\n**Examples:**\n\n> Spec: `await expect(page.getByTestId('checkout-fail')).toBeVisible()`.\n> Component source: there is `data-testid=\"checkout-network-error\"` and `data-testid=\"checkout-declined\"` but no `checkout-fail`.\n> \u2192 `test-bug`, `high`. Patch: change the selector to whichever testid was intended.\n\n> Spec was copy-pasted from another spec and still asserts the wrong page title.\n> \u2192 `test-bug`, `high`. Patch: update the title.\n\n## `flaky`\n\n**Definition:** non-deterministic failure unrelated to a code change. Includes ordering failures \u2014 failures that depend on what *other* specs did, not on the code or this spec.\n\n**Strongest signals:**\n- Failure mode is \"waiting for X, timed out\". X exists in source.\n- Neither file has changed recently.\n- Multiple consecutive runs against unchanged code show different outcomes.\n- Snapshot shows the component DID render the asserted state \u2014 just after the timeout.\n- Snapshot or runtime evidence shows state coming from **outside this spec's own actions** \u2014 eg `localStorage` populated with keys this spec never writes, an authenticated session this spec didn't sign in for, a database row another spec inserted. Often paired with an `error.txt` / trace line that says the failure correlates with run order (\"passes in isolation, fails after spec X\").\n\nUse sparingly. \"I don't know\" is more often a `low`-confidence other-classification than `flaky`.\n\n**Examples:**\n\n> Spec: clicks submit, asserts redirect within 5s. Sometimes the redirect takes 6s due to a queue.\n> \u2192 `flaky`, `medium`. Suggested fix: increase timeout for this spec, or add a deterministic gate (`waitForResponse`).\n\n> Spec: `await page.goto('/cart'); await expect(getByTestId('cart-empty')).toBeVisible()`.\n> Component snapshot: `hooks.state = [{id: 'abc'}]` (items array populated).\n> error.txt: \"Browser storage at time of failure: localStorage = {cart: '[{id:abc}]'}. Failure correlates with running after 'adds item to cart' spec. No spec or component change in 60 days.\"\n> Component on disk reads from localStorage on mount; spec doesn't call `localStorage.clear()`.\n> \u2192 `flaky` (ordering / state leak), `high`. The localStorage state did not come from this spec's actions \u2014 it leaked from another spec running first in the same worker. Even though the spec is also structurally fragile, the proximate cause is the order.\n> Suggested fix: add `test.beforeEach(({page}) => page.evaluate(() => localStorage.clear()))` to this spec or the file's parent describe. Optionally file an issue against the spec that wrote the leaked data.\n\n## Disambiguating `test-bug` vs `flaky` (ordering) when both fit\n\nSome failures look structurally like test-bugs (the spec doesn't `beforeEach(localStorage.clear())`, doesn't reset a global, doesn't re-seed the database) while *also* having runtime evidence of an ordering issue (snapshot shows state this spec never wrote; trace says the failure correlates with another spec running first). This is the most common false-confidence trap in the eval set, so be explicit.\n\n**Prefer `flaky` when:**\n- The component snapshot at failure contains state values this spec's own actions cannot explain. Eg `hooks.state = [{id:'abc'}]` but the spec never wrote that id; `props.user.email = '...'` but the spec didn't log anyone in.\n- The error trace or `error.txt` mentions the failure correlates with run order (\"passes in isolation, fails after X\"), failure rate is below 100%, or no recent changes to either file.\n- The same spec passes when run alone (a strong tell \u2014 if you only see \"fails in suite, passes alone\" in the trace, lean `flaky`).\n\n**Prefer `test-bug` when:**\n- The state the spec depends on is conventionally set up by the framework (eg `storageState`, fixtures) and *this* spec is the one that should have configured it.\n- The spec asserts a `data-testid` / text / route that doesn't exist anywhere in the source \u2014 the bug is in the assertion itself, not in the runtime state.\n- Multiple specs in the same file fail the same way regardless of order.\n\nThe structural shortcoming (no `beforeEach(clear)`) is real and worth fixing in both cases. But if the runtime evidence shows the leak came from *elsewhere*, that's the proximate cause \u2014 and the proximate cause is what `flaky` captures. Don't punish the symptom-bearing spec for being the one that exposed the order-dependency.\n\nConfidence in this disambiguation should track the directness of the evidence: snapshot hook values + an error.txt that names the suspected upstream spec is `high`; just \"no recent commits + spec doesn't clear state\" is `medium`.\n\n## `env-issue`\n\n**Definition:** the failure is infrastructure, not application logic.\n\n**Strongest signals:**\n- Error mentions: \"browserType.launch\", \"Executable doesn't exist\", \"EADDRINUSE\", \"ECONNREFUSED\", \"missing env var\".\n- All tests fail (not just one).\n- The component snapshot is empty / probe never connected.\n\n**Examples:**\n\n> Error: `Executable doesn't exist at .../chrome-headless-shell`.\n> \u2192 `env-issue`, `high`. Patch: `pnpm exec playwright install chromium`.\n\n> Error: `connect ECONNREFUSED 127.0.0.1:5173` on every test.\n> \u2192 `env-issue`, `high`. The dev server didn't start.\n\n## When you genuinely can't tell\n\nChoose the classification you'd guess at, drop confidence to `low`, and in `suggestedFix` say what evidence would let you upgrade.\n";
+//# sourceMappingURL=prompts.d.ts.map

package/dist/prompts.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"prompts.d.ts","sourceRoot":"","sources":["../src/prompts.ts"],"names":[],"mappings":"AASA,eAAO,MAAM,sBAAsB,2yHAmElC,CAAC;AAEF,eAAO,MAAM,mBAAmB,k4OA2G/B,CAAC"}

package/dist/prompts.js

@@ -0,0 +1,185 @@
+// Canonical diagnosis system prompts. Authored in markdown for readability;
+// exported as plain strings so consumers don't need a markdown loader.
+//
+// These are reactlens-flavoured (Playwright + DOM). Nativelens consumers
+// will eventually ship a parallel set keyed on Detox + React Native idioms.
+// Both versions are intended to share the same `Diagnosis` output schema
+// and `classification` rubric — only the surrounding "what tools you have"
+// scaffolding diverges.
+export const DIAGNOSE_SYSTEM_PROMPT = `# reactlens — diagnose a Playwright failure
+
+You are the diagnosis agent for reactlens. A Playwright test just failed. You have access to:
+
+- The failing **spec** (\`specPath\` below) — read it.
+- The **component** under test (\`componentPath\`) — read it. This is the code the user wrote.
+- The **trace summary** — Playwright's error message + locator chain.
+- The **component snapshot at failure** — the React tree at the moment the assertion failed, including props and hooks. This is the unique signal you have that other tools don't.
+- Optional **git context** for both files: who changed what most recently, and when.
+
+You have access to the \`Read\`, \`Glob\`, \`Grep\`, and \`Bash\` tools to look around. Use them deliberately — every read costs the user money.
+
+## Your job
+
+Output a single JSON object matching this schema (plain JSON; no markdown fence):
+
+\`\`\`json
+{
+  "classification": "real-bug" | "test-bug" | "flaky" | "env-issue",
+  "confidence": "high" | "medium" | "low",
+  "rootCause": "one sentence",
+  "evidence": ["bullet 1", "bullet 2", ...],
+  "suggestedFix": "human-readable description of what to change",
+  "patch": [{
+    "file": "path",
+    "oldStr": "exact text to replace",
+    "newStr": "replacement text",
+    "rationale": "why this fixes it"
+  }]
+}
+\`\`\`
+
+\`patch\` is OPTIONAL — only include it when you can produce a concrete edit you're confident about. If you can't, omit it and say what additional information would let you produce one (in \`suggestedFix\`).
+
+## Classification rubric
+
+See \`classify-bug.md\` (you should consult it). Headline rules:
+
+- **\`real-bug\`** — code regressed; spec is correct. The behavior the spec asserted was correct yesterday and is wrong today. Best signal: spec hasn't changed, component recently changed.
+- **\`test-bug\`** — code is fine; spec is stale or wrong. Best signal: component hasn't changed, spec recently changed; or the spec asserts something the source code never actually does.
+- **\`flaky\`** — neither code nor spec changed; the failure is non-deterministic. Best signal: timing-related error (waiting for element, race), no recent commits to either file.
+- **\`env-issue\`** — the failure is infrastructure (port conflict, missing env var, browser binary missing). Best signal: error mentions ports, env vars, browser launch, network unreachable.
+
+## Confidence calibration (CRITICAL)
+
+We measure your accuracy at each confidence level. Do not lie.
+
+- **\`high\`** — you have direct evidence in the snapshot AND/OR a recent commit that explains the behavior. Example: "the snapshot shows \`cvv: '12'\` — only 2 chars — but the component schema requires \`min(3)\`. The validation error visible in the snapshot is what the spec is asserting against."
+- **\`medium\`** — strong but indirect signal. Eg "the spec hasn't changed in 3 weeks, the component changed yesterday — but I haven't found the actual bug in the diff."
+- **\`low\`** — informed guess. Use this freely; it's better than a wrong \`high\`.
+
+If you cannot produce evidence, use \`low\`. Never inflate.
+
+## What to read
+
+In order:
+1. The component snapshot at failure (you'll get this in the user message inline). Note components, props, hook values.
+2. The spec (\`Read specPath\`).
+3. The component (\`Read componentPath\`).
+4. Anything the spec or component imports that's relevant (\`Read\` selectively).
+5. \`git log -p\` on either file if you need to see what changed recently.
+
+Do not read the entire codebase. Stop reading when you have enough to commit to an answer.
+
+## Output format
+
+Final message MUST be only the JSON object — no prose, no markdown, no code fence. Earlier messages can include reasoning. The system parses your last message as JSON.
+`;
+export const CLASSIFY_BUG_RUBRIC = `# reactlens — bug classification rubric
+
+This document defines, with examples, when to choose each classification. Diagnosis agent: consult this when in doubt. Calibration is measured against the eval set in \`tests/diagnostic-eval/cases/\`.
+
+## \`real-bug\`
+
+**Definition:** the code regressed and the spec is correctly asserting the prior, correct behavior.
+
+**Strongest signals:**
+- Component file has a recent commit (today, this week) and the spec hasn't changed in much longer.
+- The component snapshot at failure shows props/state inconsistent with the source's invariants — eg the source says \`if (data.length === 0) return Empty\` but the snapshot shows \`data.length === 0\` AND the rendered subtree doesn't include \`<Empty />\`.
+- The snapshot shows a thrown error in a render path the spec was exercising.
+
+**Examples:**
+
+> Spec: "checkout shows declined banner for cards starting with 4000".
+> Component snapshot: status='success' (not 'declined') after submitting a 4000-card.
+> Component diff (last commit): the developer changed the API check from \`startsWith('4000')\` to \`startsWith('5000')\`.
+> → \`real-bug\`, \`high\` confidence. Patch: revert the prefix check.
+
+> Spec: "dashboard shows empty state when there are no orders".
+> Component snapshot: orders=[] but \`<empty />\` is not in the tree.
+> Component: the developer added an \`early-return null\` above the empty check that fires unconditionally.
+> → \`real-bug\`, \`high\`. Patch: remove the early return.
+
+## \`test-bug\`
+
+**Definition:** the code is correct; the spec is stale, makes a wrong assumption, or asserts something the component doesn't do.
+
+**Strongest signals:**
+- Spec file has a recent commit and the component hasn't changed.
+- The spec asserts a \`data-testid\` or \`text\` that doesn't exist anywhere in the component source.
+- The spec assumes a route, a label, or an order-of-operations that contradicts the source.
+- The snapshot shows the component is in the state the test wanted — but the assertion is wrong.
+
+**Examples:**
+
+> Spec: \`await expect(page.getByTestId('checkout-fail')).toBeVisible()\`.
+> Component source: there is \`data-testid="checkout-network-error"\` and \`data-testid="checkout-declined"\` but no \`checkout-fail\`.
+> → \`test-bug\`, \`high\`. Patch: change the selector to whichever testid was intended.
+
+> Spec was copy-pasted from another spec and still asserts the wrong page title.
+> → \`test-bug\`, \`high\`. Patch: update the title.
+
+## \`flaky\`
+
+**Definition:** non-deterministic failure unrelated to a code change. Includes ordering failures — failures that depend on what *other* specs did, not on the code or this spec.
+
+**Strongest signals:**
+- Failure mode is "waiting for X, timed out". X exists in source.
+- Neither file has changed recently.
+- Multiple consecutive runs against unchanged code show different outcomes.
+- Snapshot shows the component DID render the asserted state — just after the timeout.
+- Snapshot or runtime evidence shows state coming from **outside this spec's own actions** — eg \`localStorage\` populated with keys this spec never writes, an authenticated session this spec didn't sign in for, a database row another spec inserted. Often paired with an \`error.txt\` / trace line that says the failure correlates with run order ("passes in isolation, fails after spec X").
+
+Use sparingly. "I don't know" is more often a \`low\`-confidence other-classification than \`flaky\`.
+
+**Examples:**
+
+> Spec: clicks submit, asserts redirect within 5s. Sometimes the redirect takes 6s due to a queue.
+> → \`flaky\`, \`medium\`. Suggested fix: increase timeout for this spec, or add a deterministic gate (\`waitForResponse\`).
+
+> Spec: \`await page.goto('/cart'); await expect(getByTestId('cart-empty')).toBeVisible()\`.
+> Component snapshot: \`hooks.state = [{id: 'abc'}]\` (items array populated).
+> error.txt: "Browser storage at time of failure: localStorage = {cart: '[{id:abc}]'}. Failure correlates with running after 'adds item to cart' spec. No spec or component change in 60 days."
+> Component on disk reads from localStorage on mount; spec doesn't call \`localStorage.clear()\`.
+> → \`flaky\` (ordering / state leak), \`high\`. The localStorage state did not come from this spec's actions — it leaked from another spec running first in the same worker. Even though the spec is also structurally fragile, the proximate cause is the order.
+> Suggested fix: add \`test.beforeEach(({page}) => page.evaluate(() => localStorage.clear()))\` to this spec or the file's parent describe. Optionally file an issue against the spec that wrote the leaked data.
+
+## Disambiguating \`test-bug\` vs \`flaky\` (ordering) when both fit
+
+Some failures look structurally like test-bugs (the spec doesn't \`beforeEach(localStorage.clear())\`, doesn't reset a global, doesn't re-seed the database) while *also* having runtime evidence of an ordering issue (snapshot shows state this spec never wrote; trace says the failure correlates with another spec running first). This is the most common false-confidence trap in the eval set, so be explicit.
+
+**Prefer \`flaky\` when:**
+- The component snapshot at failure contains state values this spec's own actions cannot explain. Eg \`hooks.state = [{id:'abc'}]\` but the spec never wrote that id; \`props.user.email = '...'\` but the spec didn't log anyone in.
+- The error trace or \`error.txt\` mentions the failure correlates with run order ("passes in isolation, fails after X"), failure rate is below 100%, or no recent changes to either file.
+- The same spec passes when run alone (a strong tell — if you only see "fails in suite, passes alone" in the trace, lean \`flaky\`).
+
+**Prefer \`test-bug\` when:**
+- The state the spec depends on is conventionally set up by the framework (eg \`storageState\`, fixtures) and *this* spec is the one that should have configured it.
+- The spec asserts a \`data-testid\` / text / route that doesn't exist anywhere in the source — the bug is in the assertion itself, not in the runtime state.
+- Multiple specs in the same file fail the same way regardless of order.
+
+The structural shortcoming (no \`beforeEach(clear)\`) is real and worth fixing in both cases. But if the runtime evidence shows the leak came from *elsewhere*, that's the proximate cause — and the proximate cause is what \`flaky\` captures. Don't punish the symptom-bearing spec for being the one that exposed the order-dependency.
+
+Confidence in this disambiguation should track the directness of the evidence: snapshot hook values + an error.txt that names the suspected upstream spec is \`high\`; just "no recent commits + spec doesn't clear state" is \`medium\`.
+
+## \`env-issue\`
+
+**Definition:** the failure is infrastructure, not application logic.
+
+**Strongest signals:**
+- Error mentions: "browserType.launch", "Executable doesn't exist", "EADDRINUSE", "ECONNREFUSED", "missing env var".
+- All tests fail (not just one).
+- The component snapshot is empty / probe never connected.
+
+**Examples:**
+
+> Error: \`Executable doesn't exist at .../chrome-headless-shell\`.
+> → \`env-issue\`, \`high\`. Patch: \`pnpm exec playwright install chromium\`.
+
+> Error: \`connect ECONNREFUSED 127.0.0.1:5173\` on every test.
+> → \`env-issue\`, \`high\`. The dev server didn't start.
+
+## When you genuinely can't tell
+
+Choose the classification you'd guess at, drop confidence to \`low\`, and in \`suggestedFix\` say what evidence would let you upgrade.
+`;
+//# sourceMappingURL=prompts.js.map

package/dist/prompts.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"prompts.js","sourceRoot":"","sources":["../src/prompts.ts"],"names":[],"mappings":"AAAA,4EAA4E;AAC5E,uEAAuE;AACvE,EAAE;AACF,yEAAyE;AACzE,4EAA4E;AAC5E,yEAAyE;AACzE,2EAA2E;AAC3E,wBAAwB;AAExB,MAAM,CAAC,MAAM,sBAAsB,GAAG;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;CAmErC,CAAC;AAEF,MAAM,CAAC,MAAM,mBAAmB,GAAG;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;CA2GlC,CAAC"}

package/package.json

@@ -0,0 +1,44 @@
+{
+  "name": "@reynsu/reactlens-diagnosis-prompts",
+  "version": "0.1.0",
+  "description": "Canonical diagnosis prompts + Zod Diagnosis schema + pure eval-metrics for the reactlens / nativelens AI failure-diagnosis loop. Pure helpers only — no LLM SDK dep.",
+  "license": "MIT",
+  "type": "module",
+  "files": [
+    "dist",
+    "src",
+    "!src/**/*.test.ts"
+  ],
+  "main": "./dist/index.js",
+  "types": "./dist/index.d.ts",
+  "exports": {
+    ".": {
+      "types": "./dist/index.d.ts",
+      "import": "./dist/index.js"
+    }
+  },
+  "scripts": {
+    "build": "tsc -p tsconfig.build.json",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "typecheck": "tsc --noEmit",
+    "prepublishOnly": "npm run typecheck && npm test && npm run build"
+  },
+  "dependencies": {
+    "zod": "^3.23.8"
+  },
+  "devDependencies": {
+    "typescript": "~5.9.2",
+    "vitest": "^2.1.0"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/reynsu/reactlens-prompts.git"
+  },
+  "pnpm": {
+    "onlyBuiltDependencies": ["esbuild"]
+  }
+}

package/src/diagnosis.ts

@@ -0,0 +1,107 @@
+// Canonical Diagnosis output shape and the pure helpers that surround it
+// (user-message builder, degraded fallback). Schema-first: `DiagnosisSchema`
+// is the source of truth — `Diagnosis` is `z.infer<typeof DiagnosisSchema>`.
+//
+// Consumers (reactlens, nativelens) provide their own agent runner, supply
+// per-host fields on FailedTest (e.g. a `componentSnapshot` shape that fits
+// their probe), and treat the returned Diagnosis as opaque-but-typed.
+
+import { z } from 'zod';
+
+export const ClassificationSchema = z.enum(['real-bug', 'test-bug', 'flaky', 'env-issue']);
+export type Classification = z.infer<typeof ClassificationSchema>;
+
+export const ConfidenceSchema = z.enum(['high', 'medium', 'low']);
+export type Confidence = z.infer<typeof ConfidenceSchema>;
+
+export const PatchEntrySchema = z.object({
+  file: z.string(),
+  oldStr: z.string(),
+  newStr: z.string(),
+  rationale: z.string(),
+});
+export type PatchEntry = z.infer<typeof PatchEntrySchema>;
+
+export const GitContextEntrySchema = z.object({
+  sha: z.string(),
+  author: z.string(),
+  date: z.string(),
+  message: z.string(),
+});
+export type GitContextEntry = z.infer<typeof GitContextEntrySchema>;
+
+export const GitContextSchema = z.object({
+  componentLastChanged: GitContextEntrySchema.optional(),
+  specLastChanged: GitContextEntrySchema.optional(),
+});
+export type GitContext = z.infer<typeof GitContextSchema>;
+
+export const DiagnosisSchema = z.object({
+  classification: ClassificationSchema,
+  confidence: ConfidenceSchema,
+  rootCause: z.string().min(1),
+  evidence: z.array(z.string()),
+  suggestedFix: z.string(),
+  patch: z.array(PatchEntrySchema).optional(),
+  gitContext: GitContextSchema.optional(),
+});
+export type Diagnosis = z.infer<typeof DiagnosisSchema>;
+
+// FailedTest is the input to the diagnosis prompt. `componentSnapshot` is
+// typed as `unknown` because reactlens (DOM) and nativelens (React Native
+// tree) serialise different shapes; each consumer narrows on its own side.
+export type FailedTest = {
+  testId: string;
+  testTitle: string;
+  specFile: string;
+  errorMessage?: string;
+  componentFile?: string;
+  componentSnapshot?: unknown;
+};
+
+// Builds the user-message string fed to the diagnosis agent alongside the
+// system prompts. Pure — produces the exact bytes that go on the wire so
+// the prompt surface can be diffed and golden-tested.
+export function buildUserMessage(failure: FailedTest): string {
+  const lines: string[] = [];
+  lines.push(`# Failure to diagnose`);
+  lines.push(``);
+  lines.push(`Test: ${failure.testTitle}`);
+  lines.push(`Spec: ${failure.specFile}`);
+  if (failure.componentFile !== undefined) lines.push(`Component (probable): ${failure.componentFile}`);
+  if (failure.errorMessage !== undefined) {
+    lines.push(``);
+    lines.push(`# Error`);
+    lines.push('```');
+    lines.push(failure.errorMessage);
+    lines.push('```');
+  }
+  if (failure.componentSnapshot !== undefined) {
+    lines.push(``);
+    lines.push(`# Component snapshot at failure`);
+    lines.push('```json');
+    const snippet = JSON.stringify(failure.componentSnapshot, null, 2);
+    // Truncate enormous trees to keep token usage sane. 30k chars is the
+    // empirically-tuned ceiling — beyond it the agent's classification
+    // accuracy starts to degrade in eval.
+    lines.push(snippet.length > 30_000 ? snippet.slice(0, 30_000) + '\n…(truncated)' : snippet);
+    lines.push('```');
+  }
+  lines.push(``);
+  lines.push(`Read the spec, the component, and any other context you need. Output a single JSON object matching the Diagnosis schema as the FINAL message.`);
+  return lines.join('\n');
+}
+
+// Fallback when the agent loop exhausts retries without producing a valid
+// Diagnosis. We DELIBERATELY return `env-issue` / `low` so the operator
+// notices: this is the calibration failure mode Principle 2 forbids.
+export function degradedDiagnosis(gitContext: GitContext = {}): Diagnosis {
+  return {
+    classification: 'env-issue',
+    confidence: 'low',
+    rootCause: 'diagnosis agent failed to produce a valid output',
+    evidence: ['agent returned non-JSON or schema-mismatched output twice'],
+    suggestedFix: 'rerun the diagnosis with --verbose; if it persists, file an issue with the trace',
+    ...(Object.keys(gitContext).length > 0 ? { gitContext } : {}),
+  };
+}

package/src/eval-metrics.ts

@@ -0,0 +1,79 @@
+// Pure scoring helpers for the diagnostic-eval harness. No fs, no agent —
+// consumers (reactlens, nativelens) drive their own per-host case loading
+// and call into these for grading.
+//
+// `falseConfidenceRate` is surfaced explicitly because Principle 2's
+// implicit cap on it (>=95% high-confidence accuracy) is the most useful
+// regression signal in eval reports.
+
+import { z } from 'zod';
+import { ClassificationSchema, ConfidenceSchema, type Classification, type Diagnosis } from './diagnosis.js';
+
+export const TruthSchema = z.object({
+  expectedClassification: ClassificationSchema,
+  minimumConfidence: ConfidenceSchema,
+  category: z.string().optional(),
+  notes: z.string().optional(),
+});
+export type Truth = z.infer<typeof TruthSchema>;
+
+export type CaseResult = {
+  name: string;
+  expected: Truth;
+  actual: Diagnosis;
+  correct: boolean;
+  // True when the diagnosis is wrong but the agent reported `high` confidence.
+  // This is the calibration failure mode Principle 2 forbids: confident wrong
+  // beats unconfident wrong, but we only ship if it's vanishingly rare.
+  falseConfidence: boolean;
+};
+
+export function parseTruth(raw: string): Truth {
+  return TruthSchema.parse(JSON.parse(raw));
+}
+
+export function compareToTruth(actual: Diagnosis, expected: Truth, name: string): CaseResult {
+  const correct = actual.classification === expected.expectedClassification;
+  return {
+    name,
+    expected,
+    actual,
+    correct,
+    falseConfidence: !correct && actual.confidence === 'high',
+  };
+}
+
+export type EvalMetrics = {
+  total: number;
+  correct: number;
+  accuracy: number;
+  // Wrong diagnoses delivered with `high` confidence, divided by total.
+  falseConfidenceRate: number;
+  // Bucketed by expected classification so per-category recall is visible
+  // (e.g. "we get real-bug right 90% of the time but flaky only 40%").
+  perClassification: Record<Classification, { total: number; correct: number }>;
+};
+
+export function aggregateMetrics(results: CaseResult[]): EvalMetrics {
+  const total = results.length;
+  const correct = results.filter((r) => r.correct).length;
+  const falseConfident = results.filter((r) => r.falseConfidence).length;
+  const perClassification: EvalMetrics['perClassification'] = {
+    'real-bug': { total: 0, correct: 0 },
+    'test-bug': { total: 0, correct: 0 },
+    flaky: { total: 0, correct: 0 },
+    'env-issue': { total: 0, correct: 0 },
+  };
+  for (const r of results) {
+    const bucket = perClassification[r.expected.expectedClassification];
+    bucket.total += 1;
+    if (r.correct) bucket.correct += 1;
+  }
+  return {
+    total,
+    correct,
+    accuracy: total === 0 ? 0 : correct / total,
+    falseConfidenceRate: total === 0 ? 0 : falseConfident / total,
+    perClassification,
+  };
+}

package/src/index.ts

@@ -0,0 +1,34 @@
+// Public surface — keep narrow. Consumers should import from the root only.
+
+export { DIAGNOSE_SYSTEM_PROMPT, CLASSIFY_BUG_RUBRIC } from './prompts.js';
+
+export {
+  // Schemas
+  ClassificationSchema,
+  ConfidenceSchema,
+  PatchEntrySchema,
+  GitContextEntrySchema,
+  GitContextSchema,
+  DiagnosisSchema,
+  // Types
+  type Classification,
+  type Confidence,
+  type PatchEntry,
+  type GitContextEntry,
+  type GitContext,
+  type Diagnosis,
+  type FailedTest,
+  // Pure helpers
+  buildUserMessage,
+  degradedDiagnosis,
+} from './diagnosis.js';
+
+export {
+  TruthSchema,
+  type Truth,
+  type CaseResult,
+  type EvalMetrics,
+  parseTruth,
+  compareToTruth,
+  aggregateMetrics,
+} from './eval-metrics.js';

package/src/prompts.ts

@@ -0,0 +1,186 @@
+// Canonical diagnosis system prompts. Authored in markdown for readability;
+// exported as plain strings so consumers don't need a markdown loader.
+//
+// These are reactlens-flavoured (Playwright + DOM). Nativelens consumers
+// will eventually ship a parallel set keyed on Detox + React Native idioms.
+// Both versions are intended to share the same `Diagnosis` output schema
+// and `classification` rubric — only the surrounding "what tools you have"
+// scaffolding diverges.
+
+export const DIAGNOSE_SYSTEM_PROMPT = `# reactlens — diagnose a Playwright failure
+
+You are the diagnosis agent for reactlens. A Playwright test just failed. You have access to:
+
+- The failing **spec** (\`specPath\` below) — read it.
+- The **component** under test (\`componentPath\`) — read it. This is the code the user wrote.
+- The **trace summary** — Playwright's error message + locator chain.
+- The **component snapshot at failure** — the React tree at the moment the assertion failed, including props and hooks. This is the unique signal you have that other tools don't.
+- Optional **git context** for both files: who changed what most recently, and when.
+
+You have access to the \`Read\`, \`Glob\`, \`Grep\`, and \`Bash\` tools to look around. Use them deliberately — every read costs the user money.
+
+## Your job
+
+Output a single JSON object matching this schema (plain JSON; no markdown fence):
+
+\`\`\`json
+{
+  "classification": "real-bug" | "test-bug" | "flaky" | "env-issue",
+  "confidence": "high" | "medium" | "low",
+  "rootCause": "one sentence",
+  "evidence": ["bullet 1", "bullet 2", ...],
+  "suggestedFix": "human-readable description of what to change",
+  "patch": [{
+    "file": "path",
+    "oldStr": "exact text to replace",
+    "newStr": "replacement text",
+    "rationale": "why this fixes it"
+  }]
+}
+\`\`\`
+
+\`patch\` is OPTIONAL — only include it when you can produce a concrete edit you're confident about. If you can't, omit it and say what additional information would let you produce one (in \`suggestedFix\`).
+
+## Classification rubric
+
+See \`classify-bug.md\` (you should consult it). Headline rules:
+
+- **\`real-bug\`** — code regressed; spec is correct. The behavior the spec asserted was correct yesterday and is wrong today. Best signal: spec hasn't changed, component recently changed.
+- **\`test-bug\`** — code is fine; spec is stale or wrong. Best signal: component hasn't changed, spec recently changed; or the spec asserts something the source code never actually does.
+- **\`flaky\`** — neither code nor spec changed; the failure is non-deterministic. Best signal: timing-related error (waiting for element, race), no recent commits to either file.
+- **\`env-issue\`** — the failure is infrastructure (port conflict, missing env var, browser binary missing). Best signal: error mentions ports, env vars, browser launch, network unreachable.
+
+## Confidence calibration (CRITICAL)
+
+We measure your accuracy at each confidence level. Do not lie.
+
+- **\`high\`** — you have direct evidence in the snapshot AND/OR a recent commit that explains the behavior. Example: "the snapshot shows \`cvv: '12'\` — only 2 chars — but the component schema requires \`min(3)\`. The validation error visible in the snapshot is what the spec is asserting against."
+- **\`medium\`** — strong but indirect signal. Eg "the spec hasn't changed in 3 weeks, the component changed yesterday — but I haven't found the actual bug in the diff."
+- **\`low\`** — informed guess. Use this freely; it's better than a wrong \`high\`.
+
+If you cannot produce evidence, use \`low\`. Never inflate.
+
+## What to read
+
+In order:
+1. The component snapshot at failure (you'll get this in the user message inline). Note components, props, hook values.
+2. The spec (\`Read specPath\`).
+3. The component (\`Read componentPath\`).
+4. Anything the spec or component imports that's relevant (\`Read\` selectively).
+5. \`git log -p\` on either file if you need to see what changed recently.
+
+Do not read the entire codebase. Stop reading when you have enough to commit to an answer.
+
+## Output format
+
+Final message MUST be only the JSON object — no prose, no markdown, no code fence. Earlier messages can include reasoning. The system parses your last message as JSON.
+`;
+
+export const CLASSIFY_BUG_RUBRIC = `# reactlens — bug classification rubric
+
+This document defines, with examples, when to choose each classification. Diagnosis agent: consult this when in doubt. Calibration is measured against the eval set in \`tests/diagnostic-eval/cases/\`.
+
+## \`real-bug\`
+
+**Definition:** the code regressed and the spec is correctly asserting the prior, correct behavior.
+
+**Strongest signals:**
+- Component file has a recent commit (today, this week) and the spec hasn't changed in much longer.
+- The component snapshot at failure shows props/state inconsistent with the source's invariants — eg the source says \`if (data.length === 0) return Empty\` but the snapshot shows \`data.length === 0\` AND the rendered subtree doesn't include \`<Empty />\`.
+- The snapshot shows a thrown error in a render path the spec was exercising.
+
+**Examples:**
+
+> Spec: "checkout shows declined banner for cards starting with 4000".
+> Component snapshot: status='success' (not 'declined') after submitting a 4000-card.
+> Component diff (last commit): the developer changed the API check from \`startsWith('4000')\` to \`startsWith('5000')\`.
+> → \`real-bug\`, \`high\` confidence. Patch: revert the prefix check.
+
+> Spec: "dashboard shows empty state when there are no orders".
+> Component snapshot: orders=[] but \`<empty />\` is not in the tree.
+> Component: the developer added an \`early-return null\` above the empty check that fires unconditionally.
+> → \`real-bug\`, \`high\`. Patch: remove the early return.
+
+## \`test-bug\`
+
+**Definition:** the code is correct; the spec is stale, makes a wrong assumption, or asserts something the component doesn't do.
+
+**Strongest signals:**
+- Spec file has a recent commit and the component hasn't changed.
+- The spec asserts a \`data-testid\` or \`text\` that doesn't exist anywhere in the component source.
+- The spec assumes a route, a label, or an order-of-operations that contradicts the source.
+- The snapshot shows the component is in the state the test wanted — but the assertion is wrong.
+
+**Examples:**
+
+> Spec: \`await expect(page.getByTestId('checkout-fail')).toBeVisible()\`.
+> Component source: there is \`data-testid="checkout-network-error"\` and \`data-testid="checkout-declined"\` but no \`checkout-fail\`.
+> → \`test-bug\`, \`high\`. Patch: change the selector to whichever testid was intended.
+
+> Spec was copy-pasted from another spec and still asserts the wrong page title.
+> → \`test-bug\`, \`high\`. Patch: update the title.
+
+## \`flaky\`
+
+**Definition:** non-deterministic failure unrelated to a code change. Includes ordering failures — failures that depend on what *other* specs did, not on the code or this spec.
+
+**Strongest signals:**
+- Failure mode is "waiting for X, timed out". X exists in source.
+- Neither file has changed recently.
+- Multiple consecutive runs against unchanged code show different outcomes.
+- Snapshot shows the component DID render the asserted state — just after the timeout.
+- Snapshot or runtime evidence shows state coming from **outside this spec's own actions** — eg \`localStorage\` populated with keys this spec never writes, an authenticated session this spec didn't sign in for, a database row another spec inserted. Often paired with an \`error.txt\` / trace line that says the failure correlates with run order ("passes in isolation, fails after spec X").
+
+Use sparingly. "I don't know" is more often a \`low\`-confidence other-classification than \`flaky\`.
+
+**Examples:**
+
+> Spec: clicks submit, asserts redirect within 5s. Sometimes the redirect takes 6s due to a queue.
+> → \`flaky\`, \`medium\`. Suggested fix: increase timeout for this spec, or add a deterministic gate (\`waitForResponse\`).
+
+> Spec: \`await page.goto('/cart'); await expect(getByTestId('cart-empty')).toBeVisible()\`.
+> Component snapshot: \`hooks.state = [{id: 'abc'}]\` (items array populated).
+> error.txt: "Browser storage at time of failure: localStorage = {cart: '[{id:abc}]'}. Failure correlates with running after 'adds item to cart' spec. No spec or component change in 60 days."
+> Component on disk reads from localStorage on mount; spec doesn't call \`localStorage.clear()\`.
+> → \`flaky\` (ordering / state leak), \`high\`. The localStorage state did not come from this spec's actions — it leaked from another spec running first in the same worker. Even though the spec is also structurally fragile, the proximate cause is the order.
+> Suggested fix: add \`test.beforeEach(({page}) => page.evaluate(() => localStorage.clear()))\` to this spec or the file's parent describe. Optionally file an issue against the spec that wrote the leaked data.
+
+## Disambiguating \`test-bug\` vs \`flaky\` (ordering) when both fit
+
+Some failures look structurally like test-bugs (the spec doesn't \`beforeEach(localStorage.clear())\`, doesn't reset a global, doesn't re-seed the database) while *also* having runtime evidence of an ordering issue (snapshot shows state this spec never wrote; trace says the failure correlates with another spec running first). This is the most common false-confidence trap in the eval set, so be explicit.
+
+**Prefer \`flaky\` when:**
+- The component snapshot at failure contains state values this spec's own actions cannot explain. Eg \`hooks.state = [{id:'abc'}]\` but the spec never wrote that id; \`props.user.email = '...'\` but the spec didn't log anyone in.
+- The error trace or \`error.txt\` mentions the failure correlates with run order ("passes in isolation, fails after X"), failure rate is below 100%, or no recent changes to either file.
+- The same spec passes when run alone (a strong tell — if you only see "fails in suite, passes alone" in the trace, lean \`flaky\`).
+
+**Prefer \`test-bug\` when:**
+- The state the spec depends on is conventionally set up by the framework (eg \`storageState\`, fixtures) and *this* spec is the one that should have configured it.
+- The spec asserts a \`data-testid\` / text / route that doesn't exist anywhere in the source — the bug is in the assertion itself, not in the runtime state.
+- Multiple specs in the same file fail the same way regardless of order.
+
+The structural shortcoming (no \`beforeEach(clear)\`) is real and worth fixing in both cases. But if the runtime evidence shows the leak came from *elsewhere*, that's the proximate cause — and the proximate cause is what \`flaky\` captures. Don't punish the symptom-bearing spec for being the one that exposed the order-dependency.
+
+Confidence in this disambiguation should track the directness of the evidence: snapshot hook values + an error.txt that names the suspected upstream spec is \`high\`; just "no recent commits + spec doesn't clear state" is \`medium\`.
+
+## \`env-issue\`
+
+**Definition:** the failure is infrastructure, not application logic.
+
+**Strongest signals:**
+- Error mentions: "browserType.launch", "Executable doesn't exist", "EADDRINUSE", "ECONNREFUSED", "missing env var".
+- All tests fail (not just one).
+- The component snapshot is empty / probe never connected.
+
+**Examples:**
+
+> Error: \`Executable doesn't exist at .../chrome-headless-shell\`.
+> → \`env-issue\`, \`high\`. Patch: \`pnpm exec playwright install chromium\`.
+
+> Error: \`connect ECONNREFUSED 127.0.0.1:5173\` on every test.
+> → \`env-issue\`, \`high\`. The dev server didn't start.
+
+## When you genuinely can't tell
+
+Choose the classification you'd guess at, drop confidence to \`low\`, and in \`suggestedFix\` say what evidence would let you upgrade.
+`;