tewip-ai 0.18.6 → 0.20.0

package/README.md

@@ -7,7 +7,7 @@ It works for every stack: Node.js (node:test, Jest, Vitest, Mocha), Python (pyte
 | Where | How |
 |---|---|
 | **Terminal** | `npm install -g tewip-ai`, then `tewip explain report.xml`, `tewip ci owner/repo --pr 12`, `tewip mcp` |
-| **VS Code, Cursor, Windsurf, VSCodium** | the Tewip extension: registers Tewip with the editor's AI assistant and adds *Explain a test report* and *Why did CI fail on this branch?* |
+| **VS Code, Cursor, Windsurf, VSCodium** | the Tewip extension ([download](https://github.com/hmamut39/tewip/releases/tag/vscode-0.5.0)): registers Tewip with the editor's AI assistant and adds six commands - explain a report, why CI failed on this branch, the risk review of your change, which flaky tests to quarantine, the security check, and what your corrections add up to |
 | **JetBrains, Claude, other AI assistants** | add the MCP server `tewip mcp` (see [editor setup](#use-it-from-your-editor-any-ide-that-speaks-mcp)) |
 | **Any model** | OpenAI, Claude, Gemini, Azure, or a model on your own machine with Ollama (no account, nothing leaves the computer): `TEWIP_PROVIDER` |
 | **GitHub CI** | the Tewip Action: a comment on every pull request, an optional merge gate, verified fix PRs ([QUICKSTART](QUICKSTART.md)) |
@@ -96,6 +96,20 @@ The report reads the history cache of the branch it runs on, usually the default
 
 The repository needs "Allow GitHub Actions to create and approve pull requests" enabled for fix PRs. Pull requests opened by the workflow token don't trigger CI themselves; the agent verified each change by re-running the test before opening the PR. See [triage-agent-sandbox](https://github.com/hmamut39/triage-agent-sandbox) for a working example. If this repository is private, other repositories can only use the action when Settings → Actions → Access allows it.
 
+### The editor extension (VS Code, Cursor, Windsurf, VSCodium)
+
+Download **[tewip-0.5.0.vsix](https://github.com/hmamut39/tewip/releases/tag/vscode-0.5.0)**, then in the
+editor press `Ctrl+Shift+P` and run **Extensions: Install from VSIX…**. It needs the command line:
+`npm install -g tewip-ai`.
+
+Press `Ctrl+Shift+P` and type "Tewip" for: *Explain a test report…*, *Why did CI fail on this
+branch?*, *Risk review of this change*, *Which flaky tests to quarantine*, *Check secrets and
+dependencies*, and *What our corrections add up to*. The editor's AI assistant also picks Tewip up
+as an MCP server with nothing to configure.
+
+It is not in the Visual Studio Marketplace: publishing there now requires an Azure subscription,
+which means a credit card for a free extension. The file above installs exactly the same thing.
+
 ### 2. From any AI assistant, through MCP (the whole IT team)
 
 `mcp/server.ts` is a read-only [Model Context Protocol](https://modelcontextprotocol.io) server. Anyone on the team can ask their own assistant about test failures, in the editor or chat they already use:

package/dist/cli/help.js

@@ -100,6 +100,17 @@ Usage:
                                     same way more than once, which rule decided it, and the failures
                                     it came from. Says "not yet" rather than inventing a pattern
                                     from two cases. No model, no cost.
+  tewip incident <sentry.json | otel.json | stack.txt> [--framework <id>] [--out incident.md]
+                                    Turn a production incident into a test that proves it: reads
+                                    Sentry, OpenTelemetry or a pasted stack trace, writes a test
+                                    against the code the trace names, runs it, and keeps it ONLY if
+                                    it fails for the incident's own reason. A test that passes, or
+                                    fails its own way, is reported and thrown away.
+  tewip attest [--run <id>] [--out attestation.md] [--json bundle.json]
+                                    What Tewip checked for a build, as one artifact: the claims with
+                                    their evidence and which layer decided each, what it refused to
+                                    decide and why, and what it did not check at all. Nothing enters
+                                    it that Tewip did not verify itself. No model, no cost.
   tewip mcp                         Serve Tewip over MCP (stdio) for an editor or AI assistant.
   tewip --version                   Print the version.
 

package/dist/cli/tewip.js

@@ -177,6 +177,22 @@ async function main() {
             runLearn({ root: process.cwd(), name: flag(args, 'name'), labels: flag(args, 'labels'), out: flag(args, 'out') });
             return;
         }
+        case 'incident': {
+            const file = args.find((a, i) => !a.startsWith('--') && !['--framework', '--out'].includes(args[i - 1] ?? ''));
+            if (!file)
+                throw new Error('usage: tewip incident <sentry.json | otel.json | stack.txt> [--framework pytest] [--keep] [--out incident.md]');
+            const { runIncident } = await import('../engine/incident/incident-command.js');
+            const result = await runIncident({ root: process.cwd(), file, framework: flag(args, 'framework'), keep: args.includes('--keep'), out: flag(args, 'out') });
+            // A reproduction that was not reproduced is not a success, and a script should be able to tell.
+            if (!result.kept)
+                process.exitCode = 1;
+            return;
+        }
+        case 'attest': {
+            const { runAttest } = await import('../engine/attest/attest-command.js');
+            runAttest({ root: process.cwd(), name: flag(args, 'name'), runId: flag(args, 'run'), out: flag(args, 'out'), json: flag(args, 'json') });
+            return;
+        }
         case 'mcp':
             // An editor starts this from any folder, and an installed tool's own folder may be read-only:
             // the AI answer cache goes to the user's home folder, which is always writable.

package/dist/engine/attest/attest-command.js

@@ -0,0 +1,63 @@
+// `tewip attest`: the evidence for one run, as an artifact a build can carry.
+import { existsSync, mkdirSync, readdirSync, writeFileSync } from 'node:fs';
+import { dirname, join, resolve } from 'node:path';
+import { datasetFiles, readJsonl, readTriageRecords } from '../../reporters/store.js';
+import { ENGINE_VERSION } from '../engine.js';
+import { attestRun, renderAttestation } from './attest.js';
+/** Everything Tewip kept for one project, newest run last. */
+function history(root, only) {
+    const dir = join(root, '.tewip', 'history');
+    const projects = existsSync(dir) ? (only ? [only] : readdirSync(dir)) : [];
+    for (const project of projects) {
+        const files = datasetFiles(join(dir, project));
+        const runs = readJsonl(files.runs);
+        if (runs.length > 0)
+            return { project, runs, records: readTriageRecords(files.triage) };
+    }
+    return { project: only ?? 'this project', runs: [], records: [] };
+}
+/**
+ * What ran, in the words of the thing that ran. Only what is recorded counts: an attestation that
+ * listed checks Tewip merely supports would be the fabrication this whole phase exists to prevent.
+ */
+function whatWasChecked(records) {
+    const checked = [];
+    const notChecked = [];
+    checked.push('every failing test in the run was read, and each was given a cause or left undecided');
+    const byRules = records.filter((r) => r.diagnosis?.source === 'rules').length;
+    const byModel = records.filter((r) => r.diagnosis?.source === 'llm').length;
+    if (byRules > 0)
+        checked.push(`${byRules} decided by deterministic rules (${ENGINE_VERSION}), which call no model and give the same answer every time`);
+    if (byModel > 0)
+        checked.push(`${byModel} sent to the model, because no rule could settle them`);
+    // What Tewip is silent about unless it was asked, and therefore must say it did not do.
+    notChecked.push('the application itself: no security scan, dependency audit or web check is part of this bundle unless `tewip secure` ran and is quoted here');
+    notChecked.push('tests that passed: a passing test is counted, not examined');
+    notChecked.push('anything no test covers, which no test report can show');
+    return { checked, notChecked };
+}
+export function runAttest(cmd) {
+    const { project, runs, records } = history(cmd.root, cmd.name);
+    if (runs.length === 0) {
+        throw new Error('nothing to attest: no run has been kept here yet. Run `tewip explain <report>` first, so there is evidence to attest to.');
+    }
+    const run = cmd.runId ? runs.find((r) => r.runId === cmd.runId) : runs[runs.length - 1];
+    if (!run)
+        throw new Error(`no run called ${cmd.runId} was kept here`);
+    const mine = records.filter((r) => r.runId === run.runId);
+    const { checked, notChecked } = whatWasChecked(mine);
+    const attestation = attestRun(project, run, mine, { checked, notChecked, tool: 'tewip', engine: ENGINE_VERSION });
+    const markdown = renderAttestation(attestation);
+    console.log(markdown);
+    if (cmd.out) {
+        mkdirSync(dirname(resolve(cmd.out)), { recursive: true });
+        writeFileSync(resolve(cmd.out), `${markdown}\n`, 'utf8');
+        console.error(`\nWritten to ${cmd.out}`);
+    }
+    if (cmd.json) {
+        mkdirSync(dirname(resolve(cmd.json)), { recursive: true });
+        writeFileSync(resolve(cmd.json), `${JSON.stringify(attestation, null, 2)}\n`, 'utf8');
+        console.error(`Machine-readable bundle written to ${cmd.json}`);
+    }
+    return attestation;
+}

package/dist/engine/attest/attest.js

@@ -0,0 +1,89 @@
+const sourceOf = (record) => String(record.diagnosis?.source ?? '');
+/** The evidence line a person reads: the first thing the deciding layer actually said. */
+function evidenceOf(record) {
+    const evidence = record.diagnosis?.evidence ?? [];
+    const first = evidence.find((e) => e.message)?.message ?? '';
+    return first.slice(0, 300);
+}
+/**
+ * The bundle for one run. `records` and `run` are what Tewip itself wrote when it read the report,
+ * so everything here is traceable; nothing is inferred and nothing is re-judged.
+ */
+export function attestRun(project, run, records, { checked, notChecked, tool, engine }) {
+    const claims = [];
+    const refusals = [];
+    for (const record of records) {
+        const diagnosis = record.diagnosis;
+        const category = diagnosis?.category ?? 'UNKNOWN';
+        const decidedBy = sourceOf(record);
+        // Undecided is not a claim. It is the refusal that makes the rest of the bundle believable.
+        if (category === 'UNKNOWN' || !decidedBy) {
+            refusals.push({
+                what: `${record.testId} failed, and Tewip did not classify it`,
+                why: evidenceOf(record) || 'no rule matched and the model was not asked or did not answer',
+                runId: record.runId,
+            });
+            continue;
+        }
+        claims.push({
+            claim: `${record.testId} failed because of ${category}`,
+            evidence: evidenceOf(record) || 'no evidence recorded',
+            runId: record.runId,
+            decidedBy,
+            confidence: Number(diagnosis?.confidence ?? 0),
+        });
+    }
+    return {
+        project,
+        commit: run.gitSha ?? '',
+        runId: run.runId,
+        startedAt: run.startedAt,
+        finishedAt: run.finishedAt,
+        testsAttempted: Object.keys(run.attempts ?? {}).length,
+        failures: records.length,
+        checked,
+        notChecked,
+        claims,
+        refusals,
+        producedBy: { tool, engine, at: new Date().toISOString() },
+    };
+}
+/** The bundle as a person reads it. The refusals are not hidden at the bottom; they are the point. */
+export function renderAttestation(a) {
+    const when = a.finishedAt ? a.finishedAt.slice(0, 19).replace('T', ' ') : 'unknown';
+    const lines = [
+        `# What Tewip checked: ${a.project}`,
+        '',
+        `Run \`${a.runId}\`${a.commit ? ` at commit \`${a.commit.slice(0, 8)}\`` : ''}, finished ${when} UTC.`,
+        `${a.testsAttempted} test(s) attempted, ${a.failures} failed.`,
+        '',
+        '## What was checked',
+        '',
+        ...(a.checked.length > 0 ? a.checked.map((c) => `- ${c}`) : ['- nothing: this bundle makes no claims']),
+        '',
+        '## What was NOT checked',
+        '',
+        ...(a.notChecked.length > 0
+            ? a.notChecked.map((c) => `- ${c}`)
+            : ['- nothing is recorded as unchecked, which is itself worth doubting: a check that was never run cannot report itself']),
+        '',
+    ];
+    lines.push(`## Claims (${a.claims.length})`, '');
+    if (a.claims.length === 0)
+        lines.push('None: nothing failed, so there is nothing to explain.', '');
+    for (const claim of a.claims) {
+        lines.push(`- **${claim.claim}** — ${claim.decidedBy}, confidence ${claim.confidence.toFixed(2)}`);
+        lines.push(`  - Evidence: ${claim.evidence}`);
+        lines.push(`  - From run \`${claim.runId}\``);
+    }
+    lines.push('', `## Refused (${a.refusals.length})`, '');
+    lines.push(a.refusals.length > 0
+        ? 'Tewip could not decide these. They are listed because a tool that never says so cannot be trusted about the rest.'
+        : 'Nothing was left undecided in this run.', '');
+    for (const refusal of a.refusals) {
+        lines.push(`- **${refusal.what}**`);
+        lines.push(`  - Why: ${refusal.why}`);
+    }
+    lines.push('', '---', '', `Produced by ${a.producedBy.tool} (${a.producedBy.engine}) at ${a.producedBy.at.slice(0, 19).replace('T', ' ')} UTC.`, 'Every claim above names the run it came from and the layer that decided it. This is a statement', 'about the checks listed under "what was checked" and nothing else.');
+    return lines.join('\n');
+}

package/dist/engine/incident/incident-command.js

@@ -0,0 +1,111 @@
+// `tewip incident <file>`: turn what happened in production into a test that proves it.
+//
+// The loop, and the reason each step is there:
+//   read the incident      - deterministic, from Sentry, OpenTelemetry or a pasted stack trace
+//   find the code it names - the project's own frames, so the test aims at real functions
+//   write a failing test   - the model's only job, and the smallest one
+//   run it                 - because a test nobody ran proves nothing
+//   judge it               - red is not proof: it has to be red for the incident's reason
+//
+// Nothing is kept unless the judgement is "reproduced". A test that passes, fails its own way, or
+// never ran is reported as exactly that and the file is removed, because a test wrongly labelled
+// "reproduces the outage" is worse than no test: the next person will trust it.
+import { existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
+import { dirname, join, resolve } from 'node:path';
+import { noModelHelp } from '../llm/providers.js';
+import { writerFromEnv } from '../quality/model.js';
+import { usageLine } from '../llm/usage.js';
+import { chooseFramework, detectFrameworks } from '../quality/frameworks.js';
+import { draftTest } from '../quality/write.js';
+import { runTest } from '../quality/write-verify.js';
+import { ourFrames, readIncident } from './incident.js';
+import { judgeReproduction, nextStep } from './reproduced.js';
+/** The code the incident points at, so the test is written against real functions, not guesses. */
+function codeFromFrames(root, incident) {
+    const seen = new Set();
+    const out = [];
+    for (const frame of ourFrames(incident)) {
+        // A production path (/app/src/pay.ts) rarely matches a checkout, so the tail is tried too.
+        const candidates = [frame.file, ...frame.file.split('/').map((_, i, all) => all.slice(i).join('/'))];
+        for (const candidate of candidates) {
+            const path = resolve(root, candidate);
+            if (seen.has(path) || !existsSync(path) || !path.startsWith(resolve(root)))
+                continue;
+            seen.add(path);
+            out.push({ file: candidate, content: readFileSync(path, 'utf8').slice(0, 8000) });
+            break;
+        }
+        if (out.length >= 3)
+            break;
+    }
+    return out;
+}
+/** The incident as a scenario the writer already knows how to take. */
+function asScenario(incident) {
+    const where = ourFrames(incident)[0];
+    return {
+        id: 'INCIDENT-1',
+        title: `reproduce: ${incident.title}`.slice(0, 120),
+        preconditions: [incident.operation ? `the system is doing: ${incident.operation}` : 'the code path below is exercised'],
+        steps: [
+            where ? `call the code in ${where.file}${where.fn ? ` (${where.fn})` : ''}${where.line ? ` around line ${where.line}` : ''} the way production did` : 'exercise the failing path',
+            'do not guard against the failure; the test must hit it',
+        ],
+        expected: `it fails with ${[incident.errorType, incident.errorMessage].filter(Boolean).join(': ') || incident.title}. Write the test asserting the behaviour that SHOULD happen, so it is red now and green once the bug is fixed.`,
+    };
+}
+export async function runIncident(cmd) {
+    const incident = readIncident(readFileSync(resolve(cmd.file), 'utf8'));
+    const framework = chooseFramework(detectFrameworks(cmd.root), cmd.framework, false);
+    if (!framework)
+        throw new Error('no test framework was found here, so there is nothing to write a test in. Run this inside the project.');
+    const model = writerFromEnv(cmd.root);
+    if (!model)
+        throw new Error(`reproducing an incident needs a model to write the test.\n${noModelHelp()}`);
+    const code = codeFromFrames(cmd.root, incident);
+    console.error(`Read a ${incident.source} incident: ${incident.title.slice(0, 90)}`);
+    console.error(code.length > 0 ? `It points at ${code.map((c) => c.file).join(', ')}.` : 'It names no file inside this project, so the test is written from the error alone.');
+    // The framework names the file, so its runner picks it up and its imports resolve.
+    const fileName = framework.fileFor(`incident-${Date.now().toString(36)}`);
+    const testFile = join(framework.testDir, fileName);
+    mkdirSync(dirname(testFile), { recursive: true });
+    const draft = await draftTest(model, { framework, fileName, pages: [], feature: `A production incident: ${incident.title}`, code }, asScenario(incident));
+    writeFileSync(testFile, draft.code, 'utf8');
+    console.error('Wrote the test; running it now, because a test nobody ran proves nothing…');
+    const run = runTest(framework, testFile);
+    const judgement = judgeReproduction(incident, { passed: run.passed, output: run.output });
+    const kept = judgement.verdict === 'reproduced';
+    if (!kept)
+        rmSync(testFile, { force: true });
+    const report = renderIncident(incident, judgement, kept ? testFile : '', draft.code);
+    console.log(report);
+    console.error(`\n${usageLine()}`);
+    if (cmd.out) {
+        mkdirSync(dirname(resolve(cmd.out)), { recursive: true });
+        writeFileSync(resolve(cmd.out), `${report}\n`, 'utf8');
+    }
+    return { incident, judgement, testFile: kept ? testFile : '', kept };
+}
+export function renderIncident(incident, judgement, testFile, code) {
+    const where = ourFrames(incident)[0];
+    const lines = [
+        `# ${judgement.verdict === 'reproduced' ? 'Reproduced' : 'Not reproduced'}: ${incident.title.slice(0, 100)}`,
+        '',
+        `Read from ${incident.source}.${incident.operation ? ` During: ${incident.operation}.` : ''}`,
+        `Production reported: ${[incident.errorType, incident.errorMessage].filter(Boolean).join(': ') || '(no error text)'}`,
+        where ? `Your code in the trace: ${where.file}${where.line ? `:${where.line}` : ''}${where.fn ? ` (${where.fn})` : ''}` : 'No file of yours appears in the trace.',
+        '',
+        `## ${judgement.verdict === 'reproduced' ? 'The test is red for the right reason' : 'Nothing is kept'}`,
+        '',
+        judgement.why,
+        '',
+        `**Do:** ${nextStep[judgement.verdict]}`,
+        '',
+    ];
+    if (testFile)
+        lines.push(`The test is at \`${testFile}\`. It should stay red until the bug is fixed, and turning it green is the proof that it was.`, '');
+    else
+        lines.push('The test Tewip wrote is below, unkept, so you can see what was tried:', '', '```', code.slice(0, 2000), '```', '');
+    lines.push('Tewip never keeps a test that has not been run, and never calls a red test a reproduction unless it is red for the reason production was.');
+    return lines.join('\n');
+}

package/dist/engine/incident/incident.js

@@ -0,0 +1,116 @@
+const LIBRARY = /[\\/](?:node_modules|site-packages|dist-packages|vendor|\.venv|venv|target[\\/]dependency|gems)[\\/]/;
+const RUNTIME = /^(?:node:|internal[\\/]|java\.|javax\.|sun\.|System\.|Microsoft\.)/;
+const ours = (file) => Boolean(file) && !LIBRARY.test(file) && !RUNTIME.test(file);
+const frame = (file, line, fn) => ({
+    file: String(file ?? '').replace(/\\/g, '/'),
+    line: Number(line) || 0,
+    fn: String(fn ?? ''),
+    ours: ours(String(file ?? '')),
+});
+/** A stack trace as every language prints it, which is the fallback when there is no tool. */
+export function framesFromStack(stack) {
+    const out = [];
+    for (const line of stack.split('\n')) {
+        // JavaScript: "at handler (/app/src/pay.ts:42:9)" or "at /app/src/pay.ts:42:9"
+        const js = /^\s*at\s+(?:(?<fn>[^(]+?)\s+\()?(?<file>[^():]+?):(?<line>\d+):\d+\)?\s*$/.exec(line);
+        if (js?.groups) {
+            out.push(frame(js.groups.file, js.groups.line, (js.groups.fn ?? '').trim()));
+            continue;
+        }
+        // Python: '  File "/app/pay.py", line 42, in charge'
+        const py = /^\s*File\s+"(?<file>[^"]+)",\s+line\s+(?<line>\d+)(?:,\s+in\s+(?<fn>\S+))?/.exec(line);
+        if (py?.groups) {
+            out.push(frame(py.groups.file, py.groups.line, py.groups.fn ?? ''));
+            continue;
+        }
+        // Java and C#: "at com.shop.Pay.charge(Pay.java:42)"
+        const jvm = /^\s*at\s+(?<fn>[\w.$<>]+)\((?<file>[^):]+):(?<line>\d+)\)/.exec(line);
+        if (jvm?.groups)
+            out.push(frame(jvm.groups.file, jvm.groups.line, jvm.groups.fn));
+    }
+    return out;
+}
+/** Sentry's event JSON, as its API and its "download JSON" button produce it. */
+function fromSentry(event) {
+    const thrown = event.exception?.values?.at?.(-1) ?? event.exception?.values?.[0];
+    if (!thrown && !event.title && !event.message)
+        return null;
+    const frames = (thrown?.stacktrace?.frames ?? [])
+        .map((f) => frame(f.abs_path ?? f.filename ?? '', f.lineno, f.function ?? ''))
+        // Sentry lists the oldest frame first; a person reads the newest first, and so does a fix.
+        .reverse();
+    const request = event.request ?? {};
+    return {
+        title: String(event.title ?? thrown?.value ?? event.message ?? 'an incident'),
+        errorType: String(thrown?.type ?? ''),
+        errorMessage: String(thrown?.value ?? event.message ?? ''),
+        frames,
+        operation: [request.method, request.url].filter(Boolean).join(' ') || String(event.culprit ?? ''),
+        when: String(event.datetime ?? event.timestamp ?? ''),
+        source: 'sentry',
+    };
+}
+/** An OpenTelemetry span that failed, carrying the standard `exception` event. */
+function fromOtel(span) {
+    const events = span.events ?? span.Events ?? [];
+    const thrown = events.find?.((e) => (e.name ?? '') === 'exception');
+    const status = span.status?.code ?? span.status?.Code ?? '';
+    if (!thrown && !/ERROR|STATUS_CODE_ERROR|2/.test(String(status)))
+        return null;
+    const attrs = {};
+    // OTLP writes attributes either as a map or as a list of {key,value}; both are seen in exports.
+    const raw = thrown?.attributes ?? {};
+    if (Array.isArray(raw))
+        for (const a of raw)
+            attrs[a.key] = a.value?.stringValue ?? a.value?.string_value ?? a.value;
+    else
+        Object.assign(attrs, raw);
+    const stack = String(attrs['exception.stacktrace'] ?? '');
+    return {
+        title: String(span.name ?? attrs['exception.type'] ?? 'a failed operation'),
+        errorType: String(attrs['exception.type'] ?? ''),
+        errorMessage: String(attrs['exception.message'] ?? span.status?.message ?? ''),
+        frames: framesFromStack(stack),
+        operation: String(span.name ?? ''),
+        when: String(span.startTime ?? span.startTimeUnixNano ?? ''),
+        source: 'opentelemetry',
+    };
+}
+/** Whatever was pasted: the first line is the error, the rest is the stack. */
+function fromText(text) {
+    const lines = text.split('\n');
+    const head = lines.find((l) => l.trim()) ?? '';
+    const typed = /^\s*(?<type>[\w.]*(?:Error|Exception|Fault))\b:?\s*(?<message>.*)$/.exec(head);
+    return {
+        title: head.trim().slice(0, 160) || 'an incident',
+        errorType: typed?.groups?.type ?? '',
+        errorMessage: (typed?.groups?.message ?? head).trim(),
+        frames: framesFromStack(text),
+        operation: '',
+        when: '',
+        source: 'stack trace',
+    };
+}
+/** Reads an incident from a file's contents, whichever of the three shapes it is in. */
+export function readIncident(contents) {
+    const text = contents.trim();
+    if (text.startsWith('{') || text.startsWith('[')) {
+        try {
+            const parsed = JSON.parse(text);
+            const one = Array.isArray(parsed) ? parsed[0] : parsed;
+            // A Sentry event and an OTel span are both objects; each is recognised by what it carries.
+            const sentry = one?.exception || one?.culprit || one?.event_id ? fromSentry(one) : null;
+            if (sentry)
+                return sentry;
+            const otel = fromOtel(one?.resourceSpans?.[0]?.scopeSpans?.[0]?.spans?.[0] ?? one?.span ?? one);
+            if (otel)
+                return otel;
+        }
+        catch {
+            // Not JSON after all: fall through and read it as a pasted stack trace.
+        }
+    }
+    return fromText(text);
+}
+/** The frames in the project's own code, newest first: where a reproduction has to aim. */
+export const ourFrames = (incident) => incident.frames.filter((f) => f.ours);

package/dist/engine/incident/reproduced.js

@@ -0,0 +1,87 @@
+// Did that test actually reproduce the incident?
+//
+// This is the whole of Phase 16 in one question, and getting it wrong is how a tool like this
+// becomes worthless. A test that fails is not a reproduction. A test with a typo fails. A test
+// against a missing fixture fails. A test that asserts the opposite of the truth fails. Each of
+// those would let Tewip announce "reproduced your incident" and hand over something that proves
+// nothing, and then the fix that makes it pass would fix nothing.
+//
+// So a reproduction has to fail **for the incident's reason**, and that is checkable without a
+// model: the error the test produced and the error production reported have to be the same error.
+// Everything else - a pass, a failure of a different kind, a crash before the test ran - is
+// refused and said out loud.
+import { errorShape } from '../signature.js';
+/** Words that mean the test never got as far as exercising anything. */
+const NEVER_RAN = /\b(?:cannot find module|ModuleNotFoundError|No module named|SyntaxError|IndentationError|fixture '[^']+' not found|ERR_MODULE_NOT_FOUND|NoClassDefFoundError|ClassNotFoundException|command not found|no tests? (?:found|ran)|collection failure)\b/i;
+const typeOf = (text) => /\b([\w.]*(?:Error|Exception|Fault))\b/.exec(text)?.[1]?.split('.').pop() ?? '';
+/** The words of an error, without the parts that differ every run. */
+const wordsOf = (text) => new Set(errorShape(text)
+    .toLowerCase()
+    .match(/[a-z][a-z-]{2,}/g)
+    ?.filter((w) => !['error', 'exception', 'the', 'and', 'for', 'with', 'from', 'test', 'failed', 'assert'].includes(w)) ?? []);
+/**
+ * Judges one test run against the incident it was written for.
+ *
+ * `passed` is not a near miss. It means the test does not exercise the broken path at all, and the
+ * honest thing is to say the reproduction failed, not to weaken the test until it goes red.
+ */
+export function judgeReproduction(incident, run) {
+    if (NEVER_RAN.test(run.output)) {
+        return {
+            verdict: 'never-ran',
+            why: 'the test never ran - something was missing or would not compile - so it shows nothing about the incident',
+            match: 'none',
+        };
+    }
+    if (run.passed) {
+        return {
+            verdict: 'passed',
+            why: 'the test passes against the code that is broken in production, so it does not exercise what broke. It is not a reproduction, and weakening it until it fails would prove nothing',
+            match: 'none',
+        };
+    }
+    const wanted = `${incident.errorType} ${incident.errorMessage}`.trim();
+    const sameType = Boolean(incident.errorType) && typeOf(run.output) === typeOf(wanted);
+    const theirs = wordsOf(wanted);
+    const ours = wordsOf(run.output);
+    const shared = [...theirs].filter((w) => ours.has(w));
+    // Most of the incident's own words, not most of the test output's: a long stack trace should not
+    // earn a match by sheer length.
+    const overlap = theirs.size > 0 ? shared.length / theirs.size : 0;
+    if (sameType && overlap >= 0.5) {
+        return { verdict: 'reproduced', why: `the test fails with the same error production reported (${typeOf(wanted)}), and most of the same words`, match: 'same type and message' };
+    }
+    if (sameType) {
+        return { verdict: 'reproduced', why: `the test fails with ${typeOf(wanted)}, the same error type production reported, though the message differs`, match: 'same type' };
+    }
+    if (overlap >= 0.7 && theirs.size >= 3) {
+        return { verdict: 'reproduced', why: 'the test fails with the same message production reported, although neither names an error type', match: 'same message' };
+    }
+    return {
+        verdict: 'failed-differently',
+        why: `the test fails, but not with the incident's error: production reported ${wanted.slice(0, 80) || 'its own error'}, and the test reported ${errorLine(run.output)}`,
+        match: 'none',
+    };
+}
+/**
+ * The line of a runner's output that is actually the error. Quoting the first non-empty line put
+ * "RUN v5.0.2 C:/Users/..." into a real report as though it were what the test said - a banner, not
+ * a finding, and exactly the loose evidence this project refuses everywhere else.
+ */
+function errorLine(output) {
+    // Two passes, because a runner prints its own status lines too: " FAIL tests/a.test.js" says
+    // which file failed, not what went wrong. A named error always wins over a status line.
+    const named = /\b(?:\w*Error|\w*Exception|expected|Expected|assert)\b/;
+    const anyFailure = /\b(?:FAIL|failed|failure)\b/i;
+    const noise = /^\s*(?:RUN|DEV|WATCH)\b/;
+    const lines = output.split('\n').filter((l) => l.trim() && !noise.test(l));
+    const found = lines.find((l) => named.test(l)) ?? lines.find((l) => anyFailure.test(l)) ?? lines[0];
+    return (found ?? '').trim().slice(0, 100);
+}
+/** What to do about each verdict, for the report. */
+export const nextStep = {
+    reproduced: 'keep this test: it is red for the reason production was, so a fix that turns it green has fixed the incident',
+    passed: 'do not keep this test. Either the incident depends on data or state the test does not create, or it is not in this code at all',
+    'failed-differently': 'do not keep this test: it is broken in its own way, which would be mistaken for the incident later',
+    'never-ran': 'fix the setup and try again; nothing has been shown either way',
+};

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "tewip-ai",
-  "version": "0.18.6",
+  "version": "0.20.0",
   "description": "Tewip finds why each failing test failed - real bug, flaky test, environment, test data or changed selector - for any stack, from the command line, CI, or your editor.",
   "keywords": [
     "test-failures",