tewip-ai 0.70.12 → 0.70.14

package/QUICKSTART.md

@@ -1,179 +1,179 @@
-# Quickstart: putting Tewip on one repository
-
-For an engineer adding the agent to a project that already runs Playwright tests in GitHub Actions. It takes about ten minutes, and the first useful output arrives with the next failing run.
-
-Start with **one** repository whose tests fail sometimes. A repository whose suite is always green has nothing to triage.
-
-## What the agent does once it is installed
-
-Every CI run, for every failed test, it decides the root cause — application bug, changed selector, race, environment, or test data — with a confidence, the evidence it used, and one concrete next action. It then delivers that to the people who need it: a comment on the pull request, the board in the job summary, one grouped issue for environment failures, and optionally a separate pull request with locator fixes that it re-ran and verified.
-
-It never merges anything, never edits application code, and never opens a fix pull request for a change it has not proved by re-running the test.
-
-## Step 1: make Playwright leave the evidence behind
-
-The agent reads Playwright's own blob report and traces, so nothing from this project goes into your test config:
-
-```ts
-// playwright.config.ts
-reporter: process.env.CI ? [['list'], ['blob']] : 'list',
-use: { trace: 'retain-on-failure' },
-```
-
-If your config already uses a reporter list in CI, add `['blob']` to it.
-
-## Step 2: add one step to the workflow that runs the tests
-
-```yaml
-permissions:
-  pull-requests: write     # the diagnosis comment
+# Quickstart: putting Tewip on one repository
+
+For an engineer adding the agent to a project that already runs Playwright tests in GitHub Actions. It takes about ten minutes, and the first useful output arrives with the next failing run.
+
+Start with **one** repository whose tests fail sometimes. A repository whose suite is always green has nothing to triage.
+
+## What the agent does once it is installed
+
+Every CI run, for every failed test, it decides the root cause — application bug, changed selector, race, environment, or test data — with a confidence, the evidence it used, and one concrete next action. It then delivers that to the people who need it: a comment on the pull request, the board in the job summary, one grouped issue for environment failures, and optionally a separate pull request with locator fixes that it re-ran and verified.
+
+It never merges anything, never edits application code, and never opens a fix pull request for a change it has not proved by re-running the test.
+
+## Step 1: make Playwright leave the evidence behind
+
+The agent reads Playwright's own blob report and traces, so nothing from this project goes into your test config:
+
+```ts
+// playwright.config.ts
+reporter: process.env.CI ? [['list'], ['blob']] : 'list',
+use: { trace: 'retain-on-failure' },
+```
+
+If your config already uses a reporter list in CI, add `['blob']` to it.
+
+## Step 2: add one step to the workflow that runs the tests
+
+```yaml
+permissions:
+  pull-requests: write     # the diagnosis comment
   issues: write
-  actions: read          # so it can read a failed job's log when no report was written            # the grouped environment issue
-  contents: write          # only if you turn autofix on
-
-steps:
-  # ... your existing checkout, install, and `npx playwright test` steps ...
-
-  - name: Tewip
-    if: always()           # it must run even though the test step failed
-    uses: hmamut39/tewip@main
-    with:
-      openai-api-key: ${{ secrets.OPENAI_API_KEY }}   # optional, see step 3
-      autofix: 'false'                                # start with this off
-```
-
-`if: always()` matters: without it the step is skipped exactly when there is something to triage.
-
-Keep `autofix: 'false'` for the first week. Read what the agent says, decide whether you trust it, and turn fixing on afterwards.
-
-## Step 3: decide whether to give it an API key
-
-| Without a key | With `OPENAI_API_KEY` |
-|---|---|
-| Only the deterministic rules run | The rules run first, and only what they cannot settle goes to the model |
-| Every answer is explainable from the error, the trace and the test's history | Adds a judgement on the rest, with its reasoning and evidence shown |
-| Free | A few cents per run in practice, because the rules answer most failures |
-
-On real projects the rules alone decide most failures and are right on 96–98% of what they decide, so starting without a key is a reasonable first week. To add one later: repository **Settings → Secrets and variables → Actions → New repository secret**, named `OPENAI_API_KEY`.
-
-Every run prints what it cost, priced from OpenAI's published prices. One guard worth setting once the key is in place:
-
-```yaml
-env:
-  TRIAGE_LLM_MAX_CALLS: '80'     # hard cap on model calls per run
-  # TRIAGE_LLM_PRICE_IN / _OUT: only if your price differs (per million tokens)
-```
-
-## Not using Playwright? Every stack works
-
-Point the agent at your test runner's **JUnit XML** report (or .NET's **TRX**) instead of a Playwright blob report. Each runner marked *verified* was run for real in CI (`fixtures/polyglot`) and its report is checked by the test suite on every change.
-
-| Stack | Runner | How to write the report | Status |
-|---|---|---|---|
-| **Node.js** | node:test (built in) | `node --test --test-reporter=junit --test-reporter-destination=report.xml` | verified |
-| Node.js, React, Vue, Angular, Next.js, NestJS, Express | Jest | `jest --reporters=default --reporters=jest-junit` | verified |
-| Vite, Vue, Svelte, Nuxt, React | Vitest | `vitest run --reporter=junit --outputFile=report.xml` | verified |
-| Node.js, Express | Mocha | `mocha --reporter mocha-junit-reporter` | verified |
-| **Python**, Django, Flask, FastAPI | pytest (also runs `unittest`) | `pytest --junitxml=report.xml` (retries: `pytest-rerunfailures`) | verified |
-| **Java**, Spring, Kotlin | JUnit 4/5, TestNG via Maven Surefire | written by default to `target/surefire-reports/` (retries: `rerunFailingTestsCount`) | verified |
-| Java, Kotlin, Android | Gradle | written by default to `build/test-results/test/` | same format as Surefire |
-| **Go** | gotestsum | `gotestsum --junitfile report.xml` | verified |
-| **.NET** (C#, F#), ASP.NET | xUnit, NUnit, MSTest | `dotnet test --logger trx` (native) or `--logger junit` | both verified |
-| **Ruby**, Rails | RSpec | `rspec --format RspecJunitFormatter --out report.xml` | verified |
-| **PHP**, Laravel, Symfony | PHPUnit | `phpunit --log-junit report.xml` | verified |
-| **Rust** | cargo-nextest | `[profile.ci.junit]` in `.config/nextest.toml` | verified |
-| End-to-end | **Selenium** (Java, Python, JavaScript, C#) | through the runner above it: Surefire, pytest, Jest/Mocha, `dotnet test` | JUnit format; error wording read |
-| End-to-end | Cypress, WebdriverIO | `mocha-junit-reporter` / `@wdio/junit-reporter` | JUnit format; error wording read |
-| End-to-end | **Playwright** | blob reporter (step 1) — adds traces, selector drift and verified fixes | verified on 5 real projects |
-
-```yaml
-      - name: Tewip
-        if: always()
-        uses: hmamut39/tewip@main
-        with:
-          junit-report: report.xml     # a file, or a directory of them (.xml and .trx)
-          suite: unit                  # only needed when one workflow triages several suites
-```
-
-What it reads from any of these: the failure text, retries, and each test's history. From that it tells apart a wrong value (a real bug once the test has a passing history), an unreachable service (environment), a data collision, and a test that failed then passed on a retry (timing), in the wording each language uses: `expected: <4> but was: <3>` in Java, `got 3, want 4` in Go, `AssertionError: 3 != 4` in Python, `Expected: 4 / Actual: 3` in .NET, and so on. It also reads each browser tool's own words for the same three problems: an element that was never found (Selenium's `NoSuchElementException`, WebdriverIO's `still not displayed`, Cypress's `Expected to find element`), a click another element received (Selenium's `Other element would receive the click`, Cypress's `is being covered by another element`), and an element the page replaced mid-test (Selenium's `StaleElementReferenceException`, Cypress's `the page updated while this command was executing`). No labelled Selenium or Cypress runs have measured those rules yet, so they report at a confidence the merge gate does not trust on its own. What it cannot do without Playwright's traces is look at the page, so selector drift and automatic fixes stay Playwright-only.
-
-A wrong value on a test's **first** recorded failure stays undecided rather than guessed: one failure alone does not show whether the test was reliable before. It becomes a real-bug diagnosis once the test's history shows it passing and then failing the same way.
-
-When one workflow triages several suites (say `unit` and `e2e`), give each a `suite` name: each then keeps its own history and its own comment, and corrections name the suite, as in `/triage TIMING unit#2 flaky on CI`.
-
-## On your own machine, without CI
-
-Install the command once from the release file, `npm install -g tewip-<version>.tgz` (Tewip is not on the npm registry yet; Node 22.18 or newer), run your tests with the report switched on (see the table above), then:
-
-```bash
-tewip explain report.xml          # JUnit XML or .trx
-tewip explain reports/            # a folder of them
-tewip explain blob-report/        # Playwright
-tewip ci owner/repo --pr 12       # what Tewip's GitHub Action said about a pull request
-```
-
-It prints the same triage board CI writes: each failure's cause, confidence, evidence and next action. Each run is kept in a `.tewip/` folder in your project (add it to `.gitignore`), so from the second run on, failures are judged against their history exactly as in CI. `--name` keeps separate histories for different projects.
-
-## What you will see on the next failing run
-
-- **On the pull request:** one comment listing each failing test, grouped by whether your change likely caused it, each with a cause, a confidence, the evidence and the next action. It updates in place rather than adding a comment per run.
-- **In the job summary:** the same failures as a triage board.
-- **As an issue:** environment failures (5xx responses, connection resets, outages) grouped into one issue instead of one per test.
-- **As an artifact** named `tewip-report`: every role's view, including the engineering-quality view with the flaky tests ranked by the CI minutes they cost.
-
-## Correcting it
-
-When it gets one wrong, reply to its comment:
-
-```
-/triage TIMING #2 this one is flaky on our CI runners, not a bug
-```
-
-From the next run on, that failure of that test gets your answer, shown as "corrected by a person" with your name and reason. A correction settles the cause; it never makes the agent change code.
-
-## Asking it questions from your editor
-
-- **VS Code, Cursor, Windsurf, VSCodium:** install the Tewip extension (*Extensions → … → Install from VSIX*, `tewip-<version>.vsix`). It connects Tewip to the editor's AI assistant by itself and adds *Tewip: Explain a test report…* and *Tewip: Why did CI fail on this branch?*
-- **JetBrains (2025.2+), Claude, and other MCP clients:** add one server whose command is `tewip mcp`.
-
-Then ask, in plain language: "why did CI fail on my PR?", "which tests are flaky in checkout?", "is main safe to release?". Every tool is read-only, and it reads the report the Action already uploaded, so you do not need the other repository cloned.
-
-## Stop flaky tests from blocking merges (optional)
-
-Once you trust its answers, let the agent decide whether a failed run should block:
-
-```yaml
-      - name: Tests
-        continue-on-error: true      # the gate below decides
-        run: npx playwright test
-      - name: Tewip
-        if: always()
-        uses: hmamut39/tewip@main
-        with:
-          gate: real-failures
-```
-
-The build then fails only for real bugs, changed selectors, test-data problems and anything undecided. Failures a rule or a person classified as flaky or environment no longer block. On 2,296 real failures this unblocked 84% of the flaky and environment ones without letting a real bug through.
-
-## Turning fixing on, once you trust it
-
-```yaml
-      autofix: 'true'
-```
-
-It then rewrites a drifted locator, re-runs that single test, and opens a **separate** pull request only with the changes that passed. Review it like any other pull request: an intended UI change and a regression can look identical from the outside, which is exactly why a human merges it.
-
-On real WordPress CI data it would have fixed 14 drifted locators with none wrong, and every locator it suggested matched what the developers themselves wrote.
-
-## If something does not appear
-
-| Symptom | Cause |
-|---|---|
-| The step ran but says "no blob report" | The blob reporter is not on in CI, or the tests step did not run |
-| No comment on the pull request | The run was not triggered by a pull request, or `pull-requests: write` is missing |
-| "Resource not accessible by integration" | The workflow's permissions block is missing an entry |
-| Diagnoses look thin on the first day | The agent judges each failure against that test's history, which it builds from your runs; it gets better as runs accumulate |
-
-## What it costs, and what it keeps
-
-The agent stores the run history in the Actions cache, never in your repository. Collected failure data and traces stay in CI. The only thing it writes to your repository is a pull request branch, and only when you turn fixing on.
+  actions: read          # so it can read a failed job's log when no report was written            # the grouped environment issue
+  contents: write          # only if you turn autofix on
+
+steps:
+  # ... your existing checkout, install, and `npx playwright test` steps ...
+
+  - name: Tewip
+    if: always()           # it must run even though the test step failed
+    uses: hmamut39/tewip@main
+    with:
+      openai-api-key: ${{ secrets.OPENAI_API_KEY }}   # optional, see step 3
+      autofix: 'false'                                # start with this off
+```
+
+`if: always()` matters: without it the step is skipped exactly when there is something to triage.
+
+Keep `autofix: 'false'` for the first week. Read what the agent says, decide whether you trust it, and turn fixing on afterwards.
+
+## Step 3: decide whether to give it an API key
+
+| Without a key | With `OPENAI_API_KEY` |
+|---|---|
+| Only the deterministic rules run | The rules run first, and only what they cannot settle goes to the model |
+| Every answer is explainable from the error, the trace and the test's history | Adds a judgement on the rest, with its reasoning and evidence shown |
+| Free | A few cents per run in practice, because the rules answer most failures |
+
+On real projects the rules alone decide most failures and are right on 96–98% of what they decide, so starting without a key is a reasonable first week. To add one later: repository **Settings → Secrets and variables → Actions → New repository secret**, named `OPENAI_API_KEY`.
+
+Every run prints what it cost, priced from OpenAI's published prices. One guard worth setting once the key is in place:
+
+```yaml
+env:
+  TRIAGE_LLM_MAX_CALLS: '80'     # hard cap on model calls per run
+  # TRIAGE_LLM_PRICE_IN / _OUT: only if your price differs (per million tokens)
+```
+
+## Not using Playwright? Every stack works
+
+Point the agent at your test runner's **JUnit XML** report (or .NET's **TRX**) instead of a Playwright blob report. Each runner marked *verified* was run for real in CI (`fixtures/polyglot`) and its report is checked by the test suite on every change.
+
+| Stack | Runner | How to write the report | Status |
+|---|---|---|---|
+| **Node.js** | node:test (built in) | `node --test --test-reporter=junit --test-reporter-destination=report.xml` | verified |
+| Node.js, React, Vue, Angular, Next.js, NestJS, Express | Jest | `jest --reporters=default --reporters=jest-junit` | verified |
+| Vite, Vue, Svelte, Nuxt, React | Vitest | `vitest run --reporter=junit --outputFile=report.xml` | verified |
+| Node.js, Express | Mocha | `mocha --reporter mocha-junit-reporter` | verified |
+| **Python**, Django, Flask, FastAPI | pytest (also runs `unittest`) | `pytest --junitxml=report.xml` (retries: `pytest-rerunfailures`) | verified |
+| **Java**, Spring, Kotlin | JUnit 4/5, TestNG via Maven Surefire | written by default to `target/surefire-reports/` (retries: `rerunFailingTestsCount`) | verified |
+| Java, Kotlin, Android | Gradle | written by default to `build/test-results/test/` | same format as Surefire |
+| **Go** | gotestsum | `gotestsum --junitfile report.xml` | verified |
+| **.NET** (C#, F#), ASP.NET | xUnit, NUnit, MSTest | `dotnet test --logger trx` (native) or `--logger junit` | both verified |
+| **Ruby**, Rails | RSpec | `rspec --format RspecJunitFormatter --out report.xml` | verified |
+| **PHP**, Laravel, Symfony | PHPUnit | `phpunit --log-junit report.xml` | verified |
+| **Rust** | cargo-nextest | `[profile.ci.junit]` in `.config/nextest.toml` | verified |
+| End-to-end | **Selenium** (Java, Python, JavaScript, C#) | through the runner above it: Surefire, pytest, Jest/Mocha, `dotnet test` | JUnit format; error wording read |
+| End-to-end | Cypress, WebdriverIO | `mocha-junit-reporter` / `@wdio/junit-reporter` | JUnit format; error wording read |
+| End-to-end | **Playwright** | blob reporter (step 1) — adds traces, selector drift and verified fixes | verified on 5 real projects |
+
+```yaml
+      - name: Tewip
+        if: always()
+        uses: hmamut39/tewip@main
+        with:
+          junit-report: report.xml     # a file, or a directory of them (.xml and .trx)
+          suite: unit                  # only needed when one workflow triages several suites
+```
+
+What it reads from any of these: the failure text, retries, and each test's history. From that it tells apart a wrong value (a real bug once the test has a passing history), an unreachable service (environment), a data collision, and a test that failed then passed on a retry (timing), in the wording each language uses: `expected: <4> but was: <3>` in Java, `got 3, want 4` in Go, `AssertionError: 3 != 4` in Python, `Expected: 4 / Actual: 3` in .NET, and so on. It also reads each browser tool's own words for the same three problems: an element that was never found (Selenium's `NoSuchElementException`, WebdriverIO's `still not displayed`, Cypress's `Expected to find element`), a click another element received (Selenium's `Other element would receive the click`, Cypress's `is being covered by another element`), and an element the page replaced mid-test (Selenium's `StaleElementReferenceException`, Cypress's `the page updated while this command was executing`). No labelled Selenium or Cypress runs have measured those rules yet, so they report at a confidence the merge gate does not trust on its own. What it cannot do without Playwright's traces is look at the page, so selector drift and automatic fixes stay Playwright-only.
+
+A wrong value on a test's **first** recorded failure stays undecided rather than guessed: one failure alone does not show whether the test was reliable before. It becomes a real-bug diagnosis once the test's history shows it passing and then failing the same way.
+
+When one workflow triages several suites (say `unit` and `e2e`), give each a `suite` name: each then keeps its own history and its own comment, and corrections name the suite, as in `/triage TIMING unit#2 flaky on CI`.
+
+## On your own machine, without CI
+
+Install the command once from the release file, `npm install -g tewip-<version>.tgz` (Tewip is not on the npm registry yet; Node 22.18 or newer), run your tests with the report switched on (see the table above), then:
+
+```bash
+tewip explain report.xml          # JUnit XML or .trx
+tewip explain reports/            # a folder of them
+tewip explain blob-report/        # Playwright
+tewip ci owner/repo --pr 12       # what Tewip's GitHub Action said about a pull request
+```
+
+It prints the same triage board CI writes: each failure's cause, confidence, evidence and next action. Each run is kept in a `.tewip/` folder in your project (add it to `.gitignore`), so from the second run on, failures are judged against their history exactly as in CI. `--name` keeps separate histories for different projects.
+
+## What you will see on the next failing run
+
+- **On the pull request:** one comment listing each failing test, grouped by whether your change likely caused it, each with a cause, a confidence, the evidence and the next action. It updates in place rather than adding a comment per run.
+- **In the job summary:** the same failures as a triage board.
+- **As an issue:** environment failures (5xx responses, connection resets, outages) grouped into one issue instead of one per test.
+- **As an artifact** named `tewip-report`: every role's view, including the engineering-quality view with the flaky tests ranked by the CI minutes they cost.
+
+## Correcting it
+
+When it gets one wrong, reply to its comment:
+
+```
+/triage TIMING #2 this one is flaky on our CI runners, not a bug
+```
+
+From the next run on, that failure of that test gets your answer, shown as "corrected by a person" with your name and reason. A correction settles the cause; it never makes the agent change code.
+
+## Asking it questions from your editor
+
+- **VS Code, Cursor, Windsurf, VSCodium:** install the Tewip extension (*Extensions → … → Install from VSIX*, `tewip-<version>.vsix`). It connects Tewip to the editor's AI assistant by itself and adds *Tewip: Explain a test report…* and *Tewip: Why did CI fail on this branch?*
+- **JetBrains (2025.2+), Claude, and other MCP clients:** add one server whose command is `tewip mcp`.
+
+Then ask, in plain language: "why did CI fail on my PR?", "which tests are flaky in checkout?", "is main safe to release?". Every tool is read-only, and it reads the report the Action already uploaded, so you do not need the other repository cloned.
+
+## Stop flaky tests from blocking merges (optional)
+
+Once you trust its answers, let the agent decide whether a failed run should block:
+
+```yaml
+      - name: Tests
+        continue-on-error: true      # the gate below decides
+        run: npx playwright test
+      - name: Tewip
+        if: always()
+        uses: hmamut39/tewip@main
+        with:
+          gate: real-failures
+```
+
+The build then fails only for real bugs, changed selectors, test-data problems and anything undecided. Failures a rule or a person classified as flaky or environment no longer block. On 2,296 real failures this unblocked 84% of the flaky and environment ones without letting a real bug through.
+
+## Turning fixing on, once you trust it
+
+```yaml
+      autofix: 'true'
+```
+
+It then rewrites a drifted locator, re-runs that single test, and opens a **separate** pull request only with the changes that passed. Review it like any other pull request: an intended UI change and a regression can look identical from the outside, which is exactly why a human merges it.
+
+On real WordPress CI data it would have fixed 14 drifted locators with none wrong, and every locator it suggested matched what the developers themselves wrote.
+
+## If something does not appear
+
+| Symptom | Cause |
+|---|---|
+| The step ran but says "no blob report" | The blob reporter is not on in CI, or the tests step did not run |
+| No comment on the pull request | The run was not triggered by a pull request, or `pull-requests: write` is missing |
+| "Resource not accessible by integration" | The workflow's permissions block is missing an entry |
+| Diagnoses look thin on the first day | The agent judges each failure against that test's history, which it builds from your runs; it gets better as runs accumulate |
+
+## What it costs, and what it keeps
+
+The agent stores the run history in the Actions cache, never in your repository. Collected failure data and traces stay in CI. The only thing it writes to your repository is a pull request branch, and only when you turn fixing on.

package/dist/engine/autofix/code-propose.js

@@ -1,12 +1,12 @@
 export const CODE_FIX_PROMPT_VERSION = 'code-fix-1';
-const SYSTEM = `You are Tewip's fixer. A test is failing against this code. Find the smallest change to the APPLICATION code that makes the test pass because the bug is gone, not because the check was avoided.
-
-Rules:
-- Change application code only. Never suggest changing the test, the expected values in it, configuration, or anything that silences the failure.
-- The change must be minimal and local: the fewest lines that fix the cause. Do not refactor, rename, reformat, add logging, or "improve" anything nearby.
-- before must be copied character for character from the file you were given, including indentation, and must appear in it exactly once. Include enough surrounding lines to be unique. Copy only what is in the file: nothing is added to it, so do not add line numbers or markers.
-- after must keep the file valid in its language and keep the behaviour every other test relies on.
-- If the code is right and the test is wrong, or if you cannot see the cause in the file you were given, set fault to "test" and return before and after as empty strings. Saying "I cannot see it" is a correct answer; a guess is not.
+const SYSTEM = `You are Tewip's fixer. A test is failing against this code. Find the smallest change to the APPLICATION code that makes the test pass because the bug is gone, not because the check was avoided.
+
+Rules:
+- Change application code only. Never suggest changing the test, the expected values in it, configuration, or anything that silences the failure.
+- The change must be minimal and local: the fewest lines that fix the cause. Do not refactor, rename, reformat, add logging, or "improve" anything nearby.
+- before must be copied character for character from the file you were given, including indentation, and must appear in it exactly once. Include enough surrounding lines to be unique. Copy only what is in the file: nothing is added to it, so do not add line numbers or markers.
+- after must keep the file valid in its language and keep the behaviour every other test relies on.
+- If the code is right and the test is wrong, or if you cannot see the cause in the file you were given, set fault to "test" and return before and after as empty strings. Saying "I cannot see it" is a correct answer; a guess is not.
 - why is for a reviewer: what was wrong and why this fixes it.`;
 const SCHEMA = {
     type: 'object',

package/dist/engine/autofix/selector-source.js

@@ -62,7 +62,8 @@ function commonPrefix(a, b) {
 }
 const git = (root, args) => {
     try {
-        return execFileSync('git', args, { cwd: root, encoding: 'utf8', maxBuffer: 32 * 1024 * 1024, timeout: 60_000 });
+        // Captured, not inherited: see the note in grounding.ts.
+        return execFileSync('git', args, { cwd: root, encoding: 'utf8', maxBuffer: 32 * 1024 * 1024, timeout: 60_000, stdio: ['ignore', 'pipe', 'pipe'] });
     }
     catch {
         return '';

package/dist/engine/build/build.js

@@ -1,29 +1,29 @@
 export const BUILD_PROMPT_VERSION = 'build-1';
-const RULES = `Rules that do not bend:
-- Use only what the project already has: its framework, its style, its helpers, its way of importing. Copy the shape of the example test and the example source given.
-- Never weaken or delete an existing test, and never change what one asserts.
-- Keep the change small and local. Add a file when the project's own layout calls for one; do not reorganise anything.
-- Write no comments that restate the code, no logging, no defensive try/catch around things that cannot fail.
+const RULES = `Rules that do not bend:
+- Use only what the project already has: its framework, its style, its helpers, its way of importing. Copy the shape of the example test and the example source given.
+- Never weaken or delete an existing test, and never change what one asserts.
+- Keep the change small and local. Add a file when the project's own layout calls for one; do not reorganise anything.
+- Write no comments that restate the code, no logging, no defensive try/catch around things that cannot fail.
 - before must be copied character for character from the file given and appear in it exactly once. For a new file, before is empty and after is the whole file.`;
-const TEST_SYSTEM = `You are Tewip, writing the test for a feature that does not exist yet.
-
-Write ONE test that checks the described behaviour, in the project's own framework and style. It must fail when run now, and fail for the right reason: the behaviour is missing, not because of a typo, a missing import or a file that does not exist. Use only functions, files and exports that already exist, or the one new export the feature obviously needs - name it as the project would.
-
-checks is one sentence naming the behaviour the test asserts ("a full URL keeps its query when the trailing slash is removed"). It is not advice about file names or where to put things.
-failsBecause is the behaviour that is missing today, in one sentence.
-
+const TEST_SYSTEM = `You are Tewip, writing the test for a feature that does not exist yet.
+
+Write ONE test that checks the described behaviour, in the project's own framework and style. It must fail when run now, and fail for the right reason: the behaviour is missing, not because of a typo, a missing import or a file that does not exist. Use only functions, files and exports that already exist, or the one new export the feature obviously needs - name it as the project would.
+
+checks is one sentence naming the behaviour the test asserts ("a full URL keeps its query when the trailing slash is removed"). It is not advice about file names or where to put things.
+failsBecause is the behaviour that is missing today, in one sentence.
+
 ${RULES}`;
-const CODE_SYSTEM = `You are Tewip, writing the code that makes a failing test pass.
-
-You are given the feature, the test, the real error it produced, and the project's relevant source. Write the smallest change that makes that test pass because the behaviour now exists - never by weakening the test, never by special-casing the test's own values.
-
-If the feature cannot be built from what you were given - the code you need is not here, or the request is ambiguous in a way that changes the answer - set outcome to "cannot" and return no edits. Saying "I cannot" is a correct answer; a guess that happens to pass one test is not.
-
+const CODE_SYSTEM = `You are Tewip, writing the code that makes a failing test pass.
+
+You are given the feature, the test, the real error it produced, and the project's relevant source. Write the smallest change that makes that test pass because the behaviour now exists - never by weakening the test, never by special-casing the test's own values.
+
+If the feature cannot be built from what you were given - the code you need is not here, or the request is ambiguous in a way that changes the answer - set outcome to "cannot" and return no edits. Saying "I cannot" is a correct answer; a guess that happens to pass one test is not.
+
 ${RULES}`;
-const REPAIR_SYSTEM = `You are Tewip, fixing code you just wrote. The test still fails, or other tests broke. You are given what you changed and the real output.
-
-Change your own code so the failing test passes and everything that was passing still passes. Never change the test. If you cannot, set outcome to "cannot".
-
+const REPAIR_SYSTEM = `You are Tewip, fixing code you just wrote. The test still fails, or other tests broke. You are given what you changed and the real output.
+
+Change your own code so the failing test passes and everything that was passing still passes. Never change the test. If you cannot, set outcome to "cannot".
+
 ${RULES}`;
 const TEST_SCHEMA = {
     type: 'object',

package/dist/engine/flaky/flaky-command.js

@@ -8,6 +8,9 @@ import { flakyTests, readyToRelease, weeklyCost } from './flaky.js';
 import { applyQuarantine, frameworkOf, quarantineFor } from './quarantine.js';
 import { loadVerdicts, saidByAPerson, verdictFor } from '../learn/verdicts.js';
 const DAY_MS = 24 * 60 * 60 * 1000;
+/** How many tests each group lists, and how many quarantine lines are written out in full. */
+const SHOW = 15;
+const SHOW_LINES = 5;
 const minutes = (ms) => `${(ms / 60_000).toFixed(1)} CI min`;
 function projects(root, only) {
     const dir = join(root, '.tewip', 'history');
@@ -50,7 +53,7 @@ export async function runFlaky(cmd) {
             if (chosen.length === 0)
                 return;
             lines.push(`### ${title} (${chosen.length})`, '', explain, '');
-            for (const test of chosen.slice(0, 15)) {
+            for (const test of chosen.slice(0, SHOW)) {
                 lines.push(`- **${test.title}**`);
                 lines.push(`  - ${test.failedRuns} of ${test.totalRuns} run(s), ${test.failedAttempts} failed attempt(s), ${minutes(test.wastedMs)} lost${test.recoveredOnRetry > 0 ? `, passed on a retry ${test.recoveredOnRetry} time(s)` : ''}`);
                 lines.push(`  - Tewip read them as: ${test.categories.map((c) => `${c.category} ×${c.count}`).join(', ')}`);
@@ -58,6 +61,10 @@ export async function runFlaky(cmd) {
                 if (test.file)
                     lines.push(`  - Where: ${test.file}`);
             }
+            // A heading that says 48 above a list of 15 is the defect this project keeps finding in
+            // itself: a true number beside a shorter answer, with nothing saying they differ.
+            if (chosen.length > SHOW)
+                lines.push(`_Showing the ${SHOW} costliest. The other ${chosen.length - SHOW} are not listed here._`);
             lines.push('');
         };
         group('quarantine', 'Quarantine these', 'Unreliable rather than wrong. Stop them blocking everyone, then fix them on purpose.');
@@ -82,6 +89,7 @@ export async function runFlaky(cmd) {
         lines.push(...applyAll(cmd.root, toQuarantine));
     else if (toQuarantine.length > 0)
         lines.push(...suggest(cmd.root, toQuarantine));
+    lines.push(...notChecked(all, cmd.days));
     const report = lines.join('\n');
     console.log(report);
     if (cmd.out) {
@@ -97,7 +105,7 @@ function readQuarantined(root) {
 }
 function suggest(root, tests) {
     const lines = ['### The line to add', ''];
-    for (const test of tests.slice(0, 5)) {
+    for (const test of tests.slice(0, SHOW_LINES)) {
         const path = testFile(root, test);
         const framework = path ? frameworkOf(path, readFileSync(path, 'utf8')) : null;
         if (!path || !framework) {
@@ -106,7 +114,10 @@ function suggest(root, tests) {
         }
         lines.push(`- ${test.title} (${test.file}):`, '```', quarantineFor(test, framework).line, '```');
     }
-    lines.push('', 'Run `tewip flaky --apply` to put these lines in, each with the evidence above it. Only the named tests are touched.', '');
+    const shown = Math.min(tests.length, SHOW_LINES);
+    lines.push('', shown < tests.length
+        ? `Showing ${shown} of ${tests.length}. \`tewip flaky --apply\` quarantines **all ${tests.length}**, not only the ${shown} printed here - each with its own evidence written above it. No test outside those ${tests.length} is touched.`
+        : 'Run `tewip flaky --apply` to put these lines in, each with the evidence above it. Only the named tests are touched.', '');
     return lines;
 }
 /** Writes the quarantine into the test files, and remembers which tests were quarantined. */
@@ -148,3 +159,28 @@ function applyAll(root, tests) {
     lines.push('', 'Nothing else in those files was changed. Review the diff before committing: a quarantined test is a decision, not a fix.', '');
     return lines;
 }
+/**
+ * What this report did not check.
+ *
+ * The quarantine rules in `decide` carry a rail: a test whose failures Tewip read as the
+ * application being wrong is never quarantined, because hiding it would hide the bug. That rail
+ * reads the diagnosed category of each failure - so on history where nothing has been diagnosed it
+ * has nothing to read and cannot fire, and the verdict rests on retry behaviour alone. On real
+ * history that is the normal case rather than the exception: of the 15 quarantine candidates this
+ * printed for WordPress/gutenberg, all 15 were UNKNOWN - every one. The recommendation is still the
+ * best reading of the evidence there is. What was missing was the sentence telling the person which
+ * of the two they were being shown.
+ */
+export function notChecked(tests, days) {
+    const quarantine = tests.filter((t) => t.verdict === 'quarantine');
+    const undiagnosed = quarantine.filter((t) => t.categories.every((c) => c.category === 'UNKNOWN'));
+    const lines = ['## What this did not check', ''];
+    lines.push(`- Any test that did not fail, and any run nobody put through \`tewip explain\` or \`tewip report\`. These numbers are read from kept history of the last ${days} day(s), and a suite nobody recorded looks exactly like a suite with no flaky tests.`);
+    if (undiagnosed.length > 0) {
+        lines.push(`- **${undiagnosed.length} of the ${quarantine.length} quarantine recommendation(s) rest on retry behaviour alone.** None of those failures had been diagnosed, so the rule that refuses to quarantine a test whose failures looked like the application being wrong had nothing to read and did not fire. A test that fails and then passes on a retry can also be a test correctly catching a race in the product - quarantine that one and the race is what you hide. Running \`tewip explain\` over those runs is what tells the two apart.`);
+    }
+    lines.push('- Whether a test is the only thing covering what it covers. Tewip does not know what stops being tested when you quarantine it.');
+    lines.push('- Whether a test is the same test. These are matched by name, so a renamed test looks like a new one and its history stays behind with the old name.');
+    lines.push('');
+    return lines;
+}

package/dist/engine/import/blob.js

@@ -161,5 +161,5 @@ function importShards(shards, meta, filesDir) {
         status: runStatus,
         attempts,
     };
-    return { records, summary, skippedShards };
+    return { records, summary, skippedShards, incomplete: skippedShards.map((shard) => `shard ${shard} could not be read, so its tests are missing from this run`) };
 }

package/dist/engine/import/junit.js

@@ -10,9 +10,10 @@
 // selector drift is not diagnosed from them. Every format below was checked against a report its
 // real runner wrote (fixtures/polyglot, run by .github/workflows/polyglot-fixtures.yml).
 import { readdirSync, readFileSync, statSync } from 'node:fs';
-import { join } from 'node:path';
+import { basename, join } from 'node:path';
 import { SCHEMA_VERSION } from '../../reporters/types.js';
 import { descendants, parseXml } from './xml.js';
+import { shortfall } from './shortfall.js';
 import { jmeterCases, jmeterCsvCases, k6Cases, looksLikeCsvJtl, looksLikeK6, robotCases, robotTime } from './other-formats.js';
 import { nunitCases, trxCases } from './dotnet.js';
 import { PASSED, failure } from './case.js';
@@ -161,6 +162,7 @@ export function importJunit(path, meta) {
     let startedAt = '';
     let anyFailed = false;
     const unread = [];
+    const incomplete = [];
     let parsedFiles = 0;
     for (const file of junitFiles(path)) {
         const raw = readFileSync(file, 'utf8');
@@ -199,6 +201,9 @@ export function importJunit(path, meta) {
         }
         else if (root.name === 'testsuites' || root.name === 'testsuite' || root.name === 'testrun') {
             cases = junitCases(root, meta.workspace);
+            const missing = shortfall(root);
+            if (missing)
+                incomplete.push(`${basename(file)}: ${missing}`);
             const stamp = root.attrs.timestamp ?? root.children.find((c) => c.name === 'testsuite')?.attrs.timestamp;
             if (stamp && (!startedAt || stamp < startedAt))
                 startedAt = stamp;
@@ -271,5 +276,5 @@ export function importJunit(path, meta) {
         }
         console.error(`[tewip] ignored ${unread.length} file(s) that are not a test report: ${list}`);
     }
-    return { records, summary, skippedShards: [] };
+    return { records, summary, skippedShards: [], incomplete };
 }

package/dist/engine/import/shortfall.js

@@ -0,0 +1,62 @@
+// Does a test report hold the run it says it holds?
+//
+// Kept apart from the importers because it asks nothing about any one format's shape: it compares a
+// report's own declared totals with what is actually inside it, which is the one check that catches
+// a file that parsed cleanly and is missing most of its contents.
+import { descendants } from './xml.js';
+/** A failing outcome, under either of the two names JUnit uses for one. */
+const FAILURE = new Set(['failure', 'error']);
+/**
+ * What the report says about itself, against what it actually contains.
+ *
+ * Every JUnit suite declares its own totals - `tests`, `failures`, `errors` - in the attributes of
+ * the element that holds the results. Nothing here ever read them, and a truncated upload is the
+ * commonest corruption there is: the runner dies, or the artifact is cut off mid-write, and what
+ * arrives is well-formed enough to parse and missing most of the run.
+ *
+ * Found by handing `tewip explain` a report whose header said `tests="500" failures="37"` and whose
+ * body held one passing testcase. The answer was **"All 1 test(s) passed. Nothing to explain."** The
+ * evidence that it was reading a fragment was in the file, two lines above the fragment.
+ *
+ * Only a shortfall is reported. A surplus is ordinary: rerun plugins write an entry per attempt
+ * while `tests` counts the test once, so finding more testcases than declared means a re-run, not a
+ * corrupt file.
+ */
+export function shortfall(root) {
+    // The one signal that does not depend on the report declaring anything. A streaming writer like
+    // node:test puts each suite's totals in its closing tag, so a report cut off in the middle loses
+    // the tests and the count of them together and has nothing left to disagree with itself about.
+    // What it cannot hide is the tag it never closed.
+    if (root.truncated) {
+        return 'the file ends in the middle of itself - an element is left open, so it was cut off while being written or uploaded. Whatever ran after that point is not in here, and Tewip cannot say how much that was.';
+    }
+    const num = (e, key) => {
+        const n = Number(e.attrs[key]);
+        return Number.isFinite(n) ? n : 0;
+    };
+    // Only the innermost suites are counted. PHPUnit wraps every suite in another suite carrying the
+    // same totals, and real PHPUnit output was read as half missing when both levels were added up -
+    // a false alarm on a perfectly good report, which is worse than the silence this replaces. A
+    // parent's totals already include its children's, so the leaves are the whole run exactly once.
+    const all = [...(root.name === 'testsuite' ? [root] : []), ...descendants(root, 'testsuite')];
+    const suites = all.filter((e) => descendants(e, 'testsuite').length === 0);
+    const totals = (key, fallback) => root.attrs[key] !== undefined
+        ? num(root, key)
+        : suites.reduce((sum, e) => sum + fallback.reduce((s, k) => s + num(e, k), 0), 0);
+    const declaredTests = totals('tests', ['tests']);
+    const declaredBad = root.attrs.failures !== undefined || root.attrs.errors !== undefined
+        ? num(root, 'failures') + num(root, 'errors')
+        : suites.reduce((sum, e) => sum + num(e, 'failures') + num(e, 'errors'), 0);
+    const cases = descendants(root, 'testcase');
+    const foundBad = cases.filter((c) => c.children.some((ch) => FAILURE.has(ch.name))).length;
+    const missingTests = declaredTests - cases.length;
+    const missingBad = declaredBad - foundBad;
+    if (missingTests <= 0 && missingBad <= 0)
+        return null;
+    const said = [];
+    if (missingTests > 0)
+        said.push(`${missingTests} of the ${declaredTests} test(s) it says it holds are not in it`);
+    if (missingBad > 0)
+        said.push(`${missingBad} of the ${declaredBad} failure(s) it reports are not in it`);
+    return `${said.join(', and ')}. This report is a fragment of its run - most likely cut off while being written or uploaded. Nothing below covers the missing part.`;
+}

package/dist/engine/import/xml.js

@@ -65,6 +65,10 @@ export function parseXml(source) {
     const first = root.children[0];
     if (!first)
         throw new Error('not an XML document: no root element');
+    // Anything still on the stack is an element nobody closed. Well-formed XML leaves only the
+    // document itself behind.
+    if (stack.length > 1)
+        first.truncated = true;
     return first;
 }
 function end(source, token, from) {

package/dist/engine/llm/prompt.js

@@ -3,28 +3,28 @@
 import { ROOT_CAUSES } from '../../reporters/types.js';
 /** Bump whenever the prompt, schema or context format changes; it is part of the cache key. */
 export const PROMPT_VERSION = 'llm-6';
-export const SYSTEM_PROMPT = `You are the root-cause classifier of a Playwright test-failure triage agent.
-You receive a structured summary of ONE failed test: the error, the test's run history, what the deterministic rules already checked, the test steps, the page at failure time (accessibility snapshot + notable DOM elements), failed network requests and console errors.
-
-Classify the root cause into exactly one of these categories:
-- REAL_BUG: the application under test behaves incorrectly. Examples: a wrong value is displayed; a feature or element the test needs is absent from the page and nothing equivalent exists; the app never reaches the expected state even though nothing external failed; uncaught page errors from application code.
-- LOCATOR_DRIFT: the element the test wants is on the page, but the selector no longer matches it (renamed id, class, test id or text). The page summary must show an element with the same purpose.
-- TIMING: the expected element or state does arrive, but not in time, or a race makes the result vary between attempts or runs. Signs: another attempt of the same code passed, intermittent history with varying errors, hard-coded waits, the element resolved but was not yet visible, stable or enabled.
-- ENV: something outside the app code and the test broke: failed or 5xx/502/503 responses from a backend or third-party service, connection or DNS errors, expired auth or credentials, or many tests in the same run failing the same way.
-- TEST_DATA: the test's preconditions or fixture data are wrong: missing or stale seeded data, a user or account in the wrong state, an expected value that depends on data the test did not set up, a 404 for a record the test assumed exists.
-
-How to decide:
-- Use only facts present in the summary. Do not invent page content, requests or history.
-- Prefer the category the evidence directly shows over one that is merely possible.
-- An element that is missing with no similar element on the page is not LOCATOR_DRIFT.
-- Playwright matches accessible names by substring, so a locator can match several elements at once ("strict mode violation"). That is LOCATOR_DRIFT only when one of the matched elements is the one the test means, which usually carries the locator's name as a whole word ("Edit" -> "Edit link"). When every match only contains the text inside a longer word or phrase ("Edit" -> "Editor canvas", "Gift" -> "Gifting options"), the element the test means is absent: judge it as REAL_BUG, TIMING or TEST_DATA on the other evidence, not LOCATOR_DRIFT.
-- A failed request or server error is only ENV evidence if the test did not provoke it. When "test.exercisesErrorHandling" is set, the test deliberately triggers failures (rejections, server errors, retries), so a failed request is the scenario under test: judge whether the app handled it as the test expects. If it did not, that is REAL_BUG, not ENV. A real environment outage also tends to hit unrelated tests and to pass on retry, rather than failing every attempt on one commit while other commits pass.
-- Check "thisRun" first. If anotherAttemptPassed is true, the same code both failed and passed, so the cause is not deterministic: a real bug or a drifted locator would fail every attempt. Choose among TIMING, ENV and TEST_DATA.
-- Then read "history.pattern", which compares this commit's attempts with other commits. If every attempt on this commit failed while other commits passed, something in this commit changed behaviour: choose among REAL_BUG, LOCATOR_DRIFT and TEST_DATA using the page summary, not TIMING. An overall failure rate that mixes different commits is not evidence of flakiness; only failing and passing on the same code is.
-- The "rules" section lists what the deterministic rules already examined and why they declined; treat those as checked facts.
-
-Confidence (0 to 1): 0.9 or higher only when the summary directly shows the cause; 0.6-0.8 when it is the most likely reading but another category remains plausible; below 0.5 when you are mostly guessing. Low confidence routes the failure to a human, so do not inflate it.
-
+export const SYSTEM_PROMPT = `You are the root-cause classifier of a Playwright test-failure triage agent.
+You receive a structured summary of ONE failed test: the error, the test's run history, what the deterministic rules already checked, the test steps, the page at failure time (accessibility snapshot + notable DOM elements), failed network requests and console errors.
+
+Classify the root cause into exactly one of these categories:
+- REAL_BUG: the application under test behaves incorrectly. Examples: a wrong value is displayed; a feature or element the test needs is absent from the page and nothing equivalent exists; the app never reaches the expected state even though nothing external failed; uncaught page errors from application code.
+- LOCATOR_DRIFT: the element the test wants is on the page, but the selector no longer matches it (renamed id, class, test id or text). The page summary must show an element with the same purpose.
+- TIMING: the expected element or state does arrive, but not in time, or a race makes the result vary between attempts or runs. Signs: another attempt of the same code passed, intermittent history with varying errors, hard-coded waits, the element resolved but was not yet visible, stable or enabled.
+- ENV: something outside the app code and the test broke: failed or 5xx/502/503 responses from a backend or third-party service, connection or DNS errors, expired auth or credentials, or many tests in the same run failing the same way.
+- TEST_DATA: the test's preconditions or fixture data are wrong: missing or stale seeded data, a user or account in the wrong state, an expected value that depends on data the test did not set up, a 404 for a record the test assumed exists.
+
+How to decide:
+- Use only facts present in the summary. Do not invent page content, requests or history.
+- Prefer the category the evidence directly shows over one that is merely possible.
+- An element that is missing with no similar element on the page is not LOCATOR_DRIFT.
+- Playwright matches accessible names by substring, so a locator can match several elements at once ("strict mode violation"). That is LOCATOR_DRIFT only when one of the matched elements is the one the test means, which usually carries the locator's name as a whole word ("Edit" -> "Edit link"). When every match only contains the text inside a longer word or phrase ("Edit" -> "Editor canvas", "Gift" -> "Gifting options"), the element the test means is absent: judge it as REAL_BUG, TIMING or TEST_DATA on the other evidence, not LOCATOR_DRIFT.
+- A failed request or server error is only ENV evidence if the test did not provoke it. When "test.exercisesErrorHandling" is set, the test deliberately triggers failures (rejections, server errors, retries), so a failed request is the scenario under test: judge whether the app handled it as the test expects. If it did not, that is REAL_BUG, not ENV. A real environment outage also tends to hit unrelated tests and to pass on retry, rather than failing every attempt on one commit while other commits pass.
+- Check "thisRun" first. If anotherAttemptPassed is true, the same code both failed and passed, so the cause is not deterministic: a real bug or a drifted locator would fail every attempt. Choose among TIMING, ENV and TEST_DATA.
+- Then read "history.pattern", which compares this commit's attempts with other commits. If every attempt on this commit failed while other commits passed, something in this commit changed behaviour: choose among REAL_BUG, LOCATOR_DRIFT and TEST_DATA using the page summary, not TIMING. An overall failure rate that mixes different commits is not evidence of flakiness; only failing and passing on the same code is.
+- The "rules" section lists what the deterministic rules already examined and why they declined; treat those as checked facts.
+
+Confidence (0 to 1): 0.9 or higher only when the summary directly shows the cause; 0.6-0.8 when it is the most likely reading but another category remains plausible; below 0.5 when you are mostly guessing. Low confidence routes the failure to a human, so do not inflate it.
+
 Return JSON only: the category, the confidence, a short reasoning (at most 3 sentences), and 1-4 evidence items quoting or pointing at specific facts from the summary.`;
 export const OUTPUT_SCHEMA_NAME = 'triage_diagnosis';
 export const OUTPUT_SCHEMA = {

package/dist/engine/quality/grounding.js

@@ -115,7 +115,11 @@ export function relatedTestSources(root, feature, maxFiles = 4, maxChars = 8_000
 /** What a git range changed (e.g. "main...HEAD"); null outside a git checkout. */
 export function changeSummary(root, range, maxPatch = 12_000) {
     try {
-        const git = (args) => execFileSync('git', args, { cwd: root, encoding: 'utf8', maxBuffer: 64 * 1024 * 1024 });
+        // `stdio` so git's own stderr is captured rather than inherited. `execFileSync` sends a child's
+        // stderr straight to the parent unless told not to, so a range that does not exist printed three
+        // lines of `fatal: ambiguous argument 'main...HEAD'` and a git usage hint **before** Tewip's own
+        // explanation of the same thing. The failure was always handled; it just looked like a crash.
+        const git = (args) => execFileSync('git', args, { cwd: root, encoding: 'utf8', maxBuffer: 64 * 1024 * 1024, stdio: ['ignore', 'pipe', 'pipe'] });
         const files = git(['diff', '--name-only', range]).split('\n').filter(Boolean);
         const patch = git(['diff', '--unified=2', range]);
         return { range, files, patch: patch.length > maxPatch ? `${patch.slice(0, maxPatch)}\n... (diff cut at ${maxPatch} characters)` : patch };

package/dist/engine/review/agreements.js

@@ -92,7 +92,9 @@ function classify(file, before, after) {
 }
 function sourceDiff(root, range) {
     try {
-        return execFileSync('git', ['diff', '--unified=0', range, '--'], { cwd: root, encoding: 'utf8', maxBuffer: 64 * 1024 * 1024 });
+        // Captured, not inherited: see the note in grounding.ts - an unknown range printed git's own
+        // error before Tewip's.
+        return execFileSync('git', ['diff', '--unified=0', range, '--'], { cwd: root, encoding: 'utf8', maxBuffer: 64 * 1024 * 1024, stdio: ['ignore', 'pipe', 'pipe'] });
     }
     catch {
         return '';

package/dist/engine/review/expectations.js

@@ -24,7 +24,8 @@ const title = (line) => /['"`]([^'"`]{4,120})['"`]/.exec(line)?.[1] ?? /\b(?:def
 /** The diff of test files only, which is all this looks at. */
 function testDiff(root, range) {
     try {
-        return execFileSync('git', ['diff', '--unified=0', range, '--'], { cwd: root, encoding: 'utf8', maxBuffer: 64 * 1024 * 1024 });
+        // Captured, not inherited: see the note in grounding.ts.
+        return execFileSync('git', ['diff', '--unified=0', range, '--'], { cwd: root, encoding: 'utf8', maxBuffer: 64 * 1024 * 1024, stdio: ['ignore', 'pipe', 'pipe'] });
     }
     catch {
         return '';

package/dist/engine/secure/deps.js

@@ -6,6 +6,48 @@ import { existsSync } from 'node:fs';
 import { join } from 'node:path';
 import { spawnSync } from 'node:child_process';
 const NEWLINE = String.fromCharCode(10);
+/**
+ * A scanner's JSON, or `undefined` when it did not produce any.
+ *
+ * Empty output counts as no report rather than as an empty one. A scanner that printed nothing
+ * cannot be told apart from a scanner that crashed before it printed, and the two possible
+ * mistakes are not equally bad: calling a clean scan unreadable sends somebody to look, while
+ * calling an unreadable scan clean is the false all-clear this whole report exists to avoid.
+ */
+function readJson(out) {
+    if (!out.trim())
+        return undefined;
+    try {
+        return JSON.parse(out);
+    }
+    catch {
+        return undefined;
+    }
+}
+/**
+ * Did this scanner produce a report, or only exit?
+ *
+ * `key` is the field its report always carries, so JSON that parses but is not a scan result - a
+ * usage message, a progress object, an error body - is caught too. The exit code is deliberately
+ * not consulted: pip-audit and npm audit both exit non-zero precisely when they have found
+ * something, so treating that as failure would discard the findings that matter most.
+ */
+export function scanOutcome(tool, out, err, shape, read) {
+    const report = readJson(out);
+    if (report === undefined)
+        return { findings: [], problem: `${tool} did not return a report${firstLine(out, err)}` };
+    // Older pip-audit prints the dependency list unwrapped, so a bare array is a real report there
+    // and nowhere else. Accepting one everywhere would let a stray array pass for an empty scan.
+    const known = report?.[shape.key] !== undefined || (shape.orArray === true && Array.isArray(report));
+    if (!known)
+        return { findings: [], problem: `${tool} returned JSON that is not a scan report` };
+    return { findings: read(report), problem: null };
+}
+/** The first line a failing tool said, for the report - whichever stream it used. */
+function firstLine(out, err) {
+    const said = [err, out].map((x) => x.trim()).find(Boolean);
+    return said ? `: ${said.split(NEWLINE)[0].slice(0, 120)}` : '';
+}
 const run = (command, cwd, timeoutMs = 5 * 60_000) => {
     const r = spawnSync(command, { cwd, shell: true, encoding: 'utf8', timeout: timeoutMs, maxBuffer: 40 * 1024 * 1024 });
     return { code: r.status ?? 1, out: `${r.stdout ?? ''}`, err: `${r.stderr ?? ''}` };
@@ -55,90 +97,94 @@ export function npmAudit(root) {
     }
     return { findings, problem: null };
 }
+/**
+ * What pip-audit found, given what it printed. Separated from running it so the shapes real
+ * pip-audit produces can be tested without the tool installed.
+ */
+export function pipFindings(report) {
+    const list = report?.dependencies ?? report;
+    return (Array.isArray(list) ? list : []).flatMap((dep) => (dep.vulns ?? []).map((v) => ({
+        package: String(dep.name ?? ''),
+        version: String(dep.version ?? ''),
+        severity: String(v.severity ?? 'unknown').toLowerCase(),
+        id: String(v.id ?? ''),
+        title: String(v.description ?? '').split('\n')[0].slice(0, 200),
+        fixedIn: (v.fix_versions ?? []).join(', '),
+        source: 'pip-audit',
+        dev: false,
+    })));
+}
 /** `pip-audit`, when the project is Python and the tool is installed. */
 export function pipAudit(root) {
     const python = ['requirements.txt', 'pyproject.toml', 'setup.py'].some((f) => existsSync(join(root, f)));
     if (!python || !has('pip-audit', root))
-        return [];
-    const { out } = run('pip-audit -f json --progress-spinner off', root);
-    try {
-        const report = JSON.parse(out);
-        const list = report.dependencies ?? report;
-        return (Array.isArray(list) ? list : []).flatMap((dep) => (dep.vulns ?? []).map((v) => ({
-            package: String(dep.name ?? ''),
-            version: String(dep.version ?? ''),
-            severity: String(v.severity ?? 'unknown').toLowerCase(),
-            id: String(v.id ?? ''),
-            title: String(v.description ?? '').split('\n')[0].slice(0, 200),
-            fixedIn: (v.fix_versions ?? []).join(', '),
-            source: 'pip-audit',
-            dev: false,
-        })));
-    }
-    catch {
-        return [];
+        return { findings: [], problem: null };
+    const { out, err } = run('pip-audit -f json --progress-spinner off', root);
+    return scanOutcome('pip-audit', out, err, { key: 'dependencies', orArray: true }, pipFindings);
+}
+/**
+ * What osv-scanner found, given what it printed. Separated from running it for the same reason as
+ * `pipFindings`: the parsing is what needs testing, and the tool is rarely installed.
+ */
+export function osvFindings(report) {
+    const findings = [];
+    for (const result of report?.results ?? []) {
+        for (const pkg of result.packages ?? []) {
+            for (const v of pkg.vulnerabilities ?? []) {
+                findings.push({
+                    package: String(pkg.package?.name ?? ''),
+                    version: String(pkg.package?.version ?? ''),
+                    severity: String(v.database_specific?.severity ?? v.severity?.[0]?.type ?? 'unknown').toLowerCase(),
+                    id: String(v.id ?? ''),
+                    title: String(v.summary ?? '').slice(0, 200),
+                    fixedIn: '',
+                    // osv-scanner's output says neither which version fixes it nor whether the project
+                    // declares the package, so neither is claimed.
+                    fixExists: false,
+                    direct: false,
+                    source: 'osv-scanner',
+                    dev: false,
+                });
+            }
+        }
     }
+    return findings;
 }
 /** OSV-Scanner, which covers Go, Rust, Java, Ruby, PHP and more, when it is installed. */
 export function osvScan(root) {
     if (!has('osv-scanner', root))
-        return [];
-    const { out } = run(`osv-scanner --format json --recursive "${root}"`, root);
-    try {
-        const report = JSON.parse(out);
-        const findings = [];
-        for (const result of report.results ?? []) {
-            for (const pkg of result.packages ?? []) {
-                for (const v of pkg.vulnerabilities ?? []) {
-                    findings.push({
-                        package: String(pkg.package?.name ?? ''),
-                        version: String(pkg.package?.version ?? ''),
-                        severity: String(v.database_specific?.severity ?? v.severity?.[0]?.type ?? 'unknown').toLowerCase(),
-                        id: String(v.id ?? ''),
-                        title: String(v.summary ?? '').slice(0, 200),
-                        fixedIn: '',
-                        // osv-scanner's output says neither which version fixes it nor whether the project
-                        // declares the package, so neither is claimed.
-                        fixExists: false,
-                        direct: false,
-                        source: 'osv-scanner',
-                        dev: false,
-                    });
-                }
-            }
-        }
-        return findings;
-    }
-    catch {
-        return [];
-    }
+        return { findings: [], problem: null };
+    const { out, err } = run(`osv-scanner --format json --recursive "${root}"`, root);
+    return scanOutcome('osv-scanner', out, err, { key: 'results' }, osvFindings);
 }
 export function auditDependencies(root) {
     const ran = [];
     const missing = [];
     const findings = [];
-    if (existsSync(join(root, 'package.json'))) {
-        const npm = npmAudit(root);
-        if (npm.problem)
-            missing.push(`npm audit could not run: ${npm.problem}`);
+    // `ran` used to be pushed before the scanner was called, so a pip-audit that was installed and
+    // failed left "Checked: pip-audit" standing above zero findings. Measured on a project pinning
+    // requests==2.19.1 and flask==0.12.2 - both long known vulnerable - with a pip-audit that exited
+    // without JSON: the report read "Checked: secrets in the working tree, pip-audit. 0 need someone
+    // now." A scanner is named as having run only once it has returned something readable.
+    const record = (tool, outcome) => {
+        if (outcome.problem)
+            missing.push(`${tool} could not run: ${outcome.problem}`);
         else
-            ran.push('npm audit');
-        findings.push(...npm.findings);
+            ran.push(tool);
+        findings.push(...outcome.findings);
+    };
+    if (existsSync(join(root, 'package.json'))) {
+        record('npm audit', npmAudit(root));
     }
     const python = ['requirements.txt', 'pyproject.toml', 'setup.py'].some((f) => existsSync(join(root, f)));
     if (python) {
-        if (has('pip-audit', root)) {
-            ran.push('pip-audit');
-            findings.push(...pipAudit(root));
-        }
-        else {
+        if (has('pip-audit', root))
+            record('pip-audit', pipAudit(root));
+        else
             missing.push('pip-audit (pip install pip-audit) for the Python dependencies');
-        }
-    }
-    if (has('osv-scanner', root)) {
-        ran.push('osv-scanner');
-        findings.push(...osvScan(root));
     }
+    if (has('osv-scanner', root))
+        record('osv-scanner', osvScan(root));
     else if (!existsSync(join(root, 'package.json')) && !python) {
         missing.push('osv-scanner (https://osv.dev) for this project\'s ecosystem');
     }

package/dist/engine/secure/secure-command.js

@@ -92,8 +92,13 @@ export async function runSecure(cmd) {
     }
     if (cmd.zapReport) {
         const fromZap = readZapReport(resolve(cmd.zapReport));
-        checked.push(`an OWASP ZAP report (${fromZap.length} finding(s))`);
-        for (const finding of fromZap)
+        // Named as checked only once it has been read. "(0 finding(s))" above a file Tewip could not
+        // open is a measurement of nothing, and a mistyped path answered exactly like a clean scan.
+        if (fromZap.problem)
+            notChecked.push(`the ZAP report you asked for: ${fromZap.problem}`);
+        else
+            checked.push(`an OWASP ZAP report (${fromZap.findings.length} finding(s))`);
+        for (const finding of fromZap.findings)
             items.push(triageWeb(finding, accepted));
     }
     else if (cmd.url) {

package/dist/engine/secure/zap.js

@@ -10,17 +10,22 @@ const RISK = { High: 'high', Medium: 'medium', Low: 'low', Informational: 'low'
 /** Findings from a ZAP JSON report, in Tewip's own shape. */
 export function readZapReport(path) {
     if (!existsSync(path))
-        return [];
+        return { findings: [], problem: `there is no file at ${path}` };
     let report;
     try {
         report = JSON.parse(readFileSync(path, 'utf8'));
     }
     catch {
-        return [];
+        return { findings: [], problem: `${path} is not JSON, so it is not a ZAP report Tewip can read` };
+    }
+    // ZAP always writes `site`, one entry per host it scanned, even when it found nothing. Anything
+    // without it is some other tool's JSON, or ZAP's own error body.
+    if (!Array.isArray(report?.site)) {
+        return { findings: [], problem: `${path} is JSON but not a ZAP report: it has no "site" list` };
     }
     const alerts = (report.site ?? []).flatMap((site) => site.alerts ?? []);
     const strip = (html) => html.replace(/<[^>]+>/g, ' ').replace(/\s+/g, ' ').trim();
-    return alerts.map((alert) => {
+    const findings = alerts.map((alert) => {
         const risk = (alert.riskdesc ?? '').split(' ')[0];
         const first = (alert.instances ?? [])[0] ?? {};
         return {
@@ -32,6 +37,7 @@ export function readZapReport(path) {
             next: strip(alert.solution ?? '').slice(0, 300) || 'see the ZAP report for the recommended fix',
         };
     });
+    return { findings, problem: null };
 }
 /** Whether a ZAP baseline scan can be run from here at all. */
 export function zapAvailable(cwd) {

package/dist/engine/vouch/vouch-command.js

@@ -73,6 +73,30 @@ export function vouchAll(root, paths) {
 }
 const count = (all, verdict) => all.filter((j) => j.verdict === verdict).length;
 export function renderVouch(files) {
+    // No test file was read, so there is nothing to say about tests.
+    //
+    // This printed the ordinary summary - "0 test(s) read in 0 file(s). **0 cannot fail at all.**" -
+    // followed by the heading **"Nothing here is unable to fail"** and the line "Every test read can
+    // be broken by a change to the code". All true of nothing, and all of it reads as a pass, in the
+    // command whose entire job is to find tests that cannot fail.
+    //
+    // The comment in `testFilesUnder` above already diagnosed this in Phase 36, when three languages
+    // were missing from the extension list and a file named outright came back as "0 test(s) read in
+    // 0 file(s) - which reads like a clean answer and is not one". The cause was fixed; the report
+    // was not, so every other way of reading nothing still produced a pass: the wrong folder, a
+    // framework Tewip does not recognise, or a project that genuinely has no tests yet.
+    if (files.length === 0) {
+        return [
+            '# Which of these tests could fail?',
+            '',
+            '**No test file was read, so nothing here has been checked.** This is not a report that your tests are sound - it is a report that Tewip found none to look at.',
+            '',
+            'Three things usually explain it: the path given holds no tests, the tests are somewhere Tewip was not pointed at (`tewip vouch <path>`), or they are written in a language it cannot read yet.',
+            '',
+            'It reads JavaScript, TypeScript, Python, Java, Kotlin, Scala, Groovy, Go, C#, F#, Ruby, PHP and Rust.',
+            '',
+        ].join(String.fromCharCode(10));
+    }
     const all = files.flatMap((f) => f.judgements);
     const cannotFail = all.filter((j) => CANNOT_FAIL.includes(j.verdict));
     const crashOnly = count(all, 'crash-only');

package/dist/scripts/explain.js

@@ -50,8 +50,19 @@ async function main() {
     const index = new HistoryIndex([...pastRecords, ...imported.records], [...pastRuns, imported.summary]);
     const items = await buildItems(imported.records, index, { dataset: `local: ${name}`, llm: llmFromEnv(), corrections: loadCorrections(historyDir) });
     const tests = Object.keys(imported.summary.attempts).length;
-    if (items.length === 0)
-        console.log(`All ${tests} test(s) passed. Nothing to explain.`);
+    // A report missing part of itself is not a report that nothing went wrong. This goes first,
+    // above any board and in place of any all-clear, because it changes what everything below means.
+    if (imported.incomplete.length > 0) {
+        console.log('**This run is incomplete.**');
+        for (const why of imported.incomplete)
+            console.log(`- ${why}`);
+        console.log('');
+    }
+    if (items.length === 0) {
+        console.log(imported.incomplete.length > 0
+            ? `The ${tests} test(s) that did arrive all passed. That is not the same as the run passing, and Tewip cannot say whether the missing part did.`
+            : `All ${tests} test(s) passed. Nothing to explain.`);
+    }
     else
         console.log(sdetBoardView(items, runId, `local: ${name}`).markdown);
     // Keep this run as history for the next one, with its diagnoses.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "tewip-ai",
-  "version": "0.70.12",
+  "version": "0.70.14",
   "mcpName": "io.github.hmamut39/tewip",
   "description": "Tewip finds why each failing test failed - real bug, flaky test, environment, test data or changed selector - for any stack, from the command line, CI, or your editor.",
   "keywords": [