tewip-ai 0.72.0 → 0.73.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (8) hide show
  1. package/QUICKSTART.md +178 -178
  2. package/dist/cli/help.js +268 -268
  3. package/dist/engine/autofix/code-propose.js +8 -8
  4. package/dist/engine/build/build.js +22 -22
  5. package/dist/engine/llm/prompt.js +22 -22
  6. package/dist/engine/vouch/assertions.js +8 -0
  7. package/dist/engine/walk.js +30 -3
  8. package/package.json +1 -1
package/QUICKSTART.md CHANGED
@@ -1,179 +1,179 @@
1
- # Quickstart: putting Tewip on one repository
2
-
3
- For an engineer adding the agent to a project that already runs Playwright tests in GitHub Actions. It takes about ten minutes, and the first useful output arrives with the next failing run.
4
-
5
- Start with **one** repository whose tests fail sometimes. A repository whose suite is always green has nothing to triage.
6
-
7
- ## What the agent does once it is installed
8
-
9
- Every CI run, for every failed test, it decides the root cause — application bug, changed selector, race, environment, or test data — with a confidence, the evidence it used, and one concrete next action. It then delivers that to the people who need it: a comment on the pull request, the board in the job summary, one grouped issue for environment failures, and optionally a separate pull request with locator fixes that it re-ran and verified.
10
-
11
- It never merges anything, never edits application code, and never opens a fix pull request for a change it has not proved by re-running the test.
12
-
13
- ## Step 1: make Playwright leave the evidence behind
14
-
15
- The agent reads Playwright's own blob report and traces, so nothing from this project goes into your test config:
16
-
17
- ```ts
18
- // playwright.config.ts
19
- reporter: process.env.CI ? [['list'], ['blob']] : 'list',
20
- use: { trace: 'retain-on-failure' },
21
- ```
22
-
23
- If your config already uses a reporter list in CI, add `['blob']` to it.
24
-
25
- ## Step 2: add one step to the workflow that runs the tests
26
-
27
- ```yaml
28
- permissions:
29
- pull-requests: write # the diagnosis comment
1
+ # Quickstart: putting Tewip on one repository
2
+
3
+ For an engineer adding the agent to a project that already runs Playwright tests in GitHub Actions. It takes about ten minutes, and the first useful output arrives with the next failing run.
4
+
5
+ Start with **one** repository whose tests fail sometimes. A repository whose suite is always green has nothing to triage.
6
+
7
+ ## What the agent does once it is installed
8
+
9
+ Every CI run, for every failed test, it decides the root cause — application bug, changed selector, race, environment, or test data — with a confidence, the evidence it used, and one concrete next action. It then delivers that to the people who need it: a comment on the pull request, the board in the job summary, one grouped issue for environment failures, and optionally a separate pull request with locator fixes that it re-ran and verified.
10
+
11
+ It never merges anything, never edits application code, and never opens a fix pull request for a change it has not proved by re-running the test.
12
+
13
+ ## Step 1: make Playwright leave the evidence behind
14
+
15
+ The agent reads Playwright's own blob report and traces, so nothing from this project goes into your test config:
16
+
17
+ ```ts
18
+ // playwright.config.ts
19
+ reporter: process.env.CI ? [['list'], ['blob']] : 'list',
20
+ use: { trace: 'retain-on-failure' },
21
+ ```
22
+
23
+ If your config already uses a reporter list in CI, add `['blob']` to it.
24
+
25
+ ## Step 2: add one step to the workflow that runs the tests
26
+
27
+ ```yaml
28
+ permissions:
29
+ pull-requests: write # the diagnosis comment
30
30
  issues: write
31
- actions: read # so it can read a failed job's log when no report was written # the grouped environment issue
32
- contents: write # only if you turn autofix on
33
-
34
- steps:
35
- # ... your existing checkout, install, and `npx playwright test` steps ...
36
-
37
- - name: Tewip
38
- if: always() # it must run even though the test step failed
39
- uses: hmamut39/tewip@main
40
- with:
41
- openai-api-key: ${{ secrets.OPENAI_API_KEY }} # optional, see step 3
42
- autofix: 'false' # start with this off
43
- ```
44
-
45
- `if: always()` matters: without it the step is skipped exactly when there is something to triage.
46
-
47
- Keep `autofix: 'false'` for the first week. Read what the agent says, decide whether you trust it, and turn fixing on afterwards.
48
-
49
- ## Step 3: decide whether to give it an API key
50
-
51
- | Without a key | With `OPENAI_API_KEY` |
52
- |---|---|
53
- | Only the deterministic rules run | The rules run first, and only what they cannot settle goes to the model |
54
- | Every answer is explainable from the error, the trace and the test's history | Adds a judgement on the rest, with its reasoning and evidence shown |
55
- | Free | A few cents per run in practice, because the rules answer most failures |
56
-
57
- On real projects the rules alone decide most failures and are right on 96–98% of what they decide, so starting without a key is a reasonable first week. To add one later: repository **Settings → Secrets and variables → Actions → New repository secret**, named `OPENAI_API_KEY`.
58
-
59
- Every run prints what it cost, priced from OpenAI's published prices. One guard worth setting once the key is in place:
60
-
61
- ```yaml
62
- env:
63
- TRIAGE_LLM_MAX_CALLS: '80' # hard cap on model calls per run
64
- # TRIAGE_LLM_PRICE_IN / _OUT: only if your price differs (per million tokens)
65
- ```
66
-
67
- ## Not using Playwright? Every stack works
68
-
69
- Point the agent at your test runner's **JUnit XML** report (or .NET's **TRX**) instead of a Playwright blob report. Each runner marked *verified* was run for real in CI (`fixtures/polyglot`) and its report is checked by the test suite on every change.
70
-
71
- | Stack | Runner | How to write the report | Status |
72
- |---|---|---|---|
73
- | **Node.js** | node:test (built in) | `node --test --test-reporter=junit --test-reporter-destination=report.xml` | verified |
74
- | Node.js, React, Vue, Angular, Next.js, NestJS, Express | Jest | `jest --reporters=default --reporters=jest-junit` | verified |
75
- | Vite, Vue, Svelte, Nuxt, React | Vitest | `vitest run --reporter=junit --outputFile=report.xml` | verified |
76
- | Node.js, Express | Mocha | `mocha --reporter mocha-junit-reporter` | verified |
77
- | **Python**, Django, Flask, FastAPI | pytest (also runs `unittest`) | `pytest --junitxml=report.xml` (retries: `pytest-rerunfailures`) | verified |
78
- | **Java**, Spring, Kotlin | JUnit 4/5, TestNG via Maven Surefire | written by default to `target/surefire-reports/` (retries: `rerunFailingTestsCount`) | verified |
79
- | Java, Kotlin, Android | Gradle | written by default to `build/test-results/test/` | same format as Surefire |
80
- | **Go** | gotestsum | `gotestsum --junitfile report.xml` | verified |
81
- | **.NET** (C#, F#), ASP.NET | xUnit, NUnit, MSTest | `dotnet test --logger trx` (native) or `--logger junit` | both verified |
82
- | **Ruby**, Rails | RSpec | `rspec --format RspecJunitFormatter --out report.xml` | verified |
83
- | **PHP**, Laravel, Symfony | PHPUnit | `phpunit --log-junit report.xml` | verified |
84
- | **Rust** | cargo-nextest | `[profile.ci.junit]` in `.config/nextest.toml` | verified |
85
- | End-to-end | **Selenium** (Java, Python, JavaScript, C#) | through the runner above it: Surefire, pytest, Jest/Mocha, `dotnet test` | JUnit format; error wording read |
86
- | End-to-end | Cypress, WebdriverIO | `mocha-junit-reporter` / `@wdio/junit-reporter` | JUnit format; error wording read |
87
- | End-to-end | **Playwright** | blob reporter (step 1) — adds traces, selector drift and verified fixes | verified on 5 real projects |
88
-
89
- ```yaml
90
- - name: Tewip
91
- if: always()
92
- uses: hmamut39/tewip@main
93
- with:
94
- junit-report: report.xml # a file, or a directory of them (.xml and .trx)
95
- suite: unit # only needed when one workflow triages several suites
96
- ```
97
-
98
- What it reads from any of these: the failure text, retries, and each test's history. From that it tells apart a wrong value (a real bug once the test has a passing history), an unreachable service (environment), a data collision, and a test that failed then passed on a retry (timing), in the wording each language uses: `expected: <4> but was: <3>` in Java, `got 3, want 4` in Go, `AssertionError: 3 != 4` in Python, `Expected: 4 / Actual: 3` in .NET, and so on. It also reads each browser tool's own words for the same three problems: an element that was never found (Selenium's `NoSuchElementException`, WebdriverIO's `still not displayed`, Cypress's `Expected to find element`), a click another element received (Selenium's `Other element would receive the click`, Cypress's `is being covered by another element`), and an element the page replaced mid-test (Selenium's `StaleElementReferenceException`, Cypress's `the page updated while this command was executing`). No labelled Selenium or Cypress runs have measured those rules yet, so they report at a confidence the merge gate does not trust on its own. What it cannot do without Playwright's traces is look at the page, so selector drift and automatic fixes stay Playwright-only.
99
-
100
- A wrong value on a test's **first** recorded failure stays undecided rather than guessed: one failure alone does not show whether the test was reliable before. It becomes a real-bug diagnosis once the test's history shows it passing and then failing the same way.
101
-
102
- When one workflow triages several suites (say `unit` and `e2e`), give each a `suite` name: each then keeps its own history and its own comment, and corrections name the suite, as in `/triage TIMING unit#2 flaky on CI`.
103
-
104
- ## On your own machine, without CI
105
-
106
- Install the command once from the release file, `npm install -g tewip-<version>.tgz` (Tewip is not on the npm registry yet; Node 22.18 or newer), run your tests with the report switched on (see the table above), then:
107
-
108
- ```bash
109
- tewip explain report.xml # JUnit XML or .trx
110
- tewip explain reports/ # a folder of them
111
- tewip explain blob-report/ # Playwright
112
- tewip ci owner/repo --pr 12 # what Tewip's GitHub Action said about a pull request
113
- ```
114
-
115
- It prints the same triage board CI writes: each failure's cause, confidence, evidence and next action. Each run is kept in a `.tewip/` folder in your project (add it to `.gitignore`), so from the second run on, failures are judged against their history exactly as in CI. `--name` keeps separate histories for different projects.
116
-
117
- ## What you will see on the next failing run
118
-
119
- - **On the pull request:** one comment listing each failing test, grouped by whether your change likely caused it, each with a cause, a confidence, the evidence and the next action. It updates in place rather than adding a comment per run.
120
- - **In the job summary:** the same failures as a triage board.
121
- - **As an issue:** environment failures (5xx responses, connection resets, outages) grouped into one issue instead of one per test.
122
- - **As an artifact** named `tewip-report`: every role's view, including the engineering-quality view with the flaky tests ranked by the CI minutes they cost.
123
-
124
- ## Correcting it
125
-
126
- When it gets one wrong, reply to its comment:
127
-
128
- ```
129
- /triage TIMING #2 this one is flaky on our CI runners, not a bug
130
- ```
131
-
132
- From the next run on, that failure of that test gets your answer, shown as "corrected by a person" with your name and reason. A correction settles the cause; it never makes the agent change code.
133
-
134
- ## Asking it questions from your editor
135
-
136
- - **VS Code, Cursor, Windsurf, VSCodium:** install the Tewip extension (*Extensions → … → Install from VSIX*, `tewip-<version>.vsix`). It connects Tewip to the editor's AI assistant by itself and adds *Tewip: Explain a test report…* and *Tewip: Why did CI fail on this branch?*
137
- - **JetBrains (2025.2+), Claude, and other MCP clients:** add one server whose command is `tewip mcp`.
138
-
139
- Then ask, in plain language: "why did CI fail on my PR?", "which tests are flaky in checkout?", "is main safe to release?". Every tool is read-only, and it reads the report the Action already uploaded, so you do not need the other repository cloned.
140
-
141
- ## Stop flaky tests from blocking merges (optional)
142
-
143
- Once you trust its answers, let the agent decide whether a failed run should block:
144
-
145
- ```yaml
146
- - name: Tests
147
- continue-on-error: true # the gate below decides
148
- run: npx playwright test
149
- - name: Tewip
150
- if: always()
151
- uses: hmamut39/tewip@main
152
- with:
153
- gate: real-failures
154
- ```
155
-
156
- The build then fails only for real bugs, changed selectors, test-data problems and anything undecided. Failures a rule or a person classified as flaky or environment no longer block. On 2,296 real failures this unblocked 84% of the flaky and environment ones without letting a real bug through.
157
-
158
- ## Turning fixing on, once you trust it
159
-
160
- ```yaml
161
- autofix: 'true'
162
- ```
163
-
164
- It then rewrites a drifted locator, re-runs that single test, and opens a **separate** pull request only with the changes that passed. Review it like any other pull request: an intended UI change and a regression can look identical from the outside, which is exactly why a human merges it.
165
-
166
- On real WordPress CI data it would have fixed 14 drifted locators with none wrong, and every locator it suggested matched what the developers themselves wrote.
167
-
168
- ## If something does not appear
169
-
170
- | Symptom | Cause |
171
- |---|---|
172
- | The step ran but says "no blob report" | The blob reporter is not on in CI, or the tests step did not run |
173
- | No comment on the pull request | The run was not triggered by a pull request, or `pull-requests: write` is missing |
174
- | "Resource not accessible by integration" | The workflow's permissions block is missing an entry |
175
- | Diagnoses look thin on the first day | The agent judges each failure against that test's history, which it builds from your runs; it gets better as runs accumulate |
176
-
177
- ## What it costs, and what it keeps
178
-
179
- The agent stores the run history in the Actions cache, never in your repository. Collected failure data and traces stay in CI. The only thing it writes to your repository is a pull request branch, and only when you turn fixing on.
31
+ actions: read # so it can read a failed job's log when no report was written # the grouped environment issue
32
+ contents: write # only if you turn autofix on
33
+
34
+ steps:
35
+ # ... your existing checkout, install, and `npx playwright test` steps ...
36
+
37
+ - name: Tewip
38
+ if: always() # it must run even though the test step failed
39
+ uses: hmamut39/tewip@main
40
+ with:
41
+ openai-api-key: ${{ secrets.OPENAI_API_KEY }} # optional, see step 3
42
+ autofix: 'false' # start with this off
43
+ ```
44
+
45
+ `if: always()` matters: without it the step is skipped exactly when there is something to triage.
46
+
47
+ Keep `autofix: 'false'` for the first week. Read what the agent says, decide whether you trust it, and turn fixing on afterwards.
48
+
49
+ ## Step 3: decide whether to give it an API key
50
+
51
+ | Without a key | With `OPENAI_API_KEY` |
52
+ |---|---|
53
+ | Only the deterministic rules run | The rules run first, and only what they cannot settle goes to the model |
54
+ | Every answer is explainable from the error, the trace and the test's history | Adds a judgement on the rest, with its reasoning and evidence shown |
55
+ | Free | A few cents per run in practice, because the rules answer most failures |
56
+
57
+ On real projects the rules alone decide most failures and are right on 96–98% of what they decide, so starting without a key is a reasonable first week. To add one later: repository **Settings → Secrets and variables → Actions → New repository secret**, named `OPENAI_API_KEY`.
58
+
59
+ Every run prints what it cost, priced from OpenAI's published prices. One guard worth setting once the key is in place:
60
+
61
+ ```yaml
62
+ env:
63
+ TRIAGE_LLM_MAX_CALLS: '80' # hard cap on model calls per run
64
+ # TRIAGE_LLM_PRICE_IN / _OUT: only if your price differs (per million tokens)
65
+ ```
66
+
67
+ ## Not using Playwright? Every stack works
68
+
69
+ Point the agent at your test runner's **JUnit XML** report (or .NET's **TRX**) instead of a Playwright blob report. Each runner marked *verified* was run for real in CI (`fixtures/polyglot`) and its report is checked by the test suite on every change.
70
+
71
+ | Stack | Runner | How to write the report | Status |
72
+ |---|---|---|---|
73
+ | **Node.js** | node:test (built in) | `node --test --test-reporter=junit --test-reporter-destination=report.xml` | verified |
74
+ | Node.js, React, Vue, Angular, Next.js, NestJS, Express | Jest | `jest --reporters=default --reporters=jest-junit` | verified |
75
+ | Vite, Vue, Svelte, Nuxt, React | Vitest | `vitest run --reporter=junit --outputFile=report.xml` | verified |
76
+ | Node.js, Express | Mocha | `mocha --reporter mocha-junit-reporter` | verified |
77
+ | **Python**, Django, Flask, FastAPI | pytest (also runs `unittest`) | `pytest --junitxml=report.xml` (retries: `pytest-rerunfailures`) | verified |
78
+ | **Java**, Spring, Kotlin | JUnit 4/5, TestNG via Maven Surefire | written by default to `target/surefire-reports/` (retries: `rerunFailingTestsCount`) | verified |
79
+ | Java, Kotlin, Android | Gradle | written by default to `build/test-results/test/` | same format as Surefire |
80
+ | **Go** | gotestsum | `gotestsum --junitfile report.xml` | verified |
81
+ | **.NET** (C#, F#), ASP.NET | xUnit, NUnit, MSTest | `dotnet test --logger trx` (native) or `--logger junit` | both verified |
82
+ | **Ruby**, Rails | RSpec | `rspec --format RspecJunitFormatter --out report.xml` | verified |
83
+ | **PHP**, Laravel, Symfony | PHPUnit | `phpunit --log-junit report.xml` | verified |
84
+ | **Rust** | cargo-nextest | `[profile.ci.junit]` in `.config/nextest.toml` | verified |
85
+ | End-to-end | **Selenium** (Java, Python, JavaScript, C#) | through the runner above it: Surefire, pytest, Jest/Mocha, `dotnet test` | JUnit format; error wording read |
86
+ | End-to-end | Cypress, WebdriverIO | `mocha-junit-reporter` / `@wdio/junit-reporter` | JUnit format; error wording read |
87
+ | End-to-end | **Playwright** | blob reporter (step 1) — adds traces, selector drift and verified fixes | verified on 5 real projects |
88
+
89
+ ```yaml
90
+ - name: Tewip
91
+ if: always()
92
+ uses: hmamut39/tewip@main
93
+ with:
94
+ junit-report: report.xml # a file, or a directory of them (.xml and .trx)
95
+ suite: unit # only needed when one workflow triages several suites
96
+ ```
97
+
98
+ What it reads from any of these: the failure text, retries, and each test's history. From that it tells apart a wrong value (a real bug once the test has a passing history), an unreachable service (environment), a data collision, and a test that failed then passed on a retry (timing), in the wording each language uses: `expected: <4> but was: <3>` in Java, `got 3, want 4` in Go, `AssertionError: 3 != 4` in Python, `Expected: 4 / Actual: 3` in .NET, and so on. It also reads each browser tool's own words for the same three problems: an element that was never found (Selenium's `NoSuchElementException`, WebdriverIO's `still not displayed`, Cypress's `Expected to find element`), a click another element received (Selenium's `Other element would receive the click`, Cypress's `is being covered by another element`), and an element the page replaced mid-test (Selenium's `StaleElementReferenceException`, Cypress's `the page updated while this command was executing`). No labelled Selenium or Cypress runs have measured those rules yet, so they report at a confidence the merge gate does not trust on its own. What it cannot do without Playwright's traces is look at the page, so selector drift and automatic fixes stay Playwright-only.
99
+
100
+ A wrong value on a test's **first** recorded failure stays undecided rather than guessed: one failure alone does not show whether the test was reliable before. It becomes a real-bug diagnosis once the test's history shows it passing and then failing the same way.
101
+
102
+ When one workflow triages several suites (say `unit` and `e2e`), give each a `suite` name: each then keeps its own history and its own comment, and corrections name the suite, as in `/triage TIMING unit#2 flaky on CI`.
103
+
104
+ ## On your own machine, without CI
105
+
106
+ Install the command once from the release file, `npm install -g tewip-<version>.tgz` (Tewip is not on the npm registry yet; Node 22.18 or newer), run your tests with the report switched on (see the table above), then:
107
+
108
+ ```bash
109
+ tewip explain report.xml # JUnit XML or .trx
110
+ tewip explain reports/ # a folder of them
111
+ tewip explain blob-report/ # Playwright
112
+ tewip ci owner/repo --pr 12 # what Tewip's GitHub Action said about a pull request
113
+ ```
114
+
115
+ It prints the same triage board CI writes: each failure's cause, confidence, evidence and next action. Each run is kept in a `.tewip/` folder in your project (add it to `.gitignore`), so from the second run on, failures are judged against their history exactly as in CI. `--name` keeps separate histories for different projects.
116
+
117
+ ## What you will see on the next failing run
118
+
119
+ - **On the pull request:** one comment listing each failing test, grouped by whether your change likely caused it, each with a cause, a confidence, the evidence and the next action. It updates in place rather than adding a comment per run.
120
+ - **In the job summary:** the same failures as a triage board.
121
+ - **As an issue:** environment failures (5xx responses, connection resets, outages) grouped into one issue instead of one per test.
122
+ - **As an artifact** named `tewip-report`: every role's view, including the engineering-quality view with the flaky tests ranked by the CI minutes they cost.
123
+
124
+ ## Correcting it
125
+
126
+ When it gets one wrong, reply to its comment:
127
+
128
+ ```
129
+ /triage TIMING #2 this one is flaky on our CI runners, not a bug
130
+ ```
131
+
132
+ From the next run on, that failure of that test gets your answer, shown as "corrected by a person" with your name and reason. A correction settles the cause; it never makes the agent change code.
133
+
134
+ ## Asking it questions from your editor
135
+
136
+ - **VS Code, Cursor, Windsurf, VSCodium:** install the Tewip extension (*Extensions → … → Install from VSIX*, `tewip-<version>.vsix`). It connects Tewip to the editor's AI assistant by itself and adds *Tewip: Explain a test report…* and *Tewip: Why did CI fail on this branch?*
137
+ - **JetBrains (2025.2+), Claude, and other MCP clients:** add one server whose command is `tewip mcp`.
138
+
139
+ Then ask, in plain language: "why did CI fail on my PR?", "which tests are flaky in checkout?", "is main safe to release?". Every tool is read-only, and it reads the report the Action already uploaded, so you do not need the other repository cloned.
140
+
141
+ ## Stop flaky tests from blocking merges (optional)
142
+
143
+ Once you trust its answers, let the agent decide whether a failed run should block:
144
+
145
+ ```yaml
146
+ - name: Tests
147
+ continue-on-error: true # the gate below decides
148
+ run: npx playwright test
149
+ - name: Tewip
150
+ if: always()
151
+ uses: hmamut39/tewip@main
152
+ with:
153
+ gate: real-failures
154
+ ```
155
+
156
+ The build then fails only for real bugs, changed selectors, test-data problems and anything undecided. Failures a rule or a person classified as flaky or environment no longer block. On 2,296 real failures this unblocked 84% of the flaky and environment ones without letting a real bug through.
157
+
158
+ ## Turning fixing on, once you trust it
159
+
160
+ ```yaml
161
+ autofix: 'true'
162
+ ```
163
+
164
+ It then rewrites a drifted locator, re-runs that single test, and opens a **separate** pull request only with the changes that passed. Review it like any other pull request: an intended UI change and a regression can look identical from the outside, which is exactly why a human merges it.
165
+
166
+ On real WordPress CI data it would have fixed 14 drifted locators with none wrong, and every locator it suggested matched what the developers themselves wrote.
167
+
168
+ ## If something does not appear
169
+
170
+ | Symptom | Cause |
171
+ |---|---|
172
+ | The step ran but says "no blob report" | The blob reporter is not on in CI, or the tests step did not run |
173
+ | No comment on the pull request | The run was not triggered by a pull request, or `pull-requests: write` is missing |
174
+ | "Resource not accessible by integration" | The workflow's permissions block is missing an entry |
175
+ | Diagnoses look thin on the first day | The agent judges each failure against that test's history, which it builds from your runs; it gets better as runs accumulate |
176
+
177
+ ## What it costs, and what it keeps
178
+
179
+ The agent stores the run history in the Actions cache, never in your repository. Collected failure data and traces stay in CI. The only thing it writes to your repository is a pull request branch, and only when you turn fixing on.
package/dist/cli/help.js CHANGED
@@ -1,272 +1,272 @@
1
1
  // The text `tewip --help` prints. It lives on its own because it is prose, not logic, and it grows
2
2
  // every time a verb is added - which would otherwise push the command file past the size any of
3
3
  // these files are allowed to be.
4
- export const HELP = `Tewip - finds why each test failed, for any stack.
5
-
6
- Usage:
7
- tewip Look at this project and name the two or three commands that
8
- tewip start will tell you something about it, with your own paths in them,
9
- and say plainly what Tewip cannot answer here. Reads the
10
- directory and stops: nothing is run and nothing is written.
11
- Start here. Everything below is the full list.
12
- tewip explain <report> Explain a test report: a JUnit XML or .trx file, a folder of
13
- them, or a Playwright blob-report folder. Keeps history per
14
- project, so later runs are judged against earlier ones.
15
- --name <project> Keep this project's history apart from others.
16
- tewip ci <owner/repo> [--pr N | --branch B]
17
- Show the latest report Tewip's GitHub Action uploaded (uses
18
- your gh login). Without --pr or --branch: the newest run.
19
- tewip plan "<feature>" | --issue PROJ-123 [--diff main...HEAD] [--url <page> ...] [--out plan.md]
20
- Write a test plan grounded in the project: its existing tests,
21
- the change, the live pages and Tewip's failure history.
22
- tewip write [--plan plan.json] [--only LS-002,LS-003] [--framework playwright] [--runs 3]
23
- Write tests for the plan's scenarios in the project's own
24
- framework (Playwright, Cypress, WebdriverIO, Jest, Vitest,
25
- pytest, JUnit, .NET, Go). Keeps a test only if it passes every
26
- run and fails when its check is broken on purpose.
27
- --move-to <folder> Move kept tests there afterwards, leaving the project as it was.
28
- --mutate-source Also change the code under test and check the test notices
29
- (unit tests only; needs the project's own mutation tool).
30
- tewip jira <plan.json> --project PROJ [--issue PROJ-123] [--create]
31
- File the plan's scenarios as test-case issues, linked to the
32
- story. Shows what it would create; --create actually files
33
- them, and the same scenario is never filed twice.
34
- --export xray|zephyr Write the cases as a CSV for Xray or Zephyr Scale instead.
35
- No account needed; importing stays your step.
36
- tewip bugs <report> --project PROJ [--create] [--max 5]
37
- File a bug ticket for each failure Tewip diagnosed as a real
38
- bug, with its evidence and run link. A flaky test or a broken
39
- environment is deliberately not filed, and the same failure is
40
- never filed twice.
41
- tewip secure [--url http://localhost:3000] [--secrets-only] [--gate] [--out security.md]
42
- Runs the scanners this project already has (npm audit,
43
- pip-audit, osv-scanner) plus Tewip's own secret scan, then says
44
- which findings need someone now and which are noise or not
45
- reachable. No model, no cost, and it never prints a credential.
46
- --url <page> Also check the running app: security headers, cookie flags, and
47
- files that must never be served (.env, .git). Ordinary requests
48
- only. An address that is not this machine needs
49
- --i-am-authorised.
50
- --zap-report zap.json Triage the findings of an OWASP ZAP baseline scan as well.
51
- tewip spec "<feature>" [--diff main...HEAD] [--url <page> ...] [--project PROJ] [--create]
52
- An epic, its user stories and their acceptance criteria, from
53
- this project's code, tests and failure history. Anything it
54
- says the system already does is quoted from a file it read;
55
- what it cannot quote becomes an open question. With --project it
56
- shows what it would file in Jira; --create files it.
57
- tewip report <report> [--gate] [--out report.md] [--name <project>] [--no-post]
58
- The CI command for any platform: GitLab CI, Jenkins, CircleCI,
59
- Azure, Bitbucket, Travis, Drone. Diagnoses the run, comments on
60
- the GitLab merge request, tells Slack or Teams only when a
61
- failure needs a person, and with --gate fails the build only on
62
- those failures.
63
- tewip build "<what it should do>" | --issue PROJ-123 [--dry-run] [--pr] [--out feature.patch]
64
- Writes a feature, test first: the test must fail without the
65
- code and pass with it, and every test that was passing must
66
- still pass - otherwise everything goes back byte for byte.
67
- --issue PROJ-123 Build from the ticket itself: its description and acceptance
68
- criteria. Works with a public tracker without an account.
69
- --pr Commit the proved work on a branch of its own and open a draft
70
- pull request. Never merges, never commits to your branch.
71
- tewip review [--diff main...HEAD] [--out review.md]
72
- Before anyone reads the diff: which tests cover this change,
73
- which of them have flaked before, what has gone wrong in these
74
- areas, what nothing tests at all, and the command that runs the
75
- telling tests first. From this project's own history, so it
76
- costs nothing.
77
- tewip watch [--cmd "npm test"] [--report <path>] [--paths src,tests]
78
- Beside your editor: runs your tests when a file changes (or
79
- watches for new reports) and explains each failure as it
80
- happens, without CI.
81
- tewip flaky [--days 30] [--apply] [--name <project>] [--out flaky.md]
82
- Which tests are wasting the team's time, how much CI time they
83
- cost, and what to do with each: quarantine, fix, or watch. Read
84
- from runs already diagnosed, so it costs nothing. --apply puts
85
- the quarantine line in the test file with the evidence above it.
86
- tewip note [--days 7] [--name <project>] [--out note.md] [--post]
87
- The weekly quality note for a lead, written from the runs Tewip
88
- already diagnosed. No model calls, so it costs nothing.
89
- tewip fix --code <report> [--only "<test title>"] [--max 3] [--apply] [--suite "<command>"]
90
- Opt-in: for failures diagnosed as a real bug, propose the
91
- smallest change to application code, and keep it only if the
92
- failing test then passes AND the whole suite still passes. The
93
- change is written as a patch; nothing is committed or merged.
94
- tewip design <figma link | frame.json> [--url <page>] [--frame "<name>"] [--out drift.md]
95
- Compare a Figma frame against the running page: the words the
96
- design specifies and the controls its layer names imply. Says
97
- "the design says X, the page says Y". No model, no cost.
98
- tewip audit [--all] [--gate]
99
- Who ran Tewip here, when, and with what. Each entry carries
100
- the hash of the one before it, so an entry changed or removed
101
- from the middle is caught and named. It says what it cannot
102
- prove, including that it does not survive being deleted.
103
- tewip portfolio <folder of checkouts> [--out page.md|page.json] [--quick]
104
- One page for many repositories: credentials, injection,
105
- containers, licences, accessibility, migrations and tests that
106
- cannot fail, counted per repository. A dash is never a zero - a
107
- check that could not run says so and why. No server, no agent,
108
- no model, nothing leaves the machine.
109
- tewip infra [--out infra.md] [--gate]
110
- What the containers let in: images that run as root, an
111
- unpinned base, a download piped into a shell, the Docker socket
112
- mounted, a privileged container. Reads Dockerfiles and compose
113
- files only, and says what it did not check - no image CVEs, no
114
- cluster, no cloud account. No model, no network.
115
- tewip sbom [--out sbom.json] [--gate]
116
- What this project ships and under what licence: a CycloneDX 1.6
117
- bill of materials, plus the licences that need a person to look
118
- - copyleft, source-available, or none stated at all. It reads
119
- what the project wrote down and names every ecosystem it could
120
- not read, rather than filling the gap. No model, no network.
121
- tewip safety [--out safety.md]
122
- What Tewip may do to your code and what it will not, with the
123
- test that enforces each promise named beside it - and how many
124
- failures it declined to explain in this project. For the person
125
- who has to approve an agent touching the codebase. No model.
126
- tewip correct <kind> "<what>" <verdict> ["why"] | --list
127
- Tell Tewip it was wrong, about anything it decides: a diagnosis,
128
- a flaky verdict, a security finding, a design difference, a
129
- refused build. It answers with your decision from then on, and
130
- says it was yours. It never acts on a correction by itself.
131
- tewip learn [--name <project>] [--labels labels.jsonl] [--out learn.md]
132
- What your corrections add up to: where Tewip is wrong in the
133
- same way more than once, which rule decided it, and the failures
134
- it came from. Says "not yet" rather than inventing a pattern
135
- from two cases. No model, no cost.
136
- tewip pursue "<a goal>" [--act] [--out next.md]
137
- Give Tewip a goal instead of a verb - "keep the checkout flow
138
- green" - and it says what it would do next and why, from what
139
- it can actually see. The order is fixed and written down: a
140
- real bug first, then the environment, then flakiness, then a
141
- test that cannot fail, then a coverage gap. It decides; it does
142
- not act unless you ask. With
143
- --act it carries out the two safe ones - writing a missing test,
144
- or attesting a green goal - by running the command that already
145
- proves them. It will not touch a real bug, and it writes what it
146
- did or refused to .tewip/actions.jsonl.
147
- tewip done [--diff main...HEAD] [--prove] [--strict] [--out done.md] [--json done.json]
148
- Is this work actually done? One verdict, from the layers that each
149
- have a measured accuracy: is anything failing, is the changed code
150
- tested, can those tests fail, would they catch a bug (--prove), and
151
- is what was asked for covered. Answers ready, not ready, or cannot
152
- say - and cannot say is never rounded up to ready. --json writes a
153
- sealed verdict that can be checked later. No model, no cost.
154
- tewip perf <k6.json | lighthouse.json | run.jtl> [--against before.jtl] [--record] [--strict]
155
- Is this change slower, and does it matter? Reads a k6 run (what
156
- the server does under load) or a Lighthouse report (what one
157
- page load costs in a browser) or a JMeter .jtl, and answers two
158
- questions apart:
159
- is anything in a bad state now - from the thresholds you wrote,
160
- or from Lighthouse's own scoring, never a budget Tewip invented
161
- - and is this slower than it was. The second only from repeated
162
- baselines: with fewer than three it says cannot say, because one
163
- run against one cannot tell a regression from a busy machine.
164
- --record adds a run to the baseline, and is deliberately not
165
- automatic: a regression recorded by accident becomes next week's
166
- normal. A .jtl is different and better: it carries every request,
167
- so --against before.jtl compares the two distributions with a
168
- real test rather than a range check - and refuses a difference
169
- that is certain but too small to act on. No cost.
170
- tewip access [<file or directory>] [--report axe.json|lighthouse.json] [--strict]
171
- Can everybody use this? Reads your markup and names the barriers
172
- that reading alone can prove - a picture with no alt text, a field
173
- with no label, a div only a mouse can use - each against its WCAG
174
- 2.2 criterion, which is what a regulator asks for. What only a
175
- person can settle is kept apart from what is proved, and every
176
- criterion it cannot reach without rendering the page (contrast,
177
- focus order, whether the alt text is any good) is printed every
178
- time. It never says a page is accessible, only what was checked.
179
- Give it --report with the JSON from an axe run or a Lighthouse
180
- report and it reads that too - Lighthouse runs axe rules, so
181
- either works. Contrast moves out of the unchecked list, rules
182
- that map to no WCAG criterion are reported as advice rather than
183
- barriers (51 of 174 findings on nine real pages run through axe;
184
- a Lighthouse report runs a WCAG-aligned subset and produced 2 of
185
- 363 on six real pages, so expect little advice from one), and what the
186
- tool could not determine becomes a question. Lighthouse's ten
187
- manual checks are questions too, never passes. No model, no cost.
188
- tewip standards [--diff main...HEAD] [--strict] [--out standards.md]
189
- Teach Tewip your house rules. Markdown files in
190
- .tewip/standards/ - a forbid or require line is checked and
191
- enforced on the files a change touches; the prose below it is
192
- guidance Tewip carries into whatever it writes or fixes. It never
193
- reports guidance as verified, because a rule nobody enforces is
194
- worse than no rule. No model, no cost.
195
- tewip schema [<files>] [--diff main...HEAD] [--strict] [--out schema.md]
196
- What will this migration do to a live database? Names the changes
197
- that break the version still running during a deploy, destroy data,
198
- block writes while they run, or cannot be undone - and searches your
199
- code for what still uses the column it takes away. Reads SQL,
200
- Django, Rails, Alembic, TypeORM, Sequelize, Knex, Laravel, EF Core and
201
- Liquibase - so Prisma, Drizzle, Flyway and golang-migrate too,
202
- since those emit plain SQL - and says so when it cannot read one.
203
- It judges only the forward path, never the rollback. No cost.
204
- tewip pick [--diff main...HEAD] [--list] [--out picked.md]
205
- Which tests could notice this change? Narrows a test run using
206
- facts rather than a prediction: a test runs when it imports a
207
- changed file or reaches one through another module, when the change
208
- edits the test itself, or when it failed recently - an unknown is
209
- never a skip. It runs the whole suite when the change touches
210
- dependencies, build or test configuration, shared setup or
211
- fixtures, or when a changed file cannot be traced to any test.
212
- --list prints the files for piping. No model, no cost.
213
- tewip prove <file> [--diff main...HEAD] [--limit 6] [--budget 600] [--strict]
214
- What would your tests NOT catch? Breaks your code in small,
215
- deliberate ways - a boundary moved by one, a condition inverted,
216
- an and that became an or - runs the tests that import that file,
217
- and reports every change nobody noticed. The line was covered; it
218
- was not checked, and no coverage number can tell you that. Every
219
- file is put back exactly as it was. Slow by nature, and it needs
220
- a unit test framework. No model, no cost.
221
- tewip vouch [<paths>] [--strict] [--out vouch.md] [--json vouch.json]
222
- Could these tests ever fail? Reads tests somebody else wrote -
223
- an agent, a contractor, last year's team - and names the ones
224
- that check nothing, compare a value with itself, leave an
225
- expectation unfinished, swallow their own failure, or never run.
226
- Where the checking is in a helper it cannot follow it says so
227
- instead of accusing. Reads JavaScript, TypeScript, Python, Java,
228
- Kotlin, Scala, Go, C#, Ruby, PHP and Rust. --strict exits
229
- non-zero on a finding, for a gate. No model, no cost.
230
- tewip trace [--only "<ticket or title>"] [--out trace.md]
231
- Is this requirement covered, and is it passing right now? Joins
232
- the plans Tewip wrote to the tests that cover them and the runs
233
- that attempted them. A scenario counts as covered only when a
234
- real test can be named, and passing only when a real run
235
- attempted it. No model, no cost.
236
- tewip recall "<an error, or a question>" [--test "<id>"] [--limit 5]
237
- What this project already concluded: earlier problems that look
238
- like this one, how many separate runs each happened in, what
239
- was decided, by which rule or which person, and the run each
240
- came from. Retries of one failure count once. Searches the
241
- history Tewip keeps - no model, no network, nothing to set up.
242
- tewip incident <sentry.json | otel.json | stack.txt> [--framework <id>] [--out incident.md]
243
- Turn a production incident into a test that proves it: reads
244
- Sentry, OpenTelemetry or a pasted stack trace, writes a test
245
- against the code the trace names, runs it, and keeps it ONLY if
246
- it fails for the incident's own reason. A test that passes, or
247
- fails its own way, is reported and thrown away.
248
- tewip attest [--run <id>] [--out attestation.md] [--json bundle.json]
249
- tewip attest --verify <bundle.json> [--key <public.pem>]
250
- What Tewip checked for a build, as one artifact: the claims with
251
- their evidence and which layer decided each, what it refused to
252
- decide and why, and what it did not check at all. Nothing enters
253
- it that Tewip did not verify itself. Every bundle carries a
254
- digest; set TEWIP_ATTEST_KEY to an ed25519 private key to sign
255
- it too. --verify says what it checked and what that does not
256
- establish - a digest means undamaged, not unedited. No cost.
257
- tewip mcp Serve Tewip over MCP (stdio) for an editor or AI assistant.
258
- tewip a2a [--port 41241] [--host 127.0.0.1]
259
- Answer other agents over A2A, so an agent that writes a diff has
260
- a peer it must satisfy. Serves the agent card at
261
- /.well-known/agent-card.json and takes JSON-RPC "SendMessage".
262
- Only the read-only skills are reachable - nothing it exposes
263
- writes a file - it listens on this machine alone unless you pass
264
- --host, and every request is recorded in .tewip/a2a.jsonl. A
265
- message it cannot place comes back as a question, not a guess.
266
- tewip --version Print the version.
267
-
268
- Reports it reads: Playwright, pytest/unittest, Jest, Vitest, Mocha, node:test, JUnit/TestNG
269
- (Maven, Gradle), Go (gotestsum), .NET (TRX or JUnit), NUnit 3, RSpec, PHPUnit, Rust (nextest),
270
- Robot Framework (output.xml), JMeter (.jtl) and k6 (--summary-export). Appium, Selenium and
271
- WebdriverIO suites report through one of the above, and their own error messages are understood.
4
+ export const HELP = `Tewip - finds why each test failed, for any stack.
5
+
6
+ Usage:
7
+ tewip Look at this project and name the two or three commands that
8
+ tewip start will tell you something about it, with your own paths in them,
9
+ and say plainly what Tewip cannot answer here. Reads the
10
+ directory and stops: nothing is run and nothing is written.
11
+ Start here. Everything below is the full list.
12
+ tewip explain <report> Explain a test report: a JUnit XML or .trx file, a folder of
13
+ them, or a Playwright blob-report folder. Keeps history per
14
+ project, so later runs are judged against earlier ones.
15
+ --name <project> Keep this project's history apart from others.
16
+ tewip ci <owner/repo> [--pr N | --branch B]
17
+ Show the latest report Tewip's GitHub Action uploaded (uses
18
+ your gh login). Without --pr or --branch: the newest run.
19
+ tewip plan "<feature>" | --issue PROJ-123 [--diff main...HEAD] [--url <page> ...] [--out plan.md]
20
+ Write a test plan grounded in the project: its existing tests,
21
+ the change, the live pages and Tewip's failure history.
22
+ tewip write [--plan plan.json] [--only LS-002,LS-003] [--framework playwright] [--runs 3]
23
+ Write tests for the plan's scenarios in the project's own
24
+ framework (Playwright, Cypress, WebdriverIO, Jest, Vitest,
25
+ pytest, JUnit, .NET, Go). Keeps a test only if it passes every
26
+ run and fails when its check is broken on purpose.
27
+ --move-to <folder> Move kept tests there afterwards, leaving the project as it was.
28
+ --mutate-source Also change the code under test and check the test notices
29
+ (unit tests only; needs the project's own mutation tool).
30
+ tewip jira <plan.json> --project PROJ [--issue PROJ-123] [--create]
31
+ File the plan's scenarios as test-case issues, linked to the
32
+ story. Shows what it would create; --create actually files
33
+ them, and the same scenario is never filed twice.
34
+ --export xray|zephyr Write the cases as a CSV for Xray or Zephyr Scale instead.
35
+ No account needed; importing stays your step.
36
+ tewip bugs <report> --project PROJ [--create] [--max 5]
37
+ File a bug ticket for each failure Tewip diagnosed as a real
38
+ bug, with its evidence and run link. A flaky test or a broken
39
+ environment is deliberately not filed, and the same failure is
40
+ never filed twice.
41
+ tewip secure [--url http://localhost:3000] [--secrets-only] [--gate] [--out security.md]
42
+ Runs the scanners this project already has (npm audit,
43
+ pip-audit, osv-scanner) plus Tewip's own secret scan, then says
44
+ which findings need someone now and which are noise or not
45
+ reachable. No model, no cost, and it never prints a credential.
46
+ --url <page> Also check the running app: security headers, cookie flags, and
47
+ files that must never be served (.env, .git). Ordinary requests
48
+ only. An address that is not this machine needs
49
+ --i-am-authorised.
50
+ --zap-report zap.json Triage the findings of an OWASP ZAP baseline scan as well.
51
+ tewip spec "<feature>" [--diff main...HEAD] [--url <page> ...] [--project PROJ] [--create]
52
+ An epic, its user stories and their acceptance criteria, from
53
+ this project's code, tests and failure history. Anything it
54
+ says the system already does is quoted from a file it read;
55
+ what it cannot quote becomes an open question. With --project it
56
+ shows what it would file in Jira; --create files it.
57
+ tewip report <report> [--gate] [--out report.md] [--name <project>] [--no-post]
58
+ The CI command for any platform: GitLab CI, Jenkins, CircleCI,
59
+ Azure, Bitbucket, Travis, Drone. Diagnoses the run, comments on
60
+ the GitLab merge request, tells Slack or Teams only when a
61
+ failure needs a person, and with --gate fails the build only on
62
+ those failures.
63
+ tewip build "<what it should do>" | --issue PROJ-123 [--dry-run] [--pr] [--out feature.patch]
64
+ Writes a feature, test first: the test must fail without the
65
+ code and pass with it, and every test that was passing must
66
+ still pass - otherwise everything goes back byte for byte.
67
+ --issue PROJ-123 Build from the ticket itself: its description and acceptance
68
+ criteria. Works with a public tracker without an account.
69
+ --pr Commit the proved work on a branch of its own and open a draft
70
+ pull request. Never merges, never commits to your branch.
71
+ tewip review [--diff main...HEAD] [--out review.md]
72
+ Before anyone reads the diff: which tests cover this change,
73
+ which of them have flaked before, what has gone wrong in these
74
+ areas, what nothing tests at all, and the command that runs the
75
+ telling tests first. From this project's own history, so it
76
+ costs nothing.
77
+ tewip watch [--cmd "npm test"] [--report <path>] [--paths src,tests]
78
+ Beside your editor: runs your tests when a file changes (or
79
+ watches for new reports) and explains each failure as it
80
+ happens, without CI.
81
+ tewip flaky [--days 30] [--apply] [--name <project>] [--out flaky.md]
82
+ Which tests are wasting the team's time, how much CI time they
83
+ cost, and what to do with each: quarantine, fix, or watch. Read
84
+ from runs already diagnosed, so it costs nothing. --apply puts
85
+ the quarantine line in the test file with the evidence above it.
86
+ tewip note [--days 7] [--name <project>] [--out note.md] [--post]
87
+ The weekly quality note for a lead, written from the runs Tewip
88
+ already diagnosed. No model calls, so it costs nothing.
89
+ tewip fix --code <report> [--only "<test title>"] [--max 3] [--apply] [--suite "<command>"]
90
+ Opt-in: for failures diagnosed as a real bug, propose the
91
+ smallest change to application code, and keep it only if the
92
+ failing test then passes AND the whole suite still passes. The
93
+ change is written as a patch; nothing is committed or merged.
94
+ tewip design <figma link | frame.json> [--url <page>] [--frame "<name>"] [--out drift.md]
95
+ Compare a Figma frame against the running page: the words the
96
+ design specifies and the controls its layer names imply. Says
97
+ "the design says X, the page says Y". No model, no cost.
98
+ tewip audit [--all] [--gate]
99
+ Who ran Tewip here, when, and with what. Each entry carries
100
+ the hash of the one before it, so an entry changed or removed
101
+ from the middle is caught and named. It says what it cannot
102
+ prove, including that it does not survive being deleted.
103
+ tewip portfolio <folder of checkouts> [--out page.md|page.json] [--quick]
104
+ One page for many repositories: credentials, injection,
105
+ containers, licences, accessibility, migrations and tests that
106
+ cannot fail, counted per repository. A dash is never a zero - a
107
+ check that could not run says so and why. No server, no agent,
108
+ no model, nothing leaves the machine.
109
+ tewip infra [--out infra.md] [--gate]
110
+ What the containers let in: images that run as root, an
111
+ unpinned base, a download piped into a shell, the Docker socket
112
+ mounted, a privileged container. Reads Dockerfiles and compose
113
+ files only, and says what it did not check - no image CVEs, no
114
+ cluster, no cloud account. No model, no network.
115
+ tewip sbom [--out sbom.json] [--gate]
116
+ What this project ships and under what licence: a CycloneDX 1.6
117
+ bill of materials, plus the licences that need a person to look
118
+ - copyleft, source-available, or none stated at all. It reads
119
+ what the project wrote down and names every ecosystem it could
120
+ not read, rather than filling the gap. No model, no network.
121
+ tewip safety [--out safety.md]
122
+ What Tewip may do to your code and what it will not, with the
123
+ test that enforces each promise named beside it - and how many
124
+ failures it declined to explain in this project. For the person
125
+ who has to approve an agent touching the codebase. No model.
126
+ tewip correct <kind> "<what>" <verdict> ["why"] | --list
127
+ Tell Tewip it was wrong, about anything it decides: a diagnosis,
128
+ a flaky verdict, a security finding, a design difference, a
129
+ refused build. It answers with your decision from then on, and
130
+ says it was yours. It never acts on a correction by itself.
131
+ tewip learn [--name <project>] [--labels labels.jsonl] [--out learn.md]
132
+ What your corrections add up to: where Tewip is wrong in the
133
+ same way more than once, which rule decided it, and the failures
134
+ it came from. Says "not yet" rather than inventing a pattern
135
+ from two cases. No model, no cost.
136
+ tewip pursue "<a goal>" [--act] [--out next.md]
137
+ Give Tewip a goal instead of a verb - "keep the checkout flow
138
+ green" - and it says what it would do next and why, from what
139
+ it can actually see. The order is fixed and written down: a
140
+ real bug first, then the environment, then flakiness, then a
141
+ test that cannot fail, then a coverage gap. It decides; it does
142
+ not act unless you ask. With
143
+ --act it carries out the two safe ones - writing a missing test,
144
+ or attesting a green goal - by running the command that already
145
+ proves them. It will not touch a real bug, and it writes what it
146
+ did or refused to .tewip/actions.jsonl.
147
+ tewip done [--diff main...HEAD] [--prove] [--strict] [--out done.md] [--json done.json]
148
+ Is this work actually done? One verdict, from the layers that each
149
+ have a measured accuracy: is anything failing, is the changed code
150
+ tested, can those tests fail, would they catch a bug (--prove), and
151
+ is what was asked for covered. Answers ready, not ready, or cannot
152
+ say - and cannot say is never rounded up to ready. --json writes a
153
+ sealed verdict that can be checked later. No model, no cost.
154
+ tewip perf <k6.json | lighthouse.json | run.jtl> [--against before.jtl] [--record] [--strict]
155
+ Is this change slower, and does it matter? Reads a k6 run (what
156
+ the server does under load) or a Lighthouse report (what one
157
+ page load costs in a browser) or a JMeter .jtl, and answers two
158
+ questions apart:
159
+ is anything in a bad state now - from the thresholds you wrote,
160
+ or from Lighthouse's own scoring, never a budget Tewip invented
161
+ - and is this slower than it was. The second only from repeated
162
+ baselines: with fewer than three it says cannot say, because one
163
+ run against one cannot tell a regression from a busy machine.
164
+ --record adds a run to the baseline, and is deliberately not
165
+ automatic: a regression recorded by accident becomes next week's
166
+ normal. A .jtl is different and better: it carries every request,
167
+ so --against before.jtl compares the two distributions with a
168
+ real test rather than a range check - and refuses a difference
169
+ that is certain but too small to act on. No cost.
170
+ tewip access [<file or directory>] [--report axe.json|lighthouse.json] [--strict]
171
+ Can everybody use this? Reads your markup and names the barriers
172
+ that reading alone can prove - a picture with no alt text, a field
173
+ with no label, a div only a mouse can use - each against its WCAG
174
+ 2.2 criterion, which is what a regulator asks for. What only a
175
+ person can settle is kept apart from what is proved, and every
176
+ criterion it cannot reach without rendering the page (contrast,
177
+ focus order, whether the alt text is any good) is printed every
178
+ time. It never says a page is accessible, only what was checked.
179
+ Give it --report with the JSON from an axe run or a Lighthouse
180
+ report and it reads that too - Lighthouse runs axe rules, so
181
+ either works. Contrast moves out of the unchecked list, rules
182
+ that map to no WCAG criterion are reported as advice rather than
183
+ barriers (51 of 174 findings on nine real pages run through axe;
184
+ a Lighthouse report runs a WCAG-aligned subset and produced 2 of
185
+ 363 on six real pages, so expect little advice from one), and what the
186
+ tool could not determine becomes a question. Lighthouse's ten
187
+ manual checks are questions too, never passes. No model, no cost.
188
+ tewip standards [--diff main...HEAD] [--strict] [--out standards.md]
189
+ Teach Tewip your house rules. Markdown files in
190
+ .tewip/standards/ - a forbid or require line is checked and
191
+ enforced on the files a change touches; the prose below it is
192
+ guidance Tewip carries into whatever it writes or fixes. It never
193
+ reports guidance as verified, because a rule nobody enforces is
194
+ worse than no rule. No model, no cost.
195
+ tewip schema [<files>] [--diff main...HEAD] [--strict] [--out schema.md]
196
+ What will this migration do to a live database? Names the changes
197
+ that break the version still running during a deploy, destroy data,
198
+ block writes while they run, or cannot be undone - and searches your
199
+ code for what still uses the column it takes away. Reads SQL,
200
+ Django, Rails, Alembic, TypeORM, Sequelize, Knex, Laravel, EF Core and
201
+ Liquibase - so Prisma, Drizzle, Flyway and golang-migrate too,
202
+ since those emit plain SQL - and says so when it cannot read one.
203
+ It judges only the forward path, never the rollback. No cost.
204
+ tewip pick [--diff main...HEAD] [--list] [--out picked.md]
205
+ Which tests could notice this change? Narrows a test run using
206
+ facts rather than a prediction: a test runs when it imports a
207
+ changed file or reaches one through another module, when the change
208
+ edits the test itself, or when it failed recently - an unknown is
209
+ never a skip. It runs the whole suite when the change touches
210
+ dependencies, build or test configuration, shared setup or
211
+ fixtures, or when a changed file cannot be traced to any test.
212
+ --list prints the files for piping. No model, no cost.
213
+ tewip prove <file> [--diff main...HEAD] [--limit 6] [--budget 600] [--strict]
214
+ What would your tests NOT catch? Breaks your code in small,
215
+ deliberate ways - a boundary moved by one, a condition inverted,
216
+ an and that became an or - runs the tests that import that file,
217
+ and reports every change nobody noticed. The line was covered; it
218
+ was not checked, and no coverage number can tell you that. Every
219
+ file is put back exactly as it was. Slow by nature, and it needs
220
+ a unit test framework. No model, no cost.
221
+ tewip vouch [<paths>] [--strict] [--out vouch.md] [--json vouch.json]
222
+ Could these tests ever fail? Reads tests somebody else wrote -
223
+ an agent, a contractor, last year's team - and names the ones
224
+ that check nothing, compare a value with itself, leave an
225
+ expectation unfinished, swallow their own failure, or never run.
226
+ Where the checking is in a helper it cannot follow it says so
227
+ instead of accusing. Reads JavaScript, TypeScript, Python, Java,
228
+ Kotlin, Scala, Go, C#, Ruby, PHP and Rust. --strict exits
229
+ non-zero on a finding, for a gate. No model, no cost.
230
+ tewip trace [--only "<ticket or title>"] [--out trace.md]
231
+ Is this requirement covered, and is it passing right now? Joins
232
+ the plans Tewip wrote to the tests that cover them and the runs
233
+ that attempted them. A scenario counts as covered only when a
234
+ real test can be named, and passing only when a real run
235
+ attempted it. No model, no cost.
236
+ tewip recall "<an error, or a question>" [--test "<id>"] [--limit 5]
237
+ What this project already concluded: earlier problems that look
238
+ like this one, how many separate runs each happened in, what
239
+ was decided, by which rule or which person, and the run each
240
+ came from. Retries of one failure count once. Searches the
241
+ history Tewip keeps - no model, no network, nothing to set up.
242
+ tewip incident <sentry.json | otel.json | stack.txt> [--framework <id>] [--out incident.md]
243
+ Turn a production incident into a test that proves it: reads
244
+ Sentry, OpenTelemetry or a pasted stack trace, writes a test
245
+ against the code the trace names, runs it, and keeps it ONLY if
246
+ it fails for the incident's own reason. A test that passes, or
247
+ fails its own way, is reported and thrown away.
248
+ tewip attest [--run <id>] [--out attestation.md] [--json bundle.json]
249
+ tewip attest --verify <bundle.json> [--key <public.pem>]
250
+ What Tewip checked for a build, as one artifact: the claims with
251
+ their evidence and which layer decided each, what it refused to
252
+ decide and why, and what it did not check at all. Nothing enters
253
+ it that Tewip did not verify itself. Every bundle carries a
254
+ digest; set TEWIP_ATTEST_KEY to an ed25519 private key to sign
255
+ it too. --verify says what it checked and what that does not
256
+ establish - a digest means undamaged, not unedited. No cost.
257
+ tewip mcp Serve Tewip over MCP (stdio) for an editor or AI assistant.
258
+ tewip a2a [--port 41241] [--host 127.0.0.1]
259
+ Answer other agents over A2A, so an agent that writes a diff has
260
+ a peer it must satisfy. Serves the agent card at
261
+ /.well-known/agent-card.json and takes JSON-RPC "SendMessage".
262
+ Only the read-only skills are reachable - nothing it exposes
263
+ writes a file - it listens on this machine alone unless you pass
264
+ --host, and every request is recorded in .tewip/a2a.jsonl. A
265
+ message it cannot place comes back as a question, not a guess.
266
+ tewip --version Print the version.
267
+
268
+ Reports it reads: Playwright, pytest/unittest, Jest, Vitest, Mocha, node:test, JUnit/TestNG
269
+ (Maven, Gradle), Go (gotestsum), .NET (TRX or JUnit), NUnit 3, RSpec, PHPUnit, Rust (nextest),
270
+ Robot Framework (output.xml), JMeter (.jtl) and k6 (--summary-export). Appium, Selenium and
271
+ WebdriverIO suites report through one of the above, and their own error messages are understood.
272
272
  Set OPENAI_API_KEY to let the AI layer decide what the rules leave open.`;
package/dist/engine/autofix/code-propose.js CHANGED
@@ -1,12 +1,12 @@
1
1
  export const CODE_FIX_PROMPT_VERSION = 'code-fix-1';
2
- const SYSTEM = `You are Tewip's fixer. A test is failing against this code. Find the smallest change to the APPLICATION code that makes the test pass because the bug is gone, not because the check was avoided.
3
-
4
- Rules:
5
- - Change application code only. Never suggest changing the test, the expected values in it, configuration, or anything that silences the failure.
6
- - The change must be minimal and local: the fewest lines that fix the cause. Do not refactor, rename, reformat, add logging, or "improve" anything nearby.
7
- - before must be copied character for character from the file you were given, including indentation, and must appear in it exactly once. Include enough surrounding lines to be unique. Copy only what is in the file: nothing is added to it, so do not add line numbers or markers.
8
- - after must keep the file valid in its language and keep the behaviour every other test relies on.
9
- - If the code is right and the test is wrong, or if you cannot see the cause in the file you were given, set fault to "test" and return before and after as empty strings. Saying "I cannot see it" is a correct answer; a guess is not.
2
+ const SYSTEM = `You are Tewip's fixer. A test is failing against this code. Find the smallest change to the APPLICATION code that makes the test pass because the bug is gone, not because the check was avoided.
3
+
4
+ Rules:
5
+ - Change application code only. Never suggest changing the test, the expected values in it, configuration, or anything that silences the failure.
6
+ - The change must be minimal and local: the fewest lines that fix the cause. Do not refactor, rename, reformat, add logging, or "improve" anything nearby.
7
+ - before must be copied character for character from the file you were given, including indentation, and must appear in it exactly once. Include enough surrounding lines to be unique. Copy only what is in the file: nothing is added to it, so do not add line numbers or markers.
8
+ - after must keep the file valid in its language and keep the behaviour every other test relies on.
9
+ - If the code is right and the test is wrong, or if you cannot see the cause in the file you were given, set fault to "test" and return before and after as empty strings. Saying "I cannot see it" is a correct answer; a guess is not.
10
10
  - why is for a reviewer: what was wrong and why this fixes it.`;
11
11
  const SCHEMA = {
12
12
  type: 'object',
package/dist/engine/build/build.js CHANGED
@@ -1,29 +1,29 @@
1
1
  export const BUILD_PROMPT_VERSION = 'build-1';
2
- const RULES = `Rules that do not bend:
3
- - Use only what the project already has: its framework, its style, its helpers, its way of importing. Copy the shape of the example test and the example source given.
4
- - Never weaken or delete an existing test, and never change what one asserts.
5
- - Keep the change small and local. Add a file when the project's own layout calls for one; do not reorganise anything.
6
- - Write no comments that restate the code, no logging, no defensive try/catch around things that cannot fail.
2
+ const RULES = `Rules that do not bend:
3
+ - Use only what the project already has: its framework, its style, its helpers, its way of importing. Copy the shape of the example test and the example source given.
4
+ - Never weaken or delete an existing test, and never change what one asserts.
5
+ - Keep the change small and local. Add a file when the project's own layout calls for one; do not reorganise anything.
6
+ - Write no comments that restate the code, no logging, no defensive try/catch around things that cannot fail.
7
7
  - before must be copied character for character from the file given and appear in it exactly once. For a new file, before is empty and after is the whole file.`;
8
- const TEST_SYSTEM = `You are Tewip, writing the test for a feature that does not exist yet.
9
-
10
- Write ONE test that checks the described behaviour, in the project's own framework and style. It must fail when run now, and fail for the right reason: the behaviour is missing, not because of a typo, a missing import or a file that does not exist. Use only functions, files and exports that already exist, or the one new export the feature obviously needs - name it as the project would.
11
-
12
- checks is one sentence naming the behaviour the test asserts ("a full URL keeps its query when the trailing slash is removed"). It is not advice about file names or where to put things.
13
- failsBecause is the behaviour that is missing today, in one sentence.
14
-
8
+ const TEST_SYSTEM = `You are Tewip, writing the test for a feature that does not exist yet.
9
+
10
+ Write ONE test that checks the described behaviour, in the project's own framework and style. It must fail when run now, and fail for the right reason: the behaviour is missing, not because of a typo, a missing import or a file that does not exist. Use only functions, files and exports that already exist, or the one new export the feature obviously needs - name it as the project would.
11
+
12
+ checks is one sentence naming the behaviour the test asserts ("a full URL keeps its query when the trailing slash is removed"). It is not advice about file names or where to put things.
13
+ failsBecause is the behaviour that is missing today, in one sentence.
14
+
15
15
  ${RULES}`;
16
- const CODE_SYSTEM = `You are Tewip, writing the code that makes a failing test pass.
17
-
18
- You are given the feature, the test, the real error it produced, and the project's relevant source. Write the smallest change that makes that test pass because the behaviour now exists - never by weakening the test, never by special-casing the test's own values.
19
-
20
- If the feature cannot be built from what you were given - the code you need is not here, or the request is ambiguous in a way that changes the answer - set outcome to "cannot" and return no edits. Saying "I cannot" is a correct answer; a guess that happens to pass one test is not.
21
-
16
+ const CODE_SYSTEM = `You are Tewip, writing the code that makes a failing test pass.
17
+
18
+ You are given the feature, the test, the real error it produced, and the project's relevant source. Write the smallest change that makes that test pass because the behaviour now exists - never by weakening the test, never by special-casing the test's own values.
19
+
20
+ If the feature cannot be built from what you were given - the code you need is not here, or the request is ambiguous in a way that changes the answer - set outcome to "cannot" and return no edits. Saying "I cannot" is a correct answer; a guess that happens to pass one test is not.
21
+
22
22
  ${RULES}`;
23
- const REPAIR_SYSTEM = `You are Tewip, fixing code you just wrote. The test still fails, or other tests broke. You are given what you changed and the real output.
24
-
25
- Change your own code so the failing test passes and everything that was passing still passes. Never change the test. If you cannot, set outcome to "cannot".
26
-
23
+ const REPAIR_SYSTEM = `You are Tewip, fixing code you just wrote. The test still fails, or other tests broke. You are given what you changed and the real output.
24
+
25
+ Change your own code so the failing test passes and everything that was passing still passes. Never change the test. If you cannot, set outcome to "cannot".
26
+
27
27
  ${RULES}`;
28
28
  const TEST_SCHEMA = {
29
29
  type: 'object',
package/dist/engine/llm/prompt.js CHANGED
@@ -3,28 +3,28 @@
3
3
  import { ROOT_CAUSES } from '../../reporters/types.js';
4
4
  /** Bump whenever the prompt, schema or context format changes; it is part of the cache key. */
5
5
  export const PROMPT_VERSION = 'llm-6';
6
- export const SYSTEM_PROMPT = `You are the root-cause classifier of a Playwright test-failure triage agent.
7
- You receive a structured summary of ONE failed test: the error, the test's run history, what the deterministic rules already checked, the test steps, the page at failure time (accessibility snapshot + notable DOM elements), failed network requests and console errors.
8
-
9
- Classify the root cause into exactly one of these categories:
10
- - REAL_BUG: the application under test behaves incorrectly. Examples: a wrong value is displayed; a feature or element the test needs is absent from the page and nothing equivalent exists; the app never reaches the expected state even though nothing external failed; uncaught page errors from application code.
11
- - LOCATOR_DRIFT: the element the test wants is on the page, but the selector no longer matches it (renamed id, class, test id or text). The page summary must show an element with the same purpose.
12
- - TIMING: the expected element or state does arrive, but not in time, or a race makes the result vary between attempts or runs. Signs: another attempt of the same code passed, intermittent history with varying errors, hard-coded waits, the element resolved but was not yet visible, stable or enabled.
13
- - ENV: something outside the app code and the test broke: failed or 5xx/502/503 responses from a backend or third-party service, connection or DNS errors, expired auth or credentials, or many tests in the same run failing the same way.
14
- - TEST_DATA: the test's preconditions or fixture data are wrong: missing or stale seeded data, a user or account in the wrong state, an expected value that depends on data the test did not set up, a 404 for a record the test assumed exists.
15
-
16
- How to decide:
17
- - Use only facts present in the summary. Do not invent page content, requests or history.
18
- - Prefer the category the evidence directly shows over one that is merely possible.
19
- - An element that is missing with no similar element on the page is not LOCATOR_DRIFT.
20
- - Playwright matches accessible names by substring, so a locator can match several elements at once ("strict mode violation"). That is LOCATOR_DRIFT only when one of the matched elements is the one the test means, which usually carries the locator's name as a whole word ("Edit" -> "Edit link"). When every match only contains the text inside a longer word or phrase ("Edit" -> "Editor canvas", "Gift" -> "Gifting options"), the element the test means is absent: judge it as REAL_BUG, TIMING or TEST_DATA on the other evidence, not LOCATOR_DRIFT.
21
- - A failed request or server error is only ENV evidence if the test did not provoke it. When "test.exercisesErrorHandling" is set, the test deliberately triggers failures (rejections, server errors, retries), so a failed request is the scenario under test: judge whether the app handled it as the test expects. If it did not, that is REAL_BUG, not ENV. A real environment outage also tends to hit unrelated tests and to pass on retry, rather than failing every attempt on one commit while other commits pass.
22
- - Check "thisRun" first. If anotherAttemptPassed is true, the same code both failed and passed, so the cause is not deterministic: a real bug or a drifted locator would fail every attempt. Choose among TIMING, ENV and TEST_DATA.
23
- - Then read "history.pattern", which compares this commit's attempts with other commits. If every attempt on this commit failed while other commits passed, something in this commit changed behaviour: choose among REAL_BUG, LOCATOR_DRIFT and TEST_DATA using the page summary, not TIMING. An overall failure rate that mixes different commits is not evidence of flakiness; only failing and passing on the same code is.
24
- - The "rules" section lists what the deterministic rules already examined and why they declined; treat those as checked facts.
25
-
26
- Confidence (0 to 1): 0.9 or higher only when the summary directly shows the cause; 0.6-0.8 when it is the most likely reading but another category remains plausible; below 0.5 when you are mostly guessing. Low confidence routes the failure to a human, so do not inflate it.
27
-
6
+ export const SYSTEM_PROMPT = `You are the root-cause classifier of a Playwright test-failure triage agent.
7
+ You receive a structured summary of ONE failed test: the error, the test's run history, what the deterministic rules already checked, the test steps, the page at failure time (accessibility snapshot + notable DOM elements), failed network requests and console errors.
8
+
9
+ Classify the root cause into exactly one of these categories:
10
+ - REAL_BUG: the application under test behaves incorrectly. Examples: a wrong value is displayed; a feature or element the test needs is absent from the page and nothing equivalent exists; the app never reaches the expected state even though nothing external failed; uncaught page errors from application code.
11
+ - LOCATOR_DRIFT: the element the test wants is on the page, but the selector no longer matches it (renamed id, class, test id or text). The page summary must show an element with the same purpose.
12
+ - TIMING: the expected element or state does arrive, but not in time, or a race makes the result vary between attempts or runs. Signs: another attempt of the same code passed, intermittent history with varying errors, hard-coded waits, the element resolved but was not yet visible, stable or enabled.
13
+ - ENV: something outside the app code and the test broke: failed or 5xx/502/503 responses from a backend or third-party service, connection or DNS errors, expired auth or credentials, or many tests in the same run failing the same way.
14
+ - TEST_DATA: the test's preconditions or fixture data are wrong: missing or stale seeded data, a user or account in the wrong state, an expected value that depends on data the test did not set up, a 404 for a record the test assumed exists.
15
+
16
+ How to decide:
17
+ - Use only facts present in the summary. Do not invent page content, requests or history.
18
+ - Prefer the category the evidence directly shows over one that is merely possible.
19
+ - An element that is missing with no similar element on the page is not LOCATOR_DRIFT.
20
+ - Playwright matches accessible names by substring, so a locator can match several elements at once ("strict mode violation"). That is LOCATOR_DRIFT only when one of the matched elements is the one the test means, which usually carries the locator's name as a whole word ("Edit" -> "Edit link"). When every match only contains the text inside a longer word or phrase ("Edit" -> "Editor canvas", "Gift" -> "Gifting options"), the element the test means is absent: judge it as REAL_BUG, TIMING or TEST_DATA on the other evidence, not LOCATOR_DRIFT.
21
+ - A failed request or server error is only ENV evidence if the test did not provoke it. When "test.exercisesErrorHandling" is set, the test deliberately triggers failures (rejections, server errors, retries), so a failed request is the scenario under test: judge whether the app handled it as the test expects. If it did not, that is REAL_BUG, not ENV. A real environment outage also tends to hit unrelated tests and to pass on retry, rather than failing every attempt on one commit while other commits pass.
22
+ - Check "thisRun" first. If anotherAttemptPassed is true, the same code both failed and passed, so the cause is not deterministic: a real bug or a drifted locator would fail every attempt. Choose among TIMING, ENV and TEST_DATA.
23
+ - Then read "history.pattern", which compares this commit's attempts with other commits. If every attempt on this commit failed while other commits passed, something in this commit changed behaviour: choose among REAL_BUG, LOCATOR_DRIFT and TEST_DATA using the page summary, not TIMING. An overall failure rate that mixes different commits is not evidence of flakiness; only failing and passing on the same code is.
24
+ - The "rules" section lists what the deterministic rules already examined and why they declined; treat those as checked facts.
25
+
26
+ Confidence (0 to 1): 0.9 or higher only when the summary directly shows the cause; 0.6-0.8 when it is the most likely reading but another category remains plausible; below 0.5 when you are mostly guessing. Low confidence routes the failure to a human, so do not inflate it.
27
+
28
28
  Return JSON only: the category, the confidence, a short reasoning (at most 3 sentences), and 1-4 evidence items quoting or pointing at specific facts from the summary.`;
29
29
  export const OUTPUT_SCHEMA_NAME = 'triage_diagnosis';
30
30
  export const OUTPUT_SCHEMA = {
package/dist/engine/vouch/assertions.js CHANGED
@@ -14,6 +14,14 @@ const CHECKS = [
14
14
  /\bshould\s*\.\s*not\b/,
15
15
  /\bassert\b\s*[.(]/,
16
16
  /\bassert(Equals?|True|False|Null|NotNull|That|Throws|Match|Contains|Rejects|Fails)\b/i,
17
+ // Any call named assertSomething. The list above needs the suffix to end on a word boundary, so
18
+ // keycloak's own `assertNullAdditionalJdbcProperty(config, null, prop)` matched nothing: `Null` is
19
+ // followed by `A`. Four of its ConfigurationTest tests delegate to a helper built out of those
20
+ // calls, and the helper was being credited only because the *string* `--db-tls-mode=verify-server`
21
+ // happened to contain the word `verify` - so narrowing `verify` to a call turned four correct
22
+ // verdicts into accusations. The corpus caught it. A call named `assertX(` is the universal
23
+ // convention for a custom assertion, and reading it as one is both broader and the actual reason.
24
+ /\bassert[A-Z]\w*\s*\(/,
17
25
  /\.\s*to\s*\.\s*(equal|be|deep|have|include|throw|match|exist)\b/,
18
26
  /\btoMatch(Snapshot|InlineSnapshot|ImageSnapshot)\s*\(/,
19
27
  /\bexpect(Type|TypeOf)\s*[<(]/,
package/dist/engine/walk.js CHANGED
@@ -7,7 +7,7 @@
7
7
  //
8
8
  // The cap now counts the files the caller actually wants. A budget spent on files the caller will
9
9
  // discard is not a budget.
10
- import { readdirSync, statSync } from 'node:fs';
10
+ import { lstatSync, readdirSync } from 'node:fs';
11
11
  import { join } from 'node:path';
12
12
  const SKIP_DIR = new Set(['node_modules', '.git', 'dist', 'build', 'out', 'coverage', 'target', 'vendor', '__pycache__', '.venv', 'venv', '.tewip', '.next', '.cache']);
13
13
  const BINARY = /\.(?:png|jpe?g|gif|webp|ico|svg|pdf|zip|gz|tar|exe|dll|so|dylib|woff2?|ttf|eot|mp4|mp3|wav|class|jar|pyc|lock)$/i;
@@ -35,15 +35,42 @@ export function scanSourceFiles(root, limit = 5000, keep) {
35
35
  catch {
36
36
  return;
37
37
  }
38
- for (const entry of entries) {
38
+ // Sorted, so a capped scan reads the same files everywhere.
39
+ //
40
+ // `readdir` returns whatever order the filesystem gives, which differs between platforms. That
41
+ // does not matter while everything is read - but `secrets` stops at 5,000 files on a large
42
+ // project, and then the order decides *which* 5,000. n8n is over the cap, and its findings came
43
+ // out as 34/21/13 on Windows and 33/18/15 on Linux from the same pinned commit: not a different
44
+ // answer about the same files, a different set of files. A number that depends on the order the
45
+ // filesystem happened to list a directory in is not reproducible, and a capped scan is exactly
46
+ // where that shows.
47
+ for (const entry of entries.sort()) {
39
48
  const path = join(dir, entry);
40
49
  let info;
41
50
  try {
42
- info = statSync(path);
51
+ // `lstat`, not `stat`: a symlink must be seen as a symlink. See below.
52
+ info = lstatSync(path);
43
53
  }
44
54
  catch {
45
55
  continue;
46
56
  }
57
+ // A symlink is neither walked nor read.
58
+ //
59
+ // `stat` follows them, so a link to a directory looked like a directory and was walked. In
60
+ // immich, `fastlane/metadata` points at `../mobile/android/fastlane/metadata` inside the same
61
+ // repository: following it read 77 files a second time, under a path they do not really have.
62
+ // A finding in one of them would have been reported twice, at two different paths, and the
63
+ // file count no longer meant what the report said it meant.
64
+ //
65
+ // Worse than the double count: a link pointing out of the project would have Tewip reading
66
+ // files outside the directory it was given, which it must never do whatever is on the other
67
+ // end. Anything genuinely inside the project is still reached by its real path.
68
+ //
69
+ // Windows was already doing the right thing by accident - git writes a symlink there as a
70
+ // small text file - which is exactly why this hid: the numbers were recorded on Windows and
71
+ // only the Linux corpus run disagreed.
72
+ if (info.isSymbolicLink())
73
+ continue;
47
74
  if (info.isDirectory()) {
48
75
  if (!SKIP_DIR.has(entry))
49
76
  walk(path);
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "tewip-ai",
3
- "version": "0.72.0",
3
+ "version": "0.73.1",
4
4
  "mcpName": "io.github.hmamut39/tewip",
5
5
  "description": "Tewip finds why each failing test failed - real bug, flaky test, environment, test data or changed selector - for any stack, from the command line, CI, or your editor.",
6
6
  "keywords": [