tewip-ai 0.16.1 → 0.16.3

package/README.md

@@ -16,6 +16,22 @@ Formerly *triage-agent*: the repository moved to `hmamut39/tewip`, and GitHub re
 
 **Status: phases 1 to 14 are built; the two auto-fix phases (5 and 8) are the ones still in progress.** Tewip collects failures from any framework, diagnoses them with rules first and a model only for what rules cannot settle, and turns that into work for each role: a CI comment and merge gate, test plans and verified tests, a flake dossier, a PR-risk review, security scanning, tickets in Jira, Rally, GitHub Issues, Azure Boards or Linear, checks generated from a Figma design, specifications written for people, and - opt in, and only when proved - fixes to test or application code. It runs live in GitHub CI and answers questions from AI assistants through MCP. On fresh real Gutenberg CI failures, with the pipeline frozen before labelling, accuracy is 93.4%; across six real datasets it ranges from 86.1% to 99.7%, and the caveats are in [eval/README.md](eval/README.md).
 
+## What it costs
+
+Most failures never reach a model. On 3,556 real CI failures from six public projects, the
+deterministic rules answered **75.1%** on their own - and on the labelled ones they answered
+correctly **1,979 times out of 2,012 (98.4%)**. A rule-answered failure costs nothing, explains
+itself, and gives the same answer every time.
+
+Only the remaining quarter is sent to a model, so triaging **1,000 real failures costs about
+$1.12** at gpt-5.4-mini's published price. That is measured, not estimated: three real uncached
+calls on held-out failures billed 3,624 prompt and 499 completion tokens each. A real week costs
+less, because a failure that repeats is answered from cache, and a failure someone has corrected
+never goes to the model again.
+
+Tewip prints what every command costs, caps its own spend, and works with no key at all - the rules
+run on their own and everything they cannot settle is left as UNKNOWN rather than guessed.
+
 ## Use it in your project
 
 **Giving it to your team (or another company): [TEAM-GUIDE.md](TEAM-GUIDE.md)** — what to install, what each role uses it for, and what it never does.
@@ -143,6 +159,15 @@ tewip write --only CO-002,CO-004 --framework pytest
 ```
 
 - **`tewip plan`** reads what is really there: the tests you already have (in any language), the change (`--diff main...HEAD`), the live pages (what a user sees on each page, via Playwright), and the failures Tewip has diagnosed in this project before. The plan lists risks and scenarios by priority. A claim that a scenario is "already covered" is checked against your real test names, and dropped if no such test exists. Saved to `.tewip/plans/` as Markdown to review and JSON for `write`.
+- **Does a plan catch real bugs?** Measured on 18 real pull requests that were later found to have
+  broken something: the repository is fetched at the pull request's own head, the plan written from
+  the diff a reviewer saw, and nothing about the bug is known to it. **In 8 of the 9 pairs that
+  could be judged, a scenario named the bug** - in `matrix-org/complement`, `NLnetLabs/domain`,
+  `Bioconductor/rhdf5`, `Try/OpenGothic` and others. One pull request was titled "Resolve nightly
+  warnings" and the plan still found the behaviour change hidden inside it. The caveats are real
+  and written out in [ROADMAP.md](ROADMAP.md): nine judged pairs is a small sample, Claude did the
+  judging rather than each project's engineers, and the one miss is the interesting case - when a
+  pull request's *intention* is wrong, a plan grounded in that intention agrees with it.
 - **`tewip write`** writes each scenario as a test in **your** framework and style: Playwright, Cypress, WebdriverIO, Angular, Jest, Vitest, pytest (with Selenium or Playwright), JUnit on Maven or Gradle (Selenium, Spring Boot), .NET or Go. It copies the style of an existing test and uses only elements seen on the real page. Front-end apps get component tests the way their own tests are written: React, Vue or Svelte with Testing Library under Jest or Vitest, and Angular through `ng test` (Karma and Jasmine, or Vitest). A project with both e2e and unit tests gets browser tests when the plan read pages, and unit or component tests when it did not. Then it runs the test. A draft is **kept only if** it passes every run (3 by default) **and** fails once its main check is broken on purpose, because a test that cannot fail is worse than no test. A failing draft is repaired from the real error, twice at most. If the test looks right and the app does something else, Tewip reports a **possible bug** instead of weakening the test until it passes. Kept tests are new files beside yours, for you to review and commit; nothing is committed, and no application code is touched.
 
 ## Check the page against the design (Phase 11, new)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "tewip-ai",
-  "version": "0.16.1",
+  "version": "0.16.3",
   "description": "Tewip finds why each failing test failed - real bug, flaky test, environment, test data or changed selector - for any stack, from the command line, CI, or your editor.",
   "keywords": [
     "test-failures",