tewip-ai 0.4.0 → 0.6.0

package/QUICKSTART.md

@@ -27,7 +27,8 @@ If your config already uses a reporter list in CI, add `['blob']` to it.
 ```yaml
 permissions:
   pull-requests: write     # the diagnosis comment
-  issues: write            # the grouped environment issue
+  issues: write
+  actions: read          # so it can read a failed job's log when no report was written            # the grouped environment issue
   contents: write          # only if you turn autofix on
 
 steps:

package/README.md

@@ -6,10 +6,10 @@ It works for every stack: Node.js (node:test, Jest, Vitest, Mocha), Python (pyte
 
 | Where | How |
 |---|---|
-| **Terminal** | `npm install -g tewip-<version>.tgz` (from the release; Tewip is not on the npm registry yet), then `tewip explain report.xml`, `tewip ci owner/repo --pr 12`, `tewip mcp` |
+| **Terminal** | `npm install -g tewip-ai`, then `tewip explain report.xml`, `tewip ci owner/repo --pr 12`, `tewip mcp` |
 | **VS Code, Cursor, Windsurf, VSCodium** | the Tewip extension: registers Tewip with the editor's AI assistant and adds *Explain a test report* and *Why did CI fail on this branch?* |
 | **JetBrains, Claude, other AI assistants** | add the MCP server `tewip mcp` (see [editor setup](#use-it-from-your-editor-any-ide-that-speaks-mcp)) |
-| **Any model** | OpenAI, Claude, Gemini, Azure, or a model on your own machine with Ollama (no account, nothing leaves the computer): `TEWIP_PROVIDER` |
+| **Any model** | OpenAI, Claude, Gemini, Azure, or a model on your own machine with Ollama (no account, nothing leaves the computer): `TEWIP_PROVIDER` |
 | **GitHub CI** | the Tewip Action: a comment on every pull request, an optional merge gate, verified fix PRs ([QUICKSTART](QUICKSTART.md)) |
 
 Formerly *triage-agent*: the repository moved to `hmamut39/tewip`, and GitHub redirects the old address. See [ROADMAP.md](ROADMAP.md) for the plan.
@@ -18,8 +18,8 @@ Formerly *triage-agent*: the repository moved to `hmamut39/tewip`, and GitHub re
 
 ## Use it in your project
 
-**Giving it to your team (or another company): [TEAM-GUIDE.md](TEAM-GUIDE.md)** — what to install, what each role uses it for, and what it never does.
-
+**Giving it to your team (or another company): [TEAM-GUIDE.md](TEAM-GUIDE.md)** — what to install, what each role uses it for, and what it never does.
+
 **Putting it on a real repository: [QUICKSTART.md](QUICKSTART.md)** — ten minutes, copy-paste, and what to check on the first failing run.
 
 ### 1. In GitHub CI (developers, SDETs, DevOps)
@@ -39,6 +39,7 @@ permissions:
   contents: write        # only needed for autofix
   pull-requests: write
   issues: write
+  actions: read          # lets it read the failed job's log when no test report was written
 
 steps:
   # ... checkout, install, npx playwright test ...
@@ -56,6 +57,7 @@ On every run, the action:
 - **DevOps:** collects environment failures (server errors, connection resets, outages) in **one** GitHub issue, and each new run adds a single grouped comment to it.
 - **Fixes (`autofix: true`):** for drifted locators that the deterministic rule matched to a renamed element, it rewrites the locator, re-runs that test, and opens a separate pull request with the changes that passed. It never merges and never edits application code.
 - **Every role:** uploads all role views as the `tewip-report` artifact.
+- **When the job dies before any test runs** (an image that would not pull, a runner out of disk, a service container that never came up): it reads the failed job's log and names the cause, instead of leaving a red X with no explanation.
 - **History:** keeps the run history in the Actions cache, never in git.
 - **Corrections:** reply to the agent's comment with `/triage CATEGORY #n reason` (e.g. `/triage TIMING #2 flaky on CI`). It stores that with the history and, next time that test fails the same way, gives the person's answer instead of its own, naming them and their reason. A corrected failure also costs nothing: it never goes to the model again.
 
@@ -89,7 +91,7 @@ The repository needs "Allow GitHub Actions to create and approve pull requests"
 | "What should I fix first from the last run?" | `triage_board` (SDET) · `pr_impact` (developer) |
 | "Is the test environment broken?" | `environment_report` (DevOps / SRE) |
 | "Can we release?" | `release_readiness` (business / UAT) |
-| "Where is our test suite fragile?" | `quality_report` (senior / principal) · `architecture_report` (architect) · `risk_digest` (lead) |
+| "Where is our test suite fragile?" | `quality_report` (senior / principal) · `architecture_report` (architect) · `risk_digest` (lead) |
 | "Write a test plan for the checkout feature in C:\work\shop" | `test_plan`: risks and scenarios grounded in that project's tests, code and pages; saved in its `.tewip` folder |
 
 Setup, after `npm install`:
@@ -143,6 +145,84 @@ tewip write --only CO-002,CO-004 --framework pytest
 - **`tewip plan`** reads what is really there: the tests you already have (in any language), the change (`--diff main...HEAD`), the live pages (what a user sees on each page, via Playwright), and the failures Tewip has diagnosed in this project before. The plan lists risks and scenarios by priority. A claim that a scenario is "already covered" is checked against your real test names, and dropped if no such test exists. Saved to `.tewip/plans/` as Markdown to review and JSON for `write`.
 - **`tewip write`** writes each scenario as a test in **your** framework and style: Playwright, Cypress, WebdriverIO, Angular, Jest, Vitest, pytest (with Selenium or Playwright), JUnit on Maven or Gradle (Selenium, Spring Boot), .NET or Go. It copies the style of an existing test and uses only elements seen on the real page. Front-end apps get component tests the way their own tests are written: React, Vue or Svelte with Testing Library under Jest or Vitest, and Angular through `ng test` (Karma and Jasmine, or Vitest). A project with both e2e and unit tests gets browser tests when the plan read pages, and unit or component tests when it did not. Then it runs the test. A draft is **kept only if** it passes every run (3 by default) **and** fails once its main check is broken on purpose, because a test that cannot fail is worse than no test. A failing draft is repaired from the real error, twice at most. If the test looks right and the app does something else, Tewip reports a **possible bug** instead of weakening the test until it passes. Kept tests are new files beside yours, for you to review and commit; nothing is committed, and no application code is touched.
 
+## Check the page against the design (Phase 11, new)
+
+```bash
+tewip design "https://www.figma.com/design/KEY/Checkout?node-id=12-34" --url http://localhost:3000/checkout
+tewip design frame.json --frame "Sign in"     # a saved frame, no Figma token needed
+```
+
+Tewip reads the frame (Figma's REST API, read only, `TEWIP_FIGMA_TOKEN`) and the running page
+(its accessibility snapshot, which is what a screen reader and a test both see), and says where they
+disagree: `the design says button "Sign in", the page says button "Log in"`. It checks the words the
+design specifies, the controls its layer names imply, and the regions its sections imply - never
+pixels, colours or spacing, because a design tool and a browser do not measure those the same way.
+No model is called, so it costs nothing and answers the same way every time.
+
+Measured on four real design/page pairs (a public repository's Figma exports and the pages built
+from them): 38 checks, **0 differences reported on pages that match their design**, and **8 of 8
+injected drifts caught** - a renamed button, a dropped sentence, a region that stopped being marked
+up. No designer has judged the output yet, so the roadmap's own metric is still open.
+
+## Epics, stories and acceptance criteria (Phase 14, new)
+
+```bash
+tewip spec "Readers can search the library and open a result"
+tewip spec "..." --project PROJ            # shows what it would file in Jira
+tewip spec "..." --project PROJ --create   # files the epic and its stories
+```
+
+An epic, user stories and Given / When / Then acceptance criteria, written from your code, your
+tests and your failure history. Anything it says the system **already** does carries a quote from a
+file it read, and Tewip checks that quote word by word; what it cannot quote becomes an open
+question instead of a confident sentence. It writes no estimates, dates or business cases, because
+it cannot check them.
+
+Measured on three real projects: 15 stories, 49 acceptance criteria, and of 46 claims about existing
+behaviour, **22 were kept with a checkable quote and 24 were turned into open questions**.
+
+## Any CI, your chat, and your own machine (Phase 13, new)
+
+```bash
+tewip report report.xml --gate      # GitLab CI, Jenkins, CircleCI, Azure, Bitbucket, Travis, Drone
+tewip watch --cmd "npm test"        # beside your editor: explains each failure as it happens
+tewip note --days 7                 # the weekly quality note for a lead - free, no model calls
+```
+
+`tewip report` reads which CI it is in from that platform's own variables, prints the triage board,
+writes it to a file, appends it to the job summary, and with `--gate` fails the build **only** on
+failures that are not known flaky or environment noise. On GitLab it posts the diagnosis as one
+merge-request note and edits that same note next run instead of piling up. With
+`TEWIP_SLACK_WEBHOOK` or `TEWIP_TEAMS_WEBHOOK` it tells the team's chat - and says nothing at all
+when no failure needs a person, so nobody mutes it.
+
+## Fix the code, but only when the tests prove it (Phase 12, new, opt-in)
+
+```bash
+tewip fix --code report.xml          # shows the patch; your files are left as they were
+tewip fix --code report.xml --apply  # leaves the proved change in your working tree
+```
+
+Tewip has refused to touch application code since day one. This is the one exception, and it only
+runs when you type `--code`. It takes a failure **it** diagnosed as a real bug (0.85 confidence or
+more), finds the file the failure points at, asks for the smallest change, and then proves it: the
+failing test must pass **and** the whole suite must still pass. It will not edit tests,
+configuration, CI files, lockfiles, migrations or anything with secrets in it, because a test going
+green after changing those proves nothing. If it cannot prove a change, it explains the bug instead
+and puts your file back byte for byte. It never commits and never merges.
+
+Measured by replaying **16 real bug fixes** from two public repositories, in two languages: the
+commit before each fix, with only that fix's tests, so the real bug really reproduces.
+
+| Project | Language | Bugs | Proved by the tests | Right file |
+|---|---|---|---|---|
+| unjs/ufo | TypeScript, Vitest | 8 | **8** | 8 of 8 |
+| python-humanize/humanize | Python, pytest | 8 | **8** | 8 of 8 |
+
+Every change is small (1 to 16 lines, median 4) and most are what the developers themselves wrote.
+**No wrong fix was offered**: anything that could not be proved was thrown away and the file put
+back byte for byte.
+
 ## Where it has been tried
 
 | Project | How | Result |