@ia-qa/self-healing 1.5.1 → 1.5.2

package/README.md

@@ -448,7 +448,7 @@ await aiClick(page, 'button#login', {
 ```
 A low-confidence, malformed, or failed call returns `null` and falls through to the deterministic heuristic — the AI never breaks a run.
 
-See **[ROADMAP.md](ROADMAP.md)** for what phase 2 (vision) and beyond add on top of this.
+See **[ROADMAP.md](ROADMAP.md)** for what comes next — and for why the vision/VLM phase was measured and dropped: no image ever leaves your machine, because none is ever sent anywhere.
 
 ## Page Object Models — one file to keep in sync
 

package/ROADMAP.md

@@ -16,6 +16,7 @@ The guardrails that make this safe (kept across all phases):
 4. **Suggestion, not auto-fix** — `--apply` needs a TTY and refuses in CI.
 5. **Zero bundled dependency** — provider calls are a raw `fetch`; keys via `SecretRef`, never on disk.
 6. `src/browser/match.js` stays dependency-free and byte-shared with the web tool — AI lives in a separate module that imports *from* it, never the reverse.
+7. **Framework parity** — every phase ships for all three adapters (Playwright, Cypress, Selenium) or degrades *identically* for all three; nothing lands Playwright-only. The AI layer earns this for free (it works on contracts, downstream of any browser); the one capability that would touch the framework layer is screenshot capture, now handled outside this roadmap in [SPEC-visual-evidence.md](./SPEC-visual-evidence.md).
 
 ---
 
@@ -34,26 +35,36 @@ shortlist of same-role candidates to the user's own LLM, which picks the match.
 - **Config** — optional `ai` block in `.ia-qa/config.json` (`provider`, `model`, `apiKey: SecretRef`, `minConfidence`), written by an interactive `ia-qa-heal-ai init` picker (`ia-qa-heal-ai models` lists the catalogue).
 - **Providers** — `openai`, `anthropic`, `google` (Gemini). Curated model list + free-form custom ids; temperature sent at 0 with an automatic no-temperature retry on a 400, so models that reject sampling params (newest Claude/GPT reasoning models) still work.
 
-**Status:** released — on npm since `1.1.0` (2026-07-19), current `1.5.0`. 17 unit tests (mocked `fetchFn`, no network), full suite green (deterministic non-regression verified). Real-key dogfood of `suggest` has **not** happened yet — a first field attempt (2026-07-23, external project, OpenAI key in env) never reached the resolver: the tester ran `ia-qa-heal --ai`, which silently no-opped, because the main CLI's help never mentioned `ia-qa-heal-ai`. Both findings fixed since (pending release): the main help now points to the AI add-on, and unknown flags error loudly — `--ai` specifically answers with the `ia-qa-heal-ai` pointer (`src/cli/args.ts`, pinned by `tests/cliArgs.test.js`). Next: redo the dogfood with `ia-qa-heal-ai suggest`.
+**Status:** released — on npm since `1.1.0` (2026-07-19), current `1.5.1`. 17 unit tests (mocked `fetchFn`, no network), full suite green (deterministic non-regression verified). Real-key dogfood of `suggest`: **done** (2026-07-23, external Playwright repo, OpenAI). First attempt never reached the resolver — the tester ran `ia-qa-heal --ai`, a silent no-op, because the main help never mentioned `ia-qa-heal-ai`; fixed in `1.5.1` (help points to the add-on; unknown flags error loudly, `--ai` answers with the `ia-qa-heal-ai` pointer — `src/cli/args.ts`, pinned by `tests/cliArgs.test.js`). Second attempt exercised the full binary against a live key: a cross-language semantic rename (`"Envoyer le formulaire"` → `"Submit form"`, different selectors) came back with a correct suggestion (`confidence` 0.76), and the no-drift case returned empty arrays; both are now pinned as key-gated e2e tests in the dogfood repo. Caveat: the renames were synthetic (hand-edited baseline). Still to collect from *organic* drift: whether `confidence` behaves at the floor (the phase-3 trigger). The other open question — how often the icon-only/no-name tranche shows up — was answered by measurement on 2026-07-25 and closed phase 2 (below).
 
 ---
 
-## Phase 2 — Vision (VLM), *conditional*
+## Phase 2 — Vision (VLM) — ❌ **dropped, 2026-07-25**
 
-The one tranche text cannot reach: an element that **lost its accessible name AND
-moved** — an icon-only button with `name: ""`, where Dice has nothing to rank on
-and even the LLM has only role + context to go by. A screenshot supplies the signal
-the contract deliberately doesn't store (visual layout).
+This phase targeted the one tranche text cannot reach: an element that **lost its
+accessible name AND moved** — an icon-only button with `name: ""` — to be resolved
+by sending a screenshot to a VLM. Its gate was *"dogfooding shows the icon-only
+tranche is a real, recurring miss"*.
 
-Prerequisites (all additive, none touch the deterministic verdict):
+**The gate was measured, and it fails.** 4 706 real elements (40 pages of a
+dogfood corpus + 12 third-party sites mapped live):
 
-- **Persist geometry** — `src/browser/extract.js` already computes `getBoundingClientRect()` for a visibility gate, then discards it. Keep it: add `rect?: {x,y,w,h}` to `MappedElement`.
-- **Capture a screenshot** — `page.screenshot()` in `mapUrl.ts` / `map.ts` / `capture.ts`, stored beside the contract. Uses the browser the package *already* launches — **no Playwright MCP, no new install.**
-- **Multimodal `aiResolve`** — a variant that sends the image + candidate crops to a VLM. Same invariant 2: the model still picks **by index**.
+- The blind tranche — no accessible name, no `hint`, **and** a positional selector, the only one a VLM could reach — is **0 elements in the wild**. Every anonymous element found there carries an identity selector (`a[href=…]`, `[data-testid=…]`), so it cannot drift positionally.
+- The 25 blind rows in the dogfood corpus are **unlabelled range sliders and colour pickers — not a single icon-only button**.
+- **100 % of them (52/52) are recoverable by a plain DOM read** (`title` on a descendant, or the nearest ancestor's own text) — including the canonical case, Excalidraw's `data-testid="main-menu-trigger"` hamburger, whose neighbouring text already says *"Exportation, préférences, langues…"*. The screenshot would be a lossy, paid re-derivation of a string the extractor already had in hand.
 
-**Gate to start it:** phase-1 dogfooding shows the icon-only tranche is a real, recurring miss worth the added weight (screenshots on disk, geometry in the contract, VLM latency/cost). Text-only already covers semantic renames without any schema change, so phase 2 earns its place only if the data says so.
+And a reason that does not depend on any measurement: **a crop of a customer's
+authenticated app is not a contract.** The AI add-on sends *structure* today —
+role, name, selector, context. An image sends whatever was on screen: customer
+names, amounts, records. That is a different consent, not an extra option.
 
-**Non-goal even here:** vision output is still a suggestion for `lost`/`ambiguous`, still outside CI.
+**What survives, and where it went:** the useful half of this phase — persisting
+`rect`, capturing a screenshot, cropping it — is real, but it serves a *human
+triage* need the roadmap never identified: the `ambiguous` population (role + name
++ context collisions) is **10,2 % of elements on the dogfood corpus and 20,3 % in
+the wild** — 6 to 20× the anonymous one. It is specified, fully local, with no
+model call at all, in **[SPEC-visual-evidence.md](./SPEC-visual-evidence.md)**
+(not shipped in the npm tarball). Measurements are reproducible from `studies/`.
 
 ---
 
@@ -76,5 +87,5 @@ speculative building.
 - **AI inside the CI gate / `fix`.** The gate stays deterministic. This is the whole point.
 - **A runtime computer-use agent** (clicking through unpredictable modals, CAPTCHAs). That is a different product — autonomous test *execution*, not selector self-healing — and would dilute the "deterministic codemod of test source" identity. The `llmResolver` seam is for a lightweight per-action resolver, not an agent loop.
 - **A bundled embedding/vision model.** Everything ships as raw `fetch` (BYOK). No multi-MB model in the package.
-- **A dependency on the Playwright MCP.** The package launches its own headless Chromium; screenshots (phase 2) are taken with it.
+- **A dependency on the Playwright MCP.** The package launches its own headless Chromium; any screenshot it ever takes is taken with that.
 - **A separate npm package for the AI layer.** It consumes the healing package's contract + diff, so it cannot be functionally standalone; a third `bin` in the same package gives the "distinct, optional add-on" feel without a second release pipeline or a PATH collision.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ia-qa/self-healing",
-  "version": "1.5.1",
+  "version": "1.5.2",
   "description": "Local-first self-healing for UI tests: a local MCP server + CLI that map your app's pages to a role/name/selector contract, diff selector drift (PASS/FIX/BLOCK), and apply deterministic fixes to Cypress/Playwright/Selenium tests. Runs on your machine — nothing leaves it.",
   "keywords": [
     "self-healing",