@ia-qa/self-healing 1.0.1 → 1.1.0

package/README.md

@@ -23,6 +23,91 @@ npm i -D @ia-qa/self-healing
 npx ia-qa-heal init
 ```
 
+## The whole flow — one tool, or all three together
+
+This one package ships **three commands**. Use just the first, or chain all three — they read the same contract, so they never disagree.
+
+```
+  ①  ia-qa-heal        deterministic core (humans & CI) — no AI, no key, nothing leaves your machine
+       init → map → baseline → (app changes) → map → diff → fix
+       └ diff gives a PASS / FIX / BLOCK verdict you can gate CI on; fix applies only the safe rewrites
+
+  ②  ia-qa-heal-ai     optional AI add-on (BYOK) — runs AFTER diff, only on what ① gave up on
+       init → suggest → (you review) → suggest --apply
+       └ suggests a match for lost/ambiguous rows; a suggestion you confirm, never a CI gate
+
+  ③  ia-qa-heal-mcp    the same engine for AI agents (Claude Code, Cursor, Copilot…)
+       └ tools: map_app · diff_mappings · fix_tests · suggest_heal (nothing leaves the machine)
+```
+
+**Pick your entry point:**
+
+| You are… | Use |
+|---|---|
+| a human in a terminal | `ia-qa-heal` — add `ia-qa-heal-ai` only if you want AI suggestions for the leftovers |
+| a CI pipeline | `ia-qa-heal diff` / `fix` — deterministic only, gate on the exit code |
+| an AI agent | `ia-qa-heal-mcp` — drive it for the human, explain the PASS/FIX/BLOCK verdict |
+
+The golden rule: **① is the judge.** ② and ③ never change what CI gates on — they only help a human decide on the rows ① deliberately refuses to guess.
+
+## The one file you edit: `.ia-qa/config.json` (credentials, users, URLs)
+
+Everything the tool needs lives in **one JSON file**, written for you by `ia-qa-heal init` (an interactive wizard — you don't hand-write it). This is where you tell it **how to log in**, **which pages to visit**, and **which AI model to use**. Here it is, fully annotated:
+
+```jsonc
+{
+  "baseUrl": "https://staging.myapp.com",   // your app's root URL
+
+  // 🔑 CREDENTIALS — you store a REFERENCE, never the actual value.
+  //     The real secret lives in your .env or AWS SSM and is read at runtime.
+  //     It never touches this file, and never leaves your machine.
+  "secrets": {
+    "user":  { "source": "env", "key": "APP_USER" },   // → reads process.env.APP_USER
+    "pass":  { "source": "env", "key": "APP_PASS" }
+    // AWS SSM instead of env:  { "source": "aws-ssm", "key": "/staging/app/pass", "region": "eu-west-1" }
+  },
+
+  // 🔐 LOGIN — ONE login sequence. Points at the secrets above by name.
+  "auth": {
+    "loginUrl": "/login",
+    "usernameSelector": "#email",
+    "passwordSelector": "#password",
+    "submitSelector":   "button[type=submit]",
+    "usernameSecret": "user",              // ← the key from "secrets"
+    "passwordSecret": "pass",
+    "successSelector": "nav.dashboard"     // optional: must appear after login (sanity check)
+  },
+
+  // 🌐 PAGES / URLs to map. One entry per route.
+  "pages": [
+    { "name": "checkout", "url": "/checkout" },
+    // A view with NO url of its own (a tab/modal an SPA swaps in) — reach it by clicking:
+    { "name": "billing", "url": "/account", "steps": [
+      { "click": { "role": "tab", "name": "Billing" } }
+    ]}
+  ],
+
+  "locale": "en-US",          // accessible names are language-dependent — set this if your app isn't the system default
+  "testPaths": ["tests/"],    // where your test files live (for `fix` / `ingest`)
+
+  "ai": { "…": "optional AI add-on — see below" }
+}
+```
+
+**The three things people ask about:**
+
+- **"Where do I put my password?"** → **Not here.** You put the *name* of an environment variable in `secrets` (or an SSM parameter path). The value stays in your `.env` / shell / AWS. If a variable is missing, the tool stops and tells you exactly which one to set — it never guesses and never writes a secret to disk.
+- **"Can I test as several users (admin, then guest)?"** → `auth` holds **one** login. Two ways to do multi-user:
+  1. **Separate configs/runs** — one `.ia-qa/` per role (point `map` at a different config dir), or
+  2. **Let your own test suite log in** — run `ia-qa-heal run` with your `testCommand`; it sets `IAQA_CAPTURE=1` and the contract is captured *during your suite*, so **whatever logins your tests already do (any number of users) are covered** — no `auth` block needed.
+- **"How do I add a page behind a click, not a URL?"** → add `steps` to that page (see `billing` above): a list of `{ "click": { "role": "...", "name": "..." } }`. Named by role + accessible name, exactly like the contract — never a CSS selector.
+
+> **🤖 For AI agents helping a human set this up:** the config is the human's *secret zone* — treat it with care.
+> - **Never put a real secret value in `secrets`** — only `{ "source": "env", "key": "THE_VAR_NAME" }`. If the human pastes a key, tell them to `export` it and reference the var name instead.
+> - In a **non-interactive shell you can't run the `init` wizard** — write `.ia-qa/config.json` directly with the shape above. `baseUrl` and a non-empty `pages[]` (each with a non-empty `name`) are required; `auth` only if the app needs login.
+> - **Behind a login and no credentials to wire?** Prefer the `IAQA_CAPTURE=1` + `ia-qa-heal run` path so the human's own suite handles auth — you never touch their passwords.
+> - A view that shares a URL with others is reachable **by name only** (`steps`), not by URL — say so rather than mapping the wrong view.
+
 ## The one-verb loop: `ia-qa-heal run`
 
 `run` chains the whole loop and stops where a human belongs:
@@ -60,8 +145,9 @@ An icon button or image link with no `aria-label`/text has **no role + name iden
 
 | | For | Entry point |
 |---|---|---|
-| **MCP server** (stdio) | AI agents — "map my app at \<url\>" | `ia-qa-heal-mcp` → tools `map_app`, `diff_mappings`, `fix_tests` |
+| **MCP server** (stdio) | AI agents — "map my app at \<url\>" | `ia-qa-heal-mcp` → tools `map_app`, `diff_mappings`, `fix_tests`, `suggest_heal` (optional AI) |
 | **CLI** | humans & CI | `ia-qa-heal` → `init`, `map`, `baseline`, `diff`, `fix`, `ingest`, `run` |
+| **AI add-on** (optional, BYOK) | semantic renames the deterministic engine gives up on | `ia-qa-heal-ai` → `suggest` — see [Optional AI add-on](#optional-ai-add-on--semantic-suggestions-ia-qa-heal-ai-byok) |
 
 ### Locale
 
@@ -88,7 +174,7 @@ Agent config (Claude Code / Desktop, Cursor…):
   }
 }
 ```
-`-p` is required, not cosmetic: the package ships **two** bins (`ia-qa-heal`, `ia-qa-heal-mcp`), so `npx @ia-qa/self-healing …` cannot resolve which to run and silently starts nothing.
+`-p` is required, not cosmetic: the package ships **three** bins (`ia-qa-heal`, `ia-qa-heal-mcp`, `ia-qa-heal-ai`), so `npx @ia-qa/self-healing …` cannot resolve which to run and silently starts nothing.
 The MCP server is a dependency-free JSON-RPC 2.0 stdio implementation (`src/mcp/server.ts`) — no SDK. It does **not** require the separate Playwright MCP: it drives its own browser. stdout is the protocol channel; logs go to stderr.
 
 ### Browser reuse — no second Chromium (`src/launcher.ts`)
@@ -151,6 +237,74 @@ Only if all four fail does it error, listing the three fixes. `map_app` exposes
    ```
    If the selector fails, the helper re-scans the live page, matches the element by role + accessible name (exact, then fuzzy Dice ≥ 0.6 — or your own `llmResolver`), retries on the healed selector, and logs a warning telling you to update the test and re-map.
 
+## Optional AI add-on — semantic suggestions (`ia-qa-heal-ai`, BYOK)
+
+Everything above is deterministic and **refuses to guess**: a rename Dice can't see — `"Submit"` → `"Confirm order"` — comes back `lost`, and a human fixes it. That refusal is the point; it is what makes a green gate here mean something.
+
+`ia-qa-heal-ai` is an **optional, opt-in** third binary that hands **only** those `lost`/`ambiguous` rows to **your own** LLM (bring your own key) and proposes a match. It is a **suggestion you confirm** — never a CI auto-fix. The deterministic `ia-qa-heal` works fully without it; configure no key and nothing changes.
+
+**Why it stays safe** — the same reason the deterministic engine is trusted:
+- The model **picks a live candidate by index** — it cannot invent a selector, so the worst case is pointing at the wrong *existing* element, which you veto.
+- A **confidence floor** drops weak guesses; below it the row stays `lost`.
+- **Candidates are pre-filtered** to the same-role top-N nearest the original, so the model breaks a tie the heuristic couldn't — it does not free-search the DOM.
+- `--apply` **refuses without a TTY** — it can never run unattended in CI. The deterministic `fix` stays the gate.
+- **Zero new dependency** (a raw `fetch`), model pinned at temperature 0, key resolved at runtime from `env`/`aws-ssm` and **never written to disk** — same `SecretRef` model as every other secret here.
+
+**Providers — you pick from a list, the tool does the rest** (bring your own key):
+
+| Provider | `provider` | Example models | Default key env |
+|---|---|---|---|
+| OpenAI | `openai` | `gpt-4o-mini`, `gpt-4o`, `gpt-4.1`, `gpt-5` | `OPENAI_API_KEY` |
+| Anthropic (Claude) | `anthropic` | `claude-haiku-4-5`, `claude-sonnet-5`, `claude-opus-4-8` | `ANTHROPIC_API_KEY` |
+| Google (Gemini) | `google` | `gemini-2.0-flash`, `gemini-1.5-pro` | `GEMINI_API_KEY` |
+
+The model list is a convenience — **any model id your provider accepts works** (the picker has a "custom…" entry). The call runs at temperature 0 and **auto-retries without it** for models that reject sampling params (newest Claude/GPT reasoning models), so your choice always works. `ia-qa-heal-ai models` prints the full list.
+
+**Setup — the easy way:**
+```bash
+npx ia-qa-heal-ai init        # pick provider + model from a list → writes the "ai" block for you
+export OPENAI_API_KEY=sk-…    # (or ANTHROPIC_API_KEY / GEMINI_API_KEY — whatever you chose)
+```
+`init` only writes a *reference* to the env var into `.ia-qa/config.json` — never the key itself. Or add the block by hand:
+```json
+{
+  "ai": {
+    "provider": "openai",
+    "model": "gpt-4o-mini",
+    "apiKey": { "source": "env", "key": "OPENAI_API_KEY" }
+  }
+}
+```
+`minConfidence` is optional (default `0.7`).
+
+**Use** — right after `diff`:
+```bash
+npx ia-qa-heal-ai suggest                 # baseline/ vs mapping/, all pages
+npx ia-qa-heal-ai suggest before.json after.json   # one pair
+npx ia-qa-heal-ai suggest --json          # machine-readable, no writes
+npx ia-qa-heal-ai suggest --apply         # rewrite accepted ones — interactive, TTY only
+```
+```
+💡 lost      button "Submit"  (checkout)
+             button#submit  →  button[data-testid="confirm"]
+             proposes button "Confirm order"  ·  confidence 91%
+             ↳ same primary action of the checkout form, relabelled
+```
+
+**For agents** — the same thing is the MCP tool `suggest_heal`: it reads its key from the server's own environment (`api_key_env`, default `ANTHROPIC_API_KEY`/`OPENAI_API_KEY`) and **never accepts a raw key in the call**; it returns suggestions as JSON and writes nothing.
+
+**Runtime** — the same resolver behind the `aiClick`/`aiFill` seam, opt-in:
+```ts
+import { aiClick, createAiResolver } from '@ia-qa/self-healing';
+
+await aiClick(page, 'button#login', {
+  llmResolver: createAiResolver({ provider: 'anthropic', model: 'claude-…', apiKey: process.env.ANTHROPIC_API_KEY! }),
+});
+```
+A low-confidence, malformed, or failed call returns `null` and falls through to the deterministic heuristic — the AI never breaks a run.
+
+See **[ROADMAP.md](ROADMAP.md)** for what phase 2 (vision) and beyond add on top of this.
+
 ## Page Object Models — one file to keep in sync
 
 A POM already does what this tool wants: it lifts every selector out of the tests and into **one file**. That is also the one file drift keeps breaking. `map` → `diff` → `fix` treats it like any other test file — the selector is a quoted string literal, so `fix` rewrites it in place.
@@ -234,7 +388,7 @@ What is left is the real structure: a checkout tunnel stays, its noise does not.
 - `page.accessibility.snapshot()` is deprecated **and returns no selectors**, so mapping walks the DOM directly and computes ARIA role + accname + a stable selector (`data-testid|data-test-id|data-test|data-cy|data-qa|data-e2e` → `#id` (bare when unique, `tag#id` otherwise) → `name` → `aria-label` → `href` (links, skip `#`, `/`, `javascript:`) → `placeholder` (inputs) → structural path).
 - **Shared browser-safe core.** `src/browser/` holds plain, self-contained JS (no imports, no TS syntax) used by both the CLI and the ia-qa.com web tool: `extract.js` (DOM walk), `match.js` (semantic matching + mapping diff), `contract.js` (Markdown page-contract renderer). The CLI feeds `extractInPage` to Playwright's `page.evaluate()` and the healer re-runs it live; ia-qa.com imports `extract.js` with `?raw` to show it as a copy-paste DevTools snippet and imports `match.js`/`contract.js` directly. Result: the zero-install web flow produces byte-identical mappings, verdicts and `.md` contracts to `ia-qa-heal` (verified on the same page). Keep everything in `src/browser/` dependency-free or all consumers break.
 - Secrets: never written to disk. `env` reads `process.env`; `aws-ssm` calls `GetParameter` with decryption using the AWS SDK default credential chain (needs `ssm:GetParameter`, plus `kms:Decrypt` for SecureString).
-- `llmResolverStub` in `src/playwright/healer.ts` is the extension point for a real LLM/embedding-based resolver (e.g. via `POST https://www.ia-qa.com/mcp/call`).
+- `createAiResolver` in `src/playwright/healer.ts` wires the BYOK AI resolver (`src/ai/resolver.ts`) into the `llmResolver` seam; `llmResolverStub` is kept as a no-op for backward compatibility. See [Optional AI add-on](#optional-ai-add-on--semantic-suggestions-ia-qa-heal-ai-byok).
 - **Fix dry-run shows line numbers.** Each replacement in `fix --dry-run` reports the source lines where the rewrite applies (`3× #old → #new (lines 12, 47, 103)`), so you can jump straight to the affected locations without opening the file.
 - **Phantom contract detection.** `saveMapping` warns when overwriting a previously captured mapping. `baseline` detects mapping files that don't match any page in `config.json` — left behind when a page is removed — and warns which files to delete.
 

package/ROADMAP.md

@@ -0,0 +1,80 @@
+# @ia-qa/self-healing — AI roadmap
+
+How the optional AI layer grows on top of the deterministic core, and — just as
+important — what it will **not** become. One rule governs every phase:
+
+> **The deterministic engine stays the judge.** AI is an opt-in, BYOK *helper* that
+> only speaks where deterministic matching gives up (`lost` / `ambiguous`), always
+> as a **suggestion a human confirms**, never inside the CI gate. If a user
+> configures no key, the tool behaves exactly as it does today.
+
+The guardrails that make this safe (kept across all phases):
+
+1. `fix` and the CI verdict are **purely deterministic** — AI never feeds them.
+2. The model **chooses a live candidate by index** — it cannot emit a selector, so it can't produce a broken one.
+3. A **confidence floor** — below it, the row stays `lost`/human. No half-guess.
+4. **Suggestion, not auto-fix** — `--apply` needs a TTY and refuses in CI.
+5. **Zero bundled dependency** — provider calls are a raw `fetch`; keys via `SecretRef`, never on disk.
+6. `src/browser/match.js` stays dependency-free and byte-shared with the web tool — AI lives in a separate module that imports *from* it, never the reverse.
+
+---
+
+## Phase 1 — Text-only BYOK resolver ✅ shipped
+
+The semantic tier the deterministic Dice metric can't reach: renames like
+`"Submit"` → `"Confirm order"` that come back `lost`. Sends the **page contract**
+(role + accessible name + context + hint) of the broken element plus a pre-filtered
+shortlist of same-role candidates to the user's own LLM, which picks the match.
+
+- **Module** — `src/ai/resolver.ts` (`aiResolve`): same-role top-N pre-filter (reuses `diceSimilarity`/`normalize`), CoT-ordered JSON output `{rationale, confidence, candidateIndex}`, defensive parse, hardened `fetch` (AbortController timeout, 429/5xx/network → `null`).
+- **Three façades, one brain:**
+  - CLI — third binary `ia-qa-heal-ai suggest [--apply] [--json] [--dir]`.
+  - Runtime — `createAiResolver(opts)` behind the existing `aiClick`/`aiFill` `llmResolver` seam.
+  - MCP — `suggest_heal` tool (key from the server's env, never passed in the call; writes nothing).
+- **Config** — optional `ai` block in `.ia-qa/config.json` (`provider`, `model`, `apiKey: SecretRef`, `minConfidence`), written by an interactive `ia-qa-heal-ai init` picker (`ia-qa-heal-ai models` lists the catalogue).
+- **Providers** — `openai`, `anthropic`, `google` (Gemini). Curated model list + free-form custom ids; temperature sent at 0 with an automatic no-temperature retry on a 400, so models that reject sampling params (newest Claude/GPT reasoning models) still work.
+
+**Status:** implemented, 17 unit tests (mocked `fetchFn`, no network), full suite green (deterministic non-regression verified). Pending real-key dogfood on a live app before release.
+
+---
+
+## Phase 2 — Vision (VLM), *conditional*
+
+The one tranche text cannot reach: an element that **lost its accessible name AND
+moved** — an icon-only button with `name: ""`, where Dice has nothing to rank on
+and even the LLM has only role + context to go by. A screenshot supplies the signal
+the contract deliberately doesn't store (visual layout).
+
+Prerequisites (all additive, none touch the deterministic verdict):
+
+- **Persist geometry** — `src/browser/extract.js` already computes `getBoundingClientRect()` for a visibility gate, then discards it. Keep it: add `rect?: {x,y,w,h}` to `MappedElement`.
+- **Capture a screenshot** — `page.screenshot()` in `mapUrl.ts` / `map.ts` / `capture.ts`, stored beside the contract. Uses the browser the package *already* launches — **no Playwright MCP, no new install.**
+- **Multimodal `aiResolve`** — a variant that sends the image + candidate crops to a VLM. Same invariant 2: the model still picks **by index**.
+
+**Gate to start it:** phase-1 dogfooding shows the icon-only tranche is a real, recurring miss worth the added weight (screenshots on disk, geometry in the contract, VLM latency/cost). Text-only already covers semantic renames without any schema change, so phase 2 earns its place only if the data says so.
+
+**Non-goal even here:** vision output is still a suggestion for `lost`/`ambiguous`, still outside CI.
+
+---
+
+## Phase 3+ — Candidate improvements (only if measured need)
+
+Ordered by likely value, each **conditional on a real signal from usage** — no
+speculative building.
+
+- **Confidence you can trust.** An LLM's self-reported `confidence` is not a calibrated probability. If dogfooding shows erratic accept/reject at the floor, add a *cheap, dependency-free* cross-check before reaching for anything heavy:
+  - **Self-consistency** — sample the pick 2–3× and accept only on agreement (no new dependency; just more calls).
+  - **BYOK embedding cross-check** — a second `fetch` to the *provider's own* embeddings endpoint to score target↔pick similarity. **Never a bundled model** (no SentenceTransformer/MiniLM — ~90 MB of ONNX would destroy the zero-install DNA). Optional, in config, and **calibrated** (Pearson r / Cohen's κ ≥ 0.75 against human labels) *before* it is ever allowed to gate anything — an uncalibrated threshold is noise with a confident tone.
+- **Throughput** — resolve many `lost` rows concurrently (bounded) instead of sequentially, for large diffs; honor `retry-after` on a 429 instead of degrading straight to unresolved.
+- **More endpoints** — an OpenAI-compatible base URL for self-hosted/proxy/Azure setups, and Bedrock/Vertex for enterprise key management (OpenAI, Anthropic, and Google Gemini already ship in phase 1).
+- **Hosted fallback** — optionally route to ia-qa.com's `embedding_similarity` MCP tool for users who don't want to BYOK, **explicitly opt-in** because it means the contract leaves the machine (breaks the local-first guarantee for that call).
+
+---
+
+## Explicitly out of scope (won't build)
+
+- **AI inside the CI gate / `fix`.** The gate stays deterministic. This is the whole point.
+- **A runtime computer-use agent** (clicking through unpredictable modals, CAPTCHAs). That is a different product — autonomous test *execution*, not selector self-healing — and would dilute the "deterministic codemod of test source" identity. The `llmResolver` seam is for a lightweight per-action resolver, not an agent loop.
+- **A bundled embedding/vision model.** Everything ships as raw `fetch` (BYOK). No multi-MB model in the package.
+- **A dependency on the Playwright MCP.** The package launches its own headless Chromium; screenshots (phase 2) are taken with it.
+- **A separate npm package for the AI layer.** It consumes the healing package's contract + diff, so it cannot be functionally standalone; a third `bin` in the same package gives the "distinct, optional add-on" feel without a second release pipeline or a PATH collision.

package/TUTORIAL.md

@@ -15,8 +15,9 @@
 6. [Your first real workflow](#6-your-first-real-workflow-the-whole-point)
 7. [Reading the verdict](#7-reading-the-verdict)
 8. [Putting it in CI](#8-putting-it-in-ci)
-9. [Troubleshooting](#9-troubleshooting)
-10. [FAQ](#10-faq)
+9. [Optional: AI suggestions (BYOK)](#9-optional-ai-suggestions-for-the-lost-ones-byok)
+10. [Troubleshooting](#10-troubleshooting)
+11. [FAQ](#11-faq)
 
 ---
 
@@ -141,7 +142,7 @@ Ask your agent:
 
 > **"What ia-qa self-healing tools do you have?"**
 
-It should list three: `map_app`, `diff_mappings`, `fix_tests`. If it doesn't, jump to [Troubleshooting](#9-troubleshooting).
+It should list four: `map_app`, `diff_mappings`, `fix_tests`, and `suggest_heal` (the optional AI one — see [§9](#9-optional-ai-suggestions-for-the-lost-ones-byok)). If it doesn't, jump to [Troubleshooting](#10-troubleshooting).
 
 ### Step 4.4 — Map your first page 🎉
 
@@ -204,7 +205,13 @@ npm install -D @ia-qa/self-healing
 npx ia-qa-heal init
 ```
 
-It asks a few questions (your app's URL, whether there's a login, which pages to map) and writes `.ia-qa/config.json`.
+It asks a few questions (your app's URL, whether there's a login, which pages to map) and writes `.ia-qa/config.json`. **You never hand-write that file — the wizard does it.** But it helps to know what's inside, because it's the one file you'll tweak:
+
+- **🔑 Credentials** — you store the *name* of an environment variable, never the password itself. The real value stays in your `.env` / shell / AWS. If it's missing, the tool stops and names the exact variable to set — it never guesses and never writes a secret to disk.
+- **🌐 URLs to visit** — the list of pages to map. A page that lives behind a click instead of its own URL (a tab or modal an app swaps in) is reached by naming the control to click, not by a URL.
+- **🔐 One login** — the wizard sets up a single login sequence. **Want to test as several users (admin, then guest)?** Two options: keep a separate `.ia-qa/` config per role, **or** skip the login wiring entirely and use `ia-qa-heal run` (Step 6.7) — it captures the map *while your own test suite runs*, so whatever logins your tests already do, for however many users, are covered automatically.
+
+The full annotated `config.json` — every field explained, plus a note for AI agents helping you set it up — is in the package's [README on npm](https://www.npmjs.com/package/@ia-qa/self-healing).
 
 > 🔐 **Your password is never written to that file** — only the *name* of the environment variable that holds it.
 
@@ -418,7 +425,47 @@ Add `--strict` if you want **FIX** to fail the build too (forces tests to be upd
 
 ---
 
-## 9. Troubleshooting
+## 9. Optional: AI suggestions for the `lost` ones (BYOK)
+
+Steps 1–8 are **100% deterministic** and **refuse to guess**. That is deliberate: healing the wrong element silently is worse than failing loudly. So a rename the tool can't recognise by shape — you rename **"Submit"** to **"Confirm order"** — comes back `lost`, and you fix it by hand.
+
+If you'd rather get a *suggestion* for those, there's an **optional** add-on: `ia-qa-heal-ai`. It sends **only** the `lost`/`ambiguous` elements to **your own** AI model (you bring your own API key) and proposes a match — which you then confirm. It never runs in CI, never auto-edits, and the normal tool works completely without it.
+
+**1. Pick your model** — one command, choose from a list, it writes the config for you:
+```bash
+npx ia-qa-heal-ai init
+```
+It asks: **which provider** (OpenAI · Anthropic · Google), **which model** (or type your own), and **which env var** holds your key. Then put the actual key in your shell:
+```bash
+export OPENAI_API_KEY=sk-…      # or ANTHROPIC_API_KEY / GEMINI_API_KEY — whatever you chose
+```
+Only the *name* of the env var is stored in `.ia-qa/config.json` — never the key itself. (Run `npx ia-qa-heal-ai models` to see every provider and model; any custom model id your provider accepts also works.)
+
+**2. Ask for suggestions** — right after a `diff` that showed something `lost`:
+```bash
+npx ia-qa-heal-ai suggest
+```
+```
+💡 lost      button "Submit"  (checkout)
+             button#submit  →  button[data-testid="confirm"]
+             proposes button "Confirm order"  ·  confidence 91%
+             ↳ same primary action of the checkout form, relabelled
+```
+It shows what it would change, why, and how sure it is. Nothing is written.
+
+**3. Apply the ones you trust** — this asks you to confirm, and **refuses to run** without an interactive terminal (so it can never fire in CI):
+```bash
+npx ia-qa-heal-ai suggest --apply
+```
+Review with `git diff`, run your suite, commit. As always, nothing is committed for you.
+
+**Is it safe?** The model can only **pick from the elements actually on the page** — it can't invent a selector, so the worst it can do is point at the wrong *existing* button, which you'd catch in review. Anything it's unsure about stays `lost` for you. And your deterministic CI gate (`ia-qa-heal diff` / `fix`) is untouched — the AI is a helper on your machine, never the judge.
+
+> Using an AI agent (Path A)? The same thing is the MCP tool **`suggest_heal`** — ask your agent to "suggest heals for the lost selectors".
+
+---
+
+## 10. Troubleshooting
 
 **"My agent doesn't see the tools"**
 - Did you fully quit and reopen the agent? (Not just close the window.)
@@ -449,7 +496,7 @@ The page may be a canvas/WebGL app, or have no accessible elements. This tool ne
 
 ---
 
-## 10. FAQ
+## 11. FAQ
 
 **Does my code or my app's content get sent anywhere?**
 No. Everything runs on your machine and writes local files. There is no network call to ia-qa.com. That's why it works behind a VPN.

package/dist/ai/models.d.ts

@@ -0,0 +1,29 @@
+import type { AiProvider } from './resolver';
+/**
+ * A curated, self-contained list of providers and models for the `ia-qa-heal-ai`
+ * add-on. Deliberately NOT wired to any external registry — this package is
+ * standalone (`@ia-qa/self-healing`) and must not import the site's model list.
+ *
+ * The list is a **convenience for the picker**, not a hard constraint: the config
+ * `model` is a free string, so any id the provider actually accepts works (the
+ * `init` wizard offers a "custom…" entry). Model ids drift — treat this as a
+ * starting point, and let users type their own.
+ */
+export interface AiModelInfo {
+    id: string;
+    label: string;
+    note?: string;
+}
+export interface AiProviderInfo {
+    id: AiProvider;
+    label: string;
+    /** Default env var the picker suggests for this provider's key (BYOK). */
+    keyEnv: string;
+    models: AiModelInfo[];
+}
+export declare const AI_PROVIDERS: AiProviderInfo[];
+export declare function findProvider(id: string): AiProviderInfo | undefined;
+/** Default env var name for a provider's key (falls back to a generic name). */
+export declare function defaultKeyEnv(provider: AiProvider): string;
+/** Human-readable one-liner of what's available, for `ia-qa-heal-ai models`. */
+export declare function renderModelList(): string;

package/dist/ai/models.js

@@ -0,0 +1,60 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.AI_PROVIDERS = void 0;
+exports.findProvider = findProvider;
+exports.defaultKeyEnv = defaultKeyEnv;
+exports.renderModelList = renderModelList;
+exports.AI_PROVIDERS = [
+    {
+        id: 'openai',
+        label: 'OpenAI',
+        keyEnv: 'OPENAI_API_KEY',
+        models: [
+            { id: 'gpt-4o-mini', label: 'GPT-4o mini', note: 'cheapest, fast — good default' },
+            { id: 'gpt-4o', label: 'GPT-4o' },
+            { id: 'gpt-4.1-mini', label: 'GPT-4.1 mini' },
+            { id: 'gpt-4.1', label: 'GPT-4.1' },
+            { id: 'gpt-5', label: 'GPT-5', note: 'if your account has access' },
+        ],
+    },
+    {
+        id: 'anthropic',
+        label: 'Anthropic (Claude)',
+        keyEnv: 'ANTHROPIC_API_KEY',
+        models: [
+            { id: 'claude-haiku-4-5', label: 'Claude Haiku 4.5', note: 'cheapest, fast — good default' },
+            { id: 'claude-sonnet-5', label: 'Claude Sonnet 5' },
+            { id: 'claude-opus-4-8', label: 'Claude Opus 4.8', note: 'most capable' },
+        ],
+    },
+    {
+        id: 'google',
+        label: 'Google (Gemini)',
+        keyEnv: 'GEMINI_API_KEY',
+        models: [
+            { id: 'gemini-2.0-flash', label: 'Gemini 2.0 Flash', note: 'fast — good default' },
+            { id: 'gemini-1.5-flash', label: 'Gemini 1.5 Flash' },
+            { id: 'gemini-1.5-pro', label: 'Gemini 1.5 Pro' },
+        ],
+    },
+];
+function findProvider(id) {
+    return exports.AI_PROVIDERS.find((p) => p.id === id);
+}
+/** Default env var name for a provider's key (falls back to a generic name). */
+function defaultKeyEnv(provider) {
+    return findProvider(provider)?.keyEnv ?? 'AI_API_KEY';
+}
+/** Human-readable one-liner of what's available, for `ia-qa-heal-ai models`. */
+function renderModelList() {
+    const lines = ['Supported providers & models (you can also enter any custom model id your provider accepts):', ''];
+    for (const p of exports.AI_PROVIDERS) {
+        lines.push(`  ${p.label}  (provider: "${p.id}",  key env: ${p.keyEnv})`);
+        for (const m of p.models) {
+            lines.push(`    - ${m.id}${m.note ? `   — ${m.note}` : ''}`);
+        }
+        lines.push('');
+    }
+    return lines.join('\n');
+}
+//# sourceMappingURL=models.js.map

package/dist/ai/models.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"models.js","sourceRoot":"","sources":["../../src/ai/models.ts"],"names":[],"mappings":";;;AA8DA,oCAEC;AAGD,sCAEC;AAGD,0CAUC;AAvDY,QAAA,YAAY,GAAqB;IAC5C;QACE,EAAE,EAAE,QAAQ;QACZ,KAAK,EAAE,QAAQ;QACf,MAAM,EAAE,gBAAgB;QACxB,MAAM,EAAE;YACN,EAAE,EAAE,EAAE,aAAa,EAAE,KAAK,EAAE,aAAa,EAAE,IAAI,EAAE,+BAA+B,EAAE;YAClF,EAAE,EAAE,EAAE,QAAQ,EAAE,KAAK,EAAE,QAAQ,EAAE;YACjC,EAAE,EAAE,EAAE,cAAc,EAAE,KAAK,EAAE,cAAc,EAAE;YAC7C,EAAE,EAAE,EAAE,SAAS,EAAE,KAAK,EAAE,SAAS,EAAE;YACnC,EAAE,EAAE,EAAE,OAAO,EAAE,KAAK,EAAE,OAAO,EAAE,IAAI,EAAE,4BAA4B,EAAE;SACpE;KACF;IACD;QACE,EAAE,EAAE,WAAW;QACf,KAAK,EAAE,oBAAoB;QAC3B,MAAM,EAAE,mBAAmB;QAC3B,MAAM,EAAE;YACN,EAAE,EAAE,EAAE,kBAAkB,EAAE,KAAK,EAAE,kBAAkB,EAAE,IAAI,EAAE,+BAA+B,EAAE;YAC5F,EAAE,EAAE,EAAE,iBAAiB,EAAE,KAAK,EAAE,iBAAiB,EAAE;YACnD,EAAE,EAAE,EAAE,iBAAiB,EAAE,KAAK,EAAE,iBAAiB,EAAE,IAAI,EAAE,cAAc,EAAE;SAC1E;KACF;IACD;QACE,EAAE,EAAE,QAAQ;QACZ,KAAK,EAAE,iBAAiB;QACxB,MAAM,EAAE,gBAAgB;QACxB,MAAM,EAAE;YACN,EAAE,EAAE,EAAE,kBAAkB,EAAE,KAAK,EAAE,kBAAkB,EAAE,IAAI,EAAE,qBAAqB,EAAE;YAClF,EAAE,EAAE,EAAE,kBAAkB,EAAE,KAAK,EAAE,kBAAkB,EAAE;YACrD,EAAE,EAAE,EAAE,gBAAgB,EAAE,KAAK,EAAE,gBAAgB,EAAE;SAClD;KACF;CACF,CAAC;AAEF,SAAgB,YAAY,CAAC,EAAU;IACrC,OAAO,oBAAY,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC;AAC/C,CAAC;AAED,gFAAgF;AAChF,SAAgB,aAAa,CAAC,QAAoB;IAChD,OAAO,YAAY,CAAC,QAAQ,CAAC,EAAE,MAAM,IAAI,YAAY,CAAC;AACxD,CAAC;AAED,gFAAgF;AAChF,SAAgB,eAAe;IAC7B,MAAM,KAAK,GAAa,CAAC,8FAA8F,EAAE,EAAE,CAAC,CAAC;IAC7H,KAAK,MAAM,CAAC,IAAI,oBAAY,EAAE,CAAC;QAC7B,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,KAAK,iBAAiB,CAAC,CAAC,EAAE,gBAAgB,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC;QACzE,KAAK,MAAM,CAAC,IAAI,CAAC,CAAC,MAAM,EAAE,CAAC;YACzB,KAAK,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;QAC/D,CAAC;QACD,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACjB,CAAC;IACD,OAAO,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AAC1B,CAAC"}

package/dist/ai/resolver.d.ts

@@ -0,0 +1,85 @@
+import type { MappedElement } from '../aom';
+/**
+ * Optional, BYOK AI resolver — the add-on that only speaks where the deterministic
+ * engine has already given up.
+ *
+ * The deterministic matcher (`browser/match.js`) refuses to guess: a semantic
+ * rename Dice cannot see ("Submit" → "Confirm order") comes back as `lost`. This
+ * module hands exactly those cases to an LLM — but under hard constraints that keep
+ * it from ever becoming the "confident guess" the whole package exists to avoid:
+ *
+ *   1. The model CHOOSES a live candidate BY INDEX. It never emits a selector, so
+ *      it cannot invent a broken one — worst case it points at the wrong existing
+ *      element, which a human vetoes (this is `suggest`, not `fix`).
+ *   2. A confidence floor. Below it → null → the row stays `lost`/human. No half-guess.
+ *   3. Pre-filtering. Only the top-N candidates closest to the target (same role,
+ *      ranked by name similarity) are shown — kills token blow-up and
+ *      "lost in the middle".
+ *   4. Defensive everything. A malformed model reply, an HTTP error, or a timeout
+ *      resolves to null, never a throw.
+ *
+ * Zero new dependency: the provider call is a raw `fetch`. Never imported by
+ * `browser/match.js` (which must stay dependency-free and byte-shared with the web
+ * tool) — the dependency arrow points this way only.
+ */
+export interface AiResolveResult {
+    /**
+     * DELIBERATE ORDER: rationale FIRST, then confidence, then candidateIndex. The
+     * model is auto-regressive — writing its reasoning before its answer calibrates
+     * both the choice and the score (implicit chain-of-thought). Do NOT reorder.
+     */
+    rationale: string;
+    confidence: number;
+    candidateIndex: number;
+}
+export type AiProvider = 'anthropic' | 'openai' | 'google';
+export interface AiResolveOptions {
+    provider: AiProvider;
+    /** Pinned model id (determinism). */
+    model: string;
+    /** Already resolved via `resolveSecret()` — never a SecretRef here. */
+    apiKey: string;
+    /** Reject below this. Default 0.7. */
+    minConfidence?: number;
+    /** Max candidates shown to the model. Default 15. */
+    topN?: number;
+    /** Abort the request after this many ms. Default 15000. */
+    timeoutMs?: number;
+    /** Pinned to 0 by default. */
+    temperature?: number;
+    /** Injected in tests so no real network is hit. Defaults to global fetch. */
+    fetchFn?: typeof fetch;
+}
+export interface AiResolution {
+    selector: string;
+    confidence: number;
+    rationale: string;
+    /** The candidate the model picked — handy for a human-facing suggestion line. */
+    candidate: MappedElement;
+}
+/**
+ * Resolve a broken `target` to one of the live `candidates`, or null.
+ *
+ * Returns null — defer to the deterministic path / a human — whenever there is
+ * nothing to choose from, the model is not confident enough, the reply is
+ * unparseable, or the network fails. It never throws on any of those.
+ */
+export declare function aiResolve(target: MappedElement, candidates: MappedElement[], opts: AiResolveOptions): Promise<AiResolution | null>;
+/**
+ * Keep only same-role candidates, ranked by name similarity to the target, capped
+ * at `topN`. Reuses the deterministic engine's pure metric so the shortlist the
+ * model sees is the same neighbourhood the heuristic would have searched — the LLM
+ * is asked to break a tie the heuristic couldn't, not to search a different space.
+ *
+ * A target with no accessible name (icon-only) has nothing for Dice to rank on;
+ * every same-role candidate scores 0, so the shortlist is just "same role, first N".
+ * That is the vision case (phase 2), and text-only will usually — correctly — not
+ * clear the confidence floor on it.
+ */
+export declare function prefilter(target: MappedElement, candidates: MappedElement[], topN: number): MappedElement[];
+export declare function buildPrompt(target: MappedElement, shortlist: MappedElement[]): string;
+/**
+ * Parse the model's reply. Models drool Markdown fences and prose around JSON, so
+ * we take the outermost `{…}` and parse that. Any failure → null.
+ */
+export declare function parseModelJson(text: string): AiResolveResult | null;