@ia-qa/self-healing 1.7.3 → 1.7.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +256 -835
- package/SPEC-selector-resolution.md +310 -0
- package/package.json +3 -2
package/README.md
CHANGED
|
@@ -1,498 +1,142 @@
|
|
|
1
1
|
# @ia-qa/self-healing
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Your E2E tests break because a selector moved, not because the app is wrong. This finds the
|
|
4
|
+
element again and rewrites the test.
|
|
4
5
|
|
|
5
|
-
|
|
6
|
+
A **CLI**, a **local MCP server** for AI agents, and an accessibility-tree page mapper.
|
|
7
|
+
Framework-agnostic — Playwright, Cypress, Selenium; JS/TS, Python, Java. The verdict is
|
|
8
|
+
deterministic: no model decides whether your build passes.
|
|
6
9
|
|
|
7
|
-
**It runs entirely on
|
|
10
|
+
**It runs entirely on your machine.** It brings its own headless Chromium, so it sees
|
|
11
|
+
`localhost`, staging, or an app behind a VPN, and **nothing ever leaves the machine**.
|
|
12
|
+
|
|
13
|
+
> **New here?** The [step-by-step tutorial](https://www.ia-qa.com/devtools/selector-drift/tutorial)
|
|
14
|
+
> walks the whole thing with no jargon (same text as `TUTORIAL.md`, shipped in this package).
|
|
15
|
+
> This README is the short reference.
|
|
8
16
|
|
|
9
17
|
## See it work
|
|
10
18
|
|
|
11
19
|
[](https://www.ia-qa.com/media/heal-demo.mp4)
|
|
12
20
|
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
*Real commands against a live site, real stdout — only the typing speed is synthesised.*
|
|
18
|
-
|
|
19
|
-
## Easiest start — have your agent set it up
|
|
20
|
-
|
|
21
|
-
If you already work with Claude Code, Cursor, Copilot or another coding agent, don't configure this
|
|
22
|
-
by hand. Paste this to it:
|
|
23
|
-
|
|
24
|
-
```text
|
|
25
|
-
Install and set up @ia-qa/self-healing in this project.
|
|
26
|
-
1. First run `npx -y -p @ia-qa/self-healing ia-qa-heal skill --print` and follow it.
|
|
27
|
-
That output is the authoritative doc — do not guess any command or flag from memory.
|
|
28
|
-
2. Register the MCP server so you can drive it yourself:
|
|
29
|
-
command `npx`, args ["-y", "-p", "@ia-qa/self-healing", "ia-qa-heal-mcp"].
|
|
30
|
-
3. Read my e2e tests and my router, propose the page list for .ia-qa/config.json,
|
|
31
|
-
and wait for me to confirm before mapping anything.
|
|
32
|
-
4. Then explain to me, in five lines: what map / baseline / diff / fix each do,
|
|
33
|
-
and which locators you will never rewrite without asking me.
|
|
34
|
-
```
|
|
35
|
-
|
|
36
|
-
Step 1 matters more than it looks: this package is newer than most models' training data, so an agent
|
|
37
|
-
left to improvise will invent flags that don't exist. `skill --print` hands it the real instructions —
|
|
38
|
-
including [the four prohibitions](#working-with-an-ai-agent--ia-qa-heal-skill) — before it touches
|
|
39
|
-
anything. Step 4 is how you check it actually read them instead of guessing.
|
|
40
|
-
|
|
41
|
-
## Quick start — by hand
|
|
21
|
+
`ia-qa-heal run` runs your suite, watches it break, diffs the live app against your baseline,
|
|
22
|
+
rewrites the dead selectors as `getByRole(…, { exact: true })` — not as fresher CSS paths —
|
|
23
|
+
then **re-runs the suite to verify** and exits on that verdict. Nothing is committed.
|
|
42
24
|
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
npx -y -p @ia-qa/self-healing ia-qa-heal baseline # → .ia-qa/baseline/ — the reference. Commit it.
|
|
47
|
-
# …change the app, then `map` again…
|
|
48
|
-
npx -y -p @ia-qa/self-healing ia-qa-heal diff # baseline/ vs mapping/ → PASS / FIX / BLOCK
|
|
49
|
-
```
|
|
25
|
+
▶ **[The full 49-second version](https://www.ia-qa.com/media/heal-demo.mp4)** also covers what
|
|
26
|
+
deterministic healing *refuses* to guess, the MCP server answering an agent, and the HTML
|
|
27
|
+
report `--report` writes for a PR.
|
|
50
28
|
|
|
51
|
-
Install
|
|
29
|
+
## Install and run the loop
|
|
52
30
|
|
|
53
31
|
```bash
|
|
54
32
|
npm i -D @ia-qa/self-healing
|
|
55
|
-
npx ia-qa-heal init
|
|
56
|
-
```
|
|
57
|
-
|
|
58
|
-
## Working with an AI agent — `ia-qa-heal skill`
|
|
59
|
-
|
|
60
|
-
An agent that asks a browser where a button went is doing by hand what `.ia-qa/mapping/*.json` already answers offline: every interactive element's **role, accessible name and selector**, captured. A `link` the suite still calls a `button` shows up in one `grep`.
|
|
61
|
-
|
|
62
|
-
`ia-qa-heal skill` installs that reflex — plus the rules that must not be broken — as a skill your agent loads (this is what step 1 of the [paste-block above](#easiest-start--have-your-agent-set-it-up) makes it read):
|
|
63
|
-
|
|
64
|
-
```bash
|
|
65
|
-
ia-qa-heal skill # where it would go, and whether it is current — writes nothing
|
|
66
|
-
ia-qa-heal skill --install # → ./.claude/skills/ia-qa-heal/SKILL.md
|
|
67
|
-
ia-qa-heal skill --install --user # → ~/.claude/skills/, for every project on the machine
|
|
68
|
-
ia-qa-heal skill --print # to stdout, for a different agent's format
|
|
69
|
-
```
|
|
70
|
-
|
|
71
|
-
It is short on purpose (which verb answers which question; the flags stay in `--help`) and its centre of gravity is the **prohibitions** — never hand-rewrite a locator reported `unattributable`, never gate on `audit`, never read a green `run` as a green app, never diff a `map` baseline against a `run` capture. Those are the mistakes only an agent makes, and it makes them while trying to help.
|
|
72
|
-
|
|
73
|
-
Installing is a separate step because agents read skills from `.claude/skills/`, never from `node_modules` — shipped alone, the file would never be loaded. The command is non-interactive, idempotent, prints the absolute path before writing, and refuses to overwrite a copy you have edited unless you pass `--force`.
|
|
74
|
-
|
|
75
|
-
## The whole flow — one tool, or all three together
|
|
76
|
-
|
|
77
|
-
This one package ships **three commands**. Use just the first, or chain all three — they read the same contract, so they never disagree.
|
|
78
|
-
|
|
79
|
-
```
|
|
80
|
-
① ia-qa-heal deterministic core (humans & CI) — no AI, no key, nothing leaves your machine
|
|
81
|
-
init → (discover) → ingest → map → baseline → (app changes) → map → diff → fix
|
|
82
|
-
└ diff gives a PASS / FIX / BLOCK verdict you can gate CI on; fix applies only the safe rewrites
|
|
83
|
-
|
|
84
|
-
② ia-qa-heal-ai optional AI add-on (BYOK) — runs AFTER diff, only on what ① gave up on
|
|
85
|
-
init → suggest → (you review) → suggest --apply
|
|
86
|
-
└ suggests a match for lost/ambiguous rows; a suggestion you confirm, never a CI gate
|
|
87
|
-
|
|
88
|
-
③ ia-qa-heal-mcp the same engine for AI agents (Claude Code, Cursor, Copilot…)
|
|
89
|
-
└ tools: map_app · diff_mappings · fix_tests · suggest_heal (nothing leaves the machine)
|
|
90
|
-
```
|
|
91
|
-
|
|
92
|
-
**Pick your entry point:**
|
|
93
|
-
|
|
94
|
-
| You are… | Use |
|
|
95
|
-
|---|---|
|
|
96
|
-
| a human in a terminal | `ia-qa-heal` — add `ia-qa-heal-ai` only if you want AI suggestions for the leftovers |
|
|
97
|
-
| a CI pipeline | `ia-qa-heal diff` / `fix` — deterministic only, gate on the exit code |
|
|
98
|
-
| an AI agent | `ia-qa-heal-mcp` — drive it for the human, explain the PASS/FIX/BLOCK verdict |
|
|
99
|
-
|
|
100
|
-
The golden rule: **① is the judge.** ② and ③ never change what CI gates on — they only help a human decide on the rows ① deliberately refuses to guess.
|
|
101
|
-
|
|
102
|
-
## The one file you edit: `.ia-qa/config.json` (credentials, users, URLs)
|
|
103
|
-
|
|
104
|
-
Everything the tool needs lives in **one JSON file**, written for you by `ia-qa-heal init` (an interactive wizard — you don't hand-write it). This is where you tell it **how to log in**, **which pages to visit**, and **which AI model to use**. Here it is, fully annotated:
|
|
105
|
-
|
|
106
|
-
```jsonc
|
|
107
|
-
{
|
|
108
|
-
"baseUrl": "https://staging.myapp.com", // your app's root URL
|
|
109
|
-
|
|
110
|
-
// 🔑 CREDENTIALS — you store a REFERENCE, never the actual value.
|
|
111
|
-
// The real secret lives in your .env or AWS SSM and is read at runtime.
|
|
112
|
-
// It never touches this file, and never leaves your machine.
|
|
113
|
-
"secrets": {
|
|
114
|
-
"user": { "source": "env", "key": "APP_USER" }, // → reads process.env.APP_USER
|
|
115
|
-
"pass": { "source": "env", "key": "APP_PASS" }
|
|
116
|
-
// AWS SSM instead of env: { "source": "aws-ssm", "key": "/staging/app/pass", "region": "eu-west-1" }
|
|
117
|
-
},
|
|
118
|
-
|
|
119
|
-
// 🔐 LOGIN — ONE login sequence. Points at the secrets above by name.
|
|
120
|
-
"auth": {
|
|
121
|
-
"loginUrl": "/login",
|
|
122
|
-
"usernameSelector": "#email",
|
|
123
|
-
"passwordSelector": "#password",
|
|
124
|
-
"submitSelector": "button[type=submit]",
|
|
125
|
-
"usernameSecret": "user", // ← the key from "secrets"
|
|
126
|
-
"passwordSecret": "pass",
|
|
127
|
-
"successSelector": "nav.dashboard" // optional: must appear after login (sanity check)
|
|
128
|
-
},
|
|
129
|
-
|
|
130
|
-
// 🌐 PAGES / URLs to map. One entry per route.
|
|
131
|
-
"pages": [
|
|
132
|
-
{ "name": "checkout", "url": "/checkout" },
|
|
133
|
-
// A view with NO url of its own (a tab/modal an SPA swaps in) — reach it by clicking:
|
|
134
|
-
{ "name": "billing", "url": "/account", "steps": [
|
|
135
|
-
{ "click": { "role": "tab", "name": "Billing" } }
|
|
136
|
-
]}
|
|
137
|
-
],
|
|
138
|
-
|
|
139
|
-
"locale": "en-US", // accessible names are language-dependent — set this if your app isn't the system default
|
|
140
|
-
"testPaths": ["tests/"], // where your test files live (for `fix` / `ingest`)
|
|
141
|
-
|
|
142
|
-
// 🌀 VOLATILE — content that rotates by design (feed links, article cards, live promos).
|
|
143
|
-
// It reads as interactive UI, so without this it enters the contract like a button
|
|
144
|
-
// does — then two captures of the same green page disagree about it and a handful
|
|
145
|
-
// of vanished feed items escalates into a false BLOCK. Matching elements never
|
|
146
|
-
// enter a contract (map and run capture both filter them).
|
|
147
|
-
// Glob per pattern (`*` = anything, case-insensitive, whole-string), optional field
|
|
148
|
-
// prefix href:/selector:/name:/role: — without one, tried against href, selector and name.
|
|
149
|
-
"volatile": ["href:*source=rss*", "href:https://blog.example.com/*"],
|
|
150
|
-
|
|
151
|
-
// 🎭 NAMEMASK — labels that carry a counter, a total or a clock ("Cart (3)",
|
|
152
|
-
// "🔌 Smithery 3.2k calls"). The *label* rotates, the element does not — so
|
|
153
|
-
// unlike `volatile`, the element stays fully under contract (role, selector and
|
|
154
|
-
// href still gate, a selector break on it still BLOCKs) and only the name stops
|
|
155
|
-
// being compared. Applied at comparison time: the contracts on disk keep the raw
|
|
156
|
-
// label, so adding or removing a mask never needs a re-map.
|
|
157
|
-
// Same glob dialect as `volatile`, no field prefix. A pattern with no stable text
|
|
158
|
-
// ("*") is refused at load. At runtime a mask is also dropped, loudly, if it would
|
|
159
|
-
// swallow a label your tests locate by name, or leave two elements of one role
|
|
160
|
-
// sharing a name.
|
|
161
|
-
"nameMask": ["Cart (*)", "🔌 Smithery * calls"],
|
|
162
|
-
|
|
163
|
-
"ai": { "…": "optional AI add-on — see below" }
|
|
164
|
-
}
|
|
165
|
-
```
|
|
166
|
-
|
|
167
|
-
**The three things people ask about:**
|
|
168
|
-
|
|
169
|
-
- **"Where do I put my password?"** → **Not here.** You put the *name* of an environment variable in `secrets` (or an SSM parameter path). The value stays in your `.env` / shell / AWS. **Loading it is your job:** the CLI reads `process.env` and never opens a `.env` file, so a key that lives only in `.env` is invisible to it — run it under `npx dotenv-cli --` (or Node's `--env-file-if-exists`, 20.12+) and it works everywhere, CI included. If a variable is missing, the tool stops and tells you exactly which one to set — it never guesses and never writes a secret to disk.
|
|
170
|
-
- **"Can I test as several users (admin, then guest)?"** → `auth` holds **one** login. Two ways to do multi-user:
|
|
171
|
-
1. **Separate configs/runs** — one `.ia-qa/` per role, selected with `--config <dir>` (see **Monorepos** below), or
|
|
172
|
-
2. **Let your own test suite log in** — run `ia-qa-heal run` with your `testCommand`; it sets `IAQA_CAPTURE=1` and the contract is captured *during your suite*, so **whatever logins your tests already do (any number of users) are covered** — no `auth` block needed.
|
|
173
|
-
- **"My diff BLOCKs on links that just… rotate (a news feed, a blog widget)?"** → That is content churn, not UI drift: the feed's links entered the contract as if they were buttons, and the next capture caught the feed mid-rotation. Name what rotates in `"volatile"` (see above) and it never enters a contract again — the diff even prints a hint pointing here when a BLOCK is made of lost external links.
|
|
174
|
-
- **"The same rows come back as `renamed` every run — a cart counter, a call total, a clock."** → Only the *label* is moving, so only the label should stop being compared: put the pattern in `"nameMask"` (`"Cart (*)"`), not in `"volatile"`. `volatile` would delete the element from the contract — counter, button, href and coverage together — so a real selector break on it would go unnoticed forever; a mask keeps all of that gating and exempts the name alone. The diff prints the exact config line to paste when it spots the pattern. Two masks it refuses to apply, and says so: one that would hide a label your tests locate by name (that is a red suite behind a green gate), and one that would leave two elements of the same role wearing one name (a locator coin flip the mask itself created).
|
|
175
|
-
- **"How do I add a page behind a click, not a URL?"** → add `steps` to that page (see `billing` above): a list of `{ "click": { "role": "...", "name": "..." } }`. Named by role + accessible name, exactly like the contract — never a CSS selector.
|
|
176
|
-
- **"My app spans several domains behind a single sign-on (federated SSO) — does `map` cover it?"** → **Not with one `auth` block.** `auth` runs **one** login against **one** domain, before the first page. A federated IdP that re-challenges each project/domain with its own scope (`?projectName=…`, a different sub-domain per portal) is never satisfied by that single login — the second domain bounces back to a login the first one never covered, and the page maps empty (map now warns when an authenticated target returns 0 elements). Two ways through: **one `.ia-qa/` per domain/portal**, each selected with `--config <dir>` (see **Monorepos** below), or **`ia-qa-heal run` with `IAQA_CAPTURE=1`** so your own suite — which already handles the full SSO dance (cookies, redirects, per-project scope) — does the auth and the contract is captured during that real run. And since 1.7.0 there is a third, which is the general one: **`ia-qa-heal login`** — you log in once in a visible browser, whatever the app asks, and `map` reuses that session (see below). For a multi-portal app, capture-during-run is the intended path in CI; `map --auth` is for single-domain apps.
|
|
177
|
-
|
|
178
|
-
### Behind any login — `ia-qa-heal login`
|
|
179
|
-
|
|
180
|
-
`auth` fills a form: a username field, a password field, a submit button. That is *one shape of
|
|
181
|
-
login*, and plenty of apps do not have it — federated SSO, MFA, a consent screen, a magic link, a
|
|
182
|
-
device check. No list of selectors gets through any of those, so for those apps "add an `auth`
|
|
183
|
-
block" is not an answer at all.
|
|
184
|
-
|
|
185
|
-
```bash
|
|
186
|
-
ia-qa-heal login # opens a visible browser at your first configured page
|
|
187
|
-
# you log in there, however your app asks
|
|
188
|
-
ia-qa-heal map # reuses that session — no auth block needed
|
|
33
|
+
npx ia-qa-heal init # interactive: writes .ia-qa/config.json
|
|
189
34
|
```
|
|
190
35
|
|
|
191
|
-
|
|
192
|
-
whatever the app requires, and the browser session is saved to `.ia-qa/session.json` in
|
|
193
|
-
Playwright's `storageState` shape. `map` and `discover --crawl` pick it up automatically and say
|
|
194
|
-
so; it takes precedence over `auth` when both exist (you ran `login` *because* the form fill
|
|
195
|
-
could not get through — re-running it silently would be the tool arguing with you).
|
|
196
|
-
|
|
197
|
-
- **That file is a secret.** It holds live cookies: whoever has it is logged in as you. It never
|
|
198
|
-
leaves your machine — nothing here uploads it — and `login` writes `.ia-qa/.gitignore` so a
|
|
199
|
-
commit cannot carry it.
|
|
200
|
-
- **It needs a person.** It refuses without a TTY and under CI, and an AI agent driving the CLI
|
|
201
|
-
cannot perform it: the agent hands the step to you by name.
|
|
202
|
-
- **It expires.** When the session dies, `map` hits the login wall again and says to re-run
|
|
203
|
-
`login` — it never silently maps a login form under the name of your dashboard.
|
|
204
|
-
- **It is not a CI mechanism.** In a pipeline, use `auth` (reproducible, secrets from env/SSM) or
|
|
205
|
-
capture-during-run (`IAQA_CAPTURE=1` + `testCommand`, your own suite logs itself in).
|
|
206
|
-
|
|
207
|
-
`--url <path>` opens a page other than the first one in `config.pages`.
|
|
208
|
-
|
|
209
|
-
#### You already have a session — use it
|
|
210
|
-
|
|
211
|
-
An authenticated Playwright suite usually writes a `storageState` in `globalSetup`. Point at
|
|
212
|
-
it and skip logging in a second time:
|
|
36
|
+
Then, **in this order**:
|
|
213
37
|
|
|
214
38
|
```bash
|
|
215
|
-
ia-qa-heal
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
39
|
+
npx ia-qa-heal ingest # inventory the selectors and names your tests use
|
|
40
|
+
npx ia-qa-heal map # capture the app, and bind your selectors to it
|
|
41
|
+
npx ia-qa-heal baseline # promote that capture to "this is how it should be"
|
|
42
|
+
# … someone changes the UI …
|
|
43
|
+
npx ia-qa-heal map
|
|
44
|
+
npx ia-qa-heal diff # PASS / FIX / BLOCK
|
|
45
|
+
npx ia-qa-heal fix --dry-run
|
|
46
|
+
npx ia-qa-heal fix
|
|
222
47
|
```
|
|
223
48
|
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
**A session you named and that cannot be read stops the run** — it never falls through to
|
|
229
|
-
"no session". Silently ignoring it would capture the login page under your pages' names, which
|
|
230
|
-
is the one failure this tool treats as worse than stopping. Same for a file that is not a
|
|
231
|
-
storageState: it says so, instead of failing later inside the browser.
|
|
49
|
+
`ingest` comes **first**, and it is not optional if your tests locate by CSS: it is what lets
|
|
50
|
+
`map` bind the strings your tests write to real elements. Re-run it when your test files
|
|
51
|
+
change — `map` says so when they have. Commit `.ia-qa/` (except `session.json`): the baseline
|
|
52
|
+
is what CI compares against.
|
|
232
53
|
|
|
54
|
+
Three binaries ship together: `ia-qa-heal` (the loop), `ia-qa-heal-mcp` (stdio MCP server for
|
|
55
|
+
agents), `ia-qa-heal-ai` (optional BYOK suggestions — never a gate).
|
|
233
56
|
|
|
234
|
-
|
|
57
|
+
## Behind a login
|
|
235
58
|
|
|
236
|
-
|
|
59
|
+
Most apps worth mapping are behind one, and only some logins are a form.
|
|
237
60
|
|
|
238
61
|
```bash
|
|
239
|
-
ia-qa-heal
|
|
240
|
-
ia-qa-heal
|
|
241
|
-
ia-qa-heal map --config apps/xcp # a different portal, its own contract
|
|
242
|
-
```
|
|
243
|
-
|
|
244
|
-
`--config <dir>` moves **only where the contract lives** (config, mapping, baseline, layouts, captures). Output paths you pass — `--report out.html`, explicit `diff before.json after.json` args — still resolve against your real shell directory. No flag ⇒ the current directory, exactly as before.
|
|
245
|
-
|
|
246
|
-
**For the capture path, use the env var instead:** `--config` is a CLI flag and cannot reach the capture, which runs inside your own test process. Set `IAQA_CONFIG_DIR=apps/xsp` there — the Playwright capture reads it directly; for Cypress, pass it as `--env IAQA_CONFIG_DIR=apps/xsp` (or a `CYPRESS_IAQA_CONFIG_DIR` env var) so the hook writes shards under that portal. Then merge with the matching `ia-qa-heal diff --config apps/xsp`.
|
|
247
|
-
|
|
248
|
-
> Deliberately **not** "profiles inside one config.json": mapping/, baseline/ and _layouts/ are per-directory on disk, and one directory per portal is what keeps a contract stage-able, review-able and revert-able as a single unit in git.
|
|
249
|
-
|
|
250
|
-
> **🤖 For AI agents helping a human set this up:** the config is the human's *secret zone* — treat it with care.
|
|
251
|
-
> - **Never put a real secret value in `secrets`** — only `{ "source": "env", "key": "THE_VAR_NAME" }`. If the human pastes a key, tell them to `export` it and reference the var name instead.
|
|
252
|
-
> - In a **non-interactive shell you can't run the `init` wizard** — write `.ia-qa/config.json` directly with the shape above. `baseUrl` and a non-empty `pages[]` (each with a non-empty `name`) are required; `auth` only if the app needs login.
|
|
253
|
-
> - **Behind a login and no credentials to wire?** Prefer the `IAQA_CAPTURE=1` + `ia-qa-heal run` path so the human's own suite handles auth — you never touch their passwords.
|
|
254
|
-
> - A view that shares a URL with others is reachable **by name only** (`steps`), not by URL — say so rather than mapping the wrong view.
|
|
255
|
-
|
|
256
|
-
## Discover pages to map — `ia-qa-heal discover`
|
|
257
|
-
|
|
258
|
-
`map` only ever sees the pages in `config.pages` — and so does drift detection. The question `discover` answers is the one nothing else does: **which pages of your app are you *not* covering?** It reads your whole surface (the sitemap, or a safe crawl), compares it to `config.pages`, and leads with the gap:
|
|
259
|
-
|
|
260
|
-
```
|
|
261
|
-
🗺 Coverage — 2 of 143 discovered pages in config.pages · 141 not yet covered:
|
|
262
|
-
all-tools /all-tools (sitemap)
|
|
263
|
-
resources /resources (sitemap)
|
|
264
|
-
…
|
|
62
|
+
npx ia-qa-heal login # opens a real browser; you log in however the app asks
|
|
63
|
+
npx ia-qa-heal map # reuses that session
|
|
265
64
|
```
|
|
266
65
|
|
|
267
|
-
It
|
|
268
|
-
|
|
269
|
-
|
|
66
|
+
It models **nothing** about your login, which is why SSO, MFA, a consent screen and magic
|
|
67
|
+
links all work — you perform them. What is saved is the browser session
|
|
68
|
+
(`.ia-qa/session.json`): live cookies, so treat it like a password. It stays on your machine,
|
|
69
|
+
`login` writes `.ia-qa/.gitignore` so a commit cannot carry it, and when it expires `map` says
|
|
70
|
+
so instead of quietly mapping the login page under your dashboard's name.
|
|
270
71
|
|
|
271
|
-
|
|
72
|
+
**Already have a session?** An authenticated Playwright suite usually writes a `storageState`
|
|
73
|
+
in `globalSetup` — point at it instead of logging in twice:
|
|
272
74
|
|
|
273
75
|
```bash
|
|
274
|
-
npx ia-qa-heal
|
|
275
|
-
npx ia-qa-heal discover --sitemap https://…/custom.xml # an explicit sitemap
|
|
276
|
-
npx ia-qa-heal discover --crawl # follow links from inside the logged-in app
|
|
277
|
-
npx ia-qa-heal discover --crawl --apply # …and write the new pages to config.json
|
|
76
|
+
npx ia-qa-heal map --session playwright/.auth/user.json
|
|
278
77
|
```
|
|
279
78
|
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
- **`--crawl` (higher reach, still ultra-safe).** Starts *inside* the app — reusing the very `auth` login `map` uses — and follows links outward, so it finds the authenticated pages a sitemap never lists. It is built to be side-effect-free:
|
|
285
|
-
- **same-origin GET navigations only**, and it **never visits a URL that reads like an action** — `/logout` (which would end the session for every page after it), `/delete`, `/pay`, `/order`, `/unsubscribe`… A GET that mutates is bad REST but real, so the guard is a denylist on the URL itself.
|
|
286
|
-
- to find links hidden behind a menu, dropdown, or dialog, it **opens** them — and does nothing else. Only pure **disclosure toggles** are clicked (an `aria-haspopup`, a collapsed `aria-expanded="false"`, a `<summary>`); never a link, never a submit, never a control whose name reads like an action, never one already open. If a click unexpectedly navigates, that was not a disclosure — it records where it went and steps back. After reading, it presses `Escape` to close. It never clicks the revealed items; it only harvests their `href`s for the crawl. Reveal goes **one level deep** — open it and that's all. Turn it off entirely with **`--no-reveal`** (crawl links only).
|
|
79
|
+
Precedence: `--session` → `IAQA_SESSION` → `"session"` in config → what `login` wrote. A
|
|
80
|
+
session you *named* and that cannot be read stops the run; it never falls through to "no
|
|
81
|
+
session", which would capture the login page under your pages' names.
|
|
287
82
|
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
--max <n> cap candidates per source
|
|
291
|
-
--strict-host require an exact host match (default: www./apex are one site)
|
|
292
|
-
```
|
|
83
|
+
`login` needs a terminal and refuses under CI. There, use `auth` (a form fill, secrets from
|
|
84
|
+
env/SSM — see **Configuration**) or capture-during-run, where your own suite logs itself in.
|
|
293
85
|
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
```
|
|
297
|
-
🗺 Coverage — 6 of 8 discovered pages in config.pages · 2 not yet covered:
|
|
298
|
-
|
|
299
|
-
name url (source)
|
|
300
|
-
dashboard /dashboard (crawl)
|
|
301
|
-
settings /settings (crawl)
|
|
302
|
-
Run again with --apply to add the uncovered pages to config.pages.
|
|
303
|
-
```
|
|
86
|
+
## Which verb answers which question
|
|
304
87
|
|
|
305
|
-
|
|
88
|
+
| Your question | Verb |
|
|
89
|
+
|---|---|
|
|
90
|
+
| my tests broke — fix them | `run` (capture → diff → confirm → fix → re-run to verify) |
|
|
91
|
+
| did anything drift? | `diff` → PASS/FIX/BLOCK, the CI gate |
|
|
92
|
+
| **my suite already ran with `IAQA_CAPTURE=1`** | **`diff`, on its own** — merges the capture, no browser, no second run |
|
|
93
|
+
| day one, no baseline — is my suite still valid? | `audit` (advisory; never gate on it) |
|
|
94
|
+
| is the app itself sound? | `check` — dead links, unnamed elements, name collisions, orphan pages |
|
|
95
|
+
| which pages am I *not* testing? | `discover` (`--sitemap`, or `--crawl` behind the login) |
|
|
96
|
+
| what do my tests actually use? | `ingest` |
|
|
97
|
+
| show me the app's structure | `graph --format mermaid\|svg\|json\|markdown` |
|
|
98
|
+
| is this getting better or worse? | `history` |
|
|
99
|
+
| let me look and decide myself | `ui` — a local console |
|
|
306
100
|
|
|
307
|
-
|
|
101
|
+
`ia-qa-heal <verb> --help` for flags. On a slow suite, note that `run` runs it **twice** (once
|
|
102
|
+
to capture, once to verify the fix): `run --no-verify` keeps only the first, and
|
|
103
|
+
`IAQA_CAPTURE=1` on the run you already do in CI plus `ia-qa-heal diff` costs nothing extra.
|
|
308
104
|
|
|
309
|
-
|
|
105
|
+
## The verdict
|
|
310
106
|
|
|
311
|
-
```bash
|
|
312
|
-
npx ia-qa-heal ui
|
|
313
107
|
```
|
|
314
|
-
|
|
315
|
-
…or, for someone who should never have to open a terminal at all:
|
|
316
|
-
|
|
317
|
-
```bash
|
|
318
|
-
npx ia-qa-heal ui --shortcut # once per project
|
|
108
|
+
🔧 FIX 2 ok · 1 renamed · 3 healable · 0 ambiguous · 0 lost
|
|
319
109
|
```
|
|
320
110
|
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
|
|
111
|
+
- **PASS** — nothing moved.
|
|
112
|
+
- **FIX** — every break has a deterministic old→new rewrite. `fix` applies it.
|
|
113
|
+
- **BLOCK** — a human decides. Nothing is rewritten.
|
|
324
114
|
|
|
325
|
-
|
|
326
|
-
- **The verbs as buttons** — capture, ingest, diff, audit, check, preview/apply the rewrites — with the output streamed back.
|
|
327
|
-
- **The run history as charts** — verdicts over time, mean coverage, and which pages actually churn.
|
|
328
|
-
|
|
329
|
-
**How it is kept safe**, because a server on a QA's machine is not free:
|
|
115
|
+
What `fix` **refuses** is the point of the tool:
|
|
330
116
|
|
|
331
117
|
| | |
|
|
332
118
|
|---|---|
|
|
333
|
-
|
|
|
334
|
-
|
|
|
335
|
-
|
|
|
336
|
-
|
|
|
337
|
-
|
|
338
|
-
**It draws the findings; it does not quote them.** The console runs `check --json` and `audit --json` and renders the report — broken links with the pages that link to them, name collisions with their role and count, unnamed elements with their selector, each card carrying the sentence that says why it matters, in a full-width panel. It used to paste the verb's terminal output into a black `<pre>`: aligned monospace, folded behind "Technical detail", in the one surface whose premise is that its reader does not want a terminal. The raw output is still there, one click away, for whoever wants it. The network announcement survives the change — the panel states the host and the number of links requested, because a flow that leaves the machine says so whatever renders it.
|
|
339
|
-
|
|
340
|
-
**A decision it states, it offers.** A result that ends *"re-run with a lower `--concurrency`"* or *"use Repair when the plan looks right"* is a terminal instruction printed to the one person who is not in a terminal. Those sentences are buttons now, inside the result card you are already reading: a check that could not verify some links offers **Verify those links again, slowly** and **Check the app without requesting links**; a diff with repairs offers the preview, and the preview offers the repair; an audit finding offers **Check for changes** and **Look at the app again** — and never a repair, because `audit` has no baseline to rewrite from and is advisory by design. Each verb also names its own verdict: `audit` never says "Repairs are ready", because it does not repair. The slower rate is a *frozen argv in the allowlist*, not a number field the page posts — a discrete choice keeps "no new authority" literally true, where a settings field would have traded the invariant for one option.
|
|
341
|
-
|
|
342
|
-
Nothing is uploaded, there is no telemetry and no external asset — the page is inline HTML/CSS/JS and every byte it shows came off your disk. It refuses to start under CI, where an interactive surface would just hang; use `--json`, `--junit` or the exit code there.
|
|
343
|
-
|
|
344
|
-
**One request does leave, and it is stated rather than buried:** at startup, at most once a day, `ui` asks `registry.npmjs.org` whether a newer version of this package exists, so the banner can tell you. The desktop shortcut launches a fixed path on disk, and the person clicking it does not own a terminal — this is the only way they would ever find out. It sends a request for a public package's version number: nothing about your app, your tests, your contracts or your results, no telemetry, no identifier. The answer is cached in `~/.ia-qa/update-check.json`, the terminal names the host before the request happens, and **`ia-qa-heal ui --offline` removes it entirely**.
|
|
345
|
-
|
|
346
|
-
The banner stops at the command to paste. It installs nothing, and there is no button that would: the console's authority is exactly the CLI's, and `npm install` is a different binary that runs whatever lifecycle scripts the registry serves — wiring that to a button would widen a leaked token from "edits my test files" to "runs arbitrary code as me". If you started through `npx`, it says so and tells you there is nothing to install.
|
|
347
|
-
|
|
348
|
-
### Several projects — the switcher
|
|
349
|
-
|
|
350
|
-
The header carries a **Project** dropdown as soon as there is more than one. It is governed by a single rule:
|
|
351
|
-
|
|
352
|
-
> **A project is reachable only if you opened a console in it.**
|
|
353
|
-
|
|
354
|
-
Nothing scans your disk. `~/.ia-qa/projects.json` is a consent list, and there are exactly two ways to consent: run `ia-qa-heal ui` in a folder, or pick **＋ Add another project…** in the dropdown and give the path. Switching between listed projects resolves the requested directory against that list *exactly* — a folder outside it cannot be named. Entries whose project has been deleted are dropped rather than offered, and actions are handed `--config <active project>` explicitly rather than inheriting anything.
|
|
355
|
-
|
|
356
|
-
Adding from the console is a deliberate action by whoever holds the session token — the same token that already authorizes *Repair my test files*. What the list defends against is a **drive-by**: another page in your browser reaching localhost. That is stopped by the token and the `Host` check, not by making you walk to a terminal.
|
|
357
|
-
|
|
358
|
-
```bash
|
|
359
|
-
ia-qa-heal ui --forget # clear the whole list
|
|
360
|
-
ia-qa-heal ui --forget ../shop # drop one project
|
|
361
|
-
```
|
|
362
|
-
|
|
363
|
-
Forgetting only removes a folder from the switcher — it deletes nothing in the project, and opening a console there again re-adds it. `IAQA_HOME` relocates the list (useful on a shared machine).
|
|
364
|
-
|
|
365
|
-
## Visualize the app — `ia-qa-heal graph`
|
|
366
|
-
|
|
367
|
-
Turn the mappings you already have into a **navigation graph** (page → page, derived from the `href` on each page's links — the same engine `map` uses for `_overview.md`). Read-only: it never rewrites your mapping files, it just renders them.
|
|
368
|
-
|
|
369
|
-
```bash
|
|
370
|
-
ia-qa-heal graph # Mermaid on stdout — paste into a PR comment or mermaid.live
|
|
371
|
-
ia-qa-heal graph --format json # ia-qa-graph@1 intermediate: { nodes[], edges[], meta }
|
|
372
|
-
ia-qa-heal graph --format markdown --out nav-kb.md # one card per page — a RAG chunk / Confluence page
|
|
373
|
-
ia-qa-heal graph --format svg --out nav.svg # the same SVG map writes as _navigation.svg
|
|
374
|
-
ia-qa-heal graph --format json | your-agent # hand the graph to an agent to analyze
|
|
375
|
-
```
|
|
376
|
-
|
|
377
|
-
`--format` is one of `mermaid` (default) · `svg` · `json` · `markdown`; `--out <file>` writes to a file instead of stdout; `--open` opens it. `mermaid` and `svg` are the same views `map` already writes to `_navigation.md`/`_navigation.svg` — offered here for stdout/pipe.
|
|
378
|
-
|
|
379
|
-
The **`json`** form is the portable `ia-qa-graph@1` IR — nodes carry `{ url, elementCount, entry, isolated }`, and `meta` carries the coverage gap (`unmapped`), collapsed hubs (`fans`), any `mesh`, and `entries`. It matches the node/edge shape the ia-qa.com Graph tool consumes, so it drops into other viewers (D3, Reagraph) without conversion.
|
|
380
|
-
|
|
381
|
-
The **`markdown`** form linearizes the graph into one section per page — *what it is, where it links, what reaches it* — so a page is a self-contained chunk you can index in a **RAG knowledge base** (split on the `##` headings) or push to **Confluence** (`create_confluence_page` on the ia-qa MCP). It carries the navigation *relations* the per-page `map` contracts don't, and ends with a **Coverage gaps** list of linked-but-unmapped destinations.
|
|
382
|
-
|
|
383
|
-
## The one-verb loop: `ia-qa-heal run`
|
|
384
|
-
|
|
385
|
-
`run` chains the whole loop and stops where a human belongs:
|
|
386
|
-
|
|
387
|
-
```
|
|
388
|
-
capture → ingest → diff → dry-run of the fixes → your confirmation → fix → re-run to verify
|
|
389
|
-
```
|
|
390
|
-
|
|
391
|
-
- If `config.testCommand` is set (e.g. `npx playwright test`), `run` spawns it with `IAQA_CAPTURE=1` and the mapping is **captured during your own suite** — login, modals, wizards included, with the credentials your suite already manages. No `testCommand` → falls back to `map`.
|
|
392
|
-
- `BLOCK` (lost / ambiguous / rebound) stops for your judgment. Healable drift is shown as a dry run first; `--yes` skips the confirmation for CI. Nothing is ever committed.
|
|
393
|
-
- **After the fix, `run` re-runs your suite once more (without `IAQA_CAPTURE`) to confirm the rewrite is actually green** — a selector fix is only worth anything if the tests pass after it. The exit code follows that re-run: `0` if the suite passes, `1` if it still fails (the rewrite landed but something else is red). `--no-verify` applies the fix and skips the re-run; the re-run is also skipped, with a reason, when nothing was rewritten or there is no `testCommand`.
|
|
394
|
-
- With `--report`, the HTML for a `run` additionally carries the **applied** `old → new` edits (file and line numbers) and the verify verdict — the file-level before/after the `diff` report only promised.
|
|
119
|
+
| `ambiguous` | several candidates — any choice is a coin flip |
|
|
120
|
+
| `lost` | the element is gone; there is nothing to rewrite towards |
|
|
121
|
+
| `rebound` | the selector now finds a *different* element — the test passes and acts on the wrong thing |
|
|
122
|
+
| `unattributable` | `getByText`, `cy.contains`, `getByTitle`… name a **string, not an element**. Nothing proves the test meant the renamed button rather than a heading that never moved, so editing one would break a *passing* test |
|
|
395
123
|
|
|
396
|
-
|
|
124
|
+
`--locators` rewrites to `getByRole(role, { name })` instead of a fresher CSS path, where that
|
|
125
|
+
is provably safe — a selector that survives the next layout change.
|
|
397
126
|
|
|
398
|
-
|
|
399
|
-
`IAQA_CAPTURE=1`, in CI or by hand — the verdict is one offline command away:
|
|
127
|
+
## Page Objects written in CSS
|
|
400
128
|
|
|
401
|
-
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
```
|
|
405
|
-
|
|
406
|
-
`diff`, `fix` and `baseline` all merge the staging directory (`.ia-qa/mapping/.capture/`) on the
|
|
407
|
-
way in and say so. **Do not run `map` to "finish" a capture**: `map` opens its own browser and
|
|
408
|
-
starts over, without whatever session your suite had.
|
|
409
|
-
|
|
410
|
-
And on a slow suite, `run` runs it **twice** — once to capture, once to verify the fix. `run
|
|
411
|
-
--no-verify` keeps the first only; the exit code then reflects the diff rather than a re-proved
|
|
412
|
-
green suite.
|
|
413
|
-
|
|
414
|
-
### Capture during your test run — `@ia-qa/self-healing/capture`
|
|
415
|
-
|
|
416
|
-
One line at the top of a spec (or wrap your own extended `test` with `withCapture`):
|
|
417
|
-
|
|
418
|
-
```ts
|
|
419
|
-
import { test, expect } from '@ia-qa/self-healing/capture';
|
|
420
|
-
```
|
|
421
|
-
|
|
422
|
-
Inert unless `IAQA_CAPTURE=1` — a normal run pays nothing. When armed, each worker records the pages it visits (per navigation, debounced, plus the final state of each test) into `.ia-qa/mapping/.capture/`; the next CLI verb merges the shards into regular page contracts, named from `config.pages` so they pair with your baseline. Views reached through `steps` share their URL and are skipped — `map` remains their path.
|
|
423
|
-
|
|
424
|
-
### Capture during your test run — Cypress (`@ia-qa/self-healing/cypress`)
|
|
425
|
-
|
|
426
|
-
Same idea, one import in `cypress/support/e2e.js`:
|
|
427
|
-
|
|
428
|
-
```js
|
|
429
|
-
import '@ia-qa/self-healing/cypress';
|
|
430
|
-
```
|
|
431
|
-
|
|
432
|
-
Inert unless `IAQA_CAPTURE=1` (set it as a real env var, or forward it in `cypress.config.js` → `env: { IAQA_CAPTURE: process.env.IAQA_CAPTURE }`). When armed, it records the interactive elements of the page **each test ends on** — post-login, modals, wizards, whatever your suite already walks — and writes them to `.ia-qa/mapping/.capture/` via `cy.writeFile`. **Nothing to wire in `setupNodeEvents`.** The next CLI verb (`diff` / `run` / `baseline`) merges the shards into the same page contracts as `map` and the Playwright capture — byte-compatible.
|
|
129
|
+
Your Page Object says `.btn-primary`. The contract says `#pay-now`. Both name the same button,
|
|
130
|
+
and until they are **bound** every drift row reads *not referenced by your tests* —
|
|
131
|
+
technically true, useless in practice.
|
|
433
132
|
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
- Coverage is whatever the suite visits — it is not a crawler. To get a page's contract, have a test that **ends** on it (v1 captures the final state of each test, not every intermediate navigation).
|
|
438
|
-
- Only the app-under-test's top document is read (iframes/shadow DOM excluded), and the extractor runs in the AUT realm via `eval` — an app whose CSP forbids `unsafe-eval` will silently capture nothing (the capture never fails your suite).
|
|
439
|
-
- Parallel runners (`cypress-parallel`, Cypress Cloud) are fine: each worker writes its own uniquely-named shard and the merge unions them.
|
|
440
|
-
- If a `.ia-qa/config.json` exists, contracts are named from `config.pages` (so they pair with your baseline) and `steps`-only views are skipped; with no config, the page name falls back to a slug of the URL.
|
|
441
|
-
|
|
442
|
-
### Capture during your test run — Selenium, JS (`@ia-qa/self-healing/selenium`)
|
|
443
|
-
|
|
444
|
-
A `selenium-webdriver` (Node) test runs in Node, so — unlike Cypress — there is nothing to hook and no `cy.writeFile`: you call it yourself after a navigation, and it writes the shard directly.
|
|
445
|
-
|
|
446
|
-
```js
|
|
447
|
-
const { capture } = require('@ia-qa/self-healing/selenium');
|
|
448
|
-
|
|
449
|
-
await driver.get('https://app.example.com/checkout');
|
|
450
|
-
await capture(driver); // inert unless IAQA_CAPTURE=1
|
|
451
|
-
```
|
|
452
|
-
|
|
453
|
-
Inert without `IAQA_CAPTURE=1`, so the calls can live in the suite permanently. It injects the extractor into the live page via `driver.executeScript` (the Selenium twin of `page.evaluate`), names the page (config-aware, `steps`-views skipped), unions repeats, and writes `.ia-qa/mapping/.capture/sel-<worker>.json` — the same shard the merge already reads. Parallel workers each write their own file. Errors are swallowed; the capture never fails your test. Honors `IAQA_CONFIG_DIR` for monorepos, like every other path.
|
|
454
|
-
|
|
455
|
-
### Any other framework / language — write a shard
|
|
456
|
-
|
|
457
|
-
The shard format **is** the interface; the Playwright/Cypress/Selenium helpers are just sugar over it. Non-JS Selenium (Python, Java, C#…), WebdriverIO, or a home-grown harness can feed capture directly: run the extractor in the page (it ships as the copy-paste DevTools snippet on ia-qa.com, and as `extract.js` in this package), then drop a JSON file into `.ia-qa/mapping/.capture/` shaped like:
|
|
458
|
-
|
|
459
|
-
```json
|
|
460
|
-
{ "pages": [ { "page": "checkout", "url": "/checkout",
|
|
461
|
-
"elements": [ { "role": "button", "name": "Pay", "selector": "button#pay" } ] } ] }
|
|
462
|
-
```
|
|
463
|
-
|
|
464
|
-
Any `*.json` in that directory is merged into contracts by the next `ia-qa-heal` verb — named to match `config.pages` so it pairs with your baseline. That is the whole contract: no import, no Node required.
|
|
465
|
-
|
|
466
|
-
### Locator inventory — `ia-qa-heal ingest`
|
|
467
|
-
|
|
468
|
-
```bash
|
|
469
|
-
npx ia-qa-heal ingest tests/ # or no args: reads config.testPaths
|
|
470
|
-
```
|
|
471
|
-
|
|
472
|
-
Statically scans your tests/POMs → `.ia-qa/usage.json`. Two halves, because suites are written two ways:
|
|
473
|
-
|
|
474
|
-
- **Selectors** — Playwright, Cypress, Selenium call shapes + page-object declarations (`loginButton = '#login .cta'`, any name, any language). `diff` annotates every drifted selector with *used N× in M test files*, and `fix` no longer needs the test paths on the command line.
|
|
475
|
-
- **Names** — the `get|find|queryBy…` family (Playwright, Testing Library, `@testing-library/cypress`): `ByRole`, `ByLabel(Text)`, `ByPlaceholder(Text)`, `ByText`, `ByTitle`, `ByAltText` — plus `cy.contains(…)` (both forms) and Selenium's `By.linkText` / `By.LINK_TEXT` / `By.PARTIAL_LINK_TEXT`. These are what makes the rename gate below possible.
|
|
476
|
-
|
|
477
|
-
Purely static: no execution, no network. A locator built by concatenation or holding `${…}` is invisible here — and invisible to `fix` too, which is why it is skipped rather than guessed at.
|
|
478
|
-
|
|
479
|
-
### Page Objects written in CSS — `_resolved/`
|
|
480
|
-
|
|
481
|
-
Your Page Object says `.btn-primary`. The contract says `#pay-now`. Both name the same
|
|
482
|
-
button, and until they are **bound** every drift row reads `not referenced by your tests` —
|
|
483
|
-
technically true, practically useless, and the reason a CSS-anchored suite used to get
|
|
484
|
-
nothing out of a diff.
|
|
485
|
-
|
|
486
|
-
So `map` binds them: with `usage.json` present (run `ia-qa-heal ingest` first), each
|
|
487
|
-
selector your tests write is evaluated against the page and recorded in
|
|
488
|
-
`.ia-qa/mapping/_resolved/<page>.json`. `baseline` promotes those alongside the contracts,
|
|
489
|
-
and `diff` compares the two moments.
|
|
490
|
-
|
|
491
|
-
What that buys you — an example that no other check in this package can catch:
|
|
133
|
+
So `map` binds them (with `usage.json` present — run `ingest` first) and records the result in
|
|
134
|
+
`.ia-qa/mapping/_resolved/`. What that buys, on a diff where the element contract did not move
|
|
135
|
+
at all:
|
|
492
136
|
|
|
493
137
|
```
|
|
494
138
|
🔧 FIX
|
|
495
|
-
2 ok · 0 renamed · 0 healable · 0 lost
|
|
139
|
+
2 ok · 0 renamed · 0 healable · 0 lost ← the contract is spotless
|
|
496
140
|
|
|
497
141
|
🔗 1 selector your tests write no longer reaches what it used to:
|
|
498
142
|
🔧 .btn-primary → #pay-now
|
|
@@ -500,38 +144,32 @@ What that buys you — an example that no other check in this package can catch:
|
|
|
500
144
|
tests/checkout.spec.js:1, tests/checkout.spec.js:4
|
|
501
145
|
```
|
|
502
146
|
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
Four outcomes, and only one of them is repaired automatically:
|
|
147
|
+
Somebody renamed a CSS class. The element diff is clean, the suite is broken, and `fix`
|
|
148
|
+
rewrites the literal in your files. Four outcomes, one repaired automatically:
|
|
507
149
|
|
|
508
150
|
| | |
|
|
509
151
|
|---|---|
|
|
510
|
-
|
|
|
511
|
-
|
|
|
512
|
-
|
|
|
513
|
-
|
|
|
514
|
-
|
|
515
|
-
**
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
But when it *does* carry one — a `data-testid`, an `id`, an `aria-label` — the repair is
|
|
520
|
-
exact:
|
|
152
|
+
| reaches **nothing**, and its element is still there | 🔧 rewritten |
|
|
153
|
+
| reaches a **different** element | ⛔ never rewritten — your test passes and acts on the wrong thing |
|
|
154
|
+
| reaches **several** elements | ⛔ a coin flip |
|
|
155
|
+
| its element is **gone** | ⛔ nothing to rewrite towards |
|
|
156
|
+
|
|
157
|
+
**Your app may already carry the answer.** The contract holds interactive elements only, so a
|
|
158
|
+
selector on a status div, a cell or an alert is not in it. Usually that stays quiet — such an
|
|
159
|
+
element has no identity independent of what drifts, so nothing could be repaired anyway. But
|
|
160
|
+
when it *does* carry one (`data-testid`, `id`, `aria-label`), the repair is exact:
|
|
521
161
|
|
|
522
162
|
```
|
|
523
163
|
🔧 .infoBlockContainer > .alert → [data-testid="cart-alert"]
|
|
524
|
-
the element is still there, only its address moved.
|
|
525
|
-
pom/CartPage.ts:2, pom/CartPage.ts:4
|
|
526
164
|
```
|
|
527
165
|
|
|
528
|
-
That is the common shape of a legacy suite: the front end added test ids years after the
|
|
529
|
-
|
|
530
|
-
|
|
531
|
-
|
|
532
|
-
**How much of your suite is in reach?** `map`
|
|
533
|
-
|
|
534
|
-
|
|
166
|
+
That is the common shape of a legacy suite: the front end added test ids years after the Page
|
|
167
|
+
Objects were written and nobody propagated them. It cannot be noisy — an element with a test
|
|
168
|
+
id has a stable identity by construction, and an anchor matching two elements is refused.
|
|
169
|
+
|
|
170
|
+
**How much of your suite is in reach?** `map` says so at the end, over the union of the pages
|
|
171
|
+
it just mapped — the number nobody can add up by hand, because the same string appears on
|
|
172
|
+
several pages:
|
|
535
173
|
|
|
536
174
|
```
|
|
537
175
|
🔗 Your 181 inventoried selectors, against the pages just mapped:
|
|
@@ -540,299 +178,63 @@ hand (the same string appears on several pages):
|
|
|
540
178
|
95 not seen on any page mapped this run — which is about your coverage, not your tests
|
|
541
179
|
```
|
|
542
180
|
|
|
543
|
-
Read the composition, not the ratio. `working` selectors are **fine
|
|
544
|
-
|
|
545
|
-
|
|
546
|
-
|
|
547
|
-
|
|
548
|
-
No `usage.json` ⇒ no bindings ⇒ nothing changes. Same rule as name drift: **no inventory,
|
|
549
|
-
no escalation.** Today the bindings are captured by `map`; capture-during-run is specified
|
|
550
|
-
in `SPEC-selector-resolution.md` and not built yet.
|
|
551
|
-
|
|
552
|
-
### The rename your suite actually depends on
|
|
553
|
-
|
|
554
|
-
A relabelled element is classified `renamed`: the selector still resolves, the click still lands on the right thing. **For a CSS-anchored suite that is harmless. For a name-anchored one it is fatal** — `getByRole('button', { name: 'Save' })` stops matching the moment the button says "Enregistrer", and a selector-only diff would exit 0 while your suite goes red.
|
|
555
|
-
|
|
556
|
-
Once `ingest` has run, `diff` cross-references every rename against what your tests reference:
|
|
557
|
-
|
|
558
|
-
| Situation | Verdict | What `fix` does |
|
|
559
|
-
|---|---|---|
|
|
560
|
-
| No test names that label | unchanged (`PASS`) | nothing — adopting this adds no noise |
|
|
561
|
-
| One element carries the new label | **FIX** | rewrites the call: `getByRole('button', { name: 'Enregistrer' })` |
|
|
562
|
-
| The **old** label now sits on another element | **BLOCK** | nothing — your locator would silently target the wrong element (the name-space twin of `rebound`) |
|
|
563
|
-
| Several elements carry the new label | **BLOCK** | nothing — a human picks |
|
|
564
|
-
| The reference is text-anchored (see below) | **BLOCK** | nothing — reported with its `file:line` |
|
|
565
|
-
|
|
566
|
-
**Only calls that state a role are ever rewritten.** That is the line between the two halves of the inventory, and it is not arbitrary:
|
|
567
|
-
|
|
568
|
-
| Rewritten | Reported, never rewritten |
|
|
569
|
-
|---|---|
|
|
570
|
-
| `getByRole` / `findByRole` / `queryByRole` (Playwright, Testing Library, `@testing-library/cypress`) | `getByText` / `findByText` |
|
|
571
|
-
| `getByLabel` · `getByLabelText` · `getByPlaceholder(Text)` — the role is implied | `cy.contains(…)`, both forms |
|
|
572
|
-
| `By.linkText` / `By.LINK_TEXT` / `By.PARTIAL_LINK_TEXT` (Selenium: role = link) | `getByTitle` · `getByAltText` |
|
|
573
|
-
|
|
574
|
-
`getByText('Archive')` names a *string*, not an element, and the contract maps interactive elements only — so it cannot prove the test meant the renamed button rather than a heading that never moved. Rewriting it would break a test that was passing, which is worse than any drift this tool detects. Those references are printed with their `file:line` and hold the verdict at BLOCK for a human instead.
|
|
575
|
-
|
|
576
|
-
Without `usage.json` nothing is escalated at all, so an un-ingested project behaves exactly as before.
|
|
577
|
-
|
|
578
|
-
### Is the app itself sound? — `ia-qa-heal check`
|
|
579
|
-
|
|
580
|
-
```bash
|
|
581
|
-
npx ia-qa-heal check # dead links + static checks
|
|
582
|
-
npx ia-qa-heal check --offline # skip the network half, fully local
|
|
583
|
-
npx ia-qa-heal check --strict # fail the pipeline on warnings too
|
|
584
|
-
npx ia-qa-heal check --concurrency 2 # gentler on a target with a strict rate limiter
|
|
585
|
-
```
|
|
586
|
-
|
|
587
|
-
`diff` answers *what moved since the baseline?*. `check` answers what a diff structurally cannot: **is the current contract healthy at all?** A green diff over a page whose every link 404s is still a green diff — verified on a sample app where `diff` exits 0 and `check` exits 1 at the same instant.
|
|
588
|
-
|
|
589
|
-
Every check is admitted under one rule: **it must be able to fail for a reason a contract diff would not already catch.** "The button still exists" is rejected by that rule — `diff` proves it better and without a browser. What survives:
|
|
590
|
-
|
|
591
|
-
| Check | Fails when | Verdict |
|
|
592
|
-
|---|---|---|
|
|
593
|
-
| **dead links** | the app answered 4xx/5xx (401/403 excluded — that is a working link behind auth; 429 excluded — see below) | **FAIL** |
|
|
594
|
-
| **unnamed** | an interactive element has no accessible name: no screen-reader announcement, no `getByRole`, no healing | WARN |
|
|
595
|
-
| **ambiguous** | two elements share role + name — every name-anchored locator for them is a coin flip, and a strict-mode violation waiting | WARN |
|
|
596
|
-
| **orphans** | a mapped page nothing links to | WARN |
|
|
597
|
-
| **unverified** | the request was throttled (429) or never answered — no verdict was reached | WARN |
|
|
181
|
+
Read the composition, not the ratio. `working` selectors are **fine**. And `not seen` is a
|
|
182
|
+
statement about how many pages you map: a selector on a page absent from `config.pages`, or in
|
|
183
|
+
a state no capture reached, looks exactly like one that is gone. `ia-qa-heal discover` lists
|
|
184
|
+
what you are missing.
|
|
598
185
|
|
|
599
|
-
|
|
186
|
+
No `usage.json` ⇒ no bindings ⇒ nothing changes. Same rule as name drift: **no inventory, no
|
|
187
|
+
escalation.**
|
|
600
188
|
|
|
601
|
-
|
|
189
|
+
## Capture during your own test run
|
|
602
190
|
|
|
603
|
-
|
|
191
|
+
`map` walks URLs. Your suite walks *states* — post-login pages, modals, wizard steps — and
|
|
192
|
+
already handles whatever auth your app demands. Arm it with `IAQA_CAPTURE=1` and the contract
|
|
193
|
+
is captured during the real run. **This is the intended path for a multi-domain / federated-SSO
|
|
194
|
+
app**, and it never touches your credentials.
|
|
604
195
|
|
|
605
|
-
**
|
|
196
|
+
**Playwright** — one import at the top of a spec (or wrap your own extended `test` with
|
|
197
|
+
`withCapture`):
|
|
606
198
|
|
|
607
|
-
|
|
608
|
-
|
|
609
|
-
```bash
|
|
610
|
-
npx ia-qa-heal audit tests/ # advisory, exit 0
|
|
611
|
-
npx ia-qa-heal audit --strict # exit 1 on a finding
|
|
199
|
+
```ts
|
|
200
|
+
import { test, expect } from '@ia-qa/self-healing/capture';
|
|
612
201
|
```
|
|
613
202
|
|
|
614
|
-
|
|
203
|
+
**Cypress** — one import in `cypress/support/e2e.js`, nothing to wire in `setupNodeEvents`:
|
|
615
204
|
|
|
616
|
-
|
|
617
|
-
|
|
618
|
-
- Only **role-anchored** locators carry the verdict, and only for roles a contract can hold. `getByText('Terms of service')` may point at a paragraph `map` never captured, and `getByRole('heading', …)` at a heading it never captures — their absence proves nothing, so they are counted and not judged.
|
|
619
|
-
- **Advisory by default** (exit 0). A "missing" locator may live on a page not in `config.pages`, or in a page *state* `map` never saw (a label behind a tab or a mode toggle). Both caveats are printed next to the findings. Add `--strict` once your coverage is real.
|
|
620
|
-
- Read-only. `audit` never edits and never applies a suggestion.
|
|
621
|
-
|
|
622
|
-
### Elements with no accessible name
|
|
623
|
-
|
|
624
|
-
An icon button or image link with no `aria-label`/text has **no role + name identity** — self-healing can't recover it, a `getByRole` locator can't target it, and a screen reader can't announce it. A live capture of a dynamic app keeps catching a shifting set of them, which is the noisiest thing in a drift diff. Each page contract now counts them and lists them in a dedicated section, and `diff` flags them (`👻`) with a **synthetic hint** derived from the element's icon/image (`icon: chevron-right`, `img: Logo`) so you can find them. The hint is a locating aid, never an accessible name — the durable fix is an `aria-label` or a `data-testid`.
|
|
625
|
-
|
|
626
|
-
## Two front doors, one engine
|
|
627
|
-
|
|
628
|
-
| | For | Entry point |
|
|
629
|
-
|---|---|---|
|
|
630
|
-
| **MCP server** (stdio) | AI agents — "map my app at \<url\>" | `ia-qa-heal-mcp` → tools `map_app`, `diff_mappings`, `fix_tests`, `suggest_heal` (optional AI) |
|
|
631
|
-
| **CLI** | humans & CI | `ia-qa-heal` → `init`, `discover`, `map`, `baseline`, `diff`, `fix`, `ingest`, `audit`, `run` |
|
|
632
|
-
| **AI add-on** (optional, BYOK) | semantic renames the deterministic engine gives up on | `ia-qa-heal-ai` → `suggest` — see [Optional AI add-on](#optional-ai-add-on--semantic-suggestions-ia-qa-heal-ai-byok) |
|
|
633
|
-
|
|
634
|
-
### Locale
|
|
635
|
-
|
|
636
|
-
The page contract's accessible names are language-dependent. If your app or browser OS renders in a different locale than your test environment, every name match will fail. Set the locale explicitly:
|
|
637
|
-
|
|
638
|
-
```bash
|
|
639
|
-
# env var — works for both CLI and MCP paths
|
|
640
|
-
export IAQA_LOCALE=fr-FR # or en-US, de-DE, ja-JP…
|
|
641
|
-
|
|
642
|
-
# or in .ia-qa/config.json:
|
|
643
|
-
{ "locale": "fr-FR", … }
|
|
205
|
+
```js
|
|
206
|
+
import '@ia-qa/self-healing/cypress';
|
|
644
207
|
```
|
|
645
208
|
|
|
646
|
-
|
|
209
|
+
**Selenium (Node)** — you call it yourself, since there is nothing to hook:
|
|
647
210
|
|
|
648
|
-
|
|
649
|
-
|
|
650
|
-
|
|
651
|
-
|
|
652
|
-
"ia-qa-self-healing": {
|
|
653
|
-
"command": "npx",
|
|
654
|
-
"args": ["-y", "-p", "@ia-qa/self-healing", "ia-qa-heal-mcp"]
|
|
655
|
-
}
|
|
656
|
-
}
|
|
657
|
-
}
|
|
211
|
+
```js
|
|
212
|
+
const { capture } = require('@ia-qa/self-healing/selenium');
|
|
213
|
+
await driver.get('https://app.example.com/checkout');
|
|
214
|
+
await capture(driver); // inert unless IAQA_CAPTURE=1
|
|
658
215
|
```
|
|
659
|
-
`-p` is required, not cosmetic: the package ships **three** bins (`ia-qa-heal`, `ia-qa-heal-mcp`, `ia-qa-heal-ai`), so `npx @ia-qa/self-healing …` cannot resolve which to run and silently starts nothing.
|
|
660
|
-
The MCP server is a dependency-free JSON-RPC 2.0 stdio implementation (`src/mcp/server.ts`) — no SDK. It does **not** require the separate Playwright MCP: it drives its own browser. stdout is the protocol channel; logs go to stderr.
|
|
661
|
-
|
|
662
|
-
### Browser reuse — no second Chromium (`src/launcher.ts`)
|
|
663
|
-
Playwright's browser cache is **machine-wide and shared** (`%LOCALAPPDATA%\ms-playwright`, `~/.cache/ms-playwright`). If the user already has *any* Playwright install — their own tests or the Playwright MCP — the bundled Chromium is already there and we reuse it for free. There is no "second Chromium" to avoid; the only real cost is **revision skew** (a different Playwright version pins a different Chromium revision). For that case, and for machines that forbid the download, the launcher resolves in this order:
|
|
664
|
-
|
|
665
|
-
1. explicit executable — `browserPath` / `$IAQA_BROWSER_PATH`
|
|
666
|
-
2. explicit channel — `browserChannel` / `$IAQA_BROWSER_CHANNEL` (`chrome`, `msedge`…) → **zero download**
|
|
667
|
-
3. Playwright's bundled Chromium (shared cache — usually already present)
|
|
668
|
-
4. automatic fallback to the system Chrome, then Edge
|
|
669
|
-
|
|
670
|
-
Only if all four fail does it error, listing the three fixes. `map_app` exposes `browser_channel` / `browser_path` and always reports which browser it used. For DOM extraction the engine is irrelevant — Chrome, Edge and Chromium yield the same contract.
|
|
671
|
-
|
|
672
|
-
## How it works
|
|
673
|
-
|
|
674
|
-
1. **`ia-qa-heal init`** — interactive wizard. This is where you define your **secret zone** and grant the tool access to it: choose the framework, where credentials live (**your environment, typically loaded from a local `.env`** — or **AWS SSM Parameter Store**), the login flow, and the pages to map. It writes `.ia-qa/config.json`, which stores only a **reference** to each secret — the env-var name or the SSM parameter path, **never the value**. At run time `map` reads the credentials from that zone and logs in; if one is missing it **stops with a message naming the exact variable to set** — it never prompts, never guesses, and never writes a secret to disk. **And they never reach ia-qa.** The credentials only fill the login form in the **local** headless browser on your machine; self-healing makes **no call to ia-qa.com or any third party** — the only network traffic is your own browser loading your own app (plus your own AWS account, if you chose SSM). (Behind a login and don't want to wire this up? Use `run` with `IAQA_CAPTURE=1` and let your own test suite handle the login — see [the one-verb loop](#the-one-verb-loop-ia-qa-heal-run).)
|
|
675
|
-
2. **`ia-qa-heal map [page]`** — resolves credentials at runtime (env or SSM via your local AWS credential chain), launches headless Chromium, logs in if configured, and extracts every **visible interactive element** (role + accessible name + stable selector) to **two files** per page: `.ia-qa/mapping/<page>.json` (for tooling / `diff`) and `.ia-qa/mapping/<page>.md` (the **page contract** — the framework-agnostic artifact you hand to an LLM to auto-heal tests):
|
|
676
|
-
```json
|
|
677
|
-
{
|
|
678
|
-
"page": "login",
|
|
679
|
-
"url": "/login",
|
|
680
|
-
"capturedAt": "2026-07-15T10:00:00.000Z",
|
|
681
|
-
"elements": [
|
|
682
|
-
{ "role": "button", "name": "Se connecter", "selector": "button#login" }
|
|
683
|
-
]
|
|
684
|
-
}
|
|
685
|
-
```
|
|
686
|
-
The `<page>.md` groups elements by role and carries a self-describing header telling a model exactly how to use it. **The CI auto-heal loop:** when a UI test fails, hand the failing test file + the current `<page>.md` to an LLM — it finds the element by role + accessible name, reads the current selector, and rewrites the test in **whatever framework and language** it is written in (Cypress, Playwright, Selenium; JS or Python). The contract is the source of truth; the model handles the syntax. The same contract is downloadable from the zero-install **Selector Drift Detector** page on ia-qa.com.
|
|
687
|
-
|
|
688
|
-
Alongside the page contracts, `map` rebuilds the app index and the navigation graph — see [Map artifacts](#map-artifacts).
|
|
689
|
-
3. **`ia-qa-heal baseline`** — promote `.ia-qa/mapping/` to `.ia-qa/baseline/`, the reference `diff` compares against. Commit it: a reference that is not in git cannot be reviewed or reverted, and accepting drift is a code-review decision like any other.
|
|
690
|
-
|
|
691
|
-
Drift is a claim about **two moments**, and `map` only ever captures one. Promotion is the second half, and it is a separate verb on purpose — deliberately *not* a flag on `map`, because promoting in the same breath as capturing would make the reference incapable of ever being older than what it is compared to. `map` records what the app **is**; `baseline` says it is also what the app is **supposed to be**. That second claim is a human judgement.
|
|
692
|
-
|
|
693
|
-
Skip it and the natural workaround is `cp current.json baseline.json && diff` — which compares a capture to a copy of itself and prints `PASS ✅ No drift`. That verdict is green because nothing was compared, not because nothing changed. `diff` now refuses it outright (exit `2`): every side carries a `capturedAt`, equal timestamps mean one `map` run, and one run cannot drift from itself. A gate that reports green without looking is worse than no gate, because it is trusted.
|
|
694
|
-
4. **`ia-qa-heal diff [before.json] [after.json]`** — CI-gate verdict. With no arguments, diffs `baseline/` against `mapping/` — the everyday case. Diffs two mappings (from `map` or the ia-qa.com browser snippet) and reports which selectors survived, which the runtime healer would recover, and which will hard-fail:
|
|
695
|
-
```
|
|
696
|
-
⛔ BLOCK
|
|
697
|
-
before.json → after.json · 3 ok · 2 healable · 1 lost · 2 new
|
|
698
|
-
🔧 healable button "Se connecter" button#login-btn → button.btn-primary (exact name)
|
|
699
|
-
⛔ lost button "Supprimer le compte" button#delete → (no match)
|
|
700
|
-
```
|
|
701
|
-
Exit codes for pipelines: `0` = PASS or FIX, `1` = BLOCK — lost, ambiguous, or rebound (add `--strict` to fail on FIX too), `2` = bad input, no baseline, or nothing to compare. Use `--json` for machine-readable output. Same diff engine as the web tool (`src/browser/match.js`).
|
|
702
|
-
|
|
703
|
-
**`--report [file.html]`** also writes a self-contained, ia-qa.com-branded HTML report of the run — a plain-language summary of what the run checked and found (how many elements across how many pages, what drifted, how many of your test files it touches, and the next action), the PASS/FIX/BLOCK verdict, totals, a per-page table and every drifted selector with its `old → new` transition and test-usage annotation — the artefact to attach to a PR instead of a console screenshot (default `ia-qa-heal-report.html`). Add **`--open`** to open it in your browser (local use; leave it off in CI). Works on `diff` and `run`.
|
|
704
|
-
|
|
705
|
-
A contract that was **not re-mapped** since the baseline carries the baseline's own `capturedAt`, so it cannot show drift. Those are listed and left **out** of the verdict rather than counted as `ok` — padding a gate with comparisons that never happened is the same lie in smaller print. `--json` reports them as `staleExcluded`.
|
|
706
|
-
|
|
707
|
-
Every element carries a **context** (nearest landmark + section heading) so two elements with the same role + accessible name (five "Delete" buttons) stay distinguishable. The diff classifies each baseline element as `ok`, `renamed` (same element, new accessible name — a *content* drift to eyeball), `healable` (deterministic old→new rewrite), `ambiguous` (several equal matches — the tool **refuses to guess**, a human decides), or `lost` (no match).
|
|
708
|
-
|
|
709
|
-
Identity comes from the contract (role + accessible name), never from the selector string. A selector that still resolves only proves the *position* is still occupied: insert one button above a tab bar and every `nth-of-type` below it silently slides onto its neighbour. Any row where that happened is flagged `rebound` — the old selector resolves, but to a **different element**, so the suite is green while clicking the wrong thing. Rebound rows are rewritten like any other healable row, and they **BLOCK**: a `FIX` verdict exits `0`, which is exactly how this ships unnoticed.
|
|
710
|
-
5. **`ia-qa-heal fix <before.json> <after.json> <test-paths…>`** — the deterministic half. Applies **only** the `healable` rewrites to your test files, replacing the selector only where it appears as a quoted string literal (`'…'`/`"…"`/`` `…` ``, quote-safe for attribute selectors), across any framework/language. Every rewrite is applied in a single pass against the selector each occurrence had in the file, so a *permuted* set (`nth-of-type(1)→(2)`, `(2)→(3)`) can never chain and drag `(1)` to `(3)`. LOST / AMBIGUOUS / RENAMED are printed but never touched — they need judgment. It **edits the working tree and never commits**: review with `git diff`, then push or discard. `--dry-run` previews the plan. With **no arguments** it defaults to `baseline/` vs `mapping/`, exactly like `diff`. This is "prepare the ground, the human decides" as a command.
|
|
711
|
-
|
|
712
|
-
**`--locators`** rewrites to a Playwright role locator — `page.getByRole('button', { name: 'Validator', exact: true })` — instead of a selector, wherever role + accessible name identifies exactly one element. A repaired positional selector is only correct until the next insertion; a role locator is what the contract already means, so it does not drift at all. Applies to `.js`/`.ts` files and Playwright call shapes (`page.click('…')`, `page.locator('…')`, `page.fill('…', v)`, …); every other file and every element without an unambiguous role + name keeps the plain selector rewrite. `exact: true` is emitted because the guard already verified exactly one element matches the exact name — without it, `getByRole` does substring matching and would match multiple elements with overlapping names (e.g. three "Wiki" buttons), causing the strict-mode throw the locator was supposed to avoid. When `--locators` is passed but no Playwright call shape is found in the scanned files, a note explains that Cypress (`cy.get(sel).click()`) and other frameworks silently keep the plain selector rewrite.
|
|
713
|
-
6. **Runtime healing** — in your Playwright tests:
|
|
714
|
-
```ts
|
|
715
|
-
import { aiClick, aiFill } from '@ia-qa/self-healing';
|
|
716
|
-
|
|
717
|
-
await aiFill(page, 'input[name="email"]', user);
|
|
718
|
-
await aiClick(page, 'button#login');
|
|
719
|
-
```
|
|
720
|
-
If the selector fails, the helper re-scans the live page, matches the element by role + accessible name (exact, then fuzzy Dice ≥ 0.6 — or your own `llmResolver`), retries on the healed selector, and logs a warning telling you to update the test and re-map.
|
|
721
|
-
|
|
722
|
-
## Optional AI add-on — semantic suggestions (`ia-qa-heal-ai`, BYOK)
|
|
723
|
-
|
|
724
|
-
Everything above is deterministic and **refuses to guess**: a rename Dice can't see — `"Submit"` → `"Confirm order"` — comes back `lost`, and a human fixes it. That refusal is the point; it is what makes a green gate here mean something.
|
|
725
|
-
|
|
726
|
-
`ia-qa-heal-ai` is an **optional, opt-in** third binary that hands **only** those `lost`/`ambiguous` rows to **your own** LLM (bring your own key) and proposes a match. It is a **suggestion you confirm** — never a CI auto-fix. The deterministic `ia-qa-heal` works fully without it; configure no key and nothing changes.
|
|
727
216
|
|
|
728
|
-
|
|
217
|
+
All three are **inert without `IAQA_CAPTURE=1`**, so they can live in the suite permanently,
|
|
218
|
+
and they never fail your tests. Each worker writes its own shard to
|
|
219
|
+
`.ia-qa/mapping/.capture/`; the next verb (`diff`, `run`, `baseline`) merges them into the same
|
|
220
|
+
page contracts `map` produces — byte-compatible, named from `config.pages` so they pair with
|
|
221
|
+
your baseline.
|
|
729
222
|
|
|
730
|
-
**
|
|
731
|
-
|
|
732
|
-
|
|
733
|
-
- **Candidates are pre-filtered** to the same-role top-N nearest the original, so the model breaks a tie the heuristic couldn't — it does not free-search the DOM.
|
|
734
|
-
- `--apply` **refuses without a TTY** — it can never run unattended in CI. The deterministic `fix` stays the gate.
|
|
735
|
-
- **Zero new dependency** (a raw `fetch`), model pinned at temperature 0, key resolved at runtime from `env`/`aws-ssm` and **never written to disk** — same `SecretRef` model as every other secret here.
|
|
223
|
+
**Any other framework or language.** The shard format *is* the interface; the helpers above are
|
|
224
|
+
sugar over it. Run the extractor in the page (`extract.js` in this package, also a copy-paste
|
|
225
|
+
DevTools snippet on ia-qa.com) and drop a JSON file into `.ia-qa/mapping/.capture/`:
|
|
736
226
|
|
|
737
|
-
**Providers — you pick from a list, the tool does the rest** (bring your own key):
|
|
738
|
-
|
|
739
|
-
| Provider | `provider` | Example models | Default key env |
|
|
740
|
-
|---|---|---|---|
|
|
741
|
-
| OpenAI | `openai` | `gpt-4o-mini`, `gpt-4.1`, `gpt-5`, `gpt-5-mini`, `gpt-5-nano` | `OPENAI_API_KEY` |
|
|
742
|
-
| Anthropic (Claude) | `anthropic` | `claude-haiku-4-5`, `claude-sonnet-5`, `claude-opus-4-8` | `ANTHROPIC_API_KEY` |
|
|
743
|
-
| Google (Gemini) | `google` | `gemini-2.0-flash`, `gemini-1.5-pro` | `GEMINI_API_KEY` |
|
|
744
|
-
|
|
745
|
-
The model list is a convenience — **any model id your provider accepts works** (the picker has a "custom…" entry), so point releases like `gpt-5.1` or dated snapshots you just type in. The call runs at temperature 0 and **auto-retries without it** for models that reject sampling params (newest Claude/GPT reasoning models), so your choice always works. `ia-qa-heal-ai models` prints the full list.
|
|
746
|
-
|
|
747
|
-
**Setup — the easy way:**
|
|
748
|
-
```bash
|
|
749
|
-
npx ia-qa-heal-ai init # pick provider + model from a list → writes the "ai" block for you
|
|
750
|
-
export OPENAI_API_KEY=sk-… # (or ANTHROPIC_API_KEY / GEMINI_API_KEY — whatever you chose)
|
|
751
|
-
```
|
|
752
|
-
`init` only writes a *reference* to the env var into `.ia-qa/config.json` — never the key itself. Or add the block by hand:
|
|
753
227
|
```json
|
|
754
|
-
{
|
|
755
|
-
|
|
756
|
-
"provider": "openai",
|
|
757
|
-
"model": "gpt-4o-mini",
|
|
758
|
-
"apiKey": { "source": "env", "key": "OPENAI_API_KEY" }
|
|
759
|
-
}
|
|
760
|
-
}
|
|
761
|
-
```
|
|
762
|
-
`minConfidence` is optional (default `0.7`).
|
|
763
|
-
|
|
764
|
-
**Use** — right after `diff`:
|
|
765
|
-
```bash
|
|
766
|
-
npx ia-qa-heal-ai suggest # baseline/ vs mapping/, all pages
|
|
767
|
-
npx ia-qa-heal-ai suggest before.json after.json # one pair
|
|
768
|
-
npx ia-qa-heal-ai suggest --json # machine-readable, no writes
|
|
769
|
-
npx ia-qa-heal-ai suggest --report r.html # branded HTML report to attach to a PR (add --open locally)
|
|
770
|
-
npx ia-qa-heal-ai suggest --apply # rewrite accepted ones — interactive, TTY only
|
|
771
|
-
```
|
|
772
|
-
|
|
773
|
-
`--report` writes the same self-contained, ia-qa.com-branded artefact as `ia-qa-heal diff --report` — but of the **AI proposals**, with a loud "these are suggestions, not applied changes" banner. Attach it to a PR for a reviewer to sign off on what the model proposed.
|
|
774
|
-
```
|
|
775
|
-
💡 lost button "Submit" (checkout)
|
|
776
|
-
button#submit → button[data-testid="confirm"]
|
|
777
|
-
proposes button "Confirm order" · confidence 91%
|
|
778
|
-
↳ same primary action of the checkout form, relabelled
|
|
779
|
-
```
|
|
780
|
-
|
|
781
|
-
**For agents** — the same thing is the MCP tool `suggest_heal`: it reads its key from the server's own environment (`api_key_env`, default `ANTHROPIC_API_KEY`/`OPENAI_API_KEY`) and **never accepts a raw key in the call**; it returns suggestions as JSON and writes nothing.
|
|
782
|
-
|
|
783
|
-
**Runtime** — the same resolver behind the `aiClick`/`aiFill` seam, opt-in:
|
|
784
|
-
```ts
|
|
785
|
-
import { aiClick, createAiResolver } from '@ia-qa/self-healing';
|
|
786
|
-
|
|
787
|
-
await aiClick(page, 'button#login', {
|
|
788
|
-
llmResolver: createAiResolver({ provider: 'anthropic', model: 'claude-…', apiKey: process.env.ANTHROPIC_API_KEY! }),
|
|
789
|
-
});
|
|
790
|
-
```
|
|
791
|
-
A low-confidence, malformed, or failed call returns `null` and falls through to the deterministic heuristic — the AI never breaks a run.
|
|
792
|
-
|
|
793
|
-
See **[ROADMAP.md](ROADMAP.md)** for what comes next — and for why the vision/VLM phase was measured and dropped: no image ever leaves your machine, because none is ever sent anywhere.
|
|
794
|
-
|
|
795
|
-
## Page Object Models — one file to keep in sync
|
|
796
|
-
|
|
797
|
-
A POM already does what this tool wants: it lifts every selector out of the tests and into **one file**. That is also the one file drift keeps breaking. `map` → `diff` → `fix` treats it like any other test file — the selector is a quoted string literal, so `fix` rewrites it in place.
|
|
798
|
-
|
|
799
|
-
```ts
|
|
800
|
-
// login.page.ts — the only place a selector is written
|
|
801
|
-
export class LoginPage {
|
|
802
|
-
readonly emailSelector = '#email';
|
|
803
|
-
readonly passwordSelector = '#password';
|
|
804
|
-
readonly submitSelector = 'button#login';
|
|
805
|
-
// used everywhere as page.fill(this.emailSelector, …)
|
|
806
|
-
}
|
|
807
|
-
```
|
|
808
|
-
|
|
809
|
-
A developer reworks the login form: `button#login` becomes `button.btn-primary`. The whole suite goes red through this one file. Heal it:
|
|
810
|
-
|
|
811
|
-
```bash
|
|
812
|
-
npx ia-qa-heal map # capture the new contract
|
|
813
|
-
npx ia-qa-heal diff # 🔧 healable button "Log in" button#login → button.btn-primary
|
|
814
|
-
npx ia-qa-heal ingest # inventory the POM's selector literals (optional — lets fix find the file itself)
|
|
815
|
-
npx ia-qa-heal fix .ia-qa/baseline/login.json .ia-qa/mapping/login.json login.page.ts
|
|
816
|
-
```
|
|
817
|
-
|
|
818
|
-
```diff
|
|
819
|
-
export class LoginPage {
|
|
820
|
-
readonly emailSelector = '#email';
|
|
821
|
-
readonly passwordSelector = '#password';
|
|
822
|
-
- readonly submitSelector = 'button#login';
|
|
823
|
-
+ readonly submitSelector = 'button.btn-primary';
|
|
824
|
-
}
|
|
228
|
+
{ "pages": [ { "page": "checkout", "url": "/checkout",
|
|
229
|
+
"elements": [ { "role": "button", "name": "Pay", "selector": "button#pay" } ] } ] }
|
|
825
230
|
```
|
|
826
231
|
|
|
827
|
-
|
|
828
|
-
|
|
829
|
-
**Naming does not matter, paths do.** `ingest` inventories a declaration whose value *reads* like a selector, whatever the property is called — `loginButton = '#login .cta'`, `submitSelector = '…'`, `LOGIN_LINK = "a[data-testid='login']"` (Python and Java page objects included) — as well as framework calls it already knows (`this.page.locator('#login')`). What it cannot do is find a file you never pointed it at: if your POMs live in `pages/` and `testPaths` says `tests/`, nothing there is inventoried, and `diff` will say *0 of N drifted selectors referenced by your tests* — which reads like good news and is not. It now says so; widen `testPaths` and re-run `ingest`.
|
|
830
|
-
|
|
831
|
-
Neighbouring constants (`baseUrl`, a fixture path, a brand colour) are excluded, and a selector built by concatenation or holding `${…}` is invisible — to `fix` too, which is why it is skipped rather than guessed at.
|
|
232
|
+
No import, no Node required. Coverage is whatever the suite visits — it is not a crawler, and a
|
|
233
|
+
page it never reached is `stale` in the next diff, never a pass.
|
|
832
234
|
|
|
833
235
|
## In CI
|
|
834
236
|
|
|
835
|
-
`diff`
|
|
237
|
+
`diff` exits `0` on PASS/FIX, `1` on BLOCK, `2` on bad input:
|
|
836
238
|
|
|
837
239
|
```yaml
|
|
838
240
|
# .github/workflows/selector-drift.yml
|
|
@@ -848,124 +250,143 @@ jobs:
|
|
|
848
250
|
- run: npm ci
|
|
849
251
|
- run: npx playwright install --with-deps chromium
|
|
850
252
|
- run: npm start & # your app on its usual port
|
|
253
|
+
- run: npx ia-qa-heal ingest # inventory what your tests use
|
|
851
254
|
- run: npx ia-qa-heal map # re-capture into .ia-qa/mapping/
|
|
852
|
-
- run: npx ia-qa-heal ingest # inventory selectors AND names your tests use
|
|
853
255
|
- run: npx ia-qa-heal diff # 0 = PASS/FIX · 1 = BLOCK · 2 = bad input
|
|
854
256
|
```
|
|
855
257
|
|
|
856
|
-
`
|
|
258
|
+
`diff` with no arguments compares the committed `.ia-qa/baseline/` against what `map` just
|
|
259
|
+
captured — which is why the baseline belongs in git. `--strict` fails on FIX too;
|
|
260
|
+
`--report drift.html` attaches a branded report to the build.
|
|
857
261
|
|
|
858
|
-
|
|
262
|
+
**Trend charts for free.** `diff --junit reports/drift.xml` writes the verdict as JUnit XML,
|
|
263
|
+
which Jenkins, GitLab, CircleCI, Azure DevOps and the GitHub Actions reporters ingest natively,
|
|
264
|
+
per-build history included. It describes your *contract*, not your suite, so every language
|
|
265
|
+
produces identically shaped XML. `healable` and BLOCK-class rows are failures typed by status,
|
|
266
|
+
`renamed` passes with the label change in `system-out`, and a page that was not compared is
|
|
267
|
+
**`skipped` — never a pass**.
|
|
859
268
|
|
|
860
|
-
|
|
269
|
+
> ⚠️ Publish it under its **own file pattern**, separate from your suite's results. Merged into
|
|
270
|
+
> one trend, both sets of numbers stop meaning anything.
|
|
861
271
|
|
|
862
|
-
`
|
|
272
|
+
`--json` on `diff`, `run`, `audit`, `check` and `history` gives the same verdict as one
|
|
273
|
+
parseable document. Locally, `ia-qa-heal history` charts what no single run can reconstruct —
|
|
274
|
+
and holds out runs captured a *different* way rather than averaging them in.
|
|
863
275
|
|
|
864
|
-
|
|
865
|
-
npx ia-qa-heal diff --junit reports/selector-drift.xml
|
|
866
|
-
```
|
|
867
|
-
|
|
868
|
-
It is language-agnostic by construction: what it describes is your **contract**, not your suite, so a Cypress/JS, Selenium/Python or Playwright/.NET project produces identically shaped XML.
|
|
869
|
-
|
|
870
|
-
| contract row | JUnit | why |
|
|
871
|
-
|---|---|---|
|
|
872
|
-
| `ok` | pass | the locator still finds the same element |
|
|
873
|
-
| `renamed` | pass + `system-out` | the selector still works; the label moved |
|
|
874
|
-
| `healable` | failure `type="healable"` | broken today — auto-repairable, but broken |
|
|
875
|
-
| `ambiguous` · `lost` · `rebound` | failure, typed | broken, and no deterministic repair exists |
|
|
876
|
-
| name drift | failure `type="name-drift-*"` | a name-anchored locator names nothing now |
|
|
877
|
-
| page not compared | **skipped** | not measured — reporting it as a pass would be the false green this tool exists to prevent |
|
|
878
|
-
|
|
879
|
-
> ⚠️ **Publish it under its own file pattern**, separate from your suite's results. Merged into one trend, the two sets of numbers stop meaning anything.
|
|
880
|
-
|
|
881
|
-
`run --json` (and `diff --json`) give the same verdict as one JSON document on stdout. For `run`, every other line moves to stderr so a pipe stays parseable.
|
|
276
|
+
## Configuration
|
|
882
277
|
|
|
883
|
-
|
|
278
|
+
`.ia-qa/config.json`, written by `init`. The minimum:
|
|
884
279
|
|
|
885
|
-
|
|
886
|
-
|
|
887
|
-
|
|
888
|
-
|
|
889
|
-
|
|
890
|
-
|
|
891
|
-
|
|
892
|
-
|
|
893
|
-
|
|
894
|
-
|
|
895
|
-
|
|
280
|
+
```jsonc
|
|
281
|
+
{
|
|
282
|
+
"baseUrl": "https://staging.myapp.com",
|
|
283
|
+
"pages": [
|
|
284
|
+
{ "name": "checkout", "url": "/checkout" },
|
|
285
|
+
// a view with no URL of its own — reach it by naming the control to click
|
|
286
|
+
{ "name": "billing", "url": "/account",
|
|
287
|
+
"steps": [ { "click": { "role": "tab", "name": "Billing" } } ] }
|
|
288
|
+
],
|
|
289
|
+
"testPaths": ["tests/", "pages/"],
|
|
290
|
+
"testCommand": "npx playwright test"
|
|
291
|
+
}
|
|
896
292
|
```
|
|
897
293
|
|
|
898
|
-
|
|
899
|
-
|
|
900
|
-
The file is small, deterministic text (unlike `_shots/`), so committing it is defensible if you want the trend to survive across machines and CI.
|
|
901
|
-
|
|
902
|
-
Agents get the same numbers through the MCP tool **`heal_history`** — one `summarize()`, three renderings (terminal, console charts, MCP).
|
|
903
|
-
|
|
904
|
-
## Map artifacts
|
|
905
|
-
|
|
906
|
-
Beyond one contract per page, every `map` run rebuilds two views of the whole app. Both are derived entirely from the mappings on disk (no extra page visit) and rebuilt on every run, including `map <page>`, so they never drift out of date.
|
|
907
|
-
|
|
908
|
-
**`.ia-qa/mapping/_overview.md`** — the app index, and the file to open first. Every page contract answers "what is on this page?"; none of them answers "what is this app, and which contract do I need?". The overview does: every page with its URL, element count and a link to its contract; the shared layout; and the surface by role. It also surfaces two things no single contract can — same-origin destinations your app links to but your config never maps (the coverage gap), and the external domains you link out to.
|
|
294
|
+
**Your password never goes in this file.** For a form login, `secrets` holds the *name* of an
|
|
295
|
+
env var (or an SSM parameter path) and `auth` points at it by name:
|
|
909
296
|
|
|
910
|
-
|
|
911
|
-
|
|
912
|
-
|
|
913
|
-
|
|
914
|
-
|
|
915
|
-
|
|
916
|
-
- **meshes** — when nearly every mapped page links to nearly every other (the related-tools-widget shape: 16 links between 5 pages), the whole drawing carries one fact — "these pages cross-link" — so the overview states it and no graph is written.
|
|
917
|
-
|
|
918
|
-
Pages **nothing links to** are called out as entry points (orange border in the `.svg` and the Mermaid graph): the way in — a login, a home — or a page only the shell reaches. The map cannot tell which; you can.
|
|
919
|
-
|
|
920
|
-
What is left is the real structure: a checkout tunnel stays, its noise does not. If nothing survives, both files are removed rather than left stale — for that app, the page list genuinely is the whole story.
|
|
921
|
-
|
|
922
|
-
**`.ia-qa/mapping/_shots/<page>.jpg`** — optional, written only by `ia-qa-heal map --screenshots`. One viewport JPEG per page, which `--report` then shows next to each page's name and URL, so a reader sees the view instead of only reading its path:
|
|
923
|
-
|
|
924
|
-
```bash
|
|
925
|
-
ia-qa-heal map --screenshots # capture
|
|
926
|
-
ia-qa-heal diff --report --open # the report now carries the pictures
|
|
297
|
+
```jsonc
|
|
298
|
+
"secrets": { "user": { "source": "env", "key": "APP_USER" },
|
|
299
|
+
"pass": { "source": "env", "key": "APP_PASS" } },
|
|
300
|
+
"auth": { "loginUrl": "/login", "usernameSelector": "#email", "passwordSelector": "#password",
|
|
301
|
+
"submitSelector": "button[type=submit]", "usernameSecret": "user", "passwordSecret": "pass",
|
|
302
|
+
"successSelector": "nav.dashboard" }
|
|
927
303
|
```
|
|
928
304
|
|
|
929
|
-
|
|
930
|
-
|
|
931
|
-
|
|
932
|
-
|
|
933
|
-
## Design notes
|
|
305
|
+
Loading the env var is your job — the CLI reads `process.env` and never opens a `.env` file, so
|
|
306
|
+
run it under `npx dotenv-cli --` or Node's `--env-file`. If a variable is missing it stops and
|
|
307
|
+
names it.
|
|
934
308
|
|
|
935
|
-
|
|
936
|
-
- **Shared browser-safe core.** `src/browser/` holds plain, self-contained JS (no imports, no TS syntax) used by both the CLI and the ia-qa.com web tool: `extract.js` (DOM walk), `match.js` (semantic matching + mapping diff), `contract.js` (Markdown page-contract renderer). The CLI feeds `extractInPage` to Playwright's `page.evaluate()` and the healer re-runs it live; ia-qa.com imports `extract.js` with `?raw` to show it as a copy-paste DevTools snippet and imports `match.js`/`contract.js` directly. Result: the zero-install web flow produces byte-identical mappings, verdicts and `.md` contracts to `ia-qa-heal` (verified on the same page). Keep everything in `src/browser/` dependency-free or all consumers break.
|
|
937
|
-
- Secrets: never written to disk. `env` reads `process.env`; `aws-ssm` calls `GetParameter` with decryption using the AWS SDK default credential chain (needs `ssm:GetParameter`, plus `kms:Decrypt` for SecureString).
|
|
938
|
-
- `createAiResolver` in `src/playwright/healer.ts` wires the BYOK AI resolver (`src/ai/resolver.ts`) into the `llmResolver` seam; `llmResolverStub` is kept as a no-op for backward compatibility. See [Optional AI add-on](#optional-ai-add-on--semantic-suggestions-ia-qa-heal-ai-byok).
|
|
939
|
-
- **Fix dry-run shows line numbers.** Each replacement in `fix --dry-run` reports the source lines where the rewrite applies (`3× #old → #new (lines 12, 47, 103)`), so you can jump straight to the affected locations without opening the file.
|
|
940
|
-
- **Phantom contract detection.** `saveMapping` warns when overwriting a previously captured mapping. `baseline` detects mapping files that don't match any page in `config.json` — left behind when a page is removed — and warns which files to delete.
|
|
309
|
+
Everything else is optional:
|
|
941
310
|
|
|
942
|
-
|
|
943
|
-
|
|
944
|
-
|
|
945
|
-
|
|
946
|
-
|
|
947
|
-
|
|
948
|
-
|
|
949
|
-
|
|
950
|
-
`map_app` verifies that the target URL shares the same origin as `config.json`'s `baseUrl`. This prevents an agent from being steered toward an internal-only endpoint reachable from the user's machine. Set `IAQA_ALLOW_ANY_URL=1` for multi-origin apps (SSO redirects, etc.). The check is skipped silently when no `config.json` exists (first `init` from an agent).
|
|
951
|
-
|
|
952
|
-
### Contract files are generated from live page content
|
|
311
|
+
| field | what it does |
|
|
312
|
+
|---|---|
|
|
313
|
+
| `session` | a Playwright `storageState` to reuse instead of logging in |
|
|
314
|
+
| `locale` | accessible names are language-dependent — set it if your app is not the system default |
|
|
315
|
+
| `volatile` | content that rotates by design (feed links, promos). Never enters a contract, so churn cannot read as drift |
|
|
316
|
+
| `nameMask` | labels carrying a counter or a clock (`"Cart (*)"`). Only the **label** stops being compared; role, selector and href still gate |
|
|
317
|
+
| `layouts` / `autoLayout` | shared shell extracted to `_layouts/`, diffed once instead of per page |
|
|
318
|
+
| `ai` | the BYOK add-on (see below) |
|
|
953
319
|
|
|
954
|
-
|
|
320
|
+
`--config <dir>` (or `IAQA_CONFIG_DIR`) points at another `.ia-qa/` — one per portal in a
|
|
321
|
+
monorepo. Every field is documented in full in the tutorial
|
|
322
|
+
(`TUTORIAL.md`, shipped in this package).
|
|
955
323
|
|
|
956
|
-
##
|
|
324
|
+
## The console — `ia-qa-heal ui`
|
|
957
325
|
|
|
958
|
-
|
|
326
|
+
A local web console for the rows where the tool gave up: highlighted crops of each candidate,
|
|
327
|
+
the `file:line` of every test that breaks, the verbs as plain-language buttons, and the history
|
|
328
|
+
as charts. `--shortcut` writes a desktop launcher.
|
|
959
329
|
|
|
960
|
-
|
|
330
|
+
It is **loopback only** (`127.0.0.1`), every request carries a per-process token, the Host
|
|
331
|
+
header is pinned, and its buttons spawn this package's own CLI with a fixed argv — never a
|
|
332
|
+
string from the request. It **refuses to start under CI** (an interactive surface there is a
|
|
333
|
+
hung build) and stops itself once the browser tab goes away.
|
|
961
334
|
|
|
962
|
-
##
|
|
335
|
+
## Optional AI add-on — `ia-qa-heal-ai` (BYOK)
|
|
963
336
|
|
|
964
|
-
|
|
337
|
+
Deterministic healing leaves `lost` and `ambiguous` rows behind on purpose. The add-on hands
|
|
338
|
+
*only those* to your own LLM and **suggests** a match with a confidence score:
|
|
965
339
|
|
|
966
340
|
```bash
|
|
967
|
-
|
|
968
|
-
|
|
969
|
-
|
|
970
|
-
|
|
971
|
-
|
|
341
|
+
npx ia-qa-heal-ai init # pick a provider, name the env var holding your key
|
|
342
|
+
npx ia-qa-heal-ai suggest # print suggestions
|
|
343
|
+
npx ia-qa-heal-ai suggest --apply # apply, after you confirm — refuses without a TTY
|
|
344
|
+
```
|
|
345
|
+
|
|
346
|
+
It never gates CI, and there is no `--ai` flag on `ia-qa-heal`: the verdict stays
|
|
347
|
+
deterministic by design. Your key stays in your environment; the prompt carries the contract
|
|
348
|
+
rows, nothing else.
|
|
349
|
+
|
|
350
|
+
## What leaves your machine
|
|
351
|
+
|
|
352
|
+
Nothing, unless you enable the BYOK add-on above.
|
|
353
|
+
|
|
354
|
+
**MCP server: path sandboxing.** The CLI is driven by a human in their own shell; the MCP
|
|
355
|
+
server receives arguments from an agent — potentially from third-party content via prompt
|
|
356
|
+
injection. Every path parameter (`out_dir`, `before_path`, `after_path`, `test_paths`) is
|
|
357
|
+
validated: the resolved absolute path must stay inside the project root.
|
|
358
|
+
`IAQA_ALLOW_OUTSIDE_PATHS=1` lifts it. `browser_path` is logged to stderr for auditability.
|
|
359
|
+
|
|
360
|
+
**MCP server: URL allowlist.** `map_app` verifies the target URL shares the origin of
|
|
361
|
+
`config.json`'s `baseUrl`, so an agent cannot be steered toward an internal-only endpoint
|
|
362
|
+
reachable from your machine. `IAQA_ALLOW_ANY_URL=1` for multi-origin apps.
|
|
363
|
+
|
|
364
|
+
**Contracts are generated from live page content.** The `.md` and `.json` files contain text
|
|
365
|
+
extracted from the DOM — accessible names, headings, link destinations. The Markdown renderer
|
|
366
|
+
escapes pipes, newlines and backticks to prevent table breakage and code-block injection. If
|
|
367
|
+
you map an untrusted page (third-party widget, ad, XSS'd content), treat the output as
|
|
368
|
+
untrusted before feeding it to an LLM: the tool does not detect or strip instruction-like text.
|
|
369
|
+
|
|
370
|
+
**`.ia-qa/session.json` is a secret** — live cookies. Gitignored on creation.
|
|
371
|
+
|
|
372
|
+
## Going deeper
|
|
373
|
+
|
|
374
|
+
- **The tutorial** — the whole thing, step by step, no jargon:
|
|
375
|
+
[ia-qa.com/devtools/selector-drift/tutorial](https://www.ia-qa.com/devtools/selector-drift/tutorial),
|
|
376
|
+
or `TUTORIAL.md` inside this package.
|
|
377
|
+
- **`ia-qa-heal skill`** — installs this package's agent instructions into
|
|
378
|
+
`<project>/.claude/skills/`, so an agent reads the contracts instead of opening a browser,
|
|
379
|
+
and knows the rules it must not break. Agents read skills from `.claude/skills/`, never from
|
|
380
|
+
`node_modules`, which is why this is a verb and not just a file in the tarball.
|
|
381
|
+
- **`ia-qa-heal-mcp`** — the stdio MCP server, for agents that prefer tool calls to a shell.
|
|
382
|
+
- **`SPEC-selector-resolution.md`** (shipped in the package) — how a Page Object's selectors are bound to the contract,
|
|
383
|
+
and why each refusal exists.
|
|
384
|
+
- **Peer dependencies** — `playwright` is a peer: your project provides it, so two installs
|
|
385
|
+
cannot drift to different browser revisions. `@aws-sdk/client-ssm` is an *optional* peer,
|
|
386
|
+
needed only by the `aws-ssm` secret source (~5 MB, imported lazily; the CLI tells you if you
|
|
387
|
+
pick it without).
|
|
388
|
+
- **Browser reuse** — it uses whatever Playwright browser is already on the machine before
|
|
389
|
+
asking you to download 150 MB. `IAQA_BROWSER_CHANNEL=chrome` (or `msedge`), or
|
|
390
|
+
`IAQA_BROWSER_PATH=/path/to/chrome`, to pin one.
|
|
391
|
+
|
|
392
|
+
MIT.
|