@ia-qa/self-healing 1.7.3 → 1.7.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +268 -830
- package/SPEC-selector-resolution.md +310 -0
- package/package.json +3 -2
package/README.md
CHANGED
|
@@ -1,498 +1,159 @@
|
|
|
1
1
|
# @ia-qa/self-healing
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Your E2E tests break because a selector moved, not because the app is wrong. This finds the
|
|
4
|
+
element again and rewrites the test.
|
|
4
5
|
|
|
5
|
-
|
|
6
|
+
A **CLI**, a **local MCP server** for AI agents, and an accessibility-tree page mapper.
|
|
7
|
+
Framework-agnostic — Playwright, Cypress, Selenium; JS/TS, Python, Java. The verdict is
|
|
8
|
+
deterministic: no model decides whether your build passes.
|
|
6
9
|
|
|
7
|
-
**It runs entirely on
|
|
10
|
+
**It runs entirely on your machine.** It brings its own headless Chromium, so it sees
|
|
11
|
+
`localhost`, staging, or an app behind a VPN, and **nothing ever leaves the machine**.
|
|
12
|
+
|
|
13
|
+
> **New here?** The [step-by-step tutorial](https://www.ia-qa.com/devtools/selector-drift/tutorial)
|
|
14
|
+
> walks the whole thing with no jargon (same text as `TUTORIAL.md`, shipped in this package).
|
|
15
|
+
> This README is the short reference.
|
|
8
16
|
|
|
9
17
|
## See it work
|
|
10
18
|
|
|
11
19
|
[](https://www.ia-qa.com/media/heal-demo.mp4)
|
|
12
20
|
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
## Easiest start — have your agent set it up
|
|
20
|
-
|
|
21
|
-
If you already work with Claude Code, Cursor, Copilot or another coding agent, don't configure this
|
|
22
|
-
by hand. Paste this to it:
|
|
23
|
-
|
|
24
|
-
```text
|
|
25
|
-
Install and set up @ia-qa/self-healing in this project.
|
|
26
|
-
1. First run `npx -y -p @ia-qa/self-healing ia-qa-heal skill --print` and follow it.
|
|
27
|
-
That output is the authoritative doc — do not guess any command or flag from memory.
|
|
28
|
-
2. Register the MCP server so you can drive it yourself:
|
|
29
|
-
command `npx`, args ["-y", "-p", "@ia-qa/self-healing", "ia-qa-heal-mcp"].
|
|
30
|
-
3. Read my e2e tests and my router, propose the page list for .ia-qa/config.json,
|
|
31
|
-
and wait for me to confirm before mapping anything.
|
|
32
|
-
4. Then explain to me, in five lines: what map / baseline / diff / fix each do,
|
|
33
|
-
and which locators you will never rewrite without asking me.
|
|
34
|
-
```
|
|
35
|
-
|
|
36
|
-
Step 1 matters more than it looks: this package is newer than most models' training data, so an agent
|
|
37
|
-
left to improvise will invent flags that don't exist. `skill --print` hands it the real instructions —
|
|
38
|
-
including [the four prohibitions](#working-with-an-ai-agent--ia-qa-heal-skill) — before it touches
|
|
39
|
-
anything. Step 4 is how you check it actually read them instead of guessing.
|
|
40
|
-
|
|
41
|
-
## Quick start — by hand
|
|
21
|
+
`ia-qa-heal run` runs your suite, watches it break, diffs the live app against your baseline,
|
|
22
|
+
rewrites the dead selectors, then **re-runs the suite to verify** and exits on that verdict.
|
|
23
|
+
Nothing is committed. By default a selector is replaced by the element's new selector, in the
|
|
24
|
+
style your suite already uses — CSS stays CSS. The demo adds `--locators`, which rewrites to
|
|
25
|
+
`getByRole(…, { exact: true })` where that is provably safe: a locator immune to the *next*
|
|
26
|
+
layout change.
|
|
42
27
|
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
npx -y -p @ia-qa/self-healing ia-qa-heal baseline # → .ia-qa/baseline/ — the reference. Commit it.
|
|
47
|
-
# …change the app, then `map` again…
|
|
48
|
-
npx -y -p @ia-qa/self-healing ia-qa-heal diff # baseline/ vs mapping/ → PASS / FIX / BLOCK
|
|
49
|
-
```
|
|
28
|
+
▶ **[The full 49-second version](https://www.ia-qa.com/media/heal-demo.mp4)** also covers what
|
|
29
|
+
deterministic healing *refuses* to guess, the MCP server answering an agent, and the HTML
|
|
30
|
+
report `--report` writes for a PR.
|
|
50
31
|
|
|
51
|
-
Install
|
|
32
|
+
## Install and run the loop
|
|
52
33
|
|
|
53
34
|
```bash
|
|
54
35
|
npm i -D @ia-qa/self-healing
|
|
55
|
-
npx ia-qa-heal init
|
|
36
|
+
npx ia-qa-heal init # interactive: writes .ia-qa/config.json
|
|
56
37
|
```
|
|
57
38
|
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
An agent that asks a browser where a button went is doing by hand what `.ia-qa/mapping/*.json` already answers offline: every interactive element's **role, accessible name and selector**, captured. A `link` the suite still calls a `button` shows up in one `grep`.
|
|
61
|
-
|
|
62
|
-
`ia-qa-heal skill` installs that reflex — plus the rules that must not be broken — as a skill your agent loads (this is what step 1 of the [paste-block above](#easiest-start--have-your-agent-set-it-up) makes it read):
|
|
39
|
+
Then, **in this order**:
|
|
63
40
|
|
|
64
41
|
```bash
|
|
65
|
-
ia-qa-heal
|
|
66
|
-
ia-qa-heal
|
|
67
|
-
ia-qa-heal
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
Installing is a separate step because agents read skills from `.claude/skills/`, never from `node_modules` — shipped alone, the file would never be loaded. The command is non-interactive, idempotent, prints the absolute path before writing, and refuses to overwrite a copy you have edited unless you pass `--force`.
|
|
74
|
-
|
|
75
|
-
## The whole flow — one tool, or all three together
|
|
76
|
-
|
|
77
|
-
This one package ships **three commands**. Use just the first, or chain all three — they read the same contract, so they never disagree.
|
|
78
|
-
|
|
42
|
+
npx ia-qa-heal ingest # inventory the selectors and names your tests use
|
|
43
|
+
npx ia-qa-heal map # capture the app, and bind your selectors to it
|
|
44
|
+
npx ia-qa-heal baseline # promote that capture to "this is how it should be"
|
|
45
|
+
# … someone changes the UI …
|
|
46
|
+
npx ia-qa-heal map
|
|
47
|
+
npx ia-qa-heal diff # PASS / FIX / BLOCK
|
|
48
|
+
npx ia-qa-heal fix --dry-run
|
|
49
|
+
npx ia-qa-heal fix
|
|
79
50
|
```
|
|
80
|
-
① ia-qa-heal deterministic core (humans & CI) — no AI, no key, nothing leaves your machine
|
|
81
|
-
init → (discover) → ingest → map → baseline → (app changes) → map → diff → fix
|
|
82
|
-
└ diff gives a PASS / FIX / BLOCK verdict you can gate CI on; fix applies only the safe rewrites
|
|
83
|
-
|
|
84
|
-
② ia-qa-heal-ai optional AI add-on (BYOK) — runs AFTER diff, only on what ① gave up on
|
|
85
|
-
init → suggest → (you review) → suggest --apply
|
|
86
|
-
└ suggests a match for lost/ambiguous rows; a suggestion you confirm, never a CI gate
|
|
87
|
-
|
|
88
|
-
③ ia-qa-heal-mcp the same engine for AI agents (Claude Code, Cursor, Copilot…)
|
|
89
|
-
└ tools: map_app · diff_mappings · fix_tests · suggest_heal (nothing leaves the machine)
|
|
90
|
-
```
|
|
91
|
-
|
|
92
|
-
**Pick your entry point:**
|
|
93
|
-
|
|
94
|
-
| You are… | Use |
|
|
95
|
-
|---|---|
|
|
96
|
-
| a human in a terminal | `ia-qa-heal` — add `ia-qa-heal-ai` only if you want AI suggestions for the leftovers |
|
|
97
|
-
| a CI pipeline | `ia-qa-heal diff` / `fix` — deterministic only, gate on the exit code |
|
|
98
|
-
| an AI agent | `ia-qa-heal-mcp` — drive it for the human, explain the PASS/FIX/BLOCK verdict |
|
|
99
|
-
|
|
100
|
-
The golden rule: **① is the judge.** ② and ③ never change what CI gates on — they only help a human decide on the rows ① deliberately refuses to guess.
|
|
101
|
-
|
|
102
|
-
## The one file you edit: `.ia-qa/config.json` (credentials, users, URLs)
|
|
103
51
|
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
"baseUrl": "https://staging.myapp.com", // your app's root URL
|
|
109
|
-
|
|
110
|
-
// 🔑 CREDENTIALS — you store a REFERENCE, never the actual value.
|
|
111
|
-
// The real secret lives in your .env or AWS SSM and is read at runtime.
|
|
112
|
-
// It never touches this file, and never leaves your machine.
|
|
113
|
-
"secrets": {
|
|
114
|
-
"user": { "source": "env", "key": "APP_USER" }, // → reads process.env.APP_USER
|
|
115
|
-
"pass": { "source": "env", "key": "APP_PASS" }
|
|
116
|
-
// AWS SSM instead of env: { "source": "aws-ssm", "key": "/staging/app/pass", "region": "eu-west-1" }
|
|
117
|
-
},
|
|
118
|
-
|
|
119
|
-
// 🔐 LOGIN — ONE login sequence. Points at the secrets above by name.
|
|
120
|
-
"auth": {
|
|
121
|
-
"loginUrl": "/login",
|
|
122
|
-
"usernameSelector": "#email",
|
|
123
|
-
"passwordSelector": "#password",
|
|
124
|
-
"submitSelector": "button[type=submit]",
|
|
125
|
-
"usernameSecret": "user", // ← the key from "secrets"
|
|
126
|
-
"passwordSecret": "pass",
|
|
127
|
-
"successSelector": "nav.dashboard" // optional: must appear after login (sanity check)
|
|
128
|
-
},
|
|
129
|
-
|
|
130
|
-
// 🌐 PAGES / URLs to map. One entry per route.
|
|
131
|
-
"pages": [
|
|
132
|
-
{ "name": "checkout", "url": "/checkout" },
|
|
133
|
-
// A view with NO url of its own (a tab/modal an SPA swaps in) — reach it by clicking:
|
|
134
|
-
{ "name": "billing", "url": "/account", "steps": [
|
|
135
|
-
{ "click": { "role": "tab", "name": "Billing" } }
|
|
136
|
-
]}
|
|
137
|
-
],
|
|
138
|
-
|
|
139
|
-
"locale": "en-US", // accessible names are language-dependent — set this if your app isn't the system default
|
|
140
|
-
"testPaths": ["tests/"], // where your test files live (for `fix` / `ingest`)
|
|
141
|
-
|
|
142
|
-
// 🌀 VOLATILE — content that rotates by design (feed links, article cards, live promos).
|
|
143
|
-
// It reads as interactive UI, so without this it enters the contract like a button
|
|
144
|
-
// does — then two captures of the same green page disagree about it and a handful
|
|
145
|
-
// of vanished feed items escalates into a false BLOCK. Matching elements never
|
|
146
|
-
// enter a contract (map and run capture both filter them).
|
|
147
|
-
// Glob per pattern (`*` = anything, case-insensitive, whole-string), optional field
|
|
148
|
-
// prefix href:/selector:/name:/role: — without one, tried against href, selector and name.
|
|
149
|
-
"volatile": ["href:*source=rss*", "href:https://blog.example.com/*"],
|
|
150
|
-
|
|
151
|
-
// 🎭 NAMEMASK — labels that carry a counter, a total or a clock ("Cart (3)",
|
|
152
|
-
// "🔌 Smithery 3.2k calls"). The *label* rotates, the element does not — so
|
|
153
|
-
// unlike `volatile`, the element stays fully under contract (role, selector and
|
|
154
|
-
// href still gate, a selector break on it still BLOCKs) and only the name stops
|
|
155
|
-
// being compared. Applied at comparison time: the contracts on disk keep the raw
|
|
156
|
-
// label, so adding or removing a mask never needs a re-map.
|
|
157
|
-
// Same glob dialect as `volatile`, no field prefix. A pattern with no stable text
|
|
158
|
-
// ("*") is refused at load. At runtime a mask is also dropped, loudly, if it would
|
|
159
|
-
// swallow a label your tests locate by name, or leave two elements of one role
|
|
160
|
-
// sharing a name.
|
|
161
|
-
"nameMask": ["Cart (*)", "🔌 Smithery * calls"],
|
|
162
|
-
|
|
163
|
-
"ai": { "…": "optional AI add-on — see below" }
|
|
164
|
-
}
|
|
165
|
-
```
|
|
52
|
+
`ingest` comes **first**, and it is not optional if your tests locate by CSS: it is what lets
|
|
53
|
+
`map` bind the strings your tests write to real elements. Re-run it when your test files
|
|
54
|
+
change — `map` says so when they have. Commit `.ia-qa/` (except `session.json`): the baseline
|
|
55
|
+
is what CI compares against.
|
|
166
56
|
|
|
167
|
-
|
|
57
|
+
Three binaries ship together: `ia-qa-heal` (the loop), `ia-qa-heal-mcp` (stdio MCP server for
|
|
58
|
+
agents), `ia-qa-heal-ai` (optional BYOK suggestions — never a gate).
|
|
168
59
|
|
|
169
|
-
|
|
170
|
-
- **"Can I test as several users (admin, then guest)?"** → `auth` holds **one** login. Two ways to do multi-user:
|
|
171
|
-
1. **Separate configs/runs** — one `.ia-qa/` per role, selected with `--config <dir>` (see **Monorepos** below), or
|
|
172
|
-
2. **Let your own test suite log in** — run `ia-qa-heal run` with your `testCommand`; it sets `IAQA_CAPTURE=1` and the contract is captured *during your suite*, so **whatever logins your tests already do (any number of users) are covered** — no `auth` block needed.
|
|
173
|
-
- **"My diff BLOCKs on links that just… rotate (a news feed, a blog widget)?"** → That is content churn, not UI drift: the feed's links entered the contract as if they were buttons, and the next capture caught the feed mid-rotation. Name what rotates in `"volatile"` (see above) and it never enters a contract again — the diff even prints a hint pointing here when a BLOCK is made of lost external links.
|
|
174
|
-
- **"The same rows come back as `renamed` every run — a cart counter, a call total, a clock."** → Only the *label* is moving, so only the label should stop being compared: put the pattern in `"nameMask"` (`"Cart (*)"`), not in `"volatile"`. `volatile` would delete the element from the contract — counter, button, href and coverage together — so a real selector break on it would go unnoticed forever; a mask keeps all of that gating and exempts the name alone. The diff prints the exact config line to paste when it spots the pattern. Two masks it refuses to apply, and says so: one that would hide a label your tests locate by name (that is a red suite behind a green gate), and one that would leave two elements of the same role wearing one name (a locator coin flip the mask itself created).
|
|
175
|
-
- **"How do I add a page behind a click, not a URL?"** → add `steps` to that page (see `billing` above): a list of `{ "click": { "role": "...", "name": "..." } }`. Named by role + accessible name, exactly like the contract — never a CSS selector.
|
|
176
|
-
- **"My app spans several domains behind a single sign-on (federated SSO) — does `map` cover it?"** → **Not with one `auth` block.** `auth` runs **one** login against **one** domain, before the first page. A federated IdP that re-challenges each project/domain with its own scope (`?projectName=…`, a different sub-domain per portal) is never satisfied by that single login — the second domain bounces back to a login the first one never covered, and the page maps empty (map now warns when an authenticated target returns 0 elements). Two ways through: **one `.ia-qa/` per domain/portal**, each selected with `--config <dir>` (see **Monorepos** below), or **`ia-qa-heal run` with `IAQA_CAPTURE=1`** so your own suite — which already handles the full SSO dance (cookies, redirects, per-project scope) — does the auth and the contract is captured during that real run. And since 1.7.0 there is a third, which is the general one: **`ia-qa-heal login`** — you log in once in a visible browser, whatever the app asks, and `map` reuses that session (see below). For a multi-portal app, capture-during-run is the intended path in CI; `map --auth` is for single-domain apps.
|
|
60
|
+
## Behind a login
|
|
177
61
|
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
`auth` fills a form: a username field, a password field, a submit button. That is *one shape of
|
|
181
|
-
login*, and plenty of apps do not have it — federated SSO, MFA, a consent screen, a magic link, a
|
|
182
|
-
device check. No list of selectors gets through any of those, so for those apps "add an `auth`
|
|
183
|
-
block" is not an answer at all.
|
|
184
|
-
|
|
185
|
-
```bash
|
|
186
|
-
ia-qa-heal login # opens a visible browser at your first configured page
|
|
187
|
-
# you log in there, however your app asks
|
|
188
|
-
ia-qa-heal map # reuses that session — no auth block needed
|
|
189
|
-
```
|
|
190
|
-
|
|
191
|
-
It models nothing about your login, which is exactly why it works anywhere: the human does
|
|
192
|
-
whatever the app requires, and the browser session is saved to `.ia-qa/session.json` in
|
|
193
|
-
Playwright's `storageState` shape. `map` and `discover --crawl` pick it up automatically and say
|
|
194
|
-
so; it takes precedence over `auth` when both exist (you ran `login` *because* the form fill
|
|
195
|
-
could not get through — re-running it silently would be the tool arguing with you).
|
|
196
|
-
|
|
197
|
-
- **That file is a secret.** It holds live cookies: whoever has it is logged in as you. It never
|
|
198
|
-
leaves your machine — nothing here uploads it — and `login` writes `.ia-qa/.gitignore` so a
|
|
199
|
-
commit cannot carry it.
|
|
200
|
-
- **It needs a person.** It refuses without a TTY and under CI, and an AI agent driving the CLI
|
|
201
|
-
cannot perform it: the agent hands the step to you by name.
|
|
202
|
-
- **It expires.** When the session dies, `map` hits the login wall again and says to re-run
|
|
203
|
-
`login` — it never silently maps a login form under the name of your dashboard.
|
|
204
|
-
- **It is not a CI mechanism.** In a pipeline, use `auth` (reproducible, secrets from env/SSM) or
|
|
205
|
-
capture-during-run (`IAQA_CAPTURE=1` + `testCommand`, your own suite logs itself in).
|
|
206
|
-
|
|
207
|
-
`--url <path>` opens a page other than the first one in `config.pages`.
|
|
208
|
-
|
|
209
|
-
#### You already have a session — use it
|
|
210
|
-
|
|
211
|
-
An authenticated Playwright suite usually writes a `storageState` in `globalSetup`. Point at
|
|
212
|
-
it and skip logging in a second time:
|
|
62
|
+
Most apps worth mapping are behind one, and only some logins are a form.
|
|
213
63
|
|
|
214
64
|
```bash
|
|
215
|
-
ia-qa-heal
|
|
216
|
-
|
|
65
|
+
npx ia-qa-heal login # opens a real browser; you log in however the app asks
|
|
66
|
+
npx ia-qa-heal map # reuses that session
|
|
217
67
|
```
|
|
218
68
|
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
Precedence: `--session` → `IAQA_SESSION` → `config.session` → the file `login` wrote. Relative
|
|
225
|
-
paths resolve from the project root, and `--session` works on `map`, `discover` and `login`
|
|
226
|
-
(there it says *where to write*, so a refresh updates the file you already point at).
|
|
227
|
-
|
|
228
|
-
**A session you named and that cannot be read stops the run** — it never falls through to
|
|
229
|
-
"no session". Silently ignoring it would capture the login page under your pages' names, which
|
|
230
|
-
is the one failure this tool treats as worse than stopping. Same for a file that is not a
|
|
231
|
-
storageState: it says so, instead of failing later inside the browser.
|
|
232
|
-
|
|
69
|
+
It models **nothing** about your login, which is why SSO, MFA, a consent screen and magic
|
|
70
|
+
links all work — you perform them. What is saved is the browser session
|
|
71
|
+
(`.ia-qa/session.json`): live cookies, so treat it like a password. It stays on your machine,
|
|
72
|
+
`login` writes `.ia-qa/.gitignore` so a commit cannot carry it, and when it expires `map` says
|
|
73
|
+
so instead of quietly mapping the login page under your dashboard's name.
|
|
233
74
|
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
By default everything lives in `.ia-qa/` under the current directory. A monorepo with several apps/portals keeps **one `.ia-qa/` per portal** and points the CLI at each:
|
|
75
|
+
**Already have a session?** An authenticated Playwright suite usually writes a `storageState`
|
|
76
|
+
in `globalSetup` — point at it instead of logging in twice:
|
|
237
77
|
|
|
238
78
|
```bash
|
|
239
|
-
ia-qa-heal map
|
|
240
|
-
ia-qa-heal diff --config apps/xsp # baseline/ vs mapping/ under apps/xsp/.ia-qa
|
|
241
|
-
ia-qa-heal map --config apps/xcp # a different portal, its own contract
|
|
79
|
+
npx ia-qa-heal map --session playwright/.auth/user.json
|
|
242
80
|
```
|
|
243
81
|
|
|
244
|
-
`--
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
> Deliberately **not** "profiles inside one config.json": mapping/, baseline/ and _layouts/ are per-directory on disk, and one directory per portal is what keeps a contract stage-able, review-able and revert-able as a single unit in git.
|
|
249
|
-
|
|
250
|
-
> **🤖 For AI agents helping a human set this up:** the config is the human's *secret zone* — treat it with care.
|
|
251
|
-
> - **Never put a real secret value in `secrets`** — only `{ "source": "env", "key": "THE_VAR_NAME" }`. If the human pastes a key, tell them to `export` it and reference the var name instead.
|
|
252
|
-
> - In a **non-interactive shell you can't run the `init` wizard** — write `.ia-qa/config.json` directly with the shape above. `baseUrl` and a non-empty `pages[]` (each with a non-empty `name`) are required; `auth` only if the app needs login.
|
|
253
|
-
> - **Behind a login and no credentials to wire?** Prefer the `IAQA_CAPTURE=1` + `ia-qa-heal run` path so the human's own suite handles auth — you never touch their passwords.
|
|
254
|
-
> - A view that shares a URL with others is reachable **by name only** (`steps`), not by URL — say so rather than mapping the wrong view.
|
|
82
|
+
Precedence: `--session` → `IAQA_SESSION` → `"session"` in config → what `login` wrote. A
|
|
83
|
+
session you *named* and that cannot be read stops the run; it never falls through to "no
|
|
84
|
+
session", which would capture the login page under your pages' names.
|
|
255
85
|
|
|
256
|
-
|
|
86
|
+
`login` needs a terminal and refuses under CI. There, use `auth` (a form fill, secrets from
|
|
87
|
+
env/SSM — see **Configuration**) or capture-during-run, where your own suite logs itself in.
|
|
257
88
|
|
|
258
|
-
|
|
89
|
+
## Which verb answers which question
|
|
259
90
|
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
```bash
|
|
274
|
-
npx ia-qa-heal discover # default: read sitemap.xml (pure HTTP, no browser)
|
|
275
|
-
npx ia-qa-heal discover --sitemap https://…/custom.xml # an explicit sitemap
|
|
276
|
-
npx ia-qa-heal discover --crawl # follow links from inside the logged-in app
|
|
277
|
-
npx ia-qa-heal discover --crawl --apply # …and write the new pages to config.json
|
|
278
|
-
```
|
|
279
|
-
|
|
280
|
-
- **`--sitemap` (the default, zero-risk).** A single GET on `<baseUrl>/sitemap.xml` — falling back to the `Sitemap:` lines in `/robots.txt`, and following a sitemap *index* one level into its children. No browser, no login, nothing touched. Its blind spot is the mirror of the crawler's: a sitemap is the **public, SEO** surface and stops exactly where the authenticated app begins.
|
|
281
|
-
|
|
282
|
-
It leads with what you'll map — `Found 140 pages.` — never a bare "0 URLs". **`www.` and the apex are treated as one site**, so an apex `baseUrl` (`https://ia-qa.com`) against a `www.` sitemap (or the reverse, or a `www.`→apex 301) just works; the kept URLs are stored as **relative paths**, so `map` loads them under whatever host `baseUrl` names. The only time it mentions a skip is the one you can act on: when URLs were dropped because they sit on a **different host**, it names that host and how to reconcile it — so the old silent "0 URLs" over an apex/`www.` mismatch can't happen. Pass **`--strict-host`** to require an exact host match instead.
|
|
283
|
-
|
|
284
|
-
- **`--crawl` (higher reach, still ultra-safe).** Starts *inside* the app — reusing the very `auth` login `map` uses — and follows links outward, so it finds the authenticated pages a sitemap never lists. It is built to be side-effect-free:
|
|
285
|
-
- **same-origin GET navigations only**, and it **never visits a URL that reads like an action** — `/logout` (which would end the session for every page after it), `/delete`, `/pay`, `/order`, `/unsubscribe`… A GET that mutates is bad REST but real, so the guard is a denylist on the URL itself.
|
|
286
|
-
- to find links hidden behind a menu, dropdown, or dialog, it **opens** them — and does nothing else. Only pure **disclosure toggles** are clicked (an `aria-haspopup`, a collapsed `aria-expanded="false"`, a `<summary>`); never a link, never a submit, never a control whose name reads like an action, never one already open. If a click unexpectedly navigates, that was not a disclosure — it records where it went and steps back. After reading, it presses `Escape` to close. It never clicks the revealed items; it only harvests their `href`s for the crawl. Reveal goes **one level deep** — open it and that's all. Turn it off entirely with **`--no-reveal`** (crawl links only).
|
|
287
|
-
|
|
288
|
-
```
|
|
289
|
-
--depth <n> crawl link-depth from the start pages (default 2)
|
|
290
|
-
--max <n> cap candidates per source
|
|
291
|
-
--strict-host require an exact host match (default: www./apex are one site)
|
|
292
|
-
```
|
|
293
|
-
|
|
294
|
-
Both sources can run together (`discover --sitemap --crawl`) — the sitemap hits seed the crawl frontier. Pages already in `config.pages` count as covered (not re-listed); a newly discovered page whose name would collide with an existing one gets a numeric suffix (a page name is a contract's filename — a collision would overwrite it). It is idempotent: once you have added the uncovered pages, a re-run reports full coverage.
|
|
295
|
-
|
|
296
|
-
```
|
|
297
|
-
🗺 Coverage — 6 of 8 discovered pages in config.pages · 2 not yet covered:
|
|
298
|
-
|
|
299
|
-
name url (source)
|
|
300
|
-
dashboard /dashboard (crawl)
|
|
301
|
-
settings /settings (crawl)
|
|
302
|
-
Run again with --apply to add the uncovered pages to config.pages.
|
|
303
|
-
```
|
|
304
|
-
|
|
305
|
-
Then `ia-qa-heal map` captures their contracts as usual. Discovery never maps, heals, or edits a test — it only ever edits `config.json`, and only with `--apply`.
|
|
91
|
+
| Your question | Verb |
|
|
92
|
+
|---|---|
|
|
93
|
+
| my tests broke — fix them | `run` (capture → diff → confirm → fix → re-run to verify) |
|
|
94
|
+
| did anything drift? | `diff` → PASS/FIX/BLOCK, the CI gate |
|
|
95
|
+
| **my suite already ran with `IAQA_CAPTURE=1`** | **`diff`, on its own** — merges the capture, no browser, no second run |
|
|
96
|
+
| day one, no baseline — is my suite still valid? | `audit` (advisory; never gate on it) |
|
|
97
|
+
| is the app itself sound? | `check` — dead links, unnamed elements, name collisions, orphan pages |
|
|
98
|
+
| which pages am I *not* testing? | `discover` (`--sitemap`, or `--crawl` behind the login) |
|
|
99
|
+
| what do my tests actually use? | `ingest` |
|
|
100
|
+
| show me the app's structure | `graph --format mermaid\|svg\|json\|markdown` |
|
|
101
|
+
| is this getting better or worse? | `history` |
|
|
102
|
+
| let me look and decide myself | `ui` — a local console |
|
|
306
103
|
|
|
307
|
-
|
|
104
|
+
`ia-qa-heal <verb> --help` for flags. On a slow suite, note that `run` runs it **twice** (once
|
|
105
|
+
to capture, once to verify the fix): `run --no-verify` keeps only the first, and
|
|
106
|
+
`IAQA_CAPTURE=1` on the run you already do in CI plus `ia-qa-heal diff` costs nothing extra.
|
|
308
107
|
|
|
309
|
-
|
|
108
|
+
## The verdict
|
|
310
109
|
|
|
311
|
-
```bash
|
|
312
|
-
npx ia-qa-heal ui
|
|
313
110
|
```
|
|
314
|
-
|
|
315
|
-
…or, for someone who should never have to open a terminal at all:
|
|
316
|
-
|
|
317
|
-
```bash
|
|
318
|
-
npx ia-qa-heal ui --shortcut # once per project
|
|
111
|
+
🔧 FIX 2 ok · 1 renamed · 3 healable · 0 ambiguous · 0 lost
|
|
319
112
|
```
|
|
320
113
|
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
- **The rows waiting on a human** — every `ambiguous`, `lost` and `rebound` row, each with a **highlighted crop** of the candidates as they look on the page right now (needs `map --screenshots`). When five identical "Copy" buttons exist, the row's selector tells you nothing; the picture tells you which one. Alongside them, the `file:line` of every test that will break.
|
|
326
|
-
- **The verbs as buttons** — capture, ingest, diff, audit, check, preview/apply the rewrites — with the output streamed back.
|
|
327
|
-
- **The run history as charts** — verdicts over time, mean coverage, and which pages actually churn.
|
|
114
|
+
- **PASS** — nothing moved.
|
|
115
|
+
- **FIX** — every break has a deterministic old→new rewrite. `fix` applies it.
|
|
116
|
+
- **BLOCK** — a human decides. Nothing is rewritten.
|
|
328
117
|
|
|
329
|
-
**
|
|
118
|
+
What `fix` **refuses** is the point of the tool:
|
|
330
119
|
|
|
331
120
|
| | |
|
|
332
121
|
|---|---|
|
|
333
|
-
|
|
|
334
|
-
|
|
|
335
|
-
|
|
|
336
|
-
|
|
|
337
|
-
|
|
338
|
-
**It draws the findings; it does not quote them.** The console runs `check --json` and `audit --json` and renders the report — broken links with the pages that link to them, name collisions with their role and count, unnamed elements with their selector, each card carrying the sentence that says why it matters, in a full-width panel. It used to paste the verb's terminal output into a black `<pre>`: aligned monospace, folded behind "Technical detail", in the one surface whose premise is that its reader does not want a terminal. The raw output is still there, one click away, for whoever wants it. The network announcement survives the change — the panel states the host and the number of links requested, because a flow that leaves the machine says so whatever renders it.
|
|
339
|
-
|
|
340
|
-
**A decision it states, it offers.** A result that ends *"re-run with a lower `--concurrency`"* or *"use Repair when the plan looks right"* is a terminal instruction printed to the one person who is not in a terminal. Those sentences are buttons now, inside the result card you are already reading: a check that could not verify some links offers **Verify those links again, slowly** and **Check the app without requesting links**; a diff with repairs offers the preview, and the preview offers the repair; an audit finding offers **Check for changes** and **Look at the app again** — and never a repair, because `audit` has no baseline to rewrite from and is advisory by design. Each verb also names its own verdict: `audit` never says "Repairs are ready", because it does not repair. The slower rate is a *frozen argv in the allowlist*, not a number field the page posts — a discrete choice keeps "no new authority" literally true, where a settings field would have traded the invariant for one option.
|
|
341
|
-
|
|
342
|
-
Nothing is uploaded, there is no telemetry and no external asset — the page is inline HTML/CSS/JS and every byte it shows came off your disk. It refuses to start under CI, where an interactive surface would just hang; use `--json`, `--junit` or the exit code there.
|
|
343
|
-
|
|
344
|
-
**One request does leave, and it is stated rather than buried:** at startup, at most once a day, `ui` asks `registry.npmjs.org` whether a newer version of this package exists, so the banner can tell you. The desktop shortcut launches a fixed path on disk, and the person clicking it does not own a terminal — this is the only way they would ever find out. It sends a request for a public package's version number: nothing about your app, your tests, your contracts or your results, no telemetry, no identifier. The answer is cached in `~/.ia-qa/update-check.json`, the terminal names the host before the request happens, and **`ia-qa-heal ui --offline` removes it entirely**.
|
|
345
|
-
|
|
346
|
-
The banner stops at the command to paste. It installs nothing, and there is no button that would: the console's authority is exactly the CLI's, and `npm install` is a different binary that runs whatever lifecycle scripts the registry serves — wiring that to a button would widen a leaked token from "edits my test files" to "runs arbitrary code as me". If you started through `npx`, it says so and tells you there is nothing to install.
|
|
347
|
-
|
|
348
|
-
### Several projects — the switcher
|
|
349
|
-
|
|
350
|
-
The header carries a **Project** dropdown as soon as there is more than one. It is governed by a single rule:
|
|
351
|
-
|
|
352
|
-
> **A project is reachable only if you opened a console in it.**
|
|
353
|
-
|
|
354
|
-
Nothing scans your disk. `~/.ia-qa/projects.json` is a consent list, and there are exactly two ways to consent: run `ia-qa-heal ui` in a folder, or pick **＋ Add another project…** in the dropdown and give the path. Switching between listed projects resolves the requested directory against that list *exactly* — a folder outside it cannot be named. Entries whose project has been deleted are dropped rather than offered, and actions are handed `--config <active project>` explicitly rather than inheriting anything.
|
|
355
|
-
|
|
356
|
-
Adding from the console is a deliberate action by whoever holds the session token — the same token that already authorizes *Repair my test files*. What the list defends against is a **drive-by**: another page in your browser reaching localhost. That is stopped by the token and the `Host` check, not by making you walk to a terminal.
|
|
357
|
-
|
|
358
|
-
```bash
|
|
359
|
-
ia-qa-heal ui --forget # clear the whole list
|
|
360
|
-
ia-qa-heal ui --forget ../shop # drop one project
|
|
361
|
-
```
|
|
362
|
-
|
|
363
|
-
Forgetting only removes a folder from the switcher — it deletes nothing in the project, and opening a console there again re-adds it. `IAQA_HOME` relocates the list (useful on a shared machine).
|
|
364
|
-
|
|
365
|
-
## Visualize the app — `ia-qa-heal graph`
|
|
366
|
-
|
|
367
|
-
Turn the mappings you already have into a **navigation graph** (page → page, derived from the `href` on each page's links — the same engine `map` uses for `_overview.md`). Read-only: it never rewrites your mapping files, it just renders them.
|
|
368
|
-
|
|
369
|
-
```bash
|
|
370
|
-
ia-qa-heal graph # Mermaid on stdout — paste into a PR comment or mermaid.live
|
|
371
|
-
ia-qa-heal graph --format json # ia-qa-graph@1 intermediate: { nodes[], edges[], meta }
|
|
372
|
-
ia-qa-heal graph --format markdown --out nav-kb.md # one card per page — a RAG chunk / Confluence page
|
|
373
|
-
ia-qa-heal graph --format svg --out nav.svg # the same SVG map writes as _navigation.svg
|
|
374
|
-
ia-qa-heal graph --format json | your-agent # hand the graph to an agent to analyze
|
|
375
|
-
```
|
|
376
|
-
|
|
377
|
-
`--format` is one of `mermaid` (default) · `svg` · `json` · `markdown`; `--out <file>` writes to a file instead of stdout; `--open` opens it. `mermaid` and `svg` are the same views `map` already writes to `_navigation.md`/`_navigation.svg` — offered here for stdout/pipe.
|
|
378
|
-
|
|
379
|
-
The **`json`** form is the portable `ia-qa-graph@1` IR — nodes carry `{ url, elementCount, entry, isolated }`, and `meta` carries the coverage gap (`unmapped`), collapsed hubs (`fans`), any `mesh`, and `entries`. It matches the node/edge shape the ia-qa.com Graph tool consumes, so it drops into other viewers (D3, Reagraph) without conversion.
|
|
380
|
-
|
|
381
|
-
The **`markdown`** form linearizes the graph into one section per page — *what it is, where it links, what reaches it* — so a page is a self-contained chunk you can index in a **RAG knowledge base** (split on the `##` headings) or push to **Confluence** (`create_confluence_page` on the ia-qa MCP). It carries the navigation *relations* the per-page `map` contracts don't, and ends with a **Coverage gaps** list of linked-but-unmapped destinations.
|
|
382
|
-
|
|
383
|
-
## The one-verb loop: `ia-qa-heal run`
|
|
384
|
-
|
|
385
|
-
`run` chains the whole loop and stops where a human belongs:
|
|
386
|
-
|
|
387
|
-
```
|
|
388
|
-
capture → ingest → diff → dry-run of the fixes → your confirmation → fix → re-run to verify
|
|
389
|
-
```
|
|
390
|
-
|
|
391
|
-
- If `config.testCommand` is set (e.g. `npx playwright test`), `run` spawns it with `IAQA_CAPTURE=1` and the mapping is **captured during your own suite** — login, modals, wizards included, with the credentials your suite already manages. No `testCommand` → falls back to `map`.
|
|
392
|
-
- `BLOCK` (lost / ambiguous / rebound) stops for your judgment. Healable drift is shown as a dry run first; `--yes` skips the confirmation for CI. Nothing is ever committed.
|
|
393
|
-
- **After the fix, `run` re-runs your suite once more (without `IAQA_CAPTURE`) to confirm the rewrite is actually green** — a selector fix is only worth anything if the tests pass after it. The exit code follows that re-run: `0` if the suite passes, `1` if it still fails (the rewrite landed but something else is red). `--no-verify` applies the fix and skips the re-run; the re-run is also skipped, with a reason, when nothing was rewritten or there is no `testCommand`.
|
|
394
|
-
- With `--report`, the HTML for a `run` additionally carries the **applied** `old → new` edits (file and line numbers) and the verify verdict — the file-level before/after the `diff` report only promised.
|
|
395
|
-
|
|
396
|
-
### Already ran your suite with `IAQA_CAPTURE=1`? Just `diff`
|
|
397
|
-
|
|
398
|
-
`run` is a convenience, not the only door. If the capture already happened — your suite ran with
|
|
399
|
-
`IAQA_CAPTURE=1`, in CI or by hand — the verdict is one offline command away:
|
|
400
|
-
|
|
401
|
-
```bash
|
|
402
|
-
IAQA_CAPTURE=1 npx playwright test # your suite, as usual
|
|
403
|
-
ia-qa-heal diff # merges .capture/ and compares — no browser, no re-run
|
|
404
|
-
```
|
|
405
|
-
|
|
406
|
-
`diff`, `fix` and `baseline` all merge the staging directory (`.ia-qa/mapping/.capture/`) on the
|
|
407
|
-
way in and say so. **Do not run `map` to "finish" a capture**: `map` opens its own browser and
|
|
408
|
-
starts over, without whatever session your suite had.
|
|
409
|
-
|
|
410
|
-
And on a slow suite, `run` runs it **twice** — once to capture, once to verify the fix. `run
|
|
411
|
-
--no-verify` keeps the first only; the exit code then reflects the diff rather than a re-proved
|
|
412
|
-
green suite.
|
|
122
|
+
| `ambiguous` | several candidates — any choice is a coin flip |
|
|
123
|
+
| `lost` | the element is gone; there is nothing to rewrite towards |
|
|
124
|
+
| `rebound` | the selector now finds a *different* element — the test passes and acts on the wrong thing |
|
|
125
|
+
| `unattributable` | `getByText`, `cy.contains`, `getByTitle`… name a **string, not an element**. Nothing proves the test meant the renamed button rather than a heading that never moved, so editing one would break a *passing* test |
|
|
413
126
|
|
|
414
|
-
|
|
127
|
+
`--locators` rewrites to `getByRole(role, { name })` instead of a fresher CSS path, where that
|
|
128
|
+
is provably safe — a selector that survives the next layout change. Without it, a CSS selector
|
|
129
|
+
is replaced by a CSS selector: the tool does not impose a locator style on your suite.
|
|
415
130
|
|
|
416
|
-
|
|
131
|
+
### At runtime, if you want it — `aiClick` / `aiFill`
|
|
417
132
|
|
|
418
133
|
```ts
|
|
419
|
-
import {
|
|
420
|
-
|
|
421
|
-
|
|
422
|
-
Inert unless `IAQA_CAPTURE=1` — a normal run pays nothing. When armed, each worker records the pages it visits (per navigation, debounced, plus the final state of each test) into `.ia-qa/mapping/.capture/`; the next CLI verb merges the shards into regular page contracts, named from `config.pages` so they pair with your baseline. Views reached through `steps` share their URL and are skipped — `map` remains their path.
|
|
423
|
-
|
|
424
|
-
### Capture during your test run — Cypress (`@ia-qa/self-healing/cypress`)
|
|
425
|
-
|
|
426
|
-
Same idea, one import in `cypress/support/e2e.js`:
|
|
427
|
-
|
|
428
|
-
```js
|
|
429
|
-
import '@ia-qa/self-healing/cypress';
|
|
430
|
-
```
|
|
431
|
-
|
|
432
|
-
Inert unless `IAQA_CAPTURE=1` (set it as a real env var, or forward it in `cypress.config.js` → `env: { IAQA_CAPTURE: process.env.IAQA_CAPTURE }`). When armed, it records the interactive elements of the page **each test ends on** — post-login, modals, wizards, whatever your suite already walks — and writes them to `.ia-qa/mapping/.capture/` via `cy.writeFile`. **Nothing to wire in `setupNodeEvents`.** The next CLI verb (`diff` / `run` / `baseline`) merges the shards into the same page contracts as `map` and the Playwright capture — byte-compatible.
|
|
433
|
-
|
|
434
|
-
**This is the intended path for a multi-domain / federated-SSO app.** `map` logs in once against one domain; a suite that already authenticates across several portals reaches states `map` structurally cannot. Capture rides that real run, so **whatever logins your tests already do — UI form, API token injection, `cy.session`, any number of domains — are covered, with no `auth` block and without self-healing ever touching your credentials.**
|
|
435
|
-
|
|
436
|
-
Details and limits (same as the Playwright path, plus a couple Cypress-specific ones):
|
|
437
|
-
- Coverage is whatever the suite visits — it is not a crawler. To get a page's contract, have a test that **ends** on it (v1 captures the final state of each test, not every intermediate navigation).
|
|
438
|
-
- Only the app-under-test's top document is read (iframes/shadow DOM excluded), and the extractor runs in the AUT realm via `eval` — an app whose CSP forbids `unsafe-eval` will silently capture nothing (the capture never fails your suite).
|
|
439
|
-
- Parallel runners (`cypress-parallel`, Cypress Cloud) are fine: each worker writes its own uniquely-named shard and the merge unions them.
|
|
440
|
-
- If a `.ia-qa/config.json` exists, contracts are named from `config.pages` (so they pair with your baseline) and `steps`-only views are skipped; with no config, the page name falls back to a slug of the URL.
|
|
441
|
-
|
|
442
|
-
### Capture during your test run — Selenium, JS (`@ia-qa/self-healing/selenium`)
|
|
443
|
-
|
|
444
|
-
A `selenium-webdriver` (Node) test runs in Node, so — unlike Cypress — there is nothing to hook and no `cy.writeFile`: you call it yourself after a navigation, and it writes the shard directly.
|
|
445
|
-
|
|
446
|
-
```js
|
|
447
|
-
const { capture } = require('@ia-qa/self-healing/selenium');
|
|
448
|
-
|
|
449
|
-
await driver.get('https://app.example.com/checkout');
|
|
450
|
-
await capture(driver); // inert unless IAQA_CAPTURE=1
|
|
451
|
-
```
|
|
452
|
-
|
|
453
|
-
Inert without `IAQA_CAPTURE=1`, so the calls can live in the suite permanently. It injects the extractor into the live page via `driver.executeScript` (the Selenium twin of `page.evaluate`), names the page (config-aware, `steps`-views skipped), unions repeats, and writes `.ia-qa/mapping/.capture/sel-<worker>.json` — the same shard the merge already reads. Parallel workers each write their own file. Errors are swallowed; the capture never fails your test. Honors `IAQA_CONFIG_DIR` for monorepos, like every other path.
|
|
454
|
-
|
|
455
|
-
### Any other framework / language — write a shard
|
|
456
|
-
|
|
457
|
-
The shard format **is** the interface; the Playwright/Cypress/Selenium helpers are just sugar over it. Non-JS Selenium (Python, Java, C#…), WebdriverIO, or a home-grown harness can feed capture directly: run the extractor in the page (it ships as the copy-paste DevTools snippet on ia-qa.com, and as `extract.js` in this package), then drop a JSON file into `.ia-qa/mapping/.capture/` shaped like:
|
|
458
|
-
|
|
459
|
-
```json
|
|
460
|
-
{ "pages": [ { "page": "checkout", "url": "/checkout",
|
|
461
|
-
"elements": [ { "role": "button", "name": "Pay", "selector": "button#pay" } ] } ] }
|
|
134
|
+
import { aiClick } from '@ia-qa/self-healing';
|
|
135
|
+
await aiClick(page, 'button#login');
|
|
462
136
|
```
|
|
463
137
|
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
npx ia-qa-heal ingest tests/ # or no args: reads config.testPaths
|
|
470
|
-
```
|
|
471
|
-
|
|
472
|
-
Statically scans your tests/POMs → `.ia-qa/usage.json`. Two halves, because suites are written two ways:
|
|
473
|
-
|
|
474
|
-
- **Selectors** — Playwright, Cypress, Selenium call shapes + page-object declarations (`loginButton = '#login .cta'`, any name, any language). `diff` annotates every drifted selector with *used N× in M test files*, and `fix` no longer needs the test paths on the command line.
|
|
475
|
-
- **Names** — the `get|find|queryBy…` family (Playwright, Testing Library, `@testing-library/cypress`): `ByRole`, `ByLabel(Text)`, `ByPlaceholder(Text)`, `ByText`, `ByTitle`, `ByAltText` — plus `cy.contains(…)` (both forms) and Selenium's `By.linkText` / `By.LINK_TEXT` / `By.PARTIAL_LINK_TEXT`. These are what makes the rename gate below possible.
|
|
476
|
-
|
|
477
|
-
Purely static: no execution, no network. A locator built by concatenation or holding `${…}` is invisible here — and invisible to `fix` too, which is why it is skipped rather than guessed at.
|
|
138
|
+
If the normal action times out, these read the page contract, re-scan the live page, retry on
|
|
139
|
+
the element that semantically matches, and log a loud warning that the test needs updating.
|
|
140
|
+
Useful to keep a suite green while you triage — but it is a **stopgap, not the loop**: healing
|
|
141
|
+
in memory hides the drift instead of putting it in a pull request. The verbs above edit your
|
|
142
|
+
files so a human reviews the change.
|
|
478
143
|
|
|
479
|
-
|
|
144
|
+
## Page Objects written in CSS
|
|
480
145
|
|
|
481
|
-
Your Page Object says `.btn-primary`. The contract says `#pay-now`. Both name the same
|
|
482
|
-
|
|
483
|
-
technically true,
|
|
484
|
-
nothing out of a diff.
|
|
146
|
+
Your Page Object says `.btn-primary`. The contract says `#pay-now`. Both name the same button,
|
|
147
|
+
and until they are **bound** every drift row reads *not referenced by your tests* —
|
|
148
|
+
technically true, useless in practice.
|
|
485
149
|
|
|
486
|
-
So `map` binds them
|
|
487
|
-
|
|
488
|
-
|
|
489
|
-
and `diff` compares the two moments.
|
|
490
|
-
|
|
491
|
-
What that buys you — an example that no other check in this package can catch:
|
|
150
|
+
So `map` binds them (with `usage.json` present — run `ingest` first) and records the result in
|
|
151
|
+
`.ia-qa/mapping/_resolved/`. What that buys, on a diff where the element contract did not move
|
|
152
|
+
at all:
|
|
492
153
|
|
|
493
154
|
```
|
|
494
155
|
🔧 FIX
|
|
495
|
-
2 ok · 0 renamed · 0 healable · 0 lost
|
|
156
|
+
2 ok · 0 renamed · 0 healable · 0 lost ← the contract is spotless
|
|
496
157
|
|
|
497
158
|
🔗 1 selector your tests write no longer reaches what it used to:
|
|
498
159
|
🔧 .btn-primary → #pay-now
|
|
@@ -500,38 +161,32 @@ What that buys you — an example that no other check in this package can catch:
|
|
|
500
161
|
tests/checkout.spec.js:1, tests/checkout.spec.js:4
|
|
501
162
|
```
|
|
502
163
|
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
Four outcomes, and only one of them is repaired automatically:
|
|
164
|
+
Somebody renamed a CSS class. The element diff is clean, the suite is broken, and `fix`
|
|
165
|
+
rewrites the literal in your files. Four outcomes, one repaired automatically:
|
|
507
166
|
|
|
508
167
|
| | |
|
|
509
168
|
|---|---|
|
|
510
|
-
|
|
|
511
|
-
|
|
|
512
|
-
|
|
|
513
|
-
|
|
|
514
|
-
|
|
515
|
-
**
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
But when it *does* carry one — a `data-testid`, an `id`, an `aria-label` — the repair is
|
|
520
|
-
exact:
|
|
169
|
+
| reaches **nothing**, and its element is still there | 🔧 rewritten |
|
|
170
|
+
| reaches a **different** element | ⛔ never rewritten — your test passes and acts on the wrong thing |
|
|
171
|
+
| reaches **several** elements | ⛔ a coin flip |
|
|
172
|
+
| its element is **gone** | ⛔ nothing to rewrite towards |
|
|
173
|
+
|
|
174
|
+
**Your app may already carry the answer.** The contract holds interactive elements only, so a
|
|
175
|
+
selector on a status div, a cell or an alert is not in it. Usually that stays quiet — such an
|
|
176
|
+
element has no identity independent of what drifts, so nothing could be repaired anyway. But
|
|
177
|
+
when it *does* carry one (`data-testid`, `id`, `aria-label`), the repair is exact:
|
|
521
178
|
|
|
522
179
|
```
|
|
523
180
|
🔧 .infoBlockContainer > .alert → [data-testid="cart-alert"]
|
|
524
|
-
the element is still there, only its address moved.
|
|
525
|
-
pom/CartPage.ts:2, pom/CartPage.ts:4
|
|
526
181
|
```
|
|
527
182
|
|
|
528
|
-
That is the common shape of a legacy suite: the front end added test ids years after the
|
|
529
|
-
|
|
530
|
-
|
|
531
|
-
|
|
532
|
-
**How much of your suite is in reach?** `map`
|
|
533
|
-
|
|
534
|
-
|
|
183
|
+
That is the common shape of a legacy suite: the front end added test ids years after the Page
|
|
184
|
+
Objects were written and nobody propagated them. It cannot be noisy — an element with a test
|
|
185
|
+
id has a stable identity by construction, and an anchor matching two elements is refused.
|
|
186
|
+
|
|
187
|
+
**How much of your suite is in reach?** `map` says so at the end, over the union of the pages
|
|
188
|
+
it just mapped — the number nobody can add up by hand, because the same string appears on
|
|
189
|
+
several pages:
|
|
535
190
|
|
|
536
191
|
```
|
|
537
192
|
🔗 Your 181 inventoried selectors, against the pages just mapped:
|
|
@@ -540,299 +195,63 @@ hand (the same string appears on several pages):
|
|
|
540
195
|
95 not seen on any page mapped this run — which is about your coverage, not your tests
|
|
541
196
|
```
|
|
542
197
|
|
|
543
|
-
Read the composition, not the ratio. `working` selectors are **fine
|
|
544
|
-
|
|
545
|
-
|
|
546
|
-
|
|
547
|
-
|
|
548
|
-
No `usage.json` ⇒ no bindings ⇒ nothing changes. Same rule as name drift: **no inventory,
|
|
549
|
-
no escalation.** Today the bindings are captured by `map`; capture-during-run is specified
|
|
550
|
-
in `SPEC-selector-resolution.md` and not built yet.
|
|
551
|
-
|
|
552
|
-
### The rename your suite actually depends on
|
|
553
|
-
|
|
554
|
-
A relabelled element is classified `renamed`: the selector still resolves, the click still lands on the right thing. **For a CSS-anchored suite that is harmless. For a name-anchored one it is fatal** — `getByRole('button', { name: 'Save' })` stops matching the moment the button says "Enregistrer", and a selector-only diff would exit 0 while your suite goes red.
|
|
555
|
-
|
|
556
|
-
Once `ingest` has run, `diff` cross-references every rename against what your tests reference:
|
|
557
|
-
|
|
558
|
-
| Situation | Verdict | What `fix` does |
|
|
559
|
-
|---|---|---|
|
|
560
|
-
| No test names that label | unchanged (`PASS`) | nothing — adopting this adds no noise |
|
|
561
|
-
| One element carries the new label | **FIX** | rewrites the call: `getByRole('button', { name: 'Enregistrer' })` |
|
|
562
|
-
| The **old** label now sits on another element | **BLOCK** | nothing — your locator would silently target the wrong element (the name-space twin of `rebound`) |
|
|
563
|
-
| Several elements carry the new label | **BLOCK** | nothing — a human picks |
|
|
564
|
-
| The reference is text-anchored (see below) | **BLOCK** | nothing — reported with its `file:line` |
|
|
565
|
-
|
|
566
|
-
**Only calls that state a role are ever rewritten.** That is the line between the two halves of the inventory, and it is not arbitrary:
|
|
567
|
-
|
|
568
|
-
| Rewritten | Reported, never rewritten |
|
|
569
|
-
|---|---|
|
|
570
|
-
| `getByRole` / `findByRole` / `queryByRole` (Playwright, Testing Library, `@testing-library/cypress`) | `getByText` / `findByText` |
|
|
571
|
-
| `getByLabel` · `getByLabelText` · `getByPlaceholder(Text)` — the role is implied | `cy.contains(…)`, both forms |
|
|
572
|
-
| `By.linkText` / `By.LINK_TEXT` / `By.PARTIAL_LINK_TEXT` (Selenium: role = link) | `getByTitle` · `getByAltText` |
|
|
573
|
-
|
|
574
|
-
`getByText('Archive')` names a *string*, not an element, and the contract maps interactive elements only — so it cannot prove the test meant the renamed button rather than a heading that never moved. Rewriting it would break a test that was passing, which is worse than any drift this tool detects. Those references are printed with their `file:line` and hold the verdict at BLOCK for a human instead.
|
|
575
|
-
|
|
576
|
-
Without `usage.json` nothing is escalated at all, so an un-ingested project behaves exactly as before.
|
|
577
|
-
|
|
578
|
-
### Is the app itself sound? — `ia-qa-heal check`
|
|
579
|
-
|
|
580
|
-
```bash
|
|
581
|
-
npx ia-qa-heal check # dead links + static checks
|
|
582
|
-
npx ia-qa-heal check --offline # skip the network half, fully local
|
|
583
|
-
npx ia-qa-heal check --strict # fail the pipeline on warnings too
|
|
584
|
-
npx ia-qa-heal check --concurrency 2 # gentler on a target with a strict rate limiter
|
|
585
|
-
```
|
|
586
|
-
|
|
587
|
-
`diff` answers *what moved since the baseline?*. `check` answers what a diff structurally cannot: **is the current contract healthy at all?** A green diff over a page whose every link 404s is still a green diff — verified on a sample app where `diff` exits 0 and `check` exits 1 at the same instant.
|
|
588
|
-
|
|
589
|
-
Every check is admitted under one rule: **it must be able to fail for a reason a contract diff would not already catch.** "The button still exists" is rejected by that rule — `diff` proves it better and without a browser. What survives:
|
|
590
|
-
|
|
591
|
-
| Check | Fails when | Verdict |
|
|
592
|
-
|---|---|---|
|
|
593
|
-
| **dead links** | the app answered 4xx/5xx (401/403 excluded — that is a working link behind auth; 429 excluded — see below) | **FAIL** |
|
|
594
|
-
| **unnamed** | an interactive element has no accessible name: no screen-reader announcement, no `getByRole`, no healing | WARN |
|
|
595
|
-
| **ambiguous** | two elements share role + name — every name-anchored locator for them is a coin flip, and a strict-mode violation waiting | WARN |
|
|
596
|
-
| **orphans** | a mapped page nothing links to | WARN |
|
|
597
|
-
| **unverified** | the request was throttled (429) or never answered — no verdict was reached | WARN |
|
|
198
|
+
Read the composition, not the ratio. `working` selectors are **fine**. And `not seen` is a
|
|
199
|
+
statement about how many pages you map: a selector on a page absent from `config.pages`, or in
|
|
200
|
+
a state no capture reached, looks exactly like one that is gone. `ia-qa-heal discover` lists
|
|
201
|
+
what you are missing.
|
|
598
202
|
|
|
599
|
-
|
|
203
|
+
No `usage.json` ⇒ no bindings ⇒ nothing changes. Same rule as name drift: **no inventory, no
|
|
204
|
+
escalation.**
|
|
600
205
|
|
|
601
|
-
|
|
206
|
+
## Capture during your own test run
|
|
602
207
|
|
|
603
|
-
|
|
208
|
+
`map` walks URLs. Your suite walks *states* — post-login pages, modals, wizard steps — and
|
|
209
|
+
already handles whatever auth your app demands. Arm it with `IAQA_CAPTURE=1` and the contract
|
|
210
|
+
is captured during the real run. **This is the intended path for a multi-domain / federated-SSO
|
|
211
|
+
app**, and it never touches your credentials.
|
|
604
212
|
|
|
605
|
-
**
|
|
213
|
+
**Playwright** — one import at the top of a spec (or wrap your own extended `test` with
|
|
214
|
+
`withCapture`):
|
|
606
215
|
|
|
607
|
-
|
|
608
|
-
|
|
609
|
-
```bash
|
|
610
|
-
npx ia-qa-heal audit tests/ # advisory, exit 0
|
|
611
|
-
npx ia-qa-heal audit --strict # exit 1 on a finding
|
|
216
|
+
```ts
|
|
217
|
+
import { test, expect } from '@ia-qa/self-healing/capture';
|
|
612
218
|
```
|
|
613
219
|
|
|
614
|
-
|
|
615
|
-
|
|
616
|
-
Deliberately conservative, and it says so out loud:
|
|
617
|
-
|
|
618
|
-
- Only **role-anchored** locators carry the verdict, and only for roles a contract can hold. `getByText('Terms of service')` may point at a paragraph `map` never captured, and `getByRole('heading', …)` at a heading it never captures — their absence proves nothing, so they are counted and not judged.
|
|
619
|
-
- **Advisory by default** (exit 0). A "missing" locator may live on a page not in `config.pages`, or in a page *state* `map` never saw (a label behind a tab or a mode toggle). Both caveats are printed next to the findings. Add `--strict` once your coverage is real.
|
|
620
|
-
- Read-only. `audit` never edits and never applies a suggestion.
|
|
220
|
+
**Cypress** — one import in `cypress/support/e2e.js`, nothing to wire in `setupNodeEvents`:
|
|
621
221
|
|
|
622
|
-
|
|
623
|
-
|
|
624
|
-
An icon button or image link with no `aria-label`/text has **no role + name identity** — self-healing can't recover it, a `getByRole` locator can't target it, and a screen reader can't announce it. A live capture of a dynamic app keeps catching a shifting set of them, which is the noisiest thing in a drift diff. Each page contract now counts them and lists them in a dedicated section, and `diff` flags them (`👻`) with a **synthetic hint** derived from the element's icon/image (`icon: chevron-right`, `img: Logo`) so you can find them. The hint is a locating aid, never an accessible name — the durable fix is an `aria-label` or a `data-testid`.
|
|
625
|
-
|
|
626
|
-
## Two front doors, one engine
|
|
627
|
-
|
|
628
|
-
| | For | Entry point |
|
|
629
|
-
|---|---|---|
|
|
630
|
-
| **MCP server** (stdio) | AI agents — "map my app at \<url\>" | `ia-qa-heal-mcp` → tools `map_app`, `diff_mappings`, `fix_tests`, `suggest_heal` (optional AI) |
|
|
631
|
-
| **CLI** | humans & CI | `ia-qa-heal` → `init`, `discover`, `map`, `baseline`, `diff`, `fix`, `ingest`, `audit`, `run` |
|
|
632
|
-
| **AI add-on** (optional, BYOK) | semantic renames the deterministic engine gives up on | `ia-qa-heal-ai` → `suggest` — see [Optional AI add-on](#optional-ai-add-on--semantic-suggestions-ia-qa-heal-ai-byok) |
|
|
633
|
-
|
|
634
|
-
### Locale
|
|
635
|
-
|
|
636
|
-
The page contract's accessible names are language-dependent. If your app or browser OS renders in a different locale than your test environment, every name match will fail. Set the locale explicitly:
|
|
637
|
-
|
|
638
|
-
```bash
|
|
639
|
-
# env var — works for both CLI and MCP paths
|
|
640
|
-
export IAQA_LOCALE=fr-FR # or en-US, de-DE, ja-JP…
|
|
641
|
-
|
|
642
|
-
# or in .ia-qa/config.json:
|
|
643
|
-
{ "locale": "fr-FR", … }
|
|
222
|
+
```js
|
|
223
|
+
import '@ia-qa/self-healing/cypress';
|
|
644
224
|
```
|
|
645
225
|
|
|
646
|
-
|
|
226
|
+
**Selenium (Node)** — you call it yourself, since there is nothing to hook:
|
|
647
227
|
|
|
648
|
-
|
|
649
|
-
|
|
650
|
-
|
|
651
|
-
|
|
652
|
-
"ia-qa-self-healing": {
|
|
653
|
-
"command": "npx",
|
|
654
|
-
"args": ["-y", "-p", "@ia-qa/self-healing", "ia-qa-heal-mcp"]
|
|
655
|
-
}
|
|
656
|
-
}
|
|
657
|
-
}
|
|
228
|
+
```js
|
|
229
|
+
const { capture } = require('@ia-qa/self-healing/selenium');
|
|
230
|
+
await driver.get('https://app.example.com/checkout');
|
|
231
|
+
await capture(driver); // inert unless IAQA_CAPTURE=1
|
|
658
232
|
```
|
|
659
|
-
`-p` is required, not cosmetic: the package ships **three** bins (`ia-qa-heal`, `ia-qa-heal-mcp`, `ia-qa-heal-ai`), so `npx @ia-qa/self-healing …` cannot resolve which to run and silently starts nothing.
|
|
660
|
-
The MCP server is a dependency-free JSON-RPC 2.0 stdio implementation (`src/mcp/server.ts`) — no SDK. It does **not** require the separate Playwright MCP: it drives its own browser. stdout is the protocol channel; logs go to stderr.
|
|
661
|
-
|
|
662
|
-
### Browser reuse — no second Chromium (`src/launcher.ts`)
|
|
663
|
-
Playwright's browser cache is **machine-wide and shared** (`%LOCALAPPDATA%\ms-playwright`, `~/.cache/ms-playwright`). If the user already has *any* Playwright install — their own tests or the Playwright MCP — the bundled Chromium is already there and we reuse it for free. There is no "second Chromium" to avoid; the only real cost is **revision skew** (a different Playwright version pins a different Chromium revision). For that case, and for machines that forbid the download, the launcher resolves in this order:
|
|
664
|
-
|
|
665
|
-
1. explicit executable — `browserPath` / `$IAQA_BROWSER_PATH`
|
|
666
|
-
2. explicit channel — `browserChannel` / `$IAQA_BROWSER_CHANNEL` (`chrome`, `msedge`…) → **zero download**
|
|
667
|
-
3. Playwright's bundled Chromium (shared cache — usually already present)
|
|
668
|
-
4. automatic fallback to the system Chrome, then Edge
|
|
669
|
-
|
|
670
|
-
Only if all four fail does it error, listing the three fixes. `map_app` exposes `browser_channel` / `browser_path` and always reports which browser it used. For DOM extraction the engine is irrelevant — Chrome, Edge and Chromium yield the same contract.
|
|
671
|
-
|
|
672
|
-
## How it works
|
|
673
|
-
|
|
674
|
-
1. **`ia-qa-heal init`** — interactive wizard. This is where you define your **secret zone** and grant the tool access to it: choose the framework, where credentials live (**your environment, typically loaded from a local `.env`** — or **AWS SSM Parameter Store**), the login flow, and the pages to map. It writes `.ia-qa/config.json`, which stores only a **reference** to each secret — the env-var name or the SSM parameter path, **never the value**. At run time `map` reads the credentials from that zone and logs in; if one is missing it **stops with a message naming the exact variable to set** — it never prompts, never guesses, and never writes a secret to disk. **And they never reach ia-qa.** The credentials only fill the login form in the **local** headless browser on your machine; self-healing makes **no call to ia-qa.com or any third party** — the only network traffic is your own browser loading your own app (plus your own AWS account, if you chose SSM). (Behind a login and don't want to wire this up? Use `run` with `IAQA_CAPTURE=1` and let your own test suite handle the login — see [the one-verb loop](#the-one-verb-loop-ia-qa-heal-run).)
|
|
675
|
-
2. **`ia-qa-heal map [page]`** — resolves credentials at runtime (env or SSM via your local AWS credential chain), launches headless Chromium, logs in if configured, and extracts every **visible interactive element** (role + accessible name + stable selector) to **two files** per page: `.ia-qa/mapping/<page>.json` (for tooling / `diff`) and `.ia-qa/mapping/<page>.md` (the **page contract** — the framework-agnostic artifact you hand to an LLM to auto-heal tests):
|
|
676
|
-
```json
|
|
677
|
-
{
|
|
678
|
-
"page": "login",
|
|
679
|
-
"url": "/login",
|
|
680
|
-
"capturedAt": "2026-07-15T10:00:00.000Z",
|
|
681
|
-
"elements": [
|
|
682
|
-
{ "role": "button", "name": "Se connecter", "selector": "button#login" }
|
|
683
|
-
]
|
|
684
|
-
}
|
|
685
|
-
```
|
|
686
|
-
The `<page>.md` groups elements by role and carries a self-describing header telling a model exactly how to use it. **The CI auto-heal loop:** when a UI test fails, hand the failing test file + the current `<page>.md` to an LLM — it finds the element by role + accessible name, reads the current selector, and rewrites the test in **whatever framework and language** it is written in (Cypress, Playwright, Selenium; JS or Python). The contract is the source of truth; the model handles the syntax. The same contract is downloadable from the zero-install **Selector Drift Detector** page on ia-qa.com.
|
|
687
|
-
|
|
688
|
-
Alongside the page contracts, `map` rebuilds the app index and the navigation graph — see [Map artifacts](#map-artifacts).
|
|
689
|
-
3. **`ia-qa-heal baseline`** — promote `.ia-qa/mapping/` to `.ia-qa/baseline/`, the reference `diff` compares against. Commit it: a reference that is not in git cannot be reviewed or reverted, and accepting drift is a code-review decision like any other.
|
|
690
|
-
|
|
691
|
-
Drift is a claim about **two moments**, and `map` only ever captures one. Promotion is the second half, and it is a separate verb on purpose — deliberately *not* a flag on `map`, because promoting in the same breath as capturing would make the reference incapable of ever being older than what it is compared to. `map` records what the app **is**; `baseline` says it is also what the app is **supposed to be**. That second claim is a human judgement.
|
|
692
|
-
|
|
693
|
-
Skip it and the natural workaround is `cp current.json baseline.json && diff` — which compares a capture to a copy of itself and prints `PASS ✅ No drift`. That verdict is green because nothing was compared, not because nothing changed. `diff` now refuses it outright (exit `2`): every side carries a `capturedAt`, equal timestamps mean one `map` run, and one run cannot drift from itself. A gate that reports green without looking is worse than no gate, because it is trusted.
|
|
694
|
-
4. **`ia-qa-heal diff [before.json] [after.json]`** — CI-gate verdict. With no arguments, diffs `baseline/` against `mapping/` — the everyday case. Diffs two mappings (from `map` or the ia-qa.com browser snippet) and reports which selectors survived, which the runtime healer would recover, and which will hard-fail:
|
|
695
|
-
```
|
|
696
|
-
⛔ BLOCK
|
|
697
|
-
before.json → after.json · 3 ok · 2 healable · 1 lost · 2 new
|
|
698
|
-
🔧 healable button "Se connecter" button#login-btn → button.btn-primary (exact name)
|
|
699
|
-
⛔ lost button "Supprimer le compte" button#delete → (no match)
|
|
700
|
-
```
|
|
701
|
-
Exit codes for pipelines: `0` = PASS or FIX, `1` = BLOCK — lost, ambiguous, or rebound (add `--strict` to fail on FIX too), `2` = bad input, no baseline, or nothing to compare. Use `--json` for machine-readable output. Same diff engine as the web tool (`src/browser/match.js`).
|
|
702
|
-
|
|
703
|
-
**`--report [file.html]`** also writes a self-contained, ia-qa.com-branded HTML report of the run — a plain-language summary of what the run checked and found (how many elements across how many pages, what drifted, how many of your test files it touches, and the next action), the PASS/FIX/BLOCK verdict, totals, a per-page table and every drifted selector with its `old → new` transition and test-usage annotation — the artefact to attach to a PR instead of a console screenshot (default `ia-qa-heal-report.html`). Add **`--open`** to open it in your browser (local use; leave it off in CI). Works on `diff` and `run`.
|
|
704
|
-
|
|
705
|
-
A contract that was **not re-mapped** since the baseline carries the baseline's own `capturedAt`, so it cannot show drift. Those are listed and left **out** of the verdict rather than counted as `ok` — padding a gate with comparisons that never happened is the same lie in smaller print. `--json` reports them as `staleExcluded`.
|
|
706
|
-
|
|
707
|
-
Every element carries a **context** (nearest landmark + section heading) so two elements with the same role + accessible name (five "Delete" buttons) stay distinguishable. The diff classifies each baseline element as `ok`, `renamed` (same element, new accessible name — a *content* drift to eyeball), `healable` (deterministic old→new rewrite), `ambiguous` (several equal matches — the tool **refuses to guess**, a human decides), or `lost` (no match).
|
|
708
|
-
|
|
709
|
-
Identity comes from the contract (role + accessible name), never from the selector string. A selector that still resolves only proves the *position* is still occupied: insert one button above a tab bar and every `nth-of-type` below it silently slides onto its neighbour. Any row where that happened is flagged `rebound` — the old selector resolves, but to a **different element**, so the suite is green while clicking the wrong thing. Rebound rows are rewritten like any other healable row, and they **BLOCK**: a `FIX` verdict exits `0`, which is exactly how this ships unnoticed.
|
|
710
|
-
5. **`ia-qa-heal fix <before.json> <after.json> <test-paths…>`** — the deterministic half. Applies **only** the `healable` rewrites to your test files, replacing the selector only where it appears as a quoted string literal (`'…'`/`"…"`/`` `…` ``, quote-safe for attribute selectors), across any framework/language. Every rewrite is applied in a single pass against the selector each occurrence had in the file, so a *permuted* set (`nth-of-type(1)→(2)`, `(2)→(3)`) can never chain and drag `(1)` to `(3)`. LOST / AMBIGUOUS / RENAMED are printed but never touched — they need judgment. It **edits the working tree and never commits**: review with `git diff`, then push or discard. `--dry-run` previews the plan. With **no arguments** it defaults to `baseline/` vs `mapping/`, exactly like `diff`. This is "prepare the ground, the human decides" as a command.
|
|
711
|
-
|
|
712
|
-
**`--locators`** rewrites to a Playwright role locator — `page.getByRole('button', { name: 'Validator', exact: true })` — instead of a selector, wherever role + accessible name identifies exactly one element. A repaired positional selector is only correct until the next insertion; a role locator is what the contract already means, so it does not drift at all. Applies to `.js`/`.ts` files and Playwright call shapes (`page.click('…')`, `page.locator('…')`, `page.fill('…', v)`, …); every other file and every element without an unambiguous role + name keeps the plain selector rewrite. `exact: true` is emitted because the guard already verified exactly one element matches the exact name — without it, `getByRole` does substring matching and would match multiple elements with overlapping names (e.g. three "Wiki" buttons), causing the strict-mode throw the locator was supposed to avoid. When `--locators` is passed but no Playwright call shape is found in the scanned files, a note explains that Cypress (`cy.get(sel).click()`) and other frameworks silently keep the plain selector rewrite.
|
|
713
|
-
6. **Runtime healing** — in your Playwright tests:
|
|
714
|
-
```ts
|
|
715
|
-
import { aiClick, aiFill } from '@ia-qa/self-healing';
|
|
716
|
-
|
|
717
|
-
await aiFill(page, 'input[name="email"]', user);
|
|
718
|
-
await aiClick(page, 'button#login');
|
|
719
|
-
```
|
|
720
|
-
If the selector fails, the helper re-scans the live page, matches the element by role + accessible name (exact, then fuzzy Dice ≥ 0.6 — or your own `llmResolver`), retries on the healed selector, and logs a warning telling you to update the test and re-map.
|
|
721
|
-
|
|
722
|
-
## Optional AI add-on — semantic suggestions (`ia-qa-heal-ai`, BYOK)
|
|
723
|
-
|
|
724
|
-
Everything above is deterministic and **refuses to guess**: a rename Dice can't see — `"Submit"` → `"Confirm order"` — comes back `lost`, and a human fixes it. That refusal is the point; it is what makes a green gate here mean something.
|
|
725
|
-
|
|
726
|
-
`ia-qa-heal-ai` is an **optional, opt-in** third binary that hands **only** those `lost`/`ambiguous` rows to **your own** LLM (bring your own key) and proposes a match. It is a **suggestion you confirm** — never a CI auto-fix. The deterministic `ia-qa-heal` works fully without it; configure no key and nothing changes.
|
|
727
|
-
|
|
728
|
-
> **There is no `--ai` flag on `ia-qa-heal`** — and there never will be: the loop and its CI verdict stay AI-free. The AI layer is always this separate binary. (`ia-qa-heal run --ai` errors and points you here.)
|
|
729
233
|
|
|
730
|
-
|
|
731
|
-
|
|
732
|
-
-
|
|
733
|
-
|
|
734
|
-
|
|
735
|
-
- **Zero new dependency** (a raw `fetch`), model pinned at temperature 0, key resolved at runtime from `env`/`aws-ssm` and **never written to disk** — same `SecretRef` model as every other secret here.
|
|
234
|
+
All three are **inert without `IAQA_CAPTURE=1`**, so they can live in the suite permanently,
|
|
235
|
+
and they never fail your tests. Each worker writes its own shard to
|
|
236
|
+
`.ia-qa/mapping/.capture/`; the next verb (`diff`, `run`, `baseline`) merges them into the same
|
|
237
|
+
page contracts `map` produces — byte-compatible, named from `config.pages` so they pair with
|
|
238
|
+
your baseline.
|
|
736
239
|
|
|
737
|
-
**
|
|
240
|
+
**Any other framework or language.** The shard format *is* the interface; the helpers above are
|
|
241
|
+
sugar over it. Run the extractor in the page (`extract.js` in this package, also a copy-paste
|
|
242
|
+
DevTools snippet on ia-qa.com) and drop a JSON file into `.ia-qa/mapping/.capture/`:
|
|
738
243
|
|
|
739
|
-
| Provider | `provider` | Example models | Default key env |
|
|
740
|
-
|---|---|---|---|
|
|
741
|
-
| OpenAI | `openai` | `gpt-4o-mini`, `gpt-4.1`, `gpt-5`, `gpt-5-mini`, `gpt-5-nano` | `OPENAI_API_KEY` |
|
|
742
|
-
| Anthropic (Claude) | `anthropic` | `claude-haiku-4-5`, `claude-sonnet-5`, `claude-opus-4-8` | `ANTHROPIC_API_KEY` |
|
|
743
|
-
| Google (Gemini) | `google` | `gemini-2.0-flash`, `gemini-1.5-pro` | `GEMINI_API_KEY` |
|
|
744
|
-
|
|
745
|
-
The model list is a convenience — **any model id your provider accepts works** (the picker has a "custom…" entry), so point releases like `gpt-5.1` or dated snapshots you just type in. The call runs at temperature 0 and **auto-retries without it** for models that reject sampling params (newest Claude/GPT reasoning models), so your choice always works. `ia-qa-heal-ai models` prints the full list.
|
|
746
|
-
|
|
747
|
-
**Setup — the easy way:**
|
|
748
|
-
```bash
|
|
749
|
-
npx ia-qa-heal-ai init # pick provider + model from a list → writes the "ai" block for you
|
|
750
|
-
export OPENAI_API_KEY=sk-… # (or ANTHROPIC_API_KEY / GEMINI_API_KEY — whatever you chose)
|
|
751
|
-
```
|
|
752
|
-
`init` only writes a *reference* to the env var into `.ia-qa/config.json` — never the key itself. Or add the block by hand:
|
|
753
244
|
```json
|
|
754
|
-
{
|
|
755
|
-
|
|
756
|
-
"provider": "openai",
|
|
757
|
-
"model": "gpt-4o-mini",
|
|
758
|
-
"apiKey": { "source": "env", "key": "OPENAI_API_KEY" }
|
|
759
|
-
}
|
|
760
|
-
}
|
|
761
|
-
```
|
|
762
|
-
`minConfidence` is optional (default `0.7`).
|
|
763
|
-
|
|
764
|
-
**Use** — right after `diff`:
|
|
765
|
-
```bash
|
|
766
|
-
npx ia-qa-heal-ai suggest # baseline/ vs mapping/, all pages
|
|
767
|
-
npx ia-qa-heal-ai suggest before.json after.json # one pair
|
|
768
|
-
npx ia-qa-heal-ai suggest --json # machine-readable, no writes
|
|
769
|
-
npx ia-qa-heal-ai suggest --report r.html # branded HTML report to attach to a PR (add --open locally)
|
|
770
|
-
npx ia-qa-heal-ai suggest --apply # rewrite accepted ones — interactive, TTY only
|
|
771
|
-
```
|
|
772
|
-
|
|
773
|
-
`--report` writes the same self-contained, ia-qa.com-branded artefact as `ia-qa-heal diff --report` — but of the **AI proposals**, with a loud "these are suggestions, not applied changes" banner. Attach it to a PR for a reviewer to sign off on what the model proposed.
|
|
774
|
-
```
|
|
775
|
-
💡 lost button "Submit" (checkout)
|
|
776
|
-
button#submit → button[data-testid="confirm"]
|
|
777
|
-
proposes button "Confirm order" · confidence 91%
|
|
778
|
-
↳ same primary action of the checkout form, relabelled
|
|
779
|
-
```
|
|
780
|
-
|
|
781
|
-
**For agents** — the same thing is the MCP tool `suggest_heal`: it reads its key from the server's own environment (`api_key_env`, default `ANTHROPIC_API_KEY`/`OPENAI_API_KEY`) and **never accepts a raw key in the call**; it returns suggestions as JSON and writes nothing.
|
|
782
|
-
|
|
783
|
-
**Runtime** — the same resolver behind the `aiClick`/`aiFill` seam, opt-in:
|
|
784
|
-
```ts
|
|
785
|
-
import { aiClick, createAiResolver } from '@ia-qa/self-healing';
|
|
786
|
-
|
|
787
|
-
await aiClick(page, 'button#login', {
|
|
788
|
-
llmResolver: createAiResolver({ provider: 'anthropic', model: 'claude-…', apiKey: process.env.ANTHROPIC_API_KEY! }),
|
|
789
|
-
});
|
|
790
|
-
```
|
|
791
|
-
A low-confidence, malformed, or failed call returns `null` and falls through to the deterministic heuristic — the AI never breaks a run.
|
|
792
|
-
|
|
793
|
-
See **[ROADMAP.md](ROADMAP.md)** for what comes next — and for why the vision/VLM phase was measured and dropped: no image ever leaves your machine, because none is ever sent anywhere.
|
|
794
|
-
|
|
795
|
-
## Page Object Models — one file to keep in sync
|
|
796
|
-
|
|
797
|
-
A POM already does what this tool wants: it lifts every selector out of the tests and into **one file**. That is also the one file drift keeps breaking. `map` → `diff` → `fix` treats it like any other test file — the selector is a quoted string literal, so `fix` rewrites it in place.
|
|
798
|
-
|
|
799
|
-
```ts
|
|
800
|
-
// login.page.ts — the only place a selector is written
|
|
801
|
-
export class LoginPage {
|
|
802
|
-
readonly emailSelector = '#email';
|
|
803
|
-
readonly passwordSelector = '#password';
|
|
804
|
-
readonly submitSelector = 'button#login';
|
|
805
|
-
// used everywhere as page.fill(this.emailSelector, …)
|
|
806
|
-
}
|
|
807
|
-
```
|
|
808
|
-
|
|
809
|
-
A developer reworks the login form: `button#login` becomes `button.btn-primary`. The whole suite goes red through this one file. Heal it:
|
|
810
|
-
|
|
811
|
-
```bash
|
|
812
|
-
npx ia-qa-heal map # capture the new contract
|
|
813
|
-
npx ia-qa-heal diff # 🔧 healable button "Log in" button#login → button.btn-primary
|
|
814
|
-
npx ia-qa-heal ingest # inventory the POM's selector literals (optional — lets fix find the file itself)
|
|
815
|
-
npx ia-qa-heal fix .ia-qa/baseline/login.json .ia-qa/mapping/login.json login.page.ts
|
|
816
|
-
```
|
|
817
|
-
|
|
818
|
-
```diff
|
|
819
|
-
export class LoginPage {
|
|
820
|
-
readonly emailSelector = '#email';
|
|
821
|
-
readonly passwordSelector = '#password';
|
|
822
|
-
- readonly submitSelector = 'button#login';
|
|
823
|
-
+ readonly submitSelector = 'button.btn-primary';
|
|
824
|
-
}
|
|
245
|
+
{ "pages": [ { "page": "checkout", "url": "/checkout",
|
|
246
|
+
"elements": [ { "role": "button", "name": "Pay", "selector": "button#pay" } ] } ] }
|
|
825
247
|
```
|
|
826
248
|
|
|
827
|
-
|
|
828
|
-
|
|
829
|
-
**Naming does not matter, paths do.** `ingest` inventories a declaration whose value *reads* like a selector, whatever the property is called — `loginButton = '#login .cta'`, `submitSelector = '…'`, `LOGIN_LINK = "a[data-testid='login']"` (Python and Java page objects included) — as well as framework calls it already knows (`this.page.locator('#login')`). What it cannot do is find a file you never pointed it at: if your POMs live in `pages/` and `testPaths` says `tests/`, nothing there is inventoried, and `diff` will say *0 of N drifted selectors referenced by your tests* — which reads like good news and is not. It now says so; widen `testPaths` and re-run `ingest`.
|
|
830
|
-
|
|
831
|
-
Neighbouring constants (`baseUrl`, a fixture path, a brand colour) are excluded, and a selector built by concatenation or holding `${…}` is invisible — to `fix` too, which is why it is skipped rather than guessed at.
|
|
249
|
+
No import, no Node required. Coverage is whatever the suite visits — it is not a crawler, and a
|
|
250
|
+
page it never reached is `stale` in the next diff, never a pass.
|
|
832
251
|
|
|
833
252
|
## In CI
|
|
834
253
|
|
|
835
|
-
`diff`
|
|
254
|
+
`diff` exits `0` on PASS/FIX, `1` on BLOCK, `2` on bad input:
|
|
836
255
|
|
|
837
256
|
```yaml
|
|
838
257
|
# .github/workflows/selector-drift.yml
|
|
@@ -848,124 +267,143 @@ jobs:
|
|
|
848
267
|
- run: npm ci
|
|
849
268
|
- run: npx playwright install --with-deps chromium
|
|
850
269
|
- run: npm start & # your app on its usual port
|
|
270
|
+
- run: npx ia-qa-heal ingest # inventory what your tests use
|
|
851
271
|
- run: npx ia-qa-heal map # re-capture into .ia-qa/mapping/
|
|
852
|
-
- run: npx ia-qa-heal ingest # inventory selectors AND names your tests use
|
|
853
272
|
- run: npx ia-qa-heal diff # 0 = PASS/FIX · 1 = BLOCK · 2 = bad input
|
|
854
273
|
```
|
|
855
274
|
|
|
856
|
-
`
|
|
857
|
-
|
|
858
|
-
|
|
859
|
-
|
|
860
|
-
### Trend charts for free — `--junit`
|
|
861
|
-
|
|
862
|
-
`diff --junit` and `run --junit` write the verdict as **JUnit XML**, which Jenkins, GitLab, CircleCI, Azure DevOps and the GitHub Actions reporters all ingest natively — including the per-build history graph. You get the trend without this package rendering anything in CI.
|
|
863
|
-
|
|
864
|
-
```bash
|
|
865
|
-
npx ia-qa-heal diff --junit reports/selector-drift.xml
|
|
866
|
-
```
|
|
867
|
-
|
|
868
|
-
It is language-agnostic by construction: what it describes is your **contract**, not your suite, so a Cypress/JS, Selenium/Python or Playwright/.NET project produces identically shaped XML.
|
|
275
|
+
`diff` with no arguments compares the committed `.ia-qa/baseline/` against what `map` just
|
|
276
|
+
captured — which is why the baseline belongs in git. `--strict` fails on FIX too;
|
|
277
|
+
`--report drift.html` attaches a branded report to the build.
|
|
869
278
|
|
|
870
|
-
|
|
871
|
-
|
|
872
|
-
|
|
873
|
-
|
|
874
|
-
|
|
875
|
-
|
|
876
|
-
| name drift | failure `type="name-drift-*"` | a name-anchored locator names nothing now |
|
|
877
|
-
| page not compared | **skipped** | not measured — reporting it as a pass would be the false green this tool exists to prevent |
|
|
279
|
+
**Trend charts for free.** `diff --junit reports/drift.xml` writes the verdict as JUnit XML,
|
|
280
|
+
which Jenkins, GitLab, CircleCI, Azure DevOps and the GitHub Actions reporters ingest natively,
|
|
281
|
+
per-build history included. It describes your *contract*, not your suite, so every language
|
|
282
|
+
produces identically shaped XML. `healable` and BLOCK-class rows are failures typed by status,
|
|
283
|
+
`renamed` passes with the label change in `system-out`, and a page that was not compared is
|
|
284
|
+
**`skipped` — never a pass**.
|
|
878
285
|
|
|
879
|
-
> ⚠️
|
|
286
|
+
> ⚠️ Publish it under its **own file pattern**, separate from your suite's results. Merged into
|
|
287
|
+
> one trend, both sets of numbers stop meaning anything.
|
|
880
288
|
|
|
881
|
-
`run
|
|
289
|
+
`--json` on `diff`, `run`, `audit`, `check` and `history` gives the same verdict as one
|
|
290
|
+
parseable document. Locally, `ia-qa-heal history` charts what no single run can reconstruct —
|
|
291
|
+
and holds out runs captured a *different* way rather than averaging them in.
|
|
882
292
|
|
|
883
|
-
|
|
293
|
+
## Configuration
|
|
884
294
|
|
|
885
|
-
|
|
295
|
+
`.ia-qa/config.json`, written by `init`. The minimum:
|
|
886
296
|
|
|
297
|
+
```jsonc
|
|
298
|
+
{
|
|
299
|
+
"baseUrl": "https://staging.myapp.com",
|
|
300
|
+
"pages": [
|
|
301
|
+
{ "name": "checkout", "url": "/checkout" },
|
|
302
|
+
// a view with no URL of its own — reach it by naming the control to click
|
|
303
|
+
{ "name": "billing", "url": "/account",
|
|
304
|
+
"steps": [ { "click": { "role": "tab", "name": "Billing" } } ] }
|
|
305
|
+
],
|
|
306
|
+
"testPaths": ["tests/", "pages/"],
|
|
307
|
+
"testCommand": "npx playwright test"
|
|
308
|
+
}
|
|
887
309
|
```
|
|
888
|
-
📈 Run history · 6 recorded
|
|
889
|
-
6 comparable runs captured by `map` · 67% ended with drift · 100% mean coverage
|
|
890
|
-
|
|
891
|
-
⛔ BLOCK 8/9/26, 6:25 PM 3 drifted 2/2 pages
|
|
892
|
-
✅ PASS 8/9/26, 6:26 PM 0 drifted 2/2 pages
|
|
893
|
-
…
|
|
894
|
-
🔥 Where the churn lives
|
|
895
|
-
checkout 13 drifted elements across 4 runs
|
|
896
|
-
```
|
|
897
|
-
|
|
898
|
-
Runs captured **differently** — a `map` baseline versus a live suite capture — are held **out** of the trend and named, never averaged in: the two read a page differently, so charting both would invent a trend out of a capture artifact. Same for a run that covered far fewer pages than the rest.
|
|
899
|
-
|
|
900
|
-
The file is small, deterministic text (unlike `_shots/`), so committing it is defensible if you want the trend to survive across machines and CI.
|
|
901
|
-
|
|
902
|
-
Agents get the same numbers through the MCP tool **`heal_history`** — one `summarize()`, three renderings (terminal, console charts, MCP).
|
|
903
|
-
|
|
904
|
-
## Map artifacts
|
|
905
|
-
|
|
906
|
-
Beyond one contract per page, every `map` run rebuilds two views of the whole app. Both are derived entirely from the mappings on disk (no extra page visit) and rebuilt on every run, including `map <page>`, so they never drift out of date.
|
|
907
310
|
|
|
908
|
-
|
|
311
|
+
**Your password never goes in this file.** For a form login, `secrets` holds the *name* of an
|
|
312
|
+
env var (or an SSM parameter path) and `auth` points at it by name:
|
|
909
313
|
|
|
910
|
-
|
|
911
|
-
|
|
912
|
-
|
|
913
|
-
|
|
914
|
-
|
|
915
|
-
|
|
916
|
-
- **meshes** — when nearly every mapped page links to nearly every other (the related-tools-widget shape: 16 links between 5 pages), the whole drawing carries one fact — "these pages cross-link" — so the overview states it and no graph is written.
|
|
917
|
-
|
|
918
|
-
Pages **nothing links to** are called out as entry points (orange border in the `.svg` and the Mermaid graph): the way in — a login, a home — or a page only the shell reaches. The map cannot tell which; you can.
|
|
919
|
-
|
|
920
|
-
What is left is the real structure: a checkout tunnel stays, its noise does not. If nothing survives, both files are removed rather than left stale — for that app, the page list genuinely is the whole story.
|
|
921
|
-
|
|
922
|
-
**`.ia-qa/mapping/_shots/<page>.jpg`** — optional, written only by `ia-qa-heal map --screenshots`. One viewport JPEG per page, which `--report` then shows next to each page's name and URL, so a reader sees the view instead of only reading its path:
|
|
923
|
-
|
|
924
|
-
```bash
|
|
925
|
-
ia-qa-heal map --screenshots # capture
|
|
926
|
-
ia-qa-heal diff --report --open # the report now carries the pictures
|
|
314
|
+
```jsonc
|
|
315
|
+
"secrets": { "user": { "source": "env", "key": "APP_USER" },
|
|
316
|
+
"pass": { "source": "env", "key": "APP_PASS" } },
|
|
317
|
+
"auth": { "loginUrl": "/login", "usernameSelector": "#email", "passwordSelector": "#password",
|
|
318
|
+
"submitSelector": "button[type=submit]", "usernameSecret": "user", "passwordSecret": "pass",
|
|
319
|
+
"successSelector": "nav.dashboard" }
|
|
927
320
|
```
|
|
928
321
|
|
|
929
|
-
|
|
930
|
-
|
|
931
|
-
|
|
932
|
-
|
|
933
|
-
## Design notes
|
|
322
|
+
Loading the env var is your job — the CLI reads `process.env` and never opens a `.env` file, so
|
|
323
|
+
run it under `npx dotenv-cli --` or Node's `--env-file`. If a variable is missing it stops and
|
|
324
|
+
names it.
|
|
934
325
|
|
|
935
|
-
|
|
936
|
-
- **Shared browser-safe core.** `src/browser/` holds plain, self-contained JS (no imports, no TS syntax) used by both the CLI and the ia-qa.com web tool: `extract.js` (DOM walk), `match.js` (semantic matching + mapping diff), `contract.js` (Markdown page-contract renderer). The CLI feeds `extractInPage` to Playwright's `page.evaluate()` and the healer re-runs it live; ia-qa.com imports `extract.js` with `?raw` to show it as a copy-paste DevTools snippet and imports `match.js`/`contract.js` directly. Result: the zero-install web flow produces byte-identical mappings, verdicts and `.md` contracts to `ia-qa-heal` (verified on the same page). Keep everything in `src/browser/` dependency-free or all consumers break.
|
|
937
|
-
- Secrets: never written to disk. `env` reads `process.env`; `aws-ssm` calls `GetParameter` with decryption using the AWS SDK default credential chain (needs `ssm:GetParameter`, plus `kms:Decrypt` for SecureString).
|
|
938
|
-
- `createAiResolver` in `src/playwright/healer.ts` wires the BYOK AI resolver (`src/ai/resolver.ts`) into the `llmResolver` seam; `llmResolverStub` is kept as a no-op for backward compatibility. See [Optional AI add-on](#optional-ai-add-on--semantic-suggestions-ia-qa-heal-ai-byok).
|
|
939
|
-
- **Fix dry-run shows line numbers.** Each replacement in `fix --dry-run` reports the source lines where the rewrite applies (`3× #old → #new (lines 12, 47, 103)`), so you can jump straight to the affected locations without opening the file.
|
|
940
|
-
- **Phantom contract detection.** `saveMapping` warns when overwriting a previously captured mapping. `baseline` detects mapping files that don't match any page in `config.json` — left behind when a page is removed — and warns which files to delete.
|
|
326
|
+
Everything else is optional:
|
|
941
327
|
|
|
942
|
-
|
|
943
|
-
|
|
944
|
-
|
|
945
|
-
|
|
946
|
-
|
|
947
|
-
|
|
948
|
-
|
|
949
|
-
|
|
950
|
-
`map_app` verifies that the target URL shares the same origin as `config.json`'s `baseUrl`. This prevents an agent from being steered toward an internal-only endpoint reachable from the user's machine. Set `IAQA_ALLOW_ANY_URL=1` for multi-origin apps (SSO redirects, etc.). The check is skipped silently when no `config.json` exists (first `init` from an agent).
|
|
951
|
-
|
|
952
|
-
### Contract files are generated from live page content
|
|
328
|
+
| field | what it does |
|
|
329
|
+
|---|---|
|
|
330
|
+
| `session` | a Playwright `storageState` to reuse instead of logging in |
|
|
331
|
+
| `locale` | accessible names are language-dependent — set it if your app is not the system default |
|
|
332
|
+
| `volatile` | content that rotates by design (feed links, promos). Never enters a contract, so churn cannot read as drift |
|
|
333
|
+
| `nameMask` | labels carrying a counter or a clock (`"Cart (*)"`). Only the **label** stops being compared; role, selector and href still gate |
|
|
334
|
+
| `layouts` / `autoLayout` | shared shell extracted to `_layouts/`, diffed once instead of per page |
|
|
335
|
+
| `ai` | the BYOK add-on (see below) |
|
|
953
336
|
|
|
954
|
-
|
|
337
|
+
`--config <dir>` (or `IAQA_CONFIG_DIR`) points at another `.ia-qa/` — one per portal in a
|
|
338
|
+
monorepo. Every field is documented in full in the tutorial
|
|
339
|
+
(`TUTORIAL.md`, shipped in this package).
|
|
955
340
|
|
|
956
|
-
##
|
|
341
|
+
## The console — `ia-qa-heal ui`
|
|
957
342
|
|
|
958
|
-
|
|
343
|
+
A local web console for the rows where the tool gave up: highlighted crops of each candidate,
|
|
344
|
+
the `file:line` of every test that breaks, the verbs as plain-language buttons, and the history
|
|
345
|
+
as charts. `--shortcut` writes a desktop launcher.
|
|
959
346
|
|
|
960
|
-
|
|
347
|
+
It is **loopback only** (`127.0.0.1`), every request carries a per-process token, the Host
|
|
348
|
+
header is pinned, and its buttons spawn this package's own CLI with a fixed argv — never a
|
|
349
|
+
string from the request. It **refuses to start under CI** (an interactive surface there is a
|
|
350
|
+
hung build) and stops itself once the browser tab goes away.
|
|
961
351
|
|
|
962
|
-
##
|
|
352
|
+
## Optional AI add-on — `ia-qa-heal-ai` (BYOK)
|
|
963
353
|
|
|
964
|
-
|
|
354
|
+
Deterministic healing leaves `lost` and `ambiguous` rows behind on purpose. The add-on hands
|
|
355
|
+
*only those* to your own LLM and **suggests** a match with a confidence score:
|
|
965
356
|
|
|
966
357
|
```bash
|
|
967
|
-
|
|
968
|
-
|
|
969
|
-
|
|
970
|
-
|
|
971
|
-
|
|
358
|
+
npx ia-qa-heal-ai init # pick a provider, name the env var holding your key
|
|
359
|
+
npx ia-qa-heal-ai suggest # print suggestions
|
|
360
|
+
npx ia-qa-heal-ai suggest --apply # apply, after you confirm — refuses without a TTY
|
|
361
|
+
```
|
|
362
|
+
|
|
363
|
+
It never gates CI, and there is no `--ai` flag on `ia-qa-heal`: the verdict stays
|
|
364
|
+
deterministic by design. Your key stays in your environment; the prompt carries the contract
|
|
365
|
+
rows, nothing else.
|
|
366
|
+
|
|
367
|
+
## What leaves your machine
|
|
368
|
+
|
|
369
|
+
Nothing, unless you enable the BYOK add-on above.
|
|
370
|
+
|
|
371
|
+
**MCP server: path sandboxing.** The CLI is driven by a human in their own shell; the MCP
|
|
372
|
+
server receives arguments from an agent — potentially from third-party content via prompt
|
|
373
|
+
injection. Every path parameter (`out_dir`, `before_path`, `after_path`, `test_paths`) is
|
|
374
|
+
validated: the resolved absolute path must stay inside the project root.
|
|
375
|
+
`IAQA_ALLOW_OUTSIDE_PATHS=1` lifts it. `browser_path` is logged to stderr for auditability.
|
|
376
|
+
|
|
377
|
+
**MCP server: URL allowlist.** `map_app` verifies the target URL shares the origin of
|
|
378
|
+
`config.json`'s `baseUrl`, so an agent cannot be steered toward an internal-only endpoint
|
|
379
|
+
reachable from your machine. `IAQA_ALLOW_ANY_URL=1` for multi-origin apps.
|
|
380
|
+
|
|
381
|
+
**Contracts are generated from live page content.** The `.md` and `.json` files contain text
|
|
382
|
+
extracted from the DOM — accessible names, headings, link destinations. The Markdown renderer
|
|
383
|
+
escapes pipes, newlines and backticks to prevent table breakage and code-block injection. If
|
|
384
|
+
you map an untrusted page (third-party widget, ad, XSS'd content), treat the output as
|
|
385
|
+
untrusted before feeding it to an LLM: the tool does not detect or strip instruction-like text.
|
|
386
|
+
|
|
387
|
+
**`.ia-qa/session.json` is a secret** — live cookies. Gitignored on creation.
|
|
388
|
+
|
|
389
|
+
## Going deeper
|
|
390
|
+
|
|
391
|
+
- **The tutorial** — the whole thing, step by step, no jargon:
|
|
392
|
+
[ia-qa.com/devtools/selector-drift/tutorial](https://www.ia-qa.com/devtools/selector-drift/tutorial),
|
|
393
|
+
or `TUTORIAL.md` inside this package.
|
|
394
|
+
- **`ia-qa-heal skill`** — installs this package's agent instructions into
|
|
395
|
+
`<project>/.claude/skills/`, so an agent reads the contracts instead of opening a browser,
|
|
396
|
+
and knows the rules it must not break. Agents read skills from `.claude/skills/`, never from
|
|
397
|
+
`node_modules`, which is why this is a verb and not just a file in the tarball.
|
|
398
|
+
- **`ia-qa-heal-mcp`** — the stdio MCP server, for agents that prefer tool calls to a shell.
|
|
399
|
+
- **`SPEC-selector-resolution.md`** (shipped in the package) — how a Page Object's selectors are bound to the contract,
|
|
400
|
+
and why each refusal exists.
|
|
401
|
+
- **Peer dependencies** — `playwright` is a peer: your project provides it, so two installs
|
|
402
|
+
cannot drift to different browser revisions. `@aws-sdk/client-ssm` is an *optional* peer,
|
|
403
|
+
needed only by the `aws-ssm` secret source (~5 MB, imported lazily; the CLI tells you if you
|
|
404
|
+
pick it without).
|
|
405
|
+
- **Browser reuse** — it uses whatever Playwright browser is already on the machine before
|
|
406
|
+
asking you to download 150 MB. `IAQA_BROWSER_CHANNEL=chrome` (or `msedge`), or
|
|
407
|
+
`IAQA_BROWSER_PATH=/path/to/chrome`, to pin one.
|
|
408
|
+
|
|
409
|
+
MIT.
|