@ia-qa/self-healing 1.5.0 → 1.5.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -0
- package/ROADMAP.md +2 -2
- package/TUTORIAL.md +2 -0
- package/dist/cli/args.d.ts +14 -0
- package/dist/cli/args.js +45 -0
- package/dist/cli/args.js.map +1 -0
- package/dist/cli/index.js +16 -1
- package/dist/cli/index.js.map +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -383,6 +383,8 @@ Everything above is deterministic and **refuses to guess**: a rename Dice can't
|
|
|
383
383
|
|
|
384
384
|
`ia-qa-heal-ai` is an **optional, opt-in** third binary that hands **only** those `lost`/`ambiguous` rows to **your own** LLM (bring your own key) and proposes a match. It is a **suggestion you confirm** — never a CI auto-fix. The deterministic `ia-qa-heal` works fully without it; configure no key and nothing changes.
|
|
385
385
|
|
|
386
|
+
> **There is no `--ai` flag on `ia-qa-heal`** — and there never will be: the loop and its CI verdict stay AI-free. The AI layer is always this separate binary. (`ia-qa-heal run --ai` errors and points you here.)
|
|
387
|
+
|
|
386
388
|
**Why it stays safe** — the same reason the deterministic engine is trusted:
|
|
387
389
|
- The model **picks a live candidate by index** — it cannot invent a selector, so the worst case is pointing at the wrong *existing* element, which you veto.
|
|
388
390
|
- A **confidence floor** drops weak guesses; below it the row stays `lost`.
|
package/ROADMAP.md
CHANGED
|
@@ -28,13 +28,13 @@ shortlist of same-role candidates to the user's own LLM, which picks the match.
|
|
|
28
28
|
|
|
29
29
|
- **Module** — `src/ai/resolver.ts` (`aiResolve`): same-role top-N pre-filter (reuses `diceSimilarity`/`normalize`), CoT-ordered JSON output `{rationale, confidence, candidateIndex}`, defensive parse, hardened `fetch` (AbortController timeout, 429/5xx/network → `null`).
|
|
30
30
|
- **Three façades, one brain:**
|
|
31
|
-
- CLI — third binary `ia-qa-heal-ai suggest [--apply] [--json] [--dir]`.
|
|
31
|
+
- CLI — third binary `ia-qa-heal-ai suggest [--apply] [--json] [--dir] [--report [file.html]] [--open]`.
|
|
32
32
|
- Runtime — `createAiResolver(opts)` behind the existing `aiClick`/`aiFill` `llmResolver` seam.
|
|
33
33
|
- MCP — `suggest_heal` tool (key from the server's env, never passed in the call; writes nothing).
|
|
34
34
|
- **Config** — optional `ai` block in `.ia-qa/config.json` (`provider`, `model`, `apiKey: SecretRef`, `minConfidence`), written by an interactive `ia-qa-heal-ai init` picker (`ia-qa-heal-ai models` lists the catalogue).
|
|
35
35
|
- **Providers** — `openai`, `anthropic`, `google` (Gemini). Curated model list + free-form custom ids; temperature sent at 0 with an automatic no-temperature retry on a 400, so models that reject sampling params (newest Claude/GPT reasoning models) still work.
|
|
36
36
|
|
|
37
|
-
**Status:**
|
|
37
|
+
**Status:** released — on npm since `1.1.0` (2026-07-19), current `1.5.0`. 17 unit tests (mocked `fetchFn`, no network), full suite green (deterministic non-regression verified). Real-key dogfood of `suggest` has **not** happened yet — a first field attempt (2026-07-23, external project, OpenAI key in env) never reached the resolver: the tester ran `ia-qa-heal --ai`, which silently no-opped, because the main CLI's help never mentioned `ia-qa-heal-ai`. Both findings fixed since (pending release): the main help now points to the AI add-on, and unknown flags error loudly — `--ai` specifically answers with the `ia-qa-heal-ai` pointer (`src/cli/args.ts`, pinned by `tests/cliArgs.test.js`). Next: redo the dogfood with `ia-qa-heal-ai suggest`.
|
|
38
38
|
|
|
39
39
|
---
|
|
40
40
|
|
package/TUTORIAL.md
CHANGED
|
@@ -480,6 +480,8 @@ Steps 1–8 are **100% deterministic** and **refuse to guess**. That is delibera
|
|
|
480
480
|
|
|
481
481
|
If you'd rather get a *suggestion* for those, there's an **optional** add-on: `ia-qa-heal-ai`. It sends **only** the `lost`/`ambiguous` elements to **your own** AI model (you bring your own API key) and proposes a match — which you then confirm. It never runs in CI, never auto-edits, and the normal tool works completely without it.
|
|
482
482
|
|
|
483
|
+
> Note it is a **separate command**, not a flag: there is no `--ai` on `ia-qa-heal` (trying it errors and points you here). Putting an API key in `.env` activates nothing by itself — the AI only ever runs when *you* run `ia-qa-heal-ai suggest`.
|
|
484
|
+
|
|
483
485
|
**1. Pick your model** — one command, choose from a list, it writes the config for you:
|
|
484
486
|
```bash
|
|
485
487
|
npx ia-qa-heal-ai init
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Flag validation for the hand-rolled CLI parser.
|
|
3
|
+
*
|
|
4
|
+
* Every subcommand parses its own args and used to swallow flags it did not
|
|
5
|
+
* know — `ia-qa-heal run --ai` ran the deterministic loop and said nothing,
|
|
6
|
+
* a reassuring no-op that made a field tester conclude the AI layer did not
|
|
7
|
+
* exist. Unknown flags now fail loudly, and `--ai` specifically points to the
|
|
8
|
+
* separate `ia-qa-heal-ai` binary (the gate stays AI-free by design).
|
|
9
|
+
*
|
|
10
|
+
* Flag *values* never start with `-` here (URLs, paths, numbers, words), so a
|
|
11
|
+
* leading dash is enough to tell a flag from a positional.
|
|
12
|
+
*/
|
|
13
|
+
export declare const KNOWN_FLAGS: Record<string, readonly string[]>;
|
|
14
|
+
export declare function rejectUnknownFlags(command: string, args: string[]): void;
|
package/dist/cli/args.js
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
|
+
exports.KNOWN_FLAGS = void 0;
|
|
4
|
+
exports.rejectUnknownFlags = rejectUnknownFlags;
|
|
5
|
+
/**
|
|
6
|
+
* Flag validation for the hand-rolled CLI parser.
|
|
7
|
+
*
|
|
8
|
+
* Every subcommand parses its own args and used to swallow flags it did not
|
|
9
|
+
* know — `ia-qa-heal run --ai` ran the deterministic loop and said nothing,
|
|
10
|
+
* a reassuring no-op that made a field tester conclude the AI layer did not
|
|
11
|
+
* exist. Unknown flags now fail loudly, and `--ai` specifically points to the
|
|
12
|
+
* separate `ia-qa-heal-ai` binary (the gate stays AI-free by design).
|
|
13
|
+
*
|
|
14
|
+
* Flag *values* never start with `-` here (URLs, paths, numbers, words), so a
|
|
15
|
+
* leading dash is enough to tell a flag from a positional.
|
|
16
|
+
*/
|
|
17
|
+
exports.KNOWN_FLAGS = {
|
|
18
|
+
init: [],
|
|
19
|
+
discover: ['--sitemap', '--crawl', '--no-reveal', '--strict-host', '--apply', '--max', '--depth'],
|
|
20
|
+
map: [],
|
|
21
|
+
baseline: [],
|
|
22
|
+
diff: ['--strict', '--json', '--dir', '--report', '--open'],
|
|
23
|
+
fix: ['--dry-run', '--locators', '--dir'],
|
|
24
|
+
ingest: [],
|
|
25
|
+
graph: ['--format', '--kind', '--out', '--open'],
|
|
26
|
+
run: ['--yes', '--no-verify', '--locators', '--report', '--open'],
|
|
27
|
+
};
|
|
28
|
+
function rejectUnknownFlags(command, args) {
|
|
29
|
+
const known = exports.KNOWN_FLAGS[command];
|
|
30
|
+
if (known === undefined)
|
|
31
|
+
return;
|
|
32
|
+
for (const a of args) {
|
|
33
|
+
if (!a.startsWith('-'))
|
|
34
|
+
continue;
|
|
35
|
+
if (known.includes(a))
|
|
36
|
+
continue;
|
|
37
|
+
if (a === '--ai' || a.startsWith('--ai=')) {
|
|
38
|
+
throw new Error(`"ia-qa-heal ${command}" has no --ai flag — the deterministic loop and its CI verdict stay AI-free by design.\n` +
|
|
39
|
+
`The AI layer is a separate binary shipped in this same package: ia-qa-heal-ai (init / models / suggest).\n` +
|
|
40
|
+
`It suggests matches for the lost/ambiguous rows a diff leaves behind: ia-qa-heal-ai suggest`);
|
|
41
|
+
}
|
|
42
|
+
throw new Error(`Unknown flag "${a}" for "ia-qa-heal ${command}". Run \`ia-qa-heal help\` to see its flags.`);
|
|
43
|
+
}
|
|
44
|
+
}
|
|
45
|
+
//# sourceMappingURL=args.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"args.js","sourceRoot":"","sources":["../../src/cli/args.ts"],"names":[],"mappings":";;;AAwBA,gDAeC;AAvCD;;;;;;;;;;;GAWG;AACU,QAAA,WAAW,GAAsC;IAC5D,IAAI,EAAE,EAAE;IACR,QAAQ,EAAE,CAAC,WAAW,EAAE,SAAS,EAAE,aAAa,EAAE,eAAe,EAAE,SAAS,EAAE,OAAO,EAAE,SAAS,CAAC;IACjG,GAAG,EAAE,EAAE;IACP,QAAQ,EAAE,EAAE;IACZ,IAAI,EAAE,CAAC,UAAU,EAAE,QAAQ,EAAE,OAAO,EAAE,UAAU,EAAE,QAAQ,CAAC;IAC3D,GAAG,EAAE,CAAC,WAAW,EAAE,YAAY,EAAE,OAAO,CAAC;IACzC,MAAM,EAAE,EAAE;IACV,KAAK,EAAE,CAAC,UAAU,EAAE,QAAQ,EAAE,OAAO,EAAE,QAAQ,CAAC;IAChD,GAAG,EAAE,CAAC,OAAO,EAAE,aAAa,EAAE,YAAY,EAAE,UAAU,EAAE,QAAQ,CAAC;CAClE,CAAC;AAEF,SAAgB,kBAAkB,CAAC,OAAe,EAAE,IAAc;IAChE,MAAM,KAAK,GAAG,mBAAW,CAAC,OAAO,CAAC,CAAC;IACnC,IAAI,KAAK,KAAK,SAAS;QAAE,OAAO;IAChC,KAAK,MAAM,CAAC,IAAI,IAAI,EAAE,CAAC;QACrB,IAAI,CAAC,CAAC,CAAC,UAAU,CAAC,GAAG,CAAC;YAAE,SAAS;QACjC,IAAI,KAAK,CAAC,QAAQ,CAAC,CAAC,CAAC;YAAE,SAAS;QAChC,IAAI,CAAC,KAAK,MAAM,IAAI,CAAC,CAAC,UAAU,CAAC,OAAO,CAAC,EAAE,CAAC;YAC1C,MAAM,IAAI,KAAK,CACb,eAAe,OAAO,0FAA0F;gBAC9G,4GAA4G;gBAC5G,8FAA8F,CACjG,CAAC;QACJ,CAAC;QACD,MAAM,IAAI,KAAK,CAAC,iBAAiB,CAAC,qBAAqB,OAAO,8CAA8C,CAAC,CAAC;IAChH,CAAC;AACH,CAAC"}
|
package/dist/cli/index.js
CHANGED
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
"use strict";
|
|
3
3
|
Object.defineProperty(exports, "__esModule", { value: true });
|
|
4
4
|
const config_1 = require("../config");
|
|
5
|
+
const args_1 = require("./args");
|
|
5
6
|
const init_1 = require("./init");
|
|
6
7
|
const map_1 = require("./map");
|
|
7
8
|
const baseline_1 = require("./baseline");
|
|
@@ -21,7 +22,9 @@ One verb: ia-qa-heal run (capture during your suite → diff → confirm → f
|
|
|
21
22
|
[FOR AI AGENTS] You are driving this for a human. Explain the loop and the
|
|
22
23
|
PASS/FIX/BLOCK verdict; propose running it when their E2E selectors may have
|
|
23
24
|
drifted. In non-interactive shells, write .ia-qa/config.json directly instead
|
|
24
|
-
of \`init\` (see the wizard's non-TTY guidance).
|
|
25
|
+
of \`init\` (see the wizard's non-TTY guidance). The AI layer is the separate
|
|
26
|
+
\`ia-qa-heal-ai\` binary (BYOK suggestions for lost/ambiguous rows) — this
|
|
27
|
+
binary has no --ai flag: the loop and its CI verdict stay deterministic.
|
|
25
28
|
|
|
26
29
|
Usage:
|
|
27
30
|
ia-qa-heal init Interactive setup wizard (writes .ia-qa/config.json)
|
|
@@ -96,6 +99,12 @@ Global:
|
|
|
96
99
|
--config <dir> Resolve .ia-qa/ under <dir> instead of the current directory
|
|
97
100
|
(monorepos: one config per portal). Also IAQA_CONFIG_DIR=<dir>,
|
|
98
101
|
the only lever that reaches capture (it runs in your test process).
|
|
102
|
+
|
|
103
|
+
AI add-on (separate binary, same package — optional, BYOK):
|
|
104
|
+
ia-qa-heal-ai init | models | suggest Suggests matches for the lost/ambiguous rows a diff leaves
|
|
105
|
+
behind, using your own LLM key (OpenAI / Anthropic / Gemini).
|
|
106
|
+
A suggestion you confirm — never part of the CI gate, which
|
|
107
|
+
is why there is no --ai flag on ia-qa-heal itself.
|
|
99
108
|
`;
|
|
100
109
|
/**
|
|
101
110
|
* Pull the global `--config <dir>` / `--config=<dir>` out of argv wherever it
|
|
@@ -134,6 +143,12 @@ async function main() {
|
|
|
134
143
|
(0, config_1.setBaseDir)(configDir);
|
|
135
144
|
}
|
|
136
145
|
const [command, ...rest] = argv;
|
|
146
|
+
if (rest.includes('--help') || rest.includes('-h')) {
|
|
147
|
+
console.log(HELP);
|
|
148
|
+
return;
|
|
149
|
+
}
|
|
150
|
+
if (command !== undefined)
|
|
151
|
+
(0, args_1.rejectUnknownFlags)(command, rest);
|
|
137
152
|
const arg = rest[0];
|
|
138
153
|
switch (command) {
|
|
139
154
|
case 'init':
|
package/dist/cli/index.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/cli/index.ts"],"names":[],"mappings":";;;AACA,sCAAuC;AACvC,iCAAiC;AACjC,+BAA+B;AAC/B,yCAAyC;AACzC,iCAAiC;AACjC,+BAA+B;AAC/B,qCAAqC;AACrC,+BAA+B;AAC/B,yCAAyC;AACzC,mCAAmC;AAEnC,qEAAqE;AACrE,MAAM,OAAO,GAAG,OAAO,CAAC,oBAAoB,CAAC,CAAC,OAAiB,CAAC;AAEhE,MAAM,IAAI,GAAG
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../../src/cli/index.ts"],"names":[],"mappings":";;;AACA,sCAAuC;AACvC,iCAA4C;AAC5C,iCAAiC;AACjC,+BAA+B;AAC/B,yCAAyC;AACzC,iCAAiC;AACjC,+BAA+B;AAC/B,qCAAqC;AACrC,+BAA+B;AAC/B,yCAAyC;AACzC,mCAAmC;AAEnC,qEAAqE;AACrE,MAAM,OAAO,GAAG,OAAO,CAAC,oBAAoB,CAAC,CAAC,OAAiB,CAAC;AAEhE,MAAM,IAAI,GAAG;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;CA2FZ,CAAC;AAEF;;;;;;GAMG;AACH,SAAS,gBAAgB,CAAC,IAAc;IACtC,MAAM,IAAI,GAAa,EAAE,CAAC;IAC1B,IAAI,GAAuB,CAAC;IAC5B,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACrC,MAAM,CAAC,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;QAClB,IAAI,CAAC,KAAK,UAAU,EAAE,CAAC;YACrB,MAAM,IAAI,GAAG,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;YACzB,IAAI,IAAI,KAAK,SAAS,IAAI,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE,CAAC;gBAC/C,MAAM,IAAI,KAAK,CAAC,8DAA8D,CAAC,CAAC;YAClF,CAAC;YACD,GAAG,GAAG,IAAI,CAAC;YACX,CAAC,EAAE,CAAC;YACJ,SAAS;QACX,CAAC;QACD,IAAI,CAAC,CAAC,UAAU,CAAC,WAAW,CAAC,EAAE,CAAC;YAC9B,GAAG,GAAG,CAAC,CAAC,KAAK,CAAC,WAAW,CAAC,MAAM,CAAC,CAAC;YAClC,SAAS;QACX,CAAC;QACD,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IACf,CAAC;IACD,OAAO,EAAE,GAAG,EAAE,IAAI,EAAE,CAAC;AACvB,CAAC;AAED,KAAK,UAAU,IAAI;IACjB,MAAM,EAAE,GAAG,EAAE,SAAS,EAAE,IAAI,EAAE,IAAI,EAAE,GAAG,gBAAgB,CAAC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;IAC/E,IAAI,SAAS,KAAK,SAAS,EAAE,CAAC;QAC5B,IAAI,CAAC,SAAS;YAAE,MAAM,IAAI,KAAK,CAAC,sCAAsC,CAAC,CAAC;QACxE,IAAA,mBAAU,EAAC,SAAS,CAAC,CAAC;IACxB,CAAC;IACD,MAAM,CAAC,OAAO,EAAE,GAAG,IAAI,CAAC,GAAG,IAAI,CAAC;IAChC,IAAI,IAAI,CAAC,QAAQ,CAAC,QAAQ,CAAC,IAAI,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC,EAAE,CAAC;QACnD,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QAClB,OAAO;IACT,CAAC;IACD,IAAI,OAAO,KAAK,SAAS;QAAE,IAAA,yBAAkB,EAAC,OAAO,EAAE,IAAI,CAAC,CAAC;IAC7D,MAAM,GAAG,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;IACpB,QAAQ,OAAO,EAAE,CAAC;QAChB,KAAK,MAAM;YACT,MAAM,IAAA,cAAO,GAAE,CAAC;YAChB,MAAM;QACR,KAAK,UAAU;YACb,MAAM,IAAA,sBAAW,EAAC,IAAI,CAAC,CAAC;YACxB,MAAM;QACR,KAAK,KAAK;YACR,MAAM,IAAA,YAAM,EAAC,GAAG,CAAC,CAAC;YAClB,MAAM;QACR,KAAK,UAAU;YACb,MAAM,IAAA,sBAAW,EAAC,IAAI,CAAC,CAAC;YACxB,MAAM;QACR,KAAK,MAAM;YACT,MAAM,IAAA,cAAO,EAAC,IAAI,CAAC,CAAC;YACpB,MAAM;QACR,KAAK,KAAK;YACR,MAAM,IAAA,YAAM,EAAC,IAAI,CAAC,CAAC;YACnB,MAAM;QACR,KAAK,QAAQ;YACX,MAAM,IAAA,kBAAS,EAAC,IAAI,CAAC,CAAC;YACtB,MAAM;QACR,KAAK,OAAO;YACV,MAAM,IAAA,gBAAQ,EAAC,IAAI,CAAC,CAAC;YACrB,MAAM;QACR,KAAK,KAAK;YACR,MAAM,IAAA,YAAM,EAAC,IAAI,CAAC,CAAC;YACnB,MAAM;QACR,KAAK,SAAS,CAAC;QACf,KAAK,WAAW,CAAC;QACjB,KAAK,IAAI;YACP,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;YACrB,MAAM;QACR,KAAK,MAAM,CAAC;QACZ,KAAK,QAAQ,CAAC;QACd,KAAK,IAAI,CAAC;QACV,KAAK,SAAS;YACZ,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;YAClB,MAAM;QACR;YACE,OAAO,CAAC,KAAK,CAAC,oBAAoB,OAAO,SAAS,IAAI,EAAE,CAAC,CAAC;YAC1D,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IACpB,CAAC;AACH,CAAC;AAED,IAAI,EAAE,CAAC,KAAK,CAAC,CAAC,GAAY,EAAE,EAAE;IAC5B,OAAO,CAAC,KAAK,CAAC,OAAO,GAAG,YAAY,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;IACzE,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;AAClB,CAAC,CAAC,CAAC"}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ia-qa/self-healing",
|
|
3
|
-
"version": "1.5.
|
|
3
|
+
"version": "1.5.1",
|
|
4
4
|
"description": "Local-first self-healing for UI tests: a local MCP server + CLI that map your app's pages to a role/name/selector contract, diff selector drift (PASS/FIX/BLOCK), and apply deterministic fixes to Cypress/Playwright/Selenium tests. Runs on your machine — nothing leaves it.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"self-healing",
|