@ia-qa/qa-discovery 0.3.0 → 0.5.0

package/README.md

@@ -27,13 +27,23 @@ npx -p @ia-qa/qa-discovery ia-qa-discover scan https://your-app.example.com --sa
 | **what does my test suite not test?** | **`coverage`** — the gap, ranked; needs `@ia-qa/self-healing` and one watched run |
 | what changed since last time? | `history` |
 | what is each entry point *for*, and how sensitive? | `ia-qa-discover-ai classify` (optional, your own key) |
+| what does somebody come to this page to **do**? | `ia-qa-discover-ai plan` (optional, your own key) |
+
+**Driving this with an AI agent?** One command gives it the whole doctrine — the verb for
+each question, what every refusal means, and what it must never do:
+
+```bash
+ia-qa-discover skill --print      # read it
+ia-qa-discover skill --install    # drop it in .claude/skills/, where agents actually look
+```
 
 `ia-qa-discover <verb> --help` for flags. Nothing here emits a verdict and nothing here gates
 CI: a page count is not coverage, and a gap is a decision for a human.
 
 **Same operations for an AI agent** — one contract served twice, never a simplified view for
-one of them. `ia-qa-discover-mcp` exposes `scan_app`, `coverage_map`, `classify_app` and
-`discovery_history`. `login` is deliberately not a tool: it waits for a person at a browser.
+one of them. `ia-qa-discover-mcp` exposes `scan_app`, `coverage_map`, `classify_app`,
+`plan_pages` and `discovery_history`. `login` is deliberately not a tool: it waits for a
+person at a browser.
 
 > **New here?** The [step-by-step tutorial](https://www.ia-qa.com/devtools/qa-discovery/tutorial)
 > walks the whole thing from an app you have never seen, with no jargon — first scan, login
@@ -243,7 +253,11 @@ The file it produces holds live cookies — whoever has it is logged in as you.
 npx ia-qa-discover coverage
 ```
 
-Reads local files only — no browser, no network, no argument. It compares the pages this scan found against the pages your suite was **observed** visiting, and ranks the gap by what each page takes as input: credentials, form submissions, required fields, API calls. The factors *are* the ranking — there is no score, because a single number invites tuning and hides that a page ranks high for a reason you may consider irrelevant.
+Reads local files only — no browser, no network, no argument. It compares the pages this scan found against the pages your suite was **observed** visiting, and ranks the gap by what each page takes as input: credentials, form submissions, required fields. The factors *are* the ranking — there is no score, because a single number invites tuning and hides that a page ranks high for a reason you may consider irrelevant.
+
+One factor comes from the other package: **how often a page has actually changed**, read from healing's run history. A page that changes every week and that no test visits is where bugs are born; a page untouched for six months does not need a new test — and no scan of an app can see the difference. Runs where more than half the mapped pages drifted at once are held out and counted: that describes a stale baseline or a changed capture, not volatile pages, and including them made every page carry the factor, which ranks nothing. Below three comparable runs the factor is simply absent, and the report says why.
+
+How much a page *holds* — its field and call counts — is shown beside the factors and never counted with them. Describing a page and ranking it are two jobs: merged, `1 input · 1 API call` out-ranked a page that had actually changed three times.
 
 It also reports **depth**: how many of each visited page's contracted elements your tests actually name. Named is not asserted — a test that clicks a button names it without checking anything, and nothing on disk can tell the two apart, so this never says "tested".
 
@@ -261,6 +275,28 @@ Writes `.ia-qa-discovery/coverage-map.json` (committable, diffable) and refreshe
 
 They stay **counted and named** as out of scope, never silently dropped, and nothing is ever added to that list on its own.
 
+### `generate`
+
+**A starting suite for the pages nothing tests — written so healing can repair it.**
+
+```bash
+npx ia-qa-discover generate
+```
+
+Writing test code is not the scarce thing; `playwright codegen` has done it for years. What it cannot do is decide *what* to record, and what nobody does is make the result survive the app changing. With no argument this generates one spec per page in the coverage gap — the pages that exist and that no test visits — and every locator it writes names an element the healing contract holds. So when a label changes, the file is **rewritten** instead of going red.
+
+That loop is not a claim: it is executed on every commit, on this verb's own output — generated, drifted, repaired, resolving again, with no human edit.
+
+**What it refuses to write, and why that is the feature.** Healing rewrites calls that state a role — `getByRole`, `getByLabel`, `getByPlaceholder`, `By.linkText` — and nothing else. `getByText` names a *string*, not an element, so nothing can prove the test meant the renamed button rather than a heading that never moved. And one such line does not merely stay unrepaired: it holds the verdict at BLOCK, which stops the repairable lines *beside* it from being repaired too. So a field that cannot be named by role, label or placeholder is left out, never written in a weaker form.
+
+Left out for the same reason: a field with no test id, id or name. Its only selector is positional, and it addresses a different element the moment a field is inserted above it — untestable by anyone, not only by this tool. Each omission is named at the top of the file it was omitted from, because a silent gap reads as a finished test.
+
+**The assertions are yours.** An application declares what it *accepts* — required, type, min, max — and never what it *promises*. "A refused card shows the right message" lives in someone's head, so the navigation and the actions are generated and what the app should *do* is a TODO with the question written out.
+
+Files land in `.ia-qa-discovery/generated/` and nothing is added to your suite. An existing file is never replaced without `--force`. If the output directory is outside healing's `testPaths`, the verb says so and names both remedies — out of reach means never repaired, which would quietly make the whole promise false.
+
+`--out <dir>` to write them where your suite lives, `--json` for a machine-readable summary, or name pages explicitly to generate for them anyway.
+
 ### `history`
 
 The trend no single scan can reconstruct: pages, forms, API surface, and what changed run over run.
@@ -388,6 +424,7 @@ A dependency-free JSON-RPC 2.0 stdio server exposing the **same** operations the
 
 - `scan_app` — crawl and capture. Check `loginWall` and `hrefBlind` in the result before reporting coverage.
 - `classify_app` — the optional BYOK layer below, for an agent. The key is read from an environment variable **named** in the call (`api_key_env`); a raw key is never a tool argument.
+- `plan_pages` — the other BYOK question: what somebody comes to each page **to do**. Same key handling, and it sends **less** — no selectors, no observed API calls. Defaults to the pages `coverage_map` says no test visits, in that tool's order. Ranks nothing, gates nothing; a `failed` page is a failed call, never a finding about the app.
 - `coverage_map` — what the suite does **not** test. Returns `measured: false` with a typed `reason` when no run was ever watched: report that as *no measurement*, never as "the suite covers nothing".
 - `discovery_history` — the trend.
 
@@ -402,8 +439,15 @@ Everything above is deterministic and never leaves your machine. This one binary
 ```bash
 ia-qa-discover-ai classify --dry-run   # see exactly what would be sent, and to whom
 ia-qa-discover-ai classify
+
+ia-qa-discover-ai plan --dry-run       # the other question, and a smaller payload
+ia-qa-discover-ai plan --report --open
 ```
 
+Two verbs, two questions, and **`plan` sends strictly less than `classify`**: no selectors and
+no observed API calls ever leave for it. Each announces its own payload, derived from the
+function that builds it, so neither can promise less than it sends.
+
 It reads the capture **offline** — it opens no browser and never touches your app again — and asks a model what each entry point is *for* (`authentication`, `payment`, `search`, `data-entry`…) and how sensitive what it handles is. The vocabulary is closed and versioned: a label outside it is rejected before anything else is checked.
 
 **Every claim cites the capture, and every citation is resolved and checked.** A classification whose evidence does not exist, or does not say what it was claimed to say, is dropped before you see it, and the drop is reported. That proves the *premise*, never the conclusion — a model can cite a real password field and still be wrong about what the page is for — so read the `why` and the confidence, not the label alone.
@@ -415,6 +459,12 @@ Configure it in `.ia-qa-discovery/config.json` — the file holds the *name* of
 ```json
 "ai": { "provider": "anthropic", "model": "claude-haiku-4-5",
         "apiKey": { "source": "env", "key": "ANTHROPIC_API_KEY" } }
+
+// or any OpenAI-compatible endpoint — DeepSeek, Groq, Mistral, OpenRouter, vLLM,
+// or a model on your own machine, in which case nothing leaves it at all:
+"ai": { "provider": "openai-compatible", "model": "deepseek-chat",
+        "baseUrl": "https://api.deepseek.com/v1",
+        "apiKey": { "source": "keychain", "key": "DEEPSEEK_KEY" } }
 ```
 
 Before the first request it prints what leaves and where it goes:
@@ -441,6 +491,62 @@ rewrite — it is your file — so `scan` warns instead.
 
 Read that list before pointing it at an authenticated or private app. The result lands in `.ia-qa-discovery/classification.json` **and** in `_overview.md`, which is rewritten so the reading appears next to the capture it was made from.
 
+### `plan` — what somebody comes to this page to do
+
+`classify` answers what an entry point *is*, against a closed vocabulary. `plan` answers a
+different question, on a different axis: **what is a person trying to do here, and what can
+they no longer do if it breaks** — in the words of whoever uses the app, not a developer's.
+
+It exists because that axis is the one nothing else in this ecosystem can reach. The
+deterministic half reports what an app *accepts* — fields, types, required, what moved.
+Nothing in a DOM states that this form is how a locked-out customer gets back in. Neither
+layer is a subset of the other, which is why one never ranks above the other.
+
+```bash
+ia-qa-discover-ai plan                 # the pages `coverage` says no test visits, in its order
+ia-qa-discover-ai plan --all           # every readable page, tested or not
+ia-qa-discover-ai plan --report --open # the same branded dossier `scan --report` writes
+```
+
+By default it reads exactly the untested pages from `ia-qa-discover coverage`, **in the order
+that verb ranked them** — so an interrupted run has read the ones that mattered most. With no
+suite tracked here there is no untested list to narrow to, and it reads every page and says
+so rather than refusing: a project with no tests is the one this package exists for.
+
+Four properties, and each is tested:
+
+- **It sends less than `classify`** — no selectors, no observed API calls. The most sensitive
+  half of a capture, and technical noise for a question that is not technical.
+- **Every reading cites the capture** and is dropped if the citation does not resolve, exactly
+  like a classification. A path the prompt never offered is refused *before* resolution, so
+  citing an API call — which this verb does not send — cannot happen.
+- **The shell is not a page's content.** Headings repeated across the app (a footer's `Legal`,
+  `Contact`, `Tools`) are dropped from the payload, using the same threshold `autoLayout`
+  uses. Measured before the change: a page whose whole interest was `🗂️ Environment Manager`
+  had its reading propped up by three footer links — citations that resolve, and support
+  nothing. A page whose *every* heading is shared keeps them all: the exclusion narrows the
+  evidence, it never makes a page unreadable.
+- **At least one citation must carry the subject** — a level-1 or level-2 heading, a form, or
+  a field. A reading standing entirely on nav labels is dropped. The rule is skipped on a page
+  that offers no such anchor, because a rule nobody can satisfy would punish a page for its own
+  markup.
+- **It ranks nothing and gates nothing.** The order is `coverage`'s measurement; this adds a
+  labelled line beside it, never a row to it.
+- **The origin travels with the artifact.** `plan.json` carries the model and the timestamp,
+  one module renders the terminal, `_overview.md` and the HTML report, a page re-captured
+  after it was read is marked **stale**, and the report's footer stops saying "no LLM".
+
+A model that cannot tell answers `unclear`: kept, labelled, never silently dropped — the
+capture may genuinely not say, and a proposal nobody can anchor is still worth reading as
+long as it is marked as one.
+
+**Its honest limit**, measured on a real run of 8 pages: the reading is worth most where the
+deterministic half is blind — a page with no form at all, whose headings name the stake — and
+thins out on content pages, where "somebody comes here to read the articles" repeats the
+capture back. Read the confidence and the citations, not the sentence alone. After the two
+rules above, that same run produced **0 citations pointing at the footer or the nav**, against
+three on a single page before them.
+
 ---
 
 ## Library

package/ROADMAP.md

@@ -84,7 +84,7 @@ be understood.
 
 | Stage | The question it answers | Status |
 |---|---|---|
-| **Value and personas** (F0) | Who uses this app, for what, and where would it hurt most if it broke? | not built |
+| **Value and personas** (F0) | Who uses this app, for what, and where would it hurt most if it broke? | partly shipped — see `plan` below |
 | **Surface reconnaissance** (F1) | What pages exist, what do they take as input, what do they call? | ✅ shipped (0.1.0) |
 | **Component classification** (F2) | What is each entry point *for*, and how sensitive is what it handles? | in progress |
 | **Zones and synthesis** (F2.5) | How do the pages group into functional areas, and what is the app as a whole? | not built |
@@ -218,9 +218,31 @@ navigation graph), then a grounded synthesis on top. Reconciles with the value s
 
 ## Value and personas (F0) — *the game changer, built with care*
 
-Who uses this app, for what, and where the moment of truth is. This is the stage that makes
-the output a senior's plan instead of a classified inventory, and the one whose claims are
-hardest to check — so it is scheduled as **careful** work, not as later work.
+**A first, deliberately narrow slice shipped: `ia-qa-discover-ai plan`.** It answers the
+value question for **one page at a time** — what somebody comes here to do, and what they can
+no longer do if it breaks — and nothing beyond that. No personas, no app-level framing, no
+"moment of truth": those are the claims the validation protocol below exists to gate, and
+none of them is made.
+
+Why this slice was safe to ship first, in the terms this file uses everywhere else:
+
+- It **produces no executable artifact**, so a wrong reading cannot become a broken test.
+- Every reading **cites the capture** and is dropped when the citation does not resolve —
+  the same mechanism as classification, on a per-page claim small enough for that check to
+  mean something.
+- It **ranks nothing**: the order is `coverage`'s measurement, and the model never adds a row
+  to it. That is the difference between commenting on a measurement and replacing one.
+- It sends **less** than `classify` — no selectors, no observed API calls.
+
+Measured limit, from a real 8-page run: the reading is worth most on a page the deterministic
+half is blind to (no fields at all, the stake living in the headings) and is near-tautological
+on content pages. That asymmetry is itself an argument for keeping this stage per-page until
+the protocol below exists.
+
+What remains unbuilt is the hard half — who the users are, and where the moment of truth is.
+This is the stage that makes the output a senior's plan instead of a classified inventory, and
+the one whose claims are hardest to check — so it is scheduled as **careful** work, not as
+later work.
 
 **It needs a validation protocol before it needs code.** Classification can be checked in seconds
 against the capture; this cannot. Label a handful of real apps by hand, compare the stage's
@@ -245,6 +267,17 @@ by the classifier's intent", so the first half is not blocked on the second.
 
 ## Scenario generation (F3)
 
+**Shipped, narrowly: `ia-qa-discover generate`.** Not scenario generation — *spec scaffolding
+from the observed surface*. It writes one Playwright file per uncovered page, naming only
+elements the healing contract holds, so the file is repaired rather than deleted when the app
+moves; that loop runs on every commit against the verb's own output. It emits only calls
+healing can rewrite (one `getByText` holds the verdict at BLOCK and blocks the repairable
+lines beside it), skips fields with no durable identity and says so in the file, and writes no
+assertion beyond what the app itself declares. What a submission *produces* stays a TODO.
+
+That is the deterministic floor. Everything below — deriving *intent* rather than structure —
+is still net-new, and still needs the grid.
+
 **The audit found the assumed inputs do not exist.** There is no closed grid of test
 patterns anywhere in the ecosystem to derive from — `QA_AGENT_RECIPES` and
 `AUGMENTED_QA_PLAYBOOK` are editorial content, not structured data. So the intent→scenario

package/TUTORIAL.md

@@ -30,7 +30,8 @@ suite, a ranked list of the pages nobody tests.
 8. [What my suite does not test](#8-what-my-suite-does-not-test)
 9. [Doing this every week](#9-doing-this-every-week)
 10. [Optional: what is each entry point *for*?](#10-optional-what-is-each-entry-point-for)
-11. [Troubleshooting](#11-troubleshooting)
+11. [Optional: what does somebody come here to *do*?](#11-optional-what-does-somebody-come-here-to-do)
+12. [Troubleshooting](#12-troubleshooting)
 
 ---
 
@@ -90,6 +91,20 @@ first run downloads Chromium once (~130 MB) into a shared cache and never again.
 If you drive Claude Code, Cursor, Windsurf, VS Code + Copilot or any MCP-capable agent, it can
 run all of this for you.
 
+### Step 4.0 — Give it the instructions first
+
+Before the server, one command. It costs nothing and removes most of the back-and-forth:
+
+```bash
+npx -p @ia-qa/qa-discovery ia-qa-discover skill --install
+```
+
+That writes `.claude/skills/ia-qa-discover/SKILL.md` — which verb answers which question, what
+each refusal means (`no-surface`, `no-healing`, `no-run` are three different sentences), what
+it must never do, and how to set an API key up without ever holding one. Agents read
+`.claude/skills/`, never `node_modules`, so the file has to land there to exist at all.
+Use `--print` to read it yourself first, or `--user` to install it for every project.
+
 ### Step 4.1 — Add the server to your agent
 
 Claude Code:
@@ -420,7 +435,75 @@ Without a key, nothing changes: every command above behaves exactly as it does t
 
 ---
 
-## 11. Troubleshooting
+## 11. Optional: what does somebody come here to *do*?
+
+Section 10 asks what an entry point **is**. This asks something a DOM cannot answer at all:
+**what is a person trying to do on this page, and what can they no longer do if it breaks.**
+
+```bash
+npx ia-qa-discover-ai plan
+```
+
+With no flags it reads exactly the pages section 8 said no test visits, **in the order that
+verb ranked them**. If you stop it halfway, the pages it already read are the ones that
+mattered most. No suite tracked yet? It reads every page and tells you that is what it did —
+you are the reader this package was written for.
+
+What you get, per page:
+
+```
+· devtools-environments
+  A person comes here to switch API keys and model configurations for different
+  environments like development, production and staging. · If it breaks: that person
+  cannot change or manage the API keys and model configurations. (confidence 0.90)
+```
+
+That page has **zero form fields**. The deterministic half could only say `nothing observed
+says it matters (2 API calls)` — it had nothing to observe. This is the gap the layer exists
+for, and the honest flip side is that on a page of articles the same model writes "somebody
+comes here to read the articles", which repeats the capture back at you. **Read the
+confidence and the citations, not the sentence.**
+
+### It sends less than `classify`, deliberately
+
+No selectors and no observed API calls leave for this question — the most sensitive half of a
+capture, and technical noise for a question about people. The notice it prints before the
+first request is derived from the function that builds the payload, so it cannot promise less
+than it sends.
+
+**Your footer does not leave either.** Headings that appear across your whole app — `Legal`,
+`Contact`, a nav title — are the shell, not this page's content, and they are dropped from
+what goes. That started as a privacy-shaped decision and turned out to be a quality one: on a
+real run, the reading of a page whose whole interest was `🗂️ Environment Manager` had been
+propped up by three footer links. They resolve, so nothing rejected them, and they support
+nothing. On the same run after the change: **zero citations pointing at a footer or a nav.**
+
+And a reading has to stand on something: at least one citation must be a level-1 or level-2
+heading, a form, or a field. One built entirely out of nav labels is dropped — unless the page
+offers no such anchor at all, in which case the rule is skipped rather than punishing the page
+for its own markup.
+
+### Three things it will not do
+
+- **It will not rank anything.** The order is the one `coverage` measured. A model reordering
+  a measured list has stopped commenting on a measurement and started replacing it.
+- **It will not gate.** Nothing here has an exit code that fails a build.
+- **It will not hide its origin.** `plan.json` carries the model and the time it ran;
+  `_overview.md` and the HTML report show the section labelled `model-read`; a page
+  re-captured *after* it was read is marked **stale**; and the report's footer stops claiming
+  the document is LLM-free, because it no longer is.
+
+If the capture genuinely does not say, the model answers `unclear` — kept and labelled, never
+quietly dropped.
+
+```bash
+npx ia-qa-discover-ai plan --all            # the tested pages too
+npx ia-qa-discover-ai plan --report --open  # the branded dossier, in a browser
+```
+
+---
+
+## 12. Troubleshooting
 
 **"Found 1 page" on an app I know has twenty.** Section 7, case 2 — the app navigates without
 links. Declare the routes in `config.json` → `pages`.

package/dist/ai/classify.d.ts

@@ -2,6 +2,7 @@ import type { AiProvider } from '@ia-qa/self-healing';
 import type { PageCapture } from '../capture/page';
 import { type ToolTrigger } from '../taxonomy';
 import { type CaptureIndex, type Citation, type CitationVerdict } from '../citations';
+import { providerError, type ModelOptions } from './model';
 /**
  * F2 — classify each entry point, offline, from F1's capture.
  *
@@ -12,22 +13,18 @@ import { type CaptureIndex, type Citation, type CitationVerdict } from '../citat
  * **On duplication, stated because this project punishes silent drift:** the
  * provider/model catalogue is imported from `@ia-qa/self-healing` rather than
  * copied — there must not be a third model list in this repo. The HTTP transport
- * below *is* a second implementation, because self-healing does not export its
- * own (`callModel` is internal). Any change to a provider's request shape owes an
- * edit here as well as there.
+ * now lives in `./model.ts`, which is a second implementation of something
+ * self-healing keeps internal (`callModel`); it moved out of this file when a
+ * second BYOK verb arrived, so the cost stays at two copies instead of three.
+ * Any change to a provider's request shape owes an edit there as well as in
+ * self-healing.
  */
 /** The document  writes, and the two strings a later stage keys on. */
 export declare const CLASSIFICATION_SCHEMA = "qa-discovery-classification@1";
 export declare const CLASSIFICATION_FILENAME = "classification.json";
-export interface ClassifyOptions {
-    provider: AiProvider;
-    model: string;
-    apiKey: string;
-    /** Injected in tests; defaults to the global fetch. */
-    fetchFn?: typeof fetch;
-    timeoutMs?: number;
-    temperature?: number;
-}
+/** The transport's options, named for this verb. One shape for every BYOK caller. */
+export type ClassifyOptions = ModelOptions;
+export { providerError };
 /** One thing on a page that takes input, addressed by a path a citation can resolve. */
 export interface EntryPoint {
     /** e.g. `pages/checkout.json#forms[0]` or `pages/checkout.json#looseFields[2]` */
@@ -125,9 +122,9 @@ export declare function collect(raws: Partial<Classification>[], index: CaptureI
  * What leaves this machine, said before it leaves — the one thing that makes
  * "everything is local except the model call" a promise rather than a claim.
  *
- * It lives beside `HOSTS` on purpose: the notice names the very constant
- * `buildRequest` sends to, so a new provider cannot be reachable while the
- * notice still lists three. And it is derived from `buildPrompt` — every line
+ * It resolves the host through `egressHost` rather than naming one of its own: the notice names the
+ * very constant `buildRequest` sends to, so a new provider cannot be reachable
+ * while the notice still lists three. And it is derived from `buildPrompt` — every line
  * below is a field that function actually puts in the payload; adding one there
  * owes a line here.
  *
@@ -146,27 +143,7 @@ export declare function collect(raws: Partial<Classification>[], index: CaptureI
  * the token here. The word "full" is gone from that line for the same reason the
  * allowlist is named: the two statements must not contradict each other.
  */
-export declare function egressNotice(provider: AiProvider, model: string, pages: number, planned?: boolean): string;
-/**
- * What the provider itself said went wrong.
- *
- * Found by execution, not review: a Gemini key that is simply wrong comes back
- * **400**, not 401 — so the previous message read `refused the call with HTTP
- * 400`, said nothing about the key, and left the one person who could fix it in
- * sixty seconds with nothing to go on. Mapping more status codes by hand is the
- * wrong shape of fix (the list rots, and it is different per provider); the
- * providers already answer the question in the body, and all three happen to
- * use the same `{error:{message}}` envelope.
- *
- * Two guards. The body is untrusted text that lands in a terminal and in
- * `classification.json`, so it is capped and flattened. And anything shaped like
- * a credential is redacted before it is printed — a provider that echoes the
- * request back would otherwise put the user's own key in a file they commit.
- */
-export declare function providerError(res: {
-    status: number;
-    text: () => Promise<string>;
-}): Promise<string>;
+export declare function egressNotice(provider: AiProvider, model: string, pages: number, planned?: boolean, baseUrl?: string): string;
 /**
  * One page in, validated classifications out. Returns an empty result rather
  * than throwing on a network failure, an unparseable reply or an empty page: