@ia-qa/qa-discovery 0.1.0

package/README.md

@@ -0,0 +1,378 @@
+# @ia-qa/qa-discovery
+
+**You have been handed a web app and no documentation. What is in it, and where would testing it start?**
+
+`ia-qa-discover` crawls an app you can reach and writes down what it actually observes: every page it could load, the headings that say what the page is for, every input the app takes, and every API call the browser really made. Deterministic — no LLM decides anything here. Local — it runs a headless browser on your machine and nothing is uploaded, ever. (One separate, optional binary, [`ia-qa-discover-ai`](#ia-qa-discover-ai--optional-byok-and-the-only-thing-here-that-sends-anything), calls a model with your own key; it is the only thing in this package that sends anything anywhere, and it says so before it does.)
+
+It answers *what is there*. It does not yet tell you what to test, and it never returns a pass/fail verdict: this is reconnaissance, and pretending otherwise would be the most useful-looking lie a tool like this could tell.
+
+**Where this is going.** Reconnaissance is step one of a longer goal: reproduce what a senior QA engineer does when handed an unfamiliar app — work out what it is *for*, where it would hurt most if it broke, and produce a prioritised test plan you can question line by line. Every stage after this one reads the capture below and must cite it, because a plan nobody can interrogate is a plan nobody can trust. See [ROADMAP.md](./ROADMAP.md) for the arc and for what this will deliberately never become.
+
+```bash
+npx @ia-qa/qa-discovery scan https://your-app.example.com --save --report --open
+```
+
+[![The qa-discovery report run on ia-qa.com itself: 14 pages captured with their titles, headings, input fields and API calls, followed by the API origins the app really contacted and which fields no test could locate durably](https://www.ia-qa.com/media/discover-report-poster.png)](https://www.ia-qa.com/media/discover-report-full.png)
+
+▶ **[See the full report](https://www.ia-qa.com/media/discover-report-full.png)** — one self-contained HTML file, written by `--report`. That is `ia-qa.com` scanned by its own tool, and it found something: all 37 of its input fields carry no `data-testid`, no `id` and no `name`, so any test written against them can only use a positional selector.
+
+---
+
+## What a real session looks like
+
+Every line below is genuine output from scanning a real React app (11 routes, most of them behind a login). Nothing here is a mock-up.
+
+**1. First run, anonymous.** It finds three pages — and refuses to let that read as success:
+
+```console
+$ ia-qa-discover scan https://app.example.com --save --report
+Discovering pages from https://app.example.com…
+🌐 Using bundled Chromium (shared Playwright cache)
+  visited / (2 links)
+  visited /terms (1 links)
+  visited /policy/cookies (0 links)
+Found 3 page(s). Capturing…
+  ✔ home
+  ✔ terms
+  ✔ policy-cookies
+
+✔ 3 of 3 page(s) captured
+  + added:   home, policy-cookies, terms
+
+⚠  This app has a login (home), and no session was used.
+   What you have is the PUBLIC surface only — whatever is behind the login was never reached.
+   Run `ia-qa-discover login` to log in by hand once, then scan again.
+
+Written to .ia-qa-discovery/capture/surface.json
+Readable index: .ia-qa-discovery/capture/_overview.md
+HTML report:    ia-qa-discover-report.html
+```
+
+**2. Log in by hand, once.** A visible browser opens; you do whatever the app asks.
+
+```console
+$ ia-qa-discover login
+
+🔑 ia-qa-discover login
+   Opens a visible browser at https://app.example.com/.
+   Log in there however your app asks — SSO, MFA, a consent screen, all of it.
+   Nothing is typed for you, and no credential is read or stored by this tool.
+
+   When you say you are done, the browser session is saved to:
+     .ia-qa-discovery/session.json
+   That file holds live session cookies: anyone who has it is logged in as you.
+
+Open the browser now? [Y/n]
+Logged in? (leave the browser open — press Enter here when the app is showing)
+
+✅ Session saved — 7 cookies, 1 origin with local storage.
+```
+
+**3. Scan again — and hit the *second* way a page count lies.** The app navigates with React Router's `useNavigate()`, so there is not one `<a href>` to follow:
+
+```console
+$ ia-qa-discover scan
+🔓 Reusing the saved session (.ia-qa-discovery/session.json).
+  visited / (0 links)
+Found 1 page(s). Capturing…
+  ✔ home
+
+✔ 1 of 1 page(s) captured
+
+⚠  No internal links were found, so discovery could not see past the entry URL.
+   "1 of 1 found" here means ONE page — not the whole app.
+   Discovery follows <a href>. An app that navigates programmatically (React Router's
+   navigate(), a button with an onClick handler) exposes no href for a crawl to follow.
+   Declare the routes you know in `.ia-qa-discovery/config.json` → "pages", and scan again.
+```
+
+**4. Declare the routes** (they were sitting in the router file) and get the real surface:
+
+```console
+$ ia-qa-discover scan --report
+🔓 Reusing the saved session (.ia-qa-discovery/session.json).
+Found 9 page(s) (9 declared in config.pages). Capturing…
+  ✔ dashboard        ✔ admin-dashboard   ✔ feature-manager
+  ✔ user-dashboard   ✔ billing           ✔ marketplace
+  ✔ smart-tools      ✔ terms             ✔ policy-cookies
+
+✔ 9 of 9 page(s) captured
+  + added:   admin-dashboard, billing, dashboard, feature-manager, marketplace, …
+
+🔓 Scanned with the saved session (.ia-qa-discovery/session.json).
+```
+
+**What that scan then told us about the app** — three things nobody had noticed, each traced to an observation:
+
+- All 9 pages share one `<title>`. The tab, the browser history and a screen reader announcing a route change cannot tell them apart.
+- `GET /api/integration/status` → **401** and `GET /api/billing/payment-history` → **404**, on a valid session.
+- Every input field on the login form has no `id`, no `name` and no `data-testid` — so any test written against it can only use a positional selector, which breaks the day a field is inserted above it.
+
+**5. Re-run any time.** An unchanged app must diff clean — that is the whole point of committing `capture/`:
+
+```console
+$ ia-qa-discover scan
+✔ 9 of 9 page(s) captured
+  (no change since the last scan)
+
+$ ia-qa-discover history
+2026-09-04T12:19:39Z  9/9 pages  +9 -1 ~0   300df55 (main)
+2026-09-04T12:33:48Z  9/9 pages  +0 -0 ~9   300df55 (main)
+2026-09-04T12:35:06Z  9/9 pages  +0 -0 ~0   (no change)  300df55 (main)
+```
+
+---
+
+## What you get
+
+```
+.ia-qa-discovery/
+  config.json              # baseUrl, crawl + capture settings, declared pages
+  history.jsonl            # one line per scan — the trend no single run can show
+  session.json             # only if you ran `login`. Live cookies: gitignored automatically
+  capture/
+    surface.json           # the manifest — machine-readable index, the entry point for tooling
+    _overview.md           # the same scan, readable — start here as a human (or an LLM)
+    _shared-calls.json     # calls that fire on most pages (session refresh, telemetry)
+    <page>.json            # one capture per page
+ia-qa-discover-report.html # optional branded HTML dossier (--report)
+```
+
+`capture/` is meant to be committed. It diffs cleanly in git, and re-running the scan tells you what moved.
+
+### One page's capture
+
+```jsonc
+{
+  "schema": "qa-discovery-page@1",
+  "page": "checkout",
+  "url": "https://app.example.com/checkout",
+  "source": "declared",              // "crawl" | "sitemap" | "declared"
+  "capturedAt": "2026-09-04T12:00:00.000Z",
+  "http": { "status": 200, "redirected": false },
+  "meta": { "title": "Checkout", "description": "…", "lang": "en" },
+  "headings": [{ "level": 1, "text": "Checkout" }],
+  "forms": [{
+    "selector": "form#checkout",
+    "method": null,                  // null = the DOM states none (a JS-handled submit)
+    "action": null,                  // never synthesized from the page URL
+    "hasSubmit": true,
+    "fields": [
+      { "type": "email", "name": "email", "selector": "#email",
+        "stableSelector": true, "required": true }
+    ]
+  }],
+  "looseFields": [                   // inputs NOT inside any <form> — most React apps
+    { "type": "search", "name": "q", "selector": "#sidebar > input:nth-of-type(1)",
+      "stableSelector": false, "required": false, "context": "main › Filters" }
+  ],
+  "apiCalls": [
+    { "method": "GET", "urlPattern": "/api/cart?page=2", "sameOrigin": true,
+      "origin": "https://app.example.com", "status": 200, "resourceType": "fetch", "count": 1 }
+  ]
+}
+```
+
+Three fields carry a promise worth knowing about:
+
+- **`stableSelector: false`** — the field has no `data-testid`, no `id`, no `name`. The selector given is positional: it resolves *today* and breaks the moment a field is inserted above it. That is a fact about the app, not a caveat about this tool, and it is worth fixing before a suite is written against it.
+- **`ambiguousSelector: true`** — rarer, and stronger: no selector could be made to resolve to exactly one element. Nothing downstream may act on that field. Uniqueness is *verified in the page*, never assumed.
+- **`method: null` / `action: null`** — the form states neither. A form with no `action` submits to the current URL per the HTML spec, but a React form usually has none because submission never reaches the network. Reporting the page URL there would invent a POST target the app never declared.
+
+---
+
+## Commands
+
+### `scan [url]`
+
+Crawl and capture. With `--save`, the URL is remembered and later runs are just `ia-qa-discover scan`.
+
+| Flag | |
+|---|---|
+| `--depth <n>` | crawl link-depth (default 2) |
+| `--max <n>` | page cap (default 60) |
+| `--no-reveal` | do not open menus/dropdowns looking for hidden links |
+| `--strict-host` | treat `www.` and the apex as different hosts |
+| `--session <file>` | reuse an existing Playwright `storageState` |
+| `--no-network` | skip API/network capture this run |
+| `--network-threshold <n>` | share threshold for promoting a call, 0-1 (default 0.6) |
+| `--save` | persist this baseUrl to `config.json` |
+| `--report [file.html]` | write the HTML dossier (default `ia-qa-discover-report.html`) |
+| `--open` | open that report in a browser |
+| `--json` | the whole result as one parseable document on stdout |
+
+### `login`
+
+Opens a **visible** browser, you log in by hand, and the session is saved for `scan` to reuse.
+
+This verb models nothing on purpose. A declarative `auth` block describes *one shape of login* — a username field, a password field, a submit button — and every app whose login is not that shape is out of reach: federated SSO, MFA, a consent screen, a magic link, a device check. Since you perform the login yourself, all of them work.
+
+It refuses under CI and without a TTY: it waits for a person twice over, so a pipeline that runs it just hangs. **An agent cannot run this** — it must ask you to. For automation, point `--session` at a `storageState` your own setup already writes.
+
+The file it produces holds live cookies — whoever has it is logged in as you. The path is announced before the browser opens, `.ia-qa-discovery/.gitignore` is written on the way out, and nothing here uploads it.
+
+### `history`
+
+The trend no single scan can reconstruct: pages, forms, API surface, and what changed run over run.
+
+### Global
+
+`--config <dir>` resolves `.ia-qa-discovery/` somewhere else (also `IAQA_DISCOVERY_CONFIG_DIR`).
+
+---
+
+## A page count is not coverage
+
+This is the failure mode this tool works hardest to avoid, because every version of it *looks* like success. Three checks exist for it, and each one fires in the terminal, in `_overview.md` and in the HTML report:
+
+**The login wall.** An app whose catch-all route renders the login page answers HTTP 200 at *every* URL. Without a check, a scan reports "3 of 3 pages captured" having captured the same wall three times. Detection is a password field observed in the DOM — inside a `<form>` or not.
+
+**Where the page list comes from.** Three sources, merged and deduped by path, and the report says which contributed what:
+
+1. **`sitemap.xml`** (falling back to `robots.txt`) — one HTTP GET, no browser, and the only source that does not care how the app navigates. Read first, and its hits also seed the crawl. `--no-sitemap` skips it. Apex and `www.` are folded together, so an apex sitemap against a `www.` baseUrl works — that mismatch is the most common way the two silently disagree.
+2. **The crawl** — follows `<a href>` from the entry URL, same-origin GET only, never a URL that acts (`/logout`, `/delete`…).
+3. **`config.pages`** — what you declare.
+
+⚠️ **A sitemap describes the *public* surface.** It is written for search crawlers, and crawlers are not logged in — so on an app whose real surface is behind a login, expect it to declare the marketing pages and nothing else. Measured on one: the sitemap listed 3 pages; the app had 11. Behind a wall, `login` plus `config.pages` is still the answer.
+
+And the cap is a budget, not a census: if more pages are known than `--max` allows, the scan says how many it left out rather than quietly shrinking the denominator.
+
+**Href blindness.** Discovery follows `<a href>`. An app that navigates programmatically — React Router's `useNavigate()`, a button with an onClick handler — exposes no href at all, so the crawl reaches exactly the entry URL and reports "1 of 1 found", i.e. 100%. Measured on a real app: 11 routes, zero `<Link>`, one page found. When zero internal links are harvested, the scan says so and points at `config.pages`.
+
+**Declared pages.** The way out of href blindness — the routes are usually sitting in your router file in plain sight:
+
+```jsonc
+{
+  "baseUrl": "https://app.example.com",
+  "pages": [
+    { "name": "billing",     "url": "/billing" },
+    { "name": "marketplace", "url": "/marketplace" }
+  ]
+}
+```
+
+Declared pages are scanned **in addition** to whatever the crawl finds, deduped by path — declaring some does not turn discovery off.
+
+**The loaded state is one state.** A page shows some of its input surface on load and hides the rest behind a click — a dialog, a tab, an accordion, a "create" button. Measured on a real app: one page carried 64 clickable controls and 2 visible fields, so a load-only capture described about 3% of its entry points while looking complete.
+
+Every scan therefore **counts those controls** and says so, whether or not you explore. `--deep` then opens them one level down and captures what they reveal, each field carrying the click that reaches it:
+
+```console
+$ ia-qa-discover scan --deep
+🔍 --deep opened 120 control(s): 2 field(s) found that are not visible at load.
+   ⚠ 2 page(s) hit the click budget, so their exploration is partial.
+```
+
+| Field | Type | Where / how to reach | Locatable |
+| --- | --- | --- | --- |
+| _(unnamed)_ | text | 🔍 click “$ search_tools…Ctrl+K” | ⚠ positional |
+
+Three things keep it safe and honest:
+
+- **Every non-GET request is blocked** for the duration of the walk, routed on the page — a click cannot mutate anything server-side. A name denylist is the second layer, for handlers that issue no request at all (a "log out" that just clears storage).
+- **One level only.** `@ia-qa/self-healing` measured its own equivalent: depth 1 finished in 59 s over 43 clicks; depth 2 took 183 s over 150 clicks and *never completed at any budget*. A permanently partial capture is a false green wearing a feature's clothes.
+- **A selector may not name two elements.** Exploration unions captures taken in different states, and a positional selector can address one element at load and another after a click. A revealed field whose selector is already spoken for is **dropped and counted** — reached but not captured is a coverage loss, and this tool states those rather than emit a selector that means two things.
+
+`--deep` roughly doubles scan time and is deterministic in practice: two consecutive runs over an unchanged app diff clean.
+
+---
+
+## Using it with `@ia-qa/self-healing`
+
+They are two halves of the same loop, and neither replaces the other.
+
+| | `@ia-qa/qa-discovery` | `@ia-qa/self-healing` |
+|---|---|---|
+| Question | *What is in this app?* | *My tests broke — did a selector move?* |
+| Needs | a URL | an existing test suite |
+| Output | app surface, no verdict | PASS / FIX / BLOCK, a CI gate |
+| Verb | `scan` | `map` |
+
+⚠️ **`scan` and `map` are not the same operation.** `map` writes a page *contract* — every interactive element by role and accessible name, for repairing broken locators. `scan` writes a *surface* — pages, headings, inputs, API calls, for deciding what to test. Do not expect one to produce the other.
+
+**Discovery → healing.** Once you know the routes, hand them to healing and start writing tests against contracts:
+
+```bash
+ia-qa-discover scan https://app.example.com --save   # find the surface
+ia-qa-heal init                                       # then map it for test maintenance
+```
+
+`config.pages` here has the same `{ name, url }` shape as self-healing's, and both packages resolve page names through the *same* function (`pageNameFromUrl`), so a page called `checkout` in one is `checkout` in the other. The two sets of artifacts line up 1:1.
+
+**Healing → discovery.** Healing knows what your suite *covers*. Discovery knows what *exists*. The gap between them is your real coverage — and neither tool can tell you that alone.
+
+**Sessions are interchangeable.** Both consume a Playwright `storageState`, so a session from either `login` verb works for the other: `ia-qa-discover scan --session ../.ia-qa/session.json`.
+
+---
+
+## MCP server
+
+```bash
+npx ia-qa-discover-mcp
+```
+
+A dependency-free JSON-RPC 2.0 stdio server exposing the **same** operations the CLI runs — a human and an agent get an identical contract, never a simplified view for one of them.
+
+- `scan_app` — crawl and capture. Check `loginWall` and `hrefBlind` in the result before reporting coverage.
+- `classify_app` — the optional BYOK layer below, for an agent. The key is read from an environment variable **named** in the call (`api_key_env`); a raw key is never a tool argument.
+- `discovery_history` — the trend.
+
+`login` is deliberately **not** a tool: it waits for a person at a browser, which an agent cannot be. Every path and URL an agent supplies is checked against the project root and the configured origin.
+
+---
+
+## `ia-qa-discover-ai` — optional, BYOK, and the only thing here that sends anything
+
+Everything above is deterministic and never leaves your machine. This one binary is the exception, and it is a separate binary precisely so that boundary is a thing you install rather than a flag you might forget.
+
+```bash
+ia-qa-discover-ai classify --dry-run   # see exactly what would be sent, and to whom
+ia-qa-discover-ai classify
+```
+
+It reads the capture **offline** — it opens no browser and never touches your app again — and asks a model what each entry point is *for* (`authentication`, `payment`, `search`, `data-entry`…) and how sensitive what it handles is. The vocabulary is closed and versioned: a label outside it is rejected before anything else is checked.
+
+**Every claim cites the capture, and every citation is resolved and checked.** A classification whose evidence does not exist, or does not say what it was claimed to say, is dropped before you see it, and the drop is reported. That proves the *premise*, never the conclusion — a model can cite a real password field and still be wrong about what the page is for — so read the `why` and the confidence, not the label alone.
+
+Three things it will not do: it is **not a gate** and emits no verdict; a `refused` entry point is an answer needing a person, not a retry; and a failed call is reported as a failed call, never as "this page has nothing on it".
+
+Configure it in `.ia-qa-discovery/config.json` — the file holds the *name* of the variable, never the key:
+
+```json
+"ai": { "provider": "anthropic", "model": "claude-haiku-4-5",
+        "apiKey": { "source": "env", "key": "ANTHROPIC_API_KEY" } }
+```
+
+Before the first request it prints what leaves and where it goes:
+
+```
+🌐 Sending 3 pages to api.anthropic.com  (anthropic · claude-haiku-4-5) — your key, your account.
+   Leaves this machine: each page's full URL, title and description, up to 25 headings,
+   every observed API call (method, normalised path, status), and each form field's name,
+   type, label and selector. Headings and labels are live text from your app.
+   Query values are stripped at capture EXCEPT an allowlist that includes `q` — a search
+   term reaches the model as typed (`capture.safeQueryParams` in config.json narrows it).
+   Does NOT leave: page HTML, screenshots, cookies or your session file, your test files,
+   your API key (sent as a header to api.anthropic.com only).
+```
+
+Read that list before pointing it at an authenticated or private app. The result lands in `.ia-qa-discovery/classification.json` **and** in `_overview.md`, which is rewritten so the reading appears next to the capture it was made from.
+
+---
+
+## Library
+
+```ts
+import { scanApp, readHistory } from '@ia-qa/qa-discovery';
+
+const result = await scanApp({ url: 'https://app.example.com', write: false });
+console.log(result.pagesCaptured, result.loginWall, result.hrefBlind);
+```
+
+## Requirements
+
+Node ≥ 18, and Playwright (`peerDependency`) with a Chromium available — `npx playwright install chromium`, or point at a browser you already have via `IAQA_BROWSER_CHANNEL=chrome`.
+
+## License
+
+MIT