@ia-qa/qa-discovery 0.4.0 → 0.5.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ia-qa/qa-discovery",
-  "version": "0.4.0",
+  "version": "0.5.0",
   "description": "Deterministic surface reconnaissance for an unknown web app — crawl, capture, diff. No LLM, no hosting: runs entirely on your machine, with a local MCP server for agents.",
   "keywords": [
     "qa",
@@ -35,6 +35,7 @@
   },
   "files": [
     "dist",
+    "skills",
     "README.md",
     "TUTORIAL.md",
     "ROADMAP.md"
@@ -49,7 +50,7 @@
     "node": ">=18.0.0"
   },
   "dependencies": {
-    "@ia-qa/self-healing": "^1.17.0"
+    "@ia-qa/self-healing": "^1.19.0"
   },
   "peerDependencies": {
     "playwright": ">=1.40.0"

package/skills/ia-qa-discover/SKILL.md

@@ -0,0 +1,166 @@
+---
+name: ia-qa-discover
+description: >-
+  Find out what a web application contains when no test suite exists yet — its pages, forms,
+  fields and the API calls they make — and state what an existing suite does NOT test. Use
+  when asked to explore an unknown app, to plan testing for one, to list its entry points, or
+  to answer "which pages are untested". Requires @ia-qa/qa-discovery (`ia-qa-discover`). Runs
+  entirely on the user's machine; the optional `-ai` binary is the only part that sends
+  anything anywhere.
+---
+
+# ia-qa-discover — read the capture, never re-crawl
+
+## 1. The capture is on disk. Read it before opening anything.
+
+`ia-qa-discover scan` writes `.ia-qa-discovery/capture/<page>.json`, one file per page:
+
+```json
+{
+  "page": "checkout", "url": "https://app/checkout", "capturedAt": "2026-09-19T…",
+  "meta": { "title": "Checkout", "description": "…" },
+  "headings": [{ "level": 1, "text": "Your order" }],
+  "forms": [{ "method": "post", "hasSubmit": true,
+             "fields": [{ "name": "card", "type": "text", "label": "Card number",
+                          "required": true, "stableSelector": true }] }],
+  "looseFields": [], "apiCalls": [{ "method": "POST", "urlPattern": "/api/pay", "status": 200 }]
+}
+```
+
+Most questions are answered by reading those files and `capture/_overview.md`. Re-running a
+scan costs minutes and a browser; reading costs nothing. Scan again only when the app has
+changed since `capturedAt`.
+
+| Path | What it is |
+|---|---|
+| `.ia-qa-discovery/capture/` | one JSON per page + `_overview.md` (the human index) |
+| `.ia-qa-discovery/capture/surface.json` | the manifest, including **what the scan could not see** |
+| `.ia-qa-discovery/coverage-map.json` | the gap, once `coverage` has run |
+| `.ia-qa-discovery/classification.json`, `plan.json` | the optional model readings, if any |
+| `.ia-qa-discovery/history.jsonl` | one line per scan |
+
+## 2. Which verb answers which question
+
+| The user's question | Verb |
+|---|---|
+| "what is even in this app?" | `scan <url> --save` |
+| "most of the inputs are behind a menu or a tab" | `scan --deep` — opens disclosures one level and captures what they reveal |
+| "the app is behind a login" | the human runs `ia-qa-heal login` (from `@ia-qa/self-healing`), then `scan --session .ia-qa/session.json`. **You cannot log in** (§4.1) |
+| **"what does my test suite not test?"** | `coverage` — needs `@ia-qa/self-healing` **and** one watched run (§3) |
+| "what changed since last time?" | `history` |
+| "what is each entry point FOR?" | `ia-qa-discover-ai classify` — optional, BYOK (§5) |
+| "what does somebody come to this page to DO?" | `ia-qa-discover-ai plan` — optional, BYOK (§5) |
+| "write me a starting spec for an untested page" | `generate` — scaffolding only; the assertions stay human |
+
+Exact flags: `ia-qa-discover <verb> --help`. Do not guess them from this file.
+`scan`, `coverage`, `history` and both `-ai` verbs take `--json`. Prefer it over parsing prose.
+
+## 3. This tool issues NO verdict, and that is the point
+
+Nothing here exits non-zero because of what it found. A gap is a decision with a human's name
+on it — an uncovered page may be deliberately out of scope.
+
+**`coverage` refuses rather than guesses**, with a typed reason. Report the refusal as *no
+measurement*, never as a finding:
+
+| `reason` | What it means | What the human does |
+|---|---|---|
+| `no-surface` | nobody has scanned this app | `ia-qa-discover scan` |
+| `no-healing` | no suite is tracked here | install `@ia-qa/self-healing`, then `map` |
+| `no-run` | contracts exist, no run was ever watched | `ia-qa-heal run`, or `ia-qa-pal tour --suite` |
+
+`measured: false` is **not** "the suite covers nothing". Saying that is the single worst
+sentence this tool can cause to be written.
+
+## 4. Never do this
+
+1. **Never run `ia-qa-discover login` or `ia-qa-heal login`.** They open a visible browser and
+   wait for a person, twice over: in your shell that is a hung command. Give the human the
+   command instead. The same goes for `ia-qa-heal ui`.
+2. **Never present a contract, or a configured page, as proof a page is tested.** A contract
+   means somebody configured that page. Only capture-during-run proves a test went there.
+3. **Never report a page count as coverage.** Read `surface.json` → `coverage` first: a login
+   wall (`loginWall`), an app that navigates without `<a href>` (`hrefBlind`) and the page cap
+   (`droppedByCap`) each make the surface look smaller than it is. State those beside any total.
+4. **Never treat `writeCalls` as evidence a page writes.** A scan never submits anything, so a
+   non-GET it observed fired at page load — measured on a real corpus, 10 of 10 were analytics
+   beacons. It is in the output as a fact and is deliberately not a ranking factor.
+5. **`named` is not `asserted`.** The depth figure says the suite *names* N contracted elements.
+   A test that clicks a button names it without checking anything.
+6. **Never call the `-ai` layer a gate.** It emits no verdict, and `unclear` / `refused` are
+   answers needing a person, not a retry. A `failed` page means the model call did not succeed
+   — say so; it is not a finding about the app.
+7. **Never widen `capture.safeQueryParams`** to make a scan "work". The deny rule on
+   `token`, `password`, `session`, `auth`, `jwt`, `email`, `key` exists so a reset link pasted
+   into `scan` does not end up in files the user commits.
+8. **Never edit `config.json` to add a page you have not seen in a capture.** `discover --apply`
+   (heal) and the config are the human's; propose, do not write.
+
+## 5. What leaves the machine
+
+Nothing, except: requests to the app the user asked you to scan (same origin, GET only, never
+a `/logout` or `/delete` URL), and — only if the user opted in — `ia-qa-discover-ai` calling
+**their own** LLM provider.
+
+The two `-ai` verbs announce what they send *before* they send it, on stderr, in every mode.
+Read that announcement to the user before running either on a private or authenticated app.
+`plan` sends strictly less than `classify`: no selectors, no observed API calls, and not the
+headings the whole app repeats.
+
+## 6. Setting up the BYOK key — ask, never invent
+
+Both `-ai` verbs need a provider key. The key **never** goes in a file.
+
+1. Ask the user which provider they have a key for: `anthropic`, `openai`, `google`, or
+   `openai-compatible` — that last one is a *shape*, not a vendor: DeepSeek, Groq, Mistral,
+   OpenRouter, Together, vLLM and Ollama all serve the same endpoint, and it needs a
+   `baseUrl`. A `localhost` URL there means the call never leaves their machine.
+   `ia-qa-discover-ai models` lists the models each one takes. Never pick for them.
+2. **Prefer the machine's credential store.** Give them the command; they run it themselves,
+   in their own terminal — it asks for the value and refuses an argument or a pipe:
+
+   ```bash
+   npx -p @ia-qa/self-healing ia-qa-heal secret set MY_KEY_NAME
+   ```
+
+   You can confirm it resolves without ever seeing it: `ia-qa-heal secret check MY_KEY_NAME`.
+   **There is no command that prints a stored value, and you must not go looking for one.**
+   The point of the store is that the value stops being ambient — it is no longer in the
+   environment every process you launch inherits.
+
+3. Otherwise — or in CI, where there is no store and no person — ask them to export it in
+   their own shell and to tell you the **variable name**, never the key. If they paste the key
+   to you anyway, tell them to rotate it.
+
+4. Write only the reference into `.ia-qa-discovery/config.json`:
+
+```json
+"ai": { "provider": "openai", "model": "gpt-4o-mini",
+        "apiKey": { "source": "keychain", "key": "MY_KEY_NAME" } }
+```
+
+`"source": "env"` is the same shape with the variable name, for CI.
+
+5. `--dry-run` first: it shows exactly what would be sent, and to whom, without a key and
+   without calling anything. Show that to the user before the real run.
+
+That file is committed. A value there is a leaked credential, and the CLI refuses it — it
+detects a key-shaped string where a variable name belongs, prints it masked, and tells the
+user to rotate it. Do not work around that refusal.
+
+The CLI reads `process.env` only and never loads `.env` files. If their key lives in `.env`:
+`npx dotenv-cli -- ia-qa-discover-ai plan`.
+
+Through the MCP server the same rule holds: `api_key_env` takes the **name** of a variable on
+that machine. A raw key is never a tool argument — a tool call can be logged and replayed.
+
+## 7. The two siblings, and where the line is
+
+`@ia-qa/self-healing` (`ia-qa-heal`) repairs an **existing** suite's locators. This package
+describes an app that may have **no** suite. They share the session file and the page-naming
+rule, and they hand off in exactly two places: when `ingest` finds no test file, and when
+`coverage` needs a watched run. `@ia-qa/pal` (`ia-qa-pal`) runs the whole round for a team.
+
+`scan` and `map` are **not** the same operation: `map` writes a contract (every interactive
+element, to repair broken locators); `scan` writes a surface (what exists and what it takes as
+input, to decide what to test when there is no suite yet).