@adia-ai/adia-ui-factory 0.8.1 → 0.8.2

package/skills/adia-verify/SKILL.md

@@ -3,7 +3,7 @@ name: adia-verify
 description: >-
   Browser-QA gate for adia-ui surfaces — renders headless with zero
   console/page errors, non-zero bounding boxes, and a screenshot actually
-  read, plus AdiaUI a11y checks. Use before shipping a surface, on
+  read, plus AdiaUI a11y checks. Use when shipping a surface, on
   "verify/QA this page", or when "tests pass" is the only evidence. NOT for
   composing/fixing UI (adia-compose) or structural lint (adia-lint hook).
 disable-model-invocation: false
@@ -27,7 +27,27 @@ Probe shape, the failure classes only this gate catches (0×0 host, empty-page-w
 
 ## Tooling boundary
 
-The advisory `adia-lint` PostToolUse hook mechanizes the structural slice on every write (shadow DOM, raw color/px, `::slotted`, native-primitive leaks, legacy shell shapes, SSR traps) and never blocks — fix its findings rather than re-deriving its rules. The framework's `audit:shell-composition` / `audit:native-primitive-leak` run in the @adia-ai framework repo; they are not shipped in this plugin and cannot be invoked from a consumer app. Everything else — the render, the screenshot read, a11y — is self-verified: no shipped script enforces it.
+The advisory `adia-lint` PostToolUse hook mechanizes the structural slice on every write (shadow DOM, raw color/px, `::slotted`, native-primitive leaks, legacy shell shapes, SSR traps, and the shell-nesting/LLM-key/genui-validate rules) and never blocks — fix its findings rather than re-deriving its rules. The framework's `audit:shell-composition` / `audit:native-primitive-leak` run in the @adia-ai framework repo; they are not shipped in this plugin and cannot be invoked from a consumer app.
+
+The render gate itself IS shipped: `node "${CLAUDE_PLUGIN_ROOT}/bin/adia-probe.mjs" <url> --selector <key-element> [--json]` runs gates 1–2 mechanically (console/page errors, bounding boxes, the `deviceScaleFactor: 2` capture — set at context creation, where it actually takes effect) and emits the VerifyProof below with verdict `pass-pending-read`. The screenshot READ and the a11y judgment slice stay yours — the probe's `imageRead` slot exists precisely so a report can't silently skip them. Requires Playwright in the consumer app (`npm i -D playwright`).
+
+## Deliverable — the VerifyProof
+
+The QA pass returns this record (the probe emits the mechanized half via `--json`; the reader completes it):
+
+```text
+VerifyProof
+url:            <the probed page>
+consoleErrors:  pass | fail — <each error verbatim; empty = pass>
+boundingBoxes:  pass | fail — <selector: w×h per key element; any 0×0 = fail>
+screenshot:     <path> @ deviceScaleFactor 2
+imageRead:      <what the pixels actually show — REQUIRED prose, never "looks fine">
+a11y:           pass | fail — region role/label · .open-driven overlays · heading roles · keyboard path · AA contrast
+structure:      adia-lint clean on every written file | <findings>
+verdict:        ship | hold — <one line naming the blocker if hold>
+```
+
+A VerifyProof with an empty `imageRead` or a bare "looks fine" is incomplete — the slot carries what was SEEN, so a later reader can challenge it.
 
 ## Verify rubric
 
@@ -39,5 +59,6 @@ A surface ships only when all pass:
 | Screenshot read `[gate]` | the `deviceScaleFactor: 2` capture was looked at, after the latest structural change |
 | Accessible `[gate]` | region role + label; `.open`-driven overlays; real heading roles; keyboard path; AA contrast |
 | Structurally clean `[gate]` | `adia-lint` reports no smells on every file written for the surface |
+| Data wired `[gate]` | the surface's Wiring Record (adia-data §Deliverable) exists with every round-trip row observed — a screen with unrecorded state wiring isn't done |
 
 Neighboring work routes out: composing or fixing the surface → `adia-compose` · shell chrome → `adia-shells` · hydration/state → `adia-data` · generated-markup validation → `adia-genui`.

package/skills/adia-verify/evals/routing-corpus.json

@@ -0,0 +1,171 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "name": "adia-verify routing accuracy corpus",
+  "version": "1.0.0",
+  "purpose": "Routing-eval corpus for adia-verify (adia-ui-factory plugin). Each phrase declares whether adia-verify SHOULD be the routing target. Scored by the estate's TF-IDF routing-eval runner against the skill's description (heuristic token overlap).",
+  "scoring_notes": "Heuristic signal, not ground truth. Treat misroutes as a prompt to tighten the skill description, never as a reason to keyword-stuff it. Real harness routing is LLM-driven.",
+  "license": "internal",
+  "scope": "adia-verify routing — does this phrase activate adia-verify (the consumer-app browser-QA exit gate), and does it correctly stay OUT of adia-release's release pre-flight gates, which is the one known central-corpus misroute for any 'verify'-flavored phrase?",
+
+  "minimums_per_spec": {
+    "trigger_phrases": 14,
+    "adversarial_phrases": 6,
+    "task_shapes_covered": "browser-gate, qa-request, a11y-check, debug-symptom, anti-pattern-tests-pass",
+    "adversarial_fraction": "30%"
+  },
+
+  "phrases": [
+    {
+      "id": "verify-gate-01",
+      "phrase": "verify this surface renders with zero console errors before we ship it",
+      "should_route_to_verify": true,
+      "expected_shape": "browser-gate",
+      "rationale": "Canonical pre-ship browser-render gate phrasing — matches the skill's own 'before shipping a surface' trigger."
+    },
+    {
+      "id": "verify-gate-02",
+      "phrase": "run the browser gate on the new settings page",
+      "should_route_to_verify": true,
+      "expected_shape": "browser-gate",
+      "rationale": "Direct 'browser gate' terminology from the skill's own name for its check."
+    },
+    {
+      "id": "verify-gate-03",
+      "phrase": "check that bounding boxes are non-zero on the dashboard before merging",
+      "should_route_to_verify": true,
+      "expected_shape": "browser-gate",
+      "rationale": "Non-zero bounding box is the gate's literal pass condition."
+    },
+
+    {
+      "id": "verify-qa-01",
+      "phrase": "verify/QA this page before we ship it",
+      "should_route_to_verify": true,
+      "expected_shape": "qa-request",
+      "rationale": "Verbatim 'verify/QA this page' phrase from the skill description."
+    },
+    {
+      "id": "verify-qa-02",
+      "phrase": "is this screen done? run the exit gate on it",
+      "should_route_to_verify": true,
+      "expected_shape": "qa-request",
+      "rationale": "'Is this screen done' — the exit-gate framing this skill owns."
+    },
+    {
+      "id": "verify-qa-03",
+      "phrase": "QA the checkout flow before it goes live for users",
+      "should_route_to_verify": true,
+      "expected_shape": "qa-request",
+      "rationale": "Consumer-app pre-launch QA request; deliberately avoids the word 'release' to not collide with adia-release."
+    },
+
+    {
+      "id": "verify-a11y-01",
+      "phrase": "run an accessibility pass on the profile page",
+      "should_route_to_verify": true,
+      "expected_shape": "a11y-check",
+      "rationale": "A11y check is one of the two gate halves this skill owns."
+    },
+    {
+      "id": "verify-a11y-02",
+      "phrase": "check AA contrast and the keyboard path on the new modal",
+      "should_route_to_verify": true,
+      "expected_shape": "a11y-check",
+      "rationale": "AA contrast + keyboard path are named rubric rows."
+    },
+    {
+      "id": "verify-a11y-03",
+      "phrase": "does this overlay have a real heading role and an aria-label on its region?",
+      "should_route_to_verify": true,
+      "expected_shape": "a11y-check",
+      "rationale": "Region role + heading role are named a11y-gate items."
+    },
+
+    {
+      "id": "verify-debug-01",
+      "phrase": "the screen renders completely blank, help me figure out why",
+      "should_route_to_verify": true,
+      "expected_shape": "debug-symptom",
+      "rationale": "'Empty page' symptom — a named failure class only this gate catches."
+    },
+    {
+      "id": "verify-debug-02",
+      "phrase": "this element reports a 0x0 bounding box, why isn't it sized",
+      "should_route_to_verify": true,
+      "expected_shape": "debug-symptom",
+      "rationale": "0x0 host is the skill's named failure class."
+    },
+    {
+      "id": "verify-debug-03",
+      "phrase": "the page looks broken visually but the console shows no errors",
+      "should_route_to_verify": true,
+      "expected_shape": "debug-symptom",
+      "rationale": "Clean-console-but-visually-broken is the anti-pattern this gate exists to catch."
+    },
+
+    {
+      "id": "verify-anti-01",
+      "phrase": "our unit tests pass, but I want someone to actually look at the rendered page before we call it done",
+      "should_route_to_verify": true,
+      "expected_shape": "anti-pattern-tests-pass",
+      "rationale": "Directly invokes the skill's named anti-pattern: 'tests pass, ship it'."
+    },
+    {
+      "id": "verify-anti-02",
+      "phrase": "confirm the surface actually renders and isn't just clipped — don't take the test suite's word for it",
+      "should_route_to_verify": true,
+      "expected_shape": "anti-pattern-tests-pass",
+      "rationale": "Clipped-content-despite-passing-tests is the gate's reason to exist."
+    },
+
+    {
+      "id": "verify-adv-01",
+      "phrase": "run the pre-flight verification gates before we cut the v0.8 release",
+      "should_route_to_verify": false,
+      "expected_alternative": "adia-release (forge plugin)",
+      "rationale": "Adversarial — the known central-corpus misroute: 'verify' language describing release pre-flight gates (F-N1, tag/publish checkpoints), not a consumer-app browser QA pass."
+    },
+    {
+      "id": "verify-adv-02",
+      "phrase": "run a dogfood sweep across every demo page in the framework repo",
+      "should_route_to_verify": false,
+      "expected_alternative": "adia-dogfood (forge plugin)",
+      "rationale": "Adversarial — framework-side demo sweep (visual probe, attr-quote typos, shell composition) is adia-dogfood's territory, not the consumer-app exit gate."
+    },
+    {
+      "id": "verify-adv-03",
+      "phrase": "the QA pass found the button is unstyled and misplaced — go fix the screen",
+      "should_route_to_verify": false,
+      "expected_alternative": "adia-compose (adia-ui-factory plugin)",
+      "rationale": "Adversarial — verify only reports the gap; fixing the composed screen routes to adia-compose (generator, not the gate)."
+    },
+    {
+      "id": "verify-adv-04",
+      "phrase": "the gate found the data isn't hydrating on load — fix the state wiring",
+      "should_route_to_verify": false,
+      "expected_alternative": "adia-data (adia-ui-factory plugin)",
+      "rationale": "Adversarial — fixing a hydration/state defect the gate surfaced belongs to adia-data, not to re-running verify."
+    },
+    {
+      "id": "verify-adv-05",
+      "phrase": "run the structural lint check on the files from my last commit",
+      "should_route_to_verify": false,
+      "expected_alternative": "adia-lint hook (advisory PostToolUse hook, not a skill)",
+      "rationale": "Adversarial — structural smells (shadow DOM, raw color/px, native-primitive leaks) are the adia-lint hook's mechanized slice; verify only checks that hook reports clean, it doesn't run the lint itself."
+    },
+    {
+      "id": "verify-adv-06",
+      "phrase": "write unit tests for the new date-parsing utility function",
+      "should_route_to_verify": false,
+      "expected_alternative": null,
+      "rationale": "Adversarial — generic unit-test authoring with no browser-render or a11y surface; no AdiaUI skill should activate."
+    }
+  ],
+
+  "evaluator_notes": {
+    "current_state": "20 cases (14 trigger + 6 adversarial). Each of the 5 task shapes has 2-3 trigger cases; adv-01 specifically targets the documented known misroute (verify-flavored phrase landing on adia-release).",
+    "promotion_criteria": "Promote to a CI gate (warn → hard-fail) when F1 ≥ 0.85 on this corpus across 3+ consecutive runs without description edits.",
+    "review_cadence": "Re-run on every adia-verify description edit. Add 1-2 new cases per checkpoint reflecting newly-observed routing failure modes.",
+    "known_limitations": "verify-adv-01 encodes the single highest-value adversarial for this skill: 'verify' as a bare word is release-gate-shaped in the central corpus, and the heuristic scorer may weight token overlap toward adia-release's own 'pre-flight gates' language. The description's browser/a11y-specific vocabulary (bounding boxes, console.error, aria-label) is the intended disambiguator."
+  }
+}

package/skills/adia-verify/references/verification.md

@@ -13,9 +13,16 @@ Render the surface in a real (headless) browser and assert four things, in order
 
 Gates 1 and 2 are independent, not redundant: `customElements.whenDefined(name)` never rejects, so a `Promise.all([...whenDefined]).then(bootstrap)` gate with one never-imported tag hangs forever — shell chrome still renders (tag-keyed CSS), the console stays **clean**, and the page is empty. Only the bounding-box and screenshot checks see it.
 
-Minimal probe shape (Playwright; works against a Vite static host or an SSR dev URL):
+The shipped probe runs this mechanically and emits the VerifyProof:
+`node "${CLAUDE_PLUGIN_ROOT}/bin/adia-probe.mjs" <url> --selector <el> --json`.
+Its logic, for when you need a bespoke variant (Playwright; works against a
+Vite static host or an SSR dev URL):
 
 ```js
+// deviceScaleFactor must be set at CONTEXT creation — scale:'device' on a
+// default context captures at 1x and silently fails gate 3's 2x claim.
+const context = await browser.newContext({ deviceScaleFactor: 2 });
+const page = await context.newPage();
 const errors = [];
 page.on('console', m => m.type() === 'error' && errors.push(m.text()));
 page.on('pageerror', e => errors.push(String(e)));

package/references/verification.md

@@ -1,35 +0,0 @@
-# Verification — the exit gate
-
-Mode-independent. A surface is done when it passes the **browser gate** + the a11y check — not when it compiles and not when unit tests pass. "Tests pass, ship it" is the anti-pattern: unit tests are necessary, not sufficient (happy-dom/vitest pass for components that render broken in real browsers).
-
-## The browser gate (the real gate)
-
-Render the surface in a real (headless) browser and assert four things:
-
-1. **Zero `console.error` / `pageerror` on load** — collect them during navigation; any is a failure.
-2. **Non-zero bounding boxes** on the key elements — catches the "upgraded but never sized" 0×0 host, where the element exists but renders nothing.
-3. **Read the screenshot** — capture at `deviceScaleFactor: 2` and actually look at it. Content can be present in the DOM yet visually clipped, overlapped, or off-canvas; only the pixels catch that. A probe that screenshots but doesn't read it hasn't verified anything.
-4. **Re-probe after every structural change** — a stale screenshot lies.
-
-Minimal probe shape (Playwright; works against a Vite static host or an SSR dev URL):
-
-```js
-const errors = [];
-page.on('console', m => m.type() === 'error' && errors.push(m.text()));
-page.on('pageerror', e => errors.push(String(e)));
-await page.goto(url, { waitUntil: 'networkidle' });
-const box = await page.locator('my-surface').boundingBox();   // expect non-zero w/h
-await page.screenshot({ path: 'probe.png', scale: 'device' }); // then READ probe.png
-// gate: errors.length === 0 && box.width > 0 && box.height > 0 && (you read the image)
-```
-
-## Diagnosing the "renders empty, zero console errors" page
-
-`customElements.whenDefined(name)` never rejects — if `name` is never imported, its Promise never resolves, so a `Promise.all([...]).then(bootstrap)` gate hangs forever on one dead await. Shell chrome still renders (tag-keyed CSS), the page body is just empty, and nothing errors. Check every awaited tag actually has a registering import before suspecting anything else.
-
-## Accessibility (the adia-ui-specific checks)
-
-Standard a11y applies (labelled landmarks, keyboard paths, AA contrast). The kit-specific traps:
-
-- **Overlays** — drive `<modal-ui>` / `<drawer-ui>` via the `.open` property; a hardcoded `open` attribute on a `showModal` overlay bricks the whole page, and only a live click/`elementFromPoint` probe catches it.
-- **Roles** — a presentational `text-ui variant="heading"` needs a real heading role or `<h*>` wrapper; don't use deprecated `aria-grabbed`.