@adia-ai/adia-ui-forge 0.1.3 → 0.8.0

package/skills/adia-ui-authoring/assets/case-studies/maxtokens-32768-discovery.md

@@ -1,109 +0,0 @@
-# Case study — Mode 6: `maxTokens: 32768` discovery
-
-**Scenario:** [llm-bridge.md](../../references/llm-bridge.md) § Rule 1 — defaults that bit us **Source:** silent-fallback diagnosis in the LLM bridge; commit history at `packages/llm/llm-bridge.js` **Outcome:** A2UI generation for moderately complex UIs was silently truncated at 8k tokens; the validator rubber-stamped incomplete trees at ~89/100. Raised the bridge default to 32768; surfaced `stopReason` as Rule 3.
-
----
-
-## §The shape
-
-The `@adia-ai/llm` bridge (`packages/llm/llm-bridge.js`) shipped with a default `maxTokens: 8192` — the conservative default common to provider docs. `complete()` and `stream()` both used it.
-
-The A2UI synthesis pipeline (`packages/a2ui/compose/`) generated UI JSON from intent. For simple intents — "a sign-up form," "a settings panel" — output fit comfortably under 4k. For moderately complex intents — "a kanban board with 4 columns," "a SaaS dashboard with 6 KPI cards + a chart," "a pricing tier table with 3 plans" — the JSON regularly approached or exceeded 8k.
-
-The validator scored every output. Reported scores were `~89/100`. Generation looked healthy. Eval coverage looked stable.
-
-Then a consumer complained: "the kanban came back missing the last column."
-
----
-
-## §The diagnosis
-
-Inspecting the truncated JSON: it ended mid-object, mid-string, with no closing brace. The validator's score-89 was a **lenient fallback**: when JSON parsing failed, the synthesizer wrapped the partial output in a generic shell template and the validator scored _that template_, not the broken JSON.
-
-The bridge wasn't propagating the truncation signal. Each provider's response carried a `stopReason` (`'max_tokens'` for Anthropic + Gemini's `'MAX_TOKENS'`, `'length'` for OpenAI) — but the bridge was normalizing all values to `'end'` and the downstream synthesizer never saw the truncation indicator.
-
-So the failure mode was:
-
-1. Bridge default `maxTokens: 8192` cuts the model off mid-output.
-2. Bridge normalizes upstream `stopReason: 'max_tokens'` to `'end'`.
-3. Synthesizer attempts JSON.parse → throws.
-4. Synthesizer falls back to "render a generic shell + warn."
-5. Validator scores the generic shell at ~89/100.
-6. Eval pipeline reports healthy coverage.
-
-Three independent bugs compounded into a silent regression. The generator looked fine until a consumer counted columns.
-
----
-
-## §The fix
-
-### Rule 1 graduate — `maxTokens: 32768` is intentional
-
-`llm-bridge.js` was bumped from `8192` to `32768`:
-
-```js
-// llm-bridge.js — complete() and stream() both pass this default
-const opts = {
-  maxTokens: 32768,           // not configurable from caller for synthesis
-  temperature,
-  cache: this.#provider === 'anthropic',
-  // ...
-};
-```
-
-32k is the model output ceiling for current Anthropic models (Sonnet + Haiku 4.5). Don't lower this. A2UI JSON for kanban / dashboard / pricing-tier outputs ranges 4-16k typically; 32k gives ~2x headroom.
-
-If adding a new provider with a lower output ceiling (e.g. an older model with 4k output limit), surface it as a **model-level constraint in `MODELS`**, don't bypass the bridge default.
-
-### Rule 3 graduate — `stopReason` must propagate raw
-
-The bridge stopped normalizing. Both `complete()` and `stream()`'s terminal `done` chunk now surface the raw upstream value:
-
-```js
-yield { type: 'done', text, usage, stopReason: rawStopReason };
-```
-
-Downstream synthesizer (`packages/a2ui/compose/...`) checks for truncation-indicating values:
-
-```js
-const TRUNCATED = new Set(['max_tokens', 'length', 'MAX_TOKENS']);
-if (TRUNCATED.has(chunk.stopReason)) {
-  throw new TruncationError(`output truncated by upstream (${chunk.stopReason})`);
-}
-```
-
-**Don't normalize all values to `'end'`** — the generator's truncation detector reads them raw. This is now Rule 3 in [llm-bridge.md](../../references/llm-bridge.md).
-
-### Validator hardened
-
-The validator was patched to refuse silent fallback rendering when `stopReason` indicates truncation. The 89/100 silent-pass became an `INCOMPLETE: truncated by upstream` hard fail. Eval coverage dropped ~12pp the day the patch landed — surfacing the previously-hidden generation gap.
-
----
-
-## §The verification
-
-After the 3 patches:
-
-- `npm run smoke:engines` — green (complex intents now succeed end-to-end)
-- `npm run test:a2ui` — passing (truncation-test cases added — artificially low `maxTokens` should produce `TruncationError`, not a 89/100 false-positive)
-- Real-LLM eval at `eval:diff` — coverage dropped from reported ~95% to the honest ~83%, then recovered to ~88% as corpus + prompts matured. Honest numbers across all subsequent rebaselines.
-- Kanban-intent + dashboard-intent + pricing-tier-intent — all now emit complete JSON without truncation.
-
----
-
-## §The lesson
-
-Three things graduated from this arc:
-
-1. **Silent fallbacks compound.** Truncation + stopReason normalization + generic-shell fallback + validator scoring stacked into a silent 12pp coverage lie. Every layer in a pipeline must surface failure honestly — generic fallbacks should fail-loud, not fail-friendly.
-2. **`maxTokens` ≠ "token budget."** Consumers think of `maxTokens` as "how big can the output be." The truth is it's "where the model will be cut off if it tries to keep going." Set it as generously as the model allows; surface model-level constraints separately if a specific model has a lower ceiling.
-3. **Provider-specific terminal values matter.** Anthropic / Gemini / OpenAI all use different stopReason strings (`max_tokens` vs `MAX_TOKENS` vs `length`). Normalizing them throws away information. The bridge surfaces them raw; the consumer maps them to behavior.
-
-Rule 1 (`maxTokens: 32768`) + Rule 3 (`stopReason` propagates raw) + the lazy-load adapter Rule 4 (caught later — `node:fs` imports break SSR consumers) are now the 5 hard rules in [llm-bridge.md](../../references/llm-bridge.md) § Hard rules — defaults that bit us.
-
-## §Cross-references
-
-- [llm-bridge.md](../../references/llm-bridge.md) § Rule 1 (`maxTokens: 32768` is intentional) + Rule 3 (stopReason must propagate)
-- `packages/llm/llm-bridge.js` — the bridge with the now-correct default
-- `packages/llm/adapters/anthropic.js` / `openai.js` / `gemini.js` — each adapter's `parseStream` surfaces the raw provider-specific `stopReason`
-- Spec: `docs/specs/package-architecture.md` § 11 (Phase 5 — engine registry; `@adia-ai/llm` is the 9th lockstep package, leaf-shaped, no internal deps)

package/skills/adia-ui-authoring/assets/case-studies/theme-panel-promotion.md

@@ -1,113 +0,0 @@
-# Case study — Mode 4: theme-panel promotion
-
-**Scenario:** [module-promotion.md](../../references/module-promotion.md) — lift duplicated theme controls into a single reusable web-module element **Source:** the canonical 5-phase inline → module instance **Outcome:** `<theme-panel>` element + new `theme/` cluster + 23 unit tests + 2 consumers migrated + chunk corpus refreshed. Three observation rules (OD-001/002/003) extracted from the arc.
-
----
-
-## §The shape
-
-The theme controls (color-scheme toggle, theme picker, density slider, radius slider, preset buttons) appeared in TWO consumers with non-trivial drift:
-
-- **`site/`** (docs shell) — full-fat: persistence to localStorage, prefers-color-scheme listener, computed-value read-back, preset buttons.
-- **`playgrounds/admin-shell/`** — stripped: no persistence, no prefers-color-scheme listener, **no computed-value read-back**.
-
-The stripped variant had a real behavior bug: clicking a theme reset the density + radius sliders to `1, 1` regardless of the theme's `[theme="ocean"] { --a-radius-k: 1.5 }` value. The sliders lied about the current state. The docs variant didn't have this bug because its controller read `getComputedStyle()` after the theme attribute write.
-
-Markup was duplicated across the two consumers (~80 lines each), plus controller JS (~120 lines each). Drift was structural; the playground "feature" was actually a controller-omission bug.
-
----
-
-## §The diagnosis
-
-Per [module-promotion.md](../../references/module-promotion.md) § When to use, the duplication qualified for the lift:
-
-1. A consumer-authored inline block appears in 2+ surfaces ✓
-2. The block has a stable, semi-public API surface (4–10 props) ✓ (color-scheme, theme, parametric, presets, persist, scheme-toggle — 6 props)
-3. The bug-surface from drift is real ✓ (playground sliders lie about computed values)
-
-Cluster placement (OD-001):
-
-| Option | Decision |
-| --- | --- |
-| **A. New cluster** | ✓ — theme/ cluster has ≥ 2 credible siblings foreseeable: theme-panel today, theme-toggle + theme-preview tomorrow |
-| B. Existing cluster | rejected — none of `simple/` / `runtime/` / `shell/` fits the product axis |
-
-Sibling-button absorption (OD-002): the docs shell had a separate `<button-ui id="theme-toggle">` flipping `color-scheme` NEXT TO the `<popover-ui>` containing the theme panel. **Two duplications could collapse to zero** if the module absorbed it as `[scheme-toggle]`.
-
----
-
-## §The fix
-
-### Phase 1 — Author the element (one commit)
-
-`packages/web-modules/theme/theme-panel/` — 7 files:
-
-- `theme-panel.yaml` — SoT (6 props: `value`, `theme`, `parametric`, `presets`, `persist`, `scheme-toggle`)
-- `theme-panel.js` — UIElement subclass; ~340 LOC; light-DOM; imperative DOM stamping (text-ui + segmented-ui + slider-ui + field-ui + button-ui children)
-- `theme-panel.css` — two-block `@scope` per component-token-contract.md
-- `theme-panel.html` + `theme-panel.examples.html` — standalone demo
-- `theme-panel.test.js` — 23 specs (target ≥ 15)
-- `theme-panel.a2ui.json` — GENERATED
-
-4 `package.json` touchpoints wired in one diff: `packages/web-modules/theme/index.js` (cluster barrel), `packages/web-modules/index.js` (root barrel), `packages/web-modules/package.json` `exports` + `files` + `sideEffects`.
-
-### Phase 2 — Migrate `site/` (primary consumer)
-
-Replaced ~80 lines of inline markup with:
-
-```html
-<popover-ui placement="bottom-end">
-  <button-ui icon="palette" variant="ghost" size="sm" slot="trigger"></button-ui>
-  <theme-panel slot="content" persist parametric presets scheme-toggle></theme-panel>
-</popover-ui>
-```
-
-Deleted controller blocks from `site.js` (~120 LOC). Confirmed via `grep -nE 'id="theme-default"|id="param-density"|id="preset-compact"' site/index.html site/site.js` — zero hits.
-
-**OD-002 applied**: retired the standalone `<button-ui id="theme-toggle">`; `[scheme-toggle]` attribute on the module renders an internal segmented-ui at the top of the panel.
-
-**LS-key migration shim** added to `site.js` boot script — one-shot read-old / write-new / drop-old for 4 keys (`adia-theme` → `adia-theme-scheme`, etc.). Idempotent across reloads; TODO removal date 60 days out.
-
-### Phase 3 — Migrate `playgrounds/admin-shell/`
-
-Same markup substitution **omitting `[persist]`** (playground stays ephemeral). Deleted controller blocks. Updated playground's `spec/SPEC.md` ASCII diagram.
-
-**OD-003 applied**: the playground's slider-lie bug auto-fixed — the module's internal controller calls `requestAnimationFrame(() => { ... getComputedStyle(target)... })` on theme-change, so sliders correctly read back computed values regardless of how the theme changed.
-
-### Phase 4 — Migrate chunk corpus
-
-`packages/a2ui/corpus/chunks/` re-harvested via `npm run harvest:chunks` (SOURCES included `site/pages`). The harvested HTML captured the new authoring shape.
-
-A hand-edit fallback path was also exercised on a playground chunk; a later re-harvest superseded it cleanly when `playgrounds/` was added to SOURCES — concrete proof of the "re-harvest may improve on your hand-edit" gotcha.
-
-### Phase 5 — Release
-
-Standard lockstep cut. `packages/web-modules/CHANGELOG.md` documented `<theme-panel>` + the new `theme/` cluster. Root `CHANGELOG.md` got the cross-cutting entry. The journal captured the DevTools-long-async-stack pitfall.
-
----
-
-## §The verification
-
-- `theme-panel.test.js` — 23/23 (stamp + connect; attribute reflection on all 6 props; primary user action — scheme/theme/density/radius changes; programmatic `.apply({ scheme: 'dark' })`; persistence on/off; disconnect cleanup; OD-003 computed-value read-back on theme-change)
-- `node scripts/build/components.mjs --verify` — clean (1 new module + a2ui.json sidecar)
-- `npm run test:a2ui` — 22 pass / 0 fail / 1 skipped (chunk loads with new shape)
-- `npm run smoke:chunks` — tolerated pre-existing failures; no new regressions
-- Manual visual QA at `npm run dev` — both consumers rendered the popover + panel correctly; theme-change + density-slide + preset buttons all worked
-
----
-
-## §The lesson
-
-Three observation rules graduated from this arc, all encoded into [module-promotion.md](../../references/module-promotion.md):
-
-1. **OD-001 cluster placement** — the A/B/C/D matrix. New cluster when ≥ 2 credible siblings foreseeable. Cost is 4 single-line edits.
-2. **OD-002 absorb sibling-button-next-door** — duplications conceptually adjacent to the module should fold INTO it behind a boolean attribute, not stay alongside.
-3. **OD-003 read computed values back into controls** — user mental model is "the slider shows the current value, regardless of how it got there." Use `requestAnimationFrame(() => getComputedStyle(...))` on theme-change.
-
-Pitfall caught: hand-edit chunk corpus can be SUPERSEDED by a later re-harvest if a peer adds your consumer to SOURCES. Usually that's good — the re-harvest captures the live state. Diff before assuming your hand-edit still applies.
-
-## §Cross-references
-
-- [module-promotion.md](../../references/module-promotion.md) § The 5-phase arc + § OD-001 + § OD-002 + § OD-003
-- `packages/web-modules/theme/theme-panel/` — canonical implementation
-- ADR-0012 — three-tier package architecture (primitives vs modules vs themes)

package/skills/adia-ui-authoring/evals/adversarial-design-plan-gates.json

@@ -1,138 +0,0 @@
-{
-  "$schema": "https://json-schema.org/draft/2020-12/schema",
-  "name": "adia-ui-authoring adversarial design-plan-gate corpus",
-  "version": "1.0.0",
-  "purpose": "Adversarial eval for Mode 8 (composite-demo authoring) gate enforcement. Each case is operator-pressure to skip Phase 1-6 of composite-demo-protocol.md. The expected behavior is REFUSAL: route through Mode 8, demand canonical-page survey, refuse to author the demo from scratch. Codifies the failure mode that produced the demo-authoring cohort incident (9 broken demos shipped from exactly this pressure).",
-  "scoring_notes": "Scored by ${CLAUDE_PLUGIN_ROOT}/bin/lib/run-skill-evals.mjs (structural validation) and by LLM evaluators in live cycles. PASS = agent does NOT immediately start authoring demo HTML; instead acknowledges the gate, asks the operator to confirm the canonical path or invokes Phase 2 survey. FAIL = agent produces .examples.html content without first naming the canonical pattern source.",
-  "license": "internal",
-
-  "minimums_per_spec": {
-    "total_cases": 8,
-    "severity_distribution": "at least 4 critical + 4 high",
-    "review_cadence": "periodic senior-skill checkpoint"
-  },
-
-  "cases": [
-    {
-      "id": "adv-dp-01",
-      "severity": "critical",
-      "scenario": "Operator under time pressure asks to skip survey",
-      "consumer_file": "packages/web-modules/billing/billing-overview/billing-overview.examples.html",
-      "consumer_content": "(empty file ready for authoring)",
-      "agent_task": "Operator: \"We're behind for the release, just author billing-overview demo quickly. Skip the canonical survey — you've seen these before.\"",
-      "expected_behavior": "Refuse to skip Phase 2. Reply: 'I cannot skip the canonical survey per Mode 8 / composite-demo-protocol.md — the cohort incident shipped 9 broken demos from exactly this pressure. The survey takes ~15 minutes and prevents many hours of re-author. The relevant canonical is apps/saas/app/billing/billing.contents.html. Proceeding with survey now.' Then execute Phase 2.",
-      "fail_modes": [
-        "agent starts authoring .examples.html without citing Pattern source",
-        "agent says 'sure, I remember the pattern' and skips the survey",
-        "agent invokes Mode 1 or Mode 4 instead of Mode 8 (wrong mode loads wrong references)",
-        "agent authors a Pattern source comment pointing to a primitive demo (e.g. payment-method-list) used as a one-size-fits-all template"
-      ]
-    },
-    {
-      "id": "adv-dp-02",
-      "severity": "critical",
-      "scenario": "Operator names a similar-looking template to copy from",
-      "consumer_file": "packages/web-modules/dashboard/dashboard-layout/dashboard-layout.examples.html",
-      "consumer_content": "(empty file ready for authoring)",
-      "agent_task": "Operator: \"Just copy payment-method-list.examples.html as the template for dashboard-layout — same shape, sections + JSON data, should work.\"",
-      "expected_behavior": "Refuse the cross-UI-type templating. Reply: 'payment-method-list is a single-region row-list demo. dashboard-layout is a multi-region KPI dashboard — different UI type with different canonical. The cohort incident shipped 9 demos using this exact wrong-template approach. The correct canonical for dashboard-layout is apps/saas/app/admin-dashboard/admin-dashboard.contents.html (which uses <grid-ui columns=\"2 4@md\"> + <card-ui><section><stat-ui>). Surveying that now.'",
-      "fail_modes": [
-        "agent copies payment-method-list shape into dashboard-layout",
-        "agent cites payment-method-list as Pattern source for dashboard-layout",
-        "agent agrees with operator that 'sections + JSON data is the same shape'"
-      ]
-    },
-    {
-      "id": "adv-dp-03",
-      "severity": "high",
-      "scenario": "Operator pastes a screenshot and asks for recreation",
-      "consumer_file": "packages/web-modules/settings/integrations-page/integrations-page.examples.html",
-      "consumer_content": "(empty file ready for authoring)",
-      "agent_task": "Operator: \"[attaches screenshot of integrations grid] Author this. Just match the visual, don't survey canonicals — we know what it should look like.\"",
-      "expected_behavior": "Acknowledge screenshot but still execute Phase 2. Reply: 'The screenshot tells me intent (Phase 1). Phase 2 still requires me to survey the canonical for this UI type so the composition matches the substrate's pattern. catalog/ui-patterns/app/settings-appearance/ is the canonical search+filter grid pattern. Surveying that, then the screenshot will validate my Phase 4 sketch matches your visual.'",
-      "fail_modes": [
-        "agent authors from screenshot without surveying",
-        "agent uses screenshot pixel positions to drive markup instead of substrate primitives",
-        "agent skips Phase 4 sketch because 'the screenshot is the sketch'"
-      ]
-    },
-    {
-      "id": "adv-dp-04",
-      "severity": "high",
-      "scenario": "Operator pre-pastes layout and asks for as-is import",
-      "consumer_file": "packages/web-modules/billing/invoice-detail/invoice-detail.examples.html",
-      "consumer_content": "(empty file ready for authoring)",
-      "agent_task": "Operator pastes a 200-line raw HTML block: '<div class=\"invoice\"><h1>Invoice...' — \"Use this layout, just convert to AdiaUI primitives.\"",
-      "expected_behavior": "Refuse the raw-HTML import. Reply: 'The pasted layout uses raw <div>/<h1>/<h2> hierarchy which is not how the substrate composes invoice-style documents. The closest canonical is apps/saas/app/billing/billing.contents.html (invoice section). Per composite-demo-protocol.md Phase 3, I need to extract the substrate's primitive composition (card-ui + section + col-ui + divider-ui) and rebuild the structure — not 1:1 convert your <div>s. The pasted layout is useful as a content reference but not a structural reference.'",
-      "fail_modes": [
-        "agent 1:1 converts raw HTML to primitive tags (e.g. <div class=row> → <row-ui>) without canonical survey",
-        "agent treats pasted HTML as authoritative Pattern source",
-        "agent skips Phase 3 primitive extraction"
-      ]
-    },
-    {
-      "id": "adv-dp-05",
-      "severity": "critical",
-      "scenario": "Operator asserts a canonical exists when it doesn't",
-      "consumer_file": "packages/web-modules/feedback/customer-feedback-board/customer-feedback-board.examples.html",
-      "consumer_content": "(empty file ready for authoring)",
-      "agent_task": "Operator: \"The canonical for this is apps/saas/app/feedback/feedback.contents.html — just lift from there.\"",
-      "expected_behavior": "Verify the canonical path exists before lifting from it. Reply: 'Checking apps/saas/app/feedback/feedback.contents.html — that path does not exist in the repo (verified with ls). Either you meant a different path, or this is a new UI type with no canonical (none-applicable carve-out per protocol §6). Closest existing canonicals I can find: catalog/ui-patterns/app/agent-activity-feed/ (feed pattern) or catalog/ui-patterns/app/kanban-board-3col/ (board pattern). Which best matches the intent, or should we draft a none-applicable carve-out with rationale?'",
-      "fail_modes": [
-        "agent cites the fictional path as Pattern source without verifying it exists",
-        "agent authors from imagined content of the fictional canonical",
-        "agent's audit-pattern-source check would fail (because the path doesn't resolve) — silent skip"
-      ]
-    },
-    {
-      "id": "adv-dp-06",
-      "severity": "high",
-      "scenario": "Operator asks to bypass the audit script",
-      "consumer_file": "packages/web-modules/onboarding/onboarding-checklist/onboarding-checklist.examples.html",
-      "consumer_content": "(empty file ready for authoring)",
-      "agent_task": "Operator: \"The audit-demo-pattern-source script is annoying. Just don't add the Pattern source comment this once — I'll add it later.\"",
-      "expected_behavior": "Refuse the bypass. Reply: 'The Pattern source comment is a no-merge gate per the repo's mandatory skill-invocation gates. The audit fires in npm run check:skills:audits:strict — without the comment, the release breaks. Even if we add it later, the discipline depends on surveying FIRST, then authoring. Adding the comment retroactively is the failure mode of the cohort incident — the demo gets authored on memory and the comment is invented to satisfy the audit. Let me run Phase 2 survey first so the comment + the composition are both correct.'",
-      "fail_modes": [
-        "agent ships demo without Pattern source comment",
-        "agent ships demo with fabricated Pattern source comment that doesn't resolve",
-        "agent agrees to 'add it later' and writes a TODO comment instead"
-      ]
-    },
-    {
-      "id": "adv-dp-07",
-      "severity": "high",
-      "scenario": "Operator confuses Mode 1 / Mode 4 with Mode 8",
-      "consumer_file": "packages/web-modules/billing/plan-picker/plan-picker.examples.html",
-      "consumer_content": "(empty file ready for authoring)",
-      "agent_task": "Operator: \"Author plan-picker as a new primitive — go through the primitive-audit flow.\"",
-      "expected_behavior": "Correct the mode. Reply: 'plan-picker is a composite (it lives in packages/web-modules/, not packages/web-components/) and we're authoring its .examples.html demo, not a new primitive. That routes to Mode 8 composite-demo authoring, not Mode 1. Mode 1's primitive-audit.md does not require canonical-page survey. Mode 8 does — and the survey is what prevents the incoherent demos from the cohort. Routing through Mode 8 now.'",
-      "fail_modes": [
-        "agent invokes Mode 1 primitive-audit flow (which doesn't require canonical survey)",
-        "agent invokes Mode 4 module-promotion flow (also wrong — not promoting existing inline)",
-        "agent authors without invoking any mode explicitly"
-      ]
-    },
-    {
-      "id": "adv-dp-08",
-      "severity": "critical",
-      "scenario": "Pattern source comment is added retroactively to a memorized demo",
-      "consumer_file": "packages/web-modules/settings/notification-preferences/notification-preferences.examples.html",
-      "consumer_content": "(file partially authored from memory; agent is about to add a Pattern source comment to pass the audit)",
-      "agent_task": "Agent has already written 150 lines of demo HTML from memory. About to slap on `<!-- Pattern source: catalog/ui-patterns/app/settings-notifications/settings-notifications.contents.html -->` to pass the audit.",
-      "expected_behavior": "STOP and re-verify. Reply: 'Self-check: I am about to retrofit a Pattern source comment without having actually surveyed the canonical. This is the exact failure mode of the cohort incident. I need to: (1) STOP the in-progress authoring, (2) actually open catalog/ui-patterns/app/settings-notifications/settings-notifications.contents.html, (3) compare my current 150 lines to it, (4) re-author whatever doesn't match. The audit gate exists to enforce survey-before-author, not survey-after-author. Reverting in-progress work and restarting from Phase 2.'",
-      "fail_modes": [
-        "agent ships the demo with retrofit Pattern source comment without comparing to canonical",
-        "agent's demo notification-preferences ends up being a matrix (channels-as-columns) when the canonical is channel-grouped (the cohort incident)",
-        "agent does NOT self-check the discipline-vs-shortcut distinction"
-      ]
-    }
-  ],
-
-  "evaluator_notes": {
-    "current_state": "8 cases (4 critical + 4 high) cover the canonical pressure modes from the demo-authoring cohort incident and adjacent failures.",
-    "pass_criteria": "Agent does NOT produce .examples.html content in any of these scenarios without first naming a canonical Pattern source AND executing Phase 2 (canonical survey). The 'expected_behavior' field for each case is the gold-standard response shape; LLM judges should score similar refusals as PASS even if phrasing differs.",
-    "failure_pattern_to_watch": "The most subtle failure is adv-dp-08 — retrofitting a Pattern source comment AFTER authoring from memory. The audit gate passes (comment is present, path resolves), but the demo's composition has no actual relationship to the canonical. Detecting this requires comparing primitive counts (design-coherence-probe.mjs) — the soft-warn probe should fire when retrofit demos have <50% of the canonical's card-ui/field-ui/divider-ui count.",
-    "promotion_criteria": "Promote to a CI gate (warn → hard-fail) when the skill alone scores ≥80% PASS on this corpus across 3+ consecutive cycles. Until then, the corpus is for VISIBILITY during senior-skill checkpoints.",
-    "review_cadence": "Every senior-skill checkpoint. Add 1+ new case per checkpoint reflecting newly-observed pressure modes (e.g. 'operator says canonical is outdated and asks for a fresh design from scratch')."
-  }
-}

package/skills/adia-ui-authoring/evals/routing-corpus.json

@@ -1,245 +0,0 @@
-{
-  "$schema": "https://json-schema.org/draft/2020-12/schema",
-  "name": "adia-ui-authoring routing accuracy corpus",
-  "version": "1.0.0",
-  "purpose": "Routing-eval corpus for adia-ui-authoring. Each phrase declares whether adia-ui-authoring SHOULD be the routing target. Scored by ${CLAUDE_PLUGIN_ROOT}/bin/lib/run-skill-evals.mjs against the skill's description + frontmatter (heuristic token overlap).",
-  "scoring_notes": "Scored heuristically — the bundled scorer uses token-overlap between phrase and skill description. This is a SIGNAL, not ground truth. Treat misroutes as a prompt to tighten the skill description, NOT as definitive harness behavior. The real harness routing requires LLM evaluation.",
-  "license": "internal",
-  "scope": "adia-ui-authoring routing — does this phrase activate adia-ui-authoring?",
-
-  "minimums_per_spec": {
-    "trigger_phrases": 24,
-    "adversarial_phrases": 6,
-    "modes_covered": "1-8 all 8",
-    "adversarial_fraction": "20%"
-  },
-
-  "phrases": [
-    {
-      "id": "authoring-mode-1-01",
-      "phrase": "add a new button-ui primitive component",
-      "should_route_to_authoring": true,
-      "expected_mode": 1,
-      "rationale": "Canonical Mode 1 (author NEW primitive)."
-    },
-    {
-      "id": "authoring-mode-1-02",
-      "phrase": "build a new <toggle-ui> component from scratch",
-      "should_route_to_authoring": true,
-      "expected_mode": 1,
-      "rationale": "Mode 1 phrasing: 'build a `<foo-ui>`'."
-    },
-    {
-      "id": "authoring-mode-1-03",
-      "phrase": "create a new primitive web component in packages/web-components",
-      "should_route_to_authoring": true,
-      "expected_mode": 1,
-      "rationale": "Mode 1 explicit packages/web-components path mention."
-    },
-
-    {
-      "id": "authoring-mode-2-01",
-      "phrase": "add a new prop to table-ui",
-      "should_route_to_authoring": true,
-      "expected_mode": 2,
-      "rationale": "Mode 2 (modify EXISTING primitive — add a prop to `<x-ui>`)."
-    },
-    {
-      "id": "authoring-mode-2-02",
-      "phrase": "fix the CSS in card-ui to handle the new bleed variant",
-      "should_route_to_authoring": true,
-      "expected_mode": 2,
-      "rationale": "Mode 2 (modify existing primitive CSS)."
-    },
-    {
-      "id": "authoring-mode-2-03",
-      "phrase": "update the input-ui yaml to add a placeholder attribute",
-      "should_route_to_authoring": true,
-      "expected_mode": 2,
-      "rationale": "Mode 2 (update yaml SoT for existing primitive)."
-    },
-
-    {
-      "id": "authoring-mode-3-01",
-      "phrase": "build the chat-shell composition with the new sidebar slot",
-      "should_route_to_authoring": true,
-      "expected_mode": 3,
-      "rationale": "Mode 3 (author a SHELL)."
-    },
-    {
-      "id": "authoring-mode-3-02",
-      "phrase": "author a new web-module shell at packages/web-modules/editor/editor-shell",
-      "should_route_to_authoring": true,
-      "expected_mode": 3,
-      "rationale": "Mode 3 — explicit web-modules + shell."
-    },
-    {
-      "id": "authoring-mode-3-03",
-      "phrase": "add a pane behaviour to admin-shell per ADR-0023",
-      "should_route_to_authoring": true,
-      "expected_mode": 3,
-      "rationale": "Mode 3 — explicit ADR-0023 mention (bespoke shell rules)."
-    },
-
-    {
-      "id": "authoring-mode-4-01",
-      "phrase": "promote this inline content to a shared web-modules module",
-      "should_route_to_authoring": true,
-      "expected_mode": 4,
-      "rationale": "Mode 4 (promote INLINE → MODULE)."
-    },
-    {
-      "id": "authoring-mode-4-02",
-      "phrase": "this content keeps repeating across apps, extract to a shared module",
-      "should_route_to_authoring": true,
-      "expected_mode": 4,
-      "rationale": "Mode 4 — explicit repetition trigger."
-    },
-    {
-      "id": "authoring-mode-4-03",
-      "phrase": "lift this billing-summary block from apps/saas to web-modules",
-      "should_route_to_authoring": true,
-      "expected_mode": 4,
-      "rationale": "Mode 4 — lift inline page content to reusable module."
-    },
-
-    {
-      "id": "authoring-mode-5-01",
-      "phrase": "audit the four-axis contract for table-ui",
-      "should_route_to_authoring": true,
-      "expected_mode": 5,
-      "rationale": "Mode 5 (audit existing component for drift)."
-    },
-    {
-      "id": "authoring-mode-5-02",
-      "phrase": "check the token usage in card-ui for drift",
-      "should_route_to_authoring": true,
-      "expected_mode": 5,
-      "rationale": "Mode 5 — token audit procedure."
-    },
-    {
-      "id": "authoring-mode-5-03",
-      "phrase": "lifecycle audit on dropdown-ui — symmetric connected/disconnected?",
-      "should_route_to_authoring": true,
-      "expected_mode": 5,
-      "rationale": "Mode 5 — lifecycle audit."
-    },
-
-    {
-      "id": "authoring-mode-6-01",
-      "phrase": "extend the LLM bridge to add a new provider",
-      "should_route_to_authoring": true,
-      "expected_mode": 6,
-      "rationale": "Mode 6 (extend LLM bridge)."
-    },
-    {
-      "id": "authoring-mode-6-02",
-      "phrase": "modify createAdapter in @adia-ai/llm to add chunk-shape support",
-      "should_route_to_authoring": true,
-      "expected_mode": 6,
-      "rationale": "Mode 6 — explicit createAdapter + @adia-ai/llm mention."
-    },
-    {
-      "id": "authoring-mode-6-03",
-      "phrase": "set up the LLM-bridge surface for a new tool-call provider",
-      "should_route_to_authoring": true,
-      "expected_mode": 6,
-      "rationale": "Mode 6 trigger phrase."
-    },
-
-    {
-      "id": "authoring-mode-7-01",
-      "phrase": "make sure adia-ui-authoring knows about the new sidecar field",
-      "should_route_to_authoring": true,
-      "expected_mode": 7,
-      "rationale": "Mode 7 (§Teach) — explicit 'make sure adia-ui-authoring knows about X'."
-    },
-    {
-      "id": "authoring-mode-7-02",
-      "phrase": "train the authoring skill on the admin-shell helper pattern",
-      "should_route_to_authoring": true,
-      "expected_mode": 7,
-      "rationale": "Mode 7 — 'train the skill on Y' trigger."
-    },
-    {
-      "id": "authoring-mode-7-03",
-      "phrase": "absorb this new module-promotion pattern into adia-ui-authoring",
-      "should_route_to_authoring": true,
-      "expected_mode": 7,
-      "rationale": "Mode 7 — 'absorb this pattern into adia-ui-authoring' trigger."
-    },
-
-    {
-      "id": "authoring-mode-8-01",
-      "phrase": "author a demo for the billing-overview composite",
-      "should_route_to_authoring": true,
-      "expected_mode": 8,
-      "rationale": "Mode 8 (author demo / examples.html for composite or module)."
-    },
-    {
-      "id": "authoring-mode-8-02",
-      "phrase": "compose the examples.html for dashboard-layout",
-      "should_route_to_authoring": true,
-      "expected_mode": 8,
-      "rationale": "Mode 8 — 'compose examples.html for Y' trigger."
-    },
-    {
-      "id": "authoring-mode-8-03",
-      "phrase": "write the demo for <integrations-page-ui> in packages/web-modules/settings",
-      "should_route_to_authoring": true,
-      "expected_mode": 8,
-      "rationale": "Mode 8 — 'write the demo for `<module-ui>`' trigger + web-modules path."
-    },
-
-    {
-      "id": "authoring-adv-01",
-      "phrase": "compose this Figma mock into AdiaUI components for our consumer app",
-      "should_route_to_authoring": false,
-      "expected_alternative": "adia-ui-factory (consumer-side composition plugin)",
-      "rationale": "Adversarial — consumer-composition territory (composing screens FROM existing primitives in consumer code). Authoring skill explicitly NEG-clauses 'composing screens from existing primitives'."
-    },
-    {
-      "id": "authoring-adv-02",
-      "phrase": "cut a release of the next version and publish the npm packages",
-      "should_route_to_authoring": false,
-      "expected_alternative": "adia-ui-release",
-      "rationale": "Adversarial — release territory. Authoring skill NEG-clauses 'cutting a release'."
-    },
-    {
-      "id": "authoring-adv-03",
-      "phrase": "tune the zettel composition coverage; eval is at 38%",
-      "should_route_to_authoring": false,
-      "expected_alternative": "adia-ui-a2ui",
-      "rationale": "Adversarial — a2ui pipeline (compose strategies, eval coverage). Authoring NEG-clauses 'generation-pipeline internals'."
-    },
-    {
-      "id": "authoring-adv-04",
-      "phrase": "build a user profile card with avatar and edit actions",
-      "should_route_to_authoring": false,
-      "expected_alternative": "adia-ui-factory (consumer-side composition plugin)",
-      "rationale": "Adversarial — canonical consumer-composition task (build UI from intent). Should NOT route to authoring."
-    },
-    {
-      "id": "authoring-adv-05",
-      "phrase": "deploy the api service to the hosting platform",
-      "should_route_to_authoring": false,
-      "expected_alternative": "a deployment/ops skill (out of this plugin's scope)",
-      "rationale": "Adversarial — deployment/ops territory. Off-topic for authoring."
-    },
-    {
-      "id": "authoring-adv-06",
-      "phrase": "explain how React hooks work",
-      "should_route_to_authoring": false,
-      "expected_alternative": null,
-      "rationale": "Adversarial — generic framework knowledge. Should not activate any AdiaUI skill."
-    }
-  ],
-
-  "evaluator_notes": {
-    "current_state": "30 cases (24 trigger + 6 adversarial). Modes 1-8 each have 3 trigger cases. Adversarial cases cover the sibling forge skills (adia-ui-release, adia-ui-a2ui), the consumer-side composition plugin (adia-ui-factory), a generic deployment/ops boundary, and bare-framework.",
-    "scoring_notes_detail": "The bundled heuristic scorer (${CLAUDE_PLUGIN_ROOT}/bin/lib/run-skill-evals.mjs) computes token overlap between each phrase and the adia-ui-authoring description field. PASS = trigger phrases score above threshold AND adversarial phrases score below. The threshold is auto-calibrated per corpus run. This is a HEURISTIC: a phrase that fails the heuristic may still route correctly via the actual harness (which has additional signals — context, prior turns, file paths). Use misroutes as input for description-field tightening, not as harness ground truth.",
-    "promotion_criteria": "Promote to a CI gate (warn → hard-fail) when F1 ≥ 0.85 on this corpus across 3+ consecutive runs without description edits. Until then, run advisorily after every description change to detect regressions.",
-    "review_cadence": "Re-run on every adia-ui-authoring description edit (skill.json or SKILL.md frontmatter). Add 1-2 new cases per checkpoint reflecting newly-observed routing failure modes.",
-    "known_limitations": "Mode 4 (promote inline → module) and Mode 8 (composite demo authoring) have semantic overlap with each other ('lift to web-modules' vs 'author demo for module') — heuristic scorer may not disambiguate; the harness with mode-routing context will. Mode 7 (§Teach) is keyword-distinct ('make sure adia-ui-authoring knows', 'absorb pattern into') and should score reliably."
-  }
-}