@adia-ai/a2ui-corpus 0.4.2 → 0.4.4

package/CHANGELOG.md

@@ -19,6 +19,69 @@ gap registry. Consumed by `@adia-ai/a2ui-retrieval` and
 
 _No pending changes._
 
+## [0.4.4] - 2026-05-12
+
+### Changed — corpus simplification arc (§36-§51, 2026-05-12)
+
+**verify:corpus warns: 993 → 148 (-845, -85%)** across §43-§51. Headline state: composition library boot reports `compositionCount=127, handAuthored=100, annotatedChunks=27`.
+
+#### Removed
+
+- **`corpus/exemplars/` retired (§36).** -3.4K lines. Exemplars had been speculative records without consumers; replaced by the growing annotated-chunk system.
+- **`corpus/fragments/` retired (§37).** 41 fragment JSON files + schema + categorical indices deleted (-99.6 KB). 85 hand-authored compositions inlined with previously-referenced fragment content (slot machinery removed); 10 fragment shapes migrated to chunks via `data-chunk` markers added in production HTML; remaining 29 fragments dropped as speculative/superseded. Net: 3 corpus formats → 2.
+
+#### Added
+
+- **`data-chunk-*` source-HTML annotation system (§40).** Chunks can now carry retrieval metadata authored directly on production HTML elements:
+  - `data-chunk-kind="<page|panel|block|field>"`
+  - `data-chunk-domain="<auth|dashboard|error|onboarding|...>"`
+  - `data-chunk-description="<concrete one-liner>"`
+  - `data-chunk-keywords="<space-separated tokens>"`
+  - `data-chunk-related="<comma-separated chunk names>"`
+  - `data-chunk-tags-<key>="<value>"` (per-tag attributes for the tags object)
+
+  Harvester (`scripts/build/harvest-chunks.mjs`, repo-level) reads these and produces chunks with a `metadata` field. Index schema bumped `@1` → `@2` to surface metadata for retrieval-without-load.
+
+- **Nested-expand-then-transpile (§42).** Annotated chunks with nested children get their `<!-- nested: <name> -->` placeholders recursively expanded then transpiled to flat A2UI templates at harvest time, making them retrievable as compositions via zettel. Cycle-safe via `visited` set.
+
+- **`verify:corpus` drift validator (§39, wired in §39; observability extended through §47/§51).** Walks every JSON in `corpus/{patterns,compositions,chunks}/**.json` against canonical component schemas. Surfaces unknown-component criticals + unknown-prop / enum-out-of-range / known-hallucination warns. §47 added `data-*` + `aria-*` HTML attribute skip rules; §42 added `annotated-chunk-no-template` warn for chunks with metadata but no template (caught the nested-expansion gaps).
+
+- **27 retrievable annotated chunks** seeded across 6 domains (§40 / §45 / §46 / §48):
+  - **Settings/data seeds (§40, 3 chunks):** `linked-record-row`, `settings-integrations`, `auth-signin-card-email`.
+  - **Auth (§45, 8 chunks):** `auth-signin-card-{password,otp,magic-link,mfa}`, `auth-signup-entry`, `auth-password-reset`, `auth-new-password`, `auth-mfa-enrollment`.
+  - **Dashboard (§46, 11 chunks):** `dashboard-admin-page` (427-component template, the full surface), `dashboard-page-header`, `dashboard-{overview,audience,acquisition,behavior,conversion,reports,notifications}-panel`, `dashboard-chart-recent`, `dashboard-team-actions-storage`.
+  - **Error + onboarding (§48, 5 chunks):** `error-404`, `error-500`, `error-maintenance`, `reg-step-shell`, `onb-step-shell`.
+
+#### Changed
+
+- **Shape B → Shape A codemod (§43, -486 warns).** 475 records / 79 files. Top transformations: `Text.color="muted"` → `variant="caption"` (244 records); `Text.weight="semibold"` → `strong: true` (146 records); `*.attrs: {...}` → flattened to top-level (146 records); `Text.content` → `textContent`, `Button.content` → `text`, `Badge.content` → `text` (84 records); 109 empty `attrs:{}` objects removed; 5 manual-review (Text.color=primary/success — needs design call).
+- **Yaml-gap renames + dead-prop deletes (§44, -301 warns).** 105 records / 48 files. Renames: `Button.block` → `stretch` (23), `Button.ariaLabel` → `aria-label` (16), `Badge.color="primary"` → `variant="primary"` (11, 1:1 mapping), `Table.rows` → `Table.data` (6), `Toast.title` → `Toast.text` (6). Deletes (runtime ignores): `Text.grow` (20, no `:scope[grow]` in text.css), `Progress.max` (15, runtime hardcodes max=100), `Skeleton.variant` (8, runtime silently drops), `Toast.description` (6, runtime only has `text`).
+- **§43 manual-review backlog retired (§50, -42 warns).** 74 records / 25 files. Subagent A's foundational finding: `attrs:{}` has zero runtime effect — no renderer/host spreads it onto the element. So every key inside `attrs:` is dead-letter; flatten or delete is always safe. Promoted 33 records from §43's MANUAL_REVIEW to HIGH-confidence codemod: `Toggle.attrs.checked` → top-level (6), `Text.attrs.muted=true` → `color: "muted"` (17), `Avatar.attrs.alt` → `text` (8, avatar.js:68 auto-mirrors name → `<img alt>`), `Avatar.attrs.fallback` → DELETE (5, redundant with auto-derived initials at avatar.js:96-110), `Divider.attrs.orientation="vertical"` → `vertical: true` (2), `Button.attrs.triggers` → DELETE (1, hallucinated wiring). Companion yaml-gap renames (§50): `Kbd.text`/`Kbd.keys` → `textContent` (10), `Pagination.totalPages`/`currentPage` → `total`/`page` (6), `Divider.orientation` → `vertical: true` (3), `Input.icon` → `prefix` (4), `Option.text` → `heading` (5).
+- **Search → Input migration (§42, 5 critical → 0).** 5 compositions migrated from deprecated `Search` component to `Input type="search" prefix="magnifying-glass"`. `Search` deliberately remains absent from the registry per registry.js line 57.
+- **Input.inputMode → Input.inputmode (§51, 3 records).** HTML standard is lowercase `inputmode`. Companion to the §51 renderer kebab-casing fix in `@adia-ai/a2ui-runtime`.
+
+### Harvester (`scripts/build/harvest-chunks.mjs`, repo-level)
+
+The harvester is repo-tooling (`scripts/build/`), not a corpus package member, but its changes power this arc's corpus output:
+
+- **§40 `data-chunk-*` attribute extraction** — reads sibling attributes from `data-chunk` elements; populates the chunk record's `metadata` field.
+- **§42 nested-expand-then-transpile pass** — after collection, recursively expands `<!-- nested: <name> -->` placeholders in annotated chunks, then transpiles the expanded HTML to A2UI templates via `transpileHTML(html, { mode: 'instant' })`.
+- **§45 source-aware tier-1 expansion fix** — replaces §42's "first wins" canonical-html rule with directory-aware selection. **Bug**: `auth-card-content` has 24 instances; the 404 page sorted alphabetically first, so EVERY auth chunk silently inlined 404 page content from §41 landing through §45 discovery (smoke tests only checked retrieval strategy, not template content). **Fix**: Tier 1 exact directory match → Tier 2 deepest common directory prefix → Tier 3 alphabetical. Without it, `apps/user-flow/app/auth/sign-in/` shares depth with `apps/user-flow/app/auth/sign-in/mfa/` and tie-break would pick wrong sibling.
+
+See root [CHANGELOG.md `[Unreleased]`](../../../CHANGELOG.md) for the cross-cutting arc narrative + [docs/journal/2026/05/2026-05-12.md](../../../docs/journal/2026/05/2026-05-12.md) §§ 36-51 for per-§ details.
+
+## [0.4.3] - 2026-05-11
+
+### Changed
+
+- **`catalog-a2ui_0_9.json` regenerated** to pick up the new `locale` prop on `<input-ui type="number">` from `@adia-ai/web-components@0.4.3`. The yaml `props:` entry drives retrieval signal for locale-aware numeric-input intents.
+- **`chunks/` re-harvested** (181 files touched). New chunk `settings-appearance.json` — multi-instance record covering both `catalog/ui-patterns/app/settings-appearance/` (primary=card-ui, standalone block pattern composing `<theme-panel>`) and `apps/saas/app/settings-page/` (primary=div, Workspace-level Appearance section). Plus a sweep of `captured_at` timestamps + `source:` field updates from the ADR-0026 reorg follow-up. Harvest now produces 180 unique chunks (was 179 in v0.4.2; +1 from the new `settings-appearance` pattern).
+- **Harvester scope fix** — `scripts/build/harvest-chunks.mjs` `SOURCES = ['site/pages', 'apps']` hadn't been updated after ADR-0026 split `apps/` → `apps/` + `playgrounds/` + `catalog/` on 2026-05-10. Every `npm run harvest:chunks` since silently dropped 26 chunks (179 unique → 152 visible) without warning. Fix adds both new sibling directories to SOURCES. (Note: harvest-chunks.mjs lives in the repo's `scripts/build/` tree, not the package; this corpus regen captures the FULL chunk set restored by the fix. Lesson captured as Category E imperative-drift in user-memory entry `feedback_build_script_sources_list_after_rename.md`.)
+
+### Lockstep
+
+9-package coordinated PATCH cut to v0.4.3. Internal `@adia-ai/*` dep ranges stay at `^0.4.0` (patch-cut asymmetry). Rides alongside `@adia-ai/web-components` v0.4.3 (input-ui locale rewrite) and `@adia-ai/a2ui-compose` + `@adia-ai/a2ui-retrieval` v0.4.3 (process.env browser-compat fix). See root [CHANGELOG.md `## [0.4.3]`](../../../CHANGELOG.md).
+
 ## [0.4.2] - 2026-05-11
 
 ### Changed

package/README.md

@@ -1,7 +1,7 @@
 # @adia-ai/a2ui-corpus
 
 Corpus and operational-learning artifacts for the gen-UI pipeline —
-patterns, fragments, compositions, exemplars, feedback, gaps, eval
+patterns, fragments, compositions, chunks, feedback, gaps, eval
 fixtures. Pure data plus the scripts that maintain it. No runtime.
 
 > The pipeline reads this package; this package never reads the pipeline.
@@ -11,40 +11,56 @@ fixtures. Pure data plus the scripts that maintain it. No runtime.
 > registry, streams, wiring),
 > [`@adia-ai/a2ui-mcp`](../mcp) for the MCP server.
 
+## Install
+
+```bash
+npm install @adia-ai/a2ui-corpus
+```
+
+Pure data — typically consumed transitively by [`@adia-ai/a2ui-compose`](../compose) and [`@adia-ai/a2ui-mcp`](../mcp), which list it as a runtime dependency. Direct installs are useful for offline eval tooling or building bespoke retrieval layers.
+
 ## Dependency direction
 
 ```
 a2ui-compose   ──reads──▶  a2ui-corpus
 a2ui-mcp       ──reads──▶  a2ui-compose, a2ui-corpus
-web-components ◀─reads──   a2ui-corpus (exemplars use AdiaUI tags)
+web-components ──used-by──▶  apps/, playgrounds/, catalog/ (chunk sources)
 ```
 
 No back-writes. No circular reads. Web-components ships UI atoms only;
-corpus lives here; runtime lives in a2ui-compose.
+corpus lives here; runtime lives in a2ui-compose. The chunk pipeline
+harvests `data-chunk`-tagged regions from the apps that consume
+web-components, so the read-direction stays one-way.
 
 ## Glossary
 
-Four overlapping concepts circulate through this package. Knowing which
+Three overlapping concepts circulate through this package. Knowing which
 is which prevents reading bugs from creeping into authoring.
 
 | Term | Source of truth | Granularity | What it carries | Engine that consumes it |
 | --- | --- | --- | --- | --- |
-| **chunk** | `chunks/<id>.json` (one file per chunk) + `_index.json` | Smallest — a single labeled HTML region, harvested via `data-chunk` markers | `html`, `intent`, `domain`, `kind` (`block` / `data-chunk` / `field`), `keywords` | `chunk-zettel` (retrieve → synthesize) |
+| **chunk** | `chunks/<id>.json` (one file per chunk) + `_index.json` | Smallest — a single labeled HTML region, harvested via `data-chunk` markers | `html`, `intent`, `domain`, `kind` (`block` / `data-chunk` / `field`), `keywords` | `chunk-zettel` + `monolithic-pro` (retrieve → synthesize or inject as structural reference) |
 | **pattern** | `patterns/<domain>/<name>.json` | Full-canvas A2UI template (≈whole-page) | A2UI message tree, retrieval keywords, exemplar provenance | `monolithic-instant` / `monolithic-pro` (retrieve → emit verbatim) |
-| **fragment** | `fragments/<domain>/<name>.json` | Atomic shape — re-usable across compositions | A2UI sub-tree, slot bindings, `metrics.leverage` | `zettel` (retrieve → compose into compositions) |
-| **composition** | `compositions/<domain>/<name>.json` | Pre-assembled set of fragments | Ordered fragment refs, slot wiring, keywords + tags, `links.used_by` | `zettel` (retrieve → emit verbatim or refine) |
+| **composition** | `compositions/<domain>/<name>.json` | Multi-section A2UI surface (between pattern and chunk) | Flat A2UI template, keywords + tags | `zettel` (retrieve → emit verbatim) |
 
 **One-line distinctions:**
 
-- **Chunk vs fragment** — chunks are *harvested*, fragments are *authored*. A chunk is whatever HTML region a `data-chunk` marker pointed at; a fragment is a deliberately-shaped A2UI sub-tree with named slots and leverage metrics.
-- **Fragment vs composition** — a fragment is *one shape*; a composition is *several fragments arranged together* into a coherent surface (e.g. `pricing-tiers` composition = header fragment + 3× tier-card fragments + footer fragment).
-- **Composition vs pattern** — both are full-surface, but compositions are *fragment-graph* native (zettel reuses the fragment library) while patterns are *flat A2UI templates* (monolithic emits them whole).
+- **Chunk vs composition** — chunks are *harvested* from `data-chunk` markers in real production HTML. Compositions are *authored* A2UI templates that don't necessarily correspond to a real page surface.
+- **Composition vs pattern** — both are full-surface A2UI trees. Patterns are flat (single-page idea), compositions are multi-section (composed surface). The distinction is fuzzy and may converge over time.
 
 **When in doubt, follow the engine:**
 
 - Looking at `monolithic-*` retrieval? You want a **pattern**.
-- Looking at `zettel` retrieval? You want a **composition** (or a fragment, if synthesis is composing one).
-- Looking at `chunk-zettel`? You want a **chunk**.
+- Looking at `zettel` retrieval? You want a **composition**.
+- Looking at `chunk-zettel` or chunk-aware monolithic-pro injection? You want a **chunk**.
+
+> **Note:** *Fragments* (atomic A2UI sub-trees with named slots) were
+> a fourth concept until §37 (2026-05-12). They were retired because
+> the slot-binding machinery added authoring + maintenance cost
+> without enough payoff (10/41 fragments had production examples,
+> 31/41 were stale). Their compositions were inlined into flat A2UI
+> templates per Option C (LLM handles parameterization at gen-time).
+> See [docs/journal/2026/05/2026-05-12.md §37](../../../docs/journal/2026/05/2026-05-12.md).
 
 ## Layout
 
@@ -55,25 +71,16 @@ a2ui/corpus/
 │   ├── _components.json       canonical name + aliases map (Carousel → Swiper)
 │   └── _taxonomy.json         concept synonyms (auth → [login, signup, otp])
 │
-├── fragments/               atomic shapes — building blocks for Zettel
-│   ├── action/                  button rows, toolbars, inline CTAs
-│   ├── content/                 stat cards, list items, media blocks
-│   ├── form/                    field, field-row, field-group recipes
-│   └── layout/                  section, grid, card scaffolding
-│
-├── compositions/            named multi-fragment recipes
+├── compositions/            named multi-section A2UI templates (flat,
 │   ├── agent/ data/ form/ layout/ navigation/ settings/
-│                              each = pre-assembled fragment set with keywords + tags
-│
-├── exemplars/               hand-authored HTML sources
-│   ├── ui/                      component-level exemplars
-│   └── prose/                   prose + mixed-content exemplars
-│                              extract.js compiles these → chunks/ + catalog
+│                              ~100 surfaces; each = flat A2UI template
+│                              with keywords + tags. Inlined per §37 —
+│                              no $fragment refs remain.
 │
 ├── evals/                   held-out.jsonl + eval fixtures
-├── chunks/                  gen-UI training chunks (702 unique, 852 instances)
+├── chunks/                  gen-UI training chunks (329 unique, ~852 instances)
 │                              — harvested from data-chunk markers across
-│                              site/pages/* + corpus exemplars (since 0.0.3).
+│                              site/pages/*, apps/*, playgrounds/*, catalog/*.
 │                              Powers compose_from_chunks in @adia-ai/a2ui-mcp.
 ├── pages/                   extracted full-page A2UI   (build output)
 ├── feedback/                daily JSONL — user feedback events
@@ -98,8 +105,7 @@ a2ui/corpus/
 |----------------------------|:---------:|:----------------:|----------------------------------------|
 | Patterns (`patterns/`)     | ✓         | ✓                | Hand-authored JSON                     |
 | Fragments/compositions     | ✓         | ✓                | Hand-authored JSON                     |
-| Exemplars (`exemplars/`)   | ✓         | ✓                | Hand-authored HTML                     |
-| Extracted chunks/pages     | ✓         | ✓                | `scripts/extract.js` (from exemplars)  |
+| Chunks (`chunks/`)         | ✓         | ✓                | `scripts/build/harvest-chunks.mjs` (from `data-chunk` markers across site/pages, apps, playgrounds, catalog) |
 | `catalog-a2ui_0_9.json`    | ✓         | ✓                | `scripts/build-catalog.js` (aggregated)|
 | `chunk-embeddings.json`    | ✓         | **✗ since 0.2.1**| `scripts/build/embeddings-chunks.mjs`  |
 | `pattern-embeddings.json`  | ✓         | **✗ since 0.2.1**| `scripts/build/embeddings.mjs`         |
@@ -193,9 +199,9 @@ Script inventory (`scripts/`):
 
 | Script                    | Purpose                                              |
 |---------------------------|------------------------------------------------------|
-| `extract.js`              | Exemplar HTML → A2UI chunks & pages                  |
-| `ingest.js`               | Index extracted chunks into the retrieval corpus     |
-| `run-pipeline.mjs`        | Orchestrates harvest → extract → ingest → index      |
+| `extract.js`              | **Retired** — fed the exemplar→chunks pipeline (removed 2026-05-11). Kept on disk for archaeology only. |
+| `ingest.js`               | **Retired** — same vintage. Still imported by `run-pipeline.mjs` for back-compat; no-op without exemplars. |
+| `run-pipeline.mjs`        | Pipeline orchestrator — now mostly a no-op on the extract/ingest side; chunks come from `harvest-chunks.mjs` instead |
 | `build-pattern-index.mjs` | Computes the keyword inverted index (`patterns/_index.json`) |
 | `chunk-library.js`        | In-memory loader + keyword/semantic search over `chunks/` |
 | `feedback-report.js`      | Aggregates feedback JSONL into a readable digest     |
@@ -206,9 +212,9 @@ Repo-side build scripts (not in tarball; run from the workspace root):
 
 | Script                              | Purpose                                                  |
 |-------------------------------------|----------------------------------------------------------|
-| `npm run harvest:chunks`            | Walks site/pages/* + corpus exemplars, harvests every `[data-chunk]` element, writes `chunks/<name>.json` + `_index.json` |
+| `npm run harvest:chunks`            | Walks site/pages/*, apps/*, playgrounds/*, catalog/*, harvests every `[data-chunk]` element, writes `chunks/<name>.json` + `_index.json` |
 | `npm run build:embeddings`          | Generates `pattern-embeddings.json` (113 patterns × 1536d) |
-| `npm run build:embeddings:chunks`   | Generates `chunk-embeddings.json` (702 chunks × 1536d)   |
+| `npm run build:embeddings:chunks`   | Generates `chunk-embeddings.json` (329 chunks × 1536d)   |
 | `npm run build:embeddings:all`      | Both embedding indices in sequence                       |
 
 ## Exports
@@ -223,32 +229,34 @@ import aliases from '@adia-ai/a2ui-corpus/patterns';
 // Held-out eval fixture
 import fixture from '@adia-ai/a2ui-corpus/evals/held-out.jsonl?raw';
 
-// Exemplar source (for scripts + docs)
-import html from '@adia-ai/a2ui-corpus/exemplars/ui/pricing-tiers.html?raw';
-
 // Chunk corpus (since 0.0.3 / 0.0.4)
 import chunkIndex from '@adia-ai/a2ui-corpus/chunks';   // _index.json (metadata only)
 import { searchChunks, searchChunksAsync, getChunk }
   from '@adia-ai/a2ui-corpus/chunk-library';            // in-memory query API
 ```
 
-## Authoring order — demo page → exemplar → training
+## Authoring order — demo page → `data-chunk` marker → training
 
 When adding coverage for a new intent:
 
-1. **Site demo page** — author the HTML in `site/pages/examples/**`
-   using pure primitive composition (no `<style>` block, no inline
-   styles, only real components). See repo-root `AGENTS.md`.
-2. **Exemplar** — copy the working markup into
-   `a2ui/corpus/exemplars/ui/<name>.html`. Strip non-essential chrome;
-   keep attributes that encode intent.
-3. **Extract + ingest** — `npm run pipeline`. The scripts write
-   `chunks/`, `pages/`, and refresh the catalog.
-4. **Verify** — `npm run eval:diff -- --engine zettel` should still hold
-   coverage 100%, avgScore ≥ 88. Regression thresholds in `AGENTS.md`.
-
-See `data-flow.md` for the full 4-signal pipeline (exemplars →
-fragments → compositions → feedback).
+1. **Live demo page** — author the HTML in
+   `apps/<name>/app/<demo>/<demo>.contents.html` (or under
+   `playgrounds/` / `catalog/`) using pure primitive composition. See
+   repo-root `AGENTS.md`.
+2. **Tag the reusable region** — add `data-chunk="<slug>"` +
+   `data-chunk-kind="<kind>"` (block / page / panel / field) on the
+   element. The harvester extracts the bounding HTML on the next
+   build. See `docs/specs/genui-chunk-marker.md` for the marker
+   convention.
+3. **Harvest + ingest** — `npm run harvest:chunks` writes
+   `chunks/<slug>.json` and refreshes `_index.json`. `npm run pipeline`
+   does the full extract → ingest → catalog refresh.
+4. **Verify** — `npm run eval:diff -- --engine zettel` should still
+   hold coverage ≥ 83%, avgScore ≥ 88 (per the regression floors in
+   `AGENTS.md`).
+
+See `data-flow.md` for the full pipeline (chunks → fragments →
+compositions → feedback).
 
 ## Regression floors