@adia-ai/a2ui-mcp 0.0.2 → 0.0.3

package/CHANGELOG.md

@@ -11,6 +11,59 @@ zettel strategies.
 
 ---
 
+## [0.0.3] - 2026-04-28
+
+Same-day follow-up to `0.0.2` shipped earlier today. Wires the corpus
+embedding vectors (from `@adia-ai/a2ui-corpus@0.0.4`) into the chunk
+retrieval path so the synthesizer mixes-and-matches against semantic
+similarity, not just keyword overlap.
+
+### Added
+
+- **`chunk-embedding-retriever.js`** (`packages/a2ui/retrieval/`) — sibling
+  to the existing pattern-embedding retriever. Lazy-loads
+  `chunk-embeddings.json`, scores cosine similarity against the same
+  provider that built the index. Graceful no-op when index is missing or
+  no API key is available.
+- **`searchChunksAsync(query, opts)`** in `chunk-library.js` — embedding-
+  blended search. Returns top candidates ranked by `keyword + 5×cosine`
+  when embeddings are available; falls back to keyword-only otherwise.
+  Sync `searchChunks()` is preserved for callers that want the keyword
+  floor without the network round-trip.
+
+### Changed
+
+- `chunk-synthesizer.composeFromIntent()` — both tier-1 retrieval AND
+  tier-2 pre-search now go through `searchChunksAsync` instead of the
+  keyword-only `searchChunks`. The pre-search filters the prompt token
+  budget more accurately; the retrieval-first path catches more direct
+  hits before the LLM is invoked.
+
+### Measured impact
+
+`eval:chunk-synthesis` (10 hold-out intents, real Anthropic LLM):
+
+| | 0.0.2 (keyword) | **0.0.3 (semantic)** |
+|---|---|---|
+| Retrieval-tier hits | 7/10 | **9/10** |
+| Synthesis-tier attempts | 3 | **1** |
+| Synthesis plans validated | 3/3 | 1/1 |
+| Total time | 5.0s | 5.1s |
+
+Examples that flipped from synthesis → direct retrieval:
+- "kpi grid with 4 stat cards" → matches `dashboard-kpi-grid` cleanly.
+- "conversion funnel chart" → matches `dashboard-funnel`.
+
+The remaining synthesis intent ("recovery page with backup-code form +
+contact-support link") is genuinely novel — it composes from parts that
+don't have a 1:1 chunk and exercises the slot-validator path correctly.
+
+### Dependencies
+
+- Bumps `@adia-ai/a2ui-corpus` requirement from `^0.0.3` to `^0.0.4`.
+
+---
+
 ## [0.0.2] - 2026-04-28
 
 Adds **gen-UI training-chunk tools** that expose the new chunk corpus

package/README.md

@@ -42,31 +42,35 @@ export GEMINI_API_KEY=AIza…
 
 ## Tools
 
-The server registers 21 tools. Shape is stable; argument schemas via Zod.
-
-| Tool                   | What it does                                              |
-|------------------------|-----------------------------------------------------------|
-| `generate_ui`          | Intent → A2UI tree. Engine (`monolithic`/`zettel`) + mode. |
-| `validate_schema`      | Run the 15-check validator on an A2UI tree; returns 0-100. |
-| `classify_intent`      | Extract concepts, entities, implied components, steelman.  |
-| `lookup_component`     | Resolve a component name (alias-aware) to its schema.      |
-| `get_component_map`    | Full tag→class map including alias normalizations.         |
-| `search_patterns`      | Keyword-rank the monolithic pattern corpus.                |
-| `assemble_context`     | Build the system prompt context for a given intent.        |
-| `check_anti_patterns`  | Scan a tree for canonical anti-patterns (chart-legend, …). |
-| `get_traits`           | List trait catalog + their host-binding rules.             |
-| `convert_html`         | Raw HTML → best-effort A2UI tree (import path).            |
-| `get_wiring_catalog`   | Declarative wiring-engine recipes.                         |
-| `import_pattern`       | Commit a generated result into the pattern library.        |
-| `submit_feedback`      | Append a user-feedback event to the feedback store.        |
-| `get_quality_metrics`  | Aggregate pass/fail scores over a window.                  |
-| `get_training_gaps`    | Intents that currently miss coverage.                      |
-| `run_eval`             | Run the held-out benchmark; return pass/fail per intent.   |
-| `get_fragment`         | Fetch a single zettel fragment by id.                      |
-| `get_composition`      | Fetch a named multi-fragment composition.                  |
-| `resolve_composition`  | Expand a composition reference into its fragments.         |
-| `get_graph`            | Dump the zettel fragment-dependency graph.                 |
-| `zettel_stats`         | Corpus counts (fragments, compositions, reuse ratio, …).   |
+The server registers 25 tools. Shape is stable; argument schemas via Zod.
+
+| Tool                    | What it does                                              |
+|-------------------------|-----------------------------------------------------------|
+| `generate_ui`           | Intent → A2UI tree. Engine (`monolithic`/`zettel`) + mode. |
+| `validate_schema`       | Run the 15-check validator on an A2UI tree; returns 0-100. |
+| `classify_intent`       | Extract concepts, entities, implied components, steelman.  |
+| `lookup_component`      | Resolve a component name (alias-aware) to its schema.      |
+| `get_component_map`     | Full tag→class map including alias normalizations.         |
+| `search_patterns`       | Keyword-rank the monolithic pattern corpus.                |
+| `assemble_context`      | Build the system prompt context for a given intent.        |
+| `check_anti_patterns`   | Scan a tree for canonical anti-patterns (chart-legend, …). |
+| `get_traits`            | List trait catalog + their host-binding rules.             |
+| `convert_html`          | Raw HTML → best-effort A2UI tree (import path).            |
+| `get_wiring_catalog`    | Declarative wiring-engine recipes.                         |
+| `import_pattern`        | Commit a generated result into the pattern library.        |
+| `submit_feedback`       | Append a user-feedback event to the feedback store.        |
+| `get_quality_metrics`   | Aggregate pass/fail scores over a window.                  |
+| `get_training_gaps`     | Intents that currently miss coverage.                      |
+| `run_eval`              | Run the held-out benchmark; return pass/fail per intent.   |
+| `get_fragment`          | Fetch a single zettel fragment by id.                      |
+| `get_composition`       | Fetch a named multi-fragment composition.                  |
+| `resolve_composition`   | Expand a composition reference into its fragments.         |
+| `get_graph`             | Dump the zettel fragment-dependency graph.                 |
+| `zettel_stats`          | Corpus counts (fragments, compositions, reuse ratio, …).   |
+| **`search_chunks`**     | Semantic + keyword search over the gen-UI training-chunk corpus (since 0.0.2). |
+| **`get_chunk`**         | Full record (HTML + metadata + slots) for one chunk.       |
+| **`lookup_chunk`**      | List every chunk whose primary element is `<component>`.   |
+| **`compose_from_chunks`** | Retrieval-first / LLM-mix-and-match composition. Picks a page chunk + binds block/panel chunks to its slots when retrieval is weak. Validator enforces slot+kind contracts. **Embedding-blended retrieval as of 0.0.3** (was keyword-only in 0.0.2). |
 
 ## Layout
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@adia-ai/a2ui-mcp",
-  "version": "0.0.2",
+  "version": "0.0.3",
   "description": "AdiaUI A2UI MCP server. Exposes the compose engine over MCP with an engine selector for monolithic + zettel strategies.",
   "type": "module",
   "bin": {
@@ -29,7 +29,7 @@
     "@adia-ai/a2ui-compose": "^0.0.1",
     "@adia-ai/a2ui-retrieval": "^0.0.1",
     "@adia-ai/a2ui-validator": "^0.0.1",
-    "@adia-ai/a2ui-corpus": "^0.0.3",
+    "@adia-ai/a2ui-corpus": "^0.0.4",
     "zod": "^3.24.0"
   }
 }

package/scripts/eval-chunk-synthesis.mjs

@@ -0,0 +1,100 @@
+#!/usr/bin/env node
+/**
+ * Real-LLM eval set for the chunk-aware composition synthesizer.
+ *
+ * Walks a hold-out set of intents that DON'T have a 1:1 chunk match in the
+ * corpus and exercises the full retrieval-first → synthesis-fallback path.
+ * For each intent, records:
+ *   - which tier handled it (retrieval direct hit vs LLM synthesis)
+ *   - whether HTML was produced
+ *   - whether the synthesizer's plan validated against slot/kind contracts
+ *   - composed HTML byte length (rough proxy for "not empty")
+ *
+ * Pass criterion: ≥ 80% of intents produce non-empty HTML; all synthesizer
+ * plans pass the slot+kind validator.
+ *
+ * Usage:
+ *   node packages/a2ui/mcp/scripts/eval-chunk-synthesis.mjs
+ *   ANTHROPIC_API_KEY=… node packages/a2ui/mcp/scripts/eval-chunk-synthesis.mjs
+ */
+
+import '../../../../scripts/load-env.mjs';
+import { composeFromIntent } from '../../compose/engines/zettel/chunk-synthesizer.js';
+import { createAdapter } from '../../compose/llm/llm-bridge.js';
+
+// Hold-out intents — chosen to NOT have a 1:1 chunk match (so synthesis path
+// is exercised). Mix of dashboard-shape, auth-shape, and novel composites.
+const INTENTS = [
+  // Dashboard composites that don't exist as a single chunk:
+  'admin dashboard with KPI grid and conversion funnel',
+  'analytics page with audience KPIs and a country list',
+  'reports page with a transactions table and a sparkline grid',
+
+  // Auth-card variations that aren't pre-baked:
+  'sign-in page with email + password + remember-me + magic-link alternative',
+  'recovery page with backup-code form and contact-support link',
+
+  // Generic page-shaped composites:
+  'page with a sticky header and a tabbed body',
+  'data-rich settings page with a members table and an integrations grid',
+
+  // Things that should retrieve directly (control group):
+  'kpi grid with 4 stat cards', // → likely matches dashboard-kpi-grid via search
+  'conversion funnel chart',     // → dashboard-funnel
+  'sign in form with email',     // → auth-signin-card-email
+];
+
+const startedAt = Date.now();
+const results = [];
+
+console.log(`▶ chunk-synthesis eval — ${INTENTS.length} intents\n`);
+
+const llmAdapter = await createAdapter();
+
+for (const intent of INTENTS) {
+  const t0 = Date.now();
+  let row = { intent, ms: 0, source: null, hasHtml: false, htmlBytes: 0, planValid: null, error: null };
+  try {
+    const result = await composeFromIntent({ intent, llmAdapter, maxAttempts: 2 });
+    row.ms = Date.now() - t0;
+    row.source = result.source;
+    row.hasHtml = !!result.html;
+    row.htmlBytes = result.html ? result.html.length : 0;
+    row.planValid = result.synthesis ? !!result.synthesis.validation?.ok : null;
+    row.warnings = (result.warnings || []).length;
+  } catch (e) {
+    row.ms = Date.now() - t0;
+    row.error = e.message;
+  }
+  results.push(row);
+  const flag = row.hasHtml ? '✓' : '✗';
+  const tag = row.source === 'retrieval' ? '[ret]' : row.source === 'synthesis' ? '[syn]' : '[err]';
+  console.log(`  ${flag} ${tag} ${row.ms.toString().padStart(5)}ms  ${intent}`);
+  if (row.error) console.log(`      error: ${row.error}`);
+  if (row.warnings) console.log(`      ${row.warnings} warning(s)`);
+}
+
+const passed = results.filter((r) => r.hasHtml).length;
+const passRate = passed / results.length;
+const synthAttempts = results.filter((r) => r.source === 'synthesis');
+const retrievalAttempts = results.filter((r) => r.source === 'retrieval');
+const synthValidPlans = synthAttempts.filter((r) => r.planValid).length;
+
+console.log(`\n── Summary ──`);
+console.log(`  Total intents: ${results.length}`);
+console.log(`  Produced HTML: ${passed} (${(passRate * 100).toFixed(0)}%)`);
+console.log(`  Retrieval-tier hits: ${retrievalAttempts.length}`);
+console.log(`  Synthesis-tier attempts: ${synthAttempts.length}`);
+if (synthAttempts.length) {
+  console.log(`  Synthesis plans validated: ${synthValidPlans}/${synthAttempts.length}`);
+}
+console.log(`  Total time: ${((Date.now() - startedAt) / 1000).toFixed(1)}s`);
+
+const passThreshold = 0.8;
+if (passRate >= passThreshold) {
+  console.log(`\n✓ PASS — ≥${passThreshold * 100}% intents produced HTML`);
+  process.exit(0);
+} else {
+  console.log(`\n✗ FAIL — pass rate ${(passRate * 100).toFixed(0)}% < ${passThreshold * 100}% threshold`);
+  process.exit(1);
+}