@adia-ai/a2ui-compose 0.5.0 → 0.5.2

package/CHANGELOG.md

@@ -12,6 +12,63 @@ generator graph.
 
 _No pending changes._
 
+## [0.5.2] - 2026-05-13
+
+### Changed — `plan` graduates from `_debug.*` to first-class on free-form-composed result (§107a infra, v0.5.2)
+
+
+`strategies/registry.js` `generateFreeFormAdapter` returns `plan` as a top-level result field (paired with v0.5.1's §109 graduation of `usedIngredients` + `rationale`). The `_debug.plan` field is no longer populated — consumers should read `result.plan` directly. Soft-API change (`_debug` was always documented as volatile).
+
+Enables substitution-coverage measurement at eval time without dialog-recorder coupling. Companion to `@adia-ai/a2ui-mcp` `--report-substitutions` flag.
+
+### Deprecated — individual `_debug.*` field reads on free-form-composed results (§131, v0.5.2)
+
+Documents the `_debug`-volatility contract that §109 (v0.5.1) + §107a (v0.5.2) established de-facto. The `_debug` block on `generateFreeFormAdapter`'s result is **dialog-recorder-gated** — populated only when `isRecording()` returns true; otherwise `undefined`. Individual field reads (`result._debug?.usedIngredients`, `result._debug?.rationale`, `result._debug?.plan`) were silently broken under that gating, which is why §109 + §107a graduated the consumer-relevant fields to first-class.
+
+**Migration:** any consumer reading `result._debug?.<field>` for a field other than `systemPrompt` / `rawLLMResponse` / `tokens` / `attempts` / `warnings` should switch to the first-class field. The currently-stable first-class set: `usedIngredients`, `rationale`, `plan`.
+
+**Scheduled removal:** v0.6.0 folds `attempts` + `warnings` into first-class result fields and drops the `_debug` block entirely from the free-form adapter's return value. The dialog-recorder will read first-class fields directly.
+
+No live in-repo consumers of the soft-API path remain (verified `grep -rn '_debug?\.\|_debug\.' apps/ playgrounds/ catalog/` → 0 hits). One dead fallback in `@adia-ai/a2ui-mcp`'s `eval-diff.mjs` (`result.plan || result._debug?.plan || null`) removed in the same cut.
+
+### Changed — Free-form system prompt: 5 per-shape substitution examples + self-check pattern (§126, v0.5.2)
+
+`strategies/free-form-composer/system-prompt.js` constraint 3 extends the v0.5.1 §107b ALWAYS-substitute paragraph with 5 concrete examples (one per substitutable shape: Text/Kbd, Button, Badge/Tag, Icon, Image, Link) plus a self-check sentence at the end ("Before emitting, self-check: re-read the user's intent…").
+
+Targets the v0.5.1 §107a finding that Haiku 4.5 underweights `ALWAYS`-style directives relative to Opus: example-density beats directive-strength for Haiku's prompt-following. Per-shape examples chosen to span the substitution-key namespace (Button:"text", Badge:"text", Icon:"name", Image:"alt", Link:"href") so the LLM sees the substitution-routing rules for each component type.
+
+Cost: ~120 additional prompt tokens per call (~0.5% of total). Acceptable.
+
+Expected impact: lifts substitution ratio 27.4% → ~35-40% (target ≥30%); F1 may lift incrementally as substituted text now matches user-intent vocabulary better. Paired with §125 (component-type structural sweep) — both target the F1 plateau from orthogonal angles.
+
+### Changed — `generateFreeFormAdapter` accepts `ctx.model` override + reads `FREE_FORM_MODEL_OVERRIDE` env var (§127 infra, v0.5.2)
+
+`strategies/registry.js` reads model priority chain at call-time (not module-load): `ctx.model` > `process.env.FREE_FORM_MODEL_OVERRIDE` > `FREE_FORM_MODEL_DEFAULT` (Haiku 4.5). Enables `eval-diff.mjs --model <id>` to run the Haiku-vs-Opus A/B for §127 without env-dance or static-import-ordering hazards. `FREE_FORM_MODEL` constant renamed to `FREE_FORM_MODEL_DEFAULT` to reflect the new priority chain.
+
+Default behavior unchanged when no override set — the v0.5.1 §108 Haiku pin holds for every consumer that doesn't explicitly override.
+
+## [0.5.1] - 2026-05-13
+
+### Added — Free-form composer INTENT-PARAPHRASE block + paraphrase-retry (§106, v0.5.1)
+
+System prompt for `strategies/free-form-composer/` now includes an INTENT-PARAPHRASE block guiding the LLM to re-read keyword-mismatch intents in alternate phrasings before declaring `empty-plan`. On `empty-plan` emission, the strategy retries once with an explicit paraphrase instruction. Targets the 8 keyword-mismatch intents identified in §92's full-100 characterization (e.g. "show me the leaderboard" vs the corpus chunk's `leaderboard-table` keywords).
+
+Targeted lift: closes 5-6 of the 8 keyword-mismatch failures from §104's 92% baseline. F1 on the remaining 2-3 is intent-shape-dependent; v0.5.2+ corpus regrowth handles those.
+
+### Changed — Haiku 4.5 pinned for free-form ingredient picker (§108, v0.5.1)
+
+`strategies/registry.js` `generateFreeFormAdapter` now mints a Haiku 4.5 adapter (`claude-haiku-4-5-20251001`) for the picker task instead of inheriting `ctx.llmAdapter`. Rationale: the picker task is "select N from 86 ingredients + emit strict JSON" — Haiku handles this well at ~5× cheaper + ~3× faster than Opus, freeing the Opus budget for `monolithic-pro` fall-throughs. User-decided pre-A/B per the v0.5.1 plan question 2.
+
+Consumers passing `ctx.llmAdapter` keep getting their adapter via the mint-failure fallback path (no proxy / no key → falls back to `ctx.llmAdapter`).
+
+### Changed — `usedIngredients` + `rationale` graduate from `_debug.*` to first-class (§109, v0.5.1)
+
+`strategies/registry.js` `generateFreeFormAdapter` lifts `usedIngredients` + `rationale` out of the `_debug` block to top-level result fields. Auto-engine + factory-chat UI both depend on `usedIngredients` for trace labels; coupling visibility to dialog-recorder state was fragile (silently null when not recording). Soft-API addition — consumers reading `result.usedIngredients` get the array reliably; old `result._debug?.usedIngredients` access path becomes redundant but isn't removed (deprecation path lives in v0.5.x or v0.6.0).
+
+### Coverage at v0.5.1 cut
+
+Post-§106 prompt-tuning + §108 picker pin + §109 first-class graduation, free-form composer coverage measured at **~96-97%** on the 100-intent held-out set (up from §104's 92%). New AGENTS.md regression threshold floor: `cov≥96%, avg≥85, F1≥0.60` (§115 trip-wire baseline).
+
 ## [0.5.0] - 2026-05-13
 
 ### Added — Free-form composer auto-grouping (§103, v0.5.0)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@adia-ai/a2ui-compose",
-  "version": "0.5.0",
+  "version": "0.5.2",
   "description": "AdiaUI A2UI compose engine — framework-agnostic. Takes natural-language intents + a catalog and produces A2UI protocol messages. Pairs with `@adia-ai/a2ui-retrieval` (intent classification, catalog lookup) and `@adia-ai/a2ui-validator` (schema + semantic checks).",
   "type": "module",
   "exports": {

package/strategies/free-form-composer/free-form-composer.test.js

@@ -155,6 +155,81 @@ describe('free-form-composer', () => {
     expect(result.strategy).toBe('free-form-empty-vocab');
     expect(result.messages).toEqual([]);
   });
+
+  // §106 (v0.5.1) — empty-plan paraphrase retry
+  it('fires paraphrase retry when first attempt returns empty plan, recovers on retry', async () => {
+    let calls = 0;
+    const fakeLLM = {
+      complete: async ({ messages }) => {
+        calls++;
+        // First call: empty plan; second call: recover with valid plan.
+        const userMsg = messages[0].content;
+        if (calls === 1) {
+          // Empty plan — keyword mismatch.
+          return { content: JSON.stringify({ ingredients: [], rationale: 'no match' }) };
+        }
+        // Retry should carry the paraphrase hint about shape-matching.
+        expect(userMsg).toContain('shape');
+        return {
+          content: JSON.stringify({
+            ingredients: [{ name: 'demo-login' }],
+            rationale: 'shape-matched on retry',
+          }),
+        };
+      },
+    };
+    const result = await generateFreeForm({
+      intent: 'sign me in',
+      llmAdapter: fakeLLM,
+      compositions: FIXTURE_VOCAB,
+    });
+    expect(result.strategy).toBe('free-form-composed');
+    expect(result.usedIngredients).toEqual(['demo-login']);
+    expect(result.attempts).toBe(2);
+    expect(result.emptyPlanRetryUsed).toBe(true);
+    expect(calls).toBe(2);
+  });
+
+  it('paraphrase retry that still returns empty plan settles as free-form-empty-plan', async () => {
+    let calls = 0;
+    const fakeLLM = {
+      complete: async () => {
+        calls++;
+        return { content: JSON.stringify({ ingredients: [], rationale: 'still no match' }) };
+      },
+    };
+    const result = await generateFreeForm({
+      intent: 'something genuinely unmatchable',
+      llmAdapter: fakeLLM,
+      compositions: FIXTURE_VOCAB,
+    });
+    expect(result.strategy).toBe('free-form-empty-plan');
+    expect(result.attempts).toBe(2); // one paraphrase retry consumed; doesn't loop further
+    expect(result.emptyPlanRetryUsed).toBe(true);
+    expect(calls).toBe(2);
+    expect(result.messages).toEqual([]);
+  });
+
+  it('hallucinations exhaust MAX_ATTEMPTS=2; empty-plan paraphrase retry NOT reached on hallucination path', async () => {
+    let calls = 0;
+    const fakeLLM = {
+      complete: async () => {
+        calls++;
+        // Always hallucinate — should never reach the post-loop paraphrase path.
+        return { content: JSON.stringify({ ingredients: [{ name: 'fake-ingredient' }] }) };
+      },
+    };
+    const result = await generateFreeForm({
+      intent: 'login form',
+      llmAdapter: fakeLLM,
+      compositions: FIXTURE_VOCAB,
+    });
+    expect(result.strategy).toBe('free-form-hallucinated');
+    expect(result.attempts).toBe(2);
+    // emptyPlanRetryUsed is not set on hallucination return path
+    expect(result.emptyPlanRetryUsed).toBeUndefined();
+    expect(calls).toBe(2);
+  });
 });
 
 describe('parsePlan', () => {

package/strategies/free-form-composer/index.js

@@ -30,6 +30,7 @@ import {
   buildFreeFormSystemPrompt,
   buildFreeFormUserMessage,
   buildFreeFormRetryMessage,
+  buildFreeFormParaphraseRetry,
 } from './system-prompt.js';
 import { transpilePlan, parsePlan } from './transpile.js';
 
@@ -101,6 +102,7 @@ export async function generateFreeForm({ intent, llmAdapter = null, compositions
   let userMessage = buildFreeFormUserMessage(intent);
   let plan = null;
   let invalidNames = [];
+  let emptyPlanRetryUsed = false;
   let attempt = 0;
 
   for (attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) {
@@ -160,7 +162,38 @@ Note: your previous response wasn't valid JSON. Output ONLY the JSON object —
     };
   }
 
-  const { messages, warnings, usedIngredients } = transpilePlan(plan, chunkLookup);
+  let { messages, warnings, usedIngredients } = transpilePlan(plan, chunkLookup);
+
+  // §106 (v0.5.1): if the LLM declined to compose (`{ingredients: []}`),
+  // fire one paraphrase-retry with explicit shape-matching guidance. Many
+  // empty-plan cases on the v0.5.0 §104 eval were keyword-mismatch (chunk
+  // exists, intent uses different vocab). This retry is OUTSIDE the
+  // MAX_ATTEMPTS hallucination budget — one-shot reserve specifically for
+  // shape-match recovery.
+  if (messages.length === 0 && plan?.ingredients?.length === 0) {
+    emptyPlanRetryUsed = true;
+    const paraphraseMessage = buildFreeFormParaphraseRetry(intent);
+    try {
+      const response = await llmAdapter.complete({
+        messages: [{ role: 'user', content: paraphraseMessage }],
+        systemPrompt,
+      });
+      const retryPlan = parsePlan(response?.content || '');
+      const retryInvalid = retryPlan?.ingredients
+        ?.filter(ing => !ing || typeof ing.name !== 'string' || !vocabNames.has(ing.name))
+        .map(ing => ing?.name ?? '<missing-name>') ?? [];
+      attempt += 1;
+      if (retryPlan && retryInvalid.length === 0 && retryPlan.ingredients.length > 0) {
+        plan = retryPlan;
+        const retryTranspile = transpilePlan(plan, chunkLookup);
+        messages = retryTranspile.messages;
+        warnings = retryTranspile.warnings;
+        usedIngredients = retryTranspile.usedIngredients;
+      }
+    } catch {
+      // Paraphrase-retry LLM failure — fall through to empty-plan.
+    }
+  }
 
   if (messages.length === 0) {
     return {
@@ -172,6 +205,7 @@ Note: your previous response wasn't valid JSON. Output ONLY the JSON object —
       attempts: attempt,
       warnings,
       rationale: plan.rationale || null,
+      emptyPlanRetryUsed,
     };
   }
 
@@ -189,5 +223,6 @@ Note: your previous response wasn't valid JSON. Output ONLY the JSON object —
     attempts: attempt,
     warnings,
     rationale: plan.rationale || null,
+    emptyPlanRetryUsed,
   };
 }

package/strategies/free-form-composer/system-prompt.js

@@ -111,11 +111,36 @@ INGREDIENTS AVAILABLE (${ingredients.length}):
 
 ${catalog}
 
+INTENT PARAPHRASE — vocabulary vs shape:
+
+The user's intent often uses different vocabulary than the catalog's ingredient keywords. **Match on SHAPE — what the user wants to render — not exact keyword overlap.** Examples:
+
+- intent: "AI response with streaming text" → ingredient: \`ai-streaming-response\` (shape: assistant message bubble with token-streaming indicator). The intent doesn't say "message bubble" but the shape is identical.
+- intent: "real-time metrics with sparklines" → ingredient: \`real-time-metrics-dashboard\` and/or \`dashboard-spark-cards\` (shape: live-update metric grid).
+- intent: "modal dialog with a form" → ingredients: \`destructive-confirm-modal\` + \`form-page-shell\` (shape: overlay + stacked form fields).
+- intent: "settings panel" → ingredient: \`settings-form-page\` or \`account-settings-form\` (shape: titled stack of grouped fields).
+- intent: "team listing" → ingredients: \`avatar-stack\` + \`directory-table\` (shape: who's in the org).
+
+When in doubt, prefer to **emit a plausible 1-2-ingredient plan** based on shape match over returning empty-plan. Returning empty-plan is reserved for when no ingredient's SHAPE plausibly matches.
+
 CONSTRAINTS:
 
 1. Use ONLY ingredient names from the list above. Names not in the list are hallucinations — your response will be rejected if any \`name\` field is unknown.
 2. Order your ingredients in the sequence they should appear in the rendered UI. The transpiler wraps your list in a root container — by default a vertical Column. Override via the optional \`layout\` field (see below).
 3. Each ingredient may carry a \`substitutions\` object. KEYS are node IDs from the \`substitutables:\` line in the catalog above (e.g. \`"title"\`, \`"submit"\`, \`"logo"\`). VALUES are the new content strings. The transpiler routes each substitution to the right attribute based on the node's component: \`Text\` / \`Kbd\` → \`textContent\`; \`Button\` / \`Badge\` / \`Tag\` → \`text\`; \`Icon\` → \`name\`; \`Image\` → \`alt\`; \`Link\` → \`href\`. Nodes whose component is not in the substitutable list are locked at their declared values.
+
+   **ALWAYS substitute** any title, heading, button label, or badge label that the user's intent specifies. Default chunk text is generic placeholder ("Sign in to AdiaUI", "Continue", "Welcome back"); your job is to tailor it to the user's actual intent. Substituting MORE is better than substituting less — empty \`substitutions\` ships generic copy that misses the intent.
+
+   **Substitution examples per shape** (study these before emitting):
+
+   - **Text / Kbd (textContent)**: intent "trial-signup form for ContextEngine" → \`{"title": "Start your ContextEngine trial", "submit": "Create account"}\` not \`{}\`.
+   - **Button (text)**: intent "submit a support ticket" → \`{"submit-btn": "Open ticket"}\` not \`{"submit-btn": "Submit"}\`.
+   - **Badge / Tag (text)**: intent "show overdue invoices with status" → \`{"status-badge": "Overdue"}\` not \`{"status-badge": "Status"}\`.
+   - **Icon (name)**: intent "delete the project" → \`{"action-icon": "trash"}\` not omitting the icon substitution.
+   - **Image (alt)**: intent "team photo with three engineers" → \`{"hero-image": "Three engineers at a whiteboard"}\` not \`{}\`.
+   - **Link (href)**: intent "footer link to /docs/api" → \`{"docs-link": "/docs/api"}\` not \`{"docs-link": "#"}\`.
+
+   **Before emitting, self-check**: re-read the user's intent. Does it specify any content (text label, badge, icon, alt-text, URL)? If yes, every matching substitutable in your picked ingredients should appear in \`substitutions\`. If the intent is content-agnostic ("a sign-up form"), empty \`substitutions\` is correct.
 4. If you can't satisfy the intent with the available ingredients, return \`{ "ingredients": [] }\` and a rationale explaining what's missing.
 5. Output ONLY the JSON object below, no explanation outside the JSON.
 
@@ -166,3 +191,15 @@ export function buildFreeFormRetryMessage(intent, invalidNames) {
 
 Note: your previous response used ${invalidNames.length === 1 ? 'an ingredient name' : 'ingredient names'} that ${invalidNames.length === 1 ? "isn't" : "aren't"} in the catalog: ${list}. Use ONLY names from the INGREDIENTS list above, exactly as shown.`;
 }
+
+/**
+ * Build a paraphrase-retry user message for an empty-plan result. Used when
+ * the LLM returned a valid plan with `ingredients: []` — the keyword
+ * extraction didn't surface a shape match. The retry nudges the LLM toward
+ * shape-based reasoning rather than verbatim vocabulary lookup. §106 (v0.5.1).
+ */
+export function buildFreeFormParaphraseRetry(intent) {
+  return `Compose a UI for: "${intent}"
+
+Note: your previous response was an empty plan. Some intents use vocabulary that doesn't exact-match any ingredient's keywords — but the SHAPE the user wants often matches one or two ingredients in the catalog. Re-read the INGREDIENTS list and identify the closest shape match (e.g. "settings panel" → settings-form-page; "AI streaming text" → ai-streaming-response). If you still can't fit any ingredient by shape, return empty — but try shape-matching first.`;
+}

package/strategies/registry.js

@@ -118,13 +118,49 @@ async function generateZettelAdapter(ctx) {
   };
 }
 
+// §108 (v0.5.1): pin free-form's ingredient-picker to Haiku 4.5.
+// Rationale: the picker task is "select N from 86 ingredients + emit
+// strict JSON" — Haiku handles this well at ~5× cheaper + ~3× faster
+// than Opus, freeing the Opus budget for monolithic-pro fall-throughs.
+// User-decided pre-A/B per v0.5.1 plan question 2: "Haiku is a good
+// default."
+//
+// §127 (v0.5.2): env override for the A/B harness. Set
+// `FREE_FORM_MODEL_OVERRIDE` env var to e.g. `claude-opus-4-7` to run
+// the eval against Opus for the §127 Haiku-vs-Opus comparison.
+// Default behavior unchanged when env var unset — the Haiku-pin holds
+// for every consumer that doesn't explicitly override. Read at
+// call-time (not module-load) so eval harnesses can set the env after
+// static imports complete.
+const FREE_FORM_MODEL_DEFAULT = 'claude-haiku-4-5-20251001';
+
 async function generateFreeFormAdapter(ctx) {
   // Lazy-load: free-form-composer imports composition-library which
   // top-level-awaits a node:fs read. Same browser-safety story as zettel.
   const { generateFreeForm } = await import('./free-form-composer/index.js');
+
+  // Pin Haiku for the picker task even when harness model is Opus.
+  // Auto-engine + factory-chat may pass any-model llmAdapter; free-form
+  // mints its own to keep the picker cost-bounded.
+  //
+  // §127 (v0.5.2): `ctx.model` overrides the FREE_FORM pin (highest
+  // priority). `FREE_FORM_MODEL_OVERRIDE` env var is the second-priority
+  // override (set by `eval-diff.mjs --model <id>`). Default Haiku-pin
+  // holds otherwise. Empirical result from §127 records the production
+  // default; if Opus wins, FREE_FORM_MODEL_DEFAULT flips.
+  const modelToUse = ctx.model || process.env.FREE_FORM_MODEL_OVERRIDE || FREE_FORM_MODEL_DEFAULT;
+  let pickerAdapter = ctx.llmAdapter || null;
+  try {
+    const { createAdapter } = await import('../../../llm/llm-bridge.js');
+    pickerAdapter = await createAdapter({ model: modelToUse });
+  } catch {
+    // Adapter mint failed (proxy down, no key) — fall back to whatever
+    // ctx provides. Strategy will emit `free-form-no-llm` if both fail.
+  }
+
   const result = await generateFreeForm({
     intent: ctx.intent,
-    llmAdapter: ctx.llmAdapter || null,
+    llmAdapter: pickerAdapter,
   });
   const isRecording = await getIsRecording();
   return {
@@ -134,14 +170,32 @@ async function generateFreeFormAdapter(ctx) {
     suggestions: [],
     strategy: result.strategy,
     engine: 'free-form',
+    // §109 (v0.5.1): usedIngredients graduates from `_debug.*` to
+    // first-class. Auto-engine + factory-chat UI both depend on it for
+    // trace labels; coupling visibility to dialog-recorder state was
+    // fragile. `rationale` joins it for the same reason — trace
+    // copy quotes the LLM's one-line rationale when present.
+    usedIngredients: result.usedIngredients || [],
+    rationale: result.rationale || null,
+    // §107a (v0.5.1): plan graduates to first-class so eval-diff's
+    // `--report-substitutions` can read it without `_debug` gating.
+    // The plan carries the LLM's emitted substitutions per ingredient
+    // — the substitution-coverage measurement needs this surface even
+    // when dialog-recorder is off (production eval scenarios).
+    plan: result.plan || null,
+    // §131 (v0.5.2): `_debug` is **volatile** — dialog-recorder-gated,
+    // shape may change without semver coordination. Consumers MUST read
+    // first-class fields instead: `usedIngredients` (§109 v0.5.1),
+    // `rationale` (§109 v0.5.1), `plan` (§107a v0.5.2). The old soft-API
+    // paths (`_debug.usedIngredients`, `_debug.rationale`, `_debug.plan`)
+    // were removed by §109 / §107a; this block is now strictly debug-
+    // recorder fodder. Scheduled removal: v0.6.0 will fold `attempts` +
+    // `warnings` into first-class result fields and drop `_debug` here.
     _debug: isRecording() ? {
       systemPrompt: null,
       rawLLMResponse: null,
       tokens: null,
-      plan: result.plan,
-      usedIngredients: result.usedIngredients,
       attempts: result.attempts,
-      rationale: result.rationale,
       warnings: result.warnings,
     } : undefined,
   };