@json-layout/core 2.8.1 → 2.9.0

package/src/webmcp/README.md

@@ -0,0 +1,144 @@
+# WebMCP form tools — what they are and why they exist
+
+A page that renders a json-layout form can expose it to an agent as a set of MCP tools.
+This is the design of those tools, the choices behind them, and how far they have actually
+been validated.
+
+## Why not just hand the model the schema and let it emit JSON?
+
+This is the first question anyone asks, and for a small form it is the right instinct —
+`contact` is a 368-character schema with five fields, and a model would fill it blind.
+The suite exists because real configuration forms are not that, and four properties break
+the blind approach:
+
+**1. Half the schema does not apply until you have chosen something.** `charts` is five
+levels of nested `oneOf`: which fields are legal depends on the chart type, then on the
+data-preparation mode, then on the grouping kind. `calendar` hides a whole section behind
+`x-if: datasets.0.isRest` — the crowd-sourcing settings do not exist as valid keys until a
+REST dataset is picked. A model emitting a document in one shot has to guess which branch's
+fields are legal; the tools let it activate a branch and be told what that branch contains.
+
+**2. The accepted values are not in the schema at all.** `calendar`'s label field takes its
+options from `${rootData.datasets[0].href}/schema?calculated=false` — an HTTP request whose
+URL is built from another field's current value. No amount of schema reading yields
+`nomevenement`. `getFieldSuggestions` performs that lookup; nothing else can.
+
+**3. The values can be far larger than the answer.** A picked data-fair dataset is a 12,486-
+character object of column definitions. The agent never handles it: it applies suggestion
+index 0 and the form stores the object. Emitting that document by hand would mean
+reproducing 12 KB of column metadata verbatim.
+
+**4. The schema is bigger than the whole conversation.** `portal-page` is 285,874 characters,
+814 layout nodes. A complete `portal-page` session costs about 7.4 KB of tool output. Sending
+the schema once would cost roughly forty times the entire interaction.
+
+There is a fifth reason that matters in production: the form is *live*. Writes land in the
+state a user is looking at, validation runs on every one, and the errors come back scoped to
+what just changed. The agent is editing a form, not drafting a file.
+
+## The tools
+
+Six, and one guide. The guide is not a tool — production pages pass it to the agent as its
+prompt via `includeSubAgent`.
+
+| tool | what it is for |
+|---|---|
+| `describeState` | every field with path, type, constraints, current value, errors. The starting point and the only way to see the form. |
+| `setFieldValue` | write one field; switch a variant by writing its index. |
+| `setData` | write the whole document at once, for when the goal already contains every value. |
+| `editArray` | add or remove an item; an added item is activated and its fields returned. |
+| `getFieldSuggestions` | fetch the accepted values of a field whose list is remote. |
+| `getData` | the data itself, whole or at a path, for when the values are wanted rather than a description of them. |
+
+## Choices, and what justified them
+
+Each of these was measured, most of them after getting it wrong first.
+
+**Responses are text, never `structuredContent`.** Tools used to return both a markdown text
+and a JSON object. MCP clients substitute the JSON, so the markdown was never read — proved
+by reading the runner's own transcript, where an agent can be seen receiving
+`{"path":"/contactMethod","type":"select","getSuggestions":true}` with no values. Five
+finished pieces of work were inert. Removing the structured half took what the model receives
+for one `describeState` from 91,197 bytes to 2,212.
+
+**One representation, one home per fact.** The suggestion protocol was once stated four times
+— in two tool descriptions, a parameter, and the guide. Copies drift: a stale duplicate of
+`getData`'s description in `index.js` silently overrode the real one for weeks and cost two
+cases a call each. Tests now assert that exactly one description explains `suggestionIndex`.
+
+**One guide, no complexity classification.** There used to be small/medium/large bands, from
+counting layout nodes, feeding three different sets of advice. The threshold was
+unjustifiable, it measured the schema when the question was about data, and it was duplicated
+in two files. Four consecutive commits fixed contradictions between the branches, and across
+four baselines the small and medium advice was never taken on a real form.
+
+**No `getSchema`.** A build-time compiled layout carries no raw schema — `serialize()` emits
+the skeleton, the layouts and the validators — so the tool cannot exist where production runs.
+`calendar` and `charts` had it available in every run and never called it. What it uniquely
+offered, the validation keywords, now reaches the state tree instead: `format`, `pattern`,
+`minLength`, `maxLength`, `minItems`, `maxItems` and `uniqueItems` are carried onto the
+skeleton node, because ajv enforces them and nothing else was telling the agent.
+
+**Large values are named, never truncated.** `<object, 12486 chars — call getData with this
+path to read it>`. A truncated value read as real would be forwarded to an API as a wrong
+value with nothing looking wrong; a marker fails loudly. `getData` itself is never abbreviated
+for the same reason — what it returns is the data.
+
+**A closed list is stated, a remote one is flagged.** `describeState` prints
+`values=["top","bottom","left","right"]` when the options are already resolved, and marks
+`suggestions` when they are only reachable by request. Before this, agents spent a round trip
+per enum.
+
+**Repeated things are said once.** A recursive schema reaches the same union at every level;
+`portal-page`'s 39 branches were printed three times in a ten-call run. They are now printed
+once, then referred back to by path.
+
+**Memorised suggestions are dropped only when they go stale.** Applying an option by index
+needs the list remembered. Every write used to clear all of it, which broke a listing an
+unrelated write could not have affected. Invalidation now compares each path's
+`itemsCacheKey` — the same value the state layer uses to decide whether to re-fetch.
+
+## How far this is validated
+
+Unit tests: 463, covering tool shapes, projection rules and the compile-level changes in both
+runtime and build-time compilation.
+
+Behavioural: `core/webmcp-eval/` runs each case as a headless `claude -p` subprocess against
+a real MCP server, and a judge reads the transcript. Six cases — a hand-written control, three
+vendored from data-fair (`app-calendar`, `app-charts`, `portals` page editor), and two that
+start from a populated document and ask for a change.
+
+At the latest baseline, every case was judged satisfactory on **three model tiers**, with the
+produced document checked field by field rather than trusting `valid: true`:
+
+| | opus | sonnet | haiku |
+|---|---|---|---|
+| cases correct | 6/6 | 6/6 | 6/6 |
+| errored calls | 1 | 0 | 0 |
+| suite cost | $0.838 | $0.502 | $0.254 |
+
+Haiku 4.5 completed `portal-page` — the 39-branch recursive union — in eight calls, the floor,
+with no wasted read. The points where a smaller model was expected to slip are the ones the
+protocol leaves to inference: which of `children`/`children2` is the left column, that a
+rendered branch is the active one, that a field's options cannot be fetched before the field
+they derive from is set. None of them bit.
+
+## What is not established
+
+- **One MCP client.** Everything ran through Claude Code. The `structuredContent` episode is
+  the standing reminder that client behaviour can silently invalidate a whole line of work.
+- **The judge is not independent** — it is the same model family reading its own transcripts.
+- **n=1 per case per model**, against run-to-run variance that has reached 3× on cost for
+  identical code. Single-run comparisons are noise; only repeated, large effects are signal.
+- **`setData` with `merge: false`** has unit tests and no behavioural evidence — no agent has
+  ever called it.
+- Open findings are recorded per baseline in `core/webmcp-eval/baselines/`. All current ones
+  are clarity, not correctness: none has produced a wrong value or an invalid form.
+
+## A note on measuring
+
+Output bytes are what a tool answers with; they are not what a run costs. A `calendar` run
+consumes about 98,500 input-side tokens, 89,000 of them cache reads of the conversation
+replayed each turn, against roughly 1,300 output. One 13 KB response is about 3% of that.
+Three commits went into suppressing such a response before anyone measured it. The runner now
+records token usage and the report prints it beside the byte count.

package/src/webmcp/index.js

@@ -12,7 +12,10 @@ import * as getData from './tools/get-data.js'
 import * as getFieldSuggestions from './tools/get-field-suggestions.js'
 import * as editArray from './tools/edit-array.js'
 import * as fillFormSkill from './tools/fill-form-skill.js'
-import { formatMutationResult, formatSuggestions } from './project.js'
+import { formatMutationResult, formatSuggestions, projectSuggestions, abbreviateValue, formatVisibilityDiff, suggestionsBlocked, suggestionsSource } from './project.js'
+import { resolveNode } from './resolve.js'
+import { SuggestionsStore } from './suggestions-store.js'
+import { VariantsMemo } from './variants-memo.js'
 
 /** @typedef {import('@mcp-b/webmcp-types').ToolDescriptor} ToolDescriptor */
 
@@ -42,22 +45,10 @@ function parseIfJsonString (value) {
  * @typedef {object} WebMCPOptions
  * @property {string} [prefixName] - Prefix for all tool names
  * @property {string} [dataTitle] - Title used in descriptions (default: 'form')
- * @property {object} [schema] - The original JSON schema
  * @property {boolean} [includeFillFormSkill] - Include the fillFormSkill tool (default: false)
  * @property {boolean} [includeSubAgent] - Include a subagent_ tool wrapping all form tools (default: false)
  */
 
-/**
- * @param {import('../state/index.js').StatefulLayout} statefulLayout
- * @returns {"small"|"medium"|"large"}
- */
-function getComplexity (statefulLayout) {
-  const nbNormalizedLayouts = Object.keys(statefulLayout.compiledLayout.normalizedLayouts).length
-  if (nbNormalizedLayouts > 50) return 'large'
-  if (nbNormalizedLayouts > 15) return 'medium'
-  return 'small'
-}
-
 /**
  * WebMCP class that provides MCP tool descriptors for a StatefulLayout instance
  */
@@ -82,32 +73,35 @@ export class WebMCP {
 
   /**
    * @readonly
-   * @type {"small"|"medium"|"large"}
+   * @type {boolean}
    */
-  _complexity
+  _includeFillFormSkill = false
 
   /**
    * @readonly
-   * @type {object | null}
+   * @type {boolean}
    */
-  _schema = null
+  _includeSubAgent = false
 
   /**
-   * @readonly
-   * @type {boolean}
+   * @type {string[]}
    */
-  _includeFillFormSkill = false
+  _registeredTools = []
 
   /**
+   * memory of the last suggestions per node path, used by setFieldValue's suggestionIndex
    * @readonly
-   * @type {boolean}
+   * @type {SuggestionsStore}
    */
-  _includeSubAgent = false
+  _suggestionsStore = new SuggestionsStore()
 
   /**
-   * @type {string[]}
+   * Variant lists already sent to the agent. Never cleared on a write: a schema's branches
+   * are a constant, so unlike memorized suggestions nothing about the data can invalidate
+   * them.
+   * @type {VariantsMemo}
    */
-  _registeredTools = []
+  _variantsMemo = new VariantsMemo()
 
   /**
    * @param {import('../state/index.js').StatefulLayout} statefulLayout
@@ -117,10 +111,8 @@ export class WebMCP {
     this._statefulLayout = statefulLayout
     this._prefixName = options.prefixName || ''
     this._dataTitle = options.dataTitle || 'form'
-    this._schema = options.schema || null
     this._includeFillFormSkill = options.includeFillFormSkill || false
     this._includeSubAgent = options.includeSubAgent || false
-    this._complexity = getComplexity(statefulLayout)
   }
 
   /**
@@ -136,17 +128,15 @@ export class WebMCP {
    */
   getTools () {
     const dataTitle = this._dataTitle
-    const complexity = this._complexity
 
     /** @type {ToolDescriptor[]} */
     const tools = []
 
     if (this._includeFillFormSkill) {
-      const skill = fillFormSkill.generateSkill(dataTitle, this._prefixName, !!this._schema, this._statefulLayout)
+      const skill = fillFormSkill.generateSkill(dataTitle, this._prefixName)
       tools.push({
         name: this._toolName('fillFormSkill'),
         description: fillFormSkill.getDescription(dataTitle),
-        outputSchema: { type: 'string' },
         execute: async (args) => {
           try {
             return {
@@ -166,15 +156,18 @@ export class WebMCP {
     tools.push(
       {
         name: this._toolName('getData'),
-        description: `Get current "${dataTitle}" data and validity status. Call this first to see what data already exists.`,
+        description: getData.getDescription(dataTitle),
         inputSchema: getData.inputSchema,
-        outputSchema: getData.outputSchema,
         execute: async (args) => {
           try {
             const result = getData.execute(this._statefulLayout, args || {})
+            // Returned whole, always. This tool's answer IS the data: an agent may hand it
+            // to an API, and a document with named placeholders where its values should be
+            // would be forwarded as those strings with nothing looking wrong. Volume on a
+            // large form is a question of when to call this at all, which the guide
+            // answers — not a licence for the tool to answer with something else.
             return {
-              content: [{ type: 'text', text: JSON.stringify(result) }],
-              structuredContent: result
+              content: [{ type: 'text', text: JSON.stringify(result) }]
             }
           } catch (err) {
             const message = err instanceof Error ? err.message : String(err)
@@ -187,9 +180,8 @@ export class WebMCP {
       },
       {
         name: this._toolName('setData'),
-        description: setData.getDescription(dataTitle, complexity),
+        description: setData.getDescription(dataTitle),
         inputSchema: setData.inputSchema,
-        outputSchema: setData.outputSchema,
         execute: async (args) => {
           try {
             if (!args?.data) {
@@ -198,11 +190,22 @@ export class WebMCP {
             args.data = parseIfJsonString(args.data)
             const result = setData.execute(
               this._statefulLayout,
-              /** @type {{ data: unknown }} */(args)
+              /** @type {{ data: unknown, merge?: boolean }} */(args)
             )
+            // the whole data was replaced, what a memorized path designates may have changed
+            this._suggestionsStore.clear()
+            const warnings = []
+            if (result.removed.length) {
+              warnings.push(`removed ${result.removed.length} key(s) not present in the data you passed: ${result.removed.join(', ')} — pass merge=true to keep them`)
+            }
+            if (result.unknownKeys.length) {
+              warnings.push(`${result.unknownKeys.length} key(s) match no field of this form and were ignored by it: ${result.unknownKeys.join(', ')} — check for a typo with describeState`)
+            }
+            const visibilityInfo = result.visibility ? formatVisibilityDiff(result.visibility).replace(/^\n/, '') : ''
+            const stored = result.written.length ? `stored ${result.written.length} key(s): ${result.written.join(', ')}` : ''
+            const text = [formatMutationResult(result.valid, result.errors), ...(stored ? [stored] : []), ...(visibilityInfo ? [visibilityInfo] : []), ...warnings].join('\n')
             return {
-              content: [{ type: 'text', text: formatMutationResult(result.valid, result.errors) }],
-              structuredContent: result
+              content: [{ type: 'text', text }]
             }
           } catch (err) {
             const message = err instanceof Error ? err.message : String(err)
@@ -215,16 +218,13 @@ export class WebMCP {
       },
       {
         name: this._toolName('describeState'),
-        description: describeState.getDescription(dataTitle, complexity),
+        description: describeState.getDescription(dataTitle),
         inputSchema: describeState.inputSchema,
-        outputSchema: describeState.outputSchema,
         execute: async (args) => {
           try {
-            const result = describeState.execute(this._statefulLayout, args || {})
-            const text = describeState.toMarkdown(this._statefulLayout, args || {})
+            const text = describeState.toMarkdown(this._statefulLayout, args || {}, this._variantsMemo)
             return {
-              content: [{ type: 'text', text }],
-              structuredContent: result
+              content: [{ type: 'text', text }]
             }
           } catch (err) {
             const message = err instanceof Error ? err.message : String(err)
@@ -239,21 +239,34 @@ export class WebMCP {
         name: this._toolName('setFieldValue'),
         description: setFieldValue.getDescription(dataTitle),
         inputSchema: setFieldValue.inputSchema,
-        outputSchema: setFieldValue.outputSchema,
         execute: async (args) => {
           try {
             if (!args?.path) {
               throw new Error('path is required')
             }
-            args.value = parseIfJsonString(args.value)
+            if (args.value !== undefined) args.value = parseIfJsonString(args.value)
             const result = setFieldValue.execute(
               this._statefulLayout,
-              /** @type {{ path: string, value: unknown }} */(args)
+              /** @type {{ path: string, value?: unknown, suggestionIndex?: number }} */(args),
+              this._suggestionsStore,
+              this._variantsMemo
             )
-            const fieldInfo = `${result.field.path} (${result.field.type}) = ${JSON.stringify(result.field.data)}`
+            // A getItems expression can depend on another field, so this write may have
+            // changed the options of a field memorized under an unchanged path — but only
+            // of a field whose list actually depends on it. Comparing each memorized path's
+            // itemsCacheKey against the node's current one is how the state layer itself
+            // decides whether to re-fetch.
+            this._suggestionsStore.retainFresh((path, cacheKey) => {
+              const node = resolveNode(this._statefulLayout.stateTree.root, path)
+              return !!node && node.itemsCacheKey === cacheKey
+            })
+            let fieldInfo = `${result.field.path} (${result.field.type}) = ${abbreviateValue(result.field.data)}`
+            if (result.visibility) fieldInfo += formatVisibilityDiff(result.visibility)
+            if (result.activatedMarkdown) {
+              fieldInfo += `\nFields of the activated variant:\n${result.activatedMarkdown}`
+            }
             return {
-              content: [{ type: 'text', text: formatMutationResult(result.valid, result.errors, fieldInfo) }],
-              structuredContent: result
+              content: [{ type: 'text', text: formatMutationResult(result.valid, result.errors, fieldInfo, result.otherErrors) }]
             }
           } catch (err) {
             const message = err instanceof Error ? err.message : String(err)
@@ -266,9 +279,8 @@ export class WebMCP {
       },
       {
         name: this._toolName('getFieldSuggestions'),
-        description: `Get allowed values for a dropdown or autocomplete field in "${dataTitle}". Required when describeState shows "suggestions" for a field. Pass the returned value directly to setFieldValue or include it in setData.`,
+        description: getFieldSuggestions.getDescription(dataTitle),
         inputSchema: getFieldSuggestions.inputSchema,
-        outputSchema: getFieldSuggestions.outputSchema,
         execute: async (args) => {
           try {
             if (!args?.path) {
@@ -276,11 +288,17 @@ export class WebMCP {
             }
             const result = await getFieldSuggestions.execute(
               this._statefulLayout,
-              /** @type {{ path: string, query?: string }} */(args)
+              /** @type {{ path: string, query?: string }} */(args),
+              this._suggestionsStore
             )
+            const suggestions = projectSuggestions(result.items, result.baseIndex)
+            // An empty answer has two causes the agent must tell apart: a query that
+            // matched nothing, and a list whose request could not be built because another
+            // field is still empty. Only the second is a reason to go somewhere else.
+            const node = resolveNode(this._statefulLayout.stateTree.root, /** @type {any} */(args).path)
+            const blockedOn = node && suggestionsBlocked(node) ? suggestionsSource(node) : undefined
             return {
-              content: [{ type: 'text', text: formatSuggestions(result.items) }],
-              structuredContent: result
+              content: [{ type: 'text', text: formatSuggestions(suggestions, blockedOn) }]
             }
           } catch (err) {
             const message = err instanceof Error ? err.message : String(err)
@@ -295,7 +313,6 @@ export class WebMCP {
         name: this._toolName('editArray'),
         description: editArray.getDescription(dataTitle),
         inputSchema: editArray.inputSchema,
-        outputSchema: editArray.outputSchema,
         execute: async (args) => {
           try {
             if (!args?.path || !args?.action) {
@@ -306,14 +323,20 @@ export class WebMCP {
             }
             const result = editArray.execute(
               this._statefulLayout,
-              /** @type {{ path: string, action: 'add'|'remove', index?: number, value?: unknown }} */(args)
+              /** @type {{ path: string, action: 'add'|'remove', index?: number, value?: unknown }} */(args),
+              this._variantsMemo
             )
-            const actionInfo = args.action === 'add'
-              ? `added item, ${result.itemCount} total`
-              : `removed item, ${result.itemCount} remaining`
+            // adding or removing an item shifts the paths of the items after it, so the
+            // suggestions memorized for those paths now designate another item
+            this._suggestionsStore.clear()
+            let actionInfo = args.action === 'add'
+              ? `added item at index ${result.index}, ${result.itemCount} total`
+              : `removed item at index ${result.index}, ${result.itemCount} remaining`
+            if (result.itemMarkdown) {
+              actionInfo += `\nFields of the new item (activated for edition):\n${result.itemMarkdown}`
+            }
             return {
-              content: [{ type: 'text', text: formatMutationResult(result.valid, result.errors, actionInfo) }],
-              structuredContent: result
+              content: [{ type: 'text', text: formatMutationResult(result.valid, result.errors, actionInfo, result.otherErrors) }]
             }
           } catch (err) {
             const message = err instanceof Error ? err.message : String(err)
@@ -326,26 +349,9 @@ export class WebMCP {
       }
     )
 
-    if (this._schema) {
-      tools.push({
-        name: this._toolName('getSchema'),
-        description: `Get the JSON schema that governs the "${dataTitle}" form.`,
-        outputSchema: {
-          type: 'object',
-          description: 'The JSON schema definition'
-        },
-        execute: async (args) => {
-          return {
-            content: [{ type: 'text', text: JSON.stringify(this._schema) }],
-            structuredContent: this._schema
-          }
-        }
-      })
-    }
-
     if (this._includeSubAgent) {
       const toolNames = tools.map(t => t.name)
-      const prompt = fillFormSkill.generateSkill(dataTitle, this._prefixName, !!this._schema, this._statefulLayout)
+      const prompt = fillFormSkill.generateSkill(dataTitle, this._prefixName)
       tools.push({
         name: `subagent_${this._toolName('form')}`,
         description: `Delegate a form-filling task for "${dataTitle}" to a specialized sub-agent`,
@@ -357,9 +363,9 @@ export class WebMCP {
           required: ['task']
         },
         execute: async () => {
+          const config = { prompt, tools: toolNames }
           return {
-            content: [{ type: 'text', text: JSON.stringify({ prompt, tools: toolNames }) }],
-            structuredContent: { prompt, tools: toolNames }
+            content: [{ type: 'text', text: JSON.stringify(config) }]
           }
         }
       })