@geometra/mcp 1.19.12 → 1.19.14

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -21,6 +21,8 @@ Geometra proxy: Chromium → DOM geometry → same WebSocket as native →
21
21
  | `geometra_connect` | Connect with `url` (ws://…) **or** `pageUrl` (https://…) to auto-start geometra-proxy; `url: "https://…"` is auto-coerced onto the proxy path |
22
22
  | `geometra_query` | Find elements by stable id, role, name, text content, ancestor/prompt context, current value, or semantic state such as `invalid`, `required`, or `busy` |
23
23
  | `geometra_wait_for` | Wait for a semantic condition instead of guessing sleeps (`busy`, `disabled`, alerts, values, etc.) |
24
+ | `geometra_form_schema` | Compact, fill-oriented form schema with stable field ids and collapsed radio/button groups |
25
+ | `geometra_fill_form` | Fill a form from `valuesById` / `valuesByLabel` in one MCP call; preferred low-token happy path for standard forms |
24
26
  | `geometra_fill_fields` | Fill labeled text/choice/toggle/file fields in one MCP call; can return final-only status for the smallest responses |
25
27
  | `geometra_run_actions` | Execute a batch of high-level actions in one MCP round trip and get one consolidated result, with optional final-only output |
26
28
  | `geometra_page_model` | Summary-first webpage model: archetypes, stable section ids, counts, top-level sections, primary actions |
@@ -275,19 +277,15 @@ With `python3 -m http.server 8080` in `demos/proxy-mcp-sample` and `npx geometra
275
277
  Agent: geometra_connect({ url: "ws://127.0.0.1:3200" })
276
278
  → Connected. UI includes textbox "Email", button "Save", …
277
279
 
278
- Agent: geometra_page_model({})
279
- → {"viewport":{"width":1024,"height":768},"archetypes":["shell","form"],"summary":{...},"forms":[{"id":"fm:1.0","fieldCount":3,"actionCount":1}], ...}
280
+ Agent: geometra_form_schema({})
281
+ → {"forms":[{"formId":"fm:1.0","fields":[{"id":"ff:1.0.0","label":"Email"}, ...]}]}
280
282
 
281
- Agent: geometra_expand_section({ id: "fm:1.0" })
282
- {"id":"fm:1.0","kind":"form","fields":[{"id":"n:1.0.0","name":"Email"}, ...], "actions":[...]}
283
-
284
- Agent: geometra_query({ role: "textbox", name: "Email" })
285
- → bounds for the email field (viewport coordinates)
286
-
287
- Agent: geometra_click({ x: <center-x>, y: <center-y> })
288
- → Focuses the input
289
-
290
- Agent: geometra_type({ text: "hello@example.com" })
283
+ Agent: geometra_fill_form({
284
+ formId: "fm:1.0",
285
+ valuesByLabel: { "Email": "hello@example.com" },
286
+ failOnInvalid: true
287
+ })
288
+ → {"completed":true,"successCount":1,"errorCount":0,"final":{"invalidCount":0,...}}
291
289
 
292
290
  Agent: geometra_query({ role: "button", name: "Save" })
293
291
  → Click center to submit the sample form; status text updates in the DOM
@@ -299,21 +297,22 @@ Agent: geometra_query({ role: "button", name: "Save" })
299
297
  2. It receives the computed layout (`{ x, y, width, height }` for every node) and the UI tree (`kind`, `semantic`, `props`, `handlers`, `children`).
300
298
  3. It builds an accessibility tree from that data — roles, names, focusable state, bounds.
301
299
  4. **`geometra_snapshot`** defaults to a **compact** flat list of viewport-visible actionable nodes (minified JSON) to reduce LLM tokens; use `view: "full"` for the complete nested tree.
302
- 5. **`geometra_page_model`** is summary-first: page archetypes, stable section ids, counts, top-level landmarks/forms/dialogs/lists, and a few primary actions. It is designed to be cheaper than dumping full previews for every section.
303
- 6. **`geometra_expand_section`** fetches richer details only for the section you care about (fields, actions, headings, nested lists, list items, text preview).
304
- 7. After interactions, action tools return a **semantic delta** when possible (dialogs opened/closed, forms appeared/removed, list counts changed, named/focusable nodes added/removed/updated). If nothing meaningful changed, they fall back to a short current-UI overview.
305
- 8. Tools expose query, click, type, snapshot, page-model, and section-expansion operations over this structured data.
306
- 9. After each interaction, the peer sends updated geometry (full `frame` or `patch`) the MCP tools interpret that into compact summaries.
300
+ 5. **`geometra_form_schema`** is the compact form-specific path: stable field ids, required/invalid state, current values, and collapsed choice groups without layout-heavy section detail.
301
+ 6. **`geometra_fill_form`** turns a compact values object into semantic field operations server-side, so the model does not need to emit one tool call per field.
302
+ 7. **`geometra_page_model`** is still the right summary-first path for non-form exploration: page archetypes, stable section ids, counts, top-level landmarks/forms/dialogs/lists, and a few primary actions.
303
+ 8. **`geometra_expand_section`** fetches richer details only for the section you care about (fields, actions, headings, nested lists, list items, text preview).
304
+ 9. After interactions, action tools return a **semantic delta** when possible (dialogs opened/closed, forms appeared/removed, list counts changed, named/focusable nodes added/removed/updated). If nothing meaningful changed, they fall back to a short current-UI overview.
305
+ 10. After each interaction, the peer sends updated geometry (full `frame` or `patch`) — the MCP tools interpret that into compact summaries.
307
306
 
308
307
  ## Long Forms
309
308
 
310
309
  For long application flows, prefer one of these patterns:
311
310
 
312
- 1. `geometra_page_model`
313
- 2. `geometra_expand_section`
311
+ 1. `geometra_form_schema`
312
+ 2. `geometra_fill_form`
314
313
  3. `geometra_reveal` for far-below-fold targets such as submit buttons
315
- 4. `geometra_fill_fields` for obvious field entry
316
- 5. `geometra_run_actions` when you need mixed navigation + waits + field entry
314
+ 4. `geometra_run_actions` when you need mixed navigation + waits + field entry
315
+ 5. `geometra_page_model` + `geometra_expand_section` when you are still exploring the page rather than filling it
317
316
 
318
317
  Typical batch:
319
318
 
@@ -335,6 +334,23 @@ For the smallest long-form responses, prefer:
335
334
  1. `detail: "minimal"` for structured step metadata instead of narrated deltas
336
335
  2. `includeSteps: false` when you only need aggregate success/error counts plus the final validation/state payload
337
336
 
337
+ Typical low-token form fill:
338
+
339
+ ```json
340
+ {
341
+ "formId": "fm:1.0",
342
+ "valuesById": {
343
+ "ff:1.0.0": "Taylor Applicant",
344
+ "ff:1.0.1": "taylor@example.com",
345
+ "ff:1.0.2": "Germany",
346
+ "ff:1.0.3": "No"
347
+ },
348
+ "failOnInvalid": true,
349
+ "includeSteps": false,
350
+ "detail": "minimal"
351
+ }
352
+ ```
353
+
338
354
  For long single-page forms:
339
355
 
340
356
  1. Use `geometra_expand_section` with `fieldOffset` / `actionOffset` to page through large forms instead of taking a full snapshot.
@@ -1,4 +1,4 @@
1
- import { existsSync, mkdirSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from 'node:fs';
1
+ import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
2
2
  import { createRequire } from 'node:module';
3
3
  import { tmpdir } from 'node:os';
4
4
  import path from 'node:path';
@@ -78,7 +78,22 @@ describe('proxy ready helpers', () => {
78
78
  const packageDir = path.join(scopeDir, 'proxy');
79
79
  const probePath = path.join(tempRoot, 'probe.cjs');
80
80
  mkdirSync(scopeDir, { recursive: true });
81
- symlinkSync(path.resolve(process.cwd(), 'packages/proxy'), packageDir, 'dir');
81
+ mkdirSync(path.join(packageDir, 'src'), { recursive: true });
82
+ writeFileSync(path.join(packageDir, 'package.json'), JSON.stringify({
83
+ name: '@geometra/proxy',
84
+ version: '0.0.0-test',
85
+ type: 'module',
86
+ }));
87
+ writeFileSync(path.join(packageDir, 'tsconfig.build.json'), JSON.stringify({
88
+ extends: path.resolve(process.cwd(), 'tsconfig.base.json'),
89
+ compilerOptions: {
90
+ outDir: 'dist',
91
+ rootDir: 'src',
92
+ noEmit: false,
93
+ },
94
+ include: ['src'],
95
+ }));
96
+ writeFileSync(path.join(packageDir, 'src', 'index.ts'), 'console.log("proxy");\n');
82
97
  writeFileSync(probePath, 'module.exports = {}');
83
98
  const customRequire = createRequire(probePath);
84
99
  const scriptPath = resolveProxyScriptPathWith(customRequire);
@@ -89,6 +104,33 @@ describe('proxy ready helpers', () => {
89
104
  rmSync(tempRoot, { recursive: true, force: true });
90
105
  }
91
106
  });
107
+ it('prefers the current workspace proxy dist over a bundled nested dependency in source checkouts', () => {
108
+ const tempRoot = mkdtempSync(path.join(tmpdir(), 'geometra-proxy-workspace-prefer-'));
109
+ try {
110
+ const workspaceDistDir = path.join(tempRoot, 'packages', 'proxy', 'dist');
111
+ const bundledProxyDir = path.join(tempRoot, 'mcp', 'node_modules', '@geometra', 'proxy');
112
+ const bundledDistDir = path.join(bundledProxyDir, 'dist');
113
+ const mcpDistDir = path.join(tempRoot, 'mcp', 'dist');
114
+ const probePath = path.join(mcpDistDir, 'proxy-spawn.cjs');
115
+ mkdirSync(workspaceDistDir, { recursive: true });
116
+ mkdirSync(bundledDistDir, { recursive: true });
117
+ mkdirSync(mcpDistDir, { recursive: true });
118
+ writeFileSync(path.join(workspaceDistDir, 'index.js'), 'export const source = "workspace";\n');
119
+ writeFileSync(path.join(bundledDistDir, 'index.js'), 'export const source = "bundled";\n');
120
+ writeFileSync(path.join(bundledProxyDir, 'package.json'), JSON.stringify({
121
+ name: '@geometra/proxy',
122
+ version: '0.0.0-test',
123
+ type: 'module',
124
+ }));
125
+ writeFileSync(probePath, 'module.exports = {};\n');
126
+ const customRequire = createRequire(probePath);
127
+ const scriptPath = resolveProxyScriptPathWith(customRequire, mcpDistDir);
128
+ expect(scriptPath).toBe(path.join(workspaceDistDir, 'index.js'));
129
+ }
130
+ finally {
131
+ rmSync(tempRoot, { recursive: true, force: true });
132
+ }
133
+ });
92
134
  it('falls back to the packaged sibling proxy dist when package exports are stale', () => {
93
135
  const tempRoot = mkdtempSync(path.join(tmpdir(), 'geometra-proxy-stale-exports-'));
94
136
  try {
@@ -1,6 +1,6 @@
1
1
  import { afterAll, describe, expect, it } from 'vitest';
2
2
  import { WebSocketServer } from 'ws';
3
- import { connect, disconnect, sendClick, sendListboxPick } from '../session.js';
3
+ import { connect, disconnect, sendClick, sendFillFields, sendListboxPick } from '../session.js';
4
4
  describe('proxy-backed MCP actions', () => {
5
5
  afterAll(() => {
6
6
  disconnect();
@@ -97,6 +97,83 @@ describe('proxy-backed MCP actions', () => {
97
97
  await new Promise((resolve, reject) => wss.close(err => (err ? reject(err) : resolve())));
98
98
  }
99
99
  });
100
+ it('sends batched fillFields messages and resolves from the resulting update', async () => {
101
+ const wss = new WebSocketServer({ port: 0 });
102
+ let seenMessage;
103
+ wss.on('connection', ws => {
104
+ ws.on('message', raw => {
105
+ const msg = JSON.parse(String(raw));
106
+ if (msg.type === 'resize') {
107
+ ws.send(JSON.stringify({
108
+ type: 'frame',
109
+ layout: { x: 0, y: 0, width: 1024, height: 768, children: [] },
110
+ tree: { kind: 'box', props: {}, semantic: { tag: 'body', role: 'group' }, children: [] },
111
+ }));
112
+ return;
113
+ }
114
+ if (msg.type === 'fillFields') {
115
+ seenMessage = msg;
116
+ ws.send(JSON.stringify({
117
+ type: 'ack',
118
+ requestId: msg.requestId,
119
+ result: {
120
+ pageUrl: 'https://jobs.example.com/application',
121
+ invalidCount: 0,
122
+ alertCount: 0,
123
+ dialogCount: 0,
124
+ busyCount: 0,
125
+ },
126
+ }));
127
+ ws.send(JSON.stringify({
128
+ type: 'frame',
129
+ layout: { x: 0, y: 0, width: 1024, height: 768, children: [] },
130
+ tree: {
131
+ kind: 'box',
132
+ props: {},
133
+ semantic: { tag: 'body', role: 'group', ariaLabel: 'Filled' },
134
+ children: [],
135
+ },
136
+ }));
137
+ }
138
+ });
139
+ });
140
+ const port = await new Promise((resolve, reject) => {
141
+ wss.once('listening', () => {
142
+ const address = wss.address();
143
+ if (typeof address === 'object' && address)
144
+ resolve(address.port);
145
+ else
146
+ reject(new Error('Failed to resolve ephemeral WebSocket port'));
147
+ });
148
+ wss.once('error', reject);
149
+ });
150
+ try {
151
+ const session = await connect(`ws://127.0.0.1:${port}`);
152
+ await expect(sendFillFields(session, [
153
+ { kind: 'text', fieldLabel: 'Full name', value: 'Taylor Applicant' },
154
+ { kind: 'choice', fieldLabel: 'Country', value: 'Germany' },
155
+ ], 80)).resolves.toMatchObject({
156
+ status: 'acknowledged',
157
+ timeoutMs: 80,
158
+ result: {
159
+ pageUrl: 'https://jobs.example.com/application',
160
+ invalidCount: 0,
161
+ alertCount: 0,
162
+ },
163
+ });
164
+ expect(seenMessage).toMatchObject({
165
+ type: 'fillFields',
166
+ fields: [
167
+ { kind: 'text', fieldLabel: 'Full name', value: 'Taylor Applicant' },
168
+ { kind: 'choice', fieldLabel: 'Country', value: 'Germany' },
169
+ ],
170
+ });
171
+ }
172
+ finally {
173
+ disconnect();
174
+ await new Promise((resolve, reject) => wss.close(err => (err ? reject(err) : resolve())));
175
+ }
176
+ });
100
177
  it('ignores invalid patch paths instead of mutating ancestor layout nodes', async () => {
101
178
  const wss = new WebSocketServer({ port: 0 });
102
179
  wss.on('connection', ws => {
@@ -23,12 +23,20 @@ const mockState = vi.hoisted(() => ({
23
23
  url: 'ws://127.0.0.1:3200',
24
24
  updateRevision: 1,
25
25
  },
26
+ formSchemas: [],
27
+ connect: vi.fn(),
28
+ connectThroughProxy: vi.fn(),
26
29
  sendClick: vi.fn(async () => ({ status: 'updated', timeoutMs: 2000 })),
27
30
  sendType: vi.fn(async () => ({ status: 'updated', timeoutMs: 2000 })),
28
31
  sendKey: vi.fn(async () => ({ status: 'updated', timeoutMs: 2000 })),
29
32
  sendFileUpload: vi.fn(async () => ({ status: 'updated', timeoutMs: 8000 })),
30
33
  sendFieldText: vi.fn(async () => ({ status: 'updated', timeoutMs: 2000 })),
31
34
  sendFieldChoice: vi.fn(async () => ({ status: 'updated', timeoutMs: 2000 })),
35
+ sendFillFields: vi.fn(async () => ({
36
+ status: 'updated',
37
+ timeoutMs: 6000,
38
+ result: undefined,
39
+ })),
32
40
  sendListboxPick: vi.fn(async () => ({ status: 'updated', timeoutMs: 4500 })),
33
41
  sendSelectOption: vi.fn(async () => ({ status: 'updated', timeoutMs: 2000 })),
34
42
  sendSetChecked: vi.fn(async () => ({ status: 'updated', timeoutMs: 2000 })),
@@ -36,8 +44,8 @@ const mockState = vi.hoisted(() => ({
36
44
  waitForUiCondition: vi.fn(async () => true),
37
45
  }));
38
46
  vi.mock('../session.js', () => ({
39
- connect: vi.fn(),
40
- connectThroughProxy: vi.fn(),
47
+ connect: mockState.connect,
48
+ connectThroughProxy: mockState.connectThroughProxy,
41
49
  disconnect: vi.fn(),
42
50
  getSession: vi.fn(() => mockState.session),
43
51
  sendClick: mockState.sendClick,
@@ -46,6 +54,7 @@ vi.mock('../session.js', () => ({
46
54
  sendFileUpload: mockState.sendFileUpload,
47
55
  sendFieldText: mockState.sendFieldText,
48
56
  sendFieldChoice: mockState.sendFieldChoice,
57
+ sendFillFields: mockState.sendFillFields,
49
58
  sendListboxPick: mockState.sendListboxPick,
50
59
  sendSelectOption: mockState.sendSelectOption,
51
60
  sendSetChecked: mockState.sendSetChecked,
@@ -62,6 +71,7 @@ vi.mock('../session.js', () => ({
62
71
  dialogs: [],
63
72
  lists: [],
64
73
  })),
74
+ buildFormSchemas: vi.fn(() => mockState.formSchemas),
65
75
  expandPageSection: vi.fn(() => null),
66
76
  buildUiDelta: vi.fn(() => ({})),
67
77
  hasUiDelta: vi.fn(() => false),
@@ -79,6 +89,9 @@ function getToolHandler(name) {
79
89
  describe('batch MCP result shaping', () => {
80
90
  beforeEach(() => {
81
91
  vi.clearAllMocks();
92
+ mockState.connect.mockResolvedValue(mockState.session);
93
+ mockState.connectThroughProxy.mockResolvedValue(mockState.session);
94
+ mockState.formSchemas = [];
82
95
  mockState.currentA11yRoot = node('group', undefined, {
83
96
  meta: { pageUrl: 'https://jobs.example.com/application', scrollX: 0, scrollY: 420 },
84
97
  children: [
@@ -203,6 +216,186 @@ describe('batch MCP result shaping', () => {
203
216
  expect(final.invalidFields.length).toBe(4);
204
217
  expect(final.alerts.length).toBe(1);
205
218
  });
219
+ it('returns a compact structured connect payload by default', async () => {
220
+ const handler = getToolHandler('geometra_connect');
221
+ const result = await handler({
222
+ pageUrl: 'https://jobs.example.com/application',
223
+ headless: true,
224
+ });
225
+ const payload = JSON.parse(result.content[0].text);
226
+ expect(payload).toMatchObject({
227
+ connected: true,
228
+ transport: 'proxy',
229
+ wsUrl: 'ws://127.0.0.1:3200',
230
+ pageUrl: 'https://jobs.example.com/application',
231
+ });
232
+ expect(payload).not.toHaveProperty('currentUi');
233
+ });
234
+ it('returns compact form schemas without requiring section expansion', async () => {
235
+ const handler = getToolHandler('geometra_form_schema');
236
+ mockState.formSchemas = [
237
+ {
238
+ formId: 'fm:0',
239
+ name: 'Application',
240
+ fieldCount: 4,
241
+ requiredCount: 3,
242
+ invalidCount: 0,
243
+ fields: [
244
+ { id: 'ff:0.0', kind: 'text', label: 'Full name', required: true },
245
+ { id: 'ff:0.1', kind: 'choice', label: 'Preferred location', required: true },
246
+ { id: 'ff:0.2', kind: 'choice', label: 'Are you legally authorized to work in Germany?', options: ['Yes', 'No'], optionCount: 2 },
247
+ { id: 'ff:0.3', kind: 'toggle', label: 'Share my profile for future roles', controlType: 'checkbox' },
248
+ ],
249
+ },
250
+ ];
251
+ const result = await handler({ maxFields: 20 });
252
+ const payload = JSON.parse(result.content[0].text);
253
+ expect(payload.forms).toEqual([
254
+ expect.objectContaining({
255
+ formId: 'fm:0',
256
+ fieldCount: 4,
257
+ requiredCount: 3,
258
+ invalidCount: 0,
259
+ }),
260
+ ]);
261
+ });
262
+ it('fills a form from ids and labels without echoing long essay content', async () => {
263
+ const longAnswer = 'B'.repeat(220);
264
+ const handler = getToolHandler('geometra_fill_form');
265
+ mockState.formSchemas = [
266
+ {
267
+ formId: 'fm:0',
268
+ name: 'Application',
269
+ fieldCount: 4,
270
+ requiredCount: 3,
271
+ invalidCount: 0,
272
+ fields: [
273
+ { id: 'ff:0.0', kind: 'text', label: 'Full name', required: true },
274
+ { id: 'ff:0.1', kind: 'choice', label: 'Are you legally authorized to work in Germany?', options: ['Yes', 'No'], optionCount: 2 },
275
+ { id: 'ff:0.2', kind: 'toggle', label: 'Share my profile for future roles', controlType: 'checkbox' },
276
+ { id: 'ff:0.3', kind: 'text', label: 'Why Geometra?' },
277
+ ],
278
+ },
279
+ ];
280
+ mockState.currentA11yRoot = node('group', undefined, {
281
+ meta: { pageUrl: 'https://jobs.example.com/application', scrollX: 0, scrollY: 640 },
282
+ children: [
283
+ node('textbox', 'Full name', { value: 'Taylor Applicant', path: [0] }),
284
+ node('textbox', 'Why Geometra?', { value: longAnswer, path: [1] }),
285
+ node('checkbox', 'Share my profile for future roles', {
286
+ path: [2],
287
+ state: { checked: true },
288
+ }),
289
+ ],
290
+ });
291
+ const result = await handler({
292
+ valuesById: {
293
+ 'ff:0.0': 'Taylor Applicant',
294
+ },
295
+ valuesByLabel: {
296
+ 'Are you legally authorized to work in Germany?': true,
297
+ 'Share my profile for future roles': true,
298
+ 'Why Geometra?': longAnswer,
299
+ },
300
+ includeSteps: true,
301
+ detail: 'minimal',
302
+ });
303
+ const text = result.content[0].text;
304
+ const payload = JSON.parse(text);
305
+ const steps = payload.steps;
306
+ expect(text).not.toContain(longAnswer);
307
+ expect(mockState.sendFieldChoice).toHaveBeenCalledWith(mockState.session, 'Are you legally authorized to work in Germany?', 'Yes', { exact: undefined, query: undefined }, undefined);
308
+ expect(payload).toMatchObject({
309
+ completed: true,
310
+ formId: 'fm:0',
311
+ requestedValueCount: 4,
312
+ fieldCount: 4,
313
+ successCount: 4,
314
+ errorCount: 0,
315
+ });
316
+ expect(steps[3]).toMatchObject({
317
+ kind: 'text',
318
+ fieldLabel: 'Why Geometra?',
319
+ valueLength: 220,
320
+ readback: { role: 'textbox', valueLength: 220 },
321
+ });
322
+ });
323
+ it('uses batched proxy fill for compact fill_form responses', async () => {
324
+ const handler = getToolHandler('geometra_fill_form');
325
+ mockState.sendFillFields.mockResolvedValueOnce({
326
+ status: 'acknowledged',
327
+ timeoutMs: 6000,
328
+ result: {
329
+ pageUrl: 'https://jobs.example.com/application',
330
+ invalidCount: 0,
331
+ alertCount: 0,
332
+ dialogCount: 0,
333
+ busyCount: 0,
334
+ },
335
+ });
336
+ mockState.formSchemas = [
337
+ {
338
+ formId: 'fm:0',
339
+ name: 'Application',
340
+ fieldCount: 3,
341
+ requiredCount: 2,
342
+ invalidCount: 0,
343
+ fields: [
344
+ { id: 'ff:0.0', kind: 'text', label: 'Full name', required: true },
345
+ { id: 'ff:0.1', kind: 'choice', label: 'Preferred location', required: true },
346
+ { id: 'ff:0.2', kind: 'toggle', label: 'Share my profile for future roles', controlType: 'checkbox' },
347
+ ],
348
+ },
349
+ ];
350
+ mockState.currentA11yRoot = node('group', undefined, {
351
+ meta: { pageUrl: 'https://jobs.example.com/application', scrollX: 0, scrollY: 640 },
352
+ children: [
353
+ node('textbox', 'Full name', { value: 'Taylor Applicant', path: [0] }),
354
+ node('combobox', 'Preferred location', { value: 'Berlin, Germany', path: [1] }),
355
+ node('checkbox', 'Share my profile for future roles', {
356
+ path: [2],
357
+ state: { checked: true },
358
+ }),
359
+ ],
360
+ });
361
+ const result = await handler({
362
+ valuesById: {
363
+ 'ff:0.0': 'Taylor Applicant',
364
+ 'ff:0.1': 'Berlin, Germany',
365
+ 'ff:0.2': true,
366
+ },
367
+ includeSteps: false,
368
+ detail: 'minimal',
369
+ });
370
+ const payload = JSON.parse(result.content[0].text);
371
+ expect(mockState.sendFillFields).toHaveBeenCalledWith(mockState.session, [
372
+ { kind: 'text', fieldLabel: 'Full name', value: 'Taylor Applicant' },
373
+ { kind: 'choice', fieldLabel: 'Preferred location', value: 'Berlin, Germany' },
374
+ {
375
+ kind: 'toggle',
376
+ label: 'Share my profile for future roles',
377
+ checked: true,
378
+ controlType: 'checkbox',
379
+ },
380
+ ]);
381
+ expect(mockState.sendFieldText).not.toHaveBeenCalled();
382
+ expect(mockState.sendFieldChoice).not.toHaveBeenCalled();
383
+ expect(mockState.sendSetChecked).not.toHaveBeenCalled();
384
+ expect(payload).toMatchObject({
385
+ completed: true,
386
+ execution: 'batched',
387
+ finalSource: 'proxy',
388
+ formId: 'fm:0',
389
+ fieldCount: 3,
390
+ successCount: 3,
391
+ errorCount: 0,
392
+ final: {
393
+ invalidCount: 0,
394
+ alertCount: 0,
395
+ },
396
+ });
397
+ expect(payload).not.toHaveProperty('steps');
398
+ });
206
399
  });
207
400
  describe('query and reveal tools', () => {
208
401
  beforeEach(() => {
@@ -1,5 +1,5 @@
1
1
  import { describe, expect, it } from 'vitest';
2
- import { buildA11yTree, buildCompactUiIndex, buildPageModel, expandPageSection, buildUiDelta, hasUiDelta, summarizeUiDelta, } from '../session.js';
2
+ import { buildA11yTree, buildCompactUiIndex, buildFormSchemas, buildPageModel, expandPageSection, buildUiDelta, hasUiDelta, summarizeUiDelta, } from '../session.js';
3
3
  function node(role, name, bounds, options) {
4
4
  return {
5
5
  role,
@@ -237,6 +237,135 @@ describe('buildPageModel', () => {
237
237
  expect(model.forms[0]?.name).toBeUndefined();
238
238
  });
239
239
  });
240
+ describe('buildFormSchemas', () => {
241
+ it('builds a compact fill-oriented schema and collapses repeated answer groups', () => {
242
+ const longEssay = 'Semantic browser automation should be reliable, compact, and predictable across large forms.';
243
+ const tree = node('group', undefined, { x: 0, y: 0, width: 1024, height: 768 }, {
244
+ children: [
245
+ node('form', 'Application', { x: 32, y: 32, width: 760, height: 1500 }, {
246
+ path: [0],
247
+ children: [
248
+ node('textbox', 'Full name', { x: 48, y: 120, width: 320, height: 36 }, {
249
+ path: [0, 0],
250
+ state: { required: true },
251
+ }),
252
+ node('combobox', 'Preferred location', { x: 48, y: 180, width: 320, height: 36 }, {
253
+ path: [0, 1],
254
+ state: { required: true },
255
+ value: 'Berlin, Germany',
256
+ }),
257
+ node('group', undefined, { x: 40, y: 260, width: 520, height: 96 }, {
258
+ path: [0, 2],
259
+ children: [
260
+ node('text', 'Are you legally authorized to work in Germany?', { x: 48, y: 260, width: 360, height: 24 }, {
261
+ path: [0, 2, 0],
262
+ }),
263
+ node('button', 'Yes', { x: 48, y: 300, width: 88, height: 40 }, {
264
+ path: [0, 2, 1],
265
+ focusable: true,
266
+ state: { required: true },
267
+ }),
268
+ node('button', 'No', { x: 148, y: 300, width: 88, height: 40 }, {
269
+ path: [0, 2, 2],
270
+ focusable: true,
271
+ }),
272
+ ],
273
+ }),
274
+ node('checkbox', 'Share my profile for future roles', { x: 48, y: 400, width: 24, height: 24 }, {
275
+ path: [0, 3],
276
+ focusable: true,
277
+ state: { checked: true },
278
+ }),
279
+ node('textbox', 'Why Geometra?', { x: 48, y: 480, width: 520, height: 180 }, {
280
+ path: [0, 4],
281
+ state: { required: true, invalid: true },
282
+ validation: { error: 'Please enter at least 40 characters.' },
283
+ value: longEssay,
284
+ }),
285
+ ],
286
+ }),
287
+ ],
288
+ });
289
+ const schemas = buildFormSchemas(tree);
290
+ expect(schemas).toHaveLength(1);
291
+ expect(schemas[0]).toMatchObject({
292
+ formId: 'fm:0',
293
+ name: 'Application',
294
+ fieldCount: 5,
295
+ requiredCount: 4,
296
+ invalidCount: 1,
297
+ });
298
+ expect(schemas[0]?.fields).toEqual([
299
+ expect.objectContaining({
300
+ kind: 'text',
301
+ label: 'Full name',
302
+ required: true,
303
+ }),
304
+ expect.objectContaining({
305
+ kind: 'choice',
306
+ label: 'Preferred location',
307
+ required: true,
308
+ value: 'Berlin, Germany',
309
+ }),
310
+ expect.objectContaining({
311
+ kind: 'choice',
312
+ label: 'Are you legally authorized to work in Germany?',
313
+ required: true,
314
+ optionCount: 2,
315
+ options: ['Yes', 'No'],
316
+ }),
317
+ expect.objectContaining({
318
+ kind: 'toggle',
319
+ label: 'Share my profile for future roles',
320
+ checked: true,
321
+ controlType: 'checkbox',
322
+ }),
323
+ expect.objectContaining({
324
+ kind: 'text',
325
+ label: 'Why Geometra?',
326
+ required: true,
327
+ invalid: true,
328
+ valueLength: longEssay.length,
329
+ }),
330
+ ]);
331
+ });
332
+ it('prefers question prompts over nearby explanatory copy for grouped choices', () => {
333
+ const tree = node('group', undefined, { x: 0, y: 0, width: 900, height: 700 }, {
334
+ children: [
335
+ node('form', 'Application', { x: 20, y: 20, width: 760, height: 480 }, {
336
+ path: [0],
337
+ children: [
338
+ node('group', undefined, { x: 32, y: 80, width: 520, height: 120 }, {
339
+ path: [0, 0],
340
+ children: [
341
+ node('text', 'Will you now or in the future require sponsorship?', { x: 40, y: 80, width: 420, height: 24 }, {
342
+ path: [0, 0, 0],
343
+ }),
344
+ node('text', 'This intentionally repeats Yes / No labels to test contextual disambiguation.', { x: 40, y: 112, width: 520, height: 24 }, {
345
+ path: [0, 0, 1],
346
+ }),
347
+ node('button', 'Yes', { x: 40, y: 152, width: 88, height: 40 }, {
348
+ path: [0, 0, 2],
349
+ focusable: true,
350
+ }),
351
+ node('button', 'No', { x: 140, y: 152, width: 88, height: 40 }, {
352
+ path: [0, 0, 3],
353
+ focusable: true,
354
+ }),
355
+ ],
356
+ }),
357
+ ],
358
+ }),
359
+ ],
360
+ });
361
+ const schema = buildFormSchemas(tree)[0];
362
+ expect(schema?.fields[0]).toMatchObject({
363
+ kind: 'choice',
364
+ label: 'Will you now or in the future require sponsorship?',
365
+ options: ['Yes', 'No'],
366
+ });
367
+ });
368
+ });
240
369
  describe('buildUiDelta', () => {
241
370
  it('captures opened dialogs, state changes, and list count changes', () => {
242
371
  const before = node('group', undefined, { x: 0, y: 0, width: 1024, height: 768 }, {