@geometra/mcp 1.19.9 → 1.19.11

package/README.md

@@ -19,13 +19,16 @@ Geometra proxy:       Chromium → DOM geometry → same WebSocket as native →
 | Tool | Description |
 |---|---|
 | `geometra_connect` | Connect with `url` (ws://…) **or** `pageUrl` (https://…) to auto-start geometra-proxy; `url: "https://…"` is auto-coerced onto the proxy path |
-| `geometra_query` | Find elements by stable id, role, name, or text content |
+| `geometra_query` | Find elements by stable id, role, name, text content, current value, or semantic state such as `invalid`, `required`, or `busy` |
+| `geometra_wait_for` | Wait for a semantic condition instead of guessing sleeps (`busy`, `disabled`, alerts, values, etc.) |
+| `geometra_fill_fields` | Fill labeled text/choice/toggle/file fields in one MCP call; can return final-only status for the smallest responses |
+| `geometra_run_actions` | Execute a batch of high-level actions in one MCP round trip and get one consolidated result, with optional final-only output |
 | `geometra_page_model` | Summary-first webpage model: archetypes, stable section ids, counts, top-level sections, primary actions |
 | `geometra_expand_section` | Expand one form/dialog/list/landmark from `geometra_page_model` on demand |
 | `geometra_click` | Click an element by coordinates |
 | `geometra_type` | Type text into the focused element |
 | `geometra_key` | Send special keys (Enter, Tab, Escape, arrows) |
-| `geometra_upload_files` | Attach files: auto / hidden input / native chooser / synthetic drop (`@geometra/proxy` only) |
+| `geometra_upload_files` | Attach files: labeled field / auto / hidden input / native chooser / synthetic drop (`@geometra/proxy` only) |
 | `geometra_pick_listbox_option` | Pick an option from a custom dropdown/searchable combobox; can open by field label (`@geometra/proxy` only) |
 | `geometra_select_option` | Choose an option on a native `<select>` (`@geometra/proxy` only) |
 | `geometra_set_checked` | Set a checkbox or radio by label instead of coordinate clicks (`@geometra/proxy` only) |
@@ -36,27 +39,155 @@ Geometra proxy:       Chromium → DOM geometry → same WebSocket as native →
 
 ## Setup
 
-### Claude Code
+<details>
+<summary>Claude Code</summary>
 
+**One-line install:**
 ```bash
-claude mcp add geometra -- npx @geometra/mcp
+claude mcp add geometra -- npx -y @geometra/mcp
 ```
 
-### Claude Desktop
+**Uninstall:**
+```bash
+claude mcp remove geometra
+```
+
+Or manually add to `.mcp.json` (project-level) or `~/.claude/settings.json` (global):
+```json
+{
+  "mcpServers": {
+    "geometra": {
+      "command": "npx",
+      "args": ["-y", "@geometra/mcp"]
+    }
+  }
+}
+```
+
+To uninstall manually, remove the `geometra` entry from the config file.
 
-Add to `claude_desktop_config.json`:
+</details>
+
+<details>
+<summary>Claude Desktop</summary>
+
+Add to your Claude Desktop MCP config:
 
 ```json
 {
   "mcpServers": {
     "geometra": {
       "command": "npx",
-      "args": ["@geometra/mcp"]
+      "args": ["-y", "@geometra/mcp"]
     }
   }
 }
 ```
 
+To uninstall, remove the `geometra` entry from the config file.
+
+</details>
+
+<details>
+<summary>OpenAI Codex</summary>
+
+Add to your Codex MCP configuration:
+
+```json
+{
+  "mcpServers": {
+    "geometra": {
+      "command": "npx",
+      "args": ["-y", "@geometra/mcp"]
+    }
+  }
+}
+```
+
+To uninstall, remove the `geometra` entry from the config file.
+
+</details>
+
+<details>
+<summary>Cursor</summary>
+
+Open Settings → MCP → Add new MCP server, or add to `.cursor/mcp.json`:
+
+```json
+{
+  "mcpServers": {
+    "geometra": {
+      "command": "npx",
+      "args": ["-y", "@geometra/mcp"]
+    }
+  }
+}
+```
+
+To uninstall, remove the entry from MCP settings.
+
+</details>
+
+<details>
+<summary>Windsurf</summary>
+
+Add to `~/.codeium/windsurf/mcp_config.json`:
+
+```json
+{
+  "mcpServers": {
+    "geometra": {
+      "command": "npx",
+      "args": ["-y", "@geometra/mcp"]
+    }
+  }
+}
+```
+
+To uninstall, remove the entry from the config file.
+
+</details>
+
+<details>
+<summary>VS Code / Copilot</summary>
+
+**One-line install:**
+```bash
+code --add-mcp '{"name":"geometra","command":"npx","args":["-y","@geometra/mcp"]}'
+```
+
+Or add to `.vscode/mcp.json`:
+```json
+{
+  "servers": {
+    "geometra": {
+      "command": "npx",
+      "args": ["-y", "@geometra/mcp"]
+    }
+  }
+}
+```
+
+To uninstall, remove the entry from MCP settings or delete the server from the MCP panel.
+
+</details>
+
+<details>
+<summary>Other MCP clients</summary>
+
+Any MCP client that supports stdio transport can use Geometra. The server config is:
+
+```json
+{
+  "command": "npx",
+  "args": ["-y", "@geometra/mcp"]
+}
+```
+
+To uninstall, remove the server entry from your client's MCP configuration.
+
+</details>
+
 ### From source (this repo)
 
 ```bash
@@ -173,4 +304,50 @@ Agent:  geometra_query({ role: "button", name: "Save" })
 8. Tools expose query, click, type, snapshot, page-model, and section-expansion operations over this structured data.
 9. After each interaction, the peer sends updated geometry (full `frame` or `patch`) — the MCP tools interpret that into compact summaries.
 
+## Long Forms
+
+For long application flows, prefer one of these patterns:
+
+1. `geometra_page_model`
+2. `geometra_expand_section`
+3. `geometra_fill_fields` for obvious field entry
+4. `geometra_run_actions` when you need mixed navigation + waits + field entry
+
+Typical batch:
+
+```json
+{
+  "actions": [
+    { "type": "click", "x": 412, "y": 228 },
+    { "type": "type", "text": "Taylor Applicant" },
+    { "type": "upload_files", "paths": ["/Users/you/resume.pdf"], "fieldLabel": "Resume" },
+    { "type": "wait_for", "text": "Parsing your resume", "present": false, "timeoutMs": 10000 }
+  ]
+}
+```
+
+Single action tools now default to terse summaries. Pass `detail: "verbose"` when you need a fuller current-UI fallback for debugging.
+
+For the smallest long-form responses, prefer:
+
+1. `detail: "minimal"` for structured step metadata instead of narrated deltas
+2. `includeSteps: false` when you only need aggregate success/error counts plus the final validation/state payload
+
+Typical field fill:
+
+```json
+{
+  "fields": [
+    { "kind": "text", "fieldLabel": "Full name", "value": "Taylor Applicant" },
+    { "kind": "text", "fieldLabel": "Email", "value": "taylor@example.com" },
+    { "kind": "choice", "fieldLabel": "Country", "value": "Germany" },
+    { "kind": "choice", "fieldLabel": "Will you require sponsorship?", "value": "No" },
+    { "kind": "file", "fieldLabel": "Resume", "paths": ["/Users/you/resume.pdf"] }
+  ],
+  "failOnInvalid": true,
+  "detail": "minimal",
+  "includeSteps": false
+}
+```
+
 With a **native** Geometra server, layout comes from Textura/Yoga. With **`@geometra/proxy`**, layout comes from the browser’s computed DOM geometry; the MCP layer is the same.

package/dist/__tests__/server-batch-results.test.d.ts

@@ -0,0 +1 @@
+export {};

package/dist/__tests__/server-batch-results.test.js

@@ -0,0 +1,206 @@
+import { beforeEach, describe, expect, it, vi } from 'vitest';
+function node(role, name, options) {
+    return {
+        role,
+        ...(name ? { name } : {}),
+        ...(options?.value ? { value: options.value } : {}),
+        ...(options?.state ? { state: options.state } : {}),
+        ...(options?.validation ? { validation: options.validation } : {}),
+        ...(options?.meta ? { meta: options.meta } : {}),
+        bounds: { x: 0, y: 0, width: 120, height: 40 },
+        path: options?.path ?? [],
+        children: options?.children ?? [],
+        focusable: role !== 'group',
+    };
+}
+const mockState = vi.hoisted(() => ({
+    currentA11yRoot: node('group', undefined, {
+        meta: { pageUrl: 'https://jobs.example.com/application', scrollX: 0, scrollY: 0 },
+    }),
+    session: {
+        tree: { kind: 'box' },
+        layout: { x: 0, y: 0, width: 1280, height: 800, children: [] },
+        url: 'ws://127.0.0.1:3200',
+        updateRevision: 1,
+    },
+    sendClick: vi.fn(async () => ({ status: 'updated', timeoutMs: 2000 })),
+    sendType: vi.fn(async () => ({ status: 'updated', timeoutMs: 2000 })),
+    sendKey: vi.fn(async () => ({ status: 'updated', timeoutMs: 2000 })),
+    sendFileUpload: vi.fn(async () => ({ status: 'updated', timeoutMs: 8000 })),
+    sendFieldText: vi.fn(async () => ({ status: 'updated', timeoutMs: 2000 })),
+    sendFieldChoice: vi.fn(async () => ({ status: 'updated', timeoutMs: 2000 })),
+    sendListboxPick: vi.fn(async () => ({ status: 'updated', timeoutMs: 4500 })),
+    sendSelectOption: vi.fn(async () => ({ status: 'updated', timeoutMs: 2000 })),
+    sendSetChecked: vi.fn(async () => ({ status: 'updated', timeoutMs: 2000 })),
+    sendWheel: vi.fn(async () => ({ status: 'updated', timeoutMs: 2000 })),
+    waitForUiCondition: vi.fn(async () => true),
+}));
+vi.mock('../session.js', () => ({
+    connect: vi.fn(),
+    connectThroughProxy: vi.fn(),
+    disconnect: vi.fn(),
+    getSession: vi.fn(() => mockState.session),
+    sendClick: mockState.sendClick,
+    sendType: mockState.sendType,
+    sendKey: mockState.sendKey,
+    sendFileUpload: mockState.sendFileUpload,
+    sendFieldText: mockState.sendFieldText,
+    sendFieldChoice: mockState.sendFieldChoice,
+    sendListboxPick: mockState.sendListboxPick,
+    sendSelectOption: mockState.sendSelectOption,
+    sendSetChecked: mockState.sendSetChecked,
+    sendWheel: mockState.sendWheel,
+    buildA11yTree: vi.fn(() => mockState.currentA11yRoot),
+    buildCompactUiIndex: vi.fn(() => ({ nodes: [], context: {} })),
+    buildPageModel: vi.fn(() => ({
+        viewport: { width: 1280, height: 800 },
+        archetypes: ['form'],
+        summary: { landmarkCount: 0, formCount: 1, dialogCount: 0, listCount: 0, focusableCount: 2 },
+        primaryActions: [],
+        landmarks: [],
+        forms: [],
+        dialogs: [],
+        lists: [],
+    })),
+    expandPageSection: vi.fn(() => null),
+    buildUiDelta: vi.fn(() => ({})),
+    hasUiDelta: vi.fn(() => false),
+    nodeIdForPath: vi.fn((path) => `n:${path.length > 0 ? path.join('.') : 'root'}`),
+    summarizeCompactIndex: vi.fn(() => ''),
+    summarizePageModel: vi.fn(() => ''),
+    summarizeUiDelta: vi.fn(() => ''),
+    waitForUiCondition: mockState.waitForUiCondition,
+}));
+const { createServer } = await import('../server.js');
+function getToolHandler(name) {
+    const server = createServer();
+    return server._registeredTools[name].handler;
+}
+describe('batch MCP result shaping', () => {
+    beforeEach(() => {
+        vi.clearAllMocks();
+        mockState.currentA11yRoot = node('group', undefined, {
+            meta: { pageUrl: 'https://jobs.example.com/application', scrollX: 0, scrollY: 420 },
+            children: [
+                node('textbox', 'Mission', {
+                    value: 'Ship calm developer tools across browsers and platforms.',
+                    path: [0],
+                }),
+                node('textbox', 'Email', {
+                    value: 'taylor@example.com',
+                    path: [1],
+                }),
+            ],
+        });
+    });
+    it('keeps fill_fields minimal output structured and does not echo long essay text', async () => {
+        const longAnswer = 'A'.repeat(180);
+        const handler = getToolHandler('geometra_fill_fields');
+        mockState.currentA11yRoot = node('group', undefined, {
+            meta: { pageUrl: 'https://jobs.example.com/application', scrollX: 0, scrollY: 420 },
+            children: [
+                node('textbox', 'Mission', { value: longAnswer, path: [0] }),
+                node('textbox', 'Email', { value: 'taylor@example.com', path: [1] }),
+            ],
+        });
+        const result = await handler({
+            fields: [
+                { kind: 'text', fieldLabel: 'Mission', value: longAnswer },
+                { kind: 'text', fieldLabel: 'Email', value: 'taylor@example.com' },
+            ],
+            stopOnError: true,
+            failOnInvalid: false,
+            includeSteps: true,
+            detail: 'minimal',
+        });
+        const text = result.content[0].text;
+        const payload = JSON.parse(text);
+        const steps = payload.steps;
+        expect(text).not.toContain(longAnswer);
+        expect(payload).toMatchObject({
+            completed: true,
+            fieldCount: 2,
+            successCount: 2,
+            errorCount: 0,
+        });
+        expect(steps[0]).toMatchObject({
+            index: 0,
+            kind: 'text',
+            ok: true,
+            fieldLabel: 'Mission',
+            valueLength: 180,
+            wait: 'updated',
+            readback: { role: 'textbox', valueLength: 180 },
+        });
+        expect(steps[1]).toMatchObject({
+            index: 1,
+            kind: 'text',
+            ok: true,
+            fieldLabel: 'Email',
+            value: 'taylor@example.com',
+            wait: 'updated',
+            readback: { role: 'textbox', value: 'taylor@example.com' },
+        });
+    });
+    it('lets run_actions omit step listings while keeping capped final validation state', async () => {
+        const handler = getToolHandler('geometra_run_actions');
+        mockState.currentA11yRoot = node('group', undefined, {
+            meta: { pageUrl: 'https://jobs.example.com/application', scrollX: 0, scrollY: 2400 },
+            children: [
+                node('textbox', 'Full name', {
+                    value: '',
+                    path: [0],
+                    state: { invalid: true, required: true },
+                    validation: { error: 'Enter your full name.' },
+                }),
+                node('textbox', 'Email', {
+                    value: '',
+                    path: [1],
+                    state: { invalid: true, required: true },
+                    validation: { error: 'Enter your email.' },
+                }),
+                node('textbox', 'Phone', {
+                    value: '',
+                    path: [2],
+                    state: { invalid: true, required: true },
+                    validation: { error: 'Enter your phone number.' },
+                }),
+                node('textbox', 'Location', {
+                    value: '',
+                    path: [3],
+                    state: { invalid: true, required: true },
+                    validation: { error: 'Choose a location.' },
+                }),
+                node('textbox', 'LinkedIn', {
+                    value: '',
+                    path: [4],
+                    state: { invalid: true },
+                    validation: { error: 'Enter a valid URL.' },
+                }),
+                node('alert', 'Your form needs corrections', { path: [5] }),
+            ],
+        });
+        const result = await handler({
+            actions: [{ type: 'click', x: 320, y: 540 }],
+            stopOnError: true,
+            includeSteps: false,
+            detail: 'minimal',
+        });
+        const payload = JSON.parse(result.content[0].text);
+        const final = payload.final;
+        expect(payload).toMatchObject({
+            completed: true,
+            stepCount: 1,
+            successCount: 1,
+            errorCount: 0,
+        });
+        expect(payload).not.toHaveProperty('steps');
+        expect(final).toMatchObject({
+            pageUrl: 'https://jobs.example.com/application',
+            alertCount: 1,
+            invalidCount: 5,
+        });
+        expect(final.invalidFields.length).toBe(4);
+        expect(final.alerts.length).toBe(1);
+    });
+});

package/dist/__tests__/server-filters.test.js

@@ -6,6 +6,7 @@ function node(role, bounds, options) {
         ...(options?.name ? { name: options.name } : {}),
         ...(options?.value ? { value: options.value } : {}),
         ...(options?.state ? { state: options.state } : {}),
+        ...(options?.validation ? { validation: options.validation } : {}),
         bounds,
         path: options?.path ?? [],
         children: options?.children ?? [],
@@ -22,8 +23,15 @@ describe('findNodes', () => {
                     value: 'Austin, Texas, United States',
                     focusable: true,
                 }),
-                node('checkbox', { x: 20, y: 72, width: 24, height: 24 }, {
+                node('textbox', { x: 20, y: 120, width: 260, height: 36 }, {
                     path: [1],
+                    name: 'Email',
+                    focusable: true,
+                    state: { invalid: true, required: true, busy: true },
+                    validation: { error: 'Please enter a valid email address.' },
+                }),
+                node('checkbox', { x: 20, y: 72, width: 24, height: 24 }, {
+                    path: [2],
                     name: 'Notion Website',
                     state: { checked: true },
                     focusable: true,
@@ -36,6 +44,12 @@ describe('findNodes', () => {
         expect(findNodes(tree, { text: 'United States' })).toEqual([
             expect.objectContaining({ role: 'combobox', value: 'Austin, Texas, United States' }),
         ]);
+        expect(findNodes(tree, { invalid: true, required: true, busy: true })).toEqual([
+            expect.objectContaining({ role: 'textbox', name: 'Email' }),
+        ]);
+        expect(findNodes(tree, { text: 'valid email address' })).toEqual([
+            expect.objectContaining({ role: 'textbox', name: 'Email' }),
+        ]);
         expect(findNodes(tree, { role: 'checkbox', checked: true })).toEqual([
             expect.objectContaining({ name: 'Notion Website' }),
         ]);

package/dist/__tests__/session-model.test.js

@@ -1,11 +1,12 @@
 import { describe, expect, it } from 'vitest';
-import { buildCompactUiIndex, buildPageModel, expandPageSection, buildUiDelta, hasUiDelta, summarizeUiDelta, } from '../session.js';
+import { buildA11yTree, buildCompactUiIndex, buildPageModel, expandPageSection, buildUiDelta, hasUiDelta, summarizeUiDelta, } from '../session.js';
 function node(role, name, bounds, options) {
     return {
         role,
         ...(name ? { name } : {}),
         ...(options?.value ? { value: options.value } : {}),
         ...(options?.state ? { state: options.state } : {}),
+        ...(options?.validation ? { validation: options.validation } : {}),
         ...(options?.meta ? { meta: options.meta } : {}),
         bounds,
         path: options?.path ?? [],
@@ -87,10 +88,13 @@ describe('buildPageModel', () => {
                                 node('textbox', 'Full name*', { x: 60, y: 160, width: 300, height: 36 }, {
                                     path: [0, 0, 1],
                                     value: 'Taylor Applicant',
+                                    state: { required: true },
                                 }),
                                 node('textbox', 'Email:', { x: 60, y: 208, width: 300, height: 36 }, {
                                     path: [0, 0, 2],
                                     value: 'taylor@example.com',
+                                    state: { invalid: true, required: true },
+                                    validation: { error: 'Please enter a valid email address.' },
                                 }),
                                 node('button', 'Submit application', { x: 60, y: 264, width: 180, height: 40 }, {
                                     path: [0, 0, 3],
@@ -116,6 +120,9 @@ describe('buildPageModel', () => {
         });
         expect(detail?.fields.map(field => field.name)).toEqual(['Full name', 'Email']);
         expect(detail?.fields.map(field => field.value)).toEqual(['Taylor Applicant', 'taylor@example.com']);
+        expect(detail?.fields[0]?.state).toEqual({ required: true });
+        expect(detail?.fields[1]?.state).toEqual({ invalid: true, required: true });
+        expect(detail?.fields[1]?.validation).toEqual({ error: 'Please enter a valid email address.' });
         expect(detail?.actions.map(action => action.id)).toEqual(['n:0.0.3']);
         expect(detail?.fields[0]).not.toHaveProperty('bounds');
     });
@@ -324,3 +331,53 @@ describe('buildUiDelta', () => {
         expect(summarizeUiDelta(delta)).toContain('value "Austin" -> "Austin, Texas, United States"');
     });
 });
+describe('buildA11yTree', () => {
+    it('maps required, invalid, busy, and validation text from raw semantic nodes', () => {
+        const tree = {
+            kind: 'box',
+            props: {},
+            semantic: {},
+            children: [
+                {
+                    kind: 'box',
+                    props: { value: '' },
+                    semantic: {
+                        role: 'textbox',
+                        ariaLabel: 'Email',
+                        ariaRequired: true,
+                        ariaInvalid: true,
+                        ariaBusy: true,
+                        validationDescription: 'We will contact you about this role.',
+                        validationError: 'Please enter a valid email address.',
+                    },
+                    handlers: { onClick: true, onKeyDown: true },
+                },
+            ],
+        };
+        const layout = {
+            x: 0,
+            y: 0,
+            width: 800,
+            height: 600,
+            children: [
+                {
+                    x: 24,
+                    y: 40,
+                    width: 320,
+                    height: 36,
+                    children: [],
+                },
+            ],
+        };
+        const a11y = buildA11yTree(tree, layout);
+        expect(a11y.children[0]).toMatchObject({
+            role: 'textbox',
+            name: 'Email',
+            state: { required: true, invalid: true, busy: true },
+            validation: {
+                description: 'We will contact you about this role.',
+                error: 'Please enter a valid email address.',
+            },
+        });
+    });
+});

package/dist/server.d.ts

@@ -12,6 +12,9 @@ interface NodeFilter {
     focused?: boolean;
     selected?: boolean;
     expanded?: boolean;
+    invalid?: boolean;
+    required?: boolean;
+    busy?: boolean;
 }
 export declare function createServer(): McpServer;
 export declare function findNodes(node: A11yNode, filter: NodeFilter): A11yNode[];