@geometra/mcp 1.19.20 → 1.19.23

package/README.md

@@ -24,25 +24,28 @@ Proxy-backed sessions stay warm by default on disconnect, and MCP now keeps a sm
 
 | Tool | Description |
 |---|---|
-| `geometra_connect` | Connect with `url` (ws://…) **or** `pageUrl` (https://…) to auto-start geometra-proxy; can inline `formSchema` and/or `pageModel` for lower-turn starts |
-| `geometra_query` | Find elements by stable id, role, name, text content, ancestor/prompt context, current value, or semantic state such as `invalid`, `required`, or `busy` |
+| `geometra_connect` | Connect with `url` (ws://…) **or** `pageUrl` (https://…) to auto-start geometra-proxy; can inline `formSchema` and/or `pageModel`, or defer the page model for a faster first connect response |
+| `geometra_prepare_browser` | Pre-launch and pre-navigate a reusable proxy/browser for `pageUrl` without creating an active session; best when the agent can prepare before the user-facing task starts |
+| `geometra_query` | Find elements by stable id, role, name, text content, prompt/section/item context, current value, or semantic state such as `invalid`, `required`, or `busy` |
 | `geometra_wait_for` | Wait for a semantic condition instead of guessing sleeps (`busy`, `disabled`, alerts, values, etc.). **Strict parameters** — use `text` plus `present: false` to wait until a substring disappears (e.g. “Parsing your resume”); there is no `textGone` field |
+| `geometra_wait_for_resume_parse` | Convenience wait until a parsing banner is gone (defaults: `text`: `"Parsing"`, `present` implied false). Same engine as `geometra_wait_for` |
 | `geometra_form_schema` | Compact, fill-oriented form schema with stable field ids and collapsed radio/button groups; can auto-connect from `pageUrl` / `url` |
 | `geometra_fill_form` | Fill a form from `valuesById` / `valuesByLabel` in one MCP call; can auto-connect from `pageUrl` / `url` for the lowest-token known-form path |
-| `geometra_fill_fields` | Fill labeled text/choice/toggle/file fields in one MCP call; can return final-only status for the smallest responses |
-| `geometra_run_actions` | Execute a batch of high-level actions in one MCP round trip and get one consolidated result, with optional final-only output |
-| `geometra_page_model` | Summary-first webpage model: archetypes, stable section ids, counts, top-level sections, primary actions |
+| `geometra_fill_fields` | Fill text/choice/toggle/file fields in one MCP call; text-only batches now use the proxy fast path when step output is omitted, and text/choice/toggle entries can use `fieldId` from `geometra_form_schema` without repeating the label |
+| `geometra_run_actions` | Execute a batch of high-level actions in one MCP round trip; can auto-connect from `pageUrl` / `url` and return a final-only payload for the smallest multi-step responses |
+| `geometra_page_model` | Summary-first webpage model: archetypes, stable section ids, counts, top-level sections, and primary actions with nearby section/item context when available |
+| `geometra_find_action` | Resolve a repeated button/link by action label plus optional `sectionText`, `promptText`, or `itemText` before clicking |
 | `geometra_expand_section` | Expand one form/dialog/list/landmark from `geometra_page_model` on demand, with paging/filtering for long sections |
-| `geometra_reveal` | Scroll until a matching node is visible instead of guessing wheel deltas |
-| `geometra_click` | Click by coordinates or semantic target, optionally waiting for a post-click semantic condition |
+| `geometra_reveal` | Scroll until a matching node is visible instead of guessing wheel deltas; auto-scales reveal steps for tall forms when omitted |
+| `geometra_click` | Click by coordinates or semantic target, including repeated-action disambiguation with `promptText`, `sectionText`, or `itemText`, optionally waiting for a post-click semantic condition |
 | `geometra_type` | Type text into the focused element |
 | `geometra_key` | Send special keys (Enter, Tab, Escape, arrows) |
 | `geometra_upload_files` | Attach files: labeled field / auto / hidden input / native chooser / synthetic drop (`@geometra/proxy` only) |
-| `geometra_pick_listbox_option` | Pick an option from a custom dropdown/searchable combobox; can open by field label (`@geometra/proxy` only) |
-| `geometra_select_option` | Choose an option on a native `<select>` (`@geometra/proxy` only) |
+| `geometra_pick_listbox_option` | Pick an option from a custom dropdown/searchable combobox; can open by field label, falls back to keyboard selection, and returns visible option hints on failure (`@geometra/proxy` only) |
+| `geometra_select_option` | Choose an option on a native `<select>` only; use `geometra_pick_listbox_option` for custom dropdowns (`@geometra/proxy` only) |
 | `geometra_set_checked` | Set a checkbox or radio by label instead of coordinate clicks (`@geometra/proxy` only) |
 | `geometra_wheel` | Mouse wheel / scroll (`@geometra/proxy` only) |
-| `geometra_snapshot` | Default **compact**: flat viewport-visible actionable nodes (minified JSON). `view=full` for nested tree |
+| `geometra_snapshot` | Default **compact**: flat viewport-visible actionable nodes (minified JSON). `view=full` for nested tree, `view=form-required` for required fields including offscreen ones |
 | `geometra_layout` | Raw computed geometry for every node |
 | `geometra_disconnect` | Close the connection |
 
@@ -50,6 +53,20 @@ Proxy-backed sessions stay warm by default on disconnect, and MCP now keeps a sm
 
 The tool input is **strict**: unknown keys are rejected (so mistaken names like `textGone` fail fast instead of being ignored). To wait until copy such as “Parsing…” or “Parsing your resume” is **gone**, pass a substring that matches the banner in **`text`** and set **`present`** to **`false`**. Example: `{ "text": "Parsing", "present": false }` (adjust the substring per site).
 
+For the common resume-upload case, prefer **`geometra_wait_for_resume_parse`** (optional `text`, default `"Parsing"`; optional `timeoutMs`, default `60000`) so agents do not have to remember `present: false`.
+
+### Custom dropdowns and searchable comboboxes
+
+Use **`geometra_pick_listbox_option`** for React Select / Radix / Headless UI / custom ATS dropdowns. Prefer **`fieldLabel`** so MCP opens the right combobox semantically, and pass **`query`** when the dropdown is searchable. If click-selection cannot be confirmed, the proxy now falls back to keyboard navigation for editable comboboxes.
+
+When selection fails, the error payload includes a capped **`visibleOptions`** list so an agent can retry with a real label instead of blindly guessing. Use **`geometra_select_option`** only for native HTML `<select>` controls.
+
+### Long forms and offscreen required fields
+
+For tall application forms, **`geometra_snapshot({ view: "form-required" })`** returns required fields even when they are offscreen, with **bounds**, **visibility**, and **scrollHint** data. This is useful for “what required fields are left?” without paging through a full tree.
+
+When a target is below the fold, **`geometra_reveal`** and semantic click actions auto-scale reveal steps when `maxSteps` / `maxRevealSteps` are omitted, so long forms do not fail after a hard-coded short scroll budget. If you already have schema ids, **`geometra_fill_fields`** can now use `fieldId` directly for text / choice / toggle entries without repeating `fieldLabel` / `label`.
+
 ## Setup
 
 <details>
@@ -304,13 +321,15 @@ Agent:  geometra_query({ role: "button", name: "Save" })
 1. The MCP server connects to a WebSocket peer that speaks GEOM v1 (`frame` with `layout` + `tree`, optional `patch` updates).
 2. It receives the computed layout (`{ x, y, width, height }` for every node) and the UI tree (`kind`, `semantic`, `props`, `handlers`, `children`).
 3. It builds an accessibility tree from that data — roles, names, focusable state, bounds.
-4. **`geometra_snapshot`** defaults to a **compact** flat list of viewport-visible actionable nodes (minified JSON) to reduce LLM tokens; use `view: "full"` for the complete nested tree.
+4. **`geometra_snapshot`** defaults to a **compact** flat list of viewport-visible actionable nodes (minified JSON) to reduce LLM tokens; use `view: "full"` for the complete nested tree or `view: "form-required"` for offscreen required fields plus scroll hints.
 5. **`geometra_form_schema`** is the compact form-specific path: stable field ids, required/invalid state, current values, and collapsed choice groups without layout-heavy section detail. It can auto-connect when you pass `pageUrl` / `url`.
 6. **`geometra_fill_form`** turns a compact values object into semantic field operations server-side, so the model does not need to emit one tool call per field. It can also auto-connect, which collapses “open page + fill known values” into a single tool call.
-7. **`geometra_page_model`** is still the right summary-first path for non-form exploration: page archetypes, stable section ids, counts, top-level landmarks/forms/dialogs/lists, and a few primary actions.
-8. **`geometra_expand_section`** fetches richer details only for the section you care about (fields, actions, headings, nested lists, list items, text preview).
-9. After interactions, action tools return a **semantic delta** when possible (dialogs opened/closed, forms appeared/removed, list counts changed, named/focusable nodes added/removed/updated). If nothing meaningful changed, they fall back to a short current-UI overview.
-10. After each interaction, the peer sends updated geometry (full `frame` or `patch`) — the MCP tools interpret that into compact summaries.
+7. **`geometra_fill_fields`** is the lower-level field-intent path when you need direct control; text / choice / toggle entries can now resolve `fieldId` from `geometra_form_schema` without repeating labels.
+8. **`geometra_page_model`** is still the right summary-first path for non-form exploration: page archetypes, stable section ids, counts, top-level landmarks/forms/dialogs/lists, and a few primary actions with nearby context.
+9. **`geometra_find_action`** is the narrow repeated-action resolver when the page has many identical buttons/links and you want one exact target by `itemText`, `sectionText`, or `promptText`.
+10. **`geometra_expand_section`** fetches richer details only for the section you care about (fields, actions, headings, nested lists, list items, text preview).
+11. After interactions, action tools return a **semantic delta** when possible (dialogs opened/closed, forms appeared/removed, list counts changed, named/focusable nodes added/removed/updated). If nothing meaningful changed, they fall back to a short current-UI overview.
+12. After each interaction, the peer sends updated geometry (full `frame` or `patch`) — the MCP tools interpret that into compact summaries.
 
 ## Long Forms
 
@@ -319,11 +338,15 @@ For long application flows, prefer one of these patterns:
 1. `geometra_fill_form({ pageUrl, valuesByLabel })` when you already know the fields you want to set
 2. otherwise `geometra_form_schema`
 3. then `geometra_fill_form`
-3. `geometra_reveal` for far-below-fold targets such as submit buttons
-4. `geometra_click({ ..., waitFor: ... })` when one action should also wait for the next semantic state
-5. `geometra_run_actions` when you need mixed navigation + waits + field entry
-6. `geometra_connect({ pageUrl, returnPageModel: true })` when you want connect + summary-first exploration in one turn
-7. `geometra_page_model` + `geometra_expand_section` when you are still exploring the page rather than filling it
+4. `geometra_snapshot({ view: "form-required" })` when you need the remaining required fields including offscreen ones
+5. `geometra_reveal` for far-below-fold targets such as submit buttons (auto-scales reveal steps when you omit `maxSteps`)
+6. `geometra_click({ ..., waitFor: ... })` when one action should also wait for the next semantic state
+7. `geometra_prepare_browser({ pageUrl, headless, width, height })` when you can hide browser startup before the real task and want the next proxy-backed flow to attach warm
+8. `geometra_run_actions` when you need mixed navigation + waits + field entry, especially with `pageUrl` / `url` for a one-call flow
+9. `geometra_connect({ pageUrl, returnPageModel: true })` when you want connect + summary-first exploration in one turn
+10. `geometra_connect({ pageUrl, returnPageModel: true, pageModelMode: "deferred" })` when first-response latency matters more than inlining the page model; follow with `geometra_page_model`
+11. `geometra_find_action` or `geometra_click({ name, itemText / sectionText / promptText, ... })` when the target action repeats across cards/rows
+12. `geometra_page_model` + `geometra_expand_section` when you are still exploring the page rather than filling it
 
 Typical batch:
 
@@ -338,12 +361,14 @@ Typical batch:
 }
 ```
 
-Single action tools now default to terse summaries. Pass `detail: "verbose"` when you need a fuller current-UI fallback for debugging.
+Single action tools default to short human-readable summaries (`detail: "minimal"`). Use `detail: "terse"` for the smallest machine-friendly JSON responses, or `detail: "verbose"` when you need a fuller current-UI fallback for debugging.
 
 For the smallest long-form responses, prefer:
 
-1. `detail: "minimal"` for structured step metadata instead of narrated deltas
+1. `detail: "terse"` for compact machine-friendly action responses
 2. `includeSteps: false` when you only need aggregate success/error counts plus the final validation/state payload
+3. `output: "final"` on `geometra_run_actions` when you only need completion state plus the final semantic signals
+4. `geometra_prepare_browser` before the flow when you can shift browser launch out of the user-visible task window
 
 Typical low-token form fill:
 

package/dist/__tests__/proxy-session-recovery.test.js

@@ -8,7 +8,7 @@ vi.mock('../proxy-spawn.js', () => ({
     startEmbeddedGeometraProxy: mockState.startEmbeddedGeometraProxy,
     spawnGeometraProxy: mockState.spawnGeometraProxy,
 }));
-const { connectThroughProxy, disconnect } = await import('../session.js');
+const { connectThroughProxy, disconnect, prewarmProxy } = await import('../session.js');
 function frame(pageUrl) {
     return {
         type: 'frame',
@@ -28,12 +28,17 @@ function frame(pageUrl) {
 async function createProxyPeer(options) {
     const wss = new WebSocketServer({ port: 0 });
     wss.on('connection', ws => {
-        ws.send(JSON.stringify(frame(options?.pageUrl ?? 'https://jobs.example.com/original')));
+        if (options?.sendInitialFrame !== false) {
+            ws.send(JSON.stringify(frame(options?.pageUrl ?? 'https://jobs.example.com/original')));
+        }
         ws.on('message', raw => {
             const msg = JSON.parse(String(raw));
             if (msg.type === 'navigate') {
                 options?.onNavigate?.(ws, msg);
             }
+            else if (msg.type === 'resize') {
+                options?.onResize?.(ws, msg);
+            }
         });
     });
     const port = await new Promise((resolve, reject) => {
@@ -78,6 +83,7 @@ describe('connectThroughProxy recovery', () => {
         });
         const staleRuntime = {
             wsUrl: stalePeer.wsUrl,
+            ready: Promise.resolve(),
             closed: false,
             close: vi.fn(async () => {
                 staleRuntime.closed = true;
@@ -85,6 +91,7 @@ describe('connectThroughProxy recovery', () => {
         };
         const freshRuntime = {
             wsUrl: freshPeer.wsUrl,
+            ready: Promise.resolve(),
             closed: false,
             close: vi.fn(async () => {
                 freshRuntime.closed = true;
@@ -124,6 +131,7 @@ describe('connectThroughProxy recovery', () => {
         });
         const headedRuntime = {
             wsUrl: headedPeer.wsUrl,
+            ready: Promise.resolve(),
             closed: false,
             close: vi.fn(async () => {
                 headedRuntime.closed = true;
@@ -131,6 +139,7 @@ describe('connectThroughProxy recovery', () => {
         };
         const headlessRuntime = {
             wsUrl: headlessPeer.wsUrl,
+            ready: Promise.resolve(),
             closed: false,
             close: vi.fn(async () => {
                 headlessRuntime.closed = true;
@@ -170,4 +179,84 @@ describe('connectThroughProxy recovery', () => {
             await closePeer(headlessPeer.wss);
         }
     });
+    it('can prewarm a reusable proxy before the first measured task', async () => {
+        const preparedPeer = await createProxyPeer({
+            pageUrl: 'https://jobs.example.com/prepared',
+        });
+        const preparedRuntime = {
+            wsUrl: preparedPeer.wsUrl,
+            ready: Promise.resolve(),
+            closed: false,
+            close: vi.fn(async () => {
+                preparedRuntime.closed = true;
+            }),
+        };
+        mockState.startEmbeddedGeometraProxy.mockResolvedValue({
+            runtime: preparedRuntime,
+            wsUrl: preparedPeer.wsUrl,
+        });
+        mockState.spawnGeometraProxy.mockRejectedValue(new Error('spawn fallback should not be used'));
+        try {
+            const prepared = await prewarmProxy({
+                pageUrl: 'https://jobs.example.com/prepared',
+                headless: true,
+            });
+            expect(prepared).toMatchObject({
+                prepared: true,
+                reused: false,
+                transport: 'embedded',
+                pageUrl: 'https://jobs.example.com/prepared',
+            });
+            const session = await connectThroughProxy({
+                pageUrl: 'https://jobs.example.com/prepared',
+                headless: true,
+            });
+            expect(session.proxyRuntime).toBe(preparedRuntime);
+            expect(mockState.startEmbeddedGeometraProxy).toHaveBeenCalledTimes(1);
+            expect(mockState.spawnGeometraProxy).not.toHaveBeenCalled();
+        }
+        finally {
+            disconnect({ closeProxy: true });
+            expect(preparedRuntime.close).toHaveBeenCalledTimes(1);
+            await closePeer(preparedPeer.wss);
+        }
+    });
+    it('starts without an eager initial extract when the caller defers the first frame', async () => {
+        const lazyPeer = await createProxyPeer({
+            pageUrl: 'https://jobs.example.com/lazy',
+            sendInitialFrame: false,
+        });
+        const lazyRuntime = {
+            wsUrl: lazyPeer.wsUrl,
+            ready: Promise.resolve(),
+            closed: false,
+            close: vi.fn(async () => {
+                lazyRuntime.closed = true;
+            }),
+        };
+        mockState.startEmbeddedGeometraProxy.mockResolvedValueOnce({
+            runtime: lazyRuntime,
+            wsUrl: lazyPeer.wsUrl,
+        });
+        mockState.spawnGeometraProxy.mockRejectedValue(new Error('spawn fallback should not be used'));
+        try {
+            const session = await connectThroughProxy({
+                pageUrl: 'https://jobs.example.com/lazy',
+                headless: true,
+                awaitInitialFrame: false,
+            });
+            expect(session.proxyRuntime).toBe(lazyRuntime);
+            expect(mockState.startEmbeddedGeometraProxy).toHaveBeenCalledWith(expect.objectContaining({
+                pageUrl: 'https://jobs.example.com/lazy',
+                headless: true,
+                eagerInitialExtract: false,
+            }));
+            expect(session.tree).toBeNull();
+        }
+        finally {
+            disconnect({ closeProxy: true });
+            expect(lazyRuntime.close).toHaveBeenCalledTimes(1);
+            await closePeer(lazyPeer.wss);
+        }
+    });
 });