surfagent 1.0.9 → 1.1.1

package/API.md

@@ -447,6 +447,27 @@ Check if the API can connect to Chrome.
 
 ---
 
+### POST /type
+
+Raw CDP key typing without clearing the field first. Use this for apps like **Google Sheets**, contenteditable elements, or any context where `/fill`'s Ctrl+A clear step causes side effects (e.g., selecting all cells instead of clearing a field).
+
+**Request:**
+```json
+{ "tab": "0", "keys": "Hello World", "submit": "tab" }
+```
+
+- `keys` (string) — characters to type via CDP `Input.dispatchKeyEvent`
+- `submit` (optional) — `"enter"` or `"tab"` to press after typing
+
+**Response:**
+```json
+{ "typed": 11, "submitted": true }
+```
+
+**Why not `/fill`?** The `/fill` endpoint focuses an element, does Ctrl+A + Backspace to clear it, then types. In Google Sheets, Ctrl+A selects all cells (not text in the current cell), wiping the entire sheet. `/type` skips the focus and clear — it types into whatever currently has focus.
+
+---
+
 ## Tab Targeting
 
 All POST endpoints accept a `tab` field. It resolves in this order:
@@ -548,6 +569,193 @@ POST /read  { "tab": "0", "selector": "#main-content" }
    → Click inside the iframe
 ```
 
+### Google Sheets workflow
+
+Google Sheets requires a special approach because `/fill` uses Ctrl+A to clear fields, which selects all cells in Sheets. Use the **name box + `/type`** pattern instead.
+
+**Navigate to a cell:** Use `/fill` on the name box (`#t-name-box`), then Enter to jump to the cell.
+
+**Type into a cell:** Use `/type` with `"submit": "tab"` (moves to next cell) or `"submit": "enter"` (moves down).
+
+```
+1. POST /click    { "tab": "sheets", "selector": "#t-name-box" }
+   → Focus the name box
+2. POST /fill     { "tab": "sheets", "fields": [{ "selector": "#t-name-box", "value": "A1", "clear": true }], "submit": "enter" }
+   → Navigate to cell A1
+3. POST /type     { "tab": "sheets", "keys": "Hello World", "submit": "tab" }
+   → Type into A1, Tab moves to B1
+4. POST /type     { "tab": "sheets", "keys": "=SUM(A1:A10)", "submit": "enter" }
+   → Type a formula, Enter commits and moves down
+```
+
+**Adding a new sheet tab:** The "+" button at the bottom does not respond to DOM `.click()`. Use CDP `Input.dispatchMouseEvent` at the button's coordinates:
+
+```bash
+# Get the Add Sheet button position
+curl -s -X POST localhost:3456/eval -d '{"tab":"0","expression":"var els = document.querySelectorAll(\"div[data-tooltip]\"); var r = \"\"; for(var i=0;i<els.length;i++){if(els[i].dataset.tooltip===\"Add Sheet\"){var b=els[i].getBoundingClientRect(); r=b.x+\",\"+b.y+\",\"+b.width+\",\"+b.height}} r"}'
+# Returns: "44,854,34,34"
+
+# Click it with a Node script using CDP mouse events
+node -e "
+const CDP = require('chrome-remote-interface');
+(async () => {
+  const targets = await CDP.List({port: 9222});
+  const tab = targets.find(t => t.url.includes('docs.google.com'));
+  const client = await CDP({target: tab, port: 9222});
+  await client.Input.dispatchMouseEvent({type:'mousePressed', x:61, y:871, button:'left', clickCount:1});
+  await client.Input.dispatchMouseEvent({type:'mouseReleased', x:61, y:871, button:'left', clickCount:1});
+  await client.close();
+})();
+"
+```
+
+**Renaming a sheet tab:** Double-click the tab name via CDP mouse events at the tab's coordinates, then use `/type` to enter the new name and press Enter.
+
+**Using the menu search:** Google Sheets has a menu search box (`input[aria-label="Menus"]` or `input[aria-label="Menus (Option+/)"]`). Use `/fill` to type a command (e.g., "Insert chart"), then `/click` on the matching result.
+
+**Key gotchas:**
+- Never use `/fill` directly on Google Sheets cells — it will wipe data via Ctrl+A
+- Always navigate to a cell via the name box first, then `/type`
+- Some buttons (Add Sheet, menu items) only respond to CDP mouse events, not DOM clicks
+- Navigating away from unsaved Sheets triggers a native Chrome dialog — see the "Native Chrome Dialogs" section below
+
+### CDP mouse clicks for unreachable elements
+
+Some UI elements don't respond to JavaScript `.click()` or the `/click` endpoint — they only react to real mouse events at their coordinates. This is common for:
+- Google Sheets buttons (Add Sheet, toolbar items)
+- Canvas-rendered elements
+- Custom widgets that listen for `mousedown`/`mouseup` events
+
+**Pattern:**
+```bash
+# 1. Get the element's coordinates via /eval
+curl -s -X POST localhost:3456/eval -d '{"tab":"0","expression":"document.querySelector(\"#my-button\").getBoundingClientRect().x"}'
+# → {"result": 100}
+
+# 2. Click via CDP Input.dispatchMouseEvent (requires a Node script)
+node -e "
+const CDP = require('chrome-remote-interface');
+(async () => {
+  const targets = await CDP.List({port: 9222});
+  const tab = targets.find(t => t.url.includes('your-site'));
+  const client = await CDP({target: tab, port: 9222});
+  await client.Input.dispatchMouseEvent({type:'mousePressed', x:100, y:200, button:'left', clickCount:1});
+  await client.Input.dispatchMouseEvent({type:'mouseReleased', x:100, y:200, button:'left', clickCount:1});
+  await client.close();
+})();
+"
+```
+
+**Double-click** (e.g., to rename a Google Sheets tab): use `clickCount: 2`.
+
+---
+
+## Native Chrome Dialogs (Not in DOM or CDP)
+
+Chrome can show browser-level popups — like "Leave page?" (`beforeunload`) dialogs — that are **not in the DOM**, **not accessible via CDP**, and will **block all CDP commands** (`/eval`, `/recon`, `/read` will all hang or timeout).
+
+**Symptoms of a stuck session:**
+- API calls hang or timeout on a tab that was previously working
+- `Page.handleJavaScriptDialog` returns "No dialog is showing" (because it's not a JS dialog — it's a native Chrome window)
+- The agent appears frozen on a page
+
+**How to detect it (macOS only):**
+
+Use CoreGraphics to list windows belonging to the surfagent Chrome process. Native dialogs appear as small unnamed windows (~260x218px) that are not visible to CDP.
+
+```bash
+# 1. Find the surfagent Chrome PID
+SURFAGENT_PID=$(ps aux | grep 'chrome.*surfagent' | grep -v grep | awk '{print $2}')
+
+# 2. List all windows for that PID using CoreGraphics
+swift -e "
+import CoreGraphics
+let windows = CGWindowListCopyWindowInfo(.optionAll, kCGNullWindowID) as! [[String: Any]]
+for w in windows {
+    let pid = w[\"kCGWindowOwnerPID\"] as? Int ?? 0
+    if pid == ${SURFAGENT_PID} {
+        let name = w[\"kCGWindowName\"] as? String ?? \"(unnamed)\"
+        let bounds = w[\"kCGWindowBounds\"] as? [String: Any] ?? [:]
+        let width = bounds[\"Width\"] as? Int ?? 0
+        let height = bounds[\"Height\"] as? Int ?? 0
+        if width > 100 && height > 100 {
+            print(\"Window: \(name) | Size: \(width)x\(height)\")
+        }
+    }
+}
+"
+```
+
+**What to look for:** A small unnamed window (typically ~260x218) alongside the main browser window. That's the native dialog.
+
+**How to dismiss it:**
+
+Native Chrome dialogs cannot be dismissed via CDP or AppleScript's `tell process "Google Chrome"` (which only sees the personal Chrome, not the surfagent debug instance). You must use the **Swift Accessibility API targeting the surfagent PID directly**.
+
+```bash
+# Find the surfagent Chrome PID
+SURFAGENT_PID=$(ps aux | grep 'chrome.*surfagent' | grep -v grep | awk '{print $2}')
+
+# Click "Cancel" (stay on page) — or change "Avbryt"/"Cancel" to "Leave"/"Gå ut" to leave
+swift -e "
+import Cocoa
+
+let pid: pid_t = ${SURFAGENT_PID}
+let app = AXUIElementCreateApplication(pid)
+
+var windowsRef: CFTypeRef?
+AXUIElementCopyAttributeValue(app, \"AXWindows\" as CFString, &windowsRef)
+
+if let windows = windowsRef as? [AXUIElement] {
+    for win in windows {
+        var subroleRef: CFTypeRef?
+        AXUIElementCopyAttributeValue(win, \"AXSubrole\" as CFString, &subroleRef)
+        let subrole = subroleRef as? String ?? \"\"
+        
+        // Native dialogs have subrole AXDialog
+        if subrole == \"AXDialog\" {
+            var childrenRef: CFTypeRef?
+            AXUIElementCopyAttributeValue(win, \"AXChildren\" as CFString, &childrenRef)
+            if let children = childrenRef as? [AXUIElement] {
+                for child in children {
+                    var roleRef: CFTypeRef?
+                    AXUIElementCopyAttributeValue(child, \"AXRole\" as CFString, &roleRef)
+                    var titleRef: CFTypeRef?
+                    AXUIElementCopyAttributeValue(child, \"AXTitle\" as CFString, &titleRef)
+                    let role = roleRef as? String ?? \"\"
+                    let title = titleRef as? String ?? \"\"
+                    
+                    // Match button by title — handles multiple languages
+                    // Cancel/Stay: \"Cancel\", \"Avbryt\" (Norwegian)
+                    // Leave: \"Leave\", \"Gå ut\" (Norwegian)
+                    let cancelNames = [\"Cancel\", \"Avbryt\"]
+                    let leaveNames = [\"Leave\", \"Gå ut\"]
+                    
+                    let targetNames = cancelNames  // Change to leaveNames to leave
+                    
+                    if role == \"AXButton\" && targetNames.contains(title) {
+                        let result = AXUIElementPerformAction(child, \"AXPress\" as CFString)
+                        print(\"Clicked \(title): \(result == .success ? \"SUCCESS\" : \"FAILED\")\")
+                    }
+                }
+            }
+        }
+    }
+}
+"
+```
+
+**Why AppleScript doesn't work:** `tell process "Google Chrome"` sees all Chrome instances as one process, but only exposes the *personal* Chrome's windows. The surfagent Chrome (launched with `--user-data-dir=/tmp/surfagent-chrome`) is invisible to it. The Swift `AXUIElementCreateApplication(pid)` approach targets the exact process by PID, which is the only way to reach the surfagent Chrome's native dialogs.
+
+**When to check:** If any API call hangs or times out unexpectedly on a tab that was previously responsive, check for a native dialog before retrying. Common triggers:
+- Navigating away from pages with unsaved changes (Google Sheets, web editors, forms)
+- `window.onbeforeunload` handlers
+- Chrome permission prompts
+
+**Decision logic for agents:**
+- **Click "Leave"** if you intentionally navigated away and don't need the page anymore
+- **Click "Cancel"** if the navigation was accidental and you want to keep working on the current page (e.g., Google Sheets with unsaved data)
+
 ---
 
 ## Important Notes

package/CLAUDE.md

@@ -76,6 +76,9 @@ curl -X POST localhost:3456/eval -H 'Content-Type: application/json' -d '{"tab":
 # Bring tab to front
 curl -X POST localhost:3456/focus -H 'Content-Type: application/json' -d '{"tab":"0"}'
 
+# Raw key typing — no clear step, for Google Sheets / contenteditable / canvas
+curl -X POST localhost:3456/type -H 'Content-Type: application/json' -d '{"tab":"0","keys":"Hello","submit":"tab"}'
+
 # Captcha detection and interaction (experimental)
 curl -X POST localhost:3456/captcha -H 'Content-Type: application/json' -d '{"tab":"0","action":"detect"}'
 
@@ -86,6 +89,25 @@ curl localhost:3456/tabs
 curl localhost:3456/health
 ```
 
+### Google Sheets
+
+Google Sheets requires `/type` instead of `/fill` for cell input (because `/fill` does Ctrl+A which selects all cells). Use the name box to navigate, then `/type` to enter data:
+
+```bash
+# 1. Click the name box
+curl -X POST localhost:3456/click -H 'Content-Type: application/json' -d '{"tab":"sheets","selector":"#t-name-box"}'
+
+# 2. Navigate to a cell
+curl -X POST localhost:3456/fill -H 'Content-Type: application/json' -d '{"tab":"sheets","fields":[{"selector":"#t-name-box","value":"A1","clear":true}],"submit":"enter"}'
+
+# 3. Type into the cell (Tab moves right, Enter moves down)
+curl -X POST localhost:3456/type -H 'Content-Type: application/json' -d '{"tab":"sheets","keys":"=SUM(B2:B10)","submit":"tab"}'
+```
+
+Some Sheets buttons (Add Sheet +, toolbar) only respond to CDP mouse events, not DOM clicks. See `API.md` for the CDP mouse click pattern.
+
+**Warning:** Navigating away from unsaved Sheets triggers a native Chrome "Leave page?" dialog that blocks ALL CDP commands. See `API.md` > "Native Chrome Dialogs" for detection and dismissal via Swift AX API.
+
 ### Tab Targeting
 
 All endpoints accept a `tab` field:

package/README.md

@@ -74,6 +74,7 @@ curl -X POST localhost:3456/read -H 'Content-Type: application/json' \
 | `/navigate` | POST | Go to URL, back, or forward in the same tab |
 | `/eval` | POST | Run JavaScript in any tab or cross-origin iframe |
 | `/captcha` | POST | Detect and interact with captchas — Arkose, reCAPTCHA, hCaptcha (experimental) |
+| `/type` | POST | Raw CDP key typing without clearing — for Google Sheets, contenteditable, canvas apps |
 | `/focus` | POST | Bring a tab to the front in Chrome |
 | `/tabs` | GET | List all open Chrome tabs |
 | `/health` | GET | Check if Chrome and API are connected |

package/dist/api/act.d.ts

@@ -91,3 +91,11 @@ export declare function focusTab(tabPattern: string, options: {
     title: string;
     url: string;
 }>;
+export declare function typeKeys(tabPattern: string, keys: string, options: {
+    port?: number;
+    host?: string;
+    submit?: string;
+}): Promise<{
+    typed: number;
+    submitted?: boolean;
+}>;

package/dist/api/act.js

@@ -37,43 +37,80 @@ export async function fillFields(request, options) {
     const results = [];
     for (const field of request.fields) {
         try {
-            // Focus the element and clear it
-            await client.Runtime.evaluate({
-                expression: `
-          (function() {
-            const el = document.querySelector(${JSON.stringify(field.selector)});
-            if (!el) throw new Error('Element not found: ${field.selector}');
-            el.focus();
-            el.click();
-            // Select all existing content so typing replaces it
-            if (el.select) el.select();
-            else if (el.setSelectionRange) el.setSelectionRange(0, el.value?.length || 0);
-          })()
-        `,
+            // Detect element type to choose fill strategy
+            const elInfo = await client.Runtime.evaluate({
+                expression: `(function() {
+          const el = document.querySelector(${JSON.stringify(field.selector)});
+          if (!el) return { found: false };
+          return {
+            found: true,
+            tag: el.tagName,
+            type: el.type || null,
+            contentEditable: el.isContentEditable || false,
+            maxLength: el.maxLength >= 0 ? el.maxLength : null
+          };
+        })()`,
                 returnByValue: true
             });
-            // Clear existing value with select-all + delete
-            await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: 'a', code: 'KeyA', modifiers: 2 }); // Ctrl+A / Cmd+A
-            await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: 'a', code: 'KeyA', modifiers: 2 });
-            await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: 'Backspace', code: 'Backspace' });
-            await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: 'Backspace', code: 'Backspace' });
-            // Type each character via CDP Input.dispatchKeyEvent
-            for (const char of field.value) {
-                await cdp.Input.dispatchKeyEvent({
-                    type: 'keyDown',
-                    key: char,
-                    text: char,
+            const info = elInfo.result.value;
+            if (!info || !info.found) {
+                results.push({ selector: field.selector, success: false, error: `Element not found: ${field.selector}` });
+                continue;
+            }
+            const isDateTimeRange = ['date', 'time', 'datetime-local', 'month', 'week', 'range', 'color'].includes(info.type);
+            const isContentEditable = info.contentEditable && info.tag !== 'INPUT' && info.tag !== 'TEXTAREA';
+            if (isDateTimeRange) {
+                // Date/time/range inputs: set value programmatically + dispatch events
+                await client.Runtime.evaluate({
+                    expression: `(function() {
+            const el = document.querySelector(${JSON.stringify(field.selector)});
+            const nativeSetter = Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, 'value').set;
+            nativeSetter.call(el, ${JSON.stringify(field.value)});
+            el.dispatchEvent(new Event('input', { bubbles: true }));
+            el.dispatchEvent(new Event('change', { bubbles: true }));
+          })()`,
+                    returnByValue: true
                 });
-                await cdp.Input.dispatchKeyEvent({
-                    type: 'keyUp',
-                    key: char,
+            }
+            else {
+                // Focus and clear
+                await client.Runtime.evaluate({
+                    expression: `
+            (function() {
+              const el = document.querySelector(${JSON.stringify(field.selector)});
+              el.focus();
+              el.click();
+              if (el.select) el.select();
+              else if (el.setSelectionRange) el.setSelectionRange(0, el.value?.length || 0);
+            })()
+          `,
+                    returnByValue: true
                 });
+                // Clear existing value with select-all + delete
+                await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: 'a', code: 'KeyA', modifiers: 2 });
+                await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: 'a', code: 'KeyA', modifiers: 2 });
+                await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: 'Backspace', code: 'Backspace' });
+                await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: 'Backspace', code: 'Backspace' });
+                // Type each character via CDP Input.dispatchKeyEvent
+                for (const char of field.value) {
+                    if (char === '\n') {
+                        await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: 'Enter', code: 'Enter', text: '\r', windowsVirtualKeyCode: 13, nativeVirtualKeyCode: 13 });
+                        await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: 'Enter', code: 'Enter', windowsVirtualKeyCode: 13, nativeVirtualKeyCode: 13 });
+                    }
+                    else if (char === '\t') {
+                        await client.Runtime.evaluate({ expression: `document.execCommand('insertText', false, '\\t')` });
+                    }
+                    else {
+                        await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: char, text: char });
+                        await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: char });
+                    }
+                }
             }
-            // Verify the value was set
-            const verify = await client.Runtime.evaluate({
-                expression: `document.querySelector(${JSON.stringify(field.selector)})?.value`,
-                returnByValue: true
-            });
+            // Verify: use value for inputs, textContent for contenteditable
+            const verifyExpr = isContentEditable
+                ? `document.querySelector(${JSON.stringify(field.selector)})?.textContent?.trim()`
+                : `document.querySelector(${JSON.stringify(field.selector)})?.value`;
+            const verify = await client.Runtime.evaluate({ expression: verifyExpr, returnByValue: true });
             const actual = verify.result.value;
             if (actual === field.value) {
                 results.push({ selector: field.selector, success: true });
@@ -81,6 +118,13 @@ export async function fillFields(request, options) {
             else if (actual === undefined || actual === null) {
                 results.push({ selector: field.selector, success: false, error: `Element not found or has no value: ${field.selector}` });
             }
+            else if (info.maxLength && actual === field.value.substring(0, info.maxLength)) {
+                // Maxlength truncation — fill worked within constraint
+                results.push({ selector: field.selector, success: true, error: `Truncated to maxlength=${info.maxLength}` });
+            }
+            else if (isContentEditable && actual.includes(field.value)) {
+                results.push({ selector: field.selector, success: true });
+            }
             else {
                 results.push({ selector: field.selector, success: false, error: `Value mismatch: expected "${field.value}", got "${actual}"` });
             }
@@ -138,13 +182,25 @@ export async function clickElement(request, options) {
           }
           if (!el && text) {
             const lower = text.toLowerCase();
-            const all = document.querySelectorAll('a, button, input[type="submit"], [role="button"], [role="option"], [role="menuitem"], [role="listitem"], [role="tab"], [role="link"], li[aria-label], [onclick]');
+            const all = document.querySelectorAll('a, button, input[type="submit"], [role="button"], [role="option"], [role="menuitem"], [role="listitem"], [role="tab"], [role="link"], li[aria-label], [onclick], label');
+            let bestMatch = null;
+            let bestScore = Infinity; // lower is better
             for (const candidate of all) {
               const t = (candidate.innerText || candidate.textContent || candidate.value || candidate.getAttribute('aria-label') || '').trim();
-              if (t.toLowerCase().includes(lower)) { el = candidate; break; }
+              const tLower = t.toLowerCase();
+              if (!tLower.includes(lower)) continue;
+              // Score: 0 = exact, 1 = starts-with, 2+ = contains (shorter text = better)
+              let score;
+              if (tLower === lower) score = 0;
+              else if (tLower.startsWith(lower)) score = 1;
+              else score = 2 + t.length;
+              if (score < bestScore) { bestMatch = candidate; bestScore = score; }
+              if (score === 0) break; // exact match, stop
             }
+            el = bestMatch;
           }
           if (!el) return { success: false, error: 'Element not found' };
+          if (el.disabled || el.getAttribute('aria-disabled') === 'true') return { success: false, error: 'Element is disabled' };
 
           el.scrollIntoView({ block: 'center' });
 
@@ -192,17 +248,39 @@ export async function scrollPage(request, options) {
           const scrollY = Math.round(window.scrollY);
           const scrollHeight = document.documentElement.scrollHeight;
           const viewportHeight = window.innerHeight;
-          const atBottom = (scrollY + viewportHeight) >= (scrollHeight - 10);
+          const atBottom = (scrollY + viewportHeight) >= (scrollHeight - 2);
 
-          // Get visible text content
-          const centerY = scrollY + viewportHeight / 2;
-          const elements = document.elementsFromPoint(window.innerWidth / 2, viewportHeight / 2);
+          // Get visible text content from elements in the current viewport
           let contentPreview = '';
-          for (const el of elements) {
+          const visibleTexts = [];
+          const mainEl = document.querySelector('main, article, [role="main"]') || document.body;
+          const allEls = mainEl.querySelectorAll('p, li, td, th, h1, h2, h3, h4, h5, h6, dd, dt, blockquote, pre');
+          for (const el of allEls) {
+            if (visibleTexts.length >= 30) break;
+            const rect = el.getBoundingClientRect();
+            // Element must be within the viewport
+            if (rect.bottom < 0 || rect.top > viewportHeight || rect.height === 0) continue;
+            // Skip fixed/sticky elements (nav, TOC, sidebars)
+            const style = window.getComputedStyle(el.closest('nav, aside, [role="navigation"]') || el);
+            if (style.position === 'fixed' || style.position === 'sticky') continue;
             const text = el.innerText?.trim();
-            if (text && text.length > 50) {
-              contentPreview = text.substring(0, 1500);
-              break;
+            if (!text || text.length < 5) continue;
+            // Skip if text is too long (likely a parent container)
+            if (text.length > 500) continue;
+            // Skip duplicates
+            if (visibleTexts.some(t => t.includes(text) || text.includes(t))) continue;
+            visibleTexts.push(text);
+          }
+          contentPreview = visibleTexts.join('\\n').substring(0, 1500);
+          if (!contentPreview) {
+            // Fallback: grab from center point
+            const elements = document.elementsFromPoint(window.innerWidth / 2, viewportHeight / 2);
+            for (const el of elements) {
+              const text = el.innerText?.trim();
+              if (text && text.length > 50 && text.length < 3000) {
+                contentPreview = text.substring(0, 1500);
+                break;
+              }
             }
           }
 
@@ -237,8 +315,16 @@ export async function navigatePage(request, options) {
             await new Promise(resolve => setTimeout(resolve, waitMs));
         }
         else if (request.url) {
+            // Block dangerous URL schemes
+            const scheme = request.url.trim().toLowerCase().split(':')[0];
+            if (['javascript', 'vbscript'].includes(scheme)) {
+                await client.close();
+                throw new Error('Blocked: javascript: URLs are not allowed');
+            }
             await client.Page.navigate({ url: request.url });
-            await client.Page.loadEventFired();
+            // Race loadEventFired against a timeout to prevent hanging on non-loading URLs
+            const loadTimeout = new Promise(resolve => setTimeout(resolve, Math.min(waitMs + 10000, 30000)));
+            await Promise.race([client.Page.loadEventFired(), loadTimeout]);
             await new Promise(resolve => setTimeout(resolve, waitMs));
         }
         const result = await client.Runtime.evaluate({
@@ -262,9 +348,18 @@ export async function evalInTab(tab, expression, options) {
         const timeout = new Promise((_, reject) => setTimeout(() => reject(new Error('Eval timed out after 30s')), 30000));
         const evalPromise = client.Runtime.evaluate({
             expression,
-            returnByValue: true
+            returnByValue: true,
+            awaitPromise: true
         });
         const result = await Promise.race([evalPromise, timeout]);
+        // Check for exceptions (syntax errors, thrown errors, etc.)
+        if (result.exceptionDetails) {
+            const desc = result.exceptionDetails.exception?.description
+                || result.exceptionDetails.text
+                || 'Unknown error';
+            await client.close();
+            return { __error: desc };
+        }
         await client.close();
         return result.result.value ?? null;
     }
@@ -579,3 +674,36 @@ export async function focusTab(tabPattern, options) {
         throw error;
     }
 }
+// Raw CDP key typing — no clear step, no element focus. Types directly into whatever has focus.
+// Designed for apps like Google Sheets where Ctrl+A/Backspace clear causes side effects.
+export async function typeKeys(tabPattern, keys, options) {
+    const port = options.port || 9222;
+    const host = options.host || 'localhost';
+    const tab = await resolveTab(tabPattern, port, host);
+    const client = await connectToTab(tab.id, port, host);
+    const cdp = client;
+    try {
+        // Type each character via CDP Input.dispatchKeyEvent
+        for (const char of keys) {
+            await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: char, text: char });
+            await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: char });
+        }
+        let submitted = false;
+        if (options.submit === 'enter') {
+            await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: 'Enter', code: 'Enter', windowsVirtualKeyCode: 13, nativeVirtualKeyCode: 13 });
+            await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: 'Enter', code: 'Enter', windowsVirtualKeyCode: 13, nativeVirtualKeyCode: 13 });
+            submitted = true;
+        }
+        else if (options.submit === 'tab') {
+            await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: 'Tab', code: 'Tab', windowsVirtualKeyCode: 9, nativeVirtualKeyCode: 9 });
+            await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: 'Tab', code: 'Tab', windowsVirtualKeyCode: 9, nativeVirtualKeyCode: 9 });
+            submitted = true;
+        }
+        await client.close();
+        return { typed: keys.length, submitted };
+    }
+    catch (error) {
+        await client.close();
+        throw error;
+    }
+}

package/dist/api/recon.d.ts

@@ -28,7 +28,9 @@ export interface ReconResult {
         role: string | null;
         x: number;
         y: number;
+        data?: Record<string, string>;
     }[];
+    totalElements: number;
     forms: {
         action: string | null;
         method: string | null;

package/dist/api/recon.js

@@ -30,7 +30,14 @@ const EXTRACTION_SCRIPT = `
     const tag = el.tagName.toLowerCase();
     if (el.getAttribute('aria-label')) return tag + '[aria-label="' + el.getAttribute('aria-label') + '"]';
     if (el.getAttribute('data-testid')) return '[data-testid="' + el.getAttribute('data-testid') + '"]';
-    if (el.getAttribute('name')) return tag + '[name="' + el.getAttribute('name') + '"]';
+    if (el.getAttribute('name')) {
+      const nameSelector = tag + '[name="' + el.getAttribute('name') + '"]';
+      // Disambiguate radio/checkbox with same name by adding value
+      if ((el.type === 'radio' || el.type === 'checkbox') && el.value) {
+        return nameSelector + '[value="' + el.value + '"]';
+      }
+      return nameSelector;
+    }
     // Positional fallback
     const parent = el.parentElement;
     if (!parent) return tag;
@@ -106,7 +113,12 @@ const EXTRACTION_SCRIPT = `
         selector: buildSelector(el),
         role: el.getAttribute('role'),
         x: Math.round(rect.x),
-        y: Math.round(rect.y)
+        y: Math.round(rect.y),
+        ...(el.dataset && Object.keys(el.dataset).length > 0 ? {
+          data: Object.fromEntries(
+            ['date','iso','value','testid','id'].filter(k => el.dataset[k]).map(k => [k, el.dataset[k]])
+          )
+        } : {})
       });
     }
   }
@@ -208,6 +220,16 @@ const EXTRACTION_SCRIPT = `
       }
     }
   }
+  // Deduplicate: remove overlays that are descendants of other overlays
+  const deduped = overlays.filter((o, i) => {
+    const el = document.querySelector(o.selector);
+    if (!el) return true;
+    return !overlays.some((other, j) => {
+      if (i === j) return false;
+      const otherEl = document.querySelector(other.selector);
+      return otherEl && otherEl !== el && otherEl.contains(el);
+    });
+  });
 
   // ---- Captcha detection ----
   const captchas = [];
@@ -238,9 +260,10 @@ const EXTRACTION_SCRIPT = `
     headings,
     navigation: navigation.slice(0, 50),
     elements: elements.slice(0, 150),
+    totalElements: elements.length,
     forms,
     landmarks,
-    overlays,
+    overlays: deduped,
     captchas,
     contentSummary
   };
@@ -291,6 +314,7 @@ export async function reconUrl(url, options) {
             headings: data.headings,
             navigation: data.navigation,
             elements: data.elements,
+            totalElements: data.totalElements || data.elements?.length || 0,
             forms: data.forms,
             contentSummary: data.contentSummary,
             landmarks: data.landmarks,
@@ -348,6 +372,7 @@ export async function reconTab(tabPattern, options) {
             headings: data.headings,
             navigation: data.navigation,
             elements: data.elements,
+            totalElements: data.totalElements || data.elements?.length || 0,
             forms: data.forms,
             contentSummary: data.contentSummary,
             landmarks: data.landmarks,

package/dist/api/server.js

@@ -1,7 +1,7 @@
 #!/usr/bin/env node
 import http from 'node:http';
 import { reconUrl, reconTab } from './recon.js';
-import { fillFields, clickElement, scrollPage, navigatePage, evalInTab, focusTab, readPage, captchaInteract, dismissOverlays } from './act.js';
+import { fillFields, clickElement, scrollPage, navigatePage, evalInTab, focusTab, readPage, captchaInteract, dismissOverlays, typeKeys } from './act.js';
 import { getAllTabs } from '../chrome/tabs.js';
 const PORT = parseInt(process.env.API_PORT || '3456', 10);
 const CDP_PORT = parseInt(process.env.CDP_PORT || '9222', 10);
@@ -136,14 +136,32 @@ const server = http.createServer(async (req, res) => {
                 return json(res, 400, { error: 'Provide "tab" and "expression"' });
             }
             const result = await evalInTab(body.tab, body.expression, { port: CDP_PORT, host: CDP_HOST });
+            if (result && result.__error) {
+                return json(res, 200, { result: null, error: result.__error });
+            }
             return json(res, 200, { result });
         }
+        // POST /type — raw CDP key typing, no clear step (for Google Sheets, contenteditable, etc.)
+        if (path === '/type' && req.method === 'POST') {
+            const body = parseBody(await readBody(req));
+            if (!body.tab || !body.keys) {
+                return json(res, 400, { error: 'Provide "tab" and "keys" (string to type), optional "submit": "enter"|"tab"' });
+            }
+            const result = await typeKeys(body.tab, body.keys, { port: CDP_PORT, host: CDP_HOST, submit: body.submit });
+            return json(res, 200, result);
+        }
         // POST /navigate — go to url, back, or forward in same tab
         if (path === '/navigate' && req.method === 'POST') {
             const body = parseBody(await readBody(req));
             if (!body.tab) {
                 return json(res, 400, { error: 'Provide "tab" and one of: "url", "back":true, "forward":true' });
             }
+            if (!body.url && !body.back && !body.forward) {
+                return json(res, 400, { error: 'Provide one of: "url", "back":true, "forward":true' });
+            }
+            if ((body.url && body.back) || (body.url && body.forward) || (body.back && body.forward)) {
+                return json(res, 400, { error: 'Provide only one of: "url", "back", "forward"' });
+            }
             const result = await navigatePage(body, { port: CDP_PORT, host: CDP_HOST });
             return json(res, 200, result);
         }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "surfagent",
-  "version": "1.0.9",
+  "version": "1.1.1",
   "description": "Browser automation API for AI agents — structured page recon, form filling, clicking, and navigation via Chrome CDP",
   "keywords": [
     "ai-agent",

package/src/api/act.ts

@@ -60,52 +60,93 @@ export async function fillFields(
 
   for (const field of request.fields) {
     try {
-      // Focus the element and clear it
-      await client.Runtime.evaluate({
-        expression: `
-          (function() {
-            const el = document.querySelector(${JSON.stringify(field.selector)});
-            if (!el) throw new Error('Element not found: ${field.selector}');
-            el.focus();
-            el.click();
-            // Select all existing content so typing replaces it
-            if (el.select) el.select();
-            else if (el.setSelectionRange) el.setSelectionRange(0, el.value?.length || 0);
-          })()
-        `,
+      // Detect element type to choose fill strategy
+      const elInfo = await client.Runtime.evaluate({
+        expression: `(function() {
+          const el = document.querySelector(${JSON.stringify(field.selector)});
+          if (!el) return { found: false };
+          return {
+            found: true,
+            tag: el.tagName,
+            type: el.type || null,
+            contentEditable: el.isContentEditable || false,
+            maxLength: el.maxLength >= 0 ? el.maxLength : null
+          };
+        })()`,
         returnByValue: true
       });
+      const info = elInfo.result.value as any;
+      if (!info || !info.found) {
+        results.push({ selector: field.selector, success: false, error: `Element not found: ${field.selector}` });
+        continue;
+      }
+
+      const isDateTimeRange = ['date', 'time', 'datetime-local', 'month', 'week', 'range', 'color'].includes(info.type);
+      const isContentEditable = info.contentEditable && info.tag !== 'INPUT' && info.tag !== 'TEXTAREA';
 
-      // Clear existing value with select-all + delete
-      await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: 'a', code: 'KeyA', modifiers: 2 }); // Ctrl+A / Cmd+A
-      await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: 'a', code: 'KeyA', modifiers: 2 });
-      await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: 'Backspace', code: 'Backspace' });
-      await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: 'Backspace', code: 'Backspace' });
-
-      // Type each character via CDP Input.dispatchKeyEvent
-      for (const char of field.value) {
-        await cdp.Input.dispatchKeyEvent({
-          type: 'keyDown',
-          key: char,
-          text: char,
+      if (isDateTimeRange) {
+        // Date/time/range inputs: set value programmatically + dispatch events
+        await client.Runtime.evaluate({
+          expression: `(function() {
+            const el = document.querySelector(${JSON.stringify(field.selector)});
+            const nativeSetter = Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, 'value').set;
+            nativeSetter.call(el, ${JSON.stringify(field.value)});
+            el.dispatchEvent(new Event('input', { bubbles: true }));
+            el.dispatchEvent(new Event('change', { bubbles: true }));
+          })()`,
+          returnByValue: true
         });
-        await cdp.Input.dispatchKeyEvent({
-          type: 'keyUp',
-          key: char,
+      } else {
+        // Focus and clear
+        await client.Runtime.evaluate({
+          expression: `
+            (function() {
+              const el = document.querySelector(${JSON.stringify(field.selector)});
+              el.focus();
+              el.click();
+              if (el.select) el.select();
+              else if (el.setSelectionRange) el.setSelectionRange(0, el.value?.length || 0);
+            })()
+          `,
+          returnByValue: true
         });
+
+        // Clear existing value with select-all + delete
+        await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: 'a', code: 'KeyA', modifiers: 2 });
+        await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: 'a', code: 'KeyA', modifiers: 2 });
+        await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: 'Backspace', code: 'Backspace' });
+        await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: 'Backspace', code: 'Backspace' });
+
+        // Type each character via CDP Input.dispatchKeyEvent
+        for (const char of field.value) {
+          if (char === '\n') {
+            await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: 'Enter', code: 'Enter', text: '\r', windowsVirtualKeyCode: 13, nativeVirtualKeyCode: 13 });
+            await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: 'Enter', code: 'Enter', windowsVirtualKeyCode: 13, nativeVirtualKeyCode: 13 });
+          } else if (char === '\t') {
+            await client.Runtime.evaluate({ expression: `document.execCommand('insertText', false, '\\t')` });
+          } else {
+            await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: char, text: char });
+            await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: char });
+          }
+        }
       }
 
-      // Verify the value was set
-      const verify = await client.Runtime.evaluate({
-        expression: `document.querySelector(${JSON.stringify(field.selector)})?.value`,
-        returnByValue: true
-      });
+      // Verify: use value for inputs, textContent for contenteditable
+      const verifyExpr = isContentEditable
+        ? `document.querySelector(${JSON.stringify(field.selector)})?.textContent?.trim()`
+        : `document.querySelector(${JSON.stringify(field.selector)})?.value`;
+      const verify = await client.Runtime.evaluate({ expression: verifyExpr, returnByValue: true });
 
       const actual = verify.result.value as string;
       if (actual === field.value) {
         results.push({ selector: field.selector, success: true });
       } else if (actual === undefined || actual === null) {
         results.push({ selector: field.selector, success: false, error: `Element not found or has no value: ${field.selector}` });
+      } else if (info.maxLength && actual === field.value.substring(0, info.maxLength)) {
+        // Maxlength truncation — fill worked within constraint
+        results.push({ selector: field.selector, success: true, error: `Truncated to maxlength=${info.maxLength}` });
+      } else if (isContentEditable && actual.includes(field.value)) {
+        results.push({ selector: field.selector, success: true });
       } else {
         results.push({ selector: field.selector, success: false, error: `Value mismatch: expected "${field.value}", got "${actual}"` });
       }
@@ -176,13 +217,25 @@ export async function clickElement(
           }
           if (!el && text) {
             const lower = text.toLowerCase();
-            const all = document.querySelectorAll('a, button, input[type="submit"], [role="button"], [role="option"], [role="menuitem"], [role="listitem"], [role="tab"], [role="link"], li[aria-label], [onclick]');
+            const all = document.querySelectorAll('a, button, input[type="submit"], [role="button"], [role="option"], [role="menuitem"], [role="listitem"], [role="tab"], [role="link"], li[aria-label], [onclick], label');
+            let bestMatch = null;
+            let bestScore = Infinity; // lower is better
             for (const candidate of all) {
               const t = (candidate.innerText || candidate.textContent || candidate.value || candidate.getAttribute('aria-label') || '').trim();
-              if (t.toLowerCase().includes(lower)) { el = candidate; break; }
+              const tLower = t.toLowerCase();
+              if (!tLower.includes(lower)) continue;
+              // Score: 0 = exact, 1 = starts-with, 2+ = contains (shorter text = better)
+              let score;
+              if (tLower === lower) score = 0;
+              else if (tLower.startsWith(lower)) score = 1;
+              else score = 2 + t.length;
+              if (score < bestScore) { bestMatch = candidate; bestScore = score; }
+              if (score === 0) break; // exact match, stop
             }
+            el = bestMatch;
           }
           if (!el) return { success: false, error: 'Element not found' };
+          if (el.disabled || el.getAttribute('aria-disabled') === 'true') return { success: false, error: 'Element is disabled' };
 
           el.scrollIntoView({ block: 'center' });
 
@@ -243,17 +296,39 @@ export async function scrollPage(
           const scrollY = Math.round(window.scrollY);
           const scrollHeight = document.documentElement.scrollHeight;
           const viewportHeight = window.innerHeight;
-          const atBottom = (scrollY + viewportHeight) >= (scrollHeight - 10);
+          const atBottom = (scrollY + viewportHeight) >= (scrollHeight - 2);
 
-          // Get visible text content
-          const centerY = scrollY + viewportHeight / 2;
-          const elements = document.elementsFromPoint(window.innerWidth / 2, viewportHeight / 2);
+          // Get visible text content from elements in the current viewport
           let contentPreview = '';
-          for (const el of elements) {
+          const visibleTexts = [];
+          const mainEl = document.querySelector('main, article, [role="main"]') || document.body;
+          const allEls = mainEl.querySelectorAll('p, li, td, th, h1, h2, h3, h4, h5, h6, dd, dt, blockquote, pre');
+          for (const el of allEls) {
+            if (visibleTexts.length >= 30) break;
+            const rect = el.getBoundingClientRect();
+            // Element must be within the viewport
+            if (rect.bottom < 0 || rect.top > viewportHeight || rect.height === 0) continue;
+            // Skip fixed/sticky elements (nav, TOC, sidebars)
+            const style = window.getComputedStyle(el.closest('nav, aside, [role="navigation"]') || el);
+            if (style.position === 'fixed' || style.position === 'sticky') continue;
             const text = el.innerText?.trim();
-            if (text && text.length > 50) {
-              contentPreview = text.substring(0, 1500);
-              break;
+            if (!text || text.length < 5) continue;
+            // Skip if text is too long (likely a parent container)
+            if (text.length > 500) continue;
+            // Skip duplicates
+            if (visibleTexts.some(t => t.includes(text) || text.includes(t))) continue;
+            visibleTexts.push(text);
+          }
+          contentPreview = visibleTexts.join('\\n').substring(0, 1500);
+          if (!contentPreview) {
+            // Fallback: grab from center point
+            const elements = document.elementsFromPoint(window.innerWidth / 2, viewportHeight / 2);
+            for (const el of elements) {
+              const text = el.innerText?.trim();
+              if (text && text.length > 50 && text.length < 3000) {
+                contentPreview = text.substring(0, 1500);
+                break;
+              }
             }
           }
 
@@ -301,8 +376,16 @@ export async function navigatePage(
       await client.Runtime.evaluate({ expression: 'window.history.forward()' });
       await new Promise(resolve => setTimeout(resolve, waitMs));
     } else if (request.url) {
+      // Block dangerous URL schemes
+      const scheme = request.url.trim().toLowerCase().split(':')[0];
+      if (['javascript', 'vbscript'].includes(scheme)) {
+        await client.close();
+        throw new Error('Blocked: javascript: URLs are not allowed');
+      }
       await (client.Page as any).navigate({ url: request.url });
-      await (client.Page as any).loadEventFired();
+      // Race loadEventFired against a timeout to prevent hanging on non-loading URLs
+      const loadTimeout = new Promise<void>(resolve => setTimeout(resolve, Math.min(waitMs + 10000, 30000)));
+      await Promise.race([(client.Page as any).loadEventFired(), loadTimeout]);
       await new Promise(resolve => setTimeout(resolve, waitMs));
     }
 
@@ -336,9 +419,18 @@ export async function evalInTab(
     );
     const evalPromise = client.Runtime.evaluate({
       expression,
-      returnByValue: true
+      returnByValue: true,
+      awaitPromise: true
     });
     const result = await Promise.race([evalPromise, timeout]);
+    // Check for exceptions (syntax errors, thrown errors, etc.)
+    if (result.exceptionDetails) {
+      const desc = result.exceptionDetails.exception?.description
+        || result.exceptionDetails.text
+        || 'Unknown error';
+      await client.close();
+      return { __error: desc };
+    }
     await client.close();
     return result.result.value ?? null;
   } catch (error) {
@@ -683,3 +775,43 @@ export async function focusTab(
     throw error;
   }
 }
+
+// Raw CDP key typing — no clear step, no element focus. Types directly into whatever has focus.
+// Designed for apps like Google Sheets where Ctrl+A/Backspace clear causes side effects.
+export async function typeKeys(
+  tabPattern: string,
+  keys: string,
+  options: { port?: number; host?: string; submit?: string }
+): Promise<{ typed: number; submitted?: boolean }> {
+  const port = options.port || 9222;
+  const host = options.host || 'localhost';
+
+  const tab = await resolveTab(tabPattern, port, host);
+  const client = await connectToTab(tab.id, port, host);
+  const cdp = client as any;
+
+  try {
+    // Type each character via CDP Input.dispatchKeyEvent
+    for (const char of keys) {
+      await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: char, text: char });
+      await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: char });
+    }
+
+    let submitted = false;
+    if (options.submit === 'enter') {
+      await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: 'Enter', code: 'Enter', windowsVirtualKeyCode: 13, nativeVirtualKeyCode: 13 });
+      await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: 'Enter', code: 'Enter', windowsVirtualKeyCode: 13, nativeVirtualKeyCode: 13 });
+      submitted = true;
+    } else if (options.submit === 'tab') {
+      await cdp.Input.dispatchKeyEvent({ type: 'keyDown', key: 'Tab', code: 'Tab', windowsVirtualKeyCode: 9, nativeVirtualKeyCode: 9 });
+      await cdp.Input.dispatchKeyEvent({ type: 'keyUp', key: 'Tab', code: 'Tab', windowsVirtualKeyCode: 9, nativeVirtualKeyCode: 9 });
+      submitted = true;
+    }
+
+    await client.close();
+    return { typed: keys.length, submitted };
+  } catch (error) {
+    await client.close();
+    throw error;
+  }
+}

package/src/api/recon.ts

@@ -25,7 +25,9 @@ export interface ReconResult {
     role: string | null;
     x: number;
     y: number;
+    data?: Record<string, string>;
   }[];
+  totalElements: number;
   forms: {
     action: string | null;
     method: string | null;
@@ -77,7 +79,14 @@ const EXTRACTION_SCRIPT = `
     const tag = el.tagName.toLowerCase();
     if (el.getAttribute('aria-label')) return tag + '[aria-label="' + el.getAttribute('aria-label') + '"]';
     if (el.getAttribute('data-testid')) return '[data-testid="' + el.getAttribute('data-testid') + '"]';
-    if (el.getAttribute('name')) return tag + '[name="' + el.getAttribute('name') + '"]';
+    if (el.getAttribute('name')) {
+      const nameSelector = tag + '[name="' + el.getAttribute('name') + '"]';
+      // Disambiguate radio/checkbox with same name by adding value
+      if ((el.type === 'radio' || el.type === 'checkbox') && el.value) {
+        return nameSelector + '[value="' + el.value + '"]';
+      }
+      return nameSelector;
+    }
     // Positional fallback
     const parent = el.parentElement;
     if (!parent) return tag;
@@ -153,7 +162,12 @@ const EXTRACTION_SCRIPT = `
         selector: buildSelector(el),
         role: el.getAttribute('role'),
         x: Math.round(rect.x),
-        y: Math.round(rect.y)
+        y: Math.round(rect.y),
+        ...(el.dataset && Object.keys(el.dataset).length > 0 ? {
+          data: Object.fromEntries(
+            ['date','iso','value','testid','id'].filter(k => el.dataset[k]).map(k => [k, el.dataset[k]])
+          )
+        } : {})
       });
     }
   }
@@ -255,6 +269,16 @@ const EXTRACTION_SCRIPT = `
       }
     }
   }
+  // Deduplicate: remove overlays that are descendants of other overlays
+  const deduped = overlays.filter((o, i) => {
+    const el = document.querySelector(o.selector);
+    if (!el) return true;
+    return !overlays.some((other, j) => {
+      if (i === j) return false;
+      const otherEl = document.querySelector(other.selector);
+      return otherEl && otherEl !== el && otherEl.contains(el);
+    });
+  });
 
   // ---- Captcha detection ----
   const captchas = [];
@@ -285,9 +309,10 @@ const EXTRACTION_SCRIPT = `
     headings,
     navigation: navigation.slice(0, 50),
     elements: elements.slice(0, 150),
+    totalElements: elements.length,
     forms,
     landmarks,
-    overlays,
+    overlays: deduped,
     captchas,
     contentSummary
   };
@@ -352,6 +377,7 @@ export async function reconUrl(
       headings: data.headings,
       navigation: data.navigation,
       elements: data.elements,
+      totalElements: data.totalElements || data.elements?.length || 0,
       forms: data.forms,
       contentSummary: data.contentSummary,
       landmarks: data.landmarks,
@@ -414,6 +440,7 @@ export async function reconTab(
       headings: data.headings,
       navigation: data.navigation,
       elements: data.elements,
+      totalElements: data.totalElements || data.elements?.length || 0,
       forms: data.forms,
       contentSummary: data.contentSummary,
       landmarks: data.landmarks,

package/src/api/server.ts

@@ -2,7 +2,7 @@
 
 import http from 'node:http';
 import { reconUrl, reconTab } from './recon.js';
-import { fillFields, clickElement, scrollPage, navigatePage, evalInTab, focusTab, readPage, captchaInteract, dismissOverlays } from './act.js';
+import { fillFields, clickElement, scrollPage, navigatePage, evalInTab, focusTab, readPage, captchaInteract, dismissOverlays, typeKeys } from './act.js';
 import { getAllTabs } from '../chrome/tabs.js';
 
 const PORT = parseInt(process.env.API_PORT || '3456', 10);
@@ -160,15 +160,34 @@ const server = http.createServer(async (req, res) => {
         return json(res, 400, { error: 'Provide "tab" and "expression"' });
       }
       const result = await evalInTab(body.tab, body.expression, { port: CDP_PORT, host: CDP_HOST });
+      if (result && result.__error) {
+        return json(res, 200, { result: null, error: result.__error });
+      }
       return json(res, 200, { result });
     }
 
+    // POST /type — raw CDP key typing, no clear step (for Google Sheets, contenteditable, etc.)
+    if (path === '/type' && req.method === 'POST') {
+      const body = parseBody(await readBody(req));
+      if (!body.tab || !body.keys) {
+        return json(res, 400, { error: 'Provide "tab" and "keys" (string to type), optional "submit": "enter"|"tab"' });
+      }
+      const result = await typeKeys(body.tab, body.keys, { port: CDP_PORT, host: CDP_HOST, submit: body.submit });
+      return json(res, 200, result);
+    }
+
     // POST /navigate — go to url, back, or forward in same tab
     if (path === '/navigate' && req.method === 'POST') {
       const body = parseBody(await readBody(req));
       if (!body.tab) {
         return json(res, 400, { error: 'Provide "tab" and one of: "url", "back":true, "forward":true' });
       }
+      if (!body.url && !body.back && !body.forward) {
+        return json(res, 400, { error: 'Provide one of: "url", "back":true, "forward":true' });
+      }
+      if ((body.url && body.back) || (body.url && body.forward) || (body.back && body.forward)) {
+        return json(res, 400, { error: 'Provide only one of: "url", "back", "forward"' });
+      }
       const result = await navigatePage(body, { port: CDP_PORT, host: CDP_HOST });
       return json(res, 200, result);
     }