machine-bridge-mcp 0.13.0 → 0.14.0

package/CHANGELOG.md

@@ -1,5 +1,24 @@
 # Changelog
 
+## 0.14.0 - 2026-07-13
+
+### Windows installation and OAuth callback reliability
+
+- Bootstrap installation through pinned npm 12.0.1 from an empty temporary directory before using `--allow-scripts`, declare the npm 12 requirement in package `engines`, and make `machine-mcp doctor` verify the active npm version. This removes the unsupported old-npm path shown by Windows and gives accurate diagnostics for `Unknown cli config "--allow-scripts"` and legacy `devEngines.node` errors without guessing which package supplied the invalid metadata.
+- Construct OAuth callback destinations with the URL API and return `303 See Other` after consent instead of manually concatenating a `Location` string. Add an end-to-end ChatGPT callback regression with reserved state characters and exact origin/path checks.
+- Extend the isolated install smoke test to require npm 12, install from an empty directory, verify the packaged npm engine requirement, and run the test on Linux, macOS, and Windows CI.
+
+### Existing-profile browser automation
+
+- Keep the authenticated Manifest V3 extension as the primary browser backend so automation operates the user's ordinary Chromium profile, tabs, extensions, cookies, and login state instead of launching a separate profile. Add tab creation/activation/closure and explicit combined waits for URL, document readiness, page text, and element state.
+- Replace fragile inspect-then-selector flows with bounded semantic snapshots containing stable per-document/frame element references, visibility/enabled/editable state, and viewport geometry. Page actions now wait for attachment, visibility, enabled/editable state, geometric stability, and unobscured pointer hit targets; ambiguous selectors and stale references fail explicitly.
+- Add double-click, hover, append-text, and scroll-into-view actions. Top-frame click, double-click, hover, key press, and text input can use fixed short-lived Chromium DevTools Input commands through explicit `auto`, `trusted`, or `dom` modes. The extension exposes no caller-selected CDP method or arbitrary JavaScript and detaches the debugger in `finally`.
+
+### Architecture, security, tests, and documentation
+
+- Extract browser command normalization into a focused local module and isolate trusted input in a fixed extension module. Add a versioned extension capability handshake, stale-build reload guidance, explicit keepalive handling, and replacement validation that preserves the current compatible connection until the candidate is accepted; accepted replacements reject in-flight direct and proxied requests with retry guidance instead of leaving them to time out. Preserve resource-backed secrets/files, strict action/value validation, bounded broker messages, cancellation, source limits, and the existing owner-only pairing model.
+- Add behavior-level tests for command contracts, trusted-input command sequences and cleanup, semantic-ref stability, deterministic scrolling, obscured-target waiting, stale refs, broker routing, catalog parity, and architecture invariants. Update browser setup, permission, security, architecture, testing, and tool documentation.
+
 ## 0.13.0 - 2026-07-13
 
 ### Automatic capability routing and observability

package/README.md

@@ -47,10 +47,38 @@ This default prioritizes usability, not least privilege. `run_process` and proce
 
 Node.js 26 or newer and npm 12 or newer are required. The repository pins the active development versions in `.node-version`, `.nvmrc`, and `packageManager`; project installs fail on older Node runtimes.
 
+Use a new temporary directory for the bootstrap. The existing `npm`/`npx` front-end may inspect nearby project metadata before it launches the requested npm version; starting from an empty directory prevents an unrelated legacy `devEngines` declaration from blocking the bootstrap. The actual installation is then executed by the pinned npm 12 CLI rather than whichever npm happens to be first on `PATH`.
+
+macOS/Linux:
+
 ```sh
-npm install -g --omit=optional --allow-scripts=esbuild,workerd,sharp,fsevents machine-bridge-mcp@latest
+install_dir="$(mktemp -d)"
+(
+  cd "$install_dir"
+  npx --yes npm@12.0.1 install --global npm@12.0.1
+  npx --yes npm@12.0.1 install --global --omit=optional --allow-scripts=esbuild,workerd,sharp,fsevents machine-bridge-mcp@latest
+)
+rm -rf "$install_dir"
+npm --version
+machine-mcp doctor
 ```
 
+Windows Command Prompt (`cmd.exe`):
+
+```bat
+set "MBM_INSTALL_DIR=%TEMP%\machine-bridge-mcp-install-%RANDOM%-%RANDOM%"
+mkdir "%MBM_INSTALL_DIR%"
+pushd "%MBM_INSTALL_DIR%"
+npx --yes npm@12.0.1 install --global npm@12.0.1
+npx --yes npm@12.0.1 install --global --omit=optional --allow-scripts=esbuild,workerd,sharp,fsevents machine-bridge-mcp@latest
+popd
+rmdir /s /q "%MBM_INSTALL_DIR%"
+npm --version
+machine-mcp doctor
+```
+
+`Unknown cli config "--allow-scripts"` proves the Machine Bridge install was executed by npm 11 or older rather than the required npm 12. `Invalid property "node"` or `Invalid property "devEngines.node"` means that npm parsed a legacy `devEngines` object; the npm debug log is required to identify which package supplied it. The empty-directory, pinned-npm procedure avoids relying on that old parser. If `npm --version` still reports an older version after the bootstrap, reopen the terminal and rerun `machine-mcp doctor`.
+
 Recent npm releases may otherwise warn that Wrangler's native dependencies (`esbuild`, `workerd`, and `sharp`) have install scripts awaiting approval. The scoped command approves the reviewed native script names that npm 12 evaluates during global resolution while `--omit=optional` keeps optional `fsevents` out of the installed runtime. `fsevents` is used for development-time filesystem watching rather than Machine Bridge runtime or deployment. Omitting `--omit=optional` can therefore produce a harmless blocked-script warning for `fsevents@2.3.3`; use the documented command rather than changing global npm policy. `machine-mcp doctor` remains the authoritative runtime check.
 
 From a source checkout, the checked-in exact-version `allowScripts` policy approves the reviewed native dependencies:
@@ -81,14 +109,13 @@ On first remote start, the CLI:
 
 A normal `machine-mcp` invocation is a foreground start: it remains attached to the terminal and prints `info` logs. Startup and other state-changing operations use an owner-token/process-identity lock and wait up to 30 seconds for an ordinary concurrent startup to finish instead of failing on a brief launchd/systemd race. After a global package upgrade, the CLI unloads the platform service and also checks the workspace lock for a detached/orphan `--daemon-only` process that the service manager no longer tracks. It terminates only a process whose PID, process start time, entrypoint, canonical workspace, canonical state root, and daemon-only arguments all match, waits up to 15 seconds for the PID and lock to disappear, and then starts the installed version in the foreground. A genuine foreground or unverifiable conflict is left untouched and reported with actionable guidance. To run only in the background, use `machine-mcp service start`; inspect its owner-only logs under `~/.local/state/machine-bridge-mcp/logs/`.
 
-Recommended upgrade sequence:
+Recommended upgrade sequence: repeat the package-free temporary-directory installation above, then run:
 
 ```sh
-npm install -g --omit=optional --allow-scripts=esbuild,workerd,sharp,fsevents machine-bridge-mcp@latest
 machine-mcp --verbose
 ```
 
-The global install replaces files on disk but cannot hot-reload an already running Node process; the second command performs the bounded service and orphan-daemon takeover. Use the command exactly as shown: `--omit=optional` prevents the development-only optional `fsevents` package from producing a blocked-install-script warning. If takeover fails, run `machine-mcp service status` first; it reports the platform service and workspace daemon separately. Then run `machine-mcp service stop` and retry.
+The global install replaces files on disk but cannot hot-reload an already running Node process; the foreground command performs the bounded service and orphan-daemon takeover. Keep `--omit=optional` in the installation step to prevent the development-only optional `fsevents` package from producing a blocked-install-script warning. If takeover fails, run `machine-mcp service status` first; it reports the platform service and workspace daemon separately. Then run `machine-mcp service stop` and retry.
 
 Use the printed values in the MCP client:
 
@@ -97,7 +124,7 @@ MCP Server URL: https://<worker>.<account>.workers.dev/mcp
 MCP connection password: mcp_password_...
 ```
 
-The remote authorization flow uses an authorization code, PKCE S256, exact redirect/resource binding, expiring access tokens stored as hashes, and a token-version value for bulk revocation.
+The remote authorization flow uses an authorization code, PKCE S256, exact redirect/resource binding, expiring access tokens stored as hashes, and a token-version value for bulk revocation. After password approval, the Worker constructs the registered callback with the URL API and returns `303 See Other`, so OAuth response parameters are encoded and the browser performs a GET to the callback.
 
 ## Optional local stdio MCP
 
@@ -159,7 +186,7 @@ Under canonical `full`, Machine Bridge also exposes structured local automation:
 
 - installed application discovery/opening and macOS Accessibility inspection/actions;
 - a packaged Chromium extension that controls the user's existing daily browser profile, active tabs, login state, and windows;
-- current DOM source and frame/open-Shadow-DOM inspection, structured page actions, complex multi-field forms, resource-backed secret fields, resource-backed file uploads, and screenshots.
+- current DOM source and frame/open-Shadow-DOM inspection, stable semantic element references, actionability waits, fixed trusted mouse/keyboard input, explicit browser waits, tab management, complex multi-field forms, resource-backed secret fields, resource-backed file uploads, and screenshots.
 
 One-time browser setup:
 
@@ -168,7 +195,7 @@ machine-mcp browser setup
 machine-mcp browser status
 ```
 
-Load the printed unpacked-extension directory once in Chrome, Edge, Brave, Vivaldi, or another compatible Chromium browser. The extension badge reports connection state, and clicking it opens the saved local pairing page. The local pairing token remains in owner-only state and the loopback pairing page; it is not returned through MCP. For a mass-market release, distribute the same extension as a signed browser-store build rather than asking end users to enable Developer mode. See [Local application and browser automation](docs/LOCAL_AUTOMATION.md).
+Load the printed unpacked-extension directory once in Chrome, Edge, Brave, Vivaldi, or another compatible Chromium browser. After an upgrade that adds permissions, reload the unpacked extension and accept the browser prompt; trusted input uses the Chromium `debugger` permission only for fixed, short-lived DevTools Input commands. The extension badge reports connection state, and clicking it opens the saved local pairing page. The local pairing token remains in owner-only state and the loopback pairing page; it is not returned through MCP. For a mass-market release, distribute the same extension as a signed browser-store build rather than asking end users to enable Developer mode. See [Local application and browser automation](docs/LOCAL_AUTOMATION.md).
 
 Machine Bridge can discover, refresh, rank, and load capabilities automatically. The ChatGPT/MCP host still owns tool selection and approval, so the server cannot force a host to expose or invoke a recommended skill, command, app, or browser operation. Check `server_info.observability.capability_routing` or `project_overview.capabilityRouting` to distinguish “the host never called the resolver” from “the resolver ran but found no relevant capability.”
 
@@ -319,9 +346,11 @@ The exact `tools/list` response reflects the active local policy. Definitions co
 - `browser_status`
 - `pair_browser_extension`
 - `browser_list_tabs`
+- `browser_manage_tabs` — create, activate, or close tabs
 - `browser_get_source` — bounded current DOM HTML, including selected frames
-- `browser_inspect_page`
-- `browser_action`
+- `browser_inspect_page` — semantic controls, stable per-document refs, state, and geometry
+- `browser_wait` — bounded URL/load/text/element-state waits
+- `browser_action` — actionability checks plus `auto`, `trusted`, or `dom` input mode
 - `browser_fill_form`
 - `browser_upload_files` — registered local resources to file inputs
 - `browser_screenshot`

package/SECURITY.md

@@ -71,11 +71,11 @@ The global `model_instructions_file` is an explicit user trust decision. It is r
 
 ## Browser and application automation
 
-Canonical `full` exposes structured browser and desktop UI authority. The browser extension operates the user's existing Chromium profile, including active login state, and has broad host access so it can inspect arbitrary pages and fill complex forms. An authorized client may therefore read page content, click controls, submit forms, upload registered files, and cause transactions with the user's browser identity. macOS Accessibility actions similarly operate applications with the local user's UI authority.
+Canonical `full` exposes structured browser and desktop UI authority. The browser extension operates the user's existing Chromium profile, including active login state, and has broad host access plus Chromium `debugger` permission so it can inspect arbitrary pages, fill complex forms, and issue trusted input. An authorized client may therefore read page content, click controls, submit forms, upload registered files, and cause transactions with the user's browser identity. macOS Accessibility actions similarly operate applications with the local user's UI authority.
 
 The browser loopback broker validates loopback `Host`, requires a random owner-only bearer subprotocol, accepts extension sockets only with a `chrome-extension://` origin, and authenticates additional local runtimes separately. Initial extension pairing is trust-on-first-use; after pairing, a different localhost page cannot replace the stored broker unless the user clicks the extension action while the genuine pairing page is active. The pairing token is never returned by MCP or written to operational logs. These controls defend against casual cross-origin and DNS-rebinding access; they do not protect against malicious code already running as the same OS user or a compromised browser extension profile.
 
-Caller-supplied JavaScript, AppleScript, and JXA source are deliberately unsupported. Actions use fixed implementation code and structured selectors. This removes an avoidable arbitrary-evaluation surface but does not make DOM or Accessibility actions safe. Pages can contain prompt injection, misleading labels, hidden consequences, cross-origin frames unavailable to inspection, or state that changes between inspection and submission. High-impact account, financial, legal, medical, publishing, deletion, and purchase actions require contextual review and any host/user confirmation the client provides.
+Caller-supplied JavaScript, AppleScript, JXA source, and arbitrary DevTools methods are deliberately unsupported. Actions use fixed implementation code and structured selectors. The debugger adapter is restricted to fixed `Input` commands, attaches only around one trusted action, and detaches in `finally`; `input_mode` makes fallback behavior explicit. The broker validates a versioned extension capability hello before activating a connection and does not let an invalid replacement candidate displace an already compatible extension. This removes an avoidable arbitrary-evaluation surface but does not make DOM or Accessibility actions safe. Pages can contain prompt injection, misleading labels, hidden consequences, cross-origin frames unavailable to inspection, or state that changes between inspection and submission. High-impact account, financial, legal, medical, publishing, deletion, and purchase actions require contextual review and any host/user confirmation the client provides.
 
 Text and file resources can be injected locally so their contents do not appear in MCP arguments or results. The destination page/application still receives the value, and remote tool metadata/results still traverse the Worker and host. Page source and screenshots can themselves contain secrets. Browser/app tool arguments, source, screenshots, field values, and results are intentionally omitted from operational logs.
 
@@ -145,7 +145,7 @@ Runner diagnostic logs are owner-only and do not receive child stdout/stderr. St
 
 ## OAuth and public endpoints
 
-Remote mode uses authorization code flow with PKCE S256, exact redirect/resource/client binding, expiring authorization codes and access tokens, hashed token storage, token-version revocation, and bounded dynamic client registration.
+Remote mode uses authorization code flow with PKCE S256, exact redirect/resource/client binding, expiring authorization codes and access tokens, hashed token storage, token-version revocation, and bounded dynamic client registration. Successful consent constructs the registered callback through the URL API and returns `303 See Other`; response parameters are encoded rather than concatenated into an unchecked header string.
 
 The authorization page displays the validated client name and redirect URI. Enter the connection password only after initiating the connection and recognizing both values.
 

package/browser-extension/devtools-input.js

@@ -0,0 +1,136 @@
+(() => {
+  if (globalThis.__machineBridgeDevtoolsInput) return;
+
+  const MODIFIERS = Object.freeze({ Alt: 1, Control: 2, Meta: 4, Shift: 8 });
+  const tabQueues = new Map();
+  const KEY_DATA = Object.freeze({
+    Enter: { code: "Enter", virtualKeyCode: 13 },
+    Tab: { code: "Tab", virtualKeyCode: 9 },
+    Escape: { code: "Escape", virtualKeyCode: 27 },
+    Backspace: { code: "Backspace", virtualKeyCode: 8 },
+    Delete: { code: "Delete", virtualKeyCode: 46 },
+    ArrowLeft: { code: "ArrowLeft", virtualKeyCode: 37 },
+    ArrowUp: { code: "ArrowUp", virtualKeyCode: 38 },
+    ArrowRight: { code: "ArrowRight", virtualKeyCode: 39 },
+    ArrowDown: { code: "ArrowDown", virtualKeyCode: 40 },
+    Home: { code: "Home", virtualKeyCode: 36 },
+    End: { code: "End", virtualKeyCode: 35 },
+    PageUp: { code: "PageUp", virtualKeyCode: 33 },
+    PageDown: { code: "PageDown", virtualKeyCode: 34 },
+    Space: { code: "Space", virtualKeyCode: 32, key: " " },
+  });
+
+  async function perform(tabId, action, details = {}) {
+    if (!Number.isInteger(tabId) || tabId < 1) throw new Error("trusted input requires a valid tab");
+    const previous = tabQueues.get(tabId) || Promise.resolve();
+    const current = previous.catch(() => {}).then(() => withDebugger(tabId, async (send) => {
+      if (action === "click") return mouseClick(send, details.point, 1);
+      if (action === "double_click") return mouseClick(send, details.point, 2);
+      if (action === "hover") return mouseMove(send, details.point);
+      if (action === "press") return pressKey(send, details.key || "Enter");
+      if (action === "type_text") return send("Input.insertText", { text: String(details.text ?? "") });
+      throw new Error(`trusted input does not support '${action}'`);
+    }));
+    tabQueues.set(tabId, current);
+    try { return await current; }
+    finally { if (tabQueues.get(tabId) === current) tabQueues.delete(tabId); }
+  }
+
+  async function withDebugger(tabId, operation) {
+    const target = { tabId };
+    let attached = false;
+    try {
+      await chrome.debugger.attach(target, "1.3");
+      attached = true;
+      const send = (method, params = {}) => chrome.debugger.sendCommand(target, method, params);
+      return await operation(send);
+    } catch (error) {
+      throw new Error(`trusted browser input unavailable: ${cleanError(error)}`);
+    } finally {
+      if (attached) {
+        try { await chrome.debugger.detach(target); } catch {}
+      }
+    }
+  }
+
+  async function mouseMove(send, point) {
+    const { x, y } = normalizePoint(point);
+    await send("Input.dispatchMouseEvent", { type: "mouseMoved", x, y, button: "none" });
+  }
+
+  async function mouseClick(send, point, clickCount) {
+    const { x, y } = normalizePoint(point);
+    await mouseMove(send, { x, y });
+    for (let count = 1; count <= clickCount; count += 1) {
+      await send("Input.dispatchMouseEvent", { type: "mousePressed", x, y, button: "left", buttons: 1, clickCount: count });
+      await send("Input.dispatchMouseEvent", { type: "mouseReleased", x, y, button: "left", buttons: 0, clickCount: count });
+    }
+  }
+
+  async function pressKey(send, shortcut) {
+    const parsed = parseShortcut(shortcut);
+    const down = {
+      type: parsed.text ? "keyDown" : "rawKeyDown",
+      key: parsed.key,
+      code: parsed.code,
+      modifiers: parsed.modifiers,
+      windowsVirtualKeyCode: parsed.virtualKeyCode,
+      nativeVirtualKeyCode: parsed.virtualKeyCode,
+      ...(parsed.text ? { text: parsed.text, unmodifiedText: parsed.text } : {}),
+    };
+    await send("Input.dispatchKeyEvent", down);
+    await send("Input.dispatchKeyEvent", {
+      type: "keyUp",
+      key: parsed.key,
+      code: parsed.code,
+      modifiers: parsed.modifiers,
+      windowsVirtualKeyCode: parsed.virtualKeyCode,
+      nativeVirtualKeyCode: parsed.virtualKeyCode,
+    });
+  }
+
+  function parseShortcut(value) {
+    const parts = String(value || "Enter").split("+").map((part) => part.trim()).filter(Boolean);
+    const keyName = parts.pop() || "Enter";
+    let modifiers = 0;
+    for (const raw of parts) {
+      const name = raw === "Ctrl" ? "Control" : raw === "Cmd" || raw === "Command" ? "Meta" : raw;
+      if (!(name in MODIFIERS)) throw new Error(`unsupported key modifier: ${raw}`);
+      modifiers |= MODIFIERS[name];
+    }
+    const known = KEY_DATA[keyName];
+    if (known) return {
+      key: known.key || keyName,
+      code: known.code,
+      virtualKeyCode: known.virtualKeyCode,
+      modifiers,
+      text: "",
+    };
+    if ([...keyName].length !== 1) throw new Error(`unsupported key: ${keyName}`);
+    const character = keyName;
+    const upper = character.toUpperCase();
+    return {
+      key: character,
+      code: /[A-Za-z]/.test(character) ? `Key${upper}` : "",
+      virtualKeyCode: upper.codePointAt(0),
+      modifiers,
+      text: modifiers & (MODIFIERS.Control | MODIFIERS.Meta | MODIFIERS.Alt) ? "" : character,
+    };
+  }
+
+  function normalizePoint(point) {
+    const x = Number(point?.x);
+    const y = Number(point?.y);
+    if (!Number.isFinite(x) || !Number.isFinite(y) || x < 0 || y < 0) throw new Error("trusted input target has no usable viewport point");
+    return { x, y };
+  }
+
+  function cleanError(error) {
+    return String(error?.message || error || "unknown error").replace(/\s+/g, " ").slice(0, 500);
+  }
+
+  Object.defineProperty(globalThis, "__machineBridgeDevtoolsInput", {
+    value: Object.freeze({ perform }),
+    configurable: false,
+  });
+})();

package/browser-extension/manifest.json

@@ -1,10 +1,11 @@
 {
   "manifest_version": 3,
   "name": "Machine Bridge Browser",
-  "version": "0.13.0",
+  "version": "0.14.0",
   "description": "Connects the current Chromium browser profile to the local Machine Bridge runtime for user-authorized automation.",
   "permissions": [
     "tabs",
+    "debugger",
     "scripting",
     "storage",
     "alarms"
@@ -29,5 +30,5 @@
   "action": {
     "default_title": "Machine Bridge Browser"
   },
-  "version_name": "0.13.0"
+  "version_name": "0.14.0"
 }