machine-bridge-mcp 0.13.0 → 0.14.0

package/docs/LOCAL_AUTOMATION.md

@@ -13,15 +13,17 @@ Launching a separate automation browser is predictable but loses the user's ordi
 
 The extension works with the current profile and supports:
 
-- listing existing tabs;
+- listing, creating, activating, and closing tabs;
 - reading the current serialized DOM HTML from the main frame or accessible subframes;
-- discovering interactive controls, labels, roles, placeholders, and field metadata across accessible frames and open Shadow DOM roots;
-- structured navigation, click, focus, fill, select, check, uncheck, key press, and form submission;
+- discovering interactive controls, stable per-document element references, actionability state, geometry, labels, roles, placeholders, and field metadata across accessible frames and open Shadow DOM roots;
+- waiting for bounded combinations of URL, load state, page text, and element state;
+- structured navigation, click, double-click, hover, focus, fill, type-text, select, check, uncheck, key press, scroll-into-view, and form submission;
+- automatic existence/visibility/enabled/editable/stability/hit-target checks before actions, with fixed DevTools mouse/keyboard input for top-frame interactions and a controlled DOM fallback;
 - multi-field complex form filling in one operation, bounded to 200 fields and 4 MiB of aggregate text;
 - populating file inputs from registered local resource files;
 - visible-tab screenshots.
 
-Current support targets Chrome, Chromium, Microsoft Edge, Brave, Vivaldi, and compatible Chromium browsers. After upgrading Machine Bridge, reload the unpacked extension from the browser extensions page so its packaged scripts match the new runtime. Browser-internal pages, extension stores, some PDF/plugin viewers, cross-origin frames denied to the extension, and pages restricted by enterprise policy may not be scriptable.
+Current support targets Chrome, Chromium, Microsoft Edge, Brave, Vivaldi, and compatible Chromium browsers. After upgrading Machine Bridge, reload the unpacked extension from the browser extensions page so its packaged scripts and permissions match the new runtime. Version 0.14.0 adds the `debugger` permission for fixed trusted-input commands; the extension attaches only for the action and detaches afterward. The broker requires a versioned extension hello with the expected capability set, keeps the last compatible connection active until a replacement completes its handshake, and reports `extension_reload_required` instead of silently routing new tools to stale extension code. Browser-internal pages, extension stores, some PDF/plugin viewers, cross-origin frames denied to the extension, and pages restricted by enterprise policy may not be scriptable.
 
 ## One-time browser setup
 
@@ -56,14 +58,16 @@ The broker is machine-global rather than workspace-global. One local owner liste
 - `browser_status` reports broker role, extension connection, supported operations, pairing URL, and extension path without returning the pairing token.
 - `pair_browser_extension` opens the local pairing page and returns setup steps.
 - `browser_list_tabs` lists current tabs.
+- `browser_manage_tabs` creates, activates, or closes a selected tab.
 - `browser_get_source` returns bounded current DOM HTML for selected frames.
-- `browser_inspect_page` returns bounded interactive-element metadata.
-- `browser_action` performs one structured navigation or DOM action. Navigation accepts absolute `http`, `https`, or `file` URLs; script/data schemes are rejected.
+- `browser_inspect_page` returns a bounded semantic snapshot. Each control includes a stable `ref` for the lifetime of that document/frame plus visibility, enabled/editable state, and viewport geometry.
+- `browser_wait` waits until all supplied URL, load-state, page-text, and element-state conditions are true.
+- `browser_action` performs one structured navigation or page action. Navigation accepts absolute `http`, `https`, or `file` URLs; script/data schemes are rejected. Pointer and keyboard actions default to `input_mode: auto`, which attempts fixed trusted DevTools input in the top frame and falls back to DOM behavior. `trusted` forbids fallback; `dom` never attaches the debugger.
 - `browser_fill_form` fills up to 200 fields and can submit once.
 - `browser_upload_files` sets a file input from up to eight registered local resources. Caller filenames must be safe single-component names; derived names have controls and separators removed. MIME overrides must be canonical media types.
 - `browser_screenshot` returns native MCP image content.
 
-Selectors can use CSS, ID, field name, label text, visible text, ARIA/implicit role, placeholder, and a zero-based match index. The fixed page module traverses open Shadow DOM roots; closed shadow roots remain inaccessible. For frame-specific work, inspect all frames first and then pass `frame_id` to an action.
+Selectors can use the `ref` returned by inspection, CSS, ID, field name, label text, visible text, ARIA/implicit role, placeholder, and a zero-based match index. `ref` cannot be combined with other fields and becomes stale after navigation or element replacement. Non-ref selectors that match multiple controls fail explicitly unless `index` disambiguates them. The fixed page module traverses open Shadow DOM roots; closed shadow roots remain inaccessible. For frame-specific work, inspect all frames first and then pass `frame_id` to an action. Trusted input currently targets the top frame because DevTools coordinates are top-level; use `input_mode: dom` for an explicitly selected subframe.
 
 ## Sensitive form values and files
 
@@ -100,13 +104,13 @@ The browser extension has broad page access because generic source inspection an
 The implementation reduces avoidable risk as follows:
 
 - all browser/app tools are `full`-only;
-- no arbitrary evaluation or caller-provided script source is accepted;
+- no arbitrary evaluation, caller-provided script source, or caller-selected DevTools method is accepted; trusted input exposes only fixed `Input.dispatchMouseEvent`, `Input.dispatchKeyEvent`, and `Input.insertText` sequences;
 - loopback HTTP validates `Host`; extension WebSockets require a random bearer subprotocol and a `chrome-extension://` origin;
 - runtime-to-broker connections require a separate authenticated subprotocol;
 - the pairing token is stored owner-only, embedded only in the non-cacheable local pairing page, and omitted from MCP results and logs;
 - an established extension pairing cannot be silently replaced by another localhost page; replacement requires clicking the extension action on the active pairing page;
 - arguments, form values, page source, screenshots, and results are not operational log data;
-- message, source, form-field, upload, traversal, concurrency, proxy-route, and timeout limits are enforced;
+- message, source, form-field, upload, traversal, concurrency, proxy-route, actionability, and timeout limits are enforced; ambiguous selectors, stale refs, hidden/disabled/edit-blocked controls, moving targets, and obscured pointer targets fail explicitly;
 - MCP cancellation clears local and broker pending state and propagates a cancellation signal to the extension; an action already delivered to a page or application may still have completed;
 - resource-backed text and files are not returned after injection.
 

package/docs/OPERATIONS.md

@@ -76,14 +76,21 @@ machine-mcp service status
 machine-mcp --verbose
 ```
 
-Recommended upgrade:
-
-```sh
-npm install -g --omit=optional --allow-scripts=esbuild,workerd,sharp,fsevents machine-bridge-mcp@latest
+For installation or upgrade, launch a pinned npm 12 CLI from an empty temporary directory. This avoids depending on the old npm version currently on `PATH` and prevents its `npx` bootstrap from validating unrelated nearby project metadata first. On Windows Command Prompt:
+
+```bat
+set "MBM_INSTALL_DIR=%TEMP%\machine-bridge-mcp-install-%RANDOM%-%RANDOM%"
+mkdir "%MBM_INSTALL_DIR%"
+pushd "%MBM_INSTALL_DIR%"
+npx --yes npm@12.0.1 install --global npm@12.0.1
+npx --yes npm@12.0.1 install --global --omit=optional --allow-scripts=esbuild,workerd,sharp,fsevents machine-bridge-mcp@latest
+popd
+rmdir /s /q "%MBM_INSTALL_DIR%"
+npm --version
 machine-mcp --verbose
 ```
 
-Keep `--omit=optional` in the install command. Without it npm may resolve optional `fsevents` and warn that its install script was not included in `allowScripts`; Machine Bridge does not require that development-time watcher at runtime.
+`Unknown cli config "--allow-scripts"` proves the package installation ran under npm 11 or older. `Invalid property "node"` or `Invalid property "devEngines.node"` means npm parsed a legacy `devEngines` object; inspect the npm debug log to identify its source rather than assuming it belongs to Machine Bridge. The published package declares both Node.js 26 and npm 12 in `engines`, and `machine-mcp doctor` checks both active versions. Keep `--omit=optional` in the install command. Without it npm may resolve optional `fsevents` and warn that its install script was not included in `allowScripts`; Machine Bridge does not require that development-time watcher at runtime.
 
 ## State-root safety and removal
 

package/docs/TESTING.md

@@ -48,11 +48,11 @@ The suite includes:
 - shared no-follow bounded-file reads for normal files, over-limit data, directories, and symbolic links;
 - CLI parsing, policy profiles, and client configuration boundaries;
 - live stdio MCP initialization with session instructions, capability resolution, discovery, calls, rich content, sessions, cancellation, managed-job acceptance, and a detached job/finally phase that survives stdio shutdown;
-- live local Worker OAuth registration, consent, PKCE, token replay rejection, throttling, CORS, protocol negotiation, daemon-backed session bootstrap, dynamic tool advertisement, rich content, daemon replacement, cancellation, malformed daemon JSON/non-object rejection, duplicate hello rejection, and unknown-message closure.
+- live local Worker OAuth registration, consent, URL-constructed `303` callbacks including the ChatGPT redirect URI and encoded state, PKCE, token replay rejection, throttling, CORS, protocol negotiation, daemon-backed session bootstrap, dynamic tool advertisement, rich content, daemon replacement, cancellation, malformed daemon JSON/non-object rejection, duplicate hello rejection, and unknown-message closure.
 
 ## Opt-in live desktop and browser validation
 
-The normal suite uses deterministic mocks for macOS Accessibility and an authenticated in-process extension peer for browser routing. Before a release that changes local UI automation, run the macOS live smoke test on a machine where the invoking Node/terminal process has Accessibility permission:
+The normal suite uses deterministic mocks for macOS Accessibility and an authenticated in-process extension peer for browser routing. Browser contract tests cover versioned extension handshakes, stale-extension rejection, compatible replacement without premature displacement, clean rejection of in-flight direct and proxied requests during replacement, keepalive handling, strict action/value validation, stable-ref validation, combined waits, tab commands, fixed trusted-input CDP sequences, attach/detach cleanup, semantic-ref stability, deterministic scrolling, obscured-target waiting, and stale-reference failure. Before a release that changes local UI automation, run the macOS live smoke test on a machine where the invoking Node/terminal process has Accessibility permission:
 
 ```sh
 npm run app-automation:live-test
@@ -60,7 +60,7 @@ npm run app-automation:live-test
 
 It opens Calculator, activates it through the fixed JXA helper, verifies structured JSON output, inspects main-window Accessibility controls with menu recursion disabled, clicks the `One` control, and quits the application. It is intentionally excluded from CI because macOS TCC permission and a graphical login session are host state.
 
-For browser changes, perform an isolated-profile smoke test with the packaged unpacked extension: pair it to a temporary source runtime, navigate a local no-store page, inspect controls, fill multiple fields, click a button, verify the live DOM, and capture a screenshot. A Playwright persistent Chromium context is an acceptable harness because it loads the real MV3 service worker while keeping the user's daily browser profile untouched.
+For browser changes, perform an isolated-profile smoke test with the packaged unpacked extension: pair it to a temporary source runtime, navigate a local no-store page, inspect controls and reuse a returned `ref`, wait for an element state, fill multiple fields, exercise `input_mode: trusted` for click/key/text, verify the live DOM, create/activate/close a tab, and capture a screenshot. A Playwright persistent Chromium context is an acceptable harness because it loads the real MV3 service worker while keeping the user's daily browser profile untouched.
 
 ## Additional release checks
 
@@ -97,7 +97,7 @@ Run `npm run privacy:check` before committing and before packaging. Run and revi
 
 ## Package manifest
 
-`npm run package:test` executes a real silent `npm pack --dry-run --json`, requires clean parseable JSON, rejects sensitive local artifacts and credential-like file classes, validates every packaged mode as `0644` or `0755`, and verifies that privacy/engineering guidance, runtime/relay/secure-file/lock/service/browser/app modules, all package-script helpers, the packaged browser extension, contribution discipline, and privacy/release-impact checkers are present. `npm run install:test` packs the real tarball, installs it into an isolated global prefix with the documented npm 12 options, rejects blocked-script warnings, confirms optional `fsevents` is absent, and runs the installed CLI.
+`npm run package:test` executes a real silent `npm pack --dry-run --json`, requires clean parseable JSON, rejects sensitive local artifacts and credential-like file classes, validates every packaged mode as `0644` or `0755`, and verifies that privacy/engineering guidance, runtime/relay/secure-file/lock/service/browser/app modules, all package-script helpers, the packaged browser extension, contribution discipline, and privacy/release-impact checkers are present. `npm run install:test` requires npm 12, installs the real tarball from a package-free directory into an isolated global prefix with the documented options, verifies the packaged npm engine requirement, rejects blocked-script warnings, confirms optional `fsevents` is absent, and runs the installed CLI. The test runs on Linux, macOS, and Windows CI.
 
 The stdio integration test also sends an oversized line, verifies bounded rejection, and confirms that the next valid request is still processed.
 

package/package.json

@@ -1,11 +1,12 @@
 {
   "name": "machine-bridge-mcp",
-  "version": "0.13.0",
+  "version": "0.14.0",
   "description": "Cross-client MCP bridge for local agent context, structured browser and application automation, files, Git, processes, resources, and durable jobs over stdio or OAuth relay.",
   "type": "module",
   "license": "MIT",
   "engines": {
-    "node": ">=26.0.0"
+    "node": ">=26.0.0",
+    "npm": ">=12.0.0"
   },
   "bin": {
     "machine-bridge-mcp": "bin/machine-mcp.mjs",
@@ -42,7 +43,7 @@
     "worker:types": "wrangler types src/worker/worker-configuration.d.ts",
     "typecheck": "npm run worker:types && tsc -p tsconfig.json --noEmit",
     "syntax": "node scripts/syntax-check.mjs",
-    "check": "npm run privacy:check && npm run privacy:test && npm run release-impact:check && npm run release-impact:test && npm run release-state:test && npm run release-ci:test && npm run network-retry:test && npm run relay:test && npm run secure-file:test && npm run architecture:test && npm run typecheck && npm run syntax && npm run process-lock:test && npm run service-lifecycle:test && npm run catalog:test && npm run agent-context:test && npm run browser-bridge:test && npm run app-automation:test && npm run self-test && npm run atomic-fs:test && npm run package:test && npm run ssh-key:test && npm run full-access:test && npm run managed-jobs:test && npm run stdio:integration-test && npm run worker:integration-test",
+    "check": "npm run privacy:check && npm run privacy:test && npm run release-impact:check && npm run release-impact:test && npm run release-state:test && npm run release-ci:test && npm run network-retry:test && npm run relay:test && npm run secure-file:test && npm run architecture:test && npm run typecheck && npm run syntax && npm run process-lock:test && npm run service-lifecycle:test && npm run catalog:test && npm run agent-context:test && npm run browser-command:test && npm run browser-devtools-input:test && npm run browser-page-automation:test && npm run browser-bridge:test && npm run app-automation:test && npm run self-test && npm run atomic-fs:test && npm run package:test && npm run ssh-key:test && npm run full-access:test && npm run managed-jobs:test && npm run stdio:integration-test && npm run worker:integration-test",
     "worker:dry-run": "wrangler deploy --dry-run",
     "worker:integration-test": "node tests/worker-integration-test.mjs",
     "release:check": "node scripts/github-release.mjs --check",
@@ -72,7 +73,10 @@
     "app-automation:test": "node tests/app-automation-test.mjs",
     "process-lock:test": "node tests/process-lock-test.mjs",
     "service-lifecycle:test": "node tests/service-lifecycle-test.mjs",
-    "app-automation:live-test": "node tests/app-automation-test.mjs --live-macos"
+    "app-automation:live-test": "node tests/app-automation-test.mjs --live-macos",
+    "browser-command:test": "node tests/browser-command-test.mjs",
+    "browser-devtools-input:test": "node tests/browser-devtools-input-test.mjs",
+    "browser-page-automation:test": "node tests/browser-page-automation-test.mjs"
   },
   "dependencies": {
     "https-proxy-agent": "9.1.0",

package/src/local/agent-context.mjs

@@ -825,7 +825,7 @@ function recommendTools(task, { commandsAvailable, commandRelevant, skillRelevan
   const tools = ["agent_context"];
   if (skillRelevant) tools.push("load_local_skill");
   if (commandRelevant) tools.push("run_local_command");
-  if (/browser|chrome|edge|brave|网页|浏览器|表单|网站/.test(lower)) tools.push("browser_status", "browser_list_tabs", "browser_inspect_page", "browser_action", "browser_fill_form");
+  if (/browser|chrome|edge|brave|网页|浏览器|表单|网站/.test(lower)) tools.push("browser_status", "browser_list_tabs", "browser_manage_tabs", "browser_inspect_page", "browser_wait", "browser_action", "browser_fill_form");
   if (/app|application|gui|window|应用|软件|窗口|界面/.test(lower)) tools.push("list_local_applications", "inspect_local_application", "operate_local_application");
   if (/git|commit|branch|diff|仓库|提交|分支/.test(lower)) tools.push("git_status", "git_diff");
   if (/test|build|lint|command|terminal|测试|构建|命令|终端/.test(lower)) tools.push(commandsAvailable ? "run_local_command" : "run_process");