machine-bridge-mcp 0.12.0 → 0.12.1

package/CHANGELOG.md

@@ -1,5 +1,18 @@
 # Changelog
 
+## 0.12.1 - 2026-07-12
+
+### Real local automation corrections
+
+- Make the fixed macOS JXA helper return JSON as the top-level `osascript` result instead of using `console.log`, which real `osascript` sends to stderr. Empty helper stdout now fails explicitly rather than being interpreted as a successful empty object.
+- Skip recursive macOS menu-bar/menu traversal by default so bounded inspection reaches main-window controls promptly. Add `include_menus` to inspection and actions for explicit menu automation.
+- Normalize hyphens, underscores, dots, and spaces during capability ranking, and strongly boost explicitly named skills and commands. Natural-language references such as `agents progressive disclosure` now select `agents-progressive-disclosure` instead of a generic competing skill.
+
+### Live verification and regression coverage
+
+- Add regression coverage for missing JXA output, menu-recursion forwarding, and punctuation-normalized Skill selection. Add an opt-in real macOS Calculator smoke path and document isolated-profile MV3 browser validation.
+- Validate the source runtime with a temporary real global instruction file, an existing user Skill, a registered direct-argv command, shell execution, Calculator discovery/open/activation/inspection/click, and a real unpacked browser extension controlling a local form through inspect/fill/click/source/screenshot operations. Test files, browser profiles, temporary global configuration, tabs, and application state are removed afterward.
+
 ## 0.12.0 - 2026-07-12
 
 ### Locking, persistence, and process lifecycle

package/browser-extension/manifest.json

@@ -1,7 +1,7 @@
 {
   "manifest_version": 3,
   "name": "Machine Bridge Browser",
-  "version": "0.12.0",
+  "version": "0.12.1",
   "description": "Connects the current Chromium browser profile to the local Machine Bridge runtime for user-authorized automation.",
   "permissions": [
     "tabs",
@@ -29,5 +29,5 @@
   "action": {
     "default_title": "Machine Bridge Browser"
   },
-  "version_name": "0.12.0"
+  "version_name": "0.12.1"
 }

package/docs/AGENT_CONTEXT.md

@@ -189,7 +189,7 @@ No persistent skill or project-context index is trusted as authoritative. `sessi
 
 ## Progressive disclosure and task selection
 
-`agent_context` returns bounded skill metadata. `load_local_skill` returns full instructions only for one selected bundle. `resolve_task_capabilities` tokenizes the current task, ranks skill names/descriptions and command names/descriptions/argv, returns matches with scores, and loads the leading skill only when its relevance threshold is met.
+`agent_context` returns bounded skill metadata. `load_local_skill` returns full instructions only for one selected bundle. `resolve_task_capabilities` tokenizes the current task, ranks skill names/descriptions and command names/descriptions/argv, returns matches with scores, and loads the leading skill only when its relevance threshold is met. Hyphens, underscores, dots, and whitespace are normalized for matching, while an explicitly named skill or registered command receives a strong deterministic boost. Thus a task that says “agents progressive disclosure” can match `agents-progressive-disclosure` without relying on generic description words.
 
 This ranking is deterministic local assistance, not semantic certainty. The model must still evaluate whether the selected skill applies. Machine Bridge does not execute skill scripts implicitly and does not fabricate a dynamically named MCP tool per skill.
 

package/docs/LOCAL_AUTOMATION.md

@@ -83,7 +83,9 @@ Then use `value_resource` for a text field or `resources` for `browser_upload_fi
 - `inspect_local_application` returns a bounded macOS Accessibility tree.
 - `operate_local_application` performs a structured Accessibility action.
 
-macOS UI inspection and actions require Accessibility permission for the Node/Machine Bridge process. Application discovery and opening are available on supported desktop platforms; structured UI inspection currently targets macOS. The application backend uses fixed JXA implementation code. The caller selects only an application, structured selector, action, and optional bounded value; NUL-containing action text is rejected. `include_values` never returns values for secure-text roles or controls whose metadata indicates passwords, tokens, one-time codes, or payment-card secrets.
+macOS UI inspection and actions require Accessibility permission for the Node/Machine Bridge process. Application discovery and opening are available on supported desktop platforms; structured UI inspection currently targets macOS. The application backend uses fixed JXA implementation code and requires a non-empty JSON result from `osascript`; missing or malformed helper output fails explicitly. The caller selects only an application, structured selector, action, and optional bounded value; NUL-containing action text is rejected. `include_values` never returns values for secure-text roles or controls whose metadata indicates passwords, tokens, one-time codes, or payment-card secrets.
+
+Menu-bar and menu subtrees are not recursively expanded by default. This keeps main-window controls within the element/time budget for applications with large localized menu hierarchies. Pass `include_menus: true` to `inspect_local_application` or `operate_local_application` only when the target is a menu item.
 
 ## Capability discovery and automatic selection
 

package/docs/TESTING.md

@@ -49,6 +49,18 @@ The suite includes:
 - live stdio MCP initialization with session instructions, capability resolution, discovery, calls, rich content, sessions, cancellation, managed-job acceptance, and a detached job/finally phase that survives stdio shutdown;
 - live local Worker OAuth registration, consent, PKCE, token replay rejection, throttling, CORS, protocol negotiation, daemon-backed session bootstrap, dynamic tool advertisement, rich content, daemon replacement, cancellation, malformed daemon JSON/non-object rejection, duplicate hello rejection, and unknown-message closure.
 
+## Opt-in live desktop and browser validation
+
+The normal suite uses deterministic mocks for macOS Accessibility and an authenticated in-process extension peer for browser routing. Before a release that changes local UI automation, run the macOS live smoke test on a machine where the invoking Node/terminal process has Accessibility permission:
+
+```sh
+npm run app-automation:live-test
+```
+
+It opens Calculator, activates it through the fixed JXA helper, verifies structured JSON output, inspects main-window Accessibility controls with menu recursion disabled, clicks the `One` control, and quits the application. It is intentionally excluded from CI because macOS TCC permission and a graphical login session are host state.
+
+For browser changes, perform an isolated-profile smoke test with the packaged unpacked extension: pair it to a temporary source runtime, navigate a local no-store page, inspect controls, fill multiple fields, click a button, verify the live DOM, and capture a screenshot. A Playwright persistent Chromium context is an acceptable harness because it loads the real MV3 service worker while keeping the user's daily browser profile untouched.
+
 ## Additional release checks
 
 ```sh

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "machine-bridge-mcp",
-  "version": "0.12.0",
+  "version": "0.12.1",
   "description": "Cross-client MCP bridge for local agent context, structured browser and application automation, files, Git, processes, resources, and durable jobs over stdio or OAuth relay.",
   "type": "module",
   "license": "MIT",
@@ -69,7 +69,8 @@
     "browser-bridge:test": "node tests/browser-bridge-test.mjs",
     "app-automation:test": "node tests/app-automation-test.mjs",
     "process-lock:test": "node tests/process-lock-test.mjs",
-    "service-lifecycle:test": "node tests/service-lifecycle-test.mjs"
+    "service-lifecycle:test": "node tests/service-lifecycle-test.mjs",
+    "app-automation:live-test": "node tests/app-automation-test.mjs --live-macos"
   },
   "dependencies": {
     "wrangler": "4.110.0",

package/src/local/agent-context.mjs

@@ -112,12 +112,12 @@ export class AgentContextManager {
     const discovered = await this.discoverSkills(state, { maxResults: MAX_SKILL_RESULTS }, context);
     const maxSkills = clampInt(args.max_skills, 10, 1, 50);
     const skillMatches = discovered.skills
-      .map((skill) => ({ skill, score: relevanceScore(task, `${skill.name} ${skill.description}`) }))
+      .map((skill) => ({ skill, score: relevanceScore(task, `${skill.name} ${skill.description}`, skill.name) }))
       .filter((item) => item.score > 0)
       .sort((a, b) => b.score - a.score || a.skill.name.localeCompare(b.skill.name))
       .slice(0, maxSkills);
     const commandMatches = [...state.commands.values()]
-      .map((command) => ({ command, score: relevanceScore(task, `${command.name} ${command.description} ${command.argv.join(" ")}`) }))
+      .map((command) => ({ command, score: relevanceScore(task, `${command.name} ${command.description} ${command.argv.join(" ")}`, command.name) }))
       .filter((item) => item.score > 0)
       .sort((a, b) => b.score - a.score || a.command.name.localeCompare(b.command.name))
       .slice(0, 20);
@@ -718,7 +718,7 @@ function capabilityFingerprint(state, skills) {
   }));
 }
 
-function relevanceScore(task, candidate) {
+function relevanceScore(task, candidate, identity = "") {
   const taskTokens = tokenize(task);
   const candidateTokens = tokenize(candidate);
   if (!taskTokens.size || !candidateTokens.size) return 0;
@@ -726,18 +726,31 @@ function relevanceScore(task, candidate) {
   for (const token of taskTokens) {
     if (candidateTokens.has(token)) score += token.length >= 6 ? 2 : 1;
   }
-  const taskLower = task.toLowerCase();
-  const candidateLower = candidate.toLowerCase();
-  if (candidateLower.includes(taskLower) || taskLower.includes(candidateLower)) score += 4;
+  const taskComparable = comparableText(task);
+  const candidateComparable = comparableText(candidate);
+  const identityComparable = comparableText(identity);
+  if (candidateComparable.includes(taskComparable) || taskComparable.includes(candidateComparable)) score += 4;
+  if (identityComparable.length >= 3 && ` ${taskComparable} `.includes(` ${identityComparable} `)) score += 12;
   return score;
 }
 
+function comparableText(value) {
+  return String(value || "")
+    .toLowerCase()
+    .replace(/[^a-z0-9\p{Script=Han}]+/gu, " ")
+    .trim()
+    .replace(/\s+/g, " ");
+}
+
 function tokenize(value) {
   const text = String(value || "").toLowerCase();
   const tokens = new Set();
   for (const raw of text.match(/[a-z0-9_][a-z0-9_.-]{1,}/g) || []) {
     const token = raw.replace(/^[.-]+|[.-]+$/g, "");
     if (token.length >= 2 && !TOKEN_STOP_WORDS.has(token)) tokens.add(token);
+    for (const part of token.split(/[._-]+/)) {
+      if (part.length >= 2 && !TOKEN_STOP_WORDS.has(part)) tokens.add(part);
+    }
   }
   for (const sequence of text.match(/[\p{Script=Han}]{1,}/gu) || []) {
     if (!TOKEN_STOP_WORDS.has(sequence)) tokens.add(sequence);

package/src/local/app-automation.mjs

@@ -109,6 +109,7 @@ export class AppAutomationManager {
       maxDepth: clampInt(args.max_depth, 6, 1, MAX_UI_DEPTH),
       maxElements: clampInt(args.max_elements, 200, 1, MAX_UI_ELEMENTS),
       includeValues: args.include_values === true,
+      includeMenus: args.include_menus === true,
     };
     const result = await this.runJxa(payload, clampInt(args.timeout_seconds, 30, 1, 120), context);
     return {
@@ -143,6 +144,7 @@ export class AppAutomationManager {
       value,
       maxDepth: clampInt(args.max_depth, 8, 1, MAX_UI_DEPTH),
       maxElements: MAX_UI_ELEMENTS,
+      includeMenus: args.include_menus === true,
     };
     const result = await this.runJxa(payload, clampInt(args.timeout_seconds, 30, 1, 120), context);
     return {
@@ -168,9 +170,11 @@ export class AppAutomationManager {
       undefined,
       input,
     );
+    const output = result.stdout.trim();
+    if (!output) throw new Error("macOS accessibility helper returned no JSON output");
     let parsed;
     try {
-      parsed = JSON.parse(result.stdout.trim() || "{}");
+      parsed = JSON.parse(output);
     } catch {
       throw new Error("macOS accessibility helper returned invalid JSON");
     }
@@ -337,7 +341,7 @@ function describe(element, index, includeValues) {
   return item;
 }
 function childrenOf(element) { return safe(() => element.uiElements(), []); }
-function flatten(root, maxDepth, maxElements, includeValues) {
+function flatten(root, maxDepth, maxElements, includeValues, includeMenus) {
   const output = [];
   const elements = [];
   const stack = [{ element: root, depth: 0 }];
@@ -346,10 +350,12 @@ function flatten(root, maxDepth, maxElements, includeValues) {
     const children = childrenOf(current.element);
     for (let i = children.length - 1; i >= 0; i--) {
       const child = children[i];
+      const item = describe(child, output.length, includeValues);
       elements.push(child);
-      output.push(describe(child, output.length, includeValues));
+      output.push(item);
       if (output.length >= maxElements) break;
-      if (current.depth + 1 < maxDepth) stack.push({ element: child, depth: current.depth + 1 });
+      const isMenuTree = String(item.role || '').startsWith('AXMenu');
+      if (current.depth + 1 < maxDepth && (includeMenus || !isMenuTree)) stack.push({ element: child, depth: current.depth + 1 });
     }
   }
   return { output, elements, truncated: stack.length > 0 || output.length >= maxElements };
@@ -366,14 +372,14 @@ function main() {
   const process = se.applicationProcesses.byName(payload.application);
   if (!safe(() => process.exists(), false)) throw new Error('application process not found or Accessibility access denied');
   if (payload.operation === 'inspect') {
-    const flattened = flatten(process, payload.maxDepth, payload.maxElements, payload.includeValues === true);
-    return { frontmost: safe(() => process.frontmost(), false), elements: flattened.output, truncated: flattened.truncated };
+    const flattened = flatten(process, payload.maxDepth, payload.maxElements, payload.includeValues === true, payload.includeMenus === true);
+    return { frontmost: safe(() => process.frontmost(), false), elements: flattened.output, truncated: flattened.truncated, menus_included: payload.includeMenus === true };
   }
   if (payload.action === 'activate') {
     process.frontmost = true;
     return { ok: true, matched: 1 };
   }
-  const flattened = flatten(process, payload.maxDepth, payload.maxElements, true);
+  const flattened = flatten(process, payload.maxDepth, payload.maxElements, true, payload.includeMenus === true);
   const matchesList = [];
   for (let i = 0; i < flattened.output.length; i++) if (matches(flattened.output[i], payload.selector)) matchesList.push(i);
   const chosen = payload.selector.index !== undefined ? matchesList[payload.selector.index] : matchesList[0];
@@ -398,5 +404,5 @@ function main() {
   }
   return { ok: true, matched: matchesList.length, selected_index: chosen, element: describe(element, chosen, false) };
 }
-try { console.log(JSON.stringify(main())); } catch (error) { console.log(JSON.stringify({ error: String(error.message || error) })); }
+(() => { try { return JSON.stringify(main()); } catch (error) { return JSON.stringify({ error: String(error.message || error) }); } })()
 `;

package/src/shared/tool-catalog.json

@@ -200,6 +200,11 @@
           "type": "boolean",
           "default": false
         },
+        "include_menus": {
+          "type": "boolean",
+          "default": false,
+          "description": "Include and recursively inspect the application menu hierarchy. Disabled by default so main-window controls are reached quickly."
+        },
         "timeout_seconds": {
           "type": "integer",
           "minimum": 1,
@@ -286,6 +291,11 @@
           "type": "string",
           "pattern": "^[a-z][a-z0-9._-]{0,63}$"
         },
+        "include_menus": {
+          "type": "boolean",
+          "default": false,
+          "description": "Include menu-bar and menu subtrees when matching the action selector."
+        },
         "max_depth": {
           "type": "integer",
           "minimum": 1,

package/src/worker/index.ts

@@ -3,7 +3,7 @@ import toolCatalog from "../shared/tool-catalog.json";
 import serverMetadata from "../shared/server-metadata.json";
 
 const SERVER_NAME = String(serverMetadata.name);
-const SERVER_VERSION = "0.12.0";
+const SERVER_VERSION = "0.12.1";
 const MCP_PROTOCOL_VERSION = String(serverMetadata.protocolVersion);
 const MCP_SUPPORTED_PROTOCOL_VERSIONS = serverMetadata.supportedProtocolVersions.map((value) => String(value));
 const JSONRPC_VERSION = "2.0";