moondesk 0.12.0 → 0.13.0

package/README.md

@@ -120,9 +120,11 @@ Select the connector and start working.
 
 ## Browser control
 
-MoonDesk owns one lazy agent Chromium process instead of attaching to your personal browser profile or launching a separate browser for every project. Inside that shared Chromium, each registered workspace gets its own isolated BrowserContext for cookies and site storage, while each ChatGPT conversation gets its own MoonDesk-routed logical tab set. Different workspaces therefore do not share cookies, localStorage, IndexedDB, or service-worker state, and separate conversations cannot accidentally act on each other's selected tabs.
+MoonDesk owns one lazy managed Chromium process instead of attaching to your personal browser profile or launching a separate browser for every project. On first browser use it provisions a pinned Chrome for Testing build for the current platform, verifies the exact archive size and SHA-256, records a verified inventory of the installed browser files including Unix permission bits where applicable, installs it atomically into MoonDesk's browser cache, and controls it directly from Rust over the Chrome DevTools Protocol (CDP)—no Playwright, `chrome-devtools-mcp`, or separate browser-control service is required. Later starts revalidate that inventory so a missing or damaged support file triggers reprovisioning instead of a persistent broken-browser loop. Inside that shared Chromium, each registered workspace gets its own isolated BrowserContext for cookies and site storage, while each ChatGPT conversation gets its own MoonDesk-routed logical tab set. Different workspaces therefore do not share cookies, localStorage, IndexedDB, or service-worker state, and separate conversations cannot accidentally act on each other's selected tabs.
 
-The browser runs headless by default and can be switched to visible mode for human-assisted steps such as logins or permission prompts. Presentation belongs to the shared Chromium process, so changing it while the browser is live requires explicit confirmation because every workspace BrowserContext and logical tab is recreated. MoonDesk never attaches to or reuses your personal browser profile, cookies, or logged-in sessions.
+The browser runs headless by default and can be switched to visible mode for human-assisted steps such as logins or permission prompts. Headless and visible are two presentations of the same agent browser and expose the same page-local tabs, navigation, DOM controls, visual computer-use controls, viewport management, screenshots, console/network inspection, and heap-snapshot capabilities. Browser-global performance tracing has stricter isolation preconditions and is rejected whenever another managed BrowserContext is active, another logical browser session has used the workspace BrowserContext in the current runtime generation, a managed page is not owned by the requester, or an unowned/default-context page could contribute unrelated trace data. Presentation belongs to the shared Chromium process, so changing it while the browser is live requires explicit confirmation because every workspace BrowserContext and logical tab is recreated. MoonDesk never attaches to or reuses your personal browser profile, cookies, or logged-in sessions.
+
+For normal agent work MoonDesk exposes a Codex-style capability facade: `browser_state` for ambient state, `browser_tabs` for conversation-owned tabs, `browser_navigate` for goto/back/forward/reload, `browser_dom` for accessibility/DOM interactions, `browser_cua` for rendered screenshots plus physical coordinate/keyboard/wheel input, and `browser_viewport` for responsive/device sizing. `browser_command` remains available as an advanced native-CDP escape hatch for console/network inspection, emulation, performance traces, V8 heap snapshots, screenshots, and lower-level page operations that are not represented by the facade.
 
 ```bash
 moondesk browser navigate_page --url=http://localhost:3000
@@ -131,7 +133,7 @@ moondesk browser take_snapshot
 moondesk browser list_console_messages
 ```
 
-The `moondesk browser` CLI shares the resolved workspace's BrowserContext/login state but uses its own logical tab session, so scripted browser commands cannot steal a ChatGPT conversation's active page. Use `view_page` when the task depends on actual rendered pixels rather than only a text/accessibility snapshot.
+The `moondesk browser` CLI is the deterministic low-level scripting path. It shares the resolved workspace's BrowserContext/login state but uses its own logical tab session, so scripted browser commands cannot steal a ChatGPT conversation's active page. For interactive agent work prefer the capability facade; use `browser_cua action=screenshot` or `view_page` when the task depends on actual rendered pixels rather than only a text/accessibility snapshot.
 
 For browser-runtime invariants and implementation details, see [`docs/BROWSER_RUNTIME_ARCHITECTURE_HARDENING.md`](docs/BROWSER_RUNTIME_ARCHITECTURE_HARDENING.md).
 
@@ -172,7 +174,7 @@ Use read-only mode when mutation is unnecessary, and use a VM or container for u
 
 **Commands:** `run_command`, `start_command`, `list_commands`, `poll_command`, `read_command_output`, `cancel_command`
 
-**Browser:** `set_browser_presentation`, `browser_command`, `view_page`
+**Browser:** `browser_state`, `browser_tabs`, `browser_navigate`, `browser_dom`, `browser_cua`, `browser_viewport`, `set_browser_presentation`, `view_page`, plus advanced `browser_command`
 
 Use `run_command` for short work. Use `start_command` + `poll_command` for builds, tests, installs, dev servers, and other long-running jobs.
 
@@ -217,7 +219,7 @@ If Windows Security or another antivirus quarantines the verified executable, up
 | Tunnel | ngrok |
 | MCP server | Custom implementation |
 | MCP protocol | `2025-11-25` |
-| Browser runtime | pinned `chrome-devtools-mcp@1.7.0` |
+| Browser runtime | MoonDesk-managed Chrome for Testing + native Rust CDP |
 | Distribution | npm + verified native binaries |
 
 ## Contributing

package/npm/moondesk.js

@@ -128,7 +128,7 @@ async function orchestrate(options = {}) {
   const nodeVersion = options.nodeVersion ?? process.versions.node;
   if (!isSupportedNodeVersion(nodeVersion)) {
     logger.error(
-      `MoonDesk requires Node.js ${SUPPORTED_NODE_RANGE} because the pinned browser runtime does not support Node.js ${nodeVersion}.`,
+      `MoonDesk requires Node.js ${SUPPORTED_NODE_RANGE} for its supported npm bootstrap and self-update runtime; detected Node.js ${nodeVersion}.`,
     );
     return { code: 1, signal: null };
   }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "moondesk",
-  "version": "0.12.0",
+  "version": "0.13.0",
   "description": "MoonDesk — use ChatGPT Chat as a local coding agent, by Shattermoon.",
   "author": "Shattermoon",
   "license": "MIT",

package/skills/browser/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: browser
-description: Control MoonDesk's shared Chromium runtime with workspace-isolated storage and conversation-owned tabs without exposing the full Chrome DevTools MCP schema to the model.
+description: Control MoonDesk's Codex-style agent browser with workspace-isolated storage, conversation-owned tabs, DOM actions, visual computer-use controls, viewport control, and advanced DevTools fallback.
 ---
 
 # MoonDesk Browser
@@ -9,37 +9,43 @@ Use MoonDesk's browser runtime for local web-app inspection, UI testing, console
 
 ## Interfaces
 
-- **MCP `set_browser_presentation`**: available in `multi-tools` mode for the rare case where the user needs the shared agent Chromium to become visible for human input. Prefer headless; changing presentation while Chromium is live restarts the host-wide browser runtime and therefore requires explicit user approval before retrying with `confirm_restart=true`.
-- **MCP `browser_command`**: preferred for one browser action at a time and the only page-action primitive needed in Browser-only mode.
-- **MCP `view_page`**: preferred whenever appearance matters. It returns the current rendered page as model-visible pixels.
-- **`moondesk browser` CLI**: preferred in Both mode when several deterministic browser actions are easier to express as a shell script or loop. It calls the same MoonDesk Chromium runtime and workspace BrowserContext, but it intentionally owns a separate local-CLI logical tab session instead of borrowing a ChatGPT conversation's active page.
+- **MCP `browser_state`**: read ambient browser state without launching Chromium: headless/visible presentation, runtime status, this conversation's logical tabs, selected tab, and available capability groups.
+- **MCP `browser_tabs`**: list/select/open/close conversation-owned logical tabs. Raw upstream Chrome page IDs and other conversations' tabs are never exposed.
+- **MCP `browser_navigate`**: goto/back/forward/reload on the selected tab.
+- **MCP `browser_dom`**: the normal DOM-oriented path for snapshots, UID interactions, forms, evaluation, waits, and uploads.
+- **MCP `browser_cua`**: the visual computer-use path for rendered screenshots, coordinate clicks, focused typing, keypresses, and scrolling. Prefer this for canvas, custom editors, maps, and other interfaces where DOM UIDs are unreliable.
+- **MCP `browser_viewport`**: inspect or set the selected tab's viewport, using exact emulation when DPR/mobile/touch/landscape characteristics are requested.
+- **MCP `set_browser_presentation`**: switch the same host-owned agent Chromium between headless and visible presentation. Prefer headless; changing presentation while Chromium is live restarts the host-wide browser runtime and therefore requires explicit user approval before retrying with `confirm_restart=true`.
+- **MCP `browser_command`**: advanced escape hatch for MoonDesk-native CDP operations that are not represented by the capability facade. Do not use it as the default interaction model.
+- **MCP `view_page`**: direct rendered-pixel helper retained for compatibility; `browser_cua action=screenshot` is the capability-facade equivalent.
+- **`moondesk browser` CLI**: deterministic low-level scripting interface in Both mode. It calls the same MoonDesk Chromium runtime and workspace BrowserContext, but it intentionally owns a separate local-CLI logical tab session instead of borrowing a ChatGPT conversation's active page.
 
-Do not invoke `npx chrome-devtools-mcp` directly. MoonDesk pins and manages the compatible Chrome DevTools runtime.
+MoonDesk provisions its own pinned Chrome for Testing build and controls it directly over CDP. Do not launch a second Playwright/CDP/MCP browser-control stack for normal MoonDesk browser work.
 
 ## Core workflow
 
-1. Navigate or select the target page.
-2. Set the target viewport before taking interaction UIDs. Use `resize_page` for normal desktop window sizes; use `emulate --viewport=<width>x<height>x<dpr>[,mobile][,touch]` for exact tablet/mobile responsive testing because Chromium may clamp very narrow desktop windows.
-3. After navigation or viewport emulation, run `take_snapshot`. Those operations can recreate the page context, so older UIDs may be stale.
-4. Use snapshot UIDs with `click`, `fill`, `hover`, `drag`, `upload_file`, etc.
-5. Take another snapshot after a state-changing action when the page structure may have changed.
-6. Use `view_page` for visual judgment. Accessibility/text snapshots are structural evidence, not a substitute for seeing the rendered page.
-7. Inspect console/network/performance data when it helps the task; do not collect large traces by default.
+1. Call `browser_state` when ambient browser state matters, then use `browser_tabs` to select or create the target tab.
+2. Navigate with `browser_navigate` and set the target size with `browser_viewport` before collecting interaction references.
+3. For ordinary web UI, call `browser_dom action=snapshot`, then use the latest UIDs with DOM click/fill/hover/drag/upload actions. Navigation, viewport changes, and substantial DOM mutations can invalidate older UIDs.
+4. For visual interfaces, use `browser_cua action=screenshot` followed by coordinate click/scroll/type/keypress actions instead of forcing an unreliable DOM path.
+5. Take another DOM snapshot after structural state changes, or another visual screenshot after appearance-sensitive changes.
+6. Use rendered pixels for visual judgment. Accessibility/text snapshots are structural evidence, not a substitute for seeing the page.
+7. Drop to `browser_command` only for advanced console/network, emulation, native performance-trace, heap-snapshot, or lower-level page functionality not covered by the facade.
 
-MoonDesk starts one host-owned Chromium lazily on the first browser operation and runs it headless by default at a deterministic 1280x800 initial viewport. The expensive Chromium/MCP process is shared, but browser ownership is not: each registered workspace gets a named isolated BrowserContext for cookies/storage, and each ChatGPT conversation gets its own logical page set inside that workspace context. MoonDesk routes page-scoped operations by the conversation's owned page ID instead of trusting Chromium's globally selected tab. Conversations in the same workspace therefore share that project's login/storage state while keeping separate tabs; different workspaces do not share cookies/localStorage/IndexedDB/service-worker state. Always use the connector that owns the project for its browser work--do not switch to another workspace connector merely because it exposes browser tools.
+MoonDesk starts one host-owned managed Chromium lazily on the first browser operation and runs it headless by default at a deterministic 1280x800 initial viewport. On an empty cache it first downloads the pinned Chrome for Testing artifact, verifies its exact size and SHA-256, and installs it atomically. The Chromium process and native CDP connection are shared, but browser ownership is not: each registered workspace gets a named isolated BrowserContext for cookies/storage, and each ChatGPT conversation gets its own logical page set inside that workspace context. MoonDesk routes page-scoped operations by the conversation's owned page ID instead of trusting Chromium's globally selected tab. Conversations in the same workspace therefore share that project's login/storage state while keeping separate tabs; different workspaces do not share cookies/localStorage/IndexedDB/service-worker state. Always use the connector that owns the project for its browser work--do not switch to another workspace connector merely because it exposes browser tools.
 
-Headless mode still supports snapshots, screenshots, `view_page`, console/network inspection, interaction, and responsive emulation. Users can switch the one agent Chromium to visible presentation from MoonDesk's dashboard. In `multi-tools`, use `set_browser_presentation` only when the user needs to see or manually interact with the browser, such as a login, CAPTCHA, or permission prompt; `view_page` already provides rendered pixels for agent inspection. Presentation is process-global. Changing it while Chromium is running closes **all** MoonDesk workspace BrowserContexts and conversation/CLI tabs, so the tool returns `confirmation_required` until the user explicitly approves that loss and the agent retries with `confirm_restart=true`. If the shared runtime is lost or a dispatched operation exceeds its deadline, MoonDesk invalidates that runtime before another browser operation can run; the next call starts a fresh Chromium generation and all callers must re-establish their page/snapshot state. MoonDesk never automatically replays an ambiguous state-changing action.
+Headless and visible presentation expose the same browser capabilities: tabs, navigation, DOM control, visual CUA, viewport control, screenshots, and advanced DevTools inspection. Users can switch the one agent Chromium to visible presentation from MoonDesk's dashboard. In `multi-tools`, use `set_browser_presentation` only when the user needs to see or manually interact with the browser, such as a login, CAPTCHA, or permission prompt; `browser_cua action=screenshot` and `view_page` already provide rendered pixels for agent inspection. Presentation is process-global. Changing it while Chromium is running closes **all** MoonDesk workspace BrowserContexts and conversation/CLI tabs, so the tool returns `confirmation_required` until the user explicitly approves that loss and the agent retries with `confirm_restart=true`. If the shared runtime is lost or a dispatched operation exceeds its deadline, MoonDesk invalidates that runtime before another browser operation can run; the next call starts a fresh Chromium generation and all callers must re-establish their page/snapshot state. MoonDesk never automatically replays an ambiguous state-changing action.
 
 ## Local dev-server verification
 
 When an agent starts a local web server, browser verification is part of completing the task rather than a separate setup step:
 
 1. Wait until the server reports its localhost URL as ready.
-2. Navigate this conversation's project browser page to that URL.
-3. Set the viewport being tested, then take a fresh snapshot.
-4. Exercise the user-visible flow with snapshot UIDs.
-5. Inspect console/network output when debugging behavior.
-6. Use `view_page` to verify the actual rendered result; do not declare visual success from DOM text alone.
+2. Navigate this conversation's selected tab to that URL with `browser_navigate`.
+3. Set the viewport being tested with `browser_viewport`, then take a fresh `browser_dom` snapshot when DOM interactions are appropriate.
+4. Exercise ordinary controls through `browser_dom`; use `browser_cua` for canvas/custom editors/maps or other visual interactions.
+5. Inspect console/network output through the advanced DevTools path when debugging behavior.
+6. Use `browser_cua action=screenshot` or `view_page` to verify the actual rendered result; do not declare visual success from DOM text alone.
 7. Repeat at the relevant desktop/tablet/mobile viewport when the change is responsive.
 
 ## Common CLI commands
@@ -73,13 +79,13 @@ moondesk browser take_snapshot
 moondesk browser list_console_messages
 ```
 
-Prefer the CLI for orchestration, but return to `view_page` whenever the task requires actual visual inspection.
+Prefer the capability facade for interactive agent work. Use the CLI for deterministic orchestration, then return to `browser_cua action=screenshot` or `view_page` whenever the task requires actual visual inspection.
 
 ## Safety and lifecycle
 
-- Do not run browser lifecycle commands (`start`, `status`, `stop`) through MCP `browser_command` or `moondesk browser`; the running MoonDesk host owns that lifecycle.
+- Do not run browser lifecycle commands (`start`, `status`, `stop`) through MCP `browser_command` or `moondesk browser`; the running MoonDesk host owns that lifecycle. `browser_state` observes lifecycle state without starting Chromium.
 - `moondesk browser` is a lightweight localhost client to the running MoonDesk host. It does not own a separate browser process. CLI calls for one workspace share that workspace's BrowserContext/storage but use a dedicated local-CLI logical tab session, separate from ChatGPT conversations.
-- ReadOnly mode permits inspection commands only; navigation, JavaScript execution, interaction, uploads, resizing, and other state-changing browser commands are blocked.
+- ReadOnly mode narrows the facade to inspection-only actions: `browser_state`, tab list/selected, DOM snapshot/wait, visual screenshot, and viewport get. Navigation, arbitrary JavaScript execution, interaction, uploads, resizing, and other state-changing browser actions are blocked.
 - Treat `evaluate_script` as code execution in this caller's currently active logical page. MoonDesk injects the owned upstream page ID; callers must not rely on Chromium's globally selected tab. Use scripts only when needed and keep them narrowly scoped.
-- Browser file paths are local machine paths. Relative input paths stay inside the active workspace. Browser-only mode keeps browser inputs workspace-scoped. When Computer tools are also enabled (`Both` mode), an explicit absolute input-file path (for example, `upload_file`) may reference another regular file readable by the MoonDesk user; MoonDesk stages a private copy before Chromium sees it. Input directories and file-producing/output paths remain workspace-bound.
-- Browser-global extension lifecycle operations are intentionally unavailable through the shared runtime because installing/reloading an extension would mutate every workspace. Performance traces and screencasts are globally singleton upstream resources, so MoonDesk leases them to the conversation and exact page that started them.
+- Browser file paths are local machine paths. Relative input paths stay inside the active workspace. Browser-only mode keeps browser inputs workspace-scoped. When Computer tools are also enabled (`Both` mode), an explicit absolute input-file path (for example, `upload_file`) may reference another regular file readable by the MoonDesk user; MoonDesk stages a private copy before Chromium sees it. File-producing/output paths remain workspace-bound.
+- The native surface intentionally omits adapter-specific extension lifecycle, Lighthouse wrapper, WebMCP/third-party-tool discovery, and screencast commands. Native CDP performance tracing is browser-global, so MoonDesk leases the active trace to the conversation and exact page that started it.