@mastra/mcp-docs-server 1.2.19-alpha.18 → 1.2.19-alpha.19

package/.docs/docs/sandbox/overview.md

@@ -73,6 +73,48 @@ Set `{ enabled: false }` on one tool to remove it, or set the top-level `enabled
 
 See the [sandbox tools reference](https://mastra.ai/reference/workspace/workspace-class) for all generated tools and the [tool configuration reference](https://mastra.ai/reference/workspace/workspace-class) for approvals, output limits, and hooks.
 
+### Computer-use tools
+
+Sandboxes that run a desktop environment can expose screenshot, mouse, and keyboard control. The workspace registers these tools when a statically configured sandbox supports the computer capability:
+
+| Tool                       | Does                                                                             |
+| -------------------------- | -------------------------------------------------------------------------------- |
+| `computer_screenshot`      | Captures the desktop as a PNG image and returns it to the model as native media. |
+| `computer_click`           | Presses and releases the left mouse button at pixel coordinates.                 |
+| `computer_double_click`    | Presses the left mouse button twice at pixel coordinates.                        |
+| `computer_right_click`     | Presses and releases the right mouse button at pixel coordinates.                |
+| `computer_move_mouse`      | Moves the cursor to pixel coordinates without pressing a button.                 |
+| `computer_drag`            | Presses, drags, and releases between two points.                                 |
+| `computer_type`            | Types text into the focused element.                                             |
+| `computer_press_key`       | Presses a key or key combination, such as `Enter` or `ctrl+s`.                   |
+| `computer_scroll`          | Scrolls up or down.                                                              |
+| `computer_get_screen_info` | Gets the screen dimensions and cursor position.                                  |
+| `computer_wait`            | Waits for the interface to settle.                                               |
+
+[`DaytonaSandbox`](https://mastra.ai/integrations/sandboxes/daytona) and [`E2BDesktopSandbox`](https://mastra.ai/integrations/sandboxes/e2b-desktop) support this capability. Other sandbox backends don't register the computer tools. Resolver-backed sandboxes don't register them because the workspace can't inspect the resolved sandbox's capabilities when it creates the tool list.
+
+Action tools take a screenshot after each action by default. Configure the screenshot behavior for each tool:
+
+```typescript
+import { Workspace, WORKSPACE_TOOLS } from '@mastra/core/workspace'
+import { E2BDesktopSandbox } from '@mastra/e2b-desktop'
+
+const workspace = new Workspace({
+  sandbox: new E2BDesktopSandbox(),
+  tools: {
+    [WORKSPACE_TOOLS.COMPUTER.CLICK]: {
+      screenshotAfterAction: true,
+      screenshotDelayMs: 1000,
+    },
+    [WORKSPACE_TOOLS.COMPUTER.TYPE]: {
+      requireApproval: true,
+    },
+  },
+})
+```
+
+Computer tools accept the same per-tool `enabled` and `requireApproval` configuration as other workspace tools.
+
 Authored runtime functions, including tools and workflow steps, can get the live sandbox from their execution context. Use it to execute commands, install dependencies, process files, or spawn a long-running process.
 
 ```typescript
@@ -136,6 +178,7 @@ Use a remote or container sandbox when commands need a stronger boundary from th
 - [Daytona](https://mastra.ai/integrations/sandboxes/daytona)
 - [Docker](https://mastra.ai/integrations/sandboxes/docker)
 - [E2B](https://mastra.ai/integrations/sandboxes/e2b)
+- [E2B Desktop](https://mastra.ai/integrations/sandboxes/e2b-desktop)
 - [Mastra](https://mastra.ai/reference/workspace/platform-sandbox)
 - [Modal](https://mastra.ai/integrations/sandboxes/modal)
 - [Railway](https://mastra.ai/integrations/sandboxes/railway)

package/.docs/integrations/sandboxes/daytona.md

@@ -298,6 +298,36 @@ const workspace = new Workspace({
 
 Inside the sandbox, the environment variable holds an opaque placeholder. Daytona's egress proxy substitutes the real value into HTTPS request headers toward the Secret's allowed hosts, so the raw credential never enters the sandbox. Secrets are applied at sandbox creation and are preserved by `clone()`.
 
+### Computer use (desktop)
+
+`DaytonaSandbox` exposes the [computer capability](https://mastra.ai/docs/sandbox/overview): screenshot, mouse, and keyboard control of a desktop environment inside the sandbox. When the sandbox is used in a workspace, agents automatically get the `mastra_workspace_computer_*` tools.
+
+The desktop processes (Xvfb, xfce4, x11vnc, noVNC) are started lazily on the first computer operation:
+
+```typescript
+const sandbox = new DaytonaSandbox()
+await sandbox.start()
+
+await sandbox.computer.leftClick(100, 200)
+await sandbox.computer.type('hello')
+const { data } = await sandbox.computer.screenshot() // PNG bytes
+
+// Live desktop view via the noVNC preview link
+const url = await sandbox.computer.streamUrl()
+```
+
+Disable the capability, or manage the desktop processes yourself, with the `computerUse` option:
+
+```typescript
+// No computer capability, no computer tools
+new DaytonaSandbox({ computerUse: false })
+
+// Capability stays on, but you call sandbox.daytona.computerUse.start() yourself
+new DaytonaSandbox({ computerUse: { autoStart: false } })
+```
+
+For Daytona-specific desktop APIs (regions, compressed screenshots, screen recording, accessibility tree), use the [direct SDK access](#direct-sdk-access) escape hatch: `sandbox.daytona.computerUse`.
+
 ## Constructor parameters
 
 **id** (`string`): Unique identifier for this sandbox instance. (Default: `Auto-generated`)
@@ -346,6 +376,8 @@ Inside the sandbox, the environment variable holds an opaque placeholder. Dayton
 
 **secrets** (`Record<string, string>`): Daytona Secrets to expose inside the sandbox, mapping environment variable names to Daytona Secret names. The env var holds an opaque placeholder; the real value is substituted into HTTPS request headers at egress toward the Secret's allowed hosts.
 
+**computerUse** (`boolean | { autoStart?: boolean; noVncPort?: number }`): Computer-use (desktop) capability configuration. Set to false to disable the capability. Set autoStart to false to manage the desktop processes yourself. noVncPort sets the noVNC viewer port used by computer.streamUrl(). (Default: `true`)
+
 ## Properties
 
 **id** (`string`): Sandbox instance identifier.
@@ -360,6 +392,8 @@ Inside the sandbox, the environment variable holds an opaque placeholder. Dayton
 
 **processes** (`DaytonaProcessManager`): Background process manager. See SandboxProcessManager reference.
 
+**computer** (`SandboxComputer | undefined`): Computer-use capability: screenshot, mouse, keyboard, and stream URL. Undefined when constructed with computerUse: false. See SandboxComputer reference.
+
 ## Background processes
 
 `DaytonaSandbox` includes a built-in process manager for spawning and managing background processes. Processes run in the Daytona cloud sandbox using session-based command execution.

package/.docs/integrations/sandboxes/e2b-desktop.md

@@ -0,0 +1,128 @@
+> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
+
+# E2B Desktop
+
+Runs a full Linux desktop environment in an isolated [E2B](https://e2b.dev) cloud sandbox with screenshot, mouse, and keyboard control. `E2BDesktopSandbox` extends [`E2BSandbox`](https://mastra.ai/integrations/sandboxes/e2b), so everything the base provider supports (command execution, background processes, file upload, pause/resume reconnection) works against the same desktop machine. For interface details, see [WorkspaceSandbox interface](https://mastra.ai/reference/workspace/sandbox).
+
+## Installation
+
+**npm**:
+
+```bash
+npm install @mastra/e2b-desktop
+```
+
+**pnpm**:
+
+```bash
+pnpm add @mastra/e2b-desktop
+```
+
+**Yarn**:
+
+```bash
+yarn add @mastra/e2b-desktop
+```
+
+**Bun**:
+
+```bash
+bun add @mastra/e2b-desktop
+```
+
+Set your E2B API key with the `E2B_API_KEY` environment variable or the `apiKey` option.
+
+## Usage
+
+Add an `E2BDesktopSandbox` to a workspace and assign it to an agent. Because the sandbox supports the [computer capability](https://mastra.ai/docs/sandbox/overview), the workspace registers the `mastra_workspace_computer_*` tools alongside the shell and process tools:
+
+```typescript
+import { Agent } from '@mastra/core/agent'
+import { Workspace } from '@mastra/core/workspace'
+import { E2BDesktopSandbox } from '@mastra/e2b-desktop'
+
+const workspace = new Workspace({
+  sandbox: new E2BDesktopSandbox({
+    resolution: [1280, 720],
+  }),
+})
+
+const agent = new Agent({
+  id: 'desktop-agent',
+  name: 'Desktop Agent',
+  instructions: 'You can control a Linux desktop and run shell commands.',
+  model: 'anthropic/claude-sonnet-4-6',
+  workspace,
+})
+
+const response = await agent.generate(
+  'Take a screenshot, then create /tmp/hello.txt with the text "hello" and cat it.',
+)
+```
+
+The agent can mix desktop actions (click, type, screenshot) with shell commands. Both surfaces operate on the same machine.
+
+### Direct desktop control
+
+Use the `computer` capability programmatically without an agent:
+
+```typescript
+const sandbox = new E2BDesktopSandbox()
+await sandbox.start()
+
+await sandbox.computer.leftClick(100, 200)
+await sandbox.computer.type('hello')
+const { data } = await sandbox.computer.screenshot() // PNG bytes
+
+const size = await sandbox.computer.getScreenSize()
+console.log(size) // { width: 1024, height: 768 }
+```
+
+### Live desktop view
+
+`streamUrl()` starts an authenticated noVNC stream inside the sandbox and returns a viewer URL. Open it in a browser to watch the agent work:
+
+```typescript
+const url = await sandbox.computer.streamUrl()
+// https://6080-<sandbox-id>.e2b.app/vnc.html?...&password=<auth-key>
+```
+
+### Desktop SDK escape hatch
+
+Desktop-only APIs such as `launch`, `open`, window helpers, and custom stream control are available on the underlying [`@e2b/desktop`](https://github.com/e2b-dev/desktop) sandbox:
+
+```typescript
+await sandbox.desktop.launch('xfce4-terminal')
+await sandbox.desktop.open('https://mastra.ai')
+```
+
+## Constructor parameters
+
+Accepts all [`E2BSandbox` options](https://mastra.ai/integrations/sandboxes/e2b) plus:
+
+**resolution** (`[number, number]`): Desktop display resolution as \[width, height] in pixels. Applies to newly created sandboxes only.
+
+**dpi** (`number`): Desktop display DPI. Applies to newly created sandboxes only.
+
+When no `template` is provided, the E2B-hosted `desktop` template is used instead of the base provider's mountable template.
+
+## Properties
+
+**computer** (`SandboxComputer`): Computer-use capability: screenshot, mouse, keyboard, and stream URL. See SandboxComputer reference.
+
+**desktop** (`Sandbox`): The underlying @e2b/desktop SDK sandbox for desktop-only APIs. Throws if the sandbox has not started.
+
+**name** (`string`): Provider name ('E2BDesktopSandbox')
+
+**provider** (`string`): Provider identifier ('e2b-desktop')
+
+## Cloud storage mounting
+
+The default `desktop` template has no FUSE tooling, so [cloud storage mounting](https://mastra.ai/integrations/sandboxes/e2b) requires a custom desktop template with `s3fs` or `gcsfuse` installed. Pass it with the `template` option.
+
+## Related
+
+- [Computer-use tools](https://mastra.ai/docs/sandbox/overview)
+- [`E2BSandbox` reference](https://mastra.ai/integrations/sandboxes/e2b)
+- [`WorkspaceSandbox` interface](https://mastra.ai/reference/workspace/sandbox)
+- [Sandbox](https://mastra.ai/docs/sandbox/overview)

package/.docs/integrations.md

@@ -40,6 +40,7 @@
 - [Daytona](https://mastra.ai/integrations/sandboxes/daytona)
 - [Docker](https://mastra.ai/integrations/sandboxes/docker)
 - [E2B](https://mastra.ai/integrations/sandboxes/e2b)
+- [E2B Desktop](https://mastra.ai/integrations/sandboxes/e2b-desktop)
 - [Mastra](https://mastra.ai/reference/workspace/platform-sandbox)
 - [Modal](https://mastra.ai/integrations/sandboxes/modal)
 - [Railway](https://mastra.ai/integrations/sandboxes/railway)

package/.docs/reference/workspace/sandbox.md

@@ -10,6 +10,8 @@ The `WorkspaceSandbox` interface defines how workspaces execute commands and man
 
 **processes** (`SandboxProcessManager`): Background process manager. If not implemented, process management tools won't be available. See SandboxProcessManager reference.
 
+**computer** (`SandboxComputer`): Computer-use (desktop) capability. If not implemented, computer tools won't be available. See Computer capability.
+
 ## Methods
 
 ### `start()`
@@ -195,6 +197,48 @@ const instructions = sandbox.getInstructions?.()
 
 **Returns:** `string`
 
+## Computer capability
+
+Sandboxes with a controllable desktop environment implement the optional `SandboxComputer` interface on the `computer` property. When present on a statically configured sandbox, the workspace tools factory registers the `mastra_workspace_computer_*` tools automatically. See [Computer-use tools](https://mastra.ai/docs/sandbox/overview).
+
+Coordinates are pixels from the top-left corner of the display. Providers normalize their SDK semantics (key names, scroll units) onto this surface and expose richer native APIs through their own accessors.
+
+**screenshot()** (`() => Promise<{ data: Uint8Array; mediaType: "image/png" }>`): Capture the current display as a PNG image.
+
+**leftClick(x, y)** (`(x: number, y: number) => Promise<void>`): Left-click at the given coordinates.
+
+**rightClick(x, y)** (`(x: number, y: number) => Promise<void>`): Right-click at the given coordinates.
+
+**doubleClick(x, y)** (`(x: number, y: number) => Promise<void>`): Double-click (left button) at the given coordinates.
+
+**moveMouse(x, y)** (`(x: number, y: number) => Promise<void>`): Move the cursor without clicking.
+
+**drag(from, to)** (`(from: ComputerPosition, to: ComputerPosition) => Promise<void>`): Press the left button at from, drag to to, and release.
+
+**scroll(direction, amount)** (`(direction: 'up' | 'down', amount: number) => Promise<void>`): Scroll the display by the given amount of ticks.
+
+**type(text)** (`(text: string) => Promise<void>`): Type text into the focused element.
+
+**press(key)** (`(key: string | string[]) => Promise<void>`): Press a key or key combination. A string presses one key (for example 'Enter'); an array presses a chord (for example \['ctrl', 's']).
+
+**getScreenSize()** (`() => Promise<{ width: number; height: number }>`): Get the display dimensions.
+
+**getCursorPosition()** (`() => Promise<{ x: number; y: number }>`): Get the current cursor position.
+
+**streamUrl()** (`() => Promise<string | null>`): Get a URL for a live view of the desktop, such as noVNC, or null when unavailable. Optional, not all providers expose a viewer.
+
+Use the `supportsComputer()` type guard to check for the capability:
+
+```typescript
+import { supportsComputer } from '@mastra/core/workspace'
+
+if (supportsComputer(sandbox)) {
+  const { data } = await sandbox.computer.screenshot()
+}
+```
+
+`DaytonaSandbox` and `E2BDesktopSandbox` implement this capability. See the [Daytona](https://mastra.ai/integrations/sandboxes/daytona) and [E2B Desktop](https://mastra.ai/integrations/sandboxes/e2b-desktop) integration pages.
+
 ## Related
 
 - [SandboxProcessManager reference](https://mastra.ai/reference/workspace/process-manager)

package/CHANGELOG.md

@@ -1,5 +1,12 @@
 # @mastra/mcp-docs-server
 
+## 1.2.19-alpha.19
+
+### Patch Changes
+
+- Updated dependencies [[`48ef1f1`](https://github.com/mastra-ai/mastra/commit/48ef1f1d24eedafbb07f64e659a81b52b67b8bf6), [`63796ba`](https://github.com/mastra-ai/mastra/commit/63796ba0fda60253be17535e68f6bbbf1e6ffa09), [`3c19dce`](https://github.com/mastra-ai/mastra/commit/3c19dcef8e73062a80627a4927eae3ec11145afd)]:
+  - @mastra/core@1.62.0-alpha.12
+
 ## 1.2.19-alpha.18
 
 ### Patch Changes

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@mastra/mcp-docs-server",
-  "version": "1.2.19-alpha.18",
+  "version": "1.2.19-alpha.19",
   "description": "MCP server for accessing Mastra.ai documentation, changelogs, and news.",
   "type": "module",
   "main": "dist/index.js",
@@ -28,8 +28,8 @@
     "jsdom": "^26.1.0",
     "local-pkg": "^1.1.2",
     "zod": "^4.4.3",
-    "@mastra/mcp": "^1.17.2-alpha.2",
-    "@mastra/core": "1.62.0-alpha.11"
+    "@mastra/core": "1.62.0-alpha.12",
+    "@mastra/mcp": "^1.17.2-alpha.2"
   },
   "devDependencies": {
     "@hono/node-server": "^2.0.0",
@@ -47,7 +47,7 @@
     "vitest": "4.1.10",
     "@internal/lint": "0.0.125",
     "@internal/types-builder": "0.0.100",
-    "@mastra/core": "1.62.0-alpha.11"
+    "@mastra/core": "1.62.0-alpha.12"
   },
   "homepage": "https://mastra.ai",
   "repository": {