@remcp/runtime 0.2.47 → 0.2.49

package/CHANGELOG.md

@@ -1,5 +1,28 @@
 # Changelog
 
+## Unreleased
+
+## 0.2.49 — 2026-09-20
+
+- Terminal process tools now return schema-validated structured session state, output ranges, match results and termination facts alongside their human-readable text.
+- The runtime catalog and hosted tool contract align on bounded structured outputs and accurate read-only/destructive/idempotent/open-world annotations for the expanded computer-use surface.
+- Preview references remain optional presentation metadata rather than replacing the source result used for model reasoning.
+- Lockstep patch release with the ReMCP client at `0.2.49`.
+
+## 0.2.48 — 2026-09-20
+
+- Migrate the local stdio runtime to the official `@modelcontextprotocol/server` v2 serving entry while preserving legacy 2025-era client compatibility.
+- Add negotiated MCP `2026-07-28` support, including `server/discover`, cache hints, full structured output validation, and subscription-based `notifications/tools/list_changed` delivery.
+- Keep the legacy `@modelcontextprotocol/sdk` and modern `@modelcontextprotocol/client` packages dev-only so CI continuously proves both protocol eras against the same runtime.
+- Add optional Tesseract-backed OCR inside `computer_snapshot`/`computer_action`, including bounded word/line boxes and HiDPI/multi-monitor coordinate mapping, without adding another MCP tool or production dependency.
+- Added 39 computer-use tools: native desktop/window automation, accessibility UI inspection/actions, browser CDP, diagnostics, and lightweight DOCX/XLSX/PDF operations.
+- Live `tools/list` now advertises only capabilities available on the current operating system and sends `notifications/tools/list_changed` when the capability set changes; `--print-tools` remains the stable full 83-tool release contract.
+- Added high-level `computer_snapshot` and `computer_action` primitives so agents can prefer semantic Accessibility/DOM targets and fall back to coordinates/screenshots only when necessary.
+- Chrome DevTools Protocol endpoints are restricted to loopback addresses. New filesystem/document operations continue through the existing resolved-path allowlist, and launch/service/power operations reuse the runtime command policy.
+- Fixed metadata output truncation once the JSON tool contract exceeded the operating-system pipe buffer by waiting for stdout writes to flush before exit.
+- Terminal session tools now expose schema-validated structured `pid`, status, exit information, output ranges/matches, and session lists alongside their human-readable text, so hosted agents can chain process calls without parsing prose.
+- No browser/document rendering stack was added; the runtime remains dependency-light and uses native OS facilities plus existing qpdf/poppler/zip helpers when available.
+
 ## 0.2.47 — 2026-09-19
 
 - Lockstep release with the ReMCP client. Runtime behavior is unchanged from 0.2.46; the client now recovers from transient HTTP handshake failures and migrates Linux systemd services to a stable launcher independent of nvm/Hermes prefixes.

package/README.md

@@ -1,12 +1,13 @@
 # ReMCP local runtime
 
 `@remcp/runtime` is the local device runtime for [ReMCP](https://remcp.site). It is an MCP
-server that runs on a computer you paired with ReMCP and executes the file, image, search, terminal,
-and process tools that the hosted ReMCP MCP endpoint exposes to ChatGPT and Codex.
+server that runs on a computer you paired with ReMCP and executes file, image, search, terminal,
+process, desktop UI, browser, diagnostics, and document tools exposed through the hosted ReMCP MCP endpoint.
 
 The ReMCP device agent starts this runtime as a child process and talks to it over stdio. The runtime
-never talks to the network on its own: it only answers the paired agent, which holds the device
-credential you can revoke at any time.
+has no independent cloud service or telemetry endpoint: browser automation connects only to a loopback
+Chrome DevTools endpoint, while the explicit `network` diagnostic and commands you run can reach the
+network when requested. The device credential remains in the paired agent and can be revoked at any time.
 
 ## Install
 
@@ -27,8 +28,10 @@ npx @remcp/runtime              # MCP server over stdio
 
 ## Tools
 
-44 tools, all implemented in this repository. Nothing is gated behind an approval step: a tool call
-executes.
+83 tools are implemented in this repository: the original 44 filesystem/search/terminal/process tools
+plus 39 computer-use, browser, diagnostics, and document tools. The release contract contains all 83;
+the live local MCP `tools/list` omits optional platform-specific tools that the current computer cannot
+support and emits `notifications/tools/list_changed` when that capability set changes. The stdio server is dual-era: existing 2025-era clients continue to work unchanged, while clients that negotiate MCP `2026-07-28` use `server/discover`, cache hints, unrestricted structured output validation, and `subscriptions/listen` for list-change delivery.
 
 | Area | Tools |
 | --- | --- |
@@ -41,18 +44,27 @@ executes.
 | Search | `start_search`, `get_more_search_results`, `stop_search`, `list_searches` |
 | Processes | `start_process`, `read_process_output`, `wait_for_process_output`, `interact_with_process`, `force_terminate`, `list_sessions`, `list_processes`, `kill_process` |
 | Runtime | `get_system_info`, `get_runtime_info`, `get_runtime_stats`, `set_config_value` |
+| Computer use | `computer_snapshot`, `computer_action`, `list_windows`, `window_action`, `launch_app`, `ui_snapshot`, `ui_find`, `ui_action`, `type_text`, `keyboard`, `pointer`, `drag_drop`, `scroll`, `wait_for_ui`, `clipboard`, `display_inventory`, `screenshot_region`, `open_path`, `reveal_path`, `notification` |
+| Browser | `browser_tabs`, `browser_navigate`, `browser_snapshot`, `browser_find`, `browser_action`, `browser_wait`, `browser_evaluate` |
+| Diagnostics | `service`, `event_log`, `network`, `installed_apps`, `environment`, `audio`, `power_action`, `record_screen` |
+| Documents | `read_document`, `edit_spreadsheet`, `edit_document`, `pdf_action` |
 
-The hosted ReMCP endpoint adds eight account/relay tools — `list_devices`, `ping_device`,
+The hosted ReMCP endpoint adds eleven account/presentation tools — `list_devices`, `ping_device`,
 `who_am_i`, `create_pairing_command`, `get_recent_tool_calls`, `get_usage_statistics`,
-`get_configuration`, and `rename_device` — for a 52-tool public surface. Everything else the
-agent may need, including service management, package installs, git and docker, runs through
-`start_process` with the permissions of the account running the agent.
+`get_configuration`, `render_file_preview`, `render_image_preview`, `render_terminal_preview`, and
+`rename_device` — for a 94-tool MCP Apps catalog. Plain MCP clients receive 91 tools because the
+three presentation-only render tools are omitted unless the UI extension is negotiated.
 
-`set_config_value` is deliberately narrow: it can change only `telemetryEnabled`,
+`computer_snapshot` and `computer_action` keep OCR inside the compact high-level surface: when a local `tesseract` binary is available, snapshot can return bounded OCR text/boxes and click targeting can fall back through Accessibility → browser DOM/CDP → OCR → coordinates. OCR is optional and never adds another MCP tool or bundled OCR dependency.
+
+For frontend QA, use semantic data to operate the page and rendered pixels to verify it: `browser_action(action="set_viewport")` sets an exact responsive-test viewport, `browser_wait` waits for the real application state, and `browser_snapshot(include_screenshot=true, selector=...)` recenters the component, returns its bounds, and attaches the real CDP-rendered viewport PNG. This intentionally treats screenshots as the visual oracle without making screenshot-driven coordinate guessing the primary control path.
+
+`set_config_value` remains deliberately narrow: it can change only `telemetryEnabled`,
 `maxReadLines`, `maxBufferedLines`, and `maxOutputBytes`. Access roots, blocked commands,
 the command guardrail, shell, write limit, runtime name, and unrestricted mode stay with the person
-at the computer. There is no `write_pdf`/spreadsheet/DOCX tooling; use `start_process` with the
-document tools already installed on your machine.
+at the computer. DOCX/XLSX edits operate directly on OOXML, while structural PDF writes use existing
+system tools such as qpdf/poppler when present; ReMCP still does not bundle Chromium, Puppeteer,
+`sharp`, or `exceljs`.
 
 `--print-tools` prints the exact JSON contract (schemas and annotations) the runtime advertises, and
 `src/catalog.mjs` is the single source of truth for it.
@@ -60,8 +72,14 @@ document tools already installed on your machine.
 ## No approval staircase
 
 ReMCP is a remote control for computers you own, with the same trust model as SSH: the tool call runs,
-and the user's request is the authorization. There is no approval prompt, no "are you sure", and no
-dry-run detour unless you ask for one.
+and the user's request is the authorization. ReMCP adds no per-tool approval prompt, no "are you sure",
+and no dry-run detour unless you ask for one.
+
+Operating-system security boundaries still apply. macOS can require Accessibility/Screen Recording grants,
+and GNOME Wayland requires a one-time XDG RemoteDesktop consent before low-level keyboard/pointer input.
+ReMCP does not bypass those controls: after GNOME grants access, the portal restore token is stored locally
+under the runtime config directory with mode `0600` and reused when the portal permits it. Semantic AT-SPI
+UI actions and the XDG Screenshot portal remain separate from that low-level input permission.
 
 - file writes replace by default (`mode: "append"` to add), moves and copies replace the destination
   (`overwrite: false` refuses instead), `replace_in_files` applies immediately (`dry_run: true`
@@ -106,9 +124,10 @@ in `~/.config/remcp/runtime.json`. `--describe` always reports the current state
 
 ## Runtime properties
 
-- **No network calls of its own.** The runtime opens no sockets: every byte it emits goes to the
-  paired agent. Shell commands it runs can of course reach the network, exactly as they would from
-  your own terminal.
+- **No independent cloud control plane.** Telemetry and device RPC still leave only through the paired
+  agent. Browser control accepts only loopback CDP endpoints. The explicit `network` diagnostic can
+  open a bounded TCP connection to a host/port, and shell/browser actions can reach the network when
+  the requested operation itself requires it.
 - **Optional confinement.** `allowedRoots` is empty by default. When you set it, it is enforced
   against the resolved real path of the deepest existing ancestor rather than the lexical string, so
   `<allowed>/link -> /etc` cannot be used to read or write outside the allowed directories.
@@ -194,8 +213,8 @@ MCP server.
 
 | | Desktop Commander 0.2.50 | ReMCP runtime |
 | --- | --- | --- |
-| Tools | 26, including config mutators and document tooling | 44, including binary transfer, bulk file operations, archives, screenshots, search, process control, and narrowly scoped runtime preferences |
-| Runtime dependencies | 34 (Supabase, Puppeteer/md-to-pdf, `sharp`, `exceljs`, Tiptap, ripgrep download) | 2 direct (`@modelcontextprotocol/sdk`, `@jellybrick/dbus-next`) |
+| Tools | 26, including config mutators and document tooling | 83, including filesystem/terminal plus native desktop UI, browser CDP, diagnostics, and lightweight document operations |
+| Runtime dependencies | 34 (Supabase, Puppeteer/md-to-pdf, `sharp`, `exceljs`, Tiptap, ripgrep download) | 2 production-direct (`@modelcontextprotocol/server` v2, `@jellybrick/dbus-next`); legacy/modern clients are dev-only compatibility tests |
 | Install scripts | `postinstall` posts an install payload that ignores the telemetry setting | none |
 | Telemetry | opt-out, 51 event names, remote feature flags, A/B assignment, third-party processor | opt-out, whitelisted event schema, no endpoint, no flags |
 | Package footprint | 3.78 MB unpacked, 249 files | small first-party runtime package; no bundled browser or document-rendering stack |
@@ -206,12 +225,12 @@ MCP server.
 | Images | file preview UI in a specific client | `read_image` returns the image to any MCP client |
 | Termination | session kill only | whole process group, plus runtime supervision and restart |
 
-What ReMCP deliberately does not implement, and why: document rendering (`write_pdf`) and spreadsheet
-handling would put Puppeteer, `sharp`, and `exceljs` on your computer; unrestricted configuration
-mutation is intentionally absent — `set_config_value` is limited to four non-security preferences,
-while access roots and command security remain local; local usage history would write your arguments
-to disk; URL reads in `read_file` would add an SSRF surface. Everything else the upstream server can
-do has an equivalent here, and the tool count is higher.
+What ReMCP deliberately does not bundle: browser/document rendering stacks such as Puppeteer,
+`sharp`, and `exceljs`. Spreadsheet and DOCX edits use OOXML directly, and PDF structural operations
+use existing system utilities when installed. Unrestricted configuration mutation is intentionally
+absent — `set_config_value` is limited to four non-security preferences, while access roots and
+command security remain local; local usage history would write arguments to disk; URL reads in
+`read_file` remain absent to avoid an implicit SSRF surface.
 
 ## License
 

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@remcp/runtime",
-  "version": "0.2.47",
-  "description": "First-party ReMCP local device runtime: file, search, terminal and process tools over MCP for computers paired with ReMCP.",
+  "version": "0.2.49",
+  "description": "First-party ReMCP local device runtime: filesystem, terminal, computer-use, browser, diagnostics and document tools over MCP.",
   "type": "module",
   "license": "MIT",
   "homepage": "https://remcp.site",
@@ -25,6 +25,10 @@
   },
   "dependencies": {
     "@jellybrick/dbus-next": "^0.11.3",
+    "@modelcontextprotocol/server": "^2.0.0"
+  },
+  "devDependencies": {
+    "@modelcontextprotocol/client": "^2.0.0",
     "@modelcontextprotocol/sdk": "^1.30.0"
   },
   "repository": {

package/src/catalog.mjs

@@ -16,6 +16,72 @@ export const TEXT_OUTPUT_SCHEMA = {
   additionalProperties: false,
 };
 
+const NULLABLE_NUMBER_SCHEMA = { anyOf:[{ type:'number' }, { type:'null' }] };
+const NULLABLE_STRING_SCHEMA = { anyOf:[{ type:'string' }, { type:'null' }] };
+const TERMINAL_SESSION_PROPERTIES = {
+  pid:{ type:'number', description:'Process/session id used by follow-up terminal tools.' },
+  status:{ type:'string', description:'running or exited, with an exit code/signal when known.' },
+  runtimeMs:{ type:'number' },
+  lines:{ type:'number', description:'Total output lines produced by the session.' },
+  exited:{ type:'boolean' },
+  exitCode:NULLABLE_NUMBER_SCHEMA,
+  signal:NULLABLE_STRING_SCHEMA,
+};
+function terminalOutputSchema(extraProperties = {}, extraRequired = []) {
+  return {
+    type:'object',
+    properties:{ ...TERMINAL_SESSION_PROPERTIES, ...extraProperties },
+    required:['pid','status','runtimeMs','lines','exited','exitCode','signal', ...extraRequired],
+    additionalProperties:false,
+  };
+}
+const START_PROCESS_OUTPUT_SCHEMA = terminalOutputSchema({
+  command:{ type:'string' },
+  output:{ type:'string' },
+  partial:{ type:'string' },
+  warning:{ type:'string' },
+}, ['command','output']);
+const READ_PROCESS_OUTPUT_SCHEMA = terminalOutputSchema({
+  range:{ type:'string' },
+  output:{ type:'string' },
+  explicitOffset:{ type:'boolean' },
+}, ['range','output','explicitOffset']);
+const WAIT_PROCESS_OUTPUT_SCHEMA = terminalOutputSchema({
+  pattern:{ type:'string' },
+  matched:{ type:'boolean' },
+  bufferedMatch:{ type:'boolean' },
+  timeoutMs:{ type:'number' },
+  output:{ type:'string' },
+}, ['pattern','matched','bufferedMatch','timeoutMs','output']);
+const INTERACT_PROCESS_OUTPUT_SCHEMA = terminalOutputSchema({
+  output:{ type:'string' },
+}, ['output']);
+const TERMINATE_PROCESS_OUTPUT_SCHEMA = terminalOutputSchema({
+  terminated:{ type:'boolean' },
+  alreadyExited:{ type:'boolean' },
+  escalated:{ type:'boolean' },
+}, ['terminated','alreadyExited','escalated']);
+const LIST_SESSIONS_OUTPUT_SCHEMA = {
+  type:'object',
+  properties:{
+    sessions:{
+      type:'array',
+      items:{
+        type:'object',
+        properties:{
+          ...TERMINAL_SESSION_PROPERTIES,
+          blocked:{ type:'string' },
+          command:{ type:'string' },
+        },
+        required:['pid','status','runtimeMs','lines','exited','exitCode','signal','blocked','command'],
+        additionalProperties:false,
+      },
+    },
+  },
+  required:['sessions'],
+  additionalProperties:false,
+};
+
 const readOnly = { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false };
 const readOnlyNonIdempotent = { readOnlyHint: true, destructiveHint: false, idempotentHint: false, openWorldHint: false };
 const additive = { readOnlyHint: false, destructiveHint: false, idempotentHint: true, openWorldHint: false };
@@ -26,7 +92,7 @@ export const toolDefinitions = [
   {
     name: 'read_file',
     title: 'Read file',
-    description: 'Read a file on this computer: text as text, and .docx or .pdf as their extracted document text (a scanned or font-obfuscated PDF says so instead of returning noise). Use offset and length to page through large files; a negative offset reads from the end of the file.',
+    description: 'Read a plain text or line-oriented file on this computer: source, config, Markdown, scripts or logs. Use read_document for PDF, DOCX, XLSX and other structured documents. Legacy .docx/.pdf extraction remains supported for compatibility. Use offset and length to page through large text files; a negative offset reads from the end.',
     inputSchema: {
       type: 'object',
       properties: {
@@ -606,6 +672,7 @@ export const toolDefinitions = [
       additionalProperties: false,
     },
     annotations: command,
+    outputSchema: START_PROCESS_OUTPUT_SCHEMA,
     handler: terminalToolHandlers.start_process,
   },
   {
@@ -626,6 +693,7 @@ export const toolDefinitions = [
     // Reading without an explicit offset advances the session cursor, so retries can return
     // different output even after the process exits. It never modifies user files.
     annotations: readOnlyNonIdempotent,
+    outputSchema: READ_PROCESS_OUTPUT_SCHEMA,
     handler: terminalToolHandlers.read_process_output,
   },
   {
@@ -643,6 +711,7 @@ export const toolDefinitions = [
       additionalProperties: false,
     },
     annotations: command,
+    outputSchema: INTERACT_PROCESS_OUTPUT_SCHEMA,
     handler: terminalToolHandlers.interact_with_process,
   },
   {
@@ -660,6 +729,7 @@ export const toolDefinitions = [
       additionalProperties: false,
     },
     annotations: readOnlyNonIdempotent,
+    outputSchema: WAIT_PROCESS_OUTPUT_SCHEMA,
     handler: terminalToolHandlers.wait_for_process_output,
   },
   {
@@ -675,6 +745,7 @@ export const toolDefinitions = [
       additionalProperties: false,
     },
     annotations: { readOnlyHint: false, destructiveHint: true, idempotentHint: true, openWorldHint: false },
+    outputSchema: TERMINATE_PROCESS_OUTPUT_SCHEMA,
     handler: terminalToolHandlers.force_terminate,
   },
   {
@@ -683,6 +754,7 @@ export const toolDefinitions = [
     description: 'List terminal sessions started during this ReMCP runtime session with their status and how long they have been running.',
     inputSchema: { type: 'object', properties: {}, additionalProperties: false },
     annotations: readOnly,
+    outputSchema: LIST_SESSIONS_OUTPUT_SCHEMA,
     handler: terminalToolHandlers.list_sessions,
   },
   {