@vdoninja/mcp 0.4.0 → 0.4.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (71) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +111 -0
  3. package/SKILL.md +8 -6
  4. package/assets/browser-agent/audio-worklet.js +17 -0
  5. package/assets/browser-agent/client.js +124 -0
  6. package/assets/browser-agent/index.html +50 -0
  7. package/assets/cloudflare/worker-proxy.js +3 -0
  8. package/docs/.nojekyll +0 -0
  9. package/docs/404.html +28 -0
  10. package/docs/advanced-diagnostics.html +101 -0
  11. package/docs/assets/apple-touch-icon.png +0 -0
  12. package/docs/assets/favicon.svg +7 -0
  13. package/docs/assets/how-it-works.svg +62 -0
  14. package/docs/assets/icon-192.png +0 -0
  15. package/docs/assets/icon-512.png +0 -0
  16. package/docs/assets/icon-maskable-512.png +0 -0
  17. package/docs/assets/icons.svg +42 -0
  18. package/docs/assets/logo.svg +7 -0
  19. package/docs/assets/og-image.png +0 -0
  20. package/docs/assets/site.css +735 -0
  21. package/docs/assets/site.js +289 -0
  22. package/docs/browser-agent.html +122 -0
  23. package/docs/browser-worker.html +165 -0
  24. package/docs/client-config-examples.html +254 -0
  25. package/docs/direction.html +175 -0
  26. package/docs/favicon.ico +0 -0
  27. package/docs/guide.html +872 -0
  28. package/docs/http-relay.html +154 -0
  29. package/docs/index.html +834 -0
  30. package/docs/mcp-tool-contract.html +226 -0
  31. package/docs/quickstart-and-compat.html +222 -0
  32. package/docs/site.webmanifest +16 -0
  33. package/docs/sitemap.xml +63 -0
  34. package/docs/ssapp-integration.html +594 -0
  35. package/docs/vdo-ninja-p2p-data-notes.html +225 -0
  36. package/package.json +67 -11
  37. package/references/advanced-diagnostics.md +6 -4
  38. package/references/browser-agent.md +53 -0
  39. package/references/browser-worker.md +142 -0
  40. package/references/client-config-examples.md +253 -0
  41. package/references/direction.md +88 -0
  42. package/references/http-relay.md +107 -0
  43. package/references/mcp-tool-contract.md +16 -6
  44. package/references/quickstart-and-compat.md +18 -6
  45. package/references/ssapp-integration.md +371 -0
  46. package/references/vdo-ninja-p2p-data-notes.md +3 -1
  47. package/scripts/demo-file-workflow.js +25 -6
  48. package/scripts/demo-message-workflow.js +25 -6
  49. package/scripts/install-mcp.js +55 -4
  50. package/scripts/lib/bridge-session.js +143 -26
  51. package/scripts/lib/bridge-utils.js +3 -0
  52. package/scripts/lib/browser-agent.js +255 -0
  53. package/scripts/lib/browser-media.js +79 -0
  54. package/scripts/lib/browser-routes.js +119 -0
  55. package/scripts/lib/browser-rpc.js +214 -0
  56. package/scripts/lib/browser-setup.js +133 -0
  57. package/scripts/lib/browser-worker.js +216 -0
  58. package/scripts/lib/fake-network-sdk.js +3 -0
  59. package/scripts/lib/http-relay-transport.js +294 -0
  60. package/scripts/lib/load-vdo-sdk.js +3 -0
  61. package/scripts/lib/mcp-stdio-parser.js +3 -0
  62. package/scripts/lib/ssapp-worker.js +198 -0
  63. package/scripts/lib/tool-profiles.js +3 -0
  64. package/scripts/vdo-data-bridge.js +3 -0
  65. package/scripts/vdo-mcp-browser-agent.js +132 -0
  66. package/scripts/vdo-mcp-browser.js +218 -0
  67. package/scripts/vdo-mcp-relay.js +207 -0
  68. package/scripts/vdo-mcp-server.js +87 -8
  69. package/scripts/vdo-mcp-ssapp.js +208 -0
  70. package/scripts/vdo-mcp-streamable-http.js +23 -4
  71. package/server.json +26 -10
package/docs/vdo-ninja-p2p-data-notes.html ADDED
@@ -0,0 +1,225 @@
1
+ <!doctype html>
2
+ <html lang="en" data-icons="assets/icons.svg">
3
+ <head>
4
+ <meta charset="utf-8">
5
+ <meta name="viewport" content="width=device-width, initial-scale=1">
6
+ <title>WebRTC data notes · VDO.Ninja MCP</title>
7
+ <meta name="description" content="Understand the SDK data-channel constraints behind MCP peer connections.">
8
+ <meta name="color-scheme" content="light dark">
9
+ <link rel="canonical" href="https://steveseguin.github.io/ninjamcp/vdo-ninja-p2p-data-notes.html">
10
+ <link rel="icon" href="assets/favicon.svg" type="image/svg+xml">
11
+ <link rel="stylesheet" href="assets/site.css">
12
+ <script>document.documentElement.classList.add('js');</script>
13
+ </head>
14
+ <body class="reference-page">
15
+ <a class="skip-link" href="#main">Skip to content</a>
16
+ <header class="nav">
17
+ <div class="wrap">
18
+ <a aria-label="VDO.Ninja MCP home" class="brand" href="./">
19
+ <img alt="" height="28" src="assets/logo.svg" width="28"/>
20
+ <span>VDO.Ninja <small>MCP</small></span>
21
+ </a>
22
+ <nav aria-label="Primary" class="nav-links">
23
+ <a href="./#how">How it works</a>
24
+ <a href="./#privacy">Privacy</a>
25
+ <a href="./#tools">Tools</a>
26
+ <a href="./#faq">FAQ</a>
27
+ <a aria-current="page" href="guide.html">Docs</a>
28
+ </nav>
29
+ <div class="nav-cta">
30
+ <a class="btn btn-ghost" href="https://github.com/steveseguin/ninjamcp"><svg aria-hidden="true" class="icon-sm"><use href="assets/icons.svg#i-github"></use></svg>GitHub</a>
31
+ <a class="btn btn-dark" href="https://www.npmjs.com/package/@vdoninja/mcp"><svg aria-hidden="true" class="icon-sm"><use href="assets/icons.svg#i-package"></use></svg>npm</a>
32
+ </div>
33
+ <button aria-expanded="false" aria-label="Menu" class="nav-toggle" type="button"><svg aria-hidden="true" class="icon"><use href="assets/icons.svg#i-menu"></use></svg></button>
34
+ </div>
35
+ </header>
36
+ <div class="docs">
37
+ <aside class="sidebar" aria-label="Documentation" id="guide-contents"><h5>Documentation</h5><a href="guide.html">Getting started</a><h5>On this page</h5><a href="#overview">Overview</a><a href="#hard-platform-constraints">Hard Platform Constraints</a><a href="#topologies-that-work">Topologies That Work</a><a href="#delivery-semantics">Delivery Semantics</a><a href="#firewallnat-guidance">Firewall/NAT Guidance</a><a href="#keep-alive-and-session-continuity">Keep-Alive and Session Continuity</a><a href="#bridge-runtime-knobs">Bridge Runtime Knobs</a><a href="#app-envelope-suggestions">App Envelope Suggestions</a><a href="#file-transfer-envelope-pattern-used-here">File Transfer Envelope Pattern Used Here</a><a href="#disk-spooling-and-large-files">Disk Spooling And Large Files</a><a href="#room-state-sync-crdt-lite">Room State Sync (CRDT-lite)</a><a href="#local-source-pointers">Local Source Pointers</a><h5>More guides</h5><a href="quickstart-and-compat.html">Quickstart</a><a href="client-config-examples.html">Client configuration</a><a href="http-relay.html">HTTP relay</a><a href="browser-worker.html">Remote Chrome</a><a href="browser-agent.html">Voice companion</a><a href="ssapp-integration.html">Remote SSApp</a><a href="mcp-tool-contract.html">Tool contract</a><a href="advanced-diagnostics.html">Advanced diagnostics</a><a href="direction.html">Project direction</a></aside>
38
+ <main class="docs-main" id="main">
39
+ <button class="btn docs-toggle" type="button" aria-controls="guide-contents" aria-expanded="false">Contents</button>
40
+ <header>
41
+ <p class="guide-breadcrumb"><a href="guide.html">Documentation</a> / WebRTC data notes</p>
42
+ <h1 id="vdoninja-p2p-data-notes-for-mcp-workflows">VDO.Ninja P2P Data Notes for MCP Workflows</h1>
43
+ <p class="guide-format"><a href="https://github.com/steveseguin/ninjamcp/blob/main/references/vdo-ninja-p2p-data-notes.md">Read in Markdown</a></p>
44
+ </header>
45
+ <section class="doc-section guide-intro" id="overview">
46
+ <p>This file captures operational constraints from this repository's SDK docs.</p>
47
+ </section><section class="doc-section" id="hard-platform-constraints"><h2>Hard Platform Constraints</h2>
48
+ <ol>
49
+ <li>
50
+ <p>Signaling is not an app-data tunnel.
51
+ - Only handshake/signaling data should traverse WebSocket.
52
+ - Application payloads are expected over WebRTC data channels.</p>
53
+ </li>
54
+ <li>
55
+ <p>Room scale is bounded.
56
+ - Documented guidance is about 80 peers per room (viewer limits can apply).
57
+ - For higher fanout, split rooms or use server-mediated patterns.</p>
58
+ </li>
59
+ <li>
60
+ <p>Serverless philosophy is enforced.
61
+ - No server-side state relay for your app protocol.
62
+ - Design as peer-to-peer first.</p>
63
+ </li>
64
+ </ol>
65
+ </section><section class="doc-section" id="topologies-that-work"><h2>Topologies That Work</h2>
66
+ <ol>
67
+ <li>
68
+ <p>Two-peer data link (recommended).
69
+ - Peer A: <code>announce({ streamID })</code>
70
+ - Peer B: <code>view(streamID, { audio:false, video:false })</code>
71
+ - Result: one bidirectional data channel.</p>
72
+ </li>
73
+ <li>
74
+ <p>Multi-peer data mesh.
75
+ - Use <code>autoConnect(room)</code> half-mesh mode to avoid duplicate pair connections.
76
+ - Move to full mesh only when media exchange actually needs two directional AV links.</p>
77
+ </li>
78
+ <li>
79
+ <p>Targeted delivery.
80
+ - Use <code>sendData(data, "uuid")</code> or advanced target objects.
81
+ - Keep <code>preference: "any"</code> unless you explicitly need publisher-only/viewer-only/all-channel behavior.</p>
82
+ </li>
83
+ </ol>
84
+ </section><section class="doc-section" id="delivery-semantics"><h2>Delivery Semantics</h2>
85
+ <ul>
86
+ <li>WebRTC data channels are typically configured for reliable, ordered delivery in SDK defaults.</li>
87
+ <li>Reliability applies after channel establishment; it does not guarantee every network can establish the channel.</li>
88
+ <li>TURN improves establishment success in restrictive NAT/firewall environments but does not guarantee universal reachability.</li>
89
+ <li>This MCP skill layers app-level reliability for file transfer:</li>
90
+ <li>chunk SHA-256 validation</li>
91
+ <li>ACK/NACK feedback</li>
92
+ <li>resume from <code>next_seq</code> after interruption</li>
93
+ </ul>
94
+ </section><section class="doc-section" id="firewallnat-guidance"><h2>Firewall/NAT Guidance</h2>
95
+ <ol>
96
+ <li>
97
+ <p>Prefer TURN when needed.
98
+ - <code>forceTURN: true</code> can push connections through relay infrastructure.
99
+ - This helps restrictive networks but changes latency/cost characteristics.</p>
100
+ </li>
101
+ <li>
102
+ <p>Treat fallback as last resort.
103
+ - <code>allowFallback: true</code> allows <code>sendData</code> fallback through signaling when P2P is unavailable.
104
+ - Use for control/survivability, not sustained throughput.</p>
105
+ </li>
106
+ <li>
107
+ <p>Expect hard failure zones.
108
+ - Some enterprise/WSL/network setups can still block WebRTC UDP.
109
+ - Build retry and reconnection logic regardless of TURN.</p>
110
+ </li>
111
+ </ol>
112
+ </section><section class="doc-section" id="keep-alive-and-session-continuity"><h2>Keep-Alive and Session Continuity</h2>
113
+ <ol>
114
+ <li>
115
+ <p>Keep a heartbeat.
116
+ - Send lightweight periodic app-level heartbeats.
117
+ - Optionally send SDK pings (<code>sendPing</code>) to connected peers.</p>
118
+ </li>
119
+ <li>
120
+ <p>Reconnect predictably.
121
+ - On disconnect/error, recreate SDK instance, reconnect, re-join room, re-announce, and re-view targets.
122
+ - Make handlers idempotent to tolerate duplicate or delayed messages around reconnection.</p>
123
+ </li>
124
+ <li>
125
+ <p>Define canonical IDs.
126
+ - Room and stream IDs should use alphanumeric + underscore.
127
+ - Rely on sanitized IDs consistently across all peers.</p>
128
+ </li>
129
+ </ol>
130
+ </section><section class="doc-section" id="bridge-runtime-knobs"><h2>Bridge Runtime Knobs</h2>
131
+ <p><code>scripts/vdo-data-bridge.js</code> supports:</p>
132
+ <ul>
133
+ <li><code>VDON_ROOM</code> (required in real deployments)</li>
134
+ <li><code>VDON_STREAM_ID</code> (optional; random if omitted)</li>
135
+ <li><code>VDON_TARGET_STREAM_ID</code> (optional; auto-view peer stream for two-peer links)</li>
136
+ <li><code>VDON_HOST</code> (default <code>wss://wss.vdo.ninja</code>)</li>
137
+ <li><code>VDON_PASSWORD</code> (<code>false</code> to disable encryption; omitted uses SDK default)</li>
138
+ <li><code>VDON_FORCE_TURN</code> (<code>true</code>/<code>false</code>)</li>
139
+ <li><code>VDON_HEARTBEAT_MS</code> (default <code>15000</code>)</li>
140
+ <li><code>VDON_RECONNECT_MS</code> (default <code>3000</code>)</li>
141
+ <li><code>VDON_MAX_RECONNECT_MS</code> (default <code>30000</code>)</li>
142
+ <li><code>VDON_DEBUG</code> (<code>true</code>/<code>false</code>)</li>
143
+ </ul>
144
+ </section><section class="doc-section" id="app-envelope-suggestions"><h2>App Envelope Suggestions</h2>
145
+ <p>Use explicit envelope types and IDs over raw strings:</p>
146
+ <pre><code class="lang-json">{
147
+ "type": "mcp.request",
148
+ "id": "req_123",
149
+ "method": "tool.call",
150
+ "payload": { "name": "lookup", "args": { "q": "abc" } }
151
+ }
152
+ </code></pre>
153
+ <pre><code class="lang-json">{
154
+ "type": "mcp.response",
155
+ "id": "req_123",
156
+ "ok": true,
157
+ "payload": { "result": "..." }
158
+ }
159
+ </code></pre>
160
+ <p>Avoid reserved SDK pub/sub types (<code>subscribe</code>, <code>unsubscribe</code>, <code>channelMessage</code>) for custom envelopes.</p>
161
+ </section><section class="doc-section" id="file-transfer-envelope-pattern-used-here"><h2>File Transfer Envelope Pattern Used Here</h2>
162
+ <p>This MCP implementation reserves protocol envelopes with:</p>
163
+ <ul>
164
+ <li><code>__vdo_mcp: \"vdo_mcp_bridge_v1\"</code></li>
165
+ <li><code>kind: \"sync.*\" | \"file.*\"</code></li>
166
+ </ul>
167
+ <p>File transfer handshake:</p>
168
+ <ol>
169
+ <li><code>file.offer</code></li>
170
+ <li><code>file.accept</code></li>
171
+ <li><code>file.chunk</code> + <code>file.ack</code> / <code>file.nack</code></li>
172
+ <li><code>file.complete</code></li>
173
+ <li><code>file.complete_ack</code></li>
174
+ </ol>
175
+ <p>Resume flow:</p>
176
+ <ol>
177
+ <li><code>file.resume_req</code></li>
178
+ <li><code>file.resume_state</code> (<code>next_seq</code>)</li>
179
+ </ol>
180
+ </section><section class="doc-section" id="disk-spooling-and-large-files"><h2>Disk Spooling And Large Files</h2>
181
+ <ul>
182
+ <li>This MCP implementation can spool large incoming transfers to disk.</li>
183
+ <li>Tune via <code>vdo_connect</code>:</li>
184
+ <li><code>file_spool_dir</code></li>
185
+ <li><code>file_spool_threshold_bytes</code></li>
186
+ <li><code>keep_spool_files</code></li>
187
+ <li>Use <code>vdo_file_send_path</code> to avoid base64 memory overhead on large local files.</li>
188
+ <li>Use <code>vdo_file_save</code> to persist completed incoming transfers directly.</li>
189
+ </ul>
190
+ </section><section class="doc-section" id="room-state-sync-crdt-lite"><h2>Room State Sync (CRDT-lite)</h2>
191
+ <ul>
192
+ <li>Shared state is replicated using <code>state.patch</code> and <code>state.snapshot</code>.</li>
193
+ <li>Conflict resolution is tuple-based LWW:</li>
194
+ <li>higher <code>clock</code> wins</li>
195
+ <li>tie breaks by <code>actor</code> lexical order</li>
196
+ <li>MCP surface:</li>
197
+ <li><code>vdo_state_set</code></li>
198
+ <li><code>vdo_state_get</code></li>
199
+ <li><code>vdo_state_sync</code></li>
200
+ </ul>
201
+ </section><section class="doc-section" id="local-source-pointers"><h2>Local Source Pointers</h2>
202
+ <ul>
203
+ <li><code>README.md</code></li>
204
+ <li><code>AI-INTEGRATION.md</code></li>
205
+ <li><code>docs/api-reference.md</code></li>
206
+ </ul></section>
207
+ <nav class="pager" aria-label="More documentation"><a href="guide.html#guides"><small>Keep reading</small><b>Browse all guides</b></a><a href="https://discord.vdo.ninja"><small>Need help?</small><b>Ask on Discord</b></a></nav>
208
+ </main>
209
+ </div>
210
+ <footer class="footer">
211
+ <div class="wrap">
212
+ <a class="brand" href="./"><img alt="" height="24" src="assets/logo.svg" width="24"/><span>VDO.Ninja <small>MCP</small></span></a>
213
+ <nav aria-label="Footer">
214
+ <a href="./">Home</a>
215
+ <a href="https://github.com/steveseguin/ninjamcp">GitHub</a>
216
+ <a href="https://www.npmjs.com/package/@vdoninja/mcp">npm</a>
217
+ <a href="https://discord.vdo.ninja">Discord</a>
218
+ <a href="https://vdo.ninja">VDO.Ninja</a>
219
+ </nav>
220
+ <span>MIT · © 2026 Steve Seguin</span>
221
+ </div>
222
+ </footer>
223
+ <script src="assets/site.js" defer></script>
224
+ </body>
225
+ </html>
package/package.json CHANGED
@@ -1,16 +1,16 @@
1
1
  {
2
2
  "name": "@vdoninja/mcp",
3
- "version": "0.4.0",
4
- "description": "VDO.Ninja MCP bridge for WebRTC data-channel messaging, file transfer, and room sync.",
5
- "license": "AGPL-3.0-only",
6
- "homepage": "https://github.com/steveseguin/ninjasdk/tree/main/MCP",
3
+ "version": "0.4.2",
4
+ "description": "Private agent rooms with messages, files, and shared state over WebRTC or a self-hosted HTTP relay.",
5
+ "author": "Steve Seguin",
6
+ "license": "MIT",
7
+ "homepage": "https://github.com/steveseguin/ninjamcp",
7
8
  "repository": {
8
9
  "type": "git",
9
- "url": "git+https://github.com/steveseguin/ninjasdk.git",
10
- "directory": "MCP"
10
+ "url": "git+https://github.com/steveseguin/ninjamcp.git"
11
11
  },
12
12
  "bugs": {
13
- "url": "https://github.com/steveseguin/ninjasdk/issues"
13
+ "url": "https://github.com/steveseguin/ninjamcp/issues"
14
14
  },
15
15
  "keywords": [
16
16
  "mcp",
@@ -25,23 +25,47 @@
25
25
  "vdon-mcp-install": "scripts/install-mcp.js",
26
26
  "vdon-mcp-server": "scripts/vdo-mcp-server.js",
27
27
  "vdon-mcp-http": "scripts/vdo-mcp-streamable-http.js",
28
+ "vdon-mcp-relay": "scripts/vdo-mcp-relay.js",
28
29
  "vdon-mcp-demo-message": "scripts/demo-message-workflow.js",
29
- "vdon-mcp-demo-file": "scripts/demo-file-workflow.js"
30
+ "vdon-mcp-demo-file": "scripts/demo-file-workflow.js",
31
+ "vdon-mcp-browser": "scripts/vdo-mcp-browser.js",
32
+ "vdon-mcp-browser-agent": "scripts/vdo-mcp-browser-agent.js",
33
+ "vdon-mcp-ssapp": "scripts/vdo-mcp-ssapp.js"
30
34
  },
31
35
  "scripts": {
36
+ "test": "node tests/run-mcp-tests-minimal.js",
37
+ "test:all": "node tests/run-mcp-tests.js",
38
+ "check": "npm run test:all",
39
+ "test:mcp": "node tests/run-mcp-tests-minimal.js",
40
+ "test:mcp:all": "node tests/run-mcp-tests.js",
41
+ "test:mcp:live": "node tests/live-turn-smoke.js",
42
+ "test:mcp:soak": "node tests/live-turn-soak.js",
43
+ "test:mcp:matrix": "node tests/live-turn-matrix.js",
44
+ "test:mcp:preset:matrix": "node tests/live-turn-preset-matrix.js",
45
+ "test:mcp:matrix:sweep": "node tests/live-turn-matrix-sweep.js",
32
46
  "mcp:install": "node scripts/install-mcp.js",
33
47
  "mcp:server": "node scripts/vdo-mcp-server.js",
34
48
  "mcp:http": "node scripts/vdo-mcp-streamable-http.js",
49
+ "mcp:relay": "node scripts/vdo-mcp-relay.js",
35
50
  "demo:message": "node scripts/demo-message-workflow.js",
36
- "demo:file": "node scripts/demo-file-workflow.js"
51
+ "demo:file": "node scripts/demo-file-workflow.js",
52
+ "prepublishOnly": "npm run test:all",
53
+ "mcp:browser": "node scripts/vdo-mcp-browser.js",
54
+ "mcp:browser:agent": "node scripts/vdo-mcp-browser-agent.js",
55
+ "mcp:ssapp": "node scripts/vdo-mcp-ssapp.js"
37
56
  },
38
57
  "files": [
58
+ "LICENSE",
59
+ "docs",
39
60
  "SKILL.md",
40
61
  "server.json",
41
62
  "assets/cloudflare/worker-proxy.js",
42
63
  "assets/cloudflare/wrangler.toml.example",
43
64
  "references/advanced-diagnostics.md",
65
+ "references/client-config-examples.md",
44
66
  "references/mcp-tool-contract.md",
67
+ "references/http-relay.md",
68
+ "references/direction.md",
45
69
  "references/quickstart-and-compat.md",
46
70
  "references/vdo-ninja-p2p-data-notes.md",
47
71
  "scripts/demo-file-workflow.js",
@@ -50,17 +74,49 @@
50
74
  "scripts/vdo-data-bridge.js",
51
75
  "scripts/vdo-mcp-server.js",
52
76
  "scripts/vdo-mcp-streamable-http.js",
77
+ "scripts/vdo-mcp-relay.js",
53
78
  "scripts/lib/bridge-session.js",
54
79
  "scripts/lib/bridge-utils.js",
55
80
  "scripts/lib/fake-network-sdk.js",
56
81
  "scripts/lib/load-vdo-sdk.js",
82
+ "scripts/lib/http-relay-transport.js",
57
83
  "scripts/lib/mcp-stdio-parser.js",
58
- "scripts/lib/tool-profiles.js"
84
+ "scripts/lib/tool-profiles.js",
85
+ "scripts/vdo-mcp-browser.js",
86
+ "scripts/lib/browser-rpc.js",
87
+ "scripts/lib/browser-worker.js",
88
+ "scripts/lib/browser-media.js",
89
+ "references/browser-worker.md",
90
+ "scripts/lib/browser-routes.js",
91
+ "scripts/lib/browser-setup.js",
92
+ "scripts/lib/browser-agent.js",
93
+ "scripts/vdo-mcp-browser-agent.js",
94
+ "assets/browser-agent",
95
+ "references/browser-agent.md",
96
+ "scripts/vdo-mcp-ssapp.js",
97
+ "scripts/lib/ssapp-worker.js",
98
+ "references/ssapp-integration.md"
59
99
  ],
60
100
  "engines": {
61
101
  "node": ">=18"
62
102
  },
63
103
  "dependencies": {
64
- "@vdoninja/sdk": "^1.3.18"
104
+ "@vdoninja/sdk": "^1.5.4",
105
+ "ws": "^8.21.1"
106
+ },
107
+ "optionalDependencies": {
108
+ "@roamhq/wrtc": "^0.9.1"
109
+ },
110
+ "peerDependencies": {
111
+ "chrome-devtools-mcp": "1.11.0",
112
+ "puppeteer-core": "24.37.5"
113
+ },
114
+ "peerDependenciesMeta": {
115
+ "chrome-devtools-mcp": {
116
+ "optional": true
117
+ },
118
+ "puppeteer-core": {
119
+ "optional": true
120
+ }
65
121
  }
66
122
  }
package/references/advanced-diagnostics.md CHANGED
@@ -1,5 +1,7 @@
1
1
  # VDO.Ninja MCP Advanced Diagnostics
2
2
 
3
+ [Read this guide in HTML](https://steveseguin.github.io/ninjamcp/advanced-diagnostics.html)
4
+
3
5
  This page is optional. Use it when you are debugging reliability or network edge cases.
4
6
 
5
7
  ## Live Test Commands
@@ -27,10 +29,10 @@ LIVE_VDO_TEST=1 VDON_MCP_JOIN_TOKEN_SECRET=change_me npm run test:mcp:preset:mat
27
29
 
28
30
  Artifacts are written to:
29
31
 
30
- - `MCP/tests/artifacts/latest-preset-matrix-report.json`
31
- - `MCP/tests/artifacts/latest-preset-matrix-summary.txt`
32
- - `MCP/tests/artifacts/history/preset-matrix-*.json`
33
- - `MCP/tests/artifacts/history/preset-matrix-*.txt`
32
+ - `tests/artifacts/latest-preset-matrix-report.json`
33
+ - `tests/artifacts/latest-preset-matrix-summary.txt`
34
+ - `tests/artifacts/history/preset-matrix-*.json`
35
+ - `tests/artifacts/history/preset-matrix-*.txt`
34
36
 
35
37
  ## Soak / Matrix Knobs
36
38
 
package/references/browser-agent.md ADDED
@@ -0,0 +1,53 @@
1
+ # Talk to an agent while sharing a browser
2
+
3
+ [Read this guide in HTML](https://steveseguin.github.io/ninjamcp/browser-agent.html)
4
+
5
+ The browser companion lets you speak or type to an OpenAI Realtime agent, share selected views of your remote Chrome tab, and let it use the browser tools. You can also watch and hear the tab through VDO.Ninja. The browser host can be on another computer; the companion runs locally beside your microphone and speakers.
6
+
7
+ ## Start a conversation
8
+
9
+ First [start a browser worker](browser-worker.md) and copy one controller credential file to your computer. Use a separate credential for each agent or companion running at the same time.
10
+
11
+ Set `OPENAI_API_KEY` in the local companion process, then launch:
12
+
13
+ ```sh
14
+ npx vdon-mcp-browser-agent --config /private/agent1.json
15
+ ```
16
+
17
+ From a checkout, run `node scripts/vdo-mcp-browser-agent.js --config /private/agent1.json`. Open the private loopback URL printed in the terminal on that same computer. Keep the terminal running. The API key stays in the local server process; it is not sent to the remote browser host or the operator page.
18
+
19
+ 1. Select **Check worker** to confirm authenticated access.
20
+ 2. Select **Start agent**, then choose **Share my microphone** to speak. You can also type a message.
21
+ 3. Enable **Share browser frames** when you want the agent to see the tab. Frames update about every three seconds; the agent can also read page snapshots.
22
+ 4. Enable **Allow browser control** when you want the agent to navigate, click, or type. This also enables browser sharing. The worker's controller role and tab lease still apply.
23
+
24
+ The default model is `gpt-realtime-2.1`. Set `VDON_BROWSER_MODEL` to select another Realtime model that supports audio, images, and function calls. Model usage is billed to your OpenAI account. Each conversation ends after 15 minutes; start a new one to continue.
25
+
26
+ ## Control what is shared
27
+
28
+ | Control | Effect |
29
+ | --- | --- |
30
+ | Share my microphone | Sends microphone audio to OpenAI while the agent session is connected. Turning it off stops microphone capture and clears pending input audio. |
31
+ | Share browser frames | Sends browser images and requested page content to OpenAI. Turning it off stops new image and tool requests and releases held control. Content already sent remains part of that model conversation. |
32
+ | Allow browser control | Allows the agent to claim the shared tab and use input tools. Turning it off releases held leases and prevents queued control requests from starting. |
33
+ | Interrupt | Stops queued speech and agent actions. An action already sent to the worker may finish; inspect its result before repeating it. |
34
+ | End session | Closes the model connection, stops microphone capture and frame sharing, and releases held control. |
35
+
36
+ The sharing indicator identifies the selected inputs and OpenAI as their recipient. The application does not record locally. OpenAI processes the microphone audio, selected browser images, typed messages, and tool results sent during the session. The voice you hear is AI generated.
37
+
38
+ Other authorized observers can still see the worker's tab. To revoke another controller, use `revoke 1` in the host terminal. Stop the worker to close the browser and end access for everyone.
39
+
40
+ ## Watch the browser
41
+
42
+ Enable `"media": { "enabled": true }` in the host's `worker.json`, or add `--media` during setup. Select **Watch live browser** in the companion and **Hear browser audio** to unmute. This uses the SDK stream directly and does not require an OpenAI key. Watching alone does not share that stream with the model.
43
+
44
+ The model receives selected browser frames and your shared microphone; captured tab audio is for the human viewer. Live viewing requires WebRTC. With an HTTP connection, keep using text, microphone conversation, page snapshots, screenshots, and browser controls even when the live view cannot connect. The companion's connection to OpenAI uses an outbound secure WebSocket.
45
+
46
+ ## Resolve connection problems
47
+
48
+ - If **Check worker** fails, run `vdon-mcp-browser --doctor --config /private/agent1.json`. Confirm that the host is running with matching credentials and a common route.
49
+ - If the model will not connect, set the API key in the companion process, check model access, and restart the companion. Viewing and worker diagnostics remain available without a key.
50
+ - If another agent holds the tab, let it release control or wait for its lease to expire. Sharing a browser does not bypass that ownership.
51
+ - If a tool reports an uncertain outcome, have the agent use `browser_result` and inspect the page before repeating an action. A returned tool result and completion of your requested task are separate outcomes.
52
+
53
+ The page accepts one local operator connection. Reopen the original private URL after a reload or disconnect; its access token is removed from the address bar after connection. Keep that URL and the agent credential file private.
package/references/browser-worker.md ADDED
@@ -0,0 +1,142 @@
1
+ # Share a remote Chrome browser
2
+
3
+ [Read this guide in HTML](https://steveseguin.github.io/ninjamcp/browser-worker.html)
4
+
5
+ Run an isolated, headless Chrome tab on one computer and let agents use it through Ninja MCP. The host connects outward using WebRTC or your HTTP relay. Chrome's debugging endpoint stays on loopback; agents need no VPN or desktop streaming software.
6
+
7
+ Each agent gets its own credential. Controllers take turns using a tab lease; observers can read snapshots, request screenshots, and watch optional live tab audio/video.
8
+
9
+ ## Start the host
10
+
11
+ Install Chrome and Node.js 22.12 or later. In your Ninja MCP installation folder, install the optional browser packages:
12
+
13
+ ```sh
14
+ npm install @vdoninja/mcp chrome-devtools-mcp@1.11.0 puppeteer-core@24.37.5
15
+ npx vdon-mcp-browser --setup ./private-browser
16
+ npx vdon-mcp-browser --worker --config ./private-browser/worker.json
17
+ ```
18
+
19
+ The setup prompts for a connection and optional live media. For a direct connection without prompts, add `--transport webrtc`. From a checkout, use `node scripts/vdo-mcp-browser.js` in place of `npx vdon-mcp-browser`. Keep the generated directory outside source control and accessible only to the host account. It contains four credential files:
20
+
21
+ | File | Give it to |
22
+ | --- | --- |
23
+ | `worker.json` | The browser host only; it contains every client key. |
24
+ | `agent1.json`, `agent2.json` | One controller each. |
25
+ | `viewer.json` | An observer who may see the shared tab but cannot control it. |
26
+
27
+ Copy each client file privately to its intended agent's machine. Give additional clients distinct random 32-byte hex keys and add their IDs, keys, and `controller` or `observer` roles to `worker.json`. Restart the worker to apply credential changes. Use one client credential per active agent process.
28
+
29
+ Setup also writes `mcp-client.json` with the local agent registration. Adjust its absolute paths when copying it to another computer. `--init` remains available for creating WebRTC credentials without prompts.
30
+
31
+ The worker starts with one empty shared tab and an ephemeral Chrome profile. Your ordinary Chrome windows and logged-in profile are separate. To select a Chrome executable, add an absolute `executable` path to `worker.json`. When browser packages live elsewhere, set `dependencies` to their installation directory.
32
+
33
+ ## Connect an agent
34
+
35
+ The agent machine needs Ninja MCP, but does not need Chrome or the optional browser packages. Add this separate server to the agent's MCP configuration, using absolute paths:
36
+
37
+ ```json
38
+ {
39
+ "mcpServers": {
40
+ "ninja-browser": {
41
+ "command": "node",
42
+ "args": [
43
+ "/absolute/path/node_modules/@vdoninja/mcp/scripts/vdo-mcp-browser.js",
44
+ "--config",
45
+ "/absolute/private/path/agent1.json"
46
+ ]
47
+ }
48
+ }
49
+ }
50
+ ```
51
+
52
+ On Windows, use paths such as `C:/Ninja/node_modules/@vdoninja/mcp/scripts/vdo-mcp-browser.js`. Existing message, file, and state tools remain available through the ordinary Ninja MCP server.
53
+
54
+ Check the host packages or authenticate to the running worker from a client:
55
+
56
+ ```sh
57
+ npx vdon-mcp-browser --doctor --config ./private-browser/worker.json
58
+ npx vdon-mcp-browser --doctor --config ./private-browser/agent1.json
59
+ ```
60
+
61
+ The client check lists tabs without taking control. Its output distinguishes relay access from an authenticated worker response and names the route that returned the result.
62
+
63
+ Ask the agent to list tabs, claim the shared tab, navigate, and take a snapshot:
64
+
65
+ ```json
66
+ { "name": "browser_tabs", "arguments": {} }
67
+ { "name": "browser_claim", "arguments": { "pageId": 1 } }
68
+ { "name": "browser_call", "arguments": { "pageId": 1, "lease": "<returned lease>", "action": "navigate", "url": "https://example.com" } }
69
+ { "name": "browser_observe", "arguments": { "pageId": 1 } }
70
+ ```
71
+
72
+ Use the returned snapshot token and element UID for input:
73
+
74
+ ```json
75
+ { "name": "browser_call", "arguments": { "pageId": 1, "lease": "<returned lease>", "snapshot": "<snapshot token>", "action": "click", "uid": "<element uid>" } }
76
+ { "name": "browser_observe", "arguments": { "pageId": 1, "screenshot": true } }
77
+ { "name": "browser_release", "arguments": { "pageId": 1, "lease": "<returned lease>" } }
78
+ ```
79
+
80
+ `fill` takes `uid` and `value`; `key` takes a page key such as `Enter`, `Tab`, or an arrow key. Navigation accepts HTTP and HTTPS URLs. Remote tools do not expose script evaluation, local file uploads, browser shortcuts, or the media publisher's private page. They share the initial tab; newly opened tabs are not exposed. Continue work in the shared tab by navigating it to the desired URL.
81
+
82
+ ## Share control and recover a connection
83
+
84
+ A lease lasts 60 seconds by default. Set `ttl_ms` between 1,000 and 120,000 when claiming; renew by passing the same lease to `browser_claim`. Another controller can claim after release or expiry. A disconnected agent's lease expires on that same deadline.
85
+
86
+ Take a fresh accessibility snapshot after claiming, navigating, or acting. Snapshot tokens expire after 30 seconds. Another agent's accessibility snapshot replaces the current element references; take a fresh one if an input call returns `fresh_snapshot_required`. Screenshots do not replace those references.
87
+
88
+ The host can enter `revoke 1` in its worker terminal to invalidate the current lease immediately. An action already executing may finish. Stop the worker to close the browser and end all access.
89
+
90
+ Requests have IDs, deadlines, and bounded retries. A lost response is retried with the same ID; the running worker returns its cached result instead of repeating the action. Recent results remain available for at least two minutes after their request deadline:
91
+
92
+ ```json
93
+ { "name": "browser_result", "arguments": { "request_id": "<uncertain request ID>" } }
94
+ ```
95
+
96
+ An `outcome_unknown` response or an unknown result does not mean the action failed to execute. Inspect the page before deciding whether to repeat a click or submission. Results are held in memory; restarting the worker creates a new browser and invalidates old commands. Results are limited to 2 MiB; screenshots cover the viewport and use JPEG compression.
97
+
98
+ ## Keep working when WebRTC is unavailable
99
+
100
+ For automatic browser routing, create a new group with a private relay configuration:
101
+
102
+ ```sh
103
+ npx vdon-mcp-browser --setup ./private-browser --transport auto --relay-config /private/relay.json
104
+ ```
105
+
106
+ The relay file uses `{ "url": "https://relay.example.com", "token": "<access token>" }`. You can reuse a ninja-p2p relay configuration and the same hosted relay service; setup creates a separate browser room and encryption key. Mailbox credentials and OAuth connections are separate and are not used here.
107
+
108
+ `auto` prefers an available WebRTC data channel. If a request needs retrying, it tries the other available route with the same encrypted request and ID. One worker owns the result cache and control leases across both routes. After a WebRTC timeout, new requests prefer HTTP for 30 seconds while the direct connection recovers. Tool results report which route replied.
109
+
110
+ Use `--transport http` for HTTP only, or `--transport webrtc` for direct connections only. An automatic participant can reach a participant using either one of those routes, provided their connection and browser credentials match. Both routes carry commands, results, and screenshots; live media still needs WebRTC.
111
+
112
+ For an existing group, set `connection.transport` to `"auto"` or `"http"` in its files, and supply the same relay settings to each participating process. These environment variables override the generated settings:
113
+
114
+ - `VDON_MCP_RELAY_URL`: your HTTPS relay origin.
115
+ - `VDON_MCP_RELAY_TOKEN`: relay access token.
116
+ - `VDON_MCP_RELAY_KEY`: room encryption key.
117
+
118
+ Follow the [relay setup guide](http-relay.md) to host a compatible relay. Browser credentials remain separate from relay credentials; sharing a room key does not grant browser access. Automatic routing applies to this browser adapter; ordinary Ninja MCP room sessions select their transport explicitly.
119
+
120
+ ## Watch live tab audio and video
121
+
122
+ Add this option to `worker.json`, then restart:
123
+
124
+ ```json
125
+ "media": { "enabled": true }
126
+ ```
127
+
128
+ Call `browser_media` to get track status and a private VDO.Ninja viewing link. The host captures only the shared tab, including its audio, through Chrome tab capture and publishes it using the VDO.Ninja SDK. The publisher uses a separate browser context and fresh media credentials on every worker start. Microphone and desktop capture are not enabled. Recording is off.
129
+
130
+ Anyone holding the viewing link can watch and hear that stream. Share it only with intended viewers; restart the worker to replace the link. Media access does not grant control. The capture persists when the shared tab navigates, so stop the worker before opening content you do not want viewers to see.
131
+
132
+ Live media requires WebRTC even when commands use HTTP. If `browser_media` reports unavailable media or an ended track, use `browser_observe` screenshots and restart after restoring capture or connectivity. An optional `media.host` selects a different SDK signaling endpoint; clients then receive connection settings instead of a default VDO.Ninja link.
133
+
134
+ For a speaking agent that sees selected browser frames and uses these tools, open the optional [browser companion](browser-agent.md). Live viewing and model sharing have separate controls.
135
+
136
+ ## Choose the host's access deliberately
137
+
138
+ All configured viewers can see the shared tab, and controllers can interact with sites using the host's network access. Run the worker under an account or container with only the network and filesystem access you intend to share. A separate Chrome profile is not an operating-system sandbox. Avoid signing into personal accounts in this shared tab.
139
+
140
+ Browser requests and results use a separate AES-GCM key for each client over either transport. Keep credential files private. Removing a client requires updating the host configuration and restarting; possession of that client's file grants its configured role until then.
141
+
142
+ The adapter uses the pinned [Chrome DevTools MCP](https://github.com/ChromeDevTools/chrome-devtools-mcp) release and [VDO.Ninja SDK](https://sdk.vdo.ninja/). Chrome runs with its sandbox enabled. The worker binds no public browser-control port and changes no ordinary Ninja MCP transport defaults.