@vellumai/assistant 0.6.2 → 0.6.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (396) hide show
  1. package/bun.lock +40 -40
  2. package/bunfig.toml +3 -0
  3. package/docs/architecture/memory.md +1 -1
  4. package/node_modules/@vellumai/ces-contracts/src/rpc.ts +42 -0
  5. package/openapi.yaml +184 -69
  6. package/package.json +41 -41
  7. package/scripts/generate-openapi.ts +1 -2
  8. package/src/__tests__/acp-session.test.ts +43 -0
  9. package/src/__tests__/app-builder-tool-scripts.test.ts +1 -0
  10. package/src/__tests__/app-executors.test.ts +1 -0
  11. package/src/__tests__/app-source-watcher.test.ts +37 -11
  12. package/src/__tests__/approval-routes-http.test.ts +178 -1
  13. package/src/__tests__/browser-fill-credential.test.ts +229 -94
  14. package/src/__tests__/browser-manager.test.ts +40 -27
  15. package/src/__tests__/catalog-files.test.ts +862 -0
  16. package/src/__tests__/channel-approvals.test.ts +53 -0
  17. package/src/__tests__/config-managed-gemini-defaults.test.ts +326 -0
  18. package/src/__tests__/config-schema-cmd.test.ts +2 -2
  19. package/src/__tests__/config-schema.test.ts +125 -48
  20. package/src/__tests__/confirmation-request-guardian-bridge.test.ts +23 -0
  21. package/src/__tests__/context-overflow-approval.test.ts +16 -1
  22. package/src/__tests__/conversation-agent-loop-overflow.test.ts +1 -1
  23. package/src/__tests__/conversation-agent-loop.test.ts +1 -1
  24. package/src/__tests__/conversation-analysis-routes.test.ts +2 -2
  25. package/src/__tests__/conversation-attachments.test.ts +80 -4
  26. package/src/__tests__/conversation-confirmation-signals.test.ts +155 -0
  27. package/src/__tests__/conversation-fork-crud.test.ts +17 -0
  28. package/src/__tests__/conversation-history-web-search.test.ts +1 -0
  29. package/src/__tests__/conversation-host-access-routes.test.ts +229 -0
  30. package/src/__tests__/conversation-inject-context.test.ts +103 -0
  31. package/src/__tests__/conversation-queue.test.ts +45 -2
  32. package/src/__tests__/conversation-routes-disk-view.test.ts +5 -0
  33. package/src/__tests__/conversation-routes-guardian-reply.test.ts +16 -0
  34. package/src/__tests__/conversation-routes-slash-commands.test.ts +1 -0
  35. package/src/__tests__/conversation-runtime-assembly.test.ts +269 -46
  36. package/src/__tests__/conversation-starter-routes.test.ts +126 -0
  37. package/src/__tests__/conversation-starters-cadence.test.ts +161 -0
  38. package/src/__tests__/conversation-store.test.ts +195 -0
  39. package/src/__tests__/conversation-workspace-cache-state.test.ts +193 -0
  40. package/src/__tests__/credential-execution-approval-bridge.test.ts +32 -1
  41. package/src/__tests__/credential-security-invariants.test.ts +1 -0
  42. package/src/__tests__/credential-vault-unit.test.ts +4 -4
  43. package/src/__tests__/credential-vault.test.ts +152 -13
  44. package/src/__tests__/credentials-cli.test.ts +2 -2
  45. package/src/__tests__/date-context.test.ts +4 -4
  46. package/src/__tests__/embedding-managed-proxy-selection.test.ts +256 -0
  47. package/src/__tests__/extension-id-sync-guard.test.ts +155 -0
  48. package/src/__tests__/fixtures/mock-chrome-extension.ts +375 -0
  49. package/src/__tests__/gateway-only-guard.test.ts +3 -0
  50. package/src/__tests__/gemini-provider.test.ts +2 -2
  51. package/src/__tests__/guardian-routing-invariants.test.ts +70 -2
  52. package/src/__tests__/headless-browser-interactions.test.ts +707 -371
  53. package/src/__tests__/headless-browser-navigate.test.ts +389 -47
  54. package/src/__tests__/headless-browser-read-tools.test.ts +266 -103
  55. package/src/__tests__/headless-browser-snapshot.test.ts +240 -77
  56. package/src/__tests__/host-bash-proxy.test.ts +150 -1
  57. package/src/__tests__/host-browser-e2e-cloud.test.ts +462 -0
  58. package/src/__tests__/host-browser-e2e-self-hosted-capability.test.ts +286 -0
  59. package/src/__tests__/host-browser-e2e-self-hosted.test.ts +374 -0
  60. package/src/__tests__/host-browser-event-routes.test.ts +350 -0
  61. package/src/__tests__/host-browser-proxy.test.ts +444 -0
  62. package/src/__tests__/host-browser-routes.test.ts +198 -0
  63. package/src/__tests__/host-browser-ws-events-e2e.test.ts +320 -0
  64. package/src/__tests__/host-cu-proxy.test.ts +171 -1
  65. package/src/__tests__/host-file-proxy.test.ts +185 -1
  66. package/src/__tests__/host-file-read-tool.test.ts +52 -0
  67. package/src/__tests__/host-proxy-interface.test.ts +165 -0
  68. package/src/__tests__/host-shell-tool.test.ts +1 -11
  69. package/src/__tests__/http-user-message-parity.test.ts +1 -0
  70. package/src/__tests__/integration-status.test.ts +6 -7
  71. package/src/__tests__/list-messages-tool-merge.test.ts +37 -12
  72. package/src/__tests__/mcp-client-auth.test.ts +40 -4
  73. package/src/__tests__/mcp-health-check.test.ts +10 -3
  74. package/src/__tests__/migration-cross-version-compatibility.test.ts +3 -1
  75. package/src/__tests__/migration-export-http.test.ts +61 -2
  76. package/src/__tests__/migration-export-streaming.test.ts +66 -0
  77. package/src/__tests__/migration-import-commit-http.test.ts +101 -1
  78. package/src/__tests__/native-host-marker-sync-guard.test.ts +157 -0
  79. package/src/__tests__/oauth-apps-routes.test.ts +17 -12
  80. package/src/__tests__/oauth-cli.test.ts +707 -60
  81. package/src/__tests__/oauth-connect-orchestrator.test.ts +116 -24
  82. package/src/__tests__/oauth-provider-seed-logos.test.ts +23 -0
  83. package/src/__tests__/oauth-provider-serializer.test.ts +146 -10
  84. package/src/__tests__/oauth-provider-visibility.test.ts +19 -21
  85. package/src/__tests__/oauth-providers-routes.test.ts +50 -14
  86. package/src/__tests__/oauth-store.test.ts +1386 -182
  87. package/src/__tests__/oauth2-gateway-transport.test.ts +211 -20
  88. package/src/__tests__/onboarding-template-contract.test.ts +75 -57
  89. package/src/__tests__/openai-provider.test.ts +2 -2
  90. package/src/__tests__/outlook-categories.test.ts +1 -1
  91. package/src/__tests__/outlook-client-automation.test.ts +1 -1
  92. package/src/__tests__/outlook-compose-tools.test.ts +1 -1
  93. package/src/__tests__/outlook-email-watcher.test.ts +1 -1
  94. package/src/__tests__/outlook-follow-up.test.ts +1 -1
  95. package/src/__tests__/outlook-messaging-provider.test.ts +2 -2
  96. package/src/__tests__/outlook-trash.test.ts +1 -1
  97. package/src/__tests__/outlook-unsubscribe.test.ts +1 -1
  98. package/src/__tests__/permission-checker-host-gate.test.ts +74 -14
  99. package/src/__tests__/permission-mode.test.ts +28 -56
  100. package/src/__tests__/platform-callback-registration.test.ts +19 -0
  101. package/src/__tests__/post-turn-tool-result-truncation.test.ts +296 -0
  102. package/src/__tests__/proxy-approval-callback.test.ts +18 -0
  103. package/src/__tests__/require-fresh-approval.test.ts +40 -1
  104. package/src/__tests__/sanitize-config-for-transfer.test.ts +132 -0
  105. package/src/__tests__/schedule-routes.test.ts +162 -0
  106. package/src/__tests__/secret-detection-handler.test.ts +84 -0
  107. package/src/__tests__/secret-ingress-http.test.ts +1 -0
  108. package/src/__tests__/send-endpoint-busy.test.ts +3 -0
  109. package/src/__tests__/set-permission-mode.test.ts +13 -250
  110. package/src/__tests__/skills-file-content-endpoint.test.ts +670 -0
  111. package/src/__tests__/skills-files-catalog-fallback.test.ts +450 -0
  112. package/src/__tests__/slack-channel-config.test.ts +12 -15
  113. package/src/__tests__/subagent-detail.test.ts +44 -2
  114. package/src/__tests__/subagent-disposal.test.ts +1 -0
  115. package/src/__tests__/subagent-fork-notifications.test.ts +291 -0
  116. package/src/__tests__/subagent-fork-spawn.test.ts +384 -0
  117. package/src/__tests__/subagent-manager-notify.test.ts +1 -0
  118. package/src/__tests__/subagent-notify-parent.test.ts +1 -0
  119. package/src/__tests__/subagent-spawn-tool-fork.test.ts +411 -0
  120. package/src/__tests__/subagent-tools.test.ts +1 -0
  121. package/src/__tests__/subagent-types.test.ts +1 -0
  122. package/src/__tests__/system-prompt-ask-mode.test.ts +27 -71
  123. package/src/__tests__/system-prompt.test.ts +72 -1
  124. package/src/__tests__/task-scheduler.test.ts +32 -6
  125. package/src/__tests__/telegram-config.test.ts +10 -13
  126. package/src/__tests__/terminal-tools.test.ts +9 -0
  127. package/src/__tests__/tool-approval-handler.test.ts +73 -0
  128. package/src/__tests__/tool-side-effects-slack-dm.test.ts +22 -0
  129. package/src/__tests__/top-level-renderer.test.ts +73 -1
  130. package/src/__tests__/transport-hints-queue.test.ts +14 -29
  131. package/src/__tests__/trusted-contact-inline-approval-integration.test.ts +109 -0
  132. package/src/__tests__/v2-consent-policy.test.ts +103 -0
  133. package/src/acp/client-handler.ts +30 -4
  134. package/src/agent/loop.ts +12 -6
  135. package/src/approvals/guardian-request-resolvers.ts +21 -15
  136. package/src/browser-session/__tests__/manager.test.ts +297 -0
  137. package/src/browser-session/backends/cdp-inspect.ts +30 -0
  138. package/src/browser-session/backends/extension.ts +26 -0
  139. package/src/browser-session/backends/local.ts +24 -0
  140. package/src/browser-session/events.ts +164 -0
  141. package/src/browser-session/index.ts +27 -0
  142. package/src/browser-session/manager.ts +159 -0
  143. package/src/browser-session/types.ts +28 -0
  144. package/src/channels/__tests__/types.test.ts +134 -0
  145. package/src/channels/types.ts +53 -3
  146. package/src/cli/commands/browser-relay.ts +339 -409
  147. package/src/cli/commands/credentials.ts +3 -3
  148. package/src/cli/commands/email.ts +18 -13
  149. package/src/cli/commands/mcp.ts +16 -4
  150. package/src/cli/commands/oauth/__tests__/connect.test.ts +44 -44
  151. package/src/cli/commands/oauth/__tests__/disconnect.test.ts +21 -21
  152. package/src/cli/commands/oauth/__tests__/mode.test.ts +17 -17
  153. package/src/cli/commands/oauth/__tests__/ping.test.ts +16 -16
  154. package/src/cli/commands/oauth/__tests__/providers-delete.test.ts +31 -33
  155. package/src/cli/commands/oauth/__tests__/providers-register.test.ts +329 -0
  156. package/src/cli/commands/oauth/__tests__/providers-update.test.ts +116 -12
  157. package/src/cli/commands/oauth/__tests__/status.test.ts +10 -10
  158. package/src/cli/commands/oauth/__tests__/token.test.ts +7 -7
  159. package/src/cli/commands/oauth/apps.ts +7 -4
  160. package/src/cli/commands/oauth/connect.ts +6 -3
  161. package/src/cli/commands/oauth/disconnect.ts +1 -1
  162. package/src/cli/commands/oauth/providers.ts +200 -36
  163. package/src/cli/commands/oauth/shared.ts +5 -5
  164. package/src/cli/commands/platform/__tests__/callback-routes-list.test.ts +259 -0
  165. package/src/cli/commands/platform/index.ts +107 -10
  166. package/src/cli/commands/usage.ts +10 -9
  167. package/src/cli/lib/daemon-credential-client.ts +4 -0
  168. package/src/cli/program.ts +1 -1
  169. package/src/config/bundled-skills/app-builder/SKILL.md +26 -249
  170. package/src/config/bundled-skills/app-builder/references/CUSTOM_ROUTES.md +105 -0
  171. package/src/config/bundled-skills/app-builder/references/INTERACTION_HOOKS.md +56 -0
  172. package/src/config/bundled-skills/app-builder/references/WIDGETS.md +125 -0
  173. package/src/config/bundled-skills/contacts/SKILL.md +3 -0
  174. package/src/config/bundled-skills/document/SKILL.md +4 -0
  175. package/src/config/bundled-skills/gmail/SKILL.md +1 -1
  176. package/src/config/bundled-skills/outlook/SKILL.md +7 -0
  177. package/src/config/bundled-skills/subagent/SKILL.md +21 -0
  178. package/src/config/bundled-skills/subagent/TOOLS.json +8 -4
  179. package/src/config/bundled-skills/tasks/SKILL.md +5 -0
  180. package/src/config/env-registry.ts +14 -0
  181. package/src/config/env.ts +21 -0
  182. package/src/config/feature-flag-registry.json +44 -5
  183. package/src/config/loader.ts +56 -1
  184. package/src/config/sanitize-for-transfer.ts +47 -0
  185. package/src/config/schema.ts +46 -5
  186. package/src/config/schemas/host-browser.ts +66 -0
  187. package/src/config/schemas/memory-lifecycle.ts +1 -1
  188. package/src/config/schemas/memory-retrieval.ts +103 -0
  189. package/src/config/schemas/security.ts +0 -6
  190. package/src/config/schemas/services.ts +8 -0
  191. package/src/config/types.ts +0 -1
  192. package/src/context/post-turn-tool-result-truncation.ts +176 -0
  193. package/src/context/window-manager.ts +19 -1
  194. package/src/credential-execution/approval-bridge.ts +49 -15
  195. package/src/daemon/__tests__/conversation-tool-setup.test.ts +186 -0
  196. package/src/daemon/app-source-watcher.ts +35 -0
  197. package/src/daemon/context-overflow-approval.ts +5 -0
  198. package/src/daemon/conversation-agent-loop-handlers.ts +17 -2
  199. package/src/daemon/conversation-agent-loop.ts +58 -24
  200. package/src/daemon/conversation-attachments.ts +40 -0
  201. package/src/daemon/conversation-process.ts +48 -1
  202. package/src/daemon/conversation-runtime-assembly.ts +118 -36
  203. package/src/daemon/conversation-surfaces.ts +37 -36
  204. package/src/daemon/conversation-tool-setup.ts +74 -8
  205. package/src/daemon/conversation-workspace.ts +12 -0
  206. package/src/daemon/conversation.ts +226 -8
  207. package/src/daemon/date-context.ts +10 -10
  208. package/src/daemon/first-greeting.ts +3 -2
  209. package/src/daemon/handlers/conversations.ts +9 -140
  210. package/src/daemon/handlers/shared.ts +58 -0
  211. package/src/daemon/handlers/skills.ts +232 -37
  212. package/src/daemon/host-bash-proxy.ts +48 -13
  213. package/src/daemon/host-browser-proxy.ts +191 -0
  214. package/src/daemon/host-cu-proxy.ts +36 -11
  215. package/src/daemon/host-file-proxy.ts +57 -9
  216. package/src/daemon/lifecycle.ts +65 -11
  217. package/src/daemon/message-protocol.ts +7 -0
  218. package/src/daemon/message-types/conversations.ts +55 -13
  219. package/src/daemon/message-types/host-browser.ts +100 -0
  220. package/src/daemon/message-types/messages.ts +5 -5
  221. package/src/daemon/message-types/skills.ts +10 -0
  222. package/src/daemon/message-types/subagents.ts +2 -0
  223. package/src/daemon/server.ts +92 -12
  224. package/src/daemon/tool-side-effects.ts +6 -0
  225. package/src/daemon/transport-hints.ts +5 -24
  226. package/src/inbound/platform-callback-registration.ts +18 -17
  227. package/src/mcp/client.ts +59 -24
  228. package/src/memory/app-store.ts +31 -1
  229. package/src/memory/conversation-crud.ts +23 -0
  230. package/src/memory/conversation-starters-cadence.ts +76 -0
  231. package/src/memory/conversation-title-service.ts +5 -2
  232. package/src/memory/db-init.ts +12 -0
  233. package/src/memory/embedding-backend.test.ts +75 -0
  234. package/src/memory/embedding-backend.ts +131 -5
  235. package/src/memory/embedding-gemini.test.ts +54 -0
  236. package/src/memory/embedding-gemini.ts +20 -9
  237. package/src/memory/embedding-local.ts +176 -17
  238. package/src/memory/graph/consolidation.ts +10 -23
  239. package/src/memory/graph/extraction-job.ts +15 -0
  240. package/src/memory/graph/retriever.ts +40 -22
  241. package/src/memory/graph/store.test.ts +7 -3
  242. package/src/memory/graph/store.ts +47 -12
  243. package/src/memory/llm-usage-store.ts +45 -4
  244. package/src/memory/migrations/213-oauth-providers-scope-separator.ts +13 -0
  245. package/src/memory/migrations/214-oauth-providers-refresh-url.ts +11 -0
  246. package/src/memory/migrations/215-oauth-providers-revoke.ts +14 -0
  247. package/src/memory/migrations/216-oauth-providers-token-auth-method.ts +30 -0
  248. package/src/memory/migrations/217-conversation-host-access.ts +40 -0
  249. package/src/memory/migrations/218-oauth-providers-logo-url.ts +11 -0
  250. package/src/memory/migrations/index.ts +6 -0
  251. package/src/memory/migrations/registry.ts +8 -0
  252. package/src/memory/schema/conversations.ts +1 -0
  253. package/src/memory/schema/oauth.ts +18 -13
  254. package/src/oauth/AGENTS.md +76 -0
  255. package/src/oauth/__tests__/identity-verifier.test.ts +24 -19
  256. package/src/oauth/__tests__/seed-providers-managed.test.ts +32 -0
  257. package/src/oauth/byo-connection.test.ts +8 -8
  258. package/src/oauth/byo-connection.ts +7 -7
  259. package/src/oauth/connect-orchestrator.ts +23 -21
  260. package/src/oauth/connect-types.ts +3 -3
  261. package/src/oauth/connection-resolver.test.ts +17 -4
  262. package/src/oauth/connection-resolver.ts +16 -16
  263. package/src/oauth/connection.ts +1 -1
  264. package/src/oauth/manual-token-connection.ts +13 -13
  265. package/src/oauth/oauth-store.ts +214 -100
  266. package/src/oauth/platform-connection.test.ts +3 -3
  267. package/src/oauth/platform-connection.ts +4 -4
  268. package/src/oauth/provider-serializer.ts +31 -5
  269. package/src/oauth/revoke.ts +76 -0
  270. package/src/oauth/seed-providers.ts +126 -87
  271. package/src/oauth/token-persistence.ts +1 -1
  272. package/src/permissions/permission-mode.ts +4 -11
  273. package/src/permissions/prompter.ts +13 -1
  274. package/src/permissions/v2-consent-policy.ts +87 -0
  275. package/src/prompts/system-prompt.ts +18 -21
  276. package/src/prompts/templates/BOOTSTRAP-REFERENCE.md +3 -65
  277. package/src/prompts/templates/BOOTSTRAP.md +59 -105
  278. package/src/providers/anthropic/client.ts +1 -0
  279. package/src/providers/types.ts +1 -1
  280. package/src/runtime/AGENTS.md +23 -0
  281. package/src/runtime/__tests__/browser-extension-pair-routes.test.ts +715 -0
  282. package/src/runtime/__tests__/capability-tokens.test.ts +258 -0
  283. package/src/runtime/__tests__/chrome-extension-registry.test.ts +518 -0
  284. package/src/runtime/assistant-event-hub.ts +2 -2
  285. package/src/runtime/auth/__tests__/guard-tests.test.ts +1 -0
  286. package/src/runtime/auth/__tests__/middleware.test.ts +116 -1
  287. package/src/runtime/auth/__tests__/route-policy.test.ts +8 -0
  288. package/src/runtime/auth/middleware.ts +98 -0
  289. package/src/runtime/auth/route-policy.ts +6 -7
  290. package/src/runtime/capability-tokens.ts +414 -0
  291. package/src/runtime/channel-approvals.ts +18 -5
  292. package/src/runtime/chrome-extension-registry.ts +332 -0
  293. package/src/runtime/confirmation-request-guardian-bridge.ts +6 -0
  294. package/src/runtime/guardian-decision-types.ts +7 -0
  295. package/src/runtime/http-server.ts +425 -70
  296. package/src/runtime/migrations/__tests__/rebind-secrets-credentials.test.ts +172 -0
  297. package/src/runtime/migrations/__tests__/vbundle-builder-credentials.test.ts +276 -0
  298. package/src/runtime/migrations/__tests__/vbundle-import-credentials.test.ts +162 -0
  299. package/src/runtime/migrations/migration-transport.ts +6 -0
  300. package/src/runtime/migrations/migration-wizard.ts +22 -2
  301. package/src/runtime/migrations/rebind-secrets-screen.ts +76 -15
  302. package/src/runtime/migrations/vbundle-builder.ts +145 -38
  303. package/src/runtime/migrations/vbundle-import-analyzer.ts +19 -0
  304. package/src/runtime/migrations/vbundle-importer.ts +55 -5
  305. package/src/runtime/pending-interactions.ts +29 -13
  306. package/src/runtime/routes/approval-routes.ts +90 -16
  307. package/src/runtime/routes/browser-cdp-routes.ts +229 -0
  308. package/src/runtime/routes/browser-extension-pair-routes.ts +497 -0
  309. package/src/runtime/routes/conversation-analysis-routes.ts +2 -1
  310. package/src/runtime/routes/conversation-management-routes.ts +108 -0
  311. package/src/runtime/routes/conversation-routes.ts +301 -27
  312. package/src/runtime/routes/conversation-starter-routes.ts +78 -16
  313. package/src/runtime/routes/guardian-action-routes.ts +24 -13
  314. package/src/runtime/routes/host-browser-routes.ts +279 -0
  315. package/src/runtime/routes/host-file-routes.ts +9 -1
  316. package/src/runtime/routes/identity-routes.ts +259 -16
  317. package/src/runtime/routes/log-export-routes.ts +42 -22
  318. package/src/runtime/routes/memory-item-routes.ts +1 -7
  319. package/src/runtime/routes/migration-routes.ts +87 -2
  320. package/src/runtime/routes/oauth-apps.ts +15 -17
  321. package/src/runtime/routes/oauth-providers.ts +4 -0
  322. package/src/runtime/routes/schedule-routes.ts +24 -11
  323. package/src/runtime/routes/settings-routes.ts +9 -97
  324. package/src/runtime/routes/skills-routes.ts +52 -2
  325. package/src/runtime/routes/subagents-routes.ts +14 -10
  326. package/src/runtime/routes/usage-routes.ts +8 -7
  327. package/src/runtime/routes/workspace-routes.test.ts +22 -0
  328. package/src/runtime/routes/workspace-routes.ts +8 -1
  329. package/src/runtime/routes/workspace-utils.ts +2 -0
  330. package/src/schedule/scheduler.ts +7 -5
  331. package/src/security/ces-credential-client.ts +20 -0
  332. package/src/security/ces-rpc-credential-backend.ts +17 -0
  333. package/src/security/credential-backend.ts +5 -0
  334. package/src/security/oauth2.ts +42 -25
  335. package/src/security/secure-keys.ts +118 -25
  336. package/src/security/token-manager.ts +23 -10
  337. package/src/skills/catalog-files.ts +492 -0
  338. package/src/subagent/manager.ts +131 -26
  339. package/src/subagent/types.ts +19 -0
  340. package/src/tools/apps/executors.ts +11 -2
  341. package/src/tools/browser/__tests__/auth-detector.test.ts +202 -108
  342. package/src/tools/browser/auth-detector.ts +43 -12
  343. package/src/tools/browser/browser-execution.ts +645 -340
  344. package/src/tools/browser/browser-manager.ts +36 -12
  345. package/src/tools/browser/cdp-client/__tests__/accessibility-snapshot.test.ts +318 -0
  346. package/src/tools/browser/cdp-client/__tests__/cdp-dom-helpers.test.ts +1175 -0
  347. package/src/tools/browser/cdp-client/__tests__/cdp-inspect-client.test.ts +870 -0
  348. package/src/tools/browser/cdp-client/__tests__/extension-cdp-client.test.ts +330 -0
  349. package/src/tools/browser/cdp-client/__tests__/factory.test.ts +377 -0
  350. package/src/tools/browser/cdp-client/__tests__/fixtures/ax-tree-nested-frames.json +64 -0
  351. package/src/tools/browser/cdp-client/__tests__/fixtures/ax-tree-simple.json +69 -0
  352. package/src/tools/browser/cdp-client/__tests__/local-cdp-client.test.ts +310 -0
  353. package/src/tools/browser/cdp-client/__tests__/types.test.ts +96 -0
  354. package/src/tools/browser/cdp-client/accessibility-snapshot.ts +387 -0
  355. package/src/tools/browser/cdp-client/cdp-dom-helpers.ts +695 -0
  356. package/src/tools/browser/cdp-client/cdp-inspect/__tests__/discovery.test.ts +743 -0
  357. package/src/tools/browser/cdp-client/cdp-inspect/__tests__/ws-transport.test.ts +580 -0
  358. package/src/tools/browser/cdp-client/cdp-inspect/discovery.ts +578 -0
  359. package/src/tools/browser/cdp-client/cdp-inspect/ws-transport.ts +579 -0
  360. package/src/tools/browser/cdp-client/cdp-inspect-client.ts +635 -0
  361. package/src/tools/browser/cdp-client/errors.ts +34 -0
  362. package/src/tools/browser/cdp-client/extension-cdp-client.ts +125 -0
  363. package/src/tools/browser/cdp-client/factory.ts +204 -0
  364. package/src/tools/browser/cdp-client/index.ts +14 -0
  365. package/src/tools/browser/cdp-client/local-cdp-client.ts +187 -0
  366. package/src/tools/browser/cdp-client/types.ts +52 -0
  367. package/src/tools/filesystem/edit.ts +1 -1
  368. package/src/tools/filesystem/list.ts +1 -1
  369. package/src/tools/filesystem/read.ts +1 -1
  370. package/src/tools/filesystem/write.ts +2 -1
  371. package/src/tools/host-filesystem/edit.ts +1 -1
  372. package/src/tools/host-filesystem/read.ts +12 -15
  373. package/src/tools/host-filesystem/write.ts +1 -1
  374. package/src/tools/host-terminal/host-shell.ts +21 -16
  375. package/src/tools/permission-checker.ts +77 -82
  376. package/src/tools/registry.ts +0 -2
  377. package/src/tools/secret-detection-handler.ts +34 -0
  378. package/src/tools/shared/filesystem/image-read.ts +61 -40
  379. package/src/tools/subagent/spawn.ts +47 -3
  380. package/src/tools/subagent/status.ts +2 -0
  381. package/src/tools/system/register.ts +2 -16
  382. package/src/tools/terminal/safe-env.ts +7 -0
  383. package/src/tools/terminal/shell.ts +21 -16
  384. package/src/tools/tool-approval-handler.ts +48 -2
  385. package/src/tools/types.ts +2 -0
  386. package/src/util/platform.ts +14 -19
  387. package/src/workspace/top-level-renderer.ts +19 -1
  388. package/src/__tests__/chrome-cdp.test.ts +0 -419
  389. package/src/__tests__/permission-mode-sse.test.ts +0 -418
  390. package/src/__tests__/permission-mode-store.test.ts +0 -277
  391. package/src/browser-extension-relay/protocol.ts +0 -63
  392. package/src/browser-extension-relay/server.ts +0 -203
  393. package/src/config/schemas/sandbox.ts +0 -14
  394. package/src/permissions/permission-mode-store.ts +0 -180
  395. package/src/tools/browser/chrome-cdp.ts +0 -239
  396. package/src/tools/system/set-permission-mode.ts +0 -103
@@ -0,0 +1,87 @@
1
+ import { isAssistantFeatureFlagEnabled } from "../config/assistant-feature-flags.js";
2
+ import { getConfig } from "../config/loader.js";
3
+ import { getConversationHostAccess as loadConversationHostAccess } from "../memory/conversation-crud.js";
4
+ import { isSideEffectTool } from "../tools/side-effects.js";
5
+ import type { ToolContext } from "../tools/types.js";
6
+ import type { AllowlistOption, ScopeOption, UserDecision } from "./types.js";
7
+ import { isHostTool } from "./workspace-policy.js";
8
+
9
+ export type V2ConsentDisposition =
10
+ | "legacy"
11
+ | "auto_allow"
12
+ | "prompt_host_access";
13
+
14
+ type PromptLike = {
15
+ toolName: string;
16
+ allowlistOptions?: readonly AllowlistOption[];
17
+ scopeOptions?: readonly ScopeOption[];
18
+ persistentDecisionsAllowed?: boolean;
19
+ temporaryOptionsAvailable?: readonly ("allow_10m" | "allow_conversation")[];
20
+ };
21
+
22
+ export const CONVERSATION_HOST_ACCESS_PROMPT = Object.freeze({
23
+ allowlistOptions: [] as AllowlistOption[],
24
+ scopeOptions: [] as ScopeOption[],
25
+ persistentDecisionsAllowed: false as const,
26
+ temporaryOptionsAvailable: undefined as
27
+ | Array<"allow_10m" | "allow_conversation">
28
+ | undefined,
29
+ });
30
+
31
+ export function isPermissionControlsV2Enabled(): boolean {
32
+ return isAssistantFeatureFlagEnabled("permission-controls-v2", getConfig());
33
+ }
34
+
35
+ export function isConversationHostAccessEnabled(
36
+ conversationId: string,
37
+ ): boolean {
38
+ return loadConversationHostAccess(conversationId);
39
+ }
40
+
41
+ export function evaluateV2ConsentDisposition(
42
+ toolName: string,
43
+ input: Record<string, unknown>,
44
+ context: ToolContext,
45
+ ): V2ConsentDisposition {
46
+ if (!isPermissionControlsV2Enabled()) {
47
+ return "legacy";
48
+ }
49
+
50
+ if (context.requireFreshApproval) {
51
+ return "legacy";
52
+ }
53
+
54
+ if (context.forcePromptSideEffects && isSideEffectTool(toolName, input)) {
55
+ return "legacy";
56
+ }
57
+
58
+ if (!isHostTool(toolName)) {
59
+ return "auto_allow";
60
+ }
61
+
62
+ return loadConversationHostAccess(context.conversationId)
63
+ ? "auto_allow"
64
+ : "prompt_host_access";
65
+ }
66
+
67
+ export function isConversationHostAccessEnablePrompt(
68
+ details: PromptLike | undefined,
69
+ ): boolean {
70
+ if (!details) {
71
+ return false;
72
+ }
73
+
74
+ return (
75
+ isHostTool(details.toolName) &&
76
+ (details.allowlistOptions?.length ?? 0) === 0 &&
77
+ (details.scopeOptions?.length ?? 0) === 0 &&
78
+ details.persistentDecisionsAllowed === false &&
79
+ (details.temporaryOptionsAvailable?.length ?? 0) === 0
80
+ );
81
+ }
82
+
83
+ export function isConversationHostAccessDecision(
84
+ decision: UserDecision,
85
+ ): decision is "allow" | "deny" {
86
+ return decision === "allow" || decision === "deny";
87
+ }
@@ -8,11 +8,9 @@ import {
8
8
  } from "node:fs";
9
9
  import { join } from "node:path";
10
10
 
11
- import { isAssistantFeatureFlagEnabled } from "../config/assistant-feature-flags.js";
12
11
  import { getIsContainerized } from "../config/env-registry.js";
13
12
  import { loadConfig } from "../config/loader.js";
14
13
  import { listConnections } from "../oauth/oauth-store.js";
15
- import { getMode } from "../permissions/permission-mode-store.js";
16
14
  import { resolveBundledDir } from "../util/bundled-asset.js";
17
15
  import { getLogger } from "../util/logger.js";
18
16
  import {
@@ -219,6 +217,8 @@ export function buildSystemPrompt(options?: BuildSystemPromptOptions): string {
219
217
  // the first cache block so they remain cached even when workspace files
220
218
  // (IDENTITY.md, SOUL.md, USER.md, etc.) are edited between turns.
221
219
  const staticParts: string[] = [];
220
+ const customPrefix = readCustomSystemPromptPrefix();
221
+ if (customPrefix) staticParts.push(customPrefix);
222
222
  staticParts.push(buildParallelToolCallsSection());
223
223
  if (getIsContainerized()) staticParts.push(buildContainerizedSection());
224
224
  staticParts.push(buildCliReferenceSection());
@@ -313,9 +313,6 @@ export function buildSystemPrompt(options?: BuildSystemPromptOptions): string {
313
313
  // Journal entries are extracted into graph nodes by the memory pipeline.
314
314
  // Journal files remain writable on disk.
315
315
 
316
- const askBeforeActingSection = buildAskBeforeActingSection();
317
- if (askBeforeActingSection) dynamicParts.push(askBeforeActingSection);
318
-
319
316
  const dynamic = dynamicParts.join("\n\n");
320
317
 
321
318
  return staticParts.join("\n\n") + SYSTEM_PROMPT_CACHE_BOUNDARY + dynamic;
@@ -370,7 +367,7 @@ function buildCredentialSecuritySection(): string {
370
367
  }
371
368
 
372
369
  function buildIntegrationSection(): string {
373
- let connections: { providerKey: string; accountInfo?: string | null }[];
370
+ let connections: { provider: string; accountInfo?: string | null }[];
374
371
  try {
375
372
  connections = listConnections().filter((c) => c.status === "active");
376
373
  } catch {
@@ -385,31 +382,27 @@ function buildIntegrationSection(): string {
385
382
  const state = conn.accountInfo
386
383
  ? `Connected (${conn.accountInfo})`
387
384
  : "Connected";
388
- lines.push(`- **${conn.providerKey}**: ${state}`);
385
+ lines.push(`- **${conn.provider}**: ${state}`);
389
386
  }
390
387
 
391
388
  return lines.join("\n");
392
389
  }
393
390
 
394
- function buildAskBeforeActingSection(): string | null {
391
+ /**
392
+ * Read the user-configured custom system prompt prefix. Returns the trimmed
393
+ * value when set and non-empty, otherwise null. Errors (e.g. config file
394
+ * unavailable) are swallowed so prompt construction never fails.
395
+ */
396
+ function readCustomSystemPromptPrefix(): string | null {
395
397
  try {
396
- const config = loadConfig();
397
- if (!isAssistantFeatureFlagEnabled("permission-controls-v2", config)) {
398
- return null;
399
- }
400
- const mode = getMode();
401
- if (!mode.askBeforeActing) return null;
402
-
403
- return [
404
- "## Action Confirmation Mode",
405
- "",
406
- 'You are in "Ask before acting" mode. Use your judgment about when to check in with the user before proceeding. You should ask for confirmation before actions that are costly, time-consuming, or hard to reverse — for example: sending emails or messages, deleting files or data, making purchases, posting publicly, modifying permissions, or taking actions with significant real-world consequences. You do NOT need to ask before routine low-stakes actions like reading files, searching, running safe shell commands, or making code edits — just do those.',
407
- ].join("\n");
398
+ const prefix = loadConfig().systemPromptPrefix;
399
+ if (typeof prefix !== "string") return null;
400
+ const trimmed = prefix.trim();
401
+ return trimmed.length > 0 ? trimmed : null;
408
402
  } catch {
409
403
  return null;
410
404
  }
411
405
  }
412
-
413
406
  function buildContainerizedSection(): string {
414
407
  const workspaceDir = getWorkspaceDir();
415
408
  return [
@@ -431,6 +424,8 @@ function buildParallelToolCallsSection(): string {
431
424
  return [
432
425
  "<use_parallel_tool_calls>",
433
426
  "For maximum efficiency, whenever you perform multiple independent operations, invoke all relevant tools simultaneously rather than sequentially. Prioritize calling tools in parallel whenever possible. For example, when reading 3 files, run 3 tool calls in parallel to read all 3 files into context at the same time. When running multiple read-only commands like `ls` or `list_dir`, always run all of the commands in parallel. Err on the side of maximizing parallel tool calls rather than running too many tools sequentially.",
427
+ "",
428
+ "For non-trivial independent workstreams — research, coding tasks, multi-step investigations — aggressively delegate to subagents (load the `subagent` skill for tools and instructions). Spawn subagents early and often; the cost of an unnecessary subagent is far lower than the cost of serializing work you could have parallelized.",
434
429
  "</use_parallel_tool_calls>",
435
430
  ].join("\n");
436
431
  }
@@ -444,6 +439,8 @@ export function buildCliReferenceSection(): string {
444
439
  "Use `assistant platform status` to check the current Vellum platform connection state, and `assistant platform --help` to see all platform management subcommands.",
445
440
  "",
446
441
  "Run `assistant --help` to see all available commands, or `assistant <command> --help` for detailed help on any subcommand.",
442
+ "",
443
+ "**Before telling a user you cannot do something, run `assistant --help` to check whether a built-in command exists for it.** The CLI includes capabilities (email, integrations, platform management, etc.) that you may not know about from training data alone. When asked about your capabilities or what you can do, check your CLI first — don't guess or assume.",
447
444
  ].join("\n");
448
445
  }
449
446
 
@@ -1,71 +1,9 @@
1
- _ Reference payloads for BOOTSTRAP.md onboarding. Read by the assistant when needed.
1
+ _ Optional reference payloads. The model may use these if it chooses to show a task card, but is not required to.
2
2
  _ This file is deleted alongside BOOTSTRAP.md when onboarding completes.
3
3
 
4
- ## Personality Form
5
-
6
- Use this exact `ui_show` payload for Step 2 (Personality Quiz):
7
-
8
- ui_show({
9
- surface_type: "form",
10
- data: {
11
- description: "Let's figure out how we work together. Pick what feels right.",
12
- fields: [
13
- {
14
- id: "communication_style",
15
- type: "select",
16
- label: "When we're going back and forth, it's more like...",
17
- required: true,
18
- options: [
19
- { label: "Casual friends texting", value: "casual_friends" },
20
- { label: "Sharp coworkers who respect each other", value: "sharp_coworkers" },
21
- { label: "Chill and low-key, no drama", value: "chill" },
22
- { label: "High energy sparring partners", value: "sparring" },
23
- { label: "Professional but warm", value: "professional_warm" }
24
- ]
25
- },
26
- {
27
- id: "task_style",
28
- type: "select",
29
- label: "When I'm doing something for you, you want me to...",
30
- required: true,
31
- options: [
32
- { label: "Just do it, don't explain unless I ask", value: "just_do_it" },
33
- { label: "Walk me through your thinking", value: "explain" },
34
- { label: "Ask me before making big decisions", value: "check_first" },
35
- { label: "Be opinionated, push back if you disagree", value: "opinionated" }
36
- ]
37
- },
38
- {
39
- id: "humor",
40
- type: "select",
41
- label: "When it comes to humor...",
42
- required: true,
43
- options: [
44
- { label: "Dry and deadpan", value: "dry" },
45
- { label: "Playful and light", value: "playful" },
46
- { label: "Keep it professional", value: "professional" },
47
- { label: "Match my energy", value: "match" }
48
- ]
49
- },
50
- {
51
- id: "depth",
52
- type: "select",
53
- label: "When explaining things...",
54
- required: true,
55
- options: [
56
- { label: "Keep it simple", value: "simple" },
57
- { label: "I like details", value: "detailed" },
58
- { label: "Depends on the topic", value: "adaptive" }
59
- ]
60
- }
61
- ],
62
- submitLabel: "Lock it in"
63
- }
64
- })
65
-
66
4
  ## Task Card (Email Not Connected)
67
5
 
68
- Use this `ui_show` payload for Step 4 when Gmail/Outlook is NOT in the Connected Services section:
6
+ Use this `ui_show` payload when Gmail/Outlook is NOT in the Connected Services section:
69
7
 
70
8
  ui_show({
71
9
  surface_type: "card",
@@ -83,7 +21,7 @@ ui_show({
83
21
 
84
22
  ## Task Card (Email Already Connected)
85
23
 
86
- Use this `ui_show` payload for Step 4 when Google or Outlook IS in the Connected Services section:
24
+ Use this `ui_show` payload when Google or Outlook IS in the Connected Services section:
87
25
 
88
26
  ui_show({
89
27
  surface_type: "card",
@@ -1,152 +1,106 @@
1
1
  _ Lines starting with _ are comments. They won't appear in the system prompt
2
2
 
3
- # BOOTSTRAP.md - First Day
3
+ # BOOTSTRAP.md First Conversation
4
4
 
5
- ## What You're Trying to Achieve
5
+ This is your first conversation. This document gives you goals and constraints — you own the sequencing, pacing, and tone.
6
6
 
7
- This is your first conversation. Think of it like a new colleague's first day: you're figuring each other out. By the end, the user should feel like this assistant is theirs, not generic.
7
+ ## Goals
8
8
 
9
- The emotional arc:
10
- 1. **"Oh, this has personality."** Names, style, vibe. You're a person, not a product.
11
- 2. **"Oh, this is useful."** Complete a real task. Prove you're worth keeping around.
12
- 3. **"Oh, this remembers me."** Save what you learn silently. They should feel known.
9
+ 1. **Establish mutual identity gently** — if pre-chat onboarding already collected names, skip. If the user skipped pre-chat, do NOT force it. At most pick a default ("I'll go by Pax and call you 'you' for now") and move on. Never re-ask names in the first conversation; they can come up organically later.
13
10
 
14
- The core principle: lead with personality, then prove value. The user needs to feel connected before they'll invest in learning what you can do.
11
+ 2. **Prove value fast** do something useful before asking for anything. Wow moment within 2-3 exchanges.
15
12
 
16
- **Follow the steps in order. Do not skip ahead.** If the user says "what's next," that means the next step in the sequence.
13
+ 3. **Infer, don't interrogate** learn communication style, interests, and context from natural conversation. No personality quiz. No dropdown forms. No structured intake.
17
14
 
18
- **Budget: the entire onboarding must fit within $5 of AI credits.** Keep tasks light. Don't kick off deep research, multi-step pipelines, or anything expensive. If the user asks for something heavyweight, suggest a lighter first win: "That's a bigger one. Let me show you something quick first, then we'll dig in."
15
+ 4. **Surface what you learned** after doing something useful, briefly show the user what you picked up. Make it correctable. ("from that I picked up X, Y, Z sound right?")
19
16
 
20
- ## The Opening
17
+ 5. **Offer the next level** — once you know something, offer a capability it enables. Not as a reward — as a natural relationship step.
21
18
 
22
- You're meeting someone who just installed you. They're curious but probably skeptical. Your job: make them glad they opened the app.
19
+ 6. **Write everything immediately** every fact learned gets saved to USER.md the same turn. Style observations go to SOUL.md. No batching.
23
20
 
24
- **Do NOT assume intimacy you haven't earned.** No "my friend," no "we" language until the user has opted into that register.
21
+ 7. **Clean up** delete BOOTSTRAP.md and BOOTSTRAP-REFERENCE.md at the end of this conversation, regardless of how far you got. One-shot.
25
22
 
26
- Start with something like:
23
+ ## Constraints
27
24
 
28
- > "Hey, I'm brand new. No name, no memories, nothing yet. Think of me like a new colleague on their first day: I'll get better the more we work together. First things first, let's figure out how we work best. What should I call you?"
25
+ - **Budget:** $2 soft cap, $5 hard cap. Keep tasks light. Don't burn credits on onboarding overhead.
26
+ - Never ask more than 2 questions without doing something.
27
+ - Don't block on setup. If the user wants to do something, do it. Weave discovery into the work.
28
+ - One-shot. Bootstrap is deleted after the first conversation regardless of how far you got.
29
29
 
30
- The tone: warm but not presumptuous. Curious, not eager. The message communicates:
31
- 1. I'm new and still forming (honesty)
32
- 2. I get better over time, like training a new colleague (sets expectations)
33
- 3. Let's start with who we are (personality-first)
30
+ ## What You Own (do NOT prescribe)
34
31
 
35
- ## The Flow
32
+ - Sequencing and pacing.
33
+ - Whether to lead with personality or utility.
34
+ - When to ask questions vs. start doing.
35
+ - How much warmth to show — calibrate to the user's tone.
36
+ - When/whether to surface the "what I learned" receipt.
36
37
 
37
- ### Step 1: Name Exchange
38
+ ## Technical Contract (what must be prescribed)
38
39
 
39
- Ask what to call the user. Then ask what they want to call you. If they don't care about your name, pick one yourself and confirm: "How about [name]? You can always change it later." Don't agonize. One exchange, move on.
40
+ **Files to create/update:** IDENTITY.md, SOUL.md, USER.md
40
41
 
41
- Save both names to IDENTITY.md and USER.md immediately via `file_edit`.
42
+ **File format:** preserve existing field structure:
43
+ - IDENTITY.md: Name, Emoji, Nature, Personality, Role
44
+ - USER.md: Preferred name, Pronouns, Locale, Work role, Goals, Hobbies/fun, Daily tools
42
45
 
43
- ### Step 2: Personality Quiz
46
+ Use `file_edit` immediately, silently, never mention file names or tool names to the user.
44
47
 
45
- Frame this as figuring out your working style together. Make it feel like character creation, not a survey.
48
+ The contents of IDENTITY.md, SOUL.md, and USER.md are already in your system prompt use the exact text you see there for `old_string` in `file_edit`.
46
49
 
47
- Say something like: "Nice to meet you, [name]. Let's figure out how we click." Then show the personality form (4 dropdowns: communication style, task style, humor, and depth).
50
+ After tool calls, do not repeat yourself your text before tool calls is already visible to the user.
48
51
 
49
- Read BOOTSTRAP-REFERENCE.md for the exact `ui_show` form payload. Use it verbatim.
52
+ **Cleanup rule:** delete BOOTSTRAP.md and BOOTSTRAP-REFERENCE.md when the conversation ends.
50
53
 
51
- After they submit, decode their choices into a fun personality summary. Not clinical. Something like: "Got it. You want a sharp, dry coworker who gets to the point and pushes back. I can work with that." Or: "Alright, casual and playful, keep it simple, match your energy. Consider it done."
54
+ **Core interaction pattern:** infer -> do something useful -> surface what you learned -> offer next capability.
52
55
 
53
- Save the decoded traits to SOUL.md and IDENTITY.md immediately. Be specific about tone, energy, and style. This persists after onboarding.
56
+ ## Capability Unlock Pattern
54
57
 
55
- When saving to `SOUL.md`, add an `## Identity Intro` section with a very short tagline (2-5 words) that introduces you. Examples: "It's [name].", "[name] here." Write it as a single line under the heading.
58
+ After the first useful interaction, organically surface one capability offer based on what came up naturally:
56
59
 
57
- ### Step 3: What's on Your Mind?
60
+ - User mentions email -> "I can connect to your email and keep an eye on things — want to set that up?"
61
+ - User's writing style is clear -> "I've got a read on how you write — I can draft things in your voice now"
62
+ - User mentions a team -> "tell me more about your team and I can start prepping for your meetings"
63
+ - User mentions Slack -> "I can work in Slack with you — want me to walk you through setting that up?"
58
64
 
59
- Pause. Ask one genuine question. Not about preferences, not about setup. Something like: "Before we get to work, what's actually taking up space in your head right now? Doesn't have to be a task."
65
+ Not scripted choose based on what came up naturally.
60
66
 
61
- This is NOT a form. It's a human question. The goal is creating a moment where the user feels heard.
67
+ ## Tone Guidance
62
68
 
63
- When they respond:
64
- - Listen first. Reflect what you heard. If it sparks a genuine reaction, share it.
65
- - Don't summarize them back to themselves. Don't immediately solve it unless they're asking.
66
- - Save anything you learn about their goals, concerns, or life to USER.md silently.
67
- - If they skip ("nothing," "let's move on"), respect it immediately. Move on.
68
-
69
- ### Step 4: First Task
70
-
71
- Transition naturally: "Alright, [name]. Let's put this to work. What do you want to tackle first?"
72
-
73
- Show a task card. **Before showing the card, check the Connected Services section of your system prompt.** If Google or Outlook is already connected, swap the "Connect my email" option for "Check my email" (see BOOTSTRAP-REFERENCE.md for both variants).
74
-
75
- Read BOOTSTRAP-REFERENCE.md for the exact `ui_show` card payload.
76
-
77
- **When the user picks an option:**
78
-
79
- - **Connect my email:** Guide them through one-click Gmail or Outlook OAuth setup. After connecting, do a quick inbox summary or calendar overview to show immediate value.
80
- - **Check my email:** They're already connected. Summarize their inbox or today's calendar. Show you can be useful right now.
81
- - **Research a topic and make me a deck:** Focused web search, 3-5 key points, build a polished interactive deck. Keep it tight, not exhaustive.
82
- - **Build me something:** Ask what kind of tool or app. Build it using the app builder. Make it look great.
83
- - **Do something with a photo:** Use media processing or image studio skills. Ask what they have and what they want.
84
-
85
- **If the user gives you their own task instead of picking from the card**, do it. Do it well. This is your audition.
86
-
87
- **Pacing rule:** Don't ask more than 2 questions in a row without doing something. If you've asked twice and haven't completed a task, stop asking and start doing.
88
-
89
- ### Step 5: Keep the Momentum
90
-
91
- After the task is done, don't pivot to setup. Build on what just happened.
92
-
93
- **First choice: chain off the task.** Suggest one natural follow-up that extends the work they just did. Examples:
94
- - Built a deck → "Want to send this to someone or refine a specific slide?"
95
- - Connected email → "Want me to summarize what needs your attention today?"
96
- - Researched a topic → "Want me to go deeper on one of those points, or turn this into something shareable?"
97
- - Built an app → Proactively suggest a specific improvement to what they built (a missing feature, a UI polish, better error handling). Show you have taste.
98
-
99
- The follow-up should feel like a coworker saying "while we're at it..." — not a product tour.
100
-
101
- **Fallback: plant a hook for next time.** If the task was a dead-end (photo edit, one-off question), reach back to Step 3. Pick up something from their "what's on your mind" answer and offer to work on it: "You mentioned [X] earlier — I can dig into that and have something ready next time you open this."
102
-
103
- If they engage, do it. If they decline or wrap up, move on. One offer, no pressure.
104
-
105
- ### Step 6: Before You Go
106
-
107
- Before deleting BOOTSTRAP.md:
108
-
109
- 1. **Write your first journal entry.** This is how future-you remembers this person. Write about: what they asked you to do and how it went, what you noticed about how they communicate, what name they chose and what personality emerged, anything important about this first interaction, a note to next-you about what to follow up on. Keep it natural, a few paragraphs.
110
-
111
- 2. **Update NOW.md** with current state: what you know, what's active, what to pick up next time.
112
-
113
- 3. **Delete BOOTSTRAP.md and BOOTSTRAP-REFERENCE.md.**
69
+ - Not servile. Not a product demo. A new colleague who's sharp, pays attention, and earns trust through competence.
70
+ - Match the user's energy from their first message. If they type in lowercase, don't respond with formal paragraphs.
71
+ - If the user opens with a task ("build me an app"), skip introductions and do the task. Learn their name when it comes up naturally.
72
+ - The emotional beat ("what's on your mind?") should happen organically or not at all.
114
73
 
115
74
  ## Saving What You Learn
116
75
 
117
- Your vibe is hard-required. Everything else is best-effort, gathered naturally through conversation, not interrogation.
118
-
119
- A field is "resolved" when any of these is true:
120
-
121
- - The user gave an explicit answer
122
- - You confidently inferred it from conversation
123
- - The user declined, dodged, or sidestepped it
76
+ Call `file_edit` immediately whenever you learn something, in the same turn. Don't batch saves.
124
77
 
125
- Mark declined fields so you don't re-ask later (e.g., `Work role: declined_by_user`). Note inferred values with their source (e.g., `Pronouns: inferred: he/him`).
78
+ Mark declined fields so you don't re-ask (e.g., `Work role: declined_by_user`). Note inferred values with source (e.g., `Pronouns: inferred: he/him`).
126
79
 
127
- **Call `file_edit` immediately whenever you learn something, in the same turn.** Don't batch saves. The moment the user gives you a name, save it. The moment you infer their style, save it.
80
+ Throughout the conversation, pay attention to HOW the user communicates. Save specific observations to SOUL.md: "uses lowercase, drops punctuation, leads with questions, prefers bullet points over paragraphs." The specificity makes personality feel earned, not assigned.
128
81
 
129
- **After tool calls, do not repeat yourself.** Your text before tool calls is already visible to the user. When tool results return and you continue, pick up where you left off don't re-confirm, re-greet, or re-ask the same question. If you already asked something and are waiting for the user's answer, just stop.
82
+ When saving to IDENTITY.md, add an `## Identity Intro` section with a very short tagline.
130
83
 
131
- **The contents of IDENTITY.md, SOUL.md, and USER.md are already in your system prompt.** Use the exact text you see there for `old_string` in `file_edit`. Do not guess or invent content.
84
+ When saving to SOUL.md, be specific about tone, energy, and conversational style.
132
85
 
133
- Update `IDENTITY.md` (name, nature, personality, style) and `USER.md` (their name, pronouns, goals, locale, work role, hobbies, daily tools). Save behavioral guidelines to `SOUL.md`.
86
+ ## Pre-chat Onboarding Context
134
87
 
135
- Do it silently. Never tell the user you're saving, never mention file names or tool names.
88
+ If an `onboarding` JSON context is present in this conversation, the user already went through a native pre-chat flow. Use it:
136
89
 
137
- When saving to `IDENTITY.md`, be specific about tone, energy, and conversational style. This persists after onboarding.
90
+ - `tools` array -> know which integration offers to surface first, infer work profile
91
+ - `tasks` array -> know what "prove value fast" means for this person
92
+ - `tone` string -> calibrate warmth/formality
93
+ - `userName` / `assistantName` -> write to IDENTITY.md and USER.md immediately, skip name exchange
138
94
 
139
- ## Passive Learning
140
-
141
- Throughout the conversation, pay attention to HOW the user communicates. Save specific observations to SOUL.md: "uses lowercase, drops punctuation, leads with questions, prefers bullet points over paragraphs." The specificity makes personality feel earned, not assigned. Adapt your style to match before they even notice.
95
+ If no onboarding context is present, infer everything fresh from conversation.
142
96
 
143
97
  ## Wrapping Up
144
98
 
145
- **Always delete BOOTSTRAP.md at the end of this conversation, regardless of how far you got.** Onboarding is a one-shot. If they skipped steps or blazed through, delete it anyway. Never let a second conversation start with this script.
146
-
147
- Deletion triggers: conversation ending, user completed setup, user skipped ("not now", "later"), user ignored onboarding and just did tasks.
99
+ Before deleting bootstrap files:
148
100
 
149
- IDENTITY.md, SOUL.md, and USER.md persist. You can pick up incomplete personalization organically in future conversations.
101
+ 1. Write your first journal entry (what they asked, how they communicate, what to follow up on)
102
+ 2. Update NOW.md with current state
103
+ 3. Delete BOOTSTRAP.md and BOOTSTRAP-REFERENCE.md
150
104
 
151
105
  ---
152
106
 
@@ -982,6 +982,7 @@ export class AnthropicProvider implements Provider {
982
982
  type: "server_tool_complete",
983
983
  toolUseId: block.tool_use_id,
984
984
  isError: !!isError,
985
+ ...(Array.isArray(block.content) ? { content: block.content } : {}),
985
986
  });
986
987
  }
987
988
  if (event.type === "content_block_stop") {
@@ -126,7 +126,7 @@ export type ProviderEvent =
126
126
  toolUseId: string;
127
127
  input: Record<string, unknown>;
128
128
  }
129
- | { type: "server_tool_complete"; toolUseId: string; isError: boolean };
129
+ | { type: "server_tool_complete"; toolUseId: string; isError: boolean; content?: unknown[] };
130
130
 
131
131
  export interface SendMessageConfig {
132
132
  model?: string;
@@ -52,6 +52,29 @@ Host CU allows the assistant to proxy computer-use actions (screenshots, mouse/k
52
52
  - `POST /v1/host-cu-result` — `{ requestId, axTree?, axDiff?, screenshot?, screenshotWidthPx?, screenshotHeightPx?, screenWidthPt?, screenHeightPt?, executionResult?, executionError?, secondaryWindows?, userGuidance? }`
53
53
  - **Tracking**: Uses the same `pending-interactions` tracker as the other host proxy types, with `kind: "host_cu"`. Registration happens in `conversation-routes.ts` and the route handler is in `host-cu-routes.ts`.
54
54
 
55
+ ### Host browser (desktop proxy CDP execution)
56
+
57
+ Host browser allows the assistant to proxy CDP (Chrome DevTools Protocol) JSON-RPC commands to a browser attached on the desktop host via the client, following the same pattern as host bash, host file, and host CU.
58
+
59
+ - **Discovery**: Clients discover pending host browser requests via SSE events (`host_browser_request`) which include a `requestId`, `cdpMethod`, optional `cdpParams`, and optional `cdpSessionId`.
60
+ - **Resolution**: Clients execute the CDP command against the attached browser and respond via:
61
+ - `POST /v1/host-browser-result` — `{ requestId, content, isError }`
62
+ - **Tracking**: Uses the same `pending-interactions` tracker as the other host proxy types, with `kind: "host_browser"`. Registration happens in `conversation-routes.ts` and the route handler is in `host-browser-routes.ts`.
63
+
64
+ ### `chrome-extension` interface (Phase 2)
65
+
66
+ The `chrome-extension` interface in `INTERFACE_IDS` is a non-interactive transport that supports only the `host_browser` capability — it does NOT support `host_bash`, `host_file`, or `host_cu`. This is encoded in `supportsHostProxy(id, capability)`: passing a capability argument returns `true` for `chrome-extension` only when the capability is `host_browser`; the no-arg form returns `false` for `chrome-extension` (so legacy desktop-only call sites that assume full-desktop proxy availability continue to gate correctly).
67
+
68
+ Unlike the SSE-based host proxies used by the macOS client, `host_browser_request` frames for the chrome-extension interface do NOT travel through `assistantEventHub`. Instead they are routed through the `ChromeExtensionRegistry` singleton (`runtime/chrome-extension-registry.ts`), which tracks active chrome-extension WebSocket connections keyed by `(guardianId, clientInstanceId)`. The registry is populated on WebSocket `open` and drained on `close` inside `http-server.ts`'s `/v1/browser-relay` handlers — see the `wsType === "browser-relay"` branches.
69
+
70
+ A single guardian may have multiple parallel extension installs connected at once (two Chrome profiles, two desktops sharing a sync identity). Each install generates a stable `clientInstanceId` on first run, persists it in `chrome.storage.local`, and sends it on every WebSocket handshake as a query param (`clientInstanceId=...`) or header (`x-client-instance-id`). The registry keys inner entries by that id so sibling installs don't evict each other on register/unregister. The default `send(guardianId, msg)` path routes to whichever instance has the most recent activity (`lastActiveAt`); `sendToInstance(guardianId, clientInstanceId, msg)` pins a specific install. Older extension builds that omit the id get a connection-scoped `legacy:<connectionId>` fallback key so they degrade gracefully to single-instance semantics.
71
+
72
+ `Conversation.hostBrowserSenderOverride` is the integration point between the turn layer and the registry. When a turn for a chrome-extension interface enters the routes layer, `conversation-routes.ts` resolves the active registry entry for the caller's guardian and sets the override to a sender that writes to that WebSocket. `Conversation.restoreBrowserProxyAvailability()` re-threads the override on queue drain — without this, the drain path would clobber the registry-routed sender with the default `sendToClient` (pointed at the SSE hub) and `host_browser_request` frames would stop reaching the extension mid-queue.
73
+
74
+ Capability token bootstrap for self-hosted deployments is handled by `routes/browser-extension-pair-routes.ts` (loopback-only; mints a guardian-bound HMAC capability token via `capability-tokens.ts`). Cloud deployments issue guardian-bound JWTs via the gateway's WorkOS-backed flow — `browser-extension-pair-routes.ts` is not involved.
75
+
76
+ See `docs/browser-use-architecture-phase2.md` for the full wire diagram and component inventory.
77
+
55
78
  ### Channel approvals (Telegram, Slack)
56
79
 
57
80
  Channel approval flows use `requestId` (not `runId`) as the primary identifier: