@vellumai/assistant 0.6.2 → 0.6.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bun.lock +40 -40
- package/bunfig.toml +3 -0
- package/docs/architecture/memory.md +1 -1
- package/node_modules/@vellumai/ces-contracts/src/rpc.ts +42 -0
- package/openapi.yaml +184 -69
- package/package.json +41 -41
- package/scripts/generate-openapi.ts +1 -2
- package/src/__tests__/acp-session.test.ts +43 -0
- package/src/__tests__/app-builder-tool-scripts.test.ts +1 -0
- package/src/__tests__/app-executors.test.ts +1 -0
- package/src/__tests__/app-source-watcher.test.ts +37 -11
- package/src/__tests__/approval-routes-http.test.ts +178 -1
- package/src/__tests__/browser-fill-credential.test.ts +229 -94
- package/src/__tests__/browser-manager.test.ts +40 -27
- package/src/__tests__/catalog-files.test.ts +862 -0
- package/src/__tests__/channel-approvals.test.ts +53 -0
- package/src/__tests__/config-managed-gemini-defaults.test.ts +326 -0
- package/src/__tests__/config-schema-cmd.test.ts +2 -2
- package/src/__tests__/config-schema.test.ts +125 -48
- package/src/__tests__/confirmation-request-guardian-bridge.test.ts +23 -0
- package/src/__tests__/context-overflow-approval.test.ts +16 -1
- package/src/__tests__/conversation-agent-loop-overflow.test.ts +1 -1
- package/src/__tests__/conversation-agent-loop.test.ts +1 -1
- package/src/__tests__/conversation-analysis-routes.test.ts +2 -2
- package/src/__tests__/conversation-attachments.test.ts +80 -4
- package/src/__tests__/conversation-confirmation-signals.test.ts +155 -0
- package/src/__tests__/conversation-fork-crud.test.ts +17 -0
- package/src/__tests__/conversation-history-web-search.test.ts +1 -0
- package/src/__tests__/conversation-host-access-routes.test.ts +229 -0
- package/src/__tests__/conversation-inject-context.test.ts +103 -0
- package/src/__tests__/conversation-queue.test.ts +45 -2
- package/src/__tests__/conversation-routes-disk-view.test.ts +5 -0
- package/src/__tests__/conversation-routes-guardian-reply.test.ts +16 -0
- package/src/__tests__/conversation-routes-slash-commands.test.ts +1 -0
- package/src/__tests__/conversation-runtime-assembly.test.ts +269 -46
- package/src/__tests__/conversation-starter-routes.test.ts +126 -0
- package/src/__tests__/conversation-starters-cadence.test.ts +161 -0
- package/src/__tests__/conversation-store.test.ts +195 -0
- package/src/__tests__/conversation-workspace-cache-state.test.ts +193 -0
- package/src/__tests__/credential-execution-approval-bridge.test.ts +32 -1
- package/src/__tests__/credential-security-invariants.test.ts +1 -0
- package/src/__tests__/credential-vault-unit.test.ts +4 -4
- package/src/__tests__/credential-vault.test.ts +152 -13
- package/src/__tests__/credentials-cli.test.ts +2 -2
- package/src/__tests__/date-context.test.ts +4 -4
- package/src/__tests__/embedding-managed-proxy-selection.test.ts +256 -0
- package/src/__tests__/extension-id-sync-guard.test.ts +155 -0
- package/src/__tests__/fixtures/mock-chrome-extension.ts +375 -0
- package/src/__tests__/gateway-only-guard.test.ts +3 -0
- package/src/__tests__/gemini-provider.test.ts +2 -2
- package/src/__tests__/guardian-routing-invariants.test.ts +70 -2
- package/src/__tests__/headless-browser-interactions.test.ts +707 -371
- package/src/__tests__/headless-browser-navigate.test.ts +389 -47
- package/src/__tests__/headless-browser-read-tools.test.ts +266 -103
- package/src/__tests__/headless-browser-snapshot.test.ts +240 -77
- package/src/__tests__/host-bash-proxy.test.ts +150 -1
- package/src/__tests__/host-browser-e2e-cloud.test.ts +462 -0
- package/src/__tests__/host-browser-e2e-self-hosted-capability.test.ts +286 -0
- package/src/__tests__/host-browser-e2e-self-hosted.test.ts +374 -0
- package/src/__tests__/host-browser-event-routes.test.ts +350 -0
- package/src/__tests__/host-browser-proxy.test.ts +444 -0
- package/src/__tests__/host-browser-routes.test.ts +198 -0
- package/src/__tests__/host-browser-ws-events-e2e.test.ts +320 -0
- package/src/__tests__/host-cu-proxy.test.ts +171 -1
- package/src/__tests__/host-file-proxy.test.ts +185 -1
- package/src/__tests__/host-file-read-tool.test.ts +52 -0
- package/src/__tests__/host-proxy-interface.test.ts +165 -0
- package/src/__tests__/host-shell-tool.test.ts +1 -11
- package/src/__tests__/http-user-message-parity.test.ts +1 -0
- package/src/__tests__/integration-status.test.ts +6 -7
- package/src/__tests__/list-messages-tool-merge.test.ts +37 -12
- package/src/__tests__/mcp-client-auth.test.ts +40 -4
- package/src/__tests__/mcp-health-check.test.ts +10 -3
- package/src/__tests__/migration-cross-version-compatibility.test.ts +3 -1
- package/src/__tests__/migration-export-http.test.ts +61 -2
- package/src/__tests__/migration-export-streaming.test.ts +66 -0
- package/src/__tests__/migration-import-commit-http.test.ts +101 -1
- package/src/__tests__/native-host-marker-sync-guard.test.ts +157 -0
- package/src/__tests__/oauth-apps-routes.test.ts +17 -12
- package/src/__tests__/oauth-cli.test.ts +707 -60
- package/src/__tests__/oauth-connect-orchestrator.test.ts +116 -24
- package/src/__tests__/oauth-provider-seed-logos.test.ts +23 -0
- package/src/__tests__/oauth-provider-serializer.test.ts +146 -10
- package/src/__tests__/oauth-provider-visibility.test.ts +19 -21
- package/src/__tests__/oauth-providers-routes.test.ts +50 -14
- package/src/__tests__/oauth-store.test.ts +1386 -182
- package/src/__tests__/oauth2-gateway-transport.test.ts +211 -20
- package/src/__tests__/onboarding-template-contract.test.ts +75 -57
- package/src/__tests__/openai-provider.test.ts +2 -2
- package/src/__tests__/outlook-categories.test.ts +1 -1
- package/src/__tests__/outlook-client-automation.test.ts +1 -1
- package/src/__tests__/outlook-compose-tools.test.ts +1 -1
- package/src/__tests__/outlook-email-watcher.test.ts +1 -1
- package/src/__tests__/outlook-follow-up.test.ts +1 -1
- package/src/__tests__/outlook-messaging-provider.test.ts +2 -2
- package/src/__tests__/outlook-trash.test.ts +1 -1
- package/src/__tests__/outlook-unsubscribe.test.ts +1 -1
- package/src/__tests__/permission-checker-host-gate.test.ts +74 -14
- package/src/__tests__/permission-mode.test.ts +28 -56
- package/src/__tests__/platform-callback-registration.test.ts +19 -0
- package/src/__tests__/post-turn-tool-result-truncation.test.ts +296 -0
- package/src/__tests__/proxy-approval-callback.test.ts +18 -0
- package/src/__tests__/require-fresh-approval.test.ts +40 -1
- package/src/__tests__/sanitize-config-for-transfer.test.ts +132 -0
- package/src/__tests__/schedule-routes.test.ts +162 -0
- package/src/__tests__/secret-detection-handler.test.ts +84 -0
- package/src/__tests__/secret-ingress-http.test.ts +1 -0
- package/src/__tests__/send-endpoint-busy.test.ts +3 -0
- package/src/__tests__/set-permission-mode.test.ts +13 -250
- package/src/__tests__/skills-file-content-endpoint.test.ts +670 -0
- package/src/__tests__/skills-files-catalog-fallback.test.ts +450 -0
- package/src/__tests__/slack-channel-config.test.ts +12 -15
- package/src/__tests__/subagent-detail.test.ts +44 -2
- package/src/__tests__/subagent-disposal.test.ts +1 -0
- package/src/__tests__/subagent-fork-notifications.test.ts +291 -0
- package/src/__tests__/subagent-fork-spawn.test.ts +384 -0
- package/src/__tests__/subagent-manager-notify.test.ts +1 -0
- package/src/__tests__/subagent-notify-parent.test.ts +1 -0
- package/src/__tests__/subagent-spawn-tool-fork.test.ts +411 -0
- package/src/__tests__/subagent-tools.test.ts +1 -0
- package/src/__tests__/subagent-types.test.ts +1 -0
- package/src/__tests__/system-prompt-ask-mode.test.ts +27 -71
- package/src/__tests__/system-prompt.test.ts +72 -1
- package/src/__tests__/task-scheduler.test.ts +32 -6
- package/src/__tests__/telegram-config.test.ts +10 -13
- package/src/__tests__/terminal-tools.test.ts +9 -0
- package/src/__tests__/tool-approval-handler.test.ts +73 -0
- package/src/__tests__/tool-side-effects-slack-dm.test.ts +22 -0
- package/src/__tests__/top-level-renderer.test.ts +73 -1
- package/src/__tests__/transport-hints-queue.test.ts +14 -29
- package/src/__tests__/trusted-contact-inline-approval-integration.test.ts +109 -0
- package/src/__tests__/v2-consent-policy.test.ts +103 -0
- package/src/acp/client-handler.ts +30 -4
- package/src/agent/loop.ts +12 -6
- package/src/approvals/guardian-request-resolvers.ts +21 -15
- package/src/browser-session/__tests__/manager.test.ts +297 -0
- package/src/browser-session/backends/cdp-inspect.ts +30 -0
- package/src/browser-session/backends/extension.ts +26 -0
- package/src/browser-session/backends/local.ts +24 -0
- package/src/browser-session/events.ts +164 -0
- package/src/browser-session/index.ts +27 -0
- package/src/browser-session/manager.ts +159 -0
- package/src/browser-session/types.ts +28 -0
- package/src/channels/__tests__/types.test.ts +134 -0
- package/src/channels/types.ts +53 -3
- package/src/cli/commands/browser-relay.ts +339 -409
- package/src/cli/commands/credentials.ts +3 -3
- package/src/cli/commands/email.ts +18 -13
- package/src/cli/commands/mcp.ts +16 -4
- package/src/cli/commands/oauth/__tests__/connect.test.ts +44 -44
- package/src/cli/commands/oauth/__tests__/disconnect.test.ts +21 -21
- package/src/cli/commands/oauth/__tests__/mode.test.ts +17 -17
- package/src/cli/commands/oauth/__tests__/ping.test.ts +16 -16
- package/src/cli/commands/oauth/__tests__/providers-delete.test.ts +31 -33
- package/src/cli/commands/oauth/__tests__/providers-register.test.ts +329 -0
- package/src/cli/commands/oauth/__tests__/providers-update.test.ts +116 -12
- package/src/cli/commands/oauth/__tests__/status.test.ts +10 -10
- package/src/cli/commands/oauth/__tests__/token.test.ts +7 -7
- package/src/cli/commands/oauth/apps.ts +7 -4
- package/src/cli/commands/oauth/connect.ts +6 -3
- package/src/cli/commands/oauth/disconnect.ts +1 -1
- package/src/cli/commands/oauth/providers.ts +200 -36
- package/src/cli/commands/oauth/shared.ts +5 -5
- package/src/cli/commands/platform/__tests__/callback-routes-list.test.ts +259 -0
- package/src/cli/commands/platform/index.ts +107 -10
- package/src/cli/commands/usage.ts +10 -9
- package/src/cli/lib/daemon-credential-client.ts +4 -0
- package/src/cli/program.ts +1 -1
- package/src/config/bundled-skills/app-builder/SKILL.md +26 -249
- package/src/config/bundled-skills/app-builder/references/CUSTOM_ROUTES.md +105 -0
- package/src/config/bundled-skills/app-builder/references/INTERACTION_HOOKS.md +56 -0
- package/src/config/bundled-skills/app-builder/references/WIDGETS.md +125 -0
- package/src/config/bundled-skills/contacts/SKILL.md +3 -0
- package/src/config/bundled-skills/document/SKILL.md +4 -0
- package/src/config/bundled-skills/gmail/SKILL.md +1 -1
- package/src/config/bundled-skills/outlook/SKILL.md +7 -0
- package/src/config/bundled-skills/subagent/SKILL.md +21 -0
- package/src/config/bundled-skills/subagent/TOOLS.json +8 -4
- package/src/config/bundled-skills/tasks/SKILL.md +5 -0
- package/src/config/env-registry.ts +14 -0
- package/src/config/env.ts +21 -0
- package/src/config/feature-flag-registry.json +44 -5
- package/src/config/loader.ts +56 -1
- package/src/config/sanitize-for-transfer.ts +47 -0
- package/src/config/schema.ts +46 -5
- package/src/config/schemas/host-browser.ts +66 -0
- package/src/config/schemas/memory-lifecycle.ts +1 -1
- package/src/config/schemas/memory-retrieval.ts +103 -0
- package/src/config/schemas/security.ts +0 -6
- package/src/config/schemas/services.ts +8 -0
- package/src/config/types.ts +0 -1
- package/src/context/post-turn-tool-result-truncation.ts +176 -0
- package/src/context/window-manager.ts +19 -1
- package/src/credential-execution/approval-bridge.ts +49 -15
- package/src/daemon/__tests__/conversation-tool-setup.test.ts +186 -0
- package/src/daemon/app-source-watcher.ts +35 -0
- package/src/daemon/context-overflow-approval.ts +5 -0
- package/src/daemon/conversation-agent-loop-handlers.ts +17 -2
- package/src/daemon/conversation-agent-loop.ts +58 -24
- package/src/daemon/conversation-attachments.ts +40 -0
- package/src/daemon/conversation-process.ts +48 -1
- package/src/daemon/conversation-runtime-assembly.ts +118 -36
- package/src/daemon/conversation-surfaces.ts +37 -36
- package/src/daemon/conversation-tool-setup.ts +74 -8
- package/src/daemon/conversation-workspace.ts +12 -0
- package/src/daemon/conversation.ts +226 -8
- package/src/daemon/date-context.ts +10 -10
- package/src/daemon/first-greeting.ts +3 -2
- package/src/daemon/handlers/conversations.ts +9 -140
- package/src/daemon/handlers/shared.ts +58 -0
- package/src/daemon/handlers/skills.ts +232 -37
- package/src/daemon/host-bash-proxy.ts +48 -13
- package/src/daemon/host-browser-proxy.ts +191 -0
- package/src/daemon/host-cu-proxy.ts +36 -11
- package/src/daemon/host-file-proxy.ts +57 -9
- package/src/daemon/lifecycle.ts +65 -11
- package/src/daemon/message-protocol.ts +7 -0
- package/src/daemon/message-types/conversations.ts +55 -13
- package/src/daemon/message-types/host-browser.ts +100 -0
- package/src/daemon/message-types/messages.ts +5 -5
- package/src/daemon/message-types/skills.ts +10 -0
- package/src/daemon/message-types/subagents.ts +2 -0
- package/src/daemon/server.ts +92 -12
- package/src/daemon/tool-side-effects.ts +6 -0
- package/src/daemon/transport-hints.ts +5 -24
- package/src/inbound/platform-callback-registration.ts +18 -17
- package/src/mcp/client.ts +59 -24
- package/src/memory/app-store.ts +31 -1
- package/src/memory/conversation-crud.ts +23 -0
- package/src/memory/conversation-starters-cadence.ts +76 -0
- package/src/memory/conversation-title-service.ts +5 -2
- package/src/memory/db-init.ts +12 -0
- package/src/memory/embedding-backend.test.ts +75 -0
- package/src/memory/embedding-backend.ts +131 -5
- package/src/memory/embedding-gemini.test.ts +54 -0
- package/src/memory/embedding-gemini.ts +20 -9
- package/src/memory/embedding-local.ts +176 -17
- package/src/memory/graph/consolidation.ts +10 -23
- package/src/memory/graph/extraction-job.ts +15 -0
- package/src/memory/graph/retriever.ts +40 -22
- package/src/memory/graph/store.test.ts +7 -3
- package/src/memory/graph/store.ts +47 -12
- package/src/memory/llm-usage-store.ts +45 -4
- package/src/memory/migrations/213-oauth-providers-scope-separator.ts +13 -0
- package/src/memory/migrations/214-oauth-providers-refresh-url.ts +11 -0
- package/src/memory/migrations/215-oauth-providers-revoke.ts +14 -0
- package/src/memory/migrations/216-oauth-providers-token-auth-method.ts +30 -0
- package/src/memory/migrations/217-conversation-host-access.ts +40 -0
- package/src/memory/migrations/218-oauth-providers-logo-url.ts +11 -0
- package/src/memory/migrations/index.ts +6 -0
- package/src/memory/migrations/registry.ts +8 -0
- package/src/memory/schema/conversations.ts +1 -0
- package/src/memory/schema/oauth.ts +18 -13
- package/src/oauth/AGENTS.md +76 -0
- package/src/oauth/__tests__/identity-verifier.test.ts +24 -19
- package/src/oauth/__tests__/seed-providers-managed.test.ts +32 -0
- package/src/oauth/byo-connection.test.ts +8 -8
- package/src/oauth/byo-connection.ts +7 -7
- package/src/oauth/connect-orchestrator.ts +23 -21
- package/src/oauth/connect-types.ts +3 -3
- package/src/oauth/connection-resolver.test.ts +17 -4
- package/src/oauth/connection-resolver.ts +16 -16
- package/src/oauth/connection.ts +1 -1
- package/src/oauth/manual-token-connection.ts +13 -13
- package/src/oauth/oauth-store.ts +214 -100
- package/src/oauth/platform-connection.test.ts +3 -3
- package/src/oauth/platform-connection.ts +4 -4
- package/src/oauth/provider-serializer.ts +31 -5
- package/src/oauth/revoke.ts +76 -0
- package/src/oauth/seed-providers.ts +126 -87
- package/src/oauth/token-persistence.ts +1 -1
- package/src/permissions/permission-mode.ts +4 -11
- package/src/permissions/prompter.ts +13 -1
- package/src/permissions/v2-consent-policy.ts +87 -0
- package/src/prompts/system-prompt.ts +18 -21
- package/src/prompts/templates/BOOTSTRAP-REFERENCE.md +3 -65
- package/src/prompts/templates/BOOTSTRAP.md +59 -105
- package/src/providers/anthropic/client.ts +1 -0
- package/src/providers/types.ts +1 -1
- package/src/runtime/AGENTS.md +23 -0
- package/src/runtime/__tests__/browser-extension-pair-routes.test.ts +715 -0
- package/src/runtime/__tests__/capability-tokens.test.ts +258 -0
- package/src/runtime/__tests__/chrome-extension-registry.test.ts +518 -0
- package/src/runtime/assistant-event-hub.ts +2 -2
- package/src/runtime/auth/__tests__/guard-tests.test.ts +1 -0
- package/src/runtime/auth/__tests__/middleware.test.ts +116 -1
- package/src/runtime/auth/__tests__/route-policy.test.ts +8 -0
- package/src/runtime/auth/middleware.ts +98 -0
- package/src/runtime/auth/route-policy.ts +6 -7
- package/src/runtime/capability-tokens.ts +414 -0
- package/src/runtime/channel-approvals.ts +18 -5
- package/src/runtime/chrome-extension-registry.ts +332 -0
- package/src/runtime/confirmation-request-guardian-bridge.ts +6 -0
- package/src/runtime/guardian-decision-types.ts +7 -0
- package/src/runtime/http-server.ts +425 -70
- package/src/runtime/migrations/__tests__/rebind-secrets-credentials.test.ts +172 -0
- package/src/runtime/migrations/__tests__/vbundle-builder-credentials.test.ts +276 -0
- package/src/runtime/migrations/__tests__/vbundle-import-credentials.test.ts +162 -0
- package/src/runtime/migrations/migration-transport.ts +6 -0
- package/src/runtime/migrations/migration-wizard.ts +22 -2
- package/src/runtime/migrations/rebind-secrets-screen.ts +76 -15
- package/src/runtime/migrations/vbundle-builder.ts +145 -38
- package/src/runtime/migrations/vbundle-import-analyzer.ts +19 -0
- package/src/runtime/migrations/vbundle-importer.ts +55 -5
- package/src/runtime/pending-interactions.ts +29 -13
- package/src/runtime/routes/approval-routes.ts +90 -16
- package/src/runtime/routes/browser-cdp-routes.ts +229 -0
- package/src/runtime/routes/browser-extension-pair-routes.ts +497 -0
- package/src/runtime/routes/conversation-analysis-routes.ts +2 -1
- package/src/runtime/routes/conversation-management-routes.ts +108 -0
- package/src/runtime/routes/conversation-routes.ts +301 -27
- package/src/runtime/routes/conversation-starter-routes.ts +78 -16
- package/src/runtime/routes/guardian-action-routes.ts +24 -13
- package/src/runtime/routes/host-browser-routes.ts +279 -0
- package/src/runtime/routes/host-file-routes.ts +9 -1
- package/src/runtime/routes/identity-routes.ts +259 -16
- package/src/runtime/routes/log-export-routes.ts +42 -22
- package/src/runtime/routes/memory-item-routes.ts +1 -7
- package/src/runtime/routes/migration-routes.ts +87 -2
- package/src/runtime/routes/oauth-apps.ts +15 -17
- package/src/runtime/routes/oauth-providers.ts +4 -0
- package/src/runtime/routes/schedule-routes.ts +24 -11
- package/src/runtime/routes/settings-routes.ts +9 -97
- package/src/runtime/routes/skills-routes.ts +52 -2
- package/src/runtime/routes/subagents-routes.ts +14 -10
- package/src/runtime/routes/usage-routes.ts +8 -7
- package/src/runtime/routes/workspace-routes.test.ts +22 -0
- package/src/runtime/routes/workspace-routes.ts +8 -1
- package/src/runtime/routes/workspace-utils.ts +2 -0
- package/src/schedule/scheduler.ts +7 -5
- package/src/security/ces-credential-client.ts +20 -0
- package/src/security/ces-rpc-credential-backend.ts +17 -0
- package/src/security/credential-backend.ts +5 -0
- package/src/security/oauth2.ts +42 -25
- package/src/security/secure-keys.ts +118 -25
- package/src/security/token-manager.ts +23 -10
- package/src/skills/catalog-files.ts +492 -0
- package/src/subagent/manager.ts +131 -26
- package/src/subagent/types.ts +19 -0
- package/src/tools/apps/executors.ts +11 -2
- package/src/tools/browser/__tests__/auth-detector.test.ts +202 -108
- package/src/tools/browser/auth-detector.ts +43 -12
- package/src/tools/browser/browser-execution.ts +645 -340
- package/src/tools/browser/browser-manager.ts +36 -12
- package/src/tools/browser/cdp-client/__tests__/accessibility-snapshot.test.ts +318 -0
- package/src/tools/browser/cdp-client/__tests__/cdp-dom-helpers.test.ts +1175 -0
- package/src/tools/browser/cdp-client/__tests__/cdp-inspect-client.test.ts +870 -0
- package/src/tools/browser/cdp-client/__tests__/extension-cdp-client.test.ts +330 -0
- package/src/tools/browser/cdp-client/__tests__/factory.test.ts +377 -0
- package/src/tools/browser/cdp-client/__tests__/fixtures/ax-tree-nested-frames.json +64 -0
- package/src/tools/browser/cdp-client/__tests__/fixtures/ax-tree-simple.json +69 -0
- package/src/tools/browser/cdp-client/__tests__/local-cdp-client.test.ts +310 -0
- package/src/tools/browser/cdp-client/__tests__/types.test.ts +96 -0
- package/src/tools/browser/cdp-client/accessibility-snapshot.ts +387 -0
- package/src/tools/browser/cdp-client/cdp-dom-helpers.ts +695 -0
- package/src/tools/browser/cdp-client/cdp-inspect/__tests__/discovery.test.ts +743 -0
- package/src/tools/browser/cdp-client/cdp-inspect/__tests__/ws-transport.test.ts +580 -0
- package/src/tools/browser/cdp-client/cdp-inspect/discovery.ts +578 -0
- package/src/tools/browser/cdp-client/cdp-inspect/ws-transport.ts +579 -0
- package/src/tools/browser/cdp-client/cdp-inspect-client.ts +635 -0
- package/src/tools/browser/cdp-client/errors.ts +34 -0
- package/src/tools/browser/cdp-client/extension-cdp-client.ts +125 -0
- package/src/tools/browser/cdp-client/factory.ts +204 -0
- package/src/tools/browser/cdp-client/index.ts +14 -0
- package/src/tools/browser/cdp-client/local-cdp-client.ts +187 -0
- package/src/tools/browser/cdp-client/types.ts +52 -0
- package/src/tools/filesystem/edit.ts +1 -1
- package/src/tools/filesystem/list.ts +1 -1
- package/src/tools/filesystem/read.ts +1 -1
- package/src/tools/filesystem/write.ts +2 -1
- package/src/tools/host-filesystem/edit.ts +1 -1
- package/src/tools/host-filesystem/read.ts +12 -15
- package/src/tools/host-filesystem/write.ts +1 -1
- package/src/tools/host-terminal/host-shell.ts +21 -16
- package/src/tools/permission-checker.ts +77 -82
- package/src/tools/registry.ts +0 -2
- package/src/tools/secret-detection-handler.ts +34 -0
- package/src/tools/shared/filesystem/image-read.ts +61 -40
- package/src/tools/subagent/spawn.ts +47 -3
- package/src/tools/subagent/status.ts +2 -0
- package/src/tools/system/register.ts +2 -16
- package/src/tools/terminal/safe-env.ts +7 -0
- package/src/tools/terminal/shell.ts +21 -16
- package/src/tools/tool-approval-handler.ts +48 -2
- package/src/tools/types.ts +2 -0
- package/src/util/platform.ts +14 -19
- package/src/workspace/top-level-renderer.ts +19 -1
- package/src/__tests__/chrome-cdp.test.ts +0 -419
- package/src/__tests__/permission-mode-sse.test.ts +0 -418
- package/src/__tests__/permission-mode-store.test.ts +0 -277
- package/src/browser-extension-relay/protocol.ts +0 -63
- package/src/browser-extension-relay/server.ts +0 -203
- package/src/config/schemas/sandbox.ts +0 -14
- package/src/permissions/permission-mode-store.ts +0 -180
- package/src/tools/browser/chrome-cdp.ts +0 -239
- package/src/tools/system/set-permission-mode.ts +0 -103
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
import { isAssistantFeatureFlagEnabled } from "../config/assistant-feature-flags.js";
|
|
2
|
+
import { getConfig } from "../config/loader.js";
|
|
3
|
+
import { getConversationHostAccess as loadConversationHostAccess } from "../memory/conversation-crud.js";
|
|
4
|
+
import { isSideEffectTool } from "../tools/side-effects.js";
|
|
5
|
+
import type { ToolContext } from "../tools/types.js";
|
|
6
|
+
import type { AllowlistOption, ScopeOption, UserDecision } from "./types.js";
|
|
7
|
+
import { isHostTool } from "./workspace-policy.js";
|
|
8
|
+
|
|
9
|
+
export type V2ConsentDisposition =
|
|
10
|
+
| "legacy"
|
|
11
|
+
| "auto_allow"
|
|
12
|
+
| "prompt_host_access";
|
|
13
|
+
|
|
14
|
+
type PromptLike = {
|
|
15
|
+
toolName: string;
|
|
16
|
+
allowlistOptions?: readonly AllowlistOption[];
|
|
17
|
+
scopeOptions?: readonly ScopeOption[];
|
|
18
|
+
persistentDecisionsAllowed?: boolean;
|
|
19
|
+
temporaryOptionsAvailable?: readonly ("allow_10m" | "allow_conversation")[];
|
|
20
|
+
};
|
|
21
|
+
|
|
22
|
+
export const CONVERSATION_HOST_ACCESS_PROMPT = Object.freeze({
|
|
23
|
+
allowlistOptions: [] as AllowlistOption[],
|
|
24
|
+
scopeOptions: [] as ScopeOption[],
|
|
25
|
+
persistentDecisionsAllowed: false as const,
|
|
26
|
+
temporaryOptionsAvailable: undefined as
|
|
27
|
+
| Array<"allow_10m" | "allow_conversation">
|
|
28
|
+
| undefined,
|
|
29
|
+
});
|
|
30
|
+
|
|
31
|
+
export function isPermissionControlsV2Enabled(): boolean {
|
|
32
|
+
return isAssistantFeatureFlagEnabled("permission-controls-v2", getConfig());
|
|
33
|
+
}
|
|
34
|
+
|
|
35
|
+
export function isConversationHostAccessEnabled(
|
|
36
|
+
conversationId: string,
|
|
37
|
+
): boolean {
|
|
38
|
+
return loadConversationHostAccess(conversationId);
|
|
39
|
+
}
|
|
40
|
+
|
|
41
|
+
export function evaluateV2ConsentDisposition(
|
|
42
|
+
toolName: string,
|
|
43
|
+
input: Record<string, unknown>,
|
|
44
|
+
context: ToolContext,
|
|
45
|
+
): V2ConsentDisposition {
|
|
46
|
+
if (!isPermissionControlsV2Enabled()) {
|
|
47
|
+
return "legacy";
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
if (context.requireFreshApproval) {
|
|
51
|
+
return "legacy";
|
|
52
|
+
}
|
|
53
|
+
|
|
54
|
+
if (context.forcePromptSideEffects && isSideEffectTool(toolName, input)) {
|
|
55
|
+
return "legacy";
|
|
56
|
+
}
|
|
57
|
+
|
|
58
|
+
if (!isHostTool(toolName)) {
|
|
59
|
+
return "auto_allow";
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
return loadConversationHostAccess(context.conversationId)
|
|
63
|
+
? "auto_allow"
|
|
64
|
+
: "prompt_host_access";
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
export function isConversationHostAccessEnablePrompt(
|
|
68
|
+
details: PromptLike | undefined,
|
|
69
|
+
): boolean {
|
|
70
|
+
if (!details) {
|
|
71
|
+
return false;
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
return (
|
|
75
|
+
isHostTool(details.toolName) &&
|
|
76
|
+
(details.allowlistOptions?.length ?? 0) === 0 &&
|
|
77
|
+
(details.scopeOptions?.length ?? 0) === 0 &&
|
|
78
|
+
details.persistentDecisionsAllowed === false &&
|
|
79
|
+
(details.temporaryOptionsAvailable?.length ?? 0) === 0
|
|
80
|
+
);
|
|
81
|
+
}
|
|
82
|
+
|
|
83
|
+
export function isConversationHostAccessDecision(
|
|
84
|
+
decision: UserDecision,
|
|
85
|
+
): decision is "allow" | "deny" {
|
|
86
|
+
return decision === "allow" || decision === "deny";
|
|
87
|
+
}
|
|
@@ -8,11 +8,9 @@ import {
|
|
|
8
8
|
} from "node:fs";
|
|
9
9
|
import { join } from "node:path";
|
|
10
10
|
|
|
11
|
-
import { isAssistantFeatureFlagEnabled } from "../config/assistant-feature-flags.js";
|
|
12
11
|
import { getIsContainerized } from "../config/env-registry.js";
|
|
13
12
|
import { loadConfig } from "../config/loader.js";
|
|
14
13
|
import { listConnections } from "../oauth/oauth-store.js";
|
|
15
|
-
import { getMode } from "../permissions/permission-mode-store.js";
|
|
16
14
|
import { resolveBundledDir } from "../util/bundled-asset.js";
|
|
17
15
|
import { getLogger } from "../util/logger.js";
|
|
18
16
|
import {
|
|
@@ -219,6 +217,8 @@ export function buildSystemPrompt(options?: BuildSystemPromptOptions): string {
|
|
|
219
217
|
// the first cache block so they remain cached even when workspace files
|
|
220
218
|
// (IDENTITY.md, SOUL.md, USER.md, etc.) are edited between turns.
|
|
221
219
|
const staticParts: string[] = [];
|
|
220
|
+
const customPrefix = readCustomSystemPromptPrefix();
|
|
221
|
+
if (customPrefix) staticParts.push(customPrefix);
|
|
222
222
|
staticParts.push(buildParallelToolCallsSection());
|
|
223
223
|
if (getIsContainerized()) staticParts.push(buildContainerizedSection());
|
|
224
224
|
staticParts.push(buildCliReferenceSection());
|
|
@@ -313,9 +313,6 @@ export function buildSystemPrompt(options?: BuildSystemPromptOptions): string {
|
|
|
313
313
|
// Journal entries are extracted into graph nodes by the memory pipeline.
|
|
314
314
|
// Journal files remain writable on disk.
|
|
315
315
|
|
|
316
|
-
const askBeforeActingSection = buildAskBeforeActingSection();
|
|
317
|
-
if (askBeforeActingSection) dynamicParts.push(askBeforeActingSection);
|
|
318
|
-
|
|
319
316
|
const dynamic = dynamicParts.join("\n\n");
|
|
320
317
|
|
|
321
318
|
return staticParts.join("\n\n") + SYSTEM_PROMPT_CACHE_BOUNDARY + dynamic;
|
|
@@ -370,7 +367,7 @@ function buildCredentialSecuritySection(): string {
|
|
|
370
367
|
}
|
|
371
368
|
|
|
372
369
|
function buildIntegrationSection(): string {
|
|
373
|
-
let connections: {
|
|
370
|
+
let connections: { provider: string; accountInfo?: string | null }[];
|
|
374
371
|
try {
|
|
375
372
|
connections = listConnections().filter((c) => c.status === "active");
|
|
376
373
|
} catch {
|
|
@@ -385,31 +382,27 @@ function buildIntegrationSection(): string {
|
|
|
385
382
|
const state = conn.accountInfo
|
|
386
383
|
? `Connected (${conn.accountInfo})`
|
|
387
384
|
: "Connected";
|
|
388
|
-
lines.push(`- **${conn.
|
|
385
|
+
lines.push(`- **${conn.provider}**: ${state}`);
|
|
389
386
|
}
|
|
390
387
|
|
|
391
388
|
return lines.join("\n");
|
|
392
389
|
}
|
|
393
390
|
|
|
394
|
-
|
|
391
|
+
/**
|
|
392
|
+
* Read the user-configured custom system prompt prefix. Returns the trimmed
|
|
393
|
+
* value when set and non-empty, otherwise null. Errors (e.g. config file
|
|
394
|
+
* unavailable) are swallowed so prompt construction never fails.
|
|
395
|
+
*/
|
|
396
|
+
function readCustomSystemPromptPrefix(): string | null {
|
|
395
397
|
try {
|
|
396
|
-
const
|
|
397
|
-
if (
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
const mode = getMode();
|
|
401
|
-
if (!mode.askBeforeActing) return null;
|
|
402
|
-
|
|
403
|
-
return [
|
|
404
|
-
"## Action Confirmation Mode",
|
|
405
|
-
"",
|
|
406
|
-
'You are in "Ask before acting" mode. Use your judgment about when to check in with the user before proceeding. You should ask for confirmation before actions that are costly, time-consuming, or hard to reverse — for example: sending emails or messages, deleting files or data, making purchases, posting publicly, modifying permissions, or taking actions with significant real-world consequences. You do NOT need to ask before routine low-stakes actions like reading files, searching, running safe shell commands, or making code edits — just do those.',
|
|
407
|
-
].join("\n");
|
|
398
|
+
const prefix = loadConfig().systemPromptPrefix;
|
|
399
|
+
if (typeof prefix !== "string") return null;
|
|
400
|
+
const trimmed = prefix.trim();
|
|
401
|
+
return trimmed.length > 0 ? trimmed : null;
|
|
408
402
|
} catch {
|
|
409
403
|
return null;
|
|
410
404
|
}
|
|
411
405
|
}
|
|
412
|
-
|
|
413
406
|
function buildContainerizedSection(): string {
|
|
414
407
|
const workspaceDir = getWorkspaceDir();
|
|
415
408
|
return [
|
|
@@ -431,6 +424,8 @@ function buildParallelToolCallsSection(): string {
|
|
|
431
424
|
return [
|
|
432
425
|
"<use_parallel_tool_calls>",
|
|
433
426
|
"For maximum efficiency, whenever you perform multiple independent operations, invoke all relevant tools simultaneously rather than sequentially. Prioritize calling tools in parallel whenever possible. For example, when reading 3 files, run 3 tool calls in parallel to read all 3 files into context at the same time. When running multiple read-only commands like `ls` or `list_dir`, always run all of the commands in parallel. Err on the side of maximizing parallel tool calls rather than running too many tools sequentially.",
|
|
427
|
+
"",
|
|
428
|
+
"For non-trivial independent workstreams — research, coding tasks, multi-step investigations — aggressively delegate to subagents (load the `subagent` skill for tools and instructions). Spawn subagents early and often; the cost of an unnecessary subagent is far lower than the cost of serializing work you could have parallelized.",
|
|
434
429
|
"</use_parallel_tool_calls>",
|
|
435
430
|
].join("\n");
|
|
436
431
|
}
|
|
@@ -444,6 +439,8 @@ export function buildCliReferenceSection(): string {
|
|
|
444
439
|
"Use `assistant platform status` to check the current Vellum platform connection state, and `assistant platform --help` to see all platform management subcommands.",
|
|
445
440
|
"",
|
|
446
441
|
"Run `assistant --help` to see all available commands, or `assistant <command> --help` for detailed help on any subcommand.",
|
|
442
|
+
"",
|
|
443
|
+
"**Before telling a user you cannot do something, run `assistant --help` to check whether a built-in command exists for it.** The CLI includes capabilities (email, integrations, platform management, etc.) that you may not know about from training data alone. When asked about your capabilities or what you can do, check your CLI first — don't guess or assume.",
|
|
447
444
|
].join("\n");
|
|
448
445
|
}
|
|
449
446
|
|
|
@@ -1,71 +1,9 @@
|
|
|
1
|
-
_
|
|
1
|
+
_ Optional reference payloads. The model may use these if it chooses to show a task card, but is not required to.
|
|
2
2
|
_ This file is deleted alongside BOOTSTRAP.md when onboarding completes.
|
|
3
3
|
|
|
4
|
-
## Personality Form
|
|
5
|
-
|
|
6
|
-
Use this exact `ui_show` payload for Step 2 (Personality Quiz):
|
|
7
|
-
|
|
8
|
-
ui_show({
|
|
9
|
-
surface_type: "form",
|
|
10
|
-
data: {
|
|
11
|
-
description: "Let's figure out how we work together. Pick what feels right.",
|
|
12
|
-
fields: [
|
|
13
|
-
{
|
|
14
|
-
id: "communication_style",
|
|
15
|
-
type: "select",
|
|
16
|
-
label: "When we're going back and forth, it's more like...",
|
|
17
|
-
required: true,
|
|
18
|
-
options: [
|
|
19
|
-
{ label: "Casual friends texting", value: "casual_friends" },
|
|
20
|
-
{ label: "Sharp coworkers who respect each other", value: "sharp_coworkers" },
|
|
21
|
-
{ label: "Chill and low-key, no drama", value: "chill" },
|
|
22
|
-
{ label: "High energy sparring partners", value: "sparring" },
|
|
23
|
-
{ label: "Professional but warm", value: "professional_warm" }
|
|
24
|
-
]
|
|
25
|
-
},
|
|
26
|
-
{
|
|
27
|
-
id: "task_style",
|
|
28
|
-
type: "select",
|
|
29
|
-
label: "When I'm doing something for you, you want me to...",
|
|
30
|
-
required: true,
|
|
31
|
-
options: [
|
|
32
|
-
{ label: "Just do it, don't explain unless I ask", value: "just_do_it" },
|
|
33
|
-
{ label: "Walk me through your thinking", value: "explain" },
|
|
34
|
-
{ label: "Ask me before making big decisions", value: "check_first" },
|
|
35
|
-
{ label: "Be opinionated, push back if you disagree", value: "opinionated" }
|
|
36
|
-
]
|
|
37
|
-
},
|
|
38
|
-
{
|
|
39
|
-
id: "humor",
|
|
40
|
-
type: "select",
|
|
41
|
-
label: "When it comes to humor...",
|
|
42
|
-
required: true,
|
|
43
|
-
options: [
|
|
44
|
-
{ label: "Dry and deadpan", value: "dry" },
|
|
45
|
-
{ label: "Playful and light", value: "playful" },
|
|
46
|
-
{ label: "Keep it professional", value: "professional" },
|
|
47
|
-
{ label: "Match my energy", value: "match" }
|
|
48
|
-
]
|
|
49
|
-
},
|
|
50
|
-
{
|
|
51
|
-
id: "depth",
|
|
52
|
-
type: "select",
|
|
53
|
-
label: "When explaining things...",
|
|
54
|
-
required: true,
|
|
55
|
-
options: [
|
|
56
|
-
{ label: "Keep it simple", value: "simple" },
|
|
57
|
-
{ label: "I like details", value: "detailed" },
|
|
58
|
-
{ label: "Depends on the topic", value: "adaptive" }
|
|
59
|
-
]
|
|
60
|
-
}
|
|
61
|
-
],
|
|
62
|
-
submitLabel: "Lock it in"
|
|
63
|
-
}
|
|
64
|
-
})
|
|
65
|
-
|
|
66
4
|
## Task Card (Email Not Connected)
|
|
67
5
|
|
|
68
|
-
Use this `ui_show` payload
|
|
6
|
+
Use this `ui_show` payload when Gmail/Outlook is NOT in the Connected Services section:
|
|
69
7
|
|
|
70
8
|
ui_show({
|
|
71
9
|
surface_type: "card",
|
|
@@ -83,7 +21,7 @@ ui_show({
|
|
|
83
21
|
|
|
84
22
|
## Task Card (Email Already Connected)
|
|
85
23
|
|
|
86
|
-
Use this `ui_show` payload
|
|
24
|
+
Use this `ui_show` payload when Google or Outlook IS in the Connected Services section:
|
|
87
25
|
|
|
88
26
|
ui_show({
|
|
89
27
|
surface_type: "card",
|
|
@@ -1,152 +1,106 @@
|
|
|
1
1
|
_ Lines starting with _ are comments. They won't appear in the system prompt
|
|
2
2
|
|
|
3
|
-
# BOOTSTRAP.md
|
|
3
|
+
# BOOTSTRAP.md — First Conversation
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
This is your first conversation. This document gives you goals and constraints — you own the sequencing, pacing, and tone.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
## Goals
|
|
8
8
|
|
|
9
|
-
|
|
10
|
-
1. **"Oh, this has personality."** Names, style, vibe. You're a person, not a product.
|
|
11
|
-
2. **"Oh, this is useful."** Complete a real task. Prove you're worth keeping around.
|
|
12
|
-
3. **"Oh, this remembers me."** Save what you learn silently. They should feel known.
|
|
9
|
+
1. **Establish mutual identity gently** — if pre-chat onboarding already collected names, skip. If the user skipped pre-chat, do NOT force it. At most pick a default ("I'll go by Pax and call you 'you' for now") and move on. Never re-ask names in the first conversation; they can come up organically later.
|
|
13
10
|
|
|
14
|
-
|
|
11
|
+
2. **Prove value fast** — do something useful before asking for anything. Wow moment within 2-3 exchanges.
|
|
15
12
|
|
|
16
|
-
**
|
|
13
|
+
3. **Infer, don't interrogate** — learn communication style, interests, and context from natural conversation. No personality quiz. No dropdown forms. No structured intake.
|
|
17
14
|
|
|
18
|
-
**
|
|
15
|
+
4. **Surface what you learned** — after doing something useful, briefly show the user what you picked up. Make it correctable. ("from that I picked up X, Y, Z — sound right?")
|
|
19
16
|
|
|
20
|
-
|
|
17
|
+
5. **Offer the next level** — once you know something, offer a capability it enables. Not as a reward — as a natural relationship step.
|
|
21
18
|
|
|
22
|
-
|
|
19
|
+
6. **Write everything immediately** — every fact learned gets saved to USER.md the same turn. Style observations go to SOUL.md. No batching.
|
|
23
20
|
|
|
24
|
-
**
|
|
21
|
+
7. **Clean up** — delete BOOTSTRAP.md and BOOTSTRAP-REFERENCE.md at the end of this conversation, regardless of how far you got. One-shot.
|
|
25
22
|
|
|
26
|
-
|
|
23
|
+
## Constraints
|
|
27
24
|
|
|
28
|
-
|
|
25
|
+
- **Budget:** $2 soft cap, $5 hard cap. Keep tasks light. Don't burn credits on onboarding overhead.
|
|
26
|
+
- Never ask more than 2 questions without doing something.
|
|
27
|
+
- Don't block on setup. If the user wants to do something, do it. Weave discovery into the work.
|
|
28
|
+
- One-shot. Bootstrap is deleted after the first conversation regardless of how far you got.
|
|
29
29
|
|
|
30
|
-
|
|
31
|
-
1. I'm new and still forming (honesty)
|
|
32
|
-
2. I get better over time, like training a new colleague (sets expectations)
|
|
33
|
-
3. Let's start with who we are (personality-first)
|
|
30
|
+
## What You Own (do NOT prescribe)
|
|
34
31
|
|
|
35
|
-
|
|
32
|
+
- Sequencing and pacing.
|
|
33
|
+
- Whether to lead with personality or utility.
|
|
34
|
+
- When to ask questions vs. start doing.
|
|
35
|
+
- How much warmth to show — calibrate to the user's tone.
|
|
36
|
+
- When/whether to surface the "what I learned" receipt.
|
|
36
37
|
|
|
37
|
-
|
|
38
|
+
## Technical Contract (what must be prescribed)
|
|
38
39
|
|
|
39
|
-
|
|
40
|
+
**Files to create/update:** IDENTITY.md, SOUL.md, USER.md
|
|
40
41
|
|
|
41
|
-
|
|
42
|
+
**File format:** preserve existing field structure:
|
|
43
|
+
- IDENTITY.md: Name, Emoji, Nature, Personality, Role
|
|
44
|
+
- USER.md: Preferred name, Pronouns, Locale, Work role, Goals, Hobbies/fun, Daily tools
|
|
42
45
|
|
|
43
|
-
|
|
46
|
+
Use `file_edit` immediately, silently, never mention file names or tool names to the user.
|
|
44
47
|
|
|
45
|
-
|
|
48
|
+
The contents of IDENTITY.md, SOUL.md, and USER.md are already in your system prompt — use the exact text you see there for `old_string` in `file_edit`.
|
|
46
49
|
|
|
47
|
-
|
|
50
|
+
After tool calls, do not repeat yourself — your text before tool calls is already visible to the user.
|
|
48
51
|
|
|
49
|
-
|
|
52
|
+
**Cleanup rule:** delete BOOTSTRAP.md and BOOTSTRAP-REFERENCE.md when the conversation ends.
|
|
50
53
|
|
|
51
|
-
|
|
54
|
+
**Core interaction pattern:** infer -> do something useful -> surface what you learned -> offer next capability.
|
|
52
55
|
|
|
53
|
-
|
|
56
|
+
## Capability Unlock Pattern
|
|
54
57
|
|
|
55
|
-
|
|
58
|
+
After the first useful interaction, organically surface one capability offer based on what came up naturally:
|
|
56
59
|
|
|
57
|
-
|
|
60
|
+
- User mentions email -> "I can connect to your email and keep an eye on things — want to set that up?"
|
|
61
|
+
- User's writing style is clear -> "I've got a read on how you write — I can draft things in your voice now"
|
|
62
|
+
- User mentions a team -> "tell me more about your team and I can start prepping for your meetings"
|
|
63
|
+
- User mentions Slack -> "I can work in Slack with you — want me to walk you through setting that up?"
|
|
58
64
|
|
|
59
|
-
|
|
65
|
+
Not scripted — choose based on what came up naturally.
|
|
60
66
|
|
|
61
|
-
|
|
67
|
+
## Tone Guidance
|
|
62
68
|
|
|
63
|
-
|
|
64
|
-
-
|
|
65
|
-
-
|
|
66
|
-
-
|
|
67
|
-
- If they skip ("nothing," "let's move on"), respect it immediately. Move on.
|
|
68
|
-
|
|
69
|
-
### Step 4: First Task
|
|
70
|
-
|
|
71
|
-
Transition naturally: "Alright, [name]. Let's put this to work. What do you want to tackle first?"
|
|
72
|
-
|
|
73
|
-
Show a task card. **Before showing the card, check the Connected Services section of your system prompt.** If Google or Outlook is already connected, swap the "Connect my email" option for "Check my email" (see BOOTSTRAP-REFERENCE.md for both variants).
|
|
74
|
-
|
|
75
|
-
Read BOOTSTRAP-REFERENCE.md for the exact `ui_show` card payload.
|
|
76
|
-
|
|
77
|
-
**When the user picks an option:**
|
|
78
|
-
|
|
79
|
-
- **Connect my email:** Guide them through one-click Gmail or Outlook OAuth setup. After connecting, do a quick inbox summary or calendar overview to show immediate value.
|
|
80
|
-
- **Check my email:** They're already connected. Summarize their inbox or today's calendar. Show you can be useful right now.
|
|
81
|
-
- **Research a topic and make me a deck:** Focused web search, 3-5 key points, build a polished interactive deck. Keep it tight, not exhaustive.
|
|
82
|
-
- **Build me something:** Ask what kind of tool or app. Build it using the app builder. Make it look great.
|
|
83
|
-
- **Do something with a photo:** Use media processing or image studio skills. Ask what they have and what they want.
|
|
84
|
-
|
|
85
|
-
**If the user gives you their own task instead of picking from the card**, do it. Do it well. This is your audition.
|
|
86
|
-
|
|
87
|
-
**Pacing rule:** Don't ask more than 2 questions in a row without doing something. If you've asked twice and haven't completed a task, stop asking and start doing.
|
|
88
|
-
|
|
89
|
-
### Step 5: Keep the Momentum
|
|
90
|
-
|
|
91
|
-
After the task is done, don't pivot to setup. Build on what just happened.
|
|
92
|
-
|
|
93
|
-
**First choice: chain off the task.** Suggest one natural follow-up that extends the work they just did. Examples:
|
|
94
|
-
- Built a deck → "Want to send this to someone or refine a specific slide?"
|
|
95
|
-
- Connected email → "Want me to summarize what needs your attention today?"
|
|
96
|
-
- Researched a topic → "Want me to go deeper on one of those points, or turn this into something shareable?"
|
|
97
|
-
- Built an app → Proactively suggest a specific improvement to what they built (a missing feature, a UI polish, better error handling). Show you have taste.
|
|
98
|
-
|
|
99
|
-
The follow-up should feel like a coworker saying "while we're at it..." — not a product tour.
|
|
100
|
-
|
|
101
|
-
**Fallback: plant a hook for next time.** If the task was a dead-end (photo edit, one-off question), reach back to Step 3. Pick up something from their "what's on your mind" answer and offer to work on it: "You mentioned [X] earlier — I can dig into that and have something ready next time you open this."
|
|
102
|
-
|
|
103
|
-
If they engage, do it. If they decline or wrap up, move on. One offer, no pressure.
|
|
104
|
-
|
|
105
|
-
### Step 6: Before You Go
|
|
106
|
-
|
|
107
|
-
Before deleting BOOTSTRAP.md:
|
|
108
|
-
|
|
109
|
-
1. **Write your first journal entry.** This is how future-you remembers this person. Write about: what they asked you to do and how it went, what you noticed about how they communicate, what name they chose and what personality emerged, anything important about this first interaction, a note to next-you about what to follow up on. Keep it natural, a few paragraphs.
|
|
110
|
-
|
|
111
|
-
2. **Update NOW.md** with current state: what you know, what's active, what to pick up next time.
|
|
112
|
-
|
|
113
|
-
3. **Delete BOOTSTRAP.md and BOOTSTRAP-REFERENCE.md.**
|
|
69
|
+
- Not servile. Not a product demo. A new colleague who's sharp, pays attention, and earns trust through competence.
|
|
70
|
+
- Match the user's energy from their first message. If they type in lowercase, don't respond with formal paragraphs.
|
|
71
|
+
- If the user opens with a task ("build me an app"), skip introductions and do the task. Learn their name when it comes up naturally.
|
|
72
|
+
- The emotional beat ("what's on your mind?") should happen organically or not at all.
|
|
114
73
|
|
|
115
74
|
## Saving What You Learn
|
|
116
75
|
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
A field is "resolved" when any of these is true:
|
|
120
|
-
|
|
121
|
-
- The user gave an explicit answer
|
|
122
|
-
- You confidently inferred it from conversation
|
|
123
|
-
- The user declined, dodged, or sidestepped it
|
|
76
|
+
Call `file_edit` immediately whenever you learn something, in the same turn. Don't batch saves.
|
|
124
77
|
|
|
125
|
-
Mark declined fields so you don't re-ask
|
|
78
|
+
Mark declined fields so you don't re-ask (e.g., `Work role: declined_by_user`). Note inferred values with source (e.g., `Pronouns: inferred: he/him`).
|
|
126
79
|
|
|
127
|
-
|
|
80
|
+
Throughout the conversation, pay attention to HOW the user communicates. Save specific observations to SOUL.md: "uses lowercase, drops punctuation, leads with questions, prefers bullet points over paragraphs." The specificity makes personality feel earned, not assigned.
|
|
128
81
|
|
|
129
|
-
|
|
82
|
+
When saving to IDENTITY.md, add an `## Identity Intro` section with a very short tagline.
|
|
130
83
|
|
|
131
|
-
|
|
84
|
+
When saving to SOUL.md, be specific about tone, energy, and conversational style.
|
|
132
85
|
|
|
133
|
-
|
|
86
|
+
## Pre-chat Onboarding Context
|
|
134
87
|
|
|
135
|
-
|
|
88
|
+
If an `onboarding` JSON context is present in this conversation, the user already went through a native pre-chat flow. Use it:
|
|
136
89
|
|
|
137
|
-
|
|
90
|
+
- `tools` array -> know which integration offers to surface first, infer work profile
|
|
91
|
+
- `tasks` array -> know what "prove value fast" means for this person
|
|
92
|
+
- `tone` string -> calibrate warmth/formality
|
|
93
|
+
- `userName` / `assistantName` -> write to IDENTITY.md and USER.md immediately, skip name exchange
|
|
138
94
|
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
Throughout the conversation, pay attention to HOW the user communicates. Save specific observations to SOUL.md: "uses lowercase, drops punctuation, leads with questions, prefers bullet points over paragraphs." The specificity makes personality feel earned, not assigned. Adapt your style to match before they even notice.
|
|
95
|
+
If no onboarding context is present, infer everything fresh from conversation.
|
|
142
96
|
|
|
143
97
|
## Wrapping Up
|
|
144
98
|
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
Deletion triggers: conversation ending, user completed setup, user skipped ("not now", "later"), user ignored onboarding and just did tasks.
|
|
99
|
+
Before deleting bootstrap files:
|
|
148
100
|
|
|
149
|
-
|
|
101
|
+
1. Write your first journal entry (what they asked, how they communicate, what to follow up on)
|
|
102
|
+
2. Update NOW.md with current state
|
|
103
|
+
3. Delete BOOTSTRAP.md and BOOTSTRAP-REFERENCE.md
|
|
150
104
|
|
|
151
105
|
---
|
|
152
106
|
|
|
@@ -982,6 +982,7 @@ export class AnthropicProvider implements Provider {
|
|
|
982
982
|
type: "server_tool_complete",
|
|
983
983
|
toolUseId: block.tool_use_id,
|
|
984
984
|
isError: !!isError,
|
|
985
|
+
...(Array.isArray(block.content) ? { content: block.content } : {}),
|
|
985
986
|
});
|
|
986
987
|
}
|
|
987
988
|
if (event.type === "content_block_stop") {
|
package/src/providers/types.ts
CHANGED
|
@@ -126,7 +126,7 @@ export type ProviderEvent =
|
|
|
126
126
|
toolUseId: string;
|
|
127
127
|
input: Record<string, unknown>;
|
|
128
128
|
}
|
|
129
|
-
| { type: "server_tool_complete"; toolUseId: string; isError: boolean };
|
|
129
|
+
| { type: "server_tool_complete"; toolUseId: string; isError: boolean; content?: unknown[] };
|
|
130
130
|
|
|
131
131
|
export interface SendMessageConfig {
|
|
132
132
|
model?: string;
|
package/src/runtime/AGENTS.md
CHANGED
|
@@ -52,6 +52,29 @@ Host CU allows the assistant to proxy computer-use actions (screenshots, mouse/k
|
|
|
52
52
|
- `POST /v1/host-cu-result` — `{ requestId, axTree?, axDiff?, screenshot?, screenshotWidthPx?, screenshotHeightPx?, screenWidthPt?, screenHeightPt?, executionResult?, executionError?, secondaryWindows?, userGuidance? }`
|
|
53
53
|
- **Tracking**: Uses the same `pending-interactions` tracker as the other host proxy types, with `kind: "host_cu"`. Registration happens in `conversation-routes.ts` and the route handler is in `host-cu-routes.ts`.
|
|
54
54
|
|
|
55
|
+
### Host browser (desktop proxy CDP execution)
|
|
56
|
+
|
|
57
|
+
Host browser allows the assistant to proxy CDP (Chrome DevTools Protocol) JSON-RPC commands to a browser attached on the desktop host via the client, following the same pattern as host bash, host file, and host CU.
|
|
58
|
+
|
|
59
|
+
- **Discovery**: Clients discover pending host browser requests via SSE events (`host_browser_request`) which include a `requestId`, `cdpMethod`, optional `cdpParams`, and optional `cdpSessionId`.
|
|
60
|
+
- **Resolution**: Clients execute the CDP command against the attached browser and respond via:
|
|
61
|
+
- `POST /v1/host-browser-result` — `{ requestId, content, isError }`
|
|
62
|
+
- **Tracking**: Uses the same `pending-interactions` tracker as the other host proxy types, with `kind: "host_browser"`. Registration happens in `conversation-routes.ts` and the route handler is in `host-browser-routes.ts`.
|
|
63
|
+
|
|
64
|
+
### `chrome-extension` interface (Phase 2)
|
|
65
|
+
|
|
66
|
+
The `chrome-extension` interface in `INTERFACE_IDS` is a non-interactive transport that supports only the `host_browser` capability — it does NOT support `host_bash`, `host_file`, or `host_cu`. This is encoded in `supportsHostProxy(id, capability)`: passing a capability argument returns `true` for `chrome-extension` only when the capability is `host_browser`; the no-arg form returns `false` for `chrome-extension` (so legacy desktop-only call sites that assume full-desktop proxy availability continue to gate correctly).
|
|
67
|
+
|
|
68
|
+
Unlike the SSE-based host proxies used by the macOS client, `host_browser_request` frames for the chrome-extension interface do NOT travel through `assistantEventHub`. Instead they are routed through the `ChromeExtensionRegistry` singleton (`runtime/chrome-extension-registry.ts`), which tracks active chrome-extension WebSocket connections keyed by `(guardianId, clientInstanceId)`. The registry is populated on WebSocket `open` and drained on `close` inside `http-server.ts`'s `/v1/browser-relay` handlers — see the `wsType === "browser-relay"` branches.
|
|
69
|
+
|
|
70
|
+
A single guardian may have multiple parallel extension installs connected at once (two Chrome profiles, two desktops sharing a sync identity). Each install generates a stable `clientInstanceId` on first run, persists it in `chrome.storage.local`, and sends it on every WebSocket handshake as a query param (`clientInstanceId=...`) or header (`x-client-instance-id`). The registry keys inner entries by that id so sibling installs don't evict each other on register/unregister. The default `send(guardianId, msg)` path routes to whichever instance has the most recent activity (`lastActiveAt`); `sendToInstance(guardianId, clientInstanceId, msg)` pins a specific install. Older extension builds that omit the id get a connection-scoped `legacy:<connectionId>` fallback key so they degrade gracefully to single-instance semantics.
|
|
71
|
+
|
|
72
|
+
`Conversation.hostBrowserSenderOverride` is the integration point between the turn layer and the registry. When a turn for a chrome-extension interface enters the routes layer, `conversation-routes.ts` resolves the active registry entry for the caller's guardian and sets the override to a sender that writes to that WebSocket. `Conversation.restoreBrowserProxyAvailability()` re-threads the override on queue drain — without this, the drain path would clobber the registry-routed sender with the default `sendToClient` (pointed at the SSE hub) and `host_browser_request` frames would stop reaching the extension mid-queue.
|
|
73
|
+
|
|
74
|
+
Capability token bootstrap for self-hosted deployments is handled by `routes/browser-extension-pair-routes.ts` (loopback-only; mints a guardian-bound HMAC capability token via `capability-tokens.ts`). Cloud deployments issue guardian-bound JWTs via the gateway's WorkOS-backed flow — `browser-extension-pair-routes.ts` is not involved.
|
|
75
|
+
|
|
76
|
+
See `docs/browser-use-architecture-phase2.md` for the full wire diagram and component inventory.
|
|
77
|
+
|
|
55
78
|
### Channel approvals (Telegram, Slack)
|
|
56
79
|
|
|
57
80
|
Channel approval flows use `requestId` (not `runId`) as the primary identifier:
|