@vellumai/assistant 0.3.4 → 0.3.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/Dockerfile +2 -0
- package/README.md +88 -2
- package/eslint.config.mjs +31 -0
- package/package.json +1 -1
- package/scripts/ipc/check-swift-decoder-drift.ts +4 -1
- package/scripts/ipc/generate-swift.ts +31 -2
- package/src/__tests__/__snapshots__/ipc-snapshot.test.ts.snap +438 -1
- package/src/__tests__/approval-conversation-turn.test.ts +214 -0
- package/src/__tests__/approval-hardcoded-copy-guard.test.ts +41 -0
- package/src/__tests__/approval-message-composer.test.ts +253 -0
- package/src/__tests__/browser-manager.test.ts +1 -0
- package/src/__tests__/call-conversation-messages.test.ts +130 -0
- package/src/__tests__/call-domain.test.ts +12 -2
- package/src/__tests__/call-orchestrator.test.ts +799 -249
- package/src/__tests__/call-pointer-messages.test.ts +148 -0
- package/src/__tests__/call-recovery.test.ts +3 -0
- package/src/__tests__/call-routes-http.test.ts +32 -2
- package/src/__tests__/call-store.test.ts +3 -0
- package/src/__tests__/channel-approval-routes.test.ts +1277 -98
- package/src/__tests__/channel-approval.test.ts +37 -0
- package/src/__tests__/channel-approvals.test.ts +36 -50
- package/src/__tests__/channel-guardian.test.ts +630 -22
- package/src/__tests__/channel-readiness-service.test.ts +324 -0
- package/src/__tests__/checker.test.ts +14 -7
- package/src/__tests__/clarification-resolver.test.ts +44 -24
- package/src/__tests__/commit-message-enrichment-service.test.ts +9 -4
- package/src/__tests__/computer-use-session-working-dir.test.ts +8 -0
- package/src/__tests__/config-schema.test.ts +14 -8
- package/src/__tests__/context-window-manager.test.ts +30 -2
- package/src/__tests__/contradiction-checker.test.ts +20 -5
- package/src/__tests__/credential-security-invariants.test.ts +7 -2
- package/src/__tests__/daemon-lifecycle.test.ts +13 -12
- package/src/__tests__/db-migration-rollback.test.ts +752 -0
- package/src/__tests__/dictation-mode-detection.test.ts +63 -0
- package/src/__tests__/dynamic-skill-workflow-prompt.test.ts +2 -0
- package/src/__tests__/entity-search.test.ts +615 -0
- package/src/__tests__/fuzzy-match-property.test.ts +5 -5
- package/src/__tests__/guardian-action-store.test.ts +123 -0
- package/src/__tests__/guardian-action-sweep.test.ts +277 -0
- package/src/__tests__/guardian-dispatch.test.ts +389 -0
- package/src/__tests__/guardian-question-copy.test.ts +47 -0
- package/src/__tests__/handlers-telegram-config.test.ts +4 -2
- package/src/__tests__/handlers-twilio-config.test.ts +533 -0
- package/src/__tests__/intent-routing.test.ts +2 -0
- package/src/__tests__/ipc-snapshot.test.ts +291 -1
- package/src/__tests__/memory-upsert-concurrency.test.ts +828 -0
- package/src/__tests__/messaging-send-tool.test.ts +65 -0
- package/src/__tests__/model-intents.test.ts +96 -0
- package/src/__tests__/no-direct-anthropic-sdk-imports.test.ts +42 -0
- package/src/__tests__/oauth2-gateway-transport.test.ts +130 -0
- package/src/__tests__/onboarding-starter-tasks.test.ts +2 -0
- package/src/__tests__/provider-commit-message-generator.test.ts +89 -13
- package/src/__tests__/provider-error-scenarios.test.ts +621 -0
- package/src/__tests__/provider-fail-open-selection.test.ts +119 -0
- package/src/__tests__/qdrant-manager.test.ts +27 -20
- package/src/__tests__/relay-server.test.ts +779 -40
- package/src/__tests__/run-orchestrator-assistant-events.test.ts +6 -0
- package/src/__tests__/run-orchestrator.test.ts +42 -4
- package/src/__tests__/runtime-runs-http.test.ts +17 -1
- package/src/__tests__/runtime-runs.test.ts +16 -0
- package/src/__tests__/schedule-store.test.ts +18 -4
- package/src/__tests__/scheduler-recurrence.test.ts +13 -4
- package/src/__tests__/session-abort-tool-results.test.ts +6 -0
- package/src/__tests__/session-agent-loop.test.ts +857 -0
- package/src/__tests__/session-conflict-gate.test.ts +6 -0
- package/src/__tests__/session-pre-run-repair.test.ts +6 -0
- package/src/__tests__/session-profile-injection.test.ts +6 -0
- package/src/__tests__/session-provider-retry-repair.test.ts +6 -0
- package/src/__tests__/session-queue.test.ts +6 -0
- package/src/__tests__/session-runtime-assembly.test.ts +321 -13
- package/src/__tests__/session-slash-known.test.ts +6 -0
- package/src/__tests__/session-slash-queue.test.ts +6 -0
- package/src/__tests__/session-slash-unknown.test.ts +6 -0
- package/src/__tests__/session-surfaces-task-progress.test.ts +2 -0
- package/src/__tests__/session-tool-setup-app-refresh.test.ts +1 -0
- package/src/__tests__/session-tool-setup-memory-scope.test.ts +1 -0
- package/src/__tests__/session-tool-setup-side-effect-flag.test.ts +1 -0
- package/src/__tests__/session-workspace-injection.test.ts +6 -0
- package/src/__tests__/session-workspace-tool-tracking.test.ts +6 -0
- package/src/__tests__/skills.test.ts +2 -0
- package/src/__tests__/sms-messaging-provider.test.ts +126 -0
- package/src/__tests__/starter-task-flow.test.ts +2 -0
- package/src/__tests__/swarm-dag-pathological.test.ts +535 -0
- package/src/__tests__/system-prompt.test.ts +2 -0
- package/src/__tests__/task-management-tools.test.ts +2 -2
- package/src/__tests__/task-runner.test.ts +14 -4
- package/src/__tests__/terminal-tools.test.ts +25 -19
- package/src/__tests__/tool-execution-abort-cleanup.test.ts +545 -0
- package/src/__tests__/tool-executor-shell-integration.test.ts +11 -11
- package/src/__tests__/tool-executor.test.ts +23 -24
- package/src/__tests__/trust-store.test.ts +3 -3
- package/src/__tests__/twilio-rest.test.ts +29 -0
- package/src/__tests__/twilio-routes-elevenlabs.test.ts +3 -0
- package/src/__tests__/twilio-routes-twiml.test.ts +11 -0
- package/src/__tests__/twilio-routes.test.ts +167 -11
- package/src/__tests__/twitter-cli-error-shaping.test.ts +2 -2
- package/src/__tests__/user-reference.test.ts +2 -0
- package/src/__tests__/voice-quality.test.ts +222 -0
- package/src/__tests__/web-search.test.ts +46 -30
- package/src/__tests__/work-item-output.test.ts +110 -0
- package/src/agent/loop.ts +1 -1
- package/src/agent-heartbeat/agent-heartbeat-service.ts +2 -10
- package/src/amazon/client.ts +1418 -0
- package/src/amazon/request-extractor.ts +135 -0
- package/src/amazon/session.ts +109 -0
- package/src/autonomy/autonomy-store.ts +5 -5
- package/src/browser-extension-relay/client.ts +124 -0
- package/src/browser-extension-relay/protocol.ts +63 -0
- package/src/browser-extension-relay/server.ts +177 -0
- package/src/bundler/app-bundler.ts +3 -3
- package/src/bundler/bundle-signer.ts +1 -1
- package/src/bundler/signature-verifier.ts +1 -1
- package/src/calls/call-conversation-messages.ts +33 -0
- package/src/calls/call-domain.ts +114 -10
- package/src/calls/call-orchestrator.ts +268 -59
- package/src/calls/call-pointer-messages.ts +53 -0
- package/src/calls/call-recovery.ts +3 -8
- package/src/calls/call-store.ts +69 -87
- package/src/calls/elevenlabs-config.ts +3 -2
- package/src/calls/guardian-action-sweep.ts +105 -0
- package/src/calls/guardian-dispatch.ts +203 -0
- package/src/calls/guardian-question-copy.ts +133 -0
- package/src/calls/relay-server.ts +466 -8
- package/src/calls/speaker-identification.ts +1 -1
- package/src/calls/twilio-config.ts +22 -14
- package/src/calls/twilio-provider.ts +6 -4
- package/src/calls/twilio-rest.ts +308 -7
- package/src/calls/twilio-routes.ts +65 -12
- package/src/calls/types.ts +3 -1
- package/src/channels/types.ts +25 -0
- package/src/cli/amazon.ts +815 -0
- package/src/cli/config-commands.ts +2 -2
- package/src/cli/core-commands.ts +4 -3
- package/src/cli/influencer.ts +244 -0
- package/src/cli/map.ts +89 -6
- package/src/cli.ts +1 -1
- package/src/config/agent-schema.ts +171 -0
- package/src/config/bundled-skills/amazon/SKILL.md +127 -0
- package/src/config/bundled-skills/amazon/icon.svg +13 -0
- package/src/config/bundled-skills/api-mapping/SKILL.md +78 -0
- package/src/config/bundled-skills/browser/SKILL.md +1 -0
- package/src/config/bundled-skills/browser/TOOLS.json +17 -0
- package/src/config/bundled-skills/browser/tools/browser-wait-for-download.ts +25 -0
- package/src/config/bundled-skills/doordash/SKILL.md +51 -51
- package/src/config/bundled-skills/email-setup/SKILL.md +14 -5
- package/src/config/bundled-skills/google-oauth-setup/SKILL.md +183 -0
- package/src/config/bundled-skills/influencer/SKILL.md +144 -0
- package/src/config/bundled-skills/knowledge-graph/SKILL.md +15 -0
- package/src/config/bundled-skills/knowledge-graph/TOOLS.json +56 -0
- package/src/config/bundled-skills/knowledge-graph/tools/graph-query.ts +185 -0
- package/src/config/bundled-skills/macos-automation/icon.svg +12 -0
- package/src/config/bundled-skills/media-processing/SKILL.md +176 -0
- package/src/config/bundled-skills/media-processing/TOOLS.json +230 -0
- package/src/config/bundled-skills/media-processing/__tests__/concurrency-pool.test.ts +77 -0
- package/src/config/bundled-skills/media-processing/__tests__/cost-tracker.test.ts +69 -0
- package/src/config/bundled-skills/media-processing/__tests__/preprocess.test.ts +303 -0
- package/src/config/bundled-skills/media-processing/services/concurrency-pool.ts +55 -0
- package/src/config/bundled-skills/media-processing/services/cost-tracker.ts +86 -0
- package/src/config/bundled-skills/media-processing/services/gemini-map.ts +339 -0
- package/src/config/bundled-skills/media-processing/services/preprocess.ts +551 -0
- package/src/config/bundled-skills/media-processing/services/processing-pipeline.ts +259 -0
- package/src/config/bundled-skills/media-processing/services/reduce.ts +197 -0
- package/src/config/bundled-skills/media-processing/tools/analyze-keyframes.ts +136 -0
- package/src/config/bundled-skills/media-processing/tools/extract-keyframes.ts +59 -0
- package/src/config/bundled-skills/media-processing/tools/generate-clip.ts +195 -0
- package/src/config/bundled-skills/media-processing/tools/ingest-media.ts +197 -0
- package/src/config/bundled-skills/media-processing/tools/media-diagnostics.ts +143 -0
- package/src/config/bundled-skills/media-processing/tools/media-status.ts +75 -0
- package/src/config/bundled-skills/media-processing/tools/query-media-events.ts +65 -0
- package/src/config/bundled-skills/messaging/SKILL.md +33 -8
- package/src/config/bundled-skills/messaging/tools/messaging-analyze-style.ts +4 -7
- package/src/config/bundled-skills/messaging/tools/messaging-reply.ts +2 -1
- package/src/config/bundled-skills/messaging/tools/messaging-send.ts +5 -1
- package/src/config/bundled-skills/phone-calls/SKILL.md +88 -23
- package/src/config/bundled-skills/twitter/SKILL.md +19 -3
- package/src/config/bundled-skills/twitter/icon.svg +14 -0
- package/src/config/bundled-tool-registry.ts +310 -0
- package/src/config/calls-schema.ts +181 -0
- package/src/config/core-schema.ts +309 -0
- package/src/config/defaults.ts +28 -3
- package/src/config/env-registry.ts +162 -0
- package/src/config/env.ts +175 -0
- package/src/config/loader.ts +6 -6
- package/src/config/memory-schema.ts +528 -0
- package/src/config/sandbox-schema.ts +55 -0
- package/src/config/schema.ts +158 -1133
- package/src/config/skill-state.ts +1 -1
- package/src/config/skills-schema.ts +32 -0
- package/src/config/skills.ts +35 -24
- package/src/config/system-prompt.ts +131 -56
- package/src/config/templates/IDENTITY.md +2 -2
- package/src/config/templates/SOUL.md +1 -1
- package/src/config/types.ts +1 -0
- package/src/config/user-reference.ts +4 -9
- package/src/config/vellum-skills/catalog.json +6 -7
- package/src/config/vellum-skills/chatgpt-import/tools/chatgpt-import.ts +5 -1
- package/src/config/vellum-skills/slack-oauth-setup/SKILL.md +4 -3
- package/src/config/vellum-skills/sms-setup/SKILL.md +216 -0
- package/src/config/vellum-skills/twilio-setup/SKILL.md +40 -8
- package/src/context/window-manager.ts +27 -7
- package/src/daemon/approval-generators.ts +186 -0
- package/src/daemon/approved-devices-store.ts +140 -0
- package/src/daemon/assistant-attachments.ts +1 -1
- package/src/daemon/classifier.ts +35 -32
- package/src/daemon/config-watcher.ts +1 -1
- package/src/daemon/daemon-control.ts +217 -0
- package/src/daemon/handlers/apps.ts +2 -3
- package/src/daemon/handlers/config-channels.ts +158 -0
- package/src/daemon/handlers/config-inbox.ts +540 -0
- package/src/daemon/handlers/config-ingress.ts +231 -0
- package/src/daemon/handlers/config-integrations.ts +258 -0
- package/src/daemon/handlers/config-model.ts +143 -0
- package/src/daemon/handlers/config-parental.ts +163 -0
- package/src/daemon/handlers/config-scheduling.ts +172 -0
- package/src/daemon/handlers/config-slack.ts +92 -0
- package/src/daemon/handlers/config-telegram.ts +301 -0
- package/src/daemon/handlers/config-tools.ts +177 -0
- package/src/daemon/handlers/config-trust.ts +104 -0
- package/src/daemon/handlers/config-twilio.ts +1080 -0
- package/src/daemon/handlers/config.ts +53 -1689
- package/src/daemon/handlers/diagnostics.ts +1 -1
- package/src/daemon/handlers/dictation.ts +180 -0
- package/src/daemon/handlers/documents.ts +18 -32
- package/src/daemon/handlers/identity.ts +14 -23
- package/src/daemon/handlers/index.ts +11 -0
- package/src/daemon/handlers/misc.ts +3 -5
- package/src/daemon/handlers/pairing.ts +98 -0
- package/src/daemon/handlers/sessions.ts +56 -5
- package/src/daemon/handlers/shared.ts +6 -1
- package/src/daemon/handlers/skills.ts +1 -1
- package/src/daemon/handlers/twitter-auth.ts +2 -0
- package/src/daemon/handlers/work-items.ts +17 -9
- package/src/daemon/handlers/workspace-files.ts +4 -3
- package/src/daemon/install-cli-launchers.ts +113 -0
- package/src/daemon/ipc-contract/apps.ts +356 -0
- package/src/daemon/ipc-contract/browser.ts +74 -0
- package/src/daemon/ipc-contract/computer-use.ts +151 -0
- package/src/daemon/ipc-contract/diagnostics.ts +56 -0
- package/src/daemon/ipc-contract/documents.ts +74 -0
- package/src/daemon/ipc-contract/inbox.ts +209 -0
- package/src/daemon/ipc-contract/integrations.ts +284 -0
- package/src/daemon/ipc-contract/memory.ts +48 -0
- package/src/daemon/ipc-contract/messages.ts +211 -0
- package/src/daemon/ipc-contract/pairing.ts +45 -0
- package/src/daemon/ipc-contract/parental-control.ts +95 -0
- package/src/daemon/ipc-contract/schedules.ts +97 -0
- package/src/daemon/ipc-contract/sessions.ts +315 -0
- package/src/daemon/ipc-contract/shared.ts +42 -0
- package/src/daemon/ipc-contract/skills.ts +120 -0
- package/src/daemon/ipc-contract/subagents.ts +58 -0
- package/src/daemon/ipc-contract/surfaces.ts +250 -0
- package/src/daemon/ipc-contract/trust.ts +60 -0
- package/src/daemon/ipc-contract/work-items.ts +225 -0
- package/src/daemon/ipc-contract/workspace.ts +113 -0
- package/src/daemon/ipc-contract-inventory.json +70 -0
- package/src/daemon/ipc-contract-inventory.ts +55 -29
- package/src/daemon/ipc-contract.ts +229 -2426
- package/src/daemon/ipc-protocol.ts +1 -1
- package/src/daemon/ipc-validate.ts +7 -0
- package/src/daemon/lifecycle.ts +97 -377
- package/src/daemon/pairing-store.ts +177 -0
- package/src/daemon/providers-setup.ts +43 -0
- package/src/daemon/ride-shotgun-handler.ts +68 -3
- package/src/daemon/server.ts +66 -46
- package/src/daemon/session-agent-loop-handlers.ts +421 -0
- package/src/daemon/session-agent-loop.ts +117 -275
- package/src/daemon/session-dynamic-profile.ts +1 -1
- package/src/daemon/session-history.ts +1 -1
- package/src/daemon/session-media-retry.ts +1 -1
- package/src/daemon/session-messaging.ts +37 -2
- package/src/daemon/session-notifiers.ts +5 -25
- package/src/daemon/session-process.ts +99 -59
- package/src/daemon/session-queue-manager.ts +96 -4
- package/src/daemon/session-runtime-assembly.ts +199 -10
- package/src/daemon/session-surfaces.ts +19 -4
- package/src/daemon/session-tool-setup.ts +30 -30
- package/src/daemon/session-workspace.ts +1 -1
- package/src/daemon/session.ts +35 -2
- package/src/daemon/shutdown-handlers.ts +122 -0
- package/src/daemon/trace-emitter.ts +1 -1
- package/src/daemon/watch-handler.ts +36 -33
- package/src/doordash/cart-queries.ts +787 -0
- package/src/doordash/client.ts +144 -127
- package/src/doordash/order-queries.ts +85 -0
- package/src/doordash/queries.ts +10 -1308
- package/src/doordash/search-queries.ts +203 -0
- package/src/doordash/session.ts +3 -2
- package/src/doordash/store-queries.ts +246 -0
- package/src/doordash/types.ts +367 -0
- package/src/email/providers/agentmail.ts +2 -1
- package/src/email/providers/index.ts +3 -2
- package/src/email/service.ts +3 -2
- package/src/errors.ts +43 -0
- package/src/home-base/prebuilt/seed.ts +1 -1
- package/src/hooks/cli.ts +6 -5
- package/src/hooks/config.ts +6 -8
- package/src/hooks/discovery.ts +6 -5
- package/src/hooks/manager.ts +4 -3
- package/src/hooks/runner.ts +2 -2
- package/src/hooks/templates.ts +5 -5
- package/src/inbound/public-ingress-urls.ts +6 -4
- package/src/index.ts +4 -2
- package/src/influencer/client.ts +1104 -0
- package/src/instrument.ts +4 -3
- package/src/logfire.ts +4 -3
- package/src/memory/admin.ts +25 -35
- package/src/memory/attachments-store.ts +4 -7
- package/src/memory/channel-delivery-store.ts +30 -1
- package/src/memory/channel-guardian-store.ts +202 -2
- package/src/memory/clarification-resolver.ts +37 -33
- package/src/memory/conflict-store.ts +67 -61
- package/src/memory/contradiction-checker.ts +141 -117
- package/src/memory/conversation-store.ts +335 -51
- package/src/memory/db-connection.ts +27 -4
- package/src/memory/db-init.ts +265 -4
- package/src/memory/db.ts +14 -1
- package/src/memory/embedding-backend.ts +27 -5
- package/src/memory/embedding-ollama.ts +2 -1
- package/src/memory/entity-extractor.ts +38 -35
- package/src/memory/guardian-action-store.ts +430 -0
- package/src/memory/inbox-escalation-projection.ts +59 -0
- package/src/memory/inbox-thread-store.ts +218 -0
- package/src/memory/ingress-invite-store.ts +338 -0
- package/src/memory/ingress-member-store.ts +350 -0
- package/src/memory/items-extractor.ts +91 -97
- package/src/memory/job-handlers/index-maintenance.ts +3 -3
- package/src/memory/job-handlers/media-processing.ts +69 -0
- package/src/memory/job-handlers/summarization.ts +32 -26
- package/src/memory/job-utils.ts +3 -10
- package/src/memory/jobs-store.ts +8 -10
- package/src/memory/jobs-worker.ts +55 -36
- package/src/memory/media-store.ts +759 -0
- package/src/memory/migrations/001-job-deferrals.ts +45 -0
- package/src/memory/migrations/002-tool-invocations-fk.ts +43 -0
- package/src/memory/migrations/003-memory-fts-backfill.ts +24 -0
- package/src/memory/migrations/004-entity-relation-dedup.ts +87 -0
- package/src/memory/migrations/005-fingerprint-scope-unique.ts +80 -0
- package/src/memory/migrations/006-scope-salted-fingerprints.ts +62 -0
- package/src/memory/migrations/007-assistant-id-to-self.ts +254 -0
- package/src/memory/migrations/008-remove-assistant-id-columns.ts +208 -0
- package/src/memory/migrations/009-llm-usage-events-drop-assistant-id.ts +83 -0
- package/src/memory/migrations/010-ext-conv-bindings-channel-chat-unique.ts +56 -0
- package/src/memory/migrations/011-call-sessions-provider-sid-dedup.ts +63 -0
- package/src/memory/migrations/012-call-sessions-add-initiated-from.ts +19 -0
- package/src/memory/migrations/013-guardian-action-tables.ts +68 -0
- package/src/memory/migrations/014-backfill-inbox-thread-state.ts +76 -0
- package/src/memory/migrations/015-drop-active-search-index.ts +27 -0
- package/src/memory/migrations/016-memory-segments-indexes.ts +11 -0
- package/src/memory/migrations/017-memory-items-indexes.ts +10 -0
- package/src/memory/migrations/018-remaining-table-indexes.ts +13 -0
- package/src/memory/migrations/index.ts +24 -0
- package/src/memory/migrations/registry.ts +79 -0
- package/src/memory/migrations/validate-migration-state.ts +69 -0
- package/src/memory/qdrant-manager.ts +49 -8
- package/src/memory/query-builder.ts +1 -1
- package/src/memory/raw-query.ts +119 -0
- package/src/memory/recall-cache.ts +4 -1
- package/src/memory/retriever.ts +165 -47
- package/src/memory/schema-migration.ts +25 -984
- package/src/memory/schema.ts +228 -7
- package/src/memory/search/entity.ts +205 -31
- package/src/memory/search/lexical.ts +81 -52
- package/src/memory/search/ranking.ts +27 -23
- package/src/memory/search/semantic.ts +157 -19
- package/src/memory/search/types.ts +24 -0
- package/src/memory/shared-app-links-store.ts +4 -5
- package/src/memory/validation.ts +19 -0
- package/src/messaging/draft-store.ts +5 -6
- package/src/messaging/provider-types.ts +2 -0
- package/src/messaging/providers/sms/adapter.ts +201 -0
- package/src/messaging/providers/sms/client.ts +93 -0
- package/src/messaging/providers/sms/types.ts +7 -0
- package/src/messaging/providers/telegram-bot/adapter.ts +2 -5
- package/src/messaging/providers/whatsapp/adapter.ts +136 -0
- package/src/messaging/providers/whatsapp/client.ts +67 -0
- package/src/messaging/style-analyzer.ts +5 -4
- package/src/messaging/thread-summarizer.ts +61 -69
- package/src/messaging/triage-engine.ts +62 -71
- package/src/migrations/config-merge.ts +53 -0
- package/src/migrations/data-layout.ts +68 -0
- package/src/migrations/data-merge.ts +33 -0
- package/src/migrations/hooks-merge.ts +90 -0
- package/src/migrations/index.ts +6 -0
- package/src/migrations/log.ts +23 -0
- package/src/migrations/skills-merge.ts +33 -0
- package/src/migrations/workspace-layout.ts +79 -0
- package/src/permissions/checker.ts +133 -11
- package/src/permissions/prompter.ts +14 -0
- package/src/permissions/shell-identity.ts +31 -1
- package/src/permissions/trust-store.ts +21 -1
- package/src/providers/anthropic/client.ts +4 -4
- package/src/providers/failover.ts +2 -2
- package/src/providers/model-intents.ts +70 -0
- package/src/providers/ollama/client.ts +2 -1
- package/src/providers/provider-send-message.ts +176 -0
- package/src/providers/registry.ts +71 -30
- package/src/providers/retry.ts +35 -1
- package/src/providers/types.ts +12 -1
- package/src/runtime/approval-conversation-turn.ts +97 -0
- package/src/runtime/approval-message-composer.ts +253 -0
- package/src/runtime/channel-approval-parser.ts +36 -2
- package/src/runtime/channel-approvals.ts +11 -24
- package/src/runtime/channel-guardian-service.ts +88 -21
- package/src/runtime/channel-readiness-service.ts +418 -0
- package/src/runtime/channel-readiness-types.ts +35 -0
- package/src/runtime/channel-retry-sweep.ts +184 -0
- package/src/runtime/guardian-context-resolver.ts +108 -0
- package/src/runtime/http-server.ts +275 -717
- package/src/runtime/http-types.ts +59 -3
- package/src/runtime/middleware/auth.ts +116 -0
- package/src/runtime/middleware/error-handler.ts +33 -0
- package/src/runtime/middleware/twilio-validation.ts +127 -0
- package/src/runtime/routes/app-routes.ts +1 -1
- package/src/runtime/routes/call-routes.ts +51 -7
- package/src/runtime/routes/channel-delivery-routes.ts +170 -0
- package/src/runtime/routes/channel-guardian-routes.ts +1191 -0
- package/src/runtime/routes/channel-inbound-routes.ts +1152 -0
- package/src/runtime/routes/channel-route-shared.ts +144 -0
- package/src/runtime/routes/channel-routes.ts +32 -1588
- package/src/runtime/routes/conversation-routes.ts +50 -7
- package/src/runtime/routes/events-routes.ts +2 -2
- package/src/runtime/routes/identity-routes.ts +126 -0
- package/src/runtime/routes/pairing-routes.ts +143 -0
- package/src/runtime/routes/run-routes.ts +15 -1
- package/src/runtime/run-orchestrator.ts +86 -35
- package/src/schedule/schedule-store.ts +36 -32
- package/src/schedule/scheduler.ts +3 -3
- package/src/security/encrypted-store.ts +5 -7
- package/src/security/oauth2.ts +45 -15
- package/src/security/parental-control-store.ts +183 -0
- package/src/security/secret-allowlist.ts +4 -3
- package/src/security/secret-scanner.ts +5 -5
- package/src/security/secure-keys.ts +1 -1
- package/src/security/token-manager.ts +3 -2
- package/src/services/vercel-deploy.ts +6 -2
- package/src/skills/tool-manifest.ts +3 -3
- package/src/skills/vellum-catalog-remote.ts +75 -16
- package/src/slack/slack-webhook.ts +2 -1
- package/src/swarm/orchestrator.ts +92 -1
- package/src/swarm/router-planner.ts +6 -9
- package/src/swarm/worker-prompts.ts +9 -12
- package/src/tasks/task-compiler.ts +19 -28
- package/src/tasks/task-runner.ts +1 -1
- package/src/tools/assets/materialize.ts +2 -2
- package/src/tools/assets/search.ts +15 -14
- package/src/tools/browser/__tests__/auth-detector.test.ts +1 -0
- package/src/tools/browser/auto-navigate.ts +1 -0
- package/src/tools/browser/browser-execution.ts +10 -1
- package/src/tools/browser/browser-manager.ts +119 -4
- package/src/tools/browser/network-recorder.ts +5 -0
- package/src/tools/calls/call-start.ts +1 -0
- package/src/tools/credentials/broker.ts +11 -2
- package/src/tools/credentials/metadata-store.ts +18 -14
- package/src/tools/credentials/post-connect-hooks.ts +61 -0
- package/src/tools/credentials/vault.ts +49 -23
- package/src/tools/execution-target.ts +11 -1
- package/src/tools/executor.ts +68 -9
- package/src/tools/host-terminal/cli-discover.ts +1 -1
- package/src/tools/network/script-proxy/http-forwarder.ts +1 -1
- package/src/tools/network/script-proxy/mitm-handler.ts +1 -1
- package/src/tools/network/script-proxy/server.ts +1 -1
- package/src/tools/network/script-proxy/session-manager.ts +6 -5
- package/src/tools/network/web-fetch.ts +18 -2
- package/src/tools/network/web-search.ts +8 -4
- package/src/tools/reminder/reminder-store.ts +14 -15
- package/src/tools/schedule/create.ts +1 -0
- package/src/tools/schedule/list.ts +2 -1
- package/src/tools/shared/filesystem/file-ops-service.ts +5 -7
- package/src/tools/skills/skill-script-runner.ts +24 -9
- package/src/tools/skills/skill-tool-factory.ts +1 -0
- package/src/tools/tasks/work-item-enqueue.ts +2 -2
- package/src/tools/terminal/evaluate-typescript.ts +21 -12
- package/src/tools/terminal/parser.ts +50 -0
- package/src/tools/types.ts +2 -0
- package/src/tools/watcher/delete.ts +6 -0
- package/src/tools/weather/service.ts +1 -1
- package/src/twitter/client.ts +190 -24
- package/src/twitter/router.ts +1 -1
- package/src/twitter/session.ts +4 -3
- package/src/util/clipboard.ts +1 -1
- package/src/util/errors.ts +65 -8
- package/src/util/fs.ts +40 -0
- package/src/util/json.ts +10 -0
- package/src/util/log-redact.ts +189 -0
- package/src/util/logger.ts +19 -17
- package/src/util/object.ts +3 -0
- package/src/util/platform.ts +105 -363
- package/src/util/pricing.ts +1 -1
- package/src/util/promise-guard.ts +1 -1
- package/src/util/retry.ts +19 -0
- package/src/util/row-mapper.ts +79 -0
- package/src/util/silently.ts +21 -0
- package/src/watcher/engine.ts +5 -1
- package/src/watcher/provider-types.ts +20 -0
- package/src/watcher/providers/github.ts +156 -0
- package/src/watcher/providers/gmail.ts +1 -0
- package/src/watcher/providers/google-calendar.ts +1 -0
- package/src/watcher/providers/linear.ts +460 -0
- package/src/watcher/providers/slack.ts +1 -0
- package/src/work-items/work-item-runner.ts +1 -1
- package/src/workspace/git-service.ts +1 -1
- package/src/workspace/provider-commit-message-generator.ts +51 -22
- package/src/__tests__/call-bridge.test.ts +0 -517
- package/src/__tests__/session-process-bridge.test.ts +0 -244
- package/src/calls/call-bridge.ts +0 -168
- package/src/config/vellum-skills/google-oauth-setup/SKILL.md +0 -199
|
@@ -1,244 +0,0 @@
|
|
|
1
|
-
import { describe, test, expect, beforeEach, mock } from 'bun:test';
|
|
2
|
-
import { mkdtempSync } from 'node:fs';
|
|
3
|
-
import { tmpdir } from 'node:os';
|
|
4
|
-
import { join } from 'node:path';
|
|
5
|
-
|
|
6
|
-
const testDir = mkdtempSync(join(tmpdir(), 'session-process-bridge-test-'));
|
|
7
|
-
|
|
8
|
-
// ── Platform + logger mocks ─────────────────────────────────────────
|
|
9
|
-
|
|
10
|
-
mock.module('../util/platform.js', () => ({
|
|
11
|
-
getDataDir: () => testDir,
|
|
12
|
-
isMacOS: () => process.platform === 'darwin',
|
|
13
|
-
isLinux: () => process.platform === 'linux',
|
|
14
|
-
isWindows: () => process.platform === 'win32',
|
|
15
|
-
getSocketPath: () => join(testDir, 'test.sock'),
|
|
16
|
-
getPidPath: () => join(testDir, 'test.pid'),
|
|
17
|
-
getDbPath: () => join(testDir, 'test.db'),
|
|
18
|
-
getLogPath: () => join(testDir, 'test.log'),
|
|
19
|
-
ensureDataDir: () => {},
|
|
20
|
-
}));
|
|
21
|
-
|
|
22
|
-
mock.module('../util/logger.js', () => ({
|
|
23
|
-
getLogger: () =>
|
|
24
|
-
new Proxy({} as Record<string, unknown>, {
|
|
25
|
-
get: () => () => {},
|
|
26
|
-
}),
|
|
27
|
-
}));
|
|
28
|
-
|
|
29
|
-
mock.module('../config/loader.js', () => ({
|
|
30
|
-
getConfig: () => ({
|
|
31
|
-
apiKeys: { anthropic: 'test-key' },
|
|
32
|
-
model: 'claude-sonnet-4-20250514',
|
|
33
|
-
provider: 'anthropic',
|
|
34
|
-
memory: { enabled: false },
|
|
35
|
-
calls: { enabled: false },
|
|
36
|
-
contextWindow: { maxInputTokens: 200000 },
|
|
37
|
-
}),
|
|
38
|
-
}));
|
|
39
|
-
|
|
40
|
-
// ── Mock the call bridge ─────────────────────────────────────────────
|
|
41
|
-
|
|
42
|
-
import type { CallBridgeResult } from '../calls/call-bridge.js';
|
|
43
|
-
|
|
44
|
-
const mockTryRouteCallMessage = mock(
|
|
45
|
-
(_convId: string, _text: string, _msgId?: string): Promise<CallBridgeResult> =>
|
|
46
|
-
Promise.resolve({ handled: false, reason: 'no_active_call' }),
|
|
47
|
-
);
|
|
48
|
-
|
|
49
|
-
mock.module('../calls/call-bridge.js', () => ({
|
|
50
|
-
tryRouteCallMessage: (...args: [string, string, string?]) => mockTryRouteCallMessage(...args),
|
|
51
|
-
}));
|
|
52
|
-
|
|
53
|
-
// ── Mock slash resolution ────────────────────────────────────────────
|
|
54
|
-
|
|
55
|
-
mock.module('./session-slash.js', () => ({
|
|
56
|
-
resolveSlash: (content: string) => ({ kind: 'passthrough' as const, content }),
|
|
57
|
-
}));
|
|
58
|
-
|
|
59
|
-
// ── Import after mocks ──────────────────────────────────────────────
|
|
60
|
-
|
|
61
|
-
import type { ServerMessage } from '../daemon/ipc-protocol.js';
|
|
62
|
-
import type { ProcessSessionContext } from '../daemon/session-process.js';
|
|
63
|
-
import { processMessage, drainQueue } from '../daemon/session-process.js';
|
|
64
|
-
import { MessageQueue } from '../daemon/session-queue-manager.js';
|
|
65
|
-
|
|
66
|
-
// ── Session mock factory ─────────────────────────────────────────────
|
|
67
|
-
|
|
68
|
-
function createMockSession(overrides?: Partial<ProcessSessionContext>): ProcessSessionContext {
|
|
69
|
-
return {
|
|
70
|
-
conversationId: 'test-conv',
|
|
71
|
-
messages: [],
|
|
72
|
-
processing: false,
|
|
73
|
-
abortController: null,
|
|
74
|
-
currentRequestId: undefined,
|
|
75
|
-
queue: new MessageQueue(),
|
|
76
|
-
traceEmitter: {
|
|
77
|
-
emit: () => {},
|
|
78
|
-
} as unknown as ProcessSessionContext['traceEmitter'],
|
|
79
|
-
usageStats: { inputTokens: 0, outputTokens: 0, estimatedCost: 0 },
|
|
80
|
-
persistUserMessage: mock((_content: string, _attachments: unknown[], _requestId?: string) => 'mock-msg-id'),
|
|
81
|
-
runAgentLoop: mock(async () => {}),
|
|
82
|
-
...overrides,
|
|
83
|
-
};
|
|
84
|
-
}
|
|
85
|
-
|
|
86
|
-
// ── Tests ────────────────────────────────────────────────────────────
|
|
87
|
-
|
|
88
|
-
describe('session-process bridge consumption', () => {
|
|
89
|
-
beforeEach(() => {
|
|
90
|
-
mockTryRouteCallMessage.mockReset();
|
|
91
|
-
});
|
|
92
|
-
|
|
93
|
-
// ── Direct processMessage path ───────────────────────────────
|
|
94
|
-
|
|
95
|
-
test('processMessage emits assistant_text_delta + message_complete when bridge consumes with userFacingText', async () => {
|
|
96
|
-
mockTryRouteCallMessage.mockResolvedValue({
|
|
97
|
-
handled: true,
|
|
98
|
-
userFacingText: 'Instruction relayed to active call.',
|
|
99
|
-
});
|
|
100
|
-
|
|
101
|
-
const events: ServerMessage[] = [];
|
|
102
|
-
const onEvent = (msg: ServerMessage) => events.push(msg);
|
|
103
|
-
const session = createMockSession();
|
|
104
|
-
|
|
105
|
-
await processMessage(session, 'ask about pricing', [], onEvent);
|
|
106
|
-
|
|
107
|
-
// Should have emitted text delta then message_complete
|
|
108
|
-
const textDelta = events.find((e) => e.type === 'assistant_text_delta');
|
|
109
|
-
expect(textDelta).toBeDefined();
|
|
110
|
-
expect((textDelta as { text: string }).text).toBe('Instruction relayed to active call.');
|
|
111
|
-
|
|
112
|
-
const complete = events.find((e) => e.type === 'message_complete');
|
|
113
|
-
expect(complete).toBeDefined();
|
|
114
|
-
|
|
115
|
-
// Should NOT have called runAgentLoop
|
|
116
|
-
expect(session.runAgentLoop).not.toHaveBeenCalled();
|
|
117
|
-
});
|
|
118
|
-
|
|
119
|
-
test('processMessage emits failure text when bridge consumes with failure userFacingText', async () => {
|
|
120
|
-
mockTryRouteCallMessage.mockResolvedValue({
|
|
121
|
-
handled: true,
|
|
122
|
-
reason: 'instruction_relay_failed',
|
|
123
|
-
userFacingText: 'Failed to relay instruction to the active call.',
|
|
124
|
-
});
|
|
125
|
-
|
|
126
|
-
const events: ServerMessage[] = [];
|
|
127
|
-
const onEvent = (msg: ServerMessage) => events.push(msg);
|
|
128
|
-
const session = createMockSession();
|
|
129
|
-
|
|
130
|
-
await processMessage(session, 'change the topic', [], onEvent);
|
|
131
|
-
|
|
132
|
-
const textDelta = events.find((e) => e.type === 'assistant_text_delta');
|
|
133
|
-
expect(textDelta).toBeDefined();
|
|
134
|
-
expect((textDelta as { text: string }).text).toBe('Failed to relay instruction to the active call.');
|
|
135
|
-
|
|
136
|
-
const complete = events.find((e) => e.type === 'message_complete');
|
|
137
|
-
expect(complete).toBeDefined();
|
|
138
|
-
|
|
139
|
-
// Only one message_complete
|
|
140
|
-
const completeCount = events.filter((e) => e.type === 'message_complete').length;
|
|
141
|
-
expect(completeCount).toBe(1);
|
|
142
|
-
|
|
143
|
-
expect(session.runAgentLoop).not.toHaveBeenCalled();
|
|
144
|
-
});
|
|
145
|
-
|
|
146
|
-
test('processMessage skips text delta when bridge consumes without userFacingText', async () => {
|
|
147
|
-
mockTryRouteCallMessage.mockResolvedValue({
|
|
148
|
-
handled: true,
|
|
149
|
-
});
|
|
150
|
-
|
|
151
|
-
const events: ServerMessage[] = [];
|
|
152
|
-
const onEvent = (msg: ServerMessage) => events.push(msg);
|
|
153
|
-
const session = createMockSession();
|
|
154
|
-
|
|
155
|
-
await processMessage(session, 'hello', [], onEvent);
|
|
156
|
-
|
|
157
|
-
const textDelta = events.find((e) => e.type === 'assistant_text_delta');
|
|
158
|
-
expect(textDelta).toBeUndefined();
|
|
159
|
-
|
|
160
|
-
const complete = events.find((e) => e.type === 'message_complete');
|
|
161
|
-
expect(complete).toBeDefined();
|
|
162
|
-
|
|
163
|
-
expect(session.runAgentLoop).not.toHaveBeenCalled();
|
|
164
|
-
});
|
|
165
|
-
|
|
166
|
-
test('processMessage falls through to agent loop when bridge does not consume', async () => {
|
|
167
|
-
mockTryRouteCallMessage.mockResolvedValue({
|
|
168
|
-
handled: false,
|
|
169
|
-
reason: 'no_active_call',
|
|
170
|
-
});
|
|
171
|
-
|
|
172
|
-
const events: ServerMessage[] = [];
|
|
173
|
-
const onEvent = (msg: ServerMessage) => events.push(msg);
|
|
174
|
-
const session = createMockSession();
|
|
175
|
-
|
|
176
|
-
await processMessage(session, 'normal message', [], onEvent);
|
|
177
|
-
|
|
178
|
-
expect(session.runAgentLoop).toHaveBeenCalled();
|
|
179
|
-
});
|
|
180
|
-
|
|
181
|
-
// ── Queued routeOrProcess path ───────────────────────────────
|
|
182
|
-
|
|
183
|
-
test('drainQueue emits assistant_text_delta + message_complete for bridge-consumed queued message', async () => {
|
|
184
|
-
mockTryRouteCallMessage.mockResolvedValue({
|
|
185
|
-
handled: true,
|
|
186
|
-
userFacingText: 'Instruction relayed to active call.',
|
|
187
|
-
});
|
|
188
|
-
|
|
189
|
-
const events: ServerMessage[] = [];
|
|
190
|
-
const onEvent = (msg: ServerMessage) => events.push(msg);
|
|
191
|
-
const session = createMockSession({ processing: true });
|
|
192
|
-
|
|
193
|
-
// Enqueue a message
|
|
194
|
-
session.queue.push({
|
|
195
|
-
content: 'ask about pricing',
|
|
196
|
-
attachments: [],
|
|
197
|
-
requestId: 'req-1',
|
|
198
|
-
onEvent,
|
|
199
|
-
});
|
|
200
|
-
|
|
201
|
-
drainQueue(session);
|
|
202
|
-
|
|
203
|
-
// Wait for async routeOrProcess
|
|
204
|
-
await new Promise((r) => setTimeout(r, 50));
|
|
205
|
-
|
|
206
|
-
const textDelta = events.find((e) => e.type === 'assistant_text_delta');
|
|
207
|
-
expect(textDelta).toBeDefined();
|
|
208
|
-
expect((textDelta as { text: string }).text).toBe('Instruction relayed to active call.');
|
|
209
|
-
|
|
210
|
-
// message_complete (from dequeue + bridge consumption — only one expected for this request)
|
|
211
|
-
const completeEvents = events.filter((e) => e.type === 'message_complete');
|
|
212
|
-
expect(completeEvents.length).toBe(1);
|
|
213
|
-
|
|
214
|
-
expect(session.runAgentLoop).not.toHaveBeenCalled();
|
|
215
|
-
});
|
|
216
|
-
|
|
217
|
-
test('drainQueue emits failure text for bridge-consumed queued message with relay failure', async () => {
|
|
218
|
-
mockTryRouteCallMessage.mockResolvedValue({
|
|
219
|
-
handled: true,
|
|
220
|
-
reason: 'instruction_relay_failed',
|
|
221
|
-
userFacingText: 'Failed to relay instruction to the active call.',
|
|
222
|
-
});
|
|
223
|
-
|
|
224
|
-
const events: ServerMessage[] = [];
|
|
225
|
-
const onEvent = (msg: ServerMessage) => events.push(msg);
|
|
226
|
-
const session = createMockSession({ processing: true });
|
|
227
|
-
|
|
228
|
-
session.queue.push({
|
|
229
|
-
content: 'change the topic',
|
|
230
|
-
attachments: [],
|
|
231
|
-
requestId: 'req-2',
|
|
232
|
-
onEvent,
|
|
233
|
-
});
|
|
234
|
-
|
|
235
|
-
drainQueue(session);
|
|
236
|
-
await new Promise((r) => setTimeout(r, 50));
|
|
237
|
-
|
|
238
|
-
const textDelta = events.find((e) => e.type === 'assistant_text_delta');
|
|
239
|
-
expect(textDelta).toBeDefined();
|
|
240
|
-
expect((textDelta as { text: string }).text).toBe('Failed to relay instruction to the active call.');
|
|
241
|
-
|
|
242
|
-
expect(session.runAgentLoop).not.toHaveBeenCalled();
|
|
243
|
-
});
|
|
244
|
-
});
|
package/src/calls/call-bridge.ts
DELETED
|
@@ -1,168 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* Call message bridge: intercepts user messages in-thread and routes them
|
|
3
|
-
* to the live call orchestrator — either as answers to pending questions
|
|
4
|
-
* or as mid-call steering instructions.
|
|
5
|
-
*
|
|
6
|
-
* Decision priority:
|
|
7
|
-
* 1. If a pending question exists → answer path (existing behavior).
|
|
8
|
-
* 2. If no pending question but an active call exists → instruction path.
|
|
9
|
-
*
|
|
10
|
-
* When the bridge consumes a message it returns `{ handled: true }` so
|
|
11
|
-
* the caller can skip agent processing.
|
|
12
|
-
*/
|
|
13
|
-
|
|
14
|
-
import { getLogger } from '../util/logger.js';
|
|
15
|
-
import {
|
|
16
|
-
getActiveCallSessionForConversation,
|
|
17
|
-
getPendingQuestion,
|
|
18
|
-
answerPendingQuestion,
|
|
19
|
-
recordCallEvent,
|
|
20
|
-
getCallSession,
|
|
21
|
-
} from './call-store.js';
|
|
22
|
-
import { getCallOrchestrator } from './call-state.js';
|
|
23
|
-
import { relayInstruction } from './call-domain.js';
|
|
24
|
-
import * as conversationStore from '../memory/conversation-store.js';
|
|
25
|
-
|
|
26
|
-
const log = getLogger('call-bridge');
|
|
27
|
-
|
|
28
|
-
export interface CallBridgeResult {
|
|
29
|
-
handled: boolean;
|
|
30
|
-
reason?: string;
|
|
31
|
-
/** User-facing text persisted in-thread by the bridge (success ack or failure notice). */
|
|
32
|
-
userFacingText?: string;
|
|
33
|
-
}
|
|
34
|
-
|
|
35
|
-
/**
|
|
36
|
-
* Attempt to route a user message to an active call — as an answer to
|
|
37
|
-
* a pending question (priority) or as a mid-call steering instruction.
|
|
38
|
-
*
|
|
39
|
-
* @param conversationId - The conversation the message belongs to.
|
|
40
|
-
* @param userText - The user's message text.
|
|
41
|
-
* @param _userMessageId - The persisted message ID (reserved for future use).
|
|
42
|
-
* @returns `{ handled: true }` if the message was consumed by the call system,
|
|
43
|
-
* `{ handled: false, reason }` otherwise.
|
|
44
|
-
*/
|
|
45
|
-
export async function tryRouteCallMessage(
|
|
46
|
-
conversationId: string,
|
|
47
|
-
userText: string,
|
|
48
|
-
_userMessageId?: string,
|
|
49
|
-
): Promise<CallBridgeResult> {
|
|
50
|
-
// 1. Find an active call for this conversation
|
|
51
|
-
const callSession = getActiveCallSessionForConversation(conversationId);
|
|
52
|
-
if (!callSession) {
|
|
53
|
-
return { handled: false, reason: 'no_active_call' };
|
|
54
|
-
}
|
|
55
|
-
|
|
56
|
-
// 2. Check for a pending question — answer path takes priority
|
|
57
|
-
const pendingQuestion = getPendingQuestion(callSession.id);
|
|
58
|
-
if (pendingQuestion) {
|
|
59
|
-
return handleAnswer(conversationId, callSession.id, pendingQuestion, userText);
|
|
60
|
-
}
|
|
61
|
-
|
|
62
|
-
// 3. No pending question — instruction path
|
|
63
|
-
return handleInstruction(conversationId, callSession.id, userText);
|
|
64
|
-
}
|
|
65
|
-
|
|
66
|
-
/** @deprecated Use `tryRouteCallMessage` instead. */
|
|
67
|
-
export const tryHandlePendingCallAnswer = tryRouteCallMessage;
|
|
68
|
-
|
|
69
|
-
// ── Answer path ─────────────────────────────────────────────────────
|
|
70
|
-
|
|
71
|
-
async function handleAnswer(
|
|
72
|
-
conversationId: string,
|
|
73
|
-
callSessionId: string,
|
|
74
|
-
pendingQuestion: { id: string; questionText: string },
|
|
75
|
-
userText: string,
|
|
76
|
-
): Promise<CallBridgeResult> {
|
|
77
|
-
// Empty text (e.g. attachment-only messages) should not be consumed as
|
|
78
|
-
// an answer — fall through to normal processing so attachments are handled.
|
|
79
|
-
if (!userText.trim()) {
|
|
80
|
-
return { handled: false, reason: 'empty_answer_text' };
|
|
81
|
-
}
|
|
82
|
-
|
|
83
|
-
const orchestrator = getCallOrchestrator(callSessionId);
|
|
84
|
-
if (!orchestrator) {
|
|
85
|
-
// The call may have ended between the question being asked and the
|
|
86
|
-
// user replying. Persist a follow-up message so the user knows.
|
|
87
|
-
const freshSession = getCallSession(callSessionId);
|
|
88
|
-
const ended = freshSession && (freshSession.status === 'completed' || freshSession.status === 'failed');
|
|
89
|
-
if (ended) {
|
|
90
|
-
conversationStore.addMessage(
|
|
91
|
-
conversationId,
|
|
92
|
-
'assistant',
|
|
93
|
-
JSON.stringify([{
|
|
94
|
-
type: 'text',
|
|
95
|
-
text: 'The call ended before your answer could be relayed to the caller.',
|
|
96
|
-
}]),
|
|
97
|
-
);
|
|
98
|
-
}
|
|
99
|
-
return { handled: false, reason: 'orchestrator_not_found' };
|
|
100
|
-
}
|
|
101
|
-
|
|
102
|
-
if (orchestrator.getState() !== 'waiting_on_user') {
|
|
103
|
-
return { handled: false, reason: 'orchestrator_not_waiting' };
|
|
104
|
-
}
|
|
105
|
-
|
|
106
|
-
const accepted = await orchestrator.handleUserAnswer(userText);
|
|
107
|
-
if (!accepted) {
|
|
108
|
-
return { handled: false, reason: 'orchestrator_rejected' };
|
|
109
|
-
}
|
|
110
|
-
|
|
111
|
-
answerPendingQuestion(pendingQuestion.id, userText);
|
|
112
|
-
recordCallEvent(callSessionId, 'user_answered', { answer: userText });
|
|
113
|
-
|
|
114
|
-
log.info(
|
|
115
|
-
{ conversationId, callSessionId, questionId: pendingQuestion.id },
|
|
116
|
-
'User reply routed as call answer via bridge',
|
|
117
|
-
);
|
|
118
|
-
|
|
119
|
-
return { handled: true };
|
|
120
|
-
}
|
|
121
|
-
|
|
122
|
-
// ── Instruction path ────────────────────────────────────────────────
|
|
123
|
-
|
|
124
|
-
async function handleInstruction(
|
|
125
|
-
conversationId: string,
|
|
126
|
-
callSessionId: string,
|
|
127
|
-
userText: string,
|
|
128
|
-
): Promise<CallBridgeResult> {
|
|
129
|
-
// Empty text (e.g. attachment-only messages) should not be relayed —
|
|
130
|
-
// fall through to normal processing so attachments are handled.
|
|
131
|
-
if (!userText.trim()) {
|
|
132
|
-
return { handled: false, reason: 'empty_instruction_text' };
|
|
133
|
-
}
|
|
134
|
-
|
|
135
|
-
const result = await relayInstruction({ callSessionId, instructionText: userText });
|
|
136
|
-
|
|
137
|
-
if (!result.ok) {
|
|
138
|
-
log.warn(
|
|
139
|
-
{ conversationId, callSessionId, error: result.error },
|
|
140
|
-
'Instruction relay failed via bridge',
|
|
141
|
-
);
|
|
142
|
-
|
|
143
|
-
const failureText = 'Failed to relay instruction to the active call.';
|
|
144
|
-
conversationStore.addMessage(
|
|
145
|
-
conversationId,
|
|
146
|
-
'assistant',
|
|
147
|
-
JSON.stringify([{ type: 'text', text: failureText }]),
|
|
148
|
-
);
|
|
149
|
-
|
|
150
|
-
// Consumed: caller should NOT fall through to the agent loop
|
|
151
|
-
return { handled: true, reason: 'instruction_relay_failed', userFacingText: failureText };
|
|
152
|
-
}
|
|
153
|
-
|
|
154
|
-
// Persist a concise acknowledgement so the user sees confirmation
|
|
155
|
-
const ackText = 'Instruction relayed to active call.';
|
|
156
|
-
conversationStore.addMessage(
|
|
157
|
-
conversationId,
|
|
158
|
-
'assistant',
|
|
159
|
-
JSON.stringify([{ type: 'text', text: ackText }]),
|
|
160
|
-
);
|
|
161
|
-
|
|
162
|
-
log.info(
|
|
163
|
-
{ conversationId, callSessionId },
|
|
164
|
-
'User message routed as call instruction via bridge',
|
|
165
|
-
);
|
|
166
|
-
|
|
167
|
-
return { handled: true, userFacingText: ackText };
|
|
168
|
-
}
|
|
@@ -1,199 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: "Google OAuth Setup"
|
|
3
|
-
description: "Create Google Cloud OAuth credentials for Gmail integration using browser automation"
|
|
4
|
-
user-invocable: true
|
|
5
|
-
includes: ["browser", "public-ingress"]
|
|
6
|
-
metadata: {"vellum": {"emoji": "\ud83d\udd11"}}
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
You are helping your user create Google Cloud OAuth credentials so the Gmail and Google Calendar integrations can connect. Walk through each step below using `browser_navigate`, `browser_snapshot`, `browser_screenshot`, `browser_click`, `browser_type`, and `browser_extract` tools.
|
|
10
|
-
|
|
11
|
-
**Tone:** Be friendly and reassuring throughout. Narrate what you're doing in plain language so the user always knows what's happening. After each step, briefly confirm what was accomplished before moving on.
|
|
12
|
-
|
|
13
|
-
## Prerequisites
|
|
14
|
-
|
|
15
|
-
Before starting, check that `ingress.publicBaseUrl` is configured (Settings > Public Ingress, or `INGRESS_PUBLIC_BASE_URL` env var). If it is not set, load and execute the **public-ingress** skill first (`skill_load` with `skill: "public-ingress"`) to set up an ngrok tunnel and persist the public URL. The OAuth redirect URI depends on this value.
|
|
16
|
-
|
|
17
|
-
## Before You Start
|
|
18
|
-
|
|
19
|
-
Tell the user:
|
|
20
|
-
- "I'll walk you through setting up Google Cloud so Vellum can connect to your Gmail and Google Calendar. The whole process takes a few minutes."
|
|
21
|
-
- "I'll be automating the Google Cloud Console in the browser — you'll be able to see everything I'm doing."
|
|
22
|
-
- "I'll ask for your approval before each major step, so nothing happens without your say-so."
|
|
23
|
-
- "No sensitive credentials will be shown in the conversation."
|
|
24
|
-
|
|
25
|
-
## Step 1: Navigate to Google Cloud Console
|
|
26
|
-
|
|
27
|
-
Tell the user: "First, let me open Google Cloud Console."
|
|
28
|
-
|
|
29
|
-
Use `browser_navigate` to go to `https://console.cloud.google.com/`.
|
|
30
|
-
|
|
31
|
-
Take a `browser_screenshot` to show the user what loaded, then take a `browser_snapshot` to check the page state:
|
|
32
|
-
- **If a sign-in page appears:** Tell the user "Please sign in to your Google account in the browser window. Let me know when you're done." Wait for their confirmation, then take another snapshot.
|
|
33
|
-
- **If a CAPTCHA appears:** Tell the user "There's a CAPTCHA to solve. Please complete it in the browser window and let me know." Wait, then retry.
|
|
34
|
-
- **If the console dashboard loads:** Tell the user "Google Cloud Console is loaded. Let's get started!" and continue to Step 2.
|
|
35
|
-
|
|
36
|
-
## Step 2: Create or Select a Project
|
|
37
|
-
|
|
38
|
-
**Ask for approval before proceeding.** Use `ui_show` with `surface_type: "confirmation"` and this message:
|
|
39
|
-
|
|
40
|
-
> **Create a Google Cloud Project**
|
|
41
|
-
>
|
|
42
|
-
> I'm about to create a new Google Cloud project called "Vellum Assistant". This is completely free and won't affect any of your existing projects. The project is just a container for the Gmail API credentials.
|
|
43
|
-
|
|
44
|
-
Wait for the user to approve. If they decline, explain that a project is required for OAuth credentials and offer to try again or cancel the setup.
|
|
45
|
-
|
|
46
|
-
Once approved, navigate to `https://console.cloud.google.com/projectcreate`.
|
|
47
|
-
|
|
48
|
-
Take a `browser_snapshot`. Fill in the project name form:
|
|
49
|
-
- Use `browser_type` to set the project name to "Vellum Assistant"
|
|
50
|
-
- Use `browser_click` to submit the "Create" button
|
|
51
|
-
|
|
52
|
-
Wait a few seconds, then take a `browser_screenshot` to show the user what happened, and a `browser_snapshot` to confirm the project was created. If the project already exists, that's fine — navigate to its dashboard.
|
|
53
|
-
|
|
54
|
-
Tell the user: "Project created! Now let's enable the Gmail API."
|
|
55
|
-
|
|
56
|
-
Note the project ID from the URL or page content for subsequent steps.
|
|
57
|
-
|
|
58
|
-
## Step 3: Enable the Gmail and Calendar APIs
|
|
59
|
-
|
|
60
|
-
**Ask for approval before proceeding.** Use `ui_show` with `surface_type: "confirmation"` and this message:
|
|
61
|
-
|
|
62
|
-
> **Enable the Gmail and Calendar APIs**
|
|
63
|
-
>
|
|
64
|
-
> I'm about to enable the Gmail API and Google Calendar API in your Google Cloud project. This allows Vellum to access your email and calendar — but only after you explicitly authorize it in a later step. Enabling the APIs alone doesn't grant any access.
|
|
65
|
-
|
|
66
|
-
Wait for the user to approve. If they decline, explain that the APIs are required for email and calendar integration and offer to try again or cancel.
|
|
67
|
-
|
|
68
|
-
Once approved, navigate to `https://console.cloud.google.com/apis/library/gmail.googleapis.com?project=PROJECT_ID` (substitute the actual project ID).
|
|
69
|
-
|
|
70
|
-
Take a `browser_snapshot`:
|
|
71
|
-
- **If the API is already enabled:** You'll see "API enabled" or a "Manage" button. Tell the user "Gmail API is already enabled — great!" and continue to enable Calendar API.
|
|
72
|
-
- **If not enabled:** Use `browser_click` on the "Enable" button.
|
|
73
|
-
|
|
74
|
-
Wait a moment, then take a `browser_screenshot` to show the result and a `browser_snapshot` to confirm it shows as enabled.
|
|
75
|
-
|
|
76
|
-
Now navigate to `https://console.cloud.google.com/apis/library/calendar-json.googleapis.com?project=PROJECT_ID` to enable the Google Calendar API.
|
|
77
|
-
|
|
78
|
-
Take a `browser_snapshot`:
|
|
79
|
-
- **If the API is already enabled:** Tell the user "Google Calendar API is already enabled — great!" and skip to Step 4.
|
|
80
|
-
- **If not enabled:** Use `browser_click` on the "Enable" button.
|
|
81
|
-
|
|
82
|
-
Wait a moment, then take a `browser_screenshot` to show the result.
|
|
83
|
-
|
|
84
|
-
Tell the user: "Gmail and Calendar APIs are enabled! Next, we need to set up an OAuth consent screen."
|
|
85
|
-
|
|
86
|
-
## Step 4: Configure the OAuth Consent Screen
|
|
87
|
-
|
|
88
|
-
**Ask for approval before proceeding.** Use `ui_show` with `surface_type: "confirmation"` and this message:
|
|
89
|
-
|
|
90
|
-
> **Configure OAuth Consent Screen**
|
|
91
|
-
>
|
|
92
|
-
> I'm about to set up an OAuth consent screen for your project. This is the page Google shows when you authorize an app. I'll configure it with your email and the Gmail permissions Vellum needs (read, modify, and send emails). The app will start in "testing" mode — only you will be able to authorize it.
|
|
93
|
-
|
|
94
|
-
Wait for the user to approve. If they decline, explain that the consent screen is required for the OAuth flow and offer to try again or cancel.
|
|
95
|
-
|
|
96
|
-
Once approved, navigate to `https://console.cloud.google.com/apis/credentials/consent?project=PROJECT_ID`.
|
|
97
|
-
|
|
98
|
-
Take a `browser_snapshot` to check the current state:
|
|
99
|
-
- **If consent screen is already configured:** Tell the user "Consent screen is already set up — skipping ahead!" and go to Step 5.
|
|
100
|
-
- **If a user type selection appears:** Select "External" and click "Create".
|
|
101
|
-
|
|
102
|
-
Fill in the required fields on the consent screen form:
|
|
103
|
-
1. **App name:** "Vellum Assistant"
|
|
104
|
-
2. **User support email:** Select the user's email from the dropdown
|
|
105
|
-
3. **Developer contact email:** Type the user's email address
|
|
106
|
-
4. Leave other fields as defaults
|
|
107
|
-
|
|
108
|
-
Use `browser_click` to proceed through each page of the wizard:
|
|
109
|
-
- App information page: Fill fields, click "Save and Continue"
|
|
110
|
-
- Scopes page: Click "Add or Remove Scopes", search for and select:
|
|
111
|
-
- `https://www.googleapis.com/auth/gmail.readonly`
|
|
112
|
-
- `https://www.googleapis.com/auth/gmail.modify`
|
|
113
|
-
- `https://www.googleapis.com/auth/gmail.send`
|
|
114
|
-
- `https://www.googleapis.com/auth/calendar.readonly`
|
|
115
|
-
- `https://www.googleapis.com/auth/calendar.events`
|
|
116
|
-
- `https://www.googleapis.com/auth/userinfo.email`
|
|
117
|
-
- Click "Update" then "Save and Continue"
|
|
118
|
-
- Test users page: Add the user's email as a test user, click "Save and Continue"
|
|
119
|
-
- Summary page: Click "Back to Dashboard"
|
|
120
|
-
|
|
121
|
-
Take a `browser_screenshot` after completing the wizard.
|
|
122
|
-
|
|
123
|
-
Tell the user: "Consent screen is configured! Almost there — just need to create the credentials."
|
|
124
|
-
|
|
125
|
-
## Step 5: Create OAuth Credentials
|
|
126
|
-
|
|
127
|
-
**Ask for approval before proceeding.** Use `ui_show` with `surface_type: "confirmation"` and this message:
|
|
128
|
-
|
|
129
|
-
> **Create OAuth Credentials**
|
|
130
|
-
>
|
|
131
|
-
> I'm about to create OAuth Web Application credentials for Vellum Assistant. This generates a client ID that Vellum uses to initiate the authorization flow. The redirect URI will point to the gateway's OAuth callback endpoint.
|
|
132
|
-
|
|
133
|
-
Wait for the user to approve. If they decline, explain that credentials are the final step needed and offer to try again or cancel.
|
|
134
|
-
|
|
135
|
-
Once approved, navigate to `https://console.cloud.google.com/apis/credentials?project=PROJECT_ID`.
|
|
136
|
-
|
|
137
|
-
Use `browser_click` on "+ Create Credentials" at the top, then select "OAuth client ID" from the dropdown.
|
|
138
|
-
|
|
139
|
-
Take a `browser_snapshot` and fill in:
|
|
140
|
-
1. **Application type:** Select "Web application" from the dropdown
|
|
141
|
-
2. **Name:** "Vellum Assistant"
|
|
142
|
-
3. **Authorized redirect URIs:** Click "Add URI" and enter `${ingress.publicBaseUrl}/webhooks/oauth/callback` (e.g. `https://abc123.ngrok-free.app/webhooks/oauth/callback`). Read the `ingress.publicBaseUrl` value from the assistant's workspace config (Settings > Public Ingress) or the `INGRESS_PUBLIC_BASE_URL` environment variable.
|
|
143
|
-
|
|
144
|
-
Use `browser_click` on the "Create" button.
|
|
145
|
-
|
|
146
|
-
## Step 6: Extract and Store the Client ID
|
|
147
|
-
|
|
148
|
-
After creation, a dialog should appear showing the client ID and client secret.
|
|
149
|
-
|
|
150
|
-
Use `browser_snapshot` or `browser_extract` to read the **Client ID** value from the dialog. The client ID looks like `NUMBERS-CHARS.apps.googleusercontent.com`.
|
|
151
|
-
|
|
152
|
-
**Important:** You only need the Client ID, not the client secret (PKCE flow is used).
|
|
153
|
-
|
|
154
|
-
Tell the user: "Credentials created! Now let's connect your Gmail account using the client ID."
|
|
155
|
-
|
|
156
|
-
## Step 7: Connect Gmail
|
|
157
|
-
|
|
158
|
-
Tell the user: "Opening Google sign-in so you can authorize Vellum to access your Gmail. You'll see a Google consent page — just click 'Allow'."
|
|
159
|
-
|
|
160
|
-
Use the `credential_store` tool to connect Gmail via OAuth2:
|
|
161
|
-
|
|
162
|
-
```
|
|
163
|
-
action: "oauth2_connect"
|
|
164
|
-
service: "integration:gmail"
|
|
165
|
-
client_id: "<the extracted client ID>"
|
|
166
|
-
auth_url: "https://accounts.google.com/o/oauth2/v2/auth"
|
|
167
|
-
token_url: "https://oauth2.googleapis.com/token"
|
|
168
|
-
scopes: ["https://www.googleapis.com/auth/gmail.readonly", "https://www.googleapis.com/auth/gmail.modify", "https://www.googleapis.com/auth/gmail.send", "https://www.googleapis.com/auth/calendar.readonly", "https://www.googleapis.com/auth/calendar.events", "https://www.googleapis.com/auth/userinfo.email"]
|
|
169
|
-
userinfo_url: "https://www.googleapis.com/oauth2/v2/userinfo"
|
|
170
|
-
extra_params: {"access_type": "offline", "prompt": "consent"}
|
|
171
|
-
```
|
|
172
|
-
|
|
173
|
-
This will open the Google authorization page in the user's browser. Wait for the flow to complete.
|
|
174
|
-
|
|
175
|
-
**If the user sees a "This app isn't verified" warning:** Tell them this is normal for apps in testing mode. They should click "Advanced" → "Go to Vellum Assistant (unsafe)" to proceed. This warning appears because the app hasn't gone through Google's verification process, which is only needed for apps used by many people.
|
|
176
|
-
|
|
177
|
-
## Step 8: Celebrate!
|
|
178
|
-
|
|
179
|
-
Once connected, tell the user:
|
|
180
|
-
|
|
181
|
-
"**Gmail and Calendar are connected!** You're all set. You can now read, search, and send emails, plus view and manage your calendar through Vellum. Try asking me to check your inbox or show your upcoming events!"
|
|
182
|
-
|
|
183
|
-
Summarize what was accomplished:
|
|
184
|
-
- Created a Google Cloud project (or used an existing one)
|
|
185
|
-
- Enabled the Gmail API and Google Calendar API
|
|
186
|
-
- Configured the OAuth consent screen with appropriate scopes (including calendar)
|
|
187
|
-
- Created OAuth Web Application credentials with gateway callback redirect URI
|
|
188
|
-
- Connected your Gmail and Google Calendar accounts
|
|
189
|
-
|
|
190
|
-
## Error Handling
|
|
191
|
-
|
|
192
|
-
- **Page load failures:** Retry navigation once. If it still fails, tell the user and ask them to check their internet connection.
|
|
193
|
-
- **Permission errors in GCP:** The user may need billing enabled or organization-level permissions. Explain what's needed clearly and ask them to resolve it.
|
|
194
|
-
- **Consent screen already configured with different settings:** Don't overwrite; skip to credential creation and tell the user you're using their existing configuration.
|
|
195
|
-
- **Element not found for click/type:** Take a fresh `browser_snapshot` to re-assess the page layout. GCP UI may have changed; adapt your selectors. Tell the user what you're looking for if you get stuck.
|
|
196
|
-
- **User declines an approval gate:** Don't push back aggressively. Explain briefly why the step matters, offer to try again, or offer to cancel the whole setup gracefully. Never proceed without approval.
|
|
197
|
-
- **OAuth flow timeout or failure:** Tell the user what happened and offer to retry the connect step. The client ID is already stored, so they can also connect later from Settings.
|
|
198
|
-
- **"This app isn't verified" warning:** Guide the user through clicking "Advanced" → "Go to Vellum Assistant (unsafe)". Reassure them this is expected for personal-use OAuth apps.
|
|
199
|
-
- **Any unexpected state:** Take a `browser_screenshot` and `browser_snapshot`, describe what you see, and ask the user for guidance rather than guessing.
|