@vellumai/assistant 0.11.3 → 0.11.4-dev.202608182307.5f98f27
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +9 -3
- package/ARCHITECTURE.md +32 -10
- package/docs/architecture/memory.md +43 -5
- package/docs/architecture/security.md +96 -152
- package/docs/architecture/turn-actor.md +70 -0
- package/docs/browser-use-architecture-phase2.md +128 -56
- package/docs/flux-turn-detection-spike.md +248 -0
- package/docs/guardian-request-flow.md +35 -0
- package/docs/stt-provider-onboarding.md +3 -1
- package/knip.json +3 -0
- package/node_modules/@vellumai/ces-client/node_modules/@vellumai/service-contracts/package.json +1 -0
- package/node_modules/@vellumai/ces-client/node_modules/@vellumai/service-contracts/src/__tests__/url-normalization.test.ts +135 -0
- package/node_modules/@vellumai/ces-client/node_modules/@vellumai/service-contracts/src/channels.ts +11 -0
- package/node_modules/@vellumai/ces-client/node_modules/@vellumai/service-contracts/src/index.ts +1 -0
- package/node_modules/@vellumai/ces-client/node_modules/@vellumai/service-contracts/src/remote-web-pairing.ts +60 -0
- package/node_modules/@vellumai/ces-client/node_modules/@vellumai/service-contracts/src/url-normalization.ts +107 -0
- package/node_modules/@vellumai/gateway-client/node_modules/@vellumai/service-contracts/package.json +1 -0
- package/node_modules/@vellumai/gateway-client/node_modules/@vellumai/service-contracts/src/__tests__/url-normalization.test.ts +135 -0
- package/node_modules/@vellumai/gateway-client/node_modules/@vellumai/service-contracts/src/channels.ts +11 -0
- package/node_modules/@vellumai/gateway-client/node_modules/@vellumai/service-contracts/src/index.ts +1 -0
- package/node_modules/@vellumai/gateway-client/node_modules/@vellumai/service-contracts/src/remote-web-pairing.ts +60 -0
- package/node_modules/@vellumai/gateway-client/node_modules/@vellumai/service-contracts/src/url-normalization.ts +107 -0
- package/node_modules/@vellumai/gateway-client/src/admission-policy-contract.ts +34 -0
- package/node_modules/@vellumai/gateway-client/src/gateway-ipc-contracts.ts +166 -1
- package/node_modules/@vellumai/gateway-client/src/index.ts +2 -0
- package/node_modules/@vellumai/service-contracts/package.json +1 -0
- package/node_modules/@vellumai/service-contracts/src/__tests__/url-normalization.test.ts +135 -0
- package/node_modules/@vellumai/service-contracts/src/channels.ts +11 -0
- package/node_modules/@vellumai/service-contracts/src/index.ts +1 -0
- package/node_modules/@vellumai/service-contracts/src/remote-web-pairing.ts +60 -0
- package/node_modules/@vellumai/service-contracts/src/url-normalization.ts +107 -0
- package/openapi.yaml +289 -113
- package/package.json +1 -1
- package/scripts/voice-ttft-spike.ts +3 -3
- package/scripts/write-plugin-api-shim.ts +10 -0
- package/src/__tests__/app-compiler.test.ts +38 -3
- package/src/__tests__/app-control-flow.test.ts +1 -0
- package/src/__tests__/approval-routes-http.test.ts +2 -2
- package/src/__tests__/assistant-feature-flag-guard.test.ts +25 -3
- package/src/__tests__/attachments-store.test.ts +21 -12
- package/src/__tests__/byok-default-profile-ensure.test.ts +17 -0
- package/src/__tests__/call-setup-flow-name-capture.test.ts +0 -1
- package/src/__tests__/call-site-routing-provider.test.ts +1 -1
- package/src/__tests__/channel-availability-routes.test.ts +14 -1
- package/src/__tests__/channel-capabilities-dedupe.test.ts +214 -0
- package/src/__tests__/channel-delivery-store.test.ts +14 -14
- package/src/__tests__/channel-inbound-disk-pressure.test.ts +4 -4
- package/src/__tests__/channel-setup-panel-ack.test.ts +1 -1
- package/src/__tests__/checker.test.ts +20 -314
- package/src/__tests__/cli-memory-v2-reembed-skills.test.ts +6 -2
- package/src/__tests__/compaction-events.test.ts +8 -10
- package/src/__tests__/config-loader-backfill.test.ts +3 -3
- package/src/__tests__/config-schema.test.ts +25 -10
- package/src/__tests__/context-search-memory-v2-source.test.ts +0 -1
- package/src/__tests__/conversation-agent-loop-inference-profile.test.ts +8 -11
- package/src/__tests__/conversation-agent-loop-overflow.test.ts +8 -11
- package/src/__tests__/conversation-agent-loop.test.ts +32 -21
- package/src/__tests__/conversation-attachments.test.ts +0 -1
- package/src/__tests__/conversation-attention-store.test.ts +63 -0
- package/src/__tests__/conversation-confirmation-signals.test.ts +112 -0
- package/src/__tests__/conversation-delete-schedule-cleanup.test.ts +0 -4
- package/src/__tests__/conversation-fork-crud.test.ts +69 -0
- package/src/__tests__/conversation-fork-referential.test.ts +67 -0
- package/src/__tests__/conversation-fork-retrospective.test.ts +24 -0
- package/src/__tests__/conversation-load-history-repair.test.ts +209 -0
- package/src/__tests__/conversation-notifiers-provenance.test.ts +59 -0
- package/src/__tests__/conversation-queue.test.ts +154 -4
- package/src/__tests__/conversation-routes-disk-view.test.ts +1 -1
- package/src/__tests__/conversation-routes-enabled-plugins.test.ts +1 -1
- package/src/__tests__/conversation-routes-guardian-reply.test.ts +9 -9
- package/src/__tests__/conversation-routes-hidden-queue.test.ts +1 -1
- package/src/__tests__/conversation-routes-slash-commands.test.ts +1 -1
- package/src/__tests__/conversation-runtime-assembly.test.ts +187 -102
- package/src/__tests__/conversation-runtime-workspace.test.ts +14 -10
- package/src/__tests__/conversation-slash-queue.test.ts +3 -0
- package/src/__tests__/conversation-surfaces-action-delivery.test.ts +1 -0
- package/src/__tests__/conversation-surfaces-activation-emit.test.ts +1 -0
- package/src/__tests__/conversation-surfaces-app-control.test.ts +1 -0
- package/src/__tests__/conversation-surfaces-app-open.test.ts +1 -1
- package/src/__tests__/conversation-surfaces-data-persist.test.ts +1 -1
- package/src/__tests__/conversation-surfaces-history-restored-completion.test.ts +21 -14
- package/src/__tests__/conversation-surfaces-queued-emit.test.ts +1 -0
- package/src/__tests__/conversation-surfaces-standalone-payloads.test.ts +1 -0
- package/src/__tests__/conversation-surfaces-standalone.test.ts +1 -0
- package/src/__tests__/conversation-surfaces-state-update.test.ts +1 -1
- package/src/__tests__/conversation-surfaces-table-action.test.ts +1 -1
- package/src/__tests__/conversation-surfaces-task-progress.test.ts +1 -1
- package/src/__tests__/conversation-tool-setup-app-refresh.test.ts +1 -1
- package/src/__tests__/conversation-tool-setup-attribution.test.ts +1 -1
- package/src/__tests__/credential-prompt-route.test.ts +7 -10
- package/src/__tests__/credential-security-invariants.test.ts +1 -0
- package/src/__tests__/cu-unified-flow.test.ts +1 -0
- package/src/__tests__/custom-profile-ensure.test.ts +5 -1
- package/src/__tests__/default-plugin-names-guard.test.ts +12 -1
- package/src/__tests__/discord-access-request-privacy.test.ts +5 -1
- package/src/__tests__/discord-requester-notice-privacy.test.ts +3 -3
- package/src/__tests__/document-append-idempotency.test.ts +233 -0
- package/src/__tests__/document-sync-tags.test.ts +0 -75
- package/src/__tests__/dynamic-skill-background-guard.test.ts +0 -2
- package/src/__tests__/edit-propagation.test.ts +107 -11
- package/src/__tests__/file-ops-service.test.ts +163 -30
- package/src/__tests__/filesystem-tools.test.ts +23 -24
- package/src/__tests__/gateway-only-guard.test.ts +2 -5
- package/src/__tests__/helpers/gateway-classify-mock.ts +27 -6
- package/src/__tests__/helpers/mock-actor-context.ts +49 -0
- package/src/__tests__/helpers/mock-conversation.ts +13 -1
- package/src/__tests__/host-file-read-tool.test.ts +16 -19
- package/src/__tests__/http-user-message-parity.test.ts +1 -1
- package/src/__tests__/init-feature-flag-overrides.test.ts +49 -0
- package/src/__tests__/injector-chain.test.ts +63 -41
- package/src/__tests__/injector-disk-pressure.test.ts +11 -23
- package/src/__tests__/inline-skill-load-permissions.test.ts +47 -36
- package/src/__tests__/llm-context-resolution.test.ts +73 -1
- package/src/__tests__/llm-schema.test.ts +5 -2
- package/src/__tests__/managed-skill-lifecycle.test.ts +7 -0
- package/src/__tests__/mcp-list-plugin-servers.test.ts +250 -0
- package/src/__tests__/media-generate-image.test.ts +131 -21
- package/src/__tests__/memory-retrieval-hook.test.ts +100 -7
- package/src/__tests__/messages-read-boundary-guard.test.ts +134 -0
- package/src/__tests__/mtime-cache.test.ts +3 -1
- package/src/__tests__/non-member-access-request.test.ts +0 -20
- package/src/__tests__/outbound-slack-persistence.test.ts +40 -1
- package/src/__tests__/platform-bash-auto-approve.test.ts +0 -4
- package/src/__tests__/plugin-api-store-credential.test.ts +261 -0
- package/src/__tests__/plugin-api-webhook-url.test.ts +10 -7
- package/src/__tests__/plugin-disabled-state.test.ts +2 -0
- package/src/__tests__/plugin-effective-enabled-set.test.ts +55 -3
- package/src/__tests__/plugin-execution-context.test.ts +73 -0
- package/src/__tests__/plugin-import-boundary-guard.test.ts +5 -0
- package/src/__tests__/plugin-import-boundary-reverse-guard.test.ts +7 -3
- package/src/__tests__/plugin-secret-pattern-contribution.test.ts +1 -1
- package/src/__tests__/post-compaction-reinjection-idempotency.test.ts +14 -7
- package/src/__tests__/provider-commit-message-generator.test.ts +20 -0
- package/src/__tests__/proxy-approval-callback.test.ts +1 -0
- package/src/__tests__/qdrant-manager.test.ts +14 -1
- package/src/__tests__/reaction-persistence.test.ts +200 -7
- package/src/__tests__/require-fresh-approval.test.ts +0 -4
- package/src/__tests__/risk-classification-boundary-guard.test.ts +115 -0
- package/src/__tests__/run-conversation-turn-persistence.test.ts +460 -105
- package/src/__tests__/run-due-schedules.test.ts +21 -0
- package/src/__tests__/scaffold-managed-skill-tool.test.ts +187 -18
- package/src/__tests__/schedule-routes.test.ts +23 -0
- package/src/__tests__/schedule-store.test.ts +17 -0
- package/src/__tests__/scoped-approval-grants.test.ts +11 -6
- package/src/__tests__/secret-ingress-channel.test.ts +0 -1
- package/src/__tests__/secret-ingress-http.test.ts +1 -1
- package/src/__tests__/send-endpoint-busy.test.ts +3 -3
- package/src/__tests__/skills.test.ts +32 -0
- package/src/__tests__/slack-edit-ordering-characterization.test.ts +0 -1
- package/src/__tests__/starter-task-flow.test.ts +5 -4
- package/src/__tests__/subagent-call-site-routing.test.ts +31 -19
- package/src/__tests__/subagent-fork-prompt-role.test.ts +1 -1
- package/src/__tests__/subagent-spawn-and-await.test.ts +18 -17
- package/src/__tests__/subagent-tool-gate-mode.test.ts +1 -1
- package/src/__tests__/subagent-tools.test.ts +81 -101
- package/src/__tests__/surface-completion-in-flight-snapshot.test.ts +1 -0
- package/src/__tests__/tool-execution-pipeline.benchmark.test.ts +1 -1
- package/src/__tests__/tool-executor-lifecycle-events.test.ts +132 -5
- package/src/__tests__/tool-executor.test.ts +68 -39
- package/src/__tests__/turn-events-store.test.ts +43 -0
- package/src/__tests__/ui-choice-copy-surfaces.test.ts +1 -1
- package/src/__tests__/ui-shape-teaching.test.ts +33 -0
- package/src/__tests__/ui-visual-surface.test.ts +1 -1
- package/src/__tests__/ui-voice-picker-surface.test.ts +128 -0
- package/src/__tests__/ui-work-result-surface.test.ts +1 -1
- package/src/__tests__/user-plugin-loader.test.ts +3 -1
- package/src/__tests__/verification-control-plane-policy.test.ts +0 -2
- package/src/__tests__/visible-app-context.test.ts +16 -9
- package/src/__tests__/voice-config-update.test.ts +40 -0
- package/src/__tests__/voice-scoped-grant-consumer.test.ts +5 -3
- package/src/__tests__/voice-session-bridge.test.ts +85 -29
- package/src/__tests__/worker-entrypoint-guards.test.ts +54 -0
- package/src/__tests__/worker-plugin-surface.test.ts +77 -0
- package/src/__tests__/workspace-migration-142-consolidate-voice-front-door.test.ts +158 -0
- package/src/__tests__/workspace-migration-143-repair-deprecated-codex-model-id.test.ts +133 -0
- package/src/__tests__/workspace-migration-144-convert-stranded-subscription-openai-profiles.test.ts +316 -0
- package/src/__tests__/workspace-migration-145-collapse-profile-bindings-to-entries.test.ts +325 -0
- package/src/__tests__/workspace-migration-146-repair-retired-fireworks-deepseek-flash-model-id.test.ts +235 -0
- package/src/acp/__tests__/acp-claude-oauth.test.ts +10 -2
- package/src/acp/__tests__/auth-required.test.ts +161 -0
- package/src/acp/acp-claude-oauth.ts +19 -2
- package/src/acp/agent-process.test.ts +100 -0
- package/src/acp/agent-process.ts +29 -26
- package/src/acp/auth-required.ts +102 -0
- package/src/acp/session-manager.test.ts +119 -0
- package/src/acp/session-manager.ts +76 -3
- package/src/api/events/acp-auth-required.ts +55 -0
- package/src/api/events/host-file.ts +2 -2
- package/src/api/index.ts +7 -0
- package/src/api/surfaces.ts +12 -3
- package/src/apps/app-store.ts +3 -0
- package/src/bundler/package-resolver.ts +2 -30
- package/src/calls/__tests__/voice-session-bridge.test.ts +194 -11
- package/src/calls/__tests__/voice-triage-escalate.test.ts +102 -0
- package/src/calls/call-controller.ts +19 -3
- package/src/calls/call-setup-flow.ts +0 -1
- package/src/calls/media-stream-stt-session.ts +15 -0
- package/src/calls/voice-session-bridge.ts +115 -41
- package/src/calls/voice-triage-escalate.ts +105 -2
- package/src/channels/__tests__/plugin-channel-declarations.test.ts +161 -0
- package/src/channels/config.ts +13 -0
- package/src/channels/plugin-channel-declarations.ts +108 -0
- package/src/channels/types.ts +30 -0
- package/src/cli/AGENTS.md +5 -2
- package/src/cli/bundled-modules.ts +29 -0
- package/src/cli/commands/credentials.help.ts +2 -2
- package/src/cli/commands/db/repair.ts +4 -8
- package/src/cli/commands/domain.ts +6 -3
- package/src/cli/commands/email.ts +6 -3
- package/src/cli/commands/inference-providers.ts +1 -1
- package/src/cli/commands/keys.ts +8 -3
- package/src/cli/commands/mcp.help.ts +13 -4
- package/src/cli/commands/mcp.ts +9 -0
- package/src/cli/commands/memory/__tests__/memory-v2.test.ts +57 -7
- package/src/cli/commands/memory/__tests__/memory-v3.test.ts +128 -5
- package/src/cli/commands/memory/index.help.ts +84 -28
- package/src/cli/commands/memory/index.ts +2 -0
- package/src/cli/commands/memory/memory-v2.ts +58 -54
- package/src/cli/commands/memory/memory-v3.ts +64 -0
- package/src/cli/commands/memory/memory-validate.ts +18 -0
- package/src/cli/commands/plugins.ts +85 -36
- package/src/cli/commands/schedules.ts +35 -1
- package/src/cli/commands/stt.help.ts +27 -2
- package/src/cli/commands/trust.ts +3 -14
- package/src/cli/lib/__tests__/upgrade-plugin.test.ts +39 -0
- package/src/cli/lib/bundled-marketplace.json +14 -1
- package/src/cli/lib/upgrade-plugin.ts +42 -0
- package/src/config/__tests__/balanced-model-experiment.test.ts +278 -0
- package/src/config/__tests__/default-profile-catalog.test.ts +34 -2
- package/src/config/__tests__/default-provider.test.ts +6 -1
- package/src/config/__tests__/profile-materialization.test.ts +75 -19
- package/src/config/assistant-feature-flags.ts +36 -15
- package/src/config/balanced-model-experiment.ts +35 -0
- package/src/config/bundled-skills/acp/SKILL.md +6 -7
- package/src/config/bundled-skills/document-editor/SKILL.md +2 -2
- package/src/config/bundled-skills/document-editor/TOOLS.json +2 -2
- package/src/config/bundled-skills/image-studio/SKILL.md +5 -4
- package/src/config/bundled-skills/image-studio/TOOLS.json +1 -1
- package/src/config/bundled-skills/image-studio/tools/media-generate-image.ts +101 -0
- package/src/config/bundled-skills/media-processing/services/preprocess.ts +14 -4
- package/src/config/bundled-skills/settings/TOOLS.json +3 -3
- package/src/config/bundled-skills/settings/tools/navigate-settings-tab.test.ts +65 -0
- package/src/config/bundled-skills/settings/tools/navigate-settings-tab.ts +7 -1
- package/src/config/bundled-skills/settings/tools/shared.ts +16 -0
- package/src/config/bundled-skills/settings/tools/voice-config-update.ts +19 -1
- package/src/config/bundled-skills/skill-management/TOOLS.json +9 -3
- package/src/config/bundled-skills/subagent/SKILL.md +17 -12
- package/src/config/bundled-skills/subagent/TOOLS.json +4 -4
- package/src/config/bundled-skills/transcribe/tools/transcribe-media.test.ts +22 -1
- package/src/config/bundled-skills/transcribe/tools/transcribe-media.ts +9 -2
- package/src/config/call-site-defaults.ts +11 -5
- package/src/config/default-profile-catalog.ts +179 -16
- package/src/config/default-profile-names.ts +4 -1
- package/src/config/default-provider-resolution.ts +4 -0
- package/src/config/feature-flag-registry.json +12 -12
- package/src/config/llm-context-resolution.ts +11 -3
- package/src/config/llm-resolver.ts +28 -1
- package/src/config/profile-materialization.ts +70 -22
- package/src/config/schemas/__tests__/live-voice.test.ts +107 -4
- package/src/config/schemas/call-site-catalog.ts +4 -4
- package/src/config/schemas/live-voice.ts +57 -23
- package/src/config/schemas/llm.ts +59 -32
- package/src/config/schemas/mcp.ts +23 -0
- package/src/config/schemas/plugin-updates.ts +6 -2
- package/src/config/schemas/stt.ts +1 -0
- package/src/config/skills.ts +9 -2
- package/src/context/outbound-sanitize.ts +96 -1
- package/src/daemon/__tests__/conversation-surfaces-launch.test.ts +1 -1
- package/src/daemon/__tests__/plugin-mcp-reconcile.test.ts +82 -0
- package/src/daemon/conversation-agent-loop-handlers.ts +15 -10
- package/src/daemon/conversation-agent-loop.ts +31 -19
- package/src/daemon/conversation-messaging.ts +5 -1
- package/src/daemon/conversation-notifiers.ts +20 -10
- package/src/daemon/conversation-process.ts +9 -6
- package/src/daemon/conversation-runtime-assembly.ts +12 -6
- package/src/daemon/conversation-store.ts +14 -4
- package/src/daemon/conversation-surfaces.ts +54 -18
- package/src/daemon/conversation-tool-setup.ts +3 -6
- package/src/daemon/conversation.ts +152 -51
- package/src/daemon/doordash-steps.ts +2 -2
- package/src/daemon/lifecycle.ts +14 -1
- package/src/daemon/mcp-reload-service.ts +36 -6
- package/src/daemon/process-message.ts +13 -26
- package/src/daemon/providers-setup.ts +6 -3
- package/src/daemon/trust-context-types.ts +29 -0
- package/src/daemon/wake-conversation-ops.ts +3 -2
- package/src/daemon/windows-compiled-entry.ts +4 -0
- package/src/documents/document-store.ts +143 -240
- package/src/hooks/hook-loader.ts +3 -3
- package/src/hooks/registry.ts +103 -45
- package/src/hooks/types.ts +5 -0
- package/src/inbound/__tests__/oauth-callback-url.test.ts +83 -0
- package/src/inbound/oauth-callback-url.ts +61 -0
- package/src/ipc/gateway-client.test.ts +1 -1
- package/src/ipc/gateway-client.ts +14 -21
- package/src/ipc/gateway-flag-listener.ts +17 -3
- package/src/live-voice/__tests__/live-voice-agent-turn.test.ts +1 -104
- package/src/live-voice/__tests__/live-voice-events.test.ts +7 -8
- package/src/live-voice/__tests__/live-voice-flux-turn-end.test.ts +1050 -0
- package/src/live-voice/__tests__/live-voice-metrics.test.ts +115 -8
- package/src/live-voice/__tests__/live-voice-photo.test.ts +100 -0
- package/src/live-voice/__tests__/live-voice-progress.test.ts +60 -194
- package/src/live-voice/__tests__/live-voice-stt.test.ts +14 -0
- package/src/live-voice/__tests__/live-voice-triage-escalate.test.ts +29 -0
- package/src/live-voice/__tests__/live-voice-tts-session.test.ts +0 -483
- package/src/live-voice/__tests__/live-voice-vad.test.ts +0 -16
- package/src/live-voice/__tests__/progress-narration.test.ts +214 -0
- package/src/live-voice/live-voice-archive.ts +2 -0
- package/src/live-voice/live-voice-manager.ts +16 -3
- package/src/live-voice/live-voice-metrics.ts +57 -32
- package/src/live-voice/live-voice-photo.ts +1 -2
- package/src/live-voice/live-voice-session.ts +598 -323
- package/src/live-voice/progress-narration.ts +277 -0
- package/src/live-voice/protocol.ts +21 -1
- package/src/live-voice/windows-compiled-live-voice.ts +4 -0
- package/src/mcp/__tests__/effective-config.test.ts +238 -0
- package/src/mcp/__tests__/mcp-auth-orchestrator.test.ts +0 -1
- package/src/mcp/__tests__/mcp-oauth-client-registration.test.ts +200 -0
- package/src/mcp/__tests__/mcp-oauth-provider.test.ts +9 -9
- package/src/mcp/__tests__/plugin-server-credential-isolation.test.ts +95 -0
- package/src/mcp/client.ts +16 -11
- package/src/mcp/effective-config.ts +113 -0
- package/src/mcp/manager.ts +11 -6
- package/src/mcp/mcp-auth-orchestrator.ts +13 -22
- package/src/mcp/mcp-oauth-provider.ts +205 -240
- package/src/monitoring/__tests__/plugin-auto-update.test.ts +166 -3
- package/src/monitoring/control.ts +1 -0
- package/src/monitoring/db-integrity-sample.ts +4 -5
- package/src/monitoring/plugin-auto-update.ts +128 -24
- package/src/notifications/AGENTS.md +2 -0
- package/src/notifications/approval-card-data.ts +33 -0
- package/src/notifications/signal.ts +1 -0
- package/src/permissions/AGENTS.md +16 -0
- package/src/permissions/checker.test.ts +75 -122
- package/src/permissions/checker.ts +44 -560
- package/src/permissions/confirmation-guardian-request.test.ts +15 -11
- package/src/permissions/confirmation-guardian-request.ts +2 -2
- package/src/permissions/prompter.ts +1 -5
- package/src/permissions/question-guardian-request.test.ts +14 -6
- package/src/permissions/question-guardian-request.ts +1 -2
- package/src/persistence/attachments-store.ts +8 -1
- package/src/persistence/bookmark-crud.ts +3 -7
- package/src/persistence/conversation-attention-store.ts +16 -45
- package/src/persistence/conversation-crud.ts +33 -4
- package/src/persistence/conversation-lineage.ts +9 -0
- package/src/persistence/conversation-queries.ts +182 -52
- package/src/persistence/delivery-crud.ts +150 -36
- package/src/persistence/embeddings/qdrant-manager.ts +84 -49
- package/src/persistence/external-conversation-store.ts +32 -4
- package/src/persistence/llm-request-log-store.ts +4 -10
- package/src/persistence/llm-usage-store.ts +8 -3
- package/src/persistence/message-reads.test.ts +197 -0
- package/src/persistence/message-reads.ts +211 -0
- package/src/persistence/migrations/360-add-document-workspace-path.ts +5 -14
- package/src/persistence/migrations/366-chatgpt-subscription-row-identity.test.ts +120 -0
- package/src/persistence/migrations/366-chatgpt-subscription-row-identity.ts +62 -0
- package/src/persistence/real-user-turn-filter.ts +27 -3
- package/src/persistence/schema/documents.ts +4 -4
- package/src/persistence/steps.ts +9 -0
- package/src/plugin-api/__tests__/oauth-callback-url-export.test.ts +29 -0
- package/src/plugin-api/constants.ts +8 -0
- package/src/plugin-api/conversation-turn.ts +195 -5
- package/src/plugin-api/index.ts +38 -6
- package/src/plugin-api/resolve-credential.ts +3 -2
- package/src/plugin-api/store-credential.ts +146 -0
- package/src/plugin-api/vision-support.test.ts +1 -1
- package/src/plugin-api/webhook-url.ts +13 -11
- package/src/plugins/__tests__/mcp-servers.test.ts +371 -0
- package/src/plugins/defaults/main.ts +21 -7
- package/src/plugins/defaults/memory/AGENTS.md +15 -3
- package/src/plugins/defaults/memory/__tests__/buffer-format.test.ts +204 -0
- package/src/plugins/defaults/memory/__tests__/memory-retrospective-accounting.test.ts +72 -0
- package/src/plugins/defaults/memory/__tests__/memory-retrospective-job.test.ts +4 -1
- package/src/plugins/defaults/memory/__tests__/memory-retrospective-provider-path.test.ts +4 -1
- package/src/plugins/defaults/memory/buffer-format.ts +165 -0
- package/src/plugins/defaults/memory/context-search/sources/conversations.ts +6 -0
- package/src/plugins/defaults/memory/graph/__tests__/conversation-graph-memory-v2-routing.test.ts +77 -0
- package/src/plugins/defaults/memory/graph/conversation-graph-memory.ts +24 -6
- package/src/plugins/defaults/memory/graph/image-ref-utils.ts +3 -0
- package/src/plugins/defaults/memory/graph/tool-handlers.ts +1 -30
- package/src/plugins/defaults/memory/graph-topology/pending-buffer.test.ts +34 -0
- package/src/plugins/defaults/memory/graph-topology/pending-buffer.ts +17 -24
- package/src/plugins/defaults/memory/hooks/post-compact.ts +1 -4
- package/src/plugins/defaults/memory/hooks/user-prompt-submit.ts +53 -5
- package/src/plugins/defaults/memory/indexer.ts +3 -1
- package/src/plugins/defaults/memory/memory-retrospective-accounting.ts +19 -7
- package/src/plugins/defaults/memory/memory-retrospective-job.ts +4 -4
- package/src/plugins/defaults/memory/src/__tests__/memory-v3-gate-stats.test.ts +281 -0
- package/src/plugins/defaults/memory/src/memory-v2-routes.ts +12 -10
- package/src/plugins/defaults/memory/src/memory-v3-routes.ts +207 -0
- package/src/plugins/defaults/memory/substrate/__tests__/consolidation-job.test.ts +147 -0
- package/src/plugins/defaults/memory/substrate/__tests__/consolidation-prompt-flag-gating-guard.test.ts +8 -0
- package/src/plugins/defaults/memory/substrate/__tests__/edge-index.test.ts +0 -30
- package/src/plugins/defaults/memory/substrate/__tests__/ingest.test.ts +73 -1
- package/src/plugins/defaults/memory/substrate/__tests__/page-index.test.ts +37 -0
- package/src/plugins/defaults/memory/substrate/__tests__/page-links.test.ts +208 -0
- package/src/plugins/defaults/memory/substrate/__tests__/prompts-consolidation.test.ts +158 -1
- package/src/plugins/defaults/memory/substrate/__tests__/static-context.test.ts +199 -2
- package/src/plugins/defaults/memory/substrate/consolidation-job.ts +93 -24
- package/src/plugins/defaults/memory/substrate/edge-index.ts +0 -31
- package/src/plugins/defaults/memory/substrate/ingest.ts +59 -0
- package/src/plugins/defaults/memory/substrate/page-index.ts +35 -8
- package/src/plugins/defaults/memory/substrate/page-links.ts +133 -0
- package/src/plugins/defaults/memory/substrate/page-store.ts +10 -0
- package/src/plugins/defaults/memory/substrate/prompts/consolidation.ts +124 -34
- package/src/plugins/defaults/memory/substrate/skill-content.ts +8 -1
- package/src/plugins/defaults/memory/substrate/static-context.ts +160 -4
- package/src/plugins/defaults/memory/substrate/sweep-job.ts +2 -4
- package/src/plugins/defaults/memory/v1/graph/extraction.ts +3 -1
- package/src/plugins/defaults/memory/v3/__tests__/injection.test.ts +18 -0
- package/src/plugins/defaults/memory/v3/__tests__/shadow-plugin.test.ts +66 -1
- package/src/plugins/defaults/memory/v3/card.ts +9 -9
- package/src/plugins/defaults/memory/v3/core-set.test.ts +7 -0
- package/src/plugins/defaults/memory/v3/core-set.ts +5 -1
- package/src/plugins/defaults/memory/v3/edge.ts +13 -57
- package/src/plugins/defaults/memory/v3/injector.ts +8 -0
- package/src/plugins/defaults/memory/v3/prune.ts +2 -0
- package/src/plugins/defaults/memory/v3/selection-log-store.ts +2 -0
- package/src/plugins/defaults/memory/v3/shadow-plugin.ts +23 -9
- package/src/plugins/defaults/memory/worker-control.ts +1 -0
- package/src/plugins/defaults/memory/worker.ts +6 -3
- package/src/plugins/defaults/worker-entrypoints.ts +3 -0
- package/src/plugins/external-plugin-loader.ts +47 -0
- package/src/plugins/mcp-servers.ts +361 -0
- package/src/plugins/mtime-cache.ts +37 -46
- package/src/plugins/pipeline.ts +14 -7
- package/src/plugins/plugin-execution-context.ts +35 -11
- package/src/plugins/worker-plugin-surface.ts +33 -0
- package/src/prompts/templates/system-sections.ts +0 -7
- package/src/providers/__tests__/connection-model-compat.test.ts +1 -1
- package/src/providers/__tests__/context-overflow-error.test.ts +24 -0
- package/src/providers/__tests__/dispatch-connection-routing.test.ts +214 -2
- package/src/providers/__tests__/preflight-resolved-config.test.ts +57 -0
- package/src/providers/__tests__/retry-callsite.test.ts +25 -2
- package/src/providers/anthropic/__tests__/pause-turn-continuation.test.ts +170 -0
- package/src/providers/anthropic/client.ts +332 -241
- package/src/providers/call-site-routing.ts +30 -3
- package/src/providers/connection-resolution.ts +194 -11
- package/src/providers/inference/__tests__/adapter-factory-openai-compatible.test.ts +4 -0
- package/src/providers/inference/adapter-factory.ts +3 -0
- package/src/providers/inference/auth.ts +6 -0
- package/src/providers/inference/connection-availability.ts +24 -2
- package/src/providers/inference/connections.ts +2 -0
- package/src/providers/model-catalog.ts +3 -3
- package/src/providers/model-intents.ts +28 -8
- package/src/providers/openai/__tests__/chat-completions-provider-reasoning.test.ts +163 -40
- package/src/providers/openai/__tests__/tool-choice-mapping.test.ts +125 -2
- package/src/providers/openai/chat-completions-provider.ts +133 -36
- package/src/providers/openai/codex-models.ts +2 -1
- package/src/providers/provider-send-message.ts +32 -3
- package/src/providers/speech-to-text/__tests__/deepgram-flux-frames.test.ts +433 -0
- package/src/providers/speech-to-text/__tests__/deepgram-flux-realtime.test.ts +625 -0
- package/src/providers/speech-to-text/__tests__/provider-catalog.test.ts +34 -0
- package/src/providers/speech-to-text/__tests__/resolve.test.ts +285 -6
- package/src/providers/speech-to-text/deepgram-flux-frames.ts +395 -0
- package/src/providers/speech-to-text/deepgram-flux-realtime.ts +679 -0
- package/src/providers/speech-to-text/provider-catalog.ts +99 -8
- package/src/providers/speech-to-text/resolve.ts +25 -2
- package/src/routes/control.ts +1 -0
- package/src/routes/route-host-client.ts +1 -0
- package/src/routes/route-host-protocol.ts +7 -0
- package/src/routes/worker.ts +48 -15
- package/src/runtime/AGENTS.md +16 -17
- package/src/runtime/access-request-helper.ts +9 -12
- package/src/runtime/agent-wake.ts +18 -15
- package/src/runtime/guardian-reply-router.ts +6 -2
- package/src/runtime/pre-first-message-gate.ts +4 -0
- package/src/runtime/routes/__tests__/acp-claude-auth-routes.test.ts +12 -4
- package/src/runtime/routes/__tests__/conversation-list-routes.test.ts +522 -2
- package/src/runtime/routes/__tests__/conversation-query-routes.test.ts +52 -0
- package/src/runtime/routes/__tests__/default-provider-routes.test.ts +61 -0
- package/src/runtime/routes/__tests__/inference-profiles-routes.test.ts +44 -0
- package/src/runtime/routes/__tests__/inference-provider-connection-routes.test.ts +102 -1
- package/src/runtime/routes/__tests__/plugins-routes.test.ts +132 -35
- package/src/runtime/routes/__tests__/schedule-routes-disarm-reason.test.ts +215 -0
- package/src/runtime/routes/__tests__/stt-routes.test.ts +25 -0
- package/src/runtime/routes/__tests__/user-route-dispatcher-host.test.ts +52 -1
- package/src/runtime/routes/__tests__/user-route-dispatcher.test.ts +133 -1
- package/src/runtime/routes/channel-availability-routes.ts +32 -14
- package/src/runtime/routes/channel-route-shared.ts +0 -6
- package/src/runtime/routes/chatgpt-subscription-auth-routes.ts +6 -6
- package/src/runtime/routes/conversation-list-routes.ts +181 -23
- package/src/runtime/routes/conversation-management-routes.ts +2 -3
- package/src/runtime/routes/conversation-query-routes.ts +40 -27
- package/src/runtime/routes/conversation-routes.ts +11 -13
- package/src/runtime/routes/credential-prompt-routes.ts +4 -7
- package/src/runtime/routes/credential-routes.ts +26 -82
- package/src/runtime/routes/default-provider-routes.ts +15 -0
- package/src/runtime/routes/documents-routes.ts +3 -222
- package/src/runtime/routes/inbound-message-handler.ts +21 -45
- package/src/runtime/routes/inbound-stages/acl-enforcement.test.ts +0 -1
- package/src/runtime/routes/inbound-stages/acl-enforcement.ts +0 -9
- package/src/runtime/routes/inbound-stages/admission-policy.ts +1 -17
- package/src/runtime/routes/inbound-stages/bootstrap-intercept.test.ts +0 -1
- package/src/runtime/routes/inbound-stages/bootstrap-intercept.ts +2 -3
- package/src/runtime/routes/inbound-stages/edit-intercept.ts +101 -79
- package/src/runtime/routes/inbound-stages/guardian-reply-intercept.test.ts +0 -1
- package/src/runtime/routes/inbound-stages/guardian-reply-intercept.ts +12 -7
- package/src/runtime/routes/inbound-stages/reaction-intercept.test.ts +91 -8
- package/src/runtime/routes/inbound-stages/reaction-intercept.ts +72 -79
- package/src/runtime/routes/inbound-stages/secret-ingress-check.ts +2 -3
- package/src/runtime/routes/inference-profiles-routes.ts +20 -11
- package/src/runtime/routes/inference-provider-connection-routes.ts +77 -15
- package/src/runtime/routes/log-export-routes.ts +3 -0
- package/src/runtime/routes/mcp-auth-routes.ts +148 -57
- package/src/runtime/routes/playground/__tests__/inject-failures.test.ts +2 -0
- package/src/runtime/routes/playground/__tests__/reset-circuit.test.ts +3 -0
- package/src/runtime/routes/playground/inject-failures.ts +2 -2
- package/src/runtime/routes/playground/reset-circuit.ts +1 -1
- package/src/runtime/routes/plugins-routes.ts +38 -8
- package/src/runtime/routes/schedule-routes.ts +94 -7
- package/src/runtime/routes/settings-routes.ts +6 -6
- package/src/runtime/routes/stt-routes.ts +31 -25
- package/src/runtime/routes/surface-conversation-resolver.ts +3 -0
- package/src/runtime/routes/user-route-dispatcher.ts +69 -15
- package/src/runtime/routes/user-route-import.ts +108 -0
- package/src/runtime/routes/user-route-resolution.ts +18 -1
- package/src/runtime/routes/workspace-routes.ts +0 -9
- package/src/runtime/routes/workspace-utils.ts +3 -13
- package/src/runtime/services/conversation-serializer.ts +7 -2
- package/src/schedule/__tests__/plugin-schedule-declarations.test.ts +68 -5
- package/src/schedule/__tests__/plugin-schedule-reconciler.test.ts +81 -0
- package/src/schedule/plugin-schedule-availability.ts +58 -0
- package/src/schedule/plugin-schedule-declarations.ts +23 -27
- package/src/schedule/plugin-schedule-reconciler.ts +12 -3
- package/src/schedule/schedule-store.ts +5 -1
- package/src/schedule/scheduler.ts +9 -4
- package/src/schedule/worker-control.ts +1 -0
- package/src/schedule/worker.ts +6 -0
- package/src/security/oauth2.ts +6 -22
- package/src/stt/__tests__/daemon-batch-transcriber.test.ts +22 -0
- package/src/stt/__tests__/types.test.ts +94 -0
- package/src/stt/daemon-batch-transcriber.ts +10 -0
- package/src/stt/stt-stream-session.ts +8 -4
- package/src/stt/types.ts +103 -0
- package/src/subagent/__tests__/consult-prompt.test.ts +26 -15
- package/src/subagent/consult-context.ts +11 -11
- package/src/subagent/consult-prompt.ts +26 -35
- package/src/subagent/manager.ts +21 -40
- package/src/subagent/notify.ts +7 -1
- package/src/subagent/types.ts +15 -12
- package/src/tools/__tests__/tool-input-schemas.test.ts +7 -7
- package/src/tools/acp/spawn.test.ts +97 -0
- package/src/tools/acp/spawn.ts +38 -4
- package/src/tools/credentials/ref-parse.ts +35 -0
- package/src/tools/credentials/resolve.ts +4 -10
- package/src/tools/credentials/store.ts +168 -0
- package/src/tools/document/document-tool.ts +12 -3
- package/src/tools/executor.ts +53 -15
- package/src/tools/filesystem/read.ts +27 -10
- package/src/tools/host-filesystem/read.ts +27 -15
- package/src/tools/network/url-safety.ts +9 -16
- package/src/tools/permission-checker.ts +43 -52
- package/src/tools/registry.ts +2 -1
- package/src/tools/shared/filesystem/file-ops-service.ts +63 -35
- package/src/tools/shared/filesystem/legacy-read-args.ts +22 -0
- package/src/tools/shared/filesystem/types.ts +5 -5
- package/src/tools/skills/scaffold-managed.ts +25 -7
- package/src/tools/subagent/spawn.ts +28 -88
- package/src/tools/tool-types.ts +26 -13
- package/src/tools/types.ts +6 -6
- package/src/tools/ui-surface/surface-shape-docs.ts +12 -1
- package/src/tools/workflows/run-workflow.ts +1 -2
- package/src/tts/__tests__/reasoning-tag-filter.test.ts +78 -0
- package/src/tts/reasoning-tag-filter.ts +89 -0
- package/src/util/__tests__/worker-process-command.test.ts +37 -0
- package/src/util/logger.ts +16 -0
- package/src/util/think-tag-stream.ts +95 -0
- package/src/util/worker-process.ts +37 -4
- package/src/windows-compiled-cli.ts +32 -0
- package/src/windows-compiled-entry.ts +4 -0
- package/src/windows-compiled-logger.ts +6 -0
- package/src/windows-compiled-worker-entry.ts +29 -0
- package/src/workspace/byok-default-profile-ensure.ts +76 -24
- package/src/workspace/custom-profile-ensure.ts +4 -24
- package/src/workspace/migrations/142-consolidate-voice-front-door.ts +70 -0
- package/src/workspace/migrations/143-repair-deprecated-codex-model-id.ts +134 -0
- package/src/workspace/migrations/144-convert-stranded-subscription-openai-profiles.ts +265 -0
- package/src/workspace/migrations/145-collapse-profile-bindings-to-entries.ts +328 -0
- package/src/workspace/migrations/146-repair-retired-fireworks-deepseek-flash-model-id.ts +195 -0
- package/src/workspace/migrations/__tests__/141-stt-english-default-to-multilingual.test.ts +0 -10
- package/src/workspace/migrations/registry.ts +10 -0
- package/src/workspace/provider-commit-message-generator.ts +7 -5
- package/src/__tests__/document-workspace-file.test.ts +0 -467
- package/src/live-voice/__tests__/front-decision.test.ts +0 -645
- package/src/live-voice/front-decision.ts +0 -476
- package/src/permissions/ipc-risk-types.ts +0 -143
- package/src/permissions/risk-types.ts +0 -76
- package/src/subagent/__tests__/consult-transcript.test.ts +0 -184
- package/src/subagent/consult-transcript.ts +0 -90
package/AGENTS.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
For error handling conventions (throw vs result objects vs null), see [docs/error-handling.md](docs/error-handling.md).
|
|
4
4
|
|
|
5
|
-
Subdirectory-scoped rules live in local AGENTS.md files: `src/cli/`, `src/runtime/`, `src/approvals/`, `src/notifications/`, `src/plugins/`, `src/workspace/migrations/`.
|
|
5
|
+
Subdirectory-scoped rules live in local AGENTS.md files: `src/cli/`, `src/runtime/`, `src/approvals/`, `src/notifications/`, `src/permissions/`, `src/plugins/`, `src/workspace/migrations/`.
|
|
6
6
|
|
|
7
7
|
## Adding new environment variables
|
|
8
8
|
|
|
@@ -41,6 +41,12 @@ Do not coordinate hook behaviour by re-parsing the tool's JSON response to infer
|
|
|
41
41
|
|
|
42
42
|
Shared mutable resources written by more than one caller (e.g. `dist/` directories produced by `compileApp()`) must be serialised per-resource so concurrent callers cannot race on `rm -rf` + write sequences.
|
|
43
43
|
|
|
44
|
+
## Conversation event delivery and turn presence
|
|
45
|
+
|
|
46
|
+
A `Conversation` has one event sink, fixed at construction and never rebound: top-level conversations are built with the SSE hub (`broadcastMessage`), subagents with the wrapper that re-envelopes their events under the parent. Emit conversation-level events (activity state, confirmation prompts, notifier output, out-of-turn pushes) through `conversation.emit`, which delivers to the sink and then to `addEventObserver` observers. Observers are for policy layered on delivery (the voice bridge auto-resolves approval prompts it has no UI for), never for delivery itself. Do not add a per-subsystem sender slot, a bind/restore step around a turn, or a manual `broadcastMessage` for something the conversation already emits: an emitter that runs outside a live turn (queue drain, ACP or subagent notification, summarize route, call notifiers) reaches every client because the sink is always live.
|
|
47
|
+
|
|
48
|
+
Presence (whether a human is present to see UI and answer prompts) is per-turn state, never derived from delivery. Every dispatch path declares it: `isInteractive` on `runAgentLoop` / `processMessage` / `enqueueMessage`, or a wake's `clientless` pin of `currentTurnIsNonInteractive`. A caller that omits it gets a non-interactive turn (approval-gated tools are denied rather than left waiting on a prompt nobody may answer). `hasNoClient` is a getter over the in-flight turn's presence with no setter, so a new dispatch path cannot inherit whatever the previous turn left behind; if you are reaching for a way to set it, declare interactivity on the turn instead.
|
|
49
|
+
|
|
44
50
|
## Route architecture: shared ROUTES array
|
|
45
51
|
|
|
46
52
|
Routes in `src/runtime/routes/` are being migrated to a **shared `ROUTES` array** that serves as the single source of truth for both the HTTP server and the IPC server. Each route module exports `ROUTES: RouteDefinition[]` (from `routes/types.ts`), and the aggregator `routes/index.ts` collects them.
|
|
@@ -61,7 +67,7 @@ Three response shapes are supported:
|
|
|
61
67
|
- **Binary**: a JSON envelope with `headers: { "content-length": "<n>" }` followed by one binary frame of exactly `n` bytes.
|
|
62
68
|
- **Chunked streaming**: a JSON envelope with `headers: { "transfer-encoding": "chunked" }` followed by one or more binary frames, terminated by a zero-length frame.
|
|
63
69
|
|
|
64
|
-
The server auto-detects legacy newline-delimited JSON from old CLI clients and handles it transparently. New code must use length-prefixed framing via `writeMessage()` / `IpcFrameReader`
|
|
70
|
+
The server auto-detects legacy newline-delimited JSON from old CLI clients and handles it transparently. New code must use length-prefixed framing via `writeMessage()` / `IpcFrameReader` from `@vellumai/ipc-server-utils` (`packages/ipc-server-utils/src/ipc-framing.ts`).
|
|
65
71
|
|
|
66
72
|
### CLI ↔ daemon version skew
|
|
67
73
|
|
|
@@ -69,7 +75,7 @@ The CLI and daemon are always shipped and upgraded together — there is no vers
|
|
|
69
75
|
|
|
70
76
|
### IPC-only routes
|
|
71
77
|
|
|
72
|
-
Some routes are IPC-only (defined in `src/ipc/routes/`, not in the shared array). These are tool/CLI-specific methods (e.g. `wake_conversation`, `upsert_contact`) that have no HTTP counterpart. They follow the existing pattern: define in `src/ipc/routes
|
|
78
|
+
Some routes are IPC-only (defined in `src/ipc/routes/`, not in the shared array). These are tool/CLI-specific methods (e.g. `wake_conversation`, `upsert_contact`) that have no HTTP counterpart. They follow the existing pattern: define a `*_IPC_METHODS` map in `src/ipc/routes/` and add it to the list `AssistantIpcServer` iterates in `src/ipc/assistant-server.ts` (there is no index file; each map is imported by hand).
|
|
73
79
|
|
|
74
80
|
The module-level dependency-injection pattern (`registerFooDeps()`) used by some IPC routes is a known antipattern. New IPC-only routes should avoid it.
|
|
75
81
|
|
package/ARCHITECTURE.md
CHANGED
|
@@ -624,16 +624,21 @@ To add a new daemon batch STT provider, follow the full checklist in `docs/stt-p
|
|
|
624
624
|
|
|
625
625
|
Real-time conversation chat message capture on macOS uses a WebSocket-based streaming STT path. When the configured `services.stt` provider supports conversation streaming (determined by the `conversationStreamingMode` field in the provider catalog), native clients open a WebSocket session through the gateway to the daemon's `/v1/stt/stream` endpoint. The daemon resolves a `StreamingTranscriber` for the configured provider and streams partial/final transcript events back to the client in real time.
|
|
626
626
|
|
|
627
|
-
|
|
627
|
+
Each provider that advertises a conversation streaming mode in the catalog ships an adapter implementing the `StreamingTranscriber` interface from `src/stt/types.ts`. The catalog (`src/providers/speech-to-text/provider-catalog.ts`) is the source of truth for which providers are streaming-capable; `resolveStreamingTranscriber()` maps each to its adapter:
|
|
628
628
|
|
|
629
|
-
| Provider
|
|
630
|
-
|
|
|
631
|
-
| **Deepgram**
|
|
632
|
-
| **
|
|
629
|
+
| Provider | Adapter | Mode | Mechanism |
|
|
630
|
+
| ------------------ | ----------------------------------------------------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
631
|
+
| **Deepgram** | `src/providers/speech-to-text/deepgram-realtime.ts` | `realtime-ws` | Opens a WebSocket to Deepgram's `/v1/listen` endpoint, forwards raw PCM audio, normalizes Deepgram's `is_final`/`speech_final` semantics into `partial`/`final` events. Uses model `nova-2`. |
|
|
632
|
+
| **Deepgram Flux** | `src/providers/speech-to-text/deepgram-flux-realtime.ts` | `realtime-ws` | Opens a WebSocket to Deepgram's `/v2/listen` conversational endpoint. The model decides turn boundaries, so the adapter also emits `turn-start`, `eager-turn-end`, `turn-resumed`, and `turn-end`. Streaming only, no batch endpoint. Uses model `flux-general-en`. |
|
|
633
|
+
| **Google Gemini** | `src/providers/speech-to-text/google-gemini-live-stream.ts` | `realtime-ws` | Opens a bidirectional streaming session against Gemini's Live API (`ai.live.connect`), forwards PCM audio frames, and normalizes `serverContent.inputTranscription` events into `partial`/`final` events. Uses model `gemini-2.5-flash-native-audio-latest`. |
|
|
634
|
+
| **Vellum managed** | `src/providers/speech-to-text/vellum-managed-realtime.ts` | `realtime-ws` | Wraps the Deepgram adapter and dials the gateway speech relay (`/v1/speech/stt/stream`) instead of Deepgram directly; both relay legs speak Deepgram's wire protocol, and the model is pinned server-side. |
|
|
635
|
+
| **xAI** | `src/providers/speech-to-text/xai-realtime.ts` | `realtime-ws` | Opens a WebSocket to `wss://api.x.ai/v1/stt` and normalizes xAI's `transcript.partial`/`transcript.done` payloads into `partial`/`final` events. |
|
|
636
|
+
| **OpenAI Whisper** | `src/providers/speech-to-text/openai-whisper-stream.ts` | `incremental-batch` | Whisper exposes no streaming endpoint, so the shared `incremental-batch-stream.ts` strategy re-transcribes an accumulating buffer and diffs successive results into `partial`/`final` events. |
|
|
633
637
|
|
|
634
638
|
**Provider-specific behavior differences:**
|
|
635
639
|
|
|
636
640
|
- **Deepgram (`realtime-ws`)**: True WebSocket streaming with sub-second partial latency. Emits `partial` events for `is_final: false` frames and `final` events for `is_final: true` frames. Supports backpressure (drops audio frames when `bufferedAmount > 1 MiB`). Sends `CloseStream` message on stop with a 5-second grace period for the provider to flush remaining finals. Inactivity timeout: 30 seconds (provider-side hang detection). Connect timeout: 10 seconds. Auth errors map to close codes 1008/4001; rate limits to 1013.
|
|
641
|
+
- **Deepgram Flux (`realtime-ws`)**: The model owns turn detection, so the adapter carries no endpointing heuristics and exposes no `finalizeUtterance`: callers feature-detect it and fall back to `stop()`. It emits the four turn-boundary events alongside transcripts, where `eager-turn-end` is a speculative end-of-turn that a later `turn-resumed` retracts or a `turn-end` confirms. Streaming only: batch callers get a diagnostic naming `deepgram` as the batch-capable provider on the same credential.
|
|
637
642
|
- **Google Gemini (`realtime-ws`)**: WebSocket-backed Live API session. Partials are emitted as Gemini streams `inputTranscription.text` fragments; a `final` is emitted when the server signals `generationComplete` or `turnComplete`. On `stop()`, the adapter sends `audioStreamEnd: true` and waits up to a 5-second grace window for the server to flush remaining transcription before force-closing. Inactivity timeout: 30 seconds. Connect timeout: 10 seconds. Close codes 1008/4001 map to `auth`; 1013 maps to `rate-limit`; other codes map to `provider-error`. The model's own text turn is suppressed via a silent system instruction so we only pay for transcription.
|
|
638
643
|
|
|
639
644
|
**Session lifecycle (daemon side):**
|
|
@@ -643,7 +648,7 @@ Two provider adapters are supported, each implementing the `StreamingTranscriber
|
|
|
643
648
|
3. The transcriber's `start()` method opens the provider session.
|
|
644
649
|
4. A `ready` event (with `provider` field) is sent to the client, signaling that audio frames are accepted.
|
|
645
650
|
5. Client sends `audio` frames (binary WebSocket frames or base64-encoded JSON) and a `stop` event when recording ends.
|
|
646
|
-
6. The transcriber emits `
|
|
651
|
+
6. The transcriber emits transcript events, plus turn-boundary events (`turn-start`, `eager-turn-end`, `turn-resumed`, `turn-end`) from providers that detect them, forwarded to the client as JSON frames with monotonic `seq` numbers.
|
|
647
652
|
7. The session closes deterministically on: client disconnect, `stop` event followed by provider `closed`, idle timeout (60 seconds), or runtime shutdown.
|
|
648
653
|
|
|
649
654
|
**Session lifecycle (client side):**
|
|
@@ -704,6 +709,8 @@ Live voice STT uses the same `resolveStreamingTranscriber()` path as conversatio
|
|
|
704
709
|
|
|
705
710
|
Live voice TTS uses `streamLiveVoiceTtsAudio()` and the configured `services.tts.provider`. The selected provider must be registered, catalog-compatible, and expose `capabilities.supportsStreaming` plus `synthesizeStream()`. Providers whose catalog entry advertises `supportsStreaming` (currently all four catalog providers: ElevenLabs, Fish Audio, Deepgram, and xAI) satisfy this requirement; a buffered-only provider would remain available for buffered message playback or other supported surfaces, but live voice reports a TTS error instead of silently falling back to buffered playback.
|
|
706
711
|
|
|
712
|
+
The `voiceFrontDoor` prompt skips current-turn legacy and v3 memory retrieval, while prior frozen memory cards and static memory context remain available. The front-door rule escalates rather than guessing when required personal context is absent. The escalated leg runs the ordinary memory pipeline against the latest visible caller prompt before the quality model, with low selector effort. This keeps memory work off the front-door TTFT path without cross-leg speculative state.
|
|
713
|
+
|
|
707
714
|
V1 is local/gateway-scoped. Managed/cloud WebSocket proxy support, cross-region routing, and p50/p95 latency guarantees are out of scope for this version. Metrics frames expose timing data for measurement, but the architecture does not promise a hard latency SLO.
|
|
708
715
|
|
|
709
716
|
**Client service-first boundary:**
|
|
@@ -881,17 +888,29 @@ startup pass in `daemon/lifecycle.ts` (after plugin init, before the
|
|
|
881
888
|
scheduler starts), the end of `reconcilePluginSourcesNow()` in
|
|
882
889
|
`plugins/mtime-cache.ts` (install/uninstall/upgrade and sentinel-driven
|
|
883
890
|
changes), the plugin enable/disable routes, and a 60s backstop sweep
|
|
884
|
-
registered with the HTTP server's background sweeps.
|
|
891
|
+
registered with the HTTP server's background sweeps. The desired set is
|
|
892
|
+
gated on activation as well as on what is on disk:
|
|
893
|
+
`collectDesiredDeclarations` skips any plugin directory that
|
|
894
|
+
`isPluginDirActivated` (`plugins/mtime-cache.ts`) does not report as brought
|
|
895
|
+
up in this process, so a directory that merely exists under the plugins root
|
|
896
|
+
never arms a row.
|
|
885
897
|
|
|
886
898
|
Reconcile lag never lets a disabled plugin run. The disable path writes a
|
|
887
899
|
`.disabled` sentinel that only a reconcile pass turns into disarmed rows, so
|
|
888
900
|
the scheduler re-reads the sentinel at fire time and records a skipped run
|
|
889
901
|
instead of executing a claimed row whose plugin is off. Run-now applies the
|
|
890
|
-
same boundary through `
|
|
902
|
+
same boundary through `pluginScheduleSourceAvailable`
|
|
903
|
+
(`schedule/plugin-schedule-availability.ts`), which composes the activation
|
|
904
|
+
ledger with `declarationExistsOnDisk`. That disk probe also covers a plugin
|
|
891
905
|
whose manifest no longer parses, a declaration directory that is gone, and a
|
|
892
906
|
plugin root or declaration directory resolving outside the tree it belongs to
|
|
893
907
|
(the same `isInsidePluginRoot` containment the loader applies before importing
|
|
894
|
-
a plugin).
|
|
908
|
+
a plugin). Fire time in the scheduler and the user re-enable path in
|
|
909
|
+
`schedule-store.ts` deliberately use the disk probe on its own, because both
|
|
910
|
+
can run outside the daemon process, where the activation ledger is empty and
|
|
911
|
+
every plugin would read as unactivated. The reconciler's sweep disarms the
|
|
912
|
+
rows of a plugin it has not activated within one pass, which bounds what
|
|
913
|
+
those disk-only probes can let through.
|
|
895
914
|
|
|
896
915
|
A declaration that stops parsing keeps its execute row armed on the message
|
|
897
916
|
already stored in the row, but disarms its script rows: a script row fires its
|
|
@@ -904,7 +923,10 @@ timezone, message/script, retry policy, `definition_hash`); the execution
|
|
|
904
923
|
engine owns runtime columns (`next_run_at`, `status`, `last_*`,
|
|
905
924
|
`retry_count`) and its latches are never overridden; the user owns
|
|
906
925
|
`user_enabled`, a sticky override consulted when computing effective
|
|
907
|
-
`enabled`.
|
|
926
|
+
`enabled`. `definition_hash` is a sha256 over the relPath and bytes of
|
|
927
|
+
exactly two files, the declaration's `config.json` and its entrypoint, so
|
|
928
|
+
nothing else under `schedules/<name>/` can produce a definition change.
|
|
929
|
+
Nothing ever writes to plugin files. Execution itself is
|
|
908
930
|
unchanged: declared rows fire through the same `claimDueSchedules` path as
|
|
909
931
|
imperative ones.
|
|
910
932
|
|
|
@@ -75,6 +75,17 @@ graph LR
|
|
|
75
75
|
- `handleRemember` (`graph/tool-handlers.ts`) appends timestamped bullets to
|
|
76
76
|
`memory/buffer.md` + the daily archive whenever memory is enabled. Facts may
|
|
77
77
|
carry `[[slug]]` page hints that consolidation reads first when filing.
|
|
78
|
+
- The buffer entry format itself is owned by `buffer-format.ts` at the plugin
|
|
79
|
+
root: the writer (`formatRememberEntry`) plus the single matcher every reader
|
|
80
|
+
uses. A fact may span several lines, so the entry and the line are different
|
|
81
|
+
units, and the readers below (consolidation's cutoff, the injected `<info>`
|
|
82
|
+
Buffer cap, the Memory tab's pending nodes) all recognize entries through
|
|
83
|
+
that one matcher rather than their own. An entry opens with a timestamped
|
|
84
|
+
bullet at column 0 and its body is indented under it, which is what makes the
|
|
85
|
+
format round-trip: the delimiter is the column-0 bullet shape, so nesting the
|
|
86
|
+
body keeps fact content from imitating one. Entries written before that
|
|
87
|
+
nesting existed still parse, since an unindented body line that is not itself
|
|
88
|
+
entry-shaped is read as a continuation.
|
|
78
89
|
- **Consolidation** (`substrate/consolidation-job.ts`) is a background
|
|
79
90
|
agent conversation that files buffer entries into concept pages, rewrites
|
|
80
91
|
the aggregate views, and trims the buffer. Scheduling
|
|
@@ -88,6 +99,17 @@ graph LR
|
|
|
88
99
|
failure-backoff-respecting);
|
|
89
100
|
- manual "Run now" via `POST /v1/consolidation/run-now`.
|
|
90
101
|
Failed runs enter an exponential backoff (transient vs billing curves).
|
|
102
|
+
- The agent writes pages through the file tools, so nothing validates a
|
|
103
|
+
page at write time. Two corpus-level defects are instead reported by the
|
|
104
|
+
page index and fed back into the next pass's prompt as repair steps:
|
|
105
|
+
pages the index could not parse (`PageIndex.parseFailures`) and
|
|
106
|
+
structural references (`links:`, inline `[[wikilinks]]`, `edges:`) whose
|
|
107
|
+
target page does not exist (`PageIndex.danglingLinks`). The read-side
|
|
108
|
+
graph drops a dangling reference silently, so the job also counts them
|
|
109
|
+
after each run (`danglingLinks` on the outcome, a warn line) without
|
|
110
|
+
withholding the reindex follow-ups: the pages that were written still
|
|
111
|
+
become retrievable. The `memory validate` CLI subcommand reports the same
|
|
112
|
+
list.
|
|
91
113
|
- **Ingestion** (`substrate/ingest.ts`, exposed as `POST /v1/memory/ingest`;
|
|
92
114
|
generated HTTP operation id `memory_ingest_post`, IPC method
|
|
93
115
|
`memory_ingest`) is the second sanctioned writer of
|
|
@@ -99,11 +121,12 @@ graph LR
|
|
|
99
121
|
minute, and its same-minute burst guard falls back to processing the
|
|
100
122
|
whole buffer in a single oversized run. Ingest writes validated pages
|
|
101
123
|
directly instead. Purely mechanical: each page is validated and reported
|
|
102
|
-
individually
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
124
|
+
individually (a `links:`/`[[wikilink]]`/`edges:` target that is neither on
|
|
125
|
+
disk nor in the batch is a per-page warning, not a rejection), writes hold
|
|
126
|
+
the consolidation lock so a batch cannot interleave with a consolidation
|
|
127
|
+
pass, and a batch that wrote at least one page enqueues the same reindex
|
|
128
|
+
follow-ups as consolidation (`memory_v2_reembed`, `memory_v3_maintain`).
|
|
129
|
+
Consolidation remains the only LLM-driven writer.
|
|
107
130
|
|
|
108
131
|
### Ingestion tracks and provenance
|
|
109
132
|
|
|
@@ -161,6 +184,21 @@ Ingested pages carry provenance frontmatter with distinct consumers:
|
|
|
161
184
|
machinery (graph extraction, summarization, PKB indexing/filing, PKB
|
|
162
185
|
injection) is suppressed while the substrate is active.
|
|
163
186
|
|
|
187
|
+
#### Live voice front-door memory
|
|
188
|
+
|
|
189
|
+
The live voice front door does not await current-turn memory retrieval. Its
|
|
190
|
+
prompt hook skips legacy graph retrieval, and both v3 injectors skip
|
|
191
|
+
orchestration for `voiceFrontDoor`. Frozen cards from prior turns and the static
|
|
192
|
+
substrate context remain available. When the answer depends on a saved personal
|
|
193
|
+
fact that is absent from that context, the front-door rule escalates instead of
|
|
194
|
+
guessing.
|
|
195
|
+
|
|
196
|
+
The escalated leg runs the ordinary memory pipeline before the quality model.
|
|
197
|
+
V3 retrieval routes on the latest visible caller message, ignoring the hidden
|
|
198
|
+
continuation message used to start that leg. The selector uses low effort to
|
|
199
|
+
keep retrieval latency bounded. This keeps current-turn memory work off the
|
|
200
|
+
front-door TTFT path without maintaining speculative cross-leg state.
|
|
201
|
+
|
|
164
202
|
### Boot-time maintenance
|
|
165
203
|
|
|
166
204
|
`substrate/boot-maintenance.ts`, invoked from the memory plugin's
|
|
@@ -4,207 +4,151 @@ Permission, trust, and credential-security architecture details.
|
|
|
4
4
|
|
|
5
5
|
## Permission and Trust Security Model
|
|
6
6
|
|
|
7
|
-
The permission system
|
|
7
|
+
The permission system decides which tool actions the agent may execute without explicit approval. Two processes share the work, and the split is the load-bearing fact of this section:
|
|
8
|
+
|
|
9
|
+
- The **gateway** owns risk classification (`gateway/src/risk/*`, entered through the `classify_risk` IPC method), the trust rules that raise or lower a classified risk (stored in the gateway's SQLite `trust_rules` table and applied inside the classifiers), the auto-approve thresholds and channel-permission cells, and the per-actor `TrustClass` verdict.
|
|
10
|
+
- The **assistant** owns the turn's actor and capabilities (`runtime/capabilities.ts`), the sensitive-tool capability floor (`tools/tool-approval-handler.ts`), the allow / prompt / deny decision over risk × threshold × capabilities (`permissions/checker.ts` `check()` and `DefaultApprovalPolicy`), the prompt UX (`permissions/prompter.ts`), execution, and the `tool_invocations` audit row.
|
|
11
|
+
|
|
12
|
+
The assistant has no local classifier and no fallback: an unreachable gateway fails closed. It classifies each tool invocation exactly once and passes that classification down; nothing in the assistant memoises or re-derives risk. `assistant/src/permissions/AGENTS.md` and `gateway/src/risk/AGENTS.md` state the rules that follow.
|
|
8
13
|
|
|
9
14
|
### Permission Evaluation Flow
|
|
10
15
|
|
|
11
16
|
```mermaid
|
|
12
17
|
graph TB
|
|
13
|
-
TOOL_CALL["Tool invocation<br/>(toolName, input,
|
|
14
|
-
CLASSIFY -->
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
RISK_CHECK -->|"High"| RISK_THRESHOLD{"Risk-based<br/>threshold fallback"}
|
|
27
|
-
|
|
28
|
-
NO_MATCH -->|"tool.origin === 'skill'"| PROMPT_SKILL["decision: prompt<br/>Skill tools always ask"]
|
|
29
|
-
NO_MATCH -->|"workspace-scoped<br/>+ Low risk"| AUTO_WS["decision: allow<br/>Workspace-scoped auto-allow"]
|
|
30
|
-
NO_MATCH -->|"otherwise"| RISK_THRESHOLD
|
|
31
|
-
|
|
32
|
-
RISK_THRESHOLD{"risk ≤ autoApproveUpTo<br/>threshold?"}
|
|
33
|
-
RISK_THRESHOLD -->|"yes"| AUTO_THRESHOLD["decision: allow<br/>within auto-approve threshold"]
|
|
34
|
-
RISK_THRESHOLD -->|"no"| PROMPT_THRESHOLD["decision: prompt<br/>above auto-approve threshold"]
|
|
18
|
+
TOOL_CALL["Tool invocation<br/>(toolName, input, context)"] --> CLASSIFY["classifyRisk() once, before the gates<br/>gateway classify_risk over IPC<br/>→ level, reason, matchType, options"]
|
|
19
|
+
CLASSIFY --> GATES["Pre-execution gates<br/>abort · unparseable args · guardian control-plane policy ·<br/>sensitive-tool floor + approval-matrix cell · disk pressure ·<br/>unknown tool · allowedToolNames · channel policy · Zod parse"]
|
|
20
|
+
GATES -->|"blocked"| GATE_OUT["denied / error<br/>audited with the classified level"]
|
|
21
|
+
GATES -->|"sensitive, non-guardian"| GRANT{"scoped grant<br/>consumed?"}
|
|
22
|
+
GRANT -->|"yes"| EXECUTE["execute<br/>(no permission check;<br/>provenance grant_scoped_consumed)"]
|
|
23
|
+
GRANT -->|"escalate-and-wait"| ESCALATE["guardian tool-grant request<br/>+ inline wait"]
|
|
24
|
+
GRANT -->|"deny actor / no grant"| GATE_OUT
|
|
25
|
+
GATES -->|"passed"| CHECK["checkPermission → check()<br/>threshold + capability context"]
|
|
26
|
+
CHECK --> POLICY["DefaultApprovalPolicy.evaluate"]
|
|
27
|
+
POLICY -->|"allow"| EXECUTE
|
|
28
|
+
POLICY -->|"prompt"| PROMPT_ROUTE{"presence?"}
|
|
29
|
+
PROMPT_ROUTE -->|"non-interactive turn"| AUTO_OR_DENY["guardian within background threshold → allow<br/>otherwise deny (no human to ask)"]
|
|
30
|
+
PROMPT_ROUTE -->|"interactive"| PROMPT["confirmation_request → user allow / deny"]
|
|
35
31
|
```
|
|
36
32
|
|
|
33
|
+
The order inside `check()`: the memory-retrospective skill-authoring grant, then the classification, then the auto-approve threshold for the turn's execution context (per-conversation override, then channel-permission cell, then global), then `DefaultApprovalPolicy.evaluate`. A `prompt` computed from a cached threshold is re-checked against a fresh read before the user is interrupted. `checkPermission` then applies `forcePromptSideEffects` and `requireFreshApproval` (allow → prompt), the non-interactive denial for uncovered inline-command skill loads, and platform-hosted sandboxed-bash auto-approve for guardians.
|
|
34
|
+
|
|
37
35
|
### Auto-Approve Threshold
|
|
38
36
|
|
|
39
|
-
|
|
37
|
+
Thresholds are **gateway-owned**: stored in the gateway's SQLite database, read by the assistant over IPC (`get_global_thresholds`, `get_conversation_threshold`), and set from the Settings UI (Permissions & Privacy) or the per-conversation risk tolerance picker. When the gateway is unreachable the assistant resolves `"none"` (Strict), fail-closed with no local fallback.
|
|
38
|
+
|
|
39
|
+
Gateway defaults per execution context (`gateway/src/ipc/threshold-handlers.ts`): `interactive` (a conversation with a client) `medium`, `autonomous` (background/scheduled) `low`, `headless` `none`. A per-conversation override wins over the global value; for non-guardian actors a channel-permission cell can only lower the effective threshold.
|
|
40
40
|
|
|
41
41
|
| `autoApproveUpTo` | Low-risk tools | Medium-risk tools | High-risk tools |
|
|
42
42
|
| ----------------- | -------------- | ----------------- | --------------- |
|
|
43
43
|
| `"none"` | Prompted | Prompted | Prompted |
|
|
44
|
-
| `"low"`
|
|
44
|
+
| `"low"` | Auto-allowed | Prompted | Prompted |
|
|
45
45
|
| `"medium"` | Auto-allowed | Auto-allowed | Prompted |
|
|
46
46
|
| `"high"` | Auto-allowed | Auto-allowed | Auto-allowed |
|
|
47
47
|
|
|
48
|
-
|
|
48
|
+
### Approval Policy
|
|
49
49
|
|
|
50
|
-
|
|
50
|
+
`DefaultApprovalPolicy.evaluate` (`assistant/src/permissions/approval-policy.ts`) turns a classified risk into allow / prompt, in this order:
|
|
51
51
|
|
|
52
|
-
|
|
52
|
+
1. `bash` with the gateway's `sandboxAutoApprove` verdict, when the threshold is not `"none"`: allow.
|
|
53
|
+
2. Third-party code (skill- or plugin-owned tools that are not first-party bundled, or a builtin running under a manifest override): allow within threshold, otherwise prompt.
|
|
54
|
+
3. Low risk, workspace-scoped invocation, within threshold: allow.
|
|
55
|
+
4. Low risk, bundled-skill tool, within threshold: allow.
|
|
56
|
+
5. Otherwise: allow when risk ≤ threshold, prompt when above.
|
|
53
57
|
|
|
54
|
-
|
|
55
|
-
| ----------------- | ---------------------- | ------------------------------------------------------------------------ |
|
|
56
|
-
| `id` | `string` | Unique identifier (UUID for user rules, `default:*` for system defaults) |
|
|
57
|
-
| `tool` | `string` | Tool name to match (e.g., `bash`, `file_write`, `skill_load`) |
|
|
58
|
-
| `pattern` | `string` | Minimatch glob pattern for the command/target string |
|
|
59
|
-
| `scope` | `string` | Path prefix or `everywhere` — restricts where the rule applies |
|
|
60
|
-
| `decision` | `allow \| deny \| ask` | What to do when the rule matches |
|
|
61
|
-
| `priority` | `number` | Higher priority wins; deny wins ties at equal priority |
|
|
62
|
-
| `executionTarget` | `string?` | `sandbox` or `host` — restricts by execution context |
|
|
58
|
+
The policy never returns deny; denials come from the gates, the capability floor, and the checks in `checkPermission`. There is no allow / deny / ask rule axis: trust rules act on the classified risk, upstream of this policy.
|
|
63
59
|
|
|
64
|
-
|
|
60
|
+
### Trust Rules (v3)
|
|
65
61
|
|
|
66
|
-
|
|
62
|
+
Rules live in the gateway (`gateway/src/db/trust-rule-store.ts`, cached in-process by `gateway/src/risk/trust-rule-cache.ts`, mutated only through the gateway HTTP routes, which refresh the cache). A rule is `{ tool, pattern, risk: low | medium | high, description, origin: default | user_defined, userModified, deleted }`, unique on `(tool, pattern)`. Default rules are seeded at gateway start from the bash command registry (`gateway/src/db/seed-trust-rules.ts`) and can be modified or reset; user rules are created, updated, and deleted through `/v1/trust-rules`.
|
|
67
63
|
|
|
68
|
-
|
|
64
|
+
Matching happens inside the classifiers: the bash classifier looks the command up exact, path-stripped, then by shorter subcommand prefixes, each in literal and `action:` form, user rules winning over defaults; the file, web, skill, and schedule classifiers look up a per-tool override. A matched rule replaces the base risk and the classification carries `matchType: "user_rule"`. The assistant sees only that: it never stores, matches, or lists rules except to proxy the list over IPC for clients.
|
|
69
65
|
|
|
70
|
-
|
|
71
|
-
| ---------------------------------------------------------------- | --------------------------- | -------------------------------------------------------------------------------------------- |
|
|
72
|
-
| `file_read`, `web_search`, `skill_load` | Low | Read-only or informational |
|
|
73
|
-
| `file_write`, `file_edit` | Medium (default) | Filesystem mutations |
|
|
74
|
-
| `file_write`, `file_edit` targeting skill source paths | **High** | `isSkillSourcePath()` detects managed/bundled/workspace/extra skill roots |
|
|
75
|
-
| `host_file_write`, `host_file_edit` targeting skill source paths | **High** | Same path classification, host variant |
|
|
76
|
-
| `bash`, `host_bash` | Varies | Parsed via tree-sitter: low-risk programs = Low, high-risk programs = High, unknown = Medium |
|
|
77
|
-
| `scaffold_managed_skill`, `delete_managed_skill` | High | Skill lifecycle mutations always high-risk |
|
|
78
|
-
| `evaluate_typescript_code` | High | Arbitrary code execution |
|
|
79
|
-
| Skill-origin tools with no matching rule | Prompted regardless of risk | Even Low-risk skill tools default to `ask` |
|
|
66
|
+
### Risk Classification
|
|
80
67
|
|
|
81
|
-
|
|
68
|
+
Classifiers (`gateway/src/risk/`), keyed by tool:
|
|
82
69
|
|
|
83
|
-
|
|
70
|
+
| Tool | Classifier | Notes |
|
|
71
|
+
| -------------------------------------------------------------- | --------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
72
|
+
| `bash`, `host_bash` | `bash-risk-classifier.ts` | Tree-sitter parse (`shell-parser.ts`, memoised on the command text) against `command-registry/`; per-command arg rules; dangerous patterns; `sandboxAutoApprove` for allowlisted sandbox commands with lexically-resolved path args |
|
|
73
|
+
| `file_*`, `host_file_*` | `file-risk-classifier.ts` | Writes to skill source, workspace code, and other code-loaded directories escalate to High; the assistant sends symlink-resolved paths and the protected/skill directories with the request |
|
|
74
|
+
| `web_fetch`, `network_request` | `web-risk-classifier.ts` | URL-based; private-network access escalates |
|
|
75
|
+
| `skill_load`, `scaffold_managed_skill`, `delete_managed_skill` | `skill-risk-classifier.ts` | A `skill_load` whose skill has inline command expansions (executes shell at load time) is High; skill lifecycle mutations are High |
|
|
76
|
+
| `schedule_create`, `schedule_update` | `schedule-risk-classifier.ts` | Scheduled command risk |
|
|
77
|
+
| Everything else | fallback in `risk-classification-handlers.ts` | The tool's `defaultRiskLevel` from the assistant's registry (`matchType: "registry"`), or `medium` with an "Unknown tool" reason. This branch consults no trust rule and emits no allowlist options, so a user rule cannot cover an MCP or other classifier-less tool today |
|
|
84
78
|
|
|
85
|
-
The `
|
|
79
|
+
The response (`ClassifyRiskIpcResponse`, the shared contract in `packages/gateway-client/src/gateway-ipc-contracts.ts`, validated by both sides) carries the level, reason, `matchType`, allowlist / scope / directory-scope options, command candidates and action keys, `sandboxAutoApprove`, and the path args it was based on. The assistant's one adjustment is the bash symlink-escape re-check: it resolves the gateway's lexically-checked path args through the real filesystem and revokes `sandboxAutoApprove` if any escapes the workspace. It runs on every classification.
|
|
86
80
|
|
|
87
|
-
|
|
88
|
-
2. `skill_load:<skill-id>` — matches any-version rules
|
|
89
|
-
3. `skill_load:<raw-selector>` — matches the raw user-provided selector
|
|
81
|
+
### Sensitive-Tool Capability Floor
|
|
90
82
|
|
|
91
|
-
|
|
83
|
+
Independently of risk, `tools/tool-approval-handler.ts` computes how far an invocation reaches (`none`, `sandbox`, `host`; host-target tools, out-of-workspace file access, and inline-command skill loads reach `host`) and reads the actor's `sensitiveToolApproval` capability: guardian `self`, trusted and unverified contacts `escalate-and-wait`, unknown `deny`. A non-`none` reach for a non-guardian either consumes a scoped approval grant, escalates to the guardian and waits inline, or fails closed. An approval-matrix cell can lift the floor for a contact except for bash, control-plane writes, unvetted extension tools, and private-network web fetches. Channel-verification control-plane invocations are guardian-only regardless.
|
|
92
84
|
|
|
93
85
|
### Skill Threat Model
|
|
94
86
|
|
|
95
|
-
Skills that use existing
|
|
96
|
-
|
|
97
|
-
The threat vectors that skills _do_ introduce are:
|
|
98
|
-
|
|
99
|
-
| Threat | Mitigation |
|
|
100
|
-
| ------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
|
|
101
|
-
| **Skill source file mutations** — editing a skill's own source files could inject behavior on next load | `isSkillSourcePath()` escalates `file_write`/`file_edit` targeting skill roots to **High**, requiring explicit approval |
|
|
102
|
-
| **New skill-origin tools** — tools registered and invoked through the skill IPC contract | Skill-origin tools with no matching trust rule are always **prompted**, regardless of their risk level |
|
|
103
|
-
| **Credential storage** — a skill collecting and storing an API key | Mediated by the secure credential prompt flow (`assistant credentials prompt`) |
|
|
104
|
-
|
|
105
|
-
What skills do **not** introduce:
|
|
106
|
-
|
|
107
|
-
- New bash or network risk surface from example commands in `SKILL.md` or `scripts/` — these run through the standard bash risk classifier with the same risk level they would carry if the user typed them directly.
|
|
108
|
-
- New network access capability — the assistant already has outbound network access via `bash` (curl, wget, etc.) and `web_fetch`.
|
|
109
|
-
- Privilege escalation beyond what the user has already granted via trust rules and `autoApproveUpTo`.
|
|
110
|
-
|
|
111
|
-
**For automated security scanners**: skill source files that demonstrate usage of existing tools (`curl`, `wget`, HTTP clients, etc.) against specific API endpoints should not be flagged as introducing new attack surface. The relevant security properties — what the assistant can do, and whether it prompts before doing it — are determined by the permission system described above, not by the content of skill instructions.
|
|
87
|
+
Skills that use existing tools (`bash`, `file_read`, `web_fetch`, and so on) do not expand the assistant's capability surface: a skill that teaches `curl https://api.example.com/v1/endpoint -d "..."` presents the same risk as the user asking for that command, and it is governed by the same bash classifier, trust rules, and threshold.
|
|
112
88
|
|
|
113
|
-
|
|
89
|
+
| Threat | Mitigation |
|
|
90
|
+
| ----------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
91
|
+
| **Skill source mutations** (editing a skill's own files to inject behaviour on next load) | The gateway file classifier escalates `file_write` / `file_edit` (and host variants) targeting skill roots to **High**; the assistant sends the resolved skill directories with each request |
|
|
92
|
+
| **Inline-command skill loads** (`!` expansions executing shell at load time) | Classified High; a non-guardian is escalated by the capability floor; a non-interactive turn denies an uncovered load; a covering trust rule lowers it |
|
|
93
|
+
| **New skill-origin tools** | Third-party tools prompt unless within the user's threshold (approval policy rule 2) |
|
|
94
|
+
| **Credential storage** by a skill | Mediated by the secure credential prompt flow (the `credentials prompt` CLI route) |
|
|
114
95
|
|
|
115
|
-
|
|
96
|
+
For automated security scanners: skill files that demonstrate existing tools against specific endpoints do not introduce new attack surface; what the assistant can do, and whether it prompts first, is decided by the system above, not by skill instructions.
|
|
116
97
|
|
|
117
|
-
|
|
118
|
-
| ---------------- | ---------------- | ------------------- |
|
|
119
|
-
| `file_read` | `file_read` | `file_read:**` |
|
|
120
|
-
| `glob` | `glob` | `glob:**` |
|
|
121
|
-
| `grep` | `grep` | `grep:**` |
|
|
122
|
-
| `list_directory` | `list_directory` | `list_directory:**` |
|
|
123
|
-
| `web_search` | `web_search` | `web_search:**` |
|
|
124
|
-
| `web_fetch` | `web_fetch` | `web_fetch:**` |
|
|
98
|
+
### Allowlist and Scope Options
|
|
125
99
|
|
|
126
|
-
|
|
100
|
+
The prompt's "always allow" ladder comes from the classification: bash offers the exact command and then `action:<program>` / `action:<tokens>` keys (max depth 3; pipelines and other complex operators offer only the exact command); file tools offer the exact path, up to three ancestor directories, then the tool; web tools offer the canonicalized URL, its origin, then the tool; skill tools offer a version-pinned and an any-version option in the `skill_load` or `skill_load_dynamic` namespace. Web URLs are canonicalized through `@vellumai/service-contracts/url-normalization`, so the pattern a rule is saved under has one spelling. Rule lookup in the classifiers is an exact-string match on the invocation as written, so a saved rule matches only an identically spelled call. A tool whose classifier produced no ladder gets none: the assistant builds no options of its own.
|
|
127
101
|
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
In addition to the opt-in starter bundle, the permission system seeds unconditional default allow rules at priority 100 for two categories:
|
|
131
|
-
|
|
132
|
-
| Rule ID | Tool | Pattern | Rationale |
|
|
133
|
-
| ---------------------------------------------- | ------------------------- | --------------------------- | -------------------------------------------------------------------------------------------------------- |
|
|
134
|
-
| `default:allow-skill_load-global` | `skill_load` | `skill_load:*` | Loading any skill is globally allowed — no prompt for activating bundled, managed, or workspace skills |
|
|
135
|
-
| `default:allow-browser_navigate-global` | `browser_navigate` | `browser_navigate:*` | Browser tools migrated from core to the bundled `browser` skill; default allow preserves frictionless UX |
|
|
136
|
-
| `default:allow-browser_snapshot-global` | `browser_snapshot` | `browser_snapshot:*` | (same) |
|
|
137
|
-
| `default:allow-browser_screenshot-global` | `browser_screenshot` | `browser_screenshot:*` | (same) |
|
|
138
|
-
| `default:allow-browser_close-global` | `browser_close` | `browser_close:*` | (same) |
|
|
139
|
-
| `default:allow-browser_click-global` | `browser_click` | `browser_click:*` | (same) |
|
|
140
|
-
| `default:allow-browser_type-global` | `browser_type` | `browser_type:*` | (same) |
|
|
141
|
-
| `default:allow-browser_press_key-global` | `browser_press_key` | `browser_press_key:*` | (same) |
|
|
142
|
-
| `default:allow-browser_wait_for-global` | `browser_wait_for` | `browser_wait_for:*` | (same) |
|
|
143
|
-
| `default:allow-browser_extract-global` | `browser_extract` | `browser_extract:*` | (same) |
|
|
144
|
-
| `default:allow-browser_fill_credential-global` | `browser_fill_credential` | `browser_fill_credential:*` | (same) |
|
|
145
|
-
|
|
146
|
-
These rules are emitted by `getDefaultRuleTemplates()` in `assistant/src/permissions/defaults.ts`. Because they use priority 100 (equal to user rules), they take effect regardless of the `autoApproveUpTo` threshold. The `skill_load` rule means skill activation never prompts; the `browser_*` rules mean the browser skill's tools behave identically to the old core `headless-browser` tool from a permission standpoint.
|
|
147
|
-
|
|
148
|
-
### Shell Command Identity and Allowlist Options
|
|
149
|
-
|
|
150
|
-
For `bash` and `host_bash` tool invocations, the permission system uses parser-derived action keys (via `shell-identity.ts`) instead of raw whitespace-split patterns. This produces more meaningful allowlist options that reflect the actual command structure.
|
|
151
|
-
|
|
152
|
-
**Candidate building** (`buildShellCommandCandidates`): The shell parser (`tools/terminal/parser.ts`) produces segments and operators. `analyzeShellCommand()` extracts segments, operators, opaque-construct flags, and dangerous patterns. `deriveShellActionKeys()` then classifies the command:
|
|
153
|
-
|
|
154
|
-
- **Simple action** (optional setup-prefix segments like `cd`, `export`, `pushd` + exactly one action segment): Produces hierarchical `action:` keys. For example, `cd /repo && gh pr view 5525 --json title` yields candidates: the full original command text (`cd /repo && gh pr view 5525 --json title`), and action keys `action:gh pr view`, `action:gh pr`, `action:gh` (narrowest to broadest, max depth 3).
|
|
155
|
-
- **Complex command** (pipelines with `|`, or multiple non-prefix action segments): Only the full original command text is returned as a candidate — no action keys.
|
|
156
|
-
|
|
157
|
-
**Allowlist option ranking** (`buildShellAllowlistOptions`): For simple actions, the prompt offers options ordered from most specific to broadest: the full original command text (exact match), then action keys from deepest to shallowest. For complex commands, only the full original command text is offered. This prevents over-generalization of pipelines into permissive rules.
|
|
158
|
-
|
|
159
|
-
**Trust rule pattern format**: Action keys use the `action:` prefix in trust rules (e.g., `action:gh pr view`). The trust store matches these via `findHighestPriorityRule()` against the candidate list produced by `buildShellCommandCandidates()`.
|
|
160
|
-
|
|
161
|
-
**Scope ordering**: Scope options for all tools (including shell) are ordered from narrowest to broadest: project > parent directories > everywhere. The macOS chat UI uses a two-step flow for persistent rules: the user first selects the allowlist pattern, then selects the scope. This explicit scope selection replaces any silent auto-selection, ensuring the user always knows where the rule will apply.
|
|
102
|
+
Directory-scope ladders (`directoryScopeOptions`) come from the gateway; the coarser workingDir scope ladder (`generateScopeOptions`) is still built assistant-side. Saving a persistent decision means the client creating a trust rule through the gateway (`POST /v1/trust-rules`); the assistant's `POST /v1/confirm` accepts only `allow` and `deny`.
|
|
162
103
|
|
|
163
104
|
### Prompt UX
|
|
164
105
|
|
|
165
|
-
|
|
106
|
+
`confirmation_request` (SSE) carries `requestId`, `toolName`, redacted `input`, `riskLevel`, `riskReason`, `isContainerized`, `executionTarget`, `allowlistOptions`, `scopeOptions`, `directoryScopeOptions`, an optional preview `diff`, `conversationId`, `persistentDecisionsAllowed`, and `toolUseId`; it is also promoted to a guardian request for channel delivery. A prompt that times out or loses its client resolves to deny.
|
|
166
107
|
|
|
167
|
-
|
|
168
|
-
| ------------------ | --------------------------------------------------- |
|
|
169
|
-
| `toolName` | The tool being invoked |
|
|
170
|
-
| `input` | Redacted tool input (sensitive fields removed) |
|
|
171
|
-
| `riskLevel` | `low`, `medium`, or `high` |
|
|
172
|
-
| `executionTarget` | `sandbox` or `host` — where the action will execute |
|
|
173
|
-
| `allowlistOptions` | Suggested patterns for "always allow" rules |
|
|
174
|
-
| `scopeOptions` | Suggested scopes for rule persistence |
|
|
108
|
+
### Canonical Paths
|
|
175
109
|
|
|
176
|
-
The
|
|
110
|
+
The assistant symlink-resolves file-tool paths and the working directory before sending them (`resolveFileToolPaths` in `permissions/checker.ts`, `normalizeFilePath` in `skills/path-classifier.ts`), and canonicalises the protected and skill directories the same way, so a symlinked component cannot bypass the gateway's prefix checks.
|
|
177
111
|
|
|
178
|
-
###
|
|
112
|
+
### Audit
|
|
179
113
|
|
|
180
|
-
|
|
114
|
+
Every invocation ends in one `tool_invocations` row (`telemetry/tool-audit.ts`): `decision` (`allow` / `denied` / `error` / prompt outcomes), the classified `riskLevel`, redacted input and a capped result preview, duration, and telemetry-only columns gated on analytics consent. Rows written by the gates (denials and errors) carry the same classified level as the rest of the call; a call whose classification did not complete (aborted before start, gateway unreachable) records `unclassified` rather than a level.
|
|
181
115
|
|
|
182
116
|
### Key Source Files
|
|
183
117
|
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
|
187
|
-
|
|
|
188
|
-
| `assistant/src/permissions/
|
|
189
|
-
| `assistant/src/permissions/
|
|
190
|
-
| `assistant/src/permissions/
|
|
191
|
-
| `
|
|
192
|
-
| `assistant/src/
|
|
193
|
-
| `assistant/src/
|
|
194
|
-
| `assistant/src/tools/executor.ts`
|
|
195
|
-
| `assistant/src/
|
|
118
|
+
Assistant:
|
|
119
|
+
|
|
120
|
+
| File | Role |
|
|
121
|
+
| -------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
|
|
122
|
+
| `assistant/src/permissions/checker.ts` | `classifyRisk()` (builds the request, calls the gateway, applies the symlink-escape re-check), `check()`, the workingDir scope ladder |
|
|
123
|
+
| `assistant/src/permissions/approval-policy.ts` | `DefaultApprovalPolicy` |
|
|
124
|
+
| `assistant/src/permissions/gateway-threshold-reader.ts`, `channel-permission-query.ts` | Threshold and channel-permission-cell reads over IPC |
|
|
125
|
+
| `packages/gateway-client/src/gateway-ipc-contracts.ts` | `ClassifyRiskIpcParamsSchema` / `ClassifyRiskIpcResponseSchema`, the `classify_risk` contract both sides import |
|
|
126
|
+
| `assistant/src/permissions/prompter.ts` | `confirmation_request` → `confirmation_response` |
|
|
127
|
+
| `assistant/src/permissions/types.ts` | `PolicyContext`, `RiskLevel`, `UserDecision`, thresholds |
|
|
128
|
+
| `assistant/src/tools/executor.ts` | `ToolExecutor`: one classification per invocation, gates, permission check, execution, audit |
|
|
129
|
+
| `assistant/src/tools/tool-approval-handler.ts` | Pre-execution gates and the sensitive-tool capability floor |
|
|
130
|
+
| `assistant/src/tools/permission-checker.ts` | `checkPermission`: policy adjustments, non-interactive routing, prompting |
|
|
131
|
+
| `assistant/src/runtime/capabilities.ts` | `resolveCapabilities(trustClass)` |
|
|
132
|
+
| `assistant/src/skills/path-classifier.ts`, `skills/version-hash.ts` | Path canonicalisation and skill version hashes sent with skill classifications |
|
|
133
|
+
| `assistant/src/telemetry/tool-audit.ts` | `tool_invocations` audit terminals |
|
|
134
|
+
|
|
135
|
+
Gateway:
|
|
136
|
+
|
|
137
|
+
| File | Role |
|
|
138
|
+
| ------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- |
|
|
139
|
+
| `gateway/src/ipc/risk-classification-handlers.ts` | `classify_risk` IPC handler, the entry point the assistant calls |
|
|
140
|
+
| `gateway/src/risk/bash-risk-classifier.ts` | Shell command risk, via `shell-parser.ts` / `shell-identity.ts` and `command-registry/` |
|
|
141
|
+
| `gateway/src/risk/file-risk-classifier.ts` | File tool risk, including code-loaded-directory escalation |
|
|
142
|
+
| `gateway/src/risk/web-risk-classifier.ts` | Web tool risk |
|
|
143
|
+
| `gateway/src/risk/skill-risk-classifier.ts` | Skill lifecycle and inline-command load risk |
|
|
144
|
+
| `gateway/src/risk/schedule-risk-classifier.ts` | Scheduled task risk |
|
|
145
|
+
| `gateway/src/risk/trust-rule-cache.ts`, `gateway/src/db/trust-rule-store.ts`, `gateway/src/db/seed-trust-rules.ts` | Trust rules: storage, seeding, in-process cache |
|
|
146
|
+
| `gateway/src/http/routes/trust-rules.ts` | Trust rule CRUD, refreshing the cache on every mutation |
|
|
147
|
+
| `gateway/src/ipc/threshold-handlers.ts` | Global and per-conversation thresholds |
|
|
196
148
|
|
|
197
149
|
### Permission Simulation (Tool Permission Tester)
|
|
198
150
|
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
**Simulation semantics:**
|
|
202
|
-
|
|
203
|
-
- The request specifies `toolName`, `input`, and optional context overrides (`workingDir`, `isInteractive`).
|
|
204
|
-
- The daemon runs `classifyRisk()` and `check()` against the live trust rules, then returns the decision (`allow`, `deny`, or `prompt`), risk level, reason, matched rule ID, and (when decision is `prompt`) the full `promptPayload` with allowlist/scope options.
|
|
205
|
-
- **Simulation-only allow/deny**: A simulated `allow` or `deny` decision does not persist any state. No trust rules are created or modified.
|
|
206
|
-
- **Always-allow persistence**: When the tester UI's "Always Allow" action is used, the client sends a separate `add_trust_rule` message that persists the rule to `trust.json`, identical to the existing confirmation flow.
|
|
207
|
-
- **Non-interactive override**: When `isInteractive` is false, `prompt` decisions are converted to `deny` (no client available to approve).
|
|
151
|
+
`POST tools/simulate-permission` (`tools_simulate_permission_post`, `assistant/src/runtime/routes/settings-routes.ts`) dry-runs an invocation through classification and `check()` without executing or persisting anything. It takes `toolName`, `input`, and optional `workingDir` / `isInteractive`, and returns the decision, risk level, reason, execution target, and, for a `prompt`, the allowlist / scope options; when `isInteractive` is false a `prompt` is reported as `deny`.
|
|
208
152
|
|
|
209
153
|
---
|
|
210
154
|
|