@vellumai/assistant 0.12.3-staging.1 → 0.12.3-staging.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ARCHITECTURE.md +43 -29
- package/Dockerfile +63 -0
- package/docs/architecture/security.md +10 -5
- package/docs/architecture/turn-actor.md +2 -0
- package/docs/desktop-browser-cli.md +45 -4
- package/docs/live-voice-task-outcomes.md +100 -0
- package/docs/mode-sessions.md +183 -0
- package/docs/voice-input-diagnostics.md +58 -0
- package/node_modules/@vellumai/ces-client/node_modules/@vellumai/service-contracts/package.json +2 -1
- package/node_modules/@vellumai/ces-client/node_modules/@vellumai/service-contracts/src/index.ts +1 -0
- package/node_modules/@vellumai/ces-client/node_modules/@vellumai/service-contracts/src/integration-categories.ts +36 -0
- package/node_modules/@vellumai/gateway-client/node_modules/@vellumai/service-contracts/package.json +2 -1
- package/node_modules/@vellumai/gateway-client/node_modules/@vellumai/service-contracts/src/index.ts +1 -0
- package/node_modules/@vellumai/gateway-client/node_modules/@vellumai/service-contracts/src/integration-categories.ts +36 -0
- package/node_modules/@vellumai/gateway-client/src/gateway-ipc-contracts.ts +20 -0
- package/node_modules/@vellumai/gateway-client/src/index.ts +4 -1
- package/node_modules/@vellumai/gateway-client/src/outbound-contract.ts +38 -7
- package/node_modules/@vellumai/gateway-client/src/trust-verdict-contract.ts +5 -6
- package/node_modules/@vellumai/service-contracts/package.json +2 -1
- package/node_modules/@vellumai/service-contracts/src/index.ts +1 -0
- package/node_modules/@vellumai/service-contracts/src/integration-categories.ts +36 -0
- package/node_modules/@vellumai/slack-text/src/index.ts +4 -8
- package/openapi.yaml +665 -14
- package/package.json +1 -1
- package/scripts/smoke-desktop-browser-cli.ts +10 -26
- package/scripts/smoke-desktop-window-theme.ts +2 -3
- package/src/__tests__/agent-loop-output-hooks.test.ts +150 -1
- package/src/__tests__/agent-loop-provider-error-recording.test.ts +87 -19
- package/src/__tests__/annotate-activity-metadata.test.ts +3 -1
- package/src/__tests__/annotate-risk-options.test.ts +2 -0
- package/src/__tests__/answered-question-persistence.test.ts +2 -0
- package/src/__tests__/anthropic-gateway-shared.test.ts +22 -0
- package/src/__tests__/approval-message-composer.test.ts +1 -118
- package/src/__tests__/approval-routes-http.test.ts +3 -0
- package/src/__tests__/assistant-event-hub-connection-history.test.ts +111 -0
- package/src/__tests__/assistant-event-hub-targeted.test.ts +50 -0
- package/src/__tests__/assistant-event-targeting.test.ts +186 -0
- package/src/__tests__/assistant-stream-state.test.ts +29 -9
- package/src/__tests__/browser-fill-credential.test.ts +11 -22
- package/src/__tests__/btw-routes.test.ts +4 -1
- package/src/__tests__/build-persisted-content.test.ts +1 -1
- package/src/__tests__/call-conversation-messages.test.ts +5 -0
- package/src/__tests__/call-recovery.test.ts +7 -2
- package/src/__tests__/call-routes-http.test.ts +12 -0
- package/src/__tests__/call-store.test.ts +7 -2
- package/src/__tests__/channel-retry-sweep.test.ts +9 -0
- package/src/__tests__/chat-reveal-guard-priming.test.ts +2 -0
- package/src/__tests__/checker.test.ts +11 -3
- package/src/__tests__/client-os-metadata-persistence.test.ts +35 -1
- package/src/__tests__/computer-use-skill-manifest-regression.test.ts +0 -25
- package/src/__tests__/computer-use-tools.test.ts +1 -233
- package/src/__tests__/config-model-image-provider.test.ts +39 -0
- package/src/__tests__/config-schema.test.ts +16 -0
- package/src/__tests__/config-watcher.test.ts +24 -0
- package/src/__tests__/context-search-agent-runner.test.ts +260 -1
- package/src/__tests__/context-search-fanout.test.ts +1 -1
- package/src/__tests__/context-search-types.test.ts +7 -7
- package/src/__tests__/conversation-agent-loop-disk-pressure.test.ts +8 -2
- package/src/__tests__/conversation-agent-loop-fatal-cleanup.test.ts +2 -0
- package/src/__tests__/conversation-agent-loop.test.ts +546 -16
- package/src/__tests__/conversation-app-control-lifecycle.test.ts +53 -1
- package/src/__tests__/conversation-evictor.test.ts +62 -37
- package/src/__tests__/conversation-history-boundary-repair-failure.test.ts +174 -0
- package/src/__tests__/conversation-loop-db-lock-resilience.test.ts +2 -0
- package/src/__tests__/conversation-process-app-control-preactivation.test.ts +2 -0
- package/src/__tests__/conversation-retry-route.test.ts +22 -0
- package/src/__tests__/conversation-routes-disk-view.test.ts +2 -0
- package/src/__tests__/conversation-routes-enabled-plugins.test.ts +2 -0
- package/src/__tests__/conversation-routes-guardian-reply.test.ts +10 -0
- package/src/__tests__/conversation-routes-hidden-queue.test.ts +2 -0
- package/src/__tests__/conversation-routes-slash-commands.test.ts +2 -0
- package/src/__tests__/conversation-runtime-assembly.test.ts +29 -0
- package/src/__tests__/conversation-surface-snapshots.test.ts +543 -0
- package/src/__tests__/conversation-surfaces-action-delivery.test.ts +151 -0
- package/src/__tests__/conversation-surfaces-point-at-budget.test.ts +63 -1
- package/src/__tests__/conversation-tool-setup-attribution.test.ts +23 -0
- package/src/__tests__/conversation-undo.test.ts +61 -0
- package/src/__tests__/credential-execution-client.test.ts +6 -6
- package/src/__tests__/credential-prompt-route.test.ts +35 -3
- package/src/__tests__/cu-unified-flow.test.ts +82 -1
- package/src/__tests__/db-mode-sessions-migration.test.ts +175 -0
- package/src/__tests__/deterministic-verification-control-plane.test.ts +2 -0
- package/src/__tests__/dm-backfill.test.ts +91 -1
- package/src/__tests__/document-create-route.test.ts +88 -0
- package/src/__tests__/document-sync-tags.test.ts +27 -0
- package/src/__tests__/emit-signal-routing-intent.test.ts +142 -2
- package/src/__tests__/events-tail-route.test.ts +25 -0
- package/src/__tests__/evict-conversations-for-reload.test.ts +57 -8
- package/src/__tests__/guardian-action-sweep.test.ts +16 -0
- package/src/__tests__/guardian-dispatch.test.ts +13 -0
- package/src/__tests__/headless-browser-interactions.test.ts +64 -20
- package/src/__tests__/heartbeat-service.test.ts +2 -59
- package/src/__tests__/helpers/mock-conversation.ts +21 -0
- package/src/__tests__/helpers/native-web-search-harness.ts +2 -0
- package/src/__tests__/helpers/verification-sessions-ipc-sim.ts +5 -0
- package/src/__tests__/host-cu-proxy.test.ts +46 -2
- package/src/__tests__/http-user-message-parity.test.ts +2 -0
- package/src/__tests__/image-credentials.test.ts +49 -0
- package/src/__tests__/image-service-dispatcher.test.ts +45 -0
- package/src/__tests__/inference-profile-session-handler.test.ts +1 -1
- package/src/__tests__/interrupt-on-send.test.ts +57 -43
- package/src/__tests__/interrupt-turn-note.test.ts +5 -6
- package/src/__tests__/llm-catalog-parity.test.ts +2 -2
- package/src/__tests__/media-generate-image.test.ts +49 -1
- package/src/__tests__/message-provenance-contact-id.test.ts +69 -0
- package/src/__tests__/migration-export-to-gcs.test.ts +142 -0
- package/src/__tests__/mode-session-history.test.ts +245 -0
- package/src/__tests__/mode-session-recovery.test.ts +222 -0
- package/src/__tests__/mode-session-wire.test.ts +141 -0
- package/src/__tests__/notification-guardian-path.test.ts +7 -0
- package/src/__tests__/notification-telegram-adapter.test.ts +2 -2
- package/src/__tests__/notification-vellum-adapter.test.ts +34 -0
- package/src/__tests__/oauth-provider-serializer.test.ts +12 -0
- package/src/__tests__/oauth-providers-routes.test.ts +1 -0
- package/src/__tests__/outbound-slack-persistence.test.ts +2 -0
- package/src/__tests__/persistence-secret-redaction.test.ts +2 -0
- package/src/__tests__/plugin-api-provider.test.ts +5 -0
- package/src/__tests__/plugin-api-resolve-credential.test.ts +97 -0
- package/src/__tests__/plugin-import-boundary-guard.test.ts +9 -0
- package/src/__tests__/process-message-display-content.test.ts +56 -0
- package/src/__tests__/reaction-persistence.test.ts +5 -0
- package/src/__tests__/request-secret-standalone-channel-gate.test.ts +25 -9
- package/src/__tests__/runtime-events-sse-reconnect.test.ts +1 -1
- package/src/__tests__/scripted-turn-metadata-persistence.test.ts +73 -0
- package/src/__tests__/secret-ingress-http.test.ts +2 -0
- package/src/__tests__/secure-keys.test.ts +43 -1
- package/src/__tests__/send-endpoint-busy.test.ts +163 -1
- package/src/__tests__/server-history-render.test.ts +91 -1
- package/src/__tests__/settings-routes.test.ts +50 -1
- package/src/__tests__/skill-script-runner-sandbox.test.ts +32 -0
- package/src/__tests__/subagent-fork-notifications.test.ts +1 -2
- package/src/__tests__/subagent-manager-notify.test.ts +1 -2
- package/src/__tests__/subagent-notify-parent.test.ts +146 -39
- package/src/__tests__/subagent-notify-stale-rebuild.test.ts +2 -4
- package/src/__tests__/surface-completion-in-flight-snapshot.test.ts +2 -0
- package/src/__tests__/thread-backfill.test.ts +53 -0
- package/src/__tests__/tool-preview-lifecycle.test.ts +2 -0
- package/src/__tests__/tool-result-metadata-plumbing.test.ts +3 -1
- package/src/__tests__/tool-start-timestamp.test.ts +2 -0
- package/src/__tests__/twilio-routes.test.ts +1 -0
- package/src/__tests__/web-search-backend-failure.test.ts +1 -1
- package/src/activation/progress-store.test.ts +12 -7
- package/src/agent/loop.ts +99 -41
- package/src/api/events/assistant-turn-start.ts +3 -0
- package/src/api/events/generation-handoff.ts +2 -0
- package/src/api/events/message-complete.ts +2 -0
- package/src/api/events/notification-conversation-created.ts +14 -16
- package/src/api/events/notification-intent.ts +4 -2
- package/src/api/events/tool-result.ts +85 -0
- package/src/api/events/tool-use-start.ts +3 -0
- package/src/api/events/user-message-echo.ts +3 -0
- package/src/api/index.ts +31 -6
- package/src/api/mode-session.test.ts +131 -0
- package/src/api/mode-session.ts +148 -0
- package/src/api/remember-tool.ts +15 -0
- package/src/api/responses/conversation-message.ts +6 -0
- package/src/approvals/AGENTS.md +4 -2
- package/src/browser/__tests__/operations.test.ts +29 -0
- package/src/browser/operation-meta.ts +2 -1
- package/src/browser/operations.ts +29 -0
- package/src/browser/virtual-desktop-target.ts +2 -8
- package/src/bundler/bundle-signer.ts +1 -3
- package/src/calls/__tests__/barge-in-guard.test.ts +3 -0
- package/src/calls/__tests__/call-controller.test.ts +94 -0
- package/src/calls/__tests__/call-funnel-telemetry.test.ts +204 -0
- package/src/calls/__tests__/voice-control-protocol.test.ts +15 -0
- package/src/calls/__tests__/voice-escalation-judge.test.ts +209 -0
- package/src/calls/__tests__/voice-leg-coordinator.test.ts +168 -0
- package/src/{live-voice/__tests__/live-voice-metrics.test.ts → calls/__tests__/voice-metrics.test.ts} +32 -38
- package/src/calls/__tests__/voice-session-bridge.test.ts +576 -122
- package/src/calls/barge-in-guard.ts +5 -0
- package/src/calls/call-controller.ts +196 -1
- package/src/calls/call-domain.ts +4 -0
- package/src/calls/call-store.ts +87 -2
- package/src/calls/guardian-action-sweep.ts +22 -8
- package/src/calls/media-stream-audio-transcode.ts +1 -3
- package/src/calls/typesafe-noul.ts +156 -0
- package/src/calls/voice-control-protocol.ts +6 -0
- package/src/calls/voice-escalation-judge.ts +145 -0
- package/src/calls/voice-escalation-target.ts +10 -0
- package/src/calls/voice-leg-coordinator.ts +121 -3
- package/src/{live-voice/live-voice-metrics.ts → calls/voice-metrics.ts} +94 -87
- package/src/calls/voice-session-bridge.ts +256 -112
- package/src/calls/voice-triage-escalate.ts +7 -0
- package/src/cli/__tests__/roadmap-cli.test.ts +4 -41
- package/src/cli/commands/__tests__/notifications.test.ts +35 -10
- package/src/cli/commands/browser.help.ts +4 -5
- package/src/cli/commands/browser.ts +0 -1
- package/src/cli/commands/clients.help.ts +46 -0
- package/src/cli/commands/clients.ts +116 -0
- package/src/cli/commands/credentials.help.ts +6 -1
- package/src/cli/commands/notifications.help.ts +16 -2
- package/src/cli/commands/notifications.ts +30 -8
- package/src/cli/commands/roadmap.help.ts +18 -21
- package/src/cli/commands/roadmap.ts +4 -21
- package/src/cli/lib/__tests__/list-installed-plugins.test.ts +5 -2
- package/src/cli/lib/bundled-marketplace.json +84 -34
- package/src/cli/lib/plugin-marketplace.ts +8 -0
- package/src/cli/lib/publish-plugin.ts +1 -2
- package/src/config/__tests__/default-provider.test.ts +3 -1
- package/src/config/__tests__/memory-retrospective-schema.test.ts +14 -1
- package/src/config/bundled-skills/computer-use/SKILL.md +1 -1
- package/src/config/bundled-skills/computer-use/TOOLS.json +89 -1
- package/src/config/bundled-skills/image-studio/SKILL.md +17 -2
- package/src/config/bundled-skills/image-studio/tools/media-generate-image.ts +7 -11
- package/src/config/bundled-skills/sequences/TOOLS.json +1 -5
- package/src/config/call-site-defaults.ts +9 -0
- package/src/config/feature-flag-registry.json +16 -0
- package/src/config/profile-text-generation.test.ts +2 -2
- package/src/config/schemas/__tests__/tools.test.ts +31 -0
- package/src/config/schemas/call-site-catalog.ts +14 -0
- package/src/config/schemas/channels.ts +0 -18
- package/src/config/schemas/llm.ts +3 -1
- package/src/config/schemas/mcp.ts +2 -1
- package/src/config/schemas/memory-retrospective.ts +11 -2
- package/src/config/schemas/services.ts +15 -6
- package/src/config/schemas/tools.ts +10 -0
- package/src/config/session-groups-gate.ts +21 -0
- package/src/config/skills.ts +2 -6
- package/src/conversations/__tests__/message-consolidation.test.ts +149 -0
- package/src/conversations/message-consolidation.ts +102 -5
- package/src/credential-execution/ces-connect.test.ts +69 -0
- package/src/credential-execution/ces-connect.ts +115 -0
- package/src/credential-execution/ces-runtime.ts +55 -93
- package/src/credential-execution/executable-discovery.ts +1 -11
- package/src/daemon/__tests__/conversation-surfaces-launch.test.ts +10 -1
- package/src/daemon/__tests__/inference-profile-notification.test.ts +2 -0
- package/src/daemon/browser-mode-session.test.ts +402 -0
- package/src/daemon/browser-mode-session.ts +186 -0
- package/src/daemon/computer-use-mode-session.test.ts +210 -0
- package/src/daemon/computer-use-mode-session.ts +105 -0
- package/src/daemon/config-watcher.ts +10 -0
- package/src/daemon/conversation-agent-loop-handlers.ts +58 -10
- package/src/daemon/conversation-agent-loop.ts +219 -25
- package/src/daemon/conversation-evictor.ts +5 -24
- package/src/daemon/conversation-history.ts +10 -0
- package/src/daemon/conversation-interrupt-repair.ts +4 -43
- package/src/daemon/conversation-interrupt.ts +7 -11
- package/src/daemon/conversation-messaging.ts +58 -5
- package/src/daemon/conversation-mode-session.test.ts +1667 -0
- package/src/daemon/conversation-mode-session.ts +1008 -0
- package/src/daemon/conversation-process.ts +5 -0
- package/src/daemon/conversation-runtime-assembly.ts +10 -20
- package/src/daemon/conversation-skill-tools.ts +1 -0
- package/src/daemon/conversation-store.ts +4 -4
- package/src/daemon/conversation-surface-snapshots.ts +135 -0
- package/src/daemon/conversation-surfaces.ts +113 -5
- package/src/daemon/conversation-tool-setup.ts +8 -3
- package/src/daemon/conversation.ts +143 -41
- package/src/daemon/handlers/config-model.ts +7 -5
- package/src/daemon/handlers/shared.ts +55 -16
- package/src/daemon/host-cu-proxy.ts +51 -22
- package/src/daemon/lifecycle.ts +30 -13
- package/src/daemon/message-protocol.ts +0 -1
- package/src/daemon/message-provenance.ts +28 -4
- package/src/daemon/mode-session-startup-recovery.ts +23 -0
- package/src/daemon/mode-session-tracking.ts +54 -0
- package/src/daemon/process-message.ts +13 -4
- package/src/daemon/virtual-desktop-context.ts +21 -0
- package/src/desktop/__tests__/fake-desktop.ts +1 -0
- package/src/desktop/desktop-accessibility-script.ts +88 -0
- package/src/desktop/desktop-accessibility.test.ts +160 -0
- package/src/desktop/desktop-accessibility.ts +169 -0
- package/src/desktop/desktop-automation-lease.test.ts +76 -68
- package/src/desktop/desktop-automation-lease.ts +41 -50
- package/src/desktop/desktop-browser-endpoint.ts +4 -0
- package/src/desktop/desktop-computer-use-routing.test.ts +330 -0
- package/src/desktop/desktop-computer-use.test.ts +449 -0
- package/src/desktop/desktop-computer-use.ts +502 -0
- package/src/desktop/desktop-dependencies.ts +47 -293
- package/src/desktop/desktop-display.ts +2 -2
- package/src/desktop/desktop-panel-config.test.ts +13 -3
- package/src/desktop/desktop-panel-config.ts +109 -15
- package/src/desktop/desktop-session-manager.test.ts +69 -21
- package/src/desktop/desktop-session-manager.ts +76 -22
- package/src/desktop/desktop-stream-bridge.test.ts +11 -122
- package/src/desktop/desktop-stream-bridge.ts +6 -8
- package/src/desktop/desktop-window-theme.ts +5 -25
- package/src/desktop/virtual-desktop-feature.ts +52 -2
- package/src/desktop/virtual-desktop-platform.test.ts +5 -9
- package/src/documents/document-store.ts +23 -0
- package/src/heartbeat/__tests__/heartbeat-service.test.ts +22 -26
- package/src/heartbeat/heartbeat-service.ts +5 -33
- package/src/ipc/__tests__/browser-ipc.test.ts +215 -6
- package/src/ipc/assistant-server.ts +2 -0
- package/src/ipc/routes/__tests__/channel-reply-ipc-routes.test.ts +111 -0
- package/src/ipc/routes/channel-reply-ipc-routes.ts +73 -0
- package/src/live-voice/__tests__/continuation-judge.test.ts +151 -0
- package/src/live-voice/__tests__/live-activity-reporter.test.ts +154 -15
- package/src/live-voice/__tests__/live-voice-flux-turn-end.test.ts +743 -31
- package/src/live-voice/__tests__/live-voice-look-follow-up.test.ts +191 -10
- package/src/live-voice/__tests__/live-voice-session-groups-gate.test.ts +87 -0
- package/src/live-voice/__tests__/live-voice-session-manager.test.ts +5 -1
- package/src/live-voice/__tests__/live-voice-sight-frame-echo.test.ts +41 -0
- package/src/live-voice/__tests__/live-voice-sight-frame.test.ts +1017 -9
- package/src/live-voice/__tests__/live-voice-triage-escalate.test.ts +224 -1
- package/src/live-voice/__tests__/live-voice-tts-session.test.ts +101 -0
- package/src/live-voice/__tests__/live-voice-tts.test.ts +44 -0
- package/src/live-voice/__tests__/live-voice-vad.test.ts +939 -75
- package/src/live-voice/__tests__/microphone-idle-gate.test.ts +87 -0
- package/src/live-voice/__tests__/session-controls.test.ts +23 -0
- package/src/live-voice/__tests__/task-outcomes.test.ts +117 -0
- package/src/live-voice/__tests__/voice-input-diagnostics.test.ts +117 -0
- package/src/live-voice/camera-mode-session.test.ts +265 -0
- package/src/live-voice/camera-mode-session.ts +277 -0
- package/src/live-voice/continuation-judge.ts +123 -0
- package/src/live-voice/live-activity-reporter.ts +115 -6
- package/src/live-voice/live-voice-manager.ts +36 -1
- package/src/live-voice/live-voice-photo.ts +88 -7
- package/src/live-voice/live-voice-session-manager.ts +15 -0
- package/src/live-voice/live-voice-session.ts +1443 -747
- package/src/live-voice/live-voice-tts.ts +22 -4
- package/src/live-voice/microphone-idle-gate.ts +56 -0
- package/src/live-voice/protocol.ts +171 -4
- package/src/live-voice/session-controls.ts +11 -3
- package/src/live-voice/task-outcomes.ts +131 -0
- package/src/live-voice/voice-input-diagnostics.ts +117 -0
- package/src/mcp/__tests__/manager-tool-caps.test.ts +63 -18
- package/src/mcp/__tests__/tool-caps.test.ts +28 -1
- package/src/mcp/manager.ts +3 -0
- package/src/mcp/tool-caps.ts +12 -4
- package/src/media/__tests__/image-models.test.ts +58 -0
- package/src/media/image-credentials.ts +5 -0
- package/src/media/image-models.ts +52 -0
- package/src/media/image-service.ts +8 -0
- package/src/media/openrouter-image-service.test.ts +133 -0
- package/src/media/openrouter-image-service.ts +153 -0
- package/src/media/types.ts +1 -1
- package/src/messaging/providers/__tests__/transport-dispatch.test.ts +65 -11
- package/src/messaging/providers/discord/send.ts +4 -17
- package/src/messaging/providers/discord/transport.ts +2 -2
- package/src/messaging/providers/outlook/client.ts +16 -22
- package/src/messaging/providers/telegram-bot/send.test.ts +236 -1
- package/src/messaging/providers/telegram-bot/send.ts +10 -7
- package/src/messaging/providers/telegram-bot/transport.ts +3 -10
- package/src/messaging/providers/whatsapp/send.test.ts +169 -4
- package/src/monitoring/__tests__/control.test.ts +17 -5
- package/src/monitoring/control.ts +15 -5
- package/src/notifications/AGENTS.md +1 -1
- package/src/notifications/README.md +83 -99
- package/src/notifications/__tests__/broadcaster.test.ts +1 -1
- package/src/notifications/__tests__/channel-allowlist.test.ts +39 -0
- package/src/notifications/__tests__/copy-composer.test.ts +1 -1
- package/src/notifications/__tests__/decision-engine.test.ts +79 -2
- package/src/notifications/__tests__/deterministic-checks.test.ts +10 -10
- package/src/notifications/__tests__/home-feed-side-effect.test.ts +93 -4
- package/src/notifications/__tests__/notification-utils.test.ts +18 -0
- package/src/notifications/adapters/macos.ts +37 -46
- package/src/notifications/broadcaster.ts +5 -6
- package/src/notifications/channel-allowlist.ts +44 -0
- package/src/notifications/copy-composer.ts +0 -23
- package/src/notifications/decision-engine.ts +24 -14
- package/src/notifications/deterministic-checks.ts +1 -1
- package/src/notifications/emit-signal.ts +123 -37
- package/src/notifications/home-feed-side-effect.ts +9 -7
- package/src/notifications/notification-utils.ts +19 -0
- package/src/notifications/signal.ts +4 -19
- package/src/notifications/types.ts +15 -0
- package/src/oauth/connection-resolver.ts +1 -2
- package/src/oauth/oauth-store.ts +1 -3
- package/src/oauth/provider-serializer.ts +7 -0
- package/src/oauth/seed-providers.ts +30 -0
- package/src/onboarding/onboarding-events-store.ts +99 -12
- package/src/persistence/__tests__/client-connection-events-store.test.ts +322 -0
- package/src/persistence/client-connection-events-store.ts +377 -0
- package/src/persistence/conversation-crud.ts +55 -6
- package/src/persistence/conversation-mode-sessions.test.ts +424 -0
- package/src/persistence/conversation-mode-sessions.ts +460 -0
- package/src/persistence/conversation-queries.ts +6 -19
- package/src/persistence/llm-usage-store.ts +3 -9
- package/src/persistence/message-metadata.ts +25 -0
- package/src/persistence/migrations/381-create-conversation-mode-sessions.ts +41 -0
- package/src/persistence/migrations/382-create-client-connection-events.ts +40 -0
- package/src/persistence/migrations/__tests__/382-create-client-connection-events.test.ts +56 -0
- package/src/persistence/schema/conversations.ts +67 -0
- package/src/persistence/schema/infrastructure.ts +29 -0
- package/src/persistence/steps.ts +17 -0
- package/src/plugin-api/__tests__/plugin-name-env.test.ts +249 -0
- package/src/plugin-api/effective-context-window.ts +33 -0
- package/src/plugin-api/index.ts +24 -5
- package/src/plugin-api/plugin-name-env.ts +125 -0
- package/src/plugin-api/resolve-credential.ts +79 -20
- package/src/plugin-api/vision-support.test.ts +22 -0
- package/src/plugin-api/vision-support.ts +17 -5
- package/src/plugins/__tests__/active-plugin-env.test.ts +136 -0
- package/src/plugins/active-plugin-env.ts +51 -0
- package/src/plugins/defaults/index.ts +24 -0
- package/src/plugins/defaults/memory/__tests__/archive-removal-framing.test.ts +64 -0
- package/src/plugins/defaults/memory/__tests__/worker-control.test.ts +19 -6
- package/src/plugins/defaults/memory/context-search/agent-protocol.ts +9 -12
- package/src/plugins/defaults/memory/context-search/agent-runner.ts +78 -22
- package/src/plugins/defaults/memory/context-search/format.ts +51 -7
- package/src/plugins/defaults/memory/context-search/limits.ts +10 -12
- package/src/plugins/defaults/memory/context-search/recall-input.ts +38 -0
- package/src/plugins/defaults/memory/context-search/search.ts +8 -14
- package/src/plugins/defaults/memory/context-search/sources/conversations.ts +1 -0
- package/src/plugins/defaults/memory/context-search/types.ts +1 -11
- package/src/plugins/defaults/memory/graph/__tests__/handle-remember-v2.test.ts +11 -4
- package/src/plugins/defaults/memory/graph/tool-handlers.ts +12 -21
- package/src/plugins/defaults/memory/graph/tools.ts +35 -64
- package/src/plugins/defaults/memory/src/memory-eval-routes.ts +2 -3
- package/src/plugins/defaults/memory/src/memory-item-routes.test.ts +5 -0
- package/src/plugins/defaults/memory/src/memory-item-routes.ts +2 -1
- package/src/plugins/defaults/memory/substrate/prompts/consolidation.ts +2 -2
- package/src/plugins/defaults/memory/tools.test.ts +124 -9
- package/src/plugins/defaults/memory/tools.ts +38 -11
- package/src/plugins/defaults/memory/v3/__tests__/pool-log-store.test.ts +165 -27
- package/src/plugins/defaults/memory/v3/__tests__/pool-select.test.ts +626 -27
- package/src/plugins/defaults/memory/v3/__tests__/shadow-plugin.test.ts +46 -27
- package/src/plugins/defaults/memory/v3/orchestrate.ts +78 -38
- package/src/plugins/defaults/memory/v3/pool-log-store.ts +65 -79
- package/src/plugins/defaults/memory/v3/pool-select.test.ts +14 -5
- package/src/plugins/defaults/memory/v3/pool-select.ts +683 -44
- package/src/plugins/defaults/memory/worker-control.ts +15 -5
- package/src/plugins/defaults/task-progress/__tests__/registration.test.ts +70 -0
- package/src/plugins/defaults/task-progress/__tests__/task-progress-context.test.ts +380 -0
- package/src/plugins/defaults/task-progress/hooks/post-compact.ts +19 -0
- package/src/plugins/defaults/task-progress/hooks/user-prompt-submit.ts +24 -0
- package/src/plugins/defaults/task-progress/package.json +14 -0
- package/src/plugins/defaults/task-progress/src/apply-task-progress-context.ts +93 -0
- package/src/plugins/defaults/task-progress/src/format-task-progress-context.ts +67 -0
- package/src/providers/__tests__/context-overflow-error.test.ts +10 -0
- package/src/providers/__tests__/search-provider-catalog.test.ts +18 -0
- package/src/providers/anthropic/client.ts +10 -1
- package/src/providers/anthropic-gateway-shared.ts +6 -0
- package/src/providers/fetch-provider-catalog.ts +11 -0
- package/src/providers/gemini/client.ts +25 -8
- package/src/providers/inference/adapter-factory.ts +1 -1
- package/src/providers/jev/client.test.ts +10 -0
- package/src/providers/jev/client.ts +20 -4
- package/src/providers/model-catalog.ts +4 -3
- package/src/providers/openai/__tests__/chat-completions-extra-body.test.ts +94 -1
- package/src/providers/openai/__tests__/chat-completions-provider-reasoning.test.ts +106 -1
- package/src/providers/openai/__tests__/chat-template-flatten.test.ts +24 -0
- package/src/providers/openai/__tests__/google-thought-signature.test.ts +1 -2
- package/src/providers/openai/chat-completions-provider.ts +100 -29
- package/src/providers/openai/google-thought-signature.ts +0 -6
- package/src/providers/openai/responses-provider.ts +23 -1
- package/src/providers/ratelimit.ts +1 -3
- package/src/providers/search-provider-catalog.ts +24 -0
- package/src/providers/speech-to-text/__tests__/deepgram-flux-frames.test.ts +24 -0
- package/src/providers/speech-to-text/__tests__/deepgram-flux-realtime.test.ts +7 -1
- package/src/providers/speech-to-text/__tests__/vellum-managed-realtime.test.ts +50 -1
- package/src/providers/speech-to-text/deepgram-flux-frames.ts +11 -0
- package/src/providers/speech-to-text/vellum-managed-flux-realtime.ts +13 -0
- package/src/providers/types.ts +5 -0
- package/src/routes/control.ts +7 -5
- package/src/runtime/AGENTS.md +4 -2
- package/src/runtime/actor-trust-resolver.ts +3 -3
- package/src/runtime/agent-wake.ts +1 -1
- package/src/runtime/approval-message-composer.ts +3 -69
- package/src/runtime/assistant-event-hub.ts +73 -75
- package/src/runtime/assistant-event-publish-options.ts +7 -0
- package/src/runtime/assistant-event-targeting.ts +79 -0
- package/src/runtime/assistant-stream-state.ts +13 -100
- package/src/runtime/background-job-runner.ts +5 -5
- package/src/runtime/channel-retry-sweep.ts +1 -0
- package/src/runtime/guardian-action-message-composer.ts +8 -161
- package/src/runtime/guardian-reply-router.ts +28 -32
- package/src/runtime/http-types.ts +6 -0
- package/src/runtime/message-composer-types.ts +3 -61
- package/src/runtime/migrations/__tests__/vbundle-import-parity.test.ts +67 -1
- package/src/runtime/migrations/vbundle-builder.ts +24 -1
- package/src/runtime/migrations/vbundle-import-analyzer.ts +13 -4
- package/src/runtime/migrations/vbundle-import-policy.ts +10 -0
- package/src/runtime/migrations/vbundle-importer.ts +4 -1
- package/src/runtime/migrations/vbundle-streaming-importer.ts +4 -1
- package/src/runtime/migrations/vbundle-validator.ts +1 -0
- package/src/runtime/pending-interactions-mode-session.test.ts +62 -0
- package/src/runtime/pending-interactions.ts +29 -0
- package/src/runtime/routes/__tests__/client-history-routes.test.ts +104 -0
- package/src/runtime/routes/__tests__/image-generation-routes.test.ts +97 -0
- package/src/runtime/routes/__tests__/inference-profiles-routes.test.ts +1 -1
- package/src/runtime/routes/__tests__/roadmap-routes.test.ts +29 -6
- package/src/runtime/routes/browser-context.ts +31 -11
- package/src/runtime/routes/browser-routes.ts +43 -4
- package/src/runtime/routes/btw-routes.ts +9 -4
- package/src/runtime/routes/client-routes.ts +108 -0
- package/src/runtime/routes/conversation-management-routes.ts +4 -1
- package/src/runtime/routes/conversation-routes.ts +113 -27
- package/src/runtime/routes/credential-prompt-routes.ts +4 -3
- package/src/runtime/routes/credential-routes.ts +1 -2
- package/src/runtime/routes/desktop-setup-routes.test.ts +9 -18
- package/src/runtime/routes/desktop-setup-routes.ts +4 -6
- package/src/runtime/routes/documents-routes.ts +51 -0
- package/src/runtime/routes/email-routes.ts +3 -7
- package/src/runtime/routes/events-routes.ts +75 -54
- package/src/runtime/routes/identity-routes.ts +1 -5
- package/src/runtime/routes/image-generation-routes.ts +8 -14
- package/src/runtime/routes/inbound-message-handler.ts +35 -38
- package/src/runtime/routes/inbound-stages/backfill-sender-provenance.test.ts +168 -0
- package/src/runtime/routes/inbound-stages/backfill-sender-provenance.ts +96 -0
- package/src/runtime/routes/inbound-stages/background-dispatch.test.ts +38 -9
- package/src/runtime/routes/inbound-stages/background-dispatch.ts +2 -5
- package/src/runtime/routes/inbound-stages/reaction-intercept.ts +2 -0
- package/src/runtime/routes/llm-context-normalization.ts +1 -7
- package/src/runtime/routes/migration-routes.ts +73 -2
- package/src/runtime/routes/notification-routes.ts +10 -0
- package/src/runtime/routes/oauth-providers.ts +7 -0
- package/src/runtime/routes/plugins-routes.ts +6 -0
- package/src/runtime/routes/roadmap-routes.ts +47 -8
- package/src/runtime/routes/secret-routes.ts +1 -1
- package/src/runtime/routes/settings-routes.ts +10 -4
- package/src/runtime/routes/watcher-routes.ts +1 -3
- package/src/runtime/trust-class.test.ts +7 -6
- package/src/runtime/trust-class.ts +9 -17
- package/src/schedule/__tests__/worker-control.test.ts +36 -5
- package/src/schedule/worker-control.ts +15 -5
- package/src/security/secure-keys.ts +53 -103
- package/src/skills/clawhub.ts +3 -8
- package/src/skills/managed-store.ts +1 -5
- package/src/stt/types.ts +4 -0
- package/src/subagent/notify.ts +54 -34
- package/src/subagent/parent-notification.ts +6 -0
- package/src/telemetry/__tests__/phone-call-funnel.test.ts +72 -0
- package/src/telemetry/phone-call-funnel.ts +144 -0
- package/src/tools/__tests__/tool-schema-root-combinator-guard.test.ts +81 -0
- package/src/tools/ask-question/ask-question-tool.ts +6 -4
- package/src/tools/browser/browser-execution.ts +22 -59
- package/src/tools/browser/cdp-client/__tests__/cdp-dom-helpers.test.ts +561 -39
- package/src/tools/browser/cdp-client/cdp-dom-helpers.ts +435 -39
- package/src/tools/browser/cdp-client/cdp-inspect-client.ts +1 -8
- package/src/tools/client-os.ts +15 -0
- package/src/tools/computer-use/skill-proxy-bridge.ts +1 -1
- package/src/tools/computer-use/target.ts +42 -0
- package/src/tools/document/document-tool.ts +13 -18
- package/src/tools/execution-target.ts +14 -16
- package/src/tools/host-terminal/host-shell.ts +4 -1
- package/src/tools/network/__tests__/web-fetch-tinyfish.test.ts +250 -0
- package/src/tools/network/__tests__/web-search.test.ts +203 -0
- package/src/tools/network/web-fetch.ts +274 -7
- package/src/tools/network/web-search.ts +468 -6
- package/src/tools/permission-checker.ts +3 -3
- package/src/tools/policy-context.ts +3 -1
- package/src/tools/skills/load.ts +2 -1
- package/src/tools/skills/sandbox-runner.ts +11 -2
- package/src/tools/skills/skill-script-runner.ts +7 -0
- package/src/tools/skills/skill-tool-factory.ts +15 -1
- package/src/tools/terminal/__tests__/safe-env.test.ts +11 -0
- package/src/tools/terminal/safe-env.ts +4 -0
- package/src/tools/terminal/shell.ts +2 -0
- package/src/tools/tool-approval-handler.ts +5 -1
- package/src/tools/tool-types.ts +3 -3
- package/src/tools/types.ts +8 -2
- package/src/util/__tests__/worker-process-command.test.ts +5 -4
- package/src/util/abort-reasons.ts +18 -26
- package/src/util/canonicalize-identity.test.ts +21 -0
- package/src/util/canonicalize-identity.ts +16 -0
- package/src/util/errors.ts +13 -3
- package/src/util/provider-error-patterns.ts +3 -0
- package/src/util/worker-process.ts +77 -35
- package/src/workspace/migrations/157-add-desktop-files-launcher.ts +94 -0
- package/src/workspace/migrations/__tests__/156-extract-workspace-mcp-json.test.ts +4 -10
- package/src/workspace/migrations/__tests__/157-add-desktop-files-launcher.test.ts +113 -0
- package/src/workspace/migrations/registry.ts +2 -0
- package/src/__tests__/computer-use-window-schema.test.ts +0 -24
- package/src/api/events/heartbeat-alert.ts +0 -20
- package/src/daemon/message-types/web-activity.ts +0 -65
- package/src/desktop/desktop-dependencies.test.ts +0 -131
- package/src/tools/computer-use/definitions.ts +0 -607
|
@@ -7,7 +7,6 @@ import {
|
|
|
7
7
|
attachGoogleThoughtSignature,
|
|
8
8
|
attachGoogleThoughtSignatureIfNeeded,
|
|
9
9
|
backfillGoogleThoughtSignatures,
|
|
10
|
-
backfillUnsignedGoogleThoughtSignatures,
|
|
11
10
|
geminiThoughtSignaturesByToolCallId,
|
|
12
11
|
googleThoughtSignatureFromUnknown,
|
|
13
12
|
type GoogleToolCallExtraContent,
|
|
@@ -177,7 +176,7 @@ describe("thought signature params helpers", () => {
|
|
|
177
176
|
|
|
178
177
|
test("backfills the dummy signature onto unsigned tool_calls", () => {
|
|
179
178
|
const params = structuredClone(unsignedParams);
|
|
180
|
-
expect(
|
|
179
|
+
expect(backfillGoogleThoughtSignatures(params)).toBe(true);
|
|
181
180
|
expect(messagesCarryGoogleThoughtSignature(params)).toBe(true);
|
|
182
181
|
expect(
|
|
183
182
|
params.messages[0].tool_calls[0].extra_content?.google?.thought_signature,
|
|
@@ -355,6 +355,41 @@ function protectJsonSchemaToolResult(payload: string): string {
|
|
|
355
355
|
: payload;
|
|
356
356
|
}
|
|
357
357
|
|
|
358
|
+
function carriesReasoningOptOut(params: unknown): boolean {
|
|
359
|
+
const p = params as {
|
|
360
|
+
reasoning_effort?: unknown;
|
|
361
|
+
reasoning?: { effort?: unknown } | null;
|
|
362
|
+
};
|
|
363
|
+
return (
|
|
364
|
+
p.reasoning_effort === "none" ||
|
|
365
|
+
(typeof p.reasoning === "object" &&
|
|
366
|
+
p.reasoning !== null &&
|
|
367
|
+
p.reasoning.effort === "none")
|
|
368
|
+
);
|
|
369
|
+
}
|
|
370
|
+
|
|
371
|
+
function stripReasoningParams(params: unknown): void {
|
|
372
|
+
const p = params as Record<string, unknown>;
|
|
373
|
+
delete p.reasoning_effort;
|
|
374
|
+
delete p.reasoning;
|
|
375
|
+
}
|
|
376
|
+
|
|
377
|
+
/**
|
|
378
|
+
* `baseURL|model|routing` keys whose request succeeded on the attempt directly
|
|
379
|
+
* after the opt-out was stripped. Later requests skip the opt-out up front
|
|
380
|
+
* instead of paying a rejected round-trip on every call. A success that needed
|
|
381
|
+
* a further compat retry proves nothing about the opt-out, so it is not
|
|
382
|
+
* recorded. Routing (OpenRouter's `provider` body field) is part of the key
|
|
383
|
+
* because opt-out support belongs to the upstream backend, not the model slug.
|
|
384
|
+
* Process-lifetime only: a restart re-learns with one rejected request per key.
|
|
385
|
+
*/
|
|
386
|
+
const reasoningOptOutRejecters = new Set<string>();
|
|
387
|
+
|
|
388
|
+
/** Test-only: forget learned reasoning opt-out rejections. */
|
|
389
|
+
export function resetReasoningOptOutRejectersForTests(): void {
|
|
390
|
+
reasoningOptOutRejecters.clear();
|
|
391
|
+
}
|
|
392
|
+
|
|
358
393
|
/**
|
|
359
394
|
* True when the request carried an explicit reasoning opt-out (`"none"` sent
|
|
360
395
|
* as flat `reasoning_effort` or nested `reasoning.effort`) and the provider
|
|
@@ -364,16 +399,7 @@ function protectJsonSchemaToolResult(payload: string): string {
|
|
|
364
399
|
* model-default reasoning beats a hard failure.
|
|
365
400
|
*/
|
|
366
401
|
function isReasoningOptOutRejection(error: unknown, params: unknown): boolean {
|
|
367
|
-
|
|
368
|
-
reasoning_effort?: unknown;
|
|
369
|
-
reasoning?: { effort?: unknown } | null;
|
|
370
|
-
};
|
|
371
|
-
const optedOut =
|
|
372
|
-
p.reasoning_effort === "none" ||
|
|
373
|
-
(typeof p.reasoning === "object" &&
|
|
374
|
-
p.reasoning !== null &&
|
|
375
|
-
p.reasoning.effort === "none");
|
|
376
|
-
if (!optedOut) {
|
|
402
|
+
if (!carriesReasoningOptOut(params)) {
|
|
377
403
|
return false;
|
|
378
404
|
}
|
|
379
405
|
if (!isClientErrorStatus(error)) {
|
|
@@ -685,10 +711,7 @@ function classifyOpenAICompatRetry(
|
|
|
685
711
|
kind: "reasoning-opt-out",
|
|
686
712
|
message:
|
|
687
713
|
"Model rejected the explicit reasoning opt-out; retrying without reasoning params",
|
|
688
|
-
apply: () =>
|
|
689
|
-
delete params.reasoning_effort;
|
|
690
|
-
delete (params as unknown as Record<string, unknown>).reasoning;
|
|
691
|
-
},
|
|
714
|
+
apply: () => stripReasoningParams(params),
|
|
692
715
|
};
|
|
693
716
|
}
|
|
694
717
|
if (isThinkingModeToolChoiceRejection(error, params)) {
|
|
@@ -788,6 +811,24 @@ const OPENAI_SUPPORTED_IMAGE_TYPES = new Set([
|
|
|
788
811
|
"image/webp",
|
|
789
812
|
]);
|
|
790
813
|
|
|
814
|
+
/**
|
|
815
|
+
* Persistable view of the chat.completions create params. A `logit_bias`
|
|
816
|
+
* preset (e.g. `suppress-cjk`) is ~5.3k deterministic entries (~68KB);
|
|
817
|
+
* summarize it so inspector rows don't balloon. The full map still went
|
|
818
|
+
* out on the wire.
|
|
819
|
+
*/
|
|
820
|
+
function inspectableChatCompletionsRequest(
|
|
821
|
+
params: OpenAI.Chat.Completions.ChatCompletionCreateParamsStreaming,
|
|
822
|
+
): unknown {
|
|
823
|
+
if (!params.logit_bias) {
|
|
824
|
+
return params;
|
|
825
|
+
}
|
|
826
|
+
return {
|
|
827
|
+
...params,
|
|
828
|
+
logit_bias: `<${Object.keys(params.logit_bias).length} token biases omitted>`,
|
|
829
|
+
};
|
|
830
|
+
}
|
|
831
|
+
|
|
791
832
|
// Think-tag scanning primitives are shared with the TTS reasoning filter
|
|
792
833
|
// (util/think-tag-stream.ts) so the two stream parsers cannot drift. This
|
|
793
834
|
// provider keeps its exact historical behavior: case-sensitive, <think> only.
|
|
@@ -883,6 +924,7 @@ export class OpenAIChatCompletionsProvider implements Provider {
|
|
|
883
924
|
// wire, to be decoded back on the response. Empty unless
|
|
884
925
|
// `coerceObjectArgsToJsonString` is enabled.
|
|
885
926
|
const coercedObjectKeys = new Map<string, string[]>();
|
|
927
|
+
let inspectableRequest: unknown | undefined;
|
|
886
928
|
|
|
887
929
|
try {
|
|
888
930
|
const thoughtSignaturesByCallId =
|
|
@@ -1125,18 +1167,35 @@ export class OpenAIChatCompletionsProvider implements Provider {
|
|
|
1125
1167
|
if (extraBody) {
|
|
1126
1168
|
Object.assign(params, extraBody);
|
|
1127
1169
|
}
|
|
1128
|
-
const
|
|
1129
|
-
|
|
1170
|
+
const optOutKey = `${this.client.baseURL}|${params.model}|${JSON.stringify(
|
|
1171
|
+
(params as { provider?: unknown }).provider ?? null,
|
|
1172
|
+
)}`;
|
|
1173
|
+
if (
|
|
1174
|
+
reasoningOptOutRejecters.has(optOutKey) &&
|
|
1175
|
+
carriesReasoningOptOut(params)
|
|
1176
|
+
) {
|
|
1177
|
+
stripReasoningParams(params);
|
|
1178
|
+
}
|
|
1179
|
+
const createStream = () => {
|
|
1180
|
+
// Snapshot after extra-body merge and any in-place compat retries
|
|
1181
|
+
// so inspector rows match the params that actually went on the wire.
|
|
1182
|
+
inspectableRequest = inspectableChatCompletionsRequest(params);
|
|
1183
|
+
return this.client.chat.completions.create(params, {
|
|
1130
1184
|
signal: timeoutSignal,
|
|
1131
1185
|
...(Object.keys(requestHeaders).length > 0
|
|
1132
1186
|
? { headers: requestHeaders }
|
|
1133
1187
|
: {}),
|
|
1134
1188
|
});
|
|
1189
|
+
};
|
|
1135
1190
|
const attemptedCompatRetries = new Set<OpenAICompatRetryKind>();
|
|
1191
|
+
let lastCompatRetry: OpenAICompatRetryKind | undefined;
|
|
1136
1192
|
let stream: Awaited<ReturnType<typeof createStream>>;
|
|
1137
1193
|
for (;;) {
|
|
1138
1194
|
try {
|
|
1139
1195
|
stream = await createStream();
|
|
1196
|
+
if (lastCompatRetry === "reasoning-opt-out") {
|
|
1197
|
+
reasoningOptOutRejecters.add(optOutKey);
|
|
1198
|
+
}
|
|
1140
1199
|
break;
|
|
1141
1200
|
} catch (error) {
|
|
1142
1201
|
const retry = classifyOpenAICompatRetry(
|
|
@@ -1148,6 +1207,7 @@ export class OpenAIChatCompletionsProvider implements Provider {
|
|
|
1148
1207
|
throw error;
|
|
1149
1208
|
}
|
|
1150
1209
|
attemptedCompatRetries.add(retry.kind);
|
|
1210
|
+
lastCompatRetry = retry.kind;
|
|
1151
1211
|
log.warn(
|
|
1152
1212
|
{
|
|
1153
1213
|
provider: this.name,
|
|
@@ -1447,15 +1507,9 @@ export class OpenAIChatCompletionsProvider implements Provider {
|
|
|
1447
1507
|
},
|
|
1448
1508
|
stopReason: finishReason,
|
|
1449
1509
|
// `rawRequest` is persisted to the request-log DB and inspector on every
|
|
1450
|
-
// call. A `logit_bias` preset
|
|
1451
|
-
//
|
|
1452
|
-
|
|
1453
|
-
rawRequest: params.logit_bias
|
|
1454
|
-
? {
|
|
1455
|
-
...params,
|
|
1456
|
-
logit_bias: `<${Object.keys(params.logit_bias).length} token biases omitted>`,
|
|
1457
|
-
}
|
|
1458
|
-
: params,
|
|
1510
|
+
// call, including 4xx throws below. A `logit_bias` preset is summarized
|
|
1511
|
+
// so logs don't balloon. The full map still went out on the wire above.
|
|
1512
|
+
rawRequest: inspectableRequest,
|
|
1459
1513
|
rawResponse,
|
|
1460
1514
|
};
|
|
1461
1515
|
} catch (error) {
|
|
@@ -1482,6 +1536,9 @@ export class OpenAIChatCompletionsProvider implements Provider {
|
|
|
1482
1536
|
maxTokens: overflow.maxTokens,
|
|
1483
1537
|
statusCode: error.status,
|
|
1484
1538
|
cause: error,
|
|
1539
|
+
...(inspectableRequest !== undefined
|
|
1540
|
+
? { rawRequest: inspectableRequest }
|
|
1541
|
+
: {}),
|
|
1485
1542
|
});
|
|
1486
1543
|
}
|
|
1487
1544
|
if (detectVisionNotSupported(error, normalized.message)) {
|
|
@@ -1492,9 +1549,13 @@ export class OpenAIChatCompletionsProvider implements Provider {
|
|
|
1492
1549
|
error.status,
|
|
1493
1550
|
// Stamp the reason so classification is status-independent (a vision
|
|
1494
1551
|
// rejection returned as 401/403 must not read as an invalid key).
|
|
1495
|
-
|
|
1496
|
-
|
|
1497
|
-
|
|
1552
|
+
{
|
|
1553
|
+
reason: "vision_unsupported" as const,
|
|
1554
|
+
...(abortReason ? { abortReason } : {}),
|
|
1555
|
+
...(inspectableRequest !== undefined
|
|
1556
|
+
? { rawRequest: inspectableRequest }
|
|
1557
|
+
: {}),
|
|
1558
|
+
},
|
|
1498
1559
|
);
|
|
1499
1560
|
}
|
|
1500
1561
|
const retryAfterMs = extractRetryAfterMs(error.headers);
|
|
@@ -1509,6 +1570,7 @@ export class OpenAIChatCompletionsProvider implements Provider {
|
|
|
1509
1570
|
apiErrorParam?: string;
|
|
1510
1571
|
requestId?: string;
|
|
1511
1572
|
rawBody?: string;
|
|
1573
|
+
rawRequest?: unknown;
|
|
1512
1574
|
reason?: ProviderErrorReason;
|
|
1513
1575
|
} = { cause: error };
|
|
1514
1576
|
if (retryAfterMs !== undefined) {
|
|
@@ -1535,6 +1597,9 @@ export class OpenAIChatCompletionsProvider implements Provider {
|
|
|
1535
1597
|
if (normalized.reason) {
|
|
1536
1598
|
errorOptions.reason = normalized.reason;
|
|
1537
1599
|
}
|
|
1600
|
+
if (inspectableRequest !== undefined) {
|
|
1601
|
+
errorOptions.rawRequest = inspectableRequest;
|
|
1602
|
+
}
|
|
1538
1603
|
throw new ProviderError(
|
|
1539
1604
|
formattedMessage,
|
|
1540
1605
|
this.name,
|
|
@@ -1548,7 +1613,13 @@ export class OpenAIChatCompletionsProvider implements Provider {
|
|
|
1548
1613
|
}`,
|
|
1549
1614
|
this.name,
|
|
1550
1615
|
undefined,
|
|
1551
|
-
|
|
1616
|
+
{
|
|
1617
|
+
cause: error,
|
|
1618
|
+
...(abortReason ? { abortReason } : {}),
|
|
1619
|
+
...(inspectableRequest !== undefined
|
|
1620
|
+
? { rawRequest: inspectableRequest }
|
|
1621
|
+
: {}),
|
|
1622
|
+
},
|
|
1552
1623
|
);
|
|
1553
1624
|
}
|
|
1554
1625
|
}
|
|
@@ -266,12 +266,6 @@ export function backfillGoogleThoughtSignatures(
|
|
|
266
266
|
return added;
|
|
267
267
|
}
|
|
268
268
|
|
|
269
|
-
export function backfillUnsignedGoogleThoughtSignatures(
|
|
270
|
-
params: unknown,
|
|
271
|
-
): boolean {
|
|
272
|
-
return backfillGoogleThoughtSignatures(params);
|
|
273
|
-
}
|
|
274
|
-
|
|
275
269
|
const UNKNOWN_EXTRA_CONTENT_REJECTION =
|
|
276
270
|
/unknown|unexpected|unrecognized|additional propert|extra (?:field|property)|not (?:a )?valid|invalid (?:argument|parameter|field|property)/i;
|
|
277
271
|
|
|
@@ -294,6 +294,7 @@ export class OpenAIResponsesProvider implements Provider {
|
|
|
294
294
|
configObj.promptCacheKey.length > 0
|
|
295
295
|
? (configObj.promptCacheKey as string)
|
|
296
296
|
: undefined;
|
|
297
|
+
let inspectableRequest: unknown | undefined;
|
|
297
298
|
|
|
298
299
|
try {
|
|
299
300
|
const effectiveModel = modelOverride ?? this.model;
|
|
@@ -428,6 +429,7 @@ export class OpenAIResponsesProvider implements Provider {
|
|
|
428
429
|
}
|
|
429
430
|
|
|
430
431
|
Object.assign(params, this.buildExtraCreateParams(options));
|
|
432
|
+
inspectableRequest = params;
|
|
431
433
|
|
|
432
434
|
const { signal: timeoutSignal, cleanup: cleanupTimeout } =
|
|
433
435
|
createStreamTimeout(this.streamTimeoutMs, signal);
|
|
@@ -736,6 +738,9 @@ export class OpenAIResponsesProvider implements Provider {
|
|
|
736
738
|
maxTokens: overflow.maxTokens,
|
|
737
739
|
statusCode: error.status,
|
|
738
740
|
cause: error,
|
|
741
|
+
...(inspectableRequest !== undefined
|
|
742
|
+
? { rawRequest: inspectableRequest }
|
|
743
|
+
: {}),
|
|
739
744
|
});
|
|
740
745
|
}
|
|
741
746
|
const retryAfterMs = extractRetryAfterMs(error.headers);
|
|
@@ -750,6 +755,7 @@ export class OpenAIResponsesProvider implements Provider {
|
|
|
750
755
|
apiErrorParam?: string;
|
|
751
756
|
requestId?: string;
|
|
752
757
|
rawBody?: string;
|
|
758
|
+
rawRequest?: unknown;
|
|
753
759
|
reason?: ProviderErrorReason;
|
|
754
760
|
} = { cause: error };
|
|
755
761
|
if (retryAfterMs !== undefined) {
|
|
@@ -776,6 +782,9 @@ export class OpenAIResponsesProvider implements Provider {
|
|
|
776
782
|
if (normalized.reason) {
|
|
777
783
|
errorOptions.reason = normalized.reason;
|
|
778
784
|
}
|
|
785
|
+
if (inspectableRequest !== undefined) {
|
|
786
|
+
errorOptions.rawRequest = inspectableRequest;
|
|
787
|
+
}
|
|
779
788
|
throw new ProviderError(
|
|
780
789
|
formattedMessage,
|
|
781
790
|
this.name,
|
|
@@ -789,7 +798,20 @@ export class OpenAIResponsesProvider implements Provider {
|
|
|
789
798
|
}`,
|
|
790
799
|
this.name,
|
|
791
800
|
undefined,
|
|
792
|
-
abortReason
|
|
801
|
+
abortReason
|
|
802
|
+
? {
|
|
803
|
+
cause: error,
|
|
804
|
+
abortReason,
|
|
805
|
+
...(inspectableRequest !== undefined
|
|
806
|
+
? { rawRequest: inspectableRequest }
|
|
807
|
+
: {}),
|
|
808
|
+
}
|
|
809
|
+
: {
|
|
810
|
+
cause: error,
|
|
811
|
+
...(inspectableRequest !== undefined
|
|
812
|
+
? { rawRequest: inspectableRequest }
|
|
813
|
+
: {}),
|
|
814
|
+
},
|
|
793
815
|
);
|
|
794
816
|
}
|
|
795
817
|
}
|
|
@@ -62,9 +62,7 @@ export class RateLimitProvider implements Provider {
|
|
|
62
62
|
// calls from bypassing the rate limit during the async gap.
|
|
63
63
|
this.recordRequest();
|
|
64
64
|
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
return response;
|
|
65
|
+
return this.inner.sendMessage(messages, options);
|
|
68
66
|
}
|
|
69
67
|
|
|
70
68
|
private enforceRequestRate(): void {
|
|
@@ -163,6 +163,30 @@ export const SEARCH_PROVIDER_CATALOG: readonly SearchProviderCatalogEntry[] = [
|
|
|
163
163
|
supportsApiBase: true,
|
|
164
164
|
defaultApiBase: "https://api.fastcrw.com",
|
|
165
165
|
},
|
|
166
|
+
{
|
|
167
|
+
id: "searxng",
|
|
168
|
+
displayName: "SearXNG",
|
|
169
|
+
kind: "byok",
|
|
170
|
+
keyless: true,
|
|
171
|
+
apiKeyPrefix: "token (optional)",
|
|
172
|
+
envVar: "SEARXNG_API_KEY",
|
|
173
|
+
secretKey: "searxng",
|
|
174
|
+
fallbackOrder: 7,
|
|
175
|
+
privacyPolicyUrl: "https://docs.searxng.org/own-instance.html",
|
|
176
|
+
supportsApiBase: true,
|
|
177
|
+
},
|
|
178
|
+
{
|
|
179
|
+
id: "tinyfish",
|
|
180
|
+
displayName: "TinyFish",
|
|
181
|
+
kind: "byok",
|
|
182
|
+
apiKeyPrefix: "TinyFish API key...",
|
|
183
|
+
envVar: "TINYFISH_API_KEY",
|
|
184
|
+
secretKey: "tinyfish",
|
|
185
|
+
fallbackOrder: 8,
|
|
186
|
+
privacyPolicyUrl: "https://www.tinyfish.ai/privacy-policy",
|
|
187
|
+
supportsApiBase: true,
|
|
188
|
+
defaultApiBase: "https://api.search.tinyfish.ai",
|
|
189
|
+
},
|
|
166
190
|
];
|
|
167
191
|
|
|
168
192
|
/** Provider ids accepted by the web-search config schema. */
|
|
@@ -129,6 +129,30 @@ describe("parseFluxFrame", () => {
|
|
|
129
129
|
]);
|
|
130
130
|
});
|
|
131
131
|
|
|
132
|
+
test.each(["model", "timeout", "manual"])(
|
|
133
|
+
"retains the %s turn-end trigger and decoded audio position",
|
|
134
|
+
(trigger) => {
|
|
135
|
+
expect(
|
|
136
|
+
parseFluxFrame({ ...END_OF_TURN, trigger, audio_window_end: 12.5 })[1],
|
|
137
|
+
).toMatchObject({
|
|
138
|
+
type: "turn-end",
|
|
139
|
+
confidence: 0.88,
|
|
140
|
+
trigger,
|
|
141
|
+
audioWindowEndSeconds: 12.5,
|
|
142
|
+
});
|
|
143
|
+
},
|
|
144
|
+
);
|
|
145
|
+
|
|
146
|
+
test("omits unknown triggers and malformed audio positions", () => {
|
|
147
|
+
expect(
|
|
148
|
+
parseFluxFrame({
|
|
149
|
+
...END_OF_TURN,
|
|
150
|
+
trigger: "future-trigger",
|
|
151
|
+
audio_window_end: "invalid",
|
|
152
|
+
}),
|
|
153
|
+
).toEqual(parseFluxFrame(END_OF_TURN));
|
|
154
|
+
});
|
|
155
|
+
|
|
132
156
|
test("flat frames without a nested event field are accepted", () => {
|
|
133
157
|
expect(parseFluxFrame({ type: "StartOfTurn" })).toEqual([
|
|
134
158
|
{ type: "turn-start" },
|
|
@@ -334,6 +334,7 @@ describe("DeepgramFluxRealtimeTranscriber", () => {
|
|
|
334
334
|
type: "turn-end",
|
|
335
335
|
text: "what is the weather",
|
|
336
336
|
confidence: 0.91,
|
|
337
|
+
audioWindowEndSeconds: 1.5,
|
|
337
338
|
turnIndex: 0,
|
|
338
339
|
},
|
|
339
340
|
]);
|
|
@@ -382,7 +383,12 @@ describe("DeepgramFluxRealtimeTranscriber", () => {
|
|
|
382
383
|
);
|
|
383
384
|
expect(events).toEqual([
|
|
384
385
|
{ type: "final", text: "still here" },
|
|
385
|
-
{
|
|
386
|
+
{
|
|
387
|
+
type: "turn-end",
|
|
388
|
+
text: "still here",
|
|
389
|
+
turnIndex: 0,
|
|
390
|
+
audioWindowEndSeconds: 1.5,
|
|
391
|
+
},
|
|
386
392
|
]);
|
|
387
393
|
});
|
|
388
394
|
});
|
|
@@ -8,6 +8,7 @@
|
|
|
8
8
|
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
|
|
9
9
|
|
|
10
10
|
import { SttError, type SttStreamServerEvent } from "../../../stt/types.js";
|
|
11
|
+
import { VellumManagedFluxRealtimeTranscriber } from "../vellum-managed-flux-realtime.js";
|
|
11
12
|
import { VellumManagedRealtimeTranscriber } from "../vellum-managed-realtime.js";
|
|
12
13
|
import type { SpeechRelayConnection } from "../vellum-speech-relay-connection.js";
|
|
13
14
|
|
|
@@ -99,7 +100,7 @@ const CONNECTION: SpeechRelayConnection = {
|
|
|
99
100
|
|
|
100
101
|
const tick = () => new Promise<void>((r) => setTimeout(r, 0));
|
|
101
102
|
|
|
102
|
-
describe("
|
|
103
|
+
describe("Vellum managed realtime transcribers", () => {
|
|
103
104
|
let sockets: MockWebSocket[];
|
|
104
105
|
let originalWebSocket: unknown;
|
|
105
106
|
let originalFetch: unknown;
|
|
@@ -262,6 +263,54 @@ describe("VellumManagedRealtimeTranscriber", () => {
|
|
|
262
263
|
expect(new URL(sockets[1]!.url).searchParams.get("key")).toBe("vk-secret");
|
|
263
264
|
});
|
|
264
265
|
|
|
266
|
+
test("Flux turn indices stay in one sequence across relay reconnects", async () => {
|
|
267
|
+
const adapter = new VellumManagedFluxRealtimeTranscriber(CONNECTION);
|
|
268
|
+
const { events, onEvent } = collector();
|
|
269
|
+
const started = adapter.start(onEvent);
|
|
270
|
+
sockets[0]!.simulateOpen();
|
|
271
|
+
await started;
|
|
272
|
+
try {
|
|
273
|
+
for (let generation = 0; generation < 3; generation += 1) {
|
|
274
|
+
const socket = sockets[generation]!;
|
|
275
|
+
const turnIndex = generation === 0 ? 5 : 0;
|
|
276
|
+
const startFrame = JSON.stringify({
|
|
277
|
+
type: "TurnInfo",
|
|
278
|
+
event: "StartOfTurn",
|
|
279
|
+
turn_index: turnIndex,
|
|
280
|
+
});
|
|
281
|
+
socket.simulateMessage(startFrame);
|
|
282
|
+
socket.simulateMessage(startFrame);
|
|
283
|
+
socket.simulateMessage(
|
|
284
|
+
JSON.stringify({
|
|
285
|
+
type: "TurnInfo",
|
|
286
|
+
event: "EndOfTurn",
|
|
287
|
+
// A later end without its start still advances the sequence.
|
|
288
|
+
turn_index: generation === 1 ? 2 : turnIndex,
|
|
289
|
+
transcript: "hello",
|
|
290
|
+
}),
|
|
291
|
+
);
|
|
292
|
+
if (generation < 2) {
|
|
293
|
+
socket.simulateMessage(velayErrorFrame("session_duration_exceeded"));
|
|
294
|
+
socket.simulateClose(1000, "session_duration_exceeded");
|
|
295
|
+
await tick();
|
|
296
|
+
sockets[generation + 1]!.simulateOpen();
|
|
297
|
+
await tick();
|
|
298
|
+
}
|
|
299
|
+
}
|
|
300
|
+
expect(
|
|
301
|
+
events.flatMap((event) =>
|
|
302
|
+
event.type === "turn-start" || event.type === "turn-end"
|
|
303
|
+
? [event.turnIndex]
|
|
304
|
+
: [],
|
|
305
|
+
),
|
|
306
|
+
).toEqual([5, 5, 5, 6, 6, 8, 9, 9, 9]);
|
|
307
|
+
expect(events.some((event) => event.type === "closed")).toBe(false);
|
|
308
|
+
} finally {
|
|
309
|
+
adapter.stop();
|
|
310
|
+
sockets.at(-1)!.simulateClose();
|
|
311
|
+
}
|
|
312
|
+
});
|
|
313
|
+
|
|
265
314
|
test("finalize during a re-dial settles immediately", async () => {
|
|
266
315
|
const { adapter, events } = await startAdapter();
|
|
267
316
|
|
|
@@ -71,6 +71,7 @@ export interface FluxTurnFrame {
|
|
|
71
71
|
transcript?: string;
|
|
72
72
|
/** Flux's end-of-turn confidence in [0, 1]. */
|
|
73
73
|
end_of_turn_confidence?: number;
|
|
74
|
+
trigger?: "model" | "manual" | "timeout";
|
|
74
75
|
}
|
|
75
76
|
|
|
76
77
|
/**
|
|
@@ -153,6 +154,8 @@ export function parseFluxFrame(raw: unknown): SttStreamServerEvent[] {
|
|
|
153
154
|
case "EndOfTurn": {
|
|
154
155
|
const text = readTranscript(frame);
|
|
155
156
|
const confidence = readNumber(frame.end_of_turn_confidence);
|
|
157
|
+
const trigger = frame.trigger;
|
|
158
|
+
const audioWindowEndSeconds = readNumber(frame.audio_window_end);
|
|
156
159
|
return [
|
|
157
160
|
{ type: "final", text },
|
|
158
161
|
{
|
|
@@ -160,6 +163,14 @@ export function parseFluxFrame(raw: unknown): SttStreamServerEvent[] {
|
|
|
160
163
|
text,
|
|
161
164
|
...(confidence !== undefined ? { confidence } : {}),
|
|
162
165
|
...(turnIndex !== undefined ? { turnIndex } : {}),
|
|
166
|
+
...(trigger === "model" ||
|
|
167
|
+
trigger === "manual" ||
|
|
168
|
+
trigger === "timeout"
|
|
169
|
+
? { trigger }
|
|
170
|
+
: {}),
|
|
171
|
+
...(audioWindowEndSeconds !== undefined
|
|
172
|
+
? { audioWindowEndSeconds }
|
|
173
|
+
: {}),
|
|
163
174
|
},
|
|
164
175
|
];
|
|
165
176
|
}
|
|
@@ -79,6 +79,9 @@ export class VellumManagedFluxRealtimeTranscriber implements StreamingTranscribe
|
|
|
79
79
|
private redialOnClose = false;
|
|
80
80
|
private stopping = false;
|
|
81
81
|
private closedEmitted = false;
|
|
82
|
+
// Consumers see one stream across relay reconnects, with one turn sequence.
|
|
83
|
+
private turnIndexOffset = 0;
|
|
84
|
+
private latestTurnIndex = -1;
|
|
82
85
|
|
|
83
86
|
constructor(
|
|
84
87
|
connection: SpeechRelayConnection,
|
|
@@ -225,6 +228,15 @@ export class VellumManagedFluxRealtimeTranscriber implements StreamingTranscribe
|
|
|
225
228
|
return;
|
|
226
229
|
}
|
|
227
230
|
|
|
231
|
+
if (
|
|
232
|
+
(event.type === "turn-start" || event.type === "turn-end") &&
|
|
233
|
+
event.turnIndex !== undefined
|
|
234
|
+
) {
|
|
235
|
+
const turnIndex = this.turnIndexOffset + event.turnIndex;
|
|
236
|
+
this.latestTurnIndex = Math.max(this.latestTurnIndex, turnIndex);
|
|
237
|
+
this.emit({ ...event, turnIndex });
|
|
238
|
+
return;
|
|
239
|
+
}
|
|
228
240
|
this.emit(event);
|
|
229
241
|
}
|
|
230
242
|
|
|
@@ -236,6 +248,7 @@ export class VellumManagedFluxRealtimeTranscriber implements StreamingTranscribe
|
|
|
236
248
|
private beginRedial(): void {
|
|
237
249
|
log.info("Managed Flux relay session hit velay's duration cap, re-dialing");
|
|
238
250
|
this.inner = null;
|
|
251
|
+
this.turnIndexOffset = this.latestTurnIndex + 1;
|
|
239
252
|
void (async () => {
|
|
240
253
|
try {
|
|
241
254
|
const inner = await this.dial();
|
package/src/providers/types.ts
CHANGED
|
@@ -551,6 +551,8 @@ export interface ContextOverflowErrorOptions {
|
|
|
551
551
|
cause?: unknown;
|
|
552
552
|
/** Semantic reason override; defaults to `context_overflow`. */
|
|
553
553
|
reason?: ProviderErrorReason;
|
|
554
|
+
/** Inspectable SDK/wire request payload for the overflowing call. */
|
|
555
|
+
rawRequest?: unknown;
|
|
554
556
|
}
|
|
555
557
|
|
|
556
558
|
/**
|
|
@@ -581,6 +583,9 @@ export class ContextOverflowError extends ProviderError {
|
|
|
581
583
|
super(message, provider, options.statusCode ?? 400, {
|
|
582
584
|
reason: options.reason ?? "context_overflow",
|
|
583
585
|
...(options.cause !== undefined ? { cause: options.cause } : {}),
|
|
586
|
+
...(options.rawRequest !== undefined
|
|
587
|
+
? { rawRequest: options.rawRequest }
|
|
588
|
+
: {}),
|
|
584
589
|
});
|
|
585
590
|
this.name = "ContextOverflowError";
|
|
586
591
|
this.actualTokens = options.actualTokens;
|
package/src/routes/control.ts
CHANGED
|
@@ -44,13 +44,15 @@ function routeHostPidPath(): string {
|
|
|
44
44
|
return getProcPidPath(ROUTE_HOST_PROC_NAME);
|
|
45
45
|
}
|
|
46
46
|
|
|
47
|
+
const ROUTE_HOST_ENTRY = new URL("./worker.ts", import.meta.url);
|
|
48
|
+
|
|
47
49
|
/**
|
|
48
50
|
* Read the PID file and report liveness. A missing or malformed file reports
|
|
49
|
-
* not_running; a file pointing at a dead process
|
|
50
|
-
* not_running.
|
|
51
|
+
* not_running; a file pointing at a dead process, or at a live process that
|
|
52
|
+
* is not this worker, is cleaned up and reported as not_running.
|
|
51
53
|
*/
|
|
52
54
|
export function probeRouteHostWorker(): WorkerProcessStatus {
|
|
53
|
-
return probeWorkerPidFile(routeHostPidPath());
|
|
55
|
+
return probeWorkerPidFile(routeHostPidPath(), ROUTE_HOST_ENTRY, "routes");
|
|
54
56
|
}
|
|
55
57
|
|
|
56
58
|
export class RouteHostSpawnError extends WorkerProcessSpawnError {}
|
|
@@ -67,7 +69,7 @@ export async function spawnRouteHostWorkerProcess(
|
|
|
67
69
|
try {
|
|
68
70
|
return await spawnWorkerProcess({
|
|
69
71
|
pidPath: routeHostPidPath(),
|
|
70
|
-
entry:
|
|
72
|
+
entry: ROUTE_HOST_ENTRY,
|
|
71
73
|
packagedEntry: "routes",
|
|
72
74
|
workerLabel: "Route host",
|
|
73
75
|
options: opts,
|
|
@@ -86,7 +88,7 @@ export async function spawnRouteHostWorkerProcess(
|
|
|
86
88
|
* (e.g. EPERM) — a not-running host is a no-op.
|
|
87
89
|
*/
|
|
88
90
|
export function stopRouteHostWorkerProcess(): WorkerProcessStatus {
|
|
89
|
-
return stopWorkerProcess(routeHostPidPath());
|
|
91
|
+
return stopWorkerProcess(routeHostPidPath(), ROUTE_HOST_ENTRY, "routes");
|
|
90
92
|
}
|
|
91
93
|
|
|
92
94
|
/**
|
package/src/runtime/AGENTS.md
CHANGED
|
@@ -24,7 +24,7 @@ Queueing remains the fallback, not a legacy path, and every non-`released` outco
|
|
|
24
24
|
|
|
25
25
|
A repair that fell back to the queue carries its durable requirement onto the drain that runs the queued message, and `pendingInterruptRepair` is what carries it: the drain reads that flag and asks `repairInterruptedToolUseBlocks` for `requireDurable`, so a second persist failure leaves the flag armed and the message queued instead of persisting the prompt behind an unmatched durable `tool_use`. Only a steered drain settles for an in-memory repair, because it has no row of its own to persist behind it.
|
|
26
26
|
|
|
27
|
-
**The
|
|
27
|
+
**The two interrupt texts are split by job, and the behavior instruction lives in exactly one of them.** A synthetic `tool_result` (`PREEMPTED_TOOL_RESULT_TEXT`) states the fact about the one call it answers: the call stopped early and may have taken effect anyway. It is repeated once per stopped call in a parallel batch, so it holds no instruction. Every completed handover arms `pendingInterruptNote`, whether or not a tool call was stopped. The persist of the interrupting user message consumes that flag exactly once: it stamps `interruptedPriorTurn` on the row and appends `INTERRUPTED_TURN_NOTE_TEXT` to the message's LLM-facing content only, so the persisted row stays what the user typed and no client renders the note. That note is the single place the behavior after an interrupt is written, it leads with the short acknowledgement the user is waiting for, and it sits after the user's words instead of before them. `reinjectInterruptTurnNote` rebuilds the identical block at history load, next to the attachment annotations and for the same reason: a reload that rebuilds it differently breaks provider prefix-cache parity. Do not add a third annotation, and do not put instructions back in the tool result.
|
|
28
28
|
|
|
29
29
|
Two ordering rules a change here must not break. The synthetic `tool_result` is persisted BEFORE the caller writes the interrupting user row, because a user message between a `tool_use` and its result is a sequence every provider rejects. And the `thinking` / `message_interrupted` activity state is emitted by the agent loop at the head of the replacement turn, never at the handover: activity states are cached in `lastActivityStateMsg` and replayed to reconnecting clients, so a send that fails before its turn starts would otherwise strand every client on a busy conversation that is running nothing.
|
|
30
30
|
|
|
@@ -163,7 +163,7 @@ In the CDP factory the bridge is the internal `"host-bridge"` candidate kind (`I
|
|
|
163
163
|
|
|
164
164
|
### Browser CLI surface defaults
|
|
165
165
|
|
|
166
|
-
The browser execute and tab routes share `browser/virtual-desktop-target.ts`. Platform-hosted web guardian conversations use
|
|
166
|
+
The browser execute and tab routes share `browser/virtual-desktop-target.ts`. Platform-hosted web guardian conversations use image-provided virtual desktop Chrome by default when enabled. The shared `isVirtualDesktopEnabled` gate requires both `IS_PLATFORM` and `IS_CONTAINERIZED` alongside the `assistant-desktop` flag. Native renderer turns also use the `web` transport, so the frozen turn `clientOs` excludes native apps from automatic streamed-browser selection. Native apps retain their existing backend selection and fallback behavior. Explicit desktop/backend/client targets and existing personal-browser sessions override the surface default. Disabled desktop support retains the existing browser path. Missing image components fail the managed operation without installing dependencies or falling back to a personal browser.
|
|
167
167
|
|
|
168
168
|
### Per-tool `browser_mode` override
|
|
169
169
|
|
|
@@ -277,6 +277,8 @@ Verification SESSION state (sessions, secrets, rate limits, validate+consume) AN
|
|
|
277
277
|
|
|
278
278
|
The daemon relays session lifecycle operations over the `verification_sessions_*` IPC routes via `assistant/src/channels/gateway-verification-sessions.ts` and keeps what is presentation: message composition and channel delivery (`channel-verification-routes.ts`, `verification-outbound-actions.ts`). `channel-verification-service.ts` retains only guardian-delivery reads (`getGuardianBinding`, `isGuardian`, `isGuardianBoundForChannel`).
|
|
279
279
|
|
|
280
|
+
The gateway answers a code (or an invite) it consumed at ingress with a reply it composes itself, and delivers it through the daemon's IPC-only `deliver_gateway_reply` method (`ipc/routes/channel-reply-ipc-routes.ts`), which hands it to the channel transport the inbound message's callback URL names. The gateway never sends to a provider itself: the transports are the one outbound path per channel, whoever composed the text. The reply is not recorded in a conversation, since the message it answers never entered one.
|
|
281
|
+
|
|
280
282
|
The verified outcome is written in-process by the gateway: the HTTP guardian-attest handler calls `ContactStore.markChannelVerified` directly (verifiedVia "manual"); the code-match paths (text and the `verification_sessions_validate_consume` engine route) apply role side effects in-engine — guardian phone binding commits in the same gateway transaction as the consume. The revoke/downgrade outcome is relayed from the daemon via `ipcCallPersistent("mark_channel_revoked", …)` to `ContactStore.markChannelRevoked`.
|
|
281
283
|
|
|
282
284
|
## Rate Limiting & Diagnostics
|
|
@@ -42,9 +42,9 @@ const log = getLogger("actor-trust-resolver");
|
|
|
42
42
|
// ---------------------------------------------------------------------------
|
|
43
43
|
|
|
44
44
|
/**
|
|
45
|
-
* Trust classification for an inbound actor.
|
|
46
|
-
*
|
|
47
|
-
*
|
|
45
|
+
* Trust classification for an inbound actor. Declared in
|
|
46
|
+
* `@vellumai/gateway-client` and re-exported here for the resolver's
|
|
47
|
+
* consumers.
|
|
48
48
|
*/
|
|
49
49
|
export type { TrustClass };
|
|
50
50
|
|