@agent-native/core 0.98.0 → 0.98.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/corpus/README.md +1 -1
- package/corpus/core/CHANGELOG.md +24 -0
- package/corpus/core/package.json +2 -2
- package/corpus/core/src/action-change-marker.ts +8 -1
- package/corpus/core/src/changelog/parse.ts +8 -3
- package/corpus/core/src/client/AgentPanel.tsx +2 -3
- package/corpus/core/src/client/AssistantChat.tsx +83 -39
- package/corpus/core/src/client/analytics.ts +29 -0
- package/corpus/core/src/client/blocks/library/wireframe-kit.tsx +16 -2
- package/corpus/core/src/client/chat-model-groups.ts +28 -0
- package/corpus/core/src/client/composer/RealtimeVoiceMode.tsx +442 -85
- package/corpus/core/src/client/composer/TiptapComposer.tsx +46 -29
- package/corpus/core/src/client/composer/VoiceButton.tsx +2 -1
- package/corpus/core/src/client/composer/index.ts +5 -0
- package/corpus/core/src/client/composer/useRealtimeVoiceMode.tsx +1206 -67
- package/corpus/core/src/client/index.ts +5 -0
- package/corpus/core/src/client/progress/RunsTray.tsx +13 -1
- package/corpus/core/src/client/settings/SecretsSection.tsx +387 -266
- package/corpus/core/src/client/settings/SettingsPanel.tsx +117 -38
- package/corpus/core/src/client/use-action.ts +12 -0
- package/corpus/core/src/client/use-agent-chat-home-handoff.ts +8 -1
- package/corpus/core/src/client/use-agent-engine-configured.ts +17 -3
- package/corpus/core/src/client/use-db-sync.ts +84 -34
- package/corpus/core/src/client/use-pausing-interval.ts +14 -4
- package/corpus/core/src/integrations/scope-store.ts +1 -1
- package/corpus/core/src/integrations/usage-budget-store.ts +1 -1
- package/corpus/core/src/localization/default-messages.ts +44 -1
- package/corpus/core/src/mcp-client/workspace-servers.ts +4 -4
- package/corpus/core/src/secrets/crypto.ts +28 -8
- package/corpus/core/src/server/action-change.ts +5 -1
- package/corpus/core/src/server/action-routes.ts +8 -0
- package/corpus/core/src/server/poll.ts +1 -0
- package/corpus/core/src/server/realtime-voice.ts +299 -13
- package/corpus/core/src/styles/agent-native.css +2 -2
- package/corpus/core/src/templates/default/.agents/skills/real-time-sync/SKILL.md +2 -2
- package/corpus/core/src/templates/workspace-core/.agents/skills/real-time-sync/SKILL.md +2 -2
- package/corpus/core/src/templates/workspace-core/.agents/skills/secrets/SKILL.md +7 -3
- package/corpus/core/src/templates/workspace-core/.agents/skills/voice-transcription/SKILL.md +39 -9
- package/corpus/core/src/vite/client.ts +9 -1
- package/corpus/templates/analytics/.agents/skills/real-time-sync/SKILL.md +2 -2
- package/corpus/templates/analytics/.agents/skills/session-replay/SKILL.md +10 -5
- package/corpus/templates/analytics/AGENTS.md +9 -13
- package/corpus/templates/analytics/actions/navigate.ts +59 -5
- package/corpus/templates/analytics/actions/view-screen.ts +3 -6
- package/corpus/templates/analytics/app/components/ProviderCorpusJobNotifier.tsx +20 -3
- package/corpus/templates/analytics/app/components/dashboard/SqlChart.tsx +2 -1
- package/corpus/templates/analytics/app/components/layout/Header.tsx +1 -1
- package/corpus/templates/analytics/app/components/layout/Sidebar.tsx +21 -6
- package/corpus/templates/analytics/app/global.css +19 -20
- package/corpus/templates/analytics/app/lib/provider-corpus-job-sync.ts +26 -0
- package/corpus/templates/analytics/app/pages/Settings.tsx +35 -341
- package/corpus/templates/analytics/app/pages/monitoring/ErrorsPanel.tsx +38 -7
- package/corpus/templates/analytics/app/pages/monitoring/MonitoringPage.tsx +1 -1
- package/corpus/templates/analytics/app/pages/monitoring/errors/i18n.ts +3 -0
- package/corpus/templates/analytics/app/pages/sessions/SessionDetailPage.tsx +333 -272
- package/corpus/templates/analytics/app/pages/sessions/SessionDevToolsPanel.tsx +72 -17
- package/corpus/templates/analytics/app/pages/sessions/SessionsPage.tsx +297 -108
- package/corpus/templates/analytics/app/root.tsx +28 -8
- package/corpus/templates/analytics/changelog/2026-06-24-first-party-signup-and-pageview-panels-now-honor-dashboard-filters.md +2 -1
- package/corpus/templates/analytics/changelog/2026-06-25-referrer-dashboard-panels-show-external-domains.md +2 -1
- package/corpus/templates/analytics/changelog/2026-07-06-mobile-chat-header-is-more-compact-and-the-menu-button-opens-navigation.md +1 -0
- package/corpus/templates/analytics/changelog/2026-07-08-dashboard-history-and-chat-context.md +1 -0
- package/corpus/templates/analytics/changelog/2026-07-08-settings-are-cleaner-searchable-and-alerts-have-their-own-tab.md +1 -0
- package/corpus/templates/analytics/changelog/2026-07-08-sidebar-sections-stay-collapsed-until-opened.md +1 -0
- package/corpus/templates/analytics/changelog/2026-07-11-analytics-no-longer-repeatedly-restarts-background-requests-.md +6 -0
- package/corpus/templates/analytics/changelog/2026-07-11-analytics-only-asks-for-replay-storage-when-you-enable-sessi.md +6 -0
- package/corpus/templates/analytics/changelog/2026-07-11-charts-now-show-their-loading-skeleton-while-refreshing.md +6 -0
- package/corpus/templates/analytics/changelog/2026-07-11-realtime-voice-now-shows-a-clear-connection-indicator-comple.md +6 -0
- package/corpus/templates/analytics/changelog/2026-07-11-replay-event-counts-stay-hidden-until-recordings-finish-load.md +6 -0
- package/corpus/templates/analytics/changelog/2026-07-11-replay-storage-key-mismatches-show-actionable-setup-guidance.md +6 -0
- package/corpus/templates/analytics/changelog/2026-07-11-replay-timelines-group-continuous-scrolling-and-dev-tools-co.md +6 -0
- package/corpus/templates/analytics/changelog/2026-07-11-session-filters-now-stay-compact-in-a-settings-menu-at-mediu.md +6 -0
- package/corpus/templates/analytics/changelog/2026-07-11-session-replay-console-errors-now-link-to-matching-issues-or.md +6 -0
- package/corpus/templates/analytics/changelog/2026-07-11-session-replays-load-large-recordings-faster-and-surface-fai.md +6 -0
- package/corpus/templates/analytics/changelog/2026-07-11-session-replays-preserve-recorded-styles-and-viewport-layout.md +6 -0
- package/corpus/templates/analytics/changelog/2026-07-11-session-replays-stay-centered-instead-of-clipping-wide-recor.md +6 -0
- package/corpus/templates/analytics/package.json +1 -1
- package/corpus/templates/analytics/seeds/dashboards/agent-native-templates-first-party.json +1 -303
- package/corpus/templates/analytics/server/handlers/session-replay.ts +57 -0
- package/corpus/templates/analytics/server/lib/dashboard-catalog.ts +1 -1
- package/corpus/templates/analytics/server/lib/demo-dashboards.ts +16 -3
- package/corpus/templates/analytics/server/lib/session-replay-agent-context.ts +12 -0
- package/corpus/templates/analytics/server/lib/session-replay.ts +208 -2
- package/corpus/templates/analytics/server/plugins/onboarding.ts +7 -79
- package/corpus/templates/analytics/server/routes/api/session-replay/recordings/[recordingId]/chunks.get.ts +1 -0
- package/corpus/templates/assets/.agents/skills/real-time-sync/SKILL.md +2 -2
- package/corpus/templates/assets/app/global.css +4 -4
- package/corpus/templates/assets/app/routes/brand-kits.$id.tsx +0 -1
- package/corpus/templates/assets/app/routes/library.tsx +0 -1
- package/corpus/templates/brain/.agents/skills/real-time-sync/SKILL.md +2 -2
- package/corpus/templates/brain/app/global.css +5 -5
- package/corpus/templates/brain/app/root.tsx +0 -3
- package/corpus/templates/calendar/.agents/skills/real-time-sync/SKILL.md +2 -2
- package/corpus/templates/calendar/app/root.tsx +1 -0
- package/corpus/templates/chat/.agents/skills/real-time-sync/SKILL.md +2 -2
- package/corpus/templates/clips/app/hooks/use-auto-title.ts +74 -8
- package/corpus/templates/clips/app/hooks/use-library.ts +12 -25
- package/corpus/templates/clips/app/routes/_app.settings._index.tsx +20 -22
- package/corpus/templates/clips/changelog/2026-07-11-generated-titles-no-longer-cause-continuous-background-refreshes.md +6 -0
- package/corpus/templates/clips/changelog/2026-07-11-s3-storage-setup-stays-collapsed-until-you-choose-it-as-the-.md +6 -0
- package/corpus/templates/content/.agents/skills/real-time-sync/SKILL.md +2 -2
- package/corpus/templates/content/app/hooks/use-comments.ts +0 -1
- package/corpus/templates/content/changelog/2026-07-11-comments-now-update-from-live-collaboration-events-without-c.md +6 -0
- package/corpus/templates/design/app/components/layout/Header.tsx +1 -1
- package/corpus/templates/forms/.agents/skills/real-time-sync/SKILL.md +2 -2
- package/corpus/templates/macros/app/components/layout/AppLayout.tsx +1 -2
- package/corpus/templates/macros/app/root.tsx +0 -3
- package/corpus/templates/mail/.agents/skills/real-time-sync/SKILL.md +2 -2
- package/corpus/templates/mail/app/hooks/use-compose-state.ts +1 -0
- package/corpus/templates/mail/app/root.tsx +1 -12
- package/corpus/templates/plan/.agents/skills/real-time-sync/SKILL.md +2 -2
- package/corpus/templates/plan/app/hooks/use-navigation-state.ts +5 -2
- package/corpus/templates/plan/app/hooks/use-plans.ts +6 -13
- package/corpus/templates/plan/app/pages/PlansPage.tsx +5 -27
- package/corpus/templates/plan/changelog/2026-07-11-plan-refreshes-only-when-content-changes.md +6 -0
- package/corpus/templates/slides/app/components/comments/SlideCommentsPanel.tsx +4 -1
- package/corpus/templates/slides/app/components/editor/EditorToolbar.tsx +1 -1
- package/corpus/templates/slides/app/components/layout/Header.tsx +1 -1
- package/corpus/templates/slides/app/context/DeckContext.tsx +1 -0
- package/corpus/templates/slides/app/hooks/use-slide-comments.ts +7 -31
- package/corpus/templates/slides/changelog/2026-07-11-comments-now-update-from-live-collaboration-events-without-c.md +6 -0
- package/dist/action-change-marker.d.ts +1 -0
- package/dist/action-change-marker.d.ts.map +1 -1
- package/dist/action-change-marker.js +7 -1
- package/dist/action-change-marker.js.map +1 -1
- package/dist/changelog/parse.d.ts +4 -2
- package/dist/changelog/parse.d.ts.map +1 -1
- package/dist/changelog/parse.js +5 -3
- package/dist/changelog/parse.js.map +1 -1
- package/dist/client/AgentPanel.d.ts.map +1 -1
- package/dist/client/AgentPanel.js +2 -3
- package/dist/client/AgentPanel.js.map +1 -1
- package/dist/client/AssistantChat.d.ts +7 -0
- package/dist/client/AssistantChat.d.ts.map +1 -1
- package/dist/client/AssistantChat.js +62 -40
- package/dist/client/AssistantChat.js.map +1 -1
- package/dist/client/analytics.d.ts.map +1 -1
- package/dist/client/analytics.js +26 -0
- package/dist/client/analytics.js.map +1 -1
- package/dist/client/blocks/library/wireframe-kit.d.ts +2 -0
- package/dist/client/blocks/library/wireframe-kit.d.ts.map +1 -1
- package/dist/client/blocks/library/wireframe-kit.js +9 -2
- package/dist/client/blocks/library/wireframe-kit.js.map +1 -1
- package/dist/client/chat-model-groups.d.ts.map +1 -1
- package/dist/client/chat-model-groups.js +24 -0
- package/dist/client/chat-model-groups.js.map +1 -1
- package/dist/client/composer/RealtimeVoiceMode.d.ts +40 -4
- package/dist/client/composer/RealtimeVoiceMode.d.ts.map +1 -1
- package/dist/client/composer/RealtimeVoiceMode.js +172 -42
- package/dist/client/composer/RealtimeVoiceMode.js.map +1 -1
- package/dist/client/composer/TiptapComposer.d.ts +4 -0
- package/dist/client/composer/TiptapComposer.d.ts.map +1 -1
- package/dist/client/composer/TiptapComposer.js +47 -29
- package/dist/client/composer/TiptapComposer.js.map +1 -1
- package/dist/client/composer/VoiceButton.d.ts.map +1 -1
- package/dist/client/composer/VoiceButton.js +2 -2
- package/dist/client/composer/VoiceButton.js.map +1 -1
- package/dist/client/composer/index.d.ts +2 -2
- package/dist/client/composer/index.d.ts.map +1 -1
- package/dist/client/composer/index.js +1 -1
- package/dist/client/composer/index.js.map +1 -1
- package/dist/client/composer/useRealtimeVoiceMode.d.ts +100 -3
- package/dist/client/composer/useRealtimeVoiceMode.d.ts.map +1 -1
- package/dist/client/composer/useRealtimeVoiceMode.js +952 -51
- package/dist/client/composer/useRealtimeVoiceMode.js.map +1 -1
- package/dist/client/index.d.ts +1 -1
- package/dist/client/index.d.ts.map +1 -1
- package/dist/client/index.js +1 -1
- package/dist/client/index.js.map +1 -1
- package/dist/client/progress/RunsTray.d.ts.map +1 -1
- package/dist/client/progress/RunsTray.js +10 -1
- package/dist/client/progress/RunsTray.js.map +1 -1
- package/dist/client/settings/SecretsSection.d.ts +2 -3
- package/dist/client/settings/SecretsSection.d.ts.map +1 -1
- package/dist/client/settings/SecretsSection.js +46 -22
- package/dist/client/settings/SecretsSection.js.map +1 -1
- package/dist/client/settings/SettingsPanel.d.ts +9 -0
- package/dist/client/settings/SettingsPanel.d.ts.map +1 -1
- package/dist/client/settings/SettingsPanel.js +46 -12
- package/dist/client/settings/SettingsPanel.js.map +1 -1
- package/dist/client/use-action.d.ts +2 -0
- package/dist/client/use-action.d.ts.map +1 -1
- package/dist/client/use-action.js +10 -0
- package/dist/client/use-action.js.map +1 -1
- package/dist/client/use-agent-chat-home-handoff.d.ts +2 -0
- package/dist/client/use-agent-chat-home-handoff.d.ts.map +1 -1
- package/dist/client/use-agent-chat-home-handoff.js +7 -1
- package/dist/client/use-agent-chat-home-handoff.js.map +1 -1
- package/dist/client/use-agent-engine-configured.d.ts.map +1 -1
- package/dist/client/use-agent-engine-configured.js +17 -3
- package/dist/client/use-agent-engine-configured.js.map +1 -1
- package/dist/client/use-db-sync.d.ts +8 -2
- package/dist/client/use-db-sync.d.ts.map +1 -1
- package/dist/client/use-db-sync.js +57 -32
- package/dist/client/use-db-sync.js.map +1 -1
- package/dist/client/use-pausing-interval.d.ts.map +1 -1
- package/dist/client/use-pausing-interval.js +16 -4
- package/dist/client/use-pausing-interval.js.map +1 -1
- package/dist/integrations/scope-store.js +1 -1
- package/dist/integrations/scope-store.js.map +1 -1
- package/dist/integrations/usage-budget-store.js +1 -1
- package/dist/integrations/usage-budget-store.js.map +1 -1
- package/dist/localization/default-messages.d.ts +41 -1
- package/dist/localization/default-messages.d.ts.map +1 -1
- package/dist/localization/default-messages.js +41 -1
- package/dist/localization/default-messages.js.map +1 -1
- package/dist/mcp-client/workspace-servers.js +4 -4
- package/dist/mcp-client/workspace-servers.js.map +1 -1
- package/dist/observability/routes.d.ts +5 -5
- package/dist/provider-api/corpus-jobs.d.ts +2 -2
- package/dist/secrets/crypto.d.ts +7 -6
- package/dist/secrets/crypto.d.ts.map +1 -1
- package/dist/secrets/crypto.js +30 -11
- package/dist/secrets/crypto.js.map +1 -1
- package/dist/secrets/routes.d.ts +9 -9
- package/dist/server/action-change.d.ts +1 -0
- package/dist/server/action-change.d.ts.map +1 -1
- package/dist/server/action-change.js +4 -1
- package/dist/server/action-change.js.map +1 -1
- package/dist/server/action-routes.d.ts.map +1 -1
- package/dist/server/action-routes.js +5 -0
- package/dist/server/action-routes.js.map +1 -1
- package/dist/server/agent-engine-api-key-route.d.ts +1 -1
- package/dist/server/poll.d.ts.map +1 -1
- package/dist/server/poll.js +1 -0
- package/dist/server/poll.js.map +1 -1
- package/dist/server/realtime-voice.d.ts +7 -0
- package/dist/server/realtime-voice.d.ts.map +1 -1
- package/dist/server/realtime-voice.js +222 -12
- package/dist/server/realtime-voice.js.map +1 -1
- package/dist/styles/agent-native.css +2 -2
- package/dist/templates/default/.agents/skills/real-time-sync/SKILL.md +2 -2
- package/dist/templates/workspace-core/.agents/skills/real-time-sync/SKILL.md +2 -2
- package/dist/templates/workspace-core/.agents/skills/secrets/SKILL.md +7 -3
- package/dist/templates/workspace-core/.agents/skills/voice-transcription/SKILL.md +39 -9
- package/dist/vite/client.d.ts.map +1 -1
- package/dist/vite/client.js +6 -1
- package/dist/vite/client.js.map +1 -1
- package/package.json +2 -2
- package/src/templates/default/.agents/skills/real-time-sync/SKILL.md +2 -2
- package/src/templates/workspace-core/.agents/skills/real-time-sync/SKILL.md +2 -2
- package/src/templates/workspace-core/.agents/skills/secrets/SKILL.md +7 -3
- package/src/templates/workspace-core/.agents/skills/voice-transcription/SKILL.md +39 -9
|
@@ -5,12 +5,13 @@
|
|
|
5
5
|
* credentials (`resolveCredential` / `saveCredential`, stored in `settings`) so
|
|
6
6
|
* there is a single crypto implementation and a single key story.
|
|
7
7
|
*
|
|
8
|
-
* The encryption key is derived from `
|
|
9
|
-
*
|
|
10
|
-
*
|
|
11
|
-
*
|
|
12
|
-
*
|
|
13
|
-
*
|
|
8
|
+
* The encryption key is derived from `<APP_NAME>_SECRETS_ENCRYPTION_KEY` when
|
|
9
|
+
* set, then `SECRETS_ENCRYPTION_KEY`, then `BETTER_AUTH_SECRET`. The app-scoped
|
|
10
|
+
* key lets a local multi-app workspace read one app's encrypted production data
|
|
11
|
+
* without replacing the shared local auth secret. In production we refuse to
|
|
12
|
+
* start without configured key material — a CWD-derived fallback would be
|
|
13
|
+
* effectively static (e.g. `/var/task` on Lambda), so anyone with read access
|
|
14
|
+
* to the DB could decrypt every secret.
|
|
14
15
|
*
|
|
15
16
|
* Encrypted values are tagged `v1:<iv-hex>:<ct-hex>:<tag-hex>`. The `v1:` prefix
|
|
16
17
|
* lets readers distinguish ciphertext from legacy plaintext during migration.
|
|
@@ -55,6 +56,17 @@ function processCwd(): string {
|
|
|
55
56
|
return process.cwd();
|
|
56
57
|
}
|
|
57
58
|
|
|
59
|
+
function appScopedEncryptionKey(): string | undefined {
|
|
60
|
+
if (typeof process === "undefined") return undefined;
|
|
61
|
+
const appName = process.env.APP_NAME?.trim() // guard:allow-env-credential — deploy-level app configuration selects the scoped encryption key.
|
|
62
|
+
.toUpperCase()
|
|
63
|
+
.replace(/[^A-Z0-9]+/g, "_")
|
|
64
|
+
.replace(/^_+|_+$/g, "");
|
|
65
|
+
return appName
|
|
66
|
+
? process.env[`${appName}_SECRETS_ENCRYPTION_KEY`] // guard:allow-env-credential — deploy-level app encryption material, never a user credential.
|
|
67
|
+
: undefined;
|
|
68
|
+
}
|
|
69
|
+
|
|
58
70
|
/**
|
|
59
71
|
* Derive a 32-byte AES key from the configured secret material via SHA-256.
|
|
60
72
|
* Re-derived per call (cheap, stateless, and makes rotation easy).
|
|
@@ -62,6 +74,7 @@ function processCwd(): string {
|
|
|
62
74
|
export function getSecretEncryptionKey(): Buffer {
|
|
63
75
|
const { createHash } = requireNodeCrypto();
|
|
64
76
|
const explicit =
|
|
77
|
+
appScopedEncryptionKey() ||
|
|
65
78
|
(typeof process === "undefined"
|
|
66
79
|
? undefined
|
|
67
80
|
: process.env.SECRETS_ENCRYPTION_KEY) ||
|
|
@@ -71,9 +84,16 @@ export function getSecretEncryptionKey(): Buffer {
|
|
|
71
84
|
|
|
72
85
|
if (!explicit) {
|
|
73
86
|
if (processNodeEnv() === "production") {
|
|
87
|
+
const appName =
|
|
88
|
+
typeof process === "undefined"
|
|
89
|
+
? undefined
|
|
90
|
+
: process.env.APP_NAME?.trim() // guard:allow-env-credential — deploy-level app configuration selects the scoped encryption key.
|
|
91
|
+
.toUpperCase()
|
|
92
|
+
.replace(/[^A-Z0-9]+/g, "_")
|
|
93
|
+
.replace(/^_+|_+$/g, "");
|
|
74
94
|
throw new Error(
|
|
75
95
|
"[agent-native/secrets] Refusing to start in production without an encryption key. " +
|
|
76
|
-
|
|
96
|
+
`Set ${appName ? `${appName}_SECRETS_ENCRYPTION_KEY, ` : ""}SECRETS_ENCRYPTION_KEY, or BETTER_AUTH_SECRET in the deploy environment. ` +
|
|
77
97
|
"The previous CWD-derived fallback was effectively static (e.g. `/var/task` on Lambda), " +
|
|
78
98
|
"which means anyone with read access to the secrets table could decrypt every user's secrets.",
|
|
79
99
|
);
|
|
@@ -83,7 +103,7 @@ export function getSecretEncryptionKey(): Buffer {
|
|
|
83
103
|
// eslint-disable-next-line no-console
|
|
84
104
|
console.warn(
|
|
85
105
|
"[agent-native/secrets] SECRETS_ENCRYPTION_KEY not set — using a machine-local fallback. " +
|
|
86
|
-
"Set SECRETS_ENCRYPTION_KEY
|
|
106
|
+
"Set an app-scoped *_SECRETS_ENCRYPTION_KEY, SECRETS_ENCRYPTION_KEY, or BETTER_AUTH_SECRET for production. " +
|
|
87
107
|
"Production deploys without one of these env vars now hard-fail.",
|
|
88
108
|
);
|
|
89
109
|
}
|
|
@@ -12,6 +12,7 @@ export interface NotifyActionChangeOptions {
|
|
|
12
12
|
actionName: string;
|
|
13
13
|
owner?: string;
|
|
14
14
|
orgId?: string;
|
|
15
|
+
requestSource?: string;
|
|
15
16
|
}
|
|
16
17
|
|
|
17
18
|
function actionChangeTarget(
|
|
@@ -22,6 +23,7 @@ function actionChangeTarget(
|
|
|
22
23
|
actionName: options.actionName,
|
|
23
24
|
owner,
|
|
24
25
|
orgId: owner ? undefined : (options.orgId ?? getRequestOrgId()),
|
|
26
|
+
requestSource: options.requestSource,
|
|
25
27
|
};
|
|
26
28
|
}
|
|
27
29
|
|
|
@@ -38,7 +40,7 @@ export async function writeActionChangeMarker(
|
|
|
38
40
|
...actionChangeMarkerValue(target),
|
|
39
41
|
nonce: `${Date.now()}-${Math.random().toString(36).slice(2)}`,
|
|
40
42
|
},
|
|
41
|
-
{ requestSource: "agent" },
|
|
43
|
+
{ requestSource: options.requestSource ?? "agent" },
|
|
42
44
|
);
|
|
43
45
|
}
|
|
44
46
|
|
|
@@ -52,11 +54,13 @@ export async function notifyActionChange(
|
|
|
52
54
|
key: options.actionName,
|
|
53
55
|
...(target.owner ? { owner: target.owner } : {}),
|
|
54
56
|
...(target.orgId ? { orgId: target.orgId } : {}),
|
|
57
|
+
...(options.requestSource ? { requestSource: options.requestSource } : {}),
|
|
55
58
|
});
|
|
56
59
|
|
|
57
60
|
await writeActionChangeMarker({
|
|
58
61
|
actionName: options.actionName,
|
|
59
62
|
...(target.owner ? { owner: target.owner } : {}),
|
|
60
63
|
...(target.orgId ? { orgId: target.orgId } : {}),
|
|
64
|
+
...(options.requestSource ? { requestSource: options.requestSource } : {}),
|
|
61
65
|
});
|
|
62
66
|
}
|
|
@@ -371,6 +371,14 @@ export function mountActionRoutes(
|
|
|
371
371
|
await notifyActionChange({
|
|
372
372
|
actionName: name,
|
|
373
373
|
...(userEmail ? { owner: userEmail } : {}),
|
|
374
|
+
...(getHeader(event, "x-request-source")
|
|
375
|
+
? {
|
|
376
|
+
requestSource: getHeader(
|
|
377
|
+
event,
|
|
378
|
+
"x-request-source",
|
|
379
|
+
) as string,
|
|
380
|
+
}
|
|
381
|
+
: {}),
|
|
374
382
|
});
|
|
375
383
|
} catch {
|
|
376
384
|
// ignore
|
|
@@ -604,6 +604,7 @@ function recordActionChanges(targets: ActionChangeTarget[]): void {
|
|
|
604
604
|
key: target.actionName ?? "*",
|
|
605
605
|
...(target.owner ? { owner: target.owner } : {}),
|
|
606
606
|
...(target.orgId ? { orgId: target.orgId } : {}),
|
|
607
|
+
...(target.requestSource ? { requestSource: target.requestSource } : {}),
|
|
607
608
|
});
|
|
608
609
|
}
|
|
609
610
|
}
|
|
@@ -18,6 +18,8 @@ import {
|
|
|
18
18
|
sanitizeToolErrorText,
|
|
19
19
|
sanitizeToolErrorValue,
|
|
20
20
|
} from "../agent/tool-error-redaction.js";
|
|
21
|
+
import { TOOL_SEARCH_ACTION_NAME } from "../agent/tool-search.js";
|
|
22
|
+
import { parseAcceptLanguage } from "../localization/server.js";
|
|
21
23
|
import { getSession } from "./auth.js";
|
|
22
24
|
import {
|
|
23
25
|
getBuilderGatewayBaseUrl,
|
|
@@ -37,6 +39,10 @@ export const REALTIME_VOICE_MAX_TOOL_OUTPUT_CHARS = 16_000;
|
|
|
37
39
|
export const REALTIME_VOICE_MAX_TOOLS = 32;
|
|
38
40
|
export const REALTIME_VOICE_MAX_TOOL_SCHEMA_BYTES = 32_000;
|
|
39
41
|
export const REALTIME_VOICE_MAX_SESSION_BYTES = 64_000;
|
|
42
|
+
export const REALTIME_VOICE_TOOL_GRANT_TTL_MS = 10 * 60 * 1_000;
|
|
43
|
+
export const REALTIME_VOICE_MAX_TOOL_GRANT_SESSIONS = 256;
|
|
44
|
+
export const REALTIME_VOICE_CAPABILITY_HEADER =
|
|
45
|
+
"X-Agent-Native-Realtime-Capability";
|
|
40
46
|
|
|
41
47
|
const OPENAI_REALTIME_CALLS_URL = "https://api.openai.com/v1/realtime/calls";
|
|
42
48
|
const DEFAULT_MODEL = "gpt-realtime-2.1";
|
|
@@ -50,6 +56,37 @@ const REALTIME_TOOL_NAME = /^[A-Za-z0-9_-]{1,64}$/;
|
|
|
50
56
|
const CALL_ID = /^[A-Za-z0-9_-]{1,256}$/;
|
|
51
57
|
const SESSION_ID = /^[A-Za-z0-9_.:-]{1,256}$/;
|
|
52
58
|
const BROWSER_TAB_ID = /^[A-Za-z0-9_-]{1,96}$/;
|
|
59
|
+
const ISO_639_1_LANGUAGE = /^[A-Za-z]{2}$/;
|
|
60
|
+
const REALTIME_VOICE_BUILT_INS = new Set([
|
|
61
|
+
"alloy",
|
|
62
|
+
"ash",
|
|
63
|
+
"ballad",
|
|
64
|
+
"coral",
|
|
65
|
+
"echo",
|
|
66
|
+
"sage",
|
|
67
|
+
"shimmer",
|
|
68
|
+
"verse",
|
|
69
|
+
"marin",
|
|
70
|
+
"cedar",
|
|
71
|
+
]);
|
|
72
|
+
const REALTIME_VOICE_REASONING_EFFORT = {
|
|
73
|
+
instant: "minimal",
|
|
74
|
+
balanced: "low",
|
|
75
|
+
deep: "medium",
|
|
76
|
+
} as const;
|
|
77
|
+
|
|
78
|
+
/**
|
|
79
|
+
* Realtime sessions have a deliberately bounded tool manifest. Keep the
|
|
80
|
+
* context/navigation tools ahead of large template registries so voice can
|
|
81
|
+
* always see and operate the same UI navigation surface as text chat.
|
|
82
|
+
*/
|
|
83
|
+
const REALTIME_VOICE_PRIORITY_TOOLS = [
|
|
84
|
+
"navigate",
|
|
85
|
+
"set-url-path",
|
|
86
|
+
"set-search-params",
|
|
87
|
+
"view-screen",
|
|
88
|
+
TOOL_SEARCH_ACTION_NAME,
|
|
89
|
+
] as const;
|
|
53
90
|
|
|
54
91
|
export interface RealtimeVoiceRequestContext {
|
|
55
92
|
event: H3Event;
|
|
@@ -105,6 +142,17 @@ interface RealtimeFunctionTool {
|
|
|
105
142
|
parameters: Record<string, unknown>;
|
|
106
143
|
}
|
|
107
144
|
|
|
145
|
+
interface RealtimeToolCapability {
|
|
146
|
+
userEmail: string;
|
|
147
|
+
orgId?: string;
|
|
148
|
+
browserTabId?: string;
|
|
149
|
+
expiresAt: number;
|
|
150
|
+
initialNames: Set<string>;
|
|
151
|
+
names: Set<string>;
|
|
152
|
+
}
|
|
153
|
+
|
|
154
|
+
type RealtimeToolCapabilityStore = Map<string, RealtimeToolCapability>;
|
|
155
|
+
|
|
108
156
|
interface AuthenticatedVoiceContext extends RealtimeVoiceRequestContext {
|
|
109
157
|
timezone?: string;
|
|
110
158
|
}
|
|
@@ -126,6 +174,50 @@ function configuredIdentifier(
|
|
|
126
174
|
: fallback;
|
|
127
175
|
}
|
|
128
176
|
|
|
177
|
+
export function resolveRealtimeVoiceTranscriptionLanguage(
|
|
178
|
+
acceptLanguage: string | null | undefined,
|
|
179
|
+
): string {
|
|
180
|
+
for (const locale of parseAcceptLanguage(acceptLanguage)) {
|
|
181
|
+
const primaryLanguage = locale.split("-")[0];
|
|
182
|
+
if (ISO_639_1_LANGUAGE.test(primaryLanguage)) {
|
|
183
|
+
return primaryLanguage.toLowerCase();
|
|
184
|
+
}
|
|
185
|
+
}
|
|
186
|
+
return "en";
|
|
187
|
+
}
|
|
188
|
+
|
|
189
|
+
export function resolveRealtimeVoiceLanguagePreference(
|
|
190
|
+
value: string | null | undefined,
|
|
191
|
+
acceptLanguage: string | null | undefined,
|
|
192
|
+
): string {
|
|
193
|
+
const normalized = value?.trim().toLowerCase();
|
|
194
|
+
return normalized && ISO_639_1_LANGUAGE.test(normalized)
|
|
195
|
+
? normalized
|
|
196
|
+
: resolveRealtimeVoiceTranscriptionLanguage(acceptLanguage);
|
|
197
|
+
}
|
|
198
|
+
|
|
199
|
+
export function resolveRealtimeVoiceReasoningEffort(
|
|
200
|
+
value: string | null | undefined,
|
|
201
|
+
): "minimal" | "low" | "medium" {
|
|
202
|
+
const normalized = value?.trim().toLowerCase();
|
|
203
|
+
return normalized &&
|
|
204
|
+
Object.hasOwn(REALTIME_VOICE_REASONING_EFFORT, normalized)
|
|
205
|
+
? REALTIME_VOICE_REASONING_EFFORT[
|
|
206
|
+
normalized as keyof typeof REALTIME_VOICE_REASONING_EFFORT
|
|
207
|
+
]
|
|
208
|
+
: "low";
|
|
209
|
+
}
|
|
210
|
+
|
|
211
|
+
export function resolveRealtimeVoicePreference(
|
|
212
|
+
value: string | null | undefined,
|
|
213
|
+
fallback: string,
|
|
214
|
+
): string {
|
|
215
|
+
const normalized = value?.trim().toLowerCase();
|
|
216
|
+
return normalized && REALTIME_VOICE_BUILT_INS.has(normalized)
|
|
217
|
+
? normalized
|
|
218
|
+
: fallback;
|
|
219
|
+
}
|
|
220
|
+
|
|
129
221
|
function truncate(value: string, maxChars: number): string {
|
|
130
222
|
if (value.length <= maxChars) return value;
|
|
131
223
|
return `${value.slice(0, maxChars)}\n\n...[truncated]`;
|
|
@@ -218,7 +310,22 @@ function buildRealtimeTools(
|
|
|
218
310
|
parameters,
|
|
219
311
|
});
|
|
220
312
|
}
|
|
221
|
-
|
|
313
|
+
const priority = new Map<string, number>(
|
|
314
|
+
REALTIME_VOICE_PRIORITY_TOOLS.map((name, index) => [name, index]),
|
|
315
|
+
);
|
|
316
|
+
return tools
|
|
317
|
+
.map((tool, index) => ({ tool, index }))
|
|
318
|
+
.sort((left, right) => {
|
|
319
|
+
const leftPriority = priority.get(left.tool.name);
|
|
320
|
+
const rightPriority = priority.get(right.tool.name);
|
|
321
|
+
if (leftPriority !== undefined || rightPriority !== undefined) {
|
|
322
|
+
if (leftPriority === undefined) return 1;
|
|
323
|
+
if (rightPriority === undefined) return -1;
|
|
324
|
+
return leftPriority - rightPriority;
|
|
325
|
+
}
|
|
326
|
+
return left.index - right.index;
|
|
327
|
+
})
|
|
328
|
+
.map(({ tool }) => tool);
|
|
222
329
|
}
|
|
223
330
|
|
|
224
331
|
function packRealtimeTools(
|
|
@@ -243,6 +350,124 @@ function packRealtimeTools(
|
|
|
243
350
|
return packed;
|
|
244
351
|
}
|
|
245
352
|
|
|
353
|
+
function mintRealtimeToolCapability(): string {
|
|
354
|
+
return globalThis.crypto.randomUUID().replaceAll("-", "");
|
|
355
|
+
}
|
|
356
|
+
|
|
357
|
+
function cleanRealtimeToolCapabilities(
|
|
358
|
+
capabilities: RealtimeToolCapabilityStore,
|
|
359
|
+
now = Date.now(),
|
|
360
|
+
): void {
|
|
361
|
+
for (const [key, capability] of capabilities) {
|
|
362
|
+
if (capability.expiresAt <= now) capabilities.delete(key);
|
|
363
|
+
}
|
|
364
|
+
while (capabilities.size > REALTIME_VOICE_MAX_TOOL_GRANT_SESSIONS) {
|
|
365
|
+
let oldestKey: string | undefined;
|
|
366
|
+
let oldestExpiry = Number.POSITIVE_INFINITY;
|
|
367
|
+
for (const [key, capability] of capabilities) {
|
|
368
|
+
if (capability.expiresAt < oldestExpiry) {
|
|
369
|
+
oldestKey = key;
|
|
370
|
+
oldestExpiry = capability.expiresAt;
|
|
371
|
+
}
|
|
372
|
+
}
|
|
373
|
+
if (!oldestKey) break;
|
|
374
|
+
capabilities.delete(oldestKey);
|
|
375
|
+
}
|
|
376
|
+
}
|
|
377
|
+
|
|
378
|
+
function registerRealtimeToolCapability(
|
|
379
|
+
capabilities: RealtimeToolCapabilityStore,
|
|
380
|
+
auth: AuthenticatedVoiceContext,
|
|
381
|
+
initialNames: Iterable<string>,
|
|
382
|
+
): string {
|
|
383
|
+
cleanRealtimeToolCapabilities(capabilities);
|
|
384
|
+
const token = mintRealtimeToolCapability();
|
|
385
|
+
capabilities.set(token, {
|
|
386
|
+
userEmail: auth.userEmail.trim().toLowerCase(),
|
|
387
|
+
...(auth.orgId ? { orgId: auth.orgId } : {}),
|
|
388
|
+
...(auth.browserTabId ? { browserTabId: auth.browserTabId } : {}),
|
|
389
|
+
expiresAt: Date.now() + REALTIME_VOICE_TOOL_GRANT_TTL_MS,
|
|
390
|
+
initialNames: new Set(initialNames),
|
|
391
|
+
names: new Set(),
|
|
392
|
+
});
|
|
393
|
+
cleanRealtimeToolCapabilities(capabilities);
|
|
394
|
+
return token;
|
|
395
|
+
}
|
|
396
|
+
|
|
397
|
+
function resolveRealtimeToolCapability(
|
|
398
|
+
capabilities: RealtimeToolCapabilityStore,
|
|
399
|
+
token: string | undefined,
|
|
400
|
+
auth: AuthenticatedVoiceContext,
|
|
401
|
+
): RealtimeToolCapability | null {
|
|
402
|
+
cleanRealtimeToolCapabilities(capabilities);
|
|
403
|
+
if (!token) return null;
|
|
404
|
+
const capability = capabilities.get(token);
|
|
405
|
+
if (!capability) return null;
|
|
406
|
+
if (
|
|
407
|
+
capability.userEmail !== auth.userEmail.trim().toLowerCase() ||
|
|
408
|
+
capability.orgId !== auth.orgId ||
|
|
409
|
+
capability.browserTabId !== auth.browserTabId
|
|
410
|
+
) {
|
|
411
|
+
return null;
|
|
412
|
+
}
|
|
413
|
+
capability.expiresAt = Date.now() + REALTIME_VOICE_TOOL_GRANT_TTL_MS;
|
|
414
|
+
return capability;
|
|
415
|
+
}
|
|
416
|
+
|
|
417
|
+
function parseSuccessfulToolSearchNames(output: string): string[] {
|
|
418
|
+
try {
|
|
419
|
+
const parsed = JSON.parse(output) as { results?: unknown };
|
|
420
|
+
if (!Array.isArray(parsed.results)) return [];
|
|
421
|
+
const names: string[] = [];
|
|
422
|
+
for (const result of parsed.results) {
|
|
423
|
+
if (!isRecord(result) || typeof result.name !== "string") continue;
|
|
424
|
+
if (!REALTIME_TOOL_NAME.test(result.name)) continue;
|
|
425
|
+
if (!names.includes(result.name)) names.push(result.name);
|
|
426
|
+
if (names.length >= REALTIME_VOICE_MAX_TOOLS) break;
|
|
427
|
+
}
|
|
428
|
+
return names;
|
|
429
|
+
} catch {
|
|
430
|
+
return [];
|
|
431
|
+
}
|
|
432
|
+
}
|
|
433
|
+
|
|
434
|
+
function grantDiscoveredRealtimeTools(input: {
|
|
435
|
+
request: NonNullable<ReturnType<typeof parseToolRequest>>;
|
|
436
|
+
result: RealtimeVoiceToolExecutionResult;
|
|
437
|
+
toolsByName: ReadonlyMap<string, RealtimeFunctionTool>;
|
|
438
|
+
initialAllowedNames: ReadonlySet<string>;
|
|
439
|
+
capability: RealtimeToolCapability;
|
|
440
|
+
}): RealtimeFunctionTool[] {
|
|
441
|
+
const query = input.request.args.query;
|
|
442
|
+
if (
|
|
443
|
+
input.request.name !== TOOL_SEARCH_ACTION_NAME ||
|
|
444
|
+
input.result.status !== "completed" ||
|
|
445
|
+
typeof query !== "string" ||
|
|
446
|
+
!query.trim()
|
|
447
|
+
) {
|
|
448
|
+
return [];
|
|
449
|
+
}
|
|
450
|
+
|
|
451
|
+
const candidates = parseSuccessfulToolSearchNames(input.result.output)
|
|
452
|
+
.filter((name) => !input.initialAllowedNames.has(name))
|
|
453
|
+
.map((name) => input.toolsByName.get(name))
|
|
454
|
+
.filter((tool): tool is RealtimeFunctionTool => Boolean(tool));
|
|
455
|
+
const boundedTools = packRealtimeTools({}, candidates);
|
|
456
|
+
const expandedTools: RealtimeFunctionTool[] = [];
|
|
457
|
+
for (const tool of boundedTools) {
|
|
458
|
+
if (
|
|
459
|
+
!input.capability.names.has(tool.name) &&
|
|
460
|
+
input.capability.names.size >= REALTIME_VOICE_MAX_TOOLS
|
|
461
|
+
) {
|
|
462
|
+
continue;
|
|
463
|
+
}
|
|
464
|
+
input.capability.names.add(tool.name);
|
|
465
|
+
expandedTools.push(tool);
|
|
466
|
+
}
|
|
467
|
+
input.capability.expiresAt = Date.now() + REALTIME_VOICE_TOOL_GRANT_TTL_MS;
|
|
468
|
+
return expandedTools;
|
|
469
|
+
}
|
|
470
|
+
|
|
246
471
|
function declaredBodyBytes(event: H3Event): number | undefined {
|
|
247
472
|
const raw = readSafeHeader(event, "content-length");
|
|
248
473
|
if (!raw) return undefined;
|
|
@@ -328,6 +553,7 @@ function invalidMethod(event: H3Event): { error: string } {
|
|
|
328
553
|
|
|
329
554
|
function createSessionHandler(
|
|
330
555
|
tools: RealtimeFunctionTool[],
|
|
556
|
+
capabilities: RealtimeToolCapabilityStore,
|
|
331
557
|
options: MountRealtimeVoiceRoutesOptions,
|
|
332
558
|
) {
|
|
333
559
|
return defineEventHandler(async (event: H3Event) => {
|
|
@@ -394,14 +620,30 @@ function createSessionHandler(
|
|
|
394
620
|
}
|
|
395
621
|
|
|
396
622
|
const instructions = await buildInstructions(auth, options);
|
|
623
|
+
const transcriptionLanguage = resolveRealtimeVoiceLanguagePreference(
|
|
624
|
+
readSafeHeader(event, "x-agent-native-realtime-language"),
|
|
625
|
+
readSafeHeader(event, "accept-language"),
|
|
626
|
+
);
|
|
627
|
+
const reasoningEffort = resolveRealtimeVoiceReasoningEffort(
|
|
628
|
+
readSafeHeader(event, "x-agent-native-realtime-intelligence"),
|
|
629
|
+
);
|
|
630
|
+
const voice = resolveRealtimeVoicePreference(
|
|
631
|
+
readSafeHeader(event, "x-agent-native-realtime-voice"),
|
|
632
|
+
configuredIdentifier(options.voice, DEFAULT_VOICE),
|
|
633
|
+
);
|
|
397
634
|
const sessionBase = {
|
|
398
635
|
type: "realtime",
|
|
399
636
|
model: configuredIdentifier(options.model, DEFAULT_MODEL),
|
|
400
637
|
instructions,
|
|
638
|
+
parallel_tool_calls: false,
|
|
639
|
+
reasoning: { effort: reasoningEffort },
|
|
401
640
|
output_modalities: ["audio"],
|
|
402
641
|
audio: {
|
|
403
642
|
input: {
|
|
404
|
-
transcription: {
|
|
643
|
+
transcription: {
|
|
644
|
+
model: "gpt-4o-mini-transcribe",
|
|
645
|
+
language: transcriptionLanguage,
|
|
646
|
+
},
|
|
405
647
|
turn_detection: {
|
|
406
648
|
type: "semantic_vad",
|
|
407
649
|
create_response: true,
|
|
@@ -410,13 +652,14 @@ function createSessionHandler(
|
|
|
410
652
|
},
|
|
411
653
|
},
|
|
412
654
|
output: {
|
|
413
|
-
voice
|
|
655
|
+
voice,
|
|
414
656
|
},
|
|
415
657
|
},
|
|
416
658
|
};
|
|
659
|
+
const packedTools = packRealtimeTools(sessionBase, tools);
|
|
417
660
|
const session = {
|
|
418
661
|
...sessionBase,
|
|
419
|
-
tools:
|
|
662
|
+
tools: packedTools,
|
|
420
663
|
tool_choice: "auto",
|
|
421
664
|
};
|
|
422
665
|
|
|
@@ -491,6 +734,15 @@ function createSessionHandler(
|
|
|
491
734
|
|
|
492
735
|
setResponseStatus(event, upstream.status);
|
|
493
736
|
setResponseHeader(event, "Content-Type", "application/sdp");
|
|
737
|
+
setResponseHeader(
|
|
738
|
+
event,
|
|
739
|
+
REALTIME_VOICE_CAPABILITY_HEADER,
|
|
740
|
+
registerRealtimeToolCapability(
|
|
741
|
+
capabilities,
|
|
742
|
+
auth,
|
|
743
|
+
packedTools.map((tool) => tool.name),
|
|
744
|
+
),
|
|
745
|
+
);
|
|
494
746
|
return answerSdp;
|
|
495
747
|
},
|
|
496
748
|
);
|
|
@@ -562,7 +814,8 @@ function normalizeExecutionResult(
|
|
|
562
814
|
}
|
|
563
815
|
|
|
564
816
|
function createToolHandler(
|
|
565
|
-
|
|
817
|
+
toolsByName: ReadonlyMap<string, RealtimeFunctionTool>,
|
|
818
|
+
capabilities: RealtimeToolCapabilityStore,
|
|
566
819
|
options: MountRealtimeVoiceRoutesOptions,
|
|
567
820
|
) {
|
|
568
821
|
return defineEventHandler(async (event: H3Event) => {
|
|
@@ -578,6 +831,15 @@ function createToolHandler(
|
|
|
578
831
|
setResponseStatus(event, 401);
|
|
579
832
|
return { error: "Authentication required" };
|
|
580
833
|
}
|
|
834
|
+
const capability = resolveRealtimeToolCapability(
|
|
835
|
+
capabilities,
|
|
836
|
+
readSafeHeader(event, REALTIME_VOICE_CAPABILITY_HEADER),
|
|
837
|
+
auth,
|
|
838
|
+
);
|
|
839
|
+
if (!capability) {
|
|
840
|
+
setResponseStatus(event, 403);
|
|
841
|
+
return { error: "Invalid or expired realtime voice capability" };
|
|
842
|
+
}
|
|
581
843
|
|
|
582
844
|
const contentType = readSafeHeader(event, "content-type")?.toLowerCase();
|
|
583
845
|
if (!contentType?.includes("application/json")) {
|
|
@@ -608,12 +870,22 @@ function createToolHandler(
|
|
|
608
870
|
setResponseStatus(event, 400);
|
|
609
871
|
return { error: "Invalid realtime tool request" };
|
|
610
872
|
}
|
|
611
|
-
if (
|
|
873
|
+
if (
|
|
874
|
+
request.browserTabId !== undefined &&
|
|
875
|
+
request.browserTabId !== capability.browserTabId
|
|
876
|
+
) {
|
|
877
|
+
setResponseStatus(event, 403);
|
|
878
|
+
return { error: "Realtime voice browser tab mismatch" };
|
|
879
|
+
}
|
|
880
|
+
if (
|
|
881
|
+
!capability.initialNames.has(request.name) &&
|
|
882
|
+
!capability.names.has(request.name)
|
|
883
|
+
) {
|
|
612
884
|
setResponseStatus(event, 404);
|
|
613
885
|
return { error: "Unknown realtime voice tool" };
|
|
614
886
|
}
|
|
615
887
|
|
|
616
|
-
const browserTabId =
|
|
888
|
+
const browserTabId = capability.browserTabId;
|
|
617
889
|
const threadId = request.sessionId
|
|
618
890
|
? `realtime:${request.sessionId}`
|
|
619
891
|
: `realtime:${request.callId}`;
|
|
@@ -635,9 +907,21 @@ function createToolHandler(
|
|
|
635
907
|
userEmail: auth.userEmail,
|
|
636
908
|
orgId: auth.orgId,
|
|
637
909
|
...request,
|
|
910
|
+
...(browserTabId ? { browserTabId } : {}),
|
|
638
911
|
}),
|
|
639
912
|
);
|
|
640
|
-
|
|
913
|
+
const expandedTools = grantDiscoveredRealtimeTools({
|
|
914
|
+
request,
|
|
915
|
+
result,
|
|
916
|
+
toolsByName,
|
|
917
|
+
initialAllowedNames: capability.initialNames,
|
|
918
|
+
capability,
|
|
919
|
+
});
|
|
920
|
+
return {
|
|
921
|
+
callId: request.callId,
|
|
922
|
+
...result,
|
|
923
|
+
...(expandedTools.length > 0 ? { expandedTools } : {}),
|
|
924
|
+
};
|
|
641
925
|
} catch (error) {
|
|
642
926
|
setResponseStatus(event, 500);
|
|
643
927
|
return {
|
|
@@ -665,14 +949,16 @@ export function mountRealtimeVoiceRoutes(
|
|
|
665
949
|
}
|
|
666
950
|
|
|
667
951
|
const tools = buildRealtimeTools(actions);
|
|
668
|
-
const
|
|
669
|
-
|
|
670
|
-
);
|
|
952
|
+
const toolsByName = new Map(tools.map((tool) => [tool.name, tool]));
|
|
953
|
+
const capabilities: RealtimeToolCapabilityStore = new Map();
|
|
671
954
|
const app = getH3App(nitroApp);
|
|
672
|
-
app.use(
|
|
955
|
+
app.use(
|
|
956
|
+
REALTIME_VOICE_SESSION_PATH,
|
|
957
|
+
createSessionHandler(tools, capabilities, options),
|
|
958
|
+
);
|
|
673
959
|
app.use(
|
|
674
960
|
REALTIME_VOICE_TOOL_PATH,
|
|
675
|
-
createToolHandler(
|
|
961
|
+
createToolHandler(toolsByName, capabilities, options),
|
|
676
962
|
);
|
|
677
963
|
return {
|
|
678
964
|
sessionPath: REALTIME_VOICE_SESSION_PATH,
|
|
@@ -733,7 +733,7 @@
|
|
|
733
733
|
}
|
|
734
734
|
|
|
735
735
|
.agent-composer-root--hero {
|
|
736
|
-
min-height:
|
|
736
|
+
min-height: 5.625rem;
|
|
737
737
|
border-radius: 1rem;
|
|
738
738
|
border-color: hsl(var(--border) / 0.75);
|
|
739
739
|
background: hsl(var(--muted) / 0.62);
|
|
@@ -751,7 +751,7 @@
|
|
|
751
751
|
}
|
|
752
752
|
|
|
753
753
|
.agent-composer-root--hero [data-agent-composer-slot="editor-input"] {
|
|
754
|
-
min-height:
|
|
754
|
+
min-height: 3rem;
|
|
755
755
|
font-size: 1rem;
|
|
756
756
|
line-height: 1.75rem;
|
|
757
757
|
}
|
|
@@ -30,7 +30,7 @@ The agent modifies data in SQL, but the UI runs in the browser. SSE bridges same
|
|
|
30
30
|
useDbSync({ queryClient });
|
|
31
31
|
```
|
|
32
32
|
|
|
33
|
-
For each non-own event, `useDbSync` bumps a per-source counter (e.g. `dashboards`, `analyses`, `settings`, `action`) and invalidates a small fixed list of framework-internal prefixes (`["action"]`, `["app-state"]`, `["__set_url__"]`, etc.). It does **not** blanket-invalidate templates' own data queries
|
|
33
|
+
For each non-own event, `useDbSync` bumps a per-source counter (e.g. `dashboards`, `analyses`, `settings`, `action`) and invalidates a small fixed list of framework-internal prefixes (`["action"]`, `["app-state"]`, `["__set_url__"]`, etc.). It does **not** blanket-invalidate templates' own data queries — that caused request storms in production. A successful mutating action refreshes active `useActionQuery` observers under the `["action"]` prefix. Browser actions carry a tab id so their originating tab ignores the sync echo while other tabs refresh. Legacy apps can opt into broader compatibility with `actionInvalidatePredicate`, but first-party apps should use action-backed or source-versioned query keys. Idle fallback polling runs once per minute; active agent work temporarily uses the faster cadence.
|
|
34
34
|
|
|
35
35
|
3. **Templates fold per-source counters into their query keys.** This is the pattern that makes "agent writes show up without a manual refresh" reliable:
|
|
36
36
|
|
|
@@ -117,7 +117,7 @@ useQuery({
|
|
|
117
117
|
| UI not updating after agent writes | Is `useDbSync` called with the correct `queryClient`? Does the affected query have an active observer? |
|
|
118
118
|
| Poll endpoint not responding | Is `/_agent-native/poll` accessible? Is the server running? |
|
|
119
119
|
| SSE not connecting | Is `/_agent-native/events` accessible and authenticated? Polling should still keep the UI fresh as fallback. |
|
|
120
|
-
| High CPU / event storms |
|
|
120
|
+
| High CPU / event storms | Use targeted source keys, settle bursty list counters, and avoid broad action invalidation. |
|
|
121
121
|
|
|
122
122
|
## Jitter Prevention
|
|
123
123
|
|
|
@@ -30,7 +30,7 @@ The agent modifies data in SQL, but the UI runs in the browser. SSE bridges same
|
|
|
30
30
|
useDbSync({ queryClient });
|
|
31
31
|
```
|
|
32
32
|
|
|
33
|
-
For each non-own event, `useDbSync` bumps a per-source counter (e.g. `dashboards`, `analyses`, `settings`, `action`) and invalidates a small fixed list of framework-internal prefixes (`["action"]`, `["app-state"]`, `["__set_url__"]`, etc.). It does **not** blanket-invalidate templates' own data queries
|
|
33
|
+
For each non-own event, `useDbSync` bumps a per-source counter (e.g. `dashboards`, `analyses`, `settings`, `action`) and invalidates a small fixed list of framework-internal prefixes (`["action"]`, `["app-state"]`, `["__set_url__"]`, etc.). It does **not** blanket-invalidate templates' own data queries — that caused request storms in production. A successful mutating action refreshes active `useActionQuery` observers under the `["action"]` prefix. Browser actions carry a tab id so their originating tab ignores the sync echo while other tabs refresh. Legacy apps can opt into broader compatibility with `actionInvalidatePredicate`, but first-party apps should use action-backed or source-versioned query keys. Idle fallback polling runs once per minute; active agent work temporarily uses the faster cadence.
|
|
34
34
|
|
|
35
35
|
3. **Templates fold per-source counters into their query keys.** This is the pattern that makes "agent writes show up without a manual refresh" reliable:
|
|
36
36
|
|
|
@@ -117,7 +117,7 @@ useQuery({
|
|
|
117
117
|
| UI not updating after agent writes | Is `useDbSync` called with the correct `queryClient`? Does the affected query have an active observer? |
|
|
118
118
|
| Poll endpoint not responding | Is `/_agent-native/poll` accessible? Is the server running? |
|
|
119
119
|
| SSE not connecting | Is `/_agent-native/events` accessible and authenticated? Polling should still keep the UI fresh as fallback. |
|
|
120
|
-
| High CPU / event storms |
|
|
120
|
+
| High CPU / event storms | Use targeted source keys, settle bursty list counters, and avoid broad action invalidation. |
|
|
121
121
|
|
|
122
122
|
## Jitter Prevention
|
|
123
123
|
|
|
@@ -170,9 +170,13 @@ Core routes plugin mounts these under `/_agent-native/secrets/` automatically:
|
|
|
170
170
|
- Values are stored in `app_secrets` (created on-demand; no migration
|
|
171
171
|
needed).
|
|
172
172
|
- Encrypted at rest with AES-256-GCM. Key material is derived from
|
|
173
|
-
`
|
|
174
|
-
|
|
175
|
-
|
|
173
|
+
`<APP_NAME>_SECRETS_ENCRYPTION_KEY` when set (for example,
|
|
174
|
+
`ANALYTICS_SECRETS_ENCRYPTION_KEY`), then `SECRETS_ENCRYPTION_KEY`, then
|
|
175
|
+
`BETTER_AUTH_SECRET`. App-scoped keys are useful when a local multi-app
|
|
176
|
+
workspace connects one app to its production database without replacing the
|
|
177
|
+
shared local authentication secret. If none is set, the framework uses a
|
|
178
|
+
machine-local fallback and logs a one-time warning — set stable key material
|
|
179
|
+
in production and in every runtime that reads the same encrypted data.
|
|
176
180
|
|
|
177
181
|
## Ad-hoc Keys
|
|
178
182
|
|