@vellumai/vellum-gateway 0.10.4 → 0.10.5-dev.202607022329.d89bf2f
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +19 -18
- package/ARCHITECTURE.md +89 -79
- package/README.md +12 -12
- package/node_modules/@vellumai/assistant-client/src/__tests__/assistant-client.test.ts +1 -1
- package/node_modules/@vellumai/gateway-client/src/__tests__/contact-read-contracts.test.ts +36 -0
- package/node_modules/@vellumai/gateway-client/src/__tests__/invite-contract.test.ts +233 -0
- package/node_modules/@vellumai/gateway-client/src/__tests__/trust-verdict-contract.test.ts +40 -0
- package/node_modules/@vellumai/gateway-client/src/gateway-ipc-contracts.ts +20 -8
- package/node_modules/@vellumai/gateway-client/src/inbound-contract.ts +10 -0
- package/node_modules/@vellumai/gateway-client/src/index.ts +33 -0
- package/node_modules/@vellumai/gateway-client/src/invite-contract.ts +216 -0
- package/node_modules/@vellumai/gateway-client/src/outbound-contract.ts +91 -0
- package/node_modules/@vellumai/gateway-client/src/trust-verdict-contract.ts +39 -3
- package/node_modules/@vellumai/service-contracts/src/__tests__/ingress.test.ts +0 -8
- package/node_modules/@vellumai/service-contracts/src/__tests__/package-boundary.test.ts +1 -5
- package/node_modules/@vellumai/service-contracts/src/twilio-ingress.ts +3 -14
- package/package.json +1 -1
- package/src/__tests__/contact-handlers.test.ts +45 -119
- package/src/__tests__/contact-prompt-submit.test.ts +50 -3
- package/src/__tests__/contact-rich-read.test.ts +96 -36
- package/src/__tests__/contact-store-invites.test.ts +250 -15
- package/src/__tests__/contact-store-mark-channel-verified.test.ts +37 -0
- package/src/__tests__/contacts-control-plane-proxy.test.ts +808 -700
- package/src/__tests__/contacts-info-client.test.ts +171 -0
- package/src/__tests__/contacts-info-joiner.test.ts +54 -43
- package/src/__tests__/credential-refresh.test.ts +105 -0
- package/src/__tests__/data-migration-m0008-upsert-acl-columns-from-assistant.test.ts +38 -1
- package/src/__tests__/data-migration-m0009-invite-fields-backfill.test.ts +399 -0
- package/src/__tests__/data-migration-m0010-drop-assistant-ingress-invites.test.ts +235 -0
- package/src/__tests__/data-migration-m0011-drop-gw-verification-sessions.test.ts +100 -0
- package/src/__tests__/db-proxy-allowlist.test.ts +163 -0
- package/src/__tests__/email-inbound-pipeline.test.ts +254 -0
- package/src/__tests__/guardian-binding-channel-reuse.test.ts +314 -0
- package/src/__tests__/guardian-bootstrap-binding.test.ts +25 -9
- package/src/__tests__/guardian-channel-create.test.ts +1 -19
- package/src/__tests__/guardian-init-lockfile.test.ts +29 -26
- package/src/__tests__/handle-inbound-invite-intercept.test.ts +453 -0
- package/src/__tests__/handle-inbound-trust-verdict.test.ts +177 -8
- package/src/__tests__/helpers/contact-fixtures.ts +87 -0
- package/src/__tests__/helpers/ipc-newline-client.ts +52 -0
- package/src/__tests__/inbound-register.test.ts +1 -19
- package/src/__tests__/invite-redemption-engine-voice.test.ts +379 -0
- package/src/__tests__/invite-redemption-engine.test.ts +586 -0
- package/src/__tests__/invite-route-auth.test.ts +212 -0
- package/src/__tests__/ipc-contact-routes.test.ts +47 -61
- package/src/__tests__/ipc-feature-flag-routes.test.ts +3 -45
- package/src/__tests__/ipc-invite-routes.test.ts +187 -121
- package/src/__tests__/ipc-server-multi-client.test.ts +2 -38
- package/src/__tests__/ipc-server-watchdog.test.ts +2 -38
- package/src/__tests__/ipc-slack-thread-routes.test.ts +2 -38
- package/src/__tests__/ipc-voice-invite-routes.test.ts +305 -0
- package/src/__tests__/m0006-reconcile-contacts.test.ts +31 -1
- package/src/__tests__/nonbash-trust-rule-overrides.test.ts +1 -0
- package/src/__tests__/route-schema-guard.test.ts +0 -4
- package/src/__tests__/runtime-client.test.ts +0 -43
- package/src/__tests__/schema.test.ts +0 -2
- package/src/__tests__/slack-display-name.test.ts +29 -4
- package/src/__tests__/slack-socket-mode-thread-tracking.test.ts +60 -0
- package/src/__tests__/twilio-webhooks.test.ts +2 -62
- package/src/__tests__/upsert-verified-contact-channel.test.ts +362 -136
- package/src/__tests__/upstream-transport.test.ts +0 -15
- package/src/auth/guardian-bootstrap.ts +32 -66
- package/src/channels/inbound-event.ts +2 -0
- package/src/credential-refresh.ts +59 -0
- package/src/db/assistant-db-proxy.ts +13 -8
- package/src/db/contact-store.ts +226 -62
- package/src/db/contacts-info-joiner.ts +15 -57
- package/src/db/data-migrations/assistant-invite-id-column.ts +15 -0
- package/src/db/data-migrations/index.ts +9 -1
- package/src/db/data-migrations/m0006-reconcile-contacts-from-assistant.ts +5 -10
- package/src/db/data-migrations/m0008-upsert-acl-columns-from-assistant.ts +3 -1
- package/src/db/data-migrations/m0009-invite-fields-backfill.ts +208 -0
- package/src/db/data-migrations/m0010-drop-assistant-ingress-invites.ts +77 -0
- package/src/db/data-migrations/m0011-drop-gw-verification-sessions.ts +34 -0
- package/src/db/schema.ts +23 -35
- package/src/email/inbound-pipeline.ts +251 -0
- package/src/email/normalize.test.ts +26 -0
- package/src/email/normalize.ts +36 -0
- package/src/feature-flag-registry.json +16 -8
- package/src/handlers/handle-inbound.ts +102 -63
- package/src/http/read-limited-body.test.ts +90 -0
- package/src/http/read-limited-body.ts +25 -4
- package/src/http/routes/contact-prompt.ts +10 -9
- package/src/http/routes/contacts-control-plane-proxy.ts +415 -448
- package/src/http/routes/email-webhook.test.ts +74 -1
- package/src/http/routes/email-webhook.ts +39 -74
- package/src/http/routes/invite-validation.ts +33 -36
- package/src/http/routes/ipc-runtime-proxy.test.ts +10 -22
- package/src/http/routes/mailgun-webhook.ts +113 -319
- package/src/http/routes/resend-webhook.ts +84 -306
- package/src/http/routes/telegram-webhook.ts +24 -41
- package/src/http/routes/trust-rules.suggest.test.ts +5 -0
- package/src/http/routes/twilio-voice-verify-callback.ts +2 -2
- package/src/http/routes/whatsapp-webhook.ts +28 -68
- package/src/index.ts +45 -33
- package/src/ipc/__tests__/invite-ipc-routes.test.ts +133 -14
- package/src/ipc/contact-handlers.ts +0 -31
- package/src/ipc/contacts-info-client.ts +101 -0
- package/src/ipc/invite-handlers.ts +104 -47
- package/src/ipc/ipc-health.test.ts +4 -6
- package/src/ipc/ipc-health.ts +1 -1
- package/src/ipc/risk-classification-handlers.test.ts +20 -0
- package/src/ipc/risk-classification-handlers.ts +2 -0
- package/src/ipc/route-schema-cache.test.ts +5 -2
- package/src/post-assistant-ready.ts +2 -4
- package/src/risk/__tests__/trust-verdict-resolver.test.ts +211 -4
- package/src/risk/bash-risk-classifier.test.ts +9 -0
- package/src/risk/command-registry/commands/assistant.ts +63 -0
- package/src/risk/command-registry.test.ts +17 -0
- package/src/risk/file-risk-classifier.test.ts +67 -0
- package/src/risk/file-risk-classifier.ts +41 -11
- package/src/risk/risk-types.ts +2 -2
- package/src/risk/trust-verdict-resolver.ts +91 -1
- package/src/runtime/client.ts +0 -40
- package/src/schema.ts +3 -102
- package/src/slack/normalize.test.ts +191 -0
- package/src/slack/normalize.ts +134 -7
- package/src/slack/socket-mode.ts +15 -2
- package/src/twilio/validate-webhook.ts +1 -6
- package/src/velay/allowed-paths.test.ts +0 -1
- package/src/velay/allowed-paths.ts +6 -6
- package/src/velay/client.test.ts +1 -1
- package/src/velay/websocket-bridge.test.ts +5 -5
- package/src/verification/contact-helpers.ts +347 -113
- package/src/verification/invite-liveness.ts +33 -0
- package/src/verification/invite-redemption.ts +635 -0
- package/src/verification/outbound-voice-verification-sync.ts +1 -1
- package/src/verification/session-helpers.ts +5 -29
- package/src/voice/verification.test.ts +202 -0
- package/src/voice/verification.ts +22 -24
- package/src/webhook-pipeline.ts +23 -1
- package/src/whatsapp/send.ts +14 -12
- package/src/__tests__/twilio-relay-websocket.test.ts +0 -508
- package/src/http/routes/twilio-connect-action-webhook.ts +0 -51
- package/src/http/routes/twilio-relay-websocket.ts +0 -211
- package/src/verification/voice-approval-sync.ts +0 -123
package/AGENTS.md
CHANGED
|
@@ -9,6 +9,7 @@ Concretely:
|
|
|
9
9
|
- Define new routes in the gateway and have the gateway forward requests to the assistant over the internal HTTP transport.
|
|
10
10
|
- The gateway's public URL is controlled by the **public ingress URL** setting. All externally-facing URLs you generate or advertise (callback URLs, webhook registration URLs, etc.) must be derived from this setting — never hardcode a hostname or port.
|
|
11
11
|
- The daemon should remain unreachable from the public internet. It only receives traffic from the gateway over the internal network.
|
|
12
|
+
- Webhook handlers must read request bodies via `readLimitedBody()` / `readLimitedBodyBytes()` (`gateway/src/http/read-limited-body.ts`), which enforce `maxWebhookPayloadBytes` on the actual streamed bytes. Never call `req.text()` / `req.json()` / `req.formData()` directly on unauthenticated ingress — the Content-Length header is attacker-controlled and absent on chunked requests, so a header-only guard can be bypassed up to the server-wide `maxRequestBodySize`.
|
|
12
13
|
|
|
13
14
|
Why: the gateway is the single point of ingress, handling TLS termination, auth, rate limiting, and routing. Exposing the daemon directly bypasses these protections and breaks the deployment model.
|
|
14
15
|
|
|
@@ -73,13 +74,13 @@ A default row per enforced channel is **seeded at startup** (`seedAdmissionPolic
|
|
|
73
74
|
|
|
74
75
|
**5 policies, ranked floors** (seed default `trusted_contacts`):
|
|
75
76
|
|
|
76
|
-
| Policy
|
|
77
|
-
|
|
78
|
-
| `no_one`
|
|
79
|
-
| `guardian_only`
|
|
80
|
-
| `trusted_contacts` | 3
|
|
81
|
-
| `any_contact`
|
|
82
|
-
| `strangers`
|
|
77
|
+
| Policy | Floor | Notes |
|
|
78
|
+
| ------------------ | ----- | ----------------------------------------------------------------------------------------------------------------------------- |
|
|
79
|
+
| `no_one` | 5 | Hard-deny at gateway _before_ forwarding (kill switch in `handle-inbound.ts`). Includes the guardian — this channel is _OFF_. |
|
|
80
|
+
| `guardian_only` | 4 | Seeded default for `vellum`. |
|
|
81
|
+
| `trusted_contacts` | 3 | Seeded default for all other channels; also the read-path safety fallback. |
|
|
82
|
+
| `any_contact` | 2 | May surface Slack DM / email upgrade challenge on deny. |
|
|
83
|
+
| `strangers` | 1 | May surface upgrade challenge. |
|
|
83
84
|
|
|
84
85
|
**Exempt channels** (no policy ever applies — gateway **AND** runtime both short-circuit):
|
|
85
86
|
|
|
@@ -88,7 +89,7 @@ A default row per enforced channel is **seeded at startup** (`seedAdmissionPolic
|
|
|
88
89
|
|
|
89
90
|
`phone` is now an enforced channel (voice ingress reads the policy): it seeds the universal default `trusted_contacts` and accepts PUT like other enforced channels.
|
|
90
91
|
|
|
91
|
-
For exempt ids, `PUT /v1/assistants/:id/channel-admission-policy/:channelType` returns **403**, the GET list omits them, and the runtime short-circuits `admitted: true` in `admission-policy.ts` (defense in depth). Codex finding from #35006 review: exemption checks must live in
|
|
92
|
+
For exempt ids, `PUT /v1/assistants/:id/channel-admission-policy/:channelType` returns **403**, the GET list omits them, and the runtime short-circuits `admitted: true` in `admission-policy.ts` (defense in depth). Codex finding from #35006 review: exemption checks must live in _both_ the gateway route handler AND the runtime stage — single-side enforcement creates a misuse wedge.
|
|
92
93
|
|
|
93
94
|
**Hidden channels** (`ADMISSION_POLICY_HIDDEN_CHANNELS` = `vellum`, `whatsapp`) — managed automatically, **not** user-configurable, but (unlike exempt channels) **still enforced at runtime**:
|
|
94
95
|
|
|
@@ -102,7 +103,7 @@ Only the **assistant-scoped** routes (`/v1/assistants/:id/channel-admission-poli
|
|
|
102
103
|
|
|
103
104
|
- **Gateway kill switch** — `handle-inbound.ts` enforces the `no_one` floor before forwarding. Zero contact-table lookups, zero daemon I/O, true kill.
|
|
104
105
|
- **Runtime floor** — every other policy flows through the gateway unchanged; the runtime evaluates rank-vs-floor inside `admission-policy.ts`. This keeps the canonical `actor-trust-resolver.ts:280` classifier as the single source of `TrustClass` truth (no fork).
|
|
105
|
-
- **Gateway vs runtime reciprocity** — the gateway section in `gateway/CLAUDE.md` records
|
|
106
|
+
- **Gateway vs runtime reciprocity** — the gateway section in `gateway/CLAUDE.md` records _which channels the gateway enforces_; the assistant section records _how the runtime classifies_. Either side getting out of sync is a bug, not an over-defended boundary.
|
|
106
107
|
|
|
107
108
|
**Adding a new policy**: extend the `AdmissionPolicy` union in `packages/gateway-client/src/admission-policy-contract.ts`, add its floor in `ADMISSION_FLOOR`, update the openapi schema, and update `gateway/src/__tests__/channel-admission-policy-routes.test.ts` + `assistant/src/runtime/routes/inbound-stages/admission-policy.test.ts`. Do not add a 6th floor without also bumping the `TRUST_CLASS_RANK` ceiling to match.
|
|
108
109
|
|
|
@@ -114,17 +115,17 @@ Only the **assistant-scoped** routes (`/v1/assistants/:id/channel-admission-poli
|
|
|
114
115
|
|
|
115
116
|
Two orthogonal axes, do not conflate them:
|
|
116
117
|
|
|
117
|
-
- **Admission** (above) —
|
|
118
|
-
- **Capabilities** —
|
|
118
|
+
- **Admission** (above) — _who gets in the door_. `TRUST_CLASS_RANK` vs `ADMISSION_FLOOR`, enforced across gateway + runtime.
|
|
119
|
+
- **Capabilities** — _what an actor may do once admitted_. Resolved in the runtime, never on the gateway.
|
|
119
120
|
|
|
120
|
-
**Trust classes** (`TrustClass` in `assistant/src/runtime/actor-trust-resolver.ts`) are the
|
|
121
|
+
**Trust classes** (`TrustClass` in `assistant/src/runtime/actor-trust-resolver.ts`) are the _role_, ranked by `TRUST_CLASS_RANK`:
|
|
121
122
|
|
|
122
|
-
| Class
|
|
123
|
-
|
|
124
|
-
| `guardian`
|
|
125
|
-
| `trusted_contact`
|
|
126
|
-
| `unverified_contact` | 2
|
|
127
|
-
| `unknown`
|
|
123
|
+
| Class | Rank | Meaning |
|
|
124
|
+
| -------------------- | ---- | ------------------------------------------------------------------------ |
|
|
125
|
+
| `guardian` | 4 | Matches the active guardian binding for this (assistant, channel). |
|
|
126
|
+
| `trusted_contact` | 3 | Active contact channel, not the guardian. |
|
|
127
|
+
| `unverified_contact` | 2 | Contact channel that is `pending`/`unverified` — known but not verified. |
|
|
128
|
+
| `unknown` | 1 | No contact record, no identity, or blocked/revoked. Fail-closed. |
|
|
128
129
|
|
|
129
130
|
The gateway classifies the actor at ingress (keyed on `actorExternalId`) and forwards the resolved `trustClass`; it is persisted in retry payloads, the journal store, and conversation CRUD. The gateway does **not** compute capabilities.
|
|
130
131
|
|
package/ARCHITECTURE.md
CHANGED
|
@@ -10,7 +10,7 @@ This boundary is enforced at three layers:
|
|
|
10
10
|
|
|
11
11
|
1. **Gateway routing:** The gateway's `fetch()` handler matches `/webhooks/*` paths to dedicated handlers before the runtime-proxy fallthrough. Webhook traffic is validated (signature checks, payload limits) and forwarded to internal runtime endpoints.
|
|
12
12
|
2. **Runtime-proxy guard:** The runtime-proxy handler rejects any `/webhooks/*` path with a 404, preventing accidental forwarding of webhook traffic to the runtime even if a new webhook path is added to the gateway without a dedicated handler.
|
|
13
|
-
3. **Runtime server:** The runtime HTTP server does not register any `/webhooks/telegram` routes. Direct Twilio webhook routes (`/webhooks/twilio/*`) return 410 with a `GATEWAY_ONLY` error code, and
|
|
13
|
+
3. **Runtime server:** The runtime HTTP server does not register any `/webhooks/telegram` routes. Direct Twilio webhook routes (`/webhooks/twilio/*`) return 410 with a `GATEWAY_ONLY` error code, and call WebSocket upgrades are restricted to private network peers.
|
|
14
14
|
|
|
15
15
|
```
|
|
16
16
|
Internet
|
|
@@ -174,7 +174,7 @@ Runtime health is exposed directly by the gateway at `GET /v1/health` and forwar
|
|
|
174
174
|
|
|
175
175
|
### Telegram + Contacts Control-Plane Proxies
|
|
176
176
|
|
|
177
|
-
Telegram integration setup/config endpoints and contacts/invites endpoints are also exposed directly by the gateway
|
|
177
|
+
Telegram integration setup/config endpoints and contacts/invites endpoints are also exposed directly by the gateway for dedicated auth handling. Contact endpoints forward to runtime handlers; invite endpoints are gateway-native — they run against the gateway DB's `ingress_invites` table with no runtime round-trip (the outbound-call relay is the one exception: the gateway validates its invite row, then delegates the provider call to the assistant).
|
|
178
178
|
|
|
179
179
|
**Forwarded Telegram endpoints:**
|
|
180
180
|
|
|
@@ -184,7 +184,7 @@ Telegram integration setup/config endpoints and contacts/invites endpoints are a
|
|
|
184
184
|
| POST | `/v1/integrations/telegram/commands` |
|
|
185
185
|
| POST | `/v1/integrations/telegram/setup` |
|
|
186
186
|
|
|
187
|
-
**Forwarded contact
|
|
187
|
+
**Forwarded contact endpoints:**
|
|
188
188
|
|
|
189
189
|
| Method | Path |
|
|
190
190
|
| -------- | ---------------------------------------- |
|
|
@@ -192,14 +192,20 @@ Telegram integration setup/config endpoints and contacts/invites endpoints are a
|
|
|
192
192
|
| GET | `/v1/contacts/:contactId` |
|
|
193
193
|
| POST | `/v1/contacts/merge` |
|
|
194
194
|
| PATCH | `/v1/contact-channels/:contactChannelId` |
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
195
|
+
|
|
196
|
+
**Gateway-native invite endpoints:**
|
|
197
|
+
|
|
198
|
+
| Method | Path |
|
|
199
|
+
| -------- | ------------------------------------- |
|
|
200
|
+
| GET/POST | `/v1/contacts/invites` |
|
|
201
|
+
| DELETE | `/v1/contacts/invites/:inviteId` |
|
|
202
|
+
| POST | `/v1/contacts/invites/:inviteId/call` |
|
|
203
|
+
| POST | `/v1/contacts/invites/redeem` |
|
|
198
204
|
|
|
199
205
|
**Authentication boundary:**
|
|
200
206
|
|
|
201
207
|
- Gateway validates the caller's JWT bearer token.
|
|
202
|
-
-
|
|
208
|
+
- Forwarded requests reach runtime with a minted JWT (`gateway_ingress_v1` or `gateway_service_v1` scope profile); invite endpoints are served from the gateway DB after the same bearer-auth check.
|
|
203
209
|
- Upstream 4xx/5xx responses are passed through, while connection errors return `502` and timeouts return `504`.
|
|
204
210
|
|
|
205
211
|
**Key source files:**
|
|
@@ -207,7 +213,7 @@ Telegram integration setup/config endpoints and contacts/invites endpoints are a
|
|
|
207
213
|
| File | Purpose |
|
|
208
214
|
| --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
|
|
209
215
|
| `gateway/src/http/routes/telegram-control-plane-proxy.ts` | Telegram control-plane proxy handlers and upstream forwarding |
|
|
210
|
-
| `gateway/src/http/routes/contacts-control-plane-proxy.ts` |
|
|
216
|
+
| `gateway/src/http/routes/contacts-control-plane-proxy.ts` | Contact proxy handlers plus gateway-native invite handlers (mint, list, revoke, redeem, call relay) |
|
|
211
217
|
| `gateway/src/index.ts` | Route registration and bearer-auth enforcement for `/v1/integrations/telegram/*` and `/v1/contacts/invites/*` |
|
|
212
218
|
|
|
213
219
|
### Twilio Control-Plane Proxy
|
|
@@ -319,7 +325,7 @@ Local platform smoke-test flow:
|
|
|
319
325
|
4. Re-hatch or restart the assistant so the gateway receives the new environment.
|
|
320
326
|
5. Confirm gateway logs show `Velay tunnel connected` and `Velay tunnel registered`.
|
|
321
327
|
6. Verify HTTP forwarding by requesting `${VELAY_PUBLIC_BASE_URL}/<assistant-id>/healthz` and `${VELAY_PUBLIC_BASE_URL}/<assistant-id>/schema`. When validating a JSON webhook route under active development, POST a small JSON body through the same Velay public URL and confirm it reaches the loopback gateway.
|
|
322
|
-
7. Verify Twilio WebSocket forwarding with a synthetic local WebSocket client against `${VELAY_PUBLIC_BASE_URL}/<assistant-id>/webhooks/twilio/
|
|
328
|
+
7. Verify Twilio WebSocket forwarding with a synthetic local WebSocket client against `${VELAY_PUBLIC_BASE_URL}/<assistant-id>/webhooks/twilio/media-stream/<callSessionId>/<token>`, then with a real Twilio call after the gateway has registered with Velay.
|
|
323
329
|
|
|
324
330
|
### URL Builders
|
|
325
331
|
|
|
@@ -330,8 +336,7 @@ All public-facing URLs are constructed by `assistant/src/inbound/public-ingress-
|
|
|
330
336
|
| `getPublicBaseUrl()` | Resolves the canonical base URL from `ingress.publicBaseUrl` in workspace config or module-level state (assistant-side; the gateway reads via `ConfigFileCache`) |
|
|
331
337
|
| `getTwilioVoiceWebhookUrl()` | `${base}/webhooks/twilio/voice?callSessionId=...`, using `ingress.publicBaseUrl` |
|
|
332
338
|
| `getTwilioStatusCallbackUrl()` | `${base}/webhooks/twilio/status`, using `ingress.publicBaseUrl` |
|
|
333
|
-
| `
|
|
334
|
-
| `getTwilioRelayUrl()` | `ws(s)://.../webhooks/twilio/relay`, using `ingress.publicBaseUrl` |
|
|
339
|
+
| `getTwilioMediaStreamUrl()` | `ws(s)://.../webhooks/twilio/media-stream`, using `ingress.publicBaseUrl` (per-call path segments appended at TwiML build time) |
|
|
335
340
|
| `getOAuthCallbackUrl()` | `${base}/webhooks/oauth/callback` |
|
|
336
341
|
| `getTelegramWebhookUrl()` | `${base}/webhooks/telegram` |
|
|
337
342
|
|
|
@@ -584,7 +589,7 @@ The channel inbound handler (`inbound-message-handler.ts`) enforces an access co
|
|
|
584
589
|
3. If a member exists but is not `active` (e.g., `revoked`, `blocked`), the message is denied.
|
|
585
590
|
4. If the member's `policy` is `deny`, the message is rejected. If `allow`, the message proceeds to normal processing. If `escalate`, the message is held for guardian approval.
|
|
586
591
|
|
|
587
|
-
**Invite-based onboarding:** Invite tokens are
|
|
592
|
+
**Invite-based onboarding:** Invite tokens are minted by the gateway via the invite HTTP API and stored SHA-256 hashed on the gateway DB's `ingress_invites` row -- the raw token is returned exactly once at creation time. External users redeem invites by sending the token as a channel message, which atomically creates a member record with `active` status and `allow` policy.
|
|
588
593
|
|
|
589
594
|
**Relationship to guardian verification:** Guardian verification and ingress contact management are independent systems. Guardian verification establishes who controls the assistant on a channel (the trust anchor for approvals and escalations). Ingress contacts control who can interact with the assistant. Escalation (`policy=escalate`) depends on a guardian binding existing for the channel -- without one, escalated messages are denied (fail-closed).
|
|
590
595
|
|
|
@@ -629,34 +634,38 @@ If no guardian binding exists for the channel, escalation fails closed -- the me
|
|
|
629
634
|
|
|
630
635
|
#### SQLite Tables
|
|
631
636
|
|
|
632
|
-
**Assistant DB** (`assistant.db` — current owner, migrating to gateway):
|
|
633
|
-
|
|
634
|
-
| Table | Purpose |
|
|
635
|
-
| --------------------------- | --------------------------------------------------------------------- |
|
|
636
|
-
| `assistant_ingress_invites` | Invite tokens with SHA-256 hashes, expiry, use counts |
|
|
637
|
-
| `contacts` | Contact records with role, relationship, and per-contact metadata |
|
|
638
|
-
| `contact_channels` | Channel bindings per contact with access policy (allow/deny/escalate) |
|
|
639
|
-
|
|
640
|
-
**Gateway DB** (`gateway.sqlite` — future owner of auth/authz):
|
|
637
|
+
**Assistant DB** (`assistant.db` — current owner of contact info, migrating to gateway):
|
|
641
638
|
|
|
642
639
|
| Table | Purpose |
|
|
643
640
|
| ------------------ | ---------------------------------------------------------------------- |
|
|
644
|
-
| `contacts` | Contact
|
|
645
|
-
| `contact_channels` | Channel bindings
|
|
641
|
+
| `contacts` | Contact records with role, relationship, and per-contact metadata |
|
|
642
|
+
| `contact_channels` | Channel bindings per contact with access policy (allow/deny/escalate) |
|
|
646
643
|
|
|
647
|
-
|
|
644
|
+
**Gateway DB** (`gateway.sqlite` — canonical owner of invites, future owner of auth/authz):
|
|
645
|
+
|
|
646
|
+
| Table | Purpose |
|
|
647
|
+
| ------------------ | ----------------------------------------------------------------------- |
|
|
648
|
+
| `contacts` | Contact auth/authz: id, display_name, role, principal_id |
|
|
649
|
+
| `contact_channels` | Channel bindings with policy, status, external IDs, verification state |
|
|
650
|
+
| `ingress_invites` | Canonical invite store — token/code hashes, expiry, use counts, voice/display fields |
|
|
651
|
+
|
|
652
|
+
The gateway's `ingress_invites` table is the sole invite store: mint, list, revoke, and redemption all run gateway-natively, and the daemon relays its invite surfaces here over IPC. The gateway data migrations `m0007`/`m0009` reference `assistant_ingress_invites` — a legacy assistant table absent from the current assistant schema — only as a one-time backfill source.
|
|
653
|
+
|
|
654
|
+
The gateway declares `contacts` and `contact_channels` tables and exposes them via IPC (`list_contacts`, `get_contact`, `get_contact_by_channel`, `get_channels_for_contact`). Contact endpoint cutover and data migration are in progress — the gateway will become the canonical owner once dual-writing is enabled.
|
|
648
655
|
|
|
649
656
|
#### Key Modules
|
|
650
657
|
|
|
651
|
-
| Module
|
|
652
|
-
|
|
|
653
|
-
| `
|
|
654
|
-
| `
|
|
655
|
-
| `
|
|
656
|
-
| `assistant/src/
|
|
657
|
-
| `assistant/src/
|
|
658
|
-
| `
|
|
659
|
-
| `
|
|
658
|
+
| Module | Purpose |
|
|
659
|
+
| --------------------------------------------------------- | -------------------------------------------------------------------------- |
|
|
660
|
+
| `gateway/src/http/routes/contacts-control-plane-proxy.ts` | Gateway-native invite lifecycle (mint, list, revoke, redeem) shared by HTTP and IPC |
|
|
661
|
+
| `gateway/src/ipc/invite-handlers.ts` | IPC routes relaying the daemon's invite surfaces to the native functions |
|
|
662
|
+
| `gateway/src/verification/invite-redemption.ts` | Redemption engine — validation, atomic claim, ACL activation |
|
|
663
|
+
| `assistant/src/contacts/contact-store.ts` | Contact and channel lookups (findContactChannel, guardian bindings) |
|
|
664
|
+
| `assistant/src/contacts/contacts-write.ts` | Contact and channel writes (upsert, policy changes, redemption info mirror) |
|
|
665
|
+
| `assistant/src/ipc/routes/invite-ipc-routes.ts` | `invite_redeemed` info mirror — local contact/channel identity upsert |
|
|
666
|
+
| `assistant/src/runtime/routes/inbound-message-handler.ts` | ACL enforcement point -- member lookup, policy check, escalation creation |
|
|
667
|
+
| `gateway/src/db/contact-store.ts` | Gateway-side ContactStore — contact/channel reads and invite CRUD |
|
|
668
|
+
| `gateway/src/ipc/contact-handlers.ts` | IPC route handlers for contact reads |
|
|
660
669
|
|
|
661
670
|
### Telegram Credential Flow
|
|
662
671
|
|
|
@@ -782,7 +791,7 @@ Any persistent-stream transport that does not buffer events for disconnected cli
|
|
|
782
791
|
|
|
783
792
|
## AI Phone Calls — Twilio Voice
|
|
784
793
|
|
|
785
|
-
The Calls subsystem supports both **outbound** and **inbound** voice calls via Twilio.
|
|
794
|
+
The Calls subsystem supports both **outbound** and **inbound** voice calls via Twilio. Every call connects over Twilio Media Streams (`<Connect><Stream>` TwiML): the daemon performs speech-to-text on the raw mu-law audio (streaming when the configured `services.stt` provider supports it, batch otherwise) and synthesizes speech via the configured `services.tts` provider, transcoded to mu-law frames. The assistant uses an LLM-driven conversation loop to speak in real time. Voice is a first-class channel with its own per-call conversation (outbound key: `asst:${assistantId}:voice:call:${callSessionId}`, inbound key: `asst:${assistantId}:voice:inbound:${callSid}`). When the AI needs guardian input during a call, it dispatches ASK_GUARDIAN requests cross-channel to mac/telegram via the guardian dispatch engine. Answer resolution uses first-writer-wins semantics -- the first channel to respond provides the answer, and remaining channels receive a "already answered" notice.
|
|
786
795
|
|
|
787
796
|
### Outbound Call Flow
|
|
788
797
|
|
|
@@ -794,7 +803,7 @@ sequenceDiagram
|
|
|
794
803
|
participant TwilioAPI as Twilio REST API
|
|
795
804
|
participant Gateway as Gateway (public)
|
|
796
805
|
participant Routes as twilio-routes.ts (runtime)
|
|
797
|
-
participant WS as
|
|
806
|
+
participant WS as MediaStreamCallSession (WebSocket)
|
|
798
807
|
participant Ctrl as CallController
|
|
799
808
|
participant Bridge as voice-session-bridge
|
|
800
809
|
participant RunOrch as RunOrchestrator
|
|
@@ -817,18 +826,20 @@ sequenceDiagram
|
|
|
817
826
|
Gateway->>Gateway: validateTwilioWebhookRequest()
|
|
818
827
|
Gateway->>Routes: forward to runtime /v1/internal/twilio/voice-webhook (+ params, originalUrl)
|
|
819
828
|
Routes->>CallStore: getCallSession()
|
|
820
|
-
Routes
|
|
829
|
+
Routes->>Routes: credential preflight (STT + TTS readiness)
|
|
830
|
+
Routes-->>Gateway: TwiML (Connect/Stream media-stream connect)
|
|
821
831
|
Gateway-->>TwilioAPI: TwiML response
|
|
822
832
|
|
|
823
|
-
TwilioAPI->>Gateway: WebSocket /webhooks/twilio/
|
|
824
|
-
Gateway->>WS: proxy WS to runtime /v1/calls/
|
|
825
|
-
WS->>WS:
|
|
833
|
+
TwilioAPI->>Gateway: WebSocket /webhooks/twilio/media-stream/{callSessionId}/{token}
|
|
834
|
+
Gateway->>WS: proxy WS to runtime /v1/calls/media-stream
|
|
835
|
+
WS->>WS: start event (streamSid, callSid)
|
|
836
|
+
WS->>WS: routeSetup() → setup outcome (CallSetupFlow for interactive outcomes)
|
|
826
837
|
WS->>Ctrl: new CallController()
|
|
827
838
|
Ctrl->>State: registerCallController()
|
|
828
839
|
|
|
829
840
|
loop Conversation turns
|
|
830
|
-
TwilioAPI->>WS:
|
|
831
|
-
WS->>WS:
|
|
841
|
+
TwilioAPI->>WS: media frames (mu-law audio)
|
|
842
|
+
WS->>WS: daemon STT (streaming or batch) → final transcript
|
|
832
843
|
WS->>Ctrl: handleCallerUtterance(transcript, speakerContext)
|
|
833
844
|
Ctrl->>Bridge: startVoiceTurn()
|
|
834
845
|
Bridge->>RunOrch: startRun(conversationId, content, {sourceChannel: 'phone', eventSink})
|
|
@@ -837,7 +848,7 @@ sequenceDiagram
|
|
|
837
848
|
LLM-->>Session: text tokens (streaming)
|
|
838
849
|
Session-->>Bridge: eventSink.onTextDelta()
|
|
839
850
|
Bridge-->>Ctrl: onTextDelta callback
|
|
840
|
-
Ctrl->>WS: sendTextToken() (
|
|
851
|
+
Ctrl->>WS: sendTextToken() (daemon TTS → mu-law media frames)
|
|
841
852
|
Ctrl->>CallStore: recordCallEvent()
|
|
842
853
|
end
|
|
843
854
|
|
|
@@ -867,7 +878,7 @@ sequenceDiagram
|
|
|
867
878
|
|
|
868
879
|
### Inbound Call Flow
|
|
869
880
|
|
|
870
|
-
Inbound calls are triggered when someone dials the assistant's Twilio phone number. The gateway resolves which assistant owns the number, the runtime bootstraps a session keyed by CallSid, and the
|
|
881
|
+
Inbound calls are triggered when someone dials the assistant's Twilio phone number. The gateway resolves which assistant owns the number, the runtime bootstraps a session keyed by CallSid, and the call setup flow gates the call (guardian verification, invite redemption, or name capture + guardian approval wait, as routed by `routeSetup`) before handing off to the CallController.
|
|
871
882
|
|
|
872
883
|
```mermaid
|
|
873
884
|
sequenceDiagram
|
|
@@ -877,7 +888,7 @@ sequenceDiagram
|
|
|
877
888
|
participant Routes as twilio-routes.ts (runtime)
|
|
878
889
|
participant CallDomain as CallDomain
|
|
879
890
|
participant CallStore as CallStore (SQLite)
|
|
880
|
-
participant WS as
|
|
891
|
+
participant WS as MediaStreamCallSession + CallSetupFlow (WebSocket)
|
|
881
892
|
participant GuardianSvc as ChannelGuardianService
|
|
882
893
|
participant Ctrl as CallController
|
|
883
894
|
participant Bridge as voice-session-bridge
|
|
@@ -904,24 +915,25 @@ sequenceDiagram
|
|
|
904
915
|
Routes->>CallDomain: createInboundVoiceSession(callSid, from, to, assistantId)
|
|
905
916
|
CallDomain->>CallStore: getOrCreateConversation(voice:inbound:${callSid})
|
|
906
917
|
CallDomain->>CallStore: createCallSession() — task=null for inbound
|
|
907
|
-
Routes
|
|
918
|
+
Routes->>Routes: credential preflight (STT + TTS readiness; Say + Hangup when not ready)
|
|
919
|
+
Routes-->>Gateway: TwiML (Connect/Stream media-stream connect)
|
|
908
920
|
Gateway-->>TwilioAPI: TwiML response
|
|
909
921
|
|
|
910
|
-
TwilioAPI->>Gateway: WebSocket /webhooks/twilio/
|
|
911
|
-
Gateway->>WS: proxy WS to runtime /v1/calls/
|
|
912
|
-
WS->>WS:
|
|
922
|
+
TwilioAPI->>Gateway: WebSocket /webhooks/twilio/media-stream/{callSessionId}/{token}
|
|
923
|
+
Gateway->>WS: proxy WS to runtime /v1/calls/media-stream
|
|
924
|
+
WS->>WS: start event (streamSid, callSid)
|
|
913
925
|
WS->>WS: detect isInbound (`session.initiatedFromConversationId == null`)
|
|
926
|
+
WS->>WS: routeSetup() → setup outcome
|
|
914
927
|
|
|
915
928
|
alt Pending voice guardian challenge exists
|
|
916
|
-
WS->>
|
|
917
|
-
WS->>WS: enter verification_pending state
|
|
929
|
+
WS->>WS: CallSetupFlow enters collecting_code state
|
|
918
930
|
WS->>Caller: TTS "Please enter your six-digit verification code"
|
|
919
931
|
loop DTMF / spoken digit attempts (max 3)
|
|
920
932
|
Caller->>WS: DTMF digits or spoken digits
|
|
921
933
|
WS->>GuardianSvc: validateAndConsumeVerification(code)
|
|
922
934
|
alt Code matches
|
|
923
935
|
GuardianSvc-->>WS: success + guardian binding created
|
|
924
|
-
WS->>Ctrl:
|
|
936
|
+
WS->>Ctrl: new CallController() + initial greeting
|
|
925
937
|
else Code incorrect + attempts remaining
|
|
926
938
|
WS->>Caller: TTS "That code was incorrect. Please try again."
|
|
927
939
|
else Max attempts exceeded
|
|
@@ -931,7 +943,7 @@ sequenceDiagram
|
|
|
931
943
|
end
|
|
932
944
|
end
|
|
933
945
|
else No pending guardian challenge
|
|
934
|
-
WS->>Ctrl:
|
|
946
|
+
WS->>Ctrl: new CallController() + initial greeting
|
|
935
947
|
end
|
|
936
948
|
|
|
937
949
|
Ctrl->>Bridge: startVoiceTurn([CALL_OPENING])
|
|
@@ -942,10 +954,11 @@ sequenceDiagram
|
|
|
942
954
|
LLM-->>Session: receptionist-style greeting
|
|
943
955
|
Session-->>Bridge: eventSink.onTextDelta()
|
|
944
956
|
Bridge-->>Ctrl: onTextDelta callback
|
|
945
|
-
Ctrl->>WS: sendTextToken() (TTS to caller)
|
|
957
|
+
Ctrl->>WS: sendTextToken() (daemon TTS → mu-law frames to caller)
|
|
946
958
|
|
|
947
959
|
loop Conversation turns
|
|
948
|
-
Caller->>WS:
|
|
960
|
+
Caller->>WS: media frames (mu-law audio)
|
|
961
|
+
WS->>WS: daemon STT (streaming or batch) → final transcript
|
|
949
962
|
WS->>Ctrl: handleCallerUtterance(transcript, speakerContext)
|
|
950
963
|
Ctrl->>Bridge: startVoiceTurn()
|
|
951
964
|
Bridge->>RunOrch: startRun(conversationId, content, {sourceChannel: 'phone', eventSink})
|
|
@@ -954,12 +967,12 @@ sequenceDiagram
|
|
|
954
967
|
LLM-->>Session: text tokens (streaming)
|
|
955
968
|
Session-->>Bridge: eventSink.onTextDelta()
|
|
956
969
|
Bridge-->>Ctrl: onTextDelta callback
|
|
957
|
-
Ctrl->>WS: sendTextToken() (
|
|
970
|
+
Ctrl->>WS: sendTextToken() (daemon TTS → mu-law media frames)
|
|
958
971
|
Ctrl->>CallStore: recordCallEvent()
|
|
959
972
|
end
|
|
960
973
|
```
|
|
961
974
|
|
|
962
|
-
**Inbound vs. outbound detection**: The
|
|
975
|
+
**Inbound vs. outbound detection**: The media-stream server determines call direction by checking `session.initiatedFromConversationId`. Outbound calls are initiated from an existing conversation (`initiatedFromConversationId` set). Inbound calls are bootstrapped from Twilio webhooks and therefore have `initiatedFromConversationId == null`.
|
|
963
976
|
|
|
964
977
|
**Inbound system prompt**: The session pipeline (via voice-session-bridge) generates system prompts appropriate for the voice channel context. For inbound calls, this produces a receptionist-style prompt that greets the caller warmly and helps them with what they need.
|
|
965
978
|
|
|
@@ -979,11 +992,15 @@ sequenceDiagram
|
|
|
979
992
|
| `assistant/src/calls/call-state-machine.ts` | Deterministic state transition validator with allowed-transition table and terminal-state enforcement |
|
|
980
993
|
| `assistant/src/calls/call-recovery.ts` | Startup reconciliation of non-terminal calls: fetches provider status and transitions stale sessions |
|
|
981
994
|
| `assistant/src/calls/twilio-provider.ts` | Twilio Voice REST API integration (initiateCall, endCall, getCallStatus) using direct fetch — no Twilio SDK dependency |
|
|
982
|
-
| `assistant/src/calls/twilio-routes.ts` | HTTP webhook handlers: voice webhook (returns TwiML with
|
|
983
|
-
| `assistant/src/calls/
|
|
995
|
+
| `assistant/src/calls/twilio-routes.ts` | HTTP webhook handlers: voice webhook (returns `<Connect><Stream>` TwiML, enforces the credential preflight with `<Say>` + `<Hangup/>` when not ready), status callback |
|
|
996
|
+
| `assistant/src/calls/media-stream-server.ts` | WebSocket handler for Twilio Media Streams; manages one MediaStreamCallSession per call, runs `routeSetup`, and drives interactive setup outcomes through `CallSetupFlow` |
|
|
997
|
+
| `assistant/src/calls/call-setup-flow.ts` | Transport-agnostic call setup flow: verification, invite-redemption, name-capture, and unverified-caller sub-flows over DTMF/spoken input |
|
|
998
|
+
| `assistant/src/calls/guardian-wait-controller.ts` | Guardian access-request wait orchestration: hold messaging, heartbeats, status polling, consultation timeout, callback handoff |
|
|
999
|
+
| `assistant/src/calls/media-stream-stt-session.ts` | Daemon-side STT for media-stream audio: streaming transcriber (utterance-boundary finals) with batch + VAD turn-detection fallback |
|
|
1000
|
+
| `assistant/src/calls/telephony-credential-preflight.ts` | Combined STT + TTS credential-readiness resolver gating inbound TwiML and outbound call placement |
|
|
984
1001
|
| `assistant/src/calls/speaker-identification.ts` | Reusable speaker recognition primitive for voice prompts: extracts provider speaker metadata (top-level and nested fields), resolves stable per-call speaker identities, and emits speaker context for personalization |
|
|
985
1002
|
| `assistant/src/calls/call-controller.ts` | Session-backed voice controller: routes voice turns through the daemon session pipeline via voice-session-bridge, detects ASK_GUARDIAN and END_CALL control markers |
|
|
986
|
-
| `assistant/src/calls/voice-session-bridge.ts` | Bridge between voice
|
|
1003
|
+
| `assistant/src/calls/voice-session-bridge.ts` | Bridge between the voice call controller and the daemon session/run pipeline: wraps RunOrchestrator.startRun() with voice-specific defaults, translating agent-loop events into callbacks for real-time TTS streaming |
|
|
987
1004
|
| `assistant/src/calls/call-state.ts` | Notifier pattern (Maps with register/unregister/fire helpers) for cross-component communication: question notifiers, completion notifiers, and controller registry |
|
|
988
1005
|
| `assistant/src/calls/call-constants.ts` | Config-backed constants: max call duration, user consultation timeout, silence timeout, denied emergency numbers |
|
|
989
1006
|
| `assistant/src/calls/voice-provider.ts` | Abstract VoiceProvider interface for provider-agnostic call initiation |
|
|
@@ -992,9 +1009,7 @@ sequenceDiagram
|
|
|
992
1009
|
| `assistant/src/calls/types.ts` | TypeScript type definitions: CallSession, CallEvent, CallPendingQuestion, CallStatus, CallEventType |
|
|
993
1010
|
| `gateway/src/http/routes/twilio-voice-webhook.ts` | Gateway route: validates Twilio signature, forwards voice webhook to runtime |
|
|
994
1011
|
| `gateway/src/http/routes/twilio-status-webhook.ts` | Gateway route: validates Twilio signature, forwards status callback to runtime |
|
|
995
|
-
| `gateway/src/http/routes/twilio-
|
|
996
|
-
| `gateway/src/http/routes/twilio-relay-websocket.ts` | Gateway route: WebSocket proxy for ConversationRelay frames between Twilio and runtime (used for Deepgram/Google native STT) |
|
|
997
|
-
| `gateway/src/http/routes/twilio-media-websocket.ts` | Gateway route: WebSocket proxy for Media Streams frames between Twilio and runtime (used for OpenAI Whisper media-stream STT) |
|
|
1012
|
+
| `gateway/src/http/routes/twilio-media-websocket.ts` | Gateway route: WebSocket proxy for Media Streams frames between Twilio and runtime (all calls) |
|
|
998
1013
|
| `gateway/src/twilio/validate-webhook.ts` | Twilio webhook validation: HMAC-SHA1 signature verification, payload size limits, fail-closed when auth token missing |
|
|
999
1014
|
|
|
1000
1015
|
### Call State Machine
|
|
@@ -1067,11 +1082,9 @@ Internet-facing Twilio callbacks terminate at the gateway, which validates signa
|
|
|
1067
1082
|
| ---------------------------------------------------------- | --------------------------------- | ---------------------------------------------------------------------------------------- |
|
|
1068
1083
|
| `POST /webhooks/twilio/voice` | HMAC-SHA1 signature, payload size | `POST /v1/internal/twilio/voice-webhook` (JSON: `{ params, originalUrl, assistantId? }`) |
|
|
1069
1084
|
| `POST /webhooks/twilio/status` | HMAC-SHA1 signature, payload size | `POST /v1/internal/twilio/status` (JSON: `{ params }`) |
|
|
1070
|
-
| `POST /webhooks/twilio/connect-action` | HMAC-SHA1 signature, payload size | `POST /v1/internal/twilio/connect-action` (JSON: `{ params }`) |
|
|
1071
|
-
| `WS /webhooks/twilio/relay` | WebSocket upgrade | `WS /v1/calls/relay` (bidirectional proxy) — ConversationRelay path |
|
|
1072
1085
|
| `WS /webhooks/twilio/media-stream/<callSessionId>/<token>` | WebSocket upgrade | `WS /v1/calls/media-stream` (bidirectional proxy) — Media Streams path |
|
|
1073
1086
|
|
|
1074
|
-
In gateway-fronted deployments, the TwiML WebSocket URL (returned by the voice webhook)
|
|
1087
|
+
In gateway-fronted deployments, the TwiML WebSocket URL (returned by the voice webhook) points to the gateway's `/webhooks/twilio/media-stream/<callSessionId>/<token>` endpoint rather than directly to the runtime. The gateway proxies frames bidirectionally between Twilio and the runtime, preserving close and error semantics for proper cleanup.
|
|
1075
1088
|
|
|
1076
1089
|
**Media Streams handshake metadata:** Twilio Media Streams does not reliably preserve URL query parameters across the WebSocket upgrade, so handshake metadata (`callSessionId` and auth `token`) is encoded as **URL path segments** (primary transport). The gateway also supports legacy query-parameter-based handshake as a fallback for backward compatibility. The metadata extractor in `twilio-media-websocket.ts` resolves values from path segments first, falling back to query parameters.
|
|
1077
1090
|
|
|
@@ -1079,7 +1092,7 @@ Signature validation is **fail-closed**: if the Twilio auth token is not configu
|
|
|
1079
1092
|
|
|
1080
1093
|
**Webhook base URL resolution:** Public ingress URL construction is centralized in `public-ingress-urls.ts`:
|
|
1081
1094
|
|
|
1082
|
-
- Twilio voice/status/
|
|
1095
|
+
- Twilio voice/status/media-stream URLs use `ingress.publicBaseUrl`.
|
|
1083
1096
|
- Velay registration publishes its public assistant URL to `ingress.publicBaseUrl` with `ingress.publicBaseUrlManagedBy: "velay"`.
|
|
1084
1097
|
- Telegram webhooks, OAuth callbacks, email callbacks, and normal JSON webhook URLs also use `ingress.publicBaseUrl`; Velay-managed URL changes are tagged so unrelated reconciliation can be skipped when appropriate.
|
|
1085
1098
|
- Module-level assistant state remains a fallback for legacy tunnel start/stop flows.
|
|
@@ -1106,9 +1119,7 @@ This makes ingress URL updates smoother in local tunnel workflows because Twilio
|
|
|
1106
1119
|
| POST | `/v1/calls/:callSessionId/answer` | Answer a pending question via HTTP (alternative to in-conversation bridge) |
|
|
1107
1120
|
| POST | `/v1/calls/:callSessionId/instruction` | Relay a steering instruction to an active call's controller (alternative to in-conversation bridge) |
|
|
1108
1121
|
| POST | `/v1/internal/twilio/status` | Internal status callback used by gateway; accepts JSON `{ params }` |
|
|
1109
|
-
|
|
|
1110
|
-
| WS | `/v1/calls/relay` | ConversationRelay WebSocket (bidirectional: prompt/interrupt/dtmf from Twilio, text tokens/end to Twilio) — Deepgram/Google path |
|
|
1111
|
-
| WS | `/v1/calls/media-stream` | Media Streams WebSocket (raw audio from Twilio, daemon-side STT) — OpenAI Whisper path |
|
|
1122
|
+
| WS | `/v1/calls/media-stream` | Media Streams WebSocket (raw audio from Twilio, daemon-side STT/TTS) — all calls |
|
|
1112
1123
|
|
|
1113
1124
|
### Tools
|
|
1114
1125
|
|
|
@@ -1118,7 +1129,7 @@ This makes ingress URL updates smoother in local tunnel workflows because Twilio
|
|
|
1118
1129
|
| `call_status` | Retrieves the current status of a call session |
|
|
1119
1130
|
| `call_end` | Terminates an active call |
|
|
1120
1131
|
|
|
1121
|
-
Both tools and HTTP routes delegate to the same domain functions in `call-domain.ts` (`startCall`, `getCallStatus`, `cancelCall`, `answerCall`, `relayInstruction`), ensuring consistent validation and behavior. Inbound calls do not use tools — they are initiated by the external caller and bootstrapped automatically by the voice webhook and
|
|
1132
|
+
Both tools and HTTP routes delegate to the same domain functions in `call-domain.ts` (`startCall`, `getCallStatus`, `cancelCall`, `answerCall`, `relayInstruction`), ensuring consistent validation and behavior. Inbound calls do not use tools — they are initiated by the external caller and bootstrapped automatically by the voice webhook and media-stream server.
|
|
1122
1133
|
|
|
1123
1134
|
### Control Markers
|
|
1124
1135
|
|
|
@@ -1155,7 +1166,7 @@ Call behavior is controlled via the `calls` config block in the assistant config
|
|
|
1155
1166
|
| `calls.safety.denyCategories` | string[] | `[]` | Categories of calls to deny (e.g., emergency numbers are always denied regardless of this setting). |
|
|
1156
1167
|
| `llm.callSites.callAgent.model` | string | _(unset — falls back to `llm.default.model`)_ | Optional override for the LLM model used in voice call conversations. |
|
|
1157
1168
|
| `calls.voice.language` | string | `'en-US'` | Language code for TTS and transcription. |
|
|
1158
|
-
| `services.stt.provider` | enum | `'deepgram'` | STT provider for all boundaries including telephony.
|
|
1169
|
+
| `services.stt.provider` | enum | `'deepgram'` | STT provider for all boundaries including telephony. The daemon transcribes media-stream call audio with this provider (streaming when supported, batch otherwise). |
|
|
1159
1170
|
| `services.tts.provider` | enum | `'elevenlabs'` | Active TTS provider for speech synthesis (catalog-driven; see [TTS Provider Abstraction](../assistant/ARCHITECTURE.md#tts-provider-abstraction-servicestts)). |
|
|
1160
1171
|
| `services.tts.providers.<id>.*` | object | _(per-provider defaults)_ | Provider-specific settings block. One block per catalog entry (e.g. `elevenlabs`, `fish-audio`). |
|
|
1161
1172
|
|
|
@@ -1178,20 +1189,19 @@ Voice and TTS settings are configurable via the `calls.voice` and `services.tts`
|
|
|
1178
1189
|
|
|
1179
1190
|
The active TTS provider is determined by `services.tts.provider` (default: `"elevenlabs"`). Provider-specific settings (voice ID, model, tuning parameters) are read from `services.tts.providers.<id>`. The call mode (`native-twilio` or `synthesized-play`) is resolved from the canonical provider catalog via `resolveCallStrategy()` in `tts-call-strategy.ts` — it reads the provider's declared `callMode` rather than inferring behavior from runtime capabilities.
|
|
1180
1191
|
|
|
1181
|
-
For `native-twilio` providers (e.g. ElevenLabs), the voice quality profile looks up a registered `NativeTwilioVoiceSpecBuilder` to construct the provider-specific voice spec string
|
|
1192
|
+
For `native-twilio` providers (e.g. ElevenLabs), the voice quality profile looks up a registered `NativeTwilioVoiceSpecBuilder` to construct the provider-specific voice spec string; on the media-stream transport spoken text sent via `sendTextToken()` is re-synthesized through daemon TTS, so this mode no longer routes to Twilio's built-in TTS — collapsing the callMode split is a documented deferred follow-up. For `synthesized-play` providers (e.g. Fish Audio), the assistant synthesises audio via the provider's HTTP API and delivers it through the audio store / `sendPlayUrl()` path.
|
|
1182
1193
|
|
|
1183
|
-
|
|
1194
|
+
On the media-stream transport, call playback requires audio the daemon can transcode to mu-law: each TTS catalog entry declares `mediaStreamPlayback.outputFormat`, `resolveTelephonyTtsCapability()` combines that with credential availability, and the call TTS resolver falls back to a credentialed playable provider rather than producing silence.
|
|
1184
1195
|
|
|
1185
1196
|
For full details on the catalog-driven TTS architecture, provider catalog, call strategy abstraction, and the provider-add checklist, see the [TTS Provider Abstraction](../assistant/ARCHITECTURE.md#tts-provider-abstraction-servicestts) section in the assistant architecture docs.
|
|
1186
1197
|
|
|
1187
|
-
### Telephony STT:
|
|
1188
|
-
|
|
1189
|
-
Telephony STT is unified under `services.stt.provider`. The voice webhook in `twilio-routes.ts` calls `resolveTelephonySttRouting()` to determine the Twilio integration path based on the active provider:
|
|
1198
|
+
### Telephony STT: Daemon-Side Media-Stream Transcription
|
|
1190
1199
|
|
|
1191
|
-
|
|
1200
|
+
Telephony STT is unified under `services.stt.provider`. The voice webhook in `twilio-routes.ts` emits `<Connect><Stream>` TwiML pointing to the gateway's media-stream proxy (`/webhooks/twilio/media-stream/<callSessionId>/<token>`); the gateway forwards raw audio frames to the daemon's media-stream server, which transcribes them itself:
|
|
1192
1201
|
|
|
1193
|
-
- **
|
|
1202
|
+
- **Streaming** (default) — when `calls.voice.telephonyStreaming` is enabled and the configured provider resolves a streaming transcriber, audio is decoded (mu-law → PCM16, 8 kHz → 16 kHz) and fed to the provider's realtime adapter; replies trigger only on utterance-boundary finals and barge-in fires from local energy VAD.
|
|
1203
|
+
- **Batch fallback** — otherwise turns are segmented with the energy-based VAD turn detector and transcribed via the provider's batch API.
|
|
1194
1204
|
|
|
1195
|
-
|
|
1205
|
+
A credential preflight (`resolveTelephonyCredentialReadiness()`) requires a credentialed telephony-capable STT provider and a media-stream-playable TTS provider before any call connects: inbound not-ready calls get `<Say>` setup-required copy plus `<Hangup/>`, and outbound placement fails before dialing. See `docs/internal-reference.md` for a provider-specific troubleshooting matrix.
|
|
1196
1206
|
|
|
1197
1207
|
---
|
package/README.md
CHANGED
|
@@ -80,7 +80,7 @@ When the voice webhook is called without a `callSessionId` query parameter, the
|
|
|
80
80
|
1. **`resolveAssistantByPhoneNumber(config, To)`** — Reverse lookup of the inbound `To` number against `assistantPhoneNumbers`. If the dialed number matches an assistant's configured phone number, that assistant handles the call.
|
|
81
81
|
2. **Fallback to `resolveAssistant(From, From)`** — If no phone number match is found, the standard routing chain is used: `conversation_id` match, `actor_id` match, then the unmapped policy.
|
|
82
82
|
3. **TwiML Reject for unmapped** — When the unmapped policy is `reject` (and no route matches), the gateway returns `<Reject reason="rejected"/>` TwiML directly to Twilio. Twilio plays a busy signal and hangs up. The call is never forwarded to the runtime.
|
|
83
|
-
4. **Forward with assistantId** — When routing succeeds, the gateway forwards the voice webhook to the runtime at `POST /v1/internal/twilio/voice-webhook` with a JSON body containing `{ params, originalUrl, assistantId }`. The runtime calls `createInboundVoiceSession()` to bootstrap a session keyed by CallSid, then returns TwiML pointing Twilio to the
|
|
83
|
+
4. **Forward with assistantId** — When routing succeeds, the gateway forwards the voice webhook to the runtime at `POST /v1/internal/twilio/voice-webhook` with a JSON body containing `{ params, originalUrl, assistantId }`. The runtime calls `createInboundVoiceSession()` to bootstrap a session keyed by CallSid, then returns `<Connect><Stream>` TwiML pointing Twilio to the gateway's media-stream WebSocket proxy (after a daemon-side STT/TTS credential preflight; calls that fail it get `<Say>` setup-required copy plus `<Hangup/>`).
|
|
84
84
|
|
|
85
85
|
### Inbound call lifecycle (gateway perspective)
|
|
86
86
|
|
|
@@ -88,9 +88,9 @@ When the voice webhook is called without a `callSessionId` query parameter, the
|
|
|
88
88
|
Caller → Twilio → Gateway /webhooks/twilio/voice (no callSessionId)
|
|
89
89
|
→ resolveAssistantByPhoneNumber(To) || resolveAssistant(From) || TwiML Reject
|
|
90
90
|
→ forward to runtime /v1/internal/twilio/voice-webhook (JSON: { params, originalUrl, assistantId })
|
|
91
|
-
→ runtime returns TwiML (
|
|
92
|
-
→ Twilio opens WebSocket → Gateway /webhooks/twilio/
|
|
93
|
-
→
|
|
91
|
+
→ runtime returns TwiML (<Connect><Stream> media-stream connect)
|
|
92
|
+
→ Twilio opens WebSocket → Gateway /webhooks/twilio/media-stream/<callSessionId>/<token> → Runtime /v1/calls/media-stream
|
|
93
|
+
→ media-stream server detects inbound (`initiatedFromConversationId == null`), runs the call setup flow (verification / invite / name-capture sub-flows as routed), then receptionist-style LLM greeting
|
|
94
94
|
```
|
|
95
95
|
|
|
96
96
|
## Callback Query Handling
|
|
@@ -151,8 +151,7 @@ The gateway serves as the single public ingress point for all external callbacks
|
|
|
151
151
|
| `/deliver/telegram` | POST | Internal endpoint for the assistant runtime to deliver outbound messages/attachments to Telegram chats |
|
|
152
152
|
| `/webhooks/twilio/voice` | POST | Twilio voice webhook (validated via HMAC-SHA1 signature) |
|
|
153
153
|
| `/webhooks/twilio/status` | POST | Twilio status callback (validated via HMAC-SHA1 signature) |
|
|
154
|
-
| `/webhooks/twilio/
|
|
155
|
-
| `/webhooks/twilio/relay` | WS | Twilio ConversationRelay WebSocket (bidirectional proxy to runtime, requires `callSessionId` query param) |
|
|
154
|
+
| `/webhooks/twilio/media-stream/:callSessionId/:token` | WS | Twilio Media Streams WebSocket (bidirectional proxy to runtime; handshake metadata in URL path segments) |
|
|
156
155
|
| `/webhooks/oauth/callback` | GET | OAuth2 callback endpoint — receives authorization codes from OAuth providers (Google, Slack, etc.) and forwards them to the assistant runtime |
|
|
157
156
|
| `/v1/channel-verification-sessions` | POST | Authenticated control-plane proxy for creating verification sessions (inbound challenge or outbound verification) |
|
|
158
157
|
| `/v1/channel-verification-sessions` | DELETE | Authenticated control-plane proxy for cancelling active verification sessions |
|
|
@@ -166,9 +165,10 @@ The gateway serves as the single public ingress point for all external callbacks
|
|
|
166
165
|
| `/v1/contacts/:id` | GET | Authenticated control-plane proxy for retrieving a contact by ID |
|
|
167
166
|
| `/v1/contacts/merge` | POST | Authenticated control-plane proxy for merging two contacts |
|
|
168
167
|
| `/v1/contact-channels/:contactChannelId` | PATCH | Authenticated control-plane proxy for updating a contact channel's status/policy |
|
|
169
|
-
| `/v1/contacts/invites` | GET/POST |
|
|
170
|
-
| `/v1/contacts/invites/:id` | DELETE |
|
|
171
|
-
| `/v1/contacts/invites/
|
|
168
|
+
| `/v1/contacts/invites` | GET/POST | Gateway-native invite list/create against the gateway DB's `ingress_invites` table |
|
|
169
|
+
| `/v1/contacts/invites/:id` | DELETE | Gateway-native invite revoke |
|
|
170
|
+
| `/v1/contacts/invites/:id/call` | POST | Gateway-native invite-call relay — validates the invite row, then delegates the provider call to the assistant |
|
|
171
|
+
| `/v1/contacts/invites/redeem` | POST | Gateway-native invite redemption (voice code and link token) |
|
|
172
172
|
| `/v1/health` | GET | Authenticated runtime health proxy (`/v1/health` on runtime) |
|
|
173
173
|
| `/healthz` | GET | Liveness probe |
|
|
174
174
|
| `/readyz` | GET | Readiness probe |
|
|
@@ -212,7 +212,7 @@ The assistant runtime uses this URL to construct all webhook and OAuth callback
|
|
|
212
212
|
|
|
213
213
|
### Velay for Twilio Testing
|
|
214
214
|
|
|
215
|
-
Velay is a managed ingress transport for assistant-hosted HTTP and WebSocket traffic. The gateway starts the Velay tunnel only after Twilio setup has been started in the workspace, or when existing Twilio config shows it was set up before. When Velay registration succeeds, the gateway writes the registered public assistant URL to `ingress.publicBaseUrl` and marks it with `ingress.publicBaseUrlManagedBy: "velay"`. Twilio URL builders use that public base URL for voice, status,
|
|
215
|
+
Velay is a managed ingress transport for assistant-hosted HTTP and WebSocket traffic. The gateway starts the Velay tunnel only after Twilio setup has been started in the workspace, or when existing Twilio config shows it was set up before. When Velay registration succeeds, the gateway writes the registered public assistant URL to `ingress.publicBaseUrl` and marks it with `ingress.publicBaseUrlManagedBy: "velay"`. Twilio URL builders use that public base URL for voice, status, and media-stream endpoints.
|
|
216
216
|
|
|
217
217
|
Use Velay when testing Twilio voice webhooks or Twilio WebSocket upgrades through the platform-managed tunnel:
|
|
218
218
|
|
|
@@ -247,10 +247,10 @@ For a synthetic Twilio WebSocket smoke test, connect a local WebSocket client to
|
|
|
247
247
|
|
|
248
248
|
```bash
|
|
249
249
|
bun -e 'const ws = new WebSocket(process.argv[1]); ws.onopen = () => { console.log("open"); ws.close(); }; ws.onerror = (event) => console.error(event);' \
|
|
250
|
-
"wss://<velay-host>/<assistant-id>/webhooks/twilio/
|
|
250
|
+
"wss://<velay-host>/<assistant-id>/webhooks/twilio/media-stream/session-123/<edge-token>"
|
|
251
251
|
```
|
|
252
252
|
|
|
253
|
-
For a real Twilio call, expose local Velay with a public HTTPS/WSS tunnel and configure the platform Velay service with that origin as `VELAY_PUBLIC_BASE_URL`. After the assistant re-registers, Twilio should fetch `/webhooks/twilio/voice` and open `/webhooks/twilio/
|
|
253
|
+
For a real Twilio call, expose local Velay with a public HTTPS/WSS tunnel and configure the platform Velay service with that origin as `VELAY_PUBLIC_BASE_URL`. After the assistant re-registers, Twilio should fetch `/webhooks/twilio/voice` and open `/webhooks/twilio/media-stream/...` through the Velay URL. Use ngrok or another custom tunnel in `ingress.publicBaseUrl` only for local/self-hosted workflows that are not routed through Velay.
|
|
254
254
|
|
|
255
255
|
## Ingress Boundary Guarantees
|
|
256
256
|
|