@vellumai/assistant 0.12.2-dev.202609181615.645c0a5 → 0.12.2-dev.202609181713.fb1ff61

package/ARCHITECTURE.md

@@ -614,6 +614,8 @@ Transcription mode is selected once per session in `media-stream-stt-session.ts`
 
 Every phone turn runs the same two-leg triage as live voice through `startVoiceTurn` (`src/calls/voice-session-bridge.ts`): `call-controller.ts` opens on a toolless front-door leg (`routingLeg: "front-door"`, the `voiceFrontDoor` call site) and drives it through the shared `createFrontDoorLegCoordinator` (`src/calls/voice-leg-coordinator.ts`), which reads the stream through the verdict machine and sequences the hand-off (pause narration, abort the leg, resolve and speak the bridge, mark it as the floor holder, start the escalated leg pinned to the conversation's own model, re-arm narration); each driver supplies only a host for how text and the bridge are spoken, how a leg is started or aborted, and (live voice only) the speculative hold and commit. Phone has no partial transcripts, so the hold verdict is never taught and routing is escalate-only. The controller also passes the bridge's turn callbacks (tool activity is recorded as `tool_use_started` / `tool_use_completed` call events, persisted row ids ride the `assistant_spoke` event), `launchedAtMs` for dispatch timing, and a `voiceTelemetry` bag keyed by the call session with a `phone_inbound` / `phone_outbound` entry. Both drivers share the spoken progress narration cadence (`src/calls/voice-progress-cadence.ts`, tuned by `voice.frontModel.progress`): the cadence owns the tool-activity log, the triggers (an ops burst, a long operation completing, a full interval of audible silence with news, the `maxSilenceMs` heartbeat) and the generated or static phrase, while each driver supplies its own view of audible silence (live voice from its TTS queue and playback-tail estimate; the media-stream transport from `isPlaybackIdle()` and a running sum of sent frame durations) and how to speak a phrase.
 
+Both drivers also share the per-turn latency marks (`src/calls/voice-metrics.ts`). The controller opens a turn's marks when it dispatches, seeded with when the caller's transcript arrived, and stamps the leg dispatch, the first assistant delta, the first audible audio, a barge-in, and how the turn settled. Live voice streams a snapshot to its client on every mark; a call has no client on the line, so the aggregate is logged once per turn, cancelled turns included. Each call also records a `phone_call_started` / `phone_call_ended` pair on the onboarding telemetry substrate (`src/telemetry/phone-call-funnel.ts`), written by the call store at session creation and at the single terminal status transition: duration is the gap between the two rows, and a call that took no caller turn carries why on the end row (`silent_no_connect` when it never connected, `silent_no_turn` when it did and nobody spoke).
+
 A credential preflight (`resolveTelephonyCredentialReadiness()` in `src/calls/telephony-credential-preflight.ts`) gates every call: it requires a credentialed, telephony-capable STT provider **and** a media-stream-playable TTS provider (the configured one or a credentialed playable fallback). Inbound calls that fail the preflight receive `<Say>` setup-required copy plus `<Hangup/>` instead of a doomed stream; outbound placement fails before dialing via `preflightVoiceIngress()` with the same user-facing message.
 
 Key modules:
@@ -623,6 +625,7 @@ Key modules:
 | `src/calls/twilio-routes.ts`                        | Voice webhook handler; generates `<Connect><Stream>` TwiML, enforces the inbound credential preflight |
 | `src/calls/telephony-credential-preflight.ts`       | Combined STT + TTS credential-readiness resolver                                                      |
 | `src/calls/media-stream-parser.ts`                  | Twilio Media Streams protocol parser                                                                  |
+| `src/calls/voice-metrics.ts`                        | Per-turn latency marks, shared with live voice                                                        |
 | `src/calls/media-turn-detector.ts`                  | Energy-based VAD turn detector for raw audio (batch mode)                                             |
 | `src/calls/media-stream-stt-session.ts`             | STT session — streaming/batch mode selection and transcription via `services.stt`                     |
 | `src/calls/media-stream-audio-transcode.ts`         | Mu-law ↔ PCM16 codecs and resampling                                                                  |
@@ -729,7 +732,7 @@ The assistant-side live voice module is intentionally bounded under `src/live-vo
 | `live-voice-session.ts`         | Session orchestration: streaming STT, push-to-talk release, voice turn bridge callbacks, assistant text deltas, TTS, archive, metrics, interrupt, and close |
 | `live-voice-tts.ts`             | Streaming TTS helper that resolves `services.tts`, requires `TtsProvider.synthesizeStream()`, and forwards audio chunks as `tts_audio` frames               |
 | `live-voice-archive.ts`         | Audio artifact creation/linking for user utterance and assistant response message IDs                                                                       |
-| `live-voice-metrics.ts`         | Per-session and per-turn latency snapshots emitted as `metrics` frames                                                                                      |
+| `../calls/voice-metrics.ts`     | Shared per-session and per-turn latency marks; live voice emits each snapshot as a `metrics` frame, phone calls log the per-turn aggregate                  |
 
 Live voice STT uses the same `resolveStreamingTranscriber()` path as conversation streaming, dialed with the provider the `liveVoice` STT role resolves to after managed-speech defaulting (`resolveEffectiveSpeechProviders`, see `config/managed-speech-defaults.ts`). For V1 latency-sensitive behavior, that provider must resolve to a `daemon-streaming` transcriber whose catalog entry has `conversationStreamingMode: "realtime-ws"` and usable credentials. Providers that only support batch or incremental-batch transcription remain valid for other voice surfaces, but do not satisfy live voice's streaming STT requirement.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@vellumai/assistant",
-  "version": "0.12.2-dev.202609181615.645c0a5",
+  "version": "0.12.2-dev.202609181713.fb1ff61",
   "license": "MIT",
   "type": "module",
   "exports": {

package/src/__tests__/call-conversation-messages.test.ts

@@ -102,6 +102,7 @@ describe("call-conversation-messages", () => {
     const conversationId = "conv-call-msg-failed";
     ensureConversation(conversationId);
     const session = createCallSession({
+      direction: "inbound",
       conversationId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -122,6 +123,7 @@ describe("call-conversation-messages", () => {
     const conversationId = "conv-call-msg-cancelled";
     ensureConversation(conversationId);
     const session = createCallSession({
+      direction: "inbound",
       conversationId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -142,6 +144,7 @@ describe("call-conversation-messages", () => {
     const conversationId = "conv-call-msg-completed";
     ensureConversation(conversationId);
     const session = createCallSession({
+      direction: "inbound",
       conversationId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -171,6 +174,7 @@ describe("call-conversation-messages", () => {
     const conversationId = "conv-call-msg-fallback";
     ensureConversation(conversationId);
     const session = createCallSession({
+      direction: "inbound",
       conversationId,
       provider: "twilio",
       fromNumber: "+12025550101",
@@ -208,6 +212,7 @@ describe("call-conversation-messages", () => {
     const conversationId = "conv-call-msg-schema";
     ensureConversation(conversationId);
     const session = createCallSession({
+      direction: "inbound",
       conversationId,
       provider: "twilio",
       fromNumber: "+12025550101",

package/src/__tests__/call-recovery.test.ts

@@ -54,9 +54,14 @@ function resetTables() {
   ensuredConvIds = new Set();
 }
 
-function createTestCallSession(opts: Parameters<typeof createCallSession>[0]) {
+function createTestCallSession(
+  opts: Omit<Parameters<typeof createCallSession>[0], "direction"> & {
+    direction?: Parameters<typeof createCallSession>[0]["direction"];
+  },
+) {
   ensureConversation(opts.conversationId);
-  return createCallSession(opts);
+  // Direction only shapes the funnel stamp; these suites test other things.
+  return createCallSession({ direction: "inbound", ...opts });
 }
 
 /** Backdate a session's createdAt so it appears older than the grace period. */

package/src/__tests__/call-routes-http.test.ts

@@ -327,6 +327,7 @@ describe("runtime call routes — HTTP layer", () => {
     ensureConversation("conv-get-1");
 
     const session = createCallSession({
+      direction: "outbound",
       conversationId: "conv-get-1",
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -378,6 +379,7 @@ describe("runtime call routes — HTTP layer", () => {
     ensureConversation("conv-cancel-1");
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: "conv-cancel-1",
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -407,6 +409,7 @@ describe("runtime call routes — HTTP layer", () => {
     ensureConversation("conv-cancel-2");
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: "conv-cancel-2",
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -447,6 +450,7 @@ describe("runtime call routes — HTTP layer", () => {
     ensureConversation("conv-answer-badjson");
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: "conv-answer-badjson",
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -469,6 +473,7 @@ describe("runtime call routes — HTTP layer", () => {
     ensureConversation("conv-answer-1");
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: "conv-answer-1",
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -495,6 +500,7 @@ describe("runtime call routes — HTTP layer", () => {
     ensureConversation("conv-answer-2");
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: "conv-answer-2",
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -517,6 +523,7 @@ describe("runtime call routes — HTTP layer", () => {
     ensureConversation("conv-answer-3");
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: "conv-answer-3",
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -548,6 +555,7 @@ describe("runtime call routes — HTTP layer", () => {
     ensureConversation("conv-instr-badjson");
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: "conv-instr-badjson",
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -570,6 +578,7 @@ describe("runtime call routes — HTTP layer", () => {
     ensureConversation("conv-instr-empty");
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: "conv-instr-empty",
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -596,6 +605,7 @@ describe("runtime call routes — HTTP layer", () => {
     ensureConversation("conv-instr-missing");
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: "conv-instr-missing",
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -640,6 +650,7 @@ describe("runtime call routes — HTTP layer", () => {
     ensureConversation("conv-instr-ended");
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: "conv-instr-ended",
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -668,6 +679,7 @@ describe("runtime call routes — HTTP layer", () => {
     ensureConversation("conv-instr-no-orch");
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: "conv-instr-no-orch",
       provider: "twilio",
       fromNumber: "+15550001111",

package/src/__tests__/call-store.test.ts

@@ -53,9 +53,14 @@ function resetTables() {
 }
 
 /** Wrapper that ensures the FK conversation row exists before creating a session. */
-function createTestCallSession(opts: Parameters<typeof createCallSession>[0]) {
+function createTestCallSession(
+  opts: Omit<Parameters<typeof createCallSession>[0], "direction"> & {
+    direction?: Parameters<typeof createCallSession>[0]["direction"];
+  },
+) {
   ensureConversation(opts.conversationId);
-  return createCallSession(opts);
+  // Direction only shapes the funnel stamp; these suites test other things.
+  return createCallSession({ direction: "inbound", ...opts });
 }
 
 describe("call-store", () => {

package/src/__tests__/deterministic-verification-control-plane.test.ts

@@ -153,6 +153,7 @@ describe("Call session mode metadata", () => {
 
     const { conversationId } = getOrCreateConversation("test-conv-mode");
     const session = createCallSession({
+      direction: "inbound",
       conversationId,
       provider: "twilio",
       fromNumber: "+15551234567",
@@ -181,6 +182,7 @@ describe("Call session mode metadata", () => {
       "test-conv-mode-default",
     );
     const session = createCallSession({
+      direction: "inbound",
       conversationId,
       provider: "twilio",
       fromNumber: "+15551234567",

package/src/__tests__/guardian-dispatch.test.ts

@@ -158,6 +158,7 @@ describe("guardian-dispatch", () => {
     ensureConversation(convId);
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -209,6 +210,7 @@ describe("guardian-dispatch", () => {
     ensureConversation(convId);
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -238,6 +240,7 @@ describe("guardian-dispatch", () => {
     ensureConversation(convId);
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -282,6 +285,7 @@ describe("guardian-dispatch", () => {
     };
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -324,6 +328,7 @@ describe("guardian-dispatch", () => {
     };
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -369,6 +374,7 @@ describe("guardian-dispatch", () => {
     };
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -396,6 +402,7 @@ describe("guardian-dispatch", () => {
     ensureConversation(convId);
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -431,6 +438,7 @@ describe("guardian-dispatch", () => {
     ensureConversation(convId);
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -456,6 +464,7 @@ describe("guardian-dispatch", () => {
     ensureConversation(convId);
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -488,6 +497,7 @@ describe("guardian-dispatch", () => {
     const sharedConversationId = "conv-shared-guardian";
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -576,6 +586,7 @@ describe("guardian-dispatch", () => {
     const sharedConversationId = "conv-affinity-guardian";
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -648,6 +659,7 @@ describe("guardian-dispatch", () => {
     const sharedConversationId = "conv-affinity-tool-guardian";
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -727,6 +739,7 @@ describe("guardian-dispatch", () => {
     const sharedConversationId = "conv-affinity-triple";
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",

package/src/__tests__/notification-guardian-path.test.ts

@@ -160,6 +160,7 @@ describe("ASK_GUARDIAN notification path", () => {
     ensureConversation(convId);
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -220,6 +221,7 @@ describe("ASK_GUARDIAN notification path", () => {
     };
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -263,6 +265,7 @@ describe("ASK_GUARDIAN notification path", () => {
     };
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -308,6 +311,7 @@ describe("ASK_GUARDIAN notification path", () => {
     };
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -338,6 +342,7 @@ describe("ASK_GUARDIAN notification path", () => {
     ensureConversation(convId);
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -370,6 +375,7 @@ describe("ASK_GUARDIAN notification path", () => {
     const sharedConvId = "conv-guardian-shared-thread";
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",
@@ -463,6 +469,7 @@ describe("ASK_GUARDIAN notification path", () => {
     };
 
     const session = createCallSession({
+      direction: "inbound",
       conversationId: convId,
       provider: "twilio",
       fromNumber: "+15550001111",

package/src/__tests__/twilio-routes.test.ts

@@ -387,6 +387,7 @@ function createTestSession(
 ) {
   ensureConversation(convId);
   const session = createCallSession({
+    direction: "inbound",
     conversationId: convId,
     provider: "twilio",
     fromNumber: "+14155550111",

package/src/calls/__tests__/call-controller.test.ts

@@ -421,6 +421,7 @@ function setupController(
 ) {
   ensureConversation("conv-ctrl-test");
   const session = createCallSession({
+    direction: "inbound",
     conversationId: "conv-ctrl-test",
     provider: "twilio",
     fromNumber: "+15551111111",
@@ -471,6 +472,7 @@ function setupControllerWithOrigin(task?: string) {
   ensureConversation("conv-ctrl-voice");
   ensureConversation("conv-ctrl-origin");
   const session = createCallSession({
+    direction: "inbound",
     conversationId: "conv-ctrl-voice",
     provider: "twilio",
     fromNumber: "+15551111111",
@@ -5028,4 +5030,96 @@ describe("call-controller", () => {
       await turnPromise.catch(() => {});
     });
   });
+  describe("turn metrics", () => {
+    test("a caller turn records the marks the latency numbers are built from", async () => {
+      const { controller } = setupController();
+
+      await controller.handleCallerUtterance("What is the weather?");
+
+      const snapshot = controller.getMetricsSnapshot();
+      expect(snapshot.recentTurns.length).toBe(1);
+      const turn = snapshot.recentTurns[0];
+      expect(turn.status).toBe("completed");
+      // The transcript mark is seeded from when the caller stopped talking,
+      // the dispatch mark from when the leg went out, and the delta mark from
+      // the first token back: the three anchors every phone latency number is
+      // subtracted from.
+      expect(turn.timestamps.finalTranscriptAtMs).not.toBeNull();
+      expect(turn.timestamps.assistantDispatchAtMs).not.toBeNull();
+      expect(turn.timestamps.firstAssistantDeltaAtMs).not.toBeNull();
+      expect(turn.durations.dispatchToFirstAssistantDeltaMs).not.toBeNull();
+      expect(turn.durations.totalTurnDurationMs).not.toBeNull();
+      // The provider's final doubles as the speech-end anchor, so the turn's
+      // round trip is measurable on phone as it is in live voice.
+      expect(turn.timestamps.utteranceEndAtMs).not.toBeNull();
+      expect(turn.durations.roundTripMs).not.toBeNull();
+
+      controller.destroy();
+    });
+
+    test("the transcript mark predates the dispatch it triggered", async () => {
+      // Seeding backdates the turn to the caller's own boundary; a turn that
+      // opened at dispatch time would report a felt latency that starts after
+      // the caller already finished speaking.
+      const { controller } = setupController();
+
+      await controller.handleCallerUtterance("Hello there");
+
+      const turn = controller.getMetricsSnapshot().recentTurns[0];
+      const finalTranscriptAtMs = turn.timestamps.finalTranscriptAtMs;
+      const assistantDispatchAtMs = turn.timestamps.assistantDispatchAtMs;
+      expect(finalTranscriptAtMs).not.toBeNull();
+      expect(assistantDispatchAtMs).not.toBeNull();
+      expect(finalTranscriptAtMs!).toBeLessThanOrEqual(assistantDispatchAtMs!);
+      expect(turn.timestamps.startedAtMs).toBeLessThanOrEqual(
+        finalTranscriptAtMs!,
+      );
+
+      controller.destroy();
+    });
+
+    test("a barge-in ends the turn once, as cancelled", async () => {
+      // One interruption is one cancelled turn: the count downstream reads as
+      // "turns the caller cut off", so a second settle would double it.
+      // A turn that never emits or completes holds the controller in the
+      // pre-speech phase, which a sustained barge-in can still cut off.
+      mockStartVoiceTurn.mockImplementation(
+        async (opts: { onComplete: () => void }) => ({
+          turnId: "run-metrics-slow",
+          abort: () => opts.onComplete(),
+        }),
+      );
+      const { controller } = setupController();
+      const turnPromise = controller.handleCallerUtterance("Tell me a story");
+      for (let i = 0; i < 5; i++) {
+        await Promise.resolve();
+      }
+      expect(controller.getState()).toBe("processing");
+
+      expect(controller.handleBargeIn()).toBe(true);
+      await turnPromise.catch(() => {});
+      controller.destroy();
+
+      const turns = controller.getMetricsSnapshot().recentTurns;
+      expect(turns.length).toBe(1);
+      expect(turns[0].status).toBe("cancelled");
+      expect(turns[0].cancellationReason).toBe("interrupted");
+      expect(turns[0].timestamps.bargeInAtMs).not.toBeNull();
+    });
+
+    test("no turn is invented between turns", async () => {
+      // Every mark opens a turn when none is active, so a barge-in on an idle
+      // controller must leave the collector empty rather than start a turn
+      // that never settles.
+      const { controller } = setupController();
+
+      expect(controller.handleBargeIn()).toBe(false);
+
+      const snapshot = controller.getMetricsSnapshot();
+      expect(snapshot.recentTurns.length).toBe(0);
+      expect(snapshot.activeTurn).toBeNull();
+
+      controller.destroy();
+    });
+  });
 });