@kuralle-syrinx/browser-client 4.3.0 → 4.4.1

package/README.md

@@ -0,0 +1,142 @@
+# @kuralle-syrinx/browser-client
+
+The browser SDK for the Syrinx WebSocket voice protocol. Microphone capture,
+opus encoding, jitter-buffered playback, reconnect — and the typed message
+stream a voice agent sends back.
+
+**This is what you build your own voice UI on.** [Syrinx Studio](https://github.com/kuralle/syrinx/tree/main/apps/studio)
+is built entirely on it; so is anything you want to embed voice into.
+
+```bash
+npm install @kuralle-syrinx/browser-client
+```
+
+## Talk to an agent
+
+```ts
+import { SyrinxBrowserClient } from "@kuralle-syrinx/browser-client";
+
+const client = new SyrinxBrowserClient({
+  url: "ws://localhost:4173/ws",
+  audioContext: new AudioContext(),
+  jitterBuffer: { targetBufferMs: 100 },
+});
+
+client.on((event) => {
+  if (event.type === "message" && event.message.type === "stt_output") {
+    console.log("you said:", event.message.transcript);
+  }
+  if (event.type === "message" && event.message.type === "agent_chunk") {
+    process.stdout.write(event.message.text);
+  }
+});
+
+await client.connect();
+```
+
+Send a turn as text instead of speaking — useful for iterating on prompts and
+tools without paying STT/TTS latency or cost:
+
+```ts
+client.sendText("What's the deadline for the CS masters?");
+```
+
+## What the server sends you
+
+`SyrinxStudioMessage` is a discriminated union of roughly twenty variants. The
+ones you will reach for first:
+
+| Message | Carries |
+| --- | --- |
+| `ready` | negotiated sample rates, encoding, resume window |
+| `speech_started` / `speech_ended` | turn boundaries |
+| `stt_chunk` / `stt_output` | interim and final transcript (+ confidence) |
+| `agent_chunk` / `agent_end` | the reply, streamed |
+| `agent_tool_call` / `agent_tool_result` | tool use |
+| `tool_call_started` \| `delayed` \| `complete` \| `failed` | the "thinking" cue lifecycle |
+| `agent_interrupted` | barge-in, **with a reason** |
+| `turn_complete`, `tts_chunk`, `tts_end` | turn and audio lifecycle |
+| `metrics` | per-turn latency decomposition |
+| `error` | `component`, `category`, `message` |
+
+> **Handle unknown types gracefully.** On Cloudflare the agents SDK also emits
+> `cf_agent_identity` and `cf_agent_mcp_servers`, which are not in this union,
+> and future versions will add more. Ignore what you do not recognise rather
+> than throwing.
+
+## Structured session state — `/record`
+
+Folding that stream by hand is tedious and easy to get wrong. The `/record`
+subpath does it for you, and is **dependency-free and DOM-free** so it also runs
+under Node (in a test, a CLI, or CI):
+
+```ts
+import { buildSessionRecord } from "@kuralle-syrinx/browser-client/record";
+
+const record = buildSessionRecord(messages.map((m, i) => ({ message: m, atMs: i })));
+
+record.turns[0]?.userTranscript;  // what the caller said
+record.turns[0]?.agentText;       // what the agent replied
+record.turns[0]?.interrupted;     // { atMs, reason } when barge-in cut it off
+record.turns[0]?.toolCalls;       // merged across all four cue phases
+record.turns[0]?.timings;         // optional — see the caveat below
+```
+
+It is a **pure reducer**: same messages in, same record out, no side effects.
+That is what lets you render a UI from a recorded fixture in a test instead of
+needing a live provider.
+
+It is also **bounded** — 50 turns and 500 events per turn by default, evicting
+oldest and reporting `droppedTurns` / `droppedEvents` rather than losing data
+silently.
+
+> **`timings` is optional and you must handle its absence.** It comes from the
+> `metrics` message, which the Node host emits and the Cloudflare Workers path
+> currently does not. Render a turn without timings rather than assuming they
+> arrive — showing zeroes would read as a zero-latency turn.
+
+## Derived views — `/agent-state` and `/turn-timeline`
+
+Both are pure, Node-safe, and derived from the same record.
+
+```ts
+import { deriveAgentState, isStalled } from "@kuralle-syrinx/browser-client/agent-state";
+import { buildTurnTimeline } from "@kuralle-syrinx/browser-client/turn-timeline";
+```
+
+`deriveAgentState` gives you `idle → listening → endpointing → thinking →
+speaking → interrupted` — what a client renders as a listening indicator. Note
+this is *conversational* state; `SessionState` in `@kuralle-syrinx/core` is
+session lifecycle and is a different thing. `isStalled` flags a state held far
+past plausibility, because a thirty-second "thinking" is a hung tool call, not
+patience.
+
+`buildTurnTimeline` turns `metrics` into a latency waterfall with the slowest
+segment marked. It also flags replies faster than `FAST_TURN_FLOOR_MS` (700ms) —
+**a sub-second reply is usually the endpointer firing while the caller is still
+speaking**, not a fast agent.
+
+## Session aggregates — `/session-metrics`
+
+```ts
+import { buildSessionMetrics } from "@kuralle-syrinx/browser-client/session-metrics";
+```
+
+Median / p95 / max per stage across a session, plus the ids of turns that
+replied implausibly fast. Percentiles are **nearest-rank**, so every number
+reported is a measurement that actually occurred rather than an interpolation no
+turn ever produced.
+
+## Exports
+
+| Path | Contents | Runs under Node? |
+| --- | --- | --- |
+| `.` | client, transport, audio, message types | no — needs `AudioContext` |
+| `./record` | `SessionRecord` assembler | yes |
+| `./agent-state` | conversational state machine | yes |
+| `./turn-timeline` | per-turn latency waterfall | yes |
+| `./session-metrics` | session aggregates | yes |
+
+## Licence
+
+MIT.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@kuralle-syrinx/browser-client",
-  "version": "4.3.0",
+  "version": "4.4.1",
   "private": false,
   "description": "Browser client for the Syrinx WebSocket audio protocol — mic capture, opus, jitter buffer, playout progress reporting, reconnect",
   "keywords": [
@@ -25,9 +25,17 @@
   "type": "module",
   "main": "./src/index.ts",
   "types": "./src/index.ts",
+  "exports": {
+    ".": "./src/index.ts",
+    "./record": "./src/session-record.ts",
+    "./agent-state": "./src/agent-state.ts",
+    "./turn-timeline": "./src/turn-timeline.ts",
+    "./session-metrics": "./src/session-metrics.ts",
+    "./turn-recorder": "./src/turn-recorder.ts"
+  },
   "dependencies": {
     "@evan/opus": "1.0.3",
-    "@kuralle-syrinx/core": "4.3.0"
+    "@kuralle-syrinx/core": "4.4.1"
   },
   "devDependencies": {
     "typescript": "^5.7.0",

package/src/agent-state.ts

@@ -0,0 +1,107 @@
+// SPDX-License-Identifier: MIT
+//
+// Conversational agent state, derived on the client from messages the server
+// already sends. Distinct from `SessionState` in core, which is session
+// *lifecycle* (uninitialized/ready/closed) — this is what the caller experiences:
+// is it listening to me, thinking, or talking?
+//
+// A pause reads as "broken" without this. It is what a client renders as a
+// listening indicator and what makes a turn timeline legible.
+//
+// Derived rather than sent: every signal already exists on the wire, so this
+// needs no server change and works identically on Node and Workers.
+
+import type { SyrinxStudioMessage } from "./index.js";
+
+export type AgentState =
+  | "idle"        // no session activity
+  | "listening"   // user is speaking
+  | "endpointing" // user stopped; deciding whether the turn is over
+  | "thinking"    // reasoner (and any tools) running
+  | "speaking"    // assistant audio playing
+  | "interrupted"; // barge-in cut the assistant off
+
+export interface AgentStateSnapshot {
+  readonly state: AgentState;
+  /** ms the state has been held, from the caller's clock. Lets a UI flag a stuck state. */
+  readonly sinceMs: number;
+  readonly turnId?: string;
+}
+
+type AnyMessage = { readonly type: string; readonly [k: string]: unknown };
+
+/**
+ * Fold one message into the state. Pure; the caller supplies the clock so this
+ * stays deterministic under test.
+ *
+ * Ordering note: `speech_started` wins over everything, because a caller talking
+ * is the highest-priority fact on the wire — that is also what makes barge-in
+ * legible (speaking → listening without waiting for the interrupt message).
+ */
+export function nextAgentState(
+  prev: AgentStateSnapshot,
+  message: SyrinxStudioMessage | AnyMessage,
+  atMs: number,
+): AgentStateSnapshot {
+  const type = typeof (message as AnyMessage).type === "string" ? (message as AnyMessage).type : "";
+  const turnId = typeof (message as AnyMessage).turnId === "string" ? ((message as AnyMessage).turnId as string) : prev.turnId;
+  const to = (state: AgentState): AgentStateSnapshot =>
+    state === prev.state ? { ...prev, turnId } : { state, sinceMs: atMs, turnId };
+
+  switch (type) {
+    case "speech_started":
+      return to("listening");
+    case "speech_ended":
+      // Speech stopped, but the turn is not necessarily over — this is exactly
+      // the window where a premature endpoint cuts a caller off.
+      return to("endpointing");
+    case "stt_output":
+    case "turn_complete":
+      // A final transcript means the turn closed; the reasoner is now running.
+      return to("thinking");
+    case "agent_tool_call":
+    case "tool_call_started":
+    case "tool_call_delayed":
+      return to("thinking");
+    case "agent_chunk":
+      // Text is arriving but nothing is audible yet.
+      return prev.state === "speaking" ? to("speaking") : to("thinking");
+    case "tts_chunk":
+      return to("speaking");
+    case "agent_interrupted":
+      return to("interrupted");
+    case "tts_end":
+    case "agent_end":
+      // Only settle to idle from a talking state; an interrupt stands until the
+      // next turn starts, so the UI can show what happened.
+      return prev.state === "speaking" ? to("idle") : prev;
+    default:
+      return prev;
+  }
+}
+
+export const INITIAL_AGENT_STATE: AgentStateSnapshot = { state: "idle", sinceMs: 0 };
+
+/** Fold a whole stream. Equivalent to repeated `nextAgentState`. */
+export function deriveAgentState(
+  messages: readonly { readonly message: SyrinxStudioMessage | AnyMessage; readonly atMs: number }[],
+): AgentStateSnapshot {
+  return messages.reduce((s, { message, atMs }) => nextAgentState(s, message, atMs), INITIAL_AGENT_STATE);
+}
+
+/**
+ * How long a state may plausibly be held before it indicates a stall. A UI should
+ * flag rather than hide these — a 30s "thinking" is a hung tool call, not patience.
+ */
+export const STATE_STALL_THRESHOLD_MS: Readonly<Record<AgentState, number>> = {
+  idle: Number.POSITIVE_INFINITY,
+  listening: 60_000,
+  endpointing: 5_000,
+  thinking: 30_000,
+  speaking: 120_000,
+  interrupted: 10_000,
+};
+
+export function isStalled(snapshot: AgentStateSnapshot, nowMs: number): boolean {
+  return nowMs - snapshot.sinceMs > STATE_STALL_THRESHOLD_MS[snapshot.state];
+}

package/src/index.ts

@@ -28,6 +28,7 @@ import {
   resamplePcm16Streaming,
   type StreamingPcm16Resampler,
 } from "@kuralle-syrinx/core/audio";
+import type { TurnEndOwner, TurnEndReason } from "@kuralle-syrinx/core";
 import {
   decodeBrowserAssistantAudio,
   encodeBrowserAudioEnvelopeFrame,
@@ -68,6 +69,8 @@ export type SyrinxStudioMessage =
         readonly encoding: "pcm_s16le" | "opus";
         readonly supportedInputCodecs?: readonly string[];
         readonly channels: 1;
+        /** Node states this; the Workers/DO host does not. Optional for that reason. */
+        readonly targetFrameDurationMs?: number;
         readonly binaryEnvelope?: "syrinx.audio.v1";
         readonly rawBinaryInput?: boolean;
       };
@@ -114,6 +117,10 @@ export type SyrinxStudioMessage =
       readonly llmTTFTMs?: number;
       readonly ttsTTFBMs?: number;
       readonly e2eMs?: number;
+      /** Which owner decided the turn ended. Omitted when the backend did not say. */
+      readonly endpointingOwner?: TurnEndOwner;
+      /** Why the turn ended. Omitted when the backend did not say. */
+      readonly endpointingReason?: TurnEndReason;
     }
   | { readonly type: "error"; readonly component?: string; readonly category?: string; readonly message: string };
 
@@ -187,6 +194,11 @@ const RECONNECT_MIN_STABLE_MS = 5_000;
 const RECONNECT_MAX_QUICK_FAILURES = 3;
 const KEEPALIVE_INTERVAL_MS = 10_000;
 
+// Endpointing decision values carried on the `metrics` message. Forward-compatible:
+// an unknown value (a newer backend) is ignored rather than dropping the message.
+const ENDPOINTING_OWNERS = new Set<string>(["provider_stt", "smart_turn", "timer", "text"]);
+const ENDPOINTING_REASONS = new Set<string>(["end_of_speech", "force_finalized", "typed"]);
+
 export class SyrinxBrowserClient {
   private readonly transport: ClientTransport;
   private readonly handlers = new Set<SyrinxBrowserClientHandler>();
@@ -809,6 +821,14 @@ function parseStudioMessage(value: unknown): SyrinxStudioMessage {
     };
   }
   if (type === "metrics") {
+    const endpointingOwner = optionalEndpointingEnum<TurnEndOwner>(
+      value.endpointingOwner,
+      ENDPOINTING_OWNERS,
+    );
+    const endpointingReason = optionalEndpointingEnum<TurnEndReason>(
+      value.endpointingReason,
+      ENDPOINTING_REASONS,
+    );
     return {
       type,
       turnId: optionalString(value.turnId, "metrics.turnId"),
@@ -822,6 +842,8 @@ function parseStudioMessage(value: unknown): SyrinxStudioMessage {
       llmTTFTMs: optionalNumber(value.llmTTFTMs, "metrics.llmTTFTMs"),
       ttsTTFBMs: optionalNumber(value.ttsTTFBMs, "metrics.ttsTTFBMs"),
       e2eMs: optionalNumber(value.e2eMs, "metrics.e2eMs"),
+      ...(endpointingOwner !== undefined ? { endpointingOwner } : {}),
+      ...(endpointingReason !== undefined ? { endpointingReason } : {}),
     };
   }
   if (type === "error") {
@@ -933,6 +955,12 @@ function optionalBoolean(value: unknown, name: string): boolean | undefined {
   return value;
 }
 
+function optionalEndpointingEnum<T extends string>(value: unknown, allowed: ReadonlySet<string>): T | undefined {
+  // Forward-compatible: an unknown value is treated as absent, not an error, so a
+  // newer backend adding an enum member does not break an older client's message.
+  return typeof value === "string" && allowed.has(value) ? (value as T) : undefined;
+}
+
 function requiredLiteral<T extends string | number>(value: unknown, expected: T, name: string): T {
   if (value !== expected) throw new Error(`${name} must be ${String(expected)}`);
   return expected;

package/src/session-metrics.ts

@@ -0,0 +1,90 @@
+// SPDX-License-Identifier: MIT
+//
+// Session-level latency aggregates. A one-off spike and a consistent regression
+// look identical turn by turn; you need the distribution to tell them apart.
+//
+// Deliberately reports a FLOOR as well as a ceiling. A sub-second reply is the
+// number every voice framework optimises for, and in a turn-taking system it is
+// usually evidence the endpointer fired while the caller was still speaking.
+// Celebrating it is how "fast" agents ship that interrupt people.
+
+import type { TurnRecord, TurnTimings } from "./session-record.js";
+import { FAST_TURN_FLOOR_MS } from "./turn-timeline.js";
+
+export interface StageStats {
+  readonly stage: string;
+  readonly label: string;
+  readonly count: number;
+  readonly medianMs: number;
+  readonly p95Ms: number;
+  readonly maxMs: number;
+}
+
+export interface SessionMetrics {
+  readonly turnCount: number;
+  /** Turns that carried timings. The Workers path sends none, so this can be 0. */
+  readonly measuredTurnCount: number;
+  readonly stages: readonly StageStats[];
+  /** Turns whose end-to-end fell below the floor — probable premature endpointing. */
+  readonly suspiciouslyFastTurnIds: readonly string[];
+  readonly floorMs: number;
+  /** True when nothing could be measured, so the UI says why instead of showing zeroes. */
+  readonly unavailable: boolean;
+}
+
+const STAGES: readonly { readonly key: keyof TurnTimings; readonly stage: string; readonly label: string }[] = [
+  { key: "sttMs", stage: "stt", label: "Hearing you" },
+  { key: "llmTTFTMs", stage: "llm", label: "Thinking (to first word)" },
+  { key: "ttsTTFBMs", stage: "tts", label: "Voice (to first audio)" },
+  { key: "e2eMs", stage: "e2e", label: "End to end" },
+];
+
+/**
+ * Nearest-rank percentile. Chosen over interpolation deliberately: every value
+ * reported is a real measurement that occurred, so a p95 can be traced back to
+ * an actual turn rather than being a number no turn ever had.
+ */
+export function percentile(sortedAsc: readonly number[], p: number): number {
+  if (sortedAsc.length === 0) return 0;
+  const rank = Math.ceil((p / 100) * sortedAsc.length);
+  const idx = Math.min(sortedAsc.length - 1, Math.max(0, rank - 1));
+  return sortedAsc[idx] ?? 0;
+}
+
+export function buildSessionMetrics(
+  turns: readonly TurnRecord[],
+  floorMs: number = FAST_TURN_FLOOR_MS,
+): SessionMetrics {
+  const measured = turns.filter((t) => t.timings !== undefined);
+
+  const stages = STAGES.map(({ key, stage, label }) => {
+    const values = measured
+      .map((t) => t.timings?.[key])
+      .filter((v): v is number => typeof v === "number")
+      .sort((a, b) => a - b);
+    return {
+      stage,
+      label,
+      count: values.length,
+      medianMs: percentile(values, 50),
+      p95Ms: percentile(values, 95),
+      maxMs: values.length > 0 ? (values[values.length - 1] ?? 0) : 0,
+    };
+  }).filter((s) => s.count > 0);
+
+  const suspiciouslyFastTurnIds = measured
+    .filter((t) => {
+      const e2e = t.timings?.e2eMs;
+      return typeof e2e === "number" && e2e > 0 && e2e < floorMs;
+    })
+    .map((t) => t.turnId);
+
+  return {
+    turnCount: turns.length,
+    measuredTurnCount: measured.length,
+    stages,
+    suspiciouslyFastTurnIds,
+    floorMs,
+    unavailable: measured.length === 0,
+  };
+}

package/src/session-record.ts

@@ -0,0 +1,307 @@
+// SPDX-License-Identifier: MIT
+//
+// SessionRecord — a structured, bounded view of what happened in a voice session,
+// assembled from the messages the server already sends. Every debugging surface
+// (turn timeline, event log, metrics, fixture capture) reads this one shape, and a
+// fixture is a SessionRecord trimmed to one turn.
+//
+// Deliberately dependency-free and DOM-free: this module is exported from the
+// `/record` subpath so a Node consumer can import it without pulling AudioContext
+// or the opus codec. Assembly is a pure reduction over messages — no sockets, no
+// timers, no React — which is what makes every consumer testable against recorded
+// fixtures instead of a live provider.
+
+import type { SyrinxStudioMessage } from "./index.js";
+
+/** Per-turn latency decomposition, from the `metrics` message. */
+export interface TurnTimings {
+  readonly speechEndMs?: number;
+  readonly textReadyMs?: number;
+  readonly firstAudioByteMs?: number;
+  readonly firstAudioPlayedMs?: number;
+  readonly lastAudioPlayedMs?: number;
+  readonly sttMs?: number;
+  readonly llmTTFTMs?: number;
+  readonly ttsTTFBMs?: number;
+  readonly e2eMs?: number;
+}
+
+/** Negotiated session parameters. Recorded so a replayed fixture cannot silently mislead. */
+export interface SessionConfig {
+  readonly wsUrl?: string;
+  readonly sessionId?: string;
+  readonly inputSampleRateHz?: number;
+  readonly outputSampleRateHz?: number;
+  readonly encoding?: "pcm_s16le" | "opus";
+  readonly binaryEnvelope?: string;
+  readonly rawBinaryInput?: boolean;
+  /**
+   * Frame duration the host paces to. Node states it; the Workers/DO host does not
+   * (measured live, 2026-07-25) — which is exactly the kind of divergence the session
+   * info panel exists to make visible, so it is captured rather than dropped.
+   */
+  readonly targetFrameDurationMs?: number;
+  readonly resumeWindowMs?: number;
+  /** Only present once the server exposes it; the timeline omits the marker rather than guessing. */
+  readonly endpointingOwner?: string;
+}
+
+export interface ToolCall {
+  readonly id?: string;
+  readonly name?: string;
+  readonly args?: unknown;
+  readonly result?: unknown;
+  /** `delayed` means it outlived the cue threshold — a slow tool, not a failed one. */
+  readonly phase?: "started" | "delayed" | "complete" | "failed";
+  readonly afterMs?: number;
+}
+
+export interface RecordedEvent {
+  /** ms since the record started — comparable across turns. */
+  readonly atMs: number;
+  readonly message: SyrinxStudioMessage | { readonly type: string; readonly [k: string]: unknown };
+}
+
+export interface TurnRecord {
+  readonly turnId: string;
+  readonly startedAtMs: number;
+  readonly events: readonly RecordedEvent[];
+  /** Absent on runtimes that do not emit `metrics` — the Workers path does not. */
+  readonly timings?: TurnTimings;
+  readonly userTranscript?: string;
+  /** Latest interim, cleared once the final arrives. */
+  readonly userInterim?: string;
+  readonly userConfidence?: number;
+  readonly agentText: string;
+  readonly toolCalls: readonly ToolCall[];
+  readonly ttsAudioBytes: number;
+  readonly interrupted?: { readonly atMs: number; readonly reason?: string };
+  readonly errors: readonly { readonly atMs: number; readonly component?: string; readonly category?: string; readonly message: string }[];
+  readonly complete: boolean;
+  /** Non-zero when this turn's event list hit the cap. Surfaced, never silent. */
+  readonly droppedEvents: number;
+  /** Which owner decided the turn ended. Omitted when the backend did not say. */
+  readonly endpointingOwner?: string;
+  /** Why the turn ended. Omitted when the backend did not say. */
+  readonly endpointingReason?: string;
+}
+
+export interface SessionRecord {
+  readonly config: SessionConfig;
+  readonly turns: readonly TurnRecord[];
+  /** Non-zero when oldest turns were evicted. Surfaced, never silent. */
+  readonly droppedTurns: number;
+  /** Messages that arrived before any turn existed, or with no turnId. */
+  readonly sessionEvents: readonly RecordedEvent[];
+}
+
+export interface SessionRecordLimits {
+  readonly maxTurns: number;
+  readonly maxEventsPerTurn: number;
+}
+
+export const DEFAULT_LIMITS: SessionRecordLimits = { maxTurns: 50, maxEventsPerTurn: 500 };
+
+export function emptySessionRecord(config: SessionConfig = {}): SessionRecord {
+  return { config, turns: [], droppedTurns: 0, sessionEvents: [] };
+}
+
+// A message we do not recognise — the Cloudflare agents SDK sends `cf_agent_identity`
+// and `cf_agent_mcp_servers` before `ready`, and future SDK versions will add more.
+// These are retained in the event stream but must never affect turn assembly.
+type AnyMessage = SyrinxStudioMessage | { readonly type: string; readonly [k: string]: unknown };
+
+const turnIdOf = (m: AnyMessage): string | undefined =>
+  typeof (m as { turnId?: unknown }).turnId === "string" ? (m as { turnId: string }).turnId : undefined;
+
+function newTurn(turnId: string, atMs: number): TurnRecord {
+  return {
+    turnId,
+    startedAtMs: atMs,
+    events: [],
+    agentText: "",
+    toolCalls: [],
+    ttsAudioBytes: 0,
+    errors: [],
+    complete: false,
+    droppedEvents: 0,
+  };
+}
+
+function pushEvent(turn: TurnRecord, ev: RecordedEvent, limits: SessionRecordLimits): TurnRecord {
+  if (turn.events.length < limits.maxEventsPerTurn) {
+    return { ...turn, events: [...turn.events, ev] };
+  }
+  // Drop the oldest, keep the newest, and count what was lost.
+  return { ...turn, events: [...turn.events.slice(1), ev], droppedEvents: turn.droppedEvents + 1 };
+}
+
+function upsertToolCall(calls: readonly ToolCall[], key: string | undefined, patch: Partial<ToolCall>): readonly ToolCall[] {
+  // A later phase message carries fewer fields than an earlier one — `tool_call_complete`
+  // has no `afterMs` and may have no `toolName`. Spreading those as explicit `undefined`
+  // would erase what `tool_call_delayed`/`agent_tool_call` already established, so drop
+  // undefined keys before merging.
+  const defined = Object.fromEntries(Object.entries(patch).filter(([, v]) => v !== undefined)) as Partial<ToolCall>;
+  const i = key === undefined ? -1 : calls.findIndex((c) => c.id === key);
+  if (i === -1) return [...calls, { ...defined, id: key }];
+  const next = [...calls];
+  next[i] = { ...next[i], ...defined };
+  return next;
+}
+
+/**
+ * Fold one message into the record. Pure: same inputs, same output, no side effects.
+ *
+ * `atMs` is the caller's clock (ms since the record started) rather than anything
+ * read from the message, so the record stays deterministic under test.
+ */
+export function applyMessage(
+  record: SessionRecord,
+  message: AnyMessage,
+  atMs: number,
+  limits: SessionRecordLimits = DEFAULT_LIMITS,
+): SessionRecord {
+  const ev: RecordedEvent = { atMs, message };
+  const type = typeof message?.type === "string" ? message.type : "";
+
+  // `ready` carries the negotiated config and belongs to the session, not a turn.
+  if (type === "ready") {
+    const m = message as Extract<SyrinxStudioMessage, { type: "ready" }>;
+    return {
+      ...record,
+      config: {
+        ...record.config,
+        sessionId: m.sessionId ?? record.config.sessionId,
+        resumeWindowMs: m.resumeWindowMs ?? record.config.resumeWindowMs,
+        inputSampleRateHz: m.audio?.inputSampleRateHz ?? record.config.inputSampleRateHz,
+        outputSampleRateHz: m.audio?.outputSampleRateHz ?? record.config.outputSampleRateHz,
+        encoding: m.audio?.encoding ?? record.config.encoding,
+        binaryEnvelope: m.audio?.binaryEnvelope ?? record.config.binaryEnvelope,
+        rawBinaryInput: m.audio?.rawBinaryInput ?? record.config.rawBinaryInput,
+        targetFrameDurationMs: m.audio?.targetFrameDurationMs ?? record.config.targetFrameDurationMs,
+      },
+      sessionEvents: [...record.sessionEvents, ev],
+    };
+  }
+
+  const turnId = turnIdOf(message);
+  // No turn to attribute this to — including every unknown type that carries no
+  // turnId, e.g. the agents-SDK `cf_agent_*` messages. Retained, but inert.
+  if (turnId === undefined) {
+    return { ...record, sessionEvents: [...record.sessionEvents, ev] };
+  }
+
+  let turns = record.turns;
+  let droppedTurns = record.droppedTurns;
+  let current = turns.find((t) => t.turnId === turnId);
+  if (!current) {
+    // Out-of-order arrival is fine: a turn is created by whichever of its
+    // messages lands first, not by a designated "start" message.
+    current = newTurn(turnId, atMs);
+    let appended = [...turns, current];
+    if (appended.length > limits.maxTurns) {
+      appended = appended.slice(appended.length - limits.maxTurns);
+      droppedTurns += 1;
+    }
+    turns = appended;
+  }
+
+  let turn = pushEvent(current, ev, limits);
+
+  switch (type) {
+    case "stt_chunk":
+      turn = { ...turn, userInterim: (message as { transcript: string }).transcript };
+      break;
+    case "stt_output": {
+      const m = message as { transcript: string; confidence?: number };
+      turn = { ...turn, userTranscript: m.transcript, userInterim: undefined, userConfidence: m.confidence };
+      break;
+    }
+    case "agent_chunk":
+      turn = { ...turn, agentText: turn.agentText + (message as { text: string }).text };
+      break;
+    case "agent_tool_call": {
+      const m = message as { id?: string; name: string; args?: unknown };
+      turn = { ...turn, toolCalls: upsertToolCall(turn.toolCalls, m.id, { name: m.name, args: m.args }) };
+      break;
+    }
+    case "agent_tool_result": {
+      const m = message as { id?: string; result?: unknown };
+      turn = { ...turn, toolCalls: upsertToolCall(turn.toolCalls, m.id, { result: m.result }) };
+      break;
+    }
+    case "tool_call_started":
+    case "tool_call_delayed":
+    case "tool_call_complete":
+    case "tool_call_failed": {
+      const m = message as { toolId?: string; toolName?: string; afterMs?: number };
+      const phase = type.slice("tool_call_".length) as ToolCall["phase"];
+      turn = {
+        ...turn,
+        toolCalls: upsertToolCall(turn.toolCalls, m.toolId, { name: m.toolName, phase, afterMs: m.afterMs }),
+      };
+      break;
+    }
+    case "tts_chunk":
+      turn = { ...turn, ttsAudioBytes: turn.ttsAudioBytes + ((message as { byteLength?: number }).byteLength ?? 0) };
+      break;
+    case "agent_interrupted":
+      turn = { ...turn, interrupted: { atMs, reason: (message as { reason?: string }).reason } };
+      break;
+    case "turn_complete":
+      turn = {
+        ...turn,
+        complete: true,
+        userTranscript: (message as { transcript?: string }).transcript ?? turn.userTranscript,
+      };
+      break;
+    case "metrics": {
+      const {
+        type: _t,
+        turnId: _i,
+        correlationId: _c,
+        endpointingOwner: owner,
+        endpointingReason: reason,
+        ...timings
+      } = message as Record<string, unknown>;
+      // The endpointing decision is a fact about the turn, not a timing, so it lives
+      // on the turn itself — not inside `timings`. Preserve an earlier value if this
+      // metrics message omits it (re-emission should not erase a known decision).
+      const nextOwner = typeof owner === "string" ? owner : turn.endpointingOwner;
+      const nextReason = typeof reason === "string" ? reason : turn.endpointingReason;
+      turn = {
+        ...turn,
+        timings: timings as TurnTimings,
+        ...(nextOwner !== undefined ? { endpointingOwner: nextOwner } : {}),
+        ...(nextReason !== undefined ? { endpointingReason: nextReason } : {}),
+      };
+      break;
+    }
+    case "error": {
+      const m = message as { component?: string; category?: string; message: string };
+      turn = {
+        ...turn,
+        errors: [...turn.errors, { atMs, component: m.component, category: m.category, message: m.message }],
+      };
+      break;
+    }
+    default:
+      // Unknown or non-mutating type (speech_started, tts_end, audio_clear, …):
+      // already recorded in `events`; nothing further to derive.
+      break;
+  }
+
+  return { ...record, turns: turns.map((t) => (t.turnId === turnId ? turn : t)), droppedTurns };
+}
+
+/** Fold a whole message stream. Equivalent to repeated `applyMessage`. */
+export function buildSessionRecord(
+  messages: readonly { readonly message: AnyMessage; readonly atMs: number }[],
+  config: SessionConfig = {},
+  limits: SessionRecordLimits = DEFAULT_LIMITS,
+): SessionRecord {
+  return messages.reduce(
+    (rec, { message, atMs }) => applyMessage(rec, message, atMs, limits),
+    emptySessionRecord(config),
+  );
+}

package/src/turn-recorder.ts

@@ -0,0 +1,207 @@
+// SPDX-License-Identifier: MIT
+//
+// TurnAudioRecorder — keeps the microphone PCM you already streamed, sliced per turn,
+// so a turn can be saved as a fixture. Without this the studio sends audio to the
+// server and keeps none of it, and the "capture a fixture" loop cannot close.
+//
+// Dependency-free and DOM-free, exported from the `/turn-recorder` subpath: it takes
+// Int16 frames and returns WAV bytes, so the same code runs in a browser, in Node, and
+// in CI. It holds no socket and no AudioContext — you push it the frames you were
+// sending anyway, and the server's own messages drive the turn boundaries.
+
+import type { SyrinxStudioMessage } from "./index.js";
+
+export interface TurnAudioRecorderOptions {
+  /** Turns to retain audio for. Older blobs are dropped; the TurnRecord itself is not ours to evict. */
+  readonly maxTurns?: number;
+  /**
+   * Audio kept from *before* `speech_started`, in ms.
+   *
+   * Load-bearing, not a nicety: VAD announces speech after onset, so a recorder that
+   * starts buffering on `speech_started` clips the first phoneme. A fixture missing its
+   * onset transcribes differently from the live turn it was captured from, which makes
+   * it worse than no fixture — it looks like a real regression.
+   */
+  readonly preRollMs?: number;
+  /** Hard ceiling per turn. A stuck endpointer must not grow a buffer without bound. */
+  readonly maxTurnMs?: number;
+}
+
+export const DEFAULT_RECORDER_OPTIONS: Required<TurnAudioRecorderOptions> = {
+  maxTurns: 10,
+  preRollMs: 300,
+  maxTurnMs: 60_000,
+};
+
+export interface RecordedTurnAudio {
+  readonly turnId: string;
+  readonly sampleRateHz: number;
+  readonly durationMs: number;
+  readonly byteLength: number;
+  /** True when the turn hit `maxTurnMs` and the tail was dropped. Surfaced, never silent. */
+  readonly truncated: boolean;
+}
+
+const DEFAULT_SAMPLE_RATE_HZ = 16_000;
+
+/** Minimal RIFF/WAVE header + samples. Mono PCM16 — the format every STT accepts. */
+export function encodeWav(samples: Int16Array, sampleRateHz: number): Uint8Array {
+  if (!Number.isFinite(sampleRateHz) || sampleRateHz <= 0) {
+    throw new Error(`encodeWav: invalid sampleRateHz ${String(sampleRateHz)}`);
+  }
+  const dataBytes = samples.length * 2;
+  const out = new Uint8Array(44 + dataBytes);
+  const view = new DataView(out.buffer);
+  const ascii = (offset: number, text: string): void => {
+    for (let i = 0; i < text.length; i += 1) out[offset + i] = text.charCodeAt(i);
+  };
+  ascii(0, "RIFF");
+  view.setUint32(4, 36 + dataBytes, true);
+  ascii(8, "WAVE");
+  ascii(12, "fmt ");
+  view.setUint32(16, 16, true); // PCM chunk size
+  view.setUint16(20, 1, true); // PCM
+  view.setUint16(22, 1, true); // mono
+  view.setUint32(24, sampleRateHz, true);
+  view.setUint32(28, sampleRateHz * 2, true); // byte rate
+  view.setUint16(32, 2, true); // block align
+  view.setUint16(34, 16, true); // bits per sample
+  ascii(36, "data");
+  view.setUint32(40, dataBytes, true);
+  // Copy through a DataView so the result is little-endian on every host, rather than
+  // inheriting the platform's byte order via Int16Array.set.
+  for (let i = 0; i < samples.length; i += 1) {
+    view.setInt16(44 + i * 2, samples[i] ?? 0, true);
+  }
+  return out;
+}
+
+interface OpenSegment {
+  samples: Int16Array[];
+  length: number;
+  truncated: boolean;
+}
+
+export class TurnAudioRecorder {
+  private readonly options: Required<TurnAudioRecorderOptions>;
+  private sampleRateHz = DEFAULT_SAMPLE_RATE_HZ;
+  /** Rolling pre-roll, trimmed to preRollMs on every push. */
+  private preRoll: Int16Array[] = [];
+  private preRollLength = 0;
+  private open: OpenSegment | undefined;
+  /** Sealed but not yet attributed to a turnId — turnId arrives with a later message. */
+  private pending: { samples: Int16Array; truncated: boolean } | undefined;
+  private readonly turns = new Map<string, { samples: Int16Array; truncated: boolean }>();
+
+  constructor(options: TurnAudioRecorderOptions = {}) {
+    this.options = { ...DEFAULT_RECORDER_OPTIONS, ...options };
+  }
+
+  /** Push the uplink frame you are already sending. Cheap: retains the reference, copies on seal. */
+  pushFrame(frame: Int16Array): void {
+    if (frame.length === 0) return;
+    if (this.open) {
+      const cap = Math.floor((this.options.maxTurnMs / 1000) * this.sampleRateHz);
+      if (this.open.length >= cap) {
+        this.open.truncated = true;
+        return;
+      }
+      this.open.samples.push(frame);
+      this.open.length += frame.length;
+      return;
+    }
+    this.preRoll.push(frame);
+    this.preRollLength += frame.length;
+    const keep = Math.floor((this.options.preRollMs / 1000) * this.sampleRateHz);
+    while (this.preRollLength - (this.preRoll[0]?.length ?? 0) >= keep && this.preRoll.length > 1) {
+      this.preRollLength -= this.preRoll.shift()?.length ?? 0;
+    }
+  }
+
+  /** Feed the same messages you feed the SessionRecord. Turn boundaries come from the server. */
+  onMessage(message: SyrinxStudioMessage | { readonly type: string; readonly [k: string]: unknown }): void {
+    const type = message.type;
+    if (type === "ready") {
+      const rate = (message as { audio?: { inputSampleRateHz?: unknown } }).audio?.inputSampleRateHz;
+      if (typeof rate === "number" && Number.isFinite(rate) && rate > 0) this.sampleRateHz = rate;
+      return;
+    }
+    const turnId = typeof (message as { turnId?: unknown }).turnId === "string"
+      ? (message as { turnId: string }).turnId
+      : undefined;
+
+    if (type === "speech_started") {
+      // Start from the pre-roll so the onset is present.
+      this.open = { samples: [...this.preRoll], length: this.preRollLength, truncated: false };
+      this.preRoll = [];
+      this.preRollLength = 0;
+      if (turnId !== undefined) this.attributeOnSeal = turnId;
+      return;
+    }
+
+    if (type === "speech_ended" || type === "stt_output" || type === "turn_complete") {
+      this.seal(turnId);
+      return;
+    }
+  }
+
+  private attributeOnSeal: string | undefined;
+
+  private seal(turnId: string | undefined): void {
+    if (this.open) {
+      const merged = new Int16Array(this.open.length);
+      let offset = 0;
+      for (const chunk of this.open.samples) {
+        merged.set(chunk, offset);
+        offset += chunk.length;
+      }
+      this.pending = { samples: merged, truncated: this.open.truncated };
+      this.open = undefined;
+    }
+    if (!this.pending) return;
+    const id = turnId ?? this.attributeOnSeal;
+    if (id === undefined) return; // wait for a message that names the turn
+    this.attributeOnSeal = undefined;
+    this.turns.set(id, this.pending);
+    this.pending = undefined;
+    // Bounded by construction: audio blobs, not the records, are the memory risk.
+    while (this.turns.size > this.options.maxTurns) {
+      const oldest = this.turns.keys().next().value;
+      if (oldest === undefined) break;
+      this.turns.delete(oldest);
+    }
+  }
+
+  /** WAV bytes for a turn, or undefined if it was never captured or has been evicted. */
+  getWav(turnId: string): Uint8Array | undefined {
+    const entry = this.turns.get(turnId);
+    return entry ? encodeWav(entry.samples, this.sampleRateHz) : undefined;
+  }
+
+  /** What audio is retained right now, oldest first. */
+  list(): readonly RecordedTurnAudio[] {
+    return [...this.turns.entries()].map(([turnId, entry]) => ({
+      turnId,
+      sampleRateHz: this.sampleRateHz,
+      durationMs: Math.round((entry.samples.length / this.sampleRateHz) * 1000),
+      byteLength: entry.samples.length * 2,
+      truncated: entry.truncated,
+    }));
+  }
+
+  /** Total retained audio in bytes — what a memory readout should show. */
+  retainedBytes(): number {
+    let total = 0;
+    for (const entry of this.turns.values()) total += entry.samples.length * 2;
+    return total;
+  }
+
+  reset(): void {
+    this.preRoll = [];
+    this.preRollLength = 0;
+    this.open = undefined;
+    this.pending = undefined;
+    this.attributeOnSeal = undefined;
+    this.turns.clear();
+  }
+}

package/src/turn-timeline.ts

@@ -0,0 +1,148 @@
+// SPDX-License-Identifier: MIT
+//
+// Turn timeline — the per-turn latency waterfall, computed from the `metrics`
+// message the server already sends.
+//
+// "It cut me off" is not actionable; a lane showing speech ending at 2.1s and the
+// endpoint firing at 2.16s is. This is the view no competing playground ships,
+// and the data has been arriving in the browser all along.
+//
+// Pure and DOM-free, so a timeline can be rendered from a recorded fixture in a
+// test rather than only from a live provider.
+
+import type { TurnRecord, TurnTimings } from "./session-record.js";
+
+export interface TimelineSegment {
+  readonly key: string;
+  /** Plain language. The reader does not know what `eos.turn_complete` is. */
+  readonly label: string;
+  readonly startMs: number;
+  readonly durationMs: number;
+  /** True for the longest segment — where the time actually went. */
+  readonly slowest: boolean;
+}
+
+export interface TurnTimeline {
+  readonly turnId: string;
+  readonly segments: readonly TimelineSegment[];
+  readonly totalMs: number;
+  /** Why there is nothing to draw, when there is nothing to draw. */
+  readonly unavailable?: "no-metrics" | "insufficient-marks";
+  /**
+   * Set when the agent replied implausibly fast, which almost always means the
+   * endpointer fired while the caller was still speaking. Most tools celebrate
+   * this number; it is usually a defect.
+   */
+  readonly suspiciouslyFast?: { readonly totalMs: number; readonly floorMs: number };
+}
+
+/**
+ * Below this, a reply is not "fast" — the endpointer almost certainly cut the
+ * caller off. Asserting a floor as well as a ceiling is the inversion no
+ * surveyed framework supports.
+ */
+export const FAST_TURN_FLOOR_MS = 700;
+
+// Ordered marks. Each segment spans one mark to the next, so a missing mark
+// collapses its segment rather than corrupting the ones around it.
+const MARKS: readonly { readonly field: keyof TurnTimings; readonly label: string }[] = [
+  { field: "speechEndMs", label: "You stopped speaking" },
+  { field: "textReadyMs", label: "Deciding you're done, transcribing, and thinking" },
+  // Segments are labelled by the mark that ENDS them, so this one covers
+  // textReady -> firstAudioByte: the reply text already exists and we are waiting on
+  // speech. That is time-to-first-audio, which `session-metrics` reports as
+  // "Voice (to first audio)" from the same field. Calling it "Thinking" made the two
+  // panels name one quantity two different things.
+  { field: "firstAudioByteMs", label: "Voice (to first audio)" },
+  { field: "firstAudioPlayedMs", label: "First audio out" },
+  { field: "lastAudioPlayedMs", label: "Agent speaking" },
+];
+
+export function buildTurnTimeline(turn: TurnRecord): TurnTimeline {
+  const t = turn.timings;
+  if (!t) {
+    // The Workers path emits no `metrics` at all (proven by LDT-18). Say so
+    // rather than rendering an empty lane that looks like a zero-latency turn.
+    return { turnId: turn.turnId, segments: [], totalMs: 0, unavailable: "no-metrics" };
+  }
+
+  const points = MARKS.map(({ field, label }) => ({ label, at: t[field] })).filter(
+    (p): p is { label: string; at: number } => typeof p.at === "number",
+  );
+
+  if (points.length < 2) {
+    return { turnId: turn.turnId, segments: [], totalMs: t.e2eMs ?? 0, unavailable: "insufficient-marks" };
+  }
+
+  const origin = points[0]?.at ?? 0;
+  const raw = points.slice(1).map((p, i) => {
+    const prev = points[i];
+    const startMs = (prev?.at ?? origin) - origin;
+    return { key: p.label, label: p.label, startMs, durationMs: Math.max(0, p.at - (prev?.at ?? origin)) };
+  });
+
+  const longest = raw.reduce((max, s) => Math.max(max, s.durationMs), 0);
+  const segments: TimelineSegment[] = raw.map((s) => ({ ...s, slowest: longest > 0 && s.durationMs === longest }));
+
+  const totalMs = t.e2eMs ?? (points[points.length - 1]?.at ?? origin) - origin;
+  const suspiciouslyFast =
+    totalMs > 0 && totalMs < FAST_TURN_FLOOR_MS ? { totalMs, floorMs: FAST_TURN_FLOOR_MS } : undefined;
+
+  return { turnId: turn.turnId, segments, totalMs, ...(suspiciouslyFast ? { suspiciouslyFast } : {}) };
+}
+
+export function buildTimelines(turns: readonly TurnRecord[]): readonly TurnTimeline[] {
+  return turns.map(buildTurnTimeline);
+}
+
+// -----------------------------------------------------------------------------
+// Endpointing decision rendering
+// -----------------------------------------------------------------------------
+// The timeline can already show WHEN a turn ended; this names WHO decided and WHY.
+// That turns "it cut me off" from a feeling into a named cause. Pure and DOM-free
+// so it is testable from a fixture; the component layer just renders `text`.
+
+export type EndpointingMarkerKind = "typed" | "endpoint" | "unknown";
+
+export interface EndpointingMarker {
+  readonly kind: EndpointingMarkerKind;
+  /** Plain language — never a packet or message name (`eos.turn_complete`, raw `provider_stt`, …). */
+  readonly text: string;
+}
+
+/**
+ * Turn an endpointing decision into the words a person reads. Internal type
+ * values (`provider_stt`, `smart_turn`, `timer`, `text`, `force_finalized`, …)
+ * stay internal; this is the only place they cross into user-facing text.
+ *
+ * A genuinely unknown decision is returned as `kind: "unknown"` rather than
+ * guessed — a debugging surface that fabricates a cause is worse than one that
+ * says it does not know.
+ */
+export function endpointingMarker(owner?: string, reason?: string): EndpointingMarker {
+  if (owner === "text") {
+    return {
+      kind: "typed",
+      text: "You typed this turn — nothing transcribed you and nothing judged when you finished, so no endpointer ran.",
+    };
+  }
+  const who =
+    owner === "provider_stt"
+      ? "The speech-to-text provider"
+      : owner === "smart_turn"
+        ? "The Smart Turn model"
+        : owner === "timer"
+          ? "A turn timer"
+          : undefined;
+  if (who === undefined) {
+    return {
+      kind: "unknown",
+      text: "This backend did not report what ended the turn, so the cause is unknown.",
+    };
+  }
+  const text =
+    reason === "force_finalized"
+      ? `${who} force-finalized the transcript after a timeout, not a natural end of speech.`
+      : `${who} decided you had finished speaking.`;
+  return { kind: "endpoint", text };
+}