@absolutejs/voice 0.0.22-beta.473 → 0.0.22-beta.474

package/README.md

@@ -8,6 +8,107 @@ Use it when you want Vapi/Retell/Bland-style voice-agent capability, but you wan
 
 ## What's new
 
+### 0.0.22-beta.474 · Phase 5 runtime hookup — auto-wire monitor sockets to live sessions
+
+The Phase 5 monitor primitive is now first-class in the voice runtime. Pass a `monitor` binding to `voice({...})` and every session opened against the voice plugin auto-registers in the monitor registry, outbound TTS audio auto-fans-out to all listeners, and the close/superseded/session-switch paths deregister automatically. Supervisors connecting to the listen route get a live audio stream without any manual `record.emit()` calls.
+
+```ts
+import { Elysia } from "elysia";
+import {
+  createVoiceInMemoryMonitorRegistry,
+  createVoiceLiveMonitorRoutes,
+  createVoiceMonitorRuntimeBinding,
+  voice,
+} from "@absolutejs/voice";
+
+const monitorRegistry = createVoiceInMemoryMonitorRegistry();
+
+const app = new Elysia()
+  .use(
+    voice({
+      path: "/voice/realtime",
+      stt: deepgram({ apiKey: process.env.DEEPGRAM_API_KEY! }),
+      tts: elevenlabs({ apiKey: process.env.ELEVENLABS_API_KEY! }),
+      onTurn: async (session, turn, api) => {
+        // your business logic
+      },
+      session: sessionStore,
+      monitor: createVoiceMonitorRuntimeBinding(monitorRegistry, {
+        audioFormat: { channels: 1, container: "raw", encoding: "pcm_s16le", sampleRateHz: 24_000 },
+      }),
+    }),
+  )
+  .use(
+    createVoiceLiveMonitorRoutes({
+      registry: monitorRegistry,
+      authenticate: async ({ request }) => await verifySupervisorJWT(request),
+    }),
+  );
+```
+
+Surface additions:
+- **`createVoiceMonitorRuntimeBinding(registry, { audioFormat?, defaultSource? })`** — returns a `VoiceMonitorRuntimeBinding` you pass to the voice plugin's new `monitor` option. Internally, each session open calls `registerSession({ handle, sessionId })`, audio fans out via `emitAudio()` on every binary `socket.send`, and the close/superseded/session-switch paths call `deregister(reason)`.
+- **`VoicePluginConfig.monitor?: VoiceMonitorRuntimeBinding`** — new optional field on the main voice plugin config.
+- **`VoiceMonitorMutableRegistry.deregister(sessionId, reason?)`** — explicit deregister path so the runtime binding can tear down stale records on re-register without throwing. The `register()`-returned deregister fn now also accepts an optional reason.
+- The runtime binding is **opt-in** — if you don't pass `monitor`, voice behaves exactly as before. No overhead on the audio-send hot path beyond a single `Map.get`.
+
+4 new tests cover the runtime binding's register/emit/deregister cycle, no-op-after-deregister, re-registration tear-down, and the audioFormat + defaultSource option threading. Full voice suite now 963 pass / 1 pre-existing fail.
+
+### 0.0.22-beta.473 · Phase 5 — live listen + control monitor sockets (Vapi `monitorPlan` parity)
+
+Two new WebSocket routes per session that mirror Vapi's `monitorPlan.listenUrl` + `monitorPlan.controlUrl`. Supervisors can subscribe to a live call's outbound audio and send control commands (transfer, hangup, escalate, voicemail, no-answer, plus caller-defined mute/say/inject) without touching the call itself.
+
+```ts
+import { Elysia } from "elysia";
+import {
+  buildVoiceMonitorPlan,
+  createVoiceInMemoryMonitorRegistry,
+  createVoiceLiveMonitorRoutes,
+  createVoiceMonitorSession,
+} from "@absolutejs/voice";
+
+const registry = createVoiceInMemoryMonitorRegistry();
+
+// In your runtime: when a session opens, register it so supervisors can listen.
+const record = createVoiceMonitorSession({ handle, sessionId: handle.id });
+const deregister = registry.register(record);
+// When audio leaves the assistant, fan it out:
+//   record.emit({ at: Date.now(), chunk, format, source: 'assistant' });
+// On call end:
+//   deregister();
+
+const app = new Elysia()
+  .use(
+    createVoiceLiveMonitorRoutes({
+      authenticate: async ({ sessionId, route, request }) =>
+        await verifySupervisorJWT(request),
+      controlHandlers: {
+        say: async ({ message, session }) => {
+          await yourTtsRuntime.sayInSession(session.sessionId, message.text);
+          return { detail: `Said: ${message.text}`, ok: true, type: "say" };
+        },
+      },
+      htmlPath: "/voice/monitor",
+      registry,
+    }),
+  );
+
+const plan = buildVoiceMonitorPlan({
+  baseUrl: "wss://api.example.com",
+  sessionId: handle.id,
+});
+// plan.listenUrl  → wss://api.example.com/api/voice/monitor/<sessionId>/listen
+// plan.controlUrl → wss://api.example.com/api/voice/monitor/<sessionId>/control
+```
+
+Surface summary:
+- **`createVoiceInMemoryMonitorRegistry()`** — `{ register, get, list, emit, emitClose }`. Voice's session runtime (or any caller) wires `register()` on open + the returned deregister on close; `emit()` fans out outbound audio frames; `emitClose()` notifies listeners that the call ended.
+- **`createVoiceLiveMonitorRoutes(options)`** — Elysia plugin that mounts two `.ws()` routes per session: `:sessionId/listen` (read-only outbound audio as binary frames) and `:sessionId/control` (JSON control messages). Default handlers map `transfer`/`hangup`/`escalate`/`voicemail`/`no-answer` onto `VoiceSessionHandle` verbs; `mute`/`say`/`inject` require caller-supplied handlers via `controlHandlers`.
+- **`buildVoiceMonitorPlan(input)`** — Vapi-shaped helper returning `{ listenUrl, controlUrl }` for inclusion in your call-create API response.
+- **Auth hook** — `authenticate?: ({ sessionId, route, request }) => Promise<boolean>` runs at the start of both routes; rejected connections get a 4401 close.
+
+This is a **primitive**: wiring voice's existing `activeSessions` runtime into the registry (so audio actually flows from a real call without manual `record.emit()` calls) is the next step and intentionally left out to keep this change additive. Until then, registries built by hand (e.g. from a custom telephony adapter where you control the outbound audio buffer) work out of the box.
+
 ### 0.0.22-beta.472 · Phase 6 — multilingual STT proof gate
 
 `runVoiceMultilingualProof(...)` turns the `voice-fixtures-multilingual` corpus (FLEURS + BSC Catalan-Spanish code-switch + CoSHE Hindi-English code-switch) into a gateable readiness/proof artifact. Buyers evaluating Vapi-replacement can now run any combination of STT adapters against the multilingual corpus and assert per-language WER / pass-rate / term-recall budgets in CI.

package/dist/index.d.ts

@@ -225,6 +225,6 @@ export { buildVoiceProofPackInput, buildVoiceProofPack, buildVoiceProofPackFromO
 export type { VoiceProofPack, VoiceProofPackBuildContext, VoiceProofPackBuildContextOptions, VoiceProofPackBuildTiming, VoiceProofPackEvidence, VoiceProofPackInput, VoiceProofPackInputBuilderLoaderInput, VoiceProofPackInputBuilderOperationsLoaderInput, VoiceProofPackInputBuilderOptions, VoiceProofPackInputBuilderSupportBundle, VoiceProofPackRefreshState, VoiceProofPackRefreshStatus, VoiceProofPackRoutesOptions, VoiceProofPackSection, VoiceProofPackSourceValue, VoiceProofPackStatus, VoiceProofPackStaleWhileRefreshSource, VoiceProofPackStaleWhileRefreshSourceOptions, VoiceProofPackWriteResult, VoiceProofRefreshSnapshot, VoiceProofRefreshSnapshotOptions, } from "./proofPack";
 export { buildVoiceMultilingualProofReadinessCheck, renderVoiceMultilingualProofMarkdown, runVoiceMultilingualProof, } from "./multilingualProof";
 export type { VoiceMultilingualLanguageCode, VoiceMultilingualProofAdapterEntry, VoiceMultilingualProofAdapterReport, VoiceMultilingualProofDefaultThresholds, VoiceMultilingualProofLanguageMetrics, VoiceMultilingualProofLanguageReport, VoiceMultilingualProofLanguageThresholds, VoiceMultilingualProofOptions, VoiceMultilingualProofReadinessCheck, VoiceMultilingualProofReadinessOptions, VoiceMultilingualProofReport, } from "./multilingualProof";
-export { buildVoiceMonitorPlan, createVoiceInMemoryMonitorRegistry, createVoiceLiveMonitorRoutes, createVoiceMonitorSession, } from "./monitor";
-export type { VoiceMonitorAudioEvent, VoiceMonitorAudioSource, VoiceMonitorAuthenticate, VoiceMonitorAuthenticateInput, VoiceMonitorControlAck, VoiceMonitorControlHandler, VoiceMonitorControlHandlerInput, VoiceMonitorControlMessage, VoiceMonitorMutableRegistry, VoiceMonitorPlan, VoiceMonitorPlanInput, VoiceMonitorRegistry, VoiceMonitorRegistryRegisterInput, VoiceLiveMonitorRoutesOptions, VoiceMonitorSessionRecord, } from "./monitor";
+export { buildVoiceMonitorPlan, createVoiceInMemoryMonitorRegistry, createVoiceLiveMonitorRoutes, createVoiceMonitorRuntimeBinding, createVoiceMonitorSession, } from "./monitor";
+export type { VoiceMonitorAudioEvent, VoiceMonitorAudioSource, VoiceMonitorAuthenticate, VoiceMonitorAuthenticateInput, VoiceMonitorControlAck, VoiceMonitorControlHandler, VoiceMonitorControlHandlerInput, VoiceMonitorControlMessage, VoiceMonitorMutableRegistry, VoiceMonitorPlan, VoiceMonitorPlanInput, VoiceMonitorRegistry, VoiceMonitorRegistryRegisterInput, VoiceLiveMonitorRoutesOptions, VoiceMonitorRuntimeBindingOptions, VoiceMonitorSessionRecord, } from "./monitor";
 export * from "./types";

package/dist/index.js

@@ -6265,6 +6265,7 @@ var voice = (config) => {
   if (!config.stt && !config.realtime) {
     throw new Error("voice requires either an stt or realtime adapter.");
   }
+  const monitorBindings = new Map;
   const runtime = {
     activeSessions: new Map,
     logger: resolveLogger(config.logger),
@@ -6272,6 +6273,57 @@ var voice = (config) => {
     profileSwitchGuardedSessions: new Set,
     socketSessions: new WeakMap
   };
+  const monitor = config.monitor;
+  const registerMonitorSession = (sessionId, handle) => {
+    if (!monitor)
+      return;
+    const existing = monitorBindings.get(sessionId);
+    if (existing) {
+      try {
+        existing.deregister("superseded");
+      } catch {}
+      monitorBindings.delete(sessionId);
+    }
+    try {
+      const binding = monitor.registerSession({ handle, sessionId });
+      monitorBindings.set(sessionId, binding);
+    } catch (error) {
+      runtime.logger.warn?.(`[voice] failed to register session "${sessionId}" with monitor runtime: ${error instanceof Error ? error.message : String(error)}`);
+    }
+  };
+  const deregisterMonitorSession = (sessionId, reason) => {
+    const binding = monitorBindings.get(sessionId);
+    if (!binding)
+      return;
+    monitorBindings.delete(sessionId);
+    try {
+      binding.deregister(reason);
+    } catch (error) {
+      runtime.logger.warn?.(`[voice] failed to deregister monitor binding for session "${sessionId}": ${error instanceof Error ? error.message : String(error)}`);
+    }
+  };
+  const buildSocketAdapter = (ws, sessionId) => {
+    if (!monitor)
+      return createSocketAdapter(ws);
+    return {
+      close: async (code, reason) => {
+        ws.close(code, reason);
+      },
+      send: async (data) => {
+        if (typeof data !== "string") {
+          const binding = monitorBindings.get(sessionId);
+          if (binding) {
+            try {
+              binding.emitAudio(data);
+            } catch (error) {
+              runtime.logger.warn?.(`[voice] monitor emitAudio failed for session "${sessionId}": ${error instanceof Error ? error.message : String(error)}`);
+            }
+          }
+        }
+        ws.send(data);
+      }
+    };
+  };
   const onTurn = normalizeOnTurn(config.onTurn);
   const sessionOptions = resolveSessionOptions(config);
   const htmxOptions = config.htmx && typeof config.htmx === "object" ? config.htmx : undefined;
@@ -6394,6 +6446,7 @@ var voice = (config) => {
       }
       const session = runtime.activeSessions.get(socketState.sessionId);
       runtime.activeSessions.delete(socketState.sessionId);
+      deregisterMonitorSession(socketState.sessionId, reason ?? `ws-close-${String(code)}`);
       if (session) {
         await session.disconnect({
           code,
@@ -6417,6 +6470,7 @@ var voice = (config) => {
         if (message.type === "close" && current) {
           await current.close(message.reason);
           runtime.activeSessions.delete(sessionState.sessionId);
+          deregisterMonitorSession(sessionState.sessionId, message.reason);
         }
         if (message.type === "call_control" && current) {
           if (message.action === "transfer") {
@@ -6459,6 +6513,7 @@ var voice = (config) => {
           if (currentSession) {
             await currentSession.close("session-switch");
             runtime.activeSessions.delete(sessionState.sessionId);
+            deregisterMonitorSession(sessionState.sessionId, "session-switch");
           }
           sessionState.sessionId = message.sessionId;
           runtime.socketSessions.set(ws, {
@@ -6482,8 +6537,10 @@ var voice = (config) => {
       }
       const session = current ?? await createManagedSession(ws, sessionState.sessionId, sessionState.scenarioId ?? undefined);
       if (!current) {
-        runtime.activeSessions.set(sessionState.sessionId, session);
-        await session.connect(createSocketAdapter(ws));
+        const typedSession = session;
+        runtime.activeSessions.set(sessionState.sessionId, typedSession);
+        registerMonitorSession(sessionState.sessionId, typedSession);
+        await session.connect(buildSocketAdapter(ws, sessionState.sessionId));
       }
       await session.receiveAudio(audio);
     },
@@ -6493,10 +6550,13 @@ var voice = (config) => {
       if (existing) {
         await existing.close("superseded");
         runtime.activeSessions.delete(sessionState.sessionId);
+        deregisterMonitorSession(sessionState.sessionId, "superseded");
       }
       const session = await createManagedSession(ws, sessionState.sessionId, sessionState.scenarioId ?? undefined);
-      runtime.activeSessions.set(sessionState.sessionId, session);
-      await session.connect(createSocketAdapter(ws));
+      const typedSession = session;
+      runtime.activeSessions.set(sessionState.sessionId, typedSession);
+      registerMonitorSession(sessionState.sessionId, typedSession);
+      await session.connect(buildSocketAdapter(ws, sessionState.sessionId));
     }
   }).use(htmxRoutes());
 };
@@ -44501,7 +44561,15 @@ var createVoiceMonitorSession = (input) => {
 };
 var createVoiceInMemoryMonitorRegistry = () => {
   const records = new Map;
+  const deregister = (sessionId, reason) => {
+    const existing = records.get(sessionId);
+    if (!existing)
+      return;
+    records.delete(sessionId);
+    existing.emitClose(reason ?? "deregistered");
+  };
   return {
+    deregister,
     emit: (sessionId, event) => {
       records.get(sessionId)?.emit(event);
     },
@@ -44525,10 +44593,7 @@ var createVoiceInMemoryMonitorRegistry = () => {
         record.onClose((reason) => wrapped.emitClose(reason));
       }
       records.set(record.sessionId, wrapped);
-      return () => {
-        records.delete(record.sessionId);
-        wrapped.emitClose("deregistered");
-      };
+      return (reason) => deregister(record.sessionId, reason);
     }
   };
 };
@@ -44665,6 +44730,52 @@ var buildVoiceMonitorPlan = (input) => {
     listenUrl: `${baseUrl}${substituteSessionId(listenTemplate, input.sessionId)}`
   };
 };
+var DEFAULT_RUNTIME_AUDIO_FORMAT = {
+  channels: 1,
+  container: "raw",
+  encoding: "pcm_s16le",
+  sampleRateHz: 16000
+};
+var toUint8 = (chunk) => chunk instanceof Uint8Array ? chunk : new Uint8Array(chunk);
+var createVoiceMonitorRuntimeBinding = (registry, options = {}) => {
+  const audioFormat = options.audioFormat ?? DEFAULT_RUNTIME_AUDIO_FORMAT;
+  const defaultSource = options.defaultSource ?? "assistant";
+  return {
+    registerSession: (input) => {
+      registry.deregister(input.sessionId, "superseded");
+      const record = createVoiceMonitorSession({
+        handle: input.handle,
+        metadata: input.metadata,
+        sessionId: input.sessionId
+      });
+      const deregisterFromRegistry = registry.register(record);
+      let closed = false;
+      return {
+        deregister: (reason) => {
+          if (closed)
+            return;
+          closed = true;
+          try {
+            deregisterFromRegistry(reason);
+          } catch {}
+        },
+        emitAudio: (chunk, opts) => {
+          if (closed)
+            return;
+          const bytes = toUint8(chunk);
+          if (bytes.byteLength === 0)
+            return;
+          record.emit({
+            at: Date.now(),
+            chunk: bytes,
+            format: audioFormat,
+            source: opts?.source ?? defaultSource
+          });
+        }
+      };
+    }
+  };
+};
 var resolveSessionId3 = (ws) => {
   const params = ws.data?.params;
   if (!params)
@@ -45358,6 +45469,7 @@ export {
   createVoiceObservabilityExportReplayRoutes,
   createVoiceMonitorWebhookNotifier,
   createVoiceMonitorSession,
+  createVoiceMonitorRuntimeBinding,
   createVoiceMonitorRunnerRoutes,
   createVoiceMonitorRunner,
   createVoiceMonitorRoutes,

package/dist/monitor.d.ts

@@ -1,5 +1,5 @@
 import { Elysia } from "elysia";
-import type { AudioFormat, VoiceSessionHandle, VoiceSessionRecord } from "./types";
+import type { AudioFormat, VoiceMonitorRuntimeBinding, VoiceSessionHandle, VoiceSessionRecord } from "./types";
 export type VoiceMonitorAudioSource = "assistant" | "caller" | (string & {});
 export type VoiceMonitorAudioEvent = {
     at: number;
@@ -23,7 +23,12 @@ export type VoiceMonitorRegistry = {
 export type VoiceMonitorMutableRegistry = VoiceMonitorRegistry & {
     emit: (sessionId: string, event: VoiceMonitorAudioEvent) => void;
     emitClose: (sessionId: string, reason?: string) => void;
-    register: (record: VoiceMonitorSessionRecord) => () => void;
+    /**
+     * Deregister a session by id. No-op if the id isn't registered. The
+     * `reason` flows into the close fan-out exactly once.
+     */
+    deregister: (sessionId: string, reason?: string) => void;
+    register: (record: VoiceMonitorSessionRecord) => (reason?: string) => void;
 };
 export type VoiceMonitorRegistryRegisterInput = {
     handle: VoiceSessionHandle<unknown, VoiceSessionRecord, unknown>;
@@ -108,6 +113,11 @@ export type VoiceMonitorPlan = {
     listenUrl: string;
 };
 export declare const buildVoiceMonitorPlan: (input: VoiceMonitorPlanInput) => VoiceMonitorPlan;
+export type VoiceMonitorRuntimeBindingOptions = {
+    audioFormat?: AudioFormat;
+    defaultSource?: VoiceMonitorAudioSource;
+};
+export declare const createVoiceMonitorRuntimeBinding: (registry: VoiceMonitorMutableRegistry, options?: VoiceMonitorRuntimeBindingOptions) => VoiceMonitorRuntimeBinding;
 export declare const createVoiceLiveMonitorRoutes: (options: VoiceLiveMonitorRoutesOptions) => Elysia<"", {
     decorator: {};
     store: {};

package/dist/types.d.ts

@@ -411,6 +411,20 @@ export type VoiceSocket = {
     send: (data: string | Uint8Array | ArrayBuffer) => void | Promise<void>;
     close: (code?: number, reason?: string) => void | Promise<void>;
 };
+export type VoiceMonitorRuntimeSessionBinding = {
+    deregister: (reason?: string) => void;
+    emitAudio: (chunk: Uint8Array | ArrayBuffer, options?: {
+        source?: "assistant" | "caller" | (string & {});
+    }) => void;
+};
+export type VoiceMonitorRuntimeRegisterInput = {
+    handle: VoiceSessionHandle<unknown, VoiceSessionRecord, unknown>;
+    metadata?: Record<string, unknown>;
+    sessionId: string;
+};
+export type VoiceMonitorRuntimeBinding = {
+    registerSession: (input: VoiceMonitorRuntimeRegisterInput) => VoiceMonitorRuntimeSessionBinding;
+};
 export type VoiceSessionHandle<TContext = unknown, TSession extends VoiceSessionRecord = VoiceSessionRecord, TResult = unknown> = {
     id: string;
     connect: (socket: VoiceSocket) => Promise<void>;
@@ -679,6 +693,7 @@ export type VoicePluginConfig<TContext = unknown, TSession extends VoiceSessionR
     handoff?: VoiceHandoffConfig<TContext, TSession, TResult>;
     ops?: VoiceRuntimeOpsConfig<TContext, TSession, TResult>;
     liveOps?: VoiceLiveOpsRuntimeConfig;
+    monitor?: VoiceMonitorRuntimeBinding;
     profileSwitchGuard?: VoicePluginProfileSwitchGuardConfig<TContext, TSession, TResult>;
     trace?: VoiceTraceEventStore;
 } & VoiceRouteConfig<TContext, TSession, TResult>;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@absolutejs/voice",
-  "version": "0.0.22-beta.473",
+  "version": "0.0.22-beta.474",
   "description": "Voice primitives and Elysia plugin for AbsoluteJS",
   "repository": {
     "type": "git",