@voicethere/agent 0.1.0

package/README.md

@@ -0,0 +1,251 @@
+# @voicethere/agent
+
+VoiceThere **customer agent SDK** — TypeScript types and runtime helpers for sandboxed child bundles running inside [`voicethere/runner`](https://github.com/voicethere/runner).
+
+**npm:** `@voicethere/agent`  
+**Repo:** [`voicethere/agent`](https://github.com/voicethere/agent)
+
+## Role
+
+| Layer     | Package                                                     | Runs in                              |
+| --------- | ----------------------------------------------------------- | ------------------------------------ |
+| Parent    | [`voicethere/runner`](https://github.com/voicethere/runner) | Trusted Node + WebRTC + speech stack |
+| **Child** | **`@voicethere/agent`**                                     | Sandboxed customer `agent.js` bundle |
+
+The child receives speech lifecycle events over IPC (same shapes as `@node-webrtc-rust/sdk/voice`) and calls `speak()` to request TTS from the parent.
+
+## Quick start
+
+```bash
+git clone https://github.com/voicethere/agent.git
+cd agent
+npm install
+npm run build
+```
+
+Run with a local [runner](https://github.com/voicethere/runner) (clone that repo alongside this one):
+
+```bash
+cd ../runner
+AGENT_BUNDLE_PATH=../agent/dist/agent.js npm run start
+```
+
+Open the runner URL in a browser, connect, and speak.
+
+## Verify locally (sandbox, no WebRTC)
+
+Before deploying to VoiceThere, confirm your bundle loads and responds under the **same Node sandbox** the runner uses:
+
+```bash
+npm run verify:local
+```
+
+| Script                    | When to use                                                                                                               |
+| ------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
+| `npm run verify:local`    | **Default** — `npm run build`, then fork `dist/agent.js` in the sandbox and assert a `speak` reply to `user_speech_final` |
+| `npm run verify:local:only` | Re-run smoke after build; optional `AGENT_BUNDLE_PATH=./dist/agent.js` or `--bundle <path>`                               |
+
+This checks bundle load, IPC, and Node permission flags. It does **not** replace a voice roundtrip — use [`voicethere/runner`](https://github.com/voicethere/runner) for mic/WebRTC E2E.
+
+Harness: [`scripts/sandbox/`](./scripts/sandbox/) (aligned with [`voicethere/runner`](https://github.com/voicethere/runner) child launcher).
+
+## API
+
+```typescript
+import {
+  agentLog,
+  defineAgent,
+  speak,
+  type SpeechEvent,
+} from '@voicethere/agent'
+import { SPEECH_EVENT_TYPE } from '@node-webrtc-rust/sdk/voice'
+
+defineAgent({
+  onSessionStart({ sessionId }) {
+    speak(sessionId, 'Hello!')
+  },
+  onUserSpeechFinal({ sessionId, text }) {
+    speak(sessionId, `You said: ${text}`)
+  },
+  onSpeechEvent({ sessionId }, speech: SpeechEvent) {
+    if (speech.type === SPEECH_EVENT_TYPE.bargeIn) {
+      agentLog('info', `User interrupted on ${sessionId}`)
+    }
+  },
+})
+```
+
+| Export                                          | Purpose                                                                               |
+| ----------------------------------------------- | ------------------------------------------------------------------------------------- |
+| `defineAgent`                                   | Register `onSessionStart`, `onSpeechEvent`, `onUserSpeechFinal`, `onSessionEnd`       |
+| `SpeechEvent`, `SpeechEventType`                | Re-exported **types** from `@node-webrtc-rust/sdk/voice`                              |
+| `SPEECH_EVENT_TYPE`                             | Import from `@node-webrtc-rust/sdk/voice` (runtime constants; not bundled into child) |
+| `speak`                                         | Request parent TTS                                                                    |
+| `agentLog`                                      | Forward structured logs to parent                                                     |
+| `ParentToChildMessage` / `ChildToParentMessage` | IPC contract shared with [`voicethere/runner`](https://github.com/voicethere/runner)   |
+
+### Speech events (parent → child)
+
+Forwarded from the runner voice pipeline as SDK `SpeechEvent` payloads on `speech_event.event` (`event.type`, optional `text` / `error`):
+
+| Event                                         | Typical use in custom agent                             |
+| --------------------------------------------- | ------------------------------------------------------- |
+| `user_speaking_start` / `user_speaking_end`   | UI state, turn-taking                                   |
+| `user_speech_partial`                         | Live captions, early barge-in logic                     |
+| `user_speech_final`                           | Primary turn boundary (`onUserSpeechFinal` convenience) |
+| `agent_speaking_start` / `agent_speaking_end` | Know when TTS playback starts/stops                     |
+| `barge_in`                                    | User interrupted agent playback                         |
+| `vad_triggered`, `stt_stream_*`, `user_stt_*` | Low-level pipeline hooks                                |
+| `error`                                       | Vendor or pipeline failure                              |
+
+Copy [`templates/agent.ts`](./templates/agent.ts) as a starting point — exhaustive `switch` over all 14 `SpeechEvent` types with per-peer state stubs and `agentLog` tracing.
+
+## Building your agent bundle
+
+**Recommended:** single ESM bundle (esbuild or similar):
+
+```bash
+npm install @voicethere/agent esbuild
+npx esbuild agent.ts --bundle --platform=node --format=esm --outfile=dist/agent.js
+```
+
+Upload `dist/agent.js` (or point `AGENT_BUNDLE_PATH` at it locally). Inlining dependencies avoids runtime `node_modules` resolution inside the sandbox.
+
+## Sandbox and security model
+
+Customer code runs in a **forked child process**, separate from the trusted runner parent (Sherpa, WebRTC, TTS). Security is layered:
+
+```text
+┌──────────────────────────────────────────────────────────────┐
+│  Runner parent (trusted) — full Node, WebRTC, speech stack   │
+│    fork(loader-entry.js, execArgv: [--permission, …])       │
+│         │ IPC (process.send / on('message'))                  │
+│         ▼                                                     │
+│  Customer child — Node Permission Model + stripped env      │
+│    loader-entry.js → import(your agent.js)                   │
+└──────────────────────────────────────────────────────────────┘
+          │ same pod network namespace (K8s)
+          ▼
+   Cilium NetworkPolicy on runner pod (egress rules)
+```
+
+### Layer 1 — Process isolation
+
+| Mechanism | What it means for your bundle |
+| --------- | ----------------------------- |
+| **Separate process** | Crash or `process.exit` in your bundle does not take down the parent voice stack |
+| **IPC only** | Talk to the platform via `defineAgent` / `speak` / `agentLog` — not direct WebRTC or STT |
+| **Stripped `process.env`** | Child receives only `NODE_ENV`, internal loader path, and allowlisted keys (`SESSION_ID`, `PROJECT_ID`, `BUILD_ID`) — not parent/kube secrets |
+| **Console redirection** | `console.log` / `warn` / `error` → IPC logs; raw stdout is not forwarded to cluster logs |
+
+### Layer 2 — Node `--permission` (runtime-enforced)
+
+The parent starts the child with Node’s [Permission Model](https://nodejs.org/api/permissions.html) (Node **22+**). Capabilities are **deny-by-default**; only explicitly granted flags apply.
+
+**Granted today** (via `execArgv` on `fork()`):
+
+| Flag | Effect |
+| ---- | ------ |
+| `--permission` | Enables restriction mode |
+| `--allow-fs-read=<loaderDir>` | Read files under the runner’s child loader directory |
+| `--allow-fs-read=<bundleParentDir>` | Read files under the **directory containing your `agent.js`** (see below) |
+
+**Not granted → blocked at runtime:**
+
+| Missing flag | What fails |
+| ------------ | ---------- |
+| No `--allow-child-process` | `child_process`, `exec`, `spawn`, `fork` |
+| No `--allow-fs-write` | Any file write (`writeFile`, logs to disk, etc.) |
+| No extra `--allow-fs-read` paths | Reading `/etc/passwd`, parent files, etc. outside bundle dir |
+| No `--allow-addons` | Native `.node` addons (`bcrypt`, `sharp`, …) |
+| No `--allow-worker-threads` | `worker_threads` |
+| No `--allow-wasi` | WASI modules |
+
+This is **not** an import allowlist — Node gates **capability classes**, not package names. Using `node:fs` inside the allowed read tree can work; using it on `/etc/passwd` does not.
+
+### Bundle directory vs single file
+
+`--allow-fs-read` is applied to **`dirname(bundlePath)`**, not only the `.js` file:
+
+```text
+/app/agents/my-build/
+  agent.js          ← entry (AGENT_BUNDLE_PATH)
+  helper.js         ← importable if your bundle references it
+  data.json         ← readable via fs if you import/read it
+  node_modules/     ← JS-only deps may resolve; native addons still blocked
+```
+
+| Artifact in bundle dir | Works? |
+| ---------------------- | ------ |
+| Single bundled `agent.js` (recommended) | Yes |
+| Extra pure `.js` / `.json` siblings | Usually yes (same allowed tree) |
+| `node_modules/` with **JavaScript-only** packages | Often yes (Node resolves imports by reading under that tree) |
+| **Native** npm packages (`.node` binaries) | **No** — requires `--allow-addons` (not enabled) |
+| Packages that **spawn subprocesses** | **No** — no `--allow-child-process` |
+
+Prefer **one esbuild bundle** so production behavior matches `npm run verify:local`.
+
+### Layer 3 — Platform policy (documented; not all enforced in-process)
+
+These are **unsupported** in customer bundles even if Node might not block them today:
+
+| Capability | Enforcement |
+| ---------- | ----------- |
+| **Outbound network** (`fetch`, `http`, `net`, `dns`) | Not gated by `--permission`; child shares the **pod** network. Runner pods use **Cilium NetworkPolicy** (public egress; cluster/RFC1918 denied). **Do not** rely on network from child — use parent/platform APIs. |
+| **`process.exit`** | Not blocked — kills your agent leg; parent may play crash TTS |
+| **Direct WebRTC / mic / STT / TTS** | Parent only — use `speak()` and speech event handlers |
+
+### Layer 4 — Kubernetes (runner pod)
+
+On cluster deploy, the runner pod also has Helm hardening (non-root, read-only rootfs, dropped caps) and **NetworkPolicy** for egress. That applies to the whole pod (parent + child).
+
+### What you should use in agent code
+
+**Supported**
+
+- `@voicethere/agent` (`defineAgent`, `speak`, `agentLog`, `onSpeechEvent`, …)
+- Pure TypeScript/JavaScript logic and in-memory state
+- Allowlisted env from `onSessionStart` (`SESSION_ID`, `PROJECT_ID`, `BUILD_ID`)
+- `SPEECH_EVENT_TYPE` from `@node-webrtc-rust/sdk/voice` at build time (avoid bundling the full SDK runtime into the child when possible)
+
+**Blocked or unsupported**
+
+- Subprocesses, shells, `child_process`
+- Arbitrary filesystem access outside your bundle deployment directory
+- File writes
+- Native Node addons (`.node`)
+- `worker_threads` (not allowed)
+- Direct media/network stack access
+
+**Pre-publish checklist**
+
+1. `npm run build` — produce `dist/agent.js`
+2. `npm run verify:local` — sandbox + IPC smoke (same flags as production child)
+3. Optional: voice E2E with [`voicethere/runner`](https://github.com/voicethere/runner)
+
+## Build outputs
+
+| Path            | Purpose                                                     |
+| --------------- | ----------------------------------------------------------- |
+| `dist/index.js` | Published npm library entry                                 |
+| `dist/agent.js` | Example bundle (`examples/agent.ts`) for local runner / verify |
+
+## Scripts
+
+```bash
+npm run build           # library + example bundle
+npm run verify:local    # sandbox smoke (build + fork bundle)
+npm run test:ci         # typecheck + vitest
+```
+
+## Release
+
+See [`scripts/RELEASE.md`](./scripts/RELEASE.md) — tag `release/X.Y.Z` triggers npm publish (same workflow pattern as [`node-webrtc-rust`](https://github.com/akirilyuk/node-webrtc-rust)).
+
+## Related
+
+| Repo                                                                          | Purpose                                                       |
+| ----------------------------------------------------------------------------- | ------------------------------------------------------------- |
+| [`voicethere/runner`](https://github.com/voicethere/runner)                   | Session worker that hosts your `agent.js` bundle              |
+| [`voicethere/cli`](https://github.com/voicethere/cli)                         | CLI for the VoiceThere platform (projects, deploys, sessions) |
+| [`akirilyuk/node-webrtc-rust`](https://github.com/akirilyuk/node-webrtc-rust) | WebRTC + voice SDK (`SpeechEvent` types)                      |

package/dist/agent.js

@@ -0,0 +1,148 @@
+// src/runtime.ts
+function isParentMessage(value) {
+  if (!value || typeof value !== "object") return false;
+  const msg = value;
+  return msg.type === "session_start" || msg.type === "speech_event" || msg.type === "session_end";
+}
+function defineAgent(handlers) {
+  process.on("message", (message) => {
+    if (!isParentMessage(message)) return;
+    void (async () => {
+      try {
+        switch (message.type) {
+          case "session_start":
+            await handlers.onSessionStart?.({
+              sessionId: message.sessionId,
+              env: message.env
+            });
+            break;
+          case "speech_event":
+            await handlers.onSpeechEvent?.(
+              { sessionId: message.sessionId },
+              message.event
+            );
+            if (message.event.type === "user_speech_final" && typeof message.event.text === "string" && message.event.text.trim()) {
+              await handlers.onUserSpeechFinal?.({
+                sessionId: message.sessionId,
+                text: message.event.text.trim()
+              });
+            }
+            break;
+          case "session_end":
+            await handlers.onSessionEnd?.({ sessionId: message.sessionId });
+            break;
+        }
+      } catch (error) {
+        const errMessage = error instanceof Error ? error.message : String(error);
+        process.send?.({
+          type: "agent_error",
+          sessionId: message.sessionId,
+          message: errMessage
+        });
+      }
+    })();
+  });
+}
+function speak(sessionId, text) {
+  process.send?.({ type: "speak", sessionId, text });
+}
+function agentLog(level, message) {
+  process.send?.({ type: "log", level, message });
+}
+
+// examples/agent.ts
+var peers = /* @__PURE__ */ new Map();
+function peerState(sessionId) {
+  let state = peers.get(sessionId);
+  if (!state) {
+    state = {
+      userSpeaking: false,
+      agentSpeaking: false,
+      lastPartial: "",
+      sttActive: false
+    };
+    peers.set(sessionId, state);
+  }
+  return state;
+}
+function formatSpeechDetail(speech) {
+  if (speech.text) return `"${speech.text}"`;
+  if (speech.error) return speech.error;
+  return "";
+}
+function handleSpeechEvent(sessionId, speech) {
+  const state = peerState(sessionId);
+  const detail = formatSpeechDetail(speech);
+  switch (speech.type) {
+    case "user_speaking_start":
+      state.userSpeaking = true;
+      agentLog("info", `[${sessionId}] user_speaking_start`);
+      break;
+    case "user_speaking_end":
+      state.userSpeaking = false;
+      agentLog("info", `[${sessionId}] user_speaking_end`);
+      break;
+    case "vad_triggered":
+      agentLog("info", `[${sessionId}] vad_triggered`);
+      break;
+    case "stt_stream_start":
+      agentLog("info", `[${sessionId}] stt_stream_start`);
+      break;
+    case "stt_stream_end":
+      agentLog("info", `[${sessionId}] stt_stream_end`);
+      break;
+    case "user_stt_start":
+      state.sttActive = true;
+      state.lastPartial = "";
+      agentLog("info", `[${sessionId}] user_stt_start`);
+      break;
+    case "user_stt_end":
+      state.sttActive = false;
+      agentLog("info", `[${sessionId}] user_stt_end`);
+      break;
+    case "user_stt_not_found":
+      agentLog("info", `[${sessionId}] user_stt_not_found`);
+      break;
+    case "user_speech_partial":
+      state.lastPartial = speech.text ?? "";
+      agentLog("info", `[${sessionId}] user_speech_partial ${detail}`);
+      break;
+    case "user_speech_final":
+      state.lastPartial = "";
+      agentLog("info", `[${sessionId}] user_speech_final ${detail}`);
+      break;
+    case "agent_speaking_start":
+      state.agentSpeaking = true;
+      agentLog("info", `[${sessionId}] agent_speaking_start`);
+      break;
+    case "agent_speaking_end":
+      state.agentSpeaking = false;
+      agentLog("info", `[${sessionId}] agent_speaking_end`);
+      break;
+    case "barge_in":
+      agentLog("info", `[${sessionId}] barge_in`);
+      break;
+    case "error":
+      agentLog("error", `[${sessionId}] error: ${speech.error ?? "unknown"}`);
+      break;
+    default: {
+      const _exhaustive = speech.type;
+      agentLog("error", `[${sessionId}] unhandled: ${String(_exhaustive)}`);
+    }
+  }
+}
+defineAgent({
+  onSessionStart({ sessionId }) {
+    speak(sessionId, "Hello! How can I help?");
+  },
+  onSpeechEvent({ sessionId }, speech) {
+    handleSpeechEvent(sessionId, speech);
+  },
+  onUserSpeechFinal({ sessionId, text }) {
+    speak(sessionId, `You said: ${text}`);
+  },
+  onSessionEnd({ sessionId }) {
+    peers.delete(sessionId);
+    agentLog("info", `session_end ${sessionId}`);
+  }
+});

package/dist/index.d.ts

@@ -0,0 +1,4 @@
+export type { SpeechEvent, SpeechEventListener, SpeechEventName, SpeechEventType, } from "@node-webrtc-rust/sdk/voice";
+export { ALLOWED_CHILD_ENV_KEYS, type AgentErrorMessage, type AgentLogMessage, type AllowedChildEnvKey, type ChildToParentMessage, type ParentToChildMessage, type SessionEndMessage, type SessionStartMessage, type SpeakMessage, type SpeechEventMessage, } from "./protocol.js";
+export { agentLog, defineAgent, speak, type AgentHandlers, type SessionContext, type SpeechContext, type SpeechEventContext, } from "./runtime.js";
+//# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,YAAY,EACV,WAAW,EACX,mBAAmB,EACnB,eAAe,EACf,eAAe,GAChB,MAAM,6BAA6B,CAAC;AAErC,OAAO,EACL,sBAAsB,EACtB,KAAK,iBAAiB,EACtB,KAAK,eAAe,EACpB,KAAK,kBAAkB,EACvB,KAAK,oBAAoB,EACzB,KAAK,oBAAoB,EACzB,KAAK,iBAAiB,EACtB,KAAK,mBAAmB,EACxB,KAAK,YAAY,EACjB,KAAK,kBAAkB,GACxB,MAAM,eAAe,CAAC;AAEvB,OAAO,EACL,QAAQ,EACR,WAAW,EACX,KAAK,EACL,KAAK,aAAa,EAClB,KAAK,cAAc,EACnB,KAAK,aAAa,EAClB,KAAK,kBAAkB,GACxB,MAAM,cAAc,CAAC"}

package/dist/index.js

@@ -0,0 +1,3 @@
+export { ALLOWED_CHILD_ENV_KEYS, } from "./protocol.js";
+export { agentLog, defineAgent, speak, } from "./runtime.js";
+//# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAOA,OAAO,EACL,sBAAsB,GAUvB,MAAM,eAAe,CAAC;AAEvB,OAAO,EACL,QAAQ,EACR,WAAW,EACX,KAAK,GAKN,MAAM,cAAc,CAAC"}

package/dist/protocol.d.ts

@@ -0,0 +1,112 @@
+/**
+ * IPC message shapes between trusted runner parent and isolated customer child.
+ *
+ * Transport: Node.js `process.send` / `process.on('message')` on a forked child.
+ * Speech payloads use {@link SpeechEvent} from `@node-webrtc-rust/sdk/voice` unchanged.
+ *
+ * IPC shapes are shared with [`voicethere/runner`](https://github.com/voicethere/runner).
+ *
+ * @packageDocumentation
+ */
+import type { SpeechEvent } from "@node-webrtc-rust/sdk/voice";
+/**
+ * Messages the trusted runner parent may send to the sandboxed customer child.
+ *
+ * Register handlers via {@link defineAgent} in `@voicethere/agent` — do not read
+ * `process.on('message')` directly in customer bundles.
+ */
+export type ParentToChildMessage = SessionStartMessage | SpeechEventMessage | SessionEndMessage;
+/**
+ * Messages the customer child may send back to the runner parent.
+ *
+ * Prefer {@link speak} and {@link agentLog} helpers over raw `process.send`.
+ */
+export type ChildToParentMessage = SpeakMessage | AgentLogMessage | AgentErrorMessage;
+/**
+ * A WebRTC peer connected to the runner and mapped to this child process.
+ *
+ * Emitted once per `sessionId` before the first {@link SpeechEventMessage}.
+ * `env.SESSION_ID` matches {@link SessionStartMessage.sessionId}.
+ */
+export interface SessionStartMessage {
+    type: "session_start";
+    /** Browser/signaling peer id for this conversation leg. */
+    sessionId: string;
+    /**
+     * Allowlisted environment variables copied from the runner process.
+     * Keys are a subset of {@link ALLOWED_CHILD_ENV_KEYS}.
+     */
+    env: Record<string, string>;
+}
+/**
+ * Forwards one speech lifecycle event from the parent Sherpa/VAD/STT/TTS pipeline.
+ *
+ * The {@link SpeechEvent} shape matches `@node-webrtc-rust/sdk/voice` — see SDK docs
+ * for `SpeechEventType` semantics (`user_speech_final`, `barge_in`, etc.).
+ *
+ * Delivered to customer code as `onSpeechEvent(ctx, message.event)`; `user_speech_final`
+ * also triggers the `onUserSpeechFinal` handler when `event.text` is non-empty.
+ */
+export interface SpeechEventMessage {
+    type: "speech_event";
+    /** Peer/session id the event belongs to. */
+    sessionId: string;
+    /** Native pipeline event payload (`type`, optional `text` / `error`). */
+    event: SpeechEvent;
+}
+/**
+ * The peer disconnected or the runner is tearing down this session leg.
+ *
+ * Emitted when the runner unregisters a peer or tears down the session.
+ */
+export interface SessionEndMessage {
+    type: "session_end";
+    /** Peer/session id that ended. */
+    sessionId: string;
+}
+/**
+ * Ask the parent to synthesize speech on the agent outbound WebRTC track.
+ *
+ * Handled by the runner parent, which synthesizes audio on the outbound WebRTC track.
+ * TTS does **not** run inside the sandboxed child.
+ */
+export interface SpeakMessage {
+    type: "speak";
+    /** Target peer/session id (must match a prior {@link SessionStartMessage}). */
+    sessionId: string;
+    /** UTF-8 text passed to the parent TTS vendor. */
+    text: string;
+}
+/**
+ * Structured log line forwarded to runner stdout / Winston.
+ *
+ * Use {@link agentLog} instead of calling `process.send` directly.
+ */
+export interface AgentLogMessage {
+    type: "log";
+    level: "info" | "error";
+    message: string;
+}
+/**
+ * Unhandled exception or rejected promise in customer agent code.
+ *
+ * The parent may play crash TTS and treat the child as failed. Prefer try/catch in
+ * handlers; use this only for fatal reporting.
+ */
+export interface AgentErrorMessage {
+    type: "agent_error";
+    /** Peer/session id active when the error occurred. */
+    sessionId: string;
+    /** Human-readable error summary (no stack traces required). */
+    message: string;
+}
+/**
+ * Environment variable names the runner may inject into {@link SessionStartMessage.env}.
+ *
+ * The runner may add more project-specific keys over time; customer bundles must
+ * not read `process.env` directly — only the `env` object on session start.
+ */
+export declare const ALLOWED_CHILD_ENV_KEYS: readonly ["SESSION_ID", "PROJECT_ID", "BUILD_ID"];
+/** Union of allowlisted env key names. */
+export type AllowedChildEnvKey = (typeof ALLOWED_CHILD_ENV_KEYS)[number];
+//# sourceMappingURL=protocol.d.ts.map

package/dist/protocol.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"protocol.d.ts","sourceRoot":"","sources":["../src/protocol.ts"],"names":[],"mappings":"AAAA;;;;;;;;;GASG;AAEH,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,6BAA6B,CAAC;AAE/D;;;;;GAKG;AACH,MAAM,MAAM,oBAAoB,GAC5B,mBAAmB,GACnB,kBAAkB,GAClB,iBAAiB,CAAC;AAEtB;;;;GAIG;AACH,MAAM,MAAM,oBAAoB,GAC5B,YAAY,GACZ,eAAe,GACf,iBAAiB,CAAC;AAEtB;;;;;GAKG;AACH,MAAM,WAAW,mBAAmB;IAClC,IAAI,EAAE,eAAe,CAAC;IACtB,2DAA2D;IAC3D,SAAS,EAAE,MAAM,CAAC;IAClB;;;OAGG;IACH,GAAG,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;CAC7B;AAED;;;;;;;;GAQG;AACH,MAAM,WAAW,kBAAkB;IACjC,IAAI,EAAE,cAAc,CAAC;IACrB,4CAA4C;IAC5C,SAAS,EAAE,MAAM,CAAC;IAClB,yEAAyE;IACzE,KAAK,EAAE,WAAW,CAAC;CACpB;AAED;;;;GAIG;AACH,MAAM,WAAW,iBAAiB;IAChC,IAAI,EAAE,aAAa,CAAC;IACpB,kCAAkC;IAClC,SAAS,EAAE,MAAM,CAAC;CACnB;AAED;;;;;GAKG;AACH,MAAM,WAAW,YAAY;IAC3B,IAAI,EAAE,OAAO,CAAC;IACd,+EAA+E;IAC/E,SAAS,EAAE,MAAM,CAAC;IAClB,kDAAkD;IAClD,IAAI,EAAE,MAAM,CAAC;CACd;AAED;;;;GAIG;AACH,MAAM,WAAW,eAAe;IAC9B,IAAI,EAAE,KAAK,CAAC;IACZ,KAAK,EAAE,MAAM,GAAG,OAAO,CAAC;IACxB,OAAO,EAAE,MAAM,CAAC;CACjB;AAED;;;;;GAKG;AACH,MAAM,WAAW,iBAAiB;IAChC,IAAI,EAAE,aAAa,CAAC;IACpB,sDAAsD;IACtD,SAAS,EAAE,MAAM,CAAC;IAClB,+DAA+D;IAC/D,OAAO,EAAE,MAAM,CAAC;CACjB;AAED;;;;;GAKG;AACH,eAAO,MAAM,sBAAsB,mDAIzB,CAAC;AAEX,0CAA0C;AAC1C,MAAM,MAAM,kBAAkB,GAAG,CAAC,OAAO,sBAAsB,CAAC,CAAC,MAAM,CAAC,CAAC"}

package/dist/protocol.js

@@ -0,0 +1,22 @@
+/**
+ * IPC message shapes between trusted runner parent and isolated customer child.
+ *
+ * Transport: Node.js `process.send` / `process.on('message')` on a forked child.
+ * Speech payloads use {@link SpeechEvent} from `@node-webrtc-rust/sdk/voice` unchanged.
+ *
+ * IPC shapes are shared with [`voicethere/runner`](https://github.com/voicethere/runner).
+ *
+ * @packageDocumentation
+ */
+/**
+ * Environment variable names the runner may inject into {@link SessionStartMessage.env}.
+ *
+ * The runner may add more project-specific keys over time; customer bundles must
+ * not read `process.env` directly — only the `env` object on session start.
+ */
+export const ALLOWED_CHILD_ENV_KEYS = [
+    "SESSION_ID",
+    "PROJECT_ID",
+    "BUILD_ID",
+];
+//# sourceMappingURL=protocol.js.map

package/dist/protocol.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"protocol.js","sourceRoot":"","sources":["../src/protocol.ts"],"names":[],"mappings":"AAAA;;;;;;;;;GASG;AA6GH;;;;;GAKG;AACH,MAAM,CAAC,MAAM,sBAAsB,GAAG;IACpC,YAAY;IACZ,YAAY;IACZ,UAAU;CACF,CAAC"}

package/dist/runtime.d.ts

@@ -0,0 +1,32 @@
+import type { SpeechEvent } from "@node-webrtc-rust/sdk/voice";
+export interface SessionContext {
+    sessionId: string;
+    env: Record<string, string>;
+}
+export interface SpeechContext {
+    sessionId: string;
+    text: string;
+}
+export interface SpeechEventContext {
+    sessionId: string;
+}
+export interface AgentHandlers {
+    onSessionStart?: (ctx: SessionContext) => void | Promise<void>;
+    /** Fired for every speech lifecycle event from the parent voice pipeline. */
+    onSpeechEvent?: (ctx: SpeechEventContext, event: SpeechEvent) => void | Promise<void>;
+    /** Convenience handler — also invoked when `speech.type` is `user_speech_final`. */
+    onUserSpeechFinal?: (ctx: SpeechContext) => void | Promise<void>;
+    onSessionEnd?: (ctx: {
+        sessionId: string;
+    }) => void | Promise<void>;
+}
+/**
+ * Register IPC handlers for a customer agent child process.
+ * Call once at bundle entry; runner parent sends {@link ParentToChildMessage} events.
+ */
+export declare function defineAgent(handlers: AgentHandlers): void;
+/** Ask the runner parent to synthesize speech for the session. */
+export declare function speak(sessionId: string, text: string): void;
+/** Structured log forwarded to the runner parent. */
+export declare function agentLog(level: "info" | "error", message: string): void;
+//# sourceMappingURL=runtime.d.ts.map

package/dist/runtime.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"runtime.d.ts","sourceRoot":"","sources":["../src/runtime.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,6BAA6B,CAAC;AAI/D,MAAM,WAAW,cAAc;IAC7B,SAAS,EAAE,MAAM,CAAC;IAClB,GAAG,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;CAC7B;AAED,MAAM,WAAW,aAAa;IAC5B,SAAS,EAAE,MAAM,CAAC;IAClB,IAAI,EAAE,MAAM,CAAC;CACd;AAED,MAAM,WAAW,kBAAkB;IACjC,SAAS,EAAE,MAAM,CAAC;CACnB;AAED,MAAM,WAAW,aAAa;IAC5B,cAAc,CAAC,EAAE,CAAC,GAAG,EAAE,cAAc,KAAK,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;IAC/D,6EAA6E;IAC7E,aAAa,CAAC,EAAE,CACd,GAAG,EAAE,kBAAkB,EACvB,KAAK,EAAE,WAAW,KACf,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;IAC1B,oFAAoF;IACpF,iBAAiB,CAAC,EAAE,CAAC,GAAG,EAAE,aAAa,KAAK,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;IACjE,YAAY,CAAC,EAAE,CAAC,GAAG,EAAE;QAAE,SAAS,EAAE,MAAM,CAAA;KAAE,KAAK,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;CACrE;AAYD;;;GAGG;AACH,wBAAgB,WAAW,CAAC,QAAQ,EAAE,aAAa,GAAG,IAAI,CA4CzD;AAED,kEAAkE;AAClE,wBAAgB,KAAK,CAAC,SAAS,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,GAAG,IAAI,CAE3D;AAED,qDAAqD;AACrD,wBAAgB,QAAQ,CAAC,KAAK,EAAE,MAAM,GAAG,OAAO,EAAE,OAAO,EAAE,MAAM,GAAG,IAAI,CAEvE"}

package/dist/runtime.js

@@ -0,0 +1,61 @@
+function isParentMessage(value) {
+    if (!value || typeof value !== "object")
+        return false;
+    const msg = value;
+    return (msg.type === "session_start" ||
+        msg.type === "speech_event" ||
+        msg.type === "session_end");
+}
+/**
+ * Register IPC handlers for a customer agent child process.
+ * Call once at bundle entry; runner parent sends {@link ParentToChildMessage} events.
+ */
+export function defineAgent(handlers) {
+    process.on("message", (message) => {
+        if (!isParentMessage(message))
+            return;
+        void (async () => {
+            try {
+                switch (message.type) {
+                    case "session_start":
+                        await handlers.onSessionStart?.({
+                            sessionId: message.sessionId,
+                            env: message.env,
+                        });
+                        break;
+                    case "speech_event":
+                        await handlers.onSpeechEvent?.({ sessionId: message.sessionId }, message.event);
+                        if (message.event.type === "user_speech_final" &&
+                            typeof message.event.text === "string" &&
+                            message.event.text.trim()) {
+                            await handlers.onUserSpeechFinal?.({
+                                sessionId: message.sessionId,
+                                text: message.event.text.trim(),
+                            });
+                        }
+                        break;
+                    case "session_end":
+                        await handlers.onSessionEnd?.({ sessionId: message.sessionId });
+                        break;
+                }
+            }
+            catch (error) {
+                const errMessage = error instanceof Error ? error.message : String(error);
+                process.send?.({
+                    type: "agent_error",
+                    sessionId: message.sessionId,
+                    message: errMessage,
+                });
+            }
+        })();
+    });
+}
+/** Ask the runner parent to synthesize speech for the session. */
+export function speak(sessionId, text) {
+    process.send?.({ type: "speak", sessionId, text });
+}
+/** Structured log forwarded to the runner parent. */
+export function agentLog(level, message) {
+    process.send?.({ type: "log", level, message });
+}
+//# sourceMappingURL=runtime.js.map

package/dist/runtime.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"runtime.js","sourceRoot":"","sources":["../src/runtime.ts"],"names":[],"mappings":"AA8BA,SAAS,eAAe,CAAC,KAAc;IACrC,IAAI,CAAC,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ;QAAE,OAAO,KAAK,CAAC;IACtD,MAAM,GAAG,GAAG,KAA0B,CAAC;IACvC,OAAO,CACL,GAAG,CAAC,IAAI,KAAK,eAAe;QAC5B,GAAG,CAAC,IAAI,KAAK,cAAc;QAC3B,GAAG,CAAC,IAAI,KAAK,aAAa,CAC3B,CAAC;AACJ,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,WAAW,CAAC,QAAuB;IACjD,OAAO,CAAC,EAAE,CAAC,SAAS,EAAE,CAAC,OAAgB,EAAE,EAAE;QACzC,IAAI,CAAC,eAAe,CAAC,OAAO,CAAC;YAAE,OAAO;QAEtC,KAAK,CAAC,KAAK,IAAI,EAAE;YACf,IAAI,CAAC;gBACH,QAAQ,OAAO,CAAC,IAAI,EAAE,CAAC;oBACrB,KAAK,eAAe;wBAClB,MAAM,QAAQ,CAAC,cAAc,EAAE,CAAC;4BAC9B,SAAS,EAAE,OAAO,CAAC,SAAS;4BAC5B,GAAG,EAAE,OAAO,CAAC,GAAG;yBACjB,CAAC,CAAC;wBACH,MAAM;oBACR,KAAK,cAAc;wBACjB,MAAM,QAAQ,CAAC,aAAa,EAAE,CAC5B,EAAE,SAAS,EAAE,OAAO,CAAC,SAAS,EAAE,EAChC,OAAO,CAAC,KAAK,CACd,CAAC;wBACF,IACE,OAAO,CAAC,KAAK,CAAC,IAAI,KAAK,mBAAmB;4BAC1C,OAAO,OAAO,CAAC,KAAK,CAAC,IAAI,KAAK,QAAQ;4BACtC,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,EAAE,EACzB,CAAC;4BACD,MAAM,QAAQ,CAAC,iBAAiB,EAAE,CAAC;gCACjC,SAAS,EAAE,OAAO,CAAC,SAAS;gCAC5B,IAAI,EAAE,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,EAAE;6BAChC,CAAC,CAAC;wBACL,CAAC;wBACD,MAAM;oBACR,KAAK,aAAa;wBAChB,MAAM,QAAQ,CAAC,YAAY,EAAE,CAAC,EAAE,SAAS,EAAE,OAAO,CAAC,SAAS,EAAE,CAAC,CAAC;wBAChE,MAAM;gBACV,CAAC;YACH,CAAC;YAAC,OAAO,KAAK,EAAE,CAAC;gBACf,MAAM,UAAU,GACd,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;gBACzD,OAAO,CAAC,IAAI,EAAE,CAAC;oBACb,IAAI,EAAE,aAAa;oBACnB,SAAS,EAAE,OAAO,CAAC,SAAS;oBAC5B,OAAO,EAAE,UAAU;iBACpB,CAAC,CAAC;YACL,CAAC;QACH,CAAC,CAAC,EAAE,CAAC;IACP,CAAC,CAAC,CAAC;AACL,CAAC;AAED,kEAAkE;AAClE,MAAM,UAAU,KAAK,CAAC,SAAiB,EAAE,IAAY;IACnD,OAAO,CAAC,IAAI,EAAE,CAAC,EAAE,IAAI,EAAE,OAAO,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;AACrD,CAAC;AAED,qDAAqD;AACrD,MAAM,UAAU,QAAQ,CAAC,KAAuB,EAAE,OAAe;IAC/D,OAAO,CAAC,IAAI,EAAE,CAAC,EAAE,IAAI,EAAE,KAAK,EAAE,KAAK,EAAE,OAAO,EAAE,CAAC,CAAC;AAClD,CAAC"}

package/package.json

@@ -0,0 +1,60 @@
+{
+  "name": "@voicethere/agent",
+  "version": "0.1.0",
+  "description": "VoiceThere customer agent SDK — IPC types and runtime helpers for sandboxed child bundles",
+  "type": "module",
+  "exports": {
+    ".": {
+      "types": "./dist/index.d.ts",
+      "import": "./dist/index.js"
+    }
+  },
+  "files": [
+    "dist",
+    "templates",
+    "README.md"
+  ],
+  "scripts": {
+    "build": "tsc -p tsconfig.json && esbuild examples/agent.ts --bundle --platform=node --format=esm --outfile=dist/agent.js",
+    "typecheck": "tsc --noEmit -p tsconfig.json && tsc --noEmit -p tsconfig.test.json",
+    "test": "vitest run",
+    "test:ci": "npm run typecheck && npm run build && npm run test",
+    "verify:local": "npm run build && tsx scripts/verify-local.ts",
+    "verify:local:only": "tsx scripts/verify-local.ts",
+    "prepublishOnly": "npm run build"
+  },
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/voicethere/agent.git"
+  },
+  "bugs": {
+    "url": "https://github.com/voicethere/agent/issues"
+  },
+  "homepage": "https://github.com/voicethere/agent#readme",
+  "publishConfig": {
+    "access": "public"
+  },
+  "keywords": [
+    "voicethere",
+    "voice-agent",
+    "webrtc",
+    "sandbox"
+  ],
+  "engines": {
+    "node": ">=22"
+  },
+  "license": "UNLICENSED",
+  "dependencies": {
+    "@node-webrtc-rust/sdk": "0.5.2"
+  },
+  "peerDependencies": {
+    "@node-webrtc-rust/sdk": ">=0.5.2"
+  },
+  "devDependencies": {
+    "@types/node": "^22.10.0",
+    "esbuild": "^0.25.12",
+    "tsx": "^4.19.2",
+    "typescript": "^5.7.0",
+    "vitest": "^2.1.8"
+  }
+}

package/templates/README.md

@@ -0,0 +1,26 @@
+# Agent templates
+
+## `agent.ts`
+
+Full starter bundle covering every speech event from `@node-webrtc-rust/sdk/voice`:
+
+| Group       | Events                                                                                       |
+| ----------- | -------------------------------------------------------------------------------------------- |
+| User VAD    | `user_speaking_start`, `user_speaking_end`, `vad_triggered`                                  |
+| STT stream  | `stt_stream_start`, `stt_stream_end`, `user_stt_start`, `user_stt_end`, `user_stt_not_found` |
+| Transcripts | `user_speech_partial`, `user_speech_final`                                                   |
+| Agent TTS   | `agent_speaking_start`, `agent_speaking_end`, `barge_in`                                     |
+| Failures    | `error`                                                                                      |
+
+**Customize:** replace `onUserSpeechFinal` body with your LLM/tools; extend `PeerState` or swap `handleSpeechEvent` for your architecture.
+
+**Build:**
+
+```bash
+npm install @voicethere/agent esbuild
+npx esbuild agent.ts --bundle --platform=node --format=esm --outfile=dist/agent.js
+```
+
+**Verify sandbox (no WebRTC):** from the agent repo, `npm run verify:local` after building your bundle.
+
+**Voice E2E:** host with [`voicethere/runner`](https://github.com/voicethere/runner) — set `AGENT_BUNDLE_PATH` to your built `dist/agent.js`.

package/templates/agent.ts

@@ -0,0 +1,173 @@
+/**
+ * Full starter template — copy into your project and build to a single `agent.js` bundle.
+ *
+ * Handles every `SpeechEvent` from the runner voice pipeline (`@node-webrtc-rust/sdk/voice`).
+ * Replace the stub bodies with your product logic; keep `onUserSpeechFinal` as the main turn hook.
+ *
+ * Build:
+ *   npm install @voicethere/agent
+ *   npx esbuild agent.ts --bundle --platform=node --format=esm --outfile=dist/agent.js
+ *
+ * Local runner (clone https://github.com/voicethere/runner):
+ *   AGENT_BUNDLE_PATH=./dist/agent.js npm run start
+ */
+
+import {
+  agentLog,
+  defineAgent,
+  speak,
+  type SpeechEvent,
+} from "@voicethere/agent";
+
+/** Per-peer conversational state — extend or replace with your store. */
+interface PeerState {
+  userSpeaking: boolean;
+  agentSpeaking: boolean;
+  lastPartial: string;
+  sttActive: boolean;
+}
+
+const peers = new Map<string, PeerState>();
+
+function peerState(sessionId: string): PeerState {
+  let state = peers.get(sessionId);
+  if (!state) {
+    state = {
+      userSpeaking: false,
+      agentSpeaking: false,
+      lastPartial: "",
+      sttActive: false,
+    };
+    peers.set(sessionId, state);
+  }
+  return state;
+}
+
+function formatSpeechDetail(speech: SpeechEvent): string {
+  if (speech.text) return `"${speech.text}"`;
+  if (speech.error) return speech.error;
+  return "";
+}
+
+/**
+ * Dispatch every speech lifecycle event from the parent Sherpa/VAD/STT/TTS pipeline.
+ * `speech.type` values match `SpeechEventType` in `@node-webrtc-rust/sdk/voice`.
+ */
+function handleSpeechEvent(sessionId: string, speech: SpeechEvent): void {
+  const state = peerState(sessionId);
+  const detail = formatSpeechDetail(speech);
+
+  switch (speech.type) {
+    // --- User turn (VAD + STT) ---
+    case "user_speaking_start":
+      state.userSpeaking = true;
+      agentLog("info", `[${sessionId}] user_speaking_start`);
+      break;
+
+    case "user_speaking_end":
+      state.userSpeaking = false;
+      agentLog("info", `[${sessionId}] user_speaking_end`);
+      break;
+
+    case "vad_triggered":
+      agentLog(
+        "info",
+        `[${sessionId}] vad_triggered — STT listen window opened`,
+      );
+      break;
+
+    case "stt_stream_start":
+      agentLog("info", `[${sessionId}] stt_stream_start`);
+      break;
+
+    case "stt_stream_end":
+      agentLog("info", `[${sessionId}] stt_stream_end`);
+      break;
+
+    case "user_stt_start":
+      state.sttActive = true;
+      state.lastPartial = "";
+      agentLog("info", `[${sessionId}] user_stt_start`);
+      break;
+
+    case "user_stt_end":
+      state.sttActive = false;
+      agentLog("info", `[${sessionId}] user_stt_end`);
+      break;
+
+    case "user_stt_not_found":
+      agentLog(
+        "info",
+        `[${sessionId}] user_stt_not_found — no speech recognized in listen window`,
+      );
+      // Optional: speak(sessionId, "I didn't catch that. Could you repeat?")
+      break;
+
+    case "user_speech_partial":
+      state.lastPartial = speech.text ?? "";
+      agentLog("info", `[${sessionId}] user_speech_partial ${detail}`);
+      // Optional: live captions, early intent detection, custom barge-in rules
+      break;
+
+    case "user_speech_final":
+      state.lastPartial = "";
+      agentLog("info", `[${sessionId}] user_speech_final ${detail}`);
+      // Primary turn boundary — `onUserSpeechFinal` runs after this for convenience
+      break;
+
+    // --- Agent playback (TTS) ---
+    case "agent_speaking_start":
+      state.agentSpeaking = true;
+      agentLog("info", `[${sessionId}] agent_speaking_start`);
+      break;
+
+    case "agent_speaking_end":
+      state.agentSpeaking = false;
+      agentLog("info", `[${sessionId}] agent_speaking_end`);
+      break;
+
+    case "barge_in":
+      agentLog(
+        "info",
+        `[${sessionId}] barge_in — user interrupted agent playback`,
+      );
+      // Optional: cancel in-flight LLM/TTS work keyed by sessionId
+      break;
+
+    case "error":
+      agentLog(
+        "error",
+        `[${sessionId}] pipeline error: ${speech.error ?? "unknown"}`,
+      );
+      break;
+
+    default: {
+      const _exhaustive: never = speech.type;
+      agentLog(
+        "error",
+        `[${sessionId}] unhandled speech event: ${String(_exhaustive)}`,
+      );
+    }
+  }
+}
+
+defineAgent({
+  onSessionStart({ sessionId, env }) {
+    agentLog("info", `session_start ${sessionId} env=${JSON.stringify(env)}`);
+    speak(sessionId, "Hello! How can I help?");
+  },
+
+  onSpeechEvent({ sessionId }, speech) {
+    handleSpeechEvent(sessionId, speech);
+  },
+
+  onUserSpeechFinal({ sessionId, text }) {
+    // Replace with LLM / tool calls / business logic
+    speak(sessionId, `You said: ${text}`);
+  },
+
+  onSessionEnd({ sessionId }) {
+    peers.delete(sessionId);
+    agentLog("info", `session_end ${sessionId}`);
+  },
+});