@ai-ecoverse/pi-audio 1.20.0 → 1.21.0

package/README.md

@@ -52,10 +52,12 @@ pi-audio does not apply that patch. Hosts such as slicc-agent record monkeypatch
 
 `@ai-ecoverse/pi-audio/gemini-live` registers `gemini-3.8-live`, `gemini-live-2.5-flash-native-audio`, and `gemini-3.1-flash-live-preview`. Setup enables sliding-window compression and session resumption. `gemini-3.8-live` leaves function behavior at the model default and accepts `vocabulary` as `customVocabulary`. The 2.5 model declares `NON_BLOCKING`. The 3.1 preview declares `BLOCKING`. `respond` maps `Delivery` onto `scheduling` (`INTERRUPT`, `WHEN_IDLE`, `SILENT`) and returns `status`, `retryable`, and `message`. A `goAway` reconnects with the latest resumable handle. There is no truncate call. The adapter takes a recorded `wire` and never calls Google.
 
-`mintRealtimeToken(model, constraints, options)` returns `{ value, expiresAt, model }` and calls `options.fetch` (or `globalThis.fetch`). OpenAI `POST /v1/realtime/client_secrets` sends a realtime session with instructions, voice, and tools. xAI `POST /v1/realtime/client_secrets` sends only `expires_after.seconds` of 300. Gemini `POST /v1beta/auth_tokens` locks `models/<id>`, instructions, voice, and tools in `liveConnectConstraints` and returns `name`, which the live adapter sends as `access_token`. ElevenLabs `GET /v1/convai/conversation/get-signed-url` takes `agent_id` (or `GET /conversation/token` when the only transport is WebRTC) and expires in 15 minutes. Fish `POST /v1/agent/sessions` sends `agent_id` plus `overrides.system_prompt` and `overrides.voice_id`. `value` is the LiveKit participant token. `expiresAt` is milliseconds. A numeric `expires_at` below `1e12` is unix seconds.
+`mintRealtimeToken(model, constraints, options)` returns `{ value, expiresAt, model, url? }` and calls `options.fetch` (or `globalThis.fetch`). OpenAI `POST /v1/realtime/client_secrets` sends a realtime session with instructions, voice, and tools. xAI `POST /v1/realtime/client_secrets` sends only `expires_after.seconds` of 300. Gemini `POST /v1beta/auth_tokens` locks `models/<id>`, instructions, voice, and tools in `liveConnectConstraints` and returns `name`, which the live adapter sends as `access_token`. ElevenLabs `GET /v1/convai/conversation/get-signed-url` takes `agent_id` (or `GET /conversation/token` when the only transport is WebRTC) and expires in 15 minutes. Fish `POST /v1/agent/sessions` sends `agent_id` plus `overrides.system_prompt` and `overrides.voice_id`. `value` is the LiveKit participant token and `url` is `livekit_url`. `expiresAt` is milliseconds. A numeric `expires_at` below `1e12` is unix seconds.
 
 `@ai-ecoverse/pi-audio/elevenlabs-agents` registers `elevenlabs-agent`, a hosted cascade (`provisioning: "agent"`, `truncation: "server"`, `liveToolUpdate: false`). `provisionAgent` creates or updates the agent and reuses it when the spec hash matches. pi tools become client tools. A string `llm` is a vendor model name. A `CustomLlm` sets `llm` to `custom-llm` and sends `custom_llm.url`, `model_id`, `api_type`, and `request_headers` (ElevenLabs `api_key` is a workspace secret reference, not a raw token). `connectRealtime` uses a `RealtimeToken` from `mintRealtimeToken` (`get-signed-url`) when one is passed. Otherwise an API key provisions the agent and mints the signed URL. Silent `inject` sends `contextual_update` with `context_id` `pi-audio`. `close()` prices `audio.inputSeconds` from the wall-clock session at $0.08 per minute.
 
+`@ai-ecoverse/pi-audio/fish-agents` registers `fish-agent` (`transports: ["webrtc"]`, `delivery.silent: "approximate"`, `delivery.interrupt: "native"`). `provisionAgent` creates client tools, writes the draft, and publishes it. A `RealtimeToken` with `url` is the LiveKit room from `mintRealtimeToken`; connect does not open a second session. Without a token, connect creates the session and fails on an unknown `transport`. Silent `inject` sends `user.message` with `audio: false` and drops that reply's transcript. The microphone is published after `openEchoMic`. The catalog `audioCost` is zero because Fish has not published a per-minute session rate. The example voice id is the one in Fish's quickstart.
+
 `talkerTools` is `ask_agent` and `read`. pi-audio defines `askAgent` (a pi `Tool` whose parameters are `request` and optional `context`) and the `Thinker` interface. The host implements `Thinker` and supplies `read`. `bindThinker` runs direct tools with `directTimeoutMs` and truncates their text to `directMaxChars`. On a blocking `ask_agent` it answers with `ack` immediately and injects the result later. On an async call it `respond`s with the result and `options.result`. Progress is injected with `options.progress`. `tool_call_cancelled` aborts the thinker and any direct tool. A blocking failure is injected after the acknowledgement, with a `thinker_result` caption. `Thinker.ask` receives the optional `context` string. Closing the session aborts work that is still running. The binding also yields `thinker_progress` / `thinker_result` captions and `thinking` / `tool` / `idle` phases.
 
 ## Development

package/dist/captions.js

@@ -59,7 +59,7 @@ function captionsFromHistory(entries) {
       type: "agent_generated",
       turnId,
       utteranceId,
-      text: blocks.join("")
+      text: joined(blocks)
     });
   }
   return captions;
@@ -71,7 +71,7 @@ function firstMessage(entry) {
 }
 function pushUser(captions, entry, message, voice) {
   const turnId = String(entry.id);
-  const text = textBlocks(message.content).join("");
+  const text = joined(textBlocks(message.content));
   if (voice.confidence === void 0) {
     captions.push({ type: "user_final", turnId, segmentId: "turn", text });
     return;
@@ -105,11 +105,10 @@ function noteCut(captions, agents, data) {
 }
 function heardInBlocks(blocks, contentIndex, heardUpTo) {
   let prior = 0;
-  let index = 0;
   for (const block of blocks) {
-    if (index === contentIndex) return prior + clampHeard(block, heardUpTo);
-    prior += block.length;
-    index += 1;
+    if (block.index === contentIndex) return prior + clampHeard(block.text, heardUpTo);
+    if (block.index > contentIndex) return prior;
+    prior += block.text.length;
   }
   return prior;
 }
@@ -118,12 +117,21 @@ function clampHeard(block, heardUpTo) {
   if (heardUpTo >= block.length) return block.length;
   return heardUpTo;
 }
+function joined(blocks) {
+  let text = "";
+  for (const block of blocks) text += block.text;
+  return text;
+}
 function textBlocks(content) {
-  if (typeof content === "string") return [content];
+  if (typeof content === "string") return [{ index: 0, text: content }];
   if (!content) return [];
   const blocks = [];
+  let index = 0;
   for (const block of content) {
-    if (block.type === "text" && typeof block.text === "string") blocks.push(block.text);
+    if (block.type === "text" && typeof block.text === "string") {
+      blocks.push({ index, text: block.text });
+    }
+    index += 1;
   }
   return blocks;
 }

package/dist/captions.js.map

@@ -1,7 +1,7 @@
 {
   "version": 3,
   "sources": ["../src/captions.ts"],
-  "sourcesContent": ["import type { CharAlignment } from './audio.ts';\nimport type { CaptionEvent } from './turns.ts';\n\nexport interface SpokenClock {\n  start: number;\n  end: number;\n  durationMs: number;\n  alignment?: CharAlignment;\n}\n\nexport interface UtteranceMap {\n  spokenStart: number;\n  spokenEnd: number;\n  kind: string;\n}\n\nexport interface CaptionHistoryEntry {\n  id: string | number;\n  kind: string;\n  model?: readonly HistoryMessage[];\n  data?: unknown;\n}\n\ninterface HistoryMessage {\n  role?: string;\n  content?: string | readonly HistoryBlock[];\n}\n\ninterface HistoryBlock {\n  type?: string;\n  text?: string;\n}\n\ninterface StoredAgent {\n  turnId: string;\n  utteranceId: string;\n  blocks: string[];\n}\n\nexport function spokenOffset(clock: SpokenClock, offsetMs: number): number {\n  const length = clock.end - clock.start;\n  if (length <= 0) return clock.start;\n  const alignment = clock.alignment;\n  if (alignment && alignment.startMs.length === length && alignment.durationMs.length === length) {\n    return clock.start + alignedHeard(alignment, offsetMs, length);\n  }\n  if (clock.durationMs <= 0) {\n    if (offsetMs <= 0) return clock.start;\n    return clock.end;\n  }\n  const ratio = offsetMs <= 0 ? 0 : offsetMs / clock.durationMs;\n  const fraction = ratio > 1 ? 1 : ratio;\n  return clock.start + Math.round(fraction * length);\n}\n\nfunction alignedHeard(alignment: CharAlignment, offsetMs: number, length: number): number {\n  const first = alignment.startMs[0];\n  let origin = 0;\n  if (first !== undefined) origin = first;\n  const at = origin + offsetMs;\n  let heard = 0;\n  for (let index = 0; index < length; index += 1) {\n    const start = alignment.startMs[index];\n    if (start !== undefined && start <= at) heard = index + 1;\n  }\n  return heard;\n}\n\nexport function utteranceIndex(feedPos: number, maps: readonly UtteranceMap[]): number {\n  let upTo = 0;\n  for (const map of maps) {\n    if (map.kind !== 'answer') continue;\n    if (feedPos <= map.spokenStart) break;\n    const end = feedPos < map.spokenEnd ? feedPos : map.spokenEnd;\n    upTo += end - map.spokenStart;\n  }\n  return upTo;\n}\n\nexport function captionsFromHistory(entries: readonly CaptionHistoryEntry[]): CaptionEvent[] {\n  const captions: CaptionEvent[] = [];\n  const agents = new Map<string, StoredAgent>();\n  for (const entry of entries) {\n    if (entry.kind === 'voice.truncate') {\n      noteCut(captions, agents, entry.data);\n      continue;\n    }\n    const message = firstMessage(entry);\n    if (!message) continue;\n    const voice = voiceMeta(entry.data);\n    if (message.role === 'user' && voice) {\n      pushUser(captions, entry, message, voice);\n      continue;\n    }\n    if (message.role !== 'assistant') continue;\n    const blocks = textBlocks(message.content);\n    const turnId = String(entry.id);\n    const utteranceId = `${turnId}:agent`;\n    agents.set(turnId, { turnId, utteranceId, blocks });\n    captions.push({\n      type: 'agent_generated',\n      turnId,\n      utteranceId,\n      text: blocks.join(''),\n    });\n  }\n  return captions;\n}\n\nfunction firstMessage(entry: CaptionHistoryEntry): HistoryMessage | undefined {\n  const model = entry.model;\n  if (!model) return undefined;\n  return model[0];\n}\n\nfunction pushUser(\n  captions: CaptionEvent[],\n  entry: CaptionHistoryEntry,\n  message: HistoryMessage,\n  voice: { confidence?: number }\n): void {\n  const turnId = String(entry.id);\n  const text = textBlocks(message.content).join('');\n  if (voice.confidence === undefined) {\n    captions.push({ type: 'user_final', turnId, segmentId: 'turn', text });\n    return;\n  }\n  captions.push({\n    type: 'user_final',\n    turnId,\n    segmentId: 'turn',\n    text,\n    confidence: voice.confidence,\n  });\n}\n\nfunction noteCut(captions: CaptionEvent[], agents: Map<string, StoredAgent>, data: unknown): void {\n  const cut = cutData(data);\n  if (!cut) return;\n  const agent = agents.get(cut.entryId);\n  if (!agent) return;\n  const heard = heardInBlocks(agent.blocks, cut.contentIndex, cut.heardUpTo);\n  captions.push({\n    type: 'agent_spoken',\n    turnId: agent.turnId,\n    utteranceId: agent.utteranceId,\n    upTo: heard,\n  });\n  captions.push({\n    type: 'agent_cut',\n    turnId: agent.turnId,\n    utteranceId: agent.utteranceId,\n    heardUpTo: heard,\n  });\n}\n\nfunction heardInBlocks(blocks: readonly string[], contentIndex: number, heardUpTo: number): number {\n  let prior = 0;\n  let index = 0;\n  for (const block of blocks) {\n    if (index === contentIndex) return prior + clampHeard(block, heardUpTo);\n    prior += block.length;\n    index += 1;\n  }\n  return prior;\n}\n\nfunction clampHeard(block: string, heardUpTo: number): number {\n  if (heardUpTo <= 0) return 0;\n  if (heardUpTo >= block.length) return block.length;\n  return heardUpTo;\n}\n\nfunction textBlocks(content: HistoryMessage['content']): string[] {\n  if (typeof content === 'string') return [content];\n  if (!content) return [];\n  const blocks: string[] = [];\n  for (const block of content) {\n    if (block.type === 'text' && typeof block.text === 'string') blocks.push(block.text);\n  }\n  return blocks;\n}\n\nfunction voiceMeta(data: unknown): { confidence?: number } | undefined {\n  if (typeof data !== 'object' || data === null) return undefined;\n  if (!('source' in data) || data.source !== 'voice') return undefined;\n  if ('confidence' in data && typeof data.confidence === 'number') {\n    return { confidence: data.confidence };\n  }\n  return {};\n}\n\nfunction cutData(\n  data: unknown\n): { entryId: string; contentIndex: number; heardUpTo: number } | undefined {\n  if (typeof data !== 'object' || data === null) return undefined;\n  if (!('entryId' in data) || typeof data.entryId !== 'string') return undefined;\n  if (!('contentIndex' in data) || typeof data.contentIndex !== 'number') return undefined;\n  if (!('heardUpTo' in data) || typeof data.heardUpTo !== 'number') return undefined;\n  return { entryId: data.entryId, contentIndex: data.contentIndex, heardUpTo: data.heardUpTo };\n}\n"],
-  "mappings": "AAuCO,SAAS,aAAa,OAAoB,UAA0B;AACzE,QAAM,SAAS,MAAM,MAAM,MAAM;AACjC,MAAI,UAAU,EAAG,QAAO,MAAM;AAC9B,QAAM,YAAY,MAAM;AACxB,MAAI,aAAa,UAAU,QAAQ,WAAW,UAAU,UAAU,WAAW,WAAW,QAAQ;AAC9F,WAAO,MAAM,QAAQ,aAAa,WAAW,UAAU,MAAM;AAAA,EAC/D;AACA,MAAI,MAAM,cAAc,GAAG;AACzB,QAAI,YAAY,EAAG,QAAO,MAAM;AAChC,WAAO,MAAM;AAAA,EACf;AACA,QAAM,QAAQ,YAAY,IAAI,IAAI,WAAW,MAAM;AACnD,QAAM,WAAW,QAAQ,IAAI,IAAI;AACjC,SAAO,MAAM,QAAQ,KAAK,MAAM,WAAW,MAAM;AACnD;AAEA,SAAS,aAAa,WAA0B,UAAkB,QAAwB;AACxF,QAAM,QAAQ,UAAU,QAAQ,CAAC;AACjC,MAAI,SAAS;AACb,MAAI,UAAU,OAAW,UAAS;AAClC,QAAM,KAAK,SAAS;AACpB,MAAI,QAAQ;AACZ,WAAS,QAAQ,GAAG,QAAQ,QAAQ,SAAS,GAAG;AAC9C,UAAM,QAAQ,UAAU,QAAQ,KAAK;AACrC,QAAI,UAAU,UAAa,SAAS,GAAI,SAAQ,QAAQ;AAAA,EAC1D;AACA,SAAO;AACT;AAEO,SAAS,eAAe,SAAiB,MAAuC;AACrF,MAAI,OAAO;AACX,aAAW,OAAO,MAAM;AACtB,QAAI,IAAI,SAAS,SAAU;AAC3B,QAAI,WAAW,IAAI,YAAa;AAChC,UAAM,MAAM,UAAU,IAAI,YAAY,UAAU,IAAI;AACpD,YAAQ,MAAM,IAAI;AAAA,EACpB;AACA,SAAO;AACT;AAEO,SAAS,oBAAoB,SAAyD;AAC3F,QAAM,WAA2B,CAAC;AAClC,QAAM,SAAS,oBAAI,IAAyB;AAC5C,aAAW,SAAS,SAAS;AAC3B,QAAI,MAAM,SAAS,kBAAkB;AACnC,cAAQ,UAAU,QAAQ,MAAM,IAAI;AACpC;AAAA,IACF;AACA,UAAM,UAAU,aAAa,KAAK;AAClC,QAAI,CAAC,QAAS;AACd,UAAM,QAAQ,UAAU,MAAM,IAAI;AAClC,QAAI,QAAQ,SAAS,UAAU,OAAO;AACpC,eAAS,UAAU,OAAO,SAAS,KAAK;AACxC;AAAA,IACF;AACA,QAAI,QAAQ,SAAS,YAAa;AAClC,UAAM,SAAS,WAAW,QAAQ,OAAO;AACzC,UAAM,SAAS,OAAO,MAAM,EAAE;AAC9B,UAAM,cAAc,GAAG,MAAM;AAC7B,WAAO,IAAI,QAAQ,EAAE,QAAQ,aAAa,OAAO,CAAC;AAClD,aAAS,KAAK;AAAA,MACZ,MAAM;AAAA,MACN;AAAA,MACA;AAAA,MACA,MAAM,OAAO,KAAK,EAAE;AAAA,IACtB,CAAC;AAAA,EACH;AACA,SAAO;AACT;AAEA,SAAS,aAAa,OAAwD;AAC5E,QAAM,QAAQ,MAAM;AACpB,MAAI,CAAC,MAAO,QAAO;AACnB,SAAO,MAAM,CAAC;AAChB;AAEA,SAAS,SACP,UACA,OACA,SACA,OACM;AACN,QAAM,SAAS,OAAO,MAAM,EAAE;AAC9B,QAAM,OAAO,WAAW,QAAQ,OAAO,EAAE,KAAK,EAAE;AAChD,MAAI,MAAM,eAAe,QAAW;AAClC,aAAS,KAAK,EAAE,MAAM,cAAc,QAAQ,WAAW,QAAQ,KAAK,CAAC;AACrE;AAAA,EACF;AACA,WAAS,KAAK;AAAA,IACZ,MAAM;AAAA,IACN;AAAA,IACA,WAAW;AAAA,IACX;AAAA,IACA,YAAY,MAAM;AAAA,EACpB,CAAC;AACH;AAEA,SAAS,QAAQ,UAA0B,QAAkC,MAAqB;AAChG,QAAM,MAAM,QAAQ,IAAI;AACxB,MAAI,CAAC,IAAK;AACV,QAAM,QAAQ,OAAO,IAAI,IAAI,OAAO;AACpC,MAAI,CAAC,MAAO;AACZ,QAAM,QAAQ,cAAc,MAAM,QAAQ,IAAI,cAAc,IAAI,SAAS;AACzE,WAAS,KAAK;AAAA,IACZ,MAAM;AAAA,IACN,QAAQ,MAAM;AAAA,IACd,aAAa,MAAM;AAAA,IACnB,MAAM;AAAA,EACR,CAAC;AACD,WAAS,KAAK;AAAA,IACZ,MAAM;AAAA,IACN,QAAQ,MAAM;AAAA,IACd,aAAa,MAAM;AAAA,IACnB,WAAW;AAAA,EACb,CAAC;AACH;AAEA,SAAS,cAAc,QAA2B,cAAsB,WAA2B;AACjG,MAAI,QAAQ;AACZ,MAAI,QAAQ;AACZ,aAAW,SAAS,QAAQ;AAC1B,QAAI,UAAU,aAAc,QAAO,QAAQ,WAAW,OAAO,SAAS;AACtE,aAAS,MAAM;AACf,aAAS;AAAA,EACX;AACA,SAAO;AACT;AAEA,SAAS,WAAW,OAAe,WAA2B;AAC5D,MAAI,aAAa,EAAG,QAAO;AAC3B,MAAI,aAAa,MAAM,OAAQ,QAAO,MAAM;AAC5C,SAAO;AACT;AAEA,SAAS,WAAW,SAA8C;AAChE,MAAI,OAAO,YAAY,SAAU,QAAO,CAAC,OAAO;AAChD,MAAI,CAAC,QAAS,QAAO,CAAC;AACtB,QAAM,SAAmB,CAAC;AAC1B,aAAW,SAAS,SAAS;AAC3B,QAAI,MAAM,SAAS,UAAU,OAAO,MAAM,SAAS,SAAU,QAAO,KAAK,MAAM,IAAI;AAAA,EACrF;AACA,SAAO;AACT;AAEA,SAAS,UAAU,MAAoD;AACrE,MAAI,OAAO,SAAS,YAAY,SAAS,KAAM,QAAO;AACtD,MAAI,EAAE,YAAY,SAAS,KAAK,WAAW,QAAS,QAAO;AAC3D,MAAI,gBAAgB,QAAQ,OAAO,KAAK,eAAe,UAAU;AAC/D,WAAO,EAAE,YAAY,KAAK,WAAW;AAAA,EACvC;AACA,SAAO,CAAC;AACV;AAEA,SAAS,QACP,MAC0E;AAC1E,MAAI,OAAO,SAAS,YAAY,SAAS,KAAM,QAAO;AACtD,MAAI,EAAE,aAAa,SAAS,OAAO,KAAK,YAAY,SAAU,QAAO;AACrE,MAAI,EAAE,kBAAkB,SAAS,OAAO,KAAK,iBAAiB,SAAU,QAAO;AAC/E,MAAI,EAAE,eAAe,SAAS,OAAO,KAAK,cAAc,SAAU,QAAO;AACzE,SAAO,EAAE,SAAS,KAAK,SAAS,cAAc,KAAK,cAAc,WAAW,KAAK,UAAU;AAC7F;",
+  "sourcesContent": ["import type { CharAlignment } from './audio.ts';\nimport type { CaptionEvent } from './turns.ts';\n\nexport interface SpokenClock {\n  start: number;\n  end: number;\n  durationMs: number;\n  alignment?: CharAlignment;\n}\n\nexport interface UtteranceMap {\n  spokenStart: number;\n  spokenEnd: number;\n  kind: string;\n}\n\nexport interface CaptionHistoryEntry {\n  id: string | number;\n  kind: string;\n  model?: readonly HistoryMessage[];\n  data?: unknown;\n}\n\ninterface HistoryMessage {\n  role?: string;\n  content?: string | readonly HistoryBlock[];\n}\n\ninterface HistoryBlock {\n  type?: string;\n  text?: string;\n}\n\ninterface IndexedBlock {\n  index: number;\n  text: string;\n}\n\ninterface StoredAgent {\n  turnId: string;\n  utteranceId: string;\n  blocks: IndexedBlock[];\n}\n\nexport function spokenOffset(clock: SpokenClock, offsetMs: number): number {\n  const length = clock.end - clock.start;\n  if (length <= 0) return clock.start;\n  const alignment = clock.alignment;\n  if (alignment && alignment.startMs.length === length && alignment.durationMs.length === length) {\n    return clock.start + alignedHeard(alignment, offsetMs, length);\n  }\n  if (clock.durationMs <= 0) {\n    if (offsetMs <= 0) return clock.start;\n    return clock.end;\n  }\n  const ratio = offsetMs <= 0 ? 0 : offsetMs / clock.durationMs;\n  const fraction = ratio > 1 ? 1 : ratio;\n  return clock.start + Math.round(fraction * length);\n}\n\nfunction alignedHeard(alignment: CharAlignment, offsetMs: number, length: number): number {\n  const first = alignment.startMs[0];\n  let origin = 0;\n  if (first !== undefined) origin = first;\n  const at = origin + offsetMs;\n  let heard = 0;\n  for (let index = 0; index < length; index += 1) {\n    const start = alignment.startMs[index];\n    if (start !== undefined && start <= at) heard = index + 1;\n  }\n  return heard;\n}\n\nexport function utteranceIndex(feedPos: number, maps: readonly UtteranceMap[]): number {\n  let upTo = 0;\n  for (const map of maps) {\n    if (map.kind !== 'answer') continue;\n    if (feedPos <= map.spokenStart) break;\n    const end = feedPos < map.spokenEnd ? feedPos : map.spokenEnd;\n    upTo += end - map.spokenStart;\n  }\n  return upTo;\n}\n\nexport function captionsFromHistory(entries: readonly CaptionHistoryEntry[]): CaptionEvent[] {\n  const captions: CaptionEvent[] = [];\n  const agents = new Map<string, StoredAgent>();\n  for (const entry of entries) {\n    if (entry.kind === 'voice.truncate') {\n      noteCut(captions, agents, entry.data);\n      continue;\n    }\n    const message = firstMessage(entry);\n    if (!message) continue;\n    const voice = voiceMeta(entry.data);\n    if (message.role === 'user' && voice) {\n      pushUser(captions, entry, message, voice);\n      continue;\n    }\n    if (message.role !== 'assistant') continue;\n    const blocks = textBlocks(message.content);\n    const turnId = String(entry.id);\n    const utteranceId = `${turnId}:agent`;\n    agents.set(turnId, { turnId, utteranceId, blocks });\n    captions.push({\n      type: 'agent_generated',\n      turnId,\n      utteranceId,\n      text: joined(blocks),\n    });\n  }\n  return captions;\n}\n\nfunction firstMessage(entry: CaptionHistoryEntry): HistoryMessage | undefined {\n  const model = entry.model;\n  if (!model) return undefined;\n  return model[0];\n}\n\nfunction pushUser(\n  captions: CaptionEvent[],\n  entry: CaptionHistoryEntry,\n  message: HistoryMessage,\n  voice: { confidence?: number }\n): void {\n  const turnId = String(entry.id);\n  const text = joined(textBlocks(message.content));\n  if (voice.confidence === undefined) {\n    captions.push({ type: 'user_final', turnId, segmentId: 'turn', text });\n    return;\n  }\n  captions.push({\n    type: 'user_final',\n    turnId,\n    segmentId: 'turn',\n    text,\n    confidence: voice.confidence,\n  });\n}\n\nfunction noteCut(captions: CaptionEvent[], agents: Map<string, StoredAgent>, data: unknown): void {\n  const cut = cutData(data);\n  if (!cut) return;\n  const agent = agents.get(cut.entryId);\n  if (!agent) return;\n  const heard = heardInBlocks(agent.blocks, cut.contentIndex, cut.heardUpTo);\n  captions.push({\n    type: 'agent_spoken',\n    turnId: agent.turnId,\n    utteranceId: agent.utteranceId,\n    upTo: heard,\n  });\n  captions.push({\n    type: 'agent_cut',\n    turnId: agent.turnId,\n    utteranceId: agent.utteranceId,\n    heardUpTo: heard,\n  });\n}\n\nfunction heardInBlocks(\n  blocks: readonly IndexedBlock[],\n  contentIndex: number,\n  heardUpTo: number\n): number {\n  let prior = 0;\n  for (const block of blocks) {\n    if (block.index === contentIndex) return prior + clampHeard(block.text, heardUpTo);\n    if (block.index > contentIndex) return prior;\n    prior += block.text.length;\n  }\n  return prior;\n}\n\nfunction clampHeard(block: string, heardUpTo: number): number {\n  if (heardUpTo <= 0) return 0;\n  if (heardUpTo >= block.length) return block.length;\n  return heardUpTo;\n}\n\nfunction joined(blocks: readonly IndexedBlock[]): string {\n  let text = '';\n  for (const block of blocks) text += block.text;\n  return text;\n}\n\nfunction textBlocks(content: HistoryMessage['content']): IndexedBlock[] {\n  if (typeof content === 'string') return [{ index: 0, text: content }];\n  if (!content) return [];\n  const blocks: IndexedBlock[] = [];\n  let index = 0;\n  for (const block of content) {\n    if (block.type === 'text' && typeof block.text === 'string') {\n      blocks.push({ index, text: block.text });\n    }\n    index += 1;\n  }\n  return blocks;\n}\n\nfunction voiceMeta(data: unknown): { confidence?: number } | undefined {\n  if (typeof data !== 'object' || data === null) return undefined;\n  if (!('source' in data) || data.source !== 'voice') return undefined;\n  if ('confidence' in data && typeof data.confidence === 'number') {\n    return { confidence: data.confidence };\n  }\n  return {};\n}\n\nfunction cutData(\n  data: unknown\n): { entryId: string; contentIndex: number; heardUpTo: number } | undefined {\n  if (typeof data !== 'object' || data === null) return undefined;\n  if (!('entryId' in data) || typeof data.entryId !== 'string') return undefined;\n  if (!('contentIndex' in data) || typeof data.contentIndex !== 'number') return undefined;\n  if (!('heardUpTo' in data) || typeof data.heardUpTo !== 'number') return undefined;\n  return { entryId: data.entryId, contentIndex: data.contentIndex, heardUpTo: data.heardUpTo };\n}\n"],
+  "mappings": "AA4CO,SAAS,aAAa,OAAoB,UAA0B;AACzE,QAAM,SAAS,MAAM,MAAM,MAAM;AACjC,MAAI,UAAU,EAAG,QAAO,MAAM;AAC9B,QAAM,YAAY,MAAM;AACxB,MAAI,aAAa,UAAU,QAAQ,WAAW,UAAU,UAAU,WAAW,WAAW,QAAQ;AAC9F,WAAO,MAAM,QAAQ,aAAa,WAAW,UAAU,MAAM;AAAA,EAC/D;AACA,MAAI,MAAM,cAAc,GAAG;AACzB,QAAI,YAAY,EAAG,QAAO,MAAM;AAChC,WAAO,MAAM;AAAA,EACf;AACA,QAAM,QAAQ,YAAY,IAAI,IAAI,WAAW,MAAM;AACnD,QAAM,WAAW,QAAQ,IAAI,IAAI;AACjC,SAAO,MAAM,QAAQ,KAAK,MAAM,WAAW,MAAM;AACnD;AAEA,SAAS,aAAa,WAA0B,UAAkB,QAAwB;AACxF,QAAM,QAAQ,UAAU,QAAQ,CAAC;AACjC,MAAI,SAAS;AACb,MAAI,UAAU,OAAW,UAAS;AAClC,QAAM,KAAK,SAAS;AACpB,MAAI,QAAQ;AACZ,WAAS,QAAQ,GAAG,QAAQ,QAAQ,SAAS,GAAG;AAC9C,UAAM,QAAQ,UAAU,QAAQ,KAAK;AACrC,QAAI,UAAU,UAAa,SAAS,GAAI,SAAQ,QAAQ;AAAA,EAC1D;AACA,SAAO;AACT;AAEO,SAAS,eAAe,SAAiB,MAAuC;AACrF,MAAI,OAAO;AACX,aAAW,OAAO,MAAM;AACtB,QAAI,IAAI,SAAS,SAAU;AAC3B,QAAI,WAAW,IAAI,YAAa;AAChC,UAAM,MAAM,UAAU,IAAI,YAAY,UAAU,IAAI;AACpD,YAAQ,MAAM,IAAI;AAAA,EACpB;AACA,SAAO;AACT;AAEO,SAAS,oBAAoB,SAAyD;AAC3F,QAAM,WAA2B,CAAC;AAClC,QAAM,SAAS,oBAAI,IAAyB;AAC5C,aAAW,SAAS,SAAS;AAC3B,QAAI,MAAM,SAAS,kBAAkB;AACnC,cAAQ,UAAU,QAAQ,MAAM,IAAI;AACpC;AAAA,IACF;AACA,UAAM,UAAU,aAAa,KAAK;AAClC,QAAI,CAAC,QAAS;AACd,UAAM,QAAQ,UAAU,MAAM,IAAI;AAClC,QAAI,QAAQ,SAAS,UAAU,OAAO;AACpC,eAAS,UAAU,OAAO,SAAS,KAAK;AACxC;AAAA,IACF;AACA,QAAI,QAAQ,SAAS,YAAa;AAClC,UAAM,SAAS,WAAW,QAAQ,OAAO;AACzC,UAAM,SAAS,OAAO,MAAM,EAAE;AAC9B,UAAM,cAAc,GAAG,MAAM;AAC7B,WAAO,IAAI,QAAQ,EAAE,QAAQ,aAAa,OAAO,CAAC;AAClD,aAAS,KAAK;AAAA,MACZ,MAAM;AAAA,MACN;AAAA,MACA;AAAA,MACA,MAAM,OAAO,MAAM;AAAA,IACrB,CAAC;AAAA,EACH;AACA,SAAO;AACT;AAEA,SAAS,aAAa,OAAwD;AAC5E,QAAM,QAAQ,MAAM;AACpB,MAAI,CAAC,MAAO,QAAO;AACnB,SAAO,MAAM,CAAC;AAChB;AAEA,SAAS,SACP,UACA,OACA,SACA,OACM;AACN,QAAM,SAAS,OAAO,MAAM,EAAE;AAC9B,QAAM,OAAO,OAAO,WAAW,QAAQ,OAAO,CAAC;AAC/C,MAAI,MAAM,eAAe,QAAW;AAClC,aAAS,KAAK,EAAE,MAAM,cAAc,QAAQ,WAAW,QAAQ,KAAK,CAAC;AACrE;AAAA,EACF;AACA,WAAS,KAAK;AAAA,IACZ,MAAM;AAAA,IACN;AAAA,IACA,WAAW;AAAA,IACX;AAAA,IACA,YAAY,MAAM;AAAA,EACpB,CAAC;AACH;AAEA,SAAS,QAAQ,UAA0B,QAAkC,MAAqB;AAChG,QAAM,MAAM,QAAQ,IAAI;AACxB,MAAI,CAAC,IAAK;AACV,QAAM,QAAQ,OAAO,IAAI,IAAI,OAAO;AACpC,MAAI,CAAC,MAAO;AACZ,QAAM,QAAQ,cAAc,MAAM,QAAQ,IAAI,cAAc,IAAI,SAAS;AACzE,WAAS,KAAK;AAAA,IACZ,MAAM;AAAA,IACN,QAAQ,MAAM;AAAA,IACd,aAAa,MAAM;AAAA,IACnB,MAAM;AAAA,EACR,CAAC;AACD,WAAS,KAAK;AAAA,IACZ,MAAM;AAAA,IACN,QAAQ,MAAM;AAAA,IACd,aAAa,MAAM;AAAA,IACnB,WAAW;AAAA,EACb,CAAC;AACH;AAEA,SAAS,cACP,QACA,cACA,WACQ;AACR,MAAI,QAAQ;AACZ,aAAW,SAAS,QAAQ;AAC1B,QAAI,MAAM,UAAU,aAAc,QAAO,QAAQ,WAAW,MAAM,MAAM,SAAS;AACjF,QAAI,MAAM,QAAQ,aAAc,QAAO;AACvC,aAAS,MAAM,KAAK;AAAA,EACtB;AACA,SAAO;AACT;AAEA,SAAS,WAAW,OAAe,WAA2B;AAC5D,MAAI,aAAa,EAAG,QAAO;AAC3B,MAAI,aAAa,MAAM,OAAQ,QAAO,MAAM;AAC5C,SAAO;AACT;AAEA,SAAS,OAAO,QAAyC;AACvD,MAAI,OAAO;AACX,aAAW,SAAS,OAAQ,SAAQ,MAAM;AAC1C,SAAO;AACT;AAEA,SAAS,WAAW,SAAoD;AACtE,MAAI,OAAO,YAAY,SAAU,QAAO,CAAC,EAAE,OAAO,GAAG,MAAM,QAAQ,CAAC;AACpE,MAAI,CAAC,QAAS,QAAO,CAAC;AACtB,QAAM,SAAyB,CAAC;AAChC,MAAI,QAAQ;AACZ,aAAW,SAAS,SAAS;AAC3B,QAAI,MAAM,SAAS,UAAU,OAAO,MAAM,SAAS,UAAU;AAC3D,aAAO,KAAK,EAAE,OAAO,MAAM,MAAM,KAAK,CAAC;AAAA,IACzC;AACA,aAAS;AAAA,EACX;AACA,SAAO;AACT;AAEA,SAAS,UAAU,MAAoD;AACrE,MAAI,OAAO,SAAS,YAAY,SAAS,KAAM,QAAO;AACtD,MAAI,EAAE,YAAY,SAAS,KAAK,WAAW,QAAS,QAAO;AAC3D,MAAI,gBAAgB,QAAQ,OAAO,KAAK,eAAe,UAAU;AAC/D,WAAO,EAAE,YAAY,KAAK,WAAW;AAAA,EACvC;AACA,SAAO,CAAC;AACV;AAEA,SAAS,QACP,MAC0E;AAC1E,MAAI,OAAO,SAAS,YAAY,SAAS,KAAM,QAAO;AACtD,MAAI,EAAE,aAAa,SAAS,OAAO,KAAK,YAAY,SAAU,QAAO;AACrE,MAAI,EAAE,kBAAkB,SAAS,OAAO,KAAK,iBAAiB,SAAU,QAAO;AAC/E,MAAI,EAAE,eAAe,SAAS,OAAO,KAAK,cAAc,SAAU,QAAO;AACzE,SAAO,EAAE,SAAS,KAAK,SAAS,cAAc,KAAK,cAAc,WAAW,KAAK,UAAU;AAC7F;",
   "names": []
 }

package/dist/durable/index.js

@@ -11,7 +11,9 @@ function keptBlock(block, index, cut) {
   const heard = block.text.slice(0, cut.heardUpTo);
   if (heard.length === 0) return void 0;
   if (heard === block.text) return block;
-  return { ...block, text: heard };
+  const shortened = { ...block, text: heard };
+  delete shortened.textSignature;
+  return shortened;
 }
 function cutAssistantMessage(message, cut) {
   const content = [];

package/dist/durable/index.js.map

@@ -1,7 +1,7 @@
 {
   "version": 3,
   "sources": ["../../src/durable/index.ts"],
-  "sourcesContent": ["import type { AssistantMessage } from '@earendil-works/pi-ai';\nimport type { ConversationId, EntryId, EntryRecord, Tx } from '@earendil-works/pi-durable';\nimport type { VoiceCut } from '../turns.ts';\n\ntype Block = AssistantMessage['content'][number];\n\ninterface TruncateSession<C> {\n  commit<T>(change: (tx: Tx) => T | Promise<T>, context: C): Promise<T>;\n}\n\nfunction entryIdOf(entryId: string): EntryId {\n  if (!/^\\d+$/.test(entryId)) {\n    throw new Error(`Voice cut entry is not a durable id: ${entryId}`);\n  }\n  return Number(entryId) as EntryId;\n}\n\nfunction keptBlock(block: Block, index: number, cut: VoiceCut): Block | undefined {\n  if (block.type === 'toolCall' || block.type === 'thinking') return block;\n  if (index < cut.contentIndex) return block;\n  if (index > cut.contentIndex) return undefined;\n  const heard = block.text.slice(0, cut.heardUpTo);\n  if (heard.length === 0) return undefined;\n  if (heard === block.text) return block;\n  return { ...block, text: heard };\n}\n\nfunction cutAssistantMessage(message: AssistantMessage, cut: VoiceCut): AssistantMessage {\n  const content: Block[] = [];\n  for (const [index, block] of message.content.entries()) {\n    const kept = keptBlock(block, index, cut);\n    if (kept === undefined) continue;\n    content.push(kept);\n  }\n  return { ...message, content };\n}\n\nexport async function appendVoiceTruncate<C>(\n  session: TruncateSession<C>,\n  conversationId: ConversationId,\n  cut: VoiceCut,\n  message: AssistantMessage,\n  context: C\n): Promise<EntryRecord> {\n  const replacement = cutAssistantMessage(message, cut);\n  const target = entryIdOf(cut.entryId);\n  return session.commit(\n    (tx) =>\n      tx.appendEntry(conversationId, {\n        kind: 'voice.truncate',\n        data: {\n          entryId: cut.entryId,\n          contentIndex: cut.contentIndex,\n          heardUpTo: cut.heardUpTo,\n          heardMs: cut.heardMs,\n        },\n        edits: [{ target, action: 'replace', messages: [replacement] }],\n      }),\n    context\n  );\n}\n"],
-  "mappings": "AAUA,SAAS,UAAU,SAA0B;AAC3C,MAAI,CAAC,QAAQ,KAAK,OAAO,GAAG;AAC1B,UAAM,IAAI,MAAM,wCAAwC,OAAO,EAAE;AAAA,EACnE;AACA,SAAO,OAAO,OAAO;AACvB;AAEA,SAAS,UAAU,OAAc,OAAe,KAAkC;AAChF,MAAI,MAAM,SAAS,cAAc,MAAM,SAAS,WAAY,QAAO;AACnE,MAAI,QAAQ,IAAI,aAAc,QAAO;AACrC,MAAI,QAAQ,IAAI,aAAc,QAAO;AACrC,QAAM,QAAQ,MAAM,KAAK,MAAM,GAAG,IAAI,SAAS;AAC/C,MAAI,MAAM,WAAW,EAAG,QAAO;AAC/B,MAAI,UAAU,MAAM,KAAM,QAAO;AACjC,SAAO,EAAE,GAAG,OAAO,MAAM,MAAM;AACjC;AAEA,SAAS,oBAAoB,SAA2B,KAAiC;AACvF,QAAM,UAAmB,CAAC;AAC1B,aAAW,CAAC,OAAO,KAAK,KAAK,QAAQ,QAAQ,QAAQ,GAAG;AACtD,UAAM,OAAO,UAAU,OAAO,OAAO,GAAG;AACxC,QAAI,SAAS,OAAW;AACxB,YAAQ,KAAK,IAAI;AAAA,EACnB;AACA,SAAO,EAAE,GAAG,SAAS,QAAQ;AAC/B;AAEA,eAAsB,oBACpB,SACA,gBACA,KACA,SACA,SACsB;AACtB,QAAM,cAAc,oBAAoB,SAAS,GAAG;AACpD,QAAM,SAAS,UAAU,IAAI,OAAO;AACpC,SAAO,QAAQ;AAAA,IACb,CAAC,OACC,GAAG,YAAY,gBAAgB;AAAA,MAC7B,MAAM;AAAA,MACN,MAAM;AAAA,QACJ,SAAS,IAAI;AAAA,QACb,cAAc,IAAI;AAAA,QAClB,WAAW,IAAI;AAAA,QACf,SAAS,IAAI;AAAA,MACf;AAAA,MACA,OAAO,CAAC,EAAE,QAAQ,QAAQ,WAAW,UAAU,CAAC,WAAW,EAAE,CAAC;AAAA,IAChE,CAAC;AAAA,IACH;AAAA,EACF;AACF;",
+  "sourcesContent": ["import type { AssistantMessage } from '@earendil-works/pi-ai';\nimport type { ConversationId, EntryId, EntryRecord, Tx } from '@earendil-works/pi-durable';\nimport type { VoiceCut } from '../turns.ts';\n\ntype Block = AssistantMessage['content'][number];\n\ninterface TruncateSession<C> {\n  commit<T>(change: (tx: Tx) => T | Promise<T>, context: C): Promise<T>;\n}\n\nfunction entryIdOf(entryId: string): EntryId {\n  if (!/^\\d+$/.test(entryId)) {\n    throw new Error(`Voice cut entry is not a durable id: ${entryId}`);\n  }\n  return Number(entryId) as EntryId;\n}\n\nfunction keptBlock(block: Block, index: number, cut: VoiceCut): Block | undefined {\n  if (block.type === 'toolCall' || block.type === 'thinking') return block;\n  if (index < cut.contentIndex) return block;\n  if (index > cut.contentIndex) return undefined;\n  const heard = block.text.slice(0, cut.heardUpTo);\n  if (heard.length === 0) return undefined;\n  if (heard === block.text) return block;\n  const shortened = { ...block, text: heard };\n  delete shortened.textSignature;\n  return shortened;\n}\n\nfunction cutAssistantMessage(message: AssistantMessage, cut: VoiceCut): AssistantMessage {\n  const content: Block[] = [];\n  for (const [index, block] of message.content.entries()) {\n    const kept = keptBlock(block, index, cut);\n    if (kept === undefined) continue;\n    content.push(kept);\n  }\n  return { ...message, content };\n}\n\nexport async function appendVoiceTruncate<C>(\n  session: TruncateSession<C>,\n  conversationId: ConversationId,\n  cut: VoiceCut,\n  message: AssistantMessage,\n  context: C\n): Promise<EntryRecord> {\n  const replacement = cutAssistantMessage(message, cut);\n  const target = entryIdOf(cut.entryId);\n  return session.commit(\n    (tx) =>\n      tx.appendEntry(conversationId, {\n        kind: 'voice.truncate',\n        data: {\n          entryId: cut.entryId,\n          contentIndex: cut.contentIndex,\n          heardUpTo: cut.heardUpTo,\n          heardMs: cut.heardMs,\n        },\n        edits: [{ target, action: 'replace', messages: [replacement] }],\n      }),\n    context\n  );\n}\n"],
+  "mappings": "AAUA,SAAS,UAAU,SAA0B;AAC3C,MAAI,CAAC,QAAQ,KAAK,OAAO,GAAG;AAC1B,UAAM,IAAI,MAAM,wCAAwC,OAAO,EAAE;AAAA,EACnE;AACA,SAAO,OAAO,OAAO;AACvB;AAEA,SAAS,UAAU,OAAc,OAAe,KAAkC;AAChF,MAAI,MAAM,SAAS,cAAc,MAAM,SAAS,WAAY,QAAO;AACnE,MAAI,QAAQ,IAAI,aAAc,QAAO;AACrC,MAAI,QAAQ,IAAI,aAAc,QAAO;AACrC,QAAM,QAAQ,MAAM,KAAK,MAAM,GAAG,IAAI,SAAS;AAC/C,MAAI,MAAM,WAAW,EAAG,QAAO;AAC/B,MAAI,UAAU,MAAM,KAAM,QAAO;AACjC,QAAM,YAAY,EAAE,GAAG,OAAO,MAAM,MAAM;AAC1C,SAAO,UAAU;AACjB,SAAO;AACT;AAEA,SAAS,oBAAoB,SAA2B,KAAiC;AACvF,QAAM,UAAmB,CAAC;AAC1B,aAAW,CAAC,OAAO,KAAK,KAAK,QAAQ,QAAQ,QAAQ,GAAG;AACtD,UAAM,OAAO,UAAU,OAAO,OAAO,GAAG;AACxC,QAAI,SAAS,OAAW;AACxB,YAAQ,KAAK,IAAI;AAAA,EACnB;AACA,SAAO,EAAE,GAAG,SAAS,QAAQ;AAC/B;AAEA,eAAsB,oBACpB,SACA,gBACA,KACA,SACA,SACsB;AACtB,QAAM,cAAc,oBAAoB,SAAS,GAAG;AACpD,QAAM,SAAS,UAAU,IAAI,OAAO;AACpC,SAAO,QAAQ;AAAA,IACb,CAAC,OACC,GAAG,YAAY,gBAAgB;AAAA,MAC7B,MAAM;AAAA,MACN,MAAM;AAAA,QACJ,SAAS,IAAI;AAAA,QACb,cAAc,IAAI;AAAA,QAClB,WAAW,IAAI;AAAA,QACf,SAAS,IAAI;AAAA,MACf;AAAA,MACA,OAAO,CAAC,EAAE,QAAQ,QAAQ,WAAW,UAAU,CAAC,WAAW,EAAE,CAAC;AAAA,IAChE,CAAC;AAAA,IACH;AAAA,EACF;AACF;",
   "names": []
 }

package/dist/fish-agents/events.js

@@ -0,0 +1,94 @@
+import { blankUsage } from "../blank-usage.js";
+function none() {
+  return [];
+}
+function isRecord(value) {
+  return typeof value === "object" && value !== null && !Array.isArray(value);
+}
+function text(value) {
+  return typeof value === "string" ? value : "";
+}
+function done(itemId, interrupted) {
+  return { type: "response_done", itemId, interrupted, usage: blankUsage() };
+}
+function ingestFishData(message) {
+  const type = message.type;
+  if (typeof type !== "string") return none();
+  if (type === "client_tool.call") return toolCall(message);
+  if (type === "error")
+    return [{ type: "error", error: text(message.code) || "error", fatal: false }];
+  if (type === "session.ended") {
+    const reason = text(message.reason);
+    return [{ type: "error", error: reason.length > 0 ? reason : "session ended", fatal: false }];
+  }
+  if (type === "tool.started" || type === "tool.completed" || type === "tool.failed") return none();
+  return none();
+}
+function toolCall(message) {
+  const name = text(message.toolName);
+  const id = text(message.callId);
+  if (name.length === 0 || id.length === 0) return none();
+  const params = isRecord(message.params) ? message.params : {};
+  const call = { type: "toolCall", id, name, arguments: params };
+  return [{ type: "tool_call", call, blocking: message.expectsResponse !== false }];
+}
+function ingestFishText(turn, packet, localIdentity) {
+  const itemId = text(packet.attributes["lk.segment_id"]);
+  const id = itemId.length > 0 ? itemId : "segment";
+  const final = packet.attributes["lk.transcription_final"] === "true";
+  if (packet.identity === localIdentity) {
+    return [{ type: "input_transcript", itemId: id, text: packet.text, final }];
+  }
+  if (turn.discardFinals > 0) {
+    if (final) turn.discardFinals -= 1;
+    return none();
+  }
+  turn.turnOpen = true;
+  return [{ type: "output_transcript", itemId: id, text: packet.text, final }];
+}
+function ingestFishAudio(turn, chunk) {
+  if (turn.dropAudio) return none();
+  turn.turnOpen = true;
+  return [{ type: "output_audio", itemId: "agent", chunk }];
+}
+function ingestFishState(turn, value) {
+  if (value === "speaking" || value === "thinking") {
+    turn.turnOpen = true;
+    return none();
+  }
+  if (value !== "listening") return none();
+  turn.dropAudio = false;
+  if (!turn.turnOpen) return none();
+  turn.turnOpen = false;
+  return [done("agent", false)];
+}
+function ingestSpeakers(turn, identities) {
+  const active = identities.length > 0;
+  if (!active) {
+    turn.speaking = false;
+    return none();
+  }
+  if (turn.speaking) return none();
+  turn.speaking = true;
+  return [{ type: "input_speech_start" }];
+}
+function userMessage(value, speak) {
+  const message = { type: "user.message", text: value };
+  if (!speak) message.audio = false;
+  return message;
+}
+function toolResult(callId, result, isError) {
+  const message = { type: "client_tool.result", callId, result };
+  if (isError) message.isError = true;
+  return message;
+}
+export {
+  ingestFishAudio,
+  ingestFishData,
+  ingestFishState,
+  ingestFishText,
+  ingestSpeakers,
+  toolResult,
+  userMessage
+};
+//# sourceMappingURL=events.js.map

package/dist/fish-agents/events.js.map

@@ -0,0 +1,7 @@
+{
+  "version": 3,
+  "sources": ["../../src/fish-agents/events.ts"],
+  "sourcesContent": ["import type { JsonObject, ToolCall } from '@earendil-works/pi-ai';\nimport type { AudioChunk } from '../audio.ts';\nimport { blankUsage } from '../blank-usage.ts';\nimport type { RealtimeEvent } from '../realtime.ts';\n\nexport interface FishTurn {\n  dropAudio: boolean;\n  discardFinals: number;\n  speaking: boolean;\n  turnOpen: boolean;\n}\n\nexport interface FishText {\n  identity: string;\n  text: string;\n  attributes: Record<string, string>;\n}\n\nfunction none(): RealtimeEvent[] {\n  return [];\n}\n\nfunction isRecord(value: unknown): value is Record<string, unknown> {\n  return typeof value === 'object' && value !== null && !Array.isArray(value);\n}\n\nfunction text(value: unknown): string {\n  return typeof value === 'string' ? value : '';\n}\n\nfunction done(itemId: string, interrupted: boolean): RealtimeEvent {\n  return { type: 'response_done', itemId, interrupted, usage: blankUsage() };\n}\n\nexport function ingestFishData(message: Record<string, unknown>): RealtimeEvent[] {\n  const type = message.type;\n  if (typeof type !== 'string') return none();\n  if (type === 'client_tool.call') return toolCall(message);\n  if (type === 'error')\n    return [{ type: 'error', error: text(message.code) || 'error', fatal: false }];\n  if (type === 'session.ended') {\n    const reason = text(message.reason);\n    return [{ type: 'error', error: reason.length > 0 ? reason : 'session ended', fatal: false }];\n  }\n  if (type === 'tool.started' || type === 'tool.completed' || type === 'tool.failed') return none();\n  return none();\n}\n\nfunction toolCall(message: Record<string, unknown>): RealtimeEvent[] {\n  const name = text(message.toolName);\n  const id = text(message.callId);\n  if (name.length === 0 || id.length === 0) return none();\n  const params = isRecord(message.params) ? message.params : {};\n  const call: ToolCall = { type: 'toolCall', id, name, arguments: params as JsonObject };\n  return [{ type: 'tool_call', call, blocking: message.expectsResponse !== false }];\n}\n\nexport function ingestFishText(\n  turn: FishTurn,\n  packet: FishText,\n  localIdentity: string\n): RealtimeEvent[] {\n  const itemId = text(packet.attributes['lk.segment_id']);\n  const id = itemId.length > 0 ? itemId : 'segment';\n  const final = packet.attributes['lk.transcription_final'] === 'true';\n  if (packet.identity === localIdentity) {\n    return [{ type: 'input_transcript', itemId: id, text: packet.text, final }];\n  }\n  if (turn.discardFinals > 0) {\n    if (final) turn.discardFinals -= 1;\n    return none();\n  }\n  turn.turnOpen = true;\n  return [{ type: 'output_transcript', itemId: id, text: packet.text, final }];\n}\n\nexport function ingestFishAudio(turn: FishTurn, chunk: AudioChunk): RealtimeEvent[] {\n  if (turn.dropAudio) return none();\n  turn.turnOpen = true;\n  return [{ type: 'output_audio', itemId: 'agent', chunk }];\n}\n\nexport function ingestFishState(turn: FishTurn, value: string): RealtimeEvent[] {\n  if (value === 'speaking' || value === 'thinking') {\n    turn.turnOpen = true;\n    return none();\n  }\n  if (value !== 'listening') return none();\n  turn.dropAudio = false;\n  if (!turn.turnOpen) return none();\n  turn.turnOpen = false;\n  return [done('agent', false)];\n}\n\nexport function ingestSpeakers(turn: FishTurn, identities: string[]): RealtimeEvent[] {\n  const active = identities.length > 0;\n  if (!active) {\n    turn.speaking = false;\n    return none();\n  }\n  if (turn.speaking) return none();\n  turn.speaking = true;\n  return [{ type: 'input_speech_start' }];\n}\n\nexport function userMessage(value: string, speak: boolean): JsonObject {\n  const message: JsonObject = { type: 'user.message', text: value };\n  if (!speak) message.audio = false;\n  return message;\n}\n\nexport function toolResult(callId: string, result: string, isError: boolean): JsonObject {\n  const message: JsonObject = { type: 'client_tool.result', callId, result };\n  if (isError) message.isError = true;\n  return message;\n}\n"],
+  "mappings": "AAEA,SAAS,kBAAkB;AAgB3B,SAAS,OAAwB;AAC/B,SAAO,CAAC;AACV;AAEA,SAAS,SAAS,OAAkD;AAClE,SAAO,OAAO,UAAU,YAAY,UAAU,QAAQ,CAAC,MAAM,QAAQ,KAAK;AAC5E;AAEA,SAAS,KAAK,OAAwB;AACpC,SAAO,OAAO,UAAU,WAAW,QAAQ;AAC7C;AAEA,SAAS,KAAK,QAAgB,aAAqC;AACjE,SAAO,EAAE,MAAM,iBAAiB,QAAQ,aAAa,OAAO,WAAW,EAAE;AAC3E;AAEO,SAAS,eAAe,SAAmD;AAChF,QAAM,OAAO,QAAQ;AACrB,MAAI,OAAO,SAAS,SAAU,QAAO,KAAK;AAC1C,MAAI,SAAS,mBAAoB,QAAO,SAAS,OAAO;AACxD,MAAI,SAAS;AACX,WAAO,CAAC,EAAE,MAAM,SAAS,OAAO,KAAK,QAAQ,IAAI,KAAK,SAAS,OAAO,MAAM,CAAC;AAC/E,MAAI,SAAS,iBAAiB;AAC5B,UAAM,SAAS,KAAK,QAAQ,MAAM;AAClC,WAAO,CAAC,EAAE,MAAM,SAAS,OAAO,OAAO,SAAS,IAAI,SAAS,iBAAiB,OAAO,MAAM,CAAC;AAAA,EAC9F;AACA,MAAI,SAAS,kBAAkB,SAAS,oBAAoB,SAAS,cAAe,QAAO,KAAK;AAChG,SAAO,KAAK;AACd;AAEA,SAAS,SAAS,SAAmD;AACnE,QAAM,OAAO,KAAK,QAAQ,QAAQ;AAClC,QAAM,KAAK,KAAK,QAAQ,MAAM;AAC9B,MAAI,KAAK,WAAW,KAAK,GAAG,WAAW,EAAG,QAAO,KAAK;AACtD,QAAM,SAAS,SAAS,QAAQ,MAAM,IAAI,QAAQ,SAAS,CAAC;AAC5D,QAAM,OAAiB,EAAE,MAAM,YAAY,IAAI,MAAM,WAAW,OAAqB;AACrF,SAAO,CAAC,EAAE,MAAM,aAAa,MAAM,UAAU,QAAQ,oBAAoB,MAAM,CAAC;AAClF;AAEO,SAAS,eACd,MACA,QACA,eACiB;AACjB,QAAM,SAAS,KAAK,OAAO,WAAW,eAAe,CAAC;AACtD,QAAM,KAAK,OAAO,SAAS,IAAI,SAAS;AACxC,QAAM,QAAQ,OAAO,WAAW,wBAAwB,MAAM;AAC9D,MAAI,OAAO,aAAa,eAAe;AACrC,WAAO,CAAC,EAAE,MAAM,oBAAoB,QAAQ,IAAI,MAAM,OAAO,MAAM,MAAM,CAAC;AAAA,EAC5E;AACA,MAAI,KAAK,gBAAgB,GAAG;AAC1B,QAAI,MAAO,MAAK,iBAAiB;AACjC,WAAO,KAAK;AAAA,EACd;AACA,OAAK,WAAW;AAChB,SAAO,CAAC,EAAE,MAAM,qBAAqB,QAAQ,IAAI,MAAM,OAAO,MAAM,MAAM,CAAC;AAC7E;AAEO,SAAS,gBAAgB,MAAgB,OAAoC;AAClF,MAAI,KAAK,UAAW,QAAO,KAAK;AAChC,OAAK,WAAW;AAChB,SAAO,CAAC,EAAE,MAAM,gBAAgB,QAAQ,SAAS,MAAM,CAAC;AAC1D;AAEO,SAAS,gBAAgB,MAAgB,OAAgC;AAC9E,MAAI,UAAU,cAAc,UAAU,YAAY;AAChD,SAAK,WAAW;AAChB,WAAO,KAAK;AAAA,EACd;AACA,MAAI,UAAU,YAAa,QAAO,KAAK;AACvC,OAAK,YAAY;AACjB,MAAI,CAAC,KAAK,SAAU,QAAO,KAAK;AAChC,OAAK,WAAW;AAChB,SAAO,CAAC,KAAK,SAAS,KAAK,CAAC;AAC9B;AAEO,SAAS,eAAe,MAAgB,YAAuC;AACpF,QAAM,SAAS,WAAW,SAAS;AACnC,MAAI,CAAC,QAAQ;AACX,SAAK,WAAW;AAChB,WAAO,KAAK;AAAA,EACd;AACA,MAAI,KAAK,SAAU,QAAO,KAAK;AAC/B,OAAK,WAAW;AAChB,SAAO,CAAC,EAAE,MAAM,qBAAqB,CAAC;AACxC;AAEO,SAAS,YAAY,OAAe,OAA4B;AACrE,QAAM,UAAsB,EAAE,MAAM,gBAAgB,MAAM,MAAM;AAChE,MAAI,CAAC,MAAO,SAAQ,QAAQ;AAC5B,SAAO;AACT;AAEO,SAAS,WAAW,QAAgB,QAAgB,SAA8B;AACvF,QAAM,UAAsB,EAAE,MAAM,sBAAsB,QAAQ,OAAO;AACzE,MAAI,QAAS,SAAQ,UAAU;AAC/B,SAAO;AACT;",
+  "names": []
+}

package/dist/fish-agents/index.js

@@ -0,0 +1,57 @@
+import { registerVoiceModel } from "../models.js";
+import { registerRealtimeApiProvider } from "../registry.js";
+import { connectFish } from "./session.js";
+const voices = [{ id: "802e3bc2b27e49c2995d23ef70e6ac89", name: "Example" }];
+function pcm16(sampleRate) {
+  return { encoding: "pcm16", sampleRate, channels: 1 };
+}
+const fishAgent = {
+  id: "fish-agent",
+  name: "Fish Agent",
+  api: "fish-agents",
+  provider: "fish",
+  baseUrl: "https://api.fish.audio",
+  type: "realtime",
+  transports: ["webrtc"],
+  toolCalls: "blocking",
+  inputFormat: pcm16(24e3),
+  outputFormat: pcm16(24e3),
+  maxSessionMs: 18e5,
+  resumable: false,
+  ephemeralAuth: true,
+  voices,
+  cascade: { stt: "deepgram:nova-3", llm: "fish", tts: "s2.1-pro" },
+  provisioning: "agent",
+  delivery: { interrupt: "native", when_idle: "approximate", silent: "approximate" },
+  truncation: "server",
+  liveToolUpdate: false,
+  toolTimeoutMs: { min: 1e3, max: 12e4 },
+  cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
+  audioCost: { perInputMinute: 0 }
+};
+function registerFishAgents(options = {}) {
+  const unregisterModel = registerVoiceModel(fishAgent);
+  const unregisterApi = registerRealtimeApiProvider({
+    api: "fish-agents",
+    connect: (model, context, connectOptions) => connectFish(model, context, connectOptions, options)
+  });
+  return () => {
+    unregisterModel();
+    unregisterApi();
+  };
+}
+import { hashHostedSpec, memoryHostedCache } from "../hosted-agent.js";
+import { agentConfig, httpsOrigin, provisionAgent } from "./provision.js";
+import { importFishSdk, openFishRoom } from "./room.js";
+export {
+  agentConfig,
+  fishAgent,
+  hashHostedSpec,
+  httpsOrigin,
+  importFishSdk,
+  memoryHostedCache,
+  openFishRoom,
+  provisionAgent,
+  registerFishAgents
+};
+//# sourceMappingURL=index.js.map

package/dist/fish-agents/index.js.map

@@ -0,0 +1,7 @@
+{
+  "version": 3,
+  "sources": ["../../src/fish-agents/index.ts"],
+  "sourcesContent": ["import type { RealtimeModel } from '../models.ts';\nimport { registerVoiceModel } from '../models.ts';\nimport { registerRealtimeApiProvider } from '../registry.ts';\nimport type { FishOptions } from './session.ts';\nimport { connectFish } from './session.ts';\n\nconst voices = [{ id: '802e3bc2b27e49c2995d23ef70e6ac89', name: 'Example' }];\n\nfunction pcm16(sampleRate: number) {\n  return { encoding: 'pcm16' as const, sampleRate, channels: 1 as const };\n}\n\nexport const fishAgent: RealtimeModel<'fish-agents'> = {\n  id: 'fish-agent',\n  name: 'Fish Agent',\n  api: 'fish-agents',\n  provider: 'fish',\n  baseUrl: 'https://api.fish.audio',\n  type: 'realtime',\n  transports: ['webrtc'],\n  toolCalls: 'blocking',\n  inputFormat: pcm16(24000),\n  outputFormat: pcm16(24000),\n  maxSessionMs: 1_800_000,\n  resumable: false,\n  ephemeralAuth: true,\n  voices,\n  cascade: { stt: 'deepgram:nova-3', llm: 'fish', tts: 's2.1-pro' },\n  provisioning: 'agent',\n  delivery: { interrupt: 'native', when_idle: 'approximate', silent: 'approximate' },\n  truncation: 'server',\n  liveToolUpdate: false,\n  toolTimeoutMs: { min: 1000, max: 120_000 },\n  cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },\n  audioCost: { perInputMinute: 0 },\n};\n\nexport function registerFishAgents(options: FishOptions = {}): () => void {\n  const unregisterModel = registerVoiceModel(fishAgent);\n  const unregisterApi = registerRealtimeApiProvider({\n    api: 'fish-agents',\n    connect: (model, context, connectOptions) =>\n      connectFish(model, context, connectOptions, options),\n  });\n  return () => {\n    unregisterModel();\n    unregisterApi();\n  };\n}\n\nexport type { CustomLlm, HostedAgent, HostedAgentSpec, HostedCache } from '../hosted-agent.ts';\nexport { hashHostedSpec, memoryHostedCache } from '../hosted-agent.ts';\nexport type { ProvisionOptions } from './provision.ts';\nexport { agentConfig, httpsOrigin, provisionAgent } from './provision.ts';\nexport type { FishRoom, FishRoomFactory, FishSdk } from './room.ts';\nexport { importFishSdk, openFishRoom } from './room.ts';\nexport type { FishOptions } from './session.ts';\n"],
+  "mappings": "AACA,SAAS,0BAA0B;AACnC,SAAS,mCAAmC;AAE5C,SAAS,mBAAmB;AAE5B,MAAM,SAAS,CAAC,EAAE,IAAI,oCAAoC,MAAM,UAAU,CAAC;AAE3E,SAAS,MAAM,YAAoB;AACjC,SAAO,EAAE,UAAU,SAAkB,YAAY,UAAU,EAAW;AACxE;AAEO,MAAM,YAA0C;AAAA,EACrD,IAAI;AAAA,EACJ,MAAM;AAAA,EACN,KAAK;AAAA,EACL,UAAU;AAAA,EACV,SAAS;AAAA,EACT,MAAM;AAAA,EACN,YAAY,CAAC,QAAQ;AAAA,EACrB,WAAW;AAAA,EACX,aAAa,MAAM,IAAK;AAAA,EACxB,cAAc,MAAM,IAAK;AAAA,EACzB,cAAc;AAAA,EACd,WAAW;AAAA,EACX,eAAe;AAAA,EACf;AAAA,EACA,SAAS,EAAE,KAAK,mBAAmB,KAAK,QAAQ,KAAK,WAAW;AAAA,EAChE,cAAc;AAAA,EACd,UAAU,EAAE,WAAW,UAAU,WAAW,eAAe,QAAQ,cAAc;AAAA,EACjF,YAAY;AAAA,EACZ,gBAAgB;AAAA,EAChB,eAAe,EAAE,KAAK,KAAM,KAAK,KAAQ;AAAA,EACzC,MAAM,EAAE,OAAO,GAAG,QAAQ,GAAG,WAAW,GAAG,YAAY,EAAE;AAAA,EACzD,WAAW,EAAE,gBAAgB,EAAE;AACjC;AAEO,SAAS,mBAAmB,UAAuB,CAAC,GAAe;AACxE,QAAM,kBAAkB,mBAAmB,SAAS;AACpD,QAAM,gBAAgB,4BAA4B;AAAA,IAChD,KAAK;AAAA,IACL,SAAS,CAAC,OAAO,SAAS,mBACxB,YAAY,OAAO,SAAS,gBAAgB,OAAO;AAAA,EACvD,CAAC;AACD,SAAO,MAAM;AACX,oBAAgB;AAChB,kBAAc;AAAA,EAChB;AACF;AAGA,SAAS,gBAAgB,yBAAyB;AAElD,SAAS,aAAa,aAAa,sBAAsB;AAEzD,SAAS,eAAe,oBAAoB;",
+  "names": []
+}

package/dist/fish-agents/provision.js

@@ -0,0 +1,178 @@
+import { VoiceRequestError } from "../errors.js";
+import {
+  hashHostedSpec,
+  hostedCacheKey,
+  memoryHostedCache
+} from "../hosted-agent.js";
+const defaultCache = memoryHostedCache();
+const toolName = /^[a-zA-Z][a-zA-Z0-9_-]{0,63}$/;
+function isRecord(value) {
+  return typeof value === "object" && value !== null && !Array.isArray(value);
+}
+function asVoice(error) {
+  if (error instanceof VoiceRequestError) return error;
+  if (error instanceof Error && error.name === "AbortError") {
+    return new VoiceRequestError("aborted", "session aborted");
+  }
+  if (error instanceof Error) return new VoiceRequestError("error", error.message);
+  return new VoiceRequestError("error", String(error));
+}
+function httpsOrigin(baseUrl) {
+  const url = new URL(baseUrl);
+  let protocol = url.protocol;
+  if (protocol === "wss:") protocol = "https:";
+  if (protocol === "ws:") protocol = "http:";
+  return `${protocol}//${url.host}`;
+}
+function mergedHeaders(options, defaults) {
+  const headers = { ...defaults };
+  const extra = options.headers;
+  if (extra === void 0) return headers;
+  for (const [key, value] of Object.entries(extra)) {
+    if (value === null) delete headers[key];
+    else headers[key] = value;
+  }
+  return headers;
+}
+function apiKey(options) {
+  const key = options.apiKey;
+  if (typeof key === "string" && key.length > 0) return key;
+  throw new VoiceRequestError("error", "missing realtime credential");
+}
+function timeoutSeconds(timeoutMs) {
+  if (timeoutMs === void 0) return 30;
+  return Math.min(120, Math.max(1, Math.round(timeoutMs / 1e3)));
+}
+function argumentsOf(tool) {
+  const parameters = tool.parameters;
+  if (!isRecord(parameters) || !isRecord(parameters.properties)) return [];
+  const args = [];
+  for (const [name, schema] of Object.entries(parameters.properties)) {
+    const description = isRecord(schema) && typeof schema.description === "string" ? schema.description : "";
+    args.push({ name, description });
+  }
+  return args;
+}
+function clientTool(tool, timeoutMs) {
+  if (!toolName.test(tool.name)) {
+    throw new VoiceRequestError("error", `invalid client tool name: ${tool.name}`);
+  }
+  return {
+    tool_type: "client",
+    name: tool.name,
+    description: tool.description,
+    arguments: argumentsOf(tool),
+    expects_response: true,
+    timeout_seconds: timeoutSeconds(timeoutMs)
+  };
+}
+function bearer(headers) {
+  if (headers === void 0) return void 0;
+  const value = headers.Authorization ?? headers.authorization;
+  if (typeof value !== "string" || value.length === 0) return void 0;
+  if (value.startsWith("Bearer ")) return value.slice("Bearer ".length);
+  return value;
+}
+function llmFields(llm) {
+  if (typeof llm === "string") return { model: llm };
+  if (llm.api === "openai-responses") {
+    throw new VoiceRequestError("error", "fish custom llm accepts openai-completions only");
+  }
+  const custom = { base_url: llm.baseUrl, model: llm.id };
+  const key = bearer(llm.headers);
+  if (key !== void 0) custom.api_key = key;
+  return { custom };
+}
+function agentConfig(spec, toolIds) {
+  const prompt = { system_prompt: spec.instructions };
+  if (spec.firstMessage !== void 0) prompt.first_message = spec.firstMessage;
+  const config = {
+    prompt,
+    voice: { voice_id: spec.voice },
+    llm: llmFields(spec.llm)
+  };
+  if (toolIds.length > 0) config.tools = { enabled: true, tool_ids: toolIds };
+  return config;
+}
+async function requestJson(fetchImpl, url, init) {
+  let response;
+  try {
+    response = await fetchImpl(url, init);
+  } catch (error) {
+    throw asVoice(error);
+  }
+  if (!response.ok) {
+    throw new VoiceRequestError("error", `agent request failed (${response.status})`);
+  }
+  if (response.status === 204) return {};
+  let json;
+  try {
+    json = await response.json();
+  } catch (error) {
+    throw asVoice(error);
+  }
+  if (!isRecord(json)) throw new VoiceRequestError("error", "agent response was not an object");
+  return json;
+}
+function textId(json, field, fallback) {
+  const id = json[field];
+  if (typeof id === "string" && id.length > 0) return id;
+  if (fallback !== void 0) return fallback;
+  throw new VoiceRequestError("error", `agent response had no ${field}`);
+}
+async function createTools(origin, spec, options, fetchImpl) {
+  const ids = [];
+  for (const tool of spec.tools) {
+    const json = await requestJson(fetchImpl, `${origin}/v1/agent/tools`, {
+      method: "POST",
+      headers: mergedHeaders(options, {
+        "Content-Type": "application/json",
+        Authorization: `Bearer ${apiKey(options)}`
+      }),
+      body: JSON.stringify(clientTool(tool, spec.clientToolTimeoutMs)),
+      signal: options.signal
+    });
+    ids.push(textId(json, "tool_id", void 0));
+  }
+  return ids;
+}
+async function provisionAgent(model, spec, options = {}) {
+  if (options.signal?.aborted === true) throw new VoiceRequestError("aborted", "session aborted");
+  const cache = options.cache ?? defaultCache;
+  const now = options.now ?? Date.now;
+  const key = hostedCacheKey(model.provider, spec.name);
+  const hash = hashHostedSpec(spec);
+  const cached = cache.get(key);
+  if (cached !== void 0 && cached.specHash === hash) return cached;
+  const origin = httpsOrigin(model.baseUrl);
+  const fetchImpl = options.fetch ?? globalThis.fetch;
+  const toolIds = await createTools(origin, spec, options, fetchImpl);
+  const config = agentConfig(spec, toolIds);
+  const existing = cached?.agentId ?? options.agentId;
+  const headers = mergedHeaders(options, {
+    "Content-Type": "application/json",
+    Authorization: `Bearer ${apiKey(options)}`
+  });
+  const url = existing === void 0 ? `${origin}/v1/agent/agents` : `${origin}/v1/agent/agents/${encodeURIComponent(existing)}/config`;
+  const json = await requestJson(fetchImpl, url, {
+    method: existing === void 0 ? "POST" : "PATCH",
+    headers,
+    body: JSON.stringify(existing === void 0 ? { name: spec.name, config } : config),
+    signal: options.signal
+  });
+  const agentId = textId(json, "agent_id", existing);
+  await requestJson(fetchImpl, `${origin}/v1/agent/agents/${encodeURIComponent(agentId)}/publish`, {
+    method: "POST",
+    headers: mergedHeaders(options, { Authorization: `Bearer ${apiKey(options)}` }),
+    signal: options.signal
+  });
+  const agent = { agentId, specHash: hash, publishedAt: now() };
+  cache.set(key, agent);
+  return agent;
+}
+export {
+  agentConfig,
+  httpsOrigin,
+  provisionAgent
+};
+//# sourceMappingURL=provision.js.map

package/dist/fish-agents/provision.js.map

@@ -0,0 +1,7 @@
+{
+  "version": 3,
+  "sources": ["../../src/fish-agents/provision.ts"],
+  "sourcesContent": ["import type { JsonObject, ProviderRequestOptions, Tool } from '@earendil-works/pi-ai';\nimport { VoiceRequestError } from '../errors.ts';\nimport {\n  type CustomLlm,\n  type HostedAgent,\n  type HostedAgentSpec,\n  type HostedCache,\n  hashHostedSpec,\n  hostedCacheKey,\n  memoryHostedCache,\n} from '../hosted-agent.ts';\nimport type { RealtimeModel } from '../models.ts';\n\nexport interface ProvisionOptions extends ProviderRequestOptions {\n  agentId?: string;\n  now?: () => number;\n  cache?: HostedCache;\n}\n\nconst defaultCache = memoryHostedCache();\nconst toolName = /^[a-zA-Z][a-zA-Z0-9_-]{0,63}$/;\n\nfunction isRecord(value: unknown): value is Record<string, unknown> {\n  return typeof value === 'object' && value !== null && !Array.isArray(value);\n}\n\nfunction asVoice(error: unknown): VoiceRequestError {\n  if (error instanceof VoiceRequestError) return error;\n  if (error instanceof Error && error.name === 'AbortError') {\n    return new VoiceRequestError('aborted', 'session aborted');\n  }\n  if (error instanceof Error) return new VoiceRequestError('error', error.message);\n  return new VoiceRequestError('error', String(error));\n}\n\nexport function httpsOrigin(baseUrl: string): string {\n  const url = new URL(baseUrl);\n  let protocol = url.protocol;\n  if (protocol === 'wss:') protocol = 'https:';\n  if (protocol === 'ws:') protocol = 'http:';\n  return `${protocol}//${url.host}`;\n}\n\nfunction mergedHeaders(\n  options: ProvisionOptions,\n  defaults: Record<string, string>\n): Record<string, string> {\n  const headers = { ...defaults };\n  const extra = options.headers;\n  if (extra === undefined) return headers;\n  for (const [key, value] of Object.entries(extra)) {\n    if (value === null) delete headers[key];\n    else headers[key] = value;\n  }\n  return headers;\n}\n\nfunction apiKey(options: ProvisionOptions): string {\n  const key = options.apiKey;\n  if (typeof key === 'string' && key.length > 0) return key;\n  throw new VoiceRequestError('error', 'missing realtime credential');\n}\n\nfunction timeoutSeconds(timeoutMs: number | undefined): number {\n  if (timeoutMs === undefined) return 30;\n  return Math.min(120, Math.max(1, Math.round(timeoutMs / 1000)));\n}\n\nfunction argumentsOf(tool: Tool): JsonObject[] {\n  const parameters = tool.parameters as unknown;\n  if (!isRecord(parameters) || !isRecord(parameters.properties)) return [];\n  const args: JsonObject[] = [];\n  for (const [name, schema] of Object.entries(parameters.properties)) {\n    const description =\n      isRecord(schema) && typeof schema.description === 'string' ? schema.description : '';\n    args.push({ name, description });\n  }\n  return args;\n}\n\nfunction clientTool(tool: Tool, timeoutMs: number | undefined): JsonObject {\n  if (!toolName.test(tool.name)) {\n    throw new VoiceRequestError('error', `invalid client tool name: ${tool.name}`);\n  }\n  return {\n    tool_type: 'client',\n    name: tool.name,\n    description: tool.description,\n    arguments: argumentsOf(tool),\n    expects_response: true,\n    timeout_seconds: timeoutSeconds(timeoutMs),\n  };\n}\n\nfunction bearer(headers: CustomLlm['headers']): string | undefined {\n  if (headers === undefined) return undefined;\n  const value = headers.Authorization ?? headers.authorization;\n  if (typeof value !== 'string' || value.length === 0) return undefined;\n  if (value.startsWith('Bearer ')) return value.slice('Bearer '.length);\n  return value;\n}\n\nfunction llmFields(llm: string | CustomLlm): JsonObject {\n  if (typeof llm === 'string') return { model: llm };\n  if (llm.api === 'openai-responses') {\n    throw new VoiceRequestError('error', 'fish custom llm accepts openai-completions only');\n  }\n  const custom: JsonObject = { base_url: llm.baseUrl, model: llm.id };\n  const key = bearer(llm.headers);\n  if (key !== undefined) custom.api_key = key;\n  return { custom };\n}\n\nexport function agentConfig(spec: HostedAgentSpec, toolIds: string[]): JsonObject {\n  const prompt: JsonObject = { system_prompt: spec.instructions };\n  if (spec.firstMessage !== undefined) prompt.first_message = spec.firstMessage;\n  const config: JsonObject = {\n    prompt,\n    voice: { voice_id: spec.voice },\n    llm: llmFields(spec.llm),\n  };\n  if (toolIds.length > 0) config.tools = { enabled: true, tool_ids: toolIds };\n  return config;\n}\n\nasync function requestJson(\n  fetchImpl: typeof fetch,\n  url: string,\n  init: RequestInit\n): Promise<Record<string, unknown>> {\n  let response: Response;\n  try {\n    response = await fetchImpl(url, init);\n  } catch (error) {\n    throw asVoice(error);\n  }\n  if (!response.ok) {\n    throw new VoiceRequestError('error', `agent request failed (${response.status})`);\n  }\n  if (response.status === 204) return {};\n  let json: unknown;\n  try {\n    json = await response.json();\n  } catch (error) {\n    throw asVoice(error);\n  }\n  if (!isRecord(json)) throw new VoiceRequestError('error', 'agent response was not an object');\n  return json;\n}\n\nfunction textId(\n  json: Record<string, unknown>,\n  field: string,\n  fallback: string | undefined\n): string {\n  const id = json[field];\n  if (typeof id === 'string' && id.length > 0) return id;\n  if (fallback !== undefined) return fallback;\n  throw new VoiceRequestError('error', `agent response had no ${field}`);\n}\n\nasync function createTools(\n  origin: string,\n  spec: HostedAgentSpec,\n  options: ProvisionOptions,\n  fetchImpl: typeof fetch\n): Promise<string[]> {\n  const ids: string[] = [];\n  for (const tool of spec.tools) {\n    const json = await requestJson(fetchImpl, `${origin}/v1/agent/tools`, {\n      method: 'POST',\n      headers: mergedHeaders(options, {\n        'Content-Type': 'application/json',\n        Authorization: `Bearer ${apiKey(options)}`,\n      }),\n      body: JSON.stringify(clientTool(tool, spec.clientToolTimeoutMs)),\n      signal: options.signal,\n    });\n    ids.push(textId(json, 'tool_id', undefined));\n  }\n  return ids;\n}\n\nexport async function provisionAgent(\n  model: RealtimeModel<string>,\n  spec: HostedAgentSpec,\n  options: ProvisionOptions = {}\n): Promise<HostedAgent> {\n  if (options.signal?.aborted === true) throw new VoiceRequestError('aborted', 'session aborted');\n  const cache = options.cache ?? defaultCache;\n  const now = options.now ?? Date.now;\n  const key = hostedCacheKey(model.provider, spec.name);\n  const hash = hashHostedSpec(spec);\n  const cached = cache.get(key);\n  if (cached !== undefined && cached.specHash === hash) return cached;\n  const origin = httpsOrigin(model.baseUrl);\n  const fetchImpl = options.fetch ?? globalThis.fetch;\n  const toolIds = await createTools(origin, spec, options, fetchImpl);\n  const config = agentConfig(spec, toolIds);\n  const existing = cached?.agentId ?? options.agentId;\n  const headers = mergedHeaders(options, {\n    'Content-Type': 'application/json',\n    Authorization: `Bearer ${apiKey(options)}`,\n  });\n  const url =\n    existing === undefined\n      ? `${origin}/v1/agent/agents`\n      : `${origin}/v1/agent/agents/${encodeURIComponent(existing)}/config`;\n  const json = await requestJson(fetchImpl, url, {\n    method: existing === undefined ? 'POST' : 'PATCH',\n    headers,\n    body: JSON.stringify(existing === undefined ? { name: spec.name, config } : config),\n    signal: options.signal,\n  });\n  const agentId = textId(json, 'agent_id', existing);\n  await requestJson(fetchImpl, `${origin}/v1/agent/agents/${encodeURIComponent(agentId)}/publish`, {\n    method: 'POST',\n    headers: mergedHeaders(options, { Authorization: `Bearer ${apiKey(options)}` }),\n    signal: options.signal,\n  });\n  const agent: HostedAgent = { agentId, specHash: hash, publishedAt: now() };\n  cache.set(key, agent);\n  return agent;\n}\n"],
+  "mappings": "AACA,SAAS,yBAAyB;AAClC;AAAA,EAKE;AAAA,EACA;AAAA,EACA;AAAA,OACK;AASP,MAAM,eAAe,kBAAkB;AACvC,MAAM,WAAW;AAEjB,SAAS,SAAS,OAAkD;AAClE,SAAO,OAAO,UAAU,YAAY,UAAU,QAAQ,CAAC,MAAM,QAAQ,KAAK;AAC5E;AAEA,SAAS,QAAQ,OAAmC;AAClD,MAAI,iBAAiB,kBAAmB,QAAO;AAC/C,MAAI,iBAAiB,SAAS,MAAM,SAAS,cAAc;AACzD,WAAO,IAAI,kBAAkB,WAAW,iBAAiB;AAAA,EAC3D;AACA,MAAI,iBAAiB,MAAO,QAAO,IAAI,kBAAkB,SAAS,MAAM,OAAO;AAC/E,SAAO,IAAI,kBAAkB,SAAS,OAAO,KAAK,CAAC;AACrD;AAEO,SAAS,YAAY,SAAyB;AACnD,QAAM,MAAM,IAAI,IAAI,OAAO;AAC3B,MAAI,WAAW,IAAI;AACnB,MAAI,aAAa,OAAQ,YAAW;AACpC,MAAI,aAAa,MAAO,YAAW;AACnC,SAAO,GAAG,QAAQ,KAAK,IAAI,IAAI;AACjC;AAEA,SAAS,cACP,SACA,UACwB;AACxB,QAAM,UAAU,EAAE,GAAG,SAAS;AAC9B,QAAM,QAAQ,QAAQ;AACtB,MAAI,UAAU,OAAW,QAAO;AAChC,aAAW,CAAC,KAAK,KAAK,KAAK,OAAO,QAAQ,KAAK,GAAG;AAChD,QAAI,UAAU,KAAM,QAAO,QAAQ,GAAG;AAAA,QACjC,SAAQ,GAAG,IAAI;AAAA,EACtB;AACA,SAAO;AACT;AAEA,SAAS,OAAO,SAAmC;AACjD,QAAM,MAAM,QAAQ;AACpB,MAAI,OAAO,QAAQ,YAAY,IAAI,SAAS,EAAG,QAAO;AACtD,QAAM,IAAI,kBAAkB,SAAS,6BAA6B;AACpE;AAEA,SAAS,eAAe,WAAuC;AAC7D,MAAI,cAAc,OAAW,QAAO;AACpC,SAAO,KAAK,IAAI,KAAK,KAAK,IAAI,GAAG,KAAK,MAAM,YAAY,GAAI,CAAC,CAAC;AAChE;AAEA,SAAS,YAAY,MAA0B;AAC7C,QAAM,aAAa,KAAK;AACxB,MAAI,CAAC,SAAS,UAAU,KAAK,CAAC,SAAS,WAAW,UAAU,EAAG,QAAO,CAAC;AACvE,QAAM,OAAqB,CAAC;AAC5B,aAAW,CAAC,MAAM,MAAM,KAAK,OAAO,QAAQ,WAAW,UAAU,GAAG;AAClE,UAAM,cACJ,SAAS,MAAM,KAAK,OAAO,OAAO,gBAAgB,WAAW,OAAO,cAAc;AACpF,SAAK,KAAK,EAAE,MAAM,YAAY,CAAC;AAAA,EACjC;AACA,SAAO;AACT;AAEA,SAAS,WAAW,MAAY,WAA2C;AACzE,MAAI,CAAC,SAAS,KAAK,KAAK,IAAI,GAAG;AAC7B,UAAM,IAAI,kBAAkB,SAAS,6BAA6B,KAAK,IAAI,EAAE;AAAA,EAC/E;AACA,SAAO;AAAA,IACL,WAAW;AAAA,IACX,MAAM,KAAK;AAAA,IACX,aAAa,KAAK;AAAA,IAClB,WAAW,YAAY,IAAI;AAAA,IAC3B,kBAAkB;AAAA,IAClB,iBAAiB,eAAe,SAAS;AAAA,EAC3C;AACF;AAEA,SAAS,OAAO,SAAmD;AACjE,MAAI,YAAY,OAAW,QAAO;AAClC,QAAM,QAAQ,QAAQ,iBAAiB,QAAQ;AAC/C,MAAI,OAAO,UAAU,YAAY,MAAM,WAAW,EAAG,QAAO;AAC5D,MAAI,MAAM,WAAW,SAAS,EAAG,QAAO,MAAM,MAAM,UAAU,MAAM;AACpE,SAAO;AACT;AAEA,SAAS,UAAU,KAAqC;AACtD,MAAI,OAAO,QAAQ,SAAU,QAAO,EAAE,OAAO,IAAI;AACjD,MAAI,IAAI,QAAQ,oBAAoB;AAClC,UAAM,IAAI,kBAAkB,SAAS,iDAAiD;AAAA,EACxF;AACA,QAAM,SAAqB,EAAE,UAAU,IAAI,SAAS,OAAO,IAAI,GAAG;AAClE,QAAM,MAAM,OAAO,IAAI,OAAO;AAC9B,MAAI,QAAQ,OAAW,QAAO,UAAU;AACxC,SAAO,EAAE,OAAO;AAClB;AAEO,SAAS,YAAY,MAAuB,SAA+B;AAChF,QAAM,SAAqB,EAAE,eAAe,KAAK,aAAa;AAC9D,MAAI,KAAK,iBAAiB,OAAW,QAAO,gBAAgB,KAAK;AACjE,QAAM,SAAqB;AAAA,IACzB;AAAA,IACA,OAAO,EAAE,UAAU,KAAK,MAAM;AAAA,IAC9B,KAAK,UAAU,KAAK,GAAG;AAAA,EACzB;AACA,MAAI,QAAQ,SAAS,EAAG,QAAO,QAAQ,EAAE,SAAS,MAAM,UAAU,QAAQ;AAC1E,SAAO;AACT;AAEA,eAAe,YACb,WACA,KACA,MACkC;AAClC,MAAI;AACJ,MAAI;AACF,eAAW,MAAM,UAAU,KAAK,IAAI;AAAA,EACtC,SAAS,OAAO;AACd,UAAM,QAAQ,KAAK;AAAA,EACrB;AACA,MAAI,CAAC,SAAS,IAAI;AAChB,UAAM,IAAI,kBAAkB,SAAS,yBAAyB,SAAS,MAAM,GAAG;AAAA,EAClF;AACA,MAAI,SAAS,WAAW,IAAK,QAAO,CAAC;AACrC,MAAI;AACJ,MAAI;AACF,WAAO,MAAM,SAAS,KAAK;AAAA,EAC7B,SAAS,OAAO;AACd,UAAM,QAAQ,KAAK;AAAA,EACrB;AACA,MAAI,CAAC,SAAS,IAAI,EAAG,OAAM,IAAI,kBAAkB,SAAS,kCAAkC;AAC5F,SAAO;AACT;AAEA,SAAS,OACP,MACA,OACA,UACQ;AACR,QAAM,KAAK,KAAK,KAAK;AACrB,MAAI,OAAO,OAAO,YAAY,GAAG,SAAS,EAAG,QAAO;AACpD,MAAI,aAAa,OAAW,QAAO;AACnC,QAAM,IAAI,kBAAkB,SAAS,yBAAyB,KAAK,EAAE;AACvE;AAEA,eAAe,YACb,QACA,MACA,SACA,WACmB;AACnB,QAAM,MAAgB,CAAC;AACvB,aAAW,QAAQ,KAAK,OAAO;AAC7B,UAAM,OAAO,MAAM,YAAY,WAAW,GAAG,MAAM,mBAAmB;AAAA,MACpE,QAAQ;AAAA,MACR,SAAS,cAAc,SAAS;AAAA,QAC9B,gBAAgB;AAAA,QAChB,eAAe,UAAU,OAAO,OAAO,CAAC;AAAA,MAC1C,CAAC;AAAA,MACD,MAAM,KAAK,UAAU,WAAW,MAAM,KAAK,mBAAmB,CAAC;AAAA,MAC/D,QAAQ,QAAQ;AAAA,IAClB,CAAC;AACD,QAAI,KAAK,OAAO,MAAM,WAAW,MAAS,CAAC;AAAA,EAC7C;AACA,SAAO;AACT;AAEA,eAAsB,eACpB,OACA,MACA,UAA4B,CAAC,GACP;AACtB,MAAI,QAAQ,QAAQ,YAAY,KAAM,OAAM,IAAI,kBAAkB,WAAW,iBAAiB;AAC9F,QAAM,QAAQ,QAAQ,SAAS;AAC/B,QAAM,MAAM,QAAQ,OAAO,KAAK;AAChC,QAAM,MAAM,eAAe,MAAM,UAAU,KAAK,IAAI;AACpD,QAAM,OAAO,eAAe,IAAI;AAChC,QAAM,SAAS,MAAM,IAAI,GAAG;AAC5B,MAAI,WAAW,UAAa,OAAO,aAAa,KAAM,QAAO;AAC7D,QAAM,SAAS,YAAY,MAAM,OAAO;AACxC,QAAM,YAAY,QAAQ,SAAS,WAAW;AAC9C,QAAM,UAAU,MAAM,YAAY,QAAQ,MAAM,SAAS,SAAS;AAClE,QAAM,SAAS,YAAY,MAAM,OAAO;AACxC,QAAM,WAAW,QAAQ,WAAW,QAAQ;AAC5C,QAAM,UAAU,cAAc,SAAS;AAAA,IACrC,gBAAgB;AAAA,IAChB,eAAe,UAAU,OAAO,OAAO,CAAC;AAAA,EAC1C,CAAC;AACD,QAAM,MACJ,aAAa,SACT,GAAG,MAAM,qBACT,GAAG,MAAM,oBAAoB,mBAAmB,QAAQ,CAAC;AAC/D,QAAM,OAAO,MAAM,YAAY,WAAW,KAAK;AAAA,IAC7C,QAAQ,aAAa,SAAY,SAAS;AAAA,IAC1C;AAAA,IACA,MAAM,KAAK,UAAU,aAAa,SAAY,EAAE,MAAM,KAAK,MAAM,OAAO,IAAI,MAAM;AAAA,IAClF,QAAQ,QAAQ;AAAA,EAClB,CAAC;AACD,QAAM,UAAU,OAAO,MAAM,YAAY,QAAQ;AACjD,QAAM,YAAY,WAAW,GAAG,MAAM,oBAAoB,mBAAmB,OAAO,CAAC,YAAY;AAAA,IAC/F,QAAQ;AAAA,IACR,SAAS,cAAc,SAAS,EAAE,eAAe,UAAU,OAAO,OAAO,CAAC,GAAG,CAAC;AAAA,IAC9E,QAAQ,QAAQ;AAAA,EAClB,CAAC;AACD,QAAM,QAAqB,EAAE,SAAS,UAAU,MAAM,aAAa,IAAI,EAAE;AACzE,QAAM,IAAI,KAAK,KAAK;AACpB,SAAO;AACT;",
+  "names": []
+}