@ai-ecoverse/pi-audio 1.24.0 → 1.26.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -0
- package/dist/brain/call.js +36 -0
- package/dist/brain/call.js.map +7 -0
- package/dist/brain/hash.js +12 -0
- package/dist/brain/hash.js.map +7 -0
- package/dist/brain/index.js +38 -0
- package/dist/brain/index.js.map +7 -0
- package/dist/brain/options.js +1 -0
- package/dist/brain/options.js.map +7 -0
- package/dist/brain/parse.js +102 -0
- package/dist/brain/parse.js.map +7 -0
- package/dist/brain/sse.js +25 -0
- package/dist/brain/sse.js.map +7 -0
- package/dist/brain/turn.js +135 -0
- package/dist/brain/turn.js.map +7 -0
- package/dist/models.js.map +2 -2
- package/dist/types/brain/call.d.ts +7 -0
- package/dist/types/brain/hash.d.ts +1 -0
- package/dist/types/brain/index.d.ts +3 -0
- package/dist/types/brain/options.d.ts +12 -0
- package/dist/types/brain/parse.d.ts +22 -0
- package/dist/types/brain/sse.d.ts +3 -0
- package/dist/types/brain/turn.d.ts +5 -0
- package/dist/types/models.d.ts +1 -1
- package/dist/types/wavespeed/catalog.d.ts +1 -0
- package/dist/types/wavespeed/index.d.ts +1 -0
- package/dist/types/wavespeed/speak.d.ts +4 -0
- package/dist/types/wavespeed/transcribe.d.ts +4 -0
- package/dist/types/wavespeed/wire.d.ts +25 -0
- package/dist/wavespeed/catalog.js +60 -0
- package/dist/wavespeed/catalog.js.map +7 -0
- package/dist/wavespeed/index.js +5 -0
- package/dist/wavespeed/index.js.map +7 -0
- package/dist/wavespeed/speak.js +66 -0
- package/dist/wavespeed/speak.js.map +7 -0
- package/dist/wavespeed/transcribe.js +85 -0
- package/dist/wavespeed/transcribe.js.map +7 -0
- package/dist/wavespeed/wire.js +138 -0
- package/dist/wavespeed/wire.js.map +7 -0
- package/package.json +9 -1
package/README.md
CHANGED
|
@@ -23,6 +23,8 @@ pi-audio does not apply that patch. Hosts such as slicc-agent record monkeypatch
|
|
|
23
23
|
|
|
24
24
|
`@ai-ecoverse/pi-audio/durable` appends a `voice.truncate` entry for a `VoiceCut`. The entry has no model messages. Its `edits` replace the target assistant message with the text heard so far and keep tool calls. The stored entry stays intact, and pi-durable derives the shorter message for the next model request. `@earendil-works/pi-durable` is an optional peer (`^1.1.0`); the root package does not import it. `VoiceCut.entryId` is the decimal string of pi-durable's numeric entry id.
|
|
25
25
|
|
|
26
|
+
`@ai-ecoverse/pi-audio/wavespeed` registers WaveSpeed adapters when `registerWaveSpeed()` is called. Both are batch only. `wavespeed-ai/openai-whisper` submits JSON to `/wavespeed-ai/openai-whisper` and polls `/predictions/{id}/result`. Timestamps stay off, so the catalog price is $0.001 per second (`perInputMinute` 0.06). A byte input is uploaded to `/media/upload/binary` first. `minimax/speech-2.6-turbo` is the same submit/poll path for MP3 (`textStreaming: false`, `streaming: false`) at $0.06 per thousand characters. `voice_id` is a caller id (`cloning: true`). These models are not for live voice.
|
|
27
|
+
|
|
26
28
|
`@ai-ecoverse/pi-audio/cloudflare` registers Workers AI speech adapters when `registerCloudflare()` is called. One Cloudflare token calls `POST /client/v4/accounts/{account_id}/ai/run/{model}`. Put the account in `baseUrl` or the `cf-account-id` header. Streaming models open `wss://gateway.ai.cloudflare.com/v1/{account}/{gateway}/workers-ai?model=…`. The gateway id is the `cf-aig-gateway-id` header, or `default`. `@cf/deepgram/nova-3` streams (`turnDetection: vad`, catalog price $0.0092 per minute). `@cf/deepgram/flux` streams with semantic end of turn ($0.0077 per minute) and has no batch transcribe. `@cf/openai/whisper-large-v3-turbo` is batch only ($0.000513 per minute). `@cf/deepgram/aura-1` ($0.015 per thousand characters), `aura-2-en`, and `aura-2-es` ($0.03) speak MP3 over HTTP and linear16 over the socket (`textStreaming: true`). `@cf/myshell-ai/melotts` is batch MP3 priced at $0.000205 per audio minute. `@cf/pipecat-ai/smart-turn-v2` is not a transcript model, so it is not registered.
|
|
27
29
|
|
|
28
30
|
`@ai-ecoverse/pi-audio/google` registers Gemini speech adapters when `registerGoogle()` is called. `gemini-3.8-flash-tts` and `gemini-3.8-flash-lite-tts` post JSON to `models/{id}:generateContent` with `responseModalities: ["AUDIO"]` and a prebuilt `voiceName`. Unary audio is WAV at 24 kHz unless the format asks for `AUDIO_L16`, `AUDIO_MULAW`, or `AUDIO_ALAW`. `textStreaming` is false. Output audio is priced from 25 tokens per second: $0.0135 per minute for Flash and $0.009 per minute for Flash-Lite through December 31, 2026. Text input is $0.50 per million tokens and is not on `audioCost`. Auth is `x-goog-api-key`.
|
|
@@ -62,6 +64,8 @@ pi-audio does not apply that patch. Hosts such as slicc-agent record monkeypatch
|
|
|
62
64
|
|
|
63
65
|
`@ai-ecoverse/pi-audio/fish-agents` registers `fish-agent` (`transports: ["webrtc"]`, `delivery.silent: "approximate"`, `delivery.interrupt: "native"`). `provisionAgent` creates client tools, writes the draft, and publishes it. `fishCustomLlm` sends `llm.custom.base_url`, `model`, and a raw `api_key`, and rejects `openai-responses`. A `RealtimeToken` with `url` is the LiveKit room from `mintRealtimeToken`; connect does not open a second session. Without a token, connect creates the session and fails on an unknown `transport`. Silent `inject` sends `user.message` with `audio: false` and drops that reply's transcript. The microphone is published after `openEchoMic`. The catalog `audioCost` is zero because Fish has not published a per-minute session rate. The example voice id is the one in Fish's quickstart.
|
|
64
66
|
|
|
67
|
+
`@ai-ecoverse/pi-audio/brain` exports `chatCompletionsBrain`, a fetch handler for a pi agent behind `CustomLlm.baseUrl`. The session key is Fish `session_id`, or `session_id` inside `fishaudio_extra_body`, `llm_extra_body`, `elevenlabs_extra_body`, or `custom_llm_extra_body` (a string extra body is the key). A hash of the streamed reply prompts only the new user text when it matches. The same user text with a different hash replays that stream. A new user text with a different hash is prompted as `The caller heard:` plus the vendor's copy. Final-answer text is streamed. Request tools are terminal `tool_calls` when `vendorTools` is `terminal`. `filler` is written when no text arrives within `firstTokenMs`. A turn still open at `hardDeadlineMs` ends as an error. Fish recommends a first token under 800 ms and caps a generation at 10 s.
|
|
68
|
+
|
|
65
69
|
`talkerTools` is `ask_agent` and `read`. pi-audio defines `askAgent` (a pi `Tool` whose parameters are `request` and optional `context`) and the `Thinker` interface. The host implements `Thinker` and supplies `read`. `bindThinker` runs direct tools with `directTimeoutMs` and truncates their text to `directMaxChars`. On a blocking `ask_agent` it answers with `ack` immediately and injects the result later. On an async call it `respond`s with the result and `options.result`. Progress is injected with `options.progress`. `tool_call_cancelled` aborts the thinker and any direct tool. A blocking failure is injected after the acknowledgement, with a `thinker_result` caption. `Thinker.ask` receives the optional `context` string. Closing the session aborts work that is still running. The binding also yields `thinker_progress` / `thinker_result` captions and `thinking` / `tool` / `idle` phases.
|
|
66
70
|
|
|
67
71
|
## Development
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
function isRecord(value) {
|
|
2
|
+
return typeof value === "object" && value !== null && !Array.isArray(value);
|
|
3
|
+
}
|
|
4
|
+
function contentBlocks(event) {
|
|
5
|
+
const direct = event.message?.content;
|
|
6
|
+
if (Array.isArray(direct)) return direct;
|
|
7
|
+
const model = event.entry?.model;
|
|
8
|
+
if (!Array.isArray(model) || model.length === 0) return [];
|
|
9
|
+
const first = model[0];
|
|
10
|
+
if (!isRecord(first) || !Array.isArray(first.content)) return [];
|
|
11
|
+
return first.content;
|
|
12
|
+
}
|
|
13
|
+
function callId(id, name) {
|
|
14
|
+
if (typeof id === "string" && id.length > 0) return id;
|
|
15
|
+
return name;
|
|
16
|
+
}
|
|
17
|
+
function vendorCall(event, tools) {
|
|
18
|
+
if (event.type === "tool_execution_start" && event.toolName && tools.has(event.toolName)) {
|
|
19
|
+
return {
|
|
20
|
+
id: callId(event.toolCallId, event.toolName),
|
|
21
|
+
name: event.toolName,
|
|
22
|
+
arguments: JSON.stringify(event.args ?? {})
|
|
23
|
+
};
|
|
24
|
+
}
|
|
25
|
+
for (const block of contentBlocks(event)) {
|
|
26
|
+
if (!isRecord(block) || block.type !== "toolCall" || typeof block.name !== "string") continue;
|
|
27
|
+
if (!tools.has(block.name)) continue;
|
|
28
|
+
const args = isRecord(block.arguments) ? block.arguments : {};
|
|
29
|
+
return { id: callId(block.id, block.name), name: block.name, arguments: JSON.stringify(args) };
|
|
30
|
+
}
|
|
31
|
+
return void 0;
|
|
32
|
+
}
|
|
33
|
+
export {
|
|
34
|
+
vendorCall
|
|
35
|
+
};
|
|
36
|
+
//# sourceMappingURL=call.js.map
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
{
|
|
2
|
+
"version": 3,
|
|
3
|
+
"sources": ["../../src/brain/call.ts"],
|
|
4
|
+
"sourcesContent": ["import type { VoiceAgentEvent } from '../spoken.ts';\n\nexport interface VendorCall {\n id: string;\n name: string;\n arguments: string;\n}\n\nfunction isRecord(value: unknown): value is Record<string, unknown> {\n return typeof value === 'object' && value !== null && !Array.isArray(value);\n}\n\nfunction contentBlocks(event: VoiceAgentEvent): readonly unknown[] {\n const direct = event.message?.content;\n if (Array.isArray(direct)) return direct;\n const model = event.entry?.model;\n if (!Array.isArray(model) || model.length === 0) return [];\n const first = model[0];\n if (!isRecord(first) || !Array.isArray(first.content)) return [];\n return first.content;\n}\n\nfunction callId(id: unknown, name: string): string {\n if (typeof id === 'string' && id.length > 0) return id;\n return name;\n}\n\nexport function vendorCall(event: VoiceAgentEvent, tools: Set<string>): VendorCall | undefined {\n if (event.type === 'tool_execution_start' && event.toolName && tools.has(event.toolName)) {\n return {\n id: callId(event.toolCallId, event.toolName),\n name: event.toolName,\n arguments: JSON.stringify(event.args ?? {}),\n };\n }\n for (const block of contentBlocks(event)) {\n if (!isRecord(block) || block.type !== 'toolCall' || typeof block.name !== 'string') continue;\n if (!tools.has(block.name)) continue;\n const args = isRecord(block.arguments) ? block.arguments : {};\n return { id: callId(block.id, block.name), name: block.name, arguments: JSON.stringify(args) };\n }\n return undefined;\n}\n"],
|
|
5
|
+
"mappings": "AAQA,SAAS,SAAS,OAAkD;AAClE,SAAO,OAAO,UAAU,YAAY,UAAU,QAAQ,CAAC,MAAM,QAAQ,KAAK;AAC5E;AAEA,SAAS,cAAc,OAA4C;AACjE,QAAM,SAAS,MAAM,SAAS;AAC9B,MAAI,MAAM,QAAQ,MAAM,EAAG,QAAO;AAClC,QAAM,QAAQ,MAAM,OAAO;AAC3B,MAAI,CAAC,MAAM,QAAQ,KAAK,KAAK,MAAM,WAAW,EAAG,QAAO,CAAC;AACzD,QAAM,QAAQ,MAAM,CAAC;AACrB,MAAI,CAAC,SAAS,KAAK,KAAK,CAAC,MAAM,QAAQ,MAAM,OAAO,EAAG,QAAO,CAAC;AAC/D,SAAO,MAAM;AACf;AAEA,SAAS,OAAO,IAAa,MAAsB;AACjD,MAAI,OAAO,OAAO,YAAY,GAAG,SAAS,EAAG,QAAO;AACpD,SAAO;AACT;AAEO,SAAS,WAAW,OAAwB,OAA4C;AAC7F,MAAI,MAAM,SAAS,0BAA0B,MAAM,YAAY,MAAM,IAAI,MAAM,QAAQ,GAAG;AACxF,WAAO;AAAA,MACL,IAAI,OAAO,MAAM,YAAY,MAAM,QAAQ;AAAA,MAC3C,MAAM,MAAM;AAAA,MACZ,WAAW,KAAK,UAAU,MAAM,QAAQ,CAAC,CAAC;AAAA,IAC5C;AAAA,EACF;AACA,aAAW,SAAS,cAAc,KAAK,GAAG;AACxC,QAAI,CAAC,SAAS,KAAK,KAAK,MAAM,SAAS,cAAc,OAAO,MAAM,SAAS,SAAU;AACrF,QAAI,CAAC,MAAM,IAAI,MAAM,IAAI,EAAG;AAC5B,UAAM,OAAO,SAAS,MAAM,SAAS,IAAI,MAAM,YAAY,CAAC;AAC5D,WAAO,EAAE,IAAI,OAAO,MAAM,IAAI,MAAM,IAAI,GAAG,MAAM,MAAM,MAAM,WAAW,KAAK,UAAU,IAAI,EAAE;AAAA,EAC/F;AACA,SAAO;AACT;",
|
|
6
|
+
"names": []
|
|
7
|
+
}
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
function replyHash(text) {
|
|
2
|
+
let hash = 2166136261;
|
|
3
|
+
for (let index = 0; index < text.length; index += 1) {
|
|
4
|
+
hash ^= text.charCodeAt(index);
|
|
5
|
+
hash = Math.imul(hash, 16777619);
|
|
6
|
+
}
|
|
7
|
+
return (hash >>> 0).toString(16).padStart(8, "0");
|
|
8
|
+
}
|
|
9
|
+
export {
|
|
10
|
+
replyHash
|
|
11
|
+
};
|
|
12
|
+
//# sourceMappingURL=hash.js.map
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
{
|
|
2
|
+
"version": 3,
|
|
3
|
+
"sources": ["../../src/brain/hash.ts"],
|
|
4
|
+
"sourcesContent": ["export function replyHash(text: string): string {\n let hash = 0x811c9dc5;\n for (let index = 0; index < text.length; index += 1) {\n hash ^= text.charCodeAt(index);\n hash = Math.imul(hash, 0x01000193);\n }\n return (hash >>> 0).toString(16).padStart(8, '0');\n}\n"],
|
|
5
|
+
"mappings": "AAAO,SAAS,UAAU,MAAsB;AAC9C,MAAI,OAAO;AACX,WAAS,QAAQ,GAAG,QAAQ,KAAK,QAAQ,SAAS,GAAG;AACnD,YAAQ,KAAK,WAAW,KAAK;AAC7B,WAAO,KAAK,KAAK,MAAM,QAAU;AAAA,EACnC;AACA,UAAQ,SAAS,GAAG,SAAS,EAAE,EAAE,SAAS,GAAG,GAAG;AAClD;",
|
|
6
|
+
"names": []
|
|
7
|
+
}
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
import { parseTurn, turnPlan } from "./parse.js";
|
|
2
|
+
import { runTurn } from "./turn.js";
|
|
3
|
+
function jsonError(message) {
|
|
4
|
+
return Response.json({ error: { message } }, { status: 400 });
|
|
5
|
+
}
|
|
6
|
+
async function readBody(request) {
|
|
7
|
+
try {
|
|
8
|
+
return await request.json();
|
|
9
|
+
} catch {
|
|
10
|
+
return void 0;
|
|
11
|
+
}
|
|
12
|
+
}
|
|
13
|
+
function chatCompletionsBrain(options) {
|
|
14
|
+
const sessions = /* @__PURE__ */ new Map();
|
|
15
|
+
return async (request) => {
|
|
16
|
+
const body = await readBody(request);
|
|
17
|
+
if (body === void 0) return jsonError("invalid chat completion");
|
|
18
|
+
const parsed = parseTurn(body);
|
|
19
|
+
if (typeof parsed === "string") return jsonError(parsed);
|
|
20
|
+
const plan = turnPlan(sessions.get(parsed.sessionKey), parsed);
|
|
21
|
+
const stream = new ReadableStream({
|
|
22
|
+
async start(controller) {
|
|
23
|
+
try {
|
|
24
|
+
await runTurn(options, sessions, parsed, plan, request, controller);
|
|
25
|
+
} finally {
|
|
26
|
+
controller.close();
|
|
27
|
+
}
|
|
28
|
+
}
|
|
29
|
+
});
|
|
30
|
+
return new Response(stream, {
|
|
31
|
+
headers: { "content-type": "text/event-stream", "cache-control": "no-cache" }
|
|
32
|
+
});
|
|
33
|
+
};
|
|
34
|
+
}
|
|
35
|
+
export {
|
|
36
|
+
chatCompletionsBrain
|
|
37
|
+
};
|
|
38
|
+
//# sourceMappingURL=index.js.map
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
{
|
|
2
|
+
"version": 3,
|
|
3
|
+
"sources": ["../../src/brain/index.ts"],
|
|
4
|
+
"sourcesContent": ["import type { BrainOptions } from './options.ts';\nimport type { Memory } from './parse.ts';\nimport { parseTurn, turnPlan } from './parse.ts';\nimport { runTurn } from './turn.ts';\n\nexport type { BrainBudget, BrainOptions } from './options.ts';\n\nfunction jsonError(message: string): Response {\n return Response.json({ error: { message } }, { status: 400 });\n}\n\nasync function readBody(request: Request): Promise<unknown> {\n try {\n return await request.json();\n } catch {\n return undefined;\n }\n}\n\nexport function chatCompletionsBrain(\n options: BrainOptions\n): (request: Request) => Promise<Response> {\n const sessions = new Map<string, Memory>();\n return async (request) => {\n const body = await readBody(request);\n if (body === undefined) return jsonError('invalid chat completion');\n const parsed = parseTurn(body);\n if (typeof parsed === 'string') return jsonError(parsed);\n const plan = turnPlan(sessions.get(parsed.sessionKey), parsed);\n const stream = new ReadableStream<Uint8Array>({\n async start(controller) {\n try {\n await runTurn(options, sessions, parsed, plan, request, controller);\n } finally {\n controller.close();\n }\n },\n });\n return new Response(stream, {\n headers: { 'content-type': 'text/event-stream', 'cache-control': 'no-cache' },\n });\n };\n}\n"],
|
|
5
|
+
"mappings": "AAEA,SAAS,WAAW,gBAAgB;AACpC,SAAS,eAAe;AAIxB,SAAS,UAAU,SAA2B;AAC5C,SAAO,SAAS,KAAK,EAAE,OAAO,EAAE,QAAQ,EAAE,GAAG,EAAE,QAAQ,IAAI,CAAC;AAC9D;AAEA,eAAe,SAAS,SAAoC;AAC1D,MAAI;AACF,WAAO,MAAM,QAAQ,KAAK;AAAA,EAC5B,QAAQ;AACN,WAAO;AAAA,EACT;AACF;AAEO,SAAS,qBACd,SACyC;AACzC,QAAM,WAAW,oBAAI,IAAoB;AACzC,SAAO,OAAO,YAAY;AACxB,UAAM,OAAO,MAAM,SAAS,OAAO;AACnC,QAAI,SAAS,OAAW,QAAO,UAAU,yBAAyB;AAClE,UAAM,SAAS,UAAU,IAAI;AAC7B,QAAI,OAAO,WAAW,SAAU,QAAO,UAAU,MAAM;AACvD,UAAM,OAAO,SAAS,SAAS,IAAI,OAAO,UAAU,GAAG,MAAM;AAC7D,UAAM,SAAS,IAAI,eAA2B;AAAA,MAC5C,MAAM,MAAM,YAAY;AACtB,YAAI;AACF,gBAAM,QAAQ,SAAS,UAAU,QAAQ,MAAM,SAAS,UAAU;AAAA,QACpE,UAAE;AACA,qBAAW,MAAM;AAAA,QACnB;AAAA,MACF;AAAA,IACF,CAAC;AACD,WAAO,IAAI,SAAS,QAAQ;AAAA,MAC1B,SAAS,EAAE,gBAAgB,qBAAqB,iBAAiB,WAAW;AAAA,IAC9E,CAAC;AAAA,EACH;AACF;",
|
|
6
|
+
"names": []
|
|
7
|
+
}
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
//# sourceMappingURL=options.js.map
|
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
import { replyHash } from "./hash.js";
|
|
2
|
+
const extraNames = [
|
|
3
|
+
"fishaudio_extra_body",
|
|
4
|
+
"llm_extra_body",
|
|
5
|
+
"elevenlabs_extra_body",
|
|
6
|
+
"custom_llm_extra_body"
|
|
7
|
+
];
|
|
8
|
+
function isRecord(value) {
|
|
9
|
+
return typeof value === "object" && value !== null && !Array.isArray(value);
|
|
10
|
+
}
|
|
11
|
+
function textOf(content) {
|
|
12
|
+
if (typeof content === "string") return content;
|
|
13
|
+
if (!Array.isArray(content)) return "";
|
|
14
|
+
let text = "";
|
|
15
|
+
for (const part of content) {
|
|
16
|
+
if (isRecord(part) && typeof part.text === "string") text += part.text;
|
|
17
|
+
}
|
|
18
|
+
return text;
|
|
19
|
+
}
|
|
20
|
+
function namedSession(extra) {
|
|
21
|
+
if (typeof extra === "string") return extra.length > 0 ? extra : void 0;
|
|
22
|
+
if (!isRecord(extra)) return void 0;
|
|
23
|
+
const id = extra.session_id;
|
|
24
|
+
if (typeof id === "string" && id.length > 0) return id;
|
|
25
|
+
return void 0;
|
|
26
|
+
}
|
|
27
|
+
function sessionOf(body) {
|
|
28
|
+
if (typeof body.session_id === "string" && body.session_id.length > 0) return body.session_id;
|
|
29
|
+
for (const name of extraNames) {
|
|
30
|
+
const key = namedSession(body[name]);
|
|
31
|
+
if (key !== void 0) return key;
|
|
32
|
+
}
|
|
33
|
+
return void 0;
|
|
34
|
+
}
|
|
35
|
+
function toolNames(body) {
|
|
36
|
+
const names = /* @__PURE__ */ new Set();
|
|
37
|
+
if (!Array.isArray(body.tools)) return names;
|
|
38
|
+
for (const tool of body.tools) {
|
|
39
|
+
if (!isRecord(tool)) continue;
|
|
40
|
+
const fn = tool.function;
|
|
41
|
+
if (!isRecord(fn)) continue;
|
|
42
|
+
if (typeof fn.name !== "string" || fn.name.length === 0) continue;
|
|
43
|
+
names.add(fn.name);
|
|
44
|
+
}
|
|
45
|
+
return names;
|
|
46
|
+
}
|
|
47
|
+
function lastTurn(messages) {
|
|
48
|
+
if (!Array.isArray(messages)) return void 0;
|
|
49
|
+
let prior;
|
|
50
|
+
let priorTools = false;
|
|
51
|
+
let user;
|
|
52
|
+
for (const message of messages) {
|
|
53
|
+
if (!isRecord(message) || message.role === "system") continue;
|
|
54
|
+
if (message.role === "assistant") {
|
|
55
|
+
prior = textOf(message.content);
|
|
56
|
+
priorTools = Array.isArray(message.tool_calls) && message.tool_calls.length > 0;
|
|
57
|
+
user = void 0;
|
|
58
|
+
continue;
|
|
59
|
+
}
|
|
60
|
+
if (message.role === "user" || message.role === "tool") user = textOf(message.content);
|
|
61
|
+
}
|
|
62
|
+
return { prior, priorTools, user };
|
|
63
|
+
}
|
|
64
|
+
function parseTurn(body) {
|
|
65
|
+
if (!isRecord(body)) return "invalid chat completion";
|
|
66
|
+
if (body.stream !== true) return "stream is required";
|
|
67
|
+
const sessionKey = sessionOf(body);
|
|
68
|
+
if (sessionKey === void 0) return "missing session id";
|
|
69
|
+
const turn = lastTurn(body.messages);
|
|
70
|
+
if (turn?.user === void 0) return "missing user turn";
|
|
71
|
+
const model = typeof body.model === "string" && body.model.length > 0 ? body.model : "brain";
|
|
72
|
+
return {
|
|
73
|
+
model,
|
|
74
|
+
sessionKey,
|
|
75
|
+
tools: toolNames(body),
|
|
76
|
+
prior: turn.prior,
|
|
77
|
+
priorTools: turn.priorTools,
|
|
78
|
+
user: turn.user
|
|
79
|
+
};
|
|
80
|
+
}
|
|
81
|
+
function sameReply(memory, prior) {
|
|
82
|
+
if (prior === void 0) return false;
|
|
83
|
+
return replyHash(prior) === memory.hash;
|
|
84
|
+
}
|
|
85
|
+
function turnPlan(memory, turn) {
|
|
86
|
+
if (memory === void 0) return { replay: false, content: turn.user };
|
|
87
|
+
const matches = sameReply(memory, turn.prior);
|
|
88
|
+
if (turn.user === memory.user && !matches && !turn.priorTools) {
|
|
89
|
+
return { replay: true, text: memory.text };
|
|
90
|
+
}
|
|
91
|
+
if (!matches && turn.prior !== void 0 && !turn.priorTools) {
|
|
92
|
+
return { replay: false, content: `The caller heard: ${turn.prior}
|
|
93
|
+
|
|
94
|
+
${turn.user}` };
|
|
95
|
+
}
|
|
96
|
+
return { replay: false, content: turn.user };
|
|
97
|
+
}
|
|
98
|
+
export {
|
|
99
|
+
parseTurn,
|
|
100
|
+
turnPlan
|
|
101
|
+
};
|
|
102
|
+
//# sourceMappingURL=parse.js.map
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
{
|
|
2
|
+
"version": 3,
|
|
3
|
+
"sources": ["../../src/brain/parse.ts"],
|
|
4
|
+
"sourcesContent": ["import { replyHash } from './hash.ts';\n\nexport interface Memory {\n hash: string;\n text: string;\n user: string;\n}\n\nexport interface ParsedTurn {\n model: string;\n sessionKey: string;\n tools: Set<string>;\n prior?: string;\n priorTools: boolean;\n user: string;\n}\n\nexport type TurnPlan = { replay: true; text: string } | { replay: false; content: string };\n\nconst extraNames = [\n 'fishaudio_extra_body',\n 'llm_extra_body',\n 'elevenlabs_extra_body',\n 'custom_llm_extra_body',\n] as const;\n\nfunction isRecord(value: unknown): value is Record<string, unknown> {\n return typeof value === 'object' && value !== null && !Array.isArray(value);\n}\n\nfunction textOf(content: unknown): string {\n if (typeof content === 'string') return content;\n if (!Array.isArray(content)) return '';\n let text = '';\n for (const part of content) {\n if (isRecord(part) && typeof part.text === 'string') text += part.text;\n }\n return text;\n}\n\nfunction namedSession(extra: unknown): string | undefined {\n if (typeof extra === 'string') return extra.length > 0 ? extra : undefined;\n if (!isRecord(extra)) return undefined;\n const id = extra.session_id;\n if (typeof id === 'string' && id.length > 0) return id;\n return undefined;\n}\n\nfunction sessionOf(body: Record<string, unknown>): string | undefined {\n if (typeof body.session_id === 'string' && body.session_id.length > 0) return body.session_id;\n for (const name of extraNames) {\n const key = namedSession(body[name]);\n if (key !== undefined) return key;\n }\n return undefined;\n}\n\nfunction toolNames(body: Record<string, unknown>): Set<string> {\n const names = new Set<string>();\n if (!Array.isArray(body.tools)) return names;\n for (const tool of body.tools) {\n if (!isRecord(tool)) continue;\n const fn = tool.function;\n if (!isRecord(fn)) continue;\n if (typeof fn.name !== 'string' || fn.name.length === 0) continue;\n names.add(fn.name);\n }\n return names;\n}\n\nfunction lastTurn(\n messages: unknown\n): { prior?: string; priorTools: boolean; user?: string } | undefined {\n if (!Array.isArray(messages)) return undefined;\n let prior: string | undefined;\n let priorTools = false;\n let user: string | undefined;\n for (const message of messages) {\n if (!isRecord(message) || message.role === 'system') continue;\n if (message.role === 'assistant') {\n prior = textOf(message.content);\n priorTools = Array.isArray(message.tool_calls) && message.tool_calls.length > 0;\n user = undefined;\n continue;\n }\n if (message.role === 'user' || message.role === 'tool') user = textOf(message.content);\n }\n return { prior, priorTools, user };\n}\n\nexport function parseTurn(body: unknown): ParsedTurn | string {\n if (!isRecord(body)) return 'invalid chat completion';\n if (body.stream !== true) return 'stream is required';\n const sessionKey = sessionOf(body);\n if (sessionKey === undefined) return 'missing session id';\n const turn = lastTurn(body.messages);\n if (turn?.user === undefined) return 'missing user turn';\n const model = typeof body.model === 'string' && body.model.length > 0 ? body.model : 'brain';\n return {\n model,\n sessionKey,\n tools: toolNames(body),\n prior: turn.prior,\n priorTools: turn.priorTools,\n user: turn.user,\n };\n}\n\nfunction sameReply(memory: Memory, prior: string | undefined): boolean {\n if (prior === undefined) return false;\n return replyHash(prior) === memory.hash;\n}\n\nexport function turnPlan(memory: Memory | undefined, turn: ParsedTurn): TurnPlan {\n if (memory === undefined) return { replay: false, content: turn.user };\n const matches = sameReply(memory, turn.prior);\n if (turn.user === memory.user && !matches && !turn.priorTools) {\n return { replay: true, text: memory.text };\n }\n if (!matches && turn.prior !== undefined && !turn.priorTools) {\n return { replay: false, content: `The caller heard: ${turn.prior}\\n\\n${turn.user}` };\n }\n return { replay: false, content: turn.user };\n}\n"],
|
|
5
|
+
"mappings": "AAAA,SAAS,iBAAiB;AAmB1B,MAAM,aAAa;AAAA,EACjB;AAAA,EACA;AAAA,EACA;AAAA,EACA;AACF;AAEA,SAAS,SAAS,OAAkD;AAClE,SAAO,OAAO,UAAU,YAAY,UAAU,QAAQ,CAAC,MAAM,QAAQ,KAAK;AAC5E;AAEA,SAAS,OAAO,SAA0B;AACxC,MAAI,OAAO,YAAY,SAAU,QAAO;AACxC,MAAI,CAAC,MAAM,QAAQ,OAAO,EAAG,QAAO;AACpC,MAAI,OAAO;AACX,aAAW,QAAQ,SAAS;AAC1B,QAAI,SAAS,IAAI,KAAK,OAAO,KAAK,SAAS,SAAU,SAAQ,KAAK;AAAA,EACpE;AACA,SAAO;AACT;AAEA,SAAS,aAAa,OAAoC;AACxD,MAAI,OAAO,UAAU,SAAU,QAAO,MAAM,SAAS,IAAI,QAAQ;AACjE,MAAI,CAAC,SAAS,KAAK,EAAG,QAAO;AAC7B,QAAM,KAAK,MAAM;AACjB,MAAI,OAAO,OAAO,YAAY,GAAG,SAAS,EAAG,QAAO;AACpD,SAAO;AACT;AAEA,SAAS,UAAU,MAAmD;AACpE,MAAI,OAAO,KAAK,eAAe,YAAY,KAAK,WAAW,SAAS,EAAG,QAAO,KAAK;AACnF,aAAW,QAAQ,YAAY;AAC7B,UAAM,MAAM,aAAa,KAAK,IAAI,CAAC;AACnC,QAAI,QAAQ,OAAW,QAAO;AAAA,EAChC;AACA,SAAO;AACT;AAEA,SAAS,UAAU,MAA4C;AAC7D,QAAM,QAAQ,oBAAI,IAAY;AAC9B,MAAI,CAAC,MAAM,QAAQ,KAAK,KAAK,EAAG,QAAO;AACvC,aAAW,QAAQ,KAAK,OAAO;AAC7B,QAAI,CAAC,SAAS,IAAI,EAAG;AACrB,UAAM,KAAK,KAAK;AAChB,QAAI,CAAC,SAAS,EAAE,EAAG;AACnB,QAAI,OAAO,GAAG,SAAS,YAAY,GAAG,KAAK,WAAW,EAAG;AACzD,UAAM,IAAI,GAAG,IAAI;AAAA,EACnB;AACA,SAAO;AACT;AAEA,SAAS,SACP,UACoE;AACpE,MAAI,CAAC,MAAM,QAAQ,QAAQ,EAAG,QAAO;AACrC,MAAI;AACJ,MAAI,aAAa;AACjB,MAAI;AACJ,aAAW,WAAW,UAAU;AAC9B,QAAI,CAAC,SAAS,OAAO,KAAK,QAAQ,SAAS,SAAU;AACrD,QAAI,QAAQ,SAAS,aAAa;AAChC,cAAQ,OAAO,QAAQ,OAAO;AAC9B,mBAAa,MAAM,QAAQ,QAAQ,UAAU,KAAK,QAAQ,WAAW,SAAS;AAC9E,aAAO;AACP;AAAA,IACF;AACA,QAAI,QAAQ,SAAS,UAAU,QAAQ,SAAS,OAAQ,QAAO,OAAO,QAAQ,OAAO;AAAA,EACvF;AACA,SAAO,EAAE,OAAO,YAAY,KAAK;AACnC;AAEO,SAAS,UAAU,MAAoC;AAC5D,MAAI,CAAC,SAAS,IAAI,EAAG,QAAO;AAC5B,MAAI,KAAK,WAAW,KAAM,QAAO;AACjC,QAAM,aAAa,UAAU,IAAI;AACjC,MAAI,eAAe,OAAW,QAAO;AACrC,QAAM,OAAO,SAAS,KAAK,QAAQ;AACnC,MAAI,MAAM,SAAS,OAAW,QAAO;AACrC,QAAM,QAAQ,OAAO,KAAK,UAAU,YAAY,KAAK,MAAM,SAAS,IAAI,KAAK,QAAQ;AACrF,SAAO;AAAA,IACL;AAAA,IACA;AAAA,IACA,OAAO,UAAU,IAAI;AAAA,IACrB,OAAO,KAAK;AAAA,IACZ,YAAY,KAAK;AAAA,IACjB,MAAM,KAAK;AAAA,EACb;AACF;AAEA,SAAS,UAAU,QAAgB,OAAoC;AACrE,MAAI,UAAU,OAAW,QAAO;AAChC,SAAO,UAAU,KAAK,MAAM,OAAO;AACrC;AAEO,SAAS,SAAS,QAA4B,MAA4B;AAC/E,MAAI,WAAW,OAAW,QAAO,EAAE,QAAQ,OAAO,SAAS,KAAK,KAAK;AACrE,QAAM,UAAU,UAAU,QAAQ,KAAK,KAAK;AAC5C,MAAI,KAAK,SAAS,OAAO,QAAQ,CAAC,WAAW,CAAC,KAAK,YAAY;AAC7D,WAAO,EAAE,QAAQ,MAAM,MAAM,OAAO,KAAK;AAAA,EAC3C;AACA,MAAI,CAAC,WAAW,KAAK,UAAU,UAAa,CAAC,KAAK,YAAY;AAC5D,WAAO,EAAE,QAAQ,OAAO,SAAS,qBAAqB,KAAK,KAAK;AAAA;AAAA,EAAO,KAAK,IAAI,GAAG;AAAA,EACrF;AACA,SAAO,EAAE,QAAQ,OAAO,SAAS,KAAK,KAAK;AAC7C;",
|
|
6
|
+
"names": []
|
|
7
|
+
}
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
function sseChunk(model, created, delta, finish) {
|
|
2
|
+
return `data: ${JSON.stringify({
|
|
3
|
+
id: "chatcmpl-brain",
|
|
4
|
+
object: "chat.completion.chunk",
|
|
5
|
+
created,
|
|
6
|
+
model,
|
|
7
|
+
choices: [{ index: 0, delta, finish_reason: finish }]
|
|
8
|
+
})}
|
|
9
|
+
|
|
10
|
+
`;
|
|
11
|
+
}
|
|
12
|
+
function sseDone() {
|
|
13
|
+
return "data: [DONE]\n\n";
|
|
14
|
+
}
|
|
15
|
+
function sseError(message) {
|
|
16
|
+
return `data: ${JSON.stringify({ error: { message } })}
|
|
17
|
+
|
|
18
|
+
`;
|
|
19
|
+
}
|
|
20
|
+
export {
|
|
21
|
+
sseChunk,
|
|
22
|
+
sseDone,
|
|
23
|
+
sseError
|
|
24
|
+
};
|
|
25
|
+
//# sourceMappingURL=sse.js.map
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
{
|
|
2
|
+
"version": 3,
|
|
3
|
+
"sources": ["../../src/brain/sse.ts"],
|
|
4
|
+
"sourcesContent": ["export function sseChunk(\n model: string,\n created: number,\n delta: Record<string, unknown>,\n finish: string | null\n): string {\n return `data: ${JSON.stringify({\n id: 'chatcmpl-brain',\n object: 'chat.completion.chunk',\n created,\n model,\n choices: [{ index: 0, delta, finish_reason: finish }],\n })}\\n\\n`;\n}\n\nexport function sseDone(): string {\n return 'data: [DONE]\\n\\n';\n}\n\nexport function sseError(message: string): string {\n return `data: ${JSON.stringify({ error: { message } })}\\n\\n`;\n}\n"],
|
|
5
|
+
"mappings": "AAAO,SAAS,SACd,OACA,SACA,OACA,QACQ;AACR,SAAO,SAAS,KAAK,UAAU;AAAA,IAC7B,IAAI;AAAA,IACJ,QAAQ;AAAA,IACR;AAAA,IACA;AAAA,IACA,SAAS,CAAC,EAAE,OAAO,GAAG,OAAO,eAAe,OAAO,CAAC;AAAA,EACtD,CAAC,CAAC;AAAA;AAAA;AACJ;AAEO,SAAS,UAAkB;AAChC,SAAO;AACT;AAEO,SAAS,SAAS,SAAyB;AAChD,SAAO,SAAS,KAAK,UAAU,EAAE,OAAO,EAAE,QAAQ,EAAE,CAAC,CAAC;AAAA;AAAA;AACxD;",
|
|
6
|
+
"names": []
|
|
7
|
+
}
|
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
import {
|
|
2
|
+
createSpokenState,
|
|
3
|
+
spokenPieces
|
|
4
|
+
} from "../spoken.js";
|
|
5
|
+
import { vendorCall } from "./call.js";
|
|
6
|
+
import { replyHash } from "./hash.js";
|
|
7
|
+
import { sseChunk, sseDone, sseError } from "./sse.js";
|
|
8
|
+
const encoder = new TextEncoder();
|
|
9
|
+
function sender(model, created, clock, controller) {
|
|
10
|
+
return (delta, finish2) => {
|
|
11
|
+
const body = clock.opened ? delta : { role: "assistant", ...delta };
|
|
12
|
+
clock.opened = true;
|
|
13
|
+
controller.enqueue(encoder.encode(sseChunk(model, created, body, finish2)));
|
|
14
|
+
};
|
|
15
|
+
}
|
|
16
|
+
function finish(send, controller, reason) {
|
|
17
|
+
send({}, reason);
|
|
18
|
+
controller.enqueue(encoder.encode(sseDone()));
|
|
19
|
+
}
|
|
20
|
+
function remember(sessions, parsed, clock) {
|
|
21
|
+
sessions.set(parsed.sessionKey, {
|
|
22
|
+
hash: replyHash(clock.streamed),
|
|
23
|
+
text: clock.streamed,
|
|
24
|
+
user: parsed.user
|
|
25
|
+
});
|
|
26
|
+
}
|
|
27
|
+
function writeReplay(parsed, text, sessions, controller) {
|
|
28
|
+
const clock = { cause: void 0, sent: false, streamed: text, opened: false };
|
|
29
|
+
const send = sender(parsed.model, Math.floor(Date.now() / 1e3), clock, controller);
|
|
30
|
+
if (text.length > 0) send({ content: text }, null);
|
|
31
|
+
finish(send, controller, "stop");
|
|
32
|
+
remember(sessions, parsed, clock);
|
|
33
|
+
}
|
|
34
|
+
function bindAbort(request, control, clock, hardMs) {
|
|
35
|
+
const fromClient = () => {
|
|
36
|
+
clock.cause = clock.cause ?? "client";
|
|
37
|
+
control.abort();
|
|
38
|
+
};
|
|
39
|
+
if (request.signal.aborted) fromClient();
|
|
40
|
+
else request.signal.addEventListener("abort", fromClient, { once: true });
|
|
41
|
+
const deadline = setTimeout(() => {
|
|
42
|
+
clock.cause = clock.cause ?? "deadline";
|
|
43
|
+
control.abort();
|
|
44
|
+
}, hardMs);
|
|
45
|
+
return () => {
|
|
46
|
+
clearTimeout(deadline);
|
|
47
|
+
request.signal.removeEventListener("abort", fromClient);
|
|
48
|
+
};
|
|
49
|
+
}
|
|
50
|
+
function armFiller(options, message, clock, send) {
|
|
51
|
+
const timer = setTimeout(() => {
|
|
52
|
+
if (clock.sent) return;
|
|
53
|
+
const text = options.filler(message);
|
|
54
|
+
if (text.length === 0) return;
|
|
55
|
+
clock.sent = true;
|
|
56
|
+
clock.streamed += text;
|
|
57
|
+
send({ content: text }, null);
|
|
58
|
+
}, options.budget.firstTokenMs);
|
|
59
|
+
return () => clearTimeout(timer);
|
|
60
|
+
}
|
|
61
|
+
function writeAnswers(event, state, clock, send) {
|
|
62
|
+
for (const piece of spokenPieces(event, state)) {
|
|
63
|
+
clock.sent = true;
|
|
64
|
+
clock.streamed += piece.text;
|
|
65
|
+
send({ content: piece.text }, null);
|
|
66
|
+
}
|
|
67
|
+
}
|
|
68
|
+
function writeTool(send, call) {
|
|
69
|
+
send(
|
|
70
|
+
{
|
|
71
|
+
tool_calls: [
|
|
72
|
+
{
|
|
73
|
+
index: 0,
|
|
74
|
+
id: call.id,
|
|
75
|
+
type: "function",
|
|
76
|
+
function: { name: call.name, arguments: call.arguments }
|
|
77
|
+
}
|
|
78
|
+
]
|
|
79
|
+
},
|
|
80
|
+
null
|
|
81
|
+
);
|
|
82
|
+
}
|
|
83
|
+
function stopFor(clock, controller) {
|
|
84
|
+
if (clock.cause === "deadline") controller.enqueue(encoder.encode(sseError("llm deadline")));
|
|
85
|
+
}
|
|
86
|
+
function errorText(error) {
|
|
87
|
+
if (error instanceof Error) return error.message;
|
|
88
|
+
return String(error);
|
|
89
|
+
}
|
|
90
|
+
async function liveTurn(options, sessions, parsed, content, request, controller) {
|
|
91
|
+
const clock = { cause: void 0, sent: false, streamed: "", opened: false };
|
|
92
|
+
const send = sender(parsed.model, Math.floor(Date.now() / 1e3), clock, controller);
|
|
93
|
+
const message = { role: "user", content, timestamp: Date.now() };
|
|
94
|
+
const control = new AbortController();
|
|
95
|
+
const releaseAbort = bindAbort(request, control, clock, options.budget.hardDeadlineMs);
|
|
96
|
+
const releaseFiller = armFiller(options, message, clock, send);
|
|
97
|
+
try {
|
|
98
|
+
if (clock.cause !== void 0) return stopFor(clock, controller);
|
|
99
|
+
const conversation = await options.conversation(parsed.sessionKey, control.signal);
|
|
100
|
+
if (clock.cause !== void 0) return stopFor(clock, controller);
|
|
101
|
+
const state = createSpokenState();
|
|
102
|
+
const meta = { source: "voice", transcriptionModel: parsed.model };
|
|
103
|
+
for await (const event of conversation.prompt(message, meta, control.signal)) {
|
|
104
|
+
if (clock.cause !== void 0) return stopFor(clock, controller);
|
|
105
|
+
writeAnswers(event, state, clock, send);
|
|
106
|
+
const call = vendorCall(event, parsed.tools);
|
|
107
|
+
if (call === void 0 || options.vendorTools === "drop") continue;
|
|
108
|
+
writeTool(send, call);
|
|
109
|
+
finish(send, controller, "tool_calls");
|
|
110
|
+
remember(sessions, parsed, clock);
|
|
111
|
+
return;
|
|
112
|
+
}
|
|
113
|
+
if (clock.cause !== void 0) return stopFor(clock, controller);
|
|
114
|
+
finish(send, controller, "stop");
|
|
115
|
+
remember(sessions, parsed, clock);
|
|
116
|
+
} catch (error) {
|
|
117
|
+
if (clock.cause === "client") return;
|
|
118
|
+
const text = clock.cause === "deadline" ? "llm deadline" : errorText(error);
|
|
119
|
+
controller.enqueue(encoder.encode(sseError(text)));
|
|
120
|
+
} finally {
|
|
121
|
+
releaseFiller();
|
|
122
|
+
releaseAbort();
|
|
123
|
+
}
|
|
124
|
+
}
|
|
125
|
+
async function runTurn(options, sessions, parsed, plan, request, controller) {
|
|
126
|
+
if (plan.replay) {
|
|
127
|
+
writeReplay(parsed, plan.text, sessions, controller);
|
|
128
|
+
return;
|
|
129
|
+
}
|
|
130
|
+
await liveTurn(options, sessions, parsed, plan.content, request, controller);
|
|
131
|
+
}
|
|
132
|
+
export {
|
|
133
|
+
runTurn
|
|
134
|
+
};
|
|
135
|
+
//# sourceMappingURL=turn.js.map
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
{
|
|
2
|
+
"version": 3,
|
|
3
|
+
"sources": ["../../src/brain/turn.ts"],
|
|
4
|
+
"sourcesContent": ["import type { UserMessage } from '@earendil-works/pi-ai';\nimport {\n createSpokenState,\n type SpokenState,\n spokenPieces,\n type VoiceAgentEvent,\n} from '../spoken.ts';\nimport { type VendorCall, vendorCall } from './call.ts';\nimport { replyHash } from './hash.ts';\nimport type { BrainOptions } from './options.ts';\nimport type { Memory, ParsedTurn, TurnPlan } from './parse.ts';\nimport { sseChunk, sseDone, sseError } from './sse.ts';\n\nconst encoder = new TextEncoder();\n\ninterface Clock {\n cause: 'client' | 'deadline' | undefined;\n sent: boolean;\n streamed: string;\n opened: boolean;\n}\n\ntype Send = (delta: Record<string, unknown>, finish: string | null) => void;\ntype Controller = ReadableStreamDefaultController<Uint8Array>;\n\nfunction sender(model: string, created: number, clock: Clock, controller: Controller): Send {\n return (delta, finish) => {\n const body = clock.opened ? delta : { role: 'assistant', ...delta };\n clock.opened = true;\n controller.enqueue(encoder.encode(sseChunk(model, created, body, finish)));\n };\n}\n\nfunction finish(send: Send, controller: Controller, reason: string): void {\n send({}, reason);\n controller.enqueue(encoder.encode(sseDone()));\n}\n\nfunction remember(sessions: Map<string, Memory>, parsed: ParsedTurn, clock: Clock): void {\n sessions.set(parsed.sessionKey, {\n hash: replyHash(clock.streamed),\n text: clock.streamed,\n user: parsed.user,\n });\n}\n\nfunction writeReplay(\n parsed: ParsedTurn,\n text: string,\n sessions: Map<string, Memory>,\n controller: Controller\n): void {\n const clock: Clock = { cause: undefined, sent: false, streamed: text, opened: false };\n const send = sender(parsed.model, Math.floor(Date.now() / 1000), clock, controller);\n if (text.length > 0) send({ content: text }, null);\n finish(send, controller, 'stop');\n remember(sessions, parsed, clock);\n}\n\nfunction bindAbort(\n request: Request,\n control: AbortController,\n clock: Clock,\n hardMs: number\n): () => void {\n const fromClient = () => {\n clock.cause = clock.cause ?? 'client';\n control.abort();\n };\n if (request.signal.aborted) fromClient();\n else request.signal.addEventListener('abort', fromClient, { once: true });\n const deadline = setTimeout(() => {\n clock.cause = clock.cause ?? 'deadline';\n control.abort();\n }, hardMs);\n return () => {\n clearTimeout(deadline);\n request.signal.removeEventListener('abort', fromClient);\n };\n}\n\nfunction armFiller(\n options: BrainOptions,\n message: UserMessage,\n clock: Clock,\n send: Send\n): () => void {\n const timer = setTimeout(() => {\n if (clock.sent) return;\n const text = options.filler(message);\n if (text.length === 0) return;\n clock.sent = true;\n clock.streamed += text;\n send({ content: text }, null);\n }, options.budget.firstTokenMs);\n return () => clearTimeout(timer);\n}\n\nfunction writeAnswers(event: VoiceAgentEvent, state: SpokenState, clock: Clock, send: Send): void {\n for (const piece of spokenPieces(event, state)) {\n clock.sent = true;\n clock.streamed += piece.text;\n send({ content: piece.text }, null);\n }\n}\n\nfunction writeTool(send: Send, call: VendorCall): void {\n send(\n {\n tool_calls: [\n {\n index: 0,\n id: call.id,\n type: 'function',\n function: { name: call.name, arguments: call.arguments },\n },\n ],\n },\n null\n );\n}\n\nfunction stopFor(clock: Clock, controller: Controller): void {\n if (clock.cause === 'deadline') controller.enqueue(encoder.encode(sseError('llm deadline')));\n}\n\nfunction errorText(error: unknown): string {\n if (error instanceof Error) return error.message;\n return String(error);\n}\n\nasync function liveTurn(\n options: BrainOptions,\n sessions: Map<string, Memory>,\n parsed: ParsedTurn,\n content: string,\n request: Request,\n controller: Controller\n): Promise<void> {\n const clock: Clock = { cause: undefined, sent: false, streamed: '', opened: false };\n const send = sender(parsed.model, Math.floor(Date.now() / 1000), clock, controller);\n const message: UserMessage = { role: 'user', content, timestamp: Date.now() };\n const control = new AbortController();\n const releaseAbort = bindAbort(request, control, clock, options.budget.hardDeadlineMs);\n const releaseFiller = armFiller(options, message, clock, send);\n try {\n if (clock.cause !== undefined) return stopFor(clock, controller);\n const conversation = await options.conversation(parsed.sessionKey, control.signal);\n if (clock.cause !== undefined) return stopFor(clock, controller);\n const state = createSpokenState();\n const meta = { source: 'voice' as const, transcriptionModel: parsed.model };\n for await (const event of conversation.prompt(message, meta, control.signal)) {\n if (clock.cause !== undefined) return stopFor(clock, controller);\n writeAnswers(event, state, clock, send);\n const call = vendorCall(event, parsed.tools);\n if (call === undefined || options.vendorTools === 'drop') continue;\n writeTool(send, call);\n finish(send, controller, 'tool_calls');\n remember(sessions, parsed, clock);\n return;\n }\n if (clock.cause !== undefined) return stopFor(clock, controller);\n finish(send, controller, 'stop');\n remember(sessions, parsed, clock);\n } catch (error) {\n if (clock.cause === 'client') return;\n const text = clock.cause === 'deadline' ? 'llm deadline' : errorText(error);\n controller.enqueue(encoder.encode(sseError(text)));\n } finally {\n releaseFiller();\n releaseAbort();\n }\n}\n\nexport async function runTurn(\n options: BrainOptions,\n sessions: Map<string, Memory>,\n parsed: ParsedTurn,\n plan: TurnPlan,\n request: Request,\n controller: Controller\n): Promise<void> {\n if (plan.replay) {\n writeReplay(parsed, plan.text, sessions, controller);\n return;\n }\n await liveTurn(options, sessions, parsed, plan.content, request, controller);\n}\n"],
|
|
5
|
+
"mappings": "AACA;AAAA,EACE;AAAA,EAEA;AAAA,OAEK;AACP,SAA0B,kBAAkB;AAC5C,SAAS,iBAAiB;AAG1B,SAAS,UAAU,SAAS,gBAAgB;AAE5C,MAAM,UAAU,IAAI,YAAY;AAYhC,SAAS,OAAO,OAAe,SAAiB,OAAc,YAA8B;AAC1F,SAAO,CAAC,OAAOA,YAAW;AACxB,UAAM,OAAO,MAAM,SAAS,QAAQ,EAAE,MAAM,aAAa,GAAG,MAAM;AAClE,UAAM,SAAS;AACf,eAAW,QAAQ,QAAQ,OAAO,SAAS,OAAO,SAAS,MAAMA,OAAM,CAAC,CAAC;AAAA,EAC3E;AACF;AAEA,SAAS,OAAO,MAAY,YAAwB,QAAsB;AACxE,OAAK,CAAC,GAAG,MAAM;AACf,aAAW,QAAQ,QAAQ,OAAO,QAAQ,CAAC,CAAC;AAC9C;AAEA,SAAS,SAAS,UAA+B,QAAoB,OAAoB;AACvF,WAAS,IAAI,OAAO,YAAY;AAAA,IAC9B,MAAM,UAAU,MAAM,QAAQ;AAAA,IAC9B,MAAM,MAAM;AAAA,IACZ,MAAM,OAAO;AAAA,EACf,CAAC;AACH;AAEA,SAAS,YACP,QACA,MACA,UACA,YACM;AACN,QAAM,QAAe,EAAE,OAAO,QAAW,MAAM,OAAO,UAAU,MAAM,QAAQ,MAAM;AACpF,QAAM,OAAO,OAAO,OAAO,OAAO,KAAK,MAAM,KAAK,IAAI,IAAI,GAAI,GAAG,OAAO,UAAU;AAClF,MAAI,KAAK,SAAS,EAAG,MAAK,EAAE,SAAS,KAAK,GAAG,IAAI;AACjD,SAAO,MAAM,YAAY,MAAM;AAC/B,WAAS,UAAU,QAAQ,KAAK;AAClC;AAEA,SAAS,UACP,SACA,SACA,OACA,QACY;AACZ,QAAM,aAAa,MAAM;AACvB,UAAM,QAAQ,MAAM,SAAS;AAC7B,YAAQ,MAAM;AAAA,EAChB;AACA,MAAI,QAAQ,OAAO,QAAS,YAAW;AAAA,MAClC,SAAQ,OAAO,iBAAiB,SAAS,YAAY,EAAE,MAAM,KAAK,CAAC;AACxE,QAAM,WAAW,WAAW,MAAM;AAChC,UAAM,QAAQ,MAAM,SAAS;AAC7B,YAAQ,MAAM;AAAA,EAChB,GAAG,MAAM;AACT,SAAO,MAAM;AACX,iBAAa,QAAQ;AACrB,YAAQ,OAAO,oBAAoB,SAAS,UAAU;AAAA,EACxD;AACF;AAEA,SAAS,UACP,SACA,SACA,OACA,MACY;AACZ,QAAM,QAAQ,WAAW,MAAM;AAC7B,QAAI,MAAM,KAAM;AAChB,UAAM,OAAO,QAAQ,OAAO,OAAO;AACnC,QAAI,KAAK,WAAW,EAAG;AACvB,UAAM,OAAO;AACb,UAAM,YAAY;AAClB,SAAK,EAAE,SAAS,KAAK,GAAG,IAAI;AAAA,EAC9B,GAAG,QAAQ,OAAO,YAAY;AAC9B,SAAO,MAAM,aAAa,KAAK;AACjC;AAEA,SAAS,aAAa,OAAwB,OAAoB,OAAc,MAAkB;AAChG,aAAW,SAAS,aAAa,OAAO,KAAK,GAAG;AAC9C,UAAM,OAAO;AACb,UAAM,YAAY,MAAM;AACxB,SAAK,EAAE,SAAS,MAAM,KAAK,GAAG,IAAI;AAAA,EACpC;AACF;AAEA,SAAS,UAAU,MAAY,MAAwB;AACrD;AAAA,IACE;AAAA,MACE,YAAY;AAAA,QACV;AAAA,UACE,OAAO;AAAA,UACP,IAAI,KAAK;AAAA,UACT,MAAM;AAAA,UACN,UAAU,EAAE,MAAM,KAAK,MAAM,WAAW,KAAK,UAAU;AAAA,QACzD;AAAA,MACF;AAAA,IACF;AAAA,IACA;AAAA,EACF;AACF;AAEA,SAAS,QAAQ,OAAc,YAA8B;AAC3D,MAAI,MAAM,UAAU,WAAY,YAAW,QAAQ,QAAQ,OAAO,SAAS,cAAc,CAAC,CAAC;AAC7F;AAEA,SAAS,UAAU,OAAwB;AACzC,MAAI,iBAAiB,MAAO,QAAO,MAAM;AACzC,SAAO,OAAO,KAAK;AACrB;AAEA,eAAe,SACb,SACA,UACA,QACA,SACA,SACA,YACe;AACf,QAAM,QAAe,EAAE,OAAO,QAAW,MAAM,OAAO,UAAU,IAAI,QAAQ,MAAM;AAClF,QAAM,OAAO,OAAO,OAAO,OAAO,KAAK,MAAM,KAAK,IAAI,IAAI,GAAI,GAAG,OAAO,UAAU;AAClF,QAAM,UAAuB,EAAE,MAAM,QAAQ,SAAS,WAAW,KAAK,IAAI,EAAE;AAC5E,QAAM,UAAU,IAAI,gBAAgB;AACpC,QAAM,eAAe,UAAU,SAAS,SAAS,OAAO,QAAQ,OAAO,cAAc;AACrF,QAAM,gBAAgB,UAAU,SAAS,SAAS,OAAO,IAAI;AAC7D,MAAI;AACF,QAAI,MAAM,UAAU,OAAW,QAAO,QAAQ,OAAO,UAAU;AAC/D,UAAM,eAAe,MAAM,QAAQ,aAAa,OAAO,YAAY,QAAQ,MAAM;AACjF,QAAI,MAAM,UAAU,OAAW,QAAO,QAAQ,OAAO,UAAU;AAC/D,UAAM,QAAQ,kBAAkB;AAChC,UAAM,OAAO,EAAE,QAAQ,SAAkB,oBAAoB,OAAO,MAAM;AAC1E,qBAAiB,SAAS,aAAa,OAAO,SAAS,MAAM,QAAQ,MAAM,GAAG;AAC5E,UAAI,MAAM,UAAU,OAAW,QAAO,QAAQ,OAAO,UAAU;AAC/D,mBAAa,OAAO,OAAO,OAAO,IAAI;AACtC,YAAM,OAAO,WAAW,OAAO,OAAO,KAAK;AAC3C,UAAI,SAAS,UAAa,QAAQ,gBAAgB,OAAQ;AAC1D,gBAAU,MAAM,IAAI;AACpB,aAAO,MAAM,YAAY,YAAY;AACrC,eAAS,UAAU,QAAQ,KAAK;AAChC;AAAA,IACF;AACA,QAAI,MAAM,UAAU,OAAW,QAAO,QAAQ,OAAO,UAAU;AAC/D,WAAO,MAAM,YAAY,MAAM;AAC/B,aAAS,UAAU,QAAQ,KAAK;AAAA,EAClC,SAAS,OAAO;AACd,QAAI,MAAM,UAAU,SAAU;AAC9B,UAAM,OAAO,MAAM,UAAU,aAAa,iBAAiB,UAAU,KAAK;AAC1E,eAAW,QAAQ,QAAQ,OAAO,SAAS,IAAI,CAAC,CAAC;AAAA,EACnD,UAAE;AACA,kBAAc;AACd,iBAAa;AAAA,EACf;AACF;AAEA,eAAsB,QACpB,SACA,UACA,QACA,MACA,SACA,YACe;AACf,MAAI,KAAK,QAAQ;AACf,gBAAY,QAAQ,KAAK,MAAM,UAAU,UAAU;AACnD;AAAA,EACF;AACA,QAAM,SAAS,SAAS,UAAU,QAAQ,KAAK,SAAS,SAAS,UAAU;AAC7E;",
|
|
6
|
+
"names": ["finish"]
|
|
7
|
+
}
|
package/dist/models.js.map
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"version": 3,
|
|
3
3
|
"sources": ["../src/models.ts"],
|
|
4
|
-
"sourcesContent": ["import type { BaseModel } from '@earendil-works/pi-ai';\nimport type { AudioFormat } from './audio.ts';\nimport type { RealtimeAudioCost, SpeechAudioCost, TranscriptionAudioCost } from './usage.ts';\n\nexport type KnownTranscriptionApi =\n | 'openai-transcriptions'\n | 'openai-realtime-transcription'\n | 'xai-stt'\n | 'elevenlabs-scribe'\n | 'mistral-transcriptions'\n | 'fish-asr'\n | 'cloudflare-ai-run';\n\nexport type TranscriptionApi = KnownTranscriptionApi | (string & {});\n\nexport type KnownSpeechApi =\n | 'openai-speech'\n | 'xai-tts'\n | 'elevenlabs-tts'\n | 'mistral-speech'\n | 'fish-tts'\n | 'google-tts'\n | 'cloudflare-ai-run'\n | 'wavespeed';\n\nexport type SpeechApi = KnownSpeechApi | (string & {});\n\nexport type KnownRealtimeApi =\n | 'openai-realtime'\n | 'xai-realtime'\n | 'gemini-live'\n | 'elevenlabs-agents'\n | 'fish-agents';\n\nexport type RealtimeApi = KnownRealtimeApi | (string & {});\n\nexport type Delivery = 'interrupt' | 'when_idle' | 'silent';\n\nexport type DeliverySupport = 'native' | 'approximate' | 'unsupported';\n\nexport interface VoiceInfo {\n id: string;\n name: string;\n languages?: string[];\n}\n\nexport interface TranscriptionModel<TApi extends TranscriptionApi>\n extends Omit<BaseModel<TApi>, 'input'> {\n type: 'transcription';\n streaming: boolean;\n turnDetection: 'none' | 'vad' | 'semantic';\n inputFormats: AudioFormat[];\n languages?: string[];\n diarization?: boolean;\n audioCost: TranscriptionAudioCost;\n}\n\nexport interface SpeechModel<TApi extends SpeechApi> extends Omit<BaseModel<TApi>, 'input'> {\n type: 'speech';\n textStreaming: boolean;\n outputFormats: AudioFormat[];\n voices?: VoiceInfo[];\n cloning?: boolean;\n audioCost: SpeechAudioCost;\n}\n\nexport interface RealtimeModel<TApi extends RealtimeApi> extends Omit<BaseModel<TApi>, 'input'> {\n type: 'realtime';\n transports: ('websocket' | 'webrtc')[];\n toolCalls: 'blocking' | 'async';\n inputFormat: AudioFormat;\n outputFormat: AudioFormat;\n video?: { maxFps: number };\n maxSessionMs?: number;\n resumable: boolean;\n ephemeralAuth: boolean;\n voices: VoiceInfo[];\n cascade?: { stt: string; llm: string; tts: string };\n provisioning: 'none' | 'agent';\n delivery: Record<Delivery, DeliverySupport>;\n truncation: 'client' | 'server' | 'none';\n liveToolUpdate: boolean;\n toolTimeoutMs?: { min: number; max: number };\n audioCost: RealtimeAudioCost;\n}\n\ndeclare module '@earendil-works/pi-ai' {\n interface ModelTypeMap {\n transcription: TranscriptionModel<TranscriptionApi>;\n speech: SpeechModel<SpeechApi>;\n realtime: RealtimeModel<RealtimeApi>;\n }\n}\n\nexport type VoiceModel =\n | TranscriptionModel<TranscriptionApi>\n | SpeechModel<SpeechApi>\n | RealtimeModel<RealtimeApi>;\n\nexport type VoiceModelType = VoiceModel['type'];\n\nconst catalog = new Map<string, VoiceModel>();\n\nfunction catalogKey(type: string, provider: string, id: string): string {\n return `${type}\\0${provider}\\0${id}`;\n}\n\nexport function registerVoiceModel(model: VoiceModel): () => void {\n const key = catalogKey(model.type, model.provider, model.id);\n catalog.set(key, model);\n return () => {\n if (catalog.get(key) === model) catalog.delete(key);\n };\n}\n\nexport function getVoiceModel<TType extends VoiceModelType>(\n type: TType,\n provider: string,\n id: string\n): Extract<VoiceModel, { type: TType }> | undefined {\n return catalog.get(catalogKey(type, provider, id)) as\n | Extract<VoiceModel, { type: TType }>\n | undefined;\n}\n\nexport function listVoiceModels<TType extends VoiceModelType>(\n type: TType,\n provider?: string\n): Extract<VoiceModel, { type: TType }>[] {\n const found: Extract<VoiceModel, { type: TType }>[] = [];\n for (const model of catalog.values()) {\n if (model.type !== type) continue;\n if (provider !== undefined && model.provider !== provider) continue;\n found.push(model as Extract<VoiceModel, { type: TType }>);\n }\n return found;\n}\n"],
|
|
5
|
-
"mappings": "
|
|
4
|
+
"sourcesContent": ["import type { BaseModel } from '@earendil-works/pi-ai';\nimport type { AudioFormat } from './audio.ts';\nimport type { RealtimeAudioCost, SpeechAudioCost, TranscriptionAudioCost } from './usage.ts';\n\nexport type KnownTranscriptionApi =\n | 'openai-transcriptions'\n | 'openai-realtime-transcription'\n | 'xai-stt'\n | 'elevenlabs-scribe'\n | 'mistral-transcriptions'\n | 'fish-asr'\n | 'cloudflare-ai-run'\n | 'wavespeed';\n\nexport type TranscriptionApi = KnownTranscriptionApi | (string & {});\n\nexport type KnownSpeechApi =\n | 'openai-speech'\n | 'xai-tts'\n | 'elevenlabs-tts'\n | 'mistral-speech'\n | 'fish-tts'\n | 'google-tts'\n | 'cloudflare-ai-run'\n | 'wavespeed';\n\nexport type SpeechApi = KnownSpeechApi | (string & {});\n\nexport type KnownRealtimeApi =\n | 'openai-realtime'\n | 'xai-realtime'\n | 'gemini-live'\n | 'elevenlabs-agents'\n | 'fish-agents';\n\nexport type RealtimeApi = KnownRealtimeApi | (string & {});\n\nexport type Delivery = 'interrupt' | 'when_idle' | 'silent';\n\nexport type DeliverySupport = 'native' | 'approximate' | 'unsupported';\n\nexport interface VoiceInfo {\n id: string;\n name: string;\n languages?: string[];\n}\n\nexport interface TranscriptionModel<TApi extends TranscriptionApi>\n extends Omit<BaseModel<TApi>, 'input'> {\n type: 'transcription';\n streaming: boolean;\n turnDetection: 'none' | 'vad' | 'semantic';\n inputFormats: AudioFormat[];\n languages?: string[];\n diarization?: boolean;\n audioCost: TranscriptionAudioCost;\n}\n\nexport interface SpeechModel<TApi extends SpeechApi> extends Omit<BaseModel<TApi>, 'input'> {\n type: 'speech';\n textStreaming: boolean;\n outputFormats: AudioFormat[];\n voices?: VoiceInfo[];\n cloning?: boolean;\n audioCost: SpeechAudioCost;\n}\n\nexport interface RealtimeModel<TApi extends RealtimeApi> extends Omit<BaseModel<TApi>, 'input'> {\n type: 'realtime';\n transports: ('websocket' | 'webrtc')[];\n toolCalls: 'blocking' | 'async';\n inputFormat: AudioFormat;\n outputFormat: AudioFormat;\n video?: { maxFps: number };\n maxSessionMs?: number;\n resumable: boolean;\n ephemeralAuth: boolean;\n voices: VoiceInfo[];\n cascade?: { stt: string; llm: string; tts: string };\n provisioning: 'none' | 'agent';\n delivery: Record<Delivery, DeliverySupport>;\n truncation: 'client' | 'server' | 'none';\n liveToolUpdate: boolean;\n toolTimeoutMs?: { min: number; max: number };\n audioCost: RealtimeAudioCost;\n}\n\ndeclare module '@earendil-works/pi-ai' {\n interface ModelTypeMap {\n transcription: TranscriptionModel<TranscriptionApi>;\n speech: SpeechModel<SpeechApi>;\n realtime: RealtimeModel<RealtimeApi>;\n }\n}\n\nexport type VoiceModel =\n | TranscriptionModel<TranscriptionApi>\n | SpeechModel<SpeechApi>\n | RealtimeModel<RealtimeApi>;\n\nexport type VoiceModelType = VoiceModel['type'];\n\nconst catalog = new Map<string, VoiceModel>();\n\nfunction catalogKey(type: string, provider: string, id: string): string {\n return `${type}\\0${provider}\\0${id}`;\n}\n\nexport function registerVoiceModel(model: VoiceModel): () => void {\n const key = catalogKey(model.type, model.provider, model.id);\n catalog.set(key, model);\n return () => {\n if (catalog.get(key) === model) catalog.delete(key);\n };\n}\n\nexport function getVoiceModel<TType extends VoiceModelType>(\n type: TType,\n provider: string,\n id: string\n): Extract<VoiceModel, { type: TType }> | undefined {\n return catalog.get(catalogKey(type, provider, id)) as\n | Extract<VoiceModel, { type: TType }>\n | undefined;\n}\n\nexport function listVoiceModels<TType extends VoiceModelType>(\n type: TType,\n provider?: string\n): Extract<VoiceModel, { type: TType }>[] {\n const found: Extract<VoiceModel, { type: TType }>[] = [];\n for (const model of catalog.values()) {\n if (model.type !== type) continue;\n if (provider !== undefined && model.provider !== provider) continue;\n found.push(model as Extract<VoiceModel, { type: TType }>);\n }\n return found;\n}\n"],
|
|
5
|
+
"mappings": "AAsGA,MAAM,UAAU,oBAAI,IAAwB;AAE5C,SAAS,WAAW,MAAc,UAAkB,IAAoB;AACtE,SAAO,GAAG,IAAI,KAAK,QAAQ,KAAK,EAAE;AACpC;AAEO,SAAS,mBAAmB,OAA+B;AAChE,QAAM,MAAM,WAAW,MAAM,MAAM,MAAM,UAAU,MAAM,EAAE;AAC3D,UAAQ,IAAI,KAAK,KAAK;AACtB,SAAO,MAAM;AACX,QAAI,QAAQ,IAAI,GAAG,MAAM,MAAO,SAAQ,OAAO,GAAG;AAAA,EACpD;AACF;AAEO,SAAS,cACd,MACA,UACA,IACkD;AAClD,SAAO,QAAQ,IAAI,WAAW,MAAM,UAAU,EAAE,CAAC;AAGnD;AAEO,SAAS,gBACd,MACA,UACwC;AACxC,QAAM,QAAgD,CAAC;AACvD,aAAW,SAAS,QAAQ,OAAO,GAAG;AACpC,QAAI,MAAM,SAAS,KAAM;AACzB,QAAI,aAAa,UAAa,MAAM,aAAa,SAAU;AAC3D,UAAM,KAAK,KAA6C;AAAA,EAC1D;AACA,SAAO;AACT;",
|
|
6
6
|
"names": []
|
|
7
7
|
}
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
export declare function replyHash(text: string): string;
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
import type { UserMessage } from '@earendil-works/pi-ai';
|
|
2
|
+
import type { VoiceConversation } from '../turns.ts';
|
|
3
|
+
export interface BrainBudget {
|
|
4
|
+
firstTokenMs: number;
|
|
5
|
+
hardDeadlineMs: number;
|
|
6
|
+
}
|
|
7
|
+
export interface BrainOptions {
|
|
8
|
+
conversation(sessionKey: string, signal: AbortSignal): Promise<VoiceConversation>;
|
|
9
|
+
vendorTools: 'terminal' | 'drop';
|
|
10
|
+
budget: BrainBudget;
|
|
11
|
+
filler(user: UserMessage): string;
|
|
12
|
+
}
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
export interface Memory {
|
|
2
|
+
hash: string;
|
|
3
|
+
text: string;
|
|
4
|
+
user: string;
|
|
5
|
+
}
|
|
6
|
+
export interface ParsedTurn {
|
|
7
|
+
model: string;
|
|
8
|
+
sessionKey: string;
|
|
9
|
+
tools: Set<string>;
|
|
10
|
+
prior?: string;
|
|
11
|
+
priorTools: boolean;
|
|
12
|
+
user: string;
|
|
13
|
+
}
|
|
14
|
+
export type TurnPlan = {
|
|
15
|
+
replay: true;
|
|
16
|
+
text: string;
|
|
17
|
+
} | {
|
|
18
|
+
replay: false;
|
|
19
|
+
content: string;
|
|
20
|
+
};
|
|
21
|
+
export declare function parseTurn(body: unknown): ParsedTurn | string;
|
|
22
|
+
export declare function turnPlan(memory: Memory | undefined, turn: ParsedTurn): TurnPlan;
|
|
@@ -0,0 +1,5 @@
|
|
|
1
|
+
import type { BrainOptions } from './options.ts';
|
|
2
|
+
import type { Memory, ParsedTurn, TurnPlan } from './parse.ts';
|
|
3
|
+
type Controller = ReadableStreamDefaultController<Uint8Array>;
|
|
4
|
+
export declare function runTurn(options: BrainOptions, sessions: Map<string, Memory>, parsed: ParsedTurn, plan: TurnPlan, request: Request, controller: Controller): Promise<void>;
|
|
5
|
+
export {};
|
package/dist/types/models.d.ts
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import type { BaseModel } from '@earendil-works/pi-ai';
|
|
2
2
|
import type { AudioFormat } from './audio.ts';
|
|
3
3
|
import type { RealtimeAudioCost, SpeechAudioCost, TranscriptionAudioCost } from './usage.ts';
|
|
4
|
-
export type KnownTranscriptionApi = 'openai-transcriptions' | 'openai-realtime-transcription' | 'xai-stt' | 'elevenlabs-scribe' | 'mistral-transcriptions' | 'fish-asr' | 'cloudflare-ai-run';
|
|
4
|
+
export type KnownTranscriptionApi = 'openai-transcriptions' | 'openai-realtime-transcription' | 'xai-stt' | 'elevenlabs-scribe' | 'mistral-transcriptions' | 'fish-asr' | 'cloudflare-ai-run' | 'wavespeed';
|
|
5
5
|
export type TranscriptionApi = KnownTranscriptionApi | (string & {});
|
|
6
6
|
export type KnownSpeechApi = 'openai-speech' | 'xai-tts' | 'elevenlabs-tts' | 'mistral-speech' | 'fish-tts' | 'google-tts' | 'cloudflare-ai-run' | 'wavespeed';
|
|
7
7
|
export type SpeechApi = KnownSpeechApi | (string & {});
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
export declare function registerWaveSpeed(): () => void;
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
export { registerWaveSpeed } from './catalog.ts';
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
import type { ProviderRequestOptions } from '@earendil-works/pi-ai';
|
|
2
|
+
import type { SpeechModel } from '../models.ts';
|
|
3
|
+
import { SpeechEventStream, type SpeechInput } from '../speech.ts';
|
|
4
|
+
export declare function speakWaveSpeed(model: SpeechModel<string>, input: SpeechInput, options?: ProviderRequestOptions): SpeechEventStream;
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
import type { ProviderRequestOptions } from '@earendil-works/pi-ai';
|
|
2
|
+
import type { TranscriptionModel } from '../models.ts';
|
|
3
|
+
import type { AudioInput, Transcript } from '../transcription.ts';
|
|
4
|
+
export declare function transcribeWaveSpeed(model: TranscriptionModel<string>, input: AudioInput, options?: ProviderRequestOptions): Promise<Transcript>;
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
import type { ProviderRequestOptions } from '@earendil-works/pi-ai';
|
|
2
|
+
export declare function taskRecord(body: unknown): Record<string, unknown>;
|
|
3
|
+
export declare function errorText(text: string, status: number): string;
|
|
4
|
+
export declare function isHttpUrl(value: string): boolean;
|
|
5
|
+
export declare function outputText(output: unknown): string | undefined;
|
|
6
|
+
export declare function outputUrl(output: unknown): string | undefined;
|
|
7
|
+
export declare function downloadUrl(body: unknown): string;
|
|
8
|
+
export declare function durationSeconds(record: Record<string, unknown>): number;
|
|
9
|
+
declare function waveFetch(model: {
|
|
10
|
+
provider: string;
|
|
11
|
+
headers?: Record<string, string>;
|
|
12
|
+
id: string;
|
|
13
|
+
}, options: ProviderRequestOptions | undefined, url: string, body: BodyInit | undefined, contentType: string | undefined): Promise<unknown>;
|
|
14
|
+
export declare function finishTask(model: {
|
|
15
|
+
provider: string;
|
|
16
|
+
baseUrl: string;
|
|
17
|
+
headers?: Record<string, string>;
|
|
18
|
+
id: string;
|
|
19
|
+
}, options: ProviderRequestOptions | undefined, payload: unknown, rejected: string): Promise<Record<string, unknown>>;
|
|
20
|
+
export declare function readHttp(model: {
|
|
21
|
+
provider: string;
|
|
22
|
+
headers?: Record<string, string>;
|
|
23
|
+
id: string;
|
|
24
|
+
}, options: ProviderRequestOptions | undefined, url: string): Promise<Response>;
|
|
25
|
+
export { waveFetch };
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
import { registerVoiceModel } from "../models.js";
|
|
2
|
+
import { registerSpeechApiProvider, registerTranscriptionApiProvider } from "../registry.js";
|
|
3
|
+
import { speakWaveSpeed } from "./speak.js";
|
|
4
|
+
import { transcribeWaveSpeed } from "./transcribe.js";
|
|
5
|
+
const tokenCost = { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 };
|
|
6
|
+
const wavespeed = "https://api.wavespeed.ai/api/v3";
|
|
7
|
+
const format = (encoding, sampleRate) => ({
|
|
8
|
+
encoding,
|
|
9
|
+
sampleRate,
|
|
10
|
+
channels: 1
|
|
11
|
+
});
|
|
12
|
+
function registerWaveSpeed() {
|
|
13
|
+
const whisper = {
|
|
14
|
+
id: "wavespeed-ai/openai-whisper",
|
|
15
|
+
name: "wavespeed-ai/openai-whisper",
|
|
16
|
+
api: "wavespeed",
|
|
17
|
+
provider: "wavespeed",
|
|
18
|
+
baseUrl: wavespeed,
|
|
19
|
+
type: "transcription",
|
|
20
|
+
streaming: false,
|
|
21
|
+
turnDetection: "none",
|
|
22
|
+
inputFormats: [format("wav", 16e3), format("mp3", 16e3), format("flac", 16e3)],
|
|
23
|
+
cost: tokenCost,
|
|
24
|
+
audioCost: { perInputMinute: 0.06 }
|
|
25
|
+
};
|
|
26
|
+
const speech = {
|
|
27
|
+
id: "minimax/speech-2.6-turbo",
|
|
28
|
+
name: "minimax/speech-2.6-turbo",
|
|
29
|
+
api: "wavespeed",
|
|
30
|
+
provider: "wavespeed",
|
|
31
|
+
baseUrl: wavespeed,
|
|
32
|
+
type: "speech",
|
|
33
|
+
textStreaming: false,
|
|
34
|
+
outputFormats: [format("mp3", 24e3)],
|
|
35
|
+
cloning: true,
|
|
36
|
+
cost: tokenCost,
|
|
37
|
+
audioCost: { perThousandChars: 0.06 }
|
|
38
|
+
};
|
|
39
|
+
const unregisterModels = [registerVoiceModel(whisper), registerVoiceModel(speech)];
|
|
40
|
+
const unregisterTranscription = registerTranscriptionApiProvider({
|
|
41
|
+
api: "wavespeed",
|
|
42
|
+
transcribe: transcribeWaveSpeed
|
|
43
|
+
});
|
|
44
|
+
const unregisterSpeech = registerSpeechApiProvider({
|
|
45
|
+
api: "wavespeed",
|
|
46
|
+
speak: speakWaveSpeed
|
|
47
|
+
});
|
|
48
|
+
let released = false;
|
|
49
|
+
return () => {
|
|
50
|
+
if (released) return;
|
|
51
|
+
released = true;
|
|
52
|
+
for (const unregister of unregisterModels) unregister();
|
|
53
|
+
unregisterTranscription();
|
|
54
|
+
unregisterSpeech();
|
|
55
|
+
};
|
|
56
|
+
}
|
|
57
|
+
export {
|
|
58
|
+
registerWaveSpeed
|
|
59
|
+
};
|
|
60
|
+
//# sourceMappingURL=catalog.js.map
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
{
|
|
2
|
+
"version": 3,
|
|
3
|
+
"sources": ["../../src/wavespeed/catalog.ts"],
|
|
4
|
+
"sourcesContent": ["import type { AudioFormat } from '../audio.ts';\nimport { registerVoiceModel, type SpeechModel, type TranscriptionModel } from '../models.ts';\nimport { registerSpeechApiProvider, registerTranscriptionApiProvider } from '../registry.ts';\nimport { speakWaveSpeed } from './speak.ts';\nimport { transcribeWaveSpeed } from './transcribe.ts';\n\nconst tokenCost = { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 };\nconst wavespeed = 'https://api.wavespeed.ai/api/v3';\n\nconst format = (encoding: 'wav' | 'mp3' | 'flac', sampleRate: number): AudioFormat => ({\n encoding,\n sampleRate,\n channels: 1,\n});\n\nexport function registerWaveSpeed(): () => void {\n const whisper: TranscriptionModel<string> = {\n id: 'wavespeed-ai/openai-whisper',\n name: 'wavespeed-ai/openai-whisper',\n api: 'wavespeed',\n provider: 'wavespeed',\n baseUrl: wavespeed,\n type: 'transcription',\n streaming: false,\n turnDetection: 'none',\n inputFormats: [format('wav', 16000), format('mp3', 16000), format('flac', 16000)],\n cost: tokenCost,\n audioCost: { perInputMinute: 0.06 },\n };\n const speech: SpeechModel<string> = {\n id: 'minimax/speech-2.6-turbo',\n name: 'minimax/speech-2.6-turbo',\n api: 'wavespeed',\n provider: 'wavespeed',\n baseUrl: wavespeed,\n type: 'speech',\n textStreaming: false,\n outputFormats: [format('mp3', 24000)],\n cloning: true,\n cost: tokenCost,\n audioCost: { perThousandChars: 0.06 },\n };\n const unregisterModels = [registerVoiceModel(whisper), registerVoiceModel(speech)];\n const unregisterTranscription = registerTranscriptionApiProvider({\n api: 'wavespeed',\n transcribe: transcribeWaveSpeed,\n });\n const unregisterSpeech = registerSpeechApiProvider({\n api: 'wavespeed',\n speak: speakWaveSpeed,\n });\n let released = false;\n return () => {\n if (released) return;\n released = true;\n for (const unregister of unregisterModels) unregister();\n unregisterTranscription();\n unregisterSpeech();\n };\n}\n"],
|
|
5
|
+
"mappings": "AACA,SAAS,0BAAqE;AAC9E,SAAS,2BAA2B,wCAAwC;AAC5E,SAAS,sBAAsB;AAC/B,SAAS,2BAA2B;AAEpC,MAAM,YAAY,EAAE,OAAO,GAAG,QAAQ,GAAG,WAAW,GAAG,YAAY,EAAE;AACrE,MAAM,YAAY;AAElB,MAAM,SAAS,CAAC,UAAkC,gBAAqC;AAAA,EACrF;AAAA,EACA;AAAA,EACA,UAAU;AACZ;AAEO,SAAS,oBAAgC;AAC9C,QAAM,UAAsC;AAAA,IAC1C,IAAI;AAAA,IACJ,MAAM;AAAA,IACN,KAAK;AAAA,IACL,UAAU;AAAA,IACV,SAAS;AAAA,IACT,MAAM;AAAA,IACN,WAAW;AAAA,IACX,eAAe;AAAA,IACf,cAAc,CAAC,OAAO,OAAO,IAAK,GAAG,OAAO,OAAO,IAAK,GAAG,OAAO,QAAQ,IAAK,CAAC;AAAA,IAChF,MAAM;AAAA,IACN,WAAW,EAAE,gBAAgB,KAAK;AAAA,EACpC;AACA,QAAM,SAA8B;AAAA,IAClC,IAAI;AAAA,IACJ,MAAM;AAAA,IACN,KAAK;AAAA,IACL,UAAU;AAAA,IACV,SAAS;AAAA,IACT,MAAM;AAAA,IACN,eAAe;AAAA,IACf,eAAe,CAAC,OAAO,OAAO,IAAK,CAAC;AAAA,IACpC,SAAS;AAAA,IACT,MAAM;AAAA,IACN,WAAW,EAAE,kBAAkB,KAAK;AAAA,EACtC;AACA,QAAM,mBAAmB,CAAC,mBAAmB,OAAO,GAAG,mBAAmB,MAAM,CAAC;AACjF,QAAM,0BAA0B,iCAAiC;AAAA,IAC/D,KAAK;AAAA,IACL,YAAY;AAAA,EACd,CAAC;AACD,QAAM,mBAAmB,0BAA0B;AAAA,IACjD,KAAK;AAAA,IACL,OAAO;AAAA,EACT,CAAC;AACD,MAAI,WAAW;AACf,SAAO,MAAM;AACX,QAAI,SAAU;AACd,eAAW;AACX,eAAW,cAAc,iBAAkB,YAAW;AACtD,4BAAwB;AACxB,qBAAiB;AAAA,EACnB;AACF;",
|
|
6
|
+
"names": []
|
|
7
|
+
}
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
import { asVoiceError } from "../openai/wire.js";
|
|
2
|
+
import { SpeechEventStream } from "../speech.js";
|
|
3
|
+
import { priceAudio } from "../usage.js";
|
|
4
|
+
import { finishTask, outputUrl, readHttp } from "./wire.js";
|
|
5
|
+
function resolveFormat(model, requested) {
|
|
6
|
+
const fallback = model.outputFormats[0];
|
|
7
|
+
if (!fallback) throw new Error("No output format");
|
|
8
|
+
if (!requested) return fallback;
|
|
9
|
+
const match = model.outputFormats.find(
|
|
10
|
+
(format) => format.encoding === requested.encoding && format.sampleRate === requested.sampleRate && format.channels === requested.channels
|
|
11
|
+
);
|
|
12
|
+
if (!match) throw new Error(`Unsupported speech format ${requested.encoding}`);
|
|
13
|
+
return match;
|
|
14
|
+
}
|
|
15
|
+
async function readText(text, signal) {
|
|
16
|
+
if (typeof text === "string") return text;
|
|
17
|
+
let combined = "";
|
|
18
|
+
for await (const part of text) {
|
|
19
|
+
if (signal?.aborted) throw new DOMException("aborted", "AbortError");
|
|
20
|
+
combined += part;
|
|
21
|
+
}
|
|
22
|
+
return combined;
|
|
23
|
+
}
|
|
24
|
+
async function audioOf(model, options, output) {
|
|
25
|
+
const url = outputUrl(output);
|
|
26
|
+
if (url === void 0) throw new Error("Speech response had no audio");
|
|
27
|
+
const response = await readHttp(model, options, url);
|
|
28
|
+
const data = new Uint8Array(await response.arrayBuffer());
|
|
29
|
+
if (data.byteLength === 0) throw new Error("Speech response had no audio");
|
|
30
|
+
return data;
|
|
31
|
+
}
|
|
32
|
+
function speakWaveSpeed(model, input, options) {
|
|
33
|
+
const stream = new SpeechEventStream();
|
|
34
|
+
void pumpSpeech(model, input, options, stream);
|
|
35
|
+
return stream;
|
|
36
|
+
}
|
|
37
|
+
async function pumpSpeech(model, input, options, stream) {
|
|
38
|
+
try {
|
|
39
|
+
if (options?.signal?.aborted) throw new DOMException("aborted", "AbortError");
|
|
40
|
+
const format = resolveFormat(model, input.format);
|
|
41
|
+
const text = await readText(input.text, options?.signal);
|
|
42
|
+
const payload = { text, voice_id: input.voice, format: "mp3" };
|
|
43
|
+
if (input.language !== void 0) payload.language_boost = input.language;
|
|
44
|
+
const record = await finishTask(model, options, payload, "Speech payload was not JSON");
|
|
45
|
+
const outputs = record.outputs;
|
|
46
|
+
const first = Array.isArray(outputs) ? outputs[0] : void 0;
|
|
47
|
+
const data = await audioOf(model, options, first);
|
|
48
|
+
stream.push({ type: "start", format });
|
|
49
|
+
stream.push({
|
|
50
|
+
type: "audio",
|
|
51
|
+
chunk: { data, format, timestamp: 0 },
|
|
52
|
+
span: { itemId: model.id, start: 0, end: text.length }
|
|
53
|
+
});
|
|
54
|
+
stream.push({
|
|
55
|
+
type: "done",
|
|
56
|
+
usage: priceAudio(model.audioCost, { characters: text.length })
|
|
57
|
+
});
|
|
58
|
+
} catch (error) {
|
|
59
|
+
const voice = asVoiceError(error);
|
|
60
|
+
stream.push({ type: "error", reason: voice.reason, error: voice.error });
|
|
61
|
+
}
|
|
62
|
+
}
|
|
63
|
+
export {
|
|
64
|
+
speakWaveSpeed
|
|
65
|
+
};
|
|
66
|
+
//# sourceMappingURL=speak.js.map
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
{
|
|
2
|
+
"version": 3,
|
|
3
|
+
"sources": ["../../src/wavespeed/speak.ts"],
|
|
4
|
+
"sourcesContent": ["import type { ProviderRequestOptions } from '@earendil-works/pi-ai';\nimport type { AudioFormat } from '../audio.ts';\nimport type { SpeechModel } from '../models.ts';\nimport { asVoiceError } from '../openai/wire.ts';\nimport { SpeechEventStream, type SpeechInput } from '../speech.ts';\nimport { priceAudio } from '../usage.ts';\nimport { finishTask, outputUrl, readHttp } from './wire.ts';\n\nfunction resolveFormat(\n model: SpeechModel<string>,\n requested: AudioFormat | undefined\n): AudioFormat {\n const fallback = model.outputFormats[0];\n if (!fallback) throw new Error('No output format');\n if (!requested) return fallback;\n const match = model.outputFormats.find(\n (format) =>\n format.encoding === requested.encoding &&\n format.sampleRate === requested.sampleRate &&\n format.channels === requested.channels\n );\n if (!match) throw new Error(`Unsupported speech format ${requested.encoding}`);\n return match;\n}\n\nasync function readText(\n text: string | AsyncIterable<string>,\n signal: AbortSignal | undefined\n): Promise<string> {\n if (typeof text === 'string') return text;\n let combined = '';\n for await (const part of text) {\n if (signal?.aborted) throw new DOMException('aborted', 'AbortError');\n combined += part;\n }\n return combined;\n}\n\nasync function audioOf(\n model: SpeechModel<string>,\n options: ProviderRequestOptions | undefined,\n output: unknown\n): Promise<Uint8Array> {\n const url = outputUrl(output);\n if (url === undefined) throw new Error('Speech response had no audio');\n const response = await readHttp(model, options, url);\n const data = new Uint8Array(await response.arrayBuffer());\n if (data.byteLength === 0) throw new Error('Speech response had no audio');\n return data;\n}\n\nexport function speakWaveSpeed(\n model: SpeechModel<string>,\n input: SpeechInput,\n options?: ProviderRequestOptions\n): SpeechEventStream {\n const stream = new SpeechEventStream();\n void pumpSpeech(model, input, options, stream);\n return stream;\n}\n\nasync function pumpSpeech(\n model: SpeechModel<string>,\n input: SpeechInput,\n options: ProviderRequestOptions | undefined,\n stream: SpeechEventStream\n) {\n try {\n if (options?.signal?.aborted) throw new DOMException('aborted', 'AbortError');\n const format = resolveFormat(model, input.format);\n const text = await readText(input.text, options?.signal);\n const payload: Record<string, unknown> = { text, voice_id: input.voice, format: 'mp3' };\n if (input.language !== undefined) payload.language_boost = input.language;\n const record = await finishTask(model, options, payload, 'Speech payload was not JSON');\n const outputs = record.outputs;\n const first = Array.isArray(outputs) ? outputs[0] : undefined;\n const data = await audioOf(model, options, first);\n stream.push({ type: 'start', format });\n stream.push({\n type: 'audio',\n chunk: { data, format, timestamp: 0 },\n span: { itemId: model.id, start: 0, end: text.length },\n });\n stream.push({\n type: 'done',\n usage: priceAudio(model.audioCost, { characters: text.length }),\n });\n } catch (error) {\n const voice = asVoiceError(error);\n stream.push({ type: 'error', reason: voice.reason, error: voice.error });\n }\n}\n"],
|
|
5
|
+
"mappings": "AAGA,SAAS,oBAAoB;AAC7B,SAAS,yBAA2C;AACpD,SAAS,kBAAkB;AAC3B,SAAS,YAAY,WAAW,gBAAgB;AAEhD,SAAS,cACP,OACA,WACa;AACb,QAAM,WAAW,MAAM,cAAc,CAAC;AACtC,MAAI,CAAC,SAAU,OAAM,IAAI,MAAM,kBAAkB;AACjD,MAAI,CAAC,UAAW,QAAO;AACvB,QAAM,QAAQ,MAAM,cAAc;AAAA,IAChC,CAAC,WACC,OAAO,aAAa,UAAU,YAC9B,OAAO,eAAe,UAAU,cAChC,OAAO,aAAa,UAAU;AAAA,EAClC;AACA,MAAI,CAAC,MAAO,OAAM,IAAI,MAAM,6BAA6B,UAAU,QAAQ,EAAE;AAC7E,SAAO;AACT;AAEA,eAAe,SACb,MACA,QACiB;AACjB,MAAI,OAAO,SAAS,SAAU,QAAO;AACrC,MAAI,WAAW;AACf,mBAAiB,QAAQ,MAAM;AAC7B,QAAI,QAAQ,QAAS,OAAM,IAAI,aAAa,WAAW,YAAY;AACnE,gBAAY;AAAA,EACd;AACA,SAAO;AACT;AAEA,eAAe,QACb,OACA,SACA,QACqB;AACrB,QAAM,MAAM,UAAU,MAAM;AAC5B,MAAI,QAAQ,OAAW,OAAM,IAAI,MAAM,8BAA8B;AACrE,QAAM,WAAW,MAAM,SAAS,OAAO,SAAS,GAAG;AACnD,QAAM,OAAO,IAAI,WAAW,MAAM,SAAS,YAAY,CAAC;AACxD,MAAI,KAAK,eAAe,EAAG,OAAM,IAAI,MAAM,8BAA8B;AACzE,SAAO;AACT;AAEO,SAAS,eACd,OACA,OACA,SACmB;AACnB,QAAM,SAAS,IAAI,kBAAkB;AACrC,OAAK,WAAW,OAAO,OAAO,SAAS,MAAM;AAC7C,SAAO;AACT;AAEA,eAAe,WACb,OACA,OACA,SACA,QACA;AACA,MAAI;AACF,QAAI,SAAS,QAAQ,QAAS,OAAM,IAAI,aAAa,WAAW,YAAY;AAC5E,UAAM,SAAS,cAAc,OAAO,MAAM,MAAM;AAChD,UAAM,OAAO,MAAM,SAAS,MAAM,MAAM,SAAS,MAAM;AACvD,UAAM,UAAmC,EAAE,MAAM,UAAU,MAAM,OAAO,QAAQ,MAAM;AACtF,QAAI,MAAM,aAAa,OAAW,SAAQ,iBAAiB,MAAM;AACjE,UAAM,SAAS,MAAM,WAAW,OAAO,SAAS,SAAS,6BAA6B;AACtF,UAAM,UAAU,OAAO;AACvB,UAAM,QAAQ,MAAM,QAAQ,OAAO,IAAI,QAAQ,CAAC,IAAI;AACpD,UAAM,OAAO,MAAM,QAAQ,OAAO,SAAS,KAAK;AAChD,WAAO,KAAK,EAAE,MAAM,SAAS,OAAO,CAAC;AACrC,WAAO,KAAK;AAAA,MACV,MAAM;AAAA,MACN,OAAO,EAAE,MAAM,QAAQ,WAAW,EAAE;AAAA,MACpC,MAAM,EAAE,QAAQ,MAAM,IAAI,OAAO,GAAG,KAAK,KAAK,OAAO;AAAA,IACvD,CAAC;AACD,WAAO,KAAK;AAAA,MACV,MAAM;AAAA,MACN,OAAO,WAAW,MAAM,WAAW,EAAE,YAAY,KAAK,OAAO,CAAC;AAAA,IAChE,CAAC;AAAA,EACH,SAAS,OAAO;AACd,UAAM,QAAQ,aAAa,KAAK;AAChC,WAAO,KAAK,EAAE,MAAM,SAAS,QAAQ,MAAM,QAAQ,OAAO,MAAM,MAAM,CAAC;AAAA,EACzE;AACF;",
|
|
6
|
+
"names": []
|
|
7
|
+
}
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
import { VoiceRequestError } from "../errors.js";
|
|
2
|
+
import {
|
|
3
|
+
asVoiceError,
|
|
4
|
+
audioFile,
|
|
5
|
+
encodeMultipart,
|
|
6
|
+
endpoint,
|
|
7
|
+
multipartType
|
|
8
|
+
} from "../openai/wire.js";
|
|
9
|
+
import { priceAudio } from "../usage.js";
|
|
10
|
+
import {
|
|
11
|
+
downloadUrl,
|
|
12
|
+
durationSeconds,
|
|
13
|
+
finishTask,
|
|
14
|
+
isHttpUrl,
|
|
15
|
+
outputText,
|
|
16
|
+
readHttp,
|
|
17
|
+
waveFetch
|
|
18
|
+
} from "./wire.js";
|
|
19
|
+
async function sourceUrl(model, input, options) {
|
|
20
|
+
if ("url" in input) return input.url;
|
|
21
|
+
const named = audioFile(input.mimeType);
|
|
22
|
+
const encoded = encodeMultipart(
|
|
23
|
+
{ data: input.data, mimeType: input.mimeType, filename: named.filename, format: named.format },
|
|
24
|
+
{}
|
|
25
|
+
);
|
|
26
|
+
const copy = new Uint8Array(encoded.byteLength);
|
|
27
|
+
copy.set(encoded);
|
|
28
|
+
const parsed = await waveFetch(
|
|
29
|
+
model,
|
|
30
|
+
options,
|
|
31
|
+
endpoint(model.baseUrl, "media/upload/binary"),
|
|
32
|
+
copy.buffer,
|
|
33
|
+
multipartType()
|
|
34
|
+
);
|
|
35
|
+
return downloadUrl(parsed);
|
|
36
|
+
}
|
|
37
|
+
async function textOf(model, options, output) {
|
|
38
|
+
if (typeof output === "string" && isHttpUrl(output)) {
|
|
39
|
+
const response = await readHttp(model, options, output);
|
|
40
|
+
const text2 = await response.text();
|
|
41
|
+
if (text2 === "") throw new Error("Transcription response had no text");
|
|
42
|
+
return text2;
|
|
43
|
+
}
|
|
44
|
+
const text = outputText(output);
|
|
45
|
+
if (text === void 0 || text === "" || isHttpUrl(text)) {
|
|
46
|
+
throw new Error("Transcription response had no text");
|
|
47
|
+
}
|
|
48
|
+
return text;
|
|
49
|
+
}
|
|
50
|
+
function languageOf(options) {
|
|
51
|
+
const record = options;
|
|
52
|
+
if (record === void 0) return void 0;
|
|
53
|
+
if (typeof record.language !== "string") return void 0;
|
|
54
|
+
return record.language;
|
|
55
|
+
}
|
|
56
|
+
async function transcribeWaveSpeed(model, input, options) {
|
|
57
|
+
try {
|
|
58
|
+
if (options?.signal?.aborted) throw new DOMException("aborted", "AbortError");
|
|
59
|
+
const audio = await sourceUrl(model, input, options);
|
|
60
|
+
const payload = {
|
|
61
|
+
audio,
|
|
62
|
+
task: "transcribe",
|
|
63
|
+
enable_timestamps: false
|
|
64
|
+
};
|
|
65
|
+
const language = languageOf(options);
|
|
66
|
+
if (language !== void 0) payload.language = language;
|
|
67
|
+
const record = await finishTask(model, options, payload, "Transcription payload was not JSON");
|
|
68
|
+
const outputs = record.outputs;
|
|
69
|
+
const first = Array.isArray(outputs) ? outputs[0] : void 0;
|
|
70
|
+
const text = await textOf(model, options, first);
|
|
71
|
+
const meters = {};
|
|
72
|
+
const seconds = durationSeconds(record);
|
|
73
|
+
if (seconds > 0) meters.inputSeconds = seconds;
|
|
74
|
+
const transcript = { text, usage: priceAudio(model.audioCost, meters) };
|
|
75
|
+
if (language !== void 0) transcript.language = language;
|
|
76
|
+
return transcript;
|
|
77
|
+
} catch (error) {
|
|
78
|
+
const voice = asVoiceError(error);
|
|
79
|
+
throw new VoiceRequestError(voice.reason, voice.error);
|
|
80
|
+
}
|
|
81
|
+
}
|
|
82
|
+
export {
|
|
83
|
+
transcribeWaveSpeed
|
|
84
|
+
};
|
|
85
|
+
//# sourceMappingURL=transcribe.js.map
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
{
|
|
2
|
+
"version": 3,
|
|
3
|
+
"sources": ["../../src/wavespeed/transcribe.ts"],
|
|
4
|
+
"sourcesContent": ["import type { ProviderRequestOptions } from '@earendil-works/pi-ai';\nimport { VoiceRequestError } from '../errors.ts';\nimport type { TranscriptionModel } from '../models.ts';\nimport {\n asVoiceError,\n audioFile,\n encodeMultipart,\n endpoint,\n multipartType,\n} from '../openai/wire.ts';\nimport type { AudioInput, Transcript } from '../transcription.ts';\nimport { type AudioCounters, priceAudio } from '../usage.ts';\nimport {\n downloadUrl,\n durationSeconds,\n finishTask,\n isHttpUrl,\n outputText,\n readHttp,\n waveFetch,\n} from './wire.ts';\n\nasync function sourceUrl(\n model: TranscriptionModel<string>,\n input: AudioInput,\n options: ProviderRequestOptions | undefined\n): Promise<string> {\n if ('url' in input) return input.url;\n const named = audioFile(input.mimeType);\n const encoded = encodeMultipart(\n { data: input.data, mimeType: input.mimeType, filename: named.filename, format: named.format },\n {}\n );\n const copy = new Uint8Array(encoded.byteLength);\n copy.set(encoded);\n const parsed = await waveFetch(\n model,\n options,\n endpoint(model.baseUrl, 'media/upload/binary'),\n copy.buffer,\n multipartType()\n );\n return downloadUrl(parsed);\n}\n\nasync function textOf(\n model: TranscriptionModel<string>,\n options: ProviderRequestOptions | undefined,\n output: unknown\n): Promise<string> {\n if (typeof output === 'string' && isHttpUrl(output)) {\n const response = await readHttp(model, options, output);\n const text = await response.text();\n if (text === '') throw new Error('Transcription response had no text');\n return text;\n }\n const text = outputText(output);\n if (text === undefined || text === '' || isHttpUrl(text)) {\n throw new Error('Transcription response had no text');\n }\n return text;\n}\n\nfunction languageOf(options: ProviderRequestOptions | undefined): string | undefined {\n const record = options as { language?: unknown } | undefined;\n if (record === undefined) return undefined;\n if (typeof record.language !== 'string') return undefined;\n return record.language;\n}\n\nexport async function transcribeWaveSpeed(\n model: TranscriptionModel<string>,\n input: AudioInput,\n options?: ProviderRequestOptions\n): Promise<Transcript> {\n try {\n if (options?.signal?.aborted) throw new DOMException('aborted', 'AbortError');\n const audio = await sourceUrl(model, input, options);\n const payload: Record<string, unknown> = {\n audio,\n task: 'transcribe',\n enable_timestamps: false,\n };\n const language = languageOf(options);\n if (language !== undefined) payload.language = language;\n const record = await finishTask(model, options, payload, 'Transcription payload was not JSON');\n const outputs = record.outputs;\n const first = Array.isArray(outputs) ? outputs[0] : undefined;\n const text = await textOf(model, options, first);\n const meters: AudioCounters = {};\n const seconds = durationSeconds(record);\n if (seconds > 0) meters.inputSeconds = seconds;\n const transcript: Transcript = { text, usage: priceAudio(model.audioCost, meters) };\n if (language !== undefined) transcript.language = language;\n return transcript;\n } catch (error) {\n const voice = asVoiceError(error);\n throw new VoiceRequestError(voice.reason, voice.error);\n }\n}\n"],
|
|
5
|
+
"mappings": "AACA,SAAS,yBAAyB;AAElC;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAEP,SAA6B,kBAAkB;AAC/C;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAEP,eAAe,UACb,OACA,OACA,SACiB;AACjB,MAAI,SAAS,MAAO,QAAO,MAAM;AACjC,QAAM,QAAQ,UAAU,MAAM,QAAQ;AACtC,QAAM,UAAU;AAAA,IACd,EAAE,MAAM,MAAM,MAAM,UAAU,MAAM,UAAU,UAAU,MAAM,UAAU,QAAQ,MAAM,OAAO;AAAA,IAC7F,CAAC;AAAA,EACH;AACA,QAAM,OAAO,IAAI,WAAW,QAAQ,UAAU;AAC9C,OAAK,IAAI,OAAO;AAChB,QAAM,SAAS,MAAM;AAAA,IACnB;AAAA,IACA;AAAA,IACA,SAAS,MAAM,SAAS,qBAAqB;AAAA,IAC7C,KAAK;AAAA,IACL,cAAc;AAAA,EAChB;AACA,SAAO,YAAY,MAAM;AAC3B;AAEA,eAAe,OACb,OACA,SACA,QACiB;AACjB,MAAI,OAAO,WAAW,YAAY,UAAU,MAAM,GAAG;AACnD,UAAM,WAAW,MAAM,SAAS,OAAO,SAAS,MAAM;AACtD,UAAMA,QAAO,MAAM,SAAS,KAAK;AACjC,QAAIA,UAAS,GAAI,OAAM,IAAI,MAAM,oCAAoC;AACrE,WAAOA;AAAA,EACT;AACA,QAAM,OAAO,WAAW,MAAM;AAC9B,MAAI,SAAS,UAAa,SAAS,MAAM,UAAU,IAAI,GAAG;AACxD,UAAM,IAAI,MAAM,oCAAoC;AAAA,EACtD;AACA,SAAO;AACT;AAEA,SAAS,WAAW,SAAiE;AACnF,QAAM,SAAS;AACf,MAAI,WAAW,OAAW,QAAO;AACjC,MAAI,OAAO,OAAO,aAAa,SAAU,QAAO;AAChD,SAAO,OAAO;AAChB;AAEA,eAAsB,oBACpB,OACA,OACA,SACqB;AACrB,MAAI;AACF,QAAI,SAAS,QAAQ,QAAS,OAAM,IAAI,aAAa,WAAW,YAAY;AAC5E,UAAM,QAAQ,MAAM,UAAU,OAAO,OAAO,OAAO;AACnD,UAAM,UAAmC;AAAA,MACvC;AAAA,MACA,MAAM;AAAA,MACN,mBAAmB;AAAA,IACrB;AACA,UAAM,WAAW,WAAW,OAAO;AACnC,QAAI,aAAa,OAAW,SAAQ,WAAW;AAC/C,UAAM,SAAS,MAAM,WAAW,OAAO,SAAS,SAAS,oCAAoC;AAC7F,UAAM,UAAU,OAAO;AACvB,UAAM,QAAQ,MAAM,QAAQ,OAAO,IAAI,QAAQ,CAAC,IAAI;AACpD,UAAM,OAAO,MAAM,OAAO,OAAO,SAAS,KAAK;AAC/C,UAAM,SAAwB,CAAC;AAC/B,UAAM,UAAU,gBAAgB,MAAM;AACtC,QAAI,UAAU,EAAG,QAAO,eAAe;AACvC,UAAM,aAAyB,EAAE,MAAM,OAAO,WAAW,MAAM,WAAW,MAAM,EAAE;AAClF,QAAI,aAAa,OAAW,YAAW,WAAW;AAClD,WAAO;AAAA,EACT,SAAS,OAAO;AACd,UAAM,QAAQ,aAAa,KAAK;AAChC,UAAM,IAAI,kBAAkB,MAAM,QAAQ,MAAM,KAAK;AAAA,EACvD;AACF;",
|
|
6
|
+
"names": ["text"]
|
|
7
|
+
}
|
|
@@ -0,0 +1,138 @@
|
|
|
1
|
+
import {
|
|
2
|
+
applyPayload,
|
|
3
|
+
endpoint,
|
|
4
|
+
isRecord,
|
|
5
|
+
numberField,
|
|
6
|
+
requestHeaders,
|
|
7
|
+
requestSignal,
|
|
8
|
+
stringField
|
|
9
|
+
} from "../openai/wire.js";
|
|
10
|
+
const terminal = /* @__PURE__ */ new Set(["failed", "cancelled", "timeout", "deleted"]);
|
|
11
|
+
function taskRecord(body) {
|
|
12
|
+
if (!isRecord(body) || Array.isArray(body)) throw new Error("WaveSpeed response was not JSON");
|
|
13
|
+
const data = body.data;
|
|
14
|
+
if (isRecord(data) && !Array.isArray(data)) return data;
|
|
15
|
+
return body;
|
|
16
|
+
}
|
|
17
|
+
function errorText(text, status) {
|
|
18
|
+
if (text === "") return `HTTP ${status}`;
|
|
19
|
+
let parsed;
|
|
20
|
+
try {
|
|
21
|
+
parsed = JSON.parse(text);
|
|
22
|
+
} catch {
|
|
23
|
+
return text;
|
|
24
|
+
}
|
|
25
|
+
if (!isRecord(parsed) || Array.isArray(parsed)) return text;
|
|
26
|
+
const message = stringField(parsed, "message");
|
|
27
|
+
if (message !== void 0 && message !== "") return message;
|
|
28
|
+
return text;
|
|
29
|
+
}
|
|
30
|
+
function isHttpUrl(value) {
|
|
31
|
+
if (value.startsWith("https://")) return true;
|
|
32
|
+
if (value.startsWith("http://")) return true;
|
|
33
|
+
return false;
|
|
34
|
+
}
|
|
35
|
+
function outputText(output) {
|
|
36
|
+
if (typeof output === "string") return output;
|
|
37
|
+
if (!isRecord(output) || Array.isArray(output)) return void 0;
|
|
38
|
+
return stringField(output, "text") ?? stringField(output, "transcript");
|
|
39
|
+
}
|
|
40
|
+
function outputUrl(output) {
|
|
41
|
+
if (typeof output === "string" && isHttpUrl(output)) return output;
|
|
42
|
+
if (!isRecord(output) || Array.isArray(output)) return void 0;
|
|
43
|
+
const url = stringField(output, "url") ?? stringField(output, "audio");
|
|
44
|
+
if (url === void 0 || !isHttpUrl(url)) return void 0;
|
|
45
|
+
return url;
|
|
46
|
+
}
|
|
47
|
+
function downloadUrl(body) {
|
|
48
|
+
const record = taskRecord(body);
|
|
49
|
+
const url = stringField(record, "download_url") ?? stringField(record, "url");
|
|
50
|
+
if (url === void 0 || url === "") throw new Error("WaveSpeed upload had no url");
|
|
51
|
+
return url;
|
|
52
|
+
}
|
|
53
|
+
function durationSeconds(record) {
|
|
54
|
+
const direct = numberField(record, "duration");
|
|
55
|
+
if (direct !== void 0) return direct;
|
|
56
|
+
return 0;
|
|
57
|
+
}
|
|
58
|
+
async function waveFetch(model, options, url, body, contentType) {
|
|
59
|
+
const headers = requestHeaders(model, options);
|
|
60
|
+
if (contentType !== void 0) headers["Content-Type"] = contentType;
|
|
61
|
+
const fetchImpl = options?.fetch ?? globalThis.fetch;
|
|
62
|
+
const response = await fetchImpl(url, {
|
|
63
|
+
method: body === void 0 ? "GET" : "POST",
|
|
64
|
+
headers,
|
|
65
|
+
body,
|
|
66
|
+
signal: requestSignal(options)
|
|
67
|
+
});
|
|
68
|
+
if (options?.onResponse !== void 0) {
|
|
69
|
+
await options.onResponse({ status: response.status, headers: {} }, model);
|
|
70
|
+
}
|
|
71
|
+
if (!response.ok) {
|
|
72
|
+
const text2 = await response.text();
|
|
73
|
+
throw new Error(errorText(text2, response.status));
|
|
74
|
+
}
|
|
75
|
+
const text = await response.text();
|
|
76
|
+
if (text === "") throw new Error("WaveSpeed response was not JSON");
|
|
77
|
+
try {
|
|
78
|
+
return JSON.parse(text);
|
|
79
|
+
} catch {
|
|
80
|
+
throw new Error("WaveSpeed response was not JSON");
|
|
81
|
+
}
|
|
82
|
+
}
|
|
83
|
+
function failure(record, status) {
|
|
84
|
+
const error = stringField(record, "error");
|
|
85
|
+
if (error === void 0 || error === "") return new Error(`WaveSpeed ${status}`);
|
|
86
|
+
return new Error(error);
|
|
87
|
+
}
|
|
88
|
+
async function finishTask(model, options, payload, rejected) {
|
|
89
|
+
const body = await applyPayload(payload, model, options);
|
|
90
|
+
if (!isRecord(body) || Array.isArray(body)) throw new Error(rejected);
|
|
91
|
+
let record = taskRecord(
|
|
92
|
+
await waveFetch(
|
|
93
|
+
model,
|
|
94
|
+
options,
|
|
95
|
+
endpoint(model.baseUrl, model.id),
|
|
96
|
+
JSON.stringify(body),
|
|
97
|
+
"application/json"
|
|
98
|
+
)
|
|
99
|
+
);
|
|
100
|
+
for (let attempt = 0; attempt < 9; attempt += 1) {
|
|
101
|
+
const status = stringField(record, "status") ?? "";
|
|
102
|
+
if (status === "completed") return record;
|
|
103
|
+
if (terminal.has(status)) throw failure(record, status);
|
|
104
|
+
if (attempt === 8) break;
|
|
105
|
+
const id = stringField(record, "id");
|
|
106
|
+
if (id === void 0 || id === "") throw new Error("WaveSpeed prediction had no id");
|
|
107
|
+
record = taskRecord(
|
|
108
|
+
await waveFetch(
|
|
109
|
+
model,
|
|
110
|
+
options,
|
|
111
|
+
endpoint(model.baseUrl, `predictions/${id}/result`),
|
|
112
|
+
void 0,
|
|
113
|
+
void 0
|
|
114
|
+
)
|
|
115
|
+
);
|
|
116
|
+
}
|
|
117
|
+
throw new Error("WaveSpeed prediction timed out");
|
|
118
|
+
}
|
|
119
|
+
async function readHttp(model, options, url) {
|
|
120
|
+
const headers = requestHeaders(model, options);
|
|
121
|
+
const fetchImpl = options?.fetch ?? globalThis.fetch;
|
|
122
|
+
const response = await fetchImpl(url, { headers, signal: requestSignal(options) });
|
|
123
|
+
if (!response.ok) throw new Error(errorText("", response.status));
|
|
124
|
+
return response;
|
|
125
|
+
}
|
|
126
|
+
export {
|
|
127
|
+
downloadUrl,
|
|
128
|
+
durationSeconds,
|
|
129
|
+
errorText,
|
|
130
|
+
finishTask,
|
|
131
|
+
isHttpUrl,
|
|
132
|
+
outputText,
|
|
133
|
+
outputUrl,
|
|
134
|
+
readHttp,
|
|
135
|
+
taskRecord,
|
|
136
|
+
waveFetch
|
|
137
|
+
};
|
|
138
|
+
//# sourceMappingURL=wire.js.map
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
{
|
|
2
|
+
"version": 3,
|
|
3
|
+
"sources": ["../../src/wavespeed/wire.ts"],
|
|
4
|
+
"sourcesContent": ["import type { ProviderRequestOptions } from '@earendil-works/pi-ai';\nimport {\n applyPayload,\n endpoint,\n isRecord,\n numberField,\n requestHeaders,\n requestSignal,\n stringField,\n} from '../openai/wire.ts';\n\nconst terminal = new Set(['failed', 'cancelled', 'timeout', 'deleted']);\n\nexport function taskRecord(body: unknown): Record<string, unknown> {\n if (!isRecord(body) || Array.isArray(body)) throw new Error('WaveSpeed response was not JSON');\n const data = body.data;\n if (isRecord(data) && !Array.isArray(data)) return data;\n return body;\n}\n\nexport function errorText(text: string, status: number): string {\n if (text === '') return `HTTP ${status}`;\n let parsed: unknown;\n try {\n parsed = JSON.parse(text);\n } catch {\n return text;\n }\n if (!isRecord(parsed) || Array.isArray(parsed)) return text;\n const message = stringField(parsed, 'message');\n if (message !== undefined && message !== '') return message;\n return text;\n}\n\nexport function isHttpUrl(value: string): boolean {\n if (value.startsWith('https://')) return true;\n if (value.startsWith('http://')) return true;\n return false;\n}\n\nexport function outputText(output: unknown): string | undefined {\n if (typeof output === 'string') return output;\n if (!isRecord(output) || Array.isArray(output)) return undefined;\n return stringField(output, 'text') ?? stringField(output, 'transcript');\n}\n\nexport function outputUrl(output: unknown): string | undefined {\n if (typeof output === 'string' && isHttpUrl(output)) return output;\n if (!isRecord(output) || Array.isArray(output)) return undefined;\n const url = stringField(output, 'url') ?? stringField(output, 'audio');\n if (url === undefined || !isHttpUrl(url)) return undefined;\n return url;\n}\n\nexport function downloadUrl(body: unknown): string {\n const record = taskRecord(body);\n const url = stringField(record, 'download_url') ?? stringField(record, 'url');\n if (url === undefined || url === '') throw new Error('WaveSpeed upload had no url');\n return url;\n}\n\nexport function durationSeconds(record: Record<string, unknown>): number {\n const direct = numberField(record, 'duration');\n if (direct !== undefined) return direct;\n return 0;\n}\n\nasync function waveFetch(\n model: { provider: string; headers?: Record<string, string>; id: string },\n options: ProviderRequestOptions | undefined,\n url: string,\n body: BodyInit | undefined,\n contentType: string | undefined\n): Promise<unknown> {\n const headers = requestHeaders(model, options);\n if (contentType !== undefined) headers['Content-Type'] = contentType;\n const fetchImpl = options?.fetch ?? globalThis.fetch;\n const response = await fetchImpl(url, {\n method: body === undefined ? 'GET' : 'POST',\n headers,\n body,\n signal: requestSignal(options),\n });\n if (options?.onResponse !== undefined) {\n await options.onResponse({ status: response.status, headers: {} }, model as never);\n }\n if (!response.ok) {\n const text = await response.text();\n throw new Error(errorText(text, response.status));\n }\n const text = await response.text();\n if (text === '') throw new Error('WaveSpeed response was not JSON');\n try {\n return JSON.parse(text);\n } catch {\n throw new Error('WaveSpeed response was not JSON');\n }\n}\n\nfunction failure(record: Record<string, unknown>, status: string): Error {\n const error = stringField(record, 'error');\n if (error === undefined || error === '') return new Error(`WaveSpeed ${status}`);\n return new Error(error);\n}\n\nexport async function finishTask(\n model: { provider: string; baseUrl: string; headers?: Record<string, string>; id: string },\n options: ProviderRequestOptions | undefined,\n payload: unknown,\n rejected: string\n): Promise<Record<string, unknown>> {\n const body = await applyPayload(payload, model, options);\n if (!isRecord(body) || Array.isArray(body)) throw new Error(rejected);\n let record = taskRecord(\n await waveFetch(\n model,\n options,\n endpoint(model.baseUrl, model.id),\n JSON.stringify(body),\n 'application/json'\n )\n );\n for (let attempt = 0; attempt < 9; attempt += 1) {\n const status = stringField(record, 'status') ?? '';\n if (status === 'completed') return record;\n if (terminal.has(status)) throw failure(record, status);\n if (attempt === 8) break;\n const id = stringField(record, 'id');\n if (id === undefined || id === '') throw new Error('WaveSpeed prediction had no id');\n record = taskRecord(\n await waveFetch(\n model,\n options,\n endpoint(model.baseUrl, `predictions/${id}/result`),\n undefined,\n undefined\n )\n );\n }\n throw new Error('WaveSpeed prediction timed out');\n}\n\nexport async function readHttp(\n model: { provider: string; headers?: Record<string, string>; id: string },\n options: ProviderRequestOptions | undefined,\n url: string\n): Promise<Response> {\n const headers = requestHeaders(model, options);\n const fetchImpl = options?.fetch ?? globalThis.fetch;\n const response = await fetchImpl(url, { headers, signal: requestSignal(options) });\n if (!response.ok) throw new Error(errorText('', response.status));\n return response;\n}\n\nexport { waveFetch };\n"],
|
|
5
|
+
"mappings": "AACA;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAEP,MAAM,WAAW,oBAAI,IAAI,CAAC,UAAU,aAAa,WAAW,SAAS,CAAC;AAE/D,SAAS,WAAW,MAAwC;AACjE,MAAI,CAAC,SAAS,IAAI,KAAK,MAAM,QAAQ,IAAI,EAAG,OAAM,IAAI,MAAM,iCAAiC;AAC7F,QAAM,OAAO,KAAK;AAClB,MAAI,SAAS,IAAI,KAAK,CAAC,MAAM,QAAQ,IAAI,EAAG,QAAO;AACnD,SAAO;AACT;AAEO,SAAS,UAAU,MAAc,QAAwB;AAC9D,MAAI,SAAS,GAAI,QAAO,QAAQ,MAAM;AACtC,MAAI;AACJ,MAAI;AACF,aAAS,KAAK,MAAM,IAAI;AAAA,EAC1B,QAAQ;AACN,WAAO;AAAA,EACT;AACA,MAAI,CAAC,SAAS,MAAM,KAAK,MAAM,QAAQ,MAAM,EAAG,QAAO;AACvD,QAAM,UAAU,YAAY,QAAQ,SAAS;AAC7C,MAAI,YAAY,UAAa,YAAY,GAAI,QAAO;AACpD,SAAO;AACT;AAEO,SAAS,UAAU,OAAwB;AAChD,MAAI,MAAM,WAAW,UAAU,EAAG,QAAO;AACzC,MAAI,MAAM,WAAW,SAAS,EAAG,QAAO;AACxC,SAAO;AACT;AAEO,SAAS,WAAW,QAAqC;AAC9D,MAAI,OAAO,WAAW,SAAU,QAAO;AACvC,MAAI,CAAC,SAAS,MAAM,KAAK,MAAM,QAAQ,MAAM,EAAG,QAAO;AACvD,SAAO,YAAY,QAAQ,MAAM,KAAK,YAAY,QAAQ,YAAY;AACxE;AAEO,SAAS,UAAU,QAAqC;AAC7D,MAAI,OAAO,WAAW,YAAY,UAAU,MAAM,EAAG,QAAO;AAC5D,MAAI,CAAC,SAAS,MAAM,KAAK,MAAM,QAAQ,MAAM,EAAG,QAAO;AACvD,QAAM,MAAM,YAAY,QAAQ,KAAK,KAAK,YAAY,QAAQ,OAAO;AACrE,MAAI,QAAQ,UAAa,CAAC,UAAU,GAAG,EAAG,QAAO;AACjD,SAAO;AACT;AAEO,SAAS,YAAY,MAAuB;AACjD,QAAM,SAAS,WAAW,IAAI;AAC9B,QAAM,MAAM,YAAY,QAAQ,cAAc,KAAK,YAAY,QAAQ,KAAK;AAC5E,MAAI,QAAQ,UAAa,QAAQ,GAAI,OAAM,IAAI,MAAM,6BAA6B;AAClF,SAAO;AACT;AAEO,SAAS,gBAAgB,QAAyC;AACvE,QAAM,SAAS,YAAY,QAAQ,UAAU;AAC7C,MAAI,WAAW,OAAW,QAAO;AACjC,SAAO;AACT;AAEA,eAAe,UACb,OACA,SACA,KACA,MACA,aACkB;AAClB,QAAM,UAAU,eAAe,OAAO,OAAO;AAC7C,MAAI,gBAAgB,OAAW,SAAQ,cAAc,IAAI;AACzD,QAAM,YAAY,SAAS,SAAS,WAAW;AAC/C,QAAM,WAAW,MAAM,UAAU,KAAK;AAAA,IACpC,QAAQ,SAAS,SAAY,QAAQ;AAAA,IACrC;AAAA,IACA;AAAA,IACA,QAAQ,cAAc,OAAO;AAAA,EAC/B,CAAC;AACD,MAAI,SAAS,eAAe,QAAW;AACrC,UAAM,QAAQ,WAAW,EAAE,QAAQ,SAAS,QAAQ,SAAS,CAAC,EAAE,GAAG,KAAc;AAAA,EACnF;AACA,MAAI,CAAC,SAAS,IAAI;AAChB,UAAMA,QAAO,MAAM,SAAS,KAAK;AACjC,UAAM,IAAI,MAAM,UAAUA,OAAM,SAAS,MAAM,CAAC;AAAA,EAClD;AACA,QAAM,OAAO,MAAM,SAAS,KAAK;AACjC,MAAI,SAAS,GAAI,OAAM,IAAI,MAAM,iCAAiC;AAClE,MAAI;AACF,WAAO,KAAK,MAAM,IAAI;AAAA,EACxB,QAAQ;AACN,UAAM,IAAI,MAAM,iCAAiC;AAAA,EACnD;AACF;AAEA,SAAS,QAAQ,QAAiC,QAAuB;AACvE,QAAM,QAAQ,YAAY,QAAQ,OAAO;AACzC,MAAI,UAAU,UAAa,UAAU,GAAI,QAAO,IAAI,MAAM,aAAa,MAAM,EAAE;AAC/E,SAAO,IAAI,MAAM,KAAK;AACxB;AAEA,eAAsB,WACpB,OACA,SACA,SACA,UACkC;AAClC,QAAM,OAAO,MAAM,aAAa,SAAS,OAAO,OAAO;AACvD,MAAI,CAAC,SAAS,IAAI,KAAK,MAAM,QAAQ,IAAI,EAAG,OAAM,IAAI,MAAM,QAAQ;AACpE,MAAI,SAAS;AAAA,IACX,MAAM;AAAA,MACJ;AAAA,MACA;AAAA,MACA,SAAS,MAAM,SAAS,MAAM,EAAE;AAAA,MAChC,KAAK,UAAU,IAAI;AAAA,MACnB;AAAA,IACF;AAAA,EACF;AACA,WAAS,UAAU,GAAG,UAAU,GAAG,WAAW,GAAG;AAC/C,UAAM,SAAS,YAAY,QAAQ,QAAQ,KAAK;AAChD,QAAI,WAAW,YAAa,QAAO;AACnC,QAAI,SAAS,IAAI,MAAM,EAAG,OAAM,QAAQ,QAAQ,MAAM;AACtD,QAAI,YAAY,EAAG;AACnB,UAAM,KAAK,YAAY,QAAQ,IAAI;AACnC,QAAI,OAAO,UAAa,OAAO,GAAI,OAAM,IAAI,MAAM,gCAAgC;AACnF,aAAS;AAAA,MACP,MAAM;AAAA,QACJ;AAAA,QACA;AAAA,QACA,SAAS,MAAM,SAAS,eAAe,EAAE,SAAS;AAAA,QAClD;AAAA,QACA;AAAA,MACF;AAAA,IACF;AAAA,EACF;AACA,QAAM,IAAI,MAAM,gCAAgC;AAClD;AAEA,eAAsB,SACpB,OACA,SACA,KACmB;AACnB,QAAM,UAAU,eAAe,OAAO,OAAO;AAC7C,QAAM,YAAY,SAAS,SAAS,WAAW;AAC/C,QAAM,WAAW,MAAM,UAAU,KAAK,EAAE,SAAS,QAAQ,cAAc,OAAO,EAAE,CAAC;AACjF,MAAI,CAAC,SAAS,GAAI,OAAM,IAAI,MAAM,UAAU,IAAI,SAAS,MAAM,CAAC;AAChE,SAAO;AACT;",
|
|
6
|
+
"names": ["text"]
|
|
7
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@ai-ecoverse/pi-audio",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.26.0",
|
|
4
4
|
"description": "Voice bindings for pi: speech-to-text, text-to-speech and realtime speech for pi-coding-agent plugins and slicc-agent",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "Apache-2.0",
|
|
@@ -66,6 +66,10 @@
|
|
|
66
66
|
"types": "./dist/types/cloudflare/index.d.ts",
|
|
67
67
|
"default": "./dist/cloudflare/index.js"
|
|
68
68
|
},
|
|
69
|
+
"./wavespeed": {
|
|
70
|
+
"types": "./dist/types/wavespeed/index.d.ts",
|
|
71
|
+
"default": "./dist/wavespeed/index.js"
|
|
72
|
+
},
|
|
69
73
|
"./durable": {
|
|
70
74
|
"types": "./dist/types/durable/index.d.ts",
|
|
71
75
|
"default": "./dist/durable/index.js"
|
|
@@ -81,6 +85,10 @@
|
|
|
81
85
|
"./fish-agents": {
|
|
82
86
|
"types": "./dist/types/fish-agents/index.d.ts",
|
|
83
87
|
"default": "./dist/fish-agents/index.js"
|
|
88
|
+
},
|
|
89
|
+
"./brain": {
|
|
90
|
+
"types": "./dist/types/brain/index.d.ts",
|
|
91
|
+
"default": "./dist/brain/index.js"
|
|
84
92
|
}
|
|
85
93
|
},
|
|
86
94
|
"files": [
|