@ai-ecoverse/pi-audio 1.15.0 → 1.17.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -0
- package/dist/elevenlabs-agents/events.js +172 -0
- package/dist/elevenlabs-agents/events.js.map +7 -0
- package/dist/elevenlabs-agents/index.js +58 -0
- package/dist/elevenlabs-agents/index.js.map +7 -0
- package/dist/elevenlabs-agents/provision.js +167 -0
- package/dist/elevenlabs-agents/provision.js.map +7 -0
- package/dist/elevenlabs-agents/session.js +272 -0
- package/dist/elevenlabs-agents/session.js.map +7 -0
- package/dist/hosted-agent.js +35 -0
- package/dist/hosted-agent.js.map +7 -0
- package/dist/index.js +2 -0
- package/dist/index.js.map +2 -2
- package/dist/soundscape.js +104 -0
- package/dist/soundscape.js.map +7 -0
- package/dist/turns.js +28 -4
- package/dist/turns.js.map +2 -2
- package/dist/types/elevenlabs-agents/events.d.ts +13 -0
- package/dist/types/elevenlabs-agents/index.d.ts +9 -0
- package/dist/types/elevenlabs-agents/provision.d.ts +11 -0
- package/dist/types/elevenlabs-agents/session.d.ts +16 -0
- package/dist/types/hosted-agent.d.ts +23 -0
- package/dist/types/index.d.ts +2 -0
- package/dist/types/soundscape.d.ts +47 -0
- package/dist/types/turns.d.ts +3 -1
- package/package.json +5 -1
package/README.md
CHANGED
|
@@ -27,6 +27,8 @@ pi-audio does not apply that patch. Hosts such as slicc-agent record monkeypatch
|
|
|
27
27
|
|
|
28
28
|
`@ai-ecoverse/pi-audio/elevenlabs` registers ElevenLabs speech adapters when `registerElevenLabs()` is called. `scribe_v2` posts multipart audio to `/speech-to-text`. `scribe_v2_realtime` streams base64 audio on `wss://…/speech-to-text/realtime` and commits turns with `turn_prediction` (`turnDetection: 'semantic'`). `eleven_flash_v2_5` and `eleven_multilingual_v2` post JSON to `/text-to-speech/{voice}/with-timestamps` for a string and stream text on `stream-input` for an async iterable, with character alignment. `eleven_v3` and `eleven_v4` stay on the HTTP path (`textStreaming: false`). Scribe prices are $0.22 per hour for batch and $0.39 per hour for realtime.
|
|
29
29
|
|
|
30
|
+
`followBed(bed, soundscape)` cues a `BedChannel` from `VoicePhase`. A loop waits `graceMs` before it starts, so a short gap stays silent. `keep` leaves the current loop running across chained tools. Speech ducks the bed to `duckUnderSpeech`; listening ducks it to `duckUnderUser`. `voiceTurns` does this when `options.soundscape` is set and `out` is an `AudioOut` (`speech`, `bed`, `reference`, `playback`). A plain `AudioSink` still receives only the spoken audio.
|
|
31
|
+
|
|
30
32
|
`voiceTurns` captions include `agent_spoken`. `upTo` follows `AudioSink` playback: per character when the chunk has alignment, and evenly across the sentence when it does not. `captionsFromHistory(entries)` rebuilds user captions from entries whose `data.source` is `voice`, and agent captions from the assistant text. A `voice.truncate` entry adds `agent_spoken` and `agent_cut` at `heardUpTo`, so the rest of the original text stays available as unheard.
|
|
31
33
|
`@ai-ecoverse/pi-audio/xai` registers xAI speech adapters when `registerXAI()` is called. `grok-voice-transcribe-2.0` posts multipart audio to `/stt` (fields before `file`) and streams raw binary frames on `wss://…/stt` after `transcript.created`, with Smart Turn (`turnDetection: 'semantic'`). The catalog id is shared, so `audioCost` uses the streaming rate of $0.20 per hour. `grok-voice-tts-1.0` posts JSON to `/tts` for a string and streams `text.delta` / `audio.delta` on `wss://…/tts` for an async iterable (`textStreaming: true`). Voices are eve, ara, rex, sal, and leo. Codecs are mp3, pcm, wav, mulaw, and alaw.
|
|
32
34
|
|
|
@@ -46,6 +48,8 @@ pi-audio does not apply that patch. Hosts such as slicc-agent record monkeypatch
|
|
|
46
48
|
|
|
47
49
|
`mintRealtimeToken(model, constraints, options)` returns `{ value, expiresAt, model }` and calls `options.fetch` (or `globalThis.fetch`). OpenAI `POST /v1/realtime/client_secrets` sends a realtime session with instructions, voice, and tools. xAI `POST /v1/realtime/client_secrets` sends only `expires_after.seconds` of 300. Gemini `POST /v1beta/auth_tokens` locks `models/<id>`, instructions, voice, and tools in `liveConnectConstraints` and returns `name`, which the live adapter sends as `access_token`. ElevenLabs `GET /v1/convai/conversation/get-signed-url` takes `agent_id` (or `GET /conversation/token` when the only transport is WebRTC) and expires in 15 minutes. Fish `POST /v1/agent/sessions` sends `agent_id` plus `overrides.system_prompt` and `overrides.voice_id`. `value` is the LiveKit participant token. `expiresAt` is milliseconds. A numeric `expires_at` below `1e12` is unix seconds.
|
|
48
50
|
|
|
51
|
+
`@ai-ecoverse/pi-audio/elevenlabs-agents` registers `elevenlabs-agent`, a hosted cascade (`provisioning: "agent"`, `truncation: "server"`, `liveToolUpdate: false`). `provisionAgent` creates or updates the agent and reuses it when the spec hash matches. pi tools become client tools. A string `llm` is a vendor model name. A `CustomLlm` sets `llm` to `custom-llm` and sends `custom_llm.url`, `model_id`, `api_type`, and `request_headers` (ElevenLabs `api_key` is a workspace secret reference, not a raw token). `connectRealtime` uses a `RealtimeToken` from `mintRealtimeToken` (`get-signed-url`) when one is passed. Otherwise an API key provisions the agent and mints the signed URL. Silent `inject` sends `contextual_update` with `context_id` `pi-audio`. `close()` prices `audio.inputSeconds` from the wall-clock session at $0.08 per minute.
|
|
52
|
+
|
|
49
53
|
`talkerTools` is `ask_agent` and `read`. pi-audio defines `askAgent` (a pi `Tool` whose parameters are `request` and optional `context`) and the `Thinker` interface. The host implements `Thinker` and supplies `read`. `bindThinker` runs direct tools with `directTimeoutMs` and truncates their text to `directMaxChars`. On a blocking `ask_agent` it answers with `ack` immediately and injects the result later. On an async call it `respond`s with the result and `options.result`. Progress is injected with `options.progress`. `tool_call_cancelled` aborts the thinker. The binding also yields `thinker_progress` / `thinker_result` captions and `thinking` / `tool` / `idle` phases.
|
|
50
54
|
|
|
51
55
|
## Development
|
|
@@ -0,0 +1,172 @@
|
|
|
1
|
+
import { blankUsage } from "../blank-usage.js";
|
|
2
|
+
import { decodeAudioBase64 } from "../convert.js";
|
|
3
|
+
function none() {
|
|
4
|
+
return { events: [], replies: [] };
|
|
5
|
+
}
|
|
6
|
+
function isRecord(value) {
|
|
7
|
+
return typeof value === "object" && value !== null && !Array.isArray(value);
|
|
8
|
+
}
|
|
9
|
+
function text(value) {
|
|
10
|
+
return typeof value === "string" ? value : "";
|
|
11
|
+
}
|
|
12
|
+
function transcript(itemId, value, final) {
|
|
13
|
+
return { type: "output_transcript", itemId, text: value, final };
|
|
14
|
+
}
|
|
15
|
+
function done(itemId, interrupted) {
|
|
16
|
+
return { type: "response_done", itemId, interrupted, usage: blankUsage() };
|
|
17
|
+
}
|
|
18
|
+
function numbers(value) {
|
|
19
|
+
if (!Array.isArray(value) || value.some((item) => typeof item !== "number")) return void 0;
|
|
20
|
+
return value;
|
|
21
|
+
}
|
|
22
|
+
function alignmentOf(value) {
|
|
23
|
+
if (!isRecord(value)) return void 0;
|
|
24
|
+
const startMs = numbers(value.char_start_times_ms);
|
|
25
|
+
const durationMs = numbers(value.char_durations_ms);
|
|
26
|
+
if (startMs === void 0 || durationMs === void 0) return void 0;
|
|
27
|
+
return { startMs, durationMs };
|
|
28
|
+
}
|
|
29
|
+
function pong(_turn, message) {
|
|
30
|
+
const nested = isRecord(message.ping_event) ? message.ping_event : void 0;
|
|
31
|
+
const raw = nested === void 0 ? message.event_id : nested.event_id;
|
|
32
|
+
const eventId = typeof raw === "number" ? raw : 0;
|
|
33
|
+
return { events: [], replies: [{ type: "pong", event_id: eventId }] };
|
|
34
|
+
}
|
|
35
|
+
function metadata(_turn, message) {
|
|
36
|
+
const event = isRecord(message.conversation_initiation_metadata_event) ? message.conversation_initiation_metadata_event : void 0;
|
|
37
|
+
const sessionId = text(event?.conversation_id);
|
|
38
|
+
return {
|
|
39
|
+
events: [
|
|
40
|
+
{ type: "session_start", sessionId: sessionId.length > 0 ? sessionId : "conversation" }
|
|
41
|
+
],
|
|
42
|
+
replies: []
|
|
43
|
+
};
|
|
44
|
+
}
|
|
45
|
+
function userTranscript(_turn, message) {
|
|
46
|
+
const event = isRecord(message.user_transcription_event) ? message.user_transcription_event : void 0;
|
|
47
|
+
return {
|
|
48
|
+
events: [
|
|
49
|
+
{
|
|
50
|
+
type: "input_transcript",
|
|
51
|
+
itemId: "user",
|
|
52
|
+
text: text(event?.user_transcript),
|
|
53
|
+
final: true
|
|
54
|
+
}
|
|
55
|
+
],
|
|
56
|
+
replies: []
|
|
57
|
+
};
|
|
58
|
+
}
|
|
59
|
+
function chatPart(_turn, message) {
|
|
60
|
+
const part = isRecord(message.text_response_part) ? message.text_response_part : void 0;
|
|
61
|
+
if (part === void 0) return none();
|
|
62
|
+
const kind = text(part.type);
|
|
63
|
+
if (kind === "start" || kind.length === 0) return none();
|
|
64
|
+
const itemId = text(part.response_id);
|
|
65
|
+
return {
|
|
66
|
+
events: [transcript(itemId.length > 0 ? itemId : "agent", text(part.text), kind === "stop")],
|
|
67
|
+
replies: []
|
|
68
|
+
};
|
|
69
|
+
}
|
|
70
|
+
function agentResponse(_turn, message) {
|
|
71
|
+
const event = isRecord(message.agent_response_event) ? message.agent_response_event : void 0;
|
|
72
|
+
return { events: [transcript("agent", text(event?.agent_response), true)], replies: [] };
|
|
73
|
+
}
|
|
74
|
+
function correction(_turn, message) {
|
|
75
|
+
const event = isRecord(message.agent_response_correction_event) ? message.agent_response_correction_event : void 0;
|
|
76
|
+
return {
|
|
77
|
+
events: [transcript("agent", text(event?.corrected_agent_response), true)],
|
|
78
|
+
replies: []
|
|
79
|
+
};
|
|
80
|
+
}
|
|
81
|
+
function audio(turn, message, format) {
|
|
82
|
+
const event = isRecord(message.audio_event) ? message.audio_event : void 0;
|
|
83
|
+
const eventId = typeof event?.event_id === "number" ? event.event_id : 0;
|
|
84
|
+
if (turn.dropAudio || eventId <= turn.lastInterruptId) return none();
|
|
85
|
+
const alignment = alignmentOf(event?.alignment);
|
|
86
|
+
const chars = isRecord(event?.alignment) ? event.alignment.chars : void 0;
|
|
87
|
+
const end = Array.isArray(chars) ? chars.length : 0;
|
|
88
|
+
const itemId = String(eventId);
|
|
89
|
+
const first = alignment?.startMs[0];
|
|
90
|
+
const spoken = {
|
|
91
|
+
type: "output_audio",
|
|
92
|
+
itemId,
|
|
93
|
+
chunk: {
|
|
94
|
+
data: decodeAudioBase64(text(event?.audio_base_64)),
|
|
95
|
+
format,
|
|
96
|
+
timestamp: typeof first === "number" ? first : 0
|
|
97
|
+
}
|
|
98
|
+
};
|
|
99
|
+
if (alignment !== void 0) spoken.alignment = alignment;
|
|
100
|
+
if (end > 0) spoken.span = { itemId, start: 0, end };
|
|
101
|
+
return { events: [spoken], replies: [] };
|
|
102
|
+
}
|
|
103
|
+
function interruption(turn, message) {
|
|
104
|
+
const event = isRecord(message.interruption_event) ? message.interruption_event : void 0;
|
|
105
|
+
const eventId = typeof event?.event_id === "number" ? event.event_id : turn.lastInterruptId;
|
|
106
|
+
turn.lastInterruptId = eventId;
|
|
107
|
+
turn.vad = true;
|
|
108
|
+
return { events: [{ type: "input_speech_start" }, done(String(eventId), true)], replies: [] };
|
|
109
|
+
}
|
|
110
|
+
function vad(turn, message) {
|
|
111
|
+
const event = isRecord(message.vad_score_event) ? message.vad_score_event : void 0;
|
|
112
|
+
const score = typeof event?.vad_score === "number" ? event.vad_score : 0;
|
|
113
|
+
const speaking = score > 0.5;
|
|
114
|
+
if (!speaking) {
|
|
115
|
+
turn.vad = false;
|
|
116
|
+
return none();
|
|
117
|
+
}
|
|
118
|
+
if (turn.vad) return none();
|
|
119
|
+
turn.vad = true;
|
|
120
|
+
return { events: [{ type: "input_speech_start" }], replies: [] };
|
|
121
|
+
}
|
|
122
|
+
function toolCall(_turn, message) {
|
|
123
|
+
const event = isRecord(message.client_tool_call) ? message.client_tool_call : void 0;
|
|
124
|
+
const name = text(event?.tool_name);
|
|
125
|
+
const id = text(event?.tool_call_id);
|
|
126
|
+
if (name.length === 0 || id.length === 0) return none();
|
|
127
|
+
const parameters = isRecord(event?.parameters) ? event.parameters : {};
|
|
128
|
+
const call = {
|
|
129
|
+
type: "toolCall",
|
|
130
|
+
id,
|
|
131
|
+
name,
|
|
132
|
+
arguments: parameters
|
|
133
|
+
};
|
|
134
|
+
return { events: [{ type: "tool_call", call, blocking: true }], replies: [] };
|
|
135
|
+
}
|
|
136
|
+
function complete(turn, _message) {
|
|
137
|
+
turn.dropAudio = false;
|
|
138
|
+
return { events: [done("agent", false)], replies: [] };
|
|
139
|
+
}
|
|
140
|
+
function clientError(_turn, message) {
|
|
141
|
+
const event = isRecord(message.client_error_event) ? message.client_error_event : void 0;
|
|
142
|
+
const reason = text(event?.message);
|
|
143
|
+
return {
|
|
144
|
+
events: [{ type: "error", error: reason.length > 0 ? reason : "client error", fatal: false }],
|
|
145
|
+
replies: []
|
|
146
|
+
};
|
|
147
|
+
}
|
|
148
|
+
const handlers = {
|
|
149
|
+
ping: pong,
|
|
150
|
+
conversation_initiation_metadata: metadata,
|
|
151
|
+
user_transcript: userTranscript,
|
|
152
|
+
agent_chat_response_part: chatPart,
|
|
153
|
+
agent_response: agentResponse,
|
|
154
|
+
agent_response_correction: correction,
|
|
155
|
+
audio,
|
|
156
|
+
interruption,
|
|
157
|
+
vad_score: vad,
|
|
158
|
+
client_tool_call: toolCall,
|
|
159
|
+
agent_response_complete: complete,
|
|
160
|
+
client_error: clientError
|
|
161
|
+
};
|
|
162
|
+
function ingestElevenLabs(turn, message, format) {
|
|
163
|
+
const type = message.type;
|
|
164
|
+
if (typeof type !== "string") return none();
|
|
165
|
+
const handler = handlers[type];
|
|
166
|
+
if (handler === void 0) return none();
|
|
167
|
+
return handler(turn, message, format);
|
|
168
|
+
}
|
|
169
|
+
export {
|
|
170
|
+
ingestElevenLabs
|
|
171
|
+
};
|
|
172
|
+
//# sourceMappingURL=events.js.map
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
{
|
|
2
|
+
"version": 3,
|
|
3
|
+
"sources": ["../../src/elevenlabs-agents/events.ts"],
|
|
4
|
+
"sourcesContent": ["import type { JsonObject, ToolCall } from '@earendil-works/pi-ai';\nimport type { AudioFormat, CharAlignment } from '../audio.ts';\nimport { blankUsage } from '../blank-usage.ts';\nimport { decodeAudioBase64 } from '../convert.ts';\nimport type { RealtimeEvent } from '../realtime.ts';\n\nexport interface TalkTurn {\n vad: boolean;\n dropAudio: boolean;\n lastInterruptId: number;\n}\n\nexport interface Ingested {\n events: RealtimeEvent[];\n replies: JsonObject[];\n}\n\nfunction none(): Ingested {\n return { events: [], replies: [] };\n}\n\nfunction isRecord(value: unknown): value is Record<string, unknown> {\n return typeof value === 'object' && value !== null && !Array.isArray(value);\n}\n\nfunction text(value: unknown): string {\n return typeof value === 'string' ? value : '';\n}\n\nfunction transcript(itemId: string, value: string, final: boolean): RealtimeEvent {\n return { type: 'output_transcript', itemId, text: value, final };\n}\n\nfunction done(itemId: string, interrupted: boolean): RealtimeEvent {\n return { type: 'response_done', itemId, interrupted, usage: blankUsage() };\n}\n\nfunction numbers(value: unknown): number[] | undefined {\n if (!Array.isArray(value) || value.some((item) => typeof item !== 'number')) return undefined;\n return value as number[];\n}\n\nfunction alignmentOf(value: unknown): CharAlignment | undefined {\n if (!isRecord(value)) return undefined;\n const startMs = numbers(value.char_start_times_ms);\n const durationMs = numbers(value.char_durations_ms);\n if (startMs === undefined || durationMs === undefined) return undefined;\n return { startMs, durationMs };\n}\n\nfunction pong(_turn: TalkTurn, message: Record<string, unknown>): Ingested {\n const nested = isRecord(message.ping_event) ? message.ping_event : undefined;\n const raw = nested === undefined ? message.event_id : nested.event_id;\n const eventId = typeof raw === 'number' ? raw : 0;\n return { events: [], replies: [{ type: 'pong', event_id: eventId }] };\n}\n\nfunction metadata(_turn: TalkTurn, message: Record<string, unknown>): Ingested {\n const event = isRecord(message.conversation_initiation_metadata_event)\n ? message.conversation_initiation_metadata_event\n : undefined;\n const sessionId = text(event?.conversation_id);\n return {\n events: [\n { type: 'session_start', sessionId: sessionId.length > 0 ? sessionId : 'conversation' },\n ],\n replies: [],\n };\n}\n\nfunction userTranscript(_turn: TalkTurn, message: Record<string, unknown>): Ingested {\n const event = isRecord(message.user_transcription_event)\n ? message.user_transcription_event\n : undefined;\n return {\n events: [\n {\n type: 'input_transcript',\n itemId: 'user',\n text: text(event?.user_transcript),\n final: true,\n },\n ],\n replies: [],\n };\n}\n\nfunction chatPart(_turn: TalkTurn, message: Record<string, unknown>): Ingested {\n const part = isRecord(message.text_response_part) ? message.text_response_part : undefined;\n if (part === undefined) return none();\n const kind = text(part.type);\n if (kind === 'start' || kind.length === 0) return none();\n const itemId = text(part.response_id);\n return {\n events: [transcript(itemId.length > 0 ? itemId : 'agent', text(part.text), kind === 'stop')],\n replies: [],\n };\n}\n\nfunction agentResponse(_turn: TalkTurn, message: Record<string, unknown>): Ingested {\n const event = isRecord(message.agent_response_event) ? message.agent_response_event : undefined;\n return { events: [transcript('agent', text(event?.agent_response), true)], replies: [] };\n}\n\nfunction correction(_turn: TalkTurn, message: Record<string, unknown>): Ingested {\n const event = isRecord(message.agent_response_correction_event)\n ? message.agent_response_correction_event\n : undefined;\n return {\n events: [transcript('agent', text(event?.corrected_agent_response), true)],\n replies: [],\n };\n}\n\nfunction audio(turn: TalkTurn, message: Record<string, unknown>, format: AudioFormat): Ingested {\n const event = isRecord(message.audio_event) ? message.audio_event : undefined;\n const eventId = typeof event?.event_id === 'number' ? event.event_id : 0;\n if (turn.dropAudio || eventId <= turn.lastInterruptId) return none();\n const alignment = alignmentOf(event?.alignment);\n const chars = isRecord(event?.alignment) ? event.alignment.chars : undefined;\n const end = Array.isArray(chars) ? chars.length : 0;\n const itemId = String(eventId);\n const first = alignment?.startMs[0];\n const spoken: RealtimeEvent = {\n type: 'output_audio',\n itemId,\n chunk: {\n data: decodeAudioBase64(text(event?.audio_base_64)),\n format,\n timestamp: typeof first === 'number' ? first : 0,\n },\n };\n if (alignment !== undefined) spoken.alignment = alignment;\n if (end > 0) spoken.span = { itemId, start: 0, end };\n return { events: [spoken], replies: [] };\n}\n\nfunction interruption(turn: TalkTurn, message: Record<string, unknown>): Ingested {\n const event = isRecord(message.interruption_event) ? message.interruption_event : undefined;\n const eventId = typeof event?.event_id === 'number' ? event.event_id : turn.lastInterruptId;\n turn.lastInterruptId = eventId;\n turn.vad = true;\n return { events: [{ type: 'input_speech_start' }, done(String(eventId), true)], replies: [] };\n}\n\nfunction vad(turn: TalkTurn, message: Record<string, unknown>): Ingested {\n const event = isRecord(message.vad_score_event) ? message.vad_score_event : undefined;\n const score = typeof event?.vad_score === 'number' ? event.vad_score : 0;\n const speaking = score > 0.5;\n if (!speaking) {\n turn.vad = false;\n return none();\n }\n if (turn.vad) return none();\n turn.vad = true;\n return { events: [{ type: 'input_speech_start' }], replies: [] };\n}\n\nfunction toolCall(_turn: TalkTurn, message: Record<string, unknown>): Ingested {\n const event = isRecord(message.client_tool_call) ? message.client_tool_call : undefined;\n const name = text(event?.tool_name);\n const id = text(event?.tool_call_id);\n if (name.length === 0 || id.length === 0) return none();\n const parameters = isRecord(event?.parameters) ? event.parameters : {};\n const call: ToolCall = {\n type: 'toolCall',\n id,\n name,\n arguments: parameters as JsonObject,\n };\n return { events: [{ type: 'tool_call', call, blocking: true }], replies: [] };\n}\n\nfunction complete(turn: TalkTurn, _message: Record<string, unknown>): Ingested {\n turn.dropAudio = false;\n return { events: [done('agent', false)], replies: [] };\n}\n\nfunction clientError(_turn: TalkTurn, message: Record<string, unknown>): Ingested {\n const event = isRecord(message.client_error_event) ? message.client_error_event : undefined;\n const reason = text(event?.message);\n return {\n events: [{ type: 'error', error: reason.length > 0 ? reason : 'client error', fatal: false }],\n replies: [],\n };\n}\n\nconst handlers: Record<\n string,\n (turn: TalkTurn, message: Record<string, unknown>, format: AudioFormat) => Ingested\n> = {\n ping: pong,\n conversation_initiation_metadata: metadata,\n user_transcript: userTranscript,\n agent_chat_response_part: chatPart,\n agent_response: agentResponse,\n agent_response_correction: correction,\n audio,\n interruption,\n vad_score: vad,\n client_tool_call: toolCall,\n agent_response_complete: complete,\n client_error: clientError,\n};\n\nexport function ingestElevenLabs(\n turn: TalkTurn,\n message: Record<string, unknown>,\n format: AudioFormat\n): Ingested {\n const type = message.type;\n if (typeof type !== 'string') return none();\n const handler = handlers[type];\n if (handler === undefined) return none();\n return handler(turn, message, format);\n}\n"],
|
|
5
|
+
"mappings": "AAEA,SAAS,kBAAkB;AAC3B,SAAS,yBAAyB;AAclC,SAAS,OAAiB;AACxB,SAAO,EAAE,QAAQ,CAAC,GAAG,SAAS,CAAC,EAAE;AACnC;AAEA,SAAS,SAAS,OAAkD;AAClE,SAAO,OAAO,UAAU,YAAY,UAAU,QAAQ,CAAC,MAAM,QAAQ,KAAK;AAC5E;AAEA,SAAS,KAAK,OAAwB;AACpC,SAAO,OAAO,UAAU,WAAW,QAAQ;AAC7C;AAEA,SAAS,WAAW,QAAgB,OAAe,OAA+B;AAChF,SAAO,EAAE,MAAM,qBAAqB,QAAQ,MAAM,OAAO,MAAM;AACjE;AAEA,SAAS,KAAK,QAAgB,aAAqC;AACjE,SAAO,EAAE,MAAM,iBAAiB,QAAQ,aAAa,OAAO,WAAW,EAAE;AAC3E;AAEA,SAAS,QAAQ,OAAsC;AACrD,MAAI,CAAC,MAAM,QAAQ,KAAK,KAAK,MAAM,KAAK,CAAC,SAAS,OAAO,SAAS,QAAQ,EAAG,QAAO;AACpF,SAAO;AACT;AAEA,SAAS,YAAY,OAA2C;AAC9D,MAAI,CAAC,SAAS,KAAK,EAAG,QAAO;AAC7B,QAAM,UAAU,QAAQ,MAAM,mBAAmB;AACjD,QAAM,aAAa,QAAQ,MAAM,iBAAiB;AAClD,MAAI,YAAY,UAAa,eAAe,OAAW,QAAO;AAC9D,SAAO,EAAE,SAAS,WAAW;AAC/B;AAEA,SAAS,KAAK,OAAiB,SAA4C;AACzE,QAAM,SAAS,SAAS,QAAQ,UAAU,IAAI,QAAQ,aAAa;AACnE,QAAM,MAAM,WAAW,SAAY,QAAQ,WAAW,OAAO;AAC7D,QAAM,UAAU,OAAO,QAAQ,WAAW,MAAM;AAChD,SAAO,EAAE,QAAQ,CAAC,GAAG,SAAS,CAAC,EAAE,MAAM,QAAQ,UAAU,QAAQ,CAAC,EAAE;AACtE;AAEA,SAAS,SAAS,OAAiB,SAA4C;AAC7E,QAAM,QAAQ,SAAS,QAAQ,sCAAsC,IACjE,QAAQ,yCACR;AACJ,QAAM,YAAY,KAAK,OAAO,eAAe;AAC7C,SAAO;AAAA,IACL,QAAQ;AAAA,MACN,EAAE,MAAM,iBAAiB,WAAW,UAAU,SAAS,IAAI,YAAY,eAAe;AAAA,IACxF;AAAA,IACA,SAAS,CAAC;AAAA,EACZ;AACF;AAEA,SAAS,eAAe,OAAiB,SAA4C;AACnF,QAAM,QAAQ,SAAS,QAAQ,wBAAwB,IACnD,QAAQ,2BACR;AACJ,SAAO;AAAA,IACL,QAAQ;AAAA,MACN;AAAA,QACE,MAAM;AAAA,QACN,QAAQ;AAAA,QACR,MAAM,KAAK,OAAO,eAAe;AAAA,QACjC,OAAO;AAAA,MACT;AAAA,IACF;AAAA,IACA,SAAS,CAAC;AAAA,EACZ;AACF;AAEA,SAAS,SAAS,OAAiB,SAA4C;AAC7E,QAAM,OAAO,SAAS,QAAQ,kBAAkB,IAAI,QAAQ,qBAAqB;AACjF,MAAI,SAAS,OAAW,QAAO,KAAK;AACpC,QAAM,OAAO,KAAK,KAAK,IAAI;AAC3B,MAAI,SAAS,WAAW,KAAK,WAAW,EAAG,QAAO,KAAK;AACvD,QAAM,SAAS,KAAK,KAAK,WAAW;AACpC,SAAO;AAAA,IACL,QAAQ,CAAC,WAAW,OAAO,SAAS,IAAI,SAAS,SAAS,KAAK,KAAK,IAAI,GAAG,SAAS,MAAM,CAAC;AAAA,IAC3F,SAAS,CAAC;AAAA,EACZ;AACF;AAEA,SAAS,cAAc,OAAiB,SAA4C;AAClF,QAAM,QAAQ,SAAS,QAAQ,oBAAoB,IAAI,QAAQ,uBAAuB;AACtF,SAAO,EAAE,QAAQ,CAAC,WAAW,SAAS,KAAK,OAAO,cAAc,GAAG,IAAI,CAAC,GAAG,SAAS,CAAC,EAAE;AACzF;AAEA,SAAS,WAAW,OAAiB,SAA4C;AAC/E,QAAM,QAAQ,SAAS,QAAQ,+BAA+B,IAC1D,QAAQ,kCACR;AACJ,SAAO;AAAA,IACL,QAAQ,CAAC,WAAW,SAAS,KAAK,OAAO,wBAAwB,GAAG,IAAI,CAAC;AAAA,IACzE,SAAS,CAAC;AAAA,EACZ;AACF;AAEA,SAAS,MAAM,MAAgB,SAAkC,QAA+B;AAC9F,QAAM,QAAQ,SAAS,QAAQ,WAAW,IAAI,QAAQ,cAAc;AACpE,QAAM,UAAU,OAAO,OAAO,aAAa,WAAW,MAAM,WAAW;AACvE,MAAI,KAAK,aAAa,WAAW,KAAK,gBAAiB,QAAO,KAAK;AACnE,QAAM,YAAY,YAAY,OAAO,SAAS;AAC9C,QAAM,QAAQ,SAAS,OAAO,SAAS,IAAI,MAAM,UAAU,QAAQ;AACnE,QAAM,MAAM,MAAM,QAAQ,KAAK,IAAI,MAAM,SAAS;AAClD,QAAM,SAAS,OAAO,OAAO;AAC7B,QAAM,QAAQ,WAAW,QAAQ,CAAC;AAClC,QAAM,SAAwB;AAAA,IAC5B,MAAM;AAAA,IACN;AAAA,IACA,OAAO;AAAA,MACL,MAAM,kBAAkB,KAAK,OAAO,aAAa,CAAC;AAAA,MAClD;AAAA,MACA,WAAW,OAAO,UAAU,WAAW,QAAQ;AAAA,IACjD;AAAA,EACF;AACA,MAAI,cAAc,OAAW,QAAO,YAAY;AAChD,MAAI,MAAM,EAAG,QAAO,OAAO,EAAE,QAAQ,OAAO,GAAG,IAAI;AACnD,SAAO,EAAE,QAAQ,CAAC,MAAM,GAAG,SAAS,CAAC,EAAE;AACzC;AAEA,SAAS,aAAa,MAAgB,SAA4C;AAChF,QAAM,QAAQ,SAAS,QAAQ,kBAAkB,IAAI,QAAQ,qBAAqB;AAClF,QAAM,UAAU,OAAO,OAAO,aAAa,WAAW,MAAM,WAAW,KAAK;AAC5E,OAAK,kBAAkB;AACvB,OAAK,MAAM;AACX,SAAO,EAAE,QAAQ,CAAC,EAAE,MAAM,qBAAqB,GAAG,KAAK,OAAO,OAAO,GAAG,IAAI,CAAC,GAAG,SAAS,CAAC,EAAE;AAC9F;AAEA,SAAS,IAAI,MAAgB,SAA4C;AACvE,QAAM,QAAQ,SAAS,QAAQ,eAAe,IAAI,QAAQ,kBAAkB;AAC5E,QAAM,QAAQ,OAAO,OAAO,cAAc,WAAW,MAAM,YAAY;AACvE,QAAM,WAAW,QAAQ;AACzB,MAAI,CAAC,UAAU;AACb,SAAK,MAAM;AACX,WAAO,KAAK;AAAA,EACd;AACA,MAAI,KAAK,IAAK,QAAO,KAAK;AAC1B,OAAK,MAAM;AACX,SAAO,EAAE,QAAQ,CAAC,EAAE,MAAM,qBAAqB,CAAC,GAAG,SAAS,CAAC,EAAE;AACjE;AAEA,SAAS,SAAS,OAAiB,SAA4C;AAC7E,QAAM,QAAQ,SAAS,QAAQ,gBAAgB,IAAI,QAAQ,mBAAmB;AAC9E,QAAM,OAAO,KAAK,OAAO,SAAS;AAClC,QAAM,KAAK,KAAK,OAAO,YAAY;AACnC,MAAI,KAAK,WAAW,KAAK,GAAG,WAAW,EAAG,QAAO,KAAK;AACtD,QAAM,aAAa,SAAS,OAAO,UAAU,IAAI,MAAM,aAAa,CAAC;AACrE,QAAM,OAAiB;AAAA,IACrB,MAAM;AAAA,IACN;AAAA,IACA;AAAA,IACA,WAAW;AAAA,EACb;AACA,SAAO,EAAE,QAAQ,CAAC,EAAE,MAAM,aAAa,MAAM,UAAU,KAAK,CAAC,GAAG,SAAS,CAAC,EAAE;AAC9E;AAEA,SAAS,SAAS,MAAgB,UAA6C;AAC7E,OAAK,YAAY;AACjB,SAAO,EAAE,QAAQ,CAAC,KAAK,SAAS,KAAK,CAAC,GAAG,SAAS,CAAC,EAAE;AACvD;AAEA,SAAS,YAAY,OAAiB,SAA4C;AAChF,QAAM,QAAQ,SAAS,QAAQ,kBAAkB,IAAI,QAAQ,qBAAqB;AAClF,QAAM,SAAS,KAAK,OAAO,OAAO;AAClC,SAAO;AAAA,IACL,QAAQ,CAAC,EAAE,MAAM,SAAS,OAAO,OAAO,SAAS,IAAI,SAAS,gBAAgB,OAAO,MAAM,CAAC;AAAA,IAC5F,SAAS,CAAC;AAAA,EACZ;AACF;AAEA,MAAM,WAGF;AAAA,EACF,MAAM;AAAA,EACN,kCAAkC;AAAA,EAClC,iBAAiB;AAAA,EACjB,0BAA0B;AAAA,EAC1B,gBAAgB;AAAA,EAChB,2BAA2B;AAAA,EAC3B;AAAA,EACA;AAAA,EACA,WAAW;AAAA,EACX,kBAAkB;AAAA,EAClB,yBAAyB;AAAA,EACzB,cAAc;AAChB;AAEO,SAAS,iBACd,MACA,SACA,QACU;AACV,QAAM,OAAO,QAAQ;AACrB,MAAI,OAAO,SAAS,SAAU,QAAO,KAAK;AAC1C,QAAM,UAAU,SAAS,IAAI;AAC7B,MAAI,YAAY,OAAW,QAAO,KAAK;AACvC,SAAO,QAAQ,MAAM,SAAS,MAAM;AACtC;",
|
|
6
|
+
"names": []
|
|
7
|
+
}
|
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
import { registerVoiceModel } from "../models.js";
|
|
2
|
+
import { registerRealtimeApiProvider } from "../registry.js";
|
|
3
|
+
import { connectElevenLabs } from "./session.js";
|
|
4
|
+
const voices = [
|
|
5
|
+
{ id: "21m00Tcm4TlvDq8ikWAM", name: "Rachel" },
|
|
6
|
+
{ id: "EXAVITQu4vr4xnSDxMaL", name: "Sarah" },
|
|
7
|
+
{ id: "pNInz6obpgDQGcFmaJgB", name: "Adam" }
|
|
8
|
+
];
|
|
9
|
+
function pcm16(sampleRate) {
|
|
10
|
+
return { encoding: "pcm16", sampleRate, channels: 1 };
|
|
11
|
+
}
|
|
12
|
+
const elevenLabsAgent = {
|
|
13
|
+
id: "elevenlabs-agent",
|
|
14
|
+
name: "ElevenLabs Agent",
|
|
15
|
+
api: "elevenlabs-agents",
|
|
16
|
+
provider: "elevenlabs",
|
|
17
|
+
baseUrl: "wss://api.elevenlabs.io/v1/convai/conversation",
|
|
18
|
+
type: "realtime",
|
|
19
|
+
transports: ["websocket"],
|
|
20
|
+
toolCalls: "blocking",
|
|
21
|
+
inputFormat: pcm16(16e3),
|
|
22
|
+
outputFormat: pcm16(16e3),
|
|
23
|
+
maxSessionMs: 6e5,
|
|
24
|
+
resumable: false,
|
|
25
|
+
ephemeralAuth: true,
|
|
26
|
+
voices,
|
|
27
|
+
cascade: { stt: "scribe_realtime", llm: "gemini-2.5-flash", tts: "eleven_flash_v2_5" },
|
|
28
|
+
provisioning: "agent",
|
|
29
|
+
delivery: { interrupt: "approximate", when_idle: "approximate", silent: "native" },
|
|
30
|
+
truncation: "server",
|
|
31
|
+
liveToolUpdate: false,
|
|
32
|
+
toolTimeoutMs: { min: 1e3, max: 12e4 },
|
|
33
|
+
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
|
|
34
|
+
audioCost: { perInputMinute: 0.08 }
|
|
35
|
+
};
|
|
36
|
+
function registerElevenLabsAgents(options = {}) {
|
|
37
|
+
const unregisterModel = registerVoiceModel(elevenLabsAgent);
|
|
38
|
+
const unregisterApi = registerRealtimeApiProvider({
|
|
39
|
+
api: "elevenlabs-agents",
|
|
40
|
+
connect: (model, context, connectOptions) => connectElevenLabs(model, context, connectOptions, options)
|
|
41
|
+
});
|
|
42
|
+
return () => {
|
|
43
|
+
unregisterModel();
|
|
44
|
+
unregisterApi();
|
|
45
|
+
};
|
|
46
|
+
}
|
|
47
|
+
import { hashHostedSpec, memoryHostedCache } from "../hosted-agent.js";
|
|
48
|
+
import { agentBody, httpsOrigin, provisionAgent } from "./provision.js";
|
|
49
|
+
export {
|
|
50
|
+
agentBody,
|
|
51
|
+
elevenLabsAgent,
|
|
52
|
+
hashHostedSpec,
|
|
53
|
+
httpsOrigin,
|
|
54
|
+
memoryHostedCache,
|
|
55
|
+
provisionAgent,
|
|
56
|
+
registerElevenLabsAgents
|
|
57
|
+
};
|
|
58
|
+
//# sourceMappingURL=index.js.map
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
{
|
|
2
|
+
"version": 3,
|
|
3
|
+
"sources": ["../../src/elevenlabs-agents/index.ts"],
|
|
4
|
+
"sourcesContent": ["import type { RealtimeModel } from '../models.ts';\nimport { registerVoiceModel } from '../models.ts';\nimport { registerRealtimeApiProvider } from '../registry.ts';\nimport type { ElevenLabsOptions } from './session.ts';\nimport { connectElevenLabs } from './session.ts';\n\nconst voices = [\n { id: '21m00Tcm4TlvDq8ikWAM', name: 'Rachel' },\n { id: 'EXAVITQu4vr4xnSDxMaL', name: 'Sarah' },\n { id: 'pNInz6obpgDQGcFmaJgB', name: 'Adam' },\n];\n\nfunction pcm16(sampleRate: number) {\n return { encoding: 'pcm16' as const, sampleRate, channels: 1 as const };\n}\n\nexport const elevenLabsAgent: RealtimeModel<'elevenlabs-agents'> = {\n id: 'elevenlabs-agent',\n name: 'ElevenLabs Agent',\n api: 'elevenlabs-agents',\n provider: 'elevenlabs',\n baseUrl: 'wss://api.elevenlabs.io/v1/convai/conversation',\n type: 'realtime',\n transports: ['websocket'],\n toolCalls: 'blocking',\n inputFormat: pcm16(16000),\n outputFormat: pcm16(16000),\n maxSessionMs: 600_000,\n resumable: false,\n ephemeralAuth: true,\n voices,\n cascade: { stt: 'scribe_realtime', llm: 'gemini-2.5-flash', tts: 'eleven_flash_v2_5' },\n provisioning: 'agent',\n delivery: { interrupt: 'approximate', when_idle: 'approximate', silent: 'native' },\n truncation: 'server',\n liveToolUpdate: false,\n toolTimeoutMs: { min: 1000, max: 120_000 },\n cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },\n audioCost: { perInputMinute: 0.08 },\n};\n\nexport function registerElevenLabsAgents(options: ElevenLabsOptions = {}): () => void {\n const unregisterModel = registerVoiceModel(elevenLabsAgent);\n const unregisterApi = registerRealtimeApiProvider({\n api: 'elevenlabs-agents',\n connect: (model, context, connectOptions) =>\n connectElevenLabs(model, context, connectOptions, options),\n });\n return () => {\n unregisterModel();\n unregisterApi();\n };\n}\n\nexport type { CustomLlm, HostedAgent, HostedAgentSpec, HostedCache } from '../hosted-agent.ts';\nexport { hashHostedSpec, memoryHostedCache } from '../hosted-agent.ts';\nexport type { ProvisionOptions } from './provision.ts';\nexport { agentBody, httpsOrigin, provisionAgent } from './provision.ts';\nexport type { ElevenLabsOptions, VoiceWireFactory } from './session.ts';\n"],
|
|
5
|
+
"mappings": "AACA,SAAS,0BAA0B;AACnC,SAAS,mCAAmC;AAE5C,SAAS,yBAAyB;AAElC,MAAM,SAAS;AAAA,EACb,EAAE,IAAI,wBAAwB,MAAM,SAAS;AAAA,EAC7C,EAAE,IAAI,wBAAwB,MAAM,QAAQ;AAAA,EAC5C,EAAE,IAAI,wBAAwB,MAAM,OAAO;AAC7C;AAEA,SAAS,MAAM,YAAoB;AACjC,SAAO,EAAE,UAAU,SAAkB,YAAY,UAAU,EAAW;AACxE;AAEO,MAAM,kBAAsD;AAAA,EACjE,IAAI;AAAA,EACJ,MAAM;AAAA,EACN,KAAK;AAAA,EACL,UAAU;AAAA,EACV,SAAS;AAAA,EACT,MAAM;AAAA,EACN,YAAY,CAAC,WAAW;AAAA,EACxB,WAAW;AAAA,EACX,aAAa,MAAM,IAAK;AAAA,EACxB,cAAc,MAAM,IAAK;AAAA,EACzB,cAAc;AAAA,EACd,WAAW;AAAA,EACX,eAAe;AAAA,EACf;AAAA,EACA,SAAS,EAAE,KAAK,mBAAmB,KAAK,oBAAoB,KAAK,oBAAoB;AAAA,EACrF,cAAc;AAAA,EACd,UAAU,EAAE,WAAW,eAAe,WAAW,eAAe,QAAQ,SAAS;AAAA,EACjF,YAAY;AAAA,EACZ,gBAAgB;AAAA,EAChB,eAAe,EAAE,KAAK,KAAM,KAAK,KAAQ;AAAA,EACzC,MAAM,EAAE,OAAO,GAAG,QAAQ,GAAG,WAAW,GAAG,YAAY,EAAE;AAAA,EACzD,WAAW,EAAE,gBAAgB,KAAK;AACpC;AAEO,SAAS,yBAAyB,UAA6B,CAAC,GAAe;AACpF,QAAM,kBAAkB,mBAAmB,eAAe;AAC1D,QAAM,gBAAgB,4BAA4B;AAAA,IAChD,KAAK;AAAA,IACL,SAAS,CAAC,OAAO,SAAS,mBACxB,kBAAkB,OAAO,SAAS,gBAAgB,OAAO;AAAA,EAC7D,CAAC;AACD,SAAO,MAAM;AACX,oBAAgB;AAChB,kBAAc;AAAA,EAChB;AACF;AAGA,SAAS,gBAAgB,yBAAyB;AAElD,SAAS,WAAW,aAAa,sBAAsB;",
|
|
6
|
+
"names": []
|
|
7
|
+
}
|
|
@@ -0,0 +1,167 @@
|
|
|
1
|
+
import { VoiceRequestError } from "../errors.js";
|
|
2
|
+
import {
|
|
3
|
+
hashHostedSpec,
|
|
4
|
+
hostedCacheKey,
|
|
5
|
+
memoryHostedCache
|
|
6
|
+
} from "../hosted-agent.js";
|
|
7
|
+
const defaultCache = memoryHostedCache();
|
|
8
|
+
const clientEvents = [
|
|
9
|
+
"conversation_initiation_metadata",
|
|
10
|
+
"ping",
|
|
11
|
+
"audio",
|
|
12
|
+
"interruption",
|
|
13
|
+
"user_transcript",
|
|
14
|
+
"agent_response",
|
|
15
|
+
"agent_response_correction",
|
|
16
|
+
"client_tool_call",
|
|
17
|
+
"vad_score",
|
|
18
|
+
"agent_chat_response_part",
|
|
19
|
+
"agent_response_complete"
|
|
20
|
+
];
|
|
21
|
+
function isRecord(value) {
|
|
22
|
+
return typeof value === "object" && value !== null && !Array.isArray(value);
|
|
23
|
+
}
|
|
24
|
+
function asVoice(error) {
|
|
25
|
+
if (error instanceof VoiceRequestError) return error;
|
|
26
|
+
if (error instanceof Error && error.name === "AbortError") {
|
|
27
|
+
return new VoiceRequestError("aborted", "session aborted");
|
|
28
|
+
}
|
|
29
|
+
if (error instanceof Error) return new VoiceRequestError("error", error.message);
|
|
30
|
+
return new VoiceRequestError("error", String(error));
|
|
31
|
+
}
|
|
32
|
+
function httpsOrigin(baseUrl) {
|
|
33
|
+
const url = new URL(baseUrl);
|
|
34
|
+
let protocol = url.protocol;
|
|
35
|
+
if (protocol === "wss:") protocol = "https:";
|
|
36
|
+
if (protocol === "ws:") protocol = "http:";
|
|
37
|
+
return `${protocol}//${url.host}`;
|
|
38
|
+
}
|
|
39
|
+
function mergedHeaders(options, defaults) {
|
|
40
|
+
const headers = { ...defaults };
|
|
41
|
+
const extra = options.headers;
|
|
42
|
+
if (extra === void 0) return headers;
|
|
43
|
+
for (const [key, value] of Object.entries(extra)) {
|
|
44
|
+
if (value === null) delete headers[key];
|
|
45
|
+
else headers[key] = value;
|
|
46
|
+
}
|
|
47
|
+
return headers;
|
|
48
|
+
}
|
|
49
|
+
function apiKey(options) {
|
|
50
|
+
const key = options.apiKey;
|
|
51
|
+
if (typeof key === "string" && key.length > 0) return key;
|
|
52
|
+
throw new VoiceRequestError("error", "missing realtime credential");
|
|
53
|
+
}
|
|
54
|
+
function timeoutSeconds(timeoutMs) {
|
|
55
|
+
if (timeoutMs === void 0) return 20;
|
|
56
|
+
return Math.min(120, Math.max(1, Math.round(timeoutMs / 1e3)));
|
|
57
|
+
}
|
|
58
|
+
function clientTool(tool, timeoutMs) {
|
|
59
|
+
return {
|
|
60
|
+
type: "client",
|
|
61
|
+
name: tool.name,
|
|
62
|
+
description: tool.description,
|
|
63
|
+
expects_response: true,
|
|
64
|
+
response_timeout_secs: timeoutSeconds(timeoutMs),
|
|
65
|
+
parameters: tool.parameters
|
|
66
|
+
};
|
|
67
|
+
}
|
|
68
|
+
function customLlmFields(llm) {
|
|
69
|
+
const custom = {
|
|
70
|
+
url: llm.baseUrl,
|
|
71
|
+
model_id: llm.id,
|
|
72
|
+
api_type: llm.api === "openai-responses" ? "responses" : "chat_completions"
|
|
73
|
+
};
|
|
74
|
+
const headers = llm.headers;
|
|
75
|
+
if (headers !== void 0 && Object.keys(headers).length > 0) custom.request_headers = headers;
|
|
76
|
+
return { llm: "custom-llm", custom_llm: custom };
|
|
77
|
+
}
|
|
78
|
+
function promptFields(spec) {
|
|
79
|
+
const llm = typeof spec.llm === "string" ? { llm: spec.llm } : customLlmFields(spec.llm);
|
|
80
|
+
return {
|
|
81
|
+
prompt: spec.instructions,
|
|
82
|
+
tools: spec.tools.map((tool) => clientTool(tool, spec.clientToolTimeoutMs)),
|
|
83
|
+
...llm
|
|
84
|
+
};
|
|
85
|
+
}
|
|
86
|
+
function agentBody(spec) {
|
|
87
|
+
const agent = { prompt: promptFields(spec) };
|
|
88
|
+
if (spec.firstMessage !== void 0) agent.first_message = spec.firstMessage;
|
|
89
|
+
return {
|
|
90
|
+
name: spec.name,
|
|
91
|
+
conversation_config: {
|
|
92
|
+
agent,
|
|
93
|
+
tts: { voice_id: spec.voice },
|
|
94
|
+
conversation: { client_events: clientEvents }
|
|
95
|
+
},
|
|
96
|
+
platform_settings: {
|
|
97
|
+
overrides: {
|
|
98
|
+
conversation_config_override: {
|
|
99
|
+
agent: { prompt: { prompt: true }, first_message: true },
|
|
100
|
+
tts: { voice_id: true }
|
|
101
|
+
},
|
|
102
|
+
custom_llm_extra_body: true
|
|
103
|
+
}
|
|
104
|
+
}
|
|
105
|
+
};
|
|
106
|
+
}
|
|
107
|
+
async function requestJson(fetchImpl, url, init) {
|
|
108
|
+
let response;
|
|
109
|
+
try {
|
|
110
|
+
response = await fetchImpl(url, init);
|
|
111
|
+
} catch (error) {
|
|
112
|
+
throw asVoice(error);
|
|
113
|
+
}
|
|
114
|
+
if (!response.ok) {
|
|
115
|
+
throw new VoiceRequestError("error", `agent request failed (${response.status})`);
|
|
116
|
+
}
|
|
117
|
+
let json;
|
|
118
|
+
try {
|
|
119
|
+
json = await response.json();
|
|
120
|
+
} catch (error) {
|
|
121
|
+
throw asVoice(error);
|
|
122
|
+
}
|
|
123
|
+
if (!isRecord(json)) throw new VoiceRequestError("error", "agent response was not an object");
|
|
124
|
+
return json;
|
|
125
|
+
}
|
|
126
|
+
function agentIdFrom(json, fallback) {
|
|
127
|
+
const id = json.agent_id;
|
|
128
|
+
if (typeof id === "string" && id.length > 0) return id;
|
|
129
|
+
if (fallback !== void 0) return fallback;
|
|
130
|
+
throw new VoiceRequestError("error", "agent response had no agent_id");
|
|
131
|
+
}
|
|
132
|
+
async function provisionAgent(model, spec, options = {}) {
|
|
133
|
+
if (options.signal?.aborted === true) throw new VoiceRequestError("aborted", "session aborted");
|
|
134
|
+
const cache = options.cache ?? defaultCache;
|
|
135
|
+
const now = options.now ?? Date.now;
|
|
136
|
+
const key = hostedCacheKey(model.provider, spec.name);
|
|
137
|
+
const hash = hashHostedSpec(spec);
|
|
138
|
+
const cached = cache.get(key);
|
|
139
|
+
if (cached !== void 0 && cached.specHash === hash) return cached;
|
|
140
|
+
const existing = cached?.agentId ?? options.agentId;
|
|
141
|
+
const origin = httpsOrigin(model.baseUrl);
|
|
142
|
+
const url = existing === void 0 ? `${origin}/v1/convai/agents/create` : `${origin}/v1/convai/agents/${encodeURIComponent(existing)}`;
|
|
143
|
+
const headers = mergedHeaders(options, {
|
|
144
|
+
"Content-Type": "application/json",
|
|
145
|
+
"xi-api-key": apiKey(options)
|
|
146
|
+
});
|
|
147
|
+
const fetchImpl = options.fetch ?? globalThis.fetch;
|
|
148
|
+
const json = await requestJson(fetchImpl, url, {
|
|
149
|
+
method: existing === void 0 ? "POST" : "PATCH",
|
|
150
|
+
headers,
|
|
151
|
+
body: JSON.stringify(agentBody(spec)),
|
|
152
|
+
signal: options.signal
|
|
153
|
+
});
|
|
154
|
+
const agent = {
|
|
155
|
+
agentId: agentIdFrom(json, existing),
|
|
156
|
+
specHash: hash,
|
|
157
|
+
publishedAt: now()
|
|
158
|
+
};
|
|
159
|
+
cache.set(key, agent);
|
|
160
|
+
return agent;
|
|
161
|
+
}
|
|
162
|
+
export {
|
|
163
|
+
agentBody,
|
|
164
|
+
httpsOrigin,
|
|
165
|
+
provisionAgent
|
|
166
|
+
};
|
|
167
|
+
//# sourceMappingURL=provision.js.map
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
{
|
|
2
|
+
"version": 3,
|
|
3
|
+
"sources": ["../../src/elevenlabs-agents/provision.ts"],
|
|
4
|
+
"sourcesContent": ["import type { JsonObject, ProviderRequestOptions, Tool } from '@earendil-works/pi-ai';\nimport { VoiceRequestError } from '../errors.ts';\nimport {\n type CustomLlm,\n type HostedAgent,\n type HostedAgentSpec,\n type HostedCache,\n hashHostedSpec,\n hostedCacheKey,\n memoryHostedCache,\n} from '../hosted-agent.ts';\nimport type { RealtimeModel } from '../models.ts';\n\nexport interface ProvisionOptions extends ProviderRequestOptions {\n agentId?: string;\n now?: () => number;\n cache?: HostedCache;\n}\n\nconst defaultCache = memoryHostedCache();\n\nconst clientEvents = [\n 'conversation_initiation_metadata',\n 'ping',\n 'audio',\n 'interruption',\n 'user_transcript',\n 'agent_response',\n 'agent_response_correction',\n 'client_tool_call',\n 'vad_score',\n 'agent_chat_response_part',\n 'agent_response_complete',\n];\n\nfunction isRecord(value: unknown): value is Record<string, unknown> {\n return typeof value === 'object' && value !== null && !Array.isArray(value);\n}\n\nfunction asVoice(error: unknown): VoiceRequestError {\n if (error instanceof VoiceRequestError) return error;\n if (error instanceof Error && error.name === 'AbortError') {\n return new VoiceRequestError('aborted', 'session aborted');\n }\n if (error instanceof Error) return new VoiceRequestError('error', error.message);\n return new VoiceRequestError('error', String(error));\n}\n\nexport function httpsOrigin(baseUrl: string): string {\n const url = new URL(baseUrl);\n let protocol = url.protocol;\n if (protocol === 'wss:') protocol = 'https:';\n if (protocol === 'ws:') protocol = 'http:';\n return `${protocol}//${url.host}`;\n}\n\nfunction mergedHeaders(\n options: ProvisionOptions,\n defaults: Record<string, string>\n): Record<string, string> {\n const headers = { ...defaults };\n const extra = options.headers;\n if (extra === undefined) return headers;\n for (const [key, value] of Object.entries(extra)) {\n if (value === null) delete headers[key];\n else headers[key] = value;\n }\n return headers;\n}\n\nfunction apiKey(options: ProvisionOptions): string {\n const key = options.apiKey;\n if (typeof key === 'string' && key.length > 0) return key;\n throw new VoiceRequestError('error', 'missing realtime credential');\n}\n\nfunction timeoutSeconds(timeoutMs: number | undefined): number {\n if (timeoutMs === undefined) return 20;\n return Math.min(120, Math.max(1, Math.round(timeoutMs / 1000)));\n}\n\nfunction clientTool(tool: Tool, timeoutMs: number | undefined): JsonObject {\n return {\n type: 'client',\n name: tool.name,\n description: tool.description,\n expects_response: true,\n response_timeout_secs: timeoutSeconds(timeoutMs),\n parameters: tool.parameters as unknown as JsonObject,\n };\n}\n\nfunction customLlmFields(llm: CustomLlm): JsonObject {\n const custom: JsonObject = {\n url: llm.baseUrl,\n model_id: llm.id,\n api_type: llm.api === 'openai-responses' ? 'responses' : 'chat_completions',\n };\n const headers = llm.headers;\n if (headers !== undefined && Object.keys(headers).length > 0) custom.request_headers = headers;\n return { llm: 'custom-llm', custom_llm: custom };\n}\n\nfunction promptFields(spec: HostedAgentSpec): JsonObject {\n const llm = typeof spec.llm === 'string' ? { llm: spec.llm } : customLlmFields(spec.llm);\n return {\n prompt: spec.instructions,\n tools: spec.tools.map((tool) => clientTool(tool, spec.clientToolTimeoutMs)),\n ...llm,\n };\n}\n\nexport function agentBody(spec: HostedAgentSpec): JsonObject {\n const agent: JsonObject = { prompt: promptFields(spec) };\n if (spec.firstMessage !== undefined) agent.first_message = spec.firstMessage;\n return {\n name: spec.name,\n conversation_config: {\n agent,\n tts: { voice_id: spec.voice },\n conversation: { client_events: clientEvents },\n },\n platform_settings: {\n overrides: {\n conversation_config_override: {\n agent: { prompt: { prompt: true }, first_message: true },\n tts: { voice_id: true },\n },\n custom_llm_extra_body: true,\n },\n },\n };\n}\n\nasync function requestJson(\n fetchImpl: typeof fetch,\n url: string,\n init: RequestInit\n): Promise<Record<string, unknown>> {\n let response: Response;\n try {\n response = await fetchImpl(url, init);\n } catch (error) {\n throw asVoice(error);\n }\n if (!response.ok) {\n throw new VoiceRequestError('error', `agent request failed (${response.status})`);\n }\n let json: unknown;\n try {\n json = await response.json();\n } catch (error) {\n throw asVoice(error);\n }\n if (!isRecord(json)) throw new VoiceRequestError('error', 'agent response was not an object');\n return json;\n}\n\nfunction agentIdFrom(json: Record<string, unknown>, fallback: string | undefined): string {\n const id = json.agent_id;\n if (typeof id === 'string' && id.length > 0) return id;\n if (fallback !== undefined) return fallback;\n throw new VoiceRequestError('error', 'agent response had no agent_id');\n}\n\nexport async function provisionAgent(\n model: RealtimeModel<string>,\n spec: HostedAgentSpec,\n options: ProvisionOptions = {}\n): Promise<HostedAgent> {\n if (options.signal?.aborted === true) throw new VoiceRequestError('aborted', 'session aborted');\n const cache = options.cache ?? defaultCache;\n const now = options.now ?? Date.now;\n const key = hostedCacheKey(model.provider, spec.name);\n const hash = hashHostedSpec(spec);\n const cached = cache.get(key);\n if (cached !== undefined && cached.specHash === hash) return cached;\n const existing = cached?.agentId ?? options.agentId;\n const origin = httpsOrigin(model.baseUrl);\n const url =\n existing === undefined\n ? `${origin}/v1/convai/agents/create`\n : `${origin}/v1/convai/agents/${encodeURIComponent(existing)}`;\n const headers = mergedHeaders(options, {\n 'Content-Type': 'application/json',\n 'xi-api-key': apiKey(options),\n });\n const fetchImpl = options.fetch ?? globalThis.fetch;\n const json = await requestJson(fetchImpl, url, {\n method: existing === undefined ? 'POST' : 'PATCH',\n headers,\n body: JSON.stringify(agentBody(spec)),\n signal: options.signal,\n });\n const agent: HostedAgent = {\n agentId: agentIdFrom(json, existing),\n specHash: hash,\n publishedAt: now(),\n };\n cache.set(key, agent);\n return agent;\n}\n"],
|
|
5
|
+
"mappings": "AACA,SAAS,yBAAyB;AAClC;AAAA,EAKE;AAAA,EACA;AAAA,EACA;AAAA,OACK;AASP,MAAM,eAAe,kBAAkB;AAEvC,MAAM,eAAe;AAAA,EACnB;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AACF;AAEA,SAAS,SAAS,OAAkD;AAClE,SAAO,OAAO,UAAU,YAAY,UAAU,QAAQ,CAAC,MAAM,QAAQ,KAAK;AAC5E;AAEA,SAAS,QAAQ,OAAmC;AAClD,MAAI,iBAAiB,kBAAmB,QAAO;AAC/C,MAAI,iBAAiB,SAAS,MAAM,SAAS,cAAc;AACzD,WAAO,IAAI,kBAAkB,WAAW,iBAAiB;AAAA,EAC3D;AACA,MAAI,iBAAiB,MAAO,QAAO,IAAI,kBAAkB,SAAS,MAAM,OAAO;AAC/E,SAAO,IAAI,kBAAkB,SAAS,OAAO,KAAK,CAAC;AACrD;AAEO,SAAS,YAAY,SAAyB;AACnD,QAAM,MAAM,IAAI,IAAI,OAAO;AAC3B,MAAI,WAAW,IAAI;AACnB,MAAI,aAAa,OAAQ,YAAW;AACpC,MAAI,aAAa,MAAO,YAAW;AACnC,SAAO,GAAG,QAAQ,KAAK,IAAI,IAAI;AACjC;AAEA,SAAS,cACP,SACA,UACwB;AACxB,QAAM,UAAU,EAAE,GAAG,SAAS;AAC9B,QAAM,QAAQ,QAAQ;AACtB,MAAI,UAAU,OAAW,QAAO;AAChC,aAAW,CAAC,KAAK,KAAK,KAAK,OAAO,QAAQ,KAAK,GAAG;AAChD,QAAI,UAAU,KAAM,QAAO,QAAQ,GAAG;AAAA,QACjC,SAAQ,GAAG,IAAI;AAAA,EACtB;AACA,SAAO;AACT;AAEA,SAAS,OAAO,SAAmC;AACjD,QAAM,MAAM,QAAQ;AACpB,MAAI,OAAO,QAAQ,YAAY,IAAI,SAAS,EAAG,QAAO;AACtD,QAAM,IAAI,kBAAkB,SAAS,6BAA6B;AACpE;AAEA,SAAS,eAAe,WAAuC;AAC7D,MAAI,cAAc,OAAW,QAAO;AACpC,SAAO,KAAK,IAAI,KAAK,KAAK,IAAI,GAAG,KAAK,MAAM,YAAY,GAAI,CAAC,CAAC;AAChE;AAEA,SAAS,WAAW,MAAY,WAA2C;AACzE,SAAO;AAAA,IACL,MAAM;AAAA,IACN,MAAM,KAAK;AAAA,IACX,aAAa,KAAK;AAAA,IAClB,kBAAkB;AAAA,IAClB,uBAAuB,eAAe,SAAS;AAAA,IAC/C,YAAY,KAAK;AAAA,EACnB;AACF;AAEA,SAAS,gBAAgB,KAA4B;AACnD,QAAM,SAAqB;AAAA,IACzB,KAAK,IAAI;AAAA,IACT,UAAU,IAAI;AAAA,IACd,UAAU,IAAI,QAAQ,qBAAqB,cAAc;AAAA,EAC3D;AACA,QAAM,UAAU,IAAI;AACpB,MAAI,YAAY,UAAa,OAAO,KAAK,OAAO,EAAE,SAAS,EAAG,QAAO,kBAAkB;AACvF,SAAO,EAAE,KAAK,cAAc,YAAY,OAAO;AACjD;AAEA,SAAS,aAAa,MAAmC;AACvD,QAAM,MAAM,OAAO,KAAK,QAAQ,WAAW,EAAE,KAAK,KAAK,IAAI,IAAI,gBAAgB,KAAK,GAAG;AACvF,SAAO;AAAA,IACL,QAAQ,KAAK;AAAA,IACb,OAAO,KAAK,MAAM,IAAI,CAAC,SAAS,WAAW,MAAM,KAAK,mBAAmB,CAAC;AAAA,IAC1E,GAAG;AAAA,EACL;AACF;AAEO,SAAS,UAAU,MAAmC;AAC3D,QAAM,QAAoB,EAAE,QAAQ,aAAa,IAAI,EAAE;AACvD,MAAI,KAAK,iBAAiB,OAAW,OAAM,gBAAgB,KAAK;AAChE,SAAO;AAAA,IACL,MAAM,KAAK;AAAA,IACX,qBAAqB;AAAA,MACnB;AAAA,MACA,KAAK,EAAE,UAAU,KAAK,MAAM;AAAA,MAC5B,cAAc,EAAE,eAAe,aAAa;AAAA,IAC9C;AAAA,IACA,mBAAmB;AAAA,MACjB,WAAW;AAAA,QACT,8BAA8B;AAAA,UAC5B,OAAO,EAAE,QAAQ,EAAE,QAAQ,KAAK,GAAG,eAAe,KAAK;AAAA,UACvD,KAAK,EAAE,UAAU,KAAK;AAAA,QACxB;AAAA,QACA,uBAAuB;AAAA,MACzB;AAAA,IACF;AAAA,EACF;AACF;AAEA,eAAe,YACb,WACA,KACA,MACkC;AAClC,MAAI;AACJ,MAAI;AACF,eAAW,MAAM,UAAU,KAAK,IAAI;AAAA,EACtC,SAAS,OAAO;AACd,UAAM,QAAQ,KAAK;AAAA,EACrB;AACA,MAAI,CAAC,SAAS,IAAI;AAChB,UAAM,IAAI,kBAAkB,SAAS,yBAAyB,SAAS,MAAM,GAAG;AAAA,EAClF;AACA,MAAI;AACJ,MAAI;AACF,WAAO,MAAM,SAAS,KAAK;AAAA,EAC7B,SAAS,OAAO;AACd,UAAM,QAAQ,KAAK;AAAA,EACrB;AACA,MAAI,CAAC,SAAS,IAAI,EAAG,OAAM,IAAI,kBAAkB,SAAS,kCAAkC;AAC5F,SAAO;AACT;AAEA,SAAS,YAAY,MAA+B,UAAsC;AACxF,QAAM,KAAK,KAAK;AAChB,MAAI,OAAO,OAAO,YAAY,GAAG,SAAS,EAAG,QAAO;AACpD,MAAI,aAAa,OAAW,QAAO;AACnC,QAAM,IAAI,kBAAkB,SAAS,gCAAgC;AACvE;AAEA,eAAsB,eACpB,OACA,MACA,UAA4B,CAAC,GACP;AACtB,MAAI,QAAQ,QAAQ,YAAY,KAAM,OAAM,IAAI,kBAAkB,WAAW,iBAAiB;AAC9F,QAAM,QAAQ,QAAQ,SAAS;AAC/B,QAAM,MAAM,QAAQ,OAAO,KAAK;AAChC,QAAM,MAAM,eAAe,MAAM,UAAU,KAAK,IAAI;AACpD,QAAM,OAAO,eAAe,IAAI;AAChC,QAAM,SAAS,MAAM,IAAI,GAAG;AAC5B,MAAI,WAAW,UAAa,OAAO,aAAa,KAAM,QAAO;AAC7D,QAAM,WAAW,QAAQ,WAAW,QAAQ;AAC5C,QAAM,SAAS,YAAY,MAAM,OAAO;AACxC,QAAM,MACJ,aAAa,SACT,GAAG,MAAM,6BACT,GAAG,MAAM,qBAAqB,mBAAmB,QAAQ,CAAC;AAChE,QAAM,UAAU,cAAc,SAAS;AAAA,IACrC,gBAAgB;AAAA,IAChB,cAAc,OAAO,OAAO;AAAA,EAC9B,CAAC;AACD,QAAM,YAAY,QAAQ,SAAS,WAAW;AAC9C,QAAM,OAAO,MAAM,YAAY,WAAW,KAAK;AAAA,IAC7C,QAAQ,aAAa,SAAY,SAAS;AAAA,IAC1C;AAAA,IACA,MAAM,KAAK,UAAU,UAAU,IAAI,CAAC;AAAA,IACpC,QAAQ,QAAQ;AAAA,EAClB,CAAC;AACD,QAAM,QAAqB;AAAA,IACzB,SAAS,YAAY,MAAM,QAAQ;AAAA,IACnC,UAAU;AAAA,IACV,aAAa,IAAI;AAAA,EACnB;AACA,QAAM,IAAI,KAAK,KAAK;AACpB,SAAO;AACT;",
|
|
6
|
+
"names": []
|
|
7
|
+
}
|