@ai-ecoverse/pi-audio 1.17.0 → 1.18.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -1
- package/dist/echo.js +157 -0
- package/dist/echo.js.map +7 -0
- package/dist/gemini-live/session.js +32 -15
- package/dist/gemini-live/session.js.map +2 -2
- package/dist/index.js +16 -0
- package/dist/index.js.map +2 -2
- package/dist/openai-realtime/session.js +35 -9
- package/dist/openai-realtime/session.js.map +2 -2
- package/dist/openai-realtime/socket.js +12 -0
- package/dist/openai-realtime/socket.js.map +2 -2
- package/dist/talker.js +71 -27
- package/dist/talker.js.map +3 -3
- package/dist/turns.js +7 -1
- package/dist/turns.js.map +2 -2
- package/dist/types/echo.d.ts +29 -0
- package/dist/types/index.d.ts +2 -0
- package/dist/types/openai-realtime/socket.d.ts +1 -0
- package/dist/types/talker.d.ts +1 -1
- package/dist/types/turns.d.ts +2 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -27,6 +27,8 @@ pi-audio does not apply that patch. Hosts such as slicc-agent record monkeypatch
|
|
|
27
27
|
|
|
28
28
|
`@ai-ecoverse/pi-audio/elevenlabs` registers ElevenLabs speech adapters when `registerElevenLabs()` is called. `scribe_v2` posts multipart audio to `/speech-to-text`. `scribe_v2_realtime` streams base64 audio on `wss://…/speech-to-text/realtime` and commits turns with `turn_prediction` (`turnDetection: 'semantic'`). `eleven_flash_v2_5` and `eleven_multilingual_v2` post JSON to `/text-to-speech/{voice}/with-timestamps` for a string and stream text on `stream-input` for an async iterable, with character alignment. `eleven_v3` and `eleven_v4` stay on the HTTP path (`textStreaming: false`). Scribe prices are $0.22 per hour for batch and $0.39 per hour for realtime.
|
|
29
29
|
|
|
30
|
+
`calibrateEcho(out, mic, probe, durationMs, signal)` plays a probe and aligns the mic with `AudioOut.reference`. `delayMs` is the lag. `cancellationDb` is how far the leaked copy sits below the reference, and `residualDb` is that leak. A quiet leak recommends `interrupt`, a moderate one `onWords`, and a loud one `headphones`. `chooseCapture` keeps `none` and `wasm`, and moves `browser` to `wasm` when cancellation is under 12 dB. `micOpen('browser')` asks for `echoCancellation` and `playback: 'rtc-loopback'`. `wasm` and `none` turn browser cancellation off and use `webaudio`. `assertNodeEcho` rejects `browser` on Node. `openEchoMic` leaves `browser` and `none` audio as captured. `wasm` runs an injected `EchoCanceller` against `reference`. The homescoop `@ai-ecoverse/wasm-*` echo-canceller package does not exist yet, so this package does not import one. `voiceTurns` passes `options.echo` through `openEchoMic` before `listen`.
|
|
31
|
+
|
|
30
32
|
`followBed(bed, soundscape)` cues a `BedChannel` from `VoicePhase`. A loop waits `graceMs` before it starts, so a short gap stays silent. `keep` leaves the current loop running across chained tools. Speech ducks the bed to `duckUnderSpeech`; listening ducks it to `duckUnderUser`. `voiceTurns` does this when `options.soundscape` is set and `out` is an `AudioOut` (`speech`, `bed`, `reference`, `playback`). A plain `AudioSink` still receives only the spoken audio.
|
|
31
33
|
|
|
32
34
|
`voiceTurns` captions include `agent_spoken`. `upTo` follows `AudioSink` playback: per character when the chunk has alignment, and evenly across the sentence when it does not. `captionsFromHistory(entries)` rebuilds user captions from entries whose `data.source` is `voice`, and agent captions from the assistant text. A `voice.truncate` entry adds `agent_spoken` and `agent_cut` at `heardUpTo`, so the rest of the original text stays available as unheard.
|
|
@@ -50,7 +52,7 @@ pi-audio does not apply that patch. Hosts such as slicc-agent record monkeypatch
|
|
|
50
52
|
|
|
51
53
|
`@ai-ecoverse/pi-audio/elevenlabs-agents` registers `elevenlabs-agent`, a hosted cascade (`provisioning: "agent"`, `truncation: "server"`, `liveToolUpdate: false`). `provisionAgent` creates or updates the agent and reuses it when the spec hash matches. pi tools become client tools. A string `llm` is a vendor model name. A `CustomLlm` sets `llm` to `custom-llm` and sends `custom_llm.url`, `model_id`, `api_type`, and `request_headers` (ElevenLabs `api_key` is a workspace secret reference, not a raw token). `connectRealtime` uses a `RealtimeToken` from `mintRealtimeToken` (`get-signed-url`) when one is passed. Otherwise an API key provisions the agent and mints the signed URL. Silent `inject` sends `contextual_update` with `context_id` `pi-audio`. `close()` prices `audio.inputSeconds` from the wall-clock session at $0.08 per minute.
|
|
52
54
|
|
|
53
|
-
`talkerTools` is `ask_agent` and `read`. pi-audio defines `askAgent` (a pi `Tool` whose parameters are `request` and optional `context`) and the `Thinker` interface. The host implements `Thinker` and supplies `read`. `bindThinker` runs direct tools with `directTimeoutMs` and truncates their text to `directMaxChars`. On a blocking `ask_agent` it answers with `ack` immediately and injects the result later. On an async call it `respond`s with the result and `options.result`. Progress is injected with `options.progress`. `tool_call_cancelled` aborts the thinker. The binding also yields `thinker_progress` / `thinker_result` captions and `thinking` / `tool` / `idle` phases.
|
|
55
|
+
`talkerTools` is `ask_agent` and `read`. pi-audio defines `askAgent` (a pi `Tool` whose parameters are `request` and optional `context`) and the `Thinker` interface. The host implements `Thinker` and supplies `read`. `bindThinker` runs direct tools with `directTimeoutMs` and truncates their text to `directMaxChars`. On a blocking `ask_agent` it answers with `ack` immediately and injects the result later. On an async call it `respond`s with the result and `options.result`. Progress is injected with `options.progress`. `tool_call_cancelled` aborts the thinker and any direct tool. A blocking failure is injected after the acknowledgement, with a `thinker_result` caption. `Thinker.ask` receives the optional `context` string. Closing the session aborts work that is still running. The binding also yields `thinker_progress` / `thinker_result` captions and `thinking` / `tool` / `idle` phases.
|
|
54
56
|
|
|
55
57
|
## Development
|
|
56
58
|
|
package/dist/echo.js
ADDED
|
@@ -0,0 +1,157 @@
|
|
|
1
|
+
import { chunkSeconds } from "./audio.js";
|
|
2
|
+
import { VoiceRequestError } from "./errors.js";
|
|
3
|
+
const floor = 1e-8;
|
|
4
|
+
function micOpen(mode, useAll) {
|
|
5
|
+
if (mode === "browser") {
|
|
6
|
+
let echoCancellation = true;
|
|
7
|
+
if (useAll) echoCancellation = "all";
|
|
8
|
+
return { echoCancellation, playback: "rtc-loopback" };
|
|
9
|
+
}
|
|
10
|
+
return { echoCancellation: false, playback: "webaudio" };
|
|
11
|
+
}
|
|
12
|
+
function echoRuntime(hasMediaDevices) {
|
|
13
|
+
if (hasMediaDevices) return "browser";
|
|
14
|
+
return "node";
|
|
15
|
+
}
|
|
16
|
+
function assertNodeEcho(mode, runtime) {
|
|
17
|
+
if (runtime === "node" && mode === "browser") {
|
|
18
|
+
throw new VoiceRequestError("error", "Node echo control is wasm or none");
|
|
19
|
+
}
|
|
20
|
+
}
|
|
21
|
+
function chooseCapture(tried, report) {
|
|
22
|
+
if (tried === "none") return "none";
|
|
23
|
+
if (tried === "wasm") return "wasm";
|
|
24
|
+
if (report.cancellationDb < 12) return "wasm";
|
|
25
|
+
return "browser";
|
|
26
|
+
}
|
|
27
|
+
function openEchoMic(control, mic) {
|
|
28
|
+
if (control.mode === "none" || control.mode === "browser") return mic;
|
|
29
|
+
const canceller = control.canceller;
|
|
30
|
+
if (canceller === void 0) {
|
|
31
|
+
throw new VoiceRequestError("error", "wasm echo control needs a canceller");
|
|
32
|
+
}
|
|
33
|
+
return cancelStream(mic, control.reference, canceller);
|
|
34
|
+
}
|
|
35
|
+
function measureEcho(reference, mic, sampleRate) {
|
|
36
|
+
if (reference.length === 0) return quiet();
|
|
37
|
+
if (mic.length === 0) return quiet();
|
|
38
|
+
let rate = sampleRate;
|
|
39
|
+
if (rate <= 0) rate = 1;
|
|
40
|
+
const maxLag = Math.min(mic.length - 1, Math.max(0, Math.round(rate * 0.25)));
|
|
41
|
+
let bestLag = 0;
|
|
42
|
+
let best = -1;
|
|
43
|
+
let bestGain = 0;
|
|
44
|
+
for (let lag = 0; lag <= maxLag; lag += 1) {
|
|
45
|
+
const scored = scoreLag(reference, mic, lag);
|
|
46
|
+
if (scored.score > best) {
|
|
47
|
+
best = scored.score;
|
|
48
|
+
bestLag = lag;
|
|
49
|
+
bestGain = scored.gain;
|
|
50
|
+
}
|
|
51
|
+
}
|
|
52
|
+
const leak = Math.abs(bestGain);
|
|
53
|
+
const residualDb = amplitudeDb(leak);
|
|
54
|
+
return {
|
|
55
|
+
delayMs: bestLag / rate * 1e3,
|
|
56
|
+
cancellationDb: -residualDb,
|
|
57
|
+
residualDb,
|
|
58
|
+
recommendation: recommend(residualDb)
|
|
59
|
+
};
|
|
60
|
+
}
|
|
61
|
+
async function calibrateEcho(out, mic, probe, durationMs, signal) {
|
|
62
|
+
if (signal.aborted) throw new VoiceRequestError("aborted", "aborted");
|
|
63
|
+
out.bed.play(probe, 0);
|
|
64
|
+
const rate = { value: 0 };
|
|
65
|
+
try {
|
|
66
|
+
const heard = readSamples(mic, durationMs, signal, rate);
|
|
67
|
+
const reference = readSamples(out.reference, durationMs, signal, rate);
|
|
68
|
+
const pair = await Promise.all([heard, reference]);
|
|
69
|
+
let used = rate.value;
|
|
70
|
+
if (used <= 0) used = 1;
|
|
71
|
+
return measureEcho(pair[1], pair[0], used);
|
|
72
|
+
} finally {
|
|
73
|
+
out.bed.stop(0);
|
|
74
|
+
}
|
|
75
|
+
}
|
|
76
|
+
function quiet() {
|
|
77
|
+
return { delayMs: 0, cancellationDb: 80, residualDb: -80, recommendation: "interrupt" };
|
|
78
|
+
}
|
|
79
|
+
function recommend(residualDb) {
|
|
80
|
+
if (residualDb <= -30) return "interrupt";
|
|
81
|
+
if (residualDb <= -12) return "onWords";
|
|
82
|
+
return "headphones";
|
|
83
|
+
}
|
|
84
|
+
function scoreLag(reference, mic, lag) {
|
|
85
|
+
let dot = 0;
|
|
86
|
+
let energy = 0;
|
|
87
|
+
const length = Math.min(reference.length, mic.length - lag);
|
|
88
|
+
for (let index = 0; index < length; index += 1) {
|
|
89
|
+
const ref = reference[index];
|
|
90
|
+
const heard = mic[index + lag];
|
|
91
|
+
dot += ref * heard;
|
|
92
|
+
energy += ref * ref;
|
|
93
|
+
}
|
|
94
|
+
if (energy <= floor) return { score: -1, gain: 0 };
|
|
95
|
+
return { score: Math.abs(dot), gain: dot / energy };
|
|
96
|
+
}
|
|
97
|
+
function amplitudeDb(amplitude) {
|
|
98
|
+
let safe = amplitude;
|
|
99
|
+
if (safe <= floor) safe = floor;
|
|
100
|
+
return 20 * Math.log10(safe);
|
|
101
|
+
}
|
|
102
|
+
async function* cancelStream(mic, reference, canceller) {
|
|
103
|
+
const refs = reference[Symbol.asyncIterator]();
|
|
104
|
+
for await (const chunk of mic) {
|
|
105
|
+
const next = await refs.next();
|
|
106
|
+
if (next.done) {
|
|
107
|
+
yield chunk;
|
|
108
|
+
continue;
|
|
109
|
+
}
|
|
110
|
+
yield canceller.cancel(chunk, next.value);
|
|
111
|
+
}
|
|
112
|
+
}
|
|
113
|
+
async function readSamples(source, durationMs, signal, rate) {
|
|
114
|
+
const samples = [];
|
|
115
|
+
let heardMs = 0;
|
|
116
|
+
for await (const chunk of source) {
|
|
117
|
+
if (signal.aborted) throw new VoiceRequestError("aborted", "aborted");
|
|
118
|
+
if (rate.value === 0) rate.value = chunk.format.sampleRate;
|
|
119
|
+
for (const sample of samplesOf(chunk)) samples.push(sample);
|
|
120
|
+
heardMs += chunkSeconds(chunk) * 1e3;
|
|
121
|
+
if (heardMs >= durationMs) break;
|
|
122
|
+
}
|
|
123
|
+
return Float32Array.from(samples);
|
|
124
|
+
}
|
|
125
|
+
function samplesOf(chunk) {
|
|
126
|
+
if (chunk.format.encoding === "pcm16") return pcm16Samples(chunk.data);
|
|
127
|
+
if (chunk.format.encoding === "pcmf32") return pcmf32Samples(chunk.data);
|
|
128
|
+
return [];
|
|
129
|
+
}
|
|
130
|
+
function pcm16Samples(data) {
|
|
131
|
+
const view = new DataView(data.buffer, data.byteOffset, data.byteLength);
|
|
132
|
+
const samples = [];
|
|
133
|
+
const count = Math.floor(data.byteLength / 2);
|
|
134
|
+
for (let index = 0; index < count; index += 1) {
|
|
135
|
+
samples.push(view.getInt16(index * 2, true) / 32768);
|
|
136
|
+
}
|
|
137
|
+
return samples;
|
|
138
|
+
}
|
|
139
|
+
function pcmf32Samples(data) {
|
|
140
|
+
const view = new DataView(data.buffer, data.byteOffset, data.byteLength);
|
|
141
|
+
const samples = [];
|
|
142
|
+
const count = Math.floor(data.byteLength / 4);
|
|
143
|
+
for (let index = 0; index < count; index += 1) {
|
|
144
|
+
samples.push(view.getFloat32(index * 4, true));
|
|
145
|
+
}
|
|
146
|
+
return samples;
|
|
147
|
+
}
|
|
148
|
+
export {
|
|
149
|
+
assertNodeEcho,
|
|
150
|
+
calibrateEcho,
|
|
151
|
+
chooseCapture,
|
|
152
|
+
echoRuntime,
|
|
153
|
+
measureEcho,
|
|
154
|
+
micOpen,
|
|
155
|
+
openEchoMic
|
|
156
|
+
};
|
|
157
|
+
//# sourceMappingURL=echo.js.map
|
package/dist/echo.js.map
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
{
|
|
2
|
+
"version": 3,
|
|
3
|
+
"sources": ["../src/echo.ts"],
|
|
4
|
+
"sourcesContent": ["import { type AudioChunk, type AudioSource, chunkSeconds } from './audio.ts';\nimport { VoiceRequestError } from './errors.ts';\nimport type { AudioOut, Cue } from './soundscape.ts';\nimport type { EchoRecommendation } from './turns.ts';\n\nexport type EchoMode = 'browser' | 'wasm' | 'none';\n\nexport interface EchoCanceller {\n cancel(mic: AudioChunk, reference: AudioChunk): AudioChunk;\n}\n\nexport interface EchoControl {\n mode: EchoMode;\n reference: AudioSource;\n canceller?: EchoCanceller;\n}\n\nexport interface EchoReport {\n delayMs: number;\n cancellationDb: number;\n residualDb: number;\n recommendation: EchoRecommendation;\n}\n\nexport interface MicOpen {\n echoCancellation: boolean | 'all';\n playback: 'webaudio' | 'rtc-loopback';\n}\n\nconst floor = 1e-8;\n\nexport function micOpen(mode: EchoMode, useAll: boolean): MicOpen {\n if (mode === 'browser') {\n let echoCancellation: boolean | 'all' = true;\n if (useAll) echoCancellation = 'all';\n return { echoCancellation, playback: 'rtc-loopback' };\n }\n return { echoCancellation: false, playback: 'webaudio' };\n}\n\nexport function echoRuntime(hasMediaDevices: boolean): 'node' | 'browser' {\n if (hasMediaDevices) return 'browser';\n return 'node';\n}\n\nexport function assertNodeEcho(mode: EchoMode, runtime: 'node' | 'browser'): void {\n if (runtime === 'node' && mode === 'browser') {\n throw new VoiceRequestError('error', 'Node echo control is wasm or none');\n }\n}\n\nexport function chooseCapture(tried: EchoMode, report: EchoReport): EchoMode {\n if (tried === 'none') return 'none';\n if (tried === 'wasm') return 'wasm';\n if (report.cancellationDb < 12) return 'wasm';\n return 'browser';\n}\n\nexport function openEchoMic(control: EchoControl, mic: AudioSource): AudioSource {\n if (control.mode === 'none' || control.mode === 'browser') return mic;\n const canceller = control.canceller;\n if (canceller === undefined) {\n throw new VoiceRequestError('error', 'wasm echo control needs a canceller');\n }\n return cancelStream(mic, control.reference, canceller);\n}\n\nexport function measureEcho(\n reference: Float32Array,\n mic: Float32Array,\n sampleRate: number\n): EchoReport {\n if (reference.length === 0) return quiet();\n if (mic.length === 0) return quiet();\n let rate = sampleRate;\n if (rate <= 0) rate = 1;\n const maxLag = Math.min(mic.length - 1, Math.max(0, Math.round(rate * 0.25)));\n let bestLag = 0;\n let best = -1;\n let bestGain = 0;\n for (let lag = 0; lag <= maxLag; lag += 1) {\n const scored = scoreLag(reference, mic, lag);\n if (scored.score > best) {\n best = scored.score;\n bestLag = lag;\n bestGain = scored.gain;\n }\n }\n const leak = Math.abs(bestGain);\n const residualDb = amplitudeDb(leak);\n return {\n delayMs: (bestLag / rate) * 1000,\n cancellationDb: -residualDb,\n residualDb,\n recommendation: recommend(residualDb),\n };\n}\n\nexport async function calibrateEcho(\n out: AudioOut,\n mic: AudioSource,\n probe: Cue,\n durationMs: number,\n signal: AbortSignal\n): Promise<EchoReport> {\n if (signal.aborted) throw new VoiceRequestError('aborted', 'aborted');\n out.bed.play(probe, 0);\n const rate = { value: 0 };\n try {\n const heard = readSamples(mic, durationMs, signal, rate);\n const reference = readSamples(out.reference, durationMs, signal, rate);\n const pair = await Promise.all([heard, reference]);\n let used = rate.value;\n if (used <= 0) used = 1;\n return measureEcho(pair[1], pair[0], used);\n } finally {\n out.bed.stop(0);\n }\n}\n\nfunction quiet(): EchoReport {\n return { delayMs: 0, cancellationDb: 80, residualDb: -80, recommendation: 'interrupt' };\n}\n\nfunction recommend(residualDb: number): EchoRecommendation {\n if (residualDb <= -30) return 'interrupt';\n if (residualDb <= -12) return 'onWords';\n return 'headphones';\n}\n\nfunction scoreLag(\n reference: Float32Array,\n mic: Float32Array,\n lag: number\n): { score: number; gain: number } {\n let dot = 0;\n let energy = 0;\n const length = Math.min(reference.length, mic.length - lag);\n for (let index = 0; index < length; index += 1) {\n const ref = reference[index] as number;\n const heard = mic[index + lag] as number;\n dot += ref * heard;\n energy += ref * ref;\n }\n if (energy <= floor) return { score: -1, gain: 0 };\n return { score: Math.abs(dot), gain: dot / energy };\n}\n\nfunction amplitudeDb(amplitude: number): number {\n let safe = amplitude;\n if (safe <= floor) safe = floor;\n return 20 * Math.log10(safe);\n}\n\nasync function* cancelStream(\n mic: AudioSource,\n reference: AudioSource,\n canceller: EchoCanceller\n): AudioSource {\n const refs = reference[Symbol.asyncIterator]();\n for await (const chunk of mic) {\n const next = await refs.next();\n if (next.done) {\n yield chunk;\n continue;\n }\n yield canceller.cancel(chunk, next.value);\n }\n}\n\nasync function readSamples(\n source: AudioSource,\n durationMs: number,\n signal: AbortSignal,\n rate: { value: number }\n): Promise<Float32Array> {\n const samples: number[] = [];\n let heardMs = 0;\n for await (const chunk of source) {\n if (signal.aborted) throw new VoiceRequestError('aborted', 'aborted');\n if (rate.value === 0) rate.value = chunk.format.sampleRate;\n for (const sample of samplesOf(chunk)) samples.push(sample);\n heardMs += chunkSeconds(chunk) * 1000;\n if (heardMs >= durationMs) break;\n }\n return Float32Array.from(samples);\n}\n\nfunction samplesOf(chunk: AudioChunk): number[] {\n if (chunk.format.encoding === 'pcm16') return pcm16Samples(chunk.data);\n if (chunk.format.encoding === 'pcmf32') return pcmf32Samples(chunk.data);\n return [];\n}\n\nfunction pcm16Samples(data: Uint8Array): number[] {\n const view = new DataView(data.buffer, data.byteOffset, data.byteLength);\n const samples: number[] = [];\n const count = Math.floor(data.byteLength / 2);\n for (let index = 0; index < count; index += 1) {\n samples.push(view.getInt16(index * 2, true) / 32768);\n }\n return samples;\n}\n\nfunction pcmf32Samples(data: Uint8Array): number[] {\n const view = new DataView(data.buffer, data.byteOffset, data.byteLength);\n const samples: number[] = [];\n const count = Math.floor(data.byteLength / 4);\n for (let index = 0; index < count; index += 1) {\n samples.push(view.getFloat32(index * 4, true));\n }\n return samples;\n}\n"],
|
|
5
|
+
"mappings": "AAAA,SAA4C,oBAAoB;AAChE,SAAS,yBAAyB;AA4BlC,MAAM,QAAQ;AAEP,SAAS,QAAQ,MAAgB,QAA0B;AAChE,MAAI,SAAS,WAAW;AACtB,QAAI,mBAAoC;AACxC,QAAI,OAAQ,oBAAmB;AAC/B,WAAO,EAAE,kBAAkB,UAAU,eAAe;AAAA,EACtD;AACA,SAAO,EAAE,kBAAkB,OAAO,UAAU,WAAW;AACzD;AAEO,SAAS,YAAY,iBAA8C;AACxE,MAAI,gBAAiB,QAAO;AAC5B,SAAO;AACT;AAEO,SAAS,eAAe,MAAgB,SAAmC;AAChF,MAAI,YAAY,UAAU,SAAS,WAAW;AAC5C,UAAM,IAAI,kBAAkB,SAAS,mCAAmC;AAAA,EAC1E;AACF;AAEO,SAAS,cAAc,OAAiB,QAA8B;AAC3E,MAAI,UAAU,OAAQ,QAAO;AAC7B,MAAI,UAAU,OAAQ,QAAO;AAC7B,MAAI,OAAO,iBAAiB,GAAI,QAAO;AACvC,SAAO;AACT;AAEO,SAAS,YAAY,SAAsB,KAA+B;AAC/E,MAAI,QAAQ,SAAS,UAAU,QAAQ,SAAS,UAAW,QAAO;AAClE,QAAM,YAAY,QAAQ;AAC1B,MAAI,cAAc,QAAW;AAC3B,UAAM,IAAI,kBAAkB,SAAS,qCAAqC;AAAA,EAC5E;AACA,SAAO,aAAa,KAAK,QAAQ,WAAW,SAAS;AACvD;AAEO,SAAS,YACd,WACA,KACA,YACY;AACZ,MAAI,UAAU,WAAW,EAAG,QAAO,MAAM;AACzC,MAAI,IAAI,WAAW,EAAG,QAAO,MAAM;AACnC,MAAI,OAAO;AACX,MAAI,QAAQ,EAAG,QAAO;AACtB,QAAM,SAAS,KAAK,IAAI,IAAI,SAAS,GAAG,KAAK,IAAI,GAAG,KAAK,MAAM,OAAO,IAAI,CAAC,CAAC;AAC5E,MAAI,UAAU;AACd,MAAI,OAAO;AACX,MAAI,WAAW;AACf,WAAS,MAAM,GAAG,OAAO,QAAQ,OAAO,GAAG;AACzC,UAAM,SAAS,SAAS,WAAW,KAAK,GAAG;AAC3C,QAAI,OAAO,QAAQ,MAAM;AACvB,aAAO,OAAO;AACd,gBAAU;AACV,iBAAW,OAAO;AAAA,IACpB;AAAA,EACF;AACA,QAAM,OAAO,KAAK,IAAI,QAAQ;AAC9B,QAAM,aAAa,YAAY,IAAI;AACnC,SAAO;AAAA,IACL,SAAU,UAAU,OAAQ;AAAA,IAC5B,gBAAgB,CAAC;AAAA,IACjB;AAAA,IACA,gBAAgB,UAAU,UAAU;AAAA,EACtC;AACF;AAEA,eAAsB,cACpB,KACA,KACA,OACA,YACA,QACqB;AACrB,MAAI,OAAO,QAAS,OAAM,IAAI,kBAAkB,WAAW,SAAS;AACpE,MAAI,IAAI,KAAK,OAAO,CAAC;AACrB,QAAM,OAAO,EAAE,OAAO,EAAE;AACxB,MAAI;AACF,UAAM,QAAQ,YAAY,KAAK,YAAY,QAAQ,IAAI;AACvD,UAAM,YAAY,YAAY,IAAI,WAAW,YAAY,QAAQ,IAAI;AACrE,UAAM,OAAO,MAAM,QAAQ,IAAI,CAAC,OAAO,SAAS,CAAC;AACjD,QAAI,OAAO,KAAK;AAChB,QAAI,QAAQ,EAAG,QAAO;AACtB,WAAO,YAAY,KAAK,CAAC,GAAG,KAAK,CAAC,GAAG,IAAI;AAAA,EAC3C,UAAE;AACA,QAAI,IAAI,KAAK,CAAC;AAAA,EAChB;AACF;AAEA,SAAS,QAAoB;AAC3B,SAAO,EAAE,SAAS,GAAG,gBAAgB,IAAI,YAAY,KAAK,gBAAgB,YAAY;AACxF;AAEA,SAAS,UAAU,YAAwC;AACzD,MAAI,cAAc,IAAK,QAAO;AAC9B,MAAI,cAAc,IAAK,QAAO;AAC9B,SAAO;AACT;AAEA,SAAS,SACP,WACA,KACA,KACiC;AACjC,MAAI,MAAM;AACV,MAAI,SAAS;AACb,QAAM,SAAS,KAAK,IAAI,UAAU,QAAQ,IAAI,SAAS,GAAG;AAC1D,WAAS,QAAQ,GAAG,QAAQ,QAAQ,SAAS,GAAG;AAC9C,UAAM,MAAM,UAAU,KAAK;AAC3B,UAAM,QAAQ,IAAI,QAAQ,GAAG;AAC7B,WAAO,MAAM;AACb,cAAU,MAAM;AAAA,EAClB;AACA,MAAI,UAAU,MAAO,QAAO,EAAE,OAAO,IAAI,MAAM,EAAE;AACjD,SAAO,EAAE,OAAO,KAAK,IAAI,GAAG,GAAG,MAAM,MAAM,OAAO;AACpD;AAEA,SAAS,YAAY,WAA2B;AAC9C,MAAI,OAAO;AACX,MAAI,QAAQ,MAAO,QAAO;AAC1B,SAAO,KAAK,KAAK,MAAM,IAAI;AAC7B;AAEA,gBAAgB,aACd,KACA,WACA,WACa;AACb,QAAM,OAAO,UAAU,OAAO,aAAa,EAAE;AAC7C,mBAAiB,SAAS,KAAK;AAC7B,UAAM,OAAO,MAAM,KAAK,KAAK;AAC7B,QAAI,KAAK,MAAM;AACb,YAAM;AACN;AAAA,IACF;AACA,UAAM,UAAU,OAAO,OAAO,KAAK,KAAK;AAAA,EAC1C;AACF;AAEA,eAAe,YACb,QACA,YACA,QACA,MACuB;AACvB,QAAM,UAAoB,CAAC;AAC3B,MAAI,UAAU;AACd,mBAAiB,SAAS,QAAQ;AAChC,QAAI,OAAO,QAAS,OAAM,IAAI,kBAAkB,WAAW,SAAS;AACpE,QAAI,KAAK,UAAU,EAAG,MAAK,QAAQ,MAAM,OAAO;AAChD,eAAW,UAAU,UAAU,KAAK,EAAG,SAAQ,KAAK,MAAM;AAC1D,eAAW,aAAa,KAAK,IAAI;AACjC,QAAI,WAAW,WAAY;AAAA,EAC7B;AACA,SAAO,aAAa,KAAK,OAAO;AAClC;AAEA,SAAS,UAAU,OAA6B;AAC9C,MAAI,MAAM,OAAO,aAAa,QAAS,QAAO,aAAa,MAAM,IAAI;AACrE,MAAI,MAAM,OAAO,aAAa,SAAU,QAAO,cAAc,MAAM,IAAI;AACvE,SAAO,CAAC;AACV;AAEA,SAAS,aAAa,MAA4B;AAChD,QAAM,OAAO,IAAI,SAAS,KAAK,QAAQ,KAAK,YAAY,KAAK,UAAU;AACvE,QAAM,UAAoB,CAAC;AAC3B,QAAM,QAAQ,KAAK,MAAM,KAAK,aAAa,CAAC;AAC5C,WAAS,QAAQ,GAAG,QAAQ,OAAO,SAAS,GAAG;AAC7C,YAAQ,KAAK,KAAK,SAAS,QAAQ,GAAG,IAAI,IAAI,KAAK;AAAA,EACrD;AACA,SAAO;AACT;AAEA,SAAS,cAAc,MAA4B;AACjD,QAAM,OAAO,IAAI,SAAS,KAAK,QAAQ,KAAK,YAAY,KAAK,UAAU;AACvE,QAAM,UAAoB,CAAC;AAC3B,QAAM,QAAQ,KAAK,MAAM,KAAK,aAAa,CAAC;AAC5C,WAAS,QAAQ,GAAG,QAAQ,OAAO,SAAS,GAAG;AAC7C,YAAQ,KAAK,KAAK,WAAW,QAAQ,GAAG,IAAI,CAAC;AAAA,EAC/C;AACA,SAAO;AACT;",
|
|
6
|
+
"names": []
|
|
7
|
+
}
|
|
@@ -205,6 +205,8 @@ class GeminiSession {
|
|
|
205
205
|
#closed = false;
|
|
206
206
|
#ended = false;
|
|
207
207
|
#meters;
|
|
208
|
+
#toolGroups = /* @__PURE__ */ new Map();
|
|
209
|
+
#toolGroupOf = /* @__PURE__ */ new Map();
|
|
208
210
|
constructor(model, context, vocabulary, open, target, wire) {
|
|
209
211
|
this.model = model;
|
|
210
212
|
this.#context = context;
|
|
@@ -285,20 +287,14 @@ class GeminiSession {
|
|
|
285
287
|
this.#assertOpen();
|
|
286
288
|
const send = () => {
|
|
287
289
|
const mode = delivery === void 0 ? "interrupt" : delivery;
|
|
288
|
-
this.#
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
status: result.isError ? "error" : "ok",
|
|
297
|
-
retryable: false,
|
|
298
|
-
message: resultText(result)
|
|
299
|
-
}
|
|
300
|
-
}
|
|
301
|
-
]
|
|
290
|
+
this.#queueTool(callId, {
|
|
291
|
+
id: callId,
|
|
292
|
+
name: result.toolName,
|
|
293
|
+
scheduling: scheduling[mode],
|
|
294
|
+
response: {
|
|
295
|
+
status: result.isError ? "error" : "ok",
|
|
296
|
+
retryable: false,
|
|
297
|
+
message: resultText(result)
|
|
302
298
|
}
|
|
303
299
|
});
|
|
304
300
|
};
|
|
@@ -440,23 +436,44 @@ class GeminiSession {
|
|
|
440
436
|
next.send(JSON.stringify({ setup: this.#setup(handle) }));
|
|
441
437
|
return next;
|
|
442
438
|
}
|
|
439
|
+
#queueTool(callId, response) {
|
|
440
|
+
const groupId = this.#toolGroupOf.get(callId) ?? "";
|
|
441
|
+
const group = groupId.length === 0 ? void 0 : this.#toolGroups.get(groupId);
|
|
442
|
+
if (group === void 0) {
|
|
443
|
+
this.#send({ toolResponse: { functionResponses: [response] } });
|
|
444
|
+
return;
|
|
445
|
+
}
|
|
446
|
+
group.responses.push(response);
|
|
447
|
+
group.pending.delete(callId);
|
|
448
|
+
this.#toolGroupOf.delete(callId);
|
|
449
|
+
if (group.pending.size > 0) return;
|
|
450
|
+
this.#toolGroups.delete(groupId);
|
|
451
|
+
this.#send({ toolResponse: { functionResponses: group.responses } });
|
|
452
|
+
}
|
|
443
453
|
#onToolCall(value) {
|
|
444
454
|
const call = record(value);
|
|
445
455
|
if (call === void 0 || !Array.isArray(call.functionCalls)) return;
|
|
456
|
+
const ids = [];
|
|
446
457
|
for (const entry of call.functionCalls) {
|
|
447
458
|
const fn = record(entry);
|
|
448
459
|
if (fn === void 0) continue;
|
|
460
|
+
const id = text(fn.id);
|
|
461
|
+
if (id.length > 0) ids.push(id);
|
|
449
462
|
this.#push({
|
|
450
463
|
type: "tool_call",
|
|
451
464
|
blocking: this.model.toolCalls === "blocking",
|
|
452
465
|
call: {
|
|
453
466
|
type: "toolCall",
|
|
454
|
-
id
|
|
467
|
+
id,
|
|
455
468
|
name: text(fn.name),
|
|
456
469
|
arguments: argumentsOf(fn.args)
|
|
457
470
|
}
|
|
458
471
|
});
|
|
459
472
|
}
|
|
473
|
+
const groupId = ids.at(0);
|
|
474
|
+
if (groupId === void 0) return;
|
|
475
|
+
this.#toolGroups.set(groupId, { pending: new Set(ids), responses: [] });
|
|
476
|
+
for (const id of ids) this.#toolGroupOf.set(id, groupId);
|
|
460
477
|
}
|
|
461
478
|
#onCancel(value) {
|
|
462
479
|
const cancellation = record(value);
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"version": 3,
|
|
3
3
|
"sources": ["../../src/gemini-live/session.ts"],
|
|
4
|
-
"sourcesContent": ["import type { JsonObject, Message, Tool, ToolResultMessage, Usage } from '@earendil-works/pi-ai';\nimport type { AudioChunk, AudioFormat, TextSpan } from '../audio.ts';\nimport { blankUsage } from '../blank-usage.ts';\nimport { decodeAudioBase64, encodeAudioBase64 } from '../convert.ts';\nimport { VoiceRequestError } from '../errors.ts';\nimport type { Delivery, RealtimeModel } from '../models.ts';\nimport {\n openVoiceSocket,\n type VoiceSocketConstructor,\n type VoiceWire,\n type VoiceWireTarget,\n} from '../openai-realtime/socket.ts';\nimport type {\n RealtimeContext,\n RealtimeEvent,\n RealtimeOptions,\n RealtimeSession,\n} from '../realtime.ts';\nimport { planTruncation, RealtimeLedger, realtimeSocket } from '../realtime.ts';\n\nexport type VoiceWireFactory = (target: VoiceWireTarget) => VoiceWire;\n\nexport interface GeminiOptions {\n wire?: VoiceWireFactory;\n socket?: VoiceSocketConstructor;\n vocabulary?: string[];\n}\n\nconst scheduling: Record<Delivery, string> = {\n interrupt: 'INTERRUPT',\n when_idle: 'WHEN_IDLE',\n silent: 'SILENT',\n};\n\ninterface PendingAudio {\n chunks: AudioChunk[];\n cursor: number;\n}\n\ninterface IteratorResultEvent {\n value?: RealtimeEvent;\n done: boolean;\n}\n\nfunction record(value: unknown): Record<string, unknown> | undefined {\n if (value === null || typeof value !== 'object' || Array.isArray(value)) return undefined;\n return value as Record<string, unknown>;\n}\n\nfunction text(value: unknown): string {\n return typeof value === 'string' ? value : '';\n}\n\nfunction credential(options: RealtimeOptions | undefined): { secret: string; ephemeral: boolean } {\n const token = options?.token?.value;\n if (typeof token === 'string' && token.length > 0) return { secret: token, ephemeral: true };\n const apiKey = options?.apiKey;\n if (typeof apiKey === 'string' && apiKey.length > 0) return { secret: apiKey, ephemeral: false };\n throw new VoiceRequestError('error', 'missing realtime credential');\n}\n\nfunction headersFor(options: RealtimeOptions | undefined): Record<string, string> {\n const headers: Record<string, string> = {};\n const extra = options?.headers;\n if (extra === undefined) return headers;\n for (const [key, value] of Object.entries(extra)) {\n if (value === null) delete headers[key];\n else headers[key] = value;\n }\n return headers;\n}\n\nfunction sessionUrl(model: RealtimeModel<string>, secret: string, ephemeral: boolean): string {\n const url = new URL(model.baseUrl);\n if (ephemeral) url.searchParams.set('access_token', secret);\n else url.searchParams.set('key', secret);\n return url.toString();\n}\n\nfunction timeLeftMs(value: unknown): number {\n if (typeof value === 'number' && Number.isFinite(value)) return Math.round(value * 1000);\n if (typeof value === 'string') {\n const matched = /^(\\d+(?:\\.\\d+)?)s$/.exec(value);\n if (matched !== null) return Math.round(Number(matched[1]) * 1000);\n const asNumber = Number(value);\n if (Number.isFinite(asNumber)) return Math.round(asNumber * 1000);\n }\n const duration = record(value);\n if (duration !== undefined && typeof duration.seconds === 'number') {\n const nanos = typeof duration.nanos === 'number' ? duration.nanos : 0;\n return Math.round(duration.seconds * 1000 + nanos / 1e6);\n }\n return 0;\n}\n\nfunction count(source: Record<string, unknown>, key: string): number {\n const value = source[key];\n return typeof value === 'number' ? value : 0;\n}\n\nfunction usageFrom(value: unknown): Usage {\n const usage = record(value);\n if (usage === undefined) return blankUsage();\n const input = count(usage, 'promptTokenCount');\n const response = count(usage, 'responseTokenCount');\n const candidates = count(usage, 'candidatesTokenCount');\n const output = response > 0 ? response : candidates;\n return {\n input,\n output,\n cacheRead: 0,\n cacheWrite: 0,\n totalTokens: input + output,\n cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },\n };\n}\n\nfunction argumentsOf(value: unknown): JsonObject {\n if (typeof value === 'string') {\n try {\n const parsed: unknown = JSON.parse(value);\n const object = record(parsed);\n if (object === undefined) return {};\n return object as JsonObject;\n } catch {\n return {};\n }\n }\n const object = record(value);\n if (object === undefined) return {};\n return object as JsonObject;\n}\n\nfunction resultText(result: ToolResultMessage): string {\n const parts: string[] = [];\n for (const part of result.content) {\n if (part.type === 'text') parts.push(part.text);\n }\n return parts.join('\\n');\n}\n\nfunction historyTurns(messages: Message[] | undefined): JsonObject[] {\n if (messages === undefined) return [];\n const turns: JsonObject[] = [];\n for (const message of messages) {\n if (message.role !== 'user' && message.role !== 'assistant') continue;\n const parts: JsonObject[] = [];\n for (const part of message.content) {\n if (typeof part === 'string') {\n parts.push({ text: part });\n continue;\n }\n if (part.type === 'text') parts.push({ text: part.text });\n }\n if (parts.length === 0) continue;\n turns.push({ role: message.role === 'assistant' ? 'model' : 'user', parts });\n }\n return turns;\n}\n\nfunction declaration(tool: Tool, model: RealtimeModel<string>): JsonObject {\n const fn: JsonObject = {\n name: tool.name,\n description: tool.description,\n parameters: tool.parameters as unknown as JsonObject,\n };\n if (model.id === 'gemini-live-2.5-flash-native-audio') fn.behavior = 'NON_BLOCKING';\n if (model.toolCalls === 'blocking') fn.behavior = 'BLOCKING';\n return fn;\n}\n\nfunction transcription(model: RealtimeModel<string>, vocabulary: string[] | undefined): JsonObject {\n if (model.id !== 'gemini-3.8-live') return {};\n if (vocabulary === undefined) return {};\n return { customVocabulary: vocabulary };\n}\n\nfunction setupBody(\n model: RealtimeModel<string>,\n context: RealtimeContext,\n handle: string | undefined,\n vocabulary: string[] | undefined,\n turns: JsonObject[]\n): JsonObject {\n const setup: JsonObject = {\n model: `models/${model.id}`,\n generationConfig: { responseModalities: ['AUDIO'] },\n systemInstruction: { parts: [{ text: context.instructions }] },\n speechConfig: {\n voiceConfig: { prebuiltVoiceConfig: { voiceName: context.voice } },\n },\n inputAudioTranscription: transcription(model, vocabulary),\n outputAudioTranscription: {},\n contextWindowCompression: {\n triggerTokens: 100_000,\n slidingWindow: { targetTokens: 80_000 },\n },\n sessionResumption: handle === undefined ? {} : { handle },\n };\n if (context.tools.length > 0) {\n setup.tools = [{ functionDeclarations: context.tools.map((tool) => declaration(tool, model)) }];\n }\n if (turns.length > 0) setup.historyConfig = { initialHistoryInClientContent: true };\n return setup;\n}\n\nfunction spanFor(itemId: string, start: number, end: number): TextSpan {\n return { itemId, start, end };\n}\n\nfunction takeAudio(\n pending: PendingAudio,\n itemId: string,\n format: AudioFormat,\n end: number | undefined\n): RealtimeEvent[] {\n const chunks = pending.chunks;\n pending.chunks = [];\n if (chunks.length === 0) return [];\n const start = pending.cursor;\n if (end !== undefined) pending.cursor = end;\n const width = end === undefined ? 0 : end - start;\n const total = chunks.reduce((sum, chunk) => sum + chunk.data.length, 0);\n let used = 0;\n let cursor = start;\n return chunks.map((chunk, index) => {\n const event: RealtimeEvent = {\n type: 'output_audio',\n itemId,\n chunk: { data: chunk.data, format, timestamp: chunk.timestamp },\n };\n if (end === undefined) return event;\n const last = index === chunks.length - 1;\n const next = last ? end : start + Math.round(((used + chunk.data.length) / total) * width);\n used += chunk.data.length;\n event.span = spanFor(itemId, cursor, next);\n cursor = next;\n return event;\n });\n}\n\nclass GeminiSession implements RealtimeSession {\n readonly model: RealtimeModel<string>;\n readonly #context: RealtimeContext;\n readonly #vocabulary: string[] | undefined;\n readonly #turns: JsonObject[];\n readonly #open: VoiceWireFactory;\n readonly #target: VoiceWireTarget;\n readonly #ledger = new RealtimeLedger();\n readonly #queue: RealtimeEvent[] = [];\n readonly #waiters: Array<(result: IteratorResultEvent) => void> = [];\n readonly #outgoing: JsonObject[] = [];\n readonly #format: AudioFormat;\n #wire: VoiceWire;\n #pending: PendingAudio = { chunks: [], cursor: 0 };\n #item = 'g0';\n #turn = 0;\n #heard = '';\n #spoken = '';\n #handle: string | undefined;\n #tokens: Usage = blankUsage();\n #ready = false;\n #historySent = false;\n #closed = false;\n #ended = false;\n #meters: Usage | undefined;\n\n constructor(\n model: RealtimeModel<string>,\n context: RealtimeContext,\n vocabulary: string[] | undefined,\n open: VoiceWireFactory,\n target: VoiceWireTarget,\n wire: VoiceWire\n ) {\n this.model = model;\n this.#context = context;\n this.#vocabulary = vocabulary;\n this.#turns = historyTurns(context.history);\n this.#open = open;\n this.#target = target;\n this.#wire = wire;\n this.#format = model.outputFormat;\n }\n\n start(handle: string | undefined): void {\n if (handle !== undefined && handle.length > 0) this.#handle = handle;\n this.#wire.send(JSON.stringify({ setup: this.#setup(handle) }));\n void this.#read(this.#wire);\n }\n\n #setup(handle: string | undefined): JsonObject {\n return setupBody(this.model, this.#context, handle, this.#vocabulary, this.#turns);\n }\n\n #assertOpen(): void {\n if (this.#closed) throw new VoiceRequestError('error', 'session is closed');\n }\n\n #send(event: JsonObject): void {\n if (!this.#ready) {\n this.#outgoing.push(event);\n return;\n }\n this.#wire.send(JSON.stringify(event));\n }\n\n #push(event: RealtimeEvent): void {\n if (this.#closed) return;\n this.#ledger.observe(event);\n const waiter = this.#waiters.shift();\n if (waiter !== undefined) waiter({ value: event, done: false });\n else this.#queue.push(event);\n }\n\n #end(): void {\n if (this.#ended) return;\n this.#ended = true;\n while (this.#waiters.length > 0) this.#waiters.shift()?.({ done: true });\n }\n\n appendAudio(chunk: AudioChunk): void {\n this.#assertOpen();\n this.#ledger.addInput(chunk);\n this.#send({\n realtimeInput: {\n audio: {\n data: encodeAudioBase64(chunk.data),\n mimeType: `audio/pcm;rate=${String(chunk.format.sampleRate)}`,\n },\n },\n });\n }\n\n appendVideoFrame(jpeg: Uint8Array): void {\n this.#assertOpen();\n if (this.model.video === undefined) {\n throw new VoiceRequestError('error', 'video is unsupported');\n }\n this.#send({\n realtimeInput: { video: { data: encodeAudioBase64(jpeg), mimeType: 'image/jpeg' } },\n });\n }\n\n sendText(value: string): void {\n this.#assertOpen();\n this.#ledger.addText();\n this.#send(this.#client(value, true));\n }\n\n inject(value: string, delivery: Delivery): void {\n this.#assertOpen();\n this.#ledger.submit(this.model.delivery, delivery, () => {\n this.#assertOpen();\n this.#ledger.addText();\n if (delivery === 'silent') this.#send(this.#client(value, false));\n else if (delivery === 'when_idle') this.#send({ realtimeInput: { text: value } });\n else this.#send(this.#client(value, true));\n });\n }\n\n respond(callId: string, result: ToolResultMessage, delivery?: Delivery): void {\n this.#assertOpen();\n const send = () => {\n const mode = delivery === undefined ? 'interrupt' : delivery;\n this.#send({\n toolResponse: {\n functionResponses: [\n {\n id: callId,\n name: result.toolName,\n scheduling: scheduling[mode],\n response: {\n status: result.isError ? 'error' : 'ok',\n retryable: false,\n message: resultText(result),\n },\n },\n ],\n },\n });\n };\n if (delivery === undefined) {\n send();\n return;\n }\n this.#ledger.submit(this.model.delivery, delivery, () => {\n this.#assertOpen();\n send();\n });\n }\n\n interrupt(heard?: { itemId: string; heardMs: number }): void {\n this.#assertOpen();\n planTruncation(this.model.truncation, heard);\n this.#ledger.settle();\n }\n\n update(patch: Partial<Pick<RealtimeContext, 'instructions' | 'tools' | 'voice'>>): void {\n this.#assertOpen();\n if (patch.tools !== undefined && !this.model.liveToolUpdate) {\n throw new VoiceRequestError('error', 'live tool update is unsupported');\n }\n }\n\n async close(): Promise<{ usage: Usage }> {\n if (this.#meters !== undefined) return { usage: this.#meters };\n this.#closed = true;\n this.#ledger.settle();\n this.#wire.close();\n this.#end();\n this.#meters = this.#ledger.usage(this.model.audioCost);\n return { usage: this.#meters };\n }\n\n #client(value: string, turnComplete: boolean): JsonObject {\n return {\n clientContent: {\n turns: [{ role: 'user', parts: [{ text: value }] }],\n turnComplete,\n },\n };\n }\n\n async *[Symbol.asyncIterator](): AsyncIterator<RealtimeEvent> {\n while (true) {\n const queued = this.#queue.shift();\n if (queued !== undefined) {\n yield queued;\n continue;\n }\n if (this.#closed) return;\n const next = await new Promise<IteratorResultEvent>((resolve) => {\n this.#waiters.push(resolve);\n });\n if (next.done === true || next.value === undefined) return;\n yield next.value;\n }\n }\n\n #audio(encoded: string): void {\n if (encoded.length === 0) return;\n this.#pending.chunks.push({\n data: decodeAudioBase64(encoded),\n format: this.#format,\n timestamp: 0,\n });\n }\n\n #transcript(delta: string, final: boolean): void {\n const end = this.#pending.cursor + delta.length;\n const tagged = delta.length === 0 ? undefined : end;\n for (const event of takeAudio(this.#pending, this.#item, this.#format, tagged)) {\n this.#push(event);\n }\n this.#spoken += delta;\n this.#push({ type: 'output_transcript', itemId: this.#item, text: delta, final });\n }\n\n #input(transcript: string, final: boolean): void {\n if (transcript.length > 0) this.#heard = transcript;\n this.#push({ type: 'input_transcript', itemId: this.#item, text: this.#heard, final });\n }\n\n #finish(interrupted: boolean): void {\n for (const event of takeAudio(this.#pending, this.#item, this.#format, undefined)) {\n this.#push(event);\n }\n if (this.#spoken.length > 0) {\n this.#push({\n type: 'output_transcript',\n itemId: this.#item,\n text: this.#spoken,\n final: true,\n });\n }\n if (this.#heard.length > 0) {\n this.#push({ type: 'input_transcript', itemId: this.#item, text: this.#heard, final: true });\n }\n this.#push({\n type: 'response_done',\n itemId: this.#item,\n interrupted,\n usage: this.#tokens,\n });\n this.#tokens = blankUsage();\n this.#spoken = '';\n this.#heard = '';\n this.#turn += 1;\n this.#item = `g${String(this.#turn)}`;\n this.#pending = { chunks: [], cursor: 0 };\n }\n\n #onSetup(value: unknown): void {\n const done = record(value);\n const sessionId = done === undefined ? '' : text(done.sessionId);\n this.#push({ type: 'session_start', sessionId });\n if (!this.#historySent && this.#turns.length > 0) {\n this.#historySent = true;\n this.#wire.send(\n JSON.stringify({ clientContent: { turns: this.#turns, turnComplete: false } })\n );\n }\n this.#ready = true;\n const pending = this.#outgoing.splice(0);\n for (const event of pending) this.#wire.send(JSON.stringify(event));\n }\n\n #onResume(value: unknown): void {\n const update = record(value);\n if (update === undefined) return;\n const handle = text(update.newHandle);\n if (handle.length === 0 || update.resumable !== true) return;\n this.#handle = handle;\n this.#push({ type: 'resumption', handle });\n }\n\n #onGoAway(value: unknown): 'stop' | VoiceWire {\n const gone = record(value);\n const left = gone === undefined ? 0 : timeLeftMs(gone.timeLeft);\n this.#push({ type: 'go_away', timeLeftMs: left });\n const handle = this.#handle;\n if (handle === undefined) {\n this.#push({ type: 'error', error: 'session ended', fatal: true });\n return 'stop';\n }\n const next = this.#open(this.#target);\n this.#wire = next;\n this.#ready = false;\n next.send(JSON.stringify({ setup: this.#setup(handle) }));\n return next;\n }\n\n #onToolCall(value: unknown): void {\n const call = record(value);\n if (call === undefined || !Array.isArray(call.functionCalls)) return;\n for (const entry of call.functionCalls) {\n const fn = record(entry);\n if (fn === undefined) continue;\n this.#push({\n type: 'tool_call',\n blocking: this.model.toolCalls === 'blocking',\n call: {\n type: 'toolCall',\n id: text(fn.id),\n name: text(fn.name),\n arguments: argumentsOf(fn.args),\n },\n });\n }\n }\n\n #onCancel(value: unknown): void {\n const cancellation = record(value);\n if (cancellation === undefined || !Array.isArray(cancellation.ids)) return;\n for (const id of cancellation.ids) {\n if (typeof id !== 'string') continue;\n this.#push({ type: 'tool_call_cancelled', callId: id });\n }\n }\n\n #onServer(value: unknown): void {\n const content = record(value);\n if (content === undefined) return;\n this.#modelTurn(content.modelTurn);\n this.#caption(content.outputTranscription, 'output');\n this.#caption(content.inputTranscription, 'input');\n if (content.interrupted === true) {\n this.#push({ type: 'input_speech_start' });\n this.#finish(true);\n return;\n }\n if (content.turnComplete === true) this.#finish(false);\n }\n\n #modelTurn(value: unknown): void {\n const turn = record(value);\n if (turn === undefined || !Array.isArray(turn.parts)) return;\n for (const part of turn.parts) {\n const entry = record(part);\n if (entry === undefined) continue;\n const inline = record(entry.inlineData);\n if (inline === undefined) continue;\n if (!text(inline.mimeType).startsWith('audio/')) continue;\n this.#audio(text(inline.data));\n }\n }\n\n #caption(value: unknown, kind: 'input' | 'output'): void {\n const caption = record(value);\n if (caption === undefined) return;\n if (kind === 'input') this.#input(text(caption.text), false);\n else this.#transcript(text(caption.text), false);\n }\n\n #onError(value: unknown): void {\n const detail = record(value);\n if (detail === undefined) {\n const message = text(value);\n this.#push({\n type: 'error',\n error: message.length > 0 ? message : 'realtime error',\n fatal: false,\n });\n return;\n }\n const code = text(detail.code);\n const message = text(detail.message);\n let fatal = false;\n if (code === 'MAX_DURATION' || code === 'SESSION_EXPIRED') fatal = true;\n this.#push({\n type: 'error',\n error: message.length > 0 ? message : 'realtime error',\n fatal,\n });\n }\n\n #apply(event: Record<string, unknown>): 'ok' | 'stop' | VoiceWire {\n if (event.usageMetadata !== undefined) this.#tokens = usageFrom(event.usageMetadata);\n if (event.inputTranscription !== undefined) this.#caption(event.inputTranscription, 'input');\n if (event.outputTranscription !== undefined) this.#caption(event.outputTranscription, 'output');\n if (event.setupComplete !== undefined) this.#onSetup(event.setupComplete);\n if (event.sessionResumptionUpdate !== undefined) this.#onResume(event.sessionResumptionUpdate);\n if (event.toolCall !== undefined) this.#onToolCall(event.toolCall);\n if (event.toolCallCancellation !== undefined) this.#onCancel(event.toolCallCancellation);\n if (event.serverContent !== undefined) this.#onServer(event.serverContent);\n if (event.error !== undefined) this.#onError(event.error);\n if (event.goAway !== undefined) return this.#onGoAway(event.goAway);\n return 'ok';\n }\n\n #ingest(raw: string): 'ok' | 'stop' | VoiceWire {\n let parsed: unknown;\n try {\n parsed = JSON.parse(raw);\n } catch {\n this.#push({ type: 'error', error: 'invalid realtime event', fatal: false });\n return 'ok';\n }\n const event = record(parsed);\n if (event === undefined) return 'ok';\n return this.#apply(event);\n }\n\n async #read(wire: VoiceWire): Promise<void> {\n let reconnect: VoiceWire | undefined;\n try {\n for await (const raw of wire.incoming) {\n const action = this.#ingest(raw);\n if (action === 'stop') return;\n if (action !== 'ok') {\n reconnect = action;\n return;\n }\n }\n } catch (error) {\n const message = error instanceof Error ? error.message : String(error);\n this.#push({ type: 'error', error: message, fatal: true });\n } finally {\n if (reconnect !== undefined) {\n wire.close();\n void this.#read(reconnect);\n } else {\n this.#closed = true;\n this.#end();\n }\n }\n }\n}\n\nexport async function connectGeminiSession(\n model: RealtimeModel<string>,\n context: RealtimeContext,\n options: RealtimeOptions | undefined,\n gemini: GeminiOptions\n): Promise<RealtimeSession> {\n if (options?.signal?.aborted === true) {\n throw new VoiceRequestError('aborted', 'session aborted');\n }\n const { secret, ephemeral } = credential(options);\n const socket = realtimeSocket(sessionUrl(model, secret, ephemeral), options, []);\n const target: VoiceWireTarget = {\n url: socket.url,\n protocols: socket.protocols,\n headers: headersFor(options),\n };\n const open =\n gemini.wire !== undefined\n ? gemini.wire\n : (wireTarget: VoiceWireTarget) => openVoiceSocket(wireTarget, gemini.socket);\n const session = new GeminiSession(model, context, gemini.vocabulary, open, target, open(target));\n const handle = options?.resumeHandle;\n session.start(handle);\n const signal = options?.signal;\n if (signal !== undefined) {\n signal.addEventListener('abort', () => void session.close(), { once: true });\n }\n return session;\n}\n\nexport function pcmFormat(sampleRate: number): AudioFormat {\n return { encoding: 'pcm16', sampleRate, channels: 1 };\n}\n"],
|
|
5
|
-
"mappings": "AAEA,SAAS,kBAAkB;AAC3B,SAAS,mBAAmB,yBAAyB;AACrD,SAAS,yBAAyB;AAElC;AAAA,EACE;AAAA,OAIK;AAOP,SAAS,gBAAgB,gBAAgB,sBAAsB;AAU/D,MAAM,aAAuC;AAAA,EAC3C,WAAW;AAAA,EACX,WAAW;AAAA,EACX,QAAQ;AACV;AAYA,SAAS,OAAO,OAAqD;AACnE,MAAI,UAAU,QAAQ,OAAO,UAAU,YAAY,MAAM,QAAQ,KAAK,EAAG,QAAO;AAChF,SAAO;AACT;AAEA,SAAS,KAAK,OAAwB;AACpC,SAAO,OAAO,UAAU,WAAW,QAAQ;AAC7C;AAEA,SAAS,WAAW,SAA8E;AAChG,QAAM,QAAQ,SAAS,OAAO;AAC9B,MAAI,OAAO,UAAU,YAAY,MAAM,SAAS,EAAG,QAAO,EAAE,QAAQ,OAAO,WAAW,KAAK;AAC3F,QAAM,SAAS,SAAS;AACxB,MAAI,OAAO,WAAW,YAAY,OAAO,SAAS,EAAG,QAAO,EAAE,QAAQ,QAAQ,WAAW,MAAM;AAC/F,QAAM,IAAI,kBAAkB,SAAS,6BAA6B;AACpE;AAEA,SAAS,WAAW,SAA8D;AAChF,QAAM,UAAkC,CAAC;AACzC,QAAM,QAAQ,SAAS;AACvB,MAAI,UAAU,OAAW,QAAO;AAChC,aAAW,CAAC,KAAK,KAAK,KAAK,OAAO,QAAQ,KAAK,GAAG;AAChD,QAAI,UAAU,KAAM,QAAO,QAAQ,GAAG;AAAA,QACjC,SAAQ,GAAG,IAAI;AAAA,EACtB;AACA,SAAO;AACT;AAEA,SAAS,WAAW,OAA8B,QAAgB,WAA4B;AAC5F,QAAM,MAAM,IAAI,IAAI,MAAM,OAAO;AACjC,MAAI,UAAW,KAAI,aAAa,IAAI,gBAAgB,MAAM;AAAA,MACrD,KAAI,aAAa,IAAI,OAAO,MAAM;AACvC,SAAO,IAAI,SAAS;AACtB;AAEA,SAAS,WAAW,OAAwB;AAC1C,MAAI,OAAO,UAAU,YAAY,OAAO,SAAS,KAAK,EAAG,QAAO,KAAK,MAAM,QAAQ,GAAI;AACvF,MAAI,OAAO,UAAU,UAAU;AAC7B,UAAM,UAAU,qBAAqB,KAAK,KAAK;AAC/C,QAAI,YAAY,KAAM,QAAO,KAAK,MAAM,OAAO,QAAQ,CAAC,CAAC,IAAI,GAAI;AACjE,UAAM,WAAW,OAAO,KAAK;AAC7B,QAAI,OAAO,SAAS,QAAQ,EAAG,QAAO,KAAK,MAAM,WAAW,GAAI;AAAA,EAClE;AACA,QAAM,WAAW,OAAO,KAAK;AAC7B,MAAI,aAAa,UAAa,OAAO,SAAS,YAAY,UAAU;AAClE,UAAM,QAAQ,OAAO,SAAS,UAAU,WAAW,SAAS,QAAQ;AACpE,WAAO,KAAK,MAAM,SAAS,UAAU,MAAO,QAAQ,GAAG;AAAA,EACzD;AACA,SAAO;AACT;AAEA,SAAS,MAAM,QAAiC,KAAqB;AACnE,QAAM,QAAQ,OAAO,GAAG;AACxB,SAAO,OAAO,UAAU,WAAW,QAAQ;AAC7C;AAEA,SAAS,UAAU,OAAuB;AACxC,QAAM,QAAQ,OAAO,KAAK;AAC1B,MAAI,UAAU,OAAW,QAAO,WAAW;AAC3C,QAAM,QAAQ,MAAM,OAAO,kBAAkB;AAC7C,QAAM,WAAW,MAAM,OAAO,oBAAoB;AAClD,QAAM,aAAa,MAAM,OAAO,sBAAsB;AACtD,QAAM,SAAS,WAAW,IAAI,WAAW;AACzC,SAAO;AAAA,IACL;AAAA,IACA;AAAA,IACA,WAAW;AAAA,IACX,YAAY;AAAA,IACZ,aAAa,QAAQ;AAAA,IACrB,MAAM,EAAE,OAAO,GAAG,QAAQ,GAAG,WAAW,GAAG,YAAY,GAAG,OAAO,EAAE;AAAA,EACrE;AACF;AAEA,SAAS,YAAY,OAA4B;AAC/C,MAAI,OAAO,UAAU,UAAU;AAC7B,QAAI;AACF,YAAM,SAAkB,KAAK,MAAM,KAAK;AACxC,YAAMA,UAAS,OAAO,MAAM;AAC5B,UAAIA,YAAW,OAAW,QAAO,CAAC;AAClC,aAAOA;AAAA,IACT,QAAQ;AACN,aAAO,CAAC;AAAA,IACV;AAAA,EACF;AACA,QAAM,SAAS,OAAO,KAAK;AAC3B,MAAI,WAAW,OAAW,QAAO,CAAC;AAClC,SAAO;AACT;AAEA,SAAS,WAAW,QAAmC;AACrD,QAAM,QAAkB,CAAC;AACzB,aAAW,QAAQ,OAAO,SAAS;AACjC,QAAI,KAAK,SAAS,OAAQ,OAAM,KAAK,KAAK,IAAI;AAAA,EAChD;AACA,SAAO,MAAM,KAAK,IAAI;AACxB;AAEA,SAAS,aAAa,UAA+C;AACnE,MAAI,aAAa,OAAW,QAAO,CAAC;AACpC,QAAM,QAAsB,CAAC;AAC7B,aAAW,WAAW,UAAU;AAC9B,QAAI,QAAQ,SAAS,UAAU,QAAQ,SAAS,YAAa;AAC7D,UAAM,QAAsB,CAAC;AAC7B,eAAW,QAAQ,QAAQ,SAAS;AAClC,UAAI,OAAO,SAAS,UAAU;AAC5B,cAAM,KAAK,EAAE,MAAM,KAAK,CAAC;AACzB;AAAA,MACF;AACA,UAAI,KAAK,SAAS,OAAQ,OAAM,KAAK,EAAE,MAAM,KAAK,KAAK,CAAC;AAAA,IAC1D;AACA,QAAI,MAAM,WAAW,EAAG;AACxB,UAAM,KAAK,EAAE,MAAM,QAAQ,SAAS,cAAc,UAAU,QAAQ,MAAM,CAAC;AAAA,EAC7E;AACA,SAAO;AACT;AAEA,SAAS,YAAY,MAAY,OAA0C;AACzE,QAAM,KAAiB;AAAA,IACrB,MAAM,KAAK;AAAA,IACX,aAAa,KAAK;AAAA,IAClB,YAAY,KAAK;AAAA,EACnB;AACA,MAAI,MAAM,OAAO,qCAAsC,IAAG,WAAW;AACrE,MAAI,MAAM,cAAc,WAAY,IAAG,WAAW;AAClD,SAAO;AACT;AAEA,SAAS,cAAc,OAA8B,YAA8C;AACjG,MAAI,MAAM,OAAO,kBAAmB,QAAO,CAAC;AAC5C,MAAI,eAAe,OAAW,QAAO,CAAC;AACtC,SAAO,EAAE,kBAAkB,WAAW;AACxC;AAEA,SAAS,UACP,OACA,SACA,QACA,YACA,OACY;AACZ,QAAM,QAAoB;AAAA,IACxB,OAAO,UAAU,MAAM,EAAE;AAAA,IACzB,kBAAkB,EAAE,oBAAoB,CAAC,OAAO,EAAE;AAAA,IAClD,mBAAmB,EAAE,OAAO,CAAC,EAAE,MAAM,QAAQ,aAAa,CAAC,EAAE;AAAA,IAC7D,cAAc;AAAA,MACZ,aAAa,EAAE,qBAAqB,EAAE,WAAW,QAAQ,MAAM,EAAE;AAAA,IACnE;AAAA,IACA,yBAAyB,cAAc,OAAO,UAAU;AAAA,IACxD,0BAA0B,CAAC;AAAA,IAC3B,0BAA0B;AAAA,MACxB,eAAe;AAAA,MACf,eAAe,EAAE,cAAc,IAAO;AAAA,IACxC;AAAA,IACA,mBAAmB,WAAW,SAAY,CAAC,IAAI,EAAE,OAAO;AAAA,EAC1D;AACA,MAAI,QAAQ,MAAM,SAAS,GAAG;AAC5B,UAAM,QAAQ,CAAC,EAAE,sBAAsB,QAAQ,MAAM,IAAI,CAAC,SAAS,YAAY,MAAM,KAAK,CAAC,EAAE,CAAC;AAAA,EAChG;AACA,MAAI,MAAM,SAAS,EAAG,OAAM,gBAAgB,EAAE,+BAA+B,KAAK;AAClF,SAAO;AACT;AAEA,SAAS,QAAQ,QAAgB,OAAe,KAAuB;AACrE,SAAO,EAAE,QAAQ,OAAO,IAAI;AAC9B;AAEA,SAAS,UACP,SACA,QACA,QACA,KACiB;AACjB,QAAM,SAAS,QAAQ;AACvB,UAAQ,SAAS,CAAC;AAClB,MAAI,OAAO,WAAW,EAAG,QAAO,CAAC;AACjC,QAAM,QAAQ,QAAQ;AACtB,MAAI,QAAQ,OAAW,SAAQ,SAAS;AACxC,QAAM,QAAQ,QAAQ,SAAY,IAAI,MAAM;AAC5C,QAAM,QAAQ,OAAO,OAAO,CAAC,KAAK,UAAU,MAAM,MAAM,KAAK,QAAQ,CAAC;AACtE,MAAI,OAAO;AACX,MAAI,SAAS;AACb,SAAO,OAAO,IAAI,CAAC,OAAO,UAAU;AAClC,UAAM,QAAuB;AAAA,MAC3B,MAAM;AAAA,MACN;AAAA,MACA,OAAO,EAAE,MAAM,MAAM,MAAM,QAAQ,WAAW,MAAM,UAAU;AAAA,IAChE;AACA,QAAI,QAAQ,OAAW,QAAO;AAC9B,UAAM,OAAO,UAAU,OAAO,SAAS;AACvC,UAAM,OAAO,OAAO,MAAM,QAAQ,KAAK,OAAQ,OAAO,MAAM,KAAK,UAAU,QAAS,KAAK;AACzF,YAAQ,MAAM,KAAK;AACnB,UAAM,OAAO,QAAQ,QAAQ,QAAQ,IAAI;AACzC,aAAS;AACT,WAAO;AAAA,EACT,CAAC;AACH;AAEA,MAAM,cAAyC;AAAA,EACpC;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA,UAAU,IAAI,eAAe;AAAA,EAC7B,SAA0B,CAAC;AAAA,EAC3B,WAAyD,CAAC;AAAA,EAC1D,YAA0B,CAAC;AAAA,EAC3B;AAAA,EACT;AAAA,EACA,WAAyB,EAAE,QAAQ,CAAC,GAAG,QAAQ,EAAE;AAAA,EACjD,QAAQ;AAAA,EACR,QAAQ;AAAA,EACR,SAAS;AAAA,EACT,UAAU;AAAA,EACV;AAAA,EACA,UAAiB,WAAW;AAAA,EAC5B,SAAS;AAAA,EACT,eAAe;AAAA,EACf,UAAU;AAAA,EACV,SAAS;AAAA,EACT;AAAA,EAEA,YACE,OACA,SACA,YACA,MACA,QACA,MACA;AACA,SAAK,QAAQ;AACb,SAAK,WAAW;AAChB,SAAK,cAAc;AACnB,SAAK,SAAS,aAAa,QAAQ,OAAO;AAC1C,SAAK,QAAQ;AACb,SAAK,UAAU;AACf,SAAK,QAAQ;AACb,SAAK,UAAU,MAAM;AAAA,EACvB;AAAA,EAEA,MAAM,QAAkC;AACtC,QAAI,WAAW,UAAa,OAAO,SAAS,EAAG,MAAK,UAAU;AAC9D,SAAK,MAAM,KAAK,KAAK,UAAU,EAAE,OAAO,KAAK,OAAO,MAAM,EAAE,CAAC,CAAC;AAC9D,SAAK,KAAK,MAAM,KAAK,KAAK;AAAA,EAC5B;AAAA,EAEA,OAAO,QAAwC;AAC7C,WAAO,UAAU,KAAK,OAAO,KAAK,UAAU,QAAQ,KAAK,aAAa,KAAK,MAAM;AAAA,EACnF;AAAA,EAEA,cAAoB;AAClB,QAAI,KAAK,QAAS,OAAM,IAAI,kBAAkB,SAAS,mBAAmB;AAAA,EAC5E;AAAA,EAEA,MAAM,OAAyB;AAC7B,QAAI,CAAC,KAAK,QAAQ;AAChB,WAAK,UAAU,KAAK,KAAK;AACzB;AAAA,IACF;AACA,SAAK,MAAM,KAAK,KAAK,UAAU,KAAK,CAAC;AAAA,EACvC;AAAA,EAEA,MAAM,OAA4B;AAChC,QAAI,KAAK,QAAS;AAClB,SAAK,QAAQ,QAAQ,KAAK;AAC1B,UAAM,SAAS,KAAK,SAAS,MAAM;AACnC,QAAI,WAAW,OAAW,QAAO,EAAE,OAAO,OAAO,MAAM,MAAM,CAAC;AAAA,QACzD,MAAK,OAAO,KAAK,KAAK;AAAA,EAC7B;AAAA,EAEA,OAAa;AACX,QAAI,KAAK,OAAQ;AACjB,SAAK,SAAS;AACd,WAAO,KAAK,SAAS,SAAS,EAAG,MAAK,SAAS,MAAM,IAAI,EAAE,MAAM,KAAK,CAAC;AAAA,EACzE;AAAA,EAEA,YAAY,OAAyB;AACnC,SAAK,YAAY;AACjB,SAAK,QAAQ,SAAS,KAAK;AAC3B,SAAK,MAAM;AAAA,MACT,eAAe;AAAA,QACb,OAAO;AAAA,UACL,MAAM,kBAAkB,MAAM,IAAI;AAAA,UAClC,UAAU,kBAAkB,OAAO,MAAM,OAAO,UAAU,CAAC;AAAA,QAC7D;AAAA,MACF;AAAA,IACF,CAAC;AAAA,EACH;AAAA,EAEA,iBAAiB,MAAwB;AACvC,SAAK,YAAY;AACjB,QAAI,KAAK,MAAM,UAAU,QAAW;AAClC,YAAM,IAAI,kBAAkB,SAAS,sBAAsB;AAAA,IAC7D;AACA,SAAK,MAAM;AAAA,MACT,eAAe,EAAE,OAAO,EAAE,MAAM,kBAAkB,IAAI,GAAG,UAAU,aAAa,EAAE;AAAA,IACpF,CAAC;AAAA,EACH;AAAA,EAEA,SAAS,OAAqB;AAC5B,SAAK,YAAY;AACjB,SAAK,QAAQ,QAAQ;AACrB,SAAK,MAAM,KAAK,QAAQ,OAAO,IAAI,CAAC;AAAA,EACtC;AAAA,EAEA,OAAO,OAAe,UAA0B;AAC9C,SAAK,YAAY;AACjB,SAAK,QAAQ,OAAO,KAAK,MAAM,UAAU,UAAU,MAAM;AACvD,WAAK,YAAY;AACjB,WAAK,QAAQ,QAAQ;AACrB,UAAI,aAAa,SAAU,MAAK,MAAM,KAAK,QAAQ,OAAO,KAAK,CAAC;AAAA,eACvD,aAAa,YAAa,MAAK,MAAM,EAAE,eAAe,EAAE,MAAM,MAAM,EAAE,CAAC;AAAA,UAC3E,MAAK,MAAM,KAAK,QAAQ,OAAO,IAAI,CAAC;AAAA,IAC3C,CAAC;AAAA,EACH;AAAA,EAEA,QAAQ,QAAgB,QAA2B,UAA2B;AAC5E,SAAK,YAAY;AACjB,UAAM,OAAO,MAAM;AACjB,YAAM,OAAO,aAAa,SAAY,cAAc;AACpD,WAAK,MAAM;AAAA,QACT,cAAc;AAAA,UACZ,mBAAmB;AAAA,YACjB;AAAA,cACE,IAAI;AAAA,cACJ,MAAM,OAAO;AAAA,cACb,YAAY,WAAW,IAAI;AAAA,cAC3B,UAAU;AAAA,gBACR,QAAQ,OAAO,UAAU,UAAU;AAAA,gBACnC,WAAW;AAAA,gBACX,SAAS,WAAW,MAAM;AAAA,cAC5B;AAAA,YACF;AAAA,UACF;AAAA,QACF;AAAA,MACF,CAAC;AAAA,IACH;AACA,QAAI,aAAa,QAAW;AAC1B,WAAK;AACL;AAAA,IACF;AACA,SAAK,QAAQ,OAAO,KAAK,MAAM,UAAU,UAAU,MAAM;AACvD,WAAK,YAAY;AACjB,WAAK;AAAA,IACP,CAAC;AAAA,EACH;AAAA,EAEA,UAAU,OAAmD;AAC3D,SAAK,YAAY;AACjB,mBAAe,KAAK,MAAM,YAAY,KAAK;AAC3C,SAAK,QAAQ,OAAO;AAAA,EACtB;AAAA,EAEA,OAAO,OAAiF;AACtF,SAAK,YAAY;AACjB,QAAI,MAAM,UAAU,UAAa,CAAC,KAAK,MAAM,gBAAgB;AAC3D,YAAM,IAAI,kBAAkB,SAAS,iCAAiC;AAAA,IACxE;AAAA,EACF;AAAA,EAEA,MAAM,QAAmC;AACvC,QAAI,KAAK,YAAY,OAAW,QAAO,EAAE,OAAO,KAAK,QAAQ;AAC7D,SAAK,UAAU;AACf,SAAK,QAAQ,OAAO;AACpB,SAAK,MAAM,MAAM;AACjB,SAAK,KAAK;AACV,SAAK,UAAU,KAAK,QAAQ,MAAM,KAAK,MAAM,SAAS;AACtD,WAAO,EAAE,OAAO,KAAK,QAAQ;AAAA,EAC/B;AAAA,EAEA,QAAQ,OAAe,cAAmC;AACxD,WAAO;AAAA,MACL,eAAe;AAAA,QACb,OAAO,CAAC,EAAE,MAAM,QAAQ,OAAO,CAAC,EAAE,MAAM,MAAM,CAAC,EAAE,CAAC;AAAA,QAClD;AAAA,MACF;AAAA,IACF;AAAA,EACF;AAAA,EAEA,QAAQ,OAAO,aAAa,IAAkC;AAC5D,WAAO,MAAM;AACX,YAAM,SAAS,KAAK,OAAO,MAAM;AACjC,UAAI,WAAW,QAAW;AACxB,cAAM;AACN;AAAA,MACF;AACA,UAAI,KAAK,QAAS;AAClB,YAAM,OAAO,MAAM,IAAI,QAA6B,CAAC,YAAY;AAC/D,aAAK,SAAS,KAAK,OAAO;AAAA,MAC5B,CAAC;AACD,UAAI,KAAK,SAAS,QAAQ,KAAK,UAAU,OAAW;AACpD,YAAM,KAAK;AAAA,IACb;AAAA,EACF;AAAA,EAEA,OAAO,SAAuB;AAC5B,QAAI,QAAQ,WAAW,EAAG;AAC1B,SAAK,SAAS,OAAO,KAAK;AAAA,MACxB,MAAM,kBAAkB,OAAO;AAAA,MAC/B,QAAQ,KAAK;AAAA,MACb,WAAW;AAAA,IACb,CAAC;AAAA,EACH;AAAA,EAEA,YAAY,OAAe,OAAsB;AAC/C,UAAM,MAAM,KAAK,SAAS,SAAS,MAAM;AACzC,UAAM,SAAS,MAAM,WAAW,IAAI,SAAY;AAChD,eAAW,SAAS,UAAU,KAAK,UAAU,KAAK,OAAO,KAAK,SAAS,MAAM,GAAG;AAC9E,WAAK,MAAM,KAAK;AAAA,IAClB;AACA,SAAK,WAAW;AAChB,SAAK,MAAM,EAAE,MAAM,qBAAqB,QAAQ,KAAK,OAAO,MAAM,OAAO,MAAM,CAAC;AAAA,EAClF;AAAA,EAEA,OAAO,YAAoB,OAAsB;AAC/C,QAAI,WAAW,SAAS,EAAG,MAAK,SAAS;AACzC,SAAK,MAAM,EAAE,MAAM,oBAAoB,QAAQ,KAAK,OAAO,MAAM,KAAK,QAAQ,MAAM,CAAC;AAAA,EACvF;AAAA,EAEA,QAAQ,aAA4B;AAClC,eAAW,SAAS,UAAU,KAAK,UAAU,KAAK,OAAO,KAAK,SAAS,MAAS,GAAG;AACjF,WAAK,MAAM,KAAK;AAAA,IAClB;AACA,QAAI,KAAK,QAAQ,SAAS,GAAG;AAC3B,WAAK,MAAM;AAAA,QACT,MAAM;AAAA,QACN,QAAQ,KAAK;AAAA,QACb,MAAM,KAAK;AAAA,QACX,OAAO;AAAA,MACT,CAAC;AAAA,IACH;AACA,QAAI,KAAK,OAAO,SAAS,GAAG;AAC1B,WAAK,MAAM,EAAE,MAAM,oBAAoB,QAAQ,KAAK,OAAO,MAAM,KAAK,QAAQ,OAAO,KAAK,CAAC;AAAA,IAC7F;AACA,SAAK,MAAM;AAAA,MACT,MAAM;AAAA,MACN,QAAQ,KAAK;AAAA,MACb;AAAA,MACA,OAAO,KAAK;AAAA,IACd,CAAC;AACD,SAAK,UAAU,WAAW;AAC1B,SAAK,UAAU;AACf,SAAK,SAAS;AACd,SAAK,SAAS;AACd,SAAK,QAAQ,IAAI,OAAO,KAAK,KAAK,CAAC;AACnC,SAAK,WAAW,EAAE,QAAQ,CAAC,GAAG,QAAQ,EAAE;AAAA,EAC1C;AAAA,EAEA,SAAS,OAAsB;AAC7B,UAAM,OAAO,OAAO,KAAK;AACzB,UAAM,YAAY,SAAS,SAAY,KAAK,KAAK,KAAK,SAAS;AAC/D,SAAK,MAAM,EAAE,MAAM,iBAAiB,UAAU,CAAC;AAC/C,QAAI,CAAC,KAAK,gBAAgB,KAAK,OAAO,SAAS,GAAG;AAChD,WAAK,eAAe;AACpB,WAAK,MAAM;AAAA,QACT,KAAK,UAAU,EAAE,eAAe,EAAE,OAAO,KAAK,QAAQ,cAAc,MAAM,EAAE,CAAC;AAAA,MAC/E;AAAA,IACF;AACA,SAAK,SAAS;AACd,UAAM,UAAU,KAAK,UAAU,OAAO,CAAC;AACvC,eAAW,SAAS,QAAS,MAAK,MAAM,KAAK,KAAK,UAAU,KAAK,CAAC;AAAA,EACpE;AAAA,EAEA,UAAU,OAAsB;AAC9B,UAAM,SAAS,OAAO,KAAK;AAC3B,QAAI,WAAW,OAAW;AAC1B,UAAM,SAAS,KAAK,OAAO,SAAS;AACpC,QAAI,OAAO,WAAW,KAAK,OAAO,cAAc,KAAM;AACtD,SAAK,UAAU;AACf,SAAK,MAAM,EAAE,MAAM,cAAc,OAAO,CAAC;AAAA,EAC3C;AAAA,EAEA,UAAU,OAAoC;AAC5C,UAAM,OAAO,OAAO,KAAK;AACzB,UAAM,OAAO,SAAS,SAAY,IAAI,WAAW,KAAK,QAAQ;AAC9D,SAAK,MAAM,EAAE,MAAM,WAAW,YAAY,KAAK,CAAC;AAChD,UAAM,SAAS,KAAK;AACpB,QAAI,WAAW,QAAW;AACxB,WAAK,MAAM,EAAE,MAAM,SAAS,OAAO,iBAAiB,OAAO,KAAK,CAAC;AACjE,aAAO;AAAA,IACT;AACA,UAAM,OAAO,KAAK,MAAM,KAAK,OAAO;AACpC,SAAK,QAAQ;AACb,SAAK,SAAS;AACd,SAAK,KAAK,KAAK,UAAU,EAAE,OAAO,KAAK,OAAO,MAAM,EAAE,CAAC,CAAC;AACxD,WAAO;AAAA,EACT;AAAA,EAEA,YAAY,OAAsB;AAChC,UAAM,OAAO,OAAO,KAAK;AACzB,QAAI,SAAS,UAAa,CAAC,MAAM,QAAQ,KAAK,aAAa,EAAG;AAC9D,eAAW,SAAS,KAAK,eAAe;AACtC,YAAM,KAAK,OAAO,KAAK;AACvB,UAAI,OAAO,OAAW;AACtB,WAAK,MAAM;AAAA,QACT,MAAM;AAAA,QACN,UAAU,KAAK,MAAM,cAAc;AAAA,QACnC,MAAM;AAAA,UACJ,MAAM;AAAA,UACN,IAAI,KAAK,GAAG,EAAE;AAAA,UACd,MAAM,KAAK,GAAG,IAAI;AAAA,UAClB,WAAW,YAAY,GAAG,IAAI;AAAA,QAChC;AAAA,MACF,CAAC;AAAA,IACH;AAAA,EACF;AAAA,EAEA,UAAU,OAAsB;AAC9B,UAAM,eAAe,OAAO,KAAK;AACjC,QAAI,iBAAiB,UAAa,CAAC,MAAM,QAAQ,aAAa,GAAG,EAAG;AACpE,eAAW,MAAM,aAAa,KAAK;AACjC,UAAI,OAAO,OAAO,SAAU;AAC5B,WAAK,MAAM,EAAE,MAAM,uBAAuB,QAAQ,GAAG,CAAC;AAAA,IACxD;AAAA,EACF;AAAA,EAEA,UAAU,OAAsB;AAC9B,UAAM,UAAU,OAAO,KAAK;AAC5B,QAAI,YAAY,OAAW;AAC3B,SAAK,WAAW,QAAQ,SAAS;AACjC,SAAK,SAAS,QAAQ,qBAAqB,QAAQ;AACnD,SAAK,SAAS,QAAQ,oBAAoB,OAAO;AACjD,QAAI,QAAQ,gBAAgB,MAAM;AAChC,WAAK,MAAM,EAAE,MAAM,qBAAqB,CAAC;AACzC,WAAK,QAAQ,IAAI;AACjB;AAAA,IACF;AACA,QAAI,QAAQ,iBAAiB,KAAM,MAAK,QAAQ,KAAK;AAAA,EACvD;AAAA,EAEA,WAAW,OAAsB;AAC/B,UAAM,OAAO,OAAO,KAAK;AACzB,QAAI,SAAS,UAAa,CAAC,MAAM,QAAQ,KAAK,KAAK,EAAG;AACtD,eAAW,QAAQ,KAAK,OAAO;AAC7B,YAAM,QAAQ,OAAO,IAAI;AACzB,UAAI,UAAU,OAAW;AACzB,YAAM,SAAS,OAAO,MAAM,UAAU;AACtC,UAAI,WAAW,OAAW;AAC1B,UAAI,CAAC,KAAK,OAAO,QAAQ,EAAE,WAAW,QAAQ,EAAG;AACjD,WAAK,OAAO,KAAK,OAAO,IAAI,CAAC;AAAA,IAC/B;AAAA,EACF;AAAA,EAEA,SAAS,OAAgB,MAAgC;AACvD,UAAM,UAAU,OAAO,KAAK;AAC5B,QAAI,YAAY,OAAW;AAC3B,QAAI,SAAS,QAAS,MAAK,OAAO,KAAK,QAAQ,IAAI,GAAG,KAAK;AAAA,QACtD,MAAK,YAAY,KAAK,QAAQ,IAAI,GAAG,KAAK;AAAA,EACjD;AAAA,EAEA,SAAS,OAAsB;AAC7B,UAAM,SAAS,OAAO,KAAK;AAC3B,QAAI,WAAW,QAAW;AACxB,YAAMC,WAAU,KAAK,KAAK;AAC1B,WAAK,MAAM;AAAA,QACT,MAAM;AAAA,QACN,OAAOA,SAAQ,SAAS,IAAIA,WAAU;AAAA,QACtC,OAAO;AAAA,MACT,CAAC;AACD;AAAA,IACF;AACA,UAAM,OAAO,KAAK,OAAO,IAAI;AAC7B,UAAM,UAAU,KAAK,OAAO,OAAO;AACnC,QAAI,QAAQ;AACZ,QAAI,SAAS,kBAAkB,SAAS,kBAAmB,SAAQ;AACnE,SAAK,MAAM;AAAA,MACT,MAAM;AAAA,MACN,OAAO,QAAQ,SAAS,IAAI,UAAU;AAAA,MACtC;AAAA,IACF,CAAC;AAAA,EACH;AAAA,EAEA,OAAO,OAA2D;AAChE,QAAI,MAAM,kBAAkB,OAAW,MAAK,UAAU,UAAU,MAAM,aAAa;AACnF,QAAI,MAAM,uBAAuB,OAAW,MAAK,SAAS,MAAM,oBAAoB,OAAO;AAC3F,QAAI,MAAM,wBAAwB,OAAW,MAAK,SAAS,MAAM,qBAAqB,QAAQ;AAC9F,QAAI,MAAM,kBAAkB,OAAW,MAAK,SAAS,MAAM,aAAa;AACxE,QAAI,MAAM,4BAA4B,OAAW,MAAK,UAAU,MAAM,uBAAuB;AAC7F,QAAI,MAAM,aAAa,OAAW,MAAK,YAAY,MAAM,QAAQ;AACjE,QAAI,MAAM,yBAAyB,OAAW,MAAK,UAAU,MAAM,oBAAoB;AACvF,QAAI,MAAM,kBAAkB,OAAW,MAAK,UAAU,MAAM,aAAa;AACzE,QAAI,MAAM,UAAU,OAAW,MAAK,SAAS,MAAM,KAAK;AACxD,QAAI,MAAM,WAAW,OAAW,QAAO,KAAK,UAAU,MAAM,MAAM;AAClE,WAAO;AAAA,EACT;AAAA,EAEA,QAAQ,KAAwC;AAC9C,QAAI;AACJ,QAAI;AACF,eAAS,KAAK,MAAM,GAAG;AAAA,IACzB,QAAQ;AACN,WAAK,MAAM,EAAE,MAAM,SAAS,OAAO,0BAA0B,OAAO,MAAM,CAAC;AAC3E,aAAO;AAAA,IACT;AACA,UAAM,QAAQ,OAAO,MAAM;AAC3B,QAAI,UAAU,OAAW,QAAO;AAChC,WAAO,KAAK,OAAO,KAAK;AAAA,EAC1B;AAAA,EAEA,MAAM,MAAM,MAAgC;AAC1C,QAAI;AACJ,QAAI;AACF,uBAAiB,OAAO,KAAK,UAAU;AACrC,cAAM,SAAS,KAAK,QAAQ,GAAG;AAC/B,YAAI,WAAW,OAAQ;AACvB,YAAI,WAAW,MAAM;AACnB,sBAAY;AACZ;AAAA,QACF;AAAA,MACF;AAAA,IACF,SAAS,OAAO;AACd,YAAM,UAAU,iBAAiB,QAAQ,MAAM,UAAU,OAAO,KAAK;AACrE,WAAK,MAAM,EAAE,MAAM,SAAS,OAAO,SAAS,OAAO,KAAK,CAAC;AAAA,IAC3D,UAAE;AACA,UAAI,cAAc,QAAW;AAC3B,aAAK,MAAM;AACX,aAAK,KAAK,MAAM,SAAS;AAAA,MAC3B,OAAO;AACL,aAAK,UAAU;AACf,aAAK,KAAK;AAAA,MACZ;AAAA,IACF;AAAA,EACF;AACF;AAEA,eAAsB,qBACpB,OACA,SACA,SACA,QAC0B;AAC1B,MAAI,SAAS,QAAQ,YAAY,MAAM;AACrC,UAAM,IAAI,kBAAkB,WAAW,iBAAiB;AAAA,EAC1D;AACA,QAAM,EAAE,QAAQ,UAAU,IAAI,WAAW,OAAO;AAChD,QAAM,SAAS,eAAe,WAAW,OAAO,QAAQ,SAAS,GAAG,SAAS,CAAC,CAAC;AAC/E,QAAM,SAA0B;AAAA,IAC9B,KAAK,OAAO;AAAA,IACZ,WAAW,OAAO;AAAA,IAClB,SAAS,WAAW,OAAO;AAAA,EAC7B;AACA,QAAM,OACJ,OAAO,SAAS,SACZ,OAAO,OACP,CAAC,eAAgC,gBAAgB,YAAY,OAAO,MAAM;AAChF,QAAM,UAAU,IAAI,cAAc,OAAO,SAAS,OAAO,YAAY,MAAM,QAAQ,KAAK,MAAM,CAAC;AAC/F,QAAM,SAAS,SAAS;AACxB,UAAQ,MAAM,MAAM;AACpB,QAAM,SAAS,SAAS;AACxB,MAAI,WAAW,QAAW;AACxB,WAAO,iBAAiB,SAAS,MAAM,KAAK,QAAQ,MAAM,GAAG,EAAE,MAAM,KAAK,CAAC;AAAA,EAC7E;AACA,SAAO;AACT;AAEO,SAAS,UAAU,YAAiC;AACzD,SAAO,EAAE,UAAU,SAAS,YAAY,UAAU,EAAE;AACtD;",
|
|
4
|
+
"sourcesContent": ["import type { JsonObject, Message, Tool, ToolResultMessage, Usage } from '@earendil-works/pi-ai';\nimport type { AudioChunk, AudioFormat, TextSpan } from '../audio.ts';\nimport { blankUsage } from '../blank-usage.ts';\nimport { decodeAudioBase64, encodeAudioBase64 } from '../convert.ts';\nimport { VoiceRequestError } from '../errors.ts';\nimport type { Delivery, RealtimeModel } from '../models.ts';\nimport {\n openVoiceSocket,\n type VoiceSocketConstructor,\n type VoiceWire,\n type VoiceWireTarget,\n} from '../openai-realtime/socket.ts';\nimport type {\n RealtimeContext,\n RealtimeEvent,\n RealtimeOptions,\n RealtimeSession,\n} from '../realtime.ts';\nimport { planTruncation, RealtimeLedger, realtimeSocket } from '../realtime.ts';\n\nexport type VoiceWireFactory = (target: VoiceWireTarget) => VoiceWire;\n\nexport interface GeminiOptions {\n wire?: VoiceWireFactory;\n socket?: VoiceSocketConstructor;\n vocabulary?: string[];\n}\n\nconst scheduling: Record<Delivery, string> = {\n interrupt: 'INTERRUPT',\n when_idle: 'WHEN_IDLE',\n silent: 'SILENT',\n};\n\ninterface PendingAudio {\n chunks: AudioChunk[];\n cursor: number;\n}\n\ninterface IteratorResultEvent {\n value?: RealtimeEvent;\n done: boolean;\n}\n\nfunction record(value: unknown): Record<string, unknown> | undefined {\n if (value === null || typeof value !== 'object' || Array.isArray(value)) return undefined;\n return value as Record<string, unknown>;\n}\n\nfunction text(value: unknown): string {\n return typeof value === 'string' ? value : '';\n}\n\nfunction credential(options: RealtimeOptions | undefined): { secret: string; ephemeral: boolean } {\n const token = options?.token?.value;\n if (typeof token === 'string' && token.length > 0) return { secret: token, ephemeral: true };\n const apiKey = options?.apiKey;\n if (typeof apiKey === 'string' && apiKey.length > 0) return { secret: apiKey, ephemeral: false };\n throw new VoiceRequestError('error', 'missing realtime credential');\n}\n\nfunction headersFor(options: RealtimeOptions | undefined): Record<string, string> {\n const headers: Record<string, string> = {};\n const extra = options?.headers;\n if (extra === undefined) return headers;\n for (const [key, value] of Object.entries(extra)) {\n if (value === null) delete headers[key];\n else headers[key] = value;\n }\n return headers;\n}\n\nfunction sessionUrl(model: RealtimeModel<string>, secret: string, ephemeral: boolean): string {\n const url = new URL(model.baseUrl);\n if (ephemeral) url.searchParams.set('access_token', secret);\n else url.searchParams.set('key', secret);\n return url.toString();\n}\n\nfunction timeLeftMs(value: unknown): number {\n if (typeof value === 'number' && Number.isFinite(value)) return Math.round(value * 1000);\n if (typeof value === 'string') {\n const matched = /^(\\d+(?:\\.\\d+)?)s$/.exec(value);\n if (matched !== null) return Math.round(Number(matched[1]) * 1000);\n const asNumber = Number(value);\n if (Number.isFinite(asNumber)) return Math.round(asNumber * 1000);\n }\n const duration = record(value);\n if (duration !== undefined && typeof duration.seconds === 'number') {\n const nanos = typeof duration.nanos === 'number' ? duration.nanos : 0;\n return Math.round(duration.seconds * 1000 + nanos / 1e6);\n }\n return 0;\n}\n\nfunction count(source: Record<string, unknown>, key: string): number {\n const value = source[key];\n return typeof value === 'number' ? value : 0;\n}\n\nfunction usageFrom(value: unknown): Usage {\n const usage = record(value);\n if (usage === undefined) return blankUsage();\n const input = count(usage, 'promptTokenCount');\n const response = count(usage, 'responseTokenCount');\n const candidates = count(usage, 'candidatesTokenCount');\n const output = response > 0 ? response : candidates;\n return {\n input,\n output,\n cacheRead: 0,\n cacheWrite: 0,\n totalTokens: input + output,\n cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },\n };\n}\n\nfunction argumentsOf(value: unknown): JsonObject {\n if (typeof value === 'string') {\n try {\n const parsed: unknown = JSON.parse(value);\n const object = record(parsed);\n if (object === undefined) return {};\n return object as JsonObject;\n } catch {\n return {};\n }\n }\n const object = record(value);\n if (object === undefined) return {};\n return object as JsonObject;\n}\n\nfunction resultText(result: ToolResultMessage): string {\n const parts: string[] = [];\n for (const part of result.content) {\n if (part.type === 'text') parts.push(part.text);\n }\n return parts.join('\\n');\n}\n\nfunction historyTurns(messages: Message[] | undefined): JsonObject[] {\n if (messages === undefined) return [];\n const turns: JsonObject[] = [];\n for (const message of messages) {\n if (message.role !== 'user' && message.role !== 'assistant') continue;\n const parts: JsonObject[] = [];\n for (const part of message.content) {\n if (typeof part === 'string') {\n parts.push({ text: part });\n continue;\n }\n if (part.type === 'text') parts.push({ text: part.text });\n }\n if (parts.length === 0) continue;\n turns.push({ role: message.role === 'assistant' ? 'model' : 'user', parts });\n }\n return turns;\n}\n\nfunction declaration(tool: Tool, model: RealtimeModel<string>): JsonObject {\n const fn: JsonObject = {\n name: tool.name,\n description: tool.description,\n parameters: tool.parameters as unknown as JsonObject,\n };\n if (model.id === 'gemini-live-2.5-flash-native-audio') fn.behavior = 'NON_BLOCKING';\n if (model.toolCalls === 'blocking') fn.behavior = 'BLOCKING';\n return fn;\n}\n\nfunction transcription(model: RealtimeModel<string>, vocabulary: string[] | undefined): JsonObject {\n if (model.id !== 'gemini-3.8-live') return {};\n if (vocabulary === undefined) return {};\n return { customVocabulary: vocabulary };\n}\n\nfunction setupBody(\n model: RealtimeModel<string>,\n context: RealtimeContext,\n handle: string | undefined,\n vocabulary: string[] | undefined,\n turns: JsonObject[]\n): JsonObject {\n const setup: JsonObject = {\n model: `models/${model.id}`,\n generationConfig: { responseModalities: ['AUDIO'] },\n systemInstruction: { parts: [{ text: context.instructions }] },\n speechConfig: {\n voiceConfig: { prebuiltVoiceConfig: { voiceName: context.voice } },\n },\n inputAudioTranscription: transcription(model, vocabulary),\n outputAudioTranscription: {},\n contextWindowCompression: {\n triggerTokens: 100_000,\n slidingWindow: { targetTokens: 80_000 },\n },\n sessionResumption: handle === undefined ? {} : { handle },\n };\n if (context.tools.length > 0) {\n setup.tools = [{ functionDeclarations: context.tools.map((tool) => declaration(tool, model)) }];\n }\n if (turns.length > 0) setup.historyConfig = { initialHistoryInClientContent: true };\n return setup;\n}\n\nfunction spanFor(itemId: string, start: number, end: number): TextSpan {\n return { itemId, start, end };\n}\n\nfunction takeAudio(\n pending: PendingAudio,\n itemId: string,\n format: AudioFormat,\n end: number | undefined\n): RealtimeEvent[] {\n const chunks = pending.chunks;\n pending.chunks = [];\n if (chunks.length === 0) return [];\n const start = pending.cursor;\n if (end !== undefined) pending.cursor = end;\n const width = end === undefined ? 0 : end - start;\n const total = chunks.reduce((sum, chunk) => sum + chunk.data.length, 0);\n let used = 0;\n let cursor = start;\n return chunks.map((chunk, index) => {\n const event: RealtimeEvent = {\n type: 'output_audio',\n itemId,\n chunk: { data: chunk.data, format, timestamp: chunk.timestamp },\n };\n if (end === undefined) return event;\n const last = index === chunks.length - 1;\n const next = last ? end : start + Math.round(((used + chunk.data.length) / total) * width);\n used += chunk.data.length;\n event.span = spanFor(itemId, cursor, next);\n cursor = next;\n return event;\n });\n}\n\nclass GeminiSession implements RealtimeSession {\n readonly model: RealtimeModel<string>;\n readonly #context: RealtimeContext;\n readonly #vocabulary: string[] | undefined;\n readonly #turns: JsonObject[];\n readonly #open: VoiceWireFactory;\n readonly #target: VoiceWireTarget;\n readonly #ledger = new RealtimeLedger();\n readonly #queue: RealtimeEvent[] = [];\n readonly #waiters: Array<(result: IteratorResultEvent) => void> = [];\n readonly #outgoing: JsonObject[] = [];\n readonly #format: AudioFormat;\n #wire: VoiceWire;\n #pending: PendingAudio = { chunks: [], cursor: 0 };\n #item = 'g0';\n #turn = 0;\n #heard = '';\n #spoken = '';\n #handle: string | undefined;\n #tokens: Usage = blankUsage();\n #ready = false;\n #historySent = false;\n #closed = false;\n #ended = false;\n #meters: Usage | undefined;\n readonly #toolGroups = new Map<string, { pending: Set<string>; responses: JsonObject[] }>();\n readonly #toolGroupOf = new Map<string, string>();\n\n constructor(\n model: RealtimeModel<string>,\n context: RealtimeContext,\n vocabulary: string[] | undefined,\n open: VoiceWireFactory,\n target: VoiceWireTarget,\n wire: VoiceWire\n ) {\n this.model = model;\n this.#context = context;\n this.#vocabulary = vocabulary;\n this.#turns = historyTurns(context.history);\n this.#open = open;\n this.#target = target;\n this.#wire = wire;\n this.#format = model.outputFormat;\n }\n\n start(handle: string | undefined): void {\n if (handle !== undefined && handle.length > 0) this.#handle = handle;\n this.#wire.send(JSON.stringify({ setup: this.#setup(handle) }));\n void this.#read(this.#wire);\n }\n\n #setup(handle: string | undefined): JsonObject {\n return setupBody(this.model, this.#context, handle, this.#vocabulary, this.#turns);\n }\n\n #assertOpen(): void {\n if (this.#closed) throw new VoiceRequestError('error', 'session is closed');\n }\n\n #send(event: JsonObject): void {\n if (!this.#ready) {\n this.#outgoing.push(event);\n return;\n }\n this.#wire.send(JSON.stringify(event));\n }\n\n #push(event: RealtimeEvent): void {\n if (this.#closed) return;\n this.#ledger.observe(event);\n const waiter = this.#waiters.shift();\n if (waiter !== undefined) waiter({ value: event, done: false });\n else this.#queue.push(event);\n }\n\n #end(): void {\n if (this.#ended) return;\n this.#ended = true;\n while (this.#waiters.length > 0) this.#waiters.shift()?.({ done: true });\n }\n\n appendAudio(chunk: AudioChunk): void {\n this.#assertOpen();\n this.#ledger.addInput(chunk);\n this.#send({\n realtimeInput: {\n audio: {\n data: encodeAudioBase64(chunk.data),\n mimeType: `audio/pcm;rate=${String(chunk.format.sampleRate)}`,\n },\n },\n });\n }\n\n appendVideoFrame(jpeg: Uint8Array): void {\n this.#assertOpen();\n if (this.model.video === undefined) {\n throw new VoiceRequestError('error', 'video is unsupported');\n }\n this.#send({\n realtimeInput: { video: { data: encodeAudioBase64(jpeg), mimeType: 'image/jpeg' } },\n });\n }\n\n sendText(value: string): void {\n this.#assertOpen();\n this.#ledger.addText();\n this.#send(this.#client(value, true));\n }\n\n inject(value: string, delivery: Delivery): void {\n this.#assertOpen();\n this.#ledger.submit(this.model.delivery, delivery, () => {\n this.#assertOpen();\n this.#ledger.addText();\n if (delivery === 'silent') this.#send(this.#client(value, false));\n else if (delivery === 'when_idle') this.#send({ realtimeInput: { text: value } });\n else this.#send(this.#client(value, true));\n });\n }\n\n respond(callId: string, result: ToolResultMessage, delivery?: Delivery): void {\n this.#assertOpen();\n const send = () => {\n const mode = delivery === undefined ? 'interrupt' : delivery;\n this.#queueTool(callId, {\n id: callId,\n name: result.toolName,\n scheduling: scheduling[mode],\n response: {\n status: result.isError ? 'error' : 'ok',\n retryable: false,\n message: resultText(result),\n },\n });\n };\n if (delivery === undefined) {\n send();\n return;\n }\n this.#ledger.submit(this.model.delivery, delivery, () => {\n this.#assertOpen();\n send();\n });\n }\n\n interrupt(heard?: { itemId: string; heardMs: number }): void {\n this.#assertOpen();\n planTruncation(this.model.truncation, heard);\n this.#ledger.settle();\n }\n\n update(patch: Partial<Pick<RealtimeContext, 'instructions' | 'tools' | 'voice'>>): void {\n this.#assertOpen();\n if (patch.tools !== undefined && !this.model.liveToolUpdate) {\n throw new VoiceRequestError('error', 'live tool update is unsupported');\n }\n }\n\n async close(): Promise<{ usage: Usage }> {\n if (this.#meters !== undefined) return { usage: this.#meters };\n this.#closed = true;\n this.#ledger.settle();\n this.#wire.close();\n this.#end();\n this.#meters = this.#ledger.usage(this.model.audioCost);\n return { usage: this.#meters };\n }\n\n #client(value: string, turnComplete: boolean): JsonObject {\n return {\n clientContent: {\n turns: [{ role: 'user', parts: [{ text: value }] }],\n turnComplete,\n },\n };\n }\n\n async *[Symbol.asyncIterator](): AsyncIterator<RealtimeEvent> {\n while (true) {\n const queued = this.#queue.shift();\n if (queued !== undefined) {\n yield queued;\n continue;\n }\n if (this.#closed) return;\n const next = await new Promise<IteratorResultEvent>((resolve) => {\n this.#waiters.push(resolve);\n });\n if (next.done === true || next.value === undefined) return;\n yield next.value;\n }\n }\n\n #audio(encoded: string): void {\n if (encoded.length === 0) return;\n this.#pending.chunks.push({\n data: decodeAudioBase64(encoded),\n format: this.#format,\n timestamp: 0,\n });\n }\n\n #transcript(delta: string, final: boolean): void {\n const end = this.#pending.cursor + delta.length;\n const tagged = delta.length === 0 ? undefined : end;\n for (const event of takeAudio(this.#pending, this.#item, this.#format, tagged)) {\n this.#push(event);\n }\n this.#spoken += delta;\n this.#push({ type: 'output_transcript', itemId: this.#item, text: delta, final });\n }\n\n #input(transcript: string, final: boolean): void {\n if (transcript.length > 0) this.#heard = transcript;\n this.#push({ type: 'input_transcript', itemId: this.#item, text: this.#heard, final });\n }\n\n #finish(interrupted: boolean): void {\n for (const event of takeAudio(this.#pending, this.#item, this.#format, undefined)) {\n this.#push(event);\n }\n if (this.#spoken.length > 0) {\n this.#push({\n type: 'output_transcript',\n itemId: this.#item,\n text: this.#spoken,\n final: true,\n });\n }\n if (this.#heard.length > 0) {\n this.#push({ type: 'input_transcript', itemId: this.#item, text: this.#heard, final: true });\n }\n this.#push({\n type: 'response_done',\n itemId: this.#item,\n interrupted,\n usage: this.#tokens,\n });\n this.#tokens = blankUsage();\n this.#spoken = '';\n this.#heard = '';\n this.#turn += 1;\n this.#item = `g${String(this.#turn)}`;\n this.#pending = { chunks: [], cursor: 0 };\n }\n\n #onSetup(value: unknown): void {\n const done = record(value);\n const sessionId = done === undefined ? '' : text(done.sessionId);\n this.#push({ type: 'session_start', sessionId });\n if (!this.#historySent && this.#turns.length > 0) {\n this.#historySent = true;\n this.#wire.send(\n JSON.stringify({ clientContent: { turns: this.#turns, turnComplete: false } })\n );\n }\n this.#ready = true;\n const pending = this.#outgoing.splice(0);\n for (const event of pending) this.#wire.send(JSON.stringify(event));\n }\n\n #onResume(value: unknown): void {\n const update = record(value);\n if (update === undefined) return;\n const handle = text(update.newHandle);\n if (handle.length === 0 || update.resumable !== true) return;\n this.#handle = handle;\n this.#push({ type: 'resumption', handle });\n }\n\n #onGoAway(value: unknown): 'stop' | VoiceWire {\n const gone = record(value);\n const left = gone === undefined ? 0 : timeLeftMs(gone.timeLeft);\n this.#push({ type: 'go_away', timeLeftMs: left });\n const handle = this.#handle;\n if (handle === undefined) {\n this.#push({ type: 'error', error: 'session ended', fatal: true });\n return 'stop';\n }\n const next = this.#open(this.#target);\n this.#wire = next;\n this.#ready = false;\n next.send(JSON.stringify({ setup: this.#setup(handle) }));\n return next;\n }\n\n #queueTool(callId: string, response: JsonObject): void {\n const groupId = this.#toolGroupOf.get(callId) ?? '';\n const group = groupId.length === 0 ? undefined : this.#toolGroups.get(groupId);\n if (group === undefined) {\n this.#send({ toolResponse: { functionResponses: [response] } });\n return;\n }\n group.responses.push(response);\n group.pending.delete(callId);\n this.#toolGroupOf.delete(callId);\n if (group.pending.size > 0) return;\n this.#toolGroups.delete(groupId);\n this.#send({ toolResponse: { functionResponses: group.responses } });\n }\n\n #onToolCall(value: unknown): void {\n const call = record(value);\n if (call === undefined || !Array.isArray(call.functionCalls)) return;\n const ids: string[] = [];\n for (const entry of call.functionCalls) {\n const fn = record(entry);\n if (fn === undefined) continue;\n const id = text(fn.id);\n if (id.length > 0) ids.push(id);\n this.#push({\n type: 'tool_call',\n blocking: this.model.toolCalls === 'blocking',\n call: {\n type: 'toolCall',\n id,\n name: text(fn.name),\n arguments: argumentsOf(fn.args),\n },\n });\n }\n const groupId = ids.at(0);\n if (groupId === undefined) return;\n this.#toolGroups.set(groupId, { pending: new Set(ids), responses: [] });\n for (const id of ids) this.#toolGroupOf.set(id, groupId);\n }\n\n #onCancel(value: unknown): void {\n const cancellation = record(value);\n if (cancellation === undefined || !Array.isArray(cancellation.ids)) return;\n for (const id of cancellation.ids) {\n if (typeof id !== 'string') continue;\n this.#push({ type: 'tool_call_cancelled', callId: id });\n }\n }\n\n #onServer(value: unknown): void {\n const content = record(value);\n if (content === undefined) return;\n this.#modelTurn(content.modelTurn);\n this.#caption(content.outputTranscription, 'output');\n this.#caption(content.inputTranscription, 'input');\n if (content.interrupted === true) {\n this.#push({ type: 'input_speech_start' });\n this.#finish(true);\n return;\n }\n if (content.turnComplete === true) this.#finish(false);\n }\n\n #modelTurn(value: unknown): void {\n const turn = record(value);\n if (turn === undefined || !Array.isArray(turn.parts)) return;\n for (const part of turn.parts) {\n const entry = record(part);\n if (entry === undefined) continue;\n const inline = record(entry.inlineData);\n if (inline === undefined) continue;\n if (!text(inline.mimeType).startsWith('audio/')) continue;\n this.#audio(text(inline.data));\n }\n }\n\n #caption(value: unknown, kind: 'input' | 'output'): void {\n const caption = record(value);\n if (caption === undefined) return;\n if (kind === 'input') this.#input(text(caption.text), false);\n else this.#transcript(text(caption.text), false);\n }\n\n #onError(value: unknown): void {\n const detail = record(value);\n if (detail === undefined) {\n const message = text(value);\n this.#push({\n type: 'error',\n error: message.length > 0 ? message : 'realtime error',\n fatal: false,\n });\n return;\n }\n const code = text(detail.code);\n const message = text(detail.message);\n let fatal = false;\n if (code === 'MAX_DURATION' || code === 'SESSION_EXPIRED') fatal = true;\n this.#push({\n type: 'error',\n error: message.length > 0 ? message : 'realtime error',\n fatal,\n });\n }\n\n #apply(event: Record<string, unknown>): 'ok' | 'stop' | VoiceWire {\n if (event.usageMetadata !== undefined) this.#tokens = usageFrom(event.usageMetadata);\n if (event.inputTranscription !== undefined) this.#caption(event.inputTranscription, 'input');\n if (event.outputTranscription !== undefined) this.#caption(event.outputTranscription, 'output');\n if (event.setupComplete !== undefined) this.#onSetup(event.setupComplete);\n if (event.sessionResumptionUpdate !== undefined) this.#onResume(event.sessionResumptionUpdate);\n if (event.toolCall !== undefined) this.#onToolCall(event.toolCall);\n if (event.toolCallCancellation !== undefined) this.#onCancel(event.toolCallCancellation);\n if (event.serverContent !== undefined) this.#onServer(event.serverContent);\n if (event.error !== undefined) this.#onError(event.error);\n if (event.goAway !== undefined) return this.#onGoAway(event.goAway);\n return 'ok';\n }\n\n #ingest(raw: string): 'ok' | 'stop' | VoiceWire {\n let parsed: unknown;\n try {\n parsed = JSON.parse(raw);\n } catch {\n this.#push({ type: 'error', error: 'invalid realtime event', fatal: false });\n return 'ok';\n }\n const event = record(parsed);\n if (event === undefined) return 'ok';\n return this.#apply(event);\n }\n\n async #read(wire: VoiceWire): Promise<void> {\n let reconnect: VoiceWire | undefined;\n try {\n for await (const raw of wire.incoming) {\n const action = this.#ingest(raw);\n if (action === 'stop') return;\n if (action !== 'ok') {\n reconnect = action;\n return;\n }\n }\n } catch (error) {\n const message = error instanceof Error ? error.message : String(error);\n this.#push({ type: 'error', error: message, fatal: true });\n } finally {\n if (reconnect !== undefined) {\n wire.close();\n void this.#read(reconnect);\n } else {\n this.#closed = true;\n this.#end();\n }\n }\n }\n}\n\nexport async function connectGeminiSession(\n model: RealtimeModel<string>,\n context: RealtimeContext,\n options: RealtimeOptions | undefined,\n gemini: GeminiOptions\n): Promise<RealtimeSession> {\n if (options?.signal?.aborted === true) {\n throw new VoiceRequestError('aborted', 'session aborted');\n }\n const { secret, ephemeral } = credential(options);\n const socket = realtimeSocket(sessionUrl(model, secret, ephemeral), options, []);\n const target: VoiceWireTarget = {\n url: socket.url,\n protocols: socket.protocols,\n headers: headersFor(options),\n };\n const open =\n gemini.wire !== undefined\n ? gemini.wire\n : (wireTarget: VoiceWireTarget) => openVoiceSocket(wireTarget, gemini.socket);\n const session = new GeminiSession(model, context, gemini.vocabulary, open, target, open(target));\n const handle = options?.resumeHandle;\n session.start(handle);\n const signal = options?.signal;\n if (signal !== undefined) {\n signal.addEventListener('abort', () => void session.close(), { once: true });\n }\n return session;\n}\n\nexport function pcmFormat(sampleRate: number): AudioFormat {\n return { encoding: 'pcm16', sampleRate, channels: 1 };\n}\n"],
|
|
5
|
+
"mappings": "AAEA,SAAS,kBAAkB;AAC3B,SAAS,mBAAmB,yBAAyB;AACrD,SAAS,yBAAyB;AAElC;AAAA,EACE;AAAA,OAIK;AAOP,SAAS,gBAAgB,gBAAgB,sBAAsB;AAU/D,MAAM,aAAuC;AAAA,EAC3C,WAAW;AAAA,EACX,WAAW;AAAA,EACX,QAAQ;AACV;AAYA,SAAS,OAAO,OAAqD;AACnE,MAAI,UAAU,QAAQ,OAAO,UAAU,YAAY,MAAM,QAAQ,KAAK,EAAG,QAAO;AAChF,SAAO;AACT;AAEA,SAAS,KAAK,OAAwB;AACpC,SAAO,OAAO,UAAU,WAAW,QAAQ;AAC7C;AAEA,SAAS,WAAW,SAA8E;AAChG,QAAM,QAAQ,SAAS,OAAO;AAC9B,MAAI,OAAO,UAAU,YAAY,MAAM,SAAS,EAAG,QAAO,EAAE,QAAQ,OAAO,WAAW,KAAK;AAC3F,QAAM,SAAS,SAAS;AACxB,MAAI,OAAO,WAAW,YAAY,OAAO,SAAS,EAAG,QAAO,EAAE,QAAQ,QAAQ,WAAW,MAAM;AAC/F,QAAM,IAAI,kBAAkB,SAAS,6BAA6B;AACpE;AAEA,SAAS,WAAW,SAA8D;AAChF,QAAM,UAAkC,CAAC;AACzC,QAAM,QAAQ,SAAS;AACvB,MAAI,UAAU,OAAW,QAAO;AAChC,aAAW,CAAC,KAAK,KAAK,KAAK,OAAO,QAAQ,KAAK,GAAG;AAChD,QAAI,UAAU,KAAM,QAAO,QAAQ,GAAG;AAAA,QACjC,SAAQ,GAAG,IAAI;AAAA,EACtB;AACA,SAAO;AACT;AAEA,SAAS,WAAW,OAA8B,QAAgB,WAA4B;AAC5F,QAAM,MAAM,IAAI,IAAI,MAAM,OAAO;AACjC,MAAI,UAAW,KAAI,aAAa,IAAI,gBAAgB,MAAM;AAAA,MACrD,KAAI,aAAa,IAAI,OAAO,MAAM;AACvC,SAAO,IAAI,SAAS;AACtB;AAEA,SAAS,WAAW,OAAwB;AAC1C,MAAI,OAAO,UAAU,YAAY,OAAO,SAAS,KAAK,EAAG,QAAO,KAAK,MAAM,QAAQ,GAAI;AACvF,MAAI,OAAO,UAAU,UAAU;AAC7B,UAAM,UAAU,qBAAqB,KAAK,KAAK;AAC/C,QAAI,YAAY,KAAM,QAAO,KAAK,MAAM,OAAO,QAAQ,CAAC,CAAC,IAAI,GAAI;AACjE,UAAM,WAAW,OAAO,KAAK;AAC7B,QAAI,OAAO,SAAS,QAAQ,EAAG,QAAO,KAAK,MAAM,WAAW,GAAI;AAAA,EAClE;AACA,QAAM,WAAW,OAAO,KAAK;AAC7B,MAAI,aAAa,UAAa,OAAO,SAAS,YAAY,UAAU;AAClE,UAAM,QAAQ,OAAO,SAAS,UAAU,WAAW,SAAS,QAAQ;AACpE,WAAO,KAAK,MAAM,SAAS,UAAU,MAAO,QAAQ,GAAG;AAAA,EACzD;AACA,SAAO;AACT;AAEA,SAAS,MAAM,QAAiC,KAAqB;AACnE,QAAM,QAAQ,OAAO,GAAG;AACxB,SAAO,OAAO,UAAU,WAAW,QAAQ;AAC7C;AAEA,SAAS,UAAU,OAAuB;AACxC,QAAM,QAAQ,OAAO,KAAK;AAC1B,MAAI,UAAU,OAAW,QAAO,WAAW;AAC3C,QAAM,QAAQ,MAAM,OAAO,kBAAkB;AAC7C,QAAM,WAAW,MAAM,OAAO,oBAAoB;AAClD,QAAM,aAAa,MAAM,OAAO,sBAAsB;AACtD,QAAM,SAAS,WAAW,IAAI,WAAW;AACzC,SAAO;AAAA,IACL;AAAA,IACA;AAAA,IACA,WAAW;AAAA,IACX,YAAY;AAAA,IACZ,aAAa,QAAQ;AAAA,IACrB,MAAM,EAAE,OAAO,GAAG,QAAQ,GAAG,WAAW,GAAG,YAAY,GAAG,OAAO,EAAE;AAAA,EACrE;AACF;AAEA,SAAS,YAAY,OAA4B;AAC/C,MAAI,OAAO,UAAU,UAAU;AAC7B,QAAI;AACF,YAAM,SAAkB,KAAK,MAAM,KAAK;AACxC,YAAMA,UAAS,OAAO,MAAM;AAC5B,UAAIA,YAAW,OAAW,QAAO,CAAC;AAClC,aAAOA;AAAA,IACT,QAAQ;AACN,aAAO,CAAC;AAAA,IACV;AAAA,EACF;AACA,QAAM,SAAS,OAAO,KAAK;AAC3B,MAAI,WAAW,OAAW,QAAO,CAAC;AAClC,SAAO;AACT;AAEA,SAAS,WAAW,QAAmC;AACrD,QAAM,QAAkB,CAAC;AACzB,aAAW,QAAQ,OAAO,SAAS;AACjC,QAAI,KAAK,SAAS,OAAQ,OAAM,KAAK,KAAK,IAAI;AAAA,EAChD;AACA,SAAO,MAAM,KAAK,IAAI;AACxB;AAEA,SAAS,aAAa,UAA+C;AACnE,MAAI,aAAa,OAAW,QAAO,CAAC;AACpC,QAAM,QAAsB,CAAC;AAC7B,aAAW,WAAW,UAAU;AAC9B,QAAI,QAAQ,SAAS,UAAU,QAAQ,SAAS,YAAa;AAC7D,UAAM,QAAsB,CAAC;AAC7B,eAAW,QAAQ,QAAQ,SAAS;AAClC,UAAI,OAAO,SAAS,UAAU;AAC5B,cAAM,KAAK,EAAE,MAAM,KAAK,CAAC;AACzB;AAAA,MACF;AACA,UAAI,KAAK,SAAS,OAAQ,OAAM,KAAK,EAAE,MAAM,KAAK,KAAK,CAAC;AAAA,IAC1D;AACA,QAAI,MAAM,WAAW,EAAG;AACxB,UAAM,KAAK,EAAE,MAAM,QAAQ,SAAS,cAAc,UAAU,QAAQ,MAAM,CAAC;AAAA,EAC7E;AACA,SAAO;AACT;AAEA,SAAS,YAAY,MAAY,OAA0C;AACzE,QAAM,KAAiB;AAAA,IACrB,MAAM,KAAK;AAAA,IACX,aAAa,KAAK;AAAA,IAClB,YAAY,KAAK;AAAA,EACnB;AACA,MAAI,MAAM,OAAO,qCAAsC,IAAG,WAAW;AACrE,MAAI,MAAM,cAAc,WAAY,IAAG,WAAW;AAClD,SAAO;AACT;AAEA,SAAS,cAAc,OAA8B,YAA8C;AACjG,MAAI,MAAM,OAAO,kBAAmB,QAAO,CAAC;AAC5C,MAAI,eAAe,OAAW,QAAO,CAAC;AACtC,SAAO,EAAE,kBAAkB,WAAW;AACxC;AAEA,SAAS,UACP,OACA,SACA,QACA,YACA,OACY;AACZ,QAAM,QAAoB;AAAA,IACxB,OAAO,UAAU,MAAM,EAAE;AAAA,IACzB,kBAAkB,EAAE,oBAAoB,CAAC,OAAO,EAAE;AAAA,IAClD,mBAAmB,EAAE,OAAO,CAAC,EAAE,MAAM,QAAQ,aAAa,CAAC,EAAE;AAAA,IAC7D,cAAc;AAAA,MACZ,aAAa,EAAE,qBAAqB,EAAE,WAAW,QAAQ,MAAM,EAAE;AAAA,IACnE;AAAA,IACA,yBAAyB,cAAc,OAAO,UAAU;AAAA,IACxD,0BAA0B,CAAC;AAAA,IAC3B,0BAA0B;AAAA,MACxB,eAAe;AAAA,MACf,eAAe,EAAE,cAAc,IAAO;AAAA,IACxC;AAAA,IACA,mBAAmB,WAAW,SAAY,CAAC,IAAI,EAAE,OAAO;AAAA,EAC1D;AACA,MAAI,QAAQ,MAAM,SAAS,GAAG;AAC5B,UAAM,QAAQ,CAAC,EAAE,sBAAsB,QAAQ,MAAM,IAAI,CAAC,SAAS,YAAY,MAAM,KAAK,CAAC,EAAE,CAAC;AAAA,EAChG;AACA,MAAI,MAAM,SAAS,EAAG,OAAM,gBAAgB,EAAE,+BAA+B,KAAK;AAClF,SAAO;AACT;AAEA,SAAS,QAAQ,QAAgB,OAAe,KAAuB;AACrE,SAAO,EAAE,QAAQ,OAAO,IAAI;AAC9B;AAEA,SAAS,UACP,SACA,QACA,QACA,KACiB;AACjB,QAAM,SAAS,QAAQ;AACvB,UAAQ,SAAS,CAAC;AAClB,MAAI,OAAO,WAAW,EAAG,QAAO,CAAC;AACjC,QAAM,QAAQ,QAAQ;AACtB,MAAI,QAAQ,OAAW,SAAQ,SAAS;AACxC,QAAM,QAAQ,QAAQ,SAAY,IAAI,MAAM;AAC5C,QAAM,QAAQ,OAAO,OAAO,CAAC,KAAK,UAAU,MAAM,MAAM,KAAK,QAAQ,CAAC;AACtE,MAAI,OAAO;AACX,MAAI,SAAS;AACb,SAAO,OAAO,IAAI,CAAC,OAAO,UAAU;AAClC,UAAM,QAAuB;AAAA,MAC3B,MAAM;AAAA,MACN;AAAA,MACA,OAAO,EAAE,MAAM,MAAM,MAAM,QAAQ,WAAW,MAAM,UAAU;AAAA,IAChE;AACA,QAAI,QAAQ,OAAW,QAAO;AAC9B,UAAM,OAAO,UAAU,OAAO,SAAS;AACvC,UAAM,OAAO,OAAO,MAAM,QAAQ,KAAK,OAAQ,OAAO,MAAM,KAAK,UAAU,QAAS,KAAK;AACzF,YAAQ,MAAM,KAAK;AACnB,UAAM,OAAO,QAAQ,QAAQ,QAAQ,IAAI;AACzC,aAAS;AACT,WAAO;AAAA,EACT,CAAC;AACH;AAEA,MAAM,cAAyC;AAAA,EACpC;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA,UAAU,IAAI,eAAe;AAAA,EAC7B,SAA0B,CAAC;AAAA,EAC3B,WAAyD,CAAC;AAAA,EAC1D,YAA0B,CAAC;AAAA,EAC3B;AAAA,EACT;AAAA,EACA,WAAyB,EAAE,QAAQ,CAAC,GAAG,QAAQ,EAAE;AAAA,EACjD,QAAQ;AAAA,EACR,QAAQ;AAAA,EACR,SAAS;AAAA,EACT,UAAU;AAAA,EACV;AAAA,EACA,UAAiB,WAAW;AAAA,EAC5B,SAAS;AAAA,EACT,eAAe;AAAA,EACf,UAAU;AAAA,EACV,SAAS;AAAA,EACT;AAAA,EACS,cAAc,oBAAI,IAA+D;AAAA,EACjF,eAAe,oBAAI,IAAoB;AAAA,EAEhD,YACE,OACA,SACA,YACA,MACA,QACA,MACA;AACA,SAAK,QAAQ;AACb,SAAK,WAAW;AAChB,SAAK,cAAc;AACnB,SAAK,SAAS,aAAa,QAAQ,OAAO;AAC1C,SAAK,QAAQ;AACb,SAAK,UAAU;AACf,SAAK,QAAQ;AACb,SAAK,UAAU,MAAM;AAAA,EACvB;AAAA,EAEA,MAAM,QAAkC;AACtC,QAAI,WAAW,UAAa,OAAO,SAAS,EAAG,MAAK,UAAU;AAC9D,SAAK,MAAM,KAAK,KAAK,UAAU,EAAE,OAAO,KAAK,OAAO,MAAM,EAAE,CAAC,CAAC;AAC9D,SAAK,KAAK,MAAM,KAAK,KAAK;AAAA,EAC5B;AAAA,EAEA,OAAO,QAAwC;AAC7C,WAAO,UAAU,KAAK,OAAO,KAAK,UAAU,QAAQ,KAAK,aAAa,KAAK,MAAM;AAAA,EACnF;AAAA,EAEA,cAAoB;AAClB,QAAI,KAAK,QAAS,OAAM,IAAI,kBAAkB,SAAS,mBAAmB;AAAA,EAC5E;AAAA,EAEA,MAAM,OAAyB;AAC7B,QAAI,CAAC,KAAK,QAAQ;AAChB,WAAK,UAAU,KAAK,KAAK;AACzB;AAAA,IACF;AACA,SAAK,MAAM,KAAK,KAAK,UAAU,KAAK,CAAC;AAAA,EACvC;AAAA,EAEA,MAAM,OAA4B;AAChC,QAAI,KAAK,QAAS;AAClB,SAAK,QAAQ,QAAQ,KAAK;AAC1B,UAAM,SAAS,KAAK,SAAS,MAAM;AACnC,QAAI,WAAW,OAAW,QAAO,EAAE,OAAO,OAAO,MAAM,MAAM,CAAC;AAAA,QACzD,MAAK,OAAO,KAAK,KAAK;AAAA,EAC7B;AAAA,EAEA,OAAa;AACX,QAAI,KAAK,OAAQ;AACjB,SAAK,SAAS;AACd,WAAO,KAAK,SAAS,SAAS,EAAG,MAAK,SAAS,MAAM,IAAI,EAAE,MAAM,KAAK,CAAC;AAAA,EACzE;AAAA,EAEA,YAAY,OAAyB;AACnC,SAAK,YAAY;AACjB,SAAK,QAAQ,SAAS,KAAK;AAC3B,SAAK,MAAM;AAAA,MACT,eAAe;AAAA,QACb,OAAO;AAAA,UACL,MAAM,kBAAkB,MAAM,IAAI;AAAA,UAClC,UAAU,kBAAkB,OAAO,MAAM,OAAO,UAAU,CAAC;AAAA,QAC7D;AAAA,MACF;AAAA,IACF,CAAC;AAAA,EACH;AAAA,EAEA,iBAAiB,MAAwB;AACvC,SAAK,YAAY;AACjB,QAAI,KAAK,MAAM,UAAU,QAAW;AAClC,YAAM,IAAI,kBAAkB,SAAS,sBAAsB;AAAA,IAC7D;AACA,SAAK,MAAM;AAAA,MACT,eAAe,EAAE,OAAO,EAAE,MAAM,kBAAkB,IAAI,GAAG,UAAU,aAAa,EAAE;AAAA,IACpF,CAAC;AAAA,EACH;AAAA,EAEA,SAAS,OAAqB;AAC5B,SAAK,YAAY;AACjB,SAAK,QAAQ,QAAQ;AACrB,SAAK,MAAM,KAAK,QAAQ,OAAO,IAAI,CAAC;AAAA,EACtC;AAAA,EAEA,OAAO,OAAe,UAA0B;AAC9C,SAAK,YAAY;AACjB,SAAK,QAAQ,OAAO,KAAK,MAAM,UAAU,UAAU,MAAM;AACvD,WAAK,YAAY;AACjB,WAAK,QAAQ,QAAQ;AACrB,UAAI,aAAa,SAAU,MAAK,MAAM,KAAK,QAAQ,OAAO,KAAK,CAAC;AAAA,eACvD,aAAa,YAAa,MAAK,MAAM,EAAE,eAAe,EAAE,MAAM,MAAM,EAAE,CAAC;AAAA,UAC3E,MAAK,MAAM,KAAK,QAAQ,OAAO,IAAI,CAAC;AAAA,IAC3C,CAAC;AAAA,EACH;AAAA,EAEA,QAAQ,QAAgB,QAA2B,UAA2B;AAC5E,SAAK,YAAY;AACjB,UAAM,OAAO,MAAM;AACjB,YAAM,OAAO,aAAa,SAAY,cAAc;AACpD,WAAK,WAAW,QAAQ;AAAA,QACtB,IAAI;AAAA,QACJ,MAAM,OAAO;AAAA,QACb,YAAY,WAAW,IAAI;AAAA,QAC3B,UAAU;AAAA,UACR,QAAQ,OAAO,UAAU,UAAU;AAAA,UACnC,WAAW;AAAA,UACX,SAAS,WAAW,MAAM;AAAA,QAC5B;AAAA,MACF,CAAC;AAAA,IACH;AACA,QAAI,aAAa,QAAW;AAC1B,WAAK;AACL;AAAA,IACF;AACA,SAAK,QAAQ,OAAO,KAAK,MAAM,UAAU,UAAU,MAAM;AACvD,WAAK,YAAY;AACjB,WAAK;AAAA,IACP,CAAC;AAAA,EACH;AAAA,EAEA,UAAU,OAAmD;AAC3D,SAAK,YAAY;AACjB,mBAAe,KAAK,MAAM,YAAY,KAAK;AAC3C,SAAK,QAAQ,OAAO;AAAA,EACtB;AAAA,EAEA,OAAO,OAAiF;AACtF,SAAK,YAAY;AACjB,QAAI,MAAM,UAAU,UAAa,CAAC,KAAK,MAAM,gBAAgB;AAC3D,YAAM,IAAI,kBAAkB,SAAS,iCAAiC;AAAA,IACxE;AAAA,EACF;AAAA,EAEA,MAAM,QAAmC;AACvC,QAAI,KAAK,YAAY,OAAW,QAAO,EAAE,OAAO,KAAK,QAAQ;AAC7D,SAAK,UAAU;AACf,SAAK,QAAQ,OAAO;AACpB,SAAK,MAAM,MAAM;AACjB,SAAK,KAAK;AACV,SAAK,UAAU,KAAK,QAAQ,MAAM,KAAK,MAAM,SAAS;AACtD,WAAO,EAAE,OAAO,KAAK,QAAQ;AAAA,EAC/B;AAAA,EAEA,QAAQ,OAAe,cAAmC;AACxD,WAAO;AAAA,MACL,eAAe;AAAA,QACb,OAAO,CAAC,EAAE,MAAM,QAAQ,OAAO,CAAC,EAAE,MAAM,MAAM,CAAC,EAAE,CAAC;AAAA,QAClD;AAAA,MACF;AAAA,IACF;AAAA,EACF;AAAA,EAEA,QAAQ,OAAO,aAAa,IAAkC;AAC5D,WAAO,MAAM;AACX,YAAM,SAAS,KAAK,OAAO,MAAM;AACjC,UAAI,WAAW,QAAW;AACxB,cAAM;AACN;AAAA,MACF;AACA,UAAI,KAAK,QAAS;AAClB,YAAM,OAAO,MAAM,IAAI,QAA6B,CAAC,YAAY;AAC/D,aAAK,SAAS,KAAK,OAAO;AAAA,MAC5B,CAAC;AACD,UAAI,KAAK,SAAS,QAAQ,KAAK,UAAU,OAAW;AACpD,YAAM,KAAK;AAAA,IACb;AAAA,EACF;AAAA,EAEA,OAAO,SAAuB;AAC5B,QAAI,QAAQ,WAAW,EAAG;AAC1B,SAAK,SAAS,OAAO,KAAK;AAAA,MACxB,MAAM,kBAAkB,OAAO;AAAA,MAC/B,QAAQ,KAAK;AAAA,MACb,WAAW;AAAA,IACb,CAAC;AAAA,EACH;AAAA,EAEA,YAAY,OAAe,OAAsB;AAC/C,UAAM,MAAM,KAAK,SAAS,SAAS,MAAM;AACzC,UAAM,SAAS,MAAM,WAAW,IAAI,SAAY;AAChD,eAAW,SAAS,UAAU,KAAK,UAAU,KAAK,OAAO,KAAK,SAAS,MAAM,GAAG;AAC9E,WAAK,MAAM,KAAK;AAAA,IAClB;AACA,SAAK,WAAW;AAChB,SAAK,MAAM,EAAE,MAAM,qBAAqB,QAAQ,KAAK,OAAO,MAAM,OAAO,MAAM,CAAC;AAAA,EAClF;AAAA,EAEA,OAAO,YAAoB,OAAsB;AAC/C,QAAI,WAAW,SAAS,EAAG,MAAK,SAAS;AACzC,SAAK,MAAM,EAAE,MAAM,oBAAoB,QAAQ,KAAK,OAAO,MAAM,KAAK,QAAQ,MAAM,CAAC;AAAA,EACvF;AAAA,EAEA,QAAQ,aAA4B;AAClC,eAAW,SAAS,UAAU,KAAK,UAAU,KAAK,OAAO,KAAK,SAAS,MAAS,GAAG;AACjF,WAAK,MAAM,KAAK;AAAA,IAClB;AACA,QAAI,KAAK,QAAQ,SAAS,GAAG;AAC3B,WAAK,MAAM;AAAA,QACT,MAAM;AAAA,QACN,QAAQ,KAAK;AAAA,QACb,MAAM,KAAK;AAAA,QACX,OAAO;AAAA,MACT,CAAC;AAAA,IACH;AACA,QAAI,KAAK,OAAO,SAAS,GAAG;AAC1B,WAAK,MAAM,EAAE,MAAM,oBAAoB,QAAQ,KAAK,OAAO,MAAM,KAAK,QAAQ,OAAO,KAAK,CAAC;AAAA,IAC7F;AACA,SAAK,MAAM;AAAA,MACT,MAAM;AAAA,MACN,QAAQ,KAAK;AAAA,MACb;AAAA,MACA,OAAO,KAAK;AAAA,IACd,CAAC;AACD,SAAK,UAAU,WAAW;AAC1B,SAAK,UAAU;AACf,SAAK,SAAS;AACd,SAAK,SAAS;AACd,SAAK,QAAQ,IAAI,OAAO,KAAK,KAAK,CAAC;AACnC,SAAK,WAAW,EAAE,QAAQ,CAAC,GAAG,QAAQ,EAAE;AAAA,EAC1C;AAAA,EAEA,SAAS,OAAsB;AAC7B,UAAM,OAAO,OAAO,KAAK;AACzB,UAAM,YAAY,SAAS,SAAY,KAAK,KAAK,KAAK,SAAS;AAC/D,SAAK,MAAM,EAAE,MAAM,iBAAiB,UAAU,CAAC;AAC/C,QAAI,CAAC,KAAK,gBAAgB,KAAK,OAAO,SAAS,GAAG;AAChD,WAAK,eAAe;AACpB,WAAK,MAAM;AAAA,QACT,KAAK,UAAU,EAAE,eAAe,EAAE,OAAO,KAAK,QAAQ,cAAc,MAAM,EAAE,CAAC;AAAA,MAC/E;AAAA,IACF;AACA,SAAK,SAAS;AACd,UAAM,UAAU,KAAK,UAAU,OAAO,CAAC;AACvC,eAAW,SAAS,QAAS,MAAK,MAAM,KAAK,KAAK,UAAU,KAAK,CAAC;AAAA,EACpE;AAAA,EAEA,UAAU,OAAsB;AAC9B,UAAM,SAAS,OAAO,KAAK;AAC3B,QAAI,WAAW,OAAW;AAC1B,UAAM,SAAS,KAAK,OAAO,SAAS;AACpC,QAAI,OAAO,WAAW,KAAK,OAAO,cAAc,KAAM;AACtD,SAAK,UAAU;AACf,SAAK,MAAM,EAAE,MAAM,cAAc,OAAO,CAAC;AAAA,EAC3C;AAAA,EAEA,UAAU,OAAoC;AAC5C,UAAM,OAAO,OAAO,KAAK;AACzB,UAAM,OAAO,SAAS,SAAY,IAAI,WAAW,KAAK,QAAQ;AAC9D,SAAK,MAAM,EAAE,MAAM,WAAW,YAAY,KAAK,CAAC;AAChD,UAAM,SAAS,KAAK;AACpB,QAAI,WAAW,QAAW;AACxB,WAAK,MAAM,EAAE,MAAM,SAAS,OAAO,iBAAiB,OAAO,KAAK,CAAC;AACjE,aAAO;AAAA,IACT;AACA,UAAM,OAAO,KAAK,MAAM,KAAK,OAAO;AACpC,SAAK,QAAQ;AACb,SAAK,SAAS;AACd,SAAK,KAAK,KAAK,UAAU,EAAE,OAAO,KAAK,OAAO,MAAM,EAAE,CAAC,CAAC;AACxD,WAAO;AAAA,EACT;AAAA,EAEA,WAAW,QAAgB,UAA4B;AACrD,UAAM,UAAU,KAAK,aAAa,IAAI,MAAM,KAAK;AACjD,UAAM,QAAQ,QAAQ,WAAW,IAAI,SAAY,KAAK,YAAY,IAAI,OAAO;AAC7E,QAAI,UAAU,QAAW;AACvB,WAAK,MAAM,EAAE,cAAc,EAAE,mBAAmB,CAAC,QAAQ,EAAE,EAAE,CAAC;AAC9D;AAAA,IACF;AACA,UAAM,UAAU,KAAK,QAAQ;AAC7B,UAAM,QAAQ,OAAO,MAAM;AAC3B,SAAK,aAAa,OAAO,MAAM;AAC/B,QAAI,MAAM,QAAQ,OAAO,EAAG;AAC5B,SAAK,YAAY,OAAO,OAAO;AAC/B,SAAK,MAAM,EAAE,cAAc,EAAE,mBAAmB,MAAM,UAAU,EAAE,CAAC;AAAA,EACrE;AAAA,EAEA,YAAY,OAAsB;AAChC,UAAM,OAAO,OAAO,KAAK;AACzB,QAAI,SAAS,UAAa,CAAC,MAAM,QAAQ,KAAK,aAAa,EAAG;AAC9D,UAAM,MAAgB,CAAC;AACvB,eAAW,SAAS,KAAK,eAAe;AACtC,YAAM,KAAK,OAAO,KAAK;AACvB,UAAI,OAAO,OAAW;AACtB,YAAM,KAAK,KAAK,GAAG,EAAE;AACrB,UAAI,GAAG,SAAS,EAAG,KAAI,KAAK,EAAE;AAC9B,WAAK,MAAM;AAAA,QACT,MAAM;AAAA,QACN,UAAU,KAAK,MAAM,cAAc;AAAA,QACnC,MAAM;AAAA,UACJ,MAAM;AAAA,UACN;AAAA,UACA,MAAM,KAAK,GAAG,IAAI;AAAA,UAClB,WAAW,YAAY,GAAG,IAAI;AAAA,QAChC;AAAA,MACF,CAAC;AAAA,IACH;AACA,UAAM,UAAU,IAAI,GAAG,CAAC;AACxB,QAAI,YAAY,OAAW;AAC3B,SAAK,YAAY,IAAI,SAAS,EAAE,SAAS,IAAI,IAAI,GAAG,GAAG,WAAW,CAAC,EAAE,CAAC;AACtE,eAAW,MAAM,IAAK,MAAK,aAAa,IAAI,IAAI,OAAO;AAAA,EACzD;AAAA,EAEA,UAAU,OAAsB;AAC9B,UAAM,eAAe,OAAO,KAAK;AACjC,QAAI,iBAAiB,UAAa,CAAC,MAAM,QAAQ,aAAa,GAAG,EAAG;AACpE,eAAW,MAAM,aAAa,KAAK;AACjC,UAAI,OAAO,OAAO,SAAU;AAC5B,WAAK,MAAM,EAAE,MAAM,uBAAuB,QAAQ,GAAG,CAAC;AAAA,IACxD;AAAA,EACF;AAAA,EAEA,UAAU,OAAsB;AAC9B,UAAM,UAAU,OAAO,KAAK;AAC5B,QAAI,YAAY,OAAW;AAC3B,SAAK,WAAW,QAAQ,SAAS;AACjC,SAAK,SAAS,QAAQ,qBAAqB,QAAQ;AACnD,SAAK,SAAS,QAAQ,oBAAoB,OAAO;AACjD,QAAI,QAAQ,gBAAgB,MAAM;AAChC,WAAK,MAAM,EAAE,MAAM,qBAAqB,CAAC;AACzC,WAAK,QAAQ,IAAI;AACjB;AAAA,IACF;AACA,QAAI,QAAQ,iBAAiB,KAAM,MAAK,QAAQ,KAAK;AAAA,EACvD;AAAA,EAEA,WAAW,OAAsB;AAC/B,UAAM,OAAO,OAAO,KAAK;AACzB,QAAI,SAAS,UAAa,CAAC,MAAM,QAAQ,KAAK,KAAK,EAAG;AACtD,eAAW,QAAQ,KAAK,OAAO;AAC7B,YAAM,QAAQ,OAAO,IAAI;AACzB,UAAI,UAAU,OAAW;AACzB,YAAM,SAAS,OAAO,MAAM,UAAU;AACtC,UAAI,WAAW,OAAW;AAC1B,UAAI,CAAC,KAAK,OAAO,QAAQ,EAAE,WAAW,QAAQ,EAAG;AACjD,WAAK,OAAO,KAAK,OAAO,IAAI,CAAC;AAAA,IAC/B;AAAA,EACF;AAAA,EAEA,SAAS,OAAgB,MAAgC;AACvD,UAAM,UAAU,OAAO,KAAK;AAC5B,QAAI,YAAY,OAAW;AAC3B,QAAI,SAAS,QAAS,MAAK,OAAO,KAAK,QAAQ,IAAI,GAAG,KAAK;AAAA,QACtD,MAAK,YAAY,KAAK,QAAQ,IAAI,GAAG,KAAK;AAAA,EACjD;AAAA,EAEA,SAAS,OAAsB;AAC7B,UAAM,SAAS,OAAO,KAAK;AAC3B,QAAI,WAAW,QAAW;AACxB,YAAMC,WAAU,KAAK,KAAK;AAC1B,WAAK,MAAM;AAAA,QACT,MAAM;AAAA,QACN,OAAOA,SAAQ,SAAS,IAAIA,WAAU;AAAA,QACtC,OAAO;AAAA,MACT,CAAC;AACD;AAAA,IACF;AACA,UAAM,OAAO,KAAK,OAAO,IAAI;AAC7B,UAAM,UAAU,KAAK,OAAO,OAAO;AACnC,QAAI,QAAQ;AACZ,QAAI,SAAS,kBAAkB,SAAS,kBAAmB,SAAQ;AACnE,SAAK,MAAM;AAAA,MACT,MAAM;AAAA,MACN,OAAO,QAAQ,SAAS,IAAI,UAAU;AAAA,MACtC;AAAA,IACF,CAAC;AAAA,EACH;AAAA,EAEA,OAAO,OAA2D;AAChE,QAAI,MAAM,kBAAkB,OAAW,MAAK,UAAU,UAAU,MAAM,aAAa;AACnF,QAAI,MAAM,uBAAuB,OAAW,MAAK,SAAS,MAAM,oBAAoB,OAAO;AAC3F,QAAI,MAAM,wBAAwB,OAAW,MAAK,SAAS,MAAM,qBAAqB,QAAQ;AAC9F,QAAI,MAAM,kBAAkB,OAAW,MAAK,SAAS,MAAM,aAAa;AACxE,QAAI,MAAM,4BAA4B,OAAW,MAAK,UAAU,MAAM,uBAAuB;AAC7F,QAAI,MAAM,aAAa,OAAW,MAAK,YAAY,MAAM,QAAQ;AACjE,QAAI,MAAM,yBAAyB,OAAW,MAAK,UAAU,MAAM,oBAAoB;AACvF,QAAI,MAAM,kBAAkB,OAAW,MAAK,UAAU,MAAM,aAAa;AACzE,QAAI,MAAM,UAAU,OAAW,MAAK,SAAS,MAAM,KAAK;AACxD,QAAI,MAAM,WAAW,OAAW,QAAO,KAAK,UAAU,MAAM,MAAM;AAClE,WAAO;AAAA,EACT;AAAA,EAEA,QAAQ,KAAwC;AAC9C,QAAI;AACJ,QAAI;AACF,eAAS,KAAK,MAAM,GAAG;AAAA,IACzB,QAAQ;AACN,WAAK,MAAM,EAAE,MAAM,SAAS,OAAO,0BAA0B,OAAO,MAAM,CAAC;AAC3E,aAAO;AAAA,IACT;AACA,UAAM,QAAQ,OAAO,MAAM;AAC3B,QAAI,UAAU,OAAW,QAAO;AAChC,WAAO,KAAK,OAAO,KAAK;AAAA,EAC1B;AAAA,EAEA,MAAM,MAAM,MAAgC;AAC1C,QAAI;AACJ,QAAI;AACF,uBAAiB,OAAO,KAAK,UAAU;AACrC,cAAM,SAAS,KAAK,QAAQ,GAAG;AAC/B,YAAI,WAAW,OAAQ;AACvB,YAAI,WAAW,MAAM;AACnB,sBAAY;AACZ;AAAA,QACF;AAAA,MACF;AAAA,IACF,SAAS,OAAO;AACd,YAAM,UAAU,iBAAiB,QAAQ,MAAM,UAAU,OAAO,KAAK;AACrE,WAAK,MAAM,EAAE,MAAM,SAAS,OAAO,SAAS,OAAO,KAAK,CAAC;AAAA,IAC3D,UAAE;AACA,UAAI,cAAc,QAAW;AAC3B,aAAK,MAAM;AACX,aAAK,KAAK,MAAM,SAAS;AAAA,MAC3B,OAAO;AACL,aAAK,UAAU;AACf,aAAK,KAAK;AAAA,MACZ;AAAA,IACF;AAAA,EACF;AACF;AAEA,eAAsB,qBACpB,OACA,SACA,SACA,QAC0B;AAC1B,MAAI,SAAS,QAAQ,YAAY,MAAM;AACrC,UAAM,IAAI,kBAAkB,WAAW,iBAAiB;AAAA,EAC1D;AACA,QAAM,EAAE,QAAQ,UAAU,IAAI,WAAW,OAAO;AAChD,QAAM,SAAS,eAAe,WAAW,OAAO,QAAQ,SAAS,GAAG,SAAS,CAAC,CAAC;AAC/E,QAAM,SAA0B;AAAA,IAC9B,KAAK,OAAO;AAAA,IACZ,WAAW,OAAO;AAAA,IAClB,SAAS,WAAW,OAAO;AAAA,EAC7B;AACA,QAAM,OACJ,OAAO,SAAS,SACZ,OAAO,OACP,CAAC,eAAgC,gBAAgB,YAAY,OAAO,MAAM;AAChF,QAAM,UAAU,IAAI,cAAc,OAAO,SAAS,OAAO,YAAY,MAAM,QAAQ,KAAK,MAAM,CAAC;AAC/F,QAAM,SAAS,SAAS;AACxB,UAAQ,MAAM,MAAM;AACpB,QAAM,SAAS,SAAS;AACxB,MAAI,WAAW,QAAW;AACxB,WAAO,iBAAiB,SAAS,MAAM,KAAK,QAAQ,MAAM,GAAG,EAAE,MAAM,KAAK,CAAC;AAAA,EAC7E;AACA,SAAO;AACT;AAEO,SAAS,UAAU,YAAiC;AACzD,SAAO,EAAE,UAAU,SAAS,YAAY,UAAU,EAAE;AACtD;",
|
|
6
6
|
"names": ["object", "message"]
|
|
7
7
|
}
|
package/dist/index.js
CHANGED
|
@@ -7,6 +7,15 @@ import {
|
|
|
7
7
|
decodeAudioBase64,
|
|
8
8
|
encodeAudioBase64
|
|
9
9
|
} from "./convert.js";
|
|
10
|
+
import {
|
|
11
|
+
assertNodeEcho,
|
|
12
|
+
calibrateEcho,
|
|
13
|
+
chooseCapture,
|
|
14
|
+
echoRuntime,
|
|
15
|
+
measureEcho,
|
|
16
|
+
micOpen,
|
|
17
|
+
openEchoMic
|
|
18
|
+
} from "./echo.js";
|
|
10
19
|
import { VoiceRequestError } from "./errors.js";
|
|
11
20
|
import {
|
|
12
21
|
getVoiceModel,
|
|
@@ -38,20 +47,27 @@ export {
|
|
|
38
47
|
VoiceRequestError,
|
|
39
48
|
addAudioCounters,
|
|
40
49
|
askAgent,
|
|
50
|
+
assertNodeEcho,
|
|
41
51
|
bindThinker,
|
|
42
52
|
blankUsage,
|
|
53
|
+
calibrateEcho,
|
|
43
54
|
captionsFromHistory,
|
|
55
|
+
chooseCapture,
|
|
44
56
|
connectRealtime,
|
|
45
57
|
convertAudio,
|
|
46
58
|
convertAudioSource,
|
|
47
59
|
decodeAudioBase64,
|
|
60
|
+
echoRuntime,
|
|
48
61
|
encodeAudioBase64,
|
|
49
62
|
energyVad,
|
|
50
63
|
followBed,
|
|
51
64
|
getVoiceModel,
|
|
52
65
|
listVoiceModels,
|
|
53
66
|
listen,
|
|
67
|
+
measureEcho,
|
|
68
|
+
micOpen,
|
|
54
69
|
mintRealtimeToken,
|
|
70
|
+
openEchoMic,
|
|
55
71
|
planDelivery,
|
|
56
72
|
planTruncation,
|
|
57
73
|
priceAudio,
|
package/dist/index.js.map
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"version": 3,
|
|
3
3
|
"sources": ["../src/index.ts"],
|
|
4
|
-
"sourcesContent": ["export type {\n AudioChunk,\n AudioEncoding,\n AudioFormat,\n AudioSink,\n AudioSource,\n AudioTag,\n CharAlignment,\n TextSpan,\n} from './audio.ts';\nexport { resolveVoiceAuth } from './auth.ts';\nexport { blankUsage } from './blank-usage.ts';\nexport type { CaptionHistoryEntry } from './captions.ts';\nexport { captionsFromHistory } from './captions.ts';\nexport {\n convertAudio,\n convertAudioSource,\n decodeAudioBase64,\n encodeAudioBase64,\n} from './convert.ts';\nexport { VoiceRequestError } from './errors.ts';\nexport type {\n Delivery,\n DeliverySupport,\n KnownRealtimeApi,\n KnownSpeechApi,\n KnownTranscriptionApi,\n RealtimeApi,\n RealtimeModel,\n SpeechApi,\n SpeechModel,\n TranscriptionApi,\n TranscriptionModel,\n VoiceInfo,\n VoiceModel,\n VoiceModelType,\n} from './models.ts';\nexport {\n getVoiceModel,\n listVoiceModels,\n registerVoiceModel,\n} from './models.ts';\nexport type {\n DeliveryPlan,\n RealtimeApiProvider,\n RealtimeContext,\n RealtimeEvent,\n RealtimeOptions,\n RealtimeSession,\n RealtimeSocket,\n RealtimeToken,\n TruncationPlan,\n} from './realtime.ts';\nexport { planDelivery, planTruncation, RealtimeLedger, realtimeSocket } from './realtime.ts';\nexport {\n connectRealtime,\n listen,\n registerRealtimeApiProvider,\n registerSpeechApiProvider,\n registerTranscriptionApiProvider,\n speak,\n transcribe,\n} from './registry.ts';\nexport type { AudioOut, BedChannel, Cue, CueSource, Soundscape } from './soundscape.ts';\nexport { followBed } from './soundscape.ts';\nexport type { SpeechApiProvider, SpeechEvent, SpeechInput } from './speech.ts';\nexport { SpeechEventStream } from './speech.ts';\nexport type { VoiceAgentEvent, VoiceMessageChange } from './spoken.ts';\nexport type {\n BindThinkerOptions,\n BoundThinker,\n Thinker,\n ThinkerCaption,\n ThinkerPhase,\n ThinkerUpdate,\n} from './talker.ts';\nexport { askAgent, bindThinker, talkerTools } from './talker.ts';\nexport type { RealtimeConstraints } from './tokens.ts';\nexport { mintRealtimeToken } from './tokens.ts';\nexport type {\n AudioInput,\n ListenOptions,\n Transcript,\n TranscriptionApiProvider,\n TranscriptionEvent,\n TranscriptSegment,\n} from './transcription.ts';\nexport { TranscriptionEventStream } from './transcription.ts';\nexport type {\n BargeIn,\n CaptionEvent,\n EchoRecommendation,\n VoiceConversation,\n VoiceCut,\n VoiceModels,\n VoicePhase,\n VoiceSession,\n VoiceTurnMeta,\n VoiceTurnOptions,\n} from './turns.ts';\nexport { voiceTurns } from './turns.ts';\nexport type {\n AudioCost,\n AudioCounters,\n AudioUsage,\n RealtimeAudioCost,\n SpeechAudioCost,\n TranscriptionAudioCost,\n} from './usage.ts';\nexport { addAudioCounters, priceAudio } from './usage.ts';\nexport type { VadHook, VadUpdate } from './vad.ts';\nexport { energyVad } from './vad.ts';\n"],
|
|
5
|
-
"mappings": "AAUA,SAAS,wBAAwB;AACjC,SAAS,kBAAkB;AAE3B,SAAS,2BAA2B;AACpC;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AACP,SAAS,yBAAyB;AAiBlC;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAYP,SAAS,cAAc,gBAAgB,gBAAgB,sBAAsB;AAC7E;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAEP,SAAS,iBAAiB;AAE1B,SAAS,yBAAyB;AAUlC,SAAS,UAAU,aAAa,mBAAmB;AAEnD,SAAS,yBAAyB;AASlC,SAAS,gCAAgC;AAazC,SAAS,kBAAkB;AAS3B,SAAS,kBAAkB,kBAAkB;AAE7C,SAAS,iBAAiB;",
|
|
4
|
+
"sourcesContent": ["export type {\n AudioChunk,\n AudioEncoding,\n AudioFormat,\n AudioSink,\n AudioSource,\n AudioTag,\n CharAlignment,\n TextSpan,\n} from './audio.ts';\nexport { resolveVoiceAuth } from './auth.ts';\nexport { blankUsage } from './blank-usage.ts';\nexport type { CaptionHistoryEntry } from './captions.ts';\nexport { captionsFromHistory } from './captions.ts';\nexport {\n convertAudio,\n convertAudioSource,\n decodeAudioBase64,\n encodeAudioBase64,\n} from './convert.ts';\nexport type { EchoCanceller, EchoControl, EchoMode, EchoReport, MicOpen } from './echo.ts';\nexport {\n assertNodeEcho,\n calibrateEcho,\n chooseCapture,\n echoRuntime,\n measureEcho,\n micOpen,\n openEchoMic,\n} from './echo.ts';\nexport { VoiceRequestError } from './errors.ts';\nexport type {\n Delivery,\n DeliverySupport,\n KnownRealtimeApi,\n KnownSpeechApi,\n KnownTranscriptionApi,\n RealtimeApi,\n RealtimeModel,\n SpeechApi,\n SpeechModel,\n TranscriptionApi,\n TranscriptionModel,\n VoiceInfo,\n VoiceModel,\n VoiceModelType,\n} from './models.ts';\nexport {\n getVoiceModel,\n listVoiceModels,\n registerVoiceModel,\n} from './models.ts';\nexport type {\n DeliveryPlan,\n RealtimeApiProvider,\n RealtimeContext,\n RealtimeEvent,\n RealtimeOptions,\n RealtimeSession,\n RealtimeSocket,\n RealtimeToken,\n TruncationPlan,\n} from './realtime.ts';\nexport { planDelivery, planTruncation, RealtimeLedger, realtimeSocket } from './realtime.ts';\nexport {\n connectRealtime,\n listen,\n registerRealtimeApiProvider,\n registerSpeechApiProvider,\n registerTranscriptionApiProvider,\n speak,\n transcribe,\n} from './registry.ts';\nexport type { AudioOut, BedChannel, Cue, CueSource, Soundscape } from './soundscape.ts';\nexport { followBed } from './soundscape.ts';\nexport type { SpeechApiProvider, SpeechEvent, SpeechInput } from './speech.ts';\nexport { SpeechEventStream } from './speech.ts';\nexport type { VoiceAgentEvent, VoiceMessageChange } from './spoken.ts';\nexport type {\n BindThinkerOptions,\n BoundThinker,\n Thinker,\n ThinkerCaption,\n ThinkerPhase,\n ThinkerUpdate,\n} from './talker.ts';\nexport { askAgent, bindThinker, talkerTools } from './talker.ts';\nexport type { RealtimeConstraints } from './tokens.ts';\nexport { mintRealtimeToken } from './tokens.ts';\nexport type {\n AudioInput,\n ListenOptions,\n Transcript,\n TranscriptionApiProvider,\n TranscriptionEvent,\n TranscriptSegment,\n} from './transcription.ts';\nexport { TranscriptionEventStream } from './transcription.ts';\nexport type {\n BargeIn,\n CaptionEvent,\n EchoRecommendation,\n VoiceConversation,\n VoiceCut,\n VoiceModels,\n VoicePhase,\n VoiceSession,\n VoiceTurnMeta,\n VoiceTurnOptions,\n} from './turns.ts';\nexport { voiceTurns } from './turns.ts';\nexport type {\n AudioCost,\n AudioCounters,\n AudioUsage,\n RealtimeAudioCost,\n SpeechAudioCost,\n TranscriptionAudioCost,\n} from './usage.ts';\nexport { addAudioCounters, priceAudio } from './usage.ts';\nexport type { VadHook, VadUpdate } from './vad.ts';\nexport { energyVad } from './vad.ts';\n"],
|
|
5
|
+
"mappings": "AAUA,SAAS,wBAAwB;AACjC,SAAS,kBAAkB;AAE3B,SAAS,2BAA2B;AACpC;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAEP;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AACP,SAAS,yBAAyB;AAiBlC;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAYP,SAAS,cAAc,gBAAgB,gBAAgB,sBAAsB;AAC7E;AAAA,EACE;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,EACA;AAAA,OACK;AAEP,SAAS,iBAAiB;AAE1B,SAAS,yBAAyB;AAUlC,SAAS,UAAU,aAAa,mBAAmB;AAEnD,SAAS,yBAAyB;AASlC,SAAS,gCAAgC;AAazC,SAAS,kBAAkB;AAS3B,SAAS,kBAAkB,kBAAkB;AAE7C,SAAS,iBAAiB;",
|
|
6
6
|
"names": []
|
|
7
7
|
}
|