@todoforai/voiceloop 0.1.12 → 0.1.14

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -17,7 +17,7 @@ A zero-dependency JavaScript library that runs the full voice loop — **VAD →
17
17
  - **Real barge-in** — triggers on transcribed *novel words*, not mic energy, so the agent's own voice never cuts it off.
18
18
  - **Self-echo filtering** — 0 self-interruptions with AEC off (Pipecat cut itself 20/30, OpenAI Realtime 17/30 [bench](bench/results/RESULTS.md#scenario-echo-smalltalk--speakermic-coupling-15db--30ms)).
19
19
  - **First audio <1s** — TTS speaks sentence 1 while the LLM writes sentence 2, and the LLM call starts speculatively during your end-of-turn pause.
20
- - **Serialized turns** — rapid-fire turns, tool results, holds and replays can never talk over each other. Locked in by 178 tests.
20
+ - **Serialized turns** — rapid-fire turns, tool results, holds and replays can never talk over each other. Locked in by 184 tests.
21
21
  - **Local-first** — Silero VAD and Piper TTS run as WASM in the tab: free, no cloud round-trip (CDN, then cache).
22
22
 
23
23
  Extra:
@@ -169,7 +169,8 @@ const agent = new VoiceAgent({
169
169
  // toolGate: only for adapters that run tools themselves — see below; ignore it otherwise
170
170
  for await (const delta of myProvider.stream({ history, system, signal })) {
171
171
  yield { text: delta }; // speech text → streamed into TTS
172
- // yield { tool: 'name', args: {...} }; // tool call → fired during playback
172
+ // yield { tool: 'name', args: {...} }; // tool call → the agent runs it during playback;
173
+ // // the NEXT user turn sees its tool_result
173
174
  }
174
175
  },
175
176
  });
@@ -190,13 +191,13 @@ const agent = new VoiceAgent({
190
191
  });
191
192
  ```
192
193
 
193
- Tool calls execute while the agent is still talking. Results are recorded in a per-turn ledger so the model never re-fires the same call, and `agent.notify('[TOOL RESULT get_weather] 22°C sunny')` relays an async outcome back for a spoken follow-up — bursts of results collapse into one reply instead of three interrupting monologues.
194
+ Tool calls execute while the agent is still talking. The default adapter is a loop: it feeds each result back to the model inside the same reply, so the model keeps talking with the answer in hand (bounded to 6 model messages per turn). Results are recorded natively (tool_use / tool_result) so the model never re-fires the same call, and `agent.notify('[TOOL RESULT get_weather] 22°C sunny')` relays an async outcome back for a spoken follow-up — bursts of results collapse into one reply instead of three interrupting monologues.
194
195
 
195
196
  Tools are **dispatched, not awaited**: the turn completes when the agent stops speaking, whatever the tool is still doing. A slow or hung tool can never stall the conversation, and a tool is free to `await agent.notify(...)` with its own result. Long work belongs in a tool that returns promptly ("checking that now") and delivers the outcome later via `notify()`.
196
197
 
197
198
  ### Adapters that run tools themselves
198
199
 
199
- Some adapters (agent loops) execute tools **inside** the generator, feeding results back to the model so it keeps talking with the answer in hand — all within one turn. Declare two flags so voiceloop adapts:
200
+ The default `makeOpenAILLM` and any agent loop execute tools **inside** the generator, feeding results back to the model so it keeps talking with the answer in hand — all within one turn. A custom loop declares two flags so voiceloop adapts, and yields `{ newMessage: true }` where the model's next message starts:
200
201
 
201
202
  ```js
202
203
  async function* myAgentLoop(history, system, signal, { toolGate } = {}) {
@@ -204,6 +205,7 @@ async function* myAgentLoop(history, system, signal, { toolGate } = {}) {
204
205
  if (ev.type === 'text') yield { text: ev.delta };
205
206
  if (ev.type === 'tool_start') yield { tool: ev.name, id: ev.id, args: ev.args, running: true };
206
207
  if (ev.type === 'tool_end') yield { tool: ev.name, id: ev.id, args: ev.args, result: ev.result };
208
+ if (ev.type === 'message_start' && !first) yield { newMessage: true };
207
209
  }
208
210
  }
209
211
  myAgentLoop.executesTools = true; // chunks carry the outcome — voiceloop reports, never re-runs
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@todoforai/voiceloop",
3
- "version": "0.1.12",
3
+ "version": "0.1.14",
4
4
  "description": "Fast, interruptible browser voice agent: VAD → STT → LLM → TTS with real barge-in, echo filtering, streaming first-sentence TTS and tap-to-seek. Zero runtime dependencies.",
5
5
  "type": "module",
6
6
  "main": "src/index.js",
package/src/index.d.ts CHANGED
@@ -43,7 +43,10 @@ export interface VoiceTool {
43
43
  // reports it instead of running it again.
44
44
  export type LLMChunk =
45
45
  | { text: string }
46
- | { tool: string; id?: string; args: Record<string, unknown>; result?: unknown; error?: string; running?: true };
46
+ | { tool: string; id?: string; args: Record<string, unknown>; result?: unknown; error?: string; running?: true }
47
+ /** Loop adapters only: the model's NEXT message starts here (after the results it fed back), so
48
+ * the agent records text/calls before and after as separate assistant messages. */
49
+ | { newMessage: true };
47
50
  // The conversation as the model sees it, Anthropic-shaped. Plain turns are strings; a turn in which
48
51
  // the assistant called tools is recorded NATIVELY: an assistant message of text + tool_use blocks,
49
52
  // then a user message holding the matching tool_result blocks (content = the result text the model
@@ -56,9 +59,10 @@ export type HistoryBlock =
56
59
  | { type: 'tool_result'; tool_use_id: string; content: string; is_error?: boolean };
57
60
  export type HistoryMessage = { role: string; content: string | HistoryBlock[] };
58
61
  // Pluggable LLM: lazy async generator over the chat history (fetch fires on first pull).
59
- // The default (makeOpenAILLM) yields bare tool chunks that the AGENT executes; their results are
60
- // recorded as tool_result and the agent starts a follow-up turn so the model reacts to them.
61
- // An agent LOOP that runs tools inside its own generator declares the two flags below.
62
+ // The default (makeOpenAILLM) is a tool LOOP: it runs each call inside the generator, feeds the
63
+ // result back to the model and marks the model's next message with { newMessage }. A plain adapter
64
+ // (no flags) yields bare tool chunks that the AGENT executes and records; the NEXT user turn sees
65
+ // the tool_result (no reaction turn is started).
62
66
  export interface LLM {
63
67
  (
64
68
  history: HistoryMessage[],
@@ -68,12 +72,12 @@ export interface LLM {
68
72
  ): AsyncIterable<LLMChunk>;
69
73
  /** This generator is a tool LOOP: it runs every call itself and feeds the result back inside the
70
74
  * same reply (chunks carry `result`/`error`, optionally preceded by a `running` announcement).
71
- * The agent never executes its calls and never starts a reaction turn for them. */
75
+ * The agent never executes its calls. */
72
76
  executesTools?: boolean;
73
77
  /** This generator awaits `options.toolGate` before ANY tool executes. Required to keep
74
78
  * speculative prefetch on: prefetch starts from an uncommitted interim, and the gate is what
75
79
  * stops a speculation from firing a real side effect. `executesTools` without this silently
76
- * disables prefetch (see voice-agent.js:793). */
80
+ * disables prefetch (see _startPrefetch in voice-agent.js). */
77
81
  acceptsToolGate?: boolean;
78
82
  }
79
83
 
package/src/llm-openai.js CHANGED
@@ -1,9 +1,10 @@
1
1
  // ── Default LLM: any OpenAI-compatible /chat/completions endpoint ───────────────────────────────
2
- // Normalizes the OpenAI streaming wire format to the VoiceAgent chunk contract:
3
- // { text } — one per content delta (streamed straight into the TTS)
4
- // { tool, args, id }— one per COMPLETED tool call (arguments accumulate across deltas; a call
5
- // whose accumulated arguments never parse as JSON is DROPPED with a warning —
6
- // firing a side-effecting tool with guessed/empty args would be worse)
2
+ // A LOOP adapter (executesTools + acceptsToolGate): one turn may span several model messages. Each
3
+ // completion streams { text } deltas straight to the TTS; a COMPLETED tool call (arguments accumulate
4
+ // across deltas; a call whose arguments never parse as JSON is DROPPED with a warning — firing a
5
+ // side-effecting tool with guessed args would be worse) is announced ({ running: true }), executed
6
+ // here, reported with its outcome, fed back to the model as a tool message, and the model's next
7
+ // message starts after a { newMessage } boundary — bounded by MAX_ROUNDS per turn.
7
8
  //
8
9
  // Works against OpenAI, Groq, Cerebras, Together, OpenRouter, Ollama, vLLM, LiteLLM… anything that
9
10
  // speaks /chat/completions SSE. NEVER ship a provider secret key to a public page — in production
@@ -37,6 +38,19 @@ export const toOpenAIMessages = (history) => history.flatMap((m) => {
37
38
  return text ? [...results, { role: m.role, content: text }] : results; // OpenAI has no is_error → text-marked
38
39
  });
39
40
 
41
+ // Render a host tool's return value as model-facing text (the tool_result content in history, and
42
+ // what adapters feed back to the model). Never throws: the side effect already RAN, so an
43
+ // unserializable result must not retro-fail the call and invite a retry.
44
+ export const toolResultText = (result) => {
45
+ if (result === undefined) return 'ok';
46
+ if (typeof result === 'string') return result;
47
+ if (result && typeof result.text === 'string' && typeof result.ok === 'boolean')
48
+ return result.ok ? result.text : `[failed] ${result.text}`;
49
+ try { return JSON.stringify(result) ?? 'ok'; } catch { return String(result); }
50
+ };
51
+
52
+ const MAX_ROUNDS = 6; // model messages per user turn — a model that keeps calling tools then waits for the user
53
+
40
54
  export function makeOpenAILLM({ llmUrl, apiKey = '', model = '', maxTokens = 1024, tools = {}, fetchFn, extraBody = {} } = {}) {
41
55
  const toolSpecs = Object.entries(tools).map(([name, t]) => ({
42
56
  type: 'function',
@@ -47,15 +61,15 @@ export function makeOpenAILLM({ llmUrl, apiKey = '', model = '', maxTokens = 102
47
61
  },
48
62
  }));
49
63
 
50
- return async function* (history, system, signal) {
51
- if (!llmUrl) throw new Error('VoiceAgent: no `llm` generator and no `llmUrl` configured — pass one of them');
64
+ // One completion: streams { text } and COMPLETED { tool, args, id } chunks for a wire-shaped message list.
65
+ async function* streamOnce(messages, signal) {
52
66
  const doFetch = fetchFn ?? fetch;
53
67
  const r = await doFetch(llmUrl, {
54
68
  method: 'POST', signal,
55
69
  headers: { 'Content-Type': 'application/json', ...(apiKey ? { Authorization: `Bearer ${apiKey}` } : {}) },
56
70
  body: JSON.stringify({
57
71
  stream: true,
58
- messages: [{ role: 'system', content: system }, ...toOpenAIMessages(history)],
72
+ messages,
59
73
  max_tokens: maxTokens,
60
74
  ...(model ? { model } : {}),
61
75
  ...(toolSpecs.length ? { tools: toolSpecs } : {}),
@@ -76,9 +90,6 @@ export function makeOpenAILLM({ llmUrl, apiKey = '', model = '', maxTokens = 102
76
90
  for (const c of pending.values()) {
77
91
  if (!c.name) continue;
78
92
  let args;
79
- // Truncated/invalid argument JSON (aborted stream, provider bug) → DROP the call. The agent
80
- // dispatches tools immediately, so guessing `{}` could fire a real side effect with the
81
- // wrong arguments.
82
93
  try { args = c.args ? JSON.parse(c.args) : {}; }
83
94
  catch { console.warn(`makeOpenAILLM: dropping tool call "${c.name}" — arguments never parsed: ${c.args}`); continue; }
84
95
  yield { tool: c.name, args, ...(c.id ? { id: c.id } : {}) };
@@ -86,9 +97,8 @@ export function makeOpenAILLM({ llmUrl, apiKey = '', model = '', maxTokens = 102
86
97
  pending.clear();
87
98
  };
88
99
 
89
- // One SSE data payload → zero or more chunks. Returns 'done' on [DONE].
100
+ // One SSE data payload → zero or more chunks.
90
101
  const handle = function* (payload) {
91
- if (payload === '[DONE]') return; // caller checks the sentinel itself
92
102
  let m; try { m = JSON.parse(payload); } catch { return; } // tolerate keep-alive junk
93
103
  if (m.error) throw new Error(m.error.message ?? String(m.error));
94
104
  const choice = m.choices?.[0];
@@ -136,5 +146,46 @@ export function makeOpenAILLM({ llmUrl, apiKey = '', model = '', maxTokens = 102
136
146
  } finally {
137
147
  reader.cancel().catch(() => {});
138
148
  }
149
+ }
150
+
151
+ const gen = async function* (history, system, signal, { toolGate } = {}) {
152
+ if (!llmUrl) throw new Error('VoiceAgent: no `llm` generator and no `llmUrl` configured — pass one of them');
153
+ // Anthropic-shaped working history (what VoiceAgent keeps); re-serialized to the wire every round
154
+ // so tool_use/tool_result pairing stays in ONE place (toOpenAIMessages).
155
+ const hist = [...history];
156
+ for (let round = 0; round < MAX_ROUNDS; round++) {
157
+ let text = '';
158
+ const calls = []; // { id, tool, args, outcome: Promise<{result}|{error}> }
159
+ const seen = new Set();
160
+ for await (const c of streamOnce([{ role: 'system', content: system }, ...toOpenAIMessages(hist)], signal)) {
161
+ if (c.text) { text += c.text; yield c; continue; }
162
+ const id = c.id || `call-${round}-${calls.length + 1}`; // announcements need an id to be replaced in place
163
+ if (seen.has(id)) continue;
164
+ seen.add(id);
165
+ yield { tool: c.tool, id, args: c.args, running: true };
166
+ // Execute NOW, in parallel with TTS. A speculative run waits at the gate: it rejects when the
167
+ // speculation is dropped, so the tool never fires for a turn the user was still amending.
168
+ const outcome = (async () => {
169
+ if (toolGate) await toolGate;
170
+ try { return { result: await tools[c.tool]?.run?.(c.args) }; }
171
+ catch (e) { return { error: e?.message || String(e ?? 'failed') }; }
172
+ })();
173
+ calls.push({ id, tool: c.tool, args: c.args, outcome });
174
+ }
175
+ if (!calls.length) return;
176
+ const results = [];
177
+ for (const c of calls) {
178
+ const o = await c.outcome;
179
+ yield { tool: c.tool, id: c.id, args: c.args, ...o };
180
+ const failed = 'error' in o || (o.result && typeof o.result === 'object' && o.result.ok === false);
181
+ results.push({ type: 'tool_result', tool_use_id: c.id, content: 'error' in o ? String(o.error) : toolResultText(o.result), ...(failed ? { is_error: true } : {}) });
182
+ }
183
+ hist.push({ role: 'assistant', content: [...(text ? [{ type: 'text', text }] : []), ...calls.map((c) => ({ type: 'tool_use', id: c.id, name: c.tool, input: c.args ?? {} }))] });
184
+ hist.push({ role: 'user', content: results });
185
+ yield { newMessage: true };
186
+ }
139
187
  };
188
+ gen.executesTools = true;
189
+ gen.acceptsToolGate = true;
190
+ return gen;
140
191
  }
@@ -19,7 +19,8 @@
19
19
 
20
20
  import { STT_PROVIDERS, resolveSttProvider } from './stt.js';
21
21
  import { TUNING } from './tuning.js';
22
- import { makeOpenAILLM } from './llm-openai.js';
22
+ import { makeOpenAILLM, toolResultText } from './llm-openai.js';
23
+ export { toolResultText };
23
24
 
24
25
  // Default persona: a minimal voice-first system prompt. Hosts replace it via `persona`
25
26
  // (who the agent is) and append live context via `sysmsg` / setSysmsg().
@@ -76,23 +77,13 @@ const withTimeout = (p, ms, fallback) => {
76
77
  ]);
77
78
  };
78
79
 
79
- const callArgs = (c) => JSON.stringify(c.args ?? {}, Object.keys(c.args ?? {}).sort());
80
+ const callArgs = (c) => JSON.stringify(c.args ?? {}, (_k, v) =>
81
+ v && typeof v === 'object' && !Array.isArray(v) ? Object.fromEntries(Object.keys(v).sort().map((k) => [k, v[k]])) : v);
80
82
  const callKey = (c) => c.id ? `id:${c.id}` : `${c.tool} ${callArgs(c)}`;
81
- // Render a host tool's return value as model-facing text (the tool_result content in history, and
82
- // what LLM adapters feed back to the model). Never throws: the side effect already RAN,
83
- // so an unserializable result must not retro-fail the call and invite a retry.
84
- export const toolResultText = (result) => {
85
- if (result === undefined) return 'ok';
86
- if (typeof result === 'string') return result;
87
- if (result && typeof result.text === 'string' && typeof result.ok === 'boolean')
88
- return result.ok ? result.text : `[failed] ${result.text}`;
89
- try { return JSON.stringify(result) ?? 'ok'; } catch { return String(result); }
90
- };
91
83
  // One model-facing history record for a settled/pending tool call: the native tool_result block that
92
84
  // pairs with its tool_use. Bounded — history is spoken-conversation-sized, not a log. A call whose
93
85
  // outcome has not landed yet says so; _runTool patches the SAME block in place when it does.
94
86
  const RESULT_MAX = 4000;
95
- const MAX_TOOL_ROUNDS = 6;
96
87
  const PENDING_RESULT = 'still running — the result will follow in a later turn';
97
88
  const bound = (text) => text.length > RESULT_MAX ? `${text.slice(0, RESULT_MAX)}… (${text.length} chars)` : text;
98
89
  const toolResultBlock = (c) => {
@@ -313,8 +304,6 @@ export class VoiceAgent {
313
304
  this.state = 'idle'; // idle | listening | thinking | speaking
314
305
  this._abort = null; // aborts the in-flight LLM
315
306
  this._replySeq = 0; // bumped per reply-generating turn; a queued turn drops itself if a newer one exists
316
- this._toolRounds = 0; // tool-result reactions since the user last spoke (see _runTurn react)
317
- this._late = []; // outcomes landed after the last request was built (see _runTurn react)
318
307
  this._preroll = []; this._wasSpeaking = false; this._closed = false; this._destroyed = false;
319
308
  // Diagnostic tap: last ~30s of the exact i16 frames handed to STT + delivery-stall gaps, for
320
309
  // dumpAudio() (replay any "it misheard me" run and prove capture clean-or-not).
@@ -636,7 +625,6 @@ export class VoiceAgent {
636
625
  // history stays ordered (…, assistant-so-far, user, user). Held pushes NEVER start an LLM turn —
637
626
  // release (_flush) owns starting the single turn, and waits for all of these to settle first.
638
627
  _pushHeld(text) {
639
- this._toolRounds = 0;
640
628
  const p = (async () => { await this._holdBarrier; this.history.push({ role: 'user', content: text }); })();
641
629
  this._heldPushes.add(p);
642
630
  p.finally(() => this._heldPushes.delete(p));
@@ -699,7 +687,6 @@ export class VoiceAgent {
699
687
  // caller (errors surface via onEvent), so it's safe to call without awaiting/catching.
700
688
  _onUserTurn(text, { speech = false } = {}) {
701
689
  if (!text.trim()) return Promise.resolve();
702
- this._toolRounds = 0;
703
690
  return this._runTurn(text, { speech });
704
691
  }
705
692
 
@@ -736,8 +723,8 @@ export class VoiceAgent {
736
723
  _startPrefetch(text) {
737
724
  // Speculating is only safe if pulling the first chunk cannot fire a real side effect on a
738
725
  // transcript the user is still speaking and may yet change — abort cannot undo a sent email.
739
- // The default LLM's chunks are inert (tools run later, in _speakTurn). An LLM that executes
740
- // tools INSIDE its generator (llm.executesTools) is safe only if it also honors `toolGate`
726
+ // An LLM whose chunks are inert (no tool execution) is always safe. One that executes tools
727
+ // INSIDE its generator (llm.executesTools) is safe only if it also honors `toolGate`
741
728
  // (llm.acceptsToolGate): we hand it a promise it must await before ANY tool runs, so text
742
729
  // streams speculatively while tools wait for the turn to commit. Neither → don't speculate.
743
730
  if (this.llm.executesTools && !this.llm.acceptsToolGate) return;
@@ -816,7 +803,7 @@ export class VoiceAgent {
816
803
  // if nothing newer has queued behind it, and never cut the reply in flight. Anything newer already
817
804
  // sees this message in history and answers over the fuller picture — so a burst of tool results
818
805
  // collapses into ONE reply instead of one interrupting monologue per result.
819
- _runTurn(text, { supersede = true, speech = false, react = false } = {}) {
806
+ _runTurn(text, { supersede = true, speech = false } = {}) {
820
807
  const prev = this._turn;
821
808
  if (supersede) {
822
809
  this._abort?.abort(); // cut the in-flight reply NOW (barge-in / superseded)
@@ -842,24 +829,9 @@ export class VoiceAgent {
842
829
  // Only a SPOKEN final may adopt (it seeded the speculation); typed/notify/held turns drop it.
843
830
  const pf = speech && text ? this._takePrefetch(text) : (this._dropPrefetch(), null);
844
831
  if (text) this.history.push({ role: 'user', content: text });
845
- // A reaction turn owes a reply only for results no request has carried yet: either the
846
- // tool_result record is still the last message, or a late outcome landed after the last
847
- // request was built (_late; cleared when a request starts) — then a short [EVENT] user turn,
848
- // the host's channel for out-of-band facts, points the model at the patched block (a request
849
- // can't end on an assistant message). Bounded per user utterance (_toolRounds, reset on
850
- // every accepted user turn): a model that keeps calling tools gets MAX_TOOL_ROUNDS
851
- // reactions, then waits for the user.
852
- if (react) {
853
- const last = this.history[this.history.length - 1];
854
- const owed = (last?.role === 'user' && Array.isArray(last.content)) || this._late.length;
855
- if (!owed || this._toolRounds >= MAX_TOOL_ROUNDS) { this._late.length = 0; return; }
856
- this._toolRounds++;
857
- if (last?.role !== 'user') this.history.push({ role: 'user', content: `[EVENT tool_result] ${this._late.map((c) => c.tool).join(', ')} finished — see the tool_result above.` });
858
- }
859
832
  // Reply only if the conversation now ends on a user turn (a raced turn may have already
860
833
  // replied). pf is always null here: if we pushed a user turn it IS the last entry.
861
834
  if (this.history[this.history.length - 1]?.role !== 'user') return;
862
- this._late.length = 0; // this request carries every result landed so far
863
835
  this._set('thinking'); // LLM stage
864
836
  this._abort = pf?.ctl ?? new AbortController(); // adopted prefetch keeps ITS controller so barge-in aborts the right fetch
865
837
  await this._speakTurn(this._abort.signal, pf?.gen, seq); // _speakTurn never throws — errors surface via onEvent
@@ -887,43 +859,33 @@ export class VoiceAgent {
887
859
  // notation it could imitate. `this.llm(...)` is a lazy generator — its fetch fires only when
888
860
  // tts.speak first pulls, by which point any prior turn is aborted; keeps LLM streams serial.
889
861
  //
890
- // Round boundary: the default adapter's stream is ONE model message (text + calls together;
891
- // the reaction to the results comes from _continue). A loop adapter feeds results back inside
892
- // the stream, so anything after an outcome landed is the model's NEXT message.
862
+ // Round boundary: a loop adapter (executesTools) feeds results back inside the stream and marks
863
+ // where the model's NEXT message starts with a { newMessage } chunk. A plain adapter's stream is
864
+ // ONE model message; the agent runs its calls and the NEXT user turn sees the results.
893
865
  let answer = '';
894
866
  const rounds = [], calls = []; // calls: every act this turn saw, for dedup — outlives recording
895
867
  const loop = !!this.llm.executesTools;
896
- const round = () => {
897
- const r = rounds[rounds.length - 1];
898
- if (r && !(loop && r.settled)) return r;
899
- rounds.push({ text: '', calls: [] }); return rounds[rounds.length - 1];
900
- };
868
+ const round = () => rounds[rounds.length - 1] ?? (rounds.push({ text: '', calls: [] }), rounds[rounds.length - 1]);
869
+ const newRound = () => { const r = rounds[rounds.length - 1]; if (r && (r.text || r.calls.length)) rounds.push({ text: '', calls: [] }); };
901
870
  let n = 0;
902
871
  // Accept a call once: identity is the provider id, else name+args (an id-less repeat is a
903
- // provider re-emitting its buffered call). `owner` says who executes it — the adapter (the
904
- // chunk announces or carries the outcome: it ran the tool itself and feeds the result back
905
- // inside its own reply) or the agent (_runTool runs it) — fixed at acceptance.
872
+ // provider re-emitting its buffered call).
906
873
  const accept = (item) => {
907
874
  const key = callKey(item);
908
875
  let c = calls.find((x) => x.key === key);
909
876
  if (c) return c;
910
- const adapterRan = loop || item.running || 'result' in item || 'error' in item;
911
- c = { ...item, key, id: item.id || `call-${turn}-${++n}`, owner: adapterRan ? 'adapter' : 'agent' };
877
+ c = { ...item, key, id: item.id || `call-${turn}-${++n}` };
912
878
  calls.push(c); round().calls.push(c);
913
879
  return c;
914
880
  };
915
- const outcome = (c, item) => {
916
- if ('result' in item) c.result = item.result; else if ('error' in item) c.error = item.error;
917
- const r = rounds.find((r) => r.calls.includes(c)); if (r) r.settled = true; // already recorded → no round to close
918
- };
881
+ const outcome = (c, item) => { if ('result' in item) c.result = item.result; else if ('error' in item) c.error = item.error; };
882
+ const hasOutcome = (c) => 'result' in c || 'error' in c;
919
883
  // An outcome may land AFTER the turn recorded the call (a hung tool, a barge-in mid-execution):
920
- // its tool_result block is patched IN PLACE so the next request carries the real result, and the
921
- // model gets to react to it (see _continue).
884
+ // its tool_result block is patched IN PLACE so the next request carries the real result.
922
885
  const settle = (c) => this._runTool(c).then(() => {
923
886
  if (!c.block) return; // not recorded yet — record() picks the outcome up
924
887
  if (!c.consumed) { Object.assign(c.block, toolResultBlock(c)); c.consumed = true; }
925
888
  this._dropPrefetch(); // a speculation serialized the pending block — stale
926
- this._late.push(c); this._continue();
927
889
  });
928
890
  const record = (keptLen) => {
929
891
  let budget = keptLen;
@@ -941,53 +903,45 @@ export class VoiceAgent {
941
903
  }
942
904
  rounds.length = 0;
943
905
  };
944
- // Follow-up only for calls the AGENT ran: a loop adapter fed its results back inside the reply.
945
- const recordAndContinue = (keptLen) => {
946
- record(keptLen);
947
- if (calls.some((c) => c.owner === 'agent' && c.consumed)) this._continue();
948
- };
949
906
  // Pre-execution markers: an LLM that runs tools inside its own generator (executesTools) can
950
907
  // announce a call with `running: true` BEFORE it executes, so the host shows a live spinner chip
951
908
  // that the outcome chunk replaces in place (same id). A turn that dies before an announced call's
952
909
  // outcome arrives finalizes the CHIP as interrupted; the history keeps the call with a pending
953
910
  // tool_result, patched in place if the outcome still lands.
954
- const announced = new Map();
955
- // Latched once this turn has finalized its chips (finishInterrupted ran). A barge-in detaches the
956
- // LLM producer from TTS, so it can wake AFTER we returned and keep pushing chunks: a late
911
+ // `finalized` latches once this turn has finalized its chips. A barge-in detaches the LLM
912
+ // producer from TTS, so it can wake AFTER we returned and keep pushing chunks: a late
957
913
  // announcement would open a spinner nothing will ever close, and a late UNEXECUTED call is work
958
914
  // the user cut off. Past the latch only outcomes flow (a tool the adapter really ran is truth).
959
915
  let finalized = false;
960
916
  const finishInterrupted = () => {
961
917
  finalized = true;
962
- for (const r of announced.values())
963
- this.onEvent({ type: 'tool', name: r.tool, id: r.id, args: r.args, result: { ok: false, text: 'interrupted — did not finish' } });
964
- announced.clear();
918
+ for (const c of calls) if (c.announced && !hasOutcome(c))
919
+ this.onEvent({ type: 'tool', name: c.tool, id: c.id, args: c.args, result: { ok: false, text: 'interrupted — did not finish' } });
965
920
  };
966
921
  const tapped = (async function* (self, src) {
967
922
  for await (const item of src) {
968
923
  // tool_use → fire NOW, in parallel with TTS. Identical repeats within one turn are the SAME
969
924
  // act (a provider re-emitting its buffered call, or the model asking twice) — run it once.
925
+ if (item.newMessage) { newRound(); continue; } // loop adapter: the model's next message starts here
970
926
  if (item.tool) {
971
- const hasOutcome = 'result' in item || 'error' in item;
972
927
  const known = calls.find((x) => x.key === callKey(item));
973
- if (item.running || (loop && !hasOutcome)) { // announcement; the outcome chunk below does the work
928
+ if (item.running || (loop && !hasOutcome(item))) { // announcement; the outcome chunk below does the work
974
929
  // An id-less announcement can never be replaced in place NOR finalized (both key on id),
975
930
  // so it could only ever become a stuck spinner — drop it. The outcome chunk still lands.
976
931
  if (finalized || !item.id || known) continue;
977
- announced.set(item.id, item); accept(item);
932
+ accept(item).announced = true;
978
933
  self.onEvent({ type: 'tool', name: item.tool, id: item.id, args: item.args, running: true });
979
934
  continue;
980
935
  }
981
- if (item.id) announced.delete(item.id);
982
936
  if (known) { // the announced call's outcome, or a re-emit
983
- if (known.ran || !hasOutcome) continue;
937
+ if (known.ran || !hasOutcome(item)) continue;
984
938
  outcome(known, item); settle(known);
985
939
  continue;
986
940
  }
987
- if (finalized && !hasOutcome) continue; // late bare call: the user cut this reply off
941
+ if (finalized && !hasOutcome(item)) continue; // late bare call: the user cut this reply off
988
942
  const c = accept(item);
989
- if (hasOutcome) outcome(c, item);
990
- if (finalized) record(0); // late outcome of an unseen call: give it a record of its own (settle schedules the reaction)
943
+ outcome(c, item);
944
+ if (finalized) record(0); // late outcome of an unseen call: give it a record of its own
991
945
  settle(c);
992
946
  continue;
993
947
  }
@@ -1051,7 +1005,7 @@ export class VoiceAgent {
1051
1005
  // Tools fire DURING streaming, so a reply that dies here may already have dispatched one.
1052
1006
  // Record what was done and heard so far — dropping it would let the next turn call the
1053
1007
  // same tool again.
1054
- recordAndContinue(signal.keepFull ? answer.length : this._replyText.length);
1008
+ record(signal.keepFull ? answer.length : this._replyText.length);
1055
1009
  finishInterrupted(); // announced-but-unfinished → failed chip, not a stuck spinner
1056
1010
  if (this.state !== 'idle') this._set('listening');
1057
1011
  return;
@@ -1075,7 +1029,7 @@ export class VoiceAgent {
1075
1029
  // tail aligns). keepFull/normal-finish → whole answer (no tail); a barge-in → the heard prefix
1076
1030
  // solid + the unspoken remainder dimmed, showing exactly where TTS got.
1077
1031
  const kept = signal.keepFull ? answer : rawPrefix(spoken);
1078
- recordAndContinue(kept.length);
1032
+ record(kept.length);
1079
1033
  if (kept) {
1080
1034
  this.onEvent({ type: 'assistant', text: kept, full: answer, final: true, turn });
1081
1035
  } else if (answer) {
@@ -1093,19 +1047,6 @@ export class VoiceAgent {
1093
1047
  if (!signal.aborted && this.state !== 'idle') this._set('listening'); // superseded turns don't touch state
1094
1048
  }
1095
1049
 
1096
- // Native tool semantics: once a tool_result is in history the model gets to react to it. Queues a
1097
- // SOFT reaction turn (behind the running one; dropped if a newer user turn supersedes it — that
1098
- // turn sees the result anyway) after the agent ran a call for the default adapter (its stream
1099
- // ends at the call; the result was never fed back), or when a LATE outcome (hung/slow tool,
1100
- // barge-in mid-execution) landed in a record. Whether a reply is still owed is decided when the
1101
- // turn RUNS (see _runTurn `react`), since the predecessor may already have carried the result.
1102
- // Loop adapters fed the result back inside the reply, so nothing fires for them. Silent while
1103
- // the user is speaking — their utterance will see the result in history anyway.
1104
- _continue() {
1105
- if (this._closed || this._speaking) return;
1106
- this._runTurn('', { supersede: false, react: true });
1107
- }
1108
-
1109
1050
  // Run one chosen tool call and report it. Fired as soon as its tool_use arrives in the stream, in
1110
1051
  // parallel with TTS, and NEVER joined: the turn records the call (tool_result pending) and finishes
1111
1052
  // when the speaking does (see _speakTurn), so a tool that hangs costs its own result and nothing