@inetafrica/open-claudia 3.5.26 → 3.5.27

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,23 +1,13 @@
1
1
  # Changelog
2
2
 
3
- ## 3.5.26
3
+ ## 3.5.27
4
4
 
5
- The Browser app's phone and web view stays sharp after a page load. A canary build for the Browser canary bot first; the fleet waits for Sumeet's OK.
5
+ Calls answer in about two seconds again, and say something while they work (Sumeet, 11 Oct 08:24 EAT on a walk, after a call where "hello" took 8.5 s and a 79 s lookup was never heard: "Yeah, let's give it a shot"). A canary build for the owner's own bot first. It builds on 3.5.24; the Browser canaries 3.5.25 and 3.5.26 are not in it yet.
6
6
 
7
- - **Sharp after every page.** On the canary's Chrome 150 (Linux) the driver's pictures dropped from the view's density to 1x at the first page load (the view kept its density; the picture did not). After a main-frame navigation or a load, the window is now resized by a pixel and back, which brings the density back: 906x1760 pictures for a 412x800 phone at 2.2x on duckduckgo.com, en.m.wikipedia.org and example.com, 1 to 4 s after each load.
8
- - **Tests:** `test-browser-drive.js`.
9
-
10
- ## 3.5.25
11
-
12
- The Browser app feels like a browser on the phone and on the web (Sumeet, 10 Oct 08:53 EAT on his phone: "it feels like a browser on my phone… like Google Chrome app", and 09:18: "the web and mobile still need to also feel native… don't forget the website"). A canary build for the Browser canary bot first; the fleet waits for his OK.
13
-
14
- - **The bot's tab takes the driver's screen.** With Chrome in its own container, a take (or a `fit` when the screen changes: a turned phone, a resized panel, a keyboard that opens) that carries the driver's screen (`viewport`: CSS size, pixel density, phone layout, touch) fits the bot's tab to it through `core/browser-drive.js`. A phone gets the page's phone layout, full screen and sharp; the web gets a desktop page at the panel's size. Handing back puts the tab back to its own size. The browser's identity (user agent) never changes, so sites signed into stay signed in after the hand back. `GET /api/browser/control` says `fit: true`.
15
- - **The driver's input goes straight to Chrome,** through the view's own DevTools session, in order. Touches keep their timing, so a swipe flings; typed text is inserted whole (`input_text`, at most 500 characters: accents, emoji, the keyboard's corrections); every point is scaled by the view's density, because Chrome reads input in the drawn picture's pixels. A bot without the view takes `input_text` as key presses.
16
- - **The page's text fields.** The view reads where the page's text fields are and what kind they are (never a value) and streams them as `fields` events, so the Browser page opens the phone's own keyboard of the right kind the moment a field is tapped.
17
- - **Pictures.** The view streams its own screencast at the driver's size (at most 1.6 million pixels, 30 a second). `/api/browser/stream` serves it while someone drives, up to 30 fps, and keeps the newest frame while a slow viewer drains instead of dropping it, so the last picture of a scroll always arrives.
18
- - **A hand back wakes the bot only when it waits for that person:** it sent them an Open Browser card (cards now record `openApp`), or one of its browser commands was refused while they drove (the tool guard marks it). Otherwise the hand back is only logged. On 9 Oct a test hand back set off a 28-minute tangent on the canary.
19
- - **Correction to 3.5.24:** in container mode, sessions are not pinned to tabs (Agent Space dropped `AGENT_BROWSER_PIN_TAB` in a98f239); a session uses the tab that is open.
20
- - **Tests:** `test-browser-drive.js` (new), `test-browser-control.js`, `test-web-browser-app.js`.
7
+ - **A quick voice says a fitting line right away** (`channels/kazee/call/quick-voice.js`). A kept-warm small model, one process per call through the provider's own utility invocation, hears each utterance and says one short line ("Sure, let me check the call logs.", "Yep, I'm here.") within about a second, then a brief "still on it" line whenever the full agent leaves the caller in 8 s of silence, at most 3 per turn. It never answers: every utterance still goes to the full agent. The line is dropped if it is not ready in 3.5 s, if the full agent already spoke, if the caller is talking, or once a newer utterance took over. Off unless `KAZEE_CALL_QUICK_VOICE=on`; `KAZEE_CALL_QUICK_MODEL` (default `claude-haiku-4-5`), `_EFFORT` (`low`) and `_THINKING` (off) tune it. Measured on 23 call lines with traps ("Did you hear me?", "Are you sure?"): Haiku 4.5 finished its line in 0.8-1.1 s with thinking off (2-3 s with it on), Sonnet 5 in 1.7-1.9 s, and both kept every rule.
8
+ - **The full agent speaks first, then works.** The live-call guidance used to forbid "speculative progress", so a lookup went silent until it ended. Now the first action is a 3-12 word call-say ("Sure, let me check the call logs."), then a progress line about every ten seconds. Replayed on the four turns of the 11 Oct call, 3 times per effort level: the holding line came first 3 of 3 times at max, xhigh, high and medium (the old guidance: 1 of 3 at xhigh), simple replies unchanged. With the quick voice on, the agent skips its own acknowledgement and starts with substance.
9
+ - **A sound with no words no longer silences the answer.** Speech start mutes the running call turn in case it is a new ask; when the recogniser found no words (wind, footsteps, a cough) the turn stayed muted for good, its call-say got 409 and its final answer was never spoken. The session now reports a noise-only turn and the turn gets its voice back, unless someone is still talking or another segment is pending.
10
+ - **Tests:** `test-call-quick-voice.js` (new, in `npm test`), `test-kazee-call-manager.js`, `test-kazee-call-session.js`, `test-provider-prompt-parity.js`.
21
11
 
22
12
  ## 3.5.24
23
13
 
@@ -5,6 +5,7 @@ const { CallLeaseHeartbeat } = require("./lease");
5
5
  const { publicCallIdentity, requestRoomAccess, supportedIceServers } = require("./access");
6
6
  const { canonicalForChannel } = require("../../../core/identity");
7
7
  const { KazeeCallSession } = require("./session");
8
+ const quickVoice = require("./quick-voice");
8
9
 
9
10
  const DEFAULT_JANUS_URL = "wss://janus.kazee.africa/ws";
10
11
  const DEFAULT_ICE_SERVERS = [
@@ -14,6 +15,34 @@ const DEFAULT_ICE_SERVERS = [
14
15
  // FCM push TTL is 60s and chat-central treats ringing logs as fresh for 90s.
15
16
  // This is the outbound invitation deadline; terminal server events end it early.
16
17
  const RING_TIMEOUT_MS = 60_000;
18
+ const qlog = (msg) => console.log(`${new Date().toISOString()} Kazee call quick: ${msg}`);
19
+ // The quick line must be ready this soon after the words, or it is skipped:
20
+ // by then the full agent is close to speaking itself.
21
+ const QUICK_LINE_MAX_MS = 3500;
22
+ // While the full agent works: a "still on it" line after this much quiet,
23
+ // at most a few per turn, never after the answer.
24
+ const QUICK_FILLER_QUIET_MS = 8000;
25
+ const QUICK_FILLER_MAX = 3;
26
+ const QUICK_TICK_MS = 500;
27
+
28
+ // The provider the call's chat runs on: the quick voice uses its CLI and
29
+ // credentials, as the full agent's turns do.
30
+ function chatProvider({ adapter, chatId, userId }) {
31
+ const { runInChat } = require("../../../core/context");
32
+ const { conversationKeyFor, roomKeyFor } = require("../../../core/identity");
33
+ const transport = adapter?.type || "kazee";
34
+ const id = runInChat({
35
+ adapter, channelId: String(chatId), userId: String(userId), transport,
36
+ canonicalUserId: canonicalForChannel(transport, String(userId)),
37
+ conversationKey: conversationKeyFor(transport, String(chatId)),
38
+ roomKey: roomKeyFor(transport, String(chatId)),
39
+ }, () => {
40
+ const state = require("../../../core/state");
41
+ return state.getActiveProvider(state.currentState());
42
+ });
43
+ return id ? require("../../../core/providers").registry.getProvider(id) : null;
44
+ }
45
+
17
46
  const OUTGOING_GREETING = "Hey, it's Claudia. Can you hear me okay?";
18
47
  // Inbound must greet too: callers get silence until they speak first, which
19
48
  // reads as a dead call (live test 2026-08-08 — media was up, caller hung up).
@@ -21,7 +50,8 @@ const INCOMING_GREETING = "Hey, it's Claudia — you're through. Can you hear me
21
50
 
22
51
  class KazeeCallManager {
23
52
  constructor({ adapter, SessionImpl = KazeeCallSession, janusUrl = process.env.KAZEE_JANUS_URL || DEFAULT_JANUS_URL,
24
- iceServers = DEFAULT_ICE_SERVERS, deviceId = randomUUID() } = {}) {
53
+ iceServers = DEFAULT_ICE_SERVERS, deviceId = randomUUID(), resolveQuickProvider = chatProvider, quickSpawn = undefined,
54
+ quickSettings = quickVoice.quickVoiceSettings, quickTiming = {} } = {}) {
25
55
  this.adapter = adapter;
26
56
  this.SessionImpl = SessionImpl;
27
57
  this.janusUrl = janusUrl;
@@ -30,6 +60,10 @@ class KazeeCallManager {
30
60
  this.deviceId = deviceId;
31
61
  this._pending = null;
32
62
  this._ringTimer = null;
63
+ this.resolveQuickProvider = resolveQuickProvider;
64
+ this.quickSpawn = quickSpawn;
65
+ this.quickSettings = quickSettings;
66
+ this.quickTiming = { lineMaxMs: QUICK_LINE_MAX_MS, fillerQuietMs: QUICK_FILLER_QUIET_MS, fillerMax: QUICK_FILLER_MAX, tickMs: QUICK_TICK_MS, ...quickTiming };
33
67
  }
34
68
 
35
69
  wire(socket) {
@@ -89,6 +123,7 @@ class KazeeCallManager {
89
123
  // Several terminal paths clear this.session before close() calls back.
90
124
  // Invalidate the closed call's work even when it no longer owns the slot.
91
125
  session?._voiceTurn?.controller.abort("call-ended");
126
+ this._stopQuickVoice(session);
92
127
  if (this.session !== session) return;
93
128
  this._detachSession(session);
94
129
  if (this.adapter._socket?.connected) this.adapter._socket.emit("call:end", {
@@ -98,6 +133,7 @@ class KazeeCallManager {
98
133
 
99
134
  _detachSession(session) {
100
135
  session?._voiceTurn?.controller.abort("call-ended");
136
+ this._stopQuickVoice(session);
101
137
  if (this.session === session) this.session = null;
102
138
  }
103
139
 
@@ -151,12 +187,14 @@ class KazeeCallManager {
151
187
  ...this._sessionOptions(config, access, boundIdentity, pending.controller.signal), greeting: INCOMING_GREETING,
152
188
  onTranscript: (text, meta) => this._transcript(text, { ...payload, ...boundIdentity }, meta, session),
153
189
  onSpeechStart: () => this._speechStart(session),
190
+ onSpeechNoise: () => this._speechNoise(session),
154
191
  onMediaReady: () => this._mediaReady(session),
155
192
  onEnded: reason => { session?._leaseHeartbeat?.close(); pending.controller.abort(); this._sessionEnded(session, reason); },
156
193
  });
157
194
  session.callIdentity = boundIdentity;
158
195
  session.chatId = chatId;
159
196
  this.session = session;
197
+ this._startQuickVoice(session, chatId, callerId);
160
198
  this._manageLease(session, access);
161
199
  await session.start();
162
200
  if (this.session !== session || session.closed || pending.controller.signal.aborted) return;
@@ -198,12 +236,14 @@ class KazeeCallManager {
198
236
  ...this._sessionOptions(config, access, identity, pending.controller.signal), asCaller: true, greeting: OUTGOING_GREETING,
199
237
  onTranscript: (text, meta) => this._transcript(text, remote, meta, session),
200
238
  onSpeechStart: () => this._speechStart(session),
239
+ onSpeechNoise: () => this._speechNoise(session),
201
240
  onMediaReady: () => this._mediaReady(session),
202
241
  onEnded: reason => { session?._leaseHeartbeat?.close(); pending.controller.abort(); this._clearRingTimer(); this._sessionEnded(session, reason); },
203
242
  });
204
243
  session.callIdentity = identity;
205
244
  session.chatId = chat;
206
245
  this.session = session;
246
+ this._startQuickVoice(session, chat, target);
207
247
  this._manageLease(session, access);
208
248
  await session.start();
209
249
  if (this.session !== session || session.closed || pending.controller.signal.aborted) throw new Error("Call cancelled");
@@ -247,6 +287,96 @@ class KazeeCallManager {
247
287
  // complete transcript exists; a cough must not discard an accepted ask.
248
288
  }
249
289
 
290
+ // The sound that muted the turn had no words in it: not a new request, so
291
+ // the answer being worked on may speak again (11 Oct: wind while walking
292
+ // muted a lookup's answer for good, and the caller heard nothing).
293
+ _speechNoise(session) {
294
+ const turn = session?._voiceTurn;
295
+ if (!turn?.muted || turn.controller.signal.aborted || this.session !== session || session.closed) return;
296
+ turn.muted = false;
297
+ console.log(`${new Date().toISOString()} Kazee call noise: muted turn resumed`);
298
+ }
299
+
300
+ // ── Quick voice ───────────────────────────────────────────────────
301
+
302
+ _startQuickVoice(session, chatId, userId) {
303
+ let settings;
304
+ try { settings = this.quickSettings(); } catch (_) { return; }
305
+ if (!settings?.enabled) return;
306
+ let provider = null;
307
+ try { provider = this.resolveQuickProvider({ adapter: this.adapter, chatId, userId }); } catch (_) {}
308
+ if (!quickVoice.QuickVoiceProcess.supported(provider)) {
309
+ qlog(`off: provider ${provider?.id || "none"} has no warm stream`);
310
+ return;
311
+ }
312
+ const quick = quickVoice.track(new quickVoice.QuickVoiceProcess({
313
+ provider, settings, spawnImpl: this.quickSpawn, log: qlog,
314
+ systemPrompt: quickVoice.buildQuickPrompt({ ownerName: quickVoice.ownerDisplayName() }),
315
+ }));
316
+ session._quick = quick;
317
+ // The quick voice says the first line itself; the session's delayed cue
318
+ // would only double it.
319
+ session.acknowledgement = async () => null;
320
+ try { quick.start(); } catch (error) {
321
+ qlog(`start failed: ${error.message}`);
322
+ session._quick = null;
323
+ quick.stop().catch?.(() => {});
324
+ }
325
+ }
326
+
327
+ _stopQuickVoice(session) {
328
+ const quick = session?._quick;
329
+ if (!quick) return;
330
+ session._quick = null;
331
+ quick.stop().catch?.(() => {});
332
+ }
333
+
334
+ // One utterance: a short fitting line as soon as the quick voice has it,
335
+ // then a "still on it" line whenever the full agent leaves the caller in
336
+ // silence, until its answer arrives. Never over the caller, never after
337
+ // the answer, never once a newer utterance took over.
338
+ _quickTurn(session, turn, text) {
339
+ const quick = session._quick;
340
+ const timing = this.quickTiming;
341
+ const live = () => !turn.muted && !turn.done && !turn.controller.signal.aborted &&
342
+ !session.closed && this.session === session && session._voiceTurn === turn && session._quick === quick;
343
+ const sayLine = async (message, label, { maxMs = Infinity, unlessSpoke = false } = {}) => {
344
+ const asked = Date.now();
345
+ const { generation, text: reply } = quick.say(message);
346
+ const cancel = () => quick.cancel(generation);
347
+ turn.controller.signal.addEventListener("abort", cancel, { once: true });
348
+ let line = "";
349
+ try { line = quickVoice.spokenLine(await reply); }
350
+ finally { turn.controller.signal.removeEventListener("abort", cancel); }
351
+ const ready = Date.now() - asked;
352
+ if (!line || !live() || ready > maxMs || (unlessSpoke && turn.spoke) || session.callerTalking?.()) {
353
+ qlog(`${label} skipped ready=${ready}ms`);
354
+ return false;
355
+ }
356
+ qlog(`${label} ready=${ready}ms chars=${line.length}`);
357
+ const spoken = await session.speak(line);
358
+ if (spoken === false || !live()) return false;
359
+ this._rememberSpeech(session, "assistant", line);
360
+ return true;
361
+ };
362
+ (async () => {
363
+ await sayLine(quickVoice.utteranceMessage(text), "line", { maxMs: timing.lineMaxMs, unlessSpoke: true });
364
+ let fillers = 0;
365
+ let lastFillerAt = Date.now();
366
+ while (fillers < timing.fillerMax && live()) {
367
+ await new Promise((resolve) => { const timer = setTimeout(resolve, timing.tickMs); timer.unref?.(); });
368
+ if (!live()) break;
369
+ if (session.speechBusy?.() || session.callerTalking?.()) continue;
370
+ const quietSince = Math.max(turn.startedAt, lastFillerAt, session._lastSpokeAt || 0);
371
+ if (Date.now() - quietSince < timing.fillerQuietMs) continue;
372
+ fillers += 1;
373
+ lastFillerAt = Date.now();
374
+ await sayLine(quickVoice.fillerMessage(text, (Date.now() - turn.startedAt) / 1000), `filler ${fillers}`);
375
+ lastFillerAt = Date.now();
376
+ }
377
+ })().catch((error) => qlog(`turn failed: ${error.message}`));
378
+ }
379
+
250
380
  _rememberSpeech(session, role, text) {
251
381
  const history = session._voiceHistory ||= [];
252
382
  if (role === "assistant" && history.at(-1)?.role === "assistant") {
@@ -262,7 +392,7 @@ class KazeeCallManager {
262
392
  const speaker = String(meta.display || "").trim();
263
393
  const attributed = (meta.participants || 0) > 1 && speaker ? `${speaker}: ${text}` : text;
264
394
  session._voiceTurn?.controller.abort("superseded");
265
- const turn = { controller: new AbortController(), muted: false };
395
+ const turn = { controller: new AbortController(), muted: false, done: false, spoke: false, startedAt: Date.now() };
266
396
  session._voiceTurn = turn;
267
397
  const callAdapter = Object.create(this.adapter);
268
398
  callAdapter.callHistory = (session._voiceHistory || []).map(entry => ({ ...entry }));
@@ -277,6 +407,8 @@ class KazeeCallManager {
277
407
  callAdapter.suppressChatVisibility = true;
278
408
  // Prompt builder switches to phone-call brevity when it sees this.
279
409
  callAdapter.liveCall = true;
410
+ // The quick voice says the first line; the prompt then skips its own.
411
+ callAdapter.quickVoice = !!session._quick?.ready();
280
412
  const spokenSentences = [];
281
413
  callAdapter.isCallTurnActive = () => !turn.muted && !turn.controller.signal.aborted &&
282
414
  !session.closed && this.session === session && session._voiceTurn === turn;
@@ -287,6 +419,7 @@ class KazeeCallManager {
287
419
  if (!callAdapter.isCallTurnActive()) return false;
288
420
  const spoken = await session.speak(response);
289
421
  if (spoken !== false && callAdapter.isCallTurnActive()) {
422
+ turn.spoke = true;
290
423
  this._rememberSpeech(session, "assistant", response);
291
424
  return true;
292
425
  }
@@ -298,6 +431,7 @@ class KazeeCallManager {
298
431
  return spoken;
299
432
  };
300
433
  callAdapter.send = async (_channelId, response) => {
434
+ turn.done = true;
301
435
  const remainder = require("../../../core/call-speech").unspokenSuffix(response, spokenSentences);
302
436
  if (remainder) await speak(remainder);
303
437
  return true;
@@ -308,6 +442,7 @@ class KazeeCallManager {
308
442
  console.warn(`Kazee call ignored file voice response ${voicePath}; streamed TTS is active`);
309
443
  return true;
310
444
  };
445
+ if (callAdapter.quickVoice) this._quickTurn(session, turn, attributed);
311
446
  this.adapter._emit("message", {
312
447
  adapter: callAdapter,
313
448
  channelId: String(incoming.chatId),
@@ -0,0 +1,295 @@
1
+ "use strict";
2
+
3
+ // The quick voice: a small, kept-warm model that says a short, fitting line
4
+ // the moment the caller finishes ("Sure, let me check the call logs."), and a
5
+ // brief "still on it" line while the full agent works. It never answers: the
6
+ // full agent's call turn still runs for every utterance and speaks the real
7
+ // reply. One process per call, spawned through the provider's own utility
8
+ // invocation (binary, env and credentials come from the provider, never from
9
+ // here). Ported from the 5 Oct front-voice test build, reduced to this job.
10
+
11
+ const { spawn } = require("child_process");
12
+ const { StringDecoder } = require("string_decoder");
13
+ const { killProcessTree } = require("../../../core/process-tree");
14
+ const { redactSensitive } = require("../../../core/redact");
15
+
16
+ const KILL_GRACE_MS = 2000;
17
+ const MAX_BUFFER_BYTES = 1024 * 1024;
18
+ const MAX_LINE_CHARS = 120;
19
+ const EFFORTS = new Set(["low", "medium", "high", "xhigh", "max"]);
20
+
21
+ // The bot's .env is parsed into the config object, not process.env: read the
22
+ // config first, the environment second.
23
+ function setting(name) {
24
+ let value = process.env[name];
25
+ try { value = require("../../../core/config").config?.[name] ?? value; } catch (_) {}
26
+ return String(value ?? "").trim();
27
+ }
28
+
29
+ // Measured 11 Oct, warm process, thinking off, 23 call lines with traps
30
+ // ("Did you hear me?", "Are you sure?"): Haiku 4.5 finished its line in
31
+ // 0.8-1.1 s (Sonnet 5 at low effort 1.7-1.9 s); both kept every rule once the
32
+ // prompt named the traps (a looser prompt let Haiku answer "you said it once").
33
+ function quickVoiceSettings() {
34
+ const effort = setting("KAZEE_CALL_QUICK_EFFORT").toLowerCase() || "low";
35
+ return {
36
+ enabled: /^(?:on|1|true|yes)$/i.test(setting("KAZEE_CALL_QUICK_VOICE")),
37
+ model: setting("KAZEE_CALL_QUICK_MODEL") || "claude-haiku-4-5",
38
+ effort: EFFORTS.has(effort) ? effort : "low",
39
+ thinking: /^(?:on|1|true|yes)$/i.test(setting("KAZEE_CALL_QUICK_THINKING")),
40
+ };
41
+ }
42
+
43
+ function ownerDisplayName() {
44
+ try {
45
+ const name = String(require("../../../core/people").owners()?.[0]?.name || "").trim();
46
+ return name && name.toLowerCase() !== "owner" ? name : "";
47
+ } catch (_) {
48
+ return "";
49
+ }
50
+ }
51
+
52
+ // Caller words and spoken history only: no file paths, links, ids or secrets
53
+ // reach the quick voice.
54
+ function scrubLine(text) {
55
+ return redactSensitive(String(text || ""))
56
+ .replace(/https?:\/\/\S+/gi, "a link")
57
+ .replace(/(?:~|\.{1,2})?\/[\w.@%+-]+(?:\/[\w.@%+-]*)+/g, "a file")
58
+ .replace(/\b(?=[A-Za-z0-9_-]*\d)[A-Za-z0-9_-]{16,}\b/g, "an id")
59
+ .replace(/\[REDACTED[^\]]*\]/g, "something private")
60
+ .replace(/[`*_#>|[\]]/g, " ")
61
+ .replace(/\s+/g, " ")
62
+ .trim();
63
+ }
64
+
65
+ function buildQuickPrompt({ ownerName = "" } = {}) {
66
+ const owner = scrubLine(ownerName).slice(0, 60) || "the caller";
67
+ return [
68
+ `You are the quick voice of Claudia, ${owner}'s assistant, on a live phone call with ${owner}.`,
69
+ "Claudia herself (the full assistant, with tools) answers every request a few seconds after you. Your only job is to fill the first moment, so the caller knows they were heard.",
70
+ "",
71
+ "Reply with ONE short spoken line, 2 to 8 words, that fits what was just said:",
72
+ "- a greeting or a check like \"Hello?\" or \"Are you there?\": a warm, brief reply, like \"Yep, I'm here.\"",
73
+ "- a question: \"Good question, one sec.\", or a line naming the topic, like \"Let me look at the budget.\"",
74
+ "- a request: \"Sure, let me check the call logs.\", naming the thing in two or three words.",
75
+ "- thanks or small talk: a short natural reply.",
76
+ "- a correction or \"that's wrong\": \"Okay, let me look again.\"",
77
+ "Never answer, even when you think you know: \"Did you hear me?\" gets \"Let me check.\", never \"Yes, once.\"; \"Are you sure?\" gets \"Let me double-check.\".",
78
+ "Never give facts, numbers, yes or no, opinions or apologies; never say anything was done, found, sent or changed. For sending, changing, booking or deleting anything, say \"Sure, one sec.\" and never that it is being done.",
79
+ "No lists, emoji, symbols or markdown.",
80
+ "",
81
+ "A message that starts with [Still working] means Claudia is still on it: say a short, varied line of 3 to 7 words, like \"Still on it, nearly there.\" Never repeat an earlier line word for word.",
82
+ ].join("\n");
83
+ }
84
+
85
+ // What the quick voice should say for one utterance, or a filler while the
86
+ // full agent is still working.
87
+ function utteranceMessage(text) {
88
+ return scrubLine(text).slice(0, 600);
89
+ }
90
+
91
+ function fillerMessage(text, seconds) {
92
+ return `[Still working] on "${scrubLine(text).slice(0, 200)}", ${Math.round(seconds)} seconds in.`;
93
+ }
94
+
95
+ // A spoken line from model text: first line only, no control or markup,
96
+ // bounded. Empty means "say nothing".
97
+ function spokenLine(text) {
98
+ const line = String(text || "").split("\n").map((part) => part.trim()).find(Boolean) || "";
99
+ const clean = scrubLine(line).replace(/^["']|["']$/g, "").trim();
100
+ if (!clean || !/[A-Za-z]/.test(clean)) return "";
101
+ return clean.length > MAX_LINE_CHARS ? `${clean.slice(0, MAX_LINE_CHARS - 1)}…` : clean;
102
+ }
103
+
104
+ class QuickVoiceProcess {
105
+ constructor({ provider, settings = quickVoiceSettings(), systemPrompt, cwd = process.cwd(), spawnImpl = spawn, maxRestarts = 1, log = () => {} } = {}) {
106
+ this.provider = provider;
107
+ this.settings = settings;
108
+ this.systemPrompt = systemPrompt;
109
+ this.cwd = cwd;
110
+ this.spawnImpl = spawnImpl;
111
+ this.maxRestarts = maxRestarts;
112
+ this.log = log;
113
+ this.proc = null;
114
+ this.restarts = 0;
115
+ this.stopped = false;
116
+ this.queue = []; // generations awaiting their result, oldest first
117
+ this._seq = 0;
118
+ this._stopPromise = null;
119
+ }
120
+
121
+ static supported(provider) {
122
+ return provider?.id === "claude" && provider.capabilities?.persistentUtilityStream?.support === "native" &&
123
+ typeof provider.buildUtilityInvocation === "function";
124
+ }
125
+
126
+ invocation() {
127
+ const { model, effort, thinking } = this.settings;
128
+ const invocation = this.provider.buildUtilityInvocation({
129
+ purpose: "call-quick-voice",
130
+ tier: "low",
131
+ model,
132
+ effort,
133
+ coreInstructions: this.systemPrompt,
134
+ dynamicContext: "",
135
+ userPrompt: "",
136
+ readOnly: true,
137
+ fresh: true,
138
+ });
139
+ const args = [...invocation.args];
140
+ // The slim prompt replaces the CLI's default system prompt (no tool docs
141
+ // to read), which is most of what makes a warm reply start in about a second.
142
+ const appendAt = args.indexOf("--append-system-prompt");
143
+ if (appendAt >= 0) args[appendAt] = "--system-prompt";
144
+ const printAt = args.indexOf("-p");
145
+ args.splice(printAt >= 0 ? printAt + 1 : 0, 0,
146
+ "--input-format", "stream-json", "--include-partial-messages",
147
+ "--tools", "", "--strict-mcp-config", "--setting-sources", "");
148
+ if (!args.includes("--no-session-persistence")) args.push("--no-session-persistence");
149
+ const env = { ...invocation.env, OC_PROVIDER: this.provider.id, OC_UTILITY_PURPOSE: "call-quick-voice" };
150
+ // Thinking off: Haiku 4.5 went from 2-3 s to 0.7 s to its first word.
151
+ if (!thinking) env.MAX_THINKING_TOKENS = "0";
152
+ return { binary: invocation.binary, args, env };
153
+ }
154
+
155
+ start() {
156
+ if (this.stopped || this.proc) return this.proc;
157
+ const { binary, args, env } = this.invocation();
158
+ const proc = this.spawnImpl(binary, args, {
159
+ cwd: this.cwd, env, stdio: ["pipe", "pipe", "pipe"], detached: process.platform !== "win32", shell: false,
160
+ });
161
+ this.proc = proc;
162
+ const decoder = new StringDecoder("utf8");
163
+ let buffer = "";
164
+ proc.stdout.on("data", (data) => {
165
+ if (this.proc !== proc) return;
166
+ buffer += decoder.write(data);
167
+ if (buffer.length > MAX_BUFFER_BYTES) { buffer = ""; this._crashed(proc, "buffer"); return; }
168
+ let newline;
169
+ while ((newline = buffer.indexOf("\n")) >= 0) {
170
+ const line = buffer.slice(0, newline).trim();
171
+ buffer = buffer.slice(newline + 1);
172
+ if (!line) continue;
173
+ let event;
174
+ try { event = JSON.parse(line); } catch (_) { continue; }
175
+ this._onEvent(event);
176
+ }
177
+ });
178
+ proc.stderr.on("data", () => {});
179
+ proc.stdin.on("error", () => {});
180
+ proc.on("error", () => this._crashed(proc, "spawn-error"));
181
+ proc.on("close", () => this._crashed(proc, "exited"));
182
+ this.log(`started pid=${proc.pid ?? "?"} model=${this.settings.model} effort=${this.settings.effort} thinking=${this.settings.thinking ? "on" : "off"}`);
183
+ return proc;
184
+ }
185
+
186
+ ready() {
187
+ const proc = this.proc;
188
+ return !this.stopped && !!proc && proc.exitCode === null && proc.signalCode === null && !proc.stdin.destroyed;
189
+ }
190
+
191
+ // One user message; resolves with the reply text as soon as its text block
192
+ // closes (before the CLI's end-of-turn work), or "" when cancelled/failed.
193
+ say(content) {
194
+ if (!this.ready()) return { generation: null, text: Promise.resolve("") };
195
+ let resolve;
196
+ const text = new Promise((done) => { resolve = done; });
197
+ const generation = { id: ++this._seq, text: "", cancelled: false, settled: false, resolve };
198
+ this.queue.push(generation);
199
+ try {
200
+ this.proc.stdin.write(`${JSON.stringify({ type: "user", message: { role: "user", content: String(content) } })}\n`);
201
+ } catch (_) {
202
+ this.queue = this.queue.filter((item) => item !== generation);
203
+ resolve("");
204
+ }
205
+ return { generation, text };
206
+ }
207
+
208
+ // Stops the in-flight generation (Claude Code's stream-json control
209
+ // protocol); the CLI then closes that turn with an error result.
210
+ cancel(generation) {
211
+ if (!generation || generation.cancelled) return;
212
+ generation.cancelled = true;
213
+ generation.resolve("");
214
+ if (generation.settled || this.queue[0] !== generation || !this.ready()) return;
215
+ try {
216
+ this.proc.stdin.write(`${JSON.stringify({ type: "control_request", request_id: `quick-interrupt-${generation.id}`, request: { subtype: "interrupt" } })}\n`);
217
+ } catch (_) {}
218
+ }
219
+
220
+ _onEvent(event) {
221
+ const generation = this.queue[0];
222
+ if (!generation) return;
223
+ if (event.type === "stream_event") {
224
+ const inner = event.event || {};
225
+ if (inner.type === "content_block_delta" && inner.delta?.type === "text_delta" && typeof inner.delta.text === "string") {
226
+ generation.text += inner.delta.text;
227
+ } else if (inner.type === "content_block_stop" && generation.text.trim()) {
228
+ generation.resolve(generation.cancelled ? "" : generation.text);
229
+ }
230
+ } else if (event.type === "assistant" && !generation.text && Array.isArray(event.message?.content)) {
231
+ // A CLI without partial messages: the whole reply at once.
232
+ generation.text = event.message.content.filter((block) => block?.type === "text").map((block) => block.text || "").join("");
233
+ } else if (event.type === "result") {
234
+ this.queue.shift();
235
+ generation.settled = true;
236
+ generation.resolve(generation.cancelled || event.is_error ? "" : generation.text);
237
+ }
238
+ }
239
+
240
+ _crashed(proc, reason) {
241
+ if (this.proc !== proc) return;
242
+ this.proc = null;
243
+ for (const generation of this.queue.splice(0)) { generation.settled = true; generation.resolve(""); }
244
+ try { killProcessTree(proc.pid, "SIGKILL"); } catch (_) {}
245
+ if (this.stopped) return;
246
+ if (this.restarts >= this.maxRestarts) {
247
+ this.log(`gave up reason=${reason} restarts=${this.restarts}`);
248
+ return;
249
+ }
250
+ this.restarts += 1;
251
+ this.log(`restart reason=${reason} restarts=${this.restarts}`);
252
+ try { this.start(); } catch (error) { this.log(`restart failed: ${error.message}`); }
253
+ }
254
+
255
+ stop() {
256
+ if (this._stopPromise) return this._stopPromise;
257
+ this.stopped = true;
258
+ live.delete(this);
259
+ const proc = this.proc;
260
+ this.proc = null;
261
+ for (const generation of this.queue.splice(0)) { generation.settled = true; generation.cancelled = true; generation.resolve(""); }
262
+ if (!proc) return (this._stopPromise = Promise.resolve());
263
+ this._stopPromise = new Promise((resolve) => {
264
+ let killTimer = null;
265
+ const finish = () => { if (killTimer) clearTimeout(killTimer); resolve(); };
266
+ if (proc.exitCode !== null || proc.signalCode !== null) return finish();
267
+ proc.once("close", finish);
268
+ try { proc.stdin.end(); } catch (_) {}
269
+ try { killProcessTree(proc.pid, "SIGTERM"); } catch (_) { try { proc.kill("SIGTERM"); } catch (_) {} }
270
+ killTimer = setTimeout(() => {
271
+ try { killProcessTree(proc.pid, "SIGKILL"); } catch (_) { try { proc.kill("SIGKILL"); } catch (_) {} }
272
+ }, KILL_GRACE_MS);
273
+ killTimer.unref?.();
274
+ });
275
+ return this._stopPromise;
276
+ }
277
+ }
278
+
279
+ // A bot exit never leaves a quick-voice process behind.
280
+ const live = new Set();
281
+ function track(quick) {
282
+ live.add(quick);
283
+ return quick;
284
+ }
285
+ process.on("exit", () => {
286
+ for (const quick of live) {
287
+ const pid = quick.proc?.pid;
288
+ if (pid) { try { killProcessTree(pid, "SIGKILL"); } catch (_) {} }
289
+ }
290
+ });
291
+
292
+ module.exports = {
293
+ QuickVoiceProcess, quickVoiceSettings, buildQuickPrompt, utteranceMessage, fillerMessage, spokenLine,
294
+ scrubLine, ownerDisplayName, track,
295
+ };
@@ -54,7 +54,7 @@ const ACK_DELAY_MS = 2800;
54
54
  class KazeeCallSession {
55
55
  constructor({ roomId, chatId, callerId, botUserId, janusUrl, janusToken, janusTokenExpiresAt, roomToken, roomTokenExpiresAt, managedRoom = false, refreshRoomAccess, iceServers = [], iceCredentialsExpiresAt, onTranscript, onEnded, onMediaReady,
56
56
  maxCallMs, idleMs, finalWaitMs, synthesize = synthesizePcm, synthesizeStream = null, asCaller = false, greeting = null,
57
- turnDecision = detectTurn, acknowledgement = selectAcknowledgement, onSpeechStart,
57
+ turnDecision = detectTurn, acknowledgement = selectAcknowledgement, onSpeechStart, onSpeechNoise,
58
58
  JanusClientImpl = JanusClient, PeerConnectionImpl = RTCPeerConnection, SpeechStreamImpl = SpeechStream } = {}) {
59
59
  this.roomId = Number(roomId);
60
60
  this.chatId = String(chatId);
@@ -83,6 +83,7 @@ class KazeeCallSession {
83
83
  this._transportExpiryTimer = null;
84
84
  this.onTranscript = onTranscript || (() => {});
85
85
  this.onSpeechStart = onSpeechStart || (() => {});
86
+ this.onSpeechNoise = onSpeechNoise || (() => {});
86
87
  this.turnDecision = turnDecision;
87
88
  this.acknowledgement = acknowledgement;
88
89
  this._cueController = null;
@@ -585,6 +586,7 @@ class KazeeCallSession {
585
586
  if (countWords(trimmed) < MIN_DISPATCH_WORDS) {
586
587
  clog(`Kazee call noise turn dropped: feed=${feedId} words=${countWords(trimmed)}`);
587
588
  if (!this._speakActive && !this._speakQueue.length) this._stopThinking();
589
+ this._speechWasNoise(feedId);
588
590
  return;
589
591
  }
590
592
  }
@@ -593,7 +595,7 @@ class KazeeCallSession {
593
595
  // Empty finalization (noise commit): nothing dispatches, no reply is
594
596
  // coming — don't tick into the void.
595
597
  else if (!this._speakActive && !this._speakQueue.length) this._stopThinking();
596
- if (!trimmed) return;
598
+ if (!trimmed) { this._speechWasNoise(feedId); return; }
597
599
  this.onTranscript(text, {
598
600
  feedId,
599
601
  display: sub?.display || "",
@@ -602,6 +604,16 @@ class KazeeCallSession {
602
604
  });
603
605
  }
604
606
 
607
+ // Speech start muted the turn being worked on, in case it was a new ask. A
608
+ // sound the recogniser found no words in (wind, footsteps, a cough) was not:
609
+ // unless someone is still talking or another segment is pending, the
610
+ // muted turn gets its voice back.
611
+ _speechWasNoise(feedId) {
612
+ if ([...this.subscribers.values()].some(peer => peer?.vad?.active || peer?.turnGate?.pending ||
613
+ peer?.awaitingWordsSince || peer?.segments?.size)) return;
614
+ this.onSpeechNoise({ feedId });
615
+ }
616
+
605
617
  _reportMediaReady() {
606
618
  if (this.closed || this._reportedMediaReady || !this._sentRtp || !this._receivedRtp) return;
607
619
  this._reportedMediaReady = true;
@@ -751,6 +763,16 @@ class KazeeCallSession {
751
763
  sub.stt.audio(pcm16).catch((error) => console.error("Kazee call STT send:", error.message));
752
764
  }
753
765
 
766
+ // For the quick voice: someone is talking (or a turn is being decided), and
767
+ // speech is playing or queued.
768
+ callerTalking() {
769
+ return [...this.subscribers.values()].some(sub => sub?.vad?.active || sub?.turnGate?.pending || sub?.awaitingWordsSince);
770
+ }
771
+
772
+ speechBusy() {
773
+ return !!(this.speaking || this._speakActive || this._speakQueue.length);
774
+ }
775
+
754
776
  speak(text) {
755
777
  if (this.closed || !String(text || "").trim()) return Promise.resolve(false);
756
778
  return new Promise((resolve, reject) => {