@agfpd/voicetalk-runtime 0.2.1 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/docs/README.md CHANGED
@@ -48,9 +48,19 @@ voice-connect, microphone).
48
48
  2. voice-connect transcribes the utterance (locally, on this host).
49
49
  3. If it opens by addressing a peer, the body is delivered to that peer as an IAP
50
50
  envelope from the owner's identity, framed for the spoken modality.
51
+ 3.5. Fragments are COMPOSED into one turn. Endpointing cuts audio, not thoughts: a
52
+ pause inside a sentence would otherwise become a second envelope and earn a second
53
+ spoken answer. The turn is sent once the owner has been silent for ~3 s since their
54
+ last word — and never while they are still speaking or a fragment is still being
55
+ recognized. A fragment left hanging on a comma or a conjunction waits longer.
51
56
  4. The agent's answer arrives on stdin, is spoken through voice-connect on the host
52
57
  speaker, and the microphone stays open for ~9 s so the owner can continue without
53
- naming the peer again.
58
+ naming the peer again. When that window closes, a soft low tone says so.
59
+
60
+ Inside an open conversation a wake word for ANOTHER peer must look deliberate — the
61
+ name heard letter for letter, followed by a real request — otherwise the fragment goes
62
+ to the peer already listening. Mis-hearing a name should cost a stray line, never the
63
+ whole conversation.
54
64
 
55
65
  Earcons mark the states: rising = the microphone is yours, short descending = heard
56
66
  you, double descending = nothing was routed. Processing is silent.
@@ -58,7 +68,13 @@ you, double descending = nothing was routed. Processing is silent.
58
68
  Everything is tunable by env: `VOICETALK_AUDIO=0` (text-only, no microphone),
59
69
  `VOICETALK_FOLLOWUP_MS`, `VOICETALK_HANGOVER_MS`, `VOICETALK_VAD_MIN_RMS`,
60
70
  `VOICETALK_VAD_RATIO`, `VOICETALK_STT_LANG`, `VOICETALK_STT_ENGINE`,
61
- `VOICETALK_TTS_MAX_CHARS`, `VOICETALK_EARCONS=0`.
71
+ `VOICETALK_TTS_MAX_CHARS`, `VOICETALK_EARCONS=0`, `VOICETALK_COMPOSE_MS`,
72
+ `VOICETALK_COMPOSE_UNFINISHED_MS`, `VOICETALK_TURN_MAX_MS`.
73
+
74
+ Two seams exist for testing turn-taking without touching a live channel:
75
+ `VOICETALK_MIC_FILE=<pcm>` replays a recording through the microphone path (paced in
76
+ real time) and `VOICETALK_PLAYBACK=0` keeps the speaker silent while synthesis is
77
+ still performed and timed. `scripts/replay-dialogue.sh` wires both into a sandbox.
62
78
 
63
79
  ## Requirements
64
80
 
package/docs/ru/README.md CHANGED
@@ -48,8 +48,18 @@ self-config-хука (`VOICETALK_WAKE_WORDS`, `VOICETALK_DEFAULT_TARGET`,
48
48
  2. voice-connect распознаёт реплику (локально, на этом хосте).
49
49
  3. Если реплика начинается с обращения к пиру, тело уходит ему IAP-конвертом от
50
50
  личности владельца, с рамкой голосовой модальности.
51
+ 3.5. Фрагменты СОБИРАЮТСЯ в один ход. Эндпойнтер режет звук, а не мысль: пауза внутри
52
+ фразы иначе стала бы вторым конвертом и заработала бы второй голосовой ответ. Ход
53
+ уходит, когда владелец молчит ~3 с после последнего слова, и никогда — пока он
54
+ ещё говорит или пока фрагмент ещё распознаётся. Фрагмент, оборванный на запятой
55
+ или союзе, ждёт дольше.
51
56
  4. Ответ агента приходит на stdin, озвучивается через voice-connect в динамик, и
52
- микрофон остаётся открытым ~9 с — продолжить можно без повторного обращения.
57
+ микрофон остаётся открытым ~9 с — продолжить можно без повторного обращения. Когда
58
+ окно закрывается, звучит тихий низкий тон.
59
+
60
+ Внутри открытого диалога wake-слово ДРУГОГО пира обязано выглядеть намеренным — имя
61
+ услышано буква в букву и за ним идёт настоящая просьба, — иначе фрагмент уходит тому,
62
+ кто уже слушает. Ослышка должна стоить лишней строки, а не всего разговора.
53
63
 
54
64
  Earcon'ы размечают состояния: восходящий — микрофон ваш, короткий нисходящий —
55
65
  принял, двойной нисходящий — не отправлено. Обработка идёт молча.
@@ -57,7 +67,13 @@ Earcon'ы размечают состояния: восходящий — мик
57
67
  Всё настраивается через env: `VOICETALK_AUDIO=0` (текстовый режим, без микрофона),
58
68
  `VOICETALK_FOLLOWUP_MS`, `VOICETALK_HANGOVER_MS`, `VOICETALK_VAD_MIN_RMS`,
59
69
  `VOICETALK_VAD_RATIO`, `VOICETALK_STT_LANG`, `VOICETALK_STT_ENGINE`,
60
- `VOICETALK_TTS_MAX_CHARS`, `VOICETALK_EARCONS=0`.
70
+ `VOICETALK_TTS_MAX_CHARS`, `VOICETALK_EARCONS=0`, `VOICETALK_COMPOSE_MS`,
71
+ `VOICETALK_COMPOSE_UNFINISHED_MS`, `VOICETALK_TURN_MAX_MS`.
72
+
73
+ Для проверки turn-taking, не трогая живой канал, есть два шва: `VOICETALK_MIC_FILE=<pcm>`
74
+ проигрывает запись через тот же путь, что и микрофон (в реальном темпе), а
75
+ `VOICETALK_PLAYBACK=0` держит динамик молча, оставляя синтез и его замеры. Оба сведены
76
+ в `scripts/replay-dialogue.sh`.
61
77
 
62
78
  ## Требования
63
79
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@agfpd/voicetalk-runtime",
3
- "version": "0.2.1",
3
+ "version": "0.3.0",
4
4
  "description": "Voice runtime-router for IAP peers.",
5
5
  "license": "Apache-2.0",
6
6
  "type": "module",
package/src/audio.ts CHANGED
@@ -9,7 +9,7 @@
9
9
 
10
10
  import { spawn, type ChildProcessByStdio } from 'child_process'
11
11
  import type { Readable } from 'stream'
12
- import { writeFileSync } from 'fs'
12
+ import { readFileSync, writeFileSync } from 'fs'
13
13
 
14
14
  export const SAMPLE_RATE = 16_000
15
15
  export const FRAME_MS = 20
@@ -125,6 +125,13 @@ export class Segmenter {
125
125
  this.warmupFrames = Math.round((opts.warmupMs ?? 600) / FRAME_MS)
126
126
  }
127
127
 
128
+ /** Is the endpointer inside an utterance right now? The turn layer asks before it
129
+ * decides a thought is finished: the owner resuming mid-pause is the difference
130
+ * between one envelope and two. */
131
+ get speaking(): boolean {
132
+ return this.inSpeech
133
+ }
134
+
128
135
  get noiseFloor(): number {
129
136
  return this.floor < 0 ? 0 : this.floor
130
137
  }
@@ -293,6 +300,11 @@ export async function resolveInputDevice(
293
300
 
294
301
  export type MicOptions = {
295
302
  deviceIndex: number
303
+ /** Replay a recording INSTEAD of opening the microphone (test seam). Frames are paced
304
+ * in real time, so endpointing and the compose window behave exactly as they do on a
305
+ * live mic — which is the whole point: turn-taking bugs only reproduce against a
306
+ * clock. Accepts a WAV or headerless mono 16-bit 16 kHz PCM file. */
307
+ filePath?: string
296
308
  ffmpegBin?: string
297
309
  onData: (chunk: Buffer) => void
298
310
  onError?: (detail: string) => void
@@ -333,6 +345,10 @@ export class Microphone {
333
345
  private spawnChild(): void {
334
346
  if (this.stopped) return
335
347
  this.attempt += 1
348
+ if (this.opts.filePath) {
349
+ this.replayFile(this.opts.filePath)
350
+ return
351
+ }
336
352
  const bin = this.opts.ffmpegBin ?? 'ffmpeg'
337
353
  const args = [
338
354
  '-hide_banner',
@@ -368,6 +384,35 @@ export class Microphone {
368
384
  })
369
385
  }
370
386
 
387
+ /** Feed a recording through the same path the microphone uses, one 20 ms frame per
388
+ * 20 ms, then fall silent (a finished recording is a quiet room, not a dead mic). */
389
+ private replayFile(path: string): void {
390
+ let pcm: Buffer
391
+ try {
392
+ const raw = readFileSync(path)
393
+ // Skip a RIFF header if present; anything else is treated as raw PCM.
394
+ pcm = raw.subarray(0, 4).toString() === 'RIFF' ? raw.subarray(44) : raw
395
+ } catch (err) {
396
+ this.opts.onError?.(`cannot replay ${path}: ${err instanceof Error ? err.message : String(err)}`)
397
+ return
398
+ }
399
+ this.opts.onStart?.(this.attempt)
400
+ let offset = 0
401
+ const tick = () => {
402
+ if (this.stopped) return
403
+ if (offset >= pcm.length) {
404
+ // Keep the stream alive with silence so compose/window timers still fire.
405
+ this.opts.onData(Buffer.alloc(FRAME_BYTES))
406
+ this.timer = setTimeout(tick, FRAME_MS)
407
+ return
408
+ }
409
+ this.opts.onData(Buffer.from(pcm.subarray(offset, offset + FRAME_BYTES)))
410
+ offset += FRAME_BYTES
411
+ this.timer = setTimeout(tick, FRAME_MS)
412
+ }
413
+ tick()
414
+ }
415
+
371
416
  private scheduleRestart(): void {
372
417
  if (this.stopped || this.timer) return
373
418
  const delay = Math.min(30_000, 1_000 * Math.min(8, this.attempt))
@@ -1,9 +1,17 @@
1
1
  // voiceSession.ts — the live voice session (Фаза 3): the runtime's ears, its turn-taking
2
2
  // and its mouth, wired between audio.ts (host mechanics) and voiceConnect.ts (speech).
3
3
  //
4
- // INBOUND mic → endpointing → STT → address match → deliver to the fleet
4
+ // INBOUND mic → endpointing → STT → address match → COMPOSE → deliver to the fleet
5
5
  // OUTBOUND agent envelope → TTS → speaker, then an open-mic follow-up window
6
6
  //
7
+ // A pause inside a sentence is not the end of a thought. Endpointing (fast, 700 ms) only
8
+ // cuts AUDIO into segments so recognition can start early; what the fleet receives is a
9
+ // TURN — every segment of one thought, joined, delivered ONCE when the owner has actually
10
+ // finished (the compose window). Without that, one spoken thought became two envelopes
11
+ // and earned two spoken answers, the second one replying to a fragment (live complaint,
12
+ // 22.07). Recognition still runs per segment and in the background, so composing costs
13
+ // only the silence the owner was taking anyway.
14
+ //
7
15
  // Turn-taking follows the established voice-assistant pattern recorded in Описание
8
16
  // voicetalk: the address (wake word) only INITIATES a conversation; after an agent has
9
17
  // answered, a short open-mic window lets the owner continue without naming the peer
@@ -39,6 +47,7 @@ import {
39
47
  buildAddressBook,
40
48
  isLikelyHallucination,
41
49
  matchWake,
50
+ objectionToSwitch,
42
51
  readFleetPeers,
43
52
  sttPrompt,
44
53
  type Address,
@@ -57,6 +66,24 @@ const FOLLOWUP_MS = num(process.env.VOICETALK_FOLLOWUP_MS, 9_000)
57
66
  * 500–800 ms for free dialogue; the upper half of the band suits a heavier recognizer —
58
67
  * it fragments one thought into two transcriptions far less often. */
59
68
  const HANGOVER_MS = num(process.env.VOICETALK_HANGOVER_MS, 700)
69
+ /** Silence (counted from the owner's last WORD) after which a turn is considered
70
+ * finished and sent. The two errors are not symmetric: waiting one second too long costs
71
+ * one second on a chain where synthesis alone is ~5 s, while not waiting costs a split
72
+ * thought, two envelopes and a spoken answer to a fragment — the owner's actual
73
+ * complaint. Measured thinking pauses in real Russian speech run to ~2.7 s, so the
74
+ * budget sits above that with room. Tunable live: VOICETALK_COMPOSE_MS. */
75
+ const COMPOSE_MS = num(process.env.VOICETALK_COMPOSE_MS, 3_000)
76
+ /** A fragment that ENDS mid-thought (no terminal punctuation, or a trailing comma or
77
+ * conjunction) buys extra patience — the speaker audibly has not finished. */
78
+ const COMPOSE_UNFINISHED_MS = num(process.env.VOICETALK_COMPOSE_UNFINISHED_MS, 4_500)
79
+ /** Segments waiting for recognition. Deep enough for a long sentence spoken over a
80
+ * slow recognizer, shallow enough that a wedged STT cannot grow it without bound. */
81
+ const SEGMENT_QUEUE_MAX = 8
82
+ /** A turn never grows past this — a monologue still reaches its addressee. */
83
+ const TURN_MAX_MS = num(process.env.VOICETALK_TURN_MAX_MS, 45_000)
84
+ /** How often to re-check a turn that is ready to send but whose speaker has not gone
85
+ * quiet yet. */
86
+ const HOLD_RECHECK_MS = num(process.env.VOICETALK_HOLD_RECHECK_MS, 300)
60
87
  /** Gate the mic for a moment after the speaker stops — room echo tail. */
61
88
  const ECHO_TAIL_MS = num(process.env.VOICETALK_ECHO_TAIL_MS, 400)
62
89
  /** Long answers are truncated for speech: a spoken monologue is worse than a summary. */
@@ -65,6 +92,12 @@ const TTS_MAX_CHARS = num(process.env.VOICETALK_TTS_MAX_CHARS, 1_200)
65
92
  const STT_LANG = process.env.VOICETALK_STT_LANG ?? 'ru'
66
93
  /** Force a local recognizer (mlx-whisper) to keep always-on listening on-device. */
67
94
  const STT_ENGINE = process.env.VOICETALK_STT_ENGINE?.trim() || undefined
95
+ /** Test seams. VOICETALK_MIC_FILE replays a recording instead of opening the microphone;
96
+ * VOICETALK_PLAYBACK=0 keeps the speaker silent (synthesis still runs and is still
97
+ * timed). Together they let a turn-taking change be verified end to end on a host whose
98
+ * real voice channel is in use by its owner. */
99
+ const MIC_FILE = process.env.VOICETALK_MIC_FILE?.trim() || undefined
100
+ const PLAYBACK_ENABLED = process.env.VOICETALK_PLAYBACK !== '0'
68
101
 
69
102
  export type DeliverFn = (target: string, text: string) => Promise<{ ok: boolean; detail?: string }>
70
103
  export type LogFn = (evt: string, fields: Record<string, unknown>) => void
@@ -80,9 +113,10 @@ export type VoiceSessionOptions = {
80
113
  }
81
114
 
82
115
  /** State sounds, per the earcon convention in the research: `open` = the microphone is
83
- * yours (rising, ≤300 ms), `end` = utterance captured, processing (short descending,
84
- * ≤200 ms), `error` = nothing routed (double descending). Processing itself is silent. */
85
- type Earcons = { open: string; end: string; error: string }
116
+ * yours (rising, ≤300 ms), `end` = turn captured, processing (short descending,
117
+ * ≤200 ms), `close` = the open-mic window expired, a name is needed again (low, soft),
118
+ * `error` = nothing routed (double descending). Processing itself is silent. */
119
+ type Earcons = { open: string; end: string; error: string; close: string }
86
120
 
87
121
  export class VoiceSession {
88
122
  private mic: Microphone | null = null
@@ -95,9 +129,18 @@ export class VoiceSession {
95
129
  private prompt = ''
96
130
  private earcons: Earcons | null = null
97
131
  private busy = false
132
+ private queue: Array<Segment & { speechEndedAt: number }> = []
98
133
  private muteUntil = 0
134
+ /** When the microphone last stopped hearing speech — the clock the compose window runs
135
+ * on, so the delivery delay the owner feels is exactly the silence they took. */
136
+ private lastSpeechEndedAt = 0
99
137
  private followTarget: string | null = null
100
138
  private followUntil = 0
139
+ private followTimer: ReturnType<typeof setTimeout> | null = null
140
+ /** The thought being assembled right now — segments of one turn, not yet sent. */
141
+ private pending: { target: string; parts: string[]; openedAt: number; timer: ReturnType<typeof setTimeout> | null } | null = null
142
+ /** When the current turn was handed to the fleet — the clock for «ответ за N с». */
143
+ private deliveredAt = new Map<string, number>()
101
144
  private stopped = false
102
145
 
103
146
  constructor(private readonly opts: VoiceSessionOptions) {}
@@ -109,9 +152,12 @@ export class VoiceSession {
109
152
  mkdirSync(this.opts.tmpDir, { recursive: true, mode: 0o700 })
110
153
  this.refreshAddressBook()
111
154
  this.earcons = this.writeEarcons()
112
- const device = await resolveInputDevice(this.opts.binding.device)
155
+ const device = MIC_FILE
156
+ ? { index: -1, name: `replay:${MIC_FILE}` }
157
+ : await resolveInputDevice(this.opts.binding.device)
113
158
  this.mic = new Microphone({
114
159
  deviceIndex: device.index,
160
+ filePath: MIC_FILE,
115
161
  onData: chunk => this.onAudio(chunk),
116
162
  onStart: attempt => this.opts.log('mic.start', { device: device.index, name: device.name, attempt }),
117
163
  onError: detail => this.opts.log('mic.error', { detail }),
@@ -122,6 +168,9 @@ export class VoiceSession {
122
168
 
123
169
  stop(): void {
124
170
  this.stopped = true
171
+ this.clearWindow()
172
+ if (this.pending?.timer) clearTimeout(this.pending.timer)
173
+ this.pending = null
125
174
  this.mic?.stop()
126
175
  this.mic = null
127
176
  }
@@ -140,6 +189,7 @@ export class VoiceSession {
140
189
  open: join(this.opts.tmpDir, 'earcon-open.wav'),
141
190
  end: join(this.opts.tmpDir, 'earcon-end.wav'),
142
191
  error: join(this.opts.tmpDir, 'earcon-error.wav'),
192
+ close: join(this.opts.tmpDir, 'earcon-close.wav'),
143
193
  }
144
194
  // A5→C6→E6 rising, 240 ms — the microphone is open.
145
195
  writeFileSync(files.open, earconWav([{ hz: 880, ms: 80 }, { hz: 1047, ms: 80 }, { hz: 1319, ms: 80 }]), { mode: 0o600 })
@@ -147,11 +197,15 @@ export class VoiceSession {
147
197
  writeFileSync(files.end, earconWav([{ hz: 880, ms: 80 }, { hz: 660, ms: 80 }]), { mode: 0o600 })
148
198
  // Double descending, 260 ms — tonally "closed": nothing was routed.
149
199
  writeFileSync(files.error, earconWav([{ hz: 520, ms: 120 }, { hz: 330, ms: 140 }]), { mode: 0o600 })
200
+ // One low soft tone, 180 ms — the open-mic window just expired; say a name to speak
201
+ // again. Deliberately quieter and lower than `end` so it reads as "door closing",
202
+ // not as another acknowledgement.
203
+ writeFileSync(files.close, earconWav([{ hz: 400, ms: 180 }], 0.14), { mode: 0o600 })
150
204
  return files
151
205
  }
152
206
 
153
207
  private async earcon(kind: keyof Earcons): Promise<void> {
154
- if (!this.earcons || process.env.VOICETALK_EARCONS === '0') return
208
+ if (!this.earcons || process.env.VOICETALK_EARCONS === '0' || !PLAYBACK_ENABLED) return
155
209
  try {
156
210
  this.muteUntil = Math.max(this.muteUntil, Date.now() + 600)
157
211
  await playFile(this.earcons[kind], { timeoutMs: 5_000 })
@@ -173,16 +227,35 @@ export class VoiceSession {
173
227
  return
174
228
  }
175
229
  for (const segment of this.segmenter.push(chunk)) {
176
- if (this.busy) {
177
- // One utterance in flight at a time: STT is the expensive step and a pile-up
178
- // would only add latency to the turn that matters.
179
- this.opts.log('segment.drop', { reason: 'busy', ms: segment.ms })
230
+ // Recognition is serialized (one STT at a time — a pile-up only adds latency), but
231
+ // segments QUEUE rather than being dropped: a dropped segment is a hole in the
232
+ // middle of the owner's sentence, which is exactly what this phase is fixing.
233
+ if (this.queue.length >= SEGMENT_QUEUE_MAX) {
234
+ this.opts.log('segment.drop', { reason: 'queue full', ms: segment.ms })
180
235
  continue
181
236
  }
182
- this.busy = true
183
- void this.handleSegment(segment).finally(() => {
184
- this.busy = false
185
- })
237
+ // The endpointer hands over a segment only after the hangover silence, so speech
238
+ // itself stopped that much earlier. The compose budget is counted from THAT
239
+ // instant: the owner's pause should not be charged twice, once as endpointing and
240
+ // again as composing.
241
+ this.queue.push({ ...segment, speechEndedAt: Date.now() - HANGOVER_MS })
242
+ void this.drainQueue()
243
+ }
244
+ }
245
+
246
+ private async drainQueue(): Promise<void> {
247
+ if (this.busy) return
248
+ this.busy = true
249
+ try {
250
+ while (this.queue.length) {
251
+ const segment = this.queue.shift()!
252
+ this.lastSpeechEndedAt = segment.speechEndedAt
253
+ await this.handleSegment(segment)
254
+ }
255
+ } finally {
256
+ this.busy = false
257
+ // A turn may have been waiting for recognition to finish before it could be sent.
258
+ if (this.pending) this.rearm()
186
259
  }
187
260
  }
188
261
 
@@ -222,50 +295,163 @@ export class VoiceSession {
222
295
  text: transcript,
223
296
  })
224
297
 
225
- const match = matchWake(transcript, this.book)
226
- const followOpen = this.followTarget && Date.now() < this.followUntil
227
- let target: string | null = null
228
- let body = ''
229
- if (match) {
230
- target = match.target
231
- body = match.body
232
- if (!body) {
233
- // Addressed with nothing to say yet («Борис?») — acknowledge and hold the mic
234
- // open for the actual request, exactly as an assistant beeps and waits.
235
- this.followTarget = target
236
- this.followUntil = Date.now() + FOLLOWUP_MS
237
- this.opts.log('wake.await-body', { target, alias: match.alias })
238
- await this.earcon('open')
239
- return
240
- }
241
- } else if (followOpen) {
242
- target = this.followTarget
243
- body = transcript.trim()
244
- this.opts.log('followup.route', { target })
245
- } else {
246
- this.opts.log('wake.no-match', { text: transcript })
298
+ this.absorb(transcript)
299
+ } finally {
300
+ rmSync(wav, { force: true })
301
+ }
302
+ }
303
+
304
+ /** Route one recognized fragment into the turn being composed.
305
+ *
306
+ * The decision is deliberately conservative in one direction: an unaddressed fragment
307
+ * arriving right after an addressed one CONTINUES that thought rather than being
308
+ * dropped, and a wake word for SOMEONE ELSE has to clear a higher bar than it does
309
+ * when no conversation is open. Mid-dialogue, the cheap error (one more line to the
310
+ * peer already listening) is far cheaper than the expensive one (the conversation
311
+ * silently handed to another agent). */
312
+ private absorb(transcript: string): void {
313
+ const active = this.pending?.target ?? (this.followTarget && Date.now() < this.followUntil ? this.followTarget : null)
314
+ const match = matchWake(transcript, this.book)
315
+ const switching = !!match && !!active && match.target !== active
316
+
317
+ if (switching) {
318
+ const decline = objectionToSwitch(match!)
319
+ if (decline) {
320
+ this.opts.log('wake.switch-declined', { from: active, to: match!.target, reason: decline, text: transcript })
321
+ this.append(active!, transcript.trim())
247
322
  return
248
323
  }
324
+ // A deliberate switch: the thought in flight belongs to the previous peer, so it
325
+ // goes out before the new one starts.
326
+ this.opts.log('wake.switch', { from: active, to: match!.target })
327
+ this.flush('switch')
328
+ this.append(match!.target, match!.body)
329
+ return
330
+ }
249
331
 
250
- // The window is extended for the whole turn: the agent's answer will extend it
251
- // again when it is spoken.
252
- this.followTarget = target
253
- this.followUntil = Date.now() + FOLLOWUP_MS
254
- // Utterance accepted → the "end of listening, now processing" sound. Delivery
255
- // itself stays SILENT (processing earns no earcon) — the agent's own voice is the
256
- // next thing the owner hears.
257
- await this.earcon('end')
258
- const verdict = await this.opts.deliver(target!, body)
259
- if (verdict.ok) {
260
- this.opts.log('voice.delivered', { target, chars: body.length, totalMs: Date.now() - started })
261
- } else {
262
- this.opts.log('voice.undelivered', { target, detail: verdict.detail })
263
- await this.earcon('error')
264
- await this.say(`Не доставлено: ${verdict.detail ?? 'неизвестная ошибка'}`)
332
+ if (match) {
333
+ if (!match.body) {
334
+ // Addressed with nothing to say yet («Борис?») — acknowledge and hold the mic
335
+ // open for the actual request, exactly as an assistant beeps and waits.
336
+ this.openWindow(match.target)
337
+ this.opts.log('wake.await-body', { target: match.target, alias: match.alias })
338
+ void this.earcon('open')
339
+ return
265
340
  }
266
- } finally {
267
- rmSync(wav, { force: true })
341
+ this.append(match.target, match.body)
342
+ return
268
343
  }
344
+
345
+ if (active) {
346
+ this.opts.log('turn.continue', { target: active, reason: this.pending ? 'compose' : 'follow-up' })
347
+ this.append(active, transcript.trim())
348
+ return
349
+ }
350
+
351
+ this.opts.log('wake.no-match', { text: transcript })
352
+ }
353
+
354
+ /** Add a fragment to the turn in flight and (re)arm the compose timer. */
355
+ private append(target: string, text: string): void {
356
+ const body = text.trim()
357
+ if (!body) return
358
+ if (this.pending && this.pending.target !== target) this.flush('retarget')
359
+ if (!this.pending) {
360
+ this.pending = { target, parts: [], openedAt: Date.now(), timer: null }
361
+ this.opts.log('turn.open', { target })
362
+ }
363
+ this.pending.parts.push(body)
364
+ this.clearWindow()
365
+ this.rearm()
366
+ }
367
+
368
+ /** (Re)arm the compose timer for the turn in flight. */
369
+ private rearm(delayMs?: number): void {
370
+ const turn = this.pending
371
+ if (!turn) return
372
+ if (turn.timer) clearTimeout(turn.timer)
373
+ if (Date.now() - turn.openedAt >= TURN_MAX_MS) {
374
+ this.flush('max-length')
375
+ return
376
+ }
377
+ const budget = looksUnfinished(turn.parts[turn.parts.length - 1] ?? '') ? COMPOSE_UNFINISHED_MS : COMPOSE_MS
378
+ const remaining = this.lastSpeechEndedAt + budget - Date.now()
379
+ const delay = Math.max(0, delayMs ?? remaining)
380
+ turn.timer = setTimeout(() => this.flush('compose'), delay)
381
+ }
382
+
383
+ /** Send the composed thought as ONE envelope. */
384
+ private flush(reason: string): void {
385
+ const turn = this.pending
386
+ if (!turn) return
387
+ // A thought is NOT finished while the owner is still speaking it. The compose timer
388
+ // counts silence since the last RECOGNIZED fragment, and recognition lags the voice
389
+ // by an endpointing pause plus the recognizer's own time — so by the time it fires,
390
+ // the owner may already be several words into the rest of the sentence. Waiting for
391
+ // the microphone and the recognizer to go quiet is what turns those words into the
392
+ // same envelope instead of a second one. (Live defect, 22.07: two envelopes, two
393
+ // spoken answers, the second one replying to a fragment.)
394
+ if (reason === 'compose' && (this.segmenter.speaking || this.busy || this.queue.length)) {
395
+ this.opts.log('turn.hold', {
396
+ target: turn.target,
397
+ reason: this.segmenter.speaking ? 'still speaking' : 'recognition in flight',
398
+ })
399
+ this.rearm(HOLD_RECHECK_MS)
400
+ return
401
+ }
402
+ this.pending = null
403
+ if (turn.timer) clearTimeout(turn.timer)
404
+ const text = joinTurn(turn.parts)
405
+ if (!text) return
406
+ const startedAt = Date.now()
407
+ this.opts.log('turn.flush', {
408
+ target: turn.target,
409
+ reason,
410
+ segments: turn.parts.length,
411
+ chars: text.length,
412
+ // The headline latency number of this phase: from the owner's last word to the
413
+ // moment the fleet is handed the thought.
414
+ sinceSpeechEndMs: this.lastSpeechEndedAt ? startedAt - this.lastSpeechEndedAt : null,
415
+ })
416
+ void this.deliverTurn(turn.target, text, startedAt)
417
+ }
418
+
419
+ private async deliverTurn(target: string, text: string, startedAt: number): Promise<void> {
420
+ // The turn is committed → the "heard you, working on it" sound. ONE per thought, not
421
+ // one per segment. Delivery itself stays silent (processing earns no earcon) — the
422
+ // agent's own voice is the next thing the owner hears.
423
+ await this.earcon('end')
424
+ this.openWindow(target)
425
+ const verdict = await this.opts.deliver(target, text)
426
+ if (verdict.ok) {
427
+ this.deliveredAt.set(target, Date.now())
428
+ this.opts.log('voice.delivered', { target, chars: text.length, deliverMs: Date.now() - startedAt })
429
+ } else {
430
+ this.opts.log('voice.undelivered', { target, detail: verdict.detail })
431
+ await this.earcon('error')
432
+ await this.say(`Не доставлено: ${verdict.detail ?? 'неизвестная ошибка'}`)
433
+ }
434
+ }
435
+
436
+ /** Open (or extend) the follow-up window, and arm the sound that tells the owner when
437
+ * it has closed. An invisible window is worse than a short one: the owner kept talking
438
+ * into a closed mic with no idea a name was needed again (live, 22.07). */
439
+ private openWindow(target: string): void {
440
+ this.followTarget = target
441
+ this.followUntil = Date.now() + FOLLOWUP_MS
442
+ if (this.followTimer) clearTimeout(this.followTimer)
443
+ this.followTimer = setTimeout(() => {
444
+ this.followTimer = null
445
+ if (this.pending) return // still composing — the window is not idle
446
+ this.opts.log('followup.closed', { target })
447
+ this.followTarget = null
448
+ void this.earcon('close')
449
+ }, FOLLOWUP_MS)
450
+ }
451
+
452
+ private clearWindow(): void {
453
+ if (this.followTimer) clearTimeout(this.followTimer)
454
+ this.followTimer = null
269
455
  }
270
456
 
271
457
  /** Speak an agent's answer on the owner's speaker, then hold the follow-up window
@@ -276,16 +462,21 @@ export class VoiceSession {
276
462
  this.opts.log('tts.skip', { from, reason: 'nothing speakable' })
277
463
  return
278
464
  }
465
+ // How long the owner waited between finishing their thought and hearing an answer —
466
+ // the number Фаза 4 is judged by. It splits into the agent's thinking time and the
467
+ // synthesis time (logged separately by `tts.spoken`).
468
+ const askedAt = this.deliveredAt.get(from)
469
+ if (askedAt) this.opts.log('turn.answering', { from, sinceDeliveryMs: Date.now() - askedAt })
279
470
  // Voice by IDENTITY, not by voice name: voice-connect resolves the sender's voice
280
471
  // after picking the engine, so the owner hears WHO is talking even when the primary
281
472
  // engine is down and a fallback rung serves. Sending a voice name instead would only
282
473
  // bind to the primary engine — the very defect seen live on 22.07 (boris answered in
283
474
  // a stranger's voice once gemini was unavailable).
284
475
  await this.say(text, { from })
285
- this.followTarget = from
286
- this.followUntil = Date.now() + FOLLOWUP_MS
287
476
  // The turn passes back to the human: the mic is open for a follow-up without a
288
- // wake word, and the rising earcon says so.
477
+ // wake word, the rising earcon says so, and the window's own timer will say when it
478
+ // closes again.
479
+ this.openWindow(from)
289
480
  await this.earcon('open')
290
481
  }
291
482
 
@@ -299,7 +490,8 @@ export class VoiceSession {
299
490
  this.muteUntil = Date.now() + 3_600_000
300
491
  this.segmenter.reset()
301
492
  try {
302
- await playFile(result.path)
493
+ if (PLAYBACK_ENABLED) await playFile(result.path)
494
+ else this.opts.log('tts.silent', { from: opts.from, chars: text.length, path: result.path })
303
495
  } finally {
304
496
  this.muteUntil = Date.now() + ECHO_TAIL_MS
305
497
  this.segmenter.reset()
@@ -324,6 +516,30 @@ export class VoiceSession {
324
516
  }
325
517
  }
326
518
 
519
+ /** Does this fragment sound like the middle of a sentence? Recognition punctuates what
520
+ * it hears, so a fragment left hanging on a comma or a conjunction — or with no terminal
521
+ * mark at all — is a thought still in progress, and the turn waits longer for the rest. */
522
+ export function looksUnfinished(text: string): boolean {
523
+ const t = text.trim()
524
+ if (!t) return false
525
+ if (/[,;:—–-]$/.test(t)) return true
526
+ if (!/[.!?…]$/.test(t)) return true
527
+ const last = t.replace(/[.!?…]+$/, '').split(/\s+/).pop()?.toLowerCase() ?? ''
528
+ return ['и', 'а', 'но', 'что', 'чтобы', 'или', 'если', 'когда', 'потому', 'то'].includes(last)
529
+ }
530
+
531
+ /** Join the fragments of one thought back into a sentence. Recognition returns each
532
+ * segment already punctuated and capitalized, so a plain space is the honest join —
533
+ * the parts were one utterance and are re-read as one. */
534
+ export function joinTurn(parts: readonly string[]): string {
535
+ return parts
536
+ .map(p => p.trim())
537
+ .filter(Boolean)
538
+ .join(' ')
539
+ .replace(/\s+/g, ' ')
540
+ .trim()
541
+ }
542
+
327
543
  /** Prepare an agent's text for the speaker: strip what only makes sense on a screen and
328
544
  * cap the length — a spoken wall of text is unusable. The modality frame already asks
329
545
  * agents for short plain answers; this is the floor for when one forgets. */
package/src/wake.ts CHANGED
@@ -205,6 +205,10 @@ export type WakeMatch = {
205
205
  /** The utterance with the address (and any leading filler) stripped. */
206
206
  body: string
207
207
  alias: string
208
+ /** True when every word of the address matched letter for letter. A match that only
209
+ * survived the one-edit tolerance is good enough to START a conversation, but not to
210
+ * TAKE one away from the peer already in it — see the switch bar in voiceSession. */
211
+ exact: boolean
208
212
  }
209
213
 
210
214
  /** Does this transcript open by addressing a fleet peer? Longest alias wins, so
@@ -229,6 +233,7 @@ export function matchWake(transcript: string, book: Address[]): WakeMatch | null
229
233
  if (tokens.length < cand.words.length) continue
230
234
  const ok = cand.words.every((w, i) => tokenMatches(tokens[i], w))
231
235
  if (!ok) continue
236
+ const exact = cand.words.every((w, i) => tokens[i] === w)
232
237
  // Rebuild the body from the ORIGINAL transcript so punctuation and casing of the
233
238
  // message survive — only the address is stripped.
234
239
  const body = rawTokens
@@ -236,11 +241,31 @@ export function matchWake(transcript: string, book: Address[]): WakeMatch | null
236
241
  .join(' ')
237
242
  .replace(/^[\s,.!:;—–-]+/, '')
238
243
  .trim()
239
- return { target: cand.target, alias: cand.alias, body }
244
+ return { target: cand.target, alias: cand.alias, body, exact }
240
245
  }
241
246
  return null
242
247
  }
243
248
 
249
+ /** May this match TAKE an open conversation away from the peer currently in it?
250
+ *
251
+ * Mid-dialogue the two errors are wildly asymmetric: one more line delivered to the peer
252
+ * already listening is nearly free, while a mis-heard name silently hands the whole
253
+ * conversation to another agent — the owner keeps talking to someone who is no longer
254
+ * there. So a switch must look deliberate: the name heard letter for letter, followed by
255
+ * a real request. Returns the objection, or null when the switch is allowed.
256
+ *
257
+ * Every rule here is a live artefact, not a hypothesis: «да, различаю» came back from
258
+ * recognition as «Дарвин, дарвин» and moved the conversation off boris (22.07). */
259
+ export function objectionToSwitch(match: WakeMatch): string | null {
260
+ if (!match.exact) return 'inexact name match'
261
+ const body = normalizeSpoken(match.body)
262
+ if (!body) return 'name with no request'
263
+ const words = body.split(' ')
264
+ if (words.every(w => w === match.alias)) return 'body echoes the name'
265
+ if (words.length < 2) return 'request too short to be an address'
266
+ return null
267
+ }
268
+
244
269
  /** Whisper-family recognizers emit stock phrases on near-silence («Продолжение
245
270
  * следует…», «Субтитры сделал…»). They are not speech; drop them before matching so a
246
271
  * quiet room cannot fabricate an utterance. */