@agfpd/voicetalk-runtime 0.2.0 → 0.3.0

package/docs/README.md

@@ -48,9 +48,19 @@ voice-connect, microphone).
 2. voice-connect transcribes the utterance (locally, on this host).
 3. If it opens by addressing a peer, the body is delivered to that peer as an IAP
    envelope from the owner's identity, framed for the spoken modality.
+3.5. Fragments are COMPOSED into one turn. Endpointing cuts audio, not thoughts: a
+   pause inside a sentence would otherwise become a second envelope and earn a second
+   spoken answer. The turn is sent once the owner has been silent for ~3 s since their
+   last word — and never while they are still speaking or a fragment is still being
+   recognized. A fragment left hanging on a comma or a conjunction waits longer.
 4. The agent's answer arrives on stdin, is spoken through voice-connect on the host
    speaker, and the microphone stays open for ~9 s so the owner can continue without
-   naming the peer again.
+   naming the peer again. When that window closes, a soft low tone says so.
+
+Inside an open conversation a wake word for ANOTHER peer must look deliberate — the
+name heard letter for letter, followed by a real request — otherwise the fragment goes
+to the peer already listening. Mis-hearing a name should cost a stray line, never the
+whole conversation.
 
 Earcons mark the states: rising = the microphone is yours, short descending = heard
 you, double descending = nothing was routed. Processing is silent.
@@ -58,13 +68,21 @@ you, double descending = nothing was routed. Processing is silent.
 Everything is tunable by env: `VOICETALK_AUDIO=0` (text-only, no microphone),
 `VOICETALK_FOLLOWUP_MS`, `VOICETALK_HANGOVER_MS`, `VOICETALK_VAD_MIN_RMS`,
 `VOICETALK_VAD_RATIO`, `VOICETALK_STT_LANG`, `VOICETALK_STT_ENGINE`,
-`VOICETALK_TTS_MAX_CHARS`, `VOICETALK_EARCONS=0`.
+`VOICETALK_TTS_MAX_CHARS`, `VOICETALK_EARCONS=0`, `VOICETALK_COMPOSE_MS`,
+`VOICETALK_COMPOSE_UNFINISHED_MS`, `VOICETALK_TURN_MAX_MS`.
+
+Two seams exist for testing turn-taking without touching a live channel:
+`VOICETALK_MIC_FILE=<pcm>` replays a recording through the microphone path (paced in
+real time) and `VOICETALK_PLAYBACK=0` keeps the speaker silent while synthesis is
+still performed and timed. `scripts/replay-dialogue.sh` wires both into a sandbox.
 
 ## Requirements
 
 - `ffmpeg` (microphone capture) and `ffplay` (playback) on `PATH`.
-- The voice-connect HTTP service running locally — it is discovered through
-  `<IAPEER_ROOT>/voice-provider.json` (override with `VOICETALK_VOICE_ENDPOINT`).
+- The voice-connect HTTP service running locally (**≥0.4.0** — replies are voiced by
+  sender identity, `peer`, so an agent keeps its own voice on every fallback rung) — it
+  is discovered through `<IAPEER_ROOT>/voice-provider.json` (override with
+  `VOICETALK_VOICE_ENDPOINT`).
 - Microphone permission (TCC) granted once to the process that runs the runtime.
 
 These pages are written as each phase lands. Russian translation lives in

package/docs/ru/README.md

@@ -48,8 +48,18 @@ self-config-хука (`VOICETALK_WAKE_WORDS`, `VOICETALK_DEFAULT_TARGET`,
 2. voice-connect распознаёт реплику (локально, на этом хосте).
 3. Если реплика начинается с обращения к пиру, тело уходит ему IAP-конвертом от
    личности владельца, с рамкой голосовой модальности.
+3.5. Фрагменты СОБИРАЮТСЯ в один ход. Эндпойнтер режет звук, а не мысль: пауза внутри
+   фразы иначе стала бы вторым конвертом и заработала бы второй голосовой ответ. Ход
+   уходит, когда владелец молчит ~3 с после последнего слова, и никогда — пока он
+   ещё говорит или пока фрагмент ещё распознаётся. Фрагмент, оборванный на запятой
+   или союзе, ждёт дольше.
 4. Ответ агента приходит на stdin, озвучивается через voice-connect в динамик, и
-   микрофон остаётся открытым ~9 с — продолжить можно без повторного обращения.
+   микрофон остаётся открытым ~9 с — продолжить можно без повторного обращения. Когда
+   окно закрывается, звучит тихий низкий тон.
+
+Внутри открытого диалога wake-слово ДРУГОГО пира обязано выглядеть намеренным — имя
+услышано буква в букву и за ним идёт настоящая просьба, — иначе фрагмент уходит тому,
+кто уже слушает. Ослышка должна стоить лишней строки, а не всего разговора.
 
 Earcon'ы размечают состояния: восходящий — микрофон ваш, короткий нисходящий —
 принял, двойной нисходящий — не отправлено. Обработка идёт молча.
@@ -57,13 +67,21 @@ Earcon'ы размечают состояния: восходящий — мик
 Всё настраивается через env: `VOICETALK_AUDIO=0` (текстовый режим, без микрофона),
 `VOICETALK_FOLLOWUP_MS`, `VOICETALK_HANGOVER_MS`, `VOICETALK_VAD_MIN_RMS`,
 `VOICETALK_VAD_RATIO`, `VOICETALK_STT_LANG`, `VOICETALK_STT_ENGINE`,
-`VOICETALK_TTS_MAX_CHARS`, `VOICETALK_EARCONS=0`.
+`VOICETALK_TTS_MAX_CHARS`, `VOICETALK_EARCONS=0`, `VOICETALK_COMPOSE_MS`,
+`VOICETALK_COMPOSE_UNFINISHED_MS`, `VOICETALK_TURN_MAX_MS`.
+
+Для проверки turn-taking, не трогая живой канал, есть два шва: `VOICETALK_MIC_FILE=<pcm>`
+проигрывает запись через тот же путь, что и микрофон (в реальном темпе), а
+`VOICETALK_PLAYBACK=0` держит динамик молча, оставляя синтез и его замеры. Оба сведены
+в `scripts/replay-dialogue.sh`.
 
 ## Требования
 
 - `ffmpeg` (захват микрофона) и `ffplay` (воспроизведение) на `PATH`.
-- Локально запущенный HTTP-сервис voice-connect — обнаруживается через
-  `<IAPEER_ROOT>/voice-provider.json` (переопределяется `VOICETALK_VOICE_ENDPOINT`).
+- Локально запущенный HTTP-сервис voice-connect (**≥0.4.0** — ответы озвучиваются по
+  идентичности отправителя, параметр `peer`, поэтому агент сохраняет свой голос на любой
+  ступени каскада) — обнаруживается через `<IAPEER_ROOT>/voice-provider.json`
+  (переопределяется `VOICETALK_VOICE_ENDPOINT`).
 - Разрешение на микрофон (TCC), выданное один раз процессу рантайма.
 
 Страницы пишутся по мере прохождения фаз.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@agfpd/voicetalk-runtime",
-  "version": "0.2.0",
+  "version": "0.3.0",
   "description": "Voice runtime-router for IAP peers.",
   "license": "Apache-2.0",
   "type": "module",

package/src/audio.ts

@@ -9,7 +9,7 @@
 
 import { spawn, type ChildProcessByStdio } from 'child_process'
 import type { Readable } from 'stream'
-import { writeFileSync } from 'fs'
+import { readFileSync, writeFileSync } from 'fs'
 
 export const SAMPLE_RATE = 16_000
 export const FRAME_MS = 20
@@ -125,6 +125,13 @@ export class Segmenter {
     this.warmupFrames = Math.round((opts.warmupMs ?? 600) / FRAME_MS)
   }
 
+  /** Is the endpointer inside an utterance right now? The turn layer asks before it
+   *  decides a thought is finished: the owner resuming mid-pause is the difference
+   *  between one envelope and two. */
+  get speaking(): boolean {
+    return this.inSpeech
+  }
+
   get noiseFloor(): number {
     return this.floor < 0 ? 0 : this.floor
   }
@@ -293,6 +300,11 @@ export async function resolveInputDevice(
 
 export type MicOptions = {
   deviceIndex: number
+  /** Replay a recording INSTEAD of opening the microphone (test seam). Frames are paced
+   *  in real time, so endpointing and the compose window behave exactly as they do on a
+   *  live mic — which is the whole point: turn-taking bugs only reproduce against a
+   *  clock. Accepts a WAV or headerless mono 16-bit 16 kHz PCM file. */
+  filePath?: string
   ffmpegBin?: string
   onData: (chunk: Buffer) => void
   onError?: (detail: string) => void
@@ -333,6 +345,10 @@ export class Microphone {
   private spawnChild(): void {
     if (this.stopped) return
     this.attempt += 1
+    if (this.opts.filePath) {
+      this.replayFile(this.opts.filePath)
+      return
+    }
     const bin = this.opts.ffmpegBin ?? 'ffmpeg'
     const args = [
       '-hide_banner',
@@ -368,6 +384,35 @@ export class Microphone {
     })
   }
 
+  /** Feed a recording through the same path the microphone uses, one 20 ms frame per
+   *  20 ms, then fall silent (a finished recording is a quiet room, not a dead mic). */
+  private replayFile(path: string): void {
+    let pcm: Buffer
+    try {
+      const raw = readFileSync(path)
+      // Skip a RIFF header if present; anything else is treated as raw PCM.
+      pcm = raw.subarray(0, 4).toString() === 'RIFF' ? raw.subarray(44) : raw
+    } catch (err) {
+      this.opts.onError?.(`cannot replay ${path}: ${err instanceof Error ? err.message : String(err)}`)
+      return
+    }
+    this.opts.onStart?.(this.attempt)
+    let offset = 0
+    const tick = () => {
+      if (this.stopped) return
+      if (offset >= pcm.length) {
+        // Keep the stream alive with silence so compose/window timers still fire.
+        this.opts.onData(Buffer.alloc(FRAME_BYTES))
+        this.timer = setTimeout(tick, FRAME_MS)
+        return
+      }
+      this.opts.onData(Buffer.from(pcm.subarray(offset, offset + FRAME_BYTES)))
+      offset += FRAME_BYTES
+      this.timer = setTimeout(tick, FRAME_MS)
+    }
+    tick()
+  }
+
   private scheduleRestart(): void {
     if (this.stopped || this.timer) return
     const delay = Math.min(30_000, 1_000 * Math.min(8, this.attempt))

package/src/voiceConnect.ts

@@ -6,7 +6,7 @@
 // CONSUMER of its OpenAI-compatible routes:
 //
 //   POST /v1/audio/transcriptions   multipart(file, language?, engine?)  → { text }
-//   POST /v1/audio/speech           json(input, voice?, lang?, engine?)  → ogg/opus bytes
+//   POST /v1/audio/speech           json(input, peer?, lang?, engine?)   → ogg/opus bytes
 //   GET  /health                                                          → { status }
 //
 // Engine selection, provider cascade and fallbacks live INSIDE that service — never
@@ -143,7 +143,14 @@ export async function transcribe(audioPath: string, opts: TranscribeOptions = {}
 }
 
 export type SynthesizeOptions = {
-  /** Per-peer voice (the sender's `interfaces.voice` entry) — so agents sound distinct. */
+  /** The SPEAKER's identity (voice-connect ≥0.4.0). The service resolves that peer's
+   *  voice from `interfaces.voice` AFTER the serving engine is known, so the identity
+   *  survives every rung of the cascade — including the F5 reference. Always prefer this
+   *  over `voice`: a voice NAME belongs to one engine's namespace, so it only applies to
+   *  the engine tried first and means nothing on the rung that actually serves. */
+  peer?: string
+  /** Legacy voice-name override — covers the PRIMARY engine only. Kept for callers with
+   *  no peer identity to offer; `peer` is what a runtime should send. */
   voice?: string
   lang?: string
   style?: string
@@ -155,6 +162,8 @@ export type SynthesizeResult = {
   path: string
   engine?: string
   voice?: string
+  /** The identity the service voiced this as (echo of `peer`). */
+  peer?: string
   fallbackFrom?: string
   bytes: number
 }
@@ -168,6 +177,7 @@ export async function synthesize(text: string, outPath: string, opts: Synthesize
     headers: { 'content-type': 'application/json' },
     body: JSON.stringify({
       input: text,
+      ...(opts.peer ? { peer: opts.peer } : {}),
       ...(opts.voice ? { voice: opts.voice } : {}),
       ...(opts.lang ? { lang: opts.lang } : {}),
       ...(opts.style ? { style: opts.style } : {}),
@@ -185,6 +195,7 @@ export async function synthesize(text: string, outPath: string, opts: Synthesize
     path: outPath,
     engine: res.headers.get('x-voice-engine') ?? undefined,
     voice: res.headers.get('x-voice-voice') ?? undefined,
+    peer: res.headers.get('x-voice-peer') ?? undefined,
     fallbackFrom: res.headers.get('x-voice-fallback-from') ?? undefined,
     bytes: buf.length,
   }

package/src/voiceSession.ts

@@ -1,9 +1,17 @@
 // voiceSession.ts — the live voice session (Фаза 3): the runtime's ears, its turn-taking
 // and its mouth, wired between audio.ts (host mechanics) and voiceConnect.ts (speech).
 //
-// INBOUND   mic → endpointing → STT → address match → deliver to the fleet
+// INBOUND   mic → endpointing → STT → address match → COMPOSE → deliver to the fleet
 // OUTBOUND  agent envelope → TTS → speaker, then an open-mic follow-up window
 //
+// A pause inside a sentence is not the end of a thought. Endpointing (fast, 700 ms) only
+// cuts AUDIO into segments so recognition can start early; what the fleet receives is a
+// TURN — every segment of one thought, joined, delivered ONCE when the owner has actually
+// finished (the compose window). Without that, one spoken thought became two envelopes
+// and earned two spoken answers, the second one replying to a fragment (live complaint,
+// 22.07). Recognition still runs per segment and in the background, so composing costs
+// only the silence the owner was taking anyway.
+//
 // Turn-taking follows the established voice-assistant pattern recorded in Описание
 // voicetalk: the address (wake word) only INITIATES a conversation; after an agent has
 // answered, a short open-mic window lets the owner continue without naming the peer
@@ -39,6 +47,7 @@ import {
   buildAddressBook,
   isLikelyHallucination,
   matchWake,
+  objectionToSwitch,
   readFleetPeers,
   sttPrompt,
   type Address,
@@ -57,6 +66,24 @@ const FOLLOWUP_MS = num(process.env.VOICETALK_FOLLOWUP_MS, 9_000)
  *  500–800 ms for free dialogue; the upper half of the band suits a heavier recognizer —
  *  it fragments one thought into two transcriptions far less often. */
 const HANGOVER_MS = num(process.env.VOICETALK_HANGOVER_MS, 700)
+/** Silence (counted from the owner's last WORD) after which a turn is considered
+ *  finished and sent. The two errors are not symmetric: waiting one second too long costs
+ *  one second on a chain where synthesis alone is ~5 s, while not waiting costs a split
+ *  thought, two envelopes and a spoken answer to a fragment — the owner's actual
+ *  complaint. Measured thinking pauses in real Russian speech run to ~2.7 s, so the
+ *  budget sits above that with room. Tunable live: VOICETALK_COMPOSE_MS. */
+const COMPOSE_MS = num(process.env.VOICETALK_COMPOSE_MS, 3_000)
+/** A fragment that ENDS mid-thought (no terminal punctuation, or a trailing comma or
+ *  conjunction) buys extra patience — the speaker audibly has not finished. */
+const COMPOSE_UNFINISHED_MS = num(process.env.VOICETALK_COMPOSE_UNFINISHED_MS, 4_500)
+/** Segments waiting for recognition. Deep enough for a long sentence spoken over a
+ *  slow recognizer, shallow enough that a wedged STT cannot grow it without bound. */
+const SEGMENT_QUEUE_MAX = 8
+/** A turn never grows past this — a monologue still reaches its addressee. */
+const TURN_MAX_MS = num(process.env.VOICETALK_TURN_MAX_MS, 45_000)
+/** How often to re-check a turn that is ready to send but whose speaker has not gone
+ *  quiet yet. */
+const HOLD_RECHECK_MS = num(process.env.VOICETALK_HOLD_RECHECK_MS, 300)
 /** Gate the mic for a moment after the speaker stops — room echo tail. */
 const ECHO_TAIL_MS = num(process.env.VOICETALK_ECHO_TAIL_MS, 400)
 /** Long answers are truncated for speech: a spoken monologue is worse than a summary. */
@@ -65,6 +92,12 @@ const TTS_MAX_CHARS = num(process.env.VOICETALK_TTS_MAX_CHARS, 1_200)
 const STT_LANG = process.env.VOICETALK_STT_LANG ?? 'ru'
 /** Force a local recognizer (mlx-whisper) to keep always-on listening on-device. */
 const STT_ENGINE = process.env.VOICETALK_STT_ENGINE?.trim() || undefined
+/** Test seams. VOICETALK_MIC_FILE replays a recording instead of opening the microphone;
+ *  VOICETALK_PLAYBACK=0 keeps the speaker silent (synthesis still runs and is still
+ *  timed). Together they let a turn-taking change be verified end to end on a host whose
+ *  real voice channel is in use by its owner. */
+const MIC_FILE = process.env.VOICETALK_MIC_FILE?.trim() || undefined
+const PLAYBACK_ENABLED = process.env.VOICETALK_PLAYBACK !== '0'
 
 export type DeliverFn = (target: string, text: string) => Promise<{ ok: boolean; detail?: string }>
 export type LogFn = (evt: string, fields: Record<string, unknown>) => void
@@ -80,9 +113,10 @@ export type VoiceSessionOptions = {
 }
 
 /** State sounds, per the earcon convention in the research: `open` = the microphone is
- *  yours (rising, ≤300 ms), `end` = utterance captured, processing (short descending,
- *  ≤200 ms), `error` = nothing routed (double descending). Processing itself is silent. */
-type Earcons = { open: string; end: string; error: string }
+ *  yours (rising, ≤300 ms), `end` = turn captured, processing (short descending,
+ *  ≤200 ms), `close` = the open-mic window expired, a name is needed again (low, soft),
+ *  `error` = nothing routed (double descending). Processing itself is silent. */
+type Earcons = { open: string; end: string; error: string; close: string }
 
 export class VoiceSession {
   private mic: Microphone | null = null
@@ -95,9 +129,18 @@ export class VoiceSession {
   private prompt = ''
   private earcons: Earcons | null = null
   private busy = false
+  private queue: Array<Segment & { speechEndedAt: number }> = []
   private muteUntil = 0
+  /** When the microphone last stopped hearing speech — the clock the compose window runs
+   *  on, so the delivery delay the owner feels is exactly the silence they took. */
+  private lastSpeechEndedAt = 0
   private followTarget: string | null = null
   private followUntil = 0
+  private followTimer: ReturnType<typeof setTimeout> | null = null
+  /** The thought being assembled right now — segments of one turn, not yet sent. */
+  private pending: { target: string; parts: string[]; openedAt: number; timer: ReturnType<typeof setTimeout> | null } | null = null
+  /** When the current turn was handed to the fleet — the clock for «ответ за N с». */
+  private deliveredAt = new Map<string, number>()
   private stopped = false
 
   constructor(private readonly opts: VoiceSessionOptions) {}
@@ -109,9 +152,12 @@ export class VoiceSession {
     mkdirSync(this.opts.tmpDir, { recursive: true, mode: 0o700 })
     this.refreshAddressBook()
     this.earcons = this.writeEarcons()
-    const device = await resolveInputDevice(this.opts.binding.device)
+    const device = MIC_FILE
+      ? { index: -1, name: `replay:${MIC_FILE}` }
+      : await resolveInputDevice(this.opts.binding.device)
     this.mic = new Microphone({
       deviceIndex: device.index,
+      filePath: MIC_FILE,
       onData: chunk => this.onAudio(chunk),
       onStart: attempt => this.opts.log('mic.start', { device: device.index, name: device.name, attempt }),
       onError: detail => this.opts.log('mic.error', { detail }),
@@ -122,6 +168,9 @@ export class VoiceSession {
 
   stop(): void {
     this.stopped = true
+    this.clearWindow()
+    if (this.pending?.timer) clearTimeout(this.pending.timer)
+    this.pending = null
     this.mic?.stop()
     this.mic = null
   }
@@ -140,6 +189,7 @@ export class VoiceSession {
       open: join(this.opts.tmpDir, 'earcon-open.wav'),
       end: join(this.opts.tmpDir, 'earcon-end.wav'),
       error: join(this.opts.tmpDir, 'earcon-error.wav'),
+      close: join(this.opts.tmpDir, 'earcon-close.wav'),
     }
     // A5→C6→E6 rising, 240 ms — the microphone is open.
     writeFileSync(files.open, earconWav([{ hz: 880, ms: 80 }, { hz: 1047, ms: 80 }, { hz: 1319, ms: 80 }]), { mode: 0o600 })
@@ -147,11 +197,15 @@ export class VoiceSession {
     writeFileSync(files.end, earconWav([{ hz: 880, ms: 80 }, { hz: 660, ms: 80 }]), { mode: 0o600 })
     // Double descending, 260 ms — tonally "closed": nothing was routed.
     writeFileSync(files.error, earconWav([{ hz: 520, ms: 120 }, { hz: 330, ms: 140 }]), { mode: 0o600 })
+    // One low soft tone, 180 ms — the open-mic window just expired; say a name to speak
+    // again. Deliberately quieter and lower than `end` so it reads as "door closing",
+    // not as another acknowledgement.
+    writeFileSync(files.close, earconWav([{ hz: 400, ms: 180 }], 0.14), { mode: 0o600 })
     return files
   }
 
   private async earcon(kind: keyof Earcons): Promise<void> {
-    if (!this.earcons || process.env.VOICETALK_EARCONS === '0') return
+    if (!this.earcons || process.env.VOICETALK_EARCONS === '0' || !PLAYBACK_ENABLED) return
     try {
       this.muteUntil = Math.max(this.muteUntil, Date.now() + 600)
       await playFile(this.earcons[kind], { timeoutMs: 5_000 })
@@ -173,16 +227,35 @@ export class VoiceSession {
       return
     }
     for (const segment of this.segmenter.push(chunk)) {
-      if (this.busy) {
-        // One utterance in flight at a time: STT is the expensive step and a pile-up
-        // would only add latency to the turn that matters.
-        this.opts.log('segment.drop', { reason: 'busy', ms: segment.ms })
+      // Recognition is serialized (one STT at a time — a pile-up only adds latency), but
+      // segments QUEUE rather than being dropped: a dropped segment is a hole in the
+      // middle of the owner's sentence, which is exactly what this phase is fixing.
+      if (this.queue.length >= SEGMENT_QUEUE_MAX) {
+        this.opts.log('segment.drop', { reason: 'queue full', ms: segment.ms })
         continue
       }
-      this.busy = true
-      void this.handleSegment(segment).finally(() => {
-        this.busy = false
-      })
+      // The endpointer hands over a segment only after the hangover silence, so speech
+      // itself stopped that much earlier. The compose budget is counted from THAT
+      // instant: the owner's pause should not be charged twice, once as endpointing and
+      // again as composing.
+      this.queue.push({ ...segment, speechEndedAt: Date.now() - HANGOVER_MS })
+      void this.drainQueue()
+    }
+  }
+
+  private async drainQueue(): Promise<void> {
+    if (this.busy) return
+    this.busy = true
+    try {
+      while (this.queue.length) {
+        const segment = this.queue.shift()!
+        this.lastSpeechEndedAt = segment.speechEndedAt
+        await this.handleSegment(segment)
+      }
+    } finally {
+      this.busy = false
+      // A turn may have been waiting for recognition to finish before it could be sent.
+      if (this.pending) this.rearm()
     }
   }
 
@@ -222,59 +295,163 @@ export class VoiceSession {
         text: transcript,
       })
 
-      const match = matchWake(transcript, this.book)
-      const followOpen = this.followTarget && Date.now() < this.followUntil
-      let target: string | null = null
-      let body = ''
-      if (match) {
-        target = match.target
-        body = match.body
-        if (!body) {
-          // Addressed with nothing to say yet («Борис?») — acknowledge and hold the mic
-          // open for the actual request, exactly as an assistant beeps and waits.
-          this.followTarget = target
-          this.followUntil = Date.now() + FOLLOWUP_MS
-          this.opts.log('wake.await-body', { target, alias: match.alias })
-          await this.earcon('open')
-          return
-        }
-      } else if (followOpen) {
-        target = this.followTarget
-        body = transcript.trim()
-        this.opts.log('followup.route', { target })
-      } else {
-        this.opts.log('wake.no-match', { text: transcript })
+      this.absorb(transcript)
+    } finally {
+      rmSync(wav, { force: true })
+    }
+  }
+
+  /** Route one recognized fragment into the turn being composed.
+   *
+   *  The decision is deliberately conservative in one direction: an unaddressed fragment
+   *  arriving right after an addressed one CONTINUES that thought rather than being
+   *  dropped, and a wake word for SOMEONE ELSE has to clear a higher bar than it does
+   *  when no conversation is open. Mid-dialogue, the cheap error (one more line to the
+   *  peer already listening) is far cheaper than the expensive one (the conversation
+   *  silently handed to another agent). */
+  private absorb(transcript: string): void {
+    const active = this.pending?.target ?? (this.followTarget && Date.now() < this.followUntil ? this.followTarget : null)
+    const match = matchWake(transcript, this.book)
+    const switching = !!match && !!active && match.target !== active
+
+    if (switching) {
+      const decline = objectionToSwitch(match!)
+      if (decline) {
+        this.opts.log('wake.switch-declined', { from: active, to: match!.target, reason: decline, text: transcript })
+        this.append(active!, transcript.trim())
         return
       }
+      // A deliberate switch: the thought in flight belongs to the previous peer, so it
+      // goes out before the new one starts.
+      this.opts.log('wake.switch', { from: active, to: match!.target })
+      this.flush('switch')
+      this.append(match!.target, match!.body)
+      return
+    }
 
-      // The window is extended for the whole turn: the agent's answer will extend it
-      // again when it is spoken.
-      this.followTarget = target
-      this.followUntil = Date.now() + FOLLOWUP_MS
-      // Utterance accepted → the "end of listening, now processing" sound. Delivery
-      // itself stays SILENT (processing earns no earcon) — the agent's own voice is the
-      // next thing the owner hears.
-      await this.earcon('end')
-      const verdict = await this.opts.deliver(target!, body)
-      if (verdict.ok) {
-        this.opts.log('voice.delivered', { target, chars: body.length, totalMs: Date.now() - started })
-      } else {
-        this.opts.log('voice.undelivered', { target, detail: verdict.detail })
-        await this.earcon('error')
-        await this.say(`Не доставлено: ${verdict.detail ?? 'неизвестная ошибка'}`)
+    if (match) {
+      if (!match.body) {
+        // Addressed with nothing to say yet («Борис?») — acknowledge and hold the mic
+        // open for the actual request, exactly as an assistant beeps and waits.
+        this.openWindow(match.target)
+        this.opts.log('wake.await-body', { target: match.target, alias: match.alias })
+        void this.earcon('open')
+        return
       }
-    } finally {
-      rmSync(wav, { force: true })
+      this.append(match.target, match.body)
+      return
+    }
+
+    if (active) {
+      this.opts.log('turn.continue', { target: active, reason: this.pending ? 'compose' : 'follow-up' })
+      this.append(active, transcript.trim())
+      return
     }
+
+    this.opts.log('wake.no-match', { text: transcript })
   }
 
-  /** Voice of the SENDER — each agent keeps the voice configured in its own passport
-   *  (`interfaces.voice`), so the owner hears who is talking. */
-  private voiceFor(personality: string): string | undefined {
-    const peers = this.opts.peers ?? readFleetPeers()
-    const voice = peers.find(p => p.personality === personality)?.voice
-    if (!voice) return undefined
-    return voice['gemini-3.1-flash-tts-preview'] ?? Object.values(voice)[0]
+  /** Add a fragment to the turn in flight and (re)arm the compose timer. */
+  private append(target: string, text: string): void {
+    const body = text.trim()
+    if (!body) return
+    if (this.pending && this.pending.target !== target) this.flush('retarget')
+    if (!this.pending) {
+      this.pending = { target, parts: [], openedAt: Date.now(), timer: null }
+      this.opts.log('turn.open', { target })
+    }
+    this.pending.parts.push(body)
+    this.clearWindow()
+    this.rearm()
+  }
+
+  /** (Re)arm the compose timer for the turn in flight. */
+  private rearm(delayMs?: number): void {
+    const turn = this.pending
+    if (!turn) return
+    if (turn.timer) clearTimeout(turn.timer)
+    if (Date.now() - turn.openedAt >= TURN_MAX_MS) {
+      this.flush('max-length')
+      return
+    }
+    const budget = looksUnfinished(turn.parts[turn.parts.length - 1] ?? '') ? COMPOSE_UNFINISHED_MS : COMPOSE_MS
+    const remaining = this.lastSpeechEndedAt + budget - Date.now()
+    const delay = Math.max(0, delayMs ?? remaining)
+    turn.timer = setTimeout(() => this.flush('compose'), delay)
+  }
+
+  /** Send the composed thought as ONE envelope. */
+  private flush(reason: string): void {
+    const turn = this.pending
+    if (!turn) return
+    // A thought is NOT finished while the owner is still speaking it. The compose timer
+    // counts silence since the last RECOGNIZED fragment, and recognition lags the voice
+    // by an endpointing pause plus the recognizer's own time — so by the time it fires,
+    // the owner may already be several words into the rest of the sentence. Waiting for
+    // the microphone and the recognizer to go quiet is what turns those words into the
+    // same envelope instead of a second one. (Live defect, 22.07: two envelopes, two
+    // spoken answers, the second one replying to a fragment.)
+    if (reason === 'compose' && (this.segmenter.speaking || this.busy || this.queue.length)) {
+      this.opts.log('turn.hold', {
+        target: turn.target,
+        reason: this.segmenter.speaking ? 'still speaking' : 'recognition in flight',
+      })
+      this.rearm(HOLD_RECHECK_MS)
+      return
+    }
+    this.pending = null
+    if (turn.timer) clearTimeout(turn.timer)
+    const text = joinTurn(turn.parts)
+    if (!text) return
+    const startedAt = Date.now()
+    this.opts.log('turn.flush', {
+      target: turn.target,
+      reason,
+      segments: turn.parts.length,
+      chars: text.length,
+      // The headline latency number of this phase: from the owner's last word to the
+      // moment the fleet is handed the thought.
+      sinceSpeechEndMs: this.lastSpeechEndedAt ? startedAt - this.lastSpeechEndedAt : null,
+    })
+    void this.deliverTurn(turn.target, text, startedAt)
+  }
+
+  private async deliverTurn(target: string, text: string, startedAt: number): Promise<void> {
+    // The turn is committed → the "heard you, working on it" sound. ONE per thought, not
+    // one per segment. Delivery itself stays silent (processing earns no earcon) — the
+    // agent's own voice is the next thing the owner hears.
+    await this.earcon('end')
+    this.openWindow(target)
+    const verdict = await this.opts.deliver(target, text)
+    if (verdict.ok) {
+      this.deliveredAt.set(target, Date.now())
+      this.opts.log('voice.delivered', { target, chars: text.length, deliverMs: Date.now() - startedAt })
+    } else {
+      this.opts.log('voice.undelivered', { target, detail: verdict.detail })
+      await this.earcon('error')
+      await this.say(`Не доставлено: ${verdict.detail ?? 'неизвестная ошибка'}`)
+    }
+  }
+
+  /** Open (or extend) the follow-up window, and arm the sound that tells the owner when
+   *  it has closed. An invisible window is worse than a short one: the owner kept talking
+   *  into a closed mic with no idea a name was needed again (live, 22.07). */
+  private openWindow(target: string): void {
+    this.followTarget = target
+    this.followUntil = Date.now() + FOLLOWUP_MS
+    if (this.followTimer) clearTimeout(this.followTimer)
+    this.followTimer = setTimeout(() => {
+      this.followTimer = null
+      if (this.pending) return // still composing — the window is not idle
+      this.opts.log('followup.closed', { target })
+      this.followTarget = null
+      void this.earcon('close')
+    }, FOLLOWUP_MS)
+  }
+
+  private clearWindow(): void {
+    if (this.followTimer) clearTimeout(this.followTimer)
+    this.followTimer = null
   }
 
   /** Speak an agent's answer on the owner's speaker, then hold the follow-up window
@@ -285,25 +462,36 @@ export class VoiceSession {
       this.opts.log('tts.skip', { from, reason: 'nothing speakable' })
       return
     }
-    await this.say(text, { from, voice: this.voiceFor(from) })
-    this.followTarget = from
-    this.followUntil = Date.now() + FOLLOWUP_MS
+    // How long the owner waited between finishing their thought and hearing an answer —
+    // the number Фаза 4 is judged by. It splits into the agent's thinking time and the
+    // synthesis time (logged separately by `tts.spoken`).
+    const askedAt = this.deliveredAt.get(from)
+    if (askedAt) this.opts.log('turn.answering', { from, sinceDeliveryMs: Date.now() - askedAt })
+    // Voice by IDENTITY, not by voice name: voice-connect resolves the sender's voice
+    // after picking the engine, so the owner hears WHO is talking even when the primary
+    // engine is down and a fallback rung serves. Sending a voice name instead would only
+    // bind to the primary engine — the very defect seen live on 22.07 (boris answered in
+    // a stranger's voice once gemini was unavailable).
+    await this.say(text, { from })
     // The turn passes back to the human: the mic is open for a follow-up without a
-    // wake word, and the rising earcon says so.
+    // wake word, the rising earcon says so, and the window's own timer will say when it
+    // closes again.
+    this.openWindow(from)
     await this.earcon('open')
   }
 
-  private async say(text: string, opts: { from?: string; voice?: string } = {}): Promise<void> {
+  private async say(text: string, opts: { from?: string } = {}): Promise<void> {
     const out = join(this.opts.tmpDir, `say-${randomUUID()}.ogg`)
     const started = Date.now()
     try {
-      const result = await synthesize(text, out, { voice: opts.voice })
+      const result = await synthesize(text, out, { peer: opts.from })
       const ttsMs = Date.now() - started
       // Gate the mic for the whole playback: the runtime must not hear itself.
       this.muteUntil = Date.now() + 3_600_000
       this.segmenter.reset()
       try {
-        await playFile(result.path)
+        if (PLAYBACK_ENABLED) await playFile(result.path)
+        else this.opts.log('tts.silent', { from: opts.from, chars: text.length, path: result.path })
       } finally {
         this.muteUntil = Date.now() + ECHO_TAIL_MS
         this.segmenter.reset()
@@ -313,6 +501,7 @@ export class VoiceSession {
         chars: text.length,
         engine: result.engine,
         voice: result.voice,
+        voicedAs: result.peer,
         fallbackFrom: result.fallbackFrom,
         ttsMs,
         totalMs: Date.now() - started,
@@ -327,6 +516,30 @@ export class VoiceSession {
   }
 }
 
+/** Does this fragment sound like the middle of a sentence? Recognition punctuates what
+ *  it hears, so a fragment left hanging on a comma or a conjunction — or with no terminal
+ *  mark at all — is a thought still in progress, and the turn waits longer for the rest. */
+export function looksUnfinished(text: string): boolean {
+  const t = text.trim()
+  if (!t) return false
+  if (/[,;:—–-]$/.test(t)) return true
+  if (!/[.!?…]$/.test(t)) return true
+  const last = t.replace(/[.!?…]+$/, '').split(/\s+/).pop()?.toLowerCase() ?? ''
+  return ['и', 'а', 'но', 'что', 'чтобы', 'или', 'если', 'когда', 'потому', 'то'].includes(last)
+}
+
+/** Join the fragments of one thought back into a sentence. Recognition returns each
+ *  segment already punctuated and capitalized, so a plain space is the honest join —
+ *  the parts were one utterance and are re-read as one. */
+export function joinTurn(parts: readonly string[]): string {
+  return parts
+    .map(p => p.trim())
+    .filter(Boolean)
+    .join(' ')
+    .replace(/\s+/g, ' ')
+    .trim()
+}
+
 /** Prepare an agent's text for the speaker: strip what only makes sense on a screen and
  *  cap the length — a spoken wall of text is unusable. The modality frame already asks
  *  agents for short plain answers; this is the floor for when one forgets. */

package/src/wake.ts

@@ -7,8 +7,7 @@
 // was explicitly REJECTED — it couples the spoken vocabulary to the roster and multiplies
 // false triggers — so nothing here ever sweeps the fleet registry for addressable names.
 // The registry is consulted only for peers the operator ALREADY named in the map, to
-// enrich them with their own human name («наталья» → «наталья вячеславовна») and their
-// voice.
+// enrich them with their own human name («наталья» → «наталья вячеславовна»).
 //
 // Why the address is matched on the TRANSCRIPT rather than by an acoustic wake-word model
 // (openWakeWord et al.): an acoustic detector needs a trained model per phrase, and Фаза 3
@@ -29,8 +28,6 @@ export type FleetPeer = {
   personality: string
   intelligence?: string
   description?: string
-  /** Per-peer TTS voice from `interfaces.voice` — so each agent answers in its own voice. */
-  voice?: Record<string, string>
 }
 
 export type Address = {
@@ -109,10 +106,6 @@ export function readFleetPeers(path = join(iapeerRoot(), 'peers-profiles.json'))
         personality: String(p.personality ?? ''),
         intelligence: typeof p.intelligence === 'string' ? p.intelligence : undefined,
         description: typeof p.description === 'string' ? p.description : undefined,
-        voice:
-          p.interfaces && typeof p.interfaces === 'object' && (p.interfaces as any).voice
-            ? ((p.interfaces as any).voice as Record<string, string>)
-            : undefined,
       }))
       .filter(p => p.personality)
   } catch {
@@ -212,6 +205,10 @@ export type WakeMatch = {
   /** The utterance with the address (and any leading filler) stripped. */
   body: string
   alias: string
+  /** True when every word of the address matched letter for letter. A match that only
+   *  survived the one-edit tolerance is good enough to START a conversation, but not to
+   *  TAKE one away from the peer already in it — see the switch bar in voiceSession. */
+  exact: boolean
 }
 
 /** Does this transcript open by addressing a fleet peer? Longest alias wins, so
@@ -236,6 +233,7 @@ export function matchWake(transcript: string, book: Address[]): WakeMatch | null
     if (tokens.length < cand.words.length) continue
     const ok = cand.words.every((w, i) => tokenMatches(tokens[i], w))
     if (!ok) continue
+    const exact = cand.words.every((w, i) => tokens[i] === w)
     // Rebuild the body from the ORIGINAL transcript so punctuation and casing of the
     // message survive — only the address is stripped.
     const body = rawTokens
@@ -243,11 +241,31 @@ export function matchWake(transcript: string, book: Address[]): WakeMatch | null
       .join(' ')
       .replace(/^[\s,.!:;—–-]+/, '')
       .trim()
-    return { target: cand.target, alias: cand.alias, body }
+    return { target: cand.target, alias: cand.alias, body, exact }
   }
   return null
 }
 
+/** May this match TAKE an open conversation away from the peer currently in it?
+ *
+ *  Mid-dialogue the two errors are wildly asymmetric: one more line delivered to the peer
+ *  already listening is nearly free, while a mis-heard name silently hands the whole
+ *  conversation to another agent — the owner keeps talking to someone who is no longer
+ *  there. So a switch must look deliberate: the name heard letter for letter, followed by
+ *  a real request. Returns the objection, or null when the switch is allowed.
+ *
+ *  Every rule here is a live artefact, not a hypothesis: «да, различаю» came back from
+ *  recognition as «Дарвин, дарвин» and moved the conversation off boris (22.07). */
+export function objectionToSwitch(match: WakeMatch): string | null {
+  if (!match.exact) return 'inexact name match'
+  const body = normalizeSpoken(match.body)
+  if (!body) return 'name with no request'
+  const words = body.split(' ')
+  if (words.every(w => w === match.alias)) return 'body echoes the name'
+  if (words.length < 2) return 'request too short to be an address'
+  return null
+}
+
 /** Whisper-family recognizers emit stock phrases on near-silence («Продолжение
  *  следует…», «Субтитры сделал…»). They are not speech; drop them before matching so a
  *  quiet room cannot fabricate an utterance. */