@agfpd/voicetalk-runtime 0.3.3 → 0.4.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -7,6 +7,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.4.1] - 2026-10-26
11
+
12
+ ### Added
13
+
14
+ - `native.ready {target, state}` on start when the `iapeer voice status` preflight
15
+ answers and the target stays in native mode — a positive receipt, so a healthy start
16
+ is distinguishable from one that never ran the preflight.
17
+
18
+ ## [0.4.0] - 2026-10-26
19
+
20
+ ### Added
21
+
22
+ - Native voice hand-off (pilot, `interfaces.voicetalk.native_voice` or
23
+ `VOICETALK_NATIVE_VOICE`): an accepted spoken wake for a listed target releases the
24
+ microphone (capture exit confirmed) and asks iapeer to open the peer's native Codex
25
+ `/voice` (`iapeer voice open|status|close <peer> --json`). The microphone returns to
26
+ wake mode only on a confirmed `closed`; failed/timed-out opens and unreadable receipts
27
+ keep it released. One wake → one open; outbound speech and earcons are held while the
28
+ peer owns the room; a restart adopts a still-open voice. Start-up preflight: if
29
+ `iapeer voice status` gives no readable answer, the target degrades to the STT path
30
+ (`native.unavailable`) instead of a listener that could never get its mic back.
31
+ - `interface --native-voice <peers>`; `Microphone.pause()/resume()` (replay skips the
32
+ handed-off span); `scripts/replay-dialogue.sh` gains a fake `iapeer voice` and a
33
+ `native-voice.pcm` fixture.
34
+
10
35
  ## [0.3.3] - 2026-10-18
11
36
 
12
37
  ### Fixed
package/docs/README.md CHANGED
@@ -46,6 +46,37 @@ decoder prompt, because prompt text can be generated back on unrelated audio. A
46
46
  dialogue opens only on an exact leading address in the unprimed transcript; repeated
47
47
  token loops and name-only echoes are rejected before an IAP envelope exists.
48
48
 
49
+ ## Native voice — `native_voice` (pilot)
50
+
51
+ ```jsonc
52
+ { "native_voice": ["darwin"] } // or env VOICETALK_NATIVE_VOICE=darwin ("" disables)
53
+ ```
54
+
55
+ For a target listed here an accepted wake word does NOT start STT → envelope → TTS. It
56
+ hands the conversation to the peer's own realtime voice (Codex CLI `/voice`) inside its
57
+ live session, which iapeer opens and closes:
58
+
59
+ 1. The wake word must pass every fail-closed check above — native voice changes only
60
+ what an accepted activation does, never what counts as one.
61
+ 2. voicetalk kills its capture process and waits until it has actually exited (no
62
+ CoreAudio client left), plays the short "heard you" earcon while nobody is
63
+ listening, then runs `iapeer voice open <peer> --json`.
64
+ 3. While the peer holds the room voicetalk stays silent: no earcons, and any agent
65
+ answer that arrives is spoken only after the hand-off ends.
66
+ 4. voicetalk polls `iapeer voice status <peer> --json` (1 s) and takes the microphone
67
+ back ONLY on `state: closed`. A failed or timed-out open, an unreadable answer, and
68
+ `opening`/`closing` all keep the microphone released — a timeout is not evidence
69
+ that the peer stopped listening. Idle-close is iapeer's; after a 30-minute ceiling
70
+ (`VOICETALK_NATIVE_MAX_MS`) voicetalk asks `iapeer voice close`, and still waits for
71
+ the confirmed `closed`.
72
+ 5. Back in wake mode a soft "door closed" tone plays; the next turn needs a name again.
73
+ A runtime restart adopts a voice that is still open instead of grabbing the mic.
74
+ If at start iapeer does not answer `voice status` at all, the target falls back to
75
+ the STT path (`native.unavailable` in the log).
76
+
77
+ The words spoken after the name in the activating phrase are not forwarded; the
78
+ conversation itself happens in the peer's native voice. Other targets keep the STT path.
79
+
49
80
  ## How a turn works
50
81
 
51
82
  1. The microphone runs always-on; energy endpointing cuts an utterance when the
package/docs/ru/README.md CHANGED
@@ -46,6 +46,37 @@ prompt: такой контекст способен сам породить и
46
46
  открывает только точное имя в начале непримированного транскрипта; циклы повторяющихся
47
47
  токенов и тело из одних настроенных имён отбрасываются до появления IAP-конверта.
48
48
 
49
+ ## Нативный голос — `native_voice` (пилот)
50
+
51
+ ```jsonc
52
+ { "native_voice": ["darwin"] } // или env VOICETALK_NATIVE_VOICE=darwin ("" выключает)
53
+ ```
54
+
55
+ Для перечисленного адресата принятое wake-слово НЕ запускает STT → конверт → TTS, а
56
+ передаёт разговор в собственный realtime-голос пира (Codex CLI `/voice`) в его живой
57
+ сессии; открывает и закрывает голос iapeer.
58
+
59
+ 1. Wake-слово проходит все fail-closed проверки выше: режим меняет только то, что делает
60
+ принятая активация, но не то, что ею считается.
61
+ 2. voicetalk гасит процесс захвата и ждёт его фактического выхода (CoreAudio-клиента
62
+ нет), играет короткий earcon «услышал», пока никто не слушает, и вызывает
63
+ `iapeer voice open <peer> --json`.
64
+ 3. Пока комната у пира, voicetalk молчит: ни earcon'ов, а пришедший ответ агента
65
+ озвучивается только после возврата микрофона.
66
+ 4. voicetalk опрашивает `iapeer voice status <peer> --json` (1 с) и забирает микрофон
67
+ ТОЛЬКО по `state: closed`. Неудачный или просроченный open, нечитаемый ответ,
68
+ `opening`/`closing` — микрофон остаётся отпущенным: таймаут не доказывает, что пир
69
+ перестал слушать. Закрытие по простою — у iapeer; по потолку 30 минут
70
+ (`VOICETALK_NATIVE_MAX_MS`) voicetalk просит `iapeer voice close` и всё равно ждёт
71
+ подтверждённого `closed`.
72
+ 5. По возврату в wake-режим звучит мягкий тон «дверь закрыта»; следующий ход снова
73
+ требует имени. Рестарт рантайма принимает ещё открытый голос, а не хватает микрофон.
74
+ Если на старте iapeer вообще не отвечает на `voice status`, адресат уходит на прежний
75
+ STT-путь (`native.unavailable` в логе).
76
+
77
+ Слова после имени в активирующей фразе не пересылаются — разговор идёт уже нативным
78
+ голосом пира. Остальные адресаты — прежним STT-путём.
79
+
49
80
  ## Как устроен ход разговора
50
81
 
51
82
  1. Микрофон слушает постоянно; энергетический эндпойнтер закрывает реплику по паузе
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@agfpd/voicetalk-runtime",
3
- "version": "0.3.3",
3
+ "version": "0.4.1",
4
4
  "description": "Voice runtime-router for IAP peers.",
5
5
  "repository": {
6
6
  "type": "git",
package/src/audio.ts CHANGED
@@ -320,6 +320,9 @@ export class Microphone {
320
320
  private stopped = false
321
321
  private attempt = 0
322
322
  private timer: ReturnType<typeof setTimeout> | null = null
323
+ /** Replay position, kept across pause/resume (test seam only). */
324
+ private replayOffset = 0
325
+ private pausedAt = 0
323
326
 
324
327
  constructor(private readonly opts: MicOptions) {}
325
328
 
@@ -342,6 +345,37 @@ export class Microphone {
342
345
  return this.child !== null
343
346
  }
344
347
 
348
+ /** Release the input device and resolve only once the capture child has actually
349
+ * EXITED — a killed-but-not-yet-reaped ffmpeg still holds the CoreAudio client. This is
350
+ * the hand-off to a second listener (native voice): the other side may open the same
351
+ * physical microphone only after this resolves, or both would hear the room. */
352
+ async pause(capMs = 3_000): Promise<{ released: boolean }> {
353
+ const child = this.child
354
+ this.stop()
355
+ if (this.opts.filePath) this.pausedAt = Date.now()
356
+ if (!child || child.exitCode !== null || child.signalCode !== null) return { released: true }
357
+ return await new Promise(resolve => {
358
+ const timer = setTimeout(() => resolve({ released: false }), capMs)
359
+ child.once('close', () => {
360
+ clearTimeout(timer)
361
+ resolve({ released: true })
362
+ })
363
+ })
364
+ }
365
+
366
+ /** Take the device back after a pause. A replayed recording does not rewind: the audio
367
+ * that "played" while the device was handed off belongs to the other listener, so the
368
+ * replay skips ahead by the paused wall-clock time — exactly what a real microphone
369
+ * would have missed. */
370
+ resume(): void {
371
+ if (this.opts.filePath && this.pausedAt) {
372
+ const frames = Math.floor((Date.now() - this.pausedAt) / FRAME_MS)
373
+ this.replayOffset += frames * FRAME_BYTES
374
+ this.pausedAt = 0
375
+ }
376
+ this.start()
377
+ }
378
+
345
379
  private spawnChild(): void {
346
380
  if (this.stopped) return
347
381
  this.attempt += 1
@@ -397,9 +431,9 @@ export class Microphone {
397
431
  return
398
432
  }
399
433
  this.opts.onStart?.(this.attempt)
400
- let offset = 0
401
434
  const tick = () => {
402
435
  if (this.stopped) return
436
+ const offset = this.replayOffset
403
437
  if (offset >= pcm.length) {
404
438
  // Keep the stream alive with silence so compose/window timers still fire.
405
439
  this.opts.onData(Buffer.alloc(FRAME_BYTES))
@@ -407,7 +441,7 @@ export class Microphone {
407
441
  return
408
442
  }
409
443
  this.opts.onData(Buffer.from(pcm.subarray(offset, offset + FRAME_BYTES)))
410
- offset += FRAME_BYTES
444
+ this.replayOffset = offset + FRAME_BYTES
411
445
  this.timer = setTimeout(tick, FRAME_MS)
412
446
  }
413
447
  tick()
package/src/cli.ts CHANGED
@@ -45,6 +45,7 @@ import { selfInstall } from './selfInstall.ts'
45
45
  import { parseWakeWords, runSelfConfig } from './selfConfig.ts'
46
46
  import { writeJsonAtomic } from './fsAtomic.ts'
47
47
  import { VoiceSession } from './voiceSession.ts'
48
+ import { cliNativeVoice, nativeVoiceTargets } from './nativeVoice.ts'
48
49
  import { voiceHealth } from './voiceConnect.ts'
49
50
  import { listInputDevices, resolveInputDevice } from './audio.ts'
50
51
  import { buildAddressBook, unresolvedWakeWords, wakeEntries, type VoicetalkBinding } from './wake.ts'
@@ -163,7 +164,7 @@ function usage(): string {
163
164
  ${BIN_NAME} # self-install (npx contract): bin on PATH + manifest
164
165
  ${BIN_NAME} self-install # explicit self-install (idempotent)
165
166
  ${BIN_NAME} self-config # per-peer self-config hook (foundation-invoked)
166
- ${BIN_NAME} interface --wake-words «дарвин=darwin,борис=boris» [--default-target <peer>] [--device <d>]
167
+ ${BIN_NAME} interface --wake-words «дарвин=darwin,борис=boris» [--default-target <peer>] [--device <d>] [--native-voice darwin]
167
168
  # set this peer's voicetalk binding (wake map)
168
169
  ${BIN_NAME} run # router loop: live mic ↔ speaker (VOICETALK_AUDIO=0 → text only)
169
170
  ${BIN_NAME} say --peer <personality> <text…> # inject an utterance without the mic (debug seam)
@@ -400,6 +401,7 @@ function setVoicetalkInterface(cwd: string, patch: VoicetalkInterface): Record<s
400
401
  if (patch.wake_words !== undefined) voicetalk.wake_words = patch.wake_words
401
402
  if (patch.default_target !== undefined) voicetalk.default_target = patch.default_target
402
403
  if (patch.device !== undefined) voicetalk.device = patch.device
404
+ if (patch.native_voice !== undefined) voicetalk.native_voice = patch.native_voice
403
405
  ;(interfaces as Record<string, unknown>).voicetalk = voicetalk
404
406
  const merged = { ...raw, interfaces }
405
407
  writeJsonAtomic(path, merged)
@@ -972,6 +974,11 @@ async function runCommand(extraArgs: string[]): Promise<void> {
972
974
  deliver: (target, text) => deliverUtterance(ctx, target, text),
973
975
  log: logEvt,
974
976
  tmpDir: join(globalVoicetalkRoot(), 'tmp'),
977
+ // Native voice is opt-in per target; with no target configured the door is not
978
+ // even built, so the STT path is untouched for everyone.
979
+ ...(nativeVoiceTargets(binding).size
980
+ ? { nativeVoice: cliNativeVoice({ bin: ctx.iapBin, env: iapEnv(owner), cwd: ctx.cwd }) }
981
+ : {}),
975
982
  })
976
983
  try {
977
984
  const started = await candidate.start()
@@ -1013,6 +1020,7 @@ async function runCommand(extraArgs: string[]): Promise<void> {
1013
1020
  owner: owner.personality,
1014
1021
  wake_words: wakeEntries(binding).map(([w]) => w),
1015
1022
  device: binding.device ?? null,
1023
+ native_voice: [...nativeVoiceTargets(binding)],
1016
1024
  iapBin: ctx.iapBin,
1017
1025
  inbound_socket: inboundSocketPath(owner.personality),
1018
1026
  audio: session ? { live: true, ...audio } : { live: false, ...audio },
@@ -1115,9 +1123,17 @@ async function interfaceCommand(args: string[]): Promise<void> {
1115
1123
  const wakeWords = parseWakeWords(stringFlag(flags, 'wake-words'))
1116
1124
  const defaultTarget = stringFlag(flags, 'default-target')
1117
1125
  const device = stringFlag(flags, 'device')
1118
- if (wakeWord === undefined && wakeWords === undefined && defaultTarget === undefined && device === undefined) {
1126
+ const nativeFlag = stringFlag(flags, 'native-voice')
1127
+ const nativeVoice = nativeFlag === undefined ? undefined : nativeFlag.split(',').map(v => v.trim()).filter(Boolean)
1128
+ if (
1129
+ wakeWord === undefined &&
1130
+ wakeWords === undefined &&
1131
+ defaultTarget === undefined &&
1132
+ device === undefined &&
1133
+ nativeVoice === undefined
1134
+ ) {
1119
1135
  throw new VoicetalkRuntimeError(
1120
- 'interface requires --wake-words <«слово=пир,…»>, --default-target <peer>, --wake-word <w> and/or --device <d>',
1136
+ 'interface requires --wake-words <«слово=пир,…»>, --default-target <peer>, --wake-word <w>, --device <d> and/or --native-voice <peers>',
1121
1137
  )
1122
1138
  }
1123
1139
  ensureCurrentProfile() // assert the profile exists before patching
@@ -1126,6 +1142,7 @@ async function interfaceCommand(args: string[]): Promise<void> {
1126
1142
  ...(wakeWords !== undefined ? { wake_words: wakeWords } : {}),
1127
1143
  ...(defaultTarget !== undefined ? { default_target: defaultTarget } : {}),
1128
1144
  ...(device !== undefined ? { device } : {}),
1145
+ ...(nativeVoice !== undefined ? { native_voice: nativeVoice } : {}),
1129
1146
  })
1130
1147
  process.stdout.write(`${peerProfilePath(process.cwd())}\n`)
1131
1148
  printJson(merged)
@@ -0,0 +1,119 @@
1
+ // nativeVoice.ts — the door to a peer's NATIVE voice mode (Codex CLI `/voice`).
2
+ //
3
+ // For a configured target the spoken wake word does not start STT → envelope → TTS; it
4
+ // hands the conversation to the peer's own realtime voice inside its live session. The
5
+ // session belongs to iapeer (the pty supervisor), so voicetalk only REQUESTS and OBSERVES
6
+ // over a small CLI contract agreed with the iapeer Implementer (topic codex-voice-wake):
7
+ //
8
+ // iapeer voice open <peer> --json → {"ok":true,"state":"open","session":"…","already":bool}
9
+ // iapeer voice status <peer> --json → {"state":"open|opening|closing|closed","session":…,"reason"?:…}
10
+ // iapeer voice close <peer> --json → {"ok":true,"state":"closed"}
11
+ //
12
+ // open returns only once the realtime session is confirmed started, close once it is
13
+ // confirmed closed; status is what the listener polls before it takes the microphone back.
14
+ // Idle-close lives in iapeer — voicetalk never decides a conversation is over, it only
15
+ // waits for the confirmation that it is.
16
+
17
+ import { spawn } from 'child_process'
18
+
19
+ export type NativeVoiceState = 'open' | 'opening' | 'closing' | 'closed' | 'unknown'
20
+
21
+ export type NativeVoiceReceipt = {
22
+ ok: boolean
23
+ state: NativeVoiceState
24
+ session?: string | null
25
+ already?: boolean
26
+ reason?: string
27
+ error?: string
28
+ /** Raw stdout/stderr tail for the log when the answer did not parse. */
29
+ raw?: string
30
+ }
31
+
32
+ export type NativeVoiceControl = {
33
+ open(peer: string): Promise<NativeVoiceReceipt>
34
+ status(peer: string): Promise<NativeVoiceReceipt>
35
+ close(peer: string): Promise<NativeVoiceReceipt>
36
+ }
37
+
38
+ const STATES = new Set(['open', 'opening', 'closing', 'closed'])
39
+
40
+ /** Parse the LAST JSON object line of a contract answer. Anything else is not a receipt:
41
+ * a confirmation that cannot be read is treated as no confirmation. */
42
+ export function parseVoiceReceipt(stdout: string, stderr: string, exitCode: number | null): NativeVoiceReceipt {
43
+ const lines = `${stdout}`.split(/\r?\n/).map(l => l.trim()).filter(l => l.startsWith('{') && l.endsWith('}'))
44
+ for (const line of lines.reverse()) {
45
+ try {
46
+ const obj = JSON.parse(line) as Record<string, unknown>
47
+ const state = typeof obj.state === 'string' && STATES.has(obj.state) ? (obj.state as NativeVoiceState) : 'unknown'
48
+ const errorText = [obj.error, obj.err, obj.detail].find(v => typeof v === 'string') as string | undefined
49
+ return {
50
+ ok: obj.ok === false ? false : exitCode === 0,
51
+ state,
52
+ session: typeof obj.session === 'string' ? obj.session : null,
53
+ already: obj.already === true,
54
+ reason: typeof obj.reason === 'string' ? obj.reason : undefined,
55
+ error: exitCode === 0 && obj.ok !== false ? undefined : (errorText ?? `exit ${exitCode}`),
56
+ }
57
+ } catch {}
58
+ }
59
+ const raw = `${stdout}\n${stderr}`.trim().slice(-300)
60
+ return { ok: false, state: 'unknown', error: raw || `exit ${exitCode}`, raw }
61
+ }
62
+
63
+ /** The CLI implementation of the contract. Never throws; a timeout is a failed receipt. */
64
+ export function cliNativeVoice(opts: {
65
+ bin: string
66
+ env?: NodeJS.ProcessEnv
67
+ cwd?: string
68
+ openTimeoutMs?: number
69
+ statusTimeoutMs?: number
70
+ closeTimeoutMs?: number
71
+ }): NativeVoiceControl {
72
+ const run = (verb: string, peer: string, timeoutMs: number) =>
73
+ new Promise<NativeVoiceReceipt>(resolve => {
74
+ let stdout = ''
75
+ let stderr = ''
76
+ let settled = false
77
+ const finish = (r: NativeVoiceReceipt) => {
78
+ if (settled) return
79
+ settled = true
80
+ clearTimeout(timer)
81
+ resolve(r)
82
+ }
83
+ let child: ReturnType<typeof spawn>
84
+ try {
85
+ child = spawn(opts.bin, ['voice', verb, peer, '--json'], {
86
+ cwd: opts.cwd,
87
+ env: opts.env ?? process.env,
88
+ stdio: ['ignore', 'pipe', 'pipe'],
89
+ })
90
+ } catch (err) {
91
+ resolve({ ok: false, state: 'unknown', error: err instanceof Error ? err.message : String(err) })
92
+ return
93
+ }
94
+ const timer = setTimeout(() => {
95
+ try {
96
+ child.kill('SIGKILL')
97
+ } catch {}
98
+ finish({ ok: false, state: 'unknown', error: `voice ${verb} timed out after ${Math.ceil(timeoutMs / 1000)}s` })
99
+ }, timeoutMs)
100
+ child.stdout?.on('data', d => (stdout += String(d)))
101
+ child.stderr?.on('data', d => (stderr += String(d)))
102
+ child.on('error', err => finish({ ok: false, state: 'unknown', error: err.message }))
103
+ child.on('close', code => finish(parseVoiceReceipt(stdout, stderr, code)))
104
+ })
105
+ return {
106
+ open: peer => run('open', peer, opts.openTimeoutMs ?? 20_000),
107
+ status: peer => run('status', peer, opts.statusTimeoutMs ?? 5_000),
108
+ close: peer => run('close', peer, opts.closeTimeoutMs ?? 15_000),
109
+ }
110
+ }
111
+
112
+ /** Which targets are answered in native voice: the passport's `native_voice` list, or the
113
+ * `VOICETALK_NATIVE_VOICE` override (comma list; empty string disables). */
114
+ export function nativeVoiceTargets(binding: { native_voice?: unknown }, env = process.env): Set<string> {
115
+ const override = env.VOICETALK_NATIVE_VOICE
116
+ const source: unknown = override !== undefined ? override.split(',') : binding.native_voice
117
+ const list = Array.isArray(source) ? source : typeof source === 'string' ? source.split(',') : []
118
+ return new Set(list.map(v => String(v).trim()).filter(Boolean))
119
+ }
@@ -43,6 +43,7 @@ import {
43
43
  type Segment,
44
44
  } from './audio.ts'
45
45
  import { synthesize, transcribe } from './voiceConnect.ts'
46
+ import { nativeVoiceTargets, type NativeVoiceControl } from './nativeVoice.ts'
46
47
  import {
47
48
  buildAddressBook,
48
49
  hallucinationReason,
@@ -104,6 +105,11 @@ const PLAYBACK_ENABLED = process.env.VOICETALK_PLAYBACK !== '0'
104
105
  * are mode 0600 under the runtime scratch dir and the oldest pair is evicted. */
105
106
  const WAKE_CAPTURE_ENABLED = process.env.VOICETALK_WAKE_CAPTURE !== '0'
106
107
  const WAKE_CAPTURE_MAX = Math.min(100, num(process.env.VOICETALK_WAKE_CAPTURE_MAX, 16))
108
+ /** Native voice hand-off: how often the listener asks whether the peer's voice is closed,
109
+ * and the hard ceiling after which it asks iapeer to close it. Idle-close itself belongs
110
+ * to iapeer; the ceiling only bounds a hand-off whose closing was never confirmed. */
111
+ const NATIVE_POLL_MS = num(process.env.VOICETALK_NATIVE_POLL_MS, 1_000)
112
+ const NATIVE_MAX_MS = num(process.env.VOICETALK_NATIVE_MAX_MS, 30 * 60_000)
107
113
 
108
114
  export type DeliverFn = (target: string, text: string) => Promise<{ ok: boolean; detail?: string }>
109
115
  export type LogFn = (evt: string, fields: Record<string, unknown>) => void
@@ -116,6 +122,21 @@ export type VoiceSessionOptions = {
116
122
  /** Scratch dir for captured WAVs and synthesized OGGs. */
117
123
  tmpDir: string
118
124
  peers?: FleetPeer[]
125
+ /** The door to a peer's native voice mode; absent = every target goes through STT. */
126
+ nativeVoice?: NativeVoiceControl
127
+ /** Override the hand-off poll / hold ceiling (tests); defaults come from the env. */
128
+ nativePollMs?: number
129
+ nativeMaxMs?: number
130
+ }
131
+
132
+ /** The microphone is lent to a peer's native voice. `releasing` and `reclaiming` are the
133
+ * edges where nobody listens (earcons are allowed there); during `opening` and `open`
134
+ * the room belongs to the peer and voicetalk stays silent. */
135
+ type NativeHandOff = {
136
+ target: string
137
+ phase: 'releasing' | 'opening' | 'open' | 'reclaiming'
138
+ since: number
139
+ session?: string | null
119
140
  }
120
141
 
121
142
  type WakeRoutingDecision = {
@@ -153,8 +174,18 @@ export class VoiceSession {
153
174
  /** When the current turn was handed to the fleet — the clock for «ответ за N с». */
154
175
  private deliveredAt = new Map<string, number>()
155
176
  private stopped = false
177
+ private nativeTargets: Set<string>
178
+ private native: NativeHandOff | null = null
179
+ private nativeWaiters: Array<() => void> = []
180
+
181
+ constructor(private readonly opts: VoiceSessionOptions) {
182
+ this.nativeTargets = opts.nativeVoice ? nativeVoiceTargets(opts.binding) : new Set()
183
+ }
156
184
 
157
- constructor(private readonly opts: VoiceSessionOptions) {}
185
+ /** Who currently holds the microphone instead of the wake listener (null = we do). */
186
+ get nativeHolder(): { target: string; phase: string } | null {
187
+ return this.native ? { target: this.native.target, phase: this.native.phase } : null
188
+ }
158
189
 
159
190
  /** Resolve the input device, prime the address book, open the mic. Throws if the mic
160
191
  * cannot be resolved — the caller decides whether that is fatal (it is not: the
@@ -173,7 +204,10 @@ export class VoiceSession {
173
204
  onStart: attempt => this.opts.log('mic.start', { device: device.index, name: device.name, attempt }),
174
205
  onError: detail => this.opts.log('mic.error', { detail }),
175
206
  })
176
- this.mic.start()
207
+ // A restart must not grab a microphone a peer is still talking on: adopt an open
208
+ // native voice and wait for its close before listening.
209
+ const adopted = await this.adoptOpenNativeVoice()
210
+ if (!adopted) this.mic.start()
177
211
  return { deviceIndex: device.index, deviceName: device.name, addresses: this.book.length }
178
212
  }
179
213
 
@@ -216,6 +250,9 @@ export class VoiceSession {
216
250
 
217
251
  private async earcon(kind: keyof Earcons): Promise<void> {
218
252
  if (!this.earcons || process.env.VOICETALK_EARCONS === '0' || !PLAYBACK_ENABLED) return
253
+ // The peer's own voice session is listening to the room — a beep from us would be
254
+ // heard (and possibly answered) by it.
255
+ if (this.native && (this.native.phase === 'opening' || this.native.phase === 'open')) return
219
256
  try {
220
257
  this.muteUntil = Math.max(this.muteUntil, Date.now() + 600)
221
258
  await playFile(this.earcons[kind], { timeoutMs: 5_000 })
@@ -229,7 +266,7 @@ export class VoiceSession {
229
266
  }
230
267
 
231
268
  private onAudio(chunk: Buffer): void {
232
- if (this.stopped) return
269
+ if (this.stopped || this.native) return
233
270
  if (this.muted()) {
234
271
  // Our own speech (or an earcon) is in the room — throw the audio away rather than
235
272
  // letting the endpointer build an utterance out of it.
@@ -439,10 +476,18 @@ export class VoiceSession {
439
476
  // goes out before the new one starts.
440
477
  this.opts.log('wake.switch', { from: active, to: match!.target })
441
478
  this.flush('switch')
442
- this.append(match!.target, match!.body)
479
+ if (this.nativeTargets.has(match!.target)) this.handOff(match!)
480
+ else this.append(match!.target, match!.body)
443
481
  return { candidate: match!, accepted: true }
444
482
  }
445
483
 
484
+ if (match && this.nativeTargets.has(match.target)) {
485
+ // An accepted, spoken, exact address to a native-voice peer: the conversation moves
486
+ // to the peer's own realtime voice. Nothing is transcribed onward or sent as text.
487
+ this.handOff(match)
488
+ return { candidate: match, accepted: true }
489
+ }
490
+
446
491
  if (match) {
447
492
  if (!match.body) {
448
493
  // Addressed with nothing to say yet («Борис?») — acknowledge and hold the mic
@@ -569,6 +614,155 @@ export class VoiceSession {
569
614
  this.followTimer = null
570
615
  }
571
616
 
617
+ // ── native voice hand-off ─────────────────────────────────────────────────────
618
+
619
+ /** One accepted wake → one hand-off. Synchronously marks the microphone as lent (so no
620
+ * second wake can be formed from audio already captured), then runs the hand-off. */
621
+ private handOff(match: WakeMatch): void {
622
+ if (this.native) {
623
+ this.opts.log('native.ignored', { target: match.target, holder: this.native.target, phase: this.native.phase })
624
+ return
625
+ }
626
+ this.native = { target: match.target, phase: 'releasing', since: Date.now() }
627
+ // Audio captured before the wake was handled belongs to no one now: the owner is
628
+ // about to talk to the peer directly. A thought in flight to someone ELSE still goes.
629
+ const dropped = this.queue.length
630
+ this.queue = []
631
+ if (this.pending) this.flush('native-voice')
632
+ this.clearWindow()
633
+ this.followTarget = null
634
+ this.opts.log('native.wake', { target: match.target, alias: match.alias, body: match.body || null, droppedSegments: dropped })
635
+ void this.runHandOff(match.target)
636
+ }
637
+
638
+ private async runHandOff(target: string): Promise<void> {
639
+ const control = this.opts.nativeVoice!
640
+ const released = this.mic ? await this.mic.pause() : { released: true }
641
+ this.segmenter.reset()
642
+ this.opts.log('mic.released', { to: target, confirmed: released.released })
643
+ if (!released.released) {
644
+ // Our capture has not provably let go of the device; opening the peer's voice now
645
+ // could put two listeners on one microphone. Abort this activation instead.
646
+ this.native!.phase = 'reclaiming'
647
+ await this.earcon('error')
648
+ this.reclaim(target, 'release-unconfirmed')
649
+ return
650
+ }
651
+ // "Heard you" — played while NOBODY is listening: our capture is gone and the peer's
652
+ // voice is not open yet.
653
+ await this.earcon('end')
654
+ if (this.stopped) return
655
+ this.native!.phase = 'opening'
656
+ this.opts.log('native.open-requested', { target })
657
+ const started = Date.now()
658
+ const receipt = await control.open(target)
659
+ if (!receipt.ok || receipt.state !== 'open') {
660
+ // A failed open is NOT evidence that the peer is off the microphone: the realtime
661
+ // session may still start late, and a timeout or an unreadable answer says nothing
662
+ // at all. The microphone comes back only on a confirmed `closed` (contract with
663
+ // iapeer, 26.10) — until then we hold exactly as if the voice were open.
664
+ this.opts.log('native.open-failed', { target, state: receipt.state, error: receipt.error, openMs: Date.now() - started })
665
+ this.native!.phase = 'open'
666
+ await this.holdUntilClosed(target, { failedOpen: true })
667
+ return
668
+ }
669
+ this.native!.phase = 'open'
670
+ this.native!.session = receipt.session ?? null
671
+ this.opts.log('native.opened', { target, session: receipt.session ?? null, already: !!receipt.already, openMs: Date.now() - started })
672
+ await this.holdUntilClosed(target)
673
+ }
674
+
675
+ /** Poll the peer's voice state until iapeer confirms it is closed, then give the
676
+ * microphone back to the wake listener. */
677
+ private async holdUntilClosed(target: string, opts: { failedOpen?: boolean } = {}): Promise<void> {
678
+ const control = this.opts.nativeVoice!
679
+ const openedAt = Date.now()
680
+ let closeRequested = false
681
+ let lastAlarm = Date.now()
682
+ let lastState = ''
683
+ while (!this.stopped) {
684
+ await new Promise(r => setTimeout(r, this.opts.nativePollMs ?? NATIVE_POLL_MS))
685
+ if (this.stopped) return
686
+ const st = await control.status(target)
687
+ if (st.state !== lastState) {
688
+ this.opts.log('native.status', { target, state: st.state, reason: st.reason ?? null, error: st.error ?? null })
689
+ lastState = st.state
690
+ }
691
+ if (st.state === 'closed') {
692
+ this.opts.log('native.closed', { target, session: st.session ?? this.native?.session ?? null, reason: st.reason ?? null, heldMs: Date.now() - openedAt })
693
+ break
694
+ }
695
+ if (!closeRequested && Date.now() - openedAt >= (this.opts.nativeMaxMs ?? NATIVE_MAX_MS)) {
696
+ closeRequested = true
697
+ this.opts.log('native.close-requested', { target, reason: 'hold ceiling', heldMs: Date.now() - openedAt })
698
+ const closed = await control.close(target)
699
+ this.opts.log('native.close-receipt', { target, ok: closed.ok, state: closed.state, error: closed.error ?? null })
700
+ }
701
+ if (closeRequested && Date.now() - lastAlarm >= 60_000) {
702
+ // Past the ceiling and still no confirmed close. Taking the microphone back
703
+ // unconfirmed would put two listeners on one device; staying deaf is the
704
+ // recoverable failure, so it is reported loudly instead of resolved by guess.
705
+ lastAlarm = Date.now()
706
+ this.opts.log('native.stuck', { target, lastState, heldMs: Date.now() - openedAt })
707
+ const closed = await control.close(target)
708
+ this.opts.log('native.close-receipt', { target, ok: closed.ok, state: closed.state, error: closed.error ?? null })
709
+ }
710
+ }
711
+ if (this.stopped) return
712
+ this.native!.phase = 'reclaiming'
713
+ if (opts.failedOpen) await this.earcon('error')
714
+ this.reclaim(target, opts.failedOpen ? 'open-failed' : 'closed')
715
+ if (!opts.failedOpen) void this.earcon('close')
716
+ }
717
+
718
+ private reclaim(target: string, reason: string): void {
719
+ this.segmenter.reset()
720
+ this.native = null
721
+ // Back to wake mode proper: after a native conversation the next turn needs a name.
722
+ this.clearWindow()
723
+ this.followTarget = null
724
+ this.mic?.resume()
725
+ this.opts.log('wake.listening', { after: target, reason })
726
+ const waiters = this.nativeWaiters
727
+ this.nativeWaiters = []
728
+ for (const w of waiters) w()
729
+ }
730
+
731
+ /** On start: if a native voice is already open (the runtime restarted mid-conversation),
732
+ * stay off the microphone until it closes. */
733
+ private async adoptOpenNativeVoice(): Promise<boolean> {
734
+ const control = this.opts.nativeVoice
735
+ if (!control) return false
736
+ for (const target of [...this.nativeTargets]) {
737
+ const st = await control.status(target)
738
+ if (st.state === 'unknown') {
739
+ // Preflight: the contract does not answer at all (an iapeer without `voice`, a
740
+ // broken CLI). Nothing can have been opened by THIS process yet, so the safe
741
+ // degradation is the STT path for this target — not a listener that goes deaf
742
+ // forever on the first wake because no `closed` can ever be confirmed.
743
+ this.nativeTargets.delete(target)
744
+ this.opts.log('native.unavailable', { target, error: st.error ?? null, fallback: 'stt' })
745
+ continue
746
+ }
747
+ if (st.state === 'open' || st.state === 'opening' || st.state === 'closing') {
748
+ this.native = { target, phase: 'open', since: Date.now(), session: st.session ?? null }
749
+ this.opts.log('native.adopted', { target, state: st.state, session: st.session ?? null })
750
+ void this.holdUntilClosed(target)
751
+ return true
752
+ }
753
+ // Positive preflight receipt: the contract answered and the target is live in
754
+ // native mode. Without this line a healthy start is indistinguishable from one that
755
+ // never ran the preflight.
756
+ this.opts.log('native.ready', { target, state: st.state })
757
+ }
758
+ return false
759
+ }
760
+
761
+ private waitNativeIdle(): Promise<void> {
762
+ if (!this.native) return Promise.resolve()
763
+ return new Promise(resolve => this.nativeWaiters.push(resolve))
764
+ }
765
+
572
766
  /** Speak an agent's answer on the owner's speaker, then hold the follow-up window
573
767
  * open — the conversational turn passes back to the human. */
574
768
  async speak(from: string, message: string): Promise<void> {
@@ -596,6 +790,11 @@ export class VoiceSession {
596
790
  }
597
791
 
598
792
  private async say(text: string, opts: { from?: string } = {}): Promise<void> {
793
+ if (this.native) {
794
+ // The room belongs to a peer's native voice; speaking now would be heard by it.
795
+ this.opts.log('tts.deferred', { from: opts.from ?? null, holder: this.native.target })
796
+ await this.waitNativeIdle()
797
+ }
599
798
  const out = join(this.opts.tmpDir, `say-${randomUUID()}.ogg`)
600
799
  const started = Date.now()
601
800
  try {
package/src/wake.ts CHANGED
@@ -120,6 +120,10 @@ export type VoicetalkBinding = {
120
120
  wake_words?: Record<string, string> | string[]
121
121
  default_target?: string
122
122
  device?: string
123
+ /** Targets answered in their own NATIVE voice mode (Codex `/voice`) instead of
124
+ * STT → envelope → TTS. The wake word still has to be spoken and pass every activation
125
+ * check — this only changes what an accepted activation does. */
126
+ native_voice?: string[]
123
127
  }
124
128
 
125
129
  export type AddressBookOptions = {