@gotcos/glasses-server 6.61.0 → 6.61.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +13 -0
- package/README.md +3 -2
- package/package.json +1 -1
- package/server/lib/nemotron-final.ts +34 -8
- package/server/lib/nemotron-live.ts +58 -15
- package/server/lib/speaker-embeddings.ts +2 -1
- package/server/lib/voiceprint-floor.ts +11 -3
- package/server/routes/transcribe-stream.ts +7 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,16 @@
|
|
|
1
|
+
## 6.61.1
|
|
2
|
+
|
|
3
|
+
### Fixes 6.61.0 labelling other people as the wearer: a split track is named only by a confident voiceprint, never by the glasses' default label
|
|
4
|
+
|
|
5
|
+
- Miles, 2026-10-02 10:19, in a live meeting on 6.61.0: "MU is being assigned to loudest speaker vs my actual voice signature ... I saw Jeremy Sokolic voice showing up as MU." In that session (`meeting_1790953219564_yhwyxs`, read from a copy), 40 turns read MU. 25 were genuine (voiceprint at 0.65 or more, Even Hub `self`). 11 had similarity 0 in two-track chunks where Even Hub heard mostly someone else ("Okay.", "I can send a message. Yeah.", "to have as we play into calendar."): a short second track took the client's label, and the client's label is the glasses' default wearer label (`MU`), not evidence of who spoke. 2 more ("Of course. All right." at 0.612, and 0.625) were a track's own voiceprint check returning the wearer from the full search, which accepts any profile from 0.55, below the 0.65 at which the voiceprint verifies the wearer.
|
|
6
|
+
- **The client's label never names a track.** In a split chunk, a short track that is not the dominant one is `Ext`. The client's label is still what 6.60.0 used it for: a whole chunk no voiceprint could run on (under 2 s, or no voiceprint model). A one-track chunk is labelled exactly as 6.60.0 labelled the chunk.
|
|
7
|
+
- **The wearer needs owner-verify confidence in a split chunk.** A track named the wearer (`owner_speaker_label`) by its own voiceprint check, or a short dominant track that would take the whole chunk's voiceprint, keeps the wearer's name only at `OWNER_VERIFY_SIMILARITY` (0.65, now one constant shared with the voiceprint's owner verify); between 0.55 and 0.65 it is `Ext`. Any other name keeps the search threshold, as before. The two 0.665 and 0.716 turns where Even Hub heard mostly someone else are above the floor and stay, as reviewed.
|
|
8
|
+
- **The final pass applies the same rule.** Its channel map was already built only from chunks at 0.65 or more. Its per-chunk fallback (the live turn a word sat in, else the chunk's label) now treats a wearer name in a split chunk below 0.65 as `Ext`, so a 6.61.0 turn like these is never carried into a final transcript.
|
|
9
|
+
- **Logged and counted.** Every chunk where a would-be wearer name was refused logs one line, `[diarizer] owner-guard chunk #N refused=K client_label=X weak_owner=Y`; the chunk's `diarizer.ownerGuard` lists the reasons, and `readiness.diarizer.ownerGuard` counts them since start.
|
|
10
|
+
- **Replayed on the field session (read-only copies of its 195 Nemotron chunks and their audio, the real CLI and the real voiceprint):** all 28 genuine wearer turns (0.65 or more, Even Hub `self`) stay MU; 0 of the 11 similarity-0 MU turns and 0 of the 2 weak ones (0.612, 0.625) are MU any more; the 2 borderline turns stay MU; the guard refused 13 client labels and 2 weak wearer matches; no chunk fell back.
|
|
11
|
+
- Tests: the field chunks rebuilt from their records ("Okay." with Even Hub self 19/121, "I can send a message. Yeah." at self 3/125 with Kyle keeping his own track, "Of course. All right." at 0.612), the wearer staying the wearer in a one-track chunk (0.703) and a split one (0.716), a weak wearer match refused on both paths (the whole-chunk voiceprint a short dominant turn takes, and a track's own check), the client label never naming a track, the owner-guard log line and count, the route passing `owner_speaker_label`, and the final pass refusing a 6.61.0 similarity-0 MU turn while keeping a confident one. The mutation gate adds a case for each guard and retargets the three 6.61.0 cases the guard replaced: baseline 169 tests in 11 files, 104 of 104 killed. Full suite 5,723 passed, 2 skipped.
|
|
12
|
+
- No Install hooks and no COS Control change.
|
|
13
|
+
|
|
1
14
|
## 6.61.0
|
|
2
15
|
|
|
3
16
|
### Nemotron is the default diarizer, for real: it splits two-voice chunks on the HUD during the call and in the final transcript, and the voiceprint names each voice
|
package/README.md
CHANGED
|
@@ -597,8 +597,9 @@ is split into turns instead of carrying one label.
|
|
|
597
597
|
`segments: [{ speaker, text, startSec, endSec, similarity }]` in time order;
|
|
598
598
|
their texts joined with one space equal `text`. A track with at least 2 s of
|
|
599
599
|
its own audio is named by the voiceprint on that audio; a shorter one takes the
|
|
600
|
-
chunk's voiceprint when it is the dominant voice, otherwise
|
|
601
|
-
|
|
600
|
+
chunk's voiceprint when it is the dominant voice, otherwise it is `Ext`. The
|
|
601
|
+
glasses' default wearer label never names a track, and in a split chunk the
|
|
602
|
+
wearer is named only at owner-verify confidence (0.65; since 6.61.1). The
|
|
602
603
|
segments are stored on the chunk, so the saved meeting keeps them.
|
|
603
604
|
- **The final transcript.** When a meeting is saved, one Nemotron pass over the
|
|
604
605
|
whole meeting maps each channel to a name from the chunks the voiceprint was
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@gotcos/glasses-server",
|
|
3
|
-
"version": "6.61.
|
|
3
|
+
"version": "6.61.1",
|
|
4
4
|
"description": "COS Glasses: self-hosted AI heads-up-display server for Even G2 smart glasses, powered by Claude Code, Codex, Cursor Agent CLI, or local Ollama",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -41,6 +41,8 @@ import {
|
|
|
41
41
|
prepareNemotronModelsDir,
|
|
42
42
|
} from './diarizer-backend.js'
|
|
43
43
|
import { CONFIDENT_SIMILARITY, isUnattributed } from './meeting-speaker-review.js'
|
|
44
|
+
import { getOwnerSpeakerLabel } from './profile.js'
|
|
45
|
+
import { OWNER_VERIFY_SIMILARITY } from './voiceprint-floor.js'
|
|
44
46
|
import {
|
|
45
47
|
NEMOTRON_TMP_PREFIX,
|
|
46
48
|
NemotronRunError,
|
|
@@ -265,6 +267,10 @@ export function crowdIdentities(windows: TimelineWindow[]): number {
|
|
|
265
267
|
return [...counts.values()].filter(count => count >= FINAL_MIN_SUPPORT).length
|
|
266
268
|
}
|
|
267
269
|
|
|
270
|
+
function ownerLabelOrNull(): string | null {
|
|
271
|
+
try { return getOwnerSpeakerLabel() } catch { return null }
|
|
272
|
+
}
|
|
273
|
+
|
|
268
274
|
function isName(label: unknown): label is string {
|
|
269
275
|
return typeof label === 'string' && !isUnattributed(label.trim())
|
|
270
276
|
}
|
|
@@ -332,7 +338,21 @@ export function mapChannelsToNames(windows: TimelineWindow[], a: Activity, optio
|
|
|
332
338
|
}
|
|
333
339
|
|
|
334
340
|
/** The live segment that held chunk-relative time t, else null. */
|
|
335
|
-
|
|
341
|
+
/**
|
|
342
|
+
* 6.61.1 owner guard for per-chunk names. 6.61.0 named split tracks as the wearer
|
|
343
|
+
* from the glasses' default label (similarity 0) or a weak search match
|
|
344
|
+
* (0.55 to 0.65). In a chunk with more than one live turn, the wearer's name
|
|
345
|
+
* counts only at owner-verify confidence; otherwise it is Ext. A one-turn chunk
|
|
346
|
+
* keeps its label exactly as 6.60.0 gave it.
|
|
347
|
+
*/
|
|
348
|
+
export function guardOwnerName(name: string, similarity: unknown, chunk: SidecarChunk, owner: string | null | undefined): string {
|
|
349
|
+
if (!owner || name !== owner) return name
|
|
350
|
+
const split = Array.isArray(chunk.segments) && chunk.segments.length > 1
|
|
351
|
+
if (!split) return name
|
|
352
|
+
return Number(similarity) >= OWNER_VERIFY_SIMILARITY ? name : 'Ext'
|
|
353
|
+
}
|
|
354
|
+
|
|
355
|
+
function liveSpeakerAt(chunk: SidecarChunk, t: number, owner?: string | null): string | null {
|
|
336
356
|
const segments = chunk.segments
|
|
337
357
|
if (!Array.isArray(segments) || segments.length === 0) return null
|
|
338
358
|
let best: ChunkSpeakerSegment | null = null
|
|
@@ -344,7 +364,7 @@ function liveSpeakerAt(chunk: SidecarChunk, t: number): string | null {
|
|
|
344
364
|
const d = t < s ? s - t : t > e ? t - e : 0
|
|
345
365
|
if (d < distance) { distance = d; best = segment }
|
|
346
366
|
}
|
|
347
|
-
return best && typeof best.speaker === 'string' ? best.speaker : null
|
|
367
|
+
return best && typeof best.speaker === 'string' ? guardOwnerName(best.speaker, best.similarity, chunk, owner) : null
|
|
348
368
|
}
|
|
349
369
|
|
|
350
370
|
export interface RelabelStats {
|
|
@@ -383,6 +403,7 @@ export function relabelBatchSegments(
|
|
|
383
403
|
timeline: MeetingTimeline,
|
|
384
404
|
a: Activity,
|
|
385
405
|
map: Map<number, string>,
|
|
406
|
+
owner?: string | null,
|
|
386
407
|
): { segments: SidecarBatchSegment[]; stats: RelabelStats } {
|
|
387
408
|
const stats: RelabelStats = { total: 0, relabeled: 0, viaChannel: 0, kept: 0 }
|
|
388
409
|
const out = segments.map(segment => ({
|
|
@@ -399,7 +420,8 @@ export function relabelBatchSegments(
|
|
|
399
420
|
}
|
|
400
421
|
if (name === null) {
|
|
401
422
|
stats.kept++
|
|
402
|
-
name = (start && liveSpeakerAt(start.window.chunk, start.inChunk))
|
|
423
|
+
name = (start && liveSpeakerAt(start.window.chunk, start.inChunk, owner))
|
|
424
|
+
?? (start ? guardOwnerName(original, start.window.chunk.similarity, start.window.chunk, owner) : original)
|
|
403
425
|
}
|
|
404
426
|
if (name !== word.speaker) stats.relabeled++
|
|
405
427
|
const next: SpeakerWord = { ...word, speaker: name }
|
|
@@ -441,6 +463,7 @@ export function relabelStreamingChunks(
|
|
|
441
463
|
timeline: MeetingTimeline,
|
|
442
464
|
a: Activity,
|
|
443
465
|
map: Map<number, string>,
|
|
466
|
+
owner?: string | null,
|
|
444
467
|
): { finals: Map<number, { speaker: string; segments: ChunkSpeakerSegment[] }>; stats: RelabelStats } {
|
|
445
468
|
const stats: RelabelStats = { total: 0, relabeled: 0, viaChannel: 0, kept: 0 }
|
|
446
469
|
const finals = new Map<number, { speaker: string; segments: ChunkSpeakerSegment[] }>()
|
|
@@ -449,17 +472,17 @@ export function relabelStreamingChunks(
|
|
|
449
472
|
if (typeof chunk.text !== 'string' || !chunk.text.trim()) continue
|
|
450
473
|
const tokens = splitTokens(chunk.text)
|
|
451
474
|
const times = streamingTokenTimes(chunk, tokens, window, a)
|
|
452
|
-
const base = typeof chunk.originalSpeaker === 'string' ? chunk.originalSpeaker : chunk.speaker
|
|
475
|
+
const base = guardOwnerName(typeof chunk.originalSpeaker === 'string' ? chunk.originalSpeaker : chunk.speaker, chunk.similarity, chunk, owner)
|
|
453
476
|
const labels: string[] = []
|
|
454
477
|
let carry: string | null = null
|
|
455
478
|
for (let i = 0; i < tokens.length; i++) {
|
|
456
|
-
if (!isWordToken(tokens[i]) || !times) { labels.push(carry ?? liveSpeakerAt(chunk, 0) ?? base); continue }
|
|
479
|
+
if (!isWordToken(tokens[i]) || !times) { labels.push(carry ?? liveSpeakerAt(chunk, 0, owner) ?? base); continue }
|
|
457
480
|
stats.total++
|
|
458
481
|
const span = times[i]
|
|
459
482
|
const channel = channelForSpan(a, window.start + span.start, window.start + Math.max(span.end, span.start + 0.01))
|
|
460
483
|
let name: string
|
|
461
484
|
if (channel !== null && map.has(channel)) { name = map.get(channel)!; stats.viaChannel++ }
|
|
462
|
-
else { name = liveSpeakerAt(chunk, span.start) ?? base; stats.kept++ }
|
|
485
|
+
else { name = liveSpeakerAt(chunk, span.start, owner) ?? base; stats.kept++ }
|
|
463
486
|
labels.push(name)
|
|
464
487
|
carry = name
|
|
465
488
|
}
|
|
@@ -646,6 +669,8 @@ export interface FinalPassOptions {
|
|
|
646
669
|
tmpRoot?: string
|
|
647
670
|
log?: (line: string) => void
|
|
648
671
|
now?: () => number
|
|
672
|
+
/** The wearer's label, for the 6.61.1 owner guard (default: the profile's). */
|
|
673
|
+
owner?: string | null
|
|
649
674
|
/** Seam: how files are written (durable atomic by default). */
|
|
650
675
|
write?: (path: string, data: string) => void
|
|
651
676
|
/** Seam: the batch relabeller (word-aligned in place by default). */
|
|
@@ -758,6 +783,7 @@ async function finalPassNow(options: FinalPassOptions): Promise<FinalPassOutcome
|
|
|
758
783
|
|
|
759
784
|
const sidecarText = readFileSync(options.sidecarPath, 'utf8')
|
|
760
785
|
const sidecar = JSON.parse(sidecarText) as Record<string, any>
|
|
786
|
+
const owner = options.owner !== undefined ? options.owner : ownerLabelOrNull()
|
|
761
787
|
if (Number(sidecar.correctionRevision ?? 0) > 0) return { status: 'skipped', reason: 'human_corrected' }
|
|
762
788
|
const markdown = readFileSync(options.meetingPath, 'utf8')
|
|
763
789
|
const transcript = readTranscriptSection(markdown)
|
|
@@ -833,7 +859,7 @@ async function finalPassNow(options: FinalPassOptions): Promise<FinalPassOutcome
|
|
|
833
859
|
mode = 'batch'
|
|
834
860
|
const segments = (Array.isArray(sidecar.batchSegments) ? sidecar.batchSegments : []) as SidecarBatchSegment[]
|
|
835
861
|
if (!segments.length) return { status: 'skipped', reason: 'no_speaker_words' }
|
|
836
|
-
const relabeled = relabelBatchSegments(segments, timeline, activity, mapping.map)
|
|
862
|
+
const relabeled = relabelBatchSegments(segments, timeline, activity, mapping.map, owner)
|
|
837
863
|
stats = relabeled.stats
|
|
838
864
|
// finalizeBatch wrote labelled turns only when every segment had speaker words.
|
|
839
865
|
const selected = selectBatchTranscriptForPersistence(
|
|
@@ -850,7 +876,7 @@ async function finalPassNow(options: FinalPassOptions): Promise<FinalPassOutcome
|
|
|
850
876
|
if (typeof sidecar.batchTranscript === 'string') nextSidecar.batchTranscript = next
|
|
851
877
|
} else {
|
|
852
878
|
mode = 'streaming'
|
|
853
|
-
const relabeled = relabelStreamingChunks(timeline, activity, mapping.map)
|
|
879
|
+
const relabeled = relabelStreamingChunks(timeline, activity, mapping.map, owner)
|
|
854
880
|
stats = relabeled.stats
|
|
855
881
|
const textEntries = entries.filter(entry => typeof entry.chunk?.text === 'string' && entry.chunk.text.trim())
|
|
856
882
|
next = relabelStreamingTranscript(transcript, textEntries.map(entry => ({ chunkIndex: entry.chunkIndex, text: entry.chunk.text })), relabeled.finals)
|
|
@@ -55,6 +55,7 @@ import {
|
|
|
55
55
|
type TokenTime,
|
|
56
56
|
} from './nemotron-segments.js'
|
|
57
57
|
import type { WhisperWord } from './whisper-local.js'
|
|
58
|
+
import { OWNER_VERIFY_SIMILARITY } from './voiceprint-floor.js'
|
|
58
59
|
|
|
59
60
|
/** The chunk response contract shared with COS Glasses 6.9.567. Keep this shape. */
|
|
60
61
|
export interface ChunkSpeakerSegment {
|
|
@@ -78,8 +79,10 @@ export interface LiveNamingInput {
|
|
|
78
79
|
audio: Buffer
|
|
79
80
|
/** Today's whole-chunk voiceprint result. */
|
|
80
81
|
chunkVoiceprint: VoiceprintResult
|
|
81
|
-
/** The client's own label (amplitude / Even Hub), e.g. the owner or Ext. */
|
|
82
|
+
/** The client's own label (amplitude / Even Hub), e.g. the owner or Ext. Never names a split track. */
|
|
82
83
|
clientSpeaker: string
|
|
84
|
+
/** The wearer's label (`owner_speaker_label`), for the owner guard. */
|
|
85
|
+
owner?: string | null
|
|
83
86
|
identify: (wav: Buffer) => VoiceprintResult | null
|
|
84
87
|
}
|
|
85
88
|
|
|
@@ -92,6 +95,8 @@ export type LiveNamingResult =
|
|
|
92
95
|
tracks: number
|
|
93
96
|
timing: 'words' | 'estimated'
|
|
94
97
|
identified: number
|
|
98
|
+
/** Would-be wearer names refused in this chunk, by reason. */
|
|
99
|
+
ownerGuard: { client_label: number; weak_owner: number }
|
|
95
100
|
}
|
|
96
101
|
| { ok: false; reason: 'no_text' | 'no_speech' }
|
|
97
102
|
|
|
@@ -105,14 +110,19 @@ function isRealLabel(label: string): boolean {
|
|
|
105
110
|
/**
|
|
106
111
|
* Split one chunk's text into named segments. Pure apart from `identify`.
|
|
107
112
|
*
|
|
108
|
-
* Naming rules (2026-10-02 brief):
|
|
109
|
-
* - one track: it is the whole chunk, so it takes the chunk's voiceprint result
|
|
113
|
+
* Naming rules (2026-10-02 brief, 6.61.1 owner guard):
|
|
114
|
+
* - one track: it is the whole chunk, so it takes the chunk's voiceprint result,
|
|
115
|
+
* exactly as 6.60.0 labelled the chunk (including the client's label when no
|
|
116
|
+
* voiceprint could run);
|
|
110
117
|
* - two or more: each track with at least NAME_TRACK_MIN_SEC of its own audio is
|
|
111
|
-
* named by the voiceprint on that audio;
|
|
112
|
-
*
|
|
113
|
-
*
|
|
114
|
-
*
|
|
115
|
-
*
|
|
118
|
+
* named by the voiceprint on that audio; a shorter dominant track takes the
|
|
119
|
+
* chunk's voiceprint; any other short track is Ext;
|
|
120
|
+
* - in a split chunk the WEARER is named only at owner-verify confidence
|
|
121
|
+
* (OWNER_VERIFY_SIMILARITY). A full search can return the wearer from 0.55,
|
|
122
|
+
* and the client's label is the glasses' default wearer label, not identity
|
|
123
|
+
* evidence: on 2026-10-02 both named other people (Jeremy, Kyle) as the
|
|
124
|
+
* wearer. Either becomes Ext and is counted (`ownerGuard`).
|
|
125
|
+
* A name is never invented.
|
|
116
126
|
* The segments' texts joined with one space equal `text` exactly.
|
|
117
127
|
*/
|
|
118
128
|
export function nameChunkSegments(input: LiveNamingInput): LiveNamingResult {
|
|
@@ -139,6 +149,7 @@ export function nameChunkSegments(input: LiveNamingInput): LiveNamingResult {
|
|
|
139
149
|
const used = new Set(labels)
|
|
140
150
|
|
|
141
151
|
const names = new Map<number, VoiceprintResult>()
|
|
152
|
+
const ownerGuard = { client_label: 0, weak_owner: 0 }
|
|
142
153
|
let identified = 0
|
|
143
154
|
const pcm = readPcm16Mono(input.audio)
|
|
144
155
|
const ordered = tracks.filter(track => used.has(track.channel))
|
|
@@ -146,16 +157,25 @@ export function nameChunkSegments(input: LiveNamingInput): LiveNamingResult {
|
|
|
146
157
|
if (ordered.length === 1) {
|
|
147
158
|
names.set(ordered[0].channel, input.chunkVoiceprint)
|
|
148
159
|
} else {
|
|
160
|
+
const owner = input.owner?.trim() || null
|
|
161
|
+
const ownerSafe = (named: VoiceprintResult): VoiceprintResult => {
|
|
162
|
+
if (!owner || named.speaker !== owner || named.similarity >= OWNER_VERIFY_SIMILARITY) return named
|
|
163
|
+
ownerGuard[named.similarity > 0 ? 'weak_owner' : 'client_label']++
|
|
164
|
+
return { speaker: 'Ext', similarity: 0 }
|
|
165
|
+
}
|
|
149
166
|
for (const track of ordered) {
|
|
150
167
|
if (track.exclusiveSec >= NAME_TRACK_MIN_SEC && pcm) {
|
|
151
168
|
const wav = exclusiveTrackWav(activity, track.channel, pcm)
|
|
152
169
|
const named = wav ? input.identify(wav) : null
|
|
153
|
-
if (named) { names.set(track.channel, named); identified++; continue }
|
|
170
|
+
if (named) { names.set(track.channel, ownerSafe(named)); identified++; continue }
|
|
154
171
|
}
|
|
155
|
-
if (track.channel === dominant) { names.set(track.channel, input.chunkVoiceprint); continue }
|
|
172
|
+
if (track.channel === dominant) { names.set(track.channel, ownerSafe(input.chunkVoiceprint)); continue }
|
|
173
|
+
// The client's label is the glasses' default wearer label, not evidence of
|
|
174
|
+
// who this voice is (6.61.0 named Jeremy Sokolic as MU through it).
|
|
156
175
|
const dominantName = names.get(dominant)?.speaker ?? input.chunkVoiceprint.speaker
|
|
157
176
|
const client = input.clientSpeaker.trim()
|
|
158
|
-
|
|
177
|
+
if (isRealLabel(client) && client !== dominantName) ownerGuard.client_label++
|
|
178
|
+
names.set(track.channel, { speaker: 'Ext', similarity: 0 })
|
|
159
179
|
}
|
|
160
180
|
}
|
|
161
181
|
|
|
@@ -196,7 +216,7 @@ export function nameChunkSegments(input: LiveNamingInput): LiveNamingResult {
|
|
|
196
216
|
let best = -1
|
|
197
217
|
for (const [name, sec] of voiced) if (sec > best) { best = sec; speaker = name }
|
|
198
218
|
const similarity = [...names.values()].find(named => named.speaker === speaker)?.similarity ?? 0
|
|
199
|
-
return { ok: true, speaker, similarity: round(similarity, 3), segments, tracks: tracks.length, timing, identified }
|
|
219
|
+
return { ok: true, speaker, similarity: round(similarity, 3), segments, tracks: tracks.length, timing, identified, ownerGuard }
|
|
200
220
|
}
|
|
201
221
|
|
|
202
222
|
/**
|
|
@@ -244,6 +264,8 @@ export interface LiveLabelInput {
|
|
|
244
264
|
audio: Buffer
|
|
245
265
|
voiceprint: VoiceprintResult
|
|
246
266
|
clientSpeaker: string
|
|
267
|
+
/** The wearer's label, for the owner guard on split chunks. */
|
|
268
|
+
owner?: string | null
|
|
247
269
|
identify: (wav: Buffer) => VoiceprintResult | null
|
|
248
270
|
clock?: () => number
|
|
249
271
|
}
|
|
@@ -260,10 +282,12 @@ export interface LiveLabels {
|
|
|
260
282
|
tracks?: number
|
|
261
283
|
timing?: 'words' | 'estimated'
|
|
262
284
|
ms?: number
|
|
285
|
+
/** 6.61.1: would-be wearer names refused in this chunk (client_label, weak_owner). */
|
|
286
|
+
ownerGuard?: string[]
|
|
263
287
|
}
|
|
264
288
|
/** What NemotronLiveRuntime.record should log; absent when the voiceprint was chosen by config. */
|
|
265
289
|
record?:
|
|
266
|
-
| { outcome: 'nemotron'; ms: number; segments: number; tracks: number; timing: string; identified: number }
|
|
290
|
+
| { outcome: 'nemotron'; ms: number; segments: number; tracks: number; timing: string; identified: number; ownerGuard?: { client_label: number; weak_owner: number } }
|
|
267
291
|
| { outcome: 'voiceprint'; reason: string; ms: number | null; detail?: string }
|
|
268
292
|
}
|
|
269
293
|
|
|
@@ -306,10 +330,15 @@ export async function resolveLiveLabels(input: LiveLabelInput): Promise<LiveLabe
|
|
|
306
330
|
audio: input.audio,
|
|
307
331
|
chunkVoiceprint: input.voiceprint,
|
|
308
332
|
clientSpeaker: input.clientSpeaker,
|
|
333
|
+
owner: input.owner,
|
|
309
334
|
identify: input.identify,
|
|
310
335
|
})
|
|
311
336
|
if (!named.ok) return voiceprintOnly(named.reason, ms)
|
|
312
337
|
const changed = named.speaker !== input.voiceprint.speaker
|
|
338
|
+
const guarded = [
|
|
339
|
+
...Array(named.ownerGuard.client_label).fill('client_label'),
|
|
340
|
+
...Array(named.ownerGuard.weak_owner).fill('weak_owner'),
|
|
341
|
+
] as string[]
|
|
313
342
|
return {
|
|
314
343
|
speaker: named.speaker,
|
|
315
344
|
similarity: named.similarity,
|
|
@@ -320,8 +349,12 @@ export async function resolveLiveLabels(input: LiveLabelInput): Promise<LiveLabe
|
|
|
320
349
|
timing: named.timing,
|
|
321
350
|
ms,
|
|
322
351
|
...(changed ? { voiceprintSpeaker: input.voiceprint.speaker, voiceprintSimilarity: input.voiceprint.similarity } : {}),
|
|
352
|
+
...(guarded.length ? { ownerGuard: guarded } : {}),
|
|
353
|
+
},
|
|
354
|
+
record: {
|
|
355
|
+
outcome: 'nemotron', ms, segments: named.segments.length, tracks: named.tracks, timing: named.timing, identified: named.identified,
|
|
356
|
+
...(guarded.length ? { ownerGuard: named.ownerGuard } : {}),
|
|
323
357
|
},
|
|
324
|
-
record: { outcome: 'nemotron', ms, segments: named.segments.length, tracks: named.tracks, timing: named.timing, identified: named.identified },
|
|
325
358
|
}
|
|
326
359
|
} catch (error) {
|
|
327
360
|
return voiceprintOnly('error', ms, error instanceof Error ? error.message : String(error))
|
|
@@ -414,6 +447,8 @@ export interface LiveSnapshot {
|
|
|
414
447
|
/** Whether the CLI binary is the v0.17.4 build the offline guarantee was verified on. */
|
|
415
448
|
cliKnown: boolean | null
|
|
416
449
|
chunks: { nemotron: number; voiceprint: number; reasons: Record<string, number> }
|
|
450
|
+
/** 6.61.1: would-be wearer names refused on split chunks since start. */
|
|
451
|
+
ownerGuard: { client_label: number; weak_owner: number }
|
|
417
452
|
last: { at: string; outcome: 'nemotron' | 'voiceprint'; reason: string | null; ms: number | null } | null
|
|
418
453
|
}
|
|
419
454
|
|
|
@@ -440,6 +475,7 @@ export class NemotronLiveRuntime {
|
|
|
440
475
|
private modelsDir: string | null = null
|
|
441
476
|
private identity: { path: string; sha256: string; known: boolean } | null = null
|
|
442
477
|
private counts = { nemotron: 0, voiceprint: 0, reasons: {} as Record<string, number> }
|
|
478
|
+
private ownerGuard = { client_label: 0, weak_owner: 0 }
|
|
443
479
|
private recent: Array<{ outcome: 'nemotron' | 'voiceprint'; reason: string | null }> = []
|
|
444
480
|
private last: LiveSnapshot['last'] = null
|
|
445
481
|
private lastStateLog: string | null = null
|
|
@@ -605,7 +641,7 @@ export class NemotronLiveRuntime {
|
|
|
605
641
|
|
|
606
642
|
/** Record one chunk's outcome. Run failures are logged per chunk; state reasons once per change. */
|
|
607
643
|
record(chunk: { sessionId: string; chunkIndex: number }, outcome:
|
|
608
|
-
| { outcome: 'nemotron'; ms: number; segments: number; tracks: number; timing: string; identified: number }
|
|
644
|
+
| { outcome: 'nemotron'; ms: number; segments: number; tracks: number; timing: string; identified: number; ownerGuard?: { client_label: number; weak_owner: number } }
|
|
609
645
|
| { outcome: 'voiceprint'; reason: string; ms: number | null; detail?: string },
|
|
610
646
|
): void {
|
|
611
647
|
const at = new Date(this.now()).toISOString()
|
|
@@ -615,6 +651,12 @@ export class NemotronLiveRuntime {
|
|
|
615
651
|
this.pushRecent('nemotron', null)
|
|
616
652
|
this.lastStateLog = null
|
|
617
653
|
this.log(`[diarizer] chunk #${chunk.chunkIndex} nemotron ${outcome.ms} ms tracks=${outcome.tracks} segments=${outcome.segments} timing=${outcome.timing} named=${outcome.identified}`)
|
|
654
|
+
const guard = outcome.ownerGuard
|
|
655
|
+
if (guard && guard.client_label + guard.weak_owner > 0) {
|
|
656
|
+
this.ownerGuard.client_label += guard.client_label
|
|
657
|
+
this.ownerGuard.weak_owner += guard.weak_owner
|
|
658
|
+
this.log(`[diarizer] owner-guard chunk #${chunk.chunkIndex} refused=${guard.client_label + guard.weak_owner} client_label=${guard.client_label} weak_owner=${guard.weak_owner}`)
|
|
659
|
+
}
|
|
618
660
|
return
|
|
619
661
|
}
|
|
620
662
|
this.counts.voiceprint++
|
|
@@ -676,6 +718,7 @@ export class NemotronLiveRuntime {
|
|
|
676
718
|
warm: { state: this.state, ms: this.warmMs, loadSec: this.warmLoadSec, error: this.warmError },
|
|
677
719
|
cliKnown: this.identity ? this.identity.known : null,
|
|
678
720
|
chunks: { nemotron: this.counts.nemotron, voiceprint: this.counts.voiceprint, reasons: { ...this.counts.reasons } },
|
|
721
|
+
ownerGuard: { ...this.ownerGuard },
|
|
679
722
|
last: this.last,
|
|
680
723
|
}
|
|
681
724
|
}
|
|
@@ -42,6 +42,7 @@ const require = createRequire(import.meta.url)
|
|
|
42
42
|
const __dirname = fileURLToPath(new URL('.', import.meta.url))
|
|
43
43
|
|
|
44
44
|
import { DATA_DIR } from './data-dir.js'
|
|
45
|
+
import { OWNER_VERIFY_SIMILARITY } from './voiceprint-floor.js'
|
|
45
46
|
|
|
46
47
|
export const SPEAKER_MODEL_FILENAME = '3dspeaker_speech_eres2net_sv_en_voxceleb_16k.onnx'
|
|
47
48
|
|
|
@@ -86,7 +87,7 @@ const PROFILES_PATH = resolve(DATA_DIR, 'voice-profiles.json')
|
|
|
86
87
|
const CALIBRATION_LOG = resolve(DATA_DIR, 'speaker-calibration.jsonl')
|
|
87
88
|
|
|
88
89
|
// Thresholds
|
|
89
|
-
const VERIFY_THRESHOLD =
|
|
90
|
+
const VERIFY_THRESHOLD = OWNER_VERIFY_SIMILARITY
|
|
90
91
|
const SEARCH_THRESHOLD = 0.55
|
|
91
92
|
/** The bar `autoEnroll` enrols at. From 6.45.4 also the bar at which a held
|
|
92
93
|
* group is offered as a person with one click ("high"): the identifier would
|
|
@@ -1,4 +1,12 @@
|
|
|
1
|
-
//
|
|
2
|
-
//
|
|
3
|
-
|
|
1
|
+
// Voiceprint floors shared by every path that names a voice, so they can never
|
|
2
|
+
// drift apart.
|
|
3
|
+
|
|
4
|
+
/** The least audio the voiceprint names a voice from: the whole-chunk check in
|
|
5
|
+
* transcribe-stream and the per-track check in Nemotron naming. */
|
|
4
6
|
export const VOICEPRINT_MIN_AUDIO_SEC = 2.0
|
|
7
|
+
|
|
8
|
+
/** The similarity at which the voiceprint verifies the WEARER (speaker-embeddings'
|
|
9
|
+
* owner verify). A full search accepts any profile from 0.55, so a voice can come
|
|
10
|
+
* back as the wearer between 0.55 and 0.65 without ever passing owner verify;
|
|
11
|
+
* 6.61.1 refuses that for a split track (it named other people as the wearer). */
|
|
12
|
+
export const OWNER_VERIFY_SIMILARITY = 0.65
|
|
@@ -9,7 +9,7 @@ import { chmodSync, readFileSync, writeFileSync, existsSync, lstatSync, mkdirSyn
|
|
|
9
9
|
import { writeFile } from 'node:fs/promises'
|
|
10
10
|
import { resolve } from 'node:path'
|
|
11
11
|
import { fileURLToPath } from 'node:url'
|
|
12
|
-
import { getVocabulary, getOwnerName } from '../lib/profile.js'
|
|
12
|
+
import { getVocabulary, getOwnerName, getOwnerSpeakerLabel } from '../lib/profile.js'
|
|
13
13
|
import { getOpenAIKey } from '../lib/openai-key.js'
|
|
14
14
|
import { getTranscriptionPolicySnapshot, isOpenAIWhisperFallbackReady } from '../lib/transcription-policy.js'
|
|
15
15
|
import { STRANDED_STALE_MS } from '../lib/stranded-sessions.js'
|
|
@@ -1908,6 +1908,11 @@ async function transcribeWithServerWhisper(audioBuffer: Buffer, whisperAudio: Bu
|
|
|
1908
1908
|
}
|
|
1909
1909
|
}
|
|
1910
1910
|
|
|
1911
|
+
/** The wearer's label for the 6.61.1 owner guard; null when the profile cannot say. */
|
|
1912
|
+
function ownerLabelOrNull(): string | null {
|
|
1913
|
+
try { return getOwnerSpeakerLabel() } catch { return null }
|
|
1914
|
+
}
|
|
1915
|
+
|
|
1911
1916
|
function identifyChunkSpeaker(audioBuffer: Buffer, sessionId: string, chunkIndex: number, clientSpeaker: string): { speaker: string; similarity: number } {
|
|
1912
1917
|
const expectedSpeakers = undefined
|
|
1913
1918
|
const audioDurationSec = Math.max(0, (audioBuffer.length - 44)) / 32000
|
|
@@ -2155,6 +2160,7 @@ async function processStreamChunk(opts: {
|
|
|
2155
2160
|
audio: audioBuffer,
|
|
2156
2161
|
voiceprint,
|
|
2157
2162
|
clientSpeaker,
|
|
2163
|
+
owner: ownerLabelOrNull(),
|
|
2158
2164
|
identify: wav => {
|
|
2159
2165
|
const named = identifySpeaker(wav, undefined, { event: 'nemotron-track' })
|
|
2160
2166
|
return named ? { speaker: named.speaker, similarity: named.similarity } : null
|