@gotcos/glasses-server 6.45.2 → 6.45.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +18 -0
- package/package.json +1 -1
- package/server/index.ts +2 -0
- package/server/lib/archive.ts +2 -1
- package/server/lib/conversation.ts +48 -4
- package/server/lib/embedding-eviction.ts +19 -0
- package/server/lib/held-voice-groups.ts +752 -0
- package/server/lib/prompt-tail-guard.ts +257 -0
- package/server/lib/recent-messages.ts +56 -0
- package/server/lib/speaker-embeddings.ts +8 -1
- package/server/lib/training-audio-provenance.ts +1 -1
- package/server/lib/transcribe-audio.ts +9 -1
- package/server/lib/vad-silero.ts +8 -3
- package/server/lib/voice-enrolment-selection.ts +31 -5
- package/server/lib/whisper-local.ts +12 -6
- package/server/lib/workspace-skills.ts +209 -0
- package/server/routes/archive.ts +3 -0
- package/server/routes/prompt-drafts.ts +13 -2
- package/server/routes/sessions.ts +25 -22
- package/server/routes/skills.ts +13 -0
- package/server/routes/transcribe-stream.ts +5 -3
- package/server/routes/voice.ts +139 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,21 @@
|
|
|
1
|
+
## 6.45.4
|
|
2
|
+
|
|
3
|
+
Two things from 2026-09-12: a dictated prompt that ended in a sentence nobody said, and held voices that could only be named one meeting at a time. Both went through a four-validator QA pass against a copy of Miles's live store before they shipped; what that pass corrected is recorded inline.
|
|
4
|
+
|
|
5
|
+
- A prompt no longer keeps a trailing sentence the speaker did not say. Chris's prompt closed with "I'm going to go to the bathroom" after he had stopped talking: the interactive path runs whisper without VAD on purpose (so a short prompt is never trimmed), and a decoder handed trailing silence plus a vocabulary prompt finishes the sentence it thinks it heard. `prompt-tail-guard.ts` judges the LAST sentence of every prompt chunk by two rules, at most twice: (a) the whole sentence is a known filler — thanks for watching or listening, subtitles or transcript by, like and subscribe, see you next time or in the next video, the bathroom line — anchored start to end, so "put the risks at the end", "see you next time we are both in Austin" and "the webinar about thanks for watching screens" never match (QA found the unanchored form eating all three); (b) the Silero VAD MEASURED the audio, the sentence's whisper tokens start at least a second after the last speech it heard, AND whisper's mean token probability for the sentence is under 0.45. Two signals must agree. Rule (b) never empties a chunk in which the VAD heard any speech; a chunk empties only when every sentence was a filler, or the VAD measured the whole chunk as silence and the decoder was unsure of what it wrote there (Miles: "as long as we don't end up cutting the final portion of a user's prompt"). The interactive high-quality decode now asks whisper-cli for token clocks (`words: true`); before this the timings existed only on the batch path, which QA caught. The VAD result carries `measured`, so a model that is absent, audio too short, or an internal error reads as unknown, never as silence. Calibrated on ten real prompt chunks: trailing sentences scored 0.72 to 1.00, one real sentence in a noisy room 0.43, which is why 0.45 never acts alone. Every drop logs as `[prompt-draft] tail_drop` with its reason; the lens preview line gets rule (a) too.
|
|
6
|
+
- Held voices are grouped by who they sound like, across every meeting in the window. `GET /api/voice/held-groups` clusters every retained unidentified sample with the same mutually-coherent rule that guards profile corrections. The chunk-embedding bank already holds each sample's voiceprint, so the ordinary listing decodes no audio (906 samples across 32 sessions in about a third of a second on Miles's store); a sample the bank lacks is decoded inside a one-second budget, cached, and finished by a background sweep that stops when it makes no progress, and the listing is memoised for 15 s while nothing held changes. Exact duplicate chunks (a recording started twice banks every chunk under two ids: 49 such pairs on the live store) fold into one sample before grouping and before enrolment. It answers `groups` — members, meetings, coherence, the seed sample to play first, and `suggestion` — plus `loose`, the samples that cohere with nothing, and `speakerModel`, whether anything outside the bank can be grouped at all. A suggestion is the profile that VOUCHES for a group: two or more of its samples above the identifier's 0.55 floor, one of them from a source that may vouch (manual, Fireflies, a G2 enrolment, a chunk banked on a live match, a correction, a named held group — never a bulk `ext-retroactive` session enrol, which on the live store had written one household voice into two people's profiles and offered it back as them), scored on the group's SEED sample against the profile's second-best — a centroid scores higher than any of its samples, and one polluted sample cannot vouch alone — `high` at the 0.88 bar `autoEnroll` itself enrols at and `likely` above the floor, with `agreeing`/`of` counts. The wearer's own profile is never offered, and when the best-scoring profile cannot vouch nothing is offered rather than the runner-up. On Miles's store that is 81 voices, 442 loose samples, 17 suggestions all "likely", and the 120-sample household voice left unidentified for him to name.
|
|
7
|
+
- `POST /api/voice/held-groups/enroll { name, members, confirm }` names a set as one person, appending when the name already has a profile — which is how a voice that missed in a new room heals. It fails closed like every sibling: without `confirm: true` it answers 400 with a preview of exactly what would change, and `dryRun: true` returns that preview. It re-checks coherence on the server, folds duplicates, writes only the coherent core (diverse subset, max 20, source `ext-group:<sessionId>` so a meeting's "not in this meeting" can retract it; `ext-group` ranks as human provenance for eviction), removes only those wavs, and leaves anything that was not that voice held. Decoding for a request is budgeted at two seconds; samples past it are `notReady`, untouched, and listed. A set that agrees on nothing is refused (409 `incoherent`); with the speaker model not loaded the request is refused before it touches anything (503); a store that refuses every sample answers 409 `nothing_enrolled` and deletes nothing. `POST /api/voice/held-groups/discard { members, confirm }` throws samples out, previewing without `confirm`. Requests take up to 4,000 distinct samples. `GET /api/voice/ext-audio` and `POST /api/voice/enroll-ext` are unchanged; COS Control 0.5.219 builds Add-a-voice on the new routes.
|
|
8
|
+
|
|
9
|
+
## 6.45.3
|
|
10
|
+
|
|
11
|
+
The phone can list the skills this Mac will actually run.
|
|
12
|
+
|
|
13
|
+
- `GET /api/skills` returns the slash catalog from the selected workspace (`.agents/skills`, then `.claude/skills`) plus `~/.claude/skills`. Nested agent folders flatten with `:`. Generated `source-command-*` duplicates stay out. Paths never leave the box.
|
|
14
|
+
- COS Glasses 6.9.468 paints that list on Views instead of a hardcoded cheat sheet.
|
|
15
|
+
- A held voice can be heard before it is named. `GET /api/voice/ext-audio` lists `chunkIndices` per held session and `GET /api/voice/ext-audio/:sessionId/sample?chunk=<index>` serves that one chunk (no `chunk` still serves the newest). COS Control's Add-a-voice panel uses it to play portions of a session before naming it (Queen, 2026-09-12: "no way to listen to the voices that are here").
|
|
16
|
+
- Recent is a rolling window. `GET /api/sessions/today/all-messages` keeps its path for COS Control and the phone but now answers with the newest 30 messages (`?limit=` up to 100) across every live session and as many archived days as it takes, deduplicated against their archive mirrors. Messages leave Recent only by ageing out of the window; nothing is hidden by the calendar. Control's Recent view and the phone's history recovery read this without change.
|
|
17
|
+
- Yesterday's turns no longer vanish for a day. The archive mirror ran every 24 h from boot (21:25 on the current uptime), so finished sessions from the previous local day were neither "today" for `GET /api/sessions/today/all-messages` nor in any day archive until the evening; on 2026-09-12 07:00 Control showed no turns while ten from the 11th sat live in `sessions.json`. The mirror now runs at boot, at the next local midnight, and lazily from the today and archive-listing routes, once per local day. `checkYesterdayArchive` keys on the local day like everything else.
|
|
18
|
+
|
|
1
19
|
## 6.45.2
|
|
2
20
|
|
|
3
21
|
The G2 Sessions list answers in under a second again instead of eleven, and loading it no longer stalls the rest of the server.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@gotcos/glasses-server",
|
|
3
|
-
"version": "6.45.
|
|
3
|
+
"version": "6.45.4",
|
|
4
4
|
"description": "COS Glasses — self-hosted AI heads-up-display server for Even G2 smart glasses, powered by Claude Code, Codex, Cursor Agent CLI, or local Ollama",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
package/server/index.ts
CHANGED
|
@@ -62,6 +62,7 @@ import { liveCuesRouter } from './routes/live-cues.js'
|
|
|
62
62
|
import { tasksRouter } from './routes/tasks.js'
|
|
63
63
|
import { memoryRouter } from './routes/memory.js'
|
|
64
64
|
import { threadsRouter } from './routes/threads.js'
|
|
65
|
+
import { skillsRouter } from './routes/skills.js'
|
|
65
66
|
import { shutdownLiveCues } from './lib/live-cues-engine.js'
|
|
66
67
|
import { prewarmContext } from './lib/context-builder.js'
|
|
67
68
|
import { preWarmCLI } from './lib/claude-bridge.js'
|
|
@@ -671,6 +672,7 @@ app.use('/api', meetingRouter)
|
|
|
671
672
|
app.use('/api', meetingsRouter)
|
|
672
673
|
app.use('/api', memoryRouter)
|
|
673
674
|
app.use('/api', threadsRouter)
|
|
675
|
+
app.use('/api', skillsRouter)
|
|
674
676
|
app.use('/api', openaiKeyRouter)
|
|
675
677
|
// v6.3.0 — Message History, cross-day 'reference message N', and history
|
|
676
678
|
// recovery for public npx users (previously full-COS-server only).
|
package/server/lib/archive.ts
CHANGED
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
// Each day's archive contains one or more "chats" (split by context breaks)
|
|
4
4
|
// Summaries are generated via `claude -p --model sonnet`, budget-capped per day.
|
|
5
5
|
|
|
6
|
+
import { localDay } from './local-day.js'
|
|
6
7
|
import { chmodSync, mkdirSync, readdirSync } from 'node:fs'
|
|
7
8
|
import { resolve, dirname } from 'node:path'
|
|
8
9
|
import { fileURLToPath } from 'node:url'
|
|
@@ -549,7 +550,7 @@ export function getArchiveDayMessages(
|
|
|
549
550
|
|
|
550
551
|
/** Check if yesterday needs archiving (handles overnight server restarts) */
|
|
551
552
|
export function checkYesterdayArchive(): void {
|
|
552
|
-
const yesterday =
|
|
553
|
+
const yesterday = localDay(Date.now() - 86_400_000)
|
|
553
554
|
const existing = loadArchive(yesterday)
|
|
554
555
|
if (!existing) {
|
|
555
556
|
// No yesterday archive exists — but we can't archive sessions that are already expired
|
|
@@ -288,11 +288,55 @@ async function runDailyArchiveMirror(): Promise<void> {
|
|
|
288
288
|
if (mirrored > 0) updateGlassesSessionCache()
|
|
289
289
|
}
|
|
290
290
|
|
|
291
|
+
// 6.45.3 — the mirror runs at every LOCAL DAY ROLLOVER, not only every 24 h from
|
|
292
|
+
// boot. With a server started at 21:24 the interval landed at 21:25 each evening,
|
|
293
|
+
// so a whole day's finished sessions sat in a blind spot until then: not "today"
|
|
294
|
+
// for `GET /api/sessions/today/all-messages`, not yet in any day archive. On
|
|
295
|
+
// 2026-09-12 07:00 Control showed "No turns today" (true) and no archive for
|
|
296
|
+
// 2026-09-11 (ten turns, #125 to #134, still live in sessions.json). Three
|
|
297
|
+
// triggers now: boot, a timer aimed at the next local midnight, and a lazy check
|
|
298
|
+
// from the routes that read the day views, so yesterday appears on the first
|
|
299
|
+
// read after midnight. Serialized: one run at a time, once per local day.
|
|
300
|
+
let lastMirrorDay = ''
|
|
301
|
+
let mirrorInFlight: Promise<void> | null = null
|
|
302
|
+
|
|
303
|
+
/** Milliseconds until thirty seconds past the next local midnight; never less than a second. */
|
|
304
|
+
export function msUntilNextLocalDay(now = Date.now()): number {
|
|
305
|
+
const d = new Date(now)
|
|
306
|
+
const next = new Date(d.getFullYear(), d.getMonth(), d.getDate() + 1, 0, 0, 30)
|
|
307
|
+
return Math.max(1_000, next.getTime() - now)
|
|
308
|
+
}
|
|
309
|
+
|
|
310
|
+
/** Mirror prior-day sessions once per local day. Returns true when a run happened. */
|
|
311
|
+
export async function ensureArchiveMirrorForDay(now = Date.now()): Promise<boolean> {
|
|
312
|
+
const today = localDay(now)
|
|
313
|
+
if (lastMirrorDay === today) return false
|
|
314
|
+
if (!mirrorInFlight) {
|
|
315
|
+
mirrorInFlight = runDailyArchiveMirror()
|
|
316
|
+
.then(() => { lastMirrorDay = today })
|
|
317
|
+
.finally(() => { mirrorInFlight = null })
|
|
318
|
+
}
|
|
319
|
+
await mirrorInFlight
|
|
320
|
+
return true
|
|
321
|
+
}
|
|
322
|
+
|
|
323
|
+
/** Test seam: forget the last mirrored day. */
|
|
324
|
+
export function __resetArchiveMirrorForTests(): void {
|
|
325
|
+
lastMirrorDay = ''
|
|
326
|
+
}
|
|
327
|
+
|
|
328
|
+
function scheduleArchiveMirrorAtNextLocalDay(): void {
|
|
329
|
+
const timer = setTimeout(() => {
|
|
330
|
+
ensureArchiveMirrorForDay()
|
|
331
|
+
.catch(err => console.error('[conversation] mirror rollover error:', err))
|
|
332
|
+
.finally(scheduleArchiveMirrorAtNextLocalDay)
|
|
333
|
+
}, msUntilNextLocalDay())
|
|
334
|
+
timer.unref?.()
|
|
335
|
+
}
|
|
336
|
+
|
|
291
337
|
// Fire-and-forget at boot so module load isn't blocked on disk + LLM fallback I/O.
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
runDailyArchiveMirror().catch(err => console.error('[conversation] mirror interval error:', err))
|
|
295
|
-
}, 24 * 60 * 60_000)
|
|
338
|
+
ensureArchiveMirrorForDay().catch(err => console.error('[conversation] mirror boot error:', err))
|
|
339
|
+
scheduleArchiveMirrorAtNextLocalDay()
|
|
296
340
|
|
|
297
341
|
// Track whether session is brand new (for first-query notification)
|
|
298
342
|
const newSessions = new Set<string>()
|
|
@@ -42,6 +42,8 @@ export function provenanceTier(source: string | undefined | null): ProvenanceTie
|
|
|
42
42
|
case 'correction':
|
|
43
43
|
case 'ext-retroactive':
|
|
44
44
|
case 'g2-enrollment':
|
|
45
|
+
// 6.45.4: a held group a human named after listening (held-voice-groups).
|
|
46
|
+
case 'ext-group':
|
|
45
47
|
return 'human'
|
|
46
48
|
// The identifier chose the label; a human approved the batch run.
|
|
47
49
|
case 'g2-training':
|
|
@@ -57,6 +59,23 @@ export function provenanceTier(source: string | undefined | null): ProvenanceTie
|
|
|
57
59
|
}
|
|
58
60
|
}
|
|
59
61
|
|
|
62
|
+
/**
|
|
63
|
+
* May a sample of this provenance VOUCH for an unidentified voice — be the
|
|
64
|
+
* corroboration behind "this held group sounds like X"?
|
|
65
|
+
*
|
|
66
|
+
* Narrower than `provenanceTier === 'human'` on purpose. `ext-retroactive` is a
|
|
67
|
+
* whole held session enrolled under one typed name: a human supplied the name,
|
|
68
|
+
* so eviction protects it, but nobody listened to each sample, and on the live
|
|
69
|
+
* store 2026-09-12 every "high" suggestion was carried by such samples alone —
|
|
70
|
+
* one household voice that a bulk enrol had written into two people's
|
|
71
|
+
* profiles, offered back as those people. `auto` and `unknown` never vouch.
|
|
72
|
+
*/
|
|
73
|
+
export function vouchesForIdentity(source: string | undefined | null): boolean {
|
|
74
|
+
const head = String(source ?? '').trim().split(/[:_]/)[0].toLowerCase()
|
|
75
|
+
return head === 'manual' || head === 'fireflies' || head === 'g2-enrollment'
|
|
76
|
+
|| head === 'g2-training' || head === 'correction' || head === 'ext-group'
|
|
77
|
+
}
|
|
78
|
+
|
|
60
79
|
/** True for a sample created by a human correcting a specific meeting. */
|
|
61
80
|
export function isCorrection(source: string | undefined | null): boolean {
|
|
62
81
|
return String(source ?? '').trim().toLowerCase().startsWith('correction')
|