@gotcos/glasses-server 6.22.1 → 6.23.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,3 +1,57 @@
1
+ ## 6.23.0
2
+
3
+ A recording whose phone goes away now becomes a meeting on its own. Miles: "we end
4
+ up with a meeting that is orphaned that we have no ability to keep."
5
+
6
+ Found live while writing this: two sessions stranded for 184 and 24 minutes, holding
7
+ the restart lock, while `GET /api/meeting/orphans` answered `count: 0`. Both saved at
8
+ 100% transfer integrity (529/529 and 23/23 chunks). Nothing had been lost — but
9
+ nothing was going to turn them into meetings either.
10
+
11
+ - **The audio was never the problem.** A stranded capture stays live in memory for 4
12
+ hours, then closes as `expired` and its chunks move to quarantine for 72 more. A
13
+ 76-hour window in which the audio exists and NOTHING converts it into a meeting
14
+ unless a human notices. Expiry produced preserved evidence, not a meeting.
15
+ - **The 60-second sweeper already existed and already detected these.** It called
16
+ `closeTranscriptSession(id, 'expired')`. The change is the disposition at the
17
+ cutoff, not new scheduling: it now finalizes through `POST /api/meeting/save`,
18
+ which keeps the live ASR transcript and its speaker labels. The quarantine recover
19
+ route was the wrong tool here — its output labels every speaker Unknown, because no
20
+ live ASR ever ran on it.
21
+ - **Staleness never closes a session early, and it must not.** The companion buffers
22
+ to IndexedDB while iOS suspends the WebView and drains on foreground, and it
23
+ restores `restoredSessionId` across a relaunch — so a phone silent for 30 minutes
24
+ can still deliver its tail into the same session id. Close it and `isSessionDeleted`
25
+ answers 410 Gone: a truncated meeting AND a second orphan. At the stale threshold a
26
+ readable draft is written and the session stays open.
27
+ - **`/api/meeting/orphans` and `/api/health` were blind to the state that matters.**
28
+ Both listed only QUARANTINED directories, and a stranded session is not quarantined
29
+ for four hours. New `stranded` / `stranded_captures` report idle minutes, captured
30
+ minutes, chunk count, when the sweeper will save it, and whether a draft exists.
31
+ - **Quiet is not failure.** A heartbeat carrying `audioState` is now kept per session
32
+ and can VETO a stale verdict — a phone that says it is recording is alive even with
33
+ no chunk arriving, because its uploads may merely be blocked. A BACKGROUNDED phone
34
+ counts as capturing; requiring `visibilityState: visible` would reap exactly the
35
+ sessions the drain path exists to rescue. Absence proves nothing in the other
36
+ direction: `clientLog` is fire-and-forget and lossy, so a missing heartbeat can
37
+ never itself mark a session dead. Chunk arrival decides.
38
+ - **One definition of stale.** `RECORDING_SESSION_STALE_MS` is now derived from
39
+ `STRANDED_STALE_MS` rather than being a second literal. Two subsystems disagreeing
40
+ about what a live recording is, is precisely how the panel could read "2
41
+ recording(s) active" while the orphan endpoint reported none.
42
+ - A failed auto-save does not throw away a savable capture: `no_token` and 5xx and
43
+ transport failures retry on the next sweep, 409 yields to the save that already owns
44
+ the session, and only a terminal 4xx (or an 8-hour backstop) falls back to the old
45
+ close-and-quarantine.
46
+
47
+ Coverage: 59 tests over the new modules, 23 mutations all caught. Three of those
48
+ initially SURVIVED — the whole stale branch deleted, the token gate deleted, and the
49
+ terminal draft cleanup deleted — because the loop lived inside a `setInterval` in a
50
+ module no test can import without executing boot recovery, a timer, and writes to the
51
+ real data home. It was extracted with injected dependencies rather than covered by
52
+ source-shape assertions. Full suite green serially (1411 tests); two unrelated files
53
+ flake under file parallelism, which is a pre-existing isolation bug.
54
+
1
55
  ## 6.22.1
2
56
 
3
57
  Notes attached from somewhere else by a symlink are now read properly. Found by
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "@gotcos/glasses-server",
3
- "version": "6.22.1",
4
- "description": "COS Glasses self-hosted AI heads-up-display server for Even G2 smart glasses, powered by Claude Code, Codex, or Cursor Agent CLI",
3
+ "version": "6.23.0",
4
+ "description": "COS Glasses \u2014 self-hosted AI heads-up-display server for Even G2 smart glasses, powered by Claude Code, Codex, or Cursor Agent CLI",
5
5
  "type": "module",
6
6
  "bin": {
7
7
  "glasses-server": "bin/cli.cjs",
@@ -0,0 +1,62 @@
1
+ // Last-known companion state per recording session.
2
+ //
3
+ // The stranded-session sweeper needs POSITIVE evidence that a phone is still
4
+ // capturing before it declares a quiet session stale. That evidence already
5
+ // arrives on POST /api/diag/client as a heartbeat carrying `audioState` and
6
+ // `visibilityState`, but it was only ever appended to client-diagnostics.jsonl —
7
+ // a file the sweeper would have to parse on every 60s tick. This keeps the newest
8
+ // heartbeat per session in memory instead.
9
+ //
10
+ // DELIBERATELY LOSSY, and safe to be. `clientLog` is fire-and-forget with a 3s
11
+ // abort and a silent catch: 63% of heartbeats were measured missing during one
12
+ // 2026-07-27 session. So an ABSENT heartbeat proves nothing and must never by
13
+ // itself mark a session dead — the sweeper's staleness signal is chunk arrival,
14
+ // and a heartbeat can only ever VETO a stale verdict, never cause one.
15
+
16
+ import type { SessionHeartbeat } from './stranded-sessions.js'
17
+
18
+ /**
19
+ * Bounded so a burst of zombie clients cannot grow this without limit. Well above
20
+ * any real fleet: a session is one phone, and the sweeper deletes entries as
21
+ * sessions close.
22
+ */
23
+ const MAX_TRACKED_SESSIONS = 64
24
+
25
+ const heartbeats = new Map<string, SessionHeartbeat>()
26
+
27
+ /** Record the newest heartbeat for a session. Older timestamps are ignored. */
28
+ export function recordSessionHeartbeat(
29
+ sessionId: string,
30
+ heartbeat: SessionHeartbeat,
31
+ ): void {
32
+ if (!sessionId) return
33
+ if (!Number.isFinite(heartbeat.at)) return
34
+ const existing = heartbeats.get(sessionId)
35
+ if (existing && existing.at > heartbeat.at) return
36
+ heartbeats.set(sessionId, heartbeat)
37
+ if (heartbeats.size <= MAX_TRACKED_SESSIONS) return
38
+ // Evict oldest-first so the entries that matter (recent, therefore capable of
39
+ // vetoing a stale verdict) are the ones retained.
40
+ const ordered = [...heartbeats.entries()].sort((a, b) => a[1].at - b[1].at)
41
+ for (const [id] of ordered.slice(0, heartbeats.size - MAX_TRACKED_SESSIONS)) {
42
+ heartbeats.delete(id)
43
+ }
44
+ }
45
+
46
+ export function getSessionHeartbeat(sessionId: string): SessionHeartbeat | null {
47
+ return heartbeats.get(sessionId) ?? null
48
+ }
49
+
50
+ /** Called when a session reaches any terminal state. */
51
+ export function forgetSessionHeartbeat(sessionId: string): void {
52
+ heartbeats.delete(sessionId)
53
+ }
54
+
55
+ /** Test seam only. */
56
+ export function resetSessionHeartbeats(): void {
57
+ heartbeats.clear()
58
+ }
59
+
60
+ export function trackedHeartbeatCount(): number {
61
+ return heartbeats.size
62
+ }
@@ -0,0 +1,302 @@
1
+ // What the sweeper DOES about a stranded session: draft it, then promote it.
2
+ //
3
+ // Two actions, deliberately asymmetric.
4
+ //
5
+ // DRAFT (at the stale threshold) is additive and reversible. It writes the
6
+ // transcript the session already holds to a file and touches nothing else — no
7
+ // store write, no audio move, no session close. That matters because a phone that
8
+ // has been silent for 25 minutes can still drain its IndexedDB buffer into the
9
+ // same session id, and closing early would answer that drain with 410 Gone. A
10
+ // draft cannot truncate anything; it only makes the capture readable and visible
11
+ // while the session stays open for the tail.
12
+ //
13
+ // It also stays OUT of the meetings store on purpose. `POST /api/meeting/save` is
14
+ // idempotent by session id — a second save replays the first receipt. If a draft
15
+ // occupied that id, the real save arriving later with the drained tail would find
16
+ // `alreadySaved` and return the SHORTER checkpoint forever. The draft is a
17
+ // sidecar, never a meeting.
18
+ //
19
+ // PROMOTE (at the retention cutoff) goes through POST /api/meeting/save over
20
+ // loopback rather than calling the store directly. That route owns the maintenance
21
+ // lease, the save-in-progress lock, the receipt contract, domain inference and the
22
+ // two-phase finalization job; reimplementing any of it here would fork the most
23
+ // load-bearing path in the product. The session is still in memory at this point,
24
+ // so the save keeps the live ASR transcript and its speaker labels — which is why
25
+ // promote must not use the quarantine recover route, whose output labels every
26
+ // speaker Unknown because no live ASR ever ran on it.
27
+
28
+ import { existsSync, mkdirSync, readFileSync, readdirSync, rmSync, statSync } from 'node:fs'
29
+ import { resolve } from 'node:path'
30
+ import { atomicWriteFileSync } from './atomic-fs.js'
31
+ import { forgetSessionHeartbeat } from './session-heartbeats.js'
32
+ import { classifyStrandedSession, type SessionHeartbeat } from './stranded-sessions.js'
33
+
34
+ /** Title given to a capture the sweeper saves on the user's behalf. */
35
+ export const STRANDED_PROMOTE_TITLE = 'Auto-saved capture'
36
+
37
+ const DRAFT_SUFFIX = '.draft.md'
38
+ /** A draft is a safety net, not an archive. Bounded so it cannot grow unwatched. */
39
+ const MAX_DRAFT_CHARS = 400_000
40
+
41
+ export interface StrandedDraftInput {
42
+ sessionId: string
43
+ transcript: string
44
+ chunkCount: number
45
+ startedAt: number
46
+ lastActivityAt: number
47
+ /** Injected for deterministic tests. */
48
+ now?: number
49
+ }
50
+
51
+ export interface StrandedDraft {
52
+ sessionId: string
53
+ path: string
54
+ bytes: number
55
+ chunkCount: number
56
+ updatedAt: string
57
+ }
58
+
59
+ function draftPath(dir: string, sessionId: string): string {
60
+ return resolve(dir, `${sessionId}${DRAFT_SUFFIX}`)
61
+ }
62
+
63
+ /**
64
+ * Write (or refresh) the draft for a stranded session.
65
+ *
66
+ * Returns null when there is nothing worth drafting, so an empty session cannot
67
+ * litter the directory with contentless files.
68
+ */
69
+ export function writeStrandedDraft(dir: string, input: StrandedDraftInput): string | null {
70
+ const transcript = input.transcript.trim()
71
+ if (!transcript) return null
72
+ if (!input.sessionId) return null
73
+ const now = input.now ?? Date.now()
74
+ const idleMin = Math.round(Math.max(0, now - input.lastActivityAt) / 60_000)
75
+ const capturedMin = Math.round(Math.max(0, input.lastActivityAt - input.startedAt) / 60_000)
76
+ const body = transcript.length > MAX_DRAFT_CHARS
77
+ ? `${transcript.slice(0, MAX_DRAFT_CHARS)}\n\n[draft truncated at ${MAX_DRAFT_CHARS} characters]`
78
+ : transcript
79
+ const doc = [
80
+ '---',
81
+ `session_id: ${input.sessionId}`,
82
+ `state: stranded_draft`,
83
+ `chunks_received: ${input.chunkCount}`,
84
+ `captured_minutes: ${capturedMin}`,
85
+ `idle_minutes: ${idleMin}`,
86
+ `drafted_at: ${new Date(now).toISOString()}`,
87
+ '---',
88
+ '',
89
+ '> Draft only. This capture stopped receiving audio and has not been saved as a',
90
+ '> meeting yet. The session is still open, so if the phone reconnects the rest of',
91
+ '> the audio will still arrive. It becomes a real meeting automatically at the',
92
+ `> retention cutoff, or immediately via POST /api/meeting/save.`,
93
+ '',
94
+ body,
95
+ '',
96
+ ].join('\n')
97
+ try {
98
+ mkdirSync(dir, { recursive: true })
99
+ atomicWriteFileSync(draftPath(dir, input.sessionId), doc, { mode: 0o600 })
100
+ return draftPath(dir, input.sessionId)
101
+ } catch {
102
+ return null
103
+ }
104
+ }
105
+
106
+ /** Every draft currently on disk, newest first. */
107
+ export function listStrandedDrafts(dir: string): StrandedDraft[] {
108
+ if (!existsSync(dir)) return []
109
+ const out: StrandedDraft[] = []
110
+ let entries: string[]
111
+ try { entries = readdirSync(dir) } catch { return [] }
112
+ for (const name of entries) {
113
+ if (!name.endsWith(DRAFT_SUFFIX)) continue
114
+ const full = resolve(dir, name)
115
+ try {
116
+ const st = statSync(full)
117
+ if (!st.isFile()) continue
118
+ const head = readFileSync(full, 'utf8').slice(0, 800)
119
+ const chunks = /^chunks_received: (\d+)$/m.exec(head)
120
+ out.push({
121
+ sessionId: name.slice(0, -DRAFT_SUFFIX.length),
122
+ path: full,
123
+ bytes: st.size,
124
+ chunkCount: chunks ? Number(chunks[1]) : 0,
125
+ updatedAt: new Date(st.mtimeMs).toISOString(),
126
+ })
127
+ } catch { /* a draft we cannot stat is not worth failing the sweep over */ }
128
+ }
129
+ return out.sort((a, b) => b.updatedAt.localeCompare(a.updatedAt))
130
+ }
131
+
132
+ /** Called once a session reaches a real terminal state. Never throws. */
133
+ export function clearStrandedDraft(dir: string, sessionId: string): void {
134
+ try { rmSync(draftPath(dir, sessionId), { force: true }) } catch {}
135
+ }
136
+
137
+ /**
138
+ * Drop every trace of stranded-state for a session that reached ANY terminal state.
139
+ *
140
+ * One function because the two halves must never diverge. A retained draft would
141
+ * keep /api/meeting/orphans advertising an unsaved capture that is now saved — the
142
+ * false alarm that trains a user to ignore the one channel built to report real
143
+ * losses. A retained heartbeat would let a dead session's last breath veto a stale
144
+ * verdict for a session id that no longer exists.
145
+ */
146
+ export function releaseStrandedState(dir: string, sessionId: string): void {
147
+ clearStrandedDraft(dir, sessionId)
148
+ forgetSessionHeartbeat(sessionId)
149
+ }
150
+
151
+ export interface SweepableSession {
152
+ lastActivityAt: number
153
+ startTime: number
154
+ chunkCount: number
155
+ }
156
+
157
+ export interface StrandedSweepInput {
158
+ now: number
159
+ /** Empty means this process does not own the API — see the guard below. */
160
+ token: string
161
+ draftDir: string
162
+ sessions: Iterable<[string, SweepableSession]>
163
+ getHeartbeat: (sessionId: string) => SessionHeartbeat | null
164
+ getTranscript: (sessionId: string) => string | null
165
+ /** Fire-and-forget: a promote can run for minutes and must not block the tick. */
166
+ onPromote: (sessionId: string, lastActivityAt: number) => void
167
+ }
168
+
169
+ export interface StrandedSweepResult {
170
+ drafted: string[]
171
+ promoted: string[]
172
+ /** Sessions the sweep deliberately left alone. */
173
+ live: string[]
174
+ }
175
+
176
+ /**
177
+ * One pass over the live sessions: draft what has gone quiet, promote what is done.
178
+ *
179
+ * EXTRACTED FROM THE INTERVAL ON PURPOSE. While this loop lived inside the
180
+ * `setInterval` in routes/transcribe-stream.ts no test could reach it, because
181
+ * importing that module executes boot recovery, a 60s timer, and reads and writes
182
+ * against the real data home. Two mutations proved the cost: deleting the entire
183
+ * stale branch, and deleting the token gate, both SURVIVED the whole suite. A loop
184
+ * that decides whether a recording ever becomes a meeting cannot be covered by
185
+ * source-shape assertions alone.
186
+ */
187
+ export function sweepStrandedSessions(input: StrandedSweepInput): StrandedSweepResult {
188
+ const result: StrandedSweepResult = { drafted: [], promoted: [], live: [] }
189
+ // Only the process that owns the API can save anything, and only it should be
190
+ // writing into the data home. No token means a test worker or a tool that
191
+ // imported the module, so the pass does nothing rather than half-acting.
192
+ if (!input.token) return result
193
+ for (const [sessionId, session] of input.sessions) {
194
+ const verdict = classifyStrandedSession(
195
+ { lastActivityAt: session.lastActivityAt, heartbeat: input.getHeartbeat(sessionId) },
196
+ input.now,
197
+ )
198
+ if (verdict === 'live') { result.live.push(sessionId); continue }
199
+ if (verdict === 'stale') {
200
+ // Draft only. The session stays OPEN so a phone that reconnects can still
201
+ // drain its buffered tail into this same id.
202
+ const written = writeStrandedDraft(input.draftDir, {
203
+ sessionId,
204
+ transcript: input.getTranscript(sessionId) ?? '',
205
+ chunkCount: session.chunkCount,
206
+ startedAt: session.startTime,
207
+ lastActivityAt: session.lastActivityAt,
208
+ now: input.now,
209
+ })
210
+ if (written) result.drafted.push(sessionId)
211
+ continue
212
+ }
213
+ input.onPromote(sessionId, session.lastActivityAt)
214
+ result.promoted.push(sessionId)
215
+ }
216
+ return result
217
+ }
218
+
219
+ export interface PromoteResult {
220
+ ok: boolean
221
+ status: number
222
+ filename?: string
223
+ reason?: string
224
+ }
225
+
226
+ /**
227
+ * After a failed auto-save, does the session close (quarantining its audio) or wait
228
+ * for the next sweep?
229
+ *
230
+ * Extracted from the sweeper because it is the decision with the real consequence:
231
+ * closing means the capture becomes quarantined audio and NOT a meeting, which is
232
+ * the exact outcome this release exists to prevent. Inside the route it was
233
+ * untestable without importing and therefore executing the whole module.
234
+ *
235
+ * @param idleForMs how long the session has been without a chunk
236
+ * @param giveUpAfterMs backstop so a permanently-failing save cannot keep a
237
+ * session in memory forever
238
+ */
239
+ export function shouldCloseAfterFailedPromote(
240
+ result: PromoteResult,
241
+ idleForMs: number,
242
+ giveUpAfterMs: number,
243
+ ): boolean {
244
+ if (result.ok) return false
245
+ // Never reached the server, so nothing was learned about whether it could be
246
+ // saved. Closing here would destroy a savable capture.
247
+ if (result.reason === 'no_token') return false
248
+ // Another save already owns this session. That one wins.
249
+ if (result.status === 409) return idleForMs >= giveUpAfterMs
250
+ // A transport failure, a timeout, or a 5xx is transient: retry next sweep.
251
+ const terminal = result.status >= 400 && result.status < 500
252
+ return terminal || idleForMs >= giveUpAfterMs
253
+ }
254
+
255
+ export interface PromoteOptions {
256
+ port: number
257
+ token: string
258
+ /** Injected in tests; defaults to global fetch. */
259
+ fetchImpl?: typeof fetch
260
+ timeoutMs?: number
261
+ }
262
+
263
+ /**
264
+ * Finalize a stranded session into a meeting through the real save route.
265
+ *
266
+ * Omits `domain` deliberately so the route's own keyword inference files it,
267
+ * rather than dumping every unattended capture into one folder.
268
+ */
269
+ export async function promoteStrandedSession(
270
+ sessionId: string,
271
+ options: PromoteOptions,
272
+ ): Promise<PromoteResult> {
273
+ // No token means this process is not the server that owns the API — a unit test
274
+ // worker, a tool importing the module. Refusing here matters because an
275
+ // unauthenticated POST returns 401, the caller would read a 4xx as terminal, and
276
+ // it would then close a session it could perfectly well have saved. `no_token`
277
+ // is deliberately NOT terminal.
278
+ if (!options.token) return { ok: false, status: 0, reason: 'no_token' }
279
+ const doFetch = options.fetchImpl ?? fetch
280
+ const controller = new AbortController()
281
+ const timer = setTimeout(() => controller.abort(), options.timeoutMs ?? 600_000)
282
+ try {
283
+ const res = await doFetch(`http://127.0.0.1:${options.port}/api/meeting/save`, {
284
+ method: 'POST',
285
+ headers: { 'Content-Type': 'application/json', 'X-Cos-Token': options.token },
286
+ body: JSON.stringify({ sessionId, title: STRANDED_PROMOTE_TITLE }),
287
+ signal: controller.signal,
288
+ })
289
+ let payload: Record<string, unknown> = {}
290
+ try { payload = await res.json() as Record<string, unknown> } catch {}
291
+ return {
292
+ ok: res.ok,
293
+ status: res.status,
294
+ filename: typeof payload.filename === 'string' ? payload.filename : undefined,
295
+ reason: typeof payload.reason === 'string' ? payload.reason : undefined,
296
+ }
297
+ } catch (error: any) {
298
+ return { ok: false, status: 0, reason: error?.name === 'AbortError' ? 'timeout' : 'request_failed' }
299
+ } finally {
300
+ clearTimeout(timer)
301
+ }
302
+ }
@@ -0,0 +1,125 @@
1
+ // A recording whose phone went away must still become a meeting.
2
+ //
3
+ // THE DEFECT THIS FIXES. When the companion quits, the bridge drops, or the G2
4
+ // disconnects mid-meeting, the server keeps the session in memory and keeps its
5
+ // ACK'd chunks on disk. Nothing then converts them into a meeting. At the 4-hour
6
+ // retention cutoff the session was closed as 'expired' and its audio moved to
7
+ // quarantine — preserved for 72 more hours, but still not a meeting, and visible
8
+ // only to whoever thought to look. Two sessions sat stranded for 184 and 24
9
+ // minutes on 2026-08-09 holding the restart lock while /api/meeting/orphans
10
+ // reported nothing at all, because that endpoint lists only QUARANTINED dirs and
11
+ // a stranded session is not quarantined for four hours.
12
+ //
13
+ // WHY THE OBVIOUS FIX IS WRONG. "Close idle sessions sooner" loses data. The
14
+ // companion buffers capture to IndexedDB while iOS suspends the WebView and
15
+ // drains on foreground (local-first-meeting-uploader.ts keeps a retry queue), and
16
+ // it restores `restoredSessionId` across a relaunch — so a phone that has been
17
+ // silent for 25 minutes can still deliver its tail into the SAME session id. Once
18
+ // the session is closed, `isSessionDeleted` answers 410 Gone and that tail is
19
+ // gone, leaving a truncated meeting AND a second orphan. So staleness must never
20
+ // close a session early. It may only make the state visible and durable.
21
+ //
22
+ // QUIET IS NOT FAILURE. `lastActivityAt` is bumped in exactly one place —
23
+ // `processStreamChunk` — so it is a pure chunk-ARRIVAL signal and a zombie
24
+ // client's heartbeats cannot mask a dead session. The inverse also has to hold:
25
+ // a phone that reports it is recording right now is alive even with no chunk
26
+ // arriving, because its uploads may merely be blocked. That is why a fresh
27
+ // heartbeat suppresses the stale verdict, and why a BACKGROUNDED phone is not
28
+ // treated as dead — buffering while suspended is designed behavior, not a fault.
29
+
30
+ import { LOCAL_FIRST_MEETING_IDLE_RETENTION_MS } from './local-first-meetings-contract.js'
31
+
32
+ /**
33
+ * No ACK'd chunk for this long and the session is worth telling the user about.
34
+ *
35
+ * MUST equal `RECORDING_SESSION_STALE_MS` in routes/transcribe-stream.ts, which
36
+ * already defines staleness for the maintenance drain gate and for aborting batch
37
+ * work while a recording is live. A second, different threshold is exactly the bug
38
+ * this release fixes: the session counter said "2 recordings active" while
39
+ * /api/meeting/orphans said nothing, because two subsystems disagreed about what
40
+ * counts as a real recording. One definition, or the disagreement comes back.
41
+ * `stranded-sessions.test.ts` imports both and fails if they drift.
42
+ *
43
+ * It lives here rather than being imported from the route because lib must not
44
+ * depend on routes; the test is what keeps them married.
45
+ *
46
+ * Chunk cadence is roughly 10s, so 30 minutes is ~180 missed chunks: decisively
47
+ * not a hiccup. It is deliberately NOT a close — see the header. Nothing is
48
+ * destroyed at this threshold; a draft is written and the session stays open so a
49
+ * late drain still lands.
50
+ */
51
+ export const STRANDED_STALE_MS = 30 * 60_000
52
+
53
+ /**
54
+ * The session is finished and becomes a meeting.
55
+ *
56
+ * Identical to the cutoff that already closed sessions as 'expired', so this
57
+ * changes the DISPOSITION (save instead of quarantine) and not the timing. Do not
58
+ * shorten it without re-reading the drain discussion above.
59
+ */
60
+ export const STRANDED_PROMOTE_MS = LOCAL_FIRST_MEETING_IDLE_RETENTION_MS
61
+
62
+ /** A heartbeat older than this proves nothing about the phone's state now. */
63
+ export const LIVENESS_GRACE_MS = 3 * 60_000
64
+
65
+ /**
66
+ * Pipeline states that mean audio is being captured this moment.
67
+ *
68
+ * Sourced from the companion's audio pipeline, whose recording state is
69
+ * `recording_continuous` for a meeting; `recording` covers the dictation path.
70
+ */
71
+ const CAPTURING_AUDIO_STATES = new Set(['recording_continuous', 'recording'])
72
+
73
+ export type StrandedVerdict = 'live' | 'stale' | 'promote'
74
+
75
+ export interface SessionHeartbeat {
76
+ /** ms since epoch when the heartbeat arrived. */
77
+ at: number
78
+ audioState?: string | null
79
+ visibilityState?: string | null
80
+ }
81
+
82
+ export interface SessionActivity {
83
+ /** ms since epoch of the last ACK'd chunk. NOT bumped by heartbeats. */
84
+ lastActivityAt: number
85
+ /** Most recent heartbeat for this session, when one has ever arrived. */
86
+ heartbeat?: SessionHeartbeat | null
87
+ }
88
+
89
+ /** How long this session has been without an ACK'd chunk. Never negative. */
90
+ export function strandedForMs(activity: SessionActivity, now: number): number {
91
+ return Math.max(0, now - activity.lastActivityAt)
92
+ }
93
+
94
+ /**
95
+ * Is the phone capturing audio right now?
96
+ *
97
+ * Deliberately does NOT require `visibilityState === 'visible'`. A backgrounded
98
+ * companion buffering to IndexedDB is the single most common legitimate reason
99
+ * for quiet, and demanding visibility here would reap exactly the sessions the
100
+ * drain path exists to rescue.
101
+ */
102
+ export function capturingNow(heartbeat: SessionHeartbeat | null | undefined, now: number): boolean {
103
+ if (!heartbeat) return false
104
+ if (!Number.isFinite(heartbeat.at)) return false
105
+ if (now - heartbeat.at > LIVENESS_GRACE_MS) return false
106
+ // A heartbeat from the future is a clock artifact, not evidence of capture.
107
+ if (heartbeat.at - now > LIVENESS_GRACE_MS) return false
108
+ return CAPTURING_AUDIO_STATES.has((heartbeat.audioState ?? '').trim())
109
+ }
110
+
111
+ /**
112
+ * What should the sweeper do with this session?
113
+ *
114
+ * `promote` is unconditional at the retention cutoff, matching the close it
115
+ * replaces — a session that has been silent for four hours becomes a meeting even
116
+ * if something is still heartbeating, because that is already the behavior today
117
+ * and weakening it would strand audio for longer than the old code did.
118
+ */
119
+ export function classifyStrandedSession(activity: SessionActivity, now: number): StrandedVerdict {
120
+ const idleFor = strandedForMs(activity, now)
121
+ if (idleFor >= STRANDED_PROMOTE_MS) return 'promote'
122
+ if (capturingNow(activity.heartbeat, now)) return 'live'
123
+ if (idleFor >= STRANDED_STALE_MS) return 'stale'
124
+ return 'live'
125
+ }
@@ -18,6 +18,7 @@ import { fileURLToPath } from 'node:url'
18
18
 
19
19
  const __dirname = fileURLToPath(new URL('.', import.meta.url))
20
20
  import { dataPath } from '../lib/data-dir.js'
21
+ import { recordSessionHeartbeat } from '../lib/session-heartbeats.js'
21
22
  const DIAG_DIR = dataPath()
22
23
  const DIAG_FILE = resolve(DIAG_DIR, 'client-diagnostics.jsonl')
23
24
  const MAX_FILE_BYTES = 10 * 1024 * 1024 // 10 MB cap
@@ -75,6 +76,19 @@ diagRouter.post('/diag/client', async (req, res) => {
75
76
  return res.status(410).json({ error: 'session_deleted', sessionId })
76
77
  }
77
78
  } catch { /* import failure: fall through to normal logging */ }
79
+ // Positive liveness for the stranded-session sweeper. A phone can be capturing
80
+ // while its uploads are blocked, so chunk silence alone must not mark a
81
+ // session stale — this is the only signal that can say "still recording".
82
+ // Lossy by nature (fire-and-forget, 3s abort, silent catch), which is why it
83
+ // may only VETO a stale verdict and never cause one.
84
+ const heartbeatData = data as Record<string, unknown>
85
+ recordSessionHeartbeat(sessionId, {
86
+ at: ts,
87
+ audioState: typeof heartbeatData.audioState === 'string' ? heartbeatData.audioState : null,
88
+ visibilityState: typeof heartbeatData.visibilityState === 'string'
89
+ ? heartbeatData.visibilityState
90
+ : null,
91
+ })
78
92
  }
79
93
 
80
94
  // Rate limit per sessionId (or remote address as fallback)
@@ -41,6 +41,7 @@ import { getLocalTtsHealth, refreshLocalTtsHealth } from '../lib/tts-local.js'
41
41
  import { liveCuesCapability } from '../lib/live-cues-capability.js'
42
42
  import { getMeetingSyncSnapshot } from '../lib/meeting-batch-progress.js'
43
43
  import { listUnsavedCaptures } from '../lib/unsaved-audio-quarantine.js'
44
+ import { getStrandedCaptures } from './transcribe-stream.js'
44
45
  import { getWhisperPreviewCapability } from '../lib/whisper-preview.js'
45
46
  import { getTranscriptionProfileStatus } from '../lib/profile.js'
46
47
  import { getHealthStaticProbes } from '../lib/health-static-probes.js'
@@ -235,6 +236,23 @@ healthRouter.get('/health', async (_req, res) => {
235
236
  recovered: item.recovered,
236
237
  })),
237
238
  }
239
+ // Captures that stopped receiving audio but are NOT yet quarantined, so they do
240
+ // not appear in unsaved_captures for up to four hours. This is the state COS
241
+ // Control could not see on 2026-08-09: its panel read "2 recording(s) active.
242
+ // Restart is locked." while both had been silent for 184 and 24 minutes. Compact
243
+ // here; full detail on the authenticated orphans route.
244
+ const strandedList = getStrandedCaptures()
245
+ const stranded_captures = {
246
+ count: strandedList.length,
247
+ items: strandedList.slice(0, 10).map(item => ({
248
+ sessionId: item.sessionId,
249
+ idleMinutes: item.idleMinutes,
250
+ capturedMinutes: item.capturedMinutes,
251
+ chunks: item.chunks,
252
+ promotesAt: item.promotesAt,
253
+ hasDraft: item.draftPath != null,
254
+ })),
255
+ }
238
256
  const meetingLibrary = resolveMeetingLibrary()
239
257
  res.json({
240
258
  ...checks,
@@ -257,6 +275,7 @@ healthRouter.get('/health', async (_req, res) => {
257
275
  warningCount: meetingLibrary.warnings.length,
258
276
  },
259
277
  unsaved_captures,
278
+ stranded_captures,
260
279
  chunk_embeddings: chunkEmbeddings,
261
280
  speaker_corrections: speakerCorrections,
262
281
  review_audio: reviewAudio,
@@ -101,6 +101,7 @@ import {
101
101
  getSessionStartTime,
102
102
  getSessionTranscript,
103
103
  getMeetingSessionStatus,
104
+ getStrandedCaptures,
104
105
  getTranscriptionSessionLiveness,
105
106
  hasSessionAudio,
106
107
  moveSessionAudioToPending,
@@ -1664,8 +1665,16 @@ export function createMeetingRouter(deps: MeetingRouteDependencies = {}): Router
1664
1665
  ? { segmentsDone: progress.segmentsDone, segmentsTotal: progress.segmentsTotal }
1665
1666
  : null
1666
1667
  }
1668
+ // `stranded` is the state this endpoint used to be blind to: a capture whose
1669
+ // phone went away, still live in memory, audio intact, NOT yet quarantined and
1670
+ // therefore absent from `items` for a full four hours. That blindness is why
1671
+ // two stranded sessions could hold the restart lock on 2026-08-09 while this
1672
+ // route answered count: 0.
1673
+ const stranded = getStrandedCaptures()
1667
1674
  res.json({
1668
1675
  count: items.filter(item => !item.recovered).length,
1676
+ strandedCount: stranded.length,
1677
+ stranded,
1669
1678
  recovering: [...recoveringOrphans],
1670
1679
  recoveringProgress,
1671
1680
  items,
@@ -12,6 +12,17 @@ import { fileURLToPath } from 'node:url'
12
12
  import { getVocabulary, getOwnerName } from '../lib/profile.js'
13
13
  import { getOpenAIKey } from '../lib/openai-key.js'
14
14
  import { getTranscriptionPolicySnapshot, isOpenAIWhisperFallbackReady } from '../lib/transcription-policy.js'
15
+ import { STRANDED_STALE_MS } from '../lib/stranded-sessions.js'
16
+ import {
17
+ clearStrandedDraft,
18
+ listStrandedDrafts,
19
+ releaseStrandedState,
20
+ shouldCloseAfterFailedPromote,
21
+ sweepStrandedSessions,
22
+ promoteStrandedSession,
23
+ writeStrandedDraft,
24
+ } from '../lib/stranded-session-actions.js'
25
+ import { getSessionHeartbeat } from '../lib/session-heartbeats.js'
15
26
  import { TranscriptionUnavailableError } from '../lib/transcribe-audio.js'
16
27
 
17
28
  const __dirname = fileURLToPath(new URL('.', import.meta.url))
@@ -168,6 +179,9 @@ const _unused_hallucination_constants_placeholder = 0 as const
168
179
  const SESSION_AUDIO_DIR = dataPath('session-audio')
169
180
  ensurePrivateDirectory(SESSION_AUDIO_DIR)
170
181
  const PENDING_BATCH_DIR = dataPath('pending-batch')
182
+ // Readable text for a capture that stopped receiving audio. A sidecar, never a
183
+ // meeting — see lib/stranded-session-actions.ts for why it stays out of the store.
184
+ const STRANDED_DRAFT_DIR = dataPath('stranded-drafts')
171
185
  ensurePrivateDirectory(PENDING_BATCH_DIR)
172
186
  const MAX_SESSION_AUDIO_BYTES = 500 * 1024 * 1024 // 500MB cap per session (~2hr meeting ≈ 260MB)
173
187
  const PRESERVED_SESSION_AUDIO_MARKER = '_meeting_save_preserved.marker'
@@ -354,8 +368,15 @@ export function getActiveTranscriptionSessionCount(): number {
354
368
  * run to minutes. It sits deliberately far below
355
369
  * LOCAL_FIRST_MEETING_IDLE_RETENTION_MS (4h), which governs how long chunks stay
356
370
  * recoverable and is far too long to hold a restart on.
371
+ *
372
+ * DERIVED, never a second literal. `STRANDED_STALE_MS` is the one definition of
373
+ * "this recording has gone quiet", and the stranded sweeper, the drain gate and
374
+ * the batch-abort check must all mean the same thing by it. Two independent
375
+ * numbers here is exactly the defect this release fixes: on 2026-08-09 the
376
+ * session counter reported two active recordings while /api/meeting/orphans
377
+ * reported none, because two subsystems disagreed on what a live recording is.
357
378
  */
358
- export const RECORDING_SESSION_STALE_MS = 30 * 60 * 1000
379
+ export const RECORDING_SESSION_STALE_MS = STRANDED_STALE_MS
359
380
 
360
381
  export interface TranscriptionSessionLiveness {
361
382
  /** Sessions still plausibly recording. These block a restart. */
@@ -397,6 +418,53 @@ export interface TranscriptionSessionLiveness {
397
418
  */
398
419
  export const __sessionsForTests = sessions
399
420
 
421
+ export interface StrandedCapture {
422
+ sessionId: string
423
+ /** Minutes since the last ACK'd chunk. */
424
+ idleMinutes: number
425
+ /** Minutes of audio actually captured before it went quiet. */
426
+ capturedMinutes: number
427
+ chunks: number
428
+ /** When the sweeper will save this on the user's behalf. */
429
+ promotesAt: string
430
+ /** Readable draft on disk, once the capture has been stale long enough. */
431
+ draftPath: string | null
432
+ draftBytes: number | null
433
+ }
434
+
435
+ /**
436
+ * Captures that stopped receiving audio but have NOT been saved yet.
437
+ *
438
+ * This is the row that was missing. `/api/meeting/orphans` listed only
439
+ * QUARANTINED directories, and a stranded session is not quarantined until the
440
+ * 4-hour cutoff — so on 2026-08-09 it answered `count: 0` while two sessions sat
441
+ * stranded for 184 and 24 minutes holding the restart lock. Nothing is wrong with
442
+ * the quarantine list; it was simply blind to the state that matters most, the one
443
+ * where the audio is still live and still rescuable.
444
+ *
445
+ * Reuses `getTranscriptionSessionLiveness().staleSessions` rather than re-deriving
446
+ * staleness, so this view and the maintenance drain gate can never disagree.
447
+ */
448
+ export function getStrandedCaptures(now = Date.now()): StrandedCapture[] {
449
+ const drafts = new Map(listStrandedDrafts(STRANDED_DRAFT_DIR).map(d => [d.sessionId, d]))
450
+ return getTranscriptionSessionLiveness(now).staleSessions.map(stale => {
451
+ const session = sessions.get(stale.sessionId)
452
+ const lastActivityAt = session?.lastActivityAt ?? now - stale.silentForMs
453
+ const draft = drafts.get(stale.sessionId) ?? null
454
+ return {
455
+ sessionId: stale.sessionId,
456
+ idleMinutes: Math.round(stale.silentForMs / 60_000),
457
+ capturedMinutes: Math.round(
458
+ Math.max(0, lastActivityAt - (session?.startTime ?? lastActivityAt)) / 60_000,
459
+ ),
460
+ chunks: stale.chunks,
461
+ promotesAt: new Date(lastActivityAt + LOCAL_FIRST_MEETING_IDLE_RETENTION_MS).toISOString(),
462
+ draftPath: draft?.path ?? null,
463
+ draftBytes: draft?.bytes ?? null,
464
+ }
465
+ })
466
+ }
467
+
400
468
  export function getTranscriptionSessionLiveness(now = Date.now()): TranscriptionSessionLiveness {
401
469
  let live = 0
402
470
  const staleSessions: TranscriptionSessionLiveness['staleSessions'] = []
@@ -768,15 +836,83 @@ if (maintenanceAdmissionsOpen()) {
768
836
  recoverSessions()
769
837
  }
770
838
 
839
+ /** Sessions with an auto-save in flight, so a later tick cannot start a second. */
840
+ const promotingStranded = new Set<string>()
841
+
842
+ /**
843
+ * Hard backstop for a capture whose save keeps failing.
844
+ *
845
+ * Without it a session that the save route refuses transiently, forever, would
846
+ * live in memory forever. At twice the cutoff it takes the historical disposition
847
+ * — close and quarantine — so the audio is at least preserved on the unsaved-audio
848
+ * clock rather than held hostage by a retry loop.
849
+ */
850
+ const STRANDED_PROMOTE_GIVE_UP_MS = 2 * LOCAL_FIRST_MEETING_IDLE_RETENTION_MS
851
+
852
+ /**
853
+ * Turn an unattended capture into a meeting, and fall back to the old behavior if
854
+ * that proves impossible.
855
+ *
856
+ * Fire-and-forget on purpose: the sweep tick must not await an HQ finalization
857
+ * that can run for minutes. `promotingStranded` is what keeps the next tick from
858
+ * starting a duplicate.
859
+ */
860
+ function promoteStrandedCapture(sessionId: string, lastActivityAt: number): void {
861
+ if (promotingStranded.has(sessionId)) return
862
+ promotingStranded.add(sessionId)
863
+ void promoteStrandedSession(sessionId, {
864
+ port: parseInt(process.env.PORT ?? '3141', 10),
865
+ token: process.env.COS_API_TOKEN ?? '',
866
+ })
867
+ .then(result => {
868
+ if (result.ok) {
869
+ console.warn(
870
+ `[stranded] Auto-saved unattended capture ${sessionId} \u2192 ${result.filename ?? 'saved'}`,
871
+ )
872
+ clearStrandedDraft(STRANDED_DRAFT_DIR, sessionId)
873
+ return
874
+ }
875
+ // Delegated, not restated: closing means the capture becomes quarantined
876
+ // audio instead of a meeting, which is the outcome this release exists to
877
+ // prevent, so the rule lives in a module a test can reach.
878
+ const idleForMs = Date.now() - lastActivityAt
879
+ if (!shouldCloseAfterFailedPromote(result, idleForMs, STRANDED_PROMOTE_GIVE_UP_MS)) {
880
+ console.warn(
881
+ `[stranded] Auto-save deferred for ${sessionId} `
882
+ + `(${result.status} ${result.reason ?? ''}); retrying next sweep`,
883
+ )
884
+ return
885
+ }
886
+ console.error(
887
+ `[stranded] Auto-save failed for ${sessionId} (${result.status} ${result.reason ?? ''}); `
888
+ + 'closing as expired so the audio reaches quarantine',
889
+ )
890
+ if (sessions.has(sessionId)) closeTranscriptSession(sessionId, 'expired')
891
+ })
892
+ .finally(() => promotingStranded.delete(sessionId))
893
+ }
894
+
771
895
  // Auto-cleanup sessions idle for the advertised retention horizon.
772
896
  setInterval(() => {
773
897
  if (!maintenanceAdmissionsOpen()) return
774
- const cutoff = Date.now() - LOCAL_FIRST_MEETING_IDLE_RETENTION_MS
775
- for (const [id, session] of sessions) {
776
- if (session.lastActivityAt < cutoff) {
777
- closeTranscriptSession(id, 'expired')
778
- }
779
- }
898
+ // A capture whose phone went away must still become a meeting. This loop used
899
+ // to close every idle session as 'expired', which quarantined its audio and
900
+ // produced NO meeting — recoverable for 72h, but only by whoever thought to
901
+ // look. See lib/stranded-sessions.ts for why staleness may not close a session
902
+ // early and why the cutoff is unchanged.
903
+ sweepStrandedSessions({
904
+ now: Date.now(),
905
+ token: process.env.COS_API_TOKEN ?? '',
906
+ draftDir: STRANDED_DRAFT_DIR,
907
+ sessions: [...sessions].map(([id, session]) => [id, {
908
+ lastActivityAt: session.lastActivityAt,
909
+ startTime: session.startTime,
910
+ chunkCount: session.chunks.filter(Boolean).length,
911
+ }] as const),
912
+ getHeartbeat: getSessionHeartbeat,
913
+ getTranscript: id => getSessionTranscript(id),
914
+ onPromote: promoteStrandedCapture,
915
+ })
780
916
  // Orphaned session-audio dirs (no matching active session): quarantine any
781
917
  // dir still holding chunk audio; delete only chunk-less dirs. Quarantine
782
918
  // itself expires on the unsaved-audio retention clock, the ONLY place
@@ -1307,6 +1443,10 @@ function finishClosingTranscriptSession(
1307
1443
  clearSessionHallucinationState(sessionId)
1308
1444
  // Track as deleted so orphan heartbeats get 410 Gone (prevents zombie client spam)
1309
1445
  rememberDeletedSession(sessionId)
1446
+ // A session that reached ANY terminal state keeps neither a liveness veto nor a
1447
+ // draft. Leaving the draft would advertise an unsaved capture that is now saved,
1448
+ // which is the false-alarm that trains the user to ignore the channel.
1449
+ releaseStrandedState(STRANDED_DRAFT_DIR, sessionId)
1310
1450
  persistClosedSessions()
1311
1451
  // Clean up persisted file
1312
1452
  try { unlinkSync(resolve(CHUNK_PERSIST_DIR, `${sessionId}.json`)) } catch {}