@gotcos/glasses-server 6.50.1 → 6.50.3

package/CHANGELOG.md

@@ -1,3 +1,15 @@
+## 6.50.3
+
+- **The server referees between copies of the COS Glasses app.** 2026-09-18 10:06: the Even Realities app kept an older WebView copy of the plugin alive (hidden since at least 09:30) behind the one Miles opened, both answered one Start Meeting to the millisecond, and the same 90-minute meeting was recorded and HQ-polished twice. Glasses 6.9.505's claim in the Even app's storage could not stop it: the new copy read the old copy's claim 36 minutes stale, and the hidden copy never refreshed its claim or read the new one, while its HTTP calls to this server kept working (it uploaded all 827 chunks). `POST /api/client-instance/claim` `{ id, bootAt, version }` answers `{ verdict: 'owner' | 'yield', owner }`: the newest boot that is still posting owns the ring (ties by id); a claim from an older boot takes over only after the owner has been quiet for 45 s (`CLIENT_INSTANCE_LIVE_MS`, three 15 s ticks). `check: true` answers without taking, so a copy that already yielded can ask whether the owner is gone without stealing the ring after a phone lock froze both copies' timers. Behind the API token. In memory: a restart forgets the owner and the next posts re-establish it within a tick. Glasses 6.9.507 asks this route first and falls back to the storage claim when the Mac is unreachable or older.
+- Tests (`client-instance.test.ts`): the arbitration executed (the zombie yields, the owner refreshes, the older reclaims one ms past the live edge, a same-millisecond tie has one owner, malformed claims refused) and over HTTP (state kept across requests, a 400 changes nothing); the mount sits after `requireApiToken` and is not a public path. Gate: `referee-dead-owner-never-released`, `referee-older-boot-wins`, `referee-not-mounted`, 3/3.
+
+## 6.50.2
+
+- **The Turbo meeting preview no longer switches itself off for good.** Miles, 2026-09-18 08:43: the `~` provisional line on the lens was gone, Mac-reachable, Turbo configured. The preview route had answered "no text" to 5,719 requests since 2026-09-17 16:00 (in under a millisecond) while the Turbo sidecar itself transcribed the same audio in 0.27 s. One failed request (`The operation was aborted due to timeout`, then `fetch failed`, with the Mac under build load) set `previewAvailable = false`, and the only code that sets it back is the sidecar's own start, which runs once per server boot. Now a failure against a sidecar that is still running is a 10 s cooldown (`PREVIEW_FAILURE_COOLDOWN_MS`) and the next request after it tries again; five failures in a row (`PREVIEW_RECYCLE_AFTER_FAILURES`) recycle the sidecar; a sidecar that exited is restarted from the request path, at most once per 60 s window, doubling to 15 min while restarts keep failing and resetting on any answer. A missing model, a missing binary or a foreign listener on port 8177 is never restarted (a restart cannot fix it). After the server's own shutdown stop, nothing revives the sidecar (a request arriving during shutdown could otherwise have spawned one that outlived the process). Prompt previews were never affected (they fall back to the canonical worker); canonical meeting ASR never was. The preview log lines now carry an ISO timestamp: the lines that recorded this failure had none.
+- Tests (`whisper-preview.test.ts`, executed against spawn and fetch fakes and a fake clock): a timeout is a cooldown and the next request after it gets text; a success resets the count so scattered misses never recycle; five in a row recycle and the new sidecar answers; an exited sidecar restarts from the request path and the backoff doubles across failed restarts; a foreign listener is never re-probed. Eight mutants (latch restored, success never resets, no revive, backoff never doubles, cooldown ignored, foreign port restarted, revive after shutdown, start after shutdown) each turn the suite red; three of them survived the first draft of the tests and the tests were tightened (a recycle's kill is synchronous, its respawn is not, so negatives now wait for the loop to settle).
+- **Coming back to a Desktop tab no longer blocks a Continue for 30 seconds.** Miles, 2026-09-17 07:00: five sends from the glasses refused "Your Mac is writing to this thread right now" until he clicked away from the tab. Claude Desktop had written its away summary (the note it drops when you return to a tab you left) to the transcript 11 s earlier, and the sub-agent that wrote it fired `SubagentStop`. The attach gate reads a transcript written inside 30 s as working unless the hooks vouch that the turn ended, and the vouch required the newest hook event to be the `Stop` itself, so the desk's own bookkeeping after it made the gate strict. The vouch now reads "no turn since the Stop": a `SubagentStop` with no sub-agent open, or a notification, keeps it; anything that opens a turn (a prompt, a tool, a sub-agent starting) still refuses, and a newer Stop still moves the clock.
+- Tests: the two bookkeeping events keep the vouch and each other guard still holds behind them; a tool, a prompt, a sub-agent start and a permission request stay strict. Gate: `turns-b6-desk-bookkeeping-is-strict-again` and `turns-b6-any-event-vouches` added; the four B6 cases killed.
+
 ## 6.50.1
 
 What the QA pass on 6.50.0 found in the history it serves (6.50.0 was published before the pass finished; nothing it changed regressed, and a client that does not ask for `?turns` sees no difference).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@gotcos/glasses-server",
-  "version": "6.50.1",
+  "version": "6.50.3",
   "description": "COS Glasses \u2014 self-hosted AI heads-up-display server for Even G2 smart glasses, powered by Claude Code, Codex, Cursor Agent CLI, or local Ollama",
   "type": "module",
   "bin": {

package/server/index.ts

@@ -14,6 +14,7 @@ import { execSync } from 'node:child_process'
 import { randomBytes } from 'node:crypto'
 import { healthRouter } from './routes/health.js'
 import { diagRouter } from './routes/diag.js'
+import { createClientInstanceRouter } from './routes/client-instance.js'
 import { queryRouter } from './routes/query.js'
 import { providerProofRouter } from './routes/provider-proof.js'
 import { transcribeRouter } from './routes/transcribe.js'
@@ -530,6 +531,8 @@ const forkThreadForRoute = (request: {
 // API routes
 app.use('/api', healthRouter)
 app.use('/api', diagRouter)
+// 6.50.3: the referee between copies of the COS Glasses app (routes/client-instance.ts).
+app.use('/api', createClientInstanceRouter())
 app.use('/api', createQueryJobsRouter(queryJobCoordinator, {
   prepareAdmission: preparePublicDurableQueryAdmission,
 }))
@@ -858,7 +861,7 @@ async function gracefulShutdown(): Promise<void> {
   try { logActiveSessionsOnShutdown() } catch { /* best-effort flush */ }
   stopCodexModelCatalogRefresh()
   stopWhisperServer()
-  await stopWhisperPreviewServer()
+  await stopWhisperPreviewServer({ final: true })
   stopLocalTtsServer()
   clearTimeout(forceExit)
   process.exit(0)

package/server/lib/client-instance-claim.ts

@@ -0,0 +1,60 @@
+// Which copy of the COS Glasses app owns the ring (6.50.3). PURE: the clock is passed in.
+//
+// WHY THE SERVER. On 2026-09-18 the Even Realities app kept an older WebView copy of the
+// plugin alive behind a newly opened one and delivered every ring event to both, so one
+// Start Meeting recorded the same 90 minutes twice. Glasses 6.9.505 put a claim in the Even
+// app's own storage: the newest boot wins, the copy behind it yields. It failed in the field
+// because the hidden copy's storage calls went unanswered (its claim sat 36 min stale and it
+// never read the newer one), while its HTTP calls to this server kept working (it uploaded
+// all 827 chunks). This server is the one party every copy can still reach.
+//
+// RULE. One owner at a time. A claim from the owner refreshes it. A claim from a NEWER boot
+// (ties broken by id) takes ownership. A claim from an older boot takes ownership only when
+// the owner has gone quiet for CLIENT_INSTANCE_LIVE_MS (the newer copy died). In memory: a
+// server restart forgets the owner, and the next posts re-establish it within one tick.
+
+export const CLIENT_INSTANCE_TICK_MS = 15_000
+/** Three missed ticks: the owner is gone. */
+export const CLIENT_INSTANCE_LIVE_MS = 3 * CLIENT_INSTANCE_TICK_MS
+
+export interface ClientInstanceClaim {
+  id: string
+  bootAt: number
+  version: string
+}
+
+export interface ClientInstanceOwner extends ClientInstanceClaim {
+  seenAt: number
+}
+
+export type ClientInstanceVerdict = 'owner' | 'yield'
+
+const ID_PATTERN = /^[a-z0-9]{1,16}-[a-z0-9]{1,8}$/
+
+/** A body the route may act on, or null. Never throws. */
+export function parseClientInstanceClaim(body: unknown): ClientInstanceClaim | null {
+  if (!body || typeof body !== 'object') return null
+  const o = body as Record<string, unknown>
+  if (typeof o.id !== 'string' || !ID_PATTERN.test(o.id)) return null
+  if (typeof o.bootAt !== 'number' || !Number.isFinite(o.bootAt) || o.bootAt <= 0) return null
+  const version = typeof o.version === 'string' ? o.version.slice(0, 32) : ''
+  return { id: o.id, bootAt: o.bootAt, version }
+}
+
+function isNewer(a: ClientInstanceClaim, b: ClientInstanceClaim): boolean {
+  return a.bootAt > b.bootAt || (a.bootAt === b.bootAt && a.id > b.id)
+}
+
+/** The owner after `claim` arrives at `now`, and what to tell its sender. */
+export function arbitrateClientInstance(
+  owner: ClientInstanceOwner | null,
+  claim: ClientInstanceClaim,
+  now: number,
+): { owner: ClientInstanceOwner; verdict: ClientInstanceVerdict; took: boolean } {
+  const fresh: ClientInstanceOwner = { ...claim, seenAt: now }
+  if (!owner) return { owner: fresh, verdict: 'owner', took: true }
+  if (owner.id === claim.id) return { owner: { ...owner, seenAt: now }, verdict: 'owner', took: false }
+  if (isNewer(claim, owner)) return { owner: fresh, verdict: 'owner', took: true }
+  if (now - owner.seenAt > CLIENT_INSTANCE_LIVE_MS) return { owner: fresh, verdict: 'owner', took: true }
+  return { owner, verdict: 'yield', took: false }
+}

package/server/lib/occupancy-probes.ts

@@ -792,15 +792,28 @@ export function buildOccupancyProbes(
  *
  * `signalFor` is the signal store's reader, taking a full session id or the registry's
  * 8-character form; `registryIdleAfterStop` reads the registry record for the session.
- * The probe answers a Stop time only when BOTH vouch: the session's newest hook event is
- * that Stop, the turn is closed, no sub-agent is open, the session has not ended, and
- * the registry says idle at or after the Stop (the hooks have returned). Codex never
+ * The probe answers a Stop time only when BOTH vouch: nothing since the Stop has opened
+ * a turn (the desk's own bookkeeping after it does not count: the sub-agent that writes
+ * Claude Desktop's away summary, an idle notification), the turn is closed, no sub-agent
+ * is open, the session has not ended, and the registry says idle at or after the Stop
+ * (the hooks have returned). Codex never
  * asks (its ids are not Claude sessions, and a Codex thread id sharing a Claude prefix
  * must not read a Claude signal); `readHolderActivity` guards the provider too.
  *
  * Wired at the composition root only when `COS_SESSION_HOOKS` is on. Absent, the
  * gate is byte-for-byte 6.48.0.
  */
+/**
+ * Hook events that can follow a Stop without a turn having started (6.50.2). Miles,
+ * 2026-09-17 07:00: five sends refused `native_thread_working` because Claude Desktop
+ * wrote its `away_summary` system row to the transcript 11 s earlier and the sub-agent
+ * that wrote it fired SubagentStop, so the newest event was not 'Stop' and the 30 s
+ * transcript window held. Every event that DOES mean a turn sets `turnOpen`
+ * (UserPromptSubmit, a tool) or `subagentsOpen` (SubagentStart), which the guards
+ * behind this set still read; a newer Stop replaces `stopAt`.
+ */
+const AFTER_THE_TURN: ReadonlySet<string> = new Set(['Stop', 'SubagentStop', 'Notification'])
+
 export function withHookTurnClock(
   probes: OccupancyProbes,
   signalFor: (sessionId: string) => HookTurnSignal | undefined,
@@ -812,7 +825,7 @@ export function withHookTurnClock(
       if (provider !== 'claude') return null
       if (!NATIVE_THREAD_ID_RE.test(threadId)) return null
       const signal = signalFor(threadId)
-      if (!signal || signal.lastEvent !== 'Stop' || signal.turnOpen || signal.subagentsOpen > 0 || signal.ended !== null) return null
+      if (!signal || !AFTER_THE_TURN.has(signal.lastEvent) || signal.turnOpen || signal.subagentsOpen > 0 || signal.ended !== null) return null
       if (typeof signal.stopAt !== 'number') return null
       // The engine's own end of turn: the registry flipped idle AFTER this Stop, which
       // happens only once every Stop hook has returned. Anything else is strict.

package/server/lib/whisper-preview.ts

@@ -69,6 +69,82 @@ let previewFailure: WhisperPreviewReason = null
 let previewWorkerModel: WhisperPreviewSidecarModel | null = null
 let warnedInvalidChoice = false
 
+/**
+ * 6.50.2: a request that fails against a sidecar that is STILL RUNNING is a miss, not a
+ * death. Before this, one timeout (the Mac under load) or one refused fetch set
+ * `previewAvailable = false`, and its only writer back to true is the sidecar's own start,
+ * which runs once per server boot, so the meeting preview lane stayed silent for the rest
+ * of the process: 2026-09-17 16:00 to 2026-09-18 08:48 (34 h), 5,719 requests answered 204
+ * in under a millisecond while Turbo itself answered the same audio in 0.27 s.
+ *
+ * Now: a failure against a live sidecar starts a short cooldown and the next request after
+ * it tries again. PREVIEW_RECYCLE_AFTER_FAILURES in a row recycles the sidecar (stop, then
+ * start). A sidecar that is gone is restarted from the request path, backing off from
+ * PREVIEW_RESTART_BASE_MS to PREVIEW_RESTART_MAX_MS so a model that cannot load is not
+ * reloaded every minute. Any successful answer resets both counters.
+ */
+export const PREVIEW_FAILURE_COOLDOWN_MS = 10_000
+export const PREVIEW_RECYCLE_AFTER_FAILURES = 5
+export const PREVIEW_RESTART_BASE_MS = 60_000
+export const PREVIEW_RESTART_MAX_MS = 15 * 60_000
+let previewCooldownUntil = 0
+let previewConsecutiveFailures = 0
+let previewLastRestartAt = 0
+let previewRestartBackoffMs = PREVIEW_RESTART_BASE_MS
+/** Set by the server's shutdown: no request may revive the sidecar after it (QA, 6.50.2). */
+let previewShutdown = false
+
+function previewSidecarAlive(): boolean {
+  const child = previewProcess
+  return !!child && child.exitCode === null && child.signalCode === null
+}
+
+function notePreviewRequestSuccess(): void {
+  previewConsecutiveFailures = 0
+  previewCooldownUntil = 0
+  previewRestartBackoffMs = PREVIEW_RESTART_BASE_MS
+}
+
+/** A request to the sidecar failed (not a preemption). Cooldown, recycle, or mark it gone. */
+function notePreviewRequestFailure(error: unknown, label: string): void {
+  previewConsecutiveFailures++
+  const message = error instanceof Error ? error.message : String(error)
+  const at = new Date().toISOString()
+  if (previewSidecarAlive() && previewConsecutiveFailures < PREVIEW_RECYCLE_AFTER_FAILURES) {
+    previewCooldownUntil = Date.now() + PREVIEW_FAILURE_COOLDOWN_MS
+    console.warn(`[whisper-preview] ${at} ${label} failed (${previewConsecutiveFailures} in a row); next try in ${PREVIEW_FAILURE_COOLDOWN_MS / 1000} s: ${message}`)
+    return
+  }
+  const alive = previewSidecarAlive()
+  previewAvailable = false
+  previewWorkerModel = null
+  previewFailure = 'preview_sidecar_unavailable'
+  console.warn(`[whisper-preview] ${at} ${label} failed (${previewConsecutiveFailures} in a row); ${alive ? 'recycling the sidecar' : 'the sidecar is gone'}: ${message}`)
+  maybeRevivePreviewSidecar()
+}
+
+/**
+ * Bring a failed sidecar back, from the request path. Only for a sidecar that ran and then
+ * failed or exited: a missing model, a missing binary or a foreign listener on the port
+ * cannot be fixed by a restart and stay as they are.
+ */
+function maybeRevivePreviewSidecar(): void {
+  if (previewShutdown || previewAvailable || previewStarting) return
+  if (previewFailure !== 'preview_sidecar_unavailable' && previewFailure !== 'preview_start_failed') return
+  const now = Date.now()
+  if (previewLastRestartAt && now - previewLastRestartAt < previewRestartBackoffMs) return
+  if (previewLastRestartAt) previewRestartBackoffMs = Math.min(previewRestartBackoffMs * 2, PREVIEW_RESTART_MAX_MS)
+  previewLastRestartAt = now
+  previewConsecutiveFailures = 0
+  console.log(`[whisper-preview] ${new Date(now).toISOString()} restarting the preview sidecar (${previewFailure}); next restart no sooner than ${Math.round(previewRestartBackoffMs / 1000)} s`)
+  void (async () => {
+    if (previewProcess) await stopWhisperPreviewServer()
+    await startWhisperPreviewServer()
+  })().catch(error => {
+    console.warn(`[whisper-preview] restart failed: ${error instanceof Error ? error.message : error}`)
+  })
+}
+
 interface ProcessEntry {
   pid: number
   ppid: number
@@ -260,6 +336,9 @@ async function endpointReady(path: '/health' | '/inference', init?: RequestInit,
  * uses Turbo while its canonical live worker remains Large-v3. Failure is
  * cosmetic: every recovery/finalization path stays untouched. */
 export async function startWhisperPreviewServer(): Promise<void> {
+  // A revive already past its stop when shutdown landed must not spawn an orphan that
+  // outlives the server.
+  if (previewShutdown) return
   const requested = requestedPreviewModel()
   const selected = selectedPreviewModel(requested)
   if (!needsPreviewSidecar(selected) || previewProcess || previewAvailable || previewStarting) return
@@ -359,7 +438,8 @@ function waitForPreviewClose(child: ChildProcess, timeoutMs: number): Promise<bo
   })
 }
 
-export async function stopWhisperPreviewServer(): Promise<void> {
+export async function stopWhisperPreviewServer(options: { final?: boolean } = {}): Promise<void> {
+  if (options.final) previewShutdown = true
   const child = previewProcess
   previewProcess = null
   previewAvailable = false
@@ -395,27 +475,23 @@ export async function transcribeWhisperPreview(audioBuffer: Buffer): Promise<{
   model: 'small.en' | WhisperCommitModel
   backend: 'whisper-preview-server' | 'whisper-server'
   }> {
-  if (previewAvailable && previewWorkerModel) {
+  if (!previewAvailable) maybeRevivePreviewSidecar()
+  if (previewAvailable && previewWorkerModel && Date.now() >= previewCooldownUntil) {
     const workerModel = previewWorkerModel
     const previewLease = tryAcquireMetalPreview()
     if (!previewLease) {
       return { text: '', model: workerModel, backend: 'whisper-preview-server' }
     }
     try {
-      return {
-        text: await transcribeViaPreviewServer(audioBuffer, previewLease.signal),
-        model: workerModel,
-        backend: 'whisper-preview-server',
-      }
+      const text = await transcribeViaPreviewServer(audioBuffer, previewLease.signal)
+      notePreviewRequestSuccess()
+      return { text, model: workerModel, backend: 'whisper-preview-server' }
     } catch (error) {
       if (previewLease.signal.aborted) {
         return { text: '', model: workerModel, backend: 'whisper-preview-server' }
       }
-      const failedModel = previewWorkerModel
-      previewAvailable = false
-      previewWorkerModel = null
-      previewFailure = 'preview_sidecar_unavailable'
-      console.warn(`[whisper-preview] ${failedModel} preview failed; falling back to canonical worker: ${error instanceof Error ? error.message : error}`)
+      // Falls through to the canonical worker below for THIS request either way.
+      notePreviewRequestFailure(error, `${workerModel} preview`)
     } finally {
       previewLease.release()
     }
@@ -448,21 +524,20 @@ export async function transcribeWhisperMeetingPreview(audioBuffer: Buffer): Prom
   model: 'large-v3-turbo'
   backend: 'whisper-preview-server'
 } | null> {
+  if (!previewAvailable) maybeRevivePreviewSidecar()
   if (!previewAvailable || previewWorkerModel !== 'large-v3-turbo') return null
+  // A cooldown after a failed request is a quiet miss, never a latch (see above).
+  if (Date.now() < previewCooldownUntil) return null
   const previewLease = tryAcquireMetalPreview()
   if (!previewLease) return null
   try {
-    return {
-      text: await transcribeViaPreviewServer(audioBuffer, previewLease.signal),
-      model: 'large-v3-turbo',
-      backend: 'whisper-preview-server',
-    }
+    const text = await transcribeViaPreviewServer(audioBuffer, previewLease.signal)
+    notePreviewRequestSuccess()
+    return { text, model: 'large-v3-turbo', backend: 'whisper-preview-server' }
   } catch (error) {
     if (previewLease.signal.aborted) return null
-    previewAvailable = false
-    previewWorkerModel = null
-    previewFailure = 'preview_sidecar_unavailable'
-    console.warn(`[whisper-preview] meeting Turbo preview failed; canonical meeting ASR was not affected: ${error instanceof Error ? error.message : error}`)
+    // Canonical meeting ASR is not affected by any of this: the preview is cosmetic.
+    notePreviewRequestFailure(error, 'meeting Turbo preview')
     return null
   } finally {
     previewLease.release()

package/server/routes/client-instance.ts

@@ -0,0 +1,35 @@
+// POST /api/client-instance/claim — the referee between copies of the COS Glasses app (6.50.3).
+// See lib/client-instance-claim.ts for the why. Behind the global API token like every /api
+// route except health and diag. Body: { id, bootAt, version, check? }. Answer: { verdict, owner }.
+//
+// `check: true` answers without taking: a copy that already yielded asks whether the owner
+// is still there, and must not steal the ring by asking (after a phone lock both copies'
+// timers were frozen, so the owner only LOOKS quiet; the older copy's first tick would
+// otherwise take it).
+
+import { Router } from 'express'
+import { arbitrateClientInstance, parseClientInstanceClaim, type ClientInstanceOwner } from '../lib/client-instance-claim.js'
+
+export function createClientInstanceRouter(now: () => number = Date.now): Router {
+  const router = Router()
+  let owner: ClientInstanceOwner | null = null
+  router.post('/client-instance/claim', (req, res) => {
+    const claim = parseClientInstanceClaim(req.body)
+    if (!claim) return void res.status(400).json({ error: 'invalid_claim' })
+    const previous = owner
+    const result = arbitrateClientInstance(owner, claim, now())
+    if (req.body?.check === true) {
+      const shown = previous ?? result.owner
+      return void res.json({ verdict: result.verdict, owner: { id: shown.id, bootAt: shown.bootAt, version: shown.version, seenAt: shown.seenAt }, check: true })
+    }
+    owner = result.owner
+    if (result.took && previous && previous.id !== claim.id) {
+      console.log(`[client-instance] ${new Date(now()).toISOString()} ${claim.id} (${claim.version}) took the ring from ${previous.id}`)
+    }
+    res.json({
+      verdict: result.verdict,
+      owner: { id: owner.id, bootAt: owner.bootAt, version: owner.version, seenAt: owner.seenAt },
+    })
+  })
+  return router
+}