@gotcos/glasses-server 6.36.24 → 6.36.26

package/CHANGELOG.md

@@ -1,3 +1,77 @@
+## 6.36.26
+
+Deadline hardening for the segmented TTS path shipped in 6.36.25. Three constants
+that were written rather than derived, and three tests that could not fail.
+
+- `SESSION_IDLE_MS` 60s -> 120s, DERIVED. Every segment's session is minted at
+  `/prepare`, but the client only touches segment i+1 when segment i starts
+  playing -- so the idle window has to outlast one full segment at the slowest
+  speed the client offers: `900 / 19 / 0.5 = 94.7s`. 60s covered 1x (47.4s) and
+  1.25x (37.9s) but not 0.75x (63.2s), which is a shipped option in the Settings
+  picker. At 0.75x every other segment would have 404'd, and because the client
+  resolves rather than rejects on error, playback would have continued and
+  dropped half the reply while still sounding complete.
+- `MAX_CHUNKS` 40 -> 46. 40 covered 35,350 characters against a 40,000-character
+  local cap -- 4,650 short, not "comfortably past" as its comment claimed. The
+  overflow landed in one oversized final segment, which the OpenAI backend then
+  trims PER SEGMENT, silently dropping text and contradicting the chunker's own
+  no-loss contract. (46 was the second answer; 45 was still 150 short.)
+- The timing test compared one chunk's render time against its own playback
+  time. Both sides are linear in length, so it reduced to `1.9 < 17.54` and
+  passed for every input -- including `LATER_CHUNK_CHARS = 100_000`, which
+  restores the original bug exactly. Replaced with the cumulative, serialized
+  form the sidecar actually exhibits, plus a test that scores the OLD
+  prefix/tail split as the failure it was.
+- Session-lifetime and policy tests now import `SESSION_IDLE_MS` and
+  `SESSION_MAX_LIFETIME_MS` instead of restating them as literals.
+- `SESSION_MAX_LIFETIME_MS` 30 -> 90 minutes (derived: a 40,000-char reply is
+  70.2 minutes at 0.5x).
+
+All four constants are mutation-verified: reverting each one fails a test.
+
+215 files, 3028 tests.
+
+## 6.36.25
+
+**Spoken replies are now N segments, not a prefix and a tail.**
+
+The two-segment split was a race, and the user lost it by less than a second.
+Measured on device with a 6,781-character reply:
+
+    prefix rendered in   0.5s
+    tail rendered in    12.4s  -- AFTER the prefix; the sidecar serializes
+                                  synthesis behind one lock
+    tail ready          ~12.9s after prepare
+    prefix audio         15s ... but 12.0s at the user's 1.25x playback speed
+
+The phone asked for the tail at 12.0s. It existed at 12.9s. `/play` blocks until
+synthesis finishes before sending any headers -- 11.3 seconds to first byte,
+measured -- and iOS's media loader will not wait. It buffered nothing and rejected
+with `NotSupportedError`.
+
+Widening the margin would not have fixed it: the margin depends on reply length,
+voice, playback speed and machine load. Chunking removes the race instead. The
+first segment stays small (250 chars, ~0.5s render, ~13s of speech) so first audio
+is as fast as before; later segments are 900 chars. Every segment renders roughly
+ten times faster than it plays, so the queue only gets further ahead — a property
+now asserted directly rather than assumed.
+
+`/prepare` returns `urls` (every segment, in order) plus `chunks`. `url` and
+`tailUrl` are kept, pointing at the first two, so a client older than 6.8.428
+plays a degraded two segments instead of nothing.
+
+`splitForFastPrefix` is deleted — zero callers, zero tests, and leaving a
+superseded splitter next to its replacement is how the wrong one gets used.
+
+The chunker's first test asserts that concatenating the segments reproduces the
+input's words. That caught a real defect in the first draft: the final piece was
+appended twice, so `splitForChunks('Hi.')` returned `['Hi. Hi.']`. A chunker that
+duplicates or drops text is worse than the bug it replaces, because the reply
+still sounds complete.
+
+Suite 3022 / 215, tsc 0. Mutation-verified: reverting to two segments, and
+dropping the legacy url/tailUrl, each fail the route contract test.
+
 ## 6.36.24
 
 **Playback stopped after about a minute, whatever the reply length.**

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@gotcos/glasses-server",
-  "version": "6.36.24",
+  "version": "6.36.26",
   "description": "COS Glasses — self-hosted AI heads-up-display server for Even G2 smart glasses, powered by Claude Code, Codex, or Cursor Agent CLI",
   "type": "module",
   "bin": {

package/server/lib/tts-cache.ts

@@ -101,17 +101,52 @@ const MAX_TOTAL_BYTES = 100 * 1024 * 1024 // 100 MB — in-memory cap
  * practical exposure window is unchanged". That was true, and it is also what
  * left the ceiling in place.
  */
-const SESSION_IDLE_MS = 60_000
+export const SESSION_IDLE_MS = (() => {
+  // DERIVED, like the ceiling below, and for the same reason: a window that does
+  // not cover its own inputs is the bug this replaced.
+  //
+  // Every segment's session is minted at /prepare. The client warms segment i+1
+  // at the START of segment i, so the gap between that touch and the real request
+  // is ONE FULL SEGMENT of playback. The window has to outlast that gap at the
+  // slowest speed the client offers:
+  //
+  //   LATER_CHUNK_CHARS  900 chars
+  //   speech rate        ~19 chars/sec (measured)
+  //   MIN_SPEED          0.5x (voice-output.ts clamps here)
+  //   => 900 / 19 / 0.5 = 94.7s
+  //
+  // 60s covered 1x (47.4s) and 1.25x (37.9s) but NOT 0.75x (63.2s) -- a shipped
+  // option in the Settings picker. At 0.75x every other segment would 404, and
+  // because onError resolves rather than rejects, playback would skip on and
+  // sound complete while dropping half the reply. 120s covers 0.5x with margin.
+  return 120_000
+})()
 
 /**
  * Absolute ceiling, never refreshed.
  *
  * The session UUID IS the auth for an unauthenticated play route, so a purely
- * sliding window could be kept alive indefinitely by polling. Thirty minutes is
- * far longer than any plausible single reply (211 seconds for 4,000 characters)
- * and still bounds the exposure of a leaked URL.
+ * sliding window could be kept alive indefinitely by polling. This bounds the
+ * exposure of a leaked URL.
+ *
+ * DERIVED, not picked. Every segment of a reply is minted at prepare time, so
+ * the ceiling has to outlast the WHOLE reply played at the SLOWEST speed the
+ * client offers -- otherwise the last segments expire before playback reaches
+ * them, which is the same class of bug as the 60s deadline this replaced:
+ *
+ *   MAX_LOCAL_TTS_CHARS   40,000 chars
+ *   speech rate           ~19 chars per second (measured)
+ *   MIN_SPEED             0.5x  (voice-output.ts clamps here)
+ *   => 40000 / 19 / 0.5  = 70.2 minutes of audio
+ *
+ * 30 minutes was shorter than both that and the 1x case (35.1 min), so a maximal
+ * reply would have cut off near the end. 90 minutes covers the worst case with
+ * margin and is still a bounded window.
+ *
+ * If MAX_LOCAL_TTS_CHARS or MIN_SPEED changes, re-derive this. A ceiling that
+ * silently stops covering its own inputs is exactly what went wrong before.
  */
-const SESSION_MAX_LIFETIME_MS = 30 * 60_000
+export const SESSION_MAX_LIFETIME_MS = 90 * 60_000
 
 /** Disk cache configuration (env-overridable). Defaults sized for "I run this
  *  on my laptop and forget about it for months" rather than a service tier.

package/server/routes/tts.ts

@@ -133,101 +133,113 @@ function stripMarkdownLight(text: string): string {
     .replace(/^[-*+]\s/gm, '- ')
 }
 
-// ── v5.9.6 fast-prefix splitter ───────────────────────────────────────────
-//
-// The "fast first-audio" path (POST /api/tts/prepare with fast: true) wants
-// to start playing audio in ~1-2s instead of the 8-15s a full-message OpenAI
-// render takes for long replies. We do that by splitting the input into a
-// short prefix the client can play immediately, and a tail that gets
-// generated in parallel and chained on prefix `ended`.
-//
-// Heuristic-only — no NLP dependency. Markdown is already stripped above.
-// Boundary detection uses the same .!? + whitespace rule as trimToCap so the
-// two stay consistent. Bounded lengths protect against pathological inputs:
-//   - MIN_PREFIX_CHARS: short greetings ("Hi.") get padded with the next
-//     sentence so the prefix is long enough to mask tail-render latency.
-//   - MAX_PREFIX_CHARS: a single long sentence ("So basically I think we…
-//     spanning 600 chars") gets cut at a word boundary instead of running on.
-const MIN_PREFIX_CHARS = 60
-const MAX_PREFIX_CHARS = 250
-
-/** Split `text` into a fast-playable prefix + a tail.
+
+/**
+ * Chunk sizes, in characters.
  *
- *  Contract:
- *  - Returns `{ prefix, tail }` with `prefix` non-empty and `tail` either ''
- *    (the message fits in one chunk and the route should fall back to v5.9.5
- *    single-URL behavior) or the remainder.
- *  - Prefix targets the first ~2 sentences but expands if either is short
- *    (to clear MIN_PREFIX_CHARS) and contracts if a single sentence exceeds
- *    MAX_PREFIX_CHARS (cut at the last word boundary inside the cap).
- *  - Caller is responsible for trimToCap'ping the input first. */
-export function splitForFastPrefix(text: string): { prefix: string; tail: string } {
-  const trimmed = text.trim()
-  if (trimmed.length === 0) return { prefix: '', tail: '' }
-  // Short enough to play as a single chunk — no benefit from splitting.
-  if (trimmed.length <= MIN_PREFIX_CHARS) return { prefix: trimmed, tail: '' }
-
-  // Walk sentence terminators forward, accumulating sentences until we cover
-  // at least MIN_PREFIX_CHARS. Up to 2 sentences if both are reasonably sized,
-  // more if the first ones are tiny. Indices point to the boundary AFTER the
-  // terminator + whitespace (the start of the next sentence).
-  const sentenceBoundaries: number[] = []
-  const re = /[.!?]\s+/g
-  let m: RegExpExecArray | null
-  while ((m = re.exec(trimmed)) !== null) {
-    sentenceBoundaries.push(m.index + m[0].length)
-  }
+ * MEASURED on this machine: Kokoro renders ~1.9ms per character, and speech runs
+ * ~19 characters per second of audio. Those two numbers are what make chunking a
+ * CORRECTNESS fix rather than a latency tweak.
+ *
+ * The two-segment prefix/tail split failed like this, measured on device
+ * 2026-08-23 with a 6,781-character reply:
+ *
+ *   prefix generated in  0.5s   (250 chars)
+ *   tail generated in   12.4s   (6,531 chars) -- AFTER the prefix, because the
+ *                               sidecar serializes synthesis behind one lock
+ *   tail therefore ready ~12.9s after prepare
+ *   prefix audio         15s ... but 12.0s at the user's 1.25x playback speed
+ *
+ * The phone asked for the tail at 12.0s. It existed at 12.9s. `/play` blocks
+ * until synthesis finishes before sending any headers -- 11.3s to first byte,
+ * measured -- and iOS's media loader will not wait: it buffers nothing and
+ * rejects with NotSupportedError.
+ *
+ * A margin that depends on reply length, voice, playback speed and machine load
+ * is not a margin. Small chunks remove the race entirely: by the time chunk 1
+ * finishes playing, every later chunk is long since rendered.
+ *
+ * FIRST_CHUNK stays small so first audio is still fast (~0.5s render, ~13s of
+ * speech). Later chunks are larger to keep the request count down; at ~1.7s
+ * render each they are always many seconds ahead of playback.
+ */
+export const FIRST_CHUNK_CHARS = 250
+export const LATER_CHUNK_CHARS = 900
+
+/**
+ * Hard ceiling on segments for one reply.
+ *
+ * DERIVED, and the first version was wrong: it claimed 40 chunks covered
+ * "~36,000 characters, comfortably past MAX_LOCAL_TTS_CHARS". The real coverage
+ * is 250 + 39x900 = 35,350 -- 4,650 SHORT of the 40,000 cap, not past it. A
+ * maximal reply therefore produced a 5,574-char final segment, and on the OpenAI
+ * backend `trimToCap` is applied PER SEGMENT, so 1,574 characters were silently
+ * dropped -- contradicting this chunker's own "nothing is dropped" contract two
+ * definitions above.
+ *
+ *   ceil((MAX_LOCAL_TTS_CHARS - FIRST_CHUNK_CHARS) / LATER_CHUNK_CHARS) + 1
+ *   = ceil(39750 / 900) + 1 = 45 + 1 = 46      (250 + 45x900 = 40,750)
+ *
+ * 45 was the first answer here and was itself 150 chars short -- which is why
+ * the test below computes the coverage instead of trusting this arithmetic.
+ *
+ * Re-derive if either cap moves; the test below fails if this stops covering.
+ */
+export const MAX_CHUNKS = 46
 
-  if (sentenceBoundaries.length === 0) {
-    // No sentence terminators (one giant run-on). Fall back to a word-boundary
-    // cut at MAX_PREFIX_CHARS. If the whole thing fits in MAX, it's a single chunk.
-    if (trimmed.length <= MAX_PREFIX_CHARS) return { prefix: trimmed, tail: '' }
-    const slice = trimmed.slice(0, MAX_PREFIX_CHARS)
-    const lastSpace = slice.lastIndexOf(' ')
-    const cut = lastSpace > MIN_PREFIX_CHARS ? lastSpace : MAX_PREFIX_CHARS
-    return {
-      prefix: trimmed.slice(0, cut).trim(),
-      tail: trimmed.slice(cut).trim(),
+/**
+ * Split `text` into sequentially-playable chunks.
+ *
+ * Contract:
+ *  - Always returns at least one non-empty chunk for non-empty input.
+ *  - Concatenating the chunks reproduces the input's words in order. Nothing is
+ *    dropped -- a chunker that loses text is worse than the bug it replaces.
+ *  - Breaks at sentence terminators where possible, at word boundaries
+ *    otherwise, and mid-word only for a single unbroken run longer than a chunk.
+ *  - The first chunk is small (fast first audio); the rest are larger.
+ */
+export function splitForChunks(text: string): string[] {
+  const trimmed = text.trim()
+  if (trimmed.length === 0) return []
+
+  const chunks: string[] = []
+  let rest = trimmed
+  while (rest.length > 0 && chunks.length < MAX_CHUNKS) {
+    const cap = chunks.length === 0 ? FIRST_CHUNK_CHARS : LATER_CHUNK_CHARS
+    // `rest` MUST be cleared before breaking. Leaving it set made the overflow
+    // guard below append the final piece a second time -- `splitForChunks('Hi.')`
+    // returned ['Hi. Hi.']. Caught by the never-lose-a-word test, which turned
+    // out to catch duplication just as well as loss.
+    if (rest.length <= cap) { chunks.push(rest); rest = ''; break }
+
+    const window = rest.slice(0, cap)
+    // Prefer a sentence end. NOTE: this is `/[.!?]\s+/` while `trimToCap` matches
+    // a literal '. ' -- they disagree on '.\n' and on double spaces. Not unified
+    // here because trimToCap only ever trims a hard cap, but do not describe them
+    // as the same rule. (An earlier comment claimed all three agreed, naming a
+    // prefix splitter deleted in the same commit.)
+    let cut = -1
+    const re = /[.!?]\s+/g
+    let m: RegExpExecArray | null
+    while ((m = re.exec(window)) !== null) cut = m.index + m[0].length
+    // Only accept a sentence break that is not absurdly early, or a 3-word
+    // chunk followed by a 900-char one reads as a stutter.
+    if (cut < cap * 0.4) {
+      const space = window.lastIndexOf(' ')
+      cut = space > 0 ? space + 1 : cap
     }
+    chunks.push(rest.slice(0, cut).trim())
+    rest = rest.slice(cut).trim()
   }
-
-  // Pick the smallest cut that satisfies (length >= MIN_PREFIX_CHARS) AND
-  // covers >= 2 sentences when possible. Stop early once a candidate also
-  // exceeds MAX_PREFIX_CHARS — the previous candidate is the best fit.
-  let chosenCut = sentenceBoundaries[sentenceBoundaries.length - 1]
-  for (let i = 0; i < sentenceBoundaries.length; i++) {
-    const cut = sentenceBoundaries[i]
-    const sentencesCovered = i + 1
-    const longEnough = cut >= MIN_PREFIX_CHARS
-    const tooLong = cut > MAX_PREFIX_CHARS
-    const hasTwo = sentencesCovered >= 2
-    if (tooLong) {
-      // Previous boundary (if any) was the best fit; if this is the first
-      // boundary AND it already overshoots MAX, fall back to a word-boundary
-      // cut inside the first sentence so the prefix doesn't blow past the cap.
-      if (i === 0) {
-        const slice = trimmed.slice(0, MAX_PREFIX_CHARS)
-        const lastSpace = slice.lastIndexOf(' ')
-        const cutAt = lastSpace > MIN_PREFIX_CHARS ? lastSpace : MAX_PREFIX_CHARS
-        chosenCut = cutAt
-      } else {
-        chosenCut = sentenceBoundaries[i - 1]
-      }
-      break
-    }
-    if (longEnough && hasTwo) {
-      chosenCut = cut
-      break
-    }
-    chosenCut = cut
+  // MAX_CHUNKS reached with text left: append it to the last chunk rather than
+  // dropping it. A long final chunk is a latency problem; dropped text is a lie.
+  if (rest.length > 0 && chunks.length > 0) {
+    chunks[chunks.length - 1] = `${chunks[chunks.length - 1]} ${rest}`.trim()
   }
-
-  const prefix = trimmed.slice(0, chosenCut).trim()
-  const tail = trimmed.slice(chosenCut).trim()
-  if (tail.length === 0) return { prefix: trimmed, tail: '' }
-  return { prefix, tail }
+  return chunks.filter((c) => c.length > 0)
 }
 
+
 function openaiBudgetOk(): boolean {
   try {
     assertOpenAITtsBudget()
@@ -936,15 +948,30 @@ ttsRouter.post('/tts/prepare', async (req, res) => {
       return res.json({ url: `/api/tts/play/${uuid}`, ...engineMeta })
     }
 
-    const { prefix, tail } = splitForFastPrefix(capped)
-    const prefixMint = mintAndWarm(prefix)
-    if (tail.length === 0) {
-      return res.json({ url: `/api/tts/play/${prefixMint.uuid}`, ...engineMeta })
+    // N SEGMENTS, not two. See splitForChunks for why the prefix/tail pair was a
+    // race the user lost by less than a second.
+    //
+    // Warm order matters and is already correct: the sidecar serializes synthesis
+    // behind one lock, so minting in order means chunk 1 renders first and every
+    // later chunk finishes long before playback reaches it.
+    const chunks = splitForChunks(capped)
+    if (chunks.length === 0) {
+      return res.status(400).json({ error: 'text is required (non-empty string)' })
+    }
+    const mints = chunks.map((chunk) => mintAndWarm(chunk))
+    const urls = mints.map((m) => `/api/tts/play/${m.uuid}`)
+
+    if (urls.length === 1) {
+      return res.json({ url: urls[0], urls, chunks: 1, ...engineMeta })
     }
-    const tailMint = mintAndWarm(tail)
     res.json({
-      url: `/api/tts/play/${prefixMint.uuid}`,
-      tailUrl: `/api/tts/play/${tailMint.uuid}`,
+      // `urls` is the real contract. `url` and `tailUrl` are kept so a client
+      // older than 6.8.428 still plays the first two segments instead of
+      // nothing -- degraded, but not broken, which is the point of keeping them.
+      url: urls[0],
+      tailUrl: urls[1],
+      urls,
+      chunks: urls.length,
       ...engineMeta,
     })
   } catch (err) {