lobes-cli 0.90.2__tar.gz → 0.91.0__tar.gz

{lobes_cli-0.90.2 → lobes_cli-0.91.0}/CHANGELOG.md

@@ -4,6 +4,34 @@ All notable changes to this project are documented here. The format is based on
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project
 adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.91.0] - 2026-10-23
+
+### Added
+
+- Hebrew Whisper STT (audio-he overlay): decode-quality knobs for long recordings (issue #329) - STT_NO_REPEAT_NGRAM_SIZE (default 6), STT_REPETITION_PENALTY (default 1.0 = off), STT_COMPRESSION_RATIO_MAX (default 2.4), STT_TEMPERATURE_FALLBACK (default 0.2,0.4), STT_LOGPROB_STAT (default median; mean = legacy), STT_MAX_WORD_RUN (default 4; 0/off disables), STT_VAD (default auto) with STT_VAD_THRESHOLD / STT_VAD_MIN_SPEECH_MS; each has an off value that restores the previous decode exactly.
+- Word-run knobs (#329 follow-up): STT_MAX_WORD_RUN (default 4; 0/off disables) TRIMS a run of one word, or one two-word phrase (Unicode word tokens, punctuation ignored), to 3 repeats and keeps the rest of the chunk, so emphatic speech is never dropped; STT_WORD_RUN_FAIL (default 8; 0/off disables) fails a multi-window chunk with a run that long like a compression-ratio failure - retried through the temperature ladder, then dropped with reason=word_run. stt_chunk lines carry word_run=, max_word_run= and word_run_fail=, and a trim logs event=trimmed.
+- Structured stt_chunk (retry / dropped / trimmed / skipped) and per-request stt_request log lines in the stt container, so dropped and retried chunks are observable server-side; stt_request names the request policy (policy=single_window|multi_window stat=mean|median); the client-facing dropped[] field waits for verbose_json (#328).
+
+### Changed
+
+- STT_LOGPROB_STAT now defaults to median (threshold STT_MIN_AVG_LOGPROB=-0.35 unchanged) for multi-window uploads: on a live A/B on the DGX Spark the mean gate dropped 118 of 215 decoded chunks and the median gate 39 of 215.
+- Single-window requests (every realtime turn) keep the pre-#329 confidence gate exactly: the mean statistic, one greedy decode, no temperature fallback, a word run trimmed but never failed. A short silence hallucination (mean -0.58, median -0.15) would pass a median gate.
+- The stt handler runs the whole transcription (resample, Silero scan, every Whisper decode) off the event loop in a worker thread, one transcription at a time behind one lock, so /v1/health/ready stays responsive during a long upload; the Silero wrapper's lock now covers a whole chunk scan (reset plus every window).
+- DecodeSettings, DecodePolicy (both frozen), ChunkOutcome and RequestSummary are dataclasses; STT_MIN_AVG_LOGPROB accepts the same off tokens as the other knobs (false and no included).
+- Dockerfile.whisper-stt installs silero-vad==6.2.1 with --no-deps (its torchaudio dependency would replace the base image torch); the server loads the bundled TorchScript model by path and falls back to decoding every chunk if it cannot.
+
+### Fixed
+
+- Hebrew batch STT no longer passes a phrase looped to the token cap (n-gram block plus a per-character zlib compression-ratio check), and re-decodes a chunk failing either gate at sampling temperatures before dropping it (#329). The live A/B measured the median gate with a fail-at-4 word-run gate; the single-window policy and the trim-versus-fail word-run design have not run live.
+- The last chunk of a multi-window upload is never VAD-skipped (a short closing word was lost), and a chunk too short for one VAD window counts as speech.
+- A retry that comes back empty after an earlier attempt failed a gate is a drop (reason=empty_after_retry), logged and counted as one, never as a recovery.
+
+## [0.90.3] - 2026-10-18
+
+### Fixed
+
+- Mesh: plain `embedder` / `reranker` on a box that does not host them (the DGX Spark) answered 404 even though several members serve identical lanes, because their fingerprints advertised `quantization: unknown` and the strict pool rule never pools an unknown (#297). Every card profile now declares `quantization = "none"` for both pooling gears (the lanes pass no `--quantization`), rendered as `EMBED_QUANTIZATION=none` / `RERANK_QUANTIZATION=none` and added to `env.example`; `lobes doctor --fix` appends them to existing deployments. The strict rule itself is unchanged.
+
 ## [0.90.2] - 2026-10-18
 
 ### Changed

{lobes_cli-0.90.2 → lobes_cli-0.91.0}/CLAUDE.md

@@ -977,7 +977,9 @@ rollback, restore the backup —, `...-accept-orin-embed.txt`). **UNVALIDATED:**
 everything beyond code retrieval — prose/docs/math, real image/video/audio
 retrieval (the probes were synthetic), MRL quality, cross-model agreement —
 is issue **#296**; the plain-`embedder`-on-the-Spark pooling finding is issue
-**#297** (a pre-existing strict-fingerprint rule, not caused by this shape). No
+**#297** (a pre-existing strict-fingerprint rule, not caused by this shape;
+fixed 2026-10-18 by declaring `quantization = "none"` for the embedder and
+reranker in every card profile, so their fingerprints carry a real value). No
 consumer is switched implicitly: `EIDETIC_EMBED_MODEL` / `COHERENCE_EMBED_MODEL`
 opt in by lane id and switching means re-embedding. See
 `docs/orin-embed-deployment.md`, `docs/embeddinggemma-2.md` and `lobes explain

{lobes_cli-0.90.2 → lobes_cli-0.91.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: lobes-cli
-Version: 0.90.2
+Version: 0.91.0
 Summary: lobes — run, assess, and switch the local vLLM model.
 Project-URL: Homepage, https://github.com/agentculture/lobes-cli
 Project-URL: Issues, https://github.com/agentculture/lobes-cli/issues

lobes_cli-0.91.0/docs/hebrew-realtime.md

@@ -0,0 +1,325 @@
+# Hebrew realtime — a duplex Hebrew voice session with tool calling
+
+An **opt-in overlay** on top of the realtime audio overlay
+([`realtime-pipeline.md`](realtime-pipeline.md)): the same `GET /v1/realtime`
+WebSocket session, serving Hebrew. A person speaks Hebrew, the box answers in
+Hebrew, and the model can call tools that the **client** owns. The English
+overlay's files are byte-identical with or without it.
+
+If you are building a client, read the contract instead:
+[`contracts/realtime-tool-calling.md`](contracts/realtime-tool-calling.md).
+
+> **Status (the #108 rule).** Everything here was built and measured on ONE box
+> — the DGX Spark GB10, 2026-09-18 — from a hand-carried deployment of this
+> branch's wheel. The measurements are real and cited; the *packaged* path
+> (`lobes init --audio --audio-lang he` from a released wheel) is
+> **DECLARED / UNVALIDATED** until an acceptance transcript from that path
+> lands under `docs/evidence/`. No Jetson has run any of it (issue #276).
+
+## The stack
+
+| stage | engine | notes |
+|---|---|---|
+| VAD / turn-taking | Silero, `server_vad` | PulseVAD was tried and false-triggered on all four non-speech clips |
+| STT | `ivrit-ai/whisper-large-v3-turbo`, transformers fp16 (`Dockerfile.whisper-stt`, `listen_server_whisper.py`) | ~170–200 ms per turn |
+| generate | the gateway's own `/v1/chat/completions`, `OPENAI_MODEL=multimodal` | on the Spark: `nvidia/Gemma-4-26B-A4B-NVFP4`, `gemma4` tool + reasoning parsers, 32–44 tok/s; native 262144 window since 2026-09-19 (was 32768; `docs/evidence/2026-09-19-accept-multimodal-gemma26b-native-context-spark.txt`) |
+| TTS (shipped default) | Chatterbox **Multilingual**, `language_id="he"`, phonikud niqqud (`Dockerfile.chatterbox-ml`) | seconds per sentence; needs niqqud; samples non-deterministically, hence a runaway guard |
+| TTS (measured best) | **BlueTTS**, CPU ONNX (`Dockerfile.bluetts`, `lobes.realtime.bluetts_server`) | 93–305 ms per sentence, its own G2P, no niqqud wanted — see *BlueTTS* below for why it is not the default |
+
+Evidence: `docs/evidence/2026-09-hebrew-*.txt` (seven files) and
+`docs/evidence/2026-09-hebrew-realtime-streaming-bluetts-spark.txt`.
+
+### Three things that are easy to get wrong
+
+- **Whisper hallucinates on non-speech** — this fine-tune emits plain text such
+  as `תודה רבה` for a door slam, and its `<|nospeech|>` probability reads 0.000.
+  The mean token log-probability separates cleanly (speech ≥ −0.12,
+  hallucination ≤ −0.58), so the sidecar drops a transcript below
+  `STT_MIN_AVG_LOGPROB` (default −0.35; `off` disables). 4/4 non-speech clips
+  dropped, 22/22 speech clips kept.
+- **Chatterbox Multilingual without niqqud is unusable for Hebrew** (92 % → ~9 %
+  round-trip WER with phonikud), and a Latin-script word derails it.
+- **An NVFP4A16 export of Gemma 4 12B damaged Hebrew** where bf16 and Google's
+  QAT checkpoint did not. Check Hebrew output before promoting a quantization.
+
+### Batch transcription: formats and long clips
+
+`POST /v1/audio/transcriptions` takes any format the image's ffmpeg decodes —
+m4a/aac, mp3, ogg/opus, webm, flac, and WAV at any bit depth — not only 16-bit
+PCM WAV. A 16-bit RIFF/WAVE upload (the realtime bridge's own turn audio) is
+still read directly and never touches ffmpeg; anything else is transcoded to
+16 kHz mono PCM16 first, and an undecodable upload is a 400 `invalid_audio`.
+
+An upload longer than Whisper's 30 s window is **chunked**, not refused: each
+cut goes at the quietest 20 ms frame in the last 5 s of its window, every
+window is judged on its own (silence skip, gates and retries, below), and the
+non-empty pieces are joined with a space. `STT_MAX_AUDIO_SECONDS` (default 600) caps the
+total; past it the reply is a 413 `clip_too_long`. The upload byte cap
+(`STT_MAX_UPLOAD_BYTES`, 16 MiB) still applies first, so ten minutes of
+uncompressed 16 kHz WAV (~19 MB) needs it raised; a compressed voice note does
+not. MEASURED on the DGX Spark, 2026-10-09, with synthetic BlueTTS speech: an
+80 s m4a was transcribed in 3 windows in 3.9 s with every sentence intact at
+the seams; mp3, ogg/opus, webm, 24-bit WAV and m4a single sentences each came
+back verbatim. A real microphone recording is still UNVALIDATED (#108).
+
+### Decode quality: loops, retries and silence (issue #329)
+
+On a real 56-minute recording, the decode above failed in two ways. A two-word
+phrase looped until the token cap, and a loop is high-confidence, so it passed
+the gate. The gate also dropped 61 of 125 chunks whole with no second try, and
+some of them were real speech. Each chunk now goes through the steps below.
+Each step has a knob whose off value gives back the old decode exactly (for
+`STT_LOGPROB_STAT`, the legacy value `mean`).
+
+**Two request policies.** A request of **one** window (30 s or less), which
+covers every realtime turn, and a **multi-window** upload are judged
+differently. The `stt_request` log line names the policy a request used
+(`policy=single_window stat=mean` or `policy=multi_window stat=median`).
+
+| | single window (realtime) | multi-window upload |
+|---|---|---|
+| confidence statistic | `mean`, always (the pre-#329 gate) | `STT_LOGPROB_STAT` (default `median`) |
+| decodes per chunk | one greedy decode, never retried | greedy, then `STT_TEMPERATURE_FALLBACK` |
+| word run of 4–7 | trimmed to 3 | trimmed to 3 |
+| word run of 8 or more | trimmed to 3 | fails the chunk: retry, then drop |
+| Silero VAD | never consulted | every chunk except the last |
+
+The −0.35 threshold was measured on the **mean**. A short silence
+hallucination can fail the mean while passing a median gate: in live data,
+4 tokens with log-probabilities −1.9, −0.1, −0.1, −0.2 have a mean of −0.58
+and a median of −0.15. Among short (6 tokens or fewer) first attempts that
+failed the mean gate, 7 of 24 would have passed a median gate. A long upload
+gains more from the median than it loses, but a realtime turn is usually
+short, so it keeps the mean. It also keeps one greedy decode: re-sampling a
+silent turn's hallucination at a higher temperature until one sample passes
+would add up to two decodes of latency for nothing.
+
+1. **Silence skipping (`STT_VAD`).** On an upload longer than one window,
+   Silero VAD reads each chunk first. Three kinds of chunk are never skipped:
+   a single-window request (the bridge's own VAD already opened that turn, and
+   a one-word reply may hold very little speech), the **last** chunk of an
+   upload (`plan_chunks` leaves it 0–5 s long, so a closing word may hold less
+   than `STT_VAD_MIN_SPEECH_MS` of speech), and a chunk shorter than one 32 ms
+   VAD window (there is nothing to judge, so it counts as speech). If Silero
+   finds less than `STT_VAD_MIN_SPEECH_MS` of speech, Whisper never decodes
+   the chunk, so a silent chunk cannot turn into `תודה רבה`. The scan stops as
+   soon as it has found enough speech, so a speech chunk costs only a few VAD
+   calls. The image installs the `silero-vad` wheel `--no-deps`, because its
+   torchaudio dependency would replace the base image's torch. The server loads
+   the bundled TorchScript model by path. If Silero fails to load, or fails on
+   a call, the server logs a warning and decodes the chunk as before.
+2. **Loop prevention at decode time.** `STT_NO_REPEAT_NGRAM_SIZE` sets the
+   transformers n-gram block. `STT_REPETITION_PENALTY` is off by default,
+   because it penalises every recurring token, Hebrew prefix letters included.
+   Beam search was considered and left out. It would cost about 4× the GPU
+   time on a shared box, and the per-token log-probabilities the gate reads
+   would need different bookkeeping.
+3. **Gates per attempt.** The first is the confidence gate,
+   `STT_MIN_AVG_LOGPROB` (−0.35) on the request policy's statistic (above).
+   The second is OpenAI Whisper's zlib compression ratio,
+   `STT_COMPRESSION_RATIO_MAX`, default 2.4, on both policies. The ratio is
+   taken one byte per character, not over UTF-8. Every Hebrew letter shares
+   one UTF-8 lead byte, which alone pushes ordinary Hebrew prose to about 2.3.
+   Counted per character, the same prose scores about 1.7, like English. Those
+   figures come from synthetic text, not from a recording. The third is the
+   **word-run failure check**, `STT_WORD_RUN_FAIL`, default 8, on multi-window
+   uploads only (next step).
+4. **Word runs: trim, or fail only when far beyond speech.** A word run is one
+   word, or one two-word phrase, repeated back to back. Words are Unicode
+   word tokens compared case-folded and without niqqud, so punctuation between
+   the repeats does not break the run. The n-gram block misses such a run (the
+   punctuation tokens make every n-gram different), and so does the
+   compression ratio (one short run inside a full chunk of speech). Emphatic
+   speech repeats words too ("no no no no, I disagree", or a realtime "yes yes
+   yes yes"), and failing the whole chunk for it would drop 30 s of correct
+   text. So a run of `STT_MAX_WORD_RUN` (4) or more is **trimmed** to
+   `STT_MAX_WORD_RUN − 1` repeats, and the rest of the chunk is kept. The cut
+   goes from the last kept repeat to the end of the run, so the punctuation
+   and words after it follow on. Only a run of `STT_WORD_RUN_FAIL` (8) or
+   more, far beyond normal speech, fails a multi-window chunk with
+   `reason=word_run`. A single window only trims. The trade-off: the live A/B
+   below found hallucinated one-word runs of 11 and 5, both absent from a
+   reference transcript. With these defaults the 11 is re-decoded or dropped,
+   but the 5 is trimmed to 3, so three words of that hallucination stay in the
+   text.
+5. **Retry before dropping (`STT_TEMPERATURE_FALLBACK`, multi-window only).**
+   A chunk that fails any gate is decoded again at each sampling temperature
+   in turn. Sampling uses the full distribution, and the gate undoes the
+   temperature, so it reads the same quantity every time. The first result
+   that passes every gate is kept. If none passes, the chunk is dropped, as
+   before, with the failing gates as its `reason` (`low_logprob`,
+   `compression_ratio`, `word_run`). An empty or tags-only transcript on the
+   first attempt counts as silence and is never retried. An empty retry after
+   a failed attempt is a **drop** (`reason=empty_after_retry`), not a
+   recovery: the first attempt heard something, and it is gone.
+
+| knob | default | off | what it does |
+|---|---|---|---|
+| `STT_LOGPROB_STAT` | `median` | `mean` | Which statistic the confidence gate reads on a multi-window upload. `median` ignores a few rare tokens, such as a name, in an otherwise confident chunk. A single window always reads `mean`. The log lines carry both. |
+| `STT_NO_REPEAT_NGRAM_SIZE` | `6` | `0` | Blocks any repeated token n-gram within a chunk. |
+| `STT_REPETITION_PENALTY` | `1.0` | `1.0` | transformers' repetition penalty. `1.1` is the issue's suggestion. |
+| `STT_COMPRESSION_RATIO_MAX` | `2.4` | `off` | Treats a chunk whose text compresses better than this as a loop. |
+| `STT_MAX_WORD_RUN` | `4` | `0` / `off` | Trims a run of one word or two-word phrase this long to one fewer repeats. Values below 2 fall back to the default. |
+| `STT_WORD_RUN_FAIL` | `8` | `0` / `off` | Fails a multi-window chunk with a run this long (retry, then drop). Values below 2 fall back to the default. |
+| `STT_TEMPERATURE_FALLBACK` | `0.2,0.4` | `off` | Re-decode temperatures for a failing multi-window chunk. Each one is another Whisper decode. |
+| `STT_VAD` | `auto` | `off` | Skips non-speech chunks of a multi-window upload with Silero before decoding. |
+| `STT_VAD_THRESHOLD` | `0.5` | — | Silero speech probability per 32 ms window. |
+| `STT_VAD_MIN_SPEECH_MS` | `250` | — | How much speech makes a chunk worth decoding. |
+
+**Off the event loop.** The handler runs the whole transcription (resample,
+VAD scan and every Whisper decode) in a worker thread, as it already did the
+ffmpeg transcode, so `/v1/health/ready` answers while a long upload decodes.
+One transcription runs at a time, behind one lock, so concurrent requests
+never share the GPU or Silero's recurrent state. A realtime turn that arrives
+during a long upload still waits for that upload to finish.
+
+**Observability.** The stt container logs one line for each retried, dropped,
+trimmed or skipped chunk:
+
+```text
+stt_chunk event=retry chunk=3/18 attempt=0 temperature=0.0 reason=compression_ratio avg_logprob=-0.04 median_logprob=-0.01 compression_ratio=9.80 word_run=30 policy=multi_window stat=median ... tokens=440
+stt_chunk event=dropped chunk=7/18 attempt=2 temperature=0.4 reason=low_logprob ... text='...'
+stt_chunk event=dropped chunk=8/18 attempt=1 temperature=0.2 reason=empty_after_retry ... previous_reason=compression_ratio
+stt_chunk event=trimmed chunk=5/18 word_run=5 runs=1 max_word_run=4
+stt_chunk event=skipped chunk=9/18 reason=no_speech
+```
+
+It also logs one summary line per request:
+`stt_request policy= stat= chunks= decoded= vad_skipped= retried= recovered= dropped= trimmed= decodes= seconds=`.
+Clients cannot see any of this yet. A `dropped[]` field and per-segment
+`avg_logprob` / `compression_ratio` wait for `response_format=verbose_json`
+(#328).
+
+**Live A/B, DGX Spark, 2026-10-23 — why the default is now the median.** The
+first build of this section ran on the live Hebrew `stt` lane against two
+private recordings, cut into 16 requests of 480 s (269 chunks). Counts only:
+
+- **Loops are fixed.** The longest run of one phrase fell from 220 (pre-#329)
+  to 3 at the defaults of the time.
+- **The mean gate was the binding constraint.** It still dropped 118 of 215
+  decoded chunks (55 %); retries recovered 7. Every drop was `low_logprob`.
+- **The median gate dropped 39 of 215 (18 %)**, and every window then carried
+  at least as much text as a reference transcript of the same span (a rough
+  speech reference, not ground truth). That is why `median` became the
+  multi-window default.
+- **It let through a confident hallucination.** In two windows a short word
+  was repeated 11 and 5 times in a row, with punctuation between, and a
+  two-word run of 5 sat in the same span. The word was absent from the
+  reference transcript. That is what the word-run knobs were added for.
+- Silero loaded in the built image and skipped 54 of the 269 chunks.
+
+A second run measured the median default with a word-run gate that **failed**
+the chunk at 4 repeats (an earlier design of step 4): still 39 of 215 dropped
+(18 %), the longest run of one phrase 3, three chunks hitting the word-run
+gate. 15 of the 16 requests in both runs were multi-window. The 16th, the
+20 s tail of one recording, was a single window: it returned 6 characters
+in the second run, where the pre-#329 build returned none. That one request
+is the only single-window evidence, and it predates the single-window
+policy.
+
+**Status (#108).** The single-window/multi-window split, the trim-versus-fail
+word-run design, the never-skipped last chunk, `empty_after_retry` and the
+off-loop, serialized transcription are unit-tested with a fake decoder and a
+fake VAD only; none of them has run live. 18 % still misses the issue's 10 %
+drop target. To validate, re-run the recordings on the live lane, plus a set
+of short realtime turns, and read the `stt_request` and `stt_chunk` lines.
+Acceptance means no phrase repeated more than 3 times in a row, fewer than
+10 % of non-silent chunks dropped, and still no text from silence.
+
+## Latency: what was built, and what it bought
+
+Measured live, a human on a reSpeaker XVF3800, from the moment the server
+decides the turn ended to the first audio byte:
+
+| configuration | first audio |
+|---|---|
+| whole-reply generate + Chatterbox Multilingual | 2.6 s (one sentence), 7.3 s (longer) |
+| + sentence streaming (`GENERATE_STREAM`, default on) + BlueTTS | 0.67–1.0 s |
+| + hidden speculation (`VAD_EAGER_MS=250`) | **1–113 ms** |
+
+…plus, always, the silence the VAD waits through before it believes the turn
+ended (`VAD_SILENCE_MS`). Once speculation is on, that wait is essentially the
+whole perceived delay:
+
+```text
+perceived delay ≈ max( VAD_SILENCE_MS , VAD_EAGER_MS + pipeline )
+pipeline ≈ 600 ms  =  STT ~170  +  first clause ~300  +  TTS ~120
+```
+
+| knob | default | what it does |
+|---|---|---|
+| `GENERATE_STREAM` | `true` | Stream the generate call; speak sentence by sentence. `false` is the exact previous path. **On for every language, English included.** |
+| `REPLY_FIRST_CLAUSE_MIN_CHARS` | `24` | How long the opening clause must be before a comma may end the first spoken piece. `5` on the Spark: first audio ~1.1 s → ~0.6 s, at the price of a short first piece. |
+| `VAD_MIN_LEVEL_PCT` | `0` (off) | **Ignore quiet voices** (background talk, a TV, the next room). A chunk counts as speech only while the PEAK input level — held ~250 ms, like a level meter — is at least this % of full scale; below it, speech opens no turn, interrupts no reply and does not keep a turn open. Silero cannot do this: a far voice scores as high as a near one. On a reSpeaker direct speech peaks at 15–55 % and the room floor at 1–2 %, so 3–5 is a sensible start (`4` on the Spark). Peak, not RMS — speech RMS is only 5–9 %. Depends on microphone gain and AGC: tune per device. |
+| `VAD_EAGER_MS` | `0` (off) | **Hidden speculation.** At this much provisional silence the bridge runs STT → generate → TTS out of sight. Speech resumes → discarded without trace. Turn confirmed → adopted, but only if the real generate request is byte-identical to the speculative one. Costs a wasted STT + generate on every mid-sentence pause. |
+| `CONTINUATION_WINDOW_MS` | `0` (off) | **Continuation merge.** An onset this soon after a silence commit is the speaker carrying on: the reply stops (inside the barge-in guard window too), the half-turn leaves history, both halves are re-transcribed as one turn. One take-back per utterance. What makes a short `VAD_SILENCE_MS` affordable. |
+| `CONTINUATION_TOOL_HOLD_MS` | `500` | With the merge on, a finished tool call is held this long after the commit — a call the client already has cannot be taken back. Speech is never held. A fixed cost on every tool turn. |
+
+The Spark runs `VAD_SILENCE_MS=500`, `VAD_EAGER_MS=160`,
+`CONTINUATION_WINDOW_MS=1200`: plain replies 328–370 ms after the commit
+(~830–870 ms after the speaker stops), tool turns 1.16–1.64 s.
+
+**Honest limits.** The continuation merge has been proven on a recording only —
+no live session has yet produced one. A noise burst with no words in it still
+counts as a barge-in. A merged turn sends two `transcription.completed` events
+(the half, then the whole) and nothing on the wire marks the second as
+superseding the first. English words inside Hebrew speech are misheard and hard
+to follow when spoken (issue #277).
+
+## BlueTTS — built, measured, deliberately not the default
+
+`lobes.realtime.bluetts_server` is a drop-in for the `chatterbox` service (same
+port keys, same `POST /v1/audio/synthesize` → raw PCM16 mono 24 kHz). CPU-only,
+resamples its native 44.1 kHz, strips any niqqud (it runs its own RenikudPlus
+G2P — leave the bridge's `PHONIKUD_MODEL_PATH` unset), and reports ready only
+after a warm-up synthesis has loaded that G2P.
+
+The engine **code** (github.com/maxmelichov/BlueTTS) is MIT and pinned by
+commit; `Dockerfile.bluetts` installs its dependencies from the engine's own
+`uv.lock`, because an unpinned resolve picked a `renikud-plus` that 404s at
+warm-up. The **weights** repo declared **no licence** when this was written, so:
+
+- nothing in this repo downloads the weights or names their repo as a default;
+- the operator downloads them deliberately and mounts the directory at
+  `BLUETTS_ONNX_DIR`;
+- the Hebrew overlay's shipped voice stays Chatterbox Multilingual, and no
+  compose file wires BlueTTS in.
+
+To run it, add a service built from `Dockerfile.bluetts` with that mount and
+point the bridge's `TTS_URL` at it. `BLUETTS_VOICE` (`noa`), `BLUETTS_STEPS`
+(5; 2–3 buys ~20–25 %, quality not auditioned), `BLUETTS_SPEED`.
+
+## Audio hardware notes (both cost hours)
+
+- **Check the pipewire card profile first.** The reSpeaker XVF3800 had flipped
+  to its *digital* output at 34 % volume: almost nothing reached the speaker on
+  its 3.5 mm jack, its echo canceller reported `converged: false` all day, and
+  the session interrupted itself (6 times in 7 turns). `pactl set-card-profile
+  <card> output:analog-stereo+input:analog-stereo` plus full sink volume →
+  `converged: true`, zero self-interruptions. The profile can flip when the
+  device re-enumerates, and the source/sink numeric suffixes change with it.
+- **Wire the speaker to the device that owns the microphone.** An echo
+  canceller only cancels what it plays itself; audio sent to a monitor over
+  HDMI has no reference and makes echo worse. For the same reason you cannot
+  test a session by playing a recording through that speaker — the canceller
+  removes it.
+- **A client must drain its player before exiting.** The server delivers ahead
+  of the playhead; closing the player on `response.done` cut replies
+  mid-sentence. In a browser the same gap shows up as stutter unless playback
+  (re)starts a jitter buffer ahead of now.
+
+## Selecting it
+
+```bash
+lobes init --fleet --audio --audio-lang he        # dry-run; add --apply
+```
+
+materialises `docker-compose.audio-he.yml`, `env.audio-he.example` (merged
+append-only into `.env`), `Dockerfile.whisper-stt`, `Dockerfile.chatterbox-ml`,
+`Dockerfile.bluetts` and `listen_server_whisper.py`; `lobes fleet up` includes
+the overlay when it is present. `--audio` alone, or `--audio-lang en`, is
+exactly the English overlay. Language is an overlay choice — not a shape, not a
+variation.
+
+`GET /capabilities` names what is served: `stt`/`tts` carry the declared
+`model`, `runtime` and — only when declared — `language`.

{lobes_cli-0.90.2 → lobes_cli-0.91.0}/lobes/profiles/builtin/base.toml

@@ -90,12 +90,18 @@ feasible = true
 model = "Qwen/Qwen3-Embedding-0.6B"
 gpu_mem_util = 0.06
 max_model_len = 8192
+# quantization "none": the lane passes no --quantization, so the engine loads the
+# bf16 checkpoint as-is. Declared so the mesh fingerprint is a real value, not
+# "unknown", and identical members pool on a box that does not host them (#297).
+quantization = "none"
 
 [roles.reranker]
 feasible = true
 model = "Qwen/Qwen3-Reranker-0.6B"
 gpu_mem_util = 0.06
 max_model_len = 8192
+# quantization "none": as for the embedder above (#297).
+quantization = "none"
 
 # The opt-in `innereye` render tenant (ComfyUI, issue #82) is vetoed on an
 # unrecognised card for the same reason as muse/worker/associate: this

{lobes_cli-0.90.2 → lobes_cli-0.91.0}/lobes/profiles/builtin/orin.toml

@@ -494,6 +494,10 @@ feasible = true
 model = "Qwen/Qwen3-Embedding-0.6B"
 gpu_mem_util = 0.06
 max_model_len = 8192
+# quantization "none": the lane passes no --quantization, so the engine loads the
+# bf16 checkpoint as-is. Declared so the mesh fingerprint is a real value, not
+# "unknown", and identical members pool on a box that does not host them (#297).
+quantization = "none"
 # attention_backend: overlaid from lobes.machines.get("orin") — TRITON_ATTN, a
 # conservative first-boot carry-over of Thor's sm_110 pooling divergence that
 # was NOT independently re-tested on sm_87. This gear booted healthy that way
@@ -504,6 +508,8 @@ feasible = true
 model = "Qwen/Qwen3-Reranker-0.6B"
 gpu_mem_util = 0.06
 max_model_len = 8192
+# quantization "none": as for the embedder above (#297).
+quantization = "none"
 # attention_backend + enforce_eager: overlaid from lobes.machines.get("orin") —
 # TRITON_ATTN + enforce_eager=true, the same carried-over Jetson safe-boot
 # choice. This gear booted healthy that way and its ordering probe passed

{lobes_cli-0.90.2 → lobes_cli-0.91.0}/lobes/profiles/builtin/spark.toml

@@ -106,12 +106,18 @@ feasible = true
 model = "Qwen/Qwen3-Embedding-0.6B"
 gpu_mem_util = 0.06
 max_model_len = 8192
+# quantization "none": the lane passes no --quantization, so the engine loads the
+# bf16 checkpoint as-is. Declared so the mesh fingerprint is a real value, not
+# "unknown", and identical members pool on a box that does not host them (#297).
+quantization = "none"
 
 [roles.reranker]
 feasible = true
 model = "Qwen/Qwen3-Reranker-0.6B"
 gpu_mem_util = 0.06
 max_model_len = 8192
+# quantization "none": as for the embedder above (#297).
+quantization = "none"
 
 # --- innereye: the declared peak-GiB figure (issue #268, plan
 # innereye-lobes-hosts-comfyui, t6) --------------------------------------

{lobes_cli-0.90.2 → lobes_cli-0.91.0}/lobes/profiles/builtin/thor.toml

@@ -141,6 +141,10 @@ feasible = true
 model = "Qwen/Qwen3-Embedding-0.6B"
 gpu_mem_util = 0.06
 max_model_len = 8192
+# quantization "none": the lane passes no --quantization, so the engine loads the
+# bf16 checkpoint as-is. Declared so the mesh fingerprint is a real value, not
+# "unknown", and identical members pool on a box that does not host them (#297).
+quantization = "none"
 # attention_backend: overlaid from the shared SM_110 trait — TRITON_ATTN
 # (the FlashInfer/FLASH_ATTN pooling path hangs on sm_110).
 
@@ -149,5 +153,7 @@ feasible = true
 model = "Qwen/Qwen3-Reranker-0.6B"
 gpu_mem_util = 0.06
 max_model_len = 8192
+# quantization "none": as for the embedder above (#297).
+quantization = "none"
 # attention_backend + enforce_eager: overlaid from the shared SM_110 trait —
 # TRITON_ATTN + enforce_eager=true (CUDA-graph capture is unstable here).

{lobes_cli-0.90.2 → lobes_cli-0.91.0}/lobes/templates/fleet/Dockerfile.whisper-stt

@@ -39,6 +39,15 @@ RUN pip install --no-cache-dir uv \
         fastapi \
         "uvicorn[standard]"
 
+# Silero VAD (issue #329, STT_VAD) — skips non-speech chunks before Whisper
+# decodes them. The same package and version the realtime bridge pins in
+# uv.lock, but installed --no-deps: its declared torchaudio dependency would
+# pull a torchaudio wheel pinned to a DIFFERENT torch and replace the base
+# image's validated one. listen_server_whisper.py loads the wheel's bundled
+# TorchScript model by path (never `import silero_vad`, whose __init__
+# imports torchaudio), and falls back to decoding every chunk if it cannot.
+RUN uv pip install --system --no-cache-dir --no-deps "silero-vad==6.2.1"
+
 # Do NOT bake STT_MODEL into the image — it is served from the mounted HF
 # cache volume (docker-compose.audio.yml's `stt` service already mounts
 # ${HF_CACHE}), exactly like every other fleet lane. HF_HUB_DISABLE_XET=1

{lobes_cli-0.90.2 → lobes_cli-0.91.0}/lobes/templates/fleet/docker-compose.audio-he.yml

@@ -87,6 +87,41 @@ services:
       # 'תודה רבה'). Measured 2026-09-18: speech >= -0.12, hallucinations <= -0.58.
       # `off` disables the gate.
       - STT_MIN_AVG_LOGPROB=${STT_MIN_AVG_LOGPROB:--0.35}
+      # Decode-quality knobs (issue #329; docs/hebrew-realtime.md). Each has
+      # an OFF value that restores the pre-#329 decode exactly.
+      # STT_LOGPROB_STAT: `median` (default since the 2026-10-23 live A/B;
+      # ignores a few rare tokens in an otherwise confident chunk) or `mean`
+      # (the legacy statistic -0.35 was first measured on). Applies to
+      # multi-window uploads only: a single-window request (every realtime
+      # turn) always uses the mean and one greedy decode, as before #329.
+      - STT_LOGPROB_STAT=${STT_LOGPROB_STAT:-median}
+      # STT_NO_REPEAT_NGRAM_SIZE: block any repeated token n-gram at decode
+      # time (stops a phrase looping to the token cap). 0 = off.
+      - STT_NO_REPEAT_NGRAM_SIZE=${STT_NO_REPEAT_NGRAM_SIZE:-6}
+      # STT_REPETITION_PENALTY: transformers' repetition penalty. 1.0 = off
+      # (default: it penalises every recurring Hebrew prefix letter too).
+      - STT_REPETITION_PENALTY=${STT_REPETITION_PENALTY:-1.0}
+      # STT_COMPRESSION_RATIO_MAX: a chunk whose text compresses better than
+      # this (zlib, one byte per character) is a loop. `off` disables.
+      - STT_COMPRESSION_RATIO_MAX=${STT_COMPRESSION_RATIO_MAX:-2.4}
+      # STT_MAX_WORD_RUN: a run of one word (or two-word phrase) this many
+      # times in a row is TRIMMED to one fewer repeats. 0/`off` disables.
+      - STT_MAX_WORD_RUN=${STT_MAX_WORD_RUN:-4}
+      # STT_WORD_RUN_FAIL: a run this long fails a multi-window chunk like a
+      # loop (retry, then drop); a single window only trims. 0/`off` disables.
+      - STT_WORD_RUN_FAIL=${STT_WORD_RUN_FAIL:-8}
+      # STT_TEMPERATURE_FALLBACK: sampling temperatures a multi-window chunk
+      # failing any gate is re-decoded at before it is dropped (a single
+      # window is never retried). `off` = drop at once.
+      - STT_TEMPERATURE_FALLBACK=${STT_TEMPERATURE_FALLBACK:-0.2,0.4}
+      # STT_VAD: `auto` skips non-speech chunks of a multi-window upload with
+      # Silero before decoding (never a single-window/realtime turn; falls
+      # back to decoding everything if Silero cannot load); `off`.
+      - STT_VAD=${STT_VAD:-auto}
+      # STT_VAD_THRESHOLD / STT_VAD_MIN_SPEECH_MS: a chunk is speech once this
+      # many ms of 32 ms windows score at or above the threshold.
+      - STT_VAD_THRESHOLD=${STT_VAD_THRESHOLD:-0.5}
+      - STT_VAD_MIN_SPEECH_MS=${STT_VAD_MIN_SPEECH_MS:-250}
       # STT_RUNTIME: informational — names the serving stack for
       # GET /capabilities; the container fixed on transformers, not a
       # switchable knob.

{lobes_cli-0.90.2 → lobes_cli-0.91.0}/lobes/templates/fleet/env.audio-he.example

@@ -117,6 +117,83 @@ STT_LANGUAGE=he
 # real speech >= -0.12, hallucinations <= -0.58. `off` disables it.
 STT_MIN_AVG_LOGPROB=-0.35
 
+# --- Decode quality (issue #329) ------------------------------------------
+# Measured on one real 56-minute recording before these knobs existed: a
+# two-word phrase looped to the token cap (a loop is HIGH-confidence, so the
+# gate above passes it) and half the chunks were dropped whole, some of them
+# real speech. Every knob below has an OFF value that restores that previous
+# decode exactly. The defaults are starting points (the issue's and OpenAI
+# Whisper's); a live A/B on the DGX Spark (2026-10-23) then moved
+# STT_LOGPROB_STAT to `median` and added the word-run knobs — see
+# docs/hebrew-realtime.md. Every retried, dropped, trimmed or skipped chunk
+# logs one `stt_chunk event=...` line (reason, avg/median log-probability,
+# compression ratio, word run, attempt), and every request one
+# `stt_request policy=...` summary line, in the stt container's log.
+#
+# Two request policies. A SINGLE-window request (<= 30 s — every realtime
+# turn) keeps the pre-#329 gate exactly: the MEAN log-probability against
+# STT_MIN_AVG_LOGPROB, one greedy decode, no retry, and a word run only
+# trimmed. A MULTI-window upload uses STT_LOGPROB_STAT, the
+# STT_TEMPERATURE_FALLBACK ladder and STT_WORD_RUN_FAIL.
+
+# STT_LOGPROB_STAT: which statistic STT_MIN_AVG_LOGPROB is compared with on
+# a multi-window upload. `median` (default — robust to a few rare tokens, e.g.
+# a name, in an otherwise confident chunk; on the live A/B it dropped 39 of
+# 215 decoded chunks against the mean's 118) or `mean` (the legacy statistic
+# -0.35 was first measured on). A single-window request always uses `mean`.
+STT_LOGPROB_STAT=median
+
+# STT_NO_REPEAT_NGRAM_SIZE: forbid any token n-gram from repeating within a
+# chunk at decode time. 0 = off.
+STT_NO_REPEAT_NGRAM_SIZE=6
+
+# STT_REPETITION_PENALTY: transformers' repetition penalty (>= 1.0). 1.0 = off,
+# the default: it penalises EVERY recurring token, including Hebrew's prefix
+# letters, so it is an accuracy risk the n-gram block is not. 1.1 to try it.
+STT_REPETITION_PENALTY=1.0
+
+# STT_COMPRESSION_RATIO_MAX: OpenAI Whisper's loop check — a chunk whose text
+# compresses better than this under zlib is treated as a loop. Measured one
+# byte per character, so ordinary Hebrew scores like English (~1.6-1.7).
+# `off` disables.
+STT_COMPRESSION_RATIO_MAX=2.4
+
+# STT_MAX_WORD_RUN: a run of one word, or one two-word phrase, this many
+# times in a row (punctuation between does not break the run) is TRIMMED to
+# STT_MAX_WORD_RUN - 1 repeats; the rest of the chunk is kept. 4 = three
+# repeats are left alone, 4-7 become three ("no no no no, I disagree" is
+# ordinary speech). 0/`off` disables trimming.
+STT_MAX_WORD_RUN=4
+
+# STT_WORD_RUN_FAIL: a run this long is far beyond normal speech, so on a
+# multi-window upload it fails the chunk like a loop: retried, then dropped
+# with reason=word_run. Catches the confident short-word hallucination the
+# n-gram block and the compression ratio miss (the live A/B found runs of 11
+# and 5: 11 is re-decoded or dropped, 5 is trimmed to 3). A single-window
+# request only trims. 0/`off` never fails on a word run.
+STT_WORD_RUN_FAIL=8
+
+# STT_TEMPERATURE_FALLBACK: comma-separated sampling temperatures in (0, 1]. A
+# multi-window chunk failing the confidence gate, the compression check or
+# the word-run check is re-decoded at each in turn and the first result
+# passing all three is kept; if none does (or a retry comes back empty), the
+# chunk is dropped. Each retry is one more Whisper decode. A single-window
+# request is never retried. `off` = no retry.
+STT_TEMPERATURE_FALLBACK=0.2,0.4
+
+# STT_VAD: `auto` runs Silero VAD over each chunk of an upload longer than one
+# 30 s window and skips the ones without speech, so silence never reaches
+# Whisper (where it hallucinates text). A single-window request — every
+# realtime turn — is never skipped, nor is the last (short) chunk of an
+# upload or a chunk under one 32 ms VAD window. Falls back to decoding every
+# chunk when Silero cannot load. `off` disables.
+STT_VAD=auto
+
+# STT_VAD_THRESHOLD / STT_VAD_MIN_SPEECH_MS: a chunk counts as speech once at
+# least STT_VAD_MIN_SPEECH_MS of its 32 ms windows score >= STT_VAD_THRESHOLD.
+STT_VAD_THRESHOLD=0.5
+STT_VAD_MIN_SPEECH_MS=250
+
 # === Hebrew TTS: Chatterbox Multilingual sidecar (hebrew-realtime plan,
 # task t11 — approved deviation d5, operator 2026-09-18) ===================
 # docker-compose.audio-he.yml's `chatterbox` service overrides

{lobes_cli-0.90.2 → lobes_cli-0.91.0}/lobes/templates/fleet/env.example

@@ -149,6 +149,9 @@ EMBED_MAX_MODEL_LEN=8192
 # 0.06 load-tested 2026-06-19; 0.025 fails with "No available memory for the cache
 # blocks" (vLLM's pooling runner still reserves a cache-block budget).
 EMBED_GPU_MEM_UTIL=0.06
+# Declared for the mesh fingerprint only (the lane passes no --quantization): a real
+# value lets identical embedder members pool on a box that does not host one (#297).
+EMBED_QUANTIZATION=none
 # Internal URL the gateway uses to reach the embedding backend.
 EMBED_URL=http://vllm-embed:8000
 # Attention backend for the pooling runner. "auto" (the shipped default) is
@@ -215,6 +218,8 @@ RERANK_MODEL=Qwen/Qwen3-Reranker-0.6B
 RERANK_SERVED_NAME=Qwen/Qwen3-Reranker-0.6B
 RERANK_MAX_MODEL_LEN=8192
 RERANK_GPU_MEM_UTIL=0.06
+# As EMBED_QUANTIZATION above (#297).
+RERANK_QUANTIZATION=none
 # Internal URL the gateway uses to reach the reranker backend.
 RERANK_URL=http://vllm-rerank:8000
 # Attention backend for the pooling runner — same "auto" sentinel/rationale as