pi-voicekit 0.2.3 → 0.3.1

package/README.md

@@ -260,6 +260,34 @@ Models from [Handy](https://github.com/cjpais/handy) (`~/Library/Application Sup
 
 ---
 
+## Performance
+
+Measured on the maintainer's machine, local CPU, no network, over 70 published utterances
+(28 Chinese, 28 English, 14 mixed Chinese–English). RTF is processing time divided by audio
+duration — lower is better, and below 1.0 is faster than real time.
+
+| Recogniser           | Chinese RTF | English RTF | Mixed RTF | Characters per second |
+| -------------------- | ----------- | ----------- | --------- | --------------------- |
+| **paraformer-zh**    | 0.014       | 0.013       | 0.016     | 247–983               |
+| **sensevoice-small** | 0.028       | 0.029       | 0.040     | 134–451               |
+| **whisper-turbo**    | 0.372       | 0.369       | 0.395     | 7–27                  |
+
+End to end — local recognition plus the remote polish call — four real dictations of
+8.1–28.0 s came back in an estimated 0.56–2.98 s, an RTF of 0.07–0.15 (recognition time is
+derived from the measured recognition RTF, not timed per dictation: see docs/BENCHMARKS.md).
+The segmented polish pass is what
+keeps that together: the 0.2.x line polished in one call over the whole transcript, so one
+slow call returned the dictation unpolished, while against a degraded endpoint the segmented
+pipeline polished 35 of 35 segments where the old path fell back on 100% of the run.
+`whisper-turbo` is the slowest of the three by an order of magnitude and the least accurate
+on this corpus — measured for comparison, not recommended for CPU-only use.
+
+Protocol, all result tables, reproduction commands and the honest limits live in
+[docs/BENCHMARKS.md](docs/BENCHMARKS.md) — GitHub only, because npm ships the extension and
+this README.
+
+---
+
 ## Features
 
 | Feature                          | Description                                                                              |
@@ -274,7 +302,7 @@ Models from [Handy](https://github.com/cjpais/handy) (`~/Library/Application Sup
 | **Pre-recording**                | Audio capture starts during warmup — you never miss the first word                       |
 | **Tail recording**               | Keeps recording 1.5s after release so your last word isn't clipped                       |
 | **Live streaming**               | Deepgram Nova 3 WebSocket (Nova 2 for Chinese locales) — live interim transcripts        |
-| **Transcript polish**            | Optional post-ASR cleanup — every dictation makes one extra model call; the last N conversation turns (default 2) are sent with it, and no conversation context at all when the turn count is zero. Disable with `/voice-polish off` |
+| **Transcript polish**            | Optional post-ASR cleanup — the local backend polishes each recogniser segment as it is decoded (up to three calls in flight); the last N conversation turns (default 2) are sent with the first segment, and no conversation context at all when the turn count is zero. Disable with `/voice-polish off` |
 | **56+ languages**                | Deepgram: 56+ with live streaming. Local: up to 57 depending on model.                   |
 | **Continuous dictation**         | `/voice dictate` for long-form input without holding keys                                |
 | **Typing cooldown**              | Space holds within 400ms of typing are ignored                                           |
@@ -367,7 +395,7 @@ Hold-to-talk delay defaults to **700 ms** (`/voice-hold-delay` accepts 200–300
 
 ### Transcript polish
 
-Transcript polish is on by default: every dictation runs one extra model call. When
+Transcript polish is on by default: every dictation runs an extra model pass. When
 the selected model is a cloud provider, the text that leaves your machine is:
 
 - the transcript of the dictation;
@@ -389,6 +417,19 @@ punctuation), while shorter ones keep it, because there it costs almost nothing
 terms and self-corrections better. The field only reaches OpenAI-compatible providers; one that
 ignores it behaves exactly as before.
 
+On the local backend, polish no longer waits for the whole transcript: each recogniser
+segment — roughly 10 s of speech — is polished as it is decoded, with up to three segment
+calls in flight. A long dictation may come back partly polished, and that is deliberate: a
+segment that times out is retried once with thinking disabled for that retry, and if it still
+fails, that segment keeps its raw text while its neighbours keep their polished text — so one
+slow call no longer costs the rest of the dictation. The conversation context is attached to
+the first segment only; a later segment sees just the previous segment's raw text.
+
+The gain is measurable: on 79.6 s of corpus audio (35 segments) against a degraded endpoint,
+the old single-call path fell back on 100% of the run, while the segmented path polished all
+35 segments. Real dictations after the change: 4 of 4 applied, with polish taking 0.4–2.3 s
+for 8–28 s of audio — roughly 5–13% of the audio duration.
+
 Assistant text can contain anything the conversation contained — file paths,
 identifiers, values the agent echoed. The character limits bound how much is sent,
 not how sensitive it is. With the local backend, nothing else leaves your machine,
@@ -402,7 +443,9 @@ analysis, and they do keep the raw text on disk for as long as the session file
 Each entry also records how the pass was configured: the transcript length on its own
 (separate from any text already in the editor), whether thinking was turned off for it,
 and the output-token cap it carried, plus the audio seconds it covered and which recogniser
-produced it — which is what makes polish time readable as a speedup.
+produced it — which is what makes polish time readable as a speedup. A segmented pass adds a
+`segments` summary to the entry: how many segments were polished, how many kept their raw
+text, and how many were retried.
 
 | Setting                   | Scope              | Default     | Notes                                                   |
 | ------------------------- | ------------------ | ----------- | ------------------------------------------------------- |

package/extensions/voice/local.ts

@@ -816,6 +816,21 @@ export function localLanguageDisplayName(code: string): string {
 
 // ─── Local session type ──────────────────────────────────────────────────────
 
+/**
+ * Callbacks a local recording session accepts. `onSegment` is optional, so callers that do
+ * not pipeline recogniser segments keep today's transcript-only behaviour.
+ */
+export interface LocalSessionCallbacks {
+	onTranscript: (interim: string, finals: string[]) => void;
+	onDone: (fullText: string, meta: { hadAudio: boolean; hadSpeech: boolean }) => void;
+	onError: (err: string) => void;
+	/**
+	 * Called as each recogniser segment decodes, before the next segment starts, with the
+	 * segment's zero-based index. Observational: it never changes the transcript.
+	 */
+	onSegment?: (text: string, index: number) => void;
+}
+
 export interface LocalSession {
 	backend: "local";
 	recProcess: ChildProcess;
@@ -825,6 +840,8 @@ export interface LocalSession {
 	onTranscript: (interim: string, finals: string[]) => void;
 	onDone: (fullText: string, meta: { hadAudio: boolean; hadSpeech: boolean }) => void;
 	onError: (err: string) => void;
+	/** Optional per-segment observer; forwarded to the in-process recogniser on stop. */
+	onSegment?: (text: string, index: number) => void;
 }
 
 // ─── WAV encoding ────────────────────────────────────────────────────────────
@@ -956,14 +973,7 @@ export async function transcribeWithServer(wavBuffer: Buffer, config: VoiceConfi
  * Start a local recording session. Audio is buffered in memory.
  * Transcription happens when stopLocalSession() is called.
  */
-export function startLocalSession(
-	recProcess: ChildProcess,
-	callbacks: {
-		onTranscript: (interim: string, finals: string[]) => void;
-		onDone: (fullText: string, meta: { hadAudio: boolean; hadSpeech: boolean }) => void;
-		onError: (err: string) => void;
-	}
-): LocalSession {
+export function startLocalSession(recProcess: ChildProcess, callbacks: LocalSessionCallbacks): LocalSession {
 	const session: LocalSession = {
 		backend: "local",
 		recProcess,
@@ -973,6 +983,7 @@ export function startLocalSession(
 		onTranscript: callbacks.onTranscript,
 		onDone: callbacks.onDone,
 		onError: callbacks.onError,
+		onSegment: callbacks.onSegment,
 	};
 
 	recProcess.stdout?.on("data", (chunk: Buffer) => {
@@ -1042,7 +1053,7 @@ export async function stopLocalSession(session: LocalSession, config: VoiceConfi
 			// In-process via sherpa-onnx (default, 120s timeout)
 			let timeoutHandle: ReturnType<typeof setTimeout> | undefined;
 			text = await Promise.race([
-				transcribeInProcess(pcmData, config),
+				transcribeInProcess(pcmData, config, session.onSegment),
 				new Promise<never>((_, reject) => {
 					timeoutHandle = setTimeout(() => reject(new Error("Transcription timed out (120s)")), 120_000);
 				}),
@@ -1087,15 +1098,13 @@ export function abortLocalSession(session: LocalSession | null): void {
  * Transcribe PCM audio using sherpa-onnx in-process.
  * Auto-downloads model on first use.
  */
-async function transcribeInProcess(pcmData: Buffer, config: VoiceConfig): Promise<string> {
-	const {
-		initSherpa,
-		isSherpaAvailable,
-		getSherpaError,
-		getOrCreateRecognizer,
-		transcribeBuffer,
-		transcribeBufferSegmented,
-	} = await import("./sherpa-engine");
+async function transcribeInProcess(
+	pcmData: Buffer,
+	config: VoiceConfig,
+	onSegment?: (text: string, index: number) => void
+): Promise<string> {
+	const { initSherpa, isSherpaAvailable, getSherpaError, getOrCreateRecognizer, transcribeBufferSegmented } =
+		await import("./sherpa-engine");
 	const { ensureModelDownloaded } = await import("./model-download");
 
 	// Initialize sherpa if needed
@@ -1116,11 +1125,10 @@ async function transcribeInProcess(pcmData: Buffer, config: VoiceConfig): Promis
 
 	// Create/reuse recognizer and transcribe
 	const recognizer = getOrCreateRecognizer(model, modelDir, config.language || "en");
-	// Qwen3-ASR caps context at 512 tokens (~18s); segment long audio via VAD before decode.
-	if (model.sherpaModel.type === "qwen3_asr") {
-		return transcribeBufferSegmented(pcmData, recognizer);
-	}
-	return transcribeBuffer(pcmData, recognizer);
+	// Every in-process model decodes long audio in VAD-sized pieces, so a pipelining caller can
+	// polish one piece while the rest still decodes; audio at or below the segment threshold takes
+	// the same single-decode fast path as before and the concatenation is unchanged.
+	return transcribeBufferSegmented(pcmData, recognizer, undefined, onSegment);
 }
 
 /** Check if a local transcription server is reachable. */