@drakulavich/kesha-voice-kit 1.10.1 → 1.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +12 -35
- package/SKILL.md +12 -2
- package/package.json +2 -2
- package/src/cli/install.ts +24 -38
- package/src/cli/main.ts +11 -1
- package/src/engine-install.ts +88 -45
- package/src/engine.ts +50 -11
- package/src/lib.ts +8 -1
- package/src/star.ts +76 -0
- package/src/status.ts +12 -0
- package/src/transcribe.ts +9 -2
package/README.md
CHANGED
|
@@ -60,52 +60,29 @@ Stdout: transcript. Stderr: errors. Pipe-friendly. Also available as `parakeet`
|
|
|
60
60
|
|
|
61
61
|
For long / silence-heavy audio, use `--vad` (auto-on past 120 s). Details: [docs/vad.md](docs/vad.md).
|
|
62
62
|
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
Kesha speaks back via Kokoro-82M (English) and Vosk-TTS (Russian) — voice auto-picks from the text's language:
|
|
63
|
+
**Speaker diarization** (darwin-arm64, post-v1.12.0):
|
|
66
64
|
|
|
67
65
|
```bash
|
|
68
|
-
kesha install --
|
|
69
|
-
kesha
|
|
70
|
-
|
|
66
|
+
kesha install --diarize # one-time, ~245MB Sortformer model
|
|
67
|
+
kesha --json --vad --speakers meeting.m4a > out.json
|
|
68
|
+
jq '.[0].segments[] | "\(.speaker)\t\(.text)"' out.json
|
|
71
69
|
```
|
|
72
70
|
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
```bash
|
|
76
|
-
# Auto-detect on by default — ФСБ reads as "эф-эс-бэ"
|
|
77
|
-
kesha say --voice ru-vosk-m02 'ФСБ объявила решение.'
|
|
71
|
+
Each segment gets a `speaker` integer (cluster ID, stable within one file). Linux / Windows: `--speakers` returns a clear "currently darwin-arm64 only" error — see [#199](https://github.com/drakulavich/kesha-voice-kit/issues/199).
|
|
78
72
|
|
|
79
|
-
|
|
80
|
-
kesha say --voice ru-vosk-m02 --no-expand-abbrev 'ФСБ.'
|
|
81
|
-
|
|
82
|
-
# Explicit SSML control (overrides the rule and the stop-list)
|
|
83
|
-
kesha say --voice ru-vosk-m02 --ssml \
|
|
84
|
-
'<speak><say-as interpret-as="characters">КОТ</say-as></speak>'
|
|
85
|
-
```
|
|
86
|
-
|
|
87
|
-
Detection rule: auto-expand fires when the token cannot be pronounced as a natural Russian syllable — length ≤ 2 (ИП, ЕС), 0 vowels (ФСБ, СНГ), 2+ consecutive vowels (ОАЭ), or 2+ consecutive consonants (США, ЦСКА). Tokens with strict CVC/CVCV alternation pass through (ВОЗ → "воз", НАТО → "нато", ОПЕК → "опек"). Small stop-list for common short words (ОН, МЫ, ВЫ, КАК, ЧТО, …) and Ъ/Ь-containing tokens are always skipped. See [#232](https://github.com/drakulavich/kesha-voice-kit/issues/232).
|
|
73
|
+
## Text-to-speech
|
|
88
74
|
|
|
89
|
-
|
|
75
|
+
Kesha speaks back via Kokoro-82M (English) and Vosk-TTS (Russian) — voice auto-picks from the text's language:
|
|
90
76
|
|
|
91
77
|
```bash
|
|
92
|
-
|
|
93
|
-
kesha say
|
|
94
|
-
|
|
95
|
-
# Force a literal reading (Kokoro fuses unknown caps tokens to one syllable)
|
|
96
|
-
kesha say --voice en-am_michael --no-expand-abbrev 'FBI.'
|
|
97
|
-
|
|
98
|
-
# Explicit SSML control (overrides the rule and the stop-list)
|
|
99
|
-
kesha say --voice en-am_michael --ssml \
|
|
100
|
-
'<speak><say-as interpret-as="characters">NASA</say-as></speak>'
|
|
78
|
+
kesha install --tts # ~990MB (Kokoro + Vosk-TTS RU, opt-in)
|
|
79
|
+
kesha say "Hello, world" > hello.wav
|
|
80
|
+
kesha say "Привет, мир" > privet.wav # auto-routes (Milena on darwin, ru-vosk-m02 elsewhere)
|
|
101
81
|
```
|
|
102
82
|
|
|
103
|
-
|
|
104
|
-
- **Letter-spell** — uppercase Latin tokens 2–5 chars (no digits, no mixed case) get expanded letter-by-letter (FBI → "ef bee eye"). Disable with `--no-expand-abbrev`.
|
|
105
|
-
- **`STOP_LIST`** (30 entries) — natural-English caps words pass through unchanged: NASA, NATO, AIDS, OPEC, IKEA, ASCII, NAFTA, LASER, RADAR, SCUBA + emphatic length-2 caps (OK, NO, GO, IT, IS, AS, AT, BY, IN, ON, OR, OF, TO, WE, US, MY, ME, HE, BE, DO).
|
|
106
|
-
- **`IPA_LEXICON`** (20 entries) — case-sensitive token → IPA-phoneme map that bypasses G2P entirely. Covers all-caps acronyms with industry pronunciations (EPAM, JSON, JPEG, SQL, ASAP, GIF, CRUD, JWT) AND mixed-case proper nouns (Anthropic, Microsoft, Claude, NVIDIA, Kubernetes, PostgreSQL, GraphQL, Linux, Tokio, macOS, Granola, OAuth). IPA hits fire even with `--no-expand-abbrev` — they're intent-explicit, parallel to `<say-as>`.
|
|
83
|
+
**Russian abbreviations** (`ru-vosk-*`): all-uppercase Cyrillic 2-5-char tokens auto-expand letter-by-letter when not pronounceable as a Russian syllable (ФСБ → "эф-эс-бэ", ВОЗ → "воз"). Disable with `--no-expand-abbrev`. See [docs/tts.md#russian-abbreviation-auto-expansion](docs/tts.md#russian-abbreviation-auto-expansion).
|
|
107
84
|
|
|
108
|
-
|
|
85
|
+
**English acronyms** (`en-*`, Kokoro): three-table mechanism (letter-spell rule + STOP_LIST + IPA_LEXICON) auto-expands FBI → "ef bee eye" and gives EPAM/JSON/Anthropic the right IPA. Disable letter-spell with `--no-expand-abbrev`. See [docs/tts.md#english-acronym-auto-expansion](docs/tts.md#english-acronym-auto-expansion).
|
|
109
86
|
|
|
110
87
|
**Russian word stress** (`ru-vosk-*` voices):
|
|
111
88
|
|
package/SKILL.md
CHANGED
|
@@ -54,6 +54,16 @@ jq '.[0].segments' voice.timestamps.json
|
|
|
54
54
|
|
|
55
55
|
Each segment has `start`, `end`, and `text` fields. `--timestamps` is available for machine-readable output (`--json`, `--toon`, or `--format json`).
|
|
56
56
|
|
|
57
|
+
**Speaker diarization** (darwin-arm64, post-v1.12.0). Add `--speakers` to label each segment with a cluster ID — useful for transcribing multi-person calls / meetings:
|
|
58
|
+
|
|
59
|
+
```bash
|
|
60
|
+
kesha install --diarize # one-time, ~245MB
|
|
61
|
+
kesha --json --vad --speakers meeting.m4a > out.json
|
|
62
|
+
jq '.[0].segments[] | "\(.speaker)\t\(.text)"' out.json
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
Each `segment.speaker` is a number (cluster id, stable within one file). On Linux / Windows the engine returns a clear "currently darwin-arm64 only" error — see [#199](https://github.com/drakulavich/kesha-voice-kit/issues/199).
|
|
66
|
+
|
|
57
67
|
**Formats:** .ogg, .opus, .mp3, .m4a, .wav, .flac, .webm — decoded via symphonia, no ffmpeg required.
|
|
58
68
|
|
|
59
69
|
**Other output modes:**
|
|
@@ -84,9 +94,9 @@ kesha say "Hi" --format ogg-opus --bitrate 16000 --out tiny.ogg # tin
|
|
|
84
94
|
|
|
85
95
|
Format is also inferred from `--out` extension (`.ogg` / `.opus` / `.oga` → OGG/Opus). `--bitrate` (6 000–510 000 bps) and `--sample-rate` (8 000 / 12 000 / 16 000 / 24 000 / 48 000 Hz) tune the encoder.
|
|
86
96
|
|
|
87
|
-
**Russian
|
|
97
|
+
**Russian abbreviations** (`ru-vosk-*`): all-uppercase Cyrillic 2-5-char tokens auto-expand letter-by-letter when not pronounceable as a Russian syllable (ФСБ → "эф-эс-бэ", ВОЗ → "воз"). Disable with `--no-expand-abbrev`. See [docs/tts.md#russian-abbreviation-auto-expansion](docs/tts.md#russian-abbreviation-auto-expansion).
|
|
88
98
|
|
|
89
|
-
**English
|
|
99
|
+
**English acronyms** (`en-*`, Kokoro): three-table mechanism (letter-spell rule + STOP_LIST + IPA_LEXICON) auto-expands FBI → "ef bee eye" and gives EPAM/JSON/Anthropic the right IPA. Disable letter-spell with `--no-expand-abbrev`. See [docs/tts.md#english-acronym-auto-expansion](docs/tts.md#english-acronym-auto-expansion).
|
|
90
100
|
|
|
91
101
|
**Russian word stress** (`ru-vosk-*` only): `<emphasis>сл+ово</emphasis>` shifts stress to the vowel marked with `+`. `<emphasis level="none">сл+ово</emphasis>` strips the `+` (cancel inherited emphasis). Other voices (`en-*`, `macos-*`) silently strip the `+` and warn once per process. Auto-stress dictionary not provided — caller writes the `+` manually. Closes [#233](https://github.com/drakulavich/kesha-voice-kit/issues/233).
|
|
92
102
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@drakulavich/kesha-voice-kit",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.12.0",
|
|
4
4
|
"description": "Open-source voice toolkit. Speech-to-text (25 langs, ~19x faster than Whisper on Apple Silicon, ONNX fallback on CPU), text-to-speech (Kokoro + Vosk-TTS + ~180 macOS system voices, SSML), voice-activity detection, language detection (107 langs). Rust engine, OpenClaw skill.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -30,7 +30,7 @@
|
|
|
30
30
|
]
|
|
31
31
|
},
|
|
32
32
|
"keshaEngine": {
|
|
33
|
-
"version": "1.
|
|
33
|
+
"version": "1.12.0"
|
|
34
34
|
},
|
|
35
35
|
"scripts": {
|
|
36
36
|
"test": "bun test",
|
package/src/cli/install.ts
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { defineCommand } from "citty";
|
|
2
2
|
import { downloadEngine } from "../engine-install";
|
|
3
3
|
import { getEngineBinPath } from "../engine";
|
|
4
|
-
import {
|
|
4
|
+
import { maybeAskForStar } from "../star";
|
|
5
5
|
import { log } from "../log";
|
|
6
6
|
|
|
7
7
|
interface InstallCommandArgs {
|
|
@@ -10,6 +10,7 @@ interface InstallCommandArgs {
|
|
|
10
10
|
"no-cache": boolean;
|
|
11
11
|
tts: boolean;
|
|
12
12
|
vad: boolean;
|
|
13
|
+
diarize: boolean;
|
|
13
14
|
}
|
|
14
15
|
|
|
15
16
|
const pkg = await Bun.file(new URL("../../package.json", import.meta.url)).json();
|
|
@@ -24,44 +25,24 @@ function resolveBackendFlag(coreml: boolean, onnx: boolean): string | undefined
|
|
|
24
25
|
return undefined;
|
|
25
26
|
}
|
|
26
27
|
|
|
27
|
-
async function
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
if (!
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
writeStarSeen(binPath, currentVersion);
|
|
41
|
-
} catch {
|
|
42
|
-
// Non-fatal — falling through to the prompt is still OK, just means we
|
|
43
|
-
// may nag again on the next install if the write failed for IO reasons.
|
|
44
|
-
}
|
|
45
|
-
|
|
46
|
-
const gh = Bun.which("gh");
|
|
47
|
-
if (!gh) {
|
|
48
|
-
log.info("\nIf you enjoy Kesha Voice Kit, consider starring the repo:");
|
|
49
|
-
log.info(" https://github.com/drakulavich/kesha-voice-kit");
|
|
50
|
-
return;
|
|
28
|
+
async function performInstall(
|
|
29
|
+
noCache: boolean,
|
|
30
|
+
backend?: string,
|
|
31
|
+
tts = false,
|
|
32
|
+
vad = false,
|
|
33
|
+
diarize = false,
|
|
34
|
+
) {
|
|
35
|
+
if (diarize && !(process.platform === "darwin" && process.arch === "arm64")) {
|
|
36
|
+
log.error(
|
|
37
|
+
"--diarize is currently darwin-arm64 only " +
|
|
38
|
+
"(see https://github.com/drakulavich/kesha-voice-kit/issues/199).",
|
|
39
|
+
);
|
|
40
|
+
process.exit(1);
|
|
51
41
|
}
|
|
52
|
-
const authCheck = Bun.spawnSync([gh, "auth", "status"], { stdout: "ignore", stderr: "ignore" });
|
|
53
|
-
if (authCheck.exitCode !== 0) return;
|
|
54
|
-
const starred = Bun.spawnSync([gh, "api", "user/starred/drakulavich/kesha-voice-kit"], { stdout: "ignore", stderr: "ignore" });
|
|
55
|
-
if (starred.exitCode === 0) return; // already starred
|
|
56
|
-
log.info("\n⭐ If you enjoy Kesha Voice Kit, star it on GitHub:");
|
|
57
|
-
log.info(" https://github.com/drakulavich/kesha-voice-kit");
|
|
58
|
-
log.info(' Or run: gh api -X PUT /user/starred/drakulavich/kesha-voice-kit');
|
|
59
|
-
}
|
|
60
|
-
|
|
61
|
-
async function performInstall(noCache: boolean, backend?: string, tts = false, vad = false) {
|
|
62
42
|
try {
|
|
63
|
-
await downloadEngine(noCache, backend, { tts, vad });
|
|
64
|
-
|
|
43
|
+
await downloadEngine(noCache, backend, { tts, vad, diarize });
|
|
44
|
+
const currentVersion = typeof pkg.version === "string" ? pkg.version : null;
|
|
45
|
+
await maybeAskForStar(getEngineBinPath(), currentVersion, log);
|
|
65
46
|
} catch (err: unknown) {
|
|
66
47
|
const message = err instanceof Error ? err.message : String(err);
|
|
67
48
|
log.error(message);
|
|
@@ -100,9 +81,14 @@ export const installCommand = defineCommand({
|
|
|
100
81
|
description: "Also install Silero VAD (~2.3MB) for long-audio preprocessing",
|
|
101
82
|
default: false,
|
|
102
83
|
},
|
|
84
|
+
diarize: {
|
|
85
|
+
type: "boolean",
|
|
86
|
+
description: "Also install the Sortformer streaming-diarization model (~245MB, darwin-arm64 only — #199)",
|
|
87
|
+
default: false,
|
|
88
|
+
},
|
|
103
89
|
},
|
|
104
90
|
async run({ args }: { args: InstallCommandArgs }) {
|
|
105
91
|
const backend = resolveBackendFlag(args.coreml, args.onnx);
|
|
106
|
-
await performInstall(args["no-cache"], backend, args.tts, args.vad);
|
|
92
|
+
await performInstall(args["no-cache"], backend, args.tts, args.vad, args.diarize);
|
|
107
93
|
},
|
|
108
94
|
});
|
package/src/cli/main.ts
CHANGED
|
@@ -24,6 +24,7 @@ interface MainCommandArgs {
|
|
|
24
24
|
vad: boolean;
|
|
25
25
|
"no-vad": boolean;
|
|
26
26
|
timestamps: boolean;
|
|
27
|
+
speakers: boolean;
|
|
27
28
|
format?: string;
|
|
28
29
|
lang?: string;
|
|
29
30
|
}
|
|
@@ -63,6 +64,11 @@ export const mainCommand = defineCommand({
|
|
|
63
64
|
description: "Include timestamped transcript segments in JSON/TOON output",
|
|
64
65
|
default: false,
|
|
65
66
|
},
|
|
67
|
+
speakers: {
|
|
68
|
+
type: "boolean",
|
|
69
|
+
description: "Include speaker labels in transcript segments. Requires --json / --toon / --format json. Implies --timestamps. Currently darwin-arm64 only (#199).",
|
|
70
|
+
default: false,
|
|
71
|
+
},
|
|
66
72
|
verbose: {
|
|
67
73
|
type: "boolean",
|
|
68
74
|
description: "Show language detection details",
|
|
@@ -109,6 +115,10 @@ export const mainCommand = defineCommand({
|
|
|
109
115
|
log.error("--timestamps requires --json, --toon, or --format json.");
|
|
110
116
|
process.exit(2);
|
|
111
117
|
}
|
|
118
|
+
if (args.speakers && !(args.json || args.toon || args.format === "json")) {
|
|
119
|
+
log.error("--speakers requires --json, --toon, or --format json.");
|
|
120
|
+
process.exit(2);
|
|
121
|
+
}
|
|
112
122
|
const vadMode = args.vad ? "on" : args["no-vad"] ? "off" : "auto";
|
|
113
123
|
|
|
114
124
|
if (files.length === 0) {
|
|
@@ -127,7 +137,7 @@ export const mainCommand = defineCommand({
|
|
|
127
137
|
// Run audio lang-id and transcription concurrently.
|
|
128
138
|
const [audioResult, transcript] = await Promise.all([
|
|
129
139
|
wantsLangId ? detectAudioLanguageEngine(file) : Promise.resolve(null),
|
|
130
|
-
transcribeWithSegments(file, { vad: vadMode, timestamps: args.timestamps }),
|
|
140
|
+
transcribeWithSegments(file, { vad: vadMode, timestamps: args.timestamps, speakers: args.speakers }),
|
|
131
141
|
]);
|
|
132
142
|
const { text, segments } = transcript;
|
|
133
143
|
|
package/src/engine-install.ts
CHANGED
|
@@ -35,36 +35,72 @@ function getEngineBinaryName(): string {
|
|
|
35
35
|
}
|
|
36
36
|
|
|
37
37
|
/**
|
|
38
|
-
*
|
|
39
|
-
*
|
|
40
|
-
*
|
|
41
|
-
*
|
|
42
|
-
* asset name.
|
|
43
|
-
*
|
|
44
|
-
* Best-effort: 404s (older engine versions predate the sidecar) and
|
|
45
|
-
* network errors log a warning and return — macos-* voices simply won't
|
|
46
|
-
* be available, which is a graceful degradation. The user keeps Kokoro +
|
|
47
|
-
* Vosk-TTS.
|
|
38
|
+
* One sidecar's identity. Each shipped Swift sidecar is described by an
|
|
39
|
+
* entry in `SIDECARS`; the helper below loops over them. Centralising the
|
|
40
|
+
* spec keeps the AVSpeech (#141) and diarize (#199) install paths in
|
|
41
|
+
* lockstep — adding a third sidecar is one entry, not a new function.
|
|
48
42
|
*/
|
|
49
|
-
|
|
43
|
+
interface SidecarSpec {
|
|
44
|
+
/** Filename written next to the engine binary. The Rust runtime probes
|
|
45
|
+
* this exact name (sometimes a list — see diarize::sidecar_path). */
|
|
46
|
+
fileBasename: string;
|
|
47
|
+
/** Release asset name. Often equals fileBasename, but AVSpeech writes
|
|
48
|
+
* `say-avspeech` while the asset is `say-avspeech-darwin-arm64`. */
|
|
49
|
+
assetName: string;
|
|
50
|
+
/** Human-readable name in log messages. */
|
|
51
|
+
displayName: string;
|
|
52
|
+
/** Trailing hint on success: "AVSpeech sidecar installed (<hint>)." */
|
|
53
|
+
availableHint: string;
|
|
54
|
+
/** Trailing hint on any failure path: "...; <hint>." */
|
|
55
|
+
unavailableHint: string;
|
|
56
|
+
}
|
|
57
|
+
|
|
58
|
+
const SIDECARS: SidecarSpec[] = [
|
|
59
|
+
{
|
|
60
|
+
fileBasename: "say-avspeech",
|
|
61
|
+
assetName: "say-avspeech-darwin-arm64",
|
|
62
|
+
displayName: "AVSpeech sidecar",
|
|
63
|
+
availableHint: "macOS voices available",
|
|
64
|
+
unavailableHint: "macos-* voices unavailable",
|
|
65
|
+
},
|
|
66
|
+
{
|
|
67
|
+
fileBasename: "kesha-diarize-darwin-arm64",
|
|
68
|
+
assetName: "kesha-diarize-darwin-arm64",
|
|
69
|
+
displayName: "Diarization sidecar",
|
|
70
|
+
availableHint: "--speakers available",
|
|
71
|
+
unavailableHint: "--speakers unavailable",
|
|
72
|
+
},
|
|
73
|
+
];
|
|
74
|
+
|
|
75
|
+
/**
|
|
76
|
+
* Fetch a single Swift sidecar and place it next to the engine binary on
|
|
77
|
+
* darwin-arm64. Best-effort: 404s (older engine versions predate this
|
|
78
|
+
* sidecar) and network errors log a warning and return — the corresponding
|
|
79
|
+
* feature simply won't be available. The user keeps everything else.
|
|
80
|
+
*/
|
|
81
|
+
async function downloadSidecar(
|
|
82
|
+
spec: SidecarSpec,
|
|
83
|
+
binPath: string,
|
|
84
|
+
engineVersion: string,
|
|
85
|
+
): Promise<void> {
|
|
50
86
|
if (process.platform !== "darwin" || process.arch !== "arm64") return;
|
|
51
87
|
|
|
52
|
-
const sidecarPath = join(dirname(binPath),
|
|
53
|
-
const url = `https://github.com/${GITHUB_REPO}/releases/download/v${engineVersion}
|
|
88
|
+
const sidecarPath = join(dirname(binPath), spec.fileBasename);
|
|
89
|
+
const url = `https://github.com/${GITHUB_REPO}/releases/download/v${engineVersion}/${spec.assetName}`;
|
|
54
90
|
|
|
55
91
|
let res: Response;
|
|
56
92
|
try {
|
|
57
93
|
res = await fetch(url, { redirect: "follow" });
|
|
58
94
|
} catch (e) {
|
|
59
95
|
log.warn(
|
|
60
|
-
`Could not fetch
|
|
96
|
+
`Could not fetch ${spec.displayName} (${e instanceof Error ? e.message : e}); ${spec.unavailableHint}.`,
|
|
61
97
|
);
|
|
62
98
|
return;
|
|
63
99
|
}
|
|
64
100
|
|
|
65
101
|
if (!res.ok) {
|
|
66
102
|
log.warn(
|
|
67
|
-
|
|
103
|
+
`${spec.displayName} not in release v${engineVersion} (HTTP ${res.status}); ${spec.unavailableHint}.`,
|
|
68
104
|
);
|
|
69
105
|
return;
|
|
70
106
|
}
|
|
@@ -72,17 +108,17 @@ async function downloadAVSpeechSidecar(binPath: string, engineVersion: string):
|
|
|
72
108
|
// Keep the best-effort contract: streamResponseToFile throws on an empty
|
|
73
109
|
// body and can fail mid-stream, and chmodSync can throw EPERM. Without
|
|
74
110
|
// this catch a stream/chmod failure would propagate through the tail
|
|
75
|
-
// `await
|
|
76
|
-
// engine install into a thrown exception after log.success
|
|
77
|
-
// announced it, which is exactly the regression the fetch/404
|
|
78
|
-
// above protect against.
|
|
111
|
+
// `await Promise.all(sidecarPromises)` in downloadEngine — converting a
|
|
112
|
+
// successful engine install into a thrown exception after log.success
|
|
113
|
+
// already announced it, which is exactly the regression the fetch/404
|
|
114
|
+
// branches above protect against.
|
|
79
115
|
try {
|
|
80
|
-
await streamResponseToFile(res, sidecarPath,
|
|
116
|
+
await streamResponseToFile(res, sidecarPath, spec.displayName);
|
|
81
117
|
chmodSync(sidecarPath, 0o755);
|
|
82
|
-
log.success(
|
|
118
|
+
log.success(`${spec.displayName} installed (${spec.availableHint}).`);
|
|
83
119
|
} catch (e) {
|
|
84
120
|
log.warn(
|
|
85
|
-
|
|
121
|
+
`${spec.displayName} install failed (${e instanceof Error ? e.message : e}); ${spec.unavailableHint}.`,
|
|
86
122
|
);
|
|
87
123
|
}
|
|
88
124
|
}
|
|
@@ -92,6 +128,9 @@ export interface InstallOptions {
|
|
|
92
128
|
tts?: boolean;
|
|
93
129
|
/** Also install Silero VAD model for long-audio preprocessing. */
|
|
94
130
|
vad?: boolean;
|
|
131
|
+
/** Also install the Sortformer streaming-diarization model (~245MB,
|
|
132
|
+
* darwin-arm64 only — see #199). */
|
|
133
|
+
diarize?: boolean;
|
|
95
134
|
}
|
|
96
135
|
|
|
97
136
|
export async function downloadEngine(
|
|
@@ -117,13 +156,14 @@ export async function downloadEngine(
|
|
|
117
156
|
|
|
118
157
|
if (cacheValid) {
|
|
119
158
|
log.success(`Engine binary already installed (v${engineVersion}).`);
|
|
120
|
-
//
|
|
121
|
-
//
|
|
122
|
-
//
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
159
|
+
// Top up any sidecars missing from this cached install. Pre-#141 / pre-#199
|
|
160
|
+
// engines never shipped them, so a cache-valid binary may still need
|
|
161
|
+
// fetching. Run independent fetches concurrently — same shape as the
|
|
162
|
+
// cold path below.
|
|
163
|
+
const missing = SIDECARS.filter(
|
|
164
|
+
(s) => !existsSync(join(dirname(binPath), s.fileBasename)),
|
|
165
|
+
);
|
|
166
|
+
await Promise.all(missing.map((s) => downloadSidecar(s, binPath, engineVersion)));
|
|
127
167
|
} else {
|
|
128
168
|
// Log why we're downloading — helps diagnose surprising re-downloads.
|
|
129
169
|
if (existsSync(binPath) && installedVersion && installedVersion !== engineVersion) {
|
|
@@ -136,32 +176,34 @@ export async function downloadEngine(
|
|
|
136
176
|
|
|
137
177
|
mkdirSync(dirname(binPath), { recursive: true });
|
|
138
178
|
|
|
139
|
-
// Kick off
|
|
140
|
-
// target github.com release assets
|
|
141
|
-
//
|
|
142
|
-
//
|
|
143
|
-
//
|
|
144
|
-
const
|
|
179
|
+
// Kick off all sidecar fetches concurrently with the engine fetch. They
|
|
180
|
+
// target independent github.com release assets, so overlapping the HTTP
|
|
181
|
+
// round-trips saves ~15-30s on a cold install. Each sidecar is
|
|
182
|
+
// best-effort (404 on older engines, warn + continue) so a failure
|
|
183
|
+
// doesn't cascade into the engine path.
|
|
184
|
+
const sidecarPromises = SIDECARS.map((s) =>
|
|
185
|
+
downloadSidecar(s, binPath, engineVersion),
|
|
186
|
+
);
|
|
187
|
+
// Defense-in-depth: if the engine fetch throws below, attach no-op
|
|
188
|
+
// rejection handlers so we don't surface unhandledRejection errors
|
|
189
|
+
// from sidecar paths whose internal try/catch ever drifts. Logs from
|
|
190
|
+
// sidecars whose own work is still in flight will print when they
|
|
191
|
+
// complete; the engine error is what the user needs to see now.
|
|
192
|
+
const muteSidecarRejections = () =>
|
|
193
|
+
sidecarPromises.forEach((p) => p.catch(() => {}));
|
|
145
194
|
|
|
146
195
|
let res: Response;
|
|
147
196
|
try {
|
|
148
197
|
res = await fetch(url, { redirect: "follow" });
|
|
149
198
|
} catch (e) {
|
|
150
|
-
|
|
151
|
-
// defense-in-depth. Today it can't reject (downloadAVSpeechSidecar
|
|
152
|
-
// catches all of its own errors), but if that contract ever drifts
|
|
153
|
-
// a bare dangling promise would surface as an unhandledRejection
|
|
154
|
-
// while we're throwing the engine error. Not waiting — the engine
|
|
155
|
-
// error is what the user needs to see now; sidecar's own logs
|
|
156
|
-
// will print whenever they finish.
|
|
157
|
-
sidecarPromise.catch(() => {});
|
|
199
|
+
muteSidecarRejections();
|
|
158
200
|
throw new Error(
|
|
159
201
|
`Failed to fetch engine binary: ${e instanceof Error ? e.message : e}\n Fix: Check your network connection and try again`,
|
|
160
202
|
);
|
|
161
203
|
}
|
|
162
204
|
|
|
163
205
|
if (!res.ok) {
|
|
164
|
-
|
|
206
|
+
muteSidecarRejections();
|
|
165
207
|
throw new Error(
|
|
166
208
|
`Failed to download engine binary (HTTP ${res.status})\n Fix: Check https://github.com/${GITHUB_REPO}/releases for available versions`,
|
|
167
209
|
);
|
|
@@ -171,7 +213,7 @@ export async function downloadEngine(
|
|
|
171
213
|
chmodSync(binPath, 0o755);
|
|
172
214
|
writeInstalledEngineVersion(binPath, engineVersion);
|
|
173
215
|
log.success(`Engine binary downloaded (v${engineVersion}).`);
|
|
174
|
-
await
|
|
216
|
+
await Promise.all(sidecarPromises);
|
|
175
217
|
}
|
|
176
218
|
|
|
177
219
|
if (backend) {
|
|
@@ -189,6 +231,7 @@ export async function downloadEngine(
|
|
|
189
231
|
...(noCache ? ["--no-cache"] : []),
|
|
190
232
|
...(options.tts ? ["--tts"] : []),
|
|
191
233
|
...(options.vad ? ["--vad"] : []),
|
|
234
|
+
...(options.diarize ? ["--diarize"] : []),
|
|
192
235
|
];
|
|
193
236
|
const proc = Bun.spawnSync([binPath, ...installArgs], {
|
|
194
237
|
stdout: "pipe",
|
package/src/engine.ts
CHANGED
|
@@ -1,8 +1,22 @@
|
|
|
1
1
|
import { join } from "path";
|
|
2
2
|
import { homedir } from "os";
|
|
3
|
-
import { existsSync } from "fs";
|
|
3
|
+
import { existsSync, statSync } from "fs";
|
|
4
4
|
import { log } from "./log";
|
|
5
5
|
|
|
6
|
+
/**
|
|
7
|
+
* Capability-flag string surfaced via `kesha-engine --capabilities-json`. Single
|
|
8
|
+
* source of truth so the engine, the TS CLI gate, and the integration tests
|
|
9
|
+
* can't drift. Mirrors `rust/src/transcribe/mod.rs::TRANSCRIBE_SEGMENTS_FEATURE`.
|
|
10
|
+
*/
|
|
11
|
+
export const TRANSCRIBE_SEGMENTS_FEATURE = "transcribe.segments";
|
|
12
|
+
|
|
13
|
+
/**
|
|
14
|
+
* Capability-flag string for speaker diarization. Engine advertises this only
|
|
15
|
+
* on darwin-arm64 builds with the `system_diarize` cargo feature (#199).
|
|
16
|
+
* Mirrors `rust/src/transcribe/mod.rs::TRANSCRIBE_DIARIZE_FEATURE`.
|
|
17
|
+
*/
|
|
18
|
+
export const TRANSCRIBE_DIARIZE_FEATURE = "transcribe.diarize";
|
|
19
|
+
|
|
6
20
|
export interface LangDetectResult {
|
|
7
21
|
code: string;
|
|
8
22
|
confidence: number;
|
|
@@ -12,6 +26,8 @@ export interface TranscriptionSegment {
|
|
|
12
26
|
start: number;
|
|
13
27
|
end: number;
|
|
14
28
|
text: string;
|
|
29
|
+
/** Speaker cluster id when `--speakers` was requested (#199). */
|
|
30
|
+
speaker?: number;
|
|
15
31
|
}
|
|
16
32
|
|
|
17
33
|
export interface TranscriptionOutput {
|
|
@@ -62,6 +78,9 @@ export type VadMode = "auto" | "on" | "off";
|
|
|
62
78
|
|
|
63
79
|
export interface TranscribeEngineOptions {
|
|
64
80
|
vad?: VadMode;
|
|
81
|
+
/** Request speaker labels in transcript segments. Requires the engine to
|
|
82
|
+
* advertise `transcribe.diarize` (darwin-arm64 only — see #199). */
|
|
83
|
+
speakers?: boolean;
|
|
65
84
|
}
|
|
66
85
|
|
|
67
86
|
export async function transcribeEngine(
|
|
@@ -93,11 +112,9 @@ function parseTranscriptionOutput(stdout: string): TranscriptionOutput {
|
|
|
93
112
|
) {
|
|
94
113
|
throw new Error("Invalid transcription segment returned by kesha-engine");
|
|
95
114
|
}
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
text: s.text,
|
|
100
|
-
};
|
|
115
|
+
const out: TranscriptionSegment = { start: s.start, end: s.end, text: s.text };
|
|
116
|
+
if (typeof s.speaker === "number") out.speaker = s.speaker;
|
|
117
|
+
return out;
|
|
101
118
|
});
|
|
102
119
|
|
|
103
120
|
return { text: parsed.text, segments };
|
|
@@ -108,7 +125,7 @@ export async function transcribeEngineWithSegments(
|
|
|
108
125
|
opts: TranscribeEngineOptions = {},
|
|
109
126
|
): Promise<TranscriptionOutput> {
|
|
110
127
|
const caps = await getEngineCapabilities();
|
|
111
|
-
if (!caps?.features.includes(
|
|
128
|
+
if (!caps?.features.includes(TRANSCRIBE_SEGMENTS_FEATURE)) {
|
|
112
129
|
throw new Error(
|
|
113
130
|
"Timestamped segments require a newer kesha-engine. Run `kesha install` after upgrading Kesha Voice Kit.",
|
|
114
131
|
);
|
|
@@ -117,6 +134,15 @@ export async function transcribeEngineWithSegments(
|
|
|
117
134
|
const args = ["transcribe", audioPath, "--json"];
|
|
118
135
|
if (opts.vad === "on") args.push("--vad");
|
|
119
136
|
else if (opts.vad === "off") args.push("--no-vad");
|
|
137
|
+
if (opts.speakers) {
|
|
138
|
+
if (!caps.features.includes(TRANSCRIBE_DIARIZE_FEATURE)) {
|
|
139
|
+
throw new Error(
|
|
140
|
+
"speaker diarization is currently darwin-arm64 only " +
|
|
141
|
+
"(see https://github.com/drakulavich/kesha-voice-kit/issues/199)",
|
|
142
|
+
);
|
|
143
|
+
}
|
|
144
|
+
args.push("--speakers");
|
|
145
|
+
}
|
|
120
146
|
const { stdout, stderr, exitCode } = await runEngine(args);
|
|
121
147
|
if (exitCode !== 0) {
|
|
122
148
|
throw new Error(stderr || `kesha-engine exited with code ${exitCode}`);
|
|
@@ -162,20 +188,33 @@ export interface EngineCapabilities {
|
|
|
162
188
|
}
|
|
163
189
|
|
|
164
190
|
let cachedEngineCapabilities:
|
|
165
|
-
| { binPath: string; capabilities: EngineCapabilities }
|
|
191
|
+
| { binPath: string; mtime: number; capabilities: EngineCapabilities }
|
|
166
192
|
| null = null;
|
|
167
193
|
|
|
168
194
|
export async function getEngineCapabilities(): Promise<EngineCapabilities | null> {
|
|
169
195
|
const binPath = getEngineBinPath();
|
|
170
|
-
|
|
196
|
+
// Cache key includes `mtimeMs` so the cache invalidates when `kesha
|
|
197
|
+
// install` overwrites the binary in-place within a single long-lived
|
|
198
|
+
// process (#248). `statSync` throws on missing-file; the catch returns
|
|
199
|
+
// `null` — same effect as the previous explicit `isEngineInstalled()`
|
|
200
|
+
// pre-flight, one fewer redundant fs call.
|
|
201
|
+
let mtime: number;
|
|
202
|
+
try {
|
|
203
|
+
mtime = statSync(binPath).mtimeMs;
|
|
204
|
+
} catch {
|
|
205
|
+
return null;
|
|
206
|
+
}
|
|
207
|
+
if (
|
|
208
|
+
cachedEngineCapabilities?.binPath === binPath &&
|
|
209
|
+
cachedEngineCapabilities.mtime === mtime
|
|
210
|
+
) {
|
|
171
211
|
return cachedEngineCapabilities.capabilities;
|
|
172
212
|
}
|
|
173
|
-
if (!existsSync(binPath)) return null;
|
|
174
213
|
const { stdout, exitCode } = await runEngine(["--capabilities-json"]);
|
|
175
214
|
if (exitCode !== 0) return null;
|
|
176
215
|
try {
|
|
177
216
|
const capabilities = JSON.parse(stdout) as EngineCapabilities;
|
|
178
|
-
cachedEngineCapabilities = { binPath, capabilities };
|
|
217
|
+
cachedEngineCapabilities = { binPath, mtime, capabilities };
|
|
179
218
|
return capabilities;
|
|
180
219
|
} catch {
|
|
181
220
|
return null;
|
package/src/lib.ts
CHANGED
|
@@ -44,7 +44,7 @@ export async function transcribe(
|
|
|
44
44
|
return internalTranscribe(audioPath, { ...options, silent: true });
|
|
45
45
|
}
|
|
46
46
|
|
|
47
|
-
export async function
|
|
47
|
+
export async function transcribeWithTimestamps(
|
|
48
48
|
audioPath: string,
|
|
49
49
|
options: TranscribeOptions = {},
|
|
50
50
|
) {
|
|
@@ -58,3 +58,10 @@ export async function transcribeWithSegments(
|
|
|
58
58
|
silent: true,
|
|
59
59
|
});
|
|
60
60
|
}
|
|
61
|
+
|
|
62
|
+
/**
|
|
63
|
+
* @deprecated Renamed to {@link transcribeWithTimestamps} (#248). The old
|
|
64
|
+
* name shipped briefly in v1.9.0; this alias keeps existing imports working
|
|
65
|
+
* for one minor-version cycle. Will be removed in v1.12.0.
|
|
66
|
+
*/
|
|
67
|
+
export const transcribeWithSegments = transcribeWithTimestamps;
|
package/src/star.ts
CHANGED
|
@@ -55,3 +55,79 @@ export function shouldShowStarPrompt(current: string, seen: string | null): bool
|
|
|
55
55
|
export function hasStarMarker(binPath: string): boolean {
|
|
56
56
|
return existsSync(starSeenPath(binPath));
|
|
57
57
|
}
|
|
58
|
+
|
|
59
|
+
/**
|
|
60
|
+
* Prompt the user to star the repo if and only if `shouldShowStarPrompt`
|
|
61
|
+
* agrees (first install + major-or-minor bumps, never on patch). Records
|
|
62
|
+
* the prompt against the current version up front so a single run never
|
|
63
|
+
* prompts twice — failures from the gh subprocess below don't reopen the
|
|
64
|
+
* gate. Shared by `kesha install` and `kesha status` so opt-in installs
|
|
65
|
+
* (`--tts`, `--diarize`) and `status` reuse the same marker; a user who
|
|
66
|
+
* saw the prompt on the base install won't see it again on the opt-in or
|
|
67
|
+
* the status check.
|
|
68
|
+
*
|
|
69
|
+
* No-ops when the gate says skip, when `currentVersion` is null, or when
|
|
70
|
+
* the user has already starred. When `gh` is missing or unauthenticated,
|
|
71
|
+
* a basic prompt is still printed (so the marker write isn't a silent
|
|
72
|
+
* slot consumption).
|
|
73
|
+
*
|
|
74
|
+
* `shims` lets tests inject deterministic `which` / `spawn` implementations.
|
|
75
|
+
* Production callers leave it undefined; the defaults route through Bun's
|
|
76
|
+
* built-ins. Bun.which() caches PATH at process start so swapping
|
|
77
|
+
* process.env.PATH in tests doesn't work — the shim is the supported way
|
|
78
|
+
* to fake gh's presence and authentication state from a unit test.
|
|
79
|
+
*/
|
|
80
|
+
export async function maybeAskForStar(
|
|
81
|
+
binPath: string,
|
|
82
|
+
currentVersion: string | null,
|
|
83
|
+
log: { info: (msg: string) => void },
|
|
84
|
+
shims?: {
|
|
85
|
+
which?: (name: string) => string | null;
|
|
86
|
+
spawn?: (cmd: string[]) => { exitCode: number | null };
|
|
87
|
+
},
|
|
88
|
+
): Promise<void> {
|
|
89
|
+
const which = shims?.which ?? ((n: string) => Bun.which(n));
|
|
90
|
+
const spawn =
|
|
91
|
+
shims?.spawn ??
|
|
92
|
+
((cmd: string[]) =>
|
|
93
|
+
Bun.spawnSync(cmd, { stdout: "ignore", stderr: "ignore" }));
|
|
94
|
+
if (!currentVersion) return;
|
|
95
|
+
const seen = readStarSeen(binPath);
|
|
96
|
+
if (!shouldShowStarPrompt(currentVersion, seen)) {
|
|
97
|
+
return;
|
|
98
|
+
}
|
|
99
|
+
try {
|
|
100
|
+
writeStarSeen(binPath, currentVersion);
|
|
101
|
+
} catch {
|
|
102
|
+
/* Non-fatal — falling through to the prompt is still OK. */
|
|
103
|
+
}
|
|
104
|
+
|
|
105
|
+
// The marker has been recorded — every path from here that reaches
|
|
106
|
+
// a `return` without printing would silently consume the user's
|
|
107
|
+
// once-per-major.minor slot. Only the "already starred" path is
|
|
108
|
+
// allowed to skip the print, since the consumption is the verification
|
|
109
|
+
// that they're already supporting the project.
|
|
110
|
+
const printBasicPrompt = () => {
|
|
111
|
+
log.info("\nIf you enjoy Kesha Voice Kit, consider starring the repo:");
|
|
112
|
+
log.info(" https://github.com/drakulavich/kesha-voice-kit");
|
|
113
|
+
};
|
|
114
|
+
|
|
115
|
+
const gh = which("gh");
|
|
116
|
+
if (!gh) {
|
|
117
|
+
printBasicPrompt();
|
|
118
|
+
return;
|
|
119
|
+
}
|
|
120
|
+
const authCheck = spawn([gh, "auth", "status"]);
|
|
121
|
+
if (authCheck.exitCode !== 0) {
|
|
122
|
+
// gh installed but unauthenticated — we can't check star status, but
|
|
123
|
+
// we can still nudge with the same basic prompt the no-gh path uses.
|
|
124
|
+
// Without this, the marker write above silently consumes the slot.
|
|
125
|
+
printBasicPrompt();
|
|
126
|
+
return;
|
|
127
|
+
}
|
|
128
|
+
const starred = spawn([gh, "api", "user/starred/drakulavich/kesha-voice-kit"]);
|
|
129
|
+
if (starred.exitCode === 0) return; // already starred — slot consumed by verification
|
|
130
|
+
log.info("\n⭐ If you enjoy Kesha Voice Kit, star it on GitHub:");
|
|
131
|
+
log.info(" https://github.com/drakulavich/kesha-voice-kit");
|
|
132
|
+
log.info(' Or run: gh api -X PUT /user/starred/drakulavich/kesha-voice-kit');
|
|
133
|
+
}
|
package/src/status.ts
CHANGED
|
@@ -3,8 +3,11 @@ import { homedir } from "os";
|
|
|
3
3
|
import { join } from "path";
|
|
4
4
|
import { isEngineInstalled, getEngineBinPath, getEngineCapabilities } from "./engine";
|
|
5
5
|
import { log } from "./log";
|
|
6
|
+
import { maybeAskForStar } from "./star";
|
|
6
7
|
import pc from "picocolors";
|
|
7
8
|
|
|
9
|
+
const pkg = await Bun.file(new URL("../package.json", import.meta.url)).json();
|
|
10
|
+
|
|
8
11
|
function humanBytes(bytes: number): string {
|
|
9
12
|
if (bytes < 1024) return `${bytes} B`;
|
|
10
13
|
const units = ["KB", "MB", "GB"];
|
|
@@ -86,7 +89,16 @@ export async function showStatus(): Promise<void> {
|
|
|
86
89
|
|
|
87
90
|
if (!installed) {
|
|
88
91
|
log.warn('Run "kesha install" to download the engine and models.');
|
|
92
|
+
return;
|
|
89
93
|
}
|
|
94
|
+
|
|
95
|
+
// Star prompt at the very end. Same gate + marker as `kesha install`, so a
|
|
96
|
+
// user prompted by either path won't be re-prompted by the other within the
|
|
97
|
+
// same major.minor. Headless `kesha install` (no `gh`, or no tty) leaves no
|
|
98
|
+
// marker only when the prompt path itself errored out — readStarSeen still
|
|
99
|
+
// sees the marker in the common case.
|
|
100
|
+
const currentVersion = typeof pkg.version === "string" ? pkg.version : null;
|
|
101
|
+
await maybeAskForStar(binPath, currentVersion, log);
|
|
90
102
|
}
|
|
91
103
|
|
|
92
104
|
function showDiskUsage(binPath: string): void {
|
package/src/transcribe.ts
CHANGED
|
@@ -15,6 +15,10 @@ export interface TranscribeOptions {
|
|
|
15
15
|
vad?: VadMode;
|
|
16
16
|
/** Request timestamped transcript segments from the engine. */
|
|
17
17
|
timestamps?: boolean;
|
|
18
|
+
/** Request speaker labels in transcript segments (#199). Implies `timestamps`.
|
|
19
|
+
* Currently darwin-arm64 only — throws when the engine doesn't advertise
|
|
20
|
+
* `transcribe.diarize`. */
|
|
21
|
+
speakers?: boolean;
|
|
18
22
|
}
|
|
19
23
|
|
|
20
24
|
export async function transcribe(audioPath: string, opts: TranscribeOptions = {}): Promise<string> {
|
|
@@ -36,8 +40,11 @@ export async function transcribeWithSegments(
|
|
|
36
40
|
);
|
|
37
41
|
}
|
|
38
42
|
|
|
39
|
-
if (opts.timestamps) {
|
|
40
|
-
return transcribeEngineWithSegments(audioPath, {
|
|
43
|
+
if (opts.timestamps || opts.speakers) {
|
|
44
|
+
return transcribeEngineWithSegments(audioPath, {
|
|
45
|
+
vad: opts.vad,
|
|
46
|
+
speakers: opts.speakers,
|
|
47
|
+
});
|
|
41
48
|
}
|
|
42
49
|
|
|
43
50
|
const text = await transcribeEngine(audioPath, { vad: opts.vad });
|