echogarden 0.12.2 → 1.0.0

package/docs/Engines.md

@@ -1,10 +1,11 @@
 
 # Supported engines
 
-## Text to speech
+## Text-to-speech
 
 **Offline**:
-* [VITS](https://github.com/jaywalnut310/vits) (`vits`): a high-quality end-to-end neural speech synthesis architecture. Available models were trained by Michael Hansen as part of his [Piper speech synthesis system](https://github.com/rhasspy/piper). Currently there are 117 voices, in a range of languages, including English (US, UK), Spanish (ES, MX), Portuguese (PT, BR), Italian, French, German, Dutch (NL, BE), Swedish, Norwegian, Danish, Finnish, Polish, Greek, Romanian, Serbian, Czech, Hungarian, Slovak, Slovenian, Turkish, Arabic, Farsi, Russian, Ukrainian, Catalan, Luxemburgish, Icelandic, Swahili, Kazakh, Georgian, Nepali, Vietnamese and Chinese. You can listen to audio samples of all voices and languages in [Piper's samples page](https://rhasspy.github.io/piper-samples/).
+
+* [VITS](https://github.com/jaywalnut310/vits) (`vits`): a high-quality end-to-end neural speech synthesis architecture. Available models were trained by Michael Hansen as part of his [Piper speech synthesis system](https://github.com/rhasspy/piper). Currently, there are 117 voices, in a range of languages, including English (US, UK), Spanish (ES, MX), Portuguese (PT, BR), Italian, French, German, Dutch (NL, BE), Swedish, Norwegian, Danish, Finnish, Polish, Greek, Romanian, Serbian, Czech, Hungarian, Slovak, Slovenian, Turkish, Arabic, Farsi, Russian, Ukrainian, Catalan, Luxembourgish, Icelandic, Swahili, Kazakh, Georgian, Nepali, Vietnamese and Chinese. You can listen to audio samples of all voices and languages in [Piper's samples page](https://rhasspy.github.io/piper-samples/).
 * [SVOX Pico](https://github.com/naggety/picotts) (`pico`): a legacy diphone-based synthesis engine. Supports English (US, UK), Spanish, Italian, French, and German.
 * [Flite](https://github.com/festvox/flite) (`flite`): a legacy diphone-based synthesis engine. Supports English (US, Scottish), and several Indic languages: Hindi, Bengali, Marathi, Telugu, Tamil, Gujarati, Kannada and Punjabi.
 * [eSpeak-NG](https://github.com/espeak-ng/espeak-ng/) (`espeak`): a lightweight "robot" sounding formant-based synthesizer. Supports 100+ languages. Extensively used internally for speech alignment, phonemization, and other internal tasks.
@@ -16,11 +17,12 @@
 
 * [Microsoft Speech Platform](https://www.microsoft.com/en-us/download/details.aspx?id=27225) (`msspeech`): Microsoft Server Speech API. Requires [installing a runtime (2.6MB)](https://www.microsoft.com/en-us/download/details.aspx?id=27225). Supports 28 dialects, which can be individually downloaded via [freely available installers](https://www.microsoft.com/en-us/download/details.aspx?id=27224), or, for convenience, bundled as [a single 358MB zip file](https://drive.google.com/u/0/uc?id=1uQdFNxLzUxpaEwVVKhMawys8cIh3F21T&export=download). Has voices for English (US, UK, AU, CA), Spanish (ES, MX), Portuguese (BR, PT), German, French (FR, CA), Italian, Norwegian, Dutch, Russian, Swedish, Danish, Catalan, Finnish, Japanese, Korean and Chinese (ZH, HK, TW). All voices are female.
 
-_Note_: both these engines require manually installing the [`winax` npm package](https://www.npmjs.com/package/winax) by running `npm install winax -g`.
+**Note**: both these engines require manually installing the [`winax` npm package](https://www.npmjs.com/package/winax) by running `npm install winax -g`.
+
+`winax` is a native module which requires the Node.js Windows build tools to successfully install. If you have issues installing this package, please ensure that you've checked the "install necessary tools" checkbox during the installation of Node.js.
 
-`winax` is a native module which requires the Node.js Windows build tools to successfully install. If you have issues installing this package, please ensure that you've checked the "install necessary tools" checkbox during the installation of node.js.
+**Client for self-hosted or remote servers**:
 
-**Client for remote or self-hosted servers**:
 * [Coqui TTS](https://github.com/coqui-ai/TTS) server (`coqui-server`)
 
 **Cloud services**:
@@ -30,20 +32,22 @@ These are commercial services that require a subscription and an API key to use:
 * [Google Cloud](https://cloud.google.com/text-to-speech) (`google-cloud`)
 * [Azure Cognitive Services](https://azure.microsoft.com/en-us/products/cognitive-services/text-to-speech/) (`microsoft-azure`)
 * [Amazon Polly](https://aws.amazon.com/polly/) (`amazon-polly`)
+* [OpenAI Cloud](https://platform.openai.com/) (`openai-cloud`)
 * [Elevenlabs](https://elevenlabs.io/) (`elevenlabs`)
 
 **Cloud services (unofficial)**:
 
-These cloud-based engines connect to public-facing cloud APIs that are not officially publicized by their operators. They are included for educational purposes only, and may be removed in the future:
+These cloud-based engines connect to public cloud APIs that are not officially publicized by their operators. They are included for educational purposes only, and may be removed in the future:
 
 * Google Translate (`google-translate`): used by the [Google Translate web UI](https://translate.google.com/) to speak written text in any one of its supported languages. Offers a single voice for each language (usually female).
 * Microsoft Edge (`microsoft-edge`): subset of the Azure Cognitive Services cloud TTS API used by the Microsoft Edge browser as part of its support for the [Web Speech API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API) and its [Read Aloud](https://www.microsoft.com/en-us/edge/features/read-aloud?form=MT00D8) feature. Using this engine requires a special token, which should be passed via the `microsoftEdge.trustedClientToken` option.
-* Streamlabs Polly (`streamlabs-polly`): a public REST API by Streamlabs, primarily intended for generating speech for TTS donations. It includes a few English (US, UK, AU, IN) voices, which are similar to some of the non-neural (Ivona-based) voices offered by Amazon Polly.
+* Streamlabs Polly (`streamlabs-polly`): a public REST API by Streamlabs, primarily intended for generating speech for TTS donations. It includes a few English (US, UK, AU, IN) voices, which are similar to some of the non-neural (Ivona-based) voices offered by Amazon Polly (**Note**: as of April 2024, the public Streamlabs Polly REST API doesn't seem to be accessible anymore).
 
-## Speech to text
+## Speech-to-text
 
 **Offline**:
-* [OpenAI Whisper](https://github.com/openai/whisper) (`whisper`): high accuracy transformer-based architecture. Supports 99 languages. There are several models of different sizes, some are multilingual, and some are English only (`.en`): `tiny`, `tiny.en`, `base`, `base.en`, `small`, `small.en`, `medium`, `medium.en`, `large`, `large-v1` and `large-v2`. _Note_: large models are not currently auto-downloaded as Echogarden packages, but may become available in the future.
+* [OpenAI Whisper](https://github.com/openai/whisper) (`whisper`): high accuracy transformer-based speech recognition architecture. Supports 99 languages. There are several models of different sizes, some are multilingual, and some are English only (`.en`): `tiny`, `tiny.en`, `base`, `base.en`, `small`, `small.en`, `medium`, `medium.en`, `large`, `large-v1` and `large-v2`, `large-v3`. **Note**: large models are not currently supported by `onnxruntime-node` due to model size restrictions.
+* [Whisper.cpp](https://github.com/ggerganov/whisper.cpp) (`whisper.cpp`): a port of the Whisper architecture to C++, by Georgi Gerganov. Supports all Whisper models, including several quantized ones (see full list in the options page). Has various different builds, including CUDA and OpenCL for GPU support.
 * [Vosk](https://github.com/alphacep/vosk-api) (`vosk`): models available for 25+ languages. _Note_: the Vosk package is not included in the default installation, but you can add support for it using `npm install @echogarden/vosk -g`. Then, you'll need to manually [download a model](https://alphacephei.com/vosk/models) and specify its directory path via the `vosk.modelPath` option.
 * [Silero](https://github.com/snakers4/silero-models) (`silero`): models available for English, Spanish, German and Ukrainian. For [non-commercial use only](https://github.com/snakers4/silero-models/blob/master/LICENSE).
 
@@ -54,24 +58,27 @@ These are commercial services that require a subscription and an API key to use:
 * [Google Cloud](https://cloud.google.com/speech-to-text) (`google-cloud`)
 * [Azure Cognitive Services](https://azure.microsoft.com/en-us/products/cognitive-services/speech-to-text/) (`microsoft-azure`)
 * [Amazon Transcribe](https://aws.amazon.com/transcribe/) (`amazon-transcribe`)
+* [OpenAI Cloud](https://platform.openai.com/) (`openai-cloud`)
 
-## Speech to transcript alignment
+## Speech-to-transcript alignment
 
 These engines' goal is to match (or "align") a given spoken recording with a given transcript as closely as possible. They will annotate each word in the transcript with approximate start and end timestamps:
 
-* Dynamic Time Warping (`dtw`): transcript is first synthesized using the eSpeak engine, then [DTW](https://en.wikipedia.org/wiki/Dynamic_time_warping) is applied to find the best mapping between the synthesized audio and the original audio.
-* Dynamic Time Warping with Recognition Assist (`dtw-ra`): recognition is applied to the audio (any recognition engine can be used), then both the ground-truth transcript and the recognized transcript are synthesized using eSpeak. Then, the best mapping is found between the two synthesized audio sequences, and the result is mapped back to the original audio using the timing information produced by the recognizer.
-* Whisper-based alignment (`whisper`): transcript is tokenized and decoded along with the audio using the Whisper model, then timestamps are extracted from the internal state of the model (_note_: currently, only supports audio inputs that are 30 seconds or less).
+* Dynamic Time Warping (`dtw`): transcript is first synthesized using the eSpeak engine, then the [DTW](https://en.wikipedia.org/wiki/Dynamic_time_warping) alignment algorithm is applied to find the best mapping between the synthesized and original audio frames.
+* Dynamic Time Warping with Recognition Assist (`dtw-ra`): recognition is applied to the audio (any recognition engine can be used), then both the ground-truth transcript and the recognized transcript are synthesized using eSpeak. Then, the best mapping is found between the two synthesized waveforms, and the result is mapped back to the original audio using the timing information produced by the recognizer.
+* Whisper-based alignment (`whisper`): transcript is tokenized and decoded along with the audio using the Whisper model, then timestamps are extracted from the internal state of the model. **Note**: currently, only supports audio inputs that are 30 seconds or less. Some words or special characters may fail to tokenize due to limitations of the tokenizer used by Whisper.
 
-## Speech translation
+## Speech-to-text translation
 
-* [Whisper](https://github.com/openai/whisper) (`whisper`): the Whisper model can transcribe speech in any one of its supported languages and output a transcript directly translated to English. Other languages are not supported as targets.
+* [Whisper](https://github.com/openai/whisper) (`whisper`): the Whisper model can recognize speech in any one of its supported languages and output a transcript directly translated to English. Other languages are not supported as targets.
+* [Whisper.cpp](https://github.com/ggerganov/whisper.cpp) (`whisper.cpp`): supports translation to English
+* [OpenAI Cloud](https://platform.openai.com/) (`openai-cloud`): OpenAI cloud service. Only translates to English.
 
 ## Language detection
 
 **Spoken language detection**:
 * [Silero Language Classifier](https://github.com/snakers4/silero-vad/wiki/Other-Models) (`silero`): a speech language classification model by Silero.
-* [Whisper](https://github.com/openai/whisper) (`whisper`): uses the language token produced by the `whisper` speech recognition model to generate a set of probabilities for the 99 languages it has been trained on (_note_: currently only uses the first 30 seconds of the audio).
+* [Whisper](https://github.com/openai/whisper) (`whisper`): uses the language token produced by the `whisper` speech recognition model to generate a set of probabilities for the 99 languages it has been trained on.
 
 **Text language detection**:
 * [TinyLD](https://www.npmjs.com/package/tinyld) (`tinyld`): a simple language detection library.
@@ -82,7 +89,12 @@ These engines' goal is to match (or "align") a given spoken recording with a giv
 * [WebRTC VAD](https://github.com/dpirch/libfvad) (`webrtc`): a voice activity detector. Originally from the Chromium browser source code.
 * [Silero VAD](https://github.com/snakers4/silero-vad) (`silero`): a voice activity detection model by Silero.
 * [RNNoise](https://github.com/xiph/rnnoise) (`rnnoise`): uses RNNoise's speech probabilities output for each audio frame as a VAD metric.
+* Adaptive Gate (`adaptive-gate`): uses a band-limited adaptive gate to identify activity in the lower voice frequencies. Reliable, but will often pass non-vocal sounds if they are loud enough. Good for clean speech and a cappella singing, where most non-vocal segments are quiet.
 
 ## Speech denoising
 
 * [RNNoise](https://github.com/xiph/rnnoise) (`rnnoise`): a noise suppression library based on a recurrent neural network.
+
+## Source separation
+
+* [MDX-NET](https://github.com/kuielab/mdx-net) (`mdx-net`): Deep learning source separation architecture by [KUIELAB (Korea University)](https://kuielab.github.io/).

package/docs/Licenses.md

@@ -3,6 +3,7 @@
 ## Engines and libraries:
 
 * `onnxruntime-node`: [MIT License](https://github.com/microsoft/onnxruntime/blob/main/LICENSE)
+* `whisper.cpp`: [MIT License](https://github.com/ggerganov/whisper.cpp/blob/master/LICENSE)
 * `espeak`: [GNU GPL v3](https://github.com/espeak-ng/espeak-ng/blob/master/COPYING)
 * `flite`: [BSD License](https://github.com/festvox/flite/blob/master/COPYING)
 * `pico`: [Apache License 2.0](https://github.com/gmn/nanotts/blob/master/LICENSE)
@@ -32,7 +33,9 @@ All are freely distributable, with varying licenses:
 * Silero speech recognition models (`silero-en-`, `silero-de-`, `silero-es-`, `silero-ua-`): [BY-NC-SA](https://github.com/snakers4/silero-models/blob/master/LICENSE)
 * VITS pre-trained models (`vits-`): licensed under various creative commons licenses: [CC0](https://creativecommons.org/share-your-work/public-domain/cc0/), [CC-BY](https://creativecommons.org/licenses/by/4.0/) and [BY-NC-SA](https://creativecommons.org/licenses/by-nc-sa/4.0/), and few are public domain (you can view the individual license for each model in the model cards on the [samples page](https://rhasspy.github.io/piper-samples/)). The [Piper system](https://github.com/rhasspy/piper) itself is published under the [MIT License](https://github.com/rhasspy/piper/blob/master/LICENSE.md)
 * Whisper pre-trained models (`whisper-`): [MIT License](https://github.com/openai/whisper/blob/main/LICENSE)
+* MDX-NET source separation models (`mdxnet-`): [MIT License](https://github.com/kuielab/mdx-net/blob/main/LICENSE)
 
 Tool binary distributions:
-* FFMpeg: [LGPL, GPL v2 and GPL v3 Licenses](https://github.com/FFmpeg/FFmpeg)
+* FFmpeg: [LGPL, GPL v2 and GPL v3 Licenses](https://github.com/FFmpeg/FFmpeg)
 * SoX: [GPL v2 License](https://github.com/chirlu/sox/blob/master/LICENSE.GPL)
+* whisper.cpp: [MIT License](https://github.com/ggerganov/whisper.cpp/blob/master/LICENSE)

package/docs/Options.md

@@ -1,20 +1,20 @@
-# Configuration options reference
+# Options reference
 
-Here is a detailed reference for the options accepted by the Echogarden API and CLI.
+Here's a detailed reference for all the options accepted by the Echogarden CLI and API.
 
-**Related resources**:
-* [A comprehensive list of all supported engines](Engines.md)
-* [A quick guide for using the command line interface](CLI.md)
-* [Developer's API reference](API.md)
+**Related pages**:
+* [List of all supported engines](Engines.md)
+* [Quick guide for the command line interface](CLI.md)
+* [Node.js API reference](API.md)
 
-## Synthesis
+## Text-to-speech
 
-Applicable to CLI commands: `speak`, `speak-file`, `speak-url`, `speak-wikipedia`, API method: `synthesize`
+Applies to CLI operations: `speak`, `speak-file`, `speak-url`, `speak-wikipedia`, API method: `synthesize`
 
 **General**:
-* `engine`: identifier of the synthesis engine to use, such as `espeak` or `vits`.
+* `engine`: identifier of the synthesis engine to use, such as `espeak`, `vits` or `google-translate` (see [the full engine list](Engines.md)). Auto-selected if not set
 * `language`: language code ([ISO 639-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes)), like `en`, `fr`, `en-US`, `pt-BR`. Auto-detected if not set
-* `voice`: name of the voice to use. Can be a search string. Optional
+* `voice`: name of the voice to use. Can be a search string. Auto-selected if not set
 * `voiceGender`: gender of the voice to use. Optional
 * `speed`: speech rate factor, relative to default. In the range `0.1`..`10.0`. Defaults to `1.0`
 * `pitch`: pitch factor, relative to default. In the range `0.1`..`10.0`. Defaults to `1.0`
@@ -23,9 +23,10 @@ Applicable to CLI commands: `speak`, `speak-file`, `speak-url`, `speak-wikipedia
 * `ssml`: the input is SSML. Defaults to `false`
 * `sentenceEndPause`: pause duration (seconds) at end of sentence. Defaults to `0.75`
 * `segmentEndPause`: pause duration (seconds) at end of segment. Defaults to `1.0`
-* `customLexiconPaths`: an array of custom lexicon file paths. Optional
-* `alignment`: prefix to provide custom options for alignment. Options detailed in section for alignment
-* `languageDetection`: prefix to provide custom options for language detection. Options detailed in section for text language detection
+* `customLexiconPaths`: a list of custom lexicon file paths. Optional
+* `alignment`: prefix to provide options for alignment. Options detailed in section for alignment
+* `subtitles`: prefix to provide options for subtitles. Options detailed in section for subtitles
+* `languageDetection`: prefix to provide options for text language detection. Options detailed in section for text language detection
 
 **Plain text processing**:
 * `plainText.paragraphBreaks`: split to paragraphs based on single (`single`), or double (`double`) line breaks. Defaults to `double`
@@ -33,22 +34,16 @@ Applicable to CLI commands: `speak`, `speak-file`, `speak-url`, `speak-wikipedia
 
 **Post-processing**:
 * `postProcessing.normalizeAudio`: should normalize output audio. Defaults to `true`
-* `postProcessing.targetPeakDb`: target peak (decibels) for normalization. Defaults to `-3`
-* `postProcessing.maxIncreaseDb`: max gain increase (decibels) when performing normalization. Defaults to `30`
+* `postProcessing.targetPeak`: target peak (decibels) for normalization. Defaults to `-3`
+* `postProcessing.maxGainIncrease`: max gain increase (decibels) when performing normalization. Defaults to `30`
 * `postProcessing.speed`: target speed for time stretching. Defaults to `1.0`
 * `postProcessing.pitch`: target pitch for pitch shifting. Defaults to `1.0`
 * `postProcessing.timePitchShiftingMethod`: method for time and pitch shifting. Can be `sonic` or `rubberband`. Defaults to `sonic`
-* `postProcessing.rubberband`: prefix for RubberBand options (TODO)
+* `postProcessing.rubberband`: prefix for RubberBand options (TODO: document options)
 
 **Output audio format**:
-* `outputAudioFormat.codec`: Codec identifier (**Note**: API only! CLI uses file extension instead), can be `wav`, `mp3`, `opus`, `m4a`, `ogg`, `flac`. Leaving as `undefined` would return a raw audio structure (see more information at the API documentation). Optional
-* `outputAudioFormat.bitrate`: Custom bitrate for encoding, applies only to  `mp3`, `opus`, `m4a`, `ogg`. By default, bitrates are selected between 48Kbps and 64Kbps, to provide a good speech quality while minimizing file size.  Optional
-
-**Subtitles**
-* `subtitles.maxLineCount`: maximum number of lines per cue. Defaults to `2`
-* `subtitles.maxLineWidth`: maximum characters in a line. Defaults to `42`
-* `subtitles.minWordsInLine`: minimum number of words in a line, such that a line break can be added. Defaults to `4`
-* `subtitles.maxAddedDuration`: maximum extra time (in seconds) that may be added after a cue's speech end time. This also ensures that very short-duration segments aren't shown in a flash. Defaults to `3`
+* `outputAudioFormat.codec`: Codec identifier (**Note**: API only. CLI uses file extensions instead), can be `wav`, `mp3`, `opus`, `m4a`, `ogg`, `flac`. Leaving as `undefined` would return a raw audio structure (see more information at the [API documentation](API.md]). Optional
+* `outputAudioFormat.bitrate`: Custom bitrate for encoding, applies only to  `mp3`, `opus`, `m4a`, `ogg`. By default, bitrates are selected between 48Kbps and 64Kbps, to provide a good speech quality while minimizing file size. Optional
 
 **VITS**:
 * `vits.speakerId`: speaker ID, for VITS models that support multiple speakers. Optional
@@ -69,7 +64,7 @@ Applicable to CLI commands: `speak`, `speak-file`, `speak-url`, `speak-wikipedia
 * `sapi.rate`: SAPI speech rate, in its native units. An integer number between `-10` and `10`. Setting `speed` would apply time stretching instead. The two options can be used together
 
 **Microsoft Speech Platform**:
-* `msspeech.rate`: same  units and effects as the SAPI speech rate
+* `msspeech.rate`: same units and effects as the SAPI speech rate
 
 **Coqui Server**:
 * `coquiServer.serverUrl`: server URL
@@ -91,7 +86,15 @@ Applicable to CLI commands: `speak`, `speak-file`, `speak-url`, `speak-wikipedia
 * `amazonPolly.accessKeyId`: access key ID (required)
 * `amazonPolly.secretAccessKey`: secret access key (required)
 * `amazonPolly.pollyEngine`: Amazon Polly engine kind, can be `standard` or `neural`. Defaults to `neural`
-* `amazonPolly.lexiconNames`: An array of lexicon names. Optional
+* `amazonPolly.lexiconNames`: an array of lexicon names. Optional
+
+**OpenAI Cloud**:
+* `openAICloud.apiKey`: API key (required)
+* `openAICloud.organization`: organization identifier. Optional
+* `openAICloud.baseURL`: override the default base URL for the API. Optional
+* `openAICloud.model`: model to use. Can be either `tts-1` or `tts-1-hd`. Defaults to `tts-1`
+* `openAICloud.timeout`: request timeout. Optional
+* `openAICloud.maxRetries`: maximum retries on failure. Defaults to 10
 
 **Elevenlabs**:
 * `elevenLabs.apiKey`: API key (required)
@@ -99,7 +102,7 @@ Applicable to CLI commands: `speak`, `speak-file`, `speak-url`, `speak-wikipedia
 * `elevenLabs.similarityBoost`: similarity boost. Defaults to `0.5`
 
 **Google Translate**:
-* `googleTranslate.tld`: top level domain to to connect to. Can change the dialect for a small number or voices. For example `us` gives American English for `en`, while `com` gives British English for `en`. Defaults to `us`
+* `googleTranslate.tld`: top level domain to connect to. Can change the dialect for a small number of voices. For example `us` gives American English for `en`, while `com` gives British English for `en`. Defaults to `us`
 
 **Microsoft Edge**:
 * `microsoftEdge.trustedClientToken`: trusted client token (required). A special token required to use the service
@@ -107,46 +110,59 @@ Applicable to CLI commands: `speak`, `speak-file`, `speak-url`, `speak-wikipedia
 
 ## Voice list request
 
-Applicable to CLI command: `list-voices`, API method: `requestVoiceList`
+Applies to CLI operation: `list-voices`, API method: `requestVoiceList`
 
-**General**
+**General**:
 * `language`: language code to filter by (optional)
 * `voice`: name or name pattern to filter by (optional)
 * `voiceGender`: gender to filter by (optional)
 
-Also accepted are engine-specific options that may be required in order to retrieve the voice list, especially for cloud engines. Examples:
+Also accepted are the following engine-specific options that may be required in order to retrieve the voice list:
 * `googleCloud.apiKey`
 * `microsoftAzure.subscriptionKey`, `microsoftAzure.serviceRegion`
 * `amazonPolly.region`, `amazonPolly.accessKeyId`, `amazonPolly.secretAccessKey`
 * `elevenLabs.apiKey`
+* `microsoftEdge.trustedClientToken`
 
-## Recognition
+## Speech-to-text
 
-Applicable to CLI command: `transcribe`, API method: `recognize`
+Applies to CLI operation: `transcribe`, API method: `recognize`
 
 **General**:
-* `engine`: identifier of the recognition engine to use, such as `whisper` or `vosk`
+* `engine`: identifier of the recognition engine to use, such as `whisper` or `vosk` (see [the full engine list](Engines.md))
 * `language`: language code ([ISO 639-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes)) for the audio, like `en`, `fr`, `de`. Auto-detected if not set
-* `alignment`: prefix to provide custom options for alignment. Options detailed in section for alignment
-* `languageDetection`: prefix to provide custom options for language detection. Options detailed in section for speech language detection
-
-**Subtitles**
-* `subtitles.mode`: subtitle generation mode. Can be `segment` (ensures each segment starts at a new cue), `sentence` (ensures each sentence starts at a new cue), `word` (one word per cue, no punctuation included), `phone` (one phone per cue), `word+phone` (include both `word` and `phone` cues, with overlapping time ranges), `line` (each text line is made a separate cue). Defaults to `sentence`
-* `subtitles.maxLineCount`: maximum number of lines per cue. Defaults to `2`
-* `subtitles.maxLineWidth`: maximum characters in a line. Defaults to `42`
-* `subtitles.minWordsInLine`: minimum number of remaining words to break to a new line. Defaults to `4`
-* `subtitles.separatePhrases`: try to separate phrases or sentences in new lines or cues, if possible. Defaults to `true`
-* `subtitles.maxAddedDuration`: maximum extra time (in seconds) that may be added after a cue's end time. This gives the reader additional time to read the cue, and also ensures that very short duration cues aren't shown in a flash. Defaults to `3.0`
-
-_Note_: options `maxLineCount`, `maxLineWidth`, `minWordsInLine`, `separatePhrases`, are only effective when using the `segment` and `sentence` modes, and are ignored in all other modes. `subtitles.maxAddedDuration` doesn't apply to modes `word`, `phone` and `word+phone` (they always use the exact start and end timestamps).
+* `crop`: crop to active parts using voice activity detection before starting recognition. Defaults to `true`
+* `isolate`: apply source separation to isolate voice before starting recognition. Defaults to `false`
+* `alignment`: prefix to provide options for alignment. Options detailed in section for alignment
+* `languageDetection`: prefix to provide options for language detection. Options detailed in section for speech language detection
+* `subtitles`: prefix to provide options for subtitles. Options detailed in section for subtitles
+* `vad`: prefix to provide options for voice activity detection when `crop` is set to `true`. Options detailed in section for voice activity detection
+* `sourceSeparation`: prefix to provide options for source separation when `isolate` is set to `true`. Options detailed in section for source separation
 
 **Whisper**:
-* `whisper.model`: selects which Whisper model to use. Can be `tiny`, `tiny.en`, `base`, `base.en`, `small`, `small.en`, `medium`, `medium.en`, `large` (same as `large-v2`), `large-v1`, `large-v2`. Defaults to `tiny` or `tiny.en`
+* `whisper.model`: selects which Whisper model to use. Can be `tiny`, `tiny.en`, `base`, `base.en`, `small`, `small.en`, `medium`, `medium.en`, `large` (same as `large-v2`), `large-v1`, `large-v2`, `large-v3` (**Note**: large models aren't yet supported by `onnxruntime-node` due to their size). Defaults to `tiny` or `tiny.en`
 * `whisper.temperature`: temperature setting for the text decoder. Impacts amount of randomization for token selection. It is recommended to leave at `0.1` (close to no randomization - almost always chooses the top ranked token) or choose a relatively low value (`0.25` or lower) for best results. Defaults to `0.1`
-* `whisper.prompt`: initial text to give the Whisper model. Can be a vocabulary, or example of some sort. Note that if the prompt is very similar to the transcript, the model may intentionally avoid producing the transcript tokens as it may assume that they have already been transcribed. Optional
+* `whisper.prompt`: initial text to give the Whisper model. Can be a vocabulary, or example text of some sort. Note that if the prompt is very similar to the transcript, the model may intentionally avoid producing the transcript tokens as it may assume that they have already been transcribed. Optional
 * `whisper.topCandidateCount`: the number of top candidate tokens to consider. Defaults to `5`
 * `whisper.punctuationThreshold`: the minimal probability for a punctuation token, included in the top candidates, to be chosen unconditionally. A lower threshold encourages the model to output more punctuation symbols. Defaults to `0.2`
 * `whisper.autoPromptParts`: use previous part's recognized text as prompt for the next part. Disabling this may help to prevent repetition carrying over between parts, in some cases. Defaults to `true`
+* `whisper.maxTokensPerPart`: maximum number of tokens to decode for each 30 second audio part. Defaults to `250`
+* `whisper.suppressRepetition`: attempt to suppress decoding repeating token patterns. Defaults to `true`
+* `whisper.seed`: use a custom seed for token selection, when temperature is greater than 0. Uses a constant seed by default
+
+**Whisper.cpp**:
+* `whisperCpp.model`: selects which `whisper.cpp` model to use.  Can be `tiny`, `tiny.en`, `base`, `base.en`, `small`, `small.en`, `medium`, `medium.en`, `large` (same as `large-v2`), `large-v1`, `large-v2`, `large-v3`. These quantized models are also supported: `tiny-q5_1`, `tiny.en-q5_1`, `tiny.en-q8_0`,`base-q5_1`, `base.en-q5_1`, `small-q5_1`, `small.en-q5_1`, `medium-q5_0`, `medium.en-q5_0`, `large-v2-q5_0`, `large-v3-q5_0`. Defaults to `base` or `base.en`
+* `whisperCpp.executablePath`: custom `whisper.cpp` executable path (currently required for macOS)
+* `whisperCpp.build`: type of `whisper.cpp` build to use. Can be set `cpu`, `cublas-11.8.0`, `cublas-12.4.0`. By default, builds are auto-selected and downloaded for Windows x64 (`cpu`, `cublas-11.8.0`, `cublas-12.4.0`) and Linux x64 (`cpu`). Using other builds requires providing a custom `executablePath`
+* `whisperCpp.threadCount`: number of threads to use, defaults to `4`
+* `whisperCpp.splitCount`: number of splits of the audio data to process in parallel (called `--processors` in the CLI). A value greater than `1` can increase memory use significantly, reduce timing accuracy, and slow down execution in some cases. Defaults to `1` (highly recommended)
+* `whisperCpp.enableGPU`: enable GPU processing. Defaults to `true` on CUDA-enabled builds, otherwise `false`
+* `whisperCpp.topCandidateCount`: the number of top candidate tokens to consider. Defaults to `5`
+* `whisperCpp.beamCount`: the number of decoding paths to use during beam search. Defaults to `5`
+* `whisperCpp.repetitionThreshold`: minimal repetition / compressibility score to cause a decoded segment to be discarded. Defaults to `2.4`
+* `whisperCpp.prompt`: initial text to give the Whisper model. Can be a vocabulary, or example text of some sort. Note that if the prompt is very similar to the transcript, the model may intentionally avoid producing the transcript tokens as it may assume that they have already been transcribed. Optional
+* `whisperCpp.enableDTW`: enable experimental `whisper.cpp` internal DTW-based token alignment to be used to derive timestamps. Defaults to `false` (recommended for now)
+* `whisperCpp.verbose`: show all CLI messages during execution. Defaults to `false`
 
 **Vosk**:
 * `vosk.modelPath`: path to the Vosk model to be used
@@ -162,83 +178,113 @@ _Note_: options `maxLineCount`, `maxLineWidth`, `minWordsInLine`, `separatePhras
 * `googleCloud.useEnhancedModel`: use enhanced model. Defaults to `true`
 
 **Azure Cognitive Services**:
-* `azureCognitiveServices.subscriptionKey`: subscription key (required)
-* `azureCognitiveServices.serviceRegion`: service region (required)
+* `microsoftAzure.subscriptionKey`: subscription key (required)
+* `microsoftAzure.serviceRegion`: service region (required)
 
 **Amazon Transcribe**:
 * `amazonTranscribe.region`: region (required)
 * `amazonTranscribe.accessKeyId`: access key ID (required)
 * `amazonTranscribe.secretAccessKey`: secret access key (required)
 
-## Alignment
+**OpenAI Cloud**:
+* `openAICloud.apiKey`: API key (required)
+* `openAICloud.model`: model to use. Can only be `whisper-1`
+* `openAICloud.organization`: organization identifier. Optional
+* `openAICloud.baseURL`: override the default base URL used by the API. Optional
+* `openAICloud.temperature`: temperature. Choosing `0` uses a dynamic temperature approach. Defaults to `0.0`
+* `openAICloud.prompt`: initial prompt for the model. Optional
+* `openAICloud.timeout`: request timeout. Optional
+* `openAICloud.maxRetries`: maximum retries on failure. Defaults to 10
+
 
-Applicable to CLI command: `align`, API method: `align`
+## Speech-to-transcript alignment
+
+Applies to CLI operation: `align`, API method: `align`
 
 **General**:
-* `engine`: what alignment algorithm to use, can be `dtw`, `dtw-ra` or `whisper`. Defaults to `dtw`
+* `engine`: alignment algorithm to use, can be `dtw`, `dtw-ra` or `whisper`. Defaults to `dtw`
 * `language`: language code for the audio and transcript ([ISO 639-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes)), like `en`, `fr`, `en-US`, `pt-BR`. Auto-detected from transcript if not set
+* `crop`: crop to active parts using voice activity detection before starting. Defaults to `true`
+* `isolate`: apply source separation to isolate voice before starting recognition. Defaults to `false`
 * `customLexiconPaths`: an array of custom lexicon file paths. Optional
+* `subtitles`: prefix to provide options for subtitles. Options detailed in section for subtitles
+* `vad`: prefix to provide options for voice activity detection when `crop` is set to `true`. Options detailed in section for voice activity detection
+* `sourceSeparation`: prefix to provide options for source separation when `isolate` is set to `true`. Options detailed in section for source separation
 
 **Plain text processing**:
 * `plainText.paragraphBreaks`: split transcript to paragraphs based on single (`single`), or double (`double`) line breaks. Defaults to `double`
 * `plainText.whitespace`: determines how to process whitespace within transcript paragraphs. Can be `preserve` (leave as is), `removeLineBreaks` (convert line breaks to spaces) or `collapse` (convert runs of whitespace characters, including line breaks, to a single space character). Defaults to `collapse`
 
-**Subtitles**
-* `subtitles.maxLineCount`: maximum number of lines per cue. Defaults to `2`
-* `subtitles.maxLineWidth`: maximum characters in a line. Defaults to `42`
-* `subtitles.minWordsInLine`: minimum number of leftover words required to break to a new line or cue. Defaults to `4`
-* `subtitles.maxAddedDuration`: maximum extra time (in seconds) that may be added after a cue's speech end time. This also ensures that very short-duration segments aren't shown in a flash. Defaults to `3`
-
 **DTW**:
 * `dtw.granularity`: adjusts the MFCC frame width and hop size based on the profile selected. Can be set to either `auto` (auto-selected based on audio duration and task), `xx-low` (400ms width, 160ms hop), `x-low` (200ms width, 80ms hop), `low` (100ms width, 40ms hop), `medium` (50ms width, 20ms hop), `high` (25ms width, 10ms hop), `x-high` (20ms width, 5ms hop). For multi-pass processing, multiple granularities can be provided, like `dtw.granularity=['low','high']`. Defaults to `auto`.
 * `dtw.windowDuration`: maximum duration (in seconds) of the Sakoe-Chiba window when performing DTW alignment. Higher values consume quadratically larger amounts of memory. The estimated memory requirement is shown in the log before alignment starts. Recommended to be set to at least 10% - 20% of total audio duration. For multi-pass processing, multiple durations can be provided, like `dtw.windowDuration=[240,20]`. Auto-selected by default
 
 **DTW-RA only**:
-* `recognition`: prefix for providing custom recognition options when using `dtw-ra` method, for example: setting `recognition.whisper.model = base.en`
+* `recognition`: prefix to provide recognition options when using `dtw-ra` method, for example: setting `recognition.engine = whisper` and `recognition.whisper.model = base.en`
 * `dtw.phoneAlignmentMethod`: algorithm to use when aligning phones: can either be set to `dtw` or `interpolate`. Defaults to `dtw`
 
-**Whisper alignment only**
-* `whisper`: prefix to provide Whisper options when the `whisper` alignment engine is used.
+**Whisper alignment only**:
+* `whisper`: prefix to provide Whisper options when the `whisper` alignment engine is used (does not apply to `dtw-ra` when `whisper` engine is used, for that use `recognition.whisper` prefix instead).
 
-## Speech translation
 
-Applicable to CLI command: `translate-speech`, API method: `translateSpeech`
+## Speech-to-text translation
+
+Applies to CLI operation: `translate-speech`, API method: `translateSpeech`
 
 **General**:
 * `engine`: only `whisper` supported
 * `sourceLanguage`: the source language code for the input speech. Auto-detected if not set
-* `targetLanguage`: the target language code for the output speech. Only `en` supported at this time
-* `languageDetection`: prefix to provide custom options for language detection. Options detailed in section for speech language detection
+* `targetLanguage`: the target language code for the output speech. Only `en` (English) supported by the `whisper` engine. Optional
+* `crop`: crop to active parts using voice activity detection before starting. Defaults to `true`
+* `isolate`: apply source separation to isolate voice before starting speech translation. Defaults to `false`
+* `languageDetection`: prefix to provide options for language detection. Options detailed in section for speech language detection
+* `vad`: prefix to provide options for voice activity detection when `crop` is set to `true`. Options detailed in section for voice activity detection
+* `sourceSeparation`: prefix to provide source separation options when `isolate` is to `true`
 
 **Whisper**:
 
-* `whisper`: prefix for options for the Whisper model. Same options as detailed in the recognition section above
+* `whisper`: prefix to provide options for the Whisper model. Same options as detailed in the recognition section above
+
+**Whisper.cpp**:
+
+* `whisper.cpp`: prefix to provide options for the Whisper.cpp model. Same options as detailed in the recognition section above
+
+**OpenAI Cloud**:
+
+* `openAICloud`: prefix to provide options for OpenAI cloud. Same options as detailed in the recognition section above
 
 ## Language detection
 
 ### Speech language detection
 
-Applicable to CLI command: `detect-speech-langauge`, API method: `detectSpeechLangauge`
+Applies to CLI operation: `detect-speech-langauge`, API method: `detectSpeechLangauge`
 
 **General**:
-* `engine`: `silero` or `whisper`. Defaults to `silero`
-* `whisper`: whisper options prefix, can be used like `whisper.model = base` to set options for the Whisper engine. See Whisper options on the recognition section
+* `engine`: `whisper` or `silero`. Defaults to `whisper`
+* `defaultLanguage`: language to fallback to when confidence for top candidate of is low. Defaults to `en`
+* `fallbackThresholdProbability`: confidence threshold to cause fallback. Defaults to `0.05`
+* `crop`: crop to active parts using voice activity detection before starting. Defaults to `true` (recommended, otherwise inactive sections may skew the probabilities towards various random languages)
+* `vad`: prefix to provide options for voice activity detection when `crop` is set to `true`. Options detailed in section for voice activity detection
+
+**Whisper**:
+* `whisper.model`: Whisper model to use. See model list in the recognition section
+* `whisper.temperature`: impacts the distribution of candidate languages when applying the softmax function to compute language probabilities over the model output. Higher temperature causes the distribution to be more uniform, while lower temperature causes it to be more strongly weighted towards the best scoring candidates. Defaults to `1.0`
 
 ### Text language detection
 
-Applicable to CLI command: `detect-text-langauge`, API method: `detectTextLangauge`
+Applies to CLI operation: `detect-text-langauge`, API method: `detectTextLangauge`
 
 **General**:
 * `engine`: `tinyld` or `fasttext`. Defaults to `tinyld`
-* `defaultLanguage`: language to fall back to when confidence of is low. Defaults to `en`
+* `defaultLanguage`: language to fallback to when confidence for top candidate is low. Defaults to `en`
 * `fallbackThresholdProbability`: confidence threshold to cause fallback. Defaults to `0.05`
 
 ## Voice activity detection
 
-Applicable to CLI command: `detect-voice-activity`, API method: `detectVoiceActivity`
+Applies to CLI operation: `detect-voice-activity`, API method: `detectVoiceActivity`
 
 **General**:
-* `engine`: VAD engine to use. Can be `webrtc`, `silero` or `rnnoise`. Defaults to `webrtc`
+* `engine`: VAD engine to use. Can be `webrtc`, `silero`, `rnnoise`, or `adaptive-gate`. Defaults to `silero`
 * `activityThreshold`: minimum predicted probability for determining a frame as having speech activity. Defaults to `0.5`
 
 **WebRTC**:
@@ -250,13 +296,47 @@ Applicable to CLI command: `detect-voice-activity`, API method: `detectVoiceActi
 
 ## Speech denoising
 
-Applicable to CLI command: `denoise`, API method: `denoise`
+Applies to CLI operation: `denoise`, API method: `denoise`
 
 **General**:
 * `engine`: can only be `rnnoise`
 
-**Postprocessing**:
+**Post-processing**:
 * `postProcessing.normalizeAudio`: should normalize output audio. Defaults to `false`
-* `postProcessing.targetPeakDb`: target peak (decibels) for normalization. Defaults to `-3`
-* `postProcessing.maxIncreaseDb`: max gain increase (decibels) when performing normalization. Defaults to `30`
-* `postProcessing.dryMixGainDb`: gain (decibels) of dry (original) signal to mix back to the denoised output. Defaults to `-20`
+* `postProcessing.targetPeak`: target peak (decibels) for normalization. Defaults to `-3`
+* `postProcessing.maxGainIncrease`: max gain increase (decibels) when performing normalization. Defaults to `30`
+* `postProcessing.dryMixGain`: gain (decibels) of dry (original) signal to mix back to the denoised (wet) signal. Defaults to `-20`
+
+## Source separation
+
+Applies to CLI operation: `isolate`, API method: `isolate`
+
+**General**:
+
+* `engine`: can only be `mdx-net`
+
+**MDX-NET**:
+
+* `mdxNet.model`: model to use. Currently available models are `UVR_MDXNET_1_9703`, `UVR_MDXNET_2_9682`, `UVR_MDXNET_3_9662`, `UVR_MDXNET_KARA`. Defaults to `UVR_MDXNET_1_9703`
+
+# Common options
+
+## Subtitles
+
+These are shared between text-to-speech, speech-to-text and alignment operations, usually prefixed with `subtitles.`.
+
+* `mode`: subtitle generation mode. Can be `segment` (ensures each segment starts at a new cue), `sentence` (ensures each sentence starts at a new cue), `word` (one word per cue, no punctuation included), `phone` (one phone per cue), `word+phone` (include both `word` and `phone` cues, with overlapping time ranges), `line` (each text line is made a separate cue). Defaults to `sentence`
+* `maxLineCount`: maximum number of lines per cue. Defaults to `2`
+* `maxLineWidth`: maximum characters in a line. Defaults to `42`
+* `minWordsInLine`: minimum number of remaining words to break to a new line. Defaults to `4`
+* `separatePhrases`: try to separate phrases or sentences in new lines or cues, if possible. Defaults to `true`
+* `maxAddedDuration`: maximum extra time (in seconds) that may be added after a cue's end time. This gives the reader additional time to read the cue, and also ensures that very short duration cues aren't shown in a flash. Defaults to `3.0`
+
+**Note**: options `maxLineCount`, `maxLineWidth`, `minWordsInLine`, `separatePhrases`, are only effective when using the `segment` and `sentence` modes, and are ignored in all other modes. `maxAddedDuration` doesn't apply to modes `word`, `phone` and `word+phone` (they always use the exact start and end timestamps).
+
+# General CLI options
+
+* `--play`, `--no-play`: enable/disable audio playback. Defaults to play if there is no output file specified
+* `--overwrite`, `--no-overwrite`: overwrite/keep existing files. Doesn't overwrite by default
+* `--debug`, `--no-debug`: show/hide the full details of JavaScript errors, if they occur. Disabled by default
+* `--config=...`: path to configuration file to use. See the [CLI guide](CLI.md) for more information about configuration files