echogarden 0.6.4 → 0.6.6

package/docs/Options.md

@@ -12,20 +12,20 @@ Applicable to CLI commands: `speak`, `speak-file`, `speak-url`, `speak-wikipedia
 
 General:
 * `engine`: identifier of the synthesis engine to use, such as `espeak` or `vits`.
-* `language`: language code, like `en`, `fr`, `en-US`, `pt-BR`. Auto-detected if not set
+* `language`: language code ([ISO 639-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes)), like `en`, `fr`, `en-US`, `pt-BR`. Auto-detected if not set
 * `voice`: name of the voice to use. Can be a search string. Optional
 * `voiceGender`: gender of the voice to use. Optional
 * `speed`: speech rate factor, relative to default. In the range `0.1`..`10.0`. Defaults to `1.0`
 * `pitch`: pitch factor, relative to default. In the range `0.1`..`10.0`. Defaults to `1.0`
 * `pitchVariation`: pitch variation factor. In the range `0.1`..`10.0`. Defaults to `1.0`
-* `splitToSentences`: split text to sentences before synthesis. Default to `true`
+* `splitToSentences`: split text to sentences before synthesis. Defaults to `true`
 * `ssml`: the input is SSML. Defaults to `false`
 * `sentenceEndPause`: pause duration (seconds) at end of sentence. Defaults to `0.75`
 * `segmentEndPause`: pause duration (seconds) at end of segment. Defaults to `1.0`
 
-Plain text preprocessing:
+Plain text processing:
 * `plainText.paragraphBreaks`: split to paragraphs based on single (`single`), or double (`double`) line breaks. Defaults to `double`
-* `plainText.preserveLineBreaks`: preserve line breaks within paragraphs. Defaults to `false`
+* `plainText.whitespace`: determines how to process whitespace within paragraphs. Can be `preserve` (leave as is), `removeLineBreaks` (convert line breaks to spaces) or `collapse` (convert runs of whitespace characters, including line breaks, to a single space character). Defaults to `collapse`
 
 Post-processing:
 * `postProcessing.normalizeAudio`: should normalize output audio. Defaults to `true`
@@ -84,9 +84,9 @@ Amazon Polly:
 
 Elevenlabs:
 * `elevenLabs.apiKey`: API key (required)
-* `elevenLabs.stability`: stability. Defaults to `0.0`
-* `elevenLabs.similarityBoost`: similarity boost. Defaults to `0.0`
 * `elevenLabs.modelId`: Model identifier. Defaults to `eleven_monolingual_v1`
+* `elevenLabs.stability`: stability. Defaults to `0.5`
+* `elevenLabs.similarityBoost`: similarity boost. Defaults to `0.5`
 
 Google Translate:
 * `googleTranslate.tld`: top level domain to to connect to. Can change the dialect for a small number or voices. For example `us` gives American English for `en`, while `com` gives British English for `en`. Defaults to `us`
@@ -101,7 +101,7 @@ Applicable to CLI command: `transcribe`.
 
 General:
 * `engine`: identifier of the recognition engine to use, such as `whisper` or `vosk`
-* `language`: language code for the audio, like `en`, `fr`, `en-US`, `pt-BR`. Auto-detected if not set
+* `language`: language code ([ISO 639-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes)) for the audio, like `en`, `fr`, `de`. Auto-detected if not set
 
 Whisper:
 * `whisper.model`: selects which Whisper model to use. Can be `tiny`, `tiny.en`, `base`, `base.en`, `small`, `small.en`, `medium`, `medium.en`, `large` (same as `large-v2`), `large-v1`, `large-v2`. Defaults to `tiny`
@@ -134,10 +134,10 @@ Applicable to CLI command: `align`.
 
 General:
 * `method`: what alignment algorithm to use, can be `dtw`, `dtw-ra` or `whisper`. Defaults to `dtw`
-* `language`: language code for the audio and transcript, like `en`, `fr`, `en-US`, `pt-BR`. Auto-detected if not set
+* `language`: language code for the audio and transcript ([ISO 639-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes)), like `en`, `fr`, `en-US`, `pt-BR`. Auto-detected from transcript if not set
 
 DTW:
-* `dtw.windowDuration`: time duration (in seconds) of the Sakoe-Chiba window when performing DTW alignment. Defaults to `120`. If your audio is longer than two minutes, consider increasing this value for better results. Note that a higher value would consume quadratically larger amounts of memory. A value of `600` (ten minutes) would already require several Gigabytes of memory when the audio duration is 10 minutes or greater.
+* `dtw.windowDuration`: time duration (in seconds) of the Sakoe-Chiba window when performing DTW alignment. Defaults to `120`. If your audio is longer than two minutes, consider increasing this value for better results. Note that a higher value would consume quadratically larger amounts of memory. A value of `600` (10 minutes) would already require several Gigabytes of memory when the audio duration is 10 minutes or greater.
 
 DTW-RA only:
 * `dtw.recognition`: prefix for providing custom recognition options when using `dtw-ra` method, for example: setting `dtw.recognition.engine = silero`
@@ -149,13 +149,14 @@ Applicable to CLI command: `translate-speech`.
 
 General:
 * `engine`: only `whisper` supported
+* `sourceLanguage`: the source language code for the input speech. Auto-detected if not set
+* `targetLanguage`: the target language code for the output speech. Only `en` supported at this time.
 
 Whisper:
-* `whisper.engine`: Whisper engine to use (multilingual engines only). Defaults to `tiny`
+* `whisper.model`: Whisper model to use (multilingual engines only). Defaults to `tiny`
 
 ## Language detection
 
-
 ### Speech language detection
 
 Applicable to CLI command: `detect-speech-langauge`.
@@ -168,7 +169,7 @@ Applicable to CLI command: `detect-speech-langauge`.
 Applicable to CLI command: `detect-text-langauge`.
 
 * `engine`: `tinyld` or `fasttext`. Defaults to `tinyld`
-* `defaultLanguage`: language to fall back to when confidence is low. Defaults to `en`
+* `defaultLanguage`: language to fall back to when confidence of is low. Defaults to `en`
 * `fallbackThresholdProbability`: confidence threshold to cause fallback. Defaults to `0.05`
 
 ## Voice activity detection

package/docs/Tasklist.md

@@ -78,6 +78,9 @@
 ### CLI / `speak-url`
 * Use the Wikipedia reader when the URL is detected to be from `wikipedia.org`
 
+### CLI / `list-voices`
+* When given a configuration file, see if you can fall back to take options from from `speak` options, for example, API keys that are required for the both the voice list request and synthesis request
+
 ### CLI / `list-packages`
 * Support filters
 
@@ -156,7 +159,8 @@
 ### VITS
 * Allow to limit how many models are cached in memory
 * Custom model paths (decide how to implement)
-* Pull voice list from JSON file based on URL? Is that a good idea?
+* Pull voice list from JSON file, or based on URL? Is that a good idea?
+* Add speaker names to voice list somehow
 
 ### Recognition
 * Add confidence to each recognized word, if available
@@ -164,7 +168,7 @@
 * Look for good split points using VAD before performing recognition
 * Option to split recognized audio to segments or sentences, as is done with synthesized audio
 
-### Whisper
+### Recognition / Whisper
 * When using `dtw-ra` alignment, pass the transcript as a prompt. Remove some of the initial transcript based on what has been detected (try to find the best matching initial segment between the transcript and recognized text, and remove it at each recognition window).
 * During language detection, if file is more than 30s, run the detection over all the segments and average the resulting probability distributions, consider how to handle very short segments
 * Timestamps extracted from cross-attention are still not as accurate as what the official Python implementation gets. Try to see if you can make them better.
@@ -173,6 +177,10 @@
 * Bring back option to use eSpeak DTW based alignment on segments, as an alternative approach
 * The segment output can be use to split to segment files, otherwise it is possible to try to guess using the pause lengths or voice activity detection
 * Way to specify model size, such that the English-only/multilingual would be auto selection for sizes other than `tiny`?
+* Accept custom prompt as through an option
+
+### Alignment
+* Warn when input is larger than DTW window (this can also happen when synthesizing SSML, which can't be split to segments)
 
 ### Postprocessing
 * When `normalize` is set to false, should obvious clipping still be prevented?
@@ -202,6 +210,7 @@
 ## Things to test
 
 * Test that SSML works where it should
+* Test that alignment works correctly when the input is SSML
 * Test synthesis, recognition and alignment with empty input. Do they still work?
 * Test everything's fine on macOS
 * Test that cloud services all still work correctly, especially with SSML inputs
@@ -231,6 +240,7 @@
 * Live vosk alternatives events
 * Implement beam search for Whisper decoder
 * Implement beam search for Silero decoder
+* Investigate exporting Whisper models to 16-bit and 8-bit quantized ONNX
 
 ### Web
 * Web based frontend UI to the server

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "echogarden",
-	"version": "0.6.4",
+	"version": "0.6.6",
 	"description": "An integrated speech system, providing a range of synthesis, recognition and other processing tools designed to be directly accessible to end-users.",
 	"author": "Rotem Dan",
 	"license": "GPL-3.0-only",