echogarden 0.4.3 → 0.5.1

package/docs/Technical.md

@@ -2,8 +2,8 @@
 
 * Echogarden is written in TypeScript and targets the Node.js platform.
 * It uses ESM modules and latest ECMAScript and TypeScript features.
-* It does not depend on essential binary executables. Instead, all of its engines either use pure JavaScript, WebAssembly, WASI, or the ONNX runtime (with some exceptions: the CLI does use executables, loaded from an expansion package, for the SoX tool [and `ffmpeg` as well], which is used only for audio playback and recording. This simplifies the installation and ensures a non-buggy version is used. Since SoX `v14.4.2` is broken on Windows, it bundles `v14.4.1`).
-* It does not depend on essential native modules requiring compilation with `node-gyp`. This greatly simplifies and ensures the success of the installation for end-users (the ONNX runtime bundles precompiled NAPI modules for all supported platforms - it doesn't require any compilation during installation).
+* It does not depend on essential binary executables. Instead, all of its engines either use pure JavaScript, WebAssembly, WASI, or the ONNX runtime (with some exceptions: the CLI does invoke a few binary executables, loaded from expansion packages, for the `SoX` and `ffmpeg` tools.Using expansion packages simplifies the installation and ensures non-buggy version are used. Since SoX `v14.4.2` is broken on Windows, it bundles `v14.4.1`).
+* It does not depend on essential native node.js modules requiring compilation with `node-gyp`. This greatly simplifies the installation experience for end-users (the ONNX runtime bundles precompiled NAPI modules for all supported platforms - it doesn't require any compilation during its installation).
 
 ## Package system
 
@@ -16,13 +16,13 @@ Packages are downloaded as `.tar.gz` files, and are extracted to `[data-folder]/
 * `Users/User/Library/Application Support/echogarden` on macOS
 * `/home/user/.local/share/echogarden` on Linux
 
-By using downloadable packages, the installed size is made significantly smaller and the installation faster. The total size of all available packages is currently about 13GB (8.45GB compressed).
+By using downloadable packages, the installed size is made significantly smaller and the installation faster. The total size of all available packages is currently about 15.4GB (10.6GB compressed).
 
-Another advantage of using a data folder is that the data does not need to be re-downloaded when the program is updated.
+The packages are currently hosted and downloaded from a dedicated [Hugging Face repository](https://huggingface.co/echogarden/echogarden-packages).
 
 ## Can the base install size be made smaller?
 
-The installed (uncompressed) size, including dependencies, is around 270MB. This is without any models or voices, which are downloaded as needed.
+The base installed (uncompressed) size, including dependencies, is around 270MB. This is without any models or voices, which are downloaded as needed.
 
 Currently, the biggest contributors to the size are:
 
@@ -34,29 +34,31 @@ Currently, the biggest contributors to the size are:
 
 `onnxruntime-node` is big because it bundles pre-compiled binaries for multiple platforms. `kuromoji` is large because of its dictionary files and some unessential test code it bundles. The other three packages include large WASM binaries.
 
-So, yes, in the future it may be possible to reduce the core installed size by dynamically installing some of these dependencies, as npm packages, only when needed.
+So, yes, in the future it may be possible to reduce the core installed size by dynamically installing some of these dependencies, or using modified, "slimmed-down" custom versions.
 
 ## Since the code is almost all JavaScript and WASM, why can't it just run in a web browser?
 
 It is technically possible, overall, since its core components: `espeak-ng` and `onnxruntime` both have WASM ports. Actually, `onnxruntime-web`, unlike `onnxruntime-node` can also make use of the GPU via WebGL, and in the future, [it will support WebGPU](https://github.com/microsoft/onnxruntime/issues/11695), which should give a performance boost.
 
-However, it is a lot of work, and only a subset of the engines can be supported (no cloud engines, in particular). There are several reasons why the web may not be the most effective platform:
+However, it is a lot of work, and only a subset of the engines can be supported (no cloud engines, in particular). There are several reasons why the web may not be the most effective platform for Echogarden:
 
-* Significantly slower inference when running on CPU
+* Significantly slower inference when using CPU for ONNX models
 * No cross-domain network connectivity - can't connect to Google Cloud, Microsoft, Amazon etc. without a proxy
 * Large initial download size would make it too heavy and slow to load as part of a standard web page directly
+* Large memory requirement for the VITS models, starting at about 800MB - 1GB, which is a bit too much for a browser
+* Due to the high code complexity, data size, and memory consumption, it is unlikely that a browser extension, internally bundling some of the models, would be accepted to the Chrome and Firefox web stores
 * Will require a virtual file system to store models and make use of downloadable packages
-* Requires duplicating a lot of prior work and increasing code complexity
+* Requires duplicating a lot of prior work, porting many node.js-only APIs, and increasing code complexity
 * Possibly lots of issues with inconsistent browser support and browser security constraints
 * Not future-proof. Due to changing restrictions of browsers, the runtime environment is not guaranteed be reliably reproducible in the future, meaning that it may need continuous maintenance to ensure it keeps working on the newest browsers
 
-It remains to be seen what type of use cases would justify this kind of endeavor. I designed the tool to make the local installation extremely easy and issue-free. I guess it could look "impressive" to be able to run it in a browser, and may be come as a nice "toy" or "technology demonstration", and get some attention, but it may eventually turn out to be simpler and more practical to just install a local instance and connect to it from the browser via a WebSocket.
+It remains to be seen if this sort of work would feel justified somehow. I designed the tool to make the local installation extremely easy and issue-free. I guess it could look "impressive" to be able to run it in a browser, and may be come as a nice "toy" or "tech-demo", and could get some attention, but it may eventually turn out to be simpler and more practical to just install a local instance and connect to it from the browser via a WebSocket API (which is already working and running, but at a development stage).
 
-Another option is to include some of the TTS modules in a browser extension and register TTS voices on the browser's Web Speech API using the [`chrome.ttsEngine`](https://developer.chrome.com/docs/extensions/reference/ttsEngine/) extension API. That sounds reasonable, but due to the size and complexity of such an extension, it is not certain it would be approved by the Chrome Web Store. Also, `chrome.ttsEngine` is currently only supported on Chromium browsers, but not on Firefox.
+A TTS-only browser extension is in development. It registers Echogarden's voices on the browser's Web Speech API using the [`chrome.ttsEngine`](https://developer.chrome.com/docs/extensions/reference/ttsEngine/) extension API and communicates with it using the WebSocket API
 
 ## Why can't I use `stdin` and `stdout` to pipe into and out from the CLI app?
 
-I don't think that `stdin` and `stdout` are able to capture the type of complex, multi-message, bidirectional communication that is needed for a full speech processing service. When a WebSocket server is implemented, it will fulfill all these needs, and more. It would provide a uniform interface for all external clients, and would also be launched and used internally by the CLI itself. It would enable the CLI to do complex asynchronous and parallel operations it can't currently do, like transcribing a live input while text is written to the terminal and audio is played, processed and transmitted, all at the same time.
+I don't think that `stdin` and `stdout` are able to capture the type of complex, multi-message, bidirectional communication that is needed for a full speech processing service. When the WebSocket server is released, it will fulfill all these needs, and more. It would provide a uniform interface for all external clients, and would also be launched and used internally by the CLI itself. It would enable the CLI to do complex asynchronous and parallel operations it can't currently do, like transcribing a live input while text is written to the terminal and audio is played, processed and transmitted, all at the same time.
 
 ## Why does the CLI use `--option=value` and not `--option value` syntax?
 
@@ -64,11 +66,11 @@ I would have allowed `--option value` if I could. The reason `--option=value` sy
 
 In order to support a syntax like `--option value` I will need to parse the command line separately for each command, but then it would be difficult to share common options between all of them.
 
-Also, `option=value` is more similar to the syntax used in the configuration file, which makes it more consistent, and since the CLI accepts arbitrary numbers of free arguments, it help prevent confusion on whether an argument is related to an option or is it a free one.
+Also, `option=value` is more similar to the syntax used in the configuration file, which makes it more consistent, and since the CLI accepts arbitrary numbers of free arguments, it helps prevent confusion on whether an argument is related to an option or a free one.
 
 ## Code organization
 
 * `src`: TypeScript source code
 * `dist`: compiled JavaScript modules
-* `data`: various data files, including phonetic lexicons and conversion tables. `data/schemas` stores JSON schemas for all configuration options, auto-generated using [`ts-json-schema-generator`](https://github.com/vega/ts-json-schema-generator) directly from the TypeScript code, and used by the CLI to parse and validate the types of options provided
+* `data`: various data files, including phonetic lexicons and language code conversion tables. `data/schemas` stores JSON schemas for all configuration options, auto-generated using [`ts-json-schema-generator`](https://github.com/vega/ts-json-schema-generator) directly from the TypeScript code, and used by the CLI to parse and validate the options provided
 * `docs`: documentation

package/package.json

@@ -1,21 +1,19 @@
 {
 	"name": "echogarden",
-	"version": "0.4.3",
-	"description": "An integrated speech toolbox designed with end-users in mind.",
+	"version": "0.5.1",
+	"description": "A fully open-source speech system designed with end-users in mind.",
 	"author": "Rotem Dan",
 	"license": "GPL-3.0-only",
 	"keywords": [
-		"tts",
 		"text-to-speech",
-		"stt",
-		"speech-to-text",
-		"speech processing",
 		"speech synthesis",
+		"speech-to-text",
 		"speech recognition",
+		"speech processing",
 		"speech alignment",
 		"forced alignment",
 		"speech translation",
-		"language recognition",
+		"language identification",
 		"language detection"
 	],
 	"repository": {
@@ -55,8 +53,8 @@
 		"echogarden": "./dist/cli/CLILauncher.js"
 	},
 	"dependencies": {
-		"@aws-sdk/client-polly": "^3.363.0",
-		"@aws-sdk/client-transcribe-streaming": "^3.363.0",
+		"@aws-sdk/client-polly": "^3.370.0",
+		"@aws-sdk/client-transcribe-streaming": "^3.370.0",
 		"@echogarden/espeak-ng-emscripten": "^0.1.2",
 		"@echogarden/fasttext-wasm": "^0.1.0",
 		"@echogarden/flite-wasi": "^0.1.1",
@@ -122,13 +120,13 @@
 		"@types/msgpack-lite": "^0.1.8",
 		"@types/ndarray": "^1.0.11",
 		"@types/ndarray-ops": "^1.2.4",
-		"@types/node": "^20.4.1",
+		"@types/node": "^20.4.2",
 		"@types/recursive-readdir": "^2.2.1",
 		"@types/tar": "^6.1.5",
 		"@types/ws": "^8.5.5",
-		"@typescript-eslint/eslint-plugin": "^5.61.0",
-		"@typescript-eslint/parser": "^5.61.0",
-		"eslint": "^8.44.0",
+		"@typescript-eslint/eslint-plugin": "^6.1.0",
+		"@typescript-eslint/parser": "^6.1.0",
+		"eslint": "^8.45.0",
 		"ts-json-schema-generator": "^1.2.0",
 		"typescript": "^5.1.6"
 	}

package/docs/Roadmap.md

@@ -1,14 +0,0 @@
-# Roadmap
-
-1. Publish a working command line application
-1. Smooth out any significant bugs
-1. Ensure platform support
-1. Add missing features, options or commands to the CLI
-1. Stabilize the library API
-1. Open up the library API. Add any extra methods or options needed for that to happen
-1. Develop a worker API that closely mirrors the library API
-1. Develop a WebSocket server based on the worker API
-1. Develop a basic Web-based client app, talking the WebSocket server
-1. Modify the CLI app to internally use the background worker
-1. Further improve the client app
-1. Port the offline speech synthesis API to the browser (in particular the `vits`, `pico`, `flite`, `eSpeak` engines)