echogarden 3.3.0 → 3.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +5 -0
- package/dist/api/Synthesis.d.ts.map +1 -1
- package/dist/api/Synthesis.js +31 -21
- package/dist/api/Synthesis.js.map +1 -1
- package/docs/API.md +51 -34
- package/docs/CLI.md +41 -6
- package/docs/Contributing.md +1 -1
- package/docs/Development.md +22 -18
- package/docs/Engines.md +0 -1
- package/docs/Licenses.md +2 -0
- package/docs/Options.md +56 -4
- package/docs/Releases.md +45 -14
- package/docs/Server.md +14 -5
- package/docs/Tasklist.md +37 -1
- package/docs/Technical.md +1 -0
- package/package.json +14 -12
- package/src/api/Synthesis.ts +39 -27
package/docs/CLI.md
CHANGED
|
@@ -9,10 +9,12 @@ echogarden [operation] [one or more inputs..] [one or more outputs...] [options.
|
|
|
9
9
|
Each operation can accept one or more options, in the form `--[optionName]=[value]` (The `=` is required).
|
|
10
10
|
|
|
11
11
|
**Keyboard shortcuts**:
|
|
12
|
+
|
|
12
13
|
* While the program is running, you can press `esc` to exit immediately
|
|
13
14
|
* When audio is playing, you can press `enter` to skip it, `space` to pause/resume, `right` to skip 1 second forward, and `left` to skip 1 second backwards
|
|
14
15
|
|
|
15
16
|
### Related pages
|
|
17
|
+
|
|
16
18
|
* [Options reference](Options.md)
|
|
17
19
|
* [List of all supported engines](Engines.md)
|
|
18
20
|
|
|
@@ -21,41 +23,49 @@ Each operation can accept one or more options, in the form `--[optionName]=[valu
|
|
|
21
23
|
**Task**: Given a text file, synthesize spoken audio for it.
|
|
22
24
|
|
|
23
25
|
This would synthesize "Hello World" and play the result in the terminal:
|
|
26
|
+
|
|
24
27
|
```bash
|
|
25
28
|
echogarden speak "Hello world!"
|
|
26
29
|
```
|
|
27
30
|
|
|
28
31
|
If no language is specified, it would attempt to detect it. This usually works better for longer texts, and may misidentify shorter ones. To ensure the right language is selected, you can specify the language explicitly:
|
|
32
|
+
|
|
29
33
|
```bash
|
|
30
34
|
echogarden speak "Hello world!" --language=en
|
|
31
35
|
```
|
|
32
36
|
|
|
33
37
|
This would save the resulting audio to `result.mp3`:
|
|
38
|
+
|
|
34
39
|
```bash
|
|
35
40
|
echogarden speak "Hello world!" result.mp3 --language=en
|
|
36
41
|
```
|
|
37
42
|
|
|
38
43
|
`speak-file` synthesizes text loaded from a textual file, which can have the extensions `txt`, `html`, `xml`, `ssml`, `srt`, `vtt`:
|
|
44
|
+
|
|
39
45
|
```bash
|
|
40
46
|
echogarden speak-file text.txt result.mp3 --language=en
|
|
41
47
|
```
|
|
42
48
|
|
|
43
49
|
You can specify an engine using the `--engine` option (a full list of engines can be found [here](Engines.md)). This would set the synthesis engine to `pico` (SVOX Pico):
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
echogarden speak-file text.txt result.mp3 --language=en --engine=pico
|
|
53
|
+
```
|
|
47
54
|
|
|
48
55
|
The CLI supports multiple output files. This would synthesize a text file, and save the resulting audio in both `result.mp3` and `result.wav`, as well as subtitles in `result.srt`:
|
|
56
|
+
|
|
49
57
|
```bash
|
|
50
58
|
echogarden speak-file text.txt result.mp3 result.wav result.srt --engine=kokoro --speed=1.1
|
|
51
59
|
```
|
|
52
60
|
|
|
53
61
|
Synthesize a web page (it will try to extract its main article parts and omit the rest):
|
|
62
|
+
|
|
54
63
|
```bash
|
|
55
64
|
echogarden speak-url https://example.com/hola
|
|
56
65
|
```
|
|
57
66
|
|
|
58
67
|
Synthesize a Wikipedia article, in any of its language editions:
|
|
68
|
+
|
|
59
69
|
```bash
|
|
60
70
|
echogarden speak-wikipedia "Psychologie" --language=fr
|
|
61
71
|
```
|
|
@@ -65,11 +75,13 @@ echogarden speak-wikipedia "Psychologie" --language=fr
|
|
|
65
75
|
**Task**: Given an audio recording containing speech, find a textual transcription that best matches it.
|
|
66
76
|
|
|
67
77
|
This would transcribe the audio file `speech.mp3`, and then play the audio, along with the recognized text, in the terminal:
|
|
78
|
+
|
|
68
79
|
```bash
|
|
69
80
|
echogarden transcribe speech.mp3
|
|
70
81
|
```
|
|
71
82
|
|
|
72
83
|
This would transcribe the audio file `speech.mp3` and store the resulting transcription in `result.txt`, subtitles in `result.srt`, and a full timeline tree in `result.json`:
|
|
84
|
+
|
|
73
85
|
```bash
|
|
74
86
|
echogarden transcribe speech.mp3 result.txt result.srt result.json
|
|
75
87
|
```
|
|
@@ -79,11 +91,13 @@ echogarden transcribe speech.mp3 result.txt result.srt result.json
|
|
|
79
91
|
**Task**: Given an audio file and its transcript, try to approximate the timing of the start and end of each spoken word (and its subparts).
|
|
80
92
|
|
|
81
93
|
This would align the audio file `speech.mp3` with the transcript provided in `transcript.txt`, and would play the synchronized result in the terminal:
|
|
94
|
+
|
|
82
95
|
```bash
|
|
83
96
|
echogarden align speech.mp3 transcript.txt
|
|
84
97
|
```
|
|
85
98
|
|
|
86
99
|
This would align the audio file `speech.mp3` with the transcript provided in `transcript.txt`, and store the resulting subtitles in `result.srt`, and a full timeline tree in `result.json`:
|
|
100
|
+
|
|
87
101
|
```bash
|
|
88
102
|
echogarden align speech.mp3 transcript.txt result.srt result.json
|
|
89
103
|
```
|
|
@@ -93,11 +107,13 @@ echogarden align speech.mp3 transcript.txt result.srt result.json
|
|
|
93
107
|
**Task**: Given an audio file containing speech in one language, transcribe it to a second language. The translated transcript should be generated directly from the speech itself, without an intermediate textual translation step.
|
|
94
108
|
|
|
95
109
|
This will detect the spoken language, apply speech translation to English, and play the original audio, synced with the translated transcript:
|
|
110
|
+
|
|
96
111
|
```bash
|
|
97
112
|
echogarden translate-speech speech.mp3
|
|
98
113
|
```
|
|
99
114
|
|
|
100
115
|
To specify the source and target languages explicitly, use the `sourceLanguage` and `targetLanguage` options:
|
|
116
|
+
|
|
101
117
|
```bash
|
|
102
118
|
echogarden translate-speech speech.mp3 translation.txt --sourceLanguage=es --targetLanguage=en
|
|
103
119
|
```
|
|
@@ -107,19 +123,23 @@ echogarden translate-speech speech.mp3 translation.txt --sourceLanguage=es --tar
|
|
|
107
123
|
## Speech-to-translated-transcript alignment
|
|
108
124
|
|
|
109
125
|
### Direct alignment (English target only)
|
|
126
|
+
|
|
110
127
|
**Task**: Given a spoken audio file and its English translated transcript, try to approximate the timing of the start and end of each translated word.
|
|
111
128
|
|
|
112
129
|
This would align the audio file `dutch-speech.mp3` with the translated transcript provided in `english-translation.txt`, and would play the synchronized result in the terminal:
|
|
130
|
+
|
|
113
131
|
```bash
|
|
114
132
|
echogarden align-translation dutch-speech.mp3 english-translation.txt
|
|
115
133
|
```
|
|
116
134
|
|
|
117
135
|
This would align the audio file `dutch-speech.mp3` with the translated transcript provided in `english-translation.txt`, and store the resulting subtitles in `result.srt`, and a full timeline tree in `result.json`:
|
|
136
|
+
|
|
118
137
|
```bash
|
|
119
138
|
echogarden align-translation dutch-speech.mp3 english-translation.txt result.srt result.json
|
|
120
139
|
```
|
|
121
140
|
|
|
122
141
|
### Two-stage alignment (any of 96 source and target languages, combined stages)
|
|
142
|
+
|
|
123
143
|
**Task**: Given a spoken audio file, its transcript, and its translated transcript, try to approximate the timing of the start and end of each translated word.
|
|
124
144
|
|
|
125
145
|
This would align the audio file `dutch-speech.mp3` with the Dutch (native language) transcript provided in `dutch-transcript.txt` and the translated transcript provided in `russian-translation.txt`, and would play the synchronized result in the terminal:
|
|
@@ -153,6 +173,7 @@ This manual two-step approach allows to reuse the already-aligned transcript in
|
|
|
153
173
|
**Stage 1**:
|
|
154
174
|
|
|
155
175
|
Align the audio with its native language transcript, to produce a timeline in the native language:
|
|
176
|
+
|
|
156
177
|
```bash
|
|
157
178
|
echogarden align dutch-speech.mp3 dutch-transcript.txt dutch-timeline.json
|
|
158
179
|
```
|
|
@@ -167,22 +188,24 @@ echogarden align-timeline-translation dutch-timeline.json russian-transcript.txt
|
|
|
167
188
|
|
|
168
189
|
(`--audio` is only used for previewing the result in the terminal. Otherwise, it is not necessary)
|
|
169
190
|
|
|
170
|
-
|
|
171
191
|
## Language detection
|
|
172
192
|
|
|
173
193
|
**Task**: Given audio or textual input, try to identify which language it is spoken or written in.
|
|
174
194
|
|
|
175
195
|
Try to identify the language of an audio file containing speech, and print the probabilities to the terminal:
|
|
196
|
+
|
|
176
197
|
```bash
|
|
177
198
|
echogarden detect-speech-language speech.mp3
|
|
178
199
|
```
|
|
179
200
|
|
|
180
201
|
Try to identify the language of a text file, and print the probabilities to the terminal:
|
|
202
|
+
|
|
181
203
|
```bash
|
|
182
204
|
echogarden detect-text-language story.txt
|
|
183
205
|
```
|
|
184
206
|
|
|
185
207
|
Try to identify the language of a text file, and store the detailed probabilities in a JSON file:
|
|
208
|
+
|
|
186
209
|
```bash
|
|
187
210
|
echogarden detect-text-language story.txt detection-results.json
|
|
188
211
|
```
|
|
@@ -192,11 +215,13 @@ echogarden detect-text-language story.txt detection-results.json
|
|
|
192
215
|
**Task**: Given an audio file, try to classify which parts of the audio contain speech, and which don't.
|
|
193
216
|
|
|
194
217
|
This would apply VAD and play the audio, synchronized with `speech` and `nonspeech` indicators, printed to the terminal.
|
|
218
|
+
|
|
195
219
|
```bash
|
|
196
220
|
echogarden detect-voice-activity speech.mp3
|
|
197
221
|
```
|
|
198
222
|
|
|
199
223
|
This would apply VAD and store the results in a timeline JSON file.
|
|
224
|
+
|
|
200
225
|
```bash
|
|
201
226
|
echogarden detect-voice-activity speech.mp3 timeline.json
|
|
202
227
|
```
|
|
@@ -206,11 +231,13 @@ echogarden detect-voice-activity speech.mp3 timeline.json
|
|
|
206
231
|
**Task**: Attempt to reduce the amount of background noise in a spoken recording.
|
|
207
232
|
|
|
208
233
|
This would apply denoising and play the denoised audio:
|
|
234
|
+
|
|
209
235
|
```bash
|
|
210
236
|
echogarden denoise speech.mp3
|
|
211
237
|
```
|
|
212
238
|
|
|
213
239
|
This would apply denoising, and save the denoised audio to a file:
|
|
240
|
+
|
|
214
241
|
```bash
|
|
215
242
|
echogarden denoise speech.mp3 denoised-speech.mp3
|
|
216
243
|
```
|
|
@@ -220,16 +247,19 @@ echogarden denoise speech.mp3 denoised-speech.mp3
|
|
|
220
247
|
**Task**: Try to isolate a vocal track (or other type of track, depending on model used), from the audio.
|
|
221
248
|
|
|
222
249
|
This would apply source separation and play the isolated audio:
|
|
250
|
+
|
|
223
251
|
```bash
|
|
224
252
|
echogarden isolate voice-with-music.mp3
|
|
225
253
|
```
|
|
226
254
|
|
|
227
255
|
This would apply source separation, and save both the isolated and background audio:
|
|
256
|
+
|
|
228
257
|
```bash
|
|
229
258
|
echogarden isolate voice-with-music.mp3 voice-isolated.mp3
|
|
230
259
|
```
|
|
231
260
|
|
|
232
261
|
Written files would be:
|
|
262
|
+
|
|
233
263
|
```
|
|
234
264
|
voice-isolated.mp3
|
|
235
265
|
voice-isolated.background.mp3
|
|
@@ -256,12 +286,15 @@ echogarden align speech.mp3 transcript.txt parts/[segment].m4a parts/[segment].s
|
|
|
256
286
|
Splitting based on sentences, using a `[sentence]` placeholder, is currently on the to-do list. Please let me know if you find this feature important, and I'll prioritize it.
|
|
257
287
|
|
|
258
288
|
## Audio playback
|
|
289
|
+
|
|
259
290
|
By default, audio isn't played in the terminal when an output file is specified, you can override this behavior by adding `--play`:
|
|
291
|
+
|
|
260
292
|
```bash
|
|
261
293
|
echogarden speak-file text.txt result.mp3 --play
|
|
262
294
|
```
|
|
263
295
|
|
|
264
296
|
Or similarly prevent playback using `--no-play`:
|
|
297
|
+
|
|
265
298
|
```bash
|
|
266
299
|
echogarden transcribe speech.mp3 --no-play
|
|
267
300
|
```
|
|
@@ -281,6 +314,7 @@ When a file named `echogarden.config` is found at the current directory, it will
|
|
|
281
314
|
The configuration file format is simple and has a dedicated section for each command (all `speak-` commands are grouped together under `speak`), `global` section for global API options, and `cli` for common CLI options. `#` is used as a comment character.
|
|
282
315
|
|
|
283
316
|
Example:
|
|
317
|
+
|
|
284
318
|
```conf
|
|
285
319
|
[global]
|
|
286
320
|
|
|
@@ -324,6 +358,7 @@ whisper.temperature = 0.15
|
|
|
324
358
|
You can also use a JSON configuration file format instead, if preferred.
|
|
325
359
|
|
|
326
360
|
Name your file `echogarden.config.json`:
|
|
361
|
+
|
|
327
362
|
```json
|
|
328
363
|
{
|
|
329
364
|
"speak": {
|
|
@@ -342,6 +377,7 @@ Name your file `echogarden.config.json`:
|
|
|
342
377
|
```
|
|
343
378
|
|
|
344
379
|
Flattened property names are also accepted:
|
|
380
|
+
|
|
345
381
|
```json
|
|
346
382
|
{
|
|
347
383
|
"transcribe": {
|
|
@@ -351,7 +387,6 @@ Flattened property names are also accepted:
|
|
|
351
387
|
}
|
|
352
388
|
```
|
|
353
389
|
|
|
354
|
-
|
|
355
390
|
## Information and lists
|
|
356
391
|
|
|
357
392
|
### `list-engines`
|
|
@@ -371,6 +406,7 @@ echogarden list-voices google-cloud
|
|
|
371
406
|
```
|
|
372
407
|
|
|
373
408
|
Saves the voice list in a JSON file:
|
|
409
|
+
|
|
374
410
|
```bash
|
|
375
411
|
echogarden list-voices google-cloud google-cloud-voices.json
|
|
376
412
|
```
|
|
@@ -390,4 +426,3 @@ Uninstall one or more packages
|
|
|
390
426
|
### `list-packages`
|
|
391
427
|
|
|
392
428
|
Show a list of installed packages
|
|
393
|
-
|
package/docs/Contributing.md
CHANGED
|
@@ -11,6 +11,7 @@ First, check the issue tracker, as well as the [task list](Tasklist.md) to see i
|
|
|
11
11
|
If you find the issue you're encountering in the task list, you can still open an issue to discuss it. This allows me to know that someone cares about a particular issue, and I may give it higher priority.
|
|
12
12
|
|
|
13
13
|
There might be some obvious errors that have gone unreported. Especially if:
|
|
14
|
+
|
|
14
15
|
* You're using the macOS architecture: I don't have access to a macOS machine, so personally, I did not and cannot perform testing on that platform.
|
|
15
16
|
* You're using cloud services: There may be changes in the service that will require updating the code. I don't often test they work correctly, since my trial periods in Google, Microsoft and Amazon have all expired, thus testing requires me to use paid requests.
|
|
16
17
|
|
|
@@ -32,7 +33,6 @@ If the problem is serious, you can report it, and we'll see what we can do.
|
|
|
32
33
|
|
|
33
34
|
See the guide for [setting up a development environment](Development.md).
|
|
34
35
|
|
|
35
|
-
|
|
36
36
|
## Notes about licensing
|
|
37
37
|
|
|
38
38
|
The code is currently licensed under GPL-v3, mainly due to one of its core dependencies, [eSpeak-NG](https://github.com/espeak-ng/espeak-ng), having this license.
|
package/docs/Development.md
CHANGED
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
Here's a quick guide on how to set up a development environment for making and testing changes to the Echogarden codebase.
|
|
4
4
|
|
|
5
5
|
**Related pages**:
|
|
6
|
+
|
|
6
7
|
* [How to help](Contributing.md)
|
|
7
8
|
* [Technical overview](Technical.md)
|
|
8
9
|
* [Node.js API reference](API.md)
|
|
@@ -20,33 +21,32 @@ Here's a quick guide on how to set up a development environment for making and t
|
|
|
20
21
|
Create the `.vscode` subdirectory at the project base directory
|
|
21
22
|
|
|
22
23
|
Add a `.vscode/tasks.json` file to run the TypeScript compiler in watch mode:
|
|
24
|
+
|
|
23
25
|
```json
|
|
24
26
|
{
|
|
25
27
|
"version": "2.0.0",
|
|
26
|
-
"command": "tsc.cmd",
|
|
27
|
-
"args": [
|
|
28
|
-
"-w",
|
|
29
|
-
"-p",
|
|
30
|
-
"."
|
|
31
|
-
],
|
|
32
|
-
"problemMatcher": "$tsc-watch",
|
|
33
|
-
"isBackground": true,
|
|
34
28
|
"tasks": [
|
|
35
29
|
{
|
|
36
|
-
"label": "
|
|
37
|
-
"type": "
|
|
38
|
-
"
|
|
39
|
-
"
|
|
40
|
-
|
|
41
|
-
"-
|
|
42
|
-
"."
|
|
30
|
+
"label": "tsc: watch",
|
|
31
|
+
"type": "typescript",
|
|
32
|
+
"tsconfig": "tsconfig.json",
|
|
33
|
+
"option": "watch",
|
|
34
|
+
"problemMatcher": [
|
|
35
|
+
"$tsc-watch"
|
|
43
36
|
],
|
|
44
|
-
"isBackground": true,
|
|
45
|
-
"problemMatcher": "$tsc-watch",
|
|
46
37
|
"group": {
|
|
47
38
|
"kind": "build",
|
|
48
39
|
"isDefault": true
|
|
49
40
|
},
|
|
41
|
+
"runOptions": {
|
|
42
|
+
"runOn": "folderOpen",
|
|
43
|
+
"instanceLimit": 1,
|
|
44
|
+
"instancePolicy": "silent"
|
|
45
|
+
},
|
|
46
|
+
"presentation": {
|
|
47
|
+
"reveal": "never",
|
|
48
|
+
"focus": false
|
|
49
|
+
}
|
|
50
50
|
}
|
|
51
51
|
]
|
|
52
52
|
}
|
|
@@ -82,7 +82,11 @@ Add a `.vscode/launch.json` file to launch the CLI in debug mode:
|
|
|
82
82
|
"program": "${workspaceFolder}/dist/cli/CLIStarter.js",
|
|
83
83
|
"outputCapture": "std",
|
|
84
84
|
"console": "integratedTerminal",
|
|
85
|
-
"runtimeArgs": [
|
|
85
|
+
"runtimeArgs": [
|
|
86
|
+
"--experimental-wasi-unstable-preview1",
|
|
87
|
+
"--no-warnings",
|
|
88
|
+
"--trace-uncaught"
|
|
89
|
+
],
|
|
86
90
|
|
|
87
91
|
"args": ["speak", "Hello World!", "--debug"]
|
|
88
92
|
}
|
package/docs/Engines.md
CHANGED
package/docs/Licenses.md
CHANGED
|
@@ -26,6 +26,7 @@
|
|
|
26
26
|
A large variety of voices, models and binaries are served from the repository.
|
|
27
27
|
|
|
28
28
|
All are freely distributable, with varying licenses:
|
|
29
|
+
|
|
29
30
|
* Kokoro model and voices (`kokoro-`) [Apache License 2.0](https://huggingface.co/hexgrad/Kokoro-82M)
|
|
30
31
|
* Flite voices (`flite-`): [BSD License](https://github.com/festvox/flite/blob/master/COPYING)
|
|
31
32
|
* SVOX Pico resources (`pico-`): [Apache License 2.0](https://github.com/gmn/nanotts/blob/master/LICENSE)
|
|
@@ -36,5 +37,6 @@ All are freely distributable, with varying licenses:
|
|
|
36
37
|
* NSNet2: [Attribution 4.0 International](https://github.com/NeonGeckoCom/nsnet2-denoiser/blob/main/LICENSE)
|
|
37
38
|
|
|
38
39
|
Tool binary distributions
|
|
40
|
+
|
|
39
41
|
* FFmpeg: [LGPL, GPL v2 and GPL v3 Licenses](https://github.com/FFmpeg/FFmpeg)
|
|
40
42
|
* whisper.cpp: [MIT License](https://github.com/ggerganov/whisper.cpp/blob/master/LICENSE)
|