RealTimeSTT 0.1.16__tar.gz → 0.2.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.1
2
- Name: RealTimeSTT
3
- Version: 0.1.16
2
+ Name: RealtimeSTT
3
+ Version: 0.2.0
4
4
  Summary: A fast Voice Activity Detection and Transcription System
5
5
  Home-page: https://github.com/KoljaB/RealTimeSTT
6
6
  Author: Kolja Beigel
@@ -17,16 +17,21 @@ Requires-Dist: faster-whisper==1.0.2
17
17
  Requires-Dist: pvporcupine==1.9.5
18
18
  Requires-Dist: webrtcvad==2.0.10
19
19
  Requires-Dist: halo==0.0.31
20
- Requires-Dist: torch==2.3.0
21
- Requires-Dist: torchaudio==2.3.0
20
+ Requires-Dist: torch==2.3.1
21
+ Requires-Dist: torchaudio==2.3.1
22
22
  Requires-Dist: scipy==1.12.0
23
23
  Requires-Dist: websockets==v12.0
24
+ Requires-Dist: openwakeword==0.6.0
24
25
 
25
26
 
26
27
  # RealtimeSTT
27
28
 
28
29
  *Easy-to-use, low-latency speech-to-text library for realtime applications*
29
30
 
31
+ ## New
32
+
33
+ Custom wake words with [OpenWakeWord](#openwakeword). Thanks to the [developers](https://github.com/dscripka/openWakeWord) of this!
34
+
30
35
  ## About the Project
31
36
 
32
37
  RealtimeSTT listens to the microphone and transcribes voice into text.
@@ -42,7 +47,7 @@ https://github.com/KoljaB/RealtimeSTT/assets/7604638/207cb9a2-4482-48e7-9d2b-072
42
47
 
43
48
  ### Updates
44
49
 
45
- Latest Version: v0.1.15
50
+ Latest Version: v0.2.0
46
51
 
47
52
  See [release history](https://github.com/KoljaB/RealtimeSTT/releases).
48
53
 
@@ -66,10 +71,10 @@ This library uses:
66
71
  - **Speech-To-Text**
67
72
  - [Faster_Whisper](https://github.com/guillaumekln/faster-whisper) for instant (GPU-accelerated) transcription.
68
73
  - **Wake Word Detection**
69
- - [Porcupine](https://github.com/Picovoice/porcupine) for wake word detection.
74
+ - [Porcupine](https://github.com/Picovoice/porcupine) or [OpenWakeWord](https://github.com/dscripka/openWakeWord) for wake word detection.
70
75
 
71
- *These components represent the "industry standard" for cutting-edge applications, providing the most modern and effective foundation for building high-end solutions.*
72
76
 
77
+ *These components represent the "industry standard" for cutting-edge applications, providing the most modern and effective foundation for building high-end solutions.*
73
78
 
74
79
  ## Installation
75
80
 
@@ -83,22 +88,64 @@ Although it is possible to run RealtimeSTT with a CPU installation only (use a s
83
88
 
84
89
  ### GPU Support with CUDA (recommended)
85
90
 
86
- Additional steps are needed for a **GPU-optimized** installation. These steps are recommended for those who require **better performance** and have a compatible NVIDIA GPU.
91
+
92
+ **Install RealtimeSTT with CUDA support**
93
+
94
+ *For Cuda 11.8:*
95
+ ```bash
96
+ pip install RealtimeSTT --index-url https://download.pytorch.org/whl/cu118
97
+ ```
98
+
99
+
100
+ *For Cuda 12.X:*
101
+ ```bash
102
+ pip install RealtimeSTT --index-url https://download.pytorch.org/whl/cu121
103
+ ```
104
+
105
+ This command reinstalls PyTorch and Torchaudio with explicit support for CUDA.
106
+
107
+ ### Updating PyTorch Only for CUDA Support
108
+
109
+ If you already have RealtimeSTT installed but want to upgrade your PyTorch installation to leverage GPU support with CUDA, follow these instructions based on your specific CUDA version. This process is useful if you initially installed RealtimeSTT with only CPU support and now wish to enhance its performance with CUDA capabilities.
110
+
111
+ #### For CUDA 11.8:
112
+ To update PyTorch and Torchaudio to support CUDA 11.8, use the following commands:
113
+
114
+ ```bash
115
+ pip install torch==2.3.1+cu118 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu118
116
+ ```
117
+
118
+ #### For CUDA 12.X:
119
+ To update PyTorch and Torchaudio to support CUDA 12.X, execute the following:
120
+
121
+ ```bash
122
+ pip install torch==2.3.1+cu121 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu121
123
+ ```
124
+
125
+ Replace `2.3.1` with the version of CUDA that matches your system and requirements.
126
+
127
+ ### Steps That Might Be Necessary Before
87
128
 
88
129
  > **Note**: *To check if your NVIDIA GPU supports CUDA, visit the [official CUDA GPUs list](https://developer.nvidia.com/cuda-gpus).*
89
130
 
90
- To use RealtimeSTT with GPU support via CUDA please follow these steps:
131
+ If you didn't use CUDA models before, some additional steps might be needed one time before installation. These steps prepare the system for CUDA support and installation of the **GPU-optimized** installation. This is recommended for those who require **better performance** and have a compatible NVIDIA GPU. To use RealtimeSTT with GPU support via CUDA please also follow these steps:
91
132
 
92
- 1. **Install NVIDIA CUDA Toolkit 11.8**:
93
- - Visit [NVIDIA CUDA Toolkit Archive](https://developer.nvidia.com/cuda-11-8-0-download-archive).
133
+ 1. **Install NVIDIA CUDA Toolkit**:
134
+ - select between CUDA 11.8 or CUDA 12.X Toolkit
135
+ - for 12.X visit [NVIDIA CUDA Toolkit Archive](https://developer.nvidia.com/cuda-toolkit-archive) and select latest version.
136
+ - for 11.8 visit [NVIDIA CUDA Toolkit 11.8](https://developer.nvidia.com/cuda-11-8-0-download-archive).
94
137
  - Select operating system and version.
95
138
  - Download and install the software.
96
139
 
97
- 2. **Install NVIDIA cuDNN 8.7.0 for CUDA 11.x**:
98
- - Visit [NVIDIA cuDNN Archive](https://developer.nvidia.com/rdp/cudnn-archive).
99
- - Click on "Download cuDNN v8.7.0 (November 28th, 2022), for CUDA 11.x".
100
- - Download and install the software.
101
-
140
+ 2. **Install NVIDIA cuDNN**:
141
+ - select between CUDA 11.8 or CUDA 12.X Toolkit
142
+ - for 12.X visit [cuDNN Downloads](https://developer.nvidia.com/cudnn-downloads).
143
+ - Select operating system and version.
144
+ - Download and install the software.
145
+ - for 11.8 visit [NVIDIA cuDNN Archive](https://developer.nvidia.com/rdp/cudnn-archive).
146
+ - Click on "Download cuDNN v8.7.0 (November 28th, 2022), for CUDA 11.x".
147
+ - Download and install the software.
148
+
102
149
  3. **Install ffmpeg**:
103
150
 
104
151
  > **Note**: *Installation of ffmpeg might not actually be needed to operate RealtimeSTT* <sup> *thanks to jgilbert2017 for pointing this out</sup>
@@ -137,12 +184,6 @@ To use RealtimeSTT with GPU support via CUDA please follow these steps:
137
184
  scoop install ffmpeg
138
185
  ```
139
186
 
140
- 4. **Install PyTorch with CUDA support**:
141
- ```bash
142
- pip uninstall torch
143
- pip install torch==2.2.2+cu118 torchaudio==2.2.2 --index-url https://download.pytorch.org/whl/cu118
144
- ```
145
-
146
187
  ## Quick Start
147
188
 
148
189
  Basic usage:
@@ -279,6 +320,8 @@ When you initialize the `AudioToTextRecorder` class, you have various options to
279
320
 
280
321
  - **gpu_device_index** (int, default=0): GPU Device Index to use. The model can also be loaded on multiple GPUs by passing a list of IDs (e.g. [0, 1, 2, 3]).
281
322
 
323
+ - **device** (str, default="cuda"): Device for model to use. Can either be "cuda" or "cpu".
324
+
282
325
  - **on_recording_start**: A callable function triggered when recording starts.
283
326
 
284
327
  - **on_recording_stop**: A callable function triggered when recording ends.
@@ -346,7 +389,13 @@ When you initialize the `AudioToTextRecorder` class, you have various options to
346
389
 
347
390
  #### Wake Word Parameters
348
391
 
349
- - **wake_words** (str, default=""): Wake words for initiating the recording. Multiple wake words can be provided as a comma-separated string. Supported wake words are: alexa, americano, blueberry, bumblebee, computer, grapefruits, grasshopper, hey google, hey siri, jarvis, ok google, picovoice, porcupine, terminator
392
+ - **wakeword_backend** (str, default="pvporcupine"): Specifies the backend library to use for wake word detection. Supported options include 'pvporcupine' for using the Porcupine wake word engine or 'oww' for using the OpenWakeWord engine.
393
+
394
+ - **openwakeword_model_paths** (str, default=None): Comma-separated paths to model files for the openwakeword library. These paths point to custom models that can be used for wake word detection when the openwakeword library is selected as the wakeword_backend.
395
+
396
+ - **openwakeword_inference_framework** (str, default="onnx"): Specifies the inference framework to use with the openwakeword library. Can be either 'onnx' for Open Neural Network Exchange format or 'tflite' for TensorFlow Lite.
397
+
398
+ - **wake_words** (str, default=""): Initiate recording when using the 'pvporcupine' wakeword backend. Multiple wake words can be provided as a comma-separated string. Supported wake words are: alexa, americano, blueberry, bumblebee, computer, grapefruits, grasshopper, hey google, hey siri, jarvis, ok google, picovoice, porcupine, terminator. For the 'openwakeword' backend, wake words are automatically extracted from the provided model files, so specifying them here is not necessary.
350
399
 
351
400
  - **wake_words_sensitivity** (float, default=0.6): Sensitivity level for wake word detection (0 for least sensitive, 1 for most sensitive).
352
401
 
@@ -354,6 +403,8 @@ When you initialize the `AudioToTextRecorder` class, you have various options to
354
403
 
355
404
  - **wake_word_timeout** (float, default=5): Duration in seconds after a wake word is recognized. If no subsequent voice activity is detected within this window, the system transitions back to an inactive state, awaiting the next wake word or voice activation.
356
405
 
406
+ - **wake_word_buffer_duration** (float, default=0.1): Duration in seconds to buffer audio data during wake word detection. This helps in cutting out the wake word from the recording buffer so it does not falsely get detected along with the following spoken text, ensuring cleaner and more accurate transcription start triggers. Increase this if parts of the wake word get detected as text.
407
+
357
408
  - **on_wakeword_detected**: A callable function triggered when a wake word is detected.
358
409
 
359
410
  - **on_wakeword_timeout**: A callable function triggered when the system goes back to an inactive state after when no speech was detected after wake word activation.
@@ -362,6 +413,33 @@ When you initialize the `AudioToTextRecorder` class, you have various options to
362
413
 
363
414
  - **on_wakeword_detection_end**: A callable function triggered when stopping to listen for wake words (e.g. because of timeout or wake word detected)
364
415
 
416
+ ## OpenWakeWord
417
+
418
+ ### Training models
419
+
420
+ Look [here](https://github.com/dscripka/openWakeWord?tab=readme-ov-file#training-new-models) for information about how to train your own OpenWakeWord models. You can use a [simple Google Colab notebook](https://colab.research.google.com/drive/1q1oe2zOyZp7UsB3jJiQ1IFn8z5YfjwEb?usp=sharing) for a start or use a [more detailed notebook](https://github.com/dscripka/openWakeWord/blob/main/notebooks/automatic_model_training.ipynb) that enables more customization (can produce high quality models, but requires more development experience).
421
+
422
+ ### Convert model to ONNX format
423
+
424
+ You might need to use tf2onnx to convert tensorflow tflite models to onnx format:
425
+
426
+ ```bash
427
+ pip install -U tf2onnx
428
+ python -m tf2onnx.convert --tflite my_model_filename.tflite --output my_model_filename.onnx
429
+ ```
430
+
431
+ ### Configure RealtimeSTT
432
+
433
+ Suggested starting parameters for OpenWakeWord usage:
434
+ ```python
435
+ with AudioToTextRecorder(
436
+ wakeword_backend="oww",
437
+ wake_words_sensitivity=0.35,
438
+ openwakeword_model_paths="word1.onnx,word2.onnx",
439
+ wake_word_buffer_duration=1,
440
+ ) as recorder:
441
+ ```
442
+
365
443
  ## Contribution
366
444
 
367
445
  Contributions are always welcome!
@@ -3,6 +3,10 @@
3
3
 
4
4
  *Easy-to-use, low-latency speech-to-text library for realtime applications*
5
5
 
6
+ ## New
7
+
8
+ Custom wake words with [OpenWakeWord](#openwakeword). Thanks to the [developers](https://github.com/dscripka/openWakeWord) of this!
9
+
6
10
  ## About the Project
7
11
 
8
12
  RealtimeSTT listens to the microphone and transcribes voice into text.
@@ -18,7 +22,7 @@ https://github.com/KoljaB/RealtimeSTT/assets/7604638/207cb9a2-4482-48e7-9d2b-072
18
22
 
19
23
  ### Updates
20
24
 
21
- Latest Version: v0.1.15
25
+ Latest Version: v0.2.0
22
26
 
23
27
  See [release history](https://github.com/KoljaB/RealtimeSTT/releases).
24
28
 
@@ -42,10 +46,10 @@ This library uses:
42
46
  - **Speech-To-Text**
43
47
  - [Faster_Whisper](https://github.com/guillaumekln/faster-whisper) for instant (GPU-accelerated) transcription.
44
48
  - **Wake Word Detection**
45
- - [Porcupine](https://github.com/Picovoice/porcupine) for wake word detection.
49
+ - [Porcupine](https://github.com/Picovoice/porcupine) or [OpenWakeWord](https://github.com/dscripka/openWakeWord) for wake word detection.
46
50
 
47
- *These components represent the "industry standard" for cutting-edge applications, providing the most modern and effective foundation for building high-end solutions.*
48
51
 
52
+ *These components represent the "industry standard" for cutting-edge applications, providing the most modern and effective foundation for building high-end solutions.*
49
53
 
50
54
  ## Installation
51
55
 
@@ -59,22 +63,64 @@ Although it is possible to run RealtimeSTT with a CPU installation only (use a s
59
63
 
60
64
  ### GPU Support with CUDA (recommended)
61
65
 
62
- Additional steps are needed for a **GPU-optimized** installation. These steps are recommended for those who require **better performance** and have a compatible NVIDIA GPU.
66
+
67
+ **Install RealtimeSTT with CUDA support**
68
+
69
+ *For Cuda 11.8:*
70
+ ```bash
71
+ pip install RealtimeSTT --index-url https://download.pytorch.org/whl/cu118
72
+ ```
73
+
74
+
75
+ *For Cuda 12.X:*
76
+ ```bash
77
+ pip install RealtimeSTT --index-url https://download.pytorch.org/whl/cu121
78
+ ```
79
+
80
+ This command reinstalls PyTorch and Torchaudio with explicit support for CUDA.
81
+
82
+ ### Updating PyTorch Only for CUDA Support
83
+
84
+ If you already have RealtimeSTT installed but want to upgrade your PyTorch installation to leverage GPU support with CUDA, follow these instructions based on your specific CUDA version. This process is useful if you initially installed RealtimeSTT with only CPU support and now wish to enhance its performance with CUDA capabilities.
85
+
86
+ #### For CUDA 11.8:
87
+ To update PyTorch and Torchaudio to support CUDA 11.8, use the following commands:
88
+
89
+ ```bash
90
+ pip install torch==2.3.1+cu118 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu118
91
+ ```
92
+
93
+ #### For CUDA 12.X:
94
+ To update PyTorch and Torchaudio to support CUDA 12.X, execute the following:
95
+
96
+ ```bash
97
+ pip install torch==2.3.1+cu121 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu121
98
+ ```
99
+
100
+ Replace `2.3.1` with the version of CUDA that matches your system and requirements.
101
+
102
+ ### Steps That Might Be Necessary Before
63
103
 
64
104
  > **Note**: *To check if your NVIDIA GPU supports CUDA, visit the [official CUDA GPUs list](https://developer.nvidia.com/cuda-gpus).*
65
105
 
66
- To use RealtimeSTT with GPU support via CUDA please follow these steps:
106
+ If you didn't use CUDA models before, some additional steps might be needed one time before installation. These steps prepare the system for CUDA support and installation of the **GPU-optimized** installation. This is recommended for those who require **better performance** and have a compatible NVIDIA GPU. To use RealtimeSTT with GPU support via CUDA please also follow these steps:
67
107
 
68
- 1. **Install NVIDIA CUDA Toolkit 11.8**:
69
- - Visit [NVIDIA CUDA Toolkit Archive](https://developer.nvidia.com/cuda-11-8-0-download-archive).
108
+ 1. **Install NVIDIA CUDA Toolkit**:
109
+ - select between CUDA 11.8 or CUDA 12.X Toolkit
110
+ - for 12.X visit [NVIDIA CUDA Toolkit Archive](https://developer.nvidia.com/cuda-toolkit-archive) and select latest version.
111
+ - for 11.8 visit [NVIDIA CUDA Toolkit 11.8](https://developer.nvidia.com/cuda-11-8-0-download-archive).
70
112
  - Select operating system and version.
71
113
  - Download and install the software.
72
114
 
73
- 2. **Install NVIDIA cuDNN 8.7.0 for CUDA 11.x**:
74
- - Visit [NVIDIA cuDNN Archive](https://developer.nvidia.com/rdp/cudnn-archive).
75
- - Click on "Download cuDNN v8.7.0 (November 28th, 2022), for CUDA 11.x".
76
- - Download and install the software.
77
-
115
+ 2. **Install NVIDIA cuDNN**:
116
+ - select between CUDA 11.8 or CUDA 12.X Toolkit
117
+ - for 12.X visit [cuDNN Downloads](https://developer.nvidia.com/cudnn-downloads).
118
+ - Select operating system and version.
119
+ - Download and install the software.
120
+ - for 11.8 visit [NVIDIA cuDNN Archive](https://developer.nvidia.com/rdp/cudnn-archive).
121
+ - Click on "Download cuDNN v8.7.0 (November 28th, 2022), for CUDA 11.x".
122
+ - Download and install the software.
123
+
78
124
  3. **Install ffmpeg**:
79
125
 
80
126
  > **Note**: *Installation of ffmpeg might not actually be needed to operate RealtimeSTT* <sup> *thanks to jgilbert2017 for pointing this out</sup>
@@ -113,12 +159,6 @@ To use RealtimeSTT with GPU support via CUDA please follow these steps:
113
159
  scoop install ffmpeg
114
160
  ```
115
161
 
116
- 4. **Install PyTorch with CUDA support**:
117
- ```bash
118
- pip uninstall torch
119
- pip install torch==2.2.2+cu118 torchaudio==2.2.2 --index-url https://download.pytorch.org/whl/cu118
120
- ```
121
-
122
162
  ## Quick Start
123
163
 
124
164
  Basic usage:
@@ -255,6 +295,8 @@ When you initialize the `AudioToTextRecorder` class, you have various options to
255
295
 
256
296
  - **gpu_device_index** (int, default=0): GPU Device Index to use. The model can also be loaded on multiple GPUs by passing a list of IDs (e.g. [0, 1, 2, 3]).
257
297
 
298
+ - **device** (str, default="cuda"): Device for model to use. Can either be "cuda" or "cpu".
299
+
258
300
  - **on_recording_start**: A callable function triggered when recording starts.
259
301
 
260
302
  - **on_recording_stop**: A callable function triggered when recording ends.
@@ -322,7 +364,13 @@ When you initialize the `AudioToTextRecorder` class, you have various options to
322
364
 
323
365
  #### Wake Word Parameters
324
366
 
325
- - **wake_words** (str, default=""): Wake words for initiating the recording. Multiple wake words can be provided as a comma-separated string. Supported wake words are: alexa, americano, blueberry, bumblebee, computer, grapefruits, grasshopper, hey google, hey siri, jarvis, ok google, picovoice, porcupine, terminator
367
+ - **wakeword_backend** (str, default="pvporcupine"): Specifies the backend library to use for wake word detection. Supported options include 'pvporcupine' for using the Porcupine wake word engine or 'oww' for using the OpenWakeWord engine.
368
+
369
+ - **openwakeword_model_paths** (str, default=None): Comma-separated paths to model files for the openwakeword library. These paths point to custom models that can be used for wake word detection when the openwakeword library is selected as the wakeword_backend.
370
+
371
+ - **openwakeword_inference_framework** (str, default="onnx"): Specifies the inference framework to use with the openwakeword library. Can be either 'onnx' for Open Neural Network Exchange format or 'tflite' for TensorFlow Lite.
372
+
373
+ - **wake_words** (str, default=""): Initiate recording when using the 'pvporcupine' wakeword backend. Multiple wake words can be provided as a comma-separated string. Supported wake words are: alexa, americano, blueberry, bumblebee, computer, grapefruits, grasshopper, hey google, hey siri, jarvis, ok google, picovoice, porcupine, terminator. For the 'openwakeword' backend, wake words are automatically extracted from the provided model files, so specifying them here is not necessary.
326
374
 
327
375
  - **wake_words_sensitivity** (float, default=0.6): Sensitivity level for wake word detection (0 for least sensitive, 1 for most sensitive).
328
376
 
@@ -330,6 +378,8 @@ When you initialize the `AudioToTextRecorder` class, you have various options to
330
378
 
331
379
  - **wake_word_timeout** (float, default=5): Duration in seconds after a wake word is recognized. If no subsequent voice activity is detected within this window, the system transitions back to an inactive state, awaiting the next wake word or voice activation.
332
380
 
381
+ - **wake_word_buffer_duration** (float, default=0.1): Duration in seconds to buffer audio data during wake word detection. This helps in cutting out the wake word from the recording buffer so it does not falsely get detected along with the following spoken text, ensuring cleaner and more accurate transcription start triggers. Increase this if parts of the wake word get detected as text.
382
+
333
383
  - **on_wakeword_detected**: A callable function triggered when a wake word is detected.
334
384
 
335
385
  - **on_wakeword_timeout**: A callable function triggered when the system goes back to an inactive state after when no speech was detected after wake word activation.
@@ -338,6 +388,33 @@ When you initialize the `AudioToTextRecorder` class, you have various options to
338
388
 
339
389
  - **on_wakeword_detection_end**: A callable function triggered when stopping to listen for wake words (e.g. because of timeout or wake word detected)
340
390
 
391
+ ## OpenWakeWord
392
+
393
+ ### Training models
394
+
395
+ Look [here](https://github.com/dscripka/openWakeWord?tab=readme-ov-file#training-new-models) for information about how to train your own OpenWakeWord models. You can use a [simple Google Colab notebook](https://colab.research.google.com/drive/1q1oe2zOyZp7UsB3jJiQ1IFn8z5YfjwEb?usp=sharing) for a start or use a [more detailed notebook](https://github.com/dscripka/openWakeWord/blob/main/notebooks/automatic_model_training.ipynb) that enables more customization (can produce high quality models, but requires more development experience).
396
+
397
+ ### Convert model to ONNX format
398
+
399
+ You might need to use tf2onnx to convert tensorflow tflite models to onnx format:
400
+
401
+ ```bash
402
+ pip install -U tf2onnx
403
+ python -m tf2onnx.convert --tflite my_model_filename.tflite --output my_model_filename.onnx
404
+ ```
405
+
406
+ ### Configure RealtimeSTT
407
+
408
+ Suggested starting parameters for OpenWakeWord usage:
409
+ ```python
410
+ with AudioToTextRecorder(
411
+ wakeword_backend="oww",
412
+ wake_words_sensitivity=0.35,
413
+ openwakeword_model_paths="word1.onnx,word2.onnx",
414
+ wake_word_buffer_duration=1,
415
+ ) as recorder:
416
+ ```
417
+
341
418
  ## Contribution
342
419
 
343
420
  Contributions are always welcome!
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.1
2
- Name: RealTimeSTT
3
- Version: 0.1.16
2
+ Name: RealtimeSTT
3
+ Version: 0.2.0
4
4
  Summary: A fast Voice Activity Detection and Transcription System
5
5
  Home-page: https://github.com/KoljaB/RealTimeSTT
6
6
  Author: Kolja Beigel
@@ -17,16 +17,21 @@ Requires-Dist: faster-whisper==1.0.2
17
17
  Requires-Dist: pvporcupine==1.9.5
18
18
  Requires-Dist: webrtcvad==2.0.10
19
19
  Requires-Dist: halo==0.0.31
20
- Requires-Dist: torch==2.3.0
21
- Requires-Dist: torchaudio==2.3.0
20
+ Requires-Dist: torch==2.3.1
21
+ Requires-Dist: torchaudio==2.3.1
22
22
  Requires-Dist: scipy==1.12.0
23
23
  Requires-Dist: websockets==v12.0
24
+ Requires-Dist: openwakeword==0.6.0
24
25
 
25
26
 
26
27
  # RealtimeSTT
27
28
 
28
29
  *Easy-to-use, low-latency speech-to-text library for realtime applications*
29
30
 
31
+ ## New
32
+
33
+ Custom wake words with [OpenWakeWord](#openwakeword). Thanks to the [developers](https://github.com/dscripka/openWakeWord) of this!
34
+
30
35
  ## About the Project
31
36
 
32
37
  RealtimeSTT listens to the microphone and transcribes voice into text.
@@ -42,7 +47,7 @@ https://github.com/KoljaB/RealtimeSTT/assets/7604638/207cb9a2-4482-48e7-9d2b-072
42
47
 
43
48
  ### Updates
44
49
 
45
- Latest Version: v0.1.15
50
+ Latest Version: v0.2.0
46
51
 
47
52
  See [release history](https://github.com/KoljaB/RealtimeSTT/releases).
48
53
 
@@ -66,10 +71,10 @@ This library uses:
66
71
  - **Speech-To-Text**
67
72
  - [Faster_Whisper](https://github.com/guillaumekln/faster-whisper) for instant (GPU-accelerated) transcription.
68
73
  - **Wake Word Detection**
69
- - [Porcupine](https://github.com/Picovoice/porcupine) for wake word detection.
74
+ - [Porcupine](https://github.com/Picovoice/porcupine) or [OpenWakeWord](https://github.com/dscripka/openWakeWord) for wake word detection.
70
75
 
71
- *These components represent the "industry standard" for cutting-edge applications, providing the most modern and effective foundation for building high-end solutions.*
72
76
 
77
+ *These components represent the "industry standard" for cutting-edge applications, providing the most modern and effective foundation for building high-end solutions.*
73
78
 
74
79
  ## Installation
75
80
 
@@ -83,22 +88,64 @@ Although it is possible to run RealtimeSTT with a CPU installation only (use a s
83
88
 
84
89
  ### GPU Support with CUDA (recommended)
85
90
 
86
- Additional steps are needed for a **GPU-optimized** installation. These steps are recommended for those who require **better performance** and have a compatible NVIDIA GPU.
91
+
92
+ **Install RealtimeSTT with CUDA support**
93
+
94
+ *For Cuda 11.8:*
95
+ ```bash
96
+ pip install RealtimeSTT --index-url https://download.pytorch.org/whl/cu118
97
+ ```
98
+
99
+
100
+ *For Cuda 12.X:*
101
+ ```bash
102
+ pip install RealtimeSTT --index-url https://download.pytorch.org/whl/cu121
103
+ ```
104
+
105
+ This command reinstalls PyTorch and Torchaudio with explicit support for CUDA.
106
+
107
+ ### Updating PyTorch Only for CUDA Support
108
+
109
+ If you already have RealtimeSTT installed but want to upgrade your PyTorch installation to leverage GPU support with CUDA, follow these instructions based on your specific CUDA version. This process is useful if you initially installed RealtimeSTT with only CPU support and now wish to enhance its performance with CUDA capabilities.
110
+
111
+ #### For CUDA 11.8:
112
+ To update PyTorch and Torchaudio to support CUDA 11.8, use the following commands:
113
+
114
+ ```bash
115
+ pip install torch==2.3.1+cu118 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu118
116
+ ```
117
+
118
+ #### For CUDA 12.X:
119
+ To update PyTorch and Torchaudio to support CUDA 12.X, execute the following:
120
+
121
+ ```bash
122
+ pip install torch==2.3.1+cu121 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu121
123
+ ```
124
+
125
+ Replace `2.3.1` with the version of CUDA that matches your system and requirements.
126
+
127
+ ### Steps That Might Be Necessary Before
87
128
 
88
129
  > **Note**: *To check if your NVIDIA GPU supports CUDA, visit the [official CUDA GPUs list](https://developer.nvidia.com/cuda-gpus).*
89
130
 
90
- To use RealtimeSTT with GPU support via CUDA please follow these steps:
131
+ If you didn't use CUDA models before, some additional steps might be needed one time before installation. These steps prepare the system for CUDA support and installation of the **GPU-optimized** installation. This is recommended for those who require **better performance** and have a compatible NVIDIA GPU. To use RealtimeSTT with GPU support via CUDA please also follow these steps:
91
132
 
92
- 1. **Install NVIDIA CUDA Toolkit 11.8**:
93
- - Visit [NVIDIA CUDA Toolkit Archive](https://developer.nvidia.com/cuda-11-8-0-download-archive).
133
+ 1. **Install NVIDIA CUDA Toolkit**:
134
+ - select between CUDA 11.8 or CUDA 12.X Toolkit
135
+ - for 12.X visit [NVIDIA CUDA Toolkit Archive](https://developer.nvidia.com/cuda-toolkit-archive) and select latest version.
136
+ - for 11.8 visit [NVIDIA CUDA Toolkit 11.8](https://developer.nvidia.com/cuda-11-8-0-download-archive).
94
137
  - Select operating system and version.
95
138
  - Download and install the software.
96
139
 
97
- 2. **Install NVIDIA cuDNN 8.7.0 for CUDA 11.x**:
98
- - Visit [NVIDIA cuDNN Archive](https://developer.nvidia.com/rdp/cudnn-archive).
99
- - Click on "Download cuDNN v8.7.0 (November 28th, 2022), for CUDA 11.x".
100
- - Download and install the software.
101
-
140
+ 2. **Install NVIDIA cuDNN**:
141
+ - select between CUDA 11.8 or CUDA 12.X Toolkit
142
+ - for 12.X visit [cuDNN Downloads](https://developer.nvidia.com/cudnn-downloads).
143
+ - Select operating system and version.
144
+ - Download and install the software.
145
+ - for 11.8 visit [NVIDIA cuDNN Archive](https://developer.nvidia.com/rdp/cudnn-archive).
146
+ - Click on "Download cuDNN v8.7.0 (November 28th, 2022), for CUDA 11.x".
147
+ - Download and install the software.
148
+
102
149
  3. **Install ffmpeg**:
103
150
 
104
151
  > **Note**: *Installation of ffmpeg might not actually be needed to operate RealtimeSTT* <sup> *thanks to jgilbert2017 for pointing this out</sup>
@@ -137,12 +184,6 @@ To use RealtimeSTT with GPU support via CUDA please follow these steps:
137
184
  scoop install ffmpeg
138
185
  ```
139
186
 
140
- 4. **Install PyTorch with CUDA support**:
141
- ```bash
142
- pip uninstall torch
143
- pip install torch==2.2.2+cu118 torchaudio==2.2.2 --index-url https://download.pytorch.org/whl/cu118
144
- ```
145
-
146
187
  ## Quick Start
147
188
 
148
189
  Basic usage:
@@ -279,6 +320,8 @@ When you initialize the `AudioToTextRecorder` class, you have various options to
279
320
 
280
321
  - **gpu_device_index** (int, default=0): GPU Device Index to use. The model can also be loaded on multiple GPUs by passing a list of IDs (e.g. [0, 1, 2, 3]).
281
322
 
323
+ - **device** (str, default="cuda"): Device for model to use. Can either be "cuda" or "cpu".
324
+
282
325
  - **on_recording_start**: A callable function triggered when recording starts.
283
326
 
284
327
  - **on_recording_stop**: A callable function triggered when recording ends.
@@ -346,7 +389,13 @@ When you initialize the `AudioToTextRecorder` class, you have various options to
346
389
 
347
390
  #### Wake Word Parameters
348
391
 
349
- - **wake_words** (str, default=""): Wake words for initiating the recording. Multiple wake words can be provided as a comma-separated string. Supported wake words are: alexa, americano, blueberry, bumblebee, computer, grapefruits, grasshopper, hey google, hey siri, jarvis, ok google, picovoice, porcupine, terminator
392
+ - **wakeword_backend** (str, default="pvporcupine"): Specifies the backend library to use for wake word detection. Supported options include 'pvporcupine' for using the Porcupine wake word engine or 'oww' for using the OpenWakeWord engine.
393
+
394
+ - **openwakeword_model_paths** (str, default=None): Comma-separated paths to model files for the openwakeword library. These paths point to custom models that can be used for wake word detection when the openwakeword library is selected as the wakeword_backend.
395
+
396
+ - **openwakeword_inference_framework** (str, default="onnx"): Specifies the inference framework to use with the openwakeword library. Can be either 'onnx' for Open Neural Network Exchange format or 'tflite' for TensorFlow Lite.
397
+
398
+ - **wake_words** (str, default=""): Initiate recording when using the 'pvporcupine' wakeword backend. Multiple wake words can be provided as a comma-separated string. Supported wake words are: alexa, americano, blueberry, bumblebee, computer, grapefruits, grasshopper, hey google, hey siri, jarvis, ok google, picovoice, porcupine, terminator. For the 'openwakeword' backend, wake words are automatically extracted from the provided model files, so specifying them here is not necessary.
350
399
 
351
400
  - **wake_words_sensitivity** (float, default=0.6): Sensitivity level for wake word detection (0 for least sensitive, 1 for most sensitive).
352
401
 
@@ -354,6 +403,8 @@ When you initialize the `AudioToTextRecorder` class, you have various options to
354
403
 
355
404
  - **wake_word_timeout** (float, default=5): Duration in seconds after a wake word is recognized. If no subsequent voice activity is detected within this window, the system transitions back to an inactive state, awaiting the next wake word or voice activation.
356
405
 
406
+ - **wake_word_buffer_duration** (float, default=0.1): Duration in seconds to buffer audio data during wake word detection. This helps in cutting out the wake word from the recording buffer so it does not falsely get detected along with the following spoken text, ensuring cleaner and more accurate transcription start triggers. Increase this if parts of the wake word get detected as text.
407
+
357
408
  - **on_wakeword_detected**: A callable function triggered when a wake word is detected.
358
409
 
359
410
  - **on_wakeword_timeout**: A callable function triggered when the system goes back to an inactive state after when no speech was detected after wake word activation.
@@ -362,6 +413,33 @@ When you initialize the `AudioToTextRecorder` class, you have various options to
362
413
 
363
414
  - **on_wakeword_detection_end**: A callable function triggered when stopping to listen for wake words (e.g. because of timeout or wake word detected)
364
415
 
416
+ ## OpenWakeWord
417
+
418
+ ### Training models
419
+
420
+ Look [here](https://github.com/dscripka/openWakeWord?tab=readme-ov-file#training-new-models) for information about how to train your own OpenWakeWord models. You can use a [simple Google Colab notebook](https://colab.research.google.com/drive/1q1oe2zOyZp7UsB3jJiQ1IFn8z5YfjwEb?usp=sharing) for a start or use a [more detailed notebook](https://github.com/dscripka/openWakeWord/blob/main/notebooks/automatic_model_training.ipynb) that enables more customization (can produce high quality models, but requires more development experience).
421
+
422
+ ### Convert model to ONNX format
423
+
424
+ You might need to use tf2onnx to convert tensorflow tflite models to onnx format:
425
+
426
+ ```bash
427
+ pip install -U tf2onnx
428
+ python -m tf2onnx.convert --tflite my_model_filename.tflite --output my_model_filename.onnx
429
+ ```
430
+
431
+ ### Configure RealtimeSTT
432
+
433
+ Suggested starting parameters for OpenWakeWord usage:
434
+ ```python
435
+ with AudioToTextRecorder(
436
+ wakeword_backend="oww",
437
+ wake_words_sensitivity=0.35,
438
+ openwakeword_model_paths="word1.onnx,word2.onnx",
439
+ wake_word_buffer_duration=1,
440
+ ) as recorder:
441
+ ```
442
+
365
443
  ## Contribution
366
444
 
367
445
  Contributions are always welcome!
@@ -7,4 +7,9 @@ RealTimeSTT.egg-info/dependency_links.txt
7
7
  RealTimeSTT.egg-info/requires.txt
8
8
  RealTimeSTT.egg-info/top_level.txt
9
9
  RealtimeSTT/__init__.py
10
- RealtimeSTT/audio_recorder.py
10
+ RealtimeSTT/audio_recorder.py
11
+ RealtimeSTT.egg-info/PKG-INFO
12
+ RealtimeSTT.egg-info/SOURCES.txt
13
+ RealtimeSTT.egg-info/dependency_links.txt
14
+ RealtimeSTT.egg-info/requires.txt
15
+ RealtimeSTT.egg-info/top_level.txt
@@ -3,7 +3,8 @@ faster-whisper==1.0.2
3
3
  pvporcupine==1.9.5
4
4
  webrtcvad==2.0.10
5
5
  halo==0.0.31
6
- torch==2.3.0
7
- torchaudio==2.3.0
6
+ torch==2.3.1
7
+ torchaudio==2.3.1
8
8
  scipy==1.12.0
9
9
  websockets==v12.0
10
+ openwakeword==0.6.0
@@ -31,9 +31,11 @@ import torch.multiprocessing as mp
31
31
  import torch
32
32
  from typing import List, Union
33
33
  from ctypes import c_bool
34
+ from openwakeword.model import Model
34
35
  from scipy.signal import resample
35
36
  from scipy import signal
36
37
  import faster_whisper
38
+ import openwakeword
37
39
  import collections
38
40
  import numpy as np
39
41
  import pvporcupine
@@ -67,6 +69,7 @@ INIT_WAKE_WORDS_SENSITIVITY = 0.6
67
69
  INIT_PRE_RECORDING_BUFFER_DURATION = 1.0
68
70
  INIT_WAKE_WORD_ACTIVATION_DELAY = 0.0
69
71
  INIT_WAKE_WORD_TIMEOUT = 5.0
72
+ INIT_WAKE_WORD_BUFFER_DURATION = 0.1
70
73
  ALLOWED_LATENCY_LIMIT = 10
71
74
 
72
75
  TIME_SLEEP = 0.02
@@ -90,8 +93,9 @@ class AudioToTextRecorder:
90
93
  model: str = INIT_MODEL_TRANSCRIPTION,
91
94
  language: str = "",
92
95
  compute_type: str = "default",
93
- input_device_index: int = 0,
96
+ input_device_index: int = None,
94
97
  gpu_device_index: Union[int, List[int]] = 0,
98
+ device: str = "cuda",
95
99
  on_recording_start=None,
96
100
  on_recording_stop=None,
97
101
  on_transcription_start=None,
@@ -128,12 +132,16 @@ class AudioToTextRecorder:
128
132
  on_vad_detect_stop=None,
129
133
 
130
134
  # Wake word parameters
135
+ wakeword_backend: str = "pvporcupine",
136
+ openwakeword_model_paths: str = None,
137
+ openwakeword_inference_framework: str = "onnx",
131
138
  wake_words: str = "",
132
139
  wake_words_sensitivity: float = INIT_WAKE_WORDS_SENSITIVITY,
133
140
  wake_word_activation_delay: float = (
134
141
  INIT_WAKE_WORD_ACTIVATION_DELAY
135
142
  ),
136
143
  wake_word_timeout: float = INIT_WAKE_WORD_TIMEOUT,
144
+ wake_word_buffer_duration: float = INIT_WAKE_WORD_BUFFER_DURATION,
137
145
  on_wakeword_detected=None,
138
146
  on_wakeword_timeout=None,
139
147
  on_wakeword_detection_start=None,
@@ -173,6 +181,8 @@ class AudioToTextRecorder:
173
181
  IDs (e.g. [0, 1, 2, 3]). In that case, multiple transcriptions can
174
182
  run in parallel when transcribe() is called from multiple Python
175
183
  threads
184
+ - device (str, default="cuda"): Device for model to use. Can either be
185
+ "cuda" or "cpu".
176
186
  - on_recording_start (callable, default=None): Callback function to be
177
187
  called when recording of audio to be transcripted starts.
178
188
  - on_recording_stop (callable, default=None): Callback function to be
@@ -241,11 +251,26 @@ class AudioToTextRecorder:
241
251
  be called when the system listens for voice activity.
242
252
  - on_vad_detect_stop (callable, default=None): Callback function to be
243
253
  called when the system stops listening for voice activity.
254
+ - wakeword_backend (str, default="pvporcupine"): Specifies the backend
255
+ library to use for wake word detection. Supported options include
256
+ 'pvporcupine' for using the Porcupine wake word engine or 'oww' for
257
+ using the OpenWakeWord engine.
258
+ - openwakeword_model_paths (str, default=None): Comma-separated paths
259
+ to model files for the openwakeword library. These paths point to
260
+ custom models that can be used for wake word detection when the
261
+ openwakeword library is selected as the wakeword_backend.
262
+ - openwakeword_inference_framework (str, default="onnx"): Specifies
263
+ the inference framework to use with the openwakeword library.
264
+ Can be either 'onnx' for Open Neural Network Exchange format
265
+ or 'tflite' for TensorFlow Lite.
244
266
  - wake_words (str, default=""): Comma-separated string of wake words to
245
- initiate recording. Supported wake words include:
246
- 'alexa', 'americano', 'blueberry', 'bumblebee', 'computer',
247
- 'grapefruits', 'grasshopper', 'hey google', 'hey siri', 'jarvis',
248
- 'ok google', 'picovoice', 'porcupine', 'terminator'.
267
+ initiate recording when using the 'pvporcupine' wakeword backend.
268
+ Supported wake words include: 'alexa', 'americano', 'blueberry',
269
+ 'bumblebee', 'computer', 'grapefruits', 'grasshopper', 'hey google',
270
+ 'hey siri', 'jarvis', 'ok google', 'picovoice', 'porcupine',
271
+ 'terminator'. For the 'openwakeword' backend, wake words are
272
+ automatically extracted from the provided model files, so specifying
273
+ them here is not necessary.
249
274
  - wake_words_sensitivity (float, default=0.5): Sensitivity for wake
250
275
  word detection, ranging from 0 (least sensitive) to 1 (most
251
276
  sensitive). Default is 0.5.
@@ -257,6 +282,12 @@ class AudioToTextRecorder:
257
282
  wake word is recognized. If no subsequent voice activity is
258
283
  detected within this window, the system transitions back to an
259
284
  inactive state, awaiting the next wake word or voice activation.
285
+ - wake_word_buffer_duration (float, default=0.1): Duration in seconds
286
+ to buffer audio data during wake word detection. This helps in
287
+ cutting out the wake word from the recording buffer so it does not
288
+ falsely get detected along with the following spoken text, ensuring
289
+ cleaner and more accurate transcription start triggers.
290
+ Increase this if parts of the wake word get detected as text.
260
291
  - on_wakeword_detected (callable, default=None): Callback function to
261
292
  be called when a wake word is detected.
262
293
  - on_wakeword_timeout (callable, default=None): Callback function to
@@ -298,9 +329,11 @@ class AudioToTextRecorder:
298
329
  self.compute_type = compute_type
299
330
  self.input_device_index = input_device_index
300
331
  self.gpu_device_index = gpu_device_index
332
+ self.device = device
301
333
  self.wake_words = wake_words
302
334
  self.wake_word_activation_delay = wake_word_activation_delay
303
335
  self.wake_word_timeout = wake_word_timeout
336
+ self.wake_word_buffer_duration = wake_word_buffer_duration
304
337
  self.ensure_sentence_starting_uppercase = (
305
338
  ensure_sentence_starting_uppercase
306
339
  )
@@ -368,6 +401,7 @@ class AudioToTextRecorder:
368
401
  self.last_transcription_bytes = None
369
402
  self.initial_prompt = initial_prompt
370
403
  self.suppress_tokens = suppress_tokens
404
+ self.use_wake_words = wake_words or wakeword_backend in {'oww', 'openwakeword', 'openwakewords'}
371
405
 
372
406
  # Initialize the logging configuration with the specified level
373
407
  log_format = 'RealTimeSTT: %(name)s - %(levelname)s - %(message)s'
@@ -397,7 +431,7 @@ class AudioToTextRecorder:
397
431
  logging.debug("Explicitly setting the multiprocessing start method to 'spawn'")
398
432
  mp.set_start_method('spawn')
399
433
  except RuntimeError as e:
400
- logging.debug("Start method has already been set. Details:", e)
434
+ logging.debug(f"Start method has already been set. Details: {e}")
401
435
 
402
436
  logging.info("Starting RealTimeSTT")
403
437
 
@@ -406,6 +440,9 @@ class AudioToTextRecorder:
406
440
  self.main_transcription_ready_event = mp.Event()
407
441
  self.parent_transcription_pipe, child_transcription_pipe = mp.Pipe()
408
442
 
443
+ # Set device for model
444
+ self.device = "cuda" if self.device == "cuda" and torch.cuda.is_available() else "cpu"
445
+
409
446
  self.transcript_process = mp.Process(
410
447
  target=AudioToTextRecorder._transcription_worker,
411
448
  args=(
@@ -413,6 +450,7 @@ class AudioToTextRecorder:
413
450
  model,
414
451
  self.compute_type,
415
452
  self.gpu_device_index,
453
+ self.device,
416
454
  self.main_transcription_ready_event,
417
455
  self.shutdown_event,
418
456
  self.interrupt_stop_event,
@@ -452,7 +490,7 @@ class AudioToTextRecorder:
452
490
  )
453
491
  self.realtime_model_type = faster_whisper.WhisperModel(
454
492
  model_size_or_path=self.realtime_model_type,
455
- device='cuda' if torch.cuda.is_available() else 'cpu',
493
+ device=self.device,
456
494
  compute_type=self.compute_type,
457
495
  device_index=self.gpu_device_index
458
496
  )
@@ -467,33 +505,82 @@ class AudioToTextRecorder:
467
505
  "transcription model initialized successfully")
468
506
 
469
507
  # Setup wake word detection
470
- if wake_words:
508
+ if wake_words or wakeword_backend in {'oww', 'openwakeword', 'openwakewords'}:
509
+ self.wakeword_backend = wakeword_backend
471
510
 
472
511
  self.wake_words_list = [
473
512
  word.strip() for word in wake_words.lower().split(',')
474
513
  ]
475
- sensitivity_list = [
514
+ self.wake_words_sensitivity = wake_words_sensitivity
515
+ self.wake_words_sensitivities = [
476
516
  float(wake_words_sensitivity)
477
517
  for _ in range(len(self.wake_words_list))
478
518
  ]
479
519
 
480
- try:
481
- self.porcupine = pvporcupine.create(
482
- keywords=self.wake_words_list,
483
- sensitivities=sensitivity_list
484
- )
485
- self.buffer_size = self.porcupine.frame_length
486
- self.sample_rate = self.porcupine.sample_rate
520
+ match self.wakeword_backend:
521
+ case 'pvp' | 'pvporcupine':
522
+ try:
523
+ self.porcupine = pvporcupine.create(
524
+ keywords=self.wake_words_list,
525
+ sensitivities=self.wake_words_sensitivities
526
+ )
527
+ self.buffer_size = self.porcupine.frame_length
528
+ self.sample_rate = self.porcupine.sample_rate
487
529
 
488
- except Exception as e:
489
- logging.exception("Error initializing porcupine "
490
- f"wake word detection engine: {e}"
491
- )
492
- raise
530
+ except Exception as e:
531
+ logging.exception(
532
+ "Error initializing porcupine "
533
+ f"wake word detection engine: {e}"
534
+ )
535
+ raise
536
+
537
+ logging.debug(
538
+ "Porcupine wake word detection engine initialized successfully"
539
+ )
540
+
541
+ case 'oww' | 'openwakeword' | 'openwakewords':
542
+ openwakeword.utils.download_models()
543
+
544
+ try:
545
+ if openwakeword_model_paths:
546
+ model_paths = openwakeword_model_paths.split(',')
547
+ self.owwModel = Model(
548
+ wakeword_models=model_paths,
549
+ inference_framework=openwakeword_inference_framework
550
+ )
551
+ logging.info(
552
+ "Successfully loaded wakeword model(s): "
553
+ f"{openwakeword_model_paths}"
554
+ )
555
+ else:
556
+ self.owwModel = Model(
557
+ inference_framework=openwakeword_inference_framework)
558
+
559
+ self.oww_n_models = len(self.owwModel.models.keys())
560
+ if not self.oww_n_models:
561
+ logging.error(
562
+ "No wake word models loaded."
563
+ )
564
+
565
+ for model_key in self.owwModel.models.keys():
566
+ logging.info(
567
+ "Successfully loaded openwakeword model: "
568
+ f"{model_key}"
569
+ )
570
+
571
+ except Exception as e:
572
+ logging.exception(
573
+ "Error initializing openwakeword "
574
+ f"wake word detection engine: {e}"
575
+ )
576
+ raise
577
+
578
+ logging.debug(
579
+ "Open wake word detection engine initialized successfully"
580
+ )
581
+ case _:
582
+ logging.exception("Wakeword engine {} unknown/unsupported. Please specify one of: pvporcupine, openwakeword.")
493
583
 
494
- logging.debug("Porcupine wake word detection "
495
- "engine initialized successfully"
496
- )
497
584
 
498
585
  # Setup voice activity detection model WebRTC
499
586
  try:
@@ -566,6 +653,7 @@ class AudioToTextRecorder:
566
653
  model_path,
567
654
  compute_type,
568
655
  gpu_device_index,
656
+ device,
569
657
  ready_event,
570
658
  shutdown_event,
571
659
  interrupt_stop_event,
@@ -593,6 +681,7 @@ class AudioToTextRecorder:
593
681
  compute_type (str): Specifies the type of computation to be used
594
682
  for transcription.
595
683
  gpu_device_index (int): Device ID to use.
684
+ device (str): Device for model to use.
596
685
  ready_event (threading.Event): An event that is set when the
597
686
  transcription model is successfully initialized and ready.
598
687
  shutdown_event (threading.Event): An event that, when set,
@@ -616,7 +705,7 @@ class AudioToTextRecorder:
616
705
  try:
617
706
  model = faster_whisper.WhisperModel(
618
707
  model_size_or_path=model_path,
619
- device='cuda' if torch.cuda.is_available() else 'cpu',
708
+ device=device,
620
709
  compute_type=compute_type,
621
710
  device_index=gpu_device_index,
622
711
  )
@@ -698,6 +787,9 @@ class AudioToTextRecorder:
698
787
  """
699
788
  try:
700
789
  audio_interface = pyaudio.PyAudio()
790
+ if input_device_index is None:
791
+ default_device = audio_interface.get_default_input_device_info()
792
+ input_device_index = default_device['index']
701
793
  stream = audio_interface.open(
702
794
  rate=sample_rate,
703
795
  format=pyaudio.paInt16,
@@ -852,6 +944,46 @@ class AudioToTextRecorder:
852
944
  logging.error(result)
853
945
  raise Exception(result)
854
946
 
947
+ def _process_wakeword(self, data):
948
+ """
949
+ Processes audio data to detect wake words.
950
+ """
951
+ match self.wakeword_backend:
952
+ case 'pvp' | 'pvporcupine':
953
+ pcm = struct.unpack_from(
954
+ "h" * self.buffer_size,
955
+ data
956
+ )
957
+ porcupine_index = self.porcupine.process(pcm)
958
+ if self.debug_mode:
959
+ print (f"wake words porcupine_index: {porcupine_index}")
960
+ return self.porcupine.process(pcm)
961
+
962
+ case 'oww' | 'openwakeword' | 'openwakewords':
963
+ pcm = np.frombuffer(data, dtype=np.int16)
964
+ prediction = self.owwModel.predict(pcm)
965
+ max_score = -1
966
+ max_index = -1
967
+ wake_words_in_prediction = len(self.owwModel.prediction_buffer.keys())
968
+ self.wake_words_sensitivities
969
+ if wake_words_in_prediction:
970
+ for idx, mdl in enumerate(self.owwModel.prediction_buffer.keys()):
971
+ scores = list(self.owwModel.prediction_buffer[mdl])
972
+ if scores[-1] >= self.wake_words_sensitivity and scores[-1] > max_score:
973
+ max_score = scores[-1]
974
+ max_index = idx
975
+ if self.debug_mode:
976
+ print (f"wake words oww max_index, max_score: {max_index} {max_score}")
977
+ return max_index
978
+ else:
979
+ if self.debug_mode:
980
+ print (f"wake words oww_index: -1")
981
+ return -1
982
+
983
+ if self.debug_mode:
984
+ print("wake words no match")
985
+ return -1
986
+
855
987
  def text(self,
856
988
  on_transcription_finished=None,
857
989
  ):
@@ -1109,14 +1241,14 @@ class AudioToTextRecorder:
1109
1241
  if wake_word_activation_delay_passed \
1110
1242
  and not delay_was_passed:
1111
1243
 
1112
- if self.wake_words and self.wake_word_activation_delay:
1244
+ if self.use_wake_words and self.wake_word_activation_delay:
1113
1245
  if self.on_wakeword_timeout:
1114
1246
  self.on_wakeword_timeout()
1115
1247
  delay_was_passed = wake_word_activation_delay_passed
1116
1248
 
1117
1249
  # Set state and spinner text
1118
1250
  if not self.recording_stop_time:
1119
- if self.wake_words \
1251
+ if self.use_wake_words \
1120
1252
  and wake_word_activation_delay_passed \
1121
1253
  and not self.wakeword_detected:
1122
1254
  self._set_state("wakeword")
@@ -1126,14 +1258,10 @@ class AudioToTextRecorder:
1126
1258
  else:
1127
1259
  self._set_state("inactive")
1128
1260
 
1129
- # Detect wake words if applicable
1130
- if self.wake_words and wake_word_activation_delay_passed:
1261
+ #self.wake_word_detect_time = time.time()
1262
+ if self.use_wake_words and wake_word_activation_delay_passed:
1131
1263
  try:
1132
- pcm = struct.unpack_from(
1133
- "h" * self.buffer_size,
1134
- data
1135
- )
1136
- wakeword_index = self.porcupine.process(pcm)
1264
+ wakeword_index = self._process_wakeword(data)
1137
1265
 
1138
1266
  except struct.error:
1139
1267
  logging.error("Error unpacking audio data "
@@ -1144,14 +1272,14 @@ class AudioToTextRecorder:
1144
1272
  logging.error(f"Wake word processing error: {e}")
1145
1273
  continue
1146
1274
 
1147
- # If a wake word is detected
1275
+ # If a wake word is detected
1148
1276
  if wakeword_index >= 0:
1149
1277
 
1150
1278
  # Removing the wake word from the recording
1151
- samples_for_0_1_sec = int(self.sample_rate * 0.1)
1279
+ samples_time = int(self.sample_rate * self.wake_word_buffer_duration)
1152
1280
  start_index = max(
1153
1281
  0,
1154
- len(self.audio_buffer) - samples_for_0_1_sec
1282
+ len(self.audio_buffer) - samples_time
1155
1283
  )
1156
1284
  temp_samples = collections.deque(
1157
1285
  itertools.islice(
@@ -1164,12 +1292,13 @@ class AudioToTextRecorder:
1164
1292
 
1165
1293
  self.wake_word_detect_time = time.time()
1166
1294
  self.wakeword_detected = True
1295
+ #self.wake_word_cooldown_time = time.time()
1167
1296
  if self.on_wakeword_detected:
1168
1297
  self.on_wakeword_detected()
1169
1298
 
1170
1299
  # Check for voice activity to
1171
1300
  # trigger the start of recording
1172
- if ((not self.wake_words
1301
+ if ((not self.use_wake_words
1173
1302
  or not wake_word_activation_delay_passed)
1174
1303
  and self.start_recording_on_voice_activity) \
1175
1304
  or self.wakeword_detected:
@@ -8,8 +8,8 @@ with open('requirements.txt') as f:
8
8
  requirements = f.read().splitlines()
9
9
 
10
10
  setuptools.setup(
11
- name="RealTimeSTT",
12
- version="0.1.16",
11
+ name="RealtimeSTT",
12
+ version="0.2.0",
13
13
  author="Kolja Beigel",
14
14
  author_email="kolja.beigel@web.de",
15
15
  description="A fast Voice Activity Detection and Transcription System",
File without changes
File without changes