rumblex 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,26 @@
1
+ __pycache__/
2
+ *.py[cod]
3
+ .venv/
4
+ dist/
5
+ build/
6
+ *.egg-info/
7
+ .pytest_cache/
8
+ .ruff_cache/
9
+ .cache/
10
+ # local audio / model downloads
11
+ samples/
12
+ *.wav
13
+ *.mp3
14
+ *.mp4
15
+
16
+ # datasets and cached encoder states
17
+ data/
18
+
19
+ # benchmark and example output
20
+ results/
21
+ logs/
22
+ results.tar.gz
23
+ session.out
24
+ *.bundle
25
+ out/
26
+ subs/
@@ -0,0 +1,18 @@
1
+ # Changelog
2
+
3
+ ## 0.1.0 (2026-11-21)
4
+
5
+ First release.
6
+
7
+ - Transcription with Reson8/Rumble-2-Flash: Silero VAD chunking (batched, on the GPU when there is one),
8
+ batched decoding whose output doesn't depend on the batch, and the model's `language`, `prompt`,
9
+ `digits`, `filler` and `verbatim` options.
10
+ - Word timestamps from the model's CTC head, reusing the decoder's encoder pass. Word ends are refined
11
+ on the audio's silences. A proportional fallback keeps a long job going if a chunk fails to align.
12
+ - Optional speaker labels with pyannote, per word.
13
+ - JSON, SRT, VTT (optionally with per-word karaoke tags), TSV and TXT output; subtitle cues split evenly at
14
+ word boundaries.
15
+ - Optional vLLM backend for decoding (`--vllm_url`).
16
+ - `rumblex` command line tool and a Python API (`load_model`, `transcribe`, `write_result`).
17
+ - Benchmarks against WhisperX on LibriSpeech, AMI, Earnings-22 and FLEURS Dutch, with scripts and raw
18
+ results in `docs/`.
rumblex-0.1.0/LICENSE ADDED
@@ -0,0 +1,202 @@
1
+
2
+ Apache License
3
+ Version 2.0, January 2004
4
+ http://www.apache.org/licenses/
5
+
6
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7
+
8
+ 1. Definitions.
9
+
10
+ "License" shall mean the terms and conditions for use, reproduction,
11
+ and distribution as defined by Sections 1 through 9 of this document.
12
+
13
+ "Licensor" shall mean the copyright owner or entity authorized by
14
+ the copyright owner that is granting the License.
15
+
16
+ "Legal Entity" shall mean the union of the acting entity and all
17
+ other entities that control, are controlled by, or are under common
18
+ control with that entity. For the purposes of this definition,
19
+ "control" means (i) the power, direct or indirect, to cause the
20
+ direction or management of such entity, whether by contract or
21
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
22
+ outstanding shares, or (iii) beneficial ownership of such entity.
23
+
24
+ "You" (or "Your") shall mean an individual or Legal Entity
25
+ exercising permissions granted by this License.
26
+
27
+ "Source" form shall mean the preferred form for making modifications,
28
+ including but not limited to software source code, documentation
29
+ source, and configuration files.
30
+
31
+ "Object" form shall mean any form resulting from mechanical
32
+ transformation or translation of a Source form, including but
33
+ not limited to compiled object code, generated documentation,
34
+ and conversions to other media types.
35
+
36
+ "Work" shall mean the work of authorship, whether in Source or
37
+ Object form, made available under the License, as indicated by a
38
+ copyright notice that is included in or attached to the work
39
+ (an example is provided in the Appendix below).
40
+
41
+ "Derivative Works" shall mean any work, whether in Source or Object
42
+ form, that is based on (or derived from) the Work and for which the
43
+ editorial revisions, annotations, elaborations, or other modifications
44
+ represent, as a whole, an original work of authorship. For the purposes
45
+ of this License, Derivative Works shall not include works that remain
46
+ separable from, or merely link (or bind by name) to the interfaces of,
47
+ the Work and Derivative Works thereof.
48
+
49
+ "Contribution" shall mean any work of authorship, including
50
+ the original version of the Work and any modifications or additions
51
+ to that Work or Derivative Works thereof, that is intentionally
52
+ submitted to Licensor for inclusion in the Work by the copyright owner
53
+ or by an individual or Legal Entity authorized to submit on behalf of
54
+ the copyright owner. For the purposes of this definition, "submitted"
55
+ means any form of electronic, verbal, or written communication sent
56
+ to the Licensor or its representatives, including but not limited to
57
+ communication on electronic mailing lists, source code control systems,
58
+ and issue tracking systems that are managed by, or on behalf of, the
59
+ Licensor for the purpose of discussing and improving the Work, but
60
+ excluding communication that is conspicuously marked or otherwise
61
+ designated in writing by the copyright owner as "Not a Contribution."
62
+
63
+ "Contributor" shall mean Licensor and any individual or Legal Entity
64
+ on behalf of whom a Contribution has been received by Licensor and
65
+ subsequently incorporated within the Work.
66
+
67
+ 2. Grant of Copyright License. Subject to the terms and conditions of
68
+ this License, each Contributor hereby grants to You a perpetual,
69
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70
+ copyright license to reproduce, prepare Derivative Works of,
71
+ publicly display, publicly perform, sublicense, and distribute the
72
+ Work and such Derivative Works in Source or Object form.
73
+
74
+ 3. Grant of Patent License. Subject to the terms and conditions of
75
+ this License, each Contributor hereby grants to You a perpetual,
76
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77
+ (except as stated in this section) patent license to make, have made,
78
+ use, offer to sell, sell, import, and otherwise transfer the Work,
79
+ where such license applies only to those patent claims licensable
80
+ by such Contributor that are necessarily infringed by their
81
+ Contribution(s) alone or by combination of their Contribution(s)
82
+ with the Work to which such Contribution(s) was submitted. If You
83
+ institute patent litigation against any entity (including a
84
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
85
+ or a Contribution incorporated within the Work constitutes direct
86
+ or contributory patent infringement, then any patent licenses
87
+ granted to You under this License for that Work shall terminate
88
+ as of the date such litigation is filed.
89
+
90
+ 4. Redistribution. You may reproduce and distribute copies of the
91
+ Work or Derivative Works thereof in any medium, with or without
92
+ modifications, and in Source or Object form, provided that You
93
+ meet the following conditions:
94
+
95
+ (a) You must give any other recipients of the Work or
96
+ Derivative Works a copy of this License; and
97
+
98
+ (b) You must cause any modified files to carry prominent notices
99
+ stating that You changed the files; and
100
+
101
+ (c) You must retain, in the Source form of any Derivative Works
102
+ that You distribute, all copyright, patent, trademark, and
103
+ attribution notices from the Source form of the Work,
104
+ excluding those notices that do not pertain to any part of
105
+ the Derivative Works; and
106
+
107
+ (d) If the Work includes a "NOTICE" text file as part of its
108
+ distribution, then any Derivative Works that You distribute must
109
+ include a readable copy of the attribution notices contained
110
+ within such NOTICE file, excluding those notices that do not
111
+ pertain to any part of the Derivative Works, in at least one
112
+ of the following places: within a NOTICE text file distributed
113
+ as part of the Derivative Works; within the Source form or
114
+ documentation, if provided along with the Derivative Works; or,
115
+ within a display generated by the Derivative Works, if and
116
+ wherever such third-party notices normally appear. The contents
117
+ of the NOTICE file are for informational purposes only and
118
+ do not modify the License. You may add Your own attribution
119
+ notices within Derivative Works that You distribute, alongside
120
+ or as an addendum to the NOTICE text from the Work, provided
121
+ that such additional attribution notices cannot be construed
122
+ as modifying the License.
123
+
124
+ You may add Your own copyright statement to Your modifications and
125
+ may provide additional or different license terms and conditions
126
+ for use, reproduction, or distribution of Your modifications, or
127
+ for any such Derivative Works as a whole, provided Your use,
128
+ reproduction, and distribution of the Work otherwise complies with
129
+ the conditions stated in this License.
130
+
131
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
132
+ any Contribution intentionally submitted for inclusion in the Work
133
+ by You to the Licensor shall be under the terms and conditions of
134
+ this License, without any additional terms or conditions.
135
+ Notwithstanding the above, nothing herein shall supersede or modify
136
+ the terms of any separate license agreement you may have executed
137
+ with Licensor regarding such Contributions.
138
+
139
+ 6. Trademarks. This License does not grant permission to use the trade
140
+ names, trademarks, service marks, or product names of the Licensor,
141
+ except as required for reasonable and customary use in describing the
142
+ origin of the Work and reproducing the content of the NOTICE file.
143
+
144
+ 7. Disclaimer of Warranty. Unless required by applicable law or
145
+ agreed to in writing, Licensor provides the Work (and each
146
+ Contributor provides its Contributions) on an "AS IS" BASIS,
147
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148
+ implied, including, without limitation, any warranties or conditions
149
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150
+ PARTICULAR PURPOSE. You are solely responsible for determining the
151
+ appropriateness of using or redistributing the Work and assume any
152
+ risks associated with Your exercise of permissions under this License.
153
+
154
+ 8. Limitation of Liability. In no event and under no legal theory,
155
+ whether in tort (including negligence), contract, or otherwise,
156
+ unless required by applicable law (such as deliberate and grossly
157
+ negligent acts) or agreed to in writing, shall any Contributor be
158
+ liable to You for damages, including any direct, indirect, special,
159
+ incidental, or consequential damages of any character arising as a
160
+ result of this License or out of the use or inability to use the
161
+ Work (including but not limited to damages for loss of goodwill,
162
+ work stoppage, computer failure or malfunction, or any and all
163
+ other commercial damages or losses), even if such Contributor
164
+ has been advised of the possibility of such damages.
165
+
166
+ 9. Accepting Warranty or Additional Liability. While redistributing
167
+ the Work or Derivative Works thereof, You may choose to offer,
168
+ and charge a fee for, acceptance of support, warranty, indemnity,
169
+ or other liability obligations and/or rights consistent with this
170
+ License. However, in accepting such obligations, You may act only
171
+ on Your own behalf and on Your sole responsibility, not on behalf
172
+ of any other Contributor, and only if You agree to indemnify,
173
+ defend, and hold each Contributor harmless for any liability
174
+ incurred by, or claims asserted against, such Contributor by reason
175
+ of your accepting any such warranty or additional liability.
176
+
177
+ END OF TERMS AND CONDITIONS
178
+
179
+ APPENDIX: How to apply the Apache License to your work.
180
+
181
+ To apply the Apache License to your work, attach the following
182
+ boilerplate notice, with the fields enclosed by brackets "[]"
183
+ replaced with your own identifying information. (Don't include
184
+ the brackets!) The text should be enclosed in the appropriate
185
+ comment syntax for the file format. We also recommend that a
186
+ file or class name and description of purpose be included on the
187
+ same "printed page" as the copyright notice for easier
188
+ identification within third-party archives.
189
+
190
+ Copyright [yyyy] [name of copyright owner]
191
+
192
+ Licensed under the Apache License, Version 2.0 (the "License");
193
+ you may not use this file except in compliance with the License.
194
+ You may obtain a copy of the License at
195
+
196
+ http://www.apache.org/licenses/LICENSE-2.0
197
+
198
+ Unless required by applicable law or agreed to in writing, software
199
+ distributed under the License is distributed on an "AS IS" BASIS,
200
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
201
+ See the License for the specific language governing permissions and
202
+ limitations under the License.
rumblex-0.1.0/PKG-INFO ADDED
@@ -0,0 +1,230 @@
1
+ Metadata-Version: 2.5
2
+ Name: rumblex
3
+ Version: 0.1.0
4
+ Summary: Transcription with word timestamps, speaker labels and subtitles on Reson8/Rumble-2-Flash
5
+ Project-URL: Homepage, https://github.com/EdwardGlockner/rumblex
6
+ Project-URL: Source, https://github.com/EdwardGlockner/rumblex
7
+ Project-URL: Issues, https://github.com/EdwardGlockner/rumblex/issues
8
+ Project-URL: Changelog, https://github.com/EdwardGlockner/rumblex/blob/main/CHANGELOG.md
9
+ Author: Edward Glockner
10
+ License-Expression: Apache-2.0
11
+ License-File: LICENSE
12
+ Keywords: asr,ctc,diarization,forced alignment,rumble-2-flash,speech recognition,srt,subtitles,transcription,vtt,whisperx,word timestamps
13
+ Classifier: Development Status :: 3 - Alpha
14
+ Classifier: Intended Audience :: Developers
15
+ Classifier: Intended Audience :: Science/Research
16
+ Classifier: Operating System :: OS Independent
17
+ Classifier: Programming Language :: Python :: 3
18
+ Classifier: Programming Language :: Python :: 3.11
19
+ Classifier: Programming Language :: Python :: 3.12
20
+ Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
21
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
22
+ Requires-Python: >=3.11
23
+ Requires-Dist: huggingface-hub
24
+ Requires-Dist: numpy
25
+ Requires-Dist: rich
26
+ Requires-Dist: safetensors
27
+ Requires-Dist: silero-vad>=6.2.3
28
+ Requires-Dist: soundfile
29
+ Requires-Dist: torch>=2.6
30
+ Requires-Dist: torchaudio>=2.6
31
+ Requires-Dist: transformers>=5.21
32
+ Provides-Extra: dev
33
+ Requires-Dist: jiwer; extra == 'dev'
34
+ Requires-Dist: pyarrow; extra == 'dev'
35
+ Requires-Dist: pytest; extra == 'dev'
36
+ Requires-Dist: ruff==0.17.6; extra == 'dev'
37
+ Requires-Dist: whisper-normalizer; extra == 'dev'
38
+ Provides-Extra: diarize
39
+ Requires-Dist: pyannote-audio<5,>=4.1; extra == 'diarize'
40
+ Provides-Extra: vllm
41
+ Requires-Dist: openai; extra == 'vllm'
42
+ Description-Content-Type: text/markdown
43
+
44
+ # rumblex
45
+
46
+ Transcription with word-level timestamps, speaker labels and subtitles, built on
47
+ [Reson8/Rumble-2-Flash](https://huggingface.co/Reson8/Rumble-2-Flash).
48
+
49
+ Rumble-2-Flash ships an auxiliary CTC head next to its decoder. rumblex uses it for word timestamps:
50
+ the decoder writes the text, and the CTC head force-aligns that text to the audio. It reuses the
51
+ encoder pass the decoder already ran, so alignment costs one linear layer plus a Viterbi search, not
52
+ a second model.
53
+
54
+ Rumble-2-Flash is a 0.5B-parameter speech recognition model fine-tuned from Qwen3-ASR-0.6B. Its
55
+ [model card](https://huggingface.co/Reson8/Rumble-2-Flash) lists 45 languages; the benchmarks here cover
56
+ English and Dutch.
57
+
58
+ ![rumblex transcribing two files and writing subtitles](https://raw.githubusercontent.com/EdwardGlockner/rumblex/main/docs/demo.gif)
59
+
60
+ > rumblex is an independent project and is not affiliated with Reson8.
61
+
62
+ ## Features
63
+
64
+ - Reads anything ffmpeg can (wav, mp3, m4a, mp4, mkv, ...). Long recordings are split into speech
65
+ chunks with Silero VAD.
66
+ - Batched transcription. A chunk's output doesn't depend on what it is batched with. The model's
67
+ `language`, `prompt` (context / hotwords), `digits`, `filler` and `verbatim` options are exposed.
68
+ - Word timestamps from the CTC head. If alignment fails on a chunk, its words get proportional times
69
+ instead, so a long job doesn't stop.
70
+ - Optional speaker labels via pyannote, per word. Segments are split where the speaker changes.
71
+ - Writes JSON, SRT, VTT (optionally with per-word karaoke tags), TSV and TXT.
72
+
73
+ ## Install
74
+
75
+ ```bash
76
+ pip install rumblex
77
+ pip install "rumblex[diarize]" # with speaker labels (pyannote)
78
+ ```
79
+
80
+ You also need Python 3.11 or 3.12 and ffmpeg on `PATH`. pip takes PyTorch from PyPI, which is a CPU-only build
81
+ on Windows. For a GPU on Windows, or for an older GPU, install torch and torchaudio for your CUDA version
82
+ from [pytorch.org](https://pytorch.org/get-started/locally/) first. GPUs from Ampere on run in bf16,
83
+ older ones (e.g. Pascal) in float32. A GPU is strongly recommended; the model runs on a 6 GB GTX 1060 in
84
+ float32 at the default batch size of 8. CPU works but is slow.
85
+
86
+ From source:
87
+
88
+ ```bash
89
+ git clone https://github.com/EdwardGlockner/rumblex && cd rumblex
90
+ uv sync # PyTorch's CUDA 12.6 builds, which still support Pascal
91
+ uv sync --extra diarize
92
+ ```
93
+
94
+ The lock file pins PyTorch's CUDA 12.6 wheels on Linux and Windows and PyPI's build on macOS. For RTX
95
+ 50-series GPUs (which need CUDA 12.8+), install torch yourself and use `pip install -e .` instead.
96
+
97
+ ## Use
98
+
99
+ ```bash
100
+ rumblex meeting.mp4 # all formats into the current folder
101
+ rumblex talk.wav -f srt vtt --max_line_width 42 # subtitles with cues split at 42 chars
102
+ rumblex interview.m4a --language Dutch --diarize -f srt txt
103
+ rumblex call.wav --prompt "Kubernetes, metoprolol" # context / hotwords for the model
104
+ ```
105
+
106
+ ```python
107
+ import rumblex
108
+
109
+ model = rumblex.load_model() # Reson8/Rumble-2-Flash, GPU if available
110
+ result = rumblex.transcribe("meeting.mp4", model) # segments with word timestamps
111
+ for seg in result.segments:
112
+ print(seg.start, seg.end, seg.text)
113
+ rumblex.write_result(result, "meeting.mp4", "out/", ["srt", "json"])
114
+ ```
115
+
116
+ More in [examples/quickstart.py](https://github.com/EdwardGlockner/rumblex/blob/main/examples/quickstart.py) and
117
+ [examples/batch_subtitles.py](https://github.com/EdwardGlockner/rumblex/blob/main/examples/batch_subtitles.py), which writes subtitles for a folder of files.
118
+
119
+ ### Faster decoding with vLLM
120
+
121
+ On Linux with a recent NVIDIA GPU, decoding can run on a [vLLM](https://github.com/vllm-project/vllm)
122
+ server. `pip install "rumblex[vllm]"` adds the client; vLLM itself goes wherever the server runs:
123
+
124
+ ```bash
125
+ vllm serve Reson8/Rumble-2-Flash --max-model-len 4096 --gpu-memory-utilization 0.5
126
+ rumblex meeting.mp4 --vllm_url http://localhost:8000/v1
127
+ ```
128
+
129
+ Word timestamps still come from a local pass of the audio encoder, so leave GPU memory for it. On our
130
+ LibriSpeech run this was 601× real time against 467× without vLLM, with the same WER and timing error
131
+ ([docs/bench_2026-11-20_rtx4090_release](https://github.com/EdwardGlockner/rumblex/blob/main/docs/bench_2026-11-20_rtx4090_release/README.md)).
132
+
133
+ ### Speaker labels
134
+
135
+ The pyannote models are gated. Accept the conditions on
136
+ [pyannote/speaker-diarization-community-1](https://huggingface.co/pyannote/speaker-diarization-community-1),
137
+ then run `hf auth login`. pyannote.audio 4 sends usage telemetry by default. rumblex turns it off
138
+ unless you set `PYANNOTE_METRICS_ENABLED=true` yourself.
139
+
140
+ ## Compared with WhisperX
141
+
142
+ Both systems ran on the same audio on one rented RTX 4090, in one session, with the code released as
143
+ 0.1.0: rumblex in bf16 at batch 32 (the default is 8), and WhisperX 3.8.6 with `large-v3` in float16 at
144
+ batch 16 with its default wav2vec2 aligners (large-v3-turbo was not measured). The language was given to
145
+ both. For the English sets rumblex ran with `--no_digits`, since their references spell numbers out.
146
+ WER uses Whisper's normalizers. Speed is one long file per dataset (20–60 min) made from that dataset's
147
+ audio, timed from decoded audio to aligned words with the models already loaded. Raw results, settings
148
+ and caveats:
149
+ [docs/bench_2026-11-20_rtx4090_release](https://github.com/EdwardGlockner/rumblex/blob/main/docs/bench_2026-11-20_rtx4090_release/README.md).
150
+
151
+ | dataset | rumblex WER | WhisperX WER | rumblex speed | WhisperX speed |
152
+ |---|---|---|---|---|
153
+ | AMI meetings (500 utt.) | 10.9% | 17.2% | 329× real time | 104× |
154
+ | Earnings-22 calls (500 utt.) | 9.3% | 10.6% | 430× | 102× |
155
+ | LibriSpeech test-clean (200 utt.) | 2.2% | 1.6% | 467× (601× with vLLM) | 92× |
156
+ | FLEURS Dutch (364 utt.) | 9.2% | 7.0% | 380× | 79× |
157
+
158
+ rumblex was 3–5× faster on every set. Over 40% of WhisperX's time goes to its alignment step (7–17 s
159
+ per file); rumblex's alignment reuses the encoder pass and took 0.3–0.9 s. rumblex had the lower WER on
160
+ meetings and earnings calls, WhisperX on clean read speech and on Dutch. Most of rumblex's lead on AMI
161
+ and Earnings-22 is fewer deleted words (177 vs 353 and 231 vs 427). rumblex passes clips of up to 20 s
162
+ to the model whole, while WhisperX runs its VAD on every clip, so on long recordings, where both use a
163
+ VAD, the gap may be smaller. These are samples of a few hundred utterances per set, and speeds also
164
+ depend on the host CPU.
165
+
166
+ The [model card](https://huggingface.co/Reson8/Rumble-2-Flash) reports 6.05% on FLEURS Dutch under the
167
+ Open ASR Leaderboard's scoring, which we did not reproduce. An earlier diagnostic run (vLLM vs
168
+ transformers, bf16 vs float32, with and without VAD) gave 9.2–9.6% every time, so none of those pipeline
169
+ choices explains the gap. Part of it is that FLEURS mixes digits and spelled-out numbers in its
170
+ references ([docs/bench_2026-11-20_rtx4090_diagnose](https://github.com/EdwardGlockner/rumblex/blob/main/docs/bench_2026-11-20_rtx4090_diagnose/README.md)).
171
+
172
+ ### Timestamp accuracy
173
+
174
+ Measured on the 200 LibriSpeech utterances in the same run, against Montreal Forced Aligner reference
175
+ alignments. The settings were chosen on `dev_clean` ([docs/alignment_eval.md](https://github.com/EdwardGlockner/rumblex/blob/main/docs/alignment_eval.md)).
176
+
177
+ | mean absolute error | rumblex | WhisperX |
178
+ |---|---|---|
179
+ | word start | 58 ms | 75 ms |
180
+ | word end | 57 ms | 34 ms |
181
+
182
+ The Rumble encoder produces 13 frames per second (80 ms apart within each second), while WhisperX's
183
+ aligner works at 20 ms, and that shows in the word ends. rumblex moves a word's end back to where the
184
+ audio goes quiet, which brought its end error down from 75 to 57 ms in development. The reference is
185
+ another forced aligner, not human labels.
186
+
187
+ ## How it works
188
+
189
+ 1. ffmpeg decodes the audio to 16 kHz mono.
190
+ 2. Silero VAD finds the speech, running over many stretches of the file at once (on the GPU if there
191
+ is one). Speech is packed into chunks of up to 20 s. Pauses under 0.5 s don't split speech unless a
192
+ chunk would run past 20 s, and pauses over 2 s never end up inside a chunk. Audio of 20 s or less
193
+ skips VAD and goes in whole.
194
+ 3. Rumble-2-Flash transcribes the chunks in batches. A hook keeps the encoder's `ln_post` states
195
+ (13 frames/s).
196
+ 4. The CTC head and a Viterbi search align the decoder's own tokens to those states. Tokens are grouped
197
+ into words at the leading-space marker. A word ends where the next one starts, or earlier if the
198
+ audio goes quiet before that.
199
+ 5. Optionally, pyannote labels speakers and each word takes the speaker it overlaps most.
200
+ 6. The writers produce JSON, SRT, VTT, TSV and TXT.
201
+
202
+ Model details measured for this project: [docs/model_inspection.md](https://github.com/EdwardGlockner/rumblex/blob/main/docs/model_inspection.md).
203
+
204
+ ## Limitations
205
+
206
+ - Word timing resolution is limited by the encoder's frame rate (13 per second). Word ends in particular
207
+ are less precise than WhisperX's 20 ms aligner (57 vs 34 ms mean error on LibriSpeech).
208
+ - Each chunk is decoded on its own, so a sentence that runs across a chunk boundary in long audio can be
209
+ broken there, sometimes with a capital letter mid-sentence.
210
+ - Words are split at spaces, so languages written without spaces (Chinese, Japanese) get one "word" per
211
+ chunk.
212
+ - On clean read English and on FLEURS Dutch, WhisperX large-v3 had the lower WER in our runs. Only
213
+ English and Dutch have been measured.
214
+ - Speaker labels need the `[diarize]` extra and access to pyannote's gated models.
215
+ - The vLLM backend needs Linux and a GPU vLLM supports.
216
+
217
+ ## Development
218
+
219
+ ```bash
220
+ uv sync --extra dev
221
+ uv run pytest
222
+ uv run ruff check . && uv run ruff format --check .
223
+ ```
224
+
225
+ Benchmarks and research scripts: [scripts/README.md](https://github.com/EdwardGlockner/rumblex/blob/main/scripts/README.md). Every measured number in this
226
+ README links to its raw results; the full list is in [docs/README.md](https://github.com/EdwardGlockner/rumblex/blob/main/docs/README.md).
227
+
228
+ ## License
229
+
230
+ Apache-2.0. Rumble-2-Flash is Apache-2.0 as well. The pyannote models have their own terms.
@@ -0,0 +1,187 @@
1
+ # rumblex
2
+
3
+ Transcription with word-level timestamps, speaker labels and subtitles, built on
4
+ [Reson8/Rumble-2-Flash](https://huggingface.co/Reson8/Rumble-2-Flash).
5
+
6
+ Rumble-2-Flash ships an auxiliary CTC head next to its decoder. rumblex uses it for word timestamps:
7
+ the decoder writes the text, and the CTC head force-aligns that text to the audio. It reuses the
8
+ encoder pass the decoder already ran, so alignment costs one linear layer plus a Viterbi search, not
9
+ a second model.
10
+
11
+ Rumble-2-Flash is a 0.5B-parameter speech recognition model fine-tuned from Qwen3-ASR-0.6B. Its
12
+ [model card](https://huggingface.co/Reson8/Rumble-2-Flash) lists 45 languages; the benchmarks here cover
13
+ English and Dutch.
14
+
15
+ ![rumblex transcribing two files and writing subtitles](docs/demo.gif)
16
+
17
+ > rumblex is an independent project and is not affiliated with Reson8.
18
+
19
+ ## Features
20
+
21
+ - Reads anything ffmpeg can (wav, mp3, m4a, mp4, mkv, ...). Long recordings are split into speech
22
+ chunks with Silero VAD.
23
+ - Batched transcription. A chunk's output doesn't depend on what it is batched with. The model's
24
+ `language`, `prompt` (context / hotwords), `digits`, `filler` and `verbatim` options are exposed.
25
+ - Word timestamps from the CTC head. If alignment fails on a chunk, its words get proportional times
26
+ instead, so a long job doesn't stop.
27
+ - Optional speaker labels via pyannote, per word. Segments are split where the speaker changes.
28
+ - Writes JSON, SRT, VTT (optionally with per-word karaoke tags), TSV and TXT.
29
+
30
+ ## Install
31
+
32
+ ```bash
33
+ pip install rumblex
34
+ pip install "rumblex[diarize]" # with speaker labels (pyannote)
35
+ ```
36
+
37
+ You also need Python 3.11 or 3.12 and ffmpeg on `PATH`. pip takes PyTorch from PyPI, which is a CPU-only build
38
+ on Windows. For a GPU on Windows, or for an older GPU, install torch and torchaudio for your CUDA version
39
+ from [pytorch.org](https://pytorch.org/get-started/locally/) first. GPUs from Ampere on run in bf16,
40
+ older ones (e.g. Pascal) in float32. A GPU is strongly recommended; the model runs on a 6 GB GTX 1060 in
41
+ float32 at the default batch size of 8. CPU works but is slow.
42
+
43
+ From source:
44
+
45
+ ```bash
46
+ git clone https://github.com/EdwardGlockner/rumblex && cd rumblex
47
+ uv sync # PyTorch's CUDA 12.6 builds, which still support Pascal
48
+ uv sync --extra diarize
49
+ ```
50
+
51
+ The lock file pins PyTorch's CUDA 12.6 wheels on Linux and Windows and PyPI's build on macOS. For RTX
52
+ 50-series GPUs (which need CUDA 12.8+), install torch yourself and use `pip install -e .` instead.
53
+
54
+ ## Use
55
+
56
+ ```bash
57
+ rumblex meeting.mp4 # all formats into the current folder
58
+ rumblex talk.wav -f srt vtt --max_line_width 42 # subtitles with cues split at 42 chars
59
+ rumblex interview.m4a --language Dutch --diarize -f srt txt
60
+ rumblex call.wav --prompt "Kubernetes, metoprolol" # context / hotwords for the model
61
+ ```
62
+
63
+ ```python
64
+ import rumblex
65
+
66
+ model = rumblex.load_model() # Reson8/Rumble-2-Flash, GPU if available
67
+ result = rumblex.transcribe("meeting.mp4", model) # segments with word timestamps
68
+ for seg in result.segments:
69
+ print(seg.start, seg.end, seg.text)
70
+ rumblex.write_result(result, "meeting.mp4", "out/", ["srt", "json"])
71
+ ```
72
+
73
+ More in [examples/quickstart.py](examples/quickstart.py) and
74
+ [examples/batch_subtitles.py](examples/batch_subtitles.py), which writes subtitles for a folder of files.
75
+
76
+ ### Faster decoding with vLLM
77
+
78
+ On Linux with a recent NVIDIA GPU, decoding can run on a [vLLM](https://github.com/vllm-project/vllm)
79
+ server. `pip install "rumblex[vllm]"` adds the client; vLLM itself goes wherever the server runs:
80
+
81
+ ```bash
82
+ vllm serve Reson8/Rumble-2-Flash --max-model-len 4096 --gpu-memory-utilization 0.5
83
+ rumblex meeting.mp4 --vllm_url http://localhost:8000/v1
84
+ ```
85
+
86
+ Word timestamps still come from a local pass of the audio encoder, so leave GPU memory for it. On our
87
+ LibriSpeech run this was 601× real time against 467× without vLLM, with the same WER and timing error
88
+ ([docs/bench_2026-11-20_rtx4090_release](docs/bench_2026-11-20_rtx4090_release/README.md)).
89
+
90
+ ### Speaker labels
91
+
92
+ The pyannote models are gated. Accept the conditions on
93
+ [pyannote/speaker-diarization-community-1](https://huggingface.co/pyannote/speaker-diarization-community-1),
94
+ then run `hf auth login`. pyannote.audio 4 sends usage telemetry by default. rumblex turns it off
95
+ unless you set `PYANNOTE_METRICS_ENABLED=true` yourself.
96
+
97
+ ## Compared with WhisperX
98
+
99
+ Both systems ran on the same audio on one rented RTX 4090, in one session, with the code released as
100
+ 0.1.0: rumblex in bf16 at batch 32 (the default is 8), and WhisperX 3.8.6 with `large-v3` in float16 at
101
+ batch 16 with its default wav2vec2 aligners (large-v3-turbo was not measured). The language was given to
102
+ both. For the English sets rumblex ran with `--no_digits`, since their references spell numbers out.
103
+ WER uses Whisper's normalizers. Speed is one long file per dataset (20–60 min) made from that dataset's
104
+ audio, timed from decoded audio to aligned words with the models already loaded. Raw results, settings
105
+ and caveats:
106
+ [docs/bench_2026-11-20_rtx4090_release](docs/bench_2026-11-20_rtx4090_release/README.md).
107
+
108
+ | dataset | rumblex WER | WhisperX WER | rumblex speed | WhisperX speed |
109
+ |---|---|---|---|---|
110
+ | AMI meetings (500 utt.) | 10.9% | 17.2% | 329× real time | 104× |
111
+ | Earnings-22 calls (500 utt.) | 9.3% | 10.6% | 430× | 102× |
112
+ | LibriSpeech test-clean (200 utt.) | 2.2% | 1.6% | 467× (601× with vLLM) | 92× |
113
+ | FLEURS Dutch (364 utt.) | 9.2% | 7.0% | 380× | 79× |
114
+
115
+ rumblex was 3–5× faster on every set. Over 40% of WhisperX's time goes to its alignment step (7–17 s
116
+ per file); rumblex's alignment reuses the encoder pass and took 0.3–0.9 s. rumblex had the lower WER on
117
+ meetings and earnings calls, WhisperX on clean read speech and on Dutch. Most of rumblex's lead on AMI
118
+ and Earnings-22 is fewer deleted words (177 vs 353 and 231 vs 427). rumblex passes clips of up to 20 s
119
+ to the model whole, while WhisperX runs its VAD on every clip, so on long recordings, where both use a
120
+ VAD, the gap may be smaller. These are samples of a few hundred utterances per set, and speeds also
121
+ depend on the host CPU.
122
+
123
+ The [model card](https://huggingface.co/Reson8/Rumble-2-Flash) reports 6.05% on FLEURS Dutch under the
124
+ Open ASR Leaderboard's scoring, which we did not reproduce. An earlier diagnostic run (vLLM vs
125
+ transformers, bf16 vs float32, with and without VAD) gave 9.2–9.6% every time, so none of those pipeline
126
+ choices explains the gap. Part of it is that FLEURS mixes digits and spelled-out numbers in its
127
+ references ([docs/bench_2026-11-20_rtx4090_diagnose](docs/bench_2026-11-20_rtx4090_diagnose/README.md)).
128
+
129
+ ### Timestamp accuracy
130
+
131
+ Measured on the 200 LibriSpeech utterances in the same run, against Montreal Forced Aligner reference
132
+ alignments. The settings were chosen on `dev_clean` ([docs/alignment_eval.md](docs/alignment_eval.md)).
133
+
134
+ | mean absolute error | rumblex | WhisperX |
135
+ |---|---|---|
136
+ | word start | 58 ms | 75 ms |
137
+ | word end | 57 ms | 34 ms |
138
+
139
+ The Rumble encoder produces 13 frames per second (80 ms apart within each second), while WhisperX's
140
+ aligner works at 20 ms, and that shows in the word ends. rumblex moves a word's end back to where the
141
+ audio goes quiet, which brought its end error down from 75 to 57 ms in development. The reference is
142
+ another forced aligner, not human labels.
143
+
144
+ ## How it works
145
+
146
+ 1. ffmpeg decodes the audio to 16 kHz mono.
147
+ 2. Silero VAD finds the speech, running over many stretches of the file at once (on the GPU if there
148
+ is one). Speech is packed into chunks of up to 20 s. Pauses under 0.5 s don't split speech unless a
149
+ chunk would run past 20 s, and pauses over 2 s never end up inside a chunk. Audio of 20 s or less
150
+ skips VAD and goes in whole.
151
+ 3. Rumble-2-Flash transcribes the chunks in batches. A hook keeps the encoder's `ln_post` states
152
+ (13 frames/s).
153
+ 4. The CTC head and a Viterbi search align the decoder's own tokens to those states. Tokens are grouped
154
+ into words at the leading-space marker. A word ends where the next one starts, or earlier if the
155
+ audio goes quiet before that.
156
+ 5. Optionally, pyannote labels speakers and each word takes the speaker it overlaps most.
157
+ 6. The writers produce JSON, SRT, VTT, TSV and TXT.
158
+
159
+ Model details measured for this project: [docs/model_inspection.md](docs/model_inspection.md).
160
+
161
+ ## Limitations
162
+
163
+ - Word timing resolution is limited by the encoder's frame rate (13 per second). Word ends in particular
164
+ are less precise than WhisperX's 20 ms aligner (57 vs 34 ms mean error on LibriSpeech).
165
+ - Each chunk is decoded on its own, so a sentence that runs across a chunk boundary in long audio can be
166
+ broken there, sometimes with a capital letter mid-sentence.
167
+ - Words are split at spaces, so languages written without spaces (Chinese, Japanese) get one "word" per
168
+ chunk.
169
+ - On clean read English and on FLEURS Dutch, WhisperX large-v3 had the lower WER in our runs. Only
170
+ English and Dutch have been measured.
171
+ - Speaker labels need the `[diarize]` extra and access to pyannote's gated models.
172
+ - The vLLM backend needs Linux and a GPU vLLM supports.
173
+
174
+ ## Development
175
+
176
+ ```bash
177
+ uv sync --extra dev
178
+ uv run pytest
179
+ uv run ruff check . && uv run ruff format --check .
180
+ ```
181
+
182
+ Benchmarks and research scripts: [scripts/README.md](scripts/README.md). Every measured number in this
183
+ README links to its raw results; the full list is in [docs/README.md](docs/README.md).
184
+
185
+ ## License
186
+
187
+ Apache-2.0. Rumble-2-Flash is Apache-2.0 as well. The pyannote models have their own terms.