whisperdrz 0.1.0__tar.gz

whisperdrz-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Fluxions
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

whisperdrz-0.1.0/PKG-INFO

@@ -0,0 +1,142 @@
+Metadata-Version: 2.4
+Name: whisperdrz
+Version: 0.1.0
+Summary: Speaker-aware ASR with word-level timestamps (inference)
+License: MIT
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: torch
+Requires-Dist: torchaudio
+Requires-Dist: torchcodec
+Requires-Dist: numpy
+Requires-Dist: scipy
+Requires-Dist: einops
+Requires-Dist: numba
+Requires-Dist: tiktoken
+Requires-Dist: safetensors
+Requires-Dist: huggingface_hub
+Provides-Extra: flash
+Requires-Dist: flash-attn; extra == "flash"
+Requires-Dist: triton; extra == "flash"
+Provides-Extra: demo
+Requires-Dist: gradio; extra == "demo"
+Provides-Extra: eval
+Requires-Dist: jiwer; extra == "eval"
+Requires-Dist: whisper-normalizer; extra == "eval"
+Dynamic: license-file
+
+# WhisperDRZ - Adding Diarization to Whisper
+
+WhisperDRZ is a speaker-aware automatic speech recognition model. It transcribes audio
+into text with word-level timestamps, per-line speaker tags, and non-speech event
+tags. It is a Whisper-style encoder-decoder model and handles long audio by
+chunking and stitching internally.
+
+This repository is inference only. Weights are distributed on the Hugging Face
+Hub at [`fluxions/whisperdrz`](https://huggingface.co/fluxions/whisperdrz).
+
+## Install
+
+```bash
+pip install -e .
+# optional: Flash Attention for much faster decoding on CUDA
+pip install -e ".[flash]"
+# optional: the gradio demo
+pip install -e ".[demo]"
+```
+
+Requires Python 3.12+. Runs on a CUDA GPU or on CPU. Flash Attention is
+optional: when it (and a CUDA GPU) is available it is used with CUDA graphs for
+fast decoding; otherwise a pure-torch attention path is used automatically.
+
+## Command line
+
+```bash
+whisperdrz audio.wav                                       # defaults: --model whisperdrz-large-v3.safetensors, --lang en
+whisperdrz audio.wav --output_format json > out.json
+whisperdrz audio.wav --model my-checkpoint.pt --lang auto  # override the defaults
+```
+
+`--model` defaults to `whisperdrz-large-v3.safetensors` and accepts a local checkpoint,
+a filename hosted in the weights repo, or a Hugging Face repo id. The weights
+download automatically on first use. `--lang` defaults to `en`; use `auto` to detect.
+
+## Python
+
+```python
+import whisperdrz
+from whisperdrz.audio import load_audio, SAMPLE_RATE
+
+transcriber = whisperdrz.load_model("whisperdrz-large-v3.safetensors", lang="en")
+
+audio, _ = load_audio("audio.wav", sample_rate=SAMPLE_RATE)
+result = transcriber.transcribe(audio.mean(0))  # mono, 16 kHz
+
+print(result.text)        # speaker-tagged text with timestamps
+print(result.segments)    # list of {speaker, start, end, text}
+```
+
+## Output format
+
+Each line begins with a speaker tag. Timed words and tags are wrapped in a
+start/end timestamp pair; not every word is timed, but the first and last word
+of each line always are:
+
+```
+[0] <|0.00|>Hello<|0.45|> there <|0.80|>world.<|1.10|>
+[1] <|1.20|>Hi<|1.40|> <|1.45|>[laugh]<|1.60|> <|1.70|>there.<|1.95|>
+```
+
+- `[0]`, `[1]`, ... are speaker IDs; `[c]` marks crowd/ambient.
+- `<|t|>` are timestamps in seconds (two decimals), always in a pair wrapping a word or tag.
+- The first and last word of every line are always timed; middle words may be bare.
+- `[laugh]`, `[breath]`, and similar are non-speech event tags.
+
+`transcribe()` returns a `TranscribeResults` with the raw `text` plus parsed
+`segments`, each a dict of `speaker`, `start`, `end`, and `text`.
+
+## Evaluation
+
+WhisperDRZ reports two metrics:
+
+- **WER** (word error rate) — Levenshtein word distance after normalization
+  (whisper-normalizer), measuring transcription accuracy.
+- **WDER** (word diarization error rate) — of the words that align between
+  reference and hypothesis, the fraction assigned to the wrong speaker under
+  the best speaker permutation. It isolates diarization quality from WER
+  (insertions/deletions don't count toward WDER).
+
+Measured on the released checkpoint:
+
+| Benchmark | Metric | Score |
+|---|---|---|
+| ESB (English ASR, 1000 utts) | WER | 9.6% macro / 5.9% micro |
+| VoxConverse dev (216 files, overlap-heavy) | DER | 40.7% (63.4% on overlapping speech) |
+| Internal conversational (26 clips) | WER / WDER | 11.1% / 33% |
+
+WhisperDRZ is an ASR-first model: transcription is strong, but diarization trails
+purpose-built systems (which reach ~10–25% DER on VoxConverse) and overlap is its
+weak spot. See [the write-up](blog.md) for full results (timing, non-speech
+events, multilingual) and analysis.
+
+Reproduce WER/WDER on your own data with the metrics in `whisperdrz.evals` and the runner:
+
+```bash
+pip install -e ".[eval]"
+# manifest.jsonl: one {"audio": "a.wav", "text": "[0] ref ... [1] ..."} per line
+python scripts/eval.py manifest.jsonl --model whisperdrz-large-v3.safetensors
+```
+
+`text` is the reference transcript; `[N]` speaker tags are optional and WDER is
+only computed for multi-speaker references.
+
+## Demo
+
+```bash
+python demo/app.py --model whisperdrz-large-v3.safetensors
+```
+
+## License
+
+MIT. See [LICENSE](LICENSE).

whisperdrz-0.1.0/README.md

@@ -0,0 +1,114 @@
+# WhisperDRZ - Adding Diarization to Whisper
+
+WhisperDRZ is a speaker-aware automatic speech recognition model. It transcribes audio
+into text with word-level timestamps, per-line speaker tags, and non-speech event
+tags. It is a Whisper-style encoder-decoder model and handles long audio by
+chunking and stitching internally.
+
+This repository is inference only. Weights are distributed on the Hugging Face
+Hub at [`fluxions/whisperdrz`](https://huggingface.co/fluxions/whisperdrz).
+
+## Install
+
+```bash
+pip install -e .
+# optional: Flash Attention for much faster decoding on CUDA
+pip install -e ".[flash]"
+# optional: the gradio demo
+pip install -e ".[demo]"
+```
+
+Requires Python 3.12+. Runs on a CUDA GPU or on CPU. Flash Attention is
+optional: when it (and a CUDA GPU) is available it is used with CUDA graphs for
+fast decoding; otherwise a pure-torch attention path is used automatically.
+
+## Command line
+
+```bash
+whisperdrz audio.wav                                       # defaults: --model whisperdrz-large-v3.safetensors, --lang en
+whisperdrz audio.wav --output_format json > out.json
+whisperdrz audio.wav --model my-checkpoint.pt --lang auto  # override the defaults
+```
+
+`--model` defaults to `whisperdrz-large-v3.safetensors` and accepts a local checkpoint,
+a filename hosted in the weights repo, or a Hugging Face repo id. The weights
+download automatically on first use. `--lang` defaults to `en`; use `auto` to detect.
+
+## Python
+
+```python
+import whisperdrz
+from whisperdrz.audio import load_audio, SAMPLE_RATE
+
+transcriber = whisperdrz.load_model("whisperdrz-large-v3.safetensors", lang="en")
+
+audio, _ = load_audio("audio.wav", sample_rate=SAMPLE_RATE)
+result = transcriber.transcribe(audio.mean(0))  # mono, 16 kHz
+
+print(result.text)        # speaker-tagged text with timestamps
+print(result.segments)    # list of {speaker, start, end, text}
+```
+
+## Output format
+
+Each line begins with a speaker tag. Timed words and tags are wrapped in a
+start/end timestamp pair; not every word is timed, but the first and last word
+of each line always are:
+
+```
+[0] <|0.00|>Hello<|0.45|> there <|0.80|>world.<|1.10|>
+[1] <|1.20|>Hi<|1.40|> <|1.45|>[laugh]<|1.60|> <|1.70|>there.<|1.95|>
+```
+
+- `[0]`, `[1]`, ... are speaker IDs; `[c]` marks crowd/ambient.
+- `<|t|>` are timestamps in seconds (two decimals), always in a pair wrapping a word or tag.
+- The first and last word of every line are always timed; middle words may be bare.
+- `[laugh]`, `[breath]`, and similar are non-speech event tags.
+
+`transcribe()` returns a `TranscribeResults` with the raw `text` plus parsed
+`segments`, each a dict of `speaker`, `start`, `end`, and `text`.
+
+## Evaluation
+
+WhisperDRZ reports two metrics:
+
+- **WER** (word error rate) — Levenshtein word distance after normalization
+  (whisper-normalizer), measuring transcription accuracy.
+- **WDER** (word diarization error rate) — of the words that align between
+  reference and hypothesis, the fraction assigned to the wrong speaker under
+  the best speaker permutation. It isolates diarization quality from WER
+  (insertions/deletions don't count toward WDER).
+
+Measured on the released checkpoint:
+
+| Benchmark | Metric | Score |
+|---|---|---|
+| ESB (English ASR, 1000 utts) | WER | 9.6% macro / 5.9% micro |
+| VoxConverse dev (216 files, overlap-heavy) | DER | 40.7% (63.4% on overlapping speech) |
+| Internal conversational (26 clips) | WER / WDER | 11.1% / 33% |
+
+WhisperDRZ is an ASR-first model: transcription is strong, but diarization trails
+purpose-built systems (which reach ~10–25% DER on VoxConverse) and overlap is its
+weak spot. See [the write-up](blog.md) for full results (timing, non-speech
+events, multilingual) and analysis.
+
+Reproduce WER/WDER on your own data with the metrics in `whisperdrz.evals` and the runner:
+
+```bash
+pip install -e ".[eval]"
+# manifest.jsonl: one {"audio": "a.wav", "text": "[0] ref ... [1] ..."} per line
+python scripts/eval.py manifest.jsonl --model whisperdrz-large-v3.safetensors
+```
+
+`text` is the reference transcript; `[N]` speaker tags are optional and WDER is
+only computed for multi-speaker references.
+
+## Demo
+
+```bash
+python demo/app.py --model whisperdrz-large-v3.safetensors
+```
+
+## License
+
+MIT. See [LICENSE](LICENSE).

whisperdrz-0.1.0/pyproject.toml

@@ -0,0 +1,38 @@
+[build-system]
+requires = ["setuptools", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "whisperdrz"
+version = "0.1.0"
+description = "Speaker-aware ASR with word-level timestamps (inference)"
+readme = "README.md"
+requires-python = ">=3.12"
+license = { text = "MIT" }
+dependencies = [
+    "torch",
+    "torchaudio",
+    "torchcodec",
+    "numpy",
+    "scipy",
+    "einops",
+    "numba",
+    "tiktoken",
+    "safetensors",
+    "huggingface_hub",
+]
+
+[project.optional-dependencies]
+flash = ["flash-attn", "triton"]
+demo = ["gradio"]
+eval = ["jiwer", "whisper-normalizer"]
+
+[project.scripts]
+whisperdrz = "whisperdrz.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+include = ["whisperdrz*"]
+
+[tool.setuptools.package-data]
+whisperdrz = ["assets/*.tiktoken"]

whisperdrz-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

whisperdrz-0.1.0/src/whisperdrz/__init__.py

@@ -0,0 +1,22 @@
+from whisperdrz.model import WhisperDRZ
+from whisperdrz.transcriber import Transcriber
+
+__version__ = "0.1.0"
+
+__all__ = ["WhisperDRZ", "Transcriber", "load_model", "__version__"]
+
+
+def load_model(name_or_path: str, lang: str | None = "en", **kwargs) -> Transcriber:
+    """Load a WhisperDRZ model for transcription.
+
+    Args:
+        name_or_path: A local checkpoint path (``.safetensors`` or ``.pt``), a
+            filename hosted in the default Hugging Face repo, or a Hugging Face
+            repo id.
+        lang: Force a language (e.g. ``"en"``). Pass ``None`` to auto-detect.
+        **kwargs: Forwarded to :class:`~whisperdrz.transcriber.Transcriber`.
+
+    Returns:
+        A :class:`~whisperdrz.transcriber.Transcriber` ready to ``transcribe(audio)``.
+    """
+    return Transcriber(model_path=name_or_path, lang=lang, **kwargs)