whisperspeech2 0.9.0__tar.gz

whisperspeech2-0.9.0/LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2023 Jakub Piotr Cłapa, Collabora Ltd.
+Copyright (c) 2025 Blair Chintella
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

whisperspeech2-0.9.0/MANIFEST.in

@@ -0,0 +1,5 @@
+include settings.ini
+include LICENSE
+include README.md
+recursive-exclude * __pycache__
+recursive-exclude examples *

whisperspeech2-0.9.0/PKG-INFO

@@ -0,0 +1,122 @@
+Metadata-Version: 2.4
+Name: whisperspeech2
+Version: 0.9.0
+Summary: An Open Source text-to-speech system built by inverting Whisper (fork of WhisperSpeech)
+Home-page: https://github.com/BBC-Esq/whisperspeech2
+Author: Blair Chintella
+Author-email: vici0549@gmail.com
+License: MIT License
+Keywords: tts text-to-speech whisper speech-synthesis
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Natural Language :: English
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: License :: OSI Approved :: MIT License
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: vocos
+Requires-Dist: torch>=2
+Requires-Dist: torchaudio
+Requires-Dist: soundfile
+Requires-Dist: huggingface_hub
+Requires-Dist: fastprogress
+Requires-Dist: fastcore
+Requires-Dist: numpy
+Provides-Extra: speaker
+Requires-Dist: speechbrain<1.0; extra == "speaker"
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: keywords
+Dynamic: license
+Dynamic: license-file
+Dynamic: provides-extra
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# WhisperSpeech2
+
+An Open Source text-to-speech system built by inverting Whisper. This is a fork of [WhisperSpeech](https://github.com/collabora/WhisperSpeech) optimized for inference.
+
+## Installation
+```bash
+pip install whisperspeech2
+```
+
+**Note:** You must also have PyTorch installed. Visit [pytorch.org](https://pytorch.org/get-started/locally/) for installation instructions.
+
+## Quick Start
+```python
+from whisperspeech2.pipeline import Pipeline
+
+# Initialize the pipeline
+pipe = Pipeline(s2a_ref='collabora/whisperspeech:s2a-q4-tiny-en+pl.model')
+
+# Generate audio and save to file
+pipe.generate_to_file('output.wav', "Hello, world!")
+
+# Or get the audio tensor directly
+audio = pipe.generate("Hello, world!")
+```
+
+## Available Models
+
+| Model | Reference |
+|-------|-----------|
+| Tiny | `collabora/whisperspeech:s2a-q4-tiny-en+pl.model` |
+| Base | `collabora/whisperspeech:s2a-q4-base-en+pl.model` |
+| Small | `collabora/whisperspeech:s2a-q4-small-en+pl.model` |
+
+## Speaker Embedding (Optional)
+
+To use custom speaker embeddings, install the optional dependency:
+```bash
+pip install whisperspeech2[speaker]
+```
+
+Then pass an audio file path to clone a voice:
+```python
+pipe.generate_to_file('output.wav', "Hello!", speaker='reference.wav')
+```
+
+## Examples
+
+See the `examples/` directory for more usage examples including GUI applications and streaming playback.
+
+## License
+
+MIT License
+```
+
+### 3. `LICENSE`
+```
+MIT License
+
+Copyright (c) 2025 Blair Chintella
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

whisperspeech2-0.9.0/README.md

@@ -0,0 +1,77 @@
+# WhisperSpeech2
+
+An Open Source text-to-speech system built by inverting Whisper. This is a fork of [WhisperSpeech](https://github.com/collabora/WhisperSpeech) optimized for inference.
+
+## Installation
+```bash
+pip install whisperspeech2
+```
+
+**Note:** You must also have PyTorch installed. Visit [pytorch.org](https://pytorch.org/get-started/locally/) for installation instructions.
+
+## Quick Start
+```python
+from whisperspeech2.pipeline import Pipeline
+
+# Initialize the pipeline
+pipe = Pipeline(s2a_ref='collabora/whisperspeech:s2a-q4-tiny-en+pl.model')
+
+# Generate audio and save to file
+pipe.generate_to_file('output.wav', "Hello, world!")
+
+# Or get the audio tensor directly
+audio = pipe.generate("Hello, world!")
+```
+
+## Available Models
+
+| Model | Reference |
+|-------|-----------|
+| Tiny | `collabora/whisperspeech:s2a-q4-tiny-en+pl.model` |
+| Base | `collabora/whisperspeech:s2a-q4-base-en+pl.model` |
+| Small | `collabora/whisperspeech:s2a-q4-small-en+pl.model` |
+
+## Speaker Embedding (Optional)
+
+To use custom speaker embeddings, install the optional dependency:
+```bash
+pip install whisperspeech2[speaker]
+```
+
+Then pass an audio file path to clone a voice:
+```python
+pipe.generate_to_file('output.wav', "Hello!", speaker='reference.wav')
+```
+
+## Examples
+
+See the `examples/` directory for more usage examples including GUI applications and streaming playback.
+
+## License
+
+MIT License
+```
+
+### 3. `LICENSE`
+```
+MIT License
+
+Copyright (c) 2025 Blair Chintella
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

whisperspeech2-0.9.0/settings.ini

@@ -0,0 +1,26 @@
+[DEFAULT]
+repo = whisperspeech2
+lib_name = %(repo)s
+version = 0.9.0
+min_python = 3.8
+license = MIT
+
+lib_path = whisperspeech2
+nbs_path = nbs
+recursive = False
+
+branch = master
+git_url = https://github.com/%(user)s/%(repo)s
+title = %(lib_name)s
+
+audience = Developers
+author = Blair Chintella
+author_email = vici0549@gmail.com
+copyright = 2025 onwards, %(author)s
+description = An Open Source text-to-speech system built by inverting Whisper (fork of WhisperSpeech)
+keywords = tts text-to-speech whisper speech-synthesis
+language = English
+status = 3
+user = BBC-Esq
+
+requirements = vocos torch>=2 torchaudio soundfile huggingface_hub fastprogress fastcore numpy

whisperspeech2-0.9.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

whisperspeech2-0.9.0/setup.py

@@ -0,0 +1,49 @@
+from pkg_resources import parse_version
+from configparser import ConfigParser
+import setuptools, shlex
+assert parse_version(setuptools.__version__)>=parse_version('36.2')
+
+config = ConfigParser(delimiters=['='])
+config.read('settings.ini')
+cfg = config['DEFAULT']
+
+cfg_keys = 'version description keywords author author_email'.split()
+expected = cfg_keys + "lib_name user branch license status min_python audience language".split()
+for o in expected: assert o in cfg, "missing expected setting: {}".format(o)
+setup_cfg = {o:cfg[o] for o in cfg_keys}
+
+licenses = {
+    'apache2': ('Apache Software License 2.0','OSI Approved :: Apache Software License'),
+    'mit': ('MIT License', 'OSI Approved :: MIT License'),
+    'gpl2': ('GNU General Public License v2', 'OSI Approved :: GNU General Public License v2 (GPLv2)'),
+    'gpl3': ('GNU General Public License v3', 'OSI Approved :: GNU General Public License v3 (GPLv3)'),
+    'bsd3': ('BSD License', 'OSI Approved :: BSD License'),
+}
+statuses = [ '1 - Planning', '2 - Pre-Alpha', '3 - Alpha',
+    '4 - Beta', '5 - Production/Stable', '6 - Mature', '7 - Inactive' ]
+py_versions = '3.8 3.9 3.10 3.11 3.12'.split()
+
+requirements = shlex.split(cfg.get('requirements', ''))
+min_python = cfg['min_python']
+lic = licenses.get(cfg['license'].lower(), (cfg['license'], None))
+
+setuptools.setup(
+    name = cfg['lib_name'],
+    license = lic[0],
+    classifiers = [
+        'Development Status :: ' + statuses[int(cfg['status'])],
+        'Intended Audience :: ' + cfg['audience'].title(),
+        'Natural Language :: ' + cfg['language'].title(),
+    ] + ['Programming Language :: Python :: '+o for o in py_versions[py_versions.index(min_python):]] + (['License :: ' + lic[1] ] if lic[1] else []),
+    url = cfg['git_url'],
+    packages = setuptools.find_packages(),
+    include_package_data = True,
+    install_requires = requirements,
+    extras_require={
+        'speaker': ['speechbrain<1.0'],
+    },
+    python_requires  = '>=' + cfg['min_python'],
+    long_description = open('README.md').read(),
+    long_description_content_type = 'text/markdown',
+    zip_safe = False,
+    **setup_cfg)

whisperspeech2-0.9.0/whisperspeech2/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.9.0"

whisperspeech2-0.9.0/whisperspeech2/a2wav.py

@@ -0,0 +1,72 @@
+__all__ = ['Vocoder']
+
+from vocos import Vocos
+from whisperspeech2 import inference
+import torch
+
+class Vocoder:
+    def __init__(self, repo_id="charactr/vocos-encodec-24khz", device=None, cache_dir=None):
+        if device is None: device = inference.get_compute_device()
+        if device == 'mps': device = 'cpu'
+        self.device = device
+        self.vocos = Vocos.from_pretrained(repo_id).to(device)
+
+    def is_notebook(self):
+        try:
+            return get_ipython().__class__.__name__ == "ZMQInteractiveShell"
+        except:
+            return False
+
+    @torch.no_grad()
+    def decode(self, atoks):
+        if len(atoks.shape) == 3:
+            b,q,t = atoks.shape
+            atoks = atoks.permute(1,0,2)
+        else:
+            q,t = atoks.shape
+        atoks = atoks.to(self.device)
+        features = self.vocos.codes_to_features(atoks)
+        bandwidth_id = torch.tensor({2: 0, 4: 1, 8: 2}[q]).to(self.device)
+        return self.vocos.decode(features, bandwidth_id=bandwidth_id)
+
+    def _save_audio(self, fname, audio_tensor, sample_rate=24000):
+        try:
+            import torchaudio
+            torchaudio.save(fname, audio_tensor, sample_rate, backend="soundfile")
+            return
+        except (ImportError, RuntimeError, TypeError):
+            pass
+        
+        try:
+            import torchaudio
+            torchaudio.save(fname, audio_tensor, sample_rate)
+            return
+        except (ImportError, RuntimeError):
+            pass
+        
+        try:
+            import soundfile as sf
+            audio_np = audio_tensor.numpy().T
+            sf.write(fname, audio_np, sample_rate)
+            return
+        except ImportError:
+            pass
+        
+        raise ImportError(
+            "No audio backend available. Please install either torchaudio or soundfile:\n"
+            "  pip install torchaudio\n"
+            "or\n"
+            "  pip install soundfile"
+        )
+
+    def decode_to_file(self, fname, atoks):
+        audio = self.decode(atoks)
+        self._save_audio(fname, audio.cpu(), 24000)
+        if self.is_notebook():
+            from IPython.display import display, HTML, Audio
+            display(HTML(f'<a href="{fname}" target="_blank">Listen to {fname}</a>'))
+
+    def decode_to_notebook(self, atoks):
+        from IPython.display import display, HTML, Audio
+        audio = self.decode(atoks)
+        display(Audio(audio.cpu().numpy(), rate=24000))

whisperspeech2-0.9.0/whisperspeech2/inference.py

@@ -0,0 +1,56 @@
+__all__ = ['get_compute_device']
+
+import torch
+import torch.nn.functional as F
+from huggingface_hub import hf_hub_download
+from contextlib import nullcontext
+
+def get_default_compute_device():
+    if torch.cuda.is_available() and (torch.version.cuda or torch.version.hip):
+        return 'cuda'
+    elif torch.backends.mps.is_available():
+        return 'mps'
+    else:
+        return 'cpu'
+
+preferred_device = None
+
+def get_compute_device():
+    global preferred_device
+    if preferred_device is None: preferred_device = get_default_compute_device()
+    return preferred_device
+
+def load_model(ref=None, spec=None, device='cpu', cache_dir=None):
+    if spec is not None: return spec
+    if ":" in ref:
+        repo_id, filename = ref.split(":", 1)
+        local_filename = hf_hub_download(repo_id=repo_id, filename=filename, cache_dir=cache_dir)
+    else:
+        local_filename = ref
+    return torch.load(local_filename, map_location=device)
+
+def inference_context():
+    if torch.cuda.is_available():
+        return torch.backends.cuda.sdp_kernel(enable_flash=False, enable_mem_efficient=False, enable_math=True)
+    else:
+        return nullcontext()
+
+def multinomial_sample_one_no_sync(probs_sort):
+    q = torch.empty_like(probs_sort).exponential_(1)
+    return torch.argmax(probs_sort / q, dim=-1, keepdim=True).to(dtype=torch.int)
+
+def logits_to_probs(logits, T=1.0, top_k=None):
+    logits = logits / max(T, 1e-5)
+
+    if top_k is not None:
+        v, _ = torch.topk(logits, min(top_k, logits.size(-1)))
+        pivot = v.select(-1, -1).unsqueeze(-1)
+        logits = torch.where(logits < pivot, -float("Inf"), logits)
+
+    probs = torch.nn.functional.softmax(logits, dim=-1)
+    return probs
+
+def sample(logits, T=1.0, top_k=None):
+    probs = logits_to_probs(logits, T, top_k)
+    idx_next = multinomial_sample_one_no_sync(probs)
+    return idx_next

whisperspeech2-0.9.0/whisperspeech2/languages.py

@@ -0,0 +1,131 @@
+# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/B. Languages.ipynb.
+
+# %% auto 0
+__all__ = ['to_id']
+
+# %% ../nbs/B. Languages.ipynb 3
+LANGUAGES = {
+    "en": "english",
+    "zh": "chinese",
+    "de": "german",
+    "es": "spanish",
+    "ru": "russian",
+    "ko": "korean",
+    "fr": "french",
+    "ja": "japanese",
+    "pt": "portuguese",
+    "tr": "turkish",
+    "pl": "polish",
+    "ca": "catalan",
+    "nl": "dutch",
+    "ar": "arabic",
+    "sv": "swedish",
+    "it": "italian",
+    "id": "indonesian",
+    "hi": "hindi",
+    "fi": "finnish",
+    "vi": "vietnamese",
+    "he": "hebrew",
+    "uk": "ukrainian",
+    "el": "greek",
+    "ms": "malay",
+    "cs": "czech",
+    "ro": "romanian",
+    "da": "danish",
+    "hu": "hungarian",
+    "ta": "tamil",
+    "no": "norwegian",
+    "th": "thai",
+    "ur": "urdu",
+    "hr": "croatian",
+    "bg": "bulgarian",
+    "lt": "lithuanian",
+    "la": "latin",
+    "mi": "maori",
+    "ml": "malayalam",
+    "cy": "welsh",
+    "sk": "slovak",
+    "te": "telugu",
+    "fa": "persian",
+    "lv": "latvian",
+    "bn": "bengali",
+    "sr": "serbian",
+    "az": "azerbaijani",
+    "sl": "slovenian",
+    "kn": "kannada",
+    "et": "estonian",
+    "mk": "macedonian",
+    "br": "breton",
+    "eu": "basque",
+    "is": "icelandic",
+    "hy": "armenian",
+    "ne": "nepali",
+    "mn": "mongolian",
+    "bs": "bosnian",
+    "kk": "kazakh",
+    "sq": "albanian",
+    "sw": "swahili",
+    "gl": "galician",
+    "mr": "marathi",
+    "pa": "punjabi",
+    "si": "sinhala",
+    "km": "khmer",
+    "sn": "shona",
+    "yo": "yoruba",
+    "so": "somali",
+    "af": "afrikaans",
+    "oc": "occitan",
+    "ka": "georgian",
+    "be": "belarusian",
+    "tg": "tajik",
+    "sd": "sindhi",
+    "gu": "gujarati",
+    "am": "amharic",
+    "yi": "yiddish",
+    "lo": "lao",
+    "uz": "uzbek",
+    "fo": "faroese",
+    "ht": "haitian creole",
+    "ps": "pashto",
+    "tk": "turkmen",
+    "nn": "nynorsk",
+    "mt": "maltese",
+    "sa": "sanskrit",
+    "lb": "luxembourgish",
+    "my": "myanmar",
+    "bo": "tibetan",
+    "tl": "tagalog",
+    "mg": "malagasy",
+    "as": "assamese",
+    "tt": "tatar",
+    "haw": "hawaiian",
+    "ln": "lingala",
+    "ha": "hausa",
+    "ba": "bashkir",
+    "jw": "javanese",
+    "su": "sundanese",
+}
+
+# %% ../nbs/B. Languages.ipynb 4
+# language code lookup by name, with a few language aliases
+TO_LANGUAGE_CODE = {
+    **{language: code for code, language in LANGUAGES.items()},
+    "burmese": "my",
+    "valencian": "ca",
+    "flemish": "nl",
+    "haitian": "ht",
+    "letzeburgesch": "lb",
+    "pushto": "ps",
+    "panjabi": "pa",
+    "moldavian": "ro",
+    "moldovan": "ro",
+    "sinhalese": "si",
+    "castilian": "es",
+}
+
+# %% ../nbs/B. Languages.ipynb 5
+languages = tuple(LANGUAGES.keys())
+
+# %% ../nbs/B. Languages.ipynb 6
+def to_id(lang):
+    return languages.index(TO_LANGUAGE_CODE.get(lang, lang))