muscriptor 0.1.0__py3-none-any.whl

muscriptor-0.1.0.dist-info/METADATA

@@ -0,0 +1,336 @@
+Metadata-Version: 2.4
+Name: muscriptor
+Version: 0.1.0
+Summary: Audio-to-MIDI transcription using a transformer language model
+Project-URL: Homepage, https://github.com/muscriptor/muscriptor
+Project-URL: Repository, https://github.com/muscriptor/muscriptor
+Project-URL: Issues, https://github.com/muscriptor/muscriptor/issues
+Author: Kyutai x Mirelo
+License-Expression: MIT
+License-File: LICENSE
+Keywords: audio,midi,music,transcription,transformer
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Multimedia :: Sound/Audio :: Analysis
+Requires-Python: >=3.10
+Requires-Dist: einops>=0.4
+Requires-Dist: fastapi>=0.136.3
+Requires-Dist: httpx>=0.28.1
+Requires-Dist: huggingface-hub>=0.13
+Requires-Dist: mido>=1.3
+Requires-Dist: numpy>=1.24
+Requires-Dist: python-multipart>=0.0.29
+Requires-Dist: safetensors>=0.4
+Requires-Dist: soundfile>=0.14.0
+Requires-Dist: torch<2.7,>=2.0
+Requires-Dist: typer>=0.10
+Requires-Dist: uvicorn[standard]>=0.48.0
+Description-Content-Type: text/markdown
+
+<p align="center">
+  <img src="web/logo_muscriptor_final.png" alt="MuScriptor logo" width="300">
+</p>
+
+# MuScriptor
+
+MuScriptor is a multi-instrument music transcription model developed by [Kyutai](https://kyutai.org) and [Mirelo](https://www.mirelo.ai).
+MuScriptor is the first music transcription model that has been trained on a large scale dataset of 170k songs from classical music to heavy metal.
+
+[Online Demo](https://muscriptor.kyutai.org) | [Paper] (coming soon) | [HuggingFace](https://huggingface.co/MuScriptor)
+
+<!-- TODO: record the demo GIF (web UI piano roll), save it as assets/demo.gif,
+     then uncomment:
+<p align="center">
+  <img src="assets/demo.gif" alt="MuScriptor web UI: live piano roll while transcribing" width="700">
+</p>
+-->
+
+## Try it locally
+
+You can try it locally with the web UI with:
+
+```bash
+uvx muscriptor serve
+```
+
+or with the CLI:
+
+```bash
+uvx muscriptor transcribe
+```  
+
+
+## Installation
+
+with uv (recommended):
+
+```bash
+uv add muscriptor
+```
+
+```bash
+pip install muscriptor
+```
+
+## Models
+
+Three variants are published under the [MuScriptor](https://huggingface.co/MuScriptor)
+HuggingFace organization. Everywhere a model is selected (`load_model()`, the
+CLI's `--model`, `serve --model`) you can pass the bare size keyword and the
+weights are downloaded and cached automatically. The architecture is a transformer decoder only. Here are the detailed model sizes:
+
+| Variant | Parameters | Layers | Dim | HuggingFace repo |
+|---|---|---|---|---|
+| `small` | 103M | 14 | 768 | [muscriptor-small](https://huggingface.co/MuScriptor/muscriptor-small) |
+| `medium` (default) | 307M | 24 | 1024 | [muscriptor-medium](https://huggingface.co/MuScriptor/muscriptor-medium) |
+| `large` | 1.4B | 48 | 1536 | [muscriptor-large](https://huggingface.co/MuScriptor/muscriptor-large) |
+
+`small` is the practical choice on CPU-only machines, `medium` is the default
+speed/accuracy trade-off, and `large` is the most accurate but really wants a
+GPU. 
+## Usage
+
+```python
+from pathlib import Path
+from muscriptor import TranscriptionModel
+
+# Downloads the default "medium" variant from HuggingFace (cached under
+# ~/.cache/muscriptor/). Also accepts "small"/"large", a local safetensors
+# path, or an hf:// / http(s):// URL.
+model = TranscriptionModel.load_model()
+
+# Stream events as they're transcribed. Optionally tell the model which
+# instruments to expect — run `muscriptor list-instruments` for the names.
+for event in model.transcribe("audio.wav", instruments=["acoustic_piano", "drums"]):
+    print(event)
+
+# Or get a MIDI file directly
+midi_bytes = model.transcribe_to_midi("audio.wav")
+Path("out.mid").write_bytes(midi_bytes)
+```
+
+```python
+from dataclasses import dataclass
+from typing import Generator
+
+
+@dataclass
+class NoteStartEvent:
+    pitch: int
+    # Start of the note in seconds, from the beginning of the audio.
+    start_time: float
+    # A unique index for this note, used to match the corresponding
+    # NoteEndEvent.
+    index: int
+    instrument: str
+
+
+@dataclass
+class NoteEndEvent:
+    end_time: float
+    # The NoteStartEvent this end matches. Convenient for consumers —
+    # when serializing (e.g. to JSON) drop this field and rely on
+    # `start_event_index` to refer back to the start by id.
+    start_event: NoteStartEvent
+
+    @property
+    def start_event_index(self) -> int:
+        return self.start_event.index
+
+
+@dataclass
+class ProgressEvent:
+    # A coarse progress anchor woven into the event stream: `completed` of
+    # `total` 5-second chunks have been transcribed. One is emitted up front
+    # with completed == 0 (so consumers learn `total`), then one per finished
+    # chunk. Advisory only — consumers that just build notes can ignore them.
+    completed: int
+    total: int
+
+
+class TranscriptionModel:
+    ...
+    def transcribe(
+            self,
+            audio: str | Path | tuple[torch.Tensor, int],
+            use_sampling: bool = False,
+            temperature: float = 1.0,
+            cfg_coef: float = 1.0,
+            instruments: list[str] | None = None,
+            batch_size: int | None = None,
+            no_eos_is_ok: bool = True,
+            beam_size: int = 1,
+        ) -> Generator[NoteStartEvent | NoteEndEvent | ProgressEvent, None, None]:
+        """Transcribe audio into a stream of note events.
+
+        Args:
+            audio: Path to an audio file, or a tuple `(tensor, sample_rate)`
+                with a float tensor of shape [T] or [1, T] at `sample_rate`
+                Hz. The tuple form is useful when the audio is already
+                loaded in memory.
+            use_sampling: Use temperature sampling instead of greedy decoding.
+            temperature: Sampling temperature (only used when use_sampling=True).
+            cfg_coef: Classifier-free guidance coefficient. Keep to 1 for the released models (they are post-RL)
+            instruments: Optional list of instrument group names to
+                condition the model on (exact names, e.g.
+                ["acoustic_piano", "drums"]). Run `muscriptor
+                list-instruments` (or GET /instruments on the server)
+                for the full list of valid names.
+            batch_size: Number of 5-second chunks processed per forward
+                pass. `None` (default) picks a value based on the device:
+                1 on CPU, 4 on GPU. Use `batch_size=1` for the lowest
+                streaming latency — larger batches process several chunks
+                together, so events belonging to later chunks of a batch
+                won't arrive until the whole batch finishes. Within a
+                batch, events are always yielded in temporal order; all
+                events from chunk N are emitted before any event from
+                chunk N+1.
+            no_eos_is_ok: If True, a chunk that doesn't emit EOS within
+                the generation budget produces a warning instead of raising.
+            beam_size: Beam search width. 1 (default) uses greedy decoding
+                (or sampling, with use_sampling=True); >= 2 enables beam
+                search, which is slower but can be more accurate.
+
+        Returns:
+            Generator of NoteStartEvent, NoteEndEvent and ProgressEvent
+            objects. Every
+            NoteStartEvent is guaranteed to be followed by exactly one
+            matching NoteEndEvent later in the stream (with the same
+            `index`). Drum hits appear as a NoteStartEvent immediately
+            followed by its matching NoteEndEvent at the same start time
+            plus a tiny duration. Note: this tokenizer does not preserve
+            velocity (loudness) — only onset/offset timing, pitch, and
+            instrument are recovered.
+        """
+
+    def transcribe_to_midi(
+            self,
+            audio: str | Path | tuple[torch.Tensor, int],
+            use_sampling: bool = False,
+            temperature: float = 1.0,
+            cfg_coef: float = 1.0,
+            instruments: list[str] | None = None,
+            batch_size: int | None = None,
+            no_eos_is_ok: bool = True,
+            beam_size: int = 1,
+        ) -> bytes:
+        """Same as `transcribe`, but returns a MIDI file as bytes instead
+        of a generator of events. Useful when you want to save the MIDI
+        to disk or send it over a network without going through the
+        event stream.
+        """
+```
+
+
+## CLI
+
+```bash
+# Transcribe to MIDI (defaults to <audio_file>.mid next to the input)
+muscriptor transcribe audio.wav -o out.mid
+
+# Pick a model variant: small / medium / large (default: medium),
+# a local safetensors path, or an hf:// / http(s):// URL
+muscriptor transcribe audio.wav --model large
+
+# Tell the model which instruments to expect (comma-separated names;
+# run `muscriptor list-instruments` for the full list). Case-insensitive,
+# and unambiguous abbreviations work: 'timp,dist' = timpani + distorted
+# electric guitar
+muscriptor transcribe audio.wav --instruments acoustic_piano,drums
+
+# Get the event stream instead of MIDI: json (single array) or
+# jsonl (one event per line, streamed while transcribing); -o - = stdout
+muscriptor transcribe audio.wav --format jsonl -o -
+
+# Decoding options: temperature sampling, or beam search (slower, can be
+# more accurate)
+muscriptor transcribe audio.wav --sampling -t 0.8
+muscriptor transcribe audio.wav --beam-size 4
+
+# Render a stereo check-mix of the result (left channel = original audio,
+# right channel = synthesized MIDI; requires fluidsynth on PATH)
+muscriptor transcribe audio.wav -o out.mid --auralize check.wav
+```
+
+See `muscriptor transcribe --help` for the full list of options.
+
+## Web UI
+
+A browser client is included under `web/`. The FastAPI server serves both
+the UI and a POST `/transcribe` endpoint that streams `NoteStart`/`NoteEnd`
+events back as Server-Sent Events. The UI accepts an audio file (WAV, or any
+format soundfile/libsndfile can read — mp3, flac, ogg, m4a, …) via drag-and-drop,
+renders a live piano roll while events arrive, auto-plays once enough notes
+are available, and crossfades between the original WAV and the synthesized
+MIDI playback.
+
+### One-time setup
+
+```bash
+uv sync
+cd web && pnpm install && pnpm run build && cd ..
+```
+
+`pnpm run build` is required once — the FastAPI server auto-mounts
+`web/dist/` if it exists. The first build downloads two soundfonts:
+`MuseScore_General.sf2` (215 MB, to the repo root, used by `/auralize`) and
+`MuseScore_General.sf3` (38 MB, the compressed build the UI plays); later
+builds skip files already on disk.
+
+### Run
+
+```bash
+uv run muscriptor serve \
+    --model medium \
+    --device cuda \
+    --host 0.0.0.0 \
+    --port 8222
+```
+
+`--model` accepts a size keyword (`small`, `medium`, `large`) that downloads
+the matching variant from HuggingFace (cached under `~/.cache/muscriptor/`),
+a local safetensors path, or an `hf://` / `http(s)://` URL. It defaults to
+`medium` when omitted.
+
+Then open <http://127.0.0.1:8222/> (or the LAN address of the host) and drop
+a WAV onto the page.
+
+- Drop `--device cuda` if running CPU-only.
+- `--host 0.0.0.0` makes it reachable on the LAN; the default `127.0.0.1`
+  is local-only.
+- Playback runs a full SoundFont synthesizer ([SpessaSynth](https://github.com/spessasus/spessasynth_lib))
+  in the browser, fed with `MuseScore_General.sf3` — the same soundfont the
+  `/auralize` endpoint uses, served by the app itself from `/soundfonts/`,
+  no third-party CDN.
+
+## License
+
+The code in this repository is released under the [MIT license](LICENSE).
+
+The model weights, published on
+[HuggingFace](https://huggingface.co/MuScriptor), are released under the
+[CC BY-NC 4.0 license](https://creativecommons.org/licenses/by-nc/4.0/)
+(non-commercial use).
+
+The MuseScore General SoundFont downloaded for playback / auralization is
+distributed under its own (MIT) license.
+
+## Citation
+
+<!-- TODO: fill in the BibTeX entry when the paper is out:
+```bibtex
+@techreport{muscriptor2026,
+  title  = {MuScriptor: ...},
+  author = {...},
+  year   = {2026},
+  url    = {...},
+}
+```
+-->
+
+The paper is coming soon — a citation entry will be added here with it.

muscriptor-0.1.0.dist-info/RECORD

@@ -0,0 +1,28 @@
+muscriptor/__init__.py,sha256=seJGp-40aQUe5ezeUTMeiChi1EXAwFUzTM0cscrOb7Q,292
+muscriptor/__main__.py,sha256=HX2B06oycjHcSo-OTJAUhfT8OEvkrkGZY1qYqLpnzYY,41
+muscriptor/events.py,sha256=LCS9iOW2G9ig8hMCuEF8au9jaLstINgOAaaZ3YbISWU,7682
+muscriptor/main.py,sha256=PgsU8RRejiNNvqYaSoaRcFK72r4C_jOETMYCg6vfKmk,10301
+muscriptor/server.py,sha256=JcxI_48kgy-1Octawza1y8vX8E4k5dlm1lEmcsbCRR8,10802
+muscriptor/transcription_model.py,sha256=S7DE0tcFiPg2BVC6Q0BKbbsQxin2_7Z9ZzoJFHIRhlc,23459
+muscriptor/models/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+muscriptor/models/lm.py,sha256=K8d0DgFAYR4hR74eItBP_y4GHuvLZnvWRND5ZILiwZU,19328
+muscriptor/modules/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+muscriptor/modules/conditioners.py,sha256=jlRRKdm3MTuNI_o0kje___6ZeiQgdrM81gZgkeRoGzI,10723
+muscriptor/modules/mel_spectrogram.py,sha256=1Fb2OHWPDpgNwhQD2yNC7D8mZH6b9Nf0QRDavYuGdHM,3299
+muscriptor/modules/streaming.py,sha256=0DoVh-hUarDBqiIhe43f1368uscKNHscFJS7ehsI5oo,2548
+muscriptor/modules/transformer.py,sha256=iJKJYvUSHJ_xNMtYz93nMFeOuKd6XdjkpKSB2ra3epI,6900
+muscriptor/tokenizer/__init__.py,sha256=LUiljAqOAGF218DM8RpoTcXnUFuv9GQXukdrscWGiIM,102
+muscriptor/tokenizer/mt3.py,sha256=IAawG5kV1e_Zxfyu8H-BGUEfIqmugJcNmeZeSz9DrvI,7423
+muscriptor/tokenizer/notes.py,sha256=l8PpyAKLSF0PvV6H8j3Dn0HBDSyjD8uHrba-K4u2cMQ,11533
+muscriptor/utils/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+muscriptor/utils/audio.py,sha256=ghzurZ9jUOMODmAZ8vokdaXHVzEY37rHIXeKdrlw2Fg,3818
+muscriptor/utils/auralization.py,sha256=NFhivpV6drHBEKZ45NNFEKVTjyhXzergLOhCHCsjDsg,5155
+muscriptor/utils/download.py,sha256=4PE5tvlirZHu1SvoTdo5e1-8F2xsLHCkLFzA-bpZ-CA,2762
+muscriptor/utils/midi.py,sha256=j2Zx8xTkuCcNY1R82EaxL64ar8h6oY8qO3DQzePRxIs,744
+muscriptor/utils/resample.py,sha256=KypbyuFzNGYwB_mBlYN7CzXU5V4NRLgBnkvyEBQCVIc,8039
+muscriptor/utils/sampling.py,sha256=sNuUUFtxPwBKqr1hMEfmbP2ejPwSCtaenZarwSNuNrI,3380
+muscriptor-0.1.0.dist-info/METADATA,sha256=CfWYkYQ_H-xcal2iL3J2UqAw8iV0ia_ggQfdXsfTbIs,12413
+muscriptor-0.1.0.dist-info/WHEEL,sha256=lCkmxWfQsSc9CfIClYeavTdQeEX2toPqufh9gI35EQA,87
+muscriptor-0.1.0.dist-info/entry_points.txt,sha256=FuoO_u9NK2uJASbxjUxubhHSeIIcxeIJODf9T81sbhQ,52
+muscriptor-0.1.0.dist-info/licenses/LICENSE,sha256=MX2df89NK_JBNrZRAzbLdv71LIc-cjOfspWWk0OHJ4s,1072
+muscriptor-0.1.0.dist-info/RECORD,,

muscriptor-0.1.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.31.0
+Root-Is-Purelib: true
+Tag: py3-none-any

muscriptor-0.1.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+muscriptor = muscriptor.main:main

muscriptor-0.1.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Kyutai x Mirelo
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.