muscriptor 0.3.0__tar.gz → 0.4.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (127) hide show
  1. muscriptor-0.4.0/.github/ISSUE_TEMPLATE/bad-transcription.yml +54 -0
  2. muscriptor-0.4.0/.github/ISSUE_TEMPLATE/bug.yml +36 -0
  3. {muscriptor-0.3.0 → muscriptor-0.4.0}/.gitignore +2 -0
  4. {muscriptor-0.3.0 → muscriptor-0.4.0}/Dockerfile +23 -1
  5. muscriptor-0.4.0/PKG-INFO +265 -0
  6. muscriptor-0.4.0/README.md +227 -0
  7. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/main.py +210 -22
  8. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/server.py +183 -6
  9. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/tokenizer/notes.py +69 -33
  10. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/transcription_model.py +53 -19
  11. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/utils/auralization.py +4 -1
  12. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/utils/beats.py +61 -19
  13. muscriptor-0.4.0/muscriptor/utils/midi.py +492 -0
  14. muscriptor-0.4.0/muscriptor/utils/sheets.py +465 -0
  15. muscriptor-0.4.0/muscriptor/web_dist/assets/index-BVRQFpi6.js +223 -0
  16. muscriptor-0.4.0/muscriptor/web_dist/assets/index-DSvjcK8_.css +2 -0
  17. muscriptor-0.4.0/muscriptor/web_dist/headache_by_lost_deposit_10s.mp3 +0 -0
  18. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/web_dist/index.html +7 -3
  19. {muscriptor-0.3.0 → muscriptor-0.4.0}/pyproject.toml +17 -5
  20. {muscriptor-0.3.0 → muscriptor-0.4.0}/swarm.yml +9 -0
  21. {muscriptor-0.3.0 → muscriptor-0.4.0}/tests/test_beats.py +37 -2
  22. {muscriptor-0.3.0 → muscriptor-0.4.0}/tests/test_cli.py +80 -2
  23. {muscriptor-0.3.0 → muscriptor-0.4.0}/tests/test_integration.py +10 -6
  24. muscriptor-0.4.0/tests/test_midi.py +599 -0
  25. {muscriptor-0.3.0 → muscriptor-0.4.0}/tests/test_server.py +323 -7
  26. muscriptor-0.4.0/tests/test_sheets.py +385 -0
  27. {muscriptor-0.3.0 → muscriptor-0.4.0}/uv.lock +604 -363
  28. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/index.html +5 -1
  29. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/package.json +1 -0
  30. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/pnpm-lock.yaml +8 -0
  31. muscriptor-0.4.0/web/public/headache_by_lost_deposit_10s.mp3 +0 -0
  32. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/App.tsx +42 -14
  33. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/components/Faq.tsx +11 -3
  34. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/components/FeedbackLine.tsx +1 -1
  35. muscriptor-0.4.0/web/src/components/MidiExportDialog.tsx +287 -0
  36. muscriptor-0.4.0/web/src/components/OutputBar.tsx +324 -0
  37. muscriptor-0.4.0/web/src/components/SheetsDialog.tsx +186 -0
  38. muscriptor-0.4.0/web/src/components/TimingPanel.tsx +228 -0
  39. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/components/WelcomeScreen.tsx +2 -2
  40. muscriptor-0.4.0/web/src/export.ts +184 -0
  41. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/hooks/useTranscription.ts +63 -38
  42. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/pianoroll.ts +82 -12
  43. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/style.css +13 -0
  44. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/vite.config.ts +2 -0
  45. muscriptor-0.3.0/PKG-INFO +0 -417
  46. muscriptor-0.3.0/README.md +0 -379
  47. muscriptor-0.3.0/muscriptor/utils/midi.py +0 -55
  48. muscriptor-0.3.0/muscriptor/web_dist/assets/index-ClK01VrS.js +0 -223
  49. muscriptor-0.3.0/muscriptor/web_dist/assets/index-DfGntwya.css +0 -2
  50. muscriptor-0.3.0/tests/test_midi.py +0 -174
  51. muscriptor-0.3.0/web/src/components/OutputBar.tsx +0 -203
  52. {muscriptor-0.3.0 → muscriptor-0.4.0}/.dockerignore +0 -0
  53. {muscriptor-0.3.0 → muscriptor-0.4.0}/.github/workflows/deploy.yml +0 -0
  54. {muscriptor-0.3.0 → muscriptor-0.4.0}/.github/workflows/id_ed25519.pub +0 -0
  55. {muscriptor-0.3.0 → muscriptor-0.4.0}/.github/workflows/pypi.yml +0 -0
  56. {muscriptor-0.3.0 → muscriptor-0.4.0}/.pre-commit-config.yaml +0 -0
  57. {muscriptor-0.3.0 → muscriptor-0.4.0}/.python-version +0 -0
  58. {muscriptor-0.3.0 → muscriptor-0.4.0}/LICENSE +0 -0
  59. {muscriptor-0.3.0 → muscriptor-0.4.0}/deploy.sh +0 -0
  60. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/__init__.py +0 -0
  61. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/__main__.py +0 -0
  62. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/accelerator.py +0 -0
  63. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/events.py +0 -0
  64. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/models/__init__.py +0 -0
  65. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/models/lm.py +0 -0
  66. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/modules/__init__.py +0 -0
  67. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/modules/conditioners.py +0 -0
  68. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/modules/mel_spectrogram.py +0 -0
  69. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/modules/streaming.py +0 -0
  70. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/modules/transformer.py +0 -0
  71. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/soundfonts.py +0 -0
  72. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/tokenizer/__init__.py +0 -0
  73. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/tokenizer/mt3.py +0 -0
  74. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/utils/__init__.py +0 -0
  75. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/utils/audio.py +0 -0
  76. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/utils/download.py +0 -0
  77. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/utils/resample.py +0 -0
  78. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/utils/sampling.py +0 -0
  79. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/web_dist/assets/spessasynth_processor.min-tdOdhV7p.js +0 -0
  80. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/web_dist/favicon.png +0 -0
  81. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/web_dist/headache_by_lost_deposit_1min.mp3 +0 -0
  82. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/web_dist/kyutai-logo.svg +0 -0
  83. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/web_dist/mirelo-logo.svg +0 -0
  84. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/web_dist/muscriptor-logo-v2.png +0 -0
  85. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/web_dist/muscriptor-logo-v4.svg +0 -0
  86. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/web_dist/muscriptor-mark-v4.svg +0 -0
  87. {muscriptor-0.3.0 → muscriptor-0.4.0}/muscriptor/web_dist/muscriptor-wave.svg +0 -0
  88. {muscriptor-0.3.0 → muscriptor-0.4.0}/tests/__init__.py +0 -0
  89. {muscriptor-0.3.0 → muscriptor-0.4.0}/tests/conftest.py +0 -0
  90. {muscriptor-0.3.0 → muscriptor-0.4.0}/tests/encode_helpers.py +0 -0
  91. {muscriptor-0.3.0 → muscriptor-0.4.0}/tests/test_conditioners.py +0 -0
  92. {muscriptor-0.3.0 → muscriptor-0.4.0}/tests/test_download.py +0 -0
  93. {muscriptor-0.3.0 → muscriptor-0.4.0}/tests/test_events.py +0 -0
  94. {muscriptor-0.3.0 → muscriptor-0.4.0}/tests/test_notes.py +0 -0
  95. {muscriptor-0.3.0 → muscriptor-0.4.0}/tests/test_prelude_forcing.py +0 -0
  96. {muscriptor-0.3.0 → muscriptor-0.4.0}/tests/test_sampling.py +0 -0
  97. {muscriptor-0.3.0 → muscriptor-0.4.0}/tests/test_strict_instruments.py +0 -0
  98. {muscriptor-0.3.0 → muscriptor-0.4.0}/tests/test_transcription_model.py +0 -0
  99. {muscriptor-0.3.0 → muscriptor-0.4.0}/tests/test_transformer.py +0 -0
  100. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/logo_muscriptor_final.png +0 -0
  101. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/package-lock.json +0 -0
  102. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/public/favicon.png +0 -0
  103. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/public/headache_by_lost_deposit_1min.mp3 +0 -0
  104. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/public/kyutai-logo.svg +0 -0
  105. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/public/mirelo-logo.svg +0 -0
  106. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/public/muscriptor-logo-v2.png +0 -0
  107. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/public/muscriptor-logo-v4.svg +0 -0
  108. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/public/muscriptor-mark-v4.svg +0 -0
  109. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/public/muscriptor-wave.svg +0 -0
  110. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/analytics.ts +0 -0
  111. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/audio.ts +0 -0
  112. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/components/Button.tsx +0 -0
  113. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/components/ConditioningPanel.tsx +0 -0
  114. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/components/ConsentBanner.tsx +0 -0
  115. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/components/Controls.tsx +0 -0
  116. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/components/DropOverlay.tsx +0 -0
  117. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/components/Footer.tsx +0 -0
  118. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/components/InstrumentList.tsx +0 -0
  119. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/components/PianoRollCanvas.tsx +0 -0
  120. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/components/icons.tsx +0 -0
  121. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/hooks/useAudioEngine.ts +0 -0
  122. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/instruments.ts +0 -0
  123. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/main.tsx +0 -0
  124. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/progress.ts +0 -0
  125. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/sse.ts +0 -0
  126. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/src/vite-env.d.ts +0 -0
  127. {muscriptor-0.3.0 → muscriptor-0.4.0}/web/tsconfig.json +0 -0
@@ -0,0 +1,54 @@
1
+ name: Bad transcription
2
+ description: The model transcribed a song wrong (missing notes, wrong instrument, wrong rhythm...)
3
+ labels: ["transcription quality"]
4
+ body:
5
+ - type: markdown
6
+ attributes:
7
+ value: |
8
+ **We can't do anything without the audio.** Every transcription report needs the
9
+ exact file you were transcribing, otherwise we cannot fix the issue.
10
+
11
+ Note that the model is not perfect: some wrong notes are an expected
12
+ failure case and the answer will be "fix it up afterwards in a DAW or
13
+ notation editor". We're looking for systematic problems - long dropouts,
14
+ everything landing on one track, a part that never appears.
15
+ - type: textarea
16
+ id: audio
17
+ attributes:
18
+ label: The audio file
19
+ description: |
20
+ Drag-and-drop the file here, or paste a Google Drive/Dropbox link. If you can't
21
+ share it publicly, email it to muscriptor@kyutai.org and write "sent by email" here.
22
+ placeholder: (drag the file into this box)
23
+ validations:
24
+ required: true
25
+ - type: textarea
26
+ id: what
27
+ attributes:
28
+ label: What's wrong, and where?
29
+ description: |
30
+ One or two sentences, plus timestamps, e.g. "The guitar disappears from 3:05 to 3:45".
31
+ A screenshot of the piano roll helps.
32
+ placeholder: |
33
+ The hi-hat is transcribed as 8 hits per bar instead of 4, from the start.
34
+ validations:
35
+ required: true
36
+ - type: input
37
+ id: command
38
+ attributes:
39
+ label: How did you run it?
40
+ description: The exact command, or "web app at muscriptor.com".
41
+ placeholder: muscriptor transcribe song.wav --model large
42
+ validations:
43
+ required: true
44
+ - type: input
45
+ id: version
46
+ attributes:
47
+ label: Version
48
+ description: Output of `muscriptor --version`. Skip if you used the web app.
49
+ placeholder: "0.3.0"
50
+ - type: markdown
51
+ attributes:
52
+ value: |
53
+ Please write your report yourself, even imperfectly. Short human-written text is
54
+ much easier for us to act on than a long AI-generated one.
@@ -0,0 +1,36 @@
1
+ name: Bug report
2
+ description: A crash, an error message, wrong output from the code (not from the model)
3
+ labels: ["bug"]
4
+ body:
5
+ - type: markdown
6
+ attributes:
7
+ value: |
8
+ For "the model transcribed my song wrong", use the *Bad transcription* template instead.
9
+ - type: textarea
10
+ id: what
11
+ attributes:
12
+ label: What happened?
13
+ description: What you expected, what you got. Paste the full traceback or error output.
14
+ validations:
15
+ required: true
16
+ - type: textarea
17
+ id: repro
18
+ attributes:
19
+ label: How to reproduce
20
+ description: "The exact command(s). If it only happens with a specific file, attach it:
21
+ Drag-and-drop the file here, or paste a Google Drive/Dropbox link. If you can't
22
+ share it publicly, email it to muscriptor@kyutai.org and write \"sent by email\" here."
23
+ placeholder: |
24
+ 1. muscriptor transcribe song.wav --format sheets
25
+ 2. ...
26
+ validations:
27
+ required: true
28
+ - type: textarea
29
+ id: env
30
+ attributes:
31
+ label: Environment
32
+ description: OS, Python version, `muscriptor --version`, device (CPU/CUDA/MPS).
33
+ placeholder: |
34
+ macOS 15.1, Python 3.11, muscriptor 0.3.0, MPS
35
+ validations:
36
+ required: true
@@ -5,6 +5,8 @@
5
5
  *.wav
6
6
  *.mp3
7
7
  !headache_by_lost_deposit_1min.mp3
8
+ # For development
9
+ !headache_by_lost_deposit_10s.mp3
8
10
  *.ogg
9
11
  *.sf2
10
12
  web/node_modules
@@ -27,10 +27,32 @@ ENV UV_COMPILE_BYTECODE=1 \
27
27
  WORKDIR /app
28
28
 
29
29
  # fluidsynth is required at runtime for MIDI auralization (the /auralize endpoint).
30
+ # The rest are what the MuseScore AppImage below needs from the system: its own
31
+ # AppRun substitutes bundled fallbacks for OpenGL/jack/nss/pipewire, but these
32
+ # five it expects to find installed.
30
33
  RUN apt-get update \
31
- && apt-get install -y --no-install-recommends fluidsynth \
34
+ && apt-get install -y --no-install-recommends \
35
+ fluidsynth \
36
+ curl \
37
+ libasound2 \
38
+ libgl1 \
39
+ libegl1 \
40
+ libfontconfig1 \
41
+ libglib2.0-0 \
32
42
  && rm -rf /var/lib/apt/lists/*
33
43
 
44
+
45
+ ARG MUSESCORE_URL=https://github.com/musescore/MuseScore/releases/download/v4.7.4/MuseScore-Studio-4.7.4.260706075-x86_64.AppImage
46
+ RUN curl -fsSL -o /tmp/musescore.AppImage "$MUSESCORE_URL" \
47
+ && chmod +x /tmp/musescore.AppImage \
48
+ && cd /opt && /tmp/musescore.AppImage --appimage-extract > /dev/null \
49
+ && mv squashfs-root musescore \
50
+ && rm /tmp/musescore.AppImage \
51
+ && QT_QPA_PLATFORM=offscreen MU_QT_QPA_PLATFORM=offscreen \
52
+ /opt/musescore/AppRun --version
53
+
54
+ ENV MUSCRIPTOR_MUSESCORE=/opt/musescore/AppRun
55
+
34
56
  RUN --mount=type=cache,target=/root/.cache/uv \
35
57
  --mount=type=bind,source=pyproject.toml,target=pyproject.toml \
36
58
  --mount=type=bind,source=uv.lock,target=uv.lock \
@@ -0,0 +1,265 @@
1
+ Metadata-Version: 2.5
2
+ Name: muscriptor
3
+ Version: 0.4.0
4
+ Summary: Music transcription to MIDI and sheet music using a transformer language model
5
+ Project-URL: Homepage, https://github.com/muscriptor/muscriptor
6
+ Project-URL: Repository, https://github.com/muscriptor/muscriptor
7
+ Project-URL: Issues, https://github.com/muscriptor/muscriptor/issues
8
+ Author: Kyutai x Mirelo
9
+ License-Expression: MIT
10
+ License-File: LICENSE
11
+ Keywords: audio,midi,music,musicxml,notation,sheet-music,tablature,transcription,transformer
12
+ Classifier: Development Status :: 4 - Beta
13
+ Classifier: Intended Audience :: Developers
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Programming Language :: Python :: 3.10
16
+ Classifier: Programming Language :: Python :: 3.11
17
+ Classifier: Programming Language :: Python :: 3.12
18
+ Classifier: Programming Language :: Python :: 3.13
19
+ Classifier: Topic :: Multimedia :: Sound/Audio :: Analysis
20
+ Requires-Python: >=3.10
21
+ Requires-Dist: beat-this>=1.1
22
+ Requires-Dist: einops>=0.4
23
+ Requires-Dist: fastapi>=0.136.3
24
+ Requires-Dist: httpx>=0.28.1
25
+ Requires-Dist: huggingface-hub>=0.13
26
+ Requires-Dist: mido>=1.3
27
+ Requires-Dist: numpy<2,>=1.24; sys_platform == 'darwin' and platform_machine == 'x86_64'
28
+ Requires-Dist: numpy>=2; sys_platform != 'darwin' or platform_machine != 'x86_64'
29
+ Requires-Dist: packaging>=20.0
30
+ Requires-Dist: python-multipart>=0.0.29
31
+ Requires-Dist: safetensors>=0.4
32
+ Requires-Dist: soundfile>=0.14.0
33
+ Requires-Dist: torch<2.3,>=2.0; sys_platform == 'darwin' and platform_machine == 'x86_64'
34
+ Requires-Dist: torch>=2.3; sys_platform != 'darwin' or platform_machine != 'x86_64'
35
+ Requires-Dist: typer>=0.10
36
+ Requires-Dist: uvicorn[standard]>=0.48.0
37
+ Description-Content-Type: text/markdown
38
+
39
+ <p align="center">
40
+ <img src="web/logo_muscriptor_final.png" alt="MuScriptor logo" width="300">
41
+ </p>
42
+
43
+ # MuScriptor
44
+
45
+ MuScriptor is a multi-instrument music transcription model developed by [Kyutai](https://kyutai.org) and [Mirelo](https://www.mirelo.ai).
46
+ It turns a recording into MIDI and into sheet music.
47
+ It's the most accurate open-source transcription model.
48
+ You can use the model [here](https://muscriptor.kyutai.org) or self-host it using this repository.
49
+
50
+
51
+ [Use it](https://muscriptor.kyutai.org) | [Paper](https://arxiv.org/abs/2607.08168v2) | [HuggingFace](https://huggingface.co/MuScriptor)
52
+
53
+ <!-- TODO: record the demo GIF (web UI piano roll), save it as assets/demo.gif,
54
+ then uncomment:
55
+ <p align="center">
56
+ <img src="assets/demo.gif" alt="MuScriptor web UI: live piano roll while transcribing" width="700">
57
+ </p>
58
+ -->
59
+
60
+ ## HuggingFace login (required)
61
+
62
+ To use MuScriptor locally, you first need to log into [HuggingFace](https://huggingface.co/MuScriptor)
63
+ and accept the CC BY-NC 4.0 license.
64
+
65
+ 1. Accept the model license on the model page for the [small](https://huggingface.co/MuScriptor/muscriptor-small),
66
+ [medium](https://huggingface.co/MuScriptor/muscriptor-medium) or [large](https://huggingface.co/MuScriptor/muscriptor-large) model
67
+ (access is granted automatically).
68
+ 2. Authenticate on your machine:
69
+
70
+ ```bash
71
+ uvx hf auth login
72
+ ```
73
+
74
+ or set a token (create one at
75
+ [huggingface.co/settings/tokens](https://huggingface.co/settings/tokens)):
76
+
77
+ ```bash
78
+ export HF_TOKEN=hf_...
79
+ ```
80
+
81
+ The weights are then automatically downloaded on first use and cached locally.
82
+
83
+ ## Try it locally
84
+
85
+ After Hugging Face authentication, you can use MuScriptor with `uvx` without having to clone this repo.
86
+
87
+ Some platforms need an extra `uvx` flag, on every `uvx muscriptor` command:
88
+
89
+ | Platform | Command |
90
+ |---|---|
91
+ | Linux, macOS with Apple Silicon | `uvx muscriptor serve` |
92
+ | Windows (to use the GPU) | `uvx --torch-backend=cu128 muscriptor serve` |
93
+ | macOS with Intel | `uvx --python 3.12 muscriptor serve` |
94
+
95
+ On Windows the default PyTorch backend is `cpu`, so the GPU needs
96
+ `--torch-backend=cu128`. On Intel Macs, PyTorch stopped shipping x86_64 wheels
97
+ after torch 2.2.2, which supports Python ≤ 3.12, so the Python version has to
98
+ be pinned (if you install with pip/uv instead, use Python 3.10–3.12).
99
+
100
+ ## Web UI
101
+
102
+ You can host the web UI locally with:
103
+
104
+ ```bash
105
+ uvx muscriptor serve
106
+ ```
107
+
108
+ This gives you the same UI as hosted on https://muscriptor.kyutai.org/, just with a different look.
109
+
110
+ The sheet music download needs **MuseScore 4 or newer** installed separately (see
111
+ [Sheet music](#sheet-music) below). Without it, everything except that download still works.
112
+
113
+ ## Command-line interface (CLI)
114
+
115
+ ```bash
116
+ uvx muscriptor transcribe path/to/audio_file.wav
117
+ ```
118
+
119
+ See `--help` for all the options.
120
+
121
+ ### Tempo and quantization
122
+
123
+ The beats, time signature and bar lines are detected from the audio and written into the MIDI.
124
+ When the tempo isn't steady (no metronome), the MIDI gets a tempo map that follows the
125
+ performance, so the bar lines stay on the music; `--fixed-tempo` writes one average tempo instead.
126
+ When the detection is off, you can correct it, and tidy the notes up for notation software or a
127
+ DAW grid:
128
+
129
+ ```bash
130
+ muscriptor transcribe audio.wav --bpm 96 --beats-per-bar 3 --first-downbeat 1.2 \
131
+ --quantize --grid 4 --fill-gaps 0.25 --trim-leading-bars
132
+ ```
133
+
134
+ - `--quantize` snaps notes onto the beat grid (`--grid 4` = sixteenths, `3` = eighth triplets;
135
+ by default the one the notes sit on best), `--quantize-strength 0.5` only halfway.
136
+ - `--fill-gaps` holds notes until the next one across rests of up to that many beats, which
137
+ notation software such as Guitar Pro would otherwise write out.
138
+ - `--trim-leading-bars` drops the empty bars before the first note.
139
+ - `--beat-scale 2` (or `0.5`) when the detected tempo is half (or double) what it should be.
140
+
141
+ The web UI has the same settings: the "Tempo & bars" panel and the MIDI download dialog.
142
+
143
+ ### Sheet music
144
+
145
+ Using the CLI with `--format sheets` engraves the transcription as readable notation instead of
146
+ writing a single MIDI file.
147
+
148
+ ```bash
149
+ muscriptor transcribe audio.wav --format sheets --output score/
150
+ ```
151
+
152
+ The output structure looks like this:
153
+
154
+ ```
155
+ score/
156
+ ├── score.mid the transcription, as quantized MIDI
157
+ ├── score.musicxml the engraved score, as MusicXML
158
+ ├── full_score.pdf every instrument on one system
159
+ ├── 01_electric_guitar.pdf one PDF per instrument …
160
+ ├── 01_electric_guitar_tab.pdf … and a tablature PDF for fretted ones
161
+ ├── 02_electric_bass.pdf
162
+ ├── 02_electric_bass_tab.pdf
163
+ └── 03_drum_kit.pdf
164
+ ```
165
+
166
+ This needs **MuseScore 4 or newer** installed separately. Downloads for every
167
+ platform are at [musescore.org/en/download](https://musescore.org/en/download).
168
+ Set `$MUSCRIPTOR_MUSESCORE` if it lives somewhere unusual.
169
+
170
+ It works best if there is a clear beat, because that allows us to quantize the notes (snap
171
+ them to a grid) for a cleaner transcription. Without a steady tempo the score follows the
172
+ performance's tempo changes; rubato recordings will still work noticeably worse.
173
+
174
+ ## Using from Python
175
+
176
+ MuScriptor is also on PyPI, so you can install it with with uv (recommended) or with pip:
177
+
178
+ ```bash
179
+ uv add muscriptor
180
+ ```
181
+
182
+ ```bash
183
+ pip install muscriptor
184
+ ```
185
+
186
+ Ask your coding agent to show you around the codebase.
187
+
188
+ ## Models
189
+
190
+ Three variants are published under the [MuScriptor](https://huggingface.co/MuScriptor)
191
+ HuggingFace organization. Everywhere a model is selected (`load_model()`, the
192
+ CLI's `--model`, `serve --model`) you can pass the bare size keyword and the
193
+ weights are downloaded and cached automatically. The architecture is a transformer decoder only. Here are the detailed model sizes:
194
+
195
+ | Variant | Parameters | Layers | Dim | HuggingFace repo |
196
+ |---|---|---|---|---|
197
+ | `small` | 103M | 14 | 768 | [muscriptor-small](https://huggingface.co/MuScriptor/muscriptor-small) |
198
+ | `medium` (default) | 307M | 24 | 1024 | [muscriptor-medium](https://huggingface.co/MuScriptor/muscriptor-medium) |
199
+ | `large` | 1.4B | 48 | 1536 | [muscriptor-large](https://huggingface.co/MuScriptor/muscriptor-large) |
200
+
201
+ `small` is the practical choice on CPU-only machines, `medium` is the default
202
+ speed/accuracy trade-off, and `large` is the most accurate but really wants a
203
+ GPU. On Apple Silicon the model runs on Metal (MPS) automatically.
204
+
205
+ ## Developing
206
+
207
+ To set up for development, get [uv](https://docs.astral.sh/uv/getting-started/installation/),
208
+ clone this repo and run:
209
+ ```bash
210
+ uv sync
211
+ ```
212
+
213
+ For the web UI, you also need [pnpm](https://pnpm.io/installation) and Node
214
+ (can be installed [via pnpm](https://pnpm.io/cli/runtime)).
215
+ Then run:
216
+
217
+ ```bash
218
+ cd web
219
+ pnpm install
220
+ pnpm run build
221
+ ```
222
+
223
+ If you're not editing the frontend, you only need to do this once.
224
+ If you are, run `pnpm dev` instead for a hot-reloading dev server.
225
+ Start the backend alongside it with it using `uv run muscriptor serve --port 8222`
226
+ and then open the frontend on http://localhost:5173/.
227
+
228
+ ### Run
229
+
230
+ After this setup, you can run Muscriptor from your local repository using
231
+ `uv` (note - not `uvx` like before):
232
+
233
+ ```bash
234
+ uv run muscriptor serve
235
+ # or
236
+ uv run muscriptor transcribe path/to/audio_file.wav
237
+ ```
238
+
239
+ Again, see `--help` for more options.
240
+
241
+ ## License
242
+
243
+ The code in this repository is released under the [MIT license](LICENSE).
244
+
245
+ The model weights, published on
246
+ [HuggingFace](https://huggingface.co/MuScriptor), are released under the
247
+ [CC BY-NC 4.0 license](https://creativecommons.org/licenses/by-nc/4.0/)
248
+ (non-commercial use).
249
+
250
+ The MuseScore General SoundFont downloaded for playback is
251
+ distributed under its own (MIT) license.
252
+
253
+ ## Citation
254
+
255
+ ```bibtex
256
+ @misc{rouard2026muscriptoropenmodelmultiinstrument,
257
+ title={MuScriptor: An Open Model for Multi-Instrument Music Transcription},
258
+ author={Simon Rouard and Michael Krause and Axel Roebel and Carl-Johann Simon-Gabriel and Alexandre Défossez},
259
+ year={2026},
260
+ eprint={2607.08168},
261
+ archivePrefix={arXiv},
262
+ primaryClass={cs.SD},
263
+ url={https://arxiv.org/abs/2607.08168},
264
+ }
265
+ ```
@@ -0,0 +1,227 @@
1
+ <p align="center">
2
+ <img src="web/logo_muscriptor_final.png" alt="MuScriptor logo" width="300">
3
+ </p>
4
+
5
+ # MuScriptor
6
+
7
+ MuScriptor is a multi-instrument music transcription model developed by [Kyutai](https://kyutai.org) and [Mirelo](https://www.mirelo.ai).
8
+ It turns a recording into MIDI and into sheet music.
9
+ It's the most accurate open-source transcription model.
10
+ You can use the model [here](https://muscriptor.kyutai.org) or self-host it using this repository.
11
+
12
+
13
+ [Use it](https://muscriptor.kyutai.org) | [Paper](https://arxiv.org/abs/2607.08168v2) | [HuggingFace](https://huggingface.co/MuScriptor)
14
+
15
+ <!-- TODO: record the demo GIF (web UI piano roll), save it as assets/demo.gif,
16
+ then uncomment:
17
+ <p align="center">
18
+ <img src="assets/demo.gif" alt="MuScriptor web UI: live piano roll while transcribing" width="700">
19
+ </p>
20
+ -->
21
+
22
+ ## HuggingFace login (required)
23
+
24
+ To use MuScriptor locally, you first need to log into [HuggingFace](https://huggingface.co/MuScriptor)
25
+ and accept the CC BY-NC 4.0 license.
26
+
27
+ 1. Accept the model license on the model page for the [small](https://huggingface.co/MuScriptor/muscriptor-small),
28
+ [medium](https://huggingface.co/MuScriptor/muscriptor-medium) or [large](https://huggingface.co/MuScriptor/muscriptor-large) model
29
+ (access is granted automatically).
30
+ 2. Authenticate on your machine:
31
+
32
+ ```bash
33
+ uvx hf auth login
34
+ ```
35
+
36
+ or set a token (create one at
37
+ [huggingface.co/settings/tokens](https://huggingface.co/settings/tokens)):
38
+
39
+ ```bash
40
+ export HF_TOKEN=hf_...
41
+ ```
42
+
43
+ The weights are then automatically downloaded on first use and cached locally.
44
+
45
+ ## Try it locally
46
+
47
+ After Hugging Face authentication, you can use MuScriptor with `uvx` without having to clone this repo.
48
+
49
+ Some platforms need an extra `uvx` flag, on every `uvx muscriptor` command:
50
+
51
+ | Platform | Command |
52
+ |---|---|
53
+ | Linux, macOS with Apple Silicon | `uvx muscriptor serve` |
54
+ | Windows (to use the GPU) | `uvx --torch-backend=cu128 muscriptor serve` |
55
+ | macOS with Intel | `uvx --python 3.12 muscriptor serve` |
56
+
57
+ On Windows the default PyTorch backend is `cpu`, so the GPU needs
58
+ `--torch-backend=cu128`. On Intel Macs, PyTorch stopped shipping x86_64 wheels
59
+ after torch 2.2.2, which supports Python ≤ 3.12, so the Python version has to
60
+ be pinned (if you install with pip/uv instead, use Python 3.10–3.12).
61
+
62
+ ## Web UI
63
+
64
+ You can host the web UI locally with:
65
+
66
+ ```bash
67
+ uvx muscriptor serve
68
+ ```
69
+
70
+ This gives you the same UI as hosted on https://muscriptor.kyutai.org/, just with a different look.
71
+
72
+ The sheet music download needs **MuseScore 4 or newer** installed separately (see
73
+ [Sheet music](#sheet-music) below). Without it, everything except that download still works.
74
+
75
+ ## Command-line interface (CLI)
76
+
77
+ ```bash
78
+ uvx muscriptor transcribe path/to/audio_file.wav
79
+ ```
80
+
81
+ See `--help` for all the options.
82
+
83
+ ### Tempo and quantization
84
+
85
+ The beats, time signature and bar lines are detected from the audio and written into the MIDI.
86
+ When the tempo isn't steady (no metronome), the MIDI gets a tempo map that follows the
87
+ performance, so the bar lines stay on the music; `--fixed-tempo` writes one average tempo instead.
88
+ When the detection is off, you can correct it, and tidy the notes up for notation software or a
89
+ DAW grid:
90
+
91
+ ```bash
92
+ muscriptor transcribe audio.wav --bpm 96 --beats-per-bar 3 --first-downbeat 1.2 \
93
+ --quantize --grid 4 --fill-gaps 0.25 --trim-leading-bars
94
+ ```
95
+
96
+ - `--quantize` snaps notes onto the beat grid (`--grid 4` = sixteenths, `3` = eighth triplets;
97
+ by default the one the notes sit on best), `--quantize-strength 0.5` only halfway.
98
+ - `--fill-gaps` holds notes until the next one across rests of up to that many beats, which
99
+ notation software such as Guitar Pro would otherwise write out.
100
+ - `--trim-leading-bars` drops the empty bars before the first note.
101
+ - `--beat-scale 2` (or `0.5`) when the detected tempo is half (or double) what it should be.
102
+
103
+ The web UI has the same settings: the "Tempo & bars" panel and the MIDI download dialog.
104
+
105
+ ### Sheet music
106
+
107
+ Using the CLI with `--format sheets` engraves the transcription as readable notation instead of
108
+ writing a single MIDI file.
109
+
110
+ ```bash
111
+ muscriptor transcribe audio.wav --format sheets --output score/
112
+ ```
113
+
114
+ The output structure looks like this:
115
+
116
+ ```
117
+ score/
118
+ ├── score.mid the transcription, as quantized MIDI
119
+ ├── score.musicxml the engraved score, as MusicXML
120
+ ├── full_score.pdf every instrument on one system
121
+ ├── 01_electric_guitar.pdf one PDF per instrument …
122
+ ├── 01_electric_guitar_tab.pdf … and a tablature PDF for fretted ones
123
+ ├── 02_electric_bass.pdf
124
+ ├── 02_electric_bass_tab.pdf
125
+ └── 03_drum_kit.pdf
126
+ ```
127
+
128
+ This needs **MuseScore 4 or newer** installed separately. Downloads for every
129
+ platform are at [musescore.org/en/download](https://musescore.org/en/download).
130
+ Set `$MUSCRIPTOR_MUSESCORE` if it lives somewhere unusual.
131
+
132
+ It works best if there is a clear beat, because that allows us to quantize the notes (snap
133
+ them to a grid) for a cleaner transcription. Without a steady tempo the score follows the
134
+ performance's tempo changes; rubato recordings will still work noticeably worse.
135
+
136
+ ## Using from Python
137
+
138
+ MuScriptor is also on PyPI, so you can install it with with uv (recommended) or with pip:
139
+
140
+ ```bash
141
+ uv add muscriptor
142
+ ```
143
+
144
+ ```bash
145
+ pip install muscriptor
146
+ ```
147
+
148
+ Ask your coding agent to show you around the codebase.
149
+
150
+ ## Models
151
+
152
+ Three variants are published under the [MuScriptor](https://huggingface.co/MuScriptor)
153
+ HuggingFace organization. Everywhere a model is selected (`load_model()`, the
154
+ CLI's `--model`, `serve --model`) you can pass the bare size keyword and the
155
+ weights are downloaded and cached automatically. The architecture is a transformer decoder only. Here are the detailed model sizes:
156
+
157
+ | Variant | Parameters | Layers | Dim | HuggingFace repo |
158
+ |---|---|---|---|---|
159
+ | `small` | 103M | 14 | 768 | [muscriptor-small](https://huggingface.co/MuScriptor/muscriptor-small) |
160
+ | `medium` (default) | 307M | 24 | 1024 | [muscriptor-medium](https://huggingface.co/MuScriptor/muscriptor-medium) |
161
+ | `large` | 1.4B | 48 | 1536 | [muscriptor-large](https://huggingface.co/MuScriptor/muscriptor-large) |
162
+
163
+ `small` is the practical choice on CPU-only machines, `medium` is the default
164
+ speed/accuracy trade-off, and `large` is the most accurate but really wants a
165
+ GPU. On Apple Silicon the model runs on Metal (MPS) automatically.
166
+
167
+ ## Developing
168
+
169
+ To set up for development, get [uv](https://docs.astral.sh/uv/getting-started/installation/),
170
+ clone this repo and run:
171
+ ```bash
172
+ uv sync
173
+ ```
174
+
175
+ For the web UI, you also need [pnpm](https://pnpm.io/installation) and Node
176
+ (can be installed [via pnpm](https://pnpm.io/cli/runtime)).
177
+ Then run:
178
+
179
+ ```bash
180
+ cd web
181
+ pnpm install
182
+ pnpm run build
183
+ ```
184
+
185
+ If you're not editing the frontend, you only need to do this once.
186
+ If you are, run `pnpm dev` instead for a hot-reloading dev server.
187
+ Start the backend alongside it with it using `uv run muscriptor serve --port 8222`
188
+ and then open the frontend on http://localhost:5173/.
189
+
190
+ ### Run
191
+
192
+ After this setup, you can run Muscriptor from your local repository using
193
+ `uv` (note - not `uvx` like before):
194
+
195
+ ```bash
196
+ uv run muscriptor serve
197
+ # or
198
+ uv run muscriptor transcribe path/to/audio_file.wav
199
+ ```
200
+
201
+ Again, see `--help` for more options.
202
+
203
+ ## License
204
+
205
+ The code in this repository is released under the [MIT license](LICENSE).
206
+
207
+ The model weights, published on
208
+ [HuggingFace](https://huggingface.co/MuScriptor), are released under the
209
+ [CC BY-NC 4.0 license](https://creativecommons.org/licenses/by-nc/4.0/)
210
+ (non-commercial use).
211
+
212
+ The MuseScore General SoundFont downloaded for playback is
213
+ distributed under its own (MIT) license.
214
+
215
+ ## Citation
216
+
217
+ ```bibtex
218
+ @misc{rouard2026muscriptoropenmodelmultiinstrument,
219
+ title={MuScriptor: An Open Model for Multi-Instrument Music Transcription},
220
+ author={Simon Rouard and Michael Krause and Axel Roebel and Carl-Johann Simon-Gabriel and Alexandre Défossez},
221
+ year={2026},
222
+ eprint={2607.08168},
223
+ archivePrefix={arXiv},
224
+ primaryClass={cs.SD},
225
+ url={https://arxiv.org/abs/2607.08168},
226
+ }
227
+ ```