transcpybe 2.0.0__tar.gz

transcpybe-2.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 David Whyatt
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

transcpybe-2.0.0/PKG-INFO

@@ -0,0 +1,356 @@
+Metadata-Version: 2.4
+Name: transcpybe
+Version: 2.0.0
+Summary: Transcribe sung melody to MIDI: onsets and pitches from monophonic vocal audio
+Author: David Whyatt
+License-Expression: MIT AND CC-BY-NC-SA-4.0
+Project-URL: Homepage, https://github.com/dmwhyatt/transcpybe
+Project-URL: Issues, https://github.com/dmwhyatt/transcpybe/issues
+Keywords: music information retrieval,transcription,midi,singing,melody,pitch tracking
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Multimedia :: Sound/Audio :: Analysis
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+License-File: src/transcpybe/landmarks/weights/LICENSE
+Requires-Dist: numpy>=1.24
+Requires-Dist: scipy>=1.10
+Requires-Dist: librosa>=0.10
+Requires-Dist: soundfile>=0.12
+Requires-Dist: pretty_midi>=0.2.10
+Requires-Dist: mir_eval>=0.7
+Requires-Dist: torch>=2.0
+Provides-Extra: crepe
+Requires-Dist: torchcrepe>=0.0.20; extra == "crepe"
+Provides-Extra: pesto
+Requires-Dist: pesto-pitch>=2.0; extra == "pesto"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: build>=1.0; extra == "dev"
+Requires-Dist: twine>=5.0; extra == "dev"
+Dynamic: license-file
+
+# transcPybe
+
+transcPybe transcribes monophonic singing into notes: onset and offset times in seconds, and pitches as MIDI note
+numbers, with the measured pitch retained as a pitch bend. It is intended for solo vocal recordings and for lead vocals
+separated from mixed recordings.
+
+Version 2 places note onsets with three small neural networks, which detect where each syllable begins to sound and
+where its vowel begins. Notes are then decoded from the pitch track by a hidden Markov model. Version 1, which relies on
+the pitch track and a spectral-flux onset detector alone, remains available.
+
+The code is released under the MIT licence. The weights of the version 2 networks are released under CC BY-NC-SA 4.0
+(see [Licence](#licence)).
+
+## Comparison to other models
+
+| System | vocadito A1 | vocadito A2 | Molina | RWC-P dev | SSVD test | MIR-ST500 test | A cappella | Pop | Five-set mean [95% CI] | GPU s/min | CPU s/min |
+|---|---|---|---|---|---|---|---|---|---|---|---|
+| Phoneme-informed\* | 0.450 | 0.593 | 0.712 | 0.317 | 0.688 | 0.372 | 0.641 | 0.345 | **0.522** [0.507, 0.537] | 13.4 | 12.4 |
+| transcPybe v2, pYIN | 0.481 | 0.580 | 0.675 | 0.339 | 0.609 | 0.391 | 0.605 | 0.365 | **0.509** [0.495, 0.523] | 10.6 | 10.9 |
+| transcPybe v2, PESTO | 0.467 | 0.574 | 0.693 | 0.348 | 0.601 | 0.348 | 0.605 | 0.348 | **0.502** [0.488, 0.516] | 2.1 | 4.4 |
+| transcPybe v2, CREPE | 0.473 | 0.573 | 0.670 | 0.347 | 0.568 | 0.354 | 0.587 | 0.350 | **0.492** [0.479, 0.505] | 3.0 | 65.5 |
+| CTC-CE\* | 0.312 | 0.433 | 0.629 | 0.359 | 0.504 | 0.538 | 0.502 | 0.449 | **0.481** [0.465, 0.496] | 6.8 | 3.6 |
+| MIR-ST500 model\* | 0.320 | 0.443 | 0.534 | 0.371 | 0.453 | 0.474 | 0.456 | 0.422 | **0.443** [0.427, 0.458] | 5.9 | 3.3 |
+| Tony † | 0.686 | 0.654 | 0.466 | 0.266 | 0.582 | 0.276 | 0.524 | 0.271 | **0.397**† [0.383, 0.412] | – | 1.1 |
+| YourMT3+\* | 0.275 | 0.350 | 0.464 | 0.328 | 0.410 | 0.426 | 0.395 | 0.377 | **0.388** [0.373, 0.403] | 3.0 | 21.8 |
+| GAME | 0.264 | 0.364 | 0.506 | 0.348 | 0.332 | 0.406 | 0.384 | 0.377 | **0.381** [0.366, 0.396] | – | 83.6 |
+| Omnizart | 0.440 | 0.494 | 0.512 | 0.180 | 0.533 | 0.178 | 0.504 | 0.179 | **0.374** [0.361, 0.387] | – | 134.6 |
+| transcPybe v1 | 0.496 | 0.480 | 0.419 | 0.231 | 0.449 | 0.237 | 0.452 | 0.234 | **0.365** [0.353, 0.376] | 3.0 | 60.7 |
+| SOME | 0.278 | 0.349 | 0.441 | 0.242 | 0.272 | 0.311 | 0.342 | 0.276 | **0.316** [0.304, 0.327] | 3.9 | 5.7 |
+| Kum et al. | 0.219 | 0.247 | 0.251 | 0.224 | 0.229 | 0.248 | 0.238 | 0.236 | **0.237** [0.227, 0.248] | – | 6.8 |
+| Basic Pitch | 0.307 | 0.325 | 0.345 | 0.133 | 0.254 | 0.137 | 0.305 | 0.135 | **0.237** [0.224, 0.250] | – | 4.6 |
+| Essentia | 0.318 | 0.263 | 0.289 | 0.120 | 0.360 | 0.117 | 0.313 | 0.118 | **0.235** [0.224, 0.247] | – | 1.8 |
+| CREPE Notes | 0.310 | 0.283 | 0.275 | 0.167 | 0.256 | 0.158 | 0.276 | 0.162 | **0.230** [0.219, 0.242] | – | 29.9 |
+| ROSVOT | 0.184 | 0.204 | 0.189 | 0.154 | 0.082 | 0.109 | 0.155 | 0.132 | **0.146** [0.137, 0.155] | 1.6 | – |
+| aubio | 0.061 | 0.060 | 0.086 | 0.019 | 0.106 | 0.026 | 0.084 | 0.022 | **0.059** [0.055, 0.064] | – | 0.4 |
+| STARS ‡ | 0.064 | 0.076 | 0.068 | 0.038 | 0.084 | 0.088 | 0.074 | 0.063 | **0.070** [0.062, 0.078] | – | – |
+
+The table compares transcPybe with fifteen published singing transcription systems on five annotated datasets.
+
+- **Runs.** Each system ran in its own pinned container on identical audio.
+- **Metric.** COnPOff@50 at whole semitones: a note counts when its onset is within 50 ms, its pitch within 50 cents,
+  and its offset within 20% of the note's duration or 50 ms, whichever is larger.
+- **Intervals.** The five-set mean has a 95% interval from a paired bootstrap over recordings, stratified by dataset.
+- **Time.** Seconds of computation per minute of audio, on an NVIDIA RTX 3080 GPU and on eight CPU cores.
+
+The methods, every system's reference and the per-recording scores are in the
+[transcription-benchmarking](https://github.com/dmwhyatt/transcription-benchmarking#results) repository.
+
+Notes to the table:
+
+- \* Trained on other recordings from one of the evaluation datasets.
+- † vocadito's reference annotations were produced by correcting Tony's output, so Tony's mean excludes vocadito.
+- ‡ STARS requires lyrics. They were obtained by automatic speech recognition.
+- transcPybe v2's settings were selected on vocadito, Molina and the RWC-P development set; SSVD test and MIR-ST500
+  test were held out.
+
+For reference, vocadito's two annotators agree with each other at 0.64 COnPOff@50.
+
+## Installation
+
+```shell
+pip install transcpybe              # version 2 with the pYIN pitch tracker
+pip install "transcpybe[crepe]"     # adds the CREPE pitch tracker
+pip install "transcpybe[pesto]"     # adds the PESTO pitch tracker (LGPL-3.0)
+```
+
+transcPybe requires Python 3.9 or later. PyTorch is a dependency, because the version 2 networks require it. On a
+machine without a GPU, installing PyTorch's CPU build first avoids downloading the CUDA libraries:
+`pip install torch --index-url https://download.pytorch.org/whl/cpu`. To install from a source checkout, use
+`pip install -e ".[dev]"`.
+
+## Usage
+
+### Command line
+
+```shell
+# A lead vocal separated from a mix: writes lead-vocal.mid, .notes.json and .notes.csv
+transcpybe lead-vocal.wav -o out/
+
+# Solo singing, with nothing else in the recording
+transcpybe solo.wav --preset acappella
+
+# Version 1, without the networks
+transcpybe lead-vocal.wav --preset rules
+
+# Every audio file in a directory, with a log file
+transcpybe stems/ -o midi/ --log-file transcribe.log
+
+# Print the notes as a table
+transcpybe lead-vocal.wav --print-notes
+
+# Write whole equal-tempered semitones only, without pitch bends
+transcpybe lead-vocal.wav --12tet
+
+# Run the built-in synthetic test suite
+transcpybe --self-test
+```
+
+`transcpybe --help` lists every option.
+
+### Presets
+
+| Preset | Version | Intended input | Evaluated on |
+|---|---|---|---|
+| `pop` (default) | 2 | a lead vocal separated from a pop mix | 82 MIR-ST500 test songs |
+| `acappella` | 2 | solo singing, with nothing else in the recording | 127 SSVD test recordings |
+| `rules` | 1 | either; no neural networks besides the pitch tracker | |
+
+The two version 2 presets were fixed on development sets and then evaluated once on recordings that had not been used
+for tuning. Each preset also sets the decoder parameters with which it was evaluated: `--min-note`, `--p-self`,
+`--note-level-floor`, `--onset-delta`, `--onset-split-strength` and `--release-drop`. The command line warns if one of
+these is given with a version 2 preset. With `--preset rules`, every parameter takes the value given.
+
+### Pitch trackers
+
+| Tracker | Option | Requires | Notes |
+|---|---|---|---|
+| pYIN | `--f0-backend pyin` (default) | | Version 2's best five-set mean; runs on the CPU |
+| CREPE | `--f0-backend crepe` | `transcpybe[crepe]` | Fastest on a GPU, slowest on a CPU. `--crepe-model tiny` is about 14 times faster on a CPU; version 2 has not been evaluated with it |
+| PESTO | `--f0-backend pesto` | `transcpybe[pesto]` | A much smaller network than CREPE |
+
+The version 2 networks were trained on CREPE's pitch tracks. pYIN and PESTO report lower confidence than CREPE. Their
+tracks therefore pass through a voicing map before decoding: a quantile map onto CREPE's confidence scale, fitted on
+the training data. Version 1's thresholds were set on CREPE. Further trackers can be added with
+`transcpybe.f0.register_f0_backend`.
+
+### Python
+
+```python
+from transcpybe import TranscriptionConfig, transcribe_file, write_midi
+
+result = transcribe_file("solo.wav", TranscriptionConfig(preset="acappella"))
+for note in result.notes:
+    print(note.onset, note.offset, note.pitch)          # seconds, seconds, MIDI note number
+
+write_midi(result.notes, "solo.mid", result.config)
+```
+
+`TranscriptionConfig` documents every parameter. The result also contains the pitch track (`f0_track`) and the onset
+track (`onsets`) from which the notes were decoded.
+
+### Output
+
+- **`.mid`:** one track. Note times are absolute seconds; the tempo in the header affects only the bar grid displayed
+  by notation software. Each note's MIDI number is the semitone nearest the sung pitch. By default, the measured pitch
+  is also written as a pitch bend at the start of the note, using General MIDI's two-semitone range. `--12tet` omits
+  the bends.
+- **`.notes.csv`:** onset, offset, duration, pitch, velocity, confidence, pitch in Hz, and the measured pitch as a
+  fractional MIDI number (`pitch_midi_float`).
+- **`.notes.json`:** the same notes, with the complete configuration that produced them.
+
+`transcpybe.midi_io.read_midi_notes` reads a MIDI file back with its pitch bends applied.
+
+## Method
+
+### Version 2: note onsets from neural networks
+
+Three networks with the same architecture were trained with different random seeds:
+
+- **Architecture:** a convolutional front end over an 80-band log-mel spectrogram, then a two-layer bidirectional GRU,
+  with 0.8 million parameters each.
+- **Inputs:** the spectrogram and five features from the pitch track.
+- **Outputs:** every 10 ms, the probability that a syllable begins to sound and the probability that its vowel
+  begins.
+- **Training data:** singing from M4Singer and GTSinger, mixed with accompaniment from MUSDB18-HQ.
+
+The three networks' outputs are averaged. Each vowel onset is paired with the nearest preceding syllable onset, and the
+note onset is placed between them: at 0.5 of the interval in the `pop` preset, and at 0.875 in the `acappella` preset.
+The decoder described below then forms notes from the pitch track, with these onsets.
+
+The networks run on the CPU. On an Apple M-series laptop with PyTorch's default four threads, the three together take
+about 3.6 s per minute of audio. The code is in `transcpybe.landmarks`. The weights, the voicing maps and their
+provenance are in `src/transcpybe/landmarks/weights/`.
+
+### The note decoder
+
+Both versions share the following stages. Version 1 takes its onsets from stage 5. Version 2 takes them from the
+networks and uses its preset's parameters throughout.
+
+1. **Pitch and voicing.** The pitch tracker estimates a fundamental frequency and a periodicity confidence every
+   10 ms. The confidence is multiplied by a loudness gate set relative to the recording's 95th-percentile level, so
+   that periodic residue left by source separation in quiet passages is not transcribed.
+2. **Note decoding.** A Viterbi decoder runs over pitch states spaced a third of a semitone apart, plus an unvoiced
+   state. Emissions favour states near the measured pitch, weighted by voicing. Transitions favour remaining on a
+   note, and penalise larger intervals. The design follows the note tracker of Tony (Mauch et al., 2015). The
+   sub-semitone states prevent a note sung between two semitones from alternating between them.
+3. **Fragments.** Very short segments adjacent to a much longer note, within two semitones of it, are merged into
+   it. Such segments typically arise from wide vibrato at the end of a note.
+4. **Glides.** Where the singer slides into a note, the transition is attributed to the note being approached. This
+   corrects the late onset the slide would otherwise produce, and removes the brief notes it would leave on each
+   semitone crossed.
+5. **Onsets.** A spectral-flux onset envelope with a frequency-domain maximum filter (SuperFlux) splits repeated
+   notes at the same pitch, and aligns note onsets with audible attacks. The maximum filter prevents vibrato from
+   registering as onsets.
+6. **Pitch and clean-up.** Each note's pitch is the confidence-weighted median of its pitch track, excluding the
+   start of the note, rounded to the nearest semitone. Notes shorter than the minimum duration, and notes far quieter
+   than the rest of the performance, are removed.
+7. **Note ends.** A note ends where its level has fallen a set amount below its own sustained level: 15 dB by default,
+   20 dB in the `pop` preset. This excludes the release and room reverberation, which pitch trackers continue to
+   report as periodic.
+8. **Tuning reference.** If the notes of a first pass show that the performance is consistently offset from the
+   A440 grid, decoding is repeated against the grid on which it was sung. Offsets beyond 40 cents are not applied,
+   because scattered intonation is otherwise better explained by a grid a quarter tone away.
+   `--no-tuning-estimate` disables the estimate.
+
+## Processing large collections
+
+Given a directory, `transcpybe` transcribes every audio file within it:
+
+- **Existing outputs:** files whose outputs already exist are skipped, so an interrupted run resumes; `--overwrite`
+  transcribes them again.
+- **Failures:** a file that cannot be read is logged and the run continues; `--stop-on-error` stops it instead.
+- **Remaining time:** estimated from audio duration rather than file count.
+
+Progress is shown on a single line in a terminal, and as periodic timestamped lines when output is redirected. The
+following options control logging and the GPU:
+
+| Option | Purpose |
+|---|---|
+| `--log-file PATH` | Append timestamped logs to a file as well as the console. |
+| `--progress-interval N` | Seconds between progress lines when output is redirected (default 5). |
+| `--no-progress` | Disable progress reporting. |
+| `--device` | Torch device for CREPE and PESTO: `auto` (default; CUDA, then Apple MPS, then CPU), or an explicit device such as `cuda:1`. The device used is logged and recorded in the JSON output. |
+| `--crepe-batch-size N` | CREPE frames per forward pass (default 512); lower it if GPU memory is limited. |
+| `--gpu-memory-every N` | Log GPU memory every N CREPE batches. |
+| `-v` | Include debugging detail. |
+
+## Reproducibility
+
+The JSON output records the complete configuration used. CREPE adds random dither of ±20 cents to its pitch estimates.
+transcPybe seeds it (`--seed`, default 0) and restores the global random state afterwards, so repeated runs give
+identical output. `--no-seed` restores CREPE's unseeded behaviour.
+
+## Adjusting version 1
+
+With `--preset rules`, the decoder's parameters may be adjusted. The most commonly used are:
+
+| Symptom | Option | Direction |
+|---|---|---|
+| Short spurious notes | `--min-note` | increase (e.g. 0.08) |
+| Sustained notes split into several | `--p-self` | increase (e.g. 0.995) |
+| Fast passages merged into single notes | `--p-self` | decrease (e.g. 0.97) |
+| Vibrato fragmenting notes | `--pitch-sigma` | increase (e.g. 0.7) |
+| Notes in silent passages | `--note-level-floor` | decrease (e.g. 22) |
+| Quiet sung notes discarded | `--note-level-floor` | increase |
+| Repeated notes at one pitch merged | `--onset-split-strength` | decrease |
+| Noisy recordings over-segmented | `--onset-delta` | increase |
+| Note ends too late | `--release-drop` | decrease (e.g. 10) |
+| Note ends too early | `--release-drop` | increase, or `--no-release-trim` |
+| Voice outside the tracked range | `--fmin`, `--fmax` | widen |
+
+## Tests
+
+```shell
+pytest tests/                 # the test suite
+pytest tests/ -m slow         # the tests that run CREPE
+transcpybe --self-test        # the synthetic suite
+```
+
+## Limitations
+
+- **Monophonic input.** Harmonies or overlapping voices in the same recording are transcribed as a single line.
+- **Note ends follow a convention.** Annotators differ on where a sung note ends. The release threshold
+  (`--release-drop`) may need adjusting for a particular corpus.
+- **Separation residue.** Loud residue from source separation can be transcribed as notes. With `--preset rules`,
+  lowering `--note-level-floor` reduces this, at the cost of quiet singing.
+- **Tuning.** A performance sung more than 40 cents from A440 is decoded on the A440 grid.
+- **No quantisation.** Output times are in seconds, without beat tracking or rhythmic quantisation.
+
+## Acknowledgements
+
+**Pitch estimation**
+
+- **CREPE:** J. W. Kim, J. Salamon, P. Li and J. P. Bello, "CREPE: A Convolutional Representation for Pitch
+  Estimation", ICASSP 2018. Used through **torchcrepe** (M. Morrison; MIT licence), which supplies the weights.
+- **pYIN:** M. Mauch and S. Dixon, "pYIN: A fundamental frequency estimator using probabilistic threshold
+  distributions", ICASSP 2014. Used through librosa's implementation.
+- **PESTO:** A. Riou, S. Lattner, G. Hadjeres and G. Peeters, "PESTO: Pitch Estimation with Self-supervised
+  Transposition-equivariant Objective", ISMIR 2023. Used through the pesto-pitch package (LGPL-3.0).
+
+**Note segmentation and onsets**
+
+- **Tony:** M. Mauch, C. Cannam, R. Bittner, G. Fazekas, J. Salamon, J. Dai, J. Bello and S. Dixon, "Computer-aided
+  melody note transcription using the Tony software: Accuracy and efficiency", TENOR 2015. The pitch-state hidden
+  Markov model is a simplified form of Tony's note tracker.
+- **SuperFlux:** S. Böck and G. Widmer, "Maximum filter vibrato suppression for onset detection", DAFx 2013. Used
+  through librosa's `onset_strength`.
+
+**Training data for the version 2 networks**
+
+- **M4Singer:** L. Zhang et al., "M4Singer: A Multi-Style, Multi-Singer and Musical Score Provided Mandarin Singing
+  Corpus", NeurIPS 2022 Datasets and Benchmarks. CC BY-NC-SA 4.0.
+- **GTSinger:** Y. Zhang et al., "GTSinger: A Global Multi-Technique Singing Corpus with Realistic Music Scores for
+  All Singing Tasks", NeurIPS 2024 Datasets and Benchmarks. CC BY-NC-SA 4.0.
+- **MUSDB18-HQ:** Z. Rafii, A. Liutkus, F.-R. Stöter, S. I. Mimilakis and R. Bittner, 2019. Accompaniment only;
+  educational and non-commercial use.
+
+**Libraries**
+
+- librosa (ISC licence)
+- pretty_midi (MIT)
+- mir_eval (MIT)
+- NumPy (BSD-3-Clause)
+- SciPy (BSD-3-Clause)
+- soundfile (BSD-3-Clause)
+- PyTorch (BSD-3-Clause)
+
+## Licence
+
+The code is released under the MIT licence (`LICENSE`).
+
+The weights of the version 2 networks and the voicing maps, in `src/transcpybe/landmarks/weights/`, are released under
+[CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/), because the singing on which they were trained
+is licensed on those terms:
+
+- **Attribution:** use requires attribution.
+- **Non-commercial:** commercial use is not permitted.
+- **Share-alike:** derived weights must be shared under the same licence.
+
+That folder's `LICENSE` and `README.md` give the full terms and the provenance. The `pop` and `acappella` presets use
+these weights; `--preset rules` does not. The pitch trackers' weights are distributed with their own packages, under
+their own licences.