@audio/denoise 0.9.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +97 -54
- package/package.json +8 -8
package/README.md
CHANGED
|
@@ -8,13 +8,13 @@ Single-pass noise reduction. 17 specialised methods + an auto-classifier.
|
|
|
8
8
|
|---|---|---|---|---|---|
|
|
9
9
|
| [denoise](#denoise) | meta | auto | — | varies | "just clean it" |
|
|
10
10
|
| [gate](#gate) | time | silence | ★ | very low | hard cut at threshold |
|
|
11
|
-
| [dehum](#dehum) | time | mains hum | ★★★★ | low | 50/60 Hz + harmonics |
|
|
11
|
+
| [dehum](#dehum) | time | mains hum, steady tones | ★★★★ | low | 50/60 Hz + harmonics; a whine, a test tone |
|
|
12
12
|
| [specsub](#specsub) | freq | broadband stationary | ★★ | medium | baseline |
|
|
13
13
|
| [wiener](#wiener) | freq | broadband stationary | ★★★ | medium | general broadband |
|
|
14
14
|
| [omlsa](#omlsa) | freq | broadband non-stationary | ★★★★ | high | speech in changing noise |
|
|
15
15
|
| [declick](#declick) | time | impulses | ★★★★ | medium | vinyl ticks, edit clicks, mouth clicks |
|
|
16
16
|
| [decrackle](#decrackle) | time | dense impulses | ★★★ | medium | shellac crackle |
|
|
17
|
-
| [declip](#declip) | time + freq | hard clipping | ★★★★ | high | clipped speech and music, rails found by itself, also through MP3/AAC |
|
|
17
|
+
| [declip](#declip) | time + freq | hard clipping, soft saturation | ★★★★ | high | clipped speech and music, rails found by itself, also through MP3/AAC; saturation's curve fitted blind |
|
|
18
18
|
| [dewind](#dewind) | freq | wind, LF rumble | ★★★ | medium | wind under a voice, its harmonics kept |
|
|
19
19
|
| [deplosive](#deplosive) | time | LF bursts | ★★★★ | low | mic plosives (p, b) |
|
|
20
20
|
| [deesser](#deesser) | time | sibilance | ★★★★ | low | voice (s, sh) |
|
|
@@ -76,7 +76,10 @@ classify(data, fs) // → { method,
|
|
|
76
76
|
3. **sibilance → `deesser`**: 5–9 kHz over 0.2–2 kHz power over 8.
|
|
77
77
|
4. **noise bed → `omlsa`**: a floor within 25 dB of the program (`BED_SNR`; the program: frames within 40 dB of the loudest), shown in its *pauses* or in *steady bands*. The pauses are runs of 0.15 s or more within 6 dB of the frame level's 10th percentile that hold noise or hold the floor; the bed is their median level. *Noise*: flat and without lines in the bands from 300 Hz. *The floor*: a bed lies under the program all the time, so in each band the pauses sit where the band sinks to over the whole take (its 5th percentile), no further over it than Gaussian noise's own spread puts them (1.645 · 4.34/√(n/1.5) dB over n bins) plus 3 dB for the bed's slow wander (the bands' median), and they hold no partials that come and go (median persistence under 0.1; babble reads 0.04–0.08). A line held through every pause and across the notes, an engine's, a fan's, a mains-like tone, is part of the floor; a quiet passage of music isn't: its notes come and go, and its pauses stand over the floors other passages sink to. *Steady bands*: a band whose quieter half varies no more than twice what Gaussian noise over its bins would, without lines. A line is a band's fine structure (log power less its ±4-bin mean) correlated with the frame N back, which shares no sample: a partial holds its bins, noise draws them anew.
|
|
78
78
|
5. **wind → `dewind`**: in over a tenth of the 0.15 s blocks, the low end (< 200 Hz) within 20 dB of the program, 6 dB over the 300–2000 Hz band, and without lines, where no bed shows. Wind is turbulence at the microphone, without a period (Nelke & Vary, IWAENC 2014); a bass line or a voice's low end repeats at its pitch.
|
|
79
|
-
6.
|
|
79
|
+
6. **steady tones → `dehum`**: dehum's `tones()` finds lines held through the program's pauses and under it, their phase turning as one line's (a whine, a test tone; see [dehum](#dehum)), A-weighted within 50 dB of the program, as hum is.
|
|
80
|
+
7. otherwise **`none`**.
|
|
81
|
+
|
|
82
|
+
Where `tones()` finds steady tones and another route is taken (clicks, sibilance, a bed, wind), `dehum` runs first: a gain floored per bin takes a line standing over the bed down by its floor at most (speech, a 7 kHz line at 0.3 and white noise at 0.01: `wiener` alone 10.5 dB SNR, `dehum` then `wiener` 24.2). `force` runs the forced method alone.
|
|
80
83
|
|
|
81
84
|
Every bed goes to `omlsa`, and a bed outranks wind, by what each reducer made of the noisy speech the router sends them (`node scripts/detect.js tune reduce`, `python scripts/detect.py score tune`; PESQ wideband, STOI, DNSMOS SIG at 16 kHz as `scripts/speech.py` scores):
|
|
82
85
|
|
|
@@ -155,13 +158,19 @@ Follows the mains phase, then subtracts the hum along it: each harmonic of the m
|
|
|
155
158
|
dehum(data, { fs }) // 50 or 60 Hz, found; nothing without hum
|
|
156
159
|
dehum(data, { fs, freq: 60 }) // the 60 Hz series, its exact frequency tracked
|
|
157
160
|
dehum(data, { fs, freq: 60, harmonics: 4 }) // remove 60–240 Hz as told
|
|
161
|
+
dehum(data, { fs, steady: true }) // a buzz held through the take, an amp's under a guitar
|
|
162
|
+
dehum(data, { fs, tones: [15734] }) // this tone, as told (true, the default: those found)
|
|
163
|
+
dehum(data, { fs, tones: false }) // the mains series alone, as 0.6
|
|
158
164
|
measure(data, fs) // → { f0, harmonics } or null (import from @audio/denoise-dehum)
|
|
165
|
+
tones(data, fs) // → [{ freq, level, … }] the steady tones found (import alike)
|
|
159
166
|
```
|
|
160
167
|
|
|
161
168
|
| Param | Default | |
|
|
162
169
|
|---|---|---|
|
|
163
170
|
| `freq` | found | Fundamental, Hz. Omitted: the 50 or 60 Hz series, whichever is found. Given: that series, its frequency tracked from there (within ±0.4 %, ±`drift` Hz with `adaptive`) |
|
|
164
171
|
| `harmonics` | to 8 kHz | Remove h = 1…`harmonics` as told. Omitted: every harmonic to 1 kHz and each line above it that stands out, to 8 kHz |
|
|
172
|
+
| `steady` | `false` | The hum held through the take: a buzz under an instrument, no edit in it. Each line one phasor through the take, the mains phase refined against them (below) |
|
|
173
|
+
| `tones` | `true` | Steady tones off the mains series, held through the take: those `tones()` finds (below); an array: those frequencies, Hz, as told; `false`: none |
|
|
165
174
|
|
|
166
175
|
Hum is found by `measure()`, one Fourier transform over the signal (its first 80–90 s, taken down to 3 kHz), where a hum line, steady for minutes, gathers into a peak 1/T wide while speech spreads: the fundamental 20 dB over the median of the ±8 Hz around it, or two of the first six harmonics 15 dB, each 6 dB over any other peak within 3 Hz, all harmonics of one fundamental. Under music a line 20 dB under the program is too faint for that, so the band to 1 kHz is also tracked along each series (below) and its lines weighed where the program is quiet around them: hum is there when two or more of the first 20 harmonics stand out by 18 dB, are no more than 60 dB under the signal and stand alone, 6 dB over any other peak within 3 Hz, a third at least of those that stand out (a bar of music repeated exactly is a comb of lines 1/bar apart, lines at 50 and 60 Hz among them, few alone). A faster pattern spaces its teeth past 3 Hz, and where its instrument rings a tooth stands over the next: a kick at 200 bpm repeats every 15 periods of 50 Hz and 18 of 60, a comb every 3.33 Hz with teeth on both series, and in the MUSDB18 preview of Dark Ride's "Burning Bridges" its 50 Hz tooth stands 7 dB over the next one; the take has no pause to hear the line persist in, and 0.5.1 took it for hum (the mix out at 6.0 dB SDR). Neither the line's level nor its neighbours tell such a tooth from hum; the program's comb does, by all its teeth: for each pattern of 1/8 s to 2 s that spans a whole number of mains periods (its comb runs through the series), the share of the positions of its other lines, 40 Hz to 1 kHz, that hold a peak, less the share midway between them, where chance puts as many. The kick scores 0.30 (synthetic kicks 0.44–0.50, two BabySlakh mixes rendered from MIDI at an exact tempo 0.33 and 0.53); the 143 other MUSDB18 previews 0.16 at most, the other 18 BabySlakh mixes 0.20, 200 VocalSet takes, 120 GuitarSet takes, ten readings and five DEMAND noises 0.06. Over a quarter, in a take with no pause, the series is the program's and is left. Through 468 clean takes (MUSDB18's 94 training and 50 test previews, Brahms, Vibe Ace, the Nutcracker and the trumpet whole, 200 VocalSet takes, 120 GuitarSet takes, 30 s each: `node bench/rx/dehum.mjs clean` in `audio`) 0.5.2 changes none; 0.5.1 changed that one. Hum under such a pattern is left too where the take has no pause; with one, its lines and the pattern's within a fraction of a hertz of them go together (59.95 Hz hum 20 dB under the 200 bpm kick after 2 s of room tone: the mix out at 2.0 dB SDR, 0.5.1 alike). A tone held right on a mains line through a take with no pause is taken for hum.
|
|
167
176
|
|
|
@@ -169,6 +178,30 @@ The mains phase is tracked first from the harmonics to 1 kHz (each harmonic's ph
|
|
|
169
178
|
|
|
170
179
|
Where the hum itself jumps (an edit's splice turns its phase, a level step, hum switched on or off), every line's phasors before and after a frame differ beyond their spread at once, a note beside one line moving that line alone: the tracking does not integrate across the jump, each stretch is fitted on its own frames, and the hum switches from one stretch's to the next at the sample that best splits the signal between them. A note within ~0.5 Hz of a line for seconds is taken for hum. Whole clip (`streaming: false`), a second at least to find hum; shorter, it removes the harmonics of a given `freq` as told and passes the audio through without one. About a tenth of real time (0.4.0: a hundredth), most of it in the 2 s fits.
|
|
171
180
|
|
|
181
|
+
`steady` (0.6.0) is for a hum that holds still through the take: an amp's ground loop under a guitar. A fit over 2 s takes in what the program puts near a line for seconds, and a chord's attack, moving many lines at once, reads as the hum's jump: on GuitarSet takes under a buzz 35 dB down (audio's `bench/rx/guitar.mjs`, players 00–02), 0.5.2 took the buzz 4.3 dB down and the guitar to 39.3 dB SDR. Steady, each line is one phasor through the take, its mean weighted to where the program around the line is quiet; the program's power there is read from the line's magnitude, not its phase, so a phase still off does not pass for program. Held, the lines show the mains phase's error frame by frame against them, all at once (Newton's steps on the held comb's correlation, smoothed by Rauch–Tung–Striebel at the likeliest rate of wander): from the trace, the phase is refined against the comb to harmonics 2, 4, … 64, then to 8 kHz twice, each step holding the higher lines tighter (on one of those takes, from 0.5 rad off the true phase at its start to under 0.001 rad throughout). A line goes when its held phasor stands 10 dB over its own spread and holds no more than twice the power the line has where the program falls silent (a note on the line, which the take's quiet does not hold), each frame's by its expected coherence, e^(−h²σ²/2) with σ² the phase's variance there: where the program masked the hum and the phase was bridged, the upper lines go less. The dense mix of the test suite under that buzz, wandering ±0.02 Hz: 48.0 dB of it down, the music at 78.4 dB SDR (by default 3.7 and 32.6).
|
|
182
|
+
|
|
183
|
+
**Steady tones** (0.7.0). Off the mains series a line held through a take is interference too: a whine, a test tone, a monitor's 15.7 kHz, an electrical or fan line. A held note can be as exact in frequency and level (an organ, a synth), so a line is taken only for what a note does not do through a take whose program comes and goes. `tones()` reads one short-time transform (Hann, ~0.19 s, hop half; 2048 frames at most, spread over a longer take), takes as candidates the peaks of the power each bin holds in four frames of five, 12 dB over the median of the bins 4–40 away, and keeps a line that
|
|
184
|
+
- **holds through the pauses**: the frames where all but the lines sits within 6 dB of its 10th percentile, half a second of them at least; it stands 6 dB over its bins' floor in 95 % of them, its level there within 3 dB (90th over 10th percentile). A take whose program never moves 10 dB (90th over 10th percentile) is left: nothing passes over its lines to tell them from notes;
|
|
185
|
+
- **is never gone under the program**: its five bins hold a quarter of its power in the pauses in 90 % of all frames. In a mix the quietest frames can be where the bass plays alone: in the MUSDB18 preview of Grants' "PunchDrunk" a 220 Hz line held 92 % of the pauses, its level within 1.6 dB and its turns' mean resultant length 0.994, and was there in 32 % of the frames;
|
|
186
|
+
- **turns as one line**: between consecutive pauses holding it, the mean resultant length of its phasor's turns is 0.9 at least (their spread under ~0.8 Hz), and its phase breaks off the turn (by over π/3) in 2 % of the transitions at most, across up to a second of the program too. A note struck again on the same pitch starts at a new phase, a kick tuned to it too (van Waterschoot & Moonen's interframe peak persistence, Proc. IEEE 99(2), 2011, a feedback detector's mark, read on the phase rather than the magnitude);
|
|
187
|
+
- stands within 60 dB of the take; under a found mains series, off its harmonics (±0.5 %).
|
|
188
|
+
|
|
189
|
+
Each is subtracted as a harmonic is: its frequency read from its turn, its phasor per 80 ms Hann frame, weighted by the program's power around it, bridged through the program by Rauch–Tung–Striebel at the likeliest rate of drift (up to a turn of ~0.1 rad a frame: a whine wandering ±0.3 Hz), resynthesized and taken out. What goes of the program is what lies within a fraction of a hertz of the line. A note held through the whole take, struck before it starts and released after it ends, under a melody, is a tone by every mark the take offers and goes (the synthetic drone below); a tone switched on or off inside the take is left, as is one that glides.
|
|
190
|
+
|
|
191
|
+
Measured by `node scripts/tones.mjs` (`case`, `synth`, `music`, `speech`; thresholds chosen on its tune split: VoiceBank's training speakers, the odd-numbered VocalSet singers, every third MUSDB18 training preview, odd BabySlakh mixes, GuitarSet; then read once on the test split, below). Test.js's steady line, audio-lena's first 4 s with a 7 kHz sine at 0.3 and white noise at 0.01: speech SNR −3.8 dB as 0.6 leaves it (and `denoise()` routed it to `none`); 22.8 dB through `dehum` (with the line gone exactly, the noise alone leaves 22.8); 24.2 then through `wiener`, where a notch at 7 kHz, told, then `wiener` reaches 23.8. Tones under VoiceBank+DEMAND's test speakers (4 clips of 4 utterances, ~10–14 s), by phase inversion; tone down / speech SDR, dB, the mean over the clips where found, against a notch (Q 30) told each frequency:
|
|
192
|
+
|
|
193
|
+
| tone under speech | −10 dB | −30 dB | −50 dB | notch, told |
|
|
194
|
+
|---|---|---|---|---|
|
|
195
|
+
| 97 Hz | 92.2 / 60.4 (4 of 4) | not found | not found | 22.8 / 25.3 |
|
|
196
|
+
| 440 Hz | 96.1 / 60.5 (4) | 79.6 / 72.6 (3) | not found | 29.4 / 14.3 |
|
|
197
|
+
| 1 kHz | 85.0 / 64.5 (4) | 80.4 / 72.7 (4) | 45.8 / 82.1 (1) | 32.9 / 25.8 |
|
|
198
|
+
| 3.15 kHz | 67.2 / 69.9 (4) | 89.3 / 77.8 (4) | 57.3 / 86.9 (2) | 37.8 / 31.5 |
|
|
199
|
+
| 7 kHz | 60.1 / 72.0 (4) | 89.4 / 83.9 (4) | 86.7 / 96.5 (3) | 40.8 / 36.8 |
|
|
200
|
+
| 12 kHz | 57.8 / 79.1 (4) | 76.9 / 84.8 (4) | 76.5 / 92.9 (4) | 41.8 / 41.9 |
|
|
201
|
+
| 15.734 kHz | 57.5 / 82.7 (4) | 72.3 / 86.0 (4) | 81.0 / 92.6 (4) | 41.2 / 47.4 |
|
|
202
|
+
|
|
203
|
+
A whine at 1234.5 Hz with harmonics 2–4, 20 dB under: 84.9 / 66.8 (notch 34.6 / 21.6); a 3 kHz tone wandering ±0.3 Hz: 33.5 / 55.2 (36.7 / 31.3); a 2 kHz tone switched on halfway: left. Held notes, nothing added: 15 synthetic takes (organ chords with rests, held sines, a pedal held 85 % of the take under a melody, strings with vibrato, piano), 62 VocalSet long tones (straight, messa, pp, forte, trill, trillo, inhaled), 50 MUSDB18 test previews, 10 BabySlakh mixes, 7 GuitarSet takes and the four pieces: none changed by the tones (one VocalSet take, `f8_long_inhaled_a`, changed by the hum lines as 0.6 changes it, bit for bit); on the tune split, 142 takes, none. A drone held from the first sample to the last under a melody goes, 56 dB of it. VoiceBank+DEMAND test set: of 824 clean utterances none changed, of 824 noisy one (`p232_381`, the bus noise's 38.2 Hz engine line, PESQ 2.211 → 2.204); `denoise()` routes every utterance as before, PESQ 2.30 → 2.30 (that utterance 2.292 → 2.290). Tune split, of 504: one changed (`p228_229`, the metro noise's 2374.6 Hz whine; through `denoise()` 2.472 → 2.491).
|
|
204
|
+
|
|
172
205
|
Measured by `node scripts/dehum.js`, 0.4.0 → 0.5.1 (0.5.2 alike, every cell): the hum (mains: 12 harmonics at −6 dB per octave; buzz: odd-heavy to 8 kHz, odd h at h^−½, even at 0.3·h^−½, rolling off over 3 kHz; levels drifting ±10 %, f0 0.05 Hz off nominal and wandering) and the program told apart by phase inversion (Hagerman & Olofsson 2004). 50 Hz hum 20 dB under the program, wandering ±0.02 Hz; program SDR per band where the band holds 1 % of the program:
|
|
173
206
|
|
|
174
207
|
| material | hum down, dB | program SDR, dB | <100 Hz | 100–300 | 300–1k | 1–4k | >4k |
|
|
@@ -219,8 +252,8 @@ Against iZotope RX 12 De-hum (2026-10, `node bench/rx/dehum.mjs` in `audio`; RX
|
|
|
219
252
|
|
|
220
253
|
RX's learned Dynamic profile acts as a gate: it takes the hum in the room tone 54–60 dB down and leaves it under the speech, where it takes as much of the voice (18.9 dB mean SDR, the input's 22.6). Tuned, it barely touches the take (3 dB of hum), the best SDR it reaches. 0.5.2 (every score as 0.5.1's) is over RX tuned in SDR in all 18 condition × material cells (buzz 30 dB under music: 35.9 against 32.2, input 31.3), and every clip comes out within 0.2 dB of its input SDR or better. Hum as recorded (MIR-1K singing, DEMAND's washing machine and hallway, five readings, a VoiceBank take): RX's select-all Learn takes its lines up to 18 dB down and the program with them (the rest of the spectrum 1.5–20 dB SDR); dehum takes them 0–14 dB down and leaves the rest 21–60 dB. VocalSet's 60 Hz lines, more than 60 dB under the singing and barely over its room tone, are left as they are (0.4.0 took them 0–7 dB down; RX up to 23, the rest of the spectrum 0.2–12.5 dB).
|
|
221
254
|
|
|
222
|
-
**Use when:** mains buzz, ground-loop hum, fixed tonal interference.<br>
|
|
223
|
-
**Not for:** broadband noise (use `wiener`/`omlsa`);
|
|
255
|
+
**Use when:** mains buzz, ground-loop hum, fixed tonal interference: a whine, a test tone, an electrical or fan line.<br>
|
|
256
|
+
**Not for:** broadband noise (use `wiener`/`omlsa`); tones that glide or switch on and off inside the take (use spectral methods).
|
|
224
257
|
|
|
225
258
|
|
|
226
259
|
### `dewind`
|
|
@@ -514,7 +547,7 @@ The share of mouth clicks gone by 10 dB or more: 65, 83 and 92%, Mouth De-click'
|
|
|
514
547
|
|
|
515
548
|
### `decrackle`
|
|
516
549
|
|
|
517
|
-
Finds crackle, a worn record's dense small impulses, where the sound departs both from its AR prediction and from its least-squares interpolation from either side, and rebuilds all of it in a 46 ms window at once. The prediction error alone also stands out at the sound's own excitation, a voice's glottal pulses, a reed's or a bow's; the two-sided error (the prediction error through its matched filter, Vaseghi & Rayner 1990) holds those back by √Σa² against its spread and lifts an impulse added to the sound by as much, so crackle is where both stand over `threshold` × their local level: per 1.5 ms block the median |error|, the median of those over ±12 ms, which crackle can't raise until it fills half the samples of half the blocks. All flagged samples of a window are solved together by exact least-squares AR interpolation ([`lpc`](https://github.com/audiojs/denoise/tree/main/packages/lpc)'s `arFill`, Janssen 1986 / Godsill-Rayner 1998), so a neighbouring impulse is never taken for sound. Then it looks again on the rebuilt sound, the models fitted to it, until no new impulse stands out (Vaseghi & Rayner's iteration): with the largest gone, the smaller ones show; a new one must stand out of the input too, so a rebuild's own departures are never taken for crackle. A ringing tick's tail and a small impulse beside a large one stay under the threshold, and a rebuild bends to fit them as if they were sound; so last, every sample from 0.1 ms before a crackle sample to 0.5 ms after it is made soft: the sound there is the posterior mean under the window's AR(64) model with a click of its own variance at each sample, the variances found by EM (impulsive noise as a scale mixture of Gaussians, Godsill & Rayner, IEEE TSAP 1998): where the sound fits the model the samples stay near as recorded, where it doesn't the model takes over. Samples farther than that from any crackle come back bit-exact.
|
|
550
|
+
Finds crackle, a worn record's dense small impulses, where the sound departs both from its AR prediction and from its least-squares interpolation from either side, and rebuilds all of it in a 46 ms window at once. The prediction error alone also stands out at the sound's own excitation, a voice's glottal pulses, a reed's or a bow's; the two-sided error (the prediction error through its matched filter, Vaseghi & Rayner 1990) holds those back by √Σa² against its spread and lifts an impulse added to the sound by as much, so crackle is where both stand over `threshold` × their local level: per 1.5 ms block the median |error|, the median of those over ±12 ms, which crackle can't raise until it fills half the samples of half the blocks. All flagged samples of a window are solved together by exact least-squares AR interpolation ([`lpc`](https://github.com/audiojs/denoise/tree/main/packages/lpc)'s `arFill`, Janssen 1986 / Godsill-Rayner 1998), so a neighbouring impulse is never taken for sound. Then it looks again on the rebuilt sound, the models fitted to it, until no new impulse stands out (Vaseghi & Rayner's iteration): with the largest gone, the smaller ones show; a new one must stand out of the input too, so a rebuild's own departures are never taken for crackle. Each event is then judged as `declick` judges a click: an impulse added to the recording against the sound's own excitation (an additive outlier against an innovational one, Chang, Tiao & Chen 1988), under AR fitted to the rebuilt sound, each explanation by its fewest freed samples at ln T of the excitation's variance each (Schwarz 1978). A voice's glottal pulse, a bow's or a reed's catch, a drum's attack is one outlier of the excitation, the recording after it its smooth response; an impulse added stands in `order` + 1 errors, and its own sample frees them all. What the excitation explains as well is left as recorded (and crackle within 0.7 ms of such a pulse with it). A ringing tick's tail and a small impulse beside a large one stay under the threshold, and a rebuild bends to fit them as if they were sound; so last, every sample from 0.1 ms before a crackle sample to 0.5 ms after it is made soft: the sound there is the posterior mean under the window's AR(64) model with a click of its own variance at each sample, the variances found by EM (impulsive noise as a scale mixture of Gaussians, Godsill & Rayner, IEEE TSAP 1998): where the sound fits the model the samples stay near as recorded, where it doesn't the model takes over. Samples farther than that from any crackle come back bit-exact.
|
|
518
551
|
|
|
519
552
|
```js
|
|
520
553
|
decrackle(data, { fs: 44100 })
|
|
@@ -530,21 +563,21 @@ decrackle(data, { fs: 44100 })
|
|
|
530
563
|
|
|
531
564
|
| crackle, peaks × RMS | 50/s | 200/s | 1000/s |
|
|
532
565
|
|---|---:|---:|---:|
|
|
533
|
-
| small (0.1–0.5×) | 38.7 →
|
|
534
|
-
| medium (0.4–2×) | 26.7 → 39.
|
|
535
|
-
| loud (1.6–8×) | 14.8 → 32.
|
|
566
|
+
| small (0.1–0.5×) | 38.7 → 45.5 · 32.1 | 32.7 → 39.4 · 31.1 | 25.8 → 31.8 · 27.5 |
|
|
567
|
+
| medium (0.4–2×) | 26.7 → 39.5 · 30.7 | 20.8 → 32.7 · 28.0 | 13.8 → 23.8 · 20.3 |
|
|
568
|
+
| loud (1.6–8×) | 14.8 → 32.7 · 26.1 | 8.6 → 25.1 · 19.7 | 1.7 → 15.7 · 8.7 |
|
|
536
569
|
|
|
537
|
-
`decrackle` does better than `adeclick` in
|
|
570
|
+
`decrackle` does better than `adeclick` in 53 of the 54 cases, by up to 23.5 dB, and ties in one (the sung vibrato under small crackle at 1000/s, 30.9 each; 0.3.1 led in all 54); the sung vibrato, where 0.2.0 trailed in 8 (loud crackle at 50/s: 24.2 · 29.1), leads in the other 8 (30.7 · 29.1). 0.3.1, before each event was judged: 44.7, 39.3 and 32.0 dB small; 39.4, 32.9, 23.9 medium; 32.5, 25.1, 15.8 loud. 0.2.0 (the rebuild alone, no soft samples): 44.2, 38.3 and 30.2 dB small; 36.4, 29.6, 20.8 medium; 29.2, 21.8, 12.0 loud; the soft samples gain up to 6.5 dB (0.2–6.5 in 51 of the 54 cases, none in one) and lose 3.3 and 1.3 dB on the trumpet with small crackle at 50 and 200/s. The clean sound through it: audio-lena and Brahms not a sample changed; VoiceBank 1.4% of the samples, to −41 dB; "Vibe Ace" and the trumpet 0.07% and 0.05%, −52 and −69 dB; the song 0.17%, −64 dB (`adeclick`: 1.9–8.5%, −26 to −47 dB). 0.3.1 took the sound's own excitation for crackle too, a voice's glottal pulses, a bow's catch, and softened 0.6 ms around each: audio-lena 0.05%, −73 dB; VoiceBank 2.2%, −41; the music 0.07–1.1%, −49 to −70; the song 0.3%, −60. 0.2.0 changed 0.01–0.22%, to −41 to −74 dB. Tuned on other material (two narrations, VoiceBank+DEMAND training takes, other parts of "Vibe Ace" and Brahms, the Nutcracker, two other VocalSet singers): there 0.2.0 trailed `adeclick` in 23 of 72 cases, now in none. 0.1.7 (2.5 × its window's MAD, 30 Gauss-Seidel sweeps of the fill) made small crackle worse than it found it (38.7 → 21.9 dB at 50/s, the trumpet to 5.6) and changed 3.7–10% of the clean samples (the trumpet to −5.5 dB, speech to −21 dB). 1–1.6× 0.2.0's time, run side by side (the most at 1000/s).
|
|
538
571
|
|
|
539
572
|
Against iZotope RX 12 Advanced De-crackle (VST3 hosted by Pedalboard; [`audio`](https://github.com/audiojs/audio)'s `bench/rx/decrackle.mjs`, October 2026), through `audio`'s `decrackle()` on other takes (12 s of VoiceBank+DEMAND's clean test utterances, two Spoken Wikipedia narrations, "Vibe Ace" and Brahms 5–15 s, the trumpet), the same crackle: RX at its defaults (quality Low, strength 5) and at its best for each size on separate takes (quality × strength, then amplitude skew): small Medium, 3; medium High, 9.5; loud Medium, 9.5, skew 5. SDR out, dB, speech at 50 / 200 / 1000 a second · music:
|
|
540
573
|
|
|
541
574
|
| crackle | in | RX, defaults | RX, best for the size | `decrackle` |
|
|
542
575
|
|---|---|---|---|---|
|
|
543
|
-
| small | 38.7 / 32.9 / 26.0 · 38.9 / 32.8 / 25.7 | 30.7 / 29.5 / 25.6 · 40.5 / 35.0 / 27.1 | 38.1 / 33.2 / 26.2 · 40.7 / 34.1 / 26.2 |
|
|
544
|
-
| medium | 27.1 / 20.9 / 14.0 · 26.7 / 20.8 / 13.7 | 28.1 / 24.0 / 16.2 · 31.2 / 25.1 / 15.9 | 22.6 / 21.0 / 17.7 · 31.3 / 26.1 / 18.9 | 38.
|
|
545
|
-
| loud | 14.8 / 8.9 / 1.9 · 14.8 / 8.6 / 1.7 | 22.1 / 15.9 / 5.4 · 22.4 / 15.0 / 4.8 | 19.4 / 15.5 / 10.1 · 22.7 / 17.2 / 11.4 |
|
|
576
|
+
| small | 38.7 / 32.9 / 26.0 · 38.9 / 32.8 / 25.7 | 30.7 / 29.5 / 25.6 · 40.5 / 35.0 / 27.1 | 38.1 / 33.2 / 26.2 · 40.7 / 34.1 / 26.2 | 43.4 / 38.7 / 31.5 · 46.5 / 40.0 / 31.9 |
|
|
577
|
+
| medium | 27.1 / 20.9 / 14.0 · 26.7 / 20.8 / 13.7 | 28.1 / 24.0 / 16.2 · 31.2 / 25.1 / 15.9 | 22.6 / 21.0 / 17.7 · 31.3 / 26.1 / 18.9 | 38.5 / 32.8 / 23.8 · 39.5 / 33.4 / 24.0 |
|
|
578
|
+
| loud | 14.8 / 8.9 / 1.9 · 14.8 / 8.6 / 1.7 | 22.1 / 15.9 / 5.4 · 22.4 / 15.0 / 4.8 | 19.4 / 15.5 / 10.1 · 22.7 / 17.2 / 11.4 | 31.6 / 25.3 / 16.7 · 32.7 / 25.5 / 16.2 |
|
|
546
579
|
|
|
547
|
-
RX at its defaults makes small crackle on speech worse (38.7 → 30.7 dB at 50 a second). The clean takes through each (samples moved over 2⁻¹⁵ · error under the sound): `decrackle` 0
|
|
580
|
+
`decrackle` is 4.8–15.9 dB over RX at its best for each size, in every rate and size (0.3.1, before each event was judged: 4.2–15.8; cell by cell 0.5 dB over these to 1.2 under). RX at its defaults makes small crackle on speech worse (38.7 → 30.7 dB at 50 a second). The clean takes through each (samples moved over 2⁻¹⁵ · error under the sound): `decrackle` 0–1.4% · −44 dB to none (Brahms comes back bit-exact); RX at its defaults 3.7–48% · −25 to −52; at its best for medium and loud crackle 4–75% · −15 to −59; at its best for small crackle, its gentlest, 0.3–10.3% · −36 to −67: more samples than `decrackle` on all six takes (6 to 110 times as many where `decrackle` moves any), more error on four (the narrations −51 and −36 against −70 and −47, Brahms −61 against none, the trumpet −67 against −69), less on two (VoiceBank −46 against −44, "Vibe Ace" −58 against −55). 0.3.1 moved 0.07–2.3% · −37 to −66 dB, less error than RX's gentlest on two takes: it took the sound's own excitation for crackle and softened 0.6 ms around each. What is left on VoiceBank is mostly its speakers' own mouth clicks (a 9-sample step at 9.58 s, a burst near 20 kHz at 0.90 s), which nothing in a recording tells from crackle; on "Vibe Ace", a percussive attack. The judging was chosen on the tuning takes (other utterances, narrations and passages, the Nutcracker): there the 54 crackle cases average 31.9 dB as with 0.3.1, and the clean takes' error falls from −38 to −69 dB to −39 dB to none (two untouched).
|
|
548
581
|
|
|
549
582
|
**Use when:** shellac / 78 RPM crackle; a worn LP's surface noise; dense small ticks under music or speech.<br>
|
|
550
583
|
**Not for:** loud isolated clicks and pops (use `declick`); long dropouts (use `repair`).
|
|
@@ -554,18 +587,20 @@ RX at its defaults makes small crackle on speech worse (38.7 → 30.7 dB at 50 a
|
|
|
554
587
|
|
|
555
588
|
Finds where a sound was cut and rebuilds what the cut took off. A hard clip piles every sample it cuts onto one level, its rail. Each side's rail is the sound's extreme there, taken when the samples within 0.1 % of it outnumber those in the next 0.1 % below tenfold and most of them sit in runs: the top-bin test of FFmpeg's `adeclip` (1000 bins, ratio 10), made relative to the rail so it holds at any level (and never finer than one step of a 16- or 24-bit grid), and per side, so one rail or two different ones are found. A loud peak is one sample; a limiter's ceiling is touched by single samples or by arches, and a sine's top is an arch, which fill the band below about as densely.
|
|
556
589
|
|
|
557
|
-
Lossy coding (MP3, AAC, Opus) leaves no flat top: it spreads the rail into a band, the coder's noise around the level the runs were cut at, a mode of the amplitude density with mass on both sides of it. A waveform's own peaks, a sine's top, a limiter's ceiling, only approach a level from below. So, where no side has a flat rail, each side's band is its highest mode standing 4 times over the density 2–6 % of the extreme below it (at 1, 2 and 4 times that scale, for coarser coders), with samples over it reaching 3 of its spreads σ (the RMS of the samples over the mode) and beyond; the sound is cut again 6σ under the mode and rebuilt as a clip there (κ = 6 chosen on VoiceBank and MUSDB18 training material through LAME at 128 kbit/s: 0.6 dB over 4 on average, speech +0.9, music −0.1; 5 and 8 within 0.2 on speech).
|
|
590
|
+
Lossy coding (MP3, AAC, Opus) leaves no flat top: it spreads the rail into a band, the coder's noise around the level the runs were cut at, a mode of the amplitude density with mass on both sides of it. A waveform's own peaks, a sine's top, a limiter's ceiling, only approach a level from below. So, where no side has a flat rail, each side's band is its highest mode standing 4 times over the density 2–6 % of the extreme below it (at 1, 2 and 4 times that scale, for coarser coders), with samples over it reaching 3 of its spreads σ (the RMS of the samples over the mode) and beyond; the sound is cut again 6σ under the mode and rebuilt as a clip there (κ = 6 chosen on VoiceBank and MUSDB18 training material through LAME at 128 kbit/s: 0.6 dB over 4 on average, speech +0.9, music −0.1; 5 and 8 within 0.2 on speech).
|
|
591
|
+
|
|
592
|
+
Soft saturation, a valve, transistor or tape stage driven hot, leaves no rail and no band: a memoryless curve bends the peaks over toward a ceiling. Where neither is found, the curve is fitted blind, by maximum likelihood, the sound before it taken for locally AR Gaussian: for a curve f, z = f⁻¹(x) is the sound it bent, and the likelihood is AR(24)'s of z per window of 46 ms, the variance profiled out, times the Jacobian of f⁻¹ (a Wiener system, a linear sound then a memoryless curve, inverted blindly; Taleb, Solé-Casals & Jutten 2001 invert one by its output's independence). Two shapes, each of unit slope at 0, with a ceiling per side and a knee k: c·tanh((|x|/c)^k)^(1/k), tanh at k = 1, an exponential approach to the ceiling; and x/(1 + (|x|/c)^k)^(1/k), an algebraic one, as arctan's. It reads the 48 windows of the highest peaks, a quarter of them first to screen, and takes a curve that gains over 0.02 nats a sample against the sound as it is and bends the loudest peak to half its slope or less. A band the recording left empty (a lowpass, a codec's) would make a curve's new harmonics there cost more than they should, so each window's fit keeps a white floor 70 dB under its power. The sound is then inverted, z = f⁻¹(x), where the curve keeps over 1/40 of its slope (its noise and the fit's error raised at most 32 dB); above, z is unknown and held at least at f⁻¹ there: a clip, rebuilt as one. A limiter is no curve: its gain moves over milliseconds and keeps each wave's shape, and undoing it is an expander's work, not a repair's. Sound with no rail, no band and no curve comes back bit-exact: none of 1736 clean recordings, as they are, peak-normalized, or driven 12 dB into a lookahead limiter, shows one (VoiceBank+DEMAND's 1328 utterances, MUSDB18's 144 mixtures whole, the survey's ten SQAM excerpts, the four pieces whole, ten Spoken Wikipedia narrations, 100 VocalSet takes, 120 GuitarSet takes, 20 Slakh mixes), nor do sines from 50 Hz to 5 kHz. The most any gains is 0.01 nats a sample, VocalSet's sung long tones, whose curve leaves the loudest peak 0.6–0.9 of its slope; MUSDB18's mixtures, as they are and limited, 10⁻⁴. On the tuning material of the comparison below, saturation to 10 dB SDR gains 0.07–0.23 nats a sample on speech and 0.5–1.1 on music, its peaks bent to 1/17 of their slope or less; the slope's limit, 40, was chosen there: against 10 and 20, up to 9 dB more on its tanh-saturated mixtures, the same on its speech and under curves outside the two (arctan, a side's ceiling apart, hiss after the curve). The screen costs about a tenth of a second on clean sound; a fit, a few seconds.
|
|
558
593
|
|
|
559
594
|
Every sample at a rail is then rebuilt three times, each rebuild held at least as far out as the sample was recorded (a clip only ever lowers a sample). AR: per window of 186 ms, AR(256) fitted under its taper, every unknown solved at once by exact least squares ([`lpc`](https://github.com/audiojs/denoise/tree/main/packages/lpc)'s `arFill`), refitted and solved again until the fill moves by under 1 %, at most 8 times (Janssen, Veldhuis & Vries 1986). Sparse: A-SPADE (Kitić, Bertin & Gribonval 2015) as the declipping survey's code runs it (Záviška, Rajmic, Ozerov & Rencker 2021), the fewest DFT lines that agree with what was recorded, per block of 93 ms and per block of 186 ms: the long block resolves a held note's partials (a held chord clipped to 1 dB, half a second: 11.6 → 14.2 dB), the short one follows speech and attacks. No one of the three wins everywhere, so they are blended, region by region, by weights chosen by cross-validation (Stone 1974; stacked regressions, Breiman 1996): the clipped sound is clipped once more, at the level that takes as large a share of what the first clip left as the first took of the whole; each method rebuilds that; the weights, on a grid of tenths over the three, are those whose blend lies nearest what is known there (the samples the second clip hid, and the bounds of those the first took), read per 93 ms over ±1 s. A blend of consistent rebuilds is consistent; each rebuild runs only where its weight is not 0. Samples off the rails come back bit-exact; a sound over 30 s is rebuilt in pieces crossfaded over 2 s.
|
|
560
595
|
|
|
561
596
|
```js
|
|
562
|
-
declip(data, { fs: 44100 }) // each side's rail (or band) found; none: untouched
|
|
563
|
-
declip(data, { fs: 44100, clipLevel: 0.95 }) // a known rail, ±0.95
|
|
597
|
+
declip(data, { fs: 44100 }) // each side's rail (or band, or curve) found; none: untouched
|
|
598
|
+
declip(data, { fs: 44100, clipLevel: 0.95 }) // a known rail, ±0.95
|
|
564
599
|
```
|
|
565
600
|
|
|
566
601
|
| Param | Default | |
|
|
567
602
|
|---|---|---|
|
|
568
|
-
| `clipLevel` | auto | the rail, ± this; omitted or 0: found per side |
|
|
603
|
+
| `clipLevel` | auto | the rail, ± this; omitted or 0: found per side (a rail, a band, or a curve) |
|
|
569
604
|
| `order` | `256` | AR order of the AR rebuild; `512` takes it on dense music up to 2 dB closer, in about 3× the time |
|
|
570
605
|
| `fs` | `44100` | sample rate, Hz |
|
|
571
606
|
|
|
@@ -594,7 +629,7 @@ declip(data, { fs: 44100, clipLevel: 0.95 }) // a known rail,
|
|
|
594
629
|
|
|
595
630
|
On the mean ΔSDR 2nd of the 7 (0.3.0: 5th), 1.0 dB behind social sparsity; from 3 dB up ahead of both SPADEs and ℓ1 at every level, of NMF but at 15 dB (0.1), behind social sparsity by 0.15–1.05 dB; at 1 dB behind A-SPADE by 2.2 and social sparsity by 2.5. On ODG 4th, behind NMF, social sparsity and ℓ1 by 0.18–0.38, at 3–7 dB by up to 0.7. The blend misjudges some held tones: the clarinet clipped to 15 dB comes back 17.1 dB, its 186 ms sparse rebuild alone 22.1 (the second clip scores a fill's overshoot inside the first clip's runs as nothing, only its bound being known there). On VoiceBank speech (below) it is 0.3.0 within 0.1 dB at every level.
|
|
596
631
|
|
|
597
|
-
Against iZotope RX 12 Advanced De-clip (VST3 hosted by Pedalboard, output aligned to the input; [audio](https://github.com/audiojs/audio)'s `bench/rx/declip.mjs`, 2026-10), through `audio`'s `declip()`. Test: the ten SQAM excerpts; "other music", four MUSDB18 test mixtures and "Vibe Ace", Brahms, the trumpet and the Sugar Plum Fairy (6 s each); speech, 16 VoiceBank clean test utterances. Each clipped at the level leaving it 1–20 dB from the original and scaled so its rails sit at 0 dBFS; asym: the positive rail at the 15 dB level, the negative at the 7 dB level; down: the 7 dB clip turned down 6 dB; MP3: the 7 dB clip through LAME at 128 kbit/s; soft: tanh(g·x), g giving 10 dB (analog saturation, no rail). RX at its defaults (threshold −1.02 dBFS, quality Low, post-limiter on), and tuned per condition and kind on other material (four MUSDB18 training mixtures, ten VoiceBank training utterances): asymmetric thresholds at each side's peak plus an offset (−0.07 to −6 dB) × quality, post-limiter off; its best: −0.07 dB on flat rails (High; Low or Medium on some speech conditions), High at −2 dB on MP3, −2 (music) and −4 (speech) on soft. Mean ΔSDR, dB, RX default · RX tuned · declip:
|
|
632
|
+
Against iZotope RX 12 Advanced De-clip (VST3 hosted by Pedalboard, output aligned to the input; [audio](https://github.com/audiojs/audio)'s `bench/rx/declip.mjs`, 2026-10), through `audio`'s `declip()`. Test: the ten SQAM excerpts; "other music", four MUSDB18 test mixtures and "Vibe Ace", Brahms, the trumpet and the Sugar Plum Fairy (6 s each); speech, 16 VoiceBank clean test utterances. Each clipped at the level leaving it 1–20 dB from the original and scaled so its rails sit at 0 dBFS; asym: the positive rail at the 15 dB level, the negative at the 7 dB level; down: the 7 dB clip turned down 6 dB; MP3: the 7 dB clip through LAME at 128 kbit/s; soft: tanh(g·x), g giving 10 dB (analog saturation, no rail); atan: (2/π)·arctan(π/2·g·x) likewise, a softer knee outside both of declip's curves. RX at its defaults (threshold −1.02 dBFS, quality Low, post-limiter on), and tuned per condition and kind on other material (four MUSDB18 training mixtures, ten VoiceBank training utterances): asymmetric thresholds at each side's peak plus an offset (−0.07 to −6 dB) × quality, post-limiter off; its best: −0.07 dB on flat rails (High; Low or Medium on some speech conditions), High at −2 dB on MP3, −2 (music) and −4 (speech) on soft, −2 and −6 on atan. Mean ΔSDR, dB, RX default · RX tuned · declip:
|
|
598
633
|
|
|
599
634
|
| condition | SQAM | other music | speech |
|
|
600
635
|
|---|---:|---:|---:|
|
|
@@ -608,30 +643,35 @@ Against iZotope RX 12 Advanced De-clip (VST3 hosted by Pedalboard, output aligne
|
|
|
608
643
|
| asym | −1.4 · 12.0 · 18.4 | −0.9 · 6.4 · 14.2 | −2.1 · 10.5 · 12.0 |
|
|
609
644
|
| down | 0.0 · 9.7 · 16.7 | 0.0 · 4.8 · 12.8 | 0.0 · 6.8 · 10.5 |
|
|
610
645
|
| MP3 | −2.0 · 7.1 · 11.7 | −2.2 · 3.5 · 9.2 | −2.2 · 5.2 · 9.5 |
|
|
611
|
-
| soft | −0.8 · 0.6 · 0.0
|
|
646
|
+
| soft | −0.8 · 0.6 · 40.0 (0.4.0: 0.0; its level given: 1.6) | −0.7 · 0.2 · 39.9 (0.0; 2.0) | −1.0 · 2.2 · 20.0 (0.0; 5.1) |
|
|
647
|
+
| atan | −0.0 · 0.2 · 16.7 | −0.0 · 0.0 · 17.0 | −0.0 · 1.1 · 17.3 |
|
|
612
648
|
|
|
613
|
-
Perceptually the same order: PEAQ ODG on the music, PESQ (wideband, 16 kHz) on speech, input · RX tuned · declip: 7 dB, SQAM −3.75 · −2.44 · −1.74, other music −3.65 · −2.72 · −1.85, speech 2.10 · 3.42 · 3.85; 20 dB −2.35 · −0.39 · −0.18, −1.29 · −0.23 · −0.12, 3.69 · 4.50 · 4.56; MP3 −3.49 · −3.27 · −2.78, −3.60 · −3.21 · −2.77, 2.10 · 3.05 · 3.74; soft
|
|
649
|
+
Perceptually the same order: PEAQ ODG on the music, PESQ (wideband, 16 kHz) on speech, input · RX tuned · declip: 7 dB, SQAM −3.75 · −2.44 · −1.74, other music −3.65 · −2.72 · −1.85, speech 2.10 · 3.42 · 3.85; 20 dB −2.35 · −0.39 · −0.18, −1.29 · −0.23 · −0.12, 3.69 · 4.50 · 4.56; MP3 −3.49 · −3.27 · −2.78, −3.60 · −3.21 · −2.77, 2.10 · 3.05 · 3.74; soft −2.86 · −2.84 · 0.11, −2.51 · −2.40 · 0.16, 3.12 · 3.41 · 4.60 (0.4.0, its level given: −2.75, −2.21, 3.85); atan −2.87 · −2.86 · −0.88, −2.50 · −2.46 · −0.32, 3.20 · 3.22 · 4.49. RX at its defaults takes SDR off everywhere: its post-limiter holds the restored peaks to 0 dBFS, and its threshold finds nothing on rails under −1 dBFS (down). On the unclipped recordings peak-normalized to 0 dBFS, and through a lookahead limiter 12 dB over a −0.2 dBFS ceiling, declip changes no sample; RX changes them (SDR to its input, median: defaults 59–68 dB, 43–49 limited; tuned 77–94, 58–74). On soft saturation 0.4.0 found no rail and returned it untouched; given the level RX's tuned threshold sits at, `declip({ clipLevel })` gained 1.6–5.1 dB. 0.5.0 fits the curve itself (on all 34 takes of each: 0.07–2.2 nats a sample under tanh, 0.05–1.3 under arctan, the peaks bent to 1/24 and 1/9 of their slope or less): under tanh, one of its curves, 20–40 dB; under arctan, outside both, 16.7–17.3; RX tuned 0.0–2.2 on either. The other conditions' 340 clipped takes each have a rail or a band, found as 0.4.0 finds them, so 0.5.0 takes 0.4.0's path there and its numbers are 0.4.0's (asym, rendered again: the same to the sample).
|
|
650
|
+
|
|
651
|
+
Under curves outside its two and the noise a real chain adds after the curve, `declip` alone (`node bench/rx/declip.mjs curves`, the same test material), SDR in → out, dB, SQAM · other music · speech: x/√(1 + x²) to 10 dB, 29.7 · 28.7 · 28.3; tanh with the negative ceiling at 0.7, 22.8 · 23.0 · 23.1; a biased tanh, tanh(x + 0.3) − tanh(0.3) (even harmonics, a valve's), 13.0 · 12.6 · 14.8; tanh to 20 dB, 50.4 · 63.4 · 26.1; to 30 dB, 35.6 · 34.7 · 30.0 (speech bent so little is left as it is); tanh to 10 dB, then hiss 60 dB under full scale, 22.4 · 22.1 · 23.0. Hiss draws the fitted ceiling outward, a biased curve is fitted by an unbiased one: still well over the input, short of the curves it fits exactly.
|
|
614
652
|
|
|
615
653
|
Rails are found at the 20 dB level clipped asymmetrically (with the 10 dB level below), on one side only, then turned down to 0.4, quantized to 16 bits plain or dithered, and on a 16-bit converter driven 2.5 dB over (rails at 32767 and −32768): 13 dB or more of SDR gained in each; through MP3 and AAC at 128 kbit/s, bands: speech 18.0 → 25.2 and 19.9 → 31.6, "Vibe Ace" 18.3 → 25.1 and 19.8 → 28.7, Brahms 18.2 → 24.8 and 19.6 → 28.0, the trumpet 18.1 → 24.9 and 19.3 → 26.9 dB (0.3.0: untouched). The unclipped material through it, as it is, peak-normalized, at 16 bits and through a lookahead limiter 12 dB over its ceiling: not a sample changed; nor in any of VoiceBank+DEMAND's 824 clean test utterances. The third rebuild costs time: 1.2–2.1× 0.3.0's side by side (0.3.0: about 7 s per second of sound averaged over the six levels, most of it at 1–3 dB), the AR rebuild's early stop giving some back (speech clipped to 3 dB: 0.8×).
|
|
616
654
|
|
|
617
|
-
**Use when:** digital clipping: a converter overdriven, a mix bounced too hot, a plug-in's hard clip; at any level, on either side or both; also after it went through MP3, AAC or Opus.<br>
|
|
618
|
-
**Not for:**
|
|
655
|
+
**Use when:** digital clipping: a converter overdriven, a mix bounced too hot, a plug-in's hard clip; at any level, on either side or both; also after it went through MP3, AAC or Opus; soft saturation, a valve, transistor or tape stage driven hot, found by itself.<br>
|
|
656
|
+
**Not for:** a limiter's or a compressor's work (a gain that moves, no curve: left as it is); saturation with memory (tape's hysteresis, a stage whose bias shifts with the level), where a curve fitted to the whole take is only near; clipping that has since been resampled or heavily filtered.
|
|
619
657
|
|
|
620
658
|
|
|
621
659
|
## Reverb
|
|
622
660
|
|
|
623
661
|
### `dereverb`
|
|
624
662
|
|
|
625
|
-
Late reverberation off a voice, one microphone, the whole take at once. First weighted prediction error, WPE (Nakatani et al. 2010): in each STFT bin, what the frames from 43 to 150 ms back predict of the current one (at 48 kHz: from the first frame that shares no sample with it) is the room's tail, and is subtracted; the prediction is fitted over the whole take, each frame weighted by its inverse power, three times. The room is one for the take and the voice is not, so the fit learns the room. One microphone cannot invert a room, and the prediction cancels a dB or two of the tail. The rest is taken in power, as Lebart et al. (2001) and Habets (2010) model the late reverberation: from the power of the frames before, here weighed by the shape of WPE's taps, the room's decay in that bin. Its scale is read off the take itself: where only the tail sounds, the power over its expectation is exponential, and a tenth of such cells fall under 0.105 of it, while the voice's cells lie above; so the 10th percentile of the power over the weighed past, per octave band, divided by 0.105, is the late reverberation's share. It reads a 3 s clip and a minute-long take alike (0.3 scaled the taps' own power, which on 3 s predicted about all the late power and on 30 s a quarter). A log-spectral-amplitude gain (Ephraim & Malah 1985) takes that power off what WPE left, floored at −14 dB, and no bin leaves louder than it came. Three checks on the whole take come first, and any returns it bit for bit: dry, when its fastest falls are faster than a room's tail lets a sound fall (90 % of dry VoiceBank takes); no diffuse tail, when its lowest cells do not fall as a room's tail does: a diffuse tail's power is exponential, its 2nd percentile 7.2 dB under its 10th (ln 0.98 / ln 0.9), while a dry sound's own decays, a voice's closures, a muted string, part them further (the training rooms 7.0–8.1 dB apart, dry VoiceBank 8.6–17.7, GuitarSet 8.3–12.6; over 9, the take passes); no pauses, when under half of its frames are under half its mean power (Scheirer & Slaney 1997): there a held note's sustain reads as a room's. A bin cut to digital silence (an edit, a gate) is left out of the fit, and the frames that reach into such a cut, or past the take's end, out of the checks and the scale: their fall is the cut's. About 0.1 s per second of 48 kHz sound (a 41 s take: 4.7 s, 0.3 4.0 s).
|
|
663
|
+
Late reverberation off a voice, one microphone, the whole take at once. First weighted prediction error, WPE (Nakatani et al. 2010): in each STFT bin, what the frames from 43 to 150 ms back predict of the current one (at 48 kHz: from the first frame that shares no sample with it) is the room's tail, and is subtracted; the prediction is fitted over the whole take, each frame weighted by its inverse power, three times. The room is one for the take and the voice is not, so the fit learns the room. One microphone cannot invert a room, and the prediction cancels a dB or two of the tail. The rest is taken in power, as Lebart et al. (2001) and Habets (2010) model the late reverberation: from the power of the frames before, here weighed by the shape of WPE's taps, the room's decay in that bin. Its scale is read off the take itself: where only the tail sounds, the power over its expectation is exponential, and a tenth of such cells fall under 0.105 of it, while the voice's cells lie above; so the 10th percentile of the power over the weighed past, per octave band, divided by 0.105, is the late reverberation's share. It reads a 3 s clip and a minute-long take alike (0.3 scaled the taps' own power, which on 3 s predicted about all the late power and on 30 s a quarter). A log-spectral-amplitude gain (Ephraim & Malah 1985) takes that power off what WPE left, floored at −14 dB, and no bin leaves louder than it came. Three checks on the whole take come first, and any returns it bit for bit: dry, when its fastest falls are faster than a room's tail lets a sound fall (90 % of dry VoiceBank takes); no diffuse tail, when its lowest cells do not fall as a room's tail does: a diffuse tail's power is exponential, its 2nd percentile 7.2 dB under its 10th (ln 0.98 / ln 0.9), while a dry sound's own decays, a voice's closures, a muted string, part them further (the training rooms 7.0–8.1 dB apart, dry VoiceBank 8.6–17.7, GuitarSet 8.3–12.6; over 9, the take passes); no pauses, when under half of its frames are under half its mean power (Scheirer & Slaney 1997): there a held note's sustain reads as a room's. Before them, from 0.6, music passes: a take that holds a partial (a spectral peak at one frequency, 200 Hz–4 kHz, its power within 3 dB for 0.3 s: a sung, bowed or blown note) or keeps a beat (its onsets recurring at one period, 40–200 bpm, past what chance gives a voice for the take's length), two marks Scheirer & Slaney tell music by; recorded in a room, music passes the three checks, and the late power the past carries over-reads a held note (below). A bin cut to digital silence (an edit, a gate) is left out of the fit, and the frames that reach into such a cut, or past the take's end, out of the checks and the scale: their fall is the cut's. About 0.1 s per second of 48 kHz sound (a 41 s take: 4.7 s, 0.3 4.0 s; the music check adds about a tenth).
|
|
626
664
|
|
|
627
665
|
```js
|
|
628
666
|
dereverb(data, { fs: 48000 })
|
|
629
667
|
dereverb(data, { fs: 48000, strength: 0.5 }) // a lighter hand
|
|
668
|
+
dereverb(data, { fs: 48000, music: 'enhance' }) // music in a room taken as a room too
|
|
630
669
|
```
|
|
631
670
|
|
|
632
671
|
| Param | Default | |
|
|
633
672
|
|---|---|---|
|
|
634
673
|
| `strength` | `1` | Scale of the late-reverberation estimate (1: as the take's own decays read it): 0 is the linear prediction alone, 2 takes more of the tail and more of the voice |
|
|
674
|
+
| `music` | `'pass'` | `'pass'`: a take holding a partial or keeping a beat comes back as it went; `'enhance'`: taken as a room too, a held note losing with the tail |
|
|
635
675
|
|
|
636
676
|
Measured against iZotope RX 12 Advanced De-reverb (VST3 hosted by Pedalboard; [`audio`](https://github.com/audiojs/audio)'s `bench/rx/dereverb.mjs`, October 2026), through `audio`'s `dereverb()`, on the takes of `scripts/dereverb.py` (test set): 42 VoiceBank test utterances (1.6–5.9 s) and four 41–60 s takes of ten utterances by one speaker, in 19 MIT IR Survey rooms the tuning never heard (T60 0.38–1.85 s), each room's response split at direct + 50 ms: under it, the voice as the room colours it; over it, the tail to take. Voice lost: the output's power where the voice is 10 dB over the tail; tail taken: where the tail is 10 dB over the voice. PESQ against the dry take, DNSMOS OVRL at the input's loudness; noisy: the same takes with DEMAND noise 10 dB under them; dry: their dry takes, the share returned bit for bit and PESQ. RX at its defaults (reduction 10 dB, tail length 1 s, artifact smoothing 9, band strengths 6); after its Learn on each take (it learns a tail length and four band strengths); and at its best on reverberant VoiceBank training utterances by PESQ, searched over reduction, tail length, smoothing, the band strengths, enhance dry signal and Learn (tail length 0.5 s, band strengths 8, 4, 6, 4, the rest at defaults). RX Dialogue Isolate, a trained separator, with its reverb gain at −∞ and its noise kept, for reference:
|
|
637
677
|
|
|
@@ -643,34 +683,35 @@ Measured against iZotope RX 12 Advanced De-reverb (VST3 hosted by Pedalboard; [`
|
|
|
643
683
|
| RX De-reverb, tuned | −0.18, −2.3 | 1.45 | 2.32 | −0.16, −1.5 | 1.43 | 2.09 | 1.27 | 1.75 | 0, 4.17 |
|
|
644
684
|
| RX Dialogue Isolate, reverb off | −3.18, −8.3 | **1.79** | **2.70** | −4.29, −6.2 | **1.77** | **2.57** | 1.34 | **2.18** | 0, 4.64 |
|
|
645
685
|
| `dereverb` 0.4 → 0.5 | −0.79, −6.3 | 1.53 | 2.49 | −0.61, −6.6 | 1.54 | 2.52 | **1.35** | 2.08 | 96 → **100 %**, 4.63 → **4.64** |
|
|
686
|
+
| `dereverb` 0.6 | −0.79, −6.3 | 1.53 | 2.49 | −0.61, −6.6 | 1.54 | 2.52 | **1.35** | 2.08 | **100 %**, **4.64** |
|
|
646
687
|
|
|
647
|
-
Take by take, `dereverb` over RX De-reverb tuned: PESQ +0.08 ± 0.02 (short), +0.10 ± 0.05 (long), +0.08 ± 0.02 (noisy), OVRL +0.17 ± 0.09, +0.43 ± 0.43, +0.33 ± 0.12 (95 % intervals); over its Learn, PESQ +0.06 ± 0.03, +0.02 ± 0.06, +0.06 ± 0.02. At the voice its Learn loses (−0.8 dB on the short takes) it takes 3.2 dB more of the tail; tuned, RX keeps more of the voice (−0.2 dB) and takes a third of the tail. On reverberant VoiceBank in 130 rooms (`vbreverb`, 206 test utterances; `audio`'s `bench/speech.mjs`): PESQ 2.33 in, RX defaults 2.42, tuned 2.46, `dereverb` 0.4 2.62, 0.5 2.61 (over RX tuned +0.14 ± 0.02), OVRL 2.82, 2.86, 2.87, 2.94, 2.94; the dry takes through it: RX 4.09 and 4.17, `dereverb` 4.63 → 4.64, untouched 94 → 100 %. On the training utterances RX's best setting gained PESQ 0.07 (2.42 → 2.49; on the half searched 2.37 → 2.44), its Learn 0.04 (2.37 → 2.41).
|
|
688
|
+
Take by take, `dereverb` over RX De-reverb tuned: PESQ +0.08 ± 0.02 (short), +0.10 ± 0.05 (long), +0.08 ± 0.02 (noisy), OVRL +0.17 ± 0.09, +0.43 ± 0.43, +0.33 ± 0.12 (95 % intervals); over its Learn, PESQ +0.06 ± 0.03, +0.02 ± 0.06, +0.06 ± 0.02. At the voice its Learn loses (−0.8 dB on the short takes) it takes 3.2 dB more of the tail; tuned, RX keeps more of the voice (−0.2 dB) and takes a third of the tail. On reverberant VoiceBank in 130 rooms (`vbreverb`, 206 test utterances; `audio`'s `bench/speech.mjs`): PESQ 2.33 in, RX defaults 2.42, tuned 2.46, `dereverb` 0.4 2.62, 0.5 and 0.6 2.61 (over RX tuned +0.14 ± 0.02; 0.6 passes one utterance more, its speaker holding a vowel's pitch: 2.606 → 2.604), OVRL 2.82, 2.86, 2.87, 2.94, 2.94; the dry takes through it: RX 4.09 and 4.17, `dereverb` 4.63 → 4.64, untouched 94 → 100 %. On the training utterances RX's best setting gained PESQ 0.07 (2.42 → 2.49; on the half searched 2.37 → 2.44), its Learn 0.04 (2.37 → 2.41).
|
|
648
689
|
|
|
649
690
|
Where the rest lies: on the training rooms the take with its late part gone exactly scores PESQ 2.03 (short) and 2.12 (long), the ideal ratio mask (each cell's true voice and tail power) 1.98 and 2.11, `dereverb` 1.72 and 1.81. Its gain fed each cell's true late power, in place of the estimate, reaches 1.73 and 1.80 (1.75 and 1.83 at a −20 dB floor): the late power is estimated as well as it can be used. The floor (−10, −14, −20 dB), a Wiener gain for the LSA, the decision-directed memory (0.7–0.95) and the strength (0.7–1.5) all moved PESQ by under 0.02. What is missing is the voice's own power in each cell, which no model of the room gives and a trained model learns: RX Dialogue Isolate scores 0.26 over `dereverb` on the short takes, taking 3.2–4.3 dB of the voice's first 50 ms with the tail.
|
|
650
691
|
|
|
651
692
|
Through it as it is, music at 44.1 kHz (untouched: the share returned bit for bit; level change per octave, the worst of 63 Hz–8 kHz; with pauses: the same music cut into 2–4 s phrases, 0.5–1.5 s apart, over a noise floor 50 dB under it; in a room: through the test rooms, music lost and tail taken as for the voice):
|
|
652
693
|
|
|
653
|
-
| | RX defaults | RX learned | RX tuned | `dereverb` 0.4 → 0.5 |
|
|
654
|
-
|
|
655
|
-
| MUSDB18 previews (25) | −0.32 dB | −5.66 | −1.27 | 80 →
|
|
656
|
-
| … with pauses | −0.58 | −1.44 | −1.91 | 76 → **100 %**, −0.35 → **0** |
|
|
657
|
-
| GuitarSet (20) | −0.81 | −0.24 | −1.03 | 75 → **100 %**, −0.77 → **0** |
|
|
658
|
-
| … with pauses | −1.67 | −0.27 | −2.50 | 95 → **100 %**, 0 |
|
|
659
|
-
| VocalSet sung straight tones (10) | −0.34 | −2.29 | −0.14 | 70 → **100 %**, −0.65 → **0** |
|
|
660
|
-
| … with pauses | −0.59 | −0.25 | −2.68 | 100 %, 0 |
|
|
661
|
-
| Brahms, the Nutcracker, a trumpet, Vibe Ace | −0.53 | −5.58 | −1.74 | 50 %, −3.07 |
|
|
662
|
-
| … with pauses | −0.84 | −0.74 | −2.02 | 50 → **100 %**, −2.16 → **0** |
|
|
663
|
-
| in a room, music lost, tail taken dB: MUSDB18 | 0.0
|
|
664
|
-
| GuitarSet | −0.2, 0.0 | −0.4, −2.8 | −0.4, −0.7 | −1.6, −1.5 |
|
|
665
|
-
| the four pieces | −0.2, −0.4 | −3.0, −7.1 | −0.3, −2.5 | −1.1, −10.1 |
|
|
666
|
-
| VocalSet sung |
|
|
667
|
-
|
|
668
|
-
In a room the music's lowest cells are the room's tail, so
|
|
669
|
-
|
|
670
|
-
0.2 fitted the prediction recursively, over its last second, and learned some of the voice as room. 0.3 scaled the taps' power by a constant, so its estimate grew as the take shortened: on 3 s it took 2–3 dB more than on a minute, and a dry voice came out at PESQ 4.25. Late-reverb subtraction at each room's measured T60 (Lebart; Habets's κ, the T60 an oracle's) scored under the taps' shape scaled by the percentile (short training takes: PESQ 1.64 against 1.71), and an exponential decay at the measured T60 in place of the taps' shape scored the same. Refitting WPE on the gain's output (its weights from the voice estimate, as DNN-WPE takes them) changed nothing. Sparing a spectral peak that holds over 43 ms (a held partial) cost 1.5–2 dB of the tail; peaks held 0.3 s (as noise-estimate's partials guard holds them) hold in a long room's reverberant speech as in music (up to 95 % of the energy). Pitch steadiness, power steadiness over three frames and 4 Hz modulation overlapped between speech in a room and music. 0.4 judged a take dry on its fastest fall alone, and passed music only when it had no pauses: GuitarSet, sung tones and music with pauses were processed.
|
|
694
|
+
| | RX defaults | RX learned | RX tuned | `dereverb` 0.4 → 0.5 | 0.6 |
|
|
695
|
+
|---|---|---|---|---|---|
|
|
696
|
+
| MUSDB18 previews (25) | −0.32 dB | −5.66 | −1.27 | 80 → 92 %, −0.47 → −0.15 | **96 %**, **−0.12** |
|
|
697
|
+
| … with pauses | −0.58 | −1.44 | −1.91 | 76 → **100 %**, −0.35 → **0** | **100 %**, **0** |
|
|
698
|
+
| GuitarSet (20) | −0.81 | −0.24 | −1.03 | 75 → **100 %**, −0.77 → **0** | **100 %**, **0** |
|
|
699
|
+
| … with pauses | −1.67 | −0.27 | −2.50 | 95 → **100 %**, 0 | **100 %**, **0** |
|
|
700
|
+
| VocalSet sung straight tones (10) | −0.34 | −2.29 | −0.14 | 70 → **100 %**, −0.65 → **0** | **100 %**, **0** |
|
|
701
|
+
| … with pauses | −0.59 | −0.25 | −2.68 | 100 %, 0 | **100 %**, **0** |
|
|
702
|
+
| Brahms, the Nutcracker, a trumpet, Vibe Ace | −0.53 | −5.58 | −1.74 | 50 %, −3.07 | **100 %**, **0** |
|
|
703
|
+
| … with pauses | −0.84 | −0.74 | −2.02 | 50 → **100 %**, −2.16 → **0** | **100 %**, **0** |
|
|
704
|
+
| in a room, music lost, tail taken dB: MUSDB18 | **0.0**, −0.6 | −3.5, −8.6 | −0.3, −3.0 | −0.5, −0.1 | −0.1, −0.1 |
|
|
705
|
+
| GuitarSet | −0.2, 0.0 | −0.4, −2.8 | −0.4, −0.7 | −1.6, −1.5 | **−0.1**, +0.3 |
|
|
706
|
+
| the four pieces | −0.2, −0.4 | −3.0, −7.1 | −0.3, −2.5 | −1.1, −10.1 | **0.0**, 0.0 |
|
|
707
|
+
| VocalSet sung | **+0.1**, −0.2 | −1.3, −3.0 | **+0.1**, −0.1 | −2.9, −7.8 | **+0.1**, +0.1 |
|
|
708
|
+
|
|
709
|
+
In a room the music's lowest cells are the room's tail, so 0.5 processed it, and the late power the past carries over-reads a held note: in steady state the estimate is the scale times the note itself, while a room with a strong direct sound adds less (Lebart's model over-estimates as the direct-to-reverberant ratio grows; Habets, IEEE SPL 16(9), 2009); a take it processed lost 1 to 5.6 dB of its music where the music stood 10 dB over the tail. 0.6 reads music's marks before fitting the room: of the 40 music takes in the training rooms and halls that the three checks let through, 31 hold a partial or keep a beat and pass (all 15 of GuitarSet's, 9 of 10 sung, 3 of 5 of the pieces, 4 of 10 MUSDB18 songs), on the test takes 36 of 41, the four pieces as recorded all of them. A voice's pitch moves: of 152 voices in the training rooms (50 short and long takes, the same with noise, 42 reverberant VoiceBank utterances, ten narrations) none held a partial for over 2.2·10⁻⁵ of its energy, against the 3·10⁻⁴ that passes a take (test: a narration 1.5·10⁻⁴; of 50 dry utterances one held a vowel's pitch, 3.7·10⁻⁴), and none came nearer the beat's 1.5 than 1.09 (test 0.88); of the 206 reverberant VoiceBank test utterances one, whose speaker holds a vowel's pitch, now passes. What still goes through as a voice: a song whose drums and voice keep its spectrum moving and whose beat 7 s cannot show (3 of 25 MUSDB18 previews in a room, −0.1 dB). The constants were chosen on `scripts/dereverb.py train` (training speakers in the even-numbered rooms), the 9 dB on those takes, the dry training takes and the even-numbered MUSDB18 previews, other GuitarSet takes and VocalSet singers; it passes 13 % of the reverberant VoiceBank training utterances that 0.4 processed (rooms with little tail, where the voice's own falls outnumber the tail's), which cost PESQ 0.03 there and 0.01 on the test set; the music check's two on `audio`'s `bench/rx/dereverb.mjs` train split, an order of magnitude over the voices' held share and 0.4 over their beat.
|
|
710
|
+
|
|
711
|
+
0.2 fitted the prediction recursively, over its last second, and learned some of the voice as room. 0.3 scaled the taps' power by a constant, so its estimate grew as the take shortened: on 3 s it took 2–3 dB more than on a minute, and a dry voice came out at PESQ 4.25. Late-reverb subtraction at each room's measured T60 (Lebart; Habets's κ, the T60 an oracle's) scored under the taps' shape scaled by the percentile (short training takes: PESQ 1.64 against 1.71), and an exponential decay at the measured T60 in place of the taps' shape scored the same. Refitting WPE on the gain's output (its weights from the voice estimate, as DNN-WPE takes them) changed nothing. Sparing a spectral peak that holds over 43 ms (a held partial) cost 1.5–2 dB of the tail; peaks held 0.3 s (as noise-estimate's partials guard holds them) hold in a long room's reverberant speech as in music (up to 95 % of the energy). Pitch steadiness, power steadiness over three frames and 4 Hz modulation overlapped between speech in a room and music; so did spectral flux, the 1/3-octave spectrum's change of shape from frame to frame, whose border moved with the speaker (training voices 5.2 dB at least, a test speaker 4.7). 0.4 judged a take dry on its fastest fall alone, and passed music only when it had no pauses: GuitarSet, sung tones and music with pauses were processed.
|
|
671
712
|
|
|
672
713
|
**Use when:** a voice in a room, one microphone, one room per take.<br>
|
|
673
|
-
**Not for:** music
|
|
714
|
+
**Not for:** music, which passes untouched when it holds a partial or keeps a beat (a song with neither in a short take is processed: 3 of 25 MUSDB18 previews in a room, −0.1 dB; `music: 'enhance'` takes any music as a room, a held note losing 0.5–3 dB with the tail); a voice holding a sung pitch passes too. Noise: denoise first.
|
|
674
715
|
|
|
675
716
|
|
|
676
717
|
## Bleed
|
|
@@ -799,7 +840,7 @@ Three parts, as iZotope RX's Guitar De-noise has; squeak on by default, pick and
|
|
|
799
840
|
|
|
800
841
|
**Pick.** A pluck's click: a 2–10 kHz millisecond 9 dB or more over the quietest 4 ms of the 40 ms before it (a strum's strings click one after another, each over the last), the loudest within 20 ms. Its attack, from 2 ms before to `attack` after (10 ms), is taken down on 5.8 ms frames, each bin over 1.5 kHz to the level the note holds 15–35 ms on, `pick` dB at most: the click goes, the partials that ring on stay.
|
|
801
842
|
|
|
802
|
-
**Amp.** OM-LSA (`@audio/denoise-omlsa`, the held-noise form: Gerkmann, Breithaupt & Martin 2008) on a print read from the take's quietest frames: those within 1 dB of its 1st-percentile frame (the amp alone before the playing, where the take has that, else the gaps), split in time into four groups, per bin their mean, or where the groups' means spread over 6 dB (a note rang into some), the least group's ×1.5. A spectral peak 10 dB or more over the print keeps its level, with its main lobe: a note ringing on is peaks well over the noise, where hiss peaks so high in one bin in 22 000 and a buzz's line sits at the print. OM-LSA alone took the last chord of a take, 22 dB over the hiss, 6 dB down.
|
|
843
|
+
**Amp.** The buzz first: its lines are locked to the mains, so they are subtracted, each held through the take ([`dehum`](#dehum) with `steady`: one phasor per line, the mains phase refined against them; no hum found, nothing changed), where a gain cannot part a buzz line from a partial in its bin. Then OM-LSA (`@audio/denoise-omlsa`, the held-noise form: Gerkmann, Breithaupt & Martin 2008) on a print read from the take's quietest frames: those within 1 dB of its 1st-percentile frame (the amp alone before the playing, where the take has that, else the gaps), split in time into four groups, per bin their mean, or where the groups' means spread over 6 dB (a note rang into some), the least group's ×1.5. A spectral peak 10 dB or more over the print keeps its level, with its main lobe: a note ringing on is peaks well over the noise, where hiss peaks so high in one bin in 22 000 and a buzz's line sits at the print. OM-LSA alone took the last chord of a take, 22 dB over the hiss, 6 dB down.
|
|
803
844
|
|
|
804
845
|
```js
|
|
805
846
|
desqueak(data, { fs: 44100 }) // squeaks, up to 30 dB down
|
|
@@ -812,7 +853,7 @@ desqueak(data, { fs: 44100, squeak: 0, amp: -20 }) // the amp's noise alone
|
|
|
812
853
|
| `squeak` | `-30` | dB, the most a squeak goes down; `0` leaves squeaks |
|
|
813
854
|
| `pick` | `0` | dB, the most a pick's attack goes down; `0` off; `-9` restores attacks made 6–12 dB harsher |
|
|
814
855
|
| `attack` | `0.01` | s, the pick's attack taken, from 2 ms before its click |
|
|
815
|
-
| `amp` | `0` | dB, how far the amp's hiss, hum and buzz
|
|
856
|
+
| `amp` | `0` | dB, how far the amp's hiss goes down, and what the subtraction leaves of its hum and buzz; `0` off; `-20` typical |
|
|
816
857
|
|
|
817
858
|
The whole take is read at once (streaming: false); about 20× real time at 44.1 kHz on one core.
|
|
818
859
|
|
|
@@ -850,9 +891,10 @@ Amp noise under the takes, from a second before them: hiss (white through a cabi
|
|
|
850
891
|
| RX 12 Guitar De-noise, Amp on | 0.0 · 0.0 · ∞ · 40.0 | 0.0 % | 0.0 · 0.0 · ∞ · 35.0 | 0.0 % | 0.0 · 0.0 · ∞ · 37.0 | 0.0 % |
|
|
851
892
|
| RX 12 Spectral De-noise, learned | 7.4 · 1.2 · 48.8 · 40.5 | 2.0 % | 6.3 · 1.1 · 48.0 · 35.8 | 2.5 % | 5.5 · 0.9 · 48.5 · 37.5 | 2.5 % |
|
|
852
893
|
| same, tuned | 10.3 · 2.0 · 47.9 · 41.0 | 3.7 % | 9.5 · 2.3 · 44.5 · 36.6 | 4.2 % | 8.0 · 1.6 · 46.7 · 38.0 | 4.6 % |
|
|
853
|
-
| `desqueak({ squeak: 0, amp: -20 })` | 11.8 · 2.5 · 49.2 · 41.7 | 1.6 % | 9.5 · 2.3 · 43.1 · 36.3 | 1.3 % | 8.6 · 1.9 · 42.6 · 37.3 | 1.9 % |
|
|
894
|
+
| `desqueak({ squeak: 0, amp: -20 })` 0.2.0 | 11.8 · 2.5 · 49.2 · 41.7 | 1.6 % | 9.5 · 2.3 · 43.1 · 36.3 | 1.3 % | 8.6 · 1.9 · 42.6 · 37.3 | 1.9 % |
|
|
895
|
+
| `desqueak({ squeak: 0, amp: -20 })` | 11.8 · 2.5 · 49.2 · 41.7 | 1.6 % | 13.1 · 9.3 · 47.3 · 42.8 | 0.0 % | 9.8 · 3.4 · 44.8 · 39.0 | 1.7 % |
|
|
854
896
|
|
|
855
|
-
Under hiss ours is ahead on every count. Under buzz
|
|
897
|
+
Under hiss ours is ahead on every count, as 0.2.0 was. Under buzz 0.2.0 trailed in the guitar's SDR (43.1 against 44.5 dB): in the takes' quiet passages, where the guitar plays 5–8 dB over the buzz, and on the lines, where a gain cannot part a buzz harmonic from a partial in its bin (about half the guitar's error lay within ±10 Hz of the 60 Hz lines, a third by chance). 0.3.0 subtracts the lines first, each held through the take (`dehum` 0.6.0, `steady`): under buzz the guitar's SDR 43.1 → 47.3 dB, the buzz taken where heard 9.5 → 13.1 dB and over the take 2.3 → 9.3, the SNR 36.3 → 42.8, no sustain moved; ahead of RX on every count. Under buzz and hiss together the SDR 42.6 → 44.8, the noise where heard 8.6 → 9.8 and the SNR 37.3 → 39.0: ahead in noise taken and SNR, 1.9 dB of SDR behind RX (46.7), the hiss and what the lines leave going by gain as before. On the tuning takes, under buzz the SDR 47.6 → 54.0 dB and the buzz taken where heard 10.0 → 23.8; under both 46.6 → 47.2 and 10.7 → 10.2. Subtracting the buzz with dehum as it is for edited material (0.5.1, 0.5.2) measured worse: on the tuning takes under buzz the guitar's SDR 47.6 → 39.3 dB, the buzz 4.3 dB down. Its fit over 2 s took in the guitar's partials near the lines, and a chord's attack, moving many lines at once, read as the hum's jump (12 of them in one 25 s take), so each stretch between was fitted alone.
|
|
856
898
|
|
|
857
899
|
**Use when:** a guitar take with fret squeaks (position shifts, slides on the wound strings), attacks too sharp, or a steady amp hiss and buzz under it.<br>
|
|
858
900
|
**Not for:** a slide you want heard (a squeak and a slide's own comb are the same sound); noise that moves (`omlsa`, `deepfilter`).
|
|
@@ -880,7 +922,7 @@ gate(data, { threshold: -45, attack: 0.005, release: 0.1, hold: 0.05, lookahead:
|
|
|
880
922
|
|
|
881
923
|
### `debreath`
|
|
882
924
|
|
|
883
|
-
Takes the breaths between phrases down by `range`; the room around them, and the speech, stay. A breath is told by what an inhalation is (Ruinskiy & Lavner 2007): unvoiced, longer than a consonant, well under the speech, between phrases, its noise shaped by the open tract. Per frame of [`vad`](#lower-level-building-blocks), on the 0.3–8 kHz band: 10 dB or more over the room (the band's 10th-percentile frame), 12 dB or more under the speech (its 95th), under half of it over 4 kHz (no sibilant). Runs of such frames are a breath when they last 0.15–1 s, their median stands
|
|
925
|
+
Takes the breaths between phrases down by `range`; the room around them, and the speech, stay. A breath is told by what an inhalation is (Ruinskiy & Lavner 2007): unvoiced, longer than a consonant, well under the speech, between phrases, its noise shaped by the open tract. Per frame of [`vad`](#lower-level-building-blocks), on the 0.3–8 kHz band: 10 dB or more over the room (the band's 10th-percentile frame), 12 dB or more under the speech (its 95th), under half of it over 4 kHz (no sibilant), under 80 % of it under 1 kHz (a breath's noise lies over the open tract's formants; a phrase's decaying vowel, its murmur and creak lie under 1 kHz, and a breath they run into is no longer taken for part of them). Runs of such frames are a breath when they last 0.15–1 s, their median stands 14 dB over the room, under 60 % of their energy lies under 1 kHz, at most half their frames are voiced (periodic in the 50–1000 Hz band, where a voice's first harmonics lie: Talkin's normalized cross-correlation 0.6 or more, the band 6 dB over its room, whose hum is periodic too; @audio/vad's periodicity, read up to 4 kHz, took a breath through narrow resonances or a codec's bands for voiced), and a room-level frame lies within 0.1 s on either side: a pause, which a word's own consonants have not. The cut works on the band over 300 Hz (split at zero phase: a room's rumble under a breath is the room's), ramps in over `attack` and out over `release` inside the breath, and never takes it under the room's level in that band, where it would leave a hole. `room` turns down what is neither speech nor breath, as 0.2 did. The whole clip is read at once (streaming: false).
|
|
884
926
|
|
|
885
927
|
```js
|
|
886
928
|
debreath(data, { fs: 48000 }) // breaths 12 dB down
|
|
@@ -900,11 +942,12 @@ Against iZotope RX 12 Breath Control, `node bench/rx/debreath.mjs` in [audio](ht
|
|
|
900
942
|
|---|---:|---:|---:|---:|---:|
|
|
901
943
|
| RX 12 defaults (Gain 0 dB) | 0.0 dB · 0 % | 0.0 % | 0.0 dB | 0.0 dB · 0 % | 0.0 % |
|
|
902
944
|
| RX 12 defaults, Gain −12 dB | −9.7 dB · 81 % | 0.69 % | 0.0 dB | −6.6 dB · 52 % | 0.59 % |
|
|
903
|
-
| RX 12 tuned |
|
|
945
|
+
| RX 12 tuned | −12.0 dB · 88 % | 1.96 % | −0.1 dB | −12.0 dB · 73 % | 0.99 % |
|
|
904
946
|
| 0.2.1 | 0.0 dB · 11 % | 0.11 % | 0.0 dB | 0.0 dB · 20 % | 0.19 % |
|
|
905
|
-
|
|
|
947
|
+
| 0.3.0 | −10.9 dB · 78 % | 0.49 % | 0.0 dB | −11.2 dB · 78 % | 0.58 % |
|
|
948
|
+
| **0.4.0** | −11.2 dB · **92 %** | 0.76 % | 0.0 dB | −11.5 dB · **88 %** | **0.49 %** |
|
|
906
949
|
|
|
907
|
-
0.2.1 turned down what the VAD did not call speech; its speech reaches 0.3 s from a vowel across gaps of 0.15 s, which holds most breaths. Music and singing (Vibe Ace, Brahms, the Nutcracker, a trumpet, four VocalSet
|
|
950
|
+
0.3.0 missed 20 of the 90 narration breaths. Eight ran into the phrase before or after them through its quiet frames (a vowel's decay, an onset's murmur), which passed for breath by level: bridged into one run, it had no pause on that side. Six, in one narration whose codec left a breath as a few narrow bands, were read as voiced by @audio/vad's periodicity, which looks up to 4 kHz. 0.4.0 catches the six and six of the eight; it asks a frame's shape too (under 80 % of its 0.3–8 kHz energy under 1 kHz) and reads voicing where a voice's first harmonics are, under 1 kHz; the median a run needs over the room went from 15 to 14 dB, chosen on the tuning narrations (one breath more there, no speech frame more). On the tuning sets, breaths caught 51 → 54 of 55 and 19 → 20 of 26, speech frames touched 0.23 → 0.15 % and 0.21 → 0.37 %. On the test narrations it touches more speech than 0.3.0 (0.76 % against 0.49 %, still under RX's 1.96 %): most of the new spans are noise between phrases, inhalations ending at the next phrase's onset or holding a few frames periodic under 1 kHz, which the measure's 0.1 s word edges count as speech; one is a phrase's creaky end. 0.2.1 turned down what the VAD did not call speech; its speech reaches 0.3 s from a vowel across gaps of 0.15 s, which holds most breaths. Music and singing (Vibe Ace, Brahms, the Nutcracker, a trumpet, four VocalSet arpeggios), frames within 30 dB of the loudest turned down by over 3 dB: none (0.3.0, counted the same way: 1.8 % of the trumpet's; 0.2.1, as counted then: Brahms 8.8 %, Vibe Ace 2.2 %). VoiceBank+DEMAND noisy test speech, every fourth utterance, frames within 30 dB of the clean utterance's loudest turned down by over 3 dB: 0.07 % (0.3.0: 0.05 %; 0.2.1, as counted then: 5.0 %).
|
|
908
951
|
|
|
909
952
|
**Use when:** breaths between phrases on a voiceover, a podcast, a narration.<br>
|
|
910
953
|
**Not for:** a breath that runs into a word (no pause on either side); hiss or room tone in pauses: `room`, `gate`, `omlsa`.
|
|
@@ -1102,7 +1145,7 @@ Frames follow the rate: the power of two nearest 32 ms, the papers' frame, where
|
|
|
1102
1145
|
|
|
1103
1146
|
STOI holds or rises for `wiener` and `specsub` and falls 0.001–0.002 for `omlsa`; SI-SDR falls 0.3–0.7 dB for `omlsa` and `wiener` at 22.05 and 44.1 kHz, and 0.8–1.2 dB for `specsub`, which alone prefers the longer frame there (so it did on the training subset).
|
|
1104
1147
|
|
|
1105
|
-
Reverberant speech (`vbreverb`): a quarter of the clean test utterances, 206, through 130 rooms of the MIT IR Survey (Traer & McDermott 2016), scored against the dry takes, DNSMOS at the input's loudness. `dereverb` 0.4 → 0.5; the rooms with a tail to hear, and iZotope RX 12, are in [`dereverb`](#dereverb)'s own section:
|
|
1148
|
+
Reverberant speech (`vbreverb`): a quarter of the clean test utterances, 206, through 130 rooms of the MIT IR Survey (Traer & McDermott 2016), scored against the dry takes, DNSMOS at the input's loudness. `dereverb` 0.4 → 0.5 (0.6 the same but for one utterance, below); the rooms with a tail to hear, and iZotope RX 12, are in [`dereverb`](#dereverb)'s own section:
|
|
1106
1149
|
|
|
1107
1150
|
| | PESQ | STOI | SI-SDR dB | SIG | BAK | OVRL |
|
|
1108
1151
|
|---|---|---|---|---|---|---|
|
|
@@ -1111,7 +1154,7 @@ Reverberant speech (`vbreverb`): a quarter of the clean test utterances, 206, th
|
|
|
1111
1154
|
| dry takes in (`vbreverb-dry`) | 4.64 | 1.000 | ∞ | 3.51 | 4.04 | 3.22 |
|
|
1112
1155
|
| dry, `dereverb` | 4.63 → **4.64** | 1.000 → 1.000 | ∞ (94 → **100 %** untouched) | 3.51 → 3.51 | 4.04 → 4.04 | 3.22 → 3.22 |
|
|
1113
1156
|
|
|
1114
|
-
On the training utterances (`vbreverb-train`, 126): PESQ 2.36 → 2.64 → 2.61 (input, 0.4, 0.5). The constants were chosen on `scripts/dereverb.py train`: VoiceBank training speakers in the even-numbered rooms, 1.8–4.5 s utterances and 25–41 s takes. Of the 206 reverberant test utterances 18 % (0.4) and 25 % (0.5) come out untouched: rooms with little tail, which the dry and diffuse checks pass.
|
|
1157
|
+
On the training utterances (`vbreverb-train`, 126): PESQ 2.36 → 2.64 → 2.61 (input, 0.4, 0.5). The constants were chosen on `scripts/dereverb.py train`: VoiceBank training speakers in the even-numbered rooms, 1.8–4.5 s utterances and 25–41 s takes. Of the 206 reverberant test utterances 18 % (0.4) and 25 % (0.5) come out untouched: rooms with little tail, which the dry and diffuse checks pass; 0.6 passes one more, p257_291, whose speaker holds a vowel's pitch as a singer holds a note (the music check), PESQ 2.606 → 2.604, the other 205 bit for bit.
|
|
1115
1158
|
|
|
1116
1159
|
|
|
1117
1160
|
## Demo
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@audio/denoise",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.11.0",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "Noise reduction and restoration: gate, spectral subtraction, Wiener, OM-LSA, dehum, declick, decrackle, declip, dewind, deplosive, deesser, debreath, dereverb, debleed, dewow, desqueak",
|
|
6
6
|
"sideEffects": false,
|
|
@@ -55,15 +55,15 @@
|
|
|
55
55
|
"@audio/denoise-spectral": "^0.3.0",
|
|
56
56
|
"@audio/denoise-wiener": "^0.4.0",
|
|
57
57
|
"@audio/denoise-omlsa": "^0.6.0",
|
|
58
|
-
"@audio/denoise-dehum": "^0.
|
|
58
|
+
"@audio/denoise-dehum": "^0.7.0",
|
|
59
59
|
"@audio/denoise-declick": "^0.4.0",
|
|
60
|
-
"@audio/denoise-decrackle": "^0.
|
|
61
|
-
"@audio/denoise-declip": "^0.
|
|
60
|
+
"@audio/denoise-decrackle": "^0.4.0",
|
|
61
|
+
"@audio/denoise-declip": "^0.5.0",
|
|
62
62
|
"@audio/denoise-dewind": "^0.4.0",
|
|
63
63
|
"@audio/denoise-deplosive": "^0.3.0",
|
|
64
|
-
"@audio/denoise-debreath": "^0.
|
|
65
|
-
"@audio/denoise-dereverb": "^0.
|
|
66
|
-
"@audio/denoise-detect": "^0.
|
|
64
|
+
"@audio/denoise-debreath": "^0.4.0",
|
|
65
|
+
"@audio/denoise-dereverb": "^0.6.0",
|
|
66
|
+
"@audio/denoise-detect": "^0.7.0",
|
|
67
67
|
"@audio/dynamics-gate": "^0.1.4",
|
|
68
68
|
"@audio/denoise-repair": "^0.4.0",
|
|
69
69
|
"@audio/quality": "^1.0.0",
|
|
@@ -71,7 +71,7 @@
|
|
|
71
71
|
"@audio/denoise-desilence": "^0.2.1",
|
|
72
72
|
"@audio/denoise-debleed": "^0.3.0",
|
|
73
73
|
"@audio/denoise-dewow": "^0.3.0",
|
|
74
|
-
"@audio/denoise-desqueak": "^0.
|
|
74
|
+
"@audio/denoise-desqueak": "^0.3.0"
|
|
75
75
|
},
|
|
76
76
|
"devDependencies": {
|
|
77
77
|
"audio-lena": "^3.0.0",
|