@audio/denoise 0.4.2 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +311 -85
- package/package.json +17 -17
package/README.md
CHANGED
|
@@ -8,13 +8,13 @@ Single-pass noise reduction. 13 specialised methods + an auto-classifier.
|
|
|
8
8
|
|---|---|---|---|---|---|
|
|
9
9
|
| [denoise](#denoise) | meta | auto | — | varies | "just clean it" |
|
|
10
10
|
| [gate](#gate) | time | silence | ★ | very low | hard cut at threshold |
|
|
11
|
-
| [dehum](#dehum) | time | mains hum | ★★★★ |
|
|
11
|
+
| [dehum](#dehum) | time | mains hum | ★★★★ | low | 50/60 Hz + harmonics |
|
|
12
12
|
| [specsub](#specsub) | freq | broadband stationary | ★★ | medium | baseline |
|
|
13
13
|
| [wiener](#wiener) | freq | broadband stationary | ★★★ | medium | general broadband |
|
|
14
14
|
| [omlsa](#omlsa) | freq | broadband non-stationary | ★★★★ | high | speech in changing noise |
|
|
15
|
-
| [declick](#declick) | time | impulses | ★★★★ | medium | vinyl
|
|
15
|
+
| [declick](#declick) | time | impulses | ★★★★ | medium | vinyl ticks, edit clicks, mouth clicks |
|
|
16
16
|
| [decrackle](#decrackle) | time | dense impulses | ★★★ | medium | shellac crackle |
|
|
17
|
-
| [declip](#declip) | time | hard clipping |
|
|
17
|
+
| [declip](#declip) | time | hard clipping | ★★★★ | medium | clipped recordings, rails found by itself |
|
|
18
18
|
| [dewind](#dewind) | time | LF rumble | ★★★ | very low | wind, handling noise |
|
|
19
19
|
| [deplosive](#deplosive) | time | LF bursts | ★★★ | low | mic plosives (p, b) |
|
|
20
20
|
| [deesser](#deesser) | time | sibilance | ★★★★ | low | voice (s, sh) |
|
|
@@ -53,49 +53,99 @@ write() // → flush rema
|
|
|
53
53
|
|
|
54
54
|
## `denoise`
|
|
55
55
|
|
|
56
|
-
|
|
56
|
+
Finds the defect a recording carries and routes it to the method for it. Each route needs evidence; with none, nothing is removed and the sound comes back as it was (`'none'`).
|
|
57
57
|
|
|
58
58
|
```js
|
|
59
|
-
denoise(data) // → cleaned Float32Array
|
|
59
|
+
denoise(data) // → cleaned Float32Array, or a copy: nothing to remove
|
|
60
60
|
denoise(data, { returnPlan: true }) // → { out, plan }
|
|
61
61
|
denoise(data, { force: 'wiener' }) // skip classifier
|
|
62
|
+
classify(data, fs) // → { method, scores }, the plan alone
|
|
62
63
|
```
|
|
63
64
|
|
|
64
65
|
| Param | Default | |
|
|
65
66
|
|---|---|---|
|
|
66
67
|
| `fs` | `44100` | Sample rate |
|
|
67
|
-
| `force` | — | One of `'dehum' \| 'declick' \| 'dewind' \| 'deesser' \| 'dereverb' \| 'omlsa' \| 'wiener'` |
|
|
68
|
+
| `force` | — | One of `'none' \| 'dehum' \| 'declick' \| 'dewind' \| 'deesser' \| 'dereverb' \| 'omlsa' \| 'wiener'` |
|
|
68
69
|
| `returnPlan` | `false` | Return `{ out, plan }` with classifier scores + chosen method |
|
|
69
70
|
|
|
70
|
-
**Routing
|
|
71
|
-
1.
|
|
72
|
-
2. impulses
|
|
73
|
-
3. sibilance
|
|
74
|
-
4.
|
|
75
|
-
5.
|
|
76
|
-
6. otherwise
|
|
71
|
+
**Routing, in priority order** (one STFT sweep, frames ≈ 46 ms, hop ¼; over 6 min, 16 spans of it spread across):
|
|
72
|
+
1. **hum → `dehum`**: dehum's own measurement finds a mains series, and the lines it finds, A-weighted (IEC 61672-1), lie within 50 dB of the program's A-weighted level. A lone 60 Hz line 30 dB under a voice is there, and the ear hears 60 Hz 26 dB less than 1 kHz.
|
|
73
|
+
2. **clicks → `declick`**: over 1 a second (`CLICK_RATE`) of impulses standing 32× out of the AR(30) error's local scale (the median of its 1.5 ms block RMS over ±12 ms, as declick judges), alone: no like of half its size 2.5–40 ms either side (a voice's pulses, creak down to 25 Hz), and no sound 10 dB louder after it than before (a plosive's burst, a note's attack).
|
|
74
|
+
3. **sibilance → `deesser`**: 5–9 kHz over 0.2–2 kHz power over 8.
|
|
75
|
+
4. **wind → `dewind`**: in over a tenth of the 0.15 s blocks, the low end (< 200 Hz) within 20 dB of the program, 6 dB over the 300–2000 Hz band, and without lines. Wind is turbulence at the microphone, without a period (Nelke & Vary, IWAENC 2014); a bass line or a voice's low end repeats at its pitch.
|
|
76
|
+
5. **noise bed → `wiener` or `omlsa`**: a floor within 25 dB of the program (`BED_SNR`; the program: frames within 40 dB of the loudest), shown in its *pauses* (runs of 0.15 s or more within 6 dB of the frame level's 10th percentile, flat and without lines in the bands from 300 Hz; the bed: their median level) or in *steady bands* (a band whose quieter half varies no more than twice what Gaussian noise over its bins would, without lines). A line is a band's fine structure (log power less its ±4-bin mean) correlated with the frame N back, which shares no sample: a partial holds its bins, noise draws them anew. Steady bands → `wiener`; a bed seen in the pauses alone wanders → `omlsa`.
|
|
77
|
+
6. otherwise **`none`**.
|
|
78
|
+
|
|
79
|
+
0.3 had no `none`: every recording went somewhere. Its floor statistic (the spread of a rolling-minimum frame energy) read a program's own dynamics as a wandering noise bed, so clean speech and music went to `omlsa`; a bass-heavy mix went to `dewind` (low over mid power above 3); clean VoiceBank went to `declick`, its impulse count firing on lip smacks and plosive bursts. A blind SNR estimate can't stand in for the bed, each assuming a speech-like program: on the training material below, clean music read (median) 5.4 dB by WADA-SNR (Kim & Stern, Interspeech 2008), 15.4 dB by a NIST STNR-like frame histogram and 7.0 dB by minimum statistics (Martin 2001), under the noisy VoiceBank takes' 13.2, 22.3 and 13.0. A bed has to show itself as noise, where the program pauses or never reaches.
|
|
80
|
+
|
|
81
|
+
`node scripts/detect.js tune|test` routes labelled material and prints what each class went to (sources and mixtures in the script); the share routed right, 0.3.2 → 0.4.0, thresholds chosen on `tune` (VoiceBank+DEMAND's training subset, Spoken Wikipedia narrations, repair/ music, VocalSet singers m1 f2 m2 f1, even Slakh mixes, MUSDB18 train), `test` run once (the VoiceBank+DEMAND test set, ten other narrations, other singers and tracks):
|
|
82
|
+
|
|
83
|
+
| material | wanted | tune | test |
|
|
84
|
+
|---|---|---:|---:|
|
|
85
|
+
| clean speech, VoiceBank (84 · 138) | `none` | 0 → 93% | 0 → 96% |
|
|
86
|
+
| narrations (20 · 20) | `none` | 0 → 90% | 0 → 85% |
|
|
87
|
+
| clean music (62 · 56) | `none` | 0 → 98% | 0 → 93% |
|
|
88
|
+
| VoiceBank+DEMAND noisy (84 · 138) | a reducer | 86 → 73% | 92 → 57% |
|
|
89
|
+
| speech + white, pink noise 0–20 dB under | a reducer | 100 → 95% | 99 → 100% |
|
|
90
|
+
| speech + babble 0–20 dB under | a reducer | 88 → 76% | 83 → 80% |
|
|
91
|
+
| music + white noise 10–20 dB under | a reducer | 97 → 92% | 100 → 89% |
|
|
92
|
+
| music + pink noise 10–20 dB under | a reducer | 97 → 68% | 100 → 68% |
|
|
93
|
+
| speech, music + mains hum 20 dB under | `dehum` | 57 → 83% | 69 → 81% |
|
|
94
|
+
| speech, music + clicks at 5× | `declick` | 100 → 94% | 99 → 90% |
|
|
95
|
+
| speech + wind gusts 0–10 dB under | `dewind` | 4 → 93% | 0 → 89% |
|
|
96
|
+
| music + wind gusts 0–10 dB under | `dewind` | 17 → 38% | 24 → 50% |
|
|
97
|
+
| all scored | | 66.4 → 84.6% | 68.8 → 84.9% |
|
|
98
|
+
|
|
99
|
+
Material with nothing to remove (tune 166, test 214): 0.3.2 processed all of it (median error −22.7 and −26.0 dB re the input); now 157 and 202 come back untouched. Of the 9 tune items processed, 5 VoiceBank takes go to `declick`, and they hold lip smacks 32–124× out of the error; a VoiceBank take and a metal track to `dehum`, each with a 50 Hz line 4–8 dB under its mean power; both excerpts of one narration to `omlsa`, its floor 21–23 dB under the voice, as audio-lena's is (22 dB). On test: `declick` 5, `omlsa` 2, `wiener` 2, `dewind` 2, `dehum` 1. Missed: noise holding a line through the pauses (an engine, a fan) unless it is steady elsewhere, so most of the test set's bus noise, and its office and living-room noise at 12.5–17.5 dB, stay; pink noise under a dense mix, masked where its power lies; a steady low rumble (traffic, a car) goes to `dewind` where its low end outweighs the mid band. ~10–17 ms per second of sound.
|
|
100
|
+
|
|
101
|
+
dereverb has no reliable single-pass signature, so auto-mode never selects it — reach it explicitly via `denoise(data, { force: 'dereverb' })` or `dereverb()`.
|
|
77
102
|
|
|
78
103
|
|
|
79
104
|
## Tonal & narrowband
|
|
80
105
|
|
|
81
106
|
### `dehum`
|
|
82
107
|
|
|
83
|
-
Measures the hum, then
|
|
108
|
+
Measures the hum, then subtracts it: each harmonic of the mains frequency is estimated as a slowly varying sinusoid and taken out, so what goes with the hum is the program within a fraction of a hertz of each line, and less of it where the program is loud there. Without hum the audio comes back untouched. Notches (0.2.0) took whatever their band held, a band that grows with the harmonic (Q 30: 1.7 Hz at 50 Hz, 33 Hz at 1 kHz), an orchestra's G2 two hertz under the 100 Hz line among it; and as the mains frequency wanders (±0.02–0.05 Hz, h times that at harmonic h) narrow ones miss the upper harmonics.
|
|
84
109
|
|
|
85
110
|
```js
|
|
86
111
|
dehum(data, { fs }) // 50 or 60 Hz, measured; nothing without hum
|
|
87
112
|
dehum(data, { fs, freq: 60 }) // the 60 Hz series, its exact frequency measured
|
|
88
|
-
dehum(data, { fs, freq: 60, harmonics: 4 }) //
|
|
113
|
+
dehum(data, { fs, freq: 60, harmonics: 4 }) // remove 60–240 Hz as told
|
|
89
114
|
measure(data, fs) // → { f0, harmonics } or null (import from @audio/denoise-dehum)
|
|
90
115
|
```
|
|
91
116
|
|
|
92
117
|
| Param | Default | |
|
|
93
118
|
|---|---|---|
|
|
94
119
|
| `freq` | measured | Fundamental, Hz. Omitted: the 50 or 60 Hz series, whichever stands out. Given: its exact frequency within ±0.4 % (±`drift` Hz with `adaptive`) |
|
|
95
|
-
| `harmonics` |
|
|
96
|
-
|
|
120
|
+
| `harmonics` | to 1 kHz | Remove h = 1…`harmonics`. Omitted: every harmonic up to 1 kHz |
|
|
121
|
+
|
|
122
|
+
The measurement is one Fourier transform over the signal (its first 80–90 s, taken down to 3 kHz), where a hum line, steady for minutes, gathers into a peak 1/T wide while speech spreads. Hum is there when the fundamental stands out by 20 dB over the median of the ±8 Hz around it, or two of the first six harmonics by 15 dB (searched within ±0.4 %, the mains tolerance), each 6 dB over any other peak within 3 Hz, all of them harmonics of one fundamental (p<sub>h</sub>/h within 0.01 Hz). A bar of music repeated exactly is a comb of lines 1/bar apart (every 2 Hz at 120 bpm), lines at 50 and 60 Hz among them: none stands alone. On 164 music clips (MUSDB18 excerpts, BabySlakh) this finds hum in one, which has it (0.2.0: in 19); on 504 clean and 504 noisy VoiceBank training utterances, in one, a steady 49.86 Hz tone at the speech's level; with hum 20 dB under the speech, in 95 % (50 Hz) and 92 % (60 Hz) of them, as before.
|
|
123
|
+
|
|
124
|
+
Then the signal is cut into Hann frames four mains cycles long, two apart, and each harmonic's phasor taken per frame; the window's zeros fall on the other harmonics. Each harmonic's phasors are fitted over 2 s by weighted local-linear least squares (normalized convolution, Knutsson & Westin 1993), each frame weighted by the inverse of the program's power around the line there, so a passing voice or note is bridged from the frames around it rather than averaged in. The mains phase is tracked from the fitted phasors' turn, harmonics combined by h²·SNR (Hajj-Ahmad, Garg & Wu 2013) over 8 s, and the sinusoids are subtracted along it (harmonic subtraction, as power-line interference is taken out of ECG: Levkov et al. 2005). A cut in the recording jumps the hum's phase: within a second of it the fit blends the two phases and takes the hum ~10 dB down, not ~40. A note within ~0.5 Hz of a line for seconds is taken for hum. Whole clip (`streaming: false`), a second at least to measure; shorter, it removes the harmonics of a given `freq` as told and passes the audio through without one.
|
|
125
|
+
|
|
126
|
+
Measured by `node scripts/dehum.js`: the hum (12 harmonics at −6 dB per octave, levels drifting ±10 %, f0 0.05 Hz off nominal and wandering) and the program told apart by phase inversion (Hagerman & Olofsson 2004). 50 Hz hum 20 dB under the program, wandering ±0.02 Hz; program SDR per band where the band holds 1 % of the program:
|
|
97
127
|
|
|
98
|
-
|
|
128
|
+
| material | hum down, dB | program SDR, dB | <100 Hz | 100–300 | 300–1k | 1–4k | >4k |
|
|
129
|
+
|---|---:|---:|---:|---:|---:|---:|---:|
|
|
130
|
+
| speech | 45.5 | 38.5 | 28.9 | 39.2 | 39.2 | 83.0 | – |
|
|
131
|
+
| narration | 43.5 | 42.7 | – | 38.8 | 42.9 | 64.5 | 73.9 |
|
|
132
|
+
| vibeace | no hum found | | | | | | |
|
|
133
|
+
| brahms | 42.2 | 25.7 | 30.2 | 17.9 | 27.0 | 57.8 | 50.2 |
|
|
134
|
+
| nutcracker | 41.6 | 27.6 | 23.5 | 30.7 | 26.8 | 63.6 | – |
|
|
135
|
+
| trumpet | 51.0 | 47.9 | – | – | 42.2 | 82.3 | – |
|
|
136
|
+
|
|
137
|
+
Hum down / program SDR, dB:
|
|
138
|
+
|
|
139
|
+
| material | 60 Hz | 50 Hz, ±0.05 Hz | 50 Hz, 30 dB under |
|
|
140
|
+
|---|---|---|---|
|
|
141
|
+
| speech | 46.3 / 42.1 | 36.6 / 38.6 | 38.7 / 38.6 |
|
|
142
|
+
| narration | 43.6 / 44.0 | 33.8 / 42.8 | 42.3 / 42.9 |
|
|
143
|
+
| vibeace | no hum found | no hum found | no hum found |
|
|
144
|
+
| brahms | 41.7 / 29.0 | 27.8 / 25.7 | 33.9 / 25.7 |
|
|
145
|
+
| nutcracker | 40.7 / 24.3 | 33.2 / 27.9 | no hum found |
|
|
146
|
+
| trumpet | 51.9 / 47.8 | 46.0 / 47.7 | 48.4 / 48.0 |
|
|
147
|
+
|
|
148
|
+
The same mixtures through 0.2.0 (its Q 30 notches at the lines it found, run on hum and program apart): hum 5–15 dB down (26 in Vibe Ace at 60 Hz), program SDR 10–38 dB (speech 22.3, Brahms 13.6, its 100–300 Hz band 5.7). Hum alone, 30 s, ±0.02 Hz: 56 dB down (0.2.0: 26). Clean speech, narration and music: no sample changed. In Vibe Ace the hum's lines stand among the track's own steady ones, none alone, so none is taken for hum (0.2.0 found hum in the clean track); `freq` and `harmonics` remove them as told.
|
|
99
149
|
|
|
100
150
|
**Use when:** mains buzz, ground-loop hum, fixed tonal interference.<br>
|
|
101
151
|
**Not for:** broadband noise (use `wiener`/`omlsa`); shifting tones (use spectral methods).
|
|
@@ -103,7 +153,7 @@ The measurement is one Fourier transform over the signal (its first 80–90 s, t
|
|
|
103
153
|
|
|
104
154
|
### `dewind`
|
|
105
155
|
|
|
106
|
-
|
|
156
|
+
A high-pass that comes in while wind blows and stays out otherwise. Wind is turbulence at the microphone: its energy lies under a few hundred Hz and has no period (Nelke & Vary 2014), where a voice's or an instrument's low end repeats at its pitch. Every 5 ms the energy under 200 Hz, weighed by how aperiodic it is, is set against the 300–2000 Hz band: 1 − r of it is noise, r the normalized autocorrelation's peak at a 2.5–25 ms lag (Talkin 1995; a harmonic H in noise N reads H / (H + N), Boersma 1993). The mid band is floored 20 dB under its peak over the last seconds, so a room's own rumble in a pause is no wind. Once the low band's noise outweighs the mid band, aperiodic for 30 ms, a Butterworth high-pass crossfades in over `attack`, its cutoff rising from `cutoffMin` toward `cutoffMax` with the wind. It goes back out over `release`, or within 5 ms once the low end turns periodic: a voice or a note began. With no wind the output equals the input, sample for sample.
|
|
107
157
|
|
|
108
158
|
```js
|
|
109
159
|
dewind(data, { cutoffMin: 60, cutoffMax: 250 })
|
|
@@ -111,53 +161,105 @@ dewind(data, { cutoffMin: 60, cutoffMax: 250 })
|
|
|
111
161
|
|
|
112
162
|
| Param | Default | |
|
|
113
163
|
|---|---|---|
|
|
114
|
-
| `cutoffMin` | `60` | Hz
|
|
115
|
-
| `cutoffMax` | `250` | Hz
|
|
116
|
-
| `order` | `2` |
|
|
117
|
-
| `
|
|
118
|
-
| `
|
|
164
|
+
| `cutoffMin` | `60` | Hz, the cutoff in light wind |
|
|
165
|
+
| `cutoffMax` | `250` | Hz, the cutoff in strong wind |
|
|
166
|
+
| `order` | `2` | Butterworth sections (each 12 dB/oct), −3 dB at the cutoff |
|
|
167
|
+
| `attack` | `0.05` | s, how fast it comes in |
|
|
168
|
+
| `release` | `0.4` | s, how slowly it goes back out |
|
|
169
|
+
| `blockSize` | 5 ms | Re-estimation interval (samples) |
|
|
170
|
+
|
|
171
|
+
`node scripts/lowend.js dewind` puts clean speech and music through it, then speech with wind added (synthetic: Gaussian noise shaped and gusting as Nelke & Vary measure it, under 100 Hz and ±8 dB at 1 Hz; no recordings of real wind over their clean speech exist here). VoiceBank+DEMAND test utterances (every fourth: p232 male, p257 female), ten Spoken Wikipedia narrations, the music `repair` uses, 0.1.9 → now; voiced frames thinned: the voiced 10 ms frames whose level under 250 Hz fell by more than 3 dB (now mostly the first syllable after a recording's opening room tone, which reads as wind until the voice comes in):
|
|
172
|
+
|
|
173
|
+
| | voiced frames thinned | 20–63 Hz | 63–125 Hz | 125–250 Hz |
|
|
174
|
+
|---|---:|---:|---:|---:|
|
|
175
|
+
| speech, male | 73.5% → 5.1% | −10.5 → −0.2 | −7.1 → 0.0 | −3.4 → 0.0 |
|
|
176
|
+
| speech, female | 45.6% → 8.8% | −23.4 → −1.2 | −2.9 → −0.1 | −1.9 → 0.0 |
|
|
177
|
+
| narrations | 33.4% → 2.4% | −6.6 → −0.1 | −3.2 → −0.1 | −1.1 → 0.0 |
|
|
178
|
+
| audio-lena | 0.2% → 0.0% | −1.6 → 0.0 | −1.0 → 0.0 | −0.1 → 0.0 |
|
|
179
|
+
| Vibe Ace (jazz) | 94.5% → 0.1% | −27.7 → 0.0 | −18.8 → 0.0 | −6.0 → 0.0 |
|
|
180
|
+
| Brahms (strings) | 22.0% → 0.0% | −5.1 → 0.0 | −2.7 → 0.0 | −0.4 → 0.0 |
|
|
181
|
+
| Nutcracker | 12.6% → 0.0% | −6.4 → 0.0 | −2.6 → 0.0 | −0.1 → 0.0 |
|
|
182
|
+
| trumpet | 0.0% → 0.0% | −7.3 → 0.0 | −0.1 → 0.0 | 0.0 → 0.0 |
|
|
183
|
+
| bass line | 100.0% → 0.0% | −32.8 → 0.0 | −18.7 → 0.0 | −5.8 → 0.0 |
|
|
184
|
+
|
|
185
|
+
Wind at a speech-to-wind ratio of +10, 0 and −10 dB, the error to the clean speech taken away (dB):
|
|
119
186
|
|
|
120
|
-
|
|
121
|
-
|
|
187
|
+
| | +10 dB | 0 dB | −10 dB |
|
|
188
|
+
|---|---:|---:|---:|
|
|
189
|
+
| speech, male | −10.9 → 0.1 | −1.2 → 2.4 | 7.7 → 5.7 |
|
|
190
|
+
| speech, female | −11.1 → −0.5 | −1.5 → 2.4 | 7.5 → 5.6 |
|
|
191
|
+
|
|
192
|
+
0.1.9 took any low end that outweighed the mid band for wind, a voice's and a bass line's too, behind a fixed 60 Hz floor whose two Q 0.707 sections were −6 dB at the cutoff. Its larger take of the wind came with as much of the voice: the error to the clean speech grew, by 11 dB at +10 dB and by 1.2–1.5 dB at 0 dB. At −10 dB, where the wind dwarfs the voice, a blanket cut still takes more (7.5–7.7 dB against 5.6–5.7). The wind under a voiced low end is what a time-domain cutoff can't take without the voice; it is now left there.
|
|
193
|
+
|
|
194
|
+
**Use when:** intermittent wind buffeting — the adaptive cutoff opens on gusts and closes between them (measured: beats `wiener` on gusty wind at ~1/10 the CPU).<br>
|
|
195
|
+
**Not for:** continuous rumble under speech — a time-domain cutoff can't separate overlapping spectra; use `wiener`/`omlsa` there (measured ~9 dB vs ~1 dB SNR gain). Nor a lone thump (`deplosive`), or a steady low tone or hum, which has a period (`dehum`, `highpass`). An LPC-null post-filter was evaluated and rejected: voiced speech is as AR-predictable as wind, so nulling wind poles whitens vowels too (LSD improves, SNR and speech level degrade).
|
|
122
196
|
|
|
123
197
|
|
|
124
198
|
### `deplosive`
|
|
125
199
|
|
|
126
|
-
|
|
200
|
+
Ducks the band under `crossover` for a close-mic `p` or `b`: a pressure pulse that rises out of nothing, outweighs the band above it and has no period, where a voice's or a bass note's low end repeats at its pitch. A duck begins when the low band's 3 ms envelope jumps over 3× its 30 ms average while standing over `triggerRatio`× the high band, and holds while that lasts and the low band stays aperiodic: its normalized autocorrelation's peak at a 2.5–25 ms lag (Talkin 1995) under ½, where a harmonic part would lead (Boersma 1993). The high band is floored 20 dB under its peak over the last seconds, so a room's own rumble in a pause is no pop. The duck crossfades toward the high-passed sound, x − (1 − g)·(x − HP(x)), so it never lifts a band; with no pop the output equals the input sample for sample.
|
|
127
201
|
|
|
128
202
|
```js
|
|
129
|
-
deplosive(data, { triggerRatio: 4, attack: 0.
|
|
203
|
+
deplosive(data, { triggerRatio: 4, attack: 0.002, release: 0.03 })
|
|
130
204
|
```
|
|
131
205
|
|
|
132
206
|
| Param | Default | |
|
|
133
207
|
|---|---|---|
|
|
134
|
-
| `triggerRatio` | `4` | LF/high
|
|
208
|
+
| `triggerRatio` | `4` | LF/high envelope ratio a pop must exceed |
|
|
135
209
|
| `attenuation` | `-18` | dB cut on the LF band when triggered |
|
|
136
210
|
| `crossover` | `200` | Hz — LF/high split point |
|
|
137
|
-
| `attack` | `0.
|
|
138
|
-
| `release` | `0.
|
|
211
|
+
| `attack` | `0.002` | s |
|
|
212
|
+
| `release` | `0.03` | s |
|
|
213
|
+
|
|
214
|
+
At full duck the response is −24 dB at 60 Hz, −17 at 100, −2.1 at 250 and −0.5 at 400 Hz; 0.1.10's g·LP + (x − LP), x − LP being no high-pass, gave −7.5, −3.1, +1.8 and +1.3 dB, and it ducked any low end over 4× the high band: a voice's, a jazz track's, a bass line's. `node scripts/lowend.js deplosive` puts clean speech and music through it, then speech with pops added (synthetic: half-sine pressure pulses of 20–60 ms under 150 Hz before each word that follows a pause; no recordings of real pops over their clean speech exist here). VoiceBank+DEMAND test utterances, Spoken Wikipedia narrations, the music `repair` uses, 0.1.10 → now; voiced frames thinned: the voiced 10 ms frames whose level under 250 Hz fell by more than 3 dB:
|
|
215
|
+
|
|
216
|
+
| | voiced frames thinned | 20–63 Hz | 63–125 Hz | 125–250 Hz |
|
|
217
|
+
|---|---:|---:|---:|---:|
|
|
218
|
+
| speech, male | 8.9% → 0.7% | −1.2 → −0.1 | −0.5 → 0.0 | −0.1 → 0.0 |
|
|
219
|
+
| speech, female | 16.3% → 2.2% | −3.4 → −0.2 | −0.3 → 0.0 | 0.2 → 0.0 |
|
|
220
|
+
| narrations | 2.4% → 0.0% | −0.7 → −0.2 | −0.2 → 0.0 | 0.0 → 0.0 |
|
|
221
|
+
| audio-lena | 0.0% → 0.0% | 0.0 → 0.0 | 0.0 → 0.0 | 0.0 → 0.0 |
|
|
222
|
+
| Vibe Ace (jazz) | 41.3% → 0.4% | −4.3 → 0.0 | −2.9 → 0.0 | −0.6 → 0.0 |
|
|
223
|
+
| Brahms (strings) | 0.4% → 0.1% | 0.0 → 0.0 | 0.0 → 0.0 | 0.0 → 0.0 |
|
|
224
|
+
| Nutcracker | 3.6% → 0.1% | 0.0 → 0.0 | −0.1 → 0.0 | 0.0 → 0.0 |
|
|
225
|
+
| trumpet | 0.0% → 0.0% | 0.0 → 0.0 | 0.0 → 0.0 | 0.0 → 0.0 |
|
|
226
|
+
| bass line | 53.1% → 0.0% | −8.3 → −0.1 | −4.6 → 0.0 | 0.2 → 0.0 |
|
|
227
|
+
|
|
228
|
+
Pops peaking at 0.5, 1 and 2× the utterance's peak, the error to the clean speech taken away over each pop and the 150 ms after it (dB):
|
|
229
|
+
|
|
230
|
+
| | 0.5× | 1× | 2× |
|
|
231
|
+
|---|---:|---:|---:|
|
|
232
|
+
| speech, male | 11.5 → 11.0 | 13.6 → 13.2 | 14.4 → 14.5 |
|
|
233
|
+
| speech, female | 13.9 → 12.6 | 14.7 → 14.2 | 14.9 → 15.0 |
|
|
234
|
+
|
|
235
|
+
0.1.10 often met a pop with its duck already down on the pause's rumble before the word; it now waits for the pop, and takes the first few ms of it a little less.
|
|
139
236
|
|
|
140
|
-
**Use when:** mic plosives (`p`, `b`, `t`) producing low-frequency thuds
|
|
237
|
+
**Use when:** mic plosives (`p`, `b`, `t`) producing low-frequency thuds.<br>
|
|
238
|
+
**Not for:** a kick drum or a bass note struck hard: so sudden a low end is ducked for its first few ms, before its period shows.
|
|
141
239
|
|
|
142
240
|
|
|
143
241
|
### `deesser`
|
|
144
242
|
|
|
145
|
-
|
|
243
|
+
Sibilance reduction, [`@audio/dynamics-deesser`](https://github.com/audiojs/dynamics) `mode: 'band'` behind this family's seconds-based options (2026-07 near-dupe merge). An 's' is told by its shape, not its level: the sibilance band at `fc` is measured against the voice body under `fc / 2`, in dB, and how far it rises over `threshold` sets a peaking cut at `fc`, held within `range`.
|
|
146
244
|
|
|
147
245
|
```js
|
|
148
|
-
deesser(data, {
|
|
246
|
+
deesser(data, { fs }) // the kernel's threshold (0 dB) and range (−6 dB)
|
|
247
|
+
deesser(data, { fs, fc: 7500, threshold: -3, range: -8 }) // softer esses too, deeper
|
|
149
248
|
```
|
|
150
249
|
|
|
151
250
|
| Param | Default | |
|
|
152
251
|
|---|---|---|
|
|
153
252
|
| `fc` | `6000` | Sibilance centre (Hz; `freq` still accepted) |
|
|
154
|
-
| `threshold` |
|
|
253
|
+
| `threshold` | kernel's (`0`) | dB of the sibilance band over the voice body where the cut starts; not a level |
|
|
254
|
+
| `range` | kernel's (`-6`) | dB, the deepest cut |
|
|
155
255
|
| `ratio` | `4` | Compression ratio above threshold |
|
|
156
256
|
| `attack` | `0.001` | s — how fast the cut engages |
|
|
157
257
|
| `release` | `0.05` | s — how slowly it recovers |
|
|
158
258
|
| `Q` | `1.4` | Peaking EQ Q |
|
|
159
259
|
| `block` | `64` | Coefficient update interval (samples) |
|
|
160
260
|
|
|
261
|
+
`threshold` was a level, −30 dBFS here; under the kernel's 0.3 it is the band over the body, and −30 cut a ride cymbal 12 dB under a mix by a median 2.7 dB.
|
|
262
|
+
|
|
161
263
|
**Use when:** voice post-production with hot s/sh; vocal bus de-essing.
|
|
162
264
|
|
|
163
265
|
|
|
@@ -165,7 +267,7 @@ deesser(data, { fc: 6500, threshold: -28, ratio: 4 })
|
|
|
165
267
|
|
|
166
268
|
### `specsub`
|
|
167
269
|
|
|
168
|
-
Power spectral subtraction with over-subtraction and a spectral floor (Berouti, Schwartz & Makhoul 1979): |Ŝ|² = |Y|² − α·N̂ where that stays above β·N̂, else β·N̂. Over-subtraction takes out the noise's peaks that plain subtraction leaves as musical tones; the floor, a fraction of the noise estimate, fills the valleys with a steady bed that masks what is left. α follows the frame's SNR (4 − 3/20·SNR: 4.75 at −5 dB down to 1 at 20 dB) unless fixed. The noise PSD is tracked by minimum statistics (Martin 2001) over a 1.5 s window, in batch and stream alike, unless a profile or a noise-only stretch is given.
|
|
270
|
+
Power spectral subtraction with over-subtraction and a spectral floor (Berouti, Schwartz & Makhoul 1979): |Ŝ|² = |Y|² − α·N̂ where that stays above β·N̂, else β·N̂. Over-subtraction takes out the noise's peaks that plain subtraction leaves as musical tones; the floor, a fraction of the noise estimate, fills the valleys with a steady bed that masks what is left. α follows the frame's SNR (4 − 3/20·SNR: 4.75 at −5 dB down to 1 at 20 dB) unless fixed. The noise PSD is tracked by minimum statistics (Martin 2001) over a 1.5 s window, in batch and stream alike, unless a profile or a noise-only stretch is given. `processor(opts)` is the gain as an @audio/stft frame process, for a host running its own frames.
|
|
169
271
|
|
|
170
272
|
```js
|
|
171
273
|
specsub(data, { fs }) // α(SNR), β 0.05, noise tracked
|
|
@@ -188,7 +290,7 @@ specsub(data, { fs, noiseFrames: 6 }) // noise from the
|
|
|
188
290
|
|
|
189
291
|
### `wiener`
|
|
190
292
|
|
|
191
|
-
MMSE log-spectral amplitude (Ephraim & Malah 1985) or Wiener (Scalart & Filho 1996) gain on the decision-directed a priori SNR ξ = α·Â²(l−1)/λ(l−1) + (1−α)·max(γ−1, 0) (Ephraim & Malah 1984, eq. 51). The noise PSD λ is tracked by minimum statistics (Martin 2001) over a 1.5 s window, in batch and stream alike, unless a profile or a noise-only stretch is given.
|
|
293
|
+
MMSE log-spectral amplitude (Ephraim & Malah 1985) or Wiener (Scalart & Filho 1996) gain on the decision-directed a priori SNR ξ = α·Â²(l−1)/λ(l−1) + (1−α)·max(γ−1, 0) (Ephraim & Malah 1984, eq. 51). α = 0.98 is theirs for an 8 ms frame step (§VI: 256 samples at 8 kHz, a new frame every 64) and is rescaled to the actual step as α<sup>Δt/8 ms</sup>, so the a priori SNR's memory holds in seconds; applied per frame until 0.3, it ran 1.8× longer at 48 kHz (10.7 ms steps) than at 44.1 kHz (5.8 ms). The floor ξ<sub>min</sub> stays −15 dB: −25 dB, Cohen's and Loizou's, left more musical noise on steady white and pink noise (log kurtosis ratio 0.98 and 1.49, against 0.53 and 1.01) and cost PESQ and SIG on the training speech (`scripts/speech.mjs`). The noise PSD λ is tracked by minimum statistics (Martin 2001) over a 1.5 s window, in batch and stream alike, unless a profile or a noise-only stretch is given. `processor(opts)` is the gain as an @audio/stft frame process, for a host running its own frames.
|
|
192
294
|
|
|
193
295
|
```js
|
|
194
296
|
wiener(data, { fs }) // LSA gain, noise tracked
|
|
@@ -199,7 +301,7 @@ wiener(data, { fs, noiseFrames: 6 }) // noise from the
|
|
|
199
301
|
| Param | Default | |
|
|
200
302
|
|---|---|---|
|
|
201
303
|
| `rule` | `'mmse-lsa'` | `'wiener'` or `'mmse-lsa'` (log-spectral, less musical noise) |
|
|
202
|
-
| `alpha` | `0.98` | Decision-directed smoothing (alias of `alphaDD`) |
|
|
304
|
+
| `alpha` | `0.98` | Decision-directed smoothing per 8 ms of frame step, rescaled to the actual step (alias of `alphaDD`) |
|
|
203
305
|
| `xiMin` | `10^−1.5` | A priori SNR floor (−15 dB) |
|
|
204
306
|
| `frameSize` | power of two nearest 32 ms | STFT frame: 512 at 16 and 22.05 kHz, 1024 at 44.1, 2048 at 48 (`frame(fs)`) |
|
|
205
307
|
| `hopSize` | `frameSize/4` | OLA hop |
|
|
@@ -211,9 +313,11 @@ wiener(data, { fs, noiseFrames: 6 }) // noise from the
|
|
|
211
313
|
|
|
212
314
|
### `omlsa`
|
|
213
315
|
|
|
214
|
-
Optimally-Modified Log-Spectral Amplitude estimator (Cohen & Berdugo 2001) with IMCRA noise estimation (Cohen 2003). The log-spectral amplitude gain when speech is present, weighed against a floor by the speech presence probability p: `G = G_H1^p · G_min^(1−p)` (eq. 16). The a priori SNR is decision-directed on G<sub>H1</sub> (eq. 18), the a priori speech absence is estimated from its spread over time and neighbouring bins (§4), the noise spectrum comes from IMCRA. As Cohen's own `omlsa.m`, a bin whose speech absence reaches 0.9 counts as noise: noise that happens to peak keeps G<sub>min</sub> and leaves no musical tone, so what remains of the noise is the noise, G<sub>min</sub> quieter. Time constants are set for 8 ms frames and rescaled to the actual frame step (`omlsa.m` rescales by the reciprocal, which lengthens them at 48 kHz).
|
|
316
|
+
Optimally-Modified Log-Spectral Amplitude estimator (Cohen & Berdugo 2001) with IMCRA noise estimation (Cohen 2003). The log-spectral amplitude gain when speech is present, weighed against a floor by the speech presence probability p: `G = max(G_H1, G_min)^p · G_min^(1−p)` (eq. 16 with G<sub>H1</sub> floored at G<sub>min</sub>: as written it took a bin below G<sub>min</sub> wherever G<sub>H1</sub> was under it, to −18 dB at G<sub>min</sub> −12, so `gMin` was not the floor). The a priori SNR is decision-directed on G<sub>H1</sub> (eq. 18), the a priori speech absence is estimated from its spread over time and neighbouring bins (§4), the noise spectrum comes from IMCRA. As Cohen's own `omlsa.m`, a bin whose speech absence reaches 0.9 counts as noise: noise that happens to peak keeps G<sub>min</sub> and leaves no musical tone, so what remains of the noise is the noise, G<sub>min</sub> quieter. Time constants, the decision-directed α among them, are set for 8 ms frames and rescaled to the actual frame step (`omlsa.m` rescales by the reciprocal, which lengthens them at 48 kHz).
|
|
317
|
+
|
|
318
|
+
α sets how soon ξ follows a word's start after a pause, and how much gain a noise peak gets. Until 0.3 it was 0.98 per frame, whatever the frame (0.985 per 8 ms at 48 kHz, 0.972 at 44.1), and a word's second and third frames after a pause lost 10 and 5 dB. It is now the lowest value per 8 ms that leaves steady white and pink noise without musical noise: 0.97 tracking the noise (log kurtosis ratio 0.00 at G<sub>min</sub> −15 and −25 dB; 0.96 left 1.16 and 0.49 at −25, the paper's 0.92 0.99 and 1.81 at −15), 0.95 on a learned noise (`omlsa.m`'s; 0.92 left 0.19 and 0.42). The paper's cap q ≤ q<sub>max</sub> = 0.95 (Table 1) in place of the gate kept a word's second frame after a pause another 3 dB but left musical noise on steady noise (0.24 and 0.46 at α 0.97), so the gate stays. The table under [Speech](#speech) has what each keeps. `scripts/reference.py` holds a numpy version written from the papers; on 16 kHz VoiceBank frames it gives `omlsa.m`'s noise track, speech absence and gains to the last bit (but for near-empty bins, where `omlsa.m`'s absolute 1e-10 floors bind), and test.js holds this code to it.
|
|
215
319
|
|
|
216
|
-
A noise that holds still can be learned instead: `profile`, the noise's PSD (`noiseProfile` of a stretch where it plays alone), or, in the batch form, `noiseFrames`/`profileFrom`/`profileTo` naming that stretch. The noise is then held (`known` of @audio/noise-estimate), and p reads the observation alone: (9) at the a priori SNR of speech ξ<sub>H1</sub> = 15 dB and q = 0.5 (Gerkmann & Hendriks 2012), and 0 where the estimated q reaches 0.9, as before. The decision-directed ξ lags a word's onset; a known noise need not wait for it. On the VoiceBank+DEMAND test set, the noise learned from the half second before each speaker starts and G<sub>min</sub> −12 dB: PESQ 2.
|
|
320
|
+
A noise that holds still can be learned instead: `profile`, the noise's PSD (`noiseProfile` of a stretch where it plays alone), or, in the batch form, `noiseFrames`/`profileFrom`/`profileTo` naming that stretch. The noise is then held (`known` of @audio/noise-estimate), and p reads the observation alone: (9) at the a priori SNR of speech ξ<sub>H1</sub> = 15 dB and q = 0.5 (Gerkmann & Hendriks 2012), and 0 where the estimated q reaches 0.9, as before. The decision-directed ξ lags a word's onset; a known noise need not wait for it. On the VoiceBank+DEMAND test set, the noise learned from the half second before each speaker starts and G<sub>min</sub> −12 dB: PESQ 2.45, STOI 0.919, SI-SDR 15.1 dB, OVRL 2.88 (0.2: 2.48, 0.915, 15.2, 2.89), against 2.36, 0.919, 14.0 and 2.84 tracked; on steady noise under speech and music the noise goes exactly G<sub>min</sub> down, with no musical noise from −12 to −20 dB (log kurtosis ratio 0.00) but for the half second after music stops (0.11 at −12 dB and 1.00 at −20, `scripts/broadband.mjs`). `processor(opts)` is the gain as an @audio/stft frame process, for a host running its own frames: audio's `denoise` op learns a print from a range and runs it so.
|
|
217
321
|
|
|
218
322
|
```js
|
|
219
323
|
omlsa(data, { fs })
|
|
@@ -223,8 +327,8 @@ omlsa(data, { fs, profileFrom: 0, profileTo: fs / 2, gMin: -12 }) // the noise
|
|
|
223
327
|
|
|
224
328
|
| Param | Default | |
|
|
225
329
|
|---|---|---|
|
|
226
|
-
| `gMin` | `-15` | dB: what noise-only bins keep (alias `gMinDb`) |
|
|
227
|
-
| `alpha` | `0.
|
|
330
|
+
| `gMin` | `-15` | dB: what noise-only bins keep, the floor (alias `gMinDb`) |
|
|
331
|
+
| `alpha` | `0.97`; `0.95` with a `profile` | Decision-directed smoothing per 8 ms of frame step, rescaled to the actual step (alias of `alphaDD`) |
|
|
228
332
|
| `xiMin` | `10^−2.5` | A priori SNR floor (−25 dB) |
|
|
229
333
|
| `qPrior` | estimated | A fixed a priori speech absence instead of the estimate |
|
|
230
334
|
| `frameSize` | power of two nearest 32 ms | STFT frame: 512 at 16 and 22.05 kHz, 1024 at 44.1, 2048 at 48 (`frame(fs)`) |
|
|
@@ -239,51 +343,94 @@ omlsa(data, { fs, profileFrom: 0, profileTo: fs / 2, gMin: -12 }) // the noise
|
|
|
239
343
|
|
|
240
344
|
### `declick`
|
|
241
345
|
|
|
242
|
-
|
|
346
|
+
Finds each click as an outlier of the AR prediction error and rebuilds it from the sound either side. The error is judged against its own median level over ±12 ms, so a click can't raise the bar it must clear; the model is fitted again with the outliers left out, since a loud click otherwise teaches it its own ringing and only its onset stands out. A click spans where the error stays over 3× that level after its onset; one with a like half its size 2.5–15 ms away is a glottal pulse or a plucked string, and is left alone. Each is rebuilt by least-squares AR interpolation over 46 ms either side ([`lpc`](https://github.com/audiojs/denoise/tree/main/packages/lpc)'s `arBridge`, Janssen 1986 / Godsill-Rayner 1998). Samples no click reaches come back bit-exact.
|
|
243
347
|
|
|
244
348
|
```js
|
|
245
|
-
declick(data, {
|
|
349
|
+
declick(data, { fs: 44100 }) // everywhere
|
|
350
|
+
declick(data, { fs: 44100, regions: [{ at: 12.31, duration: 0.02 }] }) // the clicks seen there
|
|
246
351
|
```
|
|
247
352
|
|
|
248
353
|
| Param | Default | |
|
|
249
354
|
|---|---|---|
|
|
250
|
-
| `threshold` | `
|
|
251
|
-
| `
|
|
252
|
-
| `
|
|
253
|
-
| `
|
|
355
|
+
| `threshold` | `8` | how far over the error's local level a click stands, multiples |
|
|
356
|
+
| `longest` | `6` | longest click rebuilt, ms; longer is taken for real sound |
|
|
357
|
+
| `order` | `32` | AR order of the detection |
|
|
358
|
+
| `regions` | — | `[{ at, duration }]`, s: look only there, and take nothing there for a pulse or too long |
|
|
359
|
+
| `fs` | `44100` | sample rate, Hz |
|
|
360
|
+
|
|
361
|
+
`node scripts/declick.js` adds clicks to speech (audio-lena) and music (librosa/data, as `repair`), one every 0.25–0.45 s at 2, 5 and 15× the sound's RMS around it, and measures how much of each click's error is gone (median dB, over the click and 1 ms either side) and the share gone by 10 dB or more:
|
|
362
|
+
|
|
363
|
+
| click | 2× | 5× | 15× |
|
|
364
|
+
|---|---:|---:|---:|
|
|
365
|
+
| tick (rings at 2–8 kHz, 0.05–0.3 ms) | 13.4 dB · 60% | 17.5 dB · 88% | 25.4 dB · 99% |
|
|
366
|
+
| pop (300–1500 Hz, 0.3–1 ms) | 5.7 dB · 30% | 12.9 dB · 60% | 18.2 dB · 86% |
|
|
367
|
+
| glitch (1–8 samples off) | 15.5 dB · 71% | 20.5 dB · 90% | 29.4 dB · 99% |
|
|
368
|
+
| mouth (1–3 ms of noise) | 9.3 dB · 49% | 17.8 dB · 86% | 25.6 dB · 100% |
|
|
254
369
|
|
|
255
|
-
|
|
256
|
-
|
|
370
|
+
0.1.7 (AR(60) on its own window, σ including the click, 2 samples either side): ticks 5–6 dB, pops 0, mouth clicks 4–6 dB. The clean material through it: speech, Brahms and the trumpet untouched (0.1.7 changed 2295, 123 and 2283 samples, the trumpet to −19 dB), "Vibe Ace" 112 samples at −42 dB. Each click's surroundings (3–20 ms either side) given as `regions`: the same within 0.5 dB; regions over the clean material change nothing. Ticks at 2× are as far as the interpolation reaches: rebuilt at their exact span, they come out the same. 10 s in 0.1 s.
|
|
371
|
+
|
|
372
|
+
**Use when:** vinyl ticks, edit clicks, digital glitches, mouth clicks on a voice; a click seen on a spectrogram, given as a region.<br>
|
|
373
|
+
**Not for:** dense crackle (use `decrackle`); long dropouts (use `repair`).
|
|
257
374
|
|
|
258
375
|
|
|
259
376
|
### `decrackle`
|
|
260
377
|
|
|
261
|
-
|
|
378
|
+
Finds crackle, a worn record's dense small impulses, where the sound departs both from its AR prediction and from its least-squares interpolation from either side, and rebuilds all of it in a 46 ms window at once. The prediction error alone also stands out at the sound's own excitation, a voice's glottal pulses, a reed's or a bow's; the two-sided error (the prediction error through its matched filter, Vaseghi & Rayner 1990) holds those back by √Σa² against its spread and lifts an impulse added to the sound by as much, so crackle is where both stand over `threshold` × their local level: per 1.5 ms block the median |error|, the median of those over ±12 ms, which crackle can't raise until it fills half the samples of half the blocks. All flagged samples of a window are solved together by exact least-squares AR interpolation ([`lpc`](https://github.com/audiojs/denoise/tree/main/packages/lpc)'s `arFill`, Janssen 1986 / Godsill-Rayner 1998), so a neighbouring impulse is never taken for sound. Then it looks again on the rebuilt sound, the models fitted to it, until no new impulse stands out (Vaseghi & Rayner's iteration): with the largest gone, the smaller ones show; a new one must stand out of the input too, so a rebuild's own departures are never taken for crackle. Samples no crackle reaches come back bit-exact.
|
|
262
379
|
|
|
263
380
|
```js
|
|
264
|
-
decrackle(data, {
|
|
381
|
+
decrackle(data, { fs: 44100 })
|
|
265
382
|
```
|
|
266
383
|
|
|
267
|
-
|
|
384
|
+
| Param | Default | |
|
|
385
|
+
|---|---|---|
|
|
386
|
+
| `threshold` | `4` | how far over each error's local level an impulse stands, multiples |
|
|
387
|
+
| `order` | `32` | AR order of the detection |
|
|
388
|
+
| `fs` | `44100` | sample rate, Hz |
|
|
389
|
+
|
|
390
|
+
`node scripts/decrackle.js` adds crackle to speech (audio-lena; 8 VoiceBank+DEMAND test utterances), music ("Vibe Ace", Brahms, the trumpet loop, as `repair`) and a sung vibrato (VocalSet), 6 s of each: impulses at random times, half 1–3 samples off, half ticks ringing at 2–8 kHz for 0.03–0.15 ms, each peaking at a fifth to all of 0.5, 2 or 8× the sound's RMS. It measures the SDR to the clean sound, input → decrackle · FFmpeg `adeclick` at its defaults, the mean over the six:
|
|
391
|
+
|
|
392
|
+
| crackle, peaks × RMS | 50/s | 200/s | 1000/s |
|
|
393
|
+
|---|---:|---:|---:|
|
|
394
|
+
| small (0.1–0.5×) | 38.7 → 44.2 · 32.1 | 32.7 → 38.3 · 31.1 | 25.8 → 30.2 · 27.5 |
|
|
395
|
+
| medium (0.4–2×) | 26.7 → 36.4 · 30.7 | 20.8 → 29.6 · 28.0 | 13.8 → 20.8 · 20.3 |
|
|
396
|
+
| loud (1.6–8×) | 14.8 → 29.2 · 26.1 | 8.6 → 21.8 · 19.7 | 1.7 → 12.0 · 8.7 |
|
|
397
|
+
|
|
398
|
+
Of the 54 cases, `adeclick` does better in 16: 8 of the sung vibrato's (by 0.1–4.9 dB, most with loud crackle at 50/s: 24.2 · 29.1), 8 of speech and Brahms by 0.3–1.4 dB; `decrackle` in the rest, by up to 22 dB (the trumpet, small crackle at 50/s: 48.6 · 26.7). The clean sound through it: audio-lena 0.01% of the samples changed, to −66 dB; VoiceBank 0.22%, −41 dB; the music 0.01–0.12%, −52 to −74 dB; the song 0.02%, −67 dB (`adeclick`: 1.9–8.5%, −26 to −47 dB). 0.1.7 (2.5 × its window's MAD, 30 Gauss-Seidel sweeps of the fill) made small crackle worse than it found it (38.7 → 21.9 dB at 50/s, the trumpet to 5.6) and changed 3.7–10% of the clean samples (the trumpet to −5.5 dB, speech to −21 dB). `threshold: 3` takes 0.6–2.5 dB more of medium and loud crackle (1000/s: 22.1 and 14.5 dB) and trails `adeclick` on the song alone, but changes up to 0.56% of the clean samples, to −40.5 dB, the trumpet above 8 kHz by as much as is in it. 10 s in 0.1–0.5 s clean, 1–4 s crackled (0.1.7: 4–11 s).
|
|
399
|
+
|
|
400
|
+
**Use when:** shellac / 78 RPM crackle; a worn LP's surface noise; dense small ticks under music or speech.<br>
|
|
401
|
+
**Not for:** loud isolated clicks and pops (use `declick`); long dropouts (use `repair`).
|
|
268
402
|
|
|
269
403
|
|
|
270
404
|
### `declip`
|
|
271
405
|
|
|
272
|
-
|
|
406
|
+
Finds where a sound was cut flat and rebuilds what the cut took off. A hard clip piles every sample it cuts onto one level, its rail. Each side's rail is the sound's extreme there, taken when the samples within 0.1 % of it outnumber those in the next 0.1 % below tenfold and most of them sit in runs: the top-bin test of FFmpeg's `adeclip` (1000 bins, ratio 10), made relative to the rail so it holds at any level (and never finer than one step of a 16- or 24-bit grid), and per side, so one rail or two different ones are found. A loud peak is one sample; a limiter's ceiling is touched by single samples or by arches, and a sine's top is an arch, which fill the band below about as densely: sound with no rail comes back bit-exact. Then, per window of 186 ms (hop half, crossfaded), every sample at a rail is unknown at once: AR(256) fitted to the window under its taper, all of them solved by exact least squares ([`lpc`](https://github.com/audiojs/denoise/tree/main/packages/lpc)'s `arFill`), each held at least as far out as it was recorded (a clip only ever lowers a sample: the consistent set of A-SPADE, Kitić, Bertin & Gribonval 2015), the model refitted to the filled window and the solve repeated, 8 times (Janssen, Veldhuis & Vries 1986). Runs of any length; samples off the rails come back bit-exact.
|
|
273
407
|
|
|
274
408
|
```js
|
|
275
|
-
declip(data, {
|
|
276
|
-
declip(data)
|
|
409
|
+
declip(data, { fs: 44100 }) // each side's rail found; none: untouched
|
|
410
|
+
declip(data, { fs: 44100, clipLevel: 0.95 }) // a known rail, ±0.95
|
|
277
411
|
```
|
|
278
412
|
|
|
279
413
|
| Param | Default | |
|
|
280
414
|
|---|---|---|
|
|
281
|
-
| `clipLevel` | auto |
|
|
282
|
-
| `order` | `
|
|
283
|
-
| `
|
|
415
|
+
| `clipLevel` | auto | the rail, ± this; omitted or 0: found per side |
|
|
416
|
+
| `order` | `256` | AR order of the rebuild; `512` takes dense music up to 2 dB closer and speech further, in about 3× the time |
|
|
417
|
+
| `fs` | `44100` | sample rate, Hz |
|
|
418
|
+
|
|
419
|
+
`node scripts/declip.js` clips speech (audio-lena) and music ("Vibe Ace", Brahms, the trumpet loop, as `repair`), 6 s of each, peak-normalized and cut at the level that leaves the clipped sound 3–20 dB from the original (the input SDR of the declipping survey, Záviška, Rajmic, Ozerov & Rencker 2021), and measures the SDR after, dB, declip (finding the rails itself) · FFmpeg `adeclip` at its defaults:
|
|
420
|
+
|
|
421
|
+
| input SDR | 3 dB | 7 dB | 10 dB | 15 dB | 20 dB |
|
|
422
|
+
|---|---:|---:|---:|---:|---:|
|
|
423
|
+
| speech | 11.6 · 5.0 | 24.2 · 12.1 | 31.2 · 16.9 | 37.7 · 24.0 | 42.4 · 30.5 |
|
|
424
|
+
| "Vibe Ace" | 9.3 · 5.0 | 15.6 · 10.3 | 19.7 · 14.0 | 26.5 · 19.9 | 32.9 · 24.6 |
|
|
425
|
+
| Brahms | 8.5 · 3.9 | 16.1 · 10.2 | 20.0 · 14.4 | 26.4 · 19.8 | 34.0 · 23.8 |
|
|
426
|
+
| trumpet | 15.1 · 4.1 | 31.4 · 10.4 | 45.6 · 14.8 | 46.1 · 21.4 | 59.0 · 25.0 |
|
|
427
|
+
|
|
428
|
+
A-SPADE as the survey's code runs it (`--spade`: 186 ms blocks, a twice-redundant DFT, s = r = 1, ε = 0.1): speech 9.7 / 22.4 / 27.6 / 33.7 / 37.7, "Vibe Ace" 13.2 / 19.0 / 24.4 / 29.0 / 33.0, Brahms 12.9 / 18.0 / 21.0 / 26.3 / 31.8, the trumpet 17.7 / 31.4 / 37.2 / 49.8 / 57.0: ahead of declip in 9 of the 15 music cases, by 1.0–4.7 dB (each at 3 dB, "Vibe Ace" up to 15 dB), within 0.1 dB in 3, behind in 3 (the trumpet at 10 dB by 8.4); behind it on speech at every level, by 1.8–4.7 dB; in 27–65 s per second of sound. VoiceBank+DEMAND's test set, every 20th utterance clipped to 10 and 20 dB: declip 19.8 and 32.4 dB, `adeclip` 16.0 and 28.2.
|
|
429
|
+
|
|
430
|
+
Rails are found at the 20 dB level clipped asymmetrically (with the 10 dB level below), on one side only, then turned down to 0.4, quantized to 16 bits plain or dithered, and on a 16-bit converter driven 2.5 dB over (rails at 32767 and −32768): 10 dB or more of SDR gained in each. The unclipped material through it, as it is, peak-normalized, at 16 bits and through a lookahead limiter 12 dB over its ceiling: not a sample changed; nor in any of VoiceBank+DEMAND's 824 clean test utterances. 0.1.7 took the most populated level over half scale for the rail: on unclipped sound peaking over it, a limiter at half scale (151–43,538 samples changed per 6 s; the overdriven converter's Brahms 41.1 → 21.3 dB), while clipping at other levels it mostly left as it was (3–20 dB in, the same out). It takes about 0.1 s per second of sound clipped at the 20 dB level, 0.5 s at 10 dB, 5 s at 3 dB (`adeclip`: 0.07, 1.3 and 23 s on the same machine).
|
|
284
431
|
|
|
285
|
-
**Use when:**
|
|
286
|
-
**Not for:**
|
|
432
|
+
**Use when:** digital clipping: a converter overdriven, a mix bounced too hot, a plug-in's hard clip; at any level, on either side or both.<br>
|
|
433
|
+
**Not for:** clipping that has since been through lossy coding or resampling (the plateau is no longer flat, and no rail is found); soft saturation, tape or tube, which has no rail.
|
|
287
434
|
|
|
288
435
|
|
|
289
436
|
## Reverb
|
|
@@ -308,28 +455,45 @@ dereverb(data, { fs: 48000 })
|
|
|
308
455
|
|
|
309
456
|
### `dewow`
|
|
310
457
|
|
|
311
|
-
Wow & flutter correction.
|
|
458
|
+
Wow & flutter correction. A speed change of the tape or disc moves every frequency in the recording by one ratio at one instant; a singer's vibrato, a glide or a melody moves one note and its harmonics. dewow measures only the first: it tracks the partials (phase-vocoder frequency over the 93 ms frame), groups each note's harmonics into one source, and takes as speed only what at least two independent sources agree on, hop to hop (Godsill & Rayner, *Digital Audio Restoration*, 1998, ch. 8, with sources in place of tracks and robust weights); a Wiener gate then keeps of that curve only what stands above its own measured uncertainty. The sound is read back through a variable-rate windowed sinc along the curve. Where nothing is evidence nothing is corrected, and a clip with none comes back bit-exact. A known steady tone — mains hum, a pilot or calibration tone — is evidence on its own: `mode: 'reference'` reads it directly (flutter too, from a calibration tone; a 50 Hz hum is read over 90 ms, wow only), and is the method to reach for when one is there. Whole-signal, length-preserving. `wowFlutter()` alone is the meter.
|
|
312
459
|
|
|
313
460
|
```js
|
|
314
|
-
dewow(data, { fs }) //
|
|
315
|
-
dewow(data, { fs, mode: 'reference', refFreq: 50 }) //
|
|
316
|
-
dewow(data, { fs, wow: true, flutter: false }) //
|
|
461
|
+
dewow(data, { fs }) // from the music's own partials (default)
|
|
462
|
+
dewow(data, { fs, mode: 'reference', refFreq: 50 }) // from 50 Hz hum
|
|
463
|
+
dewow(data, { fs, wow: true, flutter: false }) // slow drift only
|
|
317
464
|
wowFlutter(data, { fs }) // → { speed, times, wow, flutter, confidence }
|
|
318
465
|
```
|
|
319
466
|
|
|
320
467
|
| Param | Default | |
|
|
321
468
|
|---|---|---|
|
|
322
|
-
| `mode` | `'partial'` | `'partial' \| 'reference' \| 'pitch'` |
|
|
469
|
+
| `mode` | `'partial'` | `'partial' \| 'reference' \| 'pitch'` (`pitch`: one voice's f0 against its own trend; takes its vibrato for speed) |
|
|
323
470
|
| `refFreq` | — | Hz — the known tone for `reference` mode |
|
|
324
|
-
| `smooth` | `0.05` | s —
|
|
325
|
-
| `wow` / `flutter` | `true` / `true` | correct
|
|
471
|
+
| `smooth` | `0.05` | s — splits the curve into wow (slower) and flutter |
|
|
472
|
+
| `wow` / `flutter` | `true` / `true` | correct each part |
|
|
326
473
|
| `maxDeviation` | `0.05` | clamp on the speed ratio (±5 %) |
|
|
474
|
+
| `minTrack` | `0.1` | s — shortest partial used (`partial`) |
|
|
475
|
+
| `minFreq` / `maxFreq` | `50` / `2000` | Hz — where partials (`partial`) or the f0 (`pitch`) are looked for |
|
|
327
476
|
| `keepLength` | `true` | output length equals input |
|
|
328
477
|
|
|
329
|
-
|
|
478
|
+
`node scripts/dewow.js` reads clean speech (audio-lena, two Spoken Wikipedia narrations), music ("Vibe Ace", "Dance of the Sugar Plum Fairy", Brahms' Hungarian Dance No. 5, a trumpet loop, three GuitarSet takes) and singing (five VocalSet excerpts) at a varying speed — an off-centre disc (0.55 Hz sine) or tape (0.3–3 Hz random) at 0.3, 1 and 2 % peak — and measures the pitch error left after dewow on the audio itself (local lag against the clean sound, differentiated; cents RMS, mean over clips; doing nothing leaves the wow):
|
|
330
479
|
|
|
331
|
-
|
|
332
|
-
|
|
480
|
+
| | wow | 0.3 % | 1 % | 2 % |
|
|
481
|
+
|---|---|---:|---:|---:|
|
|
482
|
+
| speech | disc | 3.7 → 3.7 | 12.4 → 12.4 | 25.5 → 25.5 |
|
|
483
|
+
| | tape | 1.3 → 1.3 | 4.4 → 4.4 | 9.1 → 9.1 |
|
|
484
|
+
| music | disc | 3.7 → 3.6 | 12.3 → 10.8 | 24.5 → 18.2 |
|
|
485
|
+
| | tape | 1.3 → 1.4 | 4.2 → 4.2 | 8.5 → 8.2 |
|
|
486
|
+
| singing | disc | 3.7 → 3.7 | 12.2 → 12.2 | 24.7 → 24.7 |
|
|
487
|
+
| | tape | 1.3 → 1.3 | 4.4 → 4.4 | 8.9 → 8.9 |
|
|
488
|
+
| steady notes (C4 E4 G♯4 D5) | disc | 3.7 → 0.2 | 12.3 → 0.6 | 24.7 → 1.2 |
|
|
489
|
+
| | tape | 1.2 → 0.2 | 4.1 → 0.3 | 8.3 → 0.6 |
|
|
490
|
+
|
|
491
|
+
Clean, the speech and singing come back bit-exact, five of the seven music clips too; a comped guitar gains 0.85 cents (its strings move together after each chord, as under a speed change), and a strummed one with 0.3 % tape wow comes out 1.3 → 2.4 cents, the one clip left less steady. A vibrato voice (±50 cents at 5.5 Hz), a 220 → 330 Hz glide and a vibrato voice over steady notes come back untouched. 0.1 took every partial's movement for speed: clean speech gained 8 cents of pitch wobble, music 16, singing 34; the glide came out 120 cents off, the vibrato at a fifth of its depth; with wow, 3 to 7 of the 7 music clips and all 5 singing clips came out worse than they went in (singing at 1 % disc: 12 → 48 cents). What dewow cannot do, measured: one voice or instrument alone, whatever its harmonics, gives no evidence — its own pitch movement and wow are the same observation; in real music it corrects only where notes hold steady (the guitars and the Sugar Plum Fairy most; Brahms' strings, "Vibe Ace" and the trumpet little or nothing) and leaves 0.3 % wow as it is, under the music's own pitch jitter; flutter needs a reference tone.
|
|
492
|
+
|
|
493
|
+
Held out (never tuned on): the 824 clean VoiceBank+DEMAND test utterances all come back bit-exact (0.1: none, the worst at −3.3 dB SNR against the input); five MUSDB18 mixes, three more GuitarSet takes and four more VocalSet singers: clean, all within 0.1 cent (all but one bit-exact), and with wow none came out worse — but only two mixes were corrected at all, at 2 % disc wow (24.4 → 22.1 and 21.8 cents); 0.1 added 10–51 cents to each of the music and singing clips, clean, and left 47 of 48 music and 23 of 24 singing cases with wow worse than it found them.
|
|
494
|
+
|
|
495
|
+
**Use when:** tape, cassette, vinyl and film transfers with audible wow over sustained, steady notes; any transfer with a hum, pilot or calibration tone (`reference`).<br>
|
|
496
|
+
**Not for:** a solo voice or instrument without a reference tone, speech (use `reference` on its hum, or leave it); dropouts; azimuth/time-skew; flutter from the music itself.
|
|
333
497
|
|
|
334
498
|
|
|
335
499
|
## Gates & inter-word
|
|
@@ -348,18 +512,44 @@ gate(data, { threshold: -45, attack: 0.005, release: 0.1, hold: 0.05, lookahead:
|
|
|
348
512
|
|
|
349
513
|
### `debreath`
|
|
350
514
|
|
|
351
|
-
VAD-driven inverse gate
|
|
515
|
+
VAD-driven inverse gate: what [`vad`](#lower-level-building-blocks) does not call speech (breaths, mouth noise, the room between phrases) goes down by `range`. Speech is voicing and the sound over the noise floor next to it, so a soft word in noise stays and a breath that a pause parts from the phrase goes. A gap under 0.15 s is no breath (a breath lasts 0.15–0.6 s, Ruinskiy & Lavner 2007) and stays; the gain holds 50 ms past speech. The whole clip is read at once (streaming: false), so the gain is zero-phase: it rises over `attack` before speech starts and falls over `release` after it ends.
|
|
352
516
|
|
|
353
517
|
```js
|
|
354
|
-
debreath(data, { range: -10 }) // -10 dB
|
|
518
|
+
debreath(data, { range: -10 }) // -10 dB between phrases (default -12)
|
|
355
519
|
```
|
|
356
520
|
|
|
357
|
-
|
|
521
|
+
| Param | Default | |
|
|
522
|
+
|---|---|---|
|
|
523
|
+
| `range` | `-12` | dB — how far everything between phrases goes down |
|
|
524
|
+
| `attack` | `0.005` | s — the gain rises over this before speech starts |
|
|
525
|
+
| `release` | `0.1` | s — and falls over this after speech ends |
|
|
526
|
+
|
|
527
|
+
`snrTh` and `flatTh` (0.1) are gone: they tuned the old detector, whose floor was the 10th-percentile frame energy of the whole input. Under noise that percentile is the noise, and every word under 9 dB over it went down. Measured with `python scripts/vad.py` (VoiceBank+DEMAND test set, ten Spoken Wikipedia narrations; defaults chosen on the training subset and ten other narrations), 0.1.8 → 0.2.0, frames turned down by over 3 dB:
|
|
528
|
+
|
|
529
|
+
| | voiced | word edges |
|
|
530
|
+
|---|---:|---:|
|
|
531
|
+
| VoiceBank+DEMAND, 824 noisy | 5.87 → **0.03** % | 24.55 → **0.17** % |
|
|
532
|
+
| the same, clean | 0.03 → 0.09 % | 1.22 → **0.53** % |
|
|
533
|
+
| 10 narrations | 0.07 → 0.02 % | 0.23 → 0.03 % |
|
|
534
|
+
|
|
535
|
+
Breaths (350 ms of noise through three wide resonances, 500/1500/2500 Hz) put into the narrations' pauses, ending G before the next phrase, 35 and 25 dB under the speech: median gain, share turned down by 6 dB or more.
|
|
536
|
+
|
|
537
|
+
| G | −35 dB | −25 dB |
|
|
538
|
+
|---|---|---|
|
|
539
|
+
| 0.05 s | −6.6 dB, 53 % → −0.1 dB, 39 % | −0.2 dB, 45 % → 0.0 dB, 11 % |
|
|
540
|
+
| 0.15 s | −2.9 dB, 44 % → **−10.5 dB, 56 %** | −0.1 dB, 44 % → 0.0 dB, 37 % |
|
|
541
|
+
| 0.3 s | −4.0 dB, 50 % → **−11.7 dB, 81 %** | −3.2 dB, 50 % → **−11.7 dB, 75 %** |
|
|
542
|
+
| 0.5 s | −11.5 dB, 55 % → **−12.0 dB, 82 %** | −11.5 dB, 55 % → **−12.0 dB, 91 %** |
|
|
543
|
+
|
|
544
|
+
A breath within 0.3 s of a vowel, parted from it by less than 0.15 s, reads as the word's onset and stays: that is where consonants lie, parted from the vowel at most by a stop's closure. Breaths that close to a phrase went down more often before (53 → 39 %, 45 → 11 %), and so did a quarter of the word edges in noise. Music, frames within 30 dB of the loudest turned down by over 3 dB: Vibe Ace 37.4 → 0.39 %, Brahms 4.29 → 0.02 %, Nutcracker 22.1 → 0 %, trumpet 0 → 0 %, four sung excerpts (VocalSet) up to 21.6 → up to 0.39 %.
|
|
545
|
+
|
|
546
|
+
**Use when:** breath, mouth noise, hiss in pauses on a voiceover.<br>
|
|
547
|
+
**Not for:** a breath that runs into a word; whispered speech (it holds no voicing: it goes down whole).
|
|
358
548
|
|
|
359
549
|
|
|
360
550
|
### `desilence`
|
|
361
551
|
|
|
362
|
-
VAD-driven silence editing. Runs `vad` once, folds frames into speech/pause segments, then cuts pauses per `mode` with equal-power crossfades at every splice — never a hard cut. Length-changing, so it is a batch call (and a `silence` stat atom), not a streaming processor.
|
|
552
|
+
VAD-driven silence editing. Runs [`vad`](#lower-level-building-blocks) once (voicing, and the sound over a tracked noise floor next to it: noise, breaths and room tone between phrases are pause, a quiet word in noise is not), folds frames into speech/pause segments, then cuts pauses per `mode` with equal-power crossfades at every splice — never a hard cut. Length-changing, so it is a batch call (and a `silence` stat atom), not a streaming processor.
|
|
363
553
|
|
|
364
554
|
```js
|
|
365
555
|
desilence(data, { mode: 'shorten', maxSilence: 0.25 }) // Overcast "Smart Speed": long pauses → 0.25 s
|
|
@@ -380,8 +570,18 @@ splitSilence(data, { fs }) // → one Floa
|
|
|
380
570
|
|
|
381
571
|
Returns `{ data, segments, removed, map }`; `project(map, t)` re-times markers/subtitles (`@audio/subtitle` `project()` takes the same map).
|
|
382
572
|
|
|
573
|
+
0.1 read silence under 11 dB over the input's 10th-percentile frame energy: under noise that percentile is the noise, and whole words went. Measured with `python scripts/vad.py` (as `debreath` above), `shorten` at its defaults, 0.1.1 → 0.2.0, frames cut:
|
|
574
|
+
|
|
575
|
+
| | voiced | word edges | removed |
|
|
576
|
+
|---|---:|---:|---:|
|
|
577
|
+
| VoiceBank+DEMAND, 824 noisy | 3.89 → **0.00** % | 3.93 → **0.00** % | 290 → 21 of 2072 s |
|
|
578
|
+
| the same, clean | 0.00 → 0.03 % | 0.01 → 0.12 % | 167 → 126 s |
|
|
579
|
+
| 10 narrations | 0.00 → 0.00 % | 0.00 → 0.02 % | 50 → 52 of 600 s |
|
|
580
|
+
|
|
581
|
+
Breaths in the narrations' pauses (as above) ending 0.3 s before the next phrase: 56 → 75 % of each removed at −35 dB, 50 → 69 % at −25 dB. Music, frames within 30 dB of the loudest cut: Vibe Ace 10.5 → 0 %, Brahms 3.5 → 0.02 %, Nutcracker 16.8 → 0 %, sung (VocalSet m8) 25.3 → 0 %.
|
|
582
|
+
|
|
383
583
|
**Use when:** podcast/lecture pause tightening, split-by-silence, auto-trim.<br>
|
|
384
|
-
**Not for:** music (a rest is a pause to this VAD); overlapping speech.
|
|
584
|
+
**Not for:** music (a rest is a pause to this VAD); overlapping speech; whispered speech (no voicing: it reads as pause).
|
|
385
585
|
|
|
386
586
|
|
|
387
587
|
## Quality measurement
|
|
@@ -414,7 +614,7 @@ import { noiseProfile, minStats, imcra } from '@audio/denoise'
|
|
|
414
614
|
```
|
|
415
615
|
|
|
416
616
|
- **`stft*`** — analysis-modification-synthesis with Hann + ∑win² OLA reconstruction. Visit `(mag, phase, state, ctx) => { mag, phase }`.
|
|
417
|
-
- **`vad`** — frame-level
|
|
617
|
+
- **`vad`** — frame-level speech: Sohn's likelihood ratio over a minimum-statistics noise floor, anchored on voicing ([`@audio/vad`](packages/vad)).
|
|
418
618
|
- **`spp`** — per-bin Speech Presence Probability under Gaussian model.
|
|
419
619
|
- **`ddSnr`** — decision-directed a-priori SNR (Ephraim-Malah).
|
|
420
620
|
- **`noiseProfile`** — average PSD over leading frames.
|
|
@@ -428,9 +628,9 @@ import { noiseProfile, minStats, imcra } from '@audio/denoise'
|
|
|
428
628
|
|
|
429
629
|
| scenario | SNR-in | best method | SNR-out | NRR | ms |
|
|
430
630
|
|---|---:|---|---:|---:|---:|
|
|
431
|
-
| 60 Hz hum + harmonics | -5.2 dB | `dehum` |
|
|
631
|
+
| 60 Hz hum + harmonics | -5.2 dB | `dehum` | 40.4 dB | 6.4 dB | 65 |
|
|
432
632
|
| white noise (~13 dB SNR) | 13.3 dB | `wiener` | 20.5 dB | 0.3 dB | 82 |
|
|
433
|
-
| clicks (vinyl-style) | 24.1 dB | `declick` |
|
|
633
|
+
| clicks (vinyl-style) | 24.1 dB | `declick` | 46.2 dB | — | 227 |
|
|
434
634
|
| 7 kHz sibilance | 2.0 dB | `deesser` | 9.2 dB | 1.9 dB | 5 |
|
|
435
635
|
|
|
436
636
|
Higher = better.
|
|
@@ -443,16 +643,37 @@ VoiceBank+DEMAND test set (Valentini-Botinhao 2017, CC BY 4.0), 824 utterances;
|
|
|
443
643
|
|
|
444
644
|
| op, before → after | PESQ | STOI | SI-SDR dB | SIG | BAK | OVRL |
|
|
445
645
|
|---|---|---|---|---|---|---|
|
|
446
|
-
| `omlsa` | 1.88 → **2.
|
|
447
|
-
| `wiener` | 2.19 → 2.
|
|
448
|
-
| `specsub` | 2.11 → 2.
|
|
646
|
+
| `omlsa` | 1.88 → **2.36** | 0.870 → 0.919 | 5.4 → **14.0** | 3.07 → 3.38 | 3.15 → **3.43** | 2.51 → **2.84** |
|
|
647
|
+
| `wiener` | 2.19 → 2.36 | 0.916 → 0.911 | 11.9 → 14.3 | 3.34 → 3.40 | 3.26 → 3.43 | 2.73 → 2.84 |
|
|
648
|
+
| `specsub` | 2.11 → 2.26 | 0.921 → 0.919 | 10.4 → 12.8 | 3.29 → 3.36 | 3.12 → 3.36 | 2.66 → 2.79 |
|
|
449
649
|
| `dehum` (no hum here) | 1.78 → 1.97 | 0.907 → 0.921 | 6.5 → 8.4 | 3.29 → 3.32 | 3.08 → 3.11 | 2.65 → 2.68 |
|
|
450
650
|
|
|
451
|
-
The same clean utterances in: PESQ 3.06 → 4.
|
|
651
|
+
The same clean utterances in: PESQ 3.06 → 4.29 (`omlsa`), 3.97 → 4.03 (`wiener`), 4.30 → 4.03 (`specsub`, which now subtracts an unbiased noise estimate), 3.20 → 4.64 (`dehum`: no hum found, output equals input). Stationary white and pink noise alone, 8 s: `omlsa` takes it 15 dB down with log kurtosis ratio 0.00 (was +0.53 and +0.30), `wiener` 15.5 dB at +0.53 and +1.01 (was 10 dB at +0.93 and +1.35), `specsub` 12 dB at +0.78 and +1.44 (was 4 dB at +0.48 and +0.58). With mains hum 20 dB under the clean utterances (12 harmonics at −6 dB per octave, 0.05 Hz off nominal), `dehum` takes PESQ from 2.64 to 4.17 at 50 Hz and from 2.48 to 4.26 at 60 Hz, SI-SDR from 20.0 to 39.1 and 41.5 dB, STOI to 0.998 (0.2.0's notches: PESQ 2.88 and 2.76, SI-SDR 23.8 and 24.2 dB); alone, that hum goes 110 and 87 dB down (0.2.0: 44.6 and 50.0).
|
|
652
|
+
|
|
653
|
+
Ten Spoken Wikipedia narrations, 60 s each (volunteers at home; raw OVRL 3.17): `omlsa` SIG 3.25 → 3.46 (raw 3.45), BAK 4.00 → 4.09, OVRL 2.96 → 3.20, speech level −1.1 → −0.04 dB, noise floor median −76 → −75 dBFS. Room tone stays: G<sub>min</sub> is 15 dB, so only a room already under −75 dBFS ends under −90 (one of the nine, raw −83).
|
|
654
|
+
|
|
655
|
+
In the rows above, `omlsa` 0.3 and `wiener` 0.3 take the decision-directed α per 8 ms of frame step, as their papers quote it, where it was per frame (the a priori SNR's memory ran 1.8× longer at 48 kHz than at 44.1); `omlsa`'s is the lowest per 8 ms that leaves steady noise free of musical noise, 0.97 tracking and 0.95 on a learned noise, `wiener`'s Ephraim & Malah's 0.98; `omlsa`'s G<sub>min</sub> is the floor; and minimum statistics (`wiener`, `specsub`; noise-estimate 2.2) starts its smoother on the mean of the first frames, not on one periodogram that left 2.6 % of bins 10 dB low for the first 1.5 s. Against 0.2 on the test set: `omlsa` PESQ 2.40 → 2.36, STOI 0.916 → 0.919, SI-SDR 14.5 → 14.0, SIG 3.39 → 3.38, OVRL 2.86 → 2.84; `wiener` 2.34 → 2.36, 0.910 → 0.911, 14.0 → 14.3, 3.38 → 3.40, 2.81 → 2.84; `specsub` 2.24 → 2.26, 0.920 → 0.919, 12.5 → 12.8, 3.35 → 3.36, 2.77 → 2.79 (chosen on the training subset, where tracked `omlsa` went STOI 0.826 → 0.832, PESQ 1.83 → 1.82). On a learned noise (`omlsa` with `profile`, as audio's `denoise` runs it, G<sub>min</sub> −12 dB, the noise learned from the lead-in): PESQ 2.48 → 2.45, STOI 0.915 → 0.919, SIG 3.36 → 3.38, OVRL 2.89 → 2.88.
|
|
656
|
+
|
|
657
|
+
What each keeps of the speech, by shadow filtering (`node scripts/broadband.mjs`: each frame's gain, computed on the noisy mix, applied to the clean speech and to the noise alone): a word's second and third frame after 85 ms of pause, in ten Spoken Wikipedia narrations under pink noise 10 dB down; speech 20–50 dB under the take's loudest frame, and bins where it stands 0–5 dB over the noise (the MMSE-optimal Wiener gain keeps −3.9 dB there), in the test set's every fourth utterance; the noise taken; the share of clean speech, and of clean music, cut by more than 3 dB. dB, 0.2 → 0.3 (`wiener` and `specsub` 0.3 alone: their 0.2 had no frame process to measure):
|
|
658
|
+
|
|
659
|
+
| op | onset +1 | onset +2 | quiet | 0–5 dB | noise, VB | noise, narr. | clean cut % | music cut % |
|
|
660
|
+
|---|---|---|---|---|---|---|---|---|
|
|
661
|
+
| `omlsa` | −11.2 → −8.9 | −7.3 → −4.8 | −2.3 → −2.0 | −4.2 → −3.6 | −7.5 → −6.7 | −8.1 → −7.5 | 0.4 → 0.3 | 9.8 → 9.6 |
|
|
662
|
+
| `omlsa`, learned noise | −9.5 → −6.1 | −6.2 → −2.7 | −3.5 → −3.0 | −8.7 → −7.9 | −9.0 → −8.3 | −8.9 → −8.4 | 0.6 → 0.4 | – |
|
|
663
|
+
| `wiener` | −5.4 | −2.4 | −2.1 | −3.7 | −7.1 | −7.1 | 0.7 | 14.5 |
|
|
664
|
+
| `specsub` | −1.5 | −0.3 | −1.9 | −3.9 | −4.8 | −5.8 | 0.3 | 10.3 |
|
|
665
|
+
|
|
666
|
+
The paper's α 0.92 kept more again (tracked, a word's second frame after a pause on the training narrations: −1.6 dB) but left musical noise on steady noise (log kurtosis ratio 0.99 and 1.81), as did the paper's q ≤ q<sub>max</sub> in place of the gate (0.24 and 0.46). Martin's time-varying optimal smoothing for minimum statistics kept more speech only by reading the noise low (pink noise 1.7 dB under; `wiener`'s noise taken on the training speech 5.0 → 3.8 dB, PESQ 1.79 → 1.73), and a −25 dB ξ<sub>min</sub> for `wiener` brought more musical noise (0.98 and 1.49) and lower PESQ and SIG: none was taken. Minimum statistics' fixed start costs clean music cut at the start of a take, where the first frame's low values used to hold the estimate down (music cut 12.5 → 14.5 % for `wiener`, all of it in the first 1.5 s of the 6 s trumpet loop, cut 8 → 20 % there; the other three tracks are unchanged). In the half second after music stops, `omlsa` on a learned noise leaves musical noise of 0.12 → 0.11 at G<sub>min</sub> −12 dB and 1.02 → 1.00 at −20 (0.00 a second later).
|
|
667
|
+
|
|
668
|
+
The same test set at other rates (`vbdemand@RATE`), PESQ, STOI and OVRL, 0.2 → 0.3. At 16 kHz (8 ms steps) `wiener`'s α is as it was; at 22.05 and 44.1 kHz (5.8 ms) it rose per frame, 0.980 → 0.985, and its STOI fell 0.004:
|
|
452
669
|
|
|
453
|
-
|
|
670
|
+
| op | 16 kHz | 22.05 kHz | 44.1 kHz | 48 kHz |
|
|
671
|
+
|---|---|---|---|---|
|
|
672
|
+
| `omlsa` | 2.42 → 2.41, 0.915 → 0.918, 2.87 → 2.85 | 2.42 → 2.44, 0.915 → 0.916, 2.86 → 2.86 | 2.42 → 2.44, 0.915 → 0.916, 2.86 → 2.86 | 2.40 → 2.36, 0.916 → 0.919, 2.86 → 2.84 |
|
|
673
|
+
| `wiener` | 2.36 → 2.39, 0.912 → 0.911, 2.82 → 2.85 | 2.37 → 2.40, 0.913 → 0.909, 2.82 → 2.86 | 2.37 → 2.40, 0.913 → 0.909, 2.82 → 2.86 | 2.34 → 2.36, 0.910 → 0.911, 2.81 → 2.84 |
|
|
674
|
+
| `specsub` | 2.25 → 2.27, 0.920 → 0.919, 2.75 → 2.77 | 2.23 → 2.25, 0.920 → 0.919, 2.74 → 2.76 | 2.23 → 2.25, 0.920 → 0.919, 2.74 → 2.76 | 2.24 → 2.26, 0.920 → 0.919, 2.77 → 2.79 |
|
|
454
675
|
|
|
455
|
-
Frames follow the rate: the power of two nearest 32 ms, the papers' frame, where they were 2048 samples at every rate (128 ms at 16 kHz). The same test set resampled (`python scripts/speech.py resample vbdemand RATE`, then `vbdemand@RATE`), PESQ and OVRL, 2048 → rate's frame; 48 kHz keeps 2048:
|
|
676
|
+
Frames follow the rate: the power of two nearest 32 ms, the papers' frame, where they were 2048 samples at every rate (128 ms at 16 kHz). The same test set resampled (`python scripts/speech.py resample vbdemand RATE`, then `vbdemand@RATE`), PESQ and OVRL, 2048 → rate's frame (0.2); 48 kHz keeps 2048:
|
|
456
677
|
|
|
457
678
|
| op | 16 kHz (512) | 22.05 kHz (512) | 44.1 kHz (1024) |
|
|
458
679
|
|---|---|---|---|
|
|
@@ -489,6 +710,8 @@ The old estimate summed every past frame's power though each already holds its o
|
|
|
489
710
|
- Ephraim & Malah, *Speech Enhancement Using a Minimum Mean-Square Error Log-Spectral Amplitude Estimator*, IEEE TASSP 1985.
|
|
490
711
|
- Martin, *Noise Power Spectral Density Estimation Based on Optimal Smoothing and Minimum Statistics*, IEEE TSAP 2001.
|
|
491
712
|
- Martin, *Bias Compensation Methods for Minimum Statistics Noise Power Spectral Density Estimation*, Signal Processing 2006.
|
|
713
|
+
- Sohn, Kim & Sung, *A Statistical Model-Based Voice Activity Detection*, IEEE SPL 1999.
|
|
714
|
+
- Ruinskiy & Lavner, *An Effective Algorithm for Automatic Detection and Exact Demarcation of Breath Sounds in Speech and Song Signals*, IEEE TASLP 2007.
|
|
492
715
|
- Scalart & Vieira Filho, *Speech Enhancement Based on a Priori Signal to Noise Estimation*, ICASSP 1996.
|
|
493
716
|
- Cohen & Berdugo, *Speech Enhancement for Non-Stationary Noise Environments*, Signal Processing 2001.
|
|
494
717
|
- Cohen, *Optimal Speech Enhancement Under Signal Presence Uncertainty Using Log-Spectral Amplitude Estimator*, IEEE SPL 2002.
|
|
@@ -504,6 +727,9 @@ The old estimate summed every past frame's power though each already holds its o
|
|
|
504
727
|
- Yoshioka & Nakatani, *Generalization of Multi-Channel Linear Prediction Methods for Blind MIMO Impulse Response Shortening*, IEEE TASLP 2012.
|
|
505
728
|
- Caroselli, Shafran, Narayanan & Rose, *Adaptive Multichannel Dereverberation for Automatic Speech Recognition*, Interspeech 2017.
|
|
506
729
|
- Traer & McDermott, *Statistics of Natural Reverberation Enable Perceptual Separation of Sound and Space*, PNAS 2016 (MIT IR Survey).
|
|
730
|
+
- Talkin, *A Robust Algorithm for Pitch Tracking (RAPT)*, in Kleijn & Paliwal (eds.), Speech Coding and Synthesis, Elsevier 1995.
|
|
731
|
+
- Boersma, *Accurate Short-Term Analysis of the Fundamental Frequency and the Harmonics-to-Noise Ratio of a Sampled Sound*, IFA Proceedings 17, 1993.
|
|
732
|
+
- Nelke & Vary, *Measurement, Analysis and Simulation of Wind Noise Signals for Mobile Communication Devices*, IWAENC 2014.
|
|
507
733
|
- RBJ Audio EQ Cookbook (biquad coefficients).
|
|
508
734
|
|
|
509
735
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@audio/denoise",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.6.0",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "Noise reduction — gate, spectral subtraction, Wiener, OM-LSA, dehum, declick, decrackle, declip, deesser, dereverb",
|
|
6
6
|
"sideEffects": false,
|
|
@@ -50,26 +50,26 @@
|
|
|
50
50
|
"node": ">=18"
|
|
51
51
|
},
|
|
52
52
|
"dependencies": {
|
|
53
|
-
"@audio/vad": "^
|
|
54
|
-
"@audio/noise-estimate": "^2.
|
|
55
|
-
"@audio/denoise-spectral": "^0.2.
|
|
56
|
-
"@audio/denoise-wiener": "^0.
|
|
57
|
-
"@audio/denoise-omlsa": "^0.
|
|
58
|
-
"@audio/denoise-dehum": "^0.
|
|
59
|
-
"@audio/denoise-declick": "^0.
|
|
60
|
-
"@audio/denoise-decrackle": "^0.
|
|
61
|
-
"@audio/denoise-declip": "^0.
|
|
62
|
-
"@audio/denoise-dewind": "^0.
|
|
63
|
-
"@audio/denoise-deplosive": "^0.
|
|
64
|
-
"@audio/denoise-debreath": "^0.
|
|
53
|
+
"@audio/vad": "^2.0.0",
|
|
54
|
+
"@audio/noise-estimate": "^2.2.0",
|
|
55
|
+
"@audio/denoise-spectral": "^0.2.2",
|
|
56
|
+
"@audio/denoise-wiener": "^0.3.0",
|
|
57
|
+
"@audio/denoise-omlsa": "^0.3.0",
|
|
58
|
+
"@audio/denoise-dehum": "^0.3.0",
|
|
59
|
+
"@audio/denoise-declick": "^0.2.0",
|
|
60
|
+
"@audio/denoise-decrackle": "^0.2.0",
|
|
61
|
+
"@audio/denoise-declip": "^0.2.0",
|
|
62
|
+
"@audio/denoise-dewind": "^0.2.0",
|
|
63
|
+
"@audio/denoise-deplosive": "^0.2.0",
|
|
64
|
+
"@audio/denoise-debreath": "^0.2.0",
|
|
65
65
|
"@audio/denoise-dereverb": "^0.2.0",
|
|
66
|
-
"@audio/denoise-detect": "^0.
|
|
66
|
+
"@audio/denoise-detect": "^0.4.0",
|
|
67
67
|
"@audio/dynamics-gate": "^0.1.4",
|
|
68
|
-
"@audio/denoise-repair": "^0.
|
|
68
|
+
"@audio/denoise-repair": "^0.3.0",
|
|
69
69
|
"@audio/quality": "^1.0.0",
|
|
70
70
|
"@audio/stft": "^2.0.0",
|
|
71
|
-
"@audio/denoise-desilence": "^0.
|
|
72
|
-
"@audio/denoise-dewow": "^0.
|
|
71
|
+
"@audio/denoise-desilence": "^0.2.0",
|
|
72
|
+
"@audio/denoise-dewow": "^0.2.0"
|
|
73
73
|
},
|
|
74
74
|
"devDependencies": {
|
|
75
75
|
"audio-lena": "^3.0.0",
|