@audio/neural-denoise 0.1.0 → 0.3.0

package/README.md

@@ -17,7 +17,7 @@ let better = await denoise(pcm, { sampleRate: 44100, model: 'deepfilternet3' })
 |---|---|---|
 | `sampleRate` | | required unless `audio` carries it |
 | `model` | `'rnnoise'` | `'rnnoise'`, `'deepfilternet3'`, or a handle from `load()` |
-| `limit` | `20` RNNoise, `12` DeepFilterNet3 | attenuation limit, dB: the input is mixed back in at 10^(−limit/20), so noise drops by at most `limit`; `0` for none, upstream's output. Unlimited, RNNoise removes the voice on 12 of the 824 VoiceBank+DEMAND files (STOI down by more than 0.2), on none at 20 dB, where PESQ rises from 2.11 to 2.46; unlimited, DeepFilterNet3 takes the pauses of home narrations down to −152 dBFS, the digital silence ACX Check flags, and at 12 dB keeps them at −91 to −57 dBFS with the same DNSMOS, at a cost on noisy speech: VoiceBank+DEMAND PESQ 2.67 against 3.16 unlimited ([Accuracy](#accuracy)) |
+| `limit` | `16` RNNoise, `18` DeepFilterNet3 | attenuation limit, dB: the input is mixed back in at 10^(−limit/20), so noise drops by at most `limit`; `0` for none. Each default is the most before the voice itself suffers. RNNoise turns down speech that has nothing to remove (clean word ends 37 dB down unlimited, 12 at 16 dB) and removes the voice on some noisy files; on VoiceBank+DEMAND's training speakers its DNSMOS SIG holds from 14 to 16 dB and falls past it, and at 16 no file loses more than 0.2 STOI (21 of 504 unlimited, 3 at 20, the default before 0.3). DeepFilterNet3's SIG holds from 12 to 18 dB and falls past it, while BAK and PESQ keep rising; unlimited, it takes the pauses of home narrations to digital silence, which ACX Check flags ([Accuracy](#accuracy)) |
 | `weights` | | RNNoise: bytes of an upstream weight blob; DeepFilterNet3: URL or bytes of an upstream ONNX export |
 | `device` | `'auto'` | DeepFilterNet3: `@audio/neural-runtime` backend, `'node'`, `'wasm'` or `'webgpu'` |
 | `sessionOptions` | | DeepFilterNet3: ONNX Runtime session options, e.g. `{ intraOpNumThreads: 4 }` |
@@ -55,7 +55,7 @@ let node = new AudioWorkletNode(ctx, 'neural-denoise', { processorOptions: { wei
 mic.connect(node).connect(ctx.destination)
 ```
 
-The processor collects 128-sample render quanta into 10 ms frames. Its output queue starts with 480 − gcd(128, 480) = 448 samples of silence, the least that never runs dry, so the node's delay is constant: 1408 samples, 29.3 ms. It allocates nothing per frame. The context must run at 48 kHz; the browser resamples the device. `limit` works as in `denoise()`: 20 dB unless given, `0` for none.
+The processor collects 128-sample render quanta into 10 ms frames. Its output queue starts with 480 − gcd(128, 480) = 448 samples of silence, the least that never runs dry, so the node's delay is constant: 1408 samples, 29.3 ms. It allocates nothing per frame. The context must run at 48 kHz; the browser resamples the device. `limit` works as in `denoise()`: 16 dB unless given, `0` for none.
 
 Frame by frame, as `rnnoise_process_frame` works:
 
@@ -68,7 +68,7 @@ let voice = st.process(input480, output480)  // int16 scale in and out; voice pr
 
 ## In `audio`: the contract atom
 
-`@audio/neural-denoise/audio` is the package's [contract](https://github.com/audiojs/compile/blob/main/CONTRACT.md) manifest: `rnnoise`, a streaming atom at any sample rate, with one parameter, `limit` (dB, default 20, 0 for none). [`audio`](https://github.com/audiojs/audio) hosts it as its `rnnoise` op, and runs DeepFilterNet3 as its `deepfilter` op (the whole input before rendering, `limit` 12 by default):
+`@audio/neural-denoise/audio` is the package's [contract](https://github.com/audiojs/compile/blob/main/CONTRACT.md) manifest: `rnnoise`, a streaming atom at any sample rate, with one parameter, `limit` (dB, default 16, 0 for none). [`audio`](https://github.com/audiojs/audio) hosts it as its `rnnoise` op, and runs DeepFilterNet3 as its `deepfilter` op (the whole input before rendering, `limit` 18 by default):
 
 ```js
 import audio from 'audio'
@@ -77,7 +77,7 @@ await audio('take.wav').rnnoise().save('clean.wav')     // streams
 await audio('take.wav').deepfilter().save('clean.wav')  // DeepFilterNet3, room tone kept
 ```
 
-Blocks of any size collect into frames; the output queue starts with 479 samples of silence (FRAME − 1, the least that never runs dry for any block size), so the delay is constant: 1439 samples at 48 kHz (30 ms). Other rates go to 48 kHz and back through a streaming form of `@audio/resample-sinc`'s kernel, in its arithmetic, and the delay grows by its lookahead: 1354 samples at 44.1 kHz (30.7 ms), 271 at 8 kHz. The output is `denoise()`'s, delayed: at 48 kHz sample for sample; at other rates, with resample-sinc 1.2.0, sample for sample but in the last 20–30 ms, where the offline resampler sees the end and a stream the host's trailing silence. The default `limit`, 20 dB, is `denoise()`'s for RNNoise: unlimited, the model removes the voice on some files (Accuracy, below).
+Blocks of any size collect into frames; the output queue starts with 479 samples of silence (FRAME − 1, the least that never runs dry for any block size), so the delay is constant: 1439 samples at 48 kHz (30 ms). Other rates go to 48 kHz and back through a streaming form of `@audio/resample-sinc`'s kernel, in its arithmetic, and the delay grows by its lookahead: 1354 samples at 44.1 kHz (30.7 ms), 271 at 8 kHz. The output is `denoise()`'s, delayed: at 48 kHz sample for sample; at other rates, with resample-sinc 1.2.0, sample for sample but in the last 20–30 ms, where the offline resampler sees the end and a stream the host's trailing silence. The default `limit`, 16 dB, is `denoise()`'s for RNNoise: unlimited, the model gates clean speech and removes the voice on some noisy files (Accuracy, below).
 
 ## RNNoise
 
@@ -96,9 +96,10 @@ The float code keeps C's float semantics: each float op is `Math.fround(a op b)`
 Upstream's three ONNX graphs (encoder, ERB-gain decoder, deep-filter decoder) run unchanged; the rest is ported from DeepFilterNet at d375b2d, `libDF` and `df.enhance.enhance()`:
 
 1. **STFT**: 960-point Vorbis window, 480 hop, the input padded by 960 zeros; the same FFT as RNNoise.
-2. **Features**: power in 32 ERB bands (libDF's `erb_fb`: 2 to 67 bins wide) in dB, mean-normalized with α = 0.99 (τ = 1 s); the lowest 96 bins, unit-normalized with the same α. Both run 2 frames ahead of the output (`conv_lookahead`).
+2. **Features**: power in 32 ERB bands (libDF's `erb_fb`: 2 to 67 bins wide) in dB, mean-normalized with α = 0.99 (τ = 1 s); the lowest 96 bins, unit-normalized with the same α. Both run 2 frames ahead of the output (`conv_lookahead`). `denoise()` computes them as if the input's speech (the louder half of its 50 ms frames) were at −20 dBFS, the median of the VoiceBank+DEMAND test set (−20.2): they are not level-free, the ERB means start at −60 to −90 dB and the complex features scale with the root of the level, and the model was trained at speech gains of −6, 0 and +6 dB (libDF `dataset.rs`). Heard as it is, a take 24 dB quieter lost its pauses' noise by 11 to 14 dB instead of 22 to 24, and the noise after 2.7 s of digital silence by 10 to 24 dB; at −20 dBFS it does as the louder take, at no cost on VoiceBank+DEMAND (PESQ 3.077 against 3.076, 206 files). The ERB bands wholly above the input's band edge hear white noise 20 dB under that speech: upstream trained on full-band 48 kHz mixtures only (its `config.ini`: `p_bandwidth_ext = 0`, libDF's `BandwidthLimiterAugmentation` off; noise at −5 to 40 dB SNR), so the network never heard a band empty, yet a 16 kHz file leaves everything above 8 kHz empty and a codec's low-pass everything above 16 to 20 kHz (nine of the ten narrations below end between 16.7 and 21 kHz). Such bands reached it as libDF's constant 1e-10 floor, a band without a trace of noise, which no mixture it trained on had, and it cleaned the band below worse: on VoiceBank+DEMAND's training speakers at 16 kHz, unlimited, PESQ 2.26 against 2.57 full-band; over the floor, 2.42 ([Accuracy](#accuracy)). The edge is the highest frequency whose long-term power lies within 60 dB of the 1 to 4 kHz median, measured at the input's own rate before resampling (a resampler's images don't count); a band holding the edge keeps the input's own content, so a 44.1 kHz input, whose top band (20.7 to 24 kHz) still holds 20.7 to 22.05 kHz, is processed as before. Every bin of an empty band draws its power from an exponential distribution in every frame, from a fixed seed, as white noise's STFT does: a constant floor would help only the first seconds, since the mean normalization (τ = 1 s) takes a constant band's feature to zero, while noise keeps moving it. The spectrum the gains and filters apply to stays as it was, and a full-band input (all of VoiceBank+DEMAND at 48 kHz) is processed bit for bit as before.
 3. **Network**: 32 gains (sigmoid) and 96 × 5 complex filter taps per frame.
 4. **Enhance**: every bin takes its band's gain; below bin 96 (4.8 kHz) the output is instead Σₖ spec[t − 2 + k] · coef[t, k], k = 0–4, over the unmasked spectrum; `limit` mixes the noisy spectrum back in. Inverse FFT, overlap-add, the 480-sample STFT delay removed.
+5. **Voice guard** (`denoise()`): the model takes held voicing for noise. A clean held note from VocalSet (Wilkins 2018) comes out 39 dB down, unlimited, its own local-SNR estimate at its floor (−15 dB), while speech keeps its level; sung phrases lose 24 to 34 dB. So where the input holds sustained voicing, runs of 0.3 s or more periodic at RNNoise's pitch (normalized correlation 0.45 or more, Praat's voicing threshold, the pitch searched every 20 ms) that stand 6 dB or more over the quietest 100 ms of the 20 s around (minimum statistics, Martin 2001: a buzz is that background, a note stands over it), each band holding a harmonic keeps at least c² of its input, c its correlation with the frame one and two periods earlier: the periodic share of its power, as RNNoise's pitch filter restores what its band gains would remove. Between the harmonics and in the pauses the model's output stands.
 
 The parameters come from the export's `config.ini`, so upstream's low-latency export (`DeepFilterNet3_ll_onnx.tar.gz`: no lookahead, 36 MB) loads through `weights` too.
 
@@ -112,9 +113,12 @@ The parameters come from the export's `config.ini`, so upstream's low-latency ex
 | RNNoise vs upstream C as `configure && make` builds it on arm64 (NEON, fused multiply-adds) | 38.8–62.4 dB SNR, voice probability within 0.087: upstream's builds differ from each other by this much (NEON's `vrecpeq_f32` reciprocal estimate inside tanh and sigmoid) |
 | `rnnoise.bin` | byte for byte what upstream's `dump_weights_blob` writes from the model tarball (sha256 `1ad07b42…`) |
 | FFT, window and DCT tables | equal to upstream's `rnnoise_tables.c`: 960 twiddles, 480 window values, 1024 DCT entries, the bit reversal; the FFT matches a direct DFT to 3.2e-9 |
-| DeepFilterNet3 vs Python `df.enhance.enhance()` (deepfilternet 0.5.6, PyTorch checkpoint), same input, `limit: 0` | 133.5 dB SNR on a 1.7 s utterance, 128.2 dB on 11.3 s (max \|d\| 5.7e-7); ERB widths and α identical |
+| DeepFilterNet3, `enhance()` with upstream's settings (no level change, no voice guard, whole file), vs Python `df.enhance.enhance()` (deepfilternet 0.5.6, PyTorch checkpoint), same input, `limit: 0` | 133.5 dB SNR on a 1.7 s utterance, 128.2 dB on 11.3 s (max \|d\| 5.7e-7), 133.8 dB on an utterance led by 2 s of noise, 134.5 dB on a 60 s narration, its first second included; ERB widths and α identical |
+| DeepFilterNet3 voice guard, no model (`test.js`) | a held /a/ (190 Hz, 5 Hz vibrato) 20 dB over white noise through a model that removes everything: kept within 2.8 dB, the noise around it removed; a 120 Hz buzz under the same noise: no frame guarded |
 | DeepFilterNet3 plumbing, no model (`test.js`) | identity gains and taps give the input back to 1.8e-7, whole and chunked (chunks identical to the whole run); a tap one frame ahead advances the band below 4.8 kHz by exactly 480 samples (to 8.9e-8) |
-| Worklet (`test.js`, AudioWorkletGlobalScope simulated) | the frame API's output, delayed by 448 samples, at `limit: 0`; at 12 and at the default 20, that output with the input mixed back, to 1e-7 |
+| DeepFilterNet3 band edge, no model (`test.js`) | the edge of white noise at 48 kHz: 24 kHz; at 16 kHz: 8 kHz; low-passed at 15 kHz: 15 to 16.5 kHz. A 16 kHz input's top band reaches the encoder as the floor (first feature 0.416, as computed; 0.2 fed −0.246, libDF's 1e-10) and moves frame to frame; a full-band input's features are bit for bit those without the floor; identity gains and taps still give the input back |
+| Edge cases (`test.js`), both models, 48 and 16 kHz | empty input, one sample, 100 samples: the same length back, finite; a second of digital silence stays digital silence |
+| Worklet (`test.js`, AudioWorkletGlobalScope simulated) | the frame API's output, delayed by 448 samples, at `limit: 0`; at 12 and at the default 16, that output with the input mixed back, to 1e-7 |
 | Worklet in Chromium 153 (headless, Playwright, `scripts/worklet.mjs`) | runs, no processor error, output non-silent; live underruns in Speed |
 | Contract atom (`test.js`) against `denoise()`, delayed by its declared latency | 48 kHz: equal sample for sample in blocks of 1, 128, 1024 and 1 to 3000 samples, limits 20 and 0; 44.1 kHz (resample-sinc 1.2.0): equal but in the last 50 ms; with resample-sinc 1.1.2, whose arithmetic differs, 59.5 dB SNR |
 
@@ -134,17 +138,57 @@ The VoiceBank+DEMAND test set (Valentini-Botinhao 2017, CC BY 4.0): 824 utteranc
 | "Enhance speech", the whole recipe, with 0.3.11's `omlsa` | 1.63 | 0.830 | 0.84 | 3.06 | 2.37 | 2.18 |
 | RNNoise, `limit: 0` (this package = upstream C) | 2.11 | 0.890 | 12.28 | 3.28 | 3.85 | 2.94 |
 | RNNoise, little model, `limit: 0` | 2.12 | 0.893 | 12.39 | 3.28 | 3.85 | 2.93 |
-| RNNoise, `limit: 20`, the default | 2.46 | 0.918 | 12.86 | 3.40 | 3.71 | 2.97 |
-| **DeepFilterNet3**, `limit: 0` (this package) | **3.16** | **0.945** | **18.96** | **3.44** | **4.08** | **3.18** |
+| RNNoise, `limit: 20` (the default before 0.3) | 2.46 | 0.918 | 12.86 | 3.40 | 3.71 | 2.97 |
+| RNNoise, `limit: 16`, the default | 2.49 | 0.922 | 13.00 | 3.43 | 3.66 | 2.96 |
+| **DeepFilterNet3**, upstream's settings, `limit: 0` (`enhance()`) | **3.16** | **0.945** | **18.96** | **3.44** | **4.08** | **3.18** |
 | DeepFilterNet3, Python (deepfilternet 0.5.6) | 3.16 | 0.945 | 18.96 | 3.44 | 4.08 | 3.18 |
 | DeepFilterNet3 in 1 s chunks, `limit: 0` | 3.16 | 0.945 | 18.94 | 3.44 | 4.08 | 3.18 |
-| DeepFilterNet3, `limit: 12`, the default | 2.67 | 0.939 | 16.18 | 3.49 | 3.69 | 3.03 |
+| DeepFilterNet3, `denoise()`, `limit: 0`: heard at −20 dBFS, voice guard | 3.15 | 0.945 | 18.95 | 3.44 | 4.08 | 3.18 |
+| DeepFilterNet3, `denoise()`, `limit: 18`, the default | 2.90 | 0.942 | 17.91 | 3.48 | 3.84 | 3.09 |
+| DeepFilterNet3, `limit: 12` (the default before 0.2) | 2.67 | 0.939 | 16.18 | 3.49 | 3.69 | 3.03 |
 
-Over the noisy input, paired, with 95% intervals: DeepFilterNet3 adds 1.20 ± 0.03 PESQ, 0.024 ± 0.002 STOI and 10.5 ± 0.3 dB SI-SDR, and beats every classical row's PESQ on 804 of 824 files; RNNoise adds 0.14 ± 0.04 PESQ and 3.8 ± 0.3 dB SI-SDR but loses 0.031 ± 0.004 STOI; `omlsa`, the best classical row, adds 0.43 ± 0.03 PESQ and 6.1 ± 0.2 dB SI-SDR and loses 0.006 STOI. DeepFilterNet3's default 12 dB limit leaves noise in on this set: against `limit: 0` it costs 0.49 ± 0.02 PESQ, 2.8 ± 0.2 dB SI-SDR and 0.39 BAK, and gains 0.05 SIG; it still adds 0.70 ± 0.02 PESQ over the input, 0.27 ± 0.02 over `omlsa` and 0.21 over RNNoise at its default. Pass `limit: 0` where the most noise removal matters more than room tone. The same scoring gives upstream's published anchors: noisy 1.97 and 0.921 (as the DeepFilterNet2 paper's table), DeepFilterNet3 3.16 and 0.945 against the DeepFilterNet3 paper's 3.17 and 0.944. That DeepFilterNet2 table measured RNNoise's 2022 model at 2.33 and 0.922; the current model scores lower here.
+Over the noisy input, paired, with 95% intervals: DeepFilterNet3 adds 1.20 ± 0.03 PESQ, 0.024 ± 0.002 STOI and 10.5 ± 0.3 dB SI-SDR, and beats every classical row's PESQ on 804 of 824 files; RNNoise adds 0.14 ± 0.04 PESQ and 3.8 ± 0.3 dB SI-SDR but loses 0.031 ± 0.004 STOI; `omlsa`, the best classical row, adds 0.43 ± 0.03 PESQ and 6.1 ± 0.2 dB SI-SDR and loses 0.006 STOI. A limit leaves noise in on this set, and spares the voice: paired, on a quarter of the files (206), DNSMOS SIG moves −0.002 ± 0.015 from 12 to 18 dB, then −0.008 ± 0.003 to 20, −0.019 ± 0.006 more to 24 and −0.049 ± 0.017 more unlimited, while BAK rises 0.18, 0.04, 0.07 and 0.16 and PESQ 0.25, 0.06, 0.08 and 0.10. Past 18 dB the speech itself loses, which DNSMOS reads as SIG and a listener as a filtered, phone-call voice; from 8 to 16 kHz the unlimited output carries 1.5 dB less than the clean speech, at 18 dB 0.9 (all 824 files). The default 18 dB still adds 0.93 PESQ over the input and 0.50 over `omlsa`. Pass `limit: 0` where the most noise removal matters more than room tone. The level change and the voice guard cost 0.01 PESQ unlimited and 0.004 at 18 dB. The same scoring gives upstream's published anchors: noisy 1.97 and 0.921 (as the DeepFilterNet2 paper's table), DeepFilterNet3 3.16 and 0.945 against the DeepFilterNet3 paper's 3.17 and 0.944. That DeepFilterNet2 table measured RNNoise's 2022 model at 2.33 and 0.922; the current model scores lower here.
 
-- **RNNoise's current model sometimes removes the voice.** STOI drops by more than 0.1 on 68 of 824 files and by more than 0.2 on 12; on p232_354 it goes from 0.870 to 0.154, the output 24 dB under the clean level while RNNoise's own voice probability averages 0.84. Upstream's C does the same, the portable and the NEON build alike (0.154, 0.153). DeepFilterNet3 never loses more than 0.066 STOI, `wiener` 0.065, `omlsa` 0.059. A 20 dB `limit` (the input mixed back at 0.1) caps it: losses over 0.1 fall to 6 files and over 0.2 to none (p232_354 keeps 0.785), and PESQ rises to 2.46, +0.50 ± 0.03 over the noisy input and +0.07 ± 0.02 over `omlsa`, at a BAK of 3.71 instead of 3.85. `denoise()`, the worklet and the atom apply that limit unless given another.
+- **RNNoise's current model sometimes removes the voice.** STOI drops by more than 0.1 on 68 of 824 files and by more than 0.2 on 12; on p232_354 it goes from 0.870 to 0.154, the output 24 dB under the clean level while RNNoise's own voice probability averages 0.84. Upstream's C does the same, the portable and the NEON build alike (0.154, 0.153). DeepFilterNet3 never loses more than 0.066 STOI, `wiener` 0.065, `omlsa` 0.059. A `limit` caps it: at the default 16 dB (the input mixed back at 0.16) losses over 0.1 fall to 4 files and over 0.2 to none (p232_354 keeps 0.824), and PESQ rises to 2.49, +0.53 ± 0.02 over the noisy input, at a BAK of 3.66 instead of 3.85. Against 0.2's 20 dB, paired: PESQ +0.029 ± 0.009, STOI +0.004, SIG +0.026 ± 0.004, BAK −0.047 ± 0.005 (6 files over 0.1, p232_354 0.786). `denoise()`, the worklet and the atom apply that limit unless given another.
 - **The recipe's delivery stages are not denoisers.** Its compressor, EQ, de-esser and loudness target change the signal against the clean reference on purpose: the full recipe scores under its own denoising part. SI-SDR also falls with the 80 Hz high-pass: on its own it takes clean speech to 6.2 dB SI-SDR at PESQ 4.45 (21 test utterances). The table says what each stage does to speech in noise, not how finished a podcast sounds.
 
+**RNNoise's limit**, chosen on VoiceBank+DEMAND's training speakers (504 utterances: 18 of each of its 28 speakers, none of them in the test set; noise 0 to 15 dB under the voice), scored as above, with the limit-0 output mixed back with the input as `denoise()` mixes it. Clean word ends and quiet syllables: the same utterances' clean takes as input (Clean speech, below).
+
+| `limit` | PESQ | STOI | DNSMOS SIG | BAK | STOI down > 0.2 | clean word ends | clean quiet syllables |
+|---|---|---|---|---|---|---|---|
+| noisy input | 1.47 | 0.841 | 3.00 | 2.50 | | | |
+| 0 (upstream) | 1.83 | 0.827 | 3.20 | 3.76 | 21 files | −36.7 dB | −11.4 dB |
+| 12 | 1.84 | 0.858 | 3.33 | 3.19 | 0 | −8.9 | −8.1 |
+| 14 | 1.88 | 0.857 | 3.34 | 3.26 | 0 | −10.3 | −8.8 |
+| **16**, the default | **1.91** | **0.855** | **3.33** | **3.32** | **0** | **−11.7** | **−9.4** |
+| 18 | 1.93 | 0.853 | 3.32 | 3.37 | 1 | −13.0 | −9.9 |
+| 20 (before 0.3) | 1.93 | 0.851 | 3.31 | 3.42 | 3 | −14.4 | −10.2 |
+
+The limit trades the noise removed (BAK, and PESQ, which rewards it) against the voice: SIG holds from 14 to 16 dB and falls past it (16 against 20, paired: SIG +0.022 ± 0.011, STOI +0.004 ± 0.001, PESQ −0.018 ± 0.008, BAK −0.10 ± 0.01), and 16 is the most at which no utterance loses more than 0.2 STOI; at 20 three do. The limit only bounds what the model takes from the voice; it never stops taking it: speech with nothing to remove still loses 12 dB at its word ends at 16, 9 at 12.
+
+**Clean speech.** A repair with nothing to repair should leave the sound alone. The clean takes of the same utterances as input, against themselves, 10 ms frames: word ends are the last 100 ms before each pause of 100 ms or more, quiet syllables the other frames 20 to 35 dB under the active speech level (ITU-T P.56's, here the mean power of the frames within 35 dB of the 99th-percentile one).
+
+| | training speakers (504): word ends | quiet syllables | SI-SDR vs input | test set (824): word ends | quiet syllables | SI-SDR vs input |
+|---|---|---|---|---|---|---|
+| RNNoise, `limit: 0` | −36.7 dB | −11.4 dB | 15.0 dB | −30.8 dB | −10.2 dB | 15.5 dB |
+| RNNoise, `limit: 20` (before 0.3) | −14.4 | −10.2 | 16.0 | −12.9 | −9.5 | 16.4 |
+| **RNNoise, `limit: 16`**, the default | **−11.7** | **−9.4** | **16.6** | **−10.5** | **−8.8** | **17.0** |
+| DeepFilterNet3, `denoise()`, `limit: 0` | −0.9 | −0.7 | 39.0 | −0.9 | −0.8 | 39.2 |
+| **DeepFilterNet3, `denoise()`, `limit: 18`**, the default | **−0.7** | **−0.6** | **40.2** | **−0.8** | **−0.7** | **40.4** |
+
+RNNoise gates: its gains close on the ends of words and on quiet syllables even with no noise at all, which a listener hears as clipped word ends; the port is bit-exact to upstream's C, so this is the model. DeepFilterNet3 leaves clean speech within a decibel.
+
+**Band-limited input.** The noisy utterances at 16 and at 44.1 kHz (`resample_poly` from 48 kHz), and low-passed at 16 kHz as an MP3 encoder cuts them (a 1023-tap Kaiser FIR, kept at 48 kHz); `denoise()` resamples them to 48 kHz and back; scored at 16 kHz as above. 0.2 heard the empty bands as they are; 0.3 hears the bands wholly above the edge over the floor (DeepFilterNet3, step 2). PESQ / STOI:
+
+| DeepFilterNet3 | training speakers (504): 0.2, unlimited | 0.3, unlimited | 0.2, `limit: 18` | **0.3, `limit: 18`** | test set (824): 0.2, `limit: 18` | **0.3, `limit: 18`** |
+|---|---|---|---|---|---|---|
+| 48 kHz, the full band | 2.57 / 0.881 | the same, bit for bit | 2.24 / 0.876 | the same | 2.90 / 0.942 | the same |
+| 44.1 kHz | 2.55 / 0.880 | the same, bit for bit | 2.23 / 0.876 | the same | 2.89 / 0.940 | the same |
+| low-passed at 16 kHz | 2.42 / 0.876 | 2.51 / 0.881 | 2.19 / 0.874 | **2.22 / 0.876** | 2.76 / 0.938 | **2.86 / 0.941** |
+| 16 kHz | 2.26 / 0.863 | 2.42 / 0.878 | 2.13 / 0.866 | **2.17 / 0.874** | 2.72 / 0.934 | **2.83 / 0.940** |
+
+On the test set, paired, at the default 18 dB: PESQ +0.11 ± 0.01 at 16 kHz and +0.10 ± 0.01 low-passed, STOI +0.006 and +0.003, DNSMOS within 0.02 (BAK −0.011 at 16 kHz, +0.022 low-passed). Unlimited the gains are larger: 16 kHz from 2.68 to 2.99 PESQ (+0.31 ± 0.02; the full band scores 3.15) and 0.924 to 0.940 STOI, low-passed from 2.90 to 3.07; at 16 kHz the model then leaves a little more noise (DNSMOS BAK −0.06, OVRL −0.04) while keeping more of the voice. A 44.1 kHz input is 0.2's output bit for bit: no band lies wholly above 22.05 kHz (a floor over the top band's empty part, tried first, cost 0.012 ± 0.010 PESQ there on the training speakers). The narrations below, eight of them with empty top bands, move by 0.008 DNSMOS at most.
+
 **Real rooms.** Ten amateur narrations from Spoken Wikipedia (volunteers at home, 2007 to 2026, the first 60 s of each; Ogg Vorbis at 44.1 and 48 kHz, decoded and resampled to 48 kHz by ffmpeg), scored without a reference: DNSMOS P.835 as above; the noise floor as the RMS of the quietest 500 ms (ACX Check's measure; one file with pauses cut to digital silence left out); speech level change over the loudest half of 50 ms frames.
 
 | System | SIG | BAK | OVRL | noise floor, dBFS (median) | speech level |
@@ -153,12 +197,31 @@ Over the noisy input, paired, with 95% intervals: DeepFilterNet3 adds 1.20 ± 0.
 | `omlsa()` | 3.46 | 4.09 | 3.19 | −98 to −69 (−75) | 0.0 dB |
 | "Enhance speech" denoising | 3.45 | 4.07 | 3.17 | −101 to −71 (−85) | −0.2 dB |
 | RNNoise, `limit: 0` | 3.42 | 4.14 | 3.19 | −127 to −106 (−112) | −0.5 dB |
-| DeepFilterNet3, `limit: 0` | 3.47 | 4.12 | 3.21 | −152 to −61 (−89) | −0.5 dB |
-| DeepFilterNet3, `limit: 12`, the default | 3.47 | 4.10 | 3.21 | −91 to −57 (−67) | −0.4 dB |
+| RNNoise, `limit: 16`, the default | 3.47 | 4.14 | 3.23 | −98 to −70 (−78) | −0.4 dB |
+| DeepFilterNet3, upstream's settings, `limit: 0` | 3.47 | 4.12 | 3.21 | −152 to −61 (−83) | −0.4 dB |
+| DeepFilterNet3, `denoise()`, `limit: 0` | 3.47 | 4.11 | 3.21 | −165 to −60 (−78) | −0.4 dB |
+| DeepFilterNet3, `denoise()` 0.2, `limit: 18` | 3.47 | 4.11 | 3.21 | −93 to −59 (−73) | −0.4 dB |
+| DeepFilterNet3, `denoise()`, `limit: 18`, the default | 3.47 | 4.11 | 3.21 | −93 to −59 (−73) | −0.4 dB |
+| DeepFilterNet3, `limit: 12` | 3.47 | 4.10 | 3.20 | −91 to −58 (−68) | −0.3 dB |
+
+These rooms are mostly quiet (raw OVRL 3.17), so the gains are small. OM-LSA keeps SIG and raises BAK a little less than the models (0.3.11's lowered SIG by 0.20); both models raise BAK, and at their default limits keep SIG. Unlimited, both can take pauses to dead silence (RNNoise's median floor is −112 dBFS, DeepFilterNet3's lowest −165), which ACX Check flags; the default limits keep the floors between −98 and −59 dBFS at the same OVRL or better, room tone left in. Eight of the narrations end between 16.7 and 19.5 kHz and hear 0.3's floor in their top bands (one ends at 21.1 kHz, inside the top band, one runs to 24): DNSMOS moves by 0.008 at most on any file. On lena (a film scene with music under the voice) RNNoise lowers OVRL from 2.76 to 2.47 unlimited and keeps 2.78 at 16 dB; DeepFilterNet3 raises it to 2.83 upstream, 2.89 at the default limit.
+
+**Singing.** VocalSet (Wilkins et al., ISMIR 2018, CC BY 4.0): 40 long tones (straight, forte, pianissimo, messa di voce; 20 singers) and 20 sung phrases (straight and vibrato), alone and in living-room noise (DEMAND, from the VoiceBank+DEMAND test set) at 15 dB SNR; level change of the voice over the frames within 20 dB of the loudest. At the default 18 dB, against 0.1's processing at the same limit (heard as it is, no voice guard): held notes −1.9 dB clean and −3.4 dB in noise, 14% of frames more than 6 dB down, against −14.2 and −16.2 dB, 96% (at 10 dB SNR −4.8 against −15.6); sung phrases −0.9 and −2.0 dB against −10.2 and −14.9; the same singers' spoken phrases −0.3 and −0.8 dB against −1.7 and −3.7 (they are quiet recordings: the −20 dBFS hearing helps them). Upstream's unlimited output takes the held notes 38.7 and 37.4 dB down. The noise in the pauses drops as before (−14.1 dB against −14.3), and a 100 or 120 Hz buzz 20 dB under the voice drops in them as without the guard (−17.2 dB), in quiet and in cafe noise: it is the background, not over it.
+
+**Music.** Both models enhance speech, and to them everything else is noise: music loses at every band, and the limit only caps the loss. The first 60 s of four pieces (`vibeace` jazz, `brahms` string orchestra, `nutcracker` celesta and strings, `trumpet` solo), five chords of three VocalSet singers each, and the 50 MUSDB18 test previews (Rafii et al. 2017, 7 s of a song each, most with vocals), at 44.1 kHz; level change per band against the input, the median over the files:
+
+| at the default limit | 0–250 Hz | 0.25–1 kHz | 1–4 kHz | 4–8 kHz | 8–16 kHz | 16–22 kHz |
+|---|---|---|---|---|---|---|
+| RNNoise, pieces | −11.5 dB | −12.9 | −13.7 | −15.3 | −14.2 | −11.0 |
+| RNNoise, chords | −7.2 | −15.4 | −15.6 | −15.4 | −14.7 | −15.3 |
+| RNNoise, songs | −14.4 | −8.0 | −10.1 | −11.4 | −12.2 | −14.2 |
+| DeepFilterNet3, pieces | −8.6 | −9.0 | −10.9 | −14.4 | −15.5 | −14.0 |
+| DeepFilterNet3, chords | −5.8 | −3.9 | −4.9 | −9.5 | −11.1 | −3.7 |
+| DeepFilterNet3, songs | −16.9 | −13.7 | −16.5 | −13.2 | −12.7 | −16.0 |
 
-These rooms are mostly quiet (raw OVRL 3.17), so the gains are small. OM-LSA keeps SIG and raises BAK a little less than the models (0.3.11's lowered SIG by 0.20); both models raise BAK and keep SIG. Both can also take pauses to dead silence (RNNoise's median floor is −112 dBFS, DeepFilterNet3's lowest −152), which ACX Check flags; a `limit` of 12 dB keeps DeepFilterNet3's floors between −91 and −57 dBFS at the same OVRL, room tone left in. On lena (a film scene with music under the voice) RNNoise lowers OVRL from 2.76 to 2.47, DeepFilterNet3 raises it to 2.83, 2.90 with the limit.
+Unlimited, single bands of single songs lose up to 54 dB (DeepFilterNet3) and 68 dB (RNNoise). The voice guard keeps one held voice: it follows one pitch, not a chord or an orchestra. Neither model passes music through, because nothing at hand tells music from noisy speech reliably enough. The guard that suggests itself, passing the input through where the unlimited model takes much of what stands 10 dB or more over each band's noise floor (minimum statistics), can't be set: noise that moves stands over the floor too. DeepFilterNet3 takes more than 9 dB of it from 17 of the 504 noisy training utterances, as from 45 of the 50 songs and 3 of the 4 pieces, but from none of the chords (5 to 8.4 dB); at 12 dB, 4 utterances would pass through untouched while 10 songs and every chord would still lose; at 6 dB, 32 utterances would. RNNoise takes more than 9 dB from 32 utterances (its voice removals among them) as from every piece and chord and 40 songs. Telling music from speech takes a classifier trained for it; none ships here.
 
-To rerun: `node scripts/accuracy.mjs vbdemand SYSTEMS` and `node scripts/accuracy.mjs rooms` write the outputs (the data sources and checksums are in its header), `python scripts/accuracy.py vbdemand SYSTEMS` and `python scripts/accuracy.py rooms` score them, and `scripts/deepfilter-reference.py --vbdemand` writes the Python row; the classical rows come from `@audio/denoise`'s `node scripts/speech.mjs vbdemand|rooms SYSTEMS` and `python scripts/speech.py score vbdemand|rooms SYSTEMS`. Per-file scores stay in `~/.cache/audiojs/data/` (`vbdemand/scores/`, `spoken/scores.json`); the scripts regenerate outputs, skipping any already there.
+To rerun: `python scripts/accuracy.py prepare` writes the band-limited inputs and decodes the MUSDB18 previews; `node scripts/accuracy.mjs SET SYSTEMS` (SET: `vbdemand`, `vbtrain`, `vbclean`, `vbtrain-clean`, either noisy set `@16000`, `@44100` or `-lp16k`, `music`) and `node scripts/accuracy.mjs rooms` write the outputs (the data sources and checksums are in its header); `python scripts/accuracy.py SET SYSTEMS`, `limits SET rnnoise 0,12,14,15,16,18,20`, `clean SET SYSTEM [LIMITS]`, `music SYSTEMS`, `presence SYSTEM` and `rooms` score them; `scripts/deepfilter-reference.py --vbdemand` writes the Python row; the classical rows come from `@audio/denoise`'s `node scripts/speech.mjs vbdemand|rooms SYSTEMS` and `python scripts/speech.py score vbdemand|rooms SYSTEMS`. Per-file scores stay in `~/.cache/audiojs/data/` (`vbdemand/scores/neural-*.csv`, `vbdemand-train/scores/neural-*.csv`, `spoken/scores.json`); the scripts regenerate outputs, skipping any already there.
 
 ## Speed and memory
 
@@ -178,7 +241,7 @@ Apple M4 Max running other jobs (load average 28 to 50 on 14 cores), so wall tim
 
 A 128-sample render quantum lasts 2.67 ms and a frame is due every 3.75 quanta: the slowest SIMD frame measured (0.94 ms) fits a quantum 2.8 times over, the mean 8 times; the JS path's slowest (up to 4.0 ms) does not, and relies on device buffers of 256 samples or more, the browsers' usual size. The live underruns came with the machine oversubscribed, the control underran too; an idle machine is untested. Memory: 3.5 MB of weights (plus a 2.8 MB copy in the kernel's memory), about 70 KB of state per channel; the Node evaluation processes peaked at 83 MB RSS.
 
-**DeepFilterNet3**, onnxruntime-node 1.30, 4 intra-op threads: real-time factor 0.047 in CPU on a 180 s file (8.4 s), 0.087 over the 824 short test utterances (2072 s of audio in 180 s; per-run overhead); Python DeepFilterNet (PyTorch 2.1, 4 threads) 0.067 on the same utterances. Peak RSS 436 MB over the test set, 563 MB for 180 s in 10 s chunks. The browser path (onnxruntime-web, wasm and WebGPU) is untested here.
+**DeepFilterNet3**, onnxruntime-node 1.30, 4 intra-op threads: real-time factor 0.047 in CPU on a 180 s file (8.4 s), 0.087 over the 824 short test utterances (2072 s of audio in 180 s; per-run overhead); Python DeepFilterNet (PyTorch 2.1, 4 threads) 0.067 on the same utterances. Peak RSS 436 MB over the test set, 563 MB for 180 s in 10 s chunks. The voice guard's pitch analysis adds 0.16 s of wall time per minute of audio to the model's 0.6 (one channel at 48 kHz, 4 threads); the band edge, 0.06 s of CPU. The browser path (onnxruntime-web, wasm and WebGPU) is untested here.
 
 ## Licenses
 
@@ -204,7 +267,7 @@ The package is BSD-3-Clause (the RNNoise translation and weights); `deepfilter.j
 J.-M. Valin, "A Hybrid DSP/Deep Learning Approach to Real-Time Full-Band Speech Enhancement", MMSP 2018, [arXiv:1709.08243](https://arxiv.org/abs/1709.08243). · H. Schröter, T. Rosenkranz, A. N. Escalante-B., A. Maier, "DeepFilterNet: Perceptually Motivated Real-Time Speech Enhancement", Interspeech 2023, [arXiv:2305.08227](https://arxiv.org/abs/2305.08227). · C. Valentini-Botinhao, "Noisy speech database for training speech enhancement algorithms and TTS models", University of Edinburgh, 2017, [doi:10.7488/ds/2117](https://doi.org/10.7488/ds/2117). · ITU-T P.862.2 (2007), wideband PESQ. · C. H. Taal, R. C. Hendriks, R. Heusdens, J. Jensen, "An Algorithm for Intelligibility Prediction of Time-Frequency Weighted Noisy Speech", IEEE TASLP 19(7), 2011. · J. Le Roux, S. Wisdom, H. Erdogan, J. R. Hershey, "SDR: Half-baked or Well Done?", ICASSP 2019. · C. K. A. Reddy, V. Gopal, R. Cutler, "DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors", ICASSP 2022. · S. A. Figueroa, "When is double rounding innocuous?", SIGNUM Newsletter 30(3), 1995. · [xiph/rnnoise](https://gitlab.xiph.org/xiph/rnnoise) at 70f1d25 · [Rikorose/DeepFilterNet](https://github.com/Rikorose/DeepFilterNet) at d375b2d.
 
 **Use when:** speech with noise a statistical denoiser leaves or smears (moving noise, a busy room); live voice in a browser (RNNoise, 30 ms); files where quality is worth an 8 MB download (DeepFilterNet3).<br>
-**Not for:** music or singing (both models keep speech and drop the rest); installs that must stay model-free ([`@audio/denoise`](https://github.com/audiojs/denoise)); reverb as such (`dereverb`); live DeepFilterNet3 (upstream's graphs take no state).
+**Not for:** music, songs included (both take 8 to 16 dB from every band at their default limits, and nothing in them tells music from noisy speech; DeepFilterNet3's voice guard keeps one held voice, RNNoise drops it); clean speech with nothing to remove through RNNoise (it gates word ends and quiet syllables, 9 to 12 dB at its default limit); installs that must stay model-free ([`@audio/denoise`](https://github.com/audiojs/denoise)); reverb as such (`dereverb`); live DeepFilterNet3 (upstream's graphs take no state).
 
 ---
 

package/audio.d.ts

@@ -10,7 +10,7 @@ type Process = (inputs: Float32Array[][], outputs: Float32Array[][], params: Liv
 
 /** Chainable-host options for 'rnnoise' */
 export interface RnnoiseOptions {
-  /** 0..100 dB (default 20) */
+  /** 0..100 dB (default 16) */
   "limit"?: Auto
   at?: number | string
   duration?: number | string
@@ -22,7 +22,7 @@ export declare const rnnoise: {
   latency: (ctx: { sampleRate: number, params: Live }) => number
   tail: 0
   params: {
-    /** 0..100 dB (default 20) */
-    "limit": { type: "number", default: 20 }
+    /** 0..100 dB (default 16) */
+    "limit": { type: "number", default: 16 }
   }
 }

package/audio.js

@@ -10,10 +10,10 @@
 // the end, where the offline resampler sees the end and a stream sees the host's trailing silence.
 //
 // limit (dB): the input is mixed back in at 10^(−limit/20), so noise drops by at most `limit`; 0 lifts it
-// (denoise()'s option, DeepFilterNet's atten_lim_db). The default departs from denoise()'s (none) on purpose:
-// unlimited, RNNoise's current model removes the voice on 12 of the 824 VoiceBank+DEMAND test files (STOI
-// down by more than 0.2); limited to 20 dB, on none, and PESQ rises from 2.11 to 2.46 (README, Accuracy).
-// The limit is read per block and applies from the next finished frame on.
+// (denoise()'s option, DeepFilterNet's atten_lim_db). The default is denoise()'s, 16 dB (rnnoise.js LIMIT):
+// unlimited, RNNoise's current model gates clean speech and removes the voice on some noisy files; 16 is the
+// most before the voice itself suffers (README, Accuracy). The limit is read per block and applies from the
+// next finished frame on.
 //
 // Channels are denoised independently; a channel's state (70 KB) is created on its first block, as the
 // worklet does, since hosts declare up to 32 channels.

package/deepfilter.js

@@ -15,20 +15,42 @@
 // activations per second of audio. Each chunk's run starts `warmup` frames early from zero GRU state
 // (those outputs dropped) and continues `fade` frames past its end, where the next chunk crossfades in
 // (raised cosine). The GRUs keep a long memory of their start, so chunked output is another valid
-// trajectory, not the whole-file one; the STFT and the features stay exact across chunks.
+// trajectory, not the whole-file one (40 to 55 dB SNR from it on a 60 s narration; longer warm-ups don't
+// converge); the STFT and the features stay exact across chunks.
+//
+// Added, for denoise(): the network can hear its input at another level (`gain`, its features only: they are
+// not level-free, the ERB means start at fixed dB and the complex features scale with the root of the level),
+// its features can hear a floor of white noise above the input's band edge (`edge`: a band-limited input leaves
+// bands empty that the model, trained on full-band mixtures only, never heard empty), and a voice guard (`voice`)
+// keeps the sustained voicing it takes for noise: held sung notes, chant.
 
 import { load, tensor, fetchModel } from '@audio/neural-runtime'
 import { WINDOW, rfft, irfft, N as FFT_N, HOP as FFT_HOP, BINS } from './fft.js'
+import { pitch } from './rnnoise.js'
 
 const f = Math.fround
 
 // models/DeepFilterNet3_onnx.tar.gz at d375b2d (7,983,136 bytes, sha256 c94d91f7…): enc, erb_dec, df_dec, config.ini
 export const MODEL = 'https://raw.githubusercontent.com/Rikorose/DeepFilterNet/d375b2d8309e0935d165700c91da9de862a99c31/models/DeepFilterNet3_onnx.tar.gz'
 
-// The attenuation limit, dB, that denoise() applies to this model unless given one (0: none, upstream's
-// output). Unlimited, it takes the pauses of ten home narrations as low as -152 dBFS, the digital silence
-// ACX Check flags; limited to 12 dB, they keep -91 to -57 dBFS at the same DNSMOS (README, Accuracy).
-export const LIMIT = 12
+// The attenuation limit, dB, that denoise() applies to this model unless given one (0: none): the most before the
+// voice itself suffers. On VoiceBank+DEMAND, DNSMOS SIG holds from 12 to 18 dB and falls past it, BAK and PESQ
+// rise; unlimited, the pauses of home narrations fall to digital silence (README, Accuracy).
+export const LIMIT = 18
+
+// The level, dBFS, at which denoise() has the network hear the speech (`gain`): the median of the VoiceBank+DEMAND
+// test set it scores its published PESQ on (level(), -20.2); its features are not level-free (README, DeepFilterNet3).
+export const LEVEL = -20
+
+// The floor the features hear in the bands above the input's band edge, dB under the speech: white noise, as the
+// model always heard there. It trained on full-band 48 kHz mixtures only (config.ini: p_bandwidth_ext 0, libDF's
+// BandwidthLimiterAugmentation off; noise at -5 to 40 dB SNR), so a band a 16 kHz file or a codec's low-pass left
+// empty reached it as libDF's constant 1e-10 floor, without the trace of noise every mixture it knew had, and it
+// cleaned the band below worse: on the VoiceBank+DEMAND training subset at 16 kHz, unlimited, PESQ 2.26 against
+// 2.57 full-band. Over a floor at -40, -30, -20 and -10 dB: 2.30, 2.37, 2.42, 2.44; low-passed at 16 kHz, 2.42 and
+// over -20 dB 2.51; at 8 kHz, 1.73 and 2.09. -20 dB, noise in the middle of the SNRs it trained on, takes most of
+// the gain (README, Accuracy).
+export const FILL = -20
 
 const isNode = typeof process !== 'undefined' && !!process.versions?.node
 
@@ -112,13 +134,73 @@ function normAlpha(sr, hop, tau) {
 // ndarray's linspace, in f32 as libDF builds its initial normalization states
 const linspace = (a, b, n) => { let step = f(f(f(b) - f(a)) / (n - 1)); return Float32Array.from({ length: n }, (_, i) => f(f(a) + f(step * i))) }
 
+// ------------------------------------------------ input analysis
+
+// level(x) → the speech level, dBFS: mean power of the louder half of 50 ms frames at 48 kHz (LEVEL where there is none)
+export function level(x) {
+	let F = 2400, p = []
+	for (let i = 0; i + F <= x.length; i += F) { let s = 0; for (let j = i; j < i + F; j++) s += x[j] * x[j]; p.push(s / F) }
+	p.sort((a, b) => b - a)
+	let k = Math.max(1, p.length >> 1), s = 0
+	for (let i = 0; i < k && i < p.length; i++) s += p[i]
+	return s > 0 ? 10 * Math.log10(s / k) : LEVEL
+}
+
+// bandEdge(x, rate) → Hz: the input's band edge, the highest frequency whose long-term power lies within 60 dB of the
+// 1 to 4 kHz median: the power of 960-point frames end to end, averaged, at the input's own rate, before any
+// resampling, so a resampler's images don't count; rate/2 for a full band, and where nothing is heard from 1 to 4 kHz.
+export function bandEdge(x, rate) {
+	let P = new Float64Array(BINS), buf = new Float32Array(FFT_N), re = new Float32Array(BINS), im = new Float32Array(BINS)
+	for (let a = 0; a + FFT_N <= x.length; a += FFT_N) {
+		for (let i = 0; i < FFT_N; i++) buf[i] = x[a + i] * WINDOW[i]
+		rfft(buf, re, im)
+		for (let k = 0; k < BINS; k++) P[k] += re[k] * re[k] + im[k] * im[k]
+	}
+	let k0 = Math.ceil(1000 * FFT_N / rate), k1 = Math.min(BINS - 1, Math.floor(4000 * FFT_N / rate))
+	let mid = Array.from(P.subarray(k0, k1 + 1)).sort((a, b) => a - b)[(k1 - k0 + 1) >> 1], e = BINS - 1
+	if (!(mid > 0)) return rate / 2
+	while (e > 0 && !(P[e] >= mid * 1e-6)) e--
+	return e * rate / FFT_N
+}
+
 // ------------------------------------------------ enhance
 
-// enhance(x, net, opts) → Float32Array: one channel at 48 kHz, as df.enhance.enhance(model, state, x, pad=True)
-export async function enhance(x, net, { limit, chunk = 1000, warmup = 300, fade = 50 } = {}) {
-	let { hop, n, nbErb, nbDf, order, dfLa, convLa, widths, alpha } = net.params
+// voicing(x, T, hop) → { period, on }: per frame, RNNoise's pitch period over the frame's 20 ms window, and `on`
+// inside sustained voicing: runs of 0.3 s or more whose normalized correlation at the period is 0.45 or more
+// (Praat's voicing threshold, Boersma 1993) and whose 100 ms stand 6 dB or more above the quietest 100 ms of the
+// 20 s around (minimum statistics, Martin 2001: that minimum lies 4 to 11 dB under the mean of the DEMAND
+// noises), digital silence (under -90 dBFS) left out. A held note stands over the background; a buzz is in it.
+export function voicing(x, T, hop) {
+	let L = x.length, pt = pitch(), frame = new Float32Array(hop), period = new Int16Array(T), on = new Uint8Array(T)
+	// power per 100 ms of input, and the least of it over the 20 s around (blocks of digital silence left out)
+	let K = Math.ceil(T / 10), e = new Float64Array(K), low = new Float64Array(K).fill(Infinity)
+	for (let i = 0; i < L; i++) e[i / (10 * hop) | 0] += x[i] * x[i]
+	for (let k = 0; k < K; k++) e[k] /= Math.max(1, Math.min(L, (k + 1) * 10 * hop) - k * 10 * hop)
+	for (let k = 0; k < K; k++) if (e[k] > 1e-9) for (let j = Math.max(0, k - 100); j <= Math.min(K - 1, k + 100); j++) low[j] = Math.min(low[j], e[k])
+	// the pitch searched every 20 ms where the frame stands over the background (every 10 ms keeps held notes no better)
+	for (let t = 0; t < T; t++) {
+		for (let i = 0, j = t * hop; i < hop; i++, j++) frame[i] = j < L ? x[j] * 32768 : 0
+		let over = e[t / 10 | 0] >= 4 * low[t / 10 | 0]
+		period[t] = pt.push(frame, over && !(t & 1))
+		on[t] = over && pt.gain >= .45
+	}
+	for (let t = 0, r = 0; t <= T; t++) {
+		if (t < T && on[t]) { r++; continue }
+		if (r < 30) on.fill(0, t - r, t)
+		r = 0
+	}
+	return { period, on }
+}
+
+// enhance(x, net, opts) → Float32Array: one channel at 48 kHz, as df.enhance.enhance(model, state, x, pad=True).
+// `gain` scales the input the network hears (its features), not the spectrum its gains and filters apply to;
+// `edge` (Hz, the input's band edge) has the ERB bands wholly above it hear white noise FILL dB under speech at LEVEL;
+// `voice` keeps sustained voicing (voicing()) the network would remove.
+export async function enhance(x, net, { limit, chunk = 1000, warmup = 300, fade = 50, gain = 1, edge = Infinity, voice = false } = {}) {
+	let { sr, hop, n, nbErb, nbDf, order, dfLa, convLa, widths, alpha } = net.params
 	let L = x.length, T = Math.floor((L + n) / hop), lead = order - 1 - dfLa, look = Math.max(convLa, dfLa)
-	let lim = limit != null && Math.abs(limit) > 0 ? 10 ** (-Math.abs(limit) / 20) : 0
+	let lim = limit != null && Math.abs(limit) > 0 ? 10 ** (-Math.abs(limit) / 20) : 0, g1 = f(gain), g2 = f(gain * gain)
+	let vo = voice && voicing(x, T, hop)
 	// per-frame store, a ring big enough for one chunk with its context
 	chunk = Math.max(1, Math.min(chunk, T)); fade = Math.min(fade, chunk)
 	let cap = chunk + fade + warmup + look + lead + 2
@@ -127,6 +209,18 @@ export async function enhance(x, net, { limit, chunk = 1000, warmup = 300, fade
 	let mem = new Float32Array(hop), buf = new Float32Array(n), made = 0
 	let erbStart = new Int32Array(nbErb), erbK = new Float32Array(nbErb)
 	for (let b = 0, s = 0; b < nbErb; s += widths[b++]) { erbStart[b] = s; erbK[b] = f(1 / widths[b]) }
+	// the floor, in the bands from `empty` on, the first wholly above the edge (a band holding the edge has the input's
+	// own content): white Gaussian noise of power 10^((LEVEL + FILL)/10) per sample has |X_k|² = that / 2n · E per bin,
+	// E exponential of mean 1 (the window's Σw² is n/2, the FFT scaled by 1/n), drawn for every bin in every frame from
+	// a fixed seed (32-bit LCG): a floor the network hears move, as noise does, not a constant its mean normalization
+	// takes to zero. fill(b) → its power per bin in band b this frame
+	let empty = erbStart.findIndex(s => s * sr / n > edge), floor = 10 ** ((LEVEL + FILL) / 10) / (2 * n), seed = 1
+	let fill = b => {
+		if (empty < 0 || b < empty) return 0
+		let p = 0
+		for (let j = 0; j < widths[b]; j++) p -= Math.log(((seed = (Math.imul(seed, 1664525) + 1013904223) >>> 0) + .5) / 2 ** 32)
+		return f(floor * p / widths[b])
+	}
 
 	// frame_analysis + feat_erb + feat_cplx for frames up to t (sequential: the normalizations carry state)
 	function make(upto) {
@@ -138,12 +232,12 @@ export async function enhance(x, net, { limit, chunk = 1000, warmup = 300, fade
 			for (let b = 0; b < nbErb; b++) {
 				let s = 0, k = erbK[b]
 				for (let j = erbStart[b], e = j + widths[b]; j < e; j++) s = f(s + f(f(f(Sr[o + j] * Sr[o + j]) + f(Si[o + j] * Si[o + j])) * k))
-				let v = f(f(Math.log10(f(s + f(1e-10)))) * 10)
+				let v = f(f(Math.log10(f(f(f(s * g2) + fill(b)) + f(1e-10)))) * 10)
 				mean[b] = f(f(v * a1) + f(mean[b] * alpha))
 				FE[slot * nbErb + b] = f(f(v - mean[b]) / 40)
 			}
 			for (let j = 0; j < nbDf; j++) {
-				let re = Sr[o + j], im = Si[o + j]
+				let re = f(Sr[o + j] * g1), im = f(Si[o + j] * g1)
 				unit[j] = f(f(f(Math.hypot(re, im)) * a1) + f(unit[j] * alpha))
 				let d = f(Math.sqrt(unit[j]))
 				FC[slot * nbDf * 2 + j] = f(re / d); FC[slot * nbDf * 2 + nbDf + j] = f(im / d)
@@ -151,6 +245,35 @@ export async function enhance(x, net, { limit, chunk = 1000, warmup = 300, fade
 		}
 	}
 
+	// sustained voicing: a band holding a harmonic of the pitch keeps at least c² of its input, c its normalized
+	// correlation with the frame one period earlier and two (the less): the periodic share of its power, as RNNoise's
+	// pitch filter restores what its band gains remove (Valin 2018). Er, Ei is the output so far.
+	let P = [[new Float32Array(BINS), new Float32Array(BINS)], [new Float32Array(BINS), new Float32Array(BINS)]]
+	function keep(t, o) {
+		let T0 = vo.period[t], f0 = n / T0
+		for (let k = 0; k < 2; k++) {
+			for (let i = 0, j = (t - 1) * hop - (k + 1) * T0; i < n; i++, j++) buf[i] = f((j >= 0 && j < L ? x[j] : 0) * WINDOW[i])
+			rfft(buf, P[k][0], P[k][1])
+		}
+		for (let b = 0; b < nbErb; b++) {
+			let j0 = erbStart[b], j1 = j0 + widths[b]
+			// h·f0, h ≥ 1, within the band's bins (sr/n apart)
+			if (Math.floor((j1 - .5) / f0) < Math.max(1, Math.ceil((j0 - .5) / f0))) continue
+			let xx = 0, ee = 0, c = 1
+			for (let j = j0; j < j1; j++) { xx += Sr[o + j] ** 2 + Si[o + j] ** 2; ee += Er[j] ** 2 + Ei[j] ** 2 }
+			for (let [Pr, Pi] of P) {
+				let xy = 0, yy = 0
+				for (let j = j0; j < j1; j++) { xy += Sr[o + j] * Pr[j] + Si[o + j] * Pi[j]; yy += Pr[j] ** 2 + Pi[j] ** 2 }
+				c = Math.min(c, xy / Math.sqrt(xx * yy + 1e-30))
+			}
+			let p = c > 0 ? c * c : 0, g = Math.min(1, Math.sqrt(ee / (xx + 1e-30)))
+			if (p <= g) continue
+			// mixing a of the input in takes the band's gain from g to p
+			let a = (p - g) / (1 - g)
+			for (let j = j0; j < j1; j++) { Er[j] = f(f(Sr[o + j] * a) + f(Er[j] * (1 - a))); Ei[j] = f(f(Si[o + j] * a) + f(Ei[j] * (1 - a))) }
+		}
+	}
+
 	let out = new Float32Array(L), syn = new Float32Array(hop), y = new Float32Array(n), d = n - hop
 	let Er = new Float32Array(BINS), Ei = new Float32Array(BINS)
 	// a chunk's run continues `fade` frames past its end; the next chunk crossfades from those spectra
@@ -191,6 +314,7 @@ export async function enhance(x, net, { limit, chunk = 1000, warmup = 300, fade
 				}
 			}
 			if (lim) for (let j = 0; j < BINS; j++) { Er[j] = f(f(Sr[o + j] * lim) + f(Er[j] * (1 - lim))); Ei[j] = f(f(Si[o + j] * lim) + f(Ei[j] * (1 - lim))) }
+			if (vo && vo.on[t]) keep(t, o)
 			if (t >= b) { tailR.set(Er, (t - b) * BINS); tailI.set(Ei, (t - b) * BINS); continue }
 			if (a > 0 && t - a < fade) {
 				let w = .5 - .5 * Math.cos(Math.PI * (t - a + .5) / fade), q = (t - a) * BINS

package/denoise.js

@@ -5,7 +5,7 @@
 
 import resample from '@audio/resample-sinc'
 import { model as parseRNNoise, create, FRAME, LIMIT as RNNOISE_LIMIT } from './rnnoise.js'
-import { loadDeepFilter, enhance, MODEL, LIMIT as DFN_LIMIT } from './deepfilter.js'
+import { loadDeepFilter, enhance, level, bandEdge, MODEL, LEVEL, LIMIT as DFN_LIMIT } from './deepfilter.js'
 
 export { create as rnnoise, FRAME } from './rnnoise.js'
 export { MODEL } from './deepfilter.js'
@@ -51,7 +51,8 @@ export default async function denoise(audio, opts = {}) {
 		let out = [], limit = opts.limit ?? LIMITS[handle.model]
 		for (let ch of channels) {
 			let x = rate === RATE ? Float32Array.from(ch) : resample(Float32Array.from(ch), { from: rate, to: RATE })
-			let y = handle.model === 'rnnoise' ? rnnoiseOffline(x, handle.net, limit) : await enhance(x, handle.net, { ...opts, limit })
+			let y = handle.model === 'rnnoise' ? rnnoiseOffline(x, handle.net, limit)
+				: await enhance(x, handle.net, { ...opts, limit, gain: 10 ** ((LEVEL - level(x)) / 20), edge: bandEdge(ch, rate), voice: true })
 			out.push(rate === RATE ? y : fit(resample(y, { from: RATE, to: rate }), ch.length))
 		}
 		return audio instanceof Float32Array ? out[0] : Array.isArray(audio) ? out : { ...audio, channelData: out, sampleRate: rate }

package/index.d.ts

@@ -28,7 +28,7 @@ export interface DenoiseOptions extends LoadOptions {
   sampleRate?: number
   /** 'rnnoise' (default), 'deepfilternet3', or a handle from load() to reuse across calls */
   model?: 'rnnoise' | 'deepfilternet3' | Model
-  /** attenuation limit in dB: mixes the input back in at 10^(−limit/20); default 20 for RNNoise, 12 for DeepFilterNet3 (README, API); 0 for none, upstream's output */
+  /** attenuation limit in dB, the most the noise drops: mixes the input back in at 10^(−limit/20); default 16 for RNNoise, 18 for DeepFilterNet3 (README, API); 0 for none. DeepFilterNet3 also hears its input with the speech at −20 dBFS and, above a band-limited input's edge, a white noise floor 20 dB under it, and keeps sustained voicing it would remove (held sung notes) */
   limit?: number
   /** DeepFilterNet3: frames (10 ms) per model run, default 1000 */
   chunk?: number

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@audio/neural-denoise",
-  "version": "0.1.0",
+  "version": "0.3.0",
   "description": "Neural speech enhancement: RNNoise (Valin 2018) in pure JS, bit-exact to its C, weights bundled; DeepFilterNet3 through ONNX Runtime, matching the Python original",
   "type": "module",
   "sideEffects": false,

package/rnnoise.js

@@ -33,9 +33,11 @@ const F = { c06: f(.6), c07: f(.7), c085: f(.85), c09: f(.9), c03: f(.3), c04: f
 
 export const FRAME = 480
 // The attenuation limit, dB, that denoise(), the worklet and the audio atom apply unless given one (0:
-// none, upstream's output). Unlimited, this model removes the voice on 12 of the 824 VoiceBank+DEMAND
-// test files (STOI down by more than 0.2); limited to 20 dB, on none (README, Accuracy).
-export const LIMIT = 20
+// none, upstream's output): the most before the voice itself suffers. Unlimited, the model gates speech
+// that has nothing to remove (clean word ends 37 dB down) and removes the voice on some noisy files. On the
+// VoiceBank+DEMAND training subset (504 files) DNSMOS SIG holds from 14 to 16 dB and falls past it, and 16
+// is the most at which no file loses more than 0.2 STOI (21 files unlimited, 3 at 20) (README, Accuracy).
+export const LIMIT = 16
 const WIN = 960, FREQ = 481, NB = 32, NF = 65
 const PMIN = 60, PMAX = 768, PFRAME = 960, PBUF = PMAX + PFRAME
 
@@ -489,6 +491,22 @@ function removeDoubling(x, T0, prevPeriod, prevGain) {
 	return T0 < PMIN ? PMIN : T0
 }
 
+// pitch() → { push(frame, search = true) → period, gain, buf }: one 10 ms frame at 48 kHz, int16 scale, as
+// rnn_compute_frame_features searches it over the last 36 ms (PITCH_BUF_SIZE, held in `buf`): the period in
+// samples (60 to 768), `gain` the normalized correlation at it; search false only takes the frame in
+export function pitch() {
+	let buf = new Float32Array(PBUF), xlp = new Float32Array(PBUF >> 1), track = { buf, period: 0, gain: 0 }
+	track.push = (x, search = true) => {
+		buf.copyWithin(0, FRAME); buf.set(x, PBUF - FRAME)
+		if (!search) return track.period
+		pitchDownsample(buf, xlp, PBUF)
+		track.period = removeDoubling(xlp, PMAX - pitchSearch(xlp, PMAX >> 1, xlp, PFRAME, PMAX - 3 * PMIN), track.period, track.gain)
+		track.gain = PG[0]
+		return track.period
+	}
+	return track
+}
+
 // ------------------------------------------------ denoiser state (DenoiseState)
 
 // create(weights) → { process(input480, output480) → VAD probability }: one 10 ms frame at 48 kHz,
@@ -496,8 +514,7 @@ function removeDoubling(x, T0, prevPeriod, prevGain) {
 export function create(net) {
 	if (!net || !net.gru1) net = model(net)
 	let analysisMem = new Float32Array(FRAME), synthesisMem = new Float32Array(FRAME)
-	let pitchBuf = new Float32Array(PBUF), memHp = new Float64Array(2), lastg = new Float32Array(NB)
-	let lastPeriod = 0, lastGain = 0
+	let pt = pitch(), pitchBuf = pt.buf, memHp = new Float64Array(2), lastg = new Float32Array(NB)
 	let dXr = new Float32Array(FREQ), dXi = new Float32Array(FREQ), dPr = new Float32Array(FREQ), dPi = new Float32Array(FREQ)
 	let dEx = new Float32Array(NB), dEp = new Float32Array(NB), dExp = new Float32Array(NB)
 	// RNNState
@@ -508,7 +525,7 @@ export function create(net) {
 	let Xr = new Float32Array(FREQ), Xi = new Float32Array(FREQ), Pr = new Float32Array(FREQ), Pi = new Float32Array(FREQ)
 	let Ex = new Float32Array(NB), Ep = new Float32Array(NB), Exp = new Float32Array(NB), Ly = new Float32Array(NB)
 	let feat = new Float32Array(NF), g = new Float32Array(NB), gf = new Float32Array(FREQ)
-	let xlp = new Float32Array(PBUF >> 1), vad = new Float32Array(1)
+	let vad = new Float32Array(1)
 	let t1 = new Float32Array(128), tmp = new Float32Array(1536), cat = new Float32Array(1536)
 	let zrh = new Float32Array(1152), recur = new Float32Array(1152)
 
@@ -521,10 +538,7 @@ export function create(net) {
 	function features(x) {
 		buf.set(analysisMem); buf.set(x, FRAME); analysisMem.set(x)
 		window(buf); rfft(buf, Xr, Xi); bandEnergy(Ex, Xr, Xi)
-		pitchBuf.copyWithin(0, FRAME); pitchBuf.set(x, PBUF - FRAME)
-		pitchDownsample(pitchBuf, xlp, PBUF)
-		let idx = removeDoubling(xlp, PMAX - pitchSearch(xlp, PMAX >> 1, xlp, PFRAME, PMAX - 3 * PMIN), lastPeriod, lastGain)
-		lastPeriod = idx; lastGain = PG[0]
+		let idx = pt.push(x)
 		for (let i = 0; i < WIN; i++) buf[i] = pitchBuf[PBUF - WIN - idx + i]
 		window(buf); rfft(buf, Pr, Pi)
 		bandEnergy(Ep, Pr, Pi); bandEnergy(Exp, Xr, Xi, Pr, Pi)

package/worklet.js

@@ -6,8 +6,8 @@
 // Render quanta (128 samples) are collected into 10 ms frames (480); the output queue starts with
 // 480 − gcd(quantum, 480) samples of silence, the least that never runs dry, so the node's latency is
 // constant: that plus RNNoise's 960 samples (1408 samples, 29.3 ms, at the 128-sample quantum).
-// processorOptions.limit (dB) caps the attenuation by mixing the input back in: 20 unless given, 0 for
-// none (upstream's output).
+// processorOptions.limit (dB) caps the attenuation by mixing the input back in: 16 unless given (LIMIT),
+// 0 for none (upstream's output).
 
 import { model, create, FRAME, LIMIT } from './rnnoise.js'