@audio/neural-denoise 0.4.0 → 0.6.0

package/README.md

@@ -17,7 +17,8 @@ let better = await denoise(pcm, { sampleRate: 44100, model: 'deepfilternet3' })
 |---|---|---|
 | `sampleRate` | | required unless `audio` carries it |
 | `model` | `'rnnoise'` | `'rnnoise'`, `'deepfilternet3'`, or a handle from `load()` |
-| `limit` | `16` RNNoise, `18` DeepFilterNet3 | attenuation limit, dB: the input is mixed back in at 10^(−limit/20), so noise drops by at most `limit`; `0` for none. Each default is the most before the voice itself suffers. RNNoise turns down speech that has nothing to remove (clean word ends 37 dB down unlimited, 12 at 16 dB) and removes the voice on some noisy files; on VoiceBank+DEMAND's training speakers its DNSMOS SIG holds from 14 to 16 dB and falls past it, and at 16 no file loses more than 0.2 STOI (21 of 504 unlimited, 3 at 20, the default before 0.3). DeepFilterNet3's SIG holds from 12 to 18 dB and falls past it, while BAK and PESQ keep rising; unlimited, it takes the pauses of home narrations to digital silence, which ACX Check flags ([Accuracy](#accuracy)) |
+| `limit` | `16` RNNoise, `18` DeepFilterNet3 | attenuation limit, dB: the input is mixed back in at 10^(−limit/20), so noise drops by `limit`; `0` for none. Each default is the most before the voice itself suffers. RNNoise turns down speech that has nothing to remove (clean word ends 37 dB down unlimited, 12 at 16 dB) and removes the voice on some noisy files; on VoiceBank+DEMAND's training speakers its DNSMOS SIG holds from 14 to 16 dB and falls past it, and at 16 no file loses more than 0.2 STOI (21 of 504 unlimited, 3 at 20, the default before 0.3). DeepFilterNet3's SIG holds from 12 to 18 dB and falls past it, while BAK and PESQ keep rising; unlimited, it takes the pauses of home narrations to digital silence, which ACX Check flags ([Accuracy](#accuracy)) |
+| `floor` | `40` | DeepFilterNet3: dB under the voice. Noise the limit would leave closer to the voice than that drops further, down to it, frame by frame (`mixback()`): quiet room tone drops by `limit` and stays, noise as loud as the voice no longer stays 18 dB under it. `0` for none, as before 0.5 ([Measured](#measured-against-izotope-rx-12-dialogue-isolate)) |
 | `music` | `'pass'` | `'pass'`: speech and noise are enhanced, music passes through untouched, segment by segment, the gain ramped over 200 ms at each switch ([Music](#music)); `'enhance'`: everything, as before 0.4 |
 | `weights` | | RNNoise: bytes of an upstream weight blob; DeepFilterNet3: URL or bytes of an upstream ONNX export |
 | `device` | `'auto'` | DeepFilterNet3: `@audio/neural-runtime` backend, `'node'`, `'wasm'` or `'webgpu'` |
@@ -69,7 +70,7 @@ let voice = st.process(input480, output480)  // int16 scale in and out; voice pr
 
 ## In `audio`: the contract atom
 
-`@audio/neural-denoise/audio` is the package's [contract](https://github.com/audiojs/compile/blob/main/CONTRACT.md) manifest: `rnnoise`, a streaming atom at any sample rate, with two parameters, `limit` (dB, default 16, 0 for none) and `music` (`'pass'`, the default, or `'enhance'`). [`audio`](https://github.com/audiojs/audio) hosts it as its `rnnoise` op, and runs DeepFilterNet3 as its `deepfilter` op (the whole input before rendering, `limit` 18 and `music: 'pass'` by default):
+`@audio/neural-denoise/audio` is the package's [contract](https://github.com/audiojs/compile/blob/main/CONTRACT.md) manifest: `rnnoise`, a streaming atom at any sample rate, with two parameters, `limit` (dB, default 16, 0 for none) and `music` (`'pass'`, the default, or `'enhance'`). [`audio`](https://github.com/audiojs/audio) hosts it as its `rnnoise` op, and runs DeepFilterNet3 as its `deepfilter` op (the whole input before rendering, unlimited, the input mixed back after by `mixback()`: `limit` 18, `floor` 40 and `music: 'pass'` by default):
 
 ```js
 import audio from 'audio'
@@ -100,7 +101,7 @@ Upstream's three ONNX graphs (encoder, ERB-gain decoder, deep-filter decoder) ru
 1. **STFT**: 960-point Vorbis window, 480 hop, the input padded by 960 zeros; the same FFT as RNNoise.
 2. **Features**: power in 32 ERB bands (libDF's `erb_fb`: 2 to 67 bins wide) in dB, mean-normalized with α = 0.99 (τ = 1 s); the lowest 96 bins, unit-normalized with the same α. Both run 2 frames ahead of the output (`conv_lookahead`). `denoise()` computes them as if the input's speech (the louder half of its 50 ms frames) were at −20 dBFS, the median of the VoiceBank+DEMAND test set (−20.2): they are not level-free, the ERB means start at −60 to −90 dB and the complex features scale with the root of the level, and the model was trained at speech gains of −6, 0 and +6 dB (libDF `dataset.rs`). Heard as it is, a take 24 dB quieter lost its pauses' noise by 11 to 14 dB instead of 22 to 24, and the noise after 2.7 s of digital silence by 10 to 24 dB; at −20 dBFS it does as the louder take, at no cost on VoiceBank+DEMAND (PESQ 3.077 against 3.076, 206 files). The ERB bands wholly above the input's band edge hear white noise 20 dB under that speech: upstream trained on full-band 48 kHz mixtures only (its `config.ini`: `p_bandwidth_ext = 0`, libDF's `BandwidthLimiterAugmentation` off; noise at −5 to 40 dB SNR), so the network never heard a band empty, yet a 16 kHz file leaves everything above 8 kHz empty and a codec's low-pass everything above 16 to 20 kHz (nine of the ten narrations below end between 16.7 and 21 kHz). Such bands reached it as libDF's constant 1e-10 floor, a band without a trace of noise, which no mixture it trained on had, and it cleaned the band below worse: on VoiceBank+DEMAND's training speakers at 16 kHz, unlimited, PESQ 2.26 against 2.57 full-band; over the floor, 2.42 ([Accuracy](#accuracy)). The edge is the highest frequency whose long-term power lies within 60 dB of the 1 to 4 kHz median, measured at the input's own rate before resampling (a resampler's images don't count); a band holding the edge keeps the input's own content, so a 44.1 kHz input, whose top band (20.7 to 24 kHz) still holds 20.7 to 22.05 kHz, is processed as before. Every bin of an empty band draws its power from an exponential distribution in every frame, from a fixed seed, as white noise's STFT does: a constant floor would help only the first seconds, since the mean normalization (τ = 1 s) takes a constant band's feature to zero, while noise keeps moving it. The spectrum the gains and filters apply to stays as it was, and a full-band input (all of VoiceBank+DEMAND at 48 kHz) is processed bit for bit as before.
 3. **Network**: 32 gains (sigmoid) and 96 × 5 complex filter taps per frame.
-4. **Enhance**: every bin takes its band's gain; below bin 96 (4.8 kHz) the output is instead Σₖ spec[t − 2 + k] · coef[t, k], k = 0–4, over the unmasked spectrum; `limit` mixes the noisy spectrum back in. Inverse FFT, overlap-add, the 480-sample STFT delay removed.
+4. **Enhance**: every bin takes its band's gain; below bin 96 (4.8 kHz) the output is instead Σₖ spec[t − 2 + k] · coef[t, k], k = 0–4, over the unmasked spectrum; `enhance()`'s `limit` mixes the noisy spectrum back in per bin, as upstream's `atten_lim_db`. Inverse FFT, overlap-add, the 480-sample STFT delay removed. `denoise()` runs it unlimited and mixes the input back after, in time, by `mixback()`: the noise the model took drops by `limit` dB, or down to `floor` dB under the voice where the limit would leave it closer (its power over the 0.5 s around each 10 ms frame against the voice's level, the louder half of the output's 50 ms frames); `floor: 0` gives the per-bin limit's output, but for the voice guard's order.
 5. **Voice guard** (`denoise()`): the model takes held voicing for noise. A clean held note from VocalSet (Wilkins 2018) comes out 39 dB down, unlimited, its own local-SNR estimate at its floor (−15 dB), while speech keeps its level; sung phrases lose 24 to 34 dB. So where the input holds sustained voicing, runs of 0.3 s or more periodic at RNNoise's pitch (normalized correlation 0.45 or more, Praat's voicing threshold, the pitch searched every 20 ms) that stand 6 dB or more over the quietest 100 ms of the 20 s around (minimum statistics, Martin 2001: a buzz is that background, a note stands over it), each band holding a harmonic keeps at least c² of its input, c its correlation with the frame one and two periods earlier: the periodic share of its power, as RNNoise's pitch filter restores what its band gains would remove. Between the harmonics and in the pauses the model's output stands.
 
 The parameters come from the export's `config.ini`, so upstream's low-latency export (`DeepFilterNet3_ll_onnx.tar.gz`: no lookahead, 36 MB) loads through `weights` too.
@@ -112,10 +113,12 @@ The parameters come from the export's `config.ini`, so upstream's low-latency ex
 Both models enhance speech, and to them everything else is noise: at their default limits music lost 8 to 16 dB in every band, songs included (Accuracy, Music). So by default (`music: 'pass'`) a classifier segments the input into speech, music and noise, and the model's output is kept only where it hears speech or noise; where it hears music the input passes through, bit for bit. What counts as what follows the classifier's training, French radio and TV: speech over a music bed is speech, so the bed under a voice-over is cleaned with the noise; a song is music, its singer included, since a speech enhancer would strip the accompaniment from under the voice and dull the voice too. A voice singing alone sits between the two (below). `music: 'enhance'` enhances everything, as before 0.4.
 
 1. **Classifier**: inaSpeechSegmenter's speech/music/noise CNN (Doukhan et al., ICASSP 2018; MIT; its `smn` engine), 782,403 weights, in the package as float16 (`guard.bin`, 1.56 MB; `scripts/guard.py` writes it from upstream's Keras export, the batch norms folded). Its input as ina computes it: 16 kHz (here a 31-tap low-pass and every third sample of the 48 kHz signal), pre-emphasis 0.97, 25 ms Hann frames every 10 ms, 24 mel bands from 100 Hz to 8 kHz, log; the lowest 21 bands over 0.68 s make a patch, standardized by its own mean and deviation, so the level does not matter. One patch every 100 ms (ina takes one every 20 ms; on the tuning material every 100 ms decides the same).
-2. **Offline** (DeepFilterNet3, `denoise()` and the `deepfilter` op): segments as ina makes them, Viterbi over the whole input, three states, a switch costing 10⁻⁸⁰ per 20 ms (ina's own constant), so a segment lasts a second or more and a stray patch moves nothing. An input that is music throughout never reaches the model.
+2. **Offline** (DeepFilterNet3, `denoise()` and the `deepfilter` op): segments as ina makes them, Viterbi over the whole input, three states, a switch costing 10⁻⁸⁰ per 20 ms (ina's own constant), so a segment lasts a second or more and a stray patch moves nothing; from 0.5 each patch counts 1 nat in favour of speech, the stream's prior (step 3). A voice under a music bed as loud as itself reaches ina as music about half the time: it hears speech there at a mean probability of 0.47, in songs at 0.01 (medians, training sets). With the prior, of VoiceBank's test takes under MUSDB18 test accompaniments at the voice's level 37% of the frames pass instead of 51%, 5 dB under it 0.6% instead of 2.9%, and the training sets' songs and accompaniments pass every frame they passed without it (at 1.5 nats songs lost 1.1%: the prior chosen there). An input that is music throughout never reaches the model.
 3. **Streaming** (RNNoise: `denoise()`, the contract atom): a stream decides on what has arrived. The forward recursion of the same chain for two states (music, the rest) keeps the log-odds of music, bounded at ±20 nats; a patch moves them by at most ln 100, and by 1 nat less (a prior for speech), and the stream switches to music above 5 and back below 0. Each output frame takes the decision made once the input frame two on is in (RNNoise's own delay), so nothing is added to the latency, and `denoise()` and the atom give the same samples. The price: music is denoised for its first second or so, and each switch comes about a second late.
 4. **The gain** between the enhanced signal and the input moves over 200 ms, raised cosine: offline centered on the segment boundary, streaming from the decision on. It is applied at the input's rate (the 48 kHz gain resampled with the signal), so what passes is the input itself, not its round trip through 48 kHz.
 
+`guard(x, sampleRate)` gives that offline gain for any enhancement, per sample of `x` at its own rate: 1 where it is speech or noise, 0 where it is music, ramped as above; `g·y + (1 − g)·x` passes music untouched. `audio`'s `dereverb` guards its de-reverb model by it.
+
 It runs in JS: a stream's `process()` can't await ONNX Runtime, and RNNoise needs no runtime. The three upper convolutions (through im2col) and the dense layers run in a 1,074-byte WebAssembly SIMD kernel (`gemm.wat`), the first convolution once per frame and shared by the overlapping patches (the max pooling after it commutes with each patch's standardization); 1.1 s of CPU per minute of audio, 11 s on the JS path where WebAssembly SIMD is missing (same bits).
 
 **Accuracy**, on labelled sets (`python scripts/accuracy.py guard-sets`, `node scripts/accuracy.mjs guard train|test`): the share of active 10 ms frames passed (within 40 dB of the loudest). The constants (one patch per 100 ms, the streaming bound, prior and hysteresis) were chosen on the training sets: VoiceBank+DEMAND's training speakers, the MUSDB18 training previews, Slakh2100 mixes 1 to 10, VocalSet singers 1 to 5, the Spoken Wikipedia narrations not in Rooms; the test sets below were run once. Beds: clean utterances over a song's accompaniment (the MUSDB18 stems but the vocals) 20, 15 and 10 dB under the speech; hiss: white and pink Gaussian noise at 20, 10 and 5 dB SNR. Programmes: 8 sequences of six segments, speech (five utterances of one kind, or 15 s of narration) and music (up to 20 s of a song, its accompaniment or a Slakh mix) in turn, scored 0.5 s or more from each cut.
@@ -126,10 +129,10 @@ It runs in JS: a stream's `process()` can't await ONNX Runtime, and RNNoise need
 | VoiceBank+DEMAND, clean / noisy (824 each) | | 0.0% / 0.0% | 0.0% / 0.0% |
 | the same in white noise, 20 / 10 / 5 dB (275 each) | | 0.0% / 0.0% / 0.0% | 0.0% / 0.0% / 0.0% |
 | the same in pink noise, 20 / 10 / 5 dB | | 0.0% / 0.0% / 0.0% | 0.0% / 0.0% / 0.0% |
-| over a music bed 20 / 15 / 10 dB down (275 each) | | 0.4% / 0.7% / 1.4% | 0.7% / 1.4% / 2.3% |
+| over a music bed 20 / 15 / 10 dB down (275 each) | | 0.4% / 0.4% / 0.8% (0.4: 0.4% / 0.7% / 1.4%) | 0.7% / 1.4% / 2.3% |
 | ten home narrations (Rooms) | | 0.0% | 0.0% |
 | VocalSet, spoken excerpts (10) | | 1.8% | 2.1% |
-| DEMAND noise alone (noisy − clean) | | 7.8% | 2.4% |
+| DEMAND noise alone (noisy − clean) | | 6.6% (0.4: 7.8%) | 2.4% |
 | **Music, to pass** | | | |
 | MUSDB18 test previews, songs (50) | | 100% | 87.2% |
 | the same, accompaniment only (50) | | 100% | 87.7% |
@@ -137,10 +140,10 @@ It runs in JS: a stream's `process()` can't await ONNX Runtime, and RNNoise need
 | the four repair pieces, 60 s each | | 95.7% | 93.8% |
 | **Programmes** (8): speech enhanced / music passed | | 100% / 100% | 98.5% / 92.9%, switches 0.96 s late (median) |
 | **Singing alone** | | | |
-| MUSDB18 vocal stems (50) | | 57.5% | 32.4% |
-| VocalSet sung phrases (20) / long tones (10) | | 66.4% / 67.8% | 34.3% / 40.0% |
+| MUSDB18 vocal stems (50) | | 47.7% (0.4: 57.5%) | 32.4% |
+| VocalSet sung phrases (20) / long tones (10) | | 58.5% / 57.1% (0.4: 66.4% / 67.8%) | 34.3% / 40.0% |
 
-Offline, every speech set keeps 98.2% or more of its frames enhanced (noisy and hissy speech all of them) and every music set but one passes all of its frames; on the programmes nothing errs 0.5 s from a cut. Streaming, speech is kept as well, and music loses its first second: the previews are 7 s long, so 13% of their frames; the 60 s Slakh mixes, 1.4%. The noise alone that passes is DEMAND's living room (30% of its frames), which has music in it. Of the pieces, `trumpet` (solo) is enhanced for its first 4 s and `brahms` for 5 s of a quiet passage. A voice singing alone is split, half to two thirds of its frames passed offline: held notes and phrases the classifier calls music pass, the rest is enhanced, where DeepFilterNet3's voice guard keeps sustained voicing ([DeepFilterNet3](#deepfilternet3), step 5). Speech over a bed 10 dB down, louder than a podcast's, still counts as speech.
+Offline, every speech set keeps 98.2% or more of its frames enhanced (noisy and hissy speech all of them) and every music set but one passes all of its frames; on the programmes nothing errs 0.5 s from a cut. Streaming, speech is kept as well, and music loses its first second: the previews are 7 s long, so 13% of their frames; the 60 s Slakh mixes, 1.4%. The noise alone that passes is DEMAND's living room (30% of its frames), which has music in it. Of the pieces, `trumpet` (solo) is enhanced for its first 4 s and `brahms` for 5 s of a quiet passage. A voice singing alone is split, about half of its frames passed offline (two thirds before 0.5's prior): held notes and phrases the classifier calls music pass, the rest is enhanced, where DeepFilterNet3's voice guard keeps sustained voicing ([DeepFilterNet3](#deepfilternet3), step 5); enhanced, a VocalSet phrase comes out 1.3 dB down and a MUSDB18 vocal stem 3.1 dB (medians of 10; held tones 0.7). Speech over a bed 10 dB down, louder than a podcast's, still counts as speech.
 
 What it changes, at the default limits (Accuracy: the same sets, `python scripts/accuracy.py music`; per band the median over the files, the worst in parentheses):
 
@@ -175,6 +178,8 @@ The classifier was chosen against two others, run in Python on the same training
 | Worklet (`test.js`, AudioWorkletGlobalScope simulated) | the frame API's output, delayed by 448 samples, at `limit: 0`; at 12 and at the default 16, that output with the input mixed back, to 1e-7 |
 | Worklet in Chromium 153 (headless, Playwright, `scripts/worklet.mjs`) | runs, no processor error, output non-silent; live underruns in Speed |
 | Music guard (`test.js`) against the float32 Keras model, the same input (`scripts/guard.py` → `fixtures/guard.json`) | lena, 106 patches: probabilities within 1.7e-3 (float16 weights); the wasm kernel and the JS loop give the same bits |
+| `mixback()` (`test.js`), a perfect model: a tone and white noise | noise 40 and 25 dB under the voice: 18 dB down (58 and 43 under it); 10 dB under and 5 dB over: 40 under it, the floor; `floor: 0`: 18 down; a noise that drops 30 dB halfway: each half as such 0.5 s from the change; where input and output agree, the input bit for bit; `denoise()`'s default is the unlimited output through it, bit for bit |
+| Music guard's prior (`test.js`, with VoiceBank+DEMAND and the guard sets in the data cache) | speaker p226's first five training takes under a MUSDB18 training accompaniment at their level: 13% of the frames passed (0.4: all) |
 | Music guard (`test.js`): a synthetic band (chords of plucked tones, a bass, a tick) | DeepFilterNet3, through a stand-in that removes everything: the band back bit for bit at 48 and 44.1 kHz, `music: 'enhance'` removes it; RNNoise: the band itself from 1.1 s on; band, noisy speech, band (with VoiceBank+DEMAND in the data cache): two switches, each within 0.5 s of its cut, gain exactly 0 and 1 between 200 ms raised-cosine ramps; RNNoise's output to frame f − 2 the same whatever follows frame f |
 | Contract atom (`test.js`) against `denoise()`, delayed by its declared latency | 48 kHz: equal sample for sample in blocks of 1, 128, 1024 and 1 to 3000 samples, limits 20 and 0; 44.1 kHz (resample-sinc 1.2.0): equal but in the last 50 ms; with resample-sinc 1.1.2, whose arithmetic differs, 59.5 dB SNR |
 
@@ -200,10 +205,11 @@ The VoiceBank+DEMAND test set (Valentini-Botinhao 2017, CC BY 4.0): 824 utteranc
 | DeepFilterNet3, Python (deepfilternet 0.5.6) | 3.16 | 0.945 | 18.96 | 3.44 | 4.08 | 3.18 |
 | DeepFilterNet3 in 1 s chunks, `limit: 0` | 3.16 | 0.945 | 18.94 | 3.44 | 4.08 | 3.18 |
 | DeepFilterNet3, `denoise()`, `limit: 0`: heard at −20 dBFS, voice guard | 3.15 | 0.945 | 18.95 | 3.44 | 4.08 | 3.18 |
-| DeepFilterNet3, `denoise()`, `limit: 18`, the default | 2.90 | 0.942 | 17.91 | 3.48 | 3.84 | 3.09 |
+| DeepFilterNet3, `denoise()`, `limit: 18`, `floor: 0` (the default before 0.5) | 2.90 | 0.942 | 17.91 | 3.48 | 3.84 | 3.09 |
+| **DeepFilterNet3, `denoise()`, `limit: 18`, `floor: 40`, the default** | **3.05** | **0.944** | **18.86** | **3.45** | **4.01** | **3.15** |
 | DeepFilterNet3, `limit: 12` (the default before 0.2) | 2.67 | 0.939 | 16.18 | 3.49 | 3.69 | 3.03 |
 
-Over the noisy input, paired, with 95% intervals: DeepFilterNet3 adds 1.20 ± 0.03 PESQ, 0.024 ± 0.002 STOI and 10.5 ± 0.3 dB SI-SDR, and beats every classical row's PESQ on 804 of 824 files; RNNoise adds 0.14 ± 0.04 PESQ and 3.8 ± 0.3 dB SI-SDR but loses 0.031 ± 0.004 STOI; `omlsa`, the best classical row, adds 0.43 ± 0.03 PESQ and 6.1 ± 0.2 dB SI-SDR and loses 0.006 STOI. A limit leaves noise in on this set, and spares the voice: paired, on a quarter of the files (206), DNSMOS SIG moves −0.002 ± 0.015 from 12 to 18 dB, then −0.008 ± 0.003 to 20, −0.019 ± 0.006 more to 24 and −0.049 ± 0.017 more unlimited, while BAK rises 0.18, 0.04, 0.07 and 0.16 and PESQ 0.25, 0.06, 0.08 and 0.10. Past 18 dB the speech itself loses, which DNSMOS reads as SIG and a listener as a filtered, phone-call voice; from 8 to 16 kHz the unlimited output carries 1.5 dB less than the clean speech, at 18 dB 0.9 (all 824 files). The default 18 dB still adds 0.93 PESQ over the input and 0.50 over `omlsa`. Pass `limit: 0` where the most noise removal matters more than room tone. The level change and the voice guard cost 0.01 PESQ unlimited and 0.004 at 18 dB. The same scoring gives upstream's published anchors: noisy 1.97 and 0.921 (as the DeepFilterNet2 paper's table), DeepFilterNet3 3.16 and 0.945 against the DeepFilterNet3 paper's 3.17 and 0.944. That DeepFilterNet2 table measured RNNoise's 2022 model at 2.33 and 0.922; the current model scores lower here.
+Over the noisy input, paired, with 95% intervals: DeepFilterNet3 adds 1.20 ± 0.03 PESQ, 0.024 ± 0.002 STOI and 10.5 ± 0.3 dB SI-SDR, and beats every classical row's PESQ on 804 of 824 files; RNNoise adds 0.14 ± 0.04 PESQ and 3.8 ± 0.3 dB SI-SDR but loses 0.031 ± 0.004 STOI; `omlsa`, the best classical row, adds 0.43 ± 0.03 PESQ and 6.1 ± 0.2 dB SI-SDR and loses 0.006 STOI. A limit leaves noise in on this set, and spares the voice: paired, on a quarter of the files (206), DNSMOS SIG moves −0.002 ± 0.015 from 12 to 18 dB, then −0.008 ± 0.003 to 20, −0.019 ± 0.006 more to 24 and −0.049 ± 0.017 more unlimited, while BAK rises 0.18, 0.04, 0.07 and 0.16 and PESQ 0.25, 0.06, 0.08 and 0.10. Past 18 dB the speech itself loses, which DNSMOS reads as SIG and a listener as a filtered, phone-call voice; from 8 to 16 kHz the unlimited output carries 1.5 dB less than the clean speech, at 18 dB 0.9 (all 824 files). 18 dB alone added 0.93 PESQ over the input and 0.50 over `omlsa`; from 0.5 the floor takes the noise that 18 dB would leave within 40 dB of the voice further down, on this set most of it (2.5 to 17.5 dB SNR): +0.15 PESQ, +0.17 BAK, +0.06 OVRL, −0.03 SIG against `floor: 0` (bench/speech.mjs `dfn-18` against `dfn-f0`, both through `audio`'s op). Pass `limit: 0` where the most noise removal matters more than room tone. The level change and the voice guard cost 0.01 PESQ unlimited and 0.004 at 18 dB. The same scoring gives upstream's published anchors: noisy 1.97 and 0.921 (as the DeepFilterNet2 paper's table), DeepFilterNet3 3.16 and 0.945 against the DeepFilterNet3 paper's 3.17 and 0.944. That DeepFilterNet2 table measured RNNoise's 2022 model at 2.33 and 0.922; the current model scores lower here.
 
 - **RNNoise's current model sometimes removes the voice.** STOI drops by more than 0.1 on 68 of 824 files and by more than 0.2 on 12; on p232_354 it goes from 0.870 to 0.154, the output 24 dB under the clean level while RNNoise's own voice probability averages 0.84. Upstream's C does the same, the portable and the NEON build alike (0.154, 0.153). DeepFilterNet3 never loses more than 0.066 STOI, `wiener` 0.065, `omlsa` 0.059. A `limit` caps it: at the default 16 dB (the input mixed back at 0.16) losses over 0.1 fall to 4 files and over 0.2 to none (p232_354 keeps 0.824), and PESQ rises to 2.49, +0.53 ± 0.02 over the noisy input, at a BAK of 3.66 instead of 3.85. Against 0.2's 20 dB, paired: PESQ +0.029 ± 0.009, STOI +0.004, SIG +0.026 ± 0.004, BAK −0.047 ± 0.005 (6 files over 0.1, p232_354 0.786). `denoise()`, the worklet and the atom apply that limit unless given another.
 - **The recipe's delivery stages are not denoisers.** Its compressor, EQ, de-esser and loudness target change the signal against the clean reference on purpose: the full recipe scores under its own denoising part. SI-SDR also falls with the 80 Hz high-pass: on its own it takes clean speech to 6.2 dB SI-SDR at PESQ 4.45 (21 test utterances). The table says what each stage does to speech in noise, not how finished a podcast sounds.
@@ -257,10 +263,10 @@ On the test set, paired, at the default 18 dB: PESQ +0.11 ± 0.01 at 16 kHz and
 | DeepFilterNet3, upstream's settings, `limit: 0` | 3.47 | 4.12 | 3.21 | −152 to −61 (−83) | −0.4 dB |
 | DeepFilterNet3, `denoise()`, `limit: 0` | 3.47 | 4.11 | 3.21 | −165 to −60 (−78) | −0.4 dB |
 | DeepFilterNet3, `denoise()` 0.2, `limit: 18` | 3.47 | 4.11 | 3.21 | −93 to −59 (−73) | −0.4 dB |
-| DeepFilterNet3, `denoise()`, `limit: 18`, the default | 3.47 | 4.11 | 3.21 | −93 to −59 (−73) | −0.4 dB |
+| DeepFilterNet3, `denoise()`, `limit: 18`, the default (0.5's `floor: 40` moves DNSMOS by 0.001) | 3.47 | 4.11 | 3.21 | −93 to −59 (−73) | −0.4 dB |
 | DeepFilterNet3, `limit: 12` | 3.47 | 4.10 | 3.20 | −91 to −58 (−68) | −0.3 dB |
 
-These rooms are mostly quiet (raw OVRL 3.17), so the gains are small. OM-LSA keeps SIG and raises BAK a little less than the models (0.3.11's lowered SIG by 0.20); both models raise BAK, and at their default limits keep SIG. Unlimited, both can take pauses to dead silence (RNNoise's median floor is −112 dBFS, DeepFilterNet3's lowest −165), which ACX Check flags; the default limits keep the floors between −98 and −59 dBFS at the same OVRL or better, room tone left in. Eight of the narrations end between 16.7 and 19.5 kHz and hear 0.3's floor in their top bands (one ends at 21.1 kHz, inside the top band, one runs to 24): DNSMOS moves by 0.008 at most on any file. On lena (a film scene with music under the voice) RNNoise lowers OVRL from 2.76 to 2.47 unlimited and keeps 2.78 at 16 dB; DeepFilterNet3 raises it to 2.83 upstream, 2.89 at the default limit.
+These rooms are mostly quiet (raw OVRL 3.17), so the gains are small, and 0.5's floor changes them by 0.001 DNSMOS (it takes the noisiest room's floor from −59.1 to −62.7 dBFS, under ACX's −60; bench/speech.py's scoring, Measured). OM-LSA keeps SIG and raises BAK a little less than the models (0.3.11's lowered SIG by 0.20); both models raise BAK, and at their default limits keep SIG. Unlimited, both can take pauses to dead silence (RNNoise's median floor is −112 dBFS, DeepFilterNet3's lowest −165), which ACX Check flags; the default limits keep the floors between −98 and −59 dBFS at the same OVRL or better, room tone left in. Eight of the narrations end between 16.7 and 19.5 kHz and hear 0.3's floor in their top bands (one ends at 21.1 kHz, inside the top band, one runs to 24): DNSMOS moves by 0.008 at most on any file. On lena (a film scene with music under the voice) RNNoise lowers OVRL from 2.76 to 2.47 unlimited and keeps 2.78 at 16 dB; DeepFilterNet3 raises it to 2.83 upstream, 2.89 at the default limit.
 
 **Singing.** VocalSet (Wilkins et al., ISMIR 2018, CC BY 4.0): 40 long tones (straight, forte, pianissimo, messa di voce; 20 singers) and 20 sung phrases (straight and vibrato), alone and in living-room noise (DEMAND, from the VoiceBank+DEMAND test set) at 15 dB SNR; level change of the voice over the frames within 20 dB of the loudest. At the default 18 dB, against 0.1's processing at the same limit (heard as it is, no voice guard): held notes −1.9 dB clean and −3.4 dB in noise, 14% of frames more than 6 dB down, against −14.2 and −16.2 dB, 96% (at 10 dB SNR −4.8 against −15.6); sung phrases −0.9 and −2.0 dB against −10.2 and −14.9; the same singers' spoken phrases −0.3 and −0.8 dB against −1.7 and −3.7 (they are quiet recordings: the −20 dBFS hearing helps them). Upstream's unlimited output takes the held notes 38.7 and 37.4 dB down. The noise in the pauses drops as before (−14.1 dB against −14.3), and a 100 or 120 Hz buzz 20 dB under the voice drops in them as without the guard (−17.2 dB), in quiet and in cafe noise: it is the background, not over it.
 
@@ -268,6 +274,38 @@ These rooms are mostly quiet (raw OVRL 3.17), so the gains are small. OM-LSA kee
 
 To rerun: `python scripts/accuracy.py prepare` writes the band-limited inputs and decodes the MUSDB18 previews; `python scripts/accuracy.py guard-sets` writes the music guard's labelled sets and `node scripts/accuracy.mjs guard train|test` measures it (systems `rnnoise-guard` and `dfn3-guard` are 0.4's defaults, the other rows take `music: 'enhance'`); `node scripts/accuracy.mjs SET SYSTEMS` (SET: `vbdemand`, `vbtrain`, `vbclean`, `vbtrain-clean`, either noisy set `@16000`, `@44100` or `-lp16k`, `music`) and `node scripts/accuracy.mjs rooms` write the outputs (the data sources and checksums are in its header); `python scripts/accuracy.py SET SYSTEMS`, `limits SET rnnoise 0,12,14,15,16,18,20`, `clean SET SYSTEM [LIMITS]`, `music SYSTEMS`, `presence SYSTEM` and `rooms` score them; `scripts/deepfilter-reference.py --vbdemand` writes the Python row; the classical rows come from `@audio/denoise`'s `node scripts/speech.mjs vbdemand|rooms SYSTEMS` and `python scripts/speech.py score vbdemand|rooms SYSTEMS`. Per-file scores stay in `~/.cache/audiojs/data/` (`vbdemand/scores/neural-*.csv`, `vbdemand-train/scores/neural-*.csv`, `spoken/scores.json`); the scripts regenerate outputs, skipping any already there.
 
+## Measured: against iZotope RX 12 Dialogue Isolate
+
+October 2026: RX 12 Advanced's Dialogue Isolate plug-in (VST3 hosted through Pedalboard 0.9, `audio`'s `bench/rx/host.py`) and `audio`'s `deepfilter()` on the same buffers. RX at its defaults with the noise off (noise gain −∞; dialogue and reverb gain 0 dB, sensitivity 5, the four band amounts 100%), and tuned: the reverb off too. Ours at its defaults, 0.4 (`floor: 0` and no speech prior: `audio`'s op bit for bit) and 0.5. Every setting, RX's and ours, was chosen on the tune split (VoiceBank's training speakers, MUSDB18's training songs, the even-numbered MIT rooms, DEMAND's first 150 s, the spoken-train narrations): RX's sensitivity 0, 2.5, 5, 7.5 and 10 and its reverb off tried, the reverb off with sensitivity 5 the best mean PESQ (1.88) and OVRL (2.71) over the speech conditions; our floor 0, 35, 40 and 45 tried (mean PESQ 1.75, 1.96, 1.98, 1.98; VoiceBank's training speakers 2.24, 2.38, 2.43, 2.46 at SIG 3.41, 3.38, 3.36, 3.35; unlimited 2.57, SIG 3.34), 40 kept: SIG is the voice, and 40 dB under it is ACX's −60 dBFS floor for a voice at −20. Then the test split once.
+
+**Harder mixtures**, `audio`'s `bench/rx/isolate.mjs`, test split: VoiceBank's clean test takes (every 8th, 103 per row) at −26 dBFS under DEMAND noises VoiceBank+DEMAND does not use (washing machine, field, park, river, hallway) at −5, 0 and 5 dB SNR (active levels), six Spoken Wikipedia narrators at once, MUSDB18 test accompaniments (drums, bass, other), and MIT IR Survey rooms (odd-numbered, a tail to hear) with the noise at 5 dB, the reference there the direct sound and 50 ms. Scored at 16 kHz: PESQ / DNSMOS OVRL (at the reference's loudness), the last column paired against RX tuned with 95% intervals.
+
+| | RX default | RX tuned | 0.4 | **0.5** | 0.5 − RX tuned, PESQ / OVRL |
+|---|---|---|---|---|---|
+| noise, −5 dB | 2.00 / 2.79 | 2.00 / 2.78 | 1.73 / 2.77 | **2.11 / 2.91** | +0.11 ± 0.05 / +0.12 ± 0.03 |
+| noise, 0 dB | 2.28 / 2.95 | 2.27 / 2.95 | 2.07 / 2.96 | **2.43 / 3.04** | +0.16 ± 0.05 / +0.09 ± 0.03 |
+| noise, 5 dB | 2.56 / 3.07 | 2.56 / 3.07 | 2.44 / 3.05 | **2.70 / 3.10** | +0.15 ± 0.07 / +0.03 ± 0.03 |
+| babble, 0 dB | 1.18 / 2.11 | 1.18 / 2.16 | 1.24 / 2.31 | **1.31 / 2.41** | +0.12 ± 0.03 / +0.25 ± 0.06 |
+| babble, 5 dB | 1.43 / 2.47 | 1.44 / 2.51 | 1.58 / 2.67 | **1.70 / 2.73** | +0.26 ± 0.04 / +0.22 ± 0.05 |
+| music bed, 0 dB | 1.65 / 2.74 | 1.65 / 2.74 | 1.26 / 2.10 | 1.63 / 2.33 | −0.02 ± 0.09 / −0.41 ± 0.17 |
+| music bed, 5 dB | 1.96 / 2.90 | 1.96 / 2.90 | 1.82 / 2.94 | **2.26 / 3.03** | +0.30 ± 0.05 / +0.13 ± 0.05 |
+| room, noise 5 dB | 1.66 / 2.21 | 1.80 / 2.51 | 1.62 / 2.28 | 1.77 / 2.42 | −0.03 ± 0.05 / −0.09 ± 0.07 |
+| mean | 1.84 / 2.66 | 1.86 / 2.70 | 1.72 / 2.63 | **1.99 / 2.75** | |
+
+SI-SDR follows PESQ but under steady noise, where RX keeps the waveform closer (13.2, 15.8, 17.8 dB against 12.7, 15.3, 17.2); in babble ours is the closer (3.5 and 8.9 against 1.7 and 8.4). DNSMOS SIG, the voice: ours 3.21 to 3.39 under noise, RX's 3.07 to 3.33.
+
+**VoiceBank+DEMAND** test set (824, `audio`'s `bench/speech.mjs` and `bench/speech.py`, scored as in Accuracy): RX default PESQ 2.73, STOI 0.929, SI-SDR 18.9 dB, SIG 3.30, BAK 3.82, OVRL 2.94; tuned 2.72, 0.929, 18.8, 3.30, 3.83, 2.94; ours 0.4 2.90, 0.942, 17.9, 3.48, 3.84, 3.10; **0.5 3.05, 0.944, 18.9, 3.45, 4.01, 3.15**. **The ten narrations** (Accuracy, Real rooms): raw OVRL 3.18; RX 3.23 (tuned 3.23), every room's floor (its quietest 500 ms) taken under −79.6 dBFS (tuned −88.2), room tone gone; ours 3.23 at 0.4 and 0.5, room tone kept, the noisiest room's floor at −62.7 dBFS (0.4: −59.1).
+
+**Nothing to isolate.** The clean test takes: RX raises them 1.4 dB and leaves PESQ 3.98 and SI-SDR 23.6 dB against the take (tuned the same); ours −0.1 dB, PESQ 4.60, SI-SDR 41.2 dB (OVRL 3.24 against RX's 3.30: RX also takes the studio's own faint noise). Music alone, 24 MUSDB18 test previews, songs and accompaniments in turn: RX takes it 37.6 dB down (every band 33 to 57 dB), its median SI-SDR to the input −16.8 dB; ours passes all 24 bit for bit.
+
+What 0.5 changed, and where RX still wins:
+
+- **The floor.** 18 dB of attenuation leaves noise as loud as the voice 18 dB under it, and RX takes it all: before 0.5 RX won every noise row. Unlimited, the model beat RX already (tune split, −5 dB: PESQ 2.12 against 1.98); the floor keeps that where the noise stands near the voice and the 18 dB limit's room tone where it is quiet. +0.38, +0.36, +0.26 PESQ at −5, 0, 5 dB over 0.4.
+- **The guard's prior.** Speech under a bed as loud as itself reached ina as music, so 0.4 passed half of those takes untouched (PESQ 1.26). With speech favoured by 1 nat a patch, as the stream already decided, 37% of their frames pass (Music): PESQ 1.63 against RX's 1.65, OVRL 2.33 against 2.74. What still passes is a voice ina cannot tell from a singer at that level: the model keeps the voice of a song as it keeps this one, 4 to 7 dB under the mix, and ina hears speech in both, so passing less would gut songs (tune split: a voice under a bed at its level kept −3.5 to −5.9 dB and heard as speech at 0.81 or more; the training songs, at most −4.6 dB and 1.00). `music: 'enhance'` enhances it all: PESQ 1.87, OVRL 2.91, SI-SDR 11.0 dB, ahead of RX on all three, and music alone then loses 28.5 dB (RX 37.6).
+- **Rooms.** RX's reverb gain takes the late reverberation (SI-SDR 7.2 dB to the direct sound and 50 ms, against ours 5.7; at its default, reverb kept, 5.2). DeepFilterNet3 trained with reverberant speech in 10% of its mixtures (its `config.ini`, `p_reverb`), and takes part of it. `audio`'s `deepfilter()` now runs it again where its output still carries a room (`@audio/denoise-dereverb` 0.6's checks, asked of the output and of the take), on the take with the room's linear prediction taken off (that package's WPE alone): 81 of the 103 room takes, PESQ 1.81, OVRL 2.44, SI-SDR 6.2 dB (RX tuned 1.80, 2.51, 7.2; paired +0.01 ± 0.05 PESQ, −0.08 ± 0.07 OVRL); the music bed at 0 dB 1.67 / 2.44, 13 takes heard again; the other rows, clean speech, music alone, VoiceBank+DEMAND (2 of 824) and the narrations (2 of 10) as above. Chosen on the tune split, reverb5: WPE before the model PESQ 1.88 → 1.96, SI-SDR 7.3 → 7.9; `dereverb()` whole after the model 1.89, before it 1.87 (OVRL −0.09); MossFormer2_SE_48K (ClearerVoice-Studio, Apache-2.0, trained with reverberant speech in 30% of its mixtures; its 221 MB PyTorch checkpoint) 1.80, OVRL 2.35, SI-SDR 6.3. Still behind RX: OVRL and SI-SDR in rooms, the late tail the model and the linear prediction leave.
+
+To rerun: in `audio`, `node bench/rx/isolate.mjs test all rx-di rx-di-tuned dfn` (renders kept in `~/.cache/audiojs/data/rx/isolate/`), `node bench/speech.mjs test rx-di,rx-di-tuned,dfn-18` then `python bench/speech.py test rx-di,rx-di-tuned,dfn-18`; tune with `tune` in place of `test`.
+
 ## Speed and memory
 
 Apple M4 Max running other jobs (load average 28 to 50 on 14 cores), so wall times ran up to 10 times CPU times; CPU times are given.
@@ -317,7 +355,7 @@ The package is BSD-3-Clause (the RNNoise translation and weights); `deepfilter.j
 J.-M. Valin, "A Hybrid DSP/Deep Learning Approach to Real-Time Full-Band Speech Enhancement", MMSP 2018, [arXiv:1709.08243](https://arxiv.org/abs/1709.08243). · H. Schröter, T. Rosenkranz, A. N. Escalante-B., A. Maier, "DeepFilterNet: Perceptually Motivated Real-Time Speech Enhancement", Interspeech 2023, [arXiv:2305.08227](https://arxiv.org/abs/2305.08227). · C. Valentini-Botinhao, "Noisy speech database for training speech enhancement algorithms and TTS models", University of Edinburgh, 2017, [doi:10.7488/ds/2117](https://doi.org/10.7488/ds/2117). · ITU-T P.862.2 (2007), wideband PESQ. · C. H. Taal, R. C. Hendriks, R. Heusdens, J. Jensen, "An Algorithm for Intelligibility Prediction of Time-Frequency Weighted Noisy Speech", IEEE TASLP 19(7), 2011. · J. Le Roux, S. Wisdom, H. Erdogan, J. R. Hershey, "SDR: Half-baked or Well Done?", ICASSP 2019. · C. K. A. Reddy, V. Gopal, R. Cutler, "DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors", ICASSP 2022. · S. A. Figueroa, "When is double rounding innocuous?", SIGNUM Newsletter 30(3), 1995. · [xiph/rnnoise](https://gitlab.xiph.org/xiph/rnnoise) at 70f1d25 · [Rikorose/DeepFilterNet](https://github.com/Rikorose/DeepFilterNet) at d375b2d. · D. Doukhan, J. Carrive, F. Vallet, A. Larcher, S. Meignier, "An Open-Source Speaker Gender Detection Framework for Monitoring Gender Equality", ICASSP 2018 · [ina-foss/inaSpeechSegmenter](https://github.com/ina-foss/inaSpeechSegmenter), models release.
 
 **Use when:** speech with noise a statistical denoiser leaves or smears (moving noise, a busy room); live voice in a browser (RNNoise, 30 ms); files where quality is worth an 8 MB download (DeepFilterNet3); programmes that mix speech and music (the music passes through untouched).<br>
-**Not for:** denoising music itself, a noisy song or concert (music passes untouched; with `music: 'enhance'` both models take 8 to 16 dB from every band); singing alone (half to two thirds of it passes, the rest is enhanced); clean speech with nothing to remove through RNNoise (it gates word ends and quiet syllables, 9 to 12 dB at its default limit); installs that must stay model-free ([`@audio/denoise`](https://github.com/audiojs/denoise)); reverb as such (`dereverb`); live DeepFilterNet3 (upstream's graphs take no state).
+**Not for:** denoising music itself, a noisy song or concert (music passes untouched; with `music: 'enhance'` both models take 8 to 16 dB from every band); singing alone (about half of it passes, the rest is enhanced); clean speech with nothing to remove through RNNoise (it gates word ends and quiet syllables, 9 to 12 dB at its default limit); installs that must stay model-free ([`@audio/denoise`](https://github.com/audiojs/denoise)); reverb as such (`dereverb`); live DeepFilterNet3 (upstream's graphs take no state).
 
 ---
 

package/audio.js

@@ -6,7 +6,7 @@
 // quanta only), so the delay is constant: 960 + 479 = 1439 samples at 48 kHz, 30 ms. Other rates go to
 // 48 kHz and back through a streaming form of @audio/resample-sinc's Lanczos kernel, the resampler denoise()
 // uses offline; its lookahead adds to the delay (1354 samples at 44.1 kHz, 30.7 ms). The output is
-// denoise()'s: at 48 kHz sample for sample; at other rates, with resample-sinc 1.2.0, except within 50 ms of
+// denoise()'s: at 48 kHz sample for sample; at other rates, with resample-sinc 1.3.0, except within 50 ms of
 // the end, where the offline resampler sees the end and a stream sees the host's trailing silence.
 //
 // limit (dB): the input is mixed back in at 10^(−limit/20), so noise drops by at most `limit`; 0 lifts it
@@ -25,6 +25,7 @@
 // Channels are denoised independently; a channel's state (70 KB, and the guard's 0.3 MB) is created on its first
 // block, as the worklet does, since hosts declare up to 32 channels.
 
+import { phases } from '@audio/resample-sinc'
 import { model, create, FRAME, LIMIT } from './rnnoise.js'
 import { weights } from './denoise.js'
 import { guardNet, online, ramp, blend } from './guard.js'
@@ -47,18 +48,34 @@ const latency = sr => sr === RATE ? DELAY + PRIME : Math.floor((taps(RATE, sr) +
 
 /**
  * @audio/resample-sinc's resample() (Lanczos, a = 16) as a stream, in its arithmetic, so its samples: `push(x)`
- * calls `emit(v)` for every output whose taps have all arrived. Output i sits at input position i·from/to;
- * taps before the start are left out and the weights renormalized, as resample() does at its edges.
+ * calls `emit(v)` for every output whose taps have all arrived. Output i sits at input position i·from/to; at integer
+ * rates it reads resample()'s phase table, at others it weighs by angle addition as resample() does there; taps before
+ * the start are left out and the weights renormalized, as resample() does at its edges.
  */
 function resampler(from, to, emit) {
 	let rate = from / to, scale = rate > 1 ? 1 / rate : 1, T = Math.ceil(HALF / scale), K = HALF / (Math.PI * Math.PI)
+	// input history: an output reads 2T samples, the newest of them just arrived
+	let mask = (1 << Math.ceil(Math.log2(2 * T + 1))) - 1, ring = new Float32Array(mask + 1), n = 0, i = 0
+	let tab = phases(from, to)
+	if (tab) {
+		let { L, M, W, J0, J1 } = tab, S = 2 * T, p = 0, base = 0
+		return x => {
+			for (let s = 0; s < x.length; s++) {
+				ring[n++ & mask] = x[s]
+				while (base + T < n) {
+					let lo = base + 1 - T, o = p * S - lo, sum = 0, w = 0
+					for (let j = Math.max(0, lo + J0[p]), e = lo + J1[p]; j < e; j++) { let k = W[o + j]; sum += ring[j & mask] * k; w += k }
+					emit(Math.fround(w !== 0 ? sum / w : 0))
+					if ((p += M) >= L) { let q = Math.floor(p / L); base += q; p -= q * L }
+				}
+			}
+		}
+	}
 	let A = new Float64Array(2 * T), B = new Float64Array(2 * T), C = new Float64Array(2 * T), D = new Float64Array(2 * T)
 	for (let j = 0, t = 1 - T; t <= T; t++, j++) {
 		let a = Math.PI * scale * t
 		A[j] = Math.sin(a); B[j] = Math.cos(a); C[j] = Math.sin(a / HALF); D[j] = Math.cos(a / HALF)
 	}
-	// input history: an output reads 2T samples, the newest of them just arrived
-	let mask = (1 << Math.ceil(Math.log2(2 * T + 1))) - 1, ring = new Float32Array(mask + 1), n = 0, i = 0
 	return x => {
 		for (let s = 0; s < x.length; s++) {
 			ring[n++ & mask] = x[s]

package/deepfilter.js

@@ -38,6 +38,12 @@ export const MODEL = 'https://raw.githubusercontent.com/Rikorose/DeepFilterNet/d
 // rise; unlimited, the pauses of home narrations fall to digital silence (README, Accuracy).
 export const LIMIT = 18
 
+// The floor, dB under the voice, to which denoise() takes noise the limit would leave closer to it (mixback() in
+// denoise.js): the limit alone keeps room tone, and keeps noise as loud as the voice 18 dB under it. 40 dB under a voice
+// at -20 dBFS is ACX's -60 dBFS floor; on audio's bench/rx/isolate.mjs tune split 35, 40 and 45 score PESQ 1.96, 1.98,
+// 1.98 (0: 1.75), on VoiceBank+DEMAND's training speakers DNSMOS SIG 3.38, 3.36, 3.35 (0: 3.41) (README, Measured).
+export const FLOOR = 40
+
 // The level, dBFS, at which denoise() has the network hear the speech (`gain`): the median of the VoiceBank+DEMAND
 // test set it scores its published PESQ on (level(), -20.2); its features are not level-free (README, DeepFilterNet3).
 export const LEVEL = -20
@@ -136,9 +142,9 @@ const linspace = (a, b, n) => { let step = f(f(f(b) - f(a)) / (n - 1)); return F
 
 // ------------------------------------------------ input analysis
 
-// level(x) → the speech level, dBFS: mean power of the louder half of 50 ms frames at 48 kHz (LEVEL where there is none)
-export function level(x) {
-	let F = 2400, p = []
+// level(x, rate) → the speech level, dBFS: mean power of the louder half of 50 ms frames (LEVEL where there is none)
+export function level(x, rate = 48000) {
+	let F = Math.round(rate / 20), p = []
 	for (let i = 0; i + F <= x.length; i += F) { let s = 0; for (let j = i; j < i + F; j++) s += x[j] * x[j]; p.push(s / F) }
 	p.sort((a, b) => b - a)
 	let k = Math.max(1, p.length >> 1), s = 0

package/denoise.js

@@ -5,13 +5,13 @@
 
 import resample from '@audio/resample-sinc'
 import { model as parseRNNoise, create, FRAME, LIMIT as RNNOISE_LIMIT } from './rnnoise.js'
-import { loadDeepFilter, enhance, level, bandEdge, MODEL, LEVEL, LIMIT as DFN_LIMIT } from './deepfilter.js'
+import { loadDeepFilter, enhance, level, bandEdge, MODEL, LEVEL, LIMIT as DFN_LIMIT, FLOOR } from './deepfilter.js'
 import { guardNet, offline, online, gains, blend } from './guard.js'
 
 export { create as rnnoise, FRAME } from './rnnoise.js'
 export { MODEL } from './deepfilter.js'
 
-const RATE = 48000
+const RATE = 48000, SMOOTH = 0.5
 // the attenuation limit (dB) each model gets when opts.limit is not given; 0 is upstream's output
 const LIMITS = { rnnoise: RNNOISE_LIMIT, deepfilternet3: DFN_LIMIT }
 
@@ -51,17 +51,18 @@ export default async function denoise(audio, opts = {}) {
 	if (music !== 'pass' && music !== 'enhance') throw new TypeError(`neural-denoise: music is 'pass' or 'enhance', not ${music}`)
 	let handle = typeof opts.model === 'object' && opts.model ? opts.model : await load(opts.model, opts)
 	try {
-		let out = [], limit = opts.limit ?? LIMITS[handle.model], nn = music === 'pass' ? await guardNet() : null
+		let out = [], limit = opts.limit ?? LIMITS[handle.model], nn = music === 'pass' ? await guardNet() : null, dfn = handle.model !== 'rnnoise'
 		for (let ch of channels) {
 			let x = rate === RATE ? Float32Array.from(ch) : resample(Float32Array.from(ch), { from: rate, to: RATE }), y, pass
 			// RNNoise decides as its stream does (guard.js online()), DeepFilterNet3 with the whole input in view
-			if (handle.model === 'rnnoise') ({ y, pass } = rnnoiseOffline(x, handle.net, limit, nn))
+			if (!dfn) ({ y, pass } = rnnoiseOffline(x, handle.net, limit, nn))
 			else {
 				pass = nn && offline(x, nn)
-				y = pass?.every(Boolean) ? x : await enhance(x, handle.net, { ...opts, limit, gain: 10 ** ((LEVEL - level(x)) / 20), edge: bandEdge(ch, rate), voice: true })
+				y = pass?.every(Boolean) ? x : await enhance(x, handle.net, { ...opts, limit: 0, gain: 10 ** ((LEVEL - level(x)) / 20), edge: bandEdge(ch, rate), voice: true })
 			}
 			if (rate !== RATE) y = fit(resample(y, { from: RATE, to: rate }), ch.length)
-			out.push(pass ? mix(y, ch, gains(pass, x.length, handle.model !== 'rnnoise'), rate) : y)
+			if (pass) y = mix(y, ch, gains(pass, x.length, dfn), rate)
+			out.push(dfn ? mixback(ch, y, { limit, floor: opts.floor, sampleRate: rate }) : y)
 		}
 		return audio instanceof Float32Array ? out[0] : Array.isArray(audio) ? out : { ...audio, channelData: out, sampleRate: rate }
 	} finally {
@@ -69,6 +70,51 @@ export default async function denoise(audio, opts = {}) {
 	}
 }
 
+/**
+ * mixback(x, y, { limit, floor, sampleRate }) → y + a·(x − y): the input x mixed back into a model's output y, the noise
+ * the model took (x − y) back at gain a. a = 10^(−limit/20), so the noise drops by `limit` dB and room tone stays; where
+ * the noise stands higher than `floor` dB under the voice, a falls until it lies there, so noise loud against the voice
+ * is not merely turned down. Per 10 ms frame: the noise's mean power over the 0.5 s around it against the voice's
+ * level, the louder half of y's 50 ms frames (level()); a interpolated between frames. Where x and y agree (music the
+ * guard passed) the output is y bit for bit. limit 0: y itself, the model's whole removal; floor 0: a constant
+ * 10^(−limit/20), as before 0.5. denoise() applies it to DeepFilterNet3; `audio`'s deepfilter op to the unlimited
+ * output it keeps.
+ */
+export function mixback(x, y, { limit = DFN_LIMIT, floor = FLOOR, sampleRate = RATE } = {}) {
+	let n = x.length, out = new Float32Array(n), cap = limit ? 10 ** (-Math.abs(limit) / 20) : 0
+	if (!cap) { out.set(y.subarray(0, n)); return out }
+	if (!floor) { for (let i = 0; i < n; i++) out[i] = y[i] + cap * (x[i] - y[i]); return out }
+	// the noise's energy per frame, summed (c[k]: frames before k), and the frames' mean over the window around each
+	let F = Math.max(1, Math.round(sampleRate / 100)), K = Math.ceil(n / F), c = new Float64Array(K + 1), g = new Float64Array(K), h = Math.round(50 * SMOOTH)
+	for (let k = 0; k < K; k++) {
+		let s = 0, e = Math.min(n, (k + 1) * F)
+		for (let i = k * F; i < e; i++) s += (x[i] - y[i]) ** 2
+		c[k + 1] = c[k] + s
+	}
+	let voice = 10 ** ((level(y, sampleRate) - Math.abs(floor)) / 10)
+	for (let k = 0; k < K; k++) {
+		let a = Math.max(0, k - h), b = Math.min(K, k + h + 1)
+		g[k] = Math.min(cap, Math.sqrt(voice / Math.max((c[b] - c[a]) / (Math.min(n, b * F) - a * F), 1e-30)))
+	}
+	for (let i = 0; i < n; i++) {
+		let t = i / F - .5, k = Math.min(K - 1, Math.max(0, Math.floor(t))), w = Math.min(1, Math.max(0, t - k))
+		out[i] = y[i] + (k + 1 < K ? g[k] + w * (g[k + 1] - g[k]) : g[k]) * (x[i] - y[i])
+	}
+	return out
+}
+
+/**
+ * guard(x, sampleRate) → per sample of x, the gain an enhancement of it takes: 1 where it is speech or noise, 0 where it
+ * is music, ramped over RAMP at 48 kHz (200 ms) centered on each change: ina's segments over the whole input, as
+ * denoise() decides for DeepFilterNet3 (guard.js offline()). Blended g·y + (1 − g)·x, a model's output y passes music
+ * untouched; a host guards any enhancement so (audio's dereverb).
+ */
+export async function guard(x, sampleRate = RATE) {
+	let nn = await guardNet(), x48 = sampleRate === RATE ? x : resample(Float32Array.from(x), { from: sampleRate, to: RATE })
+	let g = gains(offline(x48, nn), x48.length, true)
+	return sampleRate === RATE ? g : fit(resample(g, { from: RATE, to: sampleRate }), x.length)
+}
+
 // the enhanced y where the gain g (at 48 kHz) is 1, the input x where it is 0: music passes untouched
 function mix(y, x, g, rate) {
 	if (rate !== RATE) g = fit(resample(g, { from: RATE, to: rate }), x.length)

package/guard.js

@@ -13,12 +13,15 @@
 //
 // A patch every STEP frames (ina takes one every 2; on the training material every 10 decides the same, at a fifth of
 // the work). Offline, the classes become segments as ina makes them, by Viterbi over the whole input: three states,
-// a switch costing 10^-80 per 20 ms (ina's SpeechMusicNoise, diag_trans_exp(80)). A stream can only use what has
-// arrived: the forward recursion of the same chain, for two states (music, the rest), keeps the log-odds of music,
-// bounded at ±BOUND, a patch moving them by at most ln(1/FLOOR), each patch PRIOR nats in favour of speech; it
-// switches to music above HYST and back below 0 (hysteresis). Either way, music passes (the enhanced signal's gain 0),
-// speech and noise are enhanced (gain 1), and the gain moves over RAMP samples, raised cosine: offline centered on the
-// segment boundary, streaming from the decision on.
+// a switch costing 10^-80 per 20 ms (ina's SpeechMusicNoise, diag_trans_exp(80)), each patch PRIOR nats in favour of
+// speech (from 0.5): a voice under a music bed as loud as itself reaches ina as music half the time, speech at a mean
+// probability of 0.47, songs at 0.01; PRIOR is the most that passes every frame of the training songs and
+// accompaniments passed without it (README, Music). A stream can only use what has arrived: the forward recursion of
+// the same chain, for two states (music, the rest), keeps the log-odds of music, bounded at ±BOUND, a patch moving them
+// by at most ln(1/FLOOR), each patch PRIOR nats in favour of speech; it switches to music above HYST and back below 0
+// (hysteresis). Either way, music passes (the enhanced signal's gain 0), speech and noise are enhanced (gain 1), and
+// the gain moves over RAMP samples, raised cosine: offline centered on the segment boundary, streaming from the
+// decision on.
 //
 // The products of the three upper convolutions and the dense layers run in a WebAssembly SIMD kernel (gemm.wat) or
 // its JS form, the same float ops in the same order, so the same bits either way.
@@ -281,10 +284,11 @@ export function offline(x, nn) {
 	let a = analyzer(nn), P = [...a(x), ...a.end()], T = P.length, n = Math.ceil(x.length / 480), pass = new Uint8Array(n)
 	if (!T) return pass
 	// V: the best path's log-likelihood ending in each state (speech, music, noise); B: the state it came from. Every
-	// switch costs COST; a patch of silence scores all states alike. Ties go to the lower state, as numpy's argmax.
+	// switch costs COST, speech gains PRIOR a patch; a patch of silence scores all states alike. Ties go to the lower
+	// state, as numpy's argmax.
 	let V = [0, 0, 0], B = new Uint8Array(T * 3)
 	for (let t = 0; t < T; t++) {
-		let e = P[t] ? P[t].map(p => lp(p, 1e-10)) : [0, 0, 0], j = V.indexOf(Math.max(...V))
+		let e = P[t] ? P[t].map((p, s) => lp(p, 1e-10) + (s ? 0 : PRIOR)) : [0, 0, 0], j = V.indexOf(Math.max(...V))
 		V = V.map((v, s) => { let b = V[j] - COST > v ? j : s; B[t * 3 + s] = b; return (b === s ? v : V[j] - COST) + e[s] })
 	}
 	let music = new Uint8Array(T)

package/index.d.ts

@@ -28,8 +28,10 @@ export interface DenoiseOptions extends LoadOptions {
   sampleRate?: number
   /** 'rnnoise' (default), 'deepfilternet3', or a handle from load() to reuse across calls */
   model?: 'rnnoise' | 'deepfilternet3' | Model
-  /** attenuation limit in dB, the most the noise drops: mixes the input back in at 10^(−limit/20); default 16 for RNNoise, 18 for DeepFilterNet3 (README, API); 0 for none. DeepFilterNet3 also hears its input with the speech at −20 dBFS and, above a band-limited input's edge, a white noise floor 20 dB under it, and keeps sustained voicing it would remove (held sung notes) */
+  /** attenuation limit in dB, how far the noise drops: mixes the input back in at 10^(−limit/20) (DeepFilterNet3: less where the noise stands within `floor` of the voice); default 16 for RNNoise, 18 for DeepFilterNet3 (README, API); 0 for none. DeepFilterNet3 also hears its input with the speech at −20 dBFS and, above a band-limited input's edge, a white noise floor 20 dB under it, and keeps sustained voicing it would remove (held sung notes) */
   limit?: number
+  /** DeepFilterNet3: dB under the voice to which noise the limit would leave louder drops (mixback()), frame by frame; default 40, 0 for none (before 0.5: every noise down by the limit only) */
+  floor?: number
   /** 'pass' (default): speech and noise are enhanced, music (songs included) passes through untouched, as inaSpeechSegmenter's speech/music/noise CNN segments the input: DeepFilterNet3 with the whole input in view, RNNoise as its stream decides (from what has arrived: music's first second or so is denoised); the gain ramps over 200 ms at each switch. 'enhance': everything, as before 0.4 */
   music?: 'pass' | 'enhance'
   /** DeepFilterNet3: frames (10 ms) per model run, default 1000 */
@@ -43,6 +45,12 @@ export interface DenoiseOptions extends LoadOptions {
 /** Denoise speech; returns the same shape as `audio`, same length, aligned with it. */
 export default function denoise<T extends Audio>(audio: T, opts?: DenoiseOptions): Promise<T>
 
+/** The input x mixed back into a model's output y, y + a·(x − y): the noise drops by `limit` dB (default 18), or to `floor` dB under the voice (default 40) where it stands higher, per 10 ms frame; `limit` 0: y; `floor` 0: a constant 10^(−limit/20). denoise() applies it to DeepFilterNet3. */
+export function mixback(x: Float32Array, y: Float32Array, opts?: { limit?: number, floor?: number, sampleRate?: number }): Float32Array
+
+/** Per sample of x, the gain an enhancement of it takes: 1 where it is speech or noise, 0 where it is music (inaSpeechSegmenter's segments over the whole input, as denoise() decides), ramped over 200 ms; blend g·y + (1 − g)·x. */
+export function guard(x: Float32Array, sampleRate?: number): Promise<Float32Array>
+
 /** Load a model once to reuse it across denoise() calls. */
 export function load(model?: 'rnnoise' | 'deepfilternet3', opts?: LoadOptions): Promise<Model>
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@audio/neural-denoise",
-  "version": "0.4.0",
+  "version": "0.6.0",
   "description": "Neural speech enhancement: RNNoise (Valin 2018) in pure JS, bit-exact to its C, weights bundled; DeepFilterNet3 through ONNX Runtime, matching the Python original; music passes through untouched (inaSpeechSegmenter's speech/music/noise CNN)",
   "type": "module",
   "sideEffects": false,
@@ -39,7 +39,7 @@
   ],
   "dependencies": {
     "@audio/neural-runtime": "^0.1.0",
-    "@audio/resample-sinc": "^1.2.0"
+    "@audio/resample-sinc": "^1.3.0"
   },
   "devDependencies": {
     "audio-lena": "^3.0.1",