@audio/neural-denoise 0.3.0 → 0.5.0

package/NOTICE

@@ -41,3 +41,29 @@ The DeepFilterNet3 model is not included. deepfilter.js downloads it on first us
 DeepFilterNet repository at the same commit (models/DeepFilterNet3_onnx.tar.gz). Upstream's
 README licenses "all code in this repository" as MIT OR Apache-2.0 and states no separate terms
 for the model files; see README.md, "Licenses".
+
+This product includes the speech/music/noise network of inaSpeechSegmenter
+(https://github.com/ina-foss/inaSpeechSegmenter), MIT License: guard.bin holds the weights of its
+keras_speech_music_noise_cnn.hdf5 (release 'models', sha256 f04b5e3c…), each batch norm folded into
+the layer before it, as float16, written by scripts/guard.py; gemm.wat and guard.js run the network.
+guard.js computes the network's input as inaSpeechSegmenter does (its sidekit_mfcc.py, from SIDEKIT by
+Anthony Larcher and Sylvain Meignier, LGPL): the framing, the pre-emphasis, the window, the power
+spectrum and the mel filters' edges and areas, written anew from those definitions.
+
+    MIT License, Copyright (c) 2018 Ina (David Doukhan - http://www.ina.fr/)
+
+    Permission is hereby granted, free of charge, to any person obtaining a copy of this
+    software and associated documentation files (the "Software"), to deal in the Software
+    without restriction, including without limitation the rights to use, copy, modify, merge,
+    publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons
+    to whom the Software is furnished to do so, subject to the following conditions:
+
+    The above copyright notice and this permission notice shall be included in all copies or
+    substantial portions of the Software.
+
+    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
+    INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
+    PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE
+    FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+    OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+    DEALINGS IN THE SOFTWARE.

package/README.md

@@ -2,7 +2,7 @@
 
 > Neural speech enhancement: RNNoise ported to JS, bit-exact to its C, weights inside; DeepFilterNet3 through `@audio/neural-runtime`, matching the Python original.
 
-The neural tier beside [`@audio/denoise`](https://github.com/audiojs/denoise)'s statistical denoisers (`omlsa`, `wiener`, `specsub`). A statistical denoiser tracks a noise floor it assumes to be steady; these models learned what speech is, so they also remove noise that moves: keyboards, traffic, a fan changing speed, a busy room.
+The neural tier beside [`@audio/denoise`](https://github.com/audiojs/denoise)'s statistical denoisers (`omlsa`, `wiener`, `specsub`). A statistical denoiser tracks a noise floor it assumes to be steady; these models learned what speech is, so they also remove noise that moves: keyboards, traffic, a fan changing speed, a busy room. To them everything that isn't speech is noise, music too, so a speech/music classifier stands guard: music, songs included, passes through untouched ([Music](#music)).
 
 ```js
 import denoise from '@audio/neural-denoise'
@@ -17,7 +17,9 @@ let better = await denoise(pcm, { sampleRate: 44100, model: 'deepfilternet3' })
 |---|---|---|
 | `sampleRate` | | required unless `audio` carries it |
 | `model` | `'rnnoise'` | `'rnnoise'`, `'deepfilternet3'`, or a handle from `load()` |
-| `limit` | `16` RNNoise, `18` DeepFilterNet3 | attenuation limit, dB: the input is mixed back in at 10^(−limit/20), so noise drops by at most `limit`; `0` for none. Each default is the most before the voice itself suffers. RNNoise turns down speech that has nothing to remove (clean word ends 37 dB down unlimited, 12 at 16 dB) and removes the voice on some noisy files; on VoiceBank+DEMAND's training speakers its DNSMOS SIG holds from 14 to 16 dB and falls past it, and at 16 no file loses more than 0.2 STOI (21 of 504 unlimited, 3 at 20, the default before 0.3). DeepFilterNet3's SIG holds from 12 to 18 dB and falls past it, while BAK and PESQ keep rising; unlimited, it takes the pauses of home narrations to digital silence, which ACX Check flags ([Accuracy](#accuracy)) |
+| `limit` | `16` RNNoise, `18` DeepFilterNet3 | attenuation limit, dB: the input is mixed back in at 10^(−limit/20), so noise drops by `limit`; `0` for none. Each default is the most before the voice itself suffers. RNNoise turns down speech that has nothing to remove (clean word ends 37 dB down unlimited, 12 at 16 dB) and removes the voice on some noisy files; on VoiceBank+DEMAND's training speakers its DNSMOS SIG holds from 14 to 16 dB and falls past it, and at 16 no file loses more than 0.2 STOI (21 of 504 unlimited, 3 at 20, the default before 0.3). DeepFilterNet3's SIG holds from 12 to 18 dB and falls past it, while BAK and PESQ keep rising; unlimited, it takes the pauses of home narrations to digital silence, which ACX Check flags ([Accuracy](#accuracy)) |
+| `floor` | `40` | DeepFilterNet3: dB under the voice. Noise the limit would leave closer to the voice than that drops further, down to it, frame by frame (`mixback()`): quiet room tone drops by `limit` and stays, noise as loud as the voice no longer stays 18 dB under it. `0` for none, as before 0.5 ([Measured](#measured-against-izotope-rx-12-dialogue-isolate)) |
+| `music` | `'pass'` | `'pass'`: speech and noise are enhanced, music passes through untouched, segment by segment, the gain ramped over 200 ms at each switch ([Music](#music)); `'enhance'`: everything, as before 0.4 |
 | `weights` | | RNNoise: bytes of an upstream weight blob; DeepFilterNet3: URL or bytes of an upstream ONNX export |
 | `device` | `'auto'` | DeepFilterNet3: `@audio/neural-runtime` backend, `'node'`, `'wasm'` or `'webgpu'` |
 | `sessionOptions` | | DeepFilterNet3: ONNX Runtime session options, e.g. `{ intraOpNumThreads: 4 }` |
@@ -55,7 +57,7 @@ let node = new AudioWorkletNode(ctx, 'neural-denoise', { processorOptions: { wei
 mic.connect(node).connect(ctx.destination)
 ```
 
-The processor collects 128-sample render quanta into 10 ms frames. Its output queue starts with 480 − gcd(128, 480) = 448 samples of silence, the least that never runs dry, so the node's delay is constant: 1408 samples, 29.3 ms. It allocates nothing per frame. The context must run at 48 kHz; the browser resamples the device. `limit` works as in `denoise()`: 16 dB unless given, `0` for none.
+The processor collects 128-sample render quanta into 10 ms frames. Its output queue starts with 480 − gcd(128, 480) = 448 samples of silence, the least that never runs dry, so the node's delay is constant: 1408 samples, 29.3 ms. It allocates nothing per frame. The context must run at 48 kHz; the browser resamples the device. `limit` works as in `denoise()`: 16 dB unless given, `0` for none. The worklet is for a live voice and has no music guard: it denoises everything.
 
 Frame by frame, as `rnnoise_process_frame` works:
 
@@ -68,16 +70,17 @@ let voice = st.process(input480, output480)  // int16 scale in and out; voice pr
 
 ## In `audio`: the contract atom
 
-`@audio/neural-denoise/audio` is the package's [contract](https://github.com/audiojs/compile/blob/main/CONTRACT.md) manifest: `rnnoise`, a streaming atom at any sample rate, with one parameter, `limit` (dB, default 16, 0 for none). [`audio`](https://github.com/audiojs/audio) hosts it as its `rnnoise` op, and runs DeepFilterNet3 as its `deepfilter` op (the whole input before rendering, `limit` 18 by default):
+`@audio/neural-denoise/audio` is the package's [contract](https://github.com/audiojs/compile/blob/main/CONTRACT.md) manifest: `rnnoise`, a streaming atom at any sample rate, with two parameters, `limit` (dB, default 16, 0 for none) and `music` (`'pass'`, the default, or `'enhance'`). [`audio`](https://github.com/audiojs/audio) hosts it as its `rnnoise` op, and runs DeepFilterNet3 as its `deepfilter` op (the whole input before rendering, unlimited, the input mixed back after by `mixback()`: `limit` 18, `floor` 40 and `music: 'pass'` by default):
 
 ```js
 import audio from 'audio'
 
 await audio('take.wav').rnnoise().save('clean.wav')     // streams
-await audio('take.wav').deepfilter().save('clean.wav')  // DeepFilterNet3, room tone kept
+await audio('take.wav').deepfilter().save('clean.wav')  // DeepFilterNet3, room tone kept, music passed
+await audio('show.wav').deepfilter({ music: 'enhance' }) // music denoised too
 ```
 
-Blocks of any size collect into frames; the output queue starts with 479 samples of silence (FRAME − 1, the least that never runs dry for any block size), so the delay is constant: 1439 samples at 48 kHz (30 ms). Other rates go to 48 kHz and back through a streaming form of `@audio/resample-sinc`'s kernel, in its arithmetic, and the delay grows by its lookahead: 1354 samples at 44.1 kHz (30.7 ms), 271 at 8 kHz. The output is `denoise()`'s, delayed: at 48 kHz sample for sample; at other rates, with resample-sinc 1.2.0, sample for sample but in the last 20–30 ms, where the offline resampler sees the end and a stream the host's trailing silence. The default `limit`, 16 dB, is `denoise()`'s for RNNoise: unlimited, the model gates clean speech and removes the voice on some noisy files (Accuracy, below).
+Blocks of any size collect into frames; the output queue starts with 479 samples of silence (FRAME − 1, the least that never runs dry for any block size), so the delay is constant: 1439 samples at 48 kHz (30 ms). Other rates go to 48 kHz and back through a streaming form of `@audio/resample-sinc`'s kernel, in its arithmetic, and the delay grows by its lookahead: 1354 samples at 44.1 kHz (30.7 ms), 271 at 8 kHz. The output is `denoise()`'s, delayed: at 48 kHz sample for sample; at other rates, with resample-sinc 1.2.0, sample for sample but in the last 20–30 ms, where the offline resampler sees the end and a stream the host's trailing silence. The default `limit`, 16 dB, is `denoise()`'s for RNNoise: unlimited, the model gates clean speech and removes the voice on some noisy files (Accuracy, below). With `music: 'pass'` the atom decides as `denoise()` does for RNNoise, from what has arrived, at no added delay, and mixes the denoised signal with the input itself, delayed by the latency, at the host's rate: what passes is the input, sample for sample.
 
 ## RNNoise
 
@@ -98,13 +101,65 @@ Upstream's three ONNX graphs (encoder, ERB-gain decoder, deep-filter decoder) ru
 1. **STFT**: 960-point Vorbis window, 480 hop, the input padded by 960 zeros; the same FFT as RNNoise.
 2. **Features**: power in 32 ERB bands (libDF's `erb_fb`: 2 to 67 bins wide) in dB, mean-normalized with α = 0.99 (τ = 1 s); the lowest 96 bins, unit-normalized with the same α. Both run 2 frames ahead of the output (`conv_lookahead`). `denoise()` computes them as if the input's speech (the louder half of its 50 ms frames) were at −20 dBFS, the median of the VoiceBank+DEMAND test set (−20.2): they are not level-free, the ERB means start at −60 to −90 dB and the complex features scale with the root of the level, and the model was trained at speech gains of −6, 0 and +6 dB (libDF `dataset.rs`). Heard as it is, a take 24 dB quieter lost its pauses' noise by 11 to 14 dB instead of 22 to 24, and the noise after 2.7 s of digital silence by 10 to 24 dB; at −20 dBFS it does as the louder take, at no cost on VoiceBank+DEMAND (PESQ 3.077 against 3.076, 206 files). The ERB bands wholly above the input's band edge hear white noise 20 dB under that speech: upstream trained on full-band 48 kHz mixtures only (its `config.ini`: `p_bandwidth_ext = 0`, libDF's `BandwidthLimiterAugmentation` off; noise at −5 to 40 dB SNR), so the network never heard a band empty, yet a 16 kHz file leaves everything above 8 kHz empty and a codec's low-pass everything above 16 to 20 kHz (nine of the ten narrations below end between 16.7 and 21 kHz). Such bands reached it as libDF's constant 1e-10 floor, a band without a trace of noise, which no mixture it trained on had, and it cleaned the band below worse: on VoiceBank+DEMAND's training speakers at 16 kHz, unlimited, PESQ 2.26 against 2.57 full-band; over the floor, 2.42 ([Accuracy](#accuracy)). The edge is the highest frequency whose long-term power lies within 60 dB of the 1 to 4 kHz median, measured at the input's own rate before resampling (a resampler's images don't count); a band holding the edge keeps the input's own content, so a 44.1 kHz input, whose top band (20.7 to 24 kHz) still holds 20.7 to 22.05 kHz, is processed as before. Every bin of an empty band draws its power from an exponential distribution in every frame, from a fixed seed, as white noise's STFT does: a constant floor would help only the first seconds, since the mean normalization (τ = 1 s) takes a constant band's feature to zero, while noise keeps moving it. The spectrum the gains and filters apply to stays as it was, and a full-band input (all of VoiceBank+DEMAND at 48 kHz) is processed bit for bit as before.
 3. **Network**: 32 gains (sigmoid) and 96 × 5 complex filter taps per frame.
-4. **Enhance**: every bin takes its band's gain; below bin 96 (4.8 kHz) the output is instead Σₖ spec[t − 2 + k] · coef[t, k], k = 0–4, over the unmasked spectrum; `limit` mixes the noisy spectrum back in. Inverse FFT, overlap-add, the 480-sample STFT delay removed.
+4. **Enhance**: every bin takes its band's gain; below bin 96 (4.8 kHz) the output is instead Σₖ spec[t − 2 + k] · coef[t, k], k = 0–4, over the unmasked spectrum; `enhance()`'s `limit` mixes the noisy spectrum back in per bin, as upstream's `atten_lim_db`. Inverse FFT, overlap-add, the 480-sample STFT delay removed. `denoise()` runs it unlimited and mixes the input back after, in time, by `mixback()`: the noise the model took drops by `limit` dB, or down to `floor` dB under the voice where the limit would leave it closer (its power over the 0.5 s around each 10 ms frame against the voice's level, the louder half of the output's 50 ms frames); `floor: 0` gives the per-bin limit's output, but for the voice guard's order.
 5. **Voice guard** (`denoise()`): the model takes held voicing for noise. A clean held note from VocalSet (Wilkins 2018) comes out 39 dB down, unlimited, its own local-SNR estimate at its floor (−15 dB), while speech keeps its level; sung phrases lose 24 to 34 dB. So where the input holds sustained voicing, runs of 0.3 s or more periodic at RNNoise's pitch (normalized correlation 0.45 or more, Praat's voicing threshold, the pitch searched every 20 ms) that stand 6 dB or more over the quietest 100 ms of the 20 s around (minimum statistics, Martin 2001: a buzz is that background, a note stands over it), each band holding a harmonic keeps at least c² of its input, c its correlation with the frame one and two periods earlier: the periodic share of its power, as RNNoise's pitch filter restores what its band gains would remove. Between the harmonics and in the pauses the model's output stands.
 
 The parameters come from the export's `config.ini`, so upstream's low-latency export (`DeepFilterNet3_ll_onnx.tar.gz`: no lookahead, 36 MB) loads through `weights` too.
 
 **Long inputs** run in chunks of `chunk` frames (10 s). Upstream feeds the whole file at once, about 8 MB of activations per second of audio (peak RSS 768 MB for 30 s, 1 GB for 60 s in onnxruntime-node); in 10 s chunks a 180 s file peaks at 563 MB, and any length stays there. The ONNX graphs are exactly causal (outputs on a prefix equal the whole run's) but take no GRU state, so each chunk after the first starts `warmup` frames (3 s) early from zero state, and its first `fade` frames (0.5 s) crossfade from the previous chunk's run, which continued that far. The GRUs remember how they started: 6 s after such a start the gains still differ from the whole-file run's by up to 0.1–0.5. Chunked output is another valid run, not the whole-file one; on VoiceBank+DEMAND in 1 s chunks (1 s warm-up, 0.2 s crossfade, 2 to 3 chunks per utterance) the scores do not move: PESQ 3.164 against 3.162 whole-file, STOI 0.9450 against 0.9452, SI-SDR 18.94 against 18.96 dB, though single files shift by up to 0.31 PESQ either way. The STFT and features stay exact across chunks. Exact chunking needs a stateful re-export (GRU states and convolution buffers as graph inputs and outputs), which is also what live DeepFilterNet3 needs.
 
+## Music
+
+Both models enhance speech, and to them everything else is noise: at their default limits music lost 8 to 16 dB in every band, songs included (Accuracy, Music). So by default (`music: 'pass'`) a classifier segments the input into speech, music and noise, and the model's output is kept only where it hears speech or noise; where it hears music the input passes through, bit for bit. What counts as what follows the classifier's training, French radio and TV: speech over a music bed is speech, so the bed under a voice-over is cleaned with the noise; a song is music, its singer included, since a speech enhancer would strip the accompaniment from under the voice and dull the voice too. A voice singing alone sits between the two (below). `music: 'enhance'` enhances everything, as before 0.4.
+
+1. **Classifier**: inaSpeechSegmenter's speech/music/noise CNN (Doukhan et al., ICASSP 2018; MIT; its `smn` engine), 782,403 weights, in the package as float16 (`guard.bin`, 1.56 MB; `scripts/guard.py` writes it from upstream's Keras export, the batch norms folded). Its input as ina computes it: 16 kHz (here a 31-tap low-pass and every third sample of the 48 kHz signal), pre-emphasis 0.97, 25 ms Hann frames every 10 ms, 24 mel bands from 100 Hz to 8 kHz, log; the lowest 21 bands over 0.68 s make a patch, standardized by its own mean and deviation, so the level does not matter. One patch every 100 ms (ina takes one every 20 ms; on the tuning material every 100 ms decides the same).
+2. **Offline** (DeepFilterNet3, `denoise()` and the `deepfilter` op): segments as ina makes them, Viterbi over the whole input, three states, a switch costing 10⁻⁸⁰ per 20 ms (ina's own constant), so a segment lasts a second or more and a stray patch moves nothing; from 0.5 each patch counts 1 nat in favour of speech, the stream's prior (step 3). A voice under a music bed as loud as itself reaches ina as music about half the time: it hears speech there at a mean probability of 0.47, in songs at 0.01 (medians, training sets). With the prior, of VoiceBank's test takes under MUSDB18 test accompaniments at the voice's level 37% of the frames pass instead of 51%, 5 dB under it 0.6% instead of 2.9%, and the training sets' songs and accompaniments pass every frame they passed without it (at 1.5 nats songs lost 1.1%: the prior chosen there). An input that is music throughout never reaches the model.
+3. **Streaming** (RNNoise: `denoise()`, the contract atom): a stream decides on what has arrived. The forward recursion of the same chain for two states (music, the rest) keeps the log-odds of music, bounded at ±20 nats; a patch moves them by at most ln 100, and by 1 nat less (a prior for speech), and the stream switches to music above 5 and back below 0. Each output frame takes the decision made once the input frame two on is in (RNNoise's own delay), so nothing is added to the latency, and `denoise()` and the atom give the same samples. The price: music is denoised for its first second or so, and each switch comes about a second late.
+4. **The gain** between the enhanced signal and the input moves over 200 ms, raised cosine: offline centered on the segment boundary, streaming from the decision on. It is applied at the input's rate (the 48 kHz gain resampled with the signal), so what passes is the input itself, not its round trip through 48 kHz.
+
+It runs in JS: a stream's `process()` can't await ONNX Runtime, and RNNoise needs no runtime. The three upper convolutions (through im2col) and the dense layers run in a 1,074-byte WebAssembly SIMD kernel (`gemm.wat`), the first convolution once per frame and shared by the overlapping patches (the max pooling after it commutes with each patch's standardization); 1.1 s of CPU per minute of audio, 11 s on the JS path where WebAssembly SIMD is missing (same bits).
+
+**Accuracy**, on labelled sets (`python scripts/accuracy.py guard-sets`, `node scripts/accuracy.mjs guard train|test`): the share of active 10 ms frames passed (within 40 dB of the loudest). The constants (one patch per 100 ms, the streaming bound, prior and hysteresis) were chosen on the training sets: VoiceBank+DEMAND's training speakers, the MUSDB18 training previews, Slakh2100 mixes 1 to 10, VocalSet singers 1 to 5, the Spoken Wikipedia narrations not in Rooms; the test sets below were run once. Beds: clean utterances over a song's accompaniment (the MUSDB18 stems but the vocals) 20, 15 and 10 dB under the speech; hiss: white and pink Gaussian noise at 20, 10 and 5 dB SNR. Programmes: 8 sequences of six segments, speech (five utterances of one kind, or 15 s of narration) and music (up to 20 s of a song, its accompaniment or a Slakh mix) in turn, scored 0.5 s or more from each cut.
+
+| Test set | Frames | Passed, offline (DeepFilterNet3) | Passed, streaming (RNNoise) |
+|---|---|---|---|
+| **Speech, to enhance** | | | |
+| VoiceBank+DEMAND, clean / noisy (824 each) | | 0.0% / 0.0% | 0.0% / 0.0% |
+| the same in white noise, 20 / 10 / 5 dB (275 each) | | 0.0% / 0.0% / 0.0% | 0.0% / 0.0% / 0.0% |
+| the same in pink noise, 20 / 10 / 5 dB | | 0.0% / 0.0% / 0.0% | 0.0% / 0.0% / 0.0% |
+| over a music bed 20 / 15 / 10 dB down (275 each) | | 0.4% / 0.4% / 0.8% (0.4: 0.4% / 0.7% / 1.4%) | 0.7% / 1.4% / 2.3% |
+| ten home narrations (Rooms) | | 0.0% | 0.0% |
+| VocalSet, spoken excerpts (10) | | 1.8% | 2.1% |
+| DEMAND noise alone (noisy − clean) | | 6.6% (0.4: 7.8%) | 2.4% |
+| **Music, to pass** | | | |
+| MUSDB18 test previews, songs (50) | | 100% | 87.2% |
+| the same, accompaniment only (50) | | 100% | 87.7% |
+| Slakh2100 mixes 11 to 20, 60 s each | | 100% | 98.6% |
+| the four repair pieces, 60 s each | | 95.7% | 93.8% |
+| **Programmes** (8): speech enhanced / music passed | | 100% / 100% | 98.5% / 92.9%, switches 0.96 s late (median) |
+| **Singing alone** | | | |
+| MUSDB18 vocal stems (50) | | 47.7% (0.4: 57.5%) | 32.4% |
+| VocalSet sung phrases (20) / long tones (10) | | 58.5% / 57.1% (0.4: 66.4% / 67.8%) | 34.3% / 40.0% |
+
+Offline, every speech set keeps 98.2% or more of its frames enhanced (noisy and hissy speech all of them) and every music set but one passes all of its frames; on the programmes nothing errs 0.5 s from a cut. Streaming, speech is kept as well, and music loses its first second: the previews are 7 s long, so 13% of their frames; the 60 s Slakh mixes, 1.4%. The noise alone that passes is DEMAND's living room (30% of its frames), which has music in it. Of the pieces, `trumpet` (solo) is enhanced for its first 4 s and `brahms` for 5 s of a quiet passage. A voice singing alone is split, about half of its frames passed offline (two thirds before 0.5's prior): held notes and phrases the classifier calls music pass, the rest is enhanced, where DeepFilterNet3's voice guard keeps sustained voicing ([DeepFilterNet3](#deepfilternet3), step 5); enhanced, a VocalSet phrase comes out 1.3 dB down and a MUSDB18 vocal stem 3.1 dB (medians of 10; held tones 0.7). Speech over a bed 10 dB down, louder than a podcast's, still counts as speech.
+
+What it changes, at the default limits (Accuracy: the same sets, `python scripts/accuracy.py music`; per band the median over the files, the worst in parentheses):
+
+| | SI-SDR to the input (median) | 0–250 Hz | 0.25–1 kHz | 1–4 kHz | 4–8 kHz | 8–16 kHz | 16–22 kHz |
+|---|---|---|---|---|---|---|---|
+| DeepFilterNet3, songs (50 MUSDB18 previews): 0.3 | 4.0 dB | −16.9 dB | −13.7 | −16.5 | −13.2 | −12.7 | −16.0 |
+| **0.4** | **the input, bit for bit (all 50)** | **0.0** | **0.0** | **0.0** | **0.0** | **0.0** | **0.0** |
+| DeepFilterNet3, pieces (4): 0.3 | 1.1 dB | −8.6 (−13.0) | −9.0 (−12.1) | −10.9 (−14.8) | −14.4 (−16.6) | −15.5 (−17.4) | −14.0 (−16.1) |
+| **0.4** | **2 of 4 bit for bit** | **0.0 (−0.0)** | **−0.0 (−1.9)** | **−0.1 (−1.7)** | **−0.1 (−4.5)** | **−0.7 (−10.2)** | **−1.0 (−13.0)** |
+| DeepFilterNet3, VocalSet chords (5): 0.3 / **0.4** | 3.5 dB / **bit for bit** | −5.8 / **0.0** | −3.9 / **0.0** | −4.9 / **0.0** | −9.5 / **0.0** | −11.1 / **0.0** | −3.7 / **0.0** |
+| RNNoise, songs: 0.3 | 3.9 dB | −14.4 (−16.0) | −8.0 (−15.1) | −10.1 (−15.8) | −11.4 (−16.0) | −12.2 (−16.0) | −14.2 (−16.0) |
+| **0.4** (the first second denoised) | **11.4 dB** | **−0.6 (−2.3)** | **−0.4 (−2.5)** | **−0.4 (−2.5)** | **−0.5 (−2.2)** | **−0.6 (−2.2)** | **−0.6 (−3.7)** |
+| RNNoise, pieces: 0.3 / **0.4** | 5.8 / **18.3 dB** | −11.5 / **−0.1** | −12.9 / **−0.1** | −13.7 / **−0.1** | −15.3 / **−0.1** | −14.2 / **−0.5** | −11.0 / **−0.6** |
+
+Speech is untouched by the guard: on VoiceBank+DEMAND's 824 noisy test utterances both models' default outputs are 0.3's bit for bit (PESQ 2.90 and 2.49, the Accuracy table's rows), and so are those of the ten narrations; lena, a film scene with music under the voice, has its musical opening passed.
+
+The classifier was chosen against two others, run in Python on the same training sets, each frame scored as music where its music score beats its speech score (Viterbi-smoothed alike): YAMNet (Google, Apache-2.0; AudioSet's 521 classes, 3.7M weights, 0.96 s frames) passed music as well (songs 100%, accompaniment 100%) but speech over a bed 10 / 15 / 20 dB down 29 / 9.6 / 0.9% of the time (ina, offline: 2.2 / 1.2 / 0.4%); Silero VAD (MIT) tells speech from everything else, noise included, and called 17 to 30% of VoiceBank's clean speech frames non-speech (the pauses). PANNs CNN14 (80M weights) and BEATs (90M) were not tried: a guard should cost a fraction of the model it guards.
+
 ## Verification
 
 | Check | Result |
@@ -120,6 +175,10 @@ The parameters come from the export's `config.ini`, so upstream's low-latency ex
 | Edge cases (`test.js`), both models, 48 and 16 kHz | empty input, one sample, 100 samples: the same length back, finite; a second of digital silence stays digital silence |
 | Worklet (`test.js`, AudioWorkletGlobalScope simulated) | the frame API's output, delayed by 448 samples, at `limit: 0`; at 12 and at the default 16, that output with the input mixed back, to 1e-7 |
 | Worklet in Chromium 153 (headless, Playwright, `scripts/worklet.mjs`) | runs, no processor error, output non-silent; live underruns in Speed |
+| Music guard (`test.js`) against the float32 Keras model, the same input (`scripts/guard.py` → `fixtures/guard.json`) | lena, 106 patches: probabilities within 1.7e-3 (float16 weights); the wasm kernel and the JS loop give the same bits |
+| `mixback()` (`test.js`), a perfect model: a tone and white noise | noise 40 and 25 dB under the voice: 18 dB down (58 and 43 under it); 10 dB under and 5 dB over: 40 under it, the floor; `floor: 0`: 18 down; a noise that drops 30 dB halfway: each half as such 0.5 s from the change; where input and output agree, the input bit for bit; `denoise()`'s default is the unlimited output through it, bit for bit |
+| Music guard's prior (`test.js`, with VoiceBank+DEMAND and the guard sets in the data cache) | speaker p226's first five training takes under a MUSDB18 training accompaniment at their level: 13% of the frames passed (0.4: all) |
+| Music guard (`test.js`): a synthetic band (chords of plucked tones, a bass, a tick) | DeepFilterNet3, through a stand-in that removes everything: the band back bit for bit at 48 and 44.1 kHz, `music: 'enhance'` removes it; RNNoise: the band itself from 1.1 s on; band, noisy speech, band (with VoiceBank+DEMAND in the data cache): two switches, each within 0.5 s of its cut, gain exactly 0 and 1 between 200 ms raised-cosine ramps; RNNoise's output to frame f − 2 the same whatever follows frame f |
 | Contract atom (`test.js`) against `denoise()`, delayed by its declared latency | 48 kHz: equal sample for sample in blocks of 1, 128, 1024 and 1 to 3000 samples, limits 20 and 0; 44.1 kHz (resample-sinc 1.2.0): equal but in the last 50 ms; with resample-sinc 1.1.2, whose arithmetic differs, 59.5 dB SNR |
 
 To rerun: `node scripts/rnnoise-reference.mjs` builds upstream and rewrites `fixtures/rnnoise.json` (and caches the little model); `python scripts/deepfilter-reference.py` (DeepFilterNet 0.5.6 in a Python 3.11 venv) writes the reference `test.js` compares with.
@@ -144,10 +203,11 @@ The VoiceBank+DEMAND test set (Valentini-Botinhao 2017, CC BY 4.0): 824 utteranc
 | DeepFilterNet3, Python (deepfilternet 0.5.6) | 3.16 | 0.945 | 18.96 | 3.44 | 4.08 | 3.18 |
 | DeepFilterNet3 in 1 s chunks, `limit: 0` | 3.16 | 0.945 | 18.94 | 3.44 | 4.08 | 3.18 |
 | DeepFilterNet3, `denoise()`, `limit: 0`: heard at −20 dBFS, voice guard | 3.15 | 0.945 | 18.95 | 3.44 | 4.08 | 3.18 |
-| DeepFilterNet3, `denoise()`, `limit: 18`, the default | 2.90 | 0.942 | 17.91 | 3.48 | 3.84 | 3.09 |
+| DeepFilterNet3, `denoise()`, `limit: 18`, `floor: 0` (the default before 0.5) | 2.90 | 0.942 | 17.91 | 3.48 | 3.84 | 3.09 |
+| **DeepFilterNet3, `denoise()`, `limit: 18`, `floor: 40`, the default** | **3.05** | **0.944** | **18.86** | **3.45** | **4.01** | **3.15** |
 | DeepFilterNet3, `limit: 12` (the default before 0.2) | 2.67 | 0.939 | 16.18 | 3.49 | 3.69 | 3.03 |
 
-Over the noisy input, paired, with 95% intervals: DeepFilterNet3 adds 1.20 ± 0.03 PESQ, 0.024 ± 0.002 STOI and 10.5 ± 0.3 dB SI-SDR, and beats every classical row's PESQ on 804 of 824 files; RNNoise adds 0.14 ± 0.04 PESQ and 3.8 ± 0.3 dB SI-SDR but loses 0.031 ± 0.004 STOI; `omlsa`, the best classical row, adds 0.43 ± 0.03 PESQ and 6.1 ± 0.2 dB SI-SDR and loses 0.006 STOI. A limit leaves noise in on this set, and spares the voice: paired, on a quarter of the files (206), DNSMOS SIG moves −0.002 ± 0.015 from 12 to 18 dB, then −0.008 ± 0.003 to 20, −0.019 ± 0.006 more to 24 and −0.049 ± 0.017 more unlimited, while BAK rises 0.18, 0.04, 0.07 and 0.16 and PESQ 0.25, 0.06, 0.08 and 0.10. Past 18 dB the speech itself loses, which DNSMOS reads as SIG and a listener as a filtered, phone-call voice; from 8 to 16 kHz the unlimited output carries 1.5 dB less than the clean speech, at 18 dB 0.9 (all 824 files). The default 18 dB still adds 0.93 PESQ over the input and 0.50 over `omlsa`. Pass `limit: 0` where the most noise removal matters more than room tone. The level change and the voice guard cost 0.01 PESQ unlimited and 0.004 at 18 dB. The same scoring gives upstream's published anchors: noisy 1.97 and 0.921 (as the DeepFilterNet2 paper's table), DeepFilterNet3 3.16 and 0.945 against the DeepFilterNet3 paper's 3.17 and 0.944. That DeepFilterNet2 table measured RNNoise's 2022 model at 2.33 and 0.922; the current model scores lower here.
+Over the noisy input, paired, with 95% intervals: DeepFilterNet3 adds 1.20 ± 0.03 PESQ, 0.024 ± 0.002 STOI and 10.5 ± 0.3 dB SI-SDR, and beats every classical row's PESQ on 804 of 824 files; RNNoise adds 0.14 ± 0.04 PESQ and 3.8 ± 0.3 dB SI-SDR but loses 0.031 ± 0.004 STOI; `omlsa`, the best classical row, adds 0.43 ± 0.03 PESQ and 6.1 ± 0.2 dB SI-SDR and loses 0.006 STOI. A limit leaves noise in on this set, and spares the voice: paired, on a quarter of the files (206), DNSMOS SIG moves −0.002 ± 0.015 from 12 to 18 dB, then −0.008 ± 0.003 to 20, −0.019 ± 0.006 more to 24 and −0.049 ± 0.017 more unlimited, while BAK rises 0.18, 0.04, 0.07 and 0.16 and PESQ 0.25, 0.06, 0.08 and 0.10. Past 18 dB the speech itself loses, which DNSMOS reads as SIG and a listener as a filtered, phone-call voice; from 8 to 16 kHz the unlimited output carries 1.5 dB less than the clean speech, at 18 dB 0.9 (all 824 files). 18 dB alone added 0.93 PESQ over the input and 0.50 over `omlsa`; from 0.5 the floor takes the noise that 18 dB would leave within 40 dB of the voice further down, on this set most of it (2.5 to 17.5 dB SNR): +0.15 PESQ, +0.17 BAK, +0.06 OVRL, −0.03 SIG against `floor: 0` (bench/speech.mjs `dfn-18` against `dfn-f0`, both through `audio`'s op). Pass `limit: 0` where the most noise removal matters more than room tone. The level change and the voice guard cost 0.01 PESQ unlimited and 0.004 at 18 dB. The same scoring gives upstream's published anchors: noisy 1.97 and 0.921 (as the DeepFilterNet2 paper's table), DeepFilterNet3 3.16 and 0.945 against the DeepFilterNet3 paper's 3.17 and 0.944. That DeepFilterNet2 table measured RNNoise's 2022 model at 2.33 and 0.922; the current model scores lower here.
 
 - **RNNoise's current model sometimes removes the voice.** STOI drops by more than 0.1 on 68 of 824 files and by more than 0.2 on 12; on p232_354 it goes from 0.870 to 0.154, the output 24 dB under the clean level while RNNoise's own voice probability averages 0.84. Upstream's C does the same, the portable and the NEON build alike (0.154, 0.153). DeepFilterNet3 never loses more than 0.066 STOI, `wiener` 0.065, `omlsa` 0.059. A `limit` caps it: at the default 16 dB (the input mixed back at 0.16) losses over 0.1 fall to 4 files and over 0.2 to none (p232_354 keeps 0.824), and PESQ rises to 2.49, +0.53 ± 0.02 over the noisy input, at a BAK of 3.66 instead of 3.85. Against 0.2's 20 dB, paired: PESQ +0.029 ± 0.009, STOI +0.004, SIG +0.026 ± 0.004, BAK −0.047 ± 0.005 (6 files over 0.1, p232_354 0.786). `denoise()`, the worklet and the atom apply that limit unless given another.
 - **The recipe's delivery stages are not denoisers.** Its compressor, EQ, de-esser and loudness target change the signal against the clean reference on purpose: the full recipe scores under its own denoising part. SI-SDR also falls with the 80 Hz high-pass: on its own it takes clean speech to 6.2 dB SI-SDR at PESQ 4.45 (21 test utterances). The table says what each stage does to speech in noise, not how finished a podcast sounds.
@@ -201,27 +261,48 @@ On the test set, paired, at the default 18 dB: PESQ +0.11 ± 0.01 at 16 kHz and
 | DeepFilterNet3, upstream's settings, `limit: 0` | 3.47 | 4.12 | 3.21 | −152 to −61 (−83) | −0.4 dB |
 | DeepFilterNet3, `denoise()`, `limit: 0` | 3.47 | 4.11 | 3.21 | −165 to −60 (−78) | −0.4 dB |
 | DeepFilterNet3, `denoise()` 0.2, `limit: 18` | 3.47 | 4.11 | 3.21 | −93 to −59 (−73) | −0.4 dB |
-| DeepFilterNet3, `denoise()`, `limit: 18`, the default | 3.47 | 4.11 | 3.21 | −93 to −59 (−73) | −0.4 dB |
+| DeepFilterNet3, `denoise()`, `limit: 18`, the default (0.5's `floor: 40` moves DNSMOS by 0.001) | 3.47 | 4.11 | 3.21 | −93 to −59 (−73) | −0.4 dB |
 | DeepFilterNet3, `limit: 12` | 3.47 | 4.10 | 3.20 | −91 to −58 (−68) | −0.3 dB |
 
-These rooms are mostly quiet (raw OVRL 3.17), so the gains are small. OM-LSA keeps SIG and raises BAK a little less than the models (0.3.11's lowered SIG by 0.20); both models raise BAK, and at their default limits keep SIG. Unlimited, both can take pauses to dead silence (RNNoise's median floor is −112 dBFS, DeepFilterNet3's lowest −165), which ACX Check flags; the default limits keep the floors between −98 and −59 dBFS at the same OVRL or better, room tone left in. Eight of the narrations end between 16.7 and 19.5 kHz and hear 0.3's floor in their top bands (one ends at 21.1 kHz, inside the top band, one runs to 24): DNSMOS moves by 0.008 at most on any file. On lena (a film scene with music under the voice) RNNoise lowers OVRL from 2.76 to 2.47 unlimited and keeps 2.78 at 16 dB; DeepFilterNet3 raises it to 2.83 upstream, 2.89 at the default limit.
+These rooms are mostly quiet (raw OVRL 3.17), so the gains are small, and 0.5's floor changes them by 0.001 DNSMOS (it takes the noisiest room's floor from −59.1 to −62.7 dBFS, under ACX's −60; bench/speech.py's scoring, Measured). OM-LSA keeps SIG and raises BAK a little less than the models (0.3.11's lowered SIG by 0.20); both models raise BAK, and at their default limits keep SIG. Unlimited, both can take pauses to dead silence (RNNoise's median floor is −112 dBFS, DeepFilterNet3's lowest −165), which ACX Check flags; the default limits keep the floors between −98 and −59 dBFS at the same OVRL or better, room tone left in. Eight of the narrations end between 16.7 and 19.5 kHz and hear 0.3's floor in their top bands (one ends at 21.1 kHz, inside the top band, one runs to 24): DNSMOS moves by 0.008 at most on any file. On lena (a film scene with music under the voice) RNNoise lowers OVRL from 2.76 to 2.47 unlimited and keeps 2.78 at 16 dB; DeepFilterNet3 raises it to 2.83 upstream, 2.89 at the default limit.
 
 **Singing.** VocalSet (Wilkins et al., ISMIR 2018, CC BY 4.0): 40 long tones (straight, forte, pianissimo, messa di voce; 20 singers) and 20 sung phrases (straight and vibrato), alone and in living-room noise (DEMAND, from the VoiceBank+DEMAND test set) at 15 dB SNR; level change of the voice over the frames within 20 dB of the loudest. At the default 18 dB, against 0.1's processing at the same limit (heard as it is, no voice guard): held notes −1.9 dB clean and −3.4 dB in noise, 14% of frames more than 6 dB down, against −14.2 and −16.2 dB, 96% (at 10 dB SNR −4.8 against −15.6); sung phrases −0.9 and −2.0 dB against −10.2 and −14.9; the same singers' spoken phrases −0.3 and −0.8 dB against −1.7 and −3.7 (they are quiet recordings: the −20 dBFS hearing helps them). Upstream's unlimited output takes the held notes 38.7 and 37.4 dB down. The noise in the pauses drops as before (−14.1 dB against −14.3), and a 100 or 120 Hz buzz 20 dB under the voice drops in them as without the guard (−17.2 dB), in quiet and in cafe noise: it is the background, not over it.
 
-**Music.** Both models enhance speech, and to them everything else is noise: music loses at every band, and the limit only caps the loss. The first 60 s of four pieces (`vibeace` jazz, `brahms` string orchestra, `nutcracker` celesta and strings, `trumpet` solo), five chords of three VocalSet singers each, and the 50 MUSDB18 test previews (Rafii et al. 2017, 7 s of a song each, most with vocals), at 44.1 kHz; level change per band against the input, the median over the files:
+**Music.** Both models enhance speech, and to them everything else is noise: before 0.4 music lost at every band, and the limit only capped the loss (DeepFilterNet3, songs: −13 to −17 dB per band, median; unlimited, single bands of single songs lost up to 54 dB, RNNoise's up to 68). From 0.4 the music guard passes it through ([Music](#music), with the before and after per band). The first 60 s of four pieces (`vibeace` jazz, `brahms` string orchestra, `nutcracker` celesta and strings, `trumpet` solo), five chords of three VocalSet singers each, and the 50 MUSDB18 test previews (Rafii et al. 2017, 7 s of a song each, most with vocals), at 44.1 kHz, are the sets of that table.
 
-| at the default limit | 0–250 Hz | 0.25–1 kHz | 1–4 kHz | 4–8 kHz | 8–16 kHz | 16–22 kHz |
-|---|---|---|---|---|---|---|
-| RNNoise, pieces | −11.5 dB | −12.9 | −13.7 | −15.3 | −14.2 | −11.0 |
-| RNNoise, chords | −7.2 | −15.4 | −15.6 | −15.4 | −14.7 | −15.3 |
-| RNNoise, songs | −14.4 | −8.0 | −10.1 | −11.4 | −12.2 | −14.2 |
-| DeepFilterNet3, pieces | −8.6 | −9.0 | −10.9 | −14.4 | −15.5 | −14.0 |
-| DeepFilterNet3, chords | −5.8 | −3.9 | −4.9 | −9.5 | −11.1 | −3.7 |
-| DeepFilterNet3, songs | −16.9 | −13.7 | −16.5 | −13.2 | −12.7 | −16.0 |
+To rerun: `python scripts/accuracy.py prepare` writes the band-limited inputs and decodes the MUSDB18 previews; `python scripts/accuracy.py guard-sets` writes the music guard's labelled sets and `node scripts/accuracy.mjs guard train|test` measures it (systems `rnnoise-guard` and `dfn3-guard` are 0.4's defaults, the other rows take `music: 'enhance'`); `node scripts/accuracy.mjs SET SYSTEMS` (SET: `vbdemand`, `vbtrain`, `vbclean`, `vbtrain-clean`, either noisy set `@16000`, `@44100` or `-lp16k`, `music`) and `node scripts/accuracy.mjs rooms` write the outputs (the data sources and checksums are in its header); `python scripts/accuracy.py SET SYSTEMS`, `limits SET rnnoise 0,12,14,15,16,18,20`, `clean SET SYSTEM [LIMITS]`, `music SYSTEMS`, `presence SYSTEM` and `rooms` score them; `scripts/deepfilter-reference.py --vbdemand` writes the Python row; the classical rows come from `@audio/denoise`'s `node scripts/speech.mjs vbdemand|rooms SYSTEMS` and `python scripts/speech.py score vbdemand|rooms SYSTEMS`. Per-file scores stay in `~/.cache/audiojs/data/` (`vbdemand/scores/neural-*.csv`, `vbdemand-train/scores/neural-*.csv`, `spoken/scores.json`); the scripts regenerate outputs, skipping any already there.
+
+## Measured: against iZotope RX 12 Dialogue Isolate
+
+October 2026: RX 12 Advanced's Dialogue Isolate plug-in (VST3 hosted through Pedalboard 0.9, `audio`'s `bench/rx/host.py`) and `audio`'s `deepfilter()` on the same buffers. RX at its defaults with the noise off (noise gain −∞; dialogue and reverb gain 0 dB, sensitivity 5, the four band amounts 100%), and tuned: the reverb off too. Ours at its defaults, 0.4 (`floor: 0` and no speech prior: `audio`'s op bit for bit) and 0.5. Every setting, RX's and ours, was chosen on the tune split (VoiceBank's training speakers, MUSDB18's training songs, the even-numbered MIT rooms, DEMAND's first 150 s, the spoken-train narrations): RX's sensitivity 0, 2.5, 5, 7.5 and 10 and its reverb off tried, the reverb off with sensitivity 5 the best mean PESQ (1.88) and OVRL (2.71) over the speech conditions; our floor 0, 35, 40 and 45 tried (mean PESQ 1.75, 1.96, 1.98, 1.98; VoiceBank's training speakers 2.24, 2.38, 2.43, 2.46 at SIG 3.41, 3.38, 3.36, 3.35; unlimited 2.57, SIG 3.34), 40 kept: SIG is the voice, and 40 dB under it is ACX's −60 dBFS floor for a voice at −20. Then the test split once.
+
+**Harder mixtures**, `audio`'s `bench/rx/isolate.mjs`, test split: VoiceBank's clean test takes (every 8th, 103 per row) at −26 dBFS under DEMAND noises VoiceBank+DEMAND does not use (washing machine, field, park, river, hallway) at −5, 0 and 5 dB SNR (active levels), six Spoken Wikipedia narrators at once, MUSDB18 test accompaniments (drums, bass, other), and MIT IR Survey rooms (odd-numbered, a tail to hear) with the noise at 5 dB, the reference there the direct sound and 50 ms. Scored at 16 kHz: PESQ / DNSMOS OVRL (at the reference's loudness), the last column paired against RX tuned with 95% intervals.
+
+| | RX default | RX tuned | 0.4 | **0.5** | 0.5 − RX tuned, PESQ / OVRL |
+|---|---|---|---|---|---|
+| noise, −5 dB | 2.00 / 2.79 | 2.00 / 2.78 | 1.73 / 2.77 | **2.11 / 2.91** | +0.11 ± 0.05 / +0.12 ± 0.03 |
+| noise, 0 dB | 2.28 / 2.95 | 2.27 / 2.95 | 2.07 / 2.96 | **2.43 / 3.04** | +0.16 ± 0.05 / +0.09 ± 0.03 |
+| noise, 5 dB | 2.56 / 3.07 | 2.56 / 3.07 | 2.44 / 3.05 | **2.70 / 3.10** | +0.15 ± 0.07 / +0.03 ± 0.03 |
+| babble, 0 dB | 1.18 / 2.11 | 1.18 / 2.16 | 1.24 / 2.31 | **1.31 / 2.41** | +0.12 ± 0.03 / +0.25 ± 0.06 |
+| babble, 5 dB | 1.43 / 2.47 | 1.44 / 2.51 | 1.58 / 2.67 | **1.70 / 2.73** | +0.26 ± 0.04 / +0.22 ± 0.05 |
+| music bed, 0 dB | 1.65 / 2.74 | 1.65 / 2.74 | 1.26 / 2.10 | 1.63 / 2.33 | −0.02 ± 0.09 / −0.41 ± 0.17 |
+| music bed, 5 dB | 1.96 / 2.90 | 1.96 / 2.90 | 1.82 / 2.94 | **2.26 / 3.03** | +0.30 ± 0.05 / +0.13 ± 0.05 |
+| room, noise 5 dB | 1.66 / 2.21 | 1.80 / 2.51 | 1.62 / 2.28 | 1.77 / 2.42 | −0.03 ± 0.05 / −0.09 ± 0.07 |
+| mean | 1.84 / 2.66 | 1.86 / 2.70 | 1.72 / 2.63 | **1.99 / 2.75** | |
 
-Unlimited, single bands of single songs lose up to 54 dB (DeepFilterNet3) and 68 dB (RNNoise). The voice guard keeps one held voice: it follows one pitch, not a chord or an orchestra. Neither model passes music through, because nothing at hand tells music from noisy speech reliably enough. The guard that suggests itself, passing the input through where the unlimited model takes much of what stands 10 dB or more over each band's noise floor (minimum statistics), can't be set: noise that moves stands over the floor too. DeepFilterNet3 takes more than 9 dB of it from 17 of the 504 noisy training utterances, as from 45 of the 50 songs and 3 of the 4 pieces, but from none of the chords (5 to 8.4 dB); at 12 dB, 4 utterances would pass through untouched while 10 songs and every chord would still lose; at 6 dB, 32 utterances would. RNNoise takes more than 9 dB from 32 utterances (its voice removals among them) as from every piece and chord and 40 songs. Telling music from speech takes a classifier trained for it; none ships here.
+SI-SDR follows PESQ but under steady noise, where RX keeps the waveform closer (13.2, 15.8, 17.8 dB against 12.7, 15.3, 17.2); in babble ours is the closer (3.5 and 8.9 against 1.7 and 8.4). DNSMOS SIG, the voice: ours 3.21 to 3.39 under noise, RX's 3.07 to 3.33.
 
-To rerun: `python scripts/accuracy.py prepare` writes the band-limited inputs and decodes the MUSDB18 previews; `node scripts/accuracy.mjs SET SYSTEMS` (SET: `vbdemand`, `vbtrain`, `vbclean`, `vbtrain-clean`, either noisy set `@16000`, `@44100` or `-lp16k`, `music`) and `node scripts/accuracy.mjs rooms` write the outputs (the data sources and checksums are in its header); `python scripts/accuracy.py SET SYSTEMS`, `limits SET rnnoise 0,12,14,15,16,18,20`, `clean SET SYSTEM [LIMITS]`, `music SYSTEMS`, `presence SYSTEM` and `rooms` score them; `scripts/deepfilter-reference.py --vbdemand` writes the Python row; the classical rows come from `@audio/denoise`'s `node scripts/speech.mjs vbdemand|rooms SYSTEMS` and `python scripts/speech.py score vbdemand|rooms SYSTEMS`. Per-file scores stay in `~/.cache/audiojs/data/` (`vbdemand/scores/neural-*.csv`, `vbdemand-train/scores/neural-*.csv`, `spoken/scores.json`); the scripts regenerate outputs, skipping any already there.
+**VoiceBank+DEMAND** test set (824, `audio`'s `bench/speech.mjs` and `bench/speech.py`, scored as in Accuracy): RX default PESQ 2.73, STOI 0.929, SI-SDR 18.9 dB, SIG 3.30, BAK 3.82, OVRL 2.94; tuned 2.72, 0.929, 18.8, 3.30, 3.83, 2.94; ours 0.4 2.90, 0.942, 17.9, 3.48, 3.84, 3.10; **0.5 3.05, 0.944, 18.9, 3.45, 4.01, 3.15**. **The ten narrations** (Accuracy, Real rooms): raw OVRL 3.18; RX 3.23 (tuned 3.23), every room's floor (its quietest 500 ms) taken under −79.6 dBFS (tuned −88.2), room tone gone; ours 3.23 at 0.4 and 0.5, room tone kept, the noisiest room's floor at −62.7 dBFS (0.4: −59.1).
+
+**Nothing to isolate.** The clean test takes: RX raises them 1.4 dB and leaves PESQ 3.98 and SI-SDR 23.6 dB against the take (tuned the same); ours −0.1 dB, PESQ 4.60, SI-SDR 41.2 dB (OVRL 3.24 against RX's 3.30: RX also takes the studio's own faint noise). Music alone, 24 MUSDB18 test previews, songs and accompaniments in turn: RX takes it 37.6 dB down (every band 33 to 57 dB), its median SI-SDR to the input −16.8 dB; ours passes all 24 bit for bit.
+
+What 0.5 changed, and where RX still wins:
+
+- **The floor.** 18 dB of attenuation leaves noise as loud as the voice 18 dB under it, and RX takes it all: before 0.5 RX won every noise row. Unlimited, the model beat RX already (tune split, −5 dB: PESQ 2.12 against 1.98); the floor keeps that where the noise stands near the voice and the 18 dB limit's room tone where it is quiet. +0.38, +0.36, +0.26 PESQ at −5, 0, 5 dB over 0.4.
+- **The guard's prior.** Speech under a bed as loud as itself reached ina as music, so 0.4 passed half of those takes untouched (PESQ 1.26). With speech favoured by 1 nat a patch, as the stream already decided, 37% of their frames pass (Music): PESQ 1.63 against RX's 1.65, OVRL 2.33 against 2.74. What still passes is a voice ina cannot tell from a singer at that level: the model keeps the voice of a song as it keeps this one, 4 to 7 dB under the mix, and ina hears speech in both, so passing less would gut songs (tune split: a voice under a bed at its level kept −3.5 to −5.9 dB and heard as speech at 0.81 or more; the training songs, at most −4.6 dB and 1.00). `music: 'enhance'` enhances it all: PESQ 1.87, OVRL 2.91, SI-SDR 11.0 dB, ahead of RX on all three, and music alone then loses 28.5 dB (RX 37.6).
+- **Rooms.** RX's reverb gain takes the late reverberation (SI-SDR 7.2 dB to the direct sound and 50 ms, against ours 5.7; at its default, reverb kept, 5.2). DeepFilterNet3 trained with reverberant speech in 10% of its mixtures (its `config.ini`, `p_reverb`), and takes part of it; `audio`'s `dereverb()` before it takes more.
+
+To rerun: in `audio`, `node bench/rx/isolate.mjs test all rx-di rx-di-tuned dfn` (renders kept in `~/.cache/audiojs/data/rx/isolate/`), `node bench/speech.mjs test rx-di,rx-di-tuned,dfn-18` then `python bench/speech.py test rx-di,rx-di-tuned,dfn-18`; tune with `tune` in place of `test`.
 
 ## Speed and memory
 
@@ -243,6 +324,8 @@ A 128-sample render quantum lasts 2.67 ms and a frame is due every 3.75 quanta:
 
 **DeepFilterNet3**, onnxruntime-node 1.30, 4 intra-op threads: real-time factor 0.047 in CPU on a 180 s file (8.4 s), 0.087 over the 824 short test utterances (2072 s of audio in 180 s; per-run overhead); Python DeepFilterNet (PyTorch 2.1, 4 threads) 0.067 on the same utterances. Peak RSS 436 MB over the test set, 563 MB for 180 s in 10 s chunks. The voice guard's pitch analysis adds 0.16 s of wall time per minute of audio to the model's 0.6 (one channel at 48 kHz, 4 threads); the band edge, 0.06 s of CPU. The browser path (onnxruntime-web, wasm and WebGPU) is untested here.
 
+**Music guard**: 1.1 s of CPU per minute of audio per channel (real-time factor 0.019; WebAssembly SIMD, Node 25, the machine as above), 11 s on the JS path. RNNoise through `denoise()` goes from 3.0 to 4.2 s of CPU per minute; DeepFilterNet3 skips the model where a whole input is music. The guard's state: 0.3 MB per channel, the network 3.1 MB once unpacked to float32.
+
 ## Licenses
 
 Audited 2026-09-27 against what each project states; a summary, not legal advice.
@@ -251,6 +334,9 @@ Audited 2026-09-27 against what each project states; a summary, not legal advice
 |---|---|---|---|---|
 | **RNNoise**, model 0a8755f8 (January 2025) | BSD-3-Clause (`COPYING`: Valin, Amazon, Mozilla, Xiph.Org, Borgerding) | BSD-3-Clause | `rnnoise_data-0a8755f8….tar.gz` (media.xiph.org, sha256 pinned in upstream's `model_version`): `src/rnnoise_data.c` (default), `src/rnnoise_data_little.c`, two `.pth` checkpoints. No header of their own; upstream's `autogen.sh` downloads them into `src/` and compiles them into the BSD library, and the README calls them "the models distributed with RNNoise". Trained on the corpora in `datasets.txt`: OpenSLR 30–86 (CC BY-SA 4.0) and Hi-Fi TTS (OpenSLR 109, CC BY 4.0) for speech, Xiph's noise collections (donated noise "freely available", terms in that archive, not checked) | bundled: `rnnoise.bin`, the default model through upstream's `dump_weights_blob`; the little model (1.55 MB) loads through `weights` |
 | **DeepFilterNet3**, d375b2d | MIT OR Apache-2.0 ("All code in this repository") | **unconfirmed** | `models/DeepFilterNet3_onnx.tar.gz`, `DeepFilterNet3.zip`, `DeepFilterNet3_ll_onnx.tar.gz`, in the same repository, no terms of their own. Four open issues ask whether "all code" covers them ([#697](https://github.com/Rikorose/DeepFilterNet/issues/697), [#700](https://github.com/Rikorose/DeepFilterNet/issues/700), [#709](https://github.com/Rikorose/DeepFilterNet/issues/709), [#712](https://github.com/Rikorose/DeepFilterNet/issues/712), July to September 2026), none answered; the last push was October 2024. The DeepFilterNet3 paper's abstract says "the framework as well as pretrained weights have been published under an open source license", naming none. Third parties republish them as MIT (Intel/deepfilternet-openvino on Hugging Face) or Apache-2.0 | fetched from upstream at the pinned commit, cached, never bundled or re-hosted |
+| **inaSpeechSegmenter**, speech/music/noise CNN (`smn`) | MIT (`LICENSE`, © 2018 Ina) | MIT: the repository's license; the weight files are its release assets (`models`), no terms of their own. Training data: Ina's annotated French radio and TV, not published | `keras_speech_music_noise_cnn.hdf5` (sha256 `f04b5e3c…`) | bundled: `guard.bin`, its weights folded and in float16 (`scripts/guard.py`); the feature recipe follows ina's `sidekit_mfcc.py` (from SIDEKIT, LGPL), written anew from its definitions |
+| YAMNet (Google) | Apache-2.0 | Apache-2.0 | TF Hub `yamnet/1` | not used: compared, passes more speech over music beds ([Music](#music)) |
+| Silero VAD | MIT | MIT | `silero_vad.onnx` | not used: speech against everything else, noise included |
 | DeepFilterNet 1 and 2 | same | same as DeepFilterNet3 | `models/DeepFilterNet{,2}*.{zip,tar.gz}` | not used; libDF deprecates DeepFilterNet2 |
 | GTCRN (Rong et al., ICASSP 2024) | MIT | MIT (checkpoints in the repository) | 48.2K parameters, trained on DNS3 or VCTK-DEMAND | not used: 16 kHz only, narration would lose everything above 8 kHz |
 | Demucs speech denoiser (facebookresearch/denoiser) | CC BY-NC 4.0 | CC BY-NC 4.0 | | excluded: non-commercial; archived |
@@ -260,14 +346,14 @@ Audited 2026-09-27 against what each project states; a summary, not legal advice
 
 In the browser, [shiguredo/rnnoise-wasm](https://github.com/shiguredo/rnnoise-wasm) (Apache-2.0) builds the same RNNoise commit with emscripten, and [sapphi-red/web-noise-suppressor](https://github.com/sapphi-red/web-noise-suppressor) (MIT) wraps its 2022 build next to Speex and GTCRN. This package needs no WebAssembly (it takes 419 bytes of it for speed where it can), is checked bit for bit against the C, and adds DeepFilterNet3.
 
-The package is BSD-3-Clause (the RNNoise translation and weights); `deepfilter.js` carries DeepFilterNet's MIT notice ([NOTICE](./NOTICE)).
+The package is BSD-3-Clause (the RNNoise translation and weights); `deepfilter.js` carries DeepFilterNet's MIT notice, `guard.bin` inaSpeechSegmenter's ([NOTICE](./NOTICE)).
 
 ## Reference
 
-J.-M. Valin, "A Hybrid DSP/Deep Learning Approach to Real-Time Full-Band Speech Enhancement", MMSP 2018, [arXiv:1709.08243](https://arxiv.org/abs/1709.08243). · H. Schröter, T. Rosenkranz, A. N. Escalante-B., A. Maier, "DeepFilterNet: Perceptually Motivated Real-Time Speech Enhancement", Interspeech 2023, [arXiv:2305.08227](https://arxiv.org/abs/2305.08227). · C. Valentini-Botinhao, "Noisy speech database for training speech enhancement algorithms and TTS models", University of Edinburgh, 2017, [doi:10.7488/ds/2117](https://doi.org/10.7488/ds/2117). · ITU-T P.862.2 (2007), wideband PESQ. · C. H. Taal, R. C. Hendriks, R. Heusdens, J. Jensen, "An Algorithm for Intelligibility Prediction of Time-Frequency Weighted Noisy Speech", IEEE TASLP 19(7), 2011. · J. Le Roux, S. Wisdom, H. Erdogan, J. R. Hershey, "SDR: Half-baked or Well Done?", ICASSP 2019. · C. K. A. Reddy, V. Gopal, R. Cutler, "DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors", ICASSP 2022. · S. A. Figueroa, "When is double rounding innocuous?", SIGNUM Newsletter 30(3), 1995. · [xiph/rnnoise](https://gitlab.xiph.org/xiph/rnnoise) at 70f1d25 · [Rikorose/DeepFilterNet](https://github.com/Rikorose/DeepFilterNet) at d375b2d.
+J.-M. Valin, "A Hybrid DSP/Deep Learning Approach to Real-Time Full-Band Speech Enhancement", MMSP 2018, [arXiv:1709.08243](https://arxiv.org/abs/1709.08243). · H. Schröter, T. Rosenkranz, A. N. Escalante-B., A. Maier, "DeepFilterNet: Perceptually Motivated Real-Time Speech Enhancement", Interspeech 2023, [arXiv:2305.08227](https://arxiv.org/abs/2305.08227). · C. Valentini-Botinhao, "Noisy speech database for training speech enhancement algorithms and TTS models", University of Edinburgh, 2017, [doi:10.7488/ds/2117](https://doi.org/10.7488/ds/2117). · ITU-T P.862.2 (2007), wideband PESQ. · C. H. Taal, R. C. Hendriks, R. Heusdens, J. Jensen, "An Algorithm for Intelligibility Prediction of Time-Frequency Weighted Noisy Speech", IEEE TASLP 19(7), 2011. · J. Le Roux, S. Wisdom, H. Erdogan, J. R. Hershey, "SDR: Half-baked or Well Done?", ICASSP 2019. · C. K. A. Reddy, V. Gopal, R. Cutler, "DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors", ICASSP 2022. · S. A. Figueroa, "When is double rounding innocuous?", SIGNUM Newsletter 30(3), 1995. · [xiph/rnnoise](https://gitlab.xiph.org/xiph/rnnoise) at 70f1d25 · [Rikorose/DeepFilterNet](https://github.com/Rikorose/DeepFilterNet) at d375b2d. · D. Doukhan, J. Carrive, F. Vallet, A. Larcher, S. Meignier, "An Open-Source Speaker Gender Detection Framework for Monitoring Gender Equality", ICASSP 2018 · [ina-foss/inaSpeechSegmenter](https://github.com/ina-foss/inaSpeechSegmenter), models release.
 
-**Use when:** speech with noise a statistical denoiser leaves or smears (moving noise, a busy room); live voice in a browser (RNNoise, 30 ms); files where quality is worth an 8 MB download (DeepFilterNet3).<br>
-**Not for:** music, songs included (both take 8 to 16 dB from every band at their default limits, and nothing in them tells music from noisy speech; DeepFilterNet3's voice guard keeps one held voice, RNNoise drops it); clean speech with nothing to remove through RNNoise (it gates word ends and quiet syllables, 9 to 12 dB at its default limit); installs that must stay model-free ([`@audio/denoise`](https://github.com/audiojs/denoise)); reverb as such (`dereverb`); live DeepFilterNet3 (upstream's graphs take no state).
+**Use when:** speech with noise a statistical denoiser leaves or smears (moving noise, a busy room); live voice in a browser (RNNoise, 30 ms); files where quality is worth an 8 MB download (DeepFilterNet3); programmes that mix speech and music (the music passes through untouched).<br>
+**Not for:** denoising music itself, a noisy song or concert (music passes untouched; with `music: 'enhance'` both models take 8 to 16 dB from every band); singing alone (about half of it passes, the rest is enhanced); clean speech with nothing to remove through RNNoise (it gates word ends and quiet syllables, 9 to 12 dB at its default limit); installs that must stay model-free ([`@audio/denoise`](https://github.com/audiojs/denoise)); reverb as such (`dereverb`); live DeepFilterNet3 (upstream's graphs take no state).
 
 ---
 

package/audio.d.ts

@@ -12,6 +12,8 @@ type Process = (inputs: Float32Array[][], outputs: Float32Array[][], params: Liv
 export interface RnnoiseOptions {
   /** 0..100 dB (default 16) */
   "limit"?: Auto
+  /** default "pass" */
+  "music"?: "pass" | "enhance"
   at?: number | string
   duration?: number | string
 }
@@ -24,5 +26,7 @@ export declare const rnnoise: {
   params: {
     /** 0..100 dB (default 16) */
     "limit": { type: "number", default: 16 }
+    /** default "pass" */
+    "music": { type: "enum", values: ["pass","enhance"], default: "pass" }
   }
 }

package/audio.js

@@ -15,14 +15,22 @@
 // most before the voice itself suffers (README, Accuracy). The limit is read per block and applies from the
 // next finished frame on.
 //
-// Channels are denoised independently; a channel's state (70 KB) is created on its first block, as the
-// worklet does, since hosts declare up to 32 channels.
+// music: 'pass' (default) has music pass through untouched, as denoise() does (guard.js): each 48 kHz input frame
+// goes to the guard too, and the output frame two before it, the one RNNoise gives back then, takes the guard's
+// decision so far; the gain between the denoised and the dry signal ramps over 200 ms, at 48 kHz, through the same
+// resampler back, and mixes at the host's rate with the input delayed by the latency, so what passes is the input
+// itself. Nothing is added to the delay: a stream decides on what has arrived, music is denoised for its first second
+// or so, and each switch comes about a second late (README, Music). 'enhance' denoises everything, as before 0.4.
+//
+// Channels are denoised independently; a channel's state (70 KB, and the guard's 0.3 MB) is created on its first
+// block, as the worklet does, since hosts declare up to 32 channels.
 
 import { model, create, FRAME, LIMIT } from './rnnoise.js'
 import { weights } from './denoise.js'
+import { guardNet, online, ramp, blend } from './guard.js'
 
 const RATE = 48000, DELAY = 2 * FRAME, PRIME = FRAME - 1, HALF = 16
-const net = model(await weights())
+const net = model(await weights()), guard = await guardNet()
 
 const sinc = x => x === 0 ? 1 : Math.sin(Math.PI * x) / (Math.PI * x)
 
@@ -89,29 +97,36 @@ function queue(len) {
 	return q
 }
 
-/** One channel at host rate sr: `run(x, y, limit)` takes host samples, gives the denoised ones `latency(sr)` later. */
+/** One channel at host rate sr: `run(x, y, limit, music)` takes host samples, gives the denoised ones `latency(sr)` later. */
 function channel(sr) {
 	let st = create(net), frame = new Float32Array(FRAME), den = new Float32Array(FRAME), z = new Float32Array(FRAME)
 	// this frame's input and the two before it: RNNoise's output for frame f is frame f − 2's time
-	let dry = new Float32Array(3 * FRAME), fill = 0, frames = 0, lim = 0, q = queue(latency(sr))
-	let back = sr === RATE ? x => { for (let k = 0; k < x.length; k++) q.push(x[k]) } : resampler(RATE, sr, q.push)
+	let dry = new Float32Array(3 * FRAME), fill = 0, frames = 0, lim = 0, pass = false, D = latency(sr)
+	let q = queue(D), gq = queue(D), dq = queue(D), decide = online(guard), next = ramp(), gz = new Float32Array(FRAME)
+	let to = q => sr === RATE ? x => { for (let k = 0; k < x.length; k++) q.push(x[k]) } : resampler(RATE, sr, q.push)
+	let back = to(q), gback = to(gq), g = new Float32Array(0), d = new Float32Array(0)
 	let push = v => {
 		frame[fill] = v * 32768; dry[(frames % 3) * FRAME + fill] = v
 		if (++fill < FRAME) return
 		st.process(frame, den)
-		// as denoise()'s offline path: the output 960 samples on, the input mixed back at `lim`
+		let on = decide(dry.subarray((frames % 3) * FRAME, (frames % 3 + 1) * FRAME))
+		// as denoise()'s offline path: the output 960 samples on, the input mixed back at `lim`, the guard's gain beside it
 		if (frames >= 2) {
-			let d = ((frames - 2) % 3) * FRAME
-			for (let k = 0; k < FRAME; k++) z[k] = (den[k] / 32768) * (1 - lim) + dry[d + k] * lim
-			back(z)
+			let o = ((frames - 2) % 3) * FRAME
+			for (let k = 0; k < FRAME; k++) { z[k] = (den[k] / 32768) * (1 - lim) + dry[o + k] * lim; gz[k] = next(!(on && pass)) }
+			back(z); gback(gz)
 		}
 		frames++; fill = 0
 	}
 	let into = sr === RATE ? x => { for (let i = 0; i < x.length; i++) push(x[i]) } : resampler(sr, RATE, push)
-	return (x, y, limit) => {
+	return (x, y, limit, music) => {
 		lim = limit ? 10 ** (-Math.abs(limit) / 20) : 0
+		pass = music !== 'enhance'
 		into(x)
-		q.pull(y)
+		for (let i = 0; i < x.length; i++) dq.push(x[i])
+		if (g.length < y.length) { g = new Float32Array(y.length); d = new Float32Array(y.length) }
+		q.pull(y); gq.pull(g.subarray(0, y.length)); dq.pull(d.subarray(0, y.length))
+		for (let i = 0; i < y.length; i++) y[i] = blend(g[i], y[i], d[i])
 	}
 }
 
@@ -120,8 +135,8 @@ export const rnnoise = (ctx) => {
 	return (inputs, outputs, params) => {
 		let inp = inputs[0], out = outputs[0]
 		if (!inp || !inp.length) return
-		let limit = params.limit[0]
-		for (let c = 0; c < inp.length; c++) (chans[c] ??= channel(sr))(inp[c], out[c], limit)
+		let limit = params.limit[0], music = params.music
+		for (let c = 0; c < inp.length; c++) (chans[c] ??= channel(sr))(inp[c], out[c], limit, music)
 	}
 }
 rnnoise.channels = 'any'
@@ -129,4 +144,5 @@ rnnoise.latency = ctx => latency(ctx.sampleRate)
 rnnoise.tail = 0
 rnnoise.params = {
 	limit: { type: 'number', min: 0, max: 100, default: LIMIT, unit: 'dB' },
+	music: { type: 'enum', values: ['pass', 'enhance'], default: 'pass' },
 }

package/deepfilter.js

@@ -38,6 +38,12 @@ export const MODEL = 'https://raw.githubusercontent.com/Rikorose/DeepFilterNet/d
 // rise; unlimited, the pauses of home narrations fall to digital silence (README, Accuracy).
 export const LIMIT = 18
 
+// The floor, dB under the voice, to which denoise() takes noise the limit would leave closer to it (mixback() in
+// denoise.js): the limit alone keeps room tone, and keeps noise as loud as the voice 18 dB under it. 40 dB under a voice
+// at -20 dBFS is ACX's -60 dBFS floor; on audio's bench/rx/isolate.mjs tune split 35, 40 and 45 score PESQ 1.96, 1.98,
+// 1.98 (0: 1.75), on VoiceBank+DEMAND's training speakers DNSMOS SIG 3.38, 3.36, 3.35 (0: 3.41) (README, Measured).
+export const FLOOR = 40
+
 // The level, dBFS, at which denoise() has the network hear the speech (`gain`): the median of the VoiceBank+DEMAND
 // test set it scores its published PESQ on (level(), -20.2); its features are not level-free (README, DeepFilterNet3).
 export const LEVEL = -20
@@ -136,9 +142,9 @@ const linspace = (a, b, n) => { let step = f(f(f(b) - f(a)) / (n - 1)); return F
 
 // ------------------------------------------------ input analysis
 
-// level(x) → the speech level, dBFS: mean power of the louder half of 50 ms frames at 48 kHz (LEVEL where there is none)
-export function level(x) {
-	let F = 2400, p = []
+// level(x, rate) → the speech level, dBFS: mean power of the louder half of 50 ms frames (LEVEL where there is none)
+export function level(x, rate = 48000) {
+	let F = Math.round(rate / 20), p = []
 	for (let i = 0; i + F <= x.length; i += F) { let s = 0; for (let j = i; j < i + F; j++) s += x[j] * x[j]; p.push(s / F) }
 	p.sort((a, b) => b - a)
 	let k = Math.max(1, p.length >> 1), s = 0

package/denoise.js

@@ -1,16 +1,17 @@
 // Neural speech enhancement: RNNoise (pure JS, weights bundled) and DeepFilterNet3 (ONNX through
 // @audio/neural-runtime, weights fetched from upstream and cached). Both run at 48 kHz on 10 ms hops;
 // other rates are resampled in and out with @audio/resample-sinc. Offline results are aligned with the
-// input: the models' delay is compensated.
+// input: the models' delay is compensated. Music passes through untouched (guard.js) unless music: 'enhance'.
 
 import resample from '@audio/resample-sinc'
 import { model as parseRNNoise, create, FRAME, LIMIT as RNNOISE_LIMIT } from './rnnoise.js'
-import { loadDeepFilter, enhance, level, bandEdge, MODEL, LEVEL, LIMIT as DFN_LIMIT } from './deepfilter.js'
+import { loadDeepFilter, enhance, level, bandEdge, MODEL, LEVEL, LIMIT as DFN_LIMIT, FLOOR } from './deepfilter.js'
+import { guardNet, offline, online, gains, blend } from './guard.js'
 
 export { create as rnnoise, FRAME } from './rnnoise.js'
 export { MODEL } from './deepfilter.js'
 
-const RATE = 48000
+const RATE = 48000, SMOOTH = 0.5
 // the attenuation limit (dB) each model gets when opts.limit is not given; 0 is upstream's output
 const LIMITS = { rnnoise: RNNOISE_LIMIT, deepfilternet3: DFN_LIMIT }
 
@@ -46,14 +47,22 @@ export default async function denoise(audio, opts = {}) {
 	else if (audio?.channelData) { channels = audio.channelData; rate = audio.sampleRate ?? rate }
 	else throw new TypeError('neural-denoise: audio must be a Float32Array, Float32Array[] or { channelData, sampleRate }')
 	if (!(rate > 0)) throw new TypeError('neural-denoise: sampleRate is required (opts.sampleRate, or audio.sampleRate)')
+	let music = opts.music ?? 'pass'
+	if (music !== 'pass' && music !== 'enhance') throw new TypeError(`neural-denoise: music is 'pass' or 'enhance', not ${music}`)
 	let handle = typeof opts.model === 'object' && opts.model ? opts.model : await load(opts.model, opts)
 	try {
-		let out = [], limit = opts.limit ?? LIMITS[handle.model]
+		let out = [], limit = opts.limit ?? LIMITS[handle.model], nn = music === 'pass' ? await guardNet() : null, dfn = handle.model !== 'rnnoise'
 		for (let ch of channels) {
-			let x = rate === RATE ? Float32Array.from(ch) : resample(Float32Array.from(ch), { from: rate, to: RATE })
-			let y = handle.model === 'rnnoise' ? rnnoiseOffline(x, handle.net, limit)
-				: await enhance(x, handle.net, { ...opts, limit, gain: 10 ** ((LEVEL - level(x)) / 20), edge: bandEdge(ch, rate), voice: true })
-			out.push(rate === RATE ? y : fit(resample(y, { from: RATE, to: rate }), ch.length))
+			let x = rate === RATE ? Float32Array.from(ch) : resample(Float32Array.from(ch), { from: rate, to: RATE }), y, pass
+			// RNNoise decides as its stream does (guard.js online()), DeepFilterNet3 with the whole input in view
+			if (!dfn) ({ y, pass } = rnnoiseOffline(x, handle.net, limit, nn))
+			else {
+				pass = nn && offline(x, nn)
+				y = pass?.every(Boolean) ? x : await enhance(x, handle.net, { ...opts, limit: 0, gain: 10 ** ((LEVEL - level(x)) / 20), edge: bandEdge(ch, rate), voice: true })
+			}
+			if (rate !== RATE) y = fit(resample(y, { from: RATE, to: rate }), ch.length)
+			if (pass) y = mix(y, ch, gains(pass, x.length, dfn), rate)
+			out.push(dfn ? mixback(ch, y, { limit, floor: opts.floor, sampleRate: rate }) : y)
 		}
 		return audio instanceof Float32Array ? out[0] : Array.isArray(audio) ? out : { ...audio, channelData: out, sampleRate: rate }
 	} finally {
@@ -61,15 +70,62 @@ export default async function denoise(audio, opts = {}) {
 	}
 }
 
+/**
+ * mixback(x, y, { limit, floor, sampleRate }) → y + a·(x − y): the input x mixed back into a model's output y, the noise
+ * the model took (x − y) back at gain a. a = 10^(−limit/20), so the noise drops by `limit` dB and room tone stays; where
+ * the noise stands higher than `floor` dB under the voice, a falls until it lies there, so noise loud against the voice
+ * is not merely turned down. Per 10 ms frame: the noise's mean power over the 0.5 s around it against the voice's
+ * level, the louder half of y's 50 ms frames (level()); a interpolated between frames. Where x and y agree (music the
+ * guard passed) the output is y bit for bit. limit 0: y itself, the model's whole removal; floor 0: a constant
+ * 10^(−limit/20), as before 0.5. denoise() applies it to DeepFilterNet3; `audio`'s deepfilter op to the unlimited
+ * output it keeps.
+ */
+export function mixback(x, y, { limit = DFN_LIMIT, floor = FLOOR, sampleRate = RATE } = {}) {
+	let n = x.length, out = new Float32Array(n), cap = limit ? 10 ** (-Math.abs(limit) / 20) : 0
+	if (!cap) { out.set(y.subarray(0, n)); return out }
+	if (!floor) { for (let i = 0; i < n; i++) out[i] = y[i] + cap * (x[i] - y[i]); return out }
+	// the noise's energy per frame, summed (c[k]: frames before k), and the frames' mean over the window around each
+	let F = Math.max(1, Math.round(sampleRate / 100)), K = Math.ceil(n / F), c = new Float64Array(K + 1), g = new Float64Array(K), h = Math.round(50 * SMOOTH)
+	for (let k = 0; k < K; k++) {
+		let s = 0, e = Math.min(n, (k + 1) * F)
+		for (let i = k * F; i < e; i++) s += (x[i] - y[i]) ** 2
+		c[k + 1] = c[k] + s
+	}
+	let voice = 10 ** ((level(y, sampleRate) - Math.abs(floor)) / 10)
+	for (let k = 0; k < K; k++) {
+		let a = Math.max(0, k - h), b = Math.min(K, k + h + 1)
+		g[k] = Math.min(cap, Math.sqrt(voice / Math.max((c[b] - c[a]) / (Math.min(n, b * F) - a * F), 1e-30)))
+	}
+	for (let i = 0; i < n; i++) {
+		let t = i / F - .5, k = Math.min(K - 1, Math.max(0, Math.floor(t))), w = Math.min(1, Math.max(0, t - k))
+		out[i] = y[i] + (k + 1 < K ? g[k] + w * (g[k + 1] - g[k]) : g[k]) * (x[i] - y[i])
+	}
+	return out
+}
+
+// the enhanced y where the gain g (at 48 kHz) is 1, the input x where it is 0: music passes untouched
+function mix(y, x, g, rate) {
+	if (rate !== RATE) g = fit(resample(g, { from: RATE, to: rate }), x.length)
+	let out = new Float32Array(x.length)
+	for (let i = 0; i < x.length; i++) out[i] = blend(g[i], y[i], x[i])
+	return out
+}
+
 const fit = (y, n) => y.length === n ? y : (() => { let o = new Float32Array(n); o.set(y.subarray(0, n)); return o })()
 
-// One channel through RNNoise at int16 scale, delay removed: out[i] lines up with x[i]
-function rnnoiseOffline(x, net, limit) {
+// One channel through RNNoise at int16 scale, delay removed: out[i] lines up with x[i]. With the guard's network,
+// also `pass` per 480 samples of the output, as the stream decides it: each output frame takes the decision made
+// once the input frame two on (RNNoise's delay) is in, from the input so far
+function rnnoiseOffline(x, net, limit, nn) {
 	let st = create(net), delay = 2 * FRAME, L = x.length, n = Math.ceil((L + delay) / FRAME) * FRAME
-	let pad = new Float32Array(n), y = new Float32Array(n)
+	let pad = new Float32Array(n), y = new Float32Array(n), raw = new Float32Array(n), decide = nn && online(nn), on = []
 	for (let i = 0; i < L; i++) pad[i] = x[i] * 32768
-	for (let i = 0; i < n; i += FRAME) st.process(pad.subarray(i, i + FRAME), y.subarray(i, i + FRAME))
+	raw.set(x)
+	for (let i = 0; i < n; i += FRAME) {
+		st.process(pad.subarray(i, i + FRAME), y.subarray(i, i + FRAME))
+		if (decide) on.push(decide(raw.subarray(i, i + FRAME)))
+	}
 	let lim = limit ? 10 ** (-Math.abs(limit) / 20) : 0, out = new Float32Array(L)
 	for (let i = 0; i < L; i++) out[i] = (y[i + delay] / 32768) * (1 - lim) + x[i] * lim
-	return out
+	return { y: out, pass: decide && Uint8Array.from(on.slice(2)) }
 }

package/gemm.wat

@@ -0,0 +1,66 @@
+;; The music guard's float32 products (guard.js: the classifier's upper convolutions through im2col, and its dense
+;; layers) in WebAssembly SIMD, embedded in guard.js as bytes. Rebuild: wat2wasm gemm.wat -o gemm.wasm (WABT).
+(module
+  (memory (export "memory") 1)
+  ;; gemm(w, x, b, y, rows, k, n): y[p, r] = max(0, b[r] + Σc w[r, c] · x[p, c]) for p < n, r < rows; w row-major
+  ;; rows × k, x n × k, y n × rows; rows and k multiples of 4. Four rows and two positions at a time, lane l of a row
+  ;; summing the columns c ≡ l (mod 4), the lanes then added (l0 + l1) + (l2 + l3), then the bias: guard.js's JS loop
+  ;; does the same float ops in the same order
+  (func (export "gemm") (param $w i32) (param $x i32) (param $b i32) (param $y i32) (param $rows i32) (param $k i32) (param $n i32)
+    (local $r i32) (local $c i32) (local $p i32) (local $x0 i32) (local $x1 i32) (local $w0 i32) (local $kb i32)
+    (local $a0 v128) (local $a1 v128) (local $a2 v128) (local $a3 v128)
+    (local $b0 v128) (local $b1 v128) (local $b2 v128) (local $b3 v128)
+    (local $u v128) (local $v v128) (local $s v128) (local $t v128)
+    (local.set $kb (i32.shl (local.get $k) (i32.const 2)))
+    (loop $pairs
+      (local.set $x0 (i32.add (local.get $x) (i32.mul (local.get $p) (local.get $kb))))
+      ;; the second of the pair: itself again when n is odd (computed, not stored)
+      (local.set $x1 (select (i32.add (local.get $x0) (local.get $kb)) (local.get $x0) (i32.lt_u (i32.add (local.get $p) (i32.const 1)) (local.get $n))))
+      (local.set $r (i32.const 0))
+      (loop $rowblk
+        (local.set $a0 (v128.const i32x4 0 0 0 0)) (local.set $a1 (v128.const i32x4 0 0 0 0))
+        (local.set $a2 (v128.const i32x4 0 0 0 0)) (local.set $a3 (v128.const i32x4 0 0 0 0))
+        (local.set $b0 (v128.const i32x4 0 0 0 0)) (local.set $b1 (v128.const i32x4 0 0 0 0))
+        (local.set $b2 (v128.const i32x4 0 0 0 0)) (local.set $b3 (v128.const i32x4 0 0 0 0))
+        (local.set $w0 (i32.add (local.get $w) (i32.mul (local.get $r) (local.get $kb))))
+        (local.set $c (i32.const 0))
+        (loop $cols
+          (local.set $u (v128.load (i32.add (local.get $x0) (local.get $c))))
+          (local.set $v (v128.load (i32.add (local.get $x1) (local.get $c))))
+          (local.set $s (v128.load (i32.add (local.get $w0) (local.get $c))))
+          (local.set $a0 (f32x4.add (local.get $a0) (f32x4.mul (local.get $s) (local.get $u))))
+          (local.set $b0 (f32x4.add (local.get $b0) (f32x4.mul (local.get $s) (local.get $v))))
+          (local.set $s (v128.load (i32.add (i32.add (local.get $w0) (local.get $kb)) (local.get $c))))
+          (local.set $a1 (f32x4.add (local.get $a1) (f32x4.mul (local.get $s) (local.get $u))))
+          (local.set $b1 (f32x4.add (local.get $b1) (f32x4.mul (local.get $s) (local.get $v))))
+          (local.set $s (v128.load (i32.add (i32.add (local.get $w0) (i32.shl (local.get $kb) (i32.const 1))) (local.get $c))))
+          (local.set $a2 (f32x4.add (local.get $a2) (f32x4.mul (local.get $s) (local.get $u))))
+          (local.set $b2 (f32x4.add (local.get $b2) (f32x4.mul (local.get $s) (local.get $v))))
+          (local.set $s (v128.load (i32.add (i32.add (local.get $w0) (i32.mul (local.get $kb) (i32.const 3))) (local.get $c))))
+          (local.set $a3 (f32x4.add (local.get $a3) (f32x4.mul (local.get $s) (local.get $u))))
+          (local.set $b3 (f32x4.add (local.get $b3) (f32x4.mul (local.get $s) (local.get $v))))
+          (local.set $c (i32.add (local.get $c) (i32.const 16)))
+          (br_if $cols (i32.lt_u (local.get $c) (local.get $kb))))
+        ;; transpose-add the four row sums: lane q = Σ a_q
+        (local.set $t (v128.load (i32.add (local.get $b) (i32.shl (local.get $r) (i32.const 2)))))
+        (local.set $s (f32x4.add
+          (f32x4.add (i8x16.shuffle 0 1 2 3 16 17 18 19 0 1 2 3 16 17 18 19 (local.get $a0) (local.get $a1)) (i8x16.shuffle 4 5 6 7 20 21 22 23 4 5 6 7 20 21 22 23 (local.get $a0) (local.get $a1)))
+          (f32x4.add (i8x16.shuffle 8 9 10 11 24 25 26 27 8 9 10 11 24 25 26 27 (local.get $a0) (local.get $a1)) (i8x16.shuffle 12 13 14 15 28 29 30 31 12 13 14 15 28 29 30 31 (local.get $a0) (local.get $a1)))))
+        (local.set $u (f32x4.add
+          (f32x4.add (i8x16.shuffle 0 1 2 3 16 17 18 19 0 1 2 3 16 17 18 19 (local.get $a2) (local.get $a3)) (i8x16.shuffle 4 5 6 7 20 21 22 23 4 5 6 7 20 21 22 23 (local.get $a2) (local.get $a3)))
+          (f32x4.add (i8x16.shuffle 8 9 10 11 24 25 26 27 8 9 10 11 24 25 26 27 (local.get $a2) (local.get $a3)) (i8x16.shuffle 12 13 14 15 28 29 30 31 12 13 14 15 28 29 30 31 (local.get $a2) (local.get $a3)))))
+        (v128.store (i32.add (local.get $y) (i32.shl (i32.add (i32.mul (local.get $p) (local.get $rows)) (local.get $r)) (i32.const 2)))
+          (f32x4.max (v128.const f32x4 0 0 0 0) (f32x4.add (local.get $t) (i8x16.shuffle 0 1 2 3 4 5 6 7 16 17 18 19 20 21 22 23 (local.get $s) (local.get $u)))))
+        (if (i32.lt_u (i32.add (local.get $p) (i32.const 1)) (local.get $n)) (then
+          (local.set $s (f32x4.add
+            (f32x4.add (i8x16.shuffle 0 1 2 3 16 17 18 19 0 1 2 3 16 17 18 19 (local.get $b0) (local.get $b1)) (i8x16.shuffle 4 5 6 7 20 21 22 23 4 5 6 7 20 21 22 23 (local.get $b0) (local.get $b1)))
+            (f32x4.add (i8x16.shuffle 8 9 10 11 24 25 26 27 8 9 10 11 24 25 26 27 (local.get $b0) (local.get $b1)) (i8x16.shuffle 12 13 14 15 28 29 30 31 12 13 14 15 28 29 30 31 (local.get $b0) (local.get $b1)))))
+          (local.set $u (f32x4.add
+            (f32x4.add (i8x16.shuffle 0 1 2 3 16 17 18 19 0 1 2 3 16 17 18 19 (local.get $b2) (local.get $b3)) (i8x16.shuffle 4 5 6 7 20 21 22 23 4 5 6 7 20 21 22 23 (local.get $b2) (local.get $b3)))
+            (f32x4.add (i8x16.shuffle 8 9 10 11 24 25 26 27 8 9 10 11 24 25 26 27 (local.get $b2) (local.get $b3)) (i8x16.shuffle 12 13 14 15 28 29 30 31 12 13 14 15 28 29 30 31 (local.get $b2) (local.get $b3)))))
+          (v128.store (i32.add (local.get $y) (i32.shl (i32.add (i32.mul (i32.add (local.get $p) (i32.const 1)) (local.get $rows)) (local.get $r)) (i32.const 2)))
+            (f32x4.max (v128.const f32x4 0 0 0 0) (f32x4.add (local.get $t) (i8x16.shuffle 0 1 2 3 4 5 6 7 16 17 18 19 20 21 22 23 (local.get $s) (local.get $u)))))))
+        (local.set $r (i32.add (local.get $r) (i32.const 4)))
+        (br_if $rowblk (i32.lt_u (local.get $r) (local.get $rows))))
+      (local.set $p (i32.add (local.get $p) (i32.const 2)))
+      (br_if $pairs (i32.lt_u (local.get $p) (local.get $n))))))

package/guard.js

@@ -0,0 +1,346 @@
+// Music guard: speech and noise are enhanced, music passes through untouched, segment by segment.
+//
+// Both models enhance speech, and to them everything else is noise: music loses 8 to 16 dB in every band at their
+// default limits. What tells music from speech here is inaSpeechSegmenter's speech/music/noise CNN (D. Doukhan,
+// J. Carrive, F. Vallet, A. Larcher, S. Meignier, "An Open-Source Speaker Gender Detection Framework for Monitoring
+// Gender Equality", ICASSP 2018; ina-foss/inaSpeechSegmenter, MIT; its 'smn' engine, keras_speech_music_noise_cnn.hdf5),
+// trained on French radio and TV, where it labels speech over a music bed as speech and songs as music. Its input as ina
+// computes it (sidekit's mfcc() at ina's settings): 16 kHz audio, pre-emphasis 0.97 within each frame, 25 ms Hann
+// frames every 10 ms, a 512-point power spectrum, 24 triangular HTK-mel filters from 100 Hz to 8 kHz, natural log. The
+// lowest 21 bands over 68 frames (0.68 s) make a patch, standardized by its own mean and deviation, so the level
+// doesn't matter. Four convolutions and four dense layers give speech, music and noise; 782,403 weights, in guard.bin
+// as float16 (1.56 MB, scripts/guard.py).
+//
+// A patch every STEP frames (ina takes one every 2; on the training material every 10 decides the same, at a fifth of
+// the work). Offline, the classes become segments as ina makes them, by Viterbi over the whole input: three states,
+// a switch costing 10^-80 per 20 ms (ina's SpeechMusicNoise, diag_trans_exp(80)), each patch PRIOR nats in favour of
+// speech (from 0.5): a voice under a music bed as loud as itself reaches ina as music half the time, speech at a mean
+// probability of 0.47, songs at 0.01; PRIOR is the most that passes every frame of the training songs and
+// accompaniments passed without it (README, Music). A stream can only use what has arrived: the forward recursion of
+// the same chain, for two states (music, the rest), keeps the log-odds of music, bounded at ±BOUND, a patch moving them
+// by at most ln(1/FLOOR), each patch PRIOR nats in favour of speech; it switches to music above HYST and back below 0
+// (hysteresis). Either way, music passes (the enhanced signal's gain 0), speech and noise are enhanced (gain 1), and
+// the gain moves over RAMP samples, raised cosine: offline centered on the segment boundary, streaming from the
+// decision on.
+//
+// The products of the three upper convolutions and the dense layers run in a WebAssembly SIMD kernel (gemm.wat) or
+// its JS form, the same float ops in the same order, so the same bits either way.
+
+const f = Math.fround
+
+export const STEP = 10, RAMP = 9600, BOUND = 20, PRIOR = 1, HYST = 5, FLOOR = 1e-2
+const COST = 80 * Math.LN10 * 2 / STEP
+
+// ------------------------------------------------ the network
+
+// conv: [out, kh, kw, in], dense: [out, in], each followed by its bias, batch norms folded into the layer before
+const LAYERS = [[64, 7, 4, 1], [128, 3, 3, 64], [128, 3, 3, 128], [256, 3, 3, 128], [256, 256], [256, 256], [256, 256], [256, 256], [3, 256]]
+export const WEIGHTS = LAYERS.reduce((n, s) => n + s.reduce((a, b) => a * b) + s[0], 0)
+
+// IEEE 754 binary16 → number
+const half = h => { let e = (h >> 10) & 31, m = h & 1023, v = e ? (1 + m / 1024) * 2 ** (e - 15) : m * 2 ** -24; return h >> 15 ? -v : v }
+
+// gemm.wat, compiled by wat2wasm (1,074 bytes): y[p, r] = max(0, b[r] + Σc w[r, c] · x[p, c]), four rows and two
+// positions at a time, each row in four float32 lanes summed (l0 + l1) + (l2 + l3)
+const GEMM = new Uint8Array([
+	0,97,115,109,1,0,0,0,1,11,1,96,7,127,127,127,127,127,127,127,0,3,2,1,0,5,3,1,0,1,7,17,2,6,109,101,109,111,114,121,2,0,
+	4,103,101,109,109,0,0,10,254,7,1,251,7,2,7,127,12,123,32,5,65,2,116,33,13,3,64,32,1,32,9,32,13,108,106,33,10,32,10,32,
+	13,106,32,10,32,9,65,1,106,32,6,73,27,33,11,65,0,33,7,3,64,253,12,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,33,14,253,12,0,0,0,
+	0,0,0,0,0,0,0,0,0,0,0,0,0,33,15,253,12,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,33,16,253,12,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
+	33,17,253,12,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,33,18,253,12,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,33,19,253,12,0,0,0,0,0,0,0,
+	0,0,0,0,0,0,0,0,0,33,20,253,12,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,33,21,32,0,32,7,32,13,108,106,33,12,65,0,33,8,3,64,32,
+	10,32,8,106,253,0,4,0,33,22,32,11,32,8,106,253,0,4,0,33,23,32,12,32,8,106,253,0,4,0,33,24,32,14,32,24,32,22,253,230,1,
+	253,228,1,33,14,32,18,32,24,32,23,253,230,1,253,228,1,33,18,32,12,32,13,106,32,8,106,253,0,4,0,33,24,32,15,32,24,32,
+	22,253,230,1,253,228,1,33,15,32,19,32,24,32,23,253,230,1,253,228,1,33,19,32,12,32,13,65,1,116,106,32,8,106,253,0,4,0,
+	33,24,32,16,32,24,32,22,253,230,1,253,228,1,33,16,32,20,32,24,32,23,253,230,1,253,228,1,33,20,32,12,32,13,65,3,108,
+	106,32,8,106,253,0,4,0,33,24,32,17,32,24,32,22,253,230,1,253,228,1,33,17,32,21,32,24,32,23,253,230,1,253,228,1,33,21,
+	32,8,65,16,106,33,8,32,8,32,13,73,13,0,11,32,2,32,7,65,2,116,106,253,0,4,0,33,25,32,14,32,15,253,13,0,1,2,3,16,17,18,
+	19,0,1,2,3,16,17,18,19,32,14,32,15,253,13,4,5,6,7,20,21,22,23,4,5,6,7,20,21,22,23,253,228,1,32,14,32,15,253,13,8,9,10,
+	11,24,25,26,27,8,9,10,11,24,25,26,27,32,14,32,15,253,13,12,13,14,15,28,29,30,31,12,13,14,15,28,29,30,31,253,228,1,253,
+	228,1,33,24,32,16,32,17,253,13,0,1,2,3,16,17,18,19,0,1,2,3,16,17,18,19,32,16,32,17,253,13,4,5,6,7,20,21,22,23,4,5,6,7,
+	20,21,22,23,253,228,1,32,16,32,17,253,13,8,9,10,11,24,25,26,27,8,9,10,11,24,25,26,27,32,16,32,17,253,13,12,13,14,15,
+	28,29,30,31,12,13,14,15,28,29,30,31,253,228,1,253,228,1,33,22,32,3,32,9,32,4,108,32,7,106,65,2,116,106,253,12,0,0,0,0,
+	0,0,0,0,0,0,0,0,0,0,0,0,32,25,32,24,32,22,253,13,0,1,2,3,4,5,6,7,16,17,18,19,20,21,22,23,253,228,1,253,233,1,253,11,4,
+	0,32,9,65,1,106,32,6,73,4,64,32,18,32,19,253,13,0,1,2,3,16,17,18,19,0,1,2,3,16,17,18,19,32,18,32,19,253,13,4,5,6,7,20,
+	21,22,23,4,5,6,7,20,21,22,23,253,228,1,32,18,32,19,253,13,8,9,10,11,24,25,26,27,8,9,10,11,24,25,26,27,32,18,32,19,253,
+	13,12,13,14,15,28,29,30,31,12,13,14,15,28,29,30,31,253,228,1,253,228,1,33,24,32,20,32,21,253,13,0,1,2,3,16,17,18,19,0,
+	1,2,3,16,17,18,19,32,20,32,21,253,13,4,5,6,7,20,21,22,23,4,5,6,7,20,21,22,23,253,228,1,32,20,32,21,253,13,8,9,10,11,
+	24,25,26,27,8,9,10,11,24,25,26,27,32,20,32,21,253,13,12,13,14,15,28,29,30,31,12,13,14,15,28,29,30,31,253,228,1,253,
+	228,1,33,22,32,3,32,9,65,1,106,32,4,108,32,7,106,65,2,116,106,253,12,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,32,25,32,24,32,
+	22,253,13,0,1,2,3,4,5,6,7,16,17,18,19,20,21,22,23,253,228,1,253,233,1,253,11,4,0,11,32,7,65,4,106,33,7,32,7,32,4,73,
+	13,0,11,32,9,65,2,106,33,9,32,9,32,6,73,13,0,11,11,
+])
+
+/** net(bytes, { simd }) → the classifier over guard.bin; rest(p1) takes conv1's pooled output, gives [speech, music, noise]. */
+export function net(bytes, { simd = true } = {}) {
+	let u = new Uint16Array(bytes.buffer.slice(bytes.byteOffset, bytes.byteOffset + bytes.byteLength)), o = 0
+	if (u.length !== WEIGHTS) throw new Error(`neural-denoise: guard weights are ${u.length} values, want ${WEIGHTS}`)
+	let take = n => { let a = new Float32Array(n); for (let i = 0; i < n; i++) a[i] = half(u[o++]); return a }
+	let [c1, c2, c3, c4, ...dense] = LAYERS.map(s => ({ s, w: take(s.reduce((a, b) => a * b)), b: take(s[0]) }))
+	let out = dense.pop(), mm = matmul([c2, c3, c4, ...dense], simd), col = new Float32Array(12 * 5 * 1152)
+	// conv1's kernel sums: a standardized patch (x − μ)/σ convolves to (conv(x) − μ·Σk)/σ
+	let ksum = new Float32Array(64)
+	for (let a = 0; a < 64; a++) for (let j = 0; j < 28; j++) ksum[a] += c1.w[a * 28 + j]
+	// valid convolution, channels last, through im2col: x [H][W][C] → [H − kh + 1][W − kw + 1][co], ReLU
+	function conv(x, H, W, C, l) {
+		let [co, kh, kw] = l.s, oh = H - kh + 1, ow = W - kw + 1, q = 0
+		for (let i = 0; i < oh; i++) for (let j = 0; j < ow; j++) for (let h = 0; h < kh; h++) {
+			let a = ((i + h) * W + j) * C
+			col.set(x.subarray(a, a + kw * C), q); q += kw * C
+		}
+		return [mm(l, col, oh * ow), oh, ow, co]
+	}
+	return {
+		c1, ksum,
+		rest(p1) {
+			for (let i = 0; i < p1.length; i++) if (p1[i] < 0) p1[i] = 0
+			let t = conv(p1, 16, 9, 64, c2)
+			t = pool(conv(...t, c3), 2, 2, 0)
+			let v = pool(conv(...t, c4), 4, 1, 0)[0]
+			for (let l of dense) v = mm(l, v, 1)
+			let z = [0, 1, 2].map(a => { let s = out.b[a]; for (let k = 0; k < 256; k++) s += out.w[a * 256 + k] * v[k]; return s })
+			let m = Math.max(...z), e = z.map(q => Math.exp(q - m)), s = e[0] + e[1] + e[2]
+			return e.map(q => q / s)
+		},
+	}
+}
+
+// matmul(layers) → mm(l, x, n): the kernel over layer l's weights for n rows of x; wasm where WebAssembly SIMD runs
+function matmul(layers, simd = true) {
+	let mod = null
+	try { mod = simd && typeof WebAssembly === 'object' ? new WebAssembly.Module(GEMM) : null } catch {}
+	if (!mod) return (l, x, n) => gemm(l.w, x, l.b, l.s[0], l.w.length / l.s[0], n)
+	let inst = new WebAssembly.Instance(mod), mem = inst.exports.memory, top = 0, at = new Map()
+	let alloc = n => { let p = top; top += n * 4; return p }
+	for (let l of layers) at.set(l, [alloc(l.w.length), alloc(l.b.length)])
+	let X = alloc(12 * 5 * 1152), Y = alloc(14 * 7 * 128)
+	mem.grow(Math.ceil(top / 65536))
+	let F = new Float32Array(mem.buffer)
+	for (let l of layers) { F.set(l.w, at.get(l)[0] / 4); F.set(l.b, at.get(l)[1] / 4) }
+	return (l, x, n) => {
+		let rows = l.s[0], k = l.w.length / rows, [w, b] = at.get(l)
+		F.set(x.subarray(0, n * k), X / 4)
+		inst.exports.gemm(w, X, b, Y, rows, k, n)
+		return F.slice(Y / 4, Y / 4 + n * rows)
+	}
+}
+
+// gemm.wat in JS
+function gemm(w, x, b, rows, k, n) {
+	let y = new Float32Array(n * rows)
+	for (let p = 0; p < n; p++) for (let r = 0; r < rows; r++) {
+		let a0 = 0, a1 = 0, a2 = 0, a3 = 0, wo = r * k, xo = p * k
+		for (let c = 0; c < k; c += 4) {
+			a0 = f(a0 + f(w[wo + c] * x[xo + c])); a1 = f(a1 + f(w[wo + c + 1] * x[xo + c + 1]))
+			a2 = f(a2 + f(w[wo + c + 2] * x[xo + c + 2])); a3 = f(a3 + f(w[wo + c + 3] * x[xo + c + 3]))
+		}
+		let s = f(b[r] + f(f(a0 + a1) + f(a2 + a3)))
+		y[p * rows + r] = s > 0 ? s : 0
+	}
+	return y
+}
+
+// max pooling, window = stride = ph × pw, `pt` rows of padding on top (keras 'same' pads the first pool's rows 1 + 1)
+function pool([x, H, W, C], ph, pw, pt) {
+	let oh = Math.ceil(H / ph), ow = Math.ceil(W / pw), y = new Float32Array(oh * ow * C).fill(-Infinity)
+	for (let i = 0; i < H; i++) for (let j = 0; j < W; j++) {
+		let a = Math.floor((i + pt) / ph), c = Math.floor(j / pw), xo = (i * W + j) * C, yo = (a * ow + c) * C
+		for (let k = 0; k < C; k++) if (x[xo + k] > y[yo + k]) y[yo + k] = x[xo + k]
+	}
+	return [y, oh, ow, C]
+}
+
+// ------------------------------------------------ features
+
+// 48 → 16 kHz: a 31-tap low-pass at 8 kHz (sinc under a Kaiser window, β = 6), causal, every third sample kept. Its
+// stop band (60 dB down from 10.6 kHz, whose alias lands at 5.4 kHz) covers the 21 bands used, which end at 5.4 kHz.
+const TAPS = 31, LP = new Float32Array(TAPS)
+{
+	let I0 = x => { let s = 1, t = 1; for (let k = 1; k < 30; k++) { t *= (x / 2 / k) ** 2; s += t } return s }, sum = 0
+	for (let i = 0; i < TAPS; i++) {
+		let t = i - (TAPS - 1) / 2, r = 2 * t / (TAPS - 1)
+		LP[i] = (t ? Math.sin(Math.PI * t / 3) / (Math.PI * t) : 1 / 3) * I0(6 * Math.sqrt(1 - r * r)) / I0(6); sum += LP[i]
+	}
+	for (let i = 0; i < TAPS; i++) LP[i] /= sum
+}
+
+// sidekit's trfbank(16000, 512, 100, 8000, 0, 24): triangles evenly spaced in HTK mel, each of area 1 (the lowest 21)
+const MELS = 24, BANDS = 21, NFFT = 512, WIN = 400, HOP = 160, PATCH = 68
+const mel = hz => 2595 * Math.log10(1 + hz / 700), hz = m => 700 * (10 ** (m / 2595) - 1)
+const FB = Array.from({ length: BANDS }, (_, i) => {
+	let edge = k => hz(mel(100) + k * (mel(8000) - mel(100)) / (MELS + 1)), [lo, c, hi] = [edge(i), edge(i + 1), edge(i + 2)]
+	let h = 2 / (hi - lo), w = [], start = Math.floor(lo * NFFT / 16000) + 1, mid = Math.floor(c * NFFT / 16000)
+	for (let k = start; k <= mid; k++) w.push(h / (c - lo) * (k * 16000 / NFFT - lo))
+	for (let k = mid + 1; k < Math.min(Math.floor(hi * NFFT / 16000) + 1, NFFT) - 1; k++) w.push(h / (hi - c) * (hi - k * 16000 / NFFT))
+	return { start, w: Float64Array.from(w) }
+})
+// numpy.hanning(400)
+const HANN = Float64Array.from({ length: WIN }, (_, n) => .5 - .5 * Math.cos(2 * Math.PI * n / (WIN - 1)))
+
+// radix-2 FFT of 512 points, the frame zero-padded → power of bins 0…256
+const REV = new Uint16Array(NFFT), COS = new Float64Array(NFFT / 2), SIN = new Float64Array(NFFT / 2)
+for (let i = 0; i < NFFT; i++) { let r = 0; for (let b = 0; b < 9; b++) r |= ((i >> b) & 1) << (8 - b); REV[i] = r }
+for (let i = 0; i < NFFT / 2; i++) { COS[i] = Math.cos(2 * Math.PI * i / NFFT); SIN[i] = -Math.sin(2 * Math.PI * i / NFFT) }
+function power(x, re, im, P) {
+	for (let i = 0; i < NFFT; i++) { re[REV[i]] = i < WIN ? x[i] : 0; im[i] = 0 }
+	for (let size = 2; size <= NFFT; size <<= 1) {
+		let half = size >> 1, step = NFFT / size
+		for (let s = 0; s < NFFT; s += size) for (let j = 0; j < half; j++) {
+			let a = s + j, b = a + half, c = COS[j * step], d = SIN[j * step]
+			let tr = re[b] * c - im[b] * d, ti = re[b] * d + im[b] * c
+			re[b] = re[a] - tr; im[b] = im[a] - ti; re[a] += tr; im[a] += ti
+		}
+	}
+	for (let k = 0; k <= NFFT / 2; k++) P[k] = re[k] * re[k] + im[k] * im[k]
+}
+
+/**
+ * analyzer(nn) → push(x): 48 kHz samples in, the [speech, music, noise] of each patch completed out, in order (null
+ * for a patch with digital silence in it: its log-mel is −∞, what ina calls not finite and scores as neither class).
+ * Patch k covers the 16 kHz frames STEP·k … STEP·k + 67, frame j the 16 kHz samples 160j … 160j + 399; at 48 kHz
+ * (the low-pass delays 15 samples) it is centered on sample 480·(STEP·k + 34.716) and complete once sample
+ * 480·(STEP·k + 67) + 1197 is in, in 48 kHz frame STEP·k + 69. push.end() → for an input shorter than a patch, its
+ * one patch, padded with its least log-mel value as ina pads.
+ */
+export function analyzer(nn) {
+	let hist = new Float32Array(TAPS), hn = 0, n48 = 0, s16 = new Float32Array(WIN), n16 = 0
+	let frame = new Float64Array(WIN), re = new Float64Array(NFFT), im = new Float64Array(NFFT), P = new Float64Array(NFFT / 2 + 1)
+	// the last 68 frames' log-mel, and conv1 over the 7 frames from each: [18][64], no bias
+	let M = new Float32Array(PATCH * BANDS), C = new Float32Array(PATCH * 1152), mels = new Float32Array(BANDS)
+	let p1 = new Float32Array(16 * 9 * 64), nf = 0, low = Infinity, out = [], { c1, ksum } = nn
+	let row = j => (j % PATCH) * BANDS
+	function addFrame() {
+		// pre-emphasis within the frame, its first sample against itself
+		for (let i = 0; i < WIN; i++) frame[i] = (s16[(n16 + i) % WIN] - .97 * s16[(n16 + Math.max(i - 1, 0)) % WIN]) * HANN[i]
+		power(frame, re, im, P)
+		for (let b = 0; b < BANDS; b++) {
+			let { start, w } = FB[b], s = 0
+			for (let k = 0; k < w.length; k++) s += P[start + k] * w[k]
+			mels[b] = Math.log(s); low = Math.min(low, mels[b])
+		}
+		addRow(mels)
+	}
+	function addRow(v) {
+		M.set(v, row(nf++))
+		if (nf >= 7) {
+			let o = ((nf - 7) % PATCH) * 1152
+			for (let j = 0; j < 18; j++) for (let a = 0; a < 64; a++) {
+				let s = 0
+				for (let h = 0; h < 7; h++) {
+					let m = row(nf - 7 + h) + j, w = (a * 7 + h) * 4
+					s += M[m] * c1.w[w] + M[m + 1] * c1.w[w + 1] + M[m + 2] * c1.w[w + 2] + M[m + 3] * c1.w[w + 3]
+				}
+				C[o + j * 64 + a] = s
+			}
+		}
+		if (nf >= PATCH && (nf - PATCH) % STEP === 0) out.push(patch(nf - PATCH))
+	}
+	// the patch from frame t: standardized by the mean and deviation of its 68 × 21 values; conv1 from the shared rows,
+	// pooled 4 × 2 (the max commutes with the patch's increasing affine map), then the rest of the network
+	function patch(t) {
+		let s = 0, ss = 0
+		for (let j = t; j < t + PATCH; j++) for (let b = 0, r = row(j); b < BANDS; b++) s += M[r + b]
+		let mu = s / (PATCH * BANDS)
+		if (!isFinite(mu)) return null
+		for (let j = t; j < t + PATCH; j++) for (let b = 0, r = row(j); b < BANDS; b++) ss += (M[r + b] - mu) ** 2
+		let sd = Math.sqrt(ss / (PATCH * BANDS))
+		if (!(sd > 0)) return null
+		for (let m = 0; m < 16; m++) for (let c = 0; c < 9; c++) for (let a = 0; a < 64; a++) {
+			let v = -Infinity
+			for (let i = Math.max(0, 4 * m - 1); i <= Math.min(61, 4 * m + 2); i++) {
+				let q = ((t + i) % PATCH) * 1152 + 2 * c * 64 + a
+				v = Math.max(v, C[q], C[q + 64])
+			}
+			p1[(m * 9 + c) * 64 + a] = (v - mu * ksum[a]) / sd + c1.b[a]
+		}
+		return nn.rest(p1)
+	}
+	function push(x) {
+		out = []
+		for (let i = 0; i < x.length; i++) {
+			hist[hn = (hn + 1) % TAPS] = x[i]
+			if (n48++ % 3) continue
+			let v = 0
+			for (let k = 0; k < TAPS; k++) v += LP[k] * hist[(hn - k + TAPS) % TAPS]
+			s16[n16++ % WIN] = v
+			if (n16 >= WIN && (n16 - WIN) % HOP === 0) addFrame()
+		}
+		return out
+	}
+	push.end = () => {
+		out = []
+		if (nf && nf < PATCH) { mels.fill(low); while (nf < PATCH) addRow(mels) }
+		return out
+	}
+	return push
+}
+
+// ------------------------------------------------ decisions
+
+const lp = (p, floor) => Math.log(Math.max(p, floor))
+
+/** offline(x, nn) → per 48 kHz frame of x (480 samples), 1 where it is music: ina's Viterbi over the whole input. */
+export function offline(x, nn) {
+	let a = analyzer(nn), P = [...a(x), ...a.end()], T = P.length, n = Math.ceil(x.length / 480), pass = new Uint8Array(n)
+	if (!T) return pass
+	// V: the best path's log-likelihood ending in each state (speech, music, noise); B: the state it came from. Every
+	// switch costs COST, speech gains PRIOR a patch; a patch of silence scores all states alike. Ties go to the lower
+	// state, as numpy's argmax.
+	let V = [0, 0, 0], B = new Uint8Array(T * 3)
+	for (let t = 0; t < T; t++) {
+		let e = P[t] ? P[t].map((p, s) => lp(p, 1e-10) + (s ? 0 : PRIOR)) : [0, 0, 0], j = V.indexOf(Math.max(...V))
+		V = V.map((v, s) => { let b = V[j] - COST > v ? j : s; B[t * 3 + s] = b; return (b === s ? v : V[j] - COST) + e[s] })
+	}
+	let music = new Uint8Array(T)
+	for (let t = T - 1, s = V.indexOf(Math.max(...V)); t >= 0; s = B[t * 3 + s], t--) music[t] = +(s === 1)
+	// each frame takes the patch centered nearest it
+	for (let g = 0; g < n; g++) pass[g] = music[Math.min(T - 1, Math.max(0, Math.round((g + .5 - 34.716) / STEP)))]
+	return pass
+}
+
+/**
+ * online(nn) → frame(x480): the stream's decision, one 48 kHz frame at a time: true once it holds music, from the
+ * patches complete in the frames so far. The output frame two before (RNNoise's delay) takes it.
+ */
+export function online(nn) {
+	let a = analyzer(nn), l = 0, on = false
+	return x => {
+		for (let p of a(x)) {
+			if (!p) continue
+			l = Math.min(BOUND, Math.max(-BOUND, l + lp(p[1], FLOOR) - lp(p[0] + p[2], FLOOR) - PRIOR))
+			on = l > (on ? 0 : HYST)
+		}
+		return on
+	}
+}
+
+/** ramp(enhance) → next(enhance): the enhanced signal's gain for the next sample, toward 1 (enhance) or 0 (pass) over
+ *  RAMP, starting at 1 or 0. */
+export function ramp(start = true) {
+	let phi = +start
+	return enhance => (phi = Math.min(1, Math.max(0, phi + (enhance ? 1 : -1) / RAMP)), .5 - .5 * Math.cos(Math.PI * phi))
+}
+
+/** blend(g, y, x): the enhanced y at gain g, the input x at 1 − g; at 1 and 0 the one signal itself, bit for bit. */
+export const blend = (g, y, x) => g === 1 ? y : g === 0 ? x : g * y + (1 - g) * x
+
+/** gains(pass, n, centered) → Float32Array(n), the enhanced signal's gain per sample from a pass flag per 480: offline,
+ *  from the first frame's and the ramps centered on each change; streaming, from 1 (nothing decided yet), the ramps
+ *  starting at each change. */
+export function gains(pass, n, centered) {
+	let g = new Float32Array(n), next = ramp(!(centered && pass[0])), lead = centered ? RAMP / 2 : 0
+	for (let i = 0; i < n; i++) g[i] = next(!pass[Math.min(pass.length - 1, Math.floor((i + lead) / 480))])
+	return g
+}
+
+// guard.bin, the network's weights (scripts/guard.py writes them from upstream's export)
+let bundled
+export async function guardNet() {
+	return bundled ??= (async () => {
+		let url = new URL('./guard.bin', import.meta.url)
+		if (url.protocol === 'file:') return net(new Uint8Array((await import('node:fs')).readFileSync(url)))
+		let res = await fetch(url)
+		if (!res.ok) throw new Error(`neural-denoise: can't fetch ${url}: ${res.status}`)
+		return net(new Uint8Array(await res.arrayBuffer()))
+	})()
+}

package/index.d.ts

@@ -1,4 +1,4 @@
-/** Neural speech enhancement: RNNoise (pure JS, weights bundled) and DeepFilterNet3 (ONNX, fetched and cached). */
+/** Neural speech enhancement: RNNoise (pure JS, weights bundled) and DeepFilterNet3 (ONNX, fetched and cached); music passes through untouched. */
 
 export type Audio = Float32Array | Float32Array[] | { channelData: Float32Array[], sampleRate: number }
 
@@ -28,8 +28,12 @@ export interface DenoiseOptions extends LoadOptions {
   sampleRate?: number
   /** 'rnnoise' (default), 'deepfilternet3', or a handle from load() to reuse across calls */
   model?: 'rnnoise' | 'deepfilternet3' | Model
-  /** attenuation limit in dB, the most the noise drops: mixes the input back in at 10^(−limit/20); default 16 for RNNoise, 18 for DeepFilterNet3 (README, API); 0 for none. DeepFilterNet3 also hears its input with the speech at −20 dBFS and, above a band-limited input's edge, a white noise floor 20 dB under it, and keeps sustained voicing it would remove (held sung notes) */
+  /** attenuation limit in dB, how far the noise drops: mixes the input back in at 10^(−limit/20) (DeepFilterNet3: less where the noise stands within `floor` of the voice); default 16 for RNNoise, 18 for DeepFilterNet3 (README, API); 0 for none. DeepFilterNet3 also hears its input with the speech at −20 dBFS and, above a band-limited input's edge, a white noise floor 20 dB under it, and keeps sustained voicing it would remove (held sung notes) */
   limit?: number
+  /** DeepFilterNet3: dB under the voice to which noise the limit would leave louder drops (mixback()), frame by frame; default 40, 0 for none (before 0.5: every noise down by the limit only) */
+  floor?: number
+  /** 'pass' (default): speech and noise are enhanced, music (songs included) passes through untouched, as inaSpeechSegmenter's speech/music/noise CNN segments the input: DeepFilterNet3 with the whole input in view, RNNoise as its stream decides (from what has arrived: music's first second or so is denoised); the gain ramps over 200 ms at each switch. 'enhance': everything, as before 0.4 */
+  music?: 'pass' | 'enhance'
   /** DeepFilterNet3: frames (10 ms) per model run, default 1000 */
   chunk?: number
   /** DeepFilterNet3: frames of context run before each chunk after the first, default 300 */
@@ -41,6 +45,9 @@ export interface DenoiseOptions extends LoadOptions {
 /** Denoise speech; returns the same shape as `audio`, same length, aligned with it. */
 export default function denoise<T extends Audio>(audio: T, opts?: DenoiseOptions): Promise<T>
 
+/** The input x mixed back into a model's output y, y + a·(x − y): the noise drops by `limit` dB (default 18), or to `floor` dB under the voice (default 40) where it stands higher, per 10 ms frame; `limit` 0: y; `floor` 0: a constant 10^(−limit/20). denoise() applies it to DeepFilterNet3. */
+export function mixback(x: Float32Array, y: Float32Array, opts?: { limit?: number, floor?: number, sampleRate?: number }): Float32Array
+
 /** Load a model once to reuse it across denoise() calls. */
 export function load(model?: 'rnnoise' | 'deepfilternet3', opts?: LoadOptions): Promise<Model>
 

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@audio/neural-denoise",
-  "version": "0.3.0",
-  "description": "Neural speech enhancement: RNNoise (Valin 2018) in pure JS, bit-exact to its C, weights bundled; DeepFilterNet3 through ONNX Runtime, matching the Python original",
+  "version": "0.5.0",
+  "description": "Neural speech enhancement: RNNoise (Valin 2018) in pure JS, bit-exact to its C, weights bundled; DeepFilterNet3 through ONNX Runtime, matching the Python original; music passes through untouched (inaSpeechSegmenter's speech/music/noise CNN)",
   "type": "module",
   "sideEffects": false,
   "main": "denoise.js",
@@ -17,6 +17,7 @@
     },
     "./worklet": "./worklet.js",
     "./rnnoise.bin": "./rnnoise.bin",
+    "./guard.bin": "./guard.bin",
     "./package.json": "./package.json"
   },
   "files": [
@@ -28,6 +29,9 @@
     "worklet.js",
     "rnnoise.bin",
     "gemv.wat",
+    "guard.js",
+    "guard.bin",
+    "gemm.wat",
     "index.d.ts",
     "LICENSE",
     "NOTICE",
@@ -61,7 +65,8 @@
   "author": "Dmitry Iv. <dfcreative@gmail.com>",
   "contributors": [
     "Jean-Marc Valin, Xiph.Org Foundation, Mozilla, Amazon and the RNNoise authors (RNNoise)",
-    "Hendrik Schröter (DeepFilterNet)"
+    "Hendrik Schröter (DeepFilterNet)",
+    "David Doukhan, Ina (inaSpeechSegmenter)"
   ],
   "audio": "./audio.js",
   "engines": {