@pipedog/resampler-wasm 0.0.0 → 0.2.0

package/LICENSE

@@ -0,0 +1,25 @@
+BSD 2-Clause License
+
+Copyright (c) 2024–2026, Daily
+Copyright (c) 2026, pipedog contributors
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

package/NOTICE

@@ -0,0 +1,9 @@
+pipedog is an unofficial TypeScript port of Pipecat (https://github.com/pipecat-ai/pipecat),
+a framework for voice and multimodal conversational AI created by Daily.
+
+It is not affiliated with or endorsed by Daily. Pipecat is distributed under
+the BSD 2-Clause License, Copyright (c) 2024–2026, Daily; that copyright notice
+and license are retained in LICENSE, and every ported source file carries a
+header naming the upstream file it was derived from.
+
+The upstream commit each file was ported from is recorded in porting/manifest.json.

package/README.md

@@ -1,5 +1,186 @@
 # @pipedog/resampler-wasm
 
-Placeholder version. Install a released version instead: `npm install @pipedog/resampler-wasm@latest`.
+High-quality PCM sample-rate conversion for pipedog, compiled to WebAssembly.
+It is the engine behind core's `SOXRAudioResampler` and `SOXRStreamAudioResampler`.
 
-Part of [pipedog](https://github.com/polvallverdu/pipedog).
+- [rubato](https://github.com/HEnquist/rubato) 5.0.1 (MIT OR Apache-2.0) windowed-sinc resampler, written in Rust,
+  built for `wasm32-unknown-unknown`. We use it instead of libsoxr, which is LGPL and needs emscripten.
+- No wasm-bindgen and no generated JS glue. The module has **zero imports** and a small C ABI
+  (`rust/src/lib.rs`), and `src/engine.ts` wraps the `WebAssembly.Instance` exports by hand.
+- `wasm/resampler.wasm` is a committed build artifact of **38,893 bytes** (18.2 KB with `gzip -9`).
+  It is built with cargo `opt-level = "z"`, LTO, `panic = "abort"`, then `wasm-opt -Oz`.
+- int16 little-endian PCM in and out, plus float32. Any channel count from 1 to 8, interleaved.
+- One-shot and streaming. Streams take chunks of any size, keep the audio continuous across
+  chunks, and support `flush()` and `reset()`.
+- Verified on Bun, Node 24, Deno 2 and workerd (Cloudflare Workers).
+
+## Usage
+
+```ts
+import { initResampler } from "@pipedog/resampler-wasm";
+
+const engine = await initResampler(); // once; everything after this is synchronous
+
+// One-shot (a whole clip, tail included). Returns the input unchanged if the rates are equal.
+const out: Uint8Array = engine.resample(pcm16, 8000, 16000, { quality: "VHQ" });
+
+// Streaming
+const stream = engine.createStream(48000, 16000, { quality: "HQ", channels: 1 });
+const a = stream.process(chunk1);   // any chunk size
+const b = stream.process(chunk2);
+const tail = stream.flush();        // emit the filter tail, then start a new stream
+stream.reset();                     // or: drop the held audio
+stream.destroy();                   // free the WASM-side state
+```
+
+There are also `resampleSamples(Int16Array)`, `resampleFloat32(Float32Array)`, and
+`processSamples`/`processFloat32`/`flushSamples`/`flushFloat32` on streams.
+
+## Loading on each runtime
+
+`package.json` `"exports"["."]` uses export conditions:
+
+| condition | entry | how the module gets loaded |
+|---|---|---|
+| `workerd` | `src/workerd.ts` | `import wasm from "../wasm/resampler.wasm"` gives a compiled `WebAssembly.Module` (wrangler's default `.wasm` rule), which is instantiated synchronously. workerd refuses to compile WASM from bytes ("Wasm code generation disallowed by embedder", confirmed by the smoke test). `getResampler()` works without calling `initResampler()` first. |
+| `default` | `src/index.ts` | `fetch(new URL("../wasm/resampler.wasm", import.meta.url))`. Bun and Deno can fetch `file:` URLs. Node 24 cannot ("not implemented... yet"), so the entry falls back to a dynamic `import("node:fs/promises")`. The specifier is held in a variable so bundlers and type checkers ignore it. If the file cannot be read at all (a bundle that did not copy the `.wasm` asset), the same bytes are taken from `src/wasm_bytes.ts` (base64, generated by `build:wasm`). |
+
+`initResampler(source?)` accepts a `WebAssembly.Module`, a `BufferSource`, a `URL` or a `Response`
+(the last two only on the default entry). With no argument it loads the bundled file. It is
+idempotent, and concurrent callers share a single load. `getResampler()` returns the shared
+engine. On the default entry it throws if `initResampler()` has not been awaited yet.
+`./engine` exports `engineFromModule`/`engineFromSource` for anyone who wants to manage
+instances themselves.
+
+## Quality presets
+
+The preset names and targets come from libsoxr's quality table (`soxr_quality_spec()` in
+libsoxr 0.1.3). In both libsoxr and this package, the stopband starts at the Nyquist frequency
+of the lower of the two rates (N below), whichever way the conversion goes:
+
+| preset | libsoxr: passband end, rejection | here: passband (±0.01 dB) | ±0.1 dB | −3 dB | stopband from 1.0 N | images (upsampling) |
+|---|---|---|---|---|---|---|
+| `QQ`  | cubic interpolation, no filter | 0.665 N | 0.695 N | 0.785 N | ≥ 75 dB  | ≤ −72 dB  |
+| `LQ`  | 0.676 N, 96 dB (16-bit)        | 0.74 N  | 0.765 N | 0.82 N  | ≥ 112 dB | ≤ −117 dB |
+| `MQ`  | 0.915 N, 96 dB (16-bit)        | 0.91 N  | 0.92 N  | 0.94 N  | ≥ 111 dB | ≤ −121 dB |
+| `HQ`  | 0.913 N, 120 dB (20-bit)       | 0.905 N | 0.915 N | 0.935 N | ≥ 138 dB | ≤ −149 dB |
+| `VHQ` | 0.911 N, 169 dB (28-bit)       | 0.91 N  | 0.915 N | 0.935 N | ≥ 157 dB (measurement floor) | ≤ −149 dB |
+
+How these were measured: float tones in, float out. Passband edges and stopband come from
+48 kHz → 8 kHz and 48 kHz → 16 kHz, which give the same values. The stopband figure is the
+worst tone between 1.0 N and the input Nyquist frequency. The image figure is the worst residual
+after the input tone is removed, for 8k → 48k, 16k → 48k and 16k → 24k.
+
+Which libsoxr numbers are matched:
+
+- **Passband edges.** MQ, HQ and VHQ are flat to libsoxr's ~0.91 N, so they differ mainly in
+  stopband attenuation, as in libsoxr. LQ is at least as wide as libsoxr's 0.676 N.
+- **Rejection.** It meets or beats libsoxr's 96 dB (LQ, MQ) and 120 dB (HQ). VHQ measures
+  ≥ 157 dB, which is the floor of a float32-input measurement. libsoxr's 169 dB cannot be
+  observed with float32 or int16 I/O.
+- **QQ.** libsoxr's QQ is plain cubic interpolation with no anti-aliasing filter. Here QQ is a
+  short filter that still rejects aliases by ≥ 75 dB.
+
+Parameters (`rust/src/lib.rs`, `preset()`). Lengths and table densities are counted at the lower
+rate:
+
+| preset | length (lower-rate samples) | window | table points / sample | interpolation | cutoff (−6 dB) |
+|---|---|---|---|---|---|
+| `QQ`  | 32  | Blackman        | 32  | Linear | 0.82 N  |
+| `LQ`  | 64  | BlackmanHarris  | 64  | Cubic  | 0.84 N  |
+| `MQ`  | 160 | BlackmanHarris  | 128 | Cubic  | 0.95 N  |
+| `HQ`  | 240 | BlackmanHarris2 | 128 | Cubic  | 0.945 N |
+| `VHQ` | 272 | BlackmanHarris2 | 256 | Cubic  | 0.945 N |
+
+- rubato measures `sinc_len` in input samples and only scales the cutoff when downsampling. So
+  the stream stretches the filter by the downsampling factor: VHQ 48k → 16k uses
+  `sinc_len = 816`. It also divides the table density by the same factor, because the
+  stretched sinc varies more slowly. Everything runs in f64; with f32 the arithmetic noise
+  floor sits near −125 dB, above VHQ's stopband.
+- Delay is half the filter length, 136 lower-rate samples (8.5 ms) for VHQ. It is
+  compensated: the output is time-aligned with the input to within about 0.01 sample (see
+  "Alignment"). In streaming mode it still shows up as latency, because the last half-filter
+  of input only comes out on the next chunk or on `flush()`.
+- Table density also sets the alignment: rubato's sinc table is offset by one table step, so
+  the output leads the input by `1 / table points` lower-rate samples (0.004 for VHQ, 0.008 for
+  HQ/MQ). That is why the densities are higher than the stopband alone would need.
+- Memory: each stream holds a sinc table of about `length × table points` f64 values: 544 KB
+  for VHQ, 240 KB for HQ, 160 KB for MQ, 32 KB for LQ and 8 KB for QQ. WASM memory grows and
+  never shrinks.
+- Speed (Bun, Apple Silicon, mono, 10 s of audio; runs vary by about ±10 ms). 16k → 48k takes
+  about 25 / 50 / 70 / 115 / 115 ms (QQ..VHQ), and 44.1k → 16k takes about 15 / 35 / 70 / 105 /
+  115 ms. A 20 ms chunk at VHQ costs about 0.23 ms (16k → 48k) or 0.21 ms (48k → 16k). A `simd128` build was measured and gave no
+  speed-up, so the artifact is plain scalar wasm.
+
+## Streaming semantics
+
+- Each `process()` call is split into pieces of at most 1024 frames.
+  `Resizable::set_chunk_size` is set to each piece's length, which keeps the filter history.
+  Nothing is held back beyond the filter delay. A sample count that is not a multiple of the
+  channel count leaves the remainder buffered for the next call.
+- Output is aligned: the start-up delay is trimmed from the front of every stream. `flush()`
+  pads with silence until the stream has emitted exactly `round(totalInputFrames × out / in)`
+  frames, then resets. This matches upstream's
+  `test_soxr_stream_audio_resampler_flush_returns_the_held_audio`: 640 samples at 16 kHz give
+  fewer than 320 samples, and adding the flush gives exactly 320.
+- `reset()` discards the filter history, any pending samples and the counters.
+- **Alignment.** rubato's first output frame sits at input position `1/ratio − sinc_len/2`, so
+  the raw delay is `sinc_len·ratio/2 − 1` output frames. For many ratios that is a fraction
+  (48 kHz → 16 kHz, for example). To make it a whole number, the stream is primed with
+  `k` zero frames, where `k < 2q` and `q` is the denominator of the reduced ratio `p/q`. The
+  whole-number delay is then trimmed. All of this is computed in `alignment()` in
+  `rust/src/lib.rs`.
+- Output samples differ from libsoxr's because the filters are different. Lengths follow the
+  rule above.
+
+## How core uses it
+
+`@pipedog/core` depends on this package and wires it in itself
+(`packages/core/src/audio/resamplers/engine.ts`): `SOXRAudioResampler` and
+`SOXRStreamAudioResampler` load it lazily on first use (`initResampler()`, then
+`createSoxrCompatibleEngine()`), fall back to a pure-TypeScript windowed-sinc engine
+with one warning if it cannot load, and accept any engine of the same shape through
+`setResamplerEngine()`. `createSoxrCompatibleEngine(engine)` returns that shape:
+
+```ts
+type SoxrQuality = "QQ" | "LQ" | "MQ" | "HQ" | "VHQ";
+interface SoxrCompatibleEngine {
+  // stands in for soxr.resample(audio, in_rate, out_rate, quality=...)
+  resample<T extends Int16Array | Float32Array>(audio: T, inRate: number, outRate: number, quality: SoxrQuality): T;
+  // stands in for soxr.ResampleStream(in_rate=, out_rate=, num_channels=, quality=, dtype="int16")
+  createStream(o: { inRate: number; outRate: number; numChannels: number; quality: SoxrQuality }): {
+    resampleChunk<T extends Int16Array | Float32Array>(chunk: T, last?: boolean): T; // last=true appends the tail and resets
+    clear(): void;   // soxr ResampleStream.clear()
+    destroy(): void; // free WASM state (optional; a FinalizationRegistry is the backstop)
+  };
+}
+```
+
+Equal rates never reach the engine, because core returns early just as upstream does. The
+engine's `resample` also returns its input unchanged for equal rates.
+
+## C ABI (for reference)
+
+| export | |
+|---|---|
+| `rs_alloc(len) -> ptr`, `rs_free(ptr, len)` | raw linear-memory blocks |
+| `rs_stream_new(in_rate, out_rate, channels, quality) -> handle` | quality: 0=QQ … 4=VHQ; 0 on bad params |
+| `rs_stream_input(h, n) -> *i16` / `rs_stream_input_f32(h, n) -> *f32` | staging buffer for `n` samples |
+| `rs_stream_process(h, n) -> count` / `rs_stream_process_f32` | output at `rs_stream_output(h)` / `rs_stream_output_f32(h)`; −1 on error |
+| `rs_stream_flush(h) -> count` / `rs_stream_flush_f32` | drain tail and reset |
+| `rs_stream_reset(h)`, `rs_stream_free(h)` | |
+| `rs_stream_delay(h)`, `rs_stream_cutoff(h)`, `rs_quality_sinc_len(q)`, `rs_quality_oversampling(q)` | introspection |
+
+The JS side copies its input into the staging buffer and copies the output out. It recreates
+typed-array views after every call, because a `memory.grow` detaches the old `ArrayBuffer`.
+
+## Building and testing
+
+```sh
+rustup target add wasm32-unknown-unknown
+bun run build:wasm      # cargo + wasm-opt -Oz (wasm-opt from PATH, else `npx -p binaryen`); --no-opt to skip
+bun test                # bun:test suite (test/resampler.test.ts)
+bun run smoke:node      # node test/smoke/node.ts (TypeScript sources run directly; Node >= 22.6)
+bun run smoke:deno      # deno run -A test/smoke/deno.ts
+bun run smoke:workerd   # bundles test/smoke/workerd/worker.ts with the "workerd" condition, serves it with `npx workerd`
+```