defuss-brotli 0.1.3 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,34 @@
1
+ # Changelog
2
+
3
+ All notable changes to `defuss-brotli`. Versions follow [Semantic Versioning](https://semver.org/); before 1.0.0, a minor version may change behavior.
4
+
5
+ ## 0.3.0 - 2026-11-09
6
+
7
+ ### Changed
8
+
9
+ - **The default compression quality is now 11 instead of 6**, the default of Node.js zlib (`BROTLI_DEFAULT_QUALITY`) and of the `brotli` CLI. `compress()` and `compressText()` calls that pass no `quality` now return the smallest output and take longer: about 3.8 s instead of 0.15 s for a 5.1 MB SVG (Node.js 26). Pass `{ quality: 6 }` to keep the previous output.
10
+ - The compressor entry points embed the worker script of `compressParallel`: `compressor-slim.js` grows from 4.8 KB to 15.2 KB and `compressor.wasm` by 11.5 KB (raw). The decompressor entry points grow by 29 B.
11
+
12
+ ### Added
13
+
14
+ - `compressParallel(input, options)` compresses up to 8 parts of the input at once on workers, each with its own instance of the compressor, and joins them into one standard Brotli stream. At quality 11 it is 2.3 to 4.3 times faster than `compress` from 256 KiB, with output 0.01 to 0.10 % larger (Node.js 26). The output depends only on the input, `quality` and `lgwin`, not on `threads` or the machine. It also exports `BrotliParallelOptions`, `PARALLEL_PARTS` and `MIN_PART_SIZE`.
15
+ - The `defuss-brotli` command (`bunx defuss-brotli compress FILE`, `bunx defuss-brotli decompress FILE.br`) takes the options, defaults, output names and refusals of the `brotli` 1.2.0 CLI. It compresses with `compressParallel`; `--threads=NUM` sets the number of workers. From 256 KiB it compresses 1.2 to 5.3 times faster than `brotli` 1.2.0, depending on the input and runtime. `--large_window`, `--comment`, `--dictionary` and `--concatenated` are not supported.
16
+ - `CHANGELOG.md`, shipped in the package.
17
+
18
+ ## 0.2.0 - 2026-11-09
19
+
20
+ ### Added
21
+
22
+ - `/slim` entry points, `defuss-brotli/compressor/slim` and `defuss-brotli/decompressor/slim`, with the same API as the inline ones. They load the binary from a separate `.wasm` file (`defuss-brotli/compressor.wasm`, `defuss-brotli/decompressor.wasm`): by default from next to the module, or from a URL, a `Response`, bytes or a compiled `WebAssembly.Module` passed to `init()`.
23
+ - `THIRD_PARTY_NOTICES.txt` with the license texts of the code compiled into the `.wasm` files.
24
+
25
+ ### Changed
26
+
27
+ - Both modules stream input and output through fixed chunks in WASM memory, so linear memory no longer grows with the payload: it holds 19.6 MiB after decoding 100 MiB, and 69 MiB after compressing it at quality 6.
28
+ - The compressor feeds the encoder one window per call instead of 4 KiB, so qualities 0 and 1 find repeats across chunks (a 28 KiB page at quality 1: 1,189 B instead of 3,853 B); the output equals one encoder call over the whole input at every quality.
29
+ - The decoder is built without the Rust standard library, which brings `defuss-brotli/decompressor/slim` to about 51 KB with Brotli quality 11 (85 KB from jsDelivr).
30
+ - `bun run build`, and with it `npm publish`, finds the pinned build tools without an activated mise.
31
+
32
+ ## 0.1.1 to 0.1.3
33
+
34
+ These releases predate this repository's history.
package/README.md CHANGED
@@ -2,12 +2,15 @@
2
2
 
3
3
  A split pure-Rust Brotli package for Node.js + browsers.
4
4
 
5
- - **Small browser decoder bundle** - import only the decompressor, skip the encoder WASM entirely
5
+ - **Small browser decoder** - `defuss-brotli/decompressor/slim` loads only the decoder: about 51 KB precompressed with Brotli quality 11 and 85 KB from jsDelivr (quality 4), most of it the Brotli dictionary every decoder carries
6
6
  - **Separate heavier encoder bundle** - used server-side or at build time
7
7
  - **Single NPM package** with split exports
8
+ - **Inlined or separate `.wasm`** - one self-contained file by default, or a `/slim` entry that loads the binary on its own
8
9
  - **Pure-Rust WebAssembly** - no native addons
9
10
  - **Typed TypeScript API** with full JSDoc
10
11
  - **Vitest coverage** in Node and browser
12
+ - **Parallel compression** - `compressParallel()` compresses parts of the input on workers: 2.3 to 4.3 times faster at quality 11 from 256 KiB, with output at most 0.1 % larger
13
+ - **`defuss-brotli` command** with the options of the `brotli` CLI, `bunx defuss-brotli compress FILE`; from 256 KiB it compresses 1.2 to 5.3 times faster than `brotli` 1.2.0
11
14
 
12
15
  ## Install
13
16
 
@@ -35,9 +38,101 @@ import { init, decompressText } from "defuss-brotli/decompressor";
35
38
  await init();
36
39
 
37
40
  const text = decompressText(compressed);
38
- // → "Hello, Brotli!"
41
+ // returns "Hello, Brotli!"
39
42
  ```
40
43
 
44
+ ### Command line
45
+
46
+ The package ships a `defuss-brotli` command with the options of the `brotli` CLI, so it can replace `brotli` in build scripts and on machines where only Node.js or Bun is installed:
47
+
48
+ ```bash
49
+ bunx defuss-brotli compress dist/app.js # writes dist/app.js.br, keeps dist/app.js
50
+ bunx defuss-brotli decompress dist/app.js.br -o app.js
51
+ bunx defuss-brotli -q 5 -v *.html *.css # brotli's spelling: compress by default
52
+ bunx defuss-brotli -d -c page.html.br | head # -d decompresses, -c writes to stdout
53
+ cat data.json | bunx defuss-brotli > data.json.br
54
+ bunx defuss-brotli -t data.json.br # test integrity, write nothing
55
+ ```
56
+
57
+ `compress` and `decompress` as the first argument select the mode; without one the command compresses unless `-d` or `-t` is given, as `brotli` does. It compresses with [`compressParallel`](#compressparallel), so inputs above 32 KiB are split into parts that compress on several workers. `npx defuss-brotli` works the same way. The command runs on Node.js when it is on the `PATH` and on Bun otherwise; `bunx --bun defuss-brotli` always uses Bun, which starts faster.
58
+
59
+ | Option | Meaning |
60
+ |---|---|
61
+ | `-0` to `-9`, `-q NUM`, `--quality=NUM`, `-Z`, `--best` | quality 0 to 11, default 11 (`-Z`) |
62
+ | `-w NUM`, `--lgwin=NUM` | window 10 to 24; `0` (default) picks the smallest window that holds the input |
63
+ | `-c`, `--stdout` | write to standard output |
64
+ | `-o FILE`, `--output=FILE` | output file (only with one input) |
65
+ | `-d`, `--decompress` / `-t`, `--test` | decompress / decompress without writing, to check integrity |
66
+ | `-f`, `--force` | overwrite existing output, allow terminal input and output |
67
+ | `-k`, `--keep` / `-j`, `--rm` | keep (default) / delete each source after writing its output |
68
+ | `-s`, `--squash` | when compressing, drop an output that is larger than its source |
69
+ | `-n`, `--no-copy-stat` | do not copy the source's mode and timestamps to the output |
70
+ | `-S SUF`, `--suffix=SUF` | suffix to add or strip, default `.br` |
71
+ | `--threads=NUM` | compress on up to `NUM` workers, default the number of CPUs, at most 8; the output is the same for every `NUM` (not a `brotli` option) |
72
+ | `-v`, `--verbose` / `-h`, `--help` / `-V`, `--version` | per-file sizes and time / usage / version |
73
+
74
+ With no file, or with `-`, it reads standard input and writes standard output. Short flags combine (`-9kf`) and `--` ends the options. Each input is written to `FILE.br` or, when decompressing, to `FILE` without `.br`. The command refuses to overwrite a file without `-f`, to write compressed bytes to a terminal, and to decompress a name that lacks the suffix. It stops at the first file that fails and exits with status 1.
75
+
76
+ VERIFIED (2026-11-09, against brotli 1.2.0 on macOS):
77
+
78
+ - The defaults, output file names and refusals match `brotli`'s. Messages keep its wording, with the error text of Node.js or Bun (`file already exists` where `brotli` prints `File exists`).
79
+ - The default output for `input/page.html` and `input/data.json` is byte-identical to `brotli`'s.
80
+ - On larger inputs, which it compresses in parts, the default output was 0.00 to 0.16 % larger than `brotli`'s: 64 KiB and 1 MiB of SVG, a 64 KiB PNG head, a 2.1 MB PNG and a 5.1 MB SVG.
81
+
82
+ Some `brotli` behavior is not supported or differs:
83
+
84
+ - `--large_window`, `-C`/`--comment`, `-D`/`--dictionary` and `-K`/`--concatenated` fail with a reason, because the WASM encoder and decoder do not implement them.
85
+ - Usage errors print a one-line hint instead of the whole help text.
86
+ - `-t -j` is refused; `brotli` 1.2.0 deletes the file it has just tested.
87
+ - Each input is read into memory whole and each output is written at once, so files must fit in memory; decompressed output is limited to 2^32 − 1 bytes, the wasm32 limit.
88
+
89
+ At the default quality 11, the command compresses 64 KiB as fast as `brotli` 1.2.0 on Bun, and from 256 KiB 2.3 to 5.3 times faster on Bun and 1.2 to 5.1 times faster on Node.js. Smaller files take longer, because it first starts a JavaScript runtime and compiles the WASM. Median of 5 interleaved runs on Apple Silicon (18 cores), startup included, Node.js 26 and Bun 1.4:
90
+
91
+ | Input | `defuss-brotli` on Node.js | on Bun (`bunx --bun`) | `brotli` 1.2.0 |
92
+ |---|---|---|---|
93
+ | 29 KB HTML (`input/page.html`), one part | 164 ms | 89 ms | 55 ms |
94
+ | 64 KiB of SVG, 2 parts | 174 ms | 68 ms | 69 ms |
95
+ | 256 KiB of SVG, 8 parts | 178 ms | 89 ms | 219 ms |
96
+ | 1 MiB of SVG, 8 parts | 320 ms | 301 ms | 879 ms |
97
+ | 2.1 MB PNG, 8 parts | 740 ms | 714 ms | 3.79 s |
98
+ | 5.1 MB SVG, 8 parts | 1.89 s | 1.71 s | 4.00 s |
99
+
100
+ The SVG inputs are the start of the 5.1 MB file. Decompressing the 5.1 MB SVG took 93 ms on Node.js and 55 ms on Bun, against 32 ms for `brotli` (median of 3). Startup (`-V`) took 59 ms on Node.js and 24 ms on Bun.
101
+
102
+ ### Inlined or separate `.wasm`
103
+
104
+ Each module comes in two builds with the same API. The default entry points embed the WASM binary as base64, so one JavaScript file is all a runtime or bundler needs. The `/slim` entry points leave the binary in its own `.wasm` file, which HTTP caches independently of your JavaScript, which compiles while it downloads when served as `application/wasm`, and which transfers 21 to 45 % fewer bytes depending on module and transfer encoding (see [Download sizes and budgets](#download-sizes-and-budgets)).
105
+
106
+ Without an argument, a slim `init()` loads the `.wasm` file from next to its module, via `new URL("./decompressor.wasm", import.meta.url)`. Browsers fetch it from there, and Node.js 20.16+ / 22.3+ and Bun read it from disk:
107
+
108
+ ```ts
109
+ import { init, decompressText } from "defuss-brotli/decompressor/slim";
110
+
111
+ await init();
112
+ ```
113
+
114
+ When your bundler or server does not place the file next to the bundle, copy `defuss-brotli/decompressor.wasm` (or `compressor.wasm`) to where you serve static assets and pass its location, or pass the response, the bytes or a compiled module:
115
+
116
+ ```ts
117
+ await init("/assets/decompressor.wasm"); // URL, fetched (streamed when served as application/wasm)
118
+ await init(fetch("https://cdn.example.com/d.wasm")); // Response or Promise<Response>
119
+ await init(await readFile(wasmPath)); // ArrayBuffer or typed array
120
+ await init(compiledModule); // WebAssembly.Module
121
+ ```
122
+
123
+ `init()` loads once; later calls return the first Promise and ignore their argument. After a failed load (for example a 404), the next call tries again.
124
+
125
+ #### Bundlers
126
+
127
+ `make e2e` builds a clean app per bundler from the packed tarball and runs it in Chromium:
128
+
129
+ | Bundler | `init()` without an argument | Explicit asset import |
130
+ |---|---|---|
131
+ | Vite 8.3.6, `vite build` and `vite` dev | works: Vite resolves `new URL(..., import.meta.url)` and emits the `.wasm` | `import url from "defuss-brotli/decompressor.wasm?url"`, then `init(url)` |
132
+ | `bun build` (Bun 1.4) | copy `compressor.wasm` / `decompressor.wasm` next to the bundle yourself: Bun leaves `new URL(...)` unchanged | `import url from "defuss-brotli/decompressor.wasm" with { type: "file" }`, then `init(url)` |
133
+
134
+ With an explicit import, the bundler copies the file and gives it a hashed name, so it can be cached long-term. Node.js and Bun as runtimes need neither: `init()` reads the file from `node_modules`.
135
+
41
136
  ## API
42
137
 
43
138
  ### `compress` vs `compressText`
@@ -51,6 +146,30 @@ const text = decompressText(compressed);
51
146
  raw bytes. `TextEncoder` converts to canonical UTF-8 - the standard encoding for
52
147
  web text. `compressText` handles this so you don't have to.
53
148
 
149
+ ### `compressParallel`
150
+
151
+ ```ts
152
+ import { init, compressParallel } from "defuss-brotli/compressor";
153
+
154
+ await init();
155
+ const compressed = await compressParallel(bytes, { quality: 11, threads: 4 });
156
+ ```
157
+
158
+ `compressParallel` splits the input into at most 8 parts of at least 32 KiB, compresses each part on a worker with its own instance of the compressor, and joins the parts into one standard Brotli stream on the calling thread. Each part gets the window of input before it as a dictionary, so matches still reach back across parts. This is the multithreaded mode of the `brotli` crate, run on workers instead of threads.
159
+
160
+ - **Output:** it depends only on the input, `quality` and `lgwin`, never on `threads` or the number of CPUs; `threads: 1` compresses the parts one after another on the calling thread and returns the same bytes. Below quality 2, or when the input fits in one part, it returns exactly what `compress` returns. Otherwise the output differs from `compress`'s and is slightly larger.
161
+ - **Workers:** Web Workers from a Blob URL in browsers and Bun, and `worker_threads` in Node.js. Pages with a Content Security Policy must allow `worker-src blob:`. No extra file is needed, because the worker script is embedded in the module. The workers are terminated before the Promise settles. A runtime that cannot start such a worker compresses the parts on the calling thread.
162
+ - **`threads`:** how many workers run at once, default the number of logical CPUs, at most 8. Each worker allocates its own encoder state, as a `compress` call does.
163
+
164
+ VERIFIED on Node.js 26 (Apple Silicon, quality 11, window 22, median of 3 in-process runs, first call):
165
+
166
+ | Input | `compress` | `compressParallel` | Size |
167
+ |---|---|---|---|
168
+ | 256 KiB of SVG | 187 ms | 44 ms | +0.01 % |
169
+ | 1 MiB of SVG | 617 ms | 189 ms | +0.05 % |
170
+ | 2.1 MB PNG | 1.79 s | 0.60 s | +0.10 % |
171
+ | 5.1 MB SVG | 3.82 s | 1.67 s | +0.09 % |
172
+
54
173
  ### `decompress` vs `decompressText`
55
174
 
56
175
  | Function | Output | Decoding | When to use |
@@ -65,7 +184,7 @@ valid UTF-8, replacement characters (U+FFFD) are inserted rather than throwing.
65
184
 
66
185
  ```ts
67
186
  interface BrotliCompressOptions {
68
- quality?: number; // 0-11, default 6
187
+ quality?: number; // 0-11, default 11
69
188
  lgwin?: number; // 10-24, default 22
70
189
  }
71
190
  ```
@@ -92,109 +211,105 @@ Hard cap on decompressed output to prevent **decompression bombs**. Raise explic
92
211
 
93
212
  ### Guarantees
94
213
 
95
- - `compressText(s)` → `decompressText(...)` is a **perfect round-trip** for any JS string.
96
- - `compress(b)` → `decompress(...)` is a **perfect round-trip** for any `Uint8Array`.
214
+ - `decompressText(compressText(s))` returns `s` exactly for any JS string.
215
+ - `decompress(compress(b))` returns the same bytes as `b` for any `Uint8Array`.
97
216
  - `init()` is **idempotent** - calling it multiple times returns the same Promise and does not reload WASM.
98
- - Importing only `defuss-brotli/decompressor` **never loads** the encoder WASM binary.
217
+ - Importing only `defuss-brotli/decompressor` (or `/decompressor/slim`) **never loads** the encoder WASM binary.
218
+ - Inline and `/slim` builds of a module run the same code and produce identical output.
99
219
 
100
220
  ### Limits
101
221
 
102
- - `quality` must be an integer in **0-11** (Brotli spec hard limit).
103
- - `lgwin` must be an integer in **10-24** (Brotli spec hard limit).
104
- - `maxOutputSize` defaults to **64 MiB** - the WASM module rejects output exceeding this.
105
- - Browser WASM memory is capped at **~2 GB** on 64-bit platforms (practical upper bound for very large payloads).
222
+ - `quality` must be an integer in **0-11** (the range of the reference Brotli encoder).
223
+ - `lgwin` must be an integer in **10-24** (the window range of the Brotli format, RFC 7932).
224
+ - `maxOutputSize` defaults to **64 MiB** - the WASM module rejects output exceeding this. Values above 2^32 − 1 act as 2^32 − 1.
225
+ - `threads` must be a positive integer. `compressParallel` copies each part and its dictionary (up to one window) to its worker, so it holds up to 8 windows more than the input in JavaScript memory while it runs.
226
+ - Calls stream the input and output through chunks of at most 1 MiB (one window of input when compressing), so WASM memory holds the codec state and two chunks however large the payload; the returned `Uint8Array` lives in JavaScript memory. Truncated input, corrupt input and bytes after the end of the Brotli stream throw.
106
227
 
107
228
  ## Why the Split Exports?
108
229
 
109
- The encoder and decoder are built from **two separate Rust Wasm crates**:
230
+ The encoder and decoder are built from **two separate Rust Wasm crates**: `brotli` behind `defuss-brotli/compressor` (server-side, build pipelines) and `brotli-decompressor` behind `defuss-brotli/decompressor` (browser clients). The encoder needs large static tables the decoder does not, so browser apps that only decompress pre-compressed data import just the decompressor and download about a sixth of the bytes. [ARCH.md](ARCH.md) explains where the sizes come from and why the build trades 1 % of size for speed.
110
231
 
111
- | Export | Rust crate | WASM size | Bundle (JS + WASM) | Typical use |
112
- |---|---|---|---|---|
113
- | `defuss-brotli/compressor` | `brotli` | 984 KB | 1.3 MB | Server-side, build pipelines |
114
- | `defuss-brotli/decompressor` | `brotli-decompressor` | 208 KB | 276 KB | Browser clients |
232
+ ## Download sizes and budgets
115
233
 
116
- Browser apps that only decompress pre-compressed data import just the decompressor, keeping the bundle small.
234
+ What a browser downloads for one import, with each file transfer-compressed on its own, as a CDN or web server does per response:
117
235
 
118
- ## Benchmarks
236
+ | Import | Files | raw | gzip -9 | brotli q4 (jsDelivr) | brotli q11 | zstd -19 |
237
+ |---|---|---:|---:|---:|---:|---:|
238
+ | `defuss-brotli/decompressor/slim` | `decompressor-slim.js` + `decompressor.wasm` | 190,289 B | 85,633 B | 85,090 B | 50,828 B | 78,609 B |
239
+ | `defuss-brotli/decompressor` | `decompressor.js` | 252,213 B | 110,006 B | 111,536 B | 92,996 B | 101,435 B |
240
+ | `defuss-brotli/compressor/slim` | `compressor-slim.js` + `compressor.wasm` | 960,020 B | 453,582 B | 510,561 B | 396,565 B | 405,487 B |
241
+ | `defuss-brotli/compressor` | `compressor.js` | 1,275,038 B | 608,203 B | 663,816 B | 499,177 B | 524,736 B |
119
242
 
120
- Measured with `vitest bench` on the payloads shown. Numbers are **ops/sec** (higher is better).
243
+ - **gzip -9** and **brotli q4** are what [jsDelivr](https://www.jsdelivr.com/) sends: measured on 2026-11-09, its brotli responses match quality 4 byte for byte and its gzip responses lie within 0.1 % of levels 6 to 9. jsDelivr serves `.wasm` files as `application/wasm` with `access-control-allow-origin: *`, and serves no zstd.
244
+ - **brotli q11** and **zstd -19** are what precompressed static hosting reaches (for example `.br` files next to the assets, or a CDN that compresses at its highest level). Quality 11 is unstable on `compressor.wasm`: passes of `wasm-opt` that change its raw size and its quality-4 size by 0.2 % move it between 340,693 B and 392,638 B, so its budget allows the upper value.
245
+ - `make metrics` prints these numbers for `dist/`. `make e2e` measures the packed tarball and fails when an import exceeds its limit in [`size-budgets.json`](size-budgets.json) (current size plus 2 %); raise a limit only for a measured, intended change.
121
246
 
122
- Run benchmarks yourself:
247
+ ### From a CDN without a bundler
123
248
 
124
- ```bash
125
- bun run bench # Node.js
126
- bun run bench:browser # Chromium via Playwright
127
- ```
128
-
129
- ### Node.js (Bun)
130
-
131
- **Text Compression - `compressText`** (ops/sec, higher is better):
249
+ An [import map](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script/type/importmap) maps the package's exports to its files on jsDelivr; pin a version (`defuss-brotli@x.y.z`) in production. The slim `init()` then fetches `decompressor.wasm` from the same CDN directory:
132
250
 
133
- | Payload | Quality 1 | Quality 6 | Quality 11 |
134
- |---|---|---|---|
135
- | Short JSON (83 B) | - | 37,239 | - |
136
- | Markdown (2 KB) | 19,819 | 10,141 | 404 |
137
- | HTML (8 KB) | 5,191 | 2,704 | 18 |
251
+ ```html
252
+ <script type="importmap">
253
+ { "imports": { "defuss-brotli/decompressor/slim": "https://cdn.jsdelivr.net/npm/defuss-brotli/dist/decompressor-slim.js" } }
254
+ </script>
255
+ <script type="module">
256
+ import { init, decompressText } from "defuss-brotli/decompressor/slim";
138
257
 
139
- **Binary Compression - `compress`** (ops/sec):
140
-
141
- | Payload | Quality 1 | Quality 6 | Quality 11 |
142
- |---|---|---|---|
143
- | Pseudorandom (512 B) | - | 29,430 | - |
144
- | SVG (4.9 MB) | 8.0 | 4.0 | 0.15 |
145
- | PNG (2.1 MB) | 39.6 | 82.8 | 0.21 |
146
-
147
- PNG compresses faster than SVG at q6 because already-compressed data triggers
148
- Brotli's fast-path fallback earlier, producing less work per byte.
149
-
150
- **Text Decompression - `decompressText`** (ops/sec):
151
-
152
- | Payload | ops/sec |
153
- |---|---|
154
- | Short JSON (83 B) | 141,241 |
155
- | Markdown (2 KB) | 47,757 |
156
- | HTML (8 KB) | 37,288 |
157
-
158
- **Binary Decompression - `decompress`** (ops/sec):
159
-
160
- | Payload | ops/sec |
161
- |---|---|
162
- | Pseudorandom (512 B) | 115,493 |
163
- | SVG (4.9 MB) | 30.7 |
164
- | PNG (2.1 MB) | 699 |
258
+ await init();
259
+ </script>
260
+ ```
165
261
 
166
- ### Browser (Chromium)
262
+ `make e2e` runs this setup in Chromium against a second origin that serves the installed package with jsDelivr's headers.
167
263
 
168
- **Text Compression - `compressText`** (ops/sec, higher is better):
264
+ ## Benchmarks
169
265
 
170
- | Payload | Quality 1 | Quality 6 | Quality 11 |
171
- |---|---|---|---|
172
- | Short JSON (83 B) | - | 36,264 | - |
173
- | Markdown (2 KB) | 36,002 | 12,932 | 408 |
174
- | HTML (8 KB) | 8,222 | 2,535 | 19 |
266
+ Numbers are operations per second from `vitest bench` (higher is better), measured on 2026-11-08 on an Apple Silicon Mac with Node.js 26.11 and headless Chromium 158. Run them yourself with `make bench` (Node.js) and `bun run bench:browser` (Chromium).
175
267
 
176
- **Binary Compression - `compress`** (ops/sec):
268
+ **Compression** (`compressText` for text, `compress` for binary), Node.js / Chromium:
177
269
 
178
270
  | Payload | Quality 1 | Quality 6 | Quality 11 |
179
271
  |---|---|---|---|
180
- | Pseudorandom (512 B) | - | 29,712 | - |
181
- | SVG (4.9 MB) | 15.9 | 4.8 | 0.15 |
182
-
183
- **Text Decompression - `decompressText`** (ops/sec):
184
-
185
- | Payload | ops/sec |
186
- |---|---|
187
- | Short JSON (83 B) | 157,906 |
188
- | Markdown (2 KB) | 51,946 |
189
- | HTML (8 KB) | 35,990 |
190
-
191
- **Binary Decompression - `decompress`** (ops/sec):
192
-
193
- | Payload | ops/sec |
194
- |---|---|
195
- | Pseudorandom (512 B) | 122,072 |
196
- | SVG (4.9 MB) | 39.7 |
197
- | PNG (2.1 MB) | 841 |
272
+ | Short JSON (83 B) | - | 55,883 / 39,120 | - |
273
+ | Markdown (2 KB) | 61,300 / 46,437 | 21,410 / 17,654 | 502 / 428 |
274
+ | HTML (28 KB) | 33,120 / 22,730 | 10,548 / 8,962 | 20.6 / 18.9 |
275
+ | Pseudorandom (512 B) | - | 43,034 / 38,924 | - |
276
+ | SVG (4.9 MB) | 43.0 / 39.7 | 6.79 / 6.15 | 0.26 / 0.28 |
277
+ | PNG (2.1 MB) | 1,505 / 1,795 | 93.1 / 139 | 0.46 / 0.67 |
278
+
279
+ **Decompression** (`decompressText` for text, `decompress` for binary):
280
+
281
+ | Payload | Node.js | Chromium |
282
+ |---|---|---|
283
+ | Short JSON (83 B) | 140,803 | 190,084 |
284
+ | Markdown (2 KB) | 68,431 | 88,380 |
285
+ | HTML (28 KB) | 44,058 | 54,016 |
286
+ | Pseudorandom (512 B) | 120,529 | 146,854 |
287
+ | SVG (4.9 MB) | 66.9 | 77.0 |
288
+ | PNG (2.1 MB) | 873 | 993 |
289
+
290
+ ### Large payloads (100 MB)
291
+
292
+ `bench/large.mjs` decompresses two inputs with `maxOutputSize: 128 * 1024 * 1024` (above the 64 MiB default): 100 MiB of generated JSON log lines, stored as a 13.3 MB Brotli file (quality 11, window 24), and a 100 MB Brotli file of incompressible bytes. "Warm" is the median of 5 calls on an initialized instance; "first call" is the first decompression on a fresh instance, which also grows and pages in WASM memory, as a one-off decode does. Native is `node:zlib`, the C Brotli library. Measured on 2026-11-09 on an Apple M5 Max:
293
+
294
+ | | Node.js 26.11 | Bun 1.4.3 | Chromium 158 |
295
+ |---|---:|---:|---:|
296
+ | 100 MiB text, warm (native) | 190 ms (132 ms) | 226 ms (141 ms) | 200 ms |
297
+ | 100 MiB text, first call | 193 ms | 272 ms | 223 ms |
298
+ | 100 MB Brotli file, warm (native) | 49 ms (11 ms) | 66 ms (7 ms) | 58 ms |
299
+ | 100 MB Brotli file, first call | 49 ms | 83 ms | 64 ms |
300
+ | WASM memory after decoding: text / file | 19.6 / 19.3 MiB | 19.6 / 19.3 MiB | 19.6 / 19.3 MiB |
301
+ | Compress the 100 MiB text at quality 6 (native) | 2,069 ms (1,173 ms) | 2,201 ms (1,021 ms) | - |
302
+ | WASM memory after compressing | 69 MiB | 69 MiB | - |
303
+ | Load + `init()`: `decompressor` / `decompressor/slim` | 2.7 / 1.3 ms | 2.6 / 1.9 ms | 3.0 / 2.2 ms |
304
+ | Load + `init()`: `compressor` / `compressor/slim` | 7.6 / 1.8 ms | 4.7 / 3.1 ms | 5.2 / 2.2 ms |
305
+
306
+ - Warm text decoding produces 440 to 530 MiB/s of output. Timings vary between runs by up to a third (native decoding of the text took 132 to 174 ms on the same machine), so compare builds within one session: interleaved with the previous whole-buffer build in Node.js, warm text decoding took 162 to 199 ms against 179 to 211 ms, faster in each of three rounds, and the file and compression times did not differ beyond noise.
307
+ - The incompressible file is mostly copying: into WASM memory, through the decoder's window and back out, which native code does once.
308
+ - WASM memory holds the decoder's window (16 MiB for these window-24 streams), its tables and two 1 MiB chunks, independent of the payload size. The previous whole-buffer build needed 283 MiB for the text and 320 MiB for the file, and kept that memory for the life of the instance, because WASM memory never shrinks. The compressor needed 214 MiB before and 69 MiB now: its encoder state at quality 6 plus one 4 MiB window of input and a 1 MiB output chunk.
309
+ - Chromium offers JavaScript no native Brotli decoder (`DecompressionStream` rejects `"brotli"` in Chromium 158); browsers decode Brotli natively only as HTTP `Content-Encoding`.
310
+ - Load + `init()` is measured from local files; over a network, the download sizes above add their transfer time.
311
+
312
+ `make bench` runs the suite and fails when a result exceeds [`bench/perf-budgets.json`](bench/perf-budgets.json): 2 times these timings (at least +5 ms) and 10 % above these memory sizes. Timings depend on the machine, so `make bench` is not part of `make verify`; recalibrate the budgets on other hardware.
198
313
 
199
314
  ## Examples
200
315
 
@@ -212,117 +327,101 @@ Sample output:
212
327
  ════════════════════════════════════════════════════════════════════════
213
328
  Quality │ Output │ Ratio │ Compress ms │ Decompress ms
214
329
  ------- │ ---------- │ ------- │ ----------- │ -------------
215
- 1 │ 87 B │ 104.8% │ 1.02 │ 0.32
216
- 4 │ 83 B │ 100.0% │ 0.55 │ 0.74
217
- 6 │ 80 B │ 96.4% │ 0.67 │ 0.16
218
- 9 │ 80 B │ 96.4% │ 2.13 │ 0.12
219
- 11 │ 85 B │ 102.4% │ 3.54 │ 0.28
330
+ 1 │ 87 B │ 104.8% │ 0.43 │ 0.23 [OK]
331
+ 4 │ 83 B │ 100.0% │ 0.36 │ 0.20 [OK]
332
+ 6 │ 80 B │ 96.4% │ 0.36 │ 0.04 [OK]
333
+ 9 │ 80 B │ 96.4% │ 1.88 │ 0.04 [OK]
334
+ 11 │ 85 B │ 102.4% │ 2.31 │ 0.11 [OK]
220
335
 
221
336
  ════════════════════════════════════════════════════════════════════════
222
337
  Pseudorandom binary (512 B) (512 B input)
223
338
  ════════════════════════════════════════════════════════════════════════
224
339
  Quality │ Output │ Ratio │ Compress ms │ Decompress ms
225
340
  ------- │ ---------- │ ------- │ ----------- │ -------------
226
- 1 │ 286 B │ 55.9% │ 0.08 │ 0.91
227
- 4 │ 278 B │ 54.3% │ 0.28 │ 0.06
228
- 6 │ 270 B │ 52.7% │ 0.27 │ 0.09
229
- 9 │ 259 B │ 50.6% │ 0.59 │ 0.05
230
- 11 │ 239 B │ 46.7% │ 2.60 │ 0.28
341
+ 1 │ 286 B │ 55.9% │ 0.08 │ 1.22 [OK]
342
+ 4 │ 278 B │ 54.3% │ 0.08 │ 0.04 [OK]
343
+ 6 │ 270 B │ 52.7% │ 0.08 │ 0.03 [OK]
344
+ 9 │ 259 B │ 50.6% │ 0.23 │ 0.03 [OK]
345
+ 11 │ 239 B │ 46.7% │ 1.57 │ 0.15 [OK]
231
346
 
232
347
  ════════════════════════════════════════════════════════════════════════
233
348
  SVG (4.9 MB) (4.9 MB input)
234
349
  ════════════════════════════════════════════════════════════════════════
235
350
  Quality │ Output │ Ratio │ Compress ms │ Decompress ms
236
351
  ------- │ ---------- │ ------- │ ----------- │ -------------
237
- 1 │ 2.8 MB │ 57.0% │ 68.27 │ 74.07
238
- 4 │ 1.9 MB │ 38.0% │ 85.92 │ 43.93
239
- 6 │ 1.8 MB │ 37.4% │ 208.86 │ 43.04
240
- 9 │ 1.8 MB │ 36.8% │ 504.55 │ 42.33
241
- 11 │ 1.7 MB │ 34.5% │ 5093.09 │ 49.22
352
+ 1 │ 1.9 MB │ 38.3% │ 25.21 │ 35.76 [OK]
353
+ 4 │ 1.8 MB │ 36.6% │ 75.33 │ 33.47 [OK]
354
+ 6 │ 1.8 MB │ 36.9% │ 158.53 │ 30.81 [OK]
355
+ 9 │ 1.8 MB │ 36.8% │ 516.73 │ 28.96 [OK]
356
+ 11 │ 1.7 MB │ 34.5% │ 3473.27 │ 38.55 [OK]
242
357
 
243
358
  ════════════════════════════════════════════════════════════════════════
244
359
  PNG (2.1 MB) (2.1 MB input)
245
360
  ════════════════════════════════════════════════════════════════════════
246
361
  Quality │ Output │ Ratio │ Compress ms │ Decompress ms
247
362
  ------- │ ---------- │ ------- │ ----------- │ -------------
248
- 1 │ 2.1 MB │ 100.1% │ 17.39 │ 11.60
249
- 4 │ 2.1 MB │ 100.0% │ 5.75 │ 11.06
250
- 6 │ 2.1 MB │ 100.0% │ 13.64 │ 10.74
251
- 9 │ 2.0 MB │ 99.0% │ 206.06 │ 23.39
252
- 11 │ 2.0 MB │ 96.2% │ 3162.39 │ 30.90
363
+ 1 │ 2.1 MB │ 100.0% │ 0.68 │ 12.58 [OK]
364
+ 4 │ 2.1 MB │ 100.0% │ 2.59 │ 9.11 [OK]
365
+ 6 │ 2.1 MB │ 100.0% │ 9.63 │ 8.56 [OK]
366
+ 9 │ 2.0 MB │ 99.0% │ 202.22 │ 13.13 [OK]
367
+ 11 │ 2.0 MB │ 96.2% │ 1556.99 │ 24.54 [OK]
253
368
 
254
369
  ════════════════════════════════════════════════════════════════════════
255
- All round-trips passed
370
+ [OK] All round-trips passed
256
371
  ════════════════════════════════════════════════════════════════════════
257
372
  ```
258
373
 
259
374
  **Key takeaways:**
260
375
 
261
- - **SVG** (text-based) compresses well: 57% → 34.5% ratio as quality increases, at the cost of much longer compression times at q11.
262
- - **PNG** (already compressed) barely shrinks: ~100% ratio at q1-q6, only 96% at q11 — Brotli can't improve on an already-compressed binary format.
263
- - **Short text** can actually *grow* at low quality levels (104.8% at q1) — the Brotli header overhead exceeds the savings on tiny payloads.
376
+ - **SVG** (text-based) compresses well: the ratio falls from 38.3 % at quality 1 to 34.5 % at quality 11, while compression time grows from about 25 ms to about 3.5 s.
377
+ - **PNG** (already compressed) barely shrinks: about 100 % at qualities 1 to 6 and 96.2 % at quality 11, because its pixel data is already deflate-compressed.
378
+ - **Short text** can *grow* at low quality (104.8 % at quality 1): on an 83-byte payload the Brotli framing outweighs the savings.
264
379
 
265
380
  ## Defaults
266
381
 
267
- For typical web text (JSON, Markdown, HTML, CSS, JS):
268
-
269
- - `quality = 6`
382
+ - `quality = 11`
270
383
  - `lgwin = 22`
271
384
 
272
- That is a sane point on the speed/ratio curve. Crank `quality` higher if you
273
- need smaller output and can afford more CPU time.
385
+ These are the defaults of Node.js zlib (`BROTLI_DEFAULT_QUALITY`, `BROTLI_DEFAULT_WINDOW`), and quality 11 gives the smallest output. Pass a lower `quality` when compression time matters more than size, for example when compressing each response as it is served.
386
+
387
+ The `defuss-brotli` command also uses quality 11 and, like the `brotli` CLI, the smallest window that holds the input.
274
388
 
275
389
  ## Build
276
390
 
277
391
  ### Prerequisites
278
392
 
279
- - [Bun](https://bun.sh/) >= 1.0
280
- - Rust toolchain + `wasm32-unknown-unknown` target
281
- - `wasm-pack`
282
- - `wasm-opt` (optional, from [binaryen](https://github.com/WebAssembly/binaryen) — produces smaller WASM)
283
-
284
- Install everything automatically (cross-platform — installs Rust/wasm-pack if missing):
393
+ The build needs [mise](https://mise.jdx.dev/), which installs the pinned Bun, `wasm-pack` and `wasm-opt` (binaryen) from `mise.toml`, and rustup, which installs the Rust toolchain from `rust-toolchain.toml`. `make setup` installs both when missing, then the locked dependencies and the Playwright Chromium:
285
394
 
286
395
  ```bash
287
396
  make setup
288
- # or
289
- bun run setup
397
+ make build
290
398
  ```
291
399
 
292
- Build everything:
400
+ On macOS, if Rust build scripts fail to link with `tapi error: malformed file ... libSystem.B.tbd`, the Command Line Tools linker is older than the newest SDK. Copy `.env.example` to `.env` and set `SDKROOT` to an older SDK under `/Library/Developer/CommandLineTools/SDKs/`.
293
401
 
294
- ```bash
295
- bun install
296
- bun run build
297
- ```
402
+ What happens during build, for each of the two crates:
298
403
 
299
- What happens during build:
404
+ 1. `wasm-pack` builds the crate for `wasm32-unknown-unknown`.
405
+ 2. `wasm-opt -O3` optimizes the binary.
406
+ 3. The binary is written as a base64 module next to the wasm-bindgen glue.
407
+ 4. `bun build` bundles `src/compressor.ts` or `src/decompressor.ts` with the glue and the base64 module into `dist/`.
300
408
 
301
- 1. `wasm-pack` builds the compressor crate
302
- 2. `wasm-pack` builds the decompressor crate
303
- 3. `wasm-opt -Oz` runs if available
304
- 4. `bun build` bundles `src/compressor.ts` and `src/decompressor.ts` into `dist/`
305
- 5. `tsc` generates declaration files
409
+ `bun build` then bundles the command, `src/bin.ts`, into `dist/cli.js`, and `tsc` generates the declaration files.
306
410
 
307
411
  ## Test and Benchmark
308
412
 
309
- Node:
310
-
311
413
  ```bash
312
- bun test
313
- bun example
314
- bun test:e2e
315
-
316
- bun bench
317
- bun bench:browser
414
+ make test # Rust unit tests, then Vitest in Node and in headless Chromium
415
+ make coverage # Rust line coverage (fails below 60 %), then the TypeScript coverage table
416
+ make e2e # packs the tarball into clean projects: Node.js, Bun, the CLI via bunx, a bun build app and a Vite app (build + dev) in Chromium
417
+ make lint # oxlint, tsc, clippy, rustfmt
418
+ make verify # all of the above, as CI runs it
419
+ make bench # Vitest bench and the 100 MB benchmark in Node, Bun and Chromium, against the perf budgets
420
+ bun run example # quality comparison table
318
421
  ```
319
422
 
320
- Browser:
321
-
322
- ```bash
323
- bun run test:browser
324
- ```
423
+ `make e2e` writes one result file per consumer to `output/` (`e2e-node.json`, `e2e-bun.json`, `e2e-cli.json`, `e2e-bun-bundle.json`, `e2e-vite-build.json`, `e2e-vite-dev.json`); `make e2e-runtime`, `make e2e-cli`, `make e2e-bun` and `make e2e-vite` run one group.
325
424
 
326
425
  ## License
327
426
 
328
- MIT
427
+ MIT. The `.wasm` files and the inline modules contain compiled third-party code under BSD-3-Clause and MIT terms (brotli, brotli-decompressor, wasm-bindgen, the Rust standard library); [THIRD_PARTY_NOTICES.txt](THIRD_PARTY_NOTICES.txt) lists each component with its license text.