defuss-brotli 0.1.3 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +34 -0
- package/README.md +245 -146
- package/THIRD_PARTY_NOTICES.txt +163 -0
- package/dist/cli.js +37 -0
- package/dist/compressor-core.d.ts +163 -0
- package/dist/compressor-encode.d.ts +16 -0
- package/dist/compressor-slim.d.ts +17 -0
- package/dist/compressor-slim.js +1 -0
- package/dist/compressor.d.ts +1 -96
- package/dist/compressor.js +1 -190
- package/dist/compressor.wasm +0 -0
- package/dist/decompressor-core.d.ts +93 -0
- package/dist/decompressor-slim.d.ts +17 -0
- package/dist/decompressor-slim.js +1 -0
- package/dist/decompressor.d.ts +1 -84
- package/dist/decompressor.js +1 -183
- package/dist/decompressor.wasm +0 -0
- package/dist/wasm.d.ts +30 -0
- package/dist/workers.d.ts +27 -0
- package/package.json +26 -10
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to `defuss-brotli`. Versions follow [Semantic Versioning](https://semver.org/); before 1.0.0, a minor version may change behavior.
|
|
4
|
+
|
|
5
|
+
## 0.3.0 - 2026-11-09
|
|
6
|
+
|
|
7
|
+
### Changed
|
|
8
|
+
|
|
9
|
+
- **The default compression quality is now 11 instead of 6**, the default of Node.js zlib (`BROTLI_DEFAULT_QUALITY`) and of the `brotli` CLI. `compress()` and `compressText()` calls that pass no `quality` now return the smallest output and take longer: about 3.8 s instead of 0.15 s for a 5.1 MB SVG (Node.js 26). Pass `{ quality: 6 }` to keep the previous output.
|
|
10
|
+
- The compressor entry points embed the worker script of `compressParallel`: `compressor-slim.js` grows from 4.8 KB to 15.2 KB and `compressor.wasm` by 11.5 KB (raw). The decompressor entry points grow by 29 B.
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
|
|
14
|
+
- `compressParallel(input, options)` compresses up to 8 parts of the input at once on workers, each with its own instance of the compressor, and joins them into one standard Brotli stream. At quality 11 it is 2.3 to 4.3 times faster than `compress` from 256 KiB, with output 0.01 to 0.10 % larger (Node.js 26). The output depends only on the input, `quality` and `lgwin`, not on `threads` or the machine. It also exports `BrotliParallelOptions`, `PARALLEL_PARTS` and `MIN_PART_SIZE`.
|
|
15
|
+
- The `defuss-brotli` command (`bunx defuss-brotli compress FILE`, `bunx defuss-brotli decompress FILE.br`) takes the options, defaults, output names and refusals of the `brotli` 1.2.0 CLI. It compresses with `compressParallel`; `--threads=NUM` sets the number of workers. From 256 KiB it compresses 1.2 to 5.3 times faster than `brotli` 1.2.0, depending on the input and runtime. `--large_window`, `--comment`, `--dictionary` and `--concatenated` are not supported.
|
|
16
|
+
- `CHANGELOG.md`, shipped in the package.
|
|
17
|
+
|
|
18
|
+
## 0.2.0 - 2026-11-09
|
|
19
|
+
|
|
20
|
+
### Added
|
|
21
|
+
|
|
22
|
+
- `/slim` entry points, `defuss-brotli/compressor/slim` and `defuss-brotli/decompressor/slim`, with the same API as the inline ones. They load the binary from a separate `.wasm` file (`defuss-brotli/compressor.wasm`, `defuss-brotli/decompressor.wasm`): by default from next to the module, or from a URL, a `Response`, bytes or a compiled `WebAssembly.Module` passed to `init()`.
|
|
23
|
+
- `THIRD_PARTY_NOTICES.txt` with the license texts of the code compiled into the `.wasm` files.
|
|
24
|
+
|
|
25
|
+
### Changed
|
|
26
|
+
|
|
27
|
+
- Both modules stream input and output through fixed chunks in WASM memory, so linear memory no longer grows with the payload: it holds 19.6 MiB after decoding 100 MiB, and 69 MiB after compressing it at quality 6.
|
|
28
|
+
- The compressor feeds the encoder one window per call instead of 4 KiB, so qualities 0 and 1 find repeats across chunks (a 28 KiB page at quality 1: 1,189 B instead of 3,853 B); the output equals one encoder call over the whole input at every quality.
|
|
29
|
+
- The decoder is built without the Rust standard library, which brings `defuss-brotli/decompressor/slim` to about 51 KB with Brotli quality 11 (85 KB from jsDelivr).
|
|
30
|
+
- `bun run build`, and with it `npm publish`, finds the pinned build tools without an activated mise.
|
|
31
|
+
|
|
32
|
+
## 0.1.1 to 0.1.3
|
|
33
|
+
|
|
34
|
+
These releases predate this repository's history.
|
package/README.md
CHANGED
|
@@ -2,12 +2,15 @@
|
|
|
2
2
|
|
|
3
3
|
A split pure-Rust Brotli package for Node.js + browsers.
|
|
4
4
|
|
|
5
|
-
- **Small browser decoder
|
|
5
|
+
- **Small browser decoder** - `defuss-brotli/decompressor/slim` loads only the decoder: about 51 KB precompressed with Brotli quality 11 and 85 KB from jsDelivr (quality 4), most of it the Brotli dictionary every decoder carries
|
|
6
6
|
- **Separate heavier encoder bundle** - used server-side or at build time
|
|
7
7
|
- **Single NPM package** with split exports
|
|
8
|
+
- **Inlined or separate `.wasm`** - one self-contained file by default, or a `/slim` entry that loads the binary on its own
|
|
8
9
|
- **Pure-Rust WebAssembly** - no native addons
|
|
9
10
|
- **Typed TypeScript API** with full JSDoc
|
|
10
11
|
- **Vitest coverage** in Node and browser
|
|
12
|
+
- **Parallel compression** - `compressParallel()` compresses parts of the input on workers: 2.3 to 4.3 times faster at quality 11 from 256 KiB, with output at most 0.1 % larger
|
|
13
|
+
- **`defuss-brotli` command** with the options of the `brotli` CLI, `bunx defuss-brotli compress FILE`; from 256 KiB it compresses 1.2 to 5.3 times faster than `brotli` 1.2.0
|
|
11
14
|
|
|
12
15
|
## Install
|
|
13
16
|
|
|
@@ -35,9 +38,101 @@ import { init, decompressText } from "defuss-brotli/decompressor";
|
|
|
35
38
|
await init();
|
|
36
39
|
|
|
37
40
|
const text = decompressText(compressed);
|
|
38
|
-
//
|
|
41
|
+
// returns "Hello, Brotli!"
|
|
39
42
|
```
|
|
40
43
|
|
|
44
|
+
### Command line
|
|
45
|
+
|
|
46
|
+
The package ships a `defuss-brotli` command with the options of the `brotli` CLI, so it can replace `brotli` in build scripts and on machines where only Node.js or Bun is installed:
|
|
47
|
+
|
|
48
|
+
```bash
|
|
49
|
+
bunx defuss-brotli compress dist/app.js # writes dist/app.js.br, keeps dist/app.js
|
|
50
|
+
bunx defuss-brotli decompress dist/app.js.br -o app.js
|
|
51
|
+
bunx defuss-brotli -q 5 -v *.html *.css # brotli's spelling: compress by default
|
|
52
|
+
bunx defuss-brotli -d -c page.html.br | head # -d decompresses, -c writes to stdout
|
|
53
|
+
cat data.json | bunx defuss-brotli > data.json.br
|
|
54
|
+
bunx defuss-brotli -t data.json.br # test integrity, write nothing
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
`compress` and `decompress` as the first argument select the mode; without one the command compresses unless `-d` or `-t` is given, as `brotli` does. It compresses with [`compressParallel`](#compressparallel), so inputs above 32 KiB are split into parts that compress on several workers. `npx defuss-brotli` works the same way. The command runs on Node.js when it is on the `PATH` and on Bun otherwise; `bunx --bun defuss-brotli` always uses Bun, which starts faster.
|
|
58
|
+
|
|
59
|
+
| Option | Meaning |
|
|
60
|
+
|---|---|
|
|
61
|
+
| `-0` to `-9`, `-q NUM`, `--quality=NUM`, `-Z`, `--best` | quality 0 to 11, default 11 (`-Z`) |
|
|
62
|
+
| `-w NUM`, `--lgwin=NUM` | window 10 to 24; `0` (default) picks the smallest window that holds the input |
|
|
63
|
+
| `-c`, `--stdout` | write to standard output |
|
|
64
|
+
| `-o FILE`, `--output=FILE` | output file (only with one input) |
|
|
65
|
+
| `-d`, `--decompress` / `-t`, `--test` | decompress / decompress without writing, to check integrity |
|
|
66
|
+
| `-f`, `--force` | overwrite existing output, allow terminal input and output |
|
|
67
|
+
| `-k`, `--keep` / `-j`, `--rm` | keep (default) / delete each source after writing its output |
|
|
68
|
+
| `-s`, `--squash` | when compressing, drop an output that is larger than its source |
|
|
69
|
+
| `-n`, `--no-copy-stat` | do not copy the source's mode and timestamps to the output |
|
|
70
|
+
| `-S SUF`, `--suffix=SUF` | suffix to add or strip, default `.br` |
|
|
71
|
+
| `--threads=NUM` | compress on up to `NUM` workers, default the number of CPUs, at most 8; the output is the same for every `NUM` (not a `brotli` option) |
|
|
72
|
+
| `-v`, `--verbose` / `-h`, `--help` / `-V`, `--version` | per-file sizes and time / usage / version |
|
|
73
|
+
|
|
74
|
+
With no file, or with `-`, it reads standard input and writes standard output. Short flags combine (`-9kf`) and `--` ends the options. Each input is written to `FILE.br` or, when decompressing, to `FILE` without `.br`. The command refuses to overwrite a file without `-f`, to write compressed bytes to a terminal, and to decompress a name that lacks the suffix. It stops at the first file that fails and exits with status 1.
|
|
75
|
+
|
|
76
|
+
VERIFIED (2026-11-09, against brotli 1.2.0 on macOS):
|
|
77
|
+
|
|
78
|
+
- The defaults, output file names and refusals match `brotli`'s. Messages keep its wording, with the error text of Node.js or Bun (`file already exists` where `brotli` prints `File exists`).
|
|
79
|
+
- The default output for `input/page.html` and `input/data.json` is byte-identical to `brotli`'s.
|
|
80
|
+
- On larger inputs, which it compresses in parts, the default output was 0.00 to 0.16 % larger than `brotli`'s: 64 KiB and 1 MiB of SVG, a 64 KiB PNG head, a 2.1 MB PNG and a 5.1 MB SVG.
|
|
81
|
+
|
|
82
|
+
Some `brotli` behavior is not supported or differs:
|
|
83
|
+
|
|
84
|
+
- `--large_window`, `-C`/`--comment`, `-D`/`--dictionary` and `-K`/`--concatenated` fail with a reason, because the WASM encoder and decoder do not implement them.
|
|
85
|
+
- Usage errors print a one-line hint instead of the whole help text.
|
|
86
|
+
- `-t -j` is refused; `brotli` 1.2.0 deletes the file it has just tested.
|
|
87
|
+
- Each input is read into memory whole and each output is written at once, so files must fit in memory; decompressed output is limited to 2^32 − 1 bytes, the wasm32 limit.
|
|
88
|
+
|
|
89
|
+
At the default quality 11, the command compresses 64 KiB as fast as `brotli` 1.2.0 on Bun, and from 256 KiB 2.3 to 5.3 times faster on Bun and 1.2 to 5.1 times faster on Node.js. Smaller files take longer, because it first starts a JavaScript runtime and compiles the WASM. Median of 5 interleaved runs on Apple Silicon (18 cores), startup included, Node.js 26 and Bun 1.4:
|
|
90
|
+
|
|
91
|
+
| Input | `defuss-brotli` on Node.js | on Bun (`bunx --bun`) | `brotli` 1.2.0 |
|
|
92
|
+
|---|---|---|---|
|
|
93
|
+
| 29 KB HTML (`input/page.html`), one part | 164 ms | 89 ms | 55 ms |
|
|
94
|
+
| 64 KiB of SVG, 2 parts | 174 ms | 68 ms | 69 ms |
|
|
95
|
+
| 256 KiB of SVG, 8 parts | 178 ms | 89 ms | 219 ms |
|
|
96
|
+
| 1 MiB of SVG, 8 parts | 320 ms | 301 ms | 879 ms |
|
|
97
|
+
| 2.1 MB PNG, 8 parts | 740 ms | 714 ms | 3.79 s |
|
|
98
|
+
| 5.1 MB SVG, 8 parts | 1.89 s | 1.71 s | 4.00 s |
|
|
99
|
+
|
|
100
|
+
The SVG inputs are the start of the 5.1 MB file. Decompressing the 5.1 MB SVG took 93 ms on Node.js and 55 ms on Bun, against 32 ms for `brotli` (median of 3). Startup (`-V`) took 59 ms on Node.js and 24 ms on Bun.
|
|
101
|
+
|
|
102
|
+
### Inlined or separate `.wasm`
|
|
103
|
+
|
|
104
|
+
Each module comes in two builds with the same API. The default entry points embed the WASM binary as base64, so one JavaScript file is all a runtime or bundler needs. The `/slim` entry points leave the binary in its own `.wasm` file, which HTTP caches independently of your JavaScript, which compiles while it downloads when served as `application/wasm`, and which transfers 21 to 45 % fewer bytes depending on module and transfer encoding (see [Download sizes and budgets](#download-sizes-and-budgets)).
|
|
105
|
+
|
|
106
|
+
Without an argument, a slim `init()` loads the `.wasm` file from next to its module, via `new URL("./decompressor.wasm", import.meta.url)`. Browsers fetch it from there, and Node.js 20.16+ / 22.3+ and Bun read it from disk:
|
|
107
|
+
|
|
108
|
+
```ts
|
|
109
|
+
import { init, decompressText } from "defuss-brotli/decompressor/slim";
|
|
110
|
+
|
|
111
|
+
await init();
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
When your bundler or server does not place the file next to the bundle, copy `defuss-brotli/decompressor.wasm` (or `compressor.wasm`) to where you serve static assets and pass its location, or pass the response, the bytes or a compiled module:
|
|
115
|
+
|
|
116
|
+
```ts
|
|
117
|
+
await init("/assets/decompressor.wasm"); // URL, fetched (streamed when served as application/wasm)
|
|
118
|
+
await init(fetch("https://cdn.example.com/d.wasm")); // Response or Promise<Response>
|
|
119
|
+
await init(await readFile(wasmPath)); // ArrayBuffer or typed array
|
|
120
|
+
await init(compiledModule); // WebAssembly.Module
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
`init()` loads once; later calls return the first Promise and ignore their argument. After a failed load (for example a 404), the next call tries again.
|
|
124
|
+
|
|
125
|
+
#### Bundlers
|
|
126
|
+
|
|
127
|
+
`make e2e` builds a clean app per bundler from the packed tarball and runs it in Chromium:
|
|
128
|
+
|
|
129
|
+
| Bundler | `init()` without an argument | Explicit asset import |
|
|
130
|
+
|---|---|---|
|
|
131
|
+
| Vite 8.3.6, `vite build` and `vite` dev | works: Vite resolves `new URL(..., import.meta.url)` and emits the `.wasm` | `import url from "defuss-brotli/decompressor.wasm?url"`, then `init(url)` |
|
|
132
|
+
| `bun build` (Bun 1.4) | copy `compressor.wasm` / `decompressor.wasm` next to the bundle yourself: Bun leaves `new URL(...)` unchanged | `import url from "defuss-brotli/decompressor.wasm" with { type: "file" }`, then `init(url)` |
|
|
133
|
+
|
|
134
|
+
With an explicit import, the bundler copies the file and gives it a hashed name, so it can be cached long-term. Node.js and Bun as runtimes need neither: `init()` reads the file from `node_modules`.
|
|
135
|
+
|
|
41
136
|
## API
|
|
42
137
|
|
|
43
138
|
### `compress` vs `compressText`
|
|
@@ -51,6 +146,30 @@ const text = decompressText(compressed);
|
|
|
51
146
|
raw bytes. `TextEncoder` converts to canonical UTF-8 - the standard encoding for
|
|
52
147
|
web text. `compressText` handles this so you don't have to.
|
|
53
148
|
|
|
149
|
+
### `compressParallel`
|
|
150
|
+
|
|
151
|
+
```ts
|
|
152
|
+
import { init, compressParallel } from "defuss-brotli/compressor";
|
|
153
|
+
|
|
154
|
+
await init();
|
|
155
|
+
const compressed = await compressParallel(bytes, { quality: 11, threads: 4 });
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
`compressParallel` splits the input into at most 8 parts of at least 32 KiB, compresses each part on a worker with its own instance of the compressor, and joins the parts into one standard Brotli stream on the calling thread. Each part gets the window of input before it as a dictionary, so matches still reach back across parts. This is the multithreaded mode of the `brotli` crate, run on workers instead of threads.
|
|
159
|
+
|
|
160
|
+
- **Output:** it depends only on the input, `quality` and `lgwin`, never on `threads` or the number of CPUs; `threads: 1` compresses the parts one after another on the calling thread and returns the same bytes. Below quality 2, or when the input fits in one part, it returns exactly what `compress` returns. Otherwise the output differs from `compress`'s and is slightly larger.
|
|
161
|
+
- **Workers:** Web Workers from a Blob URL in browsers and Bun, and `worker_threads` in Node.js. Pages with a Content Security Policy must allow `worker-src blob:`. No extra file is needed, because the worker script is embedded in the module. The workers are terminated before the Promise settles. A runtime that cannot start such a worker compresses the parts on the calling thread.
|
|
162
|
+
- **`threads`:** how many workers run at once, default the number of logical CPUs, at most 8. Each worker allocates its own encoder state, as a `compress` call does.
|
|
163
|
+
|
|
164
|
+
VERIFIED on Node.js 26 (Apple Silicon, quality 11, window 22, median of 3 in-process runs, first call):
|
|
165
|
+
|
|
166
|
+
| Input | `compress` | `compressParallel` | Size |
|
|
167
|
+
|---|---|---|---|
|
|
168
|
+
| 256 KiB of SVG | 187 ms | 44 ms | +0.01 % |
|
|
169
|
+
| 1 MiB of SVG | 617 ms | 189 ms | +0.05 % |
|
|
170
|
+
| 2.1 MB PNG | 1.79 s | 0.60 s | +0.10 % |
|
|
171
|
+
| 5.1 MB SVG | 3.82 s | 1.67 s | +0.09 % |
|
|
172
|
+
|
|
54
173
|
### `decompress` vs `decompressText`
|
|
55
174
|
|
|
56
175
|
| Function | Output | Decoding | When to use |
|
|
@@ -65,7 +184,7 @@ valid UTF-8, replacement characters (U+FFFD) are inserted rather than throwing.
|
|
|
65
184
|
|
|
66
185
|
```ts
|
|
67
186
|
interface BrotliCompressOptions {
|
|
68
|
-
quality?: number; // 0-11, default
|
|
187
|
+
quality?: number; // 0-11, default 11
|
|
69
188
|
lgwin?: number; // 10-24, default 22
|
|
70
189
|
}
|
|
71
190
|
```
|
|
@@ -92,109 +211,105 @@ Hard cap on decompressed output to prevent **decompression bombs**. Raise explic
|
|
|
92
211
|
|
|
93
212
|
### Guarantees
|
|
94
213
|
|
|
95
|
-
- `compressText(s)`
|
|
96
|
-
- `compress(b)`
|
|
214
|
+
- `decompressText(compressText(s))` returns `s` exactly for any JS string.
|
|
215
|
+
- `decompress(compress(b))` returns the same bytes as `b` for any `Uint8Array`.
|
|
97
216
|
- `init()` is **idempotent** - calling it multiple times returns the same Promise and does not reload WASM.
|
|
98
|
-
- Importing only `defuss-brotli/decompressor` **never loads** the encoder WASM binary.
|
|
217
|
+
- Importing only `defuss-brotli/decompressor` (or `/decompressor/slim`) **never loads** the encoder WASM binary.
|
|
218
|
+
- Inline and `/slim` builds of a module run the same code and produce identical output.
|
|
99
219
|
|
|
100
220
|
### Limits
|
|
101
221
|
|
|
102
|
-
- `quality` must be an integer in **0-11** (
|
|
103
|
-
- `lgwin` must be an integer in **10-24** (Brotli
|
|
104
|
-
- `maxOutputSize` defaults to **64 MiB** - the WASM module rejects output exceeding this.
|
|
105
|
-
-
|
|
222
|
+
- `quality` must be an integer in **0-11** (the range of the reference Brotli encoder).
|
|
223
|
+
- `lgwin` must be an integer in **10-24** (the window range of the Brotli format, RFC 7932).
|
|
224
|
+
- `maxOutputSize` defaults to **64 MiB** - the WASM module rejects output exceeding this. Values above 2^32 − 1 act as 2^32 − 1.
|
|
225
|
+
- `threads` must be a positive integer. `compressParallel` copies each part and its dictionary (up to one window) to its worker, so it holds up to 8 windows more than the input in JavaScript memory while it runs.
|
|
226
|
+
- Calls stream the input and output through chunks of at most 1 MiB (one window of input when compressing), so WASM memory holds the codec state and two chunks however large the payload; the returned `Uint8Array` lives in JavaScript memory. Truncated input, corrupt input and bytes after the end of the Brotli stream throw.
|
|
106
227
|
|
|
107
228
|
## Why the Split Exports?
|
|
108
229
|
|
|
109
|
-
The encoder and decoder are built from **two separate Rust Wasm crates**:
|
|
230
|
+
The encoder and decoder are built from **two separate Rust Wasm crates**: `brotli` behind `defuss-brotli/compressor` (server-side, build pipelines) and `brotli-decompressor` behind `defuss-brotli/decompressor` (browser clients). The encoder needs large static tables the decoder does not, so browser apps that only decompress pre-compressed data import just the decompressor and download about a sixth of the bytes. [ARCH.md](ARCH.md) explains where the sizes come from and why the build trades 1 % of size for speed.
|
|
110
231
|
|
|
111
|
-
|
|
112
|
-
|---|---|---|---|---|
|
|
113
|
-
| `defuss-brotli/compressor` | `brotli` | 984 KB | 1.3 MB | Server-side, build pipelines |
|
|
114
|
-
| `defuss-brotli/decompressor` | `brotli-decompressor` | 208 KB | 276 KB | Browser clients |
|
|
232
|
+
## Download sizes and budgets
|
|
115
233
|
|
|
116
|
-
|
|
234
|
+
What a browser downloads for one import, with each file transfer-compressed on its own, as a CDN or web server does per response:
|
|
117
235
|
|
|
118
|
-
|
|
236
|
+
| Import | Files | raw | gzip -9 | brotli q4 (jsDelivr) | brotli q11 | zstd -19 |
|
|
237
|
+
|---|---|---:|---:|---:|---:|---:|
|
|
238
|
+
| `defuss-brotli/decompressor/slim` | `decompressor-slim.js` + `decompressor.wasm` | 190,289 B | 85,633 B | 85,090 B | 50,828 B | 78,609 B |
|
|
239
|
+
| `defuss-brotli/decompressor` | `decompressor.js` | 252,213 B | 110,006 B | 111,536 B | 92,996 B | 101,435 B |
|
|
240
|
+
| `defuss-brotli/compressor/slim` | `compressor-slim.js` + `compressor.wasm` | 960,020 B | 453,582 B | 510,561 B | 396,565 B | 405,487 B |
|
|
241
|
+
| `defuss-brotli/compressor` | `compressor.js` | 1,275,038 B | 608,203 B | 663,816 B | 499,177 B | 524,736 B |
|
|
119
242
|
|
|
120
|
-
|
|
243
|
+
- **gzip -9** and **brotli q4** are what [jsDelivr](https://www.jsdelivr.com/) sends: measured on 2026-11-09, its brotli responses match quality 4 byte for byte and its gzip responses lie within 0.1 % of levels 6 to 9. jsDelivr serves `.wasm` files as `application/wasm` with `access-control-allow-origin: *`, and serves no zstd.
|
|
244
|
+
- **brotli q11** and **zstd -19** are what precompressed static hosting reaches (for example `.br` files next to the assets, or a CDN that compresses at its highest level). Quality 11 is unstable on `compressor.wasm`: passes of `wasm-opt` that change its raw size and its quality-4 size by 0.2 % move it between 340,693 B and 392,638 B, so its budget allows the upper value.
|
|
245
|
+
- `make metrics` prints these numbers for `dist/`. `make e2e` measures the packed tarball and fails when an import exceeds its limit in [`size-budgets.json`](size-budgets.json) (current size plus 2 %); raise a limit only for a measured, intended change.
|
|
121
246
|
|
|
122
|
-
|
|
247
|
+
### From a CDN without a bundler
|
|
123
248
|
|
|
124
|
-
|
|
125
|
-
bun run bench # Node.js
|
|
126
|
-
bun run bench:browser # Chromium via Playwright
|
|
127
|
-
```
|
|
128
|
-
|
|
129
|
-
### Node.js (Bun)
|
|
130
|
-
|
|
131
|
-
**Text Compression - `compressText`** (ops/sec, higher is better):
|
|
249
|
+
An [import map](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script/type/importmap) maps the package's exports to its files on jsDelivr; pin a version (`defuss-brotli@x.y.z`) in production. The slim `init()` then fetches `decompressor.wasm` from the same CDN directory:
|
|
132
250
|
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
251
|
+
```html
|
|
252
|
+
<script type="importmap">
|
|
253
|
+
{ "imports": { "defuss-brotli/decompressor/slim": "https://cdn.jsdelivr.net/npm/defuss-brotli/dist/decompressor-slim.js" } }
|
|
254
|
+
</script>
|
|
255
|
+
<script type="module">
|
|
256
|
+
import { init, decompressText } from "defuss-brotli/decompressor/slim";
|
|
138
257
|
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|---|---|---|---|
|
|
143
|
-
| Pseudorandom (512 B) | - | 29,430 | - |
|
|
144
|
-
| SVG (4.9 MB) | 8.0 | 4.0 | 0.15 |
|
|
145
|
-
| PNG (2.1 MB) | 39.6 | 82.8 | 0.21 |
|
|
146
|
-
|
|
147
|
-
PNG compresses faster than SVG at q6 because already-compressed data triggers
|
|
148
|
-
Brotli's fast-path fallback earlier, producing less work per byte.
|
|
149
|
-
|
|
150
|
-
**Text Decompression - `decompressText`** (ops/sec):
|
|
151
|
-
|
|
152
|
-
| Payload | ops/sec |
|
|
153
|
-
|---|---|
|
|
154
|
-
| Short JSON (83 B) | 141,241 |
|
|
155
|
-
| Markdown (2 KB) | 47,757 |
|
|
156
|
-
| HTML (8 KB) | 37,288 |
|
|
157
|
-
|
|
158
|
-
**Binary Decompression - `decompress`** (ops/sec):
|
|
159
|
-
|
|
160
|
-
| Payload | ops/sec |
|
|
161
|
-
|---|---|
|
|
162
|
-
| Pseudorandom (512 B) | 115,493 |
|
|
163
|
-
| SVG (4.9 MB) | 30.7 |
|
|
164
|
-
| PNG (2.1 MB) | 699 |
|
|
258
|
+
await init();
|
|
259
|
+
</script>
|
|
260
|
+
```
|
|
165
261
|
|
|
166
|
-
|
|
262
|
+
`make e2e` runs this setup in Chromium against a second origin that serves the installed package with jsDelivr's headers.
|
|
167
263
|
|
|
168
|
-
|
|
264
|
+
## Benchmarks
|
|
169
265
|
|
|
170
|
-
|
|
171
|
-
|---|---|---|---|
|
|
172
|
-
| Short JSON (83 B) | - | 36,264 | - |
|
|
173
|
-
| Markdown (2 KB) | 36,002 | 12,932 | 408 |
|
|
174
|
-
| HTML (8 KB) | 8,222 | 2,535 | 19 |
|
|
266
|
+
Numbers are operations per second from `vitest bench` (higher is better), measured on 2026-11-08 on an Apple Silicon Mac with Node.js 26.11 and headless Chromium 158. Run them yourself with `make bench` (Node.js) and `bun run bench:browser` (Chromium).
|
|
175
267
|
|
|
176
|
-
**
|
|
268
|
+
**Compression** (`compressText` for text, `compress` for binary), Node.js / Chromium:
|
|
177
269
|
|
|
178
270
|
| Payload | Quality 1 | Quality 6 | Quality 11 |
|
|
179
271
|
|---|---|---|---|
|
|
180
|
-
|
|
|
181
|
-
|
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
|
194
|
-
|
|
195
|
-
|
|
|
196
|
-
|
|
|
197
|
-
|
|
272
|
+
| Short JSON (83 B) | - | 55,883 / 39,120 | - |
|
|
273
|
+
| Markdown (2 KB) | 61,300 / 46,437 | 21,410 / 17,654 | 502 / 428 |
|
|
274
|
+
| HTML (28 KB) | 33,120 / 22,730 | 10,548 / 8,962 | 20.6 / 18.9 |
|
|
275
|
+
| Pseudorandom (512 B) | - | 43,034 / 38,924 | - |
|
|
276
|
+
| SVG (4.9 MB) | 43.0 / 39.7 | 6.79 / 6.15 | 0.26 / 0.28 |
|
|
277
|
+
| PNG (2.1 MB) | 1,505 / 1,795 | 93.1 / 139 | 0.46 / 0.67 |
|
|
278
|
+
|
|
279
|
+
**Decompression** (`decompressText` for text, `decompress` for binary):
|
|
280
|
+
|
|
281
|
+
| Payload | Node.js | Chromium |
|
|
282
|
+
|---|---|---|
|
|
283
|
+
| Short JSON (83 B) | 140,803 | 190,084 |
|
|
284
|
+
| Markdown (2 KB) | 68,431 | 88,380 |
|
|
285
|
+
| HTML (28 KB) | 44,058 | 54,016 |
|
|
286
|
+
| Pseudorandom (512 B) | 120,529 | 146,854 |
|
|
287
|
+
| SVG (4.9 MB) | 66.9 | 77.0 |
|
|
288
|
+
| PNG (2.1 MB) | 873 | 993 |
|
|
289
|
+
|
|
290
|
+
### Large payloads (100 MB)
|
|
291
|
+
|
|
292
|
+
`bench/large.mjs` decompresses two inputs with `maxOutputSize: 128 * 1024 * 1024` (above the 64 MiB default): 100 MiB of generated JSON log lines, stored as a 13.3 MB Brotli file (quality 11, window 24), and a 100 MB Brotli file of incompressible bytes. "Warm" is the median of 5 calls on an initialized instance; "first call" is the first decompression on a fresh instance, which also grows and pages in WASM memory, as a one-off decode does. Native is `node:zlib`, the C Brotli library. Measured on 2026-11-09 on an Apple M5 Max:
|
|
293
|
+
|
|
294
|
+
| | Node.js 26.11 | Bun 1.4.3 | Chromium 158 |
|
|
295
|
+
|---|---:|---:|---:|
|
|
296
|
+
| 100 MiB text, warm (native) | 190 ms (132 ms) | 226 ms (141 ms) | 200 ms |
|
|
297
|
+
| 100 MiB text, first call | 193 ms | 272 ms | 223 ms |
|
|
298
|
+
| 100 MB Brotli file, warm (native) | 49 ms (11 ms) | 66 ms (7 ms) | 58 ms |
|
|
299
|
+
| 100 MB Brotli file, first call | 49 ms | 83 ms | 64 ms |
|
|
300
|
+
| WASM memory after decoding: text / file | 19.6 / 19.3 MiB | 19.6 / 19.3 MiB | 19.6 / 19.3 MiB |
|
|
301
|
+
| Compress the 100 MiB text at quality 6 (native) | 2,069 ms (1,173 ms) | 2,201 ms (1,021 ms) | - |
|
|
302
|
+
| WASM memory after compressing | 69 MiB | 69 MiB | - |
|
|
303
|
+
| Load + `init()`: `decompressor` / `decompressor/slim` | 2.7 / 1.3 ms | 2.6 / 1.9 ms | 3.0 / 2.2 ms |
|
|
304
|
+
| Load + `init()`: `compressor` / `compressor/slim` | 7.6 / 1.8 ms | 4.7 / 3.1 ms | 5.2 / 2.2 ms |
|
|
305
|
+
|
|
306
|
+
- Warm text decoding produces 440 to 530 MiB/s of output. Timings vary between runs by up to a third (native decoding of the text took 132 to 174 ms on the same machine), so compare builds within one session: interleaved with the previous whole-buffer build in Node.js, warm text decoding took 162 to 199 ms against 179 to 211 ms, faster in each of three rounds, and the file and compression times did not differ beyond noise.
|
|
307
|
+
- The incompressible file is mostly copying: into WASM memory, through the decoder's window and back out, which native code does once.
|
|
308
|
+
- WASM memory holds the decoder's window (16 MiB for these window-24 streams), its tables and two 1 MiB chunks, independent of the payload size. The previous whole-buffer build needed 283 MiB for the text and 320 MiB for the file, and kept that memory for the life of the instance, because WASM memory never shrinks. The compressor needed 214 MiB before and 69 MiB now: its encoder state at quality 6 plus one 4 MiB window of input and a 1 MiB output chunk.
|
|
309
|
+
- Chromium offers JavaScript no native Brotli decoder (`DecompressionStream` rejects `"brotli"` in Chromium 158); browsers decode Brotli natively only as HTTP `Content-Encoding`.
|
|
310
|
+
- Load + `init()` is measured from local files; over a network, the download sizes above add their transfer time.
|
|
311
|
+
|
|
312
|
+
`make bench` runs the suite and fails when a result exceeds [`bench/perf-budgets.json`](bench/perf-budgets.json): 2 times these timings (at least +5 ms) and 10 % above these memory sizes. Timings depend on the machine, so `make bench` is not part of `make verify`; recalibrate the budgets on other hardware.
|
|
198
313
|
|
|
199
314
|
## Examples
|
|
200
315
|
|
|
@@ -212,117 +327,101 @@ Sample output:
|
|
|
212
327
|
════════════════════════════════════════════════════════════════════════
|
|
213
328
|
Quality │ Output │ Ratio │ Compress ms │ Decompress ms
|
|
214
329
|
------- │ ---------- │ ------- │ ----------- │ -------------
|
|
215
|
-
1 │ 87 B │ 104.8% │
|
|
216
|
-
4 │ 83 B │ 100.0% │ 0.
|
|
217
|
-
6 │ 80 B │ 96.4% │ 0.
|
|
218
|
-
9 │ 80 B │ 96.4% │
|
|
219
|
-
11 │ 85 B │ 102.4% │
|
|
330
|
+
1 │ 87 B │ 104.8% │ 0.43 │ 0.23 [OK]
|
|
331
|
+
4 │ 83 B │ 100.0% │ 0.36 │ 0.20 [OK]
|
|
332
|
+
6 │ 80 B │ 96.4% │ 0.36 │ 0.04 [OK]
|
|
333
|
+
9 │ 80 B │ 96.4% │ 1.88 │ 0.04 [OK]
|
|
334
|
+
11 │ 85 B │ 102.4% │ 2.31 │ 0.11 [OK]
|
|
220
335
|
|
|
221
336
|
════════════════════════════════════════════════════════════════════════
|
|
222
337
|
Pseudorandom binary (512 B) (512 B input)
|
|
223
338
|
════════════════════════════════════════════════════════════════════════
|
|
224
339
|
Quality │ Output │ Ratio │ Compress ms │ Decompress ms
|
|
225
340
|
------- │ ---------- │ ------- │ ----------- │ -------------
|
|
226
|
-
1 │ 286 B │ 55.9% │ 0.08 │
|
|
227
|
-
4 │ 278 B │ 54.3% │ 0.
|
|
228
|
-
6 │ 270 B │ 52.7% │ 0.
|
|
229
|
-
9 │ 259 B │ 50.6% │ 0.
|
|
230
|
-
11 │ 239 B │ 46.7% │
|
|
341
|
+
1 │ 286 B │ 55.9% │ 0.08 │ 1.22 [OK]
|
|
342
|
+
4 │ 278 B │ 54.3% │ 0.08 │ 0.04 [OK]
|
|
343
|
+
6 │ 270 B │ 52.7% │ 0.08 │ 0.03 [OK]
|
|
344
|
+
9 │ 259 B │ 50.6% │ 0.23 │ 0.03 [OK]
|
|
345
|
+
11 │ 239 B │ 46.7% │ 1.57 │ 0.15 [OK]
|
|
231
346
|
|
|
232
347
|
════════════════════════════════════════════════════════════════════════
|
|
233
348
|
SVG (4.9 MB) (4.9 MB input)
|
|
234
349
|
════════════════════════════════════════════════════════════════════════
|
|
235
350
|
Quality │ Output │ Ratio │ Compress ms │ Decompress ms
|
|
236
351
|
------- │ ---------- │ ------- │ ----------- │ -------------
|
|
237
|
-
1 │
|
|
238
|
-
4 │ 1.
|
|
239
|
-
6 │ 1.8 MB │
|
|
240
|
-
9 │ 1.8 MB │ 36.8% │
|
|
241
|
-
11 │ 1.7 MB │ 34.5% │
|
|
352
|
+
1 │ 1.9 MB │ 38.3% │ 25.21 │ 35.76 [OK]
|
|
353
|
+
4 │ 1.8 MB │ 36.6% │ 75.33 │ 33.47 [OK]
|
|
354
|
+
6 │ 1.8 MB │ 36.9% │ 158.53 │ 30.81 [OK]
|
|
355
|
+
9 │ 1.8 MB │ 36.8% │ 516.73 │ 28.96 [OK]
|
|
356
|
+
11 │ 1.7 MB │ 34.5% │ 3473.27 │ 38.55 [OK]
|
|
242
357
|
|
|
243
358
|
════════════════════════════════════════════════════════════════════════
|
|
244
359
|
PNG (2.1 MB) (2.1 MB input)
|
|
245
360
|
════════════════════════════════════════════════════════════════════════
|
|
246
361
|
Quality │ Output │ Ratio │ Compress ms │ Decompress ms
|
|
247
362
|
------- │ ---------- │ ------- │ ----------- │ -------------
|
|
248
|
-
1 │ 2.1 MB │ 100.
|
|
249
|
-
4 │ 2.1 MB │ 100.0% │
|
|
250
|
-
6 │ 2.1 MB │ 100.0% │
|
|
251
|
-
9 │ 2.0 MB │ 99.0% │
|
|
252
|
-
11 │ 2.0 MB │ 96.2% │
|
|
363
|
+
1 │ 2.1 MB │ 100.0% │ 0.68 │ 12.58 [OK]
|
|
364
|
+
4 │ 2.1 MB │ 100.0% │ 2.59 │ 9.11 [OK]
|
|
365
|
+
6 │ 2.1 MB │ 100.0% │ 9.63 │ 8.56 [OK]
|
|
366
|
+
9 │ 2.0 MB │ 99.0% │ 202.22 │ 13.13 [OK]
|
|
367
|
+
11 │ 2.0 MB │ 96.2% │ 1556.99 │ 24.54 [OK]
|
|
253
368
|
|
|
254
369
|
════════════════════════════════════════════════════════════════════════
|
|
255
|
-
All round-trips passed
|
|
370
|
+
[OK] All round-trips passed
|
|
256
371
|
════════════════════════════════════════════════════════════════════════
|
|
257
372
|
```
|
|
258
373
|
|
|
259
374
|
**Key takeaways:**
|
|
260
375
|
|
|
261
|
-
- **SVG** (text-based) compresses well:
|
|
262
|
-
- **PNG** (already compressed) barely shrinks:
|
|
263
|
-
- **Short text** can
|
|
376
|
+
- **SVG** (text-based) compresses well: the ratio falls from 38.3 % at quality 1 to 34.5 % at quality 11, while compression time grows from about 25 ms to about 3.5 s.
|
|
377
|
+
- **PNG** (already compressed) barely shrinks: about 100 % at qualities 1 to 6 and 96.2 % at quality 11, because its pixel data is already deflate-compressed.
|
|
378
|
+
- **Short text** can *grow* at low quality (104.8 % at quality 1): on an 83-byte payload the Brotli framing outweighs the savings.
|
|
264
379
|
|
|
265
380
|
## Defaults
|
|
266
381
|
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
- `quality = 6`
|
|
382
|
+
- `quality = 11`
|
|
270
383
|
- `lgwin = 22`
|
|
271
384
|
|
|
272
|
-
|
|
273
|
-
|
|
385
|
+
These are the defaults of Node.js zlib (`BROTLI_DEFAULT_QUALITY`, `BROTLI_DEFAULT_WINDOW`), and quality 11 gives the smallest output. Pass a lower `quality` when compression time matters more than size, for example when compressing each response as it is served.
|
|
386
|
+
|
|
387
|
+
The `defuss-brotli` command also uses quality 11 and, like the `brotli` CLI, the smallest window that holds the input.
|
|
274
388
|
|
|
275
389
|
## Build
|
|
276
390
|
|
|
277
391
|
### Prerequisites
|
|
278
392
|
|
|
279
|
-
|
|
280
|
-
- Rust toolchain + `wasm32-unknown-unknown` target
|
|
281
|
-
- `wasm-pack`
|
|
282
|
-
- `wasm-opt` (optional, from [binaryen](https://github.com/WebAssembly/binaryen) — produces smaller WASM)
|
|
283
|
-
|
|
284
|
-
Install everything automatically (cross-platform — installs Rust/wasm-pack if missing):
|
|
393
|
+
The build needs [mise](https://mise.jdx.dev/), which installs the pinned Bun, `wasm-pack` and `wasm-opt` (binaryen) from `mise.toml`, and rustup, which installs the Rust toolchain from `rust-toolchain.toml`. `make setup` installs both when missing, then the locked dependencies and the Playwright Chromium:
|
|
285
394
|
|
|
286
395
|
```bash
|
|
287
396
|
make setup
|
|
288
|
-
|
|
289
|
-
bun run setup
|
|
397
|
+
make build
|
|
290
398
|
```
|
|
291
399
|
|
|
292
|
-
|
|
400
|
+
On macOS, if Rust build scripts fail to link with `tapi error: malformed file ... libSystem.B.tbd`, the Command Line Tools linker is older than the newest SDK. Copy `.env.example` to `.env` and set `SDKROOT` to an older SDK under `/Library/Developer/CommandLineTools/SDKs/`.
|
|
293
401
|
|
|
294
|
-
|
|
295
|
-
bun install
|
|
296
|
-
bun run build
|
|
297
|
-
```
|
|
402
|
+
What happens during build, for each of the two crates:
|
|
298
403
|
|
|
299
|
-
|
|
404
|
+
1. `wasm-pack` builds the crate for `wasm32-unknown-unknown`.
|
|
405
|
+
2. `wasm-opt -O3` optimizes the binary.
|
|
406
|
+
3. The binary is written as a base64 module next to the wasm-bindgen glue.
|
|
407
|
+
4. `bun build` bundles `src/compressor.ts` or `src/decompressor.ts` with the glue and the base64 module into `dist/`.
|
|
300
408
|
|
|
301
|
-
|
|
302
|
-
2. `wasm-pack` builds the decompressor crate
|
|
303
|
-
3. `wasm-opt -Oz` runs if available
|
|
304
|
-
4. `bun build` bundles `src/compressor.ts` and `src/decompressor.ts` into `dist/`
|
|
305
|
-
5. `tsc` generates declaration files
|
|
409
|
+
`bun build` then bundles the command, `src/bin.ts`, into `dist/cli.js`, and `tsc` generates the declaration files.
|
|
306
410
|
|
|
307
411
|
## Test and Benchmark
|
|
308
412
|
|
|
309
|
-
Node:
|
|
310
|
-
|
|
311
413
|
```bash
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
bun
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
414
|
+
make test # Rust unit tests, then Vitest in Node and in headless Chromium
|
|
415
|
+
make coverage # Rust line coverage (fails below 60 %), then the TypeScript coverage table
|
|
416
|
+
make e2e # packs the tarball into clean projects: Node.js, Bun, the CLI via bunx, a bun build app and a Vite app (build + dev) in Chromium
|
|
417
|
+
make lint # oxlint, tsc, clippy, rustfmt
|
|
418
|
+
make verify # all of the above, as CI runs it
|
|
419
|
+
make bench # Vitest bench and the 100 MB benchmark in Node, Bun and Chromium, against the perf budgets
|
|
420
|
+
bun run example # quality comparison table
|
|
318
421
|
```
|
|
319
422
|
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
```bash
|
|
323
|
-
bun run test:browser
|
|
324
|
-
```
|
|
423
|
+
`make e2e` writes one result file per consumer to `output/` (`e2e-node.json`, `e2e-bun.json`, `e2e-cli.json`, `e2e-bun-bundle.json`, `e2e-vite-build.json`, `e2e-vite-dev.json`); `make e2e-runtime`, `make e2e-cli`, `make e2e-bun` and `make e2e-vite` run one group.
|
|
325
424
|
|
|
326
425
|
## License
|
|
327
426
|
|
|
328
|
-
MIT
|
|
427
|
+
MIT. The `.wasm` files and the inline modules contain compiled third-party code under BSD-3-Clause and MIT terms (brotli, brotli-decompressor, wasm-bindgen, the Rust standard library); [THIRD_PARTY_NOTICES.txt](THIRD_PARTY_NOTICES.txt) lists each component with its license text.
|