defuss-brotli 0.1.3 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +155 -140
- package/THIRD_PARTY_NOTICES.txt +163 -0
- package/dist/compressor-core.d.ts +104 -0
- package/dist/compressor-slim.d.ts +17 -0
- package/dist/compressor-slim.js +1 -0
- package/dist/compressor.d.ts +1 -96
- package/dist/compressor.js +1 -190
- package/dist/compressor.wasm +0 -0
- package/dist/decompressor-core.d.ts +93 -0
- package/dist/decompressor-slim.d.ts +17 -0
- package/dist/decompressor-slim.js +1 -0
- package/dist/decompressor.d.ts +1 -84
- package/dist/decompressor.js +1 -183
- package/dist/decompressor.wasm +0 -0
- package/dist/wasm.d.ts +24 -0
- package/package.json +22 -10
package/README.md
CHANGED
|
@@ -2,9 +2,10 @@
|
|
|
2
2
|
|
|
3
3
|
A split pure-Rust Brotli package for Node.js + browsers.
|
|
4
4
|
|
|
5
|
-
- **Small browser decoder
|
|
5
|
+
- **Small browser decoder** - `defuss-brotli/decompressor/slim` loads only the decoder: about 51 KB precompressed with Brotli quality 11 and 85 KB from jsDelivr (quality 4), most of it the Brotli dictionary every decoder carries
|
|
6
6
|
- **Separate heavier encoder bundle** - used server-side or at build time
|
|
7
7
|
- **Single NPM package** with split exports
|
|
8
|
+
- **Inlined or separate `.wasm`** - one self-contained file by default, or a `/slim` entry that loads the binary on its own
|
|
8
9
|
- **Pure-Rust WebAssembly** - no native addons
|
|
9
10
|
- **Typed TypeScript API** with full JSDoc
|
|
10
11
|
- **Vitest coverage** in Node and browser
|
|
@@ -35,9 +36,43 @@ import { init, decompressText } from "defuss-brotli/decompressor";
|
|
|
35
36
|
await init();
|
|
36
37
|
|
|
37
38
|
const text = decompressText(compressed);
|
|
38
|
-
//
|
|
39
|
+
// returns "Hello, Brotli!"
|
|
39
40
|
```
|
|
40
41
|
|
|
42
|
+
### Inlined or separate `.wasm`
|
|
43
|
+
|
|
44
|
+
Each module comes in two builds with the same API. The default entry points embed the WASM binary as base64, so one JavaScript file is all a runtime or bundler needs. The `/slim` entry points leave the binary in its own `.wasm` file, which HTTP caches independently of your JavaScript, which compiles while it downloads when served as `application/wasm`, and which transfers 21 to 45 % fewer bytes depending on module and transfer encoding (see [Download sizes and budgets](#download-sizes-and-budgets)).
|
|
45
|
+
|
|
46
|
+
Without an argument, a slim `init()` loads the `.wasm` file from next to its module, via `new URL("./decompressor.wasm", import.meta.url)`. Browsers fetch it from there, and Node.js 20.16+ / 22.3+ and Bun read it from disk:
|
|
47
|
+
|
|
48
|
+
```ts
|
|
49
|
+
import { init, decompressText } from "defuss-brotli/decompressor/slim";
|
|
50
|
+
|
|
51
|
+
await init();
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
When your bundler or server does not place the file next to the bundle, copy `defuss-brotli/decompressor.wasm` (or `compressor.wasm`) to where you serve static assets and pass its location, or pass the response, the bytes or a compiled module:
|
|
55
|
+
|
|
56
|
+
```ts
|
|
57
|
+
await init("/assets/decompressor.wasm"); // URL, fetched (streamed when served as application/wasm)
|
|
58
|
+
await init(fetch("https://cdn.example.com/d.wasm")); // Response or Promise<Response>
|
|
59
|
+
await init(await readFile(wasmPath)); // ArrayBuffer or typed array
|
|
60
|
+
await init(compiledModule); // WebAssembly.Module
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
`init()` loads once; later calls return the first Promise and ignore their argument. After a failed load (for example a 404), the next call tries again.
|
|
64
|
+
|
|
65
|
+
#### Bundlers
|
|
66
|
+
|
|
67
|
+
`make e2e` builds a clean app per bundler from the packed tarball and runs it in Chromium:
|
|
68
|
+
|
|
69
|
+
| Bundler | `init()` without an argument | Explicit asset import |
|
|
70
|
+
|---|---|---|
|
|
71
|
+
| Vite 8.3.6, `vite build` and `vite` dev | works: Vite resolves `new URL(..., import.meta.url)` and emits the `.wasm` | `import url from "defuss-brotli/decompressor.wasm?url"`, then `init(url)` |
|
|
72
|
+
| `bun build` (Bun 1.4) | copy `compressor.wasm` / `decompressor.wasm` next to the bundle yourself: Bun leaves `new URL(...)` unchanged | `import url from "defuss-brotli/decompressor.wasm" with { type: "file" }`, then `init(url)` |
|
|
73
|
+
|
|
74
|
+
With an explicit import, the bundler copies the file and gives it a hashed name, so it can be cached long-term. Node.js and Bun as runtimes need neither: `init()` reads the file from `node_modules`.
|
|
75
|
+
|
|
41
76
|
## API
|
|
42
77
|
|
|
43
78
|
### `compress` vs `compressText`
|
|
@@ -92,109 +127,104 @@ Hard cap on decompressed output to prevent **decompression bombs**. Raise explic
|
|
|
92
127
|
|
|
93
128
|
### Guarantees
|
|
94
129
|
|
|
95
|
-
- `compressText(s)`
|
|
96
|
-
- `compress(b)`
|
|
130
|
+
- `decompressText(compressText(s))` returns `s` exactly for any JS string.
|
|
131
|
+
- `decompress(compress(b))` returns the same bytes as `b` for any `Uint8Array`.
|
|
97
132
|
- `init()` is **idempotent** - calling it multiple times returns the same Promise and does not reload WASM.
|
|
98
|
-
- Importing only `defuss-brotli/decompressor` **never loads** the encoder WASM binary.
|
|
133
|
+
- Importing only `defuss-brotli/decompressor` (or `/decompressor/slim`) **never loads** the encoder WASM binary.
|
|
134
|
+
- Inline and `/slim` builds of a module run the same code and produce identical output.
|
|
99
135
|
|
|
100
136
|
### Limits
|
|
101
137
|
|
|
102
|
-
- `quality` must be an integer in **0-11** (
|
|
103
|
-
- `lgwin` must be an integer in **10-24** (Brotli
|
|
104
|
-
- `maxOutputSize` defaults to **64 MiB** - the WASM module rejects output exceeding this.
|
|
105
|
-
-
|
|
138
|
+
- `quality` must be an integer in **0-11** (the range of the reference Brotli encoder).
|
|
139
|
+
- `lgwin` must be an integer in **10-24** (the window range of the Brotli format, RFC 7932).
|
|
140
|
+
- `maxOutputSize` defaults to **64 MiB** - the WASM module rejects output exceeding this. Values above 2^32 − 1 act as 2^32 − 1.
|
|
141
|
+
- Calls stream the input and output through chunks of at most 1 MiB (one window of input when compressing), so WASM memory holds the codec state and two chunks however large the payload; the returned `Uint8Array` lives in JavaScript memory. Truncated input, corrupt input and bytes after the end of the Brotli stream throw.
|
|
106
142
|
|
|
107
143
|
## Why the Split Exports?
|
|
108
144
|
|
|
109
|
-
The encoder and decoder are built from **two separate Rust Wasm crates**:
|
|
110
|
-
|
|
111
|
-
| Export | Rust crate | WASM size | Bundle (JS + WASM) | Typical use |
|
|
112
|
-
|---|---|---|---|---|
|
|
113
|
-
| `defuss-brotli/compressor` | `brotli` | 984 KB | 1.3 MB | Server-side, build pipelines |
|
|
114
|
-
| `defuss-brotli/decompressor` | `brotli-decompressor` | 208 KB | 276 KB | Browser clients |
|
|
115
|
-
|
|
116
|
-
Browser apps that only decompress pre-compressed data import just the decompressor, keeping the bundle small.
|
|
117
|
-
|
|
118
|
-
## Benchmarks
|
|
119
|
-
|
|
120
|
-
Measured with `vitest bench` on the payloads shown. Numbers are **ops/sec** (higher is better).
|
|
121
|
-
|
|
122
|
-
Run benchmarks yourself:
|
|
123
|
-
|
|
124
|
-
```bash
|
|
125
|
-
bun run bench # Node.js
|
|
126
|
-
bun run bench:browser # Chromium via Playwright
|
|
127
|
-
```
|
|
128
|
-
|
|
129
|
-
### Node.js (Bun)
|
|
145
|
+
The encoder and decoder are built from **two separate Rust Wasm crates**: `brotli` behind `defuss-brotli/compressor` (server-side, build pipelines) and `brotli-decompressor` behind `defuss-brotli/decompressor` (browser clients). The encoder needs large static tables the decoder does not, so browser apps that only decompress pre-compressed data import just the decompressor and download about a sixth of the bytes. [ARCH.md](ARCH.md) explains where the sizes come from and why the build trades 1 % of size for speed.
|
|
130
146
|
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
| Payload | Quality 1 | Quality 6 | Quality 11 |
|
|
134
|
-
|---|---|---|---|
|
|
135
|
-
| Short JSON (83 B) | - | 37,239 | - |
|
|
136
|
-
| Markdown (2 KB) | 19,819 | 10,141 | 404 |
|
|
137
|
-
| HTML (8 KB) | 5,191 | 2,704 | 18 |
|
|
147
|
+
## Download sizes and budgets
|
|
138
148
|
|
|
139
|
-
|
|
149
|
+
What a browser downloads for one import, with each file transfer-compressed on its own, as a CDN or web server does per response:
|
|
140
150
|
|
|
141
|
-
|
|
|
142
|
-
|
|
143
|
-
|
|
|
144
|
-
|
|
|
145
|
-
|
|
|
151
|
+
| Import | Files | raw | gzip -9 | brotli q4 (jsDelivr) | brotli q11 | zstd -19 |
|
|
152
|
+
|---|---|---:|---:|---:|---:|---:|
|
|
153
|
+
| `defuss-brotli/decompressor/slim` | `decompressor-slim.js` + `decompressor.wasm` | 190,260 B | 85,609 B | 85,064 B | 50,801 B | 78,590 B |
|
|
154
|
+
| `defuss-brotli/decompressor` | `decompressor.js` | 252,184 B | 109,982 B | 111,509 B | 92,957 B | 101,421 B |
|
|
155
|
+
| `defuss-brotli/compressor/slim` | `compressor-slim.js` + `compressor.wasm` | 938,084 B | 447,231 B | 503,917 B | 394,522 B | 400,635 B |
|
|
156
|
+
| `defuss-brotli/compressor` | `compressor.js` | 1,249,264 B | 599,668 B | 655,950 B | 500,472 B | 520,192 B |
|
|
146
157
|
|
|
147
|
-
|
|
148
|
-
|
|
158
|
+
- **gzip -9** and **brotli q4** are what [jsDelivr](https://www.jsdelivr.com/) sends: measured on 2026-11-09, its brotli responses match quality 4 byte for byte and its gzip responses lie within 0.1 % of levels 6 to 9. jsDelivr serves `.wasm` files as `application/wasm` with `access-control-allow-origin: *`, and serves no zstd.
|
|
159
|
+
- **brotli q11** and **zstd -19** are what precompressed static hosting reaches (for example `.br` files next to the assets, or a CDN that compresses at its highest level). Quality 11 is unstable on `compressor.wasm`: passes of `wasm-opt` that change its raw size and its quality-4 size by 0.2 % move it between 340,693 B and 392,638 B, so its budget allows the upper value.
|
|
160
|
+
- `make metrics` prints these numbers for `dist/`. `make e2e` measures the packed tarball and fails when an import exceeds its limit in [`size-budgets.json`](size-budgets.json) (current size plus 2 %); raise a limit only for a measured, intended change.
|
|
149
161
|
|
|
150
|
-
|
|
162
|
+
### From a CDN without a bundler
|
|
151
163
|
|
|
152
|
-
|
|
153
|
-
|---|---|
|
|
154
|
-
| Short JSON (83 B) | 141,241 |
|
|
155
|
-
| Markdown (2 KB) | 47,757 |
|
|
156
|
-
| HTML (8 KB) | 37,288 |
|
|
164
|
+
An [import map](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script/type/importmap) maps the package's exports to its files on jsDelivr; pin a version (`defuss-brotli@x.y.z`) in production. The slim `init()` then fetches `decompressor.wasm` from the same CDN directory:
|
|
157
165
|
|
|
158
|
-
|
|
166
|
+
```html
|
|
167
|
+
<script type="importmap">
|
|
168
|
+
{ "imports": { "defuss-brotli/decompressor/slim": "https://cdn.jsdelivr.net/npm/defuss-brotli/dist/decompressor-slim.js" } }
|
|
169
|
+
</script>
|
|
170
|
+
<script type="module">
|
|
171
|
+
import { init, decompressText } from "defuss-brotli/decompressor/slim";
|
|
159
172
|
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
| SVG (4.9 MB) | 30.7 |
|
|
164
|
-
| PNG (2.1 MB) | 699 |
|
|
173
|
+
await init();
|
|
174
|
+
</script>
|
|
175
|
+
```
|
|
165
176
|
|
|
166
|
-
|
|
177
|
+
`make e2e` runs this setup in Chromium against a second origin that serves the installed package with jsDelivr's headers.
|
|
167
178
|
|
|
168
|
-
|
|
179
|
+
## Benchmarks
|
|
169
180
|
|
|
170
|
-
|
|
171
|
-
|---|---|---|---|
|
|
172
|
-
| Short JSON (83 B) | - | 36,264 | - |
|
|
173
|
-
| Markdown (2 KB) | 36,002 | 12,932 | 408 |
|
|
174
|
-
| HTML (8 KB) | 8,222 | 2,535 | 19 |
|
|
181
|
+
Numbers are operations per second from `vitest bench` (higher is better), measured on 2026-11-08 on an Apple Silicon Mac with Node.js 26.11 and headless Chromium 158. Run them yourself with `make bench` (Node.js) and `bun run bench:browser` (Chromium).
|
|
175
182
|
|
|
176
|
-
**
|
|
183
|
+
**Compression** (`compressText` for text, `compress` for binary), Node.js / Chromium:
|
|
177
184
|
|
|
178
185
|
| Payload | Quality 1 | Quality 6 | Quality 11 |
|
|
179
186
|
|---|---|---|---|
|
|
180
|
-
|
|
|
181
|
-
|
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
|
194
|
-
|
|
195
|
-
|
|
|
196
|
-
|
|
|
197
|
-
|
|
187
|
+
| Short JSON (83 B) | - | 55,883 / 39,120 | - |
|
|
188
|
+
| Markdown (2 KB) | 61,300 / 46,437 | 21,410 / 17,654 | 502 / 428 |
|
|
189
|
+
| HTML (28 KB) | 33,120 / 22,730 | 10,548 / 8,962 | 20.6 / 18.9 |
|
|
190
|
+
| Pseudorandom (512 B) | - | 43,034 / 38,924 | - |
|
|
191
|
+
| SVG (4.9 MB) | 43.0 / 39.7 | 6.79 / 6.15 | 0.26 / 0.28 |
|
|
192
|
+
| PNG (2.1 MB) | 1,505 / 1,795 | 93.1 / 139 | 0.46 / 0.67 |
|
|
193
|
+
|
|
194
|
+
**Decompression** (`decompressText` for text, `decompress` for binary):
|
|
195
|
+
|
|
196
|
+
| Payload | Node.js | Chromium |
|
|
197
|
+
|---|---|---|
|
|
198
|
+
| Short JSON (83 B) | 140,803 | 190,084 |
|
|
199
|
+
| Markdown (2 KB) | 68,431 | 88,380 |
|
|
200
|
+
| HTML (28 KB) | 44,058 | 54,016 |
|
|
201
|
+
| Pseudorandom (512 B) | 120,529 | 146,854 |
|
|
202
|
+
| SVG (4.9 MB) | 66.9 | 77.0 |
|
|
203
|
+
| PNG (2.1 MB) | 873 | 993 |
|
|
204
|
+
|
|
205
|
+
### Large payloads (100 MB)
|
|
206
|
+
|
|
207
|
+
`bench/large.mjs` decompresses two inputs with `maxOutputSize: 128 * 1024 * 1024` (above the 64 MiB default): 100 MiB of generated JSON log lines, stored as a 13.3 MB Brotli file (quality 11, window 24), and a 100 MB Brotli file of incompressible bytes. "Warm" is the median of 5 calls on an initialized instance; "first call" is the first decompression on a fresh instance, which also grows and pages in WASM memory, as a one-off decode does. Native is `node:zlib`, the C Brotli library. Measured on 2026-11-09 on an Apple M5 Max:
|
|
208
|
+
|
|
209
|
+
| | Node.js 26.11 | Bun 1.4.3 | Chromium 158 |
|
|
210
|
+
|---|---:|---:|---:|
|
|
211
|
+
| 100 MiB text, warm (native) | 190 ms (132 ms) | 226 ms (141 ms) | 200 ms |
|
|
212
|
+
| 100 MiB text, first call | 193 ms | 272 ms | 223 ms |
|
|
213
|
+
| 100 MB Brotli file, warm (native) | 49 ms (11 ms) | 66 ms (7 ms) | 58 ms |
|
|
214
|
+
| 100 MB Brotli file, first call | 49 ms | 83 ms | 64 ms |
|
|
215
|
+
| WASM memory after decoding: text / file | 19.6 / 19.3 MiB | 19.6 / 19.3 MiB | 19.6 / 19.3 MiB |
|
|
216
|
+
| Compress the 100 MiB text at quality 6 (native) | 2,069 ms (1,173 ms) | 2,201 ms (1,021 ms) | - |
|
|
217
|
+
| WASM memory after compressing | 69 MiB | 69 MiB | - |
|
|
218
|
+
| Load + `init()`: `decompressor` / `decompressor/slim` | 2.7 / 1.3 ms | 2.6 / 1.9 ms | 3.0 / 2.2 ms |
|
|
219
|
+
| Load + `init()`: `compressor` / `compressor/slim` | 7.6 / 1.8 ms | 4.7 / 3.1 ms | 5.2 / 2.2 ms |
|
|
220
|
+
|
|
221
|
+
- Warm text decoding produces 440 to 530 MiB/s of output. Timings vary between runs by up to a third (native decoding of the text took 132 to 174 ms on the same machine), so compare builds within one session: interleaved with the previous whole-buffer build in Node.js, warm text decoding took 162 to 199 ms against 179 to 211 ms, faster in each of three rounds, and the file and compression times did not differ beyond noise.
|
|
222
|
+
- The incompressible file is mostly copying: into WASM memory, through the decoder's window and back out, which native code does once.
|
|
223
|
+
- WASM memory holds the decoder's window (16 MiB for these window-24 streams), its tables and two 1 MiB chunks, independent of the payload size. The previous whole-buffer build needed 283 MiB for the text and 320 MiB for the file, and kept that memory for the life of the instance, because WASM memory never shrinks. The compressor needed 214 MiB before and 69 MiB now: its encoder state at quality 6 plus one 4 MiB window of input and a 1 MiB output chunk.
|
|
224
|
+
- Chromium offers JavaScript no native Brotli decoder (`DecompressionStream` rejects `"brotli"` in Chromium 158); browsers decode Brotli natively only as HTTP `Content-Encoding`.
|
|
225
|
+
- Load + `init()` is measured from local files; over a network, the download sizes above add their transfer time.
|
|
226
|
+
|
|
227
|
+
`make bench` runs the suite and fails when a result exceeds [`bench/perf-budgets.json`](bench/perf-budgets.json): 2 times these timings (at least +5 ms) and 10 % above these memory sizes. Timings depend on the machine, so `make bench` is not part of `make verify`; recalibrate the budgets on other hardware.
|
|
198
228
|
|
|
199
229
|
## Examples
|
|
200
230
|
|
|
@@ -212,55 +242,55 @@ Sample output:
|
|
|
212
242
|
════════════════════════════════════════════════════════════════════════
|
|
213
243
|
Quality │ Output │ Ratio │ Compress ms │ Decompress ms
|
|
214
244
|
------- │ ---------- │ ------- │ ----------- │ -------------
|
|
215
|
-
1 │ 87 B │ 104.8% │
|
|
216
|
-
4 │ 83 B │ 100.0% │ 0.
|
|
217
|
-
6 │ 80 B │ 96.4% │ 0.
|
|
218
|
-
9 │ 80 B │ 96.4% │
|
|
219
|
-
11 │ 85 B │ 102.4% │
|
|
245
|
+
1 │ 87 B │ 104.8% │ 0.43 │ 0.23 [OK]
|
|
246
|
+
4 │ 83 B │ 100.0% │ 0.36 │ 0.20 [OK]
|
|
247
|
+
6 │ 80 B │ 96.4% │ 0.36 │ 0.04 [OK]
|
|
248
|
+
9 │ 80 B │ 96.4% │ 1.88 │ 0.04 [OK]
|
|
249
|
+
11 │ 85 B │ 102.4% │ 2.31 │ 0.11 [OK]
|
|
220
250
|
|
|
221
251
|
════════════════════════════════════════════════════════════════════════
|
|
222
252
|
Pseudorandom binary (512 B) (512 B input)
|
|
223
253
|
════════════════════════════════════════════════════════════════════════
|
|
224
254
|
Quality │ Output │ Ratio │ Compress ms │ Decompress ms
|
|
225
255
|
------- │ ---------- │ ------- │ ----------- │ -------------
|
|
226
|
-
1 │ 286 B │ 55.9% │ 0.08 │
|
|
227
|
-
4 │ 278 B │ 54.3% │ 0.
|
|
228
|
-
6 │ 270 B │ 52.7% │ 0.
|
|
229
|
-
9 │ 259 B │ 50.6% │ 0.
|
|
230
|
-
11 │ 239 B │ 46.7% │
|
|
256
|
+
1 │ 286 B │ 55.9% │ 0.08 │ 1.22 [OK]
|
|
257
|
+
4 │ 278 B │ 54.3% │ 0.08 │ 0.04 [OK]
|
|
258
|
+
6 │ 270 B │ 52.7% │ 0.08 │ 0.03 [OK]
|
|
259
|
+
9 │ 259 B │ 50.6% │ 0.23 │ 0.03 [OK]
|
|
260
|
+
11 │ 239 B │ 46.7% │ 1.57 │ 0.15 [OK]
|
|
231
261
|
|
|
232
262
|
════════════════════════════════════════════════════════════════════════
|
|
233
263
|
SVG (4.9 MB) (4.9 MB input)
|
|
234
264
|
════════════════════════════════════════════════════════════════════════
|
|
235
265
|
Quality │ Output │ Ratio │ Compress ms │ Decompress ms
|
|
236
266
|
------- │ ---------- │ ------- │ ----------- │ -------------
|
|
237
|
-
1 │
|
|
238
|
-
4 │ 1.
|
|
239
|
-
6 │ 1.8 MB │
|
|
240
|
-
9 │ 1.8 MB │ 36.8% │
|
|
241
|
-
11 │ 1.7 MB │ 34.5% │
|
|
267
|
+
1 │ 1.9 MB │ 38.3% │ 25.21 │ 35.76 [OK]
|
|
268
|
+
4 │ 1.8 MB │ 36.6% │ 75.33 │ 33.47 [OK]
|
|
269
|
+
6 │ 1.8 MB │ 36.9% │ 158.53 │ 30.81 [OK]
|
|
270
|
+
9 │ 1.8 MB │ 36.8% │ 516.73 │ 28.96 [OK]
|
|
271
|
+
11 │ 1.7 MB │ 34.5% │ 3473.27 │ 38.55 [OK]
|
|
242
272
|
|
|
243
273
|
════════════════════════════════════════════════════════════════════════
|
|
244
274
|
PNG (2.1 MB) (2.1 MB input)
|
|
245
275
|
════════════════════════════════════════════════════════════════════════
|
|
246
276
|
Quality │ Output │ Ratio │ Compress ms │ Decompress ms
|
|
247
277
|
------- │ ---------- │ ------- │ ----------- │ -------------
|
|
248
|
-
1 │ 2.1 MB │ 100.
|
|
249
|
-
4 │ 2.1 MB │ 100.0% │
|
|
250
|
-
6 │ 2.1 MB │ 100.0% │
|
|
251
|
-
9 │ 2.0 MB │ 99.0% │
|
|
252
|
-
11 │ 2.0 MB │ 96.2% │
|
|
278
|
+
1 │ 2.1 MB │ 100.0% │ 0.68 │ 12.58 [OK]
|
|
279
|
+
4 │ 2.1 MB │ 100.0% │ 2.59 │ 9.11 [OK]
|
|
280
|
+
6 │ 2.1 MB │ 100.0% │ 9.63 │ 8.56 [OK]
|
|
281
|
+
9 │ 2.0 MB │ 99.0% │ 202.22 │ 13.13 [OK]
|
|
282
|
+
11 │ 2.0 MB │ 96.2% │ 1556.99 │ 24.54 [OK]
|
|
253
283
|
|
|
254
284
|
════════════════════════════════════════════════════════════════════════
|
|
255
|
-
All round-trips passed
|
|
285
|
+
[OK] All round-trips passed
|
|
256
286
|
════════════════════════════════════════════════════════════════════════
|
|
257
287
|
```
|
|
258
288
|
|
|
259
289
|
**Key takeaways:**
|
|
260
290
|
|
|
261
|
-
- **SVG** (text-based) compresses well:
|
|
262
|
-
- **PNG** (already compressed) barely shrinks:
|
|
263
|
-
- **Short text** can
|
|
291
|
+
- **SVG** (text-based) compresses well: the ratio falls from 38.3 % at quality 1 to 34.5 % at quality 11, while compression time grows from about 25 ms to about 3.5 s.
|
|
292
|
+
- **PNG** (already compressed) barely shrinks: about 100 % at qualities 1 to 6 and 96.2 % at quality 11, because its pixel data is already deflate-compressed.
|
|
293
|
+
- **Short text** can *grow* at low quality (104.8 % at quality 1): on an 83-byte payload the Brotli framing outweighs the savings.
|
|
264
294
|
|
|
265
295
|
## Defaults
|
|
266
296
|
|
|
@@ -276,53 +306,38 @@ need smaller output and can afford more CPU time.
|
|
|
276
306
|
|
|
277
307
|
### Prerequisites
|
|
278
308
|
|
|
279
|
-
|
|
280
|
-
- Rust toolchain + `wasm32-unknown-unknown` target
|
|
281
|
-
- `wasm-pack`
|
|
282
|
-
- `wasm-opt` (optional, from [binaryen](https://github.com/WebAssembly/binaryen) — produces smaller WASM)
|
|
283
|
-
|
|
284
|
-
Install everything automatically (cross-platform — installs Rust/wasm-pack if missing):
|
|
309
|
+
The build needs [mise](https://mise.jdx.dev/), which installs the pinned Bun, `wasm-pack` and `wasm-opt` (binaryen) from `mise.toml`, and rustup, which installs the Rust toolchain from `rust-toolchain.toml`. `make setup` installs both when missing, then the locked dependencies and the Playwright Chromium:
|
|
285
310
|
|
|
286
311
|
```bash
|
|
287
312
|
make setup
|
|
288
|
-
|
|
289
|
-
bun run setup
|
|
313
|
+
make build
|
|
290
314
|
```
|
|
291
315
|
|
|
292
|
-
|
|
316
|
+
On macOS, if Rust build scripts fail to link with `tapi error: malformed file ... libSystem.B.tbd`, the Command Line Tools linker is older than the newest SDK. Copy `.env.example` to `.env` and set `SDKROOT` to an older SDK under `/Library/Developer/CommandLineTools/SDKs/`.
|
|
293
317
|
|
|
294
|
-
|
|
295
|
-
bun install
|
|
296
|
-
bun run build
|
|
297
|
-
```
|
|
318
|
+
What happens during build, for each of the two crates:
|
|
298
319
|
|
|
299
|
-
|
|
320
|
+
1. `wasm-pack` builds the crate for `wasm32-unknown-unknown`.
|
|
321
|
+
2. `wasm-opt -O3` optimizes the binary.
|
|
322
|
+
3. The binary is written as a base64 module next to the wasm-bindgen glue.
|
|
323
|
+
4. `bun build` bundles `src/compressor.ts` or `src/decompressor.ts` with the glue and the base64 module into `dist/`.
|
|
300
324
|
|
|
301
|
-
|
|
302
|
-
2. `wasm-pack` builds the decompressor crate
|
|
303
|
-
3. `wasm-opt -Oz` runs if available
|
|
304
|
-
4. `bun build` bundles `src/compressor.ts` and `src/decompressor.ts` into `dist/`
|
|
305
|
-
5. `tsc` generates declaration files
|
|
325
|
+
`tsc` then generates the declaration files.
|
|
306
326
|
|
|
307
327
|
## Test and Benchmark
|
|
308
328
|
|
|
309
|
-
Node:
|
|
310
|
-
|
|
311
329
|
```bash
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
bun
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
330
|
+
make test # Rust unit tests, then Vitest in Node and in headless Chromium
|
|
331
|
+
make coverage # Rust line coverage (fails below 60 %), then the TypeScript coverage table
|
|
332
|
+
make e2e # packs the tarball into clean projects: Node.js, Bun, a bun build app and a Vite app (build + dev) in Chromium
|
|
333
|
+
make lint # oxlint, tsc, clippy, rustfmt
|
|
334
|
+
make verify # all of the above, as CI runs it
|
|
335
|
+
make bench # Vitest bench and the 100 MB benchmark in Node, Bun and Chromium, against the perf budgets
|
|
336
|
+
bun run example # quality comparison table
|
|
318
337
|
```
|
|
319
338
|
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
```bash
|
|
323
|
-
bun run test:browser
|
|
324
|
-
```
|
|
339
|
+
`make e2e` writes one result file per consumer to `output/` (`e2e-node.json`, `e2e-bun.json`, `e2e-bun-bundle.json`, `e2e-vite-build.json`, `e2e-vite-dev.json`); `make e2e-runtime`, `make e2e-bun` and `make e2e-vite` run one group.
|
|
325
340
|
|
|
326
341
|
## License
|
|
327
342
|
|
|
328
|
-
MIT
|
|
343
|
+
MIT. The `.wasm` files and the inline modules contain compiled third-party code under BSD-3-Clause and MIT terms (brotli, brotli-decompressor, wasm-bindgen, the Rust standard library); [THIRD_PARTY_NOTICES.txt](THIRD_PARTY_NOTICES.txt) lists each component with its license text.
|
|
@@ -0,0 +1,163 @@
|
|
|
1
|
+
defuss-brotli: third-party notices
|
|
2
|
+
==============================================================================
|
|
3
|
+
|
|
4
|
+
dist/compressor.wasm and dist/decompressor.wasm, also embedded as base64 in
|
|
5
|
+
dist/compressor.js and dist/decompressor.js, contain code compiled from the
|
|
6
|
+
components below; the JavaScript glue in dist/*.js is generated by
|
|
7
|
+
wasm-bindgen. Crates that run only at build time (procedural macros:
|
|
8
|
+
wasm-bindgen-macro, wasm-bindgen-macro-support, syn, quote, proc-macro2,
|
|
9
|
+
unicode-ident, bumpalo) are not part of the shipped files.
|
|
10
|
+
|
|
11
|
+
Component License Used in
|
|
12
|
+
-------------------------------- ------------------------------ ------------
|
|
13
|
+
brotli 8.0.2 BSD-3-Clause AND MIT compressor
|
|
14
|
+
brotli-decompressor 5.0.0 BSD-3-Clause OR MIT both
|
|
15
|
+
alloc-no-stdlib 2.0.4 BSD-3-Clause both
|
|
16
|
+
alloc-stdlib 0.2.2 BSD-3-Clause compressor
|
|
17
|
+
wasm-bindgen 0.2.114 MIT OR Apache-2.0 both
|
|
18
|
+
wasm-bindgen-shared 0.2.114 MIT OR Apache-2.0 both
|
|
19
|
+
once_cell 1.21.3 MIT OR Apache-2.0 both
|
|
20
|
+
cfg-if 1.0.4 MIT OR Apache-2.0 both
|
|
21
|
+
Rust standard library (rustc Apache-2.0 OR MIT, with the both
|
|
22
|
+
1.99.1): core, alloc; std in exceptions listed in its
|
|
23
|
+
the compressor COPYRIGHT-library notice
|
|
24
|
+
|
|
25
|
+
Where a component offers a choice of licenses, it is used under BSD-3-Clause
|
|
26
|
+
(brotli-decompressor) or MIT (all others).
|
|
27
|
+
|
|
28
|
+
|
|
29
|
+
==============================================================================
|
|
30
|
+
BSD-3-Clause: brotli (its BSD-3-Clause part), brotli-decompressor,
|
|
31
|
+
alloc-no-stdlib, alloc-stdlib
|
|
32
|
+
==============================================================================
|
|
33
|
+
|
|
34
|
+
Copyright (c) 2016 Dropbox, Inc.
|
|
35
|
+
All rights reserved.
|
|
36
|
+
|
|
37
|
+
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
|
|
38
|
+
|
|
39
|
+
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
|
|
40
|
+
|
|
41
|
+
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
|
|
42
|
+
|
|
43
|
+
3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
|
|
44
|
+
|
|
45
|
+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
|
46
|
+
|
|
47
|
+
|
|
48
|
+
==============================================================================
|
|
49
|
+
MIT: brotli (its MIT part, including the Brotli dictionary)
|
|
50
|
+
==============================================================================
|
|
51
|
+
|
|
52
|
+
Copyright (c) 2009, 2010, 2013-2016 by the Brotli Authors.
|
|
53
|
+
|
|
54
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
55
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
56
|
+
in the Software without restriction, including without limitation the rights
|
|
57
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
58
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
59
|
+
furnished to do so, subject to the following conditions:
|
|
60
|
+
|
|
61
|
+
The above copyright notice and this permission notice shall be included in
|
|
62
|
+
all copies or substantial portions of the Software.
|
|
63
|
+
|
|
64
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
65
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
66
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
67
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
68
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
69
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
|
70
|
+
THE SOFTWARE.
|
|
71
|
+
|
|
72
|
+
|
|
73
|
+
==============================================================================
|
|
74
|
+
MIT: wasm-bindgen, wasm-bindgen-shared, cfg-if
|
|
75
|
+
==============================================================================
|
|
76
|
+
|
|
77
|
+
Copyright (c) 2014 Alex Crichton
|
|
78
|
+
|
|
79
|
+
Permission is hereby granted, free of charge, to any
|
|
80
|
+
person obtaining a copy of this software and associated
|
|
81
|
+
documentation files (the "Software"), to deal in the
|
|
82
|
+
Software without restriction, including without
|
|
83
|
+
limitation the rights to use, copy, modify, merge,
|
|
84
|
+
publish, distribute, sublicense, and/or sell copies of
|
|
85
|
+
the Software, and to permit persons to whom the Software
|
|
86
|
+
is furnished to do so, subject to the following
|
|
87
|
+
conditions:
|
|
88
|
+
|
|
89
|
+
The above copyright notice and this permission notice
|
|
90
|
+
shall be included in all copies or substantial portions
|
|
91
|
+
of the Software.
|
|
92
|
+
|
|
93
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
|
|
94
|
+
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
|
|
95
|
+
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
|
|
96
|
+
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
|
|
97
|
+
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
|
98
|
+
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
|
99
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
|
|
100
|
+
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
|
|
101
|
+
DEALINGS IN THE SOFTWARE.
|
|
102
|
+
|
|
103
|
+
|
|
104
|
+
==============================================================================
|
|
105
|
+
MIT: once_cell (Aleksey Kladov)
|
|
106
|
+
==============================================================================
|
|
107
|
+
|
|
108
|
+
Permission is hereby granted, free of charge, to any
|
|
109
|
+
person obtaining a copy of this software and associated
|
|
110
|
+
documentation files (the "Software"), to deal in the
|
|
111
|
+
Software without restriction, including without
|
|
112
|
+
limitation the rights to use, copy, modify, merge,
|
|
113
|
+
publish, distribute, sublicense, and/or sell copies of
|
|
114
|
+
the Software, and to permit persons to whom the Software
|
|
115
|
+
is furnished to do so, subject to the following
|
|
116
|
+
conditions:
|
|
117
|
+
|
|
118
|
+
The above copyright notice and this permission notice
|
|
119
|
+
shall be included in all copies or substantial portions
|
|
120
|
+
of the Software.
|
|
121
|
+
|
|
122
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
|
|
123
|
+
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
|
|
124
|
+
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
|
|
125
|
+
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
|
|
126
|
+
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
|
127
|
+
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
|
128
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
|
|
129
|
+
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
|
|
130
|
+
DEALINGS IN THE SOFTWARE.
|
|
131
|
+
|
|
132
|
+
|
|
133
|
+
==============================================================================
|
|
134
|
+
MIT: Rust standard library
|
|
135
|
+
Copyright: The Rust Project Developers (see https://thanks.rust-lang.org)
|
|
136
|
+
Per-file exceptions and the standard library's own dependencies:
|
|
137
|
+
https://github.com/rust-lang/rust/blob/1.99.1/COPYRIGHT-library.html
|
|
138
|
+
(also installed with the toolchain as share/doc/rust/COPYRIGHT-library.html)
|
|
139
|
+
==============================================================================
|
|
140
|
+
|
|
141
|
+
Permission is hereby granted, free of charge, to any
|
|
142
|
+
person obtaining a copy of this software and associated
|
|
143
|
+
documentation files (the "Software"), to deal in the
|
|
144
|
+
Software without restriction, including without
|
|
145
|
+
limitation the rights to use, copy, modify, merge,
|
|
146
|
+
publish, distribute, sublicense, and/or sell copies of
|
|
147
|
+
the Software, and to permit persons to whom the Software
|
|
148
|
+
is furnished to do so, subject to the following
|
|
149
|
+
conditions:
|
|
150
|
+
|
|
151
|
+
The above copyright notice and this permission notice
|
|
152
|
+
shall be included in all copies or substantial portions
|
|
153
|
+
of the Software.
|
|
154
|
+
|
|
155
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
|
|
156
|
+
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
|
|
157
|
+
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
|
|
158
|
+
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
|
|
159
|
+
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
|
160
|
+
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
|
161
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
|
|
162
|
+
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
|
|
163
|
+
DEALINGS IN THE SOFTWARE.
|