node-liblzma 2.1.0 → 3.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +431 -727
- package/index.d.ts +34 -0
- package/lib/cli/nxz.d.ts +7 -0
- package/lib/cli/nxz.d.ts.map +1 -0
- package/lib/cli/nxz.js +578 -0
- package/lib/cli/nxz.js.map +1 -0
- package/lib/lzma.browser.d.ts +24 -0
- package/lib/lzma.browser.d.ts.map +1 -0
- package/lib/lzma.browser.js +30 -0
- package/lib/lzma.browser.js.map +1 -0
- package/lib/lzma.d.ts +83 -0
- package/lib/lzma.d.ts.map +1 -1
- package/lib/lzma.inline.d.ts +30 -0
- package/lib/lzma.inline.d.ts.map +1 -0
- package/lib/lzma.inline.js +68 -0
- package/lib/lzma.inline.js.map +1 -0
- package/lib/lzma.js +78 -0
- package/lib/lzma.js.map +1 -1
- package/lib/wasm/bindings.d.ts +109 -0
- package/lib/wasm/bindings.d.ts.map +1 -0
- package/lib/wasm/bindings.js +307 -0
- package/lib/wasm/bindings.js.map +1 -0
- package/lib/wasm/compress.d.ts +32 -0
- package/lib/wasm/compress.d.ts.map +1 -0
- package/lib/wasm/compress.js +47 -0
- package/lib/wasm/compress.js.map +1 -0
- package/lib/wasm/decompress.d.ts +32 -0
- package/lib/wasm/decompress.d.ts.map +1 -0
- package/lib/wasm/decompress.js +45 -0
- package/lib/wasm/decompress.js.map +1 -0
- package/lib/wasm/index.d.ts +14 -0
- package/lib/wasm/index.d.ts.map +1 -0
- package/lib/wasm/index.js +18 -0
- package/lib/wasm/index.js.map +1 -0
- package/lib/wasm/liblzma.inline.d.ts +10 -0
- package/lib/wasm/liblzma.inline.d.ts.map +1 -0
- package/lib/wasm/liblzma.inline.js +10 -0
- package/lib/wasm/liblzma.inline.js.map +1 -0
- package/lib/wasm/memory.d.ts +57 -0
- package/lib/wasm/memory.d.ts.map +1 -0
- package/lib/wasm/memory.js +108 -0
- package/lib/wasm/memory.js.map +1 -0
- package/lib/wasm/stream.d.ts +35 -0
- package/lib/wasm/stream.d.ts.map +1 -0
- package/lib/wasm/stream.js +164 -0
- package/lib/wasm/stream.js.map +1 -0
- package/lib/wasm/types.d.ts +77 -0
- package/lib/wasm/types.d.ts.map +1 -0
- package/lib/wasm/types.js +55 -0
- package/lib/wasm/types.js.map +1 -0
- package/lib/wasm/utils.d.ts +62 -0
- package/lib/wasm/utils.d.ts.map +1 -0
- package/lib/wasm/utils.js +162 -0
- package/lib/wasm/utils.js.map +1 -0
- package/package.json +27 -3
- package/src/bindings/module.cpp +196 -0
package/README.md
CHANGED
|
@@ -3,34 +3,50 @@ Node-liblzma
|
|
|
3
3
|
|
|
4
4
|
[](https://npmjs.org/package/node-liblzma)
|
|
5
5
|
[](https://npmjs.org/package/node-liblzma)
|
|
6
|
-
[](https://github.com/oorabona/node-liblzma/actions/workflows/ci.yml)
|
|
7
7
|
[](https://oorabona.github.io/node-liblzma/)
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
[
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
8
|
+
[](https://github.com/oorabona/node-liblzma/blob/master/LICENSE)
|
|
9
|
+
[](https://nodejs.org)
|
|
10
|
+
[](https://www.typescriptlang.org/)
|
|
11
|
+
[](https://docs.npmjs.com/generating-provenance-statements)
|
|
12
|
+
|
|
13
|
+
Native Node.js bindings for liblzma — XZ/LZMA2 compression with **browser support via WebAssembly**.
|
|
14
|
+
|
|
15
|
+
## Table of Contents
|
|
16
|
+
|
|
17
|
+
- [Quick Start](#quick-start)
|
|
18
|
+
- [What's New](#whats-new)
|
|
19
|
+
- [v3.0.0 — Browser & WASM Support](#v300--browser--wasm-support)
|
|
20
|
+
- [v2.0.0 — TypeScript Modernization](#v200--typescript-modernization)
|
|
21
|
+
- [Browser Usage](#browser-usage)
|
|
22
|
+
- [CLI Tool (nxz)](#cli-tool-nxz)
|
|
23
|
+
- [API Reference](#api-reference)
|
|
24
|
+
- [API Comparison with Zlib](#api-comparison-with-zlib)
|
|
25
|
+
- [Options](#options)
|
|
26
|
+
- [Advanced Configuration](#advanced-configuration)
|
|
27
|
+
- [Thread Support](#thread-support)
|
|
28
|
+
- [Progress Monitoring](#progress-monitoring)
|
|
29
|
+
- [Concurrency Control (LZMAPool)](#concurrency-control-with-lzmapool)
|
|
30
|
+
- [File Compression Helpers](#file-compression-helpers)
|
|
31
|
+
- [Error Handling](#error-handling)
|
|
32
|
+
- [Benchmark](#benchmark)
|
|
33
|
+
- [Installation](#installation)
|
|
34
|
+
- [Testing](#testing)
|
|
35
|
+
- [Migration Guide (v1 → v2)](#migration-guide)
|
|
36
|
+
- [Contributing](#contributing)
|
|
37
|
+
- [Troubleshooting](#troubleshooting)
|
|
38
|
+
- [Bugs](#bugs)
|
|
39
|
+
- [Acknowledgements](#acknowledgements)
|
|
40
|
+
- [License](#license)
|
|
41
|
+
|
|
42
|
+
## What is liblzma/XZ?
|
|
43
|
+
|
|
44
|
+
[XZ](https://tukaani.org/xz/xz-file-format.txt) is a container for compressed archives. It offers one of the best compression ratios available, with a good balance between compression time and decompression speed/memory.
|
|
29
45
|
|
|
30
46
|
> Only LZMA2 is supported for compression output.
|
|
31
|
-
But the library can open and read any LZMA1 or LZMA2 compressed file.
|
|
47
|
+
> But the library can open and read any LZMA1 or LZMA2 compressed file.
|
|
32
48
|
|
|
33
|
-
|
|
49
|
+
## Quick Start
|
|
34
50
|
|
|
35
51
|
```bash
|
|
36
52
|
npm install node-liblzma
|
|
@@ -57,7 +73,8 @@ compressor.on('progress', ({ bytesRead, bytesWritten }) => {
|
|
|
57
73
|
});
|
|
58
74
|
```
|
|
59
75
|
|
|
60
|
-
|
|
76
|
+
<details>
|
|
77
|
+
<summary><strong>Promise style (with .then())</strong></summary>
|
|
61
78
|
|
|
62
79
|
```typescript
|
|
63
80
|
import { xzAsync, unxzAsync } from 'node-liblzma';
|
|
@@ -75,7 +92,10 @@ xzAsync(Buffer.from('Hello, World!'))
|
|
|
75
92
|
});
|
|
76
93
|
```
|
|
77
94
|
|
|
78
|
-
|
|
95
|
+
</details>
|
|
96
|
+
|
|
97
|
+
<details>
|
|
98
|
+
<summary><strong>Callback style (Node.js traditional)</strong></summary>
|
|
79
99
|
|
|
80
100
|
```typescript
|
|
81
101
|
import { xz, unxz } from 'node-liblzma';
|
|
@@ -89,88 +109,211 @@ xz(Buffer.from('Hello, World!'), (err, compressed) => {
|
|
|
89
109
|
});
|
|
90
110
|
```
|
|
91
111
|
|
|
112
|
+
</details>
|
|
113
|
+
|
|
92
114
|
📖 **Full API documentation**: [oorabona.github.io/node-liblzma](https://oorabona.github.io/node-liblzma/)
|
|
93
115
|
|
|
94
|
-
|
|
116
|
+
## What's New
|
|
95
117
|
|
|
96
|
-
|
|
118
|
+
### v3.0.0 — Browser & WASM Support
|
|
97
119
|
|
|
98
|
-
|
|
99
|
-
```typescript
|
|
100
|
-
const compressor = createXz();
|
|
101
|
-
compressor.on('progress', ({ bytesRead, bytesWritten }) => {
|
|
102
|
-
console.log(`Read: ${bytesRead}, Written: ${bytesWritten}`);
|
|
103
|
-
});
|
|
104
|
-
```
|
|
105
|
-
* **API Documentation**: Full TypeDoc documentation with Material theme at [oorabona.github.io/node-liblzma](https://oorabona.github.io/node-liblzma/)
|
|
106
|
-
* **XZ Utils 5.8.2**: Updated to latest stable version
|
|
120
|
+
> **[Live Demo](https://oorabona.github.io/node-liblzma/demo/)** — Try XZ compression in your browser.
|
|
107
121
|
|
|
108
|
-
|
|
122
|
+
- **Browser/WASM support**: Full XZ compression and decompression via WebAssembly
|
|
123
|
+
- Same API as Node.js (`xzAsync`, `unxzAsync`, `createXz`, `createUnxz`)
|
|
124
|
+
- WASM binary: ~52KB gzipped (under 100KB budget)
|
|
125
|
+
- Web Streams API for streaming compression/decompression
|
|
126
|
+
- Zero-config inline mode: `import from 'node-liblzma/inline'`
|
|
127
|
+
- **CLI tool (nxz)**: Portable xz-like command line tool
|
|
128
|
+
- Full xz compatibility: `-z`, `-d`, `-l`, `-k`, `-f`, `-c`, `-o`, `-v`, `-q`
|
|
129
|
+
- Compression presets 0-9 with extreme mode (`-e`)
|
|
130
|
+
- Progress display, stdin/stdout piping, benchmarking (`-B`)
|
|
131
|
+
- **Progress events**: Monitor compression/decompression in real-time
|
|
132
|
+
- **XZ Utils 5.8.x**: Updated to latest stable version
|
|
133
|
+
- **458+ tests**: Comprehensive test suite with 100% code coverage
|
|
109
134
|
|
|
110
|
-
|
|
135
|
+
### v2.0.0 — TypeScript Modernization
|
|
111
136
|
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
- [Biome](https://biomejs.dev/) for fast linting and formatting
|
|
117
|
-
- Pre-commit hooks with nano-staged and simple-git-hooks
|
|
118
|
-
- pnpm as package manager for better dependency management
|
|
119
|
-
* **Updated Node.js support**: Requires Node.js >= 16 (updated from >= 12)
|
|
137
|
+
- **Full TypeScript migration**: Complete rewrite from CoffeeScript
|
|
138
|
+
- **Promise-based APIs**: `xzAsync()` and `unxzAsync()`
|
|
139
|
+
- **Modern tooling**: Vitest, Biome, pnpm, pre-commit hooks
|
|
140
|
+
- **Node.js >= 16** required (updated from >= 12)
|
|
120
141
|
|
|
121
|
-
|
|
142
|
+
<details>
|
|
143
|
+
<summary><strong>Legacy (N-API migration)</strong></summary>
|
|
122
144
|
|
|
123
|
-
In previous versions, [N-API](https://nodejs.org/api/n-api.html)
|
|
145
|
+
In previous versions, [N-API](https://nodejs.org/api/n-api.html) replaced [nan](https://github.com/nodejs/nan) as the stable ABI for native modules.
|
|
124
146
|
|
|
125
|
-
|
|
147
|
+
Tested on: Linux x64, macOS (x64/arm64), Raspberry Pi 2/3/4, Windows.
|
|
126
148
|
|
|
127
|
-
|
|
128
|
-
* OSX (`macos-11`)
|
|
129
|
-
* Raspberry Pi 2/3/4 (both on 32-bit and 64-bit architectures)
|
|
130
|
-
* Windows (`windows-2019` and `windows-2022` are part of GitHub CI)
|
|
149
|
+
**Prebuilt binaries** are bundled for: Windows x64, Linux x64, macOS x64/arm64.
|
|
131
150
|
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
* For [Linux](https://github.com/oorabona/node-liblzma/actions/workflows/ci-linux.yml)
|
|
151
|
+
| Flag | Description | Default | Values |
|
|
152
|
+
|------|-------------|---------|--------|
|
|
153
|
+
| `USE_GLOBAL` | Use system liblzma library | `yes` (`no` on Windows) | `yes`, `no` |
|
|
154
|
+
| `RUNTIME_LINK` | Static or shared linking | `shared` | `static`, `shared` |
|
|
155
|
+
| `ENABLE_THREAD_SUPPORT` | Enable thread support | `yes` | `yes`, `no` |
|
|
138
156
|
|
|
139
|
-
|
|
157
|
+
If no prebuilt binary matches your platform, `node-gyp` will compile from source automatically.
|
|
140
158
|
|
|
141
|
-
|
|
159
|
+
</details>
|
|
142
160
|
|
|
143
|
-
|
|
161
|
+
## Browser Usage
|
|
144
162
|
|
|
145
|
-
|
|
146
|
-
* Linux x86_64
|
|
147
|
-
* MacOS x86_64 / Arm64
|
|
163
|
+
> **[Live Demo](https://oorabona.github.io/node-liblzma/demo/)** — Try XZ compression in your browser right now.
|
|
148
164
|
|
|
149
|
-
|
|
165
|
+
node-liblzma v3.0.0+ supports XZ compression in the browser via WebAssembly. The same API works in both Node.js and browsers — bundlers (Vite, Webpack, esbuild) automatically resolve the WASM-backed implementation.
|
|
150
166
|
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
167
|
+
### Basic Usage
|
|
168
|
+
|
|
169
|
+
```typescript
|
|
170
|
+
// Bundlers auto-resolve to WASM in browser, native in Node.js
|
|
171
|
+
import { xzAsync, unxzAsync, isXZ } from 'node-liblzma';
|
|
172
|
+
|
|
173
|
+
// Compress
|
|
174
|
+
const compressed = await xzAsync('Hello, browser!');
|
|
175
|
+
|
|
176
|
+
// Decompress
|
|
177
|
+
const original = await unxzAsync(compressed);
|
|
178
|
+
|
|
179
|
+
// Check if data is XZ-compressed
|
|
180
|
+
if (isXZ(someBuffer)) {
|
|
181
|
+
const data = await unxzAsync(someBuffer);
|
|
182
|
+
}
|
|
183
|
+
```
|
|
156
184
|
|
|
157
|
-
|
|
185
|
+
### Streaming with Web Streams API
|
|
158
186
|
|
|
159
|
-
|
|
187
|
+
```typescript
|
|
188
|
+
import { createXz, createUnxz } from 'node-liblzma';
|
|
189
|
+
|
|
190
|
+
// Compress a fetch response
|
|
191
|
+
const response = await fetch('/large-file.bin');
|
|
192
|
+
const compressed = response.body.pipeThrough(createXz({ preset: 6 }));
|
|
193
|
+
|
|
194
|
+
// Decompress
|
|
195
|
+
const decompressed = compressedStream.pipeThrough(createUnxz());
|
|
196
|
+
```
|
|
160
197
|
|
|
161
|
-
|
|
198
|
+
### Import Modes
|
|
162
199
|
|
|
163
|
-
|
|
200
|
+
| Import | When to use |
|
|
201
|
+
|--------|-------------|
|
|
202
|
+
| `node-liblzma` | Standard — bundler resolves to WASM (browser) or native (Node.js) |
|
|
203
|
+
| `node-liblzma/wasm` | Explicit WASM usage in Node.js (no native addon needed) |
|
|
204
|
+
| `node-liblzma/inline` | Zero-config — WASM embedded as base64 (no external file to serve) |
|
|
205
|
+
|
|
206
|
+
```typescript
|
|
207
|
+
// Explicit WASM (works in Node.js too, no native build required)
|
|
208
|
+
import { xzAsync } from 'node-liblzma/wasm';
|
|
209
|
+
|
|
210
|
+
// Inline mode (larger bundle, but no WASM file to configure)
|
|
211
|
+
import { ensureInlineInit, xzAsync } from 'node-liblzma/inline';
|
|
212
|
+
await ensureInlineInit(); // Decodes embedded base64 WASM
|
|
213
|
+
const compressed = await xzAsync(data);
|
|
214
|
+
```
|
|
164
215
|
|
|
165
|
-
|
|
166
|
-
A pure JavaScript implementation of the algorithm
|
|
167
|
-
* [node-xz](https://github.com/robey/node-xz)
|
|
168
|
-
Node binding of XZ library
|
|
169
|
-
* [lzma-native](https://github.com/addaleax/lzma-native)
|
|
170
|
-
A very complete implementation of XZ library bindings
|
|
171
|
-
* Others are also available but they fork "xz" process in the background.
|
|
216
|
+
### Browser Limitations
|
|
172
217
|
|
|
173
|
-
|
|
218
|
+
- **No sync APIs**: `xzSync()` / `unxzSync()` throw `LZMAError` in browsers
|
|
219
|
+
- **Presets 0-6 only**: Presets 7-9 require more memory than WASM's 256MB limit
|
|
220
|
+
- **No filesystem**: `xzFile()` / `unxzFile()` are not available
|
|
221
|
+
- **No Node Streams**: Use `createXz()` / `createUnxz()` (Web TransformStream) instead of `Xz` / `Unxz` classes
|
|
222
|
+
|
|
223
|
+
### Bundle Size
|
|
224
|
+
|
|
225
|
+
| Component | Raw | Gzipped |
|
|
226
|
+
|-----------|-----|---------|
|
|
227
|
+
| liblzma.wasm | ~107KB | ~52KB |
|
|
228
|
+
| Glue code (liblzma.js) | ~6KB | ~2KB |
|
|
229
|
+
| **Total** | **~113KB** | **~54KB** |
|
|
230
|
+
|
|
231
|
+
For detailed browser setup instructions, see [docs/BROWSER.md](docs/BROWSER.md).
|
|
232
|
+
|
|
233
|
+
## CLI Tool (nxz)
|
|
234
|
+
|
|
235
|
+
This package includes `nxz`, a portable xz-like CLI tool that works on any platform with Node.js.
|
|
236
|
+
|
|
237
|
+
### Installation
|
|
238
|
+
|
|
239
|
+
```bash
|
|
240
|
+
# Global installation (recommended for CLI usage)
|
|
241
|
+
npm install -g node-liblzma
|
|
242
|
+
|
|
243
|
+
# Then use directly
|
|
244
|
+
nxz --help
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
### Quick Examples
|
|
248
|
+
|
|
249
|
+
```bash
|
|
250
|
+
# Compress a file (creates file.txt.xz, deletes original)
|
|
251
|
+
nxz file.txt
|
|
252
|
+
|
|
253
|
+
# Decompress (auto-detected from .xz extension)
|
|
254
|
+
nxz file.txt.xz
|
|
255
|
+
|
|
256
|
+
# Keep original file (-k)
|
|
257
|
+
nxz -k file.txt
|
|
258
|
+
|
|
259
|
+
# Maximum compression (-9) with extreme mode (-e)
|
|
260
|
+
nxz -9e large-file.bin
|
|
261
|
+
|
|
262
|
+
# Compress to stdout (-c) for piping
|
|
263
|
+
nxz -c file.txt > file.txt.xz
|
|
264
|
+
|
|
265
|
+
# List archive info (-l / -lv for verbose)
|
|
266
|
+
nxz -l file.txt.xz
|
|
267
|
+
|
|
268
|
+
# Benchmark native vs WASM performance (-B)
|
|
269
|
+
nxz -B file.txt
|
|
270
|
+
```
|
|
271
|
+
|
|
272
|
+
### All Options
|
|
273
|
+
|
|
274
|
+
| Option | Long | Description |
|
|
275
|
+
|--------|------|-------------|
|
|
276
|
+
| `-z` | `--compress` | Force compression mode |
|
|
277
|
+
| `-d` | `--decompress` | Force decompression mode |
|
|
278
|
+
| `-l` | `--list` | List archive information |
|
|
279
|
+
| `-B` | `--benchmark` | Benchmark native vs WASM performance |
|
|
280
|
+
| `-k` | `--keep` | Keep original file (don't delete) |
|
|
281
|
+
| `-f` | `--force` | Overwrite existing output file |
|
|
282
|
+
| `-c` | `--stdout` | Write to stdout, keep original file |
|
|
283
|
+
| `-o` | `--output=FILE` | Write output to specified file |
|
|
284
|
+
| `-v` | `--verbose` | Show progress for large files |
|
|
285
|
+
| `-q` | `--quiet` | Suppress warning messages |
|
|
286
|
+
| `-0`..`-9` | | Compression level (default: 6) |
|
|
287
|
+
| `-e` | `--extreme` | Extreme compression (slower) |
|
|
288
|
+
| `-h` | `--help` | Show help |
|
|
289
|
+
| `-V` | `--version` | Show version |
|
|
290
|
+
|
|
291
|
+
<details>
|
|
292
|
+
<summary><strong>One-shot usage (without global install)</strong></summary>
|
|
293
|
+
|
|
294
|
+
```bash
|
|
295
|
+
# npm/npx
|
|
296
|
+
npx --package node-liblzma nxz --help
|
|
297
|
+
|
|
298
|
+
# pnpm
|
|
299
|
+
pnpm dlx --package node-liblzma nxz --help
|
|
300
|
+
```
|
|
301
|
+
|
|
302
|
+
</details>
|
|
303
|
+
|
|
304
|
+
### Exit Codes
|
|
305
|
+
|
|
306
|
+
| Code | Meaning |
|
|
307
|
+
|------|---------|
|
|
308
|
+
| 0 | Success |
|
|
309
|
+
| 1 | Error (file not found, format error, etc.) |
|
|
310
|
+
| 130 | Interrupted (SIGINT/Ctrl+C) |
|
|
311
|
+
|
|
312
|
+
## API Reference
|
|
313
|
+
|
|
314
|
+
### API Comparison with Zlib
|
|
315
|
+
|
|
316
|
+
The API mirrors Node.js Zlib for easy adoption:
|
|
174
317
|
|
|
175
318
|
```js
|
|
176
319
|
// CommonJS
|
|
@@ -193,9 +336,7 @@ import * as lzma from 'node-liblzma';
|
|
|
193
336
|
| - | `xzFile` | `(input, output, [options])` → `Promise<void>` |
|
|
194
337
|
| - | `unxzFile` | `(input, output, [options])` → `Promise<void>` |
|
|
195
338
|
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
The `options` object accepts the following attributes:
|
|
339
|
+
### Options
|
|
199
340
|
|
|
200
341
|
| Attribute | Type | Description | Values |
|
|
201
342
|
|-----------|------|-------------|--------|
|
|
@@ -206,15 +347,13 @@ The `options` object accepts the following attributes:
|
|
|
206
347
|
| `filters` | array | Filter chain | `filter.LZMA2`, `filter.X86`, `filter.ARM`, etc. |
|
|
207
348
|
| `chunkSize` | number | Processing chunk size | Default: 64KB |
|
|
208
349
|
|
|
209
|
-
For further information
|
|
350
|
+
For further information, see the [XZ SDK documentation](http://7-zip.org/sdk.html).
|
|
210
351
|
|
|
211
352
|
## Advanced Configuration
|
|
212
353
|
|
|
213
354
|
### Thread Support
|
|
214
355
|
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
**Thread values:**
|
|
356
|
+
Multi-threaded compression is available when built with `ENABLE_THREAD_SUPPORT=yes` (default).
|
|
218
357
|
|
|
219
358
|
| Value | Behavior |
|
|
220
359
|
|-------|----------|
|
|
@@ -222,26 +361,15 @@ The library supports multi-threaded compression when built with `ENABLE_THREAD_S
|
|
|
222
361
|
| `1` | Single-threaded (default) |
|
|
223
362
|
| `N` | Use exactly N threads |
|
|
224
363
|
|
|
225
|
-
**Example:**
|
|
226
|
-
|
|
227
364
|
```typescript
|
|
228
365
|
import { createXz, hasThreads } from 'node-liblzma';
|
|
229
366
|
|
|
230
|
-
// Check if threading is available
|
|
231
367
|
if (hasThreads()) {
|
|
232
|
-
|
|
233
|
-
const compressor = createXz({ threads: 0 });
|
|
234
|
-
|
|
235
|
-
// Or specify exact thread count
|
|
236
|
-
const compressor4 = createXz({ threads: 4 });
|
|
368
|
+
const compressor = createXz({ threads: 0 }); // auto-detect
|
|
237
369
|
}
|
|
238
370
|
```
|
|
239
371
|
|
|
240
|
-
**
|
|
241
|
-
- Thread support only applies to **compression**, not decompression
|
|
242
|
-
- Requires LZMA library built with pthread support (`ENABLE_THREAD_SUPPORT=yes`)
|
|
243
|
-
- Default is `threads: 1` (single-threaded) for predictable behavior
|
|
244
|
-
- Check availability: `hasThreads()` returns `true` if multi-threading is supported
|
|
372
|
+
> **Note**: Threads only apply to compression, not decompression.
|
|
245
373
|
|
|
246
374
|
### Progress Monitoring
|
|
247
375
|
|
|
@@ -254,65 +382,52 @@ const compressor = createXz({ preset: 6 });
|
|
|
254
382
|
|
|
255
383
|
compressor.on('progress', ({ bytesRead, bytesWritten }) => {
|
|
256
384
|
const ratio = bytesWritten / bytesRead;
|
|
257
|
-
console.log(`Progress: ${bytesRead}
|
|
258
|
-
});
|
|
259
|
-
|
|
260
|
-
// Works with both compression and decompression
|
|
261
|
-
const decompressor = createUnxz();
|
|
262
|
-
decompressor.on('progress', ({ bytesRead, bytesWritten }) => {
|
|
263
|
-
console.log(`Decompressing: ${bytesRead} → ${bytesWritten} bytes`);
|
|
385
|
+
console.log(`Progress: ${bytesRead} in, ${bytesWritten} out (ratio: ${ratio.toFixed(2)})`);
|
|
264
386
|
});
|
|
265
387
|
|
|
266
388
|
inputStream.pipe(compressor).pipe(outputStream);
|
|
267
389
|
```
|
|
268
390
|
|
|
269
|
-
|
|
270
|
-
- Progress events fire after each chunk is processed
|
|
271
|
-
- `bytesRead`: Total input bytes processed so far
|
|
272
|
-
- `bytesWritten`: Total output bytes produced so far
|
|
273
|
-
- Works with streams, not buffer APIs (`xz`/`unxz`)
|
|
391
|
+
Progress events fire after each chunk is processed. Works with streams, not buffer APIs.
|
|
274
392
|
|
|
275
|
-
###
|
|
393
|
+
### Concurrency Control with LZMAPool
|
|
276
394
|
|
|
277
|
-
For
|
|
395
|
+
For production environments with high concurrency needs:
|
|
278
396
|
|
|
279
397
|
```typescript
|
|
280
|
-
|
|
281
|
-
preset: lzma.preset.DEFAULT,
|
|
282
|
-
chunkSize: 256 * 1024 // 256KB chunks (default: 64KB)
|
|
283
|
-
});
|
|
284
|
-
```
|
|
398
|
+
import { LZMAPool } from 'node-liblzma';
|
|
285
399
|
|
|
286
|
-
|
|
287
|
-
- **Small files (< 1MB)**: Use default 64KB chunks
|
|
288
|
-
- **Medium files (1-10MB)**: Use 128-256KB chunks
|
|
289
|
-
- **Large files (> 10MB)**: Use 512KB-1MB chunks
|
|
290
|
-
- **Maximum buffer size**: 512MB per operation (security limit)
|
|
400
|
+
const pool = new LZMAPool(10); // Max 10 concurrent operations
|
|
291
401
|
|
|
292
|
-
|
|
402
|
+
pool.on('metrics', (metrics) => {
|
|
403
|
+
console.log(`Active: ${metrics.active}, Queued: ${metrics.queued}`);
|
|
404
|
+
});
|
|
405
|
+
|
|
406
|
+
const compressed = await pool.compress(buffer);
|
|
407
|
+
const decompressed = await pool.decompress(compressed);
|
|
408
|
+
```
|
|
293
409
|
|
|
294
|
-
|
|
410
|
+
### File Compression Helpers
|
|
295
411
|
|
|
296
412
|
```typescript
|
|
297
|
-
import {
|
|
298
|
-
import { createXz } from 'node-liblzma';
|
|
413
|
+
import { xzFile, unxzFile } from 'node-liblzma';
|
|
299
414
|
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
415
|
+
await xzFile('input.txt', 'output.txt.xz');
|
|
416
|
+
await unxzFile('output.txt.xz', 'restored.txt');
|
|
417
|
+
|
|
418
|
+
// With options
|
|
419
|
+
await xzFile('large-file.bin', 'compressed.xz', { preset: 9, threads: 4 });
|
|
303
420
|
```
|
|
304
421
|
|
|
422
|
+
Handles files > 512MB automatically via streams with lower memory footprint.
|
|
423
|
+
|
|
305
424
|
### Error Handling
|
|
306
425
|
|
|
307
|
-
|
|
426
|
+
Typed error classes for precise error handling:
|
|
308
427
|
|
|
309
428
|
```typescript
|
|
310
429
|
import {
|
|
311
|
-
xzAsync,
|
|
312
|
-
LZMAError,
|
|
313
|
-
LZMAMemoryError,
|
|
314
|
-
LZMADataError,
|
|
315
|
-
LZMAFormatError
|
|
430
|
+
xzAsync, LZMAError, LZMAMemoryError, LZMADataError, LZMAFormatError
|
|
316
431
|
} from 'node-liblzma';
|
|
317
432
|
|
|
318
433
|
try {
|
|
@@ -324,339 +439,106 @@ try {
|
|
|
324
439
|
console.error('Corrupt data:', error.message);
|
|
325
440
|
} else if (error instanceof LZMAFormatError) {
|
|
326
441
|
console.error('Invalid format:', error.message);
|
|
327
|
-
} else {
|
|
328
|
-
console.error('Unknown error:', error);
|
|
329
442
|
}
|
|
330
443
|
}
|
|
331
444
|
```
|
|
332
445
|
|
|
333
|
-
**Available error classes
|
|
334
|
-
- `LZMAError` - Base error class
|
|
335
|
-
- `LZMAMemoryError` - Memory allocation failed
|
|
336
|
-
- `LZMAMemoryLimitError` - Memory limit exceeded
|
|
337
|
-
- `LZMAFormatError` - Unrecognized file format
|
|
338
|
-
- `LZMAOptionsError` - Invalid compression options
|
|
339
|
-
- `LZMADataError` - Corrupt compressed data
|
|
340
|
-
- `LZMABufferError` - Buffer size issues
|
|
341
|
-
- `LZMAProgrammingError` - Internal errors
|
|
342
|
-
|
|
343
|
-
### Error Recovery
|
|
446
|
+
**Available error classes**: `LZMAError` (base), `LZMAMemoryError`, `LZMAMemoryLimitError`, `LZMAFormatError`, `LZMAOptionsError`, `LZMADataError`, `LZMABufferError`, `LZMAProgrammingError`.
|
|
344
447
|
|
|
345
|
-
|
|
448
|
+
### Buffer Size Optimization
|
|
346
449
|
|
|
347
450
|
```typescript
|
|
348
|
-
const
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
console.error('Decompression error:', error.errno, error.message);
|
|
352
|
-
// Stream will emit 'close' event after error
|
|
353
|
-
});
|
|
354
|
-
|
|
355
|
-
decompressor.on('close', () => {
|
|
356
|
-
console.log('Stream closed, safe to cleanup');
|
|
451
|
+
const stream = createXz({
|
|
452
|
+
preset: lzma.preset.DEFAULT,
|
|
453
|
+
chunkSize: 256 * 1024 // 256KB chunks (default: 64KB)
|
|
357
454
|
});
|
|
358
455
|
```
|
|
359
456
|
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
import { LZMAPool } from 'node-liblzma';
|
|
366
|
-
|
|
367
|
-
const pool = new LZMAPool(10); // Max 10 concurrent operations
|
|
368
|
-
|
|
369
|
-
// Monitor pool metrics
|
|
370
|
-
pool.on('metrics', (metrics) => {
|
|
371
|
-
console.log(`Active: ${metrics.active}, Queued: ${metrics.queued}`);
|
|
372
|
-
console.log(`Completed: ${metrics.completed}, Failed: ${metrics.failed}`);
|
|
373
|
-
});
|
|
457
|
+
| File Size | Recommended chunkSize |
|
|
458
|
+
|-----------|-----------------------|
|
|
459
|
+
| < 1MB | 64KB (default) |
|
|
460
|
+
| 1-10MB | 128-256KB |
|
|
461
|
+
| > 10MB | 512KB-1MB |
|
|
374
462
|
|
|
375
|
-
|
|
376
|
-
const compressed = await pool.compress(buffer);
|
|
377
|
-
const decompressed = await pool.decompress(compressed);
|
|
463
|
+
Maximum buffer size: 512MB per operation (security limit). For larger files, use streaming APIs.
|
|
378
464
|
|
|
379
|
-
|
|
380
|
-
const status = pool.getMetrics();
|
|
381
|
-
```
|
|
465
|
+
### Async Callback Contract (errno-based)
|
|
382
466
|
|
|
383
|
-
|
|
384
|
-
- `queue` - Task added to queue
|
|
385
|
-
- `start` - Task started processing
|
|
386
|
-
- `complete` - Task completed successfully
|
|
387
|
-
- `error-task` - Task failed
|
|
388
|
-
- `metrics` - Metrics updated (after each state change)
|
|
467
|
+
The low-level native callback follows an errno-style contract matching liblzma behavior:
|
|
389
468
|
|
|
390
|
-
**
|
|
391
|
-
-
|
|
392
|
-
-
|
|
393
|
-
- ✅ Production-ready monitoring
|
|
394
|
-
- ✅ Zero breaking changes (opt-in)
|
|
469
|
+
- **Signature**: `(errno: number, availInAfter: number, availOutAfter: number)`
|
|
470
|
+
- **Success**: `errno` is `LZMA_OK` or `LZMA_STREAM_END`
|
|
471
|
+
- **Error**: any other `errno` value
|
|
395
472
|
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
Simplified API for file-based compression:
|
|
473
|
+
High-level APIs remain ergonomic — Promise functions resolve to `Buffer` or reject with `Error`, stream users listen to `error` events.
|
|
399
474
|
|
|
400
|
-
|
|
401
|
-
import { xzFile, unxzFile } from 'node-liblzma';
|
|
475
|
+
## Benchmark
|
|
402
476
|
|
|
403
|
-
|
|
404
|
-
await xzFile('input.txt', 'output.txt.xz');
|
|
477
|
+
### Performance Hierarchy
|
|
405
478
|
|
|
406
|
-
|
|
407
|
-
await unxzFile('output.txt.xz', 'restored.txt');
|
|
479
|
+
All three backends use the same liblzma library and produce **identical compression ratios**:
|
|
408
480
|
|
|
409
|
-
// With options
|
|
410
|
-
await xzFile('large-file.bin', 'compressed.xz', {
|
|
411
|
-
preset: 9,
|
|
412
|
-
threads: 4
|
|
413
|
-
});
|
|
414
481
|
```
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
- ✅ Built-in backpressure via streams
|
|
419
|
-
- ✅ Lower memory footprint
|
|
420
|
-
- ✅ Simpler API for common use cases
|
|
421
|
-
|
|
422
|
-
## Async callback contract (errno-based)
|
|
423
|
-
|
|
424
|
-
The low-level native callback used internally by streams follows an errno-style contract to match liblzma behavior and to avoid mixing exception channels:
|
|
425
|
-
|
|
426
|
-
- Signature: `(errno: number, availInAfter: number, availOutAfter: number)`
|
|
427
|
-
- Success: `errno` is either `LZMA_OK` or `LZMA_STREAM_END`.
|
|
428
|
-
- Recoverable/other conditions: any other `errno` value (for example, `LZMA_BUF_ERROR`, `LZMA_DATA_ERROR`, `LZMA_PROG_ERROR`) indicates an error state.
|
|
429
|
-
- Streams emit `onerror` with the numeric `errno` when `errno !== LZMA_OK && errno !== LZMA_STREAM_END`.
|
|
430
|
-
|
|
431
|
-
Why errno instead of JS exceptions?
|
|
432
|
-
|
|
433
|
-
- The binding mirrors liblzma’s status codes and keeps a single error channel that’s easy to reason about in tight processing loops.
|
|
434
|
-
- This avoids throwing across async worker boundaries and keeps cleanup deterministic.
|
|
435
|
-
|
|
436
|
-
High-level APIs remain ergonomic:
|
|
437
|
-
|
|
438
|
-
- Promise-based functions `xzAsync()`/`unxzAsync()` still resolve to `Buffer` or reject with `Error` as expected.
|
|
439
|
-
- Stream users can listen to `error` events, where we map `errno` to a human-friendly message (`messages[errno]`).
|
|
440
|
-
|
|
441
|
-
If you prefer Node’s error-first callbacks, you can wrap the APIs and translate `errno` to `Error` objects at your boundaries without changing the native layer.
|
|
442
|
-
|
|
443
|
-
# Installation
|
|
444
|
-
|
|
445
|
-
Well, as simple as this one-liner:
|
|
446
|
-
|
|
447
|
-
```sh
|
|
448
|
-
npm i node-liblzma --save
|
|
482
|
+
System xz > nxz native (C++ addon) > nxz WASM (Emscripten)
|
|
483
|
+
fastest ~1-2x slower ~2-5x slower (decompress)
|
|
484
|
+
~1x (compress, large files)
|
|
449
485
|
```
|
|
450
486
|
|
|
451
|
-
|
|
487
|
+
### Full Comparison (246 KB source code, preset 6)
|
|
452
488
|
|
|
453
|
-
|
|
454
|
-
|
|
455
|
-
|
|
489
|
+
| Backend | Compress | Decompress | Size | Environment |
|
|
490
|
+
|---------|----------|------------|------|-------------|
|
|
491
|
+
| **System `xz` 5.8** | 81 ms | 4 ms | 76.7 KB | C binary |
|
|
492
|
+
| **nxz native** | 90 ms | 3.4 ms | 76.7 KB | Node.js + C++ addon |
|
|
493
|
+
| **nxz WASM** | 86 ms | 7.9 ms | 76.7 KB | Node.js + Emscripten |
|
|
456
494
|
|
|
457
|
-
|
|
495
|
+
### Native vs WASM (nxz -B, preset 6)
|
|
458
496
|
|
|
459
|
-
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
``` bash
|
|
466
|
-
ENABLE_THREAD_SUPPORT=no npm install node-liblzma --build-from-source
|
|
467
|
-
```
|
|
468
|
-
|
|
469
|
-
> Note:
|
|
470
|
-
Enabling thread support in the library will __NOT__ work if the LZMA library itself has been built without such support.
|
|
471
|
-
|
|
472
|
-
To build the module, you have the following options:
|
|
473
|
-
|
|
474
|
-
1. Using system development libraries
|
|
475
|
-
2. Ask the build system to download `xz` and build it
|
|
476
|
-
3. Compile `xz` yourself, outside `node-liblzma`, and have it use it after
|
|
477
|
-
|
|
478
|
-
## Using system dev libraries to compile
|
|
479
|
-
|
|
480
|
-
You need to have the development package installed on your system. If you have Debian based distro:
|
|
481
|
-
|
|
482
|
-
```
|
|
483
|
-
# apt-get install liblzma-dev
|
|
484
|
-
```
|
|
485
|
-
|
|
486
|
-
## Automatic download and compilation to statically link `xz`
|
|
487
|
-
|
|
488
|
-
If you do not plan on having a local install, you can ask for automatic download and build of whatever version of `xz` you want.
|
|
489
|
-
|
|
490
|
-
Just do:
|
|
491
|
-
|
|
492
|
-
```sh
|
|
493
|
-
npm install node-liblzma --build-from-source
|
|
494
|
-
```
|
|
495
|
-
|
|
496
|
-
When no option is given in the commandline arguments, it will build with default values.
|
|
497
|
-
|
|
498
|
-
## Local install of `xz` sources (outside `node-liblzma`)
|
|
497
|
+
| Data | Compress | Decompress | Notes |
|
|
498
|
+
|------|----------|------------|-------|
|
|
499
|
+
| 1 KB text | WASM 2.8x slower | WASM 4.9x slower | Startup overhead dominates |
|
|
500
|
+
| 135 KB binary | ~1:1 | WASM 2x slower | Compression near-parity |
|
|
501
|
+
| 246 KB source | ~1:1 | WASM 2.3x slower | Realistic workload |
|
|
502
|
+
| 1 MB random | ~1:1 | WASM 1.6x slower | Gap narrows with size |
|
|
499
503
|
|
|
500
|
-
|
|
504
|
+
<details>
|
|
505
|
+
<summary><strong>Running benchmarks</strong></summary>
|
|
501
506
|
|
|
502
|
-
|
|
507
|
+
```bash
|
|
508
|
+
# Compare nxz (native) vs system xz across file sizes
|
|
509
|
+
./scripts/benchmark.sh
|
|
510
|
+
./scripts/benchmark.sh -s 1,50,200 -p 6,9 # custom sizes/presets
|
|
511
|
+
./scripts/benchmark.sh -o csv > results.csv # export as CSV/JSON
|
|
503
512
|
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
|
|
507
|
-
export LD_LIBRARY_PATH=$HOME/path/to/lib:$LD_LIBRARY_PATH
|
|
513
|
+
# Compare native addon vs WASM backend
|
|
514
|
+
nxz --benchmark file.txt
|
|
515
|
+
nxz -B -3 large-file.bin # with preset 3
|
|
508
516
|
```
|
|
509
517
|
|
|
510
|
-
|
|
511
|
-
|
|
512
|
-
Once done, this should suffice:
|
|
513
|
-
|
|
514
|
-
```sh
|
|
515
|
-
npm install
|
|
516
|
-
```
|
|
518
|
+
</details>
|
|
517
519
|
|
|
518
|
-
|
|
520
|
+
### When to Use What
|
|
519
521
|
|
|
520
|
-
|
|
522
|
+
| Scenario | Recommended |
|
|
523
|
+
|----------|-------------|
|
|
524
|
+
| Browser | WASM (only option) |
|
|
525
|
+
| Node.js, performance-critical | Native addon |
|
|
526
|
+
| Node.js, no C++ toolchain available | WASM (`node-liblzma/wasm`) |
|
|
527
|
+
| Cross-platform scripts | nxz CLI |
|
|
528
|
+
| Batch processing many files | System xz |
|
|
529
|
+
| CI/CD with Node.js already installed | nxz CLI |
|
|
521
530
|
|
|
522
|
-
|
|
531
|
+
## Installation
|
|
523
532
|
|
|
524
|
-
```
|
|
525
|
-
npm
|
|
533
|
+
```bash
|
|
534
|
+
npm install node-liblzma
|
|
526
535
|
# or
|
|
527
|
-
pnpm
|
|
528
|
-
```
|
|
529
|
-
|
|
530
|
-
It will build and launch the test suite (325+ tests) with [Vitest](https://vitest.dev/) with TypeScript support and coverage reporting.
|
|
531
|
-
|
|
532
|
-
Additional testing commands:
|
|
533
|
-
|
|
534
|
-
```sh
|
|
535
|
-
# Watch mode for development
|
|
536
|
-
pnpm test:watch
|
|
537
|
-
|
|
538
|
-
# Coverage report
|
|
539
|
-
pnpm test:coverage
|
|
540
|
-
|
|
541
|
-
# Type checking
|
|
542
|
-
pnpm type-check
|
|
543
|
-
```
|
|
544
|
-
|
|
545
|
-
# Usage
|
|
546
|
-
|
|
547
|
-
As the API is very close to NodeJS Zlib, you will probably find a good reference
|
|
548
|
-
[there](http://www.nodejs.org/api/zlib.html).
|
|
549
|
-
|
|
550
|
-
Otherwise examples can be found as part of the test suite, so feel free to use them!
|
|
551
|
-
They are written in TypeScript with full type definitions.
|
|
552
|
-
|
|
553
|
-
# Migration Guide
|
|
554
|
-
|
|
555
|
-
## Migrating from v1.x to v2.0
|
|
556
|
-
|
|
557
|
-
Version 2.0 introduces several breaking changes along with powerful new features.
|
|
558
|
-
|
|
559
|
-
### Breaking Changes
|
|
560
|
-
|
|
561
|
-
1. **Node.js Version Requirement**
|
|
562
|
-
```diff
|
|
563
|
-
- Requires Node.js >= 12
|
|
564
|
-
+ Requires Node.js >= 16
|
|
565
|
-
```
|
|
566
|
-
|
|
567
|
-
2. **ESM Module Format**
|
|
568
|
-
```diff
|
|
569
|
-
- CommonJS: var lzma = require('node-liblzma');
|
|
570
|
-
+ ESM: import * as lzma from 'node-liblzma';
|
|
571
|
-
+ CommonJS still works via dynamic import
|
|
572
|
-
```
|
|
573
|
-
|
|
574
|
-
3. **TypeScript Migration**
|
|
575
|
-
- Source code migrated from CoffeeScript to TypeScript
|
|
576
|
-
- Full type definitions included
|
|
577
|
-
- Better IDE autocomplete and type safety
|
|
578
|
-
|
|
579
|
-
### New Features You Should Adopt
|
|
580
|
-
|
|
581
|
-
1. **Promise-based APIs** (Recommended for new code)
|
|
582
|
-
```typescript
|
|
583
|
-
// Old callback style (still works)
|
|
584
|
-
xz(buffer, (err, compressed) => {
|
|
585
|
-
if (err) throw err;
|
|
586
|
-
// use compressed
|
|
587
|
-
});
|
|
588
|
-
|
|
589
|
-
// New Promise style
|
|
590
|
-
try {
|
|
591
|
-
const compressed = await xzAsync(buffer);
|
|
592
|
-
// use compressed
|
|
593
|
-
} catch (err) {
|
|
594
|
-
// handle error
|
|
595
|
-
}
|
|
596
|
-
```
|
|
597
|
-
|
|
598
|
-
2. **Typed Error Classes** (Better error handling)
|
|
599
|
-
```typescript
|
|
600
|
-
import { LZMAMemoryError, LZMADataError } from 'node-liblzma';
|
|
601
|
-
|
|
602
|
-
try {
|
|
603
|
-
await unxzAsync(corruptData);
|
|
604
|
-
} catch (error) {
|
|
605
|
-
if (error instanceof LZMADataError) {
|
|
606
|
-
console.error('Corrupt compressed data');
|
|
607
|
-
} else if (error instanceof LZMAMemoryError) {
|
|
608
|
-
console.error('Out of memory');
|
|
609
|
-
}
|
|
610
|
-
}
|
|
611
|
-
```
|
|
612
|
-
|
|
613
|
-
3. **Concurrency Control** (For high-throughput applications)
|
|
614
|
-
```typescript
|
|
615
|
-
import { LZMAPool } from 'node-liblzma';
|
|
616
|
-
|
|
617
|
-
const pool = new LZMAPool(10); // Max 10 concurrent operations
|
|
618
|
-
|
|
619
|
-
// Automatic queuing and backpressure
|
|
620
|
-
const results = await Promise.all(
|
|
621
|
-
files.map(file => pool.compress(file))
|
|
622
|
-
);
|
|
623
|
-
```
|
|
624
|
-
|
|
625
|
-
4. **File Helpers** (Simpler file compression)
|
|
626
|
-
```typescript
|
|
627
|
-
import { xzFile, unxzFile } from 'node-liblzma';
|
|
628
|
-
|
|
629
|
-
// Compress a file (handles streaming automatically)
|
|
630
|
-
await xzFile('input.txt', 'output.txt.xz');
|
|
631
|
-
|
|
632
|
-
// Decompress a file
|
|
633
|
-
await unxzFile('output.txt.xz', 'restored.txt');
|
|
634
|
-
```
|
|
635
|
-
|
|
636
|
-
### Testing Framework Change
|
|
637
|
-
|
|
638
|
-
If you maintain tests for code using node-liblzma:
|
|
639
|
-
|
|
640
|
-
```diff
|
|
641
|
-
- Mocha test framework
|
|
642
|
-
+ Vitest test framework (faster, better TypeScript support)
|
|
536
|
+
pnpm add node-liblzma
|
|
643
537
|
```
|
|
644
538
|
|
|
645
|
-
###
|
|
646
|
-
|
|
647
|
-
Development tooling has been modernized:
|
|
648
|
-
|
|
649
|
-
- **Linter**: Biome (replaces ESLint + Prettier)
|
|
650
|
-
- **Package Manager**: pnpm recommended (npm/yarn still work)
|
|
651
|
-
- **Pre-commit Hooks**: nano-staged + simple-git-hooks
|
|
652
|
-
|
|
653
|
-
# Troubleshooting
|
|
539
|
+
### System Libraries
|
|
654
540
|
|
|
655
|
-
|
|
656
|
-
|
|
657
|
-
### Issue: "Cannot find liblzma library"
|
|
658
|
-
|
|
659
|
-
**Solution**: Install system development package or let node-gyp download it:
|
|
541
|
+
If prebuilt binaries don't match your platform, install system development libraries:
|
|
660
542
|
|
|
661
543
|
```bash
|
|
662
544
|
# Debian/Ubuntu
|
|
@@ -665,374 +547,196 @@ sudo apt-get install liblzma-dev
|
|
|
665
547
|
# macOS
|
|
666
548
|
brew install xz
|
|
667
549
|
|
|
668
|
-
# Windows (
|
|
550
|
+
# Windows (automatic download and build)
|
|
669
551
|
npm install node-liblzma --build-from-source
|
|
670
552
|
```
|
|
671
553
|
|
|
672
|
-
###
|
|
673
|
-
|
|
674
|
-
**Symptoms**: Build fails with C++ compilation errors
|
|
675
|
-
|
|
676
|
-
**Solutions**:
|
|
677
|
-
1. Install build tools:
|
|
678
|
-
```bash
|
|
679
|
-
# Ubuntu/Debian
|
|
680
|
-
sudo apt-get install build-essential python3
|
|
681
|
-
|
|
682
|
-
# macOS (install Xcode Command Line Tools)
|
|
683
|
-
xcode-select --install
|
|
684
|
-
|
|
685
|
-
# Windows
|
|
686
|
-
npm install --global windows-build-tools
|
|
687
|
-
```
|
|
688
|
-
|
|
689
|
-
2. Clear build cache and retry:
|
|
690
|
-
```bash
|
|
691
|
-
rm -rf build node_modules
|
|
692
|
-
npm install
|
|
693
|
-
```
|
|
694
|
-
|
|
695
|
-
### Issue: "Prebuilt binary not found"
|
|
696
|
-
|
|
697
|
-
**Solution**: Your platform might not have prebuilt binaries. Build from source:
|
|
554
|
+
### Build from Source
|
|
698
555
|
|
|
699
556
|
```bash
|
|
557
|
+
# Force rebuild with default options
|
|
700
558
|
npm install node-liblzma --build-from-source
|
|
701
|
-
```
|
|
702
|
-
|
|
703
|
-
## Runtime Issues
|
|
704
|
-
|
|
705
|
-
### Issue: "Memory allocation failed" (LZMAMemoryError)
|
|
706
|
-
|
|
707
|
-
**Causes**:
|
|
708
|
-
- Input buffer exceeds 512MB limit (security protection)
|
|
709
|
-
- System out of memory
|
|
710
|
-
- Trying to decompress extremely large archive
|
|
711
|
-
|
|
712
|
-
**Solutions**:
|
|
713
|
-
1. For files > 512MB, use streaming APIs:
|
|
714
|
-
```typescript
|
|
715
|
-
import { createReadStream, createWriteStream } from 'fs';
|
|
716
|
-
import { createXz } from 'node-liblzma';
|
|
717
|
-
|
|
718
|
-
createReadStream('large-file.bin')
|
|
719
|
-
.pipe(createXz())
|
|
720
|
-
.pipe(createWriteStream('large-file.xz'));
|
|
721
|
-
```
|
|
722
|
-
|
|
723
|
-
2. Or use file helpers (automatically handle large files):
|
|
724
|
-
```typescript
|
|
725
|
-
await xzFile('large-file.bin', 'large-file.xz');
|
|
726
|
-
```
|
|
727
|
-
|
|
728
|
-
### Issue: "Corrupt compressed data" (LZMADataError)
|
|
729
|
-
|
|
730
|
-
**Symptoms**: Decompression fails with `LZMADataError`
|
|
731
|
-
|
|
732
|
-
**Causes**:
|
|
733
|
-
- File is not actually XZ/LZMA compressed
|
|
734
|
-
- File is corrupted or incomplete
|
|
735
|
-
- Wrong file format (LZMA1 vs LZMA2)
|
|
736
|
-
|
|
737
|
-
**Solutions**:
|
|
738
|
-
1. Verify file format:
|
|
739
|
-
```bash
|
|
740
|
-
file compressed.xz
|
|
741
|
-
# Should show: "XZ compressed data"
|
|
742
|
-
```
|
|
743
|
-
|
|
744
|
-
2. Check file integrity:
|
|
745
|
-
```bash
|
|
746
|
-
xz -t compressed.xz
|
|
747
|
-
```
|
|
748
|
-
|
|
749
|
-
3. Handle errors gracefully:
|
|
750
|
-
```typescript
|
|
751
|
-
try {
|
|
752
|
-
const data = await unxzAsync(buffer);
|
|
753
|
-
} catch (error) {
|
|
754
|
-
if (error instanceof LZMADataError) {
|
|
755
|
-
console.error('Invalid or corrupt XZ file');
|
|
756
|
-
}
|
|
757
|
-
}
|
|
758
|
-
```
|
|
759
559
|
|
|
760
|
-
|
|
761
|
-
|
|
762
|
-
**Symptoms**: Compiler warnings about `-Wmissing-field-initializers`
|
|
763
|
-
|
|
764
|
-
**Status**: This is normal and does not affect functionality. Thread support still works correctly.
|
|
765
|
-
|
|
766
|
-
**Disable thread support** (if warnings are problematic):
|
|
767
|
-
```bash
|
|
560
|
+
# Disable thread support
|
|
768
561
|
ENABLE_THREAD_SUPPORT=no npm install node-liblzma --build-from-source
|
|
769
562
|
```
|
|
770
563
|
|
|
771
|
-
|
|
772
|
-
|
|
773
|
-
### Issue: Compression is slow on multi-core systems
|
|
564
|
+
<details>
|
|
565
|
+
<summary><strong>Custom XZ installation</strong></summary>
|
|
774
566
|
|
|
775
|
-
|
|
567
|
+
If you compiled XZ outside of node-liblzma:
|
|
776
568
|
|
|
777
|
-
```
|
|
778
|
-
|
|
779
|
-
|
|
780
|
-
|
|
781
|
-
|
|
782
|
-
});
|
|
569
|
+
```bash
|
|
570
|
+
export CPATH=$HOME/path/to/headers
|
|
571
|
+
export LIBRARY_PATH=$HOME/path/to/lib
|
|
572
|
+
export LD_LIBRARY_PATH=$HOME/path/to/lib:$LD_LIBRARY_PATH
|
|
573
|
+
npm install
|
|
783
574
|
```
|
|
784
575
|
|
|
785
|
-
|
|
786
|
-
|
|
787
|
-
### Issue: High memory usage with concurrent operations
|
|
788
|
-
|
|
789
|
-
**Solution**: Use `LZMAPool` to limit concurrency:
|
|
790
|
-
|
|
791
|
-
```typescript
|
|
792
|
-
import { LZMAPool } from 'node-liblzma';
|
|
576
|
+
</details>
|
|
793
577
|
|
|
794
|
-
|
|
578
|
+
## Testing
|
|
795
579
|
|
|
796
|
-
|
|
797
|
-
|
|
798
|
-
|
|
799
|
-
|
|
580
|
+
```bash
|
|
581
|
+
pnpm test # Run all 458+ tests
|
|
582
|
+
pnpm test:watch # Watch mode
|
|
583
|
+
pnpm test:coverage # Coverage report
|
|
584
|
+
pnpm type-check # TypeScript type checking
|
|
800
585
|
```
|
|
801
586
|
|
|
802
|
-
|
|
587
|
+
Tests use [Vitest](https://vitest.dev/) with 100% code coverage across statements, branches, functions, and lines.
|
|
803
588
|
|
|
804
|
-
|
|
589
|
+
## Migration Guide
|
|
805
590
|
|
|
806
|
-
|
|
807
|
-
1. Install Visual Studio Build Tools:
|
|
808
|
-
```powershell
|
|
809
|
-
npm install --global windows-build-tools
|
|
810
|
-
```
|
|
591
|
+
### v1.x → v2.0
|
|
811
592
|
|
|
812
|
-
|
|
813
|
-
|
|
814
|
-
npm config set python python3
|
|
815
|
-
```
|
|
593
|
+
<details>
|
|
594
|
+
<summary><strong>Breaking changes and new features</strong></summary>
|
|
816
595
|
|
|
817
|
-
|
|
818
|
-
```powershell
|
|
819
|
-
npm install node-liblzma --build-from-source
|
|
820
|
-
```
|
|
596
|
+
#### Breaking Changes
|
|
821
597
|
|
|
822
|
-
|
|
598
|
+
1. **Node.js >= 16** required (was >= 12)
|
|
599
|
+
2. **ESM module format** (`import` instead of `require`)
|
|
600
|
+
3. **TypeScript source** (CoffeeScript removed)
|
|
823
601
|
|
|
824
|
-
|
|
602
|
+
#### New Features
|
|
825
603
|
|
|
826
|
-
**Solution**: Use forward slashes or `path.join()`:
|
|
827
604
|
```typescript
|
|
828
|
-
|
|
829
|
-
|
|
830
|
-
```
|
|
605
|
+
// Promise-based APIs (recommended)
|
|
606
|
+
const compressed = await xzAsync(buffer);
|
|
831
607
|
|
|
832
|
-
|
|
608
|
+
// Typed error classes
|
|
609
|
+
import { LZMAMemoryError, LZMADataError } from 'node-liblzma';
|
|
833
610
|
|
|
834
|
-
|
|
611
|
+
// Concurrency control
|
|
612
|
+
const pool = new LZMAPool(10);
|
|
613
|
+
const results = await Promise.all(files.map(f => pool.compress(f)));
|
|
835
614
|
|
|
836
|
-
|
|
615
|
+
// File helpers
|
|
616
|
+
await xzFile('input.txt', 'output.txt.xz');
|
|
617
|
+
```
|
|
837
618
|
|
|
838
|
-
|
|
839
|
-
```bash
|
|
840
|
-
git clone https://github.com/oorabona/node-liblzma.git
|
|
841
|
-
cd node-liblzma
|
|
842
|
-
```
|
|
619
|
+
#### Tooling Changes
|
|
843
620
|
|
|
844
|
-
|
|
845
|
-
|
|
846
|
-
|
|
847
|
-
# or
|
|
848
|
-
npm install
|
|
849
|
-
```
|
|
621
|
+
- **Linter**: Biome (replaces ESLint + Prettier)
|
|
622
|
+
- **Tests**: Vitest (replaces Mocha)
|
|
623
|
+
- **Package Manager**: pnpm recommended
|
|
850
624
|
|
|
851
|
-
|
|
852
|
-
```bash
|
|
853
|
-
pnpm build
|
|
854
|
-
```
|
|
625
|
+
</details>
|
|
855
626
|
|
|
856
|
-
|
|
857
|
-
```bash
|
|
858
|
-
pnpm test
|
|
859
|
-
```
|
|
627
|
+
## Contributing
|
|
860
628
|
|
|
861
|
-
|
|
629
|
+
We welcome contributions! See the full [contributing guidelines](#development-workflow) below.
|
|
862
630
|
|
|
863
|
-
###
|
|
631
|
+
### Development Setup
|
|
864
632
|
|
|
865
633
|
```bash
|
|
866
|
-
|
|
634
|
+
git clone https://github.com/oorabona/node-liblzma.git
|
|
635
|
+
cd node-liblzma
|
|
636
|
+
pnpm install
|
|
637
|
+
pnpm build
|
|
867
638
|
pnpm test
|
|
868
|
-
|
|
869
|
-
# Watch mode (re-run on changes)
|
|
870
|
-
pnpm test:watch
|
|
871
|
-
|
|
872
|
-
# Coverage report
|
|
873
|
-
pnpm test:coverage
|
|
874
|
-
|
|
875
|
-
# Interactive UI
|
|
876
|
-
pnpm test:ui
|
|
877
639
|
```
|
|
878
640
|
|
|
879
|
-
###
|
|
880
|
-
|
|
881
|
-
We use [Biome](https://biomejs.dev/) for linting and formatting:
|
|
641
|
+
### Development Workflow
|
|
882
642
|
|
|
883
643
|
```bash
|
|
884
|
-
#
|
|
885
|
-
pnpm
|
|
886
|
-
|
|
887
|
-
#
|
|
888
|
-
pnpm check:write
|
|
889
|
-
|
|
890
|
-
# Lint only
|
|
891
|
-
pnpm lint
|
|
892
|
-
|
|
893
|
-
# Format only
|
|
894
|
-
pnpm format:write
|
|
644
|
+
pnpm test # Run tests
|
|
645
|
+
pnpm test:watch # Watch mode
|
|
646
|
+
pnpm test:coverage # Coverage report
|
|
647
|
+
pnpm check # Lint + format check (Biome)
|
|
648
|
+
pnpm check:write # Auto-fix lint/format
|
|
649
|
+
pnpm type-check # TypeScript types
|
|
895
650
|
```
|
|
896
651
|
|
|
897
|
-
|
|
652
|
+
We follow [Conventional Commits](https://www.conventionalcommits.org/):
|
|
898
653
|
|
|
899
|
-
```
|
|
900
|
-
|
|
654
|
+
```
|
|
655
|
+
feat: add LZMAPool for concurrency control
|
|
656
|
+
fix: resolve memory leak in FunctionReference
|
|
657
|
+
docs: add migration guide for v2.0
|
|
901
658
|
```
|
|
902
659
|
|
|
903
|
-
|
|
660
|
+
### Pull Request Process
|
|
904
661
|
|
|
905
|
-
|
|
906
|
-
|
|
907
|
-
|
|
908
|
-
|
|
662
|
+
1. Fork and create a feature branch (`feat/`, `fix/`, `refactor/`, `docs/`)
|
|
663
|
+
2. Add tests for new functionality (100% coverage required)
|
|
664
|
+
3. Run `pnpm check:write && pnpm type-check && pnpm test`
|
|
665
|
+
4. Commit with conventional commits and push
|
|
666
|
+
5. CI checks run automatically on PR
|
|
909
667
|
|
|
910
|
-
##
|
|
668
|
+
## Troubleshooting
|
|
911
669
|
|
|
912
|
-
|
|
670
|
+
<details>
|
|
671
|
+
<summary><strong>Build issues</strong></summary>
|
|
913
672
|
|
|
673
|
+
**"Cannot find liblzma library"** — Install system dev package:
|
|
674
|
+
```bash
|
|
675
|
+
sudo apt-get install liblzma-dev # Debian/Ubuntu
|
|
676
|
+
brew install xz # macOS
|
|
914
677
|
```
|
|
915
|
-
<type>(<scope>): <description>
|
|
916
|
-
|
|
917
|
-
[optional body]
|
|
918
678
|
|
|
919
|
-
|
|
679
|
+
**"node-gyp rebuild failed"** — Install build tools:
|
|
680
|
+
```bash
|
|
681
|
+
sudo apt-get install build-essential python3 # Linux
|
|
682
|
+
xcode-select --install # macOS
|
|
683
|
+
npm install --global windows-build-tools # Windows
|
|
920
684
|
```
|
|
921
685
|
|
|
922
|
-
**
|
|
923
|
-
- `feat`: New feature
|
|
924
|
-
- `fix`: Bug fix
|
|
925
|
-
- `docs`: Documentation changes
|
|
926
|
-
- `refactor`: Code refactoring
|
|
927
|
-
- `test`: Test changes
|
|
928
|
-
- `chore`: Build/tooling changes
|
|
929
|
-
- `perf`: Performance improvements
|
|
930
|
-
|
|
931
|
-
**Examples**:
|
|
686
|
+
**"Prebuilt binary not found"** — Build from source:
|
|
932
687
|
```bash
|
|
933
|
-
|
|
934
|
-
git commit -m "fix(bindings): resolve memory leak in FunctionReference"
|
|
935
|
-
git commit -m "docs(readme): add migration guide for v2.0"
|
|
688
|
+
npm install node-liblzma --build-from-source
|
|
936
689
|
```
|
|
937
690
|
|
|
938
|
-
|
|
939
|
-
|
|
940
|
-
1. **Fork the repository** and create a feature branch:
|
|
941
|
-
```bash
|
|
942
|
-
git checkout -b feat/my-new-feature
|
|
943
|
-
```
|
|
944
|
-
|
|
945
|
-
2. **Make your changes** following code style guidelines
|
|
946
|
-
|
|
947
|
-
3. **Add tests** for new functionality:
|
|
948
|
-
- All new code must have 100% test coverage
|
|
949
|
-
- Tests go in `test/` directory
|
|
950
|
-
- Use Vitest testing framework
|
|
951
|
-
|
|
952
|
-
4. **Ensure all checks pass**:
|
|
953
|
-
```bash
|
|
954
|
-
pnpm check:write # Fix code style
|
|
955
|
-
pnpm type-check # Verify TypeScript types
|
|
956
|
-
pnpm test # Run test suite
|
|
957
|
-
```
|
|
958
|
-
|
|
959
|
-
5. **Commit with conventional commits**:
|
|
960
|
-
```bash
|
|
961
|
-
git add .
|
|
962
|
-
git commit -m "feat: add new feature"
|
|
963
|
-
```
|
|
964
|
-
|
|
965
|
-
6. **Push and create Pull Request**:
|
|
966
|
-
```bash
|
|
967
|
-
git push origin feat/my-new-feature
|
|
968
|
-
```
|
|
691
|
+
</details>
|
|
969
692
|
|
|
970
|
-
|
|
693
|
+
<details>
|
|
694
|
+
<summary><strong>Runtime issues</strong></summary>
|
|
971
695
|
|
|
972
|
-
|
|
973
|
-
|
|
974
|
-
- **Coverage**: Maintain 100% code coverage (statements, branches, functions, lines)
|
|
975
|
-
- **Test files**: Name tests `*.test.ts` in `test/` directory
|
|
976
|
-
- **Structure**: Use `describe` and `it` blocks with clear descriptions
|
|
977
|
-
- **Assertions**: Use Vitest's `expect()` API
|
|
978
|
-
|
|
979
|
-
**Example test**:
|
|
696
|
+
**LZMAMemoryError** — Input too large for buffer API. Use streaming:
|
|
980
697
|
```typescript
|
|
981
|
-
|
|
982
|
-
import { xzAsync, unxzAsync } from '../src/lzma.js';
|
|
983
|
-
|
|
984
|
-
describe('Compression', () => {
|
|
985
|
-
it('should compress and decompress data', async () => {
|
|
986
|
-
const original = Buffer.from('test data');
|
|
987
|
-
const compressed = await xzAsync(original);
|
|
988
|
-
const decompressed = await unxzAsync(compressed);
|
|
989
|
-
|
|
990
|
-
expect(decompressed.equals(original)).toBe(true);
|
|
991
|
-
});
|
|
992
|
-
});
|
|
698
|
+
createReadStream('large.bin').pipe(createXz()).pipe(createWriteStream('large.xz'));
|
|
993
699
|
```
|
|
994
700
|
|
|
995
|
-
|
|
701
|
+
**LZMADataError** — File is not XZ-compressed or is corrupted. Verify with `file compressed.xz` or `xz -t compressed.xz`.
|
|
996
702
|
|
|
997
|
-
|
|
703
|
+
**Slow on multi-core** — Enable threads: `createXz({ threads: 0 })` (auto-detect cores).
|
|
998
704
|
|
|
999
|
-
|
|
1000
|
-
# Standard release (patch/minor/major based on commits)
|
|
1001
|
-
pnpm release
|
|
705
|
+
**High memory with concurrency** — Use `LZMAPool` to limit simultaneous operations.
|
|
1002
706
|
|
|
1003
|
-
|
|
1004
|
-
pnpm release:manual
|
|
707
|
+
</details>
|
|
1005
708
|
|
|
1006
|
-
|
|
1007
|
-
|
|
709
|
+
<details>
|
|
710
|
+
<summary><strong>Windows-specific</strong></summary>
|
|
1008
711
|
|
|
1009
|
-
|
|
1010
|
-
|
|
712
|
+
**Build fails** — Install Visual Studio Build Tools and set Python:
|
|
713
|
+
```powershell
|
|
714
|
+
npm install --global windows-build-tools
|
|
715
|
+
npm config set python python3
|
|
1011
716
|
```
|
|
1012
717
|
|
|
1013
|
-
**
|
|
718
|
+
**Path issues** — Use `path.join()` instead of hardcoded separators.
|
|
1014
719
|
|
|
1015
|
-
|
|
720
|
+
</details>
|
|
1016
721
|
|
|
1017
|
-
|
|
1018
|
-
- **Bugs**: Open an [Issue](https://github.com/oorabona/node-liblzma/issues)
|
|
1019
|
-
- **Security**: Email security@example.com (do not open public issues)
|
|
722
|
+
## Related Projects
|
|
1020
723
|
|
|
1021
|
-
|
|
724
|
+
- [lzma-purejs](https://github.com/cscott/lzma-purejs) — Pure JavaScript LZMA implementation
|
|
725
|
+
- [node-xz](https://github.com/robey/node-xz) — Node binding of XZ library
|
|
726
|
+
- [lzma-native](https://github.com/addaleax/lzma-native) — Complete XZ library bindings
|
|
1022
727
|
|
|
1023
|
-
|
|
728
|
+
## Bugs
|
|
1024
729
|
|
|
1025
|
-
|
|
730
|
+
If you find one, feel free to contribute and post a new [issue](https://github.com/oorabona/node-liblzma/issues)!
|
|
731
|
+
PRs are accepted as well :)
|
|
1026
732
|
|
|
1027
|
-
If you
|
|
1028
|
-
|
|
733
|
+
If you compile with threads, you may see warnings about `-Wmissing-field-initializers`.
|
|
734
|
+
This is normal and does not prevent threading from being active and working.
|
|
1029
735
|
|
|
1030
|
-
|
|
736
|
+
## Acknowledgements
|
|
1031
737
|
|
|
1032
|
-
|
|
1033
|
-
This is _normal_ and does not prevent threading from being active and working.
|
|
1034
|
-
I did not yet figure how to fix this except by masking the warning..
|
|
738
|
+
Kudos to [addaleax](https://github.com/addaleax) for helping out with C++ stuff!
|
|
1035
739
|
|
|
1036
|
-
|
|
740
|
+
## License
|
|
1037
741
|
|
|
1038
|
-
This software is released under [
|
|
742
|
+
This software is released under [LGPL-3.0+](LICENSE).
|