node-liblzma 1.1.9 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/settings.local.json +92 -0
- package/.gitattributes +3 -0
- package/.release-it.json +6 -0
- package/CHANGELOG.md +209 -0
- package/History.md +20 -0
- package/README.md +750 -30
- package/RELEASING.md +131 -0
- package/binding.gyp +159 -438
- package/biome.json +81 -0
- package/coverage/base.css +224 -0
- package/coverage/block-navigation.js +87 -0
- package/coverage/errors.ts.html +586 -0
- package/coverage/favicon.png +0 -0
- package/coverage/index.html +146 -0
- package/coverage/lcov-report/base.css +224 -0
- package/coverage/lcov-report/block-navigation.js +87 -0
- package/coverage/lcov-report/errors.ts.html +586 -0
- package/coverage/lcov-report/favicon.png +0 -0
- package/coverage/lcov-report/index.html +146 -0
- package/coverage/lcov-report/lzma.ts.html +2596 -0
- package/coverage/lcov-report/pool.ts.html +769 -0
- package/coverage/lcov-report/prettify.css +1 -0
- package/coverage/lcov-report/prettify.js +2 -0
- package/coverage/lcov-report/sort-arrow-sprite.png +0 -0
- package/coverage/lcov-report/sorter.js +210 -0
- package/coverage/lcov.info +636 -0
- package/coverage/lzma.ts.html +2596 -0
- package/coverage/pool.ts.html +769 -0
- package/coverage/prettify.css +1 -0
- package/coverage/prettify.js +2 -0
- package/coverage/sort-arrow-sprite.png +0 -0
- package/coverage/sorter.js +210 -0
- package/coverage-reports/assets/monocart-coverage-app.js +2 -0
- package/coverage-reports/coverage-data.js +1 -0
- package/coverage-reports/index.html +48 -0
- package/err.log +26 -0
- package/index.d.ts +254 -0
- package/lib/errors.d.ts +72 -0
- package/lib/errors.d.ts.map +1 -0
- package/lib/errors.js +153 -0
- package/lib/errors.js.map +1 -0
- package/lib/lzma.d.ts +245 -0
- package/lib/lzma.d.ts.map +1 -0
- package/lib/lzma.js +626 -345
- package/lib/lzma.js.map +1 -0
- package/lib/pool.d.ts +123 -0
- package/lib/pool.d.ts.map +1 -0
- package/lib/pool.js +188 -0
- package/lib/pool.js.map +1 -0
- package/lib/types.d.ts +27 -0
- package/lib/types.d.ts.map +1 -0
- package/lib/types.js +5 -0
- package/lib/types.js.map +1 -0
- package/package.json +60 -21
- package/pnpm-workspace.yaml +3 -0
- package/scripts/analyze-coverage.js +132 -0
- package/scripts/build_xz_with_cmake.py +390 -0
- package/scripts/compare-coverage-tools.js +93 -0
- package/scripts/copy_dll.py +51 -0
- package/scripts/download_xz_from_github.py +375 -0
- package/src/bindings/node-liblzma.cpp +411 -229
- package/src/bindings/node-liblzma.hpp +101 -48
- package/src/errors.ts +167 -0
- package/src/lzma.ts +839 -0
- package/src/pool.ts +228 -0
- package/src/types.ts +30 -0
- package/tsconfig.json +50 -0
- package/vitest.config.istanbul.ts +29 -0
- package/vitest.config.monocart.ts +44 -0
- package/vitest.config.ts +44 -0
- package/xz-version.json +8 -0
- package/prebuilds/darwin-x64/node.napi.node +0 -0
- package/prebuilds/linux-x64/node.napi.node +0 -0
- package/prebuilds/win32-x64/node.napi.node +0 -0
- package/scripts/download_extract_deps.py +0 -29
- package/src/lzma.coffee +0 -344
package/README.md
CHANGED
|
@@ -3,13 +3,13 @@ Node-liblzma
|
|
|
3
3
|
|
|
4
4
|
[](https://npmjs.org/package/node-liblzma)
|
|
5
5
|
[](https://npmjs.org/package/node-liblzma)
|
|
6
|
-
[](https://github.com/oorabona/node-liblzma/actions/workflows/ci-windows.yml)
|
|
6
|
+
[](https://github.com/oorabona/node-liblzma/actions/workflows/ci-unified.yml)
|
|
7
|
+
[](#testing)
|
|
9
8
|
|
|
10
9
|
# What is liblzma/XZ ?
|
|
11
10
|
|
|
12
11
|
[XZ](https://tukaani.org/xz/xz-file-format.txt) is a container for compressed archives. It is among the best compressors out there according to several benchmarks:
|
|
12
|
+
|
|
13
13
|
* [Gzip vs Bzip2 vs LZMA vs XZ vs LZ4 vs LZO](http://pokecraft.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO)
|
|
14
14
|
* [Large Text Compression Benchmark](http://mattmahoney.net/dc/text.html#2118)
|
|
15
15
|
* [Linux Compression Comparison (GZIP vs BZIP2 vs LZMA vs ZIP vs Compress)](http://bashitout.com/2009/08/30/Linux-Compression-Comparison-GZIP-vs-BZIP2-vs-LZMA-vs-ZIP-vs-Compress.html)
|
|
@@ -19,6 +19,7 @@ It has a good balance between compression time/ratio and decompression time/memo
|
|
|
19
19
|
# About this project
|
|
20
20
|
|
|
21
21
|
This project aims towards providing:
|
|
22
|
+
|
|
22
23
|
* A quick and easy way to play with XZ compression:
|
|
23
24
|
Quick and easy as it conforms to zlib API, so that switching from __zlib/deflate__ to __xz__ might be as easy as a string search/replace in your code editor :smile:
|
|
24
25
|
|
|
@@ -31,31 +32,46 @@ But the library can open and read any LZMA1 or LZMA2 compressed file.
|
|
|
31
32
|
|
|
32
33
|
# What's new ?
|
|
33
34
|
|
|
34
|
-
|
|
35
|
-
|
|
35
|
+
## Version 2.0 (2025) - Complete Modernization
|
|
36
|
+
|
|
37
|
+
This major release brings the library into 2025 with modern tooling and TypeScript support:
|
|
36
38
|
|
|
37
|
-
|
|
39
|
+
* **Full TypeScript migration**: Complete rewrite from CoffeeScript to TypeScript for better type safety and developer experience
|
|
40
|
+
* **Promise-based APIs**: New async functions `xzAsync()` and `unxzAsync()` with Promise support
|
|
41
|
+
* **Modern testing**: Migrated from Mocha to Vitest with improved performance and better TypeScript integration
|
|
42
|
+
* **Enhanced tooling**:
|
|
43
|
+
- [Biome](https://biomejs.dev/) for fast linting and formatting
|
|
44
|
+
- Pre-commit hooks with nano-staged and simple-git-hooks
|
|
45
|
+
- pnpm as package manager for better dependency management
|
|
46
|
+
* **Updated Node.js support**: Requires Node.js >= 16 (updated from >= 12)
|
|
38
47
|
|
|
39
|
-
|
|
48
|
+
## Legacy (N-API migration)
|
|
49
|
+
|
|
50
|
+
In previous versions, [N-API](https://nodejs.org/api/n-api.html) became the _de facto_ standard to provide stable ABI API for NodeJS Native Modules, replacing [nan](https://github.com/nodejs/nan).
|
|
40
51
|
|
|
41
52
|
It has been tested and works on:
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
53
|
+
|
|
54
|
+
* Linux x64 (Ubuntu)
|
|
55
|
+
* OSX (`macos-11`)
|
|
56
|
+
* Raspberry Pi 2/3/4 (both on 32-bit and 64-bit architectures)
|
|
57
|
+
* Windows (`windows-2019` and `windows-2022` are part of GitHub CI)
|
|
46
58
|
|
|
47
59
|
> Notes:
|
|
48
|
-
>
|
|
60
|
+
>
|
|
61
|
+
> * For [Windows](https://github.com/oorabona/node-liblzma/actions/workflows/ci-windows.yml)
|
|
49
62
|
> There is no "global" installation of the LZMA library on the Windows machine provisionned by GitHub, so it is pointless to build with this config
|
|
50
|
-
|
|
51
|
-
|
|
63
|
+
>
|
|
64
|
+
* For [Linux](https://github.com/oorabona/node-liblzma/actions/workflows/ci-linux.yml)
|
|
65
|
+
|
|
66
|
+
* For [MacOS](https://github.com/oorabona/node-liblzma/actions/workflows/ci-macos.yml)
|
|
52
67
|
|
|
53
68
|
## Prebuilt images
|
|
54
69
|
|
|
55
70
|
Several prebuilt versions are bundled within the package.
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
71
|
+
|
|
72
|
+
* Windows x86_64
|
|
73
|
+
* Linux x86_64
|
|
74
|
+
* MacOS x86_64 / Arm64
|
|
59
75
|
|
|
60
76
|
If your OS/architecture matches, you will use this version which has been compiled using the following default flags:
|
|
61
77
|
|
|
@@ -72,6 +88,7 @@ If you want to change compilation flags, please read on [here](#installation).
|
|
|
72
88
|
# Related projects
|
|
73
89
|
|
|
74
90
|
Thanks to the community, there are several choices out there:
|
|
91
|
+
|
|
75
92
|
* [lzma-purejs](https://github.com/cscott/lzma-purejs)
|
|
76
93
|
A pure JavaScript implementation of the algorithm
|
|
77
94
|
* [node-xz](https://github.com/robey/node-xz)
|
|
@@ -83,7 +100,11 @@ A very complete implementation of XZ library bindings
|
|
|
83
100
|
# API comparison
|
|
84
101
|
|
|
85
102
|
```js
|
|
103
|
+
// CommonJS
|
|
86
104
|
var lzma = require('node-liblzma');
|
|
105
|
+
|
|
106
|
+
// TypeScript / ES6 modules
|
|
107
|
+
import * as lzma from 'node-liblzma';
|
|
87
108
|
```
|
|
88
109
|
|
|
89
110
|
Zlib | XZlib | Arguments
|
|
@@ -94,6 +115,8 @@ gzip | xz | (buf, [options], callback)
|
|
|
94
115
|
gunzip | unxz | (buf, [options], callback)
|
|
95
116
|
gzipSync | xzSync | (buf, [options])
|
|
96
117
|
gunzipSync | unxzSync | (buf, [options])
|
|
118
|
+
- | xzAsync | (buf, [options]) ⇒ Promise\<Buffer>
|
|
119
|
+
- | unxzAsync | (buf, [options]) ⇒ Promise\<Buffer>
|
|
97
120
|
|
|
98
121
|
## Constants
|
|
99
122
|
|
|
@@ -123,30 +146,238 @@ filters | Array | LZMA2 (added by default)
|
|
|
123
146
|
|
|
124
147
|
For further information about each of these flags, you will find reference at [XZ SDK](http://7-zip.org/sdk.html).
|
|
125
148
|
|
|
149
|
+
## Advanced Configuration
|
|
150
|
+
|
|
151
|
+
### Thread Support
|
|
152
|
+
|
|
153
|
+
The library supports multi-threaded compression when built with `ENABLE_THREAD_SUPPORT=yes` (default). Thread support allows parallel compression on multi-core systems, significantly improving performance for large files.
|
|
154
|
+
|
|
155
|
+
**Using threads in compression:**
|
|
156
|
+
|
|
157
|
+
```typescript
|
|
158
|
+
import { xz, createXz } from 'node-liblzma';
|
|
159
|
+
|
|
160
|
+
// Specify number of threads (1-N, where N is CPU core count)
|
|
161
|
+
const options = {
|
|
162
|
+
preset: lzma.preset.DEFAULT,
|
|
163
|
+
threads: 4 // Use 4 threads for compression
|
|
164
|
+
};
|
|
165
|
+
|
|
166
|
+
// With buffer compression
|
|
167
|
+
xz(buffer, options, (err, compressed) => {
|
|
168
|
+
// ...
|
|
169
|
+
});
|
|
170
|
+
|
|
171
|
+
// With streams
|
|
172
|
+
const compressor = createXz(options);
|
|
173
|
+
inputStream.pipe(compressor).pipe(outputStream);
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
**Important notes:**
|
|
177
|
+
- Thread support only applies to **compression**, not decompression
|
|
178
|
+
- Requires LZMA library built with pthread support
|
|
179
|
+
- `threads: 1` disables multi-threading (falls back to single-threaded encoder)
|
|
180
|
+
- Check if threads are available: `import { hasThreads } from 'node-liblzma';`
|
|
181
|
+
|
|
182
|
+
### Buffer Size Optimization
|
|
183
|
+
|
|
184
|
+
For optimal performance, the library uses configurable chunk sizes:
|
|
185
|
+
|
|
186
|
+
```typescript
|
|
187
|
+
const stream = createXz({
|
|
188
|
+
preset: lzma.preset.DEFAULT,
|
|
189
|
+
chunkSize: 256 * 1024 // 256KB chunks (default: 64KB)
|
|
190
|
+
});
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
**Recommendations:**
|
|
194
|
+
- **Small files (< 1MB)**: Use default 64KB chunks
|
|
195
|
+
- **Medium files (1-10MB)**: Use 128-256KB chunks
|
|
196
|
+
- **Large files (> 10MB)**: Use 512KB-1MB chunks
|
|
197
|
+
- **Maximum buffer size**: 512MB per operation (security limit)
|
|
198
|
+
|
|
199
|
+
### Memory Usage Limits
|
|
200
|
+
|
|
201
|
+
The library enforces a 512MB maximum buffer size to prevent DoS attacks via resource exhaustion. For files larger than 512MB, use streaming APIs:
|
|
202
|
+
|
|
203
|
+
```typescript
|
|
204
|
+
import { createReadStream, createWriteStream } from 'fs';
|
|
205
|
+
import { createXz } from 'node-liblzma';
|
|
206
|
+
|
|
207
|
+
createReadStream('large-file.bin')
|
|
208
|
+
.pipe(createXz())
|
|
209
|
+
.pipe(createWriteStream('large-file.xz'));
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
### Error Handling
|
|
213
|
+
|
|
214
|
+
The library provides typed error classes for better error handling:
|
|
215
|
+
|
|
216
|
+
```typescript
|
|
217
|
+
import {
|
|
218
|
+
xzAsync,
|
|
219
|
+
LZMAError,
|
|
220
|
+
LZMAMemoryError,
|
|
221
|
+
LZMADataError,
|
|
222
|
+
LZMAFormatError
|
|
223
|
+
} from 'node-liblzma';
|
|
224
|
+
|
|
225
|
+
try {
|
|
226
|
+
const compressed = await xzAsync(buffer);
|
|
227
|
+
} catch (error) {
|
|
228
|
+
if (error instanceof LZMAMemoryError) {
|
|
229
|
+
console.error('Out of memory:', error.message);
|
|
230
|
+
} else if (error instanceof LZMADataError) {
|
|
231
|
+
console.error('Corrupt data:', error.message);
|
|
232
|
+
} else if (error instanceof LZMAFormatError) {
|
|
233
|
+
console.error('Invalid format:', error.message);
|
|
234
|
+
} else {
|
|
235
|
+
console.error('Unknown error:', error);
|
|
236
|
+
}
|
|
237
|
+
}
|
|
238
|
+
```
|
|
239
|
+
|
|
240
|
+
**Available error classes:**
|
|
241
|
+
- `LZMAError` - Base error class
|
|
242
|
+
- `LZMAMemoryError` - Memory allocation failed
|
|
243
|
+
- `LZMAMemoryLimitError` - Memory limit exceeded
|
|
244
|
+
- `LZMAFormatError` - Unrecognized file format
|
|
245
|
+
- `LZMAOptionsError` - Invalid compression options
|
|
246
|
+
- `LZMADataError` - Corrupt compressed data
|
|
247
|
+
- `LZMABufferError` - Buffer size issues
|
|
248
|
+
- `LZMAProgrammingError` - Internal errors
|
|
249
|
+
|
|
250
|
+
### Error Recovery
|
|
251
|
+
|
|
252
|
+
Streams automatically handle recoverable errors and provide state transition hooks:
|
|
253
|
+
|
|
254
|
+
```typescript
|
|
255
|
+
const decompressor = createUnxz();
|
|
256
|
+
|
|
257
|
+
decompressor.on('error', (error) => {
|
|
258
|
+
console.error('Decompression error:', error.errno, error.message);
|
|
259
|
+
// Stream will emit 'close' event after error
|
|
260
|
+
});
|
|
261
|
+
|
|
262
|
+
decompressor.on('close', () => {
|
|
263
|
+
console.log('Stream closed, safe to cleanup');
|
|
264
|
+
});
|
|
265
|
+
```
|
|
266
|
+
|
|
267
|
+
### Concurrency Control with LZMAPool
|
|
268
|
+
|
|
269
|
+
For production environments with high concurrency needs, use `LZMAPool` to limit simultaneous operations:
|
|
270
|
+
|
|
271
|
+
```typescript
|
|
272
|
+
import { LZMAPool } from 'node-liblzma';
|
|
273
|
+
|
|
274
|
+
const pool = new LZMAPool(10); // Max 10 concurrent operations
|
|
275
|
+
|
|
276
|
+
// Monitor pool metrics
|
|
277
|
+
pool.on('metrics', (metrics) => {
|
|
278
|
+
console.log(`Active: ${metrics.active}, Queued: ${metrics.queued}`);
|
|
279
|
+
console.log(`Completed: ${metrics.completed}, Failed: ${metrics.failed}`);
|
|
280
|
+
});
|
|
281
|
+
|
|
282
|
+
// Compress with automatic queuing
|
|
283
|
+
const compressed = await pool.compress(buffer);
|
|
284
|
+
const decompressed = await pool.decompress(compressed);
|
|
285
|
+
|
|
286
|
+
// Get current metrics
|
|
287
|
+
const status = pool.getMetrics();
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
**Pool Events:**
|
|
291
|
+
- `queue` - Task added to queue
|
|
292
|
+
- `start` - Task started processing
|
|
293
|
+
- `complete` - Task completed successfully
|
|
294
|
+
- `error-task` - Task failed
|
|
295
|
+
- `metrics` - Metrics updated (after each state change)
|
|
296
|
+
|
|
297
|
+
**Benefits:**
|
|
298
|
+
- ✅ Automatic backpressure
|
|
299
|
+
- ✅ Prevents resource exhaustion
|
|
300
|
+
- ✅ Production-ready monitoring
|
|
301
|
+
- ✅ Zero breaking changes (opt-in)
|
|
302
|
+
|
|
303
|
+
### File Compression Helpers
|
|
304
|
+
|
|
305
|
+
Simplified API for file-based compression:
|
|
306
|
+
|
|
307
|
+
```typescript
|
|
308
|
+
import { xzFile, unxzFile } from 'node-liblzma';
|
|
309
|
+
|
|
310
|
+
// Compress a file
|
|
311
|
+
await xzFile('input.txt', 'output.txt.xz');
|
|
312
|
+
|
|
313
|
+
// Decompress a file
|
|
314
|
+
await unxzFile('output.txt.xz', 'restored.txt');
|
|
315
|
+
|
|
316
|
+
// With options
|
|
317
|
+
await xzFile('large-file.bin', 'compressed.xz', {
|
|
318
|
+
preset: 9,
|
|
319
|
+
threads: 4
|
|
320
|
+
});
|
|
321
|
+
```
|
|
322
|
+
|
|
323
|
+
**Advantages over buffer APIs:**
|
|
324
|
+
- ✅ Handles files > 512MB automatically
|
|
325
|
+
- ✅ Built-in backpressure via streams
|
|
326
|
+
- ✅ Lower memory footprint
|
|
327
|
+
- ✅ Simpler API for common use cases
|
|
328
|
+
|
|
329
|
+
## Async callback contract (errno-based)
|
|
330
|
+
|
|
331
|
+
The low-level native callback used internally by streams follows an errno-style contract to match liblzma behavior and to avoid mixing exception channels:
|
|
332
|
+
|
|
333
|
+
- Signature: `(errno: number, availInAfter: number, availOutAfter: number)`
|
|
334
|
+
- Success: `errno` is either `LZMA_OK` or `LZMA_STREAM_END`.
|
|
335
|
+
- Recoverable/other conditions: any other `errno` value (for example, `LZMA_BUF_ERROR`, `LZMA_DATA_ERROR`, `LZMA_PROG_ERROR`) indicates an error state.
|
|
336
|
+
- Streams emit `onerror` with the numeric `errno` when `errno !== LZMA_OK && errno !== LZMA_STREAM_END`.
|
|
337
|
+
|
|
338
|
+
Why errno instead of JS exceptions?
|
|
339
|
+
|
|
340
|
+
- The binding mirrors liblzma’s status codes and keeps a single error channel that’s easy to reason about in tight processing loops.
|
|
341
|
+
- This avoids throwing across async worker boundaries and keeps cleanup deterministic.
|
|
342
|
+
|
|
343
|
+
High-level APIs remain ergonomic:
|
|
344
|
+
|
|
345
|
+
- Promise-based functions `xzAsync()`/`unxzAsync()` still resolve to `Buffer` or reject with `Error` as expected.
|
|
346
|
+
- Stream users can listen to `error` events, where we map `errno` to a human-friendly message (`messages[errno]`).
|
|
347
|
+
|
|
348
|
+
If you prefer Node’s error-first callbacks, you can wrap the APIs and translate `errno` to `Error` objects at your boundaries without changing the native layer.
|
|
349
|
+
|
|
126
350
|
# Installation
|
|
127
351
|
|
|
128
352
|
Well, as simple as this one-liner:
|
|
129
353
|
|
|
130
354
|
```sh
|
|
131
|
-
|
|
355
|
+
npm i node-liblzma --save
|
|
132
356
|
```
|
|
133
357
|
|
|
134
358
|
--OR--
|
|
135
359
|
|
|
136
360
|
```sh
|
|
137
|
-
|
|
361
|
+
yarn add node-liblzma
|
|
362
|
+
```
|
|
363
|
+
|
|
364
|
+
--OR-- (recommended for development)
|
|
365
|
+
|
|
366
|
+
```sh
|
|
367
|
+
pnpm add node-liblzma
|
|
138
368
|
```
|
|
139
369
|
|
|
140
370
|
If you want to recompile the source, for example to disable threading support in the module, then you have to opt out with:
|
|
141
371
|
|
|
142
372
|
``` bash
|
|
143
|
-
|
|
373
|
+
ENABLE_THREAD_SUPPORT=no npm install node-liblzma --build-from-source
|
|
144
374
|
```
|
|
145
375
|
|
|
146
376
|
> Note:
|
|
147
|
-
Enabling thread support in the library will
|
|
377
|
+
Enabling thread support in the library will __NOT__ work if the LZMA library itself has been built without such support.
|
|
148
378
|
|
|
149
379
|
To build the module, you have the following options:
|
|
380
|
+
|
|
150
381
|
1. Using system development libraries
|
|
151
382
|
2. Ask the build system to download `xz` and build it
|
|
152
383
|
3. Compile `xz` yourself, outside `node-liblzma`, and have it use it after
|
|
@@ -166,7 +397,7 @@ If you do not plan on having a local install, you can ask for automatic download
|
|
|
166
397
|
Just do:
|
|
167
398
|
|
|
168
399
|
```sh
|
|
169
|
-
|
|
400
|
+
npm install node-liblzma --build-from-source
|
|
170
401
|
```
|
|
171
402
|
|
|
172
403
|
When no option is given in the commandline arguments, it will build with default values.
|
|
@@ -178,9 +409,9 @@ So you did install `xz` somewhere outside the module and want the module to use
|
|
|
178
409
|
For that, you need to set the include directory and library directory search paths as GCC [environment variables](https://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html).
|
|
179
410
|
|
|
180
411
|
```sh
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
412
|
+
export CPATH=$HOME/path/to/headers
|
|
413
|
+
export LIBRARY_PATH=$HOME/path/to/lib
|
|
414
|
+
export LD_LIBRARY_PATH=$HOME/path/to/lib:$LD_LIBRARY_PATH
|
|
184
415
|
```
|
|
185
416
|
|
|
186
417
|
The latest is needed for tests to be run right after.
|
|
@@ -188,18 +419,35 @@ The latest is needed for tests to be run right after.
|
|
|
188
419
|
Once done, this should suffice:
|
|
189
420
|
|
|
190
421
|
```sh
|
|
191
|
-
|
|
422
|
+
npm install
|
|
192
423
|
```
|
|
193
424
|
|
|
194
|
-
#
|
|
425
|
+
# Testing
|
|
426
|
+
|
|
427
|
+
This project maintains **100% code coverage** across all statements, branches, functions, and lines.
|
|
195
428
|
|
|
196
429
|
You can run tests with:
|
|
197
430
|
|
|
198
431
|
```sh
|
|
199
|
-
|
|
432
|
+
npm test
|
|
433
|
+
# or
|
|
434
|
+
pnpm test
|
|
200
435
|
```
|
|
201
436
|
|
|
202
|
-
It will build and launch
|
|
437
|
+
It will build and launch the test suite (51 tests) with [Vitest](https://vitest.dev/) with TypeScript support and coverage reporting.
|
|
438
|
+
|
|
439
|
+
Additional testing commands:
|
|
440
|
+
|
|
441
|
+
```sh
|
|
442
|
+
# Watch mode for development
|
|
443
|
+
pnpm test:watch
|
|
444
|
+
|
|
445
|
+
# Coverage report
|
|
446
|
+
pnpm test:coverage
|
|
447
|
+
|
|
448
|
+
# Type checking
|
|
449
|
+
pnpm type-check
|
|
450
|
+
```
|
|
203
451
|
|
|
204
452
|
# Usage
|
|
205
453
|
|
|
@@ -207,7 +455,479 @@ As the API is very close to NodeJS Zlib, you will probably find a good reference
|
|
|
207
455
|
[there](http://www.nodejs.org/api/zlib.html).
|
|
208
456
|
|
|
209
457
|
Otherwise examples can be found as part of the test suite, so feel free to use them!
|
|
210
|
-
They are written in
|
|
458
|
+
They are written in TypeScript with full type definitions.
|
|
459
|
+
|
|
460
|
+
# Migration Guide
|
|
461
|
+
|
|
462
|
+
## Migrating from v1.x to v2.0
|
|
463
|
+
|
|
464
|
+
Version 2.0 introduces several breaking changes along with powerful new features.
|
|
465
|
+
|
|
466
|
+
### Breaking Changes
|
|
467
|
+
|
|
468
|
+
1. **Node.js Version Requirement**
|
|
469
|
+
```diff
|
|
470
|
+
- Requires Node.js >= 12
|
|
471
|
+
+ Requires Node.js >= 16
|
|
472
|
+
```
|
|
473
|
+
|
|
474
|
+
2. **ESM Module Format**
|
|
475
|
+
```diff
|
|
476
|
+
- CommonJS: var lzma = require('node-liblzma');
|
|
477
|
+
+ ESM: import * as lzma from 'node-liblzma';
|
|
478
|
+
+ CommonJS still works via dynamic import
|
|
479
|
+
```
|
|
480
|
+
|
|
481
|
+
3. **TypeScript Migration**
|
|
482
|
+
- Source code migrated from CoffeeScript to TypeScript
|
|
483
|
+
- Full type definitions included
|
|
484
|
+
- Better IDE autocomplete and type safety
|
|
485
|
+
|
|
486
|
+
### New Features You Should Adopt
|
|
487
|
+
|
|
488
|
+
1. **Promise-based APIs** (Recommended for new code)
|
|
489
|
+
```typescript
|
|
490
|
+
// Old callback style (still works)
|
|
491
|
+
xz(buffer, (err, compressed) => {
|
|
492
|
+
if (err) throw err;
|
|
493
|
+
// use compressed
|
|
494
|
+
});
|
|
495
|
+
|
|
496
|
+
// New Promise style
|
|
497
|
+
try {
|
|
498
|
+
const compressed = await xzAsync(buffer);
|
|
499
|
+
// use compressed
|
|
500
|
+
} catch (err) {
|
|
501
|
+
// handle error
|
|
502
|
+
}
|
|
503
|
+
```
|
|
504
|
+
|
|
505
|
+
2. **Typed Error Classes** (Better error handling)
|
|
506
|
+
```typescript
|
|
507
|
+
import { LZMAMemoryError, LZMADataError } from 'node-liblzma';
|
|
508
|
+
|
|
509
|
+
try {
|
|
510
|
+
await unxzAsync(corruptData);
|
|
511
|
+
} catch (error) {
|
|
512
|
+
if (error instanceof LZMADataError) {
|
|
513
|
+
console.error('Corrupt compressed data');
|
|
514
|
+
} else if (error instanceof LZMAMemoryError) {
|
|
515
|
+
console.error('Out of memory');
|
|
516
|
+
}
|
|
517
|
+
}
|
|
518
|
+
```
|
|
519
|
+
|
|
520
|
+
3. **Concurrency Control** (For high-throughput applications)
|
|
521
|
+
```typescript
|
|
522
|
+
import { LZMAPool } from 'node-liblzma';
|
|
523
|
+
|
|
524
|
+
const pool = new LZMAPool(10); // Max 10 concurrent operations
|
|
525
|
+
|
|
526
|
+
// Automatic queuing and backpressure
|
|
527
|
+
const results = await Promise.all(
|
|
528
|
+
files.map(file => pool.compress(file))
|
|
529
|
+
);
|
|
530
|
+
```
|
|
531
|
+
|
|
532
|
+
4. **File Helpers** (Simpler file compression)
|
|
533
|
+
```typescript
|
|
534
|
+
import { xzFile, unxzFile } from 'node-liblzma';
|
|
535
|
+
|
|
536
|
+
// Compress a file (handles streaming automatically)
|
|
537
|
+
await xzFile('input.txt', 'output.txt.xz');
|
|
538
|
+
|
|
539
|
+
// Decompress a file
|
|
540
|
+
await unxzFile('output.txt.xz', 'restored.txt');
|
|
541
|
+
```
|
|
542
|
+
|
|
543
|
+
### Testing Framework Change
|
|
544
|
+
|
|
545
|
+
If you maintain tests for code using node-liblzma:
|
|
546
|
+
|
|
547
|
+
```diff
|
|
548
|
+
- Mocha test framework
|
|
549
|
+
+ Vitest test framework (faster, better TypeScript support)
|
|
550
|
+
```
|
|
551
|
+
|
|
552
|
+
### Tooling Updates
|
|
553
|
+
|
|
554
|
+
Development tooling has been modernized:
|
|
555
|
+
|
|
556
|
+
- **Linter**: Biome (replaces ESLint + Prettier)
|
|
557
|
+
- **Package Manager**: pnpm recommended (npm/yarn still work)
|
|
558
|
+
- **Pre-commit Hooks**: nano-staged + simple-git-hooks
|
|
559
|
+
|
|
560
|
+
# Troubleshooting
|
|
561
|
+
|
|
562
|
+
## Common Build Issues
|
|
563
|
+
|
|
564
|
+
### Issue: "Cannot find liblzma library"
|
|
565
|
+
|
|
566
|
+
**Solution**: Install system development package or let node-gyp download it:
|
|
567
|
+
|
|
568
|
+
```bash
|
|
569
|
+
# Debian/Ubuntu
|
|
570
|
+
sudo apt-get install liblzma-dev
|
|
571
|
+
|
|
572
|
+
# macOS
|
|
573
|
+
brew install xz
|
|
574
|
+
|
|
575
|
+
# Windows (let node-gyp download and build)
|
|
576
|
+
npm install node-liblzma --build-from-source
|
|
577
|
+
```
|
|
578
|
+
|
|
579
|
+
### Issue: "node-gyp rebuild failed"
|
|
580
|
+
|
|
581
|
+
**Symptoms**: Build fails with C++ compilation errors
|
|
582
|
+
|
|
583
|
+
**Solutions**:
|
|
584
|
+
1. Install build tools:
|
|
585
|
+
```bash
|
|
586
|
+
# Ubuntu/Debian
|
|
587
|
+
sudo apt-get install build-essential python3
|
|
588
|
+
|
|
589
|
+
# macOS (install Xcode Command Line Tools)
|
|
590
|
+
xcode-select --install
|
|
591
|
+
|
|
592
|
+
# Windows
|
|
593
|
+
npm install --global windows-build-tools
|
|
594
|
+
```
|
|
595
|
+
|
|
596
|
+
2. Clear build cache and retry:
|
|
597
|
+
```bash
|
|
598
|
+
rm -rf build node_modules
|
|
599
|
+
npm install
|
|
600
|
+
```
|
|
601
|
+
|
|
602
|
+
### Issue: "Prebuilt binary not found"
|
|
603
|
+
|
|
604
|
+
**Solution**: Your platform might not have prebuilt binaries. Build from source:
|
|
605
|
+
|
|
606
|
+
```bash
|
|
607
|
+
npm install node-liblzma --build-from-source
|
|
608
|
+
```
|
|
609
|
+
|
|
610
|
+
## Runtime Issues
|
|
611
|
+
|
|
612
|
+
### Issue: "Memory allocation failed" (LZMAMemoryError)
|
|
613
|
+
|
|
614
|
+
**Causes**:
|
|
615
|
+
- Input buffer exceeds 512MB limit (security protection)
|
|
616
|
+
- System out of memory
|
|
617
|
+
- Trying to decompress extremely large archive
|
|
618
|
+
|
|
619
|
+
**Solutions**:
|
|
620
|
+
1. For files > 512MB, use streaming APIs:
|
|
621
|
+
```typescript
|
|
622
|
+
import { createReadStream, createWriteStream } from 'fs';
|
|
623
|
+
import { createXz } from 'node-liblzma';
|
|
624
|
+
|
|
625
|
+
createReadStream('large-file.bin')
|
|
626
|
+
.pipe(createXz())
|
|
627
|
+
.pipe(createWriteStream('large-file.xz'));
|
|
628
|
+
```
|
|
629
|
+
|
|
630
|
+
2. Or use file helpers (automatically handle large files):
|
|
631
|
+
```typescript
|
|
632
|
+
await xzFile('large-file.bin', 'large-file.xz');
|
|
633
|
+
```
|
|
634
|
+
|
|
635
|
+
### Issue: "Corrupt compressed data" (LZMADataError)
|
|
636
|
+
|
|
637
|
+
**Symptoms**: Decompression fails with `LZMADataError`
|
|
638
|
+
|
|
639
|
+
**Causes**:
|
|
640
|
+
- File is not actually XZ/LZMA compressed
|
|
641
|
+
- File is corrupted or incomplete
|
|
642
|
+
- Wrong file format (LZMA1 vs LZMA2)
|
|
643
|
+
|
|
644
|
+
**Solutions**:
|
|
645
|
+
1. Verify file format:
|
|
646
|
+
```bash
|
|
647
|
+
file compressed.xz
|
|
648
|
+
# Should show: "XZ compressed data"
|
|
649
|
+
```
|
|
650
|
+
|
|
651
|
+
2. Check file integrity:
|
|
652
|
+
```bash
|
|
653
|
+
xz -t compressed.xz
|
|
654
|
+
```
|
|
655
|
+
|
|
656
|
+
3. Handle errors gracefully:
|
|
657
|
+
```typescript
|
|
658
|
+
try {
|
|
659
|
+
const data = await unxzAsync(buffer);
|
|
660
|
+
} catch (error) {
|
|
661
|
+
if (error instanceof LZMADataError) {
|
|
662
|
+
console.error('Invalid or corrupt XZ file');
|
|
663
|
+
}
|
|
664
|
+
}
|
|
665
|
+
```
|
|
666
|
+
|
|
667
|
+
### Issue: Thread support warnings during compilation
|
|
668
|
+
|
|
669
|
+
**Symptoms**: Compiler warnings about `-Wmissing-field-initializers`
|
|
670
|
+
|
|
671
|
+
**Status**: This is normal and does not affect functionality. Thread support still works correctly.
|
|
672
|
+
|
|
673
|
+
**Disable thread support** (if warnings are problematic):
|
|
674
|
+
```bash
|
|
675
|
+
ENABLE_THREAD_SUPPORT=no npm install node-liblzma --build-from-source
|
|
676
|
+
```
|
|
677
|
+
|
|
678
|
+
## Performance Issues
|
|
679
|
+
|
|
680
|
+
### Issue: Compression is slow on multi-core systems
|
|
681
|
+
|
|
682
|
+
**Solution**: Enable multi-threaded compression:
|
|
683
|
+
|
|
684
|
+
```typescript
|
|
685
|
+
import { xz } from 'node-liblzma';
|
|
686
|
+
|
|
687
|
+
xz(buffer, { threads: 4 }, (err, compressed) => {
|
|
688
|
+
// 4 threads used for compression
|
|
689
|
+
});
|
|
690
|
+
```
|
|
691
|
+
|
|
692
|
+
**Note**: Threads only apply to compression, not decompression.
|
|
693
|
+
|
|
694
|
+
### Issue: High memory usage with concurrent operations
|
|
695
|
+
|
|
696
|
+
**Solution**: Use `LZMAPool` to limit concurrency:
|
|
697
|
+
|
|
698
|
+
```typescript
|
|
699
|
+
import { LZMAPool } from 'node-liblzma';
|
|
700
|
+
|
|
701
|
+
const pool = new LZMAPool(5); // Limit to 5 concurrent operations
|
|
702
|
+
|
|
703
|
+
// Pool automatically queues excess operations
|
|
704
|
+
const results = await Promise.all(
|
|
705
|
+
largeArray.map(item => pool.compress(item))
|
|
706
|
+
);
|
|
707
|
+
```
|
|
708
|
+
|
|
709
|
+
## Windows-Specific Issues
|
|
710
|
+
|
|
711
|
+
### Issue: Build fails on Windows
|
|
712
|
+
|
|
713
|
+
**Solutions**:
|
|
714
|
+
1. Install Visual Studio Build Tools:
|
|
715
|
+
```powershell
|
|
716
|
+
npm install --global windows-build-tools
|
|
717
|
+
```
|
|
718
|
+
|
|
719
|
+
2. Use the correct Python version:
|
|
720
|
+
```powershell
|
|
721
|
+
npm config set python python3
|
|
722
|
+
```
|
|
723
|
+
|
|
724
|
+
3. Let the build system download XZ automatically:
|
|
725
|
+
```powershell
|
|
726
|
+
npm install node-liblzma --build-from-source
|
|
727
|
+
```
|
|
728
|
+
|
|
729
|
+
### Issue: "Cannot find module" on Windows
|
|
730
|
+
|
|
731
|
+
**Cause**: Path separator issues in Windows
|
|
732
|
+
|
|
733
|
+
**Solution**: Use forward slashes or `path.join()`:
|
|
734
|
+
```typescript
|
|
735
|
+
import { join } from 'path';
|
|
736
|
+
await xzFile(join('data', 'input.txt'), join('data', 'output.xz'));
|
|
737
|
+
```
|
|
738
|
+
|
|
739
|
+
# Contributing
|
|
740
|
+
|
|
741
|
+
We welcome contributions! Here's how to get started.
|
|
742
|
+
|
|
743
|
+
## Development Setup
|
|
744
|
+
|
|
745
|
+
1. **Clone the repository**:
|
|
746
|
+
```bash
|
|
747
|
+
git clone https://github.com/oorabona/node-liblzma.git
|
|
748
|
+
cd node-liblzma
|
|
749
|
+
```
|
|
750
|
+
|
|
751
|
+
2. **Install dependencies** (pnpm recommended):
|
|
752
|
+
```bash
|
|
753
|
+
pnpm install
|
|
754
|
+
# or
|
|
755
|
+
npm install
|
|
756
|
+
```
|
|
757
|
+
|
|
758
|
+
3. **Build the project**:
|
|
759
|
+
```bash
|
|
760
|
+
pnpm build
|
|
761
|
+
```
|
|
762
|
+
|
|
763
|
+
4. **Run tests**:
|
|
764
|
+
```bash
|
|
765
|
+
pnpm test
|
|
766
|
+
```
|
|
767
|
+
|
|
768
|
+
## Development Workflow
|
|
769
|
+
|
|
770
|
+
### Running Tests
|
|
771
|
+
|
|
772
|
+
```bash
|
|
773
|
+
# Run all tests
|
|
774
|
+
pnpm test
|
|
775
|
+
|
|
776
|
+
# Watch mode (re-run on changes)
|
|
777
|
+
pnpm test:watch
|
|
778
|
+
|
|
779
|
+
# Coverage report
|
|
780
|
+
pnpm test:coverage
|
|
781
|
+
|
|
782
|
+
# Interactive UI
|
|
783
|
+
pnpm test:ui
|
|
784
|
+
```
|
|
785
|
+
|
|
786
|
+
### Code Quality
|
|
787
|
+
|
|
788
|
+
We use [Biome](https://biomejs.dev/) for linting and formatting:
|
|
789
|
+
|
|
790
|
+
```bash
|
|
791
|
+
# Check code style
|
|
792
|
+
pnpm check
|
|
793
|
+
|
|
794
|
+
# Auto-fix issues
|
|
795
|
+
pnpm check:write
|
|
796
|
+
|
|
797
|
+
# Lint only
|
|
798
|
+
pnpm lint
|
|
799
|
+
|
|
800
|
+
# Format only
|
|
801
|
+
pnpm format:write
|
|
802
|
+
```
|
|
803
|
+
|
|
804
|
+
### Type Checking
|
|
805
|
+
|
|
806
|
+
```bash
|
|
807
|
+
pnpm type-check
|
|
808
|
+
```
|
|
809
|
+
|
|
810
|
+
## Code Style
|
|
811
|
+
|
|
812
|
+
- **Linter**: Biome (configured in `biome.json`)
|
|
813
|
+
- **Formatting**: Biome handles both linting and formatting
|
|
814
|
+
- **Pre-commit hooks**: Automatically run via nano-staged + simple-git-hooks
|
|
815
|
+
- **TypeScript**: Strict mode enabled
|
|
816
|
+
|
|
817
|
+
## Commit Convention
|
|
818
|
+
|
|
819
|
+
We follow [Conventional Commits](https://www.conventionalcommits.org/):
|
|
820
|
+
|
|
821
|
+
```
|
|
822
|
+
<type>(<scope>): <description>
|
|
823
|
+
|
|
824
|
+
[optional body]
|
|
825
|
+
|
|
826
|
+
[optional footer]
|
|
827
|
+
```
|
|
828
|
+
|
|
829
|
+
**Types**:
|
|
830
|
+
- `feat`: New feature
|
|
831
|
+
- `fix`: Bug fix
|
|
832
|
+
- `docs`: Documentation changes
|
|
833
|
+
- `refactor`: Code refactoring
|
|
834
|
+
- `test`: Test changes
|
|
835
|
+
- `chore`: Build/tooling changes
|
|
836
|
+
- `perf`: Performance improvements
|
|
837
|
+
|
|
838
|
+
**Examples**:
|
|
839
|
+
```bash
|
|
840
|
+
git commit -m "feat(pool): add LZMAPool for concurrency control"
|
|
841
|
+
git commit -m "fix(bindings): resolve memory leak in FunctionReference"
|
|
842
|
+
git commit -m "docs(readme): add migration guide for v2.0"
|
|
843
|
+
```
|
|
844
|
+
|
|
845
|
+
## Pull Request Process
|
|
846
|
+
|
|
847
|
+
1. **Fork the repository** and create a feature branch:
|
|
848
|
+
```bash
|
|
849
|
+
git checkout -b feat/my-new-feature
|
|
850
|
+
```
|
|
851
|
+
|
|
852
|
+
2. **Make your changes** following code style guidelines
|
|
853
|
+
|
|
854
|
+
3. **Add tests** for new functionality:
|
|
855
|
+
- All new code must have 100% test coverage
|
|
856
|
+
- Tests go in `test/` directory
|
|
857
|
+
- Use Vitest testing framework
|
|
858
|
+
|
|
859
|
+
4. **Ensure all checks pass**:
|
|
860
|
+
```bash
|
|
861
|
+
pnpm check:write # Fix code style
|
|
862
|
+
pnpm type-check # Verify TypeScript types
|
|
863
|
+
pnpm test # Run test suite
|
|
864
|
+
```
|
|
865
|
+
|
|
866
|
+
5. **Commit with conventional commits**:
|
|
867
|
+
```bash
|
|
868
|
+
git add .
|
|
869
|
+
git commit -m "feat: add new feature"
|
|
870
|
+
```
|
|
871
|
+
|
|
872
|
+
6. **Push and create Pull Request**:
|
|
873
|
+
```bash
|
|
874
|
+
git push origin feat/my-new-feature
|
|
875
|
+
```
|
|
876
|
+
|
|
877
|
+
7. **Wait for CI checks** to pass (GitHub Actions will run automatically)
|
|
878
|
+
|
|
879
|
+
## Testing Guidelines
|
|
880
|
+
|
|
881
|
+
- **Coverage**: Maintain 100% code coverage (statements, branches, functions, lines)
|
|
882
|
+
- **Test files**: Name tests `*.test.ts` in `test/` directory
|
|
883
|
+
- **Structure**: Use `describe` and `it` blocks with clear descriptions
|
|
884
|
+
- **Assertions**: Use Vitest's `expect()` API
|
|
885
|
+
|
|
886
|
+
**Example test**:
|
|
887
|
+
```typescript
|
|
888
|
+
import { describe, it, expect } from 'vitest';
|
|
889
|
+
import { xzAsync, unxzAsync } from '../src/lzma.js';
|
|
890
|
+
|
|
891
|
+
describe('Compression', () => {
|
|
892
|
+
it('should compress and decompress data', async () => {
|
|
893
|
+
const original = Buffer.from('test data');
|
|
894
|
+
const compressed = await xzAsync(original);
|
|
895
|
+
const decompressed = await unxzAsync(compressed);
|
|
896
|
+
|
|
897
|
+
expect(decompressed.equals(original)).toBe(true);
|
|
898
|
+
});
|
|
899
|
+
});
|
|
900
|
+
```
|
|
901
|
+
|
|
902
|
+
## Release Process
|
|
903
|
+
|
|
904
|
+
Releases are automated using [@oorabona/release-it-preset](https://github.com/oorabona/release-it-preset):
|
|
905
|
+
|
|
906
|
+
```bash
|
|
907
|
+
# Standard release (patch/minor/major based on commits)
|
|
908
|
+
pnpm release
|
|
909
|
+
|
|
910
|
+
# Manual changelog editing
|
|
911
|
+
pnpm release:manual
|
|
912
|
+
|
|
913
|
+
# Hotfix release
|
|
914
|
+
pnpm release:hotfix
|
|
915
|
+
|
|
916
|
+
# Update changelog only (no release)
|
|
917
|
+
pnpm changelog:update
|
|
918
|
+
```
|
|
919
|
+
|
|
920
|
+
**For maintainers only**. Contributors should submit PRs; maintainers handle releases.
|
|
921
|
+
|
|
922
|
+
## Getting Help
|
|
923
|
+
|
|
924
|
+
- **Questions**: Open a [Discussion](https://github.com/oorabona/node-liblzma/discussions)
|
|
925
|
+
- **Bugs**: Open an [Issue](https://github.com/oorabona/node-liblzma/issues)
|
|
926
|
+
- **Security**: Email security@example.com (do not open public issues)
|
|
927
|
+
|
|
928
|
+
## License
|
|
929
|
+
|
|
930
|
+
By contributing, you agree that your contributions will be licensed under [LGPL-3.0+](LICENSE).
|
|
211
931
|
|
|
212
932
|
# Bugs
|
|
213
933
|
|