7zip-native 0.0.1 → 0.0.6

package/README.md

@@ -3,102 +3,298 @@
 
 # 7zip-native
 
-Native Node-API bindings for the 7-Zip archive engine. Parsing and extraction through the `Archive` API run in native worker tasks and never invoke `7z`, `7zz`, or another archive executable. The optional comparison benchmark invokes `7zip-bin-full` only as a command-line baseline.
+### Current version `26.05`
+Upstream 7-Zip archive engine from https://github.com/ip7z/7zip/
 
-> **Platform status:** the release workflow builds and tests Linux x64/arm64, macOS x64/arm64, and Windows x64/arm64/x86. Linux x86 (ia32) and ARM32 artifacts are cross-compiled and checked for their target architecture, but are not runtime-tested in CI. This checkout has only been verified locally on Linux x64 with Node.js 22; do not treat other targets as runtime-tested until the corresponding GitHub Actions runs pass. This remains an early end-to-end implementation, not a claim of full parity with every requirement in the project specification.
+Native Node-API bindings for the 7-Zip archive engine. Use a typed, Promise-based Node.js API to inspect archives, read entry data, stream large files, or extract entries without launching `7z`, `7zz`, or another archive executable.
 
-## Build from source
+The package includes the native addon and the matching 7-Zip engine for each supported target. Compression and decompression are provided by the upstream 7-Zip implementation.
+
+## Installation
 
 ```sh
-git clone --recurse-submodules https://github.com/ollm/7zip-native.git
-cd 7zip-native
-npm ci
-npm run build
-npm test
+npm install 7zip-native
 ```
 
-Source builds require a C++17 toolchain, Python 3, Node.js headers (downloaded by `node-gyp`), and the upstream submodule. Linux and macOS use GNU make; Windows builds the upstream bundle with `nmake` and Visual C++ build tools. `npm run build:native` compiles the `Format7zF` handler bundle and Node-API addon, then stages both under `prebuilds/<platform>-<arch>/`. Archive operations never shell out.
+The package requires Node.js 18 or newer.
+
+## Quick start
 
-The `Build and publish` GitHub Actions workflow builds the nine listed targets for version tags (`v*`) or a manual dispatch. It runs tests on the seven native-runner targets; Linux x86 (ia32) and ARM32 use cross-compilers and receive architecture checks because the runners are x64. For manual runs, publishing is off by default; enable it only when dispatching the matching `v<package-version>` tag. npm publishing uses trusted publishing (OIDC), so configure this repository as a trusted publisher in npm before publishing. The release matrix uses GitHub-hosted arm64 runners and depends on their availability for the repository.
+```ts
+import {Archive} from '7zip-native';
 
-To update upstream deliberately:
+const archive = await Archive.open('./comics.cbz');
 
-```sh
-git submodule update --remote vendor/7zip
-git -C vendor/7zip rev-parse HEAD
-git add .gitmodules vendor/7zip
+try
+{
+	const entries = await archive.entries();
+	const cover = await archive.getEntry('chapter/cover.jpg');
+
+	if(cover && !cover.isDirectory)
+	{
+		const preview = await archive.read(cover.path, {offset: 0, length: 64 * 1024});
+		const image = await archive.extract(cover.path);
+		console.log(entries.length, preview.length, image.length);
+	}
+}
+finally
+{
+	await archive.close();
+}
 ```
 
-Review and commit the new gitlink revision. The current submodule is pinned to 7-Zip 26.05 (`b88c6bc09339de32f7d18bbab900686b244f42ba`).
+The package supports both ES modules and CommonJS:
+
+```js
+// ES module
+import {Archive} from '7zip-native';
+
+// CommonJS
+const {Archive} = require('7zip-native');
+```
 
 ## API
 
+### `Archive.open(path, options?)`
+
+Opens an archive from a filesystem path and indexes its metadata asynchronously.
+
 ```ts
-import {Archive} from '7zip-native';
+const archive = await Archive.open('./archive.7z', {password: 'secret'});
+```
 
-const archive = await Archive.open('/path/to/file.zip');
+`options.password` is passed to the upstream handler when it requests a password. It is held for the lifetime of the archive handle, is not logged, and is limited to 1 MiB of UTF-8 data.
+
+### Listing and looking up entries
+
+```ts
 const entries = await archive.entries();
-const first = entries[0];
-if (first && !first.isDirectory) {
-	const header = await archive.read(first.id, {offset: 0, length: 64 * 1024});
-	const [cover, page] = await archive.extract([first.id, entries[1]!.id]);
+const byPath = await archive.getEntry('folder/page.jpg');
+const byId = await archive.getEntry(0);
+```
+
+`entries()` returns cached metadata. Entry IDs are zero-based and stable for the lifetime of the opened archive. `getEntry()` returns `undefined` if the ID or full path does not match.
+
+All entry-targeted methods accept either the entry's numeric `id` or its exact archive-internal `path`: `extract()`, `read()`, `createReadStream()`, and `extractToStream()`. Batch `extract()` calls can mix IDs and paths. For folder extraction, use either the `ids` or `paths` option; omit both to extract every entry.
+
+Each entry includes:
+
+| Property | Type | Description |
+| --- | --- | --- |
+| `id` | `number` | Stable ID within this opened archive |
+| `path` | `string` | Entry path, using `/` separators |
+| `size` | `bigint` | Uncompressed size |
+| `compressedSize` | `bigint` | Compressed size |
+| `isDirectory` | `boolean` | Whether the entry is a directory |
+| `encrypted` | `boolean` | Whether the entry is encrypted |
+| `attributes` | `number` | Format-specific attributes |
+| `crc` | `number \| undefined` | CRC when supplied by the format |
+
+### Extracting to memory
+
+```ts
+const image = await archive.extract(entry.id); // Buffer
+const sameImage = await archive.extract(entry.path); // Buffer
+const pages = await archive.extract([page1.id, page2.path]); // Buffer[]
+```
+
+Paths use `/` separators. Batch results are returned in the same order as the requested selectors; duplicate entries are rejected. A missing path rejects with `ERR_INVALID_ENTRY`. A buffered extraction is limited to 256 MiB per entry, with a 512 MiB cap on native results pending transfer to JavaScript. For large entries, use a stream or extract to a folder instead.
+
+### Reading a byte range
+
+```ts
+const header = await archive.read(entry.path, {
+	offset: 0,
+	length: 64 * 1024,
+});
+```
+
+Offsets and lengths accept safe integers or `bigint`. A range read returns only the requested bytes and is limited to 64 MiB. It uses the handler's normal extraction path, however, so a sequential or solid format may still need to decompress preceding data or the whole entry. It is not guaranteed to be constant-time random access.
+
+### Streams and output destinations
+
+Create a standard Node.js `Readable` for a large entry:
+
+```ts
+for await (const chunk of archive.createReadStream(entry.path, {
+	highWaterMark: 256 * 1024,
+	signal: abortController.signal,
+}))
+{
+	// Consume each Buffer chunk before requesting more data.
 }
+```
+
+The native engine emits bounded 64 KiB chunks and waits for stream demand, preserving backpressure. Destroying the stream or aborting its signal cancels extraction at the next output boundary. Streams default to a 4 GiB output limit; set `maxBytes` to another non-negative `number` or `bigint`, or to `null` to disable the limit.
+
+To pipe directly to a Node.js `Writable`:
+
+```ts
+import {createWriteStream} from 'node:fs';
+
+await archive.extractToStream(entry.path, createWriteStream('./page.jpg'), {
+	signal: abortController.signal,
+});
+```
+
+This respects backpressure and ends the destination when extraction succeeds. A destination error, early close, or abort rejects the Promise.
+
+### Extracting to a folder
+
+```ts
+await archive.extractToFolder('./unpacked', {
+	paths: ['chapter/page1.jpg', 'chapter/page2.jpg'], // alternatively, use ids
+	overwrite: true,
+	onProgress: ({filesExtracted, totalFiles, bytesProcessed, totalBytes, percent}) => {
+		console.log(`${filesExtracted}/${totalFiles} files, ${percent}% (${bytesProcessed}/${totalBytes} bytes)`);
+	},
+	maxBytes: 2n * 1024n * 1024n * 1024n,
+	signal: abortController.signal,
+});
+```
+
+Folder extraction accepts either `ids` or exact `paths`; omit both to extract every entry, and do not provide both together. Paths use archive-internal `/` separators. Extraction is sequential and uses bounded memory. Its default total output limit is 4 GiB; `maxBytes: null` disables that limit. Set `onProgress` to receive the number of regular files selected and fully extracted, the bytes emitted so far, the total uncompressed bytes, and the calculated percentage. It is called before extraction, as data chunks are processed, and after each file completes. By default, existing files are not overwritten; set `overwrite: true` to replace existing regular files. Replacement is committed only after that entry has been fully extracted, using a temporary file alongside the destination so a failure does not truncate the existing file. This can temporarily require extra disk space up to the size of the entry being replaced. Symlink or non-file targets are rejected. Unsafe archive paths, path collisions, and symlinked destination components are also rejected. Archive links are written as regular file data; the API does not create symbolic or hard links. If extraction fails partway through, files already written (or partially written) may remain in the destination.
+
+### Concurrency
+
+```ts
+const current = Archive.concurrency();
+Archive.concurrency(8);
+Archive.concurrency(0); // reset to the default
+```
+
+The setting controls the process-wide native worker pool used by extraction, streams, and range reads. The default is the lesser of 4 and the parallelism available to Node.js; the allowed range is 1–128 workers, with `0` restoring the default.
+
+Concurrent operations on one archive use independent decoder contexts rather than calling one 7-Zip archive object simultaneously. This is safe for handlers that support independent contexts, but it does not guarantee a speedup. Solid archives may decompress shared data more than once; batching related entries may be faster.
+
+### Closing an archive
+
+Call `close()` when finished. It rejects future API calls; operations already submitted keep the native state alive until they finish.
+
+```ts
 await archive.close();
 ```
 
-- `Archive.open(path, {password})` supplies the archive password to upstream handlers for encrypted headers and entries. The password is kept only for this archive handle and is never logged. Passwords are limited to 1 MiB of UTF-8 data.
-- `Archive.open(path)` opens and indexes the archive off the event loop.
-- `Archive.concurrency()` returns the process-wide native extraction worker count. `Archive.concurrency(n)` sets it; use `0` to reset to the default. The default is Node's available parallelism, capped at 4 workers to avoid oversubscribing typical archive workloads. The setting applies to extraction, streams, and ranged reads.
-- `entries()` returns cached metadata. IDs are zero-based and stable for that archive handle.
-- `getEntry(idOrPath)` looks up a cached item by ID or full path.
-- `extract(id)` returns one `Buffer`; `extract(ids)` returns buffers in request order. The current in-memory limit is 256 MiB per operation and 512 MiB for native extraction results pending transfer to JavaScript.
-- `createReadStream(id, {highWaterMark, maxBytes, signal})` returns a Node.js `Readable`. Native output is delivered in bounded 64 KiB chunks on demand; destroying the stream or aborting its signal cancels extraction. A stream is limited to 4 GiB by default; set `maxBytes` to another non-negative integer or `bigint`, or to `null` for no output-size limit.
-- `extractToStream(id, writable, options)` pipes an entry to a Node.js `Writable`, respecting backpressure and ending the destination on success. Destination errors and abort signals stop the source stream.
-- `extractToFolder(directory, {ids, maxBytes, signal})` extracts all entries (or the selected IDs) sequentially with bounded memory. Its default total output limit is 4 GiB; `maxBytes: null` disables that limit. Existing files are never overwritten. Unsafe paths, path collisions, and symlinked destination components are rejected. Archive links are written as regular file data; the API does not create symbolic or hard links. A failed extraction can leave already-written or partial files in the destination.
-- `read(id, {offset, length})` returns only the selected bytes. Ranges accept safe integers or `bigint`; requested reads are limited to 64 MiB.
-- `close()` rejects future calls. Work already submitted retains the native state until that work completes.
-- Concurrent `extract()` or `read()` calls on one handle use independent, lazily opened decoder contexts and can run in parallel. Solid archives may need to decompress the same earlier data more than once, so parallelism does not guarantee a speedup; use batched extraction to let 7-Zip process related entries together.
-- `ArchiveError` exposes a stable `code`; input validation also uses standard `TypeError` and `RangeError`.
+## Errors and limits
+
+Invalid argument types and values throw standard `TypeError` or `RangeError`. Archive and extraction failures use `ArchiveError`, which has a stable `code` property:
+
+| Code | Meaning |
+| --- | --- |
+| `ERR_ARCHIVE_CLOSED` | Operation attempted after `close()` |
+| `ERR_INVALID_ENTRY` | Entry ID is invalid or out of range |
+| `ERR_INVALID_RANGE` | Byte range is invalid or exceeds the native offset limit |
+| `ERR_RESOURCE_LIMIT` | Configured or default output limit was exceeded |
+| `ERR_ARCHIVE_OPEN` | Archive could not be opened or indexed |
+| `ERR_ARCHIVE_PASSWORD` | Password is missing or incorrect when requested by the handler |
+| `ERR_ARCHIVE_PATH` | Unsafe or colliding path during folder extraction |
+| `ERR_ARCHIVE_EXTRACT` | Entry extraction failed |
+
+Buffered `extract()` and `read()` do not currently support cancellation. Streams and folder extraction accept an `AbortSignal`.
+
+## Supported formats
+
+The package uses the handlers registered in the pinned upstream `Format7zF` bundle, including 7z, ZIP, RAR, TAR, GZIP, BZIP2, XZ, WIM, ISO/UDF, CAB, ARJ, LZH, CPIO, Zstandard, and additional formats enabled by that upstream revision. Whether a particular archive can be extracted depends on its handler and compression method; unsupported methods, corrupt data, and integrity failures reject the operation. Standalone compression codecs are not necessarily archive formats.
+
+### Format-specific notes
 
-Entry metadata includes `id`, `path`, `size`, `compressedSize`, `isDirectory`, `encrypted`, `attributes`, and `crc` when provided by the format. Sizes are `bigint`.
+- **RAR:** extraction uses upstream code subject to its LGPL terms and the unRAR restriction. This package does not create RAR archives or implement the proprietary RAR compression algorithm. Review the upstream notices before redistribution.
+- **ACE:** not supported by this 7-Zip revision.
+- **Encryption:** password callbacks are connected to archive opening and extraction. AES-256 encrypted ZIP extraction and range reads are tested on Linux x64; encrypted-header behavior and password variants in other formats are not yet verified.
+- **Partial reads:** memory use is limited to the requested range, but the decoder may still process more data internally, especially for solid or sequential formats.
 
-## Formats and limitations
+See [THIRD_PARTY_NOTICES.md](./THIRD_PARTY_NOTICES.md) for third-party redistribution terms.
 
-The addon uses the pinned upstream `Format7zF` bundle and its registered handlers rather than a hand-maintained subset. The bundle includes handlers for 7z, ZIP, RAR, TAR, GZIP, BZIP2, XZ, WIM, ISO/UDF, CAB, ARJ, LZH, CPIO, Zstandard, and additional formats enabled by the upstream makefile. An archive's actual extraction still depends on the handler and decoder method being available; unsupported methods, corrupt data, and integrity failures reject extraction.
+## Platform support
 
-- RAR decompression is upstream code under its LGPL terms and the unRAR restriction. This package does not create RAR archives or implement the proprietary RAR compression algorithm. Read upstream's unRAR notice before redistribution.
-- ACE is not supplied by this 7-Zip revision and is not supported.
-- The upstream 7z handler is covered by a native open-and-extract integration test on Linux x64. Support for individual compression methods and encrypted variants still depends on the capabilities of the upstream handler.
-- Password callbacks are wired for upstream archive opening and extraction. AES-256 encrypted ZIP entry extraction and ranged reads are tested on Linux x64. Wrong or missing passwords requested by a handler are reported with `ArchiveError.code === 'ERR_ARCHIVE_PASSWORD'`. Encrypted-header behavior and password variants in other formats are not yet verified.
-- Partial reads bound memory to the selected range but use the normal `IInArchive::Extract` path. Sequential or solid decoders may decompress preceding data or the whole entry; this is not constant-time random access and does not promise early decoder termination.
-- Opening and metadata queries use Node's libuv workers. Extraction and ranged reads use a persistent native worker pool, defaulting to the lesser of 4 and the processors available to Node. `Archive.concurrency(n)` sets the process-wide pool size; submitted native operations remain bounded.
-- The addon never calls one `IInArchive` instance concurrently. It retains independent archive and input-stream contexts per handle and lazily creates more as concurrent work needs them. Context creation checks the archive's size and modification time; reopen the archive after changing it.
-- Concurrent extraction from a solid archive is safe with independent contexts but may repeat decompression of shared solid blocks. There is no decompressed-block cache, so batching related entries can be faster.
-- Cancellation of buffered `extract()`/`read()` requests and decompressed-block caching are not implemented. Streaming and folder extraction have bounded defaults that can be adjusted per operation; buffered methods retain their fixed in-memory limits.
-- Cross-platform builds and runtime behavior are not claimed until the corresponding matrix jobs succeed. No performance claim is made without measurements on the target platform.
+The package includes prebuilt binaries for:
 
-## Benchmark harness
+| Platform | Architectures |
+| --- | --- |
+| Linux | x64, x86 (ia32), ARM32, ARM64 |
+| macOS | x64, ARM64 |
+| Windows | x64, ARM64 |
 
-Run `npm run benchmark -- ./archive.zip` to measure native open/list, first 64 KiB, full extraction, and 1/2/4/8 reads across independent archive handles.
+Windows x86 (ia32) is not supported by the published prebuilt binaries.
 
-For a direct CLI comparison, install the development dependencies and run:
+## Electron
+
+The addon uses Node-API, but the addon and 7-Zip engine still need to match the Electron application's operating system and architecture. Use a matching package prebuild when available, or rebuild for the Electron version and target architecture using your Electron build tooling.
+
+When packaging with ASAR, ensure both `7zip_native.node` and the matching engine library (`7z.dll`, `7z.so`, or `lib7z.dylib`) are physically unpacked and that the runtime paths resolve to those unpacked files. For `electron-builder`, an `asarUnpack` configuration can include:
+
+```json
+{
+	"build": {
+		"asarUnpack": [
+			"node_modules/7zip-native/prebuilds/**/*.node",
+			"node_modules/7zip-native/prebuilds/**/*.dll",
+			"node_modules/7zip-native/prebuilds/**/*.so",
+			"node_modules/7zip-native/prebuilds/**/*.dylib"
+		]
+	}
+}
+```
+
+Test the packaged application on each target; ASAR unpacking does not replace checking that the addon and engine can both be loaded from the packaged runtime.
+
+## Build from source
 
 ```sh
-npm run benchmark:compare -- --archive "./archive.zip" --entry "chapter/page.jpg" --iterations 20 --read-bytes 65536 --concurrency 4
+git clone --recurse-submodules https://github.com/ollm/7zip-native.git
+cd 7zip-native
+npm ci --ignore-scripts
+npm run build
+npm test
 ```
 
-The comparison uses `7zip-bin-full`'s platform binary to measure open/list, single and batched extraction, prefix reads, and a range read. It validates that both backends list the same number of entries and return identical data before reporting median and p95 latency. It also compares extracting every file sequentially and with the selected concurrency in both backends, using the same bounded batches and validating each batch's output. CLI timings include process startup. The CLI has no ranged-read API, so range measurements extract the whole entry to stdout and slice the requested bytes; this is intentionally reported as a baseline limitation, not as an equivalent random-access implementation. The benchmark verifies the complete entry path/type listing, selected entry, and SHA-256 of every operation's output; result byte counts are included where applicable. By default, the benchmark samples 15 times, reads up to 64 KiB, and uses the native pool's configured default concurrency. Set `--iterations`, `--read-bytes`, and `--concurrency` to change these values. Captured output is capped at 256 MiB; full-archive comparison is skipped if an entry exceeds that limit. Use representative archives; smoke-test timings are not performance claims.
+Source builds require a C++17 toolchain, Python 3, Node.js headers (downloaded by `node-gyp`), and the upstream submodule. Linux and macOS use GNU make; Windows requires Visual C++ build tools and `nmake`. `npm run build:native` compiles the upstream `Format7zF` bundle and Node-API addon, then stages them under `prebuilds/<platform>-<arch>/`.
 
-Compare extracting every file from one archive sequentially versus the configured native concurrency:
+To update the upstream submodule deliberately:
 
 ```sh
-npm run benchmark:concurrency -- --archive "/path/to/archive.7z" --iterations 20 --concurrencies 1,2,3,4,5,8,16
+git submodule update --remote vendor/7zip
+git -C vendor/7zip rev-parse HEAD
+git add .gitmodules vendor/7zip
 ```
 
-The report includes median and p95 latency, throughput, and speedup. It defaults to the native pool's configured concurrency. Use `--concurrencies 1,2,4,8,16` to compare several worker counts in one run, or `--concurrency` to compare the serial baseline with one selected count. Each worker count processes the same single-entry jobs and fixed memory-limited batches, and the report shows the maximum number of entries that can overlap. Use `--max-files 20` to select evenly spaced entries from a larger archive; the report shows both the sample and archive entry counts. This can prevent excessive repeated decompression when benchmarking solid archives. It reports measurements only and does not claim a speedup.
+Review the upstream changes and commit the new submodule revision. The current submodule is pinned to 7-Zip 26.05 (`b88c6bc09339de32f7d18bbab900686b244f42ba`).
 
-## Electron
+## Benchmarks
+
+Run the native benchmark against a representative archive:
+
+```sh
+BENCH_ITERATIONS=20 npm run benchmark -- "./archive.zip"
+```
+
+It measures native open/list, prefix reads, full extraction, and reads across independent archive handles. `BENCH_ITERATIONS` defaults to 15 and must be at least 3.
+
+Compare the native API with the `7zip-bin-full` command-line baseline:
+
+```sh
+npm run benchmark:compare -- \
+	--archive "./archive.zip" \
+	--entry "chapter/page.jpg" \
+	--iterations 20 \
+	--read-bytes 65536 \
+	--concurrency 4
+```
+
+The comparison validates entry listings and output data before reporting median and p95 latency. CLI timings include process startup. Since the CLI has no ranged-read API, its range baseline extracts the entry to stdout and slices the requested bytes; this is not equivalent to native range reading. Captured output is capped at 256 MiB, and full-archive comparison is skipped if an entry exceeds that limit.
+
+Compare extraction concurrency:
+
+```sh
+npm run benchmark:concurrency -- \
+	--archive "./archive.7z" \
+	--iterations 20 \
+	--concurrencies 1,2,3,4,5,8,16 \
+	--max-files 20
+```
+
+The report includes median and p95 latency, throughput, speedup, and the number of entries that can overlap. `--max-files` selects evenly spaced entries, which is useful when sampling solid archives. Benchmarks report measurements only; results depend on the archive, machine, and workload and do not guarantee a speedup.
 
-The addon uses Node-API, but its native engine and addon still need to match the target OS and architecture. Electron applications must rebuild from source for the Electron target (for example with `npm rebuild` and the appropriate `--runtime=electron --target=<version>` arguments), or package the matching prebuild from `prebuilds/<platform>-<arch>/`. Keep the engine (`7z.so` or `7z.dll`) beside `7zip_native.node`.
+## License
 
-See [docs/architecture.md](./docs/architecture.md) for verified source and concurrency constraints, and [THIRD_PARTY_NOTICES.md](./THIRD_PARTY_NOTICES.md) for redistribution requirements.
+This package is MIT-licensed. The integrated 7-Zip code and other bundled dependencies have their own licensing terms; see [THIRD_PARTY_NOTICES.md](./THIRD_PARTY_NOTICES.md) before redistribution.