spd-lib 1.3.5 → 1.3.6

package/README.md

@@ -6,16 +6,41 @@ Encrypt any JavaScript value (strings, numbers, objects, typed arrays, Maps, Set
 
 ---
 
-## Security model
+## Security model (v29)
 
-| Layer | Algorithm | Post-quantum strength |
+| Layer | Algorithm | Notes |
 |---|---|---|
 | Key derivation | Argon2id (128 MiB, 6 iterations) | Raises brute-force cost against quantum search |
-| Encryption | XChaCha20-Poly1305 (256-bit key) | 128-bit PQ security via Grover's algorithm |
-| Authentication | HMAC-SHA3-512 (256-bit key, domain-separated) | 128-bit PQ security |
-| Salt wrapping | Argon2id-derived key + XChaCha20-Poly1305 | Same as above |
+| Key expansion | HKDF-SHA3-512 (domain-separated) | Separate AEAD and MAC keys from one master secret |
+| Encryption | XChaCha20-Poly1305 (256-bit key, per-entry) | 128-bit PQ security via Grover's algorithm |
+| Key commitment | CMT-4: SHA3-256(key ∥ nonce) prefix | Prevents partitioning oracle / multi-key attacks |
+| Authentication | HMAC-SHA3-512 (256-bit key, full-file) | 128-bit PQ security, domain-separated |
+| Name encryption | XChaCha20-Poly1305 (dedicated name key) | Entry names are never stored in plaintext |
+| Compression privacy | Pad-to-256B blocks before deflate | Mitigates CRIME/BREACH length-based oracle |
+| Salt wrapping | Argon2id + XChaCha20-Poly1305 | Encrypts the KDF salt under the passcode |
+
+**Key derivation chain:**
+1. Argon2id(passcode, random 16-byte salt) → 96-byte master secret
+2. HKDF-SHA3-512(master, `'spd-aead-key-v1'`) → 32-byte AEAD key
+3. HKDF-SHA3-512(master, `'spd-mac-key-v1'`) → 64-byte MAC key
+4. Per-entry key: HKDF-SHA3-512(AEAD key, entry name) → 32-byte entry key
 
-Keys are domain-separated: the 512-bit Argon2id master secret is split — the first 32 bytes go to encryption, the last 32 bytes go to authentication. They are never used interchangeably.
+---
+
+## Performance (Node.js 22, Apple M3, standard profile)
+
+| Operation | Throughput / Latency |
+|---|---|
+| Key derivation (Argon2id, 128 MiB) | ~380 ms (one-time per session) |
+| addData ~36 B | ~10k ops/s |
+| addData 10 KB | ~3k ops/s |
+| addData 100 KB | ~750 ops/s |
+| addData 1 MB | ~100 ops/s |
+| saveToFileStreaming / loadFromFileStreaming 10×10 KB | ~400 ms (dominated by KDF) |
+| getEntry (random-access lookup, binary file) | ~400 ms (dominated by KDF) |
+| addMany 1000 items (parallel workers) | ~1–2 s |
+
+The dominant cost in all file operations is the single Argon2id call (~380 ms). CMT-4 commitment, compression padding, HKDF, and name encryption add < 5 µs per entry.
 
 ---
 
@@ -41,13 +66,14 @@ await spd.addData('username', 'alice');
 await spd.addData('apiKey', 'sk-abc123');
 await spd.addData('config', { theme: 'dark', retries: 3 });
 
-// Save to file
-await spd.saveToFile('./vault.spd', 'MyStr0ng!Passphrase#2024');
+// Save to file (binary streaming format — recommended)
+await spd.saveToFileStreaming('./vault.spd', 'MyStr0ng!Passphrase#2024');
 
 // Load from file
-const loaded = await SPD.loadFromFile('./vault.spd', 'MyStr0ng!Passphrase#2024');
+const loaded = await SPD.loadFromFileStreaming('./vault.spd', 'MyStr0ng!Passphrase#2024');
 const data = await loaded.extractData();
 console.log(data.username); // 'alice'
+loaded.destroy();
 ```
 
 ---
@@ -62,9 +88,9 @@ Creates a new empty SPD instance. You must call `setPassKey()` before adding or
 
 ### `spd.setPassKey(passcode: string): Promise<void>`
 
-Derives encryption and authentication keys from the passcode using Argon2id. Must be called before any data operation.
+Derives encryption and authentication keys from the passcode using Argon2id + HKDF. Must be called before any data operation.
 
-**Passcode requirements:** minimum 12 characters, must contain at least 3 of: lowercase, uppercase, digits, special characters.
+**Passcode requirements:** minimum 12 characters, must contain at least 3 of: lowercase, uppercase, digits, special characters. Alternatively, a passphrase of 5+ space-separated lowercase words (e.g. from `generatePassphrase()`) is accepted.
 
 ```ts
 await spd.setPassKey('MyStr0ng!Passphrase#2024');
@@ -72,9 +98,36 @@ await spd.setPassKey('MyStr0ng!Passphrase#2024');
 
 ---
 
+### `spd.setKeyProfile(profile: 'standard' | 'paranoid'): void`
+
+Sets the Argon2id memory and time parameters before calling `setPassKey`.
+
+| Profile | Memory | Iterations | Use case |
+|---|---|---|---|
+| `'standard'` | 128 MiB | 6 | Default — interactive applications |
+| `'paranoid'` | 2048 MiB | 16 | Long-term archive, server-side, offline |
+
+```ts
+spd.setKeyProfile('standard'); // default
+spd.setKeyProfile('paranoid'); // maximum resistance
+```
+
+---
+
+### `SPD.generatePassphrase(wordCount?: number): string`
+
+Generates a cryptographically random space-separated word passphrase using the EFF large wordlist. Default is 7 words.
+
+```ts
+const pass = SPD.generatePassphrase();       // 7 words, ~56 bits entropy
+const pass = SPD.generatePassphrase(10);     // 10 words, ~80 bits entropy
+```
+
+---
+
 ### `spd.addData(name: string, value: unknown): Promise<void>`
 
-Encrypts and stores a single value. The name is sanitized (lowercased, non-alphanumeric chars replaced with `_`).
+Encrypts and stores a single value. The name is sanitized (lowercased, non-alphanumeric chars replaced with `_`). Entry names are encrypted and never stored in plaintext (v29+).
 
 **Supported types:** `string`, `number`, `boolean`, `null`, `object`, `Array`, `Uint8Array`, `Uint16Array`, `Uint32Array`, `BigInt64Array`, `BigUint64Array`, `Float32Array`, `Float64Array`, `Map`, `Set`, `Date`, `RegExp`, `Error`
 
@@ -90,7 +143,7 @@ await spd.addData('lookup', new Map([['key', 'val']]));
 
 ### `spd.addMany(items: { name: string; data: unknown }[]): Promise<void>`
 
-Batch version of `addData`. All items are encrypted in parallel.
+Batch version of `addData`. Large batches (≥ 256 items) are encrypted in parallel using worker threads.
 
 ```ts
 await spd.addMany([
@@ -113,34 +166,38 @@ console.log(data.firstName); // 'Alice'
 
 ---
 
-### `spd.saveToFile(path: string, passcode: string): Promise<void>`
+### `spd.saveToFileStreaming(path: string, passcode: string): Promise<void>`
+### `SPD.loadFromFileStreaming(path: string, passcode: string): Promise<SPD>`
 
-Saves the encrypted payload to a file (mode `0600`). Suitable for files up to ~1–2 GB.
+**Recommended for all file I/O.** Writes/reads the binary SPD wire format. Heap usage stays bounded regardless of file size. Required format for `getEntry` random-access lookups.
+
+The file embeds an offset index tail (`SPDx`) for O(1) entry lookups without a sidecar file.
 
 ```ts
-await spd.saveToFile('./secrets.spd', 'MyStr0ng!Passphrase#2024');
+await spd.saveToFileStreaming('./large.spd', 'MyStr0ng!Passphrase#2024');
+const spd = await SPD.loadFromFileStreaming('./large.spd', 'MyStr0ng!Passphrase#2024');
 ```
 
 ---
 
+### `spd.saveToFile(path: string, passcode: string): Promise<void>`
 ### `SPD.loadFromFile(path: string, passcode: string): Promise<SPD>`
 
-Loads and verifies an SPD file. The hash algorithm and Argon2 parameters are read from the file itself — no need to pass them.
+JSON-envelope file format. Suitable for files up to ~1–2 GB. Not compatible with `getEntry`.
 
 ```ts
+await spd.saveToFile('./secrets.spd', 'MyStr0ng!Passphrase#2024');
 const spd = await SPD.loadFromFile('./secrets.spd', 'MyStr0ng!Passphrase#2024');
 ```
 
 ---
 
-### `spd.saveToFileStreaming(path: string, passcode: string): Promise<void>`
-### `SPD.loadFromFileStreaming(path: string, passcode: string): Promise<SPD>`
+### `SPD.getEntry(filePath: string, passcode: string, entryName: string): Promise<unknown>`
 
-Streaming variants for files larger than 2 GB. Uses a binary wire format with 64 MB write chunks — heap usage stays bounded regardless of file size.
+Random-access lookup of a single entry from a binary SPD file (written by `saveToFileStreaming`). Uses the embedded `SPDx` index tail to jump directly to the entry — no full scan. MAC-verified before decryption.
 
 ```ts
-await spd.saveToFileStreaming('./large.spd', 'MyStr0ng!Passphrase#2024');
-const spd = await SPD.loadFromFileStreaming('./large.spd', 'MyStr0ng!Passphrase#2024');
+const value = await SPD.getEntry('./vault.spd', 'MyStr0ng!Passphrase#2024', 'username');
 ```
 
 ---
@@ -173,12 +230,6 @@ await spd.extractDataStreaming('/tmp/spd_scratch', async (name, value) => {
 
 True constant-memory extraction from a binary `.spd` file (written by `saveToFileStreaming`). Processes one entry at a time using `.ssf` skeleton files — the full plaintext never lives in RAM.
 
-Steps per entry:
-1. Writes encrypted blob to `<name>_enc.ssf`
-2. Decrypts into `<name>_dec.ssf`
-3. Reads the `.ssf` file and calls your callback
-4. Deletes both `.ssf` files before the next entry
-
 Peak RAM usage is proportional to the largest single entry, not the file size.
 
 ```ts
@@ -232,7 +283,7 @@ const spd = await SPD.loadFromChunks(chunks, 'MyStr0ng!Passphrase#2024');
 
 The last element of the array is always a JSON manifest (`{ totalChunks, chunkSize, totalBytes, version }`). The receiver validates chunk count and byte count before decrypting.
 
-**Auto chunk size** (when `chunkSize` is omitted): targets ~32 chunks, clamped between 64 KB and 8 MB, rounded to the nearest 64 KB boundary. Pass an explicit `chunkSize` to override (e.g. `256 * 1024` for strict API body limits).
+**Auto chunk size** (when `chunkSize` is omitted): targets ~32 chunks, clamped between 64 KB and 8 MB, rounded to the nearest 64 KB boundary.
 
 ---
 
@@ -258,14 +309,41 @@ spd.setHash('sha3-512'); // default, recommended
 
 ### `spd.setCompressionLevel(level: number): void`
 
-Sets the zlib deflate compression level (1–9). Default is `9` (maximum compression).
+Sets the zlib deflate compression level (1–9). Default is `6` (balanced speed/size).
 
 ```ts
-spd.setCompressionLevel(6); // faster, slightly larger
+spd.setCompressionLevel(9); // maximum compression
+spd.setCompressionLevel(1); // fastest, largest output
 ```
 
 ---
 
+### `spd.setSigningKey(privateKeyPem: string): void` / `spd.setVerifyKey(publicKeyPem: string): void`
+
+Attach an Ed25519 signing key to the session. When set, `saveToFile` / `saveData` / `saveToFileStreaming` append an Ed25519 signature over the entire payload, and `loadFromFile` / `loadFromString` / `loadFromFileStreaming` verify it before decryption.
+
+```ts
+const { privateKey, publicKey } = crypto.generateKeyPairSync('ed25519', {
+  publicKeyEncoding:  { type: 'spki',  format: 'pem' },
+  privateKeyEncoding: { type: 'pkcs8', format: 'pem' },
+});
+
+spd.setSigningKey(privateKey);
+await spd.saveToFileStreaming('./signed.spd', PASS);
+
+const loaded = await SPD.loadFromFileStreaming('./signed.spd', PASS);
+loaded.setVerifyKey(publicKey);
+const data = await loaded.extractData(); // throws if signature invalid
+```
+
+---
+
+### `spd.setEpochSize(bytes: number): void`
+
+Controls the epoch size (in bytes) for the key-ratcheting mechanism used in chunked exports. Defaults to 4 MB. Smaller values ratchet keys more frequently (higher security, lower throughput).
+
+---
+
 ### `spd.destroy()` / `spd.clearCache()`
 
 Securely zeros all key material in place using `sodium.memzero()`, then clears all stored data. Call this when you are done with a session.
@@ -276,9 +354,35 @@ spd.destroy();
 
 ---
 
-### `SPD.deriveKeys(passcode, salt)` / `SPD.derivePBK(passcode, salt)`
+## SPDWriter — streaming disk-backed writer
+
+`SPDWriter` encrypts and writes entries one at a time to disk as they are produced — the full plaintext never exists in RAM simultaneously. Suitable for constructing large SPD files from a stream of records.
+
+```ts
+import { SPDWriter } from 'spd-lib-ts';
+
+const writer = new SPDWriter('./output.spd', 'MyStr0ng!Passphrase#2024');
+await writer.init();
 
-Low-level static helpers exposed for advanced use cases (e.g. deriving keys outside of an SPD container).
+await writer.addEntry('username', 'alice');
+await writer.addEntry('config', { theme: 'dark' });
+// ... add as many entries as needed
+
+await writer.finalize();
+```
+
+After `finalize()`, the file contains an embedded `SPDx` index tail compatible with `SPD.getEntry()` — no sidecar file needed.
+
+**Options** (passed as third argument to constructor):
+
+| Option | Default | Description |
+|---|---|---|
+| `compressionLevel` | `9` | zlib deflate level |
+| `hashAlgorithm` | `'sha3-512'` | Per-entry hash |
+| `argon2Memory` | `131072` (128 MiB) | Argon2id memory cost |
+| `argon2Time` | `6` | Argon2id time cost |
+
+Call `writer.destroy()` to zero keys and delete the partial file if `finalize()` was never called.
 
 ---
 
@@ -343,12 +447,12 @@ async function saveConfig(config: Record<string, unknown>) {
   const spd = new SPD();
   await spd.setPassKey(PASS);
   await spd.addMany(Object.entries(config).map(([name, data]) => ({ name, data })));
-  await spd.saveToFile(FILE, PASS);
+  await spd.saveToFileStreaming(FILE, PASS);
   spd.destroy();
 }
 
 async function loadConfig(): Promise<Record<string, unknown>> {
-  const spd = await SPD.loadFromFile(FILE, PASS);
+  const spd = await SPD.loadFromFileStreaming(FILE, PASS);
   const data = await spd.extractData();
   spd.destroy();
   return data;
@@ -360,7 +464,7 @@ async function loadConfig(): Promise<Record<string, unknown>> {
 ## Full example — chunked HTTP upload
 
 ```ts
-// server sends:
+// sender:
 const spd = new SPD();
 await spd.setPassKey(PASS);
 await spd.addData('payload', largeObject);
@@ -374,7 +478,7 @@ for (let i = 0; i < chunks.length; i++) {
   });
 }
 
-// receiver collects all chunks in order and loads:
+// receiver:
 const spd = await SPD.loadFromChunks(receivedChunks, PASS);
 const data = await spd.extractData();
 spd.destroy();
@@ -386,11 +490,27 @@ spd.destroy();
 
 | npm version | SPD format version | Notes |
 |---|---|---|
+| 1.4.0 | v29 | CMT-4 key commitment, HKDF key expansion, encrypted entry names, 256-byte compression padding |
 | 1.3.1 | v26 | Hash algo + Argon2 params embedded in payload, secure key zeroing, `null` type support |
 | 1.3.0 | v25 | 512-bit Argon2id master secret, domain-separated AEAD + HMAC keys, HMAC-SHA3-512 |
 | < 1.3.0 | v24 | 256-bit Argon2id, single key for AEAD + MAC |
 
-SPD format versions are not cross-compatible. Use `SPDLegacy` to read v24 and earlier files.
+SPD format versions are not cross-compatible. v29 files cannot be read by older library versions. Use `SPDLegacy` to read v24 and earlier files.
+
+---
+
+## Wire format (v29 binary)
+
+Each entry in the binary file uses the following layout (all integers little-endian):
+
+```
+[2B encNameLen][encName][1B typeLen][type][2B hashLen][hash][3B nonceLen][nonce][8B dataLen][data]
+```
+
+- `encName` = `[24B XChaCha20 nonce || CMT-4 ciphertext of entry name]`
+- `data` = `[32B CMT block || XChaCha20-Poly1305 ciphertext of padded+compressed plaintext]`
+
+The full file is zlib-deflated and HMAC-SHA3-512 authenticated. A 72-byte header stores the plaintext length and MAC. An `SPDx` index tail is appended after the compressed stream for O(1) lookups.
 
 ---