libfw-client 0.4.3 → 0.4.4

package/README.md

@@ -3,6 +3,11 @@
 The browser SDK for [libfw](../README.md): a zero-config wrapper around the
 WASM engine, the File System Access API and IndexedDB.
 
+> The same protocol engine is also available as a **native Rust client** for
+> non-browser programs — `libfw_client::native::NativeClient` (async
+> `tokio` + `reqwest`), with a runnable CLI in
+> [`examples/rust-client`](../examples/rust-client/README.md).
+
 ## Usage
 
 ```js
@@ -16,8 +21,12 @@ const client = new LibfwClient({
   downloadWindow: 4,    // parallel byte-range GETs per single file download
                         // (raise to reduce download stutter on high-latency links)
   compress: true,       // zrip per-block compression
+  compressLevel: 'auto',// zrip level policy: 'auto' benchmarks uploads
+                        // (needs autoTune) / 'fast' / 'balanced' / 'max' / N
   autoTune: true,       // adaptive tuning: probes /capabilities and ramps
-                        // concurrency/windows/chunk sizes from real stats
+  tuneTtlMs: 3600000,   // how long a settled result stays cached (browser,
+                        // localStorage); 0 = never cache, re-ramp each time
+                        // concurrency/windows/chunk size from real stats
   onEvent: (e) => {
     if (e.type === 'progress') updateProgressBar(e.done, e.total);
     else if (e.type === 'tuning') renderTuning(e.phase, e.params, e.stats);
@@ -71,8 +80,14 @@ client.cancel();
   into place. Only the chunks the server still misses are re-sent
   (`x-libfw-session-status` probe seeds resume), so interrupted uploads
   resume BitTorrent-style (only the broken/lost parts are re-transmitted).
+  A dropped connection (page refresh, crashed tab) keeps that partial on the
+  server, so a reloaded page resumes exactly where it stopped.
 - Resume state (`etag`, `offset`, `size`) is persisted per path in
-  IndexedDB and re-validated on every retry.
+  IndexedDB and re-validated on every retry. `createWritable()` only
+  publishes a file on `close()`, so a download **checkpoints** its prefix to
+  disk every time the engine reports a durable offset (~4 MiB) — a hard page
+  refresh mid-download therefore resumes from the last checkpoint instead of
+  restarting from byte 0.
 - Pause/resume/cancel drive the WASM state machine
   (`idle → downloading/uploading → paused → resumed → completed/failed`).
 
@@ -106,21 +121,50 @@ dist/libfw-client.umd.js   UMD bundle (after build:umd)
     both upload chunks and parallel download ranges; the engine reorders
     in-flight chunks in memory (worst case ≈ `downloadWindow * chunkSize`
     bytes) so the SDK still receives data in order.
+  - `compress: boolean` (default `true`) — master switch for zrip
+    compression; `false` sends every body as identity.
+  - `compressLevel: number | 'auto' | 'fast' | 'balanced' | 'max'` (default
+    `'balanced'`) — zrip level policy when `compress` is on. `'fast'` is the
+    advertised minimum (least CPU, worst ratio), `'balanced'` the advertised
+    default, `'max'` the advertised maximum (best ratio); a number is clamped
+    into the advertised range. `'auto'` additionally micro-benchmarks the
+    advertised range against a real sample of the first uploaded file (while
+    `autoTune` is enabled) and picks the best bytes-saved-vs-CPU-time
+    trade-off for the measured link speed; downloads request the resolved
+    level from the server.
   - `downloadMode: 'auto' | 'fs' | 'browser'` (default `'auto'`) — `'fs'`
     streams downloads through the File System Access API; `'browser'` buffers
     and triggers a traditional browser download (folders become `.zip`);
     `'auto'` uses `'fs'` when the API exists and falls back to `'browser'`.
+    **Download resume requires `'fs'`** (or an injected `directoryHandle`):
+    an interrupted fs-mode download is continued from the bytes already on
+    disk, while the memory-backed `'browser'` fallback always restarts from
+    byte 0 — there is no partial file to continue from.
   - `maxFallbackBytes: number` (default `536870912`, 512 MiB) — memory cap
     for the in-memory `'browser'` fallback. File sizes are pre-checked
     against it before buffering; a download that would exceed it rejects
     with a `too-large` `LibfwError` instead of risking an OOM. `0` disables.
   - `autoTune: boolean` (default `false`) — enable the adaptive tuning
     engine. The engine probes the server's `/capabilities` limits and
-    TCP-style ramps concurrency / windows / chunk sizes (and the zrip
-    level) from the advertised minimums using real transfer stats. When
-    disabled, the configured static values are used as-is.
-  - `tuneTtlMs: number` (default `3600000`, 1 h) — how long a settled
-    tuning result is reused for the same server origin before re-ramping.
+    TCP-style ramps the per-file window and cross-file concurrency from the
+    advertised minimums using real transfer stats; the chunk size follows the
+    measured throughput (~100 ms of it, clamped into the advertised range) and
+    the zrip level is a client policy from `compressLevel`, never ramped. When
+    disabled, the configured static values are used as-is. Tuning state is
+    **in memory** for the lifetime of the client (a settle is reused by later
+    transfers and dropped on failure) **and** is cached in `localStorage` per
+    origin + direction, so a page refresh does not re-ramp — see `tuneTtlMs`.
+  - `tuneTtlMs: number` (default `3600000`, 1 hour) — how long a cached
+    tuning result stays usable. The cache is keyed by origin **and**
+    direction (an upload settle says nothing about a download), is tagged with
+    the `/capabilities` it was measured against, and expires `tuneTtlMs`
+    *after the ramp settled* — not after the last reuse — so a link measured
+    long ago is re-measured even if it is used constantly. An entry is also
+    discarded early when the capabilities change or a transfer fails. `0`
+    disables the cache entirely (every transfer re-ramps). Ignored unless
+    `autoTune` is enabled. Storage failures (private mode, quota, disabled
+    storage) are swallowed: caching is an optimisation and never fails a
+    transfer.
 - `downloadFolder(token, dirPath?) → Promise<number>`
 - `downloadFile(token, filePath) → Promise<number>`
 - `upload(token, files?) → Promise<number>`
@@ -130,7 +174,9 @@ dist/libfw-client.umd.js   UMD bundle (after build:umd)
 - `tuneStatus() → { phase, params, stats, capsHash } | null` — live
   adaptive-tuning status. `phase` is `uninitialized | ramping | settled |
   degraded`; `params` is `{ concurrency, uploadWindow, downloadWindow,
-  chunkSize, compressLevel }`; `stats` is
+  chunkSize, compressLevel }` (the zrip level is the resolved *policy*, i.e.
+  what downloads request — uploads may use an `'auto'`-benchmarked level for
+  the session); `stats` is
   `{ rttMs, mbps }` (EWMA request RTT, last-window throughput). `null`
   until the WASM engine is initialised.
 - Events: with `autoTune` enabled, `onEvent` additionally receives

package/index.d.ts

@@ -127,7 +127,19 @@ export interface LibfwClientOptions {
   downloadWindow?: number;
   /** Negotiate zrip compression. Default `true`. */
   compress?: boolean;
-  /** Upload chunk size in bytes. Default 2 MiB. */
+  /**
+   * zrip level policy when `compress` is on. `'fast'` = advertised minimum
+   * (least CPU), `'balanced'` (default) = advertised default, `'max'` =
+   * advertised maximum (best ratio); a number is that level clamped into the
+   * advertised range.
+   *
+   * `'auto'` additionally micro-benchmarks the advertised range against a
+   * real sample of the first uploaded file while `autoTune` is enabled,
+   * choosing the level with the best bytes-saved-vs-CPU-time trade-off.
+   * Downloads request the resolved level from the server.
+   */
+  compressLevel?: number | 'auto' | 'fast' | 'balanced' | 'max';
+  /** Shared chunk size in bytes for uploads and parallel downloads. Default 2 MiB. */
   chunkSize?: number;
   /** Retries per chunk/file before failing. Default `3`. */
   maxRetries?: number;
@@ -160,15 +172,28 @@ export interface LibfwClientOptions {
   maxFallbackBytes?: number;
   /**
    * Enable the adaptive tuning engine: the engine probes the server's
-   * `/capabilities` limits and TCP-style ramps concurrency / windows /
-   * chunk sizes (and the zrip level) from the advertised minimums using
-   * real transfer stats. When disabled the configured static values are
-   * used as-is. Default `false`.
+   * `/capabilities` limits and TCP-style ramps the per-file window and
+   * cross-file concurrency from the advertised minimums using real transfer
+   * stats. The chunk size follows the measured throughput (~100 ms of it,
+   * clamped into the advertised range) and the zrip level is a client policy
+   * from `compressLevel`, never ramped. When disabled the configured static
+   * values are used as-is. Default `false`.
+   *
+   * Tuning state lives in memory for this client instance (a settle is reused
+   * by later transfers and dropped on failure) **and** is cached in
+   * `localStorage` per origin + direction, so a page reload does not re-ramp.
+   * See `tuneTtlMs` for the lifetime of that cache. Default `false`.
    */
   autoTune?: boolean;
   /**
-   * How long (ms) a settled tuning result is reused for the same server
-   * origin before re-ramping. Default `3600000` (1 hour).
+   * How long a cached tuning result stays usable, in milliseconds. Default
+   * `3600000` (1 hour); `0` disables the cache so every transfer re-ramps.
+   *
+   * The cache is browser-only (the Rust native client keeps its settle in
+   * memory) and is keyed by origin + direction. The TTL counts from the moment
+   * the ramp settled, and an entry is discarded early when the server's
+   * `/capabilities` change or a transfer fails. Ignored unless `autoTune` is
+   * enabled.
    */
   tuneTtlMs?: number;
   /** Optional progress/state listener. Tuning updates arrive as `{ type: 'tuning', phase, params, stats }`. */

package/index.js

@@ -207,7 +207,20 @@ export class LibfwClient {
    *        transfer), so a single file's throughput is bounded by bandwidth
    *        instead of one connection's `chunkSize / RTT` on high-latency
    *        links. `1` disables parallelism (sequential downloads).
-   * @param {boolean} [options.compress=true] negotiate zrip compression
+   * @param {boolean} [options.compress=true] negotiate zrip compression.
+   *        Master switch: `false` sends every body as identity, regardless of
+   *        `compressLevel`.
+   * @param {number|'auto'|'fast'|'balanced'|'max'} [options.compressLevel='balanced']
+   *        zrip level policy when `compress` is on. `'fast'` = advertised
+   *        minimum (least CPU, worst ratio), `'balanced'`/`'auto'`-without-a-
+   *        sample = advertised default, `'max'` = advertised maximum (best
+   *        ratio), a number = that level clamped into the advertised range.
+   *        `'auto'` additionally micro-benchmarks the advertised range against
+   *        a real sample of the first uploaded file (only while `autoTune` is
+   *        enabled) and picks the level with the best
+   *        bytes-saved-vs-CPU-time trade-off for the measured link speed.
+   *        Downloads request the resolved level from the server (no sample is
+   *        available before the transfer starts).
    * @param {number} [options.chunkSize=2097152] shared chunk size in bytes for
    *        both upload chunks and parallel download ranges. The same value is
    *        used on both paths, and any value works as long as the server and
@@ -232,11 +245,23 @@ export class LibfwClient {
    *        `0` disables the limit.
    * @param {boolean} [options.autoTune=false] enable the adaptive tuning
    *        engine: the engine probes the server's `/capabilities` limits and
-   *        TCP-style ramps concurrency / windows / chunk sizes (and the zrip
-   *        level) from the advertised minimums using real transfer stats.
-   *        When disabled the configured static values are used as-is.
-   * @param {number} [options.tuneTtlMs=3600000] how long a settled tuning
-   *        result is reused for the same server origin before re-ramping
+   *        TCP-style ramps the per-file window and cross-file concurrency from
+   *        the advertised minimums using real transfer stats; the chunk size
+   *        follows the measured throughput (~100 ms of it, clamped into the
+   *        advertised range) and the zrip level is a client policy from
+   *        `compressLevel`, never ramped. When disabled the configured static
+   *        values are used as-is. A settled result is kept in memory for this
+   *        client instance (reused by later transfers, dropped on failure)
+   *        **and** cached in `localStorage` per origin + direction, so a page
+   *        reload does not re-ramp; see `tuneTtlMs` for how long that cache
+   *        lives.
+   * @param {number} [options.tuneTtlMs=3600000] how long a cached tuning
+   *        result (browser only — the Rust native client keeps it in memory)
+   *        stays usable, in milliseconds. The TTL counts from the moment the
+   *        ramp settled, is scoped to one origin + direction, and is
+   *        invalidated early if the server's `/capabilities` change or a
+   *        transfer fails. `0` disables the cache entirely, so every transfer
+   *        (and every page load) re-ramps. Ignored unless `autoTune` is on.
    * @param {(event: {type: string, done: number, total: number, path?: string, error?: string}) => void} [options.onEvent]
    *        optional progress/state listener. Tuning updates arrive as
    *        `{ type: 'tuning', phase, params, stats }` events.
@@ -268,6 +293,7 @@ export class LibfwClient {
       uploadWindow: 8,
       downloadWindow: 4,
       compress: true,
+      compressLevel: null,
       chunkSize: 2 * 1024 * 1024,
       maxRetries: 3,
       baseRetryDelayMs: 500,
@@ -277,7 +303,7 @@ export class LibfwClient {
       downloadMode: 'auto',
       maxFallbackBytes: 512 * 1024 * 1024,
       autoTune: false,
-      tuneTtlMs: 3600000,
+      tuneTtlMs: 60 * 60 * 1000,
       onEvent: null,
       directoryHandle: null,
       resolveDisplayName: null,
@@ -333,6 +359,7 @@ export class LibfwClient {
       uploadWindow: this._options.uploadWindow,
       downloadWindow: this._options.downloadWindow,
       compress: this._options.compress,
+      compressLevel: this._options.compressLevel,
       chunkSize: this._options.chunkSize,
       maxRetries: this._options.maxRetries,
       baseRetryDelayMs: this._options.baseRetryDelayMs,
@@ -417,7 +444,15 @@ export class LibfwClient {
         // poisons a later FS-API resume. Skip persisting download state
         // while a fallback transfer is active.
         if (direction === 'download' && this._fallback) return Promise.resolve();
-        return Idb.saveState(`${direction}:${path}`, state);
+        return Idb.saveState(`${direction}:${path}`, state).then(() => {
+          // The engine reports a durable offset every few MiB: use it to
+          // COMMIT the prefix to disk, so an interrupted download (a tab
+          // refresh or a crash — no `finally` block runs) still has a partial
+          // to resume from instead of starting over.
+          if (direction === 'download') {
+            return this._checkpointDownload(path, Number(state?.offset) || 0);
+          }
+        });
       },
       getFileList: () => this._getFileList(),
       readFile: (path, offset, length) => this._readFile(path, offset, length),
@@ -818,6 +853,35 @@ export class LibfwClient {
       const writable = await handle.createWritable(
         isResume ? { keepExistingData: true } : undefined
       );
+      if (isResume) {
+        // `createWritable({ keepExistingData: true })` KEEPS the existing
+        // bytes but still positions the stream at 0 — it is not an append. A
+        // resumed tail written without seeking would overwrite the prefix and
+        // then close, leaving a file that is only the tail (measured: a
+        // 20 MiB download resumed at 6.8 MiB produced a 14.1 MiB, corrupt
+        // file). Seek past what is already on disk so the tail really
+        // appends; `_loadResumeState` clamped `offset` to the on-disk length.
+        const existing = (await handle.getFile()).size;
+        if (existing < offset) {
+          // The prefix we are asked to resume past is not on disk: appending
+          // would leave a hole. Fail loudly instead of committing a corrupt
+          // file (the caller can retry, which restarts cleanly).
+          await this._discardWritable(path, writable);
+          throw new LibfwError(
+            `cannot resume \`${path}\`: only ${existing} of ${offset} bytes are on disk`,
+            'storage'
+          );
+        }
+        try {
+          await writable.seek(offset);
+        } catch (err) {
+          await this._discardWritable(path, writable);
+          throw new LibfwError(
+            `cannot resume \`${path}\`: the destination cannot be seeked (${err?.message ?? err})`,
+            'storage'
+          );
+        }
+      }
       entry = { writable, dir, name, lastOffset: offset };
       this._writables.set(path, entry);
     }
@@ -825,6 +889,74 @@ export class LibfwClient {
     await entry.writable.write(data);
   }
 
+  /**
+   * Commit the bytes received so far for a download to disk.
+   *
+   * `createWritable()` only publishes the file on `close()` — everything else
+   * lives in a Chromium swap file (`.name.crswap`) that is DISCARDED when the
+   * page dies. A hard refresh mid-download would therefore find a 0-byte (or
+   * stale) target and restart from byte 0. The engine reports a durable
+   * offset every few MiB (`RESUME_SAVE_EVERY`), so this closes the writable
+   * (publishing the prefix) and immediately reopens it with
+   * `keepExistingData` + `seek`, leaving a resumable partial on disk at the
+   * cost of one close/reopen per checkpoint.
+   *
+   * Called from the `saveState` callback, which the engine awaits between
+   * chunks — no write for this path is in flight here.
+   * @param {string} path
+   * @param {number} offset durable offset reported by the engine
+   * @returns {Promise<void>}
+   * @private
+   */
+  async _checkpointDownload(path, offset) {
+    const entry = this._writables.get(path);
+    const handle = this._fileHandles.get(path);
+    if (!entry || !handle || offset <= 0) return;
+    let onDisk = 0;
+    try {
+      onDisk = (await handle.getFile()).size;
+    } catch {
+      return; // handle gone: nothing safe to commit
+    }
+    // Nothing new to publish (an earlier checkpoint already covers it), or
+    // the engine is behind the bytes we committed: leave the stream alone.
+    if (onDisk >= offset) return;
+    try {
+      await entry.writable.close();
+    } catch {
+      return; // keep the (still open) stream; the next chunk continues on it
+    }
+    try {
+      const writable = await handle.createWritable({ keepExistingData: true });
+      // The committed file now holds everything written so far; `close()`
+      // published exactly `lastOffset` bytes, so appending resumes there.
+      await writable.seek((await handle.getFile()).size);
+      entry.writable = writable;
+    } catch {
+      // Reopening failed: forget the stream so the next chunk re-opens it
+      // through the normal resume path (which re-validates the prefix).
+      this._writables.delete(path);
+    }
+  }
+
+  /**
+   * Drop an open writable (and its uncommitted swap file) without failing the
+   * caller. Used when a resume cannot be honoured so no partial/corrupt file
+   * is committed.
+   * @param {string} path
+   * @param {FileSystemWritableFileStream} writable
+   * @returns {Promise<void>}
+   * @private
+   */
+  async _discardWritable(path, writable) {
+    this._writables.delete(path);
+    try {
+      await writable.abort();
+    } catch {
+      /* best-effort discard */
+    }
+  }
+
   /**
    * Close the destination writable once a file's transfer completes.
    * @param {string} path virtual path

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "libfw-client",
-  "version": "0.4.3",
+  "version": "0.4.4",
   "description": "High-performance streaming file & folder transfer SDK for the browser (libfw WASM engine + File System Access API + IndexedDB resume).",
   "license": "MIT",
   "type": "module",

package/pkg/libfw_client.js

@@ -592,7 +592,7 @@ function __wbg_get_imports() {
             return ret;
         }, arguments); },
         __wbindgen_cast_0000000000000001: function(arg0, arg1) {
-            // Cast intrinsic for `Closure(Closure { owned: true, function: Function { arguments: [Externref], shim_idx: 209, ret: Result(Unit), inner_ret: Some(Result(Unit)) }, mutable: true }) -> Externref`.
+            // Cast intrinsic for `Closure(Closure { owned: true, function: Function { arguments: [Externref], shim_idx: 206, ret: Result(Unit), inner_ret: Some(Result(Unit)) }, mutable: true }) -> Externref`.
             const ret = makeMutClosure(arg0, arg1, wasm_bindgen_b2c787b844a3ac79___convert__closures_____invoke___wasm_bindgen_b2c787b844a3ac79___JsValue__core_ed718c3d60ebd546___result__Result_____wasm_bindgen_b2c787b844a3ac79___JsError___true_);
             return ret;
         },

package/pkg/package.json

@@ -1,8 +1,8 @@
 {
   "name": "libfw-client",
   "type": "module",
-  "description": "WASM engine + JS SDK for libfw browser clients",
-  "version": "0.4.3",
+  "description": "libfw client: native Rust client + WASM engine + JS SDK",
+  "version": "0.4.4",
   "license": "MIT",
   "repository": {
     "type": "git",