@nemesis-oss/ollama-sdk 1.16.0 → 1.17.0

package/CHANGELOG.md

@@ -1,5 +1,32 @@
 # Changelog
 
+## [1.17.0] — Streaming blob uploads (`duplex:'half'`, two-pass file upload), typed-array view-safety pins, tracing-header verdicts
+
+The October 14 digest ("Raw Model Blob Ingestion Streaming Backpressure, W3C Distributed Tracing Baggage Injection, Defensive Serialization for Typed Arrays") targeted heavy binary asset streaming, operational provenance, and low-level buffer safety. Audited against the codebase first (see [ADR 0032](./docs/adr/0032-streaming-blob-uploads-tracing-verdicts-view-safety.md)): the heap-exhaustion claim was real on the blob path — `createBlobFromFile` bulk-read the whole file (`fs.readFile`) before uploading, and the `ReadableStream` member of `BinaryBody` was a promise the transport couldn't keep (Node 18+ `fetch` rejects stream bodies without `duplex: 'half'`) — though the `/api/create` framing was misdirected (its request body is a small JSON digest map, not a stream). The tracing claim was stale (client-level `headers`, per-request headers, and the middleware seam already forward any W3C headers verbatim — nothing is stripped). The typed-array claim was stale as a defect (`src/` has zero `Buffer.from` call sites; the SDK's Web APIs are view-safe by construction, and adopting the proposed `Buffer`-coupled helpers would regress the edge-runtime zero-Node-builtins gate) — but the invariant is now pinned by tests. The companion upstream-specs digest (System One batching, payload caps, `expires_at`, model warmth) was comprehensively stale — every recommendation already ships. All additive, no breaking changes. 1012 pre-existing tests stay green, 12 new tests added.
+
+### Streaming blob uploads (MOD-01 — multi-GB GGUFs never enter the heap)
+
+- **Transport**: both fetch call sites (middleware `finalHandler` and the direct path, previously duplicated init construction) now share one `buildFetchInit` that sets `duplex: 'half'` exactly when the body is a web `ReadableStream`. The guard is structural (`getReader` duck-typing) so cross-realm streams (workers, VM contexts) still get the member; the init type is a local `RequestInit & { duplex?: 'half' }` intersection because `@types/node` 22's `RequestInit` doesn't carry the member yet — assignable to the plain `RequestInit` `FetchLike` expects. Harmless (and spec'd) in browsers.
+- **`createBlobFromStream(digest, stream, request?)`** — the multi-gigabyte-safe counterpart of `createBlobFromData`: pipes any web `ReadableStream<Uint8Array>` straight into the request body with HEAD-dedupe semantics identical to the data-based variant. The digest must be known up front — it _is_ the upload URL, and the server verifies streamed content against it, so a truncated or corrupted stream fails the upload instead of poisoning the store.
+- **`createBlobFromFile` rewritten as two streaming passes** (same signature, same result shape): a SHA-256 hash pass through `node:crypto`'s incremental digest in 256 KiB chunks (skipped entirely when `opts.digest` is provided), then an upload pass piping `createReadStream` → `Readable.toWeb` into the streaming upload. Peak heap is O(chunk) in both passes; `createModelFromGguf` (which routes every shard through this method) inherits the bound. The method was already Node-only (hidden-specifier dynamic `node:fs`); the new `node:crypto`/`node:stream` imports follow the same pattern, keeping the main entry edge-portable.
+- **Cancellation plumbing**: `createBlob`/`checkBlob` now accept `RequestCancellationOptions` (`signal`/`timeoutMs`) like their siblings, and `createBlobFromData`'s previously decorative `signal` option is actually wired through. Aborts surface as `OllamaAbortError` from both file passes (the hash pass checks between chunks and destroys the fd; the upload pass forwards the signal to `fetch` and destroys the stream on abort).
+- New coverage pins the whole story: duplex injection (unit, injected `fetch`), byte-exact streamed uploads against a real capture server, the two-pass file flow (streaming digest matches the whole-file digest; wire bytes equal the file byte-for-byte), abort-before-traffic and abort-mid-stalled-upload, and daemonless public-mock round trips including `createModelFromGguf`'s headline flow (blob POST carries the shard, `/api/create` references it under its basename).
+
+### Typed-array view safety (BUF-01 — stale defect, pinned invariant)
+
+- The audit found zero `Buffer.from(view.buffer)`-style call sites: `computeBlobDigest` uses Web Crypto, which hashes a `BufferSource` view's bytes per spec, and `fetch` serializes `Uint8Array` bodies per-view. The digest's `safeUint8ArrayToBuffer`/`toUint8Array` helpers were declined — they couple a runtime-portable SDK to Node's `Buffer` and would regress the edge-runtime gate (ADR 0032).
+- The invariant is now pinned by the digest's own suggested probe, hardened with an external oracle: a `subarray(10, 20)` view of a 1024-byte patterned pool hashes to the standalone slice's digest (cross-checked against `node:crypto`) and arrives on the wire as exactly its 10 bytes — never the parent buffer.
+- Cleanup: `computeBlobDigest`'s misleading `data as unknown as ArrayBuffer` cast removed (the local structural `SubtleCrypto` type already accepts `ArrayBufferView`).
+
+### Tracing headers (TRC-01 — stale, documented)
+
+- The SDK forwards headers verbatim through three existing seams: client-level `headers` configuration (every request, every endpoint), per-request `headers` on transport request options (merged last, so they win), and middleware's mutable `RequestContext.headers` record — an OpenTelemetry-style extractor plugs in without touching a single method signature. The README's middleware section now shows the W3C `traceparent`/`tracestate`/`baggage` propagation pattern explicitly.
+- The digest's `traceContext` option on every client method and ambient `AsyncLocalStorage` capture were declined (ADR 0032): signature inflation and hidden global state for capability that already exists.
+
+### Upstream-specs digest verdicts (all stale — pointers, no new API)
+
+- **System One batching**: `client.systemOne()` already evaluates N named questions against shared state in one round trip, with compile-time question→answer type correlation the sketch lacks. **Payload caps**: `MAX_SYSTEM_ONE_REQUEST_BYTES` (64 KiB) / `MAX_SYSTEM_ONE_IMAGES_BYTES` (32 MiB) are exported, `estimateSystemOneRequestBytes()` is the pre-flight estimator, and the runtime enforces both client-side. **`expires_at`**: typed on `ModelResponse` from `/api/ps`, returned by `client.ps()`. **Model warmth**: `pin()`/`unload()`/`KEEP_ALIVE_INDEFINITE`/per-call `keep_alive`/`warmModelAffinity()` are the supported primitives; a background `ModelWarmthKeeper` timer daemon was declined (hidden event-loop work that escapes the client-teardown contract — reasoning in ADR 0032).
+
 ## [1.16.0] — Stream observation (`observeStream`), network-error classification & context-window recommendations
 
 The October 13 second digest ("Stream Multi-Sink Teeing, Context Window Overflow, Socket Hangup Recovery") targeted edge transport hazards and memory boundaries during high-throughput agent execution. Audited against the codebase first (see [ADR 0031](./docs/adr/0031-stream-observation-network-classification-context-recommendations.md)): the stream-teeing claim was the real gap (no observer utility existed, and `OllamaStream`'s dual modes are mutually exclusive — you couldn't tap a stream you were iterating); the context-overflow claim was comprehensively stale (a superior CJK-aware estimator with three-tier window resolution, `onContextOverflow` policies, and `getContextLength()` discovery has shipped since the v1.9.0 era — only its concrete-window suggestion was missing); and the socket-retry claim was inverted on the headline (every SDK-routed call already retries with backoff _and_ fails over — three retries where the digest asked for one) but real in one seam: network-error classification was `TypeError`-only, so code-carrying socket errors from custom `fetch` implementations and nested undici `cause` chains died un-retried on first attempt. All additive, no breaking changes. 977 pre-existing tests stay green, 35 new tests added.

package/README.md

@@ -33,7 +33,7 @@ Nexum’s model harness (2026-10-09) exercised this SDK via `gemma4:cloud` and `
 - 🌉 **OpenAI & Anthropic Compatibility Bridges**: Built-in clients for `/v1/chat/completions`, `/v1/responses`, `/v1/models`, and `/v1/messages`, including `reasoning_effort`/`reasoning.effort` for thinking models — plus an ergonomic dual-mode `client.responses.create()` bridge that prefers native `/v1/responses` and transparently re-issues via `/api/chat` on older servers (pre-v0.13.3).
 - 💬 **KV-Cache-Aware Conversation Sessions**: `client.session(model, systemPrompt)` keeps an append-only, prefix-stable history that maximizes Ollama's KV-cache reuse across turns, surfaces per-turn + cumulative cache hit rates (`prompt_eval_cached_count`) so cache degradation is visible instead of silent, and offers a one-call `compact()` sliding-window escape hatch when a session outgrows its window.
 - 🛑 **Context-Window Safety**: Heuristic client-side token estimation warns (or throws, `onContextOverflow: 'throw'`) before a request is sent when the prompt approaches the effective `num_ctx` window — Ollama's default behavior is to _silently truncate_ oversized prompts — and `defaultContextLength` makes the window explicit on every request.
-- 📦 **Blob & Custom Model Publishing**: Content-addressed blob management per Ollama's documented protocol — `computeBlobDigest()` (SHA-256 → `sha256:<hex>`), `createBlobFromData()`/`createBlobFromFile()` (HEAD-check + upload, skipping existing blobs), and one-shot `createModelFromGguf()` (upload GGUF shard blobs, then `/api/create` with `files: {name: digest}`) — with **speculative-decoding assembly**: `draftGguf` uploads the draft network's shards into `draft_files`, `draft_quantize` compresses them independently, and `parameters.draft_num_predict` tunes the per-step draft length.
+- 📦 **Blob & Custom Model Publishing**: Content-addressed blob management per Ollama's documented protocol — `computeBlobDigest()` (SHA-256 → `sha256:<hex>`), `createBlobFromData()`/`createBlobFromFile()` (HEAD-check + upload, skipping existing blobs), **streaming uploads with O(chunk) heap** (`createBlobFromFile` hashes and uploads in 256 KiB chunks — multi-GB GGUFs never enter the Node heap; `createBlobFromStream()` pipes any web `ReadableStream` straight into `fetch` with `duplex: 'half'`), and one-shot `createModelFromGguf()` (upload GGUF shard blobs, then `/api/create` with `files: {name: digest}`) — with **speculative-decoding assembly**: `draftGguf` uploads the draft network's shards into `draft_files`, `draft_quantize` compresses them independently, and `parameters.draft_num_predict` tunes the per-step draft length.
 - 🏁 **Speculative Draft Benchmark (`compareDraftPerformance`)**: One call answers "does drafting pay off for _this_ prompt?" — identical prompt on both sides, deterministic sampling (`temperature: 0` + fixed seed), warm-up discard, several samples per side with **medians** compared, throughput via `formatTelemetry` — reported as baseline vs. speculative tok/s, `speedupMultiplier`, and `effectiveGain` past a configurable threshold. A ≈1.0 speedup is itself diagnostic: the model has no draft network, so the server silently ignores the option.
 - 🧪 **In-Memory Mock Server (`@nemesis-oss/ollama-sdk/testing`)**: The SDK's own deterministic test double, promoted public — `createMockOllamaServer()` with scriptable NDJSON routes, built-in liveness/version defaults (so `ping()` works out of the box), request recording for wire-level assertions, **request-aware routes** (`responder` — per-request responses for fail-twice-then-succeed retry drills and body-dependent payloads), mid-stream **in-band error injection** (`{"error": "..."}` frames at HTTP 200, exactly how a real daemon reports GPU OOM), connection drops, and correctly-shaped chunk builders for streaming text and **parallel tool calls** — zero network, zero daemon, CI-deterministic.
 - 🧵 **Batched Embeddings with Backpressure**: `client.embedBatch()` splits corpora of any size into bounded `/api/embed` batches (default 32 inputs × 3 in flight) — order-preserving, fail-fast, progress-reporting — so RAG/vector ingestion can't OOM a local daemon or saturate `OLLAMA_MAX_QUEUE` the way `Promise.all` floods do, with per-string context-window pre-flight against silent truncation.
@@ -500,6 +500,34 @@ await client.models.createModelFromGguf('my-model', [
 ]);
 ```
 
+#### Streaming uploads — multi-gigabyte weights with O(chunk) heap
+
+Both `createBlobFromFile` and `createModelFromGguf` (which routes every shard
+through it) stream instead of bulk-reading: the SHA-256 hash pass and the
+upload pass each consume the file in 256 KiB chunks, so peak heap is a chunk,
+never the file. Pass `digest` when you already know it to skip the hash pass
+entirely. When you own the bytes (a chunk producer, a WASM/SharedArrayBuffer
+pipeline, a resumed download), `createBlobFromStream` pipes any web
+`ReadableStream<Uint8Array>` straight into the request body — the transport
+sets `duplex: 'half'` (required by Node 18+ `fetch` for stream bodies)
+automatically, and the server verifies the streamed content against the
+digest, so a truncated stream fails the upload instead of corrupting the
+store:
+
+```typescript
+// digest known up front (hash first when it isn't — the digest IS the URL)
+const { alreadyExisted } = await client.models.createBlobFromStream(
+  'sha256:…',
+  myChunkProducer, // ReadableStream<Uint8Array>
+);
+
+// Both methods accept { signal, timeoutMs } for cancellation/timeout:
+await client.models.createBlobFromFile('/models/model.gguf', {
+  digest: 'sha256:…', // optional: skip the hash pass
+  signal: controller.signal,
+});
+```
+
 ### Speculative Decoding (Draft Models)
 
 Ollama packages speculative decoding as a small **draft network** whose token
@@ -1934,6 +1962,25 @@ around the underlying HTTP request for native endpoints, OpenAI/Anthropic compat
 model health checks, and the hosted web tools. Lifecycle events expose `start`, `success`,
 `retry`, and `error` events with a request id, timing, and status/error information.
 
+**Distributed tracing (W3C `traceparent`/`tracestate`/`baggage`)**: the SDK forwards
+whatever headers it is given and never strips or rewrites yours — three seams cover
+propagation. Client-wide (every request, every endpoint):
+
+```ts
+const client = new OllamaClient({
+  headers: {
+    traceparent: carrierHeader('traceparent'), // from your inbound request
+    baggage: carrierHeader('baggage'),
+  },
+});
+```
+
+Per-request or computed (the middleware seam from above — `request.headers` is a mutable
+record, so an OpenTelemetry-style extractor plugs in directly and can vary headers per
+call). A `traceContext` option on every SDK method was considered and declined
+(ADR 0032): it would inflate every signature for something these seams already do
+without touching them.
+
 ```ts
 const client = new OllamaClient({
   middleware: [

package/dist/index.cjs

@@ -1113,6 +1113,18 @@ function createRequestId() {
   requestSequence += 1;
   return `ollama-http-${requestSequence}`;
 }
+function isWebReadableStreamBody(body) {
+  return typeof body === "object" && body !== null && typeof body.getReader === "function";
+}
+function buildFetchInit(req) {
+  return {
+    method: req.method,
+    headers: req.headers,
+    ...req.body !== void 0 ? { body: req.body } : {},
+    ...req.signal !== void 0 ? { signal: req.signal } : {},
+    ...isWebReadableStreamBody(req.body) ? { duplex: "half" } : {}
+  };
+}
 function sameHeaders(actual, expected) {
   const actualEntries = Object.fromEntries(
     Object.entries(actual).map(([key, value]) => [key.toLowerCase(), value])
@@ -1162,13 +1174,7 @@ var HttpClient = class {
       timestamp: startedAt
     });
     const finalHandler = async (req) => {
-      const init = {
-        method: req.method,
-        headers: req.headers,
-        ...req.body !== void 0 ? { body: req.body } : {},
-        ...req.signal !== void 0 ? { signal: req.signal } : {}
-      };
-      const response = await this.fetchImpl(req.url, init);
+      const response = await this.fetchImpl(req.url, buildFetchInit(req));
       rawResponse = response;
       const responseHeaders = response.headers && typeof response.headers.entries === "function" ? Object.fromEntries(response.headers.entries()) : {};
       return {
@@ -1179,12 +1185,7 @@ var HttpClient = class {
     };
     try {
       if (this.middleware.length === 0) {
-        const response2 = await this.fetchImpl(request.url, {
-          method: request.method,
-          headers: request.headers,
-          ...request.body !== void 0 ? { body: request.body } : {},
-          ...request.signal !== void 0 ? { signal: request.signal } : {}
-        });
+        const response2 = await this.fetchImpl(request.url, buildFetchInit(request));
         if (response2.status >= 200 && response2.status < 300) {
           this.onLifecycleEvent?.({
             type: "success",
@@ -2775,7 +2776,7 @@ var ModelsClient = class {
       { model, singleEndpoint: true }
     );
   }
-  async createBlob(digest, data) {
+  async createBlob(digest, data, request) {
     await this.runner(
       (http, signal) => http.request({
         path: `/api/blobs/${encodeURIComponent(digest)}`,
@@ -2783,10 +2784,10 @@ var ModelsClient = class {
         rawBody: data,
         signal
       }),
-      { singleEndpoint: true }
+      { ...request, singleEndpoint: true }
     );
   }
-  async checkBlob(digest) {
+  async checkBlob(digest, request) {
     try {
       await this.runner(
         (http, signal) => http.request({
@@ -2794,7 +2795,7 @@ var ModelsClient = class {
           method: "HEAD",
           signal
         }),
-        { singleEndpoint: true }
+        { ...request, singleEndpoint: true }
       );
       return true;
     } catch (err) {
@@ -2836,9 +2837,32 @@ var ModelsClient = class {
    */
   async createBlobFromData(data, opts) {
     const digest = opts?.digest ?? await this.computeBlobDigest(data);
-    const alreadyExisted = await this.checkBlob(digest);
+    const alreadyExisted = await this.checkBlob(digest, opts);
     if (!alreadyExisted) {
-      await this.createBlob(digest, data);
+      await this.createBlob(digest, data, opts);
+    }
+    return { digest, alreadyExisted };
+  }
+  /**
+   * Streams a blob upload straight from a web `ReadableStream` — the
+   * multi-gigabyte-safe counterpart of {@link createBlobFromData}. The body
+   * is never materialized in memory: the transport hands the stream to
+   * `fetch` directly (setting `duplex: 'half'`, which Node 18+ requires for
+   * stream bodies), so peak heap usage is a transport-level chunk, not the
+   * blob size.
+   *
+   * The digest must be known up front — it *is* the upload URL
+   * (`/api/blobs/<digest>`), the same content-addressed protocol
+   * `curl -T` performs. Hash first when you don't have it: in-memory bytes
+   * via {@link computeBlobDigest}, files via {@link createBlobFromFile}
+   * (which hashes in a streaming pass). The server verifies the streamed
+   * bytes against the digest and rejects mismatches, so a corrupted or
+   * truncated stream fails the upload instead of poisoning the store.
+   */
+  async createBlobFromStream(digest, stream, request) {
+    const alreadyExisted = await this.checkBlob(digest, request);
+    if (!alreadyExisted) {
+      await this.createBlob(digest, stream, request);
     }
     return { digest, alreadyExisted };
   }
@@ -2847,16 +2871,61 @@ var ModelsClient = class {
    * `node:fs` import, so browser bundles never pull it in). The file name
    * (basename) is what `/api/create`'s `files` map expects as the key, so
    * this returns it alongside the digest.
+   *
+   * Streams both passes in fixed-size chunks — peak heap is O(chunk), not
+   * O(file), so multi-gigabyte GGUFs never enter the Node heap:
+   *
+   *  1. **Hash pass** (skipped when `opts.digest` is provided): the file is
+   *     streamed through `node:crypto`'s incremental SHA-256.
+   *  2. **Upload pass**: `node:fs` createReadStream → web ReadableStream →
+   *     `fetch` body with `duplex: 'half'` (see {@link createBlobFromStream}).
+   *
+   * A pre-existing blob (HEAD 200) skips the upload pass entirely, same as
+   * {@link createBlobFromData}.
    */
   async createBlobFromFile(path, opts) {
     const nodeFsModule = "node:fs";
-    const { promises: fs } = await import(nodeFsModule);
-    const data = await fs.readFile(path);
-    const result = await this.createBlobFromData(new Uint8Array(data), {
-      ...opts
-    });
+    const { createReadStream } = await import(nodeFsModule);
+    const nodeCryptoModule = "node:crypto";
+    const { createHash } = await import(nodeCryptoModule);
+    const nodeStreamModule = "node:stream";
+    const { Readable } = await import(nodeStreamModule);
+    const CHUNK_BYTES = 256 * 1024;
+    let digest = opts?.digest;
+    if (digest === void 0) {
+      const hash = createHash("sha256");
+      const hashStream = createReadStream(path, { highWaterMark: CHUNK_BYTES });
+      try {
+        for await (const chunk of hashStream) {
+          if (opts?.signal?.aborted) {
+            throw new OllamaAbortError("createBlobFromFile: aborted while hashing", {});
+          }
+          hash.update(chunk);
+        }
+      } finally {
+        hashStream.destroy();
+      }
+      digest = `sha256:${hash.digest("hex")}`;
+    }
+    const alreadyExisted = await this.checkBlob(digest, opts);
+    if (!alreadyExisted) {
+      const uploadStream = createReadStream(path, { highWaterMark: CHUNK_BYTES });
+      const onAbort = () => {
+        uploadStream.destroy();
+      };
+      opts?.signal?.addEventListener("abort", onAbort, { once: true });
+      try {
+        await this.createBlob(
+          digest,
+          Readable.toWeb(uploadStream),
+          opts
+        );
+      } finally {
+        opts?.signal?.removeEventListener("abort", onAbort);
+      }
+    }
     const fileName = path.split(/[\\/]/).pop() ?? path;
-    return { ...result, fileName };
+    return { digest, alreadyExisted, fileName };
   }
   /**
    * One-shot custom-model publishing from raw GGUF weights, following the
@@ -6196,8 +6265,8 @@ var OllamaClient = class _OllamaClient {
    */
   getContextLength = (request) => this.models.getContextLength(request);
   version = () => this.models.version();
-  createBlob = (digest, data) => this.models.createBlob(digest, data);
-  checkBlob = (digest) => this.models.checkBlob(digest);
+  createBlob = (digest, data, request) => this.models.createBlob(digest, data, request);
+  checkBlob = (digest, request) => this.models.checkBlob(digest, request);
   /**
    * Immediately unloads `model` from VRAM. Convenience alias for
    * {@link ModelsClient.unload} — see that method for the full