@nemesis-oss/ollama-sdk 1.15.0 → 1.16.0

package/CHANGELOG.md

@@ -1,5 +1,28 @@
 # Changelog
 
+## [1.16.0] — Stream observation (`observeStream`), network-error classification & context-window recommendations
+
+The October 13 second digest ("Stream Multi-Sink Teeing, Context Window Overflow, Socket Hangup Recovery") targeted edge transport hazards and memory boundaries during high-throughput agent execution. Audited against the codebase first (see [ADR 0031](./docs/adr/0031-stream-observation-network-classification-context-recommendations.md)): the stream-teeing claim was the real gap (no observer utility existed, and `OllamaStream`'s dual modes are mutually exclusive — you couldn't tap a stream you were iterating); the context-overflow claim was comprehensively stale (a superior CJK-aware estimator with three-tier window resolution, `onContextOverflow` policies, and `getContextLength()` discovery has shipped since the v1.9.0 era — only its concrete-window suggestion was missing); and the socket-retry claim was inverted on the headline (every SDK-routed call already retries with backoff _and_ fails over — three retries where the digest asked for one) but real in one seam: network-error classification was `TypeError`-only, so code-carrying socket errors from custom `fetch` implementations and nested undici `cause` chains died un-retried on first attempt. All additive, no breaking changes. 977 pre-existing tests stay green, 35 new tests added.
+
+### Zero-copy stream observation (`observeStream`)
+
+- **`observeStream(source, callbacks)`** — the safe multi-sink alternative to `ReadableStream.tee()`: one queue, no forked branch, no shared backpressure. Tap any `AsyncIterable` (streaming `chat`/`generate` streams, NDJSON chunk iterables, your own sources) for audit logging, token counting, or telemetry while the primary consumer iterates it.
+- The contract is one-sided in favor of the primary consumer: observers **never block** (a serialized microtask chain — sync observers pay one hop, async observers run strictly FIFO with concurrency of exactly one, so a slow logger throttles only its own tail), **never fail the stream** (sync throws and async rejections route to `onListenerError` with the phase they came from — never unhandled rejections, never pipeline poisoning; a failing `onListenerError` is swallowed the same way), and **stay ordered** (`onDone` is enqueued only after the last chunk's observer settles, so an audit log has seen every chunk before its "done" marker).
+- **Full lifecycle**: `onDone` on natural completion; `onError` + rethrow on source failure (the observer is transparent, not a circuit breaker); `onAbandon` when the consumer breaks early — firing after the iterator-close protocol has already closed the source. The digest's sketch (`void onChunk(chunk)`, no early-exit hook, fire-and-forget async observers) had exactly these three defects; the semantics are documented in ADR 0031 and pinned by adversarial tests (deferred resolution out of order, a never-resolving sink, `process`-level unhandled-rejection capture).
+- Exported from the root entry alongside the rest of the streaming toolkit — a `./stream` subpath for a single function was declined (subpaths are reserved for distinct runtimes/concerns like `./testing` and `./mcp/*`).
+
+### Network-error classification (stale-socket recovery for every `fetch` shape)
+
+- `mapError` now recognizes socket/connection-level failures **structurally**: any error carrying a `code` from the exported `NETWORK_ERROR_CODES` set (`ECONNRESET`, `ECONNREFUSED`, `EPIPE`, `ETIMEDOUT`, `EAI_AGAIN`, `UND_ERR_SOCKET`, `UND_ERR_CONNECT_TIMEOUT`, `UND_ERR_HEADERS_TIMEOUT`, `UND_ERR_BODY_TIMEOUT`, `UND_ERR_CLOSED`, …), including codes nested in `cause` chains (bounded depth, cycle-safe), maps to retryable + failover-eligible `OllamaNetworkError`. Previously only WHATWG `fetch`'s `TypeError` wrapper was classified — a custom `fetch` implementation built on `node:http` (whose `ECONNRESET` is a plain `Error` with a `code`) failed outright on first attempt with no retry and no failover. The digest's `String(err).includes(...)` matching was declined: message wording isn't a contract, `error.code` is.
+- Two deliberate exclusions, both tested: `UND_ERR_ABORTED` stays an abort (retrying it would re-dispatch a request the caller explicitly cancelled), and non-transport codes like `EACCES` stay non-retryable.
+- New e2e coverage pins the daemon-restart story end-to-end: a real `node:http` server that destroys the socket on first request per path — `list()` and `ps()` recover via re-dispatch and the caller never sees the reset; an injected `fetch` rejecting once with a raw `ECONNRESET` code retries exactly once and succeeds.
+- The digest's "pre-flight health validation on stale sockets" was declined (undici owns the pool with no staleness introspection; the failed request is the cheapest detector) and its per-method idempotency gating was declined (the retry window is pre-response; mid-stream failures are never auto-retried) — reasoning in ADR 0031.
+
+### Context-window recommendations (the one nugget in a stale claim)
+
+- `ContextCheck` gains `recommendedNumCtx` — present exactly when the threshold is exceeded — computed by the new exported `recommendContextWindow(estimatedTokens)`: the smallest power-of-two window covering `estimate × 1.3`, floored at 4096, via an exact doubling loop (float `Math.log2` of exact powers of two can round a perfect fit up a whole power of two). The pre-flight warning now names the concrete window **with the native-ceiling caveat** — verify the model's maximum via `client.models.getContextLength()` before adopting it, because a prompt that outgrows the native maximum needs compaction, not a bigger `num_ctx`.
+- Non-finite/non-positive inputs degrade to the 4096 floor rather than throwing: the function is advisory, invoked after the caller's own estimate already flagged a problem.
+
 ## [1.15.0] — Draft benchmark utility & request-aware mock routes: `compareDraftPerformance`, mock `responder`
 
 The October 13 digest ("Speculative Performance & Telemetry Observations") reviewed how upstream reports speculative efficiency — no dedicated draft-token acceptance counter, throughput-only observability, a noisy signal (rejected drafts _add_ compute on creative prompts; accepted drafts slash `eval_duration` on structured ones) — and asked for a benchmark helper plus mock scenario hooks. Audited against the codebase first (see [ADR 0030](./docs/adr/0030-draft-benchmark-request-aware-mock-routes.md)): the "manual nanosecond ratios" claim was partially stale — `formatTelemetry` derives tokens/second since v1.12.0 and `lab/15` implements the full protocol, but neither reaches npm consumers (`files` publishes only `dist/`), so the packaged-harness gap was real. The mock scenario claims were stale: parallel tool calls in one frame, malformed-argument tool calls (string `arguments`), in-band mid-stream error injection, and 503 bodies all shipped in v1.14.0. The genuine engine gap the audit surfaced: routes couldn't vary responses **per request**. All additive, no breaking changes. 960 pre-existing tests stay green, 17 new tests added.

package/README.md

@@ -46,6 +46,7 @@ Nexum’s model harness (2026-10-09) exercised this SDK via `gemma4:cloud` and `
 - 🩺 **Root Liveness Probe (`ping`)**: `client.ping()` is the official ultra-lightweight `HEAD /` health check — single-shot (no retry, no failover: the answer is about one concrete host), resolves instead of throwing, reports latency; `healthCheck()` remains the fleet-wide audit.
 - 🧼 **Vision History Hygiene**: `sanitizeHistoryForNextTurn()` evicts stale base64 image payloads from consumer-managed multi-turn vision histories — every `/api/chat` request re-sends the whole history, so old images silently bloat payload, context, and heap; the helper keeps images only on the trailing window you choose, conversation text fully preserved.
 - 🌊 **Web Stream Adapters**: Drop-in adapters (`toTextStream`, `toDataStream`, `toResponse`) for Next.js Route Handlers and Vercel AI SDK.
+- 👀 **Zero-Copy Stream Observation (`observeStream`)**: The safe multi-sink alternative to `ReadableStream.tee()` — tap any async stream for audit logging, token counting, or telemetry while the primary consumer iterates it: observers run serialized FIFO on microtasks, a slow async sink (a logger waiting on I/O) never delays or backpressures the consumer, and observer throws/rejections route to an `onListenerError` channel instead of poisoning the pipeline or surfacing as unhandled rejections.
 - 📈 **OpenTelemetry Instrumentation**: Automatic spans for HTTP requests, endpoint failover, chat/generate calls, and agent runs — zero-cost when OpenTelemetry isn't installed.
 - 📊 **Client-Side Quota Monitoring**: `QuotaManager` tracks token/request usage against budgets you configure across rolling windows (e.g. Ollama Cloud's 5-hour session / 7-day weekly resets) and fails fast with `OllamaQuotaExceededError` before a request is sent.
 - ⚡ **Edge Runtime Verified**: CI bundles and runs the client in a real Edge Runtime sandbox (Cloudflare Workers/Vercel Edge-compatible) with zero Node.js APIs.
@@ -352,7 +353,7 @@ const client = new OllamaClient({
 //  likely silently truncate …"
 ```
 
-The estimators are exported for custom pipelines: `estimateTokens(text)` (CJK-aware — ~1 token per CJK character, ~4 chars/token elsewhere), `estimateChatRequestTokens(req)` / `estimateGenerateRequestTokens(req)` (message overhead + tool schemas + `IMAGE_TOKEN_ESTIMATE` per image), and the underlying `checkChatContext` / `checkGenerateContext` / `contextWarningMessage`. All are heuristics (±20–30%) — for exact counts, ask the server (`prompt_eval_count`).
+The estimators are exported for custom pipelines: `estimateTokens(text)` (CJK-aware — ~1 token per CJK character, ~4 chars/token elsewhere), `estimateChatRequestTokens(req)` / `estimateGenerateRequestTokens(req)` (message overhead + tool schemas + `IMAGE_TOKEN_ESTIMATE` per image), and the underlying `checkChatContext` / `checkGenerateContext` / `contextWarningMessage`. Every failing check now carries a concrete `recommendedNumCtx` — the smallest power-of-two window with ~30% headroom (via the exported `recommendContextWindow(estimatedTokens)`) — and the warning says so explicitly, with the caveat to verify the model's native ceiling via `getContextLength()` before adopting it. All are heuristics (±20–30%) — for exact counts, ask the server (`prompt_eval_count`).
 
 ### Discovering the Real Context Window
 
@@ -974,6 +975,44 @@ export async function POST(req: Request) {
 }
 ```
 
+### Zero-Copy Stream Observation (multi-sink without `tee()`)
+
+Feeding the same stream to a UI _and_ a logger (token counter, audit trail, telemetry exporter) is where `ReadableStream.tee()` bites: both branches share one upstream pull, so a slow branch — a logger waiting on a database write — forces every intermediate chunk to be queued in that branch, throttles the fast branch through shared backpressure, and leaks heap if the slow sink stalls. `observeStream` is the single-pass alternative: one queue, no fork, and a contract that is one-sided in favor of the primary consumer.
+
+```typescript
+import { observeStream } from '@nemesis-oss/ollama-sdk';
+
+const stream = await client.chat({ model: 'llama4', messages, stream: true });
+
+let tokens = 0;
+const observed = observeStream(stream, {
+  // Sync or async — async observers run strictly FIFO, one at a time, and
+  // never delay the consumer below.
+  onChunk: (event) => {
+    if (event.type === 'token') {
+      tokens += 1;
+      auditLog.write(event.data.delta); // slow sink: throttles only itself
+    }
+  },
+  onDone: () => auditLog.flush(), // fires after the last chunk's sink settles
+  onAbandon: () => auditLog.markIncomplete(), // consumer broke early
+  onListenerError: (err, phase) => telemetry.count('audit_failure', { err, phase }),
+});
+
+for await (const event of observed) {
+  ui.render(event); // primary consumer — full speed, full order
+}
+```
+
+The guarantees, in priority order:
+
+- **Never blocks.** Observer callbacks run on a serialized microtask chain off the hot path. A slow async observer delays only its own tail; the consumer receives chunks at source speed.
+- **Never fails.** A sync throw or async rejection in any observer routes to `onListenerError` (with the phase it came from: `chunk` / `done` / `error` / `abandoned`) and is otherwise discarded — observer failures can never poison the pipeline or surface as unhandled rejections. A failing `onListenerError` itself is swallowed the same way.
+- **Ordered.** Observers run strictly FIFO with concurrency of exactly one, and `onDone` is enqueued only after the last chunk's observer settles — the audit log has seen every chunk before its "done" marker.
+- **Full lifecycle.** Natural completion fires `onDone`; a source error fires `onError` and re-throws to the consumer (the observer is transparent, not a circuit breaker); an early `break` in the consuming loop fires `onAbandon` — after the source has already been closed by the iterator-`return()` protocol.
+
+Works over any `AsyncIterable<T>` — `OllamaStream` from streaming `chat`/`generate`, raw NDJSON chunk iterables, or your own sources. (Note `OllamaStream` itself is dual-mode but mode-exclusive — `.on()` listeners can't be combined with `for await`; `observeStream` is the supported way to tap a stream you're already iterating.) Stopping early is a plain `break`; if you need programmatic cancellation of an SDK stream, keep a handle on the original stream's `abort()`. The only state the wrapper retains is one closure per backlogged observer callback — if your sink can stall _forever_, bound it on your side (queue, drop policy), exactly as you would any fire-and-forget log sink.
+
 ### OpenAI & Anthropic Compatibility Bridges
 
 ```typescript
@@ -1833,6 +1872,32 @@ favor of healthy endpoints for `cooldownMs`, and only used again — sorted soon
 every endpoint is cooling down. Call `client.healthCheck()` or inspect the registry's `status()` to
 observe per-endpoint circuit state directly.
 
+#### Stale sockets & automatic network-error recovery
+
+Every SDK-routed request already runs through same-endpoint retry (default 3 attempts, jittered
+exponential backoff — tune or disable per client via `retries`) _and_ cross-endpoint failover. That
+covers the classic daemon-restart hazard: when Ollama is restarted or recycles idle keep-alive
+connections, a pooled socket dies mid-dispatch with `ECONNRESET`/`UND_ERR_SOCKET`, the request is
+re-dispatched on a fresh connection, and the caller never sees the reset — `list()`, `show()`,
+`ps()`, `version()`, and every other call alike.
+
+What counts as a network failure is decided **structurally**, never by message text: `mapError`
+classifies WHATWG `fetch`'s `TypeError` rejections, any error carrying a `code` from
+`NETWORK_ERROR_CODES` (`ECONNRESET`, `ECONNREFUSED`, `EPIPE`, `ETIMEDOUT`, `EAI_AGAIN`,
+`UND_ERR_SOCKET`, `UND_ERR_CONNECT_TIMEOUT`, `UND_ERR_HEADERS_TIMEOUT`, `UND_ERR_BODY_TIMEOUT`,
+`UND_ERR_CLOSED`, …), and codes found nested in `cause` chains (bounded depth) — so custom `fetch`
+implementations built on `node:http`, whose socket errors aren't `TypeError`s, get the same
+automatic retry. The predicate (`isNetworkLevelError`) and the code set are exported for your own
+error handling. Two deliberate exclusions: `UND_ERR_ABORTED` stays an abort (retrying it would
+re-dispatch a request the caller explicitly cancelled), and non-transport codes like `EACCES` stay
+non-retryable (retrying a permission error just burns attempts).
+
+A note on idempotency: the retry window is the pre-response phase — for the daemon-restart scenario
+the inference was lost server-side anyway, so re-dispatch does not double-charge. Requests that
+fail _mid-stream_ (after headers) are never auto-retried: partial generation can't be safely
+replayed. For a manual pre-flight when you'd rather probe than recover (e.g. right after a
+supervised daemon upgrade), `client.ping()` is the single-shot `HEAD /` liveness check.
+
 ---
 
 ## Documentation

package/dist/{anthropic-DrDUfbfQ.d.ts → anthropic-AKbXnWJU.d.cts}

@@ -1,4 +1,4 @@
-import { O as OllamaRuntime, H as HttpClient, A as AbortableAsyncIterable, af as SseEvent, y as OllamaClientError } from './runtime-BIjr5ME4.js';
+import { O as OllamaRuntime, H as HttpClient, A as AbortableAsyncIterable, ag as SseEvent, z as OllamaClientError } from './runtime-CdZP-N7j.cjs';
 
 /**
  * AUTO-GENERATED by `npm run contract:generate`. DO NOT EDIT.

package/dist/{anthropic-UeaVjLeP.d.cts → anthropic-CTv2acbj.d.ts}

@@ -1,4 +1,4 @@
-import { O as OllamaRuntime, H as HttpClient, A as AbortableAsyncIterable, af as SseEvent, y as OllamaClientError } from './runtime-C4RRHqOs.cjs';
+import { O as OllamaRuntime, H as HttpClient, A as AbortableAsyncIterable, ag as SseEvent, z as OllamaClientError } from './runtime-CFuv7Ifh.js';
 
 /**
  * AUTO-GENERATED by `npm run contract:generate`. DO NOT EDIT.

package/dist/generated-api.d.cts

@@ -1,7 +1,7 @@
-import { O as OpenAIChatCompletionRequest, a as OpenAIChatCompletionResponse, b as OpenAIChatCompletionChunk, c as OpenAICompletionRequest, d as OpenAICompletionResponse, e as OpenAICompletionChunk, f as OpenAIEmbeddingRequest, g as OpenAIEmbeddingResponse, h as OpenAIListModelsResponse, i as OpenAIModelItem, j as OpenAIResponsesRequest, k as OpenAIResponsesResponse, l as OpenAIResponsesStreamEvent, A as AnthropicMessagesRequest, m as AnthropicMessagesResponse, n as AnthropicMessageStreamEvent } from './anthropic-UeaVjLeP.cjs';
-export { N as NativeApi } from './anthropic-UeaVjLeP.cjs';
-import { O as OllamaRuntime } from './runtime-C4RRHqOs.cjs';
-export { a as OperationDefinition } from './runtime-C4RRHqOs.cjs';
+import { O as OpenAIChatCompletionRequest, a as OpenAIChatCompletionResponse, b as OpenAIChatCompletionChunk, c as OpenAICompletionRequest, d as OpenAICompletionResponse, e as OpenAICompletionChunk, f as OpenAIEmbeddingRequest, g as OpenAIEmbeddingResponse, h as OpenAIListModelsResponse, i as OpenAIModelItem, j as OpenAIResponsesRequest, k as OpenAIResponsesResponse, l as OpenAIResponsesStreamEvent, A as AnthropicMessagesRequest, m as AnthropicMessagesResponse, n as AnthropicMessageStreamEvent } from './anthropic-AKbXnWJU.cjs';
+export { N as NativeApi } from './anthropic-AKbXnWJU.cjs';
+import { O as OllamaRuntime } from './runtime-CdZP-N7j.cjs';
+export { a as OperationDefinition } from './runtime-CdZP-N7j.cjs';
 import './types-DbPnfVh-.cjs';
 import 'zod';
 

package/dist/generated-api.d.ts

@@ -1,7 +1,7 @@
-import { O as OpenAIChatCompletionRequest, a as OpenAIChatCompletionResponse, b as OpenAIChatCompletionChunk, c as OpenAICompletionRequest, d as OpenAICompletionResponse, e as OpenAICompletionChunk, f as OpenAIEmbeddingRequest, g as OpenAIEmbeddingResponse, h as OpenAIListModelsResponse, i as OpenAIModelItem, j as OpenAIResponsesRequest, k as OpenAIResponsesResponse, l as OpenAIResponsesStreamEvent, A as AnthropicMessagesRequest, m as AnthropicMessagesResponse, n as AnthropicMessageStreamEvent } from './anthropic-DrDUfbfQ.js';
-export { N as NativeApi } from './anthropic-DrDUfbfQ.js';
-import { O as OllamaRuntime } from './runtime-BIjr5ME4.js';
-export { a as OperationDefinition } from './runtime-BIjr5ME4.js';
+import { O as OpenAIChatCompletionRequest, a as OpenAIChatCompletionResponse, b as OpenAIChatCompletionChunk, c as OpenAICompletionRequest, d as OpenAICompletionResponse, e as OpenAICompletionChunk, f as OpenAIEmbeddingRequest, g as OpenAIEmbeddingResponse, h as OpenAIListModelsResponse, i as OpenAIModelItem, j as OpenAIResponsesRequest, k as OpenAIResponsesResponse, l as OpenAIResponsesStreamEvent, A as AnthropicMessagesRequest, m as AnthropicMessagesResponse, n as AnthropicMessageStreamEvent } from './anthropic-CTv2acbj.js';
+export { N as NativeApi } from './anthropic-CTv2acbj.js';
+import { O as OllamaRuntime } from './runtime-CFuv7Ifh.js';
+export { a as OperationDefinition } from './runtime-CFuv7Ifh.js';
 import './types-DbPnfVh-.js';
 import 'zod';