@nemesis-oss/ollama-sdk 1.16.0 → 1.18.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +60 -0
- package/README.md +127 -1
- package/dist/{anthropic-AKbXnWJU.d.cts → anthropic-BvHOsidk.d.cts} +3 -3
- package/dist/{anthropic-CTv2acbj.d.ts → anthropic-CvGVSoVP.d.ts} +3 -3
- package/dist/generated-api.d.cts +2 -2
- package/dist/generated-api.d.ts +2 -2
- package/dist/generated-models-schemas.cjs +2 -2
- package/dist/generated-models-schemas.cjs.map +1 -1
- package/dist/generated-models-schemas.js +2 -2
- package/dist/generated-models-schemas.js.map +1 -1
- package/dist/generated-runtime.cjs +2 -2
- package/dist/generated-runtime.cjs.map +1 -1
- package/dist/generated-runtime.js +2 -2
- package/dist/generated-runtime.js.map +1 -1
- package/dist/index.cjs +180 -33
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +255 -14
- package/dist/index.d.ts +255 -14
- package/dist/index.js +177 -34
- package/dist/index.js.map +1 -1
- package/dist/mcp/tools.json +6 -8
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,65 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [1.18.0] — Pull/push progress with speed & ETA, decision confidence gates, abort-safety pin, System One score 26→10 refresh
|
|
4
|
+
|
|
5
|
+
The October 15 digest ("Structured Output Type Inference, Stream Cancellation Memory Barriers, Model Pull/Push Progress Multiplexing") plus its morning upstream-specs companion (System One confidence calibration, a claimed zero-truncation 413 rule) were audited against the codebase before any implementation (see [ADR 0033](./docs/adr/0033-progress-speed-decision-gates-abort-pin-systemone-refresh.md)). Verdicts: the structured-output claim was stale (`chatWithSchema`/`generateWithSchema` already ship full Zod inference + validation + markdown-JSON extraction — the proposed `structured?: T` field injection was declined as both weaker and contract-breaking); the stream-abort claim was stale as a defect (every teardown seam is guarded — most load-bearingly, `holdUntil` attaches a catch handler to every client stream's `finalResult` before the stream is returned) but the invariant is now pinned by real-wire tests; and the pull/push claim was half right — `onProgress()` with a pre-computed `percent` existed internally with 13 tests but was **not exported from the root entry**, and transfer speed didn't exist at all. The morning digest's 413/2-token scoring-head specification appears nowhere upstream (the real `/v1/systemone` 413 is the request-byte cap the SDK has enforced client-side since the System One waves); its one honest catch — taken from its own "OpenAPI Parity Inspection" focus item — is real: upstream tightened System One score criteria from 2–26 to 2–10 levels. All additive, no breaking changes. 1024 pre-existing tests stay green, 23 new tests added.
|
|
6
|
+
|
|
7
|
+
### Model pull & push progress, with speed and ETA (OPR-01)
|
|
8
|
+
|
|
9
|
+
- **`onProgress(stream, callback)` is now root-exported** — it was fully implemented (typed `PullProgressEvent` with a clamped two-decimal `percent`, `computeProgressPercent()`, `toPullProgressEvent()`, 13 tests) but absent from `src/index.ts`, so npm consumers couldn't import any of it. All four functions and the three types are public now.
|
|
10
|
+
- **`createProgressSpeedTracker()`** — the stateful per-layer byte counter the digest correctly said consumers were forced to hand-roll: feed it every raw progress chunk and it returns `bytesPerSecond` + `etaSeconds` for that chunk's layer. Semantics chosen for stable progress bars: a **cumulative average** since the layer's first byte-bearing frame (instantaneous frame-to-frame rates jitter wildly), **per-layer epochs** (the tracker resets whenever `digest` changes or `completed` regresses, so a multi-layer pull never averages across layers — the classic hand-rolled-counter bug), `performance.now()` monotonic timing, and guards for zero-elapsed, zero-transfer, and missing-total frames (ETA `undefined`, never `NaN` or fake `0`).
|
|
11
|
+
- `onProgress` wires the tracker in: every callback event now carries `bytesPerSecond`/`etaSeconds` alongside `percent` once two byte-bearing frames of a layer have arrived. The pure wrapper (`toPullProgressEvent`) never measures time — deterministic tests mock `performance.now()` with an exact manually-advanced clock.
|
|
12
|
+
- Declined: a `client.pullWithProgress()` method or progress-multiplexer object — the typed dual-mode stream already _is_ the multiplexed channel; a callback helper composes over it without forking the request path (ADR 0033).
|
|
13
|
+
|
|
14
|
+
### System One decision confidence gates (morning-digest kernel)
|
|
15
|
+
|
|
16
|
+
- The six `client.decision` helpers now take **opt-in** calibration gates for autonomous flows — the digest's real concern (an agent acting on a 0.51 coin-flip `true` it never inspected):
|
|
17
|
+
- `choice()` / `score()` / `route()` / `rank()` accept `minConfidence` (inclusive lower bound in (0, 1]; `RangeError` otherwise, validated **before** any request is sent) and flag results with `isConfident: confidence >= minConfidence`. The value is still returned — the caller decides whether to act.
|
|
18
|
+
- `noul()` / `verify()` accept `minProbability`, which raises the bar their derived boolean must clear (default 0.5 unchanged). Noul answers carry no separate confidence field, so the threshold _is_ the boolean bar — a redundant `isConfident` would always equal `bool`.
|
|
19
|
+
- Without the options, every result shape and default behavior is byte-identical — gates are purely additive, and `isConfident` is present only when configured.
|
|
20
|
+
- Declined with reasoning in ADR 0033: the `SystemOneDecisionHelper.evaluateDecisions` wrapper class (zero config around one map iteration) and the `ContextBudgetGuard` token estimator (a pre-flight guard against a spec that doesn't exist — the claimed HTTP-413-on-context-overflow and two-token scoring-head reservation appear in neither the official decision docs nor the OpenAPI; inventing an estimation formula for undocumented server behavior manufactures false confidence).
|
|
21
|
+
|
|
22
|
+
### Stream abort safety (STR-03 — stale defect, invariant pinned)
|
|
23
|
+
|
|
24
|
+
- The digest's claimed unhandled-rejection race is guarded at every seam: `requestStream` wires the caller signal to an `AbortController` with listener cleanup in the generator's `finally`; `parseNdjsonStream` runs under `await using disposableReader` (cancel rejections swallowed, lock released); `OllamaStream.abort()` cancels the source before rejecting `finalResult`; and `executeWithFailover` attaches `void holdUntil(stream).catch(() => undefined)` to every client-produced stream's `finalResult` **synchronously, before the stream is returned** — the internally-held promise can never surface as a process-level `unhandledRejection`. The digest's `guardedStreamReader` sketch re-implements this stack with weaker structure.
|
|
25
|
+
- The invariant is now pinned (BUF-01 precedent): `test/stream-abort-mid-read.test.ts` aborts a real wire stream with the next frame genuinely in flight (public mock server, 50 ms chunk delays) in both iterator and events mode — asserting the mapped `OllamaAbortError` is delivered to the consumer, zero process-level `unhandledRejection` events fire after the rejection chains settle, the endpoint slot releases (`activeRequests` back to 0 under `maxConcurrentPerEndpoint: 1`), and a follow-up request on that single-slot client completes immediately.
|
|
26
|
+
|
|
27
|
+
### System One contract refresh (score criteria 2–26 → 2–10)
|
|
28
|
+
|
|
29
|
+
- Upstream's `docs/openapi.yaml` now natively carries the System One schemas (the SDK pin previously hand-assembled them from the rendered docs) and tightens **score criteria to `maxItems: 10`** (answer scale `maximum: 9`), drops the noul criteria's hardcoded `'Yes'`/`'No'` defaults ("omitted entries use model-specific defaults"), and simplifies the model/images descriptions. The pin was hand-refreshed in its cleaned style (running `contract:fetch` verbatim would clobber its documented intentional deviations) and the contract regenerated: IR, `SystemOneScoreQuestion`/`SystemOneScoreAnswer` models + Zod schemas (`score: z.number().min(0).max(9)`), and MCP tool descriptors. Boundary tests updated (11-criteria requests now rejected client-side before the round-trip; 10-criteria accepted at the new boundary) with the upstream change date recorded.
|
|
30
|
+
- `decision.ts`'s score JSDoc and the README examples now say 2–10.
|
|
31
|
+
|
|
32
|
+
### Structured outputs (TYP-04 — stale, documented)
|
|
33
|
+
|
|
34
|
+
- `chatWithSchema<T>(req, zodSchema)` / `generateWithSchema<T>(req, zodSchema)` already deliver exactly what the digest asked for — inferred `T` returns, runtime validation (`OllamaToolValidationError` with Zod issues when the model's JSON drifts from the schema), and resilient extraction from markdown-fenced output — all root-exported and README-documented since the Zod waves. The digest's redesign (a `structured?: T` field on the response message, populated by a blind `JSON.parse(...) as T`) was declined: no runtime validation is precisely the unsafety the finding complains about, and injecting SDK-computed fields into wire-contract response types breaks the contract-parity discipline (ADR 0033).
|
|
35
|
+
|
|
36
|
+
## [1.17.0] — Streaming blob uploads (`duplex:'half'`, two-pass file upload), typed-array view-safety pins, tracing-header verdicts
|
|
37
|
+
|
|
38
|
+
The October 14 digest ("Raw Model Blob Ingestion Streaming Backpressure, W3C Distributed Tracing Baggage Injection, Defensive Serialization for Typed Arrays") targeted heavy binary asset streaming, operational provenance, and low-level buffer safety. Audited against the codebase first (see [ADR 0032](./docs/adr/0032-streaming-blob-uploads-tracing-verdicts-view-safety.md)): the heap-exhaustion claim was real on the blob path — `createBlobFromFile` bulk-read the whole file (`fs.readFile`) before uploading, and the `ReadableStream` member of `BinaryBody` was a promise the transport couldn't keep (Node 18+ `fetch` rejects stream bodies without `duplex: 'half'`) — though the `/api/create` framing was misdirected (its request body is a small JSON digest map, not a stream). The tracing claim was stale (client-level `headers`, per-request headers, and the middleware seam already forward any W3C headers verbatim — nothing is stripped). The typed-array claim was stale as a defect (`src/` has zero `Buffer.from` call sites; the SDK's Web APIs are view-safe by construction, and adopting the proposed `Buffer`-coupled helpers would regress the edge-runtime zero-Node-builtins gate) — but the invariant is now pinned by tests. The companion upstream-specs digest (System One batching, payload caps, `expires_at`, model warmth) was comprehensively stale — every recommendation already ships. All additive, no breaking changes. 1012 pre-existing tests stay green, 12 new tests added.
|
|
39
|
+
|
|
40
|
+
### Streaming blob uploads (MOD-01 — multi-GB GGUFs never enter the heap)
|
|
41
|
+
|
|
42
|
+
- **Transport**: both fetch call sites (middleware `finalHandler` and the direct path, previously duplicated init construction) now share one `buildFetchInit` that sets `duplex: 'half'` exactly when the body is a web `ReadableStream`. The guard is structural (`getReader` duck-typing) so cross-realm streams (workers, VM contexts) still get the member; the init type is a local `RequestInit & { duplex?: 'half' }` intersection because `@types/node` 22's `RequestInit` doesn't carry the member yet — assignable to the plain `RequestInit` `FetchLike` expects. Harmless (and spec'd) in browsers.
|
|
43
|
+
- **`createBlobFromStream(digest, stream, request?)`** — the multi-gigabyte-safe counterpart of `createBlobFromData`: pipes any web `ReadableStream<Uint8Array>` straight into the request body with HEAD-dedupe semantics identical to the data-based variant. The digest must be known up front — it _is_ the upload URL, and the server verifies streamed content against it, so a truncated or corrupted stream fails the upload instead of poisoning the store.
|
|
44
|
+
- **`createBlobFromFile` rewritten as two streaming passes** (same signature, same result shape): a SHA-256 hash pass through `node:crypto`'s incremental digest in 256 KiB chunks (skipped entirely when `opts.digest` is provided), then an upload pass piping `createReadStream` → `Readable.toWeb` into the streaming upload. Peak heap is O(chunk) in both passes; `createModelFromGguf` (which routes every shard through this method) inherits the bound. The method was already Node-only (hidden-specifier dynamic `node:fs`); the new `node:crypto`/`node:stream` imports follow the same pattern, keeping the main entry edge-portable.
|
|
45
|
+
- **Cancellation plumbing**: `createBlob`/`checkBlob` now accept `RequestCancellationOptions` (`signal`/`timeoutMs`) like their siblings, and `createBlobFromData`'s previously decorative `signal` option is actually wired through. Aborts surface as `OllamaAbortError` from both file passes (the hash pass checks between chunks and destroys the fd; the upload pass forwards the signal to `fetch` and destroys the stream on abort).
|
|
46
|
+
- New coverage pins the whole story: duplex injection (unit, injected `fetch`), byte-exact streamed uploads against a real capture server, the two-pass file flow (streaming digest matches the whole-file digest; wire bytes equal the file byte-for-byte), abort-before-traffic and abort-mid-stalled-upload, and daemonless public-mock round trips including `createModelFromGguf`'s headline flow (blob POST carries the shard, `/api/create` references it under its basename).
|
|
47
|
+
|
|
48
|
+
### Typed-array view safety (BUF-01 — stale defect, pinned invariant)
|
|
49
|
+
|
|
50
|
+
- The audit found zero `Buffer.from(view.buffer)`-style call sites: `computeBlobDigest` uses Web Crypto, which hashes a `BufferSource` view's bytes per spec, and `fetch` serializes `Uint8Array` bodies per-view. The digest's `safeUint8ArrayToBuffer`/`toUint8Array` helpers were declined — they couple a runtime-portable SDK to Node's `Buffer` and would regress the edge-runtime gate (ADR 0032).
|
|
51
|
+
- The invariant is now pinned by the digest's own suggested probe, hardened with an external oracle: a `subarray(10, 20)` view of a 1024-byte patterned pool hashes to the standalone slice's digest (cross-checked against `node:crypto`) and arrives on the wire as exactly its 10 bytes — never the parent buffer.
|
|
52
|
+
- Cleanup: `computeBlobDigest`'s misleading `data as unknown as ArrayBuffer` cast removed (the local structural `SubtleCrypto` type already accepts `ArrayBufferView`).
|
|
53
|
+
|
|
54
|
+
### Tracing headers (TRC-01 — stale, documented)
|
|
55
|
+
|
|
56
|
+
- The SDK forwards headers verbatim through three existing seams: client-level `headers` configuration (every request, every endpoint), per-request `headers` on transport request options (merged last, so they win), and middleware's mutable `RequestContext.headers` record — an OpenTelemetry-style extractor plugs in without touching a single method signature. The README's middleware section now shows the W3C `traceparent`/`tracestate`/`baggage` propagation pattern explicitly.
|
|
57
|
+
- The digest's `traceContext` option on every client method and ambient `AsyncLocalStorage` capture were declined (ADR 0032): signature inflation and hidden global state for capability that already exists.
|
|
58
|
+
|
|
59
|
+
### Upstream-specs digest verdicts (all stale — pointers, no new API)
|
|
60
|
+
|
|
61
|
+
- **System One batching**: `client.systemOne()` already evaluates N named questions against shared state in one round trip, with compile-time question→answer type correlation the sketch lacks. **Payload caps**: `MAX_SYSTEM_ONE_REQUEST_BYTES` (64 KiB) / `MAX_SYSTEM_ONE_IMAGES_BYTES` (32 MiB) are exported, `estimateSystemOneRequestBytes()` is the pre-flight estimator, and the runtime enforces both client-side. **`expires_at`**: typed on `ModelResponse` from `/api/ps`, returned by `client.ps()`. **Model warmth**: `pin()`/`unload()`/`KEEP_ALIVE_INDEFINITE`/per-call `keep_alive`/`warmModelAffinity()` are the supported primitives; a background `ModelWarmthKeeper` timer daemon was declined (hidden event-loop work that escapes the client-teardown contract — reasoning in ADR 0032).
|
|
62
|
+
|
|
3
63
|
## [1.16.0] — Stream observation (`observeStream`), network-error classification & context-window recommendations
|
|
4
64
|
|
|
5
65
|
The October 13 second digest ("Stream Multi-Sink Teeing, Context Window Overflow, Socket Hangup Recovery") targeted edge transport hazards and memory boundaries during high-throughput agent execution. Audited against the codebase first (see [ADR 0031](./docs/adr/0031-stream-observation-network-classification-context-recommendations.md)): the stream-teeing claim was the real gap (no observer utility existed, and `OllamaStream`'s dual modes are mutually exclusive — you couldn't tap a stream you were iterating); the context-overflow claim was comprehensively stale (a superior CJK-aware estimator with three-tier window resolution, `onContextOverflow` policies, and `getContextLength()` discovery has shipped since the v1.9.0 era — only its concrete-window suggestion was missing); and the socket-retry claim was inverted on the headline (every SDK-routed call already retries with backoff _and_ fails over — three retries where the digest asked for one) but real in one seam: network-error classification was `TypeError`-only, so code-carrying socket errors from custom `fetch` implementations and nested undici `cause` chains died un-retried on first attempt. All additive, no breaking changes. 977 pre-existing tests stay green, 35 new tests added.
|
package/README.md
CHANGED
|
@@ -24,16 +24,18 @@ Nexum’s model harness (2026-10-09) exercised this SDK via `gemma4:cloud` and `
|
|
|
24
24
|
- 🧠 **Reasoning & Thinking Tokens**: First-class support for reasoning models (`qwen3:8b`, `deepseek-r1:8b`) via the native `think` parameter, with discrete `thinking` and `token` streaming events, plus `logprobs`/`top_logprobs` for token-level confidence scoring.
|
|
25
25
|
- 🖼️ **Multimodal / Vision Input**: `images` on `Message`/`generate()` is a universal resolver — pass raw base64 strings, `data:image/...;base64,` data URIs, `http(s)://` URLs (fetched), local file paths (`.png`/`.jpg`/`.webp`/…, read via `node:fs` on Node), or raw `Buffer`/`Uint8Array` bytes — everything is normalized to the raw base64 Ollama's REST API expects, matching the official SDK vision convention.
|
|
26
26
|
- 🎯 **Zod-Powered Structured Outputs**: Strictly typed schema enforcement via `chatWithSchema` and `generateWithSchema` with resilient markdown JSON parsing.
|
|
27
|
+
- ⚖️ **System One Decision Helpers**: `client.decision` wraps Ollama's `/v1/systemone` classification engine with six ergonomic single-question APIs — `choice` / `noul` (yes-no) / `score` / `route` / `verify` / `rank` — each returning calibrated probabilities, and **opt-in confidence gates** (`minConfidence` / `minProbability`) so autonomous flows refuse to act on barely-majority decisions (a 0.51 coin-flip) without seeing the numbers.
|
|
27
28
|
- 🛠️ **Autonomous Agent & Tool Calling**: Multi-turn agent loop (`Agent`) with automated tool execution, parameter validation, and self-correcting error recovery.
|
|
28
29
|
- 🌐 **High Availability & Failover**: Multi-endpoint registry with priority routing, circuit breaker failover, active health checks, and per-endpoint `models` allow-lists for routing several model-specific API keys through one client.
|
|
29
30
|
- 🔌 **Model Context Protocol (MCP)**: First-class, transport-neutral `McpBridge` for converting MCP tool descriptors into Ollama function definitions and registering executable MCP-backed tools.
|
|
30
31
|
- 📚 **Full Model Lifecycle**: `pullModel`, `pushModel`, `createModel`, `copyModel`, `deleteModel`, `listModels`, `showModel`, and `ps()` (currently loaded models) — full parity with Ollama's model management API.
|
|
32
|
+
- 📶 **Pull & Push Progress with Speed/ETA**: `onProgress()` subscribes a typed callback to `pullModel`/`pushModel` NDJSON streams with a pre-computed `percent`, and `createProgressSpeedTracker()` adds per-layer `bytesPerSecond` + `etaSeconds` — the stateful byte counter CLIs and dashboards used to hand-roll, with layer-epoch reset semantics built in.
|
|
31
33
|
- 🔎 **Ollama Cloud Web Tools**: `webSearch`/`webFetch` wrap Ollama's hosted `/api/web_search` and `/api/web_fetch` tools at `ollama.com` (requires an `OLLAMA_API_KEY`), independent of any local `baseUrl` — and `createWebTools(client)` exposes them as model-callable `web_search`/`web_fetch` tools with bounded, valid-JSON output for the documented search-agent pattern.
|
|
32
34
|
- 💳 **Ollama Cloud Usage & Balance**: `usage()`/`balance()` wrap Ollama's hosted account endpoints (`GET /api/usage`, `GET /api/balance`) — request counts, USD spend, cached-token totals bucketed by hour/day, and remaining included/purchased credits (including the legacy session/weekly plan shapes) — with the same fixed-cloud-host, API-key, timeout, and retry semantics as the web tools.
|
|
33
35
|
- 🌉 **OpenAI & Anthropic Compatibility Bridges**: Built-in clients for `/v1/chat/completions`, `/v1/responses`, `/v1/models`, and `/v1/messages`, including `reasoning_effort`/`reasoning.effort` for thinking models — plus an ergonomic dual-mode `client.responses.create()` bridge that prefers native `/v1/responses` and transparently re-issues via `/api/chat` on older servers (pre-v0.13.3).
|
|
34
36
|
- 💬 **KV-Cache-Aware Conversation Sessions**: `client.session(model, systemPrompt)` keeps an append-only, prefix-stable history that maximizes Ollama's KV-cache reuse across turns, surfaces per-turn + cumulative cache hit rates (`prompt_eval_cached_count`) so cache degradation is visible instead of silent, and offers a one-call `compact()` sliding-window escape hatch when a session outgrows its window.
|
|
35
37
|
- 🛑 **Context-Window Safety**: Heuristic client-side token estimation warns (or throws, `onContextOverflow: 'throw'`) before a request is sent when the prompt approaches the effective `num_ctx` window — Ollama's default behavior is to _silently truncate_ oversized prompts — and `defaultContextLength` makes the window explicit on every request.
|
|
36
|
-
- 📦 **Blob & Custom Model Publishing**: Content-addressed blob management per Ollama's documented protocol — `computeBlobDigest()` (SHA-256 → `sha256:<hex>`), `createBlobFromData()`/`createBlobFromFile()` (HEAD-check + upload, skipping existing blobs), and one-shot `createModelFromGguf()` (upload GGUF shard blobs, then `/api/create` with `files: {name: digest}`) — with **speculative-decoding assembly**: `draftGguf` uploads the draft network's shards into `draft_files`, `draft_quantize` compresses them independently, and `parameters.draft_num_predict` tunes the per-step draft length.
|
|
38
|
+
- 📦 **Blob & Custom Model Publishing**: Content-addressed blob management per Ollama's documented protocol — `computeBlobDigest()` (SHA-256 → `sha256:<hex>`), `createBlobFromData()`/`createBlobFromFile()` (HEAD-check + upload, skipping existing blobs), **streaming uploads with O(chunk) heap** (`createBlobFromFile` hashes and uploads in 256 KiB chunks — multi-GB GGUFs never enter the Node heap; `createBlobFromStream()` pipes any web `ReadableStream` straight into `fetch` with `duplex: 'half'`), and one-shot `createModelFromGguf()` (upload GGUF shard blobs, then `/api/create` with `files: {name: digest}`) — with **speculative-decoding assembly**: `draftGguf` uploads the draft network's shards into `draft_files`, `draft_quantize` compresses them independently, and `parameters.draft_num_predict` tunes the per-step draft length.
|
|
37
39
|
- 🏁 **Speculative Draft Benchmark (`compareDraftPerformance`)**: One call answers "does drafting pay off for _this_ prompt?" — identical prompt on both sides, deterministic sampling (`temperature: 0` + fixed seed), warm-up discard, several samples per side with **medians** compared, throughput via `formatTelemetry` — reported as baseline vs. speculative tok/s, `speedupMultiplier`, and `effectiveGain` past a configurable threshold. A ≈1.0 speedup is itself diagnostic: the model has no draft network, so the server silently ignores the option.
|
|
38
40
|
- 🧪 **In-Memory Mock Server (`@nemesis-oss/ollama-sdk/testing`)**: The SDK's own deterministic test double, promoted public — `createMockOllamaServer()` with scriptable NDJSON routes, built-in liveness/version defaults (so `ping()` works out of the box), request recording for wire-level assertions, **request-aware routes** (`responder` — per-request responses for fail-twice-then-succeed retry drills and body-dependent payloads), mid-stream **in-band error injection** (`{"error": "..."}` frames at HTTP 200, exactly how a real daemon reports GPU OOM), connection drops, and correctly-shaped chunk builders for streaming text and **parallel tool calls** — zero network, zero daemon, CI-deterministic.
|
|
39
41
|
- 🧵 **Batched Embeddings with Backpressure**: `client.embedBatch()` splits corpora of any size into bounded `/api/embed` batches (default 32 inputs × 3 in flight) — order-preserving, fail-fast, progress-reporting — so RAG/vector ingestion can't OOM a local daemon or saturate `OLLAMA_MAX_QUEUE` the way `Promise.all` floods do, with per-string context-window pre-flight against silent truncation.
|
|
@@ -500,6 +502,34 @@ await client.models.createModelFromGguf('my-model', [
|
|
|
500
502
|
]);
|
|
501
503
|
```
|
|
502
504
|
|
|
505
|
+
#### Streaming uploads — multi-gigabyte weights with O(chunk) heap
|
|
506
|
+
|
|
507
|
+
Both `createBlobFromFile` and `createModelFromGguf` (which routes every shard
|
|
508
|
+
through it) stream instead of bulk-reading: the SHA-256 hash pass and the
|
|
509
|
+
upload pass each consume the file in 256 KiB chunks, so peak heap is a chunk,
|
|
510
|
+
never the file. Pass `digest` when you already know it to skip the hash pass
|
|
511
|
+
entirely. When you own the bytes (a chunk producer, a WASM/SharedArrayBuffer
|
|
512
|
+
pipeline, a resumed download), `createBlobFromStream` pipes any web
|
|
513
|
+
`ReadableStream<Uint8Array>` straight into the request body — the transport
|
|
514
|
+
sets `duplex: 'half'` (required by Node 18+ `fetch` for stream bodies)
|
|
515
|
+
automatically, and the server verifies the streamed content against the
|
|
516
|
+
digest, so a truncated stream fails the upload instead of corrupting the
|
|
517
|
+
store:
|
|
518
|
+
|
|
519
|
+
```typescript
|
|
520
|
+
// digest known up front (hash first when it isn't — the digest IS the URL)
|
|
521
|
+
const { alreadyExisted } = await client.models.createBlobFromStream(
|
|
522
|
+
'sha256:…',
|
|
523
|
+
myChunkProducer, // ReadableStream<Uint8Array>
|
|
524
|
+
);
|
|
525
|
+
|
|
526
|
+
// Both methods accept { signal, timeoutMs } for cancellation/timeout:
|
|
527
|
+
await client.models.createBlobFromFile('/models/model.gguf', {
|
|
528
|
+
digest: 'sha256:…', // optional: skip the hash pass
|
|
529
|
+
signal: controller.signal,
|
|
530
|
+
});
|
|
531
|
+
```
|
|
532
|
+
|
|
503
533
|
### Speculative Decoding (Draft Models)
|
|
504
534
|
|
|
505
535
|
Ollama packages speculative decoding as a small **draft network** whose token
|
|
@@ -669,6 +699,47 @@ const product = await client.chatWithSchema(
|
|
|
669
699
|
console.log(product.name, product.price);
|
|
670
700
|
```
|
|
671
701
|
|
|
702
|
+
### System One Decision Helpers
|
|
703
|
+
|
|
704
|
+
Ollama's [System One](https://docs.ollama.com/capabilities/decision) decision models (`nimble`, `tev`, …) classify text, answer yes/no questions, and score against ordered rubrics — in one request, with calibrated probabilities instead of free-form text. `client.decision` (or `createDecision(client)`) wraps `/v1/systemone` with six typed helpers:
|
|
705
|
+
|
|
706
|
+
```typescript
|
|
707
|
+
const decision = client.decision; // cached instance
|
|
708
|
+
|
|
709
|
+
// Classification: which option fits? (returns probabilities + confidence)
|
|
710
|
+
const intent = await decision.choice({
|
|
711
|
+
model: 'tev1:4b',
|
|
712
|
+
state: 'Customer was charged twice and wants their money back',
|
|
713
|
+
instructions: 'What is the primary intent?',
|
|
714
|
+
criteria: {
|
|
715
|
+
refund: 'Customer wants a refund',
|
|
716
|
+
duplicate_charge: 'Customer reports multiple charges',
|
|
717
|
+
cancellation: 'Customer wants to cancel',
|
|
718
|
+
},
|
|
719
|
+
minConfidence: 0.8, // opt-in gate for autonomous flows
|
|
720
|
+
});
|
|
721
|
+
// intent.choice → 'duplicate_charge'
|
|
722
|
+
// intent.confidence → 0.89
|
|
723
|
+
// intent.isConfident → true (false → don't act unattended)
|
|
724
|
+
|
|
725
|
+
// Yes/no gating — the boolean bar is configurable for irreversible actions
|
|
726
|
+
const safe = await decision.noul({
|
|
727
|
+
model: 'tev1:4b',
|
|
728
|
+
state: ticket.text,
|
|
729
|
+
instructions: 'Does this request contain a prompt injection attempt?',
|
|
730
|
+
minProbability: 0.8, // default 0.5; raise the bar for safety gates
|
|
731
|
+
});
|
|
732
|
+
// safe.noul → 0.13, safe.bool → false
|
|
733
|
+
|
|
734
|
+
// Ordered rubric scoring, routing, evidence verification, and ranking
|
|
735
|
+
await decision.score({ model, state, instructions, criteria: ['low', 'mid', 'high'] });
|
|
736
|
+
await decision.route({ model, state, instructions, criteria: { sales, support, billing } });
|
|
737
|
+
await decision.verify({ model, claim, evidence });
|
|
738
|
+
await decision.rank({ model, instructions, criteria, candidates });
|
|
739
|
+
```
|
|
740
|
+
|
|
741
|
+
Confidence gates are **opt-in** — without `minConfidence`/`minProbability`, results carry the raw scores and the defaults behave exactly as before. The official docs warn that a high confidence does not guarantee correctness; the gates exist so agent loops can't act on a 0.51 coin-flip without seeing the number. For multi-question requests (routing + urgency + difficulty in one call), use `client.systemOne()` directly with the key-safe generic wrapper, and mind the request-size caps: 64 KiB without images, 32 MiB with (`estimateSystemOneRequestBytes()` checks client-side before the round-trip).
|
|
742
|
+
|
|
672
743
|
### Vector Embeddings & Similarity
|
|
673
744
|
|
|
674
745
|
```typescript
|
|
@@ -735,6 +806,42 @@ await client.pushModel({ model: 'my-namespace/my-assistant' });
|
|
|
735
806
|
await client.deleteModel({ model: 'my-assistant-backup' });
|
|
736
807
|
```
|
|
737
808
|
|
|
809
|
+
### Model Pull & Push Progress
|
|
810
|
+
|
|
811
|
+
`pullModel`/`pushModel` with `stream: true` return a typed `OllamaStream` of NDJSON progress frames. `onProgress()` subscribes a callback with the fields progress bars actually need — a pre-computed `percent`, plus per-layer `bytesPerSecond` and `etaSeconds` once two byte-bearing frames have arrived:
|
|
812
|
+
|
|
813
|
+
```typescript
|
|
814
|
+
import { onProgress } from '@nemesis-oss/ollama-sdk';
|
|
815
|
+
|
|
816
|
+
const stream = await client.pullModel({ model: 'llama3.2', stream: true });
|
|
817
|
+
await onProgress(stream, (event) => {
|
|
818
|
+
if (event.percent !== undefined) {
|
|
819
|
+
const speed = event.bytesPerSecond
|
|
820
|
+
? ` at ${(event.bytesPerSecond / 1024 / 1024).toFixed(1)} MiB/s`
|
|
821
|
+
: '';
|
|
822
|
+
const eta = event.etaSeconds ? `, ETA ${Math.ceil(event.etaSeconds)}s` : '';
|
|
823
|
+
console.log(`${event.status}: ${event.percent.toFixed(2)}%${speed}${eta}`);
|
|
824
|
+
} else {
|
|
825
|
+
console.log(event.status); // "pulling manifest", "verifying sha256 digest", …
|
|
826
|
+
}
|
|
827
|
+
});
|
|
828
|
+
console.log('pull complete');
|
|
829
|
+
```
|
|
830
|
+
|
|
831
|
+
Speed semantics are chosen for stable UIs: a **cumulative average** since the layer's first byte-bearing frame (instantaneous frame-to-frame rates jitter wildly), with automatic **per-layer epochs** — the tracker resets whenever the `digest` changes or `completed` regresses, so a 4-layer pull never averages across layers. Consumers iterating the raw stream themselves (structured logging, custom dashboards) can use `createProgressSpeedTracker()` directly — the same stateful byte counter, without hand-rolling one:
|
|
832
|
+
|
|
833
|
+
```typescript
|
|
834
|
+
import { createProgressSpeedTracker } from '@nemesis-oss/ollama-sdk';
|
|
835
|
+
|
|
836
|
+
const speed = createProgressSpeedTracker();
|
|
837
|
+
for await (const event of stream) {
|
|
838
|
+
if (event.type === 'message') {
|
|
839
|
+
const { bytesPerSecond, etaSeconds } = speed(event.data.chunk);
|
|
840
|
+
// …render your own bar
|
|
841
|
+
}
|
|
842
|
+
}
|
|
843
|
+
```
|
|
844
|
+
|
|
738
845
|
### Web Search & Web Fetch (Ollama Cloud)
|
|
739
846
|
|
|
740
847
|
`webSearch`/`webFetch` wrap Ollama's **hosted** web tools (`POST https://ollama.com/api/web_search` and `/api/web_fetch`) — a fixed Ollama Cloud service, entirely separate from whatever local `baseUrl`/`endpoints` the client is configured with. They require an Ollama account API key (`apiKey` on the client, or the `OLLAMA_API_KEY` environment variable) regardless of where your inference traffic goes:
|
|
@@ -1934,6 +2041,25 @@ around the underlying HTTP request for native endpoints, OpenAI/Anthropic compat
|
|
|
1934
2041
|
model health checks, and the hosted web tools. Lifecycle events expose `start`, `success`,
|
|
1935
2042
|
`retry`, and `error` events with a request id, timing, and status/error information.
|
|
1936
2043
|
|
|
2044
|
+
**Distributed tracing (W3C `traceparent`/`tracestate`/`baggage`)**: the SDK forwards
|
|
2045
|
+
whatever headers it is given and never strips or rewrites yours — three seams cover
|
|
2046
|
+
propagation. Client-wide (every request, every endpoint):
|
|
2047
|
+
|
|
2048
|
+
```ts
|
|
2049
|
+
const client = new OllamaClient({
|
|
2050
|
+
headers: {
|
|
2051
|
+
traceparent: carrierHeader('traceparent'), // from your inbound request
|
|
2052
|
+
baggage: carrierHeader('baggage'),
|
|
2053
|
+
},
|
|
2054
|
+
});
|
|
2055
|
+
```
|
|
2056
|
+
|
|
2057
|
+
Per-request or computed (the middleware seam from above — `request.headers` is a mutable
|
|
2058
|
+
record, so an OpenTelemetry-style extractor plugs in directly and can vary headers per
|
|
2059
|
+
call). A `traceContext` option on every SDK method was considered and declined
|
|
2060
|
+
(ADR 0032): it would inflate every signature for something these seams already do
|
|
2061
|
+
without touching them.
|
|
2062
|
+
|
|
1937
2063
|
```ts
|
|
1938
2064
|
const client = new OllamaClient({
|
|
1939
2065
|
middleware: [
|
|
@@ -552,10 +552,10 @@ type SystemOneProbabilities = {
|
|
|
552
552
|
*/
|
|
553
553
|
|
|
554
554
|
interface SystemOneRequest {
|
|
555
|
-
/** Local model trained for System One, such as nimble.
|
|
555
|
+
/** Local model trained for System One, such as nimble. */
|
|
556
556
|
readonly model: string;
|
|
557
557
|
readonly state: SystemOneContent;
|
|
558
|
-
/** Base64-encoded images shared by all questions, in request order. Requires
|
|
558
|
+
/** Base64-encoded images shared by all questions, in request order. Requires a model with vision, such as Clef. URLs and data URLs are not supported. */
|
|
559
559
|
readonly images?: readonly string[] | undefined;
|
|
560
560
|
/** Named questions about the shared state. Answers are not passed to later questions. */
|
|
561
561
|
readonly questions: {
|
|
@@ -614,7 +614,7 @@ interface SystemOneScoreAnswer {
|
|
|
614
614
|
interface SystemOneScoreQuestion {
|
|
615
615
|
readonly type: 'score';
|
|
616
616
|
readonly instructions: SystemOneContent;
|
|
617
|
-
/** Descriptions ordered from the lowest score (index 0) to the highest. Defines a scale from 0 to the number of criteria minus 1.
|
|
617
|
+
/** Descriptions ordered from the lowest score (index 0) to the highest. Defines a scale from 0 to the number of criteria minus 1. */
|
|
618
618
|
readonly criteria: readonly string[];
|
|
619
619
|
}
|
|
620
620
|
|
|
@@ -552,10 +552,10 @@ type SystemOneProbabilities = {
|
|
|
552
552
|
*/
|
|
553
553
|
|
|
554
554
|
interface SystemOneRequest {
|
|
555
|
-
/** Local model trained for System One, such as nimble.
|
|
555
|
+
/** Local model trained for System One, such as nimble. */
|
|
556
556
|
readonly model: string;
|
|
557
557
|
readonly state: SystemOneContent;
|
|
558
|
-
/** Base64-encoded images shared by all questions, in request order. Requires
|
|
558
|
+
/** Base64-encoded images shared by all questions, in request order. Requires a model with vision, such as Clef. URLs and data URLs are not supported. */
|
|
559
559
|
readonly images?: readonly string[] | undefined;
|
|
560
560
|
/** Named questions about the shared state. Answers are not passed to later questions. */
|
|
561
561
|
readonly questions: {
|
|
@@ -614,7 +614,7 @@ interface SystemOneScoreAnswer {
|
|
|
614
614
|
interface SystemOneScoreQuestion {
|
|
615
615
|
readonly type: 'score';
|
|
616
616
|
readonly instructions: SystemOneContent;
|
|
617
|
-
/** Descriptions ordered from the lowest score (index 0) to the highest. Defines a scale from 0 to the number of criteria minus 1.
|
|
617
|
+
/** Descriptions ordered from the lowest score (index 0) to the highest. Defines a scale from 0 to the number of criteria minus 1. */
|
|
618
618
|
readonly criteria: readonly string[];
|
|
619
619
|
}
|
|
620
620
|
|
package/dist/generated-api.d.cts
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
|
-
import { O as OpenAIChatCompletionRequest, a as OpenAIChatCompletionResponse, b as OpenAIChatCompletionChunk, c as OpenAICompletionRequest, d as OpenAICompletionResponse, e as OpenAICompletionChunk, f as OpenAIEmbeddingRequest, g as OpenAIEmbeddingResponse, h as OpenAIListModelsResponse, i as OpenAIModelItem, j as OpenAIResponsesRequest, k as OpenAIResponsesResponse, l as OpenAIResponsesStreamEvent, A as AnthropicMessagesRequest, m as AnthropicMessagesResponse, n as AnthropicMessageStreamEvent } from './anthropic-
|
|
2
|
-
export { N as NativeApi } from './anthropic-
|
|
1
|
+
import { O as OpenAIChatCompletionRequest, a as OpenAIChatCompletionResponse, b as OpenAIChatCompletionChunk, c as OpenAICompletionRequest, d as OpenAICompletionResponse, e as OpenAICompletionChunk, f as OpenAIEmbeddingRequest, g as OpenAIEmbeddingResponse, h as OpenAIListModelsResponse, i as OpenAIModelItem, j as OpenAIResponsesRequest, k as OpenAIResponsesResponse, l as OpenAIResponsesStreamEvent, A as AnthropicMessagesRequest, m as AnthropicMessagesResponse, n as AnthropicMessageStreamEvent } from './anthropic-BvHOsidk.cjs';
|
|
2
|
+
export { N as NativeApi } from './anthropic-BvHOsidk.cjs';
|
|
3
3
|
import { O as OllamaRuntime } from './runtime-CdZP-N7j.cjs';
|
|
4
4
|
export { a as OperationDefinition } from './runtime-CdZP-N7j.cjs';
|
|
5
5
|
import './types-DbPnfVh-.cjs';
|
package/dist/generated-api.d.ts
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
|
-
import { O as OpenAIChatCompletionRequest, a as OpenAIChatCompletionResponse, b as OpenAIChatCompletionChunk, c as OpenAICompletionRequest, d as OpenAICompletionResponse, e as OpenAICompletionChunk, f as OpenAIEmbeddingRequest, g as OpenAIEmbeddingResponse, h as OpenAIListModelsResponse, i as OpenAIModelItem, j as OpenAIResponsesRequest, k as OpenAIResponsesResponse, l as OpenAIResponsesStreamEvent, A as AnthropicMessagesRequest, m as AnthropicMessagesResponse, n as AnthropicMessageStreamEvent } from './anthropic-
|
|
2
|
-
export { N as NativeApi } from './anthropic-
|
|
1
|
+
import { O as OpenAIChatCompletionRequest, a as OpenAIChatCompletionResponse, b as OpenAIChatCompletionChunk, c as OpenAICompletionRequest, d as OpenAICompletionResponse, e as OpenAICompletionChunk, f as OpenAIEmbeddingRequest, g as OpenAIEmbeddingResponse, h as OpenAIListModelsResponse, i as OpenAIModelItem, j as OpenAIResponsesRequest, k as OpenAIResponsesResponse, l as OpenAIResponsesStreamEvent, A as AnthropicMessagesRequest, m as AnthropicMessagesResponse, n as AnthropicMessageStreamEvent } from './anthropic-CvGVSoVP.js';
|
|
2
|
+
export { N as NativeApi } from './anthropic-CvGVSoVP.js';
|
|
3
3
|
import { O as OllamaRuntime } from './runtime-CFuv7Ifh.js';
|
|
4
4
|
export { a as OperationDefinition } from './runtime-CFuv7Ifh.js';
|
|
5
5
|
import './types-DbPnfVh-.js';
|
|
@@ -328,7 +328,7 @@ var SystemOneNoulQuestionSchema = zod.z.object({
|
|
|
328
328
|
var SystemOneScoreQuestionSchema = zod.z.object({
|
|
329
329
|
type: zod.z.enum(["score"]),
|
|
330
330
|
instructions: SystemOneContentSchema,
|
|
331
|
-
criteria: zod.z.array(zod.z.string()).min(2).max(
|
|
331
|
+
criteria: zod.z.array(zod.z.string()).min(2).max(10)
|
|
332
332
|
});
|
|
333
333
|
|
|
334
334
|
// src/generated/models/SystemOneRequest.schema.ts
|
|
@@ -349,7 +349,7 @@ var SystemOneRequestSchema = zod.z.object({
|
|
|
349
349
|
});
|
|
350
350
|
var SystemOneScoreAnswerSchema = zod.z.object({
|
|
351
351
|
type: zod.z.enum(["score"]),
|
|
352
|
-
score: zod.z.number().min(0).max(
|
|
352
|
+
score: zod.z.number().min(0).max(9),
|
|
353
353
|
legend: zod.z.object({}).catchall(zod.z.string()),
|
|
354
354
|
probabilities: SystemOneProbabilitiesSchema,
|
|
355
355
|
confidence: SystemOneConfidenceSchema
|