@nemesis-oss/ollama-sdk 1.13.0 → 1.15.0

package/CHANGELOG.md

@@ -1,5 +1,48 @@
 # Changelog
 
+## [1.15.0] — Draft benchmark utility & request-aware mock routes: `compareDraftPerformance`, mock `responder`
+
+The October 13 digest ("Speculative Performance & Telemetry Observations") reviewed how upstream reports speculative efficiency — no dedicated draft-token acceptance counter, throughput-only observability, a noisy signal (rejected drafts _add_ compute on creative prompts; accepted drafts slash `eval_duration` on structured ones) — and asked for a benchmark helper plus mock scenario hooks. Audited against the codebase first (see [ADR 0030](./docs/adr/0030-draft-benchmark-request-aware-mock-routes.md)): the "manual nanosecond ratios" claim was partially stale — `formatTelemetry` derives tokens/second since v1.12.0 and `lab/15` implements the full protocol, but neither reaches npm consumers (`files` publishes only `dist/`), so the packaged-harness gap was real. The mock scenario claims were stale: parallel tool calls in one frame, malformed-argument tool calls (string `arguments`), in-band mid-stream error injection, and 503 bodies all shipped in v1.14.0. The genuine engine gap the audit surfaced: routes couldn't vary responses **per request**. All additive, no breaking changes. 960 pre-existing tests stay green, 17 new tests added.
+
+### Speculative draft benchmark (`compareDraftPerformance`)
+
+- **`compareDraftPerformance(client, options)`** — the deterministic A/B for "does `draft_num_predict` pay off for _this_ prompt": identical prompt on both sides, `temperature: 0` forced + a fixed `seed` (default 42), one warm-up generation per setting (pays the model load), `samples` measured runs per side (default 5) with the **median** compared — single samples mislead. Throughput flows through `formatTelemetry` (the official Usage-doc math; never hand-rolled nanosecond division).
+- Result: `baselineTokensPerSec` / `speculativeTokensPerSec` (1dp), `speedupMultiplier` (2dp, computed from the two reported medians so the numbers are self-consistent), `effectiveGain` (`> gainThreshold`, default 1.05), plus full per-side `DraftRunStats` (every sample, medians).
+- **Structural dependency** — accepts any `DraftBenchmarkClient` (`{ generate }` with the non-streaming signature); `OllamaClient` satisfies it without being named, and tests can pass a plain object. A `client.benchmark` namespace was deliberately not added, and the digest's `SpeculativeBenchmarkRunner` class was folded into the function — rationale in ADR 0030.
+- Guarded: `RangeError` on invalid numeric options; non-positive/unusable samples are skipped, and a side with **no** valid samples throws `OllamaClientError` (`draft_benchmark_no_valid_samples`) instead of fabricating a zero/Infinity speedup; the abort signal is checked between runs and forwarded to every call.
+- The no-draft-network reading is documented on the utility and in the README: a ≈1.0 speedup means the model has no draft network (and no embedded MTP tensors) — the server silently ignores the option.
+
+### Request-aware mock routes (`responder`)
+
+- `MockEndpointOptions` now accepts `responder(record) => MockRouteResponse | undefined` — per-request responses computed from the recorded method/path/body. Returning `undefined` falls back to the route's static fields, so fail-twice-then-succeed retry drills, body-dependent payloads (e.g. draft-setting-aware frames), and call-count scenarios are closure-local one-liners on the same engine — no subclassing, no one-shot global scenario queue. The digest's `ScenarioMockOllamaServer` sketch was declined for exactly that reason (it forks the request path ADR 0029 protects; see ADR 0030).
+- The emission pipeline is shared: `chunkDelayMs`, `injectErrorFrame`, `dropConnectionMidStream`, and `rawBody` all keep their semantics inside responder routes; request recording and `onRequest` still fire.
+- Type-level: the response fields moved to a new `MockRouteResponse` interface that `MockEndpointOptions` extends — pure refactor, existing routes unaffected. `MockRouteResponse` is exported from the `/testing` subpath.
+- The SDK's own benchmark tests run daemonless through responder routes (draft-aware `/api/generate` frames) — the exact consumer CI pattern the digest's downstream stories describe, exercising the public engine rather than test-only fixtures.
+
+## [1.14.0] — Speculative-decoding model assembly & the public testing subpath: `draftGguf`, `draft_num_predict` docs, `@nemesis-oss/ollama-sdk/testing`
+
+The October 12 digest ("Speculative Decoding & Draft Model Creation") reviewed the official model-creation surfaces — `draft_files` + `draft_quantize` on `POST /api/create`, the `draft_num_predict` Modelfile parameter and runtime option — and asked for an exportable zero-network mock server. Audited against the codebase first (see [ADR 0029](./docs/adr/0029-public-testing-subpath-speculative-draft-conveniences.md)): the digest's headline typing claims were stale — `draft_files`/`draft_quantize` already shipped on both the generated `CreateRequest` and the hand-written `CreateRequestOptions` (official OpenAPI, 2026-10-07 pin), and `draft_num_predict` was already typed + documented on the runtime `ModelOptions`; the "DRAFT `<model>`" Modelfile instruction the digest describes is not in the current official docs (only the parameter is). The real gaps — a path-based draft upload convenience, wire tests, discoverable documentation, and the not-exported mock server — are closed here. All additive, no breaking changes. 932 pre-existing tests stay green, 28 new tests added.
+
+### Speculative model assembly (`createModelFromGguf` → `draftGguf`)
+
+- **`CreateModelFromGgufOptions.draftGguf`**: pass the draft network's GGUF path(s) alongside the primary shards — each is uploaded as a blob and assembled into `draft_files` under its basename, the same protocol as primary shards. Split draft GGUFs, explicit `draft_files` digest maps (merged; path uploads win on basename collision), and `draft_quantize` passthrough all work; no draft → no `draft_files` key on the wire at all.
+- **Discoverable speculative-decoding docs**: `CreateRequestOptions.parameters` now documents `draft_num_predict` (separate draft models default to 4; `0` disables drafting; embedded MTP tensors require setting it explicitly — official Modelfile reference), and `draft_files`/`draft_quantize` carry the speculative-decoding context with the verification-semantics note that makes aggressive draft compression safe.
+- Wire tests pin the whole contract: `createModel` round-trips `draft_files` + `draft_quantize` + `parameters.draft_num_predict`; `createModelFromGguf` uploads draft blobs separately (HEAD + POST per shard) and never leaks `draftGguf` or empty maps into the request body.
+
+### Public testing subpath (`@nemesis-oss/ollama-sdk/testing`)
+
+- The SDK's internal mock engine is **promoted, not forked** (ADR 0029, same discipline as ADRs 0027/0028): `test/mocks/ollama-mock-server.ts` is now a re-export shim, so the SDK's own suite exercises exactly the code consumers import.
+- **`createMockOllamaServer(options?)`** — the recommended factory. Built-in **default routes** answer the official liveness probe (`HEAD /` → 200, `GET /` → the plain-text body "Ollama is running" byte-for-byte) and `GET /api/version` (configurable version, default `0.14.0`), so `client.ping()` works against a fresh mock with zero setup. `defaultRoutes: false` restores bare 404s; `register()` always wins over the defaults.
+- **Request recording** — `server.requests` captures every request (method, path, parsed body, timestamp), including default-route hits and 404s; `clearRequests()` for `beforeEach` hooks. Wire-level assertions without per-route callbacks.
+- **In-band mid-stream error injection** — `injectErrorFrame: { message, after? }` emits an `{"error": "..."}` NDJSON frame while HTTP stays 200, exactly how a real daemon reports GPU OOM/driver crashes mid-generation. The default injection point is `chunks.length - 1`: every content chunk, then the error frame, with the final `done` frame never emitted — an error frame after `done` would be unread (the stream has already completed cleanly). `after: 0` errors before any content; explicit larger values move the position. The SDK's stream readers raise `OllamaStreamError` with the partial content.
+- **`rawBody`** route option for exact non-NDJSON bodies (the liveness body uses it — no trailing newline).
+- **Chunk builders** producing correctly-shaped `/api/chat` frames: `mockChatChunk` (one frame, counter defaults on the done frame for telemetry paths), `mockTextChunks` (chunked text + done frame), and `mockToolCallChunks` — parallel tool calls in one frame, the documented native shape, followed by the done frame.
+- **Packaging**: dedicated `./testing` entry in the exports map (paired `.d.ts`/`.d.cts`), a new tsup entry, and the consumer-type verification now compiles a real CJS (node16) **and** ESM (nodenext) consumer importing the subpath. Node.js-only by design (`node:http`) — never imported from the Edge-safe main entry, so `verify:edge-runtime` stays clean.
+
+### Lab
+
+- `lab/15-speculative-draft/measure-draft-speedup.ts`: live harness comparing `draft_num_predict` 0/4/8/16 — warm-up per setting, 5 samples each, median `tokensPerSecond` via `formatTelemetry`, and a speedup table against drafting-disabled. Makes "the server has no draft network, the option is being ignored" obvious at a glance.
+
 ## [1.13.0] — docs.ollama.com re-audit: parity verifier reads checkbox state, strict compat types, web-search agent tools
 
 A full re-read of docs.ollama.com (API reference, capability guides, integrations, OpenAPI spec) against the SDK. The native API was already at parity except for two upstream changes; the real defect was in the compatibility-parity tooling, which had been reporting fields Ollama documents as unsupported as "supported".

package/README.md

@@ -33,7 +33,9 @@ Nexum’s model harness (2026-10-09) exercised this SDK via `gemma4:cloud` and `
 - 🌉 **OpenAI & Anthropic Compatibility Bridges**: Built-in clients for `/v1/chat/completions`, `/v1/responses`, `/v1/models`, and `/v1/messages`, including `reasoning_effort`/`reasoning.effort` for thinking models — plus an ergonomic dual-mode `client.responses.create()` bridge that prefers native `/v1/responses` and transparently re-issues via `/api/chat` on older servers (pre-v0.13.3).
 - 💬 **KV-Cache-Aware Conversation Sessions**: `client.session(model, systemPrompt)` keeps an append-only, prefix-stable history that maximizes Ollama's KV-cache reuse across turns, surfaces per-turn + cumulative cache hit rates (`prompt_eval_cached_count`) so cache degradation is visible instead of silent, and offers a one-call `compact()` sliding-window escape hatch when a session outgrows its window.
 - 🛑 **Context-Window Safety**: Heuristic client-side token estimation warns (or throws, `onContextOverflow: 'throw'`) before a request is sent when the prompt approaches the effective `num_ctx` window — Ollama's default behavior is to _silently truncate_ oversized prompts — and `defaultContextLength` makes the window explicit on every request.
-- 📦 **Blob & Custom Model Publishing**: Content-addressed blob management per Ollama's documented protocol — `computeBlobDigest()` (SHA-256 → `sha256:<hex>`), `createBlobFromData()`/`createBlobFromFile()` (HEAD-check + upload, skipping existing blobs), and one-shot `createModelFromGguf()` (upload GGUF shard blobs, then `/api/create` with `files: {name: digest}`).
+- 📦 **Blob & Custom Model Publishing**: Content-addressed blob management per Ollama's documented protocol — `computeBlobDigest()` (SHA-256 → `sha256:<hex>`), `createBlobFromData()`/`createBlobFromFile()` (HEAD-check + upload, skipping existing blobs), and one-shot `createModelFromGguf()` (upload GGUF shard blobs, then `/api/create` with `files: {name: digest}`) — with **speculative-decoding assembly**: `draftGguf` uploads the draft network's shards into `draft_files`, `draft_quantize` compresses them independently, and `parameters.draft_num_predict` tunes the per-step draft length.
+- 🏁 **Speculative Draft Benchmark (`compareDraftPerformance`)**: One call answers "does drafting pay off for _this_ prompt?" — identical prompt on both sides, deterministic sampling (`temperature: 0` + fixed seed), warm-up discard, several samples per side with **medians** compared, throughput via `formatTelemetry` — reported as baseline vs. speculative tok/s, `speedupMultiplier`, and `effectiveGain` past a configurable threshold. A ≈1.0 speedup is itself diagnostic: the model has no draft network, so the server silently ignores the option.
+- 🧪 **In-Memory Mock Server (`@nemesis-oss/ollama-sdk/testing`)**: The SDK's own deterministic test double, promoted public — `createMockOllamaServer()` with scriptable NDJSON routes, built-in liveness/version defaults (so `ping()` works out of the box), request recording for wire-level assertions, **request-aware routes** (`responder` — per-request responses for fail-twice-then-succeed retry drills and body-dependent payloads), mid-stream **in-band error injection** (`{"error": "..."}` frames at HTTP 200, exactly how a real daemon reports GPU OOM), connection drops, and correctly-shaped chunk builders for streaming text and **parallel tool calls** — zero network, zero daemon, CI-deterministic.
 - 🧵 **Batched Embeddings with Backpressure**: `client.embedBatch()` splits corpora of any size into bounded `/api/embed` batches (default 32 inputs × 3 in flight) — order-preserving, fail-fast, progress-reporting — so RAG/vector ingestion can't OOM a local daemon or saturate `OLLAMA_MAX_QUEUE` the way `Promise.all` floods do, with per-string context-window pre-flight against silent truncation.
 - 🧹 **Client Teardown (`destroy()`)**: One call aborts every in-flight request, active stream, and queued capacity waiter with `OllamaAbortError` — the clean-exit path for `worker_threads`, CLI runners, and short-lived scripts where dangling fetch bodies keep the event loop alive.
 - 🧠 **Dynamic Context-Window Discovery**: `client.models.getContextLength({ model })` resolves the real window from the server — the allocated window of the running instance (`/api/ps`), the Modelfile `num_ctx` default, and the native GGUF maximum (`model_info`), in precedence order — so applications stop hardcoding 2048/4096 guesses.
@@ -497,6 +499,76 @@ await client.models.createModelFromGguf('my-model', [
 ]);
 ```
 
+### Speculative Decoding (Draft Models)
+
+Ollama packages speculative decoding as a small **draft network** whose token
+guesses are batch-verified against the primary weights in a single forward
+pass — the draft runs ahead, the primary model confirms. On the wire this is
+`draft_files` (draft GGUF digests) + `draft_quantize` on `/api/create`, the
+`draft_num_predict` Modelfile parameter, and the `draft_num_predict` runtime
+option at generation time. The SDK covers the full flow:
+
+```typescript
+// 1. Assemble: primary + draft shards, draft weights quantized independently
+//    (draft tokens are verified, never returned — aggressive compression is
+//    usually safe). Separate draft models default draft_num_predict to 4;
+//    0 disables drafting; models with embedded MTP tensors require setting
+//    it explicitly.
+await client.models.createModelFromGguf('my-model', '/models/model.gguf', {
+  draftGguf: '/models/draft.gguf',
+  draftQuantize: 'q8_0',
+  parameters: { draft_num_predict: 8 },
+});
+
+// 2. Generate with drafting (also available ad hoc via options at runtime)
+const response = await client.chat({
+  model: 'my-model',
+  messages: [{ role: 'user', content: 'Describe a lighthouse in a storm.' }],
+  options: { draft_num_predict: 8 },
+});
+
+// 3. Measure the speedup with the standardized telemetry
+const t = formatTelemetry(response);
+console.log(`${t.tokensPerSecond} tok/s (load ${t.modelLoadMs}ms)`);
+```
+
+Already have the draft blobs uploaded? Pass the digest map directly —
+`createModel({ draft_files: { 'draft.gguf': digest } })` — or merge both
+(path uploads win on basename collision).
+
+**Does drafting actually pay off?** Upstream emits no draft-token
+acceptance counter — speculative efficiency is only observable as effective
+token throughput, and the signal is noisy: rejected draft predictions _add_
+compute (high-temperature, creative prompts) while accepted drafts slash
+`eval_duration` on predictable workloads (structured code, JSON extraction).
+`compareDraftPerformance()` runs the deterministic A/B for a specific
+prompt — identical prompt both sides, `temperature: 0` + a fixed seed, one
+warm-up generation per setting (pays the model load), several samples per
+side with the **median** compared, throughput derived by `formatTelemetry`:
+
+```typescript
+import { OllamaClient, compareDraftPerformance } from '@nemesis-oss/ollama-sdk';
+
+const comparison = await compareDraftPerformance(new OllamaClient(), {
+  model: 'my-model',
+  prompt: 'Extract the sender, subject, and date from this email as JSON: ...',
+  draftNumPredict: 4, // default; the baseline side always runs with 0
+  samples: 5, // median per side — single runs mislead
+});
+
+console.log(
+  `${comparison.baselineTokensPerSec} → ${comparison.speculativeTokensPerSec} tok/s ` +
+    `(${comparison.speedupMultiplier}x, gain: ${comparison.effectiveGain})`,
+);
+```
+
+`effectiveGain` flips to `true` when the reported multiplier exceeds
+`gainThreshold` (default `1.05` — five percent, past run-to-run noise). A
+speedup of ≈1.0 is itself the diagnosis: the model has no draft network
+(and no embedded MTP tensors), so the server silently ignores
+`draft_num_predict` — the same reading `lab/15-speculative-draft/measure-draft-speedup.ts`
+makes obvious across 0/4/8/16.
+
 ### Token Log Probabilities (`logprobs`)
 
 Set `logprobs: true` (optionally with `top_logprobs`) on `chat()`/`generate()` to get per-token log probabilities back — useful for confidence scoring, speculative decoding, or agent routing decisions.
@@ -955,9 +1027,19 @@ rejects `x-api-key`-only auth; the SDK always sends the bearer header plus
 `anthropic-version: 2023-06-01`):
 
 ```typescript
-const cloud = new OllamaClient({ baseUrl: 'https://ollama.com', apiKey: process.env.OLLAMA_API_KEY });
-await cloud.openai.chatCompletions({ model: 'gemma4:31b', messages: [{ role: 'user', content: 'Hi' }] });
-await cloud.anthropic.messages({ model: 'gemma4:31b', max_tokens: 1024, messages: [{ role: 'user', content: 'Hi' }] });
+const cloud = new OllamaClient({
+  baseUrl: 'https://ollama.com',
+  apiKey: process.env.OLLAMA_API_KEY,
+});
+await cloud.openai.chatCompletions({
+  model: 'gemma4:31b',
+  messages: [{ role: 'user', content: 'Hi' }],
+});
+await cloud.anthropic.messages({
+  model: 'gemma4:31b',
+  max_tokens: 1024,
+  messages: [{ role: 'user', content: 'Hi' }],
+});
 ```
 
 The cloud API does not support stateful Responses, built-in web search through
@@ -1454,6 +1536,90 @@ windows: [...] })` directly with your own `windowMs`/`maxTokens`/`maxRequests` p
 
 ---
 
+### Testing with the In-Memory Mock Server (`/testing`)
+
+The SDK ships its own deterministic test double as a dedicated subpath —
+`@nemesis-oss/ollama-sdk/testing` — the same engine the SDK's own test suite
+runs against (ADR 0029: promote, don't fork). No daemon, no network, no GPU:
+scriptable NDJSON routes on a real `node:http` server, built-in defaults for
+the official liveness probe and `/api/version`, wire-level request recording,
+and failure injection that covers what static cassettes can't.
+
+```typescript
+import {
+  createMockOllamaServer,
+  mockTextChunks,
+  mockToolCallChunks,
+} from '@nemesis-oss/ollama-sdk/testing';
+import { OllamaClient } from '@nemesis-oss/ollama-sdk';
+
+const server = createMockOllamaServer(); // ephemeral port, defaults on
+await server.start();
+const client = new OllamaClient({ baseUrl: server.baseUrl });
+
+// Defaults answer out of the box: HEAD / (ping) and GET /api/version.
+// Register everything else — register() always wins over the defaults.
+server.register('/api/chat', {
+  chunks: mockToolCallChunks([
+    // Parallel tool calls: one frame, complete calls — the native shape.
+    { name: 'get_weather', arguments: { city: 'Tokyo' } },
+    { name: 'get_time', arguments: { tz: 'Asia/Tokyo' } },
+  ]),
+  chunkDelayMs: 10, // inter-chunk backpressure testing
+});
+
+// Mid-stream failure, GPU-OOM style: HTTP stays 200, the done frame is
+// replaced by an {"error": "..."} frame — the SDK raises OllamaStreamError.
+server.register('/api/generate', {
+  chunks: mockTextChunks('partial answer', { chunkSize: 8 }),
+  injectErrorFrame: { message: 'CUDA out of memory' }, // after: N to move it
+});
+
+// ... run your code against the client ...
+
+// Wire-level assertions: every request was recorded.
+expect(server.requests[0]?.body).toMatchObject({ model: 'llama3.2' });
+server.clearRequests(); // and server.clear() between tests
+
+await server.stop(); // idempotent — safe in afterEach
+```
+
+**Request-aware routes** — `responder` computes the response _per request_
+from the recorded method/path/body, covering the scenario drills a static
+route can't: fail-twice-then-succeed retries, body-dependent payloads,
+call-count behavior:
+
+```typescript
+// Fail twice, then serve — retry/backoff drills:
+let hits = 0;
+server.register('/api/chat', {
+  chunks: mockTextChunks('recovered'),
+  responder: () =>
+    hits++ < 2 ? { status: 503, chunks: ['{"error":"server queue is full"}'] } : undefined,
+});
+
+// Body-dependent — answer differently by what the client sent:
+server.register('/api/generate', {
+  responder: (record) => ({
+    chunks: [frameFor(readDraftSetting(record.body))],
+  }),
+});
+```
+
+Returning `undefined` falls back to the route's static fields; a returned
+response feeds the full emission pipeline, so `chunkDelayMs`,
+`injectErrorFrame`, `dropConnectionMidStream`, and `rawBody` all keep their
+semantics inside responder routes.
+
+Per-route controls: `status` (503/502/429 bodies), `chunkDelayMs`,
+`dropConnectionMidStream` (connection reset mid-response), `onRequest`
+(per-route body capture), and `injectErrorFrame.after` (error position).
+Chunk builders: `mockChatChunk` (one frame with counter defaults),
+`mockTextChunks` (chunked text + done frame), `mockToolCallChunks` (parallel
+tool calls + done frame — malformed-argument cases via string
+`arguments`). Node.js only — the subpath builds on `node:http`
+and is never imported from the Edge-safe main entry.
+
 ### Edge Runtime Compatibility
 
 The core client (`OllamaClient`, `Agent`, `ToolRegistry`, and everything exported from

package/dist/{anthropic-B9AFyzXI.d.ts → anthropic-DrDUfbfQ.d.ts}

@@ -1,4 +1,4 @@
-import { O as OllamaRuntime, H as HttpClient, A as AbortableAsyncIterable, af as SseEvent, y as OllamaClientError } from './runtime-CIQYCRL3.js';
+import { O as OllamaRuntime, H as HttpClient, A as AbortableAsyncIterable, af as SseEvent, y as OllamaClientError } from './runtime-BIjr5ME4.js';
 
 /**
  * AUTO-GENERATED by `npm run contract:generate`. DO NOT EDIT.

package/dist/{anthropic-CYy_ECam.d.cts → anthropic-UeaVjLeP.d.cts}

@@ -1,4 +1,4 @@
-import { O as OllamaRuntime, H as HttpClient, A as AbortableAsyncIterable, af as SseEvent, y as OllamaClientError } from './runtime-DmI1cOEq.cjs';
+import { O as OllamaRuntime, H as HttpClient, A as AbortableAsyncIterable, af as SseEvent, y as OllamaClientError } from './runtime-C4RRHqOs.cjs';
 
 /**
  * AUTO-GENERATED by `npm run contract:generate`. DO NOT EDIT.

package/dist/{compose-6UOPEknN.d.ts → compose-CeFbHJeO.d.ts}

@@ -1,4 +1,4 @@
-import { l as ToolDefinition, i as ToolCall, n as Message } from './types-B57ZZfp3.js';
+import { l as ToolDefinition, i as ToolCall, n as Message } from './types-DbPnfVh-.js';
 import { z } from 'zod';
 
 /**

package/dist/{compose-DhvoUrWa.d.cts → compose-DFkmtUOP.d.cts}

@@ -1,4 +1,4 @@
-import { l as ToolDefinition, i as ToolCall, n as Message } from './types-B57ZZfp3.cjs';
+import { l as ToolDefinition, i as ToolCall, n as Message } from './types-DbPnfVh-.cjs';
 import { z } from 'zod';
 
 /**

package/dist/generated-api.d.cts

@@ -1,8 +1,8 @@
-import { O as OpenAIChatCompletionRequest, a as OpenAIChatCompletionResponse, b as OpenAIChatCompletionChunk, c as OpenAICompletionRequest, d as OpenAICompletionResponse, e as OpenAICompletionChunk, f as OpenAIEmbeddingRequest, g as OpenAIEmbeddingResponse, h as OpenAIListModelsResponse, i as OpenAIModelItem, j as OpenAIResponsesRequest, k as OpenAIResponsesResponse, l as OpenAIResponsesStreamEvent, A as AnthropicMessagesRequest, m as AnthropicMessagesResponse, n as AnthropicMessageStreamEvent } from './anthropic-CYy_ECam.cjs';
-export { N as NativeApi } from './anthropic-CYy_ECam.cjs';
-import { O as OllamaRuntime } from './runtime-DmI1cOEq.cjs';
-export { a as OperationDefinition } from './runtime-DmI1cOEq.cjs';
-import './types-B57ZZfp3.cjs';
+import { O as OpenAIChatCompletionRequest, a as OpenAIChatCompletionResponse, b as OpenAIChatCompletionChunk, c as OpenAICompletionRequest, d as OpenAICompletionResponse, e as OpenAICompletionChunk, f as OpenAIEmbeddingRequest, g as OpenAIEmbeddingResponse, h as OpenAIListModelsResponse, i as OpenAIModelItem, j as OpenAIResponsesRequest, k as OpenAIResponsesResponse, l as OpenAIResponsesStreamEvent, A as AnthropicMessagesRequest, m as AnthropicMessagesResponse, n as AnthropicMessageStreamEvent } from './anthropic-UeaVjLeP.cjs';
+export { N as NativeApi } from './anthropic-UeaVjLeP.cjs';
+import { O as OllamaRuntime } from './runtime-C4RRHqOs.cjs';
+export { a as OperationDefinition } from './runtime-C4RRHqOs.cjs';
+import './types-DbPnfVh-.cjs';
 import 'zod';
 
 /**

package/dist/generated-api.d.ts

@@ -1,8 +1,8 @@
-import { O as OpenAIChatCompletionRequest, a as OpenAIChatCompletionResponse, b as OpenAIChatCompletionChunk, c as OpenAICompletionRequest, d as OpenAICompletionResponse, e as OpenAICompletionChunk, f as OpenAIEmbeddingRequest, g as OpenAIEmbeddingResponse, h as OpenAIListModelsResponse, i as OpenAIModelItem, j as OpenAIResponsesRequest, k as OpenAIResponsesResponse, l as OpenAIResponsesStreamEvent, A as AnthropicMessagesRequest, m as AnthropicMessagesResponse, n as AnthropicMessageStreamEvent } from './anthropic-B9AFyzXI.js';
-export { N as NativeApi } from './anthropic-B9AFyzXI.js';
-import { O as OllamaRuntime } from './runtime-CIQYCRL3.js';
-export { a as OperationDefinition } from './runtime-CIQYCRL3.js';
-import './types-B57ZZfp3.js';
+import { O as OpenAIChatCompletionRequest, a as OpenAIChatCompletionResponse, b as OpenAIChatCompletionChunk, c as OpenAICompletionRequest, d as OpenAICompletionResponse, e as OpenAICompletionChunk, f as OpenAIEmbeddingRequest, g as OpenAIEmbeddingResponse, h as OpenAIListModelsResponse, i as OpenAIModelItem, j as OpenAIResponsesRequest, k as OpenAIResponsesResponse, l as OpenAIResponsesStreamEvent, A as AnthropicMessagesRequest, m as AnthropicMessagesResponse, n as AnthropicMessageStreamEvent } from './anthropic-DrDUfbfQ.js';
+export { N as NativeApi } from './anthropic-DrDUfbfQ.js';
+import { O as OllamaRuntime } from './runtime-BIjr5ME4.js';
+export { a as OperationDefinition } from './runtime-BIjr5ME4.js';
+import './types-DbPnfVh-.js';
 import 'zod';
 
 /**

package/dist/generated-runtime.d.cts

@@ -1,6 +1,6 @@
-export { I as InvokeRequest, O as OllamaRuntime, b as OllamaRuntimeOptions, a as OperationDefinition, S as SchemaRef } from './runtime-DmI1cOEq.cjs';
+export { I as InvokeRequest, O as OllamaRuntime, b as OllamaRuntimeOptions, a as OperationDefinition, S as SchemaRef } from './runtime-C4RRHqOs.cjs';
 import { z } from 'zod';
-import './types-B57ZZfp3.cjs';
+import './types-DbPnfVh-.cjs';
 
 /**
  * AUTO-GENERATED by `npm run contract:generate`. DO NOT EDIT.

package/dist/generated-runtime.d.ts

@@ -1,6 +1,6 @@
-export { I as InvokeRequest, O as OllamaRuntime, b as OllamaRuntimeOptions, a as OperationDefinition, S as SchemaRef } from './runtime-CIQYCRL3.js';
+export { I as InvokeRequest, O as OllamaRuntime, b as OllamaRuntimeOptions, a as OperationDefinition, S as SchemaRef } from './runtime-BIjr5ME4.js';
 import { z } from 'zod';
-import './types-B57ZZfp3.js';
+import './types-DbPnfVh-.js';
 
 /**
  * AUTO-GENERATED by `npm run contract:generate`. DO NOT EDIT.

package/dist/index.cjs

@@ -2817,12 +2817,29 @@ var ModelsClient = class {
    * await client.models.createModelFromGguf('my-model', '/path/to/model.gguf', {
    *   template: '{{ .Prompt }}',
    * });
+   *
+   * // Speculative decoding: primary + draft shards + tuning:
+   * await client.models.createModelFromGguf('my-model', '/path/to/model.gguf', {
+   *   draftGguf: '/path/to/draft.gguf',
+   *   draftQuantize: 'q8_0',
+   *   parameters: { draft_num_predict: 8 },
+   * });
    * ```
    *
+   * Speculative-decoding models assemble the same way: pass the draft
+   * network's GGUF(s) as `draftGguf` (uploaded into `draft_files`), optionally
+   * set `draft_quantize` to compress the draft weights independently of the
+   * primary model, and set `parameters.draft_num_predict` to tune how many
+   * tokens the draft network predicts per step (Ollama defaults separate
+   * draft models to 4; `0` disables drafting; models with embedded MTP
+   * tensors require setting it explicitly).
+   *
    * @param model Name for the new model (e.g. `'my-model:latest'`).
    * @param gguf Path(s) to GGUF file(s) on the local filesystem.
    * @param opts Everything else `/api/create` accepts (`template`, `system`,
-   *   `parameters`, `license`, `quantize`, …) except `files` and `from`.
+   *   `parameters`, `license`, `quantize`, `draftGguf`, …) except `files`,
+   *   `draft_files` (use `draftGguf` for path-based drafts; an explicit digest
+   *   map is still accepted and merged), `from`, and `stream`.
    */
   async createModelFromGguf(model, gguf, opts) {
     const paths = typeof gguf === "string" ? [gguf] : gguf;
@@ -2836,14 +2853,29 @@ var ModelsClient = class {
       const uploaded = await this.createBlobFromFile(path, { signal: opts?.signal });
       files[uploaded.fileName] = uploaded.digest;
     }
+    const draftOpt = opts?.draftGguf;
+    const draftPaths = typeof draftOpt === "string" ? [draftOpt] : draftOpt ?? [];
     const {
       model: _model,
       files: _files,
       from: _from,
       stream: _stream,
+      draftGguf: _draftGguf,
+      draft_files: _draftFiles,
       ...rest
     } = opts ?? {};
-    return this.create({ ...rest, model, files, stream: false });
+    const draftFiles = { ..._draftFiles ?? {} };
+    for (const path of draftPaths) {
+      const uploaded = await this.createBlobFromFile(path, { signal: opts?.signal });
+      draftFiles[uploaded.fileName] = uploaded.digest;
+    }
+    return this.create({
+      ...rest,
+      model,
+      files,
+      ...Object.keys(draftFiles).length > 0 ? { draft_files: draftFiles } : {},
+      stream: false
+    });
   }
 };
 
@@ -7718,6 +7750,111 @@ var Agent = class {
   }
 };
 
+// src/draft-benchmark.ts
+var DEFAULT_DRAFT_TOKENS = 4;
+var DEFAULT_SAMPLES = 5;
+var DEFAULT_NUM_PREDICT = 256;
+var DEFAULT_SEED = 42;
+var DEFAULT_GAIN_THRESHOLD = 1.05;
+function median(values) {
+  const sorted = [...values].sort((a, b) => a - b);
+  const mid = Math.floor(sorted.length / 2);
+  return sorted.length % 2 === 1 ? sorted[mid] ?? 0 : ((sorted[mid - 1] ?? 0) + (sorted[mid] ?? 0)) / 2;
+}
+function roundTo(value, decimals) {
+  const factor = 10 ** decimals;
+  return Math.round(value * factor) / factor;
+}
+function requirePositiveInteger(name, value) {
+  if (!Number.isInteger(value) || value < 1) {
+    throw new RangeError(
+      `compareDraftPerformance: ${name} must be an integer >= 1 (got ${value}).`
+    );
+  }
+}
+async function runSide(client, options, draftNumPredict, samples, numPredict, seed) {
+  const samplingOptions = {
+    num_predict: numPredict,
+    temperature: 0,
+    seed,
+    draft_num_predict: draftNumPredict
+  };
+  if (options.warmup ?? true) {
+    await client.generate({
+      model: options.model,
+      prompt: options.prompt,
+      options: { ...samplingOptions, num_predict: 1 },
+      stream: false,
+      signal: options.signal
+    });
+  }
+  const tokensPerSecond = [];
+  const generationMs = [];
+  for (let i = 0; i < samples; i++) {
+    options.signal?.throwIfAborted();
+    const response = await client.generate({
+      model: options.model,
+      prompt: options.prompt,
+      options: samplingOptions,
+      stream: false,
+      signal: options.signal
+    });
+    const telemetry = formatTelemetry(response);
+    if (Number.isFinite(telemetry.tokensPerSecond) && telemetry.tokensPerSecond > 0) {
+      tokensPerSecond.push(telemetry.tokensPerSecond);
+      generationMs.push(telemetry.generationMs);
+    }
+  }
+  if (tokensPerSecond.length === 0) {
+    throw new OllamaClientError(
+      `compareDraftPerformance: every draft_num_predict=${draftNumPredict} sample returned a non-positive tokens/second. The model name is likely wrong, or the server returned empty generations.`,
+      { code: "draft_benchmark_no_valid_samples" }
+    );
+  }
+  return {
+    draftNumPredict,
+    medianTokensPerSecond: median(tokensPerSecond),
+    sampleTokensPerSecond: tokensPerSecond,
+    medianEvalDurationMs: median(generationMs)
+  };
+}
+async function compareDraftPerformance(client, options) {
+  const draftNumPredict = options.draftNumPredict ?? DEFAULT_DRAFT_TOKENS;
+  requirePositiveInteger("draftNumPredict", draftNumPredict);
+  const samples = options.samples ?? DEFAULT_SAMPLES;
+  requirePositiveInteger("samples", samples);
+  const numPredict = options.numPredict ?? DEFAULT_NUM_PREDICT;
+  requirePositiveInteger("numPredict", numPredict);
+  const seed = options.seed ?? DEFAULT_SEED;
+  if (!Number.isInteger(seed)) {
+    throw new RangeError(`compareDraftPerformance: seed must be an integer (got ${seed}).`);
+  }
+  const gainThreshold = options.gainThreshold ?? DEFAULT_GAIN_THRESHOLD;
+  if (!Number.isFinite(gainThreshold) || gainThreshold <= 0) {
+    throw new RangeError(
+      `compareDraftPerformance: gainThreshold must be a positive finite number (got ${gainThreshold}).`
+    );
+  }
+  options.signal?.throwIfAborted();
+  const baseline = await runSide(client, options, 0, samples, numPredict, seed);
+  const speculative = await runSide(client, options, draftNumPredict, samples, numPredict, seed);
+  const baselineTokensPerSec = roundTo(baseline.medianTokensPerSecond, 1);
+  const speculativeTokensPerSec = roundTo(speculative.medianTokensPerSecond, 1);
+  const speedupMultiplier = baselineTokensPerSec > 0 ? roundTo(speculativeTokensPerSec / baselineTokensPerSec, 2) : 0;
+  return {
+    model: options.model,
+    prompt: options.prompt,
+    draftNumPredict,
+    baselineTokensPerSec,
+    speculativeTokensPerSec,
+    speedupMultiplier,
+    effectiveGain: speedupMultiplier > gainThreshold,
+    gainThreshold,
+    baseline,
+    speculative
+  };
+}
+
 // src/affinity.ts
 var DEFAULT_CONCURRENT_MODELS = 1;
 var DEFAULT_PER_MODEL_CONCURRENCY = 1;
@@ -7979,6 +8116,7 @@ exports.checkChatContext = checkChatContext;
 exports.checkEndpointHealth = checkEndpointHealth;
 exports.checkGenerateContext = checkGenerateContext;
 exports.compactConversationHistory = compactConversationHistory;
+exports.compareDraftPerformance = compareDraftPerformance;
 exports.composeMiddleware = composeMiddleware;
 exports.contextWarningMessage = contextWarningMessage;
 exports.createConsoleLogger = createConsoleLogger;