@nemesis-oss/ollama-sdk 1.13.0 → 1.14.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,29 @@
1
1
  # Changelog
2
2
 
3
+ ## [1.14.0] — Speculative-decoding model assembly & the public testing subpath: `draftGguf`, `draft_num_predict` docs, `@nemesis-oss/ollama-sdk/testing`
4
+
5
+ The October 12 digest ("Speculative Decoding & Draft Model Creation") reviewed the official model-creation surfaces — `draft_files` + `draft_quantize` on `POST /api/create`, the `draft_num_predict` Modelfile parameter and runtime option — and asked for an exportable zero-network mock server. Audited against the codebase first (see [ADR 0029](./docs/adr/0029-public-testing-subpath-speculative-draft-conveniences.md)): the digest's headline typing claims were stale — `draft_files`/`draft_quantize` already shipped on both the generated `CreateRequest` and the hand-written `CreateRequestOptions` (official OpenAPI, 2026-10-07 pin), and `draft_num_predict` was already typed + documented on the runtime `ModelOptions`; the "DRAFT `<model>`" Modelfile instruction the digest describes is not in the current official docs (only the parameter is). The real gaps — a path-based draft upload convenience, wire tests, discoverable documentation, and the not-exported mock server — are closed here. All additive, no breaking changes. 932 pre-existing tests stay green, 28 new tests added.
6
+
7
+ ### Speculative model assembly (`createModelFromGguf` → `draftGguf`)
8
+
9
+ - **`CreateModelFromGgufOptions.draftGguf`**: pass the draft network's GGUF path(s) alongside the primary shards — each is uploaded as a blob and assembled into `draft_files` under its basename, the same protocol as primary shards. Split draft GGUFs, explicit `draft_files` digest maps (merged; path uploads win on basename collision), and `draft_quantize` passthrough all work; no draft → no `draft_files` key on the wire at all.
10
+ - **Discoverable speculative-decoding docs**: `CreateRequestOptions.parameters` now documents `draft_num_predict` (separate draft models default to 4; `0` disables drafting; embedded MTP tensors require setting it explicitly — official Modelfile reference), and `draft_files`/`draft_quantize` carry the speculative-decoding context with the verification-semantics note that makes aggressive draft compression safe.
11
+ - Wire tests pin the whole contract: `createModel` round-trips `draft_files` + `draft_quantize` + `parameters.draft_num_predict`; `createModelFromGguf` uploads draft blobs separately (HEAD + POST per shard) and never leaks `draftGguf` or empty maps into the request body.
12
+
13
+ ### Public testing subpath (`@nemesis-oss/ollama-sdk/testing`)
14
+
15
+ - The SDK's internal mock engine is **promoted, not forked** (ADR 0029, same discipline as ADRs 0027/0028): `test/mocks/ollama-mock-server.ts` is now a re-export shim, so the SDK's own suite exercises exactly the code consumers import.
16
+ - **`createMockOllamaServer(options?)`** — the recommended factory. Built-in **default routes** answer the official liveness probe (`HEAD /` → 200, `GET /` → the plain-text body "Ollama is running" byte-for-byte) and `GET /api/version` (configurable version, default `0.14.0`), so `client.ping()` works against a fresh mock with zero setup. `defaultRoutes: false` restores bare 404s; `register()` always wins over the defaults.
17
+ - **Request recording** — `server.requests` captures every request (method, path, parsed body, timestamp), including default-route hits and 404s; `clearRequests()` for `beforeEach` hooks. Wire-level assertions without per-route callbacks.
18
+ - **In-band mid-stream error injection** — `injectErrorFrame: { message, after? }` emits an `{"error": "..."}` NDJSON frame while HTTP stays 200, exactly how a real daemon reports GPU OOM/driver crashes mid-generation. The default injection point is `chunks.length - 1`: every content chunk, then the error frame, with the final `done` frame never emitted — an error frame after `done` would be unread (the stream has already completed cleanly). `after: 0` errors before any content; explicit larger values move the position. The SDK's stream readers raise `OllamaStreamError` with the partial content.
19
+ - **`rawBody`** route option for exact non-NDJSON bodies (the liveness body uses it — no trailing newline).
20
+ - **Chunk builders** producing correctly-shaped `/api/chat` frames: `mockChatChunk` (one frame, counter defaults on the done frame for telemetry paths), `mockTextChunks` (chunked text + done frame), and `mockToolCallChunks` — parallel tool calls in one frame, the documented native shape, followed by the done frame.
21
+ - **Packaging**: dedicated `./testing` entry in the exports map (paired `.d.ts`/`.d.cts`), a new tsup entry, and the consumer-type verification now compiles a real CJS (node16) **and** ESM (nodenext) consumer importing the subpath. Node.js-only by design (`node:http`) — never imported from the Edge-safe main entry, so `verify:edge-runtime` stays clean.
22
+
23
+ ### Lab
24
+
25
+ - `lab/15-speculative-draft/measure-draft-speedup.ts`: live harness comparing `draft_num_predict` 0/4/8/16 — warm-up per setting, 5 samples each, median `tokensPerSecond` via `formatTelemetry`, and a speedup table against drafting-disabled. Makes "the server has no draft network, the option is being ignored" obvious at a glance.
26
+
3
27
  ## [1.13.0] — docs.ollama.com re-audit: parity verifier reads checkbox state, strict compat types, web-search agent tools
4
28
 
5
29
  A full re-read of docs.ollama.com (API reference, capability guides, integrations, OpenAPI spec) against the SDK. The native API was already at parity except for two upstream changes; the real defect was in the compatibility-parity tooling, which had been reporting fields Ollama documents as unsupported as "supported".
package/README.md CHANGED
@@ -33,7 +33,8 @@ Nexum’s model harness (2026-10-09) exercised this SDK via `gemma4:cloud` and `
33
33
  - 🌉 **OpenAI & Anthropic Compatibility Bridges**: Built-in clients for `/v1/chat/completions`, `/v1/responses`, `/v1/models`, and `/v1/messages`, including `reasoning_effort`/`reasoning.effort` for thinking models — plus an ergonomic dual-mode `client.responses.create()` bridge that prefers native `/v1/responses` and transparently re-issues via `/api/chat` on older servers (pre-v0.13.3).
34
34
  - 💬 **KV-Cache-Aware Conversation Sessions**: `client.session(model, systemPrompt)` keeps an append-only, prefix-stable history that maximizes Ollama's KV-cache reuse across turns, surfaces per-turn + cumulative cache hit rates (`prompt_eval_cached_count`) so cache degradation is visible instead of silent, and offers a one-call `compact()` sliding-window escape hatch when a session outgrows its window.
35
35
  - 🛑 **Context-Window Safety**: Heuristic client-side token estimation warns (or throws, `onContextOverflow: 'throw'`) before a request is sent when the prompt approaches the effective `num_ctx` window — Ollama's default behavior is to _silently truncate_ oversized prompts — and `defaultContextLength` makes the window explicit on every request.
36
- - 📦 **Blob & Custom Model Publishing**: Content-addressed blob management per Ollama's documented protocol — `computeBlobDigest()` (SHA-256 → `sha256:<hex>`), `createBlobFromData()`/`createBlobFromFile()` (HEAD-check + upload, skipping existing blobs), and one-shot `createModelFromGguf()` (upload GGUF shard blobs, then `/api/create` with `files: {name: digest}`).
36
+ - 📦 **Blob & Custom Model Publishing**: Content-addressed blob management per Ollama's documented protocol — `computeBlobDigest()` (SHA-256 → `sha256:<hex>`), `createBlobFromData()`/`createBlobFromFile()` (HEAD-check + upload, skipping existing blobs), and one-shot `createModelFromGguf()` (upload GGUF shard blobs, then `/api/create` with `files: {name: digest}`) — with **speculative-decoding assembly**: `draftGguf` uploads the draft network's shards into `draft_files`, `draft_quantize` compresses them independently, and `parameters.draft_num_predict` tunes the per-step draft length.
37
+ - 🧪 **In-Memory Mock Server (`@nemesis-oss/ollama-sdk/testing`)**: The SDK's own deterministic test double, promoted public — `createMockOllamaServer()` with scriptable NDJSON routes, built-in liveness/version defaults (so `ping()` works out of the box), request recording for wire-level assertions, mid-stream **in-band error injection** (`{"error": "..."}` frames at HTTP 200, exactly how a real daemon reports GPU OOM), connection drops, and correctly-shaped chunk builders for streaming text and **parallel tool calls** — zero network, zero daemon, CI-deterministic.
37
38
  - 🧵 **Batched Embeddings with Backpressure**: `client.embedBatch()` splits corpora of any size into bounded `/api/embed` batches (default 32 inputs × 3 in flight) — order-preserving, fail-fast, progress-reporting — so RAG/vector ingestion can't OOM a local daemon or saturate `OLLAMA_MAX_QUEUE` the way `Promise.all` floods do, with per-string context-window pre-flight against silent truncation.
38
39
  - 🧹 **Client Teardown (`destroy()`)**: One call aborts every in-flight request, active stream, and queued capacity waiter with `OllamaAbortError` — the clean-exit path for `worker_threads`, CLI runners, and short-lived scripts where dangling fetch bodies keep the event loop alive.
39
40
  - 🧠 **Dynamic Context-Window Discovery**: `client.models.getContextLength({ model })` resolves the real window from the server — the allocated window of the running instance (`/api/ps`), the Modelfile `num_ctx` default, and the native GGUF maximum (`model_info`), in precedence order — so applications stop hardcoding 2048/4096 guesses.
@@ -497,6 +498,45 @@ await client.models.createModelFromGguf('my-model', [
497
498
  ]);
498
499
  ```
499
500
 
501
+ ### Speculative Decoding (Draft Models)
502
+
503
+ Ollama packages speculative decoding as a small **draft network** whose token
504
+ guesses are batch-verified against the primary weights in a single forward
505
+ pass — the draft runs ahead, the primary model confirms. On the wire this is
506
+ `draft_files` (draft GGUF digests) + `draft_quantize` on `/api/create`, the
507
+ `draft_num_predict` Modelfile parameter, and the `draft_num_predict` runtime
508
+ option at generation time. The SDK covers the full flow:
509
+
510
+ ```typescript
511
+ // 1. Assemble: primary + draft shards, draft weights quantized independently
512
+ // (draft tokens are verified, never returned — aggressive compression is
513
+ // usually safe). Separate draft models default draft_num_predict to 4;
514
+ // 0 disables drafting; models with embedded MTP tensors require setting
515
+ // it explicitly.
516
+ await client.models.createModelFromGguf('my-model', '/models/model.gguf', {
517
+ draftGguf: '/models/draft.gguf',
518
+ draftQuantize: 'q8_0',
519
+ parameters: { draft_num_predict: 8 },
520
+ });
521
+
522
+ // 2. Generate with drafting (also available ad hoc via options at runtime)
523
+ const response = await client.chat({
524
+ model: 'my-model',
525
+ messages: [{ role: 'user', content: 'Describe a lighthouse in a storm.' }],
526
+ options: { draft_num_predict: 8 },
527
+ });
528
+
529
+ // 3. Measure the speedup with the standardized telemetry
530
+ const t = formatTelemetry(response);
531
+ console.log(`${t.tokensPerSecond} tok/s (load ${t.modelLoadMs}ms)`);
532
+ ```
533
+
534
+ Already have the draft blobs uploaded? Pass the digest map directly —
535
+ `createModel({ draft_files: { 'draft.gguf': digest } })` — or merge both
536
+ (path uploads win on basename collision). A live measurement harness
537
+ comparing `draft_num_predict` settings lives at
538
+ `lab/15-speculative-draft/measure-draft-speedup.ts`.
539
+
500
540
  ### Token Log Probabilities (`logprobs`)
501
541
 
502
542
  Set `logprobs: true` (optionally with `top_logprobs`) on `chat()`/`generate()` to get per-token log probabilities back — useful for confidence scoring, speculative decoding, or agent routing decisions.
@@ -955,9 +995,19 @@ rejects `x-api-key`-only auth; the SDK always sends the bearer header plus
955
995
  `anthropic-version: 2023-06-01`):
956
996
 
957
997
  ```typescript
958
- const cloud = new OllamaClient({ baseUrl: 'https://ollama.com', apiKey: process.env.OLLAMA_API_KEY });
959
- await cloud.openai.chatCompletions({ model: 'gemma4:31b', messages: [{ role: 'user', content: 'Hi' }] });
960
- await cloud.anthropic.messages({ model: 'gemma4:31b', max_tokens: 1024, messages: [{ role: 'user', content: 'Hi' }] });
998
+ const cloud = new OllamaClient({
999
+ baseUrl: 'https://ollama.com',
1000
+ apiKey: process.env.OLLAMA_API_KEY,
1001
+ });
1002
+ await cloud.openai.chatCompletions({
1003
+ model: 'gemma4:31b',
1004
+ messages: [{ role: 'user', content: 'Hi' }],
1005
+ });
1006
+ await cloud.anthropic.messages({
1007
+ model: 'gemma4:31b',
1008
+ max_tokens: 1024,
1009
+ messages: [{ role: 'user', content: 'Hi' }],
1010
+ });
961
1011
  ```
962
1012
 
963
1013
  The cloud API does not support stateful Responses, built-in web search through
@@ -1454,6 +1504,62 @@ windows: [...] })` directly with your own `windowMs`/`maxTokens`/`maxRequests` p
1454
1504
 
1455
1505
  ---
1456
1506
 
1507
+ ### Testing with the In-Memory Mock Server (`/testing`)
1508
+
1509
+ The SDK ships its own deterministic test double as a dedicated subpath —
1510
+ `@nemesis-oss/ollama-sdk/testing` — the same engine the SDK's own test suite
1511
+ runs against (ADR 0029: promote, don't fork). No daemon, no network, no GPU:
1512
+ scriptable NDJSON routes on a real `node:http` server, built-in defaults for
1513
+ the official liveness probe and `/api/version`, wire-level request recording,
1514
+ and failure injection that covers what static cassettes can't.
1515
+
1516
+ ```typescript
1517
+ import {
1518
+ createMockOllamaServer,
1519
+ mockTextChunks,
1520
+ mockToolCallChunks,
1521
+ } from '@nemesis-oss/ollama-sdk/testing';
1522
+ import { OllamaClient } from '@nemesis-oss/ollama-sdk';
1523
+
1524
+ const server = createMockOllamaServer(); // ephemeral port, defaults on
1525
+ await server.start();
1526
+ const client = new OllamaClient({ baseUrl: server.baseUrl });
1527
+
1528
+ // Defaults answer out of the box: HEAD / (ping) and GET /api/version.
1529
+ // Register everything else — register() always wins over the defaults.
1530
+ server.register('/api/chat', {
1531
+ chunks: mockToolCallChunks([
1532
+ // Parallel tool calls: one frame, complete calls — the native shape.
1533
+ { name: 'get_weather', arguments: { city: 'Tokyo' } },
1534
+ { name: 'get_time', arguments: { tz: 'Asia/Tokyo' } },
1535
+ ]),
1536
+ chunkDelayMs: 10, // inter-chunk backpressure testing
1537
+ });
1538
+
1539
+ // Mid-stream failure, GPU-OOM style: HTTP stays 200, the done frame is
1540
+ // replaced by an {"error": "..."} frame — the SDK raises OllamaStreamError.
1541
+ server.register('/api/generate', {
1542
+ chunks: mockTextChunks('partial answer', { chunkSize: 8 }),
1543
+ injectErrorFrame: { message: 'CUDA out of memory' }, // after: N to move it
1544
+ });
1545
+
1546
+ // ... run your code against the client ...
1547
+
1548
+ // Wire-level assertions: every request was recorded.
1549
+ expect(server.requests[0]?.body).toMatchObject({ model: 'llama3.2' });
1550
+ server.clearRequests(); // and server.clear() between tests
1551
+
1552
+ await server.stop(); // idempotent — safe in afterEach
1553
+ ```
1554
+
1555
+ Per-route controls: `status` (503/502/429 bodies), `chunkDelayMs`,
1556
+ `dropConnectionMidStream` (connection reset mid-response), `onRequest`
1557
+ (per-route body capture), and `injectErrorFrame.after` (error position).
1558
+ Chunk builders: `mockChatChunk` (one frame with counter defaults),
1559
+ `mockTextChunks` (chunked text + done frame), `mockToolCallChunks` (parallel
1560
+ tool calls + done frame). Node.js only — the subpath builds on `node:http`
1561
+ and is never imported from the Edge-safe main entry.
1562
+
1457
1563
  ### Edge Runtime Compatibility
1458
1564
 
1459
1565
  The core client (`OllamaClient`, `Agent`, `ToolRegistry`, and everything exported from
@@ -1,4 +1,4 @@
1
- import { O as OllamaRuntime, H as HttpClient, A as AbortableAsyncIterable, af as SseEvent, y as OllamaClientError } from './runtime-CIQYCRL3.js';
1
+ import { O as OllamaRuntime, H as HttpClient, A as AbortableAsyncIterable, af as SseEvent, y as OllamaClientError } from './runtime-BIjr5ME4.js';
2
2
 
3
3
  /**
4
4
  * AUTO-GENERATED by `npm run contract:generate`. DO NOT EDIT.
@@ -1,4 +1,4 @@
1
- import { O as OllamaRuntime, H as HttpClient, A as AbortableAsyncIterable, af as SseEvent, y as OllamaClientError } from './runtime-DmI1cOEq.cjs';
1
+ import { O as OllamaRuntime, H as HttpClient, A as AbortableAsyncIterable, af as SseEvent, y as OllamaClientError } from './runtime-C4RRHqOs.cjs';
2
2
 
3
3
  /**
4
4
  * AUTO-GENERATED by `npm run contract:generate`. DO NOT EDIT.
@@ -1,4 +1,4 @@
1
- import { l as ToolDefinition, i as ToolCall, n as Message } from './types-B57ZZfp3.js';
1
+ import { l as ToolDefinition, i as ToolCall, n as Message } from './types-DbPnfVh-.js';
2
2
  import { z } from 'zod';
3
3
 
4
4
  /**
@@ -1,4 +1,4 @@
1
- import { l as ToolDefinition, i as ToolCall, n as Message } from './types-B57ZZfp3.cjs';
1
+ import { l as ToolDefinition, i as ToolCall, n as Message } from './types-DbPnfVh-.cjs';
2
2
  import { z } from 'zod';
3
3
 
4
4
  /**
@@ -1,8 +1,8 @@
1
- import { O as OpenAIChatCompletionRequest, a as OpenAIChatCompletionResponse, b as OpenAIChatCompletionChunk, c as OpenAICompletionRequest, d as OpenAICompletionResponse, e as OpenAICompletionChunk, f as OpenAIEmbeddingRequest, g as OpenAIEmbeddingResponse, h as OpenAIListModelsResponse, i as OpenAIModelItem, j as OpenAIResponsesRequest, k as OpenAIResponsesResponse, l as OpenAIResponsesStreamEvent, A as AnthropicMessagesRequest, m as AnthropicMessagesResponse, n as AnthropicMessageStreamEvent } from './anthropic-CYy_ECam.cjs';
2
- export { N as NativeApi } from './anthropic-CYy_ECam.cjs';
3
- import { O as OllamaRuntime } from './runtime-DmI1cOEq.cjs';
4
- export { a as OperationDefinition } from './runtime-DmI1cOEq.cjs';
5
- import './types-B57ZZfp3.cjs';
1
+ import { O as OpenAIChatCompletionRequest, a as OpenAIChatCompletionResponse, b as OpenAIChatCompletionChunk, c as OpenAICompletionRequest, d as OpenAICompletionResponse, e as OpenAICompletionChunk, f as OpenAIEmbeddingRequest, g as OpenAIEmbeddingResponse, h as OpenAIListModelsResponse, i as OpenAIModelItem, j as OpenAIResponsesRequest, k as OpenAIResponsesResponse, l as OpenAIResponsesStreamEvent, A as AnthropicMessagesRequest, m as AnthropicMessagesResponse, n as AnthropicMessageStreamEvent } from './anthropic-UeaVjLeP.cjs';
2
+ export { N as NativeApi } from './anthropic-UeaVjLeP.cjs';
3
+ import { O as OllamaRuntime } from './runtime-C4RRHqOs.cjs';
4
+ export { a as OperationDefinition } from './runtime-C4RRHqOs.cjs';
5
+ import './types-DbPnfVh-.cjs';
6
6
  import 'zod';
7
7
 
8
8
  /**
@@ -1,8 +1,8 @@
1
- import { O as OpenAIChatCompletionRequest, a as OpenAIChatCompletionResponse, b as OpenAIChatCompletionChunk, c as OpenAICompletionRequest, d as OpenAICompletionResponse, e as OpenAICompletionChunk, f as OpenAIEmbeddingRequest, g as OpenAIEmbeddingResponse, h as OpenAIListModelsResponse, i as OpenAIModelItem, j as OpenAIResponsesRequest, k as OpenAIResponsesResponse, l as OpenAIResponsesStreamEvent, A as AnthropicMessagesRequest, m as AnthropicMessagesResponse, n as AnthropicMessageStreamEvent } from './anthropic-B9AFyzXI.js';
2
- export { N as NativeApi } from './anthropic-B9AFyzXI.js';
3
- import { O as OllamaRuntime } from './runtime-CIQYCRL3.js';
4
- export { a as OperationDefinition } from './runtime-CIQYCRL3.js';
5
- import './types-B57ZZfp3.js';
1
+ import { O as OpenAIChatCompletionRequest, a as OpenAIChatCompletionResponse, b as OpenAIChatCompletionChunk, c as OpenAICompletionRequest, d as OpenAICompletionResponse, e as OpenAICompletionChunk, f as OpenAIEmbeddingRequest, g as OpenAIEmbeddingResponse, h as OpenAIListModelsResponse, i as OpenAIModelItem, j as OpenAIResponsesRequest, k as OpenAIResponsesResponse, l as OpenAIResponsesStreamEvent, A as AnthropicMessagesRequest, m as AnthropicMessagesResponse, n as AnthropicMessageStreamEvent } from './anthropic-DrDUfbfQ.js';
2
+ export { N as NativeApi } from './anthropic-DrDUfbfQ.js';
3
+ import { O as OllamaRuntime } from './runtime-BIjr5ME4.js';
4
+ export { a as OperationDefinition } from './runtime-BIjr5ME4.js';
5
+ import './types-DbPnfVh-.js';
6
6
  import 'zod';
7
7
 
8
8
  /**
@@ -1,6 +1,6 @@
1
- export { I as InvokeRequest, O as OllamaRuntime, b as OllamaRuntimeOptions, a as OperationDefinition, S as SchemaRef } from './runtime-DmI1cOEq.cjs';
1
+ export { I as InvokeRequest, O as OllamaRuntime, b as OllamaRuntimeOptions, a as OperationDefinition, S as SchemaRef } from './runtime-C4RRHqOs.cjs';
2
2
  import { z } from 'zod';
3
- import './types-B57ZZfp3.cjs';
3
+ import './types-DbPnfVh-.cjs';
4
4
 
5
5
  /**
6
6
  * AUTO-GENERATED by `npm run contract:generate`. DO NOT EDIT.
@@ -1,6 +1,6 @@
1
- export { I as InvokeRequest, O as OllamaRuntime, b as OllamaRuntimeOptions, a as OperationDefinition, S as SchemaRef } from './runtime-CIQYCRL3.js';
1
+ export { I as InvokeRequest, O as OllamaRuntime, b as OllamaRuntimeOptions, a as OperationDefinition, S as SchemaRef } from './runtime-BIjr5ME4.js';
2
2
  import { z } from 'zod';
3
- import './types-B57ZZfp3.js';
3
+ import './types-DbPnfVh-.js';
4
4
 
5
5
  /**
6
6
  * AUTO-GENERATED by `npm run contract:generate`. DO NOT EDIT.
package/dist/index.cjs CHANGED
@@ -2817,12 +2817,29 @@ var ModelsClient = class {
2817
2817
  * await client.models.createModelFromGguf('my-model', '/path/to/model.gguf', {
2818
2818
  * template: '{{ .Prompt }}',
2819
2819
  * });
2820
+ *
2821
+ * // Speculative decoding: primary + draft shards + tuning:
2822
+ * await client.models.createModelFromGguf('my-model', '/path/to/model.gguf', {
2823
+ * draftGguf: '/path/to/draft.gguf',
2824
+ * draftQuantize: 'q8_0',
2825
+ * parameters: { draft_num_predict: 8 },
2826
+ * });
2820
2827
  * ```
2821
2828
  *
2829
+ * Speculative-decoding models assemble the same way: pass the draft
2830
+ * network's GGUF(s) as `draftGguf` (uploaded into `draft_files`), optionally
2831
+ * set `draft_quantize` to compress the draft weights independently of the
2832
+ * primary model, and set `parameters.draft_num_predict` to tune how many
2833
+ * tokens the draft network predicts per step (Ollama defaults separate
2834
+ * draft models to 4; `0` disables drafting; models with embedded MTP
2835
+ * tensors require setting it explicitly).
2836
+ *
2822
2837
  * @param model Name for the new model (e.g. `'my-model:latest'`).
2823
2838
  * @param gguf Path(s) to GGUF file(s) on the local filesystem.
2824
2839
  * @param opts Everything else `/api/create` accepts (`template`, `system`,
2825
- * `parameters`, `license`, `quantize`, …) except `files` and `from`.
2840
+ * `parameters`, `license`, `quantize`, `draftGguf`, …) except `files`,
2841
+ * `draft_files` (use `draftGguf` for path-based drafts; an explicit digest
2842
+ * map is still accepted and merged), `from`, and `stream`.
2826
2843
  */
2827
2844
  async createModelFromGguf(model, gguf, opts) {
2828
2845
  const paths = typeof gguf === "string" ? [gguf] : gguf;
@@ -2836,14 +2853,29 @@ var ModelsClient = class {
2836
2853
  const uploaded = await this.createBlobFromFile(path, { signal: opts?.signal });
2837
2854
  files[uploaded.fileName] = uploaded.digest;
2838
2855
  }
2856
+ const draftOpt = opts?.draftGguf;
2857
+ const draftPaths = typeof draftOpt === "string" ? [draftOpt] : draftOpt ?? [];
2839
2858
  const {
2840
2859
  model: _model,
2841
2860
  files: _files,
2842
2861
  from: _from,
2843
2862
  stream: _stream,
2863
+ draftGguf: _draftGguf,
2864
+ draft_files: _draftFiles,
2844
2865
  ...rest
2845
2866
  } = opts ?? {};
2846
- return this.create({ ...rest, model, files, stream: false });
2867
+ const draftFiles = { ..._draftFiles ?? {} };
2868
+ for (const path of draftPaths) {
2869
+ const uploaded = await this.createBlobFromFile(path, { signal: opts?.signal });
2870
+ draftFiles[uploaded.fileName] = uploaded.digest;
2871
+ }
2872
+ return this.create({
2873
+ ...rest,
2874
+ model,
2875
+ files,
2876
+ ...Object.keys(draftFiles).length > 0 ? { draft_files: draftFiles } : {},
2877
+ stream: false
2878
+ });
2847
2879
  }
2848
2880
  };
2849
2881