@nemesis-oss/ollama-sdk 1.3.0 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +43 -212
- package/README.md +391 -18
- package/dist/compose-BR70yb4W.d.cts +164 -0
- package/dist/compose-DfAqKAFj.d.ts +164 -0
- package/dist/generated-api.cjs +454 -0
- package/dist/generated-api.cjs.map +1 -0
- package/dist/generated-api.d.cts +1651 -0
- package/dist/generated-api.d.ts +1651 -0
- package/dist/generated-api.js +428 -0
- package/dist/generated-api.js.map +1 -0
- package/dist/generated-models-schemas.cjs +328 -0
- package/dist/generated-models-schemas.cjs.map +1 -0
- package/dist/generated-models-schemas.d.cts +463 -0
- package/dist/generated-models-schemas.d.ts +463 -0
- package/dist/generated-models-schemas.js +291 -0
- package/dist/generated-models-schemas.js.map +1 -0
- package/dist/generated-runtime.cjs +145 -0
- package/dist/generated-runtime.cjs.map +1 -0
- package/dist/generated-runtime.d.cts +2 -0
- package/dist/generated-runtime.d.ts +2 -0
- package/dist/generated-runtime.js +143 -0
- package/dist/generated-runtime.js.map +1 -0
- package/dist/index-2RIErNT5.d.cts +583 -0
- package/dist/index-DIQ8NUbU.d.ts +583 -0
- package/dist/index.cjs +1886 -154
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +817 -453
- package/dist/index.d.ts +817 -453
- package/dist/index.js +1877 -155
- package/dist/index.js.map +1 -1
- package/dist/mcp/tools.json +1058 -0
- package/dist/mcp-generated.cjs +386 -0
- package/dist/mcp-generated.cjs.map +1 -0
- package/dist/mcp-generated.d.cts +35 -0
- package/dist/mcp-generated.d.ts +35 -0
- package/dist/mcp-generated.js +381 -0
- package/dist/mcp-generated.js.map +1 -0
- package/dist/mcp-http.cjs +100 -0
- package/dist/mcp-http.cjs.map +1 -0
- package/dist/mcp-http.d.cts +48 -0
- package/dist/mcp-http.d.ts +48 -0
- package/dist/mcp-http.js +96 -0
- package/dist/mcp-http.js.map +1 -0
- package/dist/mcp-stdio.cjs +49 -0
- package/dist/mcp-stdio.cjs.map +1 -0
- package/dist/mcp-stdio.d.cts +34 -0
- package/dist/mcp-stdio.d.ts +34 -0
- package/dist/mcp-stdio.js +47 -0
- package/dist/mcp-stdio.js.map +1 -0
- package/dist/skills.cjs.map +1 -1
- package/dist/skills.d.cts +3 -2
- package/dist/skills.d.ts +3 -2
- package/dist/skills.js.map +1 -1
- package/dist/types-B9_axjum.d.cts +66 -0
- package/dist/types-B9_axjum.d.ts +66 -0
- package/dist/{compose-CDWq8oxJ.d.cts → types-BQhgW-du.d.cts} +68 -171
- package/dist/{compose-CDWq8oxJ.d.ts → types-BQhgW-du.d.ts} +68 -171
- package/package.json +96 -5
package/README.md
CHANGED
|
@@ -16,7 +16,7 @@
|
|
|
16
16
|
- 🎯 **Zod-Powered Structured Outputs**: Strictly typed schema enforcement via `chatWithSchema` and `generateWithSchema` with resilient markdown JSON parsing.
|
|
17
17
|
- 🛠️ **Autonomous Agent & Tool Calling**: Multi-turn agent loop (`Agent`) with automated tool execution, parameter validation, and self-correcting error recovery.
|
|
18
18
|
- 🌐 **High Availability & Failover**: Multi-endpoint registry with priority routing, circuit breaker failover, active health checks, and per-endpoint `models` allow-lists for routing several model-specific API keys through one client.
|
|
19
|
-
- 🔌 **Model Context Protocol (MCP)**:
|
|
19
|
+
- 🔌 **Model Context Protocol (MCP)**: First-class, transport-neutral `McpBridge` for converting MCP tool descriptors into Ollama function definitions and registering executable MCP-backed tools.
|
|
20
20
|
- 📚 **Full Model Lifecycle**: `pullModel`, `pushModel`, `createModel`, `copyModel`, `deleteModel`, `listModels`, `showModel`, and `ps()` (currently loaded models) — full parity with Ollama's model management API.
|
|
21
21
|
- 🔎 **Ollama Cloud Web Tools**: `webSearch`/`webFetch` wrap Ollama's hosted `/api/web_search` and `/api/web_fetch` tools at `ollama.com` (requires an `OLLAMA_API_KEY`), independent of any local `baseUrl`.
|
|
22
22
|
- 🌉 **OpenAI & Anthropic Compatibility Bridges**: Built-in clients for `/v1/chat/completions`, `/v1/responses`, `/v1/models`, and `/v1/messages`, including `reasoning_effort`/`reasoning.effort` for thinking models.
|
|
@@ -25,9 +25,126 @@
|
|
|
25
25
|
- 📊 **Client-Side Quota Monitoring**: `QuotaManager` tracks token/request usage against budgets you configure across rolling windows (e.g. Ollama Cloud's 5-hour session / 7-day weekly resets) and fails fast with `OllamaQuotaExceededError` before a request is sent.
|
|
26
26
|
- ⚡ **Edge Runtime Verified**: CI bundles and runs the client in a real Edge Runtime sandbox (Cloudflare Workers/Vercel Edge-compatible) with zero Node.js APIs.
|
|
27
27
|
- 📦 **Dual ESM & CJS Build**: Full module support with clean TypeScript `.d.ts` declaration maps.
|
|
28
|
+
- 🧩 **Contract-First Architecture**: A single canonical IR (`contracts/ir/ollama.ir.json`) drives TypeScript interfaces, generated API classes (`NativeApi`/`OpenAIApi`/`AnthropicApi`), MCP tool descriptors, Zod schemas, and field-level parity verification. New Ollama endpoints (like `/v1/systemone`) are caught automatically by bidirectional endpoint discovery. See [ADRs 0013-0019](./docs/adr/README.md).
|
|
28
29
|
|
|
29
30
|
---
|
|
30
31
|
|
|
32
|
+
### Contract parity verification
|
|
33
|
+
|
|
34
|
+
The repository keeps the Ollama compatibility contract in the canonical IR at
|
|
35
|
+
`contracts/ir/ollama.ir.json` (compiled from `contracts/sources/` + `contracts/overlays/`)
|
|
36
|
+
and checks it against the current official documentation with
|
|
37
|
+
`npm run verify:contract-parity`. Each operation's `parity:` block distinguishes
|
|
38
|
+
**supported**, **explicitly unsupported**, and **SDK-only** fields, and also tracks
|
|
39
|
+
documented response fields plus the public streaming-event union for compatibility
|
|
40
|
+
adapters. This prevents a field merely being mentioned in upstream documentation
|
|
41
|
+
from being mistaken for a supported Ollama feature.
|
|
42
|
+
|
|
43
|
+
For callers who want compile-time enforcement of the documented Ollama subset, the package
|
|
44
|
+
also exports strict request types such as `OllamaOpenAIChatCompletionRequest`,
|
|
45
|
+
`OllamaOpenAIResponsesRequest`, `OllamaOpenAIEmbeddingRequest`, and
|
|
46
|
+
`OllamaAnthropicMessagesRequest`. The broader compatibility request types remain
|
|
47
|
+
available for pass-through interoperability and vendor-specific fields.
|
|
48
|
+
|
|
49
|
+
### SSE Streaming Foundation
|
|
50
|
+
|
|
51
|
+
Compatibility endpoints use Server-Sent Events when `stream: true`. The SDK now exposes a provider-neutral `parseSseStream()` and `HttpClient.requestSseStream()`, plus typed adapters for OpenAI Chat/Completions/Responses and Anthropic Messages. Native Ollama NDJSON streaming remains separate.
|
|
52
|
+
|
|
53
|
+
```typescript
|
|
54
|
+
const stream = await client.openai.chatCompletions({
|
|
55
|
+
model: 'qwen3',
|
|
56
|
+
messages: [{ role: 'user', content: 'Explain SSE.' }],
|
|
57
|
+
stream: true,
|
|
58
|
+
});
|
|
59
|
+
|
|
60
|
+
for await (const chunk of stream) {
|
|
61
|
+
process.stdout.write(chunk.choices[0]?.delta.content ?? '');
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
const final = await stream.finalResult;
|
|
65
|
+
console.log(final.usage);
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
## MCP bridge
|
|
69
|
+
|
|
70
|
+
The SDK exposes a transport-neutral `McpBridge` that converts MCP `tools/list` descriptors into native Ollama function tools and registers executable MCP-backed tools. MCP tool arguments are validated against the advertised JSON Schema at the `ToolRegistry` boundary (including object, array, scalar, required, enum, and common numeric/string constraints).
|
|
71
|
+
|
|
72
|
+
- Paginated `tools/list` discovery follows `nextCursor`, with repeated-cursor and page-limit protection.
|
|
73
|
+
- MCP JSON Schema is preserved in the generated Ollama tool definition.
|
|
74
|
+
- `structuredContent` and non-text MCP content blocks are retained in the model-visible tool result.
|
|
75
|
+
- MCP `isError: true` results remain model-readable; they are not treated as transport failures.
|
|
76
|
+
|
|
77
|
+
`loadMcpTools()` keeps the historical model-oriented string result by default. Set `resultMode: 'structured'` when application code needs the raw MCP `CallToolResult`, including `structuredContent`, content blocks, `isError`, and `_meta`:
|
|
78
|
+
|
|
79
|
+
```typescript
|
|
80
|
+
const tools = await loadMcpTools(mcpClient, { resultMode: 'structured' });
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
- Input JSON Schema validation happens before the MCP server is called, preventing invalid model-generated arguments from crossing the protocol boundary.
|
|
84
|
+
- `resultMode: 'structured'` is available for lossless programmatic access to raw MCP tool results while preserving the legacy text mode by default.
|
|
85
|
+
- `AbortSignal` propagates through discovery and tool execution.
|
|
86
|
+
- The optional Node-only `@nemesis-oss/ollama-sdk/mcp/stdio` subpath uses the official MCP v2 stdio transport without importing Node-only code from the root package.
|
|
87
|
+
|
|
88
|
+
For remote MCP servers, the optional `@nemesis-oss/ollama-sdk/mcp/http` subpath uses the official MCP client v2 transport. Streamable HTTP is the default; legacy SSE can be selected explicitly, or `transport: 'auto'` can try Streamable HTTP first and fall back to SSE only on a compatible non-authentication 4xx response.
|
|
89
|
+
|
|
90
|
+
```typescript
|
|
91
|
+
import { Agent, McpBridge, OllamaClient, ToolRegistry } from '@nemesis-oss/ollama-sdk';
|
|
92
|
+
import { connectMcpHttpClient } from '@nemesis-oss/ollama-sdk/mcp/http';
|
|
93
|
+
|
|
94
|
+
const connection = await connectMcpHttpClient({
|
|
95
|
+
url: 'https://example.com/mcp',
|
|
96
|
+
transport: 'auto',
|
|
97
|
+
requestInit: {
|
|
98
|
+
headers: { Authorization: `Bearer ${process.env.MCP_TOKEN}` },
|
|
99
|
+
},
|
|
100
|
+
});
|
|
101
|
+
|
|
102
|
+
const bridge = new McpBridge(connection.client, { namePrefix: 'mcp_' });
|
|
103
|
+
const registry = new ToolRegistry();
|
|
104
|
+
await bridge.register(registry);
|
|
105
|
+
|
|
106
|
+
const agent = new Agent(new OllamaClient(), { tools: registry });
|
|
107
|
+
const result = await agent.run({
|
|
108
|
+
model: 'qwen3',
|
|
109
|
+
messages: [{ role: 'user', content: 'Use the MCP tools to complete this task.' }],
|
|
110
|
+
});
|
|
111
|
+
|
|
112
|
+
await connection.close();
|
|
113
|
+
console.log(result.finalMessage.content);
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
Use `connection.terminateSession()` when the remote Streamable HTTP server exposes a session you want to terminate explicitly. Legacy SSE connections do not expose that method.
|
|
117
|
+
|
|
118
|
+
The MCP TypeScript SDK v2 treats Streamable HTTP as the preferred remote transport and keeps SSE as a legacy fallback during migration. citeturn514654search0turn514654search6
|
|
119
|
+
|
|
120
|
+
The MCP TypeScript SDK v2 implements the 2026-07-28 protocol revision. Its `listTools()` client path aggregates pagination, and `CallToolResult` represents tool failures as ordinary results with `isError`, while `structuredContent` is available for machine-readable output. citeturn0search0turn1search4turn0search6
|
|
121
|
+
|
|
122
|
+
### MCP stdio example
|
|
123
|
+
|
|
124
|
+
```typescript
|
|
125
|
+
import { connectStdioMcpClient } from '@nemesis-oss/ollama-sdk/mcp/stdio';
|
|
126
|
+
import { Agent, McpBridge, OllamaClient, ToolRegistry } from '@nemesis-oss/ollama-sdk';
|
|
127
|
+
|
|
128
|
+
const connection = await connectStdioMcpClient({
|
|
129
|
+
command: 'npx',
|
|
130
|
+
args: ['-y', '@modelcontextprotocol/server-filesystem', '/tmp'],
|
|
131
|
+
});
|
|
132
|
+
|
|
133
|
+
const bridge = new McpBridge(connection.client, { namePrefix: 'mcp_' });
|
|
134
|
+
const registry = new ToolRegistry();
|
|
135
|
+
await bridge.register(registry);
|
|
136
|
+
|
|
137
|
+
const client = new OllamaClient();
|
|
138
|
+
const agent = new Agent(client, { tools: registry });
|
|
139
|
+
const result = await agent.run({
|
|
140
|
+
model: 'qwen3',
|
|
141
|
+
messages: [{ role: 'user', content: 'List the available files.' }],
|
|
142
|
+
});
|
|
143
|
+
|
|
144
|
+
await connection.close();
|
|
145
|
+
console.log(result.finalMessage.content);
|
|
146
|
+
```
|
|
147
|
+
|
|
31
148
|
## Installation
|
|
32
149
|
|
|
33
150
|
```bash
|
|
@@ -57,7 +174,7 @@ console.log(answer);
|
|
|
57
174
|
|
|
58
175
|
### Thinking & Reasoning Token Streams
|
|
59
176
|
|
|
60
|
-
`think` is Ollama's native reasoning-effort parameter, exposed directly on `chat()`/`generate()
|
|
177
|
+
`think` is Ollama's native reasoning-effort parameter, exposed directly on `chat()`/`generate()`. It accepts `true`, `false`, `null` (model default), or a model-defined string. Use `client.capabilities(model).thinking` to discover the exact string values and default reported by `/api/show`. The OpenAI bridge's `reasoning_effort`/`reasoning.effort` follows the same model-defined convention.
|
|
61
178
|
|
|
62
179
|
```typescript
|
|
63
180
|
const stream = await client.chatStream({
|
|
@@ -79,6 +196,20 @@ const final = await stream.finalResult;
|
|
|
79
196
|
console.log(`\nEval tokens/sec: ${final.usage?.tokensPerSecond}`);
|
|
80
197
|
```
|
|
81
198
|
|
|
199
|
+
### Cached Prompt Tokens
|
|
200
|
+
|
|
201
|
+
Ollama reports `prompt_eval_cached_count` as the number of prompt tokens read from the KV cache. The raw value is preserved on native chat/generate responses, and normalized stream usage exposes it as `cachedPromptTokens`.
|
|
202
|
+
|
|
203
|
+
```typescript
|
|
204
|
+
const res = await client.chat({
|
|
205
|
+
model: 'gpt-oss:20b',
|
|
206
|
+
messages: [{ role: 'user', content: 'hello' }],
|
|
207
|
+
});
|
|
208
|
+
|
|
209
|
+
console.log(res.prompt_eval_count, res.prompt_eval_cached_count);
|
|
210
|
+
console.log(res.prompt_eval_cached_count ?? 0);
|
|
211
|
+
```
|
|
212
|
+
|
|
82
213
|
### Token Log Probabilities (`logprobs`)
|
|
83
214
|
|
|
84
215
|
Set `logprobs: true` (optionally with `top_logprobs`) on `chat()`/`generate()` to get per-token log probabilities back — useful for confidence scoring, speculative decoding, or agent routing decisions.
|
|
@@ -231,12 +362,12 @@ const response = await agent.run({
|
|
|
231
362
|
console.log(response.finalMessage.content);
|
|
232
363
|
```
|
|
233
364
|
|
|
234
|
-
Ollama's native tool-calling protocol has no OpenAI-style call ID,
|
|
235
|
-
synthesizes a stable
|
|
236
|
-
|
|
237
|
-
`response.turns[0].toolResults[0].toolCallId
|
|
238
|
-
|
|
239
|
-
[ADR 0007](./docs/adr/0007-synthetic-tool-call-ids.md).
|
|
365
|
+
Ollama's native tool-calling protocol has no OpenAI-style call ID, so the SDK
|
|
366
|
+
synthesizes a stable client-side ID for tracing and execution correlation:
|
|
367
|
+
`response.turns[0].toolCalls[0].id` matches
|
|
368
|
+
`response.turns[0].toolResults[0].toolCallId`. Native `role: 'tool'` history entries
|
|
369
|
+
use Ollama's documented `tool_name` field; the SDK-local `toolCallId` is not sent on
|
|
370
|
+
the wire. See [ADR 0007](./docs/adr/0007-synthetic-tool-call-ids.md).
|
|
240
371
|
|
|
241
372
|
Running several `Agent`s against different models/API keys for different roles (e.g. a
|
|
242
373
|
planning model, a coding model, a research model) is a single `OllamaClient` with
|
|
@@ -247,6 +378,44 @@ unaware of credentials entirely. See
|
|
|
247
378
|
[Guide: Benchmarking Agent Models Across Multiple Ollama Cloud Keys](./docs/guides/multi-model-agent-benchmarking.md)
|
|
248
379
|
for a worked example and a runnable scenario.
|
|
249
380
|
|
|
381
|
+
### MCP Bridge & Agent Capability Preflight
|
|
382
|
+
|
|
383
|
+
The SDK exposes `McpBridge` as a transport-neutral adapter between an MCP client and
|
|
384
|
+
Ollama's native function-tool format. The bridge does not own the MCP transport or spawn
|
|
385
|
+
processes; your application supplies an MCP client with `listTools()` and `callTool()`.
|
|
386
|
+
This keeps the package root Edge-runtime safe while still supporting Node-specific
|
|
387
|
+
transports such as stdio through your chosen MCP implementation.
|
|
388
|
+
|
|
389
|
+
```typescript
|
|
390
|
+
import { McpBridge, ToolRegistry } from '@nemesis-oss/ollama-sdk';
|
|
391
|
+
|
|
392
|
+
const bridge = new McpBridge(mcpClient, { namePrefix: 'mcp_' });
|
|
393
|
+
|
|
394
|
+
const registry = new ToolRegistry();
|
|
395
|
+
await bridge.register(registry);
|
|
396
|
+
|
|
397
|
+
// The same MCP definitions can also be inspected before registration.
|
|
398
|
+
const definitions = await bridge.definitions();
|
|
399
|
+
console.log(definitions);
|
|
400
|
+
```
|
|
401
|
+
|
|
402
|
+
For tool-enabled `Agent` runs, capability preflight is enabled by default when the
|
|
403
|
+
chat client exposes `capabilities()` (including `OllamaClient`). The agent queries
|
|
404
|
+
`/api/show` before the first model turn and throws `OllamaIncompatibleModelError` when
|
|
405
|
+
the selected model does not advertise `tools`.
|
|
406
|
+
|
|
407
|
+
The same metadata can size the tool context automatically: when `options.num_ctx` is
|
|
408
|
+
not supplied, the default is `32768`, clamped to the model-reported
|
|
409
|
+
`capabilities.contextLength` when available. An explicit `options.num_ctx` always
|
|
410
|
+
takes precedence.
|
|
411
|
+
|
|
412
|
+
For legacy or custom `AgentChatClient` implementations, set
|
|
413
|
+
`validateToolCapability: false` to skip capability discovery and the automatic
|
|
414
|
+
context-size override.
|
|
415
|
+
|
|
416
|
+
See [ADR 0011](./docs/adr/0011-mcp-boundary-and-agent-tool-preconditions.md) for the
|
|
417
|
+
MCP boundary and agent-precondition rationale.
|
|
418
|
+
|
|
250
419
|
### Tool Execution Safety & Sandboxing
|
|
251
420
|
|
|
252
421
|
Tool arguments and, indirectly, which tools get called at all are driven by model
|
|
@@ -385,9 +554,24 @@ so it never burns a request retrying an unrelated key:
|
|
|
385
554
|
const client = new OllamaClient({
|
|
386
555
|
baseUrl: 'https://ollama.com',
|
|
387
556
|
endpoints: [
|
|
388
|
-
{
|
|
389
|
-
|
|
390
|
-
|
|
557
|
+
{
|
|
558
|
+
name: 'gpt-oss-key',
|
|
559
|
+
apiKey: process.env.OLLAMA_KEY_1!,
|
|
560
|
+
baseUrl: 'https://ollama.com',
|
|
561
|
+
models: ['gpt-oss:120b'],
|
|
562
|
+
},
|
|
563
|
+
{
|
|
564
|
+
name: 'minimax-key',
|
|
565
|
+
apiKey: process.env.OLLAMA_KEY_2!,
|
|
566
|
+
baseUrl: 'https://ollama.com',
|
|
567
|
+
models: ['minimax-m3'],
|
|
568
|
+
},
|
|
569
|
+
{
|
|
570
|
+
name: 'nemotron-key',
|
|
571
|
+
apiKey: process.env.OLLAMA_KEY_3!,
|
|
572
|
+
baseUrl: 'https://ollama.com',
|
|
573
|
+
models: ['nemotron-3-super'],
|
|
574
|
+
},
|
|
391
575
|
],
|
|
392
576
|
});
|
|
393
577
|
|
|
@@ -526,7 +710,7 @@ always tried first regardless of its active count.
|
|
|
526
710
|
|
|
527
711
|
#### Queueing past capacity, instead of overrunning an account
|
|
528
712
|
|
|
529
|
-
`'least-connections'` alone only guarantees no collision for up to `N`
|
|
713
|
+
`'least-connections'` alone only guarantees no collision for up to `N` _simultaneous_
|
|
530
714
|
calls against `N` candidates — an `(N+1)`th concurrent call would still be routed to
|
|
531
715
|
whichever account looks least busy at that instant, which, once all `N` already have one
|
|
532
716
|
request each, means sending it to an account that's already at its real limit. Add
|
|
@@ -621,7 +805,7 @@ compute, not a fixed token count, so a budget you set for one model won't transf
|
|
|
621
805
|
exactly to another.
|
|
622
806
|
|
|
623
807
|
`QuotaManager` is a client-side safety net, not a mirror of Ollama's real limits: it
|
|
624
|
-
tracks usage you record against budgets
|
|
808
|
+
tracks usage you record against budgets _you_ configure over one or more rolling
|
|
625
809
|
windows, and fails fast — before a request is even sent — once a window's budget is
|
|
626
810
|
spent. Pair it with catching `OllamaRateLimitError` (the server's actual `429`) as the
|
|
627
811
|
authoritative signal.
|
|
@@ -647,7 +831,10 @@ async function chatWithQuota(prompt: string) {
|
|
|
647
831
|
quota.assertCanProceed(); // throws OllamaQuotaExceededError if any window is spent
|
|
648
832
|
|
|
649
833
|
try {
|
|
650
|
-
const res = await client.chat({
|
|
834
|
+
const res = await client.chat({
|
|
835
|
+
model: 'qwen3:8b',
|
|
836
|
+
messages: [{ role: 'user', content: prompt }],
|
|
837
|
+
});
|
|
651
838
|
quota.recordUsage(res); // reads prompt_eval_count/eval_count off the raw response
|
|
652
839
|
return res.message.content;
|
|
653
840
|
} catch (error) {
|
|
@@ -685,6 +872,130 @@ real Edge Runtime sandbox exposing only Web Standard globals. See
|
|
|
685
872
|
|
|
686
873
|
---
|
|
687
874
|
|
|
875
|
+
## Contract-First Architecture
|
|
876
|
+
|
|
877
|
+
Starting with v1.4.0, the SDK ships a **contract-first hybrid architecture** where a single
|
|
878
|
+
canonical IR (`contracts/ir/ollama.ir.json`) drives seven consumers:
|
|
879
|
+
|
|
880
|
+
1. **TypeScript interfaces** — `src/generated/models/<name>.ts` (36 schemas)
|
|
881
|
+
2. **Generated API classes** — `NativeApi` / `OpenAIApi` / `AnthropicApi` in `src/generated/api/`
|
|
882
|
+
3. **MCP tool descriptors** — `src/generated/mcp/tools.json` (21 tools, one per documented operation)
|
|
883
|
+
4. **Operation metadata** — `src/generated/metadata/operations.json`
|
|
884
|
+
5. **Field-level parity** — overlay `parity:` blocks verified by `npm run verify:contract-parity`
|
|
885
|
+
6. **Zod schemas** — `src/generated/models/<name>.schema.ts` (paired with every TypeScript interface)
|
|
886
|
+
7. **Bidirectional endpoint discovery** — catches new Ollama endpoints like `/v1/systemone` that the OpenAPI spec doesn't yet cover
|
|
887
|
+
|
|
888
|
+
The IR is compiled from upstream OpenAPI + hand-maintained behavioral overlays by
|
|
889
|
+
`npm run contract:normalize`. Run `npm run contract:generate` to regenerate every TypeScript /
|
|
890
|
+
Zod / MCP artifact from the IR. See [ADRs 0013-0019](./docs/adr/README.md) for the full design.
|
|
891
|
+
|
|
892
|
+
### Using the generated `NativeApi` (recommended for new code)
|
|
893
|
+
|
|
894
|
+
The generated surface is opt-in — existing `OllamaClient` callers don't need to change anything.
|
|
895
|
+
For new code, the generated API inherits every contract-layer guarantee (environment guards,
|
|
896
|
+
version guards, streaming defaults) automatically:
|
|
897
|
+
|
|
898
|
+
```typescript
|
|
899
|
+
import { HttpClient } from '@nemesis-oss/ollama-sdk';
|
|
900
|
+
import { OllamaRuntime } from '@nemesis-oss/ollama-sdk/generated/runtime';
|
|
901
|
+
import { NativeApi } from '@nemesis-oss/ollama-sdk/generated/api';
|
|
902
|
+
|
|
903
|
+
const http = new HttpClient({ baseUrl: 'http://localhost:11434' });
|
|
904
|
+
const runtime = new OllamaRuntime({ http });
|
|
905
|
+
const api = new NativeApi(runtime);
|
|
906
|
+
|
|
907
|
+
// Non-streaming chat:
|
|
908
|
+
const res = await api.chat({
|
|
909
|
+
model: 'qwen3:8b',
|
|
910
|
+
messages: [{ role: 'user', content: 'Hello' }],
|
|
911
|
+
stream: false,
|
|
912
|
+
});
|
|
913
|
+
|
|
914
|
+
// Streaming chat — explicitly request a stream:
|
|
915
|
+
const stream = await api.chat({
|
|
916
|
+
model: 'qwen3:8b',
|
|
917
|
+
messages: [{ role: 'user', content: 'Hello' }],
|
|
918
|
+
stream: true,
|
|
919
|
+
});
|
|
920
|
+
for await (const chunk of stream) {
|
|
921
|
+
console.log(chunk.message?.content);
|
|
922
|
+
}
|
|
923
|
+
```
|
|
924
|
+
|
|
925
|
+
### Mixing the legacy client with the generated surface
|
|
926
|
+
|
|
927
|
+
`OllamaClient.runtime` returns a cached `OllamaRuntime` that shares the client's
|
|
928
|
+
transport (HttpClient + middleware + retry + telemetry), so you can mix both
|
|
929
|
+
surfaces in the same process without configuring two HttpClient instances:
|
|
930
|
+
|
|
931
|
+
```typescript
|
|
932
|
+
import { OllamaClient } from '@nemesis-oss/ollama-sdk';
|
|
933
|
+
import { NativeApi } from '@nemesis-oss/ollama-sdk/generated/api';
|
|
934
|
+
|
|
935
|
+
const client = new OllamaClient({ baseUrl: 'http://localhost:11434' });
|
|
936
|
+
|
|
937
|
+
// Existing API:
|
|
938
|
+
const res = await client.chat({ model: 'qwen3:8b', messages });
|
|
939
|
+
|
|
940
|
+
// Generated API (shares transport):
|
|
941
|
+
const api = new NativeApi(client.runtime);
|
|
942
|
+
const res2 = await api.chat({ model: 'qwen3:8b', messages, stream: false });
|
|
943
|
+
```
|
|
944
|
+
|
|
945
|
+
`OllamaClient` carries a deprecation notice pointing to `NativeApi` for new code, but
|
|
946
|
+
no method signatures have changed — existing callers continue to work.
|
|
947
|
+
|
|
948
|
+
### Runtime validation with generated Zod schemas
|
|
949
|
+
|
|
950
|
+
Every TypeScript interface in `src/generated/models/<name>.ts` has a paired Zod schema
|
|
951
|
+
in `src/generated/models/<name>.schema.ts`. Use them for runtime validation of user
|
|
952
|
+
input, server responses, or anywhere you need to confirm a value matches the IR shape:
|
|
953
|
+
|
|
954
|
+
```typescript
|
|
955
|
+
import { ChatRequestSchema } from '@nemesis-oss/ollama-sdk/generated/models/schemas';
|
|
956
|
+
|
|
957
|
+
const result = ChatRequestSchema.safeParse(userInput);
|
|
958
|
+
if (!result.success) {
|
|
959
|
+
console.error(result.error.issues);
|
|
960
|
+
} else {
|
|
961
|
+
// result.data is typed as ChatRequest
|
|
962
|
+
}
|
|
963
|
+
```
|
|
964
|
+
|
|
965
|
+
### Generated MCP tools
|
|
966
|
+
|
|
967
|
+
The IR also produces 21 MCP tool descriptors (one per documented Ollama operation) at
|
|
968
|
+
`src/generated/mcp/tools.json`. The runtime adapter at
|
|
969
|
+
`@nemesis-oss/ollama-sdk/mcp/generated` exposes them as a callable tool registry:
|
|
970
|
+
|
|
971
|
+
```typescript
|
|
972
|
+
import { OllamaRuntime } from '@nemesis-oss/ollama-sdk/generated/runtime';
|
|
973
|
+
import {
|
|
974
|
+
listGeneratedOllamaTools,
|
|
975
|
+
callGeneratedOllamaTool,
|
|
976
|
+
} from '@nemesis-oss/ollama-sdk/mcp/generated';
|
|
977
|
+
|
|
978
|
+
const tools = listGeneratedOllamaTools();
|
|
979
|
+
// tools: [{ name: 'ollama_chat', inputSchema: {...}, annotations: {...} }, ...]
|
|
980
|
+
|
|
981
|
+
const result = await callGeneratedOllamaTool(runtime, 'ollama_version', {});
|
|
982
|
+
console.log(result.structuredContent); // { version: '0.5.0' }
|
|
983
|
+
```
|
|
984
|
+
|
|
985
|
+
### Contract maintenance commands
|
|
986
|
+
|
|
987
|
+
| Command | What it does |
|
|
988
|
+
| -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
|
|
989
|
+
| `npm run contract:fetch` | Re-pull the upstream OpenAPI spec into `contracts/sources/` |
|
|
990
|
+
| `npm run contract:normalize` | Compile sources + overlays → `contracts/ir/ollama.ir.json` |
|
|
991
|
+
| `npm run contract:validate` | Run schema + compatibility + bidirectional endpoint discovery validators |
|
|
992
|
+
| `npm run contract:diff` | Fail if the committed IR is stale (CI gate) |
|
|
993
|
+
| `npm run contract:generate` | Regenerate every TypeScript / Zod / MCP artifact from the IR (idempotent) |
|
|
994
|
+
| `npm run contract:drift` | Print drift report between generated types and `src/types.ts` (pass `--strict` to fail CI on unexpected drift) |
|
|
995
|
+
| `npm run verify:contract-parity` | Verify parity blocks against `src/types.ts` (structural) and `docs.ollama.com` (live-docs); pass `--skip-live-docs` for offline CI |
|
|
996
|
+
|
|
997
|
+
---
|
|
998
|
+
|
|
688
999
|
## Error Handling
|
|
689
1000
|
|
|
690
1001
|
Every failure thrown by the client is an `OllamaClientError` subclass, so you can catch the base
|
|
@@ -697,13 +1008,14 @@ class or narrow to a specific `code`:
|
|
|
697
1008
|
| `OllamaAuthError` | `auth_error` | `false` | The endpoint returned `401`/`403`. |
|
|
698
1009
|
| `OllamaNotFoundError` | `not_found` | `false` | The endpoint returned `404` (e.g. unknown model). |
|
|
699
1010
|
| `OllamaRateLimitError` | `rate_limited` | `true` | The endpoint returned `429`. |
|
|
700
|
-
| `OllamaQuotaExceededError` | `quota_exceeded` | `false` | `QuotaManager.assertCanProceed` was called and would exceed a configured usage budget. Thrown client-side, before any network call — see [Quota Monitoring](#quota-monitoring).
|
|
701
|
-
| `OllamaModelRoutingError` | `model_routing_error` | `false` | No configured `endpoints` entry's `models` allow-list includes the requested model. Thrown client-side, before any network call — see [Multiple API keys, each entitled to different models](#multiple-api-keys-each-entitled-to-different-models).
|
|
1011
|
+
| `OllamaQuotaExceededError` | `quota_exceeded` | `false` | `QuotaManager.assertCanProceed` was called and would exceed a configured usage budget. Thrown client-side, before any network call — see [Quota Monitoring](#quota-monitoring). |
|
|
1012
|
+
| `OllamaModelRoutingError` | `model_routing_error` | `false` | No configured `endpoints` entry's `models` allow-list includes the requested model. Thrown client-side, before any network call — see [Multiple API keys, each entitled to different models](#multiple-api-keys-each-entitled-to-different-models). |
|
|
702
1013
|
| `OllamaServerError` | `server_error` | `true` | The endpoint returned `5xx`. |
|
|
703
1014
|
| `OllamaAbortError` | `aborted` | `false` | The request was cancelled via `AbortSignal`. |
|
|
704
1015
|
| `OllamaToolValidationError` | `tool_validation_error` | `false` | A tool call's arguments, or a `chatWithSchema`/`generateWithSchema` result, failed Zod validation. |
|
|
705
1016
|
| `OllamaUnsupportedCapabilityError` | `unsupported_capability` | `false` | A `format` (structured output) request was made against an endpoint inferred as Ollama Cloud, which doesn't currently support it. Thrown before any network call; in `DEFAULT_FAILOVER_CODES`, so a multi-endpoint setup tries the next candidate first. |
|
|
706
|
-
| `
|
|
1017
|
+
| `OllamaIncompatibleModelError` | `incompatible_model` | `false` | A tool-enabled `Agent` run was blocked by capability preflight because `/api/show` did not advertise `tools`. |
|
|
1018
|
+
| `OllamaAgentMaxIterationsError` | `agent_max_iterations_exceeded` | `false` | An `Agent` run exceeded `maxIterations` without producing a final answer. |
|
|
707
1019
|
| `OllamaMcpError` | `mcp_error` | varies | An MCP `listTools`/`callTool` call failed. |
|
|
708
1020
|
| `OllamaSkillNotFoundError` | `skill_not_found` | `false` | `applySkill` referenced a skill that isn't registered. |
|
|
709
1021
|
| `OllamaSkillInvalidError` | `skill_invalid` | `false` | A skill's frontmatter or contents failed to parse. |
|
|
@@ -736,9 +1048,70 @@ observe per-endpoint circuit state directly.
|
|
|
736
1048
|
|
|
737
1049
|
---
|
|
738
1050
|
|
|
1051
|
+
## Documentation
|
|
1052
|
+
|
|
1053
|
+
The repository maintains implementation-facing documentation alongside the package README:
|
|
1054
|
+
|
|
1055
|
+
- [Architecture Decision Records](./docs/adr/README.md) — rationale for durable API and architecture choices.
|
|
1056
|
+
- [Canonical Ollama IR](./contracts/ir/ollama.ir.json) — machine-readable Ollama contract compiled from sources + overlays, checked by CI.
|
|
1057
|
+
- [Multi-model agent benchmarking guide](./docs/guides/multi-model-agent-benchmarking.md) — running agent roles across multiple Ollama endpoints.
|
|
1058
|
+
- [Upstream compatibility notes](./docs/upstream/) — pinned OpenAI/Anthropic compatibility references and the upstream OpenAPI snapshot.
|
|
1059
|
+
- [Manual laboratory](./LAB_README.md) — runnable experiments for protocol, tool, streaming, and agent behavior.
|
|
1060
|
+
|
|
1061
|
+
## Contract parity verification
|
|
1062
|
+
|
|
1063
|
+
`npm run verify:contract-parity` checks the canonical IR against the official Ollama
|
|
1064
|
+
documentation. The structural half verifies that every `parity.request.fields` /
|
|
1065
|
+
`parity.response.fields` entry exists on the hand-written TypeScript interfaces
|
|
1066
|
+
in `src/types.ts` and `src/integrations/*`. The live-docs half (when run without
|
|
1067
|
+
`--skip-live-docs`) fetches the rendered docs at `docs.ollama.com` and asserts each
|
|
1068
|
+
supported field is documented as supported, each unsupported field is explicitly
|
|
1069
|
+
marked as such, and each streaming event type is present.
|
|
1070
|
+
|
|
1071
|
+
The gate covers every documented native REST endpoint (`chat`, `generate`, `embed`,
|
|
1072
|
+
`tags`, `ps`, `show`, `create`, `copy`, `pull`, `push`, `delete`, and `version`)
|
|
1073
|
+
plus the OpenAI and Anthropic compatibility request surfaces. SDK-only fields
|
|
1074
|
+
that exist in `src/types.ts` but are absent from the OpenAPI snapshot are declared
|
|
1075
|
+
as `sdkOnlyFields` in the overlay parity blocks and treated as expected drift,
|
|
1076
|
+
not findings.
|
|
1077
|
+
|
|
1078
|
+
## Middleware and request lifecycle
|
|
1079
|
+
|
|
1080
|
+
`OllamaClient` accepts `middleware` and `onLifecycleEvent` configuration. Middleware runs
|
|
1081
|
+
around the underlying HTTP request for native endpoints, OpenAI/Anthropic compatibility,
|
|
1082
|
+
model health checks, and the hosted web tools. Lifecycle events expose `start`, `success`,
|
|
1083
|
+
`retry`, and `error` events with a request id, timing, and status/error information.
|
|
1084
|
+
|
|
1085
|
+
```ts
|
|
1086
|
+
const client = new OllamaClient({
|
|
1087
|
+
middleware: [
|
|
1088
|
+
async ({ request, next }) => {
|
|
1089
|
+
request.headers['X-Request-Source'] = 'my-app';
|
|
1090
|
+
return next();
|
|
1091
|
+
},
|
|
1092
|
+
],
|
|
1093
|
+
onLifecycleEvent: (event) => {
|
|
1094
|
+
console.log(event.type, event.requestId);
|
|
1095
|
+
},
|
|
1096
|
+
});
|
|
1097
|
+
```
|
|
1098
|
+
|
|
1099
|
+
Retry backoff is also cancellation-aware: passing an `AbortSignal` to `withRetry` or cancelling an `OllamaClient` request interrupts an in-progress backoff immediately instead of waiting for the next retry delay.
|
|
1100
|
+
|
|
1101
|
+
## Compatibility routing and stream lifecycle
|
|
1102
|
+
|
|
1103
|
+
OpenAI and Anthropic compatibility requests use the same endpoint registry, model-scoped
|
|
1104
|
+
routing, failover, concurrency limits, and request cancellation as native inference calls.
|
|
1105
|
+
When `stream: true`, an endpoint capacity slot remains held until the compatibility stream
|
|
1106
|
+
finishes, errors, or is explicitly aborted.
|
|
1107
|
+
|
|
1108
|
+
Compatibility stream objects expose `.abort()` and `finalResult`. The configured request
|
|
1109
|
+
timeout remains active for the lifetime of the stream rather than ending when HTTP headers
|
|
1110
|
+
arrive.
|
|
1111
|
+
|
|
739
1112
|
## Testing
|
|
740
1113
|
|
|
741
|
-
The test suite
|
|
1114
|
+
The test suite is exercised across unit, integration, and functional coverage:
|
|
742
1115
|
|
|
743
1116
|
```bash
|
|
744
1117
|
# Run unit, integration, and functional test suite
|