@nemesis-oss/ollama-sdk 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,71 @@
1
+ # Changelog
2
+
3
+ All notable changes to `@nemesis-oss/ollama-sdk` will be documented in this file.
4
+
5
+ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6
+ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
+
8
+ ## [1.0.0] - 2026-08-17
9
+
10
+ First public release. The version was `0.1.0` throughout development but was never
11
+ published to npm, so the entire feature set below ships as the initial `1.0.0`.
12
+
13
+ `1.0.0` marks the public API surface (`OllamaClient`, `Agent`, `ToolRegistry`, the
14
+ streaming adapters, the error hierarchy, and the compatibility bridges) as stable under
15
+ [Semantic Versioning](https://semver.org/spec/v2.0.0.html) — breaking changes to it now
16
+ require a major version bump.
17
+
18
+ ### Added
19
+
20
+ - **Core Ollama REST API Client:**
21
+ - Full support for `chat`, `generate`, `embed`, `embeddings`, `ps`, and `version`.
22
+ - Zero-runtime-dependency HTTP transport with native `fetch`, streaming NDJSON parsing, and binary body blob uploads.
23
+ - Multi-endpoint high availability registry with circuit breaker failover, priority routing, and health checks.
24
+ - Configurable exponential backoff with full jitter, retryable error predicates, and timeout signal propagation.
25
+ - **Model Lifecycle & Blob Management:**
26
+ - `createModel`, `pullModel`, `pushModel`, `copyModel`, `deleteModel`, `listModels`, `showModel`.
27
+ - Dedicated blob management endpoints: `createBlob` (`POST /api/blobs/:digest`) and `checkBlob` (`HEAD /api/blobs/:digest`).
28
+ - **Structured Outputs & Schema Validation:**
29
+ - Seamless Zod schema conversion to JSON Schema, supporting both Zod v4 (native `z.toJSONSchema`) and Zod v3 (structural fallback).
30
+ - Structured output parsing with resilient markdown code fence JSON extraction and strict validation errors (`OllamaToolValidationError`).
31
+ - `zod` is a peer dependency (`^3.22.0 || ^4.0.0`) rather than a bundled dependency, so consumers don't end up with a duplicate copy in `node_modules`.
32
+ - **Reasoning & Thinking Tokens:**
33
+ - Native parsing of reasoning traces (`<think>` tags and `message.thinking`) with dual stream events (`thinking` vs `token`).
34
+ - **Agentic Workflow & Tool Calling:**
35
+ - Autonomous multi-turn agent execution loop (`Agent`).
36
+ - Tool definition helper (`defineTool`) with Zod parameter schemas.
37
+ - Tool registry with duplicate detection and execution error recovery.
38
+ - Model Context Protocol (MCP) server integration (`createMcpToolSet`, `registerMcpTools`).
39
+ - Opt-in tool execution sandboxing: per-tool/registry `timeoutMs` (cooperative cancellation via `AbortSignal`, surfaced as `OllamaToolTimeoutError`), `maxConcurrency` bounding parallel tool calls, and `maxOutputChars` truncating oversized tool output before it re-enters the conversation history. See [ADR 0004](./docs/adr/0004-tool-execution-sandboxing.md).
40
+ - Synthetic, client-generated `tool_call_id` correlation: `ToolCall.id`, `Message.tool_call_id`, and `ToolExecutionResult.toolCallId` — Ollama's native protocol has no call ID, so the SDK synthesizes a stable one (`crypto.randomUUID()`, the Web Standard global, not `node:crypto`, to stay Edge-safe) the first time it sees a call without one, and reuses it consistently across the streamed `tool_call` event, the final aggregated message, the tool execution result, and the `role: 'tool'` history entry `Agent` appends. Execution/result correlation by array order still works exactly as before; `id` is an additive convenience. See [ADR 0007](./docs/adr/0007-synthetic-tool-call-ids.md).
41
+ - **Architecture Decision Records:** `docs/adr/` documents the rationale behind the circuit breaker failure model, the dual ESM/CJS packaging strategy, Zod v3/v4 dual support, the tool execution sandboxing model, OpenTelemetry instrumentation, Edge runtime CI verification, synthetic tool-call IDs, endpoint failover scope, and registry parameter variance.
42
+ - **OpenTelemetry Instrumentation:** Automatic spans (`@opentelemetry/api` is an optional peer dependency, a no-op when absent or unconfigured) for HTTP requests, endpoint failover attempts, non-streaming `chat`/`generate` calls (using the Gen AI semantic conventions, including token usage), and `Agent` runs (`invoke_agent` → `ollama.agent.turn` → `execute_tool`). See [ADR 0005](./docs/adr/0005-opentelemetry-instrumentation.md).
43
+ - **Edge Runtime CI Verification:** `npm run verify:edge-runtime` bundles `dist/index.js` for a browser/edge platform with `esbuild` (failing on any `node:*` import, matching Cloudflare Workers/Vercel Edge Runtime's own bundlers) and runs a full `OllamaClient` + `Agent` + tool-calling round trip inside `@edge-runtime/vm`'s sandboxed Edge Runtime — a real V8 context exposing only Web Standard globals. Wired into CI as its own job and into `verify`/`prepublishOnly`. See [ADR 0006](./docs/adr/0006-edge-runtime-ci-and-benchmarks.md).
44
+ - **Benchmarks:** `npm run bench` (via `vitest bench`, no new dependency) covers NDJSON stream parsing, Zod schema conversion/structured output parsing, `ToolRegistry` dispatch overhead, and `OllamaClient.chat`'s end-to-end request pipeline overhead. Wired into CI as its own job.
45
+ - **Protocol Compatibility Bridges:**
46
+ - OpenAI compatibility bridge (`/v1/chat/completions`, `/v1/models`), including `stream_options.include_usage` and `tools` (function calling) typing — both confirmed supported by Ollama's OpenAI-compat layer; `tool_choice`/`parallel_tool_calls` are deliberately not typed, since Ollama documents `tool_choice` as explicitly unsupported. `@remarks` JSDoc on `OpenAICompatClient` scopes it as a documented subset of the OpenAI API, not the full Responses API surface.
47
+ - Anthropic compatibility bridge (`/v1/messages`), including `cache_control` typing on content blocks for prompt caching. `@remarks` JSDoc on `AnthropicCompatClient` scopes it as a subset of the Messages API (no tool use, extended thinking, citations, files, or Batches API).
48
+ - **Capability Detection Fixes:** `ModelCapabilities.supportsStructuredOutputRequest` is no longer hardcoded `true` — it's inferred `false` for endpoints classified as `cloud` by `inferRuntimeMode` (Ollama Cloud does not currently support structured outputs) and documented as a best-effort heuristic, not a guarantee, since Ollama doesn't expose this as a queryable capability. Added `supportsThinking`, derived from the model's reported `thinking` capability.
49
+ - **Fail-Fast `OllamaUnsupportedCapabilityError`:** `chat`/`chatStream`/`chatWithSchema` and `generate`/`generateStream`/`generateWithSchema` now throw this error _before making any network call_ when `format` is set against an endpoint inferred as Ollama Cloud, instead of sending a request Ollama Cloud is known to reject. `unsupported_capability` is included in `DEFAULT_FAILOVER_CODES`, so a multi-endpoint setup tries the next candidate (e.g. a local fallback) before the error ever reaches the caller — verified with a test asserting the rejected cloud endpoint's URL is never actually fetched.
50
+ - **`dimensions` on `embed()`:** `EmbedRequestOptions.dimensions` lets callers request truncated embedding vectors, matching Ollama's `/api/embed` parameter.
51
+ - **Environment Variable Fallbacks:** `OllamaClient`'s default single-endpoint `baseUrl`/`apiKey` fall back to `OLLAMA_HOST`/`OLLAMA_API_KEY` (the same variables the official `ollama` CLI and client libraries read) when not passed explicitly, matching the convention of other major LLM SDKs. Guarded to remain a no-op (not a `ReferenceError`) on Edge runtimes where `process` doesn't exist. Explicit `config.baseUrl`/`config.apiKey`, and any use of `config.endpoints`, always take precedence.
52
+ - **Web Standard Stream Adapters:**
53
+ - `toTextStream`, `toDataStream`, and `toResponse` for direct integration with Next.js, Vercel AI SDK, and Web standard streams.
54
+ - **Skills System:**
55
+ - Frontmatter parser for `SKILL.md` documents.
56
+ - Skill composition and prompt injection into system messages (`applySkill`).
57
+ - **Documentation Hygiene:** `OllamaClient.embeddings()` is now marked `@deprecated` (Ollama's `/api/embeddings` was superseded by `/api/embed`, exposed as `embed()`). `ToolCall`, `Agent`, and `ToolRegistry.executeToolCalls` document that Ollama's native tool-calling protocol has no wire-level call ID, dispatch/results are still ordered/concurrency-bounded rather than ID-driven, and that `id`/`tool_call_id` (see above) is a client-synthesized convenience layered on top, not a protocol guarantee.
58
+ - **Testing & Quality Assurance:**
59
+ - New cancellation tests prove `AbortSignal` genuinely aborts the underlying `fetch` call (not just racing a promise) and propagates from `Agent.run` into `ToolExecutionContext.signal`, including the early-abort race where the signal is already aborted before the request starts.
60
+ - 4-tier test architecture: Unit, Integration, Functional, and Behavioral testing (50 tests).
61
+ - VCR record and replay harness with real cassettes generated against `qwen3.5:2b` and `nomic-embed-text:latest`.
62
+ - Multi-node CI/CD workflow (Node 18, 20, 22) and automated npm release with provenance.
63
+
64
+ ### Fixed
65
+
66
+ - **`ToolRegistry` rejected every concretely-typed tool.** Its constructor, `register`, and `registerMany` accepted `Tool<never, unknown>`; because `Tool` is invariant in `TParams` (it appears in both `schema: z.ZodType<TParams>` and `execute`), _nothing_ was assignable to it — so `new ToolRegistry([myTool])`, `new ToolRegistry({ tools: [myTool] })`, and `.register(myTool)` all failed to compile for consumers, including the exact example in this README. Replaced with a new exported `AnyTool` alias. Tool authoring keeps full type safety (`defineTool` still infers `TParams` from the Zod schema and type-checks `execute` against it); only the registry's storage type is widened. Found by installing the packed tarball into a throwaway TypeScript project during pre-publish verification. See [ADR 0009](./docs/adr/0009-anytool-registry-variance.md).
67
+ - **Tests are now type-checked.** `tsconfig.json` scoped checking to `src/`, and Vitest's esbuild transform strips types without verifying them — so all 20 test files were unchecked, and had masked the bug above with `as never` casts. `npm run typecheck` now uses `tsconfig.typecheck.json`, covering `src/`, `test/`, `bench/`, and `scripts/`. The casts (and a matching `as unknown as z.ZodType<never>` in `mcp-tools.ts`) are gone.
68
+
69
+ ### Changed
70
+
71
+ - **Endpoint Failover Scope:** `ModelsClient` operations (`list`, `show`, `pull`, `push`, `create`, `delete`, `copy`, `ps`, `version`, `createBlob`, `checkBlob`) and `OllamaClient.capabilities()` no longer participate in cross-endpoint failover — each now targets only the single best candidate endpoint (same-endpoint retry via backoff still applies). These operations act on a specific server's local model catalog/blob store, which isn't an interchangeable resource across endpoints the way `chat`/`generate`/`embed` inference is; failing `deleteModel`/`listModels`/etc. over to a different endpoint was silently operating on the wrong server's state, not retrying "the same" request. `chat`/`generate`/`embed`/`embeddings`/`webSearch`/`webFetch` failover behavior is unchanged. See [ADR 0008](./docs/adr/0008-endpoint-failover-scope.md).
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Shubham Taywade
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md ADDED
@@ -0,0 +1,378 @@
1
+ # @nemesis-oss/ollama-sdk
2
+
3
+ [![CI](https://github.com/shubhamtaywade82/ollama-client-ts/actions/workflows/ci.yml/badge.svg)](https://github.com/shubhamtaywade82/ollama-client-ts/actions/workflows/ci.yml)
4
+ [![npm version](https://img.shields.io/npm/v/@nemesis-oss/ollama-sdk.svg)](https://www.npmjs.com/package/@nemesis-oss/ollama-sdk)
5
+ [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
6
+
7
+ > Production-grade TypeScript SDK for Ollama. Built with native fetch, high availability failover, multi-turn tool calling, structured outputs with Zod, reasoning stream tokens, OpenAI & Anthropic compatibility bridges, MCP integration, and Web Stream adapters.
8
+
9
+ ---
10
+
11
+ ## Key Features
12
+
13
+ - 🚀 **Native Web Standards**: Built on native `fetch` and Web Streams. Zero external HTTP dependencies.
14
+ - 🧠 **Reasoning & Thinking Tokens**: First-class support for reasoning models (`qwen3.5:2b`, `deepseek-r1`) with discrete `thinking` and `token` streaming events.
15
+ - 🎯 **Zod-Powered Structured Outputs**: Strictly typed schema enforcement via `chatWithSchema` and `generateWithSchema` with resilient markdown JSON parsing.
16
+ - 🛠️ **Autonomous Agent & Tool Calling**: Multi-turn agent loop (`Agent`) with automated tool execution, parameter validation, and self-correcting error recovery.
17
+ - 🌐 **High Availability & Failover**: Multi-endpoint registry with priority routing, circuit breaker failover, and active health checks.
18
+ - 🔌 **Model Context Protocol (MCP)**: Native adapters to convert MCP tools into Ollama-compatible function schemas.
19
+ - 🌉 **OpenAI & Anthropic Compatibility Bridges**: Built-in clients for `/v1/chat/completions`, `/v1/models`, and `/v1/messages`.
20
+ - 🌊 **Web Stream Adapters**: Drop-in adapters (`toTextStream`, `toDataStream`, `toResponse`) for Next.js Route Handlers and Vercel AI SDK.
21
+ - 📈 **OpenTelemetry Instrumentation**: Automatic spans for HTTP requests, endpoint failover, chat/generate calls, and agent runs — zero-cost when OpenTelemetry isn't installed.
22
+ - ⚡ **Edge Runtime Verified**: CI bundles and runs the client in a real Edge Runtime sandbox (Cloudflare Workers/Vercel Edge-compatible) with zero Node.js APIs.
23
+ - 📦 **Dual ESM & CJS Build**: Full module support with clean TypeScript `.d.ts` declaration maps.
24
+
25
+ ---
26
+
27
+ ## Installation
28
+
29
+ ```bash
30
+ npm install @nemesis-oss/ollama-sdk zod
31
+ ```
32
+
33
+ `zod` is a peer dependency (`^3.22.0 || ^4.0.0`) — install whichever major version your project already uses instead of getting a second copy bundled in.
34
+
35
+ ---
36
+
37
+ ## Quick Start
38
+
39
+ ### Basic Chat & Completion
40
+
41
+ ```typescript
42
+ import { OllamaClient } from '@nemesis-oss/ollama-sdk';
43
+
44
+ const client = new OllamaClient();
45
+
46
+ // Text helper
47
+ const answer = await client.chatText({
48
+ model: 'qwen3.5:2b',
49
+ messages: [{ role: 'user', content: 'Explain quantum computing in one sentence.' }],
50
+ });
51
+ console.log(answer);
52
+ ```
53
+
54
+ ### Thinking & Reasoning Token Streams
55
+
56
+ ```typescript
57
+ const stream = await client.chatStream({
58
+ model: 'qwen3.5:2b',
59
+ messages: [{ role: 'user', content: 'What is 18 * 4?' }],
60
+ options: { temperature: 0 },
61
+ });
62
+
63
+ for await (const event of stream) {
64
+ if (event.type === 'thinking') {
65
+ process.stdout.write(`\x1b[33m${event.data.delta}\x1b[0m`); // Thinking trace
66
+ } else if (event.type === 'token') {
67
+ process.stdout.write(event.data.delta); // Final answer token
68
+ }
69
+ }
70
+
71
+ const final = await stream.finalResult;
72
+ console.log(`\nEval tokens/sec: ${final.usage?.tokensPerSecond}`);
73
+ ```
74
+
75
+ ### Structured Outputs with Zod
76
+
77
+ ```typescript
78
+ import { z } from 'zod';
79
+
80
+ const ProductSchema = z.object({
81
+ name: z.string(),
82
+ category: z.enum(['electronics', 'books', 'apparel']),
83
+ price: z.number(),
84
+ tags: z.array(z.string()),
85
+ });
86
+
87
+ const product = await client.chatWithSchema(
88
+ {
89
+ model: 'qwen3.5:2b',
90
+ messages: [{ role: 'user', content: 'Generate a gaming keyboard item.' }],
91
+ },
92
+ ProductSchema,
93
+ );
94
+
95
+ console.log(product.name, product.price);
96
+ ```
97
+
98
+ ### Vector Embeddings & Similarity
99
+
100
+ ```typescript
101
+ const res = await client.embed({
102
+ model: 'nomic-embed-text:latest',
103
+ input: [
104
+ 'Machine learning and neural networks',
105
+ 'Artificial intelligence algorithms',
106
+ 'Baking traditional French sourdough bread',
107
+ ],
108
+ });
109
+
110
+ console.log(
111
+ `Generated ${res.embeddings.length} vectors with dimension ${res.embeddings[0].length}`,
112
+ );
113
+ ```
114
+
115
+ ### Autonomous Agent & Tool Calling
116
+
117
+ ```typescript
118
+ import { Agent, defineTool, ToolRegistry, OllamaClient } from '@nemesis-oss/ollama-sdk';
119
+ import { z } from 'zod';
120
+
121
+ const client = new OllamaClient();
122
+
123
+ const weatherTool = defineTool({
124
+ name: 'get_weather',
125
+ description: 'Get the current weather for a city',
126
+ schema: z.object({ city: z.string() }),
127
+ execute: async ({ city }) => ({ city, temperature: '22°C', condition: 'Sunny' }),
128
+ });
129
+
130
+ const registry = new ToolRegistry([weatherTool]);
131
+ const agent = new Agent(client, { tools: registry, maxIterations: 5 });
132
+
133
+ const response = await agent.run({
134
+ model: 'qwen3.5:2b',
135
+ messages: [{ role: 'user', content: 'What is the weather in Tokyo?' }],
136
+ });
137
+
138
+ console.log(response.finalMessage.content);
139
+ ```
140
+
141
+ Ollama's native tool-calling protocol has no OpenAI-style call ID, but the SDK
142
+ synthesizes a stable one so results are still correlatable without tracking array
143
+ position by hand: `response.turns[0].toolCalls[0].id` matches
144
+ `response.turns[0].toolResults[0].toolCallId`, and the `role: 'tool'` message `Agent`
145
+ appends to history carries the same value as `tool_call_id`. See
146
+ [ADR 0007](./docs/adr/0007-synthetic-tool-call-ids.md).
147
+
148
+ ### Tool Execution Safety & Sandboxing
149
+
150
+ Tool arguments and, indirectly, which tools get called at all are driven by model
151
+ output — treat them as untrusted input. `ToolRegistry` supports three defensive
152
+ controls, all opt-in (disabled by default, matching prior behavior) so existing agents
153
+ aren't affected until you turn them on:
154
+
155
+ ```typescript
156
+ const registry = new ToolRegistry({
157
+ tools: [weatherTool],
158
+ // Fail a call that runs longer than this instead of stalling the agent loop forever.
159
+ // Override per-tool via `defineTool({ ..., timeoutMs: 2_000 })`.
160
+ timeoutMs: 10_000,
161
+ // Cap how many tool calls run in parallel when the model requests several at once.
162
+ maxConcurrency: 4,
163
+ // Truncate oversized tool output before it re-enters the conversation history.
164
+ maxOutputChars: 20_000,
165
+ });
166
+ ```
167
+
168
+ - **`timeoutMs`** races the tool call against a timer and rejects with
169
+ `OllamaToolTimeoutError` on expiry. Enforcement is cooperative: it stops the _agent_
170
+ from waiting indefinitely, but genuinely halting a tool's in-flight work still
171
+ requires the tool itself to check `ToolExecutionContext.signal` (which the registry
172
+ aborts on timeout) — plain synchronous or non-abort-aware async code cannot be
173
+ force-killed from the same thread. See [ADR 0004](./docs/adr/0004-tool-execution-sandboxing.md)
174
+ for the full rationale and what a stronger guarantee would require.
175
+ - **`maxConcurrency`** bounds parallel execution instead of the previous unconditional
176
+ `Promise.all`, so a model requesting dozens of simultaneous tool calls can't exhaust
177
+ connection pools, rate limits, or memory all at once.
178
+ - **`maxOutputChars`** truncates `outputString` (what gets fed back into the
179
+ conversation) while leaving the untruncated value on `result.result` for callers who
180
+ need it — bounding how much a single tool call can inflate context size or memory.
181
+ - Zod's `safeParse` already validates every tool call's arguments against its schema
182
+ before `execute` runs (`OllamaToolValidationError` on mismatch). By default, Zod
183
+ objects silently strip unrecognized keys rather than rejecting them; call `.strict()`
184
+ on a tool's schema if you need to reject unexpected extra arguments outright.
185
+
186
+ ### Web Standard Streams & Next.js Integration
187
+
188
+ ```typescript
189
+ import { toResponse } from '@nemesis-oss/ollama-sdk';
190
+
191
+ export async function POST(req: Request) {
192
+ const { messages } = await req.json();
193
+ const stream = await client.chatStream({
194
+ model: 'qwen3.5:2b',
195
+ messages,
196
+ });
197
+
198
+ return toResponse(stream);
199
+ }
200
+ ```
201
+
202
+ ### OpenAI & Anthropic Compatibility Bridges
203
+
204
+ ```typescript
205
+ // OpenAI compatibility endpoint (/v1/chat/completions)
206
+ const openAIRes = await client.openai.chatCompletions({
207
+ model: 'llama3.2',
208
+ messages: [{ role: 'user', content: 'Hello via OpenAI bridge' }],
209
+ });
210
+
211
+ // Anthropic compatibility endpoint (/v1/messages)
212
+ const anthropicRes = await client.anthropic.messages({
213
+ model: 'llama3.2',
214
+ messages: [{ role: 'user', content: 'Hello via Anthropic bridge' }],
215
+ });
216
+ ```
217
+
218
+ ### Multi-Endpoint High Availability Failover
219
+
220
+ ```typescript
221
+ const client = new OllamaClient({
222
+ endpoints: [
223
+ { name: 'local-gpu', baseUrl: 'http://localhost:11434', priority: 10 },
224
+ {
225
+ name: 'cloud-replica',
226
+ baseUrl: 'https://ollama.internal.net',
227
+ apiKey: 'secret',
228
+ priority: 5,
229
+ },
230
+ ],
231
+ timeoutMs: 30_000,
232
+ retries: 3,
233
+ });
234
+
235
+ // Active health check probe
236
+ const health = await client.healthCheck();
237
+ console.log(health);
238
+ ```
239
+
240
+ Failover applies to inference calls (`chat`, `generate`, `embed`, `embeddings`,
241
+ `webSearch`, `webFetch`) — a different endpoint serving the same model is a genuine
242
+ substitute for those. Model/blob management (`listModels`, `pullModel`, `deleteModel`,
243
+ etc.) and `capabilities()` target one specific endpoint's local state and deliberately do
244
+ **not** fail over to a different candidate: retrying `deleteModel` against a different
245
+ server doesn't retry the same operation, it silently acts on a different model catalog.
246
+ See [ADR 0008](./docs/adr/0008-endpoint-failover-scope.md).
247
+
248
+ ---
249
+
250
+ ### Observability with OpenTelemetry
251
+
252
+ The client automatically emits [OpenTelemetry](https://opentelemetry.io/) spans for HTTP
253
+ requests, endpoint failover attempts, `chat`/`generate` calls (using the
254
+ [Gen AI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/)), and
255
+ `Agent` runs (`invoke_agent` → `ollama.agent.turn` → `execute_tool`) — no client
256
+ configuration required. `@opentelemetry/api` is an **optional peer dependency**: if it
257
+ isn't installed, or if your process hasn't registered a `TracerProvider`, tracing is a
258
+ no-op and costs nothing beyond a single cached import attempt.
259
+
260
+ ```bash
261
+ npm install @opentelemetry/api @opentelemetry/sdk-node @opentelemetry/auto-instrumentations-node
262
+ ```
263
+
264
+ ```typescript
265
+ // instrumentation.ts — run before importing the rest of your app
266
+ import { NodeSDK } from '@opentelemetry/sdk-node';
267
+ import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
268
+
269
+ const sdk = new NodeSDK({
270
+ instrumentations: [getNodeAutoInstrumentations()],
271
+ });
272
+ sdk.start();
273
+ ```
274
+
275
+ Once a `TracerProvider` is registered, every `OllamaClient`/`Agent` call in your process
276
+ produces spans automatically. See [ADR 0005](./docs/adr/0005-opentelemetry-instrumentation.md)
277
+ for exactly which spans and attributes are emitted, and the tradeoffs behind that design.
278
+
279
+ ---
280
+
281
+ ### Edge Runtime Compatibility
282
+
283
+ The core client (`OllamaClient`, `Agent`, `ToolRegistry`, and everything exported from
284
+ the package root) is built entirely on native `fetch` and Web Streams, so it runs
285
+ unmodified on Cloudflare Workers, Vercel Edge Runtime, and Next.js Edge middleware/route
286
+ handlers — no Node.js APIs required. The only Node-specific code (`SkillRegistry`, which
287
+ reads `SKILL.md` files from disk) lives behind the separate `@nemesis-oss/ollama-sdk/skills`
288
+ subpath export and is never pulled into the main bundle.
289
+
290
+ This is enforced in CI, not just asserted: `npm run verify:edge-runtime` bundles
291
+ `dist/index.js` with `esbuild` targeting a browser/edge platform (which hard-fails on any
292
+ `node:*` import, the same way Cloudflare's and Vercel's own bundlers do) and then runs a
293
+ full `OllamaClient` + `Agent` + tool-calling round trip inside `@edge-runtime/vm` — a
294
+ real Edge Runtime sandbox exposing only Web Standard globals. See
295
+ [ADR 0006](./docs/adr/0006-edge-runtime-ci-and-benchmarks.md) for the full rationale.
296
+
297
+ ---
298
+
299
+ ## Error Handling
300
+
301
+ Every failure thrown by the client is an `OllamaClientError` subclass, so you can catch the base
302
+ class or narrow to a specific `code`:
303
+
304
+ | Class | `code` | `retryable` | Thrown when |
305
+ | ---------------------------------- | ------------------------------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
306
+ | `OllamaNetworkError` | `network_error` | `true` | The request failed before a response was received (DNS, connection refused, etc). |
307
+ | `OllamaTimeoutError` | `timeout` | `true` | The request exceeded `timeoutMs`. |
308
+ | `OllamaAuthError` | `auth_error` | `false` | The endpoint returned `401`/`403`. |
309
+ | `OllamaNotFoundError` | `not_found` | `false` | The endpoint returned `404` (e.g. unknown model). |
310
+ | `OllamaRateLimitError` | `rate_limited` | `true` | The endpoint returned `429`. |
311
+ | `OllamaServerError` | `server_error` | `true` | The endpoint returned `5xx`. |
312
+ | `OllamaAbortError` | `aborted` | `false` | The request was cancelled via `AbortSignal`. |
313
+ | `OllamaToolValidationError` | `tool_validation_error` | `false` | A tool call's arguments, or a `chatWithSchema`/`generateWithSchema` result, failed Zod validation. |
314
+ | `OllamaUnsupportedCapabilityError` | `unsupported_capability` | `false` | A `format` (structured output) request was made against an endpoint inferred as Ollama Cloud, which doesn't currently support it. Thrown before any network call; in `DEFAULT_FAILOVER_CODES`, so a multi-endpoint setup tries the next candidate first. |
315
+ | `OllamaAgentMaxIterationsError` | `agent_max_iterations_exceeded` | `false` | An `Agent` run exceeded `maxTurns` without producing a final answer. |
316
+ | `OllamaMcpError` | `mcp_error` | varies | An MCP `listTools`/`callTool` call failed. |
317
+ | `OllamaSkillNotFoundError` | `skill_not_found` | `false` | `applySkill` referenced a skill that isn't registered. |
318
+ | `OllamaSkillInvalidError` | `skill_invalid` | `false` | A skill's frontmatter or contents failed to parse. |
319
+ | `OllamaGenericClientError` | `client_error` | `false` | Any other non-2xx response not covered above. |
320
+
321
+ All subclasses carry `status`, `retryable`, and optional `request`/`response` context, and preserve
322
+ the original error via the standard `cause` property:
323
+
324
+ ```typescript
325
+ import { OllamaClientError, OllamaRateLimitError } from '@nemesis-oss/ollama-sdk';
326
+
327
+ try {
328
+ await client.chatText({ model: 'qwen3.5:2b', messages: [...] });
329
+ } catch (err) {
330
+ if (err instanceof OllamaRateLimitError) {
331
+ console.warn(`Rate limited, retry after ${err.retryAfterMs}ms`);
332
+ } else if (err instanceof OllamaClientError) {
333
+ console.error(`[${err.code}] ${err.message}`, { retryable: err.retryable, cause: err.cause });
334
+ } else {
335
+ throw err;
336
+ }
337
+ }
338
+ ```
339
+
340
+ Multi-endpoint failover (`endpoints: [...]`) fails open rather than throwing a dedicated
341
+ "circuit open" error: once an endpoint's failure count crosses `failureThreshold`, it's skipped in
342
+ favor of healthy endpoints for `cooldownMs`, and only used again — sorted soonest-to-recover — if
343
+ every endpoint is cooling down. Call `client.healthCheck()` or inspect the registry's `status()` to
344
+ observe per-endpoint circuit state directly.
345
+
346
+ ---
347
+
348
+ ## Testing
349
+
350
+ The test suite contains 105 automated tests across 4 testing tiers:
351
+
352
+ ```bash
353
+ # Run unit, integration, and functional test suite
354
+ npm test
355
+
356
+ # Run typechecker
357
+ npm run typecheck
358
+
359
+ # Run linter
360
+ npm run lint
361
+
362
+ # Verify the built package runs correctly in a real Edge Runtime sandbox with zero
363
+ # Node.js APIs (see "Edge Runtime Compatibility" above) — requires `npm run build` first
364
+ npm run verify:edge-runtime
365
+
366
+ # Run the benchmark suite (NDJSON streaming, schema conversion, tool dispatch, the
367
+ # request pipeline)
368
+ npm run bench
369
+
370
+ # Run full CI verification pipeline (typecheck, lint, test, build, edge runtime check)
371
+ npm run verify
372
+ ```
373
+
374
+ ---
375
+
376
+ ## License
377
+
378
+ MIT © [Shubham Taywade](https://github.com/shubhamtaywade82)