@nemesis-oss/ollama-sdk 1.12.0 → 1.12.1

package/CHANGELOG.md

@@ -1,5 +1,14 @@
 # Changelog
 
+## [1.12.1] — Fix: model-affinity routing respects priority tiers
+
+### Fixed
+
+- **`endpointHealth.modelAffinity` no longer overrides endpoint `priority`.** In 1.12.0, `ModelAffinityRouter.reorder()` moved every host holding the requested model to the front of the _whole_ candidate list, so a lower-priority host with the model in VRAM was tried before a higher-priority host without it. That contradicted the README and ADR 0028, which state that priority tiers keep working. Affinity now reorders only within each priority tier (runs of consecutive equal `priority`, unset = `0`, as in the registry): a higher-priority endpoint is always tried first, and residency only breaks ties inside a tier. The order between tiers, including the registry's fail-open recovery order, is unchanged.
+- **Behavior change for configurations that mix `priority` values with `modelAffinity`:** requests that 1.12.0 sent to a resident lower-priority host now go to the higher-priority host first. Pools where every endpoint has the same priority (the documented fleet-of-workstations setup) are unaffected.
+- 6 new tests: tier isolation, independent per-tier reordering, unset priority as `0`, consecutive-run grouping, an unchanged input returned as-is, and an end-to-end `OllamaClient` routing check.
+- `package-lock.json` root version corrected (it still read `1.8.0`).
+
 ## [1.12.0] — Daemon liveness & telemetry standardization: formatTelemetry, dynamic host model-affinity routing, root ping
 
 The October 10 documentation digest ("Daemon Liveness & Telemetry Standardization") reviewed the official operational surfaces — the root liveness probe (`HEAD /` → 200, body "Ollama is running"), `GET /api/version`, `GET /api/ps` runner introspection, and the nanosecond duration counters on every generation response. Verified against the codebase first (see [ADR 0028](./docs/adr/0028-telemetry-normalization-host-affinity-root-ping.md)): two of the digest's gap assessments were stale — multi-host pooling with least-connections routing, capacity caps, and 503/`OLLAMA_MAX_QUEUE` saturation failover already exist (`endpoints` + `endpointHealth` + `DEFAULT_FAILOVER_CODES`), so the proposed `LoadBalancedOllamaClient` was **not** added (it would fork the existing routing engine); the real gaps — telemetry math, dynamic multi-host affinity, and the root ping — are implemented, all additive, no breaking changes. 865 pre-existing tests stay green, 33 new tests added.

package/README.md

@@ -6,6 +6,16 @@
 
 > Production-grade TypeScript SDK for Ollama. Built with native fetch, high availability failover, multi-turn tool calling, structured outputs with Zod, reasoning stream tokens, OpenAI & Anthropic compatibility bridges, MCP integration, and Web Stream adapters.
 
+### How this compares
+
+| Need                                                                       | Typical choice                                                                        |
+| -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |
+| Default Ollama integration in JS/TS                                        | Official [`ollama`](https://www.npmjs.com/package/ollama) client (`ollama/ollama-js`) |
+| Vercel AI SDK (`ai` package) apps                                          | [`ai-sdk-ollama`](https://www.npmjs.com/package/ai-sdk-ollama)                        |
+| Agent runtime, MCP bridge, HA failover, context safety, structured outputs | **This package** — powers [Nexum](https://github.com/shubhamtaywade82/nexum)          |
+
+Nexum’s model harness (2026-10-09) exercised this SDK via `gemma4:cloud` and `minicpm5:2b` with 100% pass on `tool-calling`, `output-format`, and `agentic-looping` categories. See [nexum/docs/guide/benchmarks.md](../nexum/docs/guide/benchmarks.md) and [validation.md](../nexum/docs/guide/validation.md).
+
 ---
 
 ## Key Features
@@ -1174,7 +1184,7 @@ initial HTTP round trip:
 
 #### Model-affinity routing across hosts (prefer the resident host)
 
-With several *interchangeable* endpoints serving the same models — the fleet-of-workstations
+With several _interchangeable_ endpoints serving the same models — the fleet-of-workstations
 case, not the per-credential Cloud case — sending a request to a host that doesn't currently
 hold the model forces that host to evict something and cold-load it: the multi-host version
 of the thrashing `ModelAffinityScheduler` prevents inside one client. Opt in with
@@ -1217,7 +1227,8 @@ The design never trades latency for affinity (see
   from being re-probed on every request.
 - **No probes at all unless configured.** Without `modelAffinity`, the router isn't even
   constructed — zero overhead, and no `/api/ps` traffic.
-- **Composes with everything above.** Affinity reorders *within* the runnable set:
+- **Composes with everything above.** Affinity reorders _within each priority tier_ of
+  the runnable set (a higher-priority host is always tried first, resident or not):
   priority tiers, `least-connections` ordering, capacity caps, and `models`-scoped
   credentials keep working exactly as before, and single-endpoint-pinned operations
   (model/blob management, `capabilities()`) are never moved to a different host.
@@ -1225,7 +1236,7 @@ The design never trades latency for affinity (see
   `llama3:latest` but never `llama3.1`), shared with context-window discovery so the
   two features can't disagree about what "loaded" means.
 
-For the *single-host* counterpart — per-model serial queues that deepen the already-hot
+For the _single-host_ counterpart — per-model serial queues that deepen the already-hot
 model before loading the next one — see
 [Model-Affinity Scheduling](#model-affinity-scheduling-multi-model-anti-thrashing); the
 two compose: a scheduler per host, affinity routing between hosts.
@@ -1305,7 +1316,7 @@ Two behaviors worth knowing:
   servers without `prompt_eval_cached_count`, embeddings) format cleanly instead of
   producing `NaN`/`Infinity`, and zero durations never divide into rates.
 - **`cacheHitRatio` uses `cached / (cached + evaluated)`**, not
-  `cached / prompt_eval_count`: Ollama reports `prompt_eval_count: 0` on a *full* cache
+  `cached / prompt_eval_count`: Ollama reports `prompt_eval_count: 0` on a _full_ cache
   hit, so dividing by it alone would read a perfect cache as "no caching at all". This
   is the same semantics as [`ConversationSession`'s
   `cacheStats.hitRate`](#kv-cache-aware-conversation-sessions), so the two surfaces can
@@ -1315,7 +1326,7 @@ Two behaviors worth knowing:
 straight into log lines, dashboards, and streaming pipelines alike. For span-based
 instrumentation of requests themselves, see
 [Observability with OpenTelemetry](#observability-with-opentelemetry) below; the two
-compose (spans for *where/when*, `formatTelemetry` for *how fast*).
+compose (spans for _where/when_, `formatTelemetry` for _how fast_).
 
 ### Observability with OpenTelemetry
 

package/dist/index.cjs

@@ -1765,9 +1765,10 @@ var ModelAffinityRouter = class {
     this.now = options.now ?? Date.now;
   }
   /**
-   * Synchronously reorders `candidates` so endpoints whose fresh `/api/ps`
+   * Synchronously reorders `candidates` so that, within each priority tier
+   * (run of consecutive equal `priority`), endpoints whose fresh `/api/ps`
    * snapshot shows `model` resident come first (relative order preserved within
-   * each group). Returns the input unchanged — and kicks a background refresh —
+   * each group). Never moves an endpoint across tiers. Returns the input unchanged — and kicks a background refresh —
    * when any candidate's snapshot is missing or stale: a wrong reorder is worse
    * than no reorder, and the first request after a cache miss simply isn't
    * reordered. No-op without a `model` or with fewer than two candidates.
@@ -1784,13 +1785,28 @@ var ModelAffinityRouter = class {
       }
     }
     if (!allFresh) return candidates;
-    const resident = candidates.filter((ep) => {
+    const isResident = (ep) => {
       const entry = this.cache.get(ep.name);
       return entry !== void 0 && isModelResident(entry.models, model);
-    });
-    if (resident.length === 0 || resident.length === candidates.length) return candidates;
-    const residentSet = new Set(resident.map((ep) => ep.name));
-    return [...resident, ...candidates.filter((ep) => !residentSet.has(ep.name))];
+    };
+    const reordered = [];
+    let changed = false;
+    for (let start = 0; start < candidates.length; ) {
+      const tierPriority = candidates[start].priority ?? 0;
+      let end = start + 1;
+      while (end < candidates.length && (candidates[end].priority ?? 0) === tierPriority) end++;
+      const tier = candidates.slice(start, end);
+      const resident = tier.filter(isResident);
+      if (resident.length === 0 || resident.length === tier.length) {
+        reordered.push(...tier);
+      } else {
+        const tierOrder = [...resident, ...tier.filter((ep) => !isResident(ep))];
+        changed ||= tierOrder.some((ep, i) => ep !== tier[i]);
+        reordered.push(...tierOrder);
+      }
+      start = end;
+    }
+    return changed ? reordered : candidates;
   }
   /**
    * Forces a refresh of the affinity cache for the given endpoints (deduped with