@gapi/gcli 1.8.238 → 1.8.239

package/docs/changelog.d/2026-10-11-local-ai-runners-runner-reports-each-model-s-loaded-context-window.md

@@ -0,0 +1,22 @@
+## 2026-10-11 - local-ai-runners: runner reports each model's loaded context window
+
+**Feature:** local-ai-runners · **Type:** fix
+
+- Heartbeat models now carry `runtimeContextLength` from Ollama `/api/ps`
+  (the window a loaded model really runs with, e.g. 32,768 for qwen3.8:27b
+  whose max is 262,144). The backend fits every request to it - fixes
+  `500 no user query found in messages` in long chats, caused by Ollama
+  silently truncating oversized prompts (backend half: graphql-server
+  `fix(local-ai-runners): fit requests to the model's real context window`).
+- Models `/api/show` answers 404/410 for (a retired `glm-5.1:cloud` was
+  still listed by `/api/tags`) are no longer offered, and the answer is
+  cached instead of re-asked every heartbeat.
+- **Test:** `npx jest test/runner.spec.ts --coverage=false` (fake Ollama
+  answers `/api/ps` with a loaded model at 65536 and 410 for a retired
+  cloud model; the heartbeat must carry the window and omit that model).
+- **Debug:** `ollama ps` shows the live window the runner reports;
+  `db["local-runners"]` -> `models[].runtimeContextLength` on the backend.
+  An unloaded model reports nothing - the backend keeps its last value.
+- **Verification:** gcli 27/27, tsc + eslint clean; live: runner reported
+  30 models (retired cloud model gone) and the dev backend stored
+  qwen3.8:27b runtimeContextLength 32768.

package/docs/features/local-ai-runners.md

@@ -34,7 +34,14 @@ Ollama: `--ollama` > `$OLLAMA_HOST` (scheme optional, `0.0.0.0` -> `127.0.0.1`)
   sequentially. `RunnerAuthError` = token refused -> exit 1 with a `--reset` hint.
 - `ollama.ts` - `ALLOWED_REQUESTS` (only `POST /v1/chat/completions`; the
   backend has its own copy), `OllamaModels` (`/api/tags` + cached
-  `/api/show` per digest; drops models without the `completion` capability).
+  `/api/show` per digest; drops models without the `completion` capability
+  and models `/api/show` answers 404/410 for - e.g. retired `:cloud`
+  models, cached so they aren't re-asked every heartbeat). Each heartbeat
+  also reads `/api/ps` and reports `runtimeContextLength` - the window a
+  LOADED model actually runs with (OLLAMA_CONTEXT_LENGTH / Ollama default).
+  The backend fits every request to it, because Ollama silently drops the
+  oldest messages of a prompt that is too long (see the canonical doc,
+  "Context window").
 - `config.ts` (saved runner, URL derivation), `start.ts`, `list.ts`,
   `models.ts`, `delete.ts`, `index.ts` (command registration), `types.ts`.
 - `src/services/gql-client.ts` - `createLocalRunner`, `listLocalRunners`,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@gapi/gcli",
-  "version": "1.8.238",
+  "version": "1.8.239",
   "repository": {
     "type": "git",
     "url": "https://github.com/Stradivario/gapi.git"

package/release/index.js

@@ -317007,12 +317007,41 @@ var init_ollama = __esm({
           throw new Error(`Ollama /api/tags returned ${res.status}`);
         }
         const { models = [] } = await res.json();
-        const described = await Promise.all(
-          models.map((tag) => this.describe(tag))
-        );
-        return described.filter(
+        const [described, loaded] = await Promise.all([
+          Promise.all(models.map((tag) => this.describe(tag))),
+          this.loadedContextLengths()
+        ]);
+        for (const model of described) {
+          model.runtimeContextLength = loaded.get(model.name);
+        }
+        return described.filter((m2) => !m2.unusable).filter(
           (m2) => !m2.capabilities.length || m2.capabilities.includes("completion")
-        );
+        ).map((m2) => {
+          delete m2.unusable;
+          return m2;
+        });
+      }
+      /**
+       * The context window each LOADED model actually runs with (`/api/ps`,
+       * what `ollama ps` shows) - OLLAMA_CONTEXT_LENGTH / Ollama's VRAM-based
+       * default, usually far below the model's own `contextLength`. Ollama
+       * silently drops the oldest messages of a prompt that doesn't fit, so the
+       * backend sizes requests with this. Unloaded models aren't listed; the
+       * backend keeps the last known value.
+       */
+      async loadedContextLengths() {
+        try {
+          const res = await this.fetchFn(`${this.baseUrl}/api/ps`);
+          if (!res.ok) {
+            return /* @__PURE__ */ new Map();
+          }
+          const { models = [] } = await res.json();
+          return new Map(
+            models.filter((m2) => typeof m2.context_length === "number").map((m2) => [m2.name ?? m2.model, m2.context_length])
+          );
+        } catch {
+          return /* @__PURE__ */ new Map();
+        }
       }
       async describe(tag) {
         const show = await this.show(tag);
@@ -317020,6 +317049,7 @@ var init_ollama = __esm({
           ([key]) => key.endsWith(".context_length")
         )?.[1];
         return {
+          unusable: show?.unusable,
           name: tag.name,
           family: tag.details?.family,
           parameterSize: tag.details?.parameter_size,
@@ -317039,6 +317069,11 @@ var init_ollama = __esm({
             headers: { "Content-Type": "application/json" },
             body: JSON.stringify({ model: tag.name })
           });
+          if (res.status === 404 || res.status === 410) {
+            const gone = { unusable: true };
+            this.showCache.set(key, gone);
+            return gone;
+          }
           if (!res.ok) {
             return void 0;
           }