pi-ollama-cloud 0.9.0 → 0.11.0

package/CHANGELOG.md

@@ -2,6 +2,22 @@
 
 All notable changes to this project will be documented in this file.
 
+## [Unreleased]
+
+- Fix `/ollama-cloud-usage` and the usage status bar failing with "unexpected response shape" after the undocumented `/api/usage` endpoint flipped between a single `limits.monthly` bucket and `limits.session` plus `limits.weekly` (the shape has flip-flopped repeatedly as of 2026-09). Any bucket present (`monthly`, `session`, `weekly`) is accepted alone or in combination, and whichever are present are displayed as `5h`/`7d`/`30d` segments. Thanks @johanngyger (#56).
+- Cache `ollama_web_search` results (24h) and `ollama_web_fetch` pages (24h success / 15 min failure) on disk under the pi agent home, so repeated queries and page reads cost 0 API calls. Expired entries are pruned on write, the cache is capped at 500 entries per kind (oldest evicted beyond the cap; `PI_OLLAMA_SEARCH_MAX_ENTRIES`), a partially corrupted cache file is validated per entry and degrades to "no cache" instead of crashing tool calls, and the file is written with `0600` permissions since it stores page content and URLs that can embed credentials. Tune with `PI_OLLAMA_SEARCH_TTL_HOURS`, `PI_OLLAMA_SEARCH_FAIL_TTL_MINUTES`, `PI_OLLAMA_SEARCH_MAX_ENTRIES`, and `PI_OLLAMA_SEARCH_CACHE_PATH`.
+- Bound web tool context usage: search snippets truncate to 500 chars with `[truncated]`/`[complete]` markers, `expand=<index>` returns a truncated result's full content from the cached search (0 extra API calls), and `ollama_web_fetch` pages long pages in 3000-char chunks via `offset`/`full` with a `Continue:` hint for the next offset (`PI_OLLAMA_SEARCH_SNIPPET_CHARS`/`PI_OLLAMA_SEARCH_CHUNK_CHARS` to tune).
+- Add `refresh=true` to both web tools to bypass the cache (including a cached failure) and re-call the API; the fresh result replaces the cache entry.
+- Failed page fetches are negative-cached for 15 min with a diagnostic message (likely cause + next steps) instead of a bare error. Auth (401/403), rate-limit (429), transport (timeout/abort/network), server (5xx), and unexpected-response-shape failures are never cached — search reports transport errors as `transport error` instead of a confusing `status 0` — so retrying after a fixed key, an expired rate-limit window, or a transient server blip re-calls the API immediately.
+- Note: `ollama_web_fetch` tool results now carry `details: { title, totalChars, links }` (previously `{ title, content, links }`); paged content is read via the tool output text, not `details.content`.
+
+## [0.10.0] - 2026-09-03
+
+- **Breaking:** Adapt to the changed `/api/usage` response shape. The endpoint now returns a single `limits.monthly` bucket (replacing `limits.session` and `limits.weekly`) and adds an `activity.models` array. `UsageData`, `isUsageResponse`, `formatUsage`, and `formatUsageStatusColored` now read the monthly limit; the status bar shows a single `30d` segment instead of `5h`/`7d`.
+- Refreshed the generated catalog from the live API: added `deepseek-v4-pro:0813`, `glm-5.3`, and `glm-5.3-flash`; removed `deepseek-v4-flash:preview` and `deepseek-v4-pro`, which are no longer listed.
+- Source per-token pricing from the official model table on ollama.com/pricing instead of models.dev estimates, which no longer track Ollama's published rates (up to ~13x off per model). `scripts/generate-pricing.ts` now scrapes the pricing page (the table is server-rendered; no JSON endpoint exists) and matches catalog IDs to pricing rows by exact or `:tag`-family match, replacing the `OLLAMA_TO_MODELSDEV` mapping. Regenerated `pricing.generated.ts` with the official rates, including new models (`glm-5.3`, `glm-5.3-flash`, `deepseek-v4-pro:0813`). Fixes #51. Thanks @Hackbard (#52).
+- Probe per-model max output tokens against the live API via `scripts/generate-limits.ts` into `limits.generated.ts`, replacing the fixed 32768 default for known models (unprobed models still fall back to 32768). The probe timeout is 60s so slow first-token models are not dropped from the table. Thanks @f440 (#49).
+
 ## [0.9.0] - 2026-08-11
 
 - Add `/ollama-cloud-usage` command to show Ollama Cloud session (5h) and weekly (7d) usage limits, per-model request counts, and the 4-week activity cost, fetched from the undocumented `/api/usage` endpoint with the already-resolved API key.

package/README.md

@@ -12,7 +12,7 @@ Registers Ollama Cloud as a model provider with dynamically fetched models, and
 - **Automatic model refresh** - On startup, `/model` open, and `pi update --models`, pi calls the extension's `refreshModels` callback to fetch the latest models from the API and persists them through pi's own model store. No manual refresh command.
 - **`ollama_web_search` tool** - Search the web for real-time information using Ollama Cloud's `/api/web_search` endpoint. Returns titles, URLs, and content snippets.
 - **`ollama_web_fetch` tool** - Fetch and extract text content from a web page URL using Ollama Cloud's `/api/web_fetch` endpoint. Returns page title, content, and links.
-- **Estimated cost tracking** - Models are registered with estimated per-token costs sourced from [models.dev](https://models.dev) (the same catalog pi uses), so Pi's `/cost` shows comparable usage. Ollama Cloud is subscription-billed (Free, Pro, Max), so these are equivalent pay-as-you-go estimates, not actual charges. See [ollama.com/pricing](https://ollama.com/pricing) for plan details.
+- **Per-token cost tracking** - Models are registered with the official per-token prices from [ollama.com/pricing](https://ollama.com/pricing), so Pi's `/cost` shows comparable usage. Ollama Cloud is subscription-billed, so these are equivalent pay-as-you-go rates, not actual charges.
 
 ## Prerequisites
 
@@ -135,8 +135,14 @@ Model metadata is derived from the `/api/show` response:
 | `thinkingLevelMap` | [`thinking-levels.ts`](thinking-levels.ts) with 5 maps (DEFAULT, GPT_OSS, QWEN3, GLM_52, NO_OFF) based on API testing |
 | `input` | `["text", "image"]` if `capabilities` includes `"vision"`, else `["text"]` |
 | `contextWindow` | `model_info.*.context_length` (falls back to 128000) |
-| `maxTokens` | Fixed at 32768 |
-| `cost` | Estimated per-1M-token prices from [models.dev](https://models.dev), generated by `scripts/generate-pricing.ts` into `pricing.generated.ts`. Ollama Cloud is subscription-billed, so these are equivalent pay-as-you-go estimates, not actual charges. Unmapped models default to zero. Prices are pinned to the installed package version and only update on a new release, so newly added models register with zero cost until then. |
+| `maxTokens` | Probed per-model limits from [`limits.generated.ts`](limits.generated.ts), generated by `scripts/generate-limits.ts` (requires `OLLAMA_API_KEY`). Models without a probed limit fall back to 32768. |
+| `cost` | Official per-1M-token prices from the [ollama.com/pricing](https://ollama.com/pricing) model table, generated by `scripts/generate-pricing.ts` into `pricing.generated.ts`. Ollama Cloud is subscription-billed, so these are equivalent pay-as-you-go rates, not actual charges. Catalog IDs with no matching pricing row default to zero. Prices are pinned to the installed package version and only update on a new release, so newly added models register with zero cost until then. |
+
+The per-model max output token table (`limits.generated.ts`) is probed against the live API by `scripts/generate-limits.ts`, since `/api/show` does not expose the limit. It needs an API key: `OLLAMA_API_KEY=<key> npm run generate-limits`. Limits ship with the package, so regenerated values take effect on the next release.
+
+The API itself returns no cost data: completion responses report only token counts (`prompt_tokens`/`completion_tokens`/`total_tokens`, including the final usage chunk when streaming), and `/api/show` exposes no pricing fields. The prices above come from the static `/pricing` page table and are only as fresh as the last regeneration.
+
+Cache pricing is informational only: the `/pricing` page lists a "Cached input" column, but the completion API does not report cache token usage (there is no `prompt_tokens_details.cached_tokens` or equivalent in any response, verified against the live API in September 2026), so pi never sees cache hits and `/cost` estimates do not reflect them. `cacheWrite` is always zero because the pricing table has no cache-write column.
 
 ### Thinking level mapping
 
@@ -161,19 +167,60 @@ See [docs/think-experiment.md](docs/think-experiment.md) for the testing methodo
 
 Both tools use the same Ollama Cloud API key configured for the provider. No local Ollama server is needed.
 
+### Caching
+
+Both tools cache results on disk (under the pi agent home, `~/.pi/agent/cache/pi-ollama-cloud/cache.json`). A repeated search query or page fetch within the TTL is served from cache and costs 0 API calls:
+
+- Successful searches and pages: cached for 24h
+- Failed page fetches: negative-cached for 15 min, so retrying a dead page does not re-call the API. Auth (401/403), rate-limit (429), transport (timeouts, aborts, network errors), and server (5xx) failures are not cached — fixing the key, waiting out the limit, or a transient blip lets a retry through immediately
+- Expired entries are pruned on write and the cache is capped at 500 entries per kind (searches/pages), evicting the oldest first. This bounds entry count, not file size: full page and search content can still make `cache.json` large, and loading it parses the whole file
+- The cache file is written with `0600` permissions. It stores full page content and raw URLs, which can embed credentials in query strings — avoid fetching URLs that carry secrets in the query string, or set a custom `PI_OLLAMA_SEARCH_CACHE_PATH`
+- Concurrent pi processes share the cache file on a last-writer-wins basis (no cross-process locking): one process's save can drop another's fresh entries, at the cost of a redundant API call
+- `refresh=true` on either tool bypasses the cache (including a cached failure) and re-calls the API; the fresh result replaces the cache entry
+
+### `ollama_web_search`
+
+Returns up to 5 results by default (`max_results`, max 10; title, URL, 500-char snippet). Snippets are marked `[truncated]` when the source is longer than the snippet. Output ends with `# live query` or `# from cache` to show whether the API was called.
+
+The search API returns each result's full content; it is cached in full, so a truncated result can be expanded without a separate fetch:
+
+- `expand=<index>` — return the full content of that result (1-based) from the cached search, 0 extra API calls. The cache key includes `max_results`, so expanding hits the cache only when the query was searched with the same `max_results`; otherwise the search runs live first.
+- Use `ollama_web_fetch` only when the search result's content is not enough (e.g. you need a different page, or the search excerpt is shorter than the full page).
+
+### `ollama_web_fetch`
+
+Returns the page title, a 3000-char slice of the content, and links. Long pages are read in chunks to keep the context window small:
+
+- `offset=N` — continue reading from character N (the output tells you the next offset)
+- `full=true` — return all remaining content from `offset` in one call
+
+A failed fetch throws a diagnostic message (likely cause + next steps) instead of a bare error.
+
+### Tuning
+
+| Env var | Default | Meaning |
+|---|---|---|
+| `PI_OLLAMA_SEARCH_TTL_HOURS` | `24` | Success cache TTL |
+| `PI_OLLAMA_SEARCH_FAIL_TTL_MINUTES` | `15` | Failure (negative) cache TTL |
+| `PI_OLLAMA_SEARCH_CACHE_PATH` | `<pi agent home>/cache/pi-ollama-cloud/cache.json` | Cache file location |
+| `PI_OLLAMA_SEARCH_MAX_ENTRIES` | `500` | Max cached entries per kind (searches/pages); oldest evicted beyond the cap |
+| `PI_OLLAMA_SEARCH_SNIPPET_CHARS` | `500` | Search snippet length |
+| `PI_OLLAMA_SEARCH_CHUNK_CHARS` | `3000` | Fetch chunk size |
+
 ## Commands
 
 | Command | Description |
 |---|---|
 | `/ollama-webtools [on\|off\|enable\|disable]` | Enable or disable the `ollama_web_search` and `ollama_web_fetch` tools. Toggles if no argument given. |
-| `/ollama-cloud-usage` | Show Ollama Cloud session (5h) and weekly (7d) usage limits, per-model request counts, and the 4-week activity cost. |
+| `/ollama-cloud-usage` | Show Ollama Cloud usage limits (one section per limit bucket the API reports), per-model request counts, and the 4-week activity cost. |
 | `/ollama-usage-status [on\|off\|enable\|disable]` | Enable or disable the footer usage status bar. Toggles if no argument given. |
 
 ## Usage status bar
 
 While an `ollama-cloud` model is the active provider, the footer shows a compact
-live usage readout (`5h ▕███░░░░░░░▏ 34% 7d ▕████░░░░░░▏ 45%`) that refreshes
-every 5 minutes and after each agent turn (but no more often than every 5 minutes). Each segment is colored by how close
+live usage readout with one segment per limit bucket the API reports
+(`5h ▕███░░░░░░░▏ 34% 7d ▕█░░░░░░░░░▏ 7%`, or a single `30d` segment) that
+refreshes every 5 minutes and after each agent turn (but no more often than every 5 minutes). It is colored by how close
 it is to the cap: green below 60%, yellow at 60-79%, red at 80%+. It reads the
 same undocumented `/api/usage` endpoint as `/ollama-cloud-usage` and clears
 itself on transient errors or when you switch to a non-Ollama-Cloud provider.
@@ -227,6 +274,7 @@ npm run check        # lint + format + type-check (auto-fix)
 npm run lint         # lint only (no fixes)
 npm run typecheck    # type-check only (tsgo --noEmit)
 npm run format       # format only
+OLLAMA_API_KEY=<key> npm run generate-limits   # probe max output tokens (writes limits.generated.ts)
 ```
 
 The project uses [Biome](https://biomejs.dev/) for linting and formatting (2-space indent, line width 120) and [tsgo](https://github.com/microsoft/typescript-go) for type-checking.
@@ -295,7 +343,8 @@ git push --tags
 Because the model catalog refreshes automatically at runtime, a release is **not** needed to ship new models. Publish only when:
 
 - A model is retired and still listed by the API: add it to `RETIRED_MODEL_IDS` in `scripts/generate-models.ts` (check https://docs.ollama.com/cloud#retirements, then regenerate `models.generated.ts`).
-- Pricing changes: models.dev prices updated, or a new model needs an `OLLAMA_TO_MODELSDEV` mapping line (regenerate `pricing.generated.ts`).
+- Pricing changes: Ollama updates the model pricing table, or a new model needs a pricing row (regenerate `pricing.generated.ts`).
+- Max output token limits changed: run `OLLAMA_API_KEY=<key> npm run generate-limits` locally and commit.
 
 The tag version must match the version in `package.json` - `npm version` handles this automatically. The workflow at `.github/workflows/publish.yml` verifies the match before publishing to npm.
 

package/cache.ts

@@ -0,0 +1,181 @@
+import { createHash, randomBytes } from "node:crypto";
+import { chmodSync, mkdirSync, readFileSync, renameSync, rmSync, writeFileSync } from "node:fs";
+import { dirname, join } from "node:path";
+import { getAgentDir } from "@earendil-works/pi-coding-agent";
+import { envInt } from "./utils.ts";
+
+export const CACHE_PATH =
+  process.env.PI_OLLAMA_SEARCH_CACHE_PATH ?? join(getAgentDir(), "cache", "pi-ollama-cloud", "cache.json");
+export const CACHE_TTL_MS = envInt("PI_OLLAMA_SEARCH_TTL_HOURS", 24) * 60 * 60 * 1000;
+export const FAIL_TTL_MS = envInt("PI_OLLAMA_SEARCH_FAIL_TTL_MINUTES", 15) * 60 * 1000;
+/** Max entries per map (searches/pages); oldest-ts entries are evicted beyond this. */
+export const MAX_ENTRIES = envInt("PI_OLLAMA_SEARCH_MAX_ENTRIES", 500);
+
+export interface SearchResult {
+  title: string;
+  url: string;
+  content: string;
+}
+
+export interface SearchCacheEntry {
+  ts: number;
+  q: string;
+  results: SearchResult[];
+}
+
+export interface PageCacheEntry {
+  ts: number;
+  status?: number;
+  title?: string;
+  content?: string;
+  links?: string[] | null;
+  error?: string;
+  errorType?: "response-shape";
+}
+
+export interface CacheData {
+  searches: Record<string, SearchCacheEntry>;
+  pages: Record<string, PageCacheEntry>;
+}
+
+interface CacheOptions {
+  path: string;
+  ttlMs: number;
+  failTtlMs: number;
+  maxEntries: number;
+}
+
+function isRecord(v: unknown): v is Record<string, unknown> {
+  return typeof v === "object" && v !== null && !Array.isArray(v);
+}
+
+/** Keys that must never come from a parsed JSON file (prototype pollution). */
+const UNSAFE_KEYS = new Set(["__proto__", "constructor", "prototype"]);
+
+export function isSafeKey(key: string): boolean {
+  return !UNSAFE_KEYS.has(key);
+}
+
+/** Shallow shape checks so a partially corrupted cache file degrades instead of crashing tool calls. */
+function isSearchEntry(v: unknown): v is SearchCacheEntry {
+  return (
+    isRecord(v) &&
+    typeof v.ts === "number" &&
+    typeof v.q === "string" &&
+    Array.isArray(v.results) &&
+    v.results.every(
+      (r) => isRecord(r) && typeof r.title === "string" && typeof r.url === "string" && typeof r.content === "string",
+    )
+  );
+}
+
+function isPageEntry(v: unknown): v is PageCacheEntry {
+  if (!isRecord(v) || typeof v.ts !== "number") return false;
+  const fieldsValid =
+    (v.status === undefined || typeof v.status === "number") &&
+    (v.title === undefined || typeof v.title === "string") &&
+    (v.content === undefined || typeof v.content === "string") &&
+    (v.links === null ||
+      v.links === undefined ||
+      (Array.isArray(v.links) && v.links.every((l) => typeof l === "string"))) &&
+    (v.error === undefined || (typeof v.error === "string" && v.error !== "")) &&
+    (v.errorType === undefined || v.errorType === "response-shape");
+  if (!fieldsValid) return false;
+  // Must be either a real failure or a real success; anything else (e.g. an
+  // entry with neither content nor a non-empty error) would render as a fake
+  // empty success.
+  return (
+    (typeof v.error === "string" && v.error !== "") || (typeof v.title === "string" && typeof v.content === "string")
+  );
+}
+
+export interface CacheStore {
+  loadCache(): CacheData;
+  saveCache(): void;
+  isFresh(entry: { ts: number; error?: string } | undefined): boolean;
+}
+
+export function createCache(options: Partial<CacheOptions> = {}): CacheStore {
+  const path = options.path ?? CACHE_PATH;
+  const ttlMs = options.ttlMs ?? CACHE_TTL_MS;
+  const failTtlMs = options.failTtlMs ?? FAIL_TTL_MS;
+  const maxEntries = options.maxEntries ?? MAX_ENTRIES;
+  let cacheData: CacheData | null = null;
+
+  function loadCache(): CacheData {
+    if (cacheData) return cacheData;
+    try {
+      const raw: unknown = JSON.parse(readFileSync(path, "utf8"));
+      if (isRecord(raw) && isRecord(raw.searches) && isRecord(raw.pages)) {
+        // Per-entry validation: drop poisoned entries so a partially corrupt
+        // file degrades to "those entries are gone" instead of crashing calls.
+        cacheData = { searches: {}, pages: {} };
+        for (const [key, entry] of Object.entries(raw.searches)) {
+          if (isSafeKey(key) && isSearchEntry(entry)) cacheData.searches[key] = entry;
+        }
+        for (const [key, entry] of Object.entries(raw.pages)) {
+          if (isSafeKey(key) && isPageEntry(entry)) cacheData.pages[key] = entry;
+        }
+        return cacheData;
+      }
+    } catch {
+      // First run or corrupt file, start fresh.
+    }
+    cacheData = { searches: {}, pages: {} };
+    return cacheData;
+  }
+
+  function isFresh(entry: { ts: number; error?: string } | undefined): boolean {
+    if (!entry) return false;
+    // A future ts (hand-edited file) would otherwise be fresh forever; treat as stale.
+    if (entry.ts > Date.now()) return false;
+    return Date.now() - entry.ts < (entry.error ? failTtlMs : ttlMs);
+  }
+
+  function evictOldest(map: Record<string, { ts: number }>): void {
+    const keys = Object.keys(map);
+    if (keys.length <= maxEntries) return;
+    const overflow = keys.sort((a, b) => map[a].ts - map[b].ts).slice(0, keys.length - maxEntries);
+    for (const key of overflow) delete map[key];
+  }
+
+  function saveCache(): void {
+    const data = loadCache();
+    for (const [key, entry] of Object.entries(data.searches)) if (!isFresh(entry)) delete data.searches[key];
+    for (const [key, entry] of Object.entries(data.pages)) if (!isFresh(entry)) delete data.pages[key];
+    // TTL bounds entry age; the cap bounds entry count so an aggressive session
+    // cannot grow the file without limit.
+    evictOldest(data.searches);
+    evictOldest(data.pages);
+    try {
+      mkdirSync(dirname(path), { recursive: true, mode: 0o700 });
+      // Use a unique temporary path so concurrent processes cannot overwrite
+      // one another's in-progress writes.
+      const tmp = `${path}.${process.pid}.${randomBytes(8).toString("hex")}.tmp`;
+      try {
+        // 0o600: the cache stores full page content and URLs, which can embed
+        // credentials in query strings; it should not be world-readable.
+        writeFileSync(tmp, JSON.stringify(data), { mode: 0o600 });
+        renameSync(tmp, path);
+        // Fix perms of a file written by a pre-0600 version.
+        chmodSync(path, 0o600);
+      } catch (error) {
+        rmSync(tmp, { force: true });
+        throw error;
+      }
+    } catch {
+      // Cache is best-effort; a failed write must not break the tool call.
+    }
+  }
+
+  return { loadCache, saveCache, isFresh };
+}
+
+export const defaultCache = createCache();
+export const loadCache = defaultCache.loadCache;
+export const saveCache = defaultCache.saveCache;
+export const isFresh = defaultCache.isFresh;
+
+export function searchCacheKey(query: string, maxResults: number): string {
+  return createHash("sha1").update(`${query}\n${maxResults}`).digest("hex");
+}

package/index.ts

@@ -134,7 +134,7 @@ export default async function (pi: ExtensionAPI) {
   // --- Usage Command ---
 
   pi.registerCommand("ollama-cloud-usage", {
-    description: "Show Ollama Cloud session and weekly usage limits.",
+    description: "Show Ollama Cloud usage limits.",
     handler: async (_args, ctx) => {
       const apiKey = await getCloudApiKey(ctx);
       if (!apiKey) {
@@ -152,7 +152,7 @@ export default async function (pi: ExtensionAPI) {
 
   // --- Usage Status Bar ---
 
-  // Footer status showing live session/weekly usage while ollama-cloud is the
+  // Footer status showing live usage while ollama-cloud is the
   // active provider. Refreshes on a 5-minute timer; agent_end also triggers a
   // refresh but is throttled to the same cooldown so a turn never hammers the
   // undocumented /api/usage endpoint. The quota-bar concept is inspired by

package/limits.generated.ts

@@ -0,0 +1,25 @@
+// Auto-generated by scripts/generate-limits.ts
+// Do not edit manually.
+// Probed models: 19 (0 failed)
+
+export const MODEL_MAX_OUTPUT_TOKENS: Record<string, number> = {
+  "deepseek-v4-flash:0731": 65536,
+  "deepseek-v4-pro:0813": 65536,
+  "gemma4:31b": 262144,
+  "glm-5.1": 131072,
+  "glm-5.2": 131072,
+  "glm-5.3": 524288,
+  "glm-5.3-flash": 524288,
+  "gpt-oss:120b": 131072,
+  "gpt-oss:20b": 131072,
+  "kimi-k2.6": 262144,
+  "kimi-k2.7-code": 262144,
+  "kimi-k3": 524288,
+  "minimax-m2.7": 131072,
+  "minimax-m3": 131072,
+  "mistral-large-3:675b": 262144,
+  "nemotron-3-nano:30b": 131072,
+  "nemotron-3-super": 65536,
+  "nemotron-3-ultra": 65536,
+  "qwen3.5:397b": 65536,
+};