@billjr99/pi-openai-compat 1.1.11 → 1.1.13

package/README.md

@@ -49,30 +49,40 @@ If pi is already running when you install, type `/reload` first.
 | **Venice AI** | `https://api.venice.ai/api/v1` | API key from venice.ai/settings/api |
 | **Fireworks AI** | `https://api.fireworks.ai/inference/v1` | API key from fireworks.ai/account/api-keys |
 | **Together AI** | `https://api.together.xyz/v1` | API key from api.together.ai/settings/api-keys |
-| **Cloudflare Workers AI** | `https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1` | API token from dash.cloudflare.com |
-| **Cloudflare AI Gateway** | `https://gateway.ai.cloudflare.com/v1/{account}/{gateway}/openai` | API token from dash.cloudflare.com |
+| **GitHub Models** | `https://models.github.ai/inference` | GitHub PAT — fine-grained, `Models → read` |
+| **Cloudflare Workers AI** | `https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1` | API token from dash.cloudflare.com (`Workers AI: Read`) |
+| **Cloudflare AI Gateway** | `https://gateway.ai.cloudflare.com/v1/{account}/{gateway}/openai` | API token from dash.cloudflare.com (`AI Gateway: Run` + `Workers AI: Read`) |
 | **Vercel AI Gateway** | `https://ai-gateway.vercel.sh/v1` | API key from vercel.com |
 | **OpenCode Zen** | `https://opencode.ai/zen/v1` | API key from opencode.ai |
 | **Ollama (local)** | `http://localhost:11434/v1` | Keyless |
 | **Ollama Cloud** | `https://ollama.com/v1` | Ollama Cloud API key from ollama.com |
 | **Custom** | Any URL you supply | Optional bearer token |
 
-> **Providers with built-in fallback model lists (as of May 2026)**
-> Some providers do not support `GET /v1/models` or return a non-JSON response.
-> For these, the extension substitutes a built-in list and shows a warning during
-> `/compat-login`.
+> **Providers whose model catalog lives at a non-standard `/models` path (as of June 2026)**
+> Some providers don't return models at `<base_url>/models`. The extension
+> handles them in one of two ways:
 >
-> | Provider | Reason |
-> |---|---|
-> | **Cloudflare Workers AI** | Returns HTTP 405 — method not supported |
-> | **Cloudflare AI Gateway** | Returns HTTP 401 — no anonymous model enumeration |
-> | **Hugging Face** | Returns HTML rather than JSON for `/v1/models` |
+> 1. **Live discovery via a per-template URL override** (`modelsUrl`,
+>    `modelsIdField`, `modelsKeepTask` on the template). Used for **GitHub
+>    Models** and **Cloudflare Workers AI** — `/compat-login` fetches the real
+>    catalog and registers actual current models, not a curated guess.
+> 2. **Built-in fallback list** when no working catalog endpoint exists. Used
+>    when discovery fails or when the upstream simply has no `/models` at all
+>    (e.g. **Hugging Face** returns HTML, **Cloudflare AI Gateway** has no
+>    catalog endpoint).
+>
+> | Provider | Default `/models` symptom | Handling |
+> |---|---|---|
+> | **GitHub Models** | HTTP 404 at `<base_url>/models` (catalog is at `/catalog/models`, not `/inference/models`) | Live discovery — `modelsUrl: https://models.github.ai/catalog/models` |
+> | **Cloudflare Workers AI** | HTTP 405 (no `GET /v1/models`; real catalog at `/ai/models/search`, id field is `name`, mixed task types) | Live discovery — `modelsUrl: …/ai/models/search`, `modelsIdField: name`, `modelsKeepTask: "Text Generation"` |
+> | **Cloudflare AI Gateway** | HTTP 401 (token missing `AI Gateway: Run`) or HTTP 400 *"Please configure AI Gateway"* (gateway slug doesn't exist, or the upstream isn't configured on it) | Built-in fallback list |
+> | **Hugging Face** | Returns HTML rather than JSON | Built-in fallback list |
 
 ---
 
 ## Commands
 
-Only two commands are needed.
+Three commands are available; `/compat-login` is the only one you need to get started.
 
 ### `/compat-login`
 
@@ -88,6 +98,18 @@ After login, the provider's models appear in pi's `/model` command and
 `Ctrl+L` picker immediately.  You can run `/compat-login` again to add a
 second provider — all providers are active simultaneously.
 
+### `/compat-refresh`
+
+Re-fetches the model list for an already-registered provider, reusing the
+saved base URL, API key, and discovery settings — no need to re-enter URLs,
+keys, or account IDs. Use it to pick up models a provider has added (or
+dropped) since you logged in, without restarting your session.
+
+If you have multiple providers registered, you are asked which one to refresh
+(or choose **All providers**); with a single provider it refreshes directly.
+A failed or empty refresh leaves the existing model list untouched, so a flaky
+network call can't blank out a working provider.
+
 ### `/compat-logout`
 
 Unregisters a provider from pi.  If you have multiple providers registered,
@@ -272,6 +294,19 @@ pi remove  npm:@billjr99/pi-openai-compat          # uninstall
 Verify the base URL does not have a trailing slash and ends with `/v1`.
 Confirm the API key is correct and has model-access permissions.
 For Ollama: ensure `ollama serve` is running.
+For self-hosted Open WebUI: the OpenAI-compatible API is under `/api/v1`, not
+the SPA root — e.g. `https://your-host/api/v1`, not `https://your-host`.
+Hitting the root returns the HTML SPA and `/compat-login` fails with a JSON
+parse error.
+
+**Cloudflare AI Gateway returns 401 Unauthorized**
+Check both: (1) the API token has `AI Gateway: Run` *and* `Workers AI: Read`
+under "Permissions", scoped to the correct account; and (2) the gateway slug
+in the URL actually exists under that account — list with
+`curl https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai-gateway/gateways
+-H "Authorization: Bearer $TOKEN"`. An empty `result` array means the gateway
+isn't there (you may need to create it in **dash.cloudflare.com → AI → AI
+Gateway**, or you're querying the wrong account).
 
 **No models appear after login**
 For Ollama: pull at least one model first (`ollama pull llama3`).

package/index.ts

@@ -8,8 +8,9 @@
  *   - Every provider saved in config.json is registered automatically at
  *     startup.  The factory is async so pi waits for registration to complete
  *     before showing the model list — no session_start delay.
- *   - /compat-login  adds a provider (fetches fresh model list, registers).
- *   - /compat-logout removes a provider (unregisters, restores previous model).
+ *   - /compat-login   adds a provider (fetches fresh model list, registers).
+ *   - /compat-refresh re-fetches the model list for a registered provider.
+ *   - /compat-logout  removes a provider (unregisters, restores previous model).
  *   - No activeProviders list — presence in config.providers means registered.
  *
  * Config: ~/.config/pi-openai-compat/config.json
@@ -35,6 +36,14 @@ interface ProviderConfig {
   baseUrl: string;
   apiKey: string | null;
   cachedModels: CachedModel[];
+  // Optional model-discovery overrides for providers whose /models endpoint
+  // lives at a non-standard path / shape (e.g. GitHub Models' /catalog/models,
+  // Cloudflare Workers AI's /ai/models/search). Persisted on the provider so
+  // session_start re-fetches (when cachedModels is empty) use the override
+  // rather than the broken default <baseUrl>/models path.
+  modelsUrl?: string;
+  modelsIdField?: string;
+  modelsKeepTask?: string;
 }
 
 interface ExtensionConfig {
@@ -48,6 +57,15 @@ interface OpenAIModelsResponse {
   data: Array<{ id: string; context_window?: number; max_tokens?: number }>;
 }
 
+/** Loose shape for a single entry in any /models response. */
+type RawModel = {
+  id?: string;
+  name?: string;
+  context_window?: number;
+  max_tokens?: number;
+  task?: { name?: string };
+};
+
 // ─────────────────────────────────────────────────────────────────────────────
 // Provider templates
 // ─────────────────────────────────────────────────────────────────────────────
@@ -64,6 +82,17 @@ const TEMPLATES: Record<string, {
   fallbackModels?: string[];
   /** Where to obtain the API key; presence implies the key is required. */
   keyHint?: string;
+  /**
+   * Optional model-discovery overrides for providers whose /models endpoint
+   * lives at a non-standard path / shape. May contain the same placeholders
+   * as baseUrl (YOUR_ACCOUNT_ID, YOUR_GATEWAY_SLUG, YOUR_PROVIDER); they are
+   * substituted alongside the baseUrl substitution in /compat-login.
+   */
+  modelsUrl?: string;
+  /** Field on each model entry that carries the upstream id (default "id"). */
+  modelsIdField?: string;
+  /** Keep only models whose task.name matches this string (case-insensitive). */
+  modelsKeepTask?: string;
 }> = {
   openrouter: {
     displayName: "OpenRouter",
@@ -97,9 +126,13 @@ const TEMPLATES: Record<string, {
   },
   github_models: {
     displayName: "GitHub Models",
-    baseUrl: "https://models.inference.ai.azure.com",
+    // The legacy Azure endpoint (models.inference.ai.azure.com) was retired.
+    // Chat now lives under /inference, and the catalog under /catalog/models
+    // on the same host — different path entirely, hence the modelsUrl override.
+    baseUrl: "https://models.github.ai/inference",
     keyless: false,
-    keyHint: "github.com/settings/tokens",
+    keyHint: "github.com/settings/tokens (fine-grained: Models → read)",
+    modelsUrl: "https://models.github.ai/catalog/models",
   },
   sambanova: {
     displayName: "SambaNova",
@@ -124,8 +157,14 @@ const TEMPLATES: Record<string, {
     baseUrl: "https://api.cloudflare.com/client/v4/accounts/YOUR_ACCOUNT_ID/ai/v1",
     keyless: false,
     promptUrl: true,
-    keyHint: "dash.cloudflare.com → My Profile → API Tokens",
-    // Cloudflare Workers AI returns 405 for GET /v1/models; use a curated list.
+    keyHint: "dash.cloudflare.com → My Profile → API Tokens (`Workers AI: Read`)",
+    // The OpenAI-compat base /ai/v1 returns 405 for GET /models. The real
+    // catalog lives at /ai/models/search, keys ids in "name" (reserving "id"
+    // for an internal UUID), and mixes Text Generation with embeddings and
+    // image tasks — so we filter by task.name.
+    modelsUrl: "https://api.cloudflare.com/client/v4/accounts/YOUR_ACCOUNT_ID/ai/models/search?per_page=100",
+    modelsIdField: "name",
+    modelsKeepTask: "Text Generation",
     fallbackModels: [
       "@cf/meta/llama-4-scout-17b-16e-instruct",
       "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
@@ -138,10 +177,16 @@ const TEMPLATES: Record<string, {
     displayName: "Cloudflare AI Gateway",
     // YOUR_PROVIDER is the upstream slug (e.g. "workers-ai", "openai"). /v1 is
     // appended so that fetchModels and chat completions hit the correct path.
+    // Common failure modes:
+    //   - 401 "Unauthorized": the token lacks `AI Gateway: Run` permission,
+    //     or the gateway has Authenticated Gateway enabled (requires a
+    //     separate cf-aig-authorization header, not yet supported here).
+    //   - 400 "Please configure AI Gateway": the gateway slug doesn't exist
+    //     under this account, or the upstream provider isn't configured on it.
     baseUrl: "https://gateway.ai.cloudflare.com/v1/YOUR_ACCOUNT_ID/YOUR_GATEWAY_SLUG/YOUR_PROVIDER/v1",
     keyless: false,
     promptUrl: true,
-    keyHint: "dash.cloudflare.com → My Profile → API Tokens",
+    keyHint: "dash.cloudflare.com → My Profile → API Tokens (`AI Gateway: Run` + `Workers AI: Read`)",
     fallbackModels: [
       "@cf/meta/llama-4-scout-17b-16e-instruct",
       "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
@@ -323,22 +368,69 @@ function isLocalUrl(url: string): boolean {
   }
 }
 
-async function fetchModels(baseUrl: string, apiKey: string | null): Promise<CachedModel[]> {
-  const url = `${baseUrl.replace(/\/+$/, "")}/models`;
+/** Optional per-provider overrides controlling how /models is fetched. */
+interface FetchOverrides {
+  /** Full URL to fetch instead of `<baseUrl>/models`. */
+  url?: string;
+  /** Field on each entry that holds the upstream id (default `id`). */
+  idField?: string;
+  /** Keep only entries whose `task.name` matches (case-insensitive). */
+  keepTask?: string;
+}
+
+async function fetchModels(
+  baseUrl: string,
+  apiKey: string | null,
+  overrides: FetchOverrides = {},
+): Promise<CachedModel[]> {
+  // Honor a per-provider override (e.g. GitHub Models' /catalog/models lives
+  // on a different path than its inference endpoint; Cloudflare Workers AI's
+  // catalog is at /ai/models/search). Fall back to <baseUrl>/models otherwise.
+  const url = overrides.url ?? `${baseUrl.replace(/\/+$/, "")}/models`;
+  const idField = overrides.idField ?? "id";
+  const keepTask = overrides.keepTask;
+
   const headers: Record<string, string> = { Accept: "application/json" };
   if (apiKey) headers["Authorization"] = `Bearer ${apiKey}`;
 
   const resp = await fetch(url, { headers });
   if (!resp.ok) {
+    // The Authorization header is never echoed here, so the error body is
+    // safe to surface even though we include the upstream's full response.
     const body = await resp.text().catch(() => "");
     throw new Error(`HTTP ${resp.status} from ${url}: ${body}`);
   }
 
-  const json = (await resp.json()) as OpenAIModelsResponse;
-  if (!Array.isArray(json?.data)) throw new Error(`Unexpected response from ${url}`);
+  // Normalize the various shapes /models can return:
+  //   - OpenAI style:                  {"data": [...]}
+  //   - Cloudflare / some gateways:    {"result": [...]}
+  //   - Together (and a few others):   [...]   (bare JSON array)
+  const json = (await resp.json()) as unknown;
+  let raw: RawModel[] | undefined;
+  if (Array.isArray(json)) {
+    raw = json as RawModel[];
+  } else if (json && typeof json === "object") {
+    const obj = json as { data?: unknown; result?: unknown };
+    if (Array.isArray(obj.data)) raw = obj.data as RawModel[];
+    else if (Array.isArray(obj.result)) raw = obj.result as RawModel[];
+  }
+  if (!raw) {
+    throw new Error(
+      `Unexpected /models payload shape from ${url} ` +
+      `(expected an array or an object with a "data" or "result" array).`
+    );
+  }
 
-  return json.data
-    .map((m) => ({ id: m.id, contextWindow: m.context_window, maxTokens: m.max_tokens }))
+  return raw
+    .filter((m) => {
+      if (keepTask === undefined) return true;
+      const taskName = m.task?.name ?? "";
+      return taskName.toLowerCase() === keepTask.toLowerCase();
+    })
+    .map((m) => {
+      const id = (m as Record<string, unknown>)[idField] as string | undefined;
+      return { id: id ?? "", contextWindow: m.context_window, maxTokens: m.max_tokens };
+    })
     .filter((m) => Boolean(m.id))
     .sort((a, b) => a.id.localeCompare(b.id));
 }
@@ -404,9 +496,14 @@ export default async function (pi: ExtensionAPI) {
         registerProvider(pi, key, p);
         registered.push(p.displayName);
       } else {
-        // Cache is empty (e.g. migrated from older config).  Try a live fetch.
+        // Cache is empty (e.g. migrated from older config).  Try a live fetch,
+        // honoring any per-provider discovery overrides stored on the config.
         try {
-          const models = await fetchModels(p.baseUrl, p.apiKey);
+          const models = await fetchModels(p.baseUrl, p.apiKey, {
+            url: p.modelsUrl,
+            idField: p.modelsIdField,
+            keepTask: p.modelsKeepTask,
+          });
           if (models.length > 0) {
             p.cachedModels = models;
             saveConfig(config);
@@ -456,6 +553,11 @@ export default async function (pi: ExtensionAPI) {
 
       // Step 2 — base URL
       let baseUrl = tpl.baseUrl;
+      // modelsUrl tracks the discovery URL through the same placeholder
+      // substitutions baseUrl goes through, so providers that put the model
+      // catalog on a different path (GitHub Models, Cloudflare Workers AI)
+      // get a fully-resolved URL by the time we call fetchModels.
+      let modelsUrl: string | undefined = tpl.modelsUrl;
       if (key === "cloudflare_workers") {
         const entered = await ctx.ui.input(
           "Account ID",
@@ -466,6 +568,7 @@ export default async function (pi: ExtensionAPI) {
         const accountId = entered.trim();
         if (!accountId) { ctx.ui.notify("Account ID cannot be empty.", "error"); return; }
         baseUrl = tpl.baseUrl.replace("YOUR_ACCOUNT_ID", accountId);
+        if (modelsUrl) modelsUrl = modelsUrl.replace("YOUR_ACCOUNT_ID", accountId);
       } else if (key === "cloudflare_ai_gateway") {
         const accountIdInput = await ctx.ui.input(
           "Account ID",
@@ -497,6 +600,12 @@ export default async function (pi: ExtensionAPI) {
           .replace("YOUR_ACCOUNT_ID", accountId)
           .replace("YOUR_GATEWAY_SLUG", gatewaySlug)
           .replace("YOUR_PROVIDER", provider);
+        if (modelsUrl) {
+          modelsUrl = modelsUrl
+            .replace("YOUR_ACCOUNT_ID", accountId)
+            .replace("YOUR_GATEWAY_SLUG", gatewaySlug)
+            .replace("YOUR_PROVIDER", provider);
+        }
       } else if (tpl.promptUrl) {
         const defaultUrl = tpl.baseUrl;
         const prompt = isLocalUrl(defaultUrl)
@@ -523,7 +632,11 @@ export default async function (pi: ExtensionAPI) {
       ctx.ui.notify(`Connecting to ${baseUrl} …`, "info");
       let models: CachedModel[];
       try {
-        models = await fetchModels(baseUrl, apiKey);
+        models = await fetchModels(baseUrl, apiKey, {
+          url: modelsUrl,
+          idField: tpl.modelsIdField,
+          keepTask: tpl.modelsKeepTask,
+        });
       } catch (err) {
         if (tpl.fallbackModels && tpl.fallbackModels.length > 0) {
           ctx.ui.notify(
@@ -555,12 +668,18 @@ export default async function (pi: ExtensionAPI) {
         }
       }
 
-      // Step 5 — save to config and register with pi
+      // Step 5 — save to config and register with pi.
+      // The discovery overrides are persisted so session_start can re-fetch
+      // correctly when cachedModels is empty (without them, the rehydrate
+      // path would hit <baseUrl>/models and 404 for these providers).
       config.providers[key] = {
         displayName: tpl.displayName,
         baseUrl,
         apiKey,
         cachedModels: models,
+        modelsUrl,
+        modelsIdField: tpl.modelsIdField,
+        modelsKeepTask: tpl.modelsKeepTask,
       };
       saveConfig(config);
       registerProvider(pi, key, config.providers[key]);
@@ -572,6 +691,72 @@ export default async function (pi: ExtensionAPI) {
     },
   });
 
+  // ── /compat-refresh ──────────────────────────────────────────────────────────
+  // Force a re-fetch of an already-registered provider's model list, reusing
+  // the persisted baseUrl/apiKey and discovery overrides — no need to re-enter
+  // URLs, keys, or account IDs like /compat-login. Picks up newly added (or
+  // dropped) upstream models without restarting the session.
+  pi.registerCommand("compat-refresh", {
+    description: "Re-fetch the model list for a registered OpenAI-compatible provider",
+    handler: async (_args, ctx) => {
+      const providerKeys = Object.keys(config.providers);
+      if (!providerKeys.length) {
+        ctx.ui.notify("No compat providers are registered. Run /compat-login first.", "info");
+        return;
+      }
+
+      // Choose which provider(s) to refresh. With one provider, refresh it
+      // directly; otherwise offer each by name plus an "All providers" option.
+      let keys: string[];
+      if (providerKeys.length === 1) {
+        keys = providerKeys;
+      } else {
+        const ALL = "All providers";
+        const labels = [ALL, ...providerKeys.map((k) => config.providers[k].displayName)];
+        const chosen = await ctx.ui.select("Refresh which provider?", labels);
+        if (!chosen) { ctx.ui.notify("Cancelled.", "info"); return; }
+        keys = chosen === ALL ? providerKeys : [providerKeys[labels.indexOf(chosen) - 1]];
+      }
+
+      const refreshed: string[] = [];
+      const failed: string[] = [];
+      for (const key of keys) {
+        const p = config.providers[key];
+        ctx.ui.notify(`Refreshing ${p.displayName} …`, "info");
+        try {
+          const models = await fetchModels(p.baseUrl, p.apiKey, {
+            url: p.modelsUrl,
+            idField: p.modelsIdField,
+            keepTask: p.modelsKeepTask,
+          });
+          if (models.length > 0) {
+            // Only overwrite the cache on a successful, non-empty fetch — a
+            // flaky refresh must never blank out a working provider's models.
+            p.cachedModels = models;
+            saveConfig(config);
+            registerProvider(pi, key, p);
+            refreshed.push(`${p.displayName} (${models.length})`);
+          } else {
+            failed.push(p.displayName);
+          }
+        } catch {
+          failed.push(p.displayName);
+        }
+      }
+
+      if (refreshed.length > 0) {
+        ctx.ui.notify(`Refreshed: ${refreshed.join(", ")}.`, "success");
+      }
+      if (failed.length > 0) {
+        ctx.ui.notify(
+          `Could not refresh ${failed.join(", ")} — kept the existing model list. ` +
+          `Run /compat-login if the provider's URL or key changed.`,
+          "warning"
+        );
+      }
+    },
+  });
+
   // ── /compat-logout ─────────────────────────────────────────────────────────
   pi.registerCommand("compat-logout", {
     description: "Remove an OpenAI-compatible provider from pi's model list",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@billjr99/pi-openai-compat",
-  "version": "1.1.11",
+  "version": "1.1.13",
   "description": "pi-coding-agent extension: OpenAI-compatible endpoint support (OpenRouter, NVIDIA NIM, Nous Portal, Ollama, custom)",
   "author": "Bill Mongan <https://github.com/BillJr99>",
   "license": "MIT",