pi-llama-cpp 0.9.1 → 0.10.0

package/README.md

@@ -9,9 +9,9 @@ A [Pi Coding Agent](https://pi.dev/) extension that integrates with running [lla
 - **Load / unload / switch** — manage models directly from the Pi command palette
 - **Multi-model router support** — works with both single-model and multi-model llama.cpp server configurations
 - **Image capabilities detection** — detects multimodal models automatically
-- **Flexible URL resolution** — configures the server URL via project config, environment variable, or global settings
+- **Flexible URL resolution** — configures the server via `llamaSettings` (project/global), environment variable, or legacy `llamaServerUrl`
 - **Auth support** — allows to login into a llama.cpp server that was secured with an API key
-- **Multiple server support** — connect to multiple llama.cpp servers simultaneously by separating URLs with semicolons
+- **Multiple server support** — connect to multiple llama.cpp servers simultaneously via `llamaSettings.servers` or semicolon-separated URLs
 - **Thinking budget support** — configurable token budgets for model reasoning/thinking, mapped to Pi's thinking levels
 - **Real-time progress tracking** — live loading progress via SSE (falls back to polling)
 
@@ -48,34 +48,107 @@ pi install https://github.com/gsanhueza/pi-llama-cpp
 
 ## Configuration
 
-The extension resolves the llama.cpp server URL(s) using the following priority order:
+The extension resolves the llama.cpp server configuration using the following priority order:
 
-1. **Per-project config** — `.pi/settings.json` in your project root:
+1. **Environment variable** — `LLAMA_SERVER_URL`
+2. **`llamaSettings`** — Main configuration format in `.pi/settings.json` (project) or `~/.pi/agent/settings.json` (global)
+3. **`llamaServerUrl`** — Legacy format in `.pi/settings.json` (project) or `~/.pi/agent/settings.json` (global)
+4. **Default** — `http://127.0.0.1:8080`
 
-   ```json
-   {
-     "llamaServerUrl": "http://127.0.0.1:8080"
-   }
-   ```
+### Server configuration
 
-2. **Environment variable** — `LLAMA_SERVER_URL`
+The recommended way to configure the extension is using the `llamaSettings` key. This provides a structured way to define multiple servers with custom names and IDs, plus additional behavior options.
 
-3. **Global settings** — `~/.pi/agent/settings.json`:
+Add this to your `.pi/settings.json` (project) or `~/.pi/agent/settings.json` (global):
 
-   ```json
-   {
-     "llamaServerUrl": "http://127.0.0.1:8080"
-   }
-   ```
+```json
+{
+  "llamaSettings": {
+    "servers": [
+      {
+        "url": "http://127.0.0.1:8080",
+        "id": "local",
+        "name": "Local Server"
+      },
+      {
+        "url": "http://10.0.0.5:8080",
+        "name": "Remote Server"
+      }
+    ],
+    "reactToModelSelect": true,
+    "autoloadOnMessage": false,
+    "pollingTimeout": 60000,
+    "serverTimeout": 1000
+  }
+}
+```
 
-4. **Default** — `http://127.0.0.1:8080`
+With this config, the servers will appear in Pi as **Llama.cpp (Local Server)** and **Llama.cpp (Remote Server)**.
+
+#### Server Options
+
+| Option | Type   | Required | Description                                                                  |
+| ------ | ------ | -------- | ---------------------------------------------------------------------------- |
+| `url`  | string | Yes      | The URL of the llama.cpp server                                              |
+| `id`   | string | No       | Custom provider ID (used for API key auth). Defaults to `llama-server=<url>` |
+| `name` | string | No       | Display name for the server in the UI (shown as `Llama.cpp — <name>`)        |
+
+> **Note:** If you set a custom `id`, you can use it in `~/.pi/agent/auth.json`. The extension will also fall back to the URL-based ID if no key is found for the custom `id`.
+
+#### Settings Options
+
+| Option               | Type    | Default | Description                                                   |
+| -------------------- | ------- | ------- | ------------------------------------------------------------- |
+| `reactToModelSelect` | boolean | `true`  | Load the model when you switch via Pi's model picker.         |
+| `autoloadOnMessage`  | boolean | `false` | Automatically load an unloaded model before sending a message |
+| `pollingTimeout`     | number  | `60000` | Max time (ms) to wait for model loading before giving up      |
+| `serverTimeout`      | number  | `1000`  | Timeout (ms) for server health checks and SSE probes          |
+
+> **Note:** `serverTimeout` controls individual HTTP request timeouts (health checks, SSE probe). `pollingTimeout` controls the total wait time for a model to finish loading. Increase `serverTimeout` for slow/high-latency servers, and `pollingTimeout` for large models or slow hardware.
+
+#### Environment variable
+
+For a quick setup, you can use the `LLAMA_SERVER_URL` environment variable instead of the JSON config:
+
+```bash
+export LLAMA_SERVER_URL="http://127.0.0.1:8080"
+```
 
-### Multiple Servers
+This is equivalent to defining a single server in `llamaSettings.servers` with just a URL.
 
-To connect to multiple llama.cpp servers simultaneously, add your URLs as a single string **separated with semicolons** in any of the examples above:
+### Legacy configuration
+
+For a simpler setup, you can use the legacy `llamaServerUrl` key:
+
+```json
+{
+  "llamaServerUrl": "http://127.0.0.1:8080"
+}
+```
+
+This is equivalent to defining a single server in `llamaSettings.servers` with just a URL.
+
+### Multiple servers
+
+To connect to multiple llama.cpp servers simultaneously:
+
+**Using `llamaSettings` (recommended):**
+
+```json
+{
+  "llamaSettings": {
+    "servers": [
+      { "url": "http://127.0.0.1:8080" },
+      { "url": "http://127.0.0.1:8081" },
+      { "url": "http://10.0.0.5:8080" }
+    ]
+  }
+}
+```
+
+**Using the environment variable:**
 
 ```bash
-# Example for env, but you can use any of the other methods
 LLAMA_SERVER_URL="http://127.0.0.1:8080;http://127.0.0.1:8081;http://10.0.0.5:8080"
 ```
 
@@ -86,7 +159,7 @@ Each server gets its own provider (e.g., **Llama.cpp (http://127.0.0.1:8080)**)
 If your llama.cpp server requires authentication, use `/login` in Pi, select the "API key" option, and choose the provider from the list that correlates with the server needing the API key.
 
 Alternatively, configure the API key in `~/.pi/agent/auth.json`:
-Use the provider ID `llama-server=<url>`:
+Use the provider ID `llama-server=<url>` (or your custom `id` if you set one in `llamaSettings.servers`):
 
 ```json
 {
@@ -199,13 +272,15 @@ When you switch models via Pi's model picker (instead of using the `/models` com
 
 This keeps the server in sync with the active model in Pi, regardless of how the switch was initiated — you don't need to manually load models before using them.
 
+You can disable this behavior by setting `reactToModelSelect` to `false` in `llamaSettings`.
+
 > **Note:** If you switch sessions while a model load is in-flight, you'll see a warning, but the load continues in the background. Use `/models` in the new session to verify the model status.
 
 ### Loading Models
 
 When you trigger a load, switch, or retry action, the extension uses SSE (Server-Sent Events) to receive real-time progress updates from the server. If SSE is not available, it falls back to polling.
 
-If loading takes longer than **60 seconds**, the operation times out with an error.
+If loading takes longer than **60 seconds** (configurable via `pollingTimeout`), the operation times out with an error.
 
 > **Note:** The timeout only applies to the progress detection. The model might still be loading in the background.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-llama-cpp",
-  "version": "0.9.1",
+  "version": "0.10.0",
   "description": "Pi extension for llama.cpp integration. Supports router, single and legacy models. Supports multiple servers.",
   "keywords": [
     "pi",
@@ -32,13 +32,14 @@
     ]
   },
   "peerDependencies": {
-    "@earendil-works/pi-ai": "*",
-    "@earendil-works/pi-coding-agent": "*",
-    "@earendil-works/pi-tui": "*"
+    "@earendil-works/pi-ai": ">=0.84.0",
+    "@earendil-works/pi-coding-agent": ">=0.84.0",
+    "@earendil-works/pi-tui": ">=0.84.0"
   },
+  "type": "module",
   "devDependencies": {
-    "@types/node": "^26.1.1",
+    "@types/node": "^26.4.0",
     "prettier-plugin-organize-imports": "^4.3.0",
-    "vitest": "^4.1.10"
+    "vitest": "^4.1.11"
   }
 }

package/src/constants.ts

@@ -21,12 +21,12 @@ export const API_KEY_PLACEHOLDER = "sk-placeholder";
 /**
  * The default URL if the resolver couldn't find it
  */
-export const DEFAULT_LLAMA_SERVER_URL = "http://127.0.0.1:8080";
+export const LLAMA_SERVER_URL = "http://127.0.0.1:8080";
 
 /**
  * The default context if the server didn't expose it
  */
-export const DEFAULT_CTX = 128000;
+export const FALLBACK_CTX = 128000;
 
 /**
  * Polling interval (ms) for checking model load status
@@ -48,10 +48,20 @@ export const READABLE_TIMEOUT = 15000;
  */
 export const SERVER_TIMEOUT = 1000;
 
+/**
+ * Default value for reactToModelSelect setting.
+ */
+export const REACT_TO_MODEL_SELECT = true;
+
+/**
+ * Default value for autoloadOnMessage setting.
+ */
+export const AUTOLOAD_ON_MESSAGE = false;
+
 /**
  * Thinking budgets to send to the server, depending on user-selected level in Pi.
  */
-export const DEFAULT_THINKING_BUDGETS = {
+export const THINKING_BUDGETS = {
   off: 0,
   minimal: 1024,
   low: 2048,

package/src/index.ts

@@ -11,13 +11,10 @@ import { ModelSelectEvent } from "./interfaces/events";
 import { CommandManager } from "./managers/command";
 import { EventManager } from "./managers/events";
 import { ServerManager } from "./managers/server";
-import { ConfigResolver } from "./resolver";
-import { Server } from "./server";
+import { settings } from "./managers/settings";
 
 export default async function (pi: ExtensionAPI) {
-  const resolver = new ConfigResolver();
-  const urls = await resolver.resolveUrls();
-  const servers = urls.map((url) => new Server(url));
+  const servers = await settings.resolveServers();
 
   const eventManager = new EventManager(servers);
   const serverManager = new ServerManager(servers);
@@ -40,15 +37,12 @@ export default async function (pi: ExtensionAPI) {
     if (event.reason !== "startup") return;
     for (const warning of serverManager.getWarnings())
       ctx.ui.notify(warning, "warning");
-
-    for (const warning of resolver.getWarnings())
-      ctx.ui.notify(warning, "warning");
   });
 
   pi.on(
     "before_provider_request",
-    async (event: BeforeProviderRequestEvent) =>
-      await eventManager.onBeforeProviderRequest(event),
+    async (event: BeforeProviderRequestEvent, ctx: ExtensionContext) =>
+      await eventManager.onBeforeProviderRequest(event, ctx),
   );
 
   pi.on(

package/src/interfaces/settings.ts

@@ -0,0 +1,60 @@
+/**
+ * A description of a server in the "llamaSettings" key
+ */
+interface LlamaServer {
+  /**
+   * The URL of the llama.cpp server.
+   */
+  url: string;
+  /**
+   * Custom provider ID for this server.
+   */
+  id?: string;
+  /**
+   * Custom display name for this server.
+   */
+  name?: string;
+}
+
+/**
+ * The main configuration interface for this extension
+ *
+ * E.g.:
+ *
+ * {
+ *   "servers": [{
+ *     "id": "server-a",
+ *     "name": "Server A",
+ *     "url": "http://localhost:8080"
+ *   }],
+ *   "reactToModelSelect": true
+ *   "autoloadOnMessage": false
+ * }
+}
+ */
+export interface LlamaSettings {
+  /**
+   * List of servers to connect to.
+   */
+  servers: LlamaServer[];
+  /**
+   * Whether to react to model selection events by loading the model.
+   * @default true
+   */
+  reactToModelSelect?: boolean;
+  /**
+   * Whether to auto-load models when a message is sent.
+   * @default false
+   */
+  autoloadOnMessage?: boolean;
+  /**
+   * Maximum time (ms) to wait for model loading before giving up.
+   * @default 60000
+   */
+  pollingTimeout?: number;
+  /**
+   * Timeout (ms) for server verification and SSE support probe.
+   * @default 1000
+   */
+  serverTimeout?: number;
+}

package/src/managers/events.ts

@@ -3,9 +3,10 @@ import {
   type ExtensionContext,
 } from "@earendil-works/pi-coding-agent";
 import { READABLE_TIMEOUT } from "../constants";
+import { Status } from "../enums/status";
 import { ModelSelectEvent } from "../interfaces/events";
+import { settings } from "../managers/settings";
 import { BaseModel } from "../models/baseModel";
-import { ConfigResolver } from "../resolver";
 import { Server } from "../server";
 
 export class EventManager {
@@ -27,6 +28,9 @@ export class EventManager {
    * @param ctx Pi context
    */
   async onModelSelect(event: ModelSelectEvent, ctx: ExtensionContext) {
+    // Check if the model_select event should be used
+    if (!settings.resolveReactToModelSelect()) return;
+
     for (const { providerId, models } of this.servers) {
       if (event.model.provider !== providerId) continue;
 
@@ -44,6 +48,20 @@ export class EventManager {
     }
   }
 
+  /**
+   * Loads the model if auto-loading is enabled and the model is unloaded.
+   *
+   * @param model The model to potentially auto-load
+   */
+  private async autoLoadIfNeeded(model: BaseModel): Promise<void> {
+    if (!settings.resolveAutoloadOnMessage()) return;
+
+    const status = await model.getStatus();
+    if (status !== Status.UNLOADED) return;
+
+    await model.load();
+  }
+
   /**
    * Session-switch handler. Registered once at extension init.
    * Only notifies if a model load is actually in-flight.
@@ -72,22 +90,28 @@ export class EventManager {
    * @param event Request event
    * @returns Updated payload
    */
-  async onBeforeProviderRequest(event: BeforeProviderRequestEvent) {
+  async onBeforeProviderRequest(
+    event: BeforeProviderRequestEvent,
+    ctx: ExtensionContext,
+  ) {
     const payload = event.payload as { model?: string };
     const { model } = payload;
     if (!model) return payload;
 
     // Check if this model belongs to one of our servers
-    const isLlamaCpp = this.servers.some((s) =>
-      s.models.some((m) => m.id === model),
-    );
+    const serverModel = this.servers
+      .flatMap((s) => s.models)
+      .find((m) => m.id === model);
+
+    if (!serverModel) return payload;
 
-    if (!isLlamaCpp) return payload;
+    // Auto-load if enabled and model is unloaded
+    await this.autoLoadIfNeeded(serverModel);
 
     // Retrieve pi's current thinking level, so we can setup a budget
-    const resolver = new ConfigResolver();
-    const level = resolver.resolveThinkingLevel() ?? "medium";
-    const budgets = resolver.resolveThinkingBudgets();
+    const level =
+      ctx.thinkingLevel ?? settings.resolveThinkingLevel() ?? "medium";
+    const budgets = settings.resolveThinkingBudgets();
     const thinking_budget_tokens = budgets[level];
 
     // Setup payload

package/src/managers/server.ts

@@ -1,8 +1,9 @@
 import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
-import { API_TYPE, PROVIDER_NAME, SERVER_TIMEOUT } from "../constants";
+import { API_TYPE, PROVIDER_NAME } from "../constants";
 import { ServerStatus } from "../enums/serverStatus";
 import { BaseModel } from "../models/baseModel";
 import { Server } from "../server";
+import { settings } from "./settings";
 
 export class ServerManager {
   readonly failedUrls: string[] = [];
@@ -16,8 +17,9 @@ export class ServerManager {
    * @param pi The Pi extension API
    */
   async initialize(pi: ExtensionAPI) {
-    // Register the providers with a timeout first
-    await this.update(pi, SERVER_TIMEOUT);
+    // Register the providers with the configured server timeout
+    const { serverTimeout } = settings.resolveTimeouts();
+    await this.update(pi, serverTimeout);
   }
 
   /**
@@ -67,7 +69,7 @@ export class ServerManager {
       } else if (status === ServerStatus.TIMEOUT) {
         const message = [
           "[pi-llama-cpp]",
-          `${PROVIDER_NAME} server initialization for '${server.baseUrl}' took more than ${SERVER_TIMEOUT} ms, so it has been skipped.`,
+          `${PROVIDER_NAME} server initialization for '${server.baseUrl}' took more than ${timeout} ms, so it has been skipped.`,
           "Run `/models` to retry without timeout and see all models.",
         ].join("\n");
         this.warnings.push(message);

package/src/managers/settings.ts

@@ -0,0 +1,206 @@
+import { ApiKeyCredential, ModelThinkingLevel } from "@earendil-works/pi-ai";
+import {
+  readStoredCredential,
+  SettingsManager,
+} from "@earendil-works/pi-coding-agent";
+import {
+  API_KEY_PLACEHOLDER,
+  AUTOLOAD_ON_MESSAGE,
+  LLAMA_SERVER_URL,
+  POLLING_TIMEOUT,
+  REACT_TO_MODEL_SELECT,
+  SERVER_TIMEOUT,
+  THINKING_BUDGETS,
+} from "../constants";
+import { LlamaSettings } from "../interfaces/settings";
+import { Server } from "../server";
+
+const SETTINGS_KEY = "llamaSettings";
+
+export class LlamaSettingsManager {
+  private settingsManager = SettingsManager.create(process.cwd());
+
+  /**
+   * Convenience getter for merged project/global settings
+   */
+  private get mergedSettings(): Record<string, any> {
+    const merged = {
+      ...this.settingsManager.getGlobalSettings(),
+      ...this.settingsManager.getProjectSettings(),
+    } as Record<string, any>;
+    return merged;
+  }
+
+  /**
+   * Convenience getter for the `llamaSettings` key
+   */
+  private get llamaSettings(): LlamaSettings {
+    return this.mergedSettings[SETTINGS_KEY] ?? {};
+  }
+
+  /**
+   * Resolves the server URLs to use in the following order:
+   *
+   * - `LLAMA_SERVER_URL` env variable
+   * - `llamaSettings` key (current - project, then global)
+   * - `llamaServerUrl` key (legacy - project, then global)
+   * - Default URL
+   *
+   * @returns The list of URLs to use
+   */
+  resolveUrls(): string[] {
+    let response = this.resolveEnvUrls();
+    if (response.length > 0) return response;
+
+    response = this.resolveServerUrls();
+    if (response.length > 0) return response;
+
+    response = this.resolveLegacyUrls();
+    if (response.length > 0) return response;
+
+    return [LLAMA_SERVER_URL];
+  }
+
+  /**
+   * Resolves the llama-server URLs from the environment variable.
+   *
+   * @returns A list of detected URLs
+   */
+  private resolveEnvUrls(): string[] {
+    const raw = process.env.LLAMA_SERVER_URL;
+    if (!raw) return [];
+
+    return this.parseUrls(raw);
+  }
+
+  /**
+   * Resolves the llama-server URLs from `llamaSettings.servers`.
+   * Settings are merged, prioritizing project over global settings.
+   *
+   * @returns A list of detected URLs
+   */
+  private resolveServerUrls(): string[] {
+    const { servers = [] } = this.llamaSettings;
+    return servers.map((s) => this.parseUrls(s.url)).flat();
+  }
+
+  /**
+   * Resolves the llama-server URLs from `llamaSettings.servers`.
+   * Settings are merged, prioritizing project over global settings.
+   *
+   * @returns A list of detected URLs
+   */
+  private resolveLegacyUrls(): string[] {
+    const { llamaServerUrl = null } = this.mergedSettings;
+    if (!llamaServerUrl) return [];
+
+    return this.parseUrls(llamaServerUrl);
+  }
+
+  /**
+   * Parses a raw URL string into an array of cleaned URLs.
+   * Splits on semicolons, trims whitespace, filters empty strings,
+   * and strips trailing slashes.
+   *
+   * @returns A sanitized URL
+   */
+  private parseUrls(raw: string): string[] {
+    return raw
+      .split(";")
+      .map((u) => u.trim())
+      .filter((u) => u.length > 0)
+      .map((u) => u.replace(/\/+$/, ""));
+  }
+
+  /**
+   * Resolves the servers that this extension will use.
+   * Uses `resolveUrls()` as the source of truth for URLs (env > settings >
+   * legacy > default), then applies `id`/`name` from `llamaSettings.servers`
+   * as overrides when available.
+   *
+   * @returns A list of Server objects
+   */
+  resolveServers(): Server[] {
+    const { pollingTimeout, serverTimeout } = this.resolveTimeouts();
+    const urls = this.resolveUrls();
+    const serverConfigs = this.llamaSettings.servers ?? [];
+
+    return urls.map((url) => {
+      const config = serverConfigs.find((s) => s.url === url);
+      return new Server(
+        url,
+        config?.id,
+        config?.name,
+        serverTimeout,
+        pollingTimeout,
+      );
+    });
+  }
+
+  /**
+   * Resolves API key for the provider ID using Pi's stored credentials.
+   *
+   * @returns The API key to use for the provider
+   */
+  resolveApiKey(providerId: string): string {
+    const credential = readStoredCredential(providerId) as ApiKeyCredential;
+    return credential?.key ?? API_KEY_PLACEHOLDER;
+  }
+
+  /**
+   * Resolves the current thinking level from Pi.
+   *
+   * @returns The thinking level
+   */
+  resolveThinkingLevel(): ModelThinkingLevel | undefined {
+    return this.settingsManager.getDefaultThinkingLevel();
+  }
+
+  /**
+   * Resolves the effective thinking budgets from settings.
+   *
+   * @returns An object with selected budgets for thinking levels
+   */
+  resolveThinkingBudgets(): Record<ModelThinkingLevel, number> {
+    const settingsBudgets = this.settingsManager.getThinkingBudgets() ?? {};
+    return {
+      ...THINKING_BUDGETS,
+      ...settingsBudgets,
+    };
+  }
+
+  /**
+   * Resolves whether the extension should react to model selection events.
+   *
+   * @returns `true` if the extension should load the model on model_select
+   */
+  resolveReactToModelSelect(): boolean {
+    return this.llamaSettings.reactToModelSelect ?? REACT_TO_MODEL_SELECT;
+  }
+
+  /**
+   * Resolves whether the extension should auto-load models on message.
+   *
+   * @returns `true` if the extension should auto-load models
+   */
+  resolveAutoloadOnMessage(): boolean {
+    return this.llamaSettings.autoloadOnMessage ?? AUTOLOAD_ON_MESSAGE;
+  }
+
+  /**
+   * Resolves the timeout settings for polling and server checks.
+   *
+   * @returns Object with polling and server timeout values
+   */
+  resolveTimeouts(): { pollingTimeout: number; serverTimeout: number } {
+    return {
+      pollingTimeout: this.llamaSettings.pollingTimeout ?? POLLING_TIMEOUT,
+      serverTimeout: this.llamaSettings.serverTimeout ?? SERVER_TIMEOUT,
+    };
+  }
+}
+
+/**
+ * Shared singleton instance used across the extension.
+ */
+export const settings = new LlamaSettingsManager();