@pwguler/pi-pengepul-provider 0.2.3 → 0.3.0

package/README.md

@@ -41,8 +41,8 @@ prefixed `pengepul/<id>`. To try it without installing, use
 
 ## What it does
 
-- Registers the `pengepul` provider against your relay's base URL
-  (`http://127.0.0.1:8317` by default).
+- Registers the `pengepul` provider against the relay base URL on your
+  credential (`http://127.0.0.1:8317` by default).
 - Discovers models from `GET /v1/models`, maps each to the right wire:
   - `claude-*` / `anthropic/*` and `owned_by: anthropic` → Anthropic Messages
     (`POST /v1/messages`),
@@ -53,24 +53,55 @@ prefixed `pengepul/<id>`. To try it without installing, use
   `max_output_tokens`, `input_modalities`, `pricing`). Fields the relay omits
   fall back to pi's builtin catalog for the same id, then to family
   heuristics.
-- Caches the last successful catalog at `<agent-dir>/pengepul-models.json`, so
-  startup does not wait on the network and a briefly absent relay is covered.
+- Caches the catalog in pi's model store, so startup does not wait on the
+  network and a briefly absent relay is covered. The cached models are
+  re-pointed at the relay base configured now, so moving the relay does not
+  leave requests aimed at the old address.
 - Reuses pi's built-in stream functions for both wires — no custom transport.
-- Registers no commands: the catalog refreshes on every startup.
+- Registers no commands: pi refreshes the catalog on every startup.
 
 ## Configuration
 
+Everything lives in `~/.pi/agent/auth.json`: the relay's API key and the relay
+base it applies to.
+
+```json
+{
+  "pengepul": {
+    "type": "api_key",
+    "key": "sk-local-...",
+    "baseUrl": "http://127.0.0.1:8317"
+  }
+}
+```
+
+`/login pengepul` writes both fields for you — it asks for the key, then for the
+relay URL, and defaults the URL to `http://127.0.0.1:8317`. Editing the file by
+hand works the same way; `baseUrl` may end in `/v1` or not.
+
+To reach a relay on another machine, put that machine's address in `baseUrl`
+(and make the relay listen beyond loopback there: `host: 0.0.0.0` in
+`~/.pengepul/config.yaml` on the relay, or forward the port over SSH). The key
+is the relay's own key — `pengepul config api-key` prints it.
+
+Optional overrides, for CI or a one-off shell. Each is a fallback: the
+credential wins when it carries a value.
+
 | Setting | Env var | Default |
 |---|---|---|
-| Relay base URL | `PENGEPUL_BASE_URL` | `http://127.0.0.1:8317` |
-| API key | `PENGEPUL_API_KEY` | read from `~/.pengepul/config.yaml` |
+| Relay base URL | `PENGEPUL_BASE_URL` | `http://127.0.0.1:8317`, or the relay's own `config.yaml` |
+| API key | `PENGEPUL_API_KEY` | `api-keys[0]` in `~/.pengepul/config.yaml` |
 | Config path | `PENGEPUL_CONFIG` | `~/.pengepul/config.yaml` |
-| Model cache path | `PENGEPUL_MODELS_CACHE` | `<agent-dir>/pengepul-models.json` |
+| Legacy cache path | `PENGEPUL_MODELS_CACHE` | `<agent-dir>/pengepul-models.json` |
 | Discovery timeout | `PENGEPUL_MODELS_TIMEOUT_MS` | `10000` |
 
-The API key is read from `~/.pengepul/config.yaml` (`api-keys[0]`, the
-`sk-local-...` key pengepul generates on first run) unless `PENGEPUL_API_KEY`
-is set.
+`PENGEPUL_MODELS_CACHE` names the pre-0.3 cache file. It is read once, when pi's
+model store has no pengepul catalog yet, and never written again.
+
+Do not set `providers.pengepul.baseUrl` in `models.json`. pi applies that value
+to every model of the provider, which collapses the two wires onto one URL —
+Anthropic Messages traffic would be sent to `/v1` and Chat Completions traffic
+to `/`.
 
 ## Notes
 
@@ -81,16 +112,40 @@ is set.
   pengepul bills against — displayed costs are upstream list prices.
 - The relay must be running and reachable for discovery to succeed. Without a
   cached catalog on a first start, pengepul models stay unavailable until a
-  start with the relay up.
+  start with the relay up. A relay that answers with 401 leaves the last known
+  catalog registered and logs a warning; discovery does not take pi down with
+  it.
 
 ## Development
 
 ```sh
+npm install
 bun test
 npx tsc --noEmit
 bun scripts/e2e-live.ts # live e2e against a running relay; sends one tiny completion
 ```
 
+The `@earendil-works/*` copies `npm install` writes into `node_modules` are for
+`tsc` and the tests only. At runtime pi serves the extension those modules from
+its own bundle (`loader.js` maps `@earendil-works/pi-ai/providers/all` to a
+virtual module), so the local copies exist to be *type-checked against*, not to
+be the ones in play.
+
+Nothing here can enforce that the two match: the peer ranges are `*` and the
+host's version is not knowable at install time. The lockfile records whichever
+version was current when it was last refreshed, so the comparison is a manual
+one, worth making whenever a measurement has to be trusted:
+
+```sh
+pi --version                                            # host pi release
+cat node_modules/@earendil-works/pi-ai/package.json     # local copy
+```
+
+When they drift, anything measured from this directory describes a different
+model catalog than the running extension sees — the 0.84.4 copy carried 40
+`:batch` entries where 0.85.1 carries 68 — and local verification quietly
+disagrees with production.
+
 ## Releasing
 
 Releases publish from CI on a version tag. Bump, tag, push:

package/extensions/api-key.ts

@@ -1,53 +1,22 @@
 /**
- * Resolve pengepul's local API key.
+ * Read pengepul's own API key.
  *
  * pengepul authenticates every request with a static key from its config:
  * `~/.pengepul/config.yaml`, under `api-keys:` (the first is generated on first
  * run and is `sk-local-...`). Clients send it as `Authorization: Bearer <key>`
  * or `x-api-key: <key>`.
  *
- * Precedence: an explicit env override wins, then the config file. The config
- * read is injected so this module stays io-free and testable.
+ * This is the fallback for the machine that runs the relay itself: a client box
+ * configures the key in `auth.json`, which pi resolves and hands to the
+ * provider. Only extraction lives here, so the file read stays with the caller
+ * and this module stays io-free.
  */
 
-const DEFAULT_CONFIG_PATH = "~/.pengepul/config.yaml"
-
-export type ApiKeySource = "env" | "config" | "none"
-
-export interface ApiKeyResolution {
-  /** The resolved key, or undefined when none could be found. */
-  key?: string
-  source: ApiKeySource
-}
+export const DEFAULT_CONFIG_PATH = "~/.pengepul/config.yaml"
 
 export const API_KEY_ENV = "PENGEPUL_API_KEY"
 export const CONFIG_PATH_ENV = "PENGEPUL_CONFIG"
 
-/**
- * Resolve the key from an env map and a config-file reader.
- *
- * @param env          the environment (or a test substitution for it).
- * @param readConfig   reads a config file's text by path, or undefined when the
- *                     path is unwritable/absent. Injected to keep this pure.
- */
-export function resolveApiKey(
-  env: Record<string, string | undefined>,
-  readConfig: (path: string) => string | undefined,
-): ApiKeyResolution {
-  const envKey = env[API_KEY_ENV]
-  if (envKey && envKey.trim() !== "") return { key: envKey, source: "env" }
-
-  const configPath = env[CONFIG_PATH_ENV] ?? DEFAULT_CONFIG_PATH
-  const configText = readConfig(configPath)
-  if (configText === undefined) return { source: "none" }
-
-  const keys = extractApiKeys(configText)
-  const first = keys[0]
-  if (first) return { key: first, source: "config" }
-
-  return { source: "none" }
-}
-
 /**
  * Extract `api-keys:` entries from pengepul's YAML config, without a YAML
  * dependency. Handles both the inline-flow form and the block-sequence form:

package/extensions/config.ts

@@ -1,40 +1,34 @@
 /**
- * Resolve pengepul connection and cache settings from the environment.
+ * Where the pengepul provider's fallback sources live.
  *
- * Kept small and pure: takes an env map and the host's agent dir, returns the
- * connection defaults a test can pin down.
+ * `auth.json` is the configuration surface. These are the escape hatches kept
+ * for tests, CI, and a machine that happens to run the relay itself: env vars
+ * for the relay base and key, and pengepul's own config on disk.
  */
 
 import { join } from "node:path"
 
-import { DEFAULT_RELAY_BASE } from "./models.ts"
+import { CONFIG_PATH_ENV, DEFAULT_CONFIG_PATH } from "./api-key.ts"
+
+export { CONFIG_PATH_ENV, DEFAULT_CONFIG_PATH }
 
 export const RELAY_BASE_ENV = "PENGEPUL_BASE_URL"
 export const MODELS_CACHE_ENV = "PENGEPUL_MODELS_CACHE"
 export const MODELS_TIMEOUT_MS_ENV = "PENGEPUL_MODELS_TIMEOUT_MS"
 
 export interface PengepulSettings {
-  /** The relay base URL (may or may not end in /v1). */
-  relayBase: string
-  /** Where the model catalog is cached; defaults to `<agent-dir>/pengepul-models.json`. */
-  modelsCachePath: string
-  /** Discovery timeout in milliseconds. */
-  modelsTimeoutMs: number
+  /** Where pengepul's own config lives, when this machine runs the relay. */
+  configPath: string
+  /** Where the pre-0.3 catalog cache lives; read once to seed pi's store. */
+  legacyCachePath: string
 }
 
 export function resolveSettings(
   env: Record<string, string | undefined>,
   agentDir: string,
 ): PengepulSettings {
-  const relayBase = env[RELAY_BASE_ENV] ?? DEFAULT_RELAY_BASE
-  const modelsCachePath =
-    env[MODELS_CACHE_ENV] ?? join(agentDir, "pengepul-models.json")
-  const rawTimeout = env[MODELS_TIMEOUT_MS_ENV]
-  const parsedTimeout = rawTimeout ? Number(rawTimeout) : NaN
-  const modelsTimeoutMs =
-    Number.isFinite(parsedTimeout) && parsedTimeout > 0
-      ? parsedTimeout
-      : 10_000
-
-  return { relayBase, modelsCachePath, modelsTimeoutMs }
+  return {
+    configPath: env[CONFIG_PATH_ENV] ?? DEFAULT_CONFIG_PATH,
+    legacyCachePath: env[MODELS_CACHE_ENV] ?? join(agentDir, "pengepul-models.json"),
+  }
 }

package/extensions/credential.ts

@@ -0,0 +1,113 @@
+/**
+ * Pengepul credential and relay-base resolution.
+ *
+ * The credential is the pengepul entry in pi's `auth.json`: an API key for the
+ * relay plus the relay base it applies to. pi core resolves the key itself but
+ * knows nothing about `baseUrl`, so the extension reads it here. Everything in
+ * this module is pure: an env map and config text in, a value out.
+ *
+ * Precedence, highest first: credential, then environment, then the relay's own
+ * config file (which only exists on the machine running the relay), then the
+ * loopback default pengepul binds to.
+ */
+
+import { normalizeRootBaseUrl } from "./dialect.ts"
+import { DEFAULT_RELAY_BASE } from "./models.ts"
+
+/** The fields this provider reads off a stored credential. */
+export interface PengepulCredential {
+  /** Always `"api_key"` in pi's store; read as unknown so a hand-edited file cannot lie its way past a cast. */
+  type?: unknown
+  key?: unknown
+  baseUrl?: unknown
+}
+
+function nonEmptyString(value: unknown): string | undefined {
+  if (typeof value !== "string") return undefined
+  const trimmed = value.trim()
+  return trimmed === "" ? undefined : trimmed
+}
+
+/** The relay base stored on the credential, when the user set one. */
+export function credentialRelayBase(credential: PengepulCredential | undefined): string | undefined {
+  return nonEmptyString(credential?.baseUrl)
+}
+
+/** The API key stored on the credential, when the user set one. */
+export function credentialApiKey(credential: PengepulCredential | undefined): string | undefined {
+  return nonEmptyString(credential?.key)
+}
+
+/**
+ * Build a relay base from pengepul's own `config.yaml`. Only the machine running
+ * the relay has this file; clients reach a relay elsewhere and skip this source.
+ *
+ * pengepul writes `host: ''` when it binds loopback ("empty binds 127.0.0.1, not
+ * every interface"), so an empty host means loopback rather than "unset". A
+ * bind-all address is not a connect address either: `0.0.0.0`, `::`, and `*` mean
+ * "every interface", and a client reaches that listener on loopback.
+ */
+export function relayBaseFromConfigText(configText: string | undefined): string | undefined {
+  if (configText === undefined) return undefined
+
+  let host = "127.0.0.1"
+  let port: number | undefined
+
+  for (const line of configText.split(/\r?\n/)) {
+    const hostMatch = /^\s*host:\s*(.*)$/.exec(line)
+    if (hostMatch) {
+      const value = (hostMatch[1] ?? "").trim().replace(/^["']|["']$/g, "")
+      if (value !== "" && !isBindAll(value)) host = value
+      continue
+    }
+
+    const portMatch = /^\s*port:\s*(.*)$/.exec(line)
+    if (portMatch) {
+      const value = (portMatch[1] ?? "").trim().replace(/^["']|["']$/g, "")
+      const parsed = Number(value)
+      if (Number.isInteger(parsed) && parsed > 0 && parsed <= 65_535) port = parsed
+    }
+  }
+
+  return port === undefined ? undefined : `http://${host}:${port}`
+}
+
+/** Whether a configured host is a bind-all address rather than a destination. */
+function isBindAll(host: string): boolean {
+  return host === "0.0.0.0" || host === "::" || host === "[::]" || host === "*"
+}
+
+export interface RelayBaseSources {
+  /** The relay base stored on the pengepul credential. */
+  credential?: string | undefined
+  /** The `PENGEPUL_BASE_URL` environment value. */
+  environment?: string | undefined
+  /** The relay base derived from pengepul's own config file. */
+  config?: string | undefined
+}
+
+/**
+ * The relay root every wire appends to: highest-precedence non-blank source, or
+ * loopback. A trailing `/v1` is accepted and stripped, so a base copied from a
+ * client config cannot produce `/v1/v1` downstream.
+ */
+export function resolveRelayBase(sources: RelayBaseSources): string {
+  const chosen =
+    nonEmptyString(sources.credential) ??
+    nonEmptyString(sources.environment) ??
+    nonEmptyString(sources.config) ??
+    DEFAULT_RELAY_BASE
+  return normalizeRootBaseUrl(chosen)
+}
+
+export interface ApiKeySources {
+  /** The key stored on the pengepul credential. */
+  credential?: string | undefined
+  /** The key resolved from `PENGEPUL_API_KEY` or pengepul's own config file. */
+  ambient?: string | undefined
+}
+
+/** The key requests and catalog fetches use, or undefined when none is configured. */
+export function resolveApiKey(sources: ApiKeySources): string | undefined {
+  return nonEmptyString(sources.credential) ?? nonEmptyString(sources.ambient)
+}

package/extensions/index.ts

@@ -1,30 +1,24 @@
 /**
  * @pwguler/pi-pengepul-provider entry point - the real edge adapter.
  *
- * Registers pengepul as a pi custom provider. pengepul is a local relay
- * (`http://127.0.0.1:8317`) that pools your Claude/Codex subscriptions and
- * speaks both native wires. The pure core lives in `./dialect.ts`, `./models.ts`
- * and `./runtime.ts`; this file adapts them to the pi ExtensionAPI seam.
+ * Registers pengepul as a pi provider. pengepul is a relay that pools your
+ * Claude/Codex subscriptions and speaks both native wires. The provider itself
+ * lives in `./provider.ts`: pi resolves auth through it and hands the credential
+ * back on every refresh, so `auth.json` carries the key and the relay base. The
+ * pure core is `./credential.ts`, `./dialect.ts`, and `./models.ts`; this file
+ * adapts them to the pi ExtensionAPI seam and reads the one file pi cannot.
  */
 
 import {
   getAgentDir,
   type ExtensionAPI,
-  type ProviderConfig,
 } from "@earendil-works/pi-coding-agent"
 import { getBuiltinModel, getBuiltinModels, getBuiltinProviders } from "@earendil-works/pi-ai/providers/all"
 import { readFileSync } from "node:fs"
 
-import { resolveApiKey } from "./api-key.ts"
 import { resolveSettings } from "./config.ts"
-import { modelsUrl } from "./dialect.ts"
-import {
-  loadCachedPengepulModels,
-  loadPengepulModels,
-  toProviderModelConfigs,
-  type PengepulModel,
-} from "./models.ts"
-import { createPengepulRuntime } from "./runtime.ts"
+import { catalogIdForms, type PengepulModel } from "./models.ts"
+import { createPengepulProvider } from "./provider.ts"
 
 function expandHome(path: string): string {
   if (path === "~") return process.env.HOME ?? path
@@ -53,11 +47,10 @@ function metaFromModel(model: NonNullable<ReturnType<typeof getBuiltinModel>>) {
 }
 
 /**
- * Multi-catalog lookup over pi's builtin models. A commandcode id can live
- * in several catalogs: verbatim under an aggregator (`openrouter`, `baseten`,
- * `together`, ...), bare under a vendor catalog (`deepseek`, `google`,
- * `xai`, ...), or bare lowercased. Try those shapes in that order and take
- * the first hit; reasoning metadata and the thinkingLevelMap flow from it.
+ * Multi-catalog lookup over pi's builtin models. A relay id can match several
+ * catalogs, so each of the id's shapes is tried in turn - verbatim, bare under
+ * a vendor catalog, last segment, and with the routing namespace removed - and
+ * the first hit wins. Reasoning metadata and the thinkingLevelMap flow from it.
  */
 function createBuiltinLookup(): (id: string, dialect: string) => ReturnType<typeof metaFromModel> | undefined {
   type Entry = { provider: string; id: string };
@@ -77,70 +70,40 @@ function createBuiltinLookup(): (id: string, dialect: string) => ReturnType<type
   }
 
   return (id, dialect) => {
-    const candidates: Array<Entry | undefined> = [
-      exact.get(id),
-      exact.get(bareOf(id)),
-      segments.get(bareOf(id)),
-      lower.get(id.toLowerCase()),
-      lower.get(bareOf(id).toLowerCase()),
-    ]
-    for (const candidate of candidates) {
-      if (candidate === undefined) continue
-      const model = getBuiltinModel(candidate.provider as never, candidate.id as never)
-      if (model) return metaFromModel(model)
+    for (const form of catalogIdForms(id)) {
+      const candidates: Array<Entry | undefined> = [
+        exact.get(form),
+        segments.get(form),
+        lower.get(form.toLowerCase()),
+      ]
+      for (const candidate of candidates) {
+        if (candidate === undefined) continue
+        const model = getBuiltinModel(candidate.provider as never, candidate.id as never)
+        if (model) return metaFromModel(model)
+      }
     }
     return undefined
   }
 }
 
-function bareOf(id: string): string {
-  const slash = id.lastIndexOf("/")
-  return slash === -1 ? id : id.slice(slash + 1)
-}
-
-function createProviderConfigFactory(relayBase: string, apiKey: string | undefined) {
-  return (models: readonly PengepulModel[]): ProviderConfig => ({
-    name: "Pengepul",
-    baseUrl: relayBase,
-    apiKey: apiKey ?? "$PENGEPUL_API_KEY",
-    api: "anthropic-messages",
-    models: toProviderModelConfigs(models, relayBase),
-  })
-}
-
 /**
- * Model discovery and provider registration are async: the relay's catalog is
- * fetched live (and cached), so the runtime handles the cache-first, then
- * live-refresh dance. The config factory pins the base URL and key once.
+ * Model discovery belongs to pi: it calls `refreshModels` with the resolved
+ * credential, first against pi's cached catalog and then, when the network is
+ * allowed, against the relay. Registration is synchronous; nothing here waits
+ * on the relay, and the catalog survives a restart through pi's model store.
  */
-export default async function (pi: ExtensionAPI) {
+export default function (pi: ExtensionAPI) {
   const settings = resolveSettings(process.env, getAgentDir())
-  const apiKey = resolveApiKey(process.env, readConfigText).key
-
-  // The relay advertises only ids; context/pricing/modality numbers come from
-  // pi's builtin catalogs, searched across providers until one knows the id
-  // (aggregator, vendor, and last-segment shapes). The lookup is injected so
-  // the catalog logic stays free of pi-ai imports.
-  const lookupBuiltin = createBuiltinLookup()
 
-  const runtime = createPengepulRuntime(pi, {
-    loadModels: (signal) =>
-      loadPengepulModels({
-        url: modelsUrl(settings.relayBase),
-        apiKey,
-        cachePath: settings.modelsCachePath,
-        relayBase: settings.relayBase,
-        timeoutMs: settings.modelsTimeoutMs,
-        lookupBuiltin,
-        signal,
-      }),
-    loadCachedModels: () => loadCachedPengepulModels(settings.modelsCachePath),
-    createProviderConfig: createProviderConfigFactory(settings.relayBase, apiKey),
+  const provider = createPengepulProvider({
+    env: process.env,
+    configText: readConfigText(settings.configPath),
+    legacyCachePath: settings.legacyCachePath,
+    lookupBuiltin: createBuiltinLookup(),
   })
 
-  pi.on("session_shutdown", () => {
-    runtime.dispose()
-  })
-
-  await runtime.initialize()
+  pi.registerProvider(provider)
 }
+
+/** The pengepul catalog shape, re-exported for callers that build on the core. */
+export type { PengepulModel }