llm-switcher 1.2.0 → 1.2.3

package/CHANGELOG.md

@@ -1,5 +1,28 @@
 # Changelog — LLM Switcher
 
+## Release 1.2.3
+
+- **Update notice:** The dashboard sidebar shows the running version. When npm has a newer release, a notice shows the update command with a copy button. The new command `switch version` prints the same information. The switcher asks the npm registry at most once in 12 hours. Without an answer, no notice shows.
+- **Codex daemon follows the route:** The interactive Codex TUI talks to a shared `codex app-server` daemon, and that daemon keeps the environment that it started with. As a result, the TUI bypassed the gateway after a switch. Now each change of the Codex route restarts the running daemon with the new variables. When no daemon runs, nothing starts.
+- **API keys stay with their host:** A catalog refresh sent the key of the active profile to the official Anthropic and OpenAI model lists. Now a key goes only to the host of its own profile.
+- **Real model catalog:** The catalog reads the model list that Codex and Claude Code keep on disk for the signed-in account. A failed refresh no longer marks the tool version as done, so the next request tries again.
+- **Dashboard saves:** Every write action reported a failure after it succeeded, because the page read `ok` and the server sends `success`. Enter in a field no longer saves a half-edited profile. A text selection that ends outside the dialog no longer closes it.
+- **Model slots:** Each slot is a searchable list of the provider models. The list loads when you open the Model Slots tab and sends the saved key. A profile without `outFormat` keeps it empty after a save.
+- **Other fixes:** `make-certs.sh` works when the path contains a dot. `switch doctor` warns again about a Codex profile without `publicModels`. Three dashboard controls have an accessible name.
+- **Tests:** The tests use the 1.2 configuration schema, never read the certificates of the checkout, and never reach the real Codex daemon, npm registry, or `claude` binary.
+
+## Release 1.2.2
+
+- **Structured output reaches every upstream:** A request that asks for JSON that matches a schema now keeps that schema. Before this release, the gateway did not read `output_config.format` (Claude Code) or `text.format` (Codex), so the provider got a free-text request. The schema now goes to the provider as `response_format` (OpenAI Chat), `output_config.format` (Anthropic), or `responseMimeType` with `responseSchema` (Gemini and Vertex). A request for JSON without a schema reaches Anthropic as plain text, because Anthropic has no JSON mode without a schema.
+
+## Release 1.2.1
+
+- **Intact Visual Architecture & Hash Routing:** Fully restructured dashboard using Intact design system, pure CSS tokens, and hash-based client routing (`#/routes`, `#/models`, `#/logs`, `#/doctor`).
+- **Official Brand Icon Assets:** Integrated official brand icons for Anthropic Claude, OpenAI Codex, Google Gemini, Intact, OpenRouter, and Ollama, served statically under `/icons/*` with immutable caching and PNG/SVG whitelisting.
+- **Prefetched Navigation Counters:** Navigation badges prefetch model discovery and request inspector counts immediately on page load, eliminating the delay where badges showed zero until the tab was selected.
+- **Hardened Profile Slot Mapping:** Synchronized Claude model tiers (`sonnet`, `opus`, `haiku`, `fable`) and Codex model roles (`main`, `review`, `subagent`), ensuring slot persistence when editing profiles and removing redundant role tabs for Claude-targeted profiles.
+- **CORS & Endpoint Fixes:** Added `x-llm-switcher-token` to preflight `Access-Control-Allow-Headers` and resolved routing precedence for `GET /api/catalog`.
+
 ## Release 1.2.0
 
 - **Zero-Mutation Interceptor Invariant:** The gateway now operates strictly in the network path via the blindfold interceptor, never modifying user configuration files like `~/.claude/settings.json` or `~/.codex/config.toml`. Loopback base URLs (`ANTHROPIC_BASE_URL` and `OPENAI_BASE_URL`) are scrubbed by shims so traffic routes through `HTTPS_PROXY` cleanly.

package/README.md

@@ -401,6 +401,7 @@ Add the server to your MCP configuration (for example `opencode.jsonc`, `claude_
 ```bash
 switch ui                      # Open the Web UI dashboard in your browser
 switch status                  # Display status for all active CLI targets
+switch version                 # Show the version and tell you when npm has a newer one
 switch doctor                  # Audit environment, settings & routing
 switch on [profile]            # Start the gateway and activate a profile
 switch <profile>               # Activate a profile for both tools

package/blindfold/make-certs.sh

@@ -103,8 +103,10 @@ openssl req -x509 -new -key "$WORK/ca.key" -sha256 -days "$CA_DAYS" \
 
 openssl ecparam -name prime256v1 -genkey -noout -out "$WORK/leaf.key"
 openssl req -new -key "$WORK/leaf.key" -config "$WORK/leaf.cnf" -out "$WORK/leaf.csr"
+# An explicit -CAserial: LibreSSL derives the default name by cutting the CA path at its first dot,
+# so a home directory such as /Users/first.last sent the serial file to /Users/first.srl.
 openssl x509 -req -in "$WORK/leaf.csr" \
-  -CA "$WORK/ca.pem" -CAkey "$WORK/ca.key" -CAcreateserial \
+  -CA "$WORK/ca.pem" -CAkey "$WORK/ca.key" -CAcreateserial -CAserial "$WORK/ca.srl" \
   -days "$LEAF_DAYS" -sha256 -extfile "$WORK/leaf.ext" \
   -out "$WORK/leaf.pem"
 

package/catalog.mjs

@@ -7,6 +7,7 @@
 // ============================================================
 
 import fs from 'node:fs';
+import os from 'node:os';
 import path from 'node:path';
 
 export const OFFICIAL_MODEL_URLS = {
@@ -69,6 +70,70 @@ export function loadCatalogCache(stateDir) {
   };
 }
 
+function defaultSources() {
+  return {
+    codexHome: process.env.CODEX_HOME || path.join(os.homedir(), '.codex'),
+    claudeDir: process.env.CLAUDE_CONFIG_DIR || path.join(os.homedir(), '.claude')
+  };
+}
+
+/**
+ * Reads the model list that the tool itself keeps on disk for the signed-in account, so no key and
+ * no network call is needed. Codex writes models_cache.json; Claude Code writes cache/model-catalog.
+ * Returns null when the tool has no list.
+ */
+export function readLocalToolModels(tool, sources = defaultSources()) {
+  try {
+    if (tool === 'codex') {
+      const d = JSON.parse(fs.readFileSync(path.join(sources.codexHome, 'models_cache.json'), 'utf8'));
+      const models = (Array.isArray(d?.models) ? d.models : [])
+        .filter(m => m && typeof m.slug === 'string' && m.slug)
+        .map(m => ({ id: m.slug, role: classifyCodexRole(m.slug), ...(Number.isInteger(m.context_window) ? { contextWindow: m.context_window } : {}) }));
+      return models.length ? { models, version: String(d.client_version || '') } : null;
+    }
+    if (tool === 'claude') {
+      // One file per account or surface; the newest fetch wins.
+      const dir = path.join(sources.claudeDir, 'cache', 'model-catalog');
+      let newest = null;
+      for (const f of fs.readdirSync(dir)) {
+        if (!f.endsWith('.json')) continue;
+        try {
+          const d = JSON.parse(fs.readFileSync(path.join(dir, f), 'utf8'));
+          const list = d?.catalog?.config?.models;
+          if (Array.isArray(list) && list.length && (!newest || (d.fetchedAt || 0) > newest.fetchedAt)) newest = { fetchedAt: d.fetchedAt || 0, list };
+        } catch {}
+      }
+      const models = (newest?.list || [])
+        .filter(m => m && typeof m.id === 'string' && m.id)
+        .map(m => ({ id: m.id, display_name: m.name || m.id, tier: classifyClaudeTier(m.id) }));
+      return models.length ? { models, version: '' } : null;
+    }
+  } catch {}
+  return null;
+}
+
+/**
+ * Copies the tools' own lists into the catalog when they changed. A tool can update and rewrite its
+ * list without any request through the gateway, so readers of the catalog call this first.
+ */
+export function syncLocalCatalog(stateDir, sources = defaultSources()) {
+  const cache = loadCatalogCache(stateDir);
+  let changed = false;
+  for (const tool of ['claude', 'codex']) {
+    const local = readLocalToolModels(tool, sources);
+    if (!local) continue;
+    const ids = (cache[tool]?.models || []).map(m => m.id).join('\n');
+    if (ids === local.models.map(m => m.id).join('\n') && cache[tool]?.source === 'local') continue;
+    cache[tool] = { ...cache[tool], models: local.models, source: 'local' };
+    changed = true;
+  }
+  if (changed) {
+    cache.updatedAt = Date.now();
+    saveCatalogCache(stateDir, cache);
+  }
+  return cache;
+}
+
 /** Atomically writes the model catalog cache to disk */
 export function saveCatalogCache(stateDir, catalog) {
   const p = catalogCachePath(stateDir);
@@ -141,19 +206,33 @@ export async function fetchToolModels(tool, { url, apiKey, timeout = 3000 } = {}
   }
 }
 
+// A profile key is a credential for that profile's baseURL. Send it to the official model list only
+// when the profile itself points at the official host, never to a third party.
+function officialKey(profile, tool) {
+  try {
+    if (profile?.apiKey && new URL(profile.baseURL).host === new URL(OFFICIAL_MODEL_URLS[tool]).host) return profile.apiKey;
+  } catch {}
+  return undefined;
+}
+
 /**
  * Refreshes the local catalog cache with models from both official endpoints
  * and saves to the state directory.
  */
-export async function refreshCatalog(stateDir, { claudeKey, codexKey } = {}) {
+export async function refreshCatalog(stateDir, { claudeProfile, codexProfile, sources = defaultSources() } = {}) {
   const cache = loadCatalogCache(stateDir);
+  const local = { claude: readLocalToolModels('claude', sources), codex: readLocalToolModels('codex', sources) };
 
+  // A local list is the list of the account itself; the built-in names are guesses and are not added.
   const [claudeRes, codexRes] = await Promise.all([
-    fetchToolModels('claude', { apiKey: claudeKey }),
-    fetchToolModels('codex', { apiKey: codexKey })
+    local.claude || fetchToolModels('claude', { apiKey: officialKey(claudeProfile, 'claude') }),
+    local.codex || fetchToolModels('codex', { apiKey: officialKey(codexProfile, 'codex') })
   ]);
+  for (const tool of ['claude', 'codex']) {
+    if (local[tool]) cache[tool] = { ...cache[tool], models: local[tool].models, source: 'local' };
+  }
 
-  if (claudeRes.ok && claudeRes.models.length > 0) {
+  if (!local.claude && claudeRes.ok && claudeRes.models.length > 0) {
     const existing = new Set(claudeRes.models.map(m => m.id));
     // Keep any baseline models that might be absent from the API
     for (const [tier, ids] of Object.entries(BASELINE_MODELS.claude)) {
@@ -161,17 +240,17 @@ export async function refreshCatalog(stateDir, { claudeKey, codexKey } = {}) {
         if (!existing.has(id)) claudeRes.models.push({ id, tier });
       }
     }
-    cache.claude = { models: claudeRes.models };
+    cache.claude = { ...cache.claude, models: claudeRes.models, source: 'official' };
   }
 
-  if (codexRes.ok && codexRes.models.length > 0) {
+  if (!local.codex && codexRes.ok && codexRes.models.length > 0) {
     const existing = new Set(codexRes.models.map(m => m.id));
     for (const [role, ids] of Object.entries(BASELINE_MODELS.codex)) {
       for (const id of ids) {
         if (!existing.has(id)) codexRes.models.push({ id, role });
       }
     }
-    cache.codex = { models: codexRes.models };
+    cache.codex = { ...cache.codex, models: codexRes.models, source: 'official' };
   }
 
   cache.updatedAt = Date.now();
@@ -199,48 +278,59 @@ const refreshingTools = new Set();
 
 /**
  * Version-triggered auto-poll:
- * When a request arrives with a new tool version not yet seen in cache,
- * immediately records the new version and kicks off an asynchronous background refresh.
- * Subsequent requests with the same version do zero network calls.
+ * When a request arrives with a tool version that has no refreshed catalog yet, the catalog is
+ * refreshed from the tool's own list on disk, or else from the official list in the background.
+ * The version is recorded only after a refresh succeeds, so a failed refresh is tried again.
  */
-export function checkVersionAndRefresh(tool, headers, stateDir, apiKey) {
+const RETRY_AFTER_MS = 10 * 60 * 1000;
+const failedAttempts = new Map();
+
+export function checkVersionAndRefresh(tool, headers, stateDir, apiKey, sources = defaultSources()) {
   const version = detectToolVersion(headers, tool);
   if (!version) return;
 
   const cache = loadCatalogCache(stateDir);
-  const toolEntry = cache[tool] || {};
-  const lastVersion = toolEntry.lastSeenVersion || '';
+  if (version === (cache[tool]?.lastSeenVersion || '')) return;
 
-  if (version !== lastVersion) {
-    toolEntry.lastSeenVersion = version;
-    cache[tool] = toolEntry;
+  const local = readLocalToolModels(tool, sources);
+  if (local) {
+    cache[tool] = { ...cache[tool], lastSeenVersion: version, models: local.models, source: 'local' };
+    cache.updatedAt = Date.now();
     saveCatalogCache(stateDir, cache);
+    console.log(`[llm-switcher:catalog] Detected ${tool} ${version} -> model catalog read from the tool (${local.models.length} models)`);
+    return;
+  }
+
+  // Without a local list every request would call the network again; wait between failed attempts.
+  const failed = failedAttempts.get(tool);
+  if (failed && failed.version === version && Date.now() - failed.at < RETRY_AFTER_MS) return;
+  if (refreshingTools.has(tool)) return;
 
-    if (!refreshingTools.has(tool)) {
-      refreshingTools.add(tool);
-      fetchToolModels(tool, { apiKey })
-        .then(res => {
-          if (res.ok && res.models.length > 0) {
-            const fresh = loadCatalogCache(stateDir);
-            const existing = new Set(res.models.map(m => m.id));
-            const baseline = BASELINE_MODELS[tool] || {};
-            for (const [, ids] of Object.entries(baseline)) {
-              for (const id of ids) {
-                if (!existing.has(id)) {
-                  res.models.push({ id, ...(tool === 'claude' ? { tier: classifyClaudeTier(id) } : { role: classifyCodexRole(id) }) });
-                }
-              }
-            }
-            fresh[tool] = { lastSeenVersion: version, models: res.models };
-            fresh.updatedAt = Date.now();
-            saveCatalogCache(stateDir, fresh);
-            console.log(`[llm-switcher:catalog] Detected ${tool} version update to ${version} -> refreshed model catalog (${res.models.length} models)`);
+  refreshingTools.add(tool);
+  fetchToolModels(tool, { apiKey })
+    .then(res => {
+      if (!res.ok || res.models.length === 0) {
+        failedAttempts.set(tool, { version, at: Date.now() });
+        console.log(`[llm-switcher:catalog] ${tool} ${version}: no local model list, official list failed (${res.error || 'empty'})`);
+        return;
+      }
+      const fresh = loadCatalogCache(stateDir);
+      const existing = new Set(res.models.map(m => m.id));
+      const baseline = BASELINE_MODELS[tool] || {};
+      for (const [, ids] of Object.entries(baseline)) {
+        for (const id of ids) {
+          if (!existing.has(id)) {
+            res.models.push({ id, ...(tool === 'claude' ? { tier: classifyClaudeTier(id) } : { role: classifyCodexRole(id) }) });
           }
-        })
-        .catch(() => {})
-        .finally(() => {
-          refreshingTools.delete(tool);
-        });
-    }
-  }
+        }
+      }
+      fresh[tool] = { ...fresh[tool], lastSeenVersion: version, models: res.models, source: 'official' };
+      fresh.updatedAt = Date.now();
+      saveCatalogCache(stateDir, fresh);
+      console.log(`[llm-switcher:catalog] Detected ${tool} version update to ${version} -> refreshed model catalog (${res.models.length} models)`);
+    })
+    .catch(() => {})
+    .finally(() => {
+      refreshingTools.delete(tool);
+    });
 }

package/classifier.mjs

@@ -0,0 +1,238 @@
+/**
+ * Jev-powered Semantic Classifier & Router for llm-switcher
+ * Evaluates prompt complexity using TypeSafe System One (Jev) models
+ * and maps intent to optimal model tier (haiku, sonnet, opus).
+ */
+
+export const DEFAULT_JEV_URL = 'https://api.typesafe.ai/v1/systemone';
+export const DEFAULT_JEV_MODEL = 'jev-latest';
+export const OPENROUTER_JEV_URL = 'https://openrouter.ai/api/alpha/decisions';
+export const OPENROUTER_JEV_MODEL = 'typesafe/jev-1.13';
+
+export const TIER_CRITERIA = {
+  haiku: 'Mechanical task, template fill, single fact lookup, extraction, binary decision, short formatting',
+  sonnet: 'Multi-step coding, refactor, search across files, general programming, document drafting',
+  opus: 'Architecture design, security analysis, deep reasoning, race conditions, migration plans, complex synthesis',
+};
+
+/**
+ * Fast synchronous heuristic fallback when offline or no API key is set.
+ */
+export function heuristicClassify(prompt = '') {
+  const text = String(prompt || '').toLowerCase();
+
+  // Heavy reasoning or safety critical indicators -> opus
+  const opusKeywords = [
+    'architecture', 'security audit', 'race condition', 'deadlock', 'concurrency',
+    'threat model', 'penetration', 'migration plan', 'adversarial', 'formal verification'
+  ];
+  if (opusKeywords.some((kw) => text.includes(kw))) {
+    return 'opus';
+  }
+
+  // Trivial or mechanical indicators -> haiku
+  const haikuKeywords = [
+    'translate', 'format json', 'extract', 'regex', 'rename', 'single word',
+    'yes or no', 'spell check', 'fix typo', 'convert to csv'
+  ];
+  if (text.length < 120 && haikuKeywords.some((kw) => text.includes(kw))) {
+    return 'haiku';
+  }
+
+  // Default balanced tier
+  return 'sonnet';
+}
+
+/**
+ * Resolves Jev API key from environment.
+ */
+export function findJevKey(env = process.env) {
+  if (typeof env?.TYPESAFE_API_KEY === 'string' && env.TYPESAFE_API_KEY.trim().length > 0) {
+    return { key: env.TYPESAFE_API_KEY.trim(), source: 'typesafe' };
+  }
+  if (typeof env?.JEV_API_KEY === 'string' && env.JEV_API_KEY.trim().length > 0) {
+    return { key: env.JEV_API_KEY.trim(), source: 'openrouter' };
+  }
+  return null;
+}
+
+/**
+ * Classifies prompt into optimal model tier using TypeSafe Jev.
+ * Falls back safely to heuristic on missing key, timeout, or network error.
+ */
+export async function classifyPrompt({
+  prompt,
+  apiKey,
+  url,
+  model,
+  timeoutMs = 5000,
+  env = process.env,
+  fetchImpl = globalThis.fetch,
+} = {}) {
+  const fallbackTier = heuristicClassify(prompt);
+
+  const keyInfo = apiKey ? { key: apiKey, source: 'custom' } : findJevKey(env);
+  if (!keyInfo) {
+    return {
+      tier: fallbackTier,
+      confidence: 0.5,
+      source: 'heuristic',
+      reason: 'no-key',
+    };
+  }
+
+  const endpointUrl = url || (keyInfo.source === 'openrouter' ? OPENROUTER_JEV_URL : DEFAULT_JEV_URL);
+  const modelName = model || (keyInfo.source === 'openrouter' ? OPENROUTER_JEV_MODEL : DEFAULT_JEV_MODEL);
+
+  const controller = new AbortController();
+  const timer = setTimeout(() => controller.abort(), timeoutMs);
+
+  try {
+    const payload = {
+      state: String(prompt || ''),
+      model: modelName,
+      questions: {
+        recommended_tier: {
+          type: 'choice',
+          instructions: 'Determine the optimal model tier for this task according to complexity and reasoning depth required.',
+          criteria: TIER_CRITERIA,
+        },
+      },
+    };
+
+    const res = await fetchImpl(endpointUrl, {
+      method: 'POST',
+      headers: {
+        Authorization: `Bearer ${keyInfo.key}`,
+        'Content-Type': 'application/json',
+      },
+      body: JSON.stringify(payload),
+      signal: controller.signal,
+    });
+
+    clearTimeout(timer);
+
+    if (!res.ok) {
+      return {
+        tier: fallbackTier,
+        confidence: 0.5,
+        source: 'heuristic',
+        reason: `http-${res.status}`,
+      };
+    }
+
+    const data = await res.json();
+    const ans = data?.answers?.recommended_tier;
+    if (!ans || typeof ans.choice !== 'string') {
+      return {
+        tier: fallbackTier,
+        confidence: 0.5,
+        source: 'heuristic',
+        reason: 'bad-response',
+      };
+    }
+
+    const tier = ['haiku', 'sonnet', 'opus'].includes(ans.choice) ? ans.choice : fallbackTier;
+    const confidence = typeof ans.confidence === 'number' ? Number(ans.confidence.toFixed(3)) : 0.8;
+
+    return {
+      tier,
+      confidence,
+      distribution: ans.distribution || null,
+      source: 'jev',
+      model: data.model || modelName,
+    };
+  } catch (err) {
+    clearTimeout(timer);
+    const reason = err?.name === 'AbortError' ? 'timeout' : 'network';
+    return {
+      tier: fallbackTier,
+      confidence: 0.5,
+      source: 'heuristic',
+      reason,
+    };
+  }
+}
+
+/**
+ * Checks semantic equivalence between candidate text and cached target using Jev Noul.
+ * Optional gate: returns false immediately if Jev is not configured or unavailable.
+ */
+export async function checkSemanticEquivalence({
+  candidate,
+  target,
+  apiKey,
+  url,
+  model,
+  timeoutMs = 3000,
+  env = process.env,
+  fetchImpl = globalThis.fetch,
+} = {}) {
+  if (!candidate || !target) return { equivalent: false, confidence: 0, reason: 'empty-input' };
+  if (candidate.trim() === target.trim()) {
+    return { equivalent: true, confidence: 1.0, reason: 'exact-match' };
+  }
+
+  const keyInfo = apiKey ? { key: apiKey, source: 'custom' } : findJevKey(env);
+  if (!keyInfo) {
+    return { equivalent: false, confidence: 0, reason: 'no-key-fallback-bypass' };
+  }
+
+  const endpointUrl = url || (keyInfo.source === 'openrouter' ? OPENROUTER_JEV_URL : DEFAULT_JEV_URL);
+  const modelName = model || (keyInfo.source === 'openrouter' ? OPENROUTER_JEV_MODEL : DEFAULT_JEV_MODEL);
+
+  const controller = new AbortController();
+  const timer = setTimeout(() => controller.abort(), timeoutMs);
+
+  try {
+    const payload = {
+      state: `REQUEST_A:\n${candidate}\n\nREQUEST_B:\n${target}`,
+      model: modelName,
+      questions: {
+        is_equivalent: {
+          type: 'noul',
+          instructions: 'Do REQUEST_A and REQUEST_B ask for the exact same semantic task or answer, such that the response to B fully satisfies A?',
+        },
+      },
+    };
+
+    const res = await fetchImpl(endpointUrl, {
+      method: 'POST',
+      headers: {
+        Authorization: `Bearer ${keyInfo.key}`,
+        'Content-Type': 'application/json',
+      },
+      body: JSON.stringify(payload),
+      signal: controller.signal,
+    });
+
+    clearTimeout(timer);
+    if (!res.ok) {
+      return { equivalent: false, confidence: 0, reason: `http-${res.status}` };
+    }
+
+    const data = await res.json();
+    const ans = data?.answers?.is_equivalent;
+    if (ans == null) {
+      return { equivalent: false, confidence: 0, reason: 'bad-response' };
+    }
+
+    // Noul returns either probability number directly or object with probability
+    const prob = typeof ans === 'number' ? ans : (typeof ans?.probability === 'number' ? ans.probability : 0);
+    const equivalent = prob >= 0.85;
+
+    return {
+      equivalent,
+      confidence: Number(prob.toFixed(3)),
+      reason: equivalent ? 'jev-confirmed' : 'below-threshold',
+      model: data.model || modelName,
+    };
+  } catch (err) {
+    clearTimeout(timer);
+    return {
+      equivalent: false,
+      confidence: 0,
+      reason: err?.name === 'AbortError' ? 'timeout' : 'network-error',
+    };
+  }
+}