@openchambery/web 1.18.2-beta.2 → 1.18.2-beta.21

package/server/lib/small-model/DOCUMENTATION.md

@@ -1,23 +1,30 @@
 # Small Model
 
-Server-side direct LLM calls that reuse the user's existing OpenCode provider
-logins (`~/.local/share/opencode/auth.json`). OpenCode uses a "small model"
-internally (titles, summaries) but does not expose it through the SDK or
-plugins — this module replicates that mechanism as an OpenChamber runtime API.
+Server-side LLM calls that reuse the user's existing OpenCode provider logins
+(`~/.local/share/opencode/auth.json`). OpenCode uses a "small model" internally
+(titles, summaries) but does not expose it through the SDK or plugins — this
+module replicates that mechanism as an OpenChamber runtime API.
 
 ## Security boundary
 
 Credentials never leave the server process. The client sends only a prompt;
-auth resolution, OAuth refresh, and provider dispatch all happen server-side.
-Routes live under `/api/*` and are gated by the ui-auth middleware like every
-other runtime API.
+auth resolution, OAuth refresh, and provider dispatch all happen server-side
+(either in dedicated adapters or inside the OpenCode runtime for the session
+path). Routes live under `/api/*` and are gated by the ui-auth middleware like
+every other runtime API.
 
 ## Files
 
 - `index.js` — `createSmallModelService(dependencies)` factory. Production
   obtains a single lazy instance from `server/index.js` (`getSmallModelService`)
   and injects it into feature routes, session assist/title/goal, and scheduled
-  tasks. Tests may call the factory with a mock `getModelCatalog`.
+  tasks. Tests may call the factory with a mock `getModelCatalog`. Returns
+  `stop()` to remove the lazy temp directory used by the session path (wired
+  from `server/index.js` `stop()`).
+- `opencode-session.js` — temporary OpenCode session path for non-dedicated
+  providers. Lazily creates `mkdtemp('openchamber-smallmodel-')` with a hidden
+  deny-all agent, then create → archive → `prompt_async` → poll idle → read
+  assistant text → best-effort `session.delete`.
 - `resolve.js` — model selection, mirroring OpenCode's `getSmallModel` chain:
   0. OpenChamber's own settings override (Settings → Sessions → Small Model):
      when `smallModelUseDefault` is `false`, `smallModelOverride`
@@ -27,23 +34,30 @@ other runtime API.
   1. `small_model` from the merged OpenCode config layers (`provider/model`).
   2. Family-priority scan (`gemini-flash` → `gpt-nano` → `claude-haiku`)
      **within the session's provider first** (`preferredProviderID`, like
-     OpenCode resolves within the current provider), then the session model,
-     then other authenticated providers' catalog families, then the four
-     minimal hardcoded candidates.
+     OpenCode resolves within the current provider), then keyword tiers on that
+     provider's catalog, then the session model, then other authenticated
+     providers' catalog families, then keyword+cost default across providers,
+     then the four minimal hardcoded candidates.
   3. GitHub Copilot hidden utility models (`gpt-*-nano/mini`) — these never
      appear in the catalog, so they participate as the `gpt-nano` family entry
      and as a final utility fallback.
   4. Session model (`preferredModelID`) when the session provider has auth but
-     no small family match.
-  Auth-type constraints for the four minimal providers: OpenAI OAuth →
+     no small family/keyword match.
+  Auth-type constraints for the four dedicated providers: OpenAI OAuth →
   hardcoded codex-small; OpenAI API key → catalog families; Anthropic/Google
   require API keys; Copilot supports auth aliases (`copilot` / `github-copilot`).
+
+  **Default ranking (no explicit config):** candidates are catalog models from
+  usable auth entries. Keyword tiers (tokenized id / known family):
+  `flash` → `nano` → `haiku` → `mini` → `lite` → `turbo` → `instant` →
+  `small` → `chat`. Within a tier: `cost.input` ascending (missing cost last),
+  then newer `release_date`.
 - Input clamp: the prompt is truncated to the resolved model's catalog
   `limit.context` (minus an output reserve, ~4 chars/token estimate;
   conservative default when the model is not in the catalog). Truncation is
   reported as `inputTruncated: true` in the response.
-- `call.js` — wire formats and per-provider auth, replicating OpenCode's
-  plugin auth loaders:
+- `call.js` — dedicated wire formats and per-provider auth for the four
+  adapters (replicating OpenCode plugin auth loaders):
   - **GitHub Copilot**: OpenAI-compatible `/chat/completions` on
     `https://api.githubcopilot.com` (or `copilot-api.<enterprise>`) with the
     stored device-OAuth token as the bearer — no token exchange, no expiry.
@@ -53,11 +67,12 @@ other runtime API.
     `auth.openai.com` (single-flight) and written back to `auth.json`.
   - **Anthropic** (`type: api`): `/v1/messages` with `x-api-key`.
   - **Google** (`type: api`): `generateContent` with `x-goog-api-key`.
-  - Everything else: OpenAI-compatible `/chat/completions`. Base URL order:
-    (1) `provider.<id>.options.baseURL` in the OpenCode merged config,
-    (2) the target model's `api.url` from the directory catalog (OpenCode SDK),
-    (3) known provider constants (`https://api.openai.com/v1` for openai).
-    Never uses provider-level models.dev `api` fields.
+  - Custom Summary AI mode: OpenAI-compatible `/chat/completions` against the
+    user-supplied base URL + token.
+  Generic `api.url` chat-completions for arbitrary providers is **not** used
+  for dispatch anymore (plugin providers need runtime rewrite); those providers
+  take the OpenCode session path instead. The helper remains in `call.js` for
+  the dedicated adapters and custom mode.
 - `catalog.js` — directory-scoped provider catalog via official
   `@opencode-ai/sdk/v2` `client.config.providers()`. Base URL/auth come from
   the composition root (`buildOpenCodeUrl` / `getOpenCodeAuthHeaders`).
@@ -65,8 +80,9 @@ other runtime API.
   an **explicit** minimal fallback catalog (OpenAI OAuth / GitHub Copilot /
   Google API / Anthropic API candidates only) — never an authoritative empty
   map. Raw small-model fields kept: `family`, `release_date`, `limit`,
-  `model.api.url`, provider `name`. Client-safe provider catalog projection
-  remains in `opencode/provider-catalog.js` for `/api/config/catalog/providers`.
+  `cost.input` / `cost.output`, `model.api.url`, provider `name`. Client-safe
+  provider catalog projection remains in `opencode/provider-catalog.js` for
+  `/api/config/catalog/providers`.
 - `routes.js` — `GET /api/small-model` (resolution preview) and
   `POST /api/small-model/generate` (`{ prompt, system?, maxOutputTokens?,
   model?, directory? }` → `{ text, providerID, modelID, source }`).
@@ -74,10 +90,57 @@ other runtime API.
   consumed by Settings → Summary AI. Callable lists receive the same
   `directory` as describe/generate so they share the directory catalog.
 
+## Dispatch
+
+| Mode / provider | Path |
+|---|---|
+| Summary AI `custom` | Direct OpenAI-compatible call (`call.js`) |
+| `openai` / `anthropic` / `google` / `github-copilot` with usable dedicated auth | Direct dedicated adapter (`call.js`) |
+| Any other usable auth + catalog model (plugin providers, …) | Temporary OpenCode session (`opencode-session.js`) |
+
+Callable Settings lists include every `isUsableAuthEntry` provider that has at
+least one catalog model id (no `api.url` gate). OpenAI OAuth still surfaces
+only `gpt-5.4-mini`; Copilot only `gpt-5.4-nano`; Copilot auth aliases merge
+to `github-copilot`.
+
+## OpenCode session lifecycle
+
+1. Lazy singleton temp directory under `os.tmpdir()` /
+   `openchamber-smallmodel-*` with `.opencode/agent/openchamber-smallmodel.md`
+   (`hidden: true`, `permissions: [{ action: "*", resource: "*", effect: deny }]`).
+2. `session.create` with title `[small-model] <purpose>`, agent name, and
+   `metadata.openchamber.smallModel = { purpose }` (sidebar invisibility also
+   depends on a parallel filter lane for that marker).
+3. Immediate `session.update({ time: { archived } })` (same pattern as
+   assistants / scheduled tasks) so ordinary lists never flash the session.
+4. `session.promptAsync` with `{ model: { providerID, modelID }, agent, system?,
+   parts: [{ type: 'text', text, synthetic: false }] }`.
+5. Poll `session.status` + `session.messages` until idle assistant completed
+   (budget `SESSION_SETTLE_TIMEOUT_MS` = 60s). Network / timeout / empty text
+   all throw with context — never an empty success.
+6. `finally`: best-effort `session.delete`. Delete failure is logged and does
+   not erase a successful text result.
+7. Server `stop()` calls `smallModelServiceInstance.stop()` to `rm` the temp
+   directory.
+
+### Skills / clean context boundary
+
+OpenCode always loads skill *guidance* via `SkillInstructions.load` into the
+system prompt (`packages/core/src/session/context.ts`). Availability is filtered
+by `Permission.evaluate("skill", skill.id, agent.permissions)` — a deny-all
+`action: "*"` rule therefore excludes every skill from the available list, so
+the guidance renders as "No skills are currently available." Skill **body**
+content is only injected when the skill tool runs; deny-all also blocks that
+tool. There is no separate agent/directory flag to disable skill guidance
+beyond permissions. Global `~/.agents/skills` may still be *discovered* by the
+OpenCode instance, but they are not listed or loadable under this agent.
+Project-level skills are absent because the temp directory has none.
+
 ## Registration
 
-`server/index.js` creates one lazy `getSmallModelService` and injects it into
-session assist/title/goal, scheduled tasks, and `createFeatureRoutesRuntime`.
+`server/index.js` creates one lazy `getSmallModelService` and injects
+`{ buildOpenCodeUrl, getOpenCodeAuthHeaders }` into the factory (and into
+session assist/title/goal, scheduled tasks, and `createFeatureRoutesRuntime`).
 Feature routes no longer import `small-model/index.js` top-level exports
 directly.
 
@@ -96,14 +159,6 @@ calls use the same effective default shown by Settings: authenticated OpenAI
 when available, otherwise the first callable provider/model. This effective
 default remains authoritative when the active session uses another provider.
 
-Provider mode uses direct provider APIs from the server rather than sending a
-chat request through the active OpenCode session. The dispatcher has dedicated
-wire formats for GitHub Copilot, OpenAI OAuth, Anthropic API keys, and Google
-API keys. Catalog providers with an OpenAI-compatible chat-completions endpoint
-use the generic dispatcher. Provider plugins that require a region, resource
-name, browser session, or another provider-specific credential chain require a
-dedicated adapter. Custom mode is an explicit OpenAI-compatible API client.
-
 ## Known limitations
 
 - OpenCode's free models (`opencode/big-pickle`, `*-free`) work without a
@@ -113,12 +168,12 @@ dedicated adapter. Custom mode is an explicit OpenAI-compatible API client.
   a session on an unauthenticated `opencode` provider falls through to the
   global scan (or a clean 404 on a vanilla setup with no logins).
 
-- Anthropic OAuth (Claude Pro/Max) entries are not supported — OpenCode itself
-  keeps those outside `auth.json` in this generation; only `type: api` keys
-  work for Anthropic.
-- Amazon Bedrock, GitLab, Azure and other credential-chain providers are out
-  of scope; they need more than a key/token (regions, resource names).
+- Anthropic OAuth (Claude Pro/Max) entries are not supported on the dedicated
+  adapter — OpenCode itself keeps those outside `auth.json` in this generation;
+  only `type: api` keys work for Anthropic direct calls. Plugin / other auth
+  shapes that OpenCode can already run may still succeed via the session path.
 - Responses from the codex backend are collected from the SSE stream; the
   endpoint itself is non-streaming by design (small utility calls).
 - The server never requests models.dev. When OpenCode is down, only the four
-  minimal hardcoded candidates remain available.
+  minimal hardcoded candidates remain available for resolution; session-path
+  generation still requires a live OpenCode runtime.

package/server/lib/small-model/catalog.js

@@ -48,8 +48,9 @@ const isRecord = (value) => value !== null && typeof value === 'object' && !Arra
 
 /**
  * Convert the official SDK provider list into the internal small-model catalog
- * shape. Keeps family / release_date / limit / model.api.url / provider name
- * for resolve + call; does not project the client-safe catalog allowlist.
+ * shape. Keeps family / release_date / limit / cost.input|output /
+ * model.api.url / provider name for resolve + call; does not project the
+ * client-safe catalog allowlist.
  */
 export function toSmallModelCatalog(source) {
   if (!isRecord(source) || !Array.isArray(source.providers)) {
@@ -77,6 +78,13 @@ export function toSmallModelCatalog(source) {
           if (Number.isFinite(model.limit.output)) limit.output = model.limit.output;
           if (Object.keys(limit).length > 0) entry.limit = limit;
         }
+        // cost.input/output feed default small-model ranking (cheapest first).
+        if (isRecord(model.cost)) {
+          const cost = Object.create(null);
+          if (Number.isFinite(model.cost.input)) cost.input = model.cost.input;
+          if (Number.isFinite(model.cost.output)) cost.output = model.cost.output;
+          if (Object.keys(cost).length > 0) entry.cost = cost;
+        }
         if (isRecord(model.api) && typeof model.api.url === 'string' && model.api.url.trim()) {
           entry.api = { url: model.api.url.trim() };
         }

package/server/lib/small-model/catalog.test.js

@@ -2,7 +2,7 @@ import { describe, expect, it } from 'vitest';
 import { createModelCatalogLoader, toSmallModelCatalog, MINIMAL_FALLBACK_CATALOG } from './catalog.js';
 
 describe('toSmallModelCatalog', () => {
-  it('keeps family, release_date, limit, model.api.url, and provider name', () => {
+  it('keeps family, release_date, limit, cost, model.api.url, and provider name', () => {
     const catalog = toSmallModelCatalog({
       providers: [{
         id: 'google',
@@ -14,6 +14,7 @@ describe('toSmallModelCatalog', () => {
             family: 'gemini-flash',
             release_date: '2025-06-01',
             limit: { context: 1_000_000, output: 8192 },
+            cost: { input: 0.15, output: 0.6, cache_read: 0.01 },
             api: { id: 'gemini-2.5-flash', url: 'https://generativelanguage.googleapis.com', npm: '@ai-sdk/google' },
           },
         },
@@ -30,6 +31,7 @@ describe('toSmallModelCatalog', () => {
           family: 'gemini-flash',
           release_date: '2025-06-01',
           limit: { context: 1_000_000, output: 8192 },
+          cost: { input: 0.15, output: 0.6 },
           api: { url: 'https://generativelanguage.googleapis.com' },
         },
       },

package/server/lib/small-model/index.js

@@ -6,6 +6,7 @@ import { readConfigLayers } from '../opencode/shared.js';
 import { createModelCatalogLoader, getCatalogProvider } from './catalog.js';
 import { resolveSmallModel, parseModelRef, isUsableAuthEntry, getAuthEntryForProvider } from './resolve.js';
 import { callSmallModel } from './call.js';
+import { generateViaOpenCodeSession, stop as stopOpenCodeSessionTemp } from './opencode-session.js';
 
 const OPENCHAMBER_SETTINGS_FILE = path.join(
   process.env.OPENCHAMBER_DATA_DIR
@@ -74,21 +75,13 @@ const clampPromptToModelLimit = ({ prompt, catalog, providerID, modelID }) => {
   return { prompt: `${prompt.slice(0, maxChars)}…`, truncated: true };
 };
 
-// Dedicated adapters (or OpenAI-compatible models that publish api.url) are
-// the only providers the small-model dispatcher can call without models.dev.
-// Auth-type constraints must match resolve/call: OpenAI api|oauth; Anthropic/
-// Google api only; Copilot via github-copilot/copilot alias token.
-const DEDICATED_CALLABLE_PROVIDERS = new Set(['openai', 'anthropic', 'google', 'github-copilot']);
-
-const providerHasCallableEndpoint = (provider) => {
-  if (!provider || typeof provider !== 'object') return false;
-  return Object.values(provider.models || {}).some(
-    (model) => typeof model?.api?.url === 'string' && model.api.url.trim().length > 0,
-  );
-};
+// Dedicated adapters keep the direct wire-format path. Everything else
+// (plugin providers, credential-chain, missing api.url, …) goes through a
+// temporary OpenCode session so auth/endpoint rewrite stay in the runtime.
+const DEDICATED_DIRECT_PROVIDERS = new Set(['openai', 'anthropic', 'google', 'github-copilot']);
 
 // Mirrors pickWithinProvider / callSmallModel auth gates so Settings pickers
-// never list providers that would only fail at dispatch time.
+// never list dedicated providers that would only fail at dispatch time.
 const isCallableDedicatedAuth = (auth, providerID) => {
   if (providerID === 'openai') {
     const entry = auth?.openai;
@@ -108,17 +101,29 @@ const isCallableDedicatedAuth = (auth, providerID) => {
   return false;
 };
 
+const providerHasCatalogModels = (provider) => {
+  if (!provider || typeof provider !== 'object') return false;
+  return Object.values(provider.models || {}).some(
+    (model) => typeof model?.id === 'string' && model.id.trim().length > 0,
+  );
+};
+
+const shouldUseDirectAdapter = (providerID, auth) => {
+  if (!DEDICATED_DIRECT_PROVIDERS.has(providerID)) return false;
+  return isCallableDedicatedAuth(auth, providerID);
+};
+
 const listCallableProviderIDsForState = (auth, catalog) => {
   const ids = new Set(
     Object.keys(auth || {}).filter((providerID) => {
       // Surface Copilot only as github-copilot (alias handled below).
       if (providerID === 'copilot') return false;
-      if (DEDICATED_CALLABLE_PROVIDERS.has(providerID)) {
+      if (DEDICATED_DIRECT_PROVIDERS.has(providerID)) {
         return isCallableDedicatedAuth(auth, providerID);
       }
       if (!isUsableAuthEntry(auth[providerID])) return false;
       const provider = getCatalogProvider(catalog, providerID);
-      return providerHasCallableEndpoint(provider);
+      return providerHasCatalogModels(provider);
     }),
   );
   if (isUsableAuthEntry(getAuthEntryForProvider(auth, 'github-copilot'))) {
@@ -181,10 +186,12 @@ const readConfiguredSmallModel = (workingDirectory) => {
  * }} dependencies
  */
 export function createSmallModelService(dependencies) {
+  const buildOpenCodeUrl = dependencies.buildOpenCodeUrl;
+  const getOpenCodeAuthHeaders = dependencies.getOpenCodeAuthHeaders;
   const getModelCatalog = dependencies.getModelCatalog
     || createModelCatalogLoader({
-      buildOpenCodeUrl: dependencies.buildOpenCodeUrl,
-      getOpenCodeAuthHeaders: dependencies.getOpenCodeAuthHeaders,
+      buildOpenCodeUrl,
+      getOpenCodeAuthHeaders,
     }).getModelCatalog;
 
   /**
@@ -273,17 +280,33 @@ export function createSmallModelService(dependencies) {
       modelID: resolved.modelID,
     });
 
-    const text = await callSmallModel({
-      auth,
-      catalog,
-      workingDirectory: directory,
-      providerID: resolved.providerID,
-      modelID: resolved.modelID,
-      prompt: clamped.prompt,
-      system: summaryPrompt || (typeof system === 'string' && system.trim() ? system.trim() : undefined),
-      maxOutputTokens,
-      custom: summaryCustom,
-    });
+    const effectiveSystem = summaryPrompt || (typeof system === 'string' && system.trim() ? system.trim() : undefined);
+
+    let text;
+    if (summaryCustom || shouldUseDirectAdapter(resolved.providerID, auth)) {
+      text = await callSmallModel({
+        auth,
+        catalog,
+        workingDirectory: directory,
+        providerID: resolved.providerID,
+        modelID: resolved.modelID,
+        prompt: clamped.prompt,
+        system: effectiveSystem,
+        maxOutputTokens,
+        custom: summaryCustom,
+      });
+    } else {
+      text = await generateViaOpenCodeSession({
+        buildOpenCodeUrl,
+        getOpenCodeAuthHeaders,
+        providerID: resolved.providerID,
+        modelID: resolved.modelID,
+        prompt: clamped.prompt,
+        system: effectiveSystem,
+        purpose: typeof purpose === 'string' && purpose.trim() ? purpose.trim() : 'generate',
+        directory,
+      });
+    }
 
     return {
       text: text.trim(),
@@ -341,5 +364,6 @@ export function createSmallModelService(dependencies) {
     listCallableProviders,
     listCallableModels,
     describeSmallModel,
+    stop: stopOpenCodeSessionTemp,
   };
 }