smoltalk 0.5.0 → 0.6.0

package/README.md

@@ -8,6 +8,15 @@ Smoltalk exposes a common API to different LLM providers, with built-in cost tra
 pnpm install smoltalk
 ```
 
+> **Upgrading to 0.6.x?** The flat API-key/host fields on `SmolConfig` have
+> been removed in favor of nested `apiKey` and `baseUrl` maps. Migration:
+> ```diff
+> -{ openAiApiKey: "sk-...",   googleApiKey: "...", ollamaHost: "http://..." }
+> +{ apiKey: { openAi: "sk-...", google: "..." }, baseUrl: { ollama: "http://..." } }
+> ```
+> Env-var fallbacks are unchanged (`OPENAI_API_KEY`, `GEMINI_API_KEY`,
+> `ANTHROPIC_API_KEY`, `OLLAMA_HOST`).
+
 ## Hello world example
 
 ```typescript
@@ -68,8 +77,10 @@ const messages = [
 const resp = await text({
   messages,
   model: "gemini-2.0-flash-lite",
-  openAiApiKey: process.env.OPENAI_API_KEY || "",
-  googleApiKey: process.env.GEMINI_API_KEY || "",
+  apiKey: {
+    openAi: process.env.OPENAI_API_KEY || "",
+    google: process.env.GEMINI_API_KEY || "",
+  },
   logLevel: "debug",
 });
 ```
@@ -80,8 +91,10 @@ If you want to construct a client once and reuse it across many calls, use `getC
 import { getClient, userMessage } from "smoltalk";
 
 const client = getClient({
-  openAiApiKey: process.env.OPENAI_API_KEY || "",
-  googleApiKey: process.env.GEMINI_API_KEY || "",
+  apiKey: {
+    openAi: process.env.OPENAI_API_KEY || "",
+    google: process.env.GEMINI_API_KEY || "",
+  },
   model: "gemini-2.0-flash-lite",
 });
 
@@ -147,12 +160,9 @@ A couple of design decisions to note:
 |--------|------|-------------|
 | `model` | `ModelName` | **Required.** The model to use (e.g. `"gpt-4o"`, `"gemini-2.0-flash-lite"`). |
 | `messages` | `Message[]` | **Required.** The conversation messages to send. |
-| `openAiApiKey` | `string` | OpenAI API key. |
-| `googleApiKey` | `string` | Google Gemini API key. |
-| `anthropicApiKey` | `string` | Anthropic API key. |
-| `ollamaApiKey` | `string` | Ollama API key (only needed for cloud Ollama). |
-| `ollamaHost` | `string` | Ollama host URL (for self-hosted or cloud Ollama). |
-| `provider` | `Provider` | Override provider detection. One of `"openai"`, `"openai-responses"`, `"google"`, `"ollama"`, `"anthropic"`, or any provider registered via `registerProvider()`. |
+| `apiKey` | `{ openAi?, google?, anthropic?, ollama?, openRouter?, deepInfra?, liteLlm?, openAiCompat? }` | API keys, nested by provider. Each falls back to its conventional env var (`OPENAI_API_KEY`, `GEMINI_API_KEY`, `ANTHROPIC_API_KEY`, `OPENROUTER_API_KEY`, `DEEPINFRA_API_KEY`, `LITELLM_API_KEY`, `OPENAI_COMPAT_API_KEY`). Ollama has no env-var fallback for the key. |
+| `baseUrl` | `{ ollama?, openRouter?, deepInfra?, liteLlm?, openAiCompat? }` | Custom base URLs. `ollama` defaults to `$OLLAMA_HOST` then `http://localhost:11434`; `openRouter`/`deepInfra` defaults are baked in; `liteLlm`/`openAiCompat` require an explicit URL (or `LITELLM_BASE_URL` / `OPENAI_COMPAT_BASE_URL` env). |
+| `provider` | `Provider` | Override provider detection. One of `"openai"`, `"openai-responses"`, `"google"`, `"ollama"`, `"anthropic"`, `"openrouter"`, `"deepinfra"`, `"litellm"`, `"openai-compat"`, or any provider registered via `registerProvider()`. |
 | `logLevel` | `LogLevel` | Logging verbosity: `"debug"`, `"info"`, `"warn"`, `"error"`. |
 | `tools` | `{ name, description?, schema }[]` | Tool definitions. `schema` is a Zod object schema. |
 | `responseFormat` | `ZodType` | Zod schema for structured output. The response is parsed and validated against this schema. |
@@ -194,6 +204,53 @@ Detects when the model is stuck in a repetitive tool-call loop.
 | `intervention` | `string` | Action to take: `"remove-tool"`, `"remove-all-tools"`, `"throw-error"`, or `"halt-execution"`. |
 | `excludeTools` | `string[]` | Tool names to ignore when counting calls. |
 
+## Hosted OpenAI-compatible providers
+
+Smoltalk ships four built-in providers for hosted open-source models that all
+speak the OpenAI chat-completions shape. Use these when you want to run a
+Llama, GLM, Qwen, etc. via someone else's hosted infrastructure without
+adding a new dependency. You must pass `provider:` explicitly because these
+model ids aren't in the smoltalk registry.
+
+| `provider:` | What it is | Required config | Cost source |
+|-------------|------------|-----------------|-------------|
+| `"openrouter"` | OpenRouter.ai aggregator | `apiKey.openRouter` (or `OPENROUTER_API_KEY`) | `usage.cost` (auto-enabled by injecting `usage:{include:true}`) |
+| `"deepinfra"` | DeepInfra hosted models | `apiKey.deepInfra` (or `DEEPINFRA_API_KEY`) | `usage.estimated_cost` |
+| `"litellm"`   | Your own LiteLLM proxy   | `apiKey.liteLlm` + `baseUrl.liteLlm` (or `LITELLM_API_KEY` / `LITELLM_BASE_URL`) | `x-litellm-response-cost` header (non-stream only) |
+| `"openai-compat"` | Any OpenAI-shape backend (vLLM, TGI, LM Studio…) | `apiKey.openAiCompat` + `baseUrl.openAiCompat` (or `OPENAI_COMPAT_API_KEY` / `OPENAI_COMPAT_BASE_URL`) | Best-effort: reads `usage.cost`/`estimated_cost`/`cost_usd` if present |
+
+```ts
+import { textSync, userMessage } from "smoltalk";
+
+const r = await textSync({
+  model: "z-ai/glm-5.2",
+  provider: "openrouter",
+  apiKey: { openRouter: process.env.OPENROUTER_API_KEY! },
+  messages: [userMessage("hi")],
+});
+// r.value.cost.totalCost is a real OpenRouter-reported USD cost.
+```
+
+**Capability matrix:**
+
+|                 | chat | embeddings | image generation | `web_search` hosted tool |
+|-----------------|------|------------|------------------|--------------------------|
+| `openrouter`    | ✅   | ❌         | ❌               | ✅ (via `:online` / web plugin) |
+| `deepinfra`     | ✅   | ✅         | ❌ (uses per-model endpoints, not OpenAI shape) | ❌ |
+| `litellm`       | ✅   | ✅         | ✅ (if the upstream model supports it) | ✅ (if upstream supports it) |
+| `openai-compat` | ✅   | ✅         | ✅ (backend-dependent) | depends on backend |
+
+Smoltalk surfaces a clear `failure(...)` from `embed()`/`image()` for the
+unsupported combinations rather than silently dropping the call.
+
+**Running a local LiteLLM proxy:**
+
+```bash
+pip install 'litellm[proxy]'
+litellm --model openai/gpt-4o
+# In your code: baseUrl: { liteLlm: "http://localhost:4000" }
+```
+
 ## Refreshing model data
 
 Smoltalk ships a baked-in model registry (pricing, context limits, capabilities).

package/dist/client.d.ts

@@ -4,6 +4,10 @@ export * from "./clients/openai.js";
 export * from "./clients/openaiResponses.js";
 export * from "./clients/baseClient.js";
 export * from "./clients/ollama.js";
+export * from "./clients/openaiCompat.js";
+export * from "./clients/openrouter.js";
+export * from "./clients/deepinfra.js";
+export * from "./clients/litellm.js";
 import { BaseClient } from "./clients/baseClient.js";
 import { SmolClientConfig } from "./types.js";
 export declare function registerProvider(providerName: string, clientClass: typeof BaseClient): void;

package/dist/client.js

@@ -4,11 +4,19 @@ export * from "./clients/openai.js";
 export * from "./clients/openaiResponses.js";
 export * from "./clients/baseClient.js";
 export * from "./clients/ollama.js";
+export * from "./clients/openaiCompat.js";
+export * from "./clients/openrouter.js";
+export * from "./clients/deepinfra.js";
+export * from "./clients/litellm.js";
 import { SmolAnthropic } from "./clients/anthropic.js";
 import { SmolGoogle } from "./clients/google.js";
 import { SmolOllama } from "./clients/ollama.js";
 import { SmolOpenAi } from "./clients/openai.js";
 import { SmolOpenAiResponses } from "./clients/openaiResponses.js";
+import { SmolOpenAiCompat } from "./clients/openaiCompat.js";
+import { SmolOpenRouter } from "./clients/openrouter.js";
+import { SmolDeepInfra } from "./clients/deepinfra.js";
+import { SmolLiteLlm } from "./clients/litellm.js";
 import { getModel, isTextModel } from "./models.js";
 import { SmolError } from "./smolError.js";
 import { resolveApiKey, resolveProvider } from "./util/provider.js";
@@ -36,43 +44,56 @@ export function getClient(config) {
             throw new SmolError(`Only text models are supported currently. ${modelName} is a ${model?.type} model.`);
         }
     }
+    // Resolve API keys from config or env vars and inject them back into the
+    // nested apiKey map. The client constructors also read env vars directly,
+    // but this lets getClient surface a clear error before construction.
     const resolvedKeys = {
-        openAiApiKey: resolveApiKey("openai", config),
-        googleApiKey: resolveApiKey("google", config),
-        anthropicApiKey: resolveApiKey("anthropic", config),
+        openAi: resolveApiKey("openai", config),
+        google: resolveApiKey("google", config),
+        anthropic: resolveApiKey("anthropic", config),
+        ollama: resolveApiKey("ollama", config),
+        openRouter: resolveApiKey("openrouter", config),
+        deepInfra: resolveApiKey("deepinfra", config),
+        liteLlm: resolveApiKey("litellm", config),
+        openAiCompat: resolveApiKey("openai-compat", config),
     };
     const clientConfig = {
         messages: [],
         ...config,
-        ...resolvedKeys,
+        apiKey: { ...resolvedKeys, ...config.apiKey },
         model: modelName,
     };
     switch (provider) {
         case "anthropic":
-            if (!resolvedKeys.anthropicApiKey) {
-                throw new SmolError("No Anthropic API key provided. Please provide an Anthropic API key in the config using anthropicApiKey, or set the ANTHROPIC_API_KEY environment variable.");
+            if (!resolvedKeys.anthropic) {
+                throw new SmolError("No Anthropic API key provided. Please provide an Anthropic API key in the config using apiKey.anthropic, or set the ANTHROPIC_API_KEY environment variable.");
             }
-            return new SmolAnthropic({
-                ...clientConfig,
-                anthropicApiKey: resolvedKeys.anthropicApiKey,
-            });
+            return new SmolAnthropic(clientConfig);
         case "openai":
-            if (!resolvedKeys.openAiApiKey) {
-                throw new SmolError("No OpenAI API key provided. Please provide an OpenAI API key in the config using openAiApiKey, or set the OPENAI_API_KEY environment variable.");
+            if (!resolvedKeys.openAi) {
+                throw new SmolError("No OpenAI API key provided. Please provide an OpenAI API key in the config using apiKey.openAi, or set the OPENAI_API_KEY environment variable.");
             }
             return new SmolOpenAi(clientConfig);
         case "openai-responses":
-            if (!resolvedKeys.openAiApiKey) {
-                throw new SmolError("No OpenAI API key provided. Please provide an OpenAI API key in the config using openAiApiKey, or set the OPENAI_API_KEY environment variable.");
+            if (!resolvedKeys.openAi) {
+                throw new SmolError("No OpenAI API key provided. Please provide an OpenAI API key in the config using apiKey.openAi, or set the OPENAI_API_KEY environment variable.");
             }
             return new SmolOpenAiResponses(clientConfig);
         case "google":
-            if (!resolvedKeys.googleApiKey) {
-                throw new SmolError("No Google API key provided. Please provide a Google API key in the config using googleApiKey, or set the GEMINI_API_KEY environment variable.");
+            if (!resolvedKeys.google) {
+                throw new SmolError("No Google API key provided. Please provide a Google API key in the config using apiKey.google, or set the GEMINI_API_KEY environment variable.");
             }
             return new SmolGoogle(clientConfig);
         case "ollama":
             return new SmolOllama(clientConfig);
+        case "openrouter":
+            return new SmolOpenRouter(clientConfig);
+        case "deepinfra":
+            return new SmolDeepInfra(clientConfig);
+        case "litellm":
+            return new SmolLiteLlm(clientConfig);
+        case "openai-compat":
+            return new SmolOpenAiCompat(clientConfig);
         default:
             if (provider in registeredProviders) {
                 const ClientClass = registeredProviders[provider];

package/dist/clients/anthropic.d.ts

@@ -3,9 +3,7 @@ import { BaseClient } from "./baseClient.js";
 import { ModelName } from "../models.js";
 export declare function anthropicWebSearchEntries(hostedTools?: string[]): any[];
 export declare function parseAnthropicHostedTools(response: any, provider: string): HostedToolResult[];
-export type SmolAnthropicConfig = SmolConfig & {
-    anthropicApiKey: string;
-};
+export type SmolAnthropicConfig = SmolConfig;
 export declare class SmolAnthropic extends BaseClient implements SmolClient {
     private client;
     private logger;

package/dist/clients/anthropic.js

@@ -149,7 +149,11 @@ export class SmolAnthropic extends BaseClient {
     model;
     constructor(config) {
         super(config);
-        this.client = new Anthropic({ apiKey: config.anthropicApiKey });
+        const apiKey = config.apiKey?.anthropic || process.env.ANTHROPIC_API_KEY;
+        if (!apiKey) {
+            throw new Error("Anthropic API key is required for SmolAnthropic client.");
+        }
+        this.client = new Anthropic({ apiKey });
         this.logger = getLogger();
         this.model = new Model(config.model, undefined, config.modelData);
     }

package/dist/clients/baseClient.js

@@ -56,7 +56,7 @@ export class BaseClient {
         const messageLimitResult = this.checkMessageLimit(promptConfig);
         if (messageLimitResult)
             return messageLimitResult;
-        const hostedError = validateHostedTools(promptConfig.hostedTools, promptConfig.model, promptConfig.modelData);
+        const hostedError = validateHostedTools(promptConfig.hostedTools, promptConfig.model, promptConfig.provider, promptConfig.modelData);
         if (hostedError) {
             return failure(hostedError);
         }
@@ -269,7 +269,7 @@ export class BaseClient {
             };
             return;
         }
-        const hostedError = validateHostedTools(config.hostedTools, config.model, config.modelData);
+        const hostedError = validateHostedTools(config.hostedTools, config.model, config.provider, config.modelData);
         if (hostedError) {
             yield { type: "error", error: hostedError };
             return;

package/dist/clients/deepinfra.d.ts

@@ -0,0 +1,17 @@
+import { SmolOpenAiCompat } from "./openaiCompat.js";
+import type { SmolConfig } from "../types.js";
+/**
+ * DeepInfra client (https://deepinfra.com). OpenAI-compatible chat completions
+ * at the `/v1/openai` base URL.
+ *
+ * - Baked base URL `https://api.deepinfra.com/v1/openai` (override via config.baseUrl.deepInfra).
+ * - Key: config.apiKey.deepInfra or env DEEPINFRA_API_KEY.
+ * - Cost: reads `usage.estimated_cost` (USD).
+ */
+export declare class SmolDeepInfra extends SmolOpenAiCompat {
+    protected resolveClientOptions(config: SmolConfig): {
+        apiKey: string;
+        baseURL: string;
+    };
+    protected resolveCostUsd(usage: any): number | undefined;
+}

package/dist/clients/deepinfra.js

@@ -0,0 +1,27 @@
+import { SmolOpenAiCompat } from "./openaiCompat.js";
+import { resolveApiKey, resolveBaseUrl } from "../util/provider.js";
+/**
+ * DeepInfra client (https://deepinfra.com). OpenAI-compatible chat completions
+ * at the `/v1/openai` base URL.
+ *
+ * - Baked base URL `https://api.deepinfra.com/v1/openai` (override via config.baseUrl.deepInfra).
+ * - Key: config.apiKey.deepInfra or env DEEPINFRA_API_KEY.
+ * - Cost: reads `usage.estimated_cost` (USD).
+ */
+export class SmolDeepInfra extends SmolOpenAiCompat {
+    resolveClientOptions(config) {
+        const apiKey = resolveApiKey("deepinfra", config);
+        // resolveBaseUrl bakes in the deepinfra default, so this is always defined.
+        const baseURL = resolveBaseUrl("deepinfra", config);
+        if (!apiKey) {
+            throw new Error("deepinfra: API key required (config.apiKey.deepInfra or DEEPINFRA_API_KEY).");
+        }
+        return { apiKey, baseURL };
+    }
+    resolveCostUsd(usage) {
+        // DeepInfra returns usage.estimated_cost (USD) on chat completions.
+        return typeof usage?.estimated_cost === "number"
+            ? usage.estimated_cost
+            : undefined;
+    }
+}

package/dist/clients/google.js

@@ -68,10 +68,11 @@ export class SmolGoogle extends BaseClient {
     model;
     constructor(config) {
         super(config);
-        if (!config.googleApiKey) {
+        const apiKey = config.apiKey?.google || process.env.GEMINI_API_KEY;
+        if (!apiKey) {
             throw new Error("Google API key is required for SmolGoogle client.");
         }
-        this.client = new GoogleGenAI({ apiKey: config.googleApiKey });
+        this.client = new GoogleGenAI({ apiKey });
         this.logger = getLogger();
         this.model = new Model(config.model, undefined, config.modelData);
     }

package/dist/clients/litellm.d.ts

@@ -0,0 +1,19 @@
+import { SmolOpenAiCompat } from "./openaiCompat.js";
+import type { SmolConfig } from "../types.js";
+/**
+ * LiteLLM proxy client (https://docs.litellm.ai/docs/simple_proxy). OpenAI-
+ * compatible chat completions through a user-run LiteLLM proxy.
+ *
+ * - Base URL is REQUIRED from config.baseUrl.liteLlm or env LITELLM_BASE_URL.
+ * - Key: config.apiKey.liteLlm or env LITELLM_API_KEY.
+ * - Cost: reads the `x-litellm-response-cost` header (USD). Only available
+ *   for non-streaming responses; streaming requests fall back to the
+ *   smoltalk model registry.
+ */
+export declare class SmolLiteLlm extends SmolOpenAiCompat {
+    protected resolveClientOptions(config: SmolConfig): {
+        apiKey: string;
+        baseURL: string;
+    };
+    protected resolveCostUsd(_usage: any, rawResponse?: Response): number | undefined;
+}

package/dist/clients/litellm.js

@@ -0,0 +1,32 @@
+import { SmolOpenAiCompat } from "./openaiCompat.js";
+import { resolveApiKey, resolveBaseUrl } from "../util/provider.js";
+/**
+ * LiteLLM proxy client (https://docs.litellm.ai/docs/simple_proxy). OpenAI-
+ * compatible chat completions through a user-run LiteLLM proxy.
+ *
+ * - Base URL is REQUIRED from config.baseUrl.liteLlm or env LITELLM_BASE_URL.
+ * - Key: config.apiKey.liteLlm or env LITELLM_API_KEY.
+ * - Cost: reads the `x-litellm-response-cost` header (USD). Only available
+ *   for non-streaming responses; streaming requests fall back to the
+ *   smoltalk model registry.
+ */
+export class SmolLiteLlm extends SmolOpenAiCompat {
+    resolveClientOptions(config) {
+        const apiKey = resolveApiKey("litellm", config);
+        const baseURL = resolveBaseUrl("litellm", config);
+        if (!apiKey) {
+            throw new Error("litellm: API key required (config.apiKey.liteLlm or LITELLM_API_KEY).");
+        }
+        if (!baseURL) {
+            throw new Error("litellm: base URL required (config.baseUrl.liteLlm or LITELLM_BASE_URL).");
+        }
+        return { apiKey, baseURL };
+    }
+    resolveCostUsd(_usage, rawResponse) {
+        const header = rawResponse?.headers?.get?.("x-litellm-response-cost");
+        if (!header)
+            return undefined;
+        const n = Number(header);
+        return Number.isFinite(n) ? n : undefined;
+    }
+}

package/dist/clients/ollama.js

@@ -4,6 +4,7 @@ import { getLogger } from "../util/logger.js";
 import { success, } from "../types.js";
 import { zodToGoogleTool } from "../util/tool.js";
 import { sanitizeAttributes } from "../util/util.js";
+import { resolveBaseUrl } from "../util/provider.js";
 import { BaseClient } from "./baseClient.js";
 import { SmolContextWindowExceededError, smolErrorForStatus, } from "../smolError.js";
 import { extractHttpErrorFields } from "../util/httpError.js";
@@ -17,14 +18,15 @@ export class SmolOllama extends BaseClient {
         super(config);
         this.logger = getLogger();
         this.model = new Model(config.model, undefined, config.modelData);
-        if (config.ollamaApiKey) {
+        const apiKey = config.apiKey?.ollama;
+        if (apiKey) {
             this.client = new Ollama({
                 host: "https://cloud.ollama.com",
-                headers: { Authorization: "Bearer " + config.ollamaApiKey },
+                headers: { Authorization: "Bearer " + apiKey },
             });
         }
         else {
-            const host = config.ollamaHost || DEFAULT_OLLAMA_HOST;
+            const host = resolveBaseUrl("ollama", config) || DEFAULT_OLLAMA_HOST;
             this.client = new Ollama({ host });
         }
     }

package/dist/clients/openai.d.ts

@@ -1,16 +1,52 @@
 import OpenAI from "openai";
-import { PromptResult, Result, SmolClient, SmolConfig, StreamChunk } from "../types.js";
+import { PromptResult, Result, SmolClient, SmolConfig, StreamChunk, HostedToolResult } from "../types.js";
+import { EgonLog } from "../util/logger.js";
 import { BaseClient } from "./baseClient.js";
 import { ModelName } from "../models.js";
+import { Model } from "../model.js";
+import { CostEstimate, TokenUsage } from "../types.js";
 export type SmolOpenAiConfig = SmolConfig;
 export declare class SmolOpenAi extends BaseClient implements SmolClient {
-    private client;
-    private logger;
-    private model;
+    protected client: OpenAI;
+    protected logger: EgonLog;
+    protected model: Model;
     constructor(config: SmolOpenAiConfig);
+    /**
+     * Build the `new OpenAI({...})` options. Subclasses override to inject a
+     * different baseURL or to pull the key from a different config field.
+     *
+     * Default behavior: read OPENAI_API_KEY (or config.apiKey.openAi). Throws
+     * if the key is missing — subclasses with required keys do their own check.
+     */
+    protected resolveClientOptions(config: SmolConfig): {
+        apiKey?: string;
+        baseURL?: string;
+    };
+    /**
+     * Inspect the provider-returned usage (and optionally the raw HTTP response)
+     * for a USD cost figure. Returns `undefined` if the provider didn't surface
+     * one, in which case `calculateUsageAndCost` falls back to the smoltalk
+     * model registry. Subclasses (OpenRouter, DeepInfra, LiteLLM) override this.
+     */
+    protected resolveCostUsd(_usage: any, _rawResponse?: Response): number | undefined;
+    /**
+     * Extra request body fields injected on every call. Subclasses override to
+     * add provider-specific request shapes (e.g. OpenRouter's `usage: { include: true }`
+     * or its `plugins: [{ id: "web" }]` for web search). Merged into the request
+     * after the standard params so it can override them.
+     */
+    protected buildRequestExtras(_config: SmolConfig): Record<string, unknown>;
+    /**
+     * Extract provider-specific hosted-tool results (e.g. OpenRouter web_search
+     * annotations) from a completion. Subclasses override; default returns none.
+     */
+    protected parseHostedToolResults(_completion: any, _config: SmolConfig): HostedToolResult[];
     getClient(): OpenAI;
     getModel(): ModelName;
-    private calculateUsageAndCost;
+    protected calculateUsageAndCost(usageData: any, rawResponse?: Response): {
+        usage?: TokenUsage;
+        cost?: CostEstimate;
+    };
     private buildRequest;
     private rethrowAsSmolError;
     _textSync(config: SmolConfig): Promise<Result<PromptResult>>;

package/dist/clients/openai.js

@@ -14,20 +14,57 @@ export class SmolOpenAi extends BaseClient {
     model;
     constructor(config) {
         super(config);
-        if (!config.openAiApiKey) {
-            throw new Error("OpenAI API key is required for SmolOpenAi client.");
-        }
-        this.client = new OpenAI({ apiKey: config.openAiApiKey });
+        const options = this.resolveClientOptions(config);
+        this.client = new OpenAI(options);
         this.logger = getLogger();
         this.model = new Model(config.model, undefined, config.modelData);
     }
+    /**
+     * Build the `new OpenAI({...})` options. Subclasses override to inject a
+     * different baseURL or to pull the key from a different config field.
+     *
+     * Default behavior: read OPENAI_API_KEY (or config.apiKey.openAi). Throws
+     * if the key is missing — subclasses with required keys do their own check.
+     */
+    resolveClientOptions(config) {
+        const apiKey = config.apiKey?.openAi || process.env.OPENAI_API_KEY;
+        if (!apiKey) {
+            throw new Error("OpenAI API key is required for SmolOpenAi client.");
+        }
+        return { apiKey };
+    }
+    /**
+     * Inspect the provider-returned usage (and optionally the raw HTTP response)
+     * for a USD cost figure. Returns `undefined` if the provider didn't surface
+     * one, in which case `calculateUsageAndCost` falls back to the smoltalk
+     * model registry. Subclasses (OpenRouter, DeepInfra, LiteLLM) override this.
+     */
+    resolveCostUsd(_usage, _rawResponse) {
+        return undefined;
+    }
+    /**
+     * Extra request body fields injected on every call. Subclasses override to
+     * add provider-specific request shapes (e.g. OpenRouter's `usage: { include: true }`
+     * or its `plugins: [{ id: "web" }]` for web search). Merged into the request
+     * after the standard params so it can override them.
+     */
+    buildRequestExtras(_config) {
+        return {};
+    }
+    /**
+     * Extract provider-specific hosted-tool results (e.g. OpenRouter web_search
+     * annotations) from a completion. Subclasses override; default returns none.
+     */
+    parseHostedToolResults(_completion, _config) {
+        return [];
+    }
     getClient() {
         return this.client;
     }
     getModel() {
         return this.model.getModel();
     }
-    calculateUsageAndCost(usageData) {
+    calculateUsageAndCost(usageData, rawResponse) {
         let usage;
         let cost;
         if (usageData) {
@@ -40,9 +77,23 @@ export class SmolOpenAi extends BaseClient {
             if (cached > 0) {
                 usage.cachedInputTokens = cached;
             }
-            const calculatedCost = this.model.calculateCost(usage);
-            if (calculatedCost) {
-                cost = calculatedCost;
+            // Prefer provider-supplied cost when available (e.g. OpenRouter
+            // usage.cost, DeepInfra usage.estimated_cost, LiteLLM header).
+            // Fall back to the smoltalk model-registry calculation.
+            const providerCost = this.resolveCostUsd(usageData, rawResponse);
+            if (typeof providerCost === "number") {
+                cost = {
+                    inputCost: 0,
+                    outputCost: 0,
+                    totalCost: providerCost,
+                    currency: "USD",
+                };
+            }
+            else {
+                const calculatedCost = this.model.calculateCost(usage);
+                if (calculatedCost) {
+                    cost = calculatedCost;
+                }
             }
         }
         return { usage, cost };
@@ -61,6 +112,7 @@ export class SmolOpenAi extends BaseClient {
                 reasoning_effort: config.reasoningEffort,
             }),
             ...sanitizeAttributes(config.rawAttributes),
+            ...this.buildRequestExtras(config),
         };
         if (config.responseFormat) {
             request.response_format = {
@@ -92,11 +144,16 @@ export class SmolOpenAi extends BaseClient {
         this.statelogClient?.promptRequest(request);
         const signal = this.getAbortSignal(config);
         let completion;
+        let rawResponse;
         try {
-            completion = await this.client.chat.completions.create({
+            const result = await this.client.chat.completions
+                .create({
                 ...request,
                 stream: false,
-            }, { ...(signal && { signal }) });
+            }, { ...(signal && { signal }) })
+                .withResponse();
+            completion = result.data;
+            rawResponse = result.response;
         }
         catch (error) {
             this.rethrowAsSmolError(error);
@@ -123,14 +180,17 @@ export class SmolOpenAi extends BaseClient {
                 }
             }
         }
-        // Extract usage and calculate cost
-        const { usage, cost } = this.calculateUsageAndCost(completion.usage);
+        // Extract usage and calculate cost. rawResponse lets subclasses read
+        // response headers (e.g. LiteLLM's x-litellm-response-cost).
+        const { usage, cost } = this.calculateUsageAndCost(completion.usage, rawResponse);
+        const hostedToolResults = this.parseHostedToolResults(completion, config);
         return success({
             output,
             toolCalls,
             usage,
             cost,
             model: this.getModel(),
+            ...(hostedToolResults.length > 0 ? { hostedToolResults } : {}),
         });
     }
     async *_textStream(config) {
@@ -156,6 +216,9 @@ export class SmolOpenAi extends BaseClient {
         for await (const chunk of completion) {
             // Extract usage from the final chunk
             if (chunk.usage) {
+                // Header-based cost (LiteLLM) is unsupported while streaming.
+                // Body-based cost (OpenRouter/DeepInfra) still works because it
+                // comes through chunk.usage.
                 const usageAndCost = this.calculateUsageAndCost(chunk.usage);
                 usage = usageAndCost.usage;
                 cost = usageAndCost.cost;

package/dist/clients/openaiCompat.d.ts

@@ -0,0 +1,22 @@
+import { SmolOpenAi } from "./openai.js";
+import type { SmolConfig } from "../types.js";
+/**
+ * Generic OpenAI-compatible client. Use when pointing smoltalk at any
+ * arbitrary OpenAI-shaped endpoint (vLLM, TGI, LM Studio, OpenLLM, etc.).
+ *
+ * Both `config.apiKey.openAiCompat` and `config.baseUrl.openAiCompat` are
+ * required (or their env-var fallbacks: `OPENAI_COMPAT_API_KEY`,
+ * `OPENAI_COMPAT_BASE_URL`).
+ *
+ * Cost: opportunistically reads `usage.cost`, `usage.estimated_cost`, or
+ * `usage.cost_usd` if the backend includes one of them. Otherwise falls back
+ * to the smoltalk model registry (which won't have these custom models, so
+ * cost stays undefined — that's expected for arbitrary backends).
+ */
+export declare class SmolOpenAiCompat extends SmolOpenAi {
+    protected resolveClientOptions(config: SmolConfig): {
+        apiKey: string;
+        baseURL: string;
+    };
+    protected resolveCostUsd(usage: any): number | undefined;
+}