smoltalk 0.13.2 → 0.14.1

package/README.md

@@ -250,7 +250,7 @@ model ids aren't in the smoltalk registry.
 
 | `provider:` | What it is | Required config | Cost source |
 |-------------|------------|-----------------|-------------|
-| `"openrouter"` | OpenRouter.ai aggregator | `apiKey.openRouter` (or `OPENROUTER_API_KEY`) | `usage.cost` (auto-enabled by injecting `usage:{include:true}`) |
+| `"openrouter"` | OpenRouter.ai aggregator | `apiKey.openRouter` (or `OPENROUTER_API_KEY`) | `usage.cost` |
 | `"deepinfra"` | DeepInfra hosted models | `apiKey.deepInfra` (or `DEEPINFRA_API_KEY`) | `usage.estimated_cost` |
 | `"litellm"`   | Your own LiteLLM proxy   | `apiKey.liteLlm` + `baseUrl.liteLlm` (or `LITELLM_API_KEY` / `LITELLM_BASE_URL`) | `x-litellm-response-cost` header (non-stream only) |
 | `"openai-compat"` | Any OpenAI-shape backend (vLLM, TGI, LM Studio…) | `apiKey.openAiCompat` + `baseUrl.openAiCompat` (or `OPENAI_COMPAT_API_KEY` / `OPENAI_COMPAT_BASE_URL`) | Best-effort: reads `usage.cost`/`estimated_cost`/`cost_usd` if present |
@@ -276,7 +276,7 @@ const r = await textSync({
 | `deepinfra`     | ✅   | ✅         | ❌ (uses per-model endpoints, not OpenAI shape) | ❌ |
 | `litellm`       | ✅   | ✅         | ✅ (if the upstream model supports it) | ✅ (if upstream supports it) |
 | `openai-compat` | ✅   | ✅         | ✅ (backend-dependent) | depends on backend |
-| `mlx`           | ✅   | ❌         | ❌               | ❌ |
+| `mlx`           | ✅   | ✅         | ❌               | ❌ |
 
 Smoltalk surfaces a clear `failure(...)` from `embed()`/`image()` for the
 unsupported combinations rather than silently dropping the call.
@@ -543,6 +543,11 @@ const { resolveModel } = await loadLlamaCpp({
 const modelPath = await resolveModel("hf:org/repo/model.gguf", "/models/cache");
 ```
 
+`embed()` with `provider: "llama-cpp"` and a local `.gguf` path works the
+same way and needs `smoltalk-llama-cpp` >= 0.5.0; the vector is computed in
+process. See that package's README for the embedding caveats (dimension
+truncation, one model file per role).
+
 ## Audio (STT/TTS)
 
 Three audio primitives. `transcribe()` (speech-to-text) and `speak()`
@@ -556,6 +561,10 @@ that exposes OpenAI-shaped `/audio/*` endpoints, use the generic
 **`openai-compat`** provider with `baseUrl` (mirrors the chat client). Anthropic,
 OpenRouter, and Ollama have no audio endpoints and return a `Failure`.
 
+`speak()` also supports **`mlx`**: a local MLX speech server at `baseUrl.mlx`,
+`MLX_BASE_URL`, or `http://127.0.0.1:8080/v1`. No API key, no retries, zero
+cost, and `"wav"` by default.
+
 ```ts
 // example: skip-typecheck
 // Groq STT (OpenAI-compatible; provider inferred from the model)
@@ -650,7 +659,11 @@ if (result.success) {
 `tts-1` and `tts-1-hd` are the only baked-in models in v1. `voice` is
 required. Options: `format` (OpenAI accepts `"mp3"` | `"opus"` | `"aac"` |
 `"flac"` | `"wav"` | `"pcm"`, default `"mp3"`; a custom provider may accept
-other strings) and `speed`. Limits are declared per model in the registry —
+other strings), `speed`, and `instructions`. `instructions` is free-text
+guidance on how the speech should sound (`"Alarmed and urgent."`). It is passed
+through as-is by the OpenAI-shaped providers (`openai`, `groq`, `openai-compat`,
+`mlx`); whether the model reads it is up to the model (`gpt-4o-mini-tts` does,
+`tts-1` does not). The `google` provider ignores it. Limits are declared per model in the registry —
 for `tts-1`/`tts-1-hd` that's a 4096-code-point input cap, a 0.25–4.0 speed
 range, and the format list above; exceeding any of them returns a `Failure`
 before the request is sent. The returned `audio` is a `Uint8Array` you own —

package/dist/clients/google.js

@@ -260,6 +260,9 @@ export class SmolGoogle extends BaseClient {
             });
         });
         const genConfig = {};
+        if (config.maxTokens !== undefined) {
+            genConfig.maxOutputTokens = config.maxTokens;
+        }
         if (systemParts.length > 0) {
             genConfig.systemInstruction = systemParts.join("\n");
         }

package/dist/clients/llamaCppLoader.d.ts

@@ -1,4 +1,5 @@
 import type { BaseClient } from "./baseClient.js";
+import type { EmbedProvider } from "../embed.js";
 /**
  * Minimal structural view of smoltalk-llama-cpp's module. Declared here (not
  * imported from the plugin) so smoltalk compiles without the plugin installed
@@ -7,6 +8,9 @@ import type { BaseClient } from "./baseClient.js";
 export type LlamaCppModule = {
     LlamaCPP: typeof BaseClient;
     resolveModel: (uriOrPath: string, cacheDir: string) => Promise<string>;
+    /** Present from smoltalk-llama-cpp 0.5.0. Absent on older plugins, which
+     *  then serve chat only. */
+    embed?: EmbedProvider;
 };
 type ImportFn = (specifier: string) => Promise<Record<string, unknown>>;
 /**

package/dist/clients/ollama.js

@@ -85,6 +85,9 @@ export class SmolOllama extends BaseClient {
         if (config.responseFormat) {
             request.format = responseFormatToJsonSchema(config.responseFormat);
         }
+        if (config.maxTokens !== undefined) {
+            request.options = { num_predict: config.maxTokens };
+        }
         Object.assign(request, sanitizeAttributes(config.rawAttributes));
         this.logger.debug("Sending request to Ollama:", JSON.stringify(redactAttachments(request), null, 2));
         this.statelogClient?.promptRequest(request);
@@ -151,6 +154,9 @@ export class SmolOllama extends BaseClient {
         if (config.responseFormat) {
             request.format = responseFormatToJsonSchema(config.responseFormat);
         }
+        if (config.maxTokens !== undefined) {
+            request.options = { num_predict: config.maxTokens };
+        }
         Object.assign(request, sanitizeAttributes(config.rawAttributes));
         this.logger.debug("Sending streaming request to Ollama:", JSON.stringify(redactAttachments(request), null, 2));
         this.statelogClient?.promptRequest(request);

package/dist/clients/openai.d.ts

@@ -32,11 +32,18 @@ export declare class SmolOpenAi extends BaseClient implements SmolClient {
     protected resolveCostUsd(_usage: any, _rawResponse?: Response): number | undefined;
     /**
      * Extra request body fields injected on every call. Subclasses override to
-     * add provider-specific request shapes (e.g. OpenRouter's `usage: { include: true }`
-     * or its `plugins: [{ id: "web" }]` for web search). Merged into the request
+     * add provider-specific request shapes (e.g. OpenRouter's
+     * `plugins: [{ id: "web" }]` for web search). Merged into the request
      * after the standard params so it can override them.
      */
     protected buildRequestExtras(_config: SmolConfig): Record<string, unknown>;
+    /**
+     * The request field carrying `config.maxTokens`. OpenAI deprecated
+     * `max_tokens` in favor of `max_completion_tokens`, and its reasoning models
+     * reject `max_tokens` outright. Compat servers override this to send
+     * `max_tokens`, which is the field they reliably accept.
+     */
+    protected maxTokensParam(config: SmolConfig): Record<string, number>;
     /**
      * Extract provider-specific hosted-tool results (e.g. OpenRouter web_search
      * annotations) from a completion. Subclasses override; default returns none.

package/dist/clients/openai.js

@@ -51,13 +51,25 @@ export class SmolOpenAi extends BaseClient {
     }
     /**
      * Extra request body fields injected on every call. Subclasses override to
-     * add provider-specific request shapes (e.g. OpenRouter's `usage: { include: true }`
-     * or its `plugins: [{ id: "web" }]` for web search). Merged into the request
+     * add provider-specific request shapes (e.g. OpenRouter's
+     * `plugins: [{ id: "web" }]` for web search). Merged into the request
      * after the standard params so it can override them.
      */
     buildRequestExtras(_config) {
         return {};
     }
+    /**
+     * The request field carrying `config.maxTokens`. OpenAI deprecated
+     * `max_tokens` in favor of `max_completion_tokens`, and its reasoning models
+     * reject `max_tokens` outright. Compat servers override this to send
+     * `max_tokens`, which is the field they reliably accept.
+     */
+    maxTokensParam(config) {
+        if (config.maxTokens === undefined) {
+            return {};
+        }
+        return { max_completion_tokens: config.maxTokens };
+    }
     /**
      * Extract provider-specific hosted-tool results (e.g. OpenRouter web_search
      * annotations) from a completion. Subclasses override; default returns none.
@@ -126,6 +138,7 @@ export class SmolOpenAi extends BaseClient {
             ...(config.reasoningEffort && {
                 reasoning_effort: config.reasoningEffort,
             }),
+            ...this.maxTokensParam(config),
             ...sanitizeAttributes(config.rawAttributes),
             ...this.buildRequestExtras(config),
         };

package/dist/clients/openaiCompat.d.ts

@@ -20,5 +20,6 @@ export declare class SmolOpenAiCompat extends SmolOpenAi {
         apiKey: string;
         baseURL: string;
     };
+    protected maxTokensParam(config: SmolConfig): Record<string, number>;
     protected resolveCostUsd(usage: any): number | undefined;
 }

package/dist/clients/openaiCompat.js

@@ -30,6 +30,14 @@ export class SmolOpenAiCompat extends SmolOpenAi {
         }
         return { apiKey, baseURL };
     }
+    // Compat servers (OpenRouter, DeepInfra, LiteLLM, vLLM, mlx_lm.server) all
+    // accept `max_tokens`; `max_completion_tokens` support is patchy.
+    maxTokensParam(config) {
+        if (config.maxTokens === undefined) {
+            return {};
+        }
+        return { max_tokens: config.maxTokens };
+    }
     resolveCostUsd(usage) {
         // Try the three common conventions across OpenAI-compatible providers.
         const c = usage?.cost ?? usage?.estimated_cost ?? usage?.cost_usd;

package/dist/clients/openrouter.d.ts

@@ -5,9 +5,8 @@ import type { SmolConfig, HostedToolResult } from "../types.js";
  *
  * - Baked base URL `https://openrouter.ai/api/v1` (override via config.baseUrl.openRouter).
  * - Key: config.apiKey.openRouter or env OPENROUTER_API_KEY.
- * - Cost: reads `usage.cost` (USD). OpenRouter only returns this when
- *   `usage: { include: true }` is set in the request body, which this client
- *   injects automatically via buildRequestExtras.
+ * - Cost: reads `usage.cost` (USD). OpenRouter includes this in every
+ *   response (in the final SSE chunk when streaming); no request flag needed.
  * - Hosted web_search: when config.hostedTools includes "web_search",
  *   injects `plugins: [{ id: "web", max_results: 5 }]` and parses
  *   message.annotations into a HostedToolResult.

package/dist/clients/openrouter.js

@@ -6,9 +6,8 @@ import { resolveApiKey, resolveBaseUrl } from "../util/provider.js";
  *
  * - Baked base URL `https://openrouter.ai/api/v1` (override via config.baseUrl.openRouter).
  * - Key: config.apiKey.openRouter or env OPENROUTER_API_KEY.
- * - Cost: reads `usage.cost` (USD). OpenRouter only returns this when
- *   `usage: { include: true }` is set in the request body, which this client
- *   injects automatically via buildRequestExtras.
+ * - Cost: reads `usage.cost` (USD). OpenRouter includes this in every
+ *   response (in the final SSE chunk when streaming); no request flag needed.
  * - Hosted web_search: when config.hostedTools includes "web_search",
  *   injects `plugins: [{ id: "web", max_results: 5 }]` and parses
  *   message.annotations into a HostedToolResult.
@@ -27,8 +26,7 @@ export class SmolOpenRouter extends SmolOpenAiCompat {
         return typeof usage?.cost === "number" ? usage.cost : undefined;
     }
     buildRequestExtras(config) {
-        // OpenRouter only returns usage.cost when this is set.
-        const extras = { usage: { include: true } };
+        const extras = {};
         if (config.hostedTools?.includes("web_search")) {
             extras.plugins = [{ id: "web", max_results: 5 }];
         }

package/dist/embed/mlx.d.ts

@@ -0,0 +1,15 @@
+import { EmbedConfig, EmbedResult } from "../embed.js";
+import { Result } from "../types/result.js";
+/**
+ * Embeddings from an MLX server on localhost (`agency local serve
+ * --embedding …`, or any server with an OpenAI-shaped /v1/embeddings
+ * route). The same call the OpenAI helper makes, with the two things the
+ * chat client also fixes: the key is a placeholder the server ignores, and
+ * the cost is zero. `dimensions` is passed through as-is; whether the
+ * server honours it depends on the server and model.
+ *
+ * Float encoding is requested because a base URL is given (see
+ * openaiEmbed): a local server that ignores the SDK's base64 default and
+ * returns float arrays would otherwise yield empty vectors.
+ */
+export declare function mlxEmbed(inputs: string[], config: EmbedConfig, baseURL: string): Promise<Result<EmbedResult>>;

package/dist/embed/mlx.js

@@ -0,0 +1,24 @@
+import { success } from "../types/result.js";
+import { openaiEmbed } from "./openai.js";
+/**
+ * Embeddings from an MLX server on localhost (`agency local serve
+ * --embedding …`, or any server with an OpenAI-shaped /v1/embeddings
+ * route). The same call the OpenAI helper makes, with the two things the
+ * chat client also fixes: the key is a placeholder the server ignores, and
+ * the cost is zero. `dimensions` is passed through as-is; whether the
+ * server honours it depends on the server and model.
+ *
+ * Float encoding is requested because a base URL is given (see
+ * openaiEmbed): a local server that ignores the SDK's base64 default and
+ * returns float arrays would otherwise yield empty vectors.
+ */
+export async function mlxEmbed(inputs, config, baseURL) {
+    const result = await openaiEmbed(inputs, config, "mlx-local", baseURL);
+    if (!result.success) {
+        return result;
+    }
+    return success({
+        ...result.value,
+        costEstimate: { inputCost: 0, outputCost: 0, totalCost: 0, currency: "USD" },
+    });
+}

package/dist/embed/openai.d.ts

@@ -1,9 +1,19 @@
 import { EmbedConfig, EmbedResult } from "../embed.js";
 import { Result } from "../types/result.js";
+export type OpenAiEmbedOptions = {
+    /**
+     * Wire encoding to ask the server for. Left unset, real OpenAI (no
+     * baseURL) keeps the SDK's base64 default, and every other backend gets
+     * "float": the SDK decodes an unrequested reply as base64 no matter what
+     * came back, so a server that ignores the field and returns float arrays
+     * would yield empty vectors. Set explicitly to override either default.
+     */
+    encodingFormat?: "float" | "base64";
+};
 /**
  * OpenAI-compatible embedding call. Used by openai directly and by other
  * OpenAI-shape backends (deepinfra, litellm, openai-compat) which pass a
  * custom `baseURL`. Cost comes from the smoltalk model registry; provider-
  * returned cost fields aren't standardized on this endpoint.
  */
-export declare function openaiEmbed(inputs: string[], config: EmbedConfig, apiKey: string, baseURL?: string): Promise<Result<EmbedResult>>;
+export declare function openaiEmbed(inputs: string[], config: EmbedConfig, apiKey: string, baseURL?: string, options?: OpenAiEmbedOptions): Promise<Result<EmbedResult>>;

package/dist/embed/openai.js

@@ -2,25 +2,47 @@ import OpenAI from "openai";
 import { success, failure } from "../types/result.js";
 import { getModel, isEmbeddingsModel } from "../models.js";
 import { round } from "../util/util.js";
+/**
+ * A server may answer a float request with base64 anyway (some always
+ * encode). Once we name an encoding the SDK returns the body untouched, so
+ * handle both shapes here. Little-endian float32, the same layout the SDK's
+ * own decoder assumes.
+ */
+function toFloats(embedding) {
+    if (typeof embedding === "string") {
+        const bytes = Buffer.from(embedding, "base64");
+        const view = new Float32Array(bytes.buffer, bytes.byteOffset, Math.floor(bytes.byteLength / 4));
+        return Array.from(view);
+    }
+    return embedding;
+}
 /**
  * OpenAI-compatible embedding call. Used by openai directly and by other
  * OpenAI-shape backends (deepinfra, litellm, openai-compat) which pass a
  * custom `baseURL`. Cost comes from the smoltalk model registry; provider-
  * returned cost fields aren't standardized on this endpoint.
  */
-export async function openaiEmbed(inputs, config, apiKey, baseURL) {
+export async function openaiEmbed(inputs, config, apiKey, baseURL, options) {
     try {
         const client = new OpenAI({ apiKey, ...(baseURL ? { baseURL } : {}) });
-        const response = await client.embeddings.create({
+        const body = {
             model: config.model,
             input: inputs,
-            ...(config.dimensions !== undefined
-                ? { dimensions: config.dimensions }
-                : {}),
-        });
+        };
+        if (config.dimensions !== undefined) {
+            body.dimensions = config.dimensions;
+        }
+        let encodingFormat = options?.encodingFormat;
+        if (encodingFormat === undefined && baseURL !== undefined) {
+            encodingFormat = "float";
+        }
+        if (encodingFormat !== undefined) {
+            body.encoding_format = encodingFormat;
+        }
+        const response = await client.embeddings.create(body);
         const embeddings = [...response.data]
             .sort((a, b) => a.index - b.index)
-            .map((d) => d.embedding);
+            .map((d) => toFloats(d.embedding));
         const inputTokens = response.usage.prompt_tokens;
         const costEstimate = calculateEmbeddingCost(config.model, inputTokens, config.modelData);
         return success({

package/dist/embed.d.ts

@@ -25,6 +25,7 @@ export type EmbedConfig = {
         deepInfra?: string;
         liteLlm?: string;
         openAiCompat?: string;
+        mlx?: string;
         /** Arbitrary provider names, for URLs targeting a custom-registered provider. */
         [provider: string]: string | undefined;
     };
@@ -39,4 +40,9 @@ export type EmbedResult = {
 };
 export type EmbedProvider = (inputs: string[], config: EmbedConfig) => Promise<Result<EmbedResult>>;
 export declare function registerEmbeddingProvider(name: string, fn: EmbedProvider): void;
+/** True when `name` has an embed provider registered through
+ *  registerEmbeddingProvider. The built-in cases in embed() are not its
+ *  concern, the same as hasProvider in client.ts. */
+export declare function hasEmbeddingProvider(name: string): boolean;
+export declare function unregisterEmbeddingProvider(name: string): boolean;
 export declare function embed(input: string | string[], config: EmbedConfig): Promise<Result<EmbedResult>>;

package/dist/embed.js

@@ -3,12 +3,33 @@ import { resolveProvider, resolveApiKey, resolveBaseUrl } from "./util/provider.
 import { openaiEmbed } from "./embed/openai.js";
 import { googleEmbed } from "./embed/google.js";
 import { ollamaEmbed } from "./embed/ollama.js";
+import { mlxEmbed } from "./embed/mlx.js";
+import { loadLlamaCpp } from "./clients/llamaCppLoader.js";
+function errorMessage(err) {
+    if (err instanceof Error) {
+        return err.message;
+    }
+    return String(err);
+}
 // Null-prototype so provider names like "toString"/"__proto__" can't collide
 // with Object.prototype or pollute the registry.
 const registeredEmbedProviders = Object.create(null);
 export function registerEmbeddingProvider(name, fn) {
     registeredEmbedProviders[name] = fn;
 }
+/** True when `name` has an embed provider registered through
+ *  registerEmbeddingProvider. The built-in cases in embed() are not its
+ *  concern, the same as hasProvider in client.ts. */
+export function hasEmbeddingProvider(name) {
+    return name in registeredEmbedProviders;
+}
+export function unregisterEmbeddingProvider(name) {
+    if (name in registeredEmbedProviders) {
+        delete registeredEmbedProviders[name];
+        return true;
+    }
+    return false;
+}
 export async function embed(input, config) {
     const inputs = Array.isArray(input) ? input : [input];
     let provider;
@@ -63,6 +84,31 @@ export async function embed(input, config) {
             }
             return openaiEmbed(inputs, config, apiKey, baseURL);
         }
+        case "mlx": {
+            // resolveBaseUrl always returns a value for "mlx" (it has a default).
+            return mlxEmbed(inputs, config, resolveBaseUrl("mlx", config));
+        }
+        case "llama-cpp": {
+            // A hand-registered provider wins, the same rule loadLlamaCpp applies
+            // to the chat class. Otherwise load the plugin the way text() does;
+            // the loader caches its import.
+            const custom = registeredEmbedProviders[provider];
+            if (custom) {
+                return custom(inputs, config);
+            }
+            let plugin;
+            try {
+                plugin = await loadLlamaCpp();
+            }
+            catch (err) {
+                return failure(errorMessage(err));
+            }
+            if (typeof plugin.embed !== "function") {
+                return failure("Your installed smoltalk-llama-cpp has no embeddings support. " +
+                    "Upgrade it (npm i smoltalk-llama-cpp@latest; >=0.5.0 required).");
+            }
+            return plugin.embed(inputs, config);
+        }
         default: {
             const custom = registeredEmbedProviders[provider];
             if (custom) {

package/dist/speech/baseSpeechClient.d.ts

@@ -11,6 +11,11 @@ export type SpeechClientConfig = {
     /** Base-URL map (for OpenAI-compatible providers); read via resolveBaseUrl. */
     baseUrl?: SmolConfig["baseUrl"];
     voice: string;
+    /** How the speech should sound, in plain words: "Alarmed and urgent."
+     *  Passed through as-is by the OpenAI-shaped providers (openai, groq,
+     *  openai-compat, mlx); whether the model reads it is up to the model
+     *  (OpenAI's tts-1 and tts-1-hd do not). The google provider ignores it. */
+    instructions?: string;
     modelData?: ModelDataBlob;
     /** Output format; provider-specific vocabulary (OpenAI: mp3/opus/aac/flac/wav/pcm). */
     format?: string;

package/dist/speech/mlx.d.ts

@@ -0,0 +1,23 @@
+import OpenAI from "openai";
+import { Result } from "../types/result.js";
+import type { SpeakFormat } from "../util/audioMime.js";
+import { OpenAISpeechClient } from "./openai.js";
+import type { SpeechResult } from "../speech.js";
+/**
+ * Speech from an MLX server on localhost, usually `agency local serve
+ * --speech <model>`. The same request the OpenAI client makes, with the
+ * things the chat and embedding mlx clients also fix: the base URL has a
+ * default, the key is a placeholder the server ignores, and the cost is
+ * zero. It also never retries. The server runs one generation at a time,
+ * so a retry would wait behind the request that just timed out and then
+ * generate the same text again.
+ */
+export declare class MlxSpeechClient extends OpenAISpeechClient {
+    protected makeClient(): OpenAI;
+    protected requiresKey(): boolean;
+    protected defaultFormat(): SpeakFormat;
+    /** Local speech is free. Set after the base class's cost pass, so model
+     *  data that prices this model cannot override it (matching the chat mlx
+     *  client, whose provider cost of 0 wins over registry pricing). */
+    speak(text: string): Promise<Result<SpeechResult>>;
+}

package/dist/speech/mlx.js

@@ -0,0 +1,49 @@
+import OpenAI from "openai";
+import { success } from "../types/result.js";
+import { resolveBaseUrl } from "../util/provider.js";
+import { OpenAISpeechClient } from "./openai.js";
+/** How long one request may take. Equal to the SDK default; stated here so
+ *  the ceiling is visible. A local model generating a long piece of text is
+ *  slow, and the caller keeps its requests short. */
+const MLX_SPEECH_TIMEOUT_MS = 600_000;
+/**
+ * Speech from an MLX server on localhost, usually `agency local serve
+ * --speech <model>`. The same request the OpenAI client makes, with the
+ * things the chat and embedding mlx clients also fix: the base URL has a
+ * default, the key is a placeholder the server ignores, and the cost is
+ * zero. It also never retries. The server runs one generation at a time,
+ * so a retry would wait behind the request that just timed out and then
+ * generate the same text again.
+ */
+export class MlxSpeechClient extends OpenAISpeechClient {
+    makeClient() {
+        // resolveBaseUrl always returns a value for "mlx" (it has a default).
+        const baseURL = resolveBaseUrl("mlx", { baseUrl: this.config.baseUrl });
+        // The OpenAI SDK refuses an empty key. The local server never reads it.
+        return new OpenAI({
+            apiKey: "mlx-local",
+            baseURL,
+            maxRetries: 0,
+            timeout: MLX_SPEECH_TIMEOUT_MS,
+        });
+    }
+    requiresKey() {
+        return false;
+    }
+    defaultFormat() {
+        return "wav";
+    }
+    /** Local speech is free. Set after the base class's cost pass, so model
+     *  data that prices this model cannot override it (matching the chat mlx
+     *  client, whose provider cost of 0 wins over registry pricing). */
+    async speak(text) {
+        const result = await super.speak(text);
+        if (!result.success) {
+            return result;
+        }
+        return success({
+            ...result.value,
+            cost: { inputCost: 0, outputCost: 0, totalCost: 0, currency: "USD" },
+        });
+    }
+}

package/dist/speech/openai.d.ts

@@ -10,5 +10,8 @@ export declare class OpenAISpeechClient extends BaseSpeechClient {
     protected defaultFormat(): SpeakFormat;
     /** Provider-specific diagnostic when no API key is resolved. Subclasses override. */
     protected noKeyMessage(): string;
+    /** Whether a request needs an API key at all. A local server ignores the
+     *  key, and its client says so by returning false. */
+    protected requiresKey(): boolean;
     protected _speak(text: string): Promise<Result<SpeechResult>>;
 }

package/dist/speech/openai.js

@@ -15,10 +15,15 @@ export class OpenAISpeechClient extends BaseSpeechClient {
     noKeyMessage() {
         return "No OpenAI API key provided. Set apiKey.openAi or OPENAI_API_KEY.";
     }
+    /** Whether a request needs an API key at all. A local server ignores the
+     *  key, and its client says so by returning false. */
+    requiresKey() {
+        return true;
+    }
     // No try/catch here: BaseSpeechClient.speak() is the single
     // redacting/logging exception boundary.
     async _speak(text) {
-        if (!this.config.apiKey) {
+        if (this.requiresKey() && !this.config.apiKey) {
             return failure(this.noKeyMessage());
         }
         // The shared contract carries format as a plain string; narrow to OpenAI's
@@ -40,6 +45,9 @@ export class OpenAISpeechClient extends BaseSpeechClient {
         if (this.config.speed !== undefined) {
             params.speed = this.config.speed;
         }
+        if (this.config.instructions !== undefined && this.config.instructions !== "") {
+            params.instructions = this.config.instructions;
+        }
         const res = await client.audio.speech.create(params, { signal: this.config.abortSignal });
         const audio = new Uint8Array(await res.arrayBuffer());
         const result = { audio, mimeType };

package/dist/speech.d.ts

@@ -7,6 +7,11 @@ import { BaseSpeechClient, SpeechClientConfig } from "./speech/baseSpeechClient.
 export type SpeakOptions = {
     model: string;
     voice: string;
+    /** How the speech should sound, in plain words: "Alarmed and urgent."
+     *  Passed through as-is by the OpenAI-shaped providers (openai, groq,
+     *  openai-compat, mlx); whether the model reads it is up to the model
+     *  (OpenAI's tts-1 and tts-1-hd do not). The google provider ignores it. */
+    instructions?: string;
     provider?: string;
     modelData?: ModelDataBlob;
     apiKey?: SmolConfig["apiKey"];

package/dist/speech.js

@@ -6,12 +6,14 @@ import { OpenAISpeechClient } from "./speech/openai.js";
 import { GroqSpeechClient } from "./speech/groq.js";
 import { GoogleSpeechClient } from "./speech/google.js";
 import { OpenAiCompatSpeechClient } from "./speech/openaiCompat.js";
+import { MlxSpeechClient } from "./speech/mlx.js";
 // Checked before the user registry so a registered "openai" can't hijack the built-in.
 const builtinClients = Object.create(null);
 builtinClients["openai"] = OpenAISpeechClient;
 builtinClients["groq"] = GroqSpeechClient;
 builtinClients["google"] = GoogleSpeechClient;
 builtinClients["openai-compat"] = OpenAiCompatSpeechClient;
+builtinClients["mlx"] = MlxSpeechClient;
 // Null-prototype so provider names like "toString"/"__proto__" can't collide
 // with Object.prototype or pollute the registry.
 const registered = Object.create(null);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "smoltalk",
-  "version": "0.13.2",
+  "version": "0.14.1",
   "description": "A common interface for LLM APIs",
   "homepage": "https://github.com/egonSchiele/smoltalk",
   "files": [