smoltalk 0.5.1 → 0.7.0

package/README.md

@@ -8,6 +8,15 @@ Smoltalk exposes a common API to different LLM providers, with built-in cost tra
 pnpm install smoltalk
 ```
 
+> **Upgrading to 0.6.x?** The flat API-key/host fields on `SmolConfig` have
+> been removed in favor of nested `apiKey` and `baseUrl` maps. Migration:
+> ```diff
+> -{ openAiApiKey: "sk-...",   googleApiKey: "...", ollamaHost: "http://..." }
+> +{ apiKey: { openAi: "sk-...", google: "..." }, baseUrl: { ollama: "http://..." } }
+> ```
+> Env-var fallbacks are unchanged (`OPENAI_API_KEY`, `GEMINI_API_KEY`,
+> `ANTHROPIC_API_KEY`, `OLLAMA_HOST`).
+
 ## Hello world example
 
 ```typescript
@@ -68,8 +77,10 @@ const messages = [
 const resp = await text({
   messages,
   model: "gemini-2.0-flash-lite",
-  openAiApiKey: process.env.OPENAI_API_KEY || "",
-  googleApiKey: process.env.GEMINI_API_KEY || "",
+  apiKey: {
+    openAi: process.env.OPENAI_API_KEY || "",
+    google: process.env.GEMINI_API_KEY || "",
+  },
   logLevel: "debug",
 });
 ```
@@ -80,8 +91,10 @@ If you want to construct a client once and reuse it across many calls, use `getC
 import { getClient, userMessage } from "smoltalk";
 
 const client = getClient({
-  openAiApiKey: process.env.OPENAI_API_KEY || "",
-  googleApiKey: process.env.GEMINI_API_KEY || "",
+  apiKey: {
+    openAi: process.env.OPENAI_API_KEY || "",
+    google: process.env.GEMINI_API_KEY || "",
+  },
   model: "gemini-2.0-flash-lite",
 });
 
@@ -147,12 +160,9 @@ A couple of design decisions to note:
 |--------|------|-------------|
 | `model` | `ModelName` | **Required.** The model to use (e.g. `"gpt-4o"`, `"gemini-2.0-flash-lite"`). |
 | `messages` | `Message[]` | **Required.** The conversation messages to send. |
-| `openAiApiKey` | `string` | OpenAI API key. |
-| `googleApiKey` | `string` | Google Gemini API key. |
-| `anthropicApiKey` | `string` | Anthropic API key. |
-| `ollamaApiKey` | `string` | Ollama API key (only needed for cloud Ollama). |
-| `ollamaHost` | `string` | Ollama host URL (for self-hosted or cloud Ollama). |
-| `provider` | `Provider` | Override provider detection. One of `"openai"`, `"openai-responses"`, `"google"`, `"ollama"`, `"anthropic"`, or any provider registered via `registerProvider()`. |
+| `apiKey` | `{ openAi?, google?, anthropic?, ollama?, openRouter?, deepInfra?, liteLlm?, openAiCompat? }` | API keys, nested by provider. Each falls back to its conventional env var (`OPENAI_API_KEY`, `GEMINI_API_KEY`, `ANTHROPIC_API_KEY`, `OPENROUTER_API_KEY`, `DEEPINFRA_API_KEY`, `LITELLM_API_KEY`, `OPENAI_COMPAT_API_KEY`). Ollama has no env-var fallback for the key. |
+| `baseUrl` | `{ ollama?, openRouter?, deepInfra?, liteLlm?, openAiCompat? }` | Custom base URLs. `ollama` defaults to `$OLLAMA_HOST` then `http://localhost:11434`; `openRouter`/`deepInfra` defaults are baked in; `liteLlm`/`openAiCompat` require an explicit URL (or `LITELLM_BASE_URL` / `OPENAI_COMPAT_BASE_URL` env). |
+| `provider` | `Provider` | Override provider detection. One of `"openai"`, `"openai-responses"`, `"google"`, `"ollama"`, `"anthropic"`, `"openrouter"`, `"deepinfra"`, `"litellm"`, `"openai-compat"`, or any provider registered via `registerProvider()`. |
 | `logLevel` | `LogLevel` | Logging verbosity: `"debug"`, `"info"`, `"warn"`, `"error"`. |
 | `tools` | `{ name, description?, schema }[]` | Tool definitions. `schema` is a Zod object schema. |
 | `responseFormat` | `ZodType` | Zod schema for structured output. The response is parsed and validated against this schema. |
@@ -194,6 +204,53 @@ Detects when the model is stuck in a repetitive tool-call loop.
 | `intervention` | `string` | Action to take: `"remove-tool"`, `"remove-all-tools"`, `"throw-error"`, or `"halt-execution"`. |
 | `excludeTools` | `string[]` | Tool names to ignore when counting calls. |
 
+## Hosted OpenAI-compatible providers
+
+Smoltalk ships four built-in providers for hosted open-source models that all
+speak the OpenAI chat-completions shape. Use these when you want to run a
+Llama, GLM, Qwen, etc. via someone else's hosted infrastructure without
+adding a new dependency. You must pass `provider:` explicitly because these
+model ids aren't in the smoltalk registry.
+
+| `provider:` | What it is | Required config | Cost source |
+|-------------|------------|-----------------|-------------|
+| `"openrouter"` | OpenRouter.ai aggregator | `apiKey.openRouter` (or `OPENROUTER_API_KEY`) | `usage.cost` (auto-enabled by injecting `usage:{include:true}`) |
+| `"deepinfra"` | DeepInfra hosted models | `apiKey.deepInfra` (or `DEEPINFRA_API_KEY`) | `usage.estimated_cost` |
+| `"litellm"`   | Your own LiteLLM proxy   | `apiKey.liteLlm` + `baseUrl.liteLlm` (or `LITELLM_API_KEY` / `LITELLM_BASE_URL`) | `x-litellm-response-cost` header (non-stream only) |
+| `"openai-compat"` | Any OpenAI-shape backend (vLLM, TGI, LM Studio…) | `apiKey.openAiCompat` + `baseUrl.openAiCompat` (or `OPENAI_COMPAT_API_KEY` / `OPENAI_COMPAT_BASE_URL`) | Best-effort: reads `usage.cost`/`estimated_cost`/`cost_usd` if present |
+
+```ts
+import { textSync, userMessage } from "smoltalk";
+
+const r = await textSync({
+  model: "z-ai/glm-5.2",
+  provider: "openrouter",
+  apiKey: { openRouter: process.env.OPENROUTER_API_KEY! },
+  messages: [userMessage("hi")],
+});
+// r.value.cost.totalCost is a real OpenRouter-reported USD cost.
+```
+
+**Capability matrix:**
+
+|                 | chat | embeddings | image generation | `web_search` hosted tool |
+|-----------------|------|------------|------------------|--------------------------|
+| `openrouter`    | ✅   | ❌         | ❌               | ✅ (via `:online` / web plugin) |
+| `deepinfra`     | ✅   | ✅         | ❌ (uses per-model endpoints, not OpenAI shape) | ❌ |
+| `litellm`       | ✅   | ✅         | ✅ (if the upstream model supports it) | ✅ (if upstream supports it) |
+| `openai-compat` | ✅   | ✅         | ✅ (backend-dependent) | depends on backend |
+
+Smoltalk surfaces a clear `failure(...)` from `embed()`/`image()` for the
+unsupported combinations rather than silently dropping the call.
+
+**Running a local LiteLLM proxy:**
+
+```bash
+pip install 'litellm[proxy]'
+litellm --model openai/gpt-4o
+# In your code: baseUrl: { liteLlm: "http://localhost:4000" }
+```
+
 ## Refreshing model data
 
 Smoltalk ships a baked-in model registry (pricing, context limits, capabilities).

package/dist/classes/message/UserMessage.d.ts

@@ -4,27 +4,88 @@ import { ChatCompletionMessageParam } from "openai/resources";
 import { Content } from "@google/genai";
 import { Message } from "ollama";
 import type { ResponseInputItem } from "openai/resources/responses/responses.js";
+import { UserContent, UserContentInput, UserContentPart } from "./contentParts.js";
 export declare const UserMessageJSONSchema: z.ZodObject<{
     role: z.ZodLiteral<"user">;
-    content: z.ZodString;
+    content: z.ZodUnion<readonly [z.ZodString, z.ZodArray<z.ZodDiscriminatedUnion<[z.ZodObject<{
+        type: z.ZodLiteral<"text">;
+        text: z.ZodString;
+    }, z.core.$strip>, z.ZodObject<{
+        type: z.ZodLiteral<"image">;
+        source: z.ZodDiscriminatedUnion<[z.ZodObject<{
+            kind: z.ZodLiteral<"bytes">;
+            data: z.ZodCustom<Uint8Array<ArrayBuffer>, Uint8Array<ArrayBuffer>>;
+            mimeType: z.ZodString;
+        }, z.core.$strip>, z.ZodObject<{
+            kind: z.ZodLiteral<"base64">;
+            base64: z.ZodString;
+            mimeType: z.ZodString;
+        }, z.core.$strip>, z.ZodObject<{
+            kind: z.ZodLiteral<"path">;
+            path: z.ZodString;
+            mimeType: z.ZodOptional<z.ZodString>;
+        }, z.core.$strip>, z.ZodObject<{
+            kind: z.ZodLiteral<"url">;
+            url: z.ZodString;
+            mimeType: z.ZodOptional<z.ZodString>;
+            timeoutMs: z.ZodOptional<z.ZodNumber>;
+        }, z.core.$strip>, z.ZodObject<{
+            kind: z.ZodLiteral<"providerFile">;
+            provider: z.ZodString;
+            id: z.ZodString;
+            uri: z.ZodOptional<z.ZodString>;
+            mimeType: z.ZodOptional<z.ZodString>;
+            expiresAt: z.ZodOptional<z.ZodNumber>;
+        }, z.core.$strip>], "kind">;
+    }, z.core.$strip>, z.ZodObject<{
+        type: z.ZodLiteral<"file">;
+        source: z.ZodDiscriminatedUnion<[z.ZodObject<{
+            kind: z.ZodLiteral<"bytes">;
+            data: z.ZodCustom<Uint8Array<ArrayBuffer>, Uint8Array<ArrayBuffer>>;
+            mimeType: z.ZodString;
+        }, z.core.$strip>, z.ZodObject<{
+            kind: z.ZodLiteral<"base64">;
+            base64: z.ZodString;
+            mimeType: z.ZodString;
+        }, z.core.$strip>, z.ZodObject<{
+            kind: z.ZodLiteral<"path">;
+            path: z.ZodString;
+            mimeType: z.ZodOptional<z.ZodString>;
+        }, z.core.$strip>, z.ZodObject<{
+            kind: z.ZodLiteral<"url">;
+            url: z.ZodString;
+            mimeType: z.ZodOptional<z.ZodString>;
+            timeoutMs: z.ZodOptional<z.ZodNumber>;
+        }, z.core.$strip>, z.ZodObject<{
+            kind: z.ZodLiteral<"providerFile">;
+            provider: z.ZodString;
+            id: z.ZodString;
+            uri: z.ZodOptional<z.ZodString>;
+            mimeType: z.ZodOptional<z.ZodString>;
+            expiresAt: z.ZodOptional<z.ZodNumber>;
+        }, z.core.$strip>], "kind">;
+        filename: z.ZodOptional<z.ZodString>;
+    }, z.core.$strip>], "type">>]>;
     name: z.ZodOptional<z.ZodString>;
     rawData: z.ZodOptional<z.ZodAny>;
 }, z.core.$strip>;
 export type UserMessageJSON = z.infer<typeof UserMessageJSONSchema>;
 export declare class UserMessage extends BaseMessage implements MessageClass {
     _role: "user";
-    _content: string;
+    _content: UserContent;
     _name?: string;
     _rawData?: any;
-    constructor(content: string, options?: {
+    constructor(content: UserContentInput, options?: {
         name?: string;
         rawData?: any;
     });
     get content(): string;
-    set content(value: string);
+    set content(value: UserContentInput);
     get role(): "user";
     get name(): string | undefined;
     get rawData(): any;
+    /** The content parts array, or null when content is a plain string. */
+    getContentParts(): UserContentPart[] | null;
     toJSON(): UserMessageJSON;
     static fromJSON(json: unknown): UserMessage;
     toOpenAIMessage(): ChatCompletionMessageParam;
@@ -33,6 +94,6 @@ export declare class UserMessage extends BaseMessage implements MessageClass {
     toOllamaMessage(): Message;
     toAnthropicMessage(): {
         role: "user";
-        content: string;
+        content: string | any[];
     };
 }

package/dist/classes/message/UserMessage.js

@@ -1,9 +1,17 @@
 import { z } from "zod";
 import { BaseMessage } from "./BaseMessage.js";
 import { getLogger } from "../../util/logger.js";
+import { UserContentSchema, } from "./contentParts.js";
+import { refToBase64 } from "../../util/attachments.js";
+import { renderParts } from "./renderers/PartRenderer.js";
+import { OpenAIChatRenderer } from "./renderers/OpenAIChatRenderer.js";
+import { OpenAIResponsesRenderer } from "./renderers/OpenAIResponsesRenderer.js";
+import { GoogleRenderer } from "./renderers/GoogleRenderer.js";
+import { AnthropicRenderer } from "./renderers/AnthropicRenderer.js";
+import { JSONRenderer } from "./renderers/JSONRenderer.js";
 export const UserMessageJSONSchema = z.object({
     role: z.literal("user"),
-    content: z.string(),
+    content: UserContentSchema,
     name: z.string().optional(),
     rawData: z.any().optional(),
 });
@@ -14,15 +22,15 @@ export class UserMessage extends BaseMessage {
     _rawData;
     constructor(content, options = {}) {
         super();
-        this._content = content;
+        this._content = normalizeUserContent(content);
         this._name = options.name;
         this._rawData = options.rawData;
     }
     get content() {
-        return this._content;
+        return userContentToText(this._content);
     }
     set content(value) {
-        this._content = value;
+        this._content = normalizeUserContent(value);
     }
     get role() {
         return this._role;
@@ -33,10 +41,17 @@ export class UserMessage extends BaseMessage {
     get rawData() {
         return this._rawData;
     }
+    /** The content parts array, or null when content is a plain string. */
+    getContentParts() {
+        if (typeof this._content === "string") {
+            return null;
+        }
+        return this._content;
+    }
     toJSON() {
         return {
             role: this.role,
-            content: this.content,
+            content: serializeUserContentForJSON(this._content),
             name: this.name,
         };
     }
@@ -55,29 +70,93 @@ export class UserMessage extends BaseMessage {
         });
     }
     toOpenAIMessage() {
-        return {
-            role: this.role,
-            content: this.content,
-            name: this.name,
-        };
+        if (typeof this._content === "string") {
+            return { role: this.role, content: this._content, name: this.name };
+        }
+        const parts = renderParts(this._content, new OpenAIChatRenderer());
+        return { role: this.role, content: parts, name: this.name };
     }
     toOpenAIResponseInputItem() {
-        return {
-            type: "message",
-            role: "user",
-            content: this.content,
-        };
+        if (typeof this._content === "string") {
+            return { type: "message", role: "user", content: this._content };
+        }
+        const content = renderParts(this._content, new OpenAIResponsesRenderer());
+        return { type: "message", role: "user", content };
     }
     toGoogleMessage() {
-        return { role: this.role, parts: [{ text: this.content }] };
+        if (typeof this._content === "string") {
+            return { role: this.role, parts: [{ text: this._content }] };
+        }
+        const parts = renderParts(this._content, new GoogleRenderer());
+        return { role: this.role, parts };
     }
     toOllamaMessage() {
-        return {
-            role: this.role,
-            content: this.content,
-        };
+        if (typeof this._content === "string") {
+            return { role: this.role, content: this._content };
+        }
+        const texts = [];
+        const images = [];
+        for (const part of this._content) {
+            if (part.type === "text") {
+                texts.push(part.text);
+                continue;
+            }
+            if (part.type === "image") {
+                if (part.source.kind === "providerFile") {
+                    throw new Error("Ollama does not support provider file references.");
+                }
+                images.push(refToBase64(part.source).base64);
+                continue;
+            }
+            if (part.source.kind === "providerFile") {
+                throw new Error("Ollama does not support provider file references.");
+            }
+            getLogger().warn("Ollama does not support file attachments; dropping a file part.");
+        }
+        const message = { role: this.role, content: texts.join("\n") };
+        if (images.length > 0) {
+            message.images = images;
+        }
+        return message;
     }
     toAnthropicMessage() {
-        return { role: "user", content: this.content };
+        if (typeof this._content === "string") {
+            return { role: "user", content: this._content };
+        }
+        const blocks = renderParts(this._content, new AnthropicRenderer());
+        return { role: "user", content: blocks };
+    }
+}
+function normalizeUserContent(content) {
+    if (typeof content === "string") {
+        return content;
+    }
+    const parts = [];
+    for (const part of content) {
+        if (typeof part === "string") {
+            parts.push({ type: "text", text: part });
+        }
+        else {
+            parts.push(part);
+        }
+    }
+    return parts;
+}
+function userContentToText(content) {
+    if (typeof content === "string") {
+        return content;
+    }
+    const texts = [];
+    for (const part of content) {
+        if (part.type === "text") {
+            texts.push(part.text);
+        }
+    }
+    return texts.join("\n");
+}
+function serializeUserContentForJSON(content) {
+    if (typeof content === "string") {
+        return content;
     }
+    return renderParts(content, new JSONRenderer());
 }

package/dist/classes/message/contentParts.d.ts

@@ -0,0 +1,259 @@
+import { z } from "zod";
+import type { ImageRef } from "../../util/imageRef.js";
+export type TextPart = {
+    type: "text";
+    text: string;
+};
+export type ImagePart = {
+    type: "image";
+    source: AttachmentSource;
+};
+export type FilePart = {
+    type: "file";
+    source: AttachmentSource;
+    filename?: string;
+};
+export type UserContentPart = TextPart | ImagePart | FilePart;
+/** Normalized user-message content: a plain string or an array of typed parts. */
+export type UserContent = string | UserContentPart[];
+/** What callers may pass: a bare string element is sugar for a text part. */
+export type UserContentInput = string | Array<string | UserContentPart>;
+export declare const TextPartSchema: z.ZodObject<{
+    type: z.ZodLiteral<"text">;
+    text: z.ZodString;
+}, z.core.$strip>;
+export declare const ImageRefSchema: z.ZodDiscriminatedUnion<[z.ZodObject<{
+    kind: z.ZodLiteral<"bytes">;
+    data: z.ZodCustom<Uint8Array<ArrayBuffer>, Uint8Array<ArrayBuffer>>;
+    mimeType: z.ZodString;
+}, z.core.$strip>, z.ZodObject<{
+    kind: z.ZodLiteral<"base64">;
+    base64: z.ZodString;
+    mimeType: z.ZodString;
+}, z.core.$strip>, z.ZodObject<{
+    kind: z.ZodLiteral<"path">;
+    path: z.ZodString;
+    mimeType: z.ZodOptional<z.ZodString>;
+}, z.core.$strip>, z.ZodObject<{
+    kind: z.ZodLiteral<"url">;
+    url: z.ZodString;
+    mimeType: z.ZodOptional<z.ZodString>;
+    timeoutMs: z.ZodOptional<z.ZodNumber>;
+}, z.core.$strip>], "kind">;
+export type ProviderFileRef = {
+    kind: "providerFile";
+    provider: string;
+    id: string;
+    uri?: string;
+    mimeType?: string;
+    expiresAt?: number;
+};
+export declare const ProviderFileRefSchema: z.ZodObject<{
+    kind: z.ZodLiteral<"providerFile">;
+    provider: z.ZodString;
+    id: z.ZodString;
+    uri: z.ZodOptional<z.ZodString>;
+    mimeType: z.ZodOptional<z.ZodString>;
+    expiresAt: z.ZodOptional<z.ZodNumber>;
+}, z.core.$strip>;
+export type AttachmentSource = ImageRef | ProviderFileRef;
+export declare const AttachmentSourceSchema: z.ZodDiscriminatedUnion<[z.ZodObject<{
+    kind: z.ZodLiteral<"bytes">;
+    data: z.ZodCustom<Uint8Array<ArrayBuffer>, Uint8Array<ArrayBuffer>>;
+    mimeType: z.ZodString;
+}, z.core.$strip>, z.ZodObject<{
+    kind: z.ZodLiteral<"base64">;
+    base64: z.ZodString;
+    mimeType: z.ZodString;
+}, z.core.$strip>, z.ZodObject<{
+    kind: z.ZodLiteral<"path">;
+    path: z.ZodString;
+    mimeType: z.ZodOptional<z.ZodString>;
+}, z.core.$strip>, z.ZodObject<{
+    kind: z.ZodLiteral<"url">;
+    url: z.ZodString;
+    mimeType: z.ZodOptional<z.ZodString>;
+    timeoutMs: z.ZodOptional<z.ZodNumber>;
+}, z.core.$strip>, z.ZodObject<{
+    kind: z.ZodLiteral<"providerFile">;
+    provider: z.ZodString;
+    id: z.ZodString;
+    uri: z.ZodOptional<z.ZodString>;
+    mimeType: z.ZodOptional<z.ZodString>;
+    expiresAt: z.ZodOptional<z.ZodNumber>;
+}, z.core.$strip>], "kind">;
+export declare const ImagePartSchema: z.ZodObject<{
+    type: z.ZodLiteral<"image">;
+    source: z.ZodDiscriminatedUnion<[z.ZodObject<{
+        kind: z.ZodLiteral<"bytes">;
+        data: z.ZodCustom<Uint8Array<ArrayBuffer>, Uint8Array<ArrayBuffer>>;
+        mimeType: z.ZodString;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"base64">;
+        base64: z.ZodString;
+        mimeType: z.ZodString;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"path">;
+        path: z.ZodString;
+        mimeType: z.ZodOptional<z.ZodString>;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"url">;
+        url: z.ZodString;
+        mimeType: z.ZodOptional<z.ZodString>;
+        timeoutMs: z.ZodOptional<z.ZodNumber>;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"providerFile">;
+        provider: z.ZodString;
+        id: z.ZodString;
+        uri: z.ZodOptional<z.ZodString>;
+        mimeType: z.ZodOptional<z.ZodString>;
+        expiresAt: z.ZodOptional<z.ZodNumber>;
+    }, z.core.$strip>], "kind">;
+}, z.core.$strip>;
+export declare const FilePartSchema: z.ZodObject<{
+    type: z.ZodLiteral<"file">;
+    source: z.ZodDiscriminatedUnion<[z.ZodObject<{
+        kind: z.ZodLiteral<"bytes">;
+        data: z.ZodCustom<Uint8Array<ArrayBuffer>, Uint8Array<ArrayBuffer>>;
+        mimeType: z.ZodString;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"base64">;
+        base64: z.ZodString;
+        mimeType: z.ZodString;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"path">;
+        path: z.ZodString;
+        mimeType: z.ZodOptional<z.ZodString>;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"url">;
+        url: z.ZodString;
+        mimeType: z.ZodOptional<z.ZodString>;
+        timeoutMs: z.ZodOptional<z.ZodNumber>;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"providerFile">;
+        provider: z.ZodString;
+        id: z.ZodString;
+        uri: z.ZodOptional<z.ZodString>;
+        mimeType: z.ZodOptional<z.ZodString>;
+        expiresAt: z.ZodOptional<z.ZodNumber>;
+    }, z.core.$strip>], "kind">;
+    filename: z.ZodOptional<z.ZodString>;
+}, z.core.$strip>;
+export declare const UserContentPartSchema: z.ZodDiscriminatedUnion<[z.ZodObject<{
+    type: z.ZodLiteral<"text">;
+    text: z.ZodString;
+}, z.core.$strip>, z.ZodObject<{
+    type: z.ZodLiteral<"image">;
+    source: z.ZodDiscriminatedUnion<[z.ZodObject<{
+        kind: z.ZodLiteral<"bytes">;
+        data: z.ZodCustom<Uint8Array<ArrayBuffer>, Uint8Array<ArrayBuffer>>;
+        mimeType: z.ZodString;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"base64">;
+        base64: z.ZodString;
+        mimeType: z.ZodString;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"path">;
+        path: z.ZodString;
+        mimeType: z.ZodOptional<z.ZodString>;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"url">;
+        url: z.ZodString;
+        mimeType: z.ZodOptional<z.ZodString>;
+        timeoutMs: z.ZodOptional<z.ZodNumber>;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"providerFile">;
+        provider: z.ZodString;
+        id: z.ZodString;
+        uri: z.ZodOptional<z.ZodString>;
+        mimeType: z.ZodOptional<z.ZodString>;
+        expiresAt: z.ZodOptional<z.ZodNumber>;
+    }, z.core.$strip>], "kind">;
+}, z.core.$strip>, z.ZodObject<{
+    type: z.ZodLiteral<"file">;
+    source: z.ZodDiscriminatedUnion<[z.ZodObject<{
+        kind: z.ZodLiteral<"bytes">;
+        data: z.ZodCustom<Uint8Array<ArrayBuffer>, Uint8Array<ArrayBuffer>>;
+        mimeType: z.ZodString;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"base64">;
+        base64: z.ZodString;
+        mimeType: z.ZodString;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"path">;
+        path: z.ZodString;
+        mimeType: z.ZodOptional<z.ZodString>;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"url">;
+        url: z.ZodString;
+        mimeType: z.ZodOptional<z.ZodString>;
+        timeoutMs: z.ZodOptional<z.ZodNumber>;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"providerFile">;
+        provider: z.ZodString;
+        id: z.ZodString;
+        uri: z.ZodOptional<z.ZodString>;
+        mimeType: z.ZodOptional<z.ZodString>;
+        expiresAt: z.ZodOptional<z.ZodNumber>;
+    }, z.core.$strip>], "kind">;
+    filename: z.ZodOptional<z.ZodString>;
+}, z.core.$strip>], "type">;
+export declare const UserContentSchema: z.ZodUnion<readonly [z.ZodString, z.ZodArray<z.ZodDiscriminatedUnion<[z.ZodObject<{
+    type: z.ZodLiteral<"text">;
+    text: z.ZodString;
+}, z.core.$strip>, z.ZodObject<{
+    type: z.ZodLiteral<"image">;
+    source: z.ZodDiscriminatedUnion<[z.ZodObject<{
+        kind: z.ZodLiteral<"bytes">;
+        data: z.ZodCustom<Uint8Array<ArrayBuffer>, Uint8Array<ArrayBuffer>>;
+        mimeType: z.ZodString;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"base64">;
+        base64: z.ZodString;
+        mimeType: z.ZodString;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"path">;
+        path: z.ZodString;
+        mimeType: z.ZodOptional<z.ZodString>;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"url">;
+        url: z.ZodString;
+        mimeType: z.ZodOptional<z.ZodString>;
+        timeoutMs: z.ZodOptional<z.ZodNumber>;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"providerFile">;
+        provider: z.ZodString;
+        id: z.ZodString;
+        uri: z.ZodOptional<z.ZodString>;
+        mimeType: z.ZodOptional<z.ZodString>;
+        expiresAt: z.ZodOptional<z.ZodNumber>;
+    }, z.core.$strip>], "kind">;
+}, z.core.$strip>, z.ZodObject<{
+    type: z.ZodLiteral<"file">;
+    source: z.ZodDiscriminatedUnion<[z.ZodObject<{
+        kind: z.ZodLiteral<"bytes">;
+        data: z.ZodCustom<Uint8Array<ArrayBuffer>, Uint8Array<ArrayBuffer>>;
+        mimeType: z.ZodString;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"base64">;
+        base64: z.ZodString;
+        mimeType: z.ZodString;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"path">;
+        path: z.ZodString;
+        mimeType: z.ZodOptional<z.ZodString>;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"url">;
+        url: z.ZodString;
+        mimeType: z.ZodOptional<z.ZodString>;
+        timeoutMs: z.ZodOptional<z.ZodNumber>;
+    }, z.core.$strip>, z.ZodObject<{
+        kind: z.ZodLiteral<"providerFile">;
+        provider: z.ZodString;
+        id: z.ZodString;
+        uri: z.ZodOptional<z.ZodString>;
+        mimeType: z.ZodOptional<z.ZodString>;
+        expiresAt: z.ZodOptional<z.ZodNumber>;
+    }, z.core.$strip>], "kind">;
+    filename: z.ZodOptional<z.ZodString>;
+}, z.core.$strip>], "type">>]>;

package/dist/classes/message/contentParts.js

@@ -0,0 +1,44 @@
+import { z } from "zod";
+export const TextPartSchema = z.object({
+    type: z.literal("text"),
+    text: z.string(),
+});
+export const ImageRefSchema = z.discriminatedUnion("kind", [
+    z.object({ kind: z.literal("bytes"), data: z.instanceof(Uint8Array), mimeType: z.string() }),
+    z.object({ kind: z.literal("base64"), base64: z.string(), mimeType: z.string() }),
+    z.object({ kind: z.literal("path"), path: z.string(), mimeType: z.string().optional() }),
+    z.object({
+        kind: z.literal("url"),
+        url: z.string(),
+        mimeType: z.string().optional(),
+        timeoutMs: z.number().optional(),
+    }),
+]);
+export const ProviderFileRefSchema = z.object({
+    kind: z.literal("providerFile"),
+    provider: z.string(),
+    id: z.string(),
+    uri: z.string().optional(),
+    mimeType: z.string().optional(),
+    expiresAt: z.number().optional(),
+});
+// Compose from ImageRefSchema so future arms don't need mirroring here.
+export const AttachmentSourceSchema = z.discriminatedUnion("kind", [
+    ...ImageRefSchema.options,
+    ProviderFileRefSchema,
+]);
+export const ImagePartSchema = z.object({
+    type: z.literal("image"),
+    source: AttachmentSourceSchema,
+});
+export const FilePartSchema = z.object({
+    type: z.literal("file"),
+    source: AttachmentSourceSchema,
+    filename: z.string().optional(),
+});
+export const UserContentPartSchema = z.discriminatedUnion("type", [
+    TextPartSchema,
+    ImagePartSchema,
+    FilePartSchema,
+]);
+export const UserContentSchema = z.union([z.string(), z.array(UserContentPartSchema)]);