@browserlessapi/inferlo 0.6.0 → 0.7.0

package/README.md

@@ -2,7 +2,7 @@
 
 **The enterprise-ready TypeScript backend framework, with AI models built in.**
 
-Inferlo is a batteries-included TypeScript backend framework for production APIs, SaaS products and enterprise services, in the same class as NestJS, Fastify and AdonisJS. It gives you routing, validation, a typed database layer with PostgreSQL and migrations, auth, rate limiting, background jobs, cron, OpenAPI and secure defaults in one package with zero runtime dependencies, running on Node.js, Bun and edge runtimes.
+Inferlo is a batteries-included TypeScript backend framework for production APIs, SaaS products and enterprise services, in the same class as NestJS, Fastify and AdonisJS. It gives you routing, validation, a typed database layer with PostgreSQL and migrations, auth, rate limiting, background jobs, cron, OpenAPI and secure defaults in one package with zero runtime dependencies, running on Node.js, Bun, Deno and edge runtimes.
 
 What sets it apart is that AI models are part of the core, not a plugin: Anthropic, OpenAI, Gemini, local models through Ollama and any OpenAI-compatible server behind one API, with fallback between models, typed structured output, streaming, tool calling, image input and per-user usage and cost tracking, plus an MCP server built from your routes.
 
@@ -47,7 +47,7 @@ That one model definition gives you the database table, insert and update valida
 - **Embeddings** for search and recommendations.
 - **Usage, cost and budgets.** Every call is tracked per user, so you can bill customers and stop calls when their credits run out.
 - **MCP server.** Mark a route `tool: true` and AI agents can call your API, with the same permissions as any other client.
-- **Use it in any framework.** `import { createAI } from "@browserlessapi/inferlo/ai"` gives you the same AI module without `createApp`, for Express, Hono, Fastify, NestJS or a plain script, and `toNodeHandler(app, { prefix: "/ai" })` mounts a whole Inferlo app (routes, OpenAPI, MCP) inside Express, Fastify, NestJS or `node:http`, `hono.use(inferloAI(ai))` from `@browserlessapi/inferlo/hono` gives Hono handlers `c.var.ai`, `server.use(inferloAI(ai))` from `@browserlessapi/inferlo/express` gives Express handlers `req.ai`, and `fastify.register(inferloPlugin, { ai })` from `@browserlessapi/inferlo/fastify` gives Fastify `request.ai`, `server.register({ plugin: inferloPlugin, options: { ai } })` from `@browserlessapi/inferlo/hapi` gives hapi `request.ai` (and `options: { app }` with `routes: { prefix }` serves a whole Inferlo app), `new Elysia().use(inferloAI(ai))` from `@browserlessapi/inferlo/elysia` gives Elysia handlers `ai`, `InferloModule.forRoot({ ai })` from `@browserlessapi/inferlo/nestjs` makes the AI client injectable in NestJS, the provider from `@browserlessapi/inferlo/adonisjs` gives AdonisJS routes `ctx.ai`, `@Configuration({ imports: [inferloAI(ai)] })` from `@browserlessapi/inferlo/tsed` makes the AI client injectable in Ts.ED as `InferloAI`, `trpc.procedure.use(inferloAI(ai))` from `@browserlessapi/inferlo/trpc` gives tRPC procedures `ctx.ai` while `trpcTools(router, { caller, procedures })` lets a model call tRPC procedures as tools, `koa.use(inferloAI(ai))` and `koa.use(toKoaMiddleware(app, { prefix: "/api" }))` from `@browserlessapi/inferlo/koa` give Koa `ctx.ai` and a whole Inferlo app, `export const { GET, POST } = toNextHandlers(app, { prefix: "/api" })` from `@browserlessapi/inferlo/nextjs` serves a whole Inferlo app from a Next.js App Router route, `export default toH3Handler(app, { prefix: "/api" })` from `@browserlessapi/inferlo/h3` serves it from a Nuxt or Nitro server route (with `event.context.ai` from a Nitro plugin), `export const { GET, POST, fallback } = toSvelteKitHandlers(app, { prefix: "/api" })` from `@browserlessapi/inferlo/sveltekit` serves it from a SvelteKit rest route (with `event.locals.ai` from a handle hook), `export const { loader, action } = toReactRouterHandlers(app, { prefix: "/api" })` from `@browserlessapi/inferlo/react-router` serves it from a React Router (Remix) resource route, `export const handler = toLambdaHandler(app)` from `@browserlessapi/inferlo/lambda` runs it on AWS Lambda (function URLs and API Gateway, with response streaming), and an Encore.ts raw endpoint serves a whole Inferlo app with `api.raw({ expose: true, method: "*", path: "/inferlo/*rest" }, toNodeHandler(app, { prefix: "/inferlo" }))` (see [Use with other frameworks](docs/integrations.md)).
+- **Use it in any framework.** `import { createAI } from "@browserlessapi/inferlo/ai"` gives you the same AI module without `createApp`, for Express, Hono, Fastify, NestJS or a plain script, and `toNodeHandler(app, { prefix: "/ai" })` mounts a whole Inferlo app (routes, OpenAPI, MCP) inside Express, Fastify, NestJS or `node:http`, `hono.use(inferloAI(ai))` from `@browserlessapi/inferlo/hono` gives Hono handlers `c.var.ai`, `server.use(inferloAI(ai))` from `@browserlessapi/inferlo/express` gives Express handlers `req.ai`, and `fastify.register(inferloPlugin, { ai })` from `@browserlessapi/inferlo/fastify` gives Fastify `request.ai`, `server.register({ plugin: inferloPlugin, options: { ai } })` from `@browserlessapi/inferlo/hapi` gives hapi `request.ai` (and `options: { app }` with `routes: { prefix }` serves a whole Inferlo app), `new Elysia().use(inferloAI(ai))` from `@browserlessapi/inferlo/elysia` gives Elysia handlers `ai`, `InferloModule.forRoot({ ai })` from `@browserlessapi/inferlo/nestjs` makes the AI client injectable in NestJS, the provider from `@browserlessapi/inferlo/adonisjs` gives AdonisJS routes `ctx.ai`, `@Configuration({ imports: [inferloAI(ai)] })` from `@browserlessapi/inferlo/tsed` makes the AI client injectable in Ts.ED as `InferloAI`, `trpc.procedure.use(inferloAI(ai))` from `@browserlessapi/inferlo/trpc` gives tRPC procedures `ctx.ai` while `trpcTools(router, { caller, procedures })` lets a model call tRPC procedures as tools, `koa.use(inferloAI(ai))` and `koa.use(toKoaMiddleware(app, { prefix: "/api" }))` from `@browserlessapi/inferlo/koa` give Koa `ctx.ai` and a whole Inferlo app, `export const { GET, POST } = toNextHandlers(app, { prefix: "/api" })` from `@browserlessapi/inferlo/nextjs` serves a whole Inferlo app from a Next.js App Router route, `export default toH3Handler(app, { prefix: "/api" })` from `@browserlessapi/inferlo/h3` serves it from a Nuxt or Nitro server route (with `event.context.ai` from a Nitro plugin), `export const { GET, POST, fallback } = toSvelteKitHandlers(app, { prefix: "/api" })` from `@browserlessapi/inferlo/sveltekit` serves it from a SvelteKit rest route (with `event.locals.ai` from a handle hook), `export const ALL = toAstroHandler(app, { prefix: "/api" })` from `@browserlessapi/inferlo/astro` serves it from an Astro endpoint (with `locals.ai` from a middleware), `export const { loader, action } = toReactRouterHandlers(app, { prefix: "/api" })` from `@browserlessapi/inferlo/react-router` serves it from a React Router (Remix) resource route, `export const handler = toLambdaHandler(app)` from `@browserlessapi/inferlo/lambda` runs it on AWS Lambda (function URLs and API Gateway, with response streaming), `handler: toAzureHandler(app, { prefix: "/api" })` from `@browserlessapi/inferlo/azure` runs it in an Azure Functions HTTP trigger (with streaming), `functions.http("inferlo", toNodeHandler(app))` runs it as a Google Cloud Run function, `export default app` runs it as a Cloudflare Worker or a Vercel Function and `export default app.fetch` as a Netlify Function, and an Encore.ts raw endpoint serves a whole Inferlo app with `api.raw({ expose: true, method: "*", path: "/inferlo/*rest" }, toNodeHandler(app, { prefix: "/inferlo" }))` An app that already uses Drizzle ORM adds Inferlo on the same Postgres pool, with Drizzle and Inferlo writes in one transaction through `tx.$pgClient()` (see [Use with other frameworks](docs/integrations.md)).
 - **Background jobs for slow AI work.** Queue a typed job from a route with `ctx.jobs.enqueue(...)` and it runs after the response, with retries, from a Postgres queue shared by every instance (`inferlo worker` runs them in their own process), and schedule jobs with `cron("0 3 * * *", job)`.
 
 ## Why Inferlo
@@ -81,9 +81,9 @@ Inferlo is also designed so AI coding tools (Claude Code, Cursor, Codex, NeonCod
 
 ## Quick start
 
-Requires Node.js 22.18+ or Bun 1.3+. TypeScript runs directly with no build step.
+Requires Node.js 22.18+, Bun 1.3+ or Deno 2.9+. TypeScript runs directly with no build step.
 
-Every release is tested on Node.js 22.18, 22, 24, 26 and the current release; Bun 1.3.0 and latest; PostgreSQL 14, 15, 16, 17, 18 and latest; and TypeScript 5.9 and latest.
+Every release is tested on Node.js 22.18, 22, 24, 26 and the current release; Bun 1.3.0 and latest; Deno 2.9.0 and latest; PostgreSQL 14, 15, 16, 17, 18 and latest; and TypeScript 5.9 and latest.
 
 ```bash
 npx @browserlessapi/inferlo new my-app
@@ -119,7 +119,9 @@ npm install @browserlessapi/inferlo
 - [Auth](docs/auth.md)
 - [AI](docs/ai.md)
 - [Modules](docs/modules.md)
+- [Service-to-service calls](docs/services.md)
 - [Caching](docs/caching.md)
+- [Redis](docs/redis.md)
 - [MCP, OpenAPI and describe](docs/mcp.md)
 - [CLI](docs/cli.md)
 - [Testing](docs/testing.md)
@@ -132,6 +134,7 @@ npm install @browserlessapi/inferlo
 - [Security](docs/security.md)
 - [Observability](docs/observability.md)
 - [Deployment](docs/deploy.md)
+- [Performance](docs/performance.md)
 - [Error codes](docs/errors.md)
 - [Support policy](docs/support.md)
 - [Example: helpdesk with AI triage](examples/helpdesk)

package/dist/ai/ai-sdk.d.ts

@@ -0,0 +1,48 @@
+import { type Provider } from "./providers.ts";
+/** The parts of an AI SDK language model (LanguageModelV2, V3 or V4) that Inferlo uses. */
+export interface AISdkLanguageModel {
+    readonly specificationVersion: string;
+    readonly provider: string;
+    readonly modelId: string;
+    doGenerate(options: any): PromiseLike<{
+        content: readonly object[];
+        usage: unknown;
+    }>;
+    doStream(options: any): PromiseLike<{
+        stream: ReadableStream<any>;
+    }>;
+}
+/** The parts of an AI SDK embedding model (EmbeddingModelV2, V3 or V4) that Inferlo uses. */
+export interface AISdkEmbeddingModel {
+    readonly specificationVersion: string;
+    readonly modelId: string;
+    readonly maxEmbeddingsPerCall?: PromiseLike<number | undefined> | number | undefined;
+    doEmbed(options: {
+        values: string[];
+        abortSignal?: AbortSignal;
+    }): PromiseLike<{
+        embeddings: number[][];
+        usage?: {
+            tokens: number;
+        };
+    }>;
+}
+/**
+ * An AI SDK provider: what createOpenAI(), createAnthropic(), bedrock, mistral, gateway and the others
+ * return, and what customProvider() from "ai" builds. Only languageModel is required.
+ */
+export interface AISdkProvider {
+    readonly specificationVersion?: string;
+    languageModel(modelId: string): AISdkLanguageModel;
+    embeddingModel?(modelId: string): AISdkEmbeddingModel;
+    textEmbeddingModel?(modelId: string): AISdkEmbeddingModel;
+}
+export interface AISdkOptions {
+    /** Provider name in errors, usage and traces. Default: the AI SDK provider's own name (the part before the first dot, e.g. "openai"). */
+    name?: string;
+}
+/**
+ * Use any Vercel AI SDK provider in createAI({ providers }) or createApp({ ai: { providers } }):
+ * providers: { openai: aiSdk(createOpenAI({ apiKey })) }, then models: { fast: "openai:gpt-5-mini" }.
+ */
+export declare function aiSdk(provider: AISdkProvider, options?: AISdkOptions): Provider;

package/dist/ai/ai-sdk.js

@@ -0,0 +1,235 @@
+/**
+ * Vercel AI SDK providers as Inferlo providers: every @ai-sdk/* provider (OpenAI, Anthropic, Google,
+ * Mistral, Amazon Bedrock, Azure, Vertex, Groq, xAI, the AI Gateway, ...) works with aliases, fallback,
+ * usage tracking, structured output, tools, streaming and embeddings.
+ *
+ *   import { createMistral } from "@ai-sdk/mistral";
+ *   createAI({ providers: { mistral: aiSdk(createMistral({ apiKey })) }, models: { fast: "mistral:mistral-small-latest" } })
+ *
+ * Inferlo has no dependency on the AI SDK: aiSdk() calls the provider's languageModel(id) and
+ * embeddingModel(id) (textEmbeddingModel(id) on older providers) and speaks the model specification
+ * versions v2 (ai 5), v3 (ai 6) and v4 (ai 7) itself.
+ */
+import { InferloError } from "../errors.js";
+import { imageData, ProviderError } from "./providers.js";
+/** The specification versions aiSdk() speaks. */
+const VERSIONS = new Set(["v2", "v3", "v4"]);
+/**
+ * Use any Vercel AI SDK provider in createAI({ providers }) or createApp({ ai: { providers } }):
+ * providers: { openai: aiSdk(createOpenAI({ apiKey })) }, then models: { fast: "openai:gpt-5-mini" }.
+ */
+export function aiSdk(provider, options = {}) {
+    if (!provider || typeof provider !== "object" && typeof provider !== "function" || typeof provider.languageModel !== "function") {
+        throw new InferloError("IL5029", "aiSdk() needs an AI SDK provider, such as createOpenAI({ apiKey }) from @ai-sdk/openai.", {
+            fix: 'Pass the provider, not a model: aiSdk(createOpenAI({ apiKey })) with models: { fast: "openai:gpt-5-mini" }. For single models use customProvider({ languageModels: { ... } }) from "ai".',
+        });
+    }
+    if (options.name !== undefined && (typeof options.name !== "string" || !/^[a-zA-Z0-9._-]{1,64}$/.test(options.name))) {
+        throw new InferloError("IL5029", `aiSdk() name must be 1 to 64 letters, digits, dots, dashes or underscores, got ${JSON.stringify(options.name)}.`);
+    }
+    const languageModels = new Map();
+    const embeddingModels = new Map();
+    let label = options.name;
+    const nameOf = (model) => (label ??= String(model.provider ?? "ai-sdk").split(".")[0] || "ai-sdk");
+    const languageModel = (id) => {
+        let model = languageModels.get(id);
+        if (!model) {
+            model = provider.languageModel(id);
+            checkVersion(model, id, "language");
+            languageModels.set(id, model);
+        }
+        return model;
+    };
+    const embeddingModel = (id) => {
+        let model = embeddingModels.get(id);
+        if (!model) {
+            const make = provider.embeddingModel ?? provider.textEmbeddingModel;
+            if (typeof make !== "function") {
+                throw new InferloError("IL5029", `The AI SDK provider${label ? ` "${label}"` : ""} has no embedding models.`, { fix: "Use an embedding model of another provider in ai.embed({ model })." });
+            }
+            model = make.call(provider, id);
+            checkVersion(model, id, "embedding");
+            embeddingModels.set(id, model);
+        }
+        return model;
+    };
+    const out = {
+        get name() {
+            return label ?? "ai-sdk";
+        },
+        async generate(r) {
+            const model = languageModel(r.model);
+            const name = nameOf(model);
+            const v2 = model.specificationVersion === "v2";
+            let result;
+            try {
+                result = (await model.doGenerate(callOptions(r, v2)));
+            }
+            catch (error) {
+                throw providerError(name, error);
+            }
+            const text = result.content.filter((c) => c.type === "text").map((c) => c.text ?? "").join("");
+            const toolCalls = result.content
+                .filter((c) => c.type === "tool-call")
+                .map((c) => ({ id: c.toolCallId ?? "", name: c.toolName ?? "", input: parseInput(c.input) }));
+            return { text, usage: usage(result.usage), ...(toolCalls.length ? { toolCalls } : {}) };
+        },
+        async *stream(r) {
+            const model = languageModel(r.model);
+            const name = nameOf(model);
+            const v2 = model.specificationVersion === "v2";
+            let stream;
+            try {
+                ({ stream } = await model.doStream(callOptions(r, v2)));
+            }
+            catch (error) {
+                throw providerError(name, error);
+            }
+            const reader = stream.getReader();
+            let total;
+            try {
+                while (true) {
+                    const { done, value: part } = await reader.read();
+                    if (done)
+                        break;
+                    if (part.type === "text-delta") {
+                        const text = part.delta ?? part.textDelta;
+                        if (text)
+                            yield text;
+                    }
+                    else if (part.type === "finish")
+                        total = usage(part.usage);
+                    else if (part.type === "error")
+                        throw providerError(name, part.error);
+                }
+            }
+            finally {
+                reader.releaseLock();
+            }
+            yield { usage: total ?? { inputTokens: 0, outputTokens: 0 } };
+        },
+        async embed({ model: id, input, signal }) {
+            const model = embeddingModel(id);
+            const name = nameOf({ provider: model.provider });
+            const max = (await model.maxEmbeddingsPerCall) ?? input.length;
+            const embeddings = [];
+            let tokens = 0;
+            for (let i = 0; i < input.length; i += Math.max(1, max)) {
+                let result;
+                try {
+                    result = await model.doEmbed({ values: input.slice(i, i + Math.max(1, max)), abortSignal: signal });
+                }
+                catch (error) {
+                    throw providerError(name, error);
+                }
+                embeddings.push(...result.embeddings.map((e) => Array.from(e)));
+                tokens += result.usage?.tokens ?? 0;
+            }
+            return { embeddings, usage: { inputTokens: tokens, outputTokens: 0 } };
+        },
+    };
+    return out;
+}
+function checkVersion(model, id, kind) {
+    if (!model || typeof model !== "object" || !VERSIONS.has(String(model.specificationVersion))) {
+        throw new InferloError("IL5029", `The AI SDK ${kind} model "${id}" uses specification ${String(model?.specificationVersion ?? "(none)")}, which aiSdk() does not speak.`, {
+            fix: "Use a provider package made for ai 5 or later (@ai-sdk/* 2.0 or later, LanguageModelV2, V3 or V4).",
+        });
+    }
+}
+/** Inferlo's request as AI SDK call options. v2 file parts carry data or a URL directly; v3 and v4 wrap it. */
+function callOptions(r, v2) {
+    const out = { prompt: prompt(r.messages, v2), abortSignal: r.signal };
+    if (r.maxTokens !== undefined)
+        out.maxOutputTokens = r.maxTokens;
+    if (r.temperature !== undefined)
+        out.temperature = r.temperature;
+    if (r.json)
+        out.responseFormat = r.jsonSchema ? { type: "json", schema: r.jsonSchema, name: "output" } : { type: "json" };
+    if (r.tools?.length) {
+        out.tools = r.tools.map((tool) => ({ type: "function", name: tool.name, description: tool.description, inputSchema: tool.inputSchema }));
+        out.toolChoice = { type: "auto" };
+    }
+    return out;
+}
+function prompt(messages, v2) {
+    const names = new Map();
+    const out = [];
+    for (const m of messages) {
+        if (m.role === "system")
+            out.push({ role: "system", content: m.content });
+        else if (m.role === "user") {
+            const content = [];
+            if (m.content)
+                content.push({ type: "text", text: m.content });
+            for (const image of m.images ?? [])
+                content.push(filePart(image, v2));
+            out.push({ role: "user", content: content.length ? content : [{ type: "text", text: "" }] });
+        }
+        else if (m.role === "assistant") {
+            const content = [];
+            if (m.content)
+                content.push({ type: "text", text: m.content });
+            for (const call of m.toolCalls ?? []) {
+                names.set(call.id, call.name);
+                content.push({ type: "tool-call", toolCallId: call.id, toolName: call.name, input: call.input ?? {} });
+            }
+            out.push({ role: "assistant", content });
+        }
+        else {
+            const id = m.toolCallId ?? "";
+            const part = { type: "tool-result", toolCallId: id, toolName: names.get(id) ?? "", output: { type: m.isError ? "error-text" : "text", value: m.content } };
+            // Results of one assistant turn's calls go in one tool message, as the AI SDK expects.
+            const last = out.at(-1);
+            if (last?.role === "tool")
+                last.content.push(part);
+            else
+                out.push({ role: "tool", content: [part] });
+        }
+    }
+    return out;
+}
+function filePart(image, v2) {
+    const data = imageData(image);
+    if (v2)
+        return data ? { type: "file", data: data.base64, mediaType: data.mediaType } : { type: "file", data: new URL(image.url), mediaType: "image/*" };
+    return data
+        ? { type: "file", data: { type: "data", data: data.base64 }, mediaType: data.mediaType }
+        : { type: "file", data: { type: "url", url: new URL(image.url) }, mediaType: "image/*" };
+}
+/** Tool call arguments: v2 to v4 send them as a JSON string. */
+function parseInput(input) {
+    if (typeof input !== "string")
+        return input ?? {};
+    if (!input.trim())
+        return {};
+    try {
+        return JSON.parse(input);
+    }
+    catch {
+        return input;
+    }
+}
+/** Usage of v2 ({ inputTokens: number }) and v3/v4 ({ inputTokens: { total } }). */
+function usage(raw) {
+    const u = (raw ?? {});
+    const count = (value) => {
+        const n = typeof value === "object" && value !== null ? value.total : value;
+        return typeof n === "number" && Number.isFinite(n) ? n : 0;
+    };
+    return { inputTokens: count(u.inputTokens), outputTokens: count(u.outputTokens) };
+}
+/** AI SDK errors (APICallError has statusCode) as ProviderError, so fallback and IL5006 work as with other providers. */
+function providerError(name, error) {
+    if (error instanceof InferloError)
+        return error;
+    if (error && typeof error === "object" && error.name === "AbortError")
+        return error;
+    const e = (error ?? {});
+    const status = typeof e.statusCode === "number" ? e.statusCode : 0;
+    const body = typeof e.responseBody === "string" && e.responseBody ? e.responseBody : String(e.message ?? error);
+    const wrapped = new ProviderError(name, status, body);
+    wrapped.cause = error;
+    return wrapped;
+}
+//# sourceMappingURL=ai-sdk.js.map

package/dist/ai/ai-sdk.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"ai-sdk.js","sourceRoot":"","sources":["../../src/ai/ai-sdk.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;GAWG;AACH,OAAO,EAAE,YAAY,EAAE,MAAM,cAAc,CAAC;AAC5C,OAAO,EAAE,SAAS,EAAE,aAAa,EAAoG,MAAM,gBAAgB,CAAC;AAqC5J,iDAAiD;AACjD,MAAM,QAAQ,GAAG,IAAI,GAAG,CAAC,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,CAAC,CAAC;AAE7C;;;GAGG;AACH,MAAM,UAAU,KAAK,CAAC,QAAuB,EAAE,UAAwB,EAAE;IACvE,IAAI,CAAC,QAAQ,IAAI,OAAO,QAAQ,KAAK,QAAQ,IAAI,OAAO,QAAQ,KAAK,UAAU,IAAI,OAAQ,QAA0B,CAAC,aAAa,KAAK,UAAU,EAAE,CAAC;QACnJ,MAAM,IAAI,YAAY,CAAC,QAAQ,EAAE,yFAAyF,EAAE;YAC1H,GAAG,EAAE,0LAA0L;SAChM,CAAC,CAAC;IACL,CAAC;IACD,IAAI,OAAO,CAAC,IAAI,KAAK,SAAS,IAAI,CAAC,OAAO,OAAO,CAAC,IAAI,KAAK,QAAQ,IAAI,CAAC,wBAAwB,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,EAAE,CAAC;QACrH,MAAM,IAAI,YAAY,CAAC,QAAQ,EAAE,kFAAkF,IAAI,CAAC,SAAS,CAAC,OAAO,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;IACtJ,CAAC;IACD,MAAM,cAAc,GAAG,IAAI,GAAG,EAA8B,CAAC;IAC7D,MAAM,eAAe,GAAG,IAAI,GAAG,EAA+B,CAAC;IAC/D,IAAI,KAAK,GAAG,OAAO,CAAC,IAAI,CAAC;IACzB,MAAM,MAAM,GAAG,CAAC,KAA4B,EAAE,EAAE,CAAC,CAAC,KAAK,KAAK,MAAM,CAAC,KAAK,CAAC,QAAQ,IAAI,QAAQ,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,QAAQ,CAAC,CAAC;IAE1H,MAAM,aAAa,GAAG,CAAC,EAAU,EAAsB,EAAE;QACvD,IAAI,KAAK,GAAG,cAAc,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;QACnC,IAAI,CAAC,KAAK,EAAE,CAAC;YACX,KAAK,GAAG,QAAQ,CAAC,aAAa,CAAC,EAAE,CAAC,CAAC;YACnC,YAAY,CAAC,KAAK,EAAE,EAAE,EAAE,UAAU,CAAC,CAAC;YACpC,cAAc,CAAC,GAAG,CAAC,EAAE,EAAE,KAAK,CAAC,CAAC;QAChC,CAAC;QACD,OAAO,KAAK,CAAC;IACf,CAAC,CAAC;IAEF,MAAM,cAAc,GAAG,CAAC,EAAU,EAAuB,EAAE;QACzD,IAAI,KAAK,GAAG,eAAe,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;QACpC,IAAI,CAAC,KAAK,EAAE,CAAC;YACX,MAAM,IAAI,GAAG,QAAQ,CAAC,cAAc,IAAI,QAAQ,CAAC,kBAAkB,CAAC;YACpE,IAAI,OAAO,IAAI,KAAK,UAAU,EAAE,CAAC;gBAC/B,MAAM,IAAI,YAAY,CAAC,QAAQ,EAAE,sBAAsB,KAAK,CAAC,CAAC,CAAC,KAAK,KAAK,GAAG,CAAC,CAAC,CAAC,EAAE,2BAA2B,EAAE,EAAE,GAAG,EAAE,oEAAoE,EAAE,CAAC,CAAC;YAC/L,CAAC;YACD,KAAK,GAAG,IAAI,CAAC,IAAI,CAAC,QAAQ,EAAE,EAAE,CAAC,CAAC;YAChC,YAAY,CAAC,KAAK,EAAE,EAAE,EAAE,WAAW,CAAC,CAAC;YACrC,eAAe,CAAC,GAAG,CAAC,EAAE,EAAE,KAAK,CAAC,CAAC;QACjC,CAAC;QACD,OAAO,KAAK,CAAC;IACf,CAAC,CAAC;IAEF,MAAM,GAAG,GAAa;QACpB,IAAI,IAAI;YACN,OAAO,KAAK,IAAI,QAAQ,CAAC;QAC3B,CAAC;QACD,KAAK,CAAC,QAAQ,CAAC,CAAC;YACd,MAAM,KAAK,GAAG,aAAa,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC;YACrC,MAAM,IAAI,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC;YAC3B,MAAM,EAAE,GAAG,KAAK,CAAC,oBAAoB,KAAK,IAAI,CAAC;YAC/C,IAAI,MAA4D,CAAC;YACjE,IAAI,CAAC;gBACH,MAAM,GAAG,CAAC,MAAM,KAAK,CAAC,UAAU,CAAC,WAAW,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAkB,CAAC;YACzE,CAAC;YAAC,OAAO,KAAK,EAAE,CAAC;gBACf,MAAM,aAAa,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;YACnC,CAAC;YACD,MAAM,IAAI,GAAG,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,KAAK,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,IAAI,EAAE,CAAC,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;YAC/F,MAAM,SAAS,GAAe,MAAM,CAAC,OAAO;iBACzC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,KAAK,WAAW,CAAC;iBACrC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC,UAAU,IAAI,EAAE,EAAE,IAAI,EAAE,CAAC,CAAC,QAAQ,IAAI,EAAE,EAAE,KAAK,EAAE,UAAU,CAAC,CAAC,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC,CAAC;YAChG,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,KAAK,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,GAAG,CAAC,SAAS,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,SAAS,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC;QAC1F,CAAC;QACD,KAAK,CAAC,CAAC,MAAM,CAAC,CAAC;YACb,MAAM,KAAK,GAAG,aAAa,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC;YACrC,MAAM,IAAI,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC;YAC3B,MAAM,EAAE,GAAG,KAAK,CAAC,oBAAoB,KAAK,IAAI,CAAC;YAC/C,IAAI,MAA8G,CAAC;YACnH,IAAI,CAAC;gBACH,CAAC,EAAE,MAAM,EAAE,GAAG,MAAM,KAAK,CAAC,QAAQ,CAAC,WAAW,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC;YAC1D,CAAC;YAAC,OAAO,KAAK,EAAE,CAAC;gBACf,MAAM,aAAa,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;YACnC,CAAC;YACD,MAAM,MAAM,GAAG,MAAM,CAAC,SAAS,EAAE,CAAC;YAClC,IAAI,KAAgC,CAAC;YACrC,IAAI,CAAC;gBACH,OAAO,IAAI,EAAE,CAAC;oBACZ,MAAM,EAAE,IAAI,EAAE,KAAK,EAAE,IAAI,EAAE,GAAG,MAAM,MAAM,CAAC,IAAI,EAAE,CAAC;oBAClD,IAAI,IAAI;wBAAE,MAAM;oBAChB,IAAI,IAAI,CAAC,IAAI,KAAK,YAAY,EAAE,CAAC;wBAC/B,MAAM,IAAI,GAAG,IAAI,CAAC,KAAK,IAAI,IAAI,CAAC,SAAS,CAAC;wBAC1C,IAAI,IAAI;4BAAE,MAAM,IAAI,CAAC;oBACvB,CAAC;yBAAM,IAAI,IAAI,CAAC,IAAI,KAAK,QAAQ;wBAAE,KAAK,GAAG,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;yBACxD,IAAI,IAAI,CAAC,IAAI,KAAK,OAAO;wBAAE,MAAM,aAAa,CAAC,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC;gBACxE,CAAC;YACH,CAAC;oBAAS,CAAC;gBACT,MAAM,CAAC,WAAW,EAAE,CAAC;YACvB,CAAC;YACD,MAAM,EAAE,KAAK,EAAE,KAAK,IAAI,EAAE,WAAW,EAAE,CAAC,EAAE,YAAY,EAAE,CAAC,EAAE,EAAE,CAAC;QAChE,CAAC;QACD,KAAK,CAAC,KAAK,CAAC,EAAE,KAAK,EAAE,EAAE,EAAE,KAAK,EAAE,MAAM,EAAE;YACtC,MAAM,KAAK,GAAG,cAAc,CAAC,EAAE,CAAC,CAAC;YACjC,MAAM,IAAI,GAAG,MAAM,CAAC,EAAE,QAAQ,EAAG,KAA+B,CAAC,QAAQ,EAAE,CAAC,CAAC;YAC7E,MAAM,GAAG,GAAG,CAAC,MAAM,KAAK,CAAC,oBAAoB,CAAC,IAAI,KAAK,CAAC,MAAM,CAAC;YAC/D,MAAM,UAAU,GAAe,EAAE,CAAC;YAClC,IAAI,MAAM,GAAG,CAAC,CAAC;YACf,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC,IAAI,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE,CAAC;gBACxD,IAAI,MAA8D,CAAC;gBACnE,IAAI,CAAC;oBACH,MAAM,GAAG,MAAM,KAAK,CAAC,OAAO,CAAC,EAAE,MAAM,EAAE,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,EAAE,WAAW,EAAE,MAAM,EAAE,CAAC,CAAC;gBACtG,CAAC;gBAAC,OAAO,KAAK,EAAE,CAAC;oBACf,MAAM,aAAa,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;gBACnC,CAAC;gBACD,UAAU,CAAC,IAAI,CAAC,GAAG,MAAM,CAAC,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;gBAChE,MAAM,IAAI,MAAM,CAAC,KAAK,EAAE,MAAM,IAAI,CAAC,CAAC;YACtC,CAAC;YACD,OAAO,EAAE,UAAU,EAAE,KAAK,EAAE,EAAE,WAAW,EAAE,MAAM,EAAE,YAAY,EAAE,CAAC,EAAE,EAAE,CAAC;QACzE,CAAC;KACF,CAAC;IACF,OAAO,GAAG,CAAC;AACb,CAAC;AAED,SAAS,YAAY,CAAC,KAAqD,EAAE,EAAU,EAAE,IAAY;IACnG,IAAI,CAAC,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,MAAM,CAAC,KAAK,CAAC,oBAAoB,CAAC,CAAC,EAAE,CAAC;QAC7F,MAAM,IAAI,YAAY,CAAC,QAAQ,EAAE,cAAc,IAAI,WAAW,EAAE,wBAAwB,MAAM,CAAC,KAAK,EAAE,oBAAoB,IAAI,QAAQ,CAAC,iCAAiC,EAAE;YACxK,GAAG,EAAE,oGAAoG;SAC1G,CAAC,CAAC;IACL,CAAC;AACH,CAAC;AAED,+GAA+G;AAC/G,SAAS,WAAW,CAAC,CAAkB,EAAE,EAAW;IAClD,MAAM,GAAG,GAA4B,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC,CAAC,QAAQ,EAAE,EAAE,CAAC,EAAE,WAAW,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC;IAC/F,IAAI,CAAC,CAAC,SAAS,KAAK,SAAS;QAAE,GAAG,CAAC,eAAe,GAAG,CAAC,CAAC,SAAS,CAAC;IACjE,IAAI,CAAC,CAAC,WAAW,KAAK,SAAS;QAAE,GAAG,CAAC,WAAW,GAAG,CAAC,CAAC,WAAW,CAAC;IACjE,IAAI,CAAC,CAAC,IAAI;QAAE,GAAG,CAAC,cAAc,GAAG,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,CAAC,CAAC,UAAU,EAAE,IAAI,EAAE,QAAQ,EAAE,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,CAAC;IAC1H,IAAI,CAAC,CAAC,KAAK,EAAE,MAAM,EAAE,CAAC;QACpB,GAAG,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,UAAU,EAAE,IAAI,EAAE,IAAI,CAAC,IAAI,EAAE,WAAW,EAAE,IAAI,CAAC,WAAW,EAAE,WAAW,EAAE,IAAI,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC;QACzI,GAAG,CAAC,UAAU,GAAG,EAAE,IAAI,EAAE,MAAM,EAAE,CAAC;IACpC,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,SAAS,MAAM,CAAC,QAAmB,EAAE,EAAW;IAC9C,MAAM,KAAK,GAAG,IAAI,GAAG,EAAkB,CAAC;IACxC,MAAM,GAAG,GAAc,EAAE,CAAC;IAC1B,KAAK,MAAM,CAAC,IAAI,QAAQ,EAAE,CAAC;QACzB,IAAI,CAAC,CAAC,IAAI,KAAK,QAAQ;YAAE,GAAG,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC;aACrE,IAAI,CAAC,CAAC,IAAI,KAAK,MAAM,EAAE,CAAC;YAC3B,MAAM,OAAO,GAAc,EAAE,CAAC;YAC9B,IAAI,CAAC,CAAC,OAAO;gBAAE,OAAO,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC;YAC/D,KAAK,MAAM,KAAK,IAAI,CAAC,CAAC,MAAM,IAAI,EAAE;gBAAE,OAAO,CAAC,IAAI,CAAC,QAAQ,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC,CAAC;YACtE,GAAG,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,EAAE,EAAE,CAAC,EAAE,CAAC,CAAC;QAC/F,CAAC;aAAM,IAAI,CAAC,CAAC,IAAI,KAAK,WAAW,EAAE,CAAC;YAClC,MAAM,OAAO,GAAc,EAAE,CAAC;YAC9B,IAAI,CAAC,CAAC,OAAO;gBAAE,OAAO,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC;YAC/D,KAAK,MAAM,IAAI,IAAI,CAAC,CAAC,SAAS,IAAI,EAAE,EAAE,CAAC;gBACrC,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,EAAE,IAAI,CAAC,IAAI,CAAC,CAAC;gBAC9B,OAAO,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,WAAW,EAAE,UAAU,EAAE,IAAI,CAAC,EAAE,EAAE,QAAQ,EAAE,IAAI,CAAC,IAAI,EAAE,KAAK,EAAE,IAAI,CAAC,KAAK,IAAI,EAAE,EAAE,CAAC,CAAC;YACzG,CAAC;YACD,GAAG,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,WAAW,EAAE,OAAO,EAAE,CAAC,CAAC;QAC3C,CAAC;aAAM,CAAC;YACN,MAAM,EAAE,GAAG,CAAC,CAAC,UAAU,IAAI,EAAE,CAAC;YAC9B,MAAM,IAAI,GAAG,EAAE,IAAI,EAAE,aAAa,EAAE,UAAU,EAAE,EAAE,EAAE,QAAQ,EAAE,KAAK,CAAC,GAAG,CAAC,EAAE,CAAC,IAAI,EAAE,EAAE,MAAM,EAAE,EAAE,IAAI,EAAE,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,MAAM,EAAE,KAAK,EAAE,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC;YAC3J,uFAAuF;YACvF,MAAM,IAAI,GAAG,GAAG,CAAC,EAAE,CAAC,CAAC,CAAC,CAAqD,CAAC;YAC5E,IAAI,IAAI,EAAE,IAAI,KAAK,MAAM;gBAAE,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;;gBAC9C,GAAG,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QACnD,CAAC;IACH,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,SAAS,QAAQ,CAAC,KAAY,EAAE,EAAW;IACzC,MAAM,IAAI,GAAG,SAAS,CAAC,KAAK,CAAC,CAAC;IAC9B,IAAI,EAAE;QAAE,OAAO,IAAI,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,IAAI,CAAC,MAAM,EAAE,SAAS,EAAE,IAAI,CAAC,SAAS,EAAE,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,IAAI,GAAG,CAAE,KAAyB,CAAC,GAAG,CAAC,EAAE,SAAS,EAAE,SAAS,EAAE,CAAC;IAC7K,OAAO,IAAI;QACT,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,IAAI,CAAC,MAAM,EAAE,EAAE,SAAS,EAAE,IAAI,CAAC,SAAS,EAAE;QACxF,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,EAAE,IAAI,EAAE,KAAK,EAAE,GAAG,EAAE,IAAI,GAAG,CAAE,KAAyB,CAAC,GAAG,CAAC,EAAE,EAAE,SAAS,EAAE,SAAS,EAAE,CAAC;AAClH,CAAC;AAED,gEAAgE;AAChE,SAAS,UAAU,CAAC,KAAc;IAChC,IAAI,OAAO,KAAK,KAAK,QAAQ;QAAE,OAAO,KAAK,IAAI,EAAE,CAAC;IAClD,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE;QAAE,OAAO,EAAE,CAAC;IAC7B,IAAI,CAAC;QACH,OAAO,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;IAC3B,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,KAAK,CAAC;IACf,CAAC;AACH,CAAC;AAED,oFAAoF;AACpF,SAAS,KAAK,CAAC,GAAY;IACzB,MAAM,CAAC,GAAG,CAAC,GAAG,IAAI,EAAE,CAAsD,CAAC;IAC3E,MAAM,KAAK,GAAG,CAAC,KAAc,EAAU,EAAE;QACvC,MAAM,CAAC,GAAG,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,KAAK,IAAI,CAAC,CAAC,CAAE,KAA6B,CAAC,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC;QACrG,OAAO,OAAO,CAAC,KAAK,QAAQ,IAAI,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IAC7D,CAAC,CAAC;IACF,OAAO,EAAE,WAAW,EAAE,KAAK,CAAC,CAAC,CAAC,WAAW,CAAC,EAAE,YAAY,EAAE,KAAK,CAAC,CAAC,CAAC,YAAY,CAAC,EAAE,CAAC;AACpF,CAAC;AAED,yHAAyH;AACzH,SAAS,aAAa,CAAC,IAAY,EAAE,KAAc;IACjD,IAAI,KAAK,YAAY,YAAY;QAAE,OAAO,KAAK,CAAC;IAChD,IAAI,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAK,KAAe,CAAC,IAAI,KAAK,YAAY;QAAE,OAAO,KAAK,CAAC;IAC/F,MAAM,CAAC,GAAG,CAAC,KAAK,IAAI,EAAE,CAAwE,CAAC;IAC/F,MAAM,MAAM,GAAG,OAAO,CAAC,CAAC,UAAU,KAAK,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC;IACnE,MAAM,IAAI,GAAG,OAAO,CAAC,CAAC,YAAY,KAAK,QAAQ,IAAI,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,OAAO,IAAI,KAAK,CAAC,CAAC;IAChH,MAAM,OAAO,GAAG,IAAI,aAAa,CAAC,IAAI,EAAE,MAAM,EAAE,IAAI,CAAC,CAAC;IACrD,OAA+B,CAAC,KAAK,GAAG,KAAK,CAAC;IAC/C,OAAO,OAAO,CAAC;AACjB,CAAC"}

package/dist/ai/index.d.ts

@@ -2,10 +2,11 @@ import { type AnySchema, type Infer } from "../schema.ts";
 import type { RouteDef } from "../route.ts";
 import { type ToolDef } from "./tool.ts";
 import { type TracingConfig } from "../tracing.ts";
-import { type Image, type Message, type Provider, type ProviderUsage, type ToolSpec } from "./providers.ts";
+import { type Image, type Message, type Provider, type ProviderUsage, type ToolCall, type ToolSpec } from "./providers.ts";
 export * from "./providers.ts";
 export { tool, isTool, ToolBuilder } from "./tool.ts";
 export type { ToolDef, FunctionToolOptions, ToolContext } from "./tool.ts";
+export { uiChatBody, fromUIMessages, type UIChatMessage } from "./ui.ts";
 export interface Pricing {
     /** USD per million input tokens. */
     input: number;
@@ -114,18 +115,51 @@ export type ToolRunner = (route: RouteDef, input: unknown) => Promise<{
     output: unknown;
     text: string;
 }>;
+/** One model call that returns the tool calls instead of running them, for agent loops such as LangChain's. */
+export interface ModelCallOptions {
+    model?: string;
+    messages: Message[];
+    tools?: ToolSpec[];
+    temperature?: number;
+    maxTokens?: number;
+    for?: string;
+    signal?: AbortSignal;
+}
+export interface ModelCallResult {
+    text: string;
+    toolCalls: ToolCall[];
+    usage: Usage;
+}
+/**
+ * One model call with the AI's aliases, fallback, budgets, usage tracking and tracing, which returns the tool calls
+ * the model asked for without running them. Internal: used by the LangChain integration.
+ */
+export declare function callModel(ai: AI, options: ModelCallOptions): Promise<ModelCallResult>;
+/** The route tools of ctx.ai (a request's AI client), or undefined for other clients. Internal. */
+export declare function routeToolsOf(ai: AI): {
+    runner: ToolRunner;
+    describe: (route: RouteDef) => ToolSpec;
+} | undefined;
 /**
  * ctx.ai for one request: the same AI, with tools that run as the request's user, and the request's signal
  * on every call that does not pass its own. Internal: used by createApp.
  */
 export declare function bindTools(ai: AI, runner: ToolRunner, describe: (route: RouteDef) => ToolSpec, signal?: AbortSignal): AI;
+/** Options of StreamResult.toResponse: a ResponseInit (status, headers) and the stream format. */
+export interface StreamResponseInit extends ResponseInit {
+    /** "sse" (default): Inferlo's text and done events. "ai-sdk": the AI SDK UI message stream protocol, for useChat. */
+    format?: "sse" | "ai-sdk";
+}
 export interface StreamResult {
     /** Text chunks as they arrive. */
     [Symbol.asyncIterator](): AsyncIterator<string>;
     /** Full text once the stream finishes. */
     text(): Promise<string>;
-    /** A server-sent events Response for browsers (events: "text", then "done" with usage). */
-    toResponse(init?: ResponseInit): Response;
+    /**
+     * A server-sent events Response for browsers. The default format sends "text" events, then "done" with usage.
+     * format: "ai-sdk" speaks the AI SDK UI message stream protocol (v1), for useChat from @ai-sdk/react.
+     */
+    toResponse(init?: StreamResponseInit): Response;
     /** Usage once the stream finishes. */
     usage(): Promise<Usage>;
 }

package/dist/ai/index.js

@@ -17,7 +17,23 @@ import { checkTracing, clientTracing } from "../tracing.js";
 import { ProviderError, imageData } from "./providers.js";
 export * from "./providers.js";
 export { tool, isTool, ToolBuilder } from "./tool.js";
+export { uiChatBody, fromUIMessages } from "./ui.js";
 const toolRunners = new WeakMap();
+const modelCalls = new WeakMap();
+/**
+ * One model call with the AI's aliases, fallback, budgets, usage tracking and tracing, which returns the tool calls
+ * the model asked for without running them. Internal: used by the LangChain integration.
+ */
+export function callModel(ai, options) {
+    const call = modelCalls.get(ai);
+    if (!call)
+        throw new InferloError("IL5030", "Expected an AI client made by createAI() or createApp({ ai }).", { fix: "Pass app.ai, ctx.ai or the client createAI() returns." });
+    return call(options);
+}
+/** The route tools of ctx.ai (a request's AI client), or undefined for other clients. Internal. */
+export function routeToolsOf(ai) {
+    return toolRunners.get(ai);
+}
 /**
  * ctx.ai for one request: the same AI, with tools that run as the request's user, and the request's signal
  * on every call that does not pass its own. Internal: used by createApp.
@@ -29,6 +45,9 @@ export function bindTools(ai, runner, describe, signal) {
     bound.generate = (options) => ai.generate.call(bound, withSignal(options));
     bound.stream = (options) => ai.stream.call(bound, withSignal(options));
     bound.embed = (options) => ai.embed.call(bound, withSignal(options));
+    const call = modelCalls.get(ai);
+    if (call)
+        modelCalls.set(bound, (options) => call(withSignal(options)));
     return bound;
 }
 function buildMessages(o) {
@@ -469,23 +488,59 @@ export function createAI(config) {
                 },
                 usage: () => usagePromise,
                 toResponse(init) {
+                    const { format = "sse", ...responseInit } = init ?? {};
+                    if (format !== "sse" && format !== "ai-sdk") {
+                        throw new InferloError("IL5027", `Unknown stream format "${String(format)}" in toResponse({ format }).`, {
+                            fix: 'Use toResponse() for Inferlo\'s server-sent events, or toResponse({ format: "ai-sdk" }) for useChat from the Vercel AI SDK.',
+                        });
+                    }
                     const encoder = new TextEncoder();
                     const source = start();
+                    const aiSdk = format === "ai-sdk";
+                    const textId = aiSdk ? `text-${crypto.randomUUID()}` : "";
+                    let opened = false;
+                    // AI SDK UI message stream chunks are server-sent events with only a data field.
+                    const chunk = (value) => encoder.encode(`data: ${JSON.stringify(value)}\n\n`);
                     const body = new ReadableStream({
+                        start(controller) {
+                            if (aiSdk)
+                                controller.enqueue(chunk({ type: "start" }));
+                        },
                         async pull(controller) {
                             try {
                                 const { value, done } = await source.next();
                                 if (done) {
                                     const usage = await usagePromise;
-                                    controller.enqueue(encoder.encode(`event: done\ndata: ${JSON.stringify({ usage })}\n\n`));
+                                    if (!aiSdk)
+                                        controller.enqueue(encoder.encode(`event: done\ndata: ${JSON.stringify({ usage })}\n\n`));
+                                    else {
+                                        if (opened)
+                                            controller.enqueue(chunk({ type: "text-end", id: textId }));
+                                        controller.enqueue(chunk({ type: "finish" }));
+                                        controller.enqueue(encoder.encode("data: [DONE]\n\n"));
+                                    }
                                     controller.close();
                                     return;
                                 }
-                                controller.enqueue(encoder.encode(`event: text\ndata: ${JSON.stringify(value)}\n\n`));
+                                if (!aiSdk) {
+                                    controller.enqueue(encoder.encode(`event: text\ndata: ${JSON.stringify(value)}\n\n`));
+                                    return;
+                                }
+                                if (!opened) {
+                                    opened = true;
+                                    controller.enqueue(chunk({ type: "text-start", id: textId }));
+                                }
+                                controller.enqueue(chunk({ type: "text-delta", id: textId, delta: value }));
                             }
                             catch (error) {
                                 const code = error instanceof InferloError ? error.code : "IL9000";
-                                controller.enqueue(encoder.encode(`event: error\ndata: ${JSON.stringify({ code })}\n\n`));
+                                if (!aiSdk)
+                                    controller.enqueue(encoder.encode(`event: error\ndata: ${JSON.stringify({ code })}\n\n`));
+                                else {
+                                    // Only the code reaches the browser; the details are logged below.
+                                    controller.enqueue(chunk({ type: "error", errorText: `The model call failed (${code}).` }));
+                                    controller.enqueue(encoder.encode("data: [DONE]\n\n"));
+                                }
                                 controller.close();
                                 console.error("[inferlo] stream failed:", redactor.value(error));
                             }
@@ -494,10 +549,13 @@ export function createAI(config) {
                             void source.return(undefined);
                         },
                     });
-                    return new Response(body, {
-                        ...init,
-                        headers: { "content-type": "text/event-stream; charset=utf-8", "cache-control": "no-cache", connection: "keep-alive", ...init?.headers },
-                    });
+                    const headers = { "content-type": "text/event-stream; charset=utf-8", "cache-control": "no-cache", connection: "keep-alive" };
+                    if (aiSdk) {
+                        headers["content-type"] = "text/event-stream";
+                        headers["x-vercel-ai-ui-message-stream"] = "v1";
+                        headers["x-accel-buffering"] = "no";
+                    }
+                    return new Response(body, { ...responseInit, headers: { ...headers, ...responseInit.headers } });
                 },
             };
         },
@@ -525,6 +583,37 @@ export function createAI(config) {
             });
         },
     };
+    modelCalls.set(ai, async (options) => {
+        const requested = options.model ?? config?.defaultModel ?? Object.keys(aliases)[0] ?? "";
+        const targets = resolve(options.model);
+        await guard(options.for, requested);
+        if (!options.messages.some((m) => m.role !== "system")) {
+            throw new InferloError("IL5007", "A model call needs at least one message besides the system message.", { fix: 'Pass messages: [{ role: "user", content: "..." }].' });
+        }
+        options.messages.forEach(checkImages);
+        const started = Date.now();
+        const call = () => withFallback(targets, async (target) => {
+            const response = await target.provider.generate({
+                model: target.model,
+                messages: options.messages,
+                temperature: options.temperature,
+                maxTokens: options.maxTokens,
+                tools: options.tools?.length ? options.tools : undefined,
+                signal: options.signal,
+            });
+            const usage = { requested, model: target.id, ...response.usage, cost: cost(target.id, response.usage), for: options.for, kind: "generate", durationMs: Date.now() - started };
+            await record(usage);
+            return { text: response.text, toolCalls: response.toolCalls ?? [], usage };
+        });
+        if (!spans)
+            return call();
+        const { name, attributes } = spanStart("chat", requested, targets, options);
+        return spans.run(name, attributes, async (span) => {
+            const result = await call();
+            spanUsage(span, result.usage);
+            return result;
+        });
+    });
     return ai;
 }
 /** Cosine similarity of two vectors, for small in-memory searches. */