@opencode-ai/ai 0.0.0-bootstrap.0 → 0.0.0-next-15617

package/README.md

@@ -1,3 +1,178 @@
 # @opencode-ai/ai
 
-Bootstrap release. Use a current `next` or `latest` version for the real package.
+Schema-first LLM core for opencode. One typed request, response, event, and tool language; provider quirks live in adapters, not in calling code.
+
+```ts
+import { Effect } from "effect"
+import { LLM, LLMClient } from "@opencode-ai/ai"
+import { OpenAI } from "@opencode-ai/ai/providers"
+
+const model = OpenAI.configure({ apiKey: process.env.OPENAI_API_KEY }).responses("gpt-4o-mini")
+
+const request = LLM.request({
+  model,
+  system: "You are concise.",
+  prompt: "Say hello in one short sentence.",
+  generation: { maxTokens: 40 },
+})
+
+const program = Effect.gen(function* () {
+  const response = yield* LLMClient.generate(request)
+  console.log(response.text)
+})
+```
+
+Run `LLMClient.stream(request)` instead of `generate` when you want incremental `LLMEvent`s. The event stream is provider-neutral — same shape across OpenAI Chat, OpenAI Responses, Anthropic Messages, Gemini, Bedrock Converse, and any OpenAI-compatible deployment.
+
+## Public API
+
+- **`LLM.request({...})`** — build a provider-neutral `LLMRequest`. Accepts ergonomic inputs (`system: string`, `prompt: string`) that normalize into the canonical Schema classes.
+- **`LLM.generate` / `LLM.stream`** — re-exported from `LLMClient` for one-import use.
+- **`Message.user(...)` / `Message.assistant(...)` / `Message.tool(...)`** — message constructors from the canonical schema model.
+- **`Model.make(...)` / `ToolCallPart.make(...)` / `ToolResultPart.make(...)` / `ToolDefinition.make(...)`** — model and tool-related constructors from the canonical schema model.
+- **`LLMClient.prepare(request)`** — compile a request through protocol body construction, validation, and HTTP preparation without sending. Useful for inspection and testing.
+- **`LLMEvent.is.*`** — typed guards (`is.textDelta`, `is.toolCall`, `is.finish`, …) for filtering streams.
+
+## Caching
+
+Prompt caching is **on by default**. Every `LLMRequest` resolves to `cache: "auto"` unless the caller opts out with `cache: "none"`. Each protocol translates `CacheHint`s to its wire format (`cache_control` on Anthropic, `cachePoint` on Bedrock; OpenAI and Gemini do implicit caching server-side and don't need inline markers — auto is a no-op there).
+
+### Auto placement
+
+`"auto"` places three breakpoints — last tool definition, last system part, latest user message. The last-user-message boundary is the load-bearing detail: in a tool-use loop, a single user turn expands into many assistant/tool round-trips, all sharing that prefix. Caching at that boundary lets every intra-turn API call hit.
+
+The math justifies the default: Anthropic's 5-minute cache write is 1.25× base, read is 0.1×, so a single reuse within 5 minutes already wins. One-shot completions below the per-model minimum-cacheable-token threshold silently no-op on the wire, so the worst case is harmless.
+
+### Opting out
+
+```ts
+LLM.request({
+  model,
+  system,
+  prompt: "one-off question",
+  cache: "none",
+})
+```
+
+### Granular policy
+
+```ts
+cache: {
+  tools?: boolean,
+  system?: boolean,
+  messages?: "latest-user-message" | "latest-assistant" | { tail: number },
+  ttlSeconds?: number,         // ≥ 3600 → 1h on Anthropic/Bedrock; else 5m
+}
+```
+
+### Manual hints
+
+Inline `CacheHint` on any text / system / tool / tool-result part overrides automatic placement. The auto policy preserves manual hints; it only fills gaps.
+
+```ts
+LLM.request({
+  model,
+  system: [
+    { type: "text", text: "stable system prompt", cache: { type: "ephemeral" } },
+  ],
+  ...
+})
+```
+
+### Provider behavior table
+
+| Protocol                | `cache: "auto"`                                                           |
+| ----------------------- | ------------------------------------------------------------------------- |
+| Anthropic Messages      | emits up to 3 `cache_control` markers (4-breakpoint cap enforced)         |
+| Bedrock Converse        | emits up to 3 `cachePoint` blocks (4-breakpoint cap enforced)             |
+| OpenAI Chat / Responses | no-op (implicit caching above 1024 tokens)                                |
+| Gemini                  | no-op (implicit caching on 2.5+; explicit `CachedContent` is out-of-band) |
+
+Normalized cache usage is read back into `response.usage.cacheReadInputTokens` and `cacheWriteInputTokens` across every provider.
+
+## Providers
+
+Provider facades configure endpoint/auth/deployment details first, then expose model selectors that take only a model or deployment id. The selected model carries the executable route value used at runtime.
+
+```ts
+import { OpenAI, CloudflareAIGateway } from "@opencode-ai/ai/providers"
+
+const openai = OpenAI.configure({ apiKey: process.env.OPENAI_API_KEY }).responses("gpt-4o-mini")
+const gateway = CloudflareAIGateway.configure({
+  accountId: process.env.CLOUDFLARE_ACCOUNT_ID,
+  gatewayApiKey: process.env.CLOUDFLARE_API_TOKEN,
+}).model("workers-ai/@cf/meta/llama-3.1-8b-instruct")
+```
+
+Included providers: OpenAI, Anthropic, Google (Gemini), Google Vertex Gemini and Anthropic, Amazon Bedrock, Azure OpenAI, Cloudflare AI Gateway, Cloudflare Workers AI, GitHub Copilot, OpenRouter, xAI, plus generic OpenAI-compatible Chat and Responses entrypoints and an Anthropic Messages-compatible entrypoint.
+
+### Package-like entrypoints
+
+Native catalog integrations load provider behavior through package-like entrypoints. These are export paths from the same `@opencode-ai/ai` npm package, not independently published packages. Each entrypoint exports the same `model(modelID, settings)` contract, and `settings` contains serializable provider configuration plus common `headers`, `body`, and `limits` overlays.
+
+```ts
+import { model } from "@opencode-ai/ai/providers/openai/responses"
+
+const selected = model("gpt-5", {
+  apiKey: process.env.OPENAI_API_KEY,
+  transport: "websocket",
+  headers: { "x-application": "opencode" },
+  limits: { context: 200_000, output: 64_000 },
+})
+```
+
+OpenAI Chat and OpenAI Responses are separate semantic entrypoints:
+
+- `@opencode-ai/ai/providers/openai/chat`
+- `@opencode-ai/ai/providers/openai/responses`
+- `@opencode-ai/ai/providers/openai-compatible/responses`
+- `@opencode-ai/ai/providers/anthropic-compatible`
+- `@opencode-ai/ai/providers/google-vertex`
+- `@opencode-ai/ai/providers/google-vertex/anthropic`
+
+Responses HTTP versus WebSocket is a scoped `transport` setting on the OpenAI Responses entrypoint, not another entrypoint. Azure follows the same Chat/Responses split at `providers/azure/chat` and `providers/azure/responses`. Generic OpenAI-compatible Chat remains at `providers/openai-compatible`; compatible Responses is separate at `providers/openai-compatible/responses`. Generic Anthropic Messages-compatible providers use `providers/anthropic-compatible`, which the named Anthropic provider composes. Google Gemini and Amazon Bedrock expose their single native API through their existing provider paths.
+
+Vertex Gemini and Vertex Anthropic are separate products with separate entrypoints. Both accept `project`, `location`, and an optional `accessToken`; when no explicit token or auth override is supplied they lazily use Google Application Default Credentials. Vertex Gemini instead selects express mode when `apiKey` or `GOOGLE_VERTEX_API_KEY` is present.
+
+Tuned Vertex Gemini deployments use model ids shaped like `endpoints/1234567890` and require OAuth or ADC; Vertex express-mode API keys support publisher models only.
+
+```ts
+import { model } from "@opencode-ai/ai/providers/google-vertex"
+
+model("gemini-3.5-flash", { project: "my-project", location: "global" })
+```
+
+```ts
+import { model } from "@opencode-ai/ai/providers/google-vertex/anthropic"
+
+model("claude-sonnet-4-6", { project: "my-project", location: "global" })
+```
+
+Provider facades such as `OpenAI.configure(...).responses(...)` remain the direct application API. Package-like entrypoints are the self-similar loading contract used when a catalog selects behavior by export path.
+
+Other provider exports listed above remain direct facades until they explicitly implement the package-like contract. Exporting a provider facade does not implicitly make it a catalog-loadable provider package.
+
+## Provider options & HTTP overlays
+
+Three escape hatches in order of stability:
+
+1. **`generation`** — portable knobs (`maxTokens`, `temperature`, `topP`, `topK`, penalties, seed, stop).
+2. **`providerOptions: { <provider>: {...} }`** — typed-at-the-facade provider-specific knobs (OpenAI `promptCacheKey`, Anthropic `thinking`, Gemini `thinkingConfig`, OpenRouter routing).
+3. **`http: { body, headers, query }`** — last-resort serializable overlays merged into the final HTTP request. Reach for this only when a stable typed path doesn't yet exist.
+
+Route/provider defaults are overridden by request-level values for each axis.
+
+## Routes
+
+Adding a new model or deployment is usually 5-15 lines using `Route.make({ protocol, endpoint, auth, framing, ... })`. The route owns endpoint/auth/framing and the protocol owns body construction plus stream parsing. Transports are reusable IO templates that receive route endpoint/auth at compile time. Capability/catalog metadata lives outside this low-level package; unsupported request shapes fail during protocol lowering. See `AGENTS.md` for the architectural detail.
+
+## Effect
+
+This package is built on Effect. Public methods return `Effect` or `Stream`; provide `LLMClient.layer` for runtime dispatch and import the provider/protocol modules for the routes you use. The example at `example/tutorial.ts` is a runnable walkthrough.
+
+## See also
+
+- `AGENTS.md` — architecture, route construction, contributor guide
+- `STATUS.md` — native provider parity status and AI SDK migration gaps
+- `example/tutorial.ts` — runnable end-to-end walkthrough
+- `test/provider/*.test.ts` — fixture-first protocol tests; `*.recorded.test.ts` files cover live cassettes

package/dist/cache-policy.d.ts

@@ -0,0 +1,2 @@
+import { LLMRequest } from "./schema/messages";
+export declare const applyCachePolicy: (request: LLMRequest) => LLMRequest;

package/dist/cache-policy.js

@@ -0,0 +1,108 @@
+// Apply an `LLMRequest.cache` policy by injecting `CacheHint`s onto the parts
+// the policy designates. Runs once at compile time, before the per-protocol
+// body builder, so the existing inline-hint lowering path handles the rest.
+//
+// The default `"auto"` shape places one breakpoint at the last tool definition,
+// one at the last system part, and one at the latest user message. This
+// matches what production agent harnesses (LangChain's caching middleware,
+// kern-ai's 10x cost-reduction playbook) converge on for tool-use loops: the
+// latest user message stays put while a single turn explodes into many
+// assistant/tool round-trips, so caching at that boundary lets every
+// intra-turn API call hit the prefix.
+//
+// Manual `cache: CacheHint` placements on individual parts are preserved —
+// this function only fills gaps the caller left empty.
+import { CacheHint } from "./schema/options";
+import { LLMRequest, Message, ToolDefinition } from "./schema/messages";
+const AUTO = {
+    tools: true,
+    system: true,
+    messages: "latest-user-message",
+};
+const NONE = {};
+// Resolution rules:
+//   - undefined   → "auto" — caching is on by default. The math favors it:
+//                   Anthropic 5m-cache write is 1.25x base, read is 0.1x,
+//                   so a single reuse within 5 minutes already wins.
+//   - "auto"      → tools + system + latest user msg.
+//   - "none"      → no auto placement; manual `CacheHint`s still flow.
+//   - object form → exactly what the caller asked for.
+const resolve = (policy) => {
+    if (policy === undefined || policy === "auto")
+        return AUTO;
+    if (policy === "none")
+        return NONE;
+    return policy;
+};
+// Protocols whose wire format ignores inline cache markers (OpenAI's implicit
+// prefix caching, Gemini's implicit + out-of-band CachedContent). Skip the
+// whole policy pass for these — emitting hints would be harmless but pointless.
+const RESPECTS_INLINE_HINTS = new Set(["anthropic-messages", "bedrock-converse"]);
+const makeHint = (ttlSeconds) => ttlSeconds !== undefined ? new CacheHint({ type: "ephemeral", ttlSeconds }) : new CacheHint({ type: "ephemeral" });
+const markLastTool = (tools, hint) => {
+    if (tools.length === 0)
+        return tools;
+    const last = tools.length - 1;
+    if (tools[last].cache)
+        return tools;
+    return tools.map((tool, i) => (i === last ? new ToolDefinition({ ...tool, cache: hint }) : tool));
+};
+const markLastSystem = (system, hint) => {
+    if (system.length === 0)
+        return system;
+    const last = system.length - 1;
+    if (system[last].cache)
+        return system;
+    return system.map((part, i) => (i === last ? { ...part, cache: hint } : part));
+};
+const lastIndexOfRole = (messages, role) => messages.findLastIndex((m) => m.role === role);
+// Mark the last text part of `messages[index]`. If no text part exists, mark
+// the last content part regardless of type — that's the breakpoint position
+// in tool-result-only messages too.
+const markMessageAt = (messages, index, hint) => {
+    if (index < 0 || index >= messages.length)
+        return messages;
+    const target = messages[index];
+    if (target.content.length === 0)
+        return messages;
+    const lastTextIndex = target.content.findLastIndex((part) => part.type === "text");
+    const markAt = lastTextIndex >= 0 ? lastTextIndex : target.content.length - 1;
+    const existing = target.content[markAt];
+    if ("cache" in existing && existing.cache)
+        return messages;
+    const nextContent = target.content.map((part, i) => (i === markAt ? { ...part, cache: hint } : part));
+    const next = new Message({ ...target, content: nextContent });
+    // Single pass over `messages`, substituting the one updated entry. Long
+    // conversations call this on every request, so avoid `.map()` here — its
+    // closure dispatch and identity copies show up in profiling.
+    const result = messages.slice();
+    result[index] = next;
+    return result;
+};
+const markMessages = (messages, strategy, hint) => {
+    if (messages.length === 0)
+        return messages;
+    if (strategy === "latest-user-message")
+        return markMessageAt(messages, lastIndexOfRole(messages, "user"), hint);
+    if (strategy === "latest-assistant")
+        return markMessageAt(messages, lastIndexOfRole(messages, "assistant"), hint);
+    const start = Math.max(0, messages.length - strategy.tail);
+    let next = messages;
+    for (let i = start; i < messages.length; i++)
+        next = markMessageAt(next, i, hint);
+    return next;
+};
+export const applyCachePolicy = (request) => {
+    if (!RESPECTS_INLINE_HINTS.has(request.model.route.id))
+        return request;
+    const policy = resolve(request.cache);
+    if (!policy.tools && !policy.system && !policy.messages)
+        return request;
+    const hint = makeHint(policy.ttlSeconds);
+    const tools = policy.tools ? markLastTool(request.tools, hint) : request.tools;
+    const system = policy.system ? markLastSystem(request.system, hint) : request.system;
+    const messages = policy.messages ? markMessages(request.messages, policy.messages, hint) : request.messages;
+    if (tools === request.tools && system === request.system && messages === request.messages)
+        return request;
+    return LLMRequest.update(request, { tools, system, messages });
+};

package/dist/index.d.ts

@@ -0,0 +1,14 @@
+export { LLMClient } from "./route/client";
+export { Auth } from "./route/auth";
+export { Provider } from "./provider";
+export { ProviderPackage } from "./provider-package";
+export { isContextOverflow, isContextOverflowFailure } from "./provider-error";
+export type { RouteModelInput, RouteRoutedModelInput, Interface as LLMClientShape, Service as LLMClientService, } from "./route/client";
+export * from "./schema";
+export { Tool, ToolFailure, toDefinitions } from "./tool";
+export { ToolRuntime } from "./tool-runtime";
+export type { DispatchResult as ToolDispatchResult, ToolSettlement } from "./tool-runtime";
+export type { AnyExecutableTool, AnyTool, ExecutableTool, ExecutableTools, Definition as ToolShape, ToolExecute, ToolExecuteContext, ToolModelOutputInput, Tools, ToolSchema, ToolToModelOutput, } from "./tool";
+export * as LLM from "./llm";
+export type { Definition as ProviderDefinition, ModelFactory as ProviderModelFactory, ModelOptions as ProviderModelOptions, } from "./provider";
+export type { Definition as ProviderPackageDefinition, Settings as ProviderPackageSettings } from "./provider-package";

package/dist/index.js

@@ -0,0 +1,9 @@
+export { LLMClient } from "./route/client";
+export { Auth } from "./route/auth";
+export { Provider } from "./provider";
+export { ProviderPackage } from "./provider-package";
+export { isContextOverflow, isContextOverflowFailure } from "./provider-error";
+export * from "./schema";
+export { Tool, ToolFailure, toDefinitions } from "./tool";
+export { ToolRuntime } from "./tool-runtime";
+export * as LLM from "./llm";

package/dist/llm.d.ts

@@ -0,0 +1,223 @@
+import { Effect, JsonSchema, Schema } from "effect";
+import { GenerationOptions, HttpOptions, LLMError, LLMRequest, LLMResponse, Message, type ModelInput as SchemaModelInput, SystemPart, ToolChoice, ToolDefinition, type ContentPart, ToolResultPart } from "./schema";
+import { type ToolSchema } from "./tool";
+export type ModelInput = SchemaModelInput;
+export type MessageInput = Message.Input;
+export type ToolChoiceInput = ToolChoice.Input;
+export type ToolChoiceMode = ToolChoice.Mode;
+export type ToolResultInput = Parameters<typeof ToolResultPart.make>[0];
+/** Input accepted by `LLM.request`, normalized into the canonical `LLMRequest` class. */
+export type RequestInput = Omit<ConstructorParameters<typeof LLMRequest>[0], "system" | "messages" | "tools" | "toolChoice" | "generation" | "http" | "providerOptions"> & {
+    readonly system?: string | SystemPart | ReadonlyArray<SystemPart>;
+    readonly prompt?: string | ContentPart | ReadonlyArray<ContentPart>;
+    readonly messages?: ReadonlyArray<Message | MessageInput>;
+    readonly tools?: ReadonlyArray<ToolDefinition.Input>;
+    readonly toolChoice?: ToolChoiceInput;
+    readonly generation?: GenerationOptions.Input;
+    readonly providerOptions?: ConstructorParameters<typeof LLMRequest>[0]["providerOptions"];
+    readonly http?: HttpOptions.Input;
+};
+export declare const generate: typeof import("./route/client").generate;
+export declare const stream: typeof import("./route/client").stream;
+export declare const requestInput: (input: LLMRequest) => RequestInput;
+export declare const request: (input: RequestInput) => LLMRequest;
+export declare const updateRequest: (input: LLMRequest, patch: Partial<RequestInput>) => LLMRequest;
+type GenerateObjectBase = Omit<RequestInput, "tools" | "toolChoice" | "responseFormat">;
+export declare class GenerateObjectResponse<T> {
+    readonly object: T;
+    readonly response: LLMResponse;
+    constructor(object: T, response: LLMResponse);
+    get events(): readonly ({
+        readonly type: "step-start";
+        readonly index: number;
+    } | {
+        readonly type: "text-start";
+        readonly id: string;
+        readonly providerMetadata?: {
+            readonly [x: string]: {
+                readonly [x: string]: unknown;
+            };
+        } | undefined;
+    } | {
+        readonly type: "text-delta";
+        readonly id: string;
+        readonly text: string;
+        readonly providerMetadata?: {
+            readonly [x: string]: {
+                readonly [x: string]: unknown;
+            };
+        } | undefined;
+    } | {
+        readonly type: "text-end";
+        readonly id: string;
+        readonly providerMetadata?: {
+            readonly [x: string]: {
+                readonly [x: string]: unknown;
+            };
+        } | undefined;
+    } | {
+        readonly type: "reasoning-start";
+        readonly id: string;
+        readonly providerMetadata?: {
+            readonly [x: string]: {
+                readonly [x: string]: unknown;
+            };
+        } | undefined;
+    } | {
+        readonly type: "reasoning-delta";
+        readonly id: string;
+        readonly text: string;
+        readonly providerMetadata?: {
+            readonly [x: string]: {
+                readonly [x: string]: unknown;
+            };
+        } | undefined;
+    } | {
+        readonly type: "reasoning-end";
+        readonly id: string;
+        readonly providerMetadata?: {
+            readonly [x: string]: {
+                readonly [x: string]: unknown;
+            };
+        } | undefined;
+    } | {
+        readonly type: "tool-input-start";
+        readonly id: string;
+        readonly name: string;
+        readonly providerMetadata?: {
+            readonly [x: string]: {
+                readonly [x: string]: unknown;
+            };
+        } | undefined;
+    } | {
+        readonly type: "tool-input-delta";
+        readonly id: string;
+        readonly name: string;
+        readonly text: string;
+    } | {
+        readonly type: "tool-input-end";
+        readonly id: string;
+        readonly name: string;
+        readonly providerMetadata?: {
+            readonly [x: string]: {
+                readonly [x: string]: unknown;
+            };
+        } | undefined;
+    } | {
+        readonly type: "tool-call";
+        readonly id: string;
+        readonly name: string;
+        readonly input: unknown;
+        readonly providerExecuted?: boolean | undefined;
+        readonly providerMetadata?: {
+            readonly [x: string]: {
+                readonly [x: string]: unknown;
+            };
+        } | undefined;
+    } | {
+        readonly type: "tool-result";
+        readonly id: string;
+        readonly name: string;
+        readonly result: {
+            readonly type: "json";
+            readonly value: unknown;
+        } | {
+            readonly type: "text";
+            readonly value: unknown;
+        } | {
+            readonly type: "error";
+            readonly value: unknown;
+        } | {
+            readonly type: "content";
+            readonly value: readonly ({
+                readonly type: "text";
+                readonly text: string;
+            } | {
+                readonly type: "file";
+                readonly uri: string;
+                readonly mime: string;
+                readonly name?: string | undefined;
+            })[];
+        };
+        readonly output?: {
+            readonly structured: unknown;
+            readonly content: readonly ({
+                readonly type: "text";
+                readonly text: string;
+            } | {
+                readonly type: "file";
+                readonly uri: string;
+                readonly mime: string;
+                readonly name?: string | undefined;
+            })[];
+        } | undefined;
+        readonly providerExecuted?: boolean | undefined;
+        readonly providerMetadata?: {
+            readonly [x: string]: {
+                readonly [x: string]: unknown;
+            };
+        } | undefined;
+    } | {
+        readonly type: "tool-error";
+        readonly id: string;
+        readonly name: string;
+        readonly message: string;
+        readonly error?: unknown;
+        readonly providerMetadata?: {
+            readonly [x: string]: {
+                readonly [x: string]: unknown;
+            };
+        } | undefined;
+    } | {
+        readonly type: "step-finish";
+        readonly index: number;
+        readonly reason: "length" | "stop" | "tool-calls" | "content-filter" | "error" | "unknown";
+        readonly usage?: import("./schema").Usage | undefined;
+        readonly providerMetadata?: {
+            readonly [x: string]: {
+                readonly [x: string]: unknown;
+            };
+        } | undefined;
+    } | {
+        readonly type: "finish";
+        readonly reason: "length" | "stop" | "tool-calls" | "content-filter" | "error" | "unknown";
+        readonly usage?: import("./schema").Usage | undefined;
+        readonly providerMetadata?: {
+            readonly [x: string]: {
+                readonly [x: string]: unknown;
+            };
+        } | undefined;
+    } | {
+        readonly type: "provider-error";
+        readonly message: string;
+        readonly classification?: "context-overflow" | undefined;
+        readonly providerMetadata?: {
+            readonly [x: string]: {
+                readonly [x: string]: unknown;
+            };
+        } | undefined;
+    })[];
+    get usage(): import("./schema").Usage | undefined;
+}
+export interface GenerateObjectOptions<S extends ToolSchema<any>> extends GenerateObjectBase {
+    readonly schema: S;
+}
+export interface GenerateObjectDynamicOptions extends GenerateObjectBase {
+    /** Raw JSON Schema object describing the expected output shape. */
+    readonly jsonSchema: JsonSchema.JsonSchema;
+}
+/**
+ * Run a model and decode its output against `schema`. Works on every protocol
+ * because it forces a synthetic tool call internally — provider-native JSON
+ * modes are intentionally avoided so behaviour is uniform.
+ *
+ * Two input modes:
+ *
+ * 1. `schema: EffectSchema<T>` — `.object` is decoded and typed as `T`.
+ *    Decode failures surface as `LLMError`.
+ * 2. `jsonSchema: JsonSchema.JsonSchema` — `.object` is `unknown`. Use when
+ *    the schema is only available at runtime (MCP, plugin manifests). Caller validates.
+ */
+export declare function generateObject<S extends ToolSchema<any>>(options: GenerateObjectOptions<S>): Effect.Effect<GenerateObjectResponse<Schema.Schema.Type<S>>, LLMError>;
+export declare function generateObject(options: GenerateObjectDynamicOptions): Effect.Effect<GenerateObjectResponse<unknown>, LLMError>;
+export {};

package/dist/llm.js

@@ -0,0 +1,81 @@
+import { Effect, JsonSchema, Schema } from "effect";
+import { LLMClient } from "./route/client";
+import { GenerationOptions, HttpOptions, InvalidProviderOutputReason, LLMError, LLMEvent, LLMRequest, LLMResponse, Message, SystemPart, ToolChoice, ToolDefinition, ToolResultPart, } from "./schema";
+import { make as makeTool, toDefinitions } from "./tool";
+export const generate = LLMClient.generate;
+export const stream = LLMClient.stream;
+export const requestInput = (input) => ({
+    ...LLMRequest.input(input),
+});
+export const request = (input) => {
+    const { system: requestSystem, prompt, messages, tools, toolChoice: requestToolChoice, generation: requestGeneration, providerOptions: requestProviderOptions, http: requestHttp, ...rest } = input;
+    return new LLMRequest({
+        ...rest,
+        system: SystemPart.content(requestSystem),
+        messages: [...(messages?.map(Message.make) ?? []), ...(prompt === undefined ? [] : [Message.user(prompt)])],
+        tools: tools?.map(ToolDefinition.make) ?? [],
+        toolChoice: requestToolChoice ? ToolChoice.make(requestToolChoice) : undefined,
+        generation: requestGeneration === undefined ? undefined : GenerationOptions.make(requestGeneration),
+        providerOptions: requestProviderOptions,
+        http: requestHttp === undefined ? undefined : HttpOptions.make(requestHttp),
+    });
+};
+export const updateRequest = (input, patch) => request({ ...requestInput(input), ...patch });
+const GENERATE_OBJECT_TOOL_NAME = "generate_object";
+const GENERATE_OBJECT_TOOL_DESCRIPTION = "Return the structured result by calling this tool.";
+export class GenerateObjectResponse {
+    object;
+    response;
+    constructor(object, response) {
+        this.object = object;
+        this.response = response;
+    }
+    get events() {
+        return this.response.events;
+    }
+    get usage() {
+        return this.response.usage;
+    }
+}
+const runGenerateObject = Effect.fn("LLM.generateObject")(function* (options, tool) {
+    const baseRequest = request(options);
+    const generateRequest = LLMRequest.update(baseRequest, {
+        tools: toDefinitions({ [GENERATE_OBJECT_TOOL_NAME]: tool }),
+        toolChoice: ToolChoice.named(GENERATE_OBJECT_TOOL_NAME),
+    });
+    const response = yield* LLMClient.generate(generateRequest);
+    const call = response.toolCalls.find((event) => LLMEvent.is.toolCall(event) && event.name === GENERATE_OBJECT_TOOL_NAME);
+    if (!call || !LLMEvent.is.toolCall(call))
+        return yield* new LLMError({
+            module: "LLM",
+            method: "generateObject",
+            reason: new InvalidProviderOutputReason({
+                message: `generateObject: model did not call the forced \`${GENERATE_OBJECT_TOOL_NAME}\` tool`,
+            }),
+        });
+    const object = yield* tool._decode(call.input).pipe(Effect.mapError((error) => new LLMError({
+        module: "LLM",
+        method: "generateObject",
+        reason: new InvalidProviderOutputReason({
+            message: `generateObject: tool input failed schema decode: ${error.message}`,
+        }),
+    })));
+    return new GenerateObjectResponse(object, response);
+});
+export function generateObject(options) {
+    if ("schema" in options) {
+        const { schema, ...rest } = options;
+        return runGenerateObject(rest, makeTool({
+            description: GENERATE_OBJECT_TOOL_DESCRIPTION,
+            parameters: schema,
+            success: Schema.Unknown,
+            execute: () => Effect.void,
+        }));
+    }
+    const { jsonSchema, ...rest } = options;
+    return runGenerateObject(rest, makeTool({
+        description: GENERATE_OBJECT_TOOL_DESCRIPTION,
+        jsonSchema,
+        execute: () => Effect.void,
+    }));
+}