@copilotkit/runtime 1.57.3 → 1.58.0-canary.thread-id-propagation

package/skills/runtime/references/agent-runners.md

@@ -0,0 +1,304 @@
+# CopilotKit Agent Runners
+
+`AgentRunner` is the abstraction that owns thread run state — active runs, the event stream
+replay, and stop semantics. Pick one per `CopilotRuntime` instance.
+
+- `InMemoryAgentRunner` — default; globalThis-keyed Map; lost on restart.
+- `SqliteAgentRunner` — file-backed; requires `better-sqlite3` peer.
+- `IntelligenceAgentRunner` — auto-wired by `CopilotIntelligenceRuntime`. You do NOT
+  construct this directly and you cannot pass `runner` alongside `intelligence`.
+- Custom — subclass `AgentRunner` for Redis / Postgres / any backend.
+
+## Setup
+
+Default (in-memory, dev only):
+
+```typescript
+import { CopilotRuntime } from "@copilotkit/runtime/v2";
+
+// Equivalent to passing `runner: new InMemoryAgentRunner()`
+const runtime = new CopilotRuntime({
+  agents: {
+    /* ... */
+  } as any,
+});
+```
+
+Production (file-backed SQLite):
+
+```typescript
+import { CopilotRuntime } from "@copilotkit/runtime/v2";
+import { SqliteAgentRunner } from "@copilotkit/sqlite-runner";
+
+const runtime = new CopilotRuntime({
+  agents: {
+    /* ... */
+  } as any,
+  runner: new SqliteAgentRunner({ dbPath: "./data/threads.db" }),
+});
+```
+
+Installation for the SQLite runner (the `better-sqlite3` peer is required):
+
+```bash
+pnpm add @copilotkit/sqlite-runner better-sqlite3
+```
+
+## Core Patterns
+
+### The AgentRunner contract
+
+```typescript
+import { AgentRunner } from "@copilotkit/runtime/v2";
+import type {
+  AgentRunnerRunRequest,
+  AgentRunnerConnectRequest,
+  AgentRunnerIsRunningRequest,
+  AgentRunnerStopRequest,
+} from "@copilotkit/runtime/v2";
+import { Observable } from "rxjs";
+import type { BaseEvent } from "@ag-ui/client";
+
+class MyRunner extends AgentRunner {
+  run(request: AgentRunnerRunRequest): Observable<BaseEvent> {
+    // Start a new run for request.threadId. Throw `new Error("Thread already running")`
+    // if a run is in flight. Stream events from agent.run(request.input).
+    return new Observable<BaseEvent>();
+  }
+  connect(request: AgentRunnerConnectRequest): Observable<BaseEvent> {
+    // Replay events for an active run, or historic runs for request.threadId.
+    return new Observable<BaseEvent>();
+  }
+  async isRunning(request: AgentRunnerIsRunningRequest): Promise<boolean> {
+    return false;
+  }
+  async stop(request: AgentRunnerStopRequest): Promise<boolean | undefined> {
+    return true;
+  }
+}
+```
+
+### Handle double-submit on the client
+
+Both `InMemoryAgentRunner` and `SqliteAgentRunner` throw
+`"Thread already running"` on concurrent `run()` calls for the same `threadId`. How
+that surfaces to the client depends on the runtime mode:
+
+- **Intelligence mode** — the Intelligence platform returns HTTP `409` when a lock is
+  held. The client core maps this to `CopilotKitCoreErrorCode.AGENT_THREAD_LOCKED`
+  and fires `onError({ code: "agent_thread_locked", ... })`. Handle this in
+  `<CopilotKitProvider onError>`.
+- **SSE mode** (default, in-memory / SQLite runners) — the runner throws
+  synchronously and the handler returns a plain `500` JSON body like
+  `{ "error": "Failed to run agent", "message": "Thread already running" }`.
+  There is no typed `agent_thread_locked` code — match on the message text or
+  just guard on the client with a busy flag.
+
+```tsx
+// client — Intelligence mode (typed code)
+import { CopilotKitProvider } from "@copilotkit/react-core/v2";
+
+<CopilotKitProvider
+  onError={({ code }) => {
+    if (code === "agent_thread_locked") {
+      alert("Agent is busy — wait for the current response to finish.");
+    }
+  }}
+/>;
+```
+
+```tsx
+// client — any mode: guard with a busy flag so double-submit is impossible
+import { useAgent } from "@copilotkit/react-core/v2";
+import { useState } from "react";
+
+function Composer() {
+  const agent = useAgent({ agentId: "default" });
+  const [busy, setBusy] = useState(false);
+
+  async function send(text: string) {
+    if (busy) return;
+    setBusy(true);
+    try {
+      await agent?.addMessage({ role: "user", content: text });
+    } finally {
+      setBusy(false);
+    }
+  }
+
+  return null;
+}
+```
+
+## Common Mistakes
+
+### HIGH Shipping InMemoryAgentRunner to production
+
+Wrong:
+
+```typescript
+// production:
+new CopilotRuntime({ agents: { default: agent } });
+```
+
+Correct:
+
+```typescript
+import { SqliteAgentRunner } from "@copilotkit/sqlite-runner";
+
+new CopilotRuntime({
+  agents: { default: agent },
+  runner: new SqliteAgentRunner({ dbPath: "./data/threads.db" }),
+});
+// Or upgrade to Intelligence mode for managed durability.
+```
+
+The default runner is `new InMemoryAgentRunner()`. It keeps state in a `globalThis`-keyed
+Map — threads are lost on restart, and horizontally-scaled instances see divergent state.
+
+Source: `packages/runtime/src/v2/runtime/runner/in-memory.ts:63-96`.
+
+### HIGH Setting runner alongside intelligence option
+
+Wrong:
+
+```typescript
+new CopilotRuntime({
+  agents,
+  intelligence,
+  runner: new SqliteAgentRunner({ dbPath: "./data/threads.db" }),
+});
+```
+
+Correct:
+
+```typescript
+new CopilotRuntime({
+  agents,
+  intelligence,
+  identifyUser: (req) => ({ id: req.headers.get("x-user-id")! }),
+});
+```
+
+`CopilotIntelligenceRuntimeOptions` does not declare a `runner` field — Intelligence mode
+auto-wires `IntelligenceAgentRunner` pointed at the Cloud socket. Excess-property checks will
+flag a `runner:` key on an Intelligence-shaped options object as a type error, and at runtime
+the auto-wired Intelligence runner wins regardless of what you pass.
+
+Source: `packages/runtime/src/v2/runtime/core/runtime.ts:149-173,285-294`.
+
+### HIGH Forgetting the better-sqlite3 peer
+
+Wrong:
+
+```bash
+pnpm add @copilotkit/sqlite-runner
+```
+
+Correct:
+
+```bash
+pnpm add @copilotkit/sqlite-runner better-sqlite3
+```
+
+`@copilotkit/sqlite-runner` imports `better-sqlite3` at the top of its module, so if the peer
+is missing, `import { SqliteAgentRunner } from "@copilotkit/sqlite-runner"` itself fails at
+module load with `Cannot find module 'better-sqlite3'` — long before the constructor runs.
+(The constructor has a friendlier multi-line install hint as a belt-and-suspenders fallback,
+but in practice you will see the bare module-resolution error first.) It is a peer dependency,
+not a direct dep.
+
+Source: `packages/sqlite-runner/src/sqlite-runner.ts:18`, `:55-66`.
+
+### HIGH Default SqliteAgentRunner with :memory: dbPath
+
+Wrong:
+
+```typescript
+new SqliteAgentRunner();
+```
+
+Correct:
+
+```typescript
+new SqliteAgentRunner({ dbPath: "./data/threads.db" });
+```
+
+The default `dbPath` is `":memory:"` — SQLite's in-memory mode. Data is lost at restart,
+defeating the reason to use the file-backed runner.
+
+Source: `packages/sqlite-runner/src/sqlite-runner.ts:48-54`.
+
+### MEDIUM Concurrent run() on the same threadId
+
+Wrong:
+
+```tsx
+// Double-click send button → two POST /agent/:id/run to the same thread
+<button onClick={() => agent.addMessage({ role: "user", content })}>
+  Send
+</button>
+```
+
+Correct:
+
+```tsx
+const [busy, setBusy] = useState(false);
+<button
+  disabled={busy}
+  onClick={async () => {
+    setBusy(true);
+    try {
+      await agent.addMessage({ role: "user", content });
+    } finally {
+      setBusy(false);
+    }
+  }}
+>
+  Send
+</button>;
+```
+
+Both runners throw `"Thread already running"` on concurrent runs. Debounce on the client.
+In Intelligence mode you can additionally handle `code === "agent_thread_locked"` in
+`<CopilotKitProvider onError>`; SSE mode surfaces only a generic 500 with that message.
+
+Source: `packages/runtime/src/v2/runtime/runner/in-memory.ts:110`;
+`packages/core/src/intelligence-agent.ts:368-369`.
+
+### HIGH In-memory runner + horizontal scaling
+
+Wrong:
+
+```typescript
+// 3 Fly.io / Cloud Run instances, each with its own InMemoryAgentRunner
+new CopilotRuntime({ agents });
+```
+
+Correct:
+
+```typescript
+// Either sticky-session a single instance per thread, or use shared state:
+new CopilotRuntime({
+  agents,
+  runner: new SqliteAgentRunner({ dbPath: process.env.THREADS_DB! }),
+});
+// Best: Intelligence mode for managed multi-instance durability.
+```
+
+`InMemoryAgentRunner`'s `globalThis` store is per-process — multi-instance deploys see
+totally different thread state per worker, making reconnects and `GET /connect` non-deterministic.
+
+Source: `packages/runtime/src/v2/runtime/runner/in-memory.ts:63-96`.
+
+## References
+
+- [InMemoryAgentRunner — internals and hot-reload note](agent-runners-in-memory.md)
+- [SqliteAgentRunner — schema, retention, ops](agent-runners-sqlite.md)
+- [Custom runner — Redis/Postgres skeleton](agent-runners-custom.md)
+
+## See also
+
+- `copilotkit/intelligence-mode` — managed durability alternative (Cloud-only)
+- `copilotkit/setup-endpoint` — runner is passed via the CopilotRuntime constructor
+- `copilotkit/scale-to-multi-agent` — horizontal scaling considerations

package/skills/runtime/references/built-in-agent-factory-modes.md

@@ -0,0 +1,232 @@
+BuiltInAgent Factory Modes — cookbook for TanStack AI, AI SDK, and custom AG-UI factories.
+
+## The AgentFactoryContext
+
+```typescript
+// packages/runtime/src/agent/index.ts
+export interface AgentFactoryContext {
+  input: RunAgentInput; // messages, tools, forwardedProps, context
+  abortController: AbortController; // prefer abortSignal
+  abortSignal: AbortSignal; // pass to AI SDK / fetch / custom
+}
+```
+
+Rule of thumb:
+
+- Prefer `abortSignal` for AI SDK, fetch, custom backends.
+- Use `abortController` for TanStack AI (its `chat()` takes the controller, not the signal).
+- NEVER call `ctx.abortController.abort()` inside the factory — use
+  `agent.abortRun()` from outside.
+
+## TanStack AI factory (preferred)
+
+```typescript
+import { BuiltInAgent, convertInputToTanStackAI } from "@copilotkit/runtime/v2";
+import { chat } from "@tanstack/ai";
+import { openaiText } from "@tanstack/ai-openai";
+
+new BuiltInAgent({
+  type: "tanstack",
+  factory: ({ input, abortController }) => {
+    const { messages, systemPrompts } = convertInputToTanStackAI(input);
+    systemPrompts.unshift("You are a helpful assistant.");
+    return chat({
+      adapter: openaiText("gpt-4o"),
+      messages,
+      systemPrompts,
+      tools: [
+        /* TanStack AI toolDefinition()s */
+      ],
+      abortController,
+    });
+  },
+});
+```
+
+### TanStack AI + forwardedProps
+
+```typescript
+new BuiltInAgent({
+  type: "tanstack",
+  factory: ({ input, abortController }) => {
+    const { messages, systemPrompts } = convertInputToTanStackAI(input);
+    const fwd = input.forwardedProps as
+      | { model?: string; temperature?: number }
+      | undefined;
+    return chat({
+      adapter: openaiText(fwd?.model ?? "gpt-4o"),
+      messages,
+      systemPrompts,
+      modelOptions: { temperature: fwd?.temperature ?? 0.2 },
+      abortController,
+    });
+  },
+});
+```
+
+## AI SDK factory (use when reasoning events are required)
+
+```typescript
+import {
+  BuiltInAgent,
+  convertMessagesToVercelAISDKMessages,
+  convertToolsToVercelAITools,
+} from "@copilotkit/runtime/v2";
+import { streamText, stepCountIs } from "ai";
+import { openai } from "@ai-sdk/openai";
+
+new BuiltInAgent({
+  type: "aisdk",
+  factory: ({ input, abortSignal }) => {
+    const messages = convertMessagesToVercelAISDKMessages(input.messages, {
+      forwardSystemMessages: true,
+    });
+    const tools = convertToolsToVercelAITools(input.tools);
+    return streamText({
+      model: openai("gpt-4o"),
+      messages,
+      tools,
+      abortSignal,
+      stopWhen: stepCountIs(5),
+    });
+  },
+});
+```
+
+The `BuiltInAgentAISDKFactoryConfig` contract requires an object with a `fullStream`
+async iterable — this is exactly what `streamText()` returns.
+
+## AI SDK + reasoning (Anthropic thinking)
+
+```typescript
+import { anthropic } from "@ai-sdk/anthropic";
+import { streamText } from "ai";
+import {
+  BuiltInAgent,
+  convertMessagesToVercelAISDKMessages,
+} from "@copilotkit/runtime/v2";
+
+new BuiltInAgent({
+  type: "aisdk",
+  factory: ({ input, abortSignal }) =>
+    streamText({
+      model: anthropic("claude-sonnet-4"),
+      messages: convertMessagesToVercelAISDKMessages(input.messages),
+      providerOptions: {
+        anthropic: { thinking: { type: "enabled", budgetTokens: 10000 } },
+      },
+      abortSignal,
+    }),
+});
+```
+
+TanStack AI silently drops reasoning events — only AI SDK surfaces them.
+
+## Custom factory (raw AG-UI events)
+
+```typescript
+import { BuiltInAgent } from "@copilotkit/runtime/v2";
+import type { BaseEvent } from "@ag-ui/client";
+import { EventType } from "@ag-ui/client";
+
+new BuiltInAgent({
+  type: "custom",
+  factory: async function* ({ input, abortSignal }): AsyncIterable<BaseEvent> {
+    // Check abortSignal.aborted on every iteration — agent.abortRun() signals
+    // cancellation via this flag, but the generator must consult it to stop yielding.
+    if (abortSignal.aborted) return;
+
+    const messageId = crypto.randomUUID();
+    yield {
+      type: EventType.TEXT_MESSAGE_START,
+      messageId,
+      role: "assistant",
+    } as any;
+
+    for (const delta of ["Hello", ", ", "world."]) {
+      if (abortSignal.aborted) return; // honor cancellation between yields
+      yield {
+        type: EventType.TEXT_MESSAGE_CONTENT,
+        messageId,
+        delta,
+      } as any;
+    }
+
+    yield { type: EventType.TEXT_MESSAGE_END, messageId } as any;
+  },
+});
+```
+
+A custom factory that never checks `abortSignal.aborted` (or registers an
+`addEventListener("abort", …)` handler to break its loop) is non-cancellable —
+`agent.abortRun()` will flip the flag but the generator will keep yielding until it
+exhausts its own source. Pass `abortSignal` through to any underlying `fetch` /
+streaming API as well so the upstream request is torn down.
+
+## Manual state-tool wiring (Factory Mode only)
+
+Simple Mode auto-injects `AGUISendStateSnapshot` / `AGUISendStateDelta`. In Factory Mode
+you must register them by hand for shared-state updates to reach the LLM. The AI SDK
+factory works out of the box because `defineTool` output adapts through
+`convertToolDefinitionsToVercelAITools`:
+
+```typescript
+import {
+  BuiltInAgent,
+  convertMessagesToVercelAISDKMessages,
+  convertToolDefinitionsToVercelAITools,
+  defineTool,
+} from "@copilotkit/runtime/v2";
+import { streamText } from "ai";
+import { openai } from "@ai-sdk/openai";
+import { z } from "zod";
+
+const sendStateSnapshot = defineTool({
+  name: "AGUISendStateSnapshot",
+  description: "Replace the entire application state with a new snapshot",
+  parameters: z.object({
+    snapshot: z.any().describe("The complete new state object"),
+  }),
+  execute: async ({ snapshot }) => ({ success: true, snapshot }),
+});
+const sendStateDelta = defineTool({
+  name: "AGUISendStateDelta",
+  description:
+    "Apply incremental updates to application state using JSON Patch operations",
+  // MUST mirror the Simple-Mode auto-injected schema (src/agent/index.ts:1140-1176)
+  // or the frontend's state handler won't recognize the payload.
+  parameters: z.object({
+    delta: z
+      .array(
+        z.object({
+          op: z.enum(["add", "replace", "remove"]),
+          path: z.string(),
+          value: z.any().optional(),
+        }),
+      )
+      .describe("Array of JSON Patch operations"),
+  }),
+  execute: async ({ delta }) => ({ success: true, delta }),
+});
+
+new BuiltInAgent({
+  type: "aisdk",
+  factory: ({ input, abortSignal }) =>
+    streamText({
+      model: openai("gpt-4o"),
+      messages: convertMessagesToVercelAISDKMessages(input.messages),
+      tools: convertToolDefinitionsToVercelAITools([
+        sendStateSnapshot,
+        sendStateDelta,
+      ]),
+      abortSignal,
+    }),
+});
+```
+
+For TanStack AI factories, `defineTool` output is NOT a TanStack tool — passing it to
+`chat({ tools })` does not work. Either switch to the AI SDK factory above, or redefine
+the tools with `toolDefinition()` from `@tanstack/ai`.
+
+Source: `packages/runtime/src/agent/index.ts`,
+`docs/content/docs/integrations/built-in-agent/custom-agent.mdx`.

package/skills/runtime/references/built-in-agent-helper-utilities.md

@@ -0,0 +1,123 @@
+BuiltInAgent helper utilities — exported from `@copilotkit/runtime/v2`.
+
+## convertInputToTanStackAI
+
+```typescript
+import { convertInputToTanStackAI } from "@copilotkit/runtime/v2";
+
+// signature (simplified):
+// convertInputToTanStackAI(input: RunAgentInput): {
+//   messages: TanStackAIMessage[];
+//   systemPrompts: string[];
+// }
+```
+
+Converts the AG-UI `RunAgentInput` into TanStack AI's `chat()` inputs. System messages in
+the input are collected into the `systemPrompts` array (not the `messages` array). Unshift
+your own system prompt onto `systemPrompts` before calling `chat()`:
+
+```typescript
+const { messages, systemPrompts } = convertInputToTanStackAI(input);
+systemPrompts.unshift("You are a helpful assistant.");
+return chat({ adapter, messages, systemPrompts, abortController });
+```
+
+Source: `packages/runtime/src/agent/converters/tanstack.ts:156`.
+
+## convertMessagesToVercelAISDKMessages
+
+```typescript
+import { convertMessagesToVercelAISDKMessages } from "@copilotkit/runtime/v2";
+
+// signature:
+// convertMessagesToVercelAISDKMessages(
+//   messages: Message[],
+//   options?: { forwardSystemMessages?: boolean; forwardDeveloperMessages?: boolean }
+// ): ModelMessage[]
+```
+
+Converts AG-UI `Message[]` to the Vercel AI SDK's `ModelMessage[]`. Handles multimodal
+content (text, image, audio/video/document, and legacy `binary`). By default drops
+`role: "system"` and `role: "developer"` messages — set the options to opt in.
+
+```typescript
+const messages = convertMessagesToVercelAISDKMessages(input.messages, {
+  forwardSystemMessages: true,
+});
+```
+
+Source: `packages/runtime/src/agent/index.ts:435`.
+
+## convertToolsToVercelAITools
+
+```typescript
+import { convertToolsToVercelAITools } from "@copilotkit/runtime/v2";
+
+// signature:
+// convertToolsToVercelAITools(tools: RunAgentInput["tools"]): ToolSet
+```
+
+Converts AG-UI `input.tools` (tools registered on the frontend — their parameters are plain
+JSON Schema) into the AI SDK's `ToolSet`. Throws `Invalid JSON schema for tool ${name}`
+when a tool's parameters aren't a JSON schema object. The resulting tools have no
+`execute` — the AI SDK emits tool-call events and the frontend handles them.
+
+```typescript
+const tools = convertToolsToVercelAITools(input.tools);
+return streamText({ model, messages, tools, abortSignal });
+```
+
+Source: `packages/runtime/src/agent/index.ts:599`.
+
+## convertToolDefinitionsToVercelAITools
+
+```typescript
+import { convertToolDefinitionsToVercelAITools } from "@copilotkit/runtime/v2";
+
+// signature:
+// convertToolDefinitionsToVercelAITools(tools: ToolDefinition[]): ToolSet
+```
+
+Converts server-side `ToolDefinition[]` (Standard Schema V1 parameters + `execute`
+function) into an AI SDK `ToolSet`. Zod schemas pass through directly; non-Zod Standard
+Schema V1 parameters (Valibot, ArkType, ...) are converted to JSON Schema via
+`schemaToJsonSchema` and wrapped with `jsonSchema()` from `ai`.
+
+```typescript
+import { defineTool } from "@copilotkit/runtime/v2";
+import { z } from "zod";
+
+const searchTool = defineTool({
+  name: "search",
+  description: "Search the web.",
+  parameters: z.object({ query: z.string() }),
+  execute: async ({ query }) => ({ results: [] }),
+});
+
+const tools = convertToolDefinitionsToVercelAITools([searchTool]);
+return streamText({ model, messages, tools, abortSignal });
+```
+
+Source: `packages/runtime/src/agent/index.ts:633`.
+
+## resolveModel
+
+```typescript
+import { resolveModel } from "@copilotkit/runtime/v2";
+
+// signature:
+// resolveModel(spec: ModelSpecifier, apiKey?: string): LanguageModel
+```
+
+Resolves a `"provider/model"` (or `"provider:model"`) string to a `LanguageModel`. If
+`spec` is already a `LanguageModel`, it's returned as-is. Throws
+`Invalid model string "..."` when the provider separator is missing.
+
+Supported providers: `openai`, `anthropic`, `google`/`gemini`/`google-gemini`, `vertex`.
+Unknown providers throw `Unknown provider "..." in "...". Supported: openai, anthropic, google (gemini).`
+
+```typescript
+const model = resolveModel("openai/gpt-4o", process.env.OPENAI_API_KEY);
+```
+
+Source: `packages/runtime/src/agent/index.ts:176-249`.