@copilotkit/runtime 1.70.3 → 1.71.1

package/skills/runtime/SKILL.md

@@ -1,98 +0,0 @@
----
-name: runtime
-description: >
-  @copilotkit/runtime — mount a fetch-native CopilotRuntime on any JS server, wire
-  middleware, pick an AgentRunner, instantiate BuiltInAgent (Factory Mode with TanStack AI
-  is the preferred default) or plug in any of 12 external agent frameworks (Mastra,
-  LangGraph, CrewAI Crews/Flows, PydanticAI, ADK, LlamaIndex, Agno, AWS Strands, MS Agent
-  Framework, AG2, A2A), enable Intelligence mode for durable threads + websocket,
-  register server-side tools via defineTool, and wire voice transcription. Uses the
-  fetch-based createCopilotRuntimeHandler primitive — the Express/Hono adapters are
-  discouraged. Load the reference under references/ that matches your task.
-type: core
-library: copilotkit
-library_version: "1.70.3"
-requires: []
-sources:
-  - "CopilotKit/CopilotKit:packages/runtime/src/v2/runtime/core/fetch-handler.ts"
-  - "CopilotKit/CopilotKit:packages/runtime/src/v2/runtime/core/runtime.ts"
-  - "CopilotKit/CopilotKit:packages/runtime/src/v2/runtime/core/hooks.ts"
-  - "CopilotKit/CopilotKit:packages/runtime/src/v2/runtime/core/middleware.ts"
-  - "CopilotKit/CopilotKit:packages/runtime/src/v2/runtime/runner/agent-runner.ts"
-  - "CopilotKit/CopilotKit:packages/runtime/src/v2/runtime/runner/in-memory.ts"
-  - "CopilotKit/CopilotKit:packages/runtime/src/v2/runtime/runner/intelligence.ts"
-  - "CopilotKit/CopilotKit:packages/runtime/src/v2/runtime/intelligence-platform/client.ts"
-  - "CopilotKit/CopilotKit:packages/runtime/src/v2/runtime/transcription-service/transcription-service.ts"
-  - "CopilotKit/CopilotKit:packages/runtime/src/v2/runtime/handlers/handle-transcribe.ts"
-  - "CopilotKit/CopilotKit:packages/runtime/src/agent/index.ts"
-  - "CopilotKit/CopilotKit:packages/runtime/src/agent/converters/tanstack.ts"
-  - "CopilotKit/CopilotKit:packages/sqlite-runner/src/sqlite-runner.ts"
-  - "CopilotKit/CopilotKit:packages/shared/src/transcription-errors.ts"
----
-
-# CopilotKit Runtime
-
-`@copilotkit/runtime` is the server half of CopilotKit: it accepts AG-UI protocol
-requests, dispatches them to an `AbstractAgent` (built-in or external), runs the
-stream through an `AgentRunner`, and responds as Server-Sent Events.
-
-This SKILL.md is the **index**. Read the reference under `references/` that matches
-your task — do not try to absorb the whole package from this file.
-
-## Mental Model — the three dictionaries you hand to `CopilotRuntime`
-
-```ts
-new CopilotRuntime({
-  agents, // Record<string, AbstractAgent>     — see wiring-external-agents or built-in-agent
-  runner, // AgentRunner (optional)            — see agent-runners
-  intelligence, // CopilotKitIntelligence (optional) — see intelligence-mode (auto-wires runner)
-  mcpApps, // McpAppsConfig (optional)          — see wiring-mcp-apps-middleware
-  a2ui, // A2UIConfig (optional)             — see packages/a2ui-renderer skill
-  hooks, // { onRequest, onBeforeHandler }    — see middleware
-  beforeRequestMiddleware,
-  afterRequestMiddleware, // legacy — see middleware
-  transcription, // TranscriptionService (optional)  — see transcription
-});
-```
-
-You then mount it:
-
-```ts
-import { createCopilotRuntimeHandler } from "@copilotkit/runtime/v2";
-const handler = createCopilotRuntimeHandler({
-  runtime,
-  basePath: "/api/copilotkit",
-});
-export default { fetch: handler };
-```
-
-## When to load which reference
-
-| Task                                                                                                                                                                           | Reference                                                                                                                                                                                                                                                                                                                                          |
-| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Mounting on any fetch-native server (Cloudflare Workers, Bun, Deno, Vercel Edge, Next.js App Router, React Router v7, TanStack Start) or delegating from Express/Node          | `references/setup-endpoint.md`                                                                                                                                                                                                                                                                                                                     |
-| Auth / logging / rate-limit / request-scoped guards via `hooks.onRequest` / `hooks.onBeforeHandler` (preferred) or legacy `beforeRequestMiddleware` / `afterRequestMiddleware` | `references/middleware.md`                                                                                                                                                                                                                                                                                                                         |
-| Choosing between `InMemoryAgentRunner`, `SqliteAgentRunner`, or a custom subclass — including thread-locking semantics and the runner/Intelligence mutual exclusion            | `references/agent-runners.md` (+ `-in-memory.md`, `-sqlite.md`, `-custom.md` for backend-specific detail)                                                                                                                                                                                                                                          |
-| Enabling durable threads + realtime websocket via CopilotKit Intelligence (a **managed service**, not self-hostable)                                                           | `references/intelligence-mode.md`                                                                                                                                                                                                                                                                                                                  |
-| Voice transcription — implementing a `TranscriptionService` subclass for the `/transcribe` endpoint                                                                            | `references/transcription.md`                                                                                                                                                                                                                                                                                                                      |
-| Instantiating `BuiltInAgent` — Simple Mode (classic) or Factory Mode with TanStack AI (preferred AG-UI-compliant default), AI SDK, or custom factory                           | `references/built-in-agent.md` (+ `-factory-modes.md`, `-helper-utilities.md`, `-model-identifiers.md`)                                                                                                                                                                                                                                            |
-| Defining server-side tools via `defineTool` for `BuiltInAgent.config.tools` (Simple Mode only)                                                                                 | `references/server-side-tools.md`                                                                                                                                                                                                                                                                                                                  |
-| Wiring an external agent framework into `CopilotRuntime({ agents })`                                                                                                           | `references/wiring-external-agents.md` (index) + per-framework refs (`wiring-mastra.md`, `wiring-langgraph.md`, `wiring-crewai-crews.md`, `wiring-crewai-flows.md`, `wiring-pydantic-ai.md`, `wiring-adk.md`, `wiring-llamaindex.md`, `wiring-agno.md`, `wiring-aws-strands.md`, `wiring-ms-agent-framework.md`, `wiring-ag2.md`, `wiring-a2a.md`) |
-| Wiring MCP Apps (runtime-level middleware, not an agent)                                                                                                                       | `references/wiring-mcp-apps-middleware.md`                                                                                                                                                                                                                                                                                                         |
-
-## Invariants and gotchas (load-once, before any reference)
-
-- `createCopilotRuntimeHandler` is the canonical primitive. `createCopilotExpressHandler` / `createCopilotHonoHandler` exist but are **avoid at all costs** — delegate from Express/Hono routes to the fetch primitive instead.
-- `publicLicenseKey` is the canonical provider-side field. `publicApiKey` is a **deprecated alias** — expect to see it in legacy code, emit the canonical name in new code.
-- Intelligence mode auto-wires `IntelligenceAgentRunner`. Passing both `runner` and `intelligence` to `CopilotRuntime` is rejected at construction.
-- Intelligence mode targets the managed CopilotKit Intelligence service (`api.cloud.copilotkit.ai`) and is **not self-hostable**.
-- `hooks.onRequest` runs **before** `beforeRequestMiddleware` (hook-based middleware wins for Response short-circuits). `beforeRequestMiddleware` runs **after** `hooks.onRequest` (see `fetch-handler.ts:136-147`).
-- `identifyUser` (Intelligence) does **not** forward thrown `Response` objects — convert to 500. Gate auth rejection in `hooks.onRequest`, which does forward Responses.
-- `agents__unsafe_dev_only` and `selfManagedAgents` are dev-only aliases of each other; do not reach for them in production. Either signals that the SPA is in dev mode.
-
-## Reading order for a first-time reader
-
-1. `setup-endpoint` — the primitive.
-2. `built-in-agent` **or** pick one from `wiring-external-agents` — the agent.
-3. `agent-runners` — production persistence choice.
-4. Optional: `middleware`, `intelligence-mode`, `server-side-tools`, `transcription`.

package/skills/runtime/references/agent-runners-custom.md

@@ -1,161 +0,0 @@
-Custom AgentRunner — subclass the abstract `AgentRunner` to back thread state with Redis,
-Postgres, Durable Objects, or anything else you own.
-
-## The abstract contract
-
-```typescript
-// packages/runtime/src/v2/runtime/runner/agent-runner.ts
-import {
-  AbstractAgent,
-  BaseEvent,
-  Message,
-  RunAgentInput,
-} from "@ag-ui/client";
-import { Observable } from "rxjs";
-
-export interface AgentRunnerRunRequest {
-  threadId: string;
-  agent: AbstractAgent;
-  input: RunAgentInput;
-  joinCode?: string;
-  persistedInputMessages?: Message[];
-}
-export interface AgentRunnerConnectRequest {
-  threadId: string;
-  headers?: Record<string, string>;
-  joinCode?: string;
-}
-export interface AgentRunnerIsRunningRequest {
-  threadId: string;
-}
-export interface AgentRunnerStopRequest {
-  threadId: string;
-}
-
-export abstract class AgentRunner {
-  abstract run(request: AgentRunnerRunRequest): Observable<BaseEvent>;
-  abstract connect(request: AgentRunnerConnectRequest): Observable<BaseEvent>;
-  abstract isRunning(request: AgentRunnerIsRunningRequest): Promise<boolean>;
-  abstract stop(request: AgentRunnerStopRequest): Promise<boolean | undefined>;
-}
-```
-
-## Redis-backed skeleton (for reference)
-
-```typescript
-import { AgentRunner } from "@copilotkit/runtime/v2";
-import type {
-  AgentRunnerRunRequest,
-  AgentRunnerConnectRequest,
-  AgentRunnerIsRunningRequest,
-  AgentRunnerStopRequest,
-} from "@copilotkit/runtime/v2";
-import { Observable, ReplaySubject } from "rxjs";
-import type { BaseEvent } from "@ag-ui/client";
-import { Redis } from "ioredis";
-
-const RUNNING_KEY = (t: string) => `copilotkit:running:${t}`;
-const STREAM_KEY = (t: string) => `copilotkit:stream:${t}`;
-
-export class RedisAgentRunner extends AgentRunner {
-  constructor(private redis: Redis) {
-    super();
-  }
-
-  run(request: AgentRunnerRunRequest): Observable<BaseEvent> {
-    const { threadId, agent, input } = request;
-    const subject = new ReplaySubject<BaseEvent>();
-
-    (async () => {
-      // NX guard — return 409-equivalent if another instance is running this thread
-      const acquired = await this.redis.set(
-        RUNNING_KEY(threadId),
-        "1",
-        "EX",
-        600,
-        "NX",
-      );
-      if (!acquired) {
-        subject.error(new Error("Thread already running"));
-        return;
-      }
-
-      const sub = agent.run(input).subscribe({
-        next: async (event) => {
-          subject.next(event);
-          await this.redis.xadd(
-            STREAM_KEY(threadId),
-            "*",
-            "event",
-            JSON.stringify(event),
-          );
-        },
-        error: async (err) => {
-          subject.error(err);
-          await this.redis.del(RUNNING_KEY(threadId));
-        },
-        complete: async () => {
-          subject.complete();
-          await this.redis.del(RUNNING_KEY(threadId));
-        },
-      });
-
-      // stop hook
-      this.stopHandlers.set(threadId, () => sub.unsubscribe());
-    })();
-
-    return subject.asObservable();
-  }
-
-  connect(request: AgentRunnerConnectRequest): Observable<BaseEvent> {
-    const subject = new ReplaySubject<BaseEvent>();
-    (async () => {
-      const entries = await this.redis.xrange(
-        STREAM_KEY(request.threadId),
-        "-",
-        "+",
-      );
-      for (const [, fields] of entries) {
-        const eventStr = fields[1];
-        if (eventStr) subject.next(JSON.parse(eventStr));
-      }
-      subject.complete();
-    })();
-    return subject.asObservable();
-  }
-
-  async isRunning(request: AgentRunnerIsRunningRequest): Promise<boolean> {
-    return (await this.redis.exists(RUNNING_KEY(request.threadId))) === 1;
-  }
-
-  async stop(request: AgentRunnerStopRequest): Promise<boolean | undefined> {
-    const stop = this.stopHandlers.get(request.threadId);
-    if (stop) {
-      stop();
-      this.stopHandlers.delete(request.threadId);
-    }
-    await this.redis.del(RUNNING_KEY(request.threadId));
-    return true;
-  }
-
-  private stopHandlers = new Map<string, () => void>();
-}
-```
-
-## Contract gotchas
-
-- `run()` must throw `Error("Thread already running")` (or let a distributed lock return a
-  non-acquired state) when a run is already active. Intelligence mode surfaces the 409 to
-  the client as the typed `agent_thread_locked` error code; SSE mode (direct runner use)
-  only emits a generic 500 response with the error message — so clients cannot depend on
-  the typed code there, and should additionally guard with a busy flag on submit.
-- `connect()` must replay historic events so late clients can catch up on an active run.
-- `stop()` is optional to implement in the sense that returning `undefined` is allowed, but
-  surface cancellations through `abortController.abort()` to the underlying agent if you can.
-- The runner does not persist user messages on its own — that is CopilotKit Intelligence's
-  job. A custom runner that persists only its own event stream is still a drop-in replacement
-  for `InMemoryAgentRunner` / `SqliteAgentRunner`.
-
-Source: `packages/runtime/src/v2/runtime/runner/agent-runner.ts`,
-`packages/runtime/src/v2/runtime/runner/in-memory.ts`,
-`packages/sqlite-runner/src/sqlite-runner.ts`.

package/skills/runtime/references/agent-runners-in-memory.md

@@ -1,79 +0,0 @@
-InMemoryAgentRunner — default ephemeral runner. Thread state lives in a bounded, process-global store shared by every runner instance in the process.
-
-## Store layout
-
-```typescript
-// packages/runtime/src/v2/runtime/runner/in-memory.ts
-export const ɵGLOBAL_STORE = new ɵBoundedThreadStore(ɵINMEMORY_DEFAULTS);
-```
-
-`ɵBoundedThreadStore` owns the `Map<threadId, InMemoryEventStore>`, LRU ordering, byte accounting, and eviction. The runner keeps all streaming logic and delegates storage to the store. The `ɵ` prefix marks internal API — exported for tests, not part of the public surface.
-
-One `InMemoryEventStore` per `threadId`. Each store tracks:
-
-- `subject: ReplaySubject<BaseEvent> | null` — current consumers; released on run completion
-- `isRunning: boolean` — gate for the `"Thread already running"` throw
-- `currentRunId: string | null`
-- `historicRuns: HistoricRun[]` — completed runs (events only; see snapshot note below)
-- `messagesSnapshot: Message[]` — the thread's latest non-empty message snapshot, held at the THREAD level so run-cap eviction can never drop it
-- `agent: AbstractAgent | null` — the instance that owns the active run
-- `runSubject`, `currentEvents`, `stopRequested`
-
-## Bounds
-
-Three limits, whichever trips first. Defaults in `ɵINMEMORY_DEFAULTS`:
-
-| Option             | Default | Enforcement                                                    |
-| ------------------ | ------- | -------------------------------------------------------------- |
-| `maxThreads`       | `1000`  | LRU eviction of the least-recently-used thread                 |
-| `maxRunsPerThread` | `100`   | FIFO drop of oldest runs; `Infinity` or `0` disables           |
-| `maxBytes`         | 512 MiB | Approximate total across all threads; evicts OTHER LRU threads |
-
-```typescript
-new InMemoryAgentRunner({
-  maxThreads: 200,
-  maxRunsPerThread: 50,
-  maxBytes: 128 * 1024 ** 2,
-});
-```
-
-Invariants worth knowing before touching this code:
-
-- A thread is never evicted while `isRunning` **or** `stopRequested` is set. `stop()` flips `isRunning` false immediately but the run finalizes asynchronously; evicting in that window would make the pending `appendRun` silently drop history.
-- `maxBytes` only bounds **committed** history. A single in-flight run's buffered events are not counted until the run completes, so it does not bound one runaway run mid-stream.
-- `maxBytes` evicts other threads and never self-evicts the just-appended thread — it is a cross-thread ceiling, not a per-thread cap. A single dominant thread is bounded by `maxRunsPerThread`.
-- Byte accounting is a `JSON.stringify().length` estimate, not exact heap bytes.
-- Two eviction forms, both steering heavy users to an Intelligence backend via one shared warn-once latch. Whole-thread eviction (`maxThreads` count or `maxBytes` ceiling) drops the entire LRU thread — it stops appearing in `GET /threads`. Per-thread `maxRunsPerThread` trimming drops only a thread's oldest runs' events, keeping the thread visible with its original `createdAt` and its thread-level `messagesSnapshot`. The latch fires **once per store** (not once per eviction), reset only by `clearThreads()`/`clear()`, so a hot thread trimming on every append logs a single line and every later eviction is silent until a clear.
-
-## Concurrency
-
-`onConcurrentRun` is per-runner (unlike the limits, which are process-global):
-
-- `"throw"` (default) — a second `run()` on a live thread throws `Error("Thread already running")`.
-- `"supersede"` — aborts the in-flight run (same path as `stop()`) and starts the new one. The superseded run's teardown is guarded on `store.currentRunId === request.input.runId` (so it cannot push history under the new run's id or reset the new run's state) and on `store.subject === nextSubject` (so releasing its ReplaySubject cannot null out the live run's subject).
-
-## Lifecycle
-
-1. `run({ threadId, agent, input })` — `sharedStore.getOrCreate(threadId)` (may evict other threads), then throw or supersede per `onConcurrentRun`. Create `ReplaySubject`s, run the agent, push events into the subjects and `currentEvents`, mark `isRunning`.
-2. On completion or error: finalize, `sharedStore.appendRun(...)` with the compacted events (which enforces the run cap and byte ceiling), clear `isRunning` / `currentRunId` / `agent`, and release `store.subject` so the infinite ReplaySubject buffer becomes collectable. History is rebuilt from `historicRuns` afterwards.
-3. `connect({ threadId })` — replays compacted `historicRuns`, then bridges the live subject while `isRunning || stopRequested`.
-4. `stop({ threadId })` — sets `stopRequested = true`, aborts the agent; teardown runs in the run's `catch`.
-
-## Config scope gotcha
-
-Limits reconfigure the shared store, so the **last-constructed runner wins for all in-memory threads**. A second runner passing limits that differ from an already-customized store logs a one-time clobber warning. Passing only `onConcurrentRun` leaves the limits untouched.
-
-## When NOT to use
-
-- Multi-instance production deploys — each process has its own store.
-- Anywhere history loss is unacceptable — eviction is history loss, same as a restart.
-- Load-balanced serverless with cold starts — new workers see empty stores.
-
-## When it is OK
-
-- Local development.
-- Single-instance preview environments.
-- Production single-instance deploys where scrollback is best-effort — the bounds make this safe against OOM, not durable.
-- Tests. Every `new InMemoryAgentRunner()` shares the same store, so use a fresh `threadId` per test or call `runner.clearThreads()` (which resets the map, byte total, and eviction warn latch) between tests. Tests that customize limits must restore them: `new InMemoryAgentRunner(ɵINMEMORY_DEFAULTS)` in an `afterEach` — a no-arg construction is inert and will NOT restore defaults.
-
-Source: `packages/runtime/src/v2/runtime/runner/in-memory.ts`.

package/skills/runtime/references/agent-runners-sqlite.md

@@ -1,90 +0,0 @@
-SqliteAgentRunner — file-backed agent runner in `@copilotkit/sqlite-runner`. Uses `better-sqlite3` as a required peer dep.
-
-## Install
-
-```bash
-pnpm add @copilotkit/sqlite-runner better-sqlite3
-```
-
-If `better-sqlite3` is missing, the `import` of `@copilotkit/sqlite-runner` itself fails
-at module load (`Cannot find module 'better-sqlite3'`). The runner's constructor has a
-friendlier multi-line install hint as a fallback, but you will see the bare resolution
-error first — install the peer before the runner import resolves.
-
-## Configure
-
-```typescript
-import { CopilotRuntime } from "@copilotkit/runtime/v2";
-import { SqliteAgentRunner } from "@copilotkit/sqlite-runner";
-
-const runtime = new CopilotRuntime({
-  agents: {
-    /* ... */
-  } as any,
-  runner: new SqliteAgentRunner({
-    dbPath: "./data/threads.db", // REQUIRED — default is ":memory:"
-  }),
-});
-```
-
-`dbPath: ":memory:"` is the default if omitted — that reverts to an in-memory store and
-loses data at restart. Always set a file path in production.
-
-## Schema
-
-Three tables are created on first use (`packages/sqlite-runner/src/sqlite-runner.ts:75-109`):
-
-```sql
-CREATE TABLE IF NOT EXISTS agent_runs (
-  id INTEGER PRIMARY KEY AUTOINCREMENT,
-  thread_id TEXT NOT NULL,
-  run_id TEXT NOT NULL UNIQUE,
-  parent_run_id TEXT,
-  events TEXT NOT NULL,    -- JSON-encoded BaseEvent[]
-  input TEXT NOT NULL,     -- JSON-encoded RunAgentInput
-  created_at INTEGER NOT NULL,
-  version INTEGER NOT NULL
-);
-CREATE TABLE IF NOT EXISTS run_state (
-  thread_id TEXT PRIMARY KEY,
-  is_running INTEGER DEFAULT 0,
-  current_run_id TEXT,
-  updated_at INTEGER NOT NULL
-);
-CREATE TABLE IF NOT EXISTS schema_version (
-  version INTEGER PRIMARY KEY,
-  applied_at INTEGER NOT NULL
-);
-CREATE INDEX IF NOT EXISTS idx_thread_id ON agent_runs(thread_id);
-CREATE INDEX IF NOT EXISTS idx_parent_run_id ON agent_runs(parent_run_id);
-```
-
-`agent_runs` is append-only — one row per completed run, full event log in the `events`
-column. `run_state` gates concurrent runs (the `"Thread already running"` check).
-`schema_version` tracks applied migrations so future releases can upgrade existing
-databases in place.
-
-## Retention
-
-There is no automatic retention. If you need bounded history, add a periodic purge:
-
-```typescript
-import Database from "better-sqlite3";
-
-const db = new Database("./data/threads.db");
-setInterval(
-  () => {
-    const cutoff = Date.now() - 30 * 24 * 60 * 60 * 1000; // 30 days
-    db.prepare("DELETE FROM agent_runs WHERE created_at < ?").run(cutoff);
-  },
-  60 * 60 * 1000,
-);
-```
-
-## When NOT to use
-
-- Multi-instance deploys without shared storage — each instance would have its own DB file.
-  Either put the DB on a shared volume (EFS, persistent disk) with a single writer, or
-  choose Intelligence mode or a custom Redis/Postgres runner.
-
-Source: `packages/sqlite-runner/src/sqlite-runner.ts`.