@copilotkit/runtime 1.71.0 → 1.71.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (116) hide show
  1. package/README.md +22 -0
  2. package/dist/agent/index.cjs +15 -3
  3. package/dist/agent/index.cjs.map +1 -1
  4. package/dist/agent/index.d.cts +1 -0
  5. package/dist/agent/index.d.cts.map +1 -1
  6. package/dist/agent/index.d.mts +1 -0
  7. package/dist/agent/index.d.mts.map +1 -1
  8. package/dist/agent/index.mjs +15 -3
  9. package/dist/agent/index.mjs.map +1 -1
  10. package/dist/index.cjs +1 -0
  11. package/dist/index.d.cts +9 -1
  12. package/dist/index.d.mts +9 -1
  13. package/dist/index.mjs +2 -2
  14. package/dist/lib/telemetry-client.cjs.map +1 -1
  15. package/dist/lib/telemetry-client.mjs.map +1 -1
  16. package/dist/package.cjs +8 -6
  17. package/dist/package.mjs +8 -6
  18. package/dist/service-adapters/anthropic/anthropic-adapter.cjs +59 -25
  19. package/dist/service-adapters/anthropic/anthropic-adapter.cjs.map +1 -1
  20. package/dist/service-adapters/anthropic/anthropic-adapter.d.cts +30 -1
  21. package/dist/service-adapters/anthropic/anthropic-adapter.d.cts.map +1 -1
  22. package/dist/service-adapters/anthropic/anthropic-adapter.d.mts +30 -1
  23. package/dist/service-adapters/anthropic/anthropic-adapter.d.mts.map +1 -1
  24. package/dist/service-adapters/anthropic/anthropic-adapter.mjs +59 -26
  25. package/dist/service-adapters/anthropic/anthropic-adapter.mjs.map +1 -1
  26. package/dist/service-adapters/index.d.cts +1 -1
  27. package/dist/service-adapters/index.d.mts +1 -1
  28. package/dist/v2/index.cjs +2 -0
  29. package/dist/v2/index.d.cts +2 -1
  30. package/dist/v2/index.d.mts +2 -1
  31. package/dist/v2/index.mjs +2 -1
  32. package/dist/v2/runtime/core/runtime.cjs.map +1 -1
  33. package/dist/v2/runtime/core/runtime.d.cts +2 -1
  34. package/dist/v2/runtime/core/runtime.d.cts.map +1 -1
  35. package/dist/v2/runtime/core/runtime.d.mts +2 -1
  36. package/dist/v2/runtime/core/runtime.d.mts.map +1 -1
  37. package/dist/v2/runtime/core/runtime.mjs.map +1 -1
  38. package/dist/v2/runtime/handlers/handle-run.cjs +2 -1
  39. package/dist/v2/runtime/handlers/handle-run.cjs.map +1 -1
  40. package/dist/v2/runtime/handlers/handle-run.mjs +2 -1
  41. package/dist/v2/runtime/handlers/handle-run.mjs.map +1 -1
  42. package/dist/v2/runtime/handlers/handle-stop.cjs +41 -1
  43. package/dist/v2/runtime/handlers/handle-stop.cjs.map +1 -1
  44. package/dist/v2/runtime/handlers/handle-stop.mjs +42 -2
  45. package/dist/v2/runtime/handlers/handle-stop.mjs.map +1 -1
  46. package/dist/v2/runtime/handlers/intelligence/run.cjs +1 -1
  47. package/dist/v2/runtime/handlers/intelligence/run.cjs.map +1 -1
  48. package/dist/v2/runtime/handlers/intelligence/run.mjs +1 -1
  49. package/dist/v2/runtime/handlers/intelligence/run.mjs.map +1 -1
  50. package/dist/v2/runtime/handlers/intelligence/threads.cjs +2 -3
  51. package/dist/v2/runtime/handlers/intelligence/threads.cjs.map +1 -1
  52. package/dist/v2/runtime/handlers/intelligence/threads.mjs +2 -3
  53. package/dist/v2/runtime/handlers/intelligence/threads.mjs.map +1 -1
  54. package/dist/v2/runtime/handlers/shared/agent-utils.cjs +8 -2
  55. package/dist/v2/runtime/handlers/shared/agent-utils.cjs.map +1 -1
  56. package/dist/v2/runtime/handlers/shared/agent-utils.mjs +8 -2
  57. package/dist/v2/runtime/handlers/shared/agent-utils.mjs.map +1 -1
  58. package/dist/v2/runtime/index.d.cts +1 -0
  59. package/dist/v2/runtime/index.d.cts.map +1 -1
  60. package/dist/v2/runtime/index.d.mts +1 -0
  61. package/dist/v2/runtime/index.d.mts.map +1 -1
  62. package/dist/v2/runtime/intelligence-platform/client.cjs +106 -1
  63. package/dist/v2/runtime/intelligence-platform/client.cjs.map +1 -1
  64. package/dist/v2/runtime/intelligence-platform/client.d.cts +11 -0
  65. package/dist/v2/runtime/intelligence-platform/client.d.cts.map +1 -1
  66. package/dist/v2/runtime/intelligence-platform/client.d.mts +11 -0
  67. package/dist/v2/runtime/intelligence-platform/client.d.mts.map +1 -1
  68. package/dist/v2/runtime/intelligence-platform/client.mjs +106 -1
  69. package/dist/v2/runtime/intelligence-platform/client.mjs.map +1 -1
  70. package/dist/v2/runtime/intelligence-platform/index.d.cts +1 -0
  71. package/dist/v2/runtime/intelligence-platform/index.d.mts +1 -0
  72. package/dist/v2/runtime/intelligence-platform/learned-skills.cjs +43 -0
  73. package/dist/v2/runtime/intelligence-platform/learned-skills.cjs.map +1 -0
  74. package/dist/v2/runtime/intelligence-platform/learned-skills.d.cts +34 -0
  75. package/dist/v2/runtime/intelligence-platform/learned-skills.d.cts.map +1 -0
  76. package/dist/v2/runtime/intelligence-platform/learned-skills.d.mts +34 -0
  77. package/dist/v2/runtime/intelligence-platform/learned-skills.d.mts.map +1 -0
  78. package/dist/v2/runtime/intelligence-platform/learned-skills.mjs +41 -0
  79. package/dist/v2/runtime/intelligence-platform/learned-skills.mjs.map +1 -0
  80. package/dist/v2/runtime/runner/intelligence.cjs +82 -25
  81. package/dist/v2/runtime/runner/intelligence.cjs.map +1 -1
  82. package/dist/v2/runtime/runner/intelligence.d.cts +3 -0
  83. package/dist/v2/runtime/runner/intelligence.d.cts.map +1 -1
  84. package/dist/v2/runtime/runner/intelligence.d.mts +3 -0
  85. package/dist/v2/runtime/runner/intelligence.d.mts.map +1 -1
  86. package/dist/v2/runtime/runner/intelligence.mjs +83 -26
  87. package/dist/v2/runtime/runner/intelligence.mjs.map +1 -1
  88. package/package.json +10 -7
  89. package/skills/runtime/SKILL.md +0 -98
  90. package/skills/runtime/references/agent-runners-custom.md +0 -161
  91. package/skills/runtime/references/agent-runners-in-memory.md +0 -79
  92. package/skills/runtime/references/agent-runners-sqlite.md +0 -90
  93. package/skills/runtime/references/agent-runners.md +0 -336
  94. package/skills/runtime/references/built-in-agent-factory-modes.md +0 -232
  95. package/skills/runtime/references/built-in-agent-helper-utilities.md +0 -123
  96. package/skills/runtime/references/built-in-agent-model-identifiers.md +0 -58
  97. package/skills/runtime/references/built-in-agent.md +0 -523
  98. package/skills/runtime/references/intelligence-mode.md +0 -364
  99. package/skills/runtime/references/middleware.md +0 -376
  100. package/skills/runtime/references/server-side-tools.md +0 -414
  101. package/skills/runtime/references/setup-endpoint.md +0 -503
  102. package/skills/runtime/references/transcription.md +0 -287
  103. package/skills/runtime/references/wiring-a2a.md +0 -40
  104. package/skills/runtime/references/wiring-adk.md +0 -45
  105. package/skills/runtime/references/wiring-ag2.md +0 -41
  106. package/skills/runtime/references/wiring-agno.md +0 -40
  107. package/skills/runtime/references/wiring-aws-strands.md +0 -59
  108. package/skills/runtime/references/wiring-crewai-crews.md +0 -51
  109. package/skills/runtime/references/wiring-crewai-flows.md +0 -45
  110. package/skills/runtime/references/wiring-external-agents.md +0 -348
  111. package/skills/runtime/references/wiring-langgraph.md +0 -49
  112. package/skills/runtime/references/wiring-llamaindex.md +0 -39
  113. package/skills/runtime/references/wiring-mastra.md +0 -70
  114. package/skills/runtime/references/wiring-mcp-apps-middleware.md +0 -73
  115. package/skills/runtime/references/wiring-ms-agent-framework.md +0 -41
  116. package/skills/runtime/references/wiring-pydantic-ai.md +0 -45
@@ -1,161 +0,0 @@
1
- Custom AgentRunner — subclass the abstract `AgentRunner` to back thread state with Redis,
2
- Postgres, Durable Objects, or anything else you own.
3
-
4
- ## The abstract contract
5
-
6
- ```typescript
7
- // packages/runtime/src/v2/runtime/runner/agent-runner.ts
8
- import {
9
- AbstractAgent,
10
- BaseEvent,
11
- Message,
12
- RunAgentInput,
13
- } from "@ag-ui/client";
14
- import { Observable } from "rxjs";
15
-
16
- export interface AgentRunnerRunRequest {
17
- threadId: string;
18
- agent: AbstractAgent;
19
- input: RunAgentInput;
20
- joinCode?: string;
21
- persistedInputMessages?: Message[];
22
- }
23
- export interface AgentRunnerConnectRequest {
24
- threadId: string;
25
- headers?: Record<string, string>;
26
- joinCode?: string;
27
- }
28
- export interface AgentRunnerIsRunningRequest {
29
- threadId: string;
30
- }
31
- export interface AgentRunnerStopRequest {
32
- threadId: string;
33
- }
34
-
35
- export abstract class AgentRunner {
36
- abstract run(request: AgentRunnerRunRequest): Observable<BaseEvent>;
37
- abstract connect(request: AgentRunnerConnectRequest): Observable<BaseEvent>;
38
- abstract isRunning(request: AgentRunnerIsRunningRequest): Promise<boolean>;
39
- abstract stop(request: AgentRunnerStopRequest): Promise<boolean | undefined>;
40
- }
41
- ```
42
-
43
- ## Redis-backed skeleton (for reference)
44
-
45
- ```typescript
46
- import { AgentRunner } from "@copilotkit/runtime/v2";
47
- import type {
48
- AgentRunnerRunRequest,
49
- AgentRunnerConnectRequest,
50
- AgentRunnerIsRunningRequest,
51
- AgentRunnerStopRequest,
52
- } from "@copilotkit/runtime/v2";
53
- import { Observable, ReplaySubject } from "rxjs";
54
- import type { BaseEvent } from "@ag-ui/client";
55
- import { Redis } from "ioredis";
56
-
57
- const RUNNING_KEY = (t: string) => `copilotkit:running:${t}`;
58
- const STREAM_KEY = (t: string) => `copilotkit:stream:${t}`;
59
-
60
- export class RedisAgentRunner extends AgentRunner {
61
- constructor(private redis: Redis) {
62
- super();
63
- }
64
-
65
- run(request: AgentRunnerRunRequest): Observable<BaseEvent> {
66
- const { threadId, agent, input } = request;
67
- const subject = new ReplaySubject<BaseEvent>();
68
-
69
- (async () => {
70
- // NX guard — return 409-equivalent if another instance is running this thread
71
- const acquired = await this.redis.set(
72
- RUNNING_KEY(threadId),
73
- "1",
74
- "EX",
75
- 600,
76
- "NX",
77
- );
78
- if (!acquired) {
79
- subject.error(new Error("Thread already running"));
80
- return;
81
- }
82
-
83
- const sub = agent.run(input).subscribe({
84
- next: async (event) => {
85
- subject.next(event);
86
- await this.redis.xadd(
87
- STREAM_KEY(threadId),
88
- "*",
89
- "event",
90
- JSON.stringify(event),
91
- );
92
- },
93
- error: async (err) => {
94
- subject.error(err);
95
- await this.redis.del(RUNNING_KEY(threadId));
96
- },
97
- complete: async () => {
98
- subject.complete();
99
- await this.redis.del(RUNNING_KEY(threadId));
100
- },
101
- });
102
-
103
- // stop hook
104
- this.stopHandlers.set(threadId, () => sub.unsubscribe());
105
- })();
106
-
107
- return subject.asObservable();
108
- }
109
-
110
- connect(request: AgentRunnerConnectRequest): Observable<BaseEvent> {
111
- const subject = new ReplaySubject<BaseEvent>();
112
- (async () => {
113
- const entries = await this.redis.xrange(
114
- STREAM_KEY(request.threadId),
115
- "-",
116
- "+",
117
- );
118
- for (const [, fields] of entries) {
119
- const eventStr = fields[1];
120
- if (eventStr) subject.next(JSON.parse(eventStr));
121
- }
122
- subject.complete();
123
- })();
124
- return subject.asObservable();
125
- }
126
-
127
- async isRunning(request: AgentRunnerIsRunningRequest): Promise<boolean> {
128
- return (await this.redis.exists(RUNNING_KEY(request.threadId))) === 1;
129
- }
130
-
131
- async stop(request: AgentRunnerStopRequest): Promise<boolean | undefined> {
132
- const stop = this.stopHandlers.get(request.threadId);
133
- if (stop) {
134
- stop();
135
- this.stopHandlers.delete(request.threadId);
136
- }
137
- await this.redis.del(RUNNING_KEY(request.threadId));
138
- return true;
139
- }
140
-
141
- private stopHandlers = new Map<string, () => void>();
142
- }
143
- ```
144
-
145
- ## Contract gotchas
146
-
147
- - `run()` must throw `Error("Thread already running")` (or let a distributed lock return a
148
- non-acquired state) when a run is already active. Intelligence mode surfaces the 409 to
149
- the client as the typed `agent_thread_locked` error code; SSE mode (direct runner use)
150
- only emits a generic 500 response with the error message — so clients cannot depend on
151
- the typed code there, and should additionally guard with a busy flag on submit.
152
- - `connect()` must replay historic events so late clients can catch up on an active run.
153
- - `stop()` is optional to implement in the sense that returning `undefined` is allowed, but
154
- surface cancellations through `abortController.abort()` to the underlying agent if you can.
155
- - The runner does not persist user messages on its own — that is CopilotKit Intelligence's
156
- job. A custom runner that persists only its own event stream is still a drop-in replacement
157
- for `InMemoryAgentRunner` / `SqliteAgentRunner`.
158
-
159
- Source: `packages/runtime/src/v2/runtime/runner/agent-runner.ts`,
160
- `packages/runtime/src/v2/runtime/runner/in-memory.ts`,
161
- `packages/sqlite-runner/src/sqlite-runner.ts`.
@@ -1,79 +0,0 @@
1
- InMemoryAgentRunner — default ephemeral runner. Thread state lives in a bounded, process-global store shared by every runner instance in the process.
2
-
3
- ## Store layout
4
-
5
- ```typescript
6
- // packages/runtime/src/v2/runtime/runner/in-memory.ts
7
- export const ɵGLOBAL_STORE = new ɵBoundedThreadStore(ɵINMEMORY_DEFAULTS);
8
- ```
9
-
10
- `ɵBoundedThreadStore` owns the `Map<threadId, InMemoryEventStore>`, LRU ordering, byte accounting, and eviction. The runner keeps all streaming logic and delegates storage to the store. The `ɵ` prefix marks internal API — exported for tests, not part of the public surface.
11
-
12
- One `InMemoryEventStore` per `threadId`. Each store tracks:
13
-
14
- - `subject: ReplaySubject<BaseEvent> | null` — current consumers; released on run completion
15
- - `isRunning: boolean` — gate for the `"Thread already running"` throw
16
- - `currentRunId: string | null`
17
- - `historicRuns: HistoricRun[]` — completed runs (events only; see snapshot note below)
18
- - `messagesSnapshot: Message[]` — the thread's latest non-empty message snapshot, held at the THREAD level so run-cap eviction can never drop it
19
- - `agent: AbstractAgent | null` — the instance that owns the active run
20
- - `runSubject`, `currentEvents`, `stopRequested`
21
-
22
- ## Bounds
23
-
24
- Three limits, whichever trips first. Defaults in `ɵINMEMORY_DEFAULTS`:
25
-
26
- | Option | Default | Enforcement |
27
- | ------------------ | ------- | -------------------------------------------------------------- |
28
- | `maxThreads` | `1000` | LRU eviction of the least-recently-used thread |
29
- | `maxRunsPerThread` | `100` | FIFO drop of oldest runs; `Infinity` or `0` disables |
30
- | `maxBytes` | 512 MiB | Approximate total across all threads; evicts OTHER LRU threads |
31
-
32
- ```typescript
33
- new InMemoryAgentRunner({
34
- maxThreads: 200,
35
- maxRunsPerThread: 50,
36
- maxBytes: 128 * 1024 ** 2,
37
- });
38
- ```
39
-
40
- Invariants worth knowing before touching this code:
41
-
42
- - A thread is never evicted while `isRunning` **or** `stopRequested` is set. `stop()` flips `isRunning` false immediately but the run finalizes asynchronously; evicting in that window would make the pending `appendRun` silently drop history.
43
- - `maxBytes` only bounds **committed** history. A single in-flight run's buffered events are not counted until the run completes, so it does not bound one runaway run mid-stream.
44
- - `maxBytes` evicts other threads and never self-evicts the just-appended thread — it is a cross-thread ceiling, not a per-thread cap. A single dominant thread is bounded by `maxRunsPerThread`.
45
- - Byte accounting is a `JSON.stringify().length` estimate, not exact heap bytes.
46
- - Two eviction forms, both steering heavy users to an Intelligence backend via one shared warn-once latch. Whole-thread eviction (`maxThreads` count or `maxBytes` ceiling) drops the entire LRU thread — it stops appearing in `GET /threads`. Per-thread `maxRunsPerThread` trimming drops only a thread's oldest runs' events, keeping the thread visible with its original `createdAt` and its thread-level `messagesSnapshot`. The latch fires **once per store** (not once per eviction), reset only by `clearThreads()`/`clear()`, so a hot thread trimming on every append logs a single line and every later eviction is silent until a clear.
47
-
48
- ## Concurrency
49
-
50
- `onConcurrentRun` is per-runner (unlike the limits, which are process-global):
51
-
52
- - `"throw"` (default) — a second `run()` on a live thread throws `Error("Thread already running")`.
53
- - `"supersede"` — aborts the in-flight run (same path as `stop()`) and starts the new one. The superseded run's teardown is guarded on `store.currentRunId === request.input.runId` (so it cannot push history under the new run's id or reset the new run's state) and on `store.subject === nextSubject` (so releasing its ReplaySubject cannot null out the live run's subject).
54
-
55
- ## Lifecycle
56
-
57
- 1. `run({ threadId, agent, input })` — `sharedStore.getOrCreate(threadId)` (may evict other threads), then throw or supersede per `onConcurrentRun`. Create `ReplaySubject`s, run the agent, push events into the subjects and `currentEvents`, mark `isRunning`.
58
- 2. On completion or error: finalize, `sharedStore.appendRun(...)` with the compacted events (which enforces the run cap and byte ceiling), clear `isRunning` / `currentRunId` / `agent`, and release `store.subject` so the infinite ReplaySubject buffer becomes collectable. History is rebuilt from `historicRuns` afterwards.
59
- 3. `connect({ threadId })` — replays compacted `historicRuns`, then bridges the live subject while `isRunning || stopRequested`.
60
- 4. `stop({ threadId })` — sets `stopRequested = true`, aborts the agent; teardown runs in the run's `catch`.
61
-
62
- ## Config scope gotcha
63
-
64
- Limits reconfigure the shared store, so the **last-constructed runner wins for all in-memory threads**. A second runner passing limits that differ from an already-customized store logs a one-time clobber warning. Passing only `onConcurrentRun` leaves the limits untouched.
65
-
66
- ## When NOT to use
67
-
68
- - Multi-instance production deploys — each process has its own store.
69
- - Anywhere history loss is unacceptable — eviction is history loss, same as a restart.
70
- - Load-balanced serverless with cold starts — new workers see empty stores.
71
-
72
- ## When it is OK
73
-
74
- - Local development.
75
- - Single-instance preview environments.
76
- - Production single-instance deploys where scrollback is best-effort — the bounds make this safe against OOM, not durable.
77
- - Tests. Every `new InMemoryAgentRunner()` shares the same store, so use a fresh `threadId` per test or call `runner.clearThreads()` (which resets the map, byte total, and eviction warn latch) between tests. Tests that customize limits must restore them: `new InMemoryAgentRunner(ɵINMEMORY_DEFAULTS)` in an `afterEach` — a no-arg construction is inert and will NOT restore defaults.
78
-
79
- Source: `packages/runtime/src/v2/runtime/runner/in-memory.ts`.
@@ -1,90 +0,0 @@
1
- SqliteAgentRunner — file-backed agent runner in `@copilotkit/sqlite-runner`. Uses `better-sqlite3` as a required peer dep.
2
-
3
- ## Install
4
-
5
- ```bash
6
- pnpm add @copilotkit/sqlite-runner better-sqlite3
7
- ```
8
-
9
- If `better-sqlite3` is missing, the `import` of `@copilotkit/sqlite-runner` itself fails
10
- at module load (`Cannot find module 'better-sqlite3'`). The runner's constructor has a
11
- friendlier multi-line install hint as a fallback, but you will see the bare resolution
12
- error first — install the peer before the runner import resolves.
13
-
14
- ## Configure
15
-
16
- ```typescript
17
- import { CopilotRuntime } from "@copilotkit/runtime/v2";
18
- import { SqliteAgentRunner } from "@copilotkit/sqlite-runner";
19
-
20
- const runtime = new CopilotRuntime({
21
- agents: {
22
- /* ... */
23
- } as any,
24
- runner: new SqliteAgentRunner({
25
- dbPath: "./data/threads.db", // REQUIRED — default is ":memory:"
26
- }),
27
- });
28
- ```
29
-
30
- `dbPath: ":memory:"` is the default if omitted — that reverts to an in-memory store and
31
- loses data at restart. Always set a file path in production.
32
-
33
- ## Schema
34
-
35
- Three tables are created on first use (`packages/sqlite-runner/src/sqlite-runner.ts:75-109`):
36
-
37
- ```sql
38
- CREATE TABLE IF NOT EXISTS agent_runs (
39
- id INTEGER PRIMARY KEY AUTOINCREMENT,
40
- thread_id TEXT NOT NULL,
41
- run_id TEXT NOT NULL UNIQUE,
42
- parent_run_id TEXT,
43
- events TEXT NOT NULL, -- JSON-encoded BaseEvent[]
44
- input TEXT NOT NULL, -- JSON-encoded RunAgentInput
45
- created_at INTEGER NOT NULL,
46
- version INTEGER NOT NULL
47
- );
48
- CREATE TABLE IF NOT EXISTS run_state (
49
- thread_id TEXT PRIMARY KEY,
50
- is_running INTEGER DEFAULT 0,
51
- current_run_id TEXT,
52
- updated_at INTEGER NOT NULL
53
- );
54
- CREATE TABLE IF NOT EXISTS schema_version (
55
- version INTEGER PRIMARY KEY,
56
- applied_at INTEGER NOT NULL
57
- );
58
- CREATE INDEX IF NOT EXISTS idx_thread_id ON agent_runs(thread_id);
59
- CREATE INDEX IF NOT EXISTS idx_parent_run_id ON agent_runs(parent_run_id);
60
- ```
61
-
62
- `agent_runs` is append-only — one row per completed run, full event log in the `events`
63
- column. `run_state` gates concurrent runs (the `"Thread already running"` check).
64
- `schema_version` tracks applied migrations so future releases can upgrade existing
65
- databases in place.
66
-
67
- ## Retention
68
-
69
- There is no automatic retention. If you need bounded history, add a periodic purge:
70
-
71
- ```typescript
72
- import Database from "better-sqlite3";
73
-
74
- const db = new Database("./data/threads.db");
75
- setInterval(
76
- () => {
77
- const cutoff = Date.now() - 30 * 24 * 60 * 60 * 1000; // 30 days
78
- db.prepare("DELETE FROM agent_runs WHERE created_at < ?").run(cutoff);
79
- },
80
- 60 * 60 * 1000,
81
- );
82
- ```
83
-
84
- ## When NOT to use
85
-
86
- - Multi-instance deploys without shared storage — each instance would have its own DB file.
87
- Either put the DB on a shared volume (EFS, persistent disk) with a single writer, or
88
- choose Intelligence mode or a custom Redis/Postgres runner.
89
-
90
- Source: `packages/sqlite-runner/src/sqlite-runner.ts`.
@@ -1,336 +0,0 @@
1
- # CopilotKit Agent Runners
2
-
3
- `AgentRunner` is the abstraction that owns thread run state — active runs, the event stream
4
- replay, and stop semantics. Pick one per `CopilotRuntime` instance.
5
-
6
- - `InMemoryAgentRunner` — default; process-global in-memory Map; lost on restart.
7
- - `SqliteAgentRunner` — file-backed; requires `better-sqlite3` peer.
8
- - `IntelligenceAgentRunner` — auto-wired by `CopilotIntelligenceRuntime`. You do NOT
9
- construct this directly and you cannot pass `runner` alongside `intelligence`.
10
- - Custom — subclass `AgentRunner` for Redis / Postgres / any backend.
11
-
12
- ## Setup
13
-
14
- Default (in-memory, dev only):
15
-
16
- ```typescript
17
- import { CopilotRuntime } from "@copilotkit/runtime/v2";
18
-
19
- // Equivalent to passing `runner: new InMemoryAgentRunner()`
20
- const runtime = new CopilotRuntime({
21
- agents: {
22
- /* ... */
23
- } as any,
24
- });
25
- ```
26
-
27
- Production (file-backed SQLite):
28
-
29
- ```typescript
30
- import { CopilotRuntime } from "@copilotkit/runtime/v2";
31
- import { SqliteAgentRunner } from "@copilotkit/sqlite-runner";
32
-
33
- const runtime = new CopilotRuntime({
34
- agents: {
35
- /* ... */
36
- } as any,
37
- runner: new SqliteAgentRunner({ dbPath: "./data/threads.db" }),
38
- });
39
- ```
40
-
41
- Installation for the SQLite runner (the `better-sqlite3` peer is required):
42
-
43
- ```bash
44
- pnpm add @copilotkit/sqlite-runner better-sqlite3
45
- ```
46
-
47
- ## Core Patterns
48
-
49
- ### The AgentRunner contract
50
-
51
- ```typescript
52
- import { AgentRunner } from "@copilotkit/runtime/v2";
53
- import type {
54
- AgentRunnerRunRequest,
55
- AgentRunnerConnectRequest,
56
- AgentRunnerIsRunningRequest,
57
- AgentRunnerStopRequest,
58
- } from "@copilotkit/runtime/v2";
59
- import { Observable } from "rxjs";
60
- import type { BaseEvent } from "@ag-ui/client";
61
-
62
- class MyRunner extends AgentRunner {
63
- run(request: AgentRunnerRunRequest): Observable<BaseEvent> {
64
- // Start a new run for request.threadId. Throw `new Error("Thread already running")`
65
- // if a run is in flight. Stream events from agent.run(request.input).
66
- return new Observable<BaseEvent>();
67
- }
68
- connect(request: AgentRunnerConnectRequest): Observable<BaseEvent> {
69
- // Replay events for an active run, or historic runs for request.threadId.
70
- return new Observable<BaseEvent>();
71
- }
72
- async isRunning(request: AgentRunnerIsRunningRequest): Promise<boolean> {
73
- return false;
74
- }
75
- async stop(request: AgentRunnerStopRequest): Promise<boolean | undefined> {
76
- return true;
77
- }
78
- }
79
- ```
80
-
81
- ### Handle double-submit on the client
82
-
83
- By default, both `InMemoryAgentRunner` and `SqliteAgentRunner` throw
84
- `"Thread already running"` on concurrent `run()` calls for the same `threadId`.
85
- `"throw"` is the default, but it is not the only option: constructing
86
- `InMemoryAgentRunner` with `onConcurrentRun: "supersede"` makes it abort the
87
- in-flight run (the same path `stop()` takes) and start the new one instead of
88
- throwing — the superseded run's partial output is discarded rather than persisted
89
- to history. `SqliteAgentRunner` has no such option and always throws. When the
90
- throw does happen, how it surfaces to the client depends on the runtime mode:
91
-
92
- - **Intelligence mode** — CopilotKit Intelligence returns HTTP `409` when a lock is
93
- held. The client core maps this to `CopilotKitCoreErrorCode.AGENT_THREAD_LOCKED`
94
- and fires `onError({ code: "agent_thread_locked", ... })`. Handle this in
95
- `<CopilotKit onError>` (the `CopilotKit` provider from `@copilotkit/react-core/v2`).
96
- - **SSE mode** (default, in-memory / SQLite runners) — the runner throws
97
- synchronously and the handler returns a plain `500` JSON body like
98
- `{ "error": "Failed to run agent", "message": "Thread already running" }`.
99
- There is no typed `agent_thread_locked` code — match on the message text or
100
- just guard on the client with a busy flag.
101
-
102
- ```tsx
103
- // client — Intelligence mode (typed code)
104
- import { CopilotKit } from "@copilotkit/react-core/v2";
105
-
106
- <CopilotKit
107
- onError={({ code }) => {
108
- if (code === "agent_thread_locked") {
109
- alert("Agent is busy — wait for the current response to finish.");
110
- }
111
- }}
112
- />;
113
- ```
114
-
115
- ```tsx
116
- // client — any mode: guard with a busy flag so double-submit is impossible
117
- import { useAgent } from "@copilotkit/react-core/v2";
118
- import { useState } from "react";
119
-
120
- function Composer() {
121
- const agent = useAgent({ agentId: "default" });
122
- const [busy, setBusy] = useState(false);
123
-
124
- async function send(text: string) {
125
- if (busy) return;
126
- setBusy(true);
127
- try {
128
- await agent?.addMessage({ role: "user", content: text });
129
- } finally {
130
- setBusy(false);
131
- }
132
- }
133
-
134
- return null;
135
- }
136
- ```
137
-
138
- ## Common Mistakes
139
-
140
- ### HIGH Shipping InMemoryAgentRunner to production
141
-
142
- Wrong:
143
-
144
- ```typescript
145
- // production:
146
- new CopilotRuntime({ agents: { default: agent } });
147
- ```
148
-
149
- Correct:
150
-
151
- ```typescript
152
- import { SqliteAgentRunner } from "@copilotkit/sqlite-runner";
153
-
154
- new CopilotRuntime({
155
- agents: { default: agent },
156
- runner: new SqliteAgentRunner({ dbPath: "./data/threads.db" }),
157
- });
158
- // Or upgrade to Intelligence mode for managed durability.
159
- ```
160
-
161
- The default runner is `new InMemoryAgentRunner()`. It keeps state in a process-global,
162
- bounded store — threads are lost on restart, evicted past the memory limits, and
163
- horizontally-scaled instances see divergent state. See `agent-runners-in-memory.md`
164
- for the bounds and how to tune them.
165
-
166
- Source: `packages/runtime/src/v2/runtime/runner/in-memory.ts`.
167
-
168
- ### HIGH Setting runner alongside intelligence option
169
-
170
- Wrong:
171
-
172
- ```typescript
173
- new CopilotRuntime({
174
- agents,
175
- intelligence,
176
- runner: new SqliteAgentRunner({ dbPath: "./data/threads.db" }),
177
- });
178
- ```
179
-
180
- Correct:
181
-
182
- ```typescript
183
- new CopilotRuntime({
184
- agents,
185
- intelligence,
186
- identifyUser: (req) => ({
187
- id: req.headers.get("x-user-id")!,
188
- name: req.headers.get("x-user-name") ?? "Anonymous",
189
- }),
190
- });
191
- ```
192
-
193
- `CopilotIntelligenceRuntimeOptions` does not declare a `runner` field — Intelligence mode
194
- auto-wires `IntelligenceAgentRunner` pointed at the Intelligence service socket. Excess-property checks will
195
- flag a `runner:` key on an Intelligence-shaped options object as a type error, and a caller who
196
- evades that check (JS, `as any`, or a non-literal options object) gets a `throw` at construction
197
- rather than a silently ignored runner.
198
-
199
- Source: `packages/runtime/src/v2/runtime/core/runtime.ts` — `runner?` is declared only on
200
- `CopilotSseRuntimeOptions` (:239); the Intelligence constructor guard is at :512 and the
201
- auto-wired runner at :582.
202
-
203
- ### HIGH Forgetting the better-sqlite3 peer
204
-
205
- Wrong:
206
-
207
- ```bash
208
- pnpm add @copilotkit/sqlite-runner
209
- ```
210
-
211
- Correct:
212
-
213
- ```bash
214
- pnpm add @copilotkit/sqlite-runner better-sqlite3
215
- ```
216
-
217
- `@copilotkit/sqlite-runner` imports `better-sqlite3` at the top of its module, so if the peer
218
- is missing, `import { SqliteAgentRunner } from "@copilotkit/sqlite-runner"` itself fails at
219
- module load with `Cannot find module 'better-sqlite3'` — long before the constructor runs.
220
- (The constructor has a friendlier multi-line install hint as a belt-and-suspenders fallback,
221
- but in practice you will see the bare module-resolution error first.) It is a peer dependency,
222
- not a direct dep.
223
-
224
- Source: `packages/sqlite-runner/src/sqlite-runner.ts:18`, `:55-66`.
225
-
226
- ### HIGH Default SqliteAgentRunner with :memory: dbPath
227
-
228
- Wrong:
229
-
230
- ```typescript
231
- new SqliteAgentRunner();
232
- ```
233
-
234
- Correct:
235
-
236
- ```typescript
237
- new SqliteAgentRunner({ dbPath: "./data/threads.db" });
238
- ```
239
-
240
- The default `dbPath` is `":memory:"` — SQLite's in-memory mode. Data is lost at restart,
241
- defeating the reason to use the file-backed runner.
242
-
243
- Source: `packages/sqlite-runner/src/sqlite-runner.ts:48-54`.
244
-
245
- ### MEDIUM Concurrent run() on the same threadId
246
-
247
- Wrong:
248
-
249
- ```tsx
250
- // Double-click send button → two POST /agent/:id/run to the same thread
251
- <button onClick={() => agent.addMessage({ role: "user", content })}>
252
- Send
253
- </button>
254
- ```
255
-
256
- Correct:
257
-
258
- ```tsx
259
- const [busy, setBusy] = useState(false);
260
- <button
261
- disabled={busy}
262
- onClick={async () => {
263
- setBusy(true);
264
- try {
265
- await agent.addMessage({ role: "user", content });
266
- } finally {
267
- setBusy(false);
268
- }
269
- }}
270
- >
271
- Send
272
- </button>;
273
- ```
274
-
275
- By default both runners throw `"Thread already running"` on concurrent runs, so
276
- debouncing on the client is still the right baseline. In Intelligence mode you can
277
- additionally handle `code === "agent_thread_locked"` in `<CopilotKit onError>`; SSE
278
- mode surfaces only a generic 500 with that message.
279
-
280
- Throwing is the default (`onConcurrentRun: "throw"`), not the only behavior:
281
- constructing `InMemoryAgentRunner` with `onConcurrentRun: "supersede"` aborts the
282
- in-flight run (the `stop()` path) and starts the new one instead of throwing,
283
- discarding the superseded run's partial output rather than persisting it. That
284
- suits a UX where a fast follow-up should displace a still-running (or wedged) turn.
285
- Unlike the process-global memory limits, `onConcurrentRun` is per-runner-instance —
286
- it affects only the runner you pass it to. `SqliteAgentRunner` has no such option
287
- and always throws.
288
-
289
- Source: the `throw new Error("Thread already running")` in `InMemoryAgentRunner.run()`,
290
- `packages/runtime/src/v2/runtime/runner/in-memory.ts`;
291
- `packages/core/src/intelligence-agent.ts:368-369`.
292
-
293
- ### HIGH In-memory runner + horizontal scaling
294
-
295
- Wrong:
296
-
297
- ```typescript
298
- // 3 Fly.io / Cloud Run instances, each with its own InMemoryAgentRunner
299
- new CopilotRuntime({ agents });
300
- ```
301
-
302
- Correct:
303
-
304
- ```typescript
305
- // Sticky-session one instance per thread (so every run for a thread lands on the
306
- // same process), OR move to Intelligence mode for managed multi-instance durability.
307
- new CopilotRuntime({ agents }); // + route by threadId at the load balancer
308
- ```
309
-
310
- `InMemoryAgentRunner`'s store is a process-global singleton — multi-instance deploys see
311
- totally different thread state per worker, making reconnects and `GET /connect` non-deterministic.
312
-
313
- Source: the exported `ɵGLOBAL_STORE` singleton in `packages/runtime/src/v2/runtime/runner/in-memory.ts`.
314
-
315
- A shared `dbPath` on `SqliteAgentRunner` is **not** a horizontal-scaling fix on its own.
316
- Sharing the file gives you durable, persisted history: runs survive process restarts, and
317
- completed runs are readable from any instance pointed at the same file. But the live-run
318
- bookkeeping used by the connect-bridge and by `stop()` lives in a process-local
319
- `ACTIVE_CONNECTIONS` map. A second instance has **no** entry for a run started elsewhere, so
320
- it can replay stored history but **cannot** reconnect to — or stop — an in-flight run on
321
- another instance. Use `SqliteAgentRunner` for restart-resilient single-instance durability;
322
- for managed multi-instance durability, use Intelligence mode.
323
-
324
- Source: `packages/sqlite-runner/src/sqlite-runner.ts:46` (module-level `ACTIVE_CONNECTIONS`).
325
-
326
- ## References
327
-
328
- - [InMemoryAgentRunner — store, bounds, concurrency, and lifecycle](agent-runners-in-memory.md)
329
- - [SqliteAgentRunner — schema, retention, ops](agent-runners-sqlite.md)
330
- - [Custom runner — Redis/Postgres skeleton](agent-runners-custom.md)
331
-
332
- ## See also
333
-
334
- - `copilotkit/intelligence-mode` — managed durability alternative (CopilotKit Intelligence managed service, not self-hostable)
335
- - `copilotkit/setup-endpoint` — runner is passed via the CopilotRuntime constructor
336
- - `copilotkit/scale-to-multi-agent` — horizontal scaling considerations