@copilotkit/runtime 1.71.0 → 1.71.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +22 -0
- package/dist/agent/index.cjs +15 -3
- package/dist/agent/index.cjs.map +1 -1
- package/dist/agent/index.d.cts +1 -0
- package/dist/agent/index.d.cts.map +1 -1
- package/dist/agent/index.d.mts +1 -0
- package/dist/agent/index.d.mts.map +1 -1
- package/dist/agent/index.mjs +15 -3
- package/dist/agent/index.mjs.map +1 -1
- package/dist/index.cjs +1 -0
- package/dist/index.d.cts +9 -1
- package/dist/index.d.mts +9 -1
- package/dist/index.mjs +2 -2
- package/dist/lib/telemetry-client.cjs.map +1 -1
- package/dist/lib/telemetry-client.mjs.map +1 -1
- package/dist/package.cjs +8 -6
- package/dist/package.mjs +8 -6
- package/dist/service-adapters/anthropic/anthropic-adapter.cjs +59 -25
- package/dist/service-adapters/anthropic/anthropic-adapter.cjs.map +1 -1
- package/dist/service-adapters/anthropic/anthropic-adapter.d.cts +30 -1
- package/dist/service-adapters/anthropic/anthropic-adapter.d.cts.map +1 -1
- package/dist/service-adapters/anthropic/anthropic-adapter.d.mts +30 -1
- package/dist/service-adapters/anthropic/anthropic-adapter.d.mts.map +1 -1
- package/dist/service-adapters/anthropic/anthropic-adapter.mjs +59 -26
- package/dist/service-adapters/anthropic/anthropic-adapter.mjs.map +1 -1
- package/dist/service-adapters/index.d.cts +1 -1
- package/dist/service-adapters/index.d.mts +1 -1
- package/dist/v2/index.cjs +2 -0
- package/dist/v2/index.d.cts +2 -1
- package/dist/v2/index.d.mts +2 -1
- package/dist/v2/index.mjs +2 -1
- package/dist/v2/runtime/core/runtime.cjs.map +1 -1
- package/dist/v2/runtime/core/runtime.d.cts +2 -1
- package/dist/v2/runtime/core/runtime.d.cts.map +1 -1
- package/dist/v2/runtime/core/runtime.d.mts +2 -1
- package/dist/v2/runtime/core/runtime.d.mts.map +1 -1
- package/dist/v2/runtime/core/runtime.mjs.map +1 -1
- package/dist/v2/runtime/handlers/handle-run.cjs +2 -1
- package/dist/v2/runtime/handlers/handle-run.cjs.map +1 -1
- package/dist/v2/runtime/handlers/handle-run.mjs +2 -1
- package/dist/v2/runtime/handlers/handle-run.mjs.map +1 -1
- package/dist/v2/runtime/handlers/handle-stop.cjs +41 -1
- package/dist/v2/runtime/handlers/handle-stop.cjs.map +1 -1
- package/dist/v2/runtime/handlers/handle-stop.mjs +42 -2
- package/dist/v2/runtime/handlers/handle-stop.mjs.map +1 -1
- package/dist/v2/runtime/handlers/intelligence/run.cjs +1 -1
- package/dist/v2/runtime/handlers/intelligence/run.cjs.map +1 -1
- package/dist/v2/runtime/handlers/intelligence/run.mjs +1 -1
- package/dist/v2/runtime/handlers/intelligence/run.mjs.map +1 -1
- package/dist/v2/runtime/handlers/intelligence/threads.cjs +2 -3
- package/dist/v2/runtime/handlers/intelligence/threads.cjs.map +1 -1
- package/dist/v2/runtime/handlers/intelligence/threads.mjs +2 -3
- package/dist/v2/runtime/handlers/intelligence/threads.mjs.map +1 -1
- package/dist/v2/runtime/handlers/shared/agent-utils.cjs +8 -2
- package/dist/v2/runtime/handlers/shared/agent-utils.cjs.map +1 -1
- package/dist/v2/runtime/handlers/shared/agent-utils.mjs +8 -2
- package/dist/v2/runtime/handlers/shared/agent-utils.mjs.map +1 -1
- package/dist/v2/runtime/index.d.cts +1 -0
- package/dist/v2/runtime/index.d.cts.map +1 -1
- package/dist/v2/runtime/index.d.mts +1 -0
- package/dist/v2/runtime/index.d.mts.map +1 -1
- package/dist/v2/runtime/intelligence-platform/client.cjs +106 -1
- package/dist/v2/runtime/intelligence-platform/client.cjs.map +1 -1
- package/dist/v2/runtime/intelligence-platform/client.d.cts +11 -0
- package/dist/v2/runtime/intelligence-platform/client.d.cts.map +1 -1
- package/dist/v2/runtime/intelligence-platform/client.d.mts +11 -0
- package/dist/v2/runtime/intelligence-platform/client.d.mts.map +1 -1
- package/dist/v2/runtime/intelligence-platform/client.mjs +106 -1
- package/dist/v2/runtime/intelligence-platform/client.mjs.map +1 -1
- package/dist/v2/runtime/intelligence-platform/index.d.cts +1 -0
- package/dist/v2/runtime/intelligence-platform/index.d.mts +1 -0
- package/dist/v2/runtime/intelligence-platform/learned-skills.cjs +43 -0
- package/dist/v2/runtime/intelligence-platform/learned-skills.cjs.map +1 -0
- package/dist/v2/runtime/intelligence-platform/learned-skills.d.cts +34 -0
- package/dist/v2/runtime/intelligence-platform/learned-skills.d.cts.map +1 -0
- package/dist/v2/runtime/intelligence-platform/learned-skills.d.mts +34 -0
- package/dist/v2/runtime/intelligence-platform/learned-skills.d.mts.map +1 -0
- package/dist/v2/runtime/intelligence-platform/learned-skills.mjs +41 -0
- package/dist/v2/runtime/intelligence-platform/learned-skills.mjs.map +1 -0
- package/dist/v2/runtime/runner/intelligence.cjs +82 -25
- package/dist/v2/runtime/runner/intelligence.cjs.map +1 -1
- package/dist/v2/runtime/runner/intelligence.d.cts +3 -0
- package/dist/v2/runtime/runner/intelligence.d.cts.map +1 -1
- package/dist/v2/runtime/runner/intelligence.d.mts +3 -0
- package/dist/v2/runtime/runner/intelligence.d.mts.map +1 -1
- package/dist/v2/runtime/runner/intelligence.mjs +83 -26
- package/dist/v2/runtime/runner/intelligence.mjs.map +1 -1
- package/package.json +10 -7
- package/skills/runtime/SKILL.md +0 -98
- package/skills/runtime/references/agent-runners-custom.md +0 -161
- package/skills/runtime/references/agent-runners-in-memory.md +0 -79
- package/skills/runtime/references/agent-runners-sqlite.md +0 -90
- package/skills/runtime/references/agent-runners.md +0 -336
- package/skills/runtime/references/built-in-agent-factory-modes.md +0 -232
- package/skills/runtime/references/built-in-agent-helper-utilities.md +0 -123
- package/skills/runtime/references/built-in-agent-model-identifiers.md +0 -58
- package/skills/runtime/references/built-in-agent.md +0 -523
- package/skills/runtime/references/intelligence-mode.md +0 -364
- package/skills/runtime/references/middleware.md +0 -376
- package/skills/runtime/references/server-side-tools.md +0 -414
- package/skills/runtime/references/setup-endpoint.md +0 -503
- package/skills/runtime/references/transcription.md +0 -287
- package/skills/runtime/references/wiring-a2a.md +0 -40
- package/skills/runtime/references/wiring-adk.md +0 -45
- package/skills/runtime/references/wiring-ag2.md +0 -41
- package/skills/runtime/references/wiring-agno.md +0 -40
- package/skills/runtime/references/wiring-aws-strands.md +0 -59
- package/skills/runtime/references/wiring-crewai-crews.md +0 -51
- package/skills/runtime/references/wiring-crewai-flows.md +0 -45
- package/skills/runtime/references/wiring-external-agents.md +0 -348
- package/skills/runtime/references/wiring-langgraph.md +0 -49
- package/skills/runtime/references/wiring-llamaindex.md +0 -39
- package/skills/runtime/references/wiring-mastra.md +0 -70
- package/skills/runtime/references/wiring-mcp-apps-middleware.md +0 -73
- package/skills/runtime/references/wiring-ms-agent-framework.md +0 -41
- package/skills/runtime/references/wiring-pydantic-ai.md +0 -45
|
@@ -1,161 +0,0 @@
|
|
|
1
|
-
Custom AgentRunner — subclass the abstract `AgentRunner` to back thread state with Redis,
|
|
2
|
-
Postgres, Durable Objects, or anything else you own.
|
|
3
|
-
|
|
4
|
-
## The abstract contract
|
|
5
|
-
|
|
6
|
-
```typescript
|
|
7
|
-
// packages/runtime/src/v2/runtime/runner/agent-runner.ts
|
|
8
|
-
import {
|
|
9
|
-
AbstractAgent,
|
|
10
|
-
BaseEvent,
|
|
11
|
-
Message,
|
|
12
|
-
RunAgentInput,
|
|
13
|
-
} from "@ag-ui/client";
|
|
14
|
-
import { Observable } from "rxjs";
|
|
15
|
-
|
|
16
|
-
export interface AgentRunnerRunRequest {
|
|
17
|
-
threadId: string;
|
|
18
|
-
agent: AbstractAgent;
|
|
19
|
-
input: RunAgentInput;
|
|
20
|
-
joinCode?: string;
|
|
21
|
-
persistedInputMessages?: Message[];
|
|
22
|
-
}
|
|
23
|
-
export interface AgentRunnerConnectRequest {
|
|
24
|
-
threadId: string;
|
|
25
|
-
headers?: Record<string, string>;
|
|
26
|
-
joinCode?: string;
|
|
27
|
-
}
|
|
28
|
-
export interface AgentRunnerIsRunningRequest {
|
|
29
|
-
threadId: string;
|
|
30
|
-
}
|
|
31
|
-
export interface AgentRunnerStopRequest {
|
|
32
|
-
threadId: string;
|
|
33
|
-
}
|
|
34
|
-
|
|
35
|
-
export abstract class AgentRunner {
|
|
36
|
-
abstract run(request: AgentRunnerRunRequest): Observable<BaseEvent>;
|
|
37
|
-
abstract connect(request: AgentRunnerConnectRequest): Observable<BaseEvent>;
|
|
38
|
-
abstract isRunning(request: AgentRunnerIsRunningRequest): Promise<boolean>;
|
|
39
|
-
abstract stop(request: AgentRunnerStopRequest): Promise<boolean | undefined>;
|
|
40
|
-
}
|
|
41
|
-
```
|
|
42
|
-
|
|
43
|
-
## Redis-backed skeleton (for reference)
|
|
44
|
-
|
|
45
|
-
```typescript
|
|
46
|
-
import { AgentRunner } from "@copilotkit/runtime/v2";
|
|
47
|
-
import type {
|
|
48
|
-
AgentRunnerRunRequest,
|
|
49
|
-
AgentRunnerConnectRequest,
|
|
50
|
-
AgentRunnerIsRunningRequest,
|
|
51
|
-
AgentRunnerStopRequest,
|
|
52
|
-
} from "@copilotkit/runtime/v2";
|
|
53
|
-
import { Observable, ReplaySubject } from "rxjs";
|
|
54
|
-
import type { BaseEvent } from "@ag-ui/client";
|
|
55
|
-
import { Redis } from "ioredis";
|
|
56
|
-
|
|
57
|
-
const RUNNING_KEY = (t: string) => `copilotkit:running:${t}`;
|
|
58
|
-
const STREAM_KEY = (t: string) => `copilotkit:stream:${t}`;
|
|
59
|
-
|
|
60
|
-
export class RedisAgentRunner extends AgentRunner {
|
|
61
|
-
constructor(private redis: Redis) {
|
|
62
|
-
super();
|
|
63
|
-
}
|
|
64
|
-
|
|
65
|
-
run(request: AgentRunnerRunRequest): Observable<BaseEvent> {
|
|
66
|
-
const { threadId, agent, input } = request;
|
|
67
|
-
const subject = new ReplaySubject<BaseEvent>();
|
|
68
|
-
|
|
69
|
-
(async () => {
|
|
70
|
-
// NX guard — return 409-equivalent if another instance is running this thread
|
|
71
|
-
const acquired = await this.redis.set(
|
|
72
|
-
RUNNING_KEY(threadId),
|
|
73
|
-
"1",
|
|
74
|
-
"EX",
|
|
75
|
-
600,
|
|
76
|
-
"NX",
|
|
77
|
-
);
|
|
78
|
-
if (!acquired) {
|
|
79
|
-
subject.error(new Error("Thread already running"));
|
|
80
|
-
return;
|
|
81
|
-
}
|
|
82
|
-
|
|
83
|
-
const sub = agent.run(input).subscribe({
|
|
84
|
-
next: async (event) => {
|
|
85
|
-
subject.next(event);
|
|
86
|
-
await this.redis.xadd(
|
|
87
|
-
STREAM_KEY(threadId),
|
|
88
|
-
"*",
|
|
89
|
-
"event",
|
|
90
|
-
JSON.stringify(event),
|
|
91
|
-
);
|
|
92
|
-
},
|
|
93
|
-
error: async (err) => {
|
|
94
|
-
subject.error(err);
|
|
95
|
-
await this.redis.del(RUNNING_KEY(threadId));
|
|
96
|
-
},
|
|
97
|
-
complete: async () => {
|
|
98
|
-
subject.complete();
|
|
99
|
-
await this.redis.del(RUNNING_KEY(threadId));
|
|
100
|
-
},
|
|
101
|
-
});
|
|
102
|
-
|
|
103
|
-
// stop hook
|
|
104
|
-
this.stopHandlers.set(threadId, () => sub.unsubscribe());
|
|
105
|
-
})();
|
|
106
|
-
|
|
107
|
-
return subject.asObservable();
|
|
108
|
-
}
|
|
109
|
-
|
|
110
|
-
connect(request: AgentRunnerConnectRequest): Observable<BaseEvent> {
|
|
111
|
-
const subject = new ReplaySubject<BaseEvent>();
|
|
112
|
-
(async () => {
|
|
113
|
-
const entries = await this.redis.xrange(
|
|
114
|
-
STREAM_KEY(request.threadId),
|
|
115
|
-
"-",
|
|
116
|
-
"+",
|
|
117
|
-
);
|
|
118
|
-
for (const [, fields] of entries) {
|
|
119
|
-
const eventStr = fields[1];
|
|
120
|
-
if (eventStr) subject.next(JSON.parse(eventStr));
|
|
121
|
-
}
|
|
122
|
-
subject.complete();
|
|
123
|
-
})();
|
|
124
|
-
return subject.asObservable();
|
|
125
|
-
}
|
|
126
|
-
|
|
127
|
-
async isRunning(request: AgentRunnerIsRunningRequest): Promise<boolean> {
|
|
128
|
-
return (await this.redis.exists(RUNNING_KEY(request.threadId))) === 1;
|
|
129
|
-
}
|
|
130
|
-
|
|
131
|
-
async stop(request: AgentRunnerStopRequest): Promise<boolean | undefined> {
|
|
132
|
-
const stop = this.stopHandlers.get(request.threadId);
|
|
133
|
-
if (stop) {
|
|
134
|
-
stop();
|
|
135
|
-
this.stopHandlers.delete(request.threadId);
|
|
136
|
-
}
|
|
137
|
-
await this.redis.del(RUNNING_KEY(request.threadId));
|
|
138
|
-
return true;
|
|
139
|
-
}
|
|
140
|
-
|
|
141
|
-
private stopHandlers = new Map<string, () => void>();
|
|
142
|
-
}
|
|
143
|
-
```
|
|
144
|
-
|
|
145
|
-
## Contract gotchas
|
|
146
|
-
|
|
147
|
-
- `run()` must throw `Error("Thread already running")` (or let a distributed lock return a
|
|
148
|
-
non-acquired state) when a run is already active. Intelligence mode surfaces the 409 to
|
|
149
|
-
the client as the typed `agent_thread_locked` error code; SSE mode (direct runner use)
|
|
150
|
-
only emits a generic 500 response with the error message — so clients cannot depend on
|
|
151
|
-
the typed code there, and should additionally guard with a busy flag on submit.
|
|
152
|
-
- `connect()` must replay historic events so late clients can catch up on an active run.
|
|
153
|
-
- `stop()` is optional to implement in the sense that returning `undefined` is allowed, but
|
|
154
|
-
surface cancellations through `abortController.abort()` to the underlying agent if you can.
|
|
155
|
-
- The runner does not persist user messages on its own — that is CopilotKit Intelligence's
|
|
156
|
-
job. A custom runner that persists only its own event stream is still a drop-in replacement
|
|
157
|
-
for `InMemoryAgentRunner` / `SqliteAgentRunner`.
|
|
158
|
-
|
|
159
|
-
Source: `packages/runtime/src/v2/runtime/runner/agent-runner.ts`,
|
|
160
|
-
`packages/runtime/src/v2/runtime/runner/in-memory.ts`,
|
|
161
|
-
`packages/sqlite-runner/src/sqlite-runner.ts`.
|
|
@@ -1,79 +0,0 @@
|
|
|
1
|
-
InMemoryAgentRunner — default ephemeral runner. Thread state lives in a bounded, process-global store shared by every runner instance in the process.
|
|
2
|
-
|
|
3
|
-
## Store layout
|
|
4
|
-
|
|
5
|
-
```typescript
|
|
6
|
-
// packages/runtime/src/v2/runtime/runner/in-memory.ts
|
|
7
|
-
export const ɵGLOBAL_STORE = new ɵBoundedThreadStore(ɵINMEMORY_DEFAULTS);
|
|
8
|
-
```
|
|
9
|
-
|
|
10
|
-
`ɵBoundedThreadStore` owns the `Map<threadId, InMemoryEventStore>`, LRU ordering, byte accounting, and eviction. The runner keeps all streaming logic and delegates storage to the store. The `ɵ` prefix marks internal API — exported for tests, not part of the public surface.
|
|
11
|
-
|
|
12
|
-
One `InMemoryEventStore` per `threadId`. Each store tracks:
|
|
13
|
-
|
|
14
|
-
- `subject: ReplaySubject<BaseEvent> | null` — current consumers; released on run completion
|
|
15
|
-
- `isRunning: boolean` — gate for the `"Thread already running"` throw
|
|
16
|
-
- `currentRunId: string | null`
|
|
17
|
-
- `historicRuns: HistoricRun[]` — completed runs (events only; see snapshot note below)
|
|
18
|
-
- `messagesSnapshot: Message[]` — the thread's latest non-empty message snapshot, held at the THREAD level so run-cap eviction can never drop it
|
|
19
|
-
- `agent: AbstractAgent | null` — the instance that owns the active run
|
|
20
|
-
- `runSubject`, `currentEvents`, `stopRequested`
|
|
21
|
-
|
|
22
|
-
## Bounds
|
|
23
|
-
|
|
24
|
-
Three limits, whichever trips first. Defaults in `ɵINMEMORY_DEFAULTS`:
|
|
25
|
-
|
|
26
|
-
| Option | Default | Enforcement |
|
|
27
|
-
| ------------------ | ------- | -------------------------------------------------------------- |
|
|
28
|
-
| `maxThreads` | `1000` | LRU eviction of the least-recently-used thread |
|
|
29
|
-
| `maxRunsPerThread` | `100` | FIFO drop of oldest runs; `Infinity` or `0` disables |
|
|
30
|
-
| `maxBytes` | 512 MiB | Approximate total across all threads; evicts OTHER LRU threads |
|
|
31
|
-
|
|
32
|
-
```typescript
|
|
33
|
-
new InMemoryAgentRunner({
|
|
34
|
-
maxThreads: 200,
|
|
35
|
-
maxRunsPerThread: 50,
|
|
36
|
-
maxBytes: 128 * 1024 ** 2,
|
|
37
|
-
});
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
Invariants worth knowing before touching this code:
|
|
41
|
-
|
|
42
|
-
- A thread is never evicted while `isRunning` **or** `stopRequested` is set. `stop()` flips `isRunning` false immediately but the run finalizes asynchronously; evicting in that window would make the pending `appendRun` silently drop history.
|
|
43
|
-
- `maxBytes` only bounds **committed** history. A single in-flight run's buffered events are not counted until the run completes, so it does not bound one runaway run mid-stream.
|
|
44
|
-
- `maxBytes` evicts other threads and never self-evicts the just-appended thread — it is a cross-thread ceiling, not a per-thread cap. A single dominant thread is bounded by `maxRunsPerThread`.
|
|
45
|
-
- Byte accounting is a `JSON.stringify().length` estimate, not exact heap bytes.
|
|
46
|
-
- Two eviction forms, both steering heavy users to an Intelligence backend via one shared warn-once latch. Whole-thread eviction (`maxThreads` count or `maxBytes` ceiling) drops the entire LRU thread — it stops appearing in `GET /threads`. Per-thread `maxRunsPerThread` trimming drops only a thread's oldest runs' events, keeping the thread visible with its original `createdAt` and its thread-level `messagesSnapshot`. The latch fires **once per store** (not once per eviction), reset only by `clearThreads()`/`clear()`, so a hot thread trimming on every append logs a single line and every later eviction is silent until a clear.
|
|
47
|
-
|
|
48
|
-
## Concurrency
|
|
49
|
-
|
|
50
|
-
`onConcurrentRun` is per-runner (unlike the limits, which are process-global):
|
|
51
|
-
|
|
52
|
-
- `"throw"` (default) — a second `run()` on a live thread throws `Error("Thread already running")`.
|
|
53
|
-
- `"supersede"` — aborts the in-flight run (same path as `stop()`) and starts the new one. The superseded run's teardown is guarded on `store.currentRunId === request.input.runId` (so it cannot push history under the new run's id or reset the new run's state) and on `store.subject === nextSubject` (so releasing its ReplaySubject cannot null out the live run's subject).
|
|
54
|
-
|
|
55
|
-
## Lifecycle
|
|
56
|
-
|
|
57
|
-
1. `run({ threadId, agent, input })` — `sharedStore.getOrCreate(threadId)` (may evict other threads), then throw or supersede per `onConcurrentRun`. Create `ReplaySubject`s, run the agent, push events into the subjects and `currentEvents`, mark `isRunning`.
|
|
58
|
-
2. On completion or error: finalize, `sharedStore.appendRun(...)` with the compacted events (which enforces the run cap and byte ceiling), clear `isRunning` / `currentRunId` / `agent`, and release `store.subject` so the infinite ReplaySubject buffer becomes collectable. History is rebuilt from `historicRuns` afterwards.
|
|
59
|
-
3. `connect({ threadId })` — replays compacted `historicRuns`, then bridges the live subject while `isRunning || stopRequested`.
|
|
60
|
-
4. `stop({ threadId })` — sets `stopRequested = true`, aborts the agent; teardown runs in the run's `catch`.
|
|
61
|
-
|
|
62
|
-
## Config scope gotcha
|
|
63
|
-
|
|
64
|
-
Limits reconfigure the shared store, so the **last-constructed runner wins for all in-memory threads**. A second runner passing limits that differ from an already-customized store logs a one-time clobber warning. Passing only `onConcurrentRun` leaves the limits untouched.
|
|
65
|
-
|
|
66
|
-
## When NOT to use
|
|
67
|
-
|
|
68
|
-
- Multi-instance production deploys — each process has its own store.
|
|
69
|
-
- Anywhere history loss is unacceptable — eviction is history loss, same as a restart.
|
|
70
|
-
- Load-balanced serverless with cold starts — new workers see empty stores.
|
|
71
|
-
|
|
72
|
-
## When it is OK
|
|
73
|
-
|
|
74
|
-
- Local development.
|
|
75
|
-
- Single-instance preview environments.
|
|
76
|
-
- Production single-instance deploys where scrollback is best-effort — the bounds make this safe against OOM, not durable.
|
|
77
|
-
- Tests. Every `new InMemoryAgentRunner()` shares the same store, so use a fresh `threadId` per test or call `runner.clearThreads()` (which resets the map, byte total, and eviction warn latch) between tests. Tests that customize limits must restore them: `new InMemoryAgentRunner(ɵINMEMORY_DEFAULTS)` in an `afterEach` — a no-arg construction is inert and will NOT restore defaults.
|
|
78
|
-
|
|
79
|
-
Source: `packages/runtime/src/v2/runtime/runner/in-memory.ts`.
|
|
@@ -1,90 +0,0 @@
|
|
|
1
|
-
SqliteAgentRunner — file-backed agent runner in `@copilotkit/sqlite-runner`. Uses `better-sqlite3` as a required peer dep.
|
|
2
|
-
|
|
3
|
-
## Install
|
|
4
|
-
|
|
5
|
-
```bash
|
|
6
|
-
pnpm add @copilotkit/sqlite-runner better-sqlite3
|
|
7
|
-
```
|
|
8
|
-
|
|
9
|
-
If `better-sqlite3` is missing, the `import` of `@copilotkit/sqlite-runner` itself fails
|
|
10
|
-
at module load (`Cannot find module 'better-sqlite3'`). The runner's constructor has a
|
|
11
|
-
friendlier multi-line install hint as a fallback, but you will see the bare resolution
|
|
12
|
-
error first — install the peer before the runner import resolves.
|
|
13
|
-
|
|
14
|
-
## Configure
|
|
15
|
-
|
|
16
|
-
```typescript
|
|
17
|
-
import { CopilotRuntime } from "@copilotkit/runtime/v2";
|
|
18
|
-
import { SqliteAgentRunner } from "@copilotkit/sqlite-runner";
|
|
19
|
-
|
|
20
|
-
const runtime = new CopilotRuntime({
|
|
21
|
-
agents: {
|
|
22
|
-
/* ... */
|
|
23
|
-
} as any,
|
|
24
|
-
runner: new SqliteAgentRunner({
|
|
25
|
-
dbPath: "./data/threads.db", // REQUIRED — default is ":memory:"
|
|
26
|
-
}),
|
|
27
|
-
});
|
|
28
|
-
```
|
|
29
|
-
|
|
30
|
-
`dbPath: ":memory:"` is the default if omitted — that reverts to an in-memory store and
|
|
31
|
-
loses data at restart. Always set a file path in production.
|
|
32
|
-
|
|
33
|
-
## Schema
|
|
34
|
-
|
|
35
|
-
Three tables are created on first use (`packages/sqlite-runner/src/sqlite-runner.ts:75-109`):
|
|
36
|
-
|
|
37
|
-
```sql
|
|
38
|
-
CREATE TABLE IF NOT EXISTS agent_runs (
|
|
39
|
-
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
40
|
-
thread_id TEXT NOT NULL,
|
|
41
|
-
run_id TEXT NOT NULL UNIQUE,
|
|
42
|
-
parent_run_id TEXT,
|
|
43
|
-
events TEXT NOT NULL, -- JSON-encoded BaseEvent[]
|
|
44
|
-
input TEXT NOT NULL, -- JSON-encoded RunAgentInput
|
|
45
|
-
created_at INTEGER NOT NULL,
|
|
46
|
-
version INTEGER NOT NULL
|
|
47
|
-
);
|
|
48
|
-
CREATE TABLE IF NOT EXISTS run_state (
|
|
49
|
-
thread_id TEXT PRIMARY KEY,
|
|
50
|
-
is_running INTEGER DEFAULT 0,
|
|
51
|
-
current_run_id TEXT,
|
|
52
|
-
updated_at INTEGER NOT NULL
|
|
53
|
-
);
|
|
54
|
-
CREATE TABLE IF NOT EXISTS schema_version (
|
|
55
|
-
version INTEGER PRIMARY KEY,
|
|
56
|
-
applied_at INTEGER NOT NULL
|
|
57
|
-
);
|
|
58
|
-
CREATE INDEX IF NOT EXISTS idx_thread_id ON agent_runs(thread_id);
|
|
59
|
-
CREATE INDEX IF NOT EXISTS idx_parent_run_id ON agent_runs(parent_run_id);
|
|
60
|
-
```
|
|
61
|
-
|
|
62
|
-
`agent_runs` is append-only — one row per completed run, full event log in the `events`
|
|
63
|
-
column. `run_state` gates concurrent runs (the `"Thread already running"` check).
|
|
64
|
-
`schema_version` tracks applied migrations so future releases can upgrade existing
|
|
65
|
-
databases in place.
|
|
66
|
-
|
|
67
|
-
## Retention
|
|
68
|
-
|
|
69
|
-
There is no automatic retention. If you need bounded history, add a periodic purge:
|
|
70
|
-
|
|
71
|
-
```typescript
|
|
72
|
-
import Database from "better-sqlite3";
|
|
73
|
-
|
|
74
|
-
const db = new Database("./data/threads.db");
|
|
75
|
-
setInterval(
|
|
76
|
-
() => {
|
|
77
|
-
const cutoff = Date.now() - 30 * 24 * 60 * 60 * 1000; // 30 days
|
|
78
|
-
db.prepare("DELETE FROM agent_runs WHERE created_at < ?").run(cutoff);
|
|
79
|
-
},
|
|
80
|
-
60 * 60 * 1000,
|
|
81
|
-
);
|
|
82
|
-
```
|
|
83
|
-
|
|
84
|
-
## When NOT to use
|
|
85
|
-
|
|
86
|
-
- Multi-instance deploys without shared storage — each instance would have its own DB file.
|
|
87
|
-
Either put the DB on a shared volume (EFS, persistent disk) with a single writer, or
|
|
88
|
-
choose Intelligence mode or a custom Redis/Postgres runner.
|
|
89
|
-
|
|
90
|
-
Source: `packages/sqlite-runner/src/sqlite-runner.ts`.
|
|
@@ -1,336 +0,0 @@
|
|
|
1
|
-
# CopilotKit Agent Runners
|
|
2
|
-
|
|
3
|
-
`AgentRunner` is the abstraction that owns thread run state — active runs, the event stream
|
|
4
|
-
replay, and stop semantics. Pick one per `CopilotRuntime` instance.
|
|
5
|
-
|
|
6
|
-
- `InMemoryAgentRunner` — default; process-global in-memory Map; lost on restart.
|
|
7
|
-
- `SqliteAgentRunner` — file-backed; requires `better-sqlite3` peer.
|
|
8
|
-
- `IntelligenceAgentRunner` — auto-wired by `CopilotIntelligenceRuntime`. You do NOT
|
|
9
|
-
construct this directly and you cannot pass `runner` alongside `intelligence`.
|
|
10
|
-
- Custom — subclass `AgentRunner` for Redis / Postgres / any backend.
|
|
11
|
-
|
|
12
|
-
## Setup
|
|
13
|
-
|
|
14
|
-
Default (in-memory, dev only):
|
|
15
|
-
|
|
16
|
-
```typescript
|
|
17
|
-
import { CopilotRuntime } from "@copilotkit/runtime/v2";
|
|
18
|
-
|
|
19
|
-
// Equivalent to passing `runner: new InMemoryAgentRunner()`
|
|
20
|
-
const runtime = new CopilotRuntime({
|
|
21
|
-
agents: {
|
|
22
|
-
/* ... */
|
|
23
|
-
} as any,
|
|
24
|
-
});
|
|
25
|
-
```
|
|
26
|
-
|
|
27
|
-
Production (file-backed SQLite):
|
|
28
|
-
|
|
29
|
-
```typescript
|
|
30
|
-
import { CopilotRuntime } from "@copilotkit/runtime/v2";
|
|
31
|
-
import { SqliteAgentRunner } from "@copilotkit/sqlite-runner";
|
|
32
|
-
|
|
33
|
-
const runtime = new CopilotRuntime({
|
|
34
|
-
agents: {
|
|
35
|
-
/* ... */
|
|
36
|
-
} as any,
|
|
37
|
-
runner: new SqliteAgentRunner({ dbPath: "./data/threads.db" }),
|
|
38
|
-
});
|
|
39
|
-
```
|
|
40
|
-
|
|
41
|
-
Installation for the SQLite runner (the `better-sqlite3` peer is required):
|
|
42
|
-
|
|
43
|
-
```bash
|
|
44
|
-
pnpm add @copilotkit/sqlite-runner better-sqlite3
|
|
45
|
-
```
|
|
46
|
-
|
|
47
|
-
## Core Patterns
|
|
48
|
-
|
|
49
|
-
### The AgentRunner contract
|
|
50
|
-
|
|
51
|
-
```typescript
|
|
52
|
-
import { AgentRunner } from "@copilotkit/runtime/v2";
|
|
53
|
-
import type {
|
|
54
|
-
AgentRunnerRunRequest,
|
|
55
|
-
AgentRunnerConnectRequest,
|
|
56
|
-
AgentRunnerIsRunningRequest,
|
|
57
|
-
AgentRunnerStopRequest,
|
|
58
|
-
} from "@copilotkit/runtime/v2";
|
|
59
|
-
import { Observable } from "rxjs";
|
|
60
|
-
import type { BaseEvent } from "@ag-ui/client";
|
|
61
|
-
|
|
62
|
-
class MyRunner extends AgentRunner {
|
|
63
|
-
run(request: AgentRunnerRunRequest): Observable<BaseEvent> {
|
|
64
|
-
// Start a new run for request.threadId. Throw `new Error("Thread already running")`
|
|
65
|
-
// if a run is in flight. Stream events from agent.run(request.input).
|
|
66
|
-
return new Observable<BaseEvent>();
|
|
67
|
-
}
|
|
68
|
-
connect(request: AgentRunnerConnectRequest): Observable<BaseEvent> {
|
|
69
|
-
// Replay events for an active run, or historic runs for request.threadId.
|
|
70
|
-
return new Observable<BaseEvent>();
|
|
71
|
-
}
|
|
72
|
-
async isRunning(request: AgentRunnerIsRunningRequest): Promise<boolean> {
|
|
73
|
-
return false;
|
|
74
|
-
}
|
|
75
|
-
async stop(request: AgentRunnerStopRequest): Promise<boolean | undefined> {
|
|
76
|
-
return true;
|
|
77
|
-
}
|
|
78
|
-
}
|
|
79
|
-
```
|
|
80
|
-
|
|
81
|
-
### Handle double-submit on the client
|
|
82
|
-
|
|
83
|
-
By default, both `InMemoryAgentRunner` and `SqliteAgentRunner` throw
|
|
84
|
-
`"Thread already running"` on concurrent `run()` calls for the same `threadId`.
|
|
85
|
-
`"throw"` is the default, but it is not the only option: constructing
|
|
86
|
-
`InMemoryAgentRunner` with `onConcurrentRun: "supersede"` makes it abort the
|
|
87
|
-
in-flight run (the same path `stop()` takes) and start the new one instead of
|
|
88
|
-
throwing — the superseded run's partial output is discarded rather than persisted
|
|
89
|
-
to history. `SqliteAgentRunner` has no such option and always throws. When the
|
|
90
|
-
throw does happen, how it surfaces to the client depends on the runtime mode:
|
|
91
|
-
|
|
92
|
-
- **Intelligence mode** — CopilotKit Intelligence returns HTTP `409` when a lock is
|
|
93
|
-
held. The client core maps this to `CopilotKitCoreErrorCode.AGENT_THREAD_LOCKED`
|
|
94
|
-
and fires `onError({ code: "agent_thread_locked", ... })`. Handle this in
|
|
95
|
-
`<CopilotKit onError>` (the `CopilotKit` provider from `@copilotkit/react-core/v2`).
|
|
96
|
-
- **SSE mode** (default, in-memory / SQLite runners) — the runner throws
|
|
97
|
-
synchronously and the handler returns a plain `500` JSON body like
|
|
98
|
-
`{ "error": "Failed to run agent", "message": "Thread already running" }`.
|
|
99
|
-
There is no typed `agent_thread_locked` code — match on the message text or
|
|
100
|
-
just guard on the client with a busy flag.
|
|
101
|
-
|
|
102
|
-
```tsx
|
|
103
|
-
// client — Intelligence mode (typed code)
|
|
104
|
-
import { CopilotKit } from "@copilotkit/react-core/v2";
|
|
105
|
-
|
|
106
|
-
<CopilotKit
|
|
107
|
-
onError={({ code }) => {
|
|
108
|
-
if (code === "agent_thread_locked") {
|
|
109
|
-
alert("Agent is busy — wait for the current response to finish.");
|
|
110
|
-
}
|
|
111
|
-
}}
|
|
112
|
-
/>;
|
|
113
|
-
```
|
|
114
|
-
|
|
115
|
-
```tsx
|
|
116
|
-
// client — any mode: guard with a busy flag so double-submit is impossible
|
|
117
|
-
import { useAgent } from "@copilotkit/react-core/v2";
|
|
118
|
-
import { useState } from "react";
|
|
119
|
-
|
|
120
|
-
function Composer() {
|
|
121
|
-
const agent = useAgent({ agentId: "default" });
|
|
122
|
-
const [busy, setBusy] = useState(false);
|
|
123
|
-
|
|
124
|
-
async function send(text: string) {
|
|
125
|
-
if (busy) return;
|
|
126
|
-
setBusy(true);
|
|
127
|
-
try {
|
|
128
|
-
await agent?.addMessage({ role: "user", content: text });
|
|
129
|
-
} finally {
|
|
130
|
-
setBusy(false);
|
|
131
|
-
}
|
|
132
|
-
}
|
|
133
|
-
|
|
134
|
-
return null;
|
|
135
|
-
}
|
|
136
|
-
```
|
|
137
|
-
|
|
138
|
-
## Common Mistakes
|
|
139
|
-
|
|
140
|
-
### HIGH Shipping InMemoryAgentRunner to production
|
|
141
|
-
|
|
142
|
-
Wrong:
|
|
143
|
-
|
|
144
|
-
```typescript
|
|
145
|
-
// production:
|
|
146
|
-
new CopilotRuntime({ agents: { default: agent } });
|
|
147
|
-
```
|
|
148
|
-
|
|
149
|
-
Correct:
|
|
150
|
-
|
|
151
|
-
```typescript
|
|
152
|
-
import { SqliteAgentRunner } from "@copilotkit/sqlite-runner";
|
|
153
|
-
|
|
154
|
-
new CopilotRuntime({
|
|
155
|
-
agents: { default: agent },
|
|
156
|
-
runner: new SqliteAgentRunner({ dbPath: "./data/threads.db" }),
|
|
157
|
-
});
|
|
158
|
-
// Or upgrade to Intelligence mode for managed durability.
|
|
159
|
-
```
|
|
160
|
-
|
|
161
|
-
The default runner is `new InMemoryAgentRunner()`. It keeps state in a process-global,
|
|
162
|
-
bounded store — threads are lost on restart, evicted past the memory limits, and
|
|
163
|
-
horizontally-scaled instances see divergent state. See `agent-runners-in-memory.md`
|
|
164
|
-
for the bounds and how to tune them.
|
|
165
|
-
|
|
166
|
-
Source: `packages/runtime/src/v2/runtime/runner/in-memory.ts`.
|
|
167
|
-
|
|
168
|
-
### HIGH Setting runner alongside intelligence option
|
|
169
|
-
|
|
170
|
-
Wrong:
|
|
171
|
-
|
|
172
|
-
```typescript
|
|
173
|
-
new CopilotRuntime({
|
|
174
|
-
agents,
|
|
175
|
-
intelligence,
|
|
176
|
-
runner: new SqliteAgentRunner({ dbPath: "./data/threads.db" }),
|
|
177
|
-
});
|
|
178
|
-
```
|
|
179
|
-
|
|
180
|
-
Correct:
|
|
181
|
-
|
|
182
|
-
```typescript
|
|
183
|
-
new CopilotRuntime({
|
|
184
|
-
agents,
|
|
185
|
-
intelligence,
|
|
186
|
-
identifyUser: (req) => ({
|
|
187
|
-
id: req.headers.get("x-user-id")!,
|
|
188
|
-
name: req.headers.get("x-user-name") ?? "Anonymous",
|
|
189
|
-
}),
|
|
190
|
-
});
|
|
191
|
-
```
|
|
192
|
-
|
|
193
|
-
`CopilotIntelligenceRuntimeOptions` does not declare a `runner` field — Intelligence mode
|
|
194
|
-
auto-wires `IntelligenceAgentRunner` pointed at the Intelligence service socket. Excess-property checks will
|
|
195
|
-
flag a `runner:` key on an Intelligence-shaped options object as a type error, and a caller who
|
|
196
|
-
evades that check (JS, `as any`, or a non-literal options object) gets a `throw` at construction
|
|
197
|
-
rather than a silently ignored runner.
|
|
198
|
-
|
|
199
|
-
Source: `packages/runtime/src/v2/runtime/core/runtime.ts` — `runner?` is declared only on
|
|
200
|
-
`CopilotSseRuntimeOptions` (:239); the Intelligence constructor guard is at :512 and the
|
|
201
|
-
auto-wired runner at :582.
|
|
202
|
-
|
|
203
|
-
### HIGH Forgetting the better-sqlite3 peer
|
|
204
|
-
|
|
205
|
-
Wrong:
|
|
206
|
-
|
|
207
|
-
```bash
|
|
208
|
-
pnpm add @copilotkit/sqlite-runner
|
|
209
|
-
```
|
|
210
|
-
|
|
211
|
-
Correct:
|
|
212
|
-
|
|
213
|
-
```bash
|
|
214
|
-
pnpm add @copilotkit/sqlite-runner better-sqlite3
|
|
215
|
-
```
|
|
216
|
-
|
|
217
|
-
`@copilotkit/sqlite-runner` imports `better-sqlite3` at the top of its module, so if the peer
|
|
218
|
-
is missing, `import { SqliteAgentRunner } from "@copilotkit/sqlite-runner"` itself fails at
|
|
219
|
-
module load with `Cannot find module 'better-sqlite3'` — long before the constructor runs.
|
|
220
|
-
(The constructor has a friendlier multi-line install hint as a belt-and-suspenders fallback,
|
|
221
|
-
but in practice you will see the bare module-resolution error first.) It is a peer dependency,
|
|
222
|
-
not a direct dep.
|
|
223
|
-
|
|
224
|
-
Source: `packages/sqlite-runner/src/sqlite-runner.ts:18`, `:55-66`.
|
|
225
|
-
|
|
226
|
-
### HIGH Default SqliteAgentRunner with :memory: dbPath
|
|
227
|
-
|
|
228
|
-
Wrong:
|
|
229
|
-
|
|
230
|
-
```typescript
|
|
231
|
-
new SqliteAgentRunner();
|
|
232
|
-
```
|
|
233
|
-
|
|
234
|
-
Correct:
|
|
235
|
-
|
|
236
|
-
```typescript
|
|
237
|
-
new SqliteAgentRunner({ dbPath: "./data/threads.db" });
|
|
238
|
-
```
|
|
239
|
-
|
|
240
|
-
The default `dbPath` is `":memory:"` — SQLite's in-memory mode. Data is lost at restart,
|
|
241
|
-
defeating the reason to use the file-backed runner.
|
|
242
|
-
|
|
243
|
-
Source: `packages/sqlite-runner/src/sqlite-runner.ts:48-54`.
|
|
244
|
-
|
|
245
|
-
### MEDIUM Concurrent run() on the same threadId
|
|
246
|
-
|
|
247
|
-
Wrong:
|
|
248
|
-
|
|
249
|
-
```tsx
|
|
250
|
-
// Double-click send button → two POST /agent/:id/run to the same thread
|
|
251
|
-
<button onClick={() => agent.addMessage({ role: "user", content })}>
|
|
252
|
-
Send
|
|
253
|
-
</button>
|
|
254
|
-
```
|
|
255
|
-
|
|
256
|
-
Correct:
|
|
257
|
-
|
|
258
|
-
```tsx
|
|
259
|
-
const [busy, setBusy] = useState(false);
|
|
260
|
-
<button
|
|
261
|
-
disabled={busy}
|
|
262
|
-
onClick={async () => {
|
|
263
|
-
setBusy(true);
|
|
264
|
-
try {
|
|
265
|
-
await agent.addMessage({ role: "user", content });
|
|
266
|
-
} finally {
|
|
267
|
-
setBusy(false);
|
|
268
|
-
}
|
|
269
|
-
}}
|
|
270
|
-
>
|
|
271
|
-
Send
|
|
272
|
-
</button>;
|
|
273
|
-
```
|
|
274
|
-
|
|
275
|
-
By default both runners throw `"Thread already running"` on concurrent runs, so
|
|
276
|
-
debouncing on the client is still the right baseline. In Intelligence mode you can
|
|
277
|
-
additionally handle `code === "agent_thread_locked"` in `<CopilotKit onError>`; SSE
|
|
278
|
-
mode surfaces only a generic 500 with that message.
|
|
279
|
-
|
|
280
|
-
Throwing is the default (`onConcurrentRun: "throw"`), not the only behavior:
|
|
281
|
-
constructing `InMemoryAgentRunner` with `onConcurrentRun: "supersede"` aborts the
|
|
282
|
-
in-flight run (the `stop()` path) and starts the new one instead of throwing,
|
|
283
|
-
discarding the superseded run's partial output rather than persisting it. That
|
|
284
|
-
suits a UX where a fast follow-up should displace a still-running (or wedged) turn.
|
|
285
|
-
Unlike the process-global memory limits, `onConcurrentRun` is per-runner-instance —
|
|
286
|
-
it affects only the runner you pass it to. `SqliteAgentRunner` has no such option
|
|
287
|
-
and always throws.
|
|
288
|
-
|
|
289
|
-
Source: the `throw new Error("Thread already running")` in `InMemoryAgentRunner.run()`,
|
|
290
|
-
`packages/runtime/src/v2/runtime/runner/in-memory.ts`;
|
|
291
|
-
`packages/core/src/intelligence-agent.ts:368-369`.
|
|
292
|
-
|
|
293
|
-
### HIGH In-memory runner + horizontal scaling
|
|
294
|
-
|
|
295
|
-
Wrong:
|
|
296
|
-
|
|
297
|
-
```typescript
|
|
298
|
-
// 3 Fly.io / Cloud Run instances, each with its own InMemoryAgentRunner
|
|
299
|
-
new CopilotRuntime({ agents });
|
|
300
|
-
```
|
|
301
|
-
|
|
302
|
-
Correct:
|
|
303
|
-
|
|
304
|
-
```typescript
|
|
305
|
-
// Sticky-session one instance per thread (so every run for a thread lands on the
|
|
306
|
-
// same process), OR move to Intelligence mode for managed multi-instance durability.
|
|
307
|
-
new CopilotRuntime({ agents }); // + route by threadId at the load balancer
|
|
308
|
-
```
|
|
309
|
-
|
|
310
|
-
`InMemoryAgentRunner`'s store is a process-global singleton — multi-instance deploys see
|
|
311
|
-
totally different thread state per worker, making reconnects and `GET /connect` non-deterministic.
|
|
312
|
-
|
|
313
|
-
Source: the exported `ɵGLOBAL_STORE` singleton in `packages/runtime/src/v2/runtime/runner/in-memory.ts`.
|
|
314
|
-
|
|
315
|
-
A shared `dbPath` on `SqliteAgentRunner` is **not** a horizontal-scaling fix on its own.
|
|
316
|
-
Sharing the file gives you durable, persisted history: runs survive process restarts, and
|
|
317
|
-
completed runs are readable from any instance pointed at the same file. But the live-run
|
|
318
|
-
bookkeeping used by the connect-bridge and by `stop()` lives in a process-local
|
|
319
|
-
`ACTIVE_CONNECTIONS` map. A second instance has **no** entry for a run started elsewhere, so
|
|
320
|
-
it can replay stored history but **cannot** reconnect to — or stop — an in-flight run on
|
|
321
|
-
another instance. Use `SqliteAgentRunner` for restart-resilient single-instance durability;
|
|
322
|
-
for managed multi-instance durability, use Intelligence mode.
|
|
323
|
-
|
|
324
|
-
Source: `packages/sqlite-runner/src/sqlite-runner.ts:46` (module-level `ACTIVE_CONNECTIONS`).
|
|
325
|
-
|
|
326
|
-
## References
|
|
327
|
-
|
|
328
|
-
- [InMemoryAgentRunner — store, bounds, concurrency, and lifecycle](agent-runners-in-memory.md)
|
|
329
|
-
- [SqliteAgentRunner — schema, retention, ops](agent-runners-sqlite.md)
|
|
330
|
-
- [Custom runner — Redis/Postgres skeleton](agent-runners-custom.md)
|
|
331
|
-
|
|
332
|
-
## See also
|
|
333
|
-
|
|
334
|
-
- `copilotkit/intelligence-mode` — managed durability alternative (CopilotKit Intelligence managed service, not self-hostable)
|
|
335
|
-
- `copilotkit/setup-endpoint` — runner is passed via the CopilotRuntime constructor
|
|
336
|
-
- `copilotkit/scale-to-multi-agent` — horizontal scaling considerations
|