@copilotkit/runtime 1.57.3 → 1.58.0-canary.thread-id-propagation

package/skills/runtime/references/intelligence-mode.md

@@ -0,0 +1,336 @@
+# CopilotKit Intelligence Mode
+
+Intelligence currently ships as a managed cloud service. The only supported `apiUrl` /
+`wsUrl` today is the CopilotKit-managed cloud Intelligence instance — the `ɵ`-prefixed
+runtime internals and REST/WebSocket contract that back Intelligence are still
+stabilizing and `organizationId` is reserved for future self-hosted deployments. If you
+need on-prem durable threads today, use SSE mode with a persistent runner
+(`SqliteAgentRunner` or a custom one) instead.
+
+Obtain `apiKey` and `organizationId` from the CopilotKit Cloud dashboard.
+
+### URL format
+
+The client prepends `/api/...` and the Intelligence websocket layer derives `/runner`
+or `/client` suffixes internally. Pass the bare base URLs — do NOT append `/api`,
+`/socket`, `/runner`, or `/client` yourself:
+
+```typescript
+// Correct — bare base URLs
+apiUrl: "https://api.copilotkit.ai",
+wsUrl:  "wss://api.copilotkit.ai",
+
+// Wrong — adding /api produces /api/api/... on every REST call; /socket/runner is not a real path
+apiUrl: "https://api.copilotkit.ai/api",
+wsUrl:  "wss://api.copilotkit.ai/socket",
+```
+
+Source: `packages/runtime/src/v2/runtime/intelligence-platform/client.ts:41-46, 259,
+356-357, 437, 468, 682-708`.
+
+## Setup
+
+```typescript
+import {
+  CopilotRuntime,
+  CopilotKitIntelligence,
+  createCopilotRuntimeHandler,
+} from "@copilotkit/runtime/v2";
+
+const intelligence = new CopilotKitIntelligence({
+  apiUrl: "https://api.copilotkit.ai",
+  wsUrl: "wss://api.copilotkit.ai",
+  apiKey: process.env.COPILOTKIT_CLOUD_API_KEY!,
+  organizationId: process.env.COPILOTKIT_CLOUD_ORG_ID!,
+});
+
+const runtime = new CopilotRuntime({
+  agents: {
+    /* ... */
+  } as any,
+  intelligence,
+  identifyUser: (request) => ({
+    id: request.headers.get("x-user-id") ?? "anonymous",
+  }),
+  // Optional tuning:
+  generateThreadNames: true, // default true — 1 LLM call per new thread
+  lockTtlSeconds: 20, // clamped to ≤ 3600
+  lockHeartbeatIntervalSeconds: 15, // clamped to ≤ 3000
+});
+
+const handler = createCopilotRuntimeHandler({
+  runtime,
+  basePath: "/api/copilotkit",
+});
+
+export default { fetch: handler };
+```
+
+When `intelligence` is set, the runtime auto-wires `IntelligenceAgentRunner` internally.
+Do NOT pass `runner` — see the failure-modes section.
+
+## Core Patterns
+
+### Identify the user from an auth cookie
+
+`identifyUser` is for user identification only — it does NOT forward thrown `Response`s.
+`resolveIntelligenceUser` (`handlers/shared/resolve-intelligence-user.ts:14-24`) wraps the
+call in try/catch and converts any thrown value (including `Response`) into a generic
+`errorResponse("Failed to identify user", 500)`. Gate auth in `hooks.onRequest`
+(see the `middleware` skill) and keep `identifyUser` focused on returning an id:
+
+```typescript
+import {
+  CopilotRuntime,
+  createCopilotRuntimeHandler,
+} from "@copilotkit/runtime/v2";
+import { parse } from "cookie";
+
+const runtime = new CopilotRuntime({
+  agents,
+  intelligence,
+  // identifyUser returns the id; auth rejection is hooked elsewhere.
+  identifyUser: async (request) => {
+    const cookies = parse(request.headers.get("cookie") ?? "");
+    const user = await resolveSession(cookies["session"]); // your auth lib
+    return { id: user?.id ?? "anonymous" };
+  },
+});
+
+const handler = createCopilotRuntimeHandler({
+  runtime,
+  basePath: "/api/copilotkit",
+  hooks: {
+    onRequest: async ({ request }) => {
+      const cookies = parse(request.headers.get("cookie") ?? "");
+      const user = await resolveSession(cookies["session"]);
+      // onRequest DOES forward thrown Responses — use it for auth rejection.
+      if (!user) throw new Response("Unauthorized", { status: 401 });
+    },
+  },
+});
+
+async function resolveSession(token: string | undefined) {
+  if (!token) return null;
+  return { id: "user-123" };
+}
+```
+
+### Disable thread-name generation to avoid a per-thread LLM call
+
+```typescript
+new CopilotRuntime({
+  agents,
+  intelligence,
+  identifyUser: (req) => ({ id: req.headers.get("x-user-id")! }),
+  generateThreadNames: false,
+});
+```
+
+### Frontend — no config change
+
+The frontend reads `GET /info` on mount. When the runtime reports `mode: "intelligence"`
+and an `intelligence.wsUrl`, `CopilotKitCore` auto-switches from SSE to the websocket
+transport. The React integration just points at the runtime URL:
+
+```tsx
+import { CopilotKitProvider } from "@copilotkit/react-core/v2";
+
+export function App({ children }: { children: React.ReactNode }) {
+  return (
+    <CopilotKitProvider runtimeUrl="/api/copilotkit">
+      {children}
+    </CopilotKitProvider>
+  );
+}
+```
+
+## Common Mistakes
+
+### CRITICAL Missing identifyUser
+
+Wrong:
+
+```typescript
+new CopilotRuntime({ agents, intelligence });
+```
+
+Correct:
+
+```typescript
+new CopilotRuntime({
+  agents,
+  intelligence,
+  identifyUser: (req) => ({ id: req.headers.get("x-user-id")! }),
+});
+```
+
+`identifyUser` is required on `CopilotIntelligenceRuntimeOptions` — omitting it is a
+TypeScript error and (if suppressed) crashes handlers at request time. Every thread is
+scoped to a user ID.
+
+Source: `packages/runtime/src/v2/runtime/core/runtime.ts:156-160`.
+
+### CRITICAL Adding /api or /socket suffixes, or pointing at an unsupported self-hosted server
+
+Wrong:
+
+```typescript
+new CopilotKitIntelligence({
+  apiUrl: "https://api.copilotkit.ai/api", // double /api prefix
+  wsUrl: "wss://api.copilotkit.ai/socket", // /socket is not a real path
+  apiKey,
+  organizationId,
+});
+
+new CopilotKitIntelligence({
+  apiUrl: "https://internal.myco.com/intelligence", // self-hosting is not yet supported
+  wsUrl: "wss://internal.myco.com/intelligence",
+  apiKey,
+  organizationId,
+});
+```
+
+Correct:
+
+```typescript
+new CopilotKitIntelligence({
+  apiUrl: "https://api.copilotkit.ai",
+  wsUrl: "wss://api.copilotkit.ai",
+  apiKey: process.env.COPILOTKIT_CLOUD_API_KEY!,
+  organizationId: process.env.COPILOTKIT_CLOUD_ORG_ID!,
+});
+// For on-prem durability without Intelligence: SSE mode + SqliteAgentRunner.
+```
+
+Two failure modes to avoid:
+
+1. The client prepends `/api/...` to every REST call (`#request` at line 356-357) and
+   the websocket layer derives `/runner` / `/client` suffixes from `wsUrl` internally.
+   Passing `apiUrl: ".../api"` produces double-prefixed `/api/api/threads`; passing
+   `wsUrl: ".../socket"` produces a broken `.../socket/runner` upgrade path.
+2. Self-hosting Intelligence is not yet supported. The `ɵ`-prefixed runtime internals
+   and REST/WebSocket contract are still stabilizing. `organizationId` is reserved for
+   future self-hosted instances. For on-prem durable threads today, use SSE mode +
+   `SqliteAgentRunner` (see `copilotkit/agent-runners`).
+
+Source: `packages/runtime/src/v2/runtime/intelligence-platform/client.ts:41-46, 68-69,
+259, 356-357, 437, 682-708`.
+
+### HIGH Setting runner alongside intelligence
+
+Wrong:
+
+```typescript
+import { SqliteAgentRunner } from "@copilotkit/sqlite-runner";
+
+new CopilotRuntime({
+  agents,
+  intelligence,
+  runner: new SqliteAgentRunner({ dbPath: "./threads.db" }),
+});
+```
+
+Correct:
+
+```typescript
+new CopilotRuntime({
+  agents,
+  intelligence,
+  identifyUser,
+});
+```
+
+`CopilotIntelligenceRuntimeOptions` excludes `runner` at the type level. Intelligence
+forces its own `IntelligenceAgentRunner` tied to the Cloud WebSocket; a user-supplied
+runner is rejected.
+
+Source: `packages/runtime/src/v2/runtime/core/runtime.ts:149-173,285-294`.
+
+### HIGH Calling /threads against an SSE-mode runtime
+
+Wrong:
+
+```typescript
+// SSE-only runtime (no `intelligence` configured)
+await fetch("/api/copilotkit/threads");
+```
+
+Correct:
+
+```typescript
+// Enable Intelligence mode first, OR don't call thread routes.
+// Client-side, the useThreads hook errors with "Runtime URL is not configured" when
+// the runtime isn't in Intelligence mode.
+```
+
+The `/threads`, `/threads/subscribe`, `PATCH /threads/:id`, `POST /threads/:id/archive`,
+`DELETE /threads/:id`, and `/threads/:id/messages` routes always resolve in the router,
+but the handlers call `requireIntelligenceRuntime(runtime)` first and return HTTP 422
+("Missing CopilotKitIntelligence configuration. Thread operations require a
+CopilotKitIntelligence instance to be provided in CopilotRuntime options.") when the
+runtime isn't an `IntelligenceRuntime`.
+
+Source: `packages/runtime/src/v2/runtime/handlers/intelligence/threads.ts:37-48`;
+route table in `dev-docs/architecture/setup-intelligence.md:179-183`.
+
+### LOW Over-clamping lockTtlSeconds
+
+Wrong:
+
+```typescript
+new CopilotRuntime({
+  agents,
+  intelligence,
+  identifyUser,
+  lockTtlSeconds: 86400, // "I want 1-day lock"
+});
+```
+
+Correct:
+
+```typescript
+new CopilotRuntime({
+  agents,
+  intelligence,
+  identifyUser,
+  lockTtlSeconds: 3600, // max is 1 hour
+});
+// Rethink long-running workflows if 1 hour is insufficient.
+```
+
+`lockTtlSeconds` is silently `Math.min(value, 3600)`; `lockHeartbeatIntervalSeconds` is
+`Math.min(value, 3000)`. Requests over the cap are clamped without warning.
+
+Source: `packages/runtime/src/v2/runtime/core/runtime.ts:281-307`.
+
+### MEDIUM generateThreadNames unset expecting no LLM cost
+
+Wrong:
+
+```typescript
+new CopilotRuntime({ agents, intelligence, identifyUser });
+// assumes no extra LLM spend
+```
+
+Correct:
+
+```typescript
+new CopilotRuntime({
+  agents,
+  intelligence,
+  identifyUser,
+  generateThreadNames: false,
+});
+```
+
+`generateThreadNames` defaults to `true`. Every newly created thread triggers an extra
+LLM call on the Cloud side to generate a short name, billed against your Cloud quota.
+
+Source: `packages/runtime/src/v2/runtime/core/runtime.ts` (generateThreadNames default).
+
+## See also
+
+- `copilotkit/agent-runners` — Intelligence forces `IntelligenceAgentRunner`
+- `copilotkit/setup-endpoint` — `/threads/*` routes flip on with Intelligence
+- `copilotkit/threads` (react-core) — `useThreads` depends on Intelligence routes

package/skills/runtime/references/middleware.md

@@ -0,0 +1,376 @@
+# CopilotKit Runtime Middleware
+
+Two coexisting middleware surfaces:
+
+- **`hooks`** (preferred, newer) — pass to `createCopilotRuntimeHandler({ hooks })`.
+  Route-aware via `onBeforeHandler({ route })`. Throw a `Response` to short-circuit.
+- **`beforeRequestMiddleware` / `afterRequestMiddleware`** (legacy) — pass to
+  `new CopilotRuntime({ ... })`. Runs **after `hooks.onRequest` but before routing** (see
+  `fetch-handler.ts:136-147` for exact order). Pre-routing only.
+
+Use **hooks** for new code.
+
+## Setup
+
+```typescript
+import {
+  CopilotRuntime,
+  createCopilotRuntimeHandler,
+} from "@copilotkit/runtime/v2";
+
+const runtime = new CopilotRuntime({
+  agents: {
+    /* ... */
+  } as any,
+});
+
+const handler = createCopilotRuntimeHandler({
+  runtime,
+  basePath: "/api/copilotkit",
+  hooks: {
+    onRequest: async ({ request }) => {
+      const token = request.headers.get("authorization");
+      if (!token) throw new Response("Unauthorized", { status: 401 });
+    },
+    onBeforeHandler: async ({ route, request }) => {
+      if (route.method === "agent/run" && route.agentId === "admin") {
+        const user = await verifyAdminToken(
+          request.headers.get("authorization"),
+        );
+        if (!user) throw new Response("Forbidden", { status: 403 });
+      }
+    },
+    onResponse: async ({ response }) => {
+      const headers = new Headers(response.headers);
+      headers.set("x-copilot-version", "2.0");
+      return new Response(response.body, {
+        status: response.status,
+        statusText: response.statusText,
+        headers,
+      });
+    },
+    onError: async ({ error, route }) => {
+      console.error("[copilotkit]", route?.method, error);
+    },
+  },
+});
+
+async function verifyAdminToken(
+  header: string | null,
+): Promise<{ id: string } | null> {
+  if (!header) return null;
+  // delegate to your auth lib
+  return { id: "admin" };
+}
+
+export default { fetch: handler };
+```
+
+## Core Patterns
+
+### Reject unauthenticated requests at the runtime boundary
+
+```typescript
+createCopilotRuntimeHandler({
+  runtime,
+  basePath: "/api/copilotkit",
+  hooks: {
+    onRequest: ({ request }) => {
+      const token = request.headers.get("authorization");
+      if (!token?.startsWith("Bearer ")) {
+        throw new Response(JSON.stringify({ error: "unauthorized" }), {
+          status: 401,
+          headers: { "content-type": "application/json" },
+        });
+      }
+    },
+  },
+});
+```
+
+### Route-aware authorization
+
+Use `onBeforeHandler` — the `route` object carries `method`, `agentId`, and (for thread/stop
+methods) `threadId`.
+
+```typescript
+createCopilotRuntimeHandler({
+  runtime,
+  basePath: "/api/copilotkit",
+  hooks: {
+    onBeforeHandler: async ({ route, request }) => {
+      if (route.method === "agent/run" && route.agentId === "billing") {
+        const ok = await canAccessBilling(request);
+        if (!ok) throw new Response("Forbidden", { status: 403 });
+      }
+    },
+  },
+});
+
+async function canAccessBilling(request: Request): Promise<boolean> {
+  // delegate to your policy engine
+  return true;
+}
+```
+
+### Rate-limit by calling an external limiter from the hook
+
+Delegate to a dedicated lib — do not implement a rate limiter inline.
+
+```typescript
+import { Ratelimit } from "@upstash/ratelimit";
+import { Redis } from "@upstash/redis";
+
+const ratelimit = new Ratelimit({
+  redis: Redis.fromEnv(),
+  limiter: Ratelimit.slidingWindow(60, "1 m"),
+});
+
+createCopilotRuntimeHandler({
+  runtime,
+  basePath: "/api/copilotkit",
+  hooks: {
+    onRequest: async ({ request }) => {
+      const userId = request.headers.get("x-user-id") ?? "anon";
+      const { success } = await ratelimit.limit(userId);
+      if (!success) throw new Response("Too Many Requests", { status: 429 });
+    },
+  },
+});
+```
+
+### Non-blocking telemetry on response
+
+`afterRequestMiddleware` runs non-blocking (errors inside only log). Do not await heavy
+work that the user's response waits on.
+
+```typescript
+import { CopilotRuntime } from "@copilotkit/runtime/v2";
+
+const runtime = new CopilotRuntime({
+  agents: {
+    /* ... */
+  } as any,
+  afterRequestMiddleware: async ({ threadId, messages }) => {
+    // fire-and-forget; do not await heavy work that blocks response
+    void queue.enqueue({ type: "chat", threadId, messages });
+  },
+});
+```
+
+## Common Mistakes
+
+### HIGH Returning a Response instead of throwing
+
+Wrong:
+
+```typescript
+new CopilotRuntime({
+  agents,
+  beforeRequestMiddleware: async () =>
+    new Response("Unauthorized", { status: 401 }),
+});
+```
+
+Correct:
+
+```typescript
+new CopilotRuntime({
+  agents,
+  beforeRequestMiddleware: async ({ request }) => {
+    if (!request.headers.get("authorization")) {
+      throw new Response("Unauthorized", { status: 401 });
+    }
+  },
+});
+```
+
+The middleware contract returns `Request | void`. Returning a Response corrupts the
+request object — `fetch-handler.ts:140-147` assigns any truthy return value back to
+`request`, so the router then tries to read `request.method` / `request.headers.get(...)`
+from the Response and downstream handling blows up. Always `throw` a Response to
+short-circuit; never return one.
+
+Source: `packages/runtime/src/v2/runtime/core/fetch-handler.ts:140-156`.
+
+### MEDIUM Defaulting to beforeRequestMiddleware when hooks are preferred
+
+Wrong:
+
+```typescript
+new CopilotRuntime({
+  agents,
+  beforeRequestMiddleware: async ({ request, path }) => {
+    if (path.includes("/agent/admin/")) {
+      /* check admin auth */
+    }
+  },
+});
+```
+
+Correct:
+
+```typescript
+const runtime = new CopilotRuntime({ agents });
+const handler = createCopilotRuntimeHandler({
+  runtime,
+  basePath: "/api/copilotkit",
+  hooks: {
+    onBeforeHandler: ({ route, request }) => {
+      if (route.method === "agent/run" && route.agentId === "admin") {
+        /* ... */
+      }
+    },
+  },
+});
+```
+
+Both surfaces coexist. For new code the hook API on `createCopilotRuntimeHandler` is
+preferred — `onBeforeHandler` receives typed `route` info, so you don't string-match paths.
+
+Source: `packages/runtime/src/v2/runtime/core/hooks.ts:84-117`; maintainer Phase 4c.
+
+### MEDIUM Route-specific auth in global beforeRequestMiddleware
+
+Wrong:
+
+```typescript
+new CopilotRuntime({
+  agents,
+  beforeRequestMiddleware: async ({ path, request }) => {
+    if (path.includes("/agent/admin/")) {
+      /* ... */
+    }
+  },
+});
+```
+
+Correct:
+
+```typescript
+createCopilotRuntimeHandler({
+  runtime,
+  basePath: "/api/copilotkit",
+  hooks: {
+    onBeforeHandler: ({ route, request }) => {
+      if (route.method === "agent/run" && route.agentId === "admin") {
+        /* ... */
+      }
+    },
+  },
+});
+```
+
+`beforeRequestMiddleware` fires before routing, so no route info exists yet — string-matching
+paths is fragile. `onBeforeHandler` fires after routing with typed `route.method`, `route.agentId`.
+
+Source: `packages/runtime/src/v2/runtime/core/hooks.ts:94-103`.
+
+### MEDIUM Blocking on afterRequestMiddleware
+
+Wrong:
+
+```typescript
+new CopilotRuntime({
+  agents,
+  afterRequestMiddleware: async ({ response, threadId, messages }) => {
+    await heavyAnalytics(response, threadId, messages);
+  },
+});
+```
+
+Correct:
+
+```typescript
+new CopilotRuntime({
+  agents,
+  afterRequestMiddleware: async ({ response, threadId, messages }) => {
+    void queue.enqueue({ type: "chat", threadId, messages, response });
+  },
+});
+```
+
+The `afterRequestMiddleware` callback receives
+`{ runtime, response, path, messages?, threadId?, runId? }` — all these fields are always
+available (`messages`/`threadId`/`runId` are populated from the SSE stream when present,
+undefined otherwise). The hook runs non-blocking via `.catch()` so errors only log and any
+heavy awaited work can be lost on process exit — fire-and-forget is the intended shape.
+
+Source: `packages/runtime/src/v2/runtime/core/fetch-handler.ts:225-234`.
+
+### MEDIUM Passing a webhook URL string as middleware
+
+Wrong:
+
+```typescript
+new CopilotRuntime({
+  agents,
+  beforeRequestMiddleware: "https://hooks.example/auth" as any,
+});
+```
+
+Correct:
+
+```typescript
+new CopilotRuntime({
+  agents,
+  beforeRequestMiddleware: async ({ request }) => {
+    await fetch("https://hooks.example/auth", {
+      method: "POST",
+      body: request.headers.get("authorization") ?? "",
+    });
+  },
+});
+```
+
+Webhook-URL middleware is dead code in v2 — the runtime logs
+`"Unsupported beforeRequestMiddleware value – skipped"` and does nothing. Only function
+middleware is wired.
+
+Source: `packages/runtime/src/v2/runtime/core/middleware.ts:72-87`.
+
+### HIGH Implementing auth / rate-limit inside CopilotKit middleware
+
+Wrong:
+
+```typescript
+new CopilotRuntime({
+  agents,
+  beforeRequestMiddleware: async ({ request }) => {
+    // hand-rolling a token-bucket rate limiter inline with Redis calls...
+  },
+});
+```
+
+Correct:
+
+```typescript
+import { Ratelimit } from "@upstash/ratelimit";
+import { Redis } from "@upstash/redis";
+
+const ratelimit = new Ratelimit({
+  redis: Redis.fromEnv(),
+  limiter: Ratelimit.slidingWindow(60, "1 m"),
+});
+
+new CopilotRuntime({
+  agents,
+  beforeRequestMiddleware: async ({ request }) => {
+    const { success } = await ratelimit.limit(
+      request.headers.get("x-user-id") ?? "anon",
+    );
+    if (!success) throw new Response("Too Many Requests", { status: 429 });
+  },
+});
+```
+
+Auth, rate-limiting, and observability are server-framework concerns. CopilotKit middleware
+is the hook to invoke them, not a replacement.
+
+Source: maintainer interview (Phase 2c).
+
+## See also
+
+- `copilotkit/setup-endpoint` — `hooks` are passed to `createCopilotRuntimeHandler`
+- `copilotkit/go-to-production` — production checklist lists auth/rate-limit wiring
+- `copilotkit/debug-and-troubleshoot` — `onError` telemetry pattern