@copilotkit/runtime 1.71.0 → 1.71.1

package/skills/runtime/references/transcription.md

@@ -1,287 +0,0 @@
-# CopilotKit Transcription
-
-Subclass `TranscriptionService`, pass an instance to `CopilotRuntime({ transcriptionService })`,
-and the `POST /transcribe` endpoint lights up. The service has a single method,
-`transcribeFile`, that returns the transcript as a plain string.
-
-## Setup
-
-```typescript
-import {
-  CopilotRuntime,
-  createCopilotRuntimeHandler,
-  TranscriptionService,
-  type TranscribeFileOptions,
-} from "@copilotkit/runtime/v2";
-import OpenAI from "openai";
-
-class OpenAIWhisperTranscription extends TranscriptionService {
-  private client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
-
-  async transcribeFile({ audioFile }: TranscribeFileOptions): Promise<string> {
-    const result = await this.client.audio.transcriptions.create({
-      file: audioFile,
-      model: "whisper-1",
-    });
-    return result.text;
-  }
-}
-
-const runtime = new CopilotRuntime({
-  agents: {
-    /* ... */
-  } as any,
-  transcriptionService: new OpenAIWhisperTranscription(),
-});
-
-const handler = createCopilotRuntimeHandler({
-  runtime,
-  basePath: "/api/copilotkit",
-});
-
-export default { fetch: handler };
-```
-
-## Core Patterns
-
-### Abstract contract
-
-```typescript
-// packages/runtime/src/v2/runtime/transcription-service/transcription-service.ts
-export interface TranscribeFileOptions {
-  audioFile: File;
-  mimeType?: string;
-  size?: number;
-}
-
-export abstract class TranscriptionService {
-  abstract transcribeFile(options: TranscribeFileOptions): Promise<string>;
-}
-```
-
-### Supported request shapes
-
-Multipart (REST mode):
-
-```typescript
-const form = new FormData();
-form.append("audio", blob, "recording.webm");
-await fetch("/api/copilotkit/transcribe", { method: "POST", body: form });
-```
-
-JSON (works in both multi-route and single-endpoint modes — dispatch is by
-`Content-Type: application/json`; `mimeType` is required in the payload):
-
-```typescript
-await fetch("/api/copilotkit/transcribe", {
-  method: "POST",
-  headers: { "Content-Type": "application/json" },
-  body: JSON.stringify({
-    audio: base64String,
-    mimeType: "audio/webm",
-    filename: "recording.webm", // optional
-  }),
-});
-```
-
-### Reject oversize audio with a graceful 400
-
-```typescript
-class OpenAIWhisperTranscription extends TranscriptionService {
-  private client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
-
-  async transcribeFile({
-    audioFile,
-    size,
-  }: TranscribeFileOptions): Promise<string> {
-    const max = 25 * 1024 * 1024; // 25 MB
-    if ((size ?? audioFile.size) > max) {
-      // "too long" keyword → audio_too_long response
-      throw new Error("Audio duration too long — max 25MB per upload");
-    }
-    const result = await this.client.audio.transcriptions.create({
-      file: audioFile,
-      model: "whisper-1",
-    });
-    return result.text;
-  }
-}
-```
-
-### Error auto-categorization
-
-The runtime inspects `String(error).toLowerCase()` thrown by your service and maps keywords
-to error codes. Let the provider error bubble up — do not re-categorize inside the service.
-
-| Keyword substrings                       | Maps to                          |
-| ---------------------------------------- | -------------------------------- |
-| `rate`, `429`, `too many`                | `rate_limited` (retryable)       |
-| `auth`, `401`, `api key`, `unauthorized` | `auth_failed` (not retryable)    |
-| `too long`, `duration`, `length`         | `audio_too_long` (not retryable) |
-| (anything else)                          | `provider_error` (retryable)     |
-
-Full error-code enum:
-
-```typescript
-// packages/shared/src/transcription-errors.ts
-export enum TranscriptionErrorCode {
-  SERVICE_NOT_CONFIGURED = "service_not_configured",
-  INVALID_AUDIO_FORMAT = "invalid_audio_format",
-  AUDIO_TOO_LONG = "audio_too_long",
-  AUDIO_TOO_SHORT = "audio_too_short",
-  RATE_LIMITED = "rate_limited",
-  AUTH_FAILED = "auth_failed",
-  PROVIDER_ERROR = "provider_error",
-  NETWORK_ERROR = "network_error",
-  INVALID_REQUEST = "invalid_request",
-}
-```
-
-## Common Mistakes
-
-### HIGH Calling /transcribe without configuring transcriptionService
-
-Wrong:
-
-```typescript
-new CopilotRuntime({ agents });
-// client calls /api/copilotkit/transcribe → 503
-```
-
-Correct:
-
-```typescript
-new CopilotRuntime({
-  agents,
-  transcriptionService: new MyWhisperService(),
-});
-```
-
-Unconfigured runtime returns HTTP 503 with
-`{ error: "service_not_configured" }`. The frontend gets no transcript with no obvious
-server-side failure.
-
-Source: `packages/runtime/src/v2/runtime/handlers/handle-transcribe.ts:203-207`.
-
-### MEDIUM Form field named "file" instead of "audio"
-
-Wrong:
-
-```typescript
-const form = new FormData();
-form.append("file", blob, "recording.webm");
-await fetch("/api/copilotkit/transcribe", { method: "POST", body: form });
-```
-
-Correct:
-
-```typescript
-const form = new FormData();
-form.append("audio", blob, "recording.webm");
-await fetch("/api/copilotkit/transcribe", { method: "POST", body: form });
-```
-
-The handler reads `formData.get("audio")` — any other field name yields `null` and returns
-`invalid_request`.
-
-Source: `packages/runtime/src/v2/runtime/handlers/handle-transcribe.ts:91-97`.
-
-### MEDIUM Base64 payload missing mimeType
-
-Wrong:
-
-```typescript
-await fetch("/api/copilotkit/transcribe", {
-  method: "POST",
-  headers: { "Content-Type": "application/json" },
-  body: JSON.stringify({ audio: b64 }),
-});
-```
-
-Correct:
-
-```typescript
-await fetch("/api/copilotkit/transcribe", {
-  method: "POST",
-  headers: { "Content-Type": "application/json" },
-  body: JSON.stringify({ audio: b64, mimeType: "audio/webm" }),
-});
-```
-
-JSON mode requires `mimeType` — the handler explicitly rejects payloads missing it with
-`invalid_request`.
-
-Source: `packages/runtime/src/v2/runtime/handlers/handle-transcribe.ts:131-136`.
-
-### LOW Re-categorizing errors inside the service
-
-Wrong:
-
-```typescript
-class MyService extends TranscriptionService {
-  async transcribeFile(opts: TranscribeFileOptions): Promise<string> {
-    try {
-      return await doTranscribe(opts);
-    } catch (e) {
-      // trying to hand-pick error codes
-      throw new Error("RATE_LIMITED");
-    }
-  }
-}
-```
-
-Correct:
-
-```typescript
-class MyService extends TranscriptionService {
-  async transcribeFile(opts: TranscribeFileOptions): Promise<string> {
-    return doTranscribe(opts); // let provider errors bubble up verbatim
-  }
-}
-```
-
-The runtime scans `String(error).toLowerCase()` for `"rate"`, `"429"`, `"auth"`, `"too long"`
-etc. Provider-native messages (`"OpenAI returned 429 rate limited"`) auto-map to the right
-code. Hand-crafted codes bypass the keyword matcher and end up as `provider_error`.
-
-Source: `packages/runtime/src/v2/runtime/handlers/handle-transcribe.ts:160-196`.
-
-### MEDIUM Returning a rich object instead of a string
-
-Wrong:
-
-```typescript
-class MyService extends TranscriptionService {
-  async transcribeFile(opts: TranscribeFileOptions): Promise<string> {
-    // @ts-expect-error returning the wrong shape
-    return {
-      text: "hi",
-      segments: [
-        /* ... */
-      ],
-    };
-  }
-}
-```
-
-Correct:
-
-```typescript
-class MyService extends TranscriptionService {
-  async transcribeFile(opts: TranscribeFileOptions): Promise<string> {
-    const result = await provider.transcribe(opts.audioFile);
-    return result.text;
-  }
-}
-```
-
-`transcribeFile` returns `Promise<string>`. The handler sends
-`{ transcription: string }` back to the client — any other shape is a TypeScript error and
-would be JSON-stringified wrongly at runtime.
-
-Source: `packages/runtime/src/v2/runtime/transcription-service/transcription-service.ts:9-11`.
-
-## See also
-
-- `copilotkit/setup-endpoint` — `/transcribe` is one of the routes the handler mounts
-- `copilotkit/debug-and-troubleshoot` — `TranscriptionErrorCode` catalog

package/skills/runtime/references/wiring-a2a.md

@@ -1,40 +0,0 @@
-A2A (Agent2Agent) — wired via `@ag-ui/a2a`. Requires a pre-built `A2AClient` (not a URL).
-
-## Install
-
-```bash
-pnpm add @ag-ui/a2a @a2a-js/sdk
-```
-
-## Minimal wire-up
-
-```typescript
-import {
-  CopilotRuntime,
-  createCopilotRuntimeHandler,
-} from "@copilotkit/runtime/v2";
-import { A2AAgent } from "@ag-ui/a2a";
-import { A2AClient } from "@a2a-js/sdk/client";
-
-const a2aClient = new A2AClient(process.env.A2A_URL!);
-
-const runtime = new CopilotRuntime({
-  agents: {
-    default: new A2AAgent({ a2aClient }),
-  },
-});
-
-const handler = createCopilotRuntimeHandler({
-  runtime,
-  basePath: "/api/copilotkit",
-});
-
-export default { fetch: handler };
-```
-
-## Gotcha — do NOT pass `{ url }`
-
-`A2AAgent` takes `{ a2aClient }`. The A2A protocol has its own handshake; the client
-object handles it. Passing `{ url: "..." }` is a type error and will fail at runtime.
-
-Source: `examples/integrations/a2a-a2ui/app/api/copilotkit/[[...slug]]/route.tsx:12`.

package/skills/runtime/references/wiring-adk.md

@@ -1,45 +0,0 @@
-Google ADK (Agent Development Kit) — wired via the bare `HttpAgent` from `@ag-ui/client`.
-
-## Install
-
-```bash
-pnpm add @ag-ui/client
-```
-
-## Minimal wire-up
-
-```typescript
-import {
-  CopilotRuntime,
-  createCopilotRuntimeHandler,
-} from "@copilotkit/runtime/v2";
-import { HttpAgent } from "@ag-ui/client";
-
-const runtime = new CopilotRuntime({
-  agents: {
-    default: new HttpAgent({
-      url: process.env.ADK_URL ?? "http://localhost:8000/",
-    }),
-  },
-});
-
-const handler = createCopilotRuntimeHandler({
-  runtime,
-  basePath: "/api/copilotkit",
-});
-
-export default { fetch: handler };
-```
-
-## Server side
-
-Your ADK Python agent must speak AG-UI. ADK ships an AG-UI FastAPI adapter — use it
-and point `HttpAgent({ url })` at the FastAPI route.
-
-## Gotcha — env-sourced credentials
-
-ADK typically authenticates to Google Cloud via service-account credentials
-(`GOOGLE_APPLICATION_CREDENTIALS`). Those live on the ADK Python server, not in the
-CopilotKit runtime. The runtime just forwards AG-UI events.
-
-Source: `docs/content/docs/integrations/adk/quickstart.mdx`.

package/skills/runtime/references/wiring-ag2.md

@@ -1,41 +0,0 @@
-AG2 — wired via the bare `HttpAgent` from `@ag-ui/client`. No dedicated `@ag-ui/ag2`
-package exists; AG2 is a standard HTTP AG-UI framework.
-
-## Install
-
-```bash
-pnpm add @ag-ui/client
-```
-
-## Minimal wire-up
-
-```typescript
-import {
-  CopilotRuntime,
-  createCopilotRuntimeHandler,
-} from "@copilotkit/runtime/v2";
-import { HttpAgent } from "@ag-ui/client";
-
-const runtime = new CopilotRuntime({
-  agents: {
-    default: new HttpAgent({
-      url: process.env.AG2_URL ?? "http://localhost:8000/",
-    }),
-  },
-});
-
-const handler = createCopilotRuntimeHandler({
-  runtime,
-  basePath: "/api/copilotkit",
-});
-
-export default { fetch: handler };
-```
-
-## Gotcha — no dedicated package
-
-Unlike Mastra / LangGraph / CrewAI / LlamaIndex / Agno, AG2 has no `@ag-ui/ag2` package.
-Always use the generic `HttpAgent`. If an older doc references a dedicated AG2 package,
-treat it as stale.
-
-Source: `docs/content/docs/integrations/ag2/quickstart.mdx`; maintainer Phase 4 resolution.

package/skills/runtime/references/wiring-agno.md

@@ -1,40 +0,0 @@
-Agno — wired via the generic `HttpAgent` from `@ag-ui/client`. Requires an `/agui` URL
-suffix.
-
-## Install
-
-```bash
-pnpm add @ag-ui/client
-```
-
-## Minimal wire-up
-
-```typescript
-import {
-  CopilotRuntime,
-  createCopilotRuntimeHandler,
-} from "@copilotkit/runtime/v2";
-import { HttpAgent } from "@ag-ui/client";
-
-const runtime = new CopilotRuntime({
-  agents: {
-    default: new HttpAgent({
-      url: process.env.AGNO_URL ?? "http://localhost:8000/agui",
-    }),
-  },
-});
-
-const handler = createCopilotRuntimeHandler({
-  runtime,
-  basePath: "/api/copilotkit",
-});
-
-export default { fetch: handler };
-```
-
-## Gotcha — the `/agui` suffix is mandatory
-
-Agno's AG-UI FastAPI app mounts at `/agui`. Pointing `url` at the server root
-(`http://localhost:8000`) returns 404. Always include `/agui`.
-
-Source: `docs/content/docs/integrations/agno/quickstart.mdx:215`.

package/skills/runtime/references/wiring-aws-strands.md

@@ -1,59 +0,0 @@
-AWS Strands — wired via the bare `HttpAgent` from `@ag-ui/client`.
-
-## Install
-
-```bash
-pnpm add @ag-ui/client
-```
-
-## Minimal wire-up
-
-```typescript
-import {
-  CopilotRuntime,
-  createCopilotRuntimeHandler,
-} from "@copilotkit/runtime/v2";
-import { HttpAgent } from "@ag-ui/client";
-
-const runtime = new CopilotRuntime({
-  agents: {
-    default: new HttpAgent({
-      url: process.env.STRANDS_URL ?? "http://localhost:8000",
-    }),
-  },
-});
-
-const handler = createCopilotRuntimeHandler({
-  runtime,
-  basePath: "/api/copilotkit",
-});
-
-export default { fetch: handler };
-```
-
-## Server side
-
-Strands agents run on AWS and typically expose an AG-UI-speaking endpoint (API Gateway or
-Lambda Function URL). Point `HttpAgent({ url })` at that endpoint.
-
-## Gotcha — AWS auth
-
-Strands deployments often require IAM SigV4 or a custom header. `HttpAgent` accepts a
-`headers: Record<string, string>` option that is attached to every outbound runtime →
-Strands call:
-
-```typescript
-new HttpAgent({
-  url: process.env.STRANDS_URL!,
-  headers: { Authorization: `Bearer ${process.env.STRANDS_TOKEN!}` },
-});
-```
-
-`hooks.onBeforeHandler` will NOT work for this — those hooks run on the inbound frontend
-→ runtime request, not on the outbound runtime → Strands call that `HttpAgent` issues.
-For SigV4 (which needs a per-request signature over the body), front Strands with a
-lightweight Lambda / API Gateway authorizer that strips client credentials and adds the
-IAM signing, then point `HttpAgent({ url })` at that shim.
-
-Source: `node_modules/@ag-ui/client/dist/index.d.ts` (`HttpAgentConfig.headers`);
-`docs/content/docs/integrations/aws-strands/quickstart.mdx`.

package/skills/runtime/references/wiring-crewai-crews.md

@@ -1,51 +0,0 @@
-CrewAI Crews — multi-agent crews wired via `@ag-ui/crewai`.
-
-## Install
-
-```bash
-pnpm add @ag-ui/crewai
-```
-
-## Minimal wire-up
-
-```typescript
-import {
-  CopilotRuntime,
-  createCopilotRuntimeHandler,
-} from "@copilotkit/runtime/v2";
-import { CrewAIAgent } from "@ag-ui/crewai";
-
-const runtime = new CopilotRuntime({
-  agents: {
-    default: new CrewAIAgent({
-      url: process.env.CREWAI_URL ?? "http://localhost:8000/",
-    }),
-  },
-});
-
-const handler = createCopilotRuntimeHandler({
-  runtime,
-  basePath: "/api/copilotkit",
-});
-
-export default { fetch: handler };
-```
-
-## Crews vs Flows
-
-CrewAI ships two products:
-
-- **Crews** — multi-agent orchestration. Use `CrewAIAgent` from `@ag-ui/crewai`.
-- **Flows** — event-driven pipelines. Use the generic `HttpAgent` from `@ag-ui/client`
-  (there's no framework-specific wrapper). See [crewai-flows.md](crewai-flows.md).
-
-## Gotcha — trailing slash
-
-The `url` for `CrewAIAgent` traditionally ends with a trailing slash
-(`http://localhost:8000/`). Follow whatever your CrewAI server exposes — don't strip it.
-
-Source: `@ag-ui/crewai` package types (`CrewAIAgent` constructor);
-`docs/content/docs/reference/v1/sdk/python/CrewAIAgent.mdx` for v1 Python-side
-reference. The v2 integrations docs currently ship only a Flows quickstart at
-`docs/content/docs/integrations/crewai-flows/quickstart.mdx` — there is no dedicated
-Crews quickstart yet.

package/skills/runtime/references/wiring-crewai-flows.md

@@ -1,45 +0,0 @@
-CrewAI Flows — wired via the bare `HttpAgent` from `@ag-ui/client`. No dedicated wrapper.
-
-## Install
-
-```bash
-pnpm add @ag-ui/client
-```
-
-## Minimal wire-up
-
-```typescript
-import {
-  CopilotRuntime,
-  createCopilotRuntimeHandler,
-} from "@copilotkit/runtime/v2";
-import { HttpAgent } from "@ag-ui/client";
-
-const runtime = new CopilotRuntime({
-  agents: {
-    default: new HttpAgent({
-      url: process.env.CREWAI_FLOWS_URL ?? "http://localhost:8000/",
-    }),
-  },
-});
-
-const handler = createCopilotRuntimeHandler({
-  runtime,
-  basePath: "/api/copilotkit",
-});
-
-export default { fetch: handler };
-```
-
-## Flows vs Crews
-
-Flows is the event-driven pipeline product. For multi-agent Crews orchestration, use
-`CrewAIAgent` from `@ag-ui/crewai` instead — see [crewai-crews.md](crewai-crews.md).
-
-## Gotcha — AG-UI compatibility
-
-Your CrewAI Flows server must speak AG-UI over HTTP. If you control the server, use the
-official CrewAI Python AG-UI adapter. `HttpAgent` is a thin bridge — any server that
-emits AG-UI events at the URL works.
-
-Source: `docs/content/docs/integrations/crewai-flows/quickstart.mdx`.