@coder/ai-sdk-agent 0.1.0 → 0.2.1

package/README.md

@@ -21,6 +21,25 @@ context compaction. The Vercel AI SDK runs its loop **client‑side**. This pack
 bridges the two, so a Coder agent looks and feels like a native AI SDK agent without
 re‑implementing the loop.
 
+## Agent vs. provider — which package?
+
+Two packages, two jobs:
+
+- **`@coder/ai-sdk-agent` (this package)** — Coder's **server‑side agent**: the
+  multi‑step tool loop, built‑in tools, MCP servers, workspace‑scoped file/shell
+  tools, sub‑agents, and compaction all run on the deployment. Each `CoderAgent`
+  is one server chat ("session") and may provision a workspace. Reach for it when
+  you need **server‑side tools, MCP, or a workspace**.
+- **[`@coder/ai-sdk-provider`](../provider)** — **plain model calls** through
+  Coder's AI Gateway. A normal AI SDK provider: `generateText`, `streamText`, and
+  **`generateObject` for schema‑constrained structured output**. No chat, no
+  workspace, natively cancelable. Reach for it when you just need **a model**
+  (plan / extract / summarize / classify) with no server‑side tools.
+
+Rule of thumb: **need server‑side tools, MCP, or a workspace → Agent; need a model
+→ provider.** They compose — a multi‑step pipeline often uses the provider for its
+pure text/JSON steps and the Agent only for the steps that touch tools.
+
 ## Install
 
 ```bash
@@ -76,6 +95,8 @@ pnpm example:generate     # non-streaming generate()
 pnpm example:stream       # streaming via textStream
 pnpm example:tool         # custom (client-executed) tool round-trip
 pnpm example:multi-turn   # multi-turn session memory
+pnpm example:file         # attach a file to a chat (optional: pass a path)
+pnpm example:structured   # typed structured output via the structured_output tool
 ```
 
 Each example creates a new chat and archives it when done — it never touches workspaces. See
@@ -93,6 +114,74 @@ AI SDK tool loop — your `execute` runs in your process.
   appear in the transcript as `providerExecuted` tool calls/results — you observe them, you
   don't execute them.
 
+## Files
+
+There are two distinct ways to get a file to the agent, depending on whether the model should
+**read** it or **operate on** it.
+
+**Chat attachments** — content for the model to read (a PDF, image, CSV…). Drop a native AI SDK
+`file` part into a message and it's uploaded transparently:
+
+```ts
+import { readFile } from "node:fs/promises";
+
+await agent.generate({
+  messages: [
+    {
+      role: "user",
+      content: [
+        { type: "text", text: "Summarize this report." },
+        {
+          type: "file",
+          data: await readFile("report.pdf"),
+          mediaType: "application/pdf",
+          filename: "report.pdf",
+        },
+      ],
+    },
+  ],
+});
+```
+
+Or upload once and reuse across turns with `attach()` — which also accepts a `Blob`/`File` or
+stream (use `fs.openAsBlob` to avoid reading the whole file into memory):
+
+```ts
+const file = await agent.attach({
+  content: await openAsBlob("report.pdf"),
+  mediaType: "application/pdf",
+});
+await agent.generate({
+  messages: [
+    {
+      role: "user",
+      content: [
+        { type: "text", text: "List the risks." },
+        file.toFilePart(), // references the upload by id — no re-upload
+      ],
+    },
+  ],
+});
+```
+
+Attachments are capped at **10 MiB** and restricted to a narrow media‑type allowlist
+(`application/pdf`, `application/json`, `text/{plain,markdown,csv}`, `image/{png,jpeg,gif,webp}`).
+Oversized or unsupported files throw a clear error up front.
+
+**Workspace files** — material for the agent to operate on (a zip of assets, a dataset, a
+binary — anything outside the allowlist or over the cap). Write it onto the workspace filesystem
+and let the agent's tools take over. This needs a `workspaceFiles` adapter (the agent core stays
+dependency‑free; whoever holds a workspace connection supplies a few‑line adapter):
+
+```ts
+const agent = new CoderAgent({ /* … */ workspaceId: ws.id, workspaceFiles });
+const { path } = await agent.uploadToWorkspace({
+  content: await openAsBlob("assets.zip"),
+  path: "assets.zip",
+});
+// Then ask the agent to `unzip assets.zip` — uploadToWorkspace writes bytes as-is; it does not unpack.
+```
+
 ## Auth
 
 Pass a Coder **API token** or **session token** as `token`; it is sent as the
@@ -113,30 +202,207 @@ One `CoderAgent` instance maps to one chat ("session") on the Coder server. The
 created on the first turn and reused for subsequent `generate()`/`stream()` calls (multi‑turn
 conversation with server‑side history). `agent.chatId` is the current chat id.
 
-- `agent.resetSession()` — start a fresh chat on the next turn.
+- `agent.resetSession()` — start a fresh chat on the next turn (reuse one instance for sequential turns; you don't need a new agent per turn).
 - `agent.interrupt()` — interrupt an in‑flight generation.
-- `agent.archive()` — archive the underlying chat (cleanup).
+- `agent.archive()` — archive the underlying chat (cleanup; see [Cleanup](#cleanup)).
+- `agent.listModels()` — list the deployment's model configs, so you don't have to guess the `model` hint.
 - Resume a prior chat: `new CoderAgent({ …, chatId: "…" })`.
 
-A single instance is **single‑flight** — don't run concurrent generations against it.
+A single instance is **single‑flight** — don't run concurrent generations against it. For concurrency, use one instance per session (and see [Workspaces & quota](#workspaces--quota)).
+
+## Timeouts & cancellation
+
+Pass an `abortSignal` to `generate()`/`stream()` to cancel a turn. Aborting
+**interrupts the server‑side run** (not just the local socket), so the chat stops
+generating and releases its resources instead of running on, orphaned. Tearing
+down a `stream()` early (cancelling the stream) interrupts the run too.
+
+For a hard ceiling, set `requestTimeoutMs`. If a segment runs longer (e.g. the
+server is wedged, or a workspace can't be scheduled), the run is interrupted and
+the call rejects with a retryable `CoderChatError` (`kind: "timeout"`) instead of
+hanging:
+
+```ts
+const agent = new CoderAgent({ /* … */ requestTimeoutMs: 120_000 });
+```
+
+`requestTimeoutMs` bounds **each server segment** — one model round‑trip until it
+settles or pauses for a client tool. A multi‑step `generate()` that drives client
+tools runs several segments, so it bounds each one, not the whole call. To cap the
+**total** wall‑clock of a multi‑step call, pass a deadline as the signal instead:
+
+```ts
+await agent.generate({ prompt: "…", abortSignal: AbortSignal.timeout(120_000) });
+```
+
+If the event stream drops before the turn settles, the call rejects with
+`CoderChatError` (`kind: "stream_closed"`, retryable) rather than returning a
+truncated result as if the turn had finished.
+
+## Cleanup
+
+`archive()` soft‑hides the chat (it stays in listings as `archived: true`; there
+is no hard delete yet). To make cleanup ride scope exit instead of a `finally` you
+have to remember, the agent is an **async disposable**:
+
+```ts
+await using agent = new CoderAgent({
+  /* … */
+});
+const { text } = await agent.generate({ prompt: "…" });
+// agent.archive() runs automatically when the scope exits (best‑effort).
+```
+
+In a request handler that returns before a fire‑and‑forget `archive()` settles, the
+archive can be abandoned — `await using` (or an awaited `archive()` in `finally`)
+avoids accumulating live chats.
+
+## Handling errors
+
+All errors extend `CoderAgentError`. Two carry structured detail you can branch on:
+
+- **`CoderApiError`** — an HTTP request failed. Fields: `status`, `method`, `path`, `detail`.
+- **`CoderChatError`** — a turn ended in an error, timed out, or lost its stream. Fields: `kind`, `retryable`, `statusCode`, `provider`.
+
+```ts
+import { CoderApiError, CoderChatError } from "@coder/ai-sdk-agent";
+
+try {
+  await agent.generate({ prompt: "…" });
+} catch (err) {
+  if (err instanceof CoderChatError && err.retryable) {
+    // transient (timeout, stream_closed, an upstream 5xx) — back off and retry
+  } else if (err instanceof CoderApiError && err.status === 429) {
+    // rate limited
+  } else {
+    throw err;
+  }
+}
+```
+
+`maxRetries` defaults to `0`: this agent owns server‑side chat state, so an
+SDK‑level retry could duplicate a turn. Prefer catching `retryable` errors and
+retrying the whole step deliberately.
+
+## Structured output
+
+Coder Agents has no server‑side `response_format`, so `CoderAgent` cannot
+constrain what the model **says** to a JSON schema — a `responseFormat` /
+`experimental_output` request emits a warning and is best‑effort at most. Pick
+by what the step needs:
+
+- **Pure text‑in / JSON‑out, no server‑side tools** → use
+  **[`@coder/ai-sdk-provider`](../provider)** with `generateObject` /
+  `Output.object` (schema‑constrained; requires AI Gateway on the deployment).
+- **The answer must come out of an agent run** (server‑side tools, MCP, a
+  workspace) → use the **`structured_output` tool pattern** below. What the
+  model _says_ isn't schema‑constrained, but what it passes **into a tool** is
+  typed — so have it submit its answer by _calling a tool_ whose `inputSchema`
+  is your Zod schema. The answer arrives as the tool call's typed `input`; no
+  fishing JSON out of prose.
+
+```ts
+import { stepCountIs, tool } from "ai";
+import { z } from "zod";
+
+const Answer = z.object({ severity: z.enum(["critical", "major", "minor"]), summary: z.string() });
+
+const agent = new CoderAgent({
+  /* … */
+  instructions: "… Submit your final answer by calling the structured_output tool exactly once.",
+  tools: {
+    structured_output: tool({
+      description:
+        "Submit your final structured answer as JSON. Call this exactly once, when your work is complete.",
+      inputSchema: Answer, // your schema IS the tool's input schema
+      // Ack instead of stopping the turn: the model finishes naturally and can
+      // wind down anything it still has running (dev servers, watchers, …).
+      execute: async () =>
+        "Output received. Wind down and end your turn. Do not call structured_output again.",
+    }),
+  },
+  stopWhen: stepCountIs(6), // happy path is 2 steps: file + ack, wind down
+});
+
+const result = await agent.generate({ prompt: "…" });
+// toolCalls only holds the LAST step's calls — scan all steps, last call wins.
+const raw = result.steps
+  .flatMap((s) => s.toolCalls)
+  .findLast((c) => c.toolName === "structured_output")?.input;
+if (raw === undefined)
+  throw new Error("model never called structured_output — nudge once on an idle chat (rule 3)");
+const answer = Answer.parse(raw); // typed: { severity: "critical" | "major" | "minor"; summary: string }
+```
+
+Rules that keep it robust — each guards against a failure mode observed live:
+
+1. **Don't force `toolChoice`, don't stop on the call.** `toolChoice` is
+   construction‑time and applies to _every_ segment, so after the ack it would
+   force the tool again and again up to the step ceiling (and it blocks any
+   other tools the step needs). A `hasToolCall` stop is worse: the server only
+   receives a client tool result as a side effect of the _next_ loop segment,
+   so ending the loop on the call strands the chat in `requires_action` —
+   follow‑up messages queue forever and `archive()` 409s. Instructions plus the
+   tool's own description are enough; models file unprompted most of the time.
+2. **Validate client‑side.** The schema is not enforced server‑side —
+   `schema.safeParse` on the tool input is the real gate. (Schema‑invalid calls
+   that the AI SDK catches in‑loop are automatically answered with a
+   `tool-error` result the model retries against.)
+3. **Nudge at most once, and only an idle chat.** If the turn ends in prose
+   (`finishReason: "stop"`) without a valid call, send one typed re‑prompt
+   ("Call the structured_output tool now …"), then fail into your normal error
+   handling. Never re‑prompt a chat that isn't idle — the message would queue
+   behind whatever the server is still doing.
+4. **Settle a turn that stopped on a tool call.** If the loop stops on a
+   tool‑call step — e.g. your `stopWhen` ceiling lands exactly on the
+   `structured_output` call (`finishReason: "tool-calls"`) — the tool results
+   ran locally but never reached the server. Submit the stranded step's
+   (`result.steps.at(-1)`) locally‑executed client results directly via
+   `agent.client.submitToolResults(agent.chatId, { results: [{ tool_call_id, output }] })`
+   before touching the chat again, or it strands as in rule 1; if a pending
+   call has no local result (or the submit fails), `agent.interrupt()` ends
+   the stranded turn instead. A settled chat resumes its wind‑down server‑side
+   for a few seconds, so retry a 409ing `archive()` under a short deadline
+   instead of giving up.
+
+[`examples/06-structured-output.ts`](./examples/06-structured-output.ts) packages
+all four rules into a small copyable helper — `structuredOutput(schema)` returns
+`agentOpts` to spread into the constructor plus a typed `ask(agent, prompt)`
+that runs the settle + one‑nudge ladder and returns a `z.infer<typeof schema>`.
+
+## Workspaces & quota
+
+A `CoderAgent` is one server‑side chat, and — depending on its configuration and
+the deployment — a chat may provision a **Coder workspace** to run its tools. A
+deployment caps how many workspaces an account may run at once, so **N agents
+running concurrently can need N free workspace slots.** Past the cap, a turn can
+sit unscheduled and never settle. This is the most important operational fact when
+running many agents at once:
+
+- Keep your own concurrency below the deployment's workspace limit.
+- Set `requestTimeoutMs` so an unschedulable turn fails loudly instead of hanging.
+- `archive()` / `await using` each agent so finished chats stop holding resources.
+- For steps that don't need server‑side tools, prefer the provider — it never touches a workspace.
 
 ## Configuration
 
 `CoderAgentSettings`:
 
-| field                             | description                                                                   |
-| --------------------------------- | ----------------------------------------------------------------------------- |
-| `client` \| (`baseUrl` + `token`) | connection (one or the other)                                                 |
-| `organizationId`                  | org UUID that owns the chat (required)                                        |
-| `model`                           | model hint: UUID, `provider:model`, model id, or display‑name substring       |
-| `instructions`                    | system prompt                                                                 |
-| `tools`                           | AI SDK `ToolSet` (client‑executed)                                            |
-| `workspaceId`                     | bind the chat to a Coder workspace (enables workspace‑scoped tools)           |
-| `mcpServerIds`                    | server‑side MCP servers to enable                                             |
-| `planMode`                        | enable plan mode (`"plan"`)                                                   |
-| `stopWhen`                        | AI SDK stop condition(s); default `stepCountIs(64)`                           |
-| `maxRetries`                      | default `0` — SDK retries can duplicate server‑side turns; override with care |
-| `chatId`                          | resume an existing chat                                                       |
+| field                             | description                                                                                      |
+| --------------------------------- | ------------------------------------------------------------------------------------------------ |
+| `client` \| (`baseUrl` + `token`) | connection (one or the other)                                                                    |
+| `organizationId`                  | org UUID that owns the chat (required)                                                           |
+| `model`                           | model hint: UUID, `provider:model`, model id, or display‑name substring                          |
+| `instructions`                    | system prompt                                                                                    |
+| `tools`                           | AI SDK `ToolSet` (client‑executed)                                                               |
+| `workspaceId`                     | bind the chat to a Coder workspace (enables workspace‑scoped tools)                              |
+| `workspaceFiles`                  | adapter enabling `uploadToWorkspace()` (write files to the workspace FS)                         |
+| `mcpServerIds`                    | server‑side MCP servers to enable                                                                |
+| `planMode`                        | enable plan mode (`"plan"`)                                                                      |
+| `stopWhen`                        | AI SDK stop condition(s); default `stepCountIs(64)`                                              |
+| `maxRetries`                      | default `0` — SDK retries can duplicate server‑side turns; override with care                    |
+| `requestTimeoutMs`                | per‑turn time budget (ms); interrupts the run and rejects (`kind: "timeout"`) instead of hanging |
+| `chatId`                          | resume an existing chat                                                                          |
 
 ## How it works
 
@@ -154,6 +420,28 @@ CoderAgent  (implements ai.Agent)
 - Streaming text is emitted from `message_part` deltas; fast turns that only produce a full
   `message` snapshot are diffed against an emitted‑length cursor — so neither double‑counts.
 
+## Durable workflows (Vercel Workflow, step functions, …)
+
+`CoderAgent` talks to Coder over its own REST + WebSocket client, so it can't ride
+a `fetch`‑shim durability layer — each turn must run **inside** a durable step. A
+few rules keep it well‑behaved across replays:
+
+- **One turn per step.** Create the agent, run a single `generate()` (not
+  `stream()`, so the checkpointed value is the finished result), return.
+- **Don't persist the instance across steps.** Persist `agent.chatId` (a string)
+  and resume with `new CoderAgent({ …, chatId })` in the next step. Never persist
+  or log the token — read it from the environment in each step.
+- **Clean up in the step.** `await using` the agent (or `await agent.archive()` in
+  a `finally`) so a step that returns early doesn't abandon the chat.
+- **Bound each step.** Set `requestTimeoutMs` so a wedged turn fails the step (and
+  lets the workflow retry) instead of hanging the whole run.
+- **Mind concurrency vs. workspaces.** Keep fan‑out width under the deployment's
+  workspace cap — see [Workspaces & quota](#workspaces--quota).
+- **Use the provider for pure steps.** Steps that don't need server‑side tools
+  (plan / extract / synthesize) are cheaper and natively structured through
+  [`@coder/ai-sdk-provider`](../provider) + `generateObject` — no chat, no
+  workspace, no cleanup.
+
 ## Testing
 
 ```bash
@@ -178,7 +466,6 @@ The e2e suite creates **new chats only** (no workspaces) and archives them after
 ## Limitations
 
 - The Coder chat API is experimental (`/api/experimental/chats`); wire types may change.
-- File/image **inputs** are not yet forwarded to the server (text prompts only).
 - Designed for Node (WebSocket via `ws`); a browser build can inject a `webSocketFactory`.
 - A v7 `@ai-sdk/harness` adapter (the conceptually exact fit) is a future direction once that
   experimental API stabilizes.