@agent-compose/sdk 0.1.0 → 0.2.0

package/README.md

@@ -1,6 +1,13 @@
 # @agent-compose/sdk
 
-Client library for building against agent-compose. Provides TypeScript types for defining agents, runtimes, and workflows, plus an HTTP client that auto-discovers dependencies and registers everything with a single call.
+TypeScript SDK for [agent-compose](https://github.com/Layr-Labs/agent-compose). Use it to:
+
+- **Author workflows** that run agentic LLM loops inside isolated sandboxes
+- **Define runtimes** that wrap a coding-CLI tool (Claude Code, OpenAI Desktop, …) into a sandbox-portable agent loop
+- **Register, invoke, observe, and cancel** workflows via the HTTP API (`AgentComposeClient`)
+- **Manage factories, secrets, API keys, and snapshots** programmatically
+
+The hierarchy: a **team** owns one or more **factories** (project containers); each factory owns workflow templates, secrets, and runs. Workflows are versioned per `(factory, name, version)`. New code that doesn't care about factories transparently lands in `default` — every team has one.
 
 ---
 
@@ -14,139 +21,428 @@ npm install zod
 
 ---
 
-## Defining an Agent
+## Authoring a workflow
+
+A workflow is `async (ctx, sandbox) => T`. Two positional args:
+
+- **`ctx`** carries the run identity (`run.id`), the caller's `input`, plus
+  observability helpers (`setMetadata`, `step`).
+- **`sandbox`** is a capability the engine constructs once for the run —
+  pass it to `runAgent({ sandbox, ... })` and to any helper that takes a
+  `SandboxProvider` (file writers, git utilities, command runners).
 
 ```typescript
-// my-agents/planner.ts
-import { defineAgent } from "@agent-compose/sdk";
-
-export default defineAgent({
-  name: "planner",
-  runtime: "claude",                                    // registered runtime name
-  prompt: "Explore the codebase and write a plan...",
-  budget: { turnsPerIteration: 70, maxIterations: 1 },
+// my-workflow.ts
+import { defineWorkflow, runAgent, claudeRuntime } from "@agent-compose/sdk";
+import PROMPT from "./prompt.md" with { type: "text" };
+
+export default defineWorkflow({
+  async run(ctx, sandbox) {
+    const repo = (ctx.input?.repo as string | undefined) ?? "owner/repo";
+
+    const result = await runAgent({
+      sandbox,
+      runtime:    claudeRuntime,
+      prompt:     PROMPT,
+      promptVars: { REPO: repo },
+      tools:      ["Bash", "Read", "Edit", "Write", "Grep", "Glob"],
+      budget:     { turnsPerIteration: 40, maxIterations: 8 },
+    });
+
+    await ctx.setMetadata({ summary: result.status?.summary });
+    return { ok: result.status?.completed ?? false };
+  },
+  // Optional: outbound network rules that the runner sandbox will enforce
+  // (Vercel only — E2B ignores). Use `$VAR` placeholders for secrets that
+  // get resolved from the per-workflow secret store at dispatch time.
+  networkPolicy: {
+    allow: {
+      "*": [],
+      "api.anthropic.com": [{ transform: [{ headers: { "x-api-key": "$ANTHROPIC_API_KEY" } }] }],
+    },
+  },
 });
 ```
 
----
-
-## Defining a Runtime
+`defineWorkflow` is a thin sugar — it returns the bare `run` function with
+`networkPolicy` / `placeholders` / `snapshot` / `saveSnapshot` attached as
+metadata that the bundler picks up at registration time. A plain
+`export default async (ctx, sandbox) => {...}` is also valid; you just lose
+the metadata channel.
+
+### What the workflow can do with `ctx`
+
+```ts
+interface WorkflowCtx {
+  run:         { id: string };
+  input?:      Record<string, unknown>;
+  setMetadata: (data: Record<string, unknown>) => Promise<void>;
+  step<T>(name: string, fn: () => Promise<T>): Promise<T>;
+}
+```
 
-```typescript
-// my-runtimes/claude.ts
-import { defineRuntime } from "@agent-compose/sdk";
-// ClaudeRunner is provided by the server — import from your server package
-import { ClaudeRunner } from "@agent-compose/server/runtimes/claude.js";
-
-export default defineRuntime({
-  provider: "claude-code",  // must match what createAgentSandbox knows about
-  create: (sandbox, opts) => new ClaudeRunner(sandbox, opts),
-});
+`step("phase-name", () => …)` wraps a phase for the run timeline — emits
+`step_started` / `step_completed` / `step_failed` lifecycle events with
+duration. Use it for setup, external API calls, or anything you want
+visible on the dashboard's run detail page.
+
+### The `runAgent` loop
+
+```ts
+runAgent({
+  sandbox,            // the workflow's sandbox arg
+  runtime,            // claudeRuntime, or your own via createClaudeRuntime / defineRuntime
+  prompt,             // raw markdown — `--- frontmatter ---` is auto-stripped
+  promptVars?,        // {{VAR}} substitutions; WORKING_DIR + DIFF_BASE auto-populate
+  tools?,             // model tool allowlist (defaults inside agentLoop)
+  budget?,            // { turnsPerIteration, maxIterations }
+  workingDir?,        // every shell command runs here
+  responseSchema?,    // zod — when set, the loop demands a `<response>` block on exit
+  onAgentEvent?,      // per-message hook (e.g. wire to telemetry)
+  onIteration?,       // per-iteration hook with parsed `<status>` block
+})
+// → AgentLoopResult { status?, response? (when responseSchema set), iterations, … }
 ```
 
+The protocol is simple: the model emits XML-tagged blocks (`<status>` /
+`<response>`) the loop parses. See `sdk/src/agent/protocol-suffix.md` for
+the full instructions appended to every prompt.
+
 ---
 
-## Writing a Workflow
+## Defining a runtime
 
-```typescript
-// my-workflow.ts
-import type { WorkflowFn } from "@agent-compose/sdk";
+A "runtime" wraps an agent's underlying execution model — usually a coding
+CLI like Claude Code or OpenAI Desktop — so `runAgent` can drive it. The SDK
+ships built-ins; you only need a custom one for an exotic provider.
 
-const myWorkflow: WorkflowFn = async (ctx) => {
-  const planner = await ctx.spawnAgent("planner", {
-    data: { task: ctx.input.task },
-  });
+### Built-in runtimes
 
-  await ctx.spawnAgent("implementer", {
-    input: planner,                    // chains branch + response
-    data: { task: ctx.input.task },
-  });
+```ts
+import {
+  createClaudeRuntime,    // factory, takes config
+  claudeRuntime,           // pre-built default (DEFAULT_CLAUDE_MODEL)
+  ClaudeRunner,            // class, if you need to override
+} from "@agent-compose/sdk";
 
-  await ctx.setMetadata({ done: true });
-};
+// openAIDesktopRuntime is NOT in the package root (it pulls in `sharp` for
+// screenshot capture; the native binding can't be cross-compiled). Import
+// directly when you actually want the desktop runtime:
+import openAIDesktopRuntime from "@agent-compose/sdk/runtimes/openai-desktop.js";
+```
 
-export default myWorkflow;
+### Custom runtime
+
+```ts
+import { defineRuntime, type AgentRuntime } from "@agent-compose/sdk";
+
+const myRuntime: AgentRuntime = defineRuntime({
+  create: (sandbox, opts) => {
+    // Return a ModelExecutionContract — see sdk/src/types/runtime.ts
+    return {
+      sendMessage({ prompt, sessionId, signal }) {
+        // Async generator that yields AgentMessage chunks the loop parses.
+        return /* … */;
+      },
+    };
+  },
+});
 ```
 
+`AgentRuntime` is a tagged record with `create(sandbox, RuntimeOptions) →
+ModelExecutionContract`. There is **no `provider` field** on it — the
+runtime is bound to the workflow at author time (you pass it to `runAgent`),
+not selected by the server.
+
 ---
 
-## Registration
+## Registering a workflow
 
-`AgentComposeClient.register()` reads the workflow file, scans for `ctx.spawnAgent("name")` calls, resolves each agent's source file, reads `runtime: "name"` from each agent, resolves runtime sources — then bundles and POSTs everything in one call.
+The `agentc` CLI handles the bundling-and-registration step for you:
 
-```typescript
-import { AgentComposeClient } from "@agent-compose/sdk";
+```bash
+agentc register my-workflow.ts -n my-workflow
+```
+
+Under the hood that calls `bundleWorkflow(workflowPath)` (resolves imports,
+inlines runtime sources via dynamic-require traversal) and `POST
+/api/v1/factories/<slug>/templates` with the bundled source. If you need
+to drive registration from your own build pipeline, you can do the same
+thing via the SDK directly:
+
+```ts
+import { AgentComposeClient, bundleWorkflow } from "@agent-compose/sdk";
 
 const client = new AgentComposeClient(
-  "http://localhost:8080",
-  process.env.API_KEY!,
+  "https://your-server.example.com",
+  process.env.AGENT_COMPOSE_API_KEY!,
 );
 
-// Auto-discovers agents and runtimes from the workflow file
+const bundled = await bundleWorkflow("./my-workflow.ts");
 await client.register({
-  name: "my-workflow",
-  workflowPath: "./my-workflow.ts",
-  // agentPaths and runtimePaths are optional overrides if auto-discovery fails
+  name:        "my-workflow",
+  source:      bundled.source,
+  runtimes:    bundled.runtimes,        // [{ name, source }] — embedded so the runner has them locally
+  schedule:    "*/30 * * * *",          // optional cron
+  factorySlug: "default",                // optional — defaults to "default"
+  // snapshot, saveSnapshot, networkPolicy, placeholders — all optional
 });
 ```
 
-**Auto-discovery convention** — for agent `"planner"`, the client looks for:
-- `./planner.ts`
-- `./planner/index.ts`
-- `./agents/planner.ts`
-- `./agents/planner/index.ts`
-
-For runtime `"claude"`, it looks for:
-- `./claude.ts`
-- `./runtimes/claude.ts`
+`register()` requires the caller's API key to carry the `admin` scope
+(or full team-access for legacy keys without scopes).
 
 ---
 
-## Invocation
+## Invoking a workflow
 
-```typescript
-// Invoke a registered workflow (returns immediately with a run ID)
+Two flavours:
+
+```ts
+// Fire-and-forget — returns the run id immediately.
 const { id } = await client.invoke("my-workflow", {
-  task: "Add rate limiting to the /projects endpoint",
+  repo: "owner/repo",
+});
+
+// Block until the run settles (default 30min timeout, 1s poll).
+const status = await client.invokeAndWait("my-workflow", { repo: "owner/repo" }, {
+  timeoutMs:      5 * 60_000,
+  pollIntervalMs: 2000,
 });
+console.log(status.status);   // "success" | "failed" | "abandoned" | "canceled"
+console.log(status.output);   // workflow's return value
+```
+
+`output` is the workflow's `run()` return value (whatever `defineWorkflow({
+async run() { return … } })` resolves to). `setMetadata()` writes to a
+separate `metadata` field — useful for "side-channel" facts (PR url, plan
+url) without polluting the structured return.
+
+`invoke` and `invokeAndWait` both accept `{ factorySlug, snapshot,
+saveSnapshot, parentRunId }` as the third argument. `factorySlug` defaults
+to `"default"`.
+
+### Auto parent/child tracing
+
+The SDK detects `process.env.RUN_ID` (set by the runner sandbox on every
+dispatch) and automatically threads it as `parentRunId` on subsequent
+`invoke()` calls. Workflows that fan out to other workflows get a
+parent/child tree in the dashboard for free. Pass `parentRunId: null`
+to opt out.
+
+### Cancelling a run
+
+```ts
+await client.cancelRun(runId);
+```
+
+Idempotent — cancelling an already-terminal run returns the current state
+without throwing. The server stamps the run as `canceled`, kills any live
+sandboxes, and emits a `run_canceled` event on the stream.
+
+### Streaming live logs
 
-// Poll until done
-let status;
-do {
-  await new Promise(r => setTimeout(r, 5000));
-  status = await client.getStatus(id);
-} while (status.status === "running");
+`streamRunLogs` returns an async generator of `RunEvent`s in real time,
+re-attaching via SSE under the hood. Pass `lastEventId` (the highest
+`seq` you've already processed) to resume after a reconnect.
 
-console.log(status.output); // workflow's setMetadata() payload
+```ts
+for await (const ev of client.streamRunLogs(runId, { lastEventId: 0 })) {
+  console.log(ev.event, ev.seq, ev.data);
+  if (ev.event === "run_complete" || ev.event === "run_failed" || ev.event === "run_canceled") {
+    break;
+  }
+}
 ```
 
+`AbortSignal` works too — pass `{ signal }` and call `controller.abort()`
+to tear the stream down from the caller side.
+
 ---
 
-## Exported Types
+## Factories
+
+Factories are project containers within a team. Each factory has its own
+workflow templates, secrets, runs, and (optionally) scoped API keys. New
+projects don't need to think about them — `default` is auto-created per
+team and is what the SDK falls back to when `factorySlug` is omitted.
+
+```ts
+// CRUD on factories
+await client.createFactory({ slug: "ci-bots", name: "CI Bots", description: "…" });
+const factories = await client.listFactories();
+const f         = await client.getFactory("ci-bots");
+await client.updateFactory("ci-bots", { name: "Continuous-Integration Bots" });
+await client.deleteFactory("ci-bots");
+
+// Templates list — flat across factories, or scoped to one
+const all   = await client.listTemplates();
+const scoped = await client.listTemplates({ factorySlug: "ci-bots" });
+
+// Register / invoke / secret operations all accept factorySlug
+await client.register({ name: "scrape", source, factorySlug: "ci-bots", … });
+await client.invoke("scrape", { url: "…" }, { factorySlug: "ci-bots" });
+await client.setSecret("scrape", "GH_TOKEN", "ghp_…", { factorySlug: "ci-bots" });
+```
 
-```typescript
-// Factory functions
-defineAgent(pkg: AgentDefinition): AgentDefinition
-defineRuntime(pkg: AgentRuntime): AgentRuntime
+CLI equivalents: `agentc factory list | create | get | update | delete`,
+plus `--factory <slug>` on every other command.
+
+---
+
+## Per-workflow secrets
+
+Secrets live in GCP Secret Manager, one row per `(factory, workflow, key)`.
+They're injected as env vars into the runner sandbox at dispatch time,
+never persisted in the VM. Values are write-only — the API only returns
+metadata (key, timestamps).
+
+```ts
+await client.setSecret("my-workflow", "ANTHROPIC_API_KEY", process.env.ANTHROPIC_API_KEY!);
+const list = await client.listSecrets("my-workflow"); // [{ key, createdAt, updatedAt }]
+await client.deleteSecret("my-workflow", "STALE_KEY");
 
-// Workflow authoring
-type WorkflowFn = (ctx: WorkflowCtx) => Promise<void>
-interface WorkflowCtx { spawnAgent, setMetadata, sendMessage, input, run, cost }
+// Scope to a non-default factory:
+await client.setSecret("scrape", "GH_TOKEN", "ghp_…", { factorySlug: "ci-bots" });
+```
+
+Mutations require `admin` scope.
+
+---
+
+## API keys
+
+Mint and list scoped keys programmatically (requires an `admin`-scoped
+caller key). New keys are returned **once**, in the same response as the
+metadata — copy the `ac_…` value immediately.
+
+```ts
+const created = await client.createApiKey({
+  name:      "ci-dispatcher",
+  scopes:    ["read", "invoke"],
+  expiresAt: new Date(Date.now() + 30 * 86_400_000).toISOString(),  // 30 days
+  // factorySlug: "ci-bots"  // optional — scopes the key to a single factory
+});
+console.log(created.key);   // "ac_…" — the only time you'll see this
+
+const all = await client.listApiKeys();
+```
+
+CLI equivalent: `agentc keys create <name> --scopes read,invoke
+--expires-in 30d`.
+
+---
+
+## Usage
+
+```ts
+const usage = await client.getUsage(
+  new Date(Date.now() - 30 * 86_400_000),
+  new Date(),
+);
+// usage.rows: [{ day, runs, sandbox_seconds, … }]
+```
+
+CLI equivalent: `agentc usage`.
+
+---
+
+## Snapshots (replay-friendly sandboxes)
+
+Long-running workflows can capture the runner sandbox as a Vercel snapshot
+on success (`saveSnapshot: true`). Other workflows reference that snapshot
+via the `snapshot` field to boot into the same prepared VM (deps installed,
+repo cloned, etc.) instead of repeating setup.
 
-// Agent types
-interface AgentDefinition { name, runtime, prompt, budget?, tools?, onStart?, onFinish? }
-interface AgentOpts { budget?, data?, input?, branch?, onStart?, onFinish? }
-interface AgentResult<T> { response: T; session: AgentSession }
+```ts
+// Capture per-invocation:
+await client.invoke("my-workflow", input, { saveSnapshot: true });
 
-// Runtime types
-interface AgentRuntime<S> { provider: string; create(sandbox: S, opts): ModelExecutionContract }
-interface ModelExecutionContract { sendMessage(opts): AsyncGenerator<AgentMessage> }
+// Boot from a different snapshot per invocation (per-invocation override):
+await client.invoke("my-workflow", input, { snapshot: "<run-id-or-name>" });
 
-// Protocol
-interface AgentStatus { summary, completed, blockers, exit_signal }
-type AgentMessage = AgentMessageInit | AgentMessageText | AgentMessageToolUse | ...
+// Or set a default at registration time:
+defineWorkflow({ run, saveSnapshot: true });
 
-// Sandbox
-interface SandboxProvider { sandboxId, cwd?, commands, files, kill() }
-interface DesktopSandboxProvider extends SandboxProvider { screenshot(), leftClick(), ... }
+// Browse / clean up:
+const snaps = await client.listSnapshots({ workflow: "my-workflow", limit: 50 });
+await client.deleteSnapshot(snaps[0].runId);
 ```
+
+CLI equivalents: `agentc snapshot list` / `agentc snapshot delete <run-id>`.
+
+---
+
+## Authentication
+
+The SDK accepts a Bearer API key (`ac_…`). Mint one from the dashboard:
+sign in at `<server-url>/login`, then **Settings → API Keys → Create key**.
+
+Default scopes (`read + invoke`) are right for a CI / dispatch caller. Tick
+`admin` only if this key needs to register templates, mint other keys, or
+manage secrets.
+
+```ts
+const client = new AgentComposeClient(
+  process.env.AGENT_COMPOSE_URL!,
+  process.env.AGENT_COMPOSE_API_KEY!,
+);
+```
+
+The dashboard itself uses the cookie-bound session path; the SDK is for
+programmatic / server-to-server callers.
+
+---
+
+## Public exports — quick reference
+
+| Export | What |
+|---|---|
+| `defineWorkflow` | Attach metadata to a workflow `run` function |
+| `defineRuntime` | Wrap an agent execution provider as an `AgentRuntime` |
+| `defineSandboxEnvironment` | Sugar for declaring a workflow whose primary purpose is to build a snapshot for others to boot from |
+| `runAgent` / `agentLoop` | Embed an LLM loop inside a workflow |
+| `runWorkflow` | Local engine for running a workflow in-process (test harness) |
+| `bundleWorkflow` | Resolve + inline a workflow's runtime sources for registration |
+| `claudeRuntime` / `createClaudeRuntime` / `ClaudeRunner` | Built-in Claude Code runtime + factory |
+| `AgentComposeClient` | HTTP client — register, invoke, cancel, stream logs, factories, snapshots, secrets, API keys, usage |
+| `AgentComposeError` | Thrown by every non-2xx HTTP response |
+| `parseAgentStatus` / `parseAgentResponse` / `AgentStatusSchema` / `AgentMessageSchema` | Protocol parsers |
+| `parseSseStream` | Generic SSE chunk decoder (used by `streamRunLogs`) |
+| `createSandbox` / `reconnectSandbox` / `killAllSandboxes` / `killSandboxById` / `getSandboxQuotas` / `listOwnedSandboxes` / `deleteSandboxSnapshot` | Sandbox-provider helpers (Vercel + E2B) |
+
+Type exports: `WorkflowFn`, `WorkflowCtx`, `WorkflowDefinition`,
+`WorkflowHooks`, `AgentBudget`, `AgentRuntime`, `RuntimeOptions`,
+`ModelExecutionContract`, `McpServerConfig`, `AgentMessage` (and its
+variants), `AgentStatus`, `RunStatus`, `RegisterResult`, `RunEvent`,
+`FactoryRow`, `SnapshotListEntry`, `ApiKey`, `ApiKeyCreated`,
+`UsageRollupRow`, `UsageResponse`, `CancelRunResponse`, `AgentLoopResult`,
+`RunAgentOpts`, `SandboxProvider`, `DesktopSandboxProvider`,
+`SandboxNetworkPolicy`, `SandboxCreateOpts`, `OwnedSandbox`,
+`BundledWorkflow`.
+
+For the canonical signatures, follow your IDE's go-to-definition into
+`@agent-compose/sdk` — `sdk/src/index.ts` is the public surface and the
+files it re-exports from carry full inline docstrings.
+
+---
+
+## Errors
+
+All non-2xx HTTP responses throw `AgentComposeError(status, message)`. The
+`message` is the server's `{ error: string }` body when present, falling
+back to the HTTP status text:
+
+```ts
+import { AgentComposeError } from "@agent-compose/sdk";
+
+try {
+  await client.invoke("missing-workflow");
+} catch (err) {
+  if (err instanceof AgentComposeError && err.status === 404) {
+    // template not registered
+  }
+}
+```
+
+`invokeAndWait` throws `AgentComposeError(504, …)` on timeout for symmetry.