agents 0.16.2 → 0.17.0

package/docs/human-in-the-loop.md

@@ -0,0 +1,661 @@
+# Human in the Loop
+
+Human-in-the-loop (HITL) patterns allow agents to pause execution and wait for human approval, confirmation, or input before proceeding. This is essential for compliance, safety, and oversight in agentic systems.
+
+## Overview
+
+### Why Human in the Loop?
+
+- **Compliance**: Regulatory requirements may mandate human approval for certain actions
+- **Safety**: High-stakes operations (payments, deletions, external communications) need oversight
+- **Quality**: Human review catches errors AI might miss
+- **Trust**: Users feel more confident when they can approve critical actions
+
+### Common Use Cases
+
+| Use Case            | Example                                  |
+| ------------------- | ---------------------------------------- |
+| Financial approvals | Expense reports, payment processing      |
+| Content moderation  | Publishing, email sending                |
+| Data operations     | Bulk deletions, exports                  |
+| AI tool execution   | Confirming LLM tool calls before running |
+| Access control      | Granting permissions, role changes       |
+
+## Choosing an Approach
+
+Agents SDK supports multiple human-in-the-loop patterns. Choose based on your use case:
+
+| Use Case               | Pattern           | Best For                                           | Example                                                                                                      |
+| ---------------------- | ----------------- | -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
+| Long-running workflows | Workflow Approval | Multi-step processes, durable approval gates       | [examples/workflows/](https://github.com/cloudflare/agents/tree/main/examples/workflows)                     |
+| AIChatAgent tools      | `needsApproval`   | Chat-based tool calls with `@cloudflare/ai-chat`   | [guides/human-in-the-loop/](https://github.com/cloudflare/agents/tree/main/guides/human-in-the-loop)         |
+| OpenAI Agents SDK      | `needsApproval`   | Using OpenAI's agent SDK with conditional approval | [openai-sdk/human-in-the-loop/](https://github.com/cloudflare/agents/tree/main/openai-sdk/human-in-the-loop) |
+| Client-side tools      | `onToolCall`      | Tools that need browser APIs or user interaction   | Pattern below                                                                                                |
+| MCP Servers            | Elicitation       | MCP tools requesting structured user input         | [examples/mcp-elicitation/](https://github.com/cloudflare/agents/tree/main/examples/mcp-elicitation)         |
+
+### Decision Guide
+
+```
+Is this part of a multi-step workflow?
+├── Yes → Use Workflow Approval (waitForApproval)
+└── No → Are you building an MCP server?
+         ├── Yes → Use MCP Elicitation (elicitInput)
+         └── No → Is this an AI chat interaction?
+                  ├── Yes → Does the tool need browser APIs?
+                  │        ├── Yes → Use onToolCall (client-side execution)
+                  │        └── No → Use needsApproval (server-side with approval)
+                  └── No → Use State + WebSocket for simple confirmations
+```
+
+## Workflow-Based Approval
+
+For durable, multi-step processes, use Cloudflare Workflows with the `waitForApproval()` helper. The workflow pauses until a human approves or rejects.
+
+### Basic Pattern
+
+```typescript
+import { Agent, AgentWorkflow, callable } from "agents";
+import type { AgentWorkflowEvent, AgentWorkflowStep } from "agents";
+
+// Workflow that pauses for approval
+export class ExpenseWorkflow extends AgentWorkflow<
+  ExpenseAgent,
+  ExpenseParams
+> {
+  async run(event: AgentWorkflowEvent<ExpenseParams>, step: AgentWorkflowStep) {
+    const expense = event.payload;
+
+    // Step 1: Validate the expense
+    const validated = await step.do("validate", async () => {
+      return validateExpense(expense);
+    });
+
+    // Step 2: Wait for manager approval
+    await this.reportProgress({
+      step: "approval",
+      status: "pending",
+      message: `Awaiting approval for $${expense.amount}`
+    });
+
+    // This pauses the workflow until approved/rejected
+    const approval = await this.waitForApproval<{ approvedBy: string }>(step, {
+      timeout: "7 days"
+    });
+
+    console.log(`Approved by: ${approval.approvedBy}`);
+
+    // Step 3: Process the approved expense
+    const result = await step.do("process", async () => {
+      return processExpense(validated);
+    });
+
+    await step.reportComplete(result);
+    return result;
+  }
+}
+```
+
+### Agent Methods for Approval
+
+The agent provides methods to approve or reject waiting workflows:
+
+```typescript
+export class ExpenseAgent extends Agent<Env, ExpenseState> {
+  initialState: ExpenseState = {
+    pendingApprovals: [],
+    status: "idle"
+  };
+
+  // Approve a waiting workflow
+  @callable()
+  async approve(workflowId: string, approvedBy: string): Promise<void> {
+    await this.approveWorkflow(workflowId, {
+      reason: "Expense approved",
+      metadata: { approvedBy, approvedAt: Date.now() }
+    });
+
+    // Update state to reflect approval
+    this.setState({
+      ...this.state,
+      pendingApprovals: this.state.pendingApprovals.filter(
+        (p) => p.workflowId !== workflowId
+      )
+    });
+  }
+
+  // Reject a waiting workflow
+  @callable()
+  async reject(workflowId: string, reason: string): Promise<void> {
+    await this.rejectWorkflow(workflowId, { reason });
+
+    this.setState({
+      ...this.state,
+      pendingApprovals: this.state.pendingApprovals.filter(
+        (p) => p.workflowId !== workflowId
+      )
+    });
+  }
+
+  // Track workflow progress
+  async onWorkflowProgress(
+    workflowName: string,
+    workflowId: string,
+    progress: unknown
+  ): Promise<void> {
+    const p = progress as { step: string; status: string };
+
+    if (p.step === "approval" && p.status === "pending") {
+      // Add to pending approvals list
+      this.setState({
+        ...this.state,
+        pendingApprovals: [
+          ...this.state.pendingApprovals,
+          { workflowId, requestedAt: Date.now() }
+        ]
+      });
+    }
+  }
+}
+```
+
+### Timeout Handling
+
+Set timeouts to prevent workflows from waiting indefinitely:
+
+```typescript
+const approval = await this.waitForApproval(step, {
+  timeout: "7 days" // or "1 hour", "30 minutes", etc.
+});
+```
+
+If the timeout expires, the workflow continues without approval data. Handle this case:
+
+```typescript
+const approval = await this.waitForApproval<{ approvedBy: string }>(step, {
+  timeout: "24 hours"
+});
+
+if (!approval) {
+  // Timeout expired - escalate or auto-reject
+  await step.reportError("Approval timeout - escalating to manager");
+  throw new Error("Approval timeout");
+}
+```
+
+For more details, see [Workflows Integration](./workflows.md).
+
+## AI Tool Approval with `needsApproval`
+
+When building AI chat agents, you often want humans to approve certain tool calls before execution. The AI SDK's `needsApproval` option pauses tool execution until the user approves or rejects.
+
+### Server
+
+Define tools with `needsApproval` to require human confirmation:
+
+```typescript
+import { AIChatAgent } from "@cloudflare/ai-chat";
+import { createWorkersAI } from "workers-ai-provider";
+import { streamText, tool, convertToModelMessages } from "ai";
+import { z } from "zod";
+
+export class MyAgent extends AIChatAgent {
+  async onChatMessage() {
+    const workersai = createWorkersAI({ binding: this.env.AI });
+
+    const result = streamText({
+      model: workersai("@cf/moonshotai/kimi-k2.7-code"),
+      messages: await convertToModelMessages(this.messages),
+      tools: {
+        // Tool with conditional approval
+        processPayment: tool({
+          description: "Process a payment",
+          inputSchema: z.object({
+            amount: z.number(),
+            recipient: z.string()
+          }),
+          // Approval required for amounts over $100
+          needsApproval: async ({ amount }) => amount > 100,
+          execute: async ({ amount, recipient }) => {
+            return await chargeCard(amount, recipient);
+          }
+        }),
+
+        // Tool that always requires approval
+        deleteAccount: tool({
+          description: "Delete a user account",
+          inputSchema: z.object({ userId: z.string() }),
+          needsApproval: true,
+          execute: async ({ userId }) => {
+            return await deleteUser(userId);
+          }
+        }),
+
+        // Tool that executes automatically (no approval)
+        getWeather: tool({
+          description: "Get weather for a city",
+          inputSchema: z.object({ city: z.string() }),
+          execute: async ({ city }) => fetchWeather(city)
+        })
+      },
+      maxSteps: 5
+    });
+
+    return result.toUIMessageStreamResponse();
+  }
+}
+```
+
+### Client
+
+Handle approval requests with `addToolApprovalResponse`:
+
+```tsx
+import { useAgent } from "agents/react";
+import { useAgentChat } from "@cloudflare/ai-chat/react";
+import { isToolUIPart, getToolName } from "ai";
+
+function Chat() {
+  const agent = useAgent({ agent: "MyAgent" });
+  const { messages, sendMessage, addToolApprovalResponse } = useAgentChat({
+    agent
+  });
+
+  return (
+    <div>
+      {messages.map((message) => (
+        <div key={message.id}>
+          {message.parts?.map((part, i) => {
+            if (part.type === "text") {
+              return <p key={i}>{part.text}</p>;
+            }
+
+            if (isToolUIPart(part)) {
+              // Tool waiting for approval
+              if ("approval" in part && part.state === "approval-requested") {
+                const approvalId = part.approval?.id;
+                return (
+                  <div key={part.toolCallId} className="approval-card">
+                    <p>
+                      Approve <strong>{getToolName(part)}</strong> with{" "}
+                      {JSON.stringify(part.input)}?
+                    </p>
+                    <button
+                      onClick={() =>
+                        addToolApprovalResponse({
+                          id: approvalId,
+                          approved: true
+                        })
+                      }
+                    >
+                      Approve
+                    </button>
+                    <button
+                      onClick={() =>
+                        addToolApprovalResponse({
+                          id: approvalId,
+                          approved: false
+                        })
+                      }
+                    >
+                      Reject
+                    </button>
+                  </div>
+                );
+              }
+
+              // Tool was denied
+              if (part.state === "output-denied") {
+                return (
+                  <div key={part.toolCallId}>{getToolName(part)}: Denied</div>
+                );
+              }
+
+              // Tool completed
+              if (part.state === "output-available") {
+                return (
+                  <div key={part.toolCallId}>
+                    {getToolName(part)}: {JSON.stringify(part.output)}
+                  </div>
+                );
+              }
+            }
+
+            return null;
+          })}
+        </div>
+      ))}
+    </div>
+  );
+}
+```
+
+### Custom denial messages with `addToolOutput`
+
+When a user rejects a tool, `addToolApprovalResponse({ id, approved: false })` sets the tool state to `output-denied` with a generic "Tool execution denied." message. If you need to give the LLM a more specific reason for the denial, use `addToolOutput` with `state: "output-error"` instead:
+
+```tsx
+const { addToolOutput } = useAgentChat({ agent });
+
+// Reject with a custom error message
+addToolOutput({
+  toolCallId: part.toolCallId,
+  state: "output-error",
+  errorText: "User declined: insufficient budget for this quarter"
+});
+```
+
+This sends a `tool_result` to the LLM with your custom error text, so it can respond appropriately (e.g. suggest an alternative, ask clarifying questions). The `addToolOutput` function also works for tools in `approval-requested` or `approval-responded` states, not just `input-available`.
+
+`addToolApprovalResponse` (with `approved: false`) auto-continues the conversation when `autoContinueAfterToolResult` is enabled (the default), so the LLM sees the denial and can respond naturally.
+
+`addToolOutput` with `state: "output-error"` does **not** auto-continue — it gives you full control over what happens next. If you want the LLM to respond to the error, call `sendMessage()` afterward.
+
+See the complete example: [guides/human-in-the-loop/](https://github.com/cloudflare/agents/tree/main/guides/human-in-the-loop)
+
+### Surviving restarts while waiting for a human
+
+A Durable Object can be evicted at any time (a deploy, an inactivity timeout, a resource limit), including while a turn is paused on an approval prompt or a client-side tool call. When [`chatRecovery`](./chat-agents.md#stream-recovery) is enabled (the default for `Think`), the SDK recognizes that such a turn is _waiting on the human_, not stuck, and does **not** seal it: the no-progress window, attempt cap, `maxRecoveryWork`, and `shouldKeepRecovering` are all suspended while the interaction is pending. Recovery parks the turn instead of failing it, and the user's eventual approval or `tool_result` resumes the conversation through the normal continuation path. A user who takes minutes to respond to a prompt that was interrupted by a deploy therefore does not see a spurious "session interrupted" error.
+
+This protection applies to interactions only the client can resolve — `approval-requested` parts and `input-available` parts for client-side tools (those without a server `execute`). A server tool whose `execute()` was killed mid-flight is a genuine orphan and recovers through the normal transcript-repair path instead.
+
+## Client-Side Tool Execution with `onToolCall`
+
+For tools that need browser APIs (geolocation, camera, clipboard) or user interaction, define the tool on the server without an `execute` function and handle it on the client with `onToolCall`:
+
+### Server
+
+```typescript
+export class MyAgent extends AIChatAgent {
+  async onChatMessage() {
+    const workersai = createWorkersAI({ binding: this.env.AI });
+
+    const result = streamText({
+      model: workersai("@cf/moonshotai/kimi-k2.7-code"),
+      messages: await convertToModelMessages(this.messages),
+      tools: {
+        // No execute function - client handles via onToolCall
+        getUserLocation: tool({
+          description: "Get the user's current location from their browser",
+          inputSchema: z.object({})
+        })
+      },
+      maxSteps: 3
+    });
+
+    return result.toUIMessageStreamResponse();
+  }
+}
+```
+
+### Client
+
+```tsx
+const { messages, sendMessage } = useAgentChat({
+  agent,
+  onToolCall: async ({ toolCall, addToolOutput }) => {
+    if (toolCall.toolName === "getUserLocation") {
+      const position = await new Promise((resolve, reject) => {
+        navigator.geolocation.getCurrentPosition(resolve, reject);
+      });
+      addToolOutput({
+        toolCallId: toolCall.toolCallId,
+        output: {
+          lat: position.coords.latitude,
+          lng: position.coords.longitude
+        }
+      });
+    }
+  }
+});
+```
+
+The server receives the tool output via `CF_AGENT_TOOL_RESULT` and can auto-continue the conversation (with `maxSteps > 1`), letting the LLM respond to the location data in the same turn.
+
+### OpenAI Agents SDK Pattern
+
+When using the [OpenAI Agents SDK](https://openai.github.io/openai-agents-js/), use the `needsApproval` function for conditional approval:
+
+```typescript
+import { Agent } from "agents";
+import { tool, run } from "@openai/agents";
+
+export class WeatherAgent extends Agent<Env, AgentState> {
+  async processQuery(query: string) {
+    const weatherTool = tool({
+      name: "get_weather",
+      description: "Get weather for a location",
+      parameters: z.object({ location: z.string() }),
+
+      // Conditional approval - only for certain locations
+      needsApproval: async (_context, { location }) => {
+        return location === "San Francisco"; // Require approval for SF
+      },
+
+      execute: async ({ location }) => {
+        const conditions = ["sunny", "cloudy", "rainy"];
+        return conditions[Math.floor(Math.random() * conditions.length)];
+      }
+    });
+
+    const result = await run(this.openai, {
+      model: "gpt-4o",
+      tools: [weatherTool],
+      input: query
+    });
+
+    return result;
+  }
+}
+```
+
+See the complete example: [openai-sdk/human-in-the-loop/](https://github.com/cloudflare/agents/tree/main/openai-sdk/human-in-the-loop)
+
+### MCP Elicitation
+
+When building MCP servers with `McpAgent`, you can request additional user input during tool execution using **elicitation**. The MCP client (like Claude Desktop) renders a form based on your JSON Schema and returns the user's response.
+
+```typescript
+import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
+import { Agent } from "agents";
+
+export class MyMcpAgent extends Agent<Env, State> {
+  server = new McpServer({
+    name: "my-mcp-server",
+    version: "1.0.0"
+  });
+
+  onStart() {
+    this.server.registerTool(
+      "increase-counter",
+      {
+        description: "Increase the counter by a user-specified amount",
+        inputSchema: {
+          confirm: z.boolean().describe("Do you want to increase the counter?")
+        }
+      },
+      async ({ confirm }, extra) => {
+        if (!confirm) {
+          return { content: [{ type: "text", text: "Cancelled." }] };
+        }
+
+        // Request additional input from the user
+        const userInput = await this.server.server.elicitInput(
+          {
+            message: "By how much do you want to increase the counter?",
+            requestedSchema: {
+              type: "object",
+              properties: {
+                amount: {
+                  type: "number",
+                  title: "Amount",
+                  description: "The amount to increase the counter by"
+                }
+              },
+              required: ["amount"]
+            }
+          },
+          { relatedRequestId: extra.requestId }
+        );
+
+        // Check if user accepted or cancelled
+        if (userInput.action !== "accept" || !userInput.content) {
+          return { content: [{ type: "text", text: "Cancelled." }] };
+        }
+
+        // Use the input
+        const amount = Number(userInput.content.amount);
+        this.setState({
+          ...this.state,
+          counter: this.state.counter + amount
+        });
+
+        return {
+          content: [
+            {
+              type: "text",
+              text: `Counter increased by ${amount}, now at ${this.state.counter}`
+            }
+          ]
+        };
+      }
+    );
+  }
+}
+```
+
+**Key differences from other patterns:**
+
+- Used by **MCP servers** exposing tools to clients, not agents calling tools
+- Uses **JSON Schema** for structured form-based input
+- The **MCP client** (Claude Desktop, etc.) handles UI rendering
+- Returns `{ action: "accept" | "decline", content: {...} }`
+
+See the complete example: [examples/mcp-elicitation/](https://github.com/cloudflare/agents/tree/main/examples/mcp-elicitation)
+
+## State Patterns for Approvals
+
+Track pending approvals in agent state for UI rendering and persistence:
+
+```typescript
+type PendingApproval = {
+  id: string;
+  workflowId?: string;
+  type: "expense" | "publish" | "delete";
+  description: string;
+  amount?: number;
+  requestedBy: string;
+  requestedAt: number;
+  expiresAt?: number;
+};
+
+type ApprovalRecord = {
+  id: string;
+  approvalId: string;
+  decision: "approved" | "rejected";
+  decidedBy: string;
+  decidedAt: number;
+  reason?: string;
+};
+
+type ApprovalState = {
+  pending: PendingApproval[];
+  history: ApprovalRecord[];
+};
+```
+
+### Multi-Approver Patterns
+
+For sensitive operations requiring multiple approvers:
+
+```typescript
+type MultiApproval = {
+  id: string;
+  requiredApprovals: number;  // e.g., 2
+  currentApprovals: Array<{
+    userId: string;
+    approvedAt: number;
+  }>;
+  rejections: Array<{
+    userId: string;
+    rejectedAt: number;
+    reason: string;
+  }>;
+};
+
+@callable()
+async approveMulti(approvalId: string, userId: string): Promise<boolean> {
+  const approval = this.state.pending.find(p => p.id === approvalId);
+  if (!approval) throw new Error("Approval not found");
+
+  // Add this user's approval
+  approval.currentApprovals.push({ userId, approvedAt: Date.now() });
+
+  // Check if we have enough approvals
+  if (approval.currentApprovals.length >= approval.requiredApprovals) {
+    // Execute the approved action
+    await this.executeApprovedAction(approval);
+    return true;
+  }
+
+  this.setState({ ...this.state });
+  return false; // Still waiting for more approvals
+}
+```
+
+## Timeouts and Escalation
+
+### Setting Approval Timeouts
+
+```typescript
+const approval = await this.waitForApproval(step, {
+  timeout: "24 hours"
+});
+```
+
+### Escalation with Scheduling
+
+Use `schedule()` to set up escalation reminders:
+
+```typescript
+@callable()
+async submitForApproval(request: ApprovalRequest): Promise<string> {
+  const approvalId = crypto.randomUUID();
+
+  // Add to pending
+  this.setState({
+    ...this.state,
+    pending: [...this.state.pending, { id: approvalId, ...request }]
+  });
+
+  // Schedule reminder after 4 hours
+  await this.schedule(
+    Date.now() + 4 * 60 * 60 * 1000,
+    "sendReminder",
+    { approvalId }
+  );
+
+  // Schedule escalation after 24 hours
+  await this.schedule(
+    Date.now() + 24 * 60 * 60 * 1000,
+    "escalateApproval",
+    { approvalId }
+  );
+
+  return approvalId;
+}
+```
+
+## Complete Examples
+
+| Pattern           | Location                                                                                                     | Description                                        |
+| ----------------- | ------------------------------------------------------------------------------------------------------------ | -------------------------------------------------- |
+| Workflow approval | [examples/workflows/](https://github.com/cloudflare/agents/tree/main/examples/workflows)                     | Multi-step task processing with approval gate      |
+| AIChatAgent tools | [guides/human-in-the-loop/](https://github.com/cloudflare/agents/tree/main/guides/human-in-the-loop)         | Chat tool approval with needsApproval + onToolCall |
+| OpenAI Agents SDK | [openai-sdk/human-in-the-loop/](https://github.com/cloudflare/agents/tree/main/openai-sdk/human-in-the-loop) | Conditional tool approval with modal               |
+| MCP Elicitation   | [examples/mcp-elicitation/](https://github.com/cloudflare/agents/tree/main/examples/mcp-elicitation)         | MCP server requesting structured user input        |
+
+For detailed API documentation, see:
+
+- [Workflows](./workflows.md) - `waitForApproval()`, `approveWorkflow()`, `rejectWorkflow()`
+- [MCP Servers](./mcp-servers.md) - `elicitInput()` for MCP elicitation
+- [Callable Methods](./callable-methods.md) - `@callable()` decorator for approval endpoints