@prmichaelsen/acp-visualizer 0.1.0

package/agent/patterns/tanstack-cloudflare.chat-engine.md

@@ -0,0 +1,353 @@
+# Chat Engine
+
+**Category**: Architecture
+**Applicable To**: Adding AI chat bots to TanStack + Cloudflare applications with tool calling, streaming, MCP integration, and message persistence
+**Status**: Stable
+
+---
+
+## Overview
+
+ChatEngine is a provider-agnostic chat orchestration layer that coordinates AI models, message storage, MCP tool servers, and vision processing through dependency-injected interfaces. It handles the complete message lifecycle: token budgeting, system prompt building with prompt injectors, MCP tool discovery and caching, multi-turn tool execution with persistence, streaming responses, message ACL, and conversation management. Designed to be extracted into a standalone package — clients of the `tanstack-cloudflare` ACP package can use this as a clear path to adding chat bots to their application.
+
+---
+
+## When to Use This Pattern
+
+**Use this pattern when:**
+- Adding an AI chat bot to a TanStack + Cloudflare application
+- Building a multi-tool AI assistant with MCP server integration
+- Implementing streaming chat with tool call persistence
+- Creating a white-label chat experience with pluggable AI providers
+
+**Don't use this pattern when:**
+- Building a simple form-based AI query (use direct API call)
+- The application doesn't need real-time streaming or tool calling
+
+---
+
+## Core Principles
+
+1. **Dependency Injection**: All external services (AI, storage, MCP, vision) are injected as interfaces — swap providers without changing orchestration logic
+2. **MCP Caching**: Server connections and tool definitions cached per-user with 24h TTL — avoids expensive RPC on every message
+3. **Tool Persistence**: Tool calls saved as intermediate messages (`is_tool_interaction: true`) so the AI sees prior tool output on continuation
+4. **Token Budgeting**: Heuristic estimation + optional preflight check via `countTokens` API prevents "prompt too long" errors
+5. **Fire-and-Forget Non-Critical Ops**: Analytics, usage tracking, and title generation don't block the response stream
+
+---
+
+## Implementation
+
+### Provider Interfaces
+
+```typescript
+interface ChatEngineDependencies {
+  aiProvider: IAIProvider
+  storageProvider: IStorageProvider
+  mcpProvider: IMCPProvider
+  visionProvider: IVisionProvider
+  logger: ILogger
+  env?: Record<string, unknown>
+}
+
+interface IAIProvider {
+  streamChat(params: {
+    messages: ChatMessage[]
+    systemPrompt: string
+    tools: Tool[]
+    onMessage: (msg: ChatEngineMessage) => void
+    executeTool: (name: string, input: any, id?: string) => Promise<any>
+    signal?: AbortSignal
+  }): Promise<void>
+}
+
+interface IStorageProvider {
+  saveMessage(params): Promise<Message>
+  loadMessages(params): Promise<Message[]>
+  ensureConversation(params): Promise<string>
+  updateConversation(params): Promise<void>
+  addToolCall(params): Promise<string>        // Persist tool invocation
+  updateToolCall(params): Promise<void>       // Update with result
+  getToolCallsForMessages(params): Promise<Map<string, PersistedToolCall[]>>
+}
+
+interface IMCPProvider {
+  getAvailableServers(params): Promise<MCPServer[]>
+  connectToServers(params): Promise<MCPConnection[]>
+  getTools(connections): Promise<Tool[]>       // Cached after first call
+  executeTool(params): Promise<any>
+  disconnect(connections): Promise<void>
+}
+
+interface IVisionProvider {
+  processImagesInMessage(params): Promise<MessageContent>
+}
+```
+
+### Instantiation in ChatRoom DO
+
+```typescript
+class ChatRoom extends DurableObject {
+  private chatEngine: ChatEngine
+  private mcpProvider: MCPProvider  // Persisted for caching
+
+  constructor(state: DurableObjectState, env: Env) {
+    super(state, env)
+    this.mcpProvider = new MCPProvider()
+    this.chatEngine = new ChatEngine({
+      aiProvider: new AnthropicAIProvider(),
+      storageProvider: new FirebaseStorageProvider(),
+      mcpProvider: this.mcpProvider,  // Reused across messages for caching
+      visionProvider: new GoogleVisionProvider(),
+      logger: chatLogger,
+      env: env as unknown as Record<string, unknown>,
+    })
+  }
+}
+```
+
+### Message Processing Pipeline
+
+```
+processMessage(userId, conversationId, message, onMessage, signal)
+
+ 1. Token limit check → reject if subscription exceeded
+ 2. Ensure conversation exists
+ 3. Detect @agent mention → determine if agent should respond
+ 4. Resolve @username mentions → @uid:userId
+ 5. Assign message ACL (visible_to_user_ids)
+ 6. Save user message → emit user_message_saved
+ 7. Process images via vision provider
+ 8. Load message history (ACL-filtered)
+ 9. Get MCP servers + connect (cached 24h)
+10. Fetch tools (cached) + build system prompt (parallel)
+11. Apply tool filters from prompt injectors
+12. Format messages with timestamps + locations
+13. Token-based truncation (60K budget, oldest first)
+14. Preflight check if estimate > 180K (countTokens API)
+15. Stream AI response with tool execution
+16. Save assistant message + tool call records
+17. Generate conversation title (fire-and-forget, 2nd message)
+18. Update conversation metadata
+```
+
+### Tool Execution Flow
+
+```typescript
+executeTool: async (toolName, toolInput, toolCallId) => {
+  // 1. Create persistent record (status: 'pending')
+  const id = await storageProvider.addToolCall({ toolName, status: 'pending', inputs: toolInput })
+
+  // 2. Execute (local tool registry or MCP provider)
+  const result = isLocalTool(toolName)
+    ? await executeLocalTool(toolName, userId, toolInput, env)
+    : await mcpProvider.executeTool({ toolName, toolInput, connections })
+
+  // 3. Update record (status: 'success', output: result)
+  storageProvider.updateToolCall({ id, status: 'success', output: result })
+
+  // 4. Track analytics (fire-and-forget)
+  AnalyticsService.trackServerEvent(userId, 'tool_executed', { tool: toolName })
+
+  return result  // Returned to AI for next turn
+}
+```
+
+### MCP Caching Strategy
+
+```typescript
+class MCPProvider implements IMCPProvider {
+  private serverCache: MCPServer[] = []
+  private connectionCache: MCPConnection[] = []
+  private toolDefsCache: Tool[] = []
+  private toolMapCache: Map<string, MCPConnection> = new Map()
+  private cacheExpiry: number = 0
+  private cacheUserId: string = ''
+  private readonly CACHE_TTL = 24 * 60 * 60 * 1000  // 24 hours
+
+  // Cache invalidated when: user changes, TTL expires, or OAuth connect/disconnect
+}
+```
+
+### Token Budgeting
+
+```typescript
+// Heuristic estimation
+const MESSAGE_TOKEN_BUDGET = 60_000   // ~120K after formatting
+const PREFLIGHT_THRESHOLD = 180_000   // 90% of 200K API limit
+const SAFE_TARGET = 170_000           // Leave room for output
+
+// Estimation ratios
+Text:       ~4 chars/token
+JSON:       ~2.5 chars/token
+Base64 img: ~1 token/300 bytes
+Per-message: 50 token overhead
+
+// TokenRatioService learns from actual token counts
+// Adjusts future estimates based on preflight feedback
+```
+
+### System Prompt Building
+
+System prompt assembled from:
+1. Anti-hallucination preamble (tool call requirements)
+2. Markdown formatting rules
+3. Web tool instructions
+4. **Prompt injectors** (modular, priority-ordered extensions):
+   - Ghost persona injector (priority 0.9, mutex: ghost)
+   - Agent memory injector (priority 0.7, mutex: memory-context)
+   - Space/group context injector
+   - Each returns: `{ content: string, toolFilters?: { allow?, deny? }[] }`
+5. Conversation type context (chat/DM/group behavior)
+
+### Message ACL
+
+```typescript
+// Group/DM: @agent responses visible only to sender
+MessageAclService.assignACL('group', hasAgentMention, userId)
+// → { visible_to_user_ids: [userId], created_for_user_id: userId }
+
+// History filtering: only load messages the user can see
+MessageAclService.filterMessagesByACL(allMessages, userId)
+```
+
+### ChatEngineMessage Types (Streaming Events)
+
+```typescript
+type ChatEngineMessage =
+  | { type: 'chunk'; content: string }
+  | { type: 'tool_call'; toolCall: ToolCall; persistedToolCallId?: string }
+  | { type: 'tool_result'; toolResult: ToolResult }
+  | { type: 'user_message_saved'; message: Message }
+  | { type: 'assistant_message_saved'; message: Message }
+  | { type: 'complete' }
+  | { type: 'cancelled' }
+  | { type: 'error'; error: string }
+  | { type: 'usage'; input_tokens: number; output_tokens: number }
+  | { type: 'token_limit_warning'; percentage: number }
+  | { type: 'progress_start' | 'progress_update' | 'progress_complete' | 'progress_error' }
+  | { type: 'status'; status: string }
+  | { type: 'debug'; message: string }
+```
+
+---
+
+## Adding Chat to a New Application
+
+### Step 1: Implement Provider Interfaces
+
+```typescript
+// Minimal: just AI + storage
+const engine = new ChatEngine({
+  aiProvider: new AnthropicAIProvider(),           // Or OpenAI, etc.
+  storageProvider: new MyDatabaseStorageProvider(), // Firestore, Postgres, etc.
+  mcpProvider: new NoOpMCPProvider(),               // No tools initially
+  visionProvider: new NoOpVisionProvider(),          // No images initially
+  logger: console,
+})
+```
+
+### Step 2: Create a Durable Object Host
+
+```typescript
+class MyChatRoom extends DurableObject {
+  private engine: ChatEngine
+
+  constructor(state, env) {
+    super(state, env)
+    this.engine = new ChatEngine({ /* providers */ })
+  }
+
+  async fetch(request: Request) {
+    // WebSocket upgrade → session management → message routing
+  }
+}
+```
+
+### Step 3: Connect Client via WebSocket
+
+```typescript
+const ws = new ChatWebSocket({
+  userId: user.uid,
+  conversationId: 'main',
+  onMessage: (msg) => {
+    switch (msg.type) {
+      case 'chunk': /* append streaming text */ break
+      case 'tool_call': /* show tool badge */ break
+      case 'complete': /* finalize message */ break
+    }
+  },
+})
+ws.connect()
+```
+
+### Step 4: Add Tools (Optional)
+
+Register local tools or connect MCP servers:
+```typescript
+// Local tools
+registerLocalTool('my_search', { description: '...', input_schema: {...} }, handler)
+
+// MCP servers
+mcpProvider = new MCPProvider()  // Auto-discovers user's connected servers
+```
+
+---
+
+## Anti-Patterns
+
+### Creating New MCPProvider Per Message
+
+```typescript
+// Bad: Loses tool/connection cache on every message
+async handleMessage(data) {
+  const mcp = new MCPProvider()  // New instance = no cache
+  await this.engine.processMessage({ ... })
+}
+
+// Good: Reuse instance across messages (persist in DO)
+constructor() {
+  this.mcpProvider = new MCPProvider()  // Cached connections/tools
+  this.engine = new ChatEngine({ mcpProvider: this.mcpProvider })
+}
+```
+
+### Blocking on Non-Critical Operations
+
+```typescript
+// Bad: Analytics blocks the response stream
+await AnalyticsService.trackServerEvent(userId, 'tool_executed', {...})
+onMessage({ type: 'complete' })
+
+// Good: Fire-and-forget
+AnalyticsService.trackServerEvent(userId, 'tool_executed', {...}).catch(() => {})
+onMessage({ type: 'complete' })
+```
+
+---
+
+## Checklist
+
+- [ ] All providers injected via constructor (no hard dependencies)
+- [ ] MCPProvider instance persisted across messages for caching
+- [ ] Tool calls persisted with `addToolCall`/`updateToolCall`
+- [ ] Message history truncated before AI call (60K token budget)
+- [ ] Preflight check runs if estimate > 180K tokens
+- [ ] System prompt built with prompt injectors (priority-ordered)
+- [ ] Message ACL applied on save and load
+- [ ] Non-critical operations (analytics, titles) are fire-and-forget
+- [ ] AbortSignal passed through for cancellation support
+
+---
+
+## Related Patterns
+
+- **[WebSocket Manager](./tanstack-cloudflare.websocket-manager.md)**: Client-side WebSocket that connects to ChatRoom DO
+- **[Firebase Firestore](./tanstack-cloudflare.firebase-firestore.md)**: IStorageProvider implementation
+- **[Firebase Auth](./tanstack-cloudflare.firebase-auth.md)**: User auth for message ownership
+
+---
+
+**Status**: Stable
+**Last Updated**: 2026-03-14
+**Contributors**: Community

package/agent/patterns/tanstack-cloudflare.confirmation-tokens.md

@@ -0,0 +1,346 @@
+# Confirmation Tokens Pattern
+
+**Category**: Architecture
+**Applicable To**: TanStack Start + Cloudflare Workers applications with AI-initiated mutations
+**Status**: Stable
+
+---
+
+## Overview
+
+The Confirmation Token pattern provides a two-step execution flow for dangerous or irreversible operations, especially when initiated by AI tools. Instead of executing a mutation immediately, the system generates a preview of the action along with a single-use confirmation token. The user (or AI confirmation tool) must explicitly confirm the action by presenting the token, which is then consumed to execute the operation.
+
+This pattern prevents accidental mutations, provides a human-in-the-loop checkpoint for AI operations, and ensures that the exact operation previewed is the one that executes (no TOCTOU race conditions).
+
+---
+
+## When to Use This Pattern
+
+✅ **Use this pattern when:**
+- AI agents can trigger mutations (create, update, delete operations)
+- Operations are irreversible or have significant side effects
+- Want a human-in-the-loop confirmation step
+- Need to prevent accidental execution of dangerous operations
+- Building tools that the AI calls which modify user data
+
+❌ **Don't use this pattern when:**
+- Operations are read-only (no mutations)
+- The user is directly clicking a button (standard UI confirmation is sufficient)
+- Operations are trivially reversible
+- Low-risk operations where speed matters more than safety
+
+---
+
+## Core Principles
+
+1. **Two-Step Execution**: Preview + confirm, never direct execution
+2. **Single-Use Tokens**: Each token can only be consumed once — prevents replay
+3. **User-Scoped**: Tokens validate that the confirming user matches the initiating user
+4. **TTL Expiration**: Tokens expire after a configurable time (default: 5 minutes)
+5. **Tamper-Proof**: Token encodes the exact operation parameters — confirming executes exactly what was previewed
+6. **In-Memory Store**: Tokens stored in memory within the Durable Object session (no database overhead)
+
+---
+
+## Implementation
+
+### Step 1: Define the Token Service
+
+```typescript
+// src/services/confirmation-token.service.ts
+import { randomBytes } from 'node:crypto'
+
+/**
+ * A pending action stored against a confirmation token.
+ * Encodes the exact operation parameters to prevent tampering.
+ */
+export interface PendingAction {
+  type: 'create_group' | 'update_group' | 'generate_group_link' | 'delete_resource'
+  userId: string
+  params: Record<string, unknown>
+  summary: string
+  createdAt: number
+}
+
+const DEFAULT_TTL_MS = 5 * 60 * 1000  // 5 minutes
+
+/**
+ * Module-level singleton — persists across calls within the same Durable Object session.
+ */
+let _instance: ConfirmationTokenService | null = null
+export function getConfirmationTokenService(): ConfirmationTokenService {
+  if (!_instance) _instance = new ConfirmationTokenService()
+  return _instance
+}
+
+export class ConfirmationTokenService {
+  private pending = new Map<string, PendingAction>()
+  private ttlMs: number
+
+  constructor(ttlMs: number = DEFAULT_TTL_MS) {
+    this.ttlMs = ttlMs
+  }
+
+  /**
+   * Generate a confirmation token for a pending action.
+   * Returns a 32-char hex string.
+   */
+  generateToken(action: PendingAction): string {
+    this.cleanup()
+    const token = randomBytes(16).toString('hex')
+    this.pending.set(token, action)
+    return token
+  }
+
+  /**
+   * Consume a confirmation token.
+   * Returns the pending action if valid, null otherwise.
+   * Tokens are single-use — consumed on retrieval.
+   * Validates userId matches the original initiator.
+   */
+  consumeToken(token: string, userId: string): PendingAction | null {
+    const action = this.pending.get(token)
+    if (!action) return null
+
+    // Always delete — single use
+    this.pending.delete(token)
+
+    // Check TTL
+    if (Date.now() - action.createdAt > this.ttlMs) return null
+
+    // Validate userId matches
+    if (action.userId !== userId) return null
+
+    return action
+  }
+
+  /**
+   * Lazy cleanup of expired tokens.
+   */
+  private cleanup(): void {
+    const now = Date.now()
+    for (const [token, action] of this.pending) {
+      if (now - action.createdAt > this.ttlMs) {
+        this.pending.delete(token)
+      }
+    }
+  }
+}
+```
+
+### Step 2: Create a Mutating Tool (Preview + Token)
+
+```typescript
+// src/lib/chat/tools/create-group.ts
+import { getConfirmationTokenService } from '@/services/confirmation-token.service'
+
+export const createGroupTool = {
+  name: 'create_group',
+  description: 'Create a new group conversation. Returns a preview and confirmation token.',
+  input_schema: {
+    type: 'object' as const,
+    properties: {
+      name: { type: 'string', description: 'Group name' },
+      description: { type: 'string', description: 'Group description' },
+    },
+    required: ['name'],
+  },
+
+  async execute(input: { name: string; description?: string }, userId: string) {
+    const tokenService = getConfirmationTokenService()
+
+    // Generate preview + token (don't execute yet)
+    const token = tokenService.generateToken({
+      type: 'create_group',
+      userId,
+      params: { name: input.name, description: input.description },
+      summary: `Create group "${input.name}"`,
+      createdAt: Date.now(),
+    })
+
+    return {
+      preview: {
+        action: 'create_group',
+        name: input.name,
+        description: input.description || '(none)',
+      },
+      confirmation_token: token,
+      message: `I'll create a group called "${input.name}". Please confirm to proceed.`,
+    }
+  },
+}
+```
+
+### Step 3: Create a Confirm Tool (Execute with Token)
+
+```typescript
+// src/lib/chat/tools/confirm.ts
+import { getConfirmationTokenService } from '@/services/confirmation-token.service'
+import { GroupConversationDatabaseService } from '@/services/group-conversation-database.service'
+
+export const confirmTool = {
+  name: 'confirm_action',
+  description: 'Confirm and execute a previously previewed action using its confirmation token.',
+  input_schema: {
+    type: 'object' as const,
+    properties: {
+      confirmation_token: { type: 'string', description: 'The token from the preview step' },
+    },
+    required: ['confirmation_token'],
+  },
+
+  async execute(input: { confirmation_token: string }, userId: string) {
+    const tokenService = getConfirmationTokenService()
+    const action = tokenService.consumeToken(input.confirmation_token, userId)
+
+    if (!action) {
+      return {
+        success: false,
+        error: 'Invalid, expired, or already-used confirmation token.',
+      }
+    }
+
+    // Execute the confirmed action
+    switch (action.type) {
+      case 'create_group': {
+        const { name, description } = action.params as { name: string; description?: string }
+        const group = await GroupConversationDatabaseService.createGroupConversation(
+          userId,
+          { name, description }
+        )
+        return { success: true, group, message: `Group "${name}" created successfully.` }
+      }
+
+      case 'delete_resource': {
+        // ... handle deletion
+      }
+
+      default:
+        return { success: false, error: `Unknown action type: ${action.type}` }
+    }
+  },
+}
+```
+
+---
+
+## Flow Diagram
+
+```
+User: "Create a group called Book Club"
+  ↓
+AI calls create_group tool
+  ↓
+Tool returns: { preview: {...}, confirmation_token: "abc123", message: "Please confirm" }
+  ↓
+AI shows preview to user: "I'll create a group called Book Club. Please confirm."
+  ↓
+User: "Yes, go ahead"
+  ↓
+AI calls confirm_action tool with token "abc123"
+  ↓
+Token consumed → action executed → group created
+  ↓
+AI: "Done! Group 'Book Club' has been created."
+```
+
+---
+
+## Security Properties
+
+| Property | How It's Enforced |
+|----------|-------------------|
+| **Single-use** | Token deleted on consumption |
+| **Time-limited** | TTL check (default 5 min) |
+| **User-scoped** | userId verified on consumption |
+| **Tamper-proof** | Action params encoded in token, not in confirm request |
+| **No replay** | Consumed tokens return null |
+
+---
+
+## Benefits
+
+### 1. Human-in-the-Loop
+AI shows a preview before executing, giving users a chance to cancel.
+
+### 2. Prevents Accidental Mutations
+No direct execution path — always preview first.
+
+### 3. TOCTOU Safety
+The exact operation previewed is what executes (params stored in token, not re-sent).
+
+### 4. Zero Database Overhead
+In-memory token store within the Durable Object session — no database writes.
+
+---
+
+## Trade-offs
+
+### 1. In-Memory Volatility
+**Downside**: If the Durable Object restarts, all pending tokens are lost.
+**Mitigation**: 5-minute TTL means most tokens are consumed quickly. Users can retry.
+
+### 2. Extra Round Trip
+**Downside**: Two tool calls instead of one for every mutation.
+**Mitigation**: Only use for high-impact operations. Read-only tools execute directly.
+
+---
+
+## Anti-Patterns
+
+### ❌ Anti-Pattern 1: Executing Without Token
+
+```typescript
+// ❌ BAD: Tool directly executes mutation
+async execute(input, userId) {
+  const group = await service.createGroup(userId, input)  // No confirmation!
+  return { group }
+}
+
+// ✅ GOOD: Tool returns preview + token
+async execute(input, userId) {
+  const token = tokenService.generateToken({ type: 'create_group', userId, params: input })
+  return { preview: input, confirmation_token: token }
+}
+```
+
+### ❌ Anti-Pattern 2: Re-Sending Params in Confirm
+
+```typescript
+// ❌ BAD: Confirm re-sends params (can be tampered)
+confirm_action({ token: 'abc', name: 'Evil Group' })  // Name changed!
+
+// ✅ GOOD: Params come from the stored token
+const action = tokenService.consumeToken(token, userId)
+const { name } = action.params  // Params from original preview
+```
+
+---
+
+## Related Patterns
+
+- **[ACL Permissions](./tanstack-cloudflare.acl-permissions.md)**: Permission checks happen before token generation
+- **[Durable Objects WebSocket](./tanstack-cloudflare.durable-objects-websocket.md)**: Token service lives within DO session
+- **[API Route Handlers](./tanstack-cloudflare.api-route-handlers.md)**: Can also be used in HTTP API flows
+
+---
+
+## Checklist for Implementation
+
+- [ ] `ConfirmationTokenService` with `generateToken` and `consumeToken`
+- [ ] Tokens are cryptographically random (32-char hex)
+- [ ] Single-use: deleted on consumption
+- [ ] TTL enforced (default 5 minutes)
+- [ ] User ID validated on consumption
+- [ ] Action params stored in token, not re-sent on confirm
+- [ ] Mutating tools return preview + token
+- [ ] Confirm tool consumes token and executes stored action
+- [ ] Expired/invalid tokens return clear error messages
+- [ ] Lazy cleanup of expired tokens on new generation
+
+---
+
+**Status**: Stable - Proven pattern for AI-initiated mutations
+**Recommendation**: Use for all mutating operations triggered by AI tools
+**Last Updated**: 2026-02-28
+**Contributors**: Patrick Michaelsen