@nextsparkjs/core 0.1.0-beta.2 → 0.1.0-beta.4

package/templates/contents/plugins/langchain/docs/03-orchestration/01-graph-orchestrator.md

@@ -1,809 +0,0 @@
-# Graph-Based Orchestration
-
-This guide covers the **LangGraph-based orchestrator**, the recommended approach for multi-agent systems. It replaces inefficient ReAct loops with an explicit state machine, achieving 25-50x faster execution with deterministic flow.
-
-> **Note**: For the legacy ReAct-based approach, see [Legacy Orchestration](./02-legacy-react.md) (deprecated).
-
-## Why Graph-Based Orchestration?
-
-The traditional ReAct (Reasoning + Acting) pattern has significant drawbacks at scale:
-
-### The Problem with ReAct Loops
-
-```
-User: "Show my tasks and find StartupXYZ account number"
-
-┌─────────────────────────────────────────────────────────────┐
-│ ORCHESTRATOR ReAct Loop (~10-20 iterations)                 │
-│ ├─ Think → Act → Observe → Think → Act...                   │
-│ └─ Decide: route_to_task                                    │
-│     ↓                                                       │
-│     ┌─────────────────────────────────────────────────┐     │
-│     │ TASK-ASSISTANT ReAct Loop (~10-15 iterations)   │     │
-│     │ └─ Think → Act → Observe → Think...             │     │
-│     └─────────────────────────────────────────────────┘     │
-│     ↓ (returns to orchestrator)                             │
-│ ├─ Think → decide second part                               │
-│ └─ Decide: route_to_customer                                │
-│     ↓                                                       │
-│     ┌─────────────────────────────────────────────────┐     │
-│     │ CUSTOMER-ASSISTANT ReAct Loop (~10-15 iter)     │     │
-│     └─────────────────────────────────────────────────┘     │
-└─────────────────────────────────────────────────────────────┘
-
-TOTAL: 50+ LLM iterations → TIMEOUT (2-5 minutes)
-```
-
-**Issues:**
-- **50+ LLM calls** for a simple multi-intent request
-- **Recursion limit errors** from nested loops
-- **Non-deterministic** - different paths each time
-- **Expensive** - each iteration costs tokens
-- **Slow** - 2-5 minute response times
-
-### The Solution: Explicit State Machine
-
-```
-User: "Show my tasks and find StartupXYZ account number"
-
-┌─────────────────────────────────────────────────────────────┐
-│                         START                                │
-│                           ↓                                  │
-│                  ┌─────────────┐                            │
-│                  │   ROUTER    │ ← 1 LLM call               │
-│                  │   (Intent)  │   (structured output)       │
-│                  └──────┬──────┘                            │
-│                         ↓                                    │
-│         ┌───────────────┴───────────────┐                   │
-│         ↓                               ↓                   │
-│   ┌──────────────┐              ┌──────────────┐           │
-│   │ TASK_HANDLER │              │ CUST_HANDLER │           │
-│   │   (0 LLM)    │              │   (0 LLM)    │           │
-│   └───────┬──────┘              └───────┬──────┘           │
-│           ↓                             ↓                   │
-│           └─────────────┬───────────────┘                   │
-│                         ↓                                    │
-│                  ┌─────────────┐                            │
-│                  │  COMBINER   │ ← 1 LLM call               │
-│                  │ (Response)  │   (optional)               │
-│                  └──────┬──────┘                            │
-│                         ↓                                    │
-│                        END                                   │
-└─────────────────────────────────────────────────────────────┘
-
-TOTAL: 2 LLM calls → 2-3 seconds
-```
-
-**Benefits:**
-- **1-2 LLM calls** instead of 50+
-- **No recursion** - explicit transitions
-- **Deterministic** - same input = same path
-- **Cost effective** - 25-50x fewer tokens
-- **Fast** - 2-10 second responses
-
----
-
-## Theoretical Foundations
-
-### What is LangGraph?
-
-LangGraph is a framework for building **stateful, multi-step AI applications** using explicit graphs. Unlike ReAct patterns where the LLM decides what to do next in a loop, LangGraph defines explicit states and transitions.
-
-Key concepts:
-
-| Concept | Description |
-|---------|-------------|
-| **State** | The data that flows through the graph |
-| **Node** | A function that transforms state |
-| **Edge** | Connection between nodes |
-| **Conditional Edge** | Edge that chooses next node based on state |
-| **Channel** | How state updates are merged |
-
-### State Machines vs ReAct Loops
-
-| Aspect | ReAct Loop | State Machine |
-|--------|------------|---------------|
-| **Control Flow** | LLM decides | Code defines |
-| **Predictability** | Variable | Deterministic |
-| **LLM Calls** | O(n) per step | O(1) total |
-| **Debugging** | Hard (black box) | Easy (explicit) |
-| **Parallelism** | Limited | Native support |
-
-### Why Deterministic Flow Matters
-
-1. **Reproducibility**: Same input produces same execution path
-2. **Debugging**: You can trace exactly which nodes executed
-3. **Testing**: Unit test each node independently
-4. **Cost Control**: Predictable token usage
-5. **SLA Compliance**: Guaranteed response times
-
----
-
-## Architecture
-
-### Graph Overview
-
-```
-                    ┌──────────────────────────────────────────────────────┐
-                    │                    ORCHESTRATOR GRAPH                 │
-                    │                                                       │
-   User Input ──────┤                                                       │
-                    │   ┌─────────┐                                        │
-                    │   │  START  │                                        │
-                    │   └────┬────┘                                        │
-                    │        │                                             │
-                    │        ▼                                             │
-                    │   ┌──────────┐     Structured Output                 │
-                    │   │  ROUTER  │ ◄── (Zod Schema)                      │
-                    │   │  (LLM)   │     Intent Classification             │
-                    │   └────┬─────┘                                       │
-                    │        │                                             │
-                    │   ┌────┴────┬────────────┬─────────────┐            │
-                    │   ▼         ▼            ▼             ▼            │
-                    │ greeting  clarify    single       multi            │
-                    │   │         │        intent       intent           │
-                    │   │         │           │            │              │
-                    │   │         │           ▼            ▼              │
-                    │   │         │     ┌──────────┐  ┌──────────┐       │
-                    │   │         │     │ HANDLER  │  │ HANDLER  │ ...   │
-                    │   │         │     │ (No LLM) │  │ (No LLM) │       │
-                    │   │         │     └────┬─────┘  └────┬─────┘       │
-                    │   │         │          │             │              │
-                    │   │         │          └──────┬──────┘              │
-                    │   │         │                 ▼                     │
-                    │   │         │          ┌──────────┐                 │
-                    │   └─────────┴─────────►│ COMBINER │                 │
-                    │                        │  (LLM)   │                 │
-                    │                        └────┬─────┘                 │
-                    │                             │                       │
-                    │                             ▼                       │
-                    │                        ┌────────┐                   │
-   Response ◄───────┤                        │  END   │                   │
-                    │                        └────────┘                   │
-                    └──────────────────────────────────────────────────────┘
-```
-
-### Node Responsibilities
-
-| Node | Purpose | LLM Calls | Input | Output |
-|------|---------|-----------|-------|--------|
-| **Router** | Classify user intent | 1 | User message | Intent[] |
-| **Task Handler** | Execute task operations | 0 | Intent | JSON result |
-| **Customer Handler** | Execute customer operations | 0 | Intent | JSON result |
-| **Page Handler** | Execute page operations | 0 | Intent | JSON result |
-| **Combiner** | Synthesize response | 0-1 | All results | User text |
-
-### Data Flow
-
-1. **User Input** → Raw message string
-2. **Router** → Parses to structured `Intent[]`
-3. **Handlers** → Execute operations, return JSON
-4. **Combiner** → Converts JSON to natural language
-5. **Response** → Text for user
-
----
-
-## Nodes Reference
-
-### Router Node
-
-The router is the "brain" that classifies user intent in a single LLM call.
-
-**Location**: `lib/graph/nodes/router.ts`
-
-**Features**:
-- **Structured Output**: Uses Zod schema for reliable JSON extraction
-- **Multi-Intent**: Detects multiple intents in one message
-- **Retry Logic**: Handles malformed responses from local models
-- **Provider Agnostic**: Works with OpenAI, Anthropic, Ollama, LM Studio
-
-**Output Schema**:
-
-```typescript
-const RouterOutputSchema = z.object({
-    intents: z.array(z.object({
-        type: z.enum(['task', 'customer', 'page', 'greeting', 'clarification']),
-        action: z.enum(['list', 'create', 'update', 'delete', 'search', 'get', 'unknown']),
-        parameters: z.record(z.unknown()),
-        originalText: z.string(),
-    })),
-    needsClarification: z.boolean(),
-    clarificationQuestion: z.string().nullable(),
-})
-```
-
-**Examples**:
-
-```
-Input: "Show my tasks"
-Output: {
-    intents: [{
-        type: "task",
-        action: "list",
-        parameters: {},
-        originalText: "Show my tasks"
-    }],
-    needsClarification: false
-}
-
-Input: "Create task 'Buy milk' high priority and find customer ABC"
-Output: {
-    intents: [
-        { type: "task", action: "create", parameters: { title: "Buy milk", priority: "high" } },
-        { type: "customer", action: "search", parameters: { query: "ABC" } }
-    ],
-    needsClarification: false
-}
-```
-
-### Handler Nodes
-
-Handlers execute operations **without LLM calls** - they directly call entity services.
-
-#### Task Handler
-
-**Location**: `lib/graph/nodes/task-handler.ts`
-
-**Supported Actions**:
-- `list` - Get all tasks with optional filters
-- `get` - Get single task by ID
-- `create` - Create new task
-- `update` - Update task fields
-- `search` - Find tasks by query
-- `delete` - Delete task
-
-**Example**:
-
-```typescript
-// Intent: { type: "task", action: "create", parameters: { title: "Buy milk", priority: "high" } }
-
-// Handler calls:
-await TasksService.create(userId, teamId, { title: "Buy milk", priority: "high" })
-
-// Returns:
-{
-    success: true,
-    operation: "create",
-    data: { id: "123", title: "Buy milk", priority: "high", status: "todo" },
-    message: "Created task: Buy milk"
-}
-```
-
-#### Customer Handler
-
-**Location**: `lib/graph/nodes/customer-handler.ts`
-
-**Supported Actions**:
-- `list` - Get all customers
-- `get` - Get single customer
-- `create` - Create new customer
-- `update` - Update customer fields
-- `search` - Find customers by name, email, phone, account
-- `delete` - Delete customer
-
-#### Page Handler
-
-**Location**: `lib/graph/nodes/page-handler.ts`
-
-**Supported Actions**:
-- `list` - Get published pages
-- `get` - Get page by ID or slug
-- `search` - Find pages by title
-
-> **Note**: Create/Update/Delete for pages returns error directing to Page Builder UI.
-
-### Combiner Node
-
-The combiner converts JSON handler results into natural language.
-
-**Location**: `lib/graph/nodes/combiner.ts`
-
-**Optimization**: For simple single-intent operations, uses **template-based responses** without LLM:
-
-```typescript
-// Template response (no LLM needed):
-// Intent: list 3 tasks
-"Found 3 task(s):
-• Buy milk (high) - todo
-• Call client (medium) - in-progress
-• Review docs (low) - done"
-
-// LLM response (multi-intent or complex):
-// Uses GPT-4o-mini to synthesize natural response
-```
-
-**When LLM is used**:
-- Multiple handler results (task + customer)
-- List/search with >5 items
-- Complex data requiring summarization
-
-**When templates are used**:
-- Single intent operations
-- Simple list/search with <=5 items
-- CRUD confirmations
-
----
-
-## State Schema
-
-### OrchestratorState
-
-The complete state that flows through the graph:
-
-```typescript
-interface OrchestratorState {
-    // ---- Input (immutable) ----
-    input: string                          // User message
-    sessionId: string                      // Session identifier
-    context: AgentContext                  // userId, teamId
-    conversationHistory: BaseMessage[]     // Recent messages (last 5)
-
-    // ---- Router Output ----
-    intents: Intent[]                      // Classified intents
-    needsClarification: boolean            // If true, ask user
-    clarificationQuestion?: string         // Question to ask
-
-    // ---- Handler Outputs ----
-    handlerResults: {                      // JSON from each handler
-        task?: TaskHandlerResult
-        customer?: CustomerHandlerResult
-        page?: PageHandlerResult
-    }
-    completedHandlers: IntentType[]        // Which handlers ran
-
-    // ---- Final Output ----
-    finalResponse: string | null           // Text for user
-    error: string | null                   // Error message
-
-    // ---- Tracing ----
-    traceId?: string                       // Observability trace
-}
-```
-
-### Intent Type
-
-```typescript
-interface Intent {
-    type: 'task' | 'customer' | 'page' | 'greeting' | 'clarification'
-    action: 'list' | 'create' | 'update' | 'delete' | 'search' | 'get' | 'unknown'
-    parameters: Record<string, unknown>
-    originalText: string
-    confidence?: number
-}
-```
-
-### Handler Results
-
-```typescript
-interface TaskHandlerResult {
-    success: boolean
-    operation: 'list' | 'create' | 'update' | 'delete' | 'search' | 'get'
-    data: TaskData[] | TaskData | null
-    count?: number
-    message: string
-    error?: string
-}
-```
-
----
-
-## Routing Logic
-
-### From Router to Handlers
-
-The `routeByIntents` function determines execution path:
-
-```typescript
-function routeByIntents(state: OrchestratorState): RouterRoute {
-    const intentTypes = state.intents.map(i => i.type)
-
-    // Handle special cases
-    if (state.error) return 'error'
-    if (state.needsClarification) return 'clarification'
-    if (intentTypes.length === 1 && intentTypes[0] === 'greeting') return 'greeting'
-
-    // Determine handler combination
-    const hasTask = intentTypes.includes('task')
-    const hasCustomer = intentTypes.includes('customer')
-    const hasPage = intentTypes.includes('page')
-
-    if (hasTask && hasCustomer && hasPage) return 'all_handlers'
-    if (hasTask && hasCustomer) return 'task_and_customer'
-    if (hasTask && hasPage) return 'task_and_page'
-    if (hasCustomer && hasPage) return 'customer_and_page'
-    if (hasTask) return 'task_only'
-    if (hasCustomer) return 'customer_only'
-    if (hasPage) return 'page_only'
-
-    return 'clarification'
-}
-```
-
-### Handler Sequencing
-
-For multi-intent requests, handlers execute sequentially:
-
-```
-task_and_customer: Task Handler → Customer Handler → Combiner
-task_and_page:     Task Handler → Page Handler → Combiner
-all_handlers:      Task Handler → Customer Handler → Page Handler → Combiner
-```
-
-Each handler's output is **merged** into `handlerResults`, preserving all results.
-
-### Conditional Transitions
-
-```typescript
-// After task handler
-function routeAfterTask(state): string {
-    if (needsCustomer && !completedCustomer) return 'customer_handler'
-    if (needsPage && !completedPage) return 'page_handler'
-    return 'combiner'
-}
-```
-
----
-
-## Multi-Model Compatibility
-
-### Structured Output Methods
-
-Different LLM providers support different structured output methods:
-
-| Provider | Method | Notes |
-|----------|--------|-------|
-| OpenAI API | `functionCalling` | Most reliable |
-| Anthropic | `functionCalling` | Tool use |
-| Ollama | `functionCalling` | Most models |
-| LM Studio | `jsonSchema` | OpenAI-compatible servers |
-
-The `getStructuredOutputMethod()` helper auto-detects the best method:
-
-```typescript
-import { getStructuredOutputMethod } from '../../providers'
-
-const method = getStructuredOutputMethod({ provider: 'openai' })
-// Returns: 'functionCalling' for real OpenAI, 'jsonSchema' for LM Studio
-
-const structuredModel = model.withStructuredOutput(schema, { method })
-```
-
-### Retry Logic with Zod Validation
-
-Local models (LM Studio, Ollama) may produce malformed JSON. The router includes retry logic:
-
-```typescript
-async function invokeRouterWithRetry(model, messages, method) {
-    for (let attempt = 1; attempt <= 3; attempt++) {
-        // Try 1: Structured output
-        const result = await tryStructuredOutput(model, messages, method)
-        if (result) return result
-
-        // Try 2: Manual JSON parsing (fallback)
-        const manual = await tryManualJsonParsing(model, messages)
-        if (manual) return manual
-
-        // Exponential backoff before retry
-        await sleep(500 * Math.pow(2, attempt - 1))
-    }
-    throw new Error('Router failed after retries')
-}
-
-function tryManualJsonParsing(model, messages) {
-    const result = await model.invoke(messages)
-    const content = result.content as string
-
-    // Extract JSON from markdown code blocks or raw text
-    const json = extractJsonFromResponse(content)
-    const parsed = JSON.parse(json)
-
-    // Validate with Zod
-    return RouterOutputSchema.safeParse(parsed)
-}
-```
-
-### Provider Configuration
-
-Configure providers in theme's `langchain.config.ts`:
-
-```typescript
-'orchestrator': {
-    provider: 'anthropic',              // Claude for router
-    model: 'claude-3-haiku-20240307',
-    temperature: 0.1,                   // Low for consistency
-},
-'task-assistant': {
-    provider: 'openai',                 // GPT-4o-mini for handlers
-    model: 'gpt-4o-mini',
-    temperature: 0.3,
-}
-```
-
----
-
-## Configuration
-
-### Environment Variables
-
-```env
-# Enable graph orchestrator (recommended)
-LANGCHAIN_USE_GRAPH_ORCHESTRATOR=true
-
-# Provider settings
-LANGCHAIN_DEFAULT_PROVIDER=openai
-
-# Debug mode
-LANGCHAIN_PLUGIN_DEBUG=true
-
-# Provider-specific
-OPENAI_API_KEY=sk-...
-ANTHROPIC_API_KEY=sk-ant-...
-```
-
-### Graph Configuration
-
-```typescript
-interface GraphConfig {
-    maxHistoryMessages: number     // Default: 5
-    routerTemperature: number      // Default: 0.1
-    combinerTemperature: number    // Default: 0.3
-    parallelExecution: boolean     // Default: true (future)
-    handlerTimeout: number         // Default: 30000ms
-}
-```
-
-### Usage
-
-```typescript
-import { invokeOrchestrator } from '@/contents/plugins/langchain/lib/graph'
-
-const result = await invokeOrchestrator(
-    message,
-    sessionId,
-    { userId, teamId },
-    conversationHistory,
-    {
-        config: {
-            maxHistoryMessages: 10,
-            routerTemperature: 0.05,
-        },
-        traceId: 'trace-123',
-    }
-)
-```
-
----
-
-## Performance Comparison
-
-### Benchmarks
-
-| Metric | Graph Orchestrator | ReAct Loops | Improvement |
-|--------|-------------------|-------------|-------------|
-| **LLM Calls (single intent)** | 1 | 20-30 | 20-30x |
-| **LLM Calls (multi intent)** | 2 | 50+ | 25-50x |
-| **Response Time (single)** | 2-3s | 10-15s | 5x |
-| **Response Time (multi)** | 3-5s | 60-120s | 20-30x |
-| **Token Cost** | ~500 | ~15,000 | 30x |
-| **Recursion Errors** | Never | Frequent | - |
-| **Timeout Errors** | Never | Common | - |
-
-### Real-World Example
-
-**Query**: "Show my tasks and find the account number for StartupXYZ"
-
-| Metric | Graph | ReAct |
-|--------|-------|-------|
-| Total LLM calls | 2 | 47 |
-| Router calls | 1 | - |
-| Orchestrator iterations | - | 15 |
-| Task agent iterations | - | 12 |
-| Customer agent iterations | - | 20 |
-| Response time | 4.2s | 127s (timeout) |
-| Tokens used | 892 | 23,450 |
-| Estimated cost | $0.002 | $0.047 |
-
----
-
-## Examples
-
-### Single Intent Flow
-
-```typescript
-// User: "List my tasks"
-
-// 1. Router (1 LLM call)
-{
-    intents: [{ type: "task", action: "list", parameters: {} }],
-    needsClarification: false
-}
-
-// 2. Route: task_only → task_handler
-
-// 3. Task Handler (0 LLM calls)
-{
-    handlerResults: {
-        task: {
-            success: true,
-            operation: "list",
-            data: [{ id: "1", title: "Task A" }, ...],
-            count: 5
-        }
-    }
-}
-
-// 4. Combiner (0 LLM calls - uses template)
-{
-    finalResponse: "Found 5 task(s):\n• Task A\n• Task B..."
-}
-
-// TOTAL: 1 LLM call
-```
-
-### Multi-Intent Flow
-
-```typescript
-// User: "Show tasks and find StartupXYZ phone number"
-
-// 1. Router (1 LLM call)
-{
-    intents: [
-        { type: "task", action: "list", parameters: {} },
-        { type: "customer", action: "search", parameters: { query: "StartupXYZ", fields: ["phone"] } }
-    ]
-}
-
-// 2. Route: task_and_customer → task_handler → customer_handler → combiner
-
-// 3. Task Handler (0 LLM)
-// 4. Customer Handler (0 LLM)
-{
-    handlerResults: {
-        task: { success: true, data: [...], count: 3 },
-        customer: { success: true, data: [{ name: "StartupXYZ", phone: "+1-555-1234" }] }
-    }
-}
-
-// 5. Combiner (1 LLM call - multi-result)
-{
-    finalResponse: "Found 3 tasks. StartupXYZ phone number is +1-555-1234."
-}
-
-// TOTAL: 2 LLM calls
-```
-
-### Error Handling
-
-```typescript
-// User: "Update task 999" (non-existent)
-
-// Task Handler returns error:
-{
-    handlerResults: {
-        task: {
-            success: false,
-            operation: "update",
-            data: null,
-            error: "Task not found",
-            message: "Could not find task with ID 999"
-        }
-    }
-}
-
-// Combiner generates friendly error:
-{
-    finalResponse: "I couldn't find that task. Would you like to see your task list?"
-}
-```
-
----
-
-## Debugging
-
-### Enable Debug Mode
-
-```env
-LANGCHAIN_PLUGIN_DEBUG=true
-```
-
-### Debug Output
-
-```
-[Router] Classifying intent for: Show my tasks
-[Router] Classified intents: [{"type":"task","action":"list"}]
-[Task Handler] Processing intent: list
-[Task Handler] Found 5 tasks
-[Combiner] Using template-based response
-```
-
-### Observability
-
-Use the [Observability Dashboard](../04-advanced/01-observability.md) to trace execution:
-
-- View trace timeline with each node
-- See provider/model used per node
-- Inspect input/output at each step
-- Track token usage and costs
-
----
-
-## Migration from ReAct
-
-### Step 1: Enable Graph Mode
-
-```env
-LANGCHAIN_USE_GRAPH_ORCHESTRATOR=true
-```
-
-### Step 2: Configure Providers
-
-Ensure providers are configured in `langchain.config.ts`:
-
-```typescript
-'orchestrator': {
-    provider: 'anthropic',  // Claude for reliable classification
-    model: 'claude-3-haiku-20240307',
-    temperature: 0.1,
-}
-```
-
-### Step 3: Test
-
-```bash
-# Run your test suite
-pnpm test
-
-# Monitor observability dashboard
-# /sector7/ai-observability
-```
-
-### Fallback
-
-The system supports automatic fallback:
-
-```typescript
-// In orchestrator.ts
-if (isGraphOrchestratorEnabled()) {
-    return processWithGraphOrchestrator(message, context)
-}
-return processWithOrchestrator(message, context)  // Legacy ReAct
-```
-
----
-
-## Best Practices
-
-### 1. Keep Router Focused
-
-The router should only classify intent. Keep the system prompt simple and focused on intent extraction.
-
-### 2. Use Low Temperature
-
-```typescript
-routerTemperature: 0.1  // Consistent classification
-combinerTemperature: 0.3  // Slightly creative responses
-```
-
-### 3. Handle All Intent Types
-
-Ensure your handlers cover all possible actions for each entity type.
-
-### 4. Monitor Performance
-
-Use observability to track:
-- Intent classification accuracy
-- Handler execution times
-- Combiner usage (template vs LLM)
-
-### 5. Test Multi-Intent
-
-Always test with multi-intent queries to ensure proper sequencing.
-
----
-
-## Related Documentation
-
-- [Observability & Tracing](../04-advanced/01-observability.md) - Debug and monitor execution
-- [Token Tracking](../04-advanced/02-token-tracking.md) - Cost management
-- [Configuration](../01-getting-started/03-configuration.md) - Provider setup
-- [Legacy Orchestration](./02-legacy-react.md) - ReAct-based approach (deprecated)