@falai/agent 0.8.1 → 0.9.0-alpha-2

package/docs/core/ai-integration/providers.md

@@ -0,0 +1,515 @@
+# AI Provider Integrations
+
+@falai/agent provides unified interfaces for multiple AI providers, enabling seamless switching between models and automatic fallback handling.
+
+## Supported Providers
+
+### OpenAI Provider
+
+Full-featured integration with GPT models, including backup model support and structured outputs.
+
+```typescript
+import { OpenAIProvider } from "@falai/agent";
+
+const provider = new OpenAIProvider({
+  apiKey: process.env.OPENAI_API_KEY,
+  model: "gpt-5",
+  backupModels: ["gpt-4"], // Fallback models
+  config: {
+    temperature: 0.7,
+    max_tokens: 1000,
+  },
+  retryConfig: {
+    timeout: 30000,
+    retries: 3,
+  },
+});
+```
+
+**Features:**
+
+- ✅ GPT-4, GPT-5, and all OpenAI models
+- ✅ Backup model fallback on failures
+- ✅ Structured outputs with JSON Schema
+- ✅ Tool calling support
+- ✅ Streaming responses
+- ✅ Automatic retry logic
+
+### Google Gemini Provider
+
+Integration with Google's Gemini models through Vertex AI or AI Studio.
+
+```typescript
+import { GeminiProvider } from "@falai/agent";
+
+const provider = new GeminiProvider({
+  apiKey: process.env.GEMINI_API_KEY,
+  model: "gemini-pro",
+  config: {
+    safetySettings: [
+      {
+        category: "HARM_CATEGORY_HARASSMENT",
+        threshold: "BLOCK_MEDIUM_AND_ABOVE",
+      },
+    ],
+  },
+});
+```
+
+**Features:**
+
+- ✅ Gemini 2.0, 2.5 Pro, and Flash models
+- ✅ Safety settings and content filtering
+- ✅ Multimodal capabilities (text, images)
+- ✅ Function calling support
+- ✅ Streaming responses
+
+### Anthropic Claude Provider
+
+Integration with Anthropic's Claude models via their API.
+
+```typescript
+import { AnthropicProvider } from "@falai/agent";
+
+const provider = new AnthropicProvider({
+  apiKey: process.env.ANTHROPIC_API_KEY,
+  model: "claude-sonnet-4.5",
+  config: {
+    maxTokens: 4096,
+    temperature: 0.7,
+  },
+});
+```
+
+**Features:**
+
+- ✅ Claude 4 Opus, Sonnet, and Haiku
+- ✅ Constitutional AI safety
+- ✅ Excellent reasoning capabilities
+- ✅ Tool calling support
+- ✅ Streaming responses
+
+### OpenRouter Provider
+
+Unified access to multiple AI models through OpenRouter's API.
+
+```typescript
+import { OpenRouterProvider } from "@falai/agent";
+
+const provider = new OpenRouterProvider({
+  apiKey: process.env.OPENROUTER_API_KEY,
+  model: "anthropic/claude-sonnet-4",
+  siteUrl: "https://your-app.com",
+  siteName: "Your App Name",
+});
+```
+
+**Features:**
+
+- ✅ Access to 100+ AI models
+- ✅ Unified pricing and billing
+- ✅ Automatic model routing
+- ✅ Fallback model support
+- ✅ Streaming responses
+
+## Provider Configuration
+
+### Common Options
+
+All providers support these configuration options:
+
+```typescript
+interface BaseProviderOptions {
+  model: string; // Primary model to use
+  backupModels?: string[]; // Fallback models on failure
+  temperature?: number; // Response randomness (0-2)
+  maxTokens?: number; // Maximum response length
+  timeout?: number; // Request timeout in ms
+  retries?: number; // Number of retry attempts
+}
+```
+
+### Advanced Configuration
+
+```typescript
+const provider = new OpenAIProvider({
+  apiKey: process.env.OPENAI_API_KEY,
+  model: "gpt-5",
+
+  // Model-specific parameters
+  config: {
+    temperature: 0.1, // Low randomness for consistent responses
+    top_p: 0.9, // Nucleus sampling
+    frequency_penalty: 0.1, // Reduce repetition
+    presence_penalty: 0.1, // Encourage topic diversity
+    max_tokens: 2048,
+  },
+
+  // Fallback configuration
+  backupModels: [
+    "gpt-4.1", // Try GPT-4 if turbo fails
+    "gpt-4", // Final fallback
+  ],
+
+  // Retry configuration
+  retryConfig: {
+    timeout: 60000, // 60 second timeout
+    retries: 3, // Retry up to 3 times
+  },
+});
+```
+
+## Unified Interface
+
+All providers implement the same interface:
+
+```typescript
+interface AiProvider {
+  name: string;
+
+  // Synchronous response generation
+  generateMessage(input: GenerateMessageInput): Promise<GenerateMessageOutput>;
+
+  // Streaming response generation
+  generateMessageStream(
+    input: GenerateMessageInput
+  ): AsyncGenerator<GenerateMessageStreamChunk>;
+}
+```
+
+### Input Format
+
+```typescript
+interface GenerateMessageInput<TContext = unknown> {
+  prompt: string; // The prompt to send
+  history: Event[]; // Conversation history
+  context?: TContext; // Additional context data
+  tools?: ToolDefinition[]; // Available tools
+  parameters?: {
+    jsonSchema?: StructuredSchema; // Response schema
+    schemaName?: string; // Schema identifier
+    maxOutputTokens?: number; // Token limit
+    reasoning?: { effort: "low" | "medium" | "high" };
+  };
+  signal?: AbortSignal; // Cancellation support
+}
+```
+
+### Output Format
+
+```typescript
+interface GenerateMessageOutput<TStructured = AgentStructuredResponse> {
+  message: string; // Main response text
+  metadata: {
+    model: string; // Model used
+    tokensUsed: number; // Total tokens consumed
+    promptTokens: number; // Input tokens
+    completionTokens: number; // Output tokens
+    finishReason?: string; // Why generation stopped
+  };
+  structured?: TStructured; // Structured data if schema provided
+}
+```
+
+## Tool Calling Support
+
+All providers support tool calling with consistent interfaces:
+
+```typescript
+const input: GenerateMessageInput = {
+  prompt: "What's the weather in Paris?",
+  history: conversationHistory,
+  tools: [
+    {
+      id: "get_weather",
+      description: "Get current weather for a location",
+      parameters: {
+        type: "object",
+        properties: {
+          location: { type: "string", description: "City name" },
+        },
+        required: ["location"],
+      },
+    },
+  ],
+};
+
+const response = await provider.generateMessage(input);
+
+// Check for tool calls
+if (response.structured?.toolCalls) {
+  for (const toolCall of response.structured.toolCalls) {
+    console.log(`AI wants to call: ${toolCall.toolName}`);
+    console.log(`With arguments:`, toolCall.arguments);
+  }
+}
+```
+
+## Streaming Responses
+
+Real-time response generation with chunked output:
+
+```typescript
+const stream = provider.generateMessageStream(input);
+
+for await (const chunk of stream) {
+  if (chunk.delta) {
+    process.stdout.write(chunk.delta); // Real-time output
+  }
+
+  if (chunk.done) {
+    console.log("\nGeneration complete!");
+    console.log("Total tokens:", chunk.metadata?.tokensUsed);
+    console.log("Structured data:", chunk.structured);
+  }
+}
+```
+
+## Error Handling & Fallbacks
+
+### Automatic Fallbacks
+
+Providers automatically try backup models on failures:
+
+```typescript
+const provider = new OpenAIProvider({
+  apiKey: process.env.OPENAI_API_KEY,
+  model: "gpt-5",
+  backupModels: ["gpt-4.1", "gpt-4"],
+});
+
+// If gpt-4 fails (rate limit, server error, etc.)
+// Automatically tries gpt-4.1, then gpt-4
+const response = await provider.generateMessage(input);
+```
+
+### Error Classification
+
+Different error types trigger different fallback behaviors:
+
+```typescript
+// Server errors (5xx) → Try backup models
+// Rate limits (429) → Try backup models
+// Invalid requests (4xx) → Don't retry
+// Timeouts → Retry with same model
+```
+
+### Custom Error Handling
+
+```typescript
+try {
+  const response = await provider.generateMessage(input);
+} catch (error) {
+  if (error.code === "rate_limit_exceeded") {
+    // Handle rate limiting
+    await delay(60000); // Wait 1 minute
+    return retryWithBackoff();
+  }
+
+  if (error.code === "model_not_available") {
+    // Switch to different provider
+    return fallbackProvider.generateMessage(input);
+  }
+
+  throw error;
+}
+```
+
+## Performance Optimization
+
+### Connection Reuse
+
+Reuse provider instances for better performance:
+
+```typescript
+// ✅ Good: Reuse provider instance
+const provider = new OpenAIProvider({ apiKey, model: "gpt-4" });
+
+const agent1 = new Agent({ provider });
+const agent2 = new Agent({ provider }); // Same instance
+
+// ❌ Bad: Create new instance each time
+const agent3 = new Agent({
+  provider: new OpenAIProvider({ apiKey, model: "gpt-4" }),
+});
+```
+
+### Request Batching
+
+Group related requests to reduce latency:
+
+```typescript
+// Process multiple conversations efficiently
+const responses = await Promise.all([
+  agent1.respond({ history: history1 }),
+  agent2.respond({ history: history2 }),
+  agent3.respond({ history: history3 }),
+]);
+```
+
+### Caching
+
+Cache responses for similar prompts:
+
+```typescript
+const cache = new Map();
+
+function getCachedResponse(prompt: string, context: any) {
+  const key = hash(`${prompt}-${JSON.stringify(context)}`);
+  return cache.get(key);
+}
+
+function setCachedResponse(prompt: string, context: any, response: any) {
+  const key = hash(`${prompt}-${JSON.stringify(context)}`);
+  cache.set(key, response);
+}
+```
+
+## Monitoring & Observability
+
+### Response Metrics
+
+Track provider performance and usage:
+
+```typescript
+const response = await provider.generateMessage(input);
+
+console.log("Provider:", response.metadata?.model);
+console.log("Tokens used:", response.metadata?.tokensUsed);
+console.log("Prompt tokens:", response.metadata?.promptTokens);
+console.log("Completion tokens:", response.metadata?.completionTokens);
+console.log("Finish reason:", response.metadata?.finishReason);
+```
+
+### Cost Tracking
+
+Monitor API usage costs:
+
+```typescript
+const costTracker = {
+  openai: {
+    "gpt-5": { prompt: 0.03, completion: 0.06 },
+    "gpt-4": { prompt: 0.002, completion: 0.002 },
+  },
+};
+
+function calculateCost(metadata: ResponseMetadata): number {
+  const rates = costTracker[metadata.model];
+  if (!rates) return 0;
+
+  return (
+    ((metadata.promptTokens || 0) * rates.prompt) / 1000 +
+    ((metadata.completionTokens || 0) * rates.completion) / 1000
+  );
+}
+```
+
+### Health Monitoring
+
+Track provider availability and latency:
+
+```typescript
+const healthChecks = new Map();
+
+async function checkProviderHealth(provider: AiProvider): Promise<boolean> {
+  const startTime = Date.now();
+
+  try {
+    await provider.generateMessage({
+      prompt: "Hello",
+      history: [],
+    });
+
+    const latency = Date.now() - startTime;
+    healthChecks.set(provider.name, { healthy: true, latency });
+    return true;
+  } catch (error) {
+    healthChecks.set(provider.name, { healthy: false, error: error.message });
+    return false;
+  }
+}
+```
+
+## Custom Provider Implementation
+
+Create providers for unsupported AI services:
+
+```typescript
+import {
+  AiProvider,
+  GenerateMessageInput,
+  GenerateMessageOutput,
+} from "@falai/agent";
+
+class CustomProvider implements AiProvider {
+  name = "custom";
+
+  async generateMessage(
+    input: GenerateMessageInput
+  ): Promise<GenerateMessageOutput> {
+    // Implement your AI service integration
+    const response = await callCustomAI(input.prompt, input.context);
+
+    return {
+      message: response.text,
+      metadata: {
+        model: "custom-model-v1",
+        tokensUsed: response.tokens,
+        promptTokens: input.prompt.split(" ").length,
+        completionTokens: response.text.split(" ").length,
+      },
+      structured: response.structured,
+    };
+  }
+
+  async *generateMessageStream(input: GenerateMessageInput) {
+    // Implement streaming if supported
+    const stream = await callCustomAIStream(input.prompt, input.context);
+
+    for await (const chunk of stream) {
+      yield {
+        delta: chunk.text,
+        accumulated: chunk.fullText,
+        done: chunk.finished,
+        metadata: chunk.metadata,
+      };
+    }
+  }
+}
+```
+
+## Best Practices
+
+### Provider Selection
+
+1. **Task Matching**: Choose providers based on task requirements
+
+   - OpenAI GPT-4: Complex reasoning, creative tasks
+   - Anthropic Claude: Safety-focused, analytical tasks
+   - Google Gemini: Multimodal, fast responses
+   - OpenRouter: Cost optimization, model experimentation
+
+2. **Cost Optimization**: Use appropriate models for task complexity
+
+   - Simple tasks: GPT-4.1, Claude Haiku, Gemini Flash
+   - Complex tasks: GPT-5, Claude Sonnet, Gemini Pro
+
+3. **Reliability**: Configure backup models for production
+   - Always have fallback options
+   - Monitor error rates and switch providers if needed
+
+### Configuration
+
+1. **Temperature**: Lower for deterministic tasks, higher for creative
+2. **Max Tokens**: Set appropriate limits to control costs
+3. **Timeouts**: Configure reasonable timeouts for your use case
+4. **Retries**: Enable retries for transient failures
+
+### Monitoring
+
+1. **Track Usage**: Monitor token consumption and costs
+2. **Error Rates**: Alert on high error rates
+3. **Latency**: Monitor response times
+4. **Fallback Usage**: Track how often backup models are used
+
+The AI provider system enables flexible, reliable integration with multiple AI services while maintaining a consistent interface for the @falai/agent framework.

package/docs/core/ai-integration/response-processing.md

@@ -0,0 +1,176 @@
+# Response Processing
+
+@fali/agent processes AI responses to extract structured data, execute tools, and update conversation state according to schema definitions and routing logic.
+
+## Overview
+
+The `ResponseEngine` handles AI response processing, including:
+
+- Schema-based data extraction
+- Tool execution coordination
+- Context and session updates
+- Route progression logic
+
+## Response Parsing
+
+AI responses are parsed to extract:
+
+1. **Natural Language Response** - The text to send to the user
+2. **Structured Data** - JSON data matching collection schemas
+3. **Tool Calls** - Instructions to execute tools
+4. **Routing Decisions** - Route or step transitions
+
+## Agent-Level Schema-Driven Extraction
+
+When steps specify `collect` fields, the AI response is validated against the agent-level JSON schema:
+
+```typescript
+// Agent defines comprehensive schema
+const agent = new Agent<{}, UserData>({
+  schema: {
+    type: "object",
+    properties: {
+      name: { type: "string" },
+      email: { type: "string", format: "email" },
+      phone: { type: "string" },
+      preferences: { type: "object" }
+    },
+    required: ["name", "email"]
+  }
+});
+
+// Steps collect into agent schema
+const step = route.initialStep.nextStep({
+  prompt: "What's your name and email?",
+  collect: ["name", "email"], // Maps to agent schema fields
+});
+```
+
+The AI receives instructions to return structured data that matches the agent-level schema, enabling cross-route data sharing.
+
+## Tool Execution Pipeline
+
+When tools are called, the response engine:
+
+1. **Validates** tool parameters against schemas
+2. **Executes** tools in the correct order
+3. **Updates** context with tool results
+4. **Merges** tool-returned data into session
+5. **Continues** conversation with enriched context
+
+```typescript
+// Tool execution result
+{
+  success: true,
+  result: {
+    data: "Flight search completed",
+    contextUpdate: { availableFlights: [...] },
+    dataUpdate: { searchPerformed: true }
+  }
+}
+```
+
+## Data Validation
+
+Extracted data is validated against the agent-level schema:
+
+- **Type checking** - Ensures correct data types against agent schema
+- **Required fields** - Validates mandatory data presence for route completion
+- **Format validation** - Email, dates, custom formats from agent schema
+- **Business rules** - Custom validation logic in agent-level hooks
+- **Cross-route consistency** - Ensures data consistency across all routes
+
+## Context Updates
+
+Response processing updates multiple context layers:
+
+### Agent-Level Data
+
+```typescript
+// Collected data merged into agent-level data structure
+agent.collectedData = {
+  ...agent.collectedData,
+  ...extractedData,
+  ...toolResults,
+};
+
+// Session references agent data
+session.data = agent.collectedData;
+```
+
+### Route Context
+
+```typescript
+// Route-specific context updates
+routeContext.lastActivity = new Date();
+routeContext.stepCount += 1;
+```
+
+### Agent Context
+
+```typescript
+// Global agent context
+agentContext.totalInteractions += 1;
+agentContext.lastResponseTime = Date.now();
+```
+
+## Error Handling
+
+Robust error handling for various failure scenarios:
+
+- **Schema validation failures** - Graceful fallback to manual extraction
+- **Tool execution errors** - Error recovery and user notification
+- **Context update failures** - Rollback and logging
+- **Routing errors** - Safe fallback to default behavior
+
+## Streaming Response Processing
+
+For streaming responses, processing happens incrementally:
+
+```typescript
+for await (const chunk of agent.respondStream({...})) {
+  if (chunk.delta) {
+    // Process partial response
+    processStreamingChunk(chunk);
+  }
+
+  if (chunk.done) {
+    // Final processing
+    await finalizeResponse(chunk);
+  }
+}
+```
+
+## Route Progression
+
+Response processing determines next conversation steps:
+
+- **Step completion** - Advances to next step in route
+- **Route completion** - Handles `END_ROUTE` logic
+- **Branching decisions** - Evaluates conditions for path selection
+- **Transition triggers** - Initiates route-to-route transitions
+
+## Performance Considerations
+
+- **Efficient parsing** - Minimal overhead for response processing
+- **Lazy validation** - Only validate when necessary
+- **Caching** - Cache parsed schemas and validation rules
+- **Async processing** - Non-blocking context updates
+
+## Monitoring & Debugging
+
+Built-in monitoring capabilities:
+
+- **Response metrics** - Token usage, processing time
+- **Error tracking** - Failed extractions, validation errors
+- **Debug logging** - Detailed processing traces
+- **Performance profiling** - Identify bottlenecks
+
+## Best Practices
+
+- Design schemas for reliable AI extraction
+- Implement comprehensive error handling
+- Monitor response quality and adjust prompts
+- Use streaming for better user experience
+- Leverage tool results for context enrichment
+- Validate data at multiple levels (schema + business rules)