@falai/agent 0.8.1 → 0.9.0-alpha-1

package/docs/core/ai-integration/prompt-composition.md

@@ -0,0 +1,196 @@
+# Prompt Composition
+
+@fali/agent automatically builds comprehensive prompts for AI providers by combining agent configuration, route context, conversation history, and current session state.
+
+## Overview
+
+The `PromptComposer` class is responsible for constructing detailed prompts that provide AI models with all necessary context to make intelligent routing and response decisions.
+
+## Prompt Structure
+
+Prompts are built with multiple sections in a specific order:
+
+1. **Agent Identity** - Core agent persona and personality
+2. **Routing Overview** - Available routes and current context
+3. **Knowledge Base** - Agent and route-specific knowledge
+4. **Guidelines & Rules** - Behavioral constraints and preferences
+5. **Conversation History** - Previous messages and context
+6. **Current Session** - Active route, collected data, and step state
+7. **Task-Specific Instructions** - Route-specific prompts and schemas
+
+## Agent Identity Section
+
+```typescript
+// Agent configuration contributes to identity
+const agent = new Agent({
+  name: "TravelBot",
+  description: "AI assistant for travel planning and booking",
+  goal: "Help users plan perfect trips efficiently",
+  personality: "Friendly, knowledgeable, and efficient",
+});
+```
+
+This generates:
+
+```
+You are TravelBot, an AI assistant for travel planning and booking.
+Your goal is to help users plan perfect trips efficiently.
+You are friendly, knowledgeable, and efficient.
+```
+
+## Routing Overview Section
+
+Provides the AI with information about available conversation routes:
+
+```
+Available Routes:
+1. Flight Booking - Help users book flights
+2. Hotel Reservation - Find and book accommodations
+3. Travel Itinerary - Create detailed trip plans
+4. Customer Support - Handle travel-related questions
+
+Current Route: Flight Booking
+Route Description: Help users book flights with the best options
+```
+
+## Knowledge Base Integration
+
+Agent and route knowledge bases are formatted and included:
+
+```typescript
+const agent = new Agent({
+  knowledgeBase: {
+    company: "Acme Travel - Premium travel services since 1995",
+    policies: {
+      cancellation: "Free cancellation up to 24 hours before departure",
+      refund: "Refunds processed within 3-5 business days",
+    },
+  },
+});
+```
+
+## Guidelines & Rules Section
+
+Combines agent-level and route-level behavioral constraints:
+
+```
+Guidelines:
+- Always be polite and helpful
+- Ask clarifying questions when needed
+- Provide options rather than single recommendations
+
+Rules for this route:
+- Always confirm flight details before booking
+- Never share personal information
+- Use specific date formats (YYYY-MM-DD)
+```
+
+## Conversation History
+
+Recent conversation history with context updates:
+
+```
+Previous Messages:
+User: I want to book a flight to Paris
+Assistant: I'd be happy to help you book a flight to Paris. When would you like to travel?
+
+User: Next week, for 3 nights
+Assistant: Great! Let me check flights for next week. Could you tell me which airport you'd like to depart from?
+```
+
+## Session State
+
+Current session information including:
+
+- Active route and step
+- Collected data (with privacy filtering)
+- Route progress and completion status
+
+## Dynamic Schema Generation
+
+For data collection steps, the prompt includes JSON schemas:
+
+```typescript
+// Route schema
+schema: {
+  type: "object",
+  properties: {
+    destination: { type: "string" },
+    departureDate: { type: "string", format: "date" },
+    passengers: { type: "number", minimum: 1 }
+  },
+  required: ["destination", "departureDate"]
+}
+```
+
+Generates:
+
+```
+Extract the following information from the user's response:
+- destination: string - Where the user wants to fly
+- departureDate: string (date format) - When they want to depart
+- passengers: number (minimum 1) - How many people are traveling
+
+Return extracted data as valid JSON matching this schema.
+```
+
+## Tool Integration
+
+When tools are available, they're described in the prompt:
+
+```
+Available Tools:
+1. searchFlights - Search for available flights
+   Parameters:
+   - origin: string - Departure airport code
+   - destination: string - Arrival airport code
+   - date: string - Travel date (YYYY-MM-DD)
+
+2. checkWeather - Get weather forecast
+   Parameters:
+   - location: string - City or airport code
+   - date: string - Date for forecast
+```
+
+## Context Updates
+
+Real-time context changes are included:
+
+```
+Current Context:
+- User tier: premium
+- Preferred language: English
+- Last login: 2024-01-15
+- Current location: New York
+```
+
+## Route-Specific Customization
+
+Routes can override agent-level settings:
+
+```typescript
+const supportRoute = agent.createRoute({
+  title: "Customer Support",
+  personality: "Extra patient and detailed when explaining issues",
+  knowledgeBase: {
+    supportHours: "24/7 for premium members, 9-5 EST for others",
+  },
+});
+```
+
+## Performance Optimization
+
+Prompts are optimized for token efficiency:
+
+- Knowledge bases are formatted as clean markdown
+- Redundant information is deduplicated
+- Context is truncated when approaching token limits
+- Only relevant route information is included
+
+## Best Practices
+
+- Keep knowledge bases structured and concise
+- Use clear, specific guidelines rather than vague instructions
+- Leverage route-specific overrides for specialized behavior
+- Monitor token usage and optimize prompt length
+- Test prompts with different AI providers for consistency

package/docs/core/ai-integration/providers.md

@@ -0,0 +1,515 @@
+# AI Provider Integrations
+
+@falai/agent provides unified interfaces for multiple AI providers, enabling seamless switching between models and automatic fallback handling.
+
+## Supported Providers
+
+### OpenAI Provider
+
+Full-featured integration with GPT models, including backup model support and structured outputs.
+
+```typescript
+import { OpenAIProvider } from "@falai/agent";
+
+const provider = new OpenAIProvider({
+  apiKey: process.env.OPENAI_API_KEY,
+  model: "gpt-5",
+  backupModels: ["gpt-4"], // Fallback models
+  config: {
+    temperature: 0.7,
+    max_tokens: 1000,
+  },
+  retryConfig: {
+    timeout: 30000,
+    retries: 3,
+  },
+});
+```
+
+**Features:**
+
+- ✅ GPT-4, GPT-5, and all OpenAI models
+- ✅ Backup model fallback on failures
+- ✅ Structured outputs with JSON Schema
+- ✅ Tool calling support
+- ✅ Streaming responses
+- ✅ Automatic retry logic
+
+### Google Gemini Provider
+
+Integration with Google's Gemini models through Vertex AI or AI Studio.
+
+```typescript
+import { GeminiProvider } from "@falai/agent";
+
+const provider = new GeminiProvider({
+  apiKey: process.env.GEMINI_API_KEY,
+  model: "gemini-pro",
+  config: {
+    safetySettings: [
+      {
+        category: "HARM_CATEGORY_HARASSMENT",
+        threshold: "BLOCK_MEDIUM_AND_ABOVE",
+      },
+    ],
+  },
+});
+```
+
+**Features:**
+
+- ✅ Gemini 2.0, 2.5 Pro, and Flash models
+- ✅ Safety settings and content filtering
+- ✅ Multimodal capabilities (text, images)
+- ✅ Function calling support
+- ✅ Streaming responses
+
+### Anthropic Claude Provider
+
+Integration with Anthropic's Claude models via their API.
+
+```typescript
+import { AnthropicProvider } from "@falai/agent";
+
+const provider = new AnthropicProvider({
+  apiKey: process.env.ANTHROPIC_API_KEY,
+  model: "claude-sonnet-4.5",
+  config: {
+    maxTokens: 4096,
+    temperature: 0.7,
+  },
+});
+```
+
+**Features:**
+
+- ✅ Claude 4 Opus, Sonnet, and Haiku
+- ✅ Constitutional AI safety
+- ✅ Excellent reasoning capabilities
+- ✅ Tool calling support
+- ✅ Streaming responses
+
+### OpenRouter Provider
+
+Unified access to multiple AI models through OpenRouter's API.
+
+```typescript
+import { OpenRouterProvider } from "@falai/agent";
+
+const provider = new OpenRouterProvider({
+  apiKey: process.env.OPENROUTER_API_KEY,
+  model: "anthropic/claude-sonnet-4",
+  siteUrl: "https://your-app.com",
+  siteName: "Your App Name",
+});
+```
+
+**Features:**
+
+- ✅ Access to 100+ AI models
+- ✅ Unified pricing and billing
+- ✅ Automatic model routing
+- ✅ Fallback model support
+- ✅ Streaming responses
+
+## Provider Configuration
+
+### Common Options
+
+All providers support these configuration options:
+
+```typescript
+interface BaseProviderOptions {
+  model: string; // Primary model to use
+  backupModels?: string[]; // Fallback models on failure
+  temperature?: number; // Response randomness (0-2)
+  maxTokens?: number; // Maximum response length
+  timeout?: number; // Request timeout in ms
+  retries?: number; // Number of retry attempts
+}
+```
+
+### Advanced Configuration
+
+```typescript
+const provider = new OpenAIProvider({
+  apiKey: process.env.OPENAI_API_KEY,
+  model: "gpt-5",
+
+  // Model-specific parameters
+  config: {
+    temperature: 0.1, // Low randomness for consistent responses
+    top_p: 0.9, // Nucleus sampling
+    frequency_penalty: 0.1, // Reduce repetition
+    presence_penalty: 0.1, // Encourage topic diversity
+    max_tokens: 2048,
+  },
+
+  // Fallback configuration
+  backupModels: [
+    "gpt-4.1", // Try GPT-4 if turbo fails
+    "gpt-4", // Final fallback
+  ],
+
+  // Retry configuration
+  retryConfig: {
+    timeout: 60000, // 60 second timeout
+    retries: 3, // Retry up to 3 times
+  },
+});
+```
+
+## Unified Interface
+
+All providers implement the same interface:
+
+```typescript
+interface AiProvider {
+  name: string;
+
+  // Synchronous response generation
+  generateMessage(input: GenerateMessageInput): Promise<GenerateMessageOutput>;
+
+  // Streaming response generation
+  generateMessageStream(
+    input: GenerateMessageInput
+  ): AsyncGenerator<GenerateMessageStreamChunk>;
+}
+```
+
+### Input Format
+
+```typescript
+interface GenerateMessageInput<TContext = unknown> {
+  prompt: string; // The prompt to send
+  history: Event[]; // Conversation history
+  context?: TContext; // Additional context data
+  tools?: ToolDefinition[]; // Available tools
+  parameters?: {
+    jsonSchema?: StructuredSchema; // Response schema
+    schemaName?: string; // Schema identifier
+    maxOutputTokens?: number; // Token limit
+    reasoning?: { effort: "low" | "medium" | "high" };
+  };
+  signal?: AbortSignal; // Cancellation support
+}
+```
+
+### Output Format
+
+```typescript
+interface GenerateMessageOutput<TStructured = AgentStructuredResponse> {
+  message: string; // Main response text
+  metadata: {
+    model: string; // Model used
+    tokensUsed: number; // Total tokens consumed
+    promptTokens: number; // Input tokens
+    completionTokens: number; // Output tokens
+    finishReason?: string; // Why generation stopped
+  };
+  structured?: TStructured; // Structured data if schema provided
+}
+```
+
+## Tool Calling Support
+
+All providers support tool calling with consistent interfaces:
+
+```typescript
+const input: GenerateMessageInput = {
+  prompt: "What's the weather in Paris?",
+  history: conversationHistory,
+  tools: [
+    {
+      id: "get_weather",
+      description: "Get current weather for a location",
+      parameters: {
+        type: "object",
+        properties: {
+          location: { type: "string", description: "City name" },
+        },
+        required: ["location"],
+      },
+    },
+  ],
+};
+
+const response = await provider.generateMessage(input);
+
+// Check for tool calls
+if (response.structured?.toolCalls) {
+  for (const toolCall of response.structured.toolCalls) {
+    console.log(`AI wants to call: ${toolCall.toolName}`);
+    console.log(`With arguments:`, toolCall.arguments);
+  }
+}
+```
+
+## Streaming Responses
+
+Real-time response generation with chunked output:
+
+```typescript
+const stream = provider.generateMessageStream(input);
+
+for await (const chunk of stream) {
+  if (chunk.delta) {
+    process.stdout.write(chunk.delta); // Real-time output
+  }
+
+  if (chunk.done) {
+    console.log("\nGeneration complete!");
+    console.log("Total tokens:", chunk.metadata?.tokensUsed);
+    console.log("Structured data:", chunk.structured);
+  }
+}
+```
+
+## Error Handling & Fallbacks
+
+### Automatic Fallbacks
+
+Providers automatically try backup models on failures:
+
+```typescript
+const provider = new OpenAIProvider({
+  apiKey: process.env.OPENAI_API_KEY,
+  model: "gpt-5",
+  backupModels: ["gpt-4.1", "gpt-4"],
+});
+
+// If gpt-4 fails (rate limit, server error, etc.)
+// Automatically tries gpt-4.1, then gpt-4
+const response = await provider.generateMessage(input);
+```
+
+### Error Classification
+
+Different error types trigger different fallback behaviors:
+
+```typescript
+// Server errors (5xx) → Try backup models
+// Rate limits (429) → Try backup models
+// Invalid requests (4xx) → Don't retry
+// Timeouts → Retry with same model
+```
+
+### Custom Error Handling
+
+```typescript
+try {
+  const response = await provider.generateMessage(input);
+} catch (error) {
+  if (error.code === "rate_limit_exceeded") {
+    // Handle rate limiting
+    await delay(60000); // Wait 1 minute
+    return retryWithBackoff();
+  }
+
+  if (error.code === "model_not_available") {
+    // Switch to different provider
+    return fallbackProvider.generateMessage(input);
+  }
+
+  throw error;
+}
+```
+
+## Performance Optimization
+
+### Connection Reuse
+
+Reuse provider instances for better performance:
+
+```typescript
+// ✅ Good: Reuse provider instance
+const provider = new OpenAIProvider({ apiKey, model: "gpt-4" });
+
+const agent1 = new Agent({ provider });
+const agent2 = new Agent({ provider }); // Same instance
+
+// ❌ Bad: Create new instance each time
+const agent3 = new Agent({
+  provider: new OpenAIProvider({ apiKey, model: "gpt-4" }),
+});
+```
+
+### Request Batching
+
+Group related requests to reduce latency:
+
+```typescript
+// Process multiple conversations efficiently
+const responses = await Promise.all([
+  agent1.respond({ history: history1 }),
+  agent2.respond({ history: history2 }),
+  agent3.respond({ history: history3 }),
+]);
+```
+
+### Caching
+
+Cache responses for similar prompts:
+
+```typescript
+const cache = new Map();
+
+function getCachedResponse(prompt: string, context: any) {
+  const key = hash(`${prompt}-${JSON.stringify(context)}`);
+  return cache.get(key);
+}
+
+function setCachedResponse(prompt: string, context: any, response: any) {
+  const key = hash(`${prompt}-${JSON.stringify(context)}`);
+  cache.set(key, response);
+}
+```
+
+## Monitoring & Observability
+
+### Response Metrics
+
+Track provider performance and usage:
+
+```typescript
+const response = await provider.generateMessage(input);
+
+console.log("Provider:", response.metadata?.model);
+console.log("Tokens used:", response.metadata?.tokensUsed);
+console.log("Prompt tokens:", response.metadata?.promptTokens);
+console.log("Completion tokens:", response.metadata?.completionTokens);
+console.log("Finish reason:", response.metadata?.finishReason);
+```
+
+### Cost Tracking
+
+Monitor API usage costs:
+
+```typescript
+const costTracker = {
+  openai: {
+    "gpt-5": { prompt: 0.03, completion: 0.06 },
+    "gpt-4": { prompt: 0.002, completion: 0.002 },
+  },
+};
+
+function calculateCost(metadata: ResponseMetadata): number {
+  const rates = costTracker[metadata.model];
+  if (!rates) return 0;
+
+  return (
+    ((metadata.promptTokens || 0) * rates.prompt) / 1000 +
+    ((metadata.completionTokens || 0) * rates.completion) / 1000
+  );
+}
+```
+
+### Health Monitoring
+
+Track provider availability and latency:
+
+```typescript
+const healthChecks = new Map();
+
+async function checkProviderHealth(provider: AiProvider): Promise<boolean> {
+  const startTime = Date.now();
+
+  try {
+    await provider.generateMessage({
+      prompt: "Hello",
+      history: [],
+    });
+
+    const latency = Date.now() - startTime;
+    healthChecks.set(provider.name, { healthy: true, latency });
+    return true;
+  } catch (error) {
+    healthChecks.set(provider.name, { healthy: false, error: error.message });
+    return false;
+  }
+}
+```
+
+## Custom Provider Implementation
+
+Create providers for unsupported AI services:
+
+```typescript
+import {
+  AiProvider,
+  GenerateMessageInput,
+  GenerateMessageOutput,
+} from "@falai/agent";
+
+class CustomProvider implements AiProvider {
+  name = "custom";
+
+  async generateMessage(
+    input: GenerateMessageInput
+  ): Promise<GenerateMessageOutput> {
+    // Implement your AI service integration
+    const response = await callCustomAI(input.prompt, input.context);
+
+    return {
+      message: response.text,
+      metadata: {
+        model: "custom-model-v1",
+        tokensUsed: response.tokens,
+        promptTokens: input.prompt.split(" ").length,
+        completionTokens: response.text.split(" ").length,
+      },
+      structured: response.structured,
+    };
+  }
+
+  async *generateMessageStream(input: GenerateMessageInput) {
+    // Implement streaming if supported
+    const stream = await callCustomAIStream(input.prompt, input.context);
+
+    for await (const chunk of stream) {
+      yield {
+        delta: chunk.text,
+        accumulated: chunk.fullText,
+        done: chunk.finished,
+        metadata: chunk.metadata,
+      };
+    }
+  }
+}
+```
+
+## Best Practices
+
+### Provider Selection
+
+1. **Task Matching**: Choose providers based on task requirements
+
+   - OpenAI GPT-4: Complex reasoning, creative tasks
+   - Anthropic Claude: Safety-focused, analytical tasks
+   - Google Gemini: Multimodal, fast responses
+   - OpenRouter: Cost optimization, model experimentation
+
+2. **Cost Optimization**: Use appropriate models for task complexity
+
+   - Simple tasks: GPT-4.1, Claude Haiku, Gemini Flash
+   - Complex tasks: GPT-5, Claude Sonnet, Gemini Pro
+
+3. **Reliability**: Configure backup models for production
+   - Always have fallback options
+   - Monitor error rates and switch providers if needed
+
+### Configuration
+
+1. **Temperature**: Lower for deterministic tasks, higher for creative
+2. **Max Tokens**: Set appropriate limits to control costs
+3. **Timeouts**: Configure reasonable timeouts for your use case
+4. **Retries**: Enable retries for transient failures
+
+### Monitoring
+
+1. **Track Usage**: Monitor token consumption and costs
+2. **Error Rates**: Alert on high error rates
+3. **Latency**: Monitor response times
+4. **Fallback Usage**: Track how often backup models are used
+
+The AI provider system enables flexible, reliable integration with multiple AI services while maintaining a consistent interface for the @falai/agent framework.