@falai/agent 0.8.1 → 0.9.0-alpha-2

package/docs/{API_REFERENCE.md → api/README.md}

@@ -1,19 +1,30 @@
 # API Reference
 
-Complete API documentation for `@falai/agent`.
+Complete API documentation for `@falai/agent`. This framework provides a strongly-typed, modular agent architecture with AI-powered routing and schema-driven data collection.
+
+## 📋 Documentation Structure
+
+- **[Complete API Overview](./overview.md)** - Comprehensive reference for all classes, interfaces, and utilities
+- **[Agent Architecture](../core/agent/README.md)** - Agent class, context management, and lifecycle
+- **[AI Routing System](../core/routing/intelligent-routing.md)** - Intelligent route and step selection
+- **[Route DSL](../core/conversation-flows/route-dsl.md)** - Declarative conversation flow design
+- **[Data Collection](../core/conversation-flows/data-collection.md)** - Schema-driven data extraction
+- **[Tool Execution](../core/tools/tool-execution.md)** - Dynamic tool calling and context updates
+- **[Session Storage](../core/persistence/session-storage.md)** - Persistence and session management
+- **[AI Providers](../core/ai-integration/providers.md)** - Provider integrations and configuration
 
 ---
 
 ## Core Classes
 
-### `Agent<TContext>`
+### `Agent<TContext, TData>`
 
-Main agent class for managing conversational AI.
+Main agent class for managing conversational AI with agent-level data collection.
 
 #### Constructor
 
 ```typescript
-new Agent<TContext>(options: AgentOptions<TContext>)
+new Agent<TContext, TData>(options: AgentOptions<TContext, TData>)
 ```
 
 See [Agent](./AGENT.md) for full details.
@@ -22,7 +33,38 @@ See [Agent](./AGENT.md) for full details.
 
 ##### `createRoute(options: RouteOptions): Route`
 
-Creates a new conversation route.
+Creates a new conversation route with required fields specification.
+
+##### `getCollectedData(): Partial<TData>`
+
+Gets the current agent-level collected data.
+
+```typescript
+const data = agent.getCollectedData();
+console.log(data); // { customerName: "John", email: "john@example.com" }
+```
+
+##### `updateCollectedData(updates: Partial<TData>): Promise<void>`
+
+Updates agent-level collected data and triggers validation.
+
+```typescript
+await agent.updateCollectedData({
+  customerId: "CUST-12345",
+  priority: "high"
+});
+```
+
+##### `validateData(data: Partial<TData>): ValidationResult`
+
+Validates data against the agent-level schema.
+
+```typescript
+const validation = agent.validateData({ email: "invalid-email" });
+if (!validation.valid) {
+  console.log(validation.errors); // Detailed validation errors
+}
+```
 
 ##### `createTerm(term: Term): this`
 
@@ -32,21 +74,108 @@ Adds a domain glossary term. Returns `this` for chaining.
 
 Adds a behavioral guideline. Returns `this` for chaining.
 
-##### `createCapability(capability: Capability): this`
+##### `respond(input: RespondInput<TContext>): Promise<RespondOutput>`
+
+Generates an AI response with session step management, tool execution, data extraction, and intelligent routing.
+
+**Enhanced Response Pipeline:**
+
+1. **Tool Execution** - Execute tools if current step has `tool` (enriches context before AI response)
+2. **Always-On Routing** - Score all routes, respect user intent to change direction
+3. **Step Traversal** - Use `skipIf` and `requires` to determine next step
+4. **Response Generation** - Build schema with `collect` fields, extract data
+5. **Session Update** - Merge collected data into session step
+
+**Tool Execution (Pre-Response):**
+Tools execute before AI response generation, allowing them to:
+
+- Enrich context with external data
+- Update collected session data
+- Perform business logic
+- Access APIs and databases
+
+This design enables more intelligent, context-aware responses.
+
+##### `getKnowledgeBase(): Record<string, unknown>`
+
+Gets the agent's knowledge base containing any JSON structure the AI should know.
+
+```typescript
+const knowledge = agent.getKnowledgeBase();
+// Returns the agent's knowledge base
+```
+
+##### `getRoutes(): Route<TContext, unknown>[]`
+
+Gets all routes configured in the agent.
+
+```typescript
+const routes = agent.getRoutes();
+// Returns array of all configured routes
+```
+
+##### `getTerms(): Term<TContext>[]`
+
+Gets all terms configured in the agent.
+
+```typescript
+const terms = agent.getTerms();
+// Returns array of all configured terms
+```
+
+##### `getGuidelines(): Guideline<TContext>[]`
+
+Gets all guidelines configured in the agent.
+
+```typescript
+const guidelines = agent.getGuidelines();
+// Returns array of all configured guidelines
+```
+
+````typescript
+
+##### `getPersistenceManager(): PersistenceManager | undefined`
+
+Gets the persistence manager if configured.
+
+```typescript
+const persistence = agent.getPersistenceManager();
+// Returns PersistenceManager instance or undefined if not configured
+````
+
+##### `hasPersistence(): boolean`
+
+Checks if persistence is enabled.
 
-Adds an agent capability. Returns `this` for chaining.
+```typescript
+if (agent.hasPersistence()) {
+  // Persistence is configured
+}
+```
+
+##### `nextStepRoute(routeIdOrTitle, session?, condition?, history?): Promise<SessionState>`
 
-##### `addDomain<TName, TDomain>(name: TName, domainObject: TDomain): void`
+Manually transition to a different route by setting a pending transition that will be executed on the next `respond()` call.
 
-Registers a domain with tools/methods.
+```typescript
+// Transition to feedback route after completion
+const updatedSession = await agent.nextStepRoute(
+  "feedback-collection",
+  session
+);
 
-##### `getDomainsForRoute(routeId: string): Record<string, Record<string, unknown>>`
+// Next respond() call will automatically transition
+const response = await agent.respond({ history, session: updatedSession });
+```
 
-Gets allowed domains for a specific route by ID. Returns filtered domains based on route's `domains` property, or all domains if route has no restrictions.
+**Parameters:**
 
-##### `getDomainsForRouteByTitle(routeTitle: string): Record<string, Record<string, unknown>>`
+- `routeIdOrTitle`: Route ID or title to transition to
+- `session`: Session step to update (uses current session if not provided)
+- `condition`: Optional AI-evaluated condition for the transition
+- `history`: Optional history for template context
 
-Gets allowed domains for a specific route by title. Returns filtered domains based on route's `domains` property, or all domains if route has no restrictions.
+**Returns:** Updated session with pending transition
 
 ##### `getData<TData>(routeId?): Partial<TData>`
 
@@ -159,87 +288,53 @@ interface RespondOutput {
 - Automatically merges new collected data with existing session data
 - **Per-route data preservation** - Collected data is organized by route ID, allowing users to switch routes without losing progress
 
-**Example with Persistence Adapters:**
+**Example with Automatic Session Management:**
 
 ```typescript
-import { createSession } from "@falai/agent";
-
-// Using built-in persistence adapters
-const { sessionData, sessionStep } =
-  await persistence.createSessionWithStep<FlightData>({
-    userId: "user_123",
-    agentName: "Travel Agent",
+// Server endpoint with automatic session management
+app.post('/chat', async (req, res) => {
+  const { sessionId, message } = req.body;
+  
+  const agent = new Agent({
+    name: "Travel Agent",
+    provider: new OpenAIProvider({ apiKey: process.env.OPENAI_API_KEY }),
+    persistence: { adapter: new PrismaAdapter({ prisma }) },
+    sessionId // Automatically loads or creates this session
+  });
+  
+  const response = await agent.respond(message);
+  
+  res.json({
+    message: response.message,
+    sessionId: agent.session.id,
+    isComplete: response.isRouteComplete
   });
-
-// Option 1: Set current session for convenience
-agent.setCurrentSession(sessionStep);
-
-const response = await agent.respond({
-  history,
-  // session: sessionStep, // No longer required!
-});
-
-// Use convenience methods without passing session
-const data = agent.getData();
-
-// Option 2: Still pass session explicitly if preferred
-const response2 = await agent.respond({
-  history,
-  session: sessionStep,
 });
 ```
 
-**Example with Custom Database (Manual):**
+**SessionManager API:**
 
 ```typescript
-import { createSession, SessionState } from "@falai/agent";
-
-// Load from your custom database
-const dbSession = await yourDb.sessions.findOne({ id: sessionId });
+// Access session manager
+const sessionManager = agent.session;
 
-// Restore or create session step
-let agentSession: SessionState<YourDataType>;
+// Get or create session (works for existing, new, or auto-generated IDs)
+await sessionManager.getOrCreate("user-123");
+await sessionManager.getOrCreate(); // Auto-generates ID
 
-if (dbSession && dbSession.currentRoute && dbSession.collectedData) {
-  // Restore existing session from database
-  agentSession = {
-    currentRoute: {
-      id: dbSession.currentRoute,
-      title:
-        dbSession.collectedData?.currentRouteTitle || dbSession.currentRoute,
-      enteredAt: new Date(),
-    },
-    currentStep: dbSession.currentStep
-      ? {
-          id: dbSession.currentStep,
-          description: dbSession.collectedData?.currentStepDescription,
-          enteredAt: new Date(),
-        }
-      : undefined,
-    data: dbSession.collectedData?.data || {},
-    routeHistory: dbSession.collectedData?.routeHistory || [],
-    metadata: {
-      sessionId: dbSession.id,
-      createdAt: dbSession.createdAt,
-      lastUpdatedAt: new Date(),
-    },
-  };
-} else {
-  // Create new session
-  agentSession = createSession<YourDataType>({
-    sessionId: dbSession?.id || "new-session-id",
-  });
-}
+// Data access
+const data = sessionManager.getData<FlightData>();
+await sessionManager.setData({ destination: "Paris" });
 
-// Use session in conversation
-const response = await agent.respond({
-  history,
-  session: agentSession,
-});
+// History management
+await sessionManager.addMessage("user", "Hello");
+const history = sessionManager.getHistory();
+sessionManager.clearHistory();
 
-// Manually save to your database
-await yourDb.sessions.update({
-  id: dbSession.id,
+// Session operations
+await sessionManager.save(); // Manual save (auto-saves on addMessage)
+await sessionManager.delete();
+const newSession = await sessionManager.reset(true); // Preserve history
   currentRoute: response.session?.currentRoute?.id,
   currentStep: response.session?.currentStep?.id,
   collectedData: {
@@ -255,7 +350,7 @@ await yourDb.sessions.update({
 // Save message
 await yourDb.messages.create({
   sessionId: dbSession.id,
-  role: "agent",
+  role: "assistant",
   content: response.message,
   route: response.session?.currentRoute?.id,
   step: response.session?.currentStep?.id,
@@ -310,21 +405,21 @@ const onboardingRoute = agent.createRoute<OnboardingData>({
 // Build steps with skipIf conditions
 const welcome = onboardingRoute.initialStep.nextStep({
   id: "ask_name",
-  instructions: "What's your name?",
+  prompt: "What's your name?",
   collect: ["name"],
   skipIf: (data) => !!data.name, // Skip if name already collected
 });
 
 const askEmail = welcome.nextStep({
   id: "ask_email",
-  instructions: "What's your email?",
+  prompt: "What's your email?",
   collect: ["email"],
   skipIf: (data) => !!data.email, // Skip if email already collected
 });
 
 const complete = askEmail.nextStep({
   id: "complete",
-  instructions: "All done! Thank you.",
+  prompt: "All done! Thank you.",
 });
 
 complete.nextStep({ step: END_ROUTE });
@@ -342,17 +437,13 @@ if (response.isRouteComplete) {
   return "Profile updated successfully!";
 }
 
-// Get route-specific data if needed
-const onboardingData = agent.getDataForRoute("onboarding");
-const bookingData = agent.getDataForRoute("booking");
-
 return response.message;
 ```
 
 **Important Notes:**
 
 - `isRouteComplete` is `true` when the route reaches an `END_ROUTE` transition
-- The `message` will be empty (`""`) when `isRouteComplete` is `true`
+- The `message` contains the completion message generated by the AI when `isRouteComplete` is `true`
 - You should check `isRouteComplete` and handle completion appropriately
 - If all steps are skipped (due to `skipIf` conditions), the route can complete immediately on entry
 - Use `agent.getData(session)` to retrieve all collected data
@@ -462,7 +553,7 @@ for await (const chunk of agent.respondStream({ history })) {
 // Save to database
 await db.agentMessages.create({
   sessionId: session.id,
-  role: "agent",
+  role: "assistant",
   content: fullMessage,
   route: finalChunk.route?.title,
   toolCalls: finalChunk.toolCalls || [],
@@ -485,9 +576,9 @@ Agent's description (readonly).
 
 Agent's goal (readonly).
 
-##### `domain: Record<string, Record<string, unknown>>`
+##### `identity(): Template<TContext> | undefined`
 
-Dynamic domain registry access.
+Agent's identity template (readonly).
 
 ---
 
@@ -504,9 +595,11 @@ interface RouteOptions<TData = unknown> {
   id?: string;              // Optional custom ID (deterministic ID generated from title if not provided)
   title: string;            // Route title
   description?: string;     // Route description
+  identity?: string;        // Optional identity prompt defining the agent's role and persona for this route
+  personality?: Template;   // Optional personality prompt defining the agent's communication style for this route
   conditions?: string[];    // Conditions that activate this route
   guidelines?: Guideline[]; // Initial guidelines for this route
-  domains?: string[];       // Domain names allowed in this route (undefined = all domains)
+  terms?: Term[];          // Initial terms for the route's domain glossary
   rules?: string[];         // Absolute rules the agent MUST follow in this route
   prohibitions?: string[];  // Absolute prohibitions the agent MUST NEVER do in this route
 
@@ -524,14 +617,20 @@ interface RouteOptions<TData = unknown> {
   // NEW: Configure the initial step
   initialStep?: {
     id?: string;              // Custom ID for the initial step
-    instructions?: string;       // Description for the initial step
+    prompt?: string;       // Description for the initial step
     collect?: string[];        // Fields to collect in the initial step
     skipIf?: (data: Partial<TData>) => boolean;  // Skip condition
     requires?: string[];  // Required data prerequisites
   };
 
   // NEW: Sequential steps for simple linear flows
-  steps?: TransitionSpec<unknown, TData>[];
+  steps?: StepOptions<unknown, TData>[];
+  /** Knowledge base specific to this route containing any JSON structure the AI should know */
+  knowledgeBase?: Record<string, unknown>;
+  /**
+   * Route lifecycle hooks
+   */
+  hooks?: RouteLifecycleHooks<TContext, TData>;
 }
 ```
 
@@ -552,9 +651,13 @@ Adds a guideline specific to this route. Returns `this` for chaining.
 
 Returns all guidelines for this route.
 
-##### `getDomains(): string[] | undefined`
+##### `createTerm(term: Term): this`
+
+Adds a term to the route's domain glossary. Returns `this` for chaining.
 
-Returns allowed domain names for this route. Returns `undefined` if all domains are allowed, or an array of domain names if restricted.
+##### `getTerms(): Term[]`
+
+Returns all terms in the route's domain glossary.
 
 ##### `getRules(): string[]`
 
@@ -564,15 +667,101 @@ Returns the rules that must be followed in this route.
 
 Returns the prohibitions that must never be done in this route.
 
+##### `getKnowledgeBase(): Record<string, unknown>`
+
+Gets the route's knowledge base containing any JSON structure the AI should know.
+
+```typescript
+const knowledge = route.getKnowledgeBase();
+// Returns the route-specific knowledge base
+```
+
 ##### `getRef(): RouteRef`
 
 Returns a reference to this route.
 
-**Note:** Routes no longer have a `getData()` method. Use `agent.getData()` or `agent.getDataForRoute(routeId)` instead.
+##### `getAllSteps(): Step<TContext, TData>[]`
+
+Gets all steps in this route via traversal from the initial step.
+
+```typescript
+const steps = route.getAllSteps();
+// Returns array of all steps in the route
+```
+
+##### `getStep(stepId: string): Step<TContext, TData> | undefined`
+
+Gets a specific step by ID.
+
+```typescript
+const step = route.getStep("ask_destination");
+// Returns the step with the specified ID or undefined
+```
 
 ##### `describe(): string`
 
-Generates a description of this route's structure.
+Generates a description of this route's structure for debugging.
+
+```typescript
+const description = route.describe();
+console.log(description);
+// Output:
+// Route: Book Flight
+// ID: route_book_flight
+// Description: N/A
+// Conditions: None
+//
+// Steps:
+//   - step_ask_destination: Ask where to fly
+//     -> step_ask_dates: Ask about travel dates
+//     -> step_ask_passengers: How many passengers?
+```
+
+##### `handleDataUpdate(data, previousCollected): Promise<Partial<TData>>`
+
+Handles data updates for this route, calling the onDataUpdate hook if configured. Returns modified data after hook processing, or original data if no hook.
+
+```typescript
+const updatedData = await route.handleDataUpdate(newData, previousData);
+```
+
+**Parameters:**
+
+- `data`: New collected data
+- `previousCollected`: Previously collected data
+
+**Returns:** Modified data after hook processing
+
+##### `handleContextUpdate(newContext, previousContext): Promise<void>`
+
+Handles context updates for this route, calling the onContextUpdate hook if configured.
+
+```typescript
+await route.handleContextUpdate(newContext, previousContext);
+```
+
+**Parameters:**
+
+- `newContext`: New context
+- `previousContext`: Previous context
+
+##### `evaluateOnComplete(session, context?): Promise<RouteTransitionConfig<TContext, TData> | undefined>`
+
+Evaluates the onComplete handler and returns transition config.
+
+```typescript
+const transition = await route.evaluateOnComplete(
+  { data: session.data },
+  context
+);
+
+if (transition) {
+  // Transition to next route
+  console.log(`Next route: ${transition.nextStep}`);
+}
+```
+
+**Note:** Routes no longer have a `getData()` method. Use `agent.getData()` or `agent.getData(routeId)` instead.
 
 #### Properties
 
@@ -596,6 +785,10 @@ Conditions that trigger this route (readonly).
 
 Starting step of the route (readonly).
 
+##### `hooks?: RouteLifecycleHooks<TContext, TData>`
+
+Route lifecycle hooks for managing route-specific data and behavior (readonly).
+
 ---
 
 ### `Step`
@@ -604,13 +797,15 @@ Represents a step within a conversation route.
 
 #### Methods
 
-##### `nextStep(spec: TransitionSpec): TransitionResult`
+##### `nextStep(spec: StepOptions): StepResult`
 
 Creates a transition from this step and returns a chainable result.
 
 ```typescript
-interface TransitionSpec<TData = unknown> {
-  instructions?: string; // Transition to a chat interaction
+interface StepOptions<TData = unknown> {
+  id?: string; // step id
+  description?: string; // Step description
+  prompt?: string; // Transition to a chat interaction
   tool?: ToolRef; // Transition to execute a tool
   step?: StepRef | symbol; // Transition to specific step or END_ROUTE
 
@@ -627,18 +822,18 @@ interface TransitionSpec<TData = unknown> {
   condition?: string;
 }
 
-interface TransitionResult<TData = unknown> {
+interface StepResult<TData = unknown> {
   id: string; // Step identifier
   routeId: string; // Route identifier
-  nextStep: (spec: TransitionSpec<TData>) => TransitionResult<TData>;
+  nextStep: (spec: StepOptions<TData>) => StepResult<TData>;
 }
 ```
 
 **Parameters:**
 
-- `spec`: The transition specification (see `TransitionSpec` above). Can include an optional `condition` property for AI-evaluated step selection guidance.
+- `spec`: The transition specification (see `StepOptions` above). Can include an optional `condition` property for AI-evaluated step selection guidance.
 
-**Returns:** A `TransitionResult` that includes the target step's reference (`id`, `routeId`) and a `nextStep` method for chaining additional transitions.
+**Returns:** A `StepResult` that includes the target step's reference (`id`, `routeId`) and a `nextStep` method for chaining additional transitions.
 
 **Example:**
 
@@ -666,14 +861,14 @@ const flightRoute = agent.createRoute<FlightData>({
 
 // Approach 1: Step-by-step with data extraction and text conditions
 const askDestination = flightRoute.initialStep.nextStep({
-  instructions: "Ask where they want to fly",
+  prompt: "Ask where they want to fly",
   collect: ["destination"],
   skipIf: (data) => !!data.destination, // Skip if already have destination
   condition: "Customer hasn't specified destination yet", // AI-evaluated condition
 });
 
 const askDates = askDestination.nextStep({
-  instructions: "Ask about travel dates",
+  prompt: "Ask about travel dates",
   collect: ["departureDate"],
   skipIf: (data) => !!data.departureDate,
   requires: ["destination"], // Must have destination first
@@ -681,7 +876,7 @@ const askDates = askDestination.nextStep({
 });
 
 const askPassengers = askDates.nextStep({
-  instructions: "How many passengers?",
+  prompt: "How many passengers?",
   collect: ["passengers"],
   skipIf: (data) => !!data.passengers,
 });
@@ -693,24 +888,77 @@ console.log(askDestination.routeId); // Route ID
 // Approach 2: Fluent chaining for linear flows
 flightRoute.initialStep
   .nextStep({
-    instructions: "Extract travel details",
+    prompt: "Extract travel details",
     collect: ["destination", "departureDate", "passengers"],
   })
   .nextStep({
-    instructions: "Present available flights",
+    prompt: "Present available flights",
   })
   .nextStep({ step: END_ROUTE });
 
-// Use with session step
-let session = createSession<FlightData>();
-const response = await agent.respond({ history, session });
-console.log(response.session?.data); // { destination: "Paris", ... }
+// Automatic session management
+const agent = new Agent({
+  name: "Travel Agent",
+  provider: new OpenAIProvider({ apiKey: process.env.OPENAI_API_KEY }),
+  persistence: { adapter: new PrismaAdapter({ prisma }) },
+  sessionId: "user-123" // Automatically loads or creates session
+});
+
+const response = await agent.respond("I want to book a flight to Paris");
+console.log(agent.session.getData<FlightData>()); // { destination: "Paris", ... }
 ```
 
 ##### `addGuideline(guideline: Guideline): void`
 
 Adds a guideline specific to this step.
 
+##### `getGuidelines(): Guideline<TContext>[]`
+
+Gets all guidelines for this step.
+
+```typescript
+const guidelines = step.getGuidelines();
+// Returns array of guidelines specific to this step
+```
+
+##### `getTransitions(): Step<TContext, TData>[]`
+
+Gets all transitions from this step.
+
+```typescript
+const nextSteps = step.getTransitions();
+// Returns array of possible next steps
+```
+
+##### `shouldSkip(data: Partial<TData>): boolean`
+
+Checks if this step should be skipped based on collected data.
+
+```typescript
+if (step.shouldSkip(session.data)) {
+  // Skip this step
+}
+```
+
+##### `hasRequires(data: Partial<TData>): boolean`
+
+Checks if this step has all required data to proceed.
+
+```typescript
+if (step.hasRequires(session.data)) {
+  // Step can proceed
+}
+```
+
+##### `asStepResult(): StepResult<TContext, TData>`
+
+Creates a transition result for this step that supports chaining.
+
+```typescript
+const result = step.asStepResult();
+// Returns StepResult with nextStep method for chaining
+```
+
 ##### `configure(config): this`
 
 Configure the step properties after creation. Useful for overriding initial step configuration. Returns `this` for chaining.
@@ -719,15 +967,16 @@ Configure the step properties after creation. Useful for overriding initial step
 // Configure initial step after route creation
 route.initialStep.configure({
   description: "Welcome! Let's get started",
-  collectFields: ["name", "email"],
+  collect: ["name", "email"],
   skipIf: (data) => !!data.name && !!data.email,
   requires: [],
+  prompt: "Hello! Let's get started with your information.",
 });
 
 // Or configure any step
-const askName = route.initialStep.nextStep({ instructions: "Ask for name" });
+const askName = route.initialStep.nextStep({ prompt: "Ask for name" });
 askName.configure({
-  collectFields: ["firstName", "lastName"],
+  collect: ["firstName", "lastName"],
 });
 ```
 
@@ -735,9 +984,10 @@ askName.configure({
 
 - `config`: Configuration object with optional properties:
   - `description?: string` - Step description
-  - `collectFields?: string[]` - Fields to collect in this step
+  - `collect?: string[]` - Fields to collect in this step
   - `skipIf?: (data: Partial<TData>) => boolean` - Skip condition function
   - `requires?: string[]` - Required data prerequisites
+  - `prompt?: Template<TContext, TData>` - Step prompt template
 
 **Returns:** `this` for method chaining
 
@@ -757,59 +1007,13 @@ Step description (readonly).
 
 ---
 
-### `DomainRegistry`
-
-Registry for organizing agent tools and methods by domain.
-
-#### Methods
-
-##### `register<TDomain>(name: string, domain: TDomain): void`
-
-Registers a new domain with its tools/methods. Throws error if domain name already exists.
-
-##### `get<TDomain>(name: string): TDomain | undefined`
-
-Gets a registered domain by name. Returns `undefined` if not found.
-
-##### `has(name: string): boolean`
-
-Checks if a domain is registered.
-
-##### `all(): Record<string, Record<string, unknown>>`
-
-Returns all registered domains as a single object.
-
-##### `getFiltered(allowedNames?: string[]): Record<string, Record<string, unknown>>`
-
-Returns filtered domains by names. If `allowedNames` is `undefined`, returns all domains.
-
-**Example:**
-
-```typescript
-const registry = new DomainRegistry();
-
-registry.register("payment", {
-  processPayment: async (amount: number) => {
-    /* ... */
-  },
-});
-
-registry.register("shipping", {
-  calculateShipping: (zipCode: string) => {
-    /* ... */
-  },
-});
-
-// Get specific domains
-const filtered = registry.getFiltered(["payment"]); // Only payment domain
+---
 
-// Get all domains
-const all = registry.getFiltered(); // payment + shipping
-```
+## Tool Execution
 
-##### `getDomainNames(): string[]`
+Tools provide a powerful way to execute custom logic, access external APIs, and enrich conversation context before AI response generation.
 
-Returns list of all registered domain names.
+**See Also:** [TOOLS.md](./TOOLS.md) - Complete guide to tool execution, lifecycle, and best practices
 
 ---
 
@@ -1077,6 +1281,192 @@ Marks a session as completed.
 
 Marks a session as abandoned.
 
+##### `updateCollectedData(sessionId: string, collectedData: Record<string, unknown>): Promise<SessionData | null>`
+
+Updates collected data in session.
+
+##### `updateRouteStep(sessionId: string, route?: string, step?: string): Promise<SessionData | null>`
+
+Updates current route and step in session.
+
+##### `getUserMessages(userId?: string, limit?: number): Promise<MessageData[]>`
+
+Gets messages for a user.
+
+##### `deleteSession(sessionId: string): Promise<boolean>`
+
+Deletes a session and all its messages.
+
+##### `messageToEvent(message: MessageData): Event | undefined`
+
+Helper: Convert message data to Event format.
+
+##### `saveSessionState(sessionId: string, sessionStep: SessionState): Promise<SessionData | null>`
+
+Saves SessionState to database by converting to SessionData.
+
+```typescript
+const saved = await persistence.saveSessionState(sessionId, sessionState);
+```
+
+##### `loadSessionState(sessionId: string): Promise<SessionState | null>`
+
+Loads SessionState from database by converting from SessionData.
+
+```typescript
+const sessionState = await persistence.loadSessionState(sessionId);
+```
+
+##### `createSessionWithStep(options: CreateSessionOptions): Promise<{ sessionData: SessionData; sessionStep: SessionState }>`
+
+Creates a new session with both SessionData and initialized SessionState.
+
+```typescript
+const { sessionData, sessionStep } = await persistence.createSessionWithStep({
+  userId: "user_123",
+  agentName: "Travel Agent",
+  initialData: { channel: "web" },
+});
+```
+
+---
+
+### `RoutingEngine<TContext, TData>`
+
+Handles route and step selection logic for conversation orchestration.
+
+#### Constructor
+
+```typescript
+new RoutingEngine<TContext, TData>(options?: RoutingEngineOptions)
+
+interface RoutingEngineOptions {
+  allowRouteSwitch?: boolean;  // Default: true
+  switchThreshold?: number;    // Default: 70 (0-100)
+  maxCandidates?: number;      // Default: 5
+}
+```
+
+#### Methods
+
+##### `decideRouteAndStep(params): Promise<RoutingDecision>`
+
+Combines route selection and step selection into a single orchestrated decision.
+
+**Parameters:**
+
+- `routes`: Array of available routes
+- `session`: Current session state
+- `history`: Conversation history
+- `agentOptions`: Agent metadata
+- `provider`: AI provider for scoring
+- `context`: Agent context
+- `signal`: Optional abort signal
+
+**Returns:** Routing decision with selected route, step, directives, and updated session
+
+---
+
+### `ResponseEngine<TContext>`
+
+Builds prompts and response schemas for AI message generation.
+
+#### Constructor
+
+```typescript
+new ResponseEngine<TContext>();
+```
+
+#### Methods
+
+##### `responseSchemaForRoute(route, currentStep?): StructuredSchema`
+
+Builds JSON schema for AI responses based on route and current step.
+
+##### `buildResponsePrompt(params): Promise<string>`
+
+Builds a comprehensive prompt for AI response generation including context, guidelines, and route information.
+
+**Parameters:**
+
+- `route`: Current route
+- `currentStep`: Current step in route
+- `rules`: Route-specific rules
+- `prohibitions`: Route-specific prohibitions
+- `directives`: Response directives
+- `history`: Conversation history
+- `lastMessage`: Last user message
+- `agentOptions`: Agent metadata
+- `context`: Agent context
+- `session`: Current session state
+
+##### `buildFallbackPrompt(params): Promise<string>`
+
+Builds a fallback prompt when no routes are configured.
+
+---
+
+### `ToolExecutor<TContext, TData>`
+
+Executes tools with context and security enforcement.
+
+#### Constructor
+
+```typescript
+new ToolExecutor<TContext, TData>();
+```
+
+#### Methods
+
+##### `executeTool(params): Promise<ToolExecutionResult>`
+
+Executes a single tool with domain security enforcement.
+
+**Parameters:**
+
+- `tool`: Tool reference to execute
+- `context`: Agent context
+- `updateContext`: Function to update context
+- `history`: Conversation history
+- `data`: Collected session data
+- `allowedDomains`: Array of allowed domain names
+
+**Returns:** Tool execution result with success status and data
+
+##### `executeTools(params): Promise<ToolExecutionResult[]>`
+
+Executes multiple tools in sequence, stopping on first failure.
+
+---
+
+### `Events`
+
+Utility functions for creating and adapting conversation events.
+
+#### Functions
+
+##### `adaptEvent(event): string`
+
+Adapts an event for inclusion in AI prompts by transforming it into serializable format.
+
+```typescript
+const promptText = adaptEvent(messageEvent);
+```
+
+##### History Format
+
+For conversation history, use the simplified `History` type instead of manually creating events. See the [History Format](#history-format) section below for details.
+
+##### `createToolEvent(source, toolCalls, timestamp?): Event`
+
+Creates a tool execution event.
+
+```typescript
+const toolEvent = createToolEvent(MessageRole.AGENT, [
+  { tool_id: "search_flights", arguments: { from: "NYC", to: "LAX" }, result: { flights: [...] } }
+]);
+```
+
 ---
 
 ### `PrismaAdapter`
@@ -1476,7 +1866,7 @@ interface MessageData {
   id: string;
   sessionId: string;
   userId?: string;
-  role: MessageRole; // "user" | "agent" | "system"
+  role: MessageRole; // "user" | "assistant" | "agent" | "system"
   content: string;
   route?: string;
   step?: string;
@@ -1532,15 +1922,45 @@ Adds domain glossary terms.
 
 Adds guidelines section.
 
-##### `addCapabilitiesForMessageGeneration(capabilities: Capability[]): this`
+##### `addActiveRoutes(routes): this`
+
+Adds active routes to the prompt.
+
+##### `build(): string`
+
+Builds the final prompt string.
 
-Adds capabilities section.
+---
+
+### `PromptComposer`
+
+Constructs prompts for AI generation.
+
+**Note:** As of the latest version, many methods in `PromptComposer` are `async` to support function-based templates.
+
+#### Methods
+
+##### `addIdentity(name: string, description?: string, goal?: string): this`
+
+Adds agent identity section.
+
+##### `addContext(variables: ContextVariable[]): this`
+
+Adds context variables.
+
+##### `addGlossary(terms: Term[]): this`
+
+Adds domain glossary terms.
+
+##### `addGuidelinesForMessageGeneration(guidelines: GuidelineMatch[]): this`
+
+Adds guidelines section.
 
 ##### `addActiveRoutes(routes): this`
 
 Adds active routes to the prompt.
 
-##### `build(): string`
+##### `build(): Promise<string>`
 
 Builds the final prompt string.
 
@@ -1548,67 +1968,181 @@ Builds the final prompt string.
 
 ## Utility Functions
 
-### `defineTool<TContext, TArgs, TReturn>()`
+### `Tool<TContext, TArgs, TResult, TData>`
 
-Defines a type-safe tool.
+Defines a type-safe tool using the Tool interface.
 
 ```typescript
-defineTool<TContext, TArgs extends unknown[], TReturn>(
-  name: string,
-  handler: ToolHandler<TContext, TArgs, TReturn>,
-  options?: {
-    id?: string;              // Optional custom ID (deterministic ID generated from name if not provided)
-    description?: string;
-    parameters?: unknown;
-  }
-): ToolRef<TContext, TArgs, TReturn>
+interface Tool<
+  TContext = unknown,
+  TArgs extends unknown[] = unknown[],
+  TResult = unknown,
+  TData = unknown
+> {
+  id: string; // Unique tool identifier
+  description?: string; // Description for AI discovery
+  parameters?: unknown; // JSON Schema for tool parameters
+  handler: ToolHandler<TContext, TArgs, TResult, TData>; // Tool execution handler
+}
+```
+
+**Example:**
+
+```typescript
+const getTool: Tool<MyContext, [id: string], Data> = {
+  id: "get_data",
+  description: "Fetches data by ID",
+  parameters: {
+    type: "object",
+    properties: {
+      id: { type: "string", description: "The data ID to fetch" },
+    },
+    required: ["id"],
+  },
+  handler: async (toolContext, args) => {
+    return { data: await fetchData(args.id) };
+  },
+};
 ```
 
-**Note on IDs:** Tool IDs are deterministic by default, generated from the name using a hash function. This ensures consistency across server restarts.
+---
+
+### `formatKnowledgeBase(data, title?, maxDepth?)`
+
+Formats a JSON structure into readable markdown format for AI prompts. Handles nested objects, arrays, and primitive values.
+
+```typescript
+formatKnowledgeBase(
+  data: Record<string, unknown> | unknown,
+  title?: string,
+  maxDepth?: number
+): string
+```
+
+**Parameters:**
+
+- `data` - The JSON data to format
+- `title` - Optional title for the knowledge base section
+- `maxDepth` - Maximum nesting depth (default: 3)
+
+**Returns:** Formatted markdown string
 
 **Example:**
 
 ```typescript
-const getTool = defineTool<MyContext, [id: string], Data>(
-  "get_data",
-  async ({ context }, id) => {
-    return { data: await fetchData(id) };
+import { formatKnowledgeBase } from "@falai/agent";
+
+const knowledge = {
+  company: {
+    name: "Acme Corp",
+    products: ["Widget A", "Widget B"],
+    locations: {
+      headquarters: "NYC",
+      branches: ["LA", "Chicago"],
+    },
   },
-  {
-    id: "custom_get_data_tool", // Optional: provide your own ID
-    description: "Fetches data by ID",
-  }
-);
+};
+
+const markdown = formatKnowledgeBase(knowledge, "Company Information");
+// Output:
+// ## Company Information
+//
+// - **name**: Acme Corp
+// - **products**:
+//   - Widget A
+//   - Widget B
+// - **locations**:
+//   - **headquarters**: NYC
+//   - **branches**:
+//     - LA
+//     - Chicago
 ```
 
 ---
 
-### `createMessageEvent()`
+### History Format
+
+The agent accepts a simplified history format for conversation context. This format is easier to use than the internal Event structure.
 
-Creates a message event for conversation history.
+#### Types
 
 ```typescript
-createMessageEvent(
-  source: EventSource,
-  participantName: string,
-  message: string,
-  timestamp?: string  // Optional: provide custom timestamp (ISO 8601 format)
-): Event
+type Role = "user" | "assistant" | "tool" | "system";
+
+type HistoryItem =
+  | {
+      role: "user";
+      content: string;
+      name?: string; // Optional participant name
+    }
+  | {
+      role: "assistant";
+      content: string | null;
+      tool_calls?: Array<{
+        id: string;
+        name: string;
+        arguments: Record<string, unknown>;
+      }>;
+    }
+  | {
+      role: "tool";
+      tool_call_id: string;
+      name: string;
+      content: any;
+    }
+  | {
+      role: "system";
+      content: string;
+    };
+
+type History = HistoryItem[];
 ```
 
 **Example:**
 
 ```typescript
-// With auto-generated timestamp
-createMessageEvent(EventSource.CUSTOMER, "Alice", "Hello!");
+// Simple conversation history
+const history: History = [
+  {
+    role: "user",
+    content: "Hello! Can you help me book a flight?",
+    name: "Alice"
+  },
+  {
+    role: "assistant",
+    content: "I'd be happy to help you book a flight. Where would you like to go?"
+  },
+  {
+    role: "user",
+    content: "I want to go to Paris next Friday for 2 people.",
+    name: "Alice"
+  }
+];
 
-// With custom timestamp (useful for historical data)
-createMessageEvent(
-  EventSource.CUSTOMER,
-  "Alice",
-  "Hello!",
-  "2025-10-13T10:30:00Z"
-);
+// Assistant with tool calls
+const historyWithTools: History = [
+  {
+    role: "assistant",
+    content: "I'll search for flights to Paris.",
+    tool_calls: [
+      {
+        id: "search_flights_1",
+        name: "search_flights",
+        arguments: {
+          destination: "Paris",
+          date: "2025-10-18",
+          passengers: 2
+        }
+      }
+    ]
+  },
+  {
+    role: "tool",
+    tool_call_id: "search_flights_1",
+    name: "search_flights",
+    content: { flights: [...] }  // Tool execution result
+  }
+];
 ```
 
 ---
@@ -1619,7 +2153,7 @@ Creates a tool execution event.
 
 ```typescript
 createToolEvent(
-  source: EventSource,
+  source: MessageRole,
   toolCalls: ToolCall[],
   timestamp?: string  // Optional: provide custom timestamp (ISO 8601 format)
 ): Event
@@ -1629,13 +2163,13 @@ createToolEvent(
 
 ```typescript
 // With auto-generated timestamp
-createToolEvent(EventSource.AI_AGENT, [
+createToolEvent(MessageRole.AGENT, [
   { tool_id: "get_data", arguments: { id: "123" }, result: { data: {...} } }
 ]);
 
 // With custom timestamp
 createToolEvent(
-  EventSource.AI_AGENT,
+  MessageRole.AGENT,
   [{ tool_id: "get_data", arguments: { id: "123" }, result: { data: {...} } }],
   "2025-10-13T10:30:00Z"
 );
@@ -1658,10 +2192,10 @@ adaptEvent(event: EmittedEvent): Event
 ### `Term`
 
 ```typescript
-interface Term {
-  name: string;
-  description: string;
-  synonyms?: string[];
+interface Term<TContext = unknown> {
+  name: Template<TContext>;
+  description: Template<TContext>;
+  synonyms?: Template<TContext>[];
 }
 ```
 
@@ -1670,10 +2204,10 @@ interface Term {
 ### `Guideline`
 
 ```typescript
-interface Guideline {
+interface Guideline<TContext = unknown> {
   id?: string;
-  condition?: string;
-  action: string;
+  condition?: Template<TContext>;
+  action: Template<TContext>;
   enabled?: boolean; // Default: true
   tags?: string[];
   tools?: ToolRef[];
@@ -1683,25 +2217,47 @@ interface Guideline {
 
 ---
 
-### `Capability`
+### `RouteLifecycleHooks<TContext, TData>`
+
+Route lifecycle hooks for managing route-specific data and behavior.
 
 ```typescript
-interface Capability {
-  id?: string;
-  title: string;
-  description: string;
-  tools?: ToolRef[];
+interface RouteLifecycleHooks<TContext = unknown, TData = unknown> {
+  /**
+   * Called after collected data is updated for this route (from AI response or tool execution)
+   * Useful for validation, enrichment, or persistence of route-specific collected data
+   * Return modified collected data or the same data to keep it unchanged
+   *
+   * Unlike Agent-level onDataUpdate, this only triggers for data changes in this specific route.
+   */
+  onDataUpdate?: (
+    data: Partial<TData>,
+    previousCollected: Partial<TData>
+  ) => Partial<TData> | Promise<Partial<TData>>;
+
+  /**
+   * Called after context is updated via updateContext() when this route is active
+   * Useful for route-specific context reactions, validation, or side effects
+   *
+   * Unlike Agent-level onContextUpdate, this only triggers when this specific route is active.
+   */
+  onContextUpdate?: (
+    newContext: TContext,
+    previousContext: TContext
+  ) => void | Promise<void>;
 }
 ```
 
 ---
 
+---
+
 ### `Event`
 
 ```typescript
 interface Event {
   kind: EventKind;
-  source: EventSource;
+  source: MessageRole;
   data: MessageEventData | ToolEventData | StatusEventData;
   creation_utc: string;
 }
@@ -1712,9 +2268,10 @@ enum EventKind {
   STATUS = "status",
 }
 
-enum EventSource {
-  CUSTOMER = "customer",
-  AI_AGENT = "agent",
+enum MessageRole {
+  USER = "user",
+  ASSISTANT = "assistant",
+  AGENT = "agent",
   SYSTEM = "system",
 }
 ```
@@ -1743,6 +2300,39 @@ interface ToolResult<TReturn> {
 
 ---
 
+### `RoutingDecision`
+
+Result of route and step selection process.
+
+```typescript
+interface RoutingDecision {
+  context: string;
+  routes: Record<string, number>; // Route scores
+  responseDirectives?: string[];
+  extractions?: unknown;
+  contextUpdate?: Record<string, unknown>;
+}
+```
+
+---
+
+### `ToolExecutionResult`
+
+Result of tool execution.
+
+```typescript
+interface ToolExecutionResult {
+  toolName: string;
+  success: boolean;
+  data?: unknown;
+  contextUpdate?: Record<string, unknown>;
+  dataUpdate?: Record<string, unknown>;
+  error?: string;
+}
+```
+
+---
+
 ## Types
 
 ### `SessionState<TData>`
@@ -1772,14 +2362,14 @@ interface SessionState<TData = Record<string, unknown>> {
    * Data collected during the current route
    * Convenience reference to dataByRoute[currentRoute.id]
    */
-  data: Partial<TData>;
+  data?: Partial<TData>;
 
   /**
-   * Collected data organized by route ID
-   * Preserves data when switching between routes
-   * Format: { "routeId": { ...dataData } }
+   * Data collected organized by route ID
+   * Persists data when switching between routes
+   * Allows resuming incomplete routes where they left off
    */
-  dataByRoute: Record<string, Partial<unknown>>;
+  dataByRoute?: Record<string, Partial<unknown>>;
 
   /** History of routes visited in this session */
   routeHistory: Array<{
@@ -1789,6 +2379,16 @@ interface SessionState<TData = Record<string, unknown>> {
     completed: boolean;
   }>;
 
+  /**
+   * Pending route transition after completion
+   * Set when a route completes with onComplete handler
+   */
+  pendingTransition?: {
+    targetRouteId: string;
+    condition?: string;
+    reason: "route_complete" | "manual";
+  };
+
   /** Session metadata */
   metadata?: {
     createdAt?: Date;
@@ -1802,9 +2402,10 @@ interface SessionState<TData = Record<string, unknown>> {
 
 - **`id`** - Optional session identifier that persists across database operations
 - **`data`** - Type-safe data collected via `schema` for the **current route**
-- **`dataByRoute`** - **NEW:** Per-route data map that preserves collected data when switching routes
+- **`dataByRoute`** - **Per-route data map that preserves collected data when switching routes**
 - **`currentRoute`** / **`currentStep`** - Track conversation position
 - **`routeHistory`** - Full audit trail of route transitions
+- **`pendingTransition`** - Handles route completion transitions
 - **`metadata`** - Custom data (timestamps, user info, etc.)
 
 **Per-Route Data Preservation:**
@@ -1902,14 +2503,14 @@ console.log(session.currentStep?.id); // "ask_destination"
 console.log(session.currentStep?.description); // "Ask where to fly"
 ```
 
-#### `mergeData<TData>(session, data): SessionState<TData>`
+#### `mergeCollected<TData>(session, data): SessionState<TData>`
 
 Merges new collected data into session. Updates timestamps automatically.
 
 **Example:**
 
 ```typescript
-session = mergeData(session, {
+session = mergeCollected(session, {
   destination: "Paris",
   departureDate: "2025-06-15",
 });
@@ -2088,7 +2689,7 @@ Symbol marking the end of a conversation route. Use this when building routes to
 import { END_ROUTE } from "@falai/agent";
 
 const thankYou = askEmail.nextStep({
-  instructions: "Thank you for your information!",
+  prompt: "Thank you for your information!",
 });
 
 // Mark the end of the route
@@ -2144,7 +2745,7 @@ enum BuiltInSection {
   GLOSSARY = "glossary",
   CONTEXT = "context",
   GUIDELINES = "guidelines",
-  CAPABILITIES = "capabilities",
+  TOOLS
   ACTIVE_ROUTES = "active_routes",
 }
 ```
@@ -2165,23 +2766,92 @@ interface AiProvider {
 }
 ```
 
-### Domain Registry
+---
+
+## Template Utilities
+
+### Dynamic Content with Templates
 
-Organize tools by domain:
+The framework supports dynamic content generation through a versatile `Template` system. A `Template` can be either a simple string with `{{variable}}` placeholders or a function that returns a string, allowing for more complex, context-aware logic.
+
+Templates are evaluated at runtime with a `TemplateContext` object that provides access to the agent's state:
 
 ```typescript
-const paymentDomain = {
-  processPayment: async (amount: number) => {
-    /*...*/
-  },
-  refund: async (transactionId: string) => {
-    /*...*/
-  },
-};
+interface TemplateContext<TContext = unknown, TData = unknown> {
+  context?: TContext;
+  session?: SessionState<TData>;
+  history?: Event[];
+  data?: Partial<TData>; // Convenience alias for session.data
+}
+```
+
+#### `Template<TContext, TData>` Type
 
-agent.addDomain("payment", paymentDomain);
+This is the core type for all dynamic content:
+
+```typescript
+type Template<TContext = unknown, TData = unknown> =
+  | string
+  | ((params: TemplateContext<TContext, TData>) => string | Promise<string>);
+```
 
-// Access via agent.domain.payment.processPayment()
+**String Templates:**
+
+Simple strings with `{{variable}}` placeholders are rendered using values from the `context` object.
+
+```typescript
+const agent = new Agent({
+  identity: "I am {{name}}",
+  context: { name: "HelperBot" },
+});
+```
+
+**Function Templates:**
+
+For more complex logic, you can provide a function that receives the `TemplateContext` and returns a string (or a `Promise<string>` for async operations).
+
+```typescript
+const agent = new Agent({
+  identity: ({ context }) => `I am here to help, ${context.user.name}`,
+  context: { user: { name: "Alice" } },
+});
+```
+
+#### `render(template, params)`
+
+An internal async utility that resolves a `Template` to a string. You won't typically call this directly, but it powers the dynamic content generation throughout the framework.
+
+**Supported in (Dynamic/Personalized):**
+
+- **Agent**: `identity`
+- **Terms**: `name`, `description`, `synonyms`
+- **Guidelines**: `condition`, `action`
+- **Routes**: `conditions`, `rules`, `prohibitions`
+- **Steps/Transitions**: `prompt`, `condition`
+
+**Not supported in (Static/Predictable):**
+
+- Agent `name`, `goal`, `description`
+- Route `title`, `description`
+- Step `description`
+
+**Example:**
+
+```typescript
+const agent = new Agent({
+  name: "Assistant", // Static
+  identity: ({ context }) =>
+    `I am here to help you, ${context.user.firstName}.`, // Dynamic
+  context: {
+    user: { firstName: "Alice", age: 30 },
+  },
+});
+
+agent.createGuideline({
+  condition: ({ context }) => context.user.age > 18, // Dynamic condition
+  action:
+    "Be helpful to {{user.firstName}} and consider their age in responses", // Dynamic action
+});
 ```
 
 ---