@falai/agent 0.4.1 → 0.5.1

package/docs/ADAPTERS.md

@@ -2,6 +2,8 @@
 
 All adapters follow the **provider pattern** - no dependencies required in the package, users install only what they need.
 
+**NEW**: All adapters now support the new **Session State** pattern with automatic persistence of extracted data, current route/state, and conversation progress!
+
 ## ✅ Implemented Adapters
 
 ### 1. **PrismaAdapter**
@@ -73,11 +75,17 @@ All adapters follow the **provider pattern** - no dependencies required in the p
 
 ## 🎯 Usage Pattern
 
-All adapters follow the same simple pattern:
+All adapters follow the same simple pattern with full session state support:
 
 ```typescript
 import { Agent, [Adapter]Adapter } from "@falai/agent";
 
+// Define your data extraction type
+interface YourDataType {
+  field1: string;
+  field2: number;
+}
+
 const adapter = new [Adapter]Adapter({
   client: yourClientInstance,
   // ... adapter-specific options
@@ -87,9 +95,45 @@ const agent = new Agent({
   persistence: {
     adapter,
     userId: "user_123",
-    autoSave: true,
+    autoSave: true, // ✨ Auto-saves session state!
+  },
+});
+
+// Create a route with data extraction
+const route = agent.createRoute<YourDataType>({
+  title: "My Route",
+  gatherSchema: {
+    type: "object",
+    properties: {
+      field1: { type: "string" },
+      field2: { type: "number" },
+    },
+    required: ["field1", "field2"],
   },
 });
+
+// Define states
+route.initialState.transitionTo({
+  chatState: "Collect data",
+  gather: ["field1", "field2"],
+});
+
+// Use with session state
+const persistence = agent.getPersistenceManager();
+const { sessionData, sessionState } =
+  await persistence.createSessionWithState<YourDataType>({
+    userId: "user_123",
+    agentName: "My Agent",
+  });
+
+// Chat with automatic session state persistence
+const response = await agent.respond({
+  history: [...],
+  session: sessionState, // Pass session state
+});
+
+// Session state auto-saved! Includes extracted data
+console.log("Extracted:", response.session?.extracted);
 ```
 
 ## 🔌 Optional Dependencies
@@ -150,9 +194,45 @@ export class MyCustomAdapter implements PersistenceAdapter {
 
 All adapters are fully typed with **zero `any` types** (except for Prisma's dynamic model access):
 
-- Generic client interfaces
+- Generic client interfaces with `SessionState<TExtracted>` support
 - Typed repository methods
 - Full IDE autocomplete
+- Type-safe data extraction throughout
+
+## 💾 What Gets Stored
+
+All adapters store session state in the `collectedData` JSON field:
+
+```json
+{
+  "extracted": {
+    "destination": "Paris",
+    "departureDate": "2025-06-15",
+    "passengers": 2
+  },
+  "routeHistory": [
+    {
+      "routeId": "book_flight",
+      "enteredAt": "2025-10-15T10:00:00Z",
+      "completed": false
+    }
+  ],
+  "currentRouteTitle": "Book a Flight",
+  "currentStateDescription": "Ask about travel dates",
+  "metadata": {
+    "sessionId": "session_123",
+    "createdAt": "2025-10-15T10:00:00Z",
+    "lastUpdatedAt": "2025-10-15T10:05:00Z"
+  }
+}
+```
+
+This allows:
+
+- ✅ Full session state recovery
+- ✅ Analytics on extracted data
+- ✅ Conversation progress tracking
+- ✅ Multi-turn conversation support
 
 ## 🚀 Coming Soon
 

package/docs/API_REFERENCE.md

@@ -36,10 +36,6 @@ Adds a behavioral guideline. Returns `this` for chaining.
 
 Adds an agent capability. Returns `this` for chaining.
 
-##### `createObservation(description: string): Observation`
-
-Creates an observation for disambiguation.
-
 ##### `addDomain<TName, TDomain>(name: TName, domainObject: TDomain): void`
 
 Registers a domain with tools/methods.
@@ -54,12 +50,12 @@ Gets allowed domains for a specific route by title. Returns filtered domains bas
 
 ##### `respond(input: RespondInput<TContext>): Promise<RespondOutput>`
 
-Generates an AI response based on conversation history with structured output including route, state, and tool call information.
+Generates an AI response with session state management, data extraction, and intelligent routing.
 
 ```typescript
 interface RespondInput<TContext> {
   history: Event[];
-  state?: StateRef;
+  session?: SessionState; // NEW: Session state for conversation tracking
   contextOverride?: Partial<TContext>;
   signal?: AbortSignal;
 }
@@ -67,11 +63,9 @@ interface RespondInput<TContext> {
 interface RespondOutput {
   /** The message to send to the user */
   message: string;
-  /** Route chosen by the agent (if applicable) */
-  route?: { id: string; title: string };
-  /** Current state within the route (if applicable) */
-  state?: { id: string; description?: string };
-  /** Tool calls requested by the agent */
+  /** Updated session state (includes extracted data, current route/state) */
+  session?: SessionState;
+  /** Tool calls executed during response (for debugging) */
   toolCalls?: Array<{
     toolName: string;
     arguments: Record<string, unknown>;
@@ -79,12 +73,19 @@ interface RespondOutput {
 }
 ```
 
-**Enhanced Response Information:**
+**Enhanced Response Pipeline:**
+
+1. **Tool Execution** - Execute tools if current state has `toolState`
+2. **Always-On Routing** - Score all routes, respect user intent to change direction
+3. **State Traversal** - Use `skipIf` and `requiredData` to determine next state
+4. **Response Generation** - Build schema with `gather` fields, extract data
+5. **Session Update** - Merge extracted data into session state
+
+**Session State Management:**
 
-- Uses structured JSON output via `responses.parse()` API (OpenAI/OpenRouter) or JSON mode (Gemini)
-- Automatically detects which route the agent chose based on conversation context
-- Provides current state information for tracking conversation flow
-- Returns tool calls for execution in your application
+- Tracks current route, state, and extracted data across turns
+- Enables "I changed my mind" scenarios with context-aware routing
+- Automatically merges new extracted data with existing session data
 
 **Example:**
 
@@ -239,7 +240,7 @@ Represents a conversation flow with states and transitions.
 ```typescript
 new Route(options: RouteOptions)
 
-interface RouteOptions {
+interface RouteOptions<TExtracted = unknown> {
   id?: string;              // Optional custom ID (deterministic ID generated from title if not provided)
   title: string;            // Route title
   description?: string;     // Route description
@@ -248,6 +249,17 @@ interface RouteOptions {
   domains?: string[];       // Domain names allowed in this route (undefined = all domains)
   rules?: string[];         // Absolute rules the agent MUST follow in this route
   prohibitions?: string[];  // Absolute prohibitions the agent MUST NEVER do in this route
+
+  // NEW: Schema-first data extraction
+  gatherSchema?: {
+    type: "object";
+    properties: Record<string, any>;
+    required?: string[];
+    additionalProperties?: boolean;
+  };
+
+  // NEW: Pre-populate extracted data when entering route
+  initialData?: Partial<TExtracted>;
 }
 ```
 
@@ -279,10 +291,6 @@ Returns the prohibitions that must never be done in this route.
 
 Returns a reference to this route.
 
-##### `toRef(): RouteRef`
-
-Returns a reference including the route instance (for disambiguation).
-
 ##### `describe(): string`
 
 Generates a description of this route's structure.
@@ -322,16 +330,28 @@ Represents a state within a conversation route.
 Creates a transition from this state and returns a chainable result.
 
 ```typescript
-interface TransitionSpec {
+interface TransitionSpec<TExtracted = unknown> {
   chatState?: string; // Transition to a chat interaction
   toolState?: ToolRef; // Transition to execute a tool
   state?: StateRef | symbol; // Transition to specific state or END_ROUTE
+
+  // NEW: Data extraction fields for this state
+  gather?: string[];
+
+  // NEW: Code-based condition to skip this state
+  skipIf?: (extracted: Partial<TExtracted>) => boolean;
+
+  // NEW: Prerequisites that must be met to enter this state
+  requiredData?: string[];
 }
 
-interface TransitionResult {
+interface TransitionResult<TExtracted = unknown> {
   id: string; // State identifier
   routeId: string; // Route identifier
-  transitionTo: (spec: TransitionSpec, condition?: string) => TransitionResult;
+  transitionTo: (
+    spec: TransitionSpec<TExtracted>,
+    condition?: string
+  ) => TransitionResult<TExtracted>;
 }
 ```
 
@@ -340,39 +360,66 @@ interface TransitionResult {
 **Example:**
 
 ```typescript
-// Approach 1: Step-by-step (ideal for complex flows with branching)
-const t0 = route.initialState.transitionTo({
-  chatState: "Ask for user name",
-});
+// Define your data extraction type
+interface FlightData {
+  destination: string;
+  departureDate: string;
+  passengers: number;
+}
 
-const t1 = t0.transitionTo({
-  chatState: "Ask for email",
+// Create a data-driven route
+const flightRoute = agent.createRoute<FlightData>({
+  title: "Book Flight",
+  gatherSchema: {
+    type: "object",
+    properties: {
+      destination: { type: "string" },
+      departureDate: { type: "string" },
+      passengers: { type: "number", minimum: 1, maximum: 9 },
+    },
+    required: ["destination", "departureDate", "passengers"],
+  },
 });
 
-const t2 = t1.transitionTo({
-  chatState: "Confirm details",
+// Approach 1: Step-by-step with data extraction
+const askDestination = flightRoute.initialState.transitionTo({
+  chatState: "Ask where they want to fly",
+  gather: ["destination"],
+  skipIf: (extracted) => !!extracted.destination, // Skip if already have destination
 });
 
-// Access state properties
-console.log(t1.id); // State ID
-console.log(t1.routeId); // Route ID
+const askDates = askDestination.transitionTo({
+  chatState: "Ask about travel dates",
+  gather: ["departureDate"],
+  skipIf: (extracted) => !!extracted.departureDate,
+  requiredData: ["destination"], // Must have destination first
+});
 
-// Use saved references for branching
-t1.transitionTo(
-  { chatState: "Handle invalid email" },
-  "Email validation failed"
-);
+const askPassengers = askDates.transitionTo({
+  chatState: "How many passengers?",
+  gather: ["passengers"],
+  skipIf: (extracted) => !!extracted.passengers,
+});
 
-// Approach 2: Fluent chaining (elegant for linear flows)
-route.initialState
-  .transitionTo({ chatState: "Ask for user name" })
-  .transitionTo({ chatState: "Ask for email" })
-  .transitionTo({ chatState: "Confirm details" })
+// Access state properties
+console.log(askDestination.id); // State ID
+console.log(askDestination.routeId); // Route ID
+
+// Approach 2: Fluent chaining for linear flows
+flightRoute.initialState
+  .transitionTo({
+    chatState: "Extract travel details",
+    gather: ["destination", "departureDate", "passengers"],
+  })
+  .transitionTo({
+    chatState: "Present available flights",
+  })
   .transitionTo({ state: END_ROUTE });
 
-// Both approaches are equivalent - choose based on your needs:
-// - Use step-by-step for complex flows with conditional branches
-// - Use chaining for simple linear flows for conciseness
+// Use with session state
+let session = createSession<FlightData>();
+const response = await agent.respond({ history, session });
+console.log(response.session?.extracted); // { destination: "Paris", ... }
 ```
 
 ##### `addGuideline(guideline: Guideline): void`
@@ -395,46 +442,6 @@ State description (readonly).
 
 ---
 
-### `Observation`
-
-Handles disambiguation between multiple routes.
-
-#### Constructor
-
-```typescript
-new Observation(options: ObservationOptions)
-
-interface ObservationOptions {
-  id?: string;          // Optional custom ID (deterministic ID generated from description if not provided)
-  description: string;  // The observation description
-  routeRefs?: string[]; // Route IDs or titles to disambiguate between
-}
-```
-
-**Note on IDs:** Observation IDs are deterministic by default, generated from the description using a hash function. This ensures consistency across server restarts.
-
-#### Methods
-
-##### `disambiguate(routes: (Route | RouteRef)[]): this`
-
-Sets routes this observation can disambiguate between. Returns `this` for chaining.
-
-##### `getRoutes(): RouteRef[]`
-
-Returns routes associated with this observation.
-
-#### Properties
-
-##### `id: string`
-
-Unique identifier (readonly).
-
-##### `description: string`
-
-Observation description (readonly).
-
----
-
 ### `DomainRegistry`
 
 Registry for organizing agent tools and methods by domain.
@@ -1214,10 +1221,6 @@ Adds guidelines section.
 
 Adds capabilities section.
 
-##### `addObservations(observations): this`
-
-Adds observations for disambiguation.
-
 ##### `addActiveRoutes(routes): this`
 
 Adds active routes to the prompt.
@@ -1427,6 +1430,240 @@ interface ToolResult<TReturn> {
 
 ## Types
 
+### `SessionState<TExtracted>`
+
+Tracks the current position in the conversation flow and data extracted during route progression.
+
+```typescript
+interface SessionState<TExtracted = Record<string, unknown>> {
+  /** Unique session identifier (useful for persistence) */
+  id?: string;
+
+  /** Current route the conversation is in */
+  currentRoute?: {
+    id: string;
+    title: string;
+    enteredAt: Date;
+  };
+
+  /** Current state within the route */
+  currentState?: {
+    id: string;
+    description?: string;
+    enteredAt: Date;
+  };
+
+  /** Data extracted during the current route */
+  extracted: Partial<TExtracted>;
+
+  /** History of routes visited in this session */
+  routeHistory: Array<{
+    routeId: string;
+    enteredAt: Date;
+    exitedAt?: Date;
+    completed: boolean;
+  }>;
+
+  /** Session metadata */
+  metadata?: {
+    createdAt?: Date;
+    lastUpdatedAt?: Date;
+    [key: string]: unknown;
+  };
+}
+```
+
+**Key Features:**
+
+- **`id`** - Optional session identifier that persists across database operations
+- **`extracted`** - Type-safe data collected via `gatherSchema`
+- **`currentRoute`** / **`currentState`** - Track conversation position
+- **`routeHistory`** - Full audit trail of route transitions
+- **`metadata`** - Custom data (timestamps, user info, etc.)
+
+**Usage:**
+
+```typescript
+interface FlightData {
+  destination: string;
+  departureDate: string;
+}
+
+// Create session with database ID
+const session = createSession<FlightData>("session_abc123");
+
+// Use in conversation
+const response = await agent.respond({ history, session });
+
+// Access extracted data
+console.log(response.session?.extracted.destination); // Type-safe!
+```
+
+---
+
+### Session Helper Functions
+
+#### `createSession<TExtracted>(sessionId?, metadata?): SessionState<TExtracted>`
+
+Creates a new session state object.
+
+**Parameters:**
+
+- `sessionId` (optional): Unique session identifier from database
+- `metadata` (optional): Additional metadata to attach
+
+**Example:**
+
+```typescript
+// Simple usage
+const session = createSession<OnboardingData>();
+
+// With database ID (when loading from persistence)
+const session = createSession<OnboardingData>("session_123");
+
+// With metadata
+const session = createSession<OnboardingData>("session_123", {
+  userId: "user_456",
+  channel: "whatsapp",
+});
+```
+
+#### `enterRoute<TExtracted>(session, routeId, routeTitle): SessionState<TExtracted>`
+
+Updates session when entering a new route. Automatically:
+
+- Exits previous route (if exists)
+- Resets extracted data
+- Adds route to history
+- Updates timestamps
+
+**Example:**
+
+```typescript
+let session = createSession<FlightData>();
+
+// Enter booking route
+session = enterRoute(session, "book_flight", "Book a Flight");
+
+console.log(session.currentRoute?.title); // "Book a Flight"
+console.log(session.extracted); // {} (reset for new route)
+```
+
+#### `enterState<TExtracted>(session, stateId, description?): SessionState<TExtracted>`
+
+Updates session when entering a new state within a route.
+
+**Example:**
+
+```typescript
+session = enterState(session, "ask_destination", "Ask where to fly");
+
+console.log(session.currentState?.id); // "ask_destination"
+console.log(session.currentState?.description); // "Ask where to fly"
+```
+
+#### `mergeExtracted<TExtracted>(session, data): SessionState<TExtracted>`
+
+Merges new extracted data into session. Updates timestamps automatically.
+
+**Example:**
+
+```typescript
+session = mergeExtracted(session, {
+  destination: "Paris",
+  departureDate: "2025-06-15",
+});
+
+console.log(session.extracted); // { destination: "Paris", departureDate: "2025-06-15" }
+```
+
+#### `sessionStateToData<TExtracted>(session): object`
+
+Converts SessionState to persistence-friendly format for database storage.
+
+**Returns:**
+
+```typescript
+{
+  currentRoute?: string;          // Route ID
+  currentState?: string;          // State ID
+  collectedData: {                // All session data
+    extracted: Partial<TExtracted>;
+    routeHistory: Array<...>;
+    currentRouteTitle?: string;
+    currentStateDescription?: string;
+    metadata?: object;
+  };
+}
+```
+
+**Example:**
+
+```typescript
+const session = createSession<FlightData>("session_123");
+// ... conversation happens ...
+
+// Save to database
+const dbData = sessionStateToData(session);
+await db.sessions.update(session.id!, {
+  currentRoute: dbData.currentRoute,
+  currentState: dbData.currentState,
+  collectedData: dbData.collectedData,
+});
+```
+
+#### `sessionDataToState<TExtracted>(sessionId, data): SessionState<TExtracted>`
+
+Converts database data back to SessionState for resuming conversations.
+
+**Parameters:**
+
+- `sessionId`: The database session ID
+- `data`: Database session data (currentRoute, currentState, collectedData)
+
+**Example:**
+
+```typescript
+// Load from database
+const dbSession = await db.sessions.findById("session_123");
+
+// Restore session state
+const session = sessionDataToState<FlightData>(dbSession.id, {
+  currentRoute: dbSession.currentRoute,
+  currentState: dbSession.currentState,
+  collectedData: dbSession.collectedData,
+});
+
+// Resume conversation
+const response = await agent.respond({ history, session });
+```
+
+**Complete Persistence Example:**
+
+```typescript
+// CREATE: New session
+let session = createSession<FlightData>(dbSession.id);
+
+// CONVERSATION: Extract data
+const response1 = await agent.respond({ history: history1, session });
+session = response1.session!; // { extracted: { destination: "Paris" } }
+
+// SAVE: To database
+const saveData = sessionStateToData(session);
+await db.sessions.update(session.id!, saveData);
+
+// ---  Later (new request) ---
+
+// LOAD: From database
+const loaded = await db.sessions.findById("session_123");
+const restored = sessionDataToState<FlightData>(loaded.id, loaded);
+
+// CONTINUE: Conversation
+const response2 = await agent.respond({ history: history2, session: restored });
+```
+
+---
+
 ### `AgentStructuredResponse`
 
 The structured response format returned by AI providers when JSON mode is enabled.
@@ -1479,17 +1716,6 @@ const stateId = generateStateId("route_123", "Ask for name");
 // Returns: "state_ask_for_name_{hash}"
 ```
 
-#### `generateObservationId(description: string): string`
-
-Generates a deterministic observation ID from a description.
-
-```typescript
-import { generateObservationId } from "@falai/agent";
-
-const obsId = generateObservationId("User intent is unclear");
-// Returns: "observation_user_intent_is_unclear_{hash}"
-```
-
 #### `generateToolId(name: string): string`
 
 Generates a deterministic tool ID from a name.
@@ -1552,7 +1778,6 @@ enum BuiltInSection {
   CONTEXT = "context",
   GUIDELINES = "guidelines",
   CAPABILITIES = "capabilities",
-  OBSERVATIONS = "observations",
   ACTIVE_ROUTES = "active_routes",
 }
 ```