jsfe 0.2.0 → 0.4.0

package/README.md

@@ -58,8 +58,8 @@ interface ContextEntry {
   role: 'user' | 'assistant' | 'system' | 'tool';  // Message role type
   content: string | Record<string, unknown>;       // Message content (text, object, etc.)
   timestamp: number;                               // Unix timestamp in milliseconds
-  stepId?: string;                                 // Optional: Associated flow step ID
-  toolName?: string;                               // Optional: Tool name for tool messages
+  stepId?: string;                                 // Optional: Used by the system when a Step calls a Tool
+  toolName?: string;                               // Optional: Used by the system to record Tool result into the the chat context
   metadata?: Record<string, unknown>;              // Optional: Additional context data
 }
 ```
@@ -71,7 +71,6 @@ interface ContextEntry {
 const userEntry = {
   role: 'user',
   content: 'I need help with my account',
-  timestamp: Date.now()
 };
 // Process the message
 await engine.updateActivity(userEntry, sessionContext);
@@ -80,8 +79,6 @@ await engine.updateActivity(userEntry, sessionContext);
 const assistantEntry = {
   role: 'assistant',
   content: 'I can help you with your account. What specific issue are you experiencing?',
-  timestamp: Date.now(),
-  stepId: 'greeting-step'
 };
 // Process the message
 await engine.updateActivity(assistantEntry, sessionContext);
@@ -100,25 +97,38 @@ The Flow Engine implements a sophisticated "stack-of-stacks" architecture that a
    - Automatic stack switching for flow interruption/resumption
    - Proper isolation between different workflow contexts
 
-2. **Flow Frame Structure**
+2. **Flow Frame Structure (Runtime Execution Context)**
    Each flow execution maintains a complete context frame:
    ```typescript
-   {
-     flowName: string,          // Human-readable flow name
-     flowId: string,            // Unique flow identifier
-     flowVersion: string,       // Flow version for compatibility
-     flowStepsStack: [...],     // Remaining steps (reversed)
-     contextStack: [...],       // Complete interaction history
-     inputStack: [...],         // Current input context
-     variables: {},             // Unified variable storage
-     transaction: TransactionObj, // Audit and transaction tracking
-     userId: string,            // User identifier
-     startTime: number,         // Flow start timestamp
-     pendingVariable: string,   // Variable awaiting user input
-     lastSayMessage: string,    // Last SAY step output
-     pendingInterruption: {}    // Interruption state management
+   interface FlowFrame {
+     flowName: string;                              // Human-readable flow name (from FlowDefinition.name)
+     flowId: string;                                // Unique flow identifier (from FlowDefinition.id)
+     flowVersion: string;                           // Flow version for compatibility (from FlowDefinition.version)
+     flowStepsStack: FlowStep[];                    // Remaining steps (reversed for efficient pop)
+     contextStack: ContextEntry[];                  // Complete interaction history with role info
+     inputStack: unknown[];                         // Current input context for step execution
+     variables: Record<string, unknown>;            // Unified variable storage (shared across sub-flows)
+     transaction: TransactionObj;                   // Comprehensive transaction and audit tracking
+     userId: string;                                // User identifier for this flow session
+     startTime: number;                             // Flow start timestamp for timing analysis
+     pendingVariable?: string;                      // Variable name awaiting user input (SAY-GET steps)
+     lastSayMessage?: string;                       // Last SAY step output for context
+     pendingInterruption?: Record<string, unknown>; // Interruption state management
+     accumulatedMessages?: string[];                // Accumulated SAY messages for batching
+     parentTransaction?: string;                    // Parent transaction ID for sub-flow tracking
+     justResumed?: boolean;                         // Flag indicating flow was just resumed
    }
    ```
+   
+   **Technical Implementation Details:**
+   - **Flow Identity**: `flowName`, `flowId`, `flowVersion` are copied from the FlowDefinition
+   - **Dynamic Properties**: Flow `prompt`, `description`, and localized prompts are accessed dynamically from FlowDefinition
+   - **flowStepsStack**: Steps stored in reverse order for efficient pop operations
+   - **contextStack**: Enhanced with role information for complete conversation context
+   - **variables**: Flat storage shared across sub-flows for seamless data passing
+   - **accumulatedMessages**: SAY steps are batched for efficient output
+   - **justResumed**: Helps engine provide appropriate resumption messages
+   ```
 
 3. **Helper Function Architecture**
    All stack operations go through centralized helper functions:
@@ -127,6 +137,179 @@ The Flow Engine implements a sophisticated "stack-of-stacks" architecture that a
    - `pushToCurrentStack(engine, frame)`: Adds flow to active stack
    - `popFromCurrentStack(engine)`: Removes flow from active stack
 
+4. **Tool Definition Structure (Runtime Integration Context)**
+   Tools provide external capabilities with comprehensive security and validation:
+   ```typescript
+   interface ToolDefinition {
+     id: string;                                    // Unique tool identifier
+     name: string;                                  // Human-readable tool name
+     description: string;                           // Tool functionality description
+     
+     parameters?: {                                 // OpenAI Function Calling Standard schema
+       type: string;                                // Parameter type structure
+       properties?: Record<string, PropertySchema>; // Parameter validation rules
+       required?: string[];                         // Required parameter names
+       additionalProperties?: boolean;              // Additional parameter handling
+     };
+     
+     implementation?: {                             // Execution configuration
+       type: 'local' | 'http';                      // Implementation type
+       function?: string;                           // Local function name (APPROVED_FUNCTIONS)
+       url?: string;                                // HTTP endpoint with {param} placeholders
+       method?: HttpMethod;                         // HTTP method
+       pathParams?: string[];                       // URL parameter substitution
+       queryParams?: string[];                      // Query string parameters
+       responseMapping?: MappingConfig;             // Response transformation config
+       timeout?: number;                            // Request timeout
+       retries?: number;                            // Retry attempts
+     };
+     
+     security?: {                                   // Security controls
+       rateLimit?: {                                // Rate limiting configuration
+         requests: number;                          // Max requests per window
+         window: number;                            // Time window in milliseconds
+       };
+     };
+     apiKey?: string;                               // Authentication token
+     riskLevel?: 'low' | 'medium' | 'high';         // Security classification
+   }
+   ```
+   
+   **Technical Implementation Details:**
+   - **Parameter Validation**: JSON Schema validation with ajv for type safety
+   - **Local Function Execution**: Secure execution through APPROVED_FUNCTIONS registry
+   - **HTTP Integration**: Full REST API support with authentication and retries
+   - **Response Mapping**: Declarative transformation without code injection
+   - **Security Controls**: Rate limiting, risk classification, audit logging
+   - **Error Handling**: Automatic retry logic and graceful degradation
+
+5. **Response Mapping Configuration (MappingConfig Interface)**
+   The comprehensive response mapping system supports multiple transformation types:
+   ```typescript
+   export type MappingConfig =
+     | JsonPathMappingConfig
+     | ObjectMappingConfig  
+     | ArrayMappingConfig
+     | TemplateMappingConfig
+     | ConditionalMappingConfig
+     | PathConfig
+     | string
+     | Record<string, unknown>;
+
+   // JSONPath-based field extraction and transformation
+   export type JsonPathMappingConfig = {
+     type: 'jsonPath';
+     mappings: Record<string, {
+       path: string;                                   // JSONPath expression for data extraction
+       transform?: ValueTransformConfig;               // Optional value transformation
+       fallback?: unknown;                             // Fallback value if path not found
+     }>;
+     strict?: boolean;                                 // Strict mode validation
+   };
+
+   // Object structure mapping with nested support
+   export type ObjectMappingConfig = {
+     type: 'object';
+     mappings: Record<string, string | PathConfig | MappingConfig | object>;
+     strict?: boolean;                                 // Strict mode validation
+   };
+
+   // Array processing with filtering, sorting, and pagination
+   export type ArrayMappingConfig = {
+     type: 'array';
+     source?: string;                                  // Source array path
+     filter?: ConditionConfig;                         // Filtering conditions
+     itemMapping?: MappingConfig;                      // Per-item transformation
+     sort?: { field: string; order?: 'asc' | 'desc' }; // Sorting configuration
+     offset?: number;                                  // Pagination offset
+     limit?: number;                                   // Pagination limit  
+     fallback?: unknown[];                             // Fallback array if source not found
+   };
+
+   // Template-based string generation with variable substitution
+   export type TemplateMappingConfig = {
+     type: 'template';
+     template: string;                                 // Template string with {{variable}} placeholders
+     dataPath?: string;                                // Optional path to resolve template data from
+   };
+
+   // Conditional logic-based mapping selection
+   export type ConditionalMappingConfig = {
+     type: 'conditional';
+     conditions: Array<{
+       if: ConditionConfig;                            // Condition evaluation
+       then: MappingConfig;                            // Mapping to apply if condition true
+     }>;
+     else?: MappingConfig;                             // Default mapping if no conditions match
+   };
+
+   // Path-based value extraction with transformation
+   export type PathConfig = {
+     path: string;                                     // Data path for extraction
+     transform?: ValueTransformConfig;                 // Optional value transformation
+     fallback?: unknown;                               // Fallback value if path not found
+   };
+
+   // Comprehensive value transformation system with 25+ transform types
+   export interface ValueTransformConfig {
+     type: 'parseInt' | 'parseFloat' | 'toLowerCase' | 'toUpperCase' | 'trim' | 
+           'replace' | 'concat' | 'regex' | 'date' | 'default' | 'conditional' | 
+           'substring' | 'split' | 'join' | 'abs' | 'round' | 'floor' | 'ceil' | 
+           'template' | 'sum' | 'average' | 'count' | 'min' | 'max' | 'multiply' | 
+           'divide' | 'percentage' | 'add' | 'subtract' | 'currentYear' | 
+           'yearDifference' | 'handlebars' | 'custom';
+     
+     // Common transformation parameters
+     fallback?: unknown;                               // Default value for failed transforms
+     prefix?: string;                                  // String prefix for concat operations
+     suffix?: string;                                  // String suffix for concat operations
+     pattern?: string;                                 // Regex pattern for replace/match operations
+     replacement?: string;                             // Replacement string for regex operations
+     template?: string;                                // Template string for template transforms
+     value?: unknown;                                  // Static value for default transforms
+     
+     // Mathematical operation parameters
+     precision?: number;                               // Decimal precision for rounding operations
+     divisor?: number;                                 // Divisor for division/percentage operations
+     multiplier?: number;                              // Multiplier for multiplication operations
+     addend?: number;                                  // Value to add for addition operations
+     subtrahend?: number;                              // Value to subtract for subtraction operations
+     
+     // Array and aggregation parameters
+     field?: string;                                   // Field name for array aggregations
+     delimiter?: string;                               // Delimiter for join/split operations
+     index?: number;                                   // Array index for element selection
+     
+     // Conditional and date parameters
+     condition?: ConditionConfig;                      // Condition for conditional transforms
+     fromYear?: number;                                // Start year for year difference calculations
+     dataPath?: string;                                // Path for accessing context data
+   }
+
+   // Flexible condition evaluation system
+   export interface ConditionConfig {
+     field: string;                               // Field path for evaluation
+     operator: 'equals' | 'eq' | 'notEquals' | 'ne' | 'contains' | 'exists' | 
+               'notExists' | 'greaterThan' | 'gt' | 'lessThan' | 'lt' | 
+               'greaterThanOrEqual' | 'gte' | 'lessThanOrEqual' | 'lte' | 
+               'startsWith' | 'endsWith' | 'matches' | 'in' | 'hasLength' | 
+               'isArray' | 'isObject' | 'isString' | 'isNumber';
+     value?: unknown;                             // Comparison value for operators
+   }
+   ```
+   
+   **Response Mapping Technical Features:**
+   - **Type Safety**: Full TypeScript interface definitions with validation
+   - **Declarative Configuration**: No code injection - pure JSON configuration
+   - **Comprehensive Transforms**: 25+ built-in transformation types
+   - **Mathematical Operations**: Arithmetic, statistical, and precision control
+   - **Date Processing**: Dynamic date calculations and formatting
+   - **Template System**: Handlebars-style variable substitution with array iteration
+   - **Conditional Logic**: Complex branching and filtering capabilities
+   - **Array Processing**: Filtering, sorting, pagination, and aggregation
+   - **Path Resolution**: JSONPath support with fallback handling
+   - **Security**: Complete input sanitization and validation
+
 ## Features & Capabilities
 
 ### Flow Execution Modes

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "jsfe",
-  "version": "0.2.0",
+  "version": "0.4.0",
   "description": "TypeScript workflow engine for conversational/automation apps: multi-flow orchestration, interrupt/resume, stack-based control, robust errors, and flexible tool/REST integration.",
   "license": "MIT",
   "author": "Ron Pinkas",