agentic-api 2.0.26 → 2.0.31

package/README.md

@@ -1,33 +1,59 @@
 # @agentic-api
 
-Super simple API for intelligent agent orchestration with automatic sequences and escalations. (inspiré du projet [OpenAI Swarm](https://github.com/openai/openai-realtime-agents))
+Comprehensive framework for intelligent agent orchestration, document processing, and enterprise workflow management. (inspired from [OpenAI Swarm](https://github.com/openai/openai-realtime-agents)) (01/2025)
 
-> **Personal Note**:  
-> This project is not meant to be better than Vercel or LangChain. It's simply less generic and optimized for a specific set of problems.
+> **Design Philosophy**:  
+> This project is not meant to be better than Vercel or LangChain. It's simply less generic and optimized for a specific set of enterprise problems.
 > It focuses on specific features that required too many dependencies with other frameworks.
 
+## 🚀 Core Capabilities
 
-## 🚀 Key Features
+### 🤖 Agent Orchestration
+- Multi-agent conversations with automatic transfers
+- State machine-driven workflows for complex processes
+- Confidence-based escalation between specialized agents
+- **StateGraph Architecture**: Modern conversation state management with automatic persistence
 
-- Agent orchestration in predefined sequences
-- Automatic model escalation to more intelligent instructions and models. 
-- Smart transfer between specialized agents with confidence threshold
-- Prompting models to follow a state machine, for example to accurately collect things like names and phone numbers.
-- **✅ StateGraph Architecture**: Modern conversation state management with automatic persistence
+### 📄 Document Processing
+- **MapLLM**: Advanced map-reduce pattern for large document analysis
+- **Structured Outputs**: OpenAI JSON schema validation support
+- **Flexible Loaders**: File and string content processing with chunking strategies
+- **Callback-Driven**: User-defined logic for accumulation and flow control
 
+### 🧪 Testing & Validation
+- **Agent Simulator**: Comprehensive testing framework with scenario-based simulations
+- **Realistic Personas**: Authentic user behavior simulation for thorough testing
+- **Automatic Validation**: Built-in success/failure detection and reporting
 
-### Key Advantages
+### 📋 Enterprise Workflow
+- **Rules Management**: Git-based workflow for business rules and procedures
+- **Pull Request Validation**: Structured review and approval processes
+- **Multi-File Operations**: Atomic operations across related documents
+- **Search Integration**: Full-text search with RAG embeddings
 
-1. **Simplicity**
-   - Minimal configuration
-   - Few dependencies
-   - Fast
-   - **Modern StateGraph**: Clean separation of concerns with automatic session management
+## 🎯 Key Advantages
+
+1. **Enterprise-Ready**
+   - Production-grade document workflows
+   - Comprehensive testing capabilities
+   - Git-based version control integration
+
+2. **Developer-Friendly**
+   - Minimal configuration required
+   - TypeScript-first with full type safety
+   - Modular architecture for easy extension
+
+3. **Performance-Optimized**
+   - Efficient chunking strategies
+   - Intelligent caching systems
+   - Parallel processing support
 
 ### Recommended Use Cases
 
-- Applications requiring specialized agent orchestration
-- Projects needing reliable content extraction
+- **Enterprise Document Management**: Rules, procedures, and knowledge base management
+- **Agent Development & Testing**: Comprehensive agent behavior validation
+- **Large Document Processing**: Analysis and synthesis of complex documents
+- **Conversational AI Systems**: Multi-agent workflows with state management
 
 
 [![](https://mermaid.ink/img/pako:eNpVkcluwjAQhl_FGgmJSoDIQoAcKhEQ7aGXNkiVmqDKTYYkUmJHxu5GeZaq575cH6EmC8sc7PnG_4xn7B1EPEZwYZPztyilQpLVImREmy81dYNqW1_VsVlwIxAlinXNi24Qwt_37w-5VQVlIbTCet2ql0TQMiU-itcswudZgkxuSdAwqbkpdrA4ExjJjDNy93CKekbg0xzPhZ4ZzNVW8gIF8VVZct3k2akV3CsujxnI4pDV7jx4XC3XLVXTkX7_msyaCSvwjAsyL8hqkzsdsuQi0h3QPEsYFnoYknKRfXImaV6LPIP0B2dFPLNhq2Gr5kVbtUn4WtIsVwK_yLxp_HD7KXrUQw_0IxQ0i_U37g6xEGSqmwnB1W6MG6pyGULI9lpKleT-B4vAlUJhDwRXSQruhuZbTaqMqcRFRvW3Fa2kpOyJ8yMm4nBTk60fFsWcKybBHVVScHfwrmFiDgzTGJvToWFY2nrwAe54OBg5pu1Yjj21Dcd29j34rGoPB5OxPT23_T90g8Qf?type=png)](https://mermaid.live/edit#pako:eNpVkcluwjAQhl_FGgmJSoDIQoAcKhEQ7aGXNkiVmqDKTYYkUmJHxu5GeZaq575cH6EmC8sc7PnG_4xn7B1EPEZwYZPztyilQpLVImREmy81dYNqW1_VsVlwIxAlinXNi24Qwt_37w-5VQVlIbTCet2ql0TQMiU-itcswudZgkxuSdAwqbkpdrA4ExjJjDNy93CKekbg0xzPhZ4ZzNVW8gIF8VVZct3k2akV3CsujxnI4pDV7jx4XC3XLVXTkX7_msyaCSvwjAsyL8hqkzsdsuQi0h3QPEsYFnoYknKRfXImaV6LPIP0B2dFPLNhq2Gr5kVbtUn4WtIsVwK_yLxp_HD7KXrUQw_0IxQ0i_U37g6xEGSqmwnB1W6MG6pyGULI9lpKleT-B4vAlUJhDwRXSQruhuZbTaqMqcRFRvW3Fa2kpOyJ8yMm4nBTk60fFsWcKybBHVVScHfwrmFiDgzTGJvToWFY2nrwAe54OBg5pu1Yjj21Dcd29j34rGoPB5OxPT23_T90g8Qf)
@@ -42,9 +68,9 @@ npm install @agentic-api
 
 ```typescript
 import OpenAI from "openai";
-import { executeAgentSet } from '@agentic-api/execute';
-import { AgenticContext } from '@agentic-api/types';
-import { AgentStateGraph } from '@agentic-api/stategraph';
+import { executeAgentSet } from '@agentic-api';
+import { AgenticContext } from '@agentic-api';
+import { AgentStateGraph } from '@agentic-api';
 
 const openai = new OpenAI({
   apiKey: process.env.OPENAI_API_KEY,
@@ -74,8 +100,7 @@ const stream = await executeAgentSet(agents, context, {
 ## 🤖 Custom Agent Creation
 
 ```typescript
-import { AgentConfig } from '@agentic-api/types';
-import { modelConfig } from '@agentic-api/execute';
+import { AgentConfig, modelConfig } from '@agentic-api';
 
 // Example specialized agent with thinking tool
 const haiku: AgentConfig = {
@@ -98,7 +123,7 @@ const welcome: AgentConfig = {
 };
 
 // Inject transfer and thinking tools
-import { injectTransferTools } from '@agentic-api/utils';
+import { injectTransferTools } from '@agentic-api';
 const myAgents = injectTransferTools([welcome, haiku]);
 ```
 
@@ -107,7 +132,7 @@ const myAgents = injectTransferTools([welcome, haiku]);
 The new StateGraph architecture provides automatic conversation state management:
 
 ```typescript
-import { AgentStateGraph, sessionStateGraphGet, sessionStateGraphSet } from '@agentic-api/stategraph';
+import { AgentStateGraph, sessionStateGraphGet, sessionStateGraphSet } from '@agentic-api';
 
 // StateGraph is automatically managed during executeAgentSet
 // But you can also work with it directly:
@@ -234,7 +259,7 @@ stateGraph.clearDiscussion("agentName");
 ### **Utility Functions**
 ```typescript
 // Get specialized (starting) agent for discussion
-import { getSpecializedAgent } from '@agentic-api/stategraph';
+import { getSpecializedAgent } from '@agentic-api';
 const specializedAgent = getSpecializedAgent(discussion);
 
 // Find discussion by ID
@@ -247,49 +272,188 @@ stateGraph.renameDiscussion("agentName", "New Name", "Description");
 stateGraph.deleteDiscussion("agentName");
 ```
 
-## 💾 Pull-based Data Digestion
+## 🗂️ MapLLM Document Processing
+
+Modern map-reduce pattern for processing large documents with flexible content handling and OpenAI structured outputs.
+
+- **Flexible Loaders**: `FileNativeLoader` and `StringNativeLoader` with configurable chunking strategies
+- **Callback-Driven**: User-defined logic for accumulation, termination, and flow control
+- **Structured Outputs**: Optional JSON schema validation via OpenAI structured outputs
+- **Robust EOF Handling**: Proper chunk boundary detection and clean termination
+
+📖 **[Complete MapLLM Documentation →](./docs/README-AGENT-REDUCE.md)**
+
+```typescript
+import { MapLLM, FileNativeLoader, modelConfig } from '@agentic-api';
+
+// Basic text processing
+const config = {
+  digestPrompt: "Analyze this chunk for key information.",
+  reducePrompt: "Merge analysis with previous results."
+};
+
+const loader = new FileNativeLoader('document.pdf', { type: 'pages', size: 1 });
+const mapper = new MapLLM(loader);
+
+const callback = (result, currentValue) => {
+  result.acc = result.acc + `\n${currentValue}\n`;
+  
+  // Stop conditions
+  if (result.acc.length > 5000) {
+    result.continue = true;
+  }
+  if (result.metadata.iterations > 20) {
+    result.maxIterations = true;
+  }
+  
+  return result;
+};
+
+const init = {
+  acc: "",
+  model: modelConfig("LOW-fast"),
+  verbose: true
+};
+
+const result = await mapper.reduce(config, callback, init);
+console.log('Final result:', result.acc);
+
+// Structured output processing with JSON schema
+const myJsonSchema = {
+  type: "object",
+  properties: {
+    summary: { type: "string" },
+    key_points: { type: "array", items: { type: "string" } }
+  },
+  required: ["summary", "key_points"]
+};
+
+const structuredCallback = (result, currentValue) => {
+  // Set structured output format
+  result.format = {
+    name: "analysis",
+    schema: myJsonSchema,
+    strict: true
+  };
+  
+  // For structured output, accumulate as object or merged object
+  result.acc = { ...result.acc, ...currentValue };
+  
+  if (result.metadata.iterations > 5) {
+    result.maxIterations = true;
+  }
+  
+  return result;
+};
+
+const structuredResult = await mapper.reduce(config, structuredCallback, { 
+  acc: {}, 
+  verbose: true 
+});
+// structuredResult.acc is validated JSON matching the schema
+```
+
+## 🤖 Agent Simulator
 
-This feature enables agents to process large documents chunk by chunk. It's designed for scenarios where the entire document cannot fit into the agent's context window.
+Advanced testing framework for agent behavior validation with scenario-based simulations.
 
-- **Chunked Processing**: The `pullContentDigestor` tool allows an agent to request and process content in manageable chunks.
-- **Stateful Digestion**: The agent receives the current chunk and relevant instructions, including the previous processing results, to maintain context throughout the digestion of the entire document.
-- **Flexible Templates**: Different processing templates (e.g., "facts", "compress", "semantic", "minutes") can be applied to each chunk, guiding the agent on how to digest the information.
-- **EOF Handling**: The tool signals the end of the document with an "EOF" message, allowing the agent to finalize its processing.
+- **Scenario-Based Testing**: Define complex test scenarios with goals, personas, and expected outcomes
+- **Conversational Simulation**: Simulate realistic user interactions with agents
+- **Automatic Validation**: Built-in success/failure detection and error reporting
+- **Exchange Limiting**: Control simulation length with configurable exchange limits
+
+📖 **[Complete Agent Simulator Documentation →](./docs/README-AGENT-SIMULATOR.md)**
 
 ```typescript
-// Example of how an agent might use the pullContentDigestor tool
-
-// Agent receives a call to process a large document
-// Initial call to the tool
-const initialChunkData = await pullContentDigestor({
-  path: "path/to/large/document.pdf",
-  template: "facts", // or any other relevant template
-  position: 0 // Initial position
+import { AgentSimulator, SimulationScenario } from '@agentic-api';
+
+// Define test scenario
+const scenario: SimulationScenario = {
+  testGoals: "Verify that the agent can help with haiku creation",
+  testEnd: "Agent provides a complete haiku poem",
+  testPersona: "A poetry enthusiast seeking creative assistance",
+  testQuery: "I want to write a haiku about nature. Can you help me?",
+  testResult: "Agent successfully guides haiku creation process",
+  testError: "Agent refuses to help or provides incorrect format"
+};
+
+// Configure simulator
+const simulator = new AgentSimulator({
+  agents: [haikuAgent, welcomeAgent],
+  start: "welcome",
+  verbose: true,
+  instructionEx: "Focus on creative writing assistance"
 });
 
-let currentChunk = initialChunkData.content;
-let nextPosition = initialChunkData.nextPosition;
-let accumulatedResults = ""; // Agent will accumulate results here
-
-while (!currentChunk.includes("EOF")) {
-  // Process the currentChunk based on instructions and template
-  // The agent would use its LLM capabilities here, 
-  // using the provided instructions, previous results, and the new chunk.
-  const processingResult = await processMyChunk(currentChunk, initialChunkData.instructions, accumulatedResults);
-  accumulatedResults += processingResult; // Append to overall result
-
-  // Request the next chunk
-  const nextChunkData = await pullContentDigestor({
-    path: "path/to/large/document.pdf",
-    template: "facts",
-    position: nextPosition
-  });
-  currentChunk = nextChunkData.content;
-  nextPosition = nextChunkData.nextPosition;
+// Run simulation
+const result = await simulator.executeSimulation({
+  scenario,
+  maxExchanges: 10,
+  onMessage: (message) => {
+    console.log(`${message.role}: ${message.content}`);
+  }
+});
+
+// Validate results
+console.log('Success:', result.success);
+console.log('Summary:', result.message);
+console.log('Exchanges:', result.exchangeCount);
+
+if (!result.success) {
+  console.error('Error:', result.error);
 }
+```
+
+### Simulation Features
+
+- **Structured Scenarios**: Define test goals, end conditions, and expected behaviors
+- **Persona Simulation**: Simulator adopts specific user personas for realistic testing
+- **Error Detection**: Automatic detection of unwanted content or behaviors
+- **Exchange Tracking**: Monitor conversation flow and agent performance
+- **Execution Metadata**: Access to token usage, actions, and performance metrics
+
+## 📋 Rules Management System
+
+Enterprise-grade Git-based workflow for managing business rules, procedures, and documentation.
+
+- **Git-Based Workflow**: Complete version control with branch-based status management
+- **Pull Request Validation**: Structured validation process with reviewer assignment
+- **Multi-File Operations**: Atomic operations across multiple related documents
+- **Content Management**: Rich metadata support with front-matter parsing
+- **Search Integration**: Full-text search with RAG embeddings support
+
+📖 **[Complete Rules System Documentation →](./docs/README-RULES-SYSTEM.md)**
+
+```typescript
+import { RulesWorkflow, gitLoad, RuleStatus } from '@agentic-api';
+
+// Initialize workflow
+const config = gitLoad(); // Loads from environment variables
+const workflow = new RulesWorkflow(config);
+
+// Create a new rule
+const newRule = await workflow.createRule('procedures/finance/budget.md', {
+  title: 'Budget Validation Process',
+  slugs: ['budget-validation'],
+  tags: ['finance', 'validation'],
+  role: 'rule'
+}, 'Initial budget validation procedure');
+
+// Create validation request (PR)
+const prBranch = await workflow.createPullRequest(
+  ['procedure-a.md', 'procedure-b.md'], 
+  'Update financial procedures'
+);
+
+// Search rules by content
+const searchResults = await workflow.searchRules('budget validation', {
+  branch: 'main',
+  tags: ['finance'],
+  limit: 10
+});
 
-// Final processing of accumulatedResults
-const finalDigestedDocument = finalizeProcessing(accumulatedResults);
+// Get rules by status
+const draftRules = await workflow.getRulesByStatus(RuleStatus.EDITING);
 ```
 
 ## 🧪 Testing

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agentic-api",
-  "version": "2.0.26",
+  "version": "2.0.31",
   "description": "API pour l'orchestration d'agents intelligents avec séquences et escalades automatiques",
   "main": "dist/src/index.js",
   "types": "dist/src/index.d.ts",