@ai.ntellect/core 0.3.3 → 0.4.1

package/README.md

@@ -1,362 +1,365 @@
 # AI.ntellect Core Framework
 
-## Table of contents
+## Overview
 
-1. [Main components](#main-components)
-   - [Agent](#agent)
+This framework is designed to execute complex workflows using advanced orchestration, memory management, and actionable intelligence. It integrates tools, interpreters, and memory systems to:
+
+- Analyze user inputs in their context.
+- Execute predefined workflows and dynamic actions.
+- Efficiently manage short-term and long-term memory.
+- Enable seamless integration with external APIs and tools.
+
+---
+
+## Table of Contents
+
+1. [Architecture Components](#architecture-components)
+   - [Agent Runtime](#agent-runtime)
    - [Orchestrator](#orchestrator)
-   - [Evaluator](#evaluator)
+   - [Queue Manager](#queue-manager)
    - [Interpreter](#interpreter)
-   - [Memory](#memory-architecture)
-2. [Action creation and management](#action-creation-and-management)
-3. [Agent processing](#agent-processing)
-4. [WIP (Work in Progress)](#wip-work-in-progress)
+   - [Memory System](#memory-system)
+   - [Listeners](#listeners)
+   - [Schedulers](#schedulers)
+2. [Defining and Executing Actions](#defining-and-executing-actions)
+3. [State Management and Recursion](#state-management-and-recursion)
+4. [Installation and Configuration](#installation-and-configuration)
+5. [Usage Example](#usage-example)
+6. [Work in Progress (WIP)](#work-in-progress-wip)
 
 ---
 
-## 1. Main components
+## Architecture Components
 
-The system relies on several key components that ensure smooth and efficient processing of user requests through an AI agent architecture.
+### Agent Runtime
 
-### Agent
+The `AgentRuntime` is the main engine coordinating the overall workflow. It connects all components and ensures tasks are executed efficiently.
 
-The agent is the core component that processes user requests and manages the entire interaction flow. It coordinates with other components to understand user needs, execute appropriate actions, and generate relevant responses.
+**Responsibilities:**
 
-- **Main role**: Process user requests and coordinate system components
-- **Key features**:
-  - Processes user prompts
-  - Manages conversation context
-  - Coordinates with orchestrator for action execution
-  - Handles response generation
-  - Maintains user state and memory
+- Build context for the current state using memory systems (RAG and CAG).
+- Orchestrate actions using the queue manager.
+- Leverage interpreters to analyze results and generate responses.
 
-### Orchestrator
+#### Context Building
 
-The orchestrator works under the agent's direction to manage the execution of actions. It analyzes requirements based on the agent's interpretation of user needs and coordinates the execution of appropriate tools.
+The `buildContext` method creates a complete context by:
 
-- **Main role**: Organize and direct the execution of actions
-- **Interactions**:
-  - Manages available tools/actions
-  - Executes actions based on agent requests
-  - Uses memory for context and caching
-  - Coordinates with evaluator for result assessment
+1. Adding tools and user requests.
+2. Retrieving recent actions via cache memory (CAG).
+3. Fetching relevant knowledge from persistent memory (RAG).
+4. Including available interpreters for the request.
 
-### Evaluator
+#### Workflow Processing
 
-The evaluator now collaborates with Interpreters to process action results and determine the next steps in the workflow.
+The `process` method:
 
-- **Main role**: Evaluate action results and coordinate with appropriate Interpreters
-- **Main functions**:
-  - Analyzes results from executed actions
-  - Determines if additional actions are needed
-  - Routes results to appropriate Interpreter
-  - Ensures completion of user requirements
-- **Interactions**:
-  - Works with orchestrator to manage workflow
-  - Coordinates with Interpreters for specialized processing
-  - Can trigger additional action cycles if needed
+1. Generates responses based on context using a language model.
+2. Handles recursive workflows for action execution.
+3. Selects appropriate interpreters to analyze results.
 
-### Interpreter
+---
 
-The Interpreter is a specialized component responsible for interpreting action results and generating appropriate responses. Each Interpreter is configured with a specific business context and produces responses in a format adapted to its domain of expertise.
+### Orchestrator
 
-**Key characteristics**:
+The **orchestrator** directs workflows by analyzing user inputs and planning actions. It interacts with tools, memory systems, and interpreters to ensure logical execution.
 
-- Business-specific context configuration
-- Domain-adapted response formatting
-- Specialized data processing based on context
+**Key Features:**
 
-**Operation flow**:
+- Dynamic action selection based on context.
+- Memory interaction management for RAG and CAG operations.
+- Multi-step workflow management with iterative refinement.
 
-- Receives action results via the Evaluator
-- Analyzes data according to its specific context
-- Produces formatted responses following domain rules
+---
 
-Examples of Interpreter implementations:
+### Queue Manager
 
-1. **MarketInterpreter**
+The **queue manager** is responsible for organizing and executing actions in the correct order, whether sequential or parallel. It acts as the central mechanism for managing workflows, ensuring each action is properly queued, validated, and executed.
 
-   - Specialized in market data analysis
-   - Format adapted for financial analysis
+**Main Responsibilities:**
 
-2. **SecurityInterpreter**
+1. **Action Queueing:**
 
-   - Dedicated to security verifications
-   - Optimized format for security reports
+   - Actions are added to a queue for execution, individually or in batches.
+   - Includes logging support for debugging and traceability.
 
-3. **GeneralInterpreter**
-   - Handles general purpose requests
-   - Flexible formatting based on context
+2. **Action Processing:**
 
-These examples demonstrate the system's flexibility, which can be extended with additional Interpreter types as needed.
+   - Executes actions while maintaining correct order.
+   - Respects dependencies between actions.
+   - Handles errors or confirmations via callbacks.
 
-[![Sans-titre-2024-11-08-0220.png](https://i.postimg.cc/nryjsx5y/Sans-titre-2024-11-08-0220.png)](https://postimg.cc/rR9FbBqj)
+3. **Confirmation Management:**
+   - Supports prompts for critical actions.
+   - Relies on callbacks to decide whether to proceed with specific actions.
 
-### Memory
+**Example:**
 
-The system implements a sophisticated memory architecture that combines different storage solutions for various types of memory:
+```typescript
+import { ActionQueueManager } from "@ai-ntellect/core";
+import { actions, callbacks } from "@ai-ntellect/core/examples";
 
-#### Installation and setup
+const queueManager = new ActionQueueManager(actions, callbacks);
+queueManager.addToQueue([{ name: "fetch-data", parameters: [...] }]);
+const results = await queueManager.processQueue();
+console.log("Results:", results);
+```
 
-##### Meilisearch (Long-term memory)
+---
 
-Meilisearch can be self-hosted for complete control over the agent's long-term memory:
+### Interpreter
 
-```bash
-# Install Meilisearch
-curl -L https://install.meilisearch.com | sh
+The **interpreter** specializes in analyzing results and generating domain-specific insights. Each interpreter is tailored for a specific use case and uses its own character configuration.
 
-# Launch Meilisearch with a master key
-./meilisearch --master-key="YOUR_MASTER_KEY"
-```
+**Examples:**
 
-##### Redis (Short-term memory)
+1. **MarketInterpreter**: Analyzes financial market data.
+2. **SecurityInterpreter**: Performs security checks.
+3. **GeneralInterpreter**: Processes general-purpose requests.
 
-Redis handles the short-term memory components:
+#### Interpretation Workflow
 
-```bash
-# Using Docker
-docker run --name redis -d -p 6379:6379 redis
+1. Builds context with the current state, including results and user requests.
+2. Uses the language model to generate actionable insights.
+3. Provides detailed responses for the end user.
 
-# Or install locally
-sudo apt-get install redis-server
-```
+---
 
-2. **Configuration**:
-   - Default port: 6379
-   - Configure memory limits
-   - Enable persistence if needed
+### Memory System
 
-#### Memory types
+The memory architecture combines short-term and long-term memory to provide contextual processing.
 
-#### Short-term memory (Redis)
+#### Memory Types
 
-1. **Procedural Memory**:
+1. **Cache Memory (Redis):**
+   - Stores temporary data for quick access.
+   - Examples: Recent actions, session data.
+2. **Persistent Memory (Meilisearch):**
+   - Stores long-term data such as historical interactions and knowledge.
+   - Supports semantic searches and vector-based retrievals.
 
-   - Stored in Redis for fast access
-   - Contains reusable action sequences and workflows
-   - Optimizes performance through caching
-   - Example: "Common token approval + swap sequence"
+---
 
-2. **Short-term episodic memory**:
-   - Recent messages and interactions
-   - Temporary context for ongoing conversations
-   - Stored in Redis for quick retrieval
-   - Example: "Last 10 messages in current conversation"
+### Listeners
 
-#### Long-term memory (Meilisearch)
+**Listeners** connect to external events via WebSocket. They listen for real-time updates and trigger specific actions or callbacks in response to events.
 
-1. **Semantic memory**:
+**Key Features:**
 
-   - Permanent storage of facts and knowledge
-   - Indexed for efficient retrieval
-   - Stores relationships between concepts
-   - Example: "Token X has contract address Y on network Z"
+- Connect to WebSockets to listen for events.
+- Manage subscriptions with custom messages.
+- Trigger callbacks to process received data.
 
-2. **Long-term episodic Memory**:
-   - Historical interactions and experiences
-   - Persistent context across sessions
-   - Searchable through vector similarity
-   - Example: "User X's past successful transactions"
+**Usage Example:**
 
-### Cache Augmented Generation (CAG)
+```typescript
+agent.addListener(
+  "listener-id",
+  "wss://example.com/socket",
+  () => JSON.stringify({ action: "subscribe" }),
+  async (data) => {
+    console.log("Received data:", data);
+  }
+);
+```
+
+---
 
-CAG optimizes workflow execution through Redis-based caching:
+### Schedulers
 
-- **Main role**: Cache frequently used procedural patterns
-- **Implementation**:
+**Schedulers** allow tasks or actions to be scheduled for later execution. They use cron expressions to define scheduling intervals.
 
-  - Uses Redis for high-performance storage
-  - Stores action sequences and their results
-  - Enables quick retrieval of common patterns
+**Key Features:**
 
-- **Benefits**:
-  - Reduces computation overhead
-  - Speeds up repeated operations
-  - Optimizes resource usage
+- Cron-based scheduling.
+- Support for recurring and non-recurring tasks.
+- Management and cancellation of scheduled tasks.
 
-### Retrieval Augmented Generation (RAG)
+**Usage Example:**
 
-The RAG system enhances long-term memory access through Meilisearch:
+```typescript
+const scheduler = new TaskScheduler(agentRuntime, redisCache);
 
-- **Implementation**:
+const taskId = await scheduler.scheduleRequest({
+  originalRequest: "Market analysis",
+  cronExpression: "0 9 * * *", // Every day at 9 AM
+});
 
-  - Vector-based search for semantic similarity
-  - Dual indexing (global and user-specific)
-  - Combines with traditional text search
+console.log(`Task scheduled with ID: ${taskId}`);
 
-- **Features**:
-  - Semantic and episodic memory retrieval
-  - Context-aware search capabilities
-  - Relevance-based result ranking
+// Cancel the task if needed
+scheduler.cancelScheduledRequest(taskId);
+```
 
 ---
 
-## 2. Action creation and management
+## Defining and Executing Actions
 
-Actions are specific tasks to be performed within a workflow. They can involve interactions with APIs, blockchain transactions, or any other necessary operations.
+### What is an Action?
 
-Each action is defined as an object containing a name, description, parameters, and an execution function.
+Actions are the fundamental tasks executed by the framework. Each action includes:
 
-### Example of an action
+- A unique name and description.
+- Input parameters validated using schemas.
+- Execution logic encapsulated in the `execute` method.
+
+### Action Example
 
 ```typescript
-import { networkConfigs } from "@config/network";
-import { parseEther } from "ethers";
 import { z } from "zod";
+import { parseEther } from "ethers";
 
 export const prepareTransaction = {
   name: "prepare-transaction",
-  description: "Prepare a transfer for the user to sign.",
+  description: "Prepares a token transfer for user approval.",
   parameters: z.object({
     walletAddress: z.string(),
-    amount: z.string().describe("Amount to send"),
-    networkId: z.string().describe("Target network (e.g., ethereum, arbitrum)"),
+    amount: z.string(),
+    networkId: z.string(),
   }),
-  execute: async ({
-    walletAddress,
-    amount,
-    network,
-  }: {
-    walletAddress: string;
-    amount: string;
-    networkId: string;
-  }) => {
-    try {
-      const networkConfig = networkConfigs[networkId];
-      if (!networkConfig) {
-        throw new Error(`Network ${network} not found`);
-      }
-
-      return {
-        to: walletAddress,
-        value: parseEther(amount).toString(),
-        chain: {
-          id: networkConfig.id,
-          rpc: networkConfig.rpc,
-        },
-        type: "transfer",
-      };
-    } catch (error) {
-      return "Error preparing the transaction";
-    }
+  execute: async ({ walletAddress, amount, networkId }) => {
+    return {
+      to: walletAddress,
+      value: parseEther(amount).toString(),
+      network: networkId,
+    };
   },
 };
 ```
 
-### How to define an action:
-
-1. **Name**: Unique identifier for the action.
-2. **Description**: Brief description of what the action does.
-3. **Parameters**: Parameters required to execute the action, validated by `zod`.
-4. **Execution**: `execute` function that performs the action.
-
 ---
 
-## 3. Agent processing
+## State Management and Recursion
 
-The agent handles the entire process of understanding user requests and coordinating responses. Here's an example of how to use the agent:
+The agent manages state and recursive workflows to ensure actions are executed in an orderly manner and to completion, while adhering to a maximum number of iterations to avoid infinite loops.
 
-```typescript
-const memory = new PersistentMemory({
-  host: "http://localhost:7700",
-  apiKey: "YOUR_API_KEY",
-});
+### State Management
 
-const orchestrator = new Orchestrator(
-  [
-    getChainsTVL,
-    getRssNews,
-    // other tools...
-  ],
-  memory
-);
+The state (`State`) includes:
 
-const agent = new Agent({
-  user: { id: "user_id" },
-  orchestrator,
-  persistentMemory: memory,
-  stream: false,
-  maxEvaluatorIteration: 1,
-});
+- `currentContext`: The current context of the user request.
+- `previousActions`: A list of previously executed actions.
 
-// Process a user request
-const result = await agent.process(prompt, context, {
-  onMessage: (message) => {
-    console.log({ message });
-  },
-});
-```
+When an action is completed, the state is updated to include:
 
-### Agent process flow:
+- Results from previous actions.
+- Remaining context to be processed.
 
-1. User sends a prompt
-2. Agent analyzes the prompt and context
-3. Orchestrator executes required tools/actions
-4. Evaluator assesses results
-5. Agent generates final response
+### Controlled Recursion
 
-## 4. WIP (Work in Progress)
+To prevent infinite loops, the system limits the number of iterations via the `maxIterations` configuration.
 
-Here are the elements currently in development or improvement:
+**Workflow:**
 
----
+1. **Initialization:** At each iteration, the agent:
 
-## Multi-agent collaboration
+   - Executes actions in the queue.
+   - Updates the state with new results.
 
-**Objective**: Enable multiple agents to collaborate on complex tasks with specialization and coordination.
+2. **Limit Validation:**
 
-**Interest**: Collaboration between agents allows breaking down complex tasks into specialized subtasks, enhancing the
-efficiency and quality of results. It also enables better resource management and faster adaptation to changes.
+   - If the number of iterations exceeds `maxIterations`, processing is stopped with a "Max iterations reached" message.
 
-**Steps to implement**:
+3. **Recursion:**
+   - If actions remain to be executed, the agent recursively calls the `process` method with the updated state.
 
-- [ ] Task delegation framework.
-- [ ] Shared context management.
-- [ ] Conflict resolution protocols.
+**State and Recursion Example:**
 
-**Status**: Research phase, architectural planning in progress.
+```typescript
+const updatedNextState: State = {
+  ...state,
+  currentContext: state.currentContext,
+  previousActions: [...(state.previousActions || []), ...(results || [])],
+};
+
+if (countIterations < this.config.maxIterations) {
+  return this.process(updatedNextState);
+} else {
+  console.log("Max iterations reached");
+  response.shouldContinue = false;
+}
+```
 
 ---
 
-## Complex on-chain interactions management
+## Installation and Configuration
 
-**Objective**: Create a model for recognizing on-chain interactions and creating workflows for complex interactions.
+### Install Dependencies
 
-**Interest**: This feature allows the agent to understand and interact with smart contracts more intuitively,
-facilitating the execution of complex actions on the blockchain. It improves accessibility and efficiency in
-interacting with smart contracts.
+```bash
+npm install
+```
+
+### Configure External Services
 
-**Steps to implement**:
+#### Redis (Cache Memory)
+
+```bash
+docker run --name redis -d -p 6379:6379 redis
+```
 
-- [ ] Extraction and processing of relevant contract ABIs.
-- [ ] Filtering of relevant functions.
-- [ ] Generation of hypothetical queries in natural language.
-- [ ] Conversion of queries into vector embeddings.
-- [ ] Storing embeddings and associated queries.
-- [ ] Similarity search based on cosine.
-- [ ] Ranking results based on relevance.
+#### Meilisearch (Persistent Memory)
 
-**Status**: Ongoing study to determine the best approach and technologies to use.
+```bash
+curl -L https://install.meilisearch.com | sh
+./meilisearch --master-key="YOUR_MASTER_KEY"
+```
 
 ---
 
-## Lit Protocol implementation
+## Usage Example
+
+### Initialize the Agent
+
+```typescript
+import { deepseek } from "@ai-ntellect/core";
+import { Agent } from "@ai-ntellect/core";
+import { checkHoneypot, fetchMarkPrice } from "@ai-ntellect/core/actions";
+import {
+  generalInterpreterCharacter,
+  marketInterpreterCharacter,
+  securityInterpreterCharacter,
+} from "@ai-ntellect/core/interpreter/context";
 
-**Objective**: Add the ability to execute Lit actions, enabling decentralized and secure calculations on the Lit
-network.
+const model = deepseek("deepseek-reasoner");
 
-**Interest**: Integrating the Lit Protocol allows executing Lit actions in a decentralized manner, using cryptographic
-keys to validate operations. These actions can be used to run JavaScript scripts in a decentralized environment,
-offering transparency as all interactions are recorded on the blockchain. The main benefit lies in automation and
-security while preserving user privacy, which enhances trust in on-chain interactions.
+const agent = new Agent({
+  orchestrator: {
+    model,
+    tools: [checkHoneypot, fetchMarkPrice],
+  },
+  interpreters: [
+    new Interpreter({
+      name: "security",
+      model,
+      character: securityInterpreterCharacter,
+    }),
+    new Interpreter({
+      name: "market",
+      model,
+      character: marketInterpreterCharacter,
+    }),
+    new Interpreter({
+      name: "general",
+      model,
+      character: generalInterpreterCharacter,
+    }),
+  ],
+  memoryManager: {
+    model,
+  },
+  maxIterations: 3,
+});
+```
 
-**Steps to Implement**:
+### Process a Request
 
-- [x] Study Lit Protocol documentation, especially the section on Lit actions and their implementation.
-- [ ] Integrate the protocol into the existing architecture to allow execution of Lit actions.
-- [ ] Develop modules for executing Lit actions, including signature management and secure script execution.
-- [ ] Test the integration, security, and transparency of Lit actions to ensure they function properly.
+```typescript
+const state = {
+  currentContext: "Analyze XRP/USD market trends",
+  previousActions: [],
+};
 
-**Status**: Under study to determine feasibility and technical implications, particularly regarding integrating
-decentralization into the existing system.
+const result = await agent.process(state);
+console.log("Result:", result);
+```