@ai.ntellect/core 0.8.3 → 1.1.0

package/README.md

@@ -1,247 +1,461 @@
 # @ai.ntellect/core
 
-A TypeScript framework for building workflow automation with event-driven architecture.
+**Stop your AI agents from going off-rails in production.**
 
-## Features
+*LLMs generate. You control.*
 
-- Graph-based workflow execution
-- Event-driven node processing
-- TypeScript type safety
-- Zod schema validation
-- Retry mechanisms
-- State observation
-- AI Agent integration
+One LLM call $\rightarrow$ deterministic execution $\rightarrow$ no routing hallucinations.
+
+---
+
+## Why
+
+LLMs are great at generating text.  
+**They are bad at controlling systems.**
+
+Most frameworks let the LLM decide everything:
+- which tool to call
+- in what order
+- with what context
+
+$\rightarrow$ This breaks as complexity grows.
+
+**This framework does the opposite:**
+- LLM decides **intent (once)**
+- Your system handles **execution (deterministically)**
+
+No drift. No hidden loops. No surprises.
+
+**This is not an agent framework. It's a control layer for LLM systems.**
+
+---
+
+### The Contrast
+
+**Most frameworks:**
+`User` $\rightarrow$ `LLM` $\rightarrow$ `LLM` $\rightarrow$ `LLM` $\rightarrow$ `Tool` $\rightarrow$ `LLM` (Probabilistic Chaos)
+
+**This framework:**
+`User` $\rightarrow$ `LLM` $\rightarrow$ `System` $\rightarrow$ `Done` (Deterministic Control)
+
+---
+
+## Why this works
+
+We replaced probabilistic guessing with explicit definitions. Routing is **defined**, not **learned**.
+
+- **No prompt drift**: Your agent won't change its behavior because of a minor prompt tweak.
+- **No hidden loops**: Execution paths are explicit and finite.
+- **No probabilistic branching**: If the intent is X, the path is ALWAYS Y.
+
+Just state + transitions.
+
+---
+
+## Quick Reference
+
+```
+User → LLM (intent) → Petri Net (routing) → GraphFlow (execution)
+```
+
+One decision. Then control.
+
+---
 
 ## Installation
 
 ```sh
-npm install @ai.ntellect/core zod
+pnpm add @ai.ntellect/core zod
 ```
 
-## Basic Usage
+**Requirements**: TypeScript 5.x+, Node.js 18+.
 
-### Building a simple workflow
+---
 
-```typescript
-import { z } from "zod";
-import { GraphFlow } from "@ai.ntellect/core";
-import { GraphContext, GraphNodeConfig } from "@ai.ntellect/core/types";
+## Table of Contents
 
-// Define the schema
-const Schema = z.object({
-  message: z.string(),
-});
+- [Core Thesis](#core-thesis)
+- [Source Structure](#source-structure)
+- [GraphFlow — Typed Execution](#graphflow--typed-execution)
+- [CortexFlow — Intent Routing](#cortexflow--intent-routing)
+- [Agent — Cognitive Loop](#agent--cognitive-loop)
+- [AgentPipeline — Declarative Pipelines](#agentpipeline--declarative-pipelines)
+- [Persistence — Memory & Checkpoints](#persistence--memory--checkpoints)
+- [Planner & Compiler](#planner--compiler)
+- [CLI (Interactive REPL)](#cli-interactive-repl)
+- [Testing & Build](#testing--build)
 
-// Define a node
-const greetNode = {
-  name: "greet",
-  execute: async (context: GraphContext<typeof Schema>) => {
-    context.message = "Hello, World!";
-  },
-};
+---
 
-// Create workflow
-const workflow = new GraphFlow({
-  name: "hello",
-  schema: Schema,
-  context: { message: "" },
-  nodes: [greetNode],
-});
+## Core Thesis
+
+The framework architecture is the **Classifier-Controller Split**:
 
-// Execute and observe
-const main = async () => {
-  // Observe state changes
-  workflow
-    .observe()
-    .state()
-    .subscribe((context) => {
-      console.log(context);
-    });
+| Layer | Role | Technology | Correctness |
+|-------|------|-----------|-------------|
+| **Intent** | Classify user input | LLM | Probabilistic (confidence threshold) |
+| **Routing** | Choose execution path | Petri Net | Deterministic (verified) |
+| **Execution** | Run business logic | GraphFlow | Typed (Zod validated) |
 
-  // Execute workflow
-  await workflow.execute("greet");
-};
+This split means the LLM is called exactly **once per turn** for intent classification. All subsequent routing and execution is handled by verified, typed code.
+
+---
+
+## Source Structure
 
-main();
 ```
+@ai.ntellect/core
+├── execution/         # GraphFlow engine (typed nodes, events, checkpoints, planner, compiler)
+├── routing/           # PetriNet, CortexFlowOrchestrator, IntentClassifier
+├── agent/             # Agent, GenericExecutor, handlers, tools
+├── persistence/       # Barrel: Memory + checkpoint adapters
+├── pipeline/          # AgentPipeline (trigger → stages → gate)
+├── modules/           # Plugins: agenda, cli, embedding, memory, nlp
+├── interfaces/        # Contract interfaces
+├── types/             # Zod schemas
+└── index.ts           # Barrel: all public API exports
+```
+
+### Path Aliases
 
-### Handling events in a workflow
+- `@ai.ntellect/core` — root barrel (all major exports)
+- `@ai.ntellect/core/execution/...` — deep imports for GraphFlow internals
+- `@ai.ntellect/core/routing/...` — deep imports for PetriNet / CortexFlow
+- `@ai.ntellect/core/agent/...` — deep imports for Agent module
+- `@ai.ntellect/core/pipeline/...` — deep imports for AgentPipeline
+
+---
+
+## GraphFlow — Typed Execution
+
+A `GraphFlow` is a **typed state machine** where each node is a discrete unit of logic.
 
 ```typescript
 import { z } from "zod";
 import { GraphFlow } from "@ai.ntellect/core";
-import { GraphContext } from "@ai.ntellect/core/types";
 
-// Define schema
-const OrderSchema = z.object({
-  orderId: z.string(),
-  status: z.string(),
-  amount: z.number(),
-});
+const schema = z.object({ count: z.number().default(0) });
 
-// Define nodes
-const paymentNode: GraphNodeConfig<typeof OrderSchema> = {
-  name: "payment",
-  when: {
-    events: ["payment.received"],
-    timeout: 30000,
-    strategy: { type: "single" },
-  },
-  execute: async (context: GraphContext<typeof OrderSchema>) => {
-    context.status = "processing";
-  },
-  next: ["validation"],
-};
-
-const validationNode: GraphNodeConfig<typeof OrderSchema> = {
-  name: "validation",
-  when: {
-    events: ["payment.validated", "inventory.checked"],
-    timeout: 5000,
-    strategy: {
-      type: "correlate",
-      correlation: (events) => {
-        return events.every(
-          (e) => e.payload.orderId === events[0].payload.orderId
-        );
-      },
+const workflow = new GraphFlow({
+  name: "counter",
+  schema,
+  context: { count: 0 },
+  nodes: [
+    {
+      name: "increment",
+      execute: async (ctx) => { ctx.count++; },
+      next: "double",
     },
-  },
-  execute: async (context: GraphContext<typeof OrderSchema>) => {
-    context.status = "validated";
-  },
-};
-
-// Create workflow
-const orderWorkflow = new GraphFlow({
-  name: "order",
-  schema: OrderSchema,
-  context: {
-    orderId: "",
-    status: "pending",
-    amount: 0,
-  },
-  nodes: [paymentNode, validationNode],
+    {
+      name: "double",
+      execute: async (ctx) => { ctx.count *= 2; },
+    },
+  ],
 });
 
-// Usage
-const main = async () => {
-  // Observe state
-  orderWorkflow
-    .observe()
-    .property("status")
-    .subscribe((status) => {
-      console.log("Status:", status);
-    });
-
-  // Start listening for events
-  orderWorkflow.execute("payment");
-
-  // Emit event after a short delay
-  setTimeout(async () => {
-    await orderWorkflow.emit("payment.received", {
-      orderId: "123",
-      amount: 100,
-    });
-  }, 100);
-
-  // Observe payment received event
-  orderWorkflow
-    .observe()
-    .event("payment.received")
-    .subscribe((event) => {
-      console.log("Payment received:", event);
-      orderWorkflow.emit("inventory.checked", {
-        orderId: event.payload.orderId,
-      });
-    });
-
-  // Observe inventory checked event
-  orderWorkflow
-    .observe()
-    .event("inventory.checked")
-    .subscribe((event) => {
-      console.log("Inventory checked:", event);
-      orderWorkflow.emit("payment.validated", {
-        orderId: event.payload.orderId,
-      });
-    });
-
-  // Observe payment validated event
-  orderWorkflow
-    .observe()
-    .event("payment.validated")
-    .subscribe((event) => {
-      console.log("Payment validated:", event);
-    });
-};
-
-main();
-```
-
-### Creating a workflow with Agent
+await workflow.execute("increment");
+console.log(workflow.context.count); // 2
+```
+
+### Node Types
+
+| Type | Description |
+|------|-------------|
+| **Sequential** | `next: "nodeName"` or `next: (ctx) => condition ? "a" : "b"` |
+| **Parallel** | `parallel: { enabled: true, joinNode: "merge" }` — fork-join model |
+| **Send API** | Dynamic fan-out: create N parallel branches at runtime |
+| **Event-Driven** | `when: "event.name"` — pause and wait for external event |
+
+### Checkpoints
+
+Save/resume workflow state with time-travel debugging and human-in-the-loop breakpoints:
 
 ```typescript
-import { z } from "zod";
-import { GraphFlow, Agent } from "@ai.ntellect/core";
-import { GraphContext } from "@ai.ntellect/core/types";
-
-const EmailSchema = z.object({
-  to: z.string(),
-  subject: z.string(),
-  content: z.string(),
-  status: z.string(),
+import { InMemoryCheckpointAdapter } from "@ai.ntellect/core";
+
+const adapter = new InMemoryCheckpointAdapter();
+const cpId = await workflow.executeWithCheckpoint("start", adapter, {
+  breakpoints: ["approve_payment"],
 });
 
-const sendNode = {
-  name: "send",
-  execute: async (context: GraphContext<typeof EmailSchema>) => {
-    context.status = "sending";
-    // Email sending implementation
-    context.status = "sent";
-  },
-};
-
-const emailFlow = new GraphFlow({
-  name: "email",
-  schema: EmailSchema,
-  context: {
-    to: "",
-    subject: "",
-    content: "",
-    status: "pending",
-  },
-  nodes: [sendNode],
+// Later, resume with optional state modification:
+await workflow.resumeFromCheckpoint(cpId, adapter, {
+  contextModifications: { status: "approved" },
+});
+```
+
+---
+
+## CortexFlow — Intent Routing
+
+CortexFlow wraps intent classification + Petri Net routing. The LLM identifies intent once; the Petri Net handles all control flow deterministically.
+
+### Basic Usage
+
+```typescript
+import { CortexFlowOrchestrator, IntentClassifier } from "@ai.ntellect/core/routing/orchestrator";
+import { ToolRegistry } from "@ai.ntellect/core";
+import { GraphFlow } from "@ai.ntellect/core";
+
+const llmFn = async (prompt: string) => { /* your LLM call */ };
+const registry = new ToolRegistry();
+const orchestrator = new CortexFlowOrchestrator("mail_assistant", registry);
+
+const classifier = new IntentClassifier(llmFn, {
+  intents: ["FETCH_MAILS", "SUMMARIZE", "UNKNOWN"],
+  confidenceThreshold: 0.7,
+});
+orchestrator.setIntentClassifier(IntentClassifier.toFn(classifier), classifier);
+orchestrator.setLLMCall(llmFn);
+
+// Build Petri Net
+const net = orchestrator.petri;
+net.addPlace({ id: "idle", type: "initial", tokens: [{ id: "start", data: {}, createdAt: 0 }] });
+net.addPlace({ id: "done", type: "final", tokens: [] });
+net.addTransition({
+  id: "process_mails",
+  from: ["idle"],
+  to: "done",
+  action: { type: "graphflow", name: "mail_fetch" },
 });
 
-const assistant = new Agent({
-  role: "Email Assistant",
-  goal: "Help users send emails",
-  tools: [emailFlow],
+registry.register({ name: "mail_fetch", description: "...", graph: new GraphFlow({...}), startNode: "fetch" });
+
+const sessionId = orchestrator.startSession();
+const result = await orchestrator.orchestrate("Summarise my last 5 emails", sessionId);
+```
+
+### Key Components
+
+| Export | Source | Description |
+|--------|--------|-------------|
+| `PetriNet` | `@ai.ntellect/core` | Core Petri Net engine |
+| `CortexFlowOrchestrator` | `@ai.ntellect/core/routing` | Intent → routing → execution |
+| `IntentClassifier` | `@ai.ntellect/core/routing` | LLM-based intent classification |
+| `HybridIntentClassifier` | `@ai.ntellect/core/routing` | Keyword + LLM fallback |
+
+### Advanced Features
+
+- **Hybrid Fallback**: Delegate to conversational LLM on low confidence
+- **Multi-Intent**: Execute multiple intents sequentially
+- **Formal Verification**: Deadlock detection via incidence matrices
+- **Persistence**: Redis/PostgreSQL checkpoint adapters
+
+---
+
+## Agent — Cognitive Loop
+
+The `Agent` class wraps a cognitive loop (think → execute → reply) on top of GraphFlow, with LLM-driven tool selection.
+
+```typescript
+import { Agent, createCalculatorTool } from "@ai.ntellect/core";
+
+const agent = new Agent({
+  role: "Math Assistant",
+  tools: [createCalculatorTool()],
   llmConfig: {
-    provider: "openai",
-    model: "gpt-4",
-    apiKey: process.env.OPENAI_API_KEY,
+    provider: "groq",
+    model: "llama-3.1-8b-instant",
+    apiKey: process.env.GROQ_API_KEY,
   },
 });
 
-const main = async () => {
-  const result = await assistant.process(
-    "Send an email to john@example.com about the project update"
-  );
-  console.log(result);
-};
+const result = await agent.process("What is 25 + 7?");
+```
+
+### Cognitive Loop Components
+
+| Component | Location | Description |
+|-----------|----------|-------------|
+| `Agent` | `agent/agent.ts` | Public API: orchestrates the cognitive loop |
+| `GenericExecutor` | `agent/generic-executor.ts` | Decision-making + action execution |
+| `DynamicGoalHandler` | `agent/handlers/` | Context-aware goal computation |
+| `DynamicNextHandler` | `agent/handlers/` | Next-state routing |
+| `LLMFactory` | `agent/llm-factory.ts` | Multi-provider LLM client |
+| `PromptBuilder` | `agent/prompt-builder.ts` | Structured prompt construction |
+| Tools (`file-system`, etc.) | `agent/tools/` | Prebuilt agent tools |
+
+### Built-in Agent Tools
+
+```typescript
+import {
+  createFileReaderTool,
+  createFileWriterTool,
+  createShellTool,
+  createDirectoryListerTool,
+  createAllAgentTools,
+  createCalculatorTool,
+} from "@ai.ntellect/core";
+```
+
+---
+
+## AgentPipeline — Declarative Pipelines
+
+Declarative pipelines with triggers, typed stages, and human gates.
+
+```typescript
+import { AgentPipeline } from "@ai.ntellect/core";
+import { InMemoryPetriCheckpointAdapter } from "@ai.ntellect/core";
+
+const pipeline = new AgentPipeline({
+  name: "price-monitor",
+  stages: [
+    {
+      id: "fetch",
+      run: async (ctx) => ({ data: await fetchData() }),
+      retry: { max: 3, delayMs: 1000 },
+    },
+    {
+      id: "process",
+      run: async (ctx) => ({ result: processData(ctx.data) }),
+    },
+  ],
+  trigger: { type: "cron", expression: "*/5 * * * *" },
+  gate: "human",
+  checkpointAdapter: new InMemoryPetriCheckpointAdapter(),
+});
+
+await pipeline.start();
+```
+
+---
+
+## Persistence — Memory & Checkpoints
+
+The `persistence/` barrel provides a unified interface for storage:
+
+```typescript
+import {
+  Memory,
+  InMemoryCheckpointAdapter,
+  InMemoryPetriCheckpointAdapter,
+  RedisPetriCheckpointAdapter,
+  PostgresPetriCheckpointAdapter,
+} from "@ai.ntellect/core";
+```
+
+### Memory
+
+```typescript
+import { Memory } from "@ai.ntellect/core";
+import { InMemoryAdapter } from "@ai.ntellect/core/modules/memory";
+
+const memory = new Memory(new InMemoryAdapter());
+await memory.init();
+await memory.createMemory({ content: "Hello", roomId: "default" });
+const results = await memory.getMemoryByIndex("Hello", { roomId: "default" });
+```
+
+### Checkpoint Adapters
+
+| Adapter | Use Case |
+|---------|----------|
+| `InMemoryCheckpointAdapter` | Testing / development |
+| `InMemoryPetriCheckpointAdapter` | PetriNet checkpoint testing |
+| `RedisPetriCheckpointAdapter` | Distributed / production |
+| `PostgresPetriCheckpointAdapter` | SQL-backed persistence |
+| `Neo4jPetriCheckpointAdapter` | Graph-backed persistence |
+
+### Graph Observability & Persistence (Neo4j)
+
+Because the framework relies on explicit routing (Petri Nets) and state machines (GraphFlow), **Neo4j** is supported as a first-class citizen. It allows you to visually audit execution traces, map long-term memory relations, and persist agent entity resolutions without polluting your business logic.
+
+```typescript
+import { 
+  createDriver,
+  Neo4jMemoryAdapter, 
+  Neo4jExecutionTracer, 
+  Neo4jEntityStore 
+} from "@ai.ntellect/core/persistence";
+
+const { driver } = await createDriver({ url: "bolt://localhost:7687", username: "neo4j", password: "password" });
+
+// 1. Graph Memory (Semantic & Relational)
+const memoryAdapter = new Neo4jMemoryAdapter(driver);
+await memoryAdapter.addRelation({ fromId: "mem-1", toId: "mem-2", type: "RELATES_TO", roomId: "user-1" });
+
+// 2. Visual Execution Tracing (Zero-invasion event listener)
+const tracer = Neo4jExecutionTracer.attach(workflow, driver, "exec-123");
+// Visually query in Neo4j Browser: MATCH path=(e:Execution)-[:HAS_RUN]->(n:NodeRun) RETURN path
+
+// 3. Persistent Entity Resolution
+const entityStore = new Neo4jEntityStore(driver);
+await entityStore.recordResolution("ent-1", "find user intent", 0.95);
+```
+
+---
+
+## Planner & Compiler
+
+For advanced workflows, the LLM generates a Zod-validated JSON plan that is compiled into a deterministic GraphFlow at runtime.
+
+```typescript
+import { Planner, Compiler } from "@ai.ntellect/core";
+
+const planner = new Planner(llmFn);
+const plan = await planner.createPlan("Send 0.5 ETH to Alice after checking balance");
+
+const compiler = new Compiler();
+const workflow = compiler.compile(plan);
+
+await workflow.execute("start");
+```
+
+---
+
+## CLI (Interactive REPL)
 
-main();
+```sh
+pnpm cli -p groq -m llama-3.1-8b-instant
+pnpm cli -p openai -m gpt-4o-mini
+pnpm cli -p ollama -m llama3
+```
+
+**Slash commands**: `/status`, `/history`, `/list`, `/resume [cpId]`, `/approve`, `/reject`, `/modify k=v`, `/clear`, `/help`.
+
+The CLI automatically loads API keys from `.env` (`GROQ_API_KEY`, `OPENAI_API_KEY`, `OPENROUTER_API_KEY`, `GOOGLE_API_KEY`).
+
+---
+
+## GraphController
+
+Orchestrate multiple GraphFlows in parallel or sequence:
+
+```typescript
+import { GraphController } from "@ai.ntellect/core";
+
+const controller = new GraphController();
+controller.add("workflow_a", graphA);
+controller.add("workflow_b", graphB);
+
+await controller.parallel("workflow_a", "workflow_b");
+await controller.sequential("workflow_a", "workflow_b");
 ```
 
-## Documentation
+---
+
+## Testing & Build
 
-See the [documentation](https://ai-ntellect.gitbook.io/core) for detailed usage examples.
+```sh
+pnpm run build                  # tsc → dist/
+pnpm test                       # Mocha via ts-node (5s timeout)
+pnpm run test:all               # Full suite (test/**/*.test.ts)
+pnpm test --grep "suite name"   # Focused suite
+pnpm run test:watch             # Watch mode
+```
 
-## Contributing
+**Real LLM tests**: `pnpm test --grep "CortexFlow Real LLM"` (requires Ollama `llama3:latest` on `localhost:11434`).
+
+**Examples**:
+
+```sh
+pnpm run example:hello
+pnpm run example:agent
+pnpm run example:onchain
+```
 
-Contributions are welcome. Please submit pull requests with tests.
+---
 
 ## License