@ai.ntellect/core 0.8.3 → 1.1.0

package/.mocharc.json

@@ -1,5 +1,6 @@
 {
   "extension": ["ts"],
   "require": ["./node_modules/ts-node/register"],
-  "timeout": 5000
+  "timeout": 5000,
+  "spec": "test/**/*.test.ts"
 }

package/AGENTS.md

@@ -0,0 +1,50 @@
+# AGENTS.md
+
+## Project
+- `@ai.ntellect/core` v0.12.0 — pnpm v10.33.0
+- **Thesis**: LLM as **Classifier** → **Routing** (Petri Net) → **Execution** (GraphFlow)
+- `pnpm install` runs `prepare` script (`tsc`), so build is automatic on install
+
+## Commands
+```sh
+pnpm run build                # tsc → dist/
+pnpm test                     # Mocha via ts-node (5s timeout, .mocharc.json)
+pnpm run test:all             # test/**/*.test.ts
+pnpm test --grep "suite"      # focused suite
+pnpm run test:watch           # watch mode
+pnpm run test:watch:execution # watch only execution tests
+```
+**Real LLM**: `pnpm test --grep "CortexFlow Real LLM"` (needs Ollama `llama3:latest` at `:11434`)
+
+## Architecture (Bounded Contexts)
+```
+index.ts          → public API barrel
+execution/        → GraphFlow (typed nodes, events, checkpoints, planner, compiler)
+routing/          → PetriNet, CortexFlowOrchestrator, IntentClassifier
+agent/            → Agent, GenericExecutor, handlers, tools
+persistence/      → barrel re-exporting Memory + checkpoint adapters
+pipeline/         → AgentPipeline (trigger → stages → gate)
+modules/          → remaining plugins: agenda, cli, embedding, memory, nlp
+interfaces/       → contract interfaces
+types/            → Zod schemas (use z.unknown(), NOT z.any())
+app/              → Next.js frontend (independent, not part of core build)
+```
+
+### Path Alias
+`@/*` → `./*` (tsconfig paths). Safe to use in core source.
+
+### TS Config Notes
+- `routing/web-server.ts` is explicitly excluded from build (missing express/socket.io types)
+- `test/` and `examples/` are excluded from tsc (ts-node handles them at runtime)
+
+## CLI Tools
+- **Agent REPL**: `pnpm cli -p <provider> -m <model>` (openai, ollama, groq, openrouter, google)
+- **Debugger**: `npx ts-node cli-dev.ts [workflow.json]`
+
+## Constraints
+- Use native `fetch` (Node 18+). **No axios.**
+- Use Zod v4+ syntax. Prefer `z.unknown()` over `z.any()`.
+- `.env` auto-loaded by CLI; test suite does NOT preload dotenv
+
+## Pre-existing Test Quirks
+- Real LLM Ollama tests (`test/routing/real-llm.test.ts`) are flaky — Ollama occasionally returns `UNKNOWN` instead of the expected intent. Not caused by code changes.

package/ARCHITECTURE.md

@@ -0,0 +1,58 @@
+# @ai.ntellect/core — Architecture
+
+## Core Architecture
+
+```
+User Message
+    ↓
+IntentClassifier  (1 LLM call — classify intent)
+    ↓
+PetriNet          (deterministic token-based routing — no LLM)
+    ↓
+GraphFlow         (node execution — tools, APIs, optional dynamic plan)
+    ↓
+Zod Validation    (typed state at every step)
+```
+
+## Key Components
+
+| Component | Role |
+|-----------|------|
+| `CortexFlowOrchestrator` | Top-level coordinator: intent → Petri net → GraphFlow |
+| `PetriNet` | Formally-verified state machine (deadlock detection, boundedness) |
+| `IntentClassifier` | Single-LLM-call intent resolution with confidence thresholding |
+| `GraphFlow` | Graph container: name, schema, context, nodes |
+| `GraphNode` | Execution engine: runs nodes, handles transitions |
+| `GraphEventManager` | Event system: RxJS Subjects + EventEmitter |
+| `GraphObserver` | Reactive API: subscribe to state/events |
+| `CheckpointAdapter` | Persistence: save/resume state |
+| `ToolRegistry` | Registry of GraphFlow tools available to transitions |
+| `AgentPipeline` | Declarative workflow pipelines with triggers, human gates, retry logic |
+
+## Design Principles
+
+### 1. LLM as Planner, Not Runtime
+The LLM makes **one decision** (intent classification), then the Petri Net handles all routing deterministically. This prevents context bloat in long sessions.
+
+### 2. Formal Verification
+Petri Nets enable static analysis before deployment:
+- **Deadlock detection** — identify blocking states before runtime
+- **Boundedness check** — verify token counts stay bounded
+- **Reachability analysis** — enumerate all reachable states
+
+These checks run via `matrix.ts` at build time.
+
+### 3. Typed State Everywhere
+Zod schemas validate context at every node transition. If types don't match, the workflow fails fast.
+
+### 4. Deterministic Routing
+Once intent is classified, all routing is token-based through the Petri Net — no additional LLM calls for routing decisions.
+
+## Why Petri Nets?
+
+Unlike state machines or simple graphs, Petri Nets provide:
+- **Concurrency modeling** — tokens can exist in multiple places simultaneously
+- **Static analysis** — mathematical properties can be verified
+- **Formal semantics** — unambiguous behavior definition
+
+This makes them ideal for workflows where correctness matters more than flexibility.

package/BENCHMARK.md

@@ -0,0 +1,38 @@
+# @ai.ntellect/core — Benchmarks
+
+## Scenario
+Fetch 5 emails → batch-classify urgency → draft replies for urgent ones → archive the rest.
+
+Run with:
+```sh
+pnpm run benchmark   # requires Ollama + llama3:latest, or GROQ_API_KEY in .env
+```
+
+Results below are measured, not projected. A warmup call is made before each timer starts to exclude model loading.
+
+CortexFlow uses a `HybridIntentClassifier` — keyword rules resolve unambiguous commands in microseconds; the LLM is only called when the message is genuinely ambiguous. All other routing is handled by the Petri Net with no additional LLM calls.
+
+### Ollama local — llama3:latest
+
+| | CortexFlow | LangGraph naive | LangGraph optimised |
+|---|---|---|---|
+| LLM calls | **2** | 7 | 2 |
+| Total time | **4.1s** | 3.7s | 4.2s |
+| vs naive | **0.9× speed** (71% fewer LLM calls) | baseline | 0.9× slower |
+
+**CortexFlow reduces LLM calls by 71% vs LangGraph naive, with full traceability and formal Petri Net verification.**
+
+### Groq API — llama-3.1-8b-instant
+
+| | CortexFlow | LangGraph naive | LangGraph optimised |
+|---|---|---|---|
+| LLM calls | **1** | 7 | 2 |
+| Total time | **1 650 ms** | 2 192 ms | 1 668 ms |
+| vs naive | **1.33× faster** | baseline | 1.31× faster |
+
+**LLM call reduction vs naive: −86% on both backends.**
+
+## Interpretation
+- CortexFlow trades a small speed overhead on local Ollama for 71% fewer LLM calls and formal verification
+- On fast API backends (Groq), CortexFlow is faster than LangGraph naive while using 86% fewer LLM calls
+- All routing beyond intent classification is deterministic and LLM-free, avoiding context bloat in long sessions

package/CHANGELOG.md

@@ -0,0 +1,224 @@
+# Change Log
+
+All notable changes to this project will be documented in this file.
+
+## [1.1.0] - 2026-06-17
+
+### 🚀 Added
+- **Graph Observability & Persistence (Neo4j)**: First-class support for Neo4j as an observability and persistence backend.
+  - `Neo4jExecutionTracer`: Zero-invasion event listener that visually maps `GraphFlow` execution paths (`Execution` -> `NodeRun`).
+  - `Neo4jMemoryAdapter`: Supports standard vector memory along with complex semantic graph relations (`RELATES_TO`).
+  - `Neo4jEntityStore`: Persistent agent entity resolution indexing file structures and intent history.
+  - `Neo4jPetriCheckpointAdapter`: Persist `PetriNet` checkpoints directly into the graph.
+
+### 🧹 Refactored & Simplified
+- **CognitiveHandler Consolidation**: `DynamicGoalHandler` and `DynamicNextHandler` have been fused into a single lightweight `CognitiveHandler` used by the `GenericExecutor`.
+- **EntityIndex Simplification**: Consolidated the redundant lookup maps inside `EntityIndex` down to a single synchronized store with write-behind support for the new EntityStore pattern.
+- **Event Manager Cleanup**: Removed duplicated RxJS event streams (`nodeStreams`) and dead loops in `execution/event-manager.ts` while maintaining public API compatibility for `GraphFlow`.
+- **Package Size Optimization**: Moved heavy dependencies only used in examples (`viem`, `googleapis`, `@nktkas/hyperliquid`, `@langchain/*`) to `devDependencies`. Neo4j driver added as an optional `peerDependency`.
+
+## [Unreleased]
+
+### 🚀 Added
+
+* **Petri Core Public API**
+  `PetriNet`, `CortexFlowOrchestrator`, `IntentClassifier`, `HybridIntentClassifier`, and checkpoint adapters (Redis, PostgreSQL) are now exported from the root `index.ts`.
+  → Makes the deterministic routing thesis accessible to consumers of the package.
+
+* **Extracted Cognitive Handlers**
+  `DynamicGoalHandler` and `DynamicNextHandler` extracted from the monolithic `GenericExecutor` into focused, independently testable classes in `agent/handlers/`.
+  → Enables unit-testing goal computation and next-state routing without instantiating the full executor.
+
+* **Extracted Agent Workflow Builder**
+  `buildAgentWorkflow()` extracted from `Agent.setupWorkflow()` into `agent/agent-workflow.ts`.
+  → Makes the cognitive node graph (defineGoal → think → execute → reply) testable and reusable without the `Agent` class.
+
+* **Persistence Barrel**
+  `persistence/index.ts` re-exports `Memory`, `InMemoryCheckpointAdapter`, and Petri checkpoint adapters as a unified storage layer.
+  → Simplifies imports when using both memory and checkpoint adapters together.
+
+### 🔧 Changed
+
+* **Agent Refactored to Pure Orchestrator**
+  `Agent` class now delegates to `GenericExecutor` (handlers) and `buildAgentWorkflow()` instead of defining inline closure-heavy nodes.
+  → Each responsibility (goal, next-state, action execution) is handled by a focused module.
+
+* **Dead Code Elimination**
+  Removed `routing/agent-pipeline.ts` (duplicate of `pipeline/agent-pipeline.ts` — confirmed zero imports).
+  → Eliminates the architectural ambiguity of having two competing `AgentPipeline` abstractions.
+
+* **Orphaned Patterns Removed**
+  Removed `execution/worker-pool.ts`, `execution/workers/cpu-worker.ts`, `execution/subgraph.ts`, and `agent/generic-assistant.ts` — all confirmed dead code with zero imports across the codebase.
+  → Reduces maintenance surface and clarifies what the core engine actually needs.
+
+* **Bounded Contexts Restructure**
+  Renamed directories to match domain boundaries:
+  - `graph/` → `execution/` (GraphFlow engine)
+  - `petri/` → `routing/` (PetriNet, CortexFlow)
+  - `modules/agent/` → `agent/` (promoted to root-level bounded context)
+  All imports updated across the codebase, test directories renamed accordingly. Public API (`index.ts` exports) and package consumers are unaffected.
+
+---
+
+## [0.12.0] - 2026-05-03
+
+### 🚀 Added
+
+* **CortexFlow Orchestration**
+  Stop LLMs from deciding your execution path.
+  $\rightarrow$ Eliminates routing hallucinations and uncontrolled tool selection.
+
+* **Hybrid Intent Classification**
+  Cut latency and eliminate classification guesswork with keyword-first detection and LLM fallback.
+  $\rightarrow$ Increases reliability in production workloads.
+
+* **Multi-Intent Execution**
+  One message, multiple workflows.
+  $\rightarrow$ Enables compound flows (e.g. fetch $\rightarrow$ process $\rightarrow$ notify) without repeated chatting.
+
+* **Formal Verification for Workflows**
+  Mathematically prove your agent can't deadlock.
+  $\rightarrow$ Prevents invalid execution graphs before runtime.
+
+* **Persistent Execution State**
+  Stop losing state on crashes with Redis and PostgreSQL checkpoint adapters.
+  $\rightarrow$ Enables long-running, fault-tolerant agents.
+
+* **Petri Pattern Library**
+  Stop starting from scratch. Prebuilt templates for Human-Approval, RAG, and Structured Data Extraction.
+  $\rightarrow$ Accelerates the building of common agent patterns.
+
+* **Benchmark Suite**
+  The Proof: CortexFlow vs LangGraph.
+  $\rightarrow$ Demonstrates significant reduction in LLM calls and lower latency.
+
+* **Production-Centric Documentation**
+  Complete overhaul of the `/docs` folder and README.
+  $\rightarrow$ Shifts focus from technical manual to product-centric "Control Layer" guide.
+
+### 🔧 Changed
+
+* **Architecture Positioning**
+  Framework redefined as: **“A control layer for LLM systems”** rather than a general-purpose agent framework.
+  $\rightarrow$ Clearly differentiates the system from probabilistic autonomous agents.
+
+* **Zod Upgrade**
+  Migrated to latest schema validation patterns for stricter runtime guarantees.
+
+### 🐛 Fixed
+
+* Improved stability in benchmark execution under high concurrency.
+* Fixed inconsistent Petri transition edge cases under concurrent token updates.
+
+---
+
+## [0.11.0] - 2026-05-02
+
+### 🚀 Added
+
+* **Plan $\rightarrow$ Compile $\rightarrow$ Execute Pattern**
+  Implemented a pattern where the LLM generates a Zod-validated plan, which is then compiled into a deterministic GraphFlow.
+  $\rightarrow$ Ensures the LLM handles the "What" (planning) while the system handles the "How" (execution).
+
+* **Advanced Verification Suite**
+  Integrated real-world LLM tests (via Groq) and on-chain tests (via Sepolia).
+  $\rightarrow$ Guarantees reliability in actual production environments.
+
+### 🔧 Changed
+
+* **AI-Optimized Instructions**
+  Updated `AGENTS.md` with high-signal guidance for AI agents contributing to the repo.
+
+---
+
+## [0.10.0] - 2026-04-30
+
+### 🚀 Added
+
+* **Checkpoint & Time-Travel System**
+  Implemented save/resume functionality with the ability to jump back to any state.
+  $\rightarrow$ Enables bulletproof debugging and long-term workflow persistence.
+
+* **Interactive Steering CLI**
+  Added a REPL with slash commands for real-time agent oversight.
+  $\rightarrow$ Allows developers to approve, reject, or modify agent state mid-execution.
+
+* **Multi-Graph Orchestration**
+  Introduced `GraphController` to manage multiple GraphFlows in parallel or sequence.
+  $\rightarrow$ Allows building complex, modular systems from simple, testable graphs.
+
+---
+
+## [0.8.0] - 2025-04-13
+
+### 🚀 Added
+
+* **Configurable Cognitive Agents**
+  Introduced an `Agent` class with cognitive loops and tool integration.
+  $\rightarrow$ Bridges the gap between pure workflows and interactive assistants.
+
+* **Low-Latency NLP Engine**
+  Integrated `@nlpjs/basic` for intent detection without needing an LLM call.
+  $\rightarrow$ Drastically reduces cost and latency for simple classification tasks.
+
+* **Standardized Prompting & LLM Factory**
+  Implemented `PromptBuilder` and `LLMFactory` for consistent model handling.
+  $\rightarrow$ Simplifies switching between providers (OpenAI, Groq, Ollama).
+
+### 🔧 Changed
+
+* **Event Correlation Logic**
+  Enhanced `waitForCorrelatedEvents` for precision in multi-user environments.
+  $\rightarrow$ Prevents event "leakage" between different workflow instances.
+
+---
+
+## [0.6.0] - 2025-02-04
+
+### 🚀 Added
+
+* **Reactive State Observation**
+  Implemented `GraphObserver` using RxJS for real-time state tracking.
+  $\rightarrow$ Enables building live dashboards and monitoring tools for agents.
+
+* **Concurrent Execution Model**
+  Added Fork-Join parallelism and event-driven nodes to `GraphFlow`.
+  $\rightarrow$ Allows agents to perform multiple tasks simultaneously and wait for external triggers.
+
+* **Enterprise Persistence**
+  Integrated Redis and Meilisearch adapters for shared, persistent state.
+  $\rightarrow$ Moves the system from "in-memory" to "distributed production" ready.
+
+---
+
+## [0.3.0] - 2025-01-20
+
+### 🚀 Added
+
+* **Advanced Memory Management**
+  Implemented `AgentRuntime` with dynamic action processing.
+  $\rightarrow$ Improves how agents remember and utilize previous tool outputs.
+
+* **Action Scheduling**
+  Added the ability to schedule and cancel future actions.
+  $\rightarrow$ Enables agents to handle time-delayed tasks and reminders.
+
+### 🔧 Changed
+
+* **Result Interpretation**
+  Replaced `Synthesizer` with a more robust `Interpreter` class.
+  $\rightarrow$ Improves the quality and structure of final AI responses.
+
+---
+
+## [0.0.x - 0.1.x] - 2025-01-15 to 2025-01-20
+
+### 🚀 Added
+
+* **Core Engine Foundations**
+  Initial implementation of `GraphFlow` and the event-driven execution model.
+  $\rightarrow$ Established the baseline for typed, deterministic workflow management.
+
+* **Agent & Memory Basics**
+  Introduced basic `Agent` classes and in-memory storage adapters.
+  $\rightarrow$ Provided the first functional tool-using AI assistants.