noormme 1.0.5 → 1.1.0

package/README.md

@@ -1,63 +1,64 @@
+![NOORMME: The Agentic Data Engine](/Users/bozoegg/.gemini/antigravity/brain/af239a4f-144c-41a9-b1c4-9d534b5243f8/noormme_hero_banner_1771742014638.png)
+
 # NOORMME: The Agentic Data Engine
 
-**NOORMME** is a sovereign persistence layer and cognitive operating system designed for **Autonomous AI Agents**. It transcends legacy ORM patterns by providing a self-healing, evolutionary data infrastructure that functions as an extension of the agent's internal reasoning loop.
+**NOORMME** is a sovereign persistence layer and cognitive operating system designed for **Autonomous AI Agents**. It transcends legacy ORM patterns by providing a self-healing, evolutionary data infrastructure that functions as a high-fidelity extension of the agent's internal reasoning loop.
 
 [![Version](https://img.shields.io/npm/v/noormme.svg)](https://www.npmjs.com/package/noormme)
 [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
-[![Cognitive](https://img.shields.io/badge/Intelligence-Autonomous_Governance-purple.svg)](#-autonomous-governance)
+[![Cognitive](https://img.shields.io/badge/Intelligence-Autonomous_Governance-purple.svg)](./docs/agentic-intelligence.md)
+[![Evolution](https://img.shields.io/badge/Evolution-Ultra_Scale-blueviolet.svg)](./docs/ultra-scale-orchestration.md)
 
 ---
 
-## 🏗 The Agentic Data Loop
+## 🧠 The Cognitive Loop
+
+NOORMME implements a closed-loop system where data is not merely "stored" but continuously distilled, questioned, and evolved. It turns your database into a living world model.
 
-NOORMME implements a closed-loop system where data is not merely "stored" but continuously distilled, questioned, and evolved.
+```mermaid
+graph LR
+    A[Raw Ingestion] --> B{Distillation}
+    B --> C[Knowledge Base]
+    C --> D[Reasoning Engine]
+    D --> E[Action / Outcome]
+    E --> F{Reflection}
+    F -->|Success| G[Goal Reinforcement]
+    F -->|Failure| H[Strategic Mutation]
+    H --> B
+```
 
-### 🧠 Cognitive Orchestration
-- **Strategic Planner**: Proactively suggests and applies persona mutations (roles, policies, capabilities) based on real-time success/latency telemetry.
-- **Knowledge Distiller**: Extracts structured entities and facts from raw history using Jaccard similarity-based consolidation and confidence gradients.
-- **Curiosity Engine**: Identifies "Knowledge Gaps" and "Factual Contradictions," generating research hypotheses and active questions for the agent to resolve.
-- **Ablation Labs**: Conducts "Zombie Pruning" and "Impact Tests"—temporarily disabling knowledge to evaluate its necessity in the reasoning path.
+### ✨ Advanced Orchestration
+- **🎭 Tiered Model Routing**: Optimize cost/performance by routing batch discovery to **Fast LLMs** and mutation logic to **Premium LLMs**.
+- **🔥 Predictive Pre-warming**: Background AI optimization of skills nearing promotion to eliminate cold-start latency.
+- **🌐 Goal Cross-Pollination**: Breakthroughs in individual persona mutations are automatically distilled into global **Systemic Best-Practices**.
+- **🌸 Bloom Filter Heuristics**: Ultra-scale knowledge de-duplication at 100x speed vs traditional DB hits.
 
 ---
 
-## 💎 Technical Moats
+## 🧬 Evolutionary Infrastructure (DNA Inversion)
 
-### 🧬 Evolutionary Infrastructure & "DNA Inversion"
-NOORMME allows agents to autonomously mutate their own schema.
-- **Dynamic DDL**: Agents can propose `CREATE TABLE`, `ADD COLUMN`, or `CREATE INDEX` mutations via the **Evolutionary Pilot**.
-- **Structural Rollback**: The **DNA Inverter** automatically generates inverse SQL for any DDL mutation (e.g., inverting an `ADD COLUMN` to a `DROP COLUMN`) to ensure safe structural experimentation.
-- **Real-Time Type Synthesis**: TypeScript interfaces and types are regenerated and written to disk the moment a structural change is committed.
+NOORMME allows agents to autonomously mutate their own structural DNA while maintaining 100% safety.
 
-### 🚥 Autonomous Governance (Logic Probes)
-The **Self-Test Registry** enables agents to maintain their own integrity through automated "probes":
-- **Integrity Audits**: Detects orphaned records and semantic memory failures.
-- **Performance Drift**: Detects when autonomous schema changes cause query latency to drift more than 50% from a rolling 100-query baseline.
-- **Consistency Probes**: Verifies that new knowledge doesn't contradict established "Hive" facts.
-
-### 📊 Deep Behavioral Telemetry
-A three-layered telemetry stack for research and production observation:
-1. **Raw Event Harvester**: High-fidelity capture of every interaction, pivot, and failure.
-2. **Cognitive Synthesizer**: Serializes the strategic "Evolution Path" and calculates dynamic **Autonomy Levels**.
-3. **Research Alchemist**: Transmutes events into high-order metrics like **Time-to-Magic**, **Discovery Index**, and **Trust Signals**.
+- **Dynamic DDL**: Agents can propose `CREATE TABLE`, `ADD COLUMN`, or `CREATE INDEX` mutations via the **Evolutionary Pilot**.
+- **Structural Rollback**: The **DNA Inverter** automatically generates inverse SQL for any DDL mutation to ensure safe structural experimentation.
+- **Meta-Meta Evolution**: A self-tuning system that modulates its own **Mutation Aggressiveness** and **Verification Windows** based on systemic success rates.
 
 ---
 
-## ⚡ Performance Engineering
-
-### Query Intelligence
-- **N+1 Detection**: Real-time pattern analysis to identify and warn about inefficient recursive queries.
-- **Semantic Caching**: Context-aware result caching that understands query patterns rather than just raw SQL.
-- **Dialect Optimization**: Native WAL-mode management for SQLite and optimized JSONB/Vector operations for PostgreSQL.
+## 🚥 Autonomous Governance
 
-### High-Fidelity Infrastructure
-- **Unified Vector Abstraction**: Seamless semantic search across `PGVector`, `SQLite-vss`, and a optimized **Manual Cosine Fallback**.
-- **Ritual Orchestration**: Automated background "rituals" for semantic compression, stale history pruning, and database vacuuming.
+- **Self-Healing Indexing**: Proactively applies performance indexes based on the agent's own observation of "slow" query contexts.
+- **Conflict Resolution**: Scans cognitive rules for semantic overlaps and logical contradictions.
+- **Zombie Pruning**: Automatically identifies and removes deprecated or unused knowledge items.
+- **Z-Score Sensitivity**: Monitors for performance collapse (latency or success rate) and triggers emergency rollbacks.
 
 ---
 
-## 🚀 Implementation
+## 🚀 Getting Started
+
+### 1. Provision the Mind
+Initialize your database with a single line to provision 25+ agentic tables for goals, knowledge, episodes, and logic probes.
 
-### Initializing the Mind
 ```typescript
 import { NOORMME } from 'noormme';
 
@@ -65,37 +66,48 @@ const db = new NOORMME({
   dialect: 'sqlite',
   connection: { database: './mind.sqlite' },
   agentic: {
-    enableSelfEvolution: true,
-    enableSelfHealing: true
+    llm: primaryModel,
+    llmFast: gpt4oMini,     // For high-throughput batching
+    llmPremium: claude35,  // For complex mutations
+    enableSelfEvolution: true
   }
 });
 
-// Provisions 20+ tables for goals, knowledge, episodes, and logic probes
 await db.initialize();
 ```
 
-### Strategic Interaction
+### 2. Strategic Interaction
 ```typescript
-// Access the higher-order cognitive facade
 const cortex = db.agent.cortex;
 
-// Execute a background knowledge distillation ritual
+// Execute background rituals (Compression, Pruning, Evolution)
 await cortex.rituals.runPendingRituals();
 
 // Challenge existing knowledge with new evidence
-await cortex.knowledge.challengeKnowledge('SystemArchitecture', 'New facts contradict old.', 0.95);
+await cortex.knowledge.challengeKnowledge('SystemArch', 'New audit results.', 0.95);
 ```
 
 ---
 
-## 🗄 Dialect Matrix
+## 🗄 Dialect Power Matrix
 
 | Feature | SQLite (Edge) | PostgreSQL (Enterprise) |
 | :--- | :--- | :--- |
-| **Search** | `sqlite-vss` / Fallback | `pgvector` |
-| **Persistence** | WAL Mode | Native Pooling / SSL |
-| **Evolution** | DNA Inversion | Structural Migrations |
-| **Reliability** | Local Atomicity | Multi-Tenant Isolation |
+| **Search** | `sqlite-vss` / Fallback | `pgvector` / FTS |
+| **Logic** | In-Process Atomicity | Multi-Tenant Governance |
+| **Evolution** | Sequential Snapshotting | Clustered Mutation |
+| **Throughput** | Sequential Direct | Bloom-Heuristic Parallel |
+
+---
+
+## 📚 Explore the Documentation
+Dive into our deep-dive guides to unlock the full potential of NOORMME:
+
+- [**Ultra-Scale Orchestration**](./docs/ultra-scale-orchestration.md)
+- [**Strategic Evolution Guides**](./docs/strategic-evolution.md)
+- [**Skill Lifecycle & DNA**](./docs/skill-lifecycle.md)
+- [**Sovereign Meta-Evolution**](./docs/meta-evolution.md)
+- [**Agentic Intelligence Deep-Dive**](./docs/agentic-intelligence.md)
 
 ---
 
@@ -104,6 +116,4 @@ NOORMME is an Apache 2.0 open-source project. We invite AI researchers and data
 
 [Contribution Guide](CONTRIBUTING.md) | [Security Audit](SECURITY.md)
 
----
-
 *Transforming passive records into sovereign intelligence.*

package/dist/cjs/agentic/CapabilityManager.d.ts

@@ -1,10 +1,12 @@
 import type { Kysely } from '../kysely.js';
-import type { AgenticConfig, AgentCapability } from '../types/index.js';
+import { AgenticConfig, AgentCapability } from '../types/index.js';
+import type { Cortex } from './Cortex.js';
 export interface CapabilityTable {
     id: number | string;
     name: string;
     version: string;
     description: string | null;
+    status: 'experimental' | 'verified' | 'blacklisted';
     reliability: number;
     metadata: string | null;
     created_at: string | Date;
@@ -19,9 +21,11 @@ export interface CapabilityDatabase {
  */
 export declare class CapabilityManager {
     private db;
+    private cortex;
     private config;
     private capabilitiesTable;
-    constructor(db: Kysely<any>, config?: AgenticConfig);
+    private evolutionConfig;
+    constructor(db: Kysely<any>, cortex: Cortex, config?: AgenticConfig);
     private get typedDb();
     /**
      * Register or update a capability (skill)
@@ -29,6 +33,7 @@ export declare class CapabilityManager {
     registerCapability(name: string, version: string, description?: string, metadata?: Record<string, any>): Promise<AgentCapability>;
     /**
      * Update reliability based on action outcome using a damped moving average.
+     * Manages the lifecycle of emergent skills (sandbox -> verified / blacklisted).
      */
     reportOutcome(name: string, success: boolean): Promise<void>;
     /**
@@ -36,8 +41,8 @@ export declare class CapabilityManager {
      */
     getReliability(name: string): Promise<number>;
     /**
-     * Get all registered capabilities
+     * Get all registered capabilities, optionally filtered by status
      */
-    getCapabilities(): Promise<AgentCapability[]>;
+    getCapabilities(status?: AgentCapability['status']): Promise<AgentCapability[]>;
     private parseCapability;
 }

package/dist/cjs/agentic/CapabilityManager.js

@@ -7,12 +7,22 @@ exports.CapabilityManager = void 0;
  */
 class CapabilityManager {
     db;
+    cortex;
     config;
     capabilitiesTable;
-    constructor(db, config = {}) {
+    evolutionConfig;
+    constructor(db, cortex, config = {}) {
         this.db = db;
+        this.cortex = cortex;
         this.config = config;
         this.capabilitiesTable = config.capabilitiesTable || 'agent_capabilities';
+        this.evolutionConfig = {
+            verificationWindow: config.evolution?.verificationWindow ?? 20,
+            rollbackThresholdZ: config.evolution?.rollbackThresholdZ ?? 2.5,
+            enableHiveLink: config.evolution?.enableHiveLink ?? true,
+            mutationAggressiveness: config.evolution?.mutationAggressiveness ?? 0.5,
+            maxSandboxSkills: config.evolution?.maxSandboxSkills ?? 5
+        };
     }
     get typedDb() {
         return this.db;
@@ -33,6 +43,7 @@ class CapabilityManager {
                     .updateTable(this.capabilitiesTable)
                     .set({
                     description: description || existing.description,
+                    status: existing.status || 'experimental',
                     metadata: JSON.stringify({ ...JSON.parse(existing.metadata || '{}'), ...metadata }),
                     updated_at: new Date()
                 })
@@ -47,6 +58,7 @@ class CapabilityManager {
                 name,
                 version,
                 description: description || null,
+                status: metadata.initialStatus || 'experimental',
                 reliability: 1.0,
                 metadata: JSON.stringify({ ...metadata, successCount: 0, totalCount: 0 }),
                 created_at: new Date(),
@@ -59,6 +71,7 @@ class CapabilityManager {
     }
     /**
      * Update reliability based on action outcome using a damped moving average.
+     * Manages the lifecycle of emergent skills (sandbox -> verified / blacklisted).
      */
     async reportOutcome(name, success) {
         await this.db.transaction().execute(async (trx) => {
@@ -80,11 +93,75 @@ class CapabilityManager {
                 const newReliability = success
                     ? Math.min(1.0, currentReliability * (1 - alpha) + alpha)
                     : Math.max(0.0, currentReliability * (1 - alpha));
+                let newStatus = cap.status || 'experimental';
+                // --- Emergent Skill Evolution Optimization ---
+                const successStreak = (metadata.successStreak || 0) + (success ? 1 : 0);
+                const failureStreak = success ? 0 : (metadata.failureStreak || 0) + 1;
+                const streakSuccess = success ? successStreak : 0;
+                const winRate = successCount / totalCount;
+                const windowSize = this.evolutionConfig.verificationWindow || 20;
+                const minSampleSize = Math.ceil(windowSize * 0.75);
+                // Fast-Track Promotion: 5 consecutive successes bypasses sample size
+                const isPromotable = (totalCount >= minSampleSize && winRate >= 0.8) || (streakSuccess >= 5);
+                // Early-Exit Rollback: 3 consecutive failures at the start immediately blacklists
+                const isCatastrophic = !success && failureStreak >= 3 && totalCount <= 5;
+                // Pass 6: Predictive Pre-warming Trigger
+                // If a skill is close to promotion, pre-warm its optimized description
+                const promoThreshold = Math.ceil(minSampleSize * 0.8);
+                const isNearingPromotion = (totalCount >= promoThreshold && winRate >= 0.8) || (streakSuccess === 4);
+                if (isNearingPromotion && newStatus === 'experimental' && this.cortex.skillSynthesizer) {
+                    // Trigger async background pre-warming
+                    this.cortex.skillSynthesizer.preWarmSkill(name).catch(() => { });
+                }
+                // --- Production Hardening: Dynamic Performance Baselining ---
+                const historyAlpha = 0.05; // Slower moving average for baseline
+                const baseline = metadata.performanceBaseline ?? winRate;
+                const newBaseline = (baseline * (1 - historyAlpha)) + (winRate * historyAlpha);
+                // Variance tracking for Z-score calculation
+                const variance = metadata.performanceVariance ?? 0.01;
+                const diff = winRate - baseline;
+                const newVariance = (variance * (1 - historyAlpha)) + (Math.pow(diff, 2) * historyAlpha);
+                const stdDev = Math.sqrt(newVariance);
+                // Z-Score: How many standard deviations is current performance from baseline?
+                const zScore = stdDev > 0 ? (winRate - baseline) / stdDev : 0;
+                // Promotion/Demotion Logic
+                if (isCatastrophic && (newStatus === 'experimental' || newStatus === 'sandbox')) {
+                    console.error(`[CapabilityManager] Skill '${name}' FAILED early-exit safety check (Streak: ${failureStreak}). Blacklisting immediately.`);
+                    newStatus = 'blacklisted';
+                }
+                else if (isPromotable && (newStatus === 'experimental' || newStatus === 'sandbox')) {
+                    console.log(`[CapabilityManager] Skill '${name}' PASSED fast-track verification (Streak: ${streakSuccess}, Rate: ${(winRate * 100).toFixed(1)}%). Promoting to Verified.`);
+                    newStatus = 'verified';
+                }
+                else if (totalCount >= minSampleSize) {
+                    if (winRate < 0.4) {
+                        console.log(`[CapabilityManager] Skill '${name}' FAILED statistical verification (Rate: ${(winRate * 100).toFixed(1)}%). Blacklisting.`);
+                        newStatus = 'blacklisted';
+                    }
+                    else if (newStatus === 'verified' && zScore < -2.0) {
+                        // Performance Collapse: Z-score indicates current run is significantly below historical baseline
+                        console.warn(`[CapabilityManager] Verified skill '${name}' PERFORMANCE COLLAPSE (Z: ${zScore.toFixed(2)}, Rate: ${(winRate * 100).toFixed(1)}%). Demoting to Experimental.`);
+                        newStatus = 'experimental';
+                    }
+                }
+                if (newStatus !== cap.status) {
+                    console.log(`[CapabilityManager] EVOLVING STATUS: ${name} (${cap.status} -> ${newStatus})`);
+                }
                 await trx
                     .updateTable(this.capabilitiesTable)
                     .set({
                     reliability: newReliability,
-                    metadata: JSON.stringify({ ...metadata, totalCount, successCount }),
+                    status: newStatus,
+                    metadata: JSON.stringify({
+                        ...metadata,
+                        totalCount,
+                        successCount,
+                        successStreak: streakSuccess,
+                        failureStreak,
+                        performanceBaseline: newBaseline,
+                        performanceVariance: newVariance,
+                        lastOutcomeType: success ? 'success' : 'failure' // Categorization point
+                    }),
                     updated_at: new Date()
                 })
                     .where('id', '=', cap.id)
@@ -105,12 +182,16 @@ class CapabilityManager {
         return cap ? cap.reliability : 0.0;
     }
     /**
-     * Get all registered capabilities
+     * Get all registered capabilities, optionally filtered by status
      */
-    async getCapabilities() {
-        const list = await this.typedDb
+    async getCapabilities(status) {
+        let query = this.typedDb
             .selectFrom(this.capabilitiesTable)
-            .selectAll()
+            .selectAll();
+        if (status) {
+            query = query.where('status', '=', status);
+        }
+        const list = await query
             .orderBy('name', 'asc')
             .execute();
         return list.map(c => this.parseCapability(c));
@@ -121,6 +202,7 @@ class CapabilityManager {
             name: cap.name,
             version: cap.version,
             description: cap.description,
+            status: cap.status || 'experimental',
             reliability: cap.reliability,
             metadata: typeof cap.metadata === 'string' ? JSON.parse(cap.metadata) : (cap.metadata || {}),
             createdAt: new Date(cap.created_at),

package/dist/cjs/agentic/Cortex.d.ts

@@ -1,5 +1,5 @@
 import type { Kysely } from '../kysely.js';
-import type { AgenticConfig, AgentMessage, NOORMConfig } from '../types/index.js';
+import type { AgenticConfig, AgentMessage, NOORMConfig, LLMProvider } from '../types/index.js';
 import { SessionManager } from './SessionManager.js';
 import { ContextBuffer } from './ContextBuffer.js';
 import { VectorIndexer } from './VectorIndexer.js';
@@ -29,6 +29,8 @@ import { StrategicPlanner } from './improvement/StrategicPlanner.js';
 import { AblationEngine } from './improvement/AblationEngine.js';
 import { SelfTestRegistry } from './improvement/SelfTestRegistry.js';
 import { TelemetryOrchestrator } from './telemetry/TelemetryOrchestrator.js';
+import { SkillSynthesizer } from './improvement/SkillSynthesizer.js';
+import { EvolutionRitual } from './improvement/EvolutionRitual.js';
 /**
  * Cortex is the unified facade for agentic operations.
  * It coordinates sessions, memory, reflection, and knowledge.
@@ -65,6 +67,11 @@ export declare class Cortex {
     ablation: AblationEngine;
     tests: SelfTestRegistry;
     telemetry: TelemetryOrchestrator;
+    skillSynthesizer: SkillSynthesizer;
+    evolutionRitual: EvolutionRitual;
+    llm: LLMProvider | null;
+    llmFast: LLMProvider | null;
+    llmPremium: LLMProvider | null;
     agenticConfig: AgenticConfig;
     constructor(db: Kysely<any>, config: NOORMConfig);
     /**

package/dist/cjs/agentic/Cortex.js

@@ -30,6 +30,8 @@ const StrategicPlanner_js_1 = require("./improvement/StrategicPlanner.js");
 const AblationEngine_js_1 = require("./improvement/AblationEngine.js");
 const SelfTestRegistry_js_1 = require("./improvement/SelfTestRegistry.js");
 const TelemetryOrchestrator_js_1 = require("./telemetry/TelemetryOrchestrator.js");
+const SkillSynthesizer_js_1 = require("./improvement/SkillSynthesizer.js");
+const EvolutionRitual_js_1 = require("./improvement/EvolutionRitual.js");
 /**
  * Cortex is the unified facade for agentic operations.
  * It coordinates sessions, memory, reflection, and knowledge.
@@ -66,12 +68,20 @@ class Cortex {
     ablation;
     tests;
     telemetry;
+    skillSynthesizer;
+    evolutionRitual;
+    llm;
+    llmFast;
+    llmPremium;
     agenticConfig;
     constructor(db, config) {
         this.db = db;
         this.config = config;
         const agenticConfig = config.agentic || {};
         this.agenticConfig = agenticConfig;
+        this.llm = agenticConfig.llm || null;
+        this.llmFast = agenticConfig.llmFast || this.llm;
+        this.llmPremium = agenticConfig.llmPremium || this.llm;
         this.telemetry = new TelemetryOrchestrator_js_1.TelemetryOrchestrator(db, agenticConfig);
         this.sessions = new SessionManager_js_1.SessionManager(db, agenticConfig, this.telemetry);
         this.buffer = new ContextBuffer_js_1.ContextBuffer({ maxMessages: agenticConfig.contextWindowSize });
@@ -83,7 +93,7 @@ class Cortex {
         this.actions = new ActionJournal_js_1.ActionJournal(db, agenticConfig, this.telemetry);
         this.resources = new ResourceMonitor_js_1.ResourceMonitor(db, agenticConfig);
         this.episodes = new EpisodicMemory_js_1.EpisodicMemory(db, agenticConfig);
-        this.capabilities = new CapabilityManager_js_1.CapabilityManager(db, agenticConfig);
+        this.capabilities = new CapabilityManager_js_1.CapabilityManager(db, this, agenticConfig);
         this.policies = new PolicyEnforcer_js_1.PolicyEnforcer(db, agenticConfig);
         this.metrics = new SovereignMetrics_js_1.SovereignMetrics(db, agenticConfig);
         this.evolution = new SelfEvolution_js_1.SelfEvolution(db, config);
@@ -103,6 +113,8 @@ class Cortex {
         this.strategy = new StrategicPlanner_js_1.StrategicPlanner(db, this, agenticConfig);
         this.ablation = new AblationEngine_js_1.AblationEngine(db, this, agenticConfig);
         this.tests = new SelfTestRegistry_js_1.SelfTestRegistry(db, this, agenticConfig);
+        this.skillSynthesizer = new SkillSynthesizer_js_1.SkillSynthesizer(db, this, agenticConfig);
+        this.evolutionRitual = new EvolutionRitual_js_1.EvolutionRitual(db, this, agenticConfig);
     }
     /**
      * The "Soul-Searching" Loop: A top-level orchestration of all self-improvement rituals.
@@ -125,9 +137,13 @@ class Cortex {
             await this.ablation.monitorAblationPerformance();
             // 4. Mutation & Strategy
             await this.strategy.mutateStrategy();
-            // 5. Broadcast knowledge
+            // 5. High-Throughput Evolution Pulse
+            await this.evolutionRitual.execute();
+            // 6. Broadcast knowledge & skills
             await this.hive.broadcastKnowledge();
-            // 6. Evolutionary pulse
+            // 6b. Emergent Skill Synthesis
+            await this.skillSynthesizer.discoverAndSynthesize();
+            // 7. Evolutionary pulse
             await this.pilot.runSelfImprovementCycle();
             console.log('[Cortex] Soul-Searching loop completed.');
         }

package/dist/cjs/agentic/improvement/CortexJanitor.d.ts

@@ -30,7 +30,7 @@ export declare class CortexJanitor {
     pruneMetrics(daysToKeep?: number): Promise<number>;
     /**
      * Autonomous Indexing: Detects common query patterns and suggests missing indexes.
-     * Production Hardening: Uses introspection and usage patterns.
+     * Production Hardening: Uses introspection and usage patterns from metrics.
      */
     autonomousIndexing(): Promise<string[]>;
     /**

package/dist/cjs/agentic/improvement/CortexJanitor.js

@@ -83,39 +83,55 @@ class CortexJanitor {
     }
     /**
      * Autonomous Indexing: Detects common query patterns and suggests missing indexes.
-     * Production Hardening: Uses introspection and usage patterns.
+     * Production Hardening: Uses introspection and usage patterns from metrics.
      */
     async autonomousIndexing() {
         console.log('[CortexJanitor] Analyzing query patterns for autonomous indexing...');
         const applied = [];
-        // 1. Identify slow-moving tables or columns without indexes
-        // For now, we focus on the core agentic tables
-        const agenticTables = [
-            this.knowledgeTable,
-            this.config.memoriesTable || 'agent_memories',
-            this.config.messagesTable || 'agent_messages'
+        // 1. Structural Heuristics: Core Identity Indexes
+        const coreIdentityTables = [
+            { table: this.knowledgeTable, col: 'entity' },
+            { table: this.config.memoriesTable || 'agent_memories', col: 'entity' },
+            { table: this.config.messagesTable || 'agent_messages', col: 'session_id' }
         ];
-        for (const table of agenticTables) {
+        for (const target of coreIdentityTables) {
             try {
-                // Heuristic: Ensure entity column is indexed in knowledge/memories
-                if (table === this.knowledgeTable || table === (this.config.memoriesTable || 'agent_memories')) {
-                    const indexName = `idx_${table}_entity_v2`;
-                    await (0, sql_js_1.sql) `CREATE INDEX IF NOT EXISTS ${sql_js_1.sql.raw(indexName)} ON ${sql_js_1.sql.table(table)}(entity)`.execute(this.db);
-                    applied.push(`Ensured index ${indexName} exists`);
-                }
-                // Heuristic: Index status and confidence for high-volume filtering
-                if (table === this.knowledgeTable) {
-                    const indexName = `idx_${table}_status_conf`;
-                    await (0, sql_js_1.sql) `CREATE INDEX IF NOT EXISTS ${sql_js_1.sql.raw(indexName)} ON ${sql_js_1.sql.table(table)}(status, confidence)`.execute(this.db);
-                    applied.push(`Ensured index ${indexName} exists`);
-                }
+                const indexName = `idx_${target.table}_${target.col}_v2`;
+                await (0, sql_js_1.sql) `CREATE INDEX IF NOT EXISTS ${sql_js_1.sql.raw(indexName)} ON ${sql_js_1.sql.table(target.table)}(${sql_js_1.sql.raw(target.col)})`.execute(this.db);
+                applied.push(`Standardized identity index: ${indexName}`);
             }
             catch (err) {
-                console.warn(`[CortexJanitor] Failed to apply autonomous index to ${table}:`, err);
+                console.warn(`[CortexJanitor] Identity indexing failed for ${target.table}:`, err);
+            }
+        }
+        // 2. Data-Driven Heuristics: Hotspot Injection
+        // We look for tables mentioned in slow-query metrics recently
+        const slowQueries = await this.db
+            .selectFrom(this.metricsTable)
+            .select('metadata')
+            .where('metric_name', '=', 'query_latency')
+            .where('metric_value', '>', 300) // Slower than 300ms
+            .limit(50)
+            .execute();
+        const tablesToRemoveFriction = new Set();
+        for (const q of slowQueries) {
+            try {
+                const meta = typeof q.metadata === 'string' ? JSON.parse(q.metadata) : (q.metadata || {});
+                if (meta.table)
+                    tablesToRemoveFriction.add(meta.table);
+            }
+            catch (e) { /* ignore */ }
+        }
+        for (const table of tablesToRemoveFriction) {
+            // Hotspot logic: if it's high-traffic knowledge, index the status/confidence pair for promoting/demoting
+            if (table === this.knowledgeTable) {
+                const indexName = `idx_${table}_hotspot_lifecycle`;
+                await (0, sql_js_1.sql) `CREATE INDEX IF NOT EXISTS ${sql_js_1.sql.raw(indexName)} ON ${sql_js_1.sql.table(table)}(status, confidence)`.execute(this.db);
+                applied.push(`Injected hotspot index for lifecycle: ${indexName}`);
             }
         }
         if (applied.length > 0) {
-            await this.logRitual('optimization', 'success', { action: 'autonomous_indexing', applied });
+            await this.logRitual('optimization', 'success', { action: 'autonomous_indexing', applied_count: applied.length });
         }
         return applied;
     }

package/dist/cjs/agentic/improvement/CuriosityEngine.d.ts

@@ -12,7 +12,7 @@ export declare class CuriosityEngine {
     private get typedDb();
     /**
      * Analyze current knowledge to identify "Gaps" or contradictions.
-     * Detects entities with low confidence or competing high-confidence facts.
+     * Detects entities with low confidence, unverified status, or competing high-confidence facts.
      */
     identifyKnowledgeGaps(): Promise<{
         entity: string;
@@ -36,7 +36,7 @@ export declare class CuriosityEngine {
     suggestQuestions(entity: string): Promise<string[]>;
     /**
      * Generate "Relationship Hypotheses" between high-confidence entities.
-     * Suggests that entities with similar tags might be related.
+     * Suggests that entities with multi-tag overlaps might be related.
      */
     generateHypotheses(): Promise<string[]>;
     /**