nyxora 26.7.2-alpha.4 → 26.7.3

package/CHANGELOG.md

@@ -3,6 +3,17 @@
 All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepashangelog.com/en/1.0.0/),
+
+## [26.7.3]
+### Bug Fixes & Improvements
+- **Daemon Graceful Shutdown (Port 8000)**: Improved the `npm run stop` behavior by injecting a `forceKill` (`SIGKILL`) method within `launcher.ts`, explicitly terminating detached ML Engine processes (`uvicorn`) and `ts-node` instances that were previously hanging and preventing clean reboots.
+- **Telegram Connectivity Timeout**: Resolved a persistent `Network request for 'getUpdates' failed!` issue in the Telegram integration. Enforced `dns.setDefaultResultOrder('ipv4first')` in `cli.ts` to bypass dual-stack IPv6 conflicts and ensure stable grammatical API fetching.
+## [26.7.2-alpha.5]
+### Features & Architecture (Python ML Engine)
+- **Local Python ML Engine Integration**: Successfully integrated a local Python-based Machine Learning Engine (FastAPI + LangChain + Pandas) alongside the core Node.js gateway. This massively enhances Nyxora's analytical and cognitive capabilities.
+- **Cognitive Memory & RAG**: Shifted Persona Dialectic Reasoning and Episodic Memory Semantic Search (RAG) to the new Python Engine. Integrated `langchain_huggingface` using the local `all-MiniLM-L6-v2` embedding model for ultra-fast, offline vector processing without API costs.
+- **Market Intelligence Delegation**: Completely refactored `marketPlugin.ts` (Node.js) to delegate deep market analysis and momentum calculations directly to the Python ML Engine (`/web3/analyze`), significantly reducing redundant API calls and code overlap.
+
 ## [26.7.2-alpha.4]
 ### Features & Platform Integrations
 - **Telegram Native Streaming (Bot API 9.3+)**: Radically overhauled the Telegram bot integration to completely bypass the standard 1-second `editMessageText` API rate limit. The engine now natively implements the modern `sendMessageDraft` method, streaming ephemeral hardware-accelerated "Typing..." animations directly to the Telegram client at 100ms intervals. This fully resolves UI stuttering and achieves ultra-smooth, real-time typewriter effects matching premium web interfaces.

package/README.md

@@ -55,9 +55,13 @@ It operates under a **Zero-Trust, Defense-in-Depth Cryptographically Bound Human
 
 **💻 Core Technologies**
 <p align="center">
+  <a href="https://nodejs.org/"><img src="https://img.shields.io/badge/Node.js-43853D?style=for-the-badge&logo=node.js&logoColor=white" alt="Node.js"></a>
   <a href="https://react.dev/"><img src="https://img.shields.io/badge/React-20232A?style=for-the-badge&logo=react&logoColor=61DAFB" alt="React"></a>
   <a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/TypeScript-007ACC?style=for-the-badge&logo=typescript&logoColor=white" alt="TypeScript"></a>
   <a href="https://viem.sh/"><img src="https://img.shields.io/badge/Viem-1E1E2E?style=for-the-badge&logo=v&logoColor=white" alt="Viem"></a>
+  <a href="https://www.python.org/"><img src="https://img.shields.io/badge/Python-3776AB?style=for-the-badge&logo=python&logoColor=white" alt="Python"></a>
+  <a href="https://fastapi.tiangolo.com/"><img src="https://img.shields.io/badge/FastAPI-009688?style=for-the-badge&logo=fastapi&logoColor=white" alt="FastAPI"></a>
+  <a href="https://www.langchain.com/"><img src="https://img.shields.io/badge/LangChain-1C3C3C?style=for-the-badge&logo=langchain&logoColor=white" alt="LangChain"></a>
 </p>
 
 <br/>
@@ -68,7 +72,7 @@ It operates under a **Zero-Trust, Defense-in-Depth Cryptographically Bound Human
 
 ### Advanced Security Architecture
 *   **🛡️ On-Chain AI Kill-Switch**: Nyxora is governed by a Base Smart Contract (`NyxoraAgentRegistry`). Users have absolute cryptographic power to instantly paralyze the AI's on-chain execution if compromised, solving the Web3 AI safety dilemma. [Read more about our Base Architecture](https://nyxoraai.github.io/Nyxora/smart-contract)
-*   **3-Tier IPC Architecture**: Nyxora is split into isolated processes: **Core** (LLM Runtime), **Policy Engine** (Guardrails on port 3001), and **Signer Vault** (Isolated Key Manager on Unix Sockets).
+*   **4-Tier IPC Architecture**: Nyxora is split into isolated processes: **Core** (Node.js LLM Runtime), **ML Engine** (Python Cognitive Sidecar on port 8000), **Policy Engine** (Guardrails on port 3001), and **Signer Vault** (Isolated Key Manager on Unix Sockets).
 *   **DeFi & Market Configuration BYOK & UI Masking**: All aggregator, provider, and oracle API keys are strictly isolated via a Bring Your Own Keys (BYOK) architecture into heavily guarded `~/.nyxora/defi_keys.yaml` and `~/.nyxora/market_keys.yaml` files. The local web Dashboard masks these injected secrets using `***********` and `IS_SET` censorship, completely neutralizing malicious browser extensions from exfiltrating your keys.
 *   **Approval Replay Protection (Nonce Guard)**: Transactions requested by the AI are drafted as hashes and signed with a randomized 16-byte Nonce. The `/api/transactions/:id/approve` endpoint strictly enforces Nonce matching to completely eliminate double-spending and Replay Attacks.
 *   **Native Asset Parameter Tampering Protection**: The internal cryptographic HMAC signature rigorously binds `toAddress`, `txData`, and `valueWei`, rendering the system mathematically immune to Native Token (ETH/BNB) destination or amount hijacking via Indirect Prompt Injections.
@@ -117,14 +121,15 @@ It operates under a **Zero-Trust, Defense-in-Depth Cryptographically Bound Human
 
 ## 📐 Architecture Workflow
 
-The following diagram illustrates Nyxora's **3-Tier Monorepo Architecture**, showing the isolated communication channels (REST API and Unix Socket).
+The following diagram illustrates Nyxora's **4-Tier Hybrid Architecture**, showing the isolated communication channels (REST API and Unix Socket).
 
 ![Architecture Workflow](https://raw.githubusercontent.com/perasyudha/Nyxora/main/assets/architecture.svg)
 
-*Nyxora separates its duties into 3 independent layers for absolute security:*
-1. **🧠 Core (The AI Brain)**: The intelligent assistant that strategizes and plans transactions, but **never** holds your funds.
-2. **🛡️ Policy Engine (The Guard)**: The security guard that verifies the Brain's plans. If the AI attempts to send funds exceeding your set limits, this engine automatically blocks it.
-3. **🔒 Signer Vault (The Safe)**: The offline vault where your Private Keys **and highly sensitive 3rd-party tokens (e.g., Google Workspace OAuth)** are securely locked natively in your OS Keyring (GNOME Keyring / macOS Keychain / Windows Credential Manager). It only signs transactions after they pass all rigorous security checks.
+*Nyxora separates its duties into 4 independent layers for absolute security and cognitive depth:*
+1. **🧠 Core (The AI Brain)**: The Node.js intelligent assistant that strategizes and plans transactions, but **never** holds your funds.
+2. **🧬 ML Engine (Cognitive Sidecar)**: A local Python/FastAPI sidecar running LangChain and HuggingFace models for hyper-fast Semantic RAG memory and Pandas-based technical market analysis.
+3. **🛡️ Policy Engine (The Guard)**: The security guard that verifies the Brain's plans. If the AI attempts to send funds exceeding your set limits, this engine automatically blocks it.
+4. **🔒 Signer Vault (The Safe)**: The offline vault where your Private Keys **and highly sensitive 3rd-party tokens (e.g., Google Workspace OAuth)** are securely locked natively in your OS Keyring. It only signs transactions after they pass all rigorous security checks.
 
 ### Web3 Separation of Concerns (Zero-Trust Routing)
 Within the AI Brain, the Web3 codebase is strictly divided to prevent the LLM from hallucinating or maliciously manipulating low-level routing paths:
@@ -144,8 +149,11 @@ To dive deeper into the technical details of our Zero-Knowledge security archite
 
 ## 🚀 Quick Start & Installation
 
+### Prerequisites
+Nyxora requires **Node.js 18+** and **Python 3.10+** (for the ML Cognitive Engine) to be installed on your system.
+
 ### Option 1: One-Line Installation (Recommended)
-The fastest way to install Nyxora is via our smart installation wrapper. This script automatically prepares Node.js (if missing) and securely fetches the Nyxora daemon directly from the NPM Registry.
+The fastest way to install Nyxora is via our smart installation wrapper. This script automatically prepares Node.js (if missing) and securely fetches the Nyxora daemon directly from the NPM Registry. *(Note: You must have Python 3.10+ pre-installed on your system, as this script only handles Node.js dependencies).*
 
 **Linux & macOS:**
 ```bash
@@ -164,7 +172,8 @@ If you already have Node.js installed, you can natively install Nyxora globally
 # Install globally
 npm install -g nyxora
 
-# Run the interactive setup wizard (API Keys, Wallet, Telegram)
+# Run the interactive setup wizard 
+# (Automatically validates Node.js & Python 3.10+ requirements, configures API Keys, Wallet, and ML Environment)
 nyxora setup
 
 # Start the background daemon
@@ -187,10 +196,10 @@ npm install
 # 2. Build the Core, MCP Server, and Dashboard UI
 npm run build
 
-# 3. Interactive Setup Wizard
+# 3. Interactive Setup Wizard (Will also install Python ML requirements via pip)
 npm run setup
 
-# 4. Start the Application
+# 4. Start the Application (Spawns Node.js Core and Python FastAPI sidecar)
 npm start
 ```
 
@@ -233,11 +242,6 @@ For complete technical deep-dives into our Cryptographic Architecture, please vi
 
 ---
 
-**❤️ Support the Project**
-
-Building and maintaining a highly secure, zero-trust architecture takes significant time and resources. If you love what we are building, you can help us keep Nyxora open, secure, and constantly evolving by sending a coffee our way:
-- **EVM :** `0x18a30D5DB50D287dbA669c5672CD71246CC4c4c6`
-
 ---
 **License:** MIT License
 

package/bin/nyxora.mjs

@@ -78,7 +78,7 @@ async function start() {
   });
 
   child.unref();
-
+  
   if (child.pid) {
     fs.writeFileSync(pidFile, child.pid.toString());
     console.log(`Nyxora daemon started (PID: ${child.pid}).`);
@@ -92,20 +92,18 @@ async function stop(preserveTracker = false) {
     console.log(`Stopping Nyxora daemon (PID: ${pid})...`);
     try {
       process.kill(-pid, 'SIGTERM');
-      
-      // Wait for process to exit to avoid race condition with flushState
       let attempts = 0;
       while (isDaemonRunning(pid.toString()) && attempts < 20) {
         await new Promise(r => setTimeout(r, 100));
         attempts++;
       }
-      
       console.log('Nyxora stopped gracefully.');
     } catch (e) {
-      console.error('Failed to kill process:', e.message);
+      console.error('Failed to kill Nyxora process:', e.message);
     }
+
     try {
-      fs.unlinkSync(pidFile);
+      if (fs.existsSync(pidFile)) fs.unlinkSync(pidFile);
       if (!preserveTracker) {
         const trackerFile = path.join(appDir, 'run', 'tracker.json');
         if (fs.existsSync(trackerFile)) fs.unlinkSync(trackerFile);

package/dist/launcher.js

@@ -26,20 +26,28 @@ const env = {
     ...process.env,
     INTERNAL_AUTH_TOKEN,
     SIGNER_SOCKET_PATH: '/tmp/nyxora-signer.sock',
-    TS_NODE_CACHE: 'false'
+    TS_NODE_CACHE: 'false',
+    PYTHONUNBUFFERED: '1'
 };
-const spawnService = (name, command, args, env, inheritStdio = false) => {
+const spawnService = (name, command, args, env, inheritStdio = false, cwd) => {
     let child;
     let crashCount = 0;
     let crashWindowStart = Date.now();
     let isShuttingDown = false;
     const startProcess = () => {
-        child = (0, child_process_1.spawn)(command, args, { env, stdio: inheritStdio ? 'inherit' : 'pipe' });
+        const spawnOpts = { env, stdio: inheritStdio ? 'inherit' : 'pipe' };
+        if (cwd)
+            spawnOpts.cwd = cwd;
+        child = (0, child_process_1.spawn)(command, args, spawnOpts);
+        child.on('error', (err) => {
+            console.error(`[Launcher] Failed to spawn ${name}:`, err.message);
+            isShuttingDown = true; // Prevent retry loop if spawn fails
+        });
         if (!inheritStdio) {
             child.stdout?.on('data', (data) => process.stdout.write(`[${name}] ${data}`));
             child.stderr?.on('data', (data) => {
                 const msg = data.toString();
-                if (msg.toLowerCase().includes('warn')) {
+                if (msg.toLowerCase().includes('warn') || msg.toLowerCase().includes('info')) {
                     process.stderr.write(`[${name}] ${msg}`);
                 }
                 else {
@@ -98,6 +106,14 @@ const spawnService = (name, command, args, env, inheritStdio = false) => {
                 }
                 catch (e) { }
             }
+        },
+        forceKill: () => {
+            if (child && !child.killed && child.pid) {
+                try {
+                    process.kill(child.pid, 'SIGKILL');
+                }
+                catch (e) { }
+            }
         }
     };
 };
@@ -122,10 +138,23 @@ setTimeout(() => {
     const policy = spawnService('Policy', cmd, [...baseArgs, policyPath], env);
     children.push(policy);
     setTimeout(() => {
-        const corePath = path_1.default.join(__dirnameResolved, `packages/core/src/gateway/cli${ext}`);
-        const args = process.argv.slice(2);
-        const core = spawnService('Core', cmd, [...baseArgs, corePath, ...args], env, true);
-        children.push(core);
+        // Spawn ML Engine (Python Sidecar)
+        const pythonPath = path_1.default.join(process.env.HOME || process.env.USERPROFILE || '', '.nyxora', 'ml-engine', 'venv', 'bin', 'python');
+        if (fs_1.default.existsSync(pythonPath)) {
+            const mlDir = path_1.default.join(__dirnameResolved, 'packages', 'ml-engine');
+            const mlArgs = ['-m', 'uvicorn', 'main:app', '--host', '127.0.0.1', '--port', '8000'];
+            const mlEngine = spawnService('ML Engine', pythonPath, mlArgs, env, false, mlDir);
+            children.push(mlEngine);
+        }
+        else {
+            console.warn('[Launcher] Warning: Python virtual environment not found. Did you run setup?');
+        }
+        setTimeout(() => {
+            const corePath = path_1.default.join(__dirnameResolved, `packages/core/src/gateway/cli${ext}`);
+            const args = process.argv.slice(2);
+            const core = spawnService('Core', cmd, [...baseArgs, corePath, ...args], env, true);
+            children.push(core);
+        }, 1000);
     }, 1000);
 }, 1000);
 // Ensure all child processes are killed when launcher exits
@@ -138,8 +167,15 @@ const cleanup = () => {
     children.forEach(c => c.kill());
     // Give them a moment to cleanup
     setTimeout(() => {
+        children.forEach(c => {
+            try {
+                c.forceKill();
+            }
+            catch (e) { }
+        });
         try {
             require('child_process').execSync('pkill -f ts-node');
+            require('child_process').execSync('pkill -f uvicorn');
         }
         catch (e) { }
         process.exit(0);

package/dist/packages/core/src/agent/cronManager.js

@@ -44,7 +44,7 @@ const crypto_1 = require("crypto");
 const picocolors_1 = __importDefault(require("picocolors"));
 class CronManager {
     jobs = new Map();
-    addJob(expression, prompt) {
+    addJob(expression, prompt, sessionId) {
         const id = (0, crypto_1.randomUUID)();
         // Validate expression
         try {
@@ -59,7 +59,7 @@ class CronManager {
                 // Dynamically import processUserInput to avoid circular dependencies
                 const { processUserInput } = await Promise.resolve().then(() => __importStar(require('./reasoning')));
                 // Execute the prompt as a background system task
-                const response = await processUserInput(prompt, 'system', undefined, `cron-${id}`);
+                const response = await processUserInput(prompt, 'system', undefined, sessionId || `cron-${id}`);
                 // Push notification to Telegram if configured
                 const config = (0, parser_1.loadConfig)();
                 if (config.integrations?.telegram?.enabled && config.integrations?.telegram?.authorized_chat_id) {

package/dist/packages/core/src/agent/llmProvider.js

@@ -51,9 +51,13 @@ class OpenAIAdapter {
                 }
             };
         }
-        catch {
+        catch (e) {
             // Fallback to non-streaming if streaming fails
-            return this.chat(request);
+            const chatRes = await this.chat(request);
+            if (chatRes.message.content) {
+                onChunk(chatRes.message.content);
+            }
+            return chatRes;
         }
     }
 }

package/dist/packages/core/src/agent/nyxDaemon.js

@@ -4,8 +4,6 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
 };
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.nyxDaemon = exports.NyxDaemon = void 0;
-const parser_1 = require("../config/parser");
-const llmUtils_1 = require("../utils/llmUtils");
 const episodic_1 = require("../memory/episodic");
 const reasoning_1 = require("./reasoning");
 const picocolors_1 = __importDefault(require("picocolors"));
@@ -16,10 +14,10 @@ class NyxDaemon {
     start() {
         if (this.interval)
             return;
-        // Initial run after 5 minutes
+        // Initial run after 30 seconds (faster persona availability for new sessions)
         this.initialTimeout = setTimeout(() => {
             this.runAudit();
-        }, 5 * 60 * 1000);
+        }, 30 * 1000);
         // Audit memory every 30 minutes
         this.interval = setInterval(() => {
             this.runAudit();
@@ -57,45 +55,34 @@ class NyxDaemon {
                 this.isProcessing = false;
                 return;
             }
-            const config = (0, parser_1.loadConfig)();
-            const prompt = `You are Nyx, Nyxora's background Persona Auditor.
-Analyze the following recent conversation between the USER and Nyxora.
-Identify any persistent user traits, behavioral preferences, trading styles, AND language preferences.
-Output your findings AS A STRICT JSON ARRAY of strings. If no strong traits are found, output an empty array [].
-Examples of valid traits:
-- Behavior: "Prefers concise answers", "Aggressive trader", "Risk-averse", "Polite", "Often trades on Arbitrum"
-- Language: "Primarily speaks Indonesian", "Uses English for technical terms", "Communicates in casual/informal tone", "Mixes Indonesian and English (code-switching)"
-IMPORTANT: Always include a language preference trait if the user's language or communication style is identifiable.
-DO NOT output markdown, just the JSON array.`;
-            const messages = conversationOnly
-                .map((m) => `${m.role === 'user' ? 'USER' : 'NYXORA'}: ${m.content}`)
-                .join('\n');
-            const response = await (0, llmUtils_1.executeWithRetry)(async (client) => {
-                return await client.chat({
-                    model: config.llm.model,
-                    messages: [
-                        { role: 'system', content: prompt },
-                        { role: 'user', content: messages }
-                    ],
-                    temperature: 0.2
-                });
+            // Kirim riwayat percakapan ke Python ML Engine untuk diproses oleh LangChain
+            const res = await fetch('http://localhost:8000/cognitive/reason', {
+                method: 'POST',
+                headers: { 'Content-Type': 'application/json' },
+                body: JSON.stringify({ messages: conversationOnly })
             });
-            let content = response.message.content || '[]';
-            // Clean markdown if any
-            content = content.replace(/\`\`\`json/g, '').replace(/\`\`\`/g, '').trim();
+            if (!res.ok) {
+                throw new Error(`Python ML Engine returned ${res.status}: ${await res.text()}`);
+            }
+            const traits = await res.json();
             try {
-                const traits = JSON.parse(content);
-                if (Array.isArray(traits) && traits.length > 0) {
-                    for (const trait of traits) {
-                        // Fix C: Start confidence at 0.4 (not 0.8) so traits build up gradually across audits.
-                        // updatePersonaTrait has upsert logic: repeated traits gain +0.4 * 0.2 per audit cycle.
-                        episodic_1.episodicDB.updatePersonaTrait(trait, 0.4, 'nyx_daemon');
-                        console.log(picocolors_1.default.magenta(`[Nyx] Discovered new trait: ${trait}`));
+                if (traits && typeof traits === 'object' && !Array.isArray(traits)) {
+                    const categories = ['language', 'tone', 'trading_style', 'behavior'];
+                    for (const cat of categories) {
+                        const value = traits[cat];
+                        if (value && typeof value === 'string' && value.trim()) {
+                            // Category-based upsert: confidence accumulates correctly per category
+                            episodic_1.episodicDB.upsertPersonaByCategory(cat, value.trim(), 0.5, 'nyx_daemon');
+                            console.log(picocolors_1.default.magenta(`[Nyx] Updated persona [${cat}]: ${value.trim()}`));
+                        }
                     }
                 }
+                else {
+                    console.log(picocolors_1.default.magenta('[Nyx] No strong traits found in this audit cycle.'));
+                }
             }
             catch (e) {
-                console.error(picocolors_1.default.red('[Nyx] Failed to parse JSON traits'), content);
+                console.error(picocolors_1.default.red('[Nyx] Failed to process traits from Python'), traits);
             }
         }
         catch (e) {

package/dist/packages/core/src/agent/osAgent.js

@@ -6,12 +6,15 @@ Object.defineProperty(exports, "__esModule", { value: true });
 exports.logger = void 0;
 exports.processOsIntent = processOsIntent;
 exports.processOsIntentStream = processOsIntentStream;
+const fs_1 = __importDefault(require("fs"));
 const parser_1 = require("../config/parser");
 const logger_1 = require("../memory/logger");
 const tracker_1 = require("../gateway/tracker");
 const episodic_1 = require("../memory/episodic");
 const skillManager_1 = require("../utils/skillManager");
 const cognitiveManager_1 = require("../cognitive/cognitiveManager");
+const reasoningScratchpad_1 = require("./reasoningScratchpad");
+const contextSummarizer_1 = require("../utils/contextSummarizer");
 const EXECUTION_DISCIPLINE = `
 <tool_persistence>
 Use tools whenever they can increase the accuracy, completeness, or factual correctness of your response.
@@ -37,10 +40,11 @@ NEVER fabricate, hallucinate, or forge tool outputs.
 </task_completion>
 `;
 const registry_1 = require("../plugin/registry");
+const paths_1 = require("../config/paths");
 const picocolors_1 = __importDefault(require("picocolors"));
 exports.logger = new logger_1.Logger();
 const llmUtils_1 = require("../utils/llmUtils");
-function getSystemPrompt(context = 'os', userInput = '') {
+async function getSystemPrompt(context = 'os', userInput = '') {
     const config = (0, parser_1.loadConfig)();
     const currentDateTime = new Date().toLocaleString('en-US');
     let basePrompt = `You are Nyxora's OS Agent (System & Automation Specialist).
@@ -50,7 +54,7 @@ Reason internally. Never reveal private reasoning. Provide only concise conclusi
 
 [OS EXECUTION WORKFLOW]
 CRITICAL RULE 1: NEVER expose internal JSON tool calls. Explain the outcome naturally.
-CRITICAL RULE 2: STRICT LANGUAGE MATCHING. Reply in the exact same language as the user's LATEST prompt.
+CRITICAL RULE 2: STRICT LANGUAGE MATCHING. Reply in the exact same language as the user's LATEST prompt, UNLESS the Episodic Memories or Cognitive Skills specify a strict language preference.
 CRITICAL RULE 3: FILE SYSTEM SAFETY. You are STRICTLY FORBIDDEN from modifying config.yaml, rpc_key.yaml, or policy.yaml using terminal commands like sed or echo.
 CRITICAL RULE 4: CRON JOBS VS LIMIT ORDERS. Do NOT use schedule_task for price-based trading triggers. Use schedule_task for time-based recurring tasks.
 CRITICAL RULE 5: TOOL CONFIDENCE. NEVER fabricate file contents or command outputs.
@@ -62,46 +66,83 @@ ${EXECUTION_DISCIPLINE}
     if (activeSOP) {
         basePrompt += `\n\n[ACTIVE COGNITIVE SKILLS]\n${activeSOP}\n`;
     }
-    // Inject Episodic Memories
+    // Inject Episodic Memories via Python RAG
     try {
-        const recentMemories = episodic_1.episodicDB.getMemories().slice(0, 10);
-        if (recentMemories.length > 0) {
-            basePrompt += `
-
---- EPISODIC MEMORIES (SMART SUGGESTIONS) ---
-`;
-            recentMemories.forEach(mem => {
-                basePrompt += `- [${mem.category.toUpperCase()}] ${mem.fact} (Confidence: ${(mem.confidence * 100).toFixed(0)}%)
-`;
+        const ragRes = await fetch('http://localhost:8000/memory/rag', {
+            method: 'POST',
+            headers: { 'Content-Type': 'application/json' },
+            body: JSON.stringify({ query: userInput, top_k: 5 })
+        });
+        if (ragRes.ok) {
+            const ragData = await ragRes.json();
+            if (ragData.memories && ragData.memories.length > 0) {
+                basePrompt += `\n\n--- EPISODIC MEMORIES (SMART SUGGESTIONS) ---\n`;
+                ragData.memories.forEach((mem) => {
+                    basePrompt += `- ${mem}\n`;
+                });
+            }
+        }
+    }
+    catch (e) {
+        // Fallback or ignore if Python ML engine is down
+    }
+    // Inject User Information & Personas from user.md
+    try {
+        const userMdPath = (0, paths_1.getPath)('user.md');
+        if (fs_1.default.existsSync(userMdPath)) {
+            const userInstructions = fs_1.default.readFileSync(userMdPath, 'utf8');
+            basePrompt += `\n\n--- USER INFORMATION & PREFERENCES ---\n${userInstructions}\n`;
+        }
+    }
+    catch (e) {
+        // Ignore error
+    }
+    // HIGHEST PRIORITY: Inject observed user communication style (NyxDaemon)
+    try {
+        const strongPersonas = episodic_1.episodicDB.getStrongPersonas(0.5);
+        if (strongPersonas.length > 0) {
+            basePrompt += `\n\n--- ⚡ OVERRIDE: USER COMMUNICATION STYLE (HIGHEST PRIORITY — OVERRIDES ALL RULES ABOVE) ---\n`;
+            basePrompt += `CRITICAL: These are CONFIRMED user preferences observed across multiple conversations. You MUST apply them IMMEDIATELY and CONSISTENTLY, overriding any default tone or style instructions above.\n\n`;
+            strongPersonas.forEach(p => {
+                const label = p.category ? `[${p.category.toUpperCase()}]` : '[PREFERENCE]';
+                basePrompt += `${label} ${p.trait}\n`;
             });
         }
     }
-    catch { }
+    catch (e) {
+        // Ignore
+    }
     return basePrompt;
 }
 async function processOsIntent(input, role = 'user', onProgress, sessionId) {
     const config = (0, parser_1.loadConfig)();
     // Add input to memory
     exports.logger.addEntry({ role, content: input }, sessionId);
-    const history = exports.logger.getHistory(sessionId);
-    // Format messages for OpenAI
     let activeTools = [...registry_1.pluginManager.getAllToolDefinitions()];
     activeTools = activeTools.filter(t => (0, skillManager_1.isSkillActive)(t.function.name));
+    // P1: Init reasoning scratchpad for this request
+    const scratchpad = new reasoningScratchpad_1.ReasoningScratchpad();
+    // P3: Build system prompt ONCE per request — not per turn
+    const cachedSystemPrompt = await getSystemPrompt('os', input);
     const { sanitizeHistoryForLLM } = require('../utils/historySanitizer');
-    const sanitizedHistory = sanitizeHistoryForLLM(history, activeTools, config.llm.provider);
-    let messages = [
-        { role: 'system', content: getSystemPrompt('os', input) },
-        ...sanitizedHistory
-    ];
     try {
         let turnCount = 0;
         const MAX_TURNS = 10;
+        let consecutiveToolErrors = 0;
         while (turnCount < MAX_TURNS) {
             turnCount++;
             const currentHistory = exports.logger.getHistory(sessionId);
-            const sanitizedHistory = sanitizeHistoryForLLM(currentHistory, activeTools, config.llm.provider);
+            // P6: Compress history if conversation is too long
+            const historyToUse = (0, contextSummarizer_1.needsCompression)(currentHistory)
+                ? await (0, contextSummarizer_1.compressHistory)(currentHistory)
+                : currentHistory;
+            const sanitizedHistory = sanitizeHistoryForLLM(historyToUse, activeTools, config.llm.provider);
+            // P1: Inject scratchpad into system prompt for turns > 1
+            const sysPrompt = turnCount === 1
+                ? cachedSystemPrompt
+                : cachedSystemPrompt + scratchpad.getInjection();
             const messages = [
-                { role: 'system', content: getSystemPrompt('os', input) },
+                { role: 'system', content: sysPrompt },
                 ...sanitizedHistory
             ];
             const response = await (0, llmUtils_1.executeWithRetry)(async (client) => {
@@ -120,16 +161,15 @@ async function processOsIntent(input, role = 'user', onProgress, sessionId) {
                 tracker_1.Tracker.addTokens(response.usage.total_tokens, config.llm.provider);
             }
             tracker_1.Tracker.addEvent('llm.response', { provider: config.llm.provider, tool_calls: responseMessage.tool_calls?.length || 0 });
+            // P1: Capture <think> blocks for scratchpad, get clean content
+            const cleanedContent = scratchpad.capture(responseMessage.content || '', turnCount);
             exports.logger.addEntry({
                 role: 'assistant',
-                content: responseMessage.content || "",
+                content: cleanedContent || '',
                 tool_calls: responseMessage.tool_calls,
             }, sessionId);
             if (!responseMessage.tool_calls || responseMessage.tool_calls.length === 0) {
-                let finalContent = responseMessage.content || "No response generated.";
-                finalContent = finalContent.replace(/<(think|thought|thinking|reasoning|analysis|reflection)>[\s\S]*?<\/\1>\n?/gi, '');
-                finalContent = finalContent.trim();
-                return finalContent;
+                return cleanedContent || 'No response generated.';
             }
             let canFastReturnAll = true;
             let accumulatedResults = [];
@@ -206,8 +246,26 @@ async function processOsIntent(input, role = 'user', onProgress, sessionId) {
                     canFastReturnAll = false;
                 }
             }
-            // V2 Optimization (Expanded in v1.7.4): Zero-LLM Fast Return for data-heavy and read-only tools
-            // If all tools already return perfectly formatted markdown, skip the second LLM call to save 5-10s latency!
+            // P4: Self-reflection — if ALL tools failed, inject reflection before next turn
+            const allFailed = accumulatedResults.length > 0 && accumulatedResults.every(r => r.startsWith('Error') || r.includes('[System Error]') || r.includes('[Security Blocked]'));
+            if (allFailed) {
+                consecutiveToolErrors++;
+                const reflection = `[SELF-REFLECTION] All ${accumulatedResults.length} tool call(s) failed (attempt ${consecutiveToolErrors}). ` +
+                    `Errors: ${accumulatedResults.join(' | ')}. ` +
+                    `Analyze WHY each failed. Options: (1) retry with corrected params, (2) use alternative tool, (3) inform user clearly. ` +
+                    `Do NOT repeat the exact same failed call.`;
+                exports.logger.addEntry({ role: 'system', content: reflection }, sessionId);
+                console.log(picocolors_1.default.magenta(`[Self-Reflection] Turn ${turnCount}: all tools failed, injecting reflection.`));
+                if (consecutiveToolErrors >= 2) {
+                    const errorSummary = `⚠️ Unable to complete this request after multiple attempts.\n${accumulatedResults.join('\n')}`;
+                    exports.logger.addEntry({ role: 'assistant', content: errorSummary }, sessionId);
+                    return errorSummary;
+                }
+            }
+            else {
+                consecutiveToolErrors = 0;
+            }
+            // V2 Optimization: Zero-LLM Fast Return for transaction tools
             if (canFastReturnAll && accumulatedResults.length > 0) {
                 const finalContent = accumulatedResults.join('\n\n---\n\n');
                 exports.logger.addEntry({ role: 'assistant', content: finalContent }, sessionId);
@@ -245,7 +303,7 @@ async function processOsIntentStream(input, onChunk, onProgress, sessionId) {
             const currentHistory = exports.logger.getHistory(sessionId);
             const sanitizedHistory = sanitizeHistoryForLLM(currentHistory, activeTools, config.llm.provider);
             const messages = [
-                { role: 'system', content: getSystemPrompt('os', input) },
+                { role: 'system', content: await getSystemPrompt('os', input) },
                 ...sanitizedHistory
             ];
             let streamedContent = '';