@senoldogann/context-manager 0.1.20 → 0.1.24

package/README.md

@@ -2,81 +2,189 @@
 
 > 🧠 The Neural Backbone for Autonomous AI Agents
 
-This is the Node.js wrapper for the **Cognitive Codebase Matrix (CCM)**. It allows you to run the CCM CLI and MCP server without manually installing Rust or building from source.
+**Node.js wrapper for Cognitive Codebase Matrix (CCM)** - Enables AI agents to understand and navigate your codebase with surgical precision.
+
+[![npm](https://img.shields.io/npm/v/@senoldogann/context-manager?color=orange)](https://www.npmjs.com/package/@senoldogann/context-manager)
+[![MCP](https://img.shields.io/badge/MCP-Compatible-blue)](https://modelcontextprotocol.io/)
+[![Rust](https://img.shields.io/badge/Built%20With-Rust-orange.svg)](https://www.rust-lang.org/)
+
+---
 
 ## 🚀 Quick Start
 
-> [!IMPORTANT]
-> **Prerequisite:** This tool relies on local AI models. You must have **[Ollama](https://ollama.com)** installed and running.
-> CCM will **automatically pull** the required embedding model (`nomic-embed-text`) if missing, but the Ollama engine itself is required.
+### Prerequisites
+
+1. **Node.js** 16+ installed
+2. **Ollama** installed and running (for local embeddings)
+   - Download: https://ollama.com
+   - Pull required model: `ollama pull mxbai-embed-large`
 
-### 🔒 Privacy by Default
-CCM uses a **Local-First** architecture by default. This means:
-*   Your code is **never** sent to 3rd party servers (OpenAI, Anthropic, etc.).
-*   All vector operations (Embedding) happen on your local machine.
-*   You can safely use it for internal or confidential projects.
+### Installation
 
-The easiest way to get started. This will automatically add CCM to your Claude or Antigravity configuration:
 ```bash
+# 1. Install and configure for Codex, Cursor, Claude Desktop, Antigravity, etc.
 npx @senoldogann/context-manager install
+
+# 2. Index your project
+npx @senoldogann/context-manager index --path .
 ```
 
-### 2. Index your Project (Automatic)
-CCM now implements **Lazy Indexing**. You don't actually need to run this command; the MCP server will automatically index your project the first time you or your AI agent run a query.
+**That's it!** Restart your AI editor and start asking questions about your code.
+
+---
+
+## 📖 What is CCM?
+
+CCM transforms static source code into a dynamic, queryable Knowledge Graph:
+
+- **🔍 Semantic Search** - Find code by meaning ("where is auth logic?")
+- **🧠 Graph Navigation** - Understand relationships ("who calls this function?")
+- **📍 Cursor Context** - Get relevant code based on your position
+
+---
+
+## 🔧 Commands
+
+The npm wrapper downloads pre-built binaries and passes commands through:
+
+| Command | Description |
+|---------|-------------|
+| `npx @senoldogann/context-manager install` | Auto-configure MCP for editors |
+| `npx @senoldogann/context-manager index --path <dir>` | Index a project |
+| `npx @senoldogann/context-manager query --text "..."` | Search codebase |
+| `npx @senoldogann/context-manager mcp` | Run MCP server directly |
+| `npx @senoldogann/context-manager eval --tasks <file>` | Run evaluation tasks |
+
+### Options
 
-If you still want to index manually:
 ```bash
-npx @senoldogann/context-manager index --path .
-```
+# Watch mode - auto-reindex on file changes
+npx @senoldogann/context-manager index --path . --watch
 
-## ⚒️ Manual Configuration (Alternative)
-
-If you prefer to configure your AI editor manually without the `install` command, add this to your `mcp_config.json`:
-
-```json
-{
-  "mcpServers": {
-    "context-manager": {
-      "command": "npx",
-      "args": ["-y", "@senoldogann/context-manager", "mcp"],
-      "env": {
-        "RUST_LOG": "info"
-      }
-    }
-  }
-}
+# Custom database path
+npx @senoldogann/context-manager index --path . --db-path /custom/path
 ```
 
-## 💡 Pro-Tip: Enforcing CCM Usage
-To ensure your AI agent (Claude, Cursor, etc.) always uses CCM for deep analysis, add this to your **Custom Instructions** or **System Prompt**:
+---
 
-> "You are an expert architect. For any question about the codebase, DO NOT guess. Use the `context-manager` tools to explore the Graph and Vector store. Always prioritize `search_code` to find entry points and `read_graph` to navigate dependencies before proposing any code changes."
+## 🔒 Privacy by Default
+
+CCM uses a **Local-First** architecture:
+
+- ✅ Your code **never** leaves your machine
+- ✅ All embeddings run locally via Ollama
+- ✅ No external API calls (unless you configure OpenAI)
 
 ---
 
-## 🇹🇷 Türkçe Özet
+## ⚙️ Configuration
 
-Bu paket, **Cognitive Codebase Matrix (CCM)** için Node.js wrapper'ıdır. Rust kurulumuna gerek kalmadan CCM araçlarını kullanmanızı sağlar.
+### Environment Variables
 
-**Hızlı Kurulum:**
-```bash
-npx @senoldogann/context-manager install
-npx @senoldogann/context-manager index --path .
+Create `~/.ccm/.env`:
+
+```ini
+# Local (Recommended)
+EMBEDDING_PROVIDER=ollama
+EMBEDDING_HOST=http://127.0.0.1:11434
+EMBEDDING_MODEL=mxbai-embed-large
+
+# Cloud (Optional)
+EMBEDDING_PROVIDER=openai
+EMBEDDING_API_KEY=sk-your-key
+EMBEDDING_MODEL=text-embedding-3-small
+
+# Networking & Limits
+EMBEDDING_TIMEOUT_SECS=30
+CCM_MAX_FILE_BYTES=2097152
+
+# MCP Security
+CCM_ALLOWED_ROOTS=/Users/you/projects:/Users/you/sandbox
+CCM_REQUIRE_ALLOWED_ROOTS=0
+
+# MCP Runtime
+CCM_MCP_ENGINE_CACHE_SIZE=8
+CCM_MCP_DEBUG=0
+
+# Optional: disable embeddings entirely (semantic search disabled)
+CCM_DISABLE_EMBEDDER=0
+
+# Optional: embed data files (md/json/yaml) into vector search
+CCM_EMBED_DATA_FILES=0
+
+# Binary checksum verification (0 = enforce, 1 = bypass)
+CCM_ALLOW_UNVERIFIED_BINARIES=0
 ```
 
+**Production Tip:** Set `CCM_ALLOWED_ROOTS` and enable `CCM_REQUIRE_ALLOWED_ROOTS=1` to restrict MCP access.
+
 ---
 
-## 📦 What this package does
-This package is a lightweight wrapper that:
-1. Detects your OS and CPU architecture.
-2. Downloads the pre-built Rust binaries from GitHub Releases if not already present.
-3. Automatically manages your global index and persistence in `~/.ccm`.
+## 🤖 Usage in AI
+
+Once configured, ask your AI agent:
+
+> "Search for the authentication flow in this codebase."
 
-For more details, visit the [Main Repository](https://github.com/senoldogann/LLM-Context-Manager).
+> "Read the graph for `UserService` and show me its callers."
+
+> "What functions call `parse_config`?"
+
+---
 
-### 🆕 v0.1.15 Updates
-*   **Lazy Indexing:** Automatic project indexing on first query. No background hogging.
-*   **Universal Support:** Falls back to generic text indexing for ALL file types.
-*   **Zero-Config:** Improved project root detection and cross-platform binary handling.
+## 📦 For Developers
+
+This package handles:
+
+1. OS/architecture detection
+2. Binary download from GitHub Releases
+3. Global persistence in `~/.ccm`
+
+### ✅ Binary Integrity
+
+Downloads are verified against `checksums.txt` from the GitHub Release.  
+If the manifest is missing or a mismatch occurs, you can set `CCM_ALLOW_UNVERIFIED_BINARIES=1` to bypass verification (not recommended).
+
+### 📄 Data Files in Search
+
+By default, data files (`.md`, `.json`, `.yaml`) are indexed but not embedded.  
+Enable `CCM_EMBED_DATA_FILES=1` to include them in semantic search.
+
+**Source:** https://github.com/senoldogann/LLM-Context-Manager
+
+---
+
+## 📝 Changelog
+
+### v0.1.24
+- ✅ Native `protoc` installation in GitHub Actions release builds
+- ✅ Updated release workflow actions for newer runner compatibility
+
+### v0.1.23
+- ✅ JSON-RPC request size limit for safer MCP stdio transport
+- ✅ Sensitive value redaction in MCP debug logs
+- ✅ Hardened path normalization for MCP graph and context tools
+- ✅ GitHub release redirect allowlist for binary downloads
+- ✅ Unused core dependency cleanup
+
+### v0.1.22
+- ✅ Codex installer support via `codex mcp add`
+- ✅ Cursor config support via `~/.cursor/mcp.json`
+- ✅ Evaluation bootstraps missing indexes before scoring
+- ✅ Golden tasks refreshed for the current repo layout
+
+### v0.1.21
+- ✅ Release checksums (`checksums.txt`) for binary integrity
+- ✅ MCP allowlist with optional strict enforcement
+- ✅ Data file embedding is optional (`CCM_EMBED_DATA_FILES`)
+- ✅ CLI/MCP integration tests
+
+### v0.1.20
+- ✅ 100% evaluation pass rate
+- ✅ Hybrid scoring (graph + semantic)
+- ✅ Lazy indexing support
+- ✅ Watch mode for auto-reindexing
+
+---
 
-Built with ❤️ in **Rust**.
+Built with ❤️ in **Rust**

package/bin/ccm.js

@@ -1,39 +1,59 @@
 #!/usr/bin/env node
 
-const { spawn } = require('child_process');
+const { spawn, spawnSync } = require('child_process');
 const path = require('path');
 const fs = require('fs');
 const os = require('os');
 const https = require('https');
+const crypto = require('crypto');
 
-const VERSION = "0.1.20";
+const VERSION = "0.1.24";
 const REPO = 'senoldogann/LLM-Context-Manager';
 const BIN_DIR = path.join(os.homedir(), '.ccm', 'bin');
+const CHECKSUMS_FILE = 'checksums.txt';
+let checksumCache = null;
+
+const MCP_SERVER_NAME = 'context-manager';
+const MCP_COMMAND = 'npx';
+const MCP_ARGS = ['-y', '@senoldogann/context-manager', 'mcp'];
+const MCP_ENV = {
+    RUST_LOG: 'info'
+};
+const ALLOWED_REDIRECT_HOSTS = new Set([
+    'github.com',
+    'objects.githubusercontent.com',
+    'release-assets.githubusercontent.com'
+]);
+
+function allowUnverifiedBinaries() {
+    const raw = process.env.CCM_ALLOW_UNVERIFIED_BINARIES || process.env.CCM_SKIP_CHECKSUM || '';
+    return ['1', 'true', 'yes'].includes(raw.toLowerCase());
+}
 
 async function installMcp() {
-    const configPaths = [];
     const home = os.homedir();
+    const jsonTargets = [];
 
     if (os.platform() === 'darwin') {
-        configPaths.push(path.join(home, 'Library', 'Application Support', 'Claude', 'claude_desktop_config.json'));
-        configPaths.push(path.join(home, '.gemini', 'antigravity', 'mcp_config.json'));
-        configPaths.push(path.join(home, 'Library', 'Application Support', 'Code', 'User', 'globalStorage', 'saoudrizwan.claude-dev', 'settings', 'cline_mcp_settings.json'));
-        configPaths.push(path.join(home, 'Library', 'Application Support', 'Code', 'User', 'globalStorage', 'rooveterinaryinc.roo-cline', 'settings', 'cline_mcp_settings.json'));
+        jsonTargets.push(path.join(home, 'Library', 'Application Support', 'Claude', 'claude_desktop_config.json'));
+        jsonTargets.push(path.join(home, '.gemini', 'antigravity', 'mcp_config.json'));
+        jsonTargets.push(path.join(home, 'Library', 'Application Support', 'Code', 'User', 'globalStorage', 'saoudrizwan.claude-dev', 'settings', 'cline_mcp_settings.json'));
+        jsonTargets.push(path.join(home, 'Library', 'Application Support', 'Code', 'User', 'globalStorage', 'rooveterinaryinc.roo-cline', 'settings', 'cline_mcp_settings.json'));
     } else if (os.platform() === 'win32') {
         const appData = process.env.APPDATA || '';
-        configPaths.push(path.join(appData, 'Claude', 'claude_desktop_config.json'));
-        configPaths.push(path.join(process.env.USERPROFILE || '', 'AppData', 'Roaming', 'Code', 'User', 'globalStorage', 'saoudrizwan.claude-dev', 'settings', 'cline_mcp_settings.json'));
+        jsonTargets.push(path.join(appData, 'Claude', 'claude_desktop_config.json'));
+        jsonTargets.push(path.join(process.env.USERPROFILE || '', 'AppData', 'Roaming', 'Code', 'User', 'globalStorage', 'saoudrizwan.claude-dev', 'settings', 'cline_mcp_settings.json'));
     } else if (os.platform() === 'linux') {
-        configPaths.push(path.join(home, '.config', 'Claude', 'claude_desktop_config.json'));
-        configPaths.push(path.join(home, '.config', 'Code', 'User', 'globalStorage', 'saoudrizwan.claude-dev', 'settings', 'cline_mcp_settings.json'));
+        jsonTargets.push(path.join(home, '.config', 'Claude', 'claude_desktop_config.json'));
+        jsonTargets.push(path.join(home, '.config', 'Code', 'User', 'globalStorage', 'saoudrizwan.claude-dev', 'settings', 'cline_mcp_settings.json'));
     }
 
+    jsonTargets.push(path.join(home, '.cursor', 'mcp.json'));
+
     const mcpConfig = {
-        "command": "npx",
-        "args": ["-y", "@senoldogann/context-manager", "mcp"],
-        "env": {
-            "RUST_LOG": "info"
-        }
+        command: MCP_COMMAND,
+        args: MCP_ARGS,
+        env: MCP_ENV
     };
 
     console.log("[CCM] Pre-downloading binaries for all tools...");
@@ -41,38 +61,104 @@ async function installMcp() {
     await getBinaryFor('ccm-mcp');
 
     let installedCount = 0;
-    for (const configPath of configPaths) {
-        const dir = path.dirname(configPath);
-        if (fs.existsSync(dir)) {
-            let config = { mcpServers: {} };
-            if (fs.existsSync(configPath)) {
-                try {
-                    const content = fs.readFileSync(configPath, 'utf8');
-                    config = JSON.parse(content);
-                    fs.copyFileSync(configPath, `${configPath}.bak`);
-                } catch (e) {
-                    console.warn(`[CCM] Could not parse ${configPath}, creating backup and starting fresh section.`);
-                }
-            }
-
-            if (!config.mcpServers) config.mcpServers = {};
-            config.mcpServers["context-manager"] = mcpConfig;
 
-            fs.writeFileSync(configPath, JSON.stringify(config, null, 2));
-            console.log(`[CCM] ✓ Successfully updated: ${configPath}`);
+    for (const configPath of jsonTargets) {
+        if (installJsonConfig(configPath, mcpConfig)) {
             installedCount++;
         }
     }
 
+    if (installCodexConfig()) {
+        installedCount++;
+    }
+
     if (installedCount === 0) {
         console.log("[CCM] No supported MCP config directories found.");
-        console.log("[CCM] Please add this to your mcp_config.json manually:");
-        console.log(JSON.stringify({ "context-manager": mcpConfig }, null, 2));
+        console.log("[CCM] Add this server manually:");
+        console.log(JSON.stringify({ [MCP_SERVER_NAME]: mcpConfig }, null, 2));
+        console.log("[CCM] Codex example:");
+        console.log(`codex mcp add ${MCP_SERVER_NAME} --env RUST_LOG=info -- ${MCP_COMMAND} ${MCP_ARGS.join(' ')}`);
     } else {
         console.log("[CCM] Installation complete! Restart your AI editor to see the changes.");
     }
 }
 
+function installJsonConfig(configPath, mcpConfig) {
+    const dir = path.dirname(configPath);
+
+    if (!fs.existsSync(dir)) {
+        return false;
+    }
+
+    let config = { mcpServers: {} };
+    if (fs.existsSync(configPath)) {
+        try {
+            const content = fs.readFileSync(configPath, 'utf8');
+            config = JSON.parse(content);
+            fs.copyFileSync(configPath, `${configPath}.bak`);
+        } catch (e) {
+            console.warn(`[CCM] Could not parse ${configPath}, creating backup and starting fresh section.`);
+        }
+    }
+
+    if (!config.mcpServers || typeof config.mcpServers !== 'object') {
+        config.mcpServers = {};
+    }
+
+    config.mcpServers[MCP_SERVER_NAME] = mcpConfig;
+    fs.writeFileSync(configPath, JSON.stringify(config, null, 2));
+    console.log(`[CCM] ✓ Successfully updated: ${configPath}`);
+    return true;
+}
+
+function installCodexConfig() {
+    const listResult = spawnSync('codex', ['mcp', 'list', '--json'], {
+        encoding: 'utf8'
+    });
+
+    if (listResult.error) {
+        return false;
+    }
+
+    if (listResult.status !== 0) {
+        console.warn(`[CCM] Codex MCP inspection failed: ${listResult.stderr.trim()}`);
+        return false;
+    }
+
+    let existingServers = [];
+    try {
+        existingServers = JSON.parse(listResult.stdout);
+    } catch (error) {
+        console.warn('[CCM] Could not parse Codex MCP list output.');
+        return false;
+    }
+
+    const existing = existingServers.find((server) => server.name === MCP_SERVER_NAME);
+    if (existing) {
+        const removeResult = spawnSync('codex', ['mcp', 'remove', MCP_SERVER_NAME], {
+            encoding: 'utf8'
+        });
+
+        if (removeResult.status !== 0) {
+            console.warn(`[CCM] Codex MCP removal failed: ${removeResult.stderr.trim()}`);
+            return false;
+        }
+    }
+
+    const addArgs = ['mcp', 'add', MCP_SERVER_NAME, '--env', 'RUST_LOG=info', '--', MCP_COMMAND, ...MCP_ARGS];
+    const addResult = spawnSync('codex', addArgs, {
+        encoding: 'utf8'
+    });
+
+    if (addResult.status !== 0) {
+        console.warn(`[CCM] Codex MCP install failed: ${addResult.stderr.trim()}`);
+        return false;
+    }
+
+    console.log('[CCM] ✓ Successfully updated: ~/.codex/config.toml');
+    return true;
+}
+
 async function getBinaryFor(commandName) {
     const platform = os.platform();
     const arch = os.arch();
@@ -88,6 +174,7 @@ async function getBinaryFor(commandName) {
 
     const binFilename = `${commandName}-v${VERSION}-${target}`;
     const binPath = path.join(BIN_DIR, binFilename);
+    const remoteFilename = `${commandName}-${target}`;
 
     // If file exists, ensure it is executable
     if (fs.existsSync(binPath)) {
@@ -110,6 +197,7 @@ async function getBinaryFor(commandName) {
 
     try {
         await downloadFile(url, tmpPath);
+        await verifyChecksum(tmpPath, [remoteFilename, binFilename]);
         fs.chmodSync(tmpPath, '755');
         fs.renameSync(tmpPath, binPath);
     } catch (err) {
@@ -142,7 +230,12 @@ function downloadFile(url, dest) {
     return new Promise((resolve, reject) => {
         https.get(url, (response) => {
             if (response.statusCode === 302 || response.statusCode === 301) {
-                return downloadFile(response.headers.location, dest).then(resolve).catch(reject);
+                try {
+                    const redirectUrl = resolveRedirectUrl(url, response.headers.location);
+                    return downloadFile(redirectUrl, dest).then(resolve).catch(reject);
+                } catch (error) {
+                    return reject(error);
+                }
             }
             if (response.statusCode !== 200) {
                 return reject(new Error(`Failed to download: ${response.statusCode}`));
@@ -163,6 +256,112 @@ function downloadFile(url, dest) {
     });
 }
 
+function downloadText(url) {
+    return new Promise((resolve, reject) => {
+        https.get(url, (response) => {
+            if (response.statusCode === 302 || response.statusCode === 301) {
+                try {
+                    const redirectUrl = resolveRedirectUrl(url, response.headers.location);
+                    return downloadText(redirectUrl).then(resolve).catch(reject);
+                } catch (error) {
+                    return reject(error);
+                }
+            }
+            if (response.statusCode !== 200) {
+                return reject(new Error(`Failed to download: ${response.statusCode}`));
+            }
+            let data = '';
+            response.setEncoding('utf8');
+            response.on('data', (chunk) => {
+                data += chunk;
+            });
+            response.on('end', () => resolve(data));
+        }).on('error', reject);
+    });
+}
+
+async function getChecksums() {
+    if (checksumCache) return checksumCache;
+
+    const url = `https://github.com/${REPO}/releases/download/v${VERSION}/${CHECKSUMS_FILE}`;
+    try {
+        const text = await downloadText(url);
+        checksumCache = parseChecksums(text);
+        return checksumCache;
+    } catch (err) {
+        return null;
+    }
+}
+
+function resolveRedirectUrl(sourceUrl, location) {
+    if (!location) {
+        throw new Error('Redirect response did not include a Location header');
+    }
+
+    const resolved = new URL(location, sourceUrl);
+    if (resolved.protocol !== 'https:') {
+        throw new Error(`Blocked redirect to non-HTTPS URL: ${resolved.href}`);
+    }
+    if (!ALLOWED_REDIRECT_HOSTS.has(resolved.hostname)) {
+        throw new Error(`Blocked redirect to unexpected host: ${resolved.hostname}`);
+    }
+
+    return resolved.toString();
+}
+
+function parseChecksums(text) {
+    const map = new Map();
+    for (const line of text.split('\n')) {
+        const trimmed = line.trim();
+        if (!trimmed) continue;
+        const parts = trimmed.split(/\s+/);
+        if (parts.length < 2) continue;
+        const hash = parts[0].toLowerCase();
+        const filename = parts[parts.length - 1].replace(/^\*/, '');
+        map.set(filename, hash);
+    }
+    return map;
+}
+
+function sha256File(filePath) {
+    return new Promise((resolve, reject) => {
+        const hash = crypto.createHash('sha256');
+        const stream = fs.createReadStream(filePath);
+        stream.on('error', reject);
+        stream.on('data', (data) => hash.update(data));
+        stream.on('end', () => resolve(hash.digest('hex')));
+    });
+}
+
+async function verifyChecksum(filePath, candidates) {
+    const checksums = await getChecksums();
+    if (!checksums) {
+        if (allowUnverifiedBinaries()) {
+            console.warn('[CCM] Checksum manifest not found. Proceeding without verification.');
+            return;
+        }
+        throw new Error('Checksum manifest not found. Set CCM_ALLOW_UNVERIFIED_BINARIES=1 to bypass.');
+    }
+
+    const expected = candidates
+        .map((name) => [name, `${name}.exe`])
+        .flat()
+        .map((name) => checksums.get(name))
+        .find(Boolean);
+    if (!expected) {
+        if (allowUnverifiedBinaries()) {
+            console.warn('[CCM] Checksum not found for binary. Proceeding without verification.');
+            return;
+        }
+        throw new Error('Checksum not found for binary. Set CCM_ALLOW_UNVERIFIED_BINARIES=1 to bypass.');
+    }
+
+    const actual = await sha256File(filePath);
+    if (actual !== expected) {
+        throw new Error(`Checksum mismatch. Expected ${expected}, got ${actual}`);
+    }
+}
+
 async function main() {
     try {
         const binPath = await getBinary();

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "@senoldogann/context-manager",
-    "version": "0.1.20",
+    "version": "0.1.24",
     "description": "LLM Context Manager MCP Server & CLI wrapper using npx",
     "main": "bin/ccm.js",
     "bin": {
@@ -20,10 +20,8 @@
     ],
     "author": "dogan",
     "license": "MIT",
-    "dependencies": {
-        "node-fetch": "^3.3.1"
-    },
+    "dependencies": {},
     "engines": {
         "node": ">=16.0.0"
     }
-}
+}