scai 0.1.19 → 0.1.21

package/README.md

@@ -1,150 +1,175 @@
 # ⚙️ scai — Smart Commit AI ✨
 
-> AI-powered CLI tools for smart commit messages, code comments, summaries, and changelogs — all powered by local models.
+> AI-powered CLI tool for smart commit messages, code comments, summaries, test generation, and changelogs — all powered by local models.
 
-**scai** (Smart Commit AI) is a lightweight, privacy-focused CLI tool that uses local AI models (via [Ollama](https://ollama.com)) to help developers work faster and cleaner:
+**scai** (Smart Commit AI) is a lightweight, privacy-focused CLI tool that helps developers work faster and cleaner:
 
 - 💬 Suggest high-quality Git commit messages  
 - ✨ Automatically comment your code  
 - 🧠 Summarize code files instantly in the terminal  
+- 🧪 Generate test files for your code (JavaScript/TypeScript)  
 - 📝 Generate changelog entries based on Git diffs  
-- 🔒 100% local — no API keys, no cloud, no telemetry  
+- 🔒 100% local — no API keys, no cloud, no telemetry
 
 ---
 
 ## 🚀 Features
 
-- ⚡️ Powered by local Ollama models like `llama3` and `codellama`
-- 🛠️ CLI built with Node.js + TypeScript
-- 🔒 No external services, full privacy by design  
-- ✅ Global options for model and language selection
+- ⚡️ Powered by local AI models (e.g., `llama3`, `codellama`)  
+- 🛠️ Built with Node.js and TypeScript  
+- 🔒 No external services or APIs required  
+- ✅ Global options to select model and programming language
 
 ---
 
 ## ❤️ Why Local AI?
 
-We believe your code — and your workflow — should stay **yours**. scai runs entirely on your machine using open-source models and tools.
+At scai, we believe your code — and your workflow — should stay **yours**. That's why we prioritize **local models** and **privacy-first** development:
 
-✅ Works entirely offline  
-✅ No API keys or cloud dependencies  
-✅ Backed by open-source models  
-✅ Designed for CLI-first developers
+- ✅ **Local models**: scai runs entirely on your machine, with no need for cloud-based processing. We use open-source models like `llama3` and `codellama`, ensuring that your code stays private and secure.
+- ✅ **Privacy by design**: We do not send your code to the cloud, store your data, or require API keys. Your data remains 100% local, ensuring full control over your project.
+- ✅ **EU & Danish friendly**: As a Danish company, we are deeply committed to respecting European privacy regulations (GDPR). This means no data collection, no tracking, and full compliance with the strictest data protection laws.
+  
+By running everything locally, you can be sure that your sensitive code and workflows stay safe, private, and within your control.
+
+---
 
 ## 📦 Installation
 
-1. **Install [Ollama](https://ollama.com)**  
+1. **Install Ollama:**  
    - On **Windows**: [Download Ollama](https://ollama.com/download)  
-   - On **macOS**:  
-     - Via Homebrew:  
-       ```bash
-       brew install ollama
-       ```  
-     - Or download directly from the website
+   - On **macOS**: via Homebrew (`brew install ollama`) or [download from Ollama](https://ollama.com)
 
-2. **Install scai globally:**
+2. **Install scai globally:**  
+    ```bash
+    npm install -g scai
+    ```
 
-```bash
-npm install -g scai
-```
+3. **Initialize the tool and models:**  
+    ```bash
+    scai init
+    ```
 
-3. **Initialize the tool and models:**
+This will download required models and set up scai.
 
-```bash
-scai init
-```
+---
 
-This will:
-- Start the Ollama background server
-- Download required models (`llama3`, etc.)
+## ⚒️ Usage Examples
 
-## 🧪 Usage Examples
+### 🔧 Git Commands
 
-### 💬 Suggest a commit message
+- **Check Git status (better than `git status`):**  
+    ```bash
+    scai git status
+    ```
 
-```bash
-git add .
-scai sugg
-```
+    This is an enhanced version of `git status`. It does more than just show the status of your working directory:
 
-> Example output:
-```
-feat(api): add error handling to user service
-```
+    - **Check if your working directory is clean**: It tells you if there are any uncommitted changes.
+    - **Check your current branch**: It displays your current branch name.
+    - **Check if you're up to date with `origin/main`**: Unlike `git status`, which only shows local status, scai ensures you know if your branch is behind or ahead of the `origin/main` branch. It compares your local commit hash with the remote `origin/main` hash, giving you more comprehensive information.
 
-To commit automatically with the suggestion:
+    Example output:
+    ```
+    ✅ Git working directory is clean
+    ✅ Up to date with origin/main
+    ```
 
-```bash
-scai sugg --commit
-```
+    If you're behind `origin/main`, it will show:
+    ```
+    🔄 Branch <your-branch-name> is not up to date with origin/main
+    ```
 
----
+- **Suggest a commit message:**  
+    ```bash
+    git add .  
+    scai git sugg  
+    ```
 
-### ✨ Comment a code file
+    *Example output:*  
+    ```
+    feat(api): add error handling to user service
+    ```
 
-```bash
-scai comm <file>
-```
+    To commit automatically with the suggestion:  
+    ```bash
+    scai git sugg --commit
+    ```
 
-Adds clear, helpful comments to the code. Optional flag:
-- `-a, --apply` — Overwrite the original file with the commented version
+### 🛠️ Generate Code-Related Output (`gen` commands)
 
----
+- **Comment a code file:**  
+    ```bash
+    scai gen comm <file>
+    ```
 
-### 🔍 Summarize a code file
+    Adds clear, helpful comments to the code. Optional flag:
+    - `-a, --apply` — Overwrite the original file with the commented version.
 
-```bash
-scai summ <file>
-```
+- **Summarize a code file:**  
+    ```bash
+    scai gen summ <file>
+    ```
 
-Prints a summary of what the code does directly to your terminal.
+    Prints a summary of what the code does directly in the terminal. You can also pipe file content:  
+    ```bash
+    cat <file> | scai gen summ
+    ```
 
-To pipe input for summarization:
+- **Generate tests for a code file:**  
+    ```bash
+    scai gen tests <file>
+    ```
 
-```bash
-cat <file> | scai summ
-```
+    Creates a Jest test file for the specified JavaScript/TypeScript module.
 
----
+- **Update the changelog:**  
+    ```bash
+    scai gen changelog
+    ```
 
-### 📝 Update the changelog
+    Analyzes the current Git diff and updates (or creates) a `CHANGELOG.md` file with relevant changes.
 
-```bash
-scai changelog
-```
+## ⚙️ Configuration
 
-Analyzes the current Git diff and updates (or creates) a `CHANGELOG.md` file with relevant public-facing changes.
+scai stores your configuration locally in the `~/.scai/config.json` file. You can configure the model and programming language settings as follows:
 
----
+- **Set the model:**  
+    ```bash
+    scai set model <model>
+    ```
+
+    e.g., `scai set model codellama:7b`
+
+- **Set the programming language:**  
+    ```bash
+    scai set lang <lang>
+    ```
 
-### 📦 Module Pipeline (Advanced Usage)
+    e.g., `scai set lang rust`
 
-For more advanced functionality, scai allows you to run custom pipelines with multiple modules to process code. You can specify multiple modules to be run, such as comment, summary, cleanup, and more.
+- **Show current configuration:**  
+    ```bash
+    scai config
+    ```
 
-```bash
-scai <file> --modules comment,summary,cleanup
-```
-Like with regular Unix commands you can pipe this to a file
+You can also use global options `--model` and `--lang` with any command to override the settings:  
+    ```bash
+    scai --model codellama:34b --lang ts git sugg
+    ```
 
-```bash
-scai <file> --modules comment,summary > test.txt
-```
-Or to both stdout and a file with tee fx. 
-```bash
-scai <file> --modules comment,summary | tee test.txt
-```
+---
 
 ## 🔐 License & Fair Use
 
 **scai is free to use** for individuals, teams, and companies — including in commercial work.
 
 You may:
-
 - ✅ Use it internally in your projects  
 - ✅ Use it at work or in commercial development  
 - ✅ Share and recommend it  
 
 But you may not:
-
 - ❌ Repackage or resell **scai** as a product or SaaS  
 - ❌ Claim ownership of the tool
 

package/dist/commands/AskCmd.js

@@ -0,0 +1,66 @@
+import { searchFiles } from "../db/fileIndex.js";
+import { generate } from "../lib/generate.js";
+import { summaryModule } from "../pipeline/modules/summaryModule.js";
+export async function runAskCommand(query) {
+    if (!query) {
+        console.error('❌ Please provide a search query.\n👉 Usage: scai ask "keyword"');
+        return;
+    }
+    console.log(`🔍 Searching for: "${query}"\n`);
+    // Use vector-based search
+    const results = await searchFiles(query, 5); // Or 3 if you want fewer
+    if (results.length === 0) {
+        console.log('⚠️ No similar embeddings found. Asking the model for context instead...');
+    }
+    else {
+        console.log('📊 Closest files based on semantic similarity:');
+        results.forEach(file => {
+            console.log(`📄 Path: ${file?.path}`);
+        });
+    }
+    let allSummaries = '';
+    for (const file of results) {
+        try {
+            if (!file?.summary) {
+                console.warn(`⚠️ No summary available for file: ${file?.path}`);
+                continue;
+            }
+            console.log(`📝 Using cached summary for file: ${file?.path}`);
+            const summaryResponse = await summaryModule.run({ content: file?.summary ? file.summary : '', filepath: file?.path });
+            if (summaryResponse.summary) {
+                allSummaries += `\n${summaryResponse.summary}`;
+            }
+        }
+        catch (err) {
+            console.error(`❌ Error processing file: ${file?.path}`, err instanceof Error ? err.message : err);
+        }
+    }
+    if (allSummaries.trim()) {
+        console.log('🧠 Summaries found, sending them to the model for synthesis...');
+        try {
+            const input = {
+                content: `${query}\n\n${allSummaries}`,
+                filepath: '',
+            };
+            const modelResponse = await generate(input, 'llama3');
+            console.log(`\n📝 Model response:\n${modelResponse.content}`);
+        }
+        catch (err) {
+            console.error('❌ Model request failed:', err);
+        }
+    }
+    else {
+        console.log('⚠️ No summaries found. Asking the model for context only...');
+        try {
+            const input = {
+                content: query,
+                filepath: '',
+            };
+            const modelResponse = await generate(input, 'llama3');
+            console.log(`\n📝 Model response:\n${modelResponse.content}`);
+        }
+        catch (err) {
+            console.error('❌ Model request failed:', err);
+        }
+    }
+}

package/dist/commands/DaemonCmd.js

@@ -4,10 +4,17 @@ import fs from 'fs/promises';
 import fsSync from 'fs';
 import os from 'os';
 import path from 'path';
-const MAX_FILES = 500;
-const DAEMON_DURATION_MINUTES = 10;
+import { generateEmbedding } from '../lib/generateEmbedding.js';
+import { IGNORED_EXTENSIONS } from '../config/IgnoredExtensions.js';
+const MAX_FILES = 1000;
+const DAEMON_DURATION_MINUTES = 25;
 const DAEMON_INTERVAL_MINUTES = 30;
 const PID_PATH = path.join(os.homedir(), '.scai/daemon.pid');
+// Helper function to check if a file should be ignored
+const shouldIgnoreFile = (filePath) => {
+    const ext = path.extname(filePath).toLowerCase();
+    return IGNORED_EXTENSIONS.includes(ext);
+};
 export async function runDaemonBatch() {
     console.log('📥 Daemon batch: scanning for files to summarize...');
     const rows = db.prepare(`
@@ -15,19 +22,34 @@ export async function runDaemonBatch() {
     WHERE summary IS NULL OR summary = ''
     ORDER BY last_modified DESC
     LIMIT ?
-`).all(MAX_FILES);
+    `).all(MAX_FILES);
     if (rows.length === 0) {
         console.log('✅ No files left to summarize.');
         return;
     }
     for (const row of rows) {
+        if (shouldIgnoreFile(row.path)) {
+            console.log(`⚠️ Ignored file (unwanted extension): ${row.path}`);
+            continue;
+        }
         try {
             const content = await fs.readFile(row.path, 'utf-8');
             const result = await summaryModule.run({ content, filepath: row.path });
             const summary = result?.summary?.trim() ? result.summary : null;
-            db.prepare(`UPDATE files SET summary = ?, indexed_at = datetime('now') WHERE path = ?`)
-                .run(summary, row.path);
+            let embedding = null;
+            if (summary) {
+                const vector = await generateEmbedding(summary);
+                if (vector)
+                    embedding = JSON.stringify(vector);
+            }
+            // Using named parameters for better readability and flexibility
+            db.prepare(`
+                UPDATE files
+                SET summary = @summary, embedding = @embedding, indexed_at = datetime('now')
+                WHERE path = @path
+            `).run({ summary, embedding, path: row.path });
             console.log(`📝 Summarized: ${row.path}`);
+            console.log(`🔢 Embedded: ${row.path}`);
         }
         catch (err) {
             console.warn(`⚠️ Failed: ${row.path}`, err instanceof Error ? err.message : err);

package/dist/commands/IndexCmd.js

@@ -6,15 +6,37 @@ import { shouldIgnoreFile } from '../utils/shouldIgnoreFiles.js';
 import { detectFileType } from '../utils/detectFileType.js';
 import { runDaemonScheduler } from './DaemonCmd.js';
 import { IGNORED_FOLDER_GLOBS } from '../config/IgnoredPaths.js';
+import { db } from '../db/client.js';
 const IGNORE = [
     '**/node_modules/**', '**/dist/**', '**/build/**',
     '**/coverage/**', '**/.git/**', '**/*.test.*'
 ];
 export async function runIndexCommand(targetDir = process.cwd(), options = {}) {
-    console.log(`📂 Indexing files in: ${targetDir}`);
+    const resolvedDir = path.resolve(targetDir);
+    console.log(`📂 Indexing files in: ${resolvedDir}`);
     initSchema();
+    // 🧠 Check if another directory has already been indexed
+    const indexedPaths = db.prepare(`
+    SELECT DISTINCT path FROM files LIMIT 100
+  `).all();
+    const knownRoot = indexedPaths.length > 0
+        ? path.dirname(indexedPaths[0].path)
+        : null;
+    if (knownRoot && !resolvedDir.startsWith(knownRoot) && !options.force) {
+        console.warn(`⚠️ You're indexing a different folder than before:
+- Previously: ${knownRoot}
+- Now:        ${resolvedDir}
+
+This will add more files into the existing index and may reduce accuracy or performance.
+
+Use --force to continue, or consider clearing the index:
+  scai reset-db
+
+Aborting.`);
+        process.exit(1);
+    }
     const files = await fg('**/*.*', {
-        cwd: targetDir,
+        cwd: resolvedDir,
         ignore: IGNORED_FOLDER_GLOBS,
         absolute: true,
     });
@@ -28,7 +50,7 @@ export async function runIndexCommand(targetDir = process.cwd(), options = {}) {
             indexFile(file, null, type); // empty summary for now
             const ext = path.extname(file);
             countByExt[ext] = (countByExt[ext] || 0) + 1;
-            console.log(`📄 Indexed: ${path.relative(targetDir, file)}`);
+            console.log(`📄 Indexed: ${path.relative(resolvedDir, file)}`);
             count++;
         }
         catch (err) {
@@ -39,6 +61,6 @@ export async function runIndexCommand(targetDir = process.cwd(), options = {}) {
     console.log(`✅ Done. Indexed ${count} files.`);
     if (options.detached) {
         console.log('🚀 Starting summarizer daemon in background mode...');
-        runDaemonScheduler(); // Infinite loop every 30 min
+        runDaemonScheduler();
     }
 }

package/dist/commands/QueryCmd.js

@@ -1,4 +1,4 @@
-import { searchFiles } from '../db/fileIndex.js';
+import { queryFiles } from '../db/fileIndex.js';
 import path from 'path';
 export async function runQueryCommand(query) {
     if (!query) {
@@ -6,7 +6,7 @@ export async function runQueryCommand(query) {
         return;
     }
     console.log(`🔍 Searching for: "${query}"\n`);
-    const results = searchFiles(query);
+    const results = queryFiles(query);
     if (results.length === 0) {
         console.log('⚠️ No matching files found.');
         return;

package/dist/config/IgnoredExtensions.js

@@ -3,23 +3,44 @@ export const IGNORED_EXTENSIONS = [
     // 🖼 Media
     '.png', '.jpg', '.jpeg', '.gif', '.webp', '.svg', '.ico',
     '.mp4', '.mp3', '.mov', '.avi', '.mkv', '.flv', '.wav', '.flac',
+    '.aac', '.m4a', '.wma', '.3gp', '.webm', '.ogg', '.aiff', '.au',
     // 📦 Archives & install packages
     '.zip', '.tar', '.gz', '.bz2', '.xz', '.rar', '.7z',
-    '.jar', '.war', '.ear', // Java packaging
-    '.deb', '.rpm', '.pkg', '.msi', '.dmg', '.cab', '.xz',
+    '.jar', '.war', '.ear', '.deb', '.rpm', '.pkg', '.msi', '.dmg', '.cab', '.apk',
+    '.tar.gz', '.tar.bz2', '.tar.xz', '.tar.lzma', '.tar.zst',
     // 🧱 Binaries & executables
     '.exe', '.dll', '.bin', '.so', '.dylib', '.a', '.lib',
-    '.iso', '.img', '.elf', '.o', '.obj',
+    '.iso', '.img', '.elf', '.o', '.obj', '.msm', '.vbs', '.jscript',
+    '.cmd', '.bat', '.ps1', '.sh', '.bash', '.run',
     // 🧪 Runtime / build / cache
-    '.log', '.lock', '.tmp', '.map',
+    '.log', '.tmp', '.map',
     '.db', '.sqlite', '.pkl', '.sav', '.rdb', '.ldb',
-    '.pyc', '.class', '.tsbuildinfo', '.coverage',
+    '.pyc', '.class', '.tsbuildinfo', '.coverage', '.eslintcache',
+    '.yarn', '.webpack', '.babel', '.compilercache',
     // 🔤 Fonts & styles
     '.woff', '.woff2', '.ttf', '.eot', '.otf', '.css.map',
+    '.scss', '.sass', '.less', '.styl',
     // 🔐 Certs, keys, credentials
-    '.crt', '.key', '.pem', '.pub', '.asc', '.gpg',
+    '.crt', '.key', '.pem', '.pub', '.asc', '.gpg', '.p12', '.csr', '.der', '.pfx',
     // ♻️ Backups / temp
-    '.bak', '.old', '.swp', '.swo', '.tmp', '.orig',
-    // 🌐 Misc
-    '.torrent', '.DS_Store', '.env.local', '.env.production', '.env.development',
+    '.bak', '.old', '.swp', '.swo', '.orig',
+    '.sublime-workspace', '.sublime-project', '.db-shm', '.db-wal',
+    // 🌐 System/config folders (still ignored by path, not extension)
+    '.DS_Store', '.bundle', '.npmrc',
+    // 🗺️ GIS / Geospatial
+    '.shp', '.shx', '.dbf', '.prj', '.qix', '.sbn', '.sbx', '.shp.xml', '.cpg', '.gpkg', '.mif', '.mid',
+    // 📊 Enterprise BI / Reporting
+    '.pbix', '.rdl', '.rpt', '.bqy', '.iqy',
+    // 🧪 ETL / DWH / Modeling
+    '.abf', '.dtsx', '.bim', '.xmi',
+    // 🏗️ CAD / Engineering
+    '.dwg', '.dxf', '.step', '.stp', '.sldprt', '.sldasm',
+    '.iges', '.igs', '.3ds', '.fbx',
+    // 🧾 Forms / Print / Publishing
+    '.xps', '.afpub', '.pub', '.indd', '.qxd', '.frm', '.frx', '.frl',
+    // 💰 ERP / Finance / Legacy DB
+    '.mbd', '.fdb', '.nav', '.accdb', '.mdb', '.gdb',
+    '.sap', '.sappkg', '.qbw', '.qbb',
+    // 🔒 Lock files (but NOT lock *configs*)
+    '.lck', '.lockfile', '.db-lock', '.pid', '.socket',
 ];

package/dist/config/IgnoredPaths.js

@@ -18,4 +18,28 @@ export const IGNORED_FOLDER_GLOBS = [
     '**/.output/**',
     '**/tmp/**',
     '**/*.test.*',
+    '**/.m2/**',
+    '**/.gradle/**',
+    '**/.tox/**',
+    '**/.nox/**',
+    '**/.hypothesis/**',
+    '**/.npm/**',
+    '**/.yarn/**',
+    '**/*.o',
+    '**/*.out',
+    '**/*.exe',
+    '**/*.dll',
+    '**/.cache/**',
+    '**/.pylint.d/**',
+    '**/.eslintcache/**',
+    '**/.cache-loader/**',
+    '**/.serverless/**',
+    '**/.docker/**',
+    '**/.sublime-workspace',
+    '**/.sublime-project',
+    '**/*.log',
+    '**/npm-debug.log',
+    '**/yarn-error.log',
+    '**/debug.log',
+    '**/Dockerfile',
 ];

package/dist/config.js

@@ -0,0 +1,68 @@
+import fs from 'fs';
+import { CONFIG_PATH, SCAI_HOME, INDEX_DIR } from './constants.js'; // Correctly import INDEX_DIR from constants
+// Default configuration values
+const defaultConfig = {
+    model: 'llama3',
+    language: 'ts',
+    indexDir: INDEX_DIR, // Default index directory from constants
+};
+// Function to ensure the configuration directory exists
+function ensureConfigDir() {
+    if (!fs.existsSync(SCAI_HOME)) {
+        fs.mkdirSync(SCAI_HOME, { recursive: true });
+    }
+}
+// Function to read the configuration file
+function readConfig() {
+    try {
+        const content = fs.readFileSync(CONFIG_PATH, 'utf-8');
+        return { ...defaultConfig, ...JSON.parse(content) };
+    }
+    catch {
+        return defaultConfig; // Return default config if read fails
+    }
+}
+// Function to write the configuration to the config file
+function writeConfig(newConfig) {
+    ensureConfigDir();
+    const current = readConfig();
+    const merged = { ...current, ...newConfig };
+    fs.writeFileSync(CONFIG_PATH, JSON.stringify(merged, null, 2));
+}
+export const Config = {
+    // Get the current model from the config
+    getModel() {
+        return readConfig().model;
+    },
+    // Set a new model in the config
+    setModel(model) {
+        writeConfig({ model });
+        console.log(`📦 Model set to: ${model}`);
+    },
+    // Get the current language from the config
+    getLanguage() {
+        return readConfig().language;
+    },
+    // Set a new language in the config
+    setLanguage(language) {
+        writeConfig({ language });
+        console.log(`🗣️ Language set to: ${language}`);
+    },
+    // Get the index directory from the config
+    getIndexDir() {
+        return readConfig().indexDir;
+    },
+    // Set a new index directory in the config
+    setIndexDir(indexDir) {
+        writeConfig({ indexDir });
+        console.log(`📁 Index directory set to: ${indexDir}`);
+    },
+    // Show the current configuration
+    show() {
+        const cfg = readConfig();
+        console.log(`🔧 Current configuration:`);
+        console.log(`   Model     : ${cfg.model}`);
+        console.log(`   Language  : ${cfg.language}`);
+        console.log(`   Index dir : ${cfg.indexDir}`);
+    }
+};

package/dist/constants.js

@@ -0,0 +1,19 @@
+import os from 'os';
+import path from 'path';
+import fs from 'fs';
+export const SCAI_HOME = path.join(os.homedir(), '.scai');
+export const DB_PATH = path.join(SCAI_HOME, 'db.sqlite');
+export const PID_PATH = path.join(SCAI_HOME, 'daemon.pid');
+export const CONFIG_PATH = path.join(SCAI_HOME, 'config.json');
+// Function to read config and get the indexDir on-demand
+export function getIndexDir() {
+    try {
+        const config = JSON.parse(fs.readFileSync(CONFIG_PATH, 'utf-8'));
+        return config.indexDir || path.join(os.homedir(), 'defaultIndex'); // Default if not set
+    }
+    catch (e) {
+        return path.join(os.homedir(), 'defaultIndex'); // Fallback if no config file
+    }
+}
+// On-demand index directory retrieval
+export const INDEX_DIR = getIndexDir();

package/dist/db/client.js

@@ -1,7 +1,5 @@
 import Database from 'better-sqlite3';
-import path from 'path';
 import fs from 'fs';
-const DB_PATH = path.resolve(process.cwd(), '.scai/db.sqlite');
-// Ensure directory exists
-fs.mkdirSync(path.dirname(DB_PATH), { recursive: true });
+import { DB_PATH, SCAI_HOME } from '../constants.js';
+fs.mkdirSync(SCAI_HOME, { recursive: true });
 export const db = new Database(DB_PATH);

package/dist/db/fileIndex.js

@@ -1,51 +1,114 @@
+// File: src/db/fileIndex.ts
 import { db } from './client.js';
 import fs from 'fs';
+import { generateEmbedding } from '../lib/generateEmbedding.js';
+import * as sqlTemplates from './sqlTemplates.js'; // Import the SQL templates
 export function indexFile(filePath, summary, type) {
     const stats = fs.statSync(filePath);
     const lastModified = stats.mtime.toISOString();
     // 1) INSERT new rows (only when path not present)
-    const insertStmt = db.prepare(`
-    INSERT OR IGNORE INTO files
-      (path, summary, type, indexed_at, last_modified)
-    VALUES (?, ?, ?, datetime('now'), ?)
-  `);
-    insertStmt.run(filePath, summary, type, lastModified);
+    const insertStmt = db.prepare(sqlTemplates.insertFileTemplate);
+    insertStmt.run({ path: filePath, summary, type, lastModified });
     // 2) UPDATE metadata if file already existed and changed
-    const updateStmt = db.prepare(`
-    UPDATE files
-    SET type       = ?,
-        last_modified = ?,
-        indexed_at = datetime('now')
-    WHERE path = ?
-      AND last_modified != ?
-  `);
-    updateStmt.run(type, lastModified, filePath, lastModified);
+    const updateStmt = db.prepare(sqlTemplates.updateFileTemplate);
+    updateStmt.run({ path: filePath, type, lastModified });
     // Step 1: Delete from FTS where the path matches
-    db.prepare(`
-  DELETE FROM files_fts
-  WHERE rowid = (SELECT id FROM files WHERE path = ?)
-`).run(filePath);
+    db.prepare(sqlTemplates.deleteFromFtsTemplate).run({ path: filePath });
     // Step 2: Insert into FTS with the same id
-    db.prepare(`
-  INSERT INTO files_fts(rowid, path, summary)
-  VALUES((SELECT id FROM files WHERE path = ?), ?, ?)
-`).run(filePath, filePath, summary);
+    db.prepare(sqlTemplates.insertIntoFtsTemplate).run({ path: filePath, summary });
 }
-export function searchFiles(query, limit = 10) {
-    // Use FTS5 MATCH for relevance-ranked results
-    const stmt = db.prepare(`
+export function queryFiles(query, limit = 10) {
+    // Sanitize the query by removing or escaping special characters
+    const safeQuery = query
+        .trim()
+        .split(/\s+/)
+        .map(token => {
+        token = token
+            .replace(/[?*\\"]/g, '') // Remove question marks, asterisks, backslashes, and double quotes
+            .replace(/'/g, "''"); // Escape single quotes for SQL safety
+        // For multi-word queries, wrap the token in quotes for exact phrase matching
+        if (token.includes(' ')) {
+            return `"${token}"`; // Exact phrase match for multi-word tokens
+        }
+        return `${token}*`; // Prefix match for single tokens
+    })
+        .join(' OR ');
+    // Log the constructed query for debugging purposes
+    console.log(`Executing search query: ${safeQuery}`);
+    // Execute the query with safeQuery and limit as parameters
+    const sql = `
     SELECT f.path, f.summary, f.type, f.last_modified, f.indexed_at,
            bm25(files_fts) AS rank
     FROM files_fts
     JOIN files f ON files_fts.rowid = f.id
-    WHERE files_fts MATCH ?
+    WHERE files_fts MATCH :query
     ORDER BY rank
-    LIMIT ?
-  `);
-    const matchQuery = query
+    LIMIT :limit
+  `;
+    const results = db.prepare(sql).all({ query: safeQuery, limit });
+    return results;
+}
+export function cosineSimilarity(a, b) {
+    const dot = a.reduce((sum, ai, i) => sum + ai * b[i], 0);
+    const magA = Math.sqrt(a.reduce((sum, ai) => sum + ai * ai, 0));
+    const magB = Math.sqrt(b.reduce((sum, bi) => sum + bi * bi, 0));
+    return dot / (magA * magB);
+}
+export async function searchFiles(query, topK = 5) {
+    // Generate the query embedding
+    const embedding = await generateEmbedding(query);
+    if (!embedding)
+        return [];
+    // Sanitize the query by removing or escaping special characters
+    const safeQuery = query
         .trim()
         .split(/\s+/)
-        .map(token => `${token}*`) // prefix search
-        .join(' ');
-    return stmt.all(matchQuery, limit);
+        .map(token => {
+        token = token
+            .replace(/[?*\\"]/g, '') // Remove question marks, asterisks, backslashes, and double quotes
+            .replace(/'/g, "''"); // Escape single quotes for SQL safety
+        // For multi-word queries, wrap the token in quotes for exact phrase matching
+        if (token.includes(' ')) {
+            return `"${token}"`; // Exact phrase match for multi-word tokens
+        }
+        return `${token}*`; // Prefix match for single tokens
+    })
+        .join(' OR ');
+    // Log the constructed query for debugging purposes
+    console.log(`Executing search query: ${safeQuery}`);
+    // Fetch BM25 scores from the FTS using the safeQuery string directly
+    const ftsResults = db.prepare(sqlTemplates.fetchBm25ScoresTemplate).all({ query: safeQuery });
+    const bm25Min = Math.min(...ftsResults.map(r => r.bm25Score));
+    const bm25Max = Math.max(...ftsResults.map(r => r.bm25Score));
+    // Calculate final score combining BM25 and cosine similarity
+    const scored = ftsResults.map(result => {
+        try {
+            // Fetch embedding for each file from the `files` table
+            const embResult = db.prepare(sqlTemplates.fetchEmbeddingTemplate).get({ path: result.path });
+            // Check if embedding exists and has the correct structure
+            if (!embResult || typeof embResult.embedding !== 'string')
+                return null;
+            // Parse the embedding
+            const vector = JSON.parse(embResult.embedding);
+            const sim = cosineSimilarity(embedding, vector);
+            // Normalize BM25 scores
+            const normalizedBm25 = 1 - ((result.bm25Score - bm25Min) / (bm25Max - bm25Min + 1e-5));
+            const normalizedSim = sim; // cosineSimilarity is already 0–1
+            const finalScore = 0.7 * normalizedSim + 0.3 * normalizedBm25;
+            return {
+                path: result.path,
+                summary: result.summary,
+                score: finalScore,
+                sim: normalizedSim,
+                bm25: normalizedBm25
+            };
+        }
+        catch (err) {
+            console.error(`Error processing embedding for file: ${result.path}`, err);
+            return null;
+        }
+    }).filter(Boolean)
+        .sort((a, b) => b.score - a.score)
+        .slice(0, topK);
+    return scored;
 }

package/dist/db/schema.js

@@ -7,7 +7,8 @@ export function initSchema() {
       summary TEXT,
       type TEXT,
       indexed_at TEXT,
-      last_modified TEXT
+      last_modified TEXT,
+      embedding TEXT
     );
 
     -- FTS5 table for fast full‑text search of summaries and paths

package/dist/db/sqlTemplates.js

@@ -0,0 +1,37 @@
+// Template for inserting or ignoring new file entries
+export const insertFileTemplate = `
+  INSERT OR IGNORE INTO files
+    (path, summary, type, indexed_at, last_modified)
+  VALUES (:path, :summary, :type, datetime('now'), :lastModified)
+`;
+// Template for updating file metadata if it has changed
+export const updateFileTemplate = `
+  UPDATE files
+  SET type = :type,
+      last_modified = :lastModified,
+      indexed_at = datetime('now')
+  WHERE path = :path
+    AND last_modified != :lastModified
+`;
+// Template for deleting a file from FTS
+export const deleteFromFtsTemplate = `
+  DELETE FROM files_fts
+  WHERE rowid = (SELECT id FROM files WHERE path = :path)
+`;
+// Template for inserting a file into FTS with its ID
+export const insertIntoFtsTemplate = `
+  INSERT INTO files_fts(rowid, path, summary)
+  VALUES((SELECT id FROM files WHERE path = :path), :path, :summary)
+`;
+// Template for fetching BM25 scores from FTS
+export const fetchBm25ScoresTemplate = `
+  SELECT f.path, f.summary, f.type, bm25(files_fts) AS bm25Score
+  FROM files_fts
+  JOIN files f ON files_fts.rowid = f.id
+  WHERE files_fts MATCH :query
+  LIMIT 50
+`;
+// Template for fetching embedding for a specific file
+export const fetchEmbeddingTemplate = `
+  SELECT embedding FROM files WHERE path = :path
+`;

package/dist/index.js

@@ -1,17 +1,17 @@
 #!/usr/bin/env node
 import { Command } from "commander";
-// Importing package.json for version
+import path from "path";
+import { Config } from './config.js';
 import { createRequire } from 'module';
 const require = createRequire(import.meta.url);
 const { version } = require('../package.json');
-// Importing commands
+// 🧠 Commands
 import { checkEnv } from "./commands/EnvCmd.js";
 import { checkGit } from "./commands/GitCmd.js";
 import { suggestCommitMessage } from "./commands/CommitSuggesterCmd.js";
 import { handleRefactor } from "./commands/RefactorCmd.js";
 import { generateTests } from "./commands/TestGenCmd.js";
 import { bootstrap } from './modelSetup.js';
-import { ModelConfig } from './config/ModelConfig.js';
 import { summarizeFile } from "./commands/SummaryCmd.js";
 import { handleChangelogUpdate } from './commands/ChangeLogUpdateCmd.js';
 import { runModulePipelineFromCLI } from './commands/ModulePipelineCmd.js';
@@ -20,12 +20,13 @@ import { resetDatabase } from './commands/ResetDbCmd.js';
 import { runQueryCommand } from './commands/QueryCmd.js';
 import { runDaemonBatch } from './commands/DaemonCmd.js';
 import { runStopDaemonCommand } from "./commands/StopDaemonCmd.js";
-// Create the CLI instance
+import { runAskCommand } from './commands/AskCmd.js';
+// 🎛️ CLI Setup
 const cmd = new Command('scai')
     .version(version)
     .option('--model <model>', 'Set the model to use (e.g., codellama:34b)')
     .option('--lang <lang>', 'Set the target language (ts, java, rust)');
-// Define CLI commands
+// 🚀 Init command
 cmd
     .command('init')
     .description('Initialize the model and download required models')
@@ -33,44 +34,96 @@ cmd
     await bootstrap();
     console.log('✅ Model initialization completed!');
 });
-cmd
+// 🔧 Group: Git-related commands
+const git = cmd.command('git').description('Git utilities');
+git
+    .command('status')
+    .description('Check Git status')
+    .action(checkGit);
+git
     .command('sugg')
     .description('Suggest a commit message from staged changes')
     .option('-c, --commit', 'Automatically commit with suggested message')
     .action(suggestCommitMessage);
-cmd
+// 🛠️ Group: `gen` commands for content generation
+const gen = cmd.command('gen').description('Generate code-related output');
+gen
     .command('comm <file>')
     .description('Write comments for the given file')
     .option('-a, --apply', 'Apply the refactored version to the original file')
     .action((file, options) => handleRefactor(file, options));
-cmd
+gen
     .command('changelog')
     .description('Update or create the CHANGELOG.md based on current Git diff')
     .action(async () => {
     await handleChangelogUpdate();
 });
-cmd
+gen
     .command('summ [file]')
     .description('Print a summary of the given file to the terminal')
     .action((file) => summarizeFile(file));
+gen
+    .command('tests <file>')
+    .description('Generate a Jest test file for the specified JS/TS module')
+    .action((file) => generateTests(file));
+// 🔍 Indexing
 cmd
-    .command('git')
-    .description('Check Git status')
-    .action(checkGit);
+    .command('index [targetDir]')
+    .description('Index supported files in the given directory (or current folder if none)')
+    .option('-d, --detached', 'Run summarizer daemon after indexing')
+    .option('--force', 'Force indexing even if another folder has already been indexed')
+    .action((targetDir, options) => {
+    const resolvedDir = targetDir ? path.resolve(targetDir) : process.cwd();
+    runIndexCommand(resolvedDir, { detached: options.detached, force: options.force });
+});
+// ⚙️ Group: Configuration settings
+const set = cmd.command('set').description('Set configuration values');
+set
+    .command('model <model>')
+    .description('Set the model to use')
+    .action((model) => {
+    Config.setModel(model);
+    Config.show();
+});
+set
+    .command('lang <lang>')
+    .description('Set the programming language')
+    .action((lang) => {
+    Config.setLanguage(lang);
+    Config.show();
+});
+set
+    .command('index-dir <dir>')
+    .description('Set the path to the indexed directory')
+    .action((dir) => {
+    Config.setIndexDir(path.resolve(dir));
+    Config.show();
+});
+// 🧪 Diagnostics and info
 cmd
     .command('env')
     .description('Check environment variables')
     .action(checkEnv);
-cmd
-    .command('gen-tests <file>')
-    .description('Generate a Jest test file for the specified JS/TS module')
-    .action((file) => generateTests(file));
 cmd
     .command('config')
     .description('Show the currently active model and language settings')
     .action(() => {
-    ModelConfig.logCurrentConfig();
+    Config.show();
 });
+// 🧠 Query and assistant
+cmd
+    .command('query <query>')
+    .description('Search indexed files by keyword')
+    .action(runQueryCommand);
+cmd
+    .command('ask')
+    .description('Ask a question using file summaries and a local model')
+    .argument('<question...>', 'The question to ask')
+    .action((question) => {
+    const q = question.join(' ');
+    runAskCommand(q);
+});
+// 🛠️ Background tasks and maintenance
 cmd
     .command('daemon')
     .description('Run background summarization of indexed files')
@@ -79,22 +132,11 @@ cmd
     .command('stop-daemon')
     .description('Stop the background summarizer daemon')
     .action(runStopDaemonCommand);
-cmd
-    .command('index [targetDir]')
-    .description('Index supported files in the given directory (or current folder if none)')
-    .option('-d, --detached', 'Run summarizer daemon after indexing')
-    .action((targetDir, options) => {
-    runIndexCommand(targetDir, { detached: options.detached });
-});
-cmd
-    .command('query <query>')
-    .description('Search indexed files by keyword')
-    .action(runQueryCommand);
 cmd
     .command('reset-db')
     .description('Delete and reset the SQLite database')
     .action(() => resetDatabase());
-// Default
+// 🧬 Fallback: Pipeline mode
 cmd
     .arguments('<file>')
     .option('-m, --modules <modules>', 'Comma-separated list of modules to run (e.g., comments,cleanup,summary)')
@@ -105,11 +147,11 @@ cmd
     }
     runModulePipelineFromCLI(file, options);
 });
-// ✅ Now that commands are defined, parse args
+// ✅ Parse CLI args
 cmd.parse(process.argv);
-// ✅ Apply global options after parsing
+// 🔁 Apply global options post-parse
 const opts = cmd.opts();
 if (opts.model)
-    ModelConfig.setModel(opts.model);
+    Config.setModel(opts.model);
 if (opts.lang)
-    ModelConfig.setLanguage(opts.lang);
+    Config.setLanguage(opts.lang);

package/dist/lib/generateEmbedding.js

@@ -0,0 +1,22 @@
+export async function generateEmbedding(text) {
+    try {
+        const res = await fetch('http://localhost:11434/api/embeddings', {
+            method: 'POST',
+            headers: { 'Content-Type': 'application/json' },
+            body: JSON.stringify({
+                model: 'mistral', // or 'llama3' — whatever works best
+                prompt: text,
+            }),
+        });
+        if (!res.ok) {
+            console.error('❌ Failed to generate embedding:', await res.text());
+            return null;
+        }
+        const data = await res.json();
+        return data.embedding;
+    }
+    catch (err) {
+        console.error('❌ Embedding error:', err instanceof Error ? err.message : err);
+        return null;
+    }
+}

package/dist/pipeline/modules/changeLogModule.js

@@ -1,10 +1,10 @@
-import { ModelConfig } from '../../config/ModelConfig.js';
+import { Config } from '../../config.js';
 import { generate } from '../../lib/generate.js';
 export const changelogModule = {
     name: 'changelogModule',
     description: 'Generates changelog entry based on Git diff',
     async run(input) {
-        const model = ModelConfig.getModel();
+        const model = Config.getModel();
         const prompt = `
 You're an experienced changelog writer. Based on this Git diff, write a markdown bullet-point entry suitable for CHANGELOG.md:
 

package/dist/pipeline/modules/commentModule.js

@@ -1,11 +1,11 @@
-import { ModelConfig } from '../../config/ModelConfig.js';
+import { Config } from '../../config.js';
 import { generate } from '../../lib/generate.js';
 export const addCommentsModule = {
     name: 'addComments',
     description: 'Adds meaningful // comments to each block of code',
     async run(input) {
-        const model = ModelConfig.getModel();
-        const lang = ModelConfig.getLanguage();
+        const model = Config.getModel();
+        const lang = Config.getLanguage();
         const prompt = `
 You are a senior ${lang.toUpperCase()} engineer reviewing source code.
 

package/dist/pipeline/modules/commitSuggesterModule.js

@@ -1,10 +1,10 @@
 import { generate } from '../../lib/generate.js';
-import { ModelConfig } from '../../config/ModelConfig.js';
+import { Config } from '../../config.js';
 export const commitSuggesterModule = {
     name: 'commitSuggester',
     description: 'Suggests conventional commit messages from Git diff',
     async run({ content }) {
-        const model = ModelConfig.getModel();
+        const model = Config.getModel();
         const prompt = `
 Suggest ALWAYS 3 concise, conventional Git commit messages based on the input code diff.
 Use this format ONLY:

package/dist/pipeline/modules/generateTestsModule.js

@@ -1,14 +1,14 @@
 // src/pipeline/modules/generateTestsModule.ts
 import fs from 'fs/promises';
 import path from 'path';
-import { ModelConfig } from '../../config/ModelConfig.js';
+import { Config } from '../../config.js';
 import { generate } from '../../lib/generate.js';
 export const generateTestsModule = {
     name: 'generateTests',
     description: 'Generate a Jest test file for the class/module',
     async run({ content, filepath }) {
-        const model = ModelConfig.getModel();
-        const lang = ModelConfig.getLanguage();
+        const model = Config.getModel();
+        const lang = Config.getLanguage();
         if (!filepath)
             throw new Error('Missing filepath in pipeline context');
         const prompt = `

package/dist/pipeline/modules/refactorModule.js

@@ -1,11 +1,11 @@
-import { ModelConfig } from '../../config/ModelConfig.js';
+import { Config } from '../../config.js';
 import { generate } from '../../lib/generate.js';
 export const refactorModule = {
     name: 'refactor',
     description: 'Break code into small, clean functions',
     async run(input) {
-        const model = ModelConfig.getModel();
-        const lang = ModelConfig.getLanguage();
+        const model = Config.getModel();
+        const lang = Config.getLanguage();
         const prompt = `
 You are a senior ${lang.toUpperCase()} engineer.
 

package/dist/pipeline/modules/summaryModule.js

@@ -1,11 +1,11 @@
-import { ModelConfig } from '../../config/ModelConfig.js';
+import { Config } from '../../config.js';
 import { generate } from '../../lib/generate.js';
 import path from 'path';
 export const summaryModule = {
     name: 'summary',
     description: 'Generates a general summary of any file content.',
     run: async ({ content, filepath }) => {
-        const model = ModelConfig.getModel();
+        const model = Config.getModel();
         const ext = filepath ? path.extname(filepath).toLowerCase() : '';
         const filename = filepath ? path.basename(filepath) : '';
         // More neutral prompt for general-purpose content
@@ -36,7 +36,6 @@ ${content}
         }
         else {
             console.warn('⚠️ No summary generated.');
-            response.summary = '⚠️ No summary generated.';
         }
         return response;
     }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "scai",
-  "version": "0.1.19",
+  "version": "0.1.21",
   "type": "module",
   "bin": {
     "scai": "./dist/index.js"

package/dist/config/ModelConfig.js

@@ -1,23 +0,0 @@
-export class ModelConfig {
-    static setModel(model) {
-        this.model = model;
-        console.log(`📦 Model set to: ${model}`);
-    }
-    static getModel() {
-        return this.model;
-    }
-    static setLanguage(lang) {
-        this.language = lang;
-        console.log(`🗣️ Language set to: ${lang}`);
-    }
-    static getLanguage() {
-        return this.language;
-    }
-    static logCurrentConfig() {
-        console.log(`🔧 Current configuration:`);
-        console.log(`   Model   : ${this.model}`);
-        console.log(`   Language: ${this.language}`);
-    }
-}
-ModelConfig.model = 'codellama:7b';
-ModelConfig.language = 'ts';