scai 0.1.163 → 0.1.165

package/README.md

@@ -1,98 +1,196 @@
 # ⚙️ SCAI — Source Code AI 🌿
 
-> **AI-powered CLI for local code analysis, commit message suggestions, and natural-language queries.**
-> **100% local • No token cost • Private by design • GDPR-friendly** — made in Denmark/EU with ❤️.
+> **A local-first AI CLI for understanding, querying, and iterating on large codebases.**
+> **100% local • No token costs • No cloud • No prompt injection • Private by design**
 
 🔗 **Website:** [https://scai.dk](https://scai.dk)
+🇪🇺 Built in Denmark / EU
 
-SCAI is your AI coding companion in the terminal. Stay focused on coding while SCAI helps you understand, analyze, and reason about your codebase using local language models.
+---
+
+## What is SCAI?
+
+**SCAI** is an AI-powered command-line tool that helps developers explore and reason about source code using **local large language models only**.
+
+Inspired by tools such as *Claude Code* and *Gemini CLI*, SCAI is designed to feel like a natural extension of the terminal. It enables natural-language interaction with your codebase while deliberately avoiding cloud dependencies and network-connected agents.
+
+SCAI runs entirely on your local system:
+
+* **No token costs** — no usage-based pricing
+* **No internet access for agents**
+* **No prompt injection from web content**
+* No external AI APIs
+* No telemetry or tracking
+* No API keys
+
+Your code never leaves your machine. All analysis and reasoning happens locally.
+
+> **Local model tradeoff**
+> SCAI uses local LLMs. Output quality depends on your hardware and selected model. Cloud-hosted systems may perform better on general reasoning tasks, but SCAI prioritizes privacy, predictability, and control.
+
+---
+
+## ⚠️ Alpha Status
+
+SCAI is currently in **alpha**.
+
+If you have previously installed SCAI, reset the local database before upgrading:
+
+```bash
+scai db reset
+scai index start
+```
+
+Breaking changes and evolving behavior should be expected.
+
+---
+
+## Why SCAI?
+
+### 🔐 Local-Only by Design
+
+SCAI agents operate **entirely offline**.
+
+They do not:
+
+* Browse the web
+* Fetch URLs
+* Ingest external documents
+* Execute remote prompts
 
-**Local Model Note:** SCAI runs entirely on local LLMs. This means **no API keys, no token cost, and full privacy**, but also **more limited capabilities** compared to cloud-hosted AI.
+**Security implications:**
 
-> ⚠️ **Alpha Version Notice**
-> If you have previously installed SCAI, please run:
->
-> bash
-> scai db reset && scai index start
-> 
->
-> before using this version.
+* No prompt injection via web content
+* No data exfiltration
+* No hidden network calls
+* Fully auditable execution
+
+This makes SCAI suitable for **private repositories, regulated environments, and GDPR-compliant workflows**.
 
 ---
 
-## 🧠 Why SCAI?
+### 🧠 Codebase-Aware Analysis
+
+SCAI builds and maintains a structured internal representation of your repository using:
+
+* Language-aware parsing
+* Symbol and dependency indexing
+* Static and heuristic analysis
+* Cross-file context tracking
+
+This enables repository-level questions that go beyond single-file inspection.
 
-SCAI is not just another AI tool — it's a **developer-first**, **privacy-focused**, and **local-first** coding companion. Here's why SCAI stands out:
+---
 
-### 🔐 **100% Local & Private by Design**
+### ✂️ Assisted Code Iteration (Early)
 
-Unlike cloud-based AI tools, SCAI runs entirely on your machine. No data leaves your environment — making it ideal for **sensitive codebases** and **GDPR-compliant workflows**.
+SCAI can assist with **lightweight, example-driven code iteration**, primarily focused on understanding and improving existing code rather than large-scale automated refactoring.
 
-### 🧠 **Deep Code Understanding**
+Current strengths include:
 
-SCAI doesn't just parse code — it **understands** it. With background indexing, static analysis, and language-aware parsing, SCAI helps you explore, refactor, and debug with confidence.
+* Explaining what functions, files, or modules do
+* Identifying patterns and responsibilities across files
+* Generating or improving comments and documentation
+* Highlighting structural or readability issues
+* Suggesting small, localized improvements
 
-### 📦 **No Token Costs or API Keys**
+Changes are **guided by indexed context and user prompts**, and are intended to support human review and decision-making.
 
-SCAI works offline, with **zero token usage**. You don’t pay for API calls or subscribe to cloud services. Just install and go.
+Large-scale or fully automated repository-wide refactoring should currently be considered **experimental**.
 
-### 🛠️ **Developer-Focused Toolset**
+---
 
-From commit message generation to architecture summaries, SCAI integrates directly into your workflow. It's built for developers, by developers.
+### 🛠 Built for Developer Workflows
 
-### 🇩🇰 **Built in Denmark/EU**
+SCAI is a **terminal-native tool** designed to integrate cleanly into daily development:
 
-SCAI is developed in the European Union, ensuring compliance with data protection laws and a focus on privacy-first development.
+* Natural-language queries over your codebase
+* Code understanding and exploration
+* Assisted iteration and suggestions
+* Commit message generation
+* Background indexing and analysis
+* Interactive REPL
 
-> ✅ SCAI is your **AI coding assistant that respects your privacy**, enhances your productivity, and works **entirely offline**.
+No browser UI. No cloud login. No vendor lock-in.
 
 ---
 
-## 🗣️ Language Support (Important)
+### 🇪🇺 Privacy & Compliance First
 
-SCAI is currently **tested and validated only on the following languages**:
+* Fully local execution
+* No telemetry
+* No cloud services
+* Developed in Denmark / EU
+* GDPR-friendly by default
+
+---
 
-* **JavaScript (JS)**
-* **TypeScript (TS)**
+## Language Support
+
+SCAI is currently **tested and supported** for:
+
+* **JavaScript**
+* **TypeScript**
 * **Java**
 
-Other languages may work partially, but analysis quality, indexing accuracy, and agent behavior are **not guaranteed** outside these languages. Broader language support is planned, but for now SCAI should be considered **JS/TS/Java-first**.
+Other languages may work partially, but indexing quality, analysis accuracy, and agent behavior are **not guaranteed**.
+
+SCAI should currently be considered **JS / TS / Java-first**.
 
 ---
 
-## 💻 Getting Started
+## Getting Started
 
-### 1️⃣ Install & Initialize
+### Install & Initialize
 
-bash
+```bash
 npm install -g scai
 scai init
 scai index start
+```
 
+This:
 
-This initializes local models (recommended: `qwen3-coder:30b`) and starts indexing your code repository.
+* Initializes local configuration
+* Starts the background daemon
+* Begins indexing the current repository
 
-> ⏳ **Note:** Initial indexing and analysis can take **minutes to hours** depending on repository size and enabled analysis tools.
+> **Note**
+> Initial indexing can take **minutes to hours**, depending on repository size and enabled analysis.
 
-### 2️⃣ Check Available Commands
+---
+
+### Starting SCAI
+
+Running the `scai` command with no arguments starts the interactive shell:
 
 ```bash
-scai --help
+scai
 ```
 
----
+You can also start it explicitly:
+
+```bash
+scai shell
+```
 
-## 🏠 REPL Mode (Local AI Queries)
+---
 
-Start an interactive REPL to ask natural-language questions about your code:
+### View Available Commands
 
 ```bash
-scai shell
+scai --help
 ```
 
-Once in the REPL, you can:
+---
+
+## Interactive REPL
+
+The REPL is the primary interface for working with SCAI.
+
+### Ask questions about your codebase
 
-### Ask questions about your codebase. Be specific for better results. 
+Be specific for better results.
 
 ```text
 scai> what does withContext function do in index.ts file?
@@ -103,32 +201,28 @@ scai> Where are all the database queries defined?
 scai> List files involved in authentication
 ```
 
-### Run CLI commands from inside the REPL
+### Run CLI commands inside the REPL
 
 ```text
-scai> /git commit
-scai> /index list
-scai> /index set /path/to/repo
-scai> /index switch
-scai> /index delete
+/index list
+/index switch
+/git commit
 ```
 
 ### Execute shell commands
 
 ```text
-scai> !ls -la
-scai> !git status
+!git status
+!ls -la
 ```
 
-> ✅ REPL queries are free, offline, GDPR-friendly, and **no token cost**.
+All interactions remain **offline and free**, with **no token usage**.
 
 ---
 
-## 📦 Repository Indexing
+## Repository Indexing
 
-Before SCAI can answer questions, your repository must be indexed.
-
-### Common Index Commands
+Repositories must be indexed before querying:
 
 ```bash
 scai index set /path/to/repo
@@ -138,47 +232,43 @@ scai index switch
 scai index delete
 ```
 
-Only indexed repositories can be queried.
+Only indexed repositories are accessible to agents.
 
 ---
 
-## 🧠 Background Indexing & Analysis (Daemon)
-
-SCAI performs **deep repository indexing and static analysis** using background workers. This includes:
+## Background Analysis (Daemon)
 
-* File structure discovery
-* Language-aware parsing (JS / TS / Java)
-* Symbol and dependency mapping
-* Heuristic analysis for tests, architecture, and patterns
+SCAI performs deep analysis in the background, including:
 
-⚠️ **Important:** On first install or on large repositories, this process can take **several hours**.
+* File discovery
+* AST parsing
+* Dependency graph construction
+* Symbol resolution
+* Heuristic structure analysis
 
-All background work is handled by the **SCAI daemon**, which can be fully controlled from the CLI.
-
-### Daemon Commands
+Daemon control:
 
 ```bash
 scai daemon start
 scai daemon stop
 scai daemon restart
 scai daemon status
-scai daemon unlock
 scai daemon logs
 ```
 
-You can safely stop the daemon at any time. Indexing and analysis will resume when restarted.
+Indexing progress resumes automatically after restart.
 
 ---
 
-## ⚙️ Configuration
+## Configuration
 
-Set the local AI model (recommended):
+Set a local model (recommended):
 
 ```bash
 scai config set-model qwen3-coder:30b
 ```
 
-View current configuration:
+View configuration:
 
 ```bash
 scai config show --raw
@@ -186,22 +276,22 @@ scai config show --raw
 
 ---
 
-## 🔧 Git Commit Assistant
+## Git Commit Assistant
 
-Generate meaningful commit messages based on staged changes:
+Generate commit messages from staged changes:
 
 ```bash
 git add .
 scai git commit
 ```
 
-All analysis is performed locally — **no token usage, no cloud calls**.
+All diff inspection and reasoning is performed locally.
 
 ---
 
-## 🔑 GitHub Authentication
+## GitHub Authentication
 
-For GitHub-related features, SCAI requires a Personal Access Token.
+Required only for GitHub-related features:
 
 ```bash
 scai auth set
@@ -211,98 +301,78 @@ scai auth reset
 
 ---
 
-## 🧠 Example Queries
-
-* `Summarize codeTransform.js`
-* `Explain utils/helpers.ts architecture`
-* `List all functions without tests in services/`
-* `Show where database queries are defined`
-* `Highlight potential memory leaks`
-* `Describe how authentication works`
-* `Summarize repo architecture`
-
----
-
-## 🔐 Privacy & GDPR
+## Privacy & Security Summary
 
-* Fully local — no cloud calls
+* 100% local execution
+* No internet access for agents
+* No prompt injection from web content
 * No API keys
-* **No token cost**
-* GDPR-friendly, built in Denmark/EU 🇩🇰
+* No token costs
+* GDPR-friendly by default
 
 ---
 
-## 🙌 Feedback & Support
-
-Feedback, bugs, and ideas are very welcome:
+## Feedback & Community
 
-* 🌍 Website: [https://scai.dk](https://scai.dk)
-* 🧵 Threads: [@scai.dk](https://threads.net/@scai.dk)
-
-<br>
+* 🌍 [https://scai.dk](https://scai.dk)
+* 🧵 [https://threads.net/@scai.dk](https://threads.net/@scai.dk)
 
 ---
 
-<br>
-<br>
-
-## 🔐 License & Usage Terms
+# License & Usage Terms
 
-Copyright © SCAI
-All rights reserved.
+© SCAI — All rights reserved.
 
-SCAI is **free to use for non-commercial purposes only**, subject to the terms below.
+SCAI is **free for non-commercial use only**.
 
 ---
 
-## ✅ Permitted Use
+## Permitted Use
 
-You may use SCAI **without charge** for the following purposes:
+You may use SCAI free of charge for:
 
 * Personal projects
 * Educational use
 * Research and experimentation
-* Non-commercial open-source contributions
-* Internal evaluation or proof-of-concept work
+* Non-commercial open-source work
+* Internal evaluation or proof-of-concepts
 
-You may also **fork, modify, and redistribute** the source code, provided that such use remains **strictly non-commercial**.
+You may fork and modify the source code **for non-commercial purposes only**.
 
 ---
 
-## 🚫 Restricted Use
+## Restricted Use
 
-The following uses are **not permitted without a commercial license**:
+The following require a **commercial license**:
 
-* Commercial use of any kind
-* Enterprise or organizational deployment
-* Use as part of a paid product, service, or subscription
-* Use in consultancy, client work, or billable services
-* Bundling or integrating SCAI into commercial software or internal enterprise tooling
-* Resale, sublicensing, or redistribution for commercial purposes
+* Any commercial or enterprise use
+* Consultancy or client work
+* Paid products or services
+* Internal enterprise tooling
+* Commercial redistribution or resale
 
 ---
 
-## 🏢 Commercial & Enterprise Licensing
+## Commercial Licensing
 
-**Commercial and enterprise use requires a paid license and explicit permission from the author.**
+Commercial and enterprise use requires a **paid license** and explicit permission from the author.
 
-Organizations or individuals wishing to use SCAI in a commercial context must obtain a commercial license **prior to such use**.
-
-Please contact the author to discuss commercial licensing terms.
+Please contact the author to discuss licensing terms.
 
 ---
 
-## ⚖️ Disclaimer
+## Disclaimer
 
-This software is provided **"as is"**, without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose, and non-infringement.
+This software is provided **“as is”**, without warranty of any kind.
 
-In no event shall the author be liable for any claim, damages, or other liability, whether in an action of contract, tort, or otherwise, arising from, out of, or in connection with the software or the use or other dealings in the software.
+The author is not liable for any damages arising from its use.
 
 ---
 
-### 📄 Summary (Non-Binding)
+### Non-Binding Summary
 
 * Free for personal and non-commercial use
-* Commercial and enterprise use requires a paid license
-* Commercial redistribution is prohibited
-
+* Fully local, offline AI
+* No token costs
+* No prompt injection surface
+* Commercial use requires a license

package/dist/db/fileIndex.js

@@ -100,54 +100,143 @@ export function queryFiles(safeQuery, limit = 10) {
 // - Uses LLM aggressively
 // - Optimizes for precision
 // --------------------------------------------------
-export async function semanticSearchFiles(originalQuery, query, topK = 5) {
+export async function semanticSearchFiles(originalQuery, _query, // ignored now – LLM owns query construction
+topK = 5) {
     const db = getDbForRepo();
-    const seen = new Map();
-    // -----------------------------
-    // Primary FTS
-    // -----------------------------
-    const safeQuery = sanitizeQueryForFts(query);
+    // --------------------------------------------------
+    // 1. LLM → primary FTS query (always)
+    // --------------------------------------------------
+    const primaryFtsQuery = await generatePrimaryFtsQuery(originalQuery);
+    logInputOutput("semanticSearchFiles LLM primary query", "output", {
+        originalQuery,
+        ftsQuery: primaryFtsQuery,
+    });
+    // --------------------------------------------------
+    // 2. Run primary FTS once
+    // --------------------------------------------------
     const primaryResults = db
         .prepare(sqlTemplates.searchFilesTemplate)
-        .all(safeQuery, RELATED_FILES_LIMIT);
-    logInputOutput("semanticSearchFiles primary FTS", "input", {
-        safeQuery,
-        count: primaryResults.length,
-    });
-    if (primaryResults.length === 0) {
-        // No FTS at all → blind expansion
-        return semanticFallbackSearch(originalQuery, topK);
+        .all(primaryFtsQuery, RELATED_FILES_LIMIT);
+    if (primaryResults.length > 0) {
+        return rankAndMap(new Map(primaryResults.map(r => [r.id, r])), topK);
     }
-    // -----------------------------
-    // Relevance check (KEY STEP)
-    // -----------------------------
-    const relevance = await checkFtsRelevanceWithModel(originalQuery, primaryResults);
-    logInputOutput("semanticSearchFiles relevance check", "output", relevance);
-    // -----------------------------
-    // If relevant → trust FTS
-    // -----------------------------
-    if (relevance.relevant) {
-        primaryResults.forEach(r => seen.set(r.id, r));
-        return rankAndMap(seen, topK);
-    }
-    // -----------------------------
-    // Not relevant → model-guided expansion
-    // -----------------------------
-    for (const term of relevance.suggestedTerms) {
-        const safeTerm = sanitizeQueryForFts(term);
+    // --------------------------------------------------
+    // 3. Fallback: LLM → 2–3 subqueries (ONLY if zero results)
+    // --------------------------------------------------
+    const subQueries = await generateFallbackFtsQueries(originalQuery, primaryFtsQuery);
+    logInputOutput("semanticSearchFiles LLM fallback queries", "output", {
+        originalQuery,
+        primaryFtsQuery,
+        subQueries,
+    });
+    // --------------------------------------------------
+    // 4. Execute fallback queries sequentially
+    // --------------------------------------------------
+    for (const subQuery of subQueries) {
         const rows = db
             .prepare(sqlTemplates.searchFilesTemplate)
-            .all(safeTerm, RELATED_FILES_LIMIT);
-        for (const row of rows) {
-            if (!seen.has(row.id))
-                seen.set(row.id, row);
+            .all(subQuery, RELATED_FILES_LIMIT);
+        if (rows.length > 0) {
+            return rankAndMap(new Map(rows.map(r => [r.id, r])), topK);
+        }
+    }
+    // --------------------------------------------------
+    // 5. Hard stop
+    // --------------------------------------------------
+    return [];
+}
+async function generatePrimaryFtsQuery(userQuery) {
+    const prompt = `
+You are generating a SQLite FTS query for searching a source code repository.
+
+Input (natural language):
+"${userQuery}"
+
+Task:
+- Produce ONE concise FTS query
+- Focus on filenames, symbols, module names, domain nouns
+- Prefer literal identifiers likely to exist in code
+- NO sentences
+- NO stopwords
+- NO explanations
+- NO wildcards unless absolutely necessary
+- Use OR between terms
+- **MAX 10 terms only** — be selective and concise
+
+Output JSON ONLY:
+{
+  "ftsQuery": "term1 OR term2 OR term3"
+}
+`.trim();
+    try {
+        const response = await generate({ content: prompt, query: "" });
+        const cleaned = await cleanupModule.run({
+            query: userQuery,
+            content: response.data,
+        });
+        if (cleaned.data &&
+            typeof cleaned.data === "object" &&
+            "ftsQuery" in cleaned.data &&
+            typeof cleaned.data.ftsQuery === "string") {
+            return cleaned.data.ftsQuery;
         }
     }
-    // Safety net: never return empty if we had FTS
-    if (seen.size === 0) {
-        primaryResults.forEach(r => seen.set(r.id, r));
+    catch (err) {
+        log(`⚠️ [semanticSearchFiles] Failed to generate primary FTS query: ${String(err)}`);
     }
-    return rankAndMap(seen, topK);
+    // Absolute safety fallback — never explode
+    return sanitizeQueryForFts(userQuery);
+}
+async function generateFallbackFtsQueries(userQuery, failedQuery) {
+    const prompt = `
+You are generating fallback SQLite FTS queries for a source code search.
+
+Original user query:
+"${userQuery}"
+
+Primary FTS query returned ZERO results:
+"${failedQuery}"
+
+Task:
+- Generate 2–3 independent FTS queries (MAX 3)
+- Each query should be concise: no more than 10 OR-joined search terms
+- Focus on filenames, symbols, module names
+- Avoid natural-language sentences
+- Avoid recursion or refinement loops
+- Use OR between terms
+
+Output JSON ONLY:
+{
+  "subQueries": [
+    "query1",
+    "query2",
+    "query3"
+  ]
+}
+`.trim();
+    try {
+        const response = await generate({ content: prompt, query: "" });
+        const cleaned = await cleanupModule.run({
+            query: userQuery,
+            content: response.data,
+        });
+        if (cleaned.data &&
+            typeof cleaned.data === "object" &&
+            Array.isArray(cleaned.data.subQueries)) {
+            return cleaned.data.subQueries
+                .filter((q) => typeof q === "string")
+                .slice(0, 3) // cap to 3 queries
+                .map((q) => q
+                .split(' OR ')
+                .map(term => sanitizeQueryForFts(term)) // sanitize each term individually
+                .slice(0, 10) // cap terms per query
+                .join(' OR '));
+        }
+    }
+    catch (err) {
+        log(`⚠️ [semanticSearchFiles] Failed to generate fallback queries: ${String(err)}`);
+    }
+    return [];
 }
 // --------------------------------------------------
 // PLANNER SEARCH (fileSearchModule, discovery)
@@ -209,71 +298,6 @@ function rankAndMap(seen, topK) {
         bm25Score: r.bm25Score,
     }));
 }
-/**
- * Ask the model whether the initial FTS hits are relevant to the query.
- * If not, suggest new search terms for another FTS pass.
- */
-async function checkFtsRelevanceWithModel(query, ftsResults) {
-    if (ftsResults.length === 0) {
-        return { relevant: false, suggestedTerms: [] };
-    }
-    const prompt = `
-You are assisting a code search system that uses full-text search (FTS)
-over source code.
-
-Query (natural language):
-"${query}"
-
-Initial FTS results (filenames and summaries):
-${JSON.stringify(ftsResults)}
-
-Task:
-1. Decide whether these results are relevant to the query.
-2. If they are NOT relevant, suggest alternative search terms.
-
-IMPORTANT RULES FOR SUGGESTED TERMS:
-- Terms MUST be likely to appear literally in source code.
-- Prefer: filenames, module names, function names, variables, symbols, config keys.
-- Use short identifiers (1–3 words max).
-- Avoid natural-language phrases or explanations.
-- Avoid conceptual or architectural descriptions.
-- Examples of GOOD terms: "api", "router", "frontend", "backend", "client", "server", "routes", "config.ts"
-- Examples of BAD terms: "frontend backend separation", "code architecture", "business logic"
-
-Output format:
-- If relevant:
-  { "relevant": true, "suggestedTerms": [] }
-- If not relevant:
-  { "relevant": false, "suggestedTerms": ["term1", "term2", "term3"] }
-
-Return ONLY valid JSON.
-`.trim();
-    try {
-        const response = await generate({ content: prompt, query: "" });
-        const cleaned = await cleanupModule.run({
-            query,
-            content: response.data,
-        });
-        const data = cleaned.data;
-        // Type guard: ensure it's an object with correct properties
-        if (data &&
-            typeof data === "object" &&
-            "relevant" in data &&
-            "suggestedTerms" in data &&
-            typeof data.relevant === "boolean" &&
-            Array.isArray(data.suggestedTerms)) {
-            const relevant = data.relevant;
-            const suggestedTerms = data.suggestedTerms.filter((t) => typeof t === "string");
-            return { relevant, suggestedTerms };
-        }
-        return { relevant: false, suggestedTerms: [] };
-    }
-    catch (err) {
-        log(`⚠️ [searchFiles] Failed to check FTS relevance: ${String(err)}`);
-        return { relevant: false, suggestedTerms: [] };
-    }
-}
-;
 async function expandQueryWithModel(query) {
     const prompt = `
 You are assisting a code search system.
@@ -305,25 +329,3 @@ Question:
         return [];
     }
 }
-async function semanticFallbackSearch(query, topK) {
-    const db = getDbForRepo();
-    const seen = new Map();
-    const llmTerms = await expandQueryWithModel(query);
-    logInputOutput("semanticSearchFiles blind expansion", "output", {
-        query,
-        suggestedTerms: llmTerms,
-    });
-    for (const term of llmTerms) {
-        const safeTerm = sanitizeQueryForFts(term);
-        const rows = db
-            .prepare(sqlTemplates.searchFilesTemplate)
-            .all(safeTerm, RELATED_FILES_LIMIT);
-        for (const row of rows) {
-            if (!seen.has(row.id))
-                seen.set(row.id, row);
-        }
-    }
-    if (seen.size === 0)
-        return [];
-    return rankAndMap(seen, topK);
-}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "scai",
-  "version": "0.1.163",
+  "version": "0.1.165",
   "type": "module",
   "bin": {
     "scai": "./dist/index.js"