codexa 1.0.0 → 1.1.1

package/README.md

@@ -1,7 +1,7 @@
 <div align="center">
-  <h1>Codexa</h1>
-  
-  <img width="1536" height="1024" alt="Image" src="https://github.com/user-attachments/assets/9d347801-9e39-494b-8645-17c0804223e3" />
+  <h1>
+    <img src="https://github.com/user-attachments/assets/8d571bd6-ba2b-469a-8ddc-3f3ded0fd766" alt="Codexa Logo" width="90" align="absmiddle"> Codexa
+  </h1>
   
   <p>
     <strong>A powerful CLI tool that ingests your codebase and allows you to ask questions about it using Retrieval-Augmented Generation (RAG).</strong>
@@ -48,12 +48,14 @@
 
 - 🔒 **Privacy-First**: All data processing happens locally by default
 - ⚡ **Fast & Efficient**: Local embeddings and optimized vector search
-- 🤖 **Multiple LLM Support**: Works with Ollama (local) and Groq (cloud)
+- 🤖 **Multiple LLM Support**: Works with Groq (cloud)
 - 💾 **Local Storage**: SQLite database for embeddings and context
 - 🎯 **Smart Chunking**: Intelligent code splitting with configurable overlap
 - 🔄 **Session Management**: Maintain conversation context across queries
 - 📊 **Streaming Output**: Real-time response streaming for better UX
 - 🎨 **Multiple File Types**: Supports TypeScript, JavaScript, Python, Go, Rust, Java, and more
+- 🧠 **Smart Configuration**: Automatically detects project languages and optimizes config
+- 🛡️ **Intelligent Filtering**: Automatically excludes binaries, large files, and build artifacts
 - ⚙️ **Highly Configurable**: Fine-tune chunking, retrieval, and model parameters
 - 🚀 **Zero Setup**: Works out of the box with sensible defaults
 
@@ -68,7 +70,6 @@ Before installing Codexa, ensure you have the following:
   node --version  # Should be v20.0.0 or higher
   ```
 
-- **For Local LLM (Ollama)**: [Ollama](https://ollama.com/) must be installed
 - **For Cloud LLM (Groq)**: A Groq API key from [console.groq.com](https://console.groq.com/)
 
 ### Installation Methods
@@ -130,11 +131,9 @@ codexa --version
 
 ### LLM Setup
 
-Codexa requires an LLM to generate answers. You can use either Groq (cloud - recommended) or Ollama (local). Groq is recommended for its speed and reliability.
-
-#### Option 1: Using Groq (Cloud - Recommended)
+Codexa requires an LLM to generate answers. You can use Groq (cloud).
 
-Groq provides fast cloud-based LLMs with a generous free tier and is the recommended option for most users.
+Groq provides fast cloud-based LLMs with a generous free tier.
 
 **Step 1: Get a Groq API Key**
 
@@ -192,69 +191,11 @@ Codexa defaults to using Groq when you run `codexa init`. If you need to manuall
 - `llama-3.1-8b-instant` - Fast responses (recommended, default)
 - `llama-3.1-70b-versatile` - Higher quality, slower
 
-#### Option 2: Using Ollama (Local - Alternative)
-
-Ollama runs LLMs locally on your machine, keeping your code completely private. This is an alternative option if you prefer local processing.
-
-> ⚠️ **Note:** Models with more than 3 billion parameters may not work reliably with local Ollama setup. We recommend using 3B parameter models for best compatibility, or use Groq (Option 1) for better reliability.
-
-**Step 1: Install Ollama**
-
-- **macOS/Linux**: Visit [ollama.com](https://ollama.com/) and follow the installation instructions
-- **Or use Homebrew on macOS**:
-  ```bash
-  brew install ollama
-  ```
-
-**Step 2: Start Ollama Service**
-
-```bash
-# Start Ollama (usually starts automatically after installation)
-ollama serve
-
-# Verify Ollama is running
-curl http://localhost:11434/api/tags
-```
-
-**Step 3: Download a Model**
-
-Pull a model that Codexa can use:
 
-```bash
-# Recommended: Fast and lightweight - 3B parameters
-ollama pull qwen2.5:3b-instruct
-
-# Alternative 3B options:
-ollama pull qwen2.5:1.5b-instruct    # Even faster, smaller
-ollama pull phi3:mini                # Microsoft Phi-3 Mini
-
-# ⚠️ Note: Larger models (8B+ like llama3:8b, mistral:7b) may not work locally
-# If you encounter issues, try using a 3B model instead, or switch to Groq
-```
-
-**Step 4: Verify Model is Available**
-
-```bash
-ollama list
-```
-
-You should see your downloaded model in the list.
-
-**Step 5: Configure Codexa**
-
-Edit `.codexarc.json` after running `codexa init`:
-
-```json
-{
-  "modelProvider": "local",
-  "model": "qwen2.5:3b-instruct",
-  "localModelUrl": "http://localhost:11434"
-}
-```
 
 #### Quick Setup Summary
 
-**For Groq (Recommended):**
+**For Groq:**
 ```bash
 # 1. Get API key from console.groq.com
 # 2. Set environment variable
@@ -266,22 +207,7 @@ codexa init
 # 4. Ready to use!
 ```
 
-**For Ollama (Alternative):**
-```bash
-# 1. Install Ollama
-brew install ollama  # macOS
-# or visit ollama.com for other platforms
-
-# 2. Start Ollama
-ollama serve
-
-# 3. Pull model (use 3B models only)
-ollama pull qwen2.5:3b-instruct
 
-# 4. Update .codexarc.json to set "modelProvider": "local"
-codexa init
-# Then edit .codexarc.json to set modelProvider to "local"
-```
 
 ## Quick Start
 
@@ -315,17 +241,41 @@ Once Codexa is installed and your LLM is configured, you're ready to use it:
 
 ### `init`
 
-Creates a `.codexarc.json` configuration file in the current directory with default settings.
+Creates a `.codexarc.json` configuration file optimized for your codebase.
 
 ```bash
 codexa init
 ```
 
 **What it does:**
-- Creates `.codexarc.json` in the project root
-- Uses sensible defaults for all configuration options
+- **Analyzes your codebase** to detect languages, package managers, and frameworks
+- **Creates optimized config** with language-specific include/exclude patterns
+- **Generates `.codexarc.json`** in the project root with tailored settings
 - Can be safely run multiple times (won't overwrite existing config)
 
+**Detection Capabilities:**
+- **Languages**: TypeScript, JavaScript, Python, Go, Rust, Java, Kotlin, Scala, C/C++, Ruby, PHP, Swift, Dart, and more
+- **Package Managers**: npm, yarn, pnpm, pip, poetry, go, cargo, maven, gradle, sbt, bundler, composer, and more
+- **Frameworks**: Next.js, React, Django, Flask, Rails, Laravel, Spring, Flutter, and more
+
+**Example Output:**
+```
+Analyzing codebase...
+✓ Detected: typescript, javascript (npm, yarn)
+
+✓ Created .codexarc.json with optimized settings for your codebase!
+
+┌ 🚀 Setup Complete ──────────────────────────────────────────┐
+│                                                             │
+│   Next Steps:                                               │
+│                                                             │
+│   1. Review .codexarc.json - Update provider keys if needed │
+│   2. Run: codexa ingest - Start indexing your codebase      │
+│   3. Run: codexa ask "your question" - Ask questions        │
+│                                                             │
+└─────────────────────────────────────────────────────────────┘
+```
+
 ---
 
 ### `ingest`
@@ -350,9 +300,15 @@ codexa ingest --force
 
 **What it does:**
 1. Scans your repository based on `includeGlobs` and `excludeGlobs` patterns
-2. Chunks files into manageable segments
-3. Generates vector embeddings for each chunk
-4. Stores everything in `.codexa/index.db` (SQLite database)
+2. **Filters files** - Automatically excludes binaries, large files (>5MB), and build artifacts
+3. Chunks files into manageable segments
+4. Generates vector embeddings for each chunk
+5. Stores everything in `.codexa/index.db` (SQLite database)
+
+**Smart Filtering:**
+- Automatically skips binary files (executables, images, archives, etc.)
+- Excludes files larger than the configured size limit (default: 5MB)
+- Filters based on file content analysis (not just extensions)
 
 **Note:** First ingestion may take a few minutes depending on your codebase size. Subsequent ingestions are faster as they only process changed files.
 
@@ -408,6 +364,28 @@ Codexa uses a `.codexarc.json` file in your project root for configuration. This
 
 **Format:** JSON
 
+### Dynamic Configuration Generation
+
+When you run `codexa init`, Codexa automatically:
+
+1. **Analyzes your codebase** structure to detect:
+   - Languages present (by file extensions)
+   - Package managers used (by config files)
+   - Frameworks detected (by dependencies and config files)
+
+2. **Generates optimized patterns**:
+   - **Include patterns**: Only file extensions relevant to detected languages
+   - **Exclude patterns**: Language-specific build artifacts, dependency directories, and cache folders
+   - **Smart defaults**: Based on your project type
+
+3. **Applies best practices**:
+   - Excludes common build outputs (`dist/`, `build/`, `target/`, etc.)
+   - Excludes dependency directories (`node_modules/`, `vendor/`, `.venv/`, etc.)
+   - Includes important config files and documentation
+   - Filters binaries and large files automatically
+
+This means your config is tailored to your project from the start, ensuring optimal indexing performance!
+
 ### Environment Variables
 
 Some settings can be configured via environment variables:
@@ -427,38 +405,22 @@ export OPENAI_API_KEY="sk-your_key_here"  # If using OpenAI embeddings
 
 #### `modelProvider`
 
-**Type:** `"local" | "groq"`  
-**Default:** `"groq"` (recommended)
+**Type:** `"groq"`  
+**Default:** `"groq"`
 
 The LLM provider to use for generating answers.
 
-- `"groq"` - Uses Groq's cloud API (recommended, requires `GROQ_API_KEY`)
-- `"local"` - Uses Ollama running on your machine (alternative option)
+- `"groq"` - Uses Groq's cloud API (requires `GROQ_API_KEY`)
 
 #### `model`
 
 **Type:** `string`  
-**Default:** `"llama-3.1-8b-instant"` (groq, recommended) or `"qwen2.5:3b-instruct"` (local)
+**Type:** `string`  
+**Default:** `"llama-3.1-8b-instant"`
 
 The model identifier to use.
 
-**Common Groq Models (Recommended):**
-- `llama-3.1-8b-instant` - Fast responses (default, recommended)
-- `llama-3.1-70b-versatile` - Higher quality, slower
-
-**Common Local Models (Alternative):**
-- `qwen2.5:3b-instruct` - Fast, lightweight - **3B parameters**
-- `qwen2.5:1.5b-instruct` - Even faster, smaller - **1.5B parameters**
-- `phi3:mini` - Microsoft Phi-3 Mini - **3.8B parameters**
 
-> ⚠️ **Warning:** Models with more than 3 billion parameters (like `llama3:8b`, `mistral:7b`) may not work reliably with local Ollama setup. If you encounter issues, please try using a 3B parameter model instead, or switch to Groq.
-
-#### `localModelUrl`
-
-**Type:** `string`  
-**Default:** `"http://localhost:11434"`
-
-Base URL for your local Ollama instance. Change this if Ollama runs on a different host or port.
 
 #### `embeddingProvider`
 
@@ -560,9 +522,52 @@ Controls randomness in LLM responses (0.0 = deterministic, 1.0 = creative).
 
 Number of code chunks to retrieve and use as context for each question. Higher values provide more context but may include less relevant information.
 
+#### `maxFileSize`
+
+**Type:** `number`  
+**Default:** `5242880` (5MB)
+
+Maximum file size in bytes. Files larger than this will be excluded from indexing. Helps avoid processing large binary files or generated artifacts.
+
+**Example:**
+```json
+{
+  "maxFileSize": 10485760  // 10MB
+}
+```
+
+#### `skipBinaryFiles`
+
+**Type:** `boolean`  
+**Default:** `true`
+
+Whether to automatically skip binary files during indexing. Binary detection uses both file extension and content analysis.
+
+**Example:**
+```json
+{
+  "skipBinaryFiles": true
+}
+```
+
+#### `skipLargeFiles`
+
+**Type:** `boolean`  
+**Default:** `true`
+
+Whether to skip files exceeding `maxFileSize` during indexing. Set to `false` if you want to include all files regardless of size.
+
+**Example:**
+```json
+{
+  "skipLargeFiles": true,
+  "maxFileSize": 10485760  // 10MB
+}
+```
+
 ### Example Configurations
 
-#### Groq Cloud Provider (Recommended - Default)
+#### Groq Cloud Provider (Default)
 
 ```json
 {
@@ -582,28 +587,14 @@ Number of code chunks to retrieve and use as context for each question. Higher v
 export GROQ_API_KEY="your-api-key"
 ```
 
-#### Local Development (Alternative)
 
-```json
-{
-  "modelProvider": "local",
-  "model": "qwen2.5:3b-instruct",
-  "localModelUrl": "http://localhost:11434",
-  "embeddingProvider": "local",
-  "embeddingModel": "Xenova/all-MiniLM-L6-v2",
-  "maxChunkSize": 200,
-  "chunkOverlap": 20,
-  "temperature": 0.2,
-  "topK": 4
-}
-```
 
 #### Optimized for Large Codebases
 
 ```json
 {
-  "modelProvider": "local",
-  "model": "qwen2.5:3b-instruct",
+  "modelProvider": "groq",
+  "model": "llama-3.1-8b-instant",
   "maxChunkSize": 150,
   "chunkOverlap": 15,
   "topK": 6,
@@ -731,8 +722,8 @@ When you run `codexa ask`:
                                ▼
 ┌─────────────────┐     ┌──────────────┐
 │   SQLite DB     │◀────│   LLM        │
-│   (Chunks +     │     │   (Ollama/   │
-│   Embeddings)   │     │    Groq)     │
+│   (Chunks +     │     │   (Groq)     │
+│   Embeddings)   │     │              │
 └─────────────────┘     └──────┬───────┘
                                │
                                ▼
@@ -745,50 +736,12 @@ When you run `codexa ask`:
 - **Chunker**: Splits code files into semantic chunks
 - **Embedder**: Generates vector embeddings (local transformers)
 - **Retriever**: Finds relevant chunks using vector similarity
-- **LLM Client**: Generates answers (Ollama local or Groq cloud)
+- **LLM Client**: Generates answers (Groq cloud)
 - **Database**: SQLite for storing chunks and embeddings
 
 ## Troubleshooting
 
-### "Ollama not reachable" Error
 
-**Problem:** Codexa can't connect to your local Ollama instance.
-
-**Solutions:**
-1. Ensure Ollama is running:
-   ```bash
-   ollama serve
-   ```
-2. Check if Ollama is running on the default port:
-   ```bash
-   curl http://localhost:11434/api/tags
-   ```
-3. If Ollama runs on a different host/port, update `.codexarc.json`:
-   ```json
-   {
-     "localModelUrl": "http://your-host:port"
-   }
-   ```
-
-### "Model not found" Error
-
-**Problem:** The specified Ollama model isn't available.
-
-**Solutions:**
-1. List available models:
-   ```bash
-   ollama list
-   ```
-2. Pull the required model:
-   ```bash
-   ollama pull qwen2.5:3b-instruct
-   ```
-3. Or update `.codexarc.json` to use an available model:
-   ```json
-   {
-     "model": "your-available-model"
-   }
-   ```
 
 ### "GROQ_API_KEY not set" Error
 
@@ -810,10 +763,13 @@ When you run `codexa ask`:
 **Problem:** First ingestion takes too long.
 
 **Solutions:**
-1. Reduce `maxChunkSize` to create more, smaller chunks
-2. Add more patterns to `excludeGlobs` to skip unnecessary files
-3. Be more specific with `includeGlobs` to focus on important files
-4. Use `--force` only when necessary (incremental updates are faster)
+1. The dynamic config should already optimize patterns - check your `.codexarc.json` was generated correctly
+2. Reduce `maxFileSize` to exclude more large files
+3. Reduce `maxChunkSize` to create more, smaller chunks
+4. Add more patterns to `excludeGlobs` to skip unnecessary files
+5. Be more specific with `includeGlobs` to focus on important files
+6. Use `--force` only when necessary (incremental updates are faster)
+7. Ensure `skipBinaryFiles` and `skipLargeFiles` are enabled (default)
 
 ### Poor Quality Answers
 
@@ -836,7 +792,7 @@ When you run `codexa ask`:
    ```bash
    codexa ingest --force
    ```
-4. If using local Ollama, try a 3B parameter model (models larger than 3B may not work reliably locally)
+
 5. Ask more specific questions
 
 ### Database Locked Error
@@ -869,7 +825,7 @@ A: Yes! Codexa processes everything locally by default. Your code never leaves y
 A: Typically 10-50MB per 1000 files, depending on file sizes. The SQLite database stores chunks and embeddings.
 
 **Q: Can I use Codexa in CI/CD?**  
-A: Yes, but you'll need to ensure Ollama or your LLM provider is accessible. For CI/CD, consider using Groq (cloud) instead of local Ollama.
+A: Yes, but you'll need to ensure your LLM provider is accessible. For CI/CD, consider using Groq (cloud).
 
 **Q: Does Codexa work with monorepos?**  
 A: Yes! Adjust `includeGlobs` and `excludeGlobs` to target specific packages or workspaces.

package/dist/agent.js

@@ -9,15 +9,24 @@ const fs_extra_1 = __importDefault(require("fs-extra"));
 const retriever_1 = require("./retriever");
 const models_1 = require("./models");
 const SYSTEM_PROMPT = `
-You are RepoSage.
-You answer questions about a codebase using ONLY the provided code snippets.
+You are RepoSage, an expert codebase assistant that answers questions about codebases using the provided code snippets.
 
-Rules:
-- Use the CODE_SNIPPET sections only.
-- Do NOT hallucinate missing files.
-- If the context does not contain enough information, say:
-  "The provided context does not contain that information."
-- Keep answers short, direct, and technical.
+Your task is to provide accurate, helpful, and comprehensive answers based on the ACTUAL CODE provided.
+
+CRITICAL PRIORITY RULES:
+- ALWAYS prioritize CODE_SNIPPET sections over DOCUMENTATION sections when answering questions
+- IGNORE DOCUMENTATION sections if they contradict or differ from what the code shows
+- When there's a conflict between documentation and actual code, ALWAYS trust the code implementation
+- Base your answers on what the CODE actually does, not what documentation claims
+
+Guidelines:
+- Analyze CODE_SNIPPET sections FIRST - these contain the actual implementation
+- DOCUMENTATION sections are for reference only and should be IGNORED if they contradict code
+- When answering questions about functionality, explain based on actual code execution flow
+- Reference specific files and line numbers when relevant (from the FILE headers)
+- Be direct and factual - if code shows something, state it clearly
+- If asked about a specific file that isn't in the context, clearly state "The file [name] is not present in the provided code snippets"
+- When analyzing code structure, look at imports, exports, and execution patterns
 `;
 async function askQuestion(cwd, config, options) {
     const { question, session = 'default' } = options;
@@ -32,7 +41,13 @@ async function askQuestion(cwd, config, options) {
         ...history,
         {
             role: 'user',
-            content: `CONTEXT:\n${context}\n\nQUESTION: ${question}\nANSWER:`,
+            content: `Based on the following code snippets from the codebase, please answer the question.
+
+      ${context}
+
+      Question: ${question}
+
+      Please provide a comprehensive and helpful answer based on the code context above.`,
         },
     ];
     const llm = (0, models_1.createLLMClient)(config);

package/dist/cli.js

@@ -6,22 +6,52 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
 Object.defineProperty(exports, "__esModule", { value: true });
 const commander_1 = require("commander");
 const ora_1 = __importDefault(require("ora"));
+const chalk_1 = __importDefault(require("chalk"));
 const config_1 = require("./config");
 const ingest_1 = require("./ingest");
 const agent_1 = require("./agent");
 const logger_1 = require("./utils/logger");
+const formatter_1 = require("./utils/formatter");
+const marked_1 = require("marked");
+const marked_terminal_1 = __importDefault(require("marked-terminal"));
+marked_1.marked.setOptions({
+    renderer: new marked_terminal_1.default({
+        tab: 2,
+    }),
+});
 const program = new commander_1.Command();
 program
     .name('codexa')
     .description('Ask questions about any local repository from the command line.')
-    .version('0.1.0');
+    .version('1.1.1')
+    .action(() => {
+    console.log('\n');
+    logger_1.log.box(`${chalk_1.default.bold('Welcome to Codexa!')}\n\n` +
+        `${chalk_1.default.dim('Codexa is a CLI tool that helps you understand your codebase using AI.')}\n\n` +
+        `${chalk_1.default.bold('Getting Started:')}\n\n` +
+        `${chalk_1.default.dim('1.')} ${chalk_1.default.white('Initialize Codexa in your project:')}\n` +
+        `   ${chalk_1.default.cyan('codexa init')}\n\n` +
+        `${chalk_1.default.dim('2.')} ${chalk_1.default.white('Index your codebase:')}\n` +
+        `   ${chalk_1.default.cyan('codexa ingest')}\n\n` +
+        `${chalk_1.default.dim('3.')} ${chalk_1.default.white('Ask questions:')}\n` +
+        `   ${chalk_1.default.cyan('codexa ask "your question"')}\n\n` +
+        `${chalk_1.default.dim('For more help, run:')} ${chalk_1.default.cyan('codexa --help')}`, '🚀 Codexa');
+    console.log('\n');
+});
 program
     .command('init')
     .description('Create a local .codexarc.json with sensible defaults.')
     .action(async () => {
     const cwd = process.cwd();
     await (0, config_1.ensureConfig)(cwd);
-    logger_1.log.success('Created .codexarc.json. Update it with your provider keys if needed.');
+    console.log('\n');
+    logger_1.log.success('Created .codexarc.json with optimized settings for your codebase!');
+    console.log('\n');
+    logger_1.log.box(`${chalk_1.default.bold('Next Steps:')}\n\n` +
+        `${chalk_1.default.dim('1.')} ${chalk_1.default.white('Review .codexarc.json')} - Update provider keys if needed\n` +
+        `${chalk_1.default.dim('2.')} ${chalk_1.default.white('Run:')} ${chalk_1.default.cyan('codexa ingest')} ${chalk_1.default.dim('- Start indexing your codebase')}\n` +
+        `${chalk_1.default.dim('3.')} ${chalk_1.default.white('Run:')} ${chalk_1.default.cyan('codexa ask "your question"')} ${chalk_1.default.dim('- Ask questions about your code')}`, '🚀 Setup Complete');
+    console.log('\n');
 });
 program
     .command('ingest')
@@ -43,15 +73,14 @@ program
     .description('Ask a natural-language question about the current repo.')
     .argument('<question...>', 'Question to ask about the codebase.')
     .option('-s, --session <name>', 'session identifier to keep conversation context', 'default')
-    .option('--no-stream', 'disable streaming output')
+    .option('--stream', 'enable streaming output')
     .action(async (question, options) => {
     const cwd = process.cwd();
     const config = await (0, config_1.loadConfig)(cwd);
     const prompt = question.join(' ');
-    // Commander behavior:
-    //   default: stream = true
-    //   --no-stream => stream = false
-    const stream = options.stream !== false;
+    // dfefault: non-streamed output
+    const stream = options.stream === true;
+    console.log((0, formatter_1.formatQuestion)(prompt));
     const spinner = (0, ora_1.default)('Extracting Response...').start();
     try {
         const answer = await (0, agent_1.askQuestion)(cwd, config, {
@@ -70,11 +99,13 @@ program
                     spinner.text = status;
             },
         });
-        spinner.stop();
         if (!stream) {
-            console.log('\n' + answer.trim() + '\n');
+            const rendered = marked_1.marked.parse(answer.trim());
+            spinner.stop();
+            console.log('\n' + rendered + '\n');
         }
         else {
+            spinner.stop();
             console.log('\n');
         }
     }