codexa 1.0.1 → 1.2.0

package/README.md

@@ -1,6 +1,6 @@
 <div align="center">
   <h1>
-    <img src="https://github.com/sahitya-chandra/codexa/blob/main/.github/assets/logo.png" alt="Codexa Logo" width="90" align="absmiddle"> Codexa
+    Codexa
   </h1>
   
   <p>
@@ -54,6 +54,8 @@
 - 🔄 **Session Management**: Maintain conversation context across queries
 - 📊 **Streaming Output**: Real-time response streaming for better UX
 - 🎨 **Multiple File Types**: Supports TypeScript, JavaScript, Python, Go, Rust, Java, and more
+- 🧠 **Smart Configuration**: Automatically detects project languages and optimizes config
+- 🛡️ **Intelligent Filtering**: Automatically excludes binaries, large files, and build artifacts
 - ⚙️ **Highly Configurable**: Fine-tune chunking, retrieval, and model parameters
 - 🚀 **Zero Setup**: Works out of the box with sensible defaults
 
@@ -239,17 +241,41 @@ Once Codexa is installed and your LLM is configured, you're ready to use it:
 
 ### `init`
 
-Creates a `.codexarc.json` configuration file in the current directory with default settings.
+Creates a `.codexarc.json` configuration file optimized for your codebase.
 
 ```bash
 codexa init
 ```
 
 **What it does:**
-- Creates `.codexarc.json` in the project root
-- Uses sensible defaults for all configuration options
+- **Analyzes your codebase** to detect languages, package managers, and frameworks
+- **Creates optimized config** with language-specific include/exclude patterns
+- **Generates `.codexarc.json`** in the project root with tailored settings
 - Can be safely run multiple times (won't overwrite existing config)
 
+**Detection Capabilities:**
+- **Languages**: TypeScript, JavaScript, Python, Go, Rust, Java, Kotlin, Scala, C/C++, Ruby, PHP, Swift, Dart, and more
+- **Package Managers**: npm, yarn, pnpm, pip, poetry, go, cargo, maven, gradle, sbt, bundler, composer, and more
+- **Frameworks**: Next.js, React, Django, Flask, Rails, Laravel, Spring, Flutter, and more
+
+**Example Output:**
+```
+Analyzing codebase...
+✓ Detected: typescript, javascript (npm, yarn)
+
+✓ Created .codexarc.json with optimized settings for your codebase!
+
+┌ 🚀 Setup Complete ──────────────────────────────────────────┐
+│                                                             │
+│   Next Steps:                                               │
+│                                                             │
+│   1. Review .codexarc.json - Update provider keys if needed │
+│   2. Run: codexa ingest - Start indexing your codebase      │
+│   3. Run: codexa ask "your question" - Ask questions        │
+│                                                             │
+└─────────────────────────────────────────────────────────────┘
+```
+
 ---
 
 ### `ingest`
@@ -274,9 +300,15 @@ codexa ingest --force
 
 **What it does:**
 1. Scans your repository based on `includeGlobs` and `excludeGlobs` patterns
-2. Chunks files into manageable segments
-3. Generates vector embeddings for each chunk
-4. Stores everything in `.codexa/index.db` (SQLite database)
+2. **Filters files** - Automatically excludes binaries, large files (>5MB), and build artifacts
+3. Chunks files into manageable segments
+4. Generates vector embeddings for each chunk
+5. Stores everything in `.codexa/index.db` (SQLite database)
+
+**Smart Filtering:**
+- Automatically skips binary files (executables, images, archives, etc.)
+- Excludes files larger than the configured size limit (default: 5MB)
+- Filters based on file content analysis (not just extensions)
 
 **Note:** First ingestion may take a few minutes depending on your codebase size. Subsequent ingestions are faster as they only process changed files.
 
@@ -332,6 +364,28 @@ Codexa uses a `.codexarc.json` file in your project root for configuration. This
 
 **Format:** JSON
 
+### Dynamic Configuration Generation
+
+When you run `codexa init`, Codexa automatically:
+
+1. **Analyzes your codebase** structure to detect:
+   - Languages present (by file extensions)
+   - Package managers used (by config files)
+   - Frameworks detected (by dependencies and config files)
+
+2. **Generates optimized patterns**:
+   - **Include patterns**: Only file extensions relevant to detected languages
+   - **Exclude patterns**: Language-specific build artifacts, dependency directories, and cache folders
+   - **Smart defaults**: Based on your project type
+
+3. **Applies best practices**:
+   - Excludes common build outputs (`dist/`, `build/`, `target/`, etc.)
+   - Excludes dependency directories (`node_modules/`, `vendor/`, `.venv/`, etc.)
+   - Includes important config files and documentation
+   - Filters binaries and large files automatically
+
+This means your config is tailored to your project from the start, ensuring optimal indexing performance!
+
 ### Environment Variables
 
 Some settings can be configured via environment variables:
@@ -468,6 +522,49 @@ Controls randomness in LLM responses (0.0 = deterministic, 1.0 = creative).
 
 Number of code chunks to retrieve and use as context for each question. Higher values provide more context but may include less relevant information.
 
+#### `maxFileSize`
+
+**Type:** `number`  
+**Default:** `5242880` (5MB)
+
+Maximum file size in bytes. Files larger than this will be excluded from indexing. Helps avoid processing large binary files or generated artifacts.
+
+**Example:**
+```json
+{
+  "maxFileSize": 10485760  // 10MB
+}
+```
+
+#### `skipBinaryFiles`
+
+**Type:** `boolean`  
+**Default:** `true`
+
+Whether to automatically skip binary files during indexing. Binary detection uses both file extension and content analysis.
+
+**Example:**
+```json
+{
+  "skipBinaryFiles": true
+}
+```
+
+#### `skipLargeFiles`
+
+**Type:** `boolean`  
+**Default:** `true`
+
+Whether to skip files exceeding `maxFileSize` during indexing. Set to `false` if you want to include all files regardless of size.
+
+**Example:**
+```json
+{
+  "skipLargeFiles": true,
+  "maxFileSize": 10485760  // 10MB
+}
+```
+
 ### Example Configurations
 
 #### Groq Cloud Provider (Default)
@@ -666,10 +763,13 @@ When you run `codexa ask`:
 **Problem:** First ingestion takes too long.
 
 **Solutions:**
-1. Reduce `maxChunkSize` to create more, smaller chunks
-2. Add more patterns to `excludeGlobs` to skip unnecessary files
-3. Be more specific with `includeGlobs` to focus on important files
-4. Use `--force` only when necessary (incremental updates are faster)
+1. The dynamic config should already optimize patterns - check your `.codexarc.json` was generated correctly
+2. Reduce `maxFileSize` to exclude more large files
+3. Reduce `maxChunkSize` to create more, smaller chunks
+4. Add more patterns to `excludeGlobs` to skip unnecessary files
+5. Be more specific with `includeGlobs` to focus on important files
+6. Use `--force` only when necessary (incremental updates are faster)
+7. Ensure `skipBinaryFiles` and `skipLargeFiles` are enabled (default)
 
 ### Poor Quality Answers
 

package/dist/cli.js

@@ -6,6 +6,7 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
 Object.defineProperty(exports, "__esModule", { value: true });
 const commander_1 = require("commander");
 const ora_1 = __importDefault(require("ora"));
+const chalk_1 = __importDefault(require("chalk"));
 const config_1 = require("./config");
 const ingest_1 = require("./ingest");
 const agent_1 = require("./agent");
@@ -22,14 +23,35 @@ const program = new commander_1.Command();
 program
     .name('codexa')
     .description('Ask questions about any local repository from the command line.')
-    .version('0.1.0');
+    .version('1.2.0')
+    .action(() => {
+    console.log('\n');
+    logger_1.log.box(`${chalk_1.default.bold('Welcome to Codexa!')}\n\n` +
+        `${chalk_1.default.dim('Codexa is a CLI tool that helps you understand your codebase using AI.')}\n\n` +
+        `${chalk_1.default.bold('Getting Started:')}\n\n` +
+        `${chalk_1.default.dim('1.')} ${chalk_1.default.white('Initialize Codexa in your project:')}\n` +
+        `   ${chalk_1.default.cyan('codexa init')}\n\n` +
+        `${chalk_1.default.dim('2.')} ${chalk_1.default.white('Index your codebase:')}\n` +
+        `   ${chalk_1.default.cyan('codexa ingest')}\n\n` +
+        `${chalk_1.default.dim('3.')} ${chalk_1.default.white('Ask questions:')}\n` +
+        `   ${chalk_1.default.cyan('codexa ask "your question"')}\n\n` +
+        `${chalk_1.default.dim('For more help, run:')} ${chalk_1.default.cyan('codexa --help')}`, '🚀 Codexa');
+    console.log('\n');
+});
 program
     .command('init')
     .description('Create a local .codexarc.json with sensible defaults.')
     .action(async () => {
     const cwd = process.cwd();
     await (0, config_1.ensureConfig)(cwd);
-    logger_1.log.success('Created .codexarc.json. Update it with your provider keys if needed.');
+    console.log('\n');
+    logger_1.log.success('Created .codexarc.json with optimized settings for your codebase!');
+    console.log('\n');
+    logger_1.log.box(`${chalk_1.default.bold('Next Steps:')}\n\n` +
+        `${chalk_1.default.dim('1.')} ${chalk_1.default.white('Review .codexarc.json')} - Update provider keys if needed\n` +
+        `${chalk_1.default.dim('2.')} ${chalk_1.default.white('Run:')} ${chalk_1.default.cyan('codexa ingest')} ${chalk_1.default.dim('- Start indexing your codebase')}\n` +
+        `${chalk_1.default.dim('3.')} ${chalk_1.default.white('Run:')} ${chalk_1.default.cyan('codexa ask "your question"')} ${chalk_1.default.dim('- Ask questions about your code')}`, '🚀 Setup Complete');
+    console.log('\n');
 });
 program
     .command('ingest')

package/dist/config/detector.js

@@ -0,0 +1,339 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.analyzeProject = analyzeProject;
+const node_path_1 = __importDefault(require("node:path"));
+const fs_extra_1 = __importDefault(require("fs-extra"));
+const globby_1 = require("globby");
+const LANGUAGE_DEFINITIONS = [
+    {
+        name: 'typescript',
+        extensions: ['.ts', '.tsx', '.d.ts'],
+        packageManagers: ['npm', 'yarn', 'pnpm'],
+        frameworks: ['nextjs', 'react', 'angular', 'vue', 'nest', 'express'],
+        configFiles: ['tsconfig.json', 'next.config.js', 'next.config.ts'],
+    },
+    {
+        name: 'javascript',
+        extensions: ['.js', '.jsx', '.mjs', '.cjs'],
+        packageManagers: ['npm', 'yarn', 'pnpm'],
+        frameworks: ['react', 'vue', 'express', 'nextjs'],
+        configFiles: ['package.json', 'webpack.config.js', 'vite.config.js'],
+    },
+    {
+        name: 'python',
+        extensions: ['.py', '.pyw', '.pyi'],
+        packageManagers: ['pip', 'poetry', 'pipenv', 'conda'],
+        frameworks: ['django', 'flask', 'fastapi', 'pytest'],
+        configFiles: [
+            'requirements.txt',
+            'setup.py',
+            'pyproject.toml',
+            'Pipfile',
+            'poetry.lock',
+            'manage.py',
+        ],
+    },
+    {
+        name: 'go',
+        extensions: ['.go'],
+        packageManagers: ['go'],
+        frameworks: ['gin', 'echo', 'fiber'],
+        configFiles: ['go.mod', 'go.sum', 'Gopkg.toml'],
+    },
+    {
+        name: 'rust',
+        extensions: ['.rs'],
+        packageManagers: ['cargo'],
+        frameworks: ['actix', 'rocket', 'axum'],
+        configFiles: ['Cargo.toml', 'Cargo.lock'],
+    },
+    {
+        name: 'java',
+        extensions: ['.java', '.kt', '.scala'],
+        packageManagers: ['maven', 'gradle', 'sbt'],
+        frameworks: ['spring', 'javafx', 'android'],
+        configFiles: ['pom.xml', 'build.gradle', 'build.sbt', 'settings.gradle'],
+    },
+    {
+        name: 'cpp',
+        extensions: ['.cpp', '.cxx', '.cc', '.c++', '.hpp', '.hxx', '.h++'],
+        packageManagers: ['cmake', 'conan'],
+        frameworks: ['qt', 'boost'],
+        configFiles: ['CMakeLists.txt', 'Makefile', 'conanfile.txt'],
+    },
+    {
+        name: 'c',
+        extensions: ['.c', '.h'],
+        packageManagers: ['cmake', 'make'],
+        configFiles: ['CMakeLists.txt', 'Makefile', 'configure'],
+    },
+    {
+        name: 'ruby',
+        extensions: ['.rb', '.rake'],
+        packageManagers: ['bundler', 'gem'],
+        frameworks: ['rails', 'sinatra'],
+        configFiles: ['Gemfile', 'Gemfile.lock', 'Rakefile'],
+    },
+    {
+        name: 'php',
+        extensions: ['.php', '.phtml'],
+        packageManagers: ['composer'],
+        frameworks: ['laravel', 'symfony', 'wordpress'],
+        configFiles: ['composer.json', 'composer.lock'],
+    },
+    {
+        name: 'swift',
+        extensions: ['.swift'],
+        packageManagers: ['swiftpm', 'cocoapods'],
+        frameworks: ['swiftui', 'uikit'],
+        configFiles: ['Package.swift', 'Podfile'],
+    },
+    {
+        name: 'kotlin',
+        extensions: ['.kt', '.kts'],
+        packageManagers: ['gradle', 'maven'],
+        frameworks: ['android', 'spring'],
+        configFiles: ['build.gradle.kts', 'pom.xml'],
+    },
+    {
+        name: 'scala',
+        extensions: ['.scala', '.sc'],
+        packageManagers: ['sbt', 'maven'],
+        frameworks: ['play', 'akka'],
+        configFiles: ['build.sbt', 'pom.xml'],
+    },
+    {
+        name: 'dart',
+        extensions: ['.dart'],
+        packageManagers: ['pub'],
+        frameworks: ['flutter'],
+        configFiles: ['pubspec.yaml', 'pubspec.lock'],
+    },
+    {
+        name: 'html',
+        extensions: ['.html', '.htm', '.xhtml'],
+    },
+    {
+        name: 'css',
+        extensions: ['.css', '.scss', '.sass', '.less', '.styl'],
+    },
+    {
+        name: 'sql',
+        extensions: ['.sql'],
+    },
+    {
+        name: 'shell',
+        extensions: ['.sh', '.bash', '.zsh', '.fish'],
+    },
+    {
+        name: 'yaml',
+        extensions: ['.yaml', '.yml'],
+    },
+    {
+        name: 'toml',
+        extensions: ['.toml'],
+    },
+    {
+        name: 'json',
+        extensions: ['.json'],
+    },
+    {
+        name: 'xml',
+        extensions: ['.xml'],
+    },
+    {
+        name: 'markdown',
+        extensions: ['.md', '.markdown', '.mdx'],
+    },
+    {
+        name: 'docker',
+        extensions: ['.dockerfile'],
+        configFiles: ['Dockerfile', 'docker-compose.yml', 'docker-compose.yaml'],
+    },
+];
+const FRAMEWORK_INDICATORS = {
+    nextjs: ['next.config.js', 'next.config.ts', '.next'],
+    react: ['package.json'],
+    django: ['manage.py', 'settings.py', 'wsgi.py'],
+    flask: ['flask', 'app.py'], // Check for flask import
+    rails: ['Gemfile', 'config.ru', 'app/models'],
+    laravel: ['artisan', 'app/Http'],
+    spring: ['pom.xml', 'application.properties'],
+    android: ['AndroidManifest.xml', 'build.gradle'],
+    flutter: ['pubspec.yaml', 'lib/main.dart'],
+};
+/**
+ * Analyzes codebase to detect languages, package managers, and frameworks
+ */
+async function analyzeProject(cwd) {
+    const languages = new Set();
+    const packageManagers = new Set();
+    const frameworks = new Set();
+    const detectedConfigFiles = [];
+    let hasLargeFiles = false;
+    try {
+        const allFiles = await (0, globby_1.globby)(['**/*'], {
+            cwd,
+            gitignore: true,
+            ignore: ['node_modules/**', '.git/**', 'dist/**', 'build/**', '.codexa/**'],
+            onlyFiles: true,
+            absolute: false,
+        });
+        const extensionMap = new Map();
+        const LARGE_FILE_THRESHOLD = 5 * 1024 * 1024;
+        for (const file of allFiles) {
+            const ext = node_path_1.default.extname(file).toLowerCase();
+            if (ext) {
+                extensionMap.set(ext, (extensionMap.get(ext) || 0) + 1);
+            }
+            // large file check
+            try {
+                const filePath = node_path_1.default.join(cwd, file);
+                const stats = await fs_extra_1.default.stat(filePath);
+                if (stats.size > LARGE_FILE_THRESHOLD) {
+                    hasLargeFiles = true;
+                }
+            }
+            catch {
+                // Ignore errors (file might not exist, permission issues, etc.)
+            }
+        }
+        for (const lang of LANGUAGE_DEFINITIONS) {
+            const hasExtension = lang.extensions.some((ext) => extensionMap.has(ext));
+            if (hasExtension) {
+                languages.add(lang.name);
+                if (lang.packageManagers) {
+                    lang.packageManagers.forEach((pm) => packageManagers.add(pm));
+                }
+            }
+        }
+        const configFileChecks = [
+            'package.json', // npm/yarn/pnpm
+            'package-lock.json', // npm
+            'yarn.lock', // yarn
+            'pnpm-lock.yaml', // pnpm
+            'requirements.txt', // pip
+            'pyproject.toml', // poetry
+            'Pipfile', // pipenv
+            'go.mod', // go
+            'Cargo.toml', // cargo
+            'pom.xml', // maven
+            'build.gradle', // gradle
+            'build.sbt', // sbt
+            'Gemfile', // bundler
+            'composer.json', // composer
+            'pubspec.yaml', // pub (dart/flutter)
+            'CMakeLists.txt', // cmake
+            'Makefile', // make
+            'Dockerfile', // docker
+            'docker-compose.yml',
+            'docker-compose.yaml',
+        ];
+        for (const configFile of configFileChecks) {
+            const configPath = node_path_1.default.join(cwd, configFile);
+            if (await fs_extra_1.default.pathExists(configPath)) {
+                detectedConfigFiles.push(configFile);
+                if (configFile === 'package.json' || configFile === 'package-lock.json') {
+                    packageManagers.add('npm');
+                }
+                else if (configFile === 'yarn.lock') {
+                    packageManagers.add('yarn');
+                }
+                else if (configFile === 'pnpm-lock.yaml') {
+                    packageManagers.add('pnpm');
+                }
+                else if (configFile === 'requirements.txt' || configFile === 'setup.py') {
+                    packageManagers.add('pip');
+                }
+                else if (configFile === 'pyproject.toml' || configFile === 'poetry.lock') {
+                    packageManagers.add('poetry');
+                }
+                else if (configFile === 'Pipfile') {
+                    packageManagers.add('pipenv');
+                }
+                else if (configFile === 'go.mod') {
+                    packageManagers.add('go');
+                }
+                else if (configFile === 'Cargo.toml') {
+                    packageManagers.add('cargo');
+                }
+                else if (configFile === 'pom.xml') {
+                    packageManagers.add('maven');
+                }
+                else if (configFile === 'build.gradle' || configFile === 'settings.gradle') {
+                    packageManagers.add('gradle');
+                }
+                else if (configFile === 'build.sbt') {
+                    packageManagers.add('sbt');
+                }
+                else if (configFile === 'Gemfile') {
+                    packageManagers.add('bundler');
+                }
+                else if (configFile === 'composer.json') {
+                    packageManagers.add('composer');
+                }
+                else if (configFile === 'pubspec.yaml') {
+                    packageManagers.add('pub');
+                }
+                else if (configFile === 'CMakeLists.txt') {
+                    packageManagers.add('cmake');
+                }
+            }
+        }
+        for (const [framework, indicators] of Object.entries(FRAMEWORK_INDICATORS)) {
+            for (const indicator of indicators) {
+                const indicatorPath = node_path_1.default.join(cwd, indicator);
+                if (await fs_extra_1.default.pathExists(indicatorPath)) {
+                    frameworks.add(framework);
+                    break;
+                }
+            }
+        }
+        // Special case: Check package.json for framework dependencies
+        const packageJsonPath = node_path_1.default.join(cwd, 'package.json');
+        if (await fs_extra_1.default.pathExists(packageJsonPath)) {
+            try {
+                const packageJson = await fs_extra_1.default.readJson(packageJsonPath);
+                const deps = {
+                    ...packageJson.dependencies,
+                    ...packageJson.devDependencies,
+                };
+                const depNames = Object.keys(deps).map((d) => d.toLowerCase());
+                if (depNames.some((d) => d.includes('react'))) {
+                    frameworks.add('react');
+                }
+                if (depNames.some((d) => d.includes('next'))) {
+                    frameworks.add('nextjs');
+                }
+                if (depNames.some((d) => d.includes('angular'))) {
+                    frameworks.add('angular');
+                }
+                if (depNames.some((d) => d.includes('vue'))) {
+                    frameworks.add('vue');
+                }
+                if (depNames.some((d) => d.includes('nestjs'))) {
+                    frameworks.add('nest');
+                }
+                if (depNames.some((d) => d.includes('express'))) {
+                    frameworks.add('express');
+                }
+            }
+            catch {
+                // Ignore JSON parse err
+            }
+        }
+    }
+    catch (error) {
+        console.warn(`Project analysis warning: ${error instanceof Error ? error.message : 'Unknown error'}`);
+    }
+    return {
+        languages: Array.from(languages).sort(),
+        packageManagers: Array.from(packageManagers).sort(),
+        frameworks: Array.from(frameworks).sort(),
+        hasLargeFiles,
+        detectedConfigFiles: detectedConfigFiles.sort(),
+    };
+}