codehere 0.1.0

package/ARCHITECTURE.md

@@ -0,0 +1,240 @@
+# CodeHere Architecture
+
+This document provides a technical overview of CodeHere's architecture. For design decisions and roadmap, see `docs/CODEHERE_LIVING_DOC.md`.
+
+## System Overview
+
+CodeHere is a local-first, CLI-based coding assistant that uses:
+- **Cohere AI** for embeddings and chat completion
+- **SQLite** (via sql.js) for local vector storage
+- **TypeScript** for type safety
+- **Commander.js** for CLI interface
+
+## Core Architecture
+
+### 1. Embeddings System
+
+**File:** `src/embed.ts`
+
+**Responsibilities:**
+- Recursively scan repository for text files
+- Chunk files into 300-token segments (50-token overlap)
+- Generate embeddings via Cohere API
+- Store in SQLite database (`data/embeddings.sqlite`)
+
+**Key Functions:**
+- `indexRepository()` - Main indexing function
+- `chunkText()` - Splits text into token-sized chunks
+- `embedText()` - Calls Cohere API with retry logic
+- `updateFileEmbeddings()` - Re-indexes a single file after edit
+
+**Incremental Indexing:**
+- Tracks file metadata (path, mtime, hash) in `file_metadata` table
+- Only re-indexes changed/new files
+- Removes deleted files from index
+
+### 2. Search System
+
+**File:** `src/search.ts`
+
+**Responsibilities:**
+- Semantic search using cosine similarity
+- Intelligent context retrieval with ranking
+- File type prioritization
+
+**Key Functions:**
+- `searchSimilar()` - Basic cosine similarity search
+- `searchIntelligent()` - Enhanced search with:
+  - File type prioritization (source files > docs)
+  - Path heuristics (`src/` boost, `test/` penalty)
+  - Exact filename matching (3x boost)
+
+**Search Flow:**
+1. Embed query using Cohere API
+2. Load all embeddings from SQLite
+3. Calculate cosine similarity
+4. Rank and boost results
+5. Return top N chunks
+
+### 3. Edit System
+
+**File:** `src/edit.ts`
+
+**Responsibilities:**
+- Generate unified diffs from natural language
+- Apply patches safely with validation
+- Never write broken files
+
+**Key Functions:**
+- `generateAndApplyEdit()` - Main edit function
+- `generateEditDryRun()` - Preview mode (no write)
+- `applyUnifiedDiff()` - Applies patch using `diff` package
+- `validateSyntax()` - Validates JS/TS syntax before write
+- `fixDuplicateExports()` - Post-processes module.exports
+
+**Safety Flow:**
+1. Generate diff via Cohere API
+2. Parse diff stats
+3. Check policy (line limits, folders)
+4. Apply patch to get new content
+5. **Validate syntax** (JS/TS only)
+6. **Only write if valid** (critical safety check)
+7. Update embeddings asynchronously
+
+**Critical Safety Rule:**
+```typescript
+// NEVER write if syntax is invalid
+if (!syntaxCheck.valid) {
+  return { success: false, error: 'Syntax validation failed. File NOT modified.' };
+}
+writeFileSync(fullPath, newContent, 'utf-8'); // Only if valid
+```
+
+### 4. Policy System
+
+**File:** `src/policy.ts`
+
+**Responsibilities:**
+- Enforce safety guardrails
+- Prevent dangerous operations
+
+**Rules:**
+- Max 50 lines changed per edit
+- Blocks `infra/` and `billing/` folders
+- Single-file only (blocks multi-file patches)
+
+### 5. Chat System
+
+**File:** `src/chat.ts`
+
+**Responsibilities:**
+- Cohere API integration
+- Error handling with retry logic
+- Circuit breaker pattern
+
+**Key Functions:**
+- `chatWithContext()` - Chat with code context
+- `generateDiff()` - Generate unified diff from instruction
+
+**Error Handling:**
+- Exponential backoff retry
+- Circuit breaker for API failures
+- Clear error messages
+
+## Data Storage
+
+### SQLite Database
+
+**Location:** `data/embeddings.sqlite`
+
+**Tables:**
+1. `embeddings`
+   - `id` (INTEGER PRIMARY KEY)
+   - `filepath` (TEXT)
+   - `chunkIndex` (INTEGER)
+   - `content` (TEXT)
+   - `embedding` (TEXT - JSON array)
+
+2. `file_metadata`
+   - `filepath` (TEXT PRIMARY KEY)
+   - `mtime` (INTEGER)
+   - `hash` (TEXT - MD5)
+
+**Indexes:**
+- `idx_filepath` on `embeddings(filepath)`
+
+### Audit Logs
+
+**Location:** `data/logs.jsonl`
+
+**Format:** JSON Lines (one JSON object per line)
+
+**Fields:**
+- `timestamp` (ISO string)
+- `command` (string)
+- `query` (string, optional)
+- `filepath` (string, optional)
+- `diffStats` (object, optional)
+- `policyStatus` ('allowed' | 'rejected')
+- `retrievedFiles` (array, optional)
+
+## CLI Structure
+
+**File:** `src/index.ts`
+
+**Commands:**
+- `codehere` (no args) - Interactive session
+- `codehere index` - Build embeddings index
+- `codehere ask "<question>"` - Semantic search + chat
+- `codehere explain <file>` - Explain a file
+- `codehere fix <file> "<instruction>"` - Edit a file
+- `codehere fix <file> "<instruction>" --dry-run` - Preview changes
+- `codehere scaffold <template> <targetDir>` - Scaffold app skeleton
+- `codehere debug-search "<query>"` - Debug search results
+
+## Templates
+
+**Location:** `templates/`
+
+**Structure:**
+- Each template is a directory
+- Files are copied recursively to target directory
+- Templates are static (not LLM-generated)
+
+**Available Templates:**
+- `node-api` - Simple Node.js HTTP API
+- `next-page` - Minimal Next.js page
+
+## Error Handling
+
+**Strategy:**
+- Retry with exponential backoff for API calls
+- Circuit breaker for repeated failures
+- Clear, actionable error messages
+- Never crash silently
+
+**Error Messages:**
+- Missing API key: Shows setup instructions
+- Missing embeddings DB: Shows how to run `index`
+- Syntax validation failure: Shows error details, file NOT modified
+- Policy rejection: Shows reason
+
+## Testing
+
+**Location:** `tests/edit.test.ts`
+
+**Test Cases:**
+1. Simple function rename
+2. Add logging line
+3. Add try/catch around async call
+4. Edit file with comments
+5. Add parameter with default
+
+**Run:** `npm test`
+
+## Performance
+
+- **Indexing:** ~2 seconds per file (rate-limited by API)
+- **Search:** 5-8 seconds (API call + DB query)
+- **Edit:** 5-8 seconds (diff generation + validation)
+- **Token Optimization:** 64-83% reduction
+
+## Security & Privacy
+
+- **Local-first:** Embeddings stored locally
+- **No code transmission:** Only chunks sent for embedding
+- **Audit logging:** Complete JSONL trail
+- **Policy enforcement:** Multiple safety checks
+
+## Future Architecture (Tier 2+)
+
+See `docs/CODEHERE_LIVING_DOC.md` for roadmap:
+- Multi-file support
+- Project graph awareness
+- Test execution integration
+- VS Code extension
+
+---
+
+For design decisions and roadmap, see `docs/CODEHERE_LIVING_DOC.md`.
+

package/CHANGELOG.md

@@ -0,0 +1,44 @@
+# Changelog
+
+All notable changes to CodeHere will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.0] - 2025-11-24
+
+### Added
+- Core CLI commands: `index`, `ask`, `explain`, `fix`, `scaffold`
+- Local-first embeddings storage using SQLite
+- Semantic search with intelligent context retrieval
+- Single-file code editing with syntax validation
+- Policy guardrails (line limits, restricted folders)
+- Dry-run mode for previewing changes
+- Incremental indexing (only re-indexes changed files)
+- Template-based app scaffolding (`node-api`, `next-page`)
+- Integration test suite (5 test cases)
+- Comprehensive documentation (living doc, architecture, contributing guide)
+
+### Safety Features
+- Syntax validation before writing files (JS/TS)
+- TypeScript type checking (`tsc --noEmit`)
+- Policy enforcement (max 50 lines, single-file only)
+- Audit logging (JSONL format)
+- Never writes broken files to disk
+
+### Fixed
+- Module.exports handling on function renames
+- Temp file cleanup in syntax validation
+- Error handling for missing embeddings database
+- Search relevance (prioritizes source files, path heuristics)
+
+### Documentation
+- Single living document (`docs/CODEHERE_LIVING_DOC.md`)
+- Architecture guide
+- Contributing guide
+- Changelog (this file)
+
+---
+
+[0.1.0]: https://github.com/muhammadegaa/codehere/releases/tag/v0.1.0
+

package/CONTRIBUTING.md

@@ -0,0 +1,171 @@
+# Contributing to CodeHere
+
+Thank you for your interest in contributing to CodeHere! This guide will help you understand the architecture, development workflow, and how to make contributions.
+
+## Architecture Overview
+
+CodeHere is a local-first, CLI-based coding assistant. See `docs/CODEHERE_LIVING_DOC.md` for the complete design and roadmap.
+
+### Core Components
+
+1. **Embeddings** (`src/embed.ts`)
+   - Indexes files, chunks text, generates embeddings via Cohere API
+   - Stores in local SQLite database
+   - Supports incremental indexing (tracks file mtime + hash)
+
+2. **Search** (`src/search.ts`)
+   - Semantic search using cosine similarity
+   - Intelligent context retrieval with file type prioritization
+   - Path heuristics (prioritizes `src/`, penalizes `test/`)
+
+3. **Edit** (`src/edit.ts`)
+   - Generates unified diffs from natural language instructions
+   - Applies patches with syntax validation
+   - **Critical safety:** Never writes broken files to disk
+
+4. **Policy** (`src/policy.ts`)
+   - Enforces safety guardrails (line limits, restricted folders)
+   - Single-file only (prevents accidental multi-file changes)
+
+5. **Chat** (`src/chat.ts`)
+   - Cohere API integration for chat and diff generation
+   - Error handling with retry logic
+
+### Data Flow
+
+```
+User Command → Policy Check → Generate Diff → Apply Patch → 
+Syntax Validation → Write File (only if valid) → Update Embeddings (async)
+```
+
+## Development Workflow
+
+### Prerequisites
+
+- Node.js 18+
+- TypeScript 5.0+
+- Cohere API key (get from https://dashboard.cohere.com/api-keys)
+
+### Setup
+
+```bash
+# Clone the repository
+git clone https://github.com/muhammadegaa/codehere.git
+cd codehere/agent
+
+# Install dependencies
+npm install
+
+# Set up environment
+echo "COHERE_API_KEY=your_key_here" > .env
+
+# Build
+npm run build
+
+# Run tests
+npm test
+```
+
+### Development Commands
+
+```bash
+# Build TypeScript
+npm run build
+
+# Run in development mode
+npm run dev
+
+# Run tests
+npm test
+
+# Install globally (for testing)
+npm install -g .
+```
+
+### Making Changes
+
+1. **Create a branch:**
+   ```bash
+   git checkout -b feature/your-feature-name
+   ```
+
+2. **Make your changes:**
+   - Follow existing code style
+   - Add tests for new features
+   - Update documentation if behavior changes
+
+3. **Test your changes:**
+   ```bash
+   npm run build
+   npm test
+   ```
+
+4. **Update documentation:**
+   - If behavior changes, update `docs/CODEHERE_LIVING_DOC.md`
+   - If CLI changes, update `README.md`
+   - If architecture changes, update `ARCHITECTURE.md`
+
+5. **Commit:**
+   ```bash
+   git commit -m "feat: add your feature"
+   ```
+
+6. **Push and create PR:**
+   ```bash
+   git push origin feature/your-feature-name
+   ```
+
+## Code Style
+
+- **TypeScript:** Use strict mode, prefer explicit types
+- **Imports:** Use ES modules (`import`/`export`)
+- **Error handling:** Always use try/catch, provide clear error messages
+- **Safety first:** Never write broken files, always validate before writing
+
+## Testing
+
+- Integration tests in `tests/edit.test.ts`
+- Run with `npm test`
+- Add tests for new features or edge cases
+
+## Documentation
+
+- **Living Doc:** `docs/CODEHERE_LIVING_DOC.md` - single source of truth
+- **Architecture:** `ARCHITECTURE.md` - technical details
+- **README:** User-facing documentation
+- **Changelog:** `CHANGELOG.md` - release notes
+
+**Important:** Always update the living doc when capabilities or behavior change.
+
+## Safety Guidelines
+
+1. **Never write broken files:**
+   - Always validate syntax before writing
+   - Use dry-run mode for testing
+   - Rollback on validation failure
+
+2. **Policy enforcement:**
+   - Respect line limits (50 lines max)
+   - Single-file only (for now)
+   - Block restricted folders
+
+3. **Error handling:**
+   - Provide clear, actionable error messages
+   - Log all operations for audit
+   - Never crash silently
+
+## Release Process
+
+1. Update version in `package.json` (semver)
+2. Update `CHANGELOG.md`
+3. Update `docs/CODEHERE_LIVING_DOC.md` changelog section
+4. Tag release: `git tag v0.1.0`
+5. Publish: `npm publish --access public`
+
+## Questions?
+
+- Open an issue on GitHub
+- Check `docs/CODEHERE_LIVING_DOC.md` for design decisions
+- Review `ARCHITECTURE.md` for technical details
+
+Thank you for contributing! 🚀

package/LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2024
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+

package/README.md

@@ -0,0 +1,203 @@
+# codehere
+
+> Enterprise-grade AI coding assistant with local embeddings and semantic search
+
+[![TypeScript](https://img.shields.io/badge/TypeScript-5.3-blue)](https://www.typescriptlang.org/)
+[![Node.js](https://img.shields.io/badge/Node.js-18+-green)](https://nodejs.org/)
+[![License](https://img.shields.io/badge/License-MIT-yellow)](LICENSE)
+
+## 🚀 Installation
+
+### Global Installation (Recommended)
+
+**Option 1: Install from Local Directory**
+```bash
+# Navigate to the agent directory
+cd codehere/agent
+
+# Install globally
+npm install -g .
+
+# Verify installation
+codehere --version
+```
+
+**Option 2: Install from npm (after publishing)**
+```bash
+npm install -g codehere
+```
+
+Once installed globally, you can use `codehere` from any directory in your terminal.
+
+### Quick Start
+
+```bash
+# Set up your Cohere API key (in your project directory)
+echo "COHERE_API_KEY=your_key_here" > .env
+
+# Index your codebase
+codehere index
+
+# Start interactive session
+codehere
+
+# Or use commands directly
+codehere ask "How does authentication work?"
+codehere explain src/utils.ts
+codehere fix src/utils.ts "Add error handling"
+codehere fix src/utils.ts "Add error handling" --dry-run  # Preview changes
+codehere scaffold node-api ./playground/todo-api  # Scaffold app skeleton
+codehere debug-search "logging"  # Debug search results
+```
+
+## ✨ Features
+
+### 🧠 Semantic Code Search
+- Builds local embedding index using Cohere's embedding API
+- Uses cosine similarity to find relevant code chunks
+- Enhanced context with file relationships and dependencies
+
+### 🤖 AI-Powered Assistance
+- Context-aware responses using retrieved code
+- Natural language code explanations
+- Intelligent code editing with unified diff generation
+- Enhanced context understanding (15+ chunks)
+
+### 🛡️ Enterprise Safety
+- **Line limit enforcement** - Maximum 50 lines changed per edit
+- **Restricted folders** - Protect critical directories (infra/, billing/)
+- **Single-file edits** - Prevent accidental multi-file changes
+- **Audit logging** - Every operation logged to JSONL
+
+### 🔧 Enterprise Reliability
+- **Automatic error recovery** - Retry with exponential backoff
+- **Circuit breaker** - Protects against API failures
+- **Health monitoring** - System health checks and metrics
+- **24/7 operation ready** - Robust error handling
+
+### 📦 Production Ready
+- Fully tested end-to-end
+- TypeScript for type safety
+- SQLite for local vector storage
+- Comprehensive error handling
+
+## 🏗️ Architecture
+
+```
+┌─────────────┐
+│   CLI Tool  │
+└──────┬──────┘
+       │
+       ├─→ Index: Scan → Chunk → Embed → Store (SQLite)
+       ├─→ Ask: Query → Embed → Search (15 chunks) → Chat
+       ├─→ Explain: Read → Chat
+       ├─→ Fix: Read → Generate Diff → Policy Check → Apply
+       └─→ Health: System Status → Metrics
+```
+
+**Tech Stack:**
+- **TypeScript** - Type-safe development
+- **Cohere AI** - Embeddings and chat completion
+- **SQL.js** - In-memory SQLite for vector storage
+- **Commander.js** - CLI interface
+- **Tiktoken** - Token counting and chunking
+
+## 📊 Performance
+
+- **Indexing:** ~2 seconds per chunk (rate-limited by API)
+- **Retrieval:** <100ms for 200+ chunks
+- **Chat:** ~2-5 seconds per query
+- **Storage:** ~13KB per chunk (1024-dim embeddings)
+
+## 🔒 Security & Privacy
+
+- **Local-first:** All embeddings stored locally in SQLite
+- **No code transmission:** Only chunks sent to API for embedding
+- **Audit trail:** Complete JSONL logging of all operations
+- **Policy enforcement:** Multiple safety checks before applying changes
+
+## 📈 Use Cases
+
+### For Developers
+- Understand large codebases quickly
+- Get context-aware answers about code
+- Safely refactor with AI assistance
+- Learn from code explanations
+
+### For Teams
+- Onboard new developers faster
+- Document code automatically
+- Enforce coding standards
+- Audit code changes
+
+### For Enterprises
+- Code review assistance
+- Compliance logging
+- Safe AI-assisted development
+- Knowledge base building
+
+## 📚 Documentation
+
+**Main Reference:** See [docs/CODEHERE_LIVING_DOC.md](docs/CODEHERE_LIVING_DOC.md) for the complete design, roadmap, and current capabilities.
+
+This is the single source of truth for CodeHere's vision, current state, limitations, and future plans.
+
+**Test Results:**
+- ✅ Indexing: Working with retry logic
+- ✅ Retrieval: 15 chunks with file relationships
+- ✅ Chat: Context-aware responses with error recovery
+- ✅ Editing: Diff generation and application
+- ✅ Policies: All safety checks enforced
+- ✅ Logging: Complete audit trail
+- ✅ Health: System monitoring operational
+
+## 📁 Project Structure
+
+```
+agent/
+├── src/
+│   ├── index.ts      # CLI entrypoint
+│   ├── embed.ts      # Repository indexing & embeddings
+│   ├── search.ts     # Semantic search & retrieval
+│   ├── chat.ts       # Cohere API integration
+│   ├── edit.ts       # Diff generation & application
+│   ├── policy.ts     # Safety policy enforcement
+│   ├── log.ts        # Audit logging
+│   ├── types.ts      # Type definitions
+│   ├── ui.ts         # CLI UI utilities
+│   ├── error-handler.ts  # Enterprise error handling
+│   ├── monitoring.ts  # Health checks & metrics
+│   └── context.ts    # Enhanced context understanding
+├── data/
+│   ├── embeddings.sqlite  # Vector database
+│   └── logs.jsonl         # Audit log
+├── test/             # Test files
+├── package.json
+├── tsconfig.json
+└── README.md
+```
+
+## 🛠️ Development
+
+```bash
+# Development mode
+npm run dev
+
+# Build
+npm run build
+
+# Run tests
+npm test
+```
+
+## 📝 License
+
+MIT License - see [LICENSE](LICENSE) for details.
+
+## 🤝 Contributing
+
+Contributions welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
+
+---
+
+**Built with ❤️ for developers who want AI assistance without sacrificing control.**