ai-first-cli 1.1.3 → 1.2.0

package/CHANGELOG.md

@@ -6,265 +6,265 @@ All notable changes to `ai-first` will be documented in this file.
 
 ## [Unreleased]
 
-### Added
-- **CLI Progress Feedback**: Added ora-based progress indicators for repository analysis
-- **Lazy Context Generation**: Staged indexing for faster CLI startup (Stage 1 minimal, Stage 2 on-demand)
-- **Lazy Analyzer Module**: `src/core/lazyAnalyzer.ts` with buildMinimalIndex, expandFeatureContext, expandFlowContext
-- **Lazy Indexing Tests**: Comprehensive test suite for lazy context generation
-- **Lazy Indexing Documentation**: English and Spanish guides for lazy context generation
-- **AI Repository Schema**: Standardized schema system for AI-First metadata (schema.json, project.json, tools.json)
-- **AI Repository Schema**: Standardized schema system for AI-First metadata (schema.json, project.json, tools.json)
-- **Schema CLI Integration**: Schema automatically generated with `ai-first init`
-- **Schema Validation**: Programmatic schema validation API
-- **Schema Tests**: Comprehensive test suite for schema generation and validation
-- **Schema Documentation**: English and Spanish guides for AI Repository Schema
-- **Incremental Analysis**: Update repository context incrementally when files change
-- **Incremental Analysis**: Update repository context incrementally when files change
-- **Update CLI Command**: `ai-first update` to perform incremental updates
-- **Self-Healing Context**: AI context updates automatically when files change
-- **Repository Knowledge Graph**: Unify all repository intelligence into a single navigable graph
-- **Graph CLI Command**: `ai-first graph` to generate knowledge graph
-- **Repository Knowledge Graph**: Unify all repository intelligence into a single navigable graph
-- **Graph CLI Command**: `ai-first graph` to generate knowledge graph
-- **Graph Output**: `ai/graph/knowledge-graph.json` with nodes and edges
-- **Git Intelligence**: Analyze git activity to provide AI context about recently changed files, features, and flows
-- **Git CLI Command**: `ai-first git` to analyze repository activity and generate metadata
-- **Git Intelligence**: Analyze git activity to provide AI context about recently changed files, features, and flows
-- **Git CLI Command**: `ai-first git` to analyze repository activity and generate metadata
-- **Git Context Files**: `ai/git/recent-files.json`, `ai/git/recent-features.json`, `ai/git/recent-flows.json`, `ai/git/commit-activity.json`
-- **Adapter SDK**: Developer-friendly `createAdapter()` API for creating custom ecosystem adapters
-- **Community Adapters**: Pre-built adapters for Laravel, NestJS, Spring Boot, Phoenix, FastAPI
-- **Adapter CLI Command**: `ai-first adapters` to list all available adapters
-- **Adapter SDK**: Developer-friendly `createAdapter()` API for creating custom ecosystem adapters
-- **Community Adapters**: Pre-built adapters for Laravel, NestJS, Spring Boot, Phoenix, FastAPI
-- **Adapter CLI Command**: `ai-first adapters` to list all available adapters
-- Enhanced symbol indexing with `filePath#symbolName` IDs
-- Enhanced symbol indexing with `filePath#symbolName` IDs
-- Extended symbol graph relationships: calls, called_by, imports, references, instantiates, extends, implements, exports
-- Reverse symbol references (`ai/graph/symbol-references.json`)
-- File index with symbol mappings (`ai/files.json`)
-- Code Context Packets (CCP) with depth, ranking, and multiple formats
-- Context CLI flags: `--depth`, `--max-symbols`, `--format`, `--save`
-- Incremental indexing with file hash cache (`ai/index-state.json`)
-- Context ranking system based on graph distance and relationships
-- Android/Kotlin Support: Android framework detection, SDK version extraction, Gradle dependency parsing, AndroidManifest parsing, Android resources indexing, Gradle multi-module detection
-- **Business Feature Detection**: Auto-detect business modules (auth, users, payments) from codebase structure
-- **Flow Detection**: Generate execution chains spanning multiple architectural layers (api → service → data)
-- Feature output: `ai/context/features/<feature>.json` with path, files, entrypoints, dependencies
-- Flow output: `ai/context/flows/<flow>.json` with depth, layers, entrypoint
-- Support for nested features at depth 1 and 2 (src/auth, src/modules/auth)
-- **Performance Documentation**: Complete guide to incremental analysis and optimization
-- **Spanish Documentation**: Complete translation for all guides
-- **Adapter System**: Multi-language/framework support (JavaScript, Python, Django, Flask, Rails, Salesforce, .NET)
-- **Spanish Documentation**: Complete translation for all guides
+### Phase 2: Code Quality - Completed (v1.2.0)
 
-### Improved
-- Incremental repository analysis with change detection
-- Performance benchmarks for various repository sizes
+**Date:** 2026-03-18
 
-### Added
-- Enhanced symbol indexing with `filePath#symbolName` IDs
-- Extended symbol graph relationships: calls, called_by, imports, references, instantiates, extends, implements, exports
-- Reverse symbol references (`ai/graph/symbol-references.json`)
-- File index with symbol mappings (`ai/files.json`)
-- Code Context Packets (CCP) with depth, ranking, and multiple formats
-- Context CLI flags: `--depth`, `--max-symbols`, `--format`, `--save`
-- Incremental indexing with file hash cache (`ai/cache.json`)
-- Context ranking system based on graph distance and relationships
-- Android/Kotlin Support: Android framework detection, SDK version extraction, Gradle dependency parsing, AndroidManifest parsing, Android resources indexing, Gradle multi-module detection
-- **Business Feature Detection**: Auto-detect business modules (auth, users, payments) from codebase structure
-- **Flow Detection**: Generate execution chains spanning multiple architectural layers (api → service → data)
-- Feature output: `ai/context/features/<feature>.json` with path, files, entrypoints, dependencies
-- Flow output: `ai/context/flows/<flow>.json` with depth, layers, entrypoint
-- Support for nested features at depth 1 and 2 (src/auth, src/modules/auth)
+#### Fixed
 
----
+- **Symbol Extraction Extension Bug**
+  - Files affected: `src/analyzers/symbols.ts`
+  - Issue: Extension checks were missing dots (`"js"` instead of `".js"`)
+  - Fix: All extension checks now use correct format with dot
 
-## [1.1.0] - 2026-03-09
+- **Flow Name Sanitization**
+  - Files affected: `src/core/semanticContexts.ts`
+  - Issue: Malformed flow names like `auth..json`, `add_.json`
+  - Fix: Added `sanitizeFlowName()` function
 
-### Added
-- Enhanced symbol indexing with `filePath#symbolName` IDs
-- Extended symbol graph relationships: calls, called_by, imports, references, instantiates, extends, implements, exports
-- Reverse symbol references (`ai/graph/symbol-references.json`)
-- File index with symbol mappings (`ai/files.json`)
-- Code Context Packets (CCP) with depth, ranking, and multiple formats
-- Context CLI flags: `--depth`, `--max-symbols`, `--format`, `--save`
-- Incremental indexing with file hash cache (`ai/cache.json`)
-YB|- Context ranking system based on graph distance and relationships
-RR|- Android/Kotlin Support: Android framework detection, SDK version extraction, Gradle dependency parsing, AndroidManifest parsing, Android resources indexing, Gradle multi-module detection
-PY|- Documentation: Architecture guide and Commands reference updated
-- Documentation: Architecture guide and Commands reference updated
+- **Duplicate Files**
+  - Issue: `repo-map.json` and `repo_map.json` coexisted
+  - Fix: Consolidated to `repo_map.json` only
 
----
+- **PHP Parser**
+  - Files affected: `src/analyzers/symbols.ts`
+  - Issue: Laravel adapter returned 0 symbols
+  - Fix: Added `parsePHP()` function
 
-## [1.1.0] - 2026-03-09
+- **Ruby Parser**
+  - Files affected: `src/analyzers/symbols.ts`
+  - Issue: Rails adapter returned 0 symbols
+  - Fix: Added `parseRuby()` function
 
-All notable changes to `ai-first` will be documented in this file.
+#### Added
 
----
+- **Functional Tests for Phase 2 Fixes**
+  - File: `tests/phase2-fixes.test.ts`
+  - Tests symbol extraction, flow sanitization, file consolidation
 
-## [Unreleased]
+- **Adapter Test Script**
+  - File: `test_adapters.mjs`
+  - Tests all 11 adapters for symbol extraction (11/11 PASS)
 
-### Added
-- `ai-first context <symbol>` - Generate context packet for specific symbol
-- `ai/graph/symbol-graph.json` - Symbol-level dependency graph with bidirectional relationships
-- `ai/context/<symbol>.json` - Code Context Packets (CCP) for AI agents
-- Unique symbol IDs (format: `module.symbolName`)
-- Symbol relationships: calls, called_by, imports, references
+#### Changed
 
----
+- **Breaking Change:** Flow files are now sanitized
+  - Old names: `auth..json`, `users..json`, `add_.json`
+  - New names: `auth.json`, `users.json`, `add.json`
 
-## [1.1.0] - 2026-03-09
+- **Breaking Change:** `repo-map.json` renamed to `repo_map.json`
 
-### Added
-- `ai-first doctor` - Health check command for repository readiness
-- `ai-first explore <module>` - Navigate module dependencies
-- `ai-first map` - Generate repository architecture map
-- `ai-first index --semantic` - Semantic indexing with embeddings
-- `index-state.json` - Track file changes for incremental indexing
-- `files.json` - List of all indexed files with metadata
-- `modules.json` - Detected modules and their structure
-- `module-graph.json` - Module dependency graph
-- `embeddings.json` - Semantic embeddings for code search
-- SQLite index with adaptive sizing for large repos
-
-### New Generated Files
-- `ai/files.json` - File inventory with hash, size, line count
-- `ai/modules.json` - Module detection and hierarchy
-- `ai/module-graph.json` - Import-based dependency graph
-- `ai/embeddings.json` - Vector embeddings for semantic search
+### Phase 3: Architecture - Completed (v1.2.0)
 
----
+**Date:** 2026-03-18
 
-## [1.0.0] - 2026-03-08
+#### Changed
 
-### Added
-- feat: Add VitePress documentation site with SEO optimization (e0b3e35)
-- fix: Add permissions and GITHUB_TOKEN to changelog workflow (062d952)
-- chore: Update CHANGELOG with all new features (8c1f6d9)
-- fix: Update roadmap to match planned features from CHANGELOG (7eca761)
-- fix: Remove hash ID markers from README (d05db53)
-- feat: Add incremental indexing and semantic search pipeline (e294a2c)
-- feat: Add improved indexing with adaptive sizing, map command, incremental state (d0a172b)
-- feat: Add doctor, explore commands and semantic indexing (50011da)
-- fix: Correct YAML syntax in changelog workflow (5a1c819)
-- Improve documentation with Quick Start, Why AI-First, Architecture diagram, Languages (7cc8f44)
+- **Breaking Change:** Directory renamed from `ai/` to `ai-context/`
+  - Reason: Hidden directory (starts with `.`) keeps project root cleaner
+  - Affects: All generated files, test projects, documentation
+  - Files updated: 21 source files + all README/docs
 
-# Changelog & Roadmap
+#### Technical Details
 
-All notable changes to `ai-first` will be documented in this file.
+- Changed: `path.join(rootDir, "ai")` → `path.join(rootDir, "ai-context")` in all source files
+- Renamed: Main project directory `ai/` → `ai-context/`
+- Renamed: All 11 test project directories `*/ai/` → `*/ai-context/`
+- Updated: README.md, README.es.md, CHANGELOG.md, PLAN_MEJORAS.md
+
+### Phase 4: Performance - Completed (v1.2.0)
+
+**Date:** 2026-03-19
+
+#### Changed
+
+- **Embeddings storage moved from JSON to SQLite**
+  - Embeddings now stored in `index.db` alongside other index data
+  - Removed `embeddings.json` file generation
+  - Performance improvement: Single database file instead of multiple JSON files
+  - Files affected: `src/core/embeddings.ts`, `src/commands/ai-first.ts`
+
+#### Technical Details
+
+- Added `embeddings` table to SQLite database
+- Schema: `(id, vector, metadata, chunk_id, file_path, start_line, end_line, type, language)`
+- Vector and metadata stored as JSON strings
+- Same `index.db` used for files, symbols, imports, and embeddings
+
+### Phase 5: Documentation - Completed (v1.2.0)
+
+**Date:** 2026-03-19
+
+#### Changed
+
+- **Documentation consistency fixed**
+  - Updated all documentation files to use `ai-context/` instead of `ai/`
+  - Files updated: README.md, README.es.md, tests/e2e/README.md, examples/README.md
+  - Files updated: docs/guide/quick-start.md, docs/guide/getting-started.md
+  - Examples updated: examples/01-express-api.md, examples/02-react-app.md, examples/03-python-django.md
 
 ---
 
-## [Unreleased]
+## [1.1.6] - 2026-03-18
 
 ### Added
-- `ai-first doctor` - Health check command for repository readiness
-- `ai-first explore <module>` - Navigate module dependencies
-- `ai-first map` - Generate repository architecture map
-- `ai-first index --semantic` - Semantic indexing with embeddings
-- `index-state.json` - Track file changes for incremental indexing
-- `files.json` - List of all indexed files with metadata
-- `modules.json` - Detected modules and their structure
-- `module-graph.json` - Module dependency graph
-- `embeddings.json` - Semantic embeddings for code search
-- SQLite index with adaptive sizing for large repos
-
-### New Generated Files
-- `ai/files.json` - File inventory with hash, size, line count
-- `ai/modules.json` - Module detection and hierarchy
-- `ai/module-graph.json` - Import-based dependency graph
-- `ai/embeddings.json` - Vector embeddings for semantic search
+- **Phase 1: Testing Infrastructure** - Complete test project coverage for 10/16 adapters (62.5%)
+  - Test projects added:
+    - `django-app` (Django Web framework)
+    - `laravel-app` (Laravel PHP framework)
+    - `fastapi-app` (FastAPI Python)
+    - `flask-app` (Flask Python micro-framework)
+    - `rails-app` (Ruby on Rails)
+    - `spring-boot-app` (Spring Boot Java)
+  - All 11 commands tested on all 11 projects (121/121 functional tests passing)
+  - All projects generate valid JSON, SQLite databases, and Markdown files
+
+### Fixed
+- **Ruby Parser**: Added `parseRuby()` function to extract symbols from Ruby files
+  - rails-app now extracts 74 symbols (was 0)
+- **Flow Name Sanitization**: Added `sanitizeFileName()` function in semanticContexts.ts
+  - Fixes malformed names: `auth..json` → `auth.json`, `add_.json` → `add.json`
+- **Laravel/PHP Relationships**: Now properly detecting relationships (was 0, now 3+)
+
+### Verified
+- All 11 test projects pass doctor command (6/6 checks)
+- All adapters detect their respective frameworks correctly
+- All 11 commands (init, map, index, doctor, graph, summarize, update, explore, query, context, adapters) work on all projects
 
 ---
 
-## [1.0.0] - 2026-03-08
+## [1.1.5] - 2026-03-17
+
+### Fixed
+- **Bug A**: `index` command now correctly generates `index.db` in the `--root/ai/` directory instead of current working directory
+- **Bug B**: `graph` command now works without git repository (uses static analysis fallback)
+- **Bug C**: `query` command now correctly finds symbols (depends on Bug A fix)
+- **Comprehensive Testing**: All 11 commands tested on 5 different project types (55/55 tests passing)
+  - Express.js API
+  - NestJS Backend
+  - Python CLI
+  - React App
+  - Salesforce CLI
+
+### Verified
+- All JSON output files are valid
+- All SQLite databases are properly formatted
+- All Markdown files are properly generated
+- Exit codes are correct for all scenarios
 
-### Added
-- Initial release
-- Core analyzers:
-  - `repo_map.md` - Repository structure tree view
-  - `summary.md` - Files by extension and directory
-  - `architecture.md` - Pattern detection (MVC, Clean Architecture, etc.)
-  - `tech_stack.md` - Languages, frameworks, libraries
-  - `entrypoints.md` - CLI, API, server entry points
-  - `conventions.md` - Naming, testing, linting conventions
-  - `ai_context.md` - Unified AI context
-- `symbols.json` - Extracts functions, classes, interfaces from code
-- `dependencies.json` - Analyzes import/require dependencies between files
-- `ai_rules.md` - Generates guidelines for AI assistants
-- `repo_map.json` - Machine-readable repository structure
-- `index.db` - SQLite database for fast queries (`ai-first index`)
-
-### Features
-- Multi-language support (TypeScript, Python, Go, Rust, Java, C#, etc.)
-- Deterministic analysis (no AI/LLM required)
-- Works offline
-- CLI interface with `--root` and `--output` options
 ---
 
-## 📋 Roadmap
+## [1.1.4] - 2026-03-17
+
+### Fixed
+- **Bug 1**: Features and flows were empty because `isEntrypoint` only checked file names, not directory paths (e.g., `/commands/`, `/handlers/`)
+- **Bug 2**: Context command now displays all relationship types (Exports, References) in addition to Calls, Called by, Imports
+- **Bug 5**: `init` command now generates `modules.json`, required for feature/flow detection
+- **Bug 6**: Features now detected in projects without `src/` prefix (Express, React, etc.)
+- **Bug 7**: Fixed incorrect paths in flows (removed npm dependencies and normalized paths)
+- **Bug 8**: Salesforce extensions (`.cls`, `.trigger`, `.apex`) added to file scanner
+- **Bug 10**: Added Apex symbol parser to extract classes, methods, and triggers from Salesforce files
+- **Bug 11**: Fixed import detection in Python and React/JSX projects (was only detecting exports)
 
-See [CHANGELOG.md](./CHANGELOG.md) for the complete roadmap:
-- **Unreleased** section shows what's being worked on
-- Version sections show completed features
-- Future planned features are listed under each phase
+### Added
+- **Functional Test Suite**: Added test projects for multiple frameworks:
+  - `express-api` (Express.js)
+  - `nestjs-backend` (NestJS)
+  - `python-cli` (Python)
+  - `react-app` (React + Vite)
+  - `salesforce-cli` (Salesforce DX)
+- **E2E Tests**: Test runner script and documentation in `tests/e2e/`
+- **BUGS.md**: Tracking file for known issues and their status
+
+### Changed
+- Reduced minimum files for features from 3 to 2 (for small projects)
+- Added `controllers`, `routes`, `handlers`, `views`, `pages` to candidate roots
+- Removed `models`, `services` from ignored folders (now detected as features)
+
+### Security
+- **Bug 4**: Reduced npm vulnerabilities from 6 to 3 (dev dependencies only)
+- Updated `vitest` from 2.x to 4.x to resolve security issues
+- Remaining 3 vulnerabilities in `esbuild` (via vitepress) - low risk, dev-only
 
 ---
 
-## How to Update
+## [1.1.3] - 2026-03-11
+
+### Added
+- **CLI Progress Feedback**: Added ora-based progress indicators for repository analysis
+- **Lazy Context Generation**: Staged indexing for faster CLI startup
+- **Lazy Analyzer Module**: `src/core/lazyAnalyzer.ts` with buildMinimalIndex, expandFeatureContext, expandFlowContext
+- **AI Repository Schema**: Standardized schema system (schema.json, project.json, tools.json)
+- **Schema Validation**: Programmatic schema validation API
+- **Incremental Analysis**: Update repository context incrementally when files change
+- **Update CLI Command**: `ai-first update` to perform incremental updates
+- **Repository Knowledge Graph**: Unify all repository intelligence into a single navigable graph
+- **Graph CLI Command**: `ai-first graph` to generate knowledge graph
+- **Git Intelligence**: Analyze git activity for AI context
+- **Git CLI Command**: `ai-first git` to analyze repository activity
+- **Adapter SDK**: Developer-friendly `createAdapter()` API for custom ecosystem adapters
+- **Community Adapters**: Pre-built adapters for Laravel, NestJS, Spring Boot, Phoenix, FastAPI
+- **Business Feature Detection**: Auto-detect business modules (auth, users, payments)
+- **Flow Detection**: Generate execution chains spanning multiple architectural layers
+- **Android/Kotlin Support**: Android framework detection, SDK version extraction, Gradle parsing
 
-After each feature release:
-1. Update this changelog with new changes
-2. Update version in `package.json` (semver)
-3. Commit and push
+### Improved
+- Incremental repository analysis with change detection
+- Performance benchmarks for various repository sizes
 
-```bash
-git add CHANGELOG.md package.json
-git commit -m "Release v1.x.x"
-git push
-```
 ---
 
-## Roadmap
+## [1.1.0] - 2026-03-09
 
-### Phase 1 (Done ✅)
-- [x] Core CLI structure
-- [x] Basic analyzers (architecture, tech stack, entrypoints, conventions)
-- [x] Unified context generation
-
-### Phase 2 (Done ✅)
-- [x] Symbol extraction (functions, classes, interfaces)
-- [x] Dependency analysis
-- [x] AI rules generation
-- [x] Machine-readable outputs (JSON)
-
-### Phase 3 (In Progress)
-- [x] Health check command (`ai-first doctor`)
-- [x] Module exploration (`ai-first explore`)
-- [x] Repository mapping (`ai-first map`)
-- [x] Semantic indexing with embeddings
-- [x] Incremental indexing (track file changes)
-- [ ] Configuration file support (`ai-first.config.json`)
-- [ ] Custom rules/plugins system
-
-### Phase 4 (Future)
-- [ ] Git integration (analyze recent changes)
-- [ ] Diff-aware context (what changed since last run)
-- [ ] Interactive mode
-- [ ] VS Code extension
+### Added
+- Enhanced symbol indexing with `filePath#symbolName` IDs
+- Extended symbol graph relationships: calls, called_by, imports, references, instantiates, extends, implements, exports
+- Reverse symbol references (`ai/graph/symbol-references.json`)
+- File index with symbol mappings (`ai/files.json`)
+- Code Context Packets (CCP) with depth, ranking, and multiple formats
+- Context CLI flags: `--depth`, `--max-symbols`, `--format`, `--save`
+- Incremental indexing with file hash cache (`ai/cache.json`)
+- Context ranking system based on graph distance and relationships
+- Documentation: Architecture guide and Commands reference
 
 ---
 
-## How to Update
+## [1.0.0] - 2026-03-01
 
-After each push to GitHub:
-1. Update this changelog with new changes
-2. Update version if needed (semver)
-3. Push changes
+### Added
+- Initial release
+- Repository scanning and structure analysis
+- Symbol extraction for multiple languages
+- Dependency analysis
+- AI context generation
+- Architecture pattern detection
+- Tech stack detection
+
+---
+
+## Roadmap
 
-```bash
-git add CHANGELOG.md
-git commit -m "Update changelog"
-git push
-```
+### Phase 1: Core Infrastructure ✅
+- Repository scanning
+- Symbol extraction
+- Dependency analysis
+
+### Phase 2: Intelligence ✅
+- Symbol graph
+- Context packets
+- Feature/flow detection
+
+### Phase 3: Ecosystem ✅
+- Adapters for popular frameworks
+- Multi-language support
+- Salesforce/Apex support (partial)
+
+### Phase 4: Optimization (In Progress)
+- Incremental updates
+- Lazy loading
+- Performance improvements
+
+### Phase 5: Community (Planned)
+- Plugin system
+- Custom rules
+- CI/CD integration