codesummary 1.2.0 → 1.2.2

package/README.md

@@ -5,21 +5,22 @@
 [![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
 [![Cross-Platform](https://img.shields.io/badge/platform-Windows%20%7C%20macOS%20%7C%20Linux-lightgrey)](#)
 
-A **cross-platform CLI tool** that scans your project's source code and generates three types of output:
+A **cross-platform CLI tool** that scans your project's source code and generates repository documentation optimized for both human review and Large Language Models (LLMs).
 
-- **PDF** — clean, professional A4 documentation for code reviews and audits
-- **RAG JSON** — semantic chunks with byte offsets and token estimates, ready for vector databases and LLM pipelines
-- **LLM Markdown** — token-optimised single file to paste directly into any chat-based LLM
+- **LLM Markdown** — Token-optimized single file designed for direct consumption by any chat-based AI.
+- **PDF Documentation** — Clean, professional A4 reports for code reviews, audits, and archival.
+- **Structured Context** — Emits `.llmsummary.json` for downstream agentic workflows.
 
 ## 🚀 Key Features
 
-- **Three output formats**: PDF, RAG JSON, LLM Markdown — pick what you need
-- **Intelligent scanning**: recursive traversal with configurable whitelist filtering
-- **Extensive language support**: 50+ file types out of the box
-- **Versioned output files**: `-v1`, `-v2` suffixes instead of overwriting or timestamps
-- **Non-interactive mode**: `--no-interactive` for CI/CD and scripted use
-- **Smart config migration**: new defaults merge into existing config without overwriting customisations
-- **Cross-platform**: identical behaviour on Windows, macOS, and Linux
+- **LLM-First Output**: Optimized Markdown with lossless compression (whitespace stripping, JSON compaction) to maximize context window efficiency.
+- **Deep Dependency Analysis**: Internal file-level graph engine with centrality metrics and hub/node identification.
+- **Semantic Clustering**: Heuristic-based capability grouping (CLI, logic, utilities, etc.) to help AI understand project structure.
+- **Language-Aware Extraction**: Specialized adapters for JavaScript/TypeScript and Python, with a universal fallback for 50+ other languages.
+- **Professional PDF Reports**: High-quality snapshots including file trees, AI-assisted executive briefs, and syntax-highlighted source code.
+- **Intelligent Scanning**: Recursive traversal with configurable filters and robust `.csignore` support (gitignore-style syntax).
+- **Versioned Output**: Automatic `-v1`, `-v2` suffixes prevent overwriting previous reports.
+- **CI/CD Ready**: `--no-interactive` mode for automated documentation pipelines.
 
 ## 📦 Installation
 
@@ -33,451 +34,116 @@ npm install -g codesummary
 
 ## 🎯 Output Modes
 
-### 📄 PDF — Documentation & audits
+### 💬 LLM — Direct Chat Context
 
-Generate a professional A4 PDF with file structure and complete source content:
-
-```bash
-codesummary
-# or explicitly:
-codesummary --format pdf
-# Output: MYPROJECT_code.pdf
-```
-
-Best for: code reviews, client handovers, compliance audits, archival snapshots.
-
----
-
-### 🤖 RAG — Vector databases & AI pipelines
-
-Generate a structured JSON file built for embedding and retrieval:
-
-```bash
-codesummary --format rag
-# Output: MYPROJECT_rag.json
-```
-
-The JSON contains semantic chunks (split by function, class, or logical block) with:
-- Byte-accurate content offsets for fast seeking
-- SHA-256 file hashes for deduplication
-- Token estimates for context budget planning
-- Import/call extraction for graph-based retrieval
-- Full statistics for monitoring
-
-Best for: building RAG systems, loading code into a vector database (Pinecone, Qdrant, Chroma, etc.), or programmatic LLM integration where you control chunking and retrieval.
-
----
-
-### 💬 LLM — Direct chat with any AI assistant
-
-Generate a single, token-optimised Markdown file to paste directly into any LLM:
+Generate a single, token-optimized Markdown file to paste into ChatGPT, Claude, or any other LLM:
 
 ```bash
 codesummary --format llm
-# Output: MYPROJECT_llm.md
+# Output: PROJECT_llm.md
 ```
 
-The file contains a project header, a complete file tree, and each file's content in a fenced code block with syntax highlighting hints. These lossless optimisations are applied automatically to reduce token count:
+The file includes a project overview, a suggested reading order, dependency graphs, and full file contents with language hints. Lossless optimizations (normalizing line endings, stripping trailing whitespace, compacting JSON) are applied automatically.
 
-- Line endings normalised to `\n`
-- Trailing whitespace stripped per line
-- Leading/trailing blank lines removed per file
-- JSON files compacted (re-serialised without indentation)
-- Markdown files: max 2 consecutive blank lines preserved
-- All other files: max 1 consecutive blank line
+### 📄 PDF — Audits & Handovers
 
-Best for: asking any LLM chat interface to review, explain, or work with your codebase in a single paste — much more token-efficient than a PDF.
-
----
-
-### 🔄 Both — PDF + RAG together
+Generate a professional A4 PDF for human consumption:
 
 ```bash
-codesummary --format both
-# Output: MYPROJECT_code.pdf + MYPROJECT_rag.json
+codesummary --format pdf
+# Output: PROJECT_code.pdf
 ```
 
-Uses a single scan pass. If one format fails, the other still completes.
+Best for: code reviews, client handovers, compliance audits, and physical archiving.
 
 ---
 
 ## 📖 Usage
 
-### Quick start
+### Quick Start
 
 ```bash
-# First run: interactive setup wizard
+# First run: Interactive setup wizard
 codesummary
 
 # Generate LLM Markdown for the current project
 codesummary --format llm
 
-# Generate RAG JSON and save to a specific directory
-codesummary --format rag --output ./ai-data
-
-# Skip all prompts (CI-friendly)
-codesummary --format llm --no-interactive
-
-# Generate everything at once
+# Generate both formats in one run
 codesummary --format both
-```
-
-### Interactive workflow
-
-#### 1. First-run setup
 
+# Generate a task-focused context pack (prioritizes relevant files)
+codesummary --format llm --focus "authentication flow" --max-tokens 12000
 ```
-Welcome to CodeSummary!
-No configuration found. Starting setup...
-
-Where should the output be saved by default?
-> [ ] Current working directory (relative mode)
-> [x] Fixed folder (absolute mode)
 
-Enter absolute path for fixed folder:
-> ~/Desktop/CodeSummaries
-```
+### Interactive Workflow
 
-#### 2. Extension selection
+1.  **First-Run Setup**: Detects missing configuration and launches a wizard to set your default output paths.
+2.  **Extension Selection**: Choose which file types to include (e.g., `.ts`, `.py`, `.md`).
+3.  **Generation**: Files are scanned, analyzed, and rendered to your chosen format.
 
-```
-Scan Summary:
-   Extensions found: .js, .ts, .md, .json
-   Total files: 127  —  Total size: 2.4 MB
-
-Select file extensions to include:
-[x] .js → JavaScript (42 files)
-[x] .ts → TypeScript (28 files)
-[x] .md → Markdown (5 files)
-[ ] .json → JSON (52 files)
-```
+### Optional AI Enrichment
 
-#### 3. Output
+Enhance your reports with AI-generated architecture insights:
 
-```
-SUCCESS: LLM-optimised Markdown generated successfully!
+```bash
+# Using local Ollama
+codesummary --format llm --ai-semantic --provider ollama --model llama3.1
 
-   Output: ~/Desktop/CodeSummaries/MYPROJECT_llm.md
-   Extensions: .js, .ts, .md
-   Total files: 75
-   File size: 1.1 MB
-   Ready to paste into any LLM chat interface
+# Using an OpenAI-compatible endpoint
+codesummary --format llm --ai-semantic --provider openai-compatible --api-key YOUR_KEY
 ```
 
-### Command reference
-
-| Command | Description |
-| ------- | ----------- |
-| `codesummary` | Scan and generate PDF (default) |
-| `codesummary --format pdf` | Generate PDF documentation |
-| `codesummary --format rag` | Generate RAG-optimised JSON |
-| `codesummary --format llm` | Generate LLM-optimised Markdown |
-| `codesummary --format both` | Generate PDF + RAG JSON |
-| `codesummary config` | Edit configuration interactively |
-| `codesummary --show-config` | Display current configuration |
-| `codesummary --reset-config` | Reset configuration to defaults |
-
-### Options
-
-| Option | Short | Description |
-| ------ | ----- | ----------- |
-| `--format <format>` | `-f` | Output format: `pdf` (default), `rag`, `llm`, or `both` |
-| `--output <path>` | `-o` | Override output directory for this run |
-| `--no-interactive` | | Skip all prompts; auto-select all extensions |
-| `--show-config` | | Display current configuration |
-| `--reset-config` | | Reset configuration to defaults |
-| `--help` | `-h` | Show help |
-| `--version` | `-v` | Show version |
+*Note: AI enrichment is optional. If the provider is unavailable, CodeSummary falls back to heuristic analysis.*
 
 ---
 
 ## ⚙️ Configuration
 
 Configuration is stored globally at:
-
-- **Linux/macOS**: `~/.codesummary/config.json`
 - **Windows**: `%APPDATA%\CodeSummary\config.json`
+- **Linux/macOS**: `~/.codesummary/config.json`
 
-Existing configuration is never overwritten on upgrade — new defaults are merged in automatically.
-
-### Default configuration
-
-```json
-{
-  "output": {
-    "mode": "fixed",
-    "fixedPath": "~/Desktop/CodeSummaries"
-  },
-  "allowedExtensions": [
-    ".js", ".jsx", ".ts", ".tsx", ".json", ".html", ".css", ".scss",
-    ".md", ".txt", ".py", ".java", ".cs", ".cpp", ".c", ".h",
-    ".xml", ".yaml", ".yml", ".sh", ".bat", ".ps1",
-    ".cfg", ".conf", ".env", ".local", ".service", ".timer",
-    ".ino", ".j2", ".csv", ".tsv", ".crt", ".sql",
-    ".toml", ".ini", ".properties", ".tf", ".tfvars", ".proto", ".prisma",
-    ".dart", ".lua", ".r", ".ex", ".exs", ".pl", ".mk", ".cmake",
-    ".mdx", ".astro", ".graphql", ".gql"
-  ],
-  "excludeDirs": [
-    "node_modules", ".git", ".vscode", "dist", "build", "coverage",
-    "out", "__pycache__", ".next", ".nuxt",
-    ".idea", "target", ".gradle", "venv", ".venv",
-    ".pytest_cache", ".mypy_cache", ".tox", ".terraform", ".turbo",
-    ".angular", ".svelte-kit", ".yarn", ".pnpm-store",
-    ".expo", ".dart_tool", "storybook-static", "htmlcov"
-  ],
-  "excludeFiles": [
-    "*-lock.json", "*.lock", "*.min.js", "*.min.css", "*.map",
-    ".DS_Store", "Thumbs.db", "desktop.ini", "ehthumbs.db",
-    "*.pyc", "*.pyo", "*.class", "*.log", "*.tmp", "*.temp",
-    "*.swp", "*.bak", "*.orig"
-  ],
-  "settings": {
-    "documentTitle": "Project Code Summary",
-    "maxFilesBeforePrompt": 500
-  }
-}
-```
-
----
-
-## 📋 PDF structure
-
-Generated PDFs are A4 with three sections:
-
-1. **Project overview** — title, project name, generation timestamp, included file types
-2. **File structure** — complete sorted file listing
-3. **File content** — full source of every selected file, monospace font, no truncation
-
----
-
-## 🤖 RAG JSON structure
-
-```json
-{
-  "metadata": {
-    "projectName": "MyProject",
-    "generatedAt": "2025-07-31T08:00:00.000Z",
-    "version": "3.1.0"
-  },
-  "files": [
-    {
-      "id": "abc123def456",
-      "path": "src/component.js",
-      "language": "JavaScript",
-      "hash": "sha256-...",
-      "chunks": [
-        {
-          "id": "chunk_abc123def456_0",
-          "content": "function myFunction() { ... }",
-          "tokenEstimate": 45,
-          "lineStart": 1,
-          "lineEnd": 15,
-          "chunkingMethod": "semantic-function",
-          "context": "function_myFunction",
-          "imports": ["lodash", "react"],
-          "calls": ["useState", "useEffect"]
-        }
-      ]
-    }
-  ],
-  "index": {
-    "chunkOffsets": {
-      "chunk_abc123def456_0": {
-        "contentStart": 12123,
-        "contentEnd": 12356,
-        "filePath": "src/component.js"
-      }
-    },
-    "statistics": { "processingTimeMs": 245, "chunksWithValidOffsets": 387 }
-  }
-}
-```
-
-### RAG integration example
-
-```javascript
-const ragData = JSON.parse(fs.readFileSync('project_rag.json'));
-
-// Extract all chunks for embedding
-const chunks = ragData.files.flatMap(file =>
-  file.chunks.map(chunk => ({
-    id: chunk.id,
-    content: chunk.content,
-    metadata: { filePath: file.path, language: file.language }
-  }))
-);
-
-// Store in your vector database
-for (const chunk of chunks) {
-  const embedding = await embed(chunk.content);
-  await vectorDB.upsert(chunk.id, embedding, chunk.metadata);
-}
-```
-
----
-
-## 💬 LLM Markdown structure
-
-```markdown
-# MyProject — Code Summary
-
-**Generated:** 2026-04-05 | **Files:** 42 | **Total size:** 1.2 MB
-
----
-
-## File Tree
-
-```
-  src/cli.js
-  src/scanner.js
-  ...
-```
-
----
-
-## src/cli.js
-
-```js
-import chalk from 'chalk';
-...
-```
-```
-
-Paste the `.md` file directly into any LLM chat interface. No further processing needed.
-
----
-
-## 🔧 Advanced features
-
-### Versioned output filenames
-
-When the target file already exists, CodeSummary creates a versioned copy instead of overwriting:
-
-```
-MYPROJECT_llm.md        ← exists
-MYPROJECT_llm-v1.md     ← created
-MYPROJECT_llm-v1.md     ← exists on next run
-MYPROJECT_llm-v2.md     ← created
-```
-
-This applies to all three output formats (PDF, RAG JSON, LLM Markdown).
-
-### Non-interactive mode
-
-Skip all prompts and auto-select all detected extensions:
-
-```bash
-codesummary --format llm --no-interactive
-```
-
-Useful for CI pipelines or scripted documentation generation.
-
----
-
-## 🎨 Supported file types
-
-| Extension | Type | Extension | Type |
-| --------- | ---- | --------- | ---- |
-| `.js` `.jsx` | JavaScript | `.ts` `.tsx` | TypeScript |
-| `.py` | Python | `.java` | Java |
-| `.cs` | C# | `.cpp` `.c` `.h` | C/C++ |
-| `.go` | Go | `.rs` | Rust |
-| `.swift` | Swift | `.kt` | Kotlin |
-| `.rb` | Ruby | `.php` | PHP |
-| `.dart` | Dart | `.lua` | Lua |
-| `.r` | R | `.ex` `.exs` | Elixir |
-| `.pl` | Perl | `.scala` | Scala |
-| `.html` | HTML | `.css` `.scss` | CSS |
-| `.vue` | Vue.js | `.svelte` | Svelte |
-| `.astro` | Astro | `.mdx` | MDX |
-| `.json` | JSON | `.yaml` `.yml` | YAML |
-| `.toml` | TOML | `.xml` | XML |
-| `.ini` | INI | `.properties` | Java Properties |
-| `.tf` `.tfvars` | Terraform | `.proto` | Protobuf |
-| `.prisma` | Prisma | `.graphql` `.gql` | GraphQL |
-| `.sql` | SQL | `.md` `.txt` | Docs |
-| `.sh` `.bash` | Shell | `.bat` | Batch |
-| `.ps1` | PowerShell | `.mk` `.cmake` | Build |
-| `.cfg` `.conf` | Config | `.env` `.local` | Environment |
-| `.service` `.timer` | Systemd | `.ino` | Arduino |
-| `.j2` | Jinja2 | `.csv` `.tsv` | Data |
-| `.crt` | Certificate | `.dockerfile` | Docker |
-
----
-
-## 🛠️ Project structure
+Existing settings are preserved during upgrades; new defaults are merged automatically.
 
-```
-codesummary/
-├── bin/
-│   └── codesummary.js      # Entry point
-├── src/
-│   ├── cli.js              # Argument parsing, orchestration
-│   ├── scanner.js          # Recursive directory scanning
-│   ├── pdfGenerator.js     # PDF generation (PDFKit)
-│   ├── ragGenerator.js     # RAG JSON generation with semantic chunking
-│   ├── llmGenerator.js     # LLM Markdown generation with optimisations
-│   ├── configManager.js    # Global config storage and migration
-│   ├── ragConfig.js        # RAG-specific configuration and YAML loading
-│   ├── errorHandler.js     # Centralised error handling and path validation
-│   └── utils.js            # Shared utilities (formatFileSize, etc.)
-├── rag-schema.json
-├── raggen.config.yaml
-└── package.json
-```
+### `.csignore` Support
+Use a `.csignore` file in your project root to exclude sensitive paths (keys, tokens, private data) using standard `.gitignore` syntax.
 
 ---
 
-## 🔍 Troubleshooting
+## 🛠️ Project Structure
 
-**No files found after scan**
-- Check `allowedExtensions` in your config (`codesummary --show-config`)
-- Verify the directory is not listed in `excludeDirs`
-
-**Output file not generated**
-- Check write permissions on the output directory
-- Try `--output ./` to write to the current directory
-
-**Non-ASCII characters in paths cause issues**
-- Update to v1.2.0+ which fixes Windows path handling for accented characters
-
-**CI pipeline hangs**
-- Add `--no-interactive` to skip all prompts
+- `bin/codesummary.js`: CLI Entry point.
+- `src/cli.js`: Argument parsing and flow orchestration.
+- `src/scanner.js`: Recursive file system traversal and filtering.
+- `src/graph/`: Dependency graph engine and language adapters.
+- `src/llmGenerator.js`: Token-optimized Markdown generation.
+- `src/pdfGenerator.js`: High-fidelity PDF generation (PDFKit).
+- `src/configManager.js`: Global configuration and migration logic.
 
 ---
 
 ## 🤝 Contributing
 
-1. Fork the repository
-2. Clone: `git clone https://github.com/skamoll/CodeSummary.git`
-3. Install: `npm install`
-4. Test: `node bin/codesummary.js --help`
-5. Submit a pull request
+1.  Fork the repository.
+2.  Clone: `git clone https://github.com/skamoll/CodeSummary.git`
+3.  Install: `npm install`
+4.  Submit a pull request.
 
 ---
 
 ## 📄 License
 
-GNU General Public License v3.0 — see [LICENSE](LICENSE) for details.
+Distributed under the **GNU General Public License v3.0**. See `LICENSE` for details.
 
 ---
 
 ## 📊 Roadmap
 
-- [ ] Syntax highlighting in PDF output
-- [ ] Clickable table of contents in PDF
-- [x] LLM-optimised Markdown output (`--format llm`)
-- [x] Versioned output filenames (`-v1`, `-v2`)
-- [x] Non-interactive mode (`--no-interactive`)
-- [x] RAG JSON with semantic chunking
-- [ ] `--format all` (PDF + RAG + LLM in one pass)
-- [ ] Git integration (document only changed files)
-- [ ] CI/CD plugin for automated documentation
-
----
-
-## 📞 Support
+- [x] LLM-optimized Markdown output.
+- [x] Versioned output filenames.
+- [x] Deep dependency graph analysis.
+- [x] VS Code Extension (MVP).
+- [ ] Automated regression test suite.
 
-- Report bugs: [GitHub Issues](https://github.com/skamoll/CodeSummary/issues)
-- Questions: [GitHub Discussions](https://github.com/skamoll/CodeSummary/discussions)
+See [ROADMAP.md](ROADMAP.md) for detailed product direction.