code-graph-llm 1.0.0 → 1.1.0

package/README.md

@@ -1,97 +1,56 @@
 # CODE-GRAPH
 
-A language-agnostic, ultra-compact codebase mapper designed specifically for LLM agents to optimize context and token usage.
+A language-agnostic, ultra-compact codebase mapper designed specifically for LLM agents to optimize context and token usage. It doesn't just list files; it provides a high-signal "map" of your project's architecture, including descriptions and signatures.
 
-## Installation
+## Features
+- **Smart Context Extraction:** Captures JSDoc, Python docstrings, and preceding comments for files and symbols.
+- **Signature Fallback:** Automatically extracts function signatures (parameters/types) if documentation is missing.
+- **Compact & Dense:** Optimized for LLM token efficiency, replacing expensive recursive file scans.
+- **Language-Agnostic:** Optimized regex support for JS/TS, Python, Go, Rust, Java, C#, C/C++, Swift, PHP, Ruby, Dart, and more.
+- **Recursive Ignore Logic:** Deeply respects `.gitignore` and standard excludes (`node_modules`, `.git`).
+- **Live Sync:** Continuous background updates or Git pre-commit hooks.
 
-### 1. Install via NPM (Recommended)
-You can install **Code-Graph** globally or as a project-specific dependency without cloning the repository.
+## Installation
 
-**Global Installation:**
+### 1. Install via NPM
 ```bash
+# Global installation for CLI use
 npm install -g code-graph-llm
-```
 
-**Local Project Dependency:**
-```bash
+# Local project dependency
 npm install --save-dev code-graph-llm
 ```
 
-### 2. Direct Usage
-Once installed, you can run it from any project directory:
+### 2. Basic Usage
 ```bash
-# Generate the map
+# Generate the llm-code-graph.md map
 code-graph generate
 
-# Start the live watcher
+# Start the live watcher for real-time updates
 code-graph watch
-```
-
-## Usage in Different Workflows
 
-### 1. Node.js (package.json)
-If installed as a dependency, add it to your scripts:
-```json
-"scripts": {
-  "postinstall": "code-graph generate",
-  "pretest": "code-graph generate"
-}
-```
-
-### 2. Git Integration
-Automatically update and stage `llm-code-graph.md` on every commit:
-```bash
+# Install the Git pre-commit hook
 code-graph install-hook
 ```
 
----
-
 ## LLM Usage & Token Efficiency
 
-To achieve the best performance and minimize token consumption, follow these guidelines:
+### The "Read First" Strategy
+Instruct your LLM agent to read `llm-code-graph.md` as its first step. The file uses a dense format that provides immediate architectural context:
 
-### 1. The "Read First" Rule
-Always instruct the LLM agent to read `llm-code-graph.md` as its first step. It provides an immediate, high-level map without scanning every directory.
+**Example Map Entry:**
+```markdown
+- src/auth.js | desc: Handles user authentication and JWT validation.
+  - syms: [login [ (username, password) ], validateToken [ (token: string) ]]
+```
 
 **Example System Prompt:**
-> "Before performing any tasks, read `llm-code-graph.md` to understand the project structure. Use this map to target specific files rather than scanning the entire codebase."
-
-### 2. Targeted File Reads
-The `|syms:[...]` metadata allows the LLM to identify exactly which file contains a specific function or class. It can jump directly to the relevant file.
-
----
-
-## Language Examples & Build Phase Integration
+> "Before acting, read `llm-code-graph.md`. It contains the project map, file descriptions, and function signatures. Use this to locate relevant logic instead of scanning the full codebase."
 
-### 1. Rust (build.rs)
-```rust
-use std::process::Command;
-fn main() {
-    Command::new("code-graph").arg("generate").status().unwrap();
-}
-```
-
-### 2. Java/Kotlin (Maven/Gradle)
-
-#### Maven (pom.xml)
-```xml
-<plugin>
-  <groupId>org.codehaus.mojo</groupId>
-  <artifactId>exec-maven-plugin</artifactId>
-  <executions>
-    <execution>
-      <phase>compile</phase>
-      <goals><goal>exec</goal></goals>
-      <configuration>
-        <executable>code-graph</executable>
-        <arguments><argument>generate</argument></arguments>
-      </configuration>
-    </execution>
-  </executions>
-</plugin>
-```
+## Build Phase Integration
 
-#### Gradle (Groovy DSL)
+### 1. Java/Kotlin (Maven/Gradle)
+**Gradle (Groovy):**
 ```groovy
 task generateCodeGraph(type: Exec) {
     commandLine 'code-graph', 'generate'
@@ -99,19 +58,31 @@ task generateCodeGraph(type: Exec) {
 compileJava.dependsOn generateCodeGraph
 ```
 
-### 3. Python Integration
-Add this to your `Makefile`:
+### 2. Python
+**Makefile:**
 ```makefile
 map:
 	code-graph generate
+test: map
+	pytest
 ```
 
----
+### 3. Rust (build.rs)
+```rust
+use std::process::Command;
+fn main() {
+    Command::new("code-graph").arg("generate").status().unwrap();
+}
+```
 
 ## How it works
-The tool scans your project directory (respecting `.gitignore`), extracts classes, functions, and exports using optimized regular expressions, and compiles them into a dense, machine-readable `llm-code-graph.md` file.
-
-## Publishing as a Package (For Developers)
-To publish this to the NPM registry:
-1. Log in: `npm login`
-2. Publish: `npm publish --access public` (Ensure the name in `package.json` is unique, e.g., `code-graph-llm`).
+1. **File Scanning:** Recursively walks the directory, ignoring patterns in `.gitignore`.
+2. **Context Extraction:** Scans for classes, functions, and variables.
+3. **Docstring Capture:** If a symbol has a preceding comment (`//`, `/**`, `#`, `"""`), it's captured as a description.
+4. **Signature Capture:** If no comment is found, it captures the declaration signature (parameters) as a fallback.
+5. **Compilation:** Writes a single, minified `llm-code-graph.md` file designed for machine consumption.
+
+## Publishing as a Package
+To share your own version:
+1. `npm login`
+2. `npm publish --access public`

package/index.js

@@ -13,14 +13,38 @@ const IGNORE_FILE = '.gitignore';
 const DEFAULT_MAP_FILE = 'llm-code-graph.md';
 
 const SYMBOL_REGEXES = [
-  /\b(?:class|interface|type|struct|enum)\s+([a-zA-Z_]\w*)/g,
-  /\b(?:function|def|fn|func|void|fun)\s+([a-zA-Z_]\w*)/g,
-  /\bconst\s+([a-zA-Z_]\w*)\s*=\s*(?:async\s*)?\([^)]*\)\s*=>/g, // Arrow functions
-  /\bexport\s+(?:const|let|var|function|class|type|interface|enum)\s+([a-zA-Z_]\w*)/g
+  // Types, Classes, Interfaces, and Containers (Universal)
+  /\b(?:class|interface|type|struct|enum|protocol|extension|trait|module|namespace|object)\s+([a-zA-Z_]\w*)/g,
+  
+  // Explicit Function Keywords (JS, Python, Go, Rust, Ruby, PHP, Swift, Kotlin, Dart)
+  /\b(?:function|def|fn|func|fun|method|procedure|sub|routine)\s+([a-zA-Z_]\w*)/g,
+  
+  // C-style / Java / C# / TypeScript Method Patterns
+  // Matches: ReturnType Name(...) or AccessModifier Name(...)
+  /\b(?:void|async|public|private|protected|static|virtual|override|readonly|int|float|double|char|bool|string|val|var|let|const|final)\s+([a-zA-Z_]\w*)(?=\s*\(|(?:\s*:\s*\w+)?\s*=>)/g,
+  
+  // Exported symbols (JS/TS specific but captures named exports)
+  /\bexport\s+(?:default\s+)?(?:const|let|var|function|class|type|interface|enum|async|val)\s+([a-zA-Z_]\w*)/g,
+  
+  // Ruby: def name, class Name, module Name (defs covered by Explicit Function Keywords)
+  
+  // PHP: class Name, interface Name, trait Name, function Name
+  
+  // Swift: func name, class Name, struct Name, protocol Name, extension Name
+  
+  // Dart: class Name, void name, var name (void/var covered by C-style pattern)
+];
+
+const SUPPORTED_EXTENSIONS = [
+  '.js', '.ts', '.jsx', '.tsx', 
+  '.py', '.go', '.rs', '.java', 
+  '.cpp', '.c', '.h', '.hpp', '.cc',
+  '.rb', '.php', '.swift', '.kt', 
+  '.cs', '.dart', '.scala', '.m', '.mm'
 ];
 
 function getIgnores(cwd) {
-  const ig = ignore().add(['.git', 'node_modules', DEFAULT_MAP_FILE]);
+  const ig = ignore().add(['.git', 'node_modules', DEFAULT_MAP_FILE, 'package-lock.json']);
   const ignorePath = path.join(cwd, IGNORE_FILE);
   if (fs.existsSync(ignorePath)) {
     ig.add(fs.readFileSync(ignorePath, 'utf8'));
@@ -29,14 +53,46 @@ function getIgnores(cwd) {
 }
 
 function extractSymbols(content) {
-  const symbols = new Set();
+  const symbols = [];
   for (const regex of SYMBOL_REGEXES) {
     let match;
+    regex.lastIndex = 0;
     while ((match = regex.exec(content)) !== null) {
-      if (match[1]) symbols.add(match[1]);
+      if (match[1]) {
+        const symbolName = match[1];
+        if (['if', 'for', 'while', 'switch', 'return', 'await', 'yield'].includes(symbolName)) continue;
+
+        // 1. Extract preceding comment/docstring
+        const linesBefore = content.substring(0, match.index).split('\n');
+        let comment = '';
+        for (let i = linesBefore.length - 1; i >= 0; i--) {
+          const line = linesBefore[i].trim();
+          if (line.startsWith('//') || line.startsWith('*') || line.startsWith('"""') || line.startsWith('#')) {
+            const clean = line.replace(/[\/*#"]/g, '').trim();
+            if (clean) comment = clean + (comment ? ' ' + comment : '');
+            if (comment.length > 80) break; 
+          } else if (line === '' && comment === '') {
+            continue;
+          } else {
+            break;
+          }
+        }
+
+        // 2. Backup: Extract Signature (Parameters/Type) if no comment
+        let context = comment;
+        if (!context) {
+          const remainingLine = content.substring(match.index + match[0].length).split('\n')[0];
+          const sigMatch = remainingLine.match(/^[^:{;]*/);
+          if (sigMatch && sigMatch[0].trim()) {
+            context = sigMatch[0].trim();
+          }
+        }
+        
+        symbols.push(context ? `${symbolName} [${context}]` : symbolName);
+      }
     }
   }
-  return Array.from(symbols).sort();
+  return Array.from(new Set(symbols)).sort();
 }
 
 async function generate(cwd = process.cwd()) {
@@ -47,20 +103,39 @@ async function generate(cwd = process.cwd()) {
     const entries = fs.readdirSync(dir, { withFileTypes: true });
     for (const entry of entries) {
       const fullPath = path.join(dir, entry.name);
-      const relativePath = path.relative(cwd, fullPath);
+      let relativePath = path.relative(cwd, fullPath);
+      const normalizedPath = relativePath.replace(/\\/g, '/');
+      const checkPath = entry.isDirectory() ? `${normalizedPath}/` : normalizedPath;
 
-      if (ig.ignores(relativePath)) continue;
+      if (ig.ignores(checkPath)) continue;
 
       if (entry.isDirectory()) {
         walk(fullPath);
       } else if (entry.isFile()) {
         const ext = path.extname(entry.name);
-        if (['.js', '.ts', '.py', '.go', '.rs', '.java', '.cpp', '.c', '.h', '.rb', '.php', '.swift', '.kt'].includes(ext)) {
+        if (SUPPORTED_EXTENSIONS.includes(ext)) {
           const content = fs.readFileSync(fullPath, 'utf8');
+          
+          // Extract file-level description
+          const firstLines = content.split('\n').slice(0, 5);
+          let fileDesc = '';
+          for (const line of firstLines) {
+            const trimmed = line.trim();
+            if (trimmed.startsWith('//') || trimmed.startsWith('#') || trimmed.startsWith('/*')) {
+              fileDesc += trimmed.replace(/[\/*#]/g, '').trim() + ' ';
+            }
+          }
+
           const symbols = extractSymbols(content);
-          files.push({ path: relativePath, symbols });
+          
+          // Backup: If no file description, provide a summary
+          if (!fileDesc.trim() && symbols.length > 0) {
+            fileDesc = `Contains ${symbols.length} symbols.`;
+          }
+
+          files.push({ path: normalizedPath, desc: fileDesc.trim(), symbols });
         } else {
-           files.push({ path: relativePath, symbols: [] });
+           files.push({ path: normalizedPath, desc: '', symbols: [] });
         }
       }
     }
@@ -69,8 +144,9 @@ async function generate(cwd = process.cwd()) {
   walk(cwd);
 
   const output = files.map(f => {
-    const symStr = f.symbols.length > 0 ? `|syms:[${f.symbols.join(',')}]` : '';
-    return `- ${f.path}${symStr}`;
+    const descStr = f.desc ? ` | desc: ${f.desc.substring(0, 100)}` : '';
+    const symStr = f.symbols.length > 0 ? `\n  - syms: [${f.symbols.join(', ')}]` : '';
+    return `- ${f.path}${descStr}${symStr}`;
   }).join('\n');
 
   const header = `# CODE_GRAPH_MAP\n> LLM_ONLY: DO NOT EDIT. COMPACT PROJECT MAP.\n\n`;

package/llm-code-graph.md

@@ -2,7 +2,7 @@
 > LLM_ONLY: DO NOT EDIT. COMPACT PROJECT MAP.
 
 - .gitignore
-- index.js|syms:[debouncedGenerate,extractSymbols,generate,getIgnores,installHook,walk,watch]
-- package-lock.json
+- index.js | desc: !usrbinenv node
+  - syms: [Name [Dart:], Name [PHP: class Name, interface Name, trait Name,], Name [PHP: class Name, interface Name,], Name [PHP: class Name,], Name [PHP:], Name [Ruby: def name, class Name,], Name [Ruby: def name,], Name [Swift: func name, class Name, struct Name, protocol Name,], Name [Swift: func name, class Name, struct Name,], Name [Swift: func name, class Name,], Name [Swift: func name,], extractSymbols [(content)], generate [(cwd = process.cwd())], getIgnores [(cwd)], installHook [(cwd = process.cwd())], name [Dart: class Name, void name,], name [Ruby:], name [Swift:], walk [(dir)], watch [(cwd = process.cwd())]]
 - package.json
 - README.md

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "code-graph-llm",
-  "version": "1.0.0",
+  "version": "1.1.0",
   "description": "Compact, language-agnostic codebase mapper for LLM token efficiency.",
   "main": "index.js",
   "bin": {