thorns 5.1.9

package/README.md

@@ -0,0 +1,175 @@
+# Thorns - Ultra-Compact Codebase Intelligence
+
+Cross-platform codebase analysis using tree-sitter for maximum insight with minimal output.
+
+## Features
+
+- **Cross-platform**: WSL, Linux, Windows, macOS (arm64/x64)
+- **12 languages**: JavaScript, TypeScript, Python, Rust, Go, C/C++, Java, C#, Ruby, PHP, JSON
+- **Dependency graph**: File-level imports, orphans, circular deps
+- **Code duplication**: AST-based clone detection
+- **Coupling metrics**: Most connected files
+- **File sizes**: Distribution and largest files
+- **Identifier usage**: Most common variables/functions
+- **Complexity hotspots**: High cx/depth files
+- **Ultra-comprehensive ignoring**: Auto-loads .gitignore, .dockerignore, .npmignore + 200+ built-in patterns
+- **Smart filtering**: Ignores node_modules, target, vendor, dist, build, .cache, etc. across all languages
+- **Ultra-compact**: Zero unnecessary tokens
+- **Fast**: Native parsers, ~1000 files/sec, skips 96%+ of irrelevant files
+
+## Usage
+
+### Quick Start
+
+Analyze current directory:
+```bash
+bunx mcp-thorns
+```
+
+Analyze specific directory:
+```bash
+bunx mcp-thorns /path/to/codebase
+```
+
+### Install Globally with Bun
+
+```bash
+bun add -g mcp-thorns
+```
+
+Then use:
+```bash
+thorns /path/to/codebase
+```
+
+### Programmatic API
+
+```javascript
+import { analyze } from 'mcp-thorns';
+
+// Get ultra-compact analysis as a string
+const output = analyze('./path/to/codebase');
+console.log(output);
+
+// Or use the raw functions for custom formatting
+import { analyzeCodebase, formatUltraCompact } from 'mcp-thorns';
+
+const data = analyzeCodebase('./path/to/codebase');
+const formatted = formatUltraCompact(data);
+```
+
+## Output Format
+
+Ultra-compact cheat sheet with maximum information density:
+
+```
+━━━ 47f 8.5kL 44fn 5cls cx:7.9 d:21.7 ━━━
+JA 97% 40f 8.3kL 44fn 5c 575i 96e cx:7.9
+JS 3% 7f 214L 0fn 0c 0i 0e cx:0
+━━━ fn ━━━
+7× Ja:main(1)
+6× Ja:uuid(1)
+5× Ja:createApp(1)
+4× Ja:verifyApps(1)
+3× Ja:processNextApp(1)
+━━━ cls ━━━
+2× Ja:StatelessMCPTools
+2× Ja:ValidationSystem
+━━━ imports ━━━
+2× import { CallToolRequestSchema...
+1× import WebSocket from 'ws';
+━━━ calls ━━━
+504× console.log
+63× ws.on
+51× setTimeout
+━━━ ⚠ hotspots ━━━
+cx:57 d:19 src/validation-system.js
+cx:20 d:28 src/stateless-mcp-tools-basic.js
+```
+
+**Legend:**
+- Abbreviations: `f`=files `L`=lines `fn`=functions `cls`=classes `i`=imports `e`=exports `cx`=complexity `d`=AST-depth `(N)`=param-count
+- Issues: `orph`=orphaned-files `dup`=duplicate-code `circ`=circular-deps `in/out`=dependency-coupling
+- `TOTALS`: Total files, lines, functions, classes, avg complexity, avg depth | Issues counts
+- Language rows: % of codebase, file/line/function/class/import/export counts, avg complexity
+- `TOP-FUNCTIONS(most-defined)`: Most common function signatures (count × lang : signature)
+- `TOP-CLASSES(most-defined)`: Most common classes
+- `TOP-IMPORTS(common-deps)`: Most frequent imports
+- `TOP-CALLS(frequent-invocations)`: Most called functions/APIs
+- `HOTSPOTS(complex-files)`: Complexity hotspots - refactor candidates (cx=complexity, d=depth)
+- `ORPHANS(unused-or-entries)`: Files not imported anywhere - potential dead code or entry points
+- `COUPLING(central-files)`: Files with most dependencies - central hubs, refactor candidates (in←imports, out→uses)
+- `DUPLICATES(code-clones)`: AST-based structural clones - consolidation candidates (count × hash : files)
+- `CIRCULAR-DEPS(import-cycles)`: Import cycles - architecture issues
+- `LARGEST-FILES(split-candidates)`: Largest files - maintainability risk
+- `FILE-SIZE-DISTRIBUTION`: File size distribution by line count
+- `TOP-IDENTIFIERS(common-names)`: Most used variable names in codebase
+
+## Supported Languages
+
+- JavaScript (.js, .mjs, .cjs, .jsx)
+- TypeScript (.ts, .tsx)
+- Python (.py)
+- Rust (.rs)
+- Go (.go)
+- C (.c, .h)
+- C++ (.cpp, .cc, .cxx, .hpp)
+- Java (.java)
+- C# (.cs)
+- Ruby (.rb)
+- PHP (.php)
+- JSON (.json)
+
+## Ignored Directories & Files
+
+Automatically skips 200+ patterns including:
+- **VCS**: `.git`, `.svn`, `.hg`, `CVS`
+- **Build artifacts**: `dist`, `build`, `out`, `target`, `vendor`
+- **Language-specific**: `node_modules`, `__pycache__`, `.gradle`, `Pods`, etc.
+- **Caches**: `.cache`, `.next`, `.nuxt`, `.parcel-cache`, `.vite`, `.turbo`
+- **IDEs**: `.vscode`, `.idea`, `.vs`, `.sublime-*`
+- **Test & coverage**: `.pytest_cache`, `.nyc_output`, `coverage`, `htmlcov`
+- **Home directories**: `~/.cache`, `~/.config`, `~/.local`, `~/.npm`, `~/.cargo`, etc. (when analyzing subdirs)
+- **Lock files**: `bun.lockb`, `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`
+- **Custom patterns**: Loads and merges `.gitignore`, `.dockerignore`, `.npmignore`, `.thornsignore`
+
+Thorns prioritizes speed by ignoring 96%+ of non-code files.
+
+## Requirements
+
+- **Runtime**: Bun >= 1.0
+- **Platform**: Linux, macOS, Windows (WSL2), Docker
+- Prebuilt binaries download automatically for supported platforms
+
+## Compatibility
+
+✅ **Linux**: Ubuntu, Debian, Alpine, Fedora, Arch
+✅ **macOS**: Intel & Apple Silicon (arm64)
+✅ **Windows**: WSL2, Git Bash, PowerShell
+✅ **Docker**: All Linux images (Alpine, Debian, Ubuntu)
+✅ **CI/CD**: GitHub Actions, GitLab CI, Jenkins, CircleCI
+✅ **Cloud**: AWS Lambda (with Node/Bun layer), GCP Cloud Functions
+
+## Execution Methods
+
+| Method | Best For | Requirements |
+|--------|----------|--------------|
+| bunx | Quick analysis | `bun` |
+| Bun global | Repeated use | `bun` |
+| Programmatic | Integration | Bun |
+
+## How It Works
+
+1. Walks directory tree, filtering by extension
+2. Parses each file with tree-sitter
+3. Analyzes AST for functions, classes, imports, exports, complexity
+4. Aggregates statistics by language
+5. Outputs compact summary
+
+## Performance
+
+- Skips files > 1MB
+- Uses native parsers (not WASM)
+- Minimal memory footprint
+- Processes ~1000 files/second on modern hardware
+# Triggered npm publishing

package/advanced-metrics.js

@@ -0,0 +1,203 @@
+import { createHash } from 'crypto';
+
+export function extractAdvancedMetrics(tree, sourceCode) {
+  const metrics = {
+    identifiers: new Map(),
+    functionLengths: [],
+    functionParams: [],
+    nestingDepths: [],
+    commentLines: 0,
+    blankLines: 0,
+    longFunctions: [],
+    deeplyNested: [],
+    manyParams: []
+  };
+
+  const lines = sourceCode.split('\n');
+  metrics.blankLines = lines.filter(l => l.trim() === '').length;
+  metrics.commentLines = lines.filter(l => {
+    const t = l.trim();
+    return t.startsWith('//') || t.startsWith('#') || t.startsWith('/*') || t.startsWith('*');
+  }).length;
+
+  function getNodeDepth(node, depth = 0) {
+    let maxDepth = depth;
+    for (const child of node.children) {
+      maxDepth = Math.max(maxDepth, getNodeDepth(child, depth + 1));
+    }
+    return maxDepth;
+  }
+
+  function countParams(node) {
+    let count = 0;
+    function traverse(n) {
+      if (n.type === 'parameter' || n.type === 'formal_parameter' ||
+          n.type.includes('param')) {
+        count++;
+      }
+      for (const child of n.children) traverse(child);
+    }
+    traverse(node);
+    return count;
+  }
+
+  function getFunctionBody(node) {
+    for (const child of node.children) {
+      if (child.type === 'block' || child.type === 'statement_block' ||
+          child.type === 'body' || child.type.includes('body')) {
+        return child;
+      }
+    }
+    return node;
+  }
+
+  function traverse(node, depth = 0) {
+    const type = node.type;
+
+    // Track identifiers
+    if (type === 'identifier' || type === 'property_identifier' ||
+        type === 'type_identifier' || type === 'field_identifier') {
+      const name = node.text;
+      if (name && name.length < 50) {
+        metrics.identifiers.set(name, (metrics.identifiers.get(name) || 0) + 1);
+      }
+    }
+
+    // Track function metrics
+    if (type.includes('function') && type.includes('declaration') ||
+        type === 'method_definition' || type === 'function_item') {
+      const body = getFunctionBody(node);
+      const bodyLines = body.text.split('\n').length;
+      const params = countParams(node);
+      const nestDepth = getNodeDepth(body);
+
+      metrics.functionLengths.push(bodyLines);
+      metrics.functionParams.push(params);
+      metrics.nestingDepths.push(nestDepth);
+
+      if (bodyLines > 50) {
+        metrics.longFunctions.push({ lines: bodyLines, text: node.text.slice(0, 100) });
+      }
+      if (nestDepth > 5) {
+        metrics.deeplyNested.push({ depth: nestDepth, text: node.text.slice(0, 100) });
+      }
+      if (params > 5) {
+        metrics.manyParams.push({ params, text: node.text.slice(0, 100) });
+      }
+    }
+
+    for (const child of node.children) {
+      traverse(child, depth + 1);
+    }
+  }
+
+  traverse(tree.rootNode);
+  return metrics;
+}
+
+export function detectDuplication(fileMetrics) {
+  const hashes = new Map();
+  const duplicates = [];
+
+  for (const [file, data] of Object.entries(fileMetrics)) {
+    if (!data.functionHashes) continue;
+
+    for (const [funcSig, hash] of Object.entries(data.functionHashes)) {
+      if (!hashes.has(hash)) {
+        hashes.set(hash, []);
+      }
+      hashes.get(hash).push({ file, func: funcSig });
+    }
+  }
+
+  for (const [hash, instances] of hashes) {
+    if (instances.length > 1) {
+      duplicates.push({
+        hash,
+        count: instances.length,
+        instances: instances.slice(0, 5)
+      });
+    }
+  }
+
+  return duplicates.sort((a, b) => b.count - a.count).slice(0, 10);
+}
+
+export function hashFunction(node) {
+  // Create structural hash of function (ignoring variable names)
+  const structure = [];
+
+  function traverse(n) {
+    structure.push(n.type);
+    for (const child of n.children) {
+      if (!child.type.includes('identifier') && !child.type.includes('comment')) {
+        traverse(child);
+      }
+    }
+  }
+
+  traverse(node);
+  return createHash('md5').update(structure.join(':')).digest('hex').slice(0, 8);
+}
+
+export function detectCircularDeps(graph) {
+  const cycles = [];
+  const visiting = new Set();
+  const visited = new Set();
+
+  function dfs(node, path) {
+    if (visiting.has(node)) {
+      const cycleStart = path.indexOf(node);
+      if (cycleStart !== -1) {
+        cycles.push(path.slice(cycleStart).concat(node));
+      }
+      return;
+    }
+
+    if (visited.has(node)) return;
+
+    visiting.add(node);
+    path.push(node);
+
+    const nodeData = graph.nodes.get(node);
+    if (nodeData) {
+      for (const dep of nodeData.importsFrom) {
+        dfs(dep, [...path]);
+      }
+    }
+
+    visiting.delete(node);
+    visited.add(node);
+  }
+
+  for (const node of graph.nodes.keys()) {
+    if (!visited.has(node)) {
+      dfs(node, []);
+    }
+  }
+
+  return cycles.slice(0, 5);
+}
+
+export function analyzeFileSizes(fileMetrics) {
+  const sizes = [];
+
+  for (const [file, data] of Object.entries(fileMetrics)) {
+    if (data.loc) {
+      sizes.push({ file, lines: data.loc });
+    }
+  }
+
+  sizes.sort((a, b) => b.lines - a.lines);
+
+  return {
+    largest: sizes.slice(0, 10),
+    distribution: {
+      tiny: sizes.filter(s => s.lines < 50).length,
+      small: sizes.filter(s => s.lines >= 50 && s.lines < 200).length,
+      medium: sizes.filter(s => s.lines >= 200 && s.lines < 500).length,
+      large: sizes.filter(s => s.lines >= 500 && s.lines < 1000).length,
+      huge: sizes.filter(s => s.lines >= 1000).length
+    }
+  };
+}

package/analyzer.js

@@ -0,0 +1,257 @@
+import Parser from 'tree-sitter';
+import { createHash } from 'crypto';
+
+export function extractEntities(tree, sourceCode, lang) {
+  const entities = {
+    functions: new Map(),
+    classes: new Map(),
+    imports: new Set(),
+    exports: new Set(),
+    patterns: new Map(),
+    asyncPatterns: { async: 0, await: 0, promise: 0, callback: 0, thenCatch: 0 },
+    errorPatterns: { tryCatch: 0, throw: 0, errorTypes: new Set() },
+    internalCalls: new Map(),
+    constants: [],
+    globalState: [],
+    envVars: new Set(),
+    urls: new Set(),
+    filePaths: new Set(),
+    eventPatterns: { emitters: 0, listeners: 0 },
+    httpPatterns: { routes: [], fetches: 0, axios: 0 },
+    storagePatterns: { sql: 0, fileOps: 0, json: 0 },
+    basicStats: { functions: 0, classes: 0, imports: 0, exports: 0, complexity: 0, lines: 0 }
+  };
+
+  entities.basicStats.lines = sourceCode.split('\n').length;
+
+  function hash(text) {
+    return createHash('md5').update(text).digest('hex').slice(0, 8);
+  }
+
+  function extractName(node) {
+    if (node.type === 'identifier' || node.type === 'property_identifier' || node.type === 'type_identifier') {
+      return node.text;
+    }
+    for (const child of node.children) {
+      if (child.type.includes('identifier')) return child.text;
+    }
+    return null;
+  }
+
+  function traverse(node) {
+    const type = node.type;
+    const text = node.text;
+
+    // Functions
+    if (type.includes('function') && type.includes('declaration') ||
+        type === 'method_definition' || type === 'function_item') {
+      const name = extractName(node) || 'anon';
+      const params = text.match(/\((.*?)\)/)?.[1] || '';
+      const sig = `${name}(${params.split(',').length})`;
+      const h = hash(text);
+      const existing = entities.functions.get(sig) || { count: 0, hash: h, params };
+      existing.count++;
+      entities.functions.set(sig, existing);
+      entities.basicStats.functions++;
+    }
+
+    // Classes/structs/types
+    if (type.includes('class') && type.includes('declaration') ||
+        type === 'struct_item' || type === 'enum_item' || type === 'interface_declaration') {
+      const name = extractName(node) || 'anon';
+      const existing = entities.classes.get(name) || { count: 0, type: type.split('_')[0] };
+      existing.count++;
+      entities.classes.set(name, existing);
+      entities.basicStats.classes++;
+    }
+
+    // Imports
+    if (type.includes('import')) {
+      const imp = text.replace(/\s+/g, ' ').slice(0, 60);
+      entities.imports.add(imp);
+      entities.basicStats.imports++;
+    }
+
+    // Exports
+    if (type.includes('export')) {
+      const exp = extractName(node) || text.slice(0, 30);
+      entities.exports.add(exp);
+      entities.basicStats.exports++;
+    }
+
+    // Complexity
+    if (type === 'if_statement' || type === 'while_statement' || type === 'for_statement' ||
+        type === 'case_statement' || type === 'catch_clause') {
+      entities.basicStats.complexity++;
+    }
+
+    // Patterns (API calls, common patterns)
+    if (type === 'call_expression' || type === 'call' || type === 'function_call') {
+      const name = node.children[0]?.text || '';
+      if (name && name.length < 30) {
+        const existing = entities.patterns.get(name) || 0;
+        entities.patterns.set(name, existing + 1);
+
+        if (!name.includes('.') && name.match(/^[a-z]/)) {
+          entities.internalCalls.set(name, (entities.internalCalls.get(name) || 0) + 1);
+        }
+
+        if (name.endsWith('.then') || name.endsWith('.catch')) {
+          entities.asyncPatterns.thenCatch++;
+        }
+        if (name === 'Promise' || name.startsWith('Promise.')) {
+          entities.asyncPatterns.promise++;
+        }
+      }
+    }
+
+    // Async patterns
+    if (type === 'async_function' || type === 'async_arrow_function' || type === 'async_function_declaration' || text.startsWith('async ')) {
+      entities.asyncPatterns.async++;
+    }
+    if (type === 'await_expression') {
+      entities.asyncPatterns.await++;
+    }
+
+    // Callback patterns (function params that look like callbacks)
+    if ((type === 'arrow_function' || type === 'function_expression') && node.parent?.type === 'arguments') {
+      entities.asyncPatterns.callback++;
+    }
+
+    // Error handling patterns
+    if (type === 'try_statement') {
+      entities.errorPatterns.tryCatch++;
+    }
+    if (type === 'throw_statement') {
+      entities.errorPatterns.throw++;
+      const errType = text.match(/throw new (\w+)/)?.[1];
+      if (errType) entities.errorPatterns.errorTypes.add(errType);
+    }
+
+    // Constants (module-level const declarations with UPPER_CASE names or literal values)
+    if (type === 'lexical_declaration' && text.startsWith('const ')) {
+      const isTopLevel = node.parent?.type === 'program' || node.parent?.type === 'export_statement';
+      if (isTopLevel) {
+        const nameMatch = text.match(/const\s+(\w+)\s*=/);
+        if (nameMatch) {
+          const name = nameMatch[1];
+          const isConstStyle = name === name.toUpperCase() && name.length > 2;
+          const hasLiteralValue = /=\s*(\d+|['"`][^'"`]*['"`]|true|false|null)/.test(text);
+          if (isConstStyle || hasLiteralValue) {
+            const valueMatch = text.match(/=\s*([^;]+)/);
+            const value = valueMatch ? valueMatch[1].trim().slice(0, 30) : '';
+            entities.constants.push({ name, value });
+          }
+        }
+      }
+    }
+
+    // Global state (module-level let/var declarations)
+    if ((type === 'lexical_declaration' && text.startsWith('let ')) || type === 'variable_declaration') {
+      const isTopLevel = node.parent?.type === 'program' || node.parent?.type === 'export_statement';
+      if (isTopLevel && !text.startsWith('const ')) {
+        const nameMatch = text.match(/(let|var)\s+(\w+)/);
+        if (nameMatch) {
+          entities.globalState.push(nameMatch[2]);
+        }
+      }
+    }
+
+    // Environment variables
+    const envMatch = text.match(/process\.env\.(\w+)/g);
+    if (envMatch) {
+      envMatch.forEach(e => entities.envVars.add(e.replace('process.env.', '')));
+    }
+
+    // URLs and paths in strings
+    if (type === 'string' || type === 'template_string') {
+      const urlMatch = text.match(/https?:\/\/[^\s'"`,)]+/g);
+      if (urlMatch) urlMatch.forEach(u => entities.urls.add(u.slice(0, 60)));
+
+      const pathMatch = text.match(/['"](\.?\.?\/[\w\-./]+)['"]/);
+      if (pathMatch && pathMatch[1].includes('/')) {
+        entities.filePaths.add(pathMatch[1]);
+      }
+    }
+
+    // Event patterns
+    if (type === 'call_expression') {
+      const callText = node.children[0]?.text || '';
+      if (callText.match(/\.(on|once|addEventListener|addListener)\s*$/)) {
+        entities.eventPatterns.listeners++;
+      }
+      if (callText.match(/\.(emit|dispatch|dispatchEvent|trigger)\s*$/)) {
+        entities.eventPatterns.emitters++;
+      }
+
+      // HTTP patterns
+      if (callText === 'fetch' || callText.endsWith('.fetch')) {
+        entities.httpPatterns.fetches++;
+      }
+      if (callText.match(/axios\.(get|post|put|delete|patch)/)) {
+        entities.httpPatterns.axios++;
+      }
+      if (callText.match(/\.(get|post|put|delete|patch|use)\s*$/) && text.includes("'/" )) {
+        const routeMatch = text.match(/['"](\/.+?)['"]/);
+        if (routeMatch) entities.httpPatterns.routes.push(routeMatch[1]);
+      }
+
+      // Storage patterns
+      if (callText.match(/\.(query|execute|prepare|run)\s*$/) || text.includes('SELECT') || text.includes('INSERT')) {
+        entities.storagePatterns.sql++;
+      }
+      if (callText.match(/(readFile|writeFile|readdir|mkdir|unlink|stat)/)) {
+        entities.storagePatterns.fileOps++;
+      }
+      if (callText.match(/JSON\.(parse|stringify)/)) {
+        entities.storagePatterns.json++;
+      }
+    }
+
+    for (const child of node.children) {
+      traverse(child);
+    }
+  }
+
+  traverse(tree.rootNode);
+  return entities;
+}
+
+export function calculateMetrics(tree, sourceCode) {
+  let depth = 0, maxDepth = 0, nodes = 0;
+  let branches = 0, loops = 0, returns = 0;
+
+  function traverse(node, level = 0) {
+    nodes++;
+    maxDepth = Math.max(maxDepth, level);
+
+    const type = node.type;
+    if (['if_statement', 'switch_statement', 'case_statement', 'conditional_expression', 'match_expression'].includes(type)) branches++;
+    if (['while_statement', 'for_statement', 'loop_expression', 'for_in_statement'].includes(type)) loops++;
+    if (type.includes('return')) returns++;
+
+    for (const child of node.children) {
+      traverse(child, level + 1);
+    }
+  }
+
+  traverse(tree.rootNode);
+
+  const lines = sourceCode.split('\n');
+  const blankLines = lines.filter(l => l.trim() === '').length;
+  const commentLines = lines.filter(l => {
+    const t = l.trim();
+    return t.startsWith('//') || t.startsWith('#') || t.startsWith('/*') || t.startsWith('*');
+  }).length;
+
+  return {
+    nodes,
+    maxDepth,
+    branches,
+    loops,
+    returns,
+    loc: lines.length,
+    sloc: lines.length - blankLines - commentLines,
+    density: nodes / lines.length
+  };
+}