depgraph-core 1.0.2 → 1.5.0

package/docs/README.md

@@ -0,0 +1,56 @@
+# DepGraph — Developer Documentation
+
+Welcome to the internal documentation for **DepGraph Core**.
+
+This folder explains how every moving part of the compiler works so you can contribute confidently, extend it with new languages, or debug an unexpected result.
+
+---
+
+## Contents
+
+| File | What it covers |
+|---|---|
+| [architecture.md](./architecture.md) | Big-picture overview — how the pipeline fits together |
+| [stage-collector.md](./stage-collector.md) | Stage 1 — File collection (scanning the project directory) |
+| [stage-parser.md](./stage-parser.md) | Stage 2 — Source file parsing (entities, imports, exports) |
+| [stage-graph.md](./stage-graph.md) | Stage 3 — Dependency graph construction |
+| [stage-metrics.md](./stage-metrics.md) | Stage 4 — Metrics computation (centrality, degrees) |
+| [stage-impact.md](./stage-impact.md) | Stage 5 — Impact simulation (BFS + risk scoring) |
+| [stage-output.md](./stage-output.md) | Stage 6 — JSON report generation |
+| [language-registry.md](./language-registry.md) | The language plugin system — how to add a new language |
+| [data-types.md](./data-types.md) | All shared TypeScript interfaces, explained |
+
+---
+
+## Quick Mental Model
+
+```
+ Your project folder
+       │
+       ▼
+ ┌─────────────┐
+ │  Collector  │  ← finds every eligible source file
+ └──────┬──────┘
+        ▼
+ ┌─────────────┐
+ │   Parser    │  ← reads each file; extracts entities, imports, exports
+ └──────┬──────┘
+        ▼
+ ┌─────────────┐
+ │    Graph    │  ← links entities together through their imports
+ └──────┬──────┘
+        ▼
+ ┌─────────────┐
+ │   Metrics   │  ← computes in/out-degree and centrality for every node
+ └──────┬──────┘
+        ▼
+ ┌─────────────┐
+ │   Impact    │  ← (optional) BFS from a target node → risk report
+ └──────┬──────┘
+        ▼
+ ┌─────────────┐
+ │   Output    │  ← serialises everything to depgraph-output.json
+ └─────────────┘
+```
+
+Each stage is self-contained. Data flows **forward only** — no stage reaches back to an earlier one.

package/docs/architecture.md

@@ -0,0 +1,89 @@
+# Architecture Overview
+
+> **File**: `src/main.ts` — the orchestration entry point  
+> **Role**: Wires together all six stages in sequential order and handles CLI argument parsing.
+
+---
+
+## The Pipeline at a Glance
+
+DepGraph is a **linear, stage-based compiler**. There is no framework magic — it's a plain chain of function calls where the output of each stage becomes the input of the next.
+
+```
+collectFiles()
+    │  string[]          (file paths)
+    ▼
+parseFiles()
+    │  ParsedFile[]      (entities, imports, exports per file)
+    ▼
+buildGraph()
+    │  DepGraph          (nodes + edges map)
+    ▼
+computeMetrics()
+    │  DepGraph          (same graph, nodes now have inDegree / outDegree / centralityScore)
+    ▼
+simulateImpact()         ← only runs when --impact flag is provided
+    │  ImpactReport
+    ▼
+writeOutput()
+       depgraph-output.json
+```
+
+Every stage lives in its own file under `src/stages/`. They are pure functions — given the same input they always produce the same output, and they never touch the file system except for the collector (reading) and output (writing).
+
+---
+
+## Entry Point — `src/main.ts`
+
+`main.ts` does four things and nothing else:
+
+1. **Parses CLI flags** — `--output`, `--impact`, `--verbose`, `--no-color`, `--help`
+2. **Runs the pipeline** inside a single `try/catch` block
+3. **Prints progress** to stdout (banner, summary, impact table)
+4. **Exits with code 1** on any unhandled error
+
+### Key variables wired at startup
+
+```ts
+const projectDir  = args[0];              // the path to scan
+const outputPath  = getFlag('--output');  // where to write the JSON
+const impactTarget = getFlag('--impact'); // entity name to simulate
+```
+
+### Execution order in the `try` block
+
+```ts
+const files   = collectFiles(projectDir);   // Stage 1
+const parsed  = parseFiles(files);          // Stage 2
+const graph   = buildGraph(parsed);         // Stage 3
+const metrics = computeMetrics(graph);      // Stage 4
+// Stage 5 — optional
+const impact  = impactTarget
+  ? simulateImpact(metrics, impactTarget, impactDesc)
+  : undefined;
+writeOutput(metrics, parsed, outputPath, impact); // Stage 6
+```
+
+---
+
+## Design Principles
+
+| Principle | How it's applied |
+|---|---|
+| **Single responsibility** | Each stage file exports exactly one primary function |
+| **No hidden state** | All data passed explicitly between stages |
+| **Fail loudly** | `process.exit(1)` on unrecoverable errors — no silent swallowing |
+| **Zero runtime dependencies** | Only Node.js built-ins (`fs`, `path`) plus TypeScript |
+| **Plugin language support** | Languages register themselves via the registry (see [language-registry.md](./language-registry.md)) |
+
+---
+
+## Adding a New Stage
+
+If you need to insert a new processing step (e.g. a linting stage), follow this pattern:
+
+1. Create `src/stages/my-stage.ts` and export a pure function.
+2. Import and call it in `main.ts` after the stage it depends on.
+3. Thread its output into downstream stages.
+
+No registration, no magic — just a function call.

package/docs/data-types.md

@@ -0,0 +1,218 @@
+# Data Types Reference
+
+> **File**: `src/types.ts`  
+> All shared TypeScript interfaces used across every stage of the pipeline.
+
+---
+
+## Data Flow Summary
+
+```
+RawEntity  ┐
+RawImport  ├──► ParsedFile  ──► DepNode / DepEdge  ──► DepGraph
+           ┘                          │
+                                      ▼
+                               AffectedNode  ──► ImpactReport
+                                      │
+                                      ▼
+                                  OutputJSON  (written to disk)
+```
+
+---
+
+## Stage 1–2 Types (Parser Output)
+
+### `RawEntity`
+
+A code construct extracted directly from source — before any cross-file resolution.
+
+```ts
+interface RawEntity {
+  name:       string;  // e.g. "getUserById"
+  type:       string;  // "function" | "class" | "component" | "hook" | "api" | "interface" | "type"
+  line:       number;  // 1-indexed line number in the source file
+  complexity: string;  // "low" | "medium" | "high"
+}
+```
+
+**Entity types by language:**
+
+| Type | Languages | Example |
+|---|---|---|
+| `function` | JS/TS, Python | `function doSomething()` |
+| `class` | JS/TS, Python | `class UserService` |
+| `component` | JS/TS (React) | `const Button = () =>` |
+| `hook` | JS/TS (React) | `const useAuth = () =>` |
+| `interface` | TypeScript | `interface User` |
+| `type` | TypeScript | `type UserId = string` |
+| `api` | JS/TS (Express) | `app.get('/users', ...)` |
+
+---
+
+### `RawImport`
+
+An import statement extracted from a source file.
+
+```ts
+interface RawImport {
+  source:  string;    // the import path, e.g. "./userService" or "express"
+  names:   string[];  // the names imported, e.g. ["getUserById", "createUser"]
+  isLocal: boolean;   // true if source starts with "." (relative path)
+}
+```
+
+**Key distinction**: Only imports where `isLocal === true` become graph edges. External package imports (`"express"`, `"lodash"`, etc.) are ignored during graph construction.
+
+---
+
+### `ParsedFile`
+
+The complete output for one source file after parsing.
+
+```ts
+interface ParsedFile {
+  filePath: string;        // absolute or relative path on disk
+  lang:     string;        // parser's lang id, e.g. "js" or "py"
+  lines:    number;        // total line count
+  entities: RawEntity[];   // all found code constructs
+  imports:  RawImport[];   // all import statements
+  exports:  string[];      // names of exported entities
+}
+```
+
+---
+
+## Stage 3–4 Types (Graph)
+
+### `DepNode`
+
+A fully resolved node in the dependency graph. Built from a `RawEntity`, enriched with graph metrics.
+
+```ts
+interface DepNode {
+  id:              string;    // composite ID: "entityName__fileBase"
+  name:            string;    // entity name, e.g. "getUserById"
+  type:            string;    // same as RawEntity.type
+  file:            string;    // source file path
+  line:            number;    // source line number
+  lang:            string;    // language id
+  complexity:      string;    // "low" | "medium" | "high"
+  inDegree:        number;    // how many other nodes import this one
+  outDegree:       number;    // how many nodes this one imports
+  centralityScore: number;    // inDegree * 2 + outDegree
+  connections:     string[];  // IDs of directly connected nodes (both directions)
+
+  // Optional — only present for specific entity types:
+  extends?: string;  // parent class/interface name
+  method?:  string;  // HTTP method for API routes, e.g. "GET"
+  route?:   string;  // URL path for API routes, e.g. "/api/users"
+}
+```
+
+**Node ID format**: `getUserById__userService`  
+(entity name + `__` + file basename without extension, with non-alphanumeric chars replaced by `_`)
+
+---
+
+### `DepEdge`
+
+A directed link from one node to another, representing an import relationship.
+
+```ts
+interface DepEdge {
+  from:        string;  // source node ID (the importer)
+  to:          string;  // target node ID (the imported entity)
+  type:        string;  // currently always "imports"
+  description: string;  // human-readable, e.g. "UserRouter imports getUserById from userService"
+}
+```
+
+**Direction convention**: `from → to` means "`from` depends on `to`".
+
+---
+
+### `DepGraph`
+
+The complete graph.
+
+```ts
+interface DepGraph {
+  nodes: Map<string, DepNode>;  // keyed by node ID
+  edges: DepEdge[];
+}
+```
+
+The `nodes` Map is converted to an Array when writing to JSON output.
+
+---
+
+## Stage 5 Types (Impact)
+
+### `AffectedNode`
+
+A downstream node discovered during BFS traversal from the impact target.
+
+```ts
+interface AffectedNode {
+  nodeId:         string;   // graph node ID
+  name:           string;   // entity name
+  file:           string;   // source file path
+  depth:          number;   // BFS depth from target (1 = direct dependent)
+  impact:         string;   // "critical" | "high" | "medium" | "low"
+  reason:         string;   // why this node is affected
+  changeRequired: string;   // what action needs to be taken
+  breakingChange: boolean;  // true if depth <= 2
+}
+```
+
+---
+
+### `ImpactReport`
+
+The full simulation result returned by `simulateImpact`.
+
+```ts
+interface ImpactReport {
+  targetNode:        string;           // the node ID of the changed entity
+  changeDescription: string;          // description from --impact flag
+  riskScore:         number;           // 0–100
+  riskLevel:         string;           // "LOW" | "MEDIUM" | "HIGH" | "CRITICAL"
+  affectedNodes:     AffectedNode[];   // all downstream nodes
+  breakingChanges:   AffectedNode[];   // subset where breakingChange === true
+  testingPlan:       string[];         // recommended test actions
+  recommendations:   string[];         // code review / deployment guidance
+}
+```
+
+**Risk score formula**: `C×30 + H×15 + M×7 + L×2 + targetInDegree×3` (capped at 100)
+
+---
+
+## Output Type
+
+### `OutputJSON`
+
+The root schema of the generated `depgraph-output.json` file.
+
+```ts
+interface OutputJSON {
+  meta: {
+    version:    string;   // tool version, e.g. "1.0.0"
+    timestamp:  string;   // ISO 8601, e.g. "2024-01-15T09:30:00.000Z"
+    totalFiles: number;
+    totalLines: number;
+  };
+  summary: {
+    totalNodes:    number;
+    totalEdges:    number;
+    entryPoints:   string[];  // node IDs with inDegree=0 & outDegree>0
+    leafNodes:     string[];  // node IDs with outDegree=0 & inDegree>0
+    isolatedNodes: string[];  // node IDs with both degrees = 0 (dead code)
+    criticalNodes: string[];  // node IDs with centralityScore > 20
+  };
+  nodes:   DepNode[];
+  edges:   DepEdge[];
+  files:   ParsedFile[];
+  impact?: ImpactReport;  // only present when --impact was used
+}
+```

package/docs/language-registry.md

@@ -0,0 +1,205 @@
+# Language Registry
+
+> **File**: `src/languages/registry.ts`  
+> **Exports**: `LanguageParser` (interface), `registerParser`, `getLanguageParser`
+
+---
+
+## Overview
+
+The language registry is a **simple plugin system**. It solves one problem: the parser stage needs to know *how* to parse a `.py` file differently from a `.ts` file, without hardcoding language logic into the stage itself.
+
+Any file that imports `registerParser` and calls it at module load time becomes a **language plugin**. The parser stage just calls `getLanguageParser(ext)` and gets back the right handler — it doesn't know or care which language it's dealing with.
+
+---
+
+## The `LanguageParser` Interface
+
+This is the contract every language plugin must fulfil:
+
+```ts
+export interface LanguageParser {
+  /** A short unique identifier for the language, e.g. "js", "py". */
+  lang: string;
+
+  /** File extensions this parser handles, e.g. [".js", ".ts", ".tsx"]. */
+  extensions: string[];
+
+  /**
+   * Extract named code entities (functions, classes, routes, etc.)
+   * from clean source code (single-line comments already stripped).
+   */
+  extractEntities: (code: string, filePath: string) => RawEntity[];
+
+  /**
+   * Extract import statements from clean source code.
+   */
+  extractImports: (code: string) => RawImport[];
+
+  /**
+   * Extract the names of exported entities from clean source code.
+   */
+  extractExports: (code: string) => string[];
+}
+```
+
+---
+
+## How Registration Works
+
+The registry stores parsers in a private module-level array:
+
+```ts
+const parsers: LanguageParser[] = [];
+
+export function registerParser(parser: LanguageParser): void {
+  parsers.push(parser);
+}
+
+export function getLanguageParser(ext: string): LanguageParser | null {
+  return parsers.find(p => p.extensions.includes(ext)) ?? null;
+}
+```
+
+That's the entire registry — an array and two functions.
+
+### Lookup
+
+`getLanguageParser(".ts")` returns the first registered parser whose `extensions` array includes `".ts"`. If none match, it returns `null` and the file is skipped by the parser stage.
+
+---
+
+## How Plugins Self-Register
+
+Each language file (e.g. `src/languages/javascript.ts`) defines a parser object and calls `registerParser` at the **bottom of the file**, at module scope:
+
+```ts
+const JavaScriptParser: LanguageParser = {
+  lang: 'js',
+  extensions: ['.js', '.jsx', '.mjs', '.cjs', '.ts', '.tsx'],
+  extractEntities,
+  extractImports,
+  extractExports,
+};
+
+registerParser(JavaScriptParser);
+```
+
+This code runs **once**, when the module is first imported.
+
+### Triggering registration
+
+Plugins must be imported somewhere in the application for their side-effect (`registerParser(...)`) to run. In `src/main.ts`:
+
+```ts
+import './languages/javascript';
+import './languages/python';
+```
+
+These are **side-effect imports** (no named export is consumed). They exist purely to execute the registration call.
+
+> If you add a new language parser file but forget to import it in `main.ts`, it will never be registered and its extensions will never be recognised.
+
+---
+
+## Currently Registered Languages
+
+| File | `lang` | Extensions |
+|---|---|---|
+| `javascript.ts` | `js` | `.js` `.jsx` `.mjs` `.cjs` `.ts` `.tsx` |
+| `python.ts` | `py` | `.py` |
+
+The following language files exist as stubs (imported but with minimal or no implementation yet):
+
+- `java.ts`
+- `csharp.ts`
+- `go.ts`
+- `kotlin.ts`
+- `php.ts`
+- `ruby.ts`
+- `swift.ts`
+
+---
+
+## How to Add a New Language
+
+Follow these steps to add full support for a new language.
+
+### 1. Create the parser file
+
+Create `src/languages/mylang.ts`:
+
+```ts
+import { RawEntity, RawImport } from '../types';
+import { LanguageParser, registerParser } from './registry';
+
+function extractEntities(code: string, filePath: string): RawEntity[] {
+  const entities: RawEntity[] = [];
+  // Use regex or any other technique to find functions, classes, etc.
+  // Populate and return the entities array.
+  return entities;
+}
+
+function extractImports(code: string): RawImport[] {
+  const imports: RawImport[] = [];
+  // Parse import/require/use statements.
+  // Set isLocal: true for relative imports ('./something').
+  return imports;
+}
+
+function extractExports(code: string): string[] {
+  // Return an array of exported entity names.
+  return [];
+}
+
+const MyLangParser: LanguageParser = {
+  lang: 'mylang',
+  extensions: ['.ml', '.mls'],
+  extractEntities,
+  extractImports,
+  extractExports,
+};
+
+registerParser(MyLangParser);
+```
+
+### 2. Import it in `main.ts`
+
+```ts
+import './languages/mylang';
+```
+
+Add this **before** the pipeline starts (with the other language imports at the top of the file).
+
+### 3. Ensure the extension is in `SUPPORTED_EXTS`
+
+In `src/constants.ts`, add your extension to the set:
+
+```ts
+export const SUPPORTED_EXTS = new Set([
+  // ... existing extensions
+  '.ml', '.mls',
+]);
+```
+
+Without this, the collector will never hand files with that extension to the parser in the first place.
+
+### 4. Test it
+
+```bash
+node depgraph.js ./path/to/mylang-project --verbose
+```
+
+Look for your files appearing in the verbose output and entities showing up in the summary.
+
+---
+
+## Rules for Writing a Good Parser
+
+| Rule | Why |
+|---|---|
+| Only use `code` (the cleaned version) — never read files | The stage already strips comments and passes clean code |
+| Reset `regex.lastIndex = 0` before each `exec` loop | Global regexes maintain state; forgetting this causes skipped matches |
+| Guard against duplicates (`entities.some(e => e.name === name)`) | The graph stage skips duplicates, but clean data is better upstream |
+| Mark relative imports with `isLocal: true` | Only local imports become graph edges; external packages are ignored |
+| Keep extraction logic regex-based or simple string parsing | The project has zero AST dependencies on purpose — keep it that way |

package/docs/stage-collector.md

@@ -0,0 +1,109 @@
+# Stage 1 — Collector
+
+> **File**: `src/stages/collector.ts`  
+> **Exports**: `collectFiles(dir: string): string[]`
+
+---
+
+## What It Does
+
+The collector is the very first thing that runs. Its job is simple: **walk a directory tree and return a flat list of file paths** that the rest of the pipeline should process.
+
+It does **not** read file contents — it only looks at names, extensions, and file sizes.
+
+---
+
+## How It Works
+
+### 1. Recursive directory walk
+
+`collectFiles` calls an internal `walk()` helper which:
+
+1. Reads the entries of the current directory with `fs.readdirSync`.
+2. For each entry:
+   - If it's a **directory**, check it against `IGNORE_DIRS`. If not ignored, recurse into it.
+   - If it's a **file**, check its extension against `SUPPORTED_EXTS` and its size against `MAX_FILE_SIZE`.
+
+```ts
+if (stat.isDirectory()) {
+  if (!IGNORE_DIRS.has(entry)) walk(fullPath);
+  continue;
+}
+
+const ext = path.extname(entry);
+if (SUPPORTED_EXTS.has(ext) && stat.size < MAX_FILE_SIZE) {
+  results.push(fullPath);
+}
+```
+
+### 2. Ignored directories
+
+Defined in `src/constants.ts`:
+
+```ts
+export const IGNORE_DIRS = new Set([
+  'node_modules', '.git', 'dist', 'build',
+  '.next', '__pycache__', 'vendor', 'venv',
+  'target', 'out', 'coverage', '.cache'
+]);
+```
+
+These are skipped entirely — the walker never descends into them.
+
+### 3. Supported extensions
+
+Also from constants:
+
+```ts
+export const SUPPORTED_EXTS = new Set([
+  '.ts', '.tsx', '.js', '.jsx', '.mjs', '.cjs',
+  '.py', '.go', '.java', '.cs', '.rb',
+  '.php', '.swift', '.kt', '.vue', '.svelte'
+]);
+```
+
+> **Note**: Collecting a file with `.java` doesn't mean it will be *parsed* successfully. The collector just gathers eligible paths. If no language parser is registered for that extension, the parser stage silently drops it.
+
+### 4. Size guard
+
+```ts
+export const MAX_FILE_SIZE = 300_000; // 300 KB
+```
+
+Files over 300 KB are skipped. This prevents the scanner from choking on minified bundles, lock files accidentally using a supported extension, or generated code.
+
+---
+
+## Error Handling
+
+The collector is fault-tolerant at the entry level:
+- If a directory can't be read (permissions, broken symlink), it logs a warning and continues.
+- If `fs.statSync` fails on an entry, it skips that entry and continues.
+
+It **never throws** — it always returns whatever it managed to collect.
+
+---
+
+## Output
+
+A flat `string[]` of absolute file paths, for example:
+
+```
+[
+  "/project/src/main.ts",
+  "/project/src/utils/auth.ts",
+  "/project/src/routes/users.ts"
+]
+```
+
+---
+
+## Configuration Quick Reference
+
+| Constant | Default | Description |
+|---|---|---|
+| `IGNORE_DIRS` | See above | Directory names to skip entirely |
+| `SUPPORTED_EXTS` | See above | File extensions that are eligible |
+| `MAX_FILE_SIZE` | `300_000` (300 KB) | Maximum file size in bytes |
+
+All constants live in [`src/constants.ts`](../src/constants.ts).