depgraph-core 1.0.2 → 1.0.3

package/docs/data-types.md

@@ -0,0 +1,218 @@
+# Data Types Reference
+
+> **File**: `src/types.ts`  
+> All shared TypeScript interfaces used across every stage of the pipeline.
+
+---
+
+## Data Flow Summary
+
+```
+RawEntity  ┐
+RawImport  ├──► ParsedFile  ──► DepNode / DepEdge  ──► DepGraph
+           ┘                          │
+                                      ▼
+                               AffectedNode  ──► ImpactReport
+                                      │
+                                      ▼
+                                  OutputJSON  (written to disk)
+```
+
+---
+
+## Stage 1–2 Types (Parser Output)
+
+### `RawEntity`
+
+A code construct extracted directly from source — before any cross-file resolution.
+
+```ts
+interface RawEntity {
+  name:       string;  // e.g. "getUserById"
+  type:       string;  // "function" | "class" | "component" | "hook" | "api" | "interface" | "type"
+  line:       number;  // 1-indexed line number in the source file
+  complexity: string;  // "low" | "medium" | "high"
+}
+```
+
+**Entity types by language:**
+
+| Type | Languages | Example |
+|---|---|---|
+| `function` | JS/TS, Python | `function doSomething()` |
+| `class` | JS/TS, Python | `class UserService` |
+| `component` | JS/TS (React) | `const Button = () =>` |
+| `hook` | JS/TS (React) | `const useAuth = () =>` |
+| `interface` | TypeScript | `interface User` |
+| `type` | TypeScript | `type UserId = string` |
+| `api` | JS/TS (Express) | `app.get('/users', ...)` |
+
+---
+
+### `RawImport`
+
+An import statement extracted from a source file.
+
+```ts
+interface RawImport {
+  source:  string;    // the import path, e.g. "./userService" or "express"
+  names:   string[];  // the names imported, e.g. ["getUserById", "createUser"]
+  isLocal: boolean;   // true if source starts with "." (relative path)
+}
+```
+
+**Key distinction**: Only imports where `isLocal === true` become graph edges. External package imports (`"express"`, `"lodash"`, etc.) are ignored during graph construction.
+
+---
+
+### `ParsedFile`
+
+The complete output for one source file after parsing.
+
+```ts
+interface ParsedFile {
+  filePath: string;        // absolute or relative path on disk
+  lang:     string;        // parser's lang id, e.g. "js" or "py"
+  lines:    number;        // total line count
+  entities: RawEntity[];   // all found code constructs
+  imports:  RawImport[];   // all import statements
+  exports:  string[];      // names of exported entities
+}
+```
+
+---
+
+## Stage 3–4 Types (Graph)
+
+### `DepNode`
+
+A fully resolved node in the dependency graph. Built from a `RawEntity`, enriched with graph metrics.
+
+```ts
+interface DepNode {
+  id:              string;    // composite ID: "entityName__fileBase"
+  name:            string;    // entity name, e.g. "getUserById"
+  type:            string;    // same as RawEntity.type
+  file:            string;    // source file path
+  line:            number;    // source line number
+  lang:            string;    // language id
+  complexity:      string;    // "low" | "medium" | "high"
+  inDegree:        number;    // how many other nodes import this one
+  outDegree:       number;    // how many nodes this one imports
+  centralityScore: number;    // inDegree * 2 + outDegree
+  connections:     string[];  // IDs of directly connected nodes (both directions)
+
+  // Optional — only present for specific entity types:
+  extends?: string;  // parent class/interface name
+  method?:  string;  // HTTP method for API routes, e.g. "GET"
+  route?:   string;  // URL path for API routes, e.g. "/api/users"
+}
+```
+
+**Node ID format**: `getUserById__userService`  
+(entity name + `__` + file basename without extension, with non-alphanumeric chars replaced by `_`)
+
+---
+
+### `DepEdge`
+
+A directed link from one node to another, representing an import relationship.
+
+```ts
+interface DepEdge {
+  from:        string;  // source node ID (the importer)
+  to:          string;  // target node ID (the imported entity)
+  type:        string;  // currently always "imports"
+  description: string;  // human-readable, e.g. "UserRouter imports getUserById from userService"
+}
+```
+
+**Direction convention**: `from → to` means "`from` depends on `to`".
+
+---
+
+### `DepGraph`
+
+The complete graph.
+
+```ts
+interface DepGraph {
+  nodes: Map<string, DepNode>;  // keyed by node ID
+  edges: DepEdge[];
+}
+```
+
+The `nodes` Map is converted to an Array when writing to JSON output.
+
+---
+
+## Stage 5 Types (Impact)
+
+### `AffectedNode`
+
+A downstream node discovered during BFS traversal from the impact target.
+
+```ts
+interface AffectedNode {
+  nodeId:         string;   // graph node ID
+  name:           string;   // entity name
+  file:           string;   // source file path
+  depth:          number;   // BFS depth from target (1 = direct dependent)
+  impact:         string;   // "critical" | "high" | "medium" | "low"
+  reason:         string;   // why this node is affected
+  changeRequired: string;   // what action needs to be taken
+  breakingChange: boolean;  // true if depth <= 2
+}
+```
+
+---
+
+### `ImpactReport`
+
+The full simulation result returned by `simulateImpact`.
+
+```ts
+interface ImpactReport {
+  targetNode:        string;           // the node ID of the changed entity
+  changeDescription: string;          // description from --impact flag
+  riskScore:         number;           // 0–100
+  riskLevel:         string;           // "LOW" | "MEDIUM" | "HIGH" | "CRITICAL"
+  affectedNodes:     AffectedNode[];   // all downstream nodes
+  breakingChanges:   AffectedNode[];   // subset where breakingChange === true
+  testingPlan:       string[];         // recommended test actions
+  recommendations:   string[];         // code review / deployment guidance
+}
+```
+
+**Risk score formula**: `C×30 + H×15 + M×7 + L×2 + targetInDegree×3` (capped at 100)
+
+---
+
+## Output Type
+
+### `OutputJSON`
+
+The root schema of the generated `depgraph-output.json` file.
+
+```ts
+interface OutputJSON {
+  meta: {
+    version:    string;   // tool version, e.g. "1.0.0"
+    timestamp:  string;   // ISO 8601, e.g. "2024-01-15T09:30:00.000Z"
+    totalFiles: number;
+    totalLines: number;
+  };
+  summary: {
+    totalNodes:    number;
+    totalEdges:    number;
+    entryPoints:   string[];  // node IDs with inDegree=0 & outDegree>0
+    leafNodes:     string[];  // node IDs with outDegree=0 & inDegree>0
+    isolatedNodes: string[];  // node IDs with both degrees = 0 (dead code)
+    criticalNodes: string[];  // node IDs with centralityScore > 20
+  };
+  nodes:   DepNode[];
+  edges:   DepEdge[];
+  files:   ParsedFile[];
+  impact?: ImpactReport;  // only present when --impact was used
+}
+```

package/docs/language-registry.md

@@ -0,0 +1,205 @@
+# Language Registry
+
+> **File**: `src/languages/registry.ts`  
+> **Exports**: `LanguageParser` (interface), `registerParser`, `getLanguageParser`
+
+---
+
+## Overview
+
+The language registry is a **simple plugin system**. It solves one problem: the parser stage needs to know *how* to parse a `.py` file differently from a `.ts` file, without hardcoding language logic into the stage itself.
+
+Any file that imports `registerParser` and calls it at module load time becomes a **language plugin**. The parser stage just calls `getLanguageParser(ext)` and gets back the right handler — it doesn't know or care which language it's dealing with.
+
+---
+
+## The `LanguageParser` Interface
+
+This is the contract every language plugin must fulfil:
+
+```ts
+export interface LanguageParser {
+  /** A short unique identifier for the language, e.g. "js", "py". */
+  lang: string;
+
+  /** File extensions this parser handles, e.g. [".js", ".ts", ".tsx"]. */
+  extensions: string[];
+
+  /**
+   * Extract named code entities (functions, classes, routes, etc.)
+   * from clean source code (single-line comments already stripped).
+   */
+  extractEntities: (code: string, filePath: string) => RawEntity[];
+
+  /**
+   * Extract import statements from clean source code.
+   */
+  extractImports: (code: string) => RawImport[];
+
+  /**
+   * Extract the names of exported entities from clean source code.
+   */
+  extractExports: (code: string) => string[];
+}
+```
+
+---
+
+## How Registration Works
+
+The registry stores parsers in a private module-level array:
+
+```ts
+const parsers: LanguageParser[] = [];
+
+export function registerParser(parser: LanguageParser): void {
+  parsers.push(parser);
+}
+
+export function getLanguageParser(ext: string): LanguageParser | null {
+  return parsers.find(p => p.extensions.includes(ext)) ?? null;
+}
+```
+
+That's the entire registry — an array and two functions.
+
+### Lookup
+
+`getLanguageParser(".ts")` returns the first registered parser whose `extensions` array includes `".ts"`. If none match, it returns `null` and the file is skipped by the parser stage.
+
+---
+
+## How Plugins Self-Register
+
+Each language file (e.g. `src/languages/javascript.ts`) defines a parser object and calls `registerParser` at the **bottom of the file**, at module scope:
+
+```ts
+const JavaScriptParser: LanguageParser = {
+  lang: 'js',
+  extensions: ['.js', '.jsx', '.mjs', '.cjs', '.ts', '.tsx'],
+  extractEntities,
+  extractImports,
+  extractExports,
+};
+
+registerParser(JavaScriptParser);
+```
+
+This code runs **once**, when the module is first imported.
+
+### Triggering registration
+
+Plugins must be imported somewhere in the application for their side-effect (`registerParser(...)`) to run. In `src/main.ts`:
+
+```ts
+import './languages/javascript';
+import './languages/python';
+```
+
+These are **side-effect imports** (no named export is consumed). They exist purely to execute the registration call.
+
+> If you add a new language parser file but forget to import it in `main.ts`, it will never be registered and its extensions will never be recognised.
+
+---
+
+## Currently Registered Languages
+
+| File | `lang` | Extensions |
+|---|---|---|
+| `javascript.ts` | `js` | `.js` `.jsx` `.mjs` `.cjs` `.ts` `.tsx` |
+| `python.ts` | `py` | `.py` |
+
+The following language files exist as stubs (imported but with minimal or no implementation yet):
+
+- `java.ts`
+- `csharp.ts`
+- `go.ts`
+- `kotlin.ts`
+- `php.ts`
+- `ruby.ts`
+- `swift.ts`
+
+---
+
+## How to Add a New Language
+
+Follow these steps to add full support for a new language.
+
+### 1. Create the parser file
+
+Create `src/languages/mylang.ts`:
+
+```ts
+import { RawEntity, RawImport } from '../types';
+import { LanguageParser, registerParser } from './registry';
+
+function extractEntities(code: string, filePath: string): RawEntity[] {
+  const entities: RawEntity[] = [];
+  // Use regex or any other technique to find functions, classes, etc.
+  // Populate and return the entities array.
+  return entities;
+}
+
+function extractImports(code: string): RawImport[] {
+  const imports: RawImport[] = [];
+  // Parse import/require/use statements.
+  // Set isLocal: true for relative imports ('./something').
+  return imports;
+}
+
+function extractExports(code: string): string[] {
+  // Return an array of exported entity names.
+  return [];
+}
+
+const MyLangParser: LanguageParser = {
+  lang: 'mylang',
+  extensions: ['.ml', '.mls'],
+  extractEntities,
+  extractImports,
+  extractExports,
+};
+
+registerParser(MyLangParser);
+```
+
+### 2. Import it in `main.ts`
+
+```ts
+import './languages/mylang';
+```
+
+Add this **before** the pipeline starts (with the other language imports at the top of the file).
+
+### 3. Ensure the extension is in `SUPPORTED_EXTS`
+
+In `src/constants.ts`, add your extension to the set:
+
+```ts
+export const SUPPORTED_EXTS = new Set([
+  // ... existing extensions
+  '.ml', '.mls',
+]);
+```
+
+Without this, the collector will never hand files with that extension to the parser in the first place.
+
+### 4. Test it
+
+```bash
+node depgraph.js ./path/to/mylang-project --verbose
+```
+
+Look for your files appearing in the verbose output and entities showing up in the summary.
+
+---
+
+## Rules for Writing a Good Parser
+
+| Rule | Why |
+|---|---|
+| Only use `code` (the cleaned version) — never read files | The stage already strips comments and passes clean code |
+| Reset `regex.lastIndex = 0` before each `exec` loop | Global regexes maintain state; forgetting this causes skipped matches |
+| Guard against duplicates (`entities.some(e => e.name === name)`) | The graph stage skips duplicates, but clean data is better upstream |
+| Mark relative imports with `isLocal: true` | Only local imports become graph edges; external packages are ignored |
+| Keep extraction logic regex-based or simple string parsing | The project has zero AST dependencies on purpose — keep it that way |

package/docs/stage-collector.md

@@ -0,0 +1,109 @@
+# Stage 1 — Collector
+
+> **File**: `src/stages/collector.ts`  
+> **Exports**: `collectFiles(dir: string): string[]`
+
+---
+
+## What It Does
+
+The collector is the very first thing that runs. Its job is simple: **walk a directory tree and return a flat list of file paths** that the rest of the pipeline should process.
+
+It does **not** read file contents — it only looks at names, extensions, and file sizes.
+
+---
+
+## How It Works
+
+### 1. Recursive directory walk
+
+`collectFiles` calls an internal `walk()` helper which:
+
+1. Reads the entries of the current directory with `fs.readdirSync`.
+2. For each entry:
+   - If it's a **directory**, check it against `IGNORE_DIRS`. If not ignored, recurse into it.
+   - If it's a **file**, check its extension against `SUPPORTED_EXTS` and its size against `MAX_FILE_SIZE`.
+
+```ts
+if (stat.isDirectory()) {
+  if (!IGNORE_DIRS.has(entry)) walk(fullPath);
+  continue;
+}
+
+const ext = path.extname(entry);
+if (SUPPORTED_EXTS.has(ext) && stat.size < MAX_FILE_SIZE) {
+  results.push(fullPath);
+}
+```
+
+### 2. Ignored directories
+
+Defined in `src/constants.ts`:
+
+```ts
+export const IGNORE_DIRS = new Set([
+  'node_modules', '.git', 'dist', 'build',
+  '.next', '__pycache__', 'vendor', 'venv',
+  'target', 'out', 'coverage', '.cache'
+]);
+```
+
+These are skipped entirely — the walker never descends into them.
+
+### 3. Supported extensions
+
+Also from constants:
+
+```ts
+export const SUPPORTED_EXTS = new Set([
+  '.ts', '.tsx', '.js', '.jsx', '.mjs', '.cjs',
+  '.py', '.go', '.java', '.cs', '.rb',
+  '.php', '.swift', '.kt', '.vue', '.svelte'
+]);
+```
+
+> **Note**: Collecting a file with `.java` doesn't mean it will be *parsed* successfully. The collector just gathers eligible paths. If no language parser is registered for that extension, the parser stage silently drops it.
+
+### 4. Size guard
+
+```ts
+export const MAX_FILE_SIZE = 300_000; // 300 KB
+```
+
+Files over 300 KB are skipped. This prevents the scanner from choking on minified bundles, lock files accidentally using a supported extension, or generated code.
+
+---
+
+## Error Handling
+
+The collector is fault-tolerant at the entry level:
+- If a directory can't be read (permissions, broken symlink), it logs a warning and continues.
+- If `fs.statSync` fails on an entry, it skips that entry and continues.
+
+It **never throws** — it always returns whatever it managed to collect.
+
+---
+
+## Output
+
+A flat `string[]` of absolute file paths, for example:
+
+```
+[
+  "/project/src/main.ts",
+  "/project/src/utils/auth.ts",
+  "/project/src/routes/users.ts"
+]
+```
+
+---
+
+## Configuration Quick Reference
+
+| Constant | Default | Description |
+|---|---|---|
+| `IGNORE_DIRS` | See above | Directory names to skip entirely |
+| `SUPPORTED_EXTS` | See above | File extensions that are eligible |
+| `MAX_FILE_SIZE` | `300_000` (300 KB) | Maximum file size in bytes |
+
+All constants live in [`src/constants.ts`](../src/constants.ts).

package/docs/stage-graph.md

@@ -0,0 +1,157 @@
+# Stage 3 — Graph Builder
+
+> **File**: `src/stages/graph.ts`  
+> **Exports**: `buildGraph(parsedFiles: ParsedFile[]): DepGraph`
+
+---
+
+## What It Does
+
+The graph builder takes the flat list of `ParsedFile` objects and turns it into a **directed dependency graph**: a collection of **nodes** (entities) connected by **edges** (import relationships).
+
+After this stage, you know exactly which entity depends on which other entity, across the entire codebase.
+
+---
+
+## The Three-Step Build Process
+
+### Step 1 — Create a node for every entity
+
+Every `RawEntity` found in every file becomes a `DepNode` in the graph:
+
+```ts
+for (const file of parsedFiles) {
+  const fileBase = path.basename(file.filePath, path.extname(file.filePath));
+
+  for (const entity of file.entities) {
+    const id = makeId(entity.name, fileBase); // e.g. "getUserById__userService"
+
+    if (nodes.has(id)) continue; // skip duplicates
+
+    nodes.set(id, {
+      id,
+      name: entity.name,
+      type: entity.type,
+      file: file.filePath,
+      line: entity.line,
+      lang: file.lang,
+      complexity: entity.complexity,
+      inDegree: 0,
+      outDegree: 0,
+      centralityScore: 0,
+      connections: [],
+    });
+  }
+}
+```
+
+**Node IDs** are composite strings: `entityName__fileBasename`.  
+For example, the function `getUserById` in `src/services/userService.ts` gets ID `getUserById__userService`.
+
+This scheme prevents collisions when two files have a function with the same name.
+
+### Step 2 — Build a file path lookup map
+
+```ts
+const fileMap = new Map<string, ParsedFile>();
+for (const file of parsedFiles) {
+  fileMap.set(file.filePath, file);
+}
+```
+
+This allows O(1) lookup from a resolved file path → its `ParsedFile` data.
+
+### Step 3 — Connect nodes through imports
+
+This is where the real graph-building happens. For every file:
+
+1. Iterate over its `imports`.
+2. Skip **external** imports (packages like `express`, `lodash`) — only local imports matter.
+3. **Resolve** the import path to an actual file on disk (see path resolution below).
+4. For each imported name, find the matching entity in the target file.
+5. Create an edge from **every entity in the importing file** to the **matching target entity**.
+
+```ts
+edges.push({
+  from: fromId,
+  to: toId,
+  type: 'imports',
+  description: `${fromEntity.name} imports ${importedName} from ${path.basename(resolvedPath)}`,
+});
+```
+
+Duplicate edges are guarded against before insertion.
+
+---
+
+## Path Resolution
+
+Import paths like `"./userService"` don't include an extension. The `resolvePath` helper tries these candidates in priority order:
+
+```
+./userService
+./userService.ts
+./userService.tsx
+./userService.js
+./userService.jsx
+./userService/index.ts
+./userService/index.js
+```
+
+It finds the first candidate that matches a known file in `parsedFiles` and returns that path. If nothing matches, the import is silently skipped.
+
+---
+
+## Output Shape — `DepGraph`
+
+```ts
+interface DepGraph {
+  nodes: Map<string, DepNode>;
+  edges: DepEdge[];
+}
+```
+
+At this point, all `inDegree`, `outDegree`, and `centralityScore` fields on each node are still `0`. The **Metrics stage** fills these in (see [stage-metrics.md](./stage-metrics.md)).
+
+---
+
+## Edge Direction Convention
+
+```
+FROM → TO
+ (importer)  (imported)
+```
+
+If `routes/users.ts` imports `getUserById` from `services/userService.ts`, the edge is:
+
+```
+GET /api/users__users  →  getUserById__userService
+```
+
+This forward-direction convention means: **following edges = following dependencies**.  
+For impact analysis (who depends on me?), edges are traversed **in reverse** (see [stage-impact.md](./stage-impact.md)).
+
+---
+
+## The `makeId` Helper
+
+```ts
+function makeId(name: string, fileBase: string): string {
+  const cleanName = name.replace(/[^a-zA-Z0-9]/g, '_');
+  const cleanBase = fileBase.replace(/[^a-zA-Z0-9]/g, '_');
+  return `${cleanName}__${cleanBase}`;
+}
+```
+
+Non-alphanumeric characters (e.g. spaces, slashes in route paths like `GET /api/users`) are replaced with underscores. This keeps IDs safe for use as JSON keys and stable across runs.
+
+---
+
+## Common Gotchas
+
+| Situation | Behaviour |
+|---|---|
+| Two files export the same function name | First one wins; second is skipped (duplicate `id` guard) |
+| Import resolves to a file with no entities | The import is dropped — nothing to link to |
+| Circular imports (A imports B, B imports A) | Both edges are created; no infinite loop — the graph is built in a single linear pass |
+| External package imports | Skipped via the `isLocal` flag on `RawImport` |