depgraph-core 1.5.2 → 1.8.0

package/docs/parser-conventions.md

@@ -1,194 +0,0 @@
-# Language Parser Conventions
-
-> **Standardized architecture, directory conventions, and coding guidelines for DepGraph language parsers.**
-
----
-
-## Overview
-
-DepGraph relies on language plugins to extract code entities, imports, and exports from source files across various languages without heavy AST dependencies. As support for languages grows in sophistication—especially when incorporating deep graph extractors (e.g. Graphify engines) or rich syntax parsing—parsers must adhere to clear structural conventions to remain readable, maintainable, and testable.
-
----
-
-## The Two-Tier Architecture
-
-To balance simplicity for straightforward languages and scalability for feature-rich languages, DepGraph uses a two-tier organization model:
-
-| Tier | Format | When to Use | Typical File Count | Target Size |
-|---|---|---|---|---|
-| **Tier 1: Single-File Plugin** | `src/languages/<lang>.ts` | Simple languages or basic regex extractors | 1 file | < 300 lines |
-| **Tier 2: Modular Directory Module** | `src/languages/<lang>/` | Complex languages, >300 lines, or dual-purpose extractors (DepGraph + Graphify) | 4–6 modular files | < 250 lines per file |
-
----
-
-## Tier 1: Single-File Plugin Convention
-
-Used by languages like Python, Go, Java, Kotlin, PHP, Ruby, Swift, C#, and JavaScript.
-
-### Structure
-
-A single file: `src/languages/<lang>.ts`
-
-```
-src/languages/
-├── go.ts
-├── python.ts
-└── java.ts
-```
-
-### Standard Section Order
-
-Every single-file parser must organize code into well-demarcated sections in the following order:
-
-```ts
-// 1. Imports & External Types
-import { RawEntity, RawImport } from '../types';
-import { EntityPattern, LanguageParser, registerParser } from './registry';
-import { COMPLEXITY_THRESHOLDS } from '../constants';
-
-// 2. Helpers & Complexity Estimation
-function estimateComplexity(code: string, name: string): string { ... }
-
-// 3. Entity Patterns (Exported for gitdiff reuse)
-export const langEntityPatterns: EntityPattern[] = [ ... ];
-
-// 4. Entity Extractor
-function extractEntities(code: string, filePath: string): RawEntity[] { ... }
-
-// 5. Import Extractor
-function extractImports(code: string): RawImport[] { ... }
-
-// 6. Export Extractor
-function extractExports(code: string): string[] { ... }
-
-// 7. Parser Registration
-export const LangParser: LanguageParser = {
-  lang: 'lang',
-  extensions: ['.ext'],
-  extractEntities,
-  extractImports,
-  extractExports,
-  entityPatterns: langEntityPatterns,
-};
-
-registerParser(LangParser);
-```
-
----
-
-## Tier 2: Modular Directory Module Convention
-
-Used when a language parser exceeds ~300 lines or integrates secondary engines (such as standalone Graphify graph extractors). Languages using this tier include **Dart** and **Rust**.
-
-### Directory Layout
-
-```
-src/languages/<lang>/
-├── index.ts          # Public entrypoint, LanguageParser definition, side-effect registration
-├── types.ts          # Specialized TypeScript interfaces (e.g., Graphify node/edge definitions)
-├── patterns.ts       # EntityPattern definitions, keyword sets, and token constants
-├── helpers.ts        # Lexing utilities, comment cleaners, brace depth counters, complexity
-├── extractor.ts      # Core DepGraph contract: extractEntities, extractImports, extractExports
-└── graphify.ts       # Standalone deep AST/regex graph extraction engine (if applicable)
-```
-
-### File Responsibilities & Contract
-
-#### 1. `index.ts` (Entry Point & Barrel)
-- Constructs the `LanguageParser` object.
-- Invokes `registerParser(...)` at module evaluation time.
-- Re-exports public helpers and secondary functions (e.g., `cleanDartComments`, `estimateComplexity`, `extractDart`) to ensure seamless backward compatibility.
-- Transparently resolves `import './languages/dart'` without altering caller syntax.
-
-```ts
-import { LanguageParser, registerParser } from '../registry';
-import { langEntityPatterns } from './patterns';
-import { extractEntities, extractImports, extractExports } from './extractor';
-
-export * from './types';
-export * from './patterns';
-export * from './helpers';
-export * from './graphify';
-
-export const LangParser: LanguageParser = {
-  lang: 'lang',
-  extensions: ['.ext'],
-  extractEntities,
-  extractImports,
-  extractExports,
-  entityPatterns: langEntityPatterns,
-};
-
-registerParser(LangParser);
-```
-
-#### 2. `types.ts` (Domain Types)
-- Houses language-specific data contracts that do not belong in global `src/types.ts`.
-- Example: Graphify node representations, edge payloads, and extractor result contracts (`DartGraphNode`, `DartGraphEdge`, `DartGraphResult`).
-
-#### 3. `patterns.ts` (Regex & Token Sets)
-- Contains `EntityPattern[]` regex rules with named or indexed capture groups (Group 1 must always be the entity name).
-- Houses language keywords and primitive type sets (e.g., `DART_KEYWORDS`, `RUST_PRIMITIVES`).
-
-#### 4. `helpers.ts` (Lexical & Analysis Utilities)
-- String manipulations, string literal escaping (`escapeRegex`).
-- Comment cleaning functions that preserve line counts (`clean<Lang>Comments`).
-- Balanced bracket/brace scanners (`_findMatchingBrace`, `_splitBalanced`).
-- Cyclomatic complexity estimation (`estimateComplexity`).
-
-#### 5. `extractor.ts` (DepGraph Stage 2 Implementation)
-- Implements the 3 required methods of `LanguageParser`:
-  - `extractEntities(code: string, filePath: string): RawEntity[]`
-  - `extractImports(code: string): RawImport[]`
-  - `extractExports(code: string): string[]`
-- Operates strictly on pre-cleaned code (single-line comments already stripped by the stage caller).
-
-#### 6. `graphify.ts` (Deep Graph Engine)
-- Isolated from core stage pipeline execution.
-- Contains the self-contained graph extraction function (`extractDart`, `extractRust`).
-- Allows comprehensive standalone graph analysis without bloating the pipeline parsing logic.
-
----
-
-## Core Implementation Rules
-
-Every parser, whether Tier 1 or Tier 2, must respect the following rules:
-
-### 1. Zero Heavy AST Dependencies
-DepGraph is intentionally designed to be lightweight, ultrafast, and dependency-free.
-- **Do not** introduce heavy AST parsers (such as Babel, tree-sitter, or Roslyn).
-- Rely on regular expressions, state machines, and balanced delimiter traversal.
-
-### 2. Preserve Line Number Offsets During Comment Cleaning
-When stripping comments or preprocessing files:
-- Replace stripped blocks with the identical number of `\n` newline characters they spanned.
-- This guarantees that entity line numbers reported in `RawEntity.line` accurately match original file source lines.
-
-### 3. Regex State Hygiene
-Global regular expressions (`/pattern/g`) retain state in `regex.lastIndex`.
-- Reset `regex.lastIndex = 0` prior to any `exec` loop or instantiate regular expressions within the function scope.
-- Never let `lastIndex` leak across calls.
-
-### 4. Group 1 Entity Identifier Contract
-Any pattern in `entityPatterns` must capture the entity identifier name in **Group 1**:
-```ts
-{
-  regex: /^[ \t]*fn\s+([a-z_]\w*)/gm,
-  type: 'function',
-}
-```
-This enables `src/stages/gitdiff.ts` to deduce modified entity names directly from git hunk headers.
-
-### 5. Local vs. External Import Flagging
-- Mark `isLocal: true` only for relative or internal project paths (e.g., `./utils`, `../services`).
-- Mark `isLocal: false` for standard library packages or third-party package dependencies (e.g., `fmt`, `dart:async`, `std::collections`).
-- Only local imports become edges in DepGraph's dependency graph.
-
----
-
-## When to Migrate from Tier 1 to Tier 2
-
-Migrate a language parser from `src/languages/<lang>.ts` to `src/languages/<lang>/` when:
-1. The single file exceeds **300 lines of code**.
-2. The language incorporates custom type systems or deep extractors (e.g., Graphify bindings).
-3. The language requires extensive grammar assistance (e.g., custom bracket matching, token splitting, or stateful tokenization).

package/docs/stage-collector.md

@@ -1,109 +0,0 @@
-# Stage 1 — Collector
-
-> **File**: `src/stages/collector.ts`  
-> **Exports**: `collectFiles(dir: string): string[]`
-
----
-
-## What It Does
-
-The collector is the very first thing that runs. Its job is simple: **walk a directory tree and return a flat list of file paths** that the rest of the pipeline should process.
-
-It does **not** read file contents — it only looks at names, extensions, and file sizes.
-
----
-
-## How It Works
-
-### 1. Recursive directory walk
-
-`collectFiles` calls an internal `walk()` helper which:
-
-1. Reads the entries of the current directory with `fs.readdirSync`.
-2. For each entry:
-   - If it's a **directory**, check it against `IGNORE_DIRS`. If not ignored, recurse into it.
-   - If it's a **file**, check its extension against `SUPPORTED_EXTS` and its size against `MAX_FILE_SIZE`.
-
-```ts
-if (stat.isDirectory()) {
-  if (!IGNORE_DIRS.has(entry)) walk(fullPath);
-  continue;
-}
-
-const ext = path.extname(entry);
-if (SUPPORTED_EXTS.has(ext) && stat.size < MAX_FILE_SIZE) {
-  results.push(fullPath);
-}
-```
-
-### 2. Ignored directories
-
-Defined in `src/constants.ts`:
-
-```ts
-export const IGNORE_DIRS = new Set([
-  'node_modules', '.git', 'dist', 'build',
-  '.next', '__pycache__', 'vendor', 'venv',
-  'target', 'out', 'coverage', '.cache'
-]);
-```
-
-These are skipped entirely — the walker never descends into them.
-
-### 3. Supported extensions
-
-Also from constants:
-
-```ts
-export const SUPPORTED_EXTS = new Set([
-  '.ts', '.tsx', '.js', '.jsx', '.mjs', '.cjs',
-  '.py', '.go', '.java', '.cs', '.rb',
-  '.php', '.swift', '.kt', '.vue', '.svelte'
-]);
-```
-
-> **Note**: Collecting a file with `.java` doesn't mean it will be *parsed* successfully. The collector just gathers eligible paths. If no language parser is registered for that extension, the parser stage silently drops it.
-
-### 4. Size guard
-
-```ts
-export const MAX_FILE_SIZE = 300_000; // 300 KB
-```
-
-Files over 300 KB are skipped. This prevents the scanner from choking on minified bundles, lock files accidentally using a supported extension, or generated code.
-
----
-
-## Error Handling
-
-The collector is fault-tolerant at the entry level:
-- If a directory can't be read (permissions, broken symlink), it logs a warning and continues.
-- If `fs.statSync` fails on an entry, it skips that entry and continues.
-
-It **never throws** — it always returns whatever it managed to collect.
-
----
-
-## Output
-
-A flat `string[]` of absolute file paths, for example:
-
-```
-[
-  "/project/src/main.ts",
-  "/project/src/utils/auth.ts",
-  "/project/src/routes/users.ts"
-]
-```
-
----
-
-## Configuration Quick Reference
-
-| Constant | Default | Description |
-|---|---|---|
-| `IGNORE_DIRS` | See above | Directory names to skip entirely |
-| `SUPPORTED_EXTS` | See above | File extensions that are eligible |
-| `MAX_FILE_SIZE` | `300_000` (300 KB) | Maximum file size in bytes |
-
-All constants live in [`src/constants.ts`](../src/constants.ts).

package/docs/stage-graph.md

@@ -1,157 +0,0 @@
-# Stage 3 — Graph Builder
-
-> **File**: `src/stages/graph.ts`  
-> **Exports**: `buildGraph(parsedFiles: ParsedFile[]): DepGraph`
-
----
-
-## What It Does
-
-The graph builder takes the flat list of `ParsedFile` objects and turns it into a **directed dependency graph**: a collection of **nodes** (entities) connected by **edges** (import relationships).
-
-After this stage, you know exactly which entity depends on which other entity, across the entire codebase.
-
----
-
-## The Three-Step Build Process
-
-### Step 1 — Create a node for every entity
-
-Every `RawEntity` found in every file becomes a `DepNode` in the graph:
-
-```ts
-for (const file of parsedFiles) {
-  const fileBase = path.basename(file.filePath, path.extname(file.filePath));
-
-  for (const entity of file.entities) {
-    const id = makeId(entity.name, fileBase); // e.g. "getUserById__userService"
-
-    if (nodes.has(id)) continue; // skip duplicates
-
-    nodes.set(id, {
-      id,
-      name: entity.name,
-      type: entity.type,
-      file: file.filePath,
-      line: entity.line,
-      lang: file.lang,
-      complexity: entity.complexity,
-      inDegree: 0,
-      outDegree: 0,
-      centralityScore: 0,
-      connections: [],
-    });
-  }
-}
-```
-
-**Node IDs** are composite strings: `entityName__fileBasename`.  
-For example, the function `getUserById` in `src/services/userService.ts` gets ID `getUserById__userService`.
-
-This scheme prevents collisions when two files have a function with the same name.
-
-### Step 2 — Build a file path lookup map
-
-```ts
-const fileMap = new Map<string, ParsedFile>();
-for (const file of parsedFiles) {
-  fileMap.set(file.filePath, file);
-}
-```
-
-This allows O(1) lookup from a resolved file path → its `ParsedFile` data.
-
-### Step 3 — Connect nodes through imports
-
-This is where the real graph-building happens. For every file:
-
-1. Iterate over its `imports`.
-2. Skip **external** imports (packages like `express`, `lodash`) — only local imports matter.
-3. **Resolve** the import path to an actual file on disk (see path resolution below).
-4. For each imported name, find the matching entity in the target file.
-5. Create an edge from **every entity in the importing file** to the **matching target entity**.
-
-```ts
-edges.push({
-  from: fromId,
-  to: toId,
-  type: 'imports',
-  description: `${fromEntity.name} imports ${importedName} from ${path.basename(resolvedPath)}`,
-});
-```
-
-Duplicate edges are guarded against before insertion.
-
----
-
-## Path Resolution
-
-Import paths like `"./userService"` don't include an extension. The `resolvePath` helper tries these candidates in priority order:
-
-```
-./userService
-./userService.ts
-./userService.tsx
-./userService.js
-./userService.jsx
-./userService/index.ts
-./userService/index.js
-```
-
-It finds the first candidate that matches a known file in `parsedFiles` and returns that path. If nothing matches, the import is silently skipped.
-
----
-
-## Output Shape — `DepGraph`
-
-```ts
-interface DepGraph {
-  nodes: Map<string, DepNode>;
-  edges: DepEdge[];
-}
-```
-
-At this point, all `inDegree`, `outDegree`, and `centralityScore` fields on each node are still `0`. The **Metrics stage** fills these in (see [stage-metrics.md](./stage-metrics.md)).
-
----
-
-## Edge Direction Convention
-
-```
-FROM → TO
- (importer)  (imported)
-```
-
-If `routes/users.ts` imports `getUserById` from `services/userService.ts`, the edge is:
-
-```
-GET /api/users__users  →  getUserById__userService
-```
-
-This forward-direction convention means: **following edges = following dependencies**.  
-For impact analysis (who depends on me?), edges are traversed **in reverse** (see [stage-impact.md](./stage-impact.md)).
-
----
-
-## The `makeId` Helper
-
-```ts
-function makeId(name: string, fileBase: string): string {
-  const cleanName = name.replace(/[^a-zA-Z0-9]/g, '_');
-  const cleanBase = fileBase.replace(/[^a-zA-Z0-9]/g, '_');
-  return `${cleanName}__${cleanBase}`;
-}
-```
-
-Non-alphanumeric characters (e.g. spaces, slashes in route paths like `GET /api/users`) are replaced with underscores. This keeps IDs safe for use as JSON keys and stable across runs.
-
----
-
-## Common Gotchas
-
-| Situation | Behaviour |
-|---|---|
-| Two files export the same function name | First one wins; second is skipped (duplicate `id` guard) |
-| Import resolves to a file with no entities | The import is dropped — nothing to link to |
-| Circular imports (A imports B, B imports A) | Both edges are created; no infinite loop — the graph is built in a single linear pass |
-| External package imports | Skipped via the `isLocal` flag on `RawImport` |

package/docs/stage-impact.md

@@ -1,154 +0,0 @@
-# Stage 5 — Impact Simulator
-
-> **File**: `src/stages/impact.ts`  
-> **Exports**: `simulateImpact(graph, targetName, changeDescription): ImpactReport`  
-> **Triggered by**: `--impact <name> <description>` CLI flag
-
----
-
-## What It Does
-
-The impact simulator answers the question: **"If I change this entity, what else breaks?"**
-
-It performs a **reverse Breadth-First Search (BFS)** starting from the target node, walking *against* the edge direction to find every node that (directly or transitively) depends on the target. It then scores each affected node by impact severity and produces a risk report with testing recommendations.
-
----
-
-## The Four-Step Algorithm
-
-### Step 1 — Find the target node
-
-```ts
-const targetNode = [...graph.nodes.values()]
-  .find(n => n.name === targetName);
-```
-
-Searches by `name` (not `id`). If the target is not found, an empty report is returned immediately.
-
-### Step 2 — Reverse BFS
-
-A standard BFS, but instead of following edges *forward* (A→B means A depends on B), we follow edges *backward* (find all edges where `edge.to === nodeId`):
-
-```ts
-function getDirectDependents(graph, nodeId) {
-  return graph.edges
-    .filter(e => e.to === nodeId)
-    .map(e => e.from);
-}
-```
-
-The BFS seeds the queue with **direct dependents** (depth 1) and expands outward, level by level, until:
-- A node has already been visited (`visited` Set), or
-- The depth exceeds `MAX_BFS_DEPTH` (default: `10`)
-
-```
-target node
-   ▲
-   │  (direct dependents — depth 1)
-[A, B, C]
-   ▲
-   │  (dependents of A, B, C — depth 2)
-[D, E, F, G]
-   ▲  ...and so on up to depth 10
-```
-
-### Step 3 — Score each affected node
-
-Impact level is determined by **BFS depth**:
-
-| Depth | Impact Level | Breaking Change? |
-|---|---|---|
-| 1 | `critical` | Yes |
-| 2 | `high` | Yes |
-| 3–4 | `medium` | No |
-| 5–10 | `low` | No |
-
-Nodes at depth ≤ 2 are flagged as `breakingChange: true` because they directly consume the target's interface.
-
-### Step 4 — Compute the risk score
-
-```ts
-const score = C * 30 + H * 15 + M * 7 + L * 2 + inDegree * 3;
-return Math.min(100, score);
-```
-
-Where `C`, `H`, `M`, `L` are the counts of critical/high/medium/low affected nodes, and `inDegree` is the target node's own in-degree.
-
-The score is capped at 100.
-
-**Risk level thresholds:**
-
-| Score | Risk Level |
-|---|---|
-| ≥ 75 | `CRITICAL` |
-| ≥ 50 | `HIGH` |
-| ≥ 25 | `MEDIUM` |
-| < 25  | `LOW` |
-
----
-
-## Output Shape — `ImpactReport`
-
-```ts
-interface ImpactReport {
-  targetNode: string;           // the ID of the changed node
-  changeDescription: string;   // from CLI --impact flag
-  riskScore: number;            // 0–100
-  riskLevel: string;            // LOW / MEDIUM / HIGH / CRITICAL
-  affectedNodes: AffectedNode[];
-  breakingChanges: AffectedNode[]; // subset of affectedNodes where breakingChange=true
-  testingPlan: string[];
-  recommendations: string[];
-}
-```
-
-### `AffectedNode`
-
-```ts
-interface AffectedNode {
-  nodeId:        string;
-  name:          string;
-  file:          string;
-  depth:         number;     // BFS distance from target
-  impact:        string;     // critical / high / medium / low
-  reason:        string;     // human-readable why
-  changeRequired: string;    // what action is needed
-  breakingChange: boolean;
-}
-```
-
----
-
-## Recommendations Logic
-
-Recommendations are generated automatically based on the risk score:
-
-| Risk Score | Recommendations |
-|---|---|
-| ≥ 75 | Full team review, phased rollout, full regression suite |
-| ≥ 50 | Tech lead review, feature flag the change |
-| ≥ 25 | Code review, test all affected modules |
-| < 25 | Standard PR process sufficient |
-
-If any breaking changes exist, an additional note is always appended.
-
----
-
-## Usage Example
-
-```bash
-node depgraph.js ./src --impact "getUserById" "removing userId parameter"
-```
-
-This runs the full pipeline and then simulates changing `getUserById`, returning a detailed report of everything that would be affected.
-
----
-
-## Edge Cases
-
-| Situation | Behaviour |
-|---|---|
-| Target not found | Returns an empty report with `riskScore: 0` and an explanatory testing plan entry |
-| Target has no dependents | Returns an empty `affectedNodes` array and `LOW` risk |
-| Circular dependencies | The `visited` Set prevents infinite loops |
-| Extremely deep dependency chains | `MAX_BFS_DEPTH = 10` hard-caps traversal |