scopecairn 0.4.0 → 0.7.0

package/README.md

@@ -22,6 +22,15 @@ No MCP server, no embeddings, no network calls, no telemetry.
   Definition of Done, so the agent changes only what matters.
 - **Change impact** — direct, indirect (2-hop), tests, and UI components
   affected by a file or symbol.
+- **Call path tracing** — `scopecairn path <A> <B>` shows the shortest
+  multi-hop call/dependency chain between two symbols.
+- **Centrality ranking** — weighted PageRank + betweenness cached per
+  symbol; `context` surfaces critical hotspots, `doctor` flags
+  single points of failure.
+- **Community detection** — Louvain clustering finds real modules by
+  link density (not just folders); `graph` shows each symbol's community.
+- **Cycle detection** — Tarjan SCC finds circular dependencies;
+  reported by `doctor`, warned in `context`, listed in `GRAPH.md`.
 - **Auto-refresh** — `context` and `impact` re-index changed files first;
   run `scan` after editing to close the loop.
 - **Framework adapters** — Next.js (App + Pages Router routes),
@@ -32,6 +41,22 @@ No MCP server, no embeddings, no network calls, no telemetry.
   ready), Antigravity, Cursor, Windsurf, Copilot, Kiro, and an OpenCode
   slash command. ScopeCairn never touches your `AGENTS.md`/`CLAUDE.md` —
   those hold your repo's details, not tool instructions.
+- **`GRAPH.md` knowledge file** — `.scopecairn/GRAPH.md` is a budgeted
+  (±150 lines) frozen summary of the graph (modules, most-used symbols,
+  PageRank hotspots, circular dependencies, routes, models, protected).
+  Written on setup, rewritten only when the
+  graph changes — for cold-start orientation; per-task precision still
+  comes from `context`.
+- **`GRAPH.html` visual explorer** — offline, dependency-free Canvas
+  explorer written next to `GRAPH.md` on every graph change.
+  Cluster ↔ File ↔ Symbol hierarchy toggle, live search, type filter,
+  click-for-details side panel (callers, callees, centrality scores).
+  Nodes sized by symbols/PageRank, colored by Louvain community.
+  Pan, zoom, hover inspect. Double-click to open, no server.
+- **`export` for external tools** — `scopecairn export --format
+  mermaid|graphml|dot|json` writes `.scopecairn/graph.<ext>` for PR
+  diagrams (Mermaid), Gephi/Cytoscape (GraphML), Graphviz (DOT),
+  or custom scripts (JSON).
 
 ## Install
 
@@ -41,6 +66,13 @@ npm install -g scopecairn
 
 Requires Node.js ≥ 20.
 
+Upgrade later with:
+
+```bash
+scopecairn update          # or: npm install -g scopecairn@latest
+scopecairn update --check  # only check for a new version
+```
+
 ## Agent-driven setup
 
 Install once, then just type `scopecairn ./` to your AI agent —
@@ -90,6 +122,8 @@ scopecairn context ./                 # repository orientation map
 scopecairn impact src/request/RequestService.ts
 scopecairn read symbol approveRequest
 scopecairn graph RequestService
+scopecairn path handleLogin dbQuery
+scopecairn export --format mermaid
 scopecairn doctor
 ```
 
@@ -108,14 +142,20 @@ scopecairn agents list
 | Command | Description |
 |---|---|
 | `init [path]` | Initial indexing + Skill, Workflow, detected agent skills (`--agents …\|all`; never writes `AGENTS.md`) |
-| `scan` / `rebuild` | Manual index management (`scan` inkremental; `rebuild` dari nol) |
+| `scan` / `rebuild` / `clean` | Manual index management (`scan` incremental; `rebuild` from scratch; `clean` removes `.scopecairn/`) |
 | `status` / `doctor` | Index status (incl. adapters, invocations) and health checks |
-| `context "<task>"` | Main entry point: context + scope (`--escalate`, `--no-refresh`); a path task returns an orientation map |
+| `context "<task>"` | Main entry point: context + scope (`--escalate`, `--no-refresh`, `--mode NORMAL|FAST|SAFE|AUDIT`); a path task returns an orientation map |
 | `impact <path\|symbol>` | Change impact: direct, indirect, tests, UI, queries, routes |
-| `graph <symbol>` | Symbol relations |
+| `graph <symbol>` | Symbol relations (+ PageRank, community, cycle status) |
+| `path <from> <to>` | Shortest multi-hop call/dependency path between two symbols |
+| `export --format <fmt>` | Graph export: `mermaid`, `graphml`, `dot`, or `json` (default `.scopecairn/graph.<ext>`, override with `--out`) |
 | `read symbol <name>` | Symbol-level source excerpt (not whole files) |
 | `<agent> install` | Per-agent setup: `antigravity`, `claude`, `gemini`, `cursor`, `windsurf`, `copilot`, `kiro` (each also `uninstall`; `agents list` to see all) |
+| `dashboard` | Generate local HTML dashboard at `.scopecairn/dashboard.html` |
+| `test-select <target>` | List test files affected by a file/symbol change target |
+| `doctor --verbose` | Also surface adapter/graph errors from `.scopecairn/last-run.log` to stderr |
 | `benchmark [--tune]` | Retrieval recall, irrelevant ratio, context reduction + weight calibration |
+| `update [--check]` | Update ScopeCairn to the latest npm release (`--check` only checks) |
 
 Agent commands (`context`, `impact`, `read`, `graph`, `status`, `doctor`)
 are read-only toward source and only write to `.scopecairn/` — safe to
@@ -131,11 +171,17 @@ No configuration needed.
 | `nextjs` | `app/`, `pages/`, `next.config.*` | Route symbols (`GET /api/x`, `PAGE /y` incl. route groups, dynamic segments), `ROUTES_TO` handler edges, `QUERIES` handler→model edges |
 | `prisma` | `schema.prisma` | `model` symbols for `QUERIES` resolution |
 | `drizzle` | `drizzle*` paths/config, `db/` schemas | `model` symbols from `pgTable`/`sqliteTable`/`mysqlTable`; `QUERIES` from `db.query.*` relational and `db.select/insert/update/delete` builder calls |
-
-Limits (honest): Server Actions, middleware, non-Prisma/Drizzle ORMs,
-inter-table references, raw SQL strings, and fully dynamic table names
-are not mapped yet — those edges are skipped silently, never hallucinated.
-QUERIES re-derive on every scan so new models resolve without a rebuild.
+| `express` | `package.json` deps | Route symbols from `app/router.METHOD(path)`, handler edges |
+| `fastapi` | `.py` routes, requirements/pyproject | Route symbols from `@app.METHOD("/path")` decorators |
+| `sqlalchemy` | requirements/pyproject | `model` symbols from `class X(Base)`/`__tablename__`; `QUERIES` from `session.query/select` |
+| `vue` | `.vue` SFCs, `package.json` deps | `component` symbols per SFC; `IMPORTS` edges from used components in template |
+
+Limits (honest): non-Prisma/Drizzle/SQLAlchemy ORMs, inter-table
+references, raw SQL strings, and fully dynamic table names are not mapped
+yet — those edges are skipped (and logged to `.scopecairn/last-run.log`),
+never hallucinated. Server Actions (`detectServerActions`) and middleware
+(`detectMiddleware`) are detected in `nextjs.ts`. QUERIES re-derive on
+every scan so new models resolve without a rebuild.
 Contributing a new adapter = one file + one registration line
 in `src/adapters/index.ts` (see `FrameworkAdapter` in `src/adapters/types.ts`).
 

package/dist/chunk-DY2AAM62.js

@@ -0,0 +1,207 @@
+#!/usr/bin/env node
+
+// src/graph/metrics.ts
+var PAGERANK_DAMPING = 0.85;
+var MAX_ITERS = 40;
+var TOLERANCE = 1e-6;
+var BETWEENNESS_SOURCE_CAP = 400;
+function readGraph(db) {
+  let ids = [];
+  try {
+    ids = db.prepare(`SELECT id FROM symbols ORDER BY id`).all().map((r) => r.id);
+  } catch {
+    return { ids: [], edges: [] };
+  }
+  let edges = [];
+  try {
+    edges = db.prepare(
+      `SELECT source_id AS s, target_id AS t,
+                  MAX(weight, 0.01) AS w FROM relationships`
+    ).all().filter((e) => e.s !== void 0 && e.t !== void 0);
+  } catch {
+    edges = [];
+  }
+  return { ids, edges };
+}
+function computePageRank(db, damping = PAGERANK_DAMPING) {
+  const { ids, edges } = readGraph(db);
+  const pr = /* @__PURE__ */ new Map();
+  const n = ids.length;
+  if (n === 0) return pr;
+  for (const id of ids) pr.set(id, 1 / n);
+  const outW = /* @__PURE__ */ new Map();
+  const incoming = /* @__PURE__ */ new Map();
+  for (const e of edges) {
+    outW.set(e.s, (outW.get(e.s) ?? 0) + e.w);
+    if (!incoming.has(e.t)) incoming.set(e.t, []);
+    incoming.get(e.t).push({ v: e.s, w: e.w });
+  }
+  const base = (1 - damping) / n;
+  for (let it = 0; it < MAX_ITERS; it++) {
+    let dangling = 0;
+    for (const id of ids) {
+      if (!outW.has(id) || outW.get(id) === 0) dangling += pr.get(id);
+    }
+    const danglingShare = damping * dangling / n;
+    let delta = 0;
+    const next = /* @__PURE__ */ new Map();
+    for (const id of ids) {
+      let s = 0;
+      for (const inc of incoming.get(id) ?? []) {
+        const ow = outW.get(inc.v) ?? 0;
+        if (ow > 0) s += (pr.get(inc.v) ?? 0) * inc.w / ow;
+      }
+      const v = base + danglingShare + damping * s;
+      next.set(id, v);
+      delta = Math.max(delta, Math.abs(v - (pr.get(id) ?? 0)));
+    }
+    for (const [k, v] of next) pr.set(k, v);
+    if (delta < TOLERANCE) break;
+  }
+  return pr;
+}
+function computeBetweenness(db) {
+  const { ids, edges } = readGraph(db);
+  const btw = /* @__PURE__ */ new Map();
+  for (const id of ids) btw.set(id, 0);
+  const n = ids.length;
+  if (n === 0) return btw;
+  const adj = /* @__PURE__ */ new Map();
+  for (const e of edges) {
+    if (!adj.has(e.s)) adj.set(e.s, []);
+    adj.get(e.s).push(e.t);
+  }
+  const sources = n > BETWEENNESS_SOURCE_CAP ? ids.filter((_, i) => i % Math.ceil(n / BETWEENNESS_SOURCE_CAP) === 0) : ids;
+  const scale = n / sources.length;
+  for (const s of sources) {
+    const stack = [];
+    const pred = /* @__PURE__ */ new Map();
+    const sigma = /* @__PURE__ */ new Map();
+    const dist = /* @__PURE__ */ new Map();
+    for (const id of ids) {
+      pred.set(id, []);
+      sigma.set(id, 0);
+      dist.set(id, -1);
+    }
+    sigma.set(s, 1);
+    dist.set(s, 0);
+    const queue = [s];
+    while (queue.length > 0) {
+      const v = queue.shift();
+      stack.push(v);
+      for (const w of adj.get(v) ?? []) {
+        if ((dist.get(w) ?? -1) < 0) {
+          dist.set(w, (dist.get(v) ?? 0) + 1);
+          queue.push(w);
+        }
+        if (dist.get(w) === (dist.get(v) ?? 0) + 1) {
+          sigma.set(w, (sigma.get(w) ?? 0) + (sigma.get(v) ?? 0));
+          pred.get(w).push(v);
+        }
+      }
+    }
+    const delta = /* @__PURE__ */ new Map();
+    for (const id of ids) delta.set(id, 0);
+    while (stack.length > 0) {
+      const w = stack.pop();
+      for (const v of pred.get(w) ?? []) {
+        const c = (sigma.get(v) ?? 0) / Math.max(1, sigma.get(w) ?? 1) * (1 + (delta.get(w) ?? 0));
+        delta.set(v, (delta.get(v) ?? 0) + c);
+      }
+      if (w !== s) btw.set(w, (btw.get(w) ?? 0) + (delta.get(w) ?? 0) * scale);
+    }
+  }
+  const norm = (n - 1) * (n - 2);
+  if (norm > 0) {
+    for (const [k, v] of btw) btw.set(k, v / norm);
+  }
+  return btw;
+}
+function degrees(db) {
+  const m = /* @__PURE__ */ new Map();
+  try {
+    for (const r of db.prepare(`SELECT target_id AS t, COUNT(*) AS n FROM relationships GROUP BY target_id`).all()) {
+      m.set(r.t, { inn: r.n, out: m.get(r.t)?.out ?? 0 });
+    }
+    for (const r of db.prepare(`SELECT source_id AS s, COUNT(*) AS n FROM relationships GROUP BY source_id`).all()) {
+      m.set(r.s, { inn: m.get(r.s)?.inn ?? 0, out: r.n });
+    }
+  } catch {
+  }
+  return m;
+}
+function refreshMetrics(db) {
+  const { ids } = readGraph(db);
+  if (ids.length === 0) return 0;
+  const pr = computePageRank(db);
+  const btw = computeBetweenness(db);
+  const deg = degrees(db);
+  try {
+    db.prepare(`DELETE FROM node_metrics WHERE node_id NOT IN (SELECT id FROM symbols)`).run();
+  } catch {
+  }
+  const up = db.prepare(
+    `INSERT INTO node_metrics(node_id, pagerank, betweenness, in_degree, out_degree, updated_at)
+     VALUES (?, ?, ?, ?, ?, datetime('now'))
+     ON CONFLICT(node_id) DO UPDATE SET
+       pagerank = excluded.pagerank, betweenness = excluded.betweenness,
+       in_degree = excluded.in_degree, out_degree = excluded.out_degree,
+       updated_at = excluded.updated_at`
+  );
+  let n = 0;
+  for (const id of ids) {
+    up.run(id, pr.get(id) ?? 0, btw.get(id) ?? 0, deg.get(id)?.inn ?? 0, deg.get(id)?.out ?? 0);
+    n++;
+  }
+  return n;
+}
+function ensureMetrics(db) {
+  try {
+    const syms = db.prepare(`SELECT COUNT(*) AS n FROM symbols`).get().n;
+    if (syms === 0) return 0;
+    const missing = db.prepare(
+      `SELECT COUNT(*) AS n FROM symbols s LEFT JOIN node_metrics m ON m.node_id = s.id WHERE m.node_id IS NULL`
+    ).get().n;
+    const orphans = db.prepare(
+      `SELECT COUNT(*) AS n FROM node_metrics m LEFT JOIN symbols s ON s.id = m.node_id WHERE s.id IS NULL`
+    ).get().n;
+    if (missing === 0 && orphans === 0) {
+      return db.prepare(`SELECT COUNT(*) AS n FROM node_metrics`).get().n;
+    }
+    return refreshMetrics(db);
+  } catch {
+    return 0;
+  }
+}
+function getMetrics(db, nodeId) {
+  try {
+    const r = db.prepare(`SELECT * FROM node_metrics WHERE node_id = ?`).get(nodeId);
+    if (!r) return null;
+    return { nodeId: r.node_id, pagerank: r.pagerank, betweenness: r.betweenness, inDegree: r.in_degree, outDegree: r.out_degree };
+  } catch {
+    return null;
+  }
+}
+function topByPageRank(db, limit = 10) {
+  ensureMetrics(db);
+  try {
+    return db.prepare(
+      `SELECT s.id, s.name, s.type, f.path AS file, m.pagerank, m.in_degree AS inDegree, m.out_degree AS outDegree
+       FROM node_metrics m JOIN symbols s ON s.id = m.node_id
+       JOIN files f ON f.id = s.file_id
+       ORDER BY m.pagerank DESC LIMIT ${Math.max(1, Math.min(50, limit))}`
+    ).all();
+  } catch {
+    return [];
+  }
+}
+
+export {
+  PAGERANK_DAMPING,
+  computePageRank,
+  computeBetweenness,
+  refreshMetrics,
+  ensureMetrics,
+  getMetrics,
+  topByPageRank
+};

package/dist/chunk-UU5WHX3D.js

@@ -0,0 +1,155 @@
+#!/usr/bin/env node
+
+// src/graph/cycles.ts
+import crypto from "crypto";
+function readAdj(db) {
+  let ids = [];
+  try {
+    ids = db.prepare(`SELECT id FROM symbols ORDER BY id`).all().map((r) => r.id);
+  } catch {
+    return { ids: [], adj: /* @__PURE__ */ new Map() };
+  }
+  const adj = /* @__PURE__ */ new Map();
+  for (const id of ids) adj.set(id, []);
+  try {
+    const rows = db.prepare(`SELECT source_id AS s, target_id AS t FROM relationships`).all();
+    for (const r of rows) {
+      if (adj.has(r.s)) adj.get(r.s).push(r.t);
+    }
+  } catch {
+  }
+  return { ids, adj };
+}
+function stronglyConnected(db) {
+  const { ids, adj } = readAdj(db);
+  const index = /* @__PURE__ */ new Map();
+  const low = /* @__PURE__ */ new Map();
+  const onStack = /* @__PURE__ */ new Set();
+  const stack = [];
+  const out = [];
+  let counter = 0;
+  for (const root of ids) {
+    if (index.has(root)) continue;
+    const work = [{ v: root, i: 0 }];
+    while (work.length > 0) {
+      const top = work[work.length - 1];
+      const v = top.v;
+      if (!index.has(v)) {
+        index.set(v, counter);
+        low.set(v, counter);
+        counter++;
+        stack.push(v);
+        onStack.add(v);
+      }
+      const nbrs = adj.get(v) ?? [];
+      if (top.i < nbrs.length) {
+        const w = nbrs[top.i++];
+        if (!index.has(w)) {
+          work.push({ v: w, i: 0 });
+        } else if (onStack.has(w)) {
+          low.set(v, Math.min(low.get(v), index.get(w)));
+        }
+      } else {
+        work.pop();
+        if (work.length > 0) {
+          const parent = work[work.length - 1].v;
+          low.set(parent, Math.min(low.get(parent), low.get(v)));
+        }
+        if (low.get(v) === index.get(v)) {
+          const comp = [];
+          let w;
+          do {
+            w = stack.pop();
+            onStack.delete(w);
+            comp.push(w);
+          } while (w !== v);
+          out.push(comp);
+        }
+      }
+    }
+  }
+  return out;
+}
+function hashIds(ids) {
+  return crypto.createHash("sha1").update([...ids].sort((a, b) => a - b).join(",")).digest("hex");
+}
+function refreshCycles(db) {
+  const { adj } = readAdj(db);
+  const comps = stronglyConnected(db);
+  const cycles = [];
+  for (const c of comps) {
+    if (c.length > 1) {
+      cycles.push(c);
+    } else if (c.length === 1 && (adj.get(c[0]) ?? []).includes(c[0])) {
+      cycles.push(c);
+    }
+  }
+  try {
+    db.prepare(`DELETE FROM cycles`).run();
+  } catch {
+  }
+  const up = db.prepare(
+    `INSERT INTO cycles(cycle_hash, length, nodes_json, created_at)
+     VALUES (?, ?, ?, datetime('now'))
+     ON CONFLICT(cycle_hash) DO UPDATE SET
+       length = excluded.length, nodes_json = excluded.nodes_json`
+  );
+  const out = [];
+  for (const c of cycles) {
+    const sorted = [...c].sort((a, b) => a - b);
+    const h = hashIds(sorted);
+    up.run(h, sorted.length, JSON.stringify(sorted));
+    const row = db.prepare(`SELECT id FROM cycles WHERE cycle_hash = ?`).get(h);
+    out.push({ id: row.id, hash: h, length: sorted.length, nodeIds: sorted });
+  }
+  return out;
+}
+function getCycles(db, limit = 20) {
+  try {
+    const rows = db.prepare(
+      `SELECT id, cycle_hash AS hash, length, nodes_json AS js FROM cycles ORDER BY length DESC LIMIT ${Math.max(1, Math.min(100, limit))}`
+    ).all();
+    return rows.map((r) => {
+      let nodeIds = [];
+      try {
+        nodeIds = JSON.parse(r.js);
+      } catch {
+        nodeIds = [];
+      }
+      return { id: r.id, hash: r.hash, length: r.length, nodeIds };
+    });
+  } catch {
+    return [];
+  }
+}
+function describeCycle(db, c, limit = 8) {
+  try {
+    const names = [];
+    for (const id of c.nodeIds.slice(0, limit)) {
+      const r = db.prepare(`SELECT s.name AS n, f.path AS p FROM symbols s JOIN files f ON f.id = s.file_id WHERE s.id = ?`).get(id);
+      if (r) names.push(`${r.n} (${r.p})`);
+    }
+    const more = c.nodeIds.length > limit ? ` +${c.nodeIds.length - limit} more` : "";
+    return names.join(" \u2192 ") + more;
+  } catch {
+    return c.nodeIds.join(" \u2192 ");
+  }
+}
+function nodeInCycle(db, nodeId) {
+  try {
+    for (const c of getCycles(db, 100)) {
+      if (c.nodeIds.includes(nodeId)) return true;
+    }
+    return false;
+  } catch {
+    return false;
+  }
+}
+
+export {
+  stronglyConnected,
+  refreshCycles,
+  getCycles,
+  describeCycle,
+  nodeInCycle
+};