@kage-core/kage-graph-mcp 1.1.19 → 1.1.21

package/README.md

@@ -9,18 +9,26 @@ This package exposes two surfaces:
 
 ## Latest Release
 
-`1.1.18` publishes the end-to-end performance pass:
-
-- read-only commands reuse current graph artifacts instead of rebuilding them
-  when inputs are fresh.
-- MCP sessions keep an in-process graph cache, so repeated agent calls do not
-  keep reparsing the same graph JSON.
-- `kage refresh` reports lightweight freshness metrics and leaves deep
-  benchmark/quality work to explicit `kage metrics` and `kage benchmark` calls.
-- recall builds graph lookup maps once per query instead of scanning all graph
-  entities and edges for every memory packet.
-- `kage init` remains a packet-only bootstrap path; full graph generation stays
-  with `kage refresh` and `kage index`.
+`1.1.21` publishes the memory-code graph quality pass:
+
+- precise memory-code links now require explicit, non-generic symbol/test
+  mentions instead of broad path-only matches.
+- generated memory symbol/route/test entities no longer carry file path aliases
+  that can collapse stale graph nodes onto code file hubs.
+- the viewer's `Memory <-> Code only` relation shows actual cross-graph links,
+  while path-level memory still appears through capped `affects_code_path`
+  bridge edges.
+
+`1.1.20` published the large-repo indexing pass:
+
+- repeated `kage refresh` calls reuse unchanged code graph artifacts by source
+  stat fingerprint, with `kage refresh --full` available for intentional clean
+  rebuilds.
+- `kage code-index` prefers SCIP via `scip-typescript` plus the `scip` CLI when
+  those tools are installed, then falls back to Kage's built-in LSP-compatible
+  symbol index.
+- read-only commands and MCP sessions reuse current graph artifacts instead of
+  rebuilding them when inputs are fresh.
 
 `1.1.17` publishes content-based graph freshness:
 
@@ -116,6 +124,7 @@ kage policy --project /path/to/repo
 kage doctor --project /path/to/repo
 kage index --project /path/to/repo
 kage refresh --project /path/to/repo
+kage refresh --project /path/to/repo --full
 kage branch --project /path/to/repo
 kage code-index --project /path/to/repo
 kage code-graph --project /path/to/repo
@@ -170,26 +179,44 @@ The graph builder writes evidence-backed graph artifacts under
   and memory type entities.
 - `edges.json`: typed facts such as `contains_memory`, `affects_path`,
   `mentions_tag`, `uses_package`, `documents_command`, and `defines_command`.
-- `graph.json`: the assembled graph with branch, head, and merge-base metadata.
+- `graph.json`: compact graph metadata plus references to the split graph files.
+
+In the viewer, path-level memory such as `affects_path: src` is bridged to a
+small representative set of matching code files at render time. This keeps the
+stored graph compact while still making broad repo memories visibly connected
+to the code graph.
 
 The code graph builder writes source-derived artifacts under
-`.agent_memory/code_graph/`:
-
-- `files.json`: source, test, config, manifest, and doc files, including
-  language and parser metadata.
-- `symbols.json`: functions, classes, constants, and test cases.
-- `imports.json`: local and external import edges.
-- `calls.json`: best-effort call edges between discovered symbols.
-- `routes.json`: best-effort Node/Express/Next route facts.
-- `tests.json`: test-to-symbol/file coverage hints.
-- `packages.json`: package scripts and dependencies.
-- `graph.json`: the assembled code graph.
+`.agent_memory/code_graph/`. `graph.json` is now a compact compatibility
+artifact: it keeps repo state plus calls, routes, tests, and packages inline, and
+references the canonical structural `files.json`, `symbols.json`, and
+`imports.json` instead of duplicating them.
 
 The code graph is multi-language by design. JS/TS/JSX/TSX files use the
 TypeScript compiler API for AST-backed symbols, imports, and call hints. Python,
 Go, Rust, Java, Kotlin, Ruby, PHP, C#, C/C++, and Swift use deterministic generic
 static extractors so every repo gets a useful graph immediately.
 
+For large repos, refresh also writes a complete structural index under
+`.agent_memory/structural/`:
+
+- `files.json`: every supported source/config/doc file, including files the
+  code graph represents as metadata-only because they are too large to parse.
+- `symbols.json`: extracted functions, classes, constants, routes, and tests.
+- `edges.json`: file-to-symbol and file-to-import edges with confidence labels.
+- `manifest.json`: mtime/hash entries, cache hits/misses, ignored files, and
+  deleted-file tracking.
+- `file-cache.json`: packed per-file structural facts keyed by source content
+  hash. Refresh migrates older `.agent_memory/structural/file-cache/*.json`
+  layouts and removes stale per-file cache entries.
+- `report.md`: a compact language/concept coverage report.
+
+This follows the same shape as fast graph indexers such as Graphify: discover
+files, skip generated/vendor paths, use a per-file content cache, parallelize
+extraction on large repos, rebuild only changed files, and keep generated
+structural facts separate from learned memory. Use `.kageignore` for
+repo-specific excludes.
+
 Kage also consumes external industry indexer artifacts when present:
 
 - `.agent_memory/code_index/tree-sitter.json`
@@ -206,11 +233,13 @@ generic extraction in metrics and file parser coverage. This keeps installation
 light while allowing teams to plug in the strongest indexer available for their
 language stack.
 
-`kage code-index --project <repo>` writes `.agent_memory/code_index/lsp-symbols.json`
-in an LSP document-symbol-compatible shape using Kage's local parser. The CI,
+`kage code-index --project <repo>` now tries the best external indexer first for
+the common JS/TS case: if `scip-typescript` and the `scip` CLI are on the repo or
+shell path, it writes `.agent_memory/code_index/scip.json` from the generated
+SCIP index. If those tools are unavailable, it writes
+`.agent_memory/code_index/lsp-symbols.json` using Kage's local parser. The CI,
 PR, and sync workflows run it before refresh so the code graph has a committed
-precise-index slot while teams can still replace or augment it with SCIP/LSIF
-artifacts from their preferred toolchain.
+precise-index slot without making first-run setup depend on external binaries.
 
 The memory graph follows the same product direction as temporal context graph
 systems such as Graphiti: immutable ingestion episodes, derived entities and
@@ -258,7 +287,9 @@ Use `kage refresh --project <repo>` or the `kage_refresh` MCP tool after
 meaningful file/content changes. Refresh rebuilds indexes, code graph, memory
 graph, metrics, and stale-memory metadata. Memory is marked stale when status or
 feedback says it is stale, its TTL expires, or grounded paths disappear. Pushes
-and empty/same-tree commits do not need another refresh.
+and empty/same-tree commits do not need another refresh. Use `--full` or
+`kage_refresh` with `full: true` only when you intentionally want to bypass
+unchanged-graph reuse and rebuild the code graph from scratch.
 
 Use `kage gc --project <repo> --dry-run` to preview stale packet cleanup.
 `kage gc --project <repo>` marks stale repo packets deprecated, rebuilds

package/dist/cli.js

@@ -24,7 +24,7 @@ Usage:
   kage daemon status --project <dir> [--json]
   kage daemon doctor --project <dir> [--json]
   kage viewer --project <dir> [--port 3113]
-  kage refresh --project <dir> [--json]
+  kage refresh --project <dir> [--full] [--json]
   kage gc --project <dir> [--dry-run] [--force] [--json]
   kage pr summarize --project <dir> [--json]
   kage pr check --project <dir> [--json]
@@ -39,6 +39,7 @@ Usage:
   kage code-graph --project <dir> [--json]
   kage code-graph "<query>" --project <dir> [--json]
   kage code-index --project <dir> [--json]
+  kage structural-index --project <dir> [--json]
   kage graph --project <dir> [--json]
   kage graph --project <dir> --mermaid
   kage graph "<query>" --project <dir> [--json]
@@ -335,7 +336,7 @@ async function main() {
         return;
     }
     if (command === "refresh") {
-        const result = (0, kernel_js_1.refreshProject)(projectArg(args));
+        const result = (0, kernel_js_1.refreshProject)(projectArg(args), { full: args.includes("--full") });
         if (args.includes("--json")) {
             console.log(JSON.stringify(result, null, 2));
             if (!result.ok)
@@ -506,7 +507,7 @@ async function main() {
         return;
     }
     if (command === "code-index") {
-        const result = (0, kernel_js_1.writeLspSymbolIndex)(projectArg(args));
+        const result = (0, kernel_js_1.writeCodeIndex)(projectArg(args));
         if (args.includes("--json")) {
             console.log(JSON.stringify(result, null, 2));
             return;
@@ -516,6 +517,11 @@ async function main() {
         console.log(`Path: ${result.path}`);
         console.log(`Documents: ${result.documents}`);
         console.log(`Symbols: ${result.symbols}`);
+        if (result.warnings.length) {
+            console.log("\nWarnings:");
+            for (const warning of result.warnings)
+                console.log(`  - ${warning}`);
+        }
         if (result.errors.length) {
             console.log("\nErrors:");
             for (const error of result.errors)
@@ -524,6 +530,22 @@ async function main() {
         }
         return;
     }
+    if (command === "structural-index") {
+        const result = (0, kernel_js_1.buildStructuralIndex)(projectArg(args));
+        if (args.includes("--json")) {
+            console.log(JSON.stringify(result, null, 2));
+            return;
+        }
+        console.log(`Kage Structural Index: ${result.manifest.project_dir}`);
+        console.log(`Files: ${result.files.length}`);
+        console.log(`Symbols: ${result.symbols.length}`);
+        console.log(`Edges: ${result.edges.length}`);
+        console.log(`Metadata-only files: ${result.manifest.files.metadata_only}`);
+        console.log(`Cache: ${result.manifest.cache.hits} hits, ${result.manifest.cache.misses} misses`);
+        console.log(`Workers: ${result.manifest.worker_count}`);
+        console.log(`Path: .agent_memory/structural`);
+        return;
+    }
     if (command === "branch") {
         const result = (0, kernel_js_1.buildBranchOverlay)(projectArg(args));
         if (args.includes("--json"))
@@ -557,6 +579,13 @@ async function main() {
         console.log(`  Indexer coverage: ${result.code_graph.indexer_coverage_percent}%`);
         console.log(`  Languages: ${Object.entries(result.code_graph.languages).map(([name, count]) => `${name}=${count}`).join(", ") || "(none)"}`);
         console.log(`  Parsers: ${Object.entries(result.code_graph.parsers).map(([name, count]) => `${name}=${count}`).join(", ") || "(none)"}`);
+        console.log("\nStructural index:");
+        console.log(`  Files: ${result.structural_index.files}`);
+        console.log(`  Symbols: ${result.structural_index.symbols}`);
+        console.log(`  Edges: ${result.structural_index.edges}`);
+        console.log(`  Metadata-only files: ${result.structural_index.metadata_only_files}`);
+        console.log(`  Workers: ${result.structural_index.worker_count}`);
+        console.log(`  Cache: ${result.structural_index.cache_hits} hits, ${result.structural_index.cache_misses} misses`);
         console.log("\nMemory graph:");
         console.log(`  Approved packets: ${result.memory_graph.approved_packets}`);
         console.log(`  Pending packets: ${result.memory_graph.pending_packets}`);
@@ -599,7 +628,7 @@ async function main() {
         console.log(`Memory inbox: ${result.checks.memory_inbox.approved_packets} approved, ${result.checks.memory_inbox.pending_packets} pending, ${result.checks.memory_inbox.stale_packets} stale`);
         console.log(`Structured memory: ${result.checks.structured_memory.structured_packets}/${result.checks.structured_memory.total_packets} (${result.checks.structured_memory.coverage_percent}%)`);
         console.log(`Code graph precision: ${result.checks.code_graph.precise_files}/${result.checks.code_graph.files} precise (${result.checks.code_graph.precise_coverage_percent}%), ${result.checks.code_graph.ast_files} AST, ${result.checks.code_graph.fallback_files} fallback`);
-        console.log(`Memory-code graph edges: ${result.checks.graph_links.memory_code_edges}`);
+        console.log(`Memory-code graph edges: ${result.checks.graph_links.memory_code_edges} (${result.checks.graph_links.precise_memory_code_edges} precise, ${result.checks.graph_links.path_memory_code_edges} path)`);
         if (result.recommendations.length) {
             console.log("\nRecommendations:");
             for (const recommendation of result.recommendations)

package/dist/daemon.js

@@ -249,12 +249,15 @@ async function startViewer(projectDir, options = {}) {
     catch {
         // non-fatal: viewer will show 404 for reports if generation fails
     }
-    const url = `http://${host}:${port}/viewer/index.html?graph=${encodeURIComponent(graphPath)}&code=${encodeURIComponent(codePath)}&metrics=${encodeURIComponent(metricsPath)}&inbox=${encodeURIComponent(inboxPath)}&review=${encodeURIComponent(reviewPath)}&pending=${encodeURIComponent(pendingDir)}`;
+    const url = `http://${host}:${port}/viewer/index.html?graph=${encodeURIComponent(graphPath)}&code=${encodeURIComponent(codePath)}&metrics=${encodeURIComponent(metricsPath)}&inbox=${encodeURIComponent(inboxPath)}&review=${encodeURIComponent(reviewPath)}&pending=${encodeURIComponent(pendingDir)}&view=code`;
     const server = (0, node_http_1.createServer)((req, res) => {
         const requestUrl = new URL(req.url ?? "/", `http://${host}:${port}`);
         let filePath = null;
         if (requestUrl.pathname === "/" || requestUrl.pathname === "/viewer") {
-            filePath = (0, node_path_1.join)(viewerDir, "index.html");
+            const viewerSearch = requestUrl.search || new URL(url).search;
+            res.writeHead(302, { location: `/viewer/index.html${viewerSearch}` });
+            res.end();
+            return;
         }
         else if (requestUrl.pathname.startsWith("/viewer/")) {
             filePath = (0, node_path_1.join)(viewerDir, (0, node_path_1.normalize)(requestUrl.pathname.replace(/^\/viewer\//, "")));

package/dist/index.js

@@ -153,6 +153,7 @@ function listTools() {
                 type: "object",
                 properties: {
                     project_dir: { type: "string" },
+                    full: { type: "boolean", description: "Force a full code graph rebuild instead of reusing unchanged graph artifacts." },
                 },
                 required: ["project_dir"],
             },
@@ -173,7 +174,18 @@ function listTools() {
         },
         {
             name: "kage_code_index",
-            description: "Write .agent_memory/code_index/lsp-symbols.json, an LSP-compatible symbol artifact consumed by the code graph for higher parser coverage.",
+            description: "Write external code index artifacts consumed by the code graph. Prefers SCIP when scip-typescript and scip are installed, then falls back to the built-in LSP-compatible symbol index.",
+            inputSchema: {
+                type: "object",
+                properties: {
+                    project_dir: { type: "string" },
+                },
+                required: ["project_dir"],
+            },
+        },
+        {
+            name: "kage_structural_index",
+            description: "Build the complete cache-backed structural index for large repos. This covers all supported source/config/doc files and writes .agent_memory/structural artifacts separate from learned memory.",
             inputSchema: {
                 type: "object",
                 properties: {
@@ -725,12 +737,18 @@ async function callTool(name, args) {
         };
     }
     if (name === "kage_code_index") {
-        const result = (0, kernel_js_1.writeLspSymbolIndex)(String(args?.project_dir ?? ""));
+        const result = (0, kernel_js_1.writeCodeIndex)(String(args?.project_dir ?? ""));
         return {
             content: [{ type: "text", text: JSON.stringify(result, null, 2) }],
             isError: !result.ok,
         };
     }
+    if (name === "kage_structural_index") {
+        const result = (0, kernel_js_1.buildStructuralIndex)(String(args?.project_dir ?? ""));
+        return {
+            content: [{ type: "text", text: JSON.stringify(result, null, 2) }],
+        };
+    }
     if (name === "kage_metrics") {
         const result = (0, kernel_js_1.kageMetrics)(String(args?.project_dir ?? ""));
         return {
@@ -752,7 +770,7 @@ async function callTool(name, args) {
         };
     }
     if (name === "kage_refresh") {
-        const result = (0, kernel_js_1.refreshProject)(String(args?.project_dir ?? ""));
+        const result = (0, kernel_js_1.refreshProject)(String(args?.project_dir ?? ""), { full: Boolean(args?.full) });
         return {
             content: [{ type: "text", text: JSON.stringify(result, null, 2) }],
             isError: !result.ok,