graph-indexer 2.1.0 → 3.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (129) hide show
  1. package/CHANGELOG.md +260 -1
  2. package/README.md +232 -312
  3. package/SECURITY.md +37 -59
  4. package/bin/graph-indexer.mjs +251 -0
  5. package/integrations/claude-code/.claude-plugin/plugin.json +8 -0
  6. package/integrations/claude-code/.mcp.json +8 -0
  7. package/integrations/claude-code/agents/code-structure.md +22 -0
  8. package/integrations/claude-code/hooks/gi-hook.mjs +99 -0
  9. package/integrations/claude-code/hooks/hooks.json +16 -0
  10. package/integrations/opencode/graph-indexer.js +124 -0
  11. package/package.json +31 -105
  12. package/src/cli/helper.mjs +67 -0
  13. package/src/cli/hook.mjs +356 -0
  14. package/src/cli/init.mjs +255 -0
  15. package/src/cli/resident.mjs +175 -0
  16. package/src/index/discover.mjs +130 -0
  17. package/src/index/fingerprint.mjs +30 -0
  18. package/src/index/graph.mjs +61 -0
  19. package/src/index/indexer.mjs +476 -0
  20. package/src/index/modules.mjs +331 -0
  21. package/src/index/resolver.mjs +954 -0
  22. package/src/mcp/render.mjs +60 -0
  23. package/src/mcp/server.mjs +147 -0
  24. package/src/mcp/tools.mjs +973 -0
  25. package/src/parse/arity.mjs +152 -0
  26. package/src/parse/extract.mjs +1031 -0
  27. package/src/parse/lang/csharp.mjs +106 -0
  28. package/src/parse/lang/ecma.mjs +438 -0
  29. package/src/parse/lang/go.mjs +194 -0
  30. package/src/parse/lang/jvm.mjs +118 -0
  31. package/src/parse/lang/kotlin.mjs +97 -0
  32. package/src/parse/lang/misc.mjs +115 -0
  33. package/src/parse/lang/native.mjs +126 -0
  34. package/src/parse/lang/python.mjs +153 -0
  35. package/src/parse/lang/rust.mjs +157 -0
  36. package/src/parse/lang/scripting.mjs +219 -0
  37. package/src/parse/languages.mjs +48 -0
  38. package/src/parse/runtime.mjs +56 -0
  39. package/src/query/check.mjs +343 -0
  40. package/src/query/facts.mjs +323 -0
  41. package/src/query/intel.mjs +680 -0
  42. package/src/query/read.mjs +81 -0
  43. package/src/query/testcmd.mjs +119 -0
  44. package/src/query/textsearch.mjs +160 -0
  45. package/src/search/search.mjs +210 -0
  46. package/src/search/tokenize.mjs +188 -0
  47. package/src/store/db.mjs +169 -0
  48. package/src/util/paths.mjs +24 -0
  49. package/vendor/grammars/bash.wasm.gz +0 -0
  50. package/vendor/grammars/c.wasm.gz +0 -0
  51. package/vendor/grammars/c_sharp.wasm.gz +0 -0
  52. package/vendor/grammars/cpp.wasm.gz +0 -0
  53. package/vendor/grammars/css.wasm.gz +0 -0
  54. package/vendor/grammars/go.wasm.gz +0 -0
  55. package/vendor/grammars/java.wasm.gz +0 -0
  56. package/vendor/grammars/javascript.wasm.gz +0 -0
  57. package/vendor/grammars/kotlin.wasm.gz +0 -0
  58. package/vendor/grammars/manifest.json +148 -0
  59. package/vendor/grammars/php.wasm.gz +0 -0
  60. package/vendor/grammars/python.wasm.gz +0 -0
  61. package/vendor/grammars/ruby.wasm.gz +0 -0
  62. package/vendor/grammars/rust.wasm.gz +0 -0
  63. package/vendor/grammars/scala.wasm.gz +0 -0
  64. package/vendor/grammars/tsx.wasm.gz +0 -0
  65. package/vendor/grammars/typescript.wasm.gz +0 -0
  66. package/vendor/web-tree-sitter/web-tree-sitter.js +3980 -0
  67. package/vendor/web-tree-sitter/web-tree-sitter.wasm +0 -0
  68. package/cli-ui.mjs +0 -78
  69. package/colbert.mjs +0 -168
  70. package/config.mjs +0 -494
  71. package/daemon-ctl.mjs +0 -175
  72. package/daemon-lock.mjs +0 -69
  73. package/embedders/python/mlx_embed_server.py +0 -61
  74. package/embedders/python/requirements-mlx.txt +0 -5
  75. package/embedders/setup-mlx.mjs +0 -110
  76. package/embeddings.mjs +0 -571
  77. package/engine/binary.mjs +0 -437
  78. package/engine/memory.mjs +0 -1003
  79. package/engine/sqlite.mjs +0 -1136
  80. package/enrichment.mjs +0 -521
  81. package/git-signals.mjs +0 -248
  82. package/indexer.mjs +0 -546
  83. package/init.mjs +0 -1809
  84. package/layout.mjs +0 -123
  85. package/mcp/centrality.mjs +0 -117
  86. package/mcp/format.mjs +0 -61
  87. package/mcp/resolver.mjs +0 -116
  88. package/mcp/symbolgraph.mjs +0 -137
  89. package/mcp/taint.mjs +0 -225
  90. package/mcp/tools.mjs +0 -1582
  91. package/mcp/topology.mjs +0 -676
  92. package/mcp-server.mjs +0 -197
  93. package/parse/extractor.mjs +0 -628
  94. package/parse/imports.mjs +0 -365
  95. package/parse/interprocedural.mjs +0 -134
  96. package/parse/languages.mjs +0 -253
  97. package/parse/metadata.mjs +0 -701
  98. package/parse/routes.mjs +0 -327
  99. package/parse/scip.mjs +0 -391
  100. package/parse/taint-patterns.mjs +0 -119
  101. package/prompts/CORE.md +0 -91
  102. package/prompts/DOMAIN_TEMPLATE.md +0 -46
  103. package/prompts/INTEGRATION.md +0 -123
  104. package/prompts/frameworks/ANDROID.md +0 -17
  105. package/prompts/frameworks/ASPNET_CORE.md +0 -19
  106. package/prompts/frameworks/FASTAPI_DJANGO.md +0 -20
  107. package/prompts/frameworks/LARAVEL_SYMFONY.md +0 -22
  108. package/prompts/frameworks/NODE_EXPRESS_NESTJS.md +0 -19
  109. package/prompts/frameworks/RAILS.md +0 -20
  110. package/prompts/frameworks/REACT.md +0 -20
  111. package/prompts/frameworks/SPRING_BOOT.md +0 -20
  112. package/prompts/languages/BASH.md +0 -22
  113. package/prompts/languages/C.md +0 -21
  114. package/prompts/languages/CSHARP.md +0 -24
  115. package/prompts/languages/CSS_SCSS.md +0 -22
  116. package/prompts/languages/GO.md +0 -24
  117. package/prompts/languages/JAVA.md +0 -23
  118. package/prompts/languages/JAVASCRIPT_TYPESCRIPT.md +0 -24
  119. package/prompts/languages/KOTLIN.md +0 -24
  120. package/prompts/languages/PHP.md +0 -23
  121. package/prompts/languages/PYTHON.md +0 -23
  122. package/prompts/languages/RUBY.md +0 -23
  123. package/prompts/languages/RUST.md +0 -21
  124. package/prompts/languages/SWIFT.md +0 -21
  125. package/seal.mjs +0 -400
  126. package/search-core.mjs +0 -777
  127. package/search-sparse.mjs +0 -183
  128. package/storage.mjs +0 -73
  129. package/watch-daemon.mjs +0 -289
package/README.md CHANGED
@@ -4,12 +4,12 @@
4
4
  <h1>Graph Indexer</h1>
5
5
 
6
6
  <p>
7
- <em>A local MCP server that gives AI coding agents an AST-precise, topology-aware index of your codebase.</em>
7
+ <em>A live code graph and search engine for AI coding agents, served locally over MCP.</em>
8
8
  </p>
9
9
 
10
10
  <a href="https://www.npmjs.com/package/graph-indexer"><img src="https://img.shields.io/npm/v/graph-indexer?color=007acc&style=for-the-badge" alt="npm version"></a>
11
11
  <a href="https://www.npmjs.com/package/graph-indexer"><img src="https://img.shields.io/npm/dt/graph-indexer?color=4caf50&style=for-the-badge" alt="NPM Downloads"></a>
12
- <a href="https://nodejs.org"><img src="https://img.shields.io/badge/Node.js-%3E%3D18-brightgreen?style=for-the-badge&logo=nodedotjs" alt="Node.js 18+"></a>
12
+ <a href="https://nodejs.org"><img src="https://img.shields.io/badge/Node.js-%3E%3D22.5-brightgreen?style=for-the-badge&logo=nodedotjs" alt="Node.js 22.5+"></a>
13
13
  <a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-yellow.svg?style=for-the-badge" alt="License: MIT"></a>
14
14
  </div>
15
15
 
@@ -17,349 +17,269 @@
17
17
 
18
18
  ## What it does
19
19
 
20
- Graph Indexer is a local [Model Context Protocol](https://modelcontextprotocol.io) server that builds an AST-precise index of your repository with Tree-sitter and serves it to AI coding agents. Instead of grepping text or embedding whole files, it indexes *semantic chunks* — functions, classes, methods — and the call graph and import topology that connect them, so an agent can find the right symbol, see every caller and dependency that would be affected by a change, and resolve references across files. It runs entirely on your machine: the default search path is lexical (BM25 + morphological stemming) and needs no model, no daemon, and no network. Dense vector embeddings, LLM enrichment, and an LLM reranker are all available but off by default — you opt into each one when you have a measured reason to.
21
-
22
- **Why it matters for AI coding agents:**
23
-
24
- - **Up to 98.2% Token Savings:** Delivers the exact AST chunk needed instead of dumping 1M token context files.
25
- - **Blast radius before every edit.** `get_call_graph` and `find_references` surface every caller, subclass, and dependency a change would touch, so the agent reasons about impact on code it never opened.
26
- - **Private by default.** Everything runs locally; the default path makes zero network calls and needs no model — your code never leaves your machine. (The very first index build fetches only the Tree-sitter grammar(s) for languages actually in your repo — see [Language grammars](#language-grammars) — cached from then on; `--sealed` enforces zero network hard.)
27
- - **One command, any language.** Guided setup wires your editors in seconds, and the lexical engine indexes 14 languages — installing each one's grammar on first use, not bundled, so a fresh `npm install graph-indexer` stays small.
28
-
29
- ## 📦 Prerequisites
30
-
31
- * **Node.js** 18+ (22+ recommended if you hit >15k chunks for automatic SQLite scaling).
32
- * **OS:** Agnostic. macOS Apple Silicon required only for the optional MLX GPU acceleration.
33
- * **Network (first index build only):** installs the Tree-sitter grammar(s) for languages in your repo (a few MB each) — see [Language grammars](#language-grammars). Cached after; fully offline from then on, or pre-install before going air-gapped.
34
- * **Optional:** Ollama for embeddings, enrichment, and reranking (see [Ollama setup](#ollama-setup)).
35
- * **Optional:** Python 3.10+ for the MLX embedder (macOS Apple Silicon only).
20
+ Graph Indexer parses your repository with tree-sitter and keeps a structural index of it:
21
+ every definition, every reference bound to the definition it actually points at, the call
22
+ graph, inheritance, imports, and which tests exercise what. Coding agents (Claude Code,
23
+ Cursor, VS Code Copilot, Codex, Gemini CLI, OpenCode, Kilo Code, Junie, Zed, Devin, …) query it
24
+ through [Model Context Protocol](https://modelcontextprotocol.io) tools, the CLI and hooks
25
+ instead of reading whole files and grepping.
26
+
27
+ - **Answers are exact, not guesses.** `find_references` binds each use through scopes,
28
+ imports and inferred receiver types, so it lists the callers of *this* `get` and not of every
29
+ method named `get`. Measured against the TypeScript compiler it finds 89% of the real
30
+ references at 98% precision; grep finds all of them buried among seven times as many false hits.
31
+ - **Answers are never stale.** The index re-syncs with the files on disk before every answer,
32
+ including edits the agent made a second ago, so line numbers and call graphs are current.
33
+ - **Answers are small.** A tool reply is typically 400–1,500 tokens: the right function with
34
+ line numbers, not a 2,000-line file.
35
+ - **Nothing to set up.** Zero dependencies, no native builds, no model downloads, no network,
36
+ no service to install. The tree-sitter runtime and 16 grammars ship as WebAssembly inside the
37
+ package. The MCP server keeps the index open while the agent runs; with hooks and no server, a
38
+ resident process started on demand does the same so hooks and CLI queries answer in about
39
+ 0.1 s, and exits after 30 idle minutes (`GRAPH_INDEXER_RESIDENT=0` turns it off).
36
40
 
37
41
  ## Quick start
38
42
 
39
- ### 1. Run the Interactive Indexer
40
- Go to your project root (works for Python, Go, Rust, TS/JS, C#, and 9 more languages) and run:
43
+ Requires **Node.js 22.5+** (for the built-in `node:sqlite`).
41
44
 
42
45
  ```bash
43
- npx graph-indexer /path/to/your/repo
46
+ cd your-repo
47
+ npx graph-indexer init # wires the MCP server into the agents it detects
48
+ npx graph-indexer index # optional: build the index now (the server also does it on start)
44
49
  ```
45
50
 
46
- That runs the [guided setup](#guided-setup) against that repo, which leaves it ready to use. It is idempotent — re-run it whenever you add a language or change a setting; it merges into what you already have and never clobbers another tool's config. Every generated file lands in `.graph-indexer/` (git-ignored), so your repo root stays clean.
47
-
48
- ### 2. Connect Your Agent
49
-
50
- **If you ran the guided setup and selected your agent in step 4, this is already done — skip ahead.**
51
-
52
- The guided setup writes the MCP config automatically (with absolute paths that survive GUI launches) for **Claude Code** (`.mcp.json`), **Claude Desktop**, **Cursor** (`.cursor/mcp.json`), and **VS Code** (`.vscode/mcp.json`). Re-run `npx graph-indexer /path/to/your/repo` at any time to add or refresh a client.
51
+ `init` writes project-level MCP configuration for the agents it finds (`.mcp.json` for Claude
52
+ Code, `.cursor/mcp.json`, `.vscode/mcp.json`, `.gemini/settings.json`, `opencode.json` for
53
+ OpenCode and Kilo Code, `.junie/mcp/mcp.json`, `.zed/settings.json`; it prints the Codex TOML
54
+ and Devin's server entry), merges instead of overwriting other servers, adds `.graph-indexer/`
55
+ to `.gitignore`, and adds a short block to `CLAUDE.md` / `AGENTS.md` on how to look code up.
56
+ For Claude Code it also writes a structural helper, `.claude/agents/code-structure.md`: a
57
+ read-only sub-agent on a small model that answers call-site, caller, subclass and impact
58
+ questions from the index, so the main agent can hand them off and get back only the list (the
59
+ Claude Code plugin ships it too). `--hooks` adds the Claude Code hooks to `.claude/settings.json`,
60
+ which Devin, Copilot CLI and Cursor also run, and the OpenCode / Kilo Code plugin that appends the
61
+ same context to their tool output. Flags: `--agents claude,cursor,vscode,gemini,codex,opencode,junie,zed,devin`,
62
+ `--all`, `--hooks`, `--local` (use this checkout instead of `npx`), `--dry-run`,
63
+ `--no-instructions`, `--no-helper`.
53
64
 
54
65
  <details>
55
- <summary>Manual wiring (if you skipped setup or use a client not listed above)</summary>
66
+ <summary>Manual configuration</summary>
56
67
 
57
- #### Claude Code — CLI
58
-
59
- ```bash
60
- claude mcp add graph-indexer -- npx -y -p graph-indexer idx-mcp --repo /path/to/your/repo
61
- ```
62
-
63
- #### Cursor — `.cursor/mcp.json`
68
+ Any MCP client can run the server over stdio:
64
69
 
65
70
  ```json
66
71
  {
67
72
  "mcpServers": {
68
- "graph-indexer": {
69
- "command": "npx",
70
- "args": ["-y", "-p", "graph-indexer", "idx-mcp", "--repo", "/path/to/your/repo"]
71
- }
72
- }
73
- }
74
- ```
75
-
76
- #### VS Code — `.vscode/mcp.json`
77
-
78
- ```json
79
- {
80
- "servers": {
81
- "graph-indexer": {
82
- "command": "npx",
83
- "args": ["-y", "-p", "graph-indexer", "idx-mcp", "--repo", "/path/to/your/repo"]
84
- }
73
+ "graph-indexer": { "command": "npx", "args": ["-y", "graph-indexer@3", "serve", "--repo", "/path/to/repo"] }
85
74
  }
86
75
  }
87
76
  ```
88
77
 
89
- #### Claude Desktop — `claude_desktop_config.json`
78
+ VS Code uses `"servers"` instead of `"mcpServers"` and `"type": "stdio"`. Codex CLI
79
+ (`~/.codex/config.toml`):
90
80
 
91
- ```json
92
- {
93
- "mcpServers": {
94
- "graph-indexer": {
95
- "command": "npx",
96
- "args": ["-y", "-p", "graph-indexer", "idx-mcp", "--repo", "/path/to/your/repo"]
97
- }
98
- }
99
- }
81
+ ```toml
82
+ [mcp_servers.graph-indexer]
83
+ command = "npx"
84
+ args = ["-y", "graph-indexer@3", "serve", "--repo", "/path/to/repo"]
100
85
  ```
101
86
 
102
- > **Note:** The `-p graph-indexer idx-mcp` form is required — `npx graph-indexer` runs the setup wizard, not the MCP server. If `npx` isn't on your GUI PATH (common on macOS when launching editors from Finder/Dock), run the guided setup instead — it writes absolute paths that always work.
103
-
87
+ Without `--repo` the server indexes the git repository containing its working directory
88
+ (`GRAPH_INDEXER_REPO` also works).
104
89
  </details>
105
90
 
106
- Once connected, the agent can call `search_code`. A query like `search_code("rate limiting middleware")` returns ranked semantic chunks, not whole files:
107
-
108
- ```jsonc
109
- [
110
- {
111
- "score": 8.41,
112
- "chunk": {
113
- "id": "src/middleware/rateLimit.ts:14",
114
- "file_path": "src/middleware/rateLimit.ts",
115
- "name": "rateLimiter",
116
- "node_type": "function_declaration",
117
- "start_line": 14, "end_line": 47,
118
- "calls": ["tokenBucket", "getClientKey"],
119
- "class_context": ""
120
- }
121
- }
122
- ]
123
- ```
124
-
125
- The agent can then call `get_call_graph("rateLimiter")` to see what calls it (the blast radius) before changing it.
126
-
127
- ## Guided setup
91
+ ## The tools
128
92
 
129
- `npx graph-indexer <path>` runs an interactive wizard that leaves the repo ready to use. It is **idempotent** — re-run it whenever you add a language or change a setting; it merges into what you already have and never clobbers another tool's config. Every generated file lands in `.graph-indexer/` (git-ignored), so your repo root stays clean.
130
-
131
- Before the steps, it scans the repo to pre-select your stack — languages from a bounded file walk, frameworks from your manifests (`package.json`, `pyproject.toml`, `composer.json`, `pom.xml`, `Gemfile`, `*.csproj`, …) — and, if it finds a pre-v1.4 layout, tidies those stray artifacts into `.graph-indexer/` first. Then six steps:
132
-
133
- | Step | You choose | What happens |
134
- |------|------------|--------------|
135
- | **1 · Languages** | Languages to index | Detected languages are pre-checked — press Enter to accept. Selecting none indexes **all** supported languages. |
136
- | **2 · Frameworks** | Prompt add-ons | Filtered to your languages and pre-checked from detection. Sharpens the agent prompt for React, Express/NestJS, FastAPI/Django, Spring, Rails, Laravel/Symfony, ASP.NET, or Android. |
137
- | **3 · Search engine & LLM** | Retrieval engine | **Press Enter for the recommended default** — lexical search, no LLM, no network. Everything heavier is opt-in (see below). |
138
- | **4 · Agents & IDEs** | Your coding tools | Multi-select which agents to wire: Claude Code/Desktop, Cursor, VS Code/Copilot, Windsurf, Cline/Roo, JetBrains Junie, Codex (AGENTS.md), Gemini CLI. Pre-checked from your saved choice, else what's detected, else all. Drives steps 5 **and** 7 — deselect a tool and graph-indexer generates nothing for it. |
139
- | **5 · MCP server wiring** | *(automatic)* | Wires the MCP server for each **selected** editor (VS Code, Cursor, Claude Desktop, Claude Code). **Merge-safe**: your other MCP servers and keys are preserved. |
140
- | **6 · Project files & daemon** | *(automatic)* | Adds `mcp:*` npm scripts (index + daemon control), a managed `.gitignore` block, and `.graph-indexer/config.json` (which remembers your agent selection). |
141
- | **7 · Agent instructions** | *(automatic)* | Always writes the canonical layered prompt (`GRAPH_INDEXER_PROMPT.md`) + a `GRAPH_INDEXER_DOMAIN.md` template for your own rules. Then, for each **selected** agent: `@`-imports (no duplication) in `CLAUDE.md` and `GEMINI.md`; rule files for Cursor, Windsurf, and Cline/Roo; and managed blocks in `.github/copilot-instructions.md`, `.junie/guidelines.md`, and `AGENTS.md` (Codex/Zed/Jules) — preserving anything already there. |
142
-
143
- It finishes with a grouped summary (created / updated / kept / skipped / warnings), offers to **build the index now**, and prints your next steps.
144
-
145
- **Most repos need nothing past step 3's default** — the lexical engine works in any language, installing that language's grammar on the first index build. Step 3 only branches when you decline the defaults:
146
-
147
- - **Storage** — `auto` (in-memory, promoting to SQLite past ~15k chunks), or force in-memory / SQLite.
148
- - **Embeddings** — `off` (default), `auto`, **Ollama**, **local** (in-process MiniLM, ~25 MB on first index), or **Apple Metal (MLX)** on macOS. Ollama is probed only when you actually pick it; choosing MLX offers to provision its Python venv on the spot.
149
- - **Enrichment / reranker** — off by default. A measured per-language note is shown before the reranker prompt (it helps Go/Python, regresses JS/TS). Model pickers are provider-aware, and if you select MLX the wizard verifies `mlx_lm.server` end-to-end — installing `mlx-lm`, starting the server, and pre-loading the model — so your first index can't fail against a server that isn't up.
150
-
151
- ### Non-interactive & preview
152
-
153
- | Flag | Effect |
154
- |------|--------|
155
- | `--yes`, `-y` | Accept all detected/default selections — no prompts (CI). |
156
- | `--dry-run` | Print every file action without writing anything. |
157
- | `--all-languages` | Index every supported language (implies `--yes`). |
158
- | `--help`, `-h` | Show usage. |
93
+ | Tool | Use it to | Typical reply |
94
+ |---|---|---|
95
+ | `read_code` | read symbols, line ranges or files — several per call — without searching for what they use | code with line numbers, then where each name it uses is defined (file:line and signature); who uses a symbol and which tests reference it; a long file's outline |
96
+ | `search_text` | grep any text across the repository (code, config, docs) | grep lines with the enclosing definition and, for identifiers, the definition each match refers to; where the definition is when the search missed it |
97
+ | `search_code` | find where a behaviour or identifier lives (natural language or names) | ranked symbols with location, signature, doc line and matching lines |
98
+ | `find_references` | see every use before renaming or changing a signature | uses grouped by file, enclosing function, confidence, calls through base types marked |
99
+ | `call_graph` | trace callers and callees across files | a tree with locations and confidence |
100
+ | `change_impact` | get the blast radius of a change, or of your uncommitted diff | transitive dependents by distance, tests to run, co-changed files, risk |
101
+ | `check_changes` | verify uncommitted edits without building | syntax errors introduced, calls that no longer fit, removed names still in use, the tests to run |
102
+ | `outline` | see a file's structure, or a ranked map of a directory or the repo | signatures and line ranges within a token budget |
103
+
104
+ All tools are read-only. Replies are plain text, capped, best result first, and end with a hint
105
+ for the next step. Example, on [gin](https://github.com/gin-gonic/gin):
106
+
107
+ ```text
108
+ > search_code("where are trusted proxies checked")
109
+ 3 results for "where are trusted proxies checked" (of 55 candidates):
110
+ 1. Engine.isTrustedProxy — method · gin.go:445-455
111
+ func (engine *Engine) isTrustedProxy(ip net.IP) bool
112
+ isTrustedProxy will check whether the IP address is included in the trusted list according to Engine.trustedCIDRs
113
+ 446: if engine.trustedCIDRs == nil {
114
+ 449: for _, cidr := range engine.trustedCIDRs {
115
+ 2. Engine.isUnsafeTrustedProxies — method · gin.go:433-435
116
+ …
117
+
118
+ > change_impact(symbols: ["Engine.isTrustedProxy"], depth: 2)
119
+ Impact of changing Engine.isTrustedProxy
120
+ dependents: 9 symbols in 3 files (non-test)
121
+ distance 1 (3):
122
+ Context.ClientIP context.go:771-806
123
+ Engine.isUnsafeTrustedProxies gin.go:433-435
124
+ Engine.validateHeader gin.go:458-477
125
+ distance 2 (6):
126
+ LoggerWithConfig logger.go:203-270 [high]
127
+ Engine.Run gin.go:376-388 [high]
128
+ …
129
+ tests to run (1 file):
130
+ context_test.go: TestRemoteIPFail, TestContextClientIP
131
+ often changed together (git): context.go (14/42), context_test.go (10/42), …
132
+ risk: medium (9 dependents, 1 test files)
133
+ ```
159
134
 
160
- Setup also runs non-interactively whenever stdin isn't a TTY, so piping into it behaves like `--yes`.
135
+ Every reference carries a confidence: **exact/high** means it was bound through a scope,
136
+ import or inferred receiver type; **likely/possible** means only the name matched. Footers
137
+ report same-name call sites that could not be bound, so an agent knows when to double-check
138
+ with grep.
139
+
140
+ ## Measured results
141
+
142
+ All numbers come from scripts in [`bench/`](bench) on public repositories pinned to exact
143
+ commits; [docs/BENCHMARKS.md](docs/BENCHMARKS.md) has the methodology and how to reproduce them.
144
+
145
+ **References vs the TypeScript compiler.** For 400 randomly sampled functions, methods, classes
146
+ and interfaces of nestjs, the references each system reports are compared with the compiler's
147
+ own `findReferences` (import lines and declarations excluded):
148
+
149
+ | | precision | recall | F1 |
150
+ |---|---|---|---|
151
+ | graph-indexer `find_references` | **0.997** | **0.962** | **0.979** |
152
+ | graph-indexer 2.x `find_references` | 0.176 | 0.886 | 0.294 |
153
+ | same-name references (no binding) | 0.252 | 0.975 | 0.400 |
154
+ | grep for the name | 0.128 | 0.993 | 0.227 |
155
+
156
+ *Dispatch semantics: calls through a base class or interface count, calls to sibling overrides
157
+ do not. Against the compiler's full rename set (sibling overrides included) recall is 0.925. The
158
+ fixture has no `node_modules`, so library-typed values are invisible to the compiler: precision is
159
+ a lower bound.* The same measurement against the Go type checker (gin, caddy, nats-server) gives
160
+ precision 0.992–1.000 and recall 0.972–0.981, and against the Java compiler (spring-petclinic,
161
+ jsoup, Commons Collections) 0.955–0.994 and 0.861–0.989; grep's precision there is 0.02–0.61.
162
+ In the server of Twenty, an open-source CRM monorepo (28,512 files), precision is 0.999 and
163
+ recall 0.924; grep's precision is 0.009 and 2.x's 0.052.
164
+
165
+ **Localization from real commits.** 169 focused commits of five repositories are replayed: the
166
+ commit subject is the query, the files and functions it changed are the answer, and the index is
167
+ rolled back to the parent commit first.
168
+
169
+ | | file Acc@1 | file Acc@5 | function Acc@5 | function MRR@10 |
170
+ |---|---|---|---|---|
171
+ | graph-indexer `search_code` | **0.544** | **0.805** | **0.528** | **0.432** |
172
+ | grep, files ranked by idf-weighted hits | 0.485 | 0.769 | 0.514 | 0.374 |
173
+ | BM25 over the same index | 0.373 | 0.722 | 0.401 | 0.290 |
174
+
175
+ **Symbol search.** 377 hand-written queries over nine repositories in eight languages, scored
176
+ strictly at symbol level: rank-1 0.703 and MRR 0.760, against 0.552 and 0.646 for graph-indexer
177
+ 2.x on the same queries. On the 169 held-out queries (never used for tuning) MRR is 0.773 vs 0.632.
178
+
179
+ **Against 2.x.** In real Claude Code sessions on nestjs and Twenty (40 tasks, 120 runs), 3.0
180
+ cost 0.43 of 2.x on code questions and 0.46 on refactors, with a quarter to a third of the
181
+ tokens spent looking code up; 2.x cost about twice as much as no index, as its tools and prompt
182
+ add 11,300 tokens to every call (3.0: 1,200). Against no index, 3.0 halves those lookup tokens at
183
+ 0.84–0.86 of the cost. Every benchmark against 2.x is in
184
+ [docs/BENCHMARKS.md](docs/BENCHMARKS.md#5-against-graph-indexer-2x).
185
+
186
+ **Agents.** The same model gets the same task with built-in tools only, with graph-indexer added,
187
+ or with graph-indexer instead of grep, and an oracle it never sees grades the result: the
188
+ TypeScript compiler for code questions and multi-site refactors, the hidden tests of the real fix
189
+ for issues reported after the model's training cutoff. In the fourth round (19 new tasks, 122
190
+ runs, all solved), graph-indexer as `init` installs it answered code questions at 0.44 of the
191
+ cost of grep alone (95% CI 0.32–0.67) and 0.37 of the time, with half the model's reasoning. A
192
+ smaller model, at half the price per token, answered the same questions exactly with graph-indexer
193
+ (7 of 7, against 5 of 7 with grep alone) at 0.26 of the cost (0.18–0.50) and 0.32 of the time — a
194
+ tenth of what the larger model spent with grep — and did multi-site refactors at 0.69
195
+ (0.56–0.87). Handed off the way a main agent delegates them, 24 new code questions were answered
196
+ by the structural helper `init` installs as exactly as by a general-purpose sub-agent with grep
197
+ (20 of 24 each), at 0.24 of the cost (0.18–0.33) and 0.28 of the time, with about 170 tokens
198
+ handed back instead of the 23k a lookup adds to the asking agent's context. Fixing real issues
199
+ does not get cheaper: 0.85 (0.76–0.96) in the fourth round, a gain the lookup rules it installs
200
+ give on their own (0.86), and 1.05 (0.82–1.46) with the smaller model. In real Claude Code
201
+ sessions (252 runs) graph-indexer as `init` installs it answered all 12 code questions exactly
202
+ against 8–9 of 12 without it — the misses were the two-level caller questions — at 0.80 of the
203
+ cost and 0.65 of the time; refactors came out the same, and on real issues, where the agent never
204
+ called it, it cost 1.02–1.05 once its tool definitions were trimmed to what agents use. In Go (57
205
+ real sessions on caddy, graded by the Go type checker) graph-indexer answered all 14 code
206
+ questions against 13 without it, at 0.87 of the cost, and did the refactors at 0.81 of the cost
207
+ and half the time. With the usual model every run with graph-indexer solved its task. Details in
208
+ [docs/AGENTIC-BENCHMARK.md](docs/AGENTIC-BENCHMARK.md).
209
+
210
+ **Speed.** A full index of nestjs (1,641 files, 96k lines) takes 3.7 s; afterwards only changed
211
+ files are re-parsed. Warm tool calls take 1–4 ms for the graph tools, about 30 ms for search and
212
+ 18 ms for a repository outline.
161
213
 
162
214
  ## How it works
163
215
 
164
- - **AST indexing.** Tree-sitter parses each file into a syntax tree, and the indexer extracts one *chunk* per top-level definition (function, class, method, struct, …) with its name, parameters, line range, call sites, and enclosing class. A "god class" is split so its methods become their own chunks. Supported languages: **TypeScript/JavaScript** (`.ts .tsx .js .jsx .mjs .cjs`), **Python**, **Go**, **Rust**, **Java**, **Kotlin**, **C#**, **C**, **Ruby**, **PHP**, **Bash**, and **Swift**, plus **CSS/SCSS**.
165
- - **Retrieval.** A hybrid ranker fuses a lexical channel (BM25 with camelCase splitting and language-agnostic Porter stemming) with an optional dense-vector channel (local embeddings) via Reciprocal Rank Fusion. With embeddings off, only the lexical channel runs — and it is the default.
166
- - **Call graph.** `get_call_graph` returns the callers and callees of a symbol — the *blast radius* of a change — so an agent can see what it might break before editing code it never read.
167
- - **Backend parity.** The in-memory and SQLite backends share the same ranking core, so they return identical top-5 results for the same query (enforced by `test/sqlite.mjs`).
168
-
169
- ### Language grammars
170
-
171
- Tree-sitter grammars aren't bundled with the package — a fresh `npm install graph-indexer` doesn't download all 14 languages' native bindings, just the ~5 MB core engine. Instead, the first `idx-index` run installs the grammar(s) for whichever languages you actually selected (or, un-narrowed, whatever's detected) into `.graph-indexer/node_modules`, and reuses that cache on every later run — no re-download, no re-build. This works the same way whether graph-indexer is a project dependency, a global install, or run ad hoc via `npx`, because the install target is always your repo's own `.graph-indexer/` directory, not graph-indexer's own location. A platform-incompatible grammar (e.g. a missing native toolchain) is skipped on its own — other languages still install and index normally, the same graceful degradation `npm`'s `optionalDependencies` used to give for free. Under `--sealed`, auto-install is disabled entirely: an unresolvable grammar fails the run with the exact `npm install --prefix .graph-indexer <pkg>@<version>` command to run first, rather than silently shipping a partial index.
172
-
173
- ### MCP tools
174
-
175
- | Tool | Returns |
176
- |------|---------|
177
- | `search_code` | Ranked semantic chunks for a natural-language or symbol query. |
178
- | `get_chunk` | The full source of one chunk by id. |
179
- | `get_chunk_summary` | A compact summary of a chunk (signature, calls, context). |
180
- | `resolve_symbol` | Exact, case-insensitive symbol lookup by name. |
181
- | `get_file_skeleton` | The top-level structure (symbols + signatures) of a file. |
182
- | `get_call_graph` | Callers and callees of a symbol — the blast radius. |
183
- | `find_references` | Where a symbol is used: callers, subclasses, and type references. On a `--resolver scip` index, adds a precise cross-file **`resolved`-tier** dimension that disambiguates same-named symbols and recovers references the name heuristics miss. |
184
- | `find_routes` | HTTP routes mapped to their handler chunks (NestJS, FastAPI/Flask, Spring, Express/Koa). |
185
- | `get_subgraph` | A bounded connected subgraph around a seed symbol — its callees, high-confidence callers, and type/inheritance users, in one call. |
186
- | `get_repo_map` | A high-level map of the repository's modules and topology. With `--symbol-graph`, also lists the most-central symbols (PageRank). |
187
- | `list_index_stats` | Index health: chunk/file/symbol/vector counts and the active config. |
188
- | `tests_for` | The test/spec chunks that exercise a symbol (call or reference it) — which tests to run or update before changing it. |
189
- | `explain_symbol` | One-call overview of a symbol: signature, callees, callers (blast radius), subclasses/type users, routes it handles, tests, git recency/co-change, and (with `--symbol-graph`) its symbol-centrality rank. |
190
- | `impact_of_edit` | The precise blast radius of a change: pass the symbols/files you're about to edit → transitively-affected code, the routes that reach it, the tests to run, and git co-change. Most precise with `--symbol-graph`; `precision: 'strict'` follows only provably-unambiguous (`resolved`) edges (best on a `--resolver precise` or `--resolver scip` index). |
191
- | `trace_taint` | Security: trace untrusted data from a source (request body/query/params, argv/env, stdin) to a dangerous sink (eval/exec, SQL, fs/path, HTML, outbound request) across the call graph — injection-class risks (rce/sqli/xss/path/ssrf) with the source→sink path. JS/TS, Python, Java, Go; heuristic finder, not a verifier. |
192
- | `find_tainted_sinks` | Security orientation: every dangerous sink grouped by category, each flagged with whether an untrusted source reaches it — map the attack surface, then `trace_taint` a specific one. |
193
-
194
- ## Configuration
195
-
196
- Everything beyond the lexical default is opt-in. The server, indexer, and daemon all print their **effective configuration** at startup (storage backend, model names, which optional features are on), and emit a visible warning whenever an opt-in feature has a known trade-off — nothing is enabled silently.
197
-
198
- ### Headline trade-offs
199
-
200
- | Option | Default | When to enable | Cost |
201
- |--------|---------|----------------|------|
202
- | `--embeddings` | off | Larger repos where recall matters; lifts success@5 | Requires Ollama or the in-process MiniLM model; slower indexing |
203
- | `--embed-model qwen3-embedding:4b` | `nomic-embed-text` | A code-specialized embedder — lifts agent-style recall on Go (and some code-heavy repos); measure first | slower indexing; requires Ollama; neutral on JS/Python in our tests |
204
- | `--enrichment` | off | Only useful paired with `--rerank`; alone it regresses | slowest indexing |
205
- | `--rerank` | off | Go/Python repos with weak semantic recall; regresses JS repos | Requires an Ollama 7B model; adds query latency |
206
- | `--rerank --rerank-provider cross-encoder` | `generative` | Want a reranker's semantic lift with **no LLM/daemon** — air-gapped, deterministic, fast | Optional `@huggingface/transformers`; smaller lift than the 7B judge |
207
- | `--use-sqlite` | `auto` | Repos past ~15k chunks or memory-constrained environments | Slightly higher query latency; needs Node 22+ |
208
-
209
- For most repos, the default (lexical + stemming, no embeddings) is the right starting point. Enable embeddings when you notice the agent missing chunks it should find. Enable the reranker only on Go or Python repos after measuring whether it helps — it is known to regress JavaScript repositories.
210
-
211
- **Reranker providers.** The reranker reorders the over-fetched candidate pool on natural-language queries; three providers are available (`--rerank-provider`):
212
-
213
- - **`generative`** (default) — a local LLM judge (Ollama 7B). The strongest reranker in our benchmarks (agent-style semantic rank-1 0.19 → 0.42 across the core suites; no symbolic regression), but it needs a running model and its output is non-deterministic.
214
- - **`cross-encoder`** — a local MS-MARCO cross-encoder (`Xenova/ms-marco-MiniLM-L-6-v2`, via the optional `@huggingface/transformers`). It scores each (query, candidate) pair and sorts. **Air-gapped, deterministic, and fast** (no Ollama/daemon; the model downloads once). It captures part of the lift — agent-style semantic rank-1 0.19 → 0.26, and notably Go (gin) 0.20 → 0.40 and Python (fastapi) 0.14 → 0.29 — but less than the 7B judge, and like all rerankers it is mixed on JavaScript. Pick it when you want a reranker without an LLM in the loop. Measure on your repo: `npm run test:eval -- --rerank --rerank-provider cross-encoder`.
215
- - **`late-interaction`** (B4) — a **ColBERT-style multi-vector MaxSim** reranker. It encodes the query into several vectors (the whole query + content sub-units, via the same local embedder) and scores `Σ_qi max_dj cos(qi, dj)` against each candidate's stored document sub-vectors — **no storage blow-up** (it reuses the base/summary/window vectors `--embeddings` already writes), air-gapped, deterministic, parity-safe. **Requires `--embeddings`** (a no-op without it). Measured at production retrieval depth: **helps Go** (gin semantic rank-1 0.00 → 0.40, overall 0.68 → 0.77, no symbolic regression) and **neutral on JavaScript** (express unchanged) — the "helps Go/Python" half of the reranker profile without the JS regression. Approximates true token-level ColBERT with a sentence embedder. Measure on your repo: `npm run test:eval -- --embeddings --rerank --rerank-provider late-interaction`.
216
-
217
- ### All CLI flags
218
-
219
- | Flag | Default | Effect |
220
- |------|---------|--------|
221
- | `--repo <path>` | current directory | Repository to index / serve. |
222
- | `--embeddings` | off | Enable the dense-vector channel. |
223
- | `--embed-model <model>` | `nomic-embed-text` | Ollama embedding model (e.g. `qwen3-embedding:4b`). |
224
- | `--embed-provider <auto\|ollama\|local\|mlx\|off>` | `auto` | Force the embedding backend (see [Embedder backends](#embedder-backends)). |
225
- | `--mlx-embed-model <model>` | `mlx-community/all-MiniLM-L6-v2-4bit` | Model loaded by the MLX (Apple Metal) embedder. |
226
- | `--use-sqlite` | `auto` | Force the disk-backed SQLite backend. |
227
- | `--enrichment` | off | Enable LLM enrichment of central chunks. |
228
- | `--enrich-model <model>` | `qwen2.5-coder:1.5b` | Model used for enrichment. |
229
- | `--enrich-max <n>` | 500 | Cap on new LLM calls per index run. |
230
- | `--enrich-concurrency <n>` | 4 | Parallel Ollama requests during enrichment. |
231
- | `--rerank` | off | Enable the reranker (reorders the over-fetched pool on NL queries). |
232
- | `--rerank-provider <generative\|cross-encoder\|late-interaction>` | `generative` | `generative` = local LLM judge; `cross-encoder` = local air-gapped MS-MARCO cross-encoder (optional `@huggingface/transformers`, no Ollama); `late-interaction` = ColBERT-style multi-vector MaxSim over the stored doc vectors (needs `--embeddings`; air-gapped, deterministic, no storage blow-up; measured helps-Go / neutral-JS at production depth). |
233
- | `--interprocedural` | off | Index-time inter-procedural receiver-type fixpoint: propagate return types along factory call chains so multi-hop receivers resolve in `get_call_graph` / `find_references`. Air-gapped, deterministic; default index byte-identical; search ranking unaffected. |
234
- | `--symbol-graph` | off | Persist a resolved chunk→chunk symbol graph (edges with kind + confidence) at index time, exposed via `getEdges`. `findCallers`/`findReferers` read it (identical sets, fall back to a scan). Also computes symbol-level centrality (confidence-weighted PageRank over the edges), surfaced by `explain_symbol` and `get_repo_map`. Air-gapped, deterministic; default index byte-identical; search ranking unaffected. |
235
- | `--taint` | off | Precompute and serialize the taint flow set (C2) at index time, so `trace_taint` / `find_tainted_sinks` answer instantly instead of walking the call graph per query. Computed against the symbol-graph edges when `--symbol-graph` is on (cross-function precision), else `chunk.calls`. Both backends serve byte-identical flows; default index byte-identical; air-gapped. |
236
- | `--ranker <rrf\|learned>` | `rrf` | Search ranker (D3). `rrf` (default) = the RRF + boost-ladder fusion. `learned` = an opt-in, zero-dependency post-fusion linear re-rank over the RRF score + symbol centrality + resolved/SCIP in-degree + git (best paired with `--symbol-graph`). **Measured ≈neutral on the benchmark** — RRF is the default; the learned ranker is provided for users to retrain on their own repo (`bench/train-ranker.mjs`). Default path byte-identical. |
237
- | `--learned-sparse` | off | Learn a corpus co-occurrence (PMI) association model (B3) at index time and serialize it, adding a **learned-sparse vocabulary-expansion** channel: NL queries expand into a weighted term set (literal terms + learned associates) fused via RRF, to close lexical-gap misses stemming can't. Zero new dependency, air-gapped, NL-asymmetric (symbolic lookups byte-identical), parity-safe. **Measured net-neutral on the strict benchmark** (does not beat lexical — like BM25F/AST); shipped opt-in for repos with genuine query/code vocabulary mismatch. Default index byte-identical. |
238
- | `--resolver <heuristic\|precise\|scip>` | `heuristic` | Resolver provider for the symbol graph. `precise` lifts provably-unambiguous edges (sole definition, or a type-pinned receiver) from `high` to a `resolved` tier. `scip` ingests a locally-generated SCIP index (`--scip-index`) and resolves edges by *real cross-file bindings* — promoting confirmed edges to `resolved` **and suppressing the wrong-target fan-out** an ambiguous same-name reference would otherwise emit, **and** adding a precise cross-file **`resolved`-tier** dimension to `find_references` (disambiguates same-named symbols, recovers references the name heuristics miss). Both power `impact_of_edit(precision: 'strict')`. Only meaningful with `--symbol-graph`. |
239
- | `--scip-index <path>` | — | Path to a SCIP index generated locally by your own toolchain (`scip-typescript`, `scip-python`, `scip-java`, `rust-analyzer --scip`), consumed by `--resolver scip`. Air-gapped (a local file read — sealed-compatible). Files the index doesn't cover fall back to heuristic confidence (never worse); coverage is printed at index time. Accepts the `.scip` protobuf or a SCIP-shaped JSON. |
240
- | `--sealed [strict\|local]` | off | **Sealed mode** — air-gapped *enforced and verifiable*. `strict` = zero network egress (lexical-only); `local` = loopback only (local LLM/embedders OK, nothing off-box). Fail-closed: refuses to start if any enabled feature would egress beyond the tier, and installs a deny-by-default runtime egress guard. Bare `--sealed` = `strict`. |
241
- | `--attest` | — | Print the deterministic sealed-mode manifest (tier, providers, egress posture) and exit — an auditor/CI artifact. Add `--sign-key <path>` to print a **cryptographically signed** envelope instead (Ed25519/RSA/EC via `node:crypto`; air-gapped). |
242
- | `--gen-attestation-key <prefix>` | — | Write an Ed25519 attestation keypair (`<prefix>.key` private, `<prefix>.pub` public) and exit. |
243
- | `--verify-attestation <file> --pub-key <path>` | — | Verify a signed attestation's signature and report whether it still matches the current effective config (policy drift → non-zero exit). |
244
- | `--no-git-signals` | (signals on) | Skip collecting local git churn/recency/co-change. |
245
- | `--git-rank-boost <0..1>` | 0 | Opt-in weight for git recency/churn in ranking (0 = ranking unchanged). |
246
- | `--llm-provider <ollama\|mlx>` | `ollama` | LLM backend for enrichment, reranking, and HyDE. `mlx` routes calls to a local `mlx_lm.server`. |
247
- | `--mlx-lm-host <url>` | `http://localhost:8080` | Endpoint for the `mlx_lm.server` when `--llm-provider mlx` is set. |
248
-
249
- ### All environment variables
250
-
251
- | Variable | Default | Effect |
252
- |----------|---------|--------|
253
- | `MCP_PROJECT_ROOT` | current directory | Repository root when `--repo` is not given. |
254
- | `OLLAMA_HOST` | `http://localhost:11434` | Ollama endpoint for embeddings/enrichment/rerank. |
255
- | `INDEXER_EMBEDDINGS` | `off` | `on` enables embeddings; `off` always wins over `--embeddings`. |
256
- | `EMBED_MODEL` | `nomic-embed-text` | Ollama embedding model (overrides config; overridden by `--embed-model`). |
257
- | `INDEXER_EMBED_PROVIDER` | `auto` | `auto` \| `ollama` \| `local` \| `mlx` \| `off`. |
258
- | `INDEXER_MLX_EMBED_MODEL` | `mlx-community/all-MiniLM-L6-v2-4bit` | Model loaded by the MLX embedder (overridden by `--mlx-embed-model`). |
259
- | `INDEXER_MLX_BATCH_SIZE` | 32 | Texts per batch sent to the MLX subprocess (raise on large unified memory). |
260
- | `INDEXER_STORAGE` | `auto` | `auto` \| `memory` \| `sqlite`. |
261
- | `ENRICH_MODEL` | (unset) | Naming a model enables enrichment and selects it. |
262
- | `RERANK_MODEL` | (unset) | Naming a model enables the (generative) reranker and selects it. |
263
- | `RERANK_PROVIDER` | `generative` | `generative` (LLM judge), `cross-encoder` (local air-gapped), or `late-interaction` (ColBERT-style MaxSim, needs embeddings); setting it enables reranking. |
264
- | `RERANK_CROSS_ENCODER_MODEL` | `Xenova/ms-marco-MiniLM-L-6-v2` | Cross-encoder model id (downloaded on first use). |
265
- | `INDEXER_GIT_SIGNALS` | (on) | Set to `off` to skip git-signal collection. |
266
- | `INDEXER_INTERPROCEDURAL` | (off) | `on` enables the index-time inter-procedural receiver-type fixpoint (`--interprocedural`). |
267
- | `INDEXER_SYMBOL_GRAPH` | (off) | `on` builds the persistent resolved symbol graph (`--symbol-graph`). |
268
- | `INDEXER_TAINT` | (off) | `on` precomputes + serializes the taint flow set at index time (`--taint`). |
269
- | `INDEXER_RANKER` | `rrf` | `learned` enables the opt-in post-fusion learned re-rank (`--ranker`). |
270
- | `INDEXER_LEARNED_SPARSE` | (off) | `on` learns + serializes the corpus association model for the learned-sparse expansion channel (`--learned-sparse`). |
271
- | `INDEXER_RESOLVER` | `heuristic` | `precise` enables the precise resolver; `scip` enables the SCIP cross-file resolver (`resolved` edge tier) for the symbol graph (`--resolver`). |
272
- | `INDEXER_SCIP_INDEX` | (unset) | Path to a locally-generated SCIP index for `--resolver scip` (`--scip-index`). |
273
- | `INDEXER_SEALED` | (off) | `strict` (or `on`) / `local` enables sealed mode (`--sealed`): fail-closed validation + a runtime egress guard. |
274
- | `INDEXER_GIT_RANK_BOOST` | 0 | Opt-in git recency/churn ranking weight (0..1). |
275
- | `INDEXER_LLM_PROVIDER` | `ollama` | LLM backend for enrichment, reranking, and HyDE: `ollama` or `mlx`. |
276
- | `INDEXER_MLX_LM_HOST` | `http://localhost:8080` | Endpoint for the `mlx_lm.server` when `INDEXER_LLM_PROVIDER=mlx`. |
277
- | `INDEXER_EMBED_CONCURRENCY` | 4 | Parallel embedding batches; lower to 1 for large models on modest hardware. |
278
- | `INDEXER_EMBED_TIMEOUT_MS` | 120000 | Per-batch embedding timeout; raise for very large models. |
279
-
280
- When embeddings are enabled, the provider is selected in this order, and every fallback is logged (never silent): Ollama with `EMBED_MODEL` if set and reachable → Ollama with `nomic-embed-text` → the in-process MiniLM model (optional `@huggingface/transformers`) → lexical-only with a warning.
281
-
282
- ### Embedder backends
283
-
284
- Set the backend with `--embed-provider` (or `INDEXER_EMBED_PROVIDER`). `mlx` is a native Python embedder that runs the Apple Metal GPU as a reused subprocess — faster in-process vectors than the bundled Xenova path, with no running Ollama daemon. The in-process (`local`) and `mlx` backends produce all-MiniLM-L6-v2-family **384-dim** vectors; the default `auto`→Ollama path uses `nomic-embed-text` (768-dim).
285
-
286
- | Backend | `--embed-provider` | Platform | Throughput¹ | Setup |
287
- |---------|--------------------|----------|-------------|-------|
288
- | Auto (Ollama → local → off) | `auto` (default) | any | varies | none |
289
- | In-process (Xenova) | `local` | any | ~18 ch/s | `npm i @huggingface/transformers` |
290
- | **Apple Metal (MLX)** | **`mlx`** | **macOS Apple Silicon** | **~42 ch/s** | `npm run embed:setup:mlx` |
291
- | Ollama daemon | `ollama` | any | ~14 ch/s² | `ollama serve` + `ollama pull` |
292
-
293
- ¹ **Throughput is hardware-dependent.** These figures are the median of 3 cold builds on an **Apple M2 Mac mini (24 GB)** indexing the `express-js` fixture (389 chunks): `node bench/cell.mjs express-js <E0|E0_MLX|O0>`. Expect different numbers on other chips; reproduce on your own machine. Throughput also varies with system load, model warm state, and corpus size — larger repos amortize cold-start better (e.g. Ollama/nomic reaches ~32 ch/s on the larger `gin` corpus).
294
-
295
- ² On Apple Silicon, Ollama already uses the Metal GPU internally (via llama.cpp). The `mlx` provider's advantage comes from a smaller model (all-MiniLM-L6-v2-4bit, 22M params, 384-dim) and no HTTP round-trip, not from GPU vs CPU.
296
-
297
- The index records which provider/model built it (in the `code-index.embeddings.bin.meta.json` sidecar) and queries with the same one, so vectors never get mixed across models. Switching providers or the MLX model between builds is detected and triggers a clean re-embed.
298
-
299
- **Code-specialized embedding models.** The default `nomic-embed-text` is a general-purpose embedder. Models trained on code — `qwen3-embedding`, `nomic-embed-code`, `jina-embeddings-v2-base-code` — embed code semantics more faithfully and can lift agent-style (natural-language) recall. Select one with `--embed-model <name>` (Ollama); it is stamped into the index meta and queried with the same model so vectors never get mixed. **Measure before adopting** — in our benchmarks `qwen3-embedding` lifted Go (gin) semantic rank-1 0.20 → 0.40 but was neutral on Python (django) and JavaScript (express), and it indexes substantially slower than `nomic-embed-text`. Run `npm run test:eval -- --embeddings` on your own repo to confirm the trade-off. We also evaluated *re-weighting the vector channel upward* for these stronger embedders and found **no measurable recall@5 gain** (it regressed rank-1/symbolic at higher weights) — the existing default weighting already extracts the benefit, so there is no separate "code weight" knob. `jina-embeddings-v2-base-code` remains opt-in only: it helped some fixtures, regressed others, and indexes far slower, so it is never a default.
300
-
301
- **macOS Apple Silicon (recommended):**
302
-
303
- ```bash
304
- npm run embed:setup:mlx # one-time: creates embedders/venv-mlx + installs MLX
305
- npx idx-index --repo . --embeddings --embed-provider mlx
216
+ 1. **Parse.** tree-sitter (WebAssembly) with one query per language extracts definitions,
217
+ references (calls, instantiations, types, inheritance, decorators, values, member reads),
218
+ imports and type evidence.
219
+ 2. **Infer types locally.** Receiver types come from annotations, initialisers, `new`/casts,
220
+ return types (declared or inferred from `return` statements), fields and constructor-injected
221
+ dependencies, collections (`xs[i]`, `for x of xs`, `xs.forEach(x => …)`, `list.get(0)`),
222
+ optionals and wrappers (`Foo | undefined`, `Optional[Foo]`, `Promise<Foo>`, `Arc<Mutex<Foo>>`),
223
+ across closures and access chains.
224
+ 3. **Resolve.** Each reference is bound by a ladder: lexical scope → enclosing type and its
225
+ bases → same file → explicit import (barrels, tsconfig paths, workspaces, Go modules, Python
226
+ packages, Rust crates, C includes) → package → global, and labelled with a confidence.
227
+ 4. **Store.** SQLite (WAL) with per-file ownership of facts, stable symbol ids across edits,
228
+ FTS5 indexes over symbols and files. The index rebuilds itself when graph-indexer is upgraded.
229
+ 5. **Stay fresh.** A file watcher plus periodic stat sweeps re-index changed files before any
230
+ answer, and files are re-validated right before their lines are rendered.
231
+ 6. **Rank.** Search fuses exact/qualified name matches, BM25F over name, signature, doc, path and
232
+ body, file-level relevance and a small concept thesaurus, then applies interpretable priors
233
+ and PageRank centrality. Weights were tuned on a tuning split only.
234
+
235
+ [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) describes each part in detail.
236
+
237
+ ## Languages
238
+
239
+ JavaScript, TypeScript/TSX, Python, Go, Java, Kotlin, Scala, C#, Rust, C, C++, Ruby, PHP, Bash
240
+ and CSS/SCSS/Less. Reference binding and type inference are deepest for TypeScript/JavaScript,
241
+ Python, Go, Java, C# and Rust.
242
+
243
+ ## Command line
244
+
245
+ The CLI runs the same tools, which is handy for scripts and for checking what an agent sees:
246
+
247
+ ```text
248
+ graph-indexer init [--repo DIR] [--agents …] [--all] [--hooks] [--local] [--dry-run] [--no-instructions] [--no-helper]
249
+ graph-indexer serve [--repo DIR] MCP server on stdio
250
+ graph-indexer index [--repo DIR] build or update the index
251
+ graph-indexer status [--repo DIR]
252
+ graph-indexer search <query> [--path P] [--kind K] [--limit N]
253
+ graph-indexer symbol <name> [--no-code]
254
+ graph-indexer refs <name> [--kind call|type|inherit|new|value|decorator] [--no-tests]
255
+ graph-indexer callgraph <name> [--direction callers|callees|both] [--depth N]
256
+ graph-indexer impact [--symbols a,b] [--files x,y] [--diff] [--depth N]
257
+ graph-indexer outline [path] [--focus TEXT] [--max-tokens N]
306
258
  ```
307
259
 
308
- `embed:setup:mlx` provisions a **dedicated Python virtualenv** under `embedders/venv-mlx` (so MLX's deps never touch your system Python) and is idempotent — re-running it is a no-op once ready. The interactive setup (`npx graph-indexer`) offers MLX directly and can run this step for you. The first index downloads the model (~90 MB) into the Hugging Face cache; later builds reuse it.
309
-
310
- Choose a different MLX model with `--mlx-embed-model <id>` (or `INDEXER_MLX_EMBED_MODEL`); it must be an `mlx_embeddings`-compatible sentence model, and the default `mlx-community/all-MiniLM-L6-v2-4bit` is the proven option. Tune batch size with `INDEXER_MLX_BATCH_SIZE` (default 32; lower to 16 on 8 GB machines, raise to 64 on M-series Max/Ultra). `mlx` is macOS-only; requesting it elsewhere fails fast with the alternative to use.
311
-
312
- ## Benchmarks
313
-
314
- Every number here is produced by the eval harness on cold, isolated builds — never hand-edited — with **strict scoring** (exact symbol match, no file-path fallback) against a **held-out** split (~20–25% per language, never used to tune). Reproduce it all with `npm run test:eval`.
260
+ The index lives in `<repo>/.graph-indexer/` (git-ignored). It contains excerpts of your code, so
261
+ treat it like the repository itself.
315
262
 
316
- What the measurements say, across 18 real-world fixtures spanning every supported language:
263
+ ## Privacy and security
317
264
 
318
- - **The zero-dependency lexical default is the right starting point.** It wins the held-out metric outright on 2 fixtures, and 6 more need only a cheap in-process embedder — so **8 of 18 run with no Ollama and no network**. Most repos never need more.
319
- - **Symbolic lookups are strong** (mean rank-1 ≈ 0.70). Behavioural, natural-language queries are harder (≈ 0.30 overall (held-out semantic rank-1 0.40 with stemming)) — that's where the optional embedding and reranker channels earn their keep.
320
- - **Optional features are measured, not assumed.** Dense embeddings lift recall on larger repos; the LLM reranker is language- and repo-dependent (it lifts 8 fixtures but taxes JavaScript and Spring); enrichment helps only where proven (just rust). Setup surfaces each trade-off before you enable it.
321
- - **Backend parity:** the in-memory and SQLite backends return byte-identical top-5 on all 18 fixtures (gated by `test/sqlite.mjs`).
265
+ Everything runs locally. graph-indexer makes no network calls, runs no code from the repository
266
+ it indexes (the only subprocesses are read-only git commands: `rev-parse`, `ls-files`, `diff`
267
+ and `log`), talks
268
+ to the agent over stdio only, and never reads outside the repository: tool paths are resolved
269
+ through the index, and symlinks pointing outside the repository are ignored. See
270
+ [SECURITY.md](SECURITY.md).
322
271
 
323
- Per-fixture best configs, 3× spreads, and copy-paste enable flags live in **[docs/benchmarks/BENCH_PER_FIXTURE.md](docs/benchmarks/BENCH_PER_FIXTURE.md)**.
272
+ ## Upgrading from 2.x
324
273
 
325
- ### Structural coverage by language
274
+ Version 3 is a rewrite; see the [changelog](CHANGELOG.md). In short: the tools changed
275
+ (`read_code`, `search_text`, `search_code`, `find_references`, `call_graph`, `change_impact`,
276
+ `check_changes`, `outline`), the
277
+ embedding, Ollama, reranker and daemon options are gone (search is lexical plus graph and needs
278
+ no model), grammars are bundled instead of installed on first use, and Node.js 22.5+ is required.
279
+ Re-run `npx graph-indexer init` to update agent configuration; old `.graph-indexer/` contents are
280
+ replaced automatically.
326
281
 
327
- `get_call_graph` and `find_references` are richest for typed languages. Every verdict is confirmed by invocation on the real index, not by reading field counts:
282
+ ## License
328
283
 
329
- | Capability | Strong | Limited |
330
- |---|---|---|
331
- | **Call graph** (callers/callees) | resolves on every supported language | Java/Spring is class-granular; SCSS effectively none |
332
- | **Caller precision** | receiver-aware: TS/JS, Python, C#, Swift, PHP, Java (call-site level — but its call graph is class-granular) | name-only: Go, Rust, Kotlin, Ruby, Bash, C |
333
- | **Typed `find_references`** | precise: TS/JS, Python · field-precise: C# | heuristic: Java/PHP/Kotlin/Swift/Rust/Go/C · empty: Ruby, Bash, SCSS, dynamic JS/Python |
334
-
335
- AST chunking and lexical search cover **every** supported language; only the typed cross-reference channel narrows for dynamic ones.
336
-
337
- **Multi-hop receivers (`--interprocedural`).** Caller precision relies on resolving a call's receiver type. Intra-procedurally, graph-indexer already resolves `const r = makeRepo(); r.save()` one hop (reading `makeRepo`'s recorded return type). The opt-in `--interprocedural` flag adds an index-time fixpoint that propagates return types *along factory call chains*, so a receiver whose type comes from a multi-hop or unannotated factory (`makeRepoIndirect()` → `makeRepo()` → `new OrderRepo()`) still resolves to its concrete class — promoting those callers from *name-only* to *high-confidence* in `get_call_graph` / `find_references`. It is air-gapped and deterministic, the result is serialized into the index (both backends identical), and it leaves search ranking and the default index byte-identical. Note: the watch daemon updates one file at a time and does not re-run the whole-program pass, so cross-file chains refresh on the next full `idx-index` (degrading to single-hop until then, never worse).
338
-
339
- ### Where it's weakest (honestly)
340
-
341
- - Behavioural, natural-language recall on the default path (rank-1 ≈ 0.30 overall; held-out semantic rank-1 0.40 — the embedding/reranker channel closes it further) — closing it needs the embedding/reranker channel, not lexical tuning.
342
- - The hardest fixtures stay below 0.65 held-out success@5 even with the full stack (rails 0.50, rust 0.61).
343
- - Java/Spring `get_call_graph` reports at class granularity (the god-class split only fires ≥200 lines); SCSS has no meaningful call graph.
344
- - LLM enrichment without reranking regresses precision — enable it only paired with `--rerank`.
345
-
346
- ## Reproducing the benchmarks
347
-
348
- ```bash
349
- npm run test:all # full unit + integration suite
350
- npm run test:setup # index the benchmark fixtures
351
- npm run test:eval # default lexical path — prints the HELD-OUT block
352
- npm run test:eval -- --suite css # any single fixture (18 authored suites)
353
- npm run test:eval -- --embeddings # hybrid eval (requires Ollama)
354
- npm run test:eval -- --verbose # per-query breakdown incl. file-rank column
355
- node test/sqlite.mjs # backend-parity gate (memory ↔ SQLite top-5)
356
- node bench/cell.mjs <fixture> L1 # cold rebuild + score one fixture
357
- ```
358
-
359
- The harness is [test/evaluate.mjs](test/evaluate.mjs); deeper reproduction notes live in [docs/benchmarks/BENCH_BASELINE.md](docs/benchmarks/BENCH_BASELINE.md) and [docs/benchmarks/BENCH_FULL_SUITE.md](docs/benchmarks/BENCH_FULL_SUITE.md).
360
-
361
- ---
362
-
363
- <div align="center">
364
- <sub>MIT licensed · runs entirely on your machine · see <a href="SECURITY.md">SECURITY.md</a> and <a href="TRADEMARK.md">TRADEMARK.md</a></sub>
365
- </div>
284
+ [MIT](LICENSE). "graph-indexer" and the logo are trademarks of MaquinaTech; see
285
+ [TRADEMARK.md](TRADEMARK.md).