graph-indexer 2.1.1 → 3.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +312 -1
- package/README.md +236 -312
- package/SECURITY.md +53 -58
- package/bin/graph-indexer.mjs +251 -0
- package/integrations/claude-code/.claude-plugin/plugin.json +9 -0
- package/integrations/claude-code/.mcp.json +8 -0
- package/integrations/claude-code/agents/code-structure.md +22 -0
- package/integrations/claude-code/hooks/gi-hook.mjs +103 -0
- package/integrations/claude-code/hooks/hooks.json +16 -0
- package/integrations/opencode/graph-indexer.js +129 -0
- package/package.json +32 -109
- package/src/cli/helper.mjs +67 -0
- package/src/cli/hook.mjs +366 -0
- package/src/cli/init.mjs +272 -0
- package/src/cli/resident.mjs +185 -0
- package/src/index/discover.mjs +128 -0
- package/src/index/fingerprint.mjs +30 -0
- package/src/index/graph.mjs +61 -0
- package/src/index/indexer.mjs +476 -0
- package/src/index/modules.mjs +332 -0
- package/src/index/resolver.mjs +961 -0
- package/src/mcp/render.mjs +60 -0
- package/src/mcp/server.mjs +147 -0
- package/src/mcp/tools.mjs +974 -0
- package/src/parse/arity.mjs +153 -0
- package/src/parse/extract.mjs +1038 -0
- package/src/parse/lang/csharp.mjs +106 -0
- package/src/parse/lang/ecma.mjs +438 -0
- package/src/parse/lang/go.mjs +194 -0
- package/src/parse/lang/jvm.mjs +118 -0
- package/src/parse/lang/kotlin.mjs +97 -0
- package/src/parse/lang/misc.mjs +115 -0
- package/src/parse/lang/native.mjs +126 -0
- package/src/parse/lang/python.mjs +153 -0
- package/src/parse/lang/rust.mjs +157 -0
- package/src/parse/lang/scripting.mjs +219 -0
- package/src/parse/languages.mjs +48 -0
- package/src/parse/runtime.mjs +56 -0
- package/src/query/check.mjs +345 -0
- package/src/query/facts.mjs +323 -0
- package/src/query/intel.mjs +683 -0
- package/src/query/read.mjs +81 -0
- package/src/query/testcmd.mjs +130 -0
- package/src/query/textsearch.mjs +178 -0
- package/src/search/search.mjs +210 -0
- package/src/search/tokenize.mjs +188 -0
- package/src/store/db.mjs +169 -0
- package/src/util/git.mjs +30 -0
- package/src/util/paths.mjs +63 -0
- package/src/util/regex.mjs +2 -0
- package/vendor/grammars/bash.wasm.gz +0 -0
- package/vendor/grammars/c.wasm.gz +0 -0
- package/vendor/grammars/c_sharp.wasm.gz +0 -0
- package/vendor/grammars/cpp.wasm.gz +0 -0
- package/vendor/grammars/css.wasm.gz +0 -0
- package/vendor/grammars/go.wasm.gz +0 -0
- package/vendor/grammars/java.wasm.gz +0 -0
- package/vendor/grammars/javascript.wasm.gz +0 -0
- package/vendor/grammars/kotlin.wasm.gz +0 -0
- package/vendor/grammars/manifest.json +148 -0
- package/vendor/grammars/php.wasm.gz +0 -0
- package/vendor/grammars/python.wasm.gz +0 -0
- package/vendor/grammars/ruby.wasm.gz +0 -0
- package/vendor/grammars/rust.wasm.gz +0 -0
- package/vendor/grammars/scala.wasm.gz +0 -0
- package/vendor/grammars/tsx.wasm.gz +0 -0
- package/vendor/grammars/typescript.wasm.gz +0 -0
- package/vendor/web-tree-sitter/web-tree-sitter.js +3980 -0
- package/vendor/web-tree-sitter/web-tree-sitter.wasm +0 -0
- package/cli-ui.mjs +0 -78
- package/colbert.mjs +0 -168
- package/config.mjs +0 -494
- package/daemon-ctl.mjs +0 -175
- package/daemon-lock.mjs +0 -69
- package/embedders/python/mlx_embed_server.py +0 -61
- package/embedders/python/requirements-mlx.txt +0 -5
- package/embedders/setup-mlx.mjs +0 -110
- package/embeddings.mjs +0 -571
- package/engine/binary.mjs +0 -437
- package/engine/memory.mjs +0 -1003
- package/engine/sqlite.mjs +0 -1136
- package/enrichment.mjs +0 -521
- package/git-signals.mjs +0 -248
- package/indexer.mjs +0 -546
- package/init.mjs +0 -1809
- package/layout.mjs +0 -123
- package/mcp/centrality.mjs +0 -117
- package/mcp/format.mjs +0 -61
- package/mcp/resolver.mjs +0 -116
- package/mcp/symbolgraph.mjs +0 -137
- package/mcp/taint.mjs +0 -225
- package/mcp/tools.mjs +0 -1582
- package/mcp/topology.mjs +0 -676
- package/mcp-server.mjs +0 -197
- package/parse/extractor.mjs +0 -628
- package/parse/imports.mjs +0 -365
- package/parse/interprocedural.mjs +0 -134
- package/parse/languages.mjs +0 -253
- package/parse/metadata.mjs +0 -701
- package/parse/routes.mjs +0 -327
- package/parse/scip.mjs +0 -391
- package/parse/taint-patterns.mjs +0 -119
- package/prompts/CORE.md +0 -91
- package/prompts/DOMAIN_TEMPLATE.md +0 -46
- package/prompts/INTEGRATION.md +0 -123
- package/prompts/frameworks/ANDROID.md +0 -17
- package/prompts/frameworks/ASPNET_CORE.md +0 -19
- package/prompts/frameworks/FASTAPI_DJANGO.md +0 -20
- package/prompts/frameworks/LARAVEL_SYMFONY.md +0 -22
- package/prompts/frameworks/NODE_EXPRESS_NESTJS.md +0 -19
- package/prompts/frameworks/RAILS.md +0 -20
- package/prompts/frameworks/REACT.md +0 -20
- package/prompts/frameworks/SPRING_BOOT.md +0 -20
- package/prompts/languages/BASH.md +0 -22
- package/prompts/languages/C.md +0 -21
- package/prompts/languages/CSHARP.md +0 -24
- package/prompts/languages/CSS_SCSS.md +0 -22
- package/prompts/languages/GO.md +0 -24
- package/prompts/languages/JAVA.md +0 -23
- package/prompts/languages/JAVASCRIPT_TYPESCRIPT.md +0 -24
- package/prompts/languages/KOTLIN.md +0 -24
- package/prompts/languages/PHP.md +0 -23
- package/prompts/languages/PYTHON.md +0 -23
- package/prompts/languages/RUBY.md +0 -23
- package/prompts/languages/RUST.md +0 -21
- package/prompts/languages/SWIFT.md +0 -21
- package/seal.mjs +0 -400
- package/search-core.mjs +0 -777
- package/search-sparse.mjs +0 -183
- package/storage.mjs +0 -73
- package/watch-daemon.mjs +0 -289
package/README.md
CHANGED
|
@@ -4,12 +4,12 @@
|
|
|
4
4
|
<h1>Graph Indexer</h1>
|
|
5
5
|
|
|
6
6
|
<p>
|
|
7
|
-
<em>A
|
|
7
|
+
<em>A live code graph and search engine for AI coding agents, served locally over MCP.</em>
|
|
8
8
|
</p>
|
|
9
9
|
|
|
10
10
|
<a href="https://www.npmjs.com/package/graph-indexer"><img src="https://img.shields.io/npm/v/graph-indexer?color=007acc&style=for-the-badge" alt="npm version"></a>
|
|
11
11
|
<a href="https://www.npmjs.com/package/graph-indexer"><img src="https://img.shields.io/npm/dt/graph-indexer?color=4caf50&style=for-the-badge" alt="NPM Downloads"></a>
|
|
12
|
-
<a href="https://nodejs.org"><img src="https://img.shields.io/badge/Node.js
|
|
12
|
+
<a href="https://nodejs.org"><img src="https://img.shields.io/badge/Node.js-22.23.4%2B%20%7C%2024.22.1%2B%20%7C%2026.12.1%2B-brightgreen?style=for-the-badge&logo=nodedotjs" alt="Node.js 22.23.4+, 24.22.1+ or 26.12.1+"></a>
|
|
13
13
|
<a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-yellow.svg?style=for-the-badge" alt="License: MIT"></a>
|
|
14
14
|
</div>
|
|
15
15
|
|
|
@@ -17,349 +17,273 @@
|
|
|
17
17
|
|
|
18
18
|
## What it does
|
|
19
19
|
|
|
20
|
-
Graph Indexer
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
- **
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
20
|
+
Graph Indexer parses your repository with tree-sitter and keeps a structural index of it:
|
|
21
|
+
every definition, every reference bound to the definition it actually points at, the call
|
|
22
|
+
graph, inheritance, imports, and which tests exercise what. Coding agents (Claude Code,
|
|
23
|
+
Cursor, VS Code Copilot, Codex, Gemini CLI, OpenCode, Kilo Code, Junie, Zed, Devin, …) query it
|
|
24
|
+
through [Model Context Protocol](https://modelcontextprotocol.io) tools, the CLI and hooks
|
|
25
|
+
instead of reading whole files and grepping.
|
|
26
|
+
|
|
27
|
+
- **Answers are exact, not guesses.** `find_references` binds each use through scopes,
|
|
28
|
+
imports and inferred receiver types, so it lists the callers of *this* `get` and not of every
|
|
29
|
+
method named `get`. Measured against the TypeScript compiler it finds 89% of the real
|
|
30
|
+
references at 98% precision; grep finds all of them buried among seven times as many false hits.
|
|
31
|
+
- **Answers are never stale.** The index re-syncs with the files on disk before every answer,
|
|
32
|
+
including edits the agent made a second ago, so line numbers and call graphs are current.
|
|
33
|
+
- **Answers are small.** A tool reply is typically 400–1,500 tokens: the right function with
|
|
34
|
+
line numbers, not a 2,000-line file.
|
|
35
|
+
- **Nothing to set up.** Zero dependencies, no native builds, no model downloads, no network,
|
|
36
|
+
no service to install. The tree-sitter runtime and 16 grammars ship as WebAssembly inside the
|
|
37
|
+
package. The MCP server keeps the index open while the agent runs; with hooks and no server, a
|
|
38
|
+
resident process started on demand does the same so hooks and CLI queries answer in about
|
|
39
|
+
0.1 s, and exits after 30 idle minutes (`GRAPH_INDEXER_RESIDENT=0` turns it off).
|
|
36
40
|
|
|
37
41
|
## Quick start
|
|
38
42
|
|
|
39
|
-
|
|
40
|
-
|
|
43
|
+
Requires **Node.js 22.23.4+, 24.22.1+ or 26.12.1+** (the built-in `node:sqlite`, and the October 2026
|
|
44
|
+
Node.js security releases, which fix CVE-2026-111497).
|
|
41
45
|
|
|
42
46
|
```bash
|
|
43
|
-
|
|
47
|
+
cd your-repo
|
|
48
|
+
npx -y --prefix / graph-indexer@3 init # wires the MCP server into the agents it detects
|
|
49
|
+
npx -y --prefix / graph-indexer@3 index # optional: build the index now (the server also does it on start)
|
|
44
50
|
```
|
|
45
51
|
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
52
|
+
`--prefix /` makes npx run the published package even in a repository that ships its own
|
|
53
|
+
`node_modules/graph-indexer`.
|
|
54
|
+
|
|
55
|
+
`init` writes project-level MCP configuration for the agents it finds (`.mcp.json` for Claude
|
|
56
|
+
Code, `.cursor/mcp.json`, `.vscode/mcp.json`, `.gemini/settings.json`, `opencode.json` for
|
|
57
|
+
OpenCode and Kilo Code, `.junie/mcp/mcp.json`, `.zed/settings.json`; it prints the Codex TOML
|
|
58
|
+
and Devin's server entry), merges instead of overwriting other servers, adds `.graph-indexer/`
|
|
59
|
+
to `.gitignore`, and adds a short block to `CLAUDE.md` / `AGENTS.md` on how to look code up.
|
|
60
|
+
For Claude Code it also writes a structural helper, `.claude/agents/code-structure.md`: a
|
|
61
|
+
read-only sub-agent on a small model that answers call-site, caller, subclass and impact
|
|
62
|
+
questions from the index, so the main agent can hand them off and get back only the list (the
|
|
63
|
+
Claude Code plugin ships it too). `--hooks` adds the Claude Code hooks to `.claude/settings.json`,
|
|
64
|
+
which Devin, Copilot CLI and Cursor also run, and the OpenCode / Kilo Code plugin that appends the
|
|
65
|
+
same context to their tool output. Flags: `--agents claude,cursor,vscode,gemini,codex,opencode,junie,zed,devin`,
|
|
66
|
+
`--all`, `--hooks`, `--local` (use this checkout instead of `npx`), `--dry-run`,
|
|
67
|
+
`--no-instructions`, `--no-helper`.
|
|
53
68
|
|
|
54
69
|
<details>
|
|
55
|
-
<summary>Manual
|
|
70
|
+
<summary>Manual configuration</summary>
|
|
56
71
|
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
```bash
|
|
60
|
-
claude mcp add graph-indexer -- npx -y -p graph-indexer idx-mcp --repo /path/to/your/repo
|
|
61
|
-
```
|
|
62
|
-
|
|
63
|
-
#### Cursor — `.cursor/mcp.json`
|
|
72
|
+
Any MCP client can run the server over stdio:
|
|
64
73
|
|
|
65
74
|
```json
|
|
66
75
|
{
|
|
67
76
|
"mcpServers": {
|
|
68
|
-
"graph-indexer": {
|
|
69
|
-
"command": "npx",
|
|
70
|
-
"args": ["-y", "-p", "graph-indexer", "idx-mcp", "--repo", "/path/to/your/repo"]
|
|
71
|
-
}
|
|
72
|
-
}
|
|
73
|
-
}
|
|
74
|
-
```
|
|
75
|
-
|
|
76
|
-
#### VS Code — `.vscode/mcp.json`
|
|
77
|
-
|
|
78
|
-
```json
|
|
79
|
-
{
|
|
80
|
-
"servers": {
|
|
81
|
-
"graph-indexer": {
|
|
82
|
-
"command": "npx",
|
|
83
|
-
"args": ["-y", "-p", "graph-indexer", "idx-mcp", "--repo", "/path/to/your/repo"]
|
|
84
|
-
}
|
|
77
|
+
"graph-indexer": { "command": "npx", "args": ["-y", "--prefix", "/", "graph-indexer@3", "serve", "--repo", "/path/to/repo"] }
|
|
85
78
|
}
|
|
86
79
|
}
|
|
87
80
|
```
|
|
88
81
|
|
|
89
|
-
|
|
82
|
+
VS Code uses `"servers"` instead of `"mcpServers"` and `"type": "stdio"`. Codex CLI
|
|
83
|
+
(`~/.codex/config.toml`):
|
|
90
84
|
|
|
91
|
-
```
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
"command": "npx",
|
|
96
|
-
"args": ["-y", "-p", "graph-indexer", "idx-mcp", "--repo", "/path/to/your/repo"]
|
|
97
|
-
}
|
|
98
|
-
}
|
|
99
|
-
}
|
|
85
|
+
```toml
|
|
86
|
+
[mcp_servers.graph-indexer]
|
|
87
|
+
command = "npx"
|
|
88
|
+
args = ["-y", "--prefix", "/", "graph-indexer@3", "serve", "--repo", "/path/to/repo"]
|
|
100
89
|
```
|
|
101
90
|
|
|
102
|
-
|
|
103
|
-
|
|
91
|
+
Without `--repo` the server indexes the git repository containing its working directory
|
|
92
|
+
(`GRAPH_INDEXER_REPO` also works).
|
|
104
93
|
</details>
|
|
105
94
|
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
```jsonc
|
|
109
|
-
[
|
|
110
|
-
{
|
|
111
|
-
"score": 8.41,
|
|
112
|
-
"chunk": {
|
|
113
|
-
"id": "src/middleware/rateLimit.ts:14",
|
|
114
|
-
"file_path": "src/middleware/rateLimit.ts",
|
|
115
|
-
"name": "rateLimiter",
|
|
116
|
-
"node_type": "function_declaration",
|
|
117
|
-
"start_line": 14, "end_line": 47,
|
|
118
|
-
"calls": ["tokenBucket", "getClientKey"],
|
|
119
|
-
"class_context": ""
|
|
120
|
-
}
|
|
121
|
-
}
|
|
122
|
-
]
|
|
123
|
-
```
|
|
124
|
-
|
|
125
|
-
The agent can then call `get_call_graph("rateLimiter")` to see what calls it (the blast radius) before changing it.
|
|
126
|
-
|
|
127
|
-
## Guided setup
|
|
95
|
+
## The tools
|
|
128
96
|
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
|
134
|
-
|
|
135
|
-
|
|
|
136
|
-
|
|
|
137
|
-
|
|
|
138
|
-
|
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
97
|
+
| Tool | Use it to | Typical reply |
|
|
98
|
+
|---|---|---|
|
|
99
|
+
| `read_code` | read symbols, line ranges or files — several per call — without searching for what they use | code with line numbers, then where each name it uses is defined (file:line and signature); who uses a symbol and which tests reference it; a long file's outline |
|
|
100
|
+
| `search_text` | grep any text across the repository (code, config, docs) | grep lines with the enclosing definition and, for identifiers, the definition each match refers to; where the definition is when the search missed it |
|
|
101
|
+
| `search_code` | find where a behaviour or identifier lives (natural language or names) | ranked symbols with location, signature, doc line and matching lines |
|
|
102
|
+
| `find_references` | see every use before renaming or changing a signature | uses grouped by file, enclosing function, confidence, calls through base types marked |
|
|
103
|
+
| `call_graph` | trace callers and callees across files | a tree with locations and confidence |
|
|
104
|
+
| `change_impact` | get the blast radius of a change, or of your uncommitted diff | transitive dependents by distance, tests to run, co-changed files, risk |
|
|
105
|
+
| `check_changes` | verify uncommitted edits without building | syntax errors introduced, calls that no longer fit, removed names still in use, the tests to run |
|
|
106
|
+
| `outline` | see a file's structure, or a ranked map of a directory or the repo | signatures and line ranges within a token budget |
|
|
107
|
+
|
|
108
|
+
All tools are read-only. Replies are plain text, capped, best result first, and end with a hint
|
|
109
|
+
for the next step. Example, on [gin](https://github.com/gin-gonic/gin):
|
|
110
|
+
|
|
111
|
+
```text
|
|
112
|
+
> search_code("where are trusted proxies checked")
|
|
113
|
+
3 results for "where are trusted proxies checked" (of 55 candidates):
|
|
114
|
+
1. Engine.isTrustedProxy — method · gin.go:445-455
|
|
115
|
+
func (engine *Engine) isTrustedProxy(ip net.IP) bool
|
|
116
|
+
isTrustedProxy will check whether the IP address is included in the trusted list according to Engine.trustedCIDRs
|
|
117
|
+
446: if engine.trustedCIDRs == nil {
|
|
118
|
+
449: for _, cidr := range engine.trustedCIDRs {
|
|
119
|
+
2. Engine.isUnsafeTrustedProxies — method · gin.go:433-435
|
|
120
|
+
…
|
|
121
|
+
|
|
122
|
+
> change_impact(symbols: ["Engine.isTrustedProxy"], depth: 2)
|
|
123
|
+
Impact of changing Engine.isTrustedProxy
|
|
124
|
+
dependents: 9 symbols in 3 files (non-test)
|
|
125
|
+
distance 1 (3):
|
|
126
|
+
Context.ClientIP context.go:771-806
|
|
127
|
+
Engine.isUnsafeTrustedProxies gin.go:433-435
|
|
128
|
+
Engine.validateHeader gin.go:458-477
|
|
129
|
+
distance 2 (6):
|
|
130
|
+
LoggerWithConfig logger.go:203-270 [high]
|
|
131
|
+
Engine.Run gin.go:376-388 [high]
|
|
132
|
+
…
|
|
133
|
+
tests to run (1 file):
|
|
134
|
+
context_test.go: TestRemoteIPFail, TestContextClientIP
|
|
135
|
+
often changed together (git): context.go (14/42), context_test.go (10/42), …
|
|
136
|
+
risk: medium (9 dependents, 1 test files)
|
|
137
|
+
```
|
|
159
138
|
|
|
160
|
-
|
|
139
|
+
Every reference carries a confidence: **exact/high** means it was bound through a scope,
|
|
140
|
+
import or inferred receiver type; **likely/possible** means only the name matched. Footers
|
|
141
|
+
report same-name call sites that could not be bound, so an agent knows when to double-check
|
|
142
|
+
with grep.
|
|
143
|
+
|
|
144
|
+
## Measured results
|
|
145
|
+
|
|
146
|
+
All numbers come from scripts in [`bench/`](bench) on public repositories pinned to exact
|
|
147
|
+
commits; [docs/BENCHMARKS.md](docs/BENCHMARKS.md) has the methodology and how to reproduce them.
|
|
148
|
+
|
|
149
|
+
**References vs the TypeScript compiler.** For 400 randomly sampled functions, methods, classes
|
|
150
|
+
and interfaces of nestjs, the references each system reports are compared with the compiler's
|
|
151
|
+
own `findReferences` (import lines and declarations excluded):
|
|
152
|
+
|
|
153
|
+
| | precision | recall | F1 |
|
|
154
|
+
|---|---|---|---|
|
|
155
|
+
| graph-indexer `find_references` | **0.997** | **0.962** | **0.979** |
|
|
156
|
+
| graph-indexer 2.x `find_references` | 0.176 | 0.886 | 0.294 |
|
|
157
|
+
| same-name references (no binding) | 0.252 | 0.975 | 0.400 |
|
|
158
|
+
| grep for the name | 0.128 | 0.993 | 0.227 |
|
|
159
|
+
|
|
160
|
+
*Dispatch semantics: calls through a base class or interface count, calls to sibling overrides
|
|
161
|
+
do not. Against the compiler's full rename set (sibling overrides included) recall is 0.925. The
|
|
162
|
+
fixture has no `node_modules`, so library-typed values are invisible to the compiler: precision is
|
|
163
|
+
a lower bound.* The same measurement against the Go type checker (gin, caddy, nats-server) gives
|
|
164
|
+
precision 0.992–1.000 and recall 0.972–0.981, and against the Java compiler (spring-petclinic,
|
|
165
|
+
jsoup, Commons Collections) 0.955–0.994 and 0.861–0.989; grep's precision there is 0.02–0.61.
|
|
166
|
+
In the server of Twenty, an open-source CRM monorepo (28,512 files), precision is 0.999 and
|
|
167
|
+
recall 0.924; grep's precision is 0.009 and 2.x's 0.052.
|
|
168
|
+
|
|
169
|
+
**Localization from real commits.** 169 focused commits of five repositories are replayed: the
|
|
170
|
+
commit subject is the query, the files and functions it changed are the answer, and the index is
|
|
171
|
+
rolled back to the parent commit first.
|
|
172
|
+
|
|
173
|
+
| | file Acc@1 | file Acc@5 | function Acc@5 | function MRR@10 |
|
|
174
|
+
|---|---|---|---|---|
|
|
175
|
+
| graph-indexer `search_code` | **0.544** | **0.805** | **0.528** | **0.432** |
|
|
176
|
+
| grep, files ranked by idf-weighted hits | 0.485 | 0.769 | 0.514 | 0.374 |
|
|
177
|
+
| BM25 over the same index | 0.373 | 0.722 | 0.401 | 0.290 |
|
|
178
|
+
|
|
179
|
+
**Symbol search.** 377 hand-written queries over nine repositories in eight languages, scored
|
|
180
|
+
strictly at symbol level: rank-1 0.703 and MRR 0.760, against 0.552 and 0.646 for graph-indexer
|
|
181
|
+
2.x on the same queries. On the 169 held-out queries (never used for tuning) MRR is 0.773 vs 0.632.
|
|
182
|
+
|
|
183
|
+
**Against 2.x.** In real Claude Code sessions on nestjs and Twenty (40 tasks, 120 runs), 3.0
|
|
184
|
+
cost 0.43 of 2.x on code questions and 0.46 on refactors, with a quarter to a third of the
|
|
185
|
+
tokens spent looking code up; 2.x cost about twice as much as no index, as its tools and prompt
|
|
186
|
+
add 11,300 tokens to every call (3.0: 1,200). Against no index, 3.0 halves those lookup tokens at
|
|
187
|
+
0.84–0.86 of the cost. Every benchmark against 2.x is in
|
|
188
|
+
[docs/BENCHMARKS.md](docs/BENCHMARKS.md#5-against-graph-indexer-2x).
|
|
189
|
+
|
|
190
|
+
**Agents.** The same model gets the same task with built-in tools only, with graph-indexer added,
|
|
191
|
+
or with graph-indexer instead of grep, and an oracle it never sees grades the result: the
|
|
192
|
+
TypeScript compiler for code questions and multi-site refactors, the hidden tests of the real fix
|
|
193
|
+
for issues reported after the model's training cutoff. In the fourth round (19 new tasks, 122
|
|
194
|
+
runs, all solved), graph-indexer as `init` installs it answered code questions at 0.44 of the
|
|
195
|
+
cost of grep alone (95% CI 0.32–0.67) and 0.37 of the time, with half the model's reasoning. A
|
|
196
|
+
smaller model, at half the price per token, answered the same questions exactly with graph-indexer
|
|
197
|
+
(7 of 7, against 5 of 7 with grep alone) at 0.26 of the cost (0.18–0.50) and 0.32 of the time — a
|
|
198
|
+
tenth of what the larger model spent with grep — and did multi-site refactors at 0.69
|
|
199
|
+
(0.56–0.87). Handed off the way a main agent delegates them, 24 new code questions were answered
|
|
200
|
+
by the structural helper `init` installs as exactly as by a general-purpose sub-agent with grep
|
|
201
|
+
(20 of 24 each), at 0.24 of the cost (0.18–0.33) and 0.28 of the time, with about 170 tokens
|
|
202
|
+
handed back instead of the 23k a lookup adds to the asking agent's context. Fixing real issues
|
|
203
|
+
does not get cheaper: 0.85 (0.76–0.96) in the fourth round, a gain the lookup rules it installs
|
|
204
|
+
give on their own (0.86), and 1.05 (0.82–1.46) with the smaller model. In real Claude Code
|
|
205
|
+
sessions (252 runs) graph-indexer as `init` installs it answered all 12 code questions exactly
|
|
206
|
+
against 8–9 of 12 without it — the misses were the two-level caller questions — at 0.80 of the
|
|
207
|
+
cost and 0.65 of the time; refactors came out the same, and on real issues, where the agent never
|
|
208
|
+
called it, it cost 1.02–1.05 once its tool definitions were trimmed to what agents use. In Go (57
|
|
209
|
+
real sessions on caddy, graded by the Go type checker) graph-indexer answered all 14 code
|
|
210
|
+
questions against 13 without it, at 0.87 of the cost, and did the refactors at 0.81 of the cost
|
|
211
|
+
and half the time. With the usual model every run with graph-indexer solved its task. Details in
|
|
212
|
+
[docs/AGENTIC-BENCHMARK.md](docs/AGENTIC-BENCHMARK.md).
|
|
213
|
+
|
|
214
|
+
**Speed.** A full index of nestjs (1,641 files, 96k lines) takes 3.7 s; afterwards only changed
|
|
215
|
+
files are re-parsed. Warm tool calls take 1–4 ms for the graph tools, about 30 ms for search and
|
|
216
|
+
18 ms for a repository outline.
|
|
161
217
|
|
|
162
218
|
## How it works
|
|
163
219
|
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
| `--rerank --rerank-provider cross-encoder` | `generative` | Want a reranker's semantic lift with **no LLM/daemon** — air-gapped, deterministic, fast | Optional `@huggingface/transformers`; smaller lift than the 7B judge |
|
|
207
|
-
| `--use-sqlite` | `auto` | Repos past ~15k chunks or memory-constrained environments | Slightly higher query latency; needs Node 22+ |
|
|
208
|
-
|
|
209
|
-
For most repos, the default (lexical + stemming, no embeddings) is the right starting point. Enable embeddings when you notice the agent missing chunks it should find. Enable the reranker only on Go or Python repos after measuring whether it helps — it is known to regress JavaScript repositories.
|
|
210
|
-
|
|
211
|
-
**Reranker providers.** The reranker reorders the over-fetched candidate pool on natural-language queries; three providers are available (`--rerank-provider`):
|
|
212
|
-
|
|
213
|
-
- **`generative`** (default) — a local LLM judge (Ollama 7B). The strongest reranker in our benchmarks (agent-style semantic rank-1 0.19 → 0.42 across the core suites; no symbolic regression), but it needs a running model and its output is non-deterministic.
|
|
214
|
-
- **`cross-encoder`** — a local MS-MARCO cross-encoder (`Xenova/ms-marco-MiniLM-L-6-v2`, via the optional `@huggingface/transformers`). It scores each (query, candidate) pair and sorts. **Air-gapped, deterministic, and fast** (no Ollama/daemon; the model downloads once). It captures part of the lift — agent-style semantic rank-1 0.19 → 0.26, and notably Go (gin) 0.20 → 0.40 and Python (fastapi) 0.14 → 0.29 — but less than the 7B judge, and like all rerankers it is mixed on JavaScript. Pick it when you want a reranker without an LLM in the loop. Measure on your repo: `npm run test:eval -- --rerank --rerank-provider cross-encoder`.
|
|
215
|
-
- **`late-interaction`** (B4) — a **ColBERT-style multi-vector MaxSim** reranker. It encodes the query into several vectors (the whole query + content sub-units, via the same local embedder) and scores `Σ_qi max_dj cos(qi, dj)` against each candidate's stored document sub-vectors — **no storage blow-up** (it reuses the base/summary/window vectors `--embeddings` already writes), air-gapped, deterministic, parity-safe. **Requires `--embeddings`** (a no-op without it). Measured at production retrieval depth: **helps Go** (gin semantic rank-1 0.00 → 0.40, overall 0.68 → 0.77, no symbolic regression) and **neutral on JavaScript** (express unchanged) — the "helps Go/Python" half of the reranker profile without the JS regression. Approximates true token-level ColBERT with a sentence embedder. Measure on your repo: `npm run test:eval -- --embeddings --rerank --rerank-provider late-interaction`.
|
|
216
|
-
|
|
217
|
-
### All CLI flags
|
|
218
|
-
|
|
219
|
-
| Flag | Default | Effect |
|
|
220
|
-
|------|---------|--------|
|
|
221
|
-
| `--repo <path>` | current directory | Repository to index / serve. |
|
|
222
|
-
| `--embeddings` | off | Enable the dense-vector channel. |
|
|
223
|
-
| `--embed-model <model>` | `nomic-embed-text` | Ollama embedding model (e.g. `qwen3-embedding:4b`). |
|
|
224
|
-
| `--embed-provider <auto\|ollama\|local\|mlx\|off>` | `auto` | Force the embedding backend (see [Embedder backends](#embedder-backends)). |
|
|
225
|
-
| `--mlx-embed-model <model>` | `mlx-community/all-MiniLM-L6-v2-4bit` | Model loaded by the MLX (Apple Metal) embedder. |
|
|
226
|
-
| `--use-sqlite` | `auto` | Force the disk-backed SQLite backend. |
|
|
227
|
-
| `--enrichment` | off | Enable LLM enrichment of central chunks. |
|
|
228
|
-
| `--enrich-model <model>` | `qwen2.5-coder:1.5b` | Model used for enrichment. |
|
|
229
|
-
| `--enrich-max <n>` | 500 | Cap on new LLM calls per index run. |
|
|
230
|
-
| `--enrich-concurrency <n>` | 4 | Parallel Ollama requests during enrichment. |
|
|
231
|
-
| `--rerank` | off | Enable the reranker (reorders the over-fetched pool on NL queries). |
|
|
232
|
-
| `--rerank-provider <generative\|cross-encoder\|late-interaction>` | `generative` | `generative` = local LLM judge; `cross-encoder` = local air-gapped MS-MARCO cross-encoder (optional `@huggingface/transformers`, no Ollama); `late-interaction` = ColBERT-style multi-vector MaxSim over the stored doc vectors (needs `--embeddings`; air-gapped, deterministic, no storage blow-up; measured helps-Go / neutral-JS at production depth). |
|
|
233
|
-
| `--interprocedural` | off | Index-time inter-procedural receiver-type fixpoint: propagate return types along factory call chains so multi-hop receivers resolve in `get_call_graph` / `find_references`. Air-gapped, deterministic; default index byte-identical; search ranking unaffected. |
|
|
234
|
-
| `--symbol-graph` | off | Persist a resolved chunk→chunk symbol graph (edges with kind + confidence) at index time, exposed via `getEdges`. `findCallers`/`findReferers` read it (identical sets, fall back to a scan). Also computes symbol-level centrality (confidence-weighted PageRank over the edges), surfaced by `explain_symbol` and `get_repo_map`. Air-gapped, deterministic; default index byte-identical; search ranking unaffected. |
|
|
235
|
-
| `--taint` | off | Precompute and serialize the taint flow set (C2) at index time, so `trace_taint` / `find_tainted_sinks` answer instantly instead of walking the call graph per query. Computed against the symbol-graph edges when `--symbol-graph` is on (cross-function precision), else `chunk.calls`. Both backends serve byte-identical flows; default index byte-identical; air-gapped. |
|
|
236
|
-
| `--ranker <rrf\|learned>` | `rrf` | Search ranker (D3). `rrf` (default) = the RRF + boost-ladder fusion. `learned` = an opt-in, zero-dependency post-fusion linear re-rank over the RRF score + symbol centrality + resolved/SCIP in-degree + git (best paired with `--symbol-graph`). **Measured ≈neutral on the benchmark** — RRF is the default; the learned ranker is provided for users to retrain on their own repo (`bench/train-ranker.mjs`). Default path byte-identical. |
|
|
237
|
-
| `--learned-sparse` | off | Learn a corpus co-occurrence (PMI) association model (B3) at index time and serialize it, adding a **learned-sparse vocabulary-expansion** channel: NL queries expand into a weighted term set (literal terms + learned associates) fused via RRF, to close lexical-gap misses stemming can't. Zero new dependency, air-gapped, NL-asymmetric (symbolic lookups byte-identical), parity-safe. **Measured net-neutral on the strict benchmark** (does not beat lexical — like BM25F/AST); shipped opt-in for repos with genuine query/code vocabulary mismatch. Default index byte-identical. |
|
|
238
|
-
| `--resolver <heuristic\|precise\|scip>` | `heuristic` | Resolver provider for the symbol graph. `precise` lifts provably-unambiguous edges (sole definition, or a type-pinned receiver) from `high` to a `resolved` tier. `scip` ingests a locally-generated SCIP index (`--scip-index`) and resolves edges by *real cross-file bindings* — promoting confirmed edges to `resolved` **and suppressing the wrong-target fan-out** an ambiguous same-name reference would otherwise emit, **and** adding a precise cross-file **`resolved`-tier** dimension to `find_references` (disambiguates same-named symbols, recovers references the name heuristics miss). Both power `impact_of_edit(precision: 'strict')`. Only meaningful with `--symbol-graph`. |
|
|
239
|
-
| `--scip-index <path>` | — | Path to a SCIP index generated locally by your own toolchain (`scip-typescript`, `scip-python`, `scip-java`, `rust-analyzer --scip`), consumed by `--resolver scip`. Air-gapped (a local file read — sealed-compatible). Files the index doesn't cover fall back to heuristic confidence (never worse); coverage is printed at index time. Accepts the `.scip` protobuf or a SCIP-shaped JSON. |
|
|
240
|
-
| `--sealed [strict\|local]` | off | **Sealed mode** — air-gapped *enforced and verifiable*. `strict` = zero network egress (lexical-only); `local` = loopback only (local LLM/embedders OK, nothing off-box). Fail-closed: refuses to start if any enabled feature would egress beyond the tier, and installs a deny-by-default runtime egress guard. Bare `--sealed` = `strict`. |
|
|
241
|
-
| `--attest` | — | Print the deterministic sealed-mode manifest (tier, providers, egress posture) and exit — an auditor/CI artifact. Add `--sign-key <path>` to print a **cryptographically signed** envelope instead (Ed25519/RSA/EC via `node:crypto`; air-gapped). |
|
|
242
|
-
| `--gen-attestation-key <prefix>` | — | Write an Ed25519 attestation keypair (`<prefix>.key` private, `<prefix>.pub` public) and exit. |
|
|
243
|
-
| `--verify-attestation <file> --pub-key <path>` | — | Verify a signed attestation's signature and report whether it still matches the current effective config (policy drift → non-zero exit). |
|
|
244
|
-
| `--no-git-signals` | (signals on) | Skip collecting local git churn/recency/co-change. |
|
|
245
|
-
| `--git-rank-boost <0..1>` | 0 | Opt-in weight for git recency/churn in ranking (0 = ranking unchanged). |
|
|
246
|
-
| `--llm-provider <ollama\|mlx>` | `ollama` | LLM backend for enrichment, reranking, and HyDE. `mlx` routes calls to a local `mlx_lm.server`. |
|
|
247
|
-
| `--mlx-lm-host <url>` | `http://localhost:8080` | Endpoint for the `mlx_lm.server` when `--llm-provider mlx` is set. |
|
|
248
|
-
|
|
249
|
-
### All environment variables
|
|
250
|
-
|
|
251
|
-
| Variable | Default | Effect |
|
|
252
|
-
|----------|---------|--------|
|
|
253
|
-
| `MCP_PROJECT_ROOT` | current directory | Repository root when `--repo` is not given. |
|
|
254
|
-
| `OLLAMA_HOST` | `http://localhost:11434` | Ollama endpoint for embeddings/enrichment/rerank. |
|
|
255
|
-
| `INDEXER_EMBEDDINGS` | `off` | `on` enables embeddings; `off` always wins over `--embeddings`. |
|
|
256
|
-
| `EMBED_MODEL` | `nomic-embed-text` | Ollama embedding model (overrides config; overridden by `--embed-model`). |
|
|
257
|
-
| `INDEXER_EMBED_PROVIDER` | `auto` | `auto` \| `ollama` \| `local` \| `mlx` \| `off`. |
|
|
258
|
-
| `INDEXER_MLX_EMBED_MODEL` | `mlx-community/all-MiniLM-L6-v2-4bit` | Model loaded by the MLX embedder (overridden by `--mlx-embed-model`). |
|
|
259
|
-
| `INDEXER_MLX_BATCH_SIZE` | 32 | Texts per batch sent to the MLX subprocess (raise on large unified memory). |
|
|
260
|
-
| `INDEXER_STORAGE` | `auto` | `auto` \| `memory` \| `sqlite`. |
|
|
261
|
-
| `ENRICH_MODEL` | (unset) | Naming a model enables enrichment and selects it. |
|
|
262
|
-
| `RERANK_MODEL` | (unset) | Naming a model enables the (generative) reranker and selects it. |
|
|
263
|
-
| `RERANK_PROVIDER` | `generative` | `generative` (LLM judge), `cross-encoder` (local air-gapped), or `late-interaction` (ColBERT-style MaxSim, needs embeddings); setting it enables reranking. |
|
|
264
|
-
| `RERANK_CROSS_ENCODER_MODEL` | `Xenova/ms-marco-MiniLM-L-6-v2` | Cross-encoder model id (downloaded on first use). |
|
|
265
|
-
| `INDEXER_GIT_SIGNALS` | (on) | Set to `off` to skip git-signal collection. |
|
|
266
|
-
| `INDEXER_INTERPROCEDURAL` | (off) | `on` enables the index-time inter-procedural receiver-type fixpoint (`--interprocedural`). |
|
|
267
|
-
| `INDEXER_SYMBOL_GRAPH` | (off) | `on` builds the persistent resolved symbol graph (`--symbol-graph`). |
|
|
268
|
-
| `INDEXER_TAINT` | (off) | `on` precomputes + serializes the taint flow set at index time (`--taint`). |
|
|
269
|
-
| `INDEXER_RANKER` | `rrf` | `learned` enables the opt-in post-fusion learned re-rank (`--ranker`). |
|
|
270
|
-
| `INDEXER_LEARNED_SPARSE` | (off) | `on` learns + serializes the corpus association model for the learned-sparse expansion channel (`--learned-sparse`). |
|
|
271
|
-
| `INDEXER_RESOLVER` | `heuristic` | `precise` enables the precise resolver; `scip` enables the SCIP cross-file resolver (`resolved` edge tier) for the symbol graph (`--resolver`). |
|
|
272
|
-
| `INDEXER_SCIP_INDEX` | (unset) | Path to a locally-generated SCIP index for `--resolver scip` (`--scip-index`). |
|
|
273
|
-
| `INDEXER_SEALED` | (off) | `strict` (or `on`) / `local` enables sealed mode (`--sealed`): fail-closed validation + a runtime egress guard. |
|
|
274
|
-
| `INDEXER_GIT_RANK_BOOST` | 0 | Opt-in git recency/churn ranking weight (0..1). |
|
|
275
|
-
| `INDEXER_LLM_PROVIDER` | `ollama` | LLM backend for enrichment, reranking, and HyDE: `ollama` or `mlx`. |
|
|
276
|
-
| `INDEXER_MLX_LM_HOST` | `http://localhost:8080` | Endpoint for the `mlx_lm.server` when `INDEXER_LLM_PROVIDER=mlx`. |
|
|
277
|
-
| `INDEXER_EMBED_CONCURRENCY` | 4 | Parallel embedding batches; lower to 1 for large models on modest hardware. |
|
|
278
|
-
| `INDEXER_EMBED_TIMEOUT_MS` | 120000 | Per-batch embedding timeout; raise for very large models. |
|
|
279
|
-
|
|
280
|
-
When embeddings are enabled, the provider is selected in this order, and every fallback is logged (never silent): Ollama with `EMBED_MODEL` if set and reachable → Ollama with `nomic-embed-text` → the in-process MiniLM model (optional `@huggingface/transformers`) → lexical-only with a warning.
|
|
281
|
-
|
|
282
|
-
### Embedder backends
|
|
283
|
-
|
|
284
|
-
Set the backend with `--embed-provider` (or `INDEXER_EMBED_PROVIDER`). `mlx` is a native Python embedder that runs the Apple Metal GPU as a reused subprocess — faster in-process vectors than the bundled Xenova path, with no running Ollama daemon. The in-process (`local`) and `mlx` backends produce all-MiniLM-L6-v2-family **384-dim** vectors; the default `auto`→Ollama path uses `nomic-embed-text` (768-dim).
|
|
285
|
-
|
|
286
|
-
| Backend | `--embed-provider` | Platform | Throughput¹ | Setup |
|
|
287
|
-
|---------|--------------------|----------|-------------|-------|
|
|
288
|
-
| Auto (Ollama → local → off) | `auto` (default) | any | varies | none |
|
|
289
|
-
| In-process (Xenova) | `local` | any | ~18 ch/s | `npm i @huggingface/transformers` |
|
|
290
|
-
| **Apple Metal (MLX)** | **`mlx`** | **macOS Apple Silicon** | **~42 ch/s** | `npm run embed:setup:mlx` |
|
|
291
|
-
| Ollama daemon | `ollama` | any | ~14 ch/s² | `ollama serve` + `ollama pull` |
|
|
292
|
-
|
|
293
|
-
¹ **Throughput is hardware-dependent.** These figures are the median of 3 cold builds on an **Apple M2 Mac mini (24 GB)** indexing the `express-js` fixture (389 chunks): `node bench/cell.mjs express-js <E0|E0_MLX|O0>`. Expect different numbers on other chips; reproduce on your own machine. Throughput also varies with system load, model warm state, and corpus size — larger repos amortize cold-start better (e.g. Ollama/nomic reaches ~32 ch/s on the larger `gin` corpus).
|
|
294
|
-
|
|
295
|
-
² On Apple Silicon, Ollama already uses the Metal GPU internally (via llama.cpp). The `mlx` provider's advantage comes from a smaller model (all-MiniLM-L6-v2-4bit, 22M params, 384-dim) and no HTTP round-trip, not from GPU vs CPU.
|
|
296
|
-
|
|
297
|
-
The index records which provider/model built it (in the `code-index.embeddings.bin.meta.json` sidecar) and queries with the same one, so vectors never get mixed across models. Switching providers or the MLX model between builds is detected and triggers a clean re-embed.
|
|
298
|
-
|
|
299
|
-
**Code-specialized embedding models.** The default `nomic-embed-text` is a general-purpose embedder. Models trained on code — `qwen3-embedding`, `nomic-embed-code`, `jina-embeddings-v2-base-code` — embed code semantics more faithfully and can lift agent-style (natural-language) recall. Select one with `--embed-model <name>` (Ollama); it is stamped into the index meta and queried with the same model so vectors never get mixed. **Measure before adopting** — in our benchmarks `qwen3-embedding` lifted Go (gin) semantic rank-1 0.20 → 0.40 but was neutral on Python (django) and JavaScript (express), and it indexes substantially slower than `nomic-embed-text`. Run `npm run test:eval -- --embeddings` on your own repo to confirm the trade-off. We also evaluated *re-weighting the vector channel upward* for these stronger embedders and found **no measurable recall@5 gain** (it regressed rank-1/symbolic at higher weights) — the existing default weighting already extracts the benefit, so there is no separate "code weight" knob. `jina-embeddings-v2-base-code` remains opt-in only: it helped some fixtures, regressed others, and indexes far slower, so it is never a default.
|
|
300
|
-
|
|
301
|
-
**macOS Apple Silicon (recommended):**
|
|
302
|
-
|
|
303
|
-
```bash
|
|
304
|
-
npm run embed:setup:mlx # one-time: creates embedders/venv-mlx + installs MLX
|
|
305
|
-
npx idx-index --repo . --embeddings --embed-provider mlx
|
|
220
|
+
1. **Parse.** tree-sitter (WebAssembly) with one query per language extracts definitions,
|
|
221
|
+
references (calls, instantiations, types, inheritance, decorators, values, member reads),
|
|
222
|
+
imports and type evidence.
|
|
223
|
+
2. **Infer types locally.** Receiver types come from annotations, initialisers, `new`/casts,
|
|
224
|
+
return types (declared or inferred from `return` statements), fields and constructor-injected
|
|
225
|
+
dependencies, collections (`xs[i]`, `for x of xs`, `xs.forEach(x => …)`, `list.get(0)`),
|
|
226
|
+
optionals and wrappers (`Foo | undefined`, `Optional[Foo]`, `Promise<Foo>`, `Arc<Mutex<Foo>>`),
|
|
227
|
+
across closures and access chains.
|
|
228
|
+
3. **Resolve.** Each reference is bound by a ladder: lexical scope → enclosing type and its
|
|
229
|
+
bases → same file → explicit import (barrels, tsconfig paths, workspaces, Go modules, Python
|
|
230
|
+
packages, Rust crates, C includes) → package → global, and labelled with a confidence.
|
|
231
|
+
4. **Store.** SQLite (WAL) with per-file ownership of facts, stable symbol ids across edits,
|
|
232
|
+
FTS5 indexes over symbols and files. The index rebuilds itself when graph-indexer is upgraded.
|
|
233
|
+
5. **Stay fresh.** A file watcher plus periodic stat sweeps re-index changed files before any
|
|
234
|
+
answer, and files are re-validated right before their lines are rendered.
|
|
235
|
+
6. **Rank.** Search fuses exact/qualified name matches, BM25F over name, signature, doc, path and
|
|
236
|
+
body, file-level relevance and a small concept thesaurus, then applies interpretable priors
|
|
237
|
+
and PageRank centrality. Weights were tuned on a tuning split only.
|
|
238
|
+
|
|
239
|
+
[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) describes each part in detail.
|
|
240
|
+
|
|
241
|
+
## Languages
|
|
242
|
+
|
|
243
|
+
JavaScript, TypeScript/TSX, Python, Go, Java, Kotlin, Scala, C#, Rust, C, C++, Ruby, PHP, Bash
|
|
244
|
+
and CSS/SCSS/Less. Reference binding and type inference are deepest for TypeScript/JavaScript,
|
|
245
|
+
Python, Go, Java, C# and Rust.
|
|
246
|
+
|
|
247
|
+
## Command line
|
|
248
|
+
|
|
249
|
+
The CLI runs the same tools, which is handy for scripts and for checking what an agent sees:
|
|
250
|
+
|
|
251
|
+
```text
|
|
252
|
+
graph-indexer init [--repo DIR] [--agents …] [--all] [--hooks] [--local] [--dry-run] [--no-instructions] [--no-helper]
|
|
253
|
+
graph-indexer serve [--repo DIR] MCP server on stdio
|
|
254
|
+
graph-indexer index [--repo DIR] build or update the index
|
|
255
|
+
graph-indexer status [--repo DIR]
|
|
256
|
+
graph-indexer search <query> [--path P] [--kind K] [--limit N]
|
|
257
|
+
graph-indexer symbol <name> [--no-code]
|
|
258
|
+
graph-indexer refs <name> [--kind call|type|inherit|new|value|decorator] [--no-tests]
|
|
259
|
+
graph-indexer callgraph <name> [--direction callers|callees|both] [--depth N]
|
|
260
|
+
graph-indexer impact [--symbols a,b] [--files x,y] [--diff] [--depth N]
|
|
261
|
+
graph-indexer outline [path] [--focus TEXT] [--max-tokens N]
|
|
306
262
|
```
|
|
307
263
|
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
Choose a different MLX model with `--mlx-embed-model <id>` (or `INDEXER_MLX_EMBED_MODEL`); it must be an `mlx_embeddings`-compatible sentence model, and the default `mlx-community/all-MiniLM-L6-v2-4bit` is the proven option. Tune batch size with `INDEXER_MLX_BATCH_SIZE` (default 32; lower to 16 on 8 GB machines, raise to 64 on M-series Max/Ultra). `mlx` is macOS-only; requesting it elsewhere fails fast with the alternative to use.
|
|
311
|
-
|
|
312
|
-
## Benchmarks
|
|
313
|
-
|
|
314
|
-
Every number here is produced by the eval harness on cold, isolated builds — never hand-edited — with **strict scoring** (exact symbol match, no file-path fallback) against a **held-out** split (~20–25% per language, never used to tune). Reproduce it all with `npm run test:eval`.
|
|
264
|
+
The index lives in `<repo>/.graph-indexer/` (git-ignored). It contains excerpts of your code, so
|
|
265
|
+
treat it like the repository itself.
|
|
315
266
|
|
|
316
|
-
|
|
267
|
+
## Privacy and security
|
|
317
268
|
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
|
|
269
|
+
Everything runs locally. graph-indexer makes no network calls, runs no code from the repository
|
|
270
|
+
it indexes (the only subprocesses are read-only git commands: `rev-parse`, `ls-files`, `diff`
|
|
271
|
+
and `log`), talks
|
|
272
|
+
to the agent over stdio only, and never reads outside the repository: tool paths are resolved
|
|
273
|
+
through the index, and symlinks pointing outside the repository are ignored. See
|
|
274
|
+
[SECURITY.md](SECURITY.md).
|
|
322
275
|
|
|
323
|
-
|
|
276
|
+
## Upgrading from 2.x
|
|
324
277
|
|
|
325
|
-
|
|
278
|
+
Version 3 is a rewrite; see the [changelog](CHANGELOG.md). In short: the tools changed
|
|
279
|
+
(`read_code`, `search_text`, `search_code`, `find_references`, `call_graph`, `change_impact`,
|
|
280
|
+
`check_changes`, `outline`), the
|
|
281
|
+
embedding, Ollama, reranker and daemon options are gone (search is lexical plus graph and needs
|
|
282
|
+
no model), grammars are bundled instead of installed on first use, and Node.js 22.5+ is required.
|
|
283
|
+
Re-run `npx -y --prefix / graph-indexer@3 init` to update agent configuration; old `.graph-indexer/` contents are
|
|
284
|
+
replaced automatically.
|
|
326
285
|
|
|
327
|
-
|
|
286
|
+
## License
|
|
328
287
|
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
| **Call graph** (callers/callees) | resolves on every supported language | Java/Spring is class-granular; SCSS effectively none |
|
|
332
|
-
| **Caller precision** | receiver-aware: TS/JS, Python, C#, Swift, PHP, Java (call-site level — but its call graph is class-granular) | name-only: Go, Rust, Kotlin, Ruby, Bash, C |
|
|
333
|
-
| **Typed `find_references`** | precise: TS/JS, Python · field-precise: C# | heuristic: Java/PHP/Kotlin/Swift/Rust/Go/C · empty: Ruby, Bash, SCSS, dynamic JS/Python |
|
|
334
|
-
|
|
335
|
-
AST chunking and lexical search cover **every** supported language; only the typed cross-reference channel narrows for dynamic ones.
|
|
336
|
-
|
|
337
|
-
**Multi-hop receivers (`--interprocedural`).** Caller precision relies on resolving a call's receiver type. Intra-procedurally, graph-indexer already resolves `const r = makeRepo(); r.save()` one hop (reading `makeRepo`'s recorded return type). The opt-in `--interprocedural` flag adds an index-time fixpoint that propagates return types *along factory call chains*, so a receiver whose type comes from a multi-hop or unannotated factory (`makeRepoIndirect()` → `makeRepo()` → `new OrderRepo()`) still resolves to its concrete class — promoting those callers from *name-only* to *high-confidence* in `get_call_graph` / `find_references`. It is air-gapped and deterministic, the result is serialized into the index (both backends identical), and it leaves search ranking and the default index byte-identical. Note: the watch daemon updates one file at a time and does not re-run the whole-program pass, so cross-file chains refresh on the next full `idx-index` (degrading to single-hop until then, never worse).
|
|
338
|
-
|
|
339
|
-
### Where it's weakest (honestly)
|
|
340
|
-
|
|
341
|
-
- Behavioural, natural-language recall on the default path (rank-1 ≈ 0.30 overall; held-out semantic rank-1 0.40 — the embedding/reranker channel closes it further) — closing it needs the embedding/reranker channel, not lexical tuning.
|
|
342
|
-
- The hardest fixtures stay below 0.65 held-out success@5 even with the full stack (rails 0.50, rust 0.61).
|
|
343
|
-
- Java/Spring `get_call_graph` reports at class granularity (the god-class split only fires ≥200 lines); SCSS has no meaningful call graph.
|
|
344
|
-
- LLM enrichment without reranking regresses precision — enable it only paired with `--rerank`.
|
|
345
|
-
|
|
346
|
-
## Reproducing the benchmarks
|
|
347
|
-
|
|
348
|
-
```bash
|
|
349
|
-
npm run test:all # full unit + integration suite
|
|
350
|
-
npm run test:setup # index the benchmark fixtures
|
|
351
|
-
npm run test:eval # default lexical path — prints the HELD-OUT block
|
|
352
|
-
npm run test:eval -- --suite css # any single fixture (18 authored suites)
|
|
353
|
-
npm run test:eval -- --embeddings # hybrid eval (requires Ollama)
|
|
354
|
-
npm run test:eval -- --verbose # per-query breakdown incl. file-rank column
|
|
355
|
-
node test/sqlite.mjs # backend-parity gate (memory ↔ SQLite top-5)
|
|
356
|
-
node bench/cell.mjs <fixture> L1 # cold rebuild + score one fixture
|
|
357
|
-
```
|
|
358
|
-
|
|
359
|
-
The harness is [test/evaluate.mjs](test/evaluate.mjs); deeper reproduction notes live in [docs/benchmarks/BENCH_BASELINE.md](docs/benchmarks/BENCH_BASELINE.md) and [docs/benchmarks/BENCH_FULL_SUITE.md](docs/benchmarks/BENCH_FULL_SUITE.md).
|
|
360
|
-
|
|
361
|
-
---
|
|
362
|
-
|
|
363
|
-
<div align="center">
|
|
364
|
-
<sub>MIT licensed · runs entirely on your machine · see <a href="SECURITY.md">SECURITY.md</a> and <a href="TRADEMARK.md">TRADEMARK.md</a></sub>
|
|
365
|
-
</div>
|
|
288
|
+
[MIT](LICENSE). "graph-indexer" and the logo are trademarks of MaquinaTech; see
|
|
289
|
+
[TRADEMARK.md](TRADEMARK.md).
|