graph-indexer 1.0.2 → 1.0.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (6) hide show
  1. package/PROMPT.md +143 -218
  2. package/README.md +169 -161
  3. package/init.mjs +179 -68
  4. package/mcp-server.mjs +1 -1
  5. package/package.json +13 -13
  6. package/parser-utils.mjs +51 -20
package/README.md CHANGED
@@ -14,31 +14,43 @@
14
14
 
15
15
  ---
16
16
 
17
- ## What it does
17
+ ## Table of Contents
18
+
19
+ - [What it does](#what-it-does)
20
+ - [Results](#results)
21
+ - [Getting started](#getting-started)
22
+ - [MCP tools](#mcp-tools)
23
+ - [Configuration](#configuration)
24
+ - [Supported languages](#supported-languages)
25
+ - [How it works](#how-it-works)
26
+ - [Best practices](#best-practices)
27
+ - [Troubleshooting](#troubleshooting)
28
+ - [Development](#development)
29
+ - [License](#license)
18
30
 
19
- graph-indexer pre-indexes your codebase into an AST-precise search index and exposes it as an MCP server. Instead of reading full files, AI agents call `search_code("payment validation")` and get back the exact functions that match — with their type signatures, docstrings, dependency graph, and call sites — in a fraction of the tokens.
31
+ ---
20
32
 
21
- It runs entirely on your machine. No external database, no cloud APIs, no telemetry.
33
+ ## What it does
22
34
 
23
- ---
35
+ graph-indexer pre-indexes your codebase into an AST-precise search index and exposes it as an MCP server. Instead of reading full files, AI agents call `search_code("payment validation")` and get back the exact functions that match — with type signatures, docstrings, dependency graph, and call sites — in a fraction of the tokens.
24
36
 
25
- ## Why it matters
37
+ It runs entirely on your machine. No external database, no cloud APIs, no telemetry.
26
38
 
27
- When an AI agent works on a real codebase without graph-indexer, it reads files to find code:
39
+ **Token comparison:**
28
40
 
29
41
  ```
30
42
  Agent task: "Add error handling to the payment processing flow"
31
43
 
32
44
  Without graph-indexer:
33
- readFile("src/payments/service.ts") → 8,400 tokens
34
- readFile("src/payments/handlers.ts") → 6,200 tokens
35
- readFile("src/types/payment.ts") → 1,800 tokens
36
- Total context consumed: ~16,400 tokens for 3 files
45
+ readFile("src/payments/service.ts") → 8,400 tokens
46
+ readFile("src/payments/handlers.ts") → 6,200 tokens
47
+ readFile("src/types/payment.ts") → 1,800 tokens
48
+ Total: ~ 16,400 tokens
37
49
 
38
50
  With graph-indexer:
39
- search_code("payment processing") → 650 tokens (5 exact chunks)
40
- get_chunk("chunk_id") → 280 tokens (full function body)
41
- Total context consumed: ~930 tokens
51
+ search_code("payment processing") → 650 tokens (5 exact chunks)
52
+ get_chunk("chunk_id") → 280 tokens (full function body)
53
+ Total: ~ 930 tokens
42
54
  ```
43
55
 
44
56
  The difference compounds across a session. Agents that read files also lack topology — they can't see which other functions call the one they're modifying, so they miss side effects.
@@ -47,91 +59,40 @@ The difference compounds across a session. Agents that read files also lack topo
47
59
 
48
60
  ## Results
49
61
 
50
- Measured across 5 production open-source codebases (7,503 total AST chunks, 65 ground-truth queries). Token savings compare the chunks returned per query against reading the full source files they came from.
51
-
52
- ### Token usage per query: with vs. without graph-indexer
62
+ Measured across 5 production open-source codebases (7,503 AST chunks, 70 ground-truth queries across 3 difficulty levels).
53
63
 
54
- | Project | Language | Codebase size | Without (full files) | With graph-indexer | Savings | Latency |
55
- | :--- | :--- | ---: | ---: | ---: | ---: | ---: |
56
- | Axios v1.6.0 | JavaScript | 111k tokens | ~2,290 tok/query | ~655 tok/query | **71.4%** | 0.2 ms |
57
- | Express 4.18.2 | JavaScript | 145k tokens | ~7,330 tok/query | ~850 tok/query | **88.4%** | 0.2 ms |
58
- | NestJS v10.4.9 | TypeScript | 700k tokens | ~2,535 tok/query | ~900 tok/query | **64.5%** | 0.7 ms |
59
- | FastAPI 0.103.0 | Python | 859k tokens | ~4,030 tok/query | ~685 tok/query | **83.0%** | 1.0 ms |
60
- | Gin v1.9.1 | Go | 131k tokens | ~2,530 tok/query | ~450 tok/query | **82.2%** | 0.4 ms |
61
- | **Mean** | | | | | **77.9%** | **0.5 ms** |
64
+ ### Token savings per query
62
65
 
63
- > "Full files" = reading the source files that contain the top-5 retrieved chunks. "With graph-indexer" = tokens in the returned chunk code snippets only. Measured at 4 chars/token.
66
+ | Project | Language | Without (full files) | With graph-indexer | Savings | Latency |
67
+ | :--- | :--- | ---: | ---: | ---: | ---: |
68
+ | Axios v1.6.0 | JavaScript | ~2,290 tok | ~655 tok | **71.4%** | 0.2 ms |
69
+ | Express 4.18.2 | JavaScript | ~7,330 tok | ~850 tok | **88.4%** | 0.2 ms |
70
+ | NestJS v10.4.9 | TypeScript | ~2,535 tok | ~900 tok | **64.5%** | 0.7 ms |
71
+ | FastAPI 0.103.0 | Python | ~4,030 tok | ~685 tok | **83.0%** | 1.0 ms |
72
+ | Gin v1.9.1 | Go | ~2,530 tok | ~450 tok | **82.2%** | 0.4 ms |
73
+ | **Mean** | | | | **77.9%** | **0.5 ms** |
64
74
 
65
- **Key observations:**
66
- - NestJS has 700k tokens of source — larger than most LLMs' full context window. graph-indexer reduces each query to ~900 tokens.
67
- - Express shows 88.4% savings because its core router logic is spread across large files with much non-relevant code.
68
- - Latency is sub-millisecond for all suites at their real corpus sizes.
75
+ > "Full files" = tokens in the source files containing the top-5 results. "With graph-indexer" = tokens in the returned chunk snippets only. Measured at 4 chars/token. NestJS is 700k tokens of source — larger than most LLMs' context windows.
69
76
 
70
77
  ### Search quality
71
78
 
72
- | Project | Chunks | Recall@1 | Recall@3 | Recall@5 | MRR | Queries |
73
- | :--- | ---: | ---: | ---: | ---: | ---: | ---: |
74
- | Axios v1.6.0 | 435 | 0.86 | 1.00 | **1.00** | 0.92 | 14 |
75
- | Express 4.18.2 | 389 | 0.93 | 0.93 | **1.00** | 0.95 | 14 |
76
- | NestJS v10.4.9 | 2,019 | 1.00 | 1.00 | **1.00** | **1.00** | 15 |
77
- | FastAPI 0.103.0 | 3,572 | 0.93 | 1.00 | **1.00** | 0.96 | 14 |
78
- | Gin v1.9.1 | 1,088 | 0.85 | 1.00 | **1.00** | 0.92 | 13 |
79
- | **Mean** | **7,503** | **0.91** | **0.99** | **1.00** | **0.95** | **70** |
79
+ | Project | Chunks | Recall@1 | Recall@3 | Recall@5 | MRR |
80
+ | :--- | ---: | ---: | ---: | ---: | ---: |
81
+ | Axios v1.6.0 | 435 | 0.86 | 1.00 | **1.00** | 0.92 |
82
+ | Express 4.18.2 | 389 | 0.93 | 0.93 | **1.00** | 0.95 |
83
+ | NestJS v10.4.9 | 2,019 | 1.00 | 1.00 | **1.00** | **1.00** |
84
+ | FastAPI 0.103.0 | 3,572 | 0.93 | 1.00 | **1.00** | 0.96 |
85
+ | Gin v1.9.1 | 1,088 | 0.85 | 1.00 | **1.00** | 0.92 |
86
+ | **Mean** | **7,503** | **0.91** | **0.99** | **1.00** | **0.95** |
80
87
 
81
- - **Recall@5 = 1.00** across every project and language: the correct answer is always in the top 5 results.
82
- - **MRR = 0.95** overall: on average, the correct answer is the first or second result.
83
- - Queries span three difficulty levels: easy (exact symbol names), medium (multi-token descriptions), hard (semantic concepts with vocabulary mismatch).
84
-
85
- ---
86
-
87
- ## How it works
88
-
89
- ### Indexing
90
-
91
- When you run `npm run mcp:index`, graph-indexer:
92
-
93
- 1. **Parses** every source file with Tree-sitter to build an AST — no regex, exact code boundaries.
94
- 2. **Extracts chunks**: named functions, classes, methods, and exports. Each chunk includes its name, docstring, parameters, return type, call sites, and type references.
95
- 3. **Builds a dependency graph**: bidirectional import map so each chunk knows what it imports and what imports it.
96
- 4. **Creates two search indexes**:
97
- - A BM25 inverted index (lexical search, O(1) per-token lookup).
98
- - A vector embedding per chunk via Ollama `nomic-embed-text` (optional; falls back to lexical-only if Ollama is unavailable).
99
- 5. **Serializes to disk**: `code-index.json` (metadata + BM25 index) and `code-index.embeddings.bin` (binary float32 vectors).
100
-
101
- A background file watcher daemon (`watch-daemon.mjs`) re-indexes changed files automatically after the initial index is built.
102
-
103
- ### Search
104
-
105
- When an agent calls `search_code("payment validation middleware")`:
106
-
107
- 1. **Lexical search** over the BM25 inverted index — returns candidates with BM25 scores in O(query_terms) time.
108
- 2. **Vector search** over pre-normalized float32 embeddings — returns semantically similar chunks.
109
- 3. **Reciprocal Rank Fusion (RRF)** merges both ranked lists with weights (lexical 1.5×, vector 1.0×).
110
- 4. **Scoring adjustments**: exact name match +2.0×, snake_case suffix match +1.4×, path token match +1.4×, test/example files demoted.
111
- 5. **Returns** top-k chunks with: code snippet, type signature, docstring, call graph, and dependency topology — all in one response.
112
-
113
- ### Architecture
114
-
115
- ```
116
- Source files (.ts .js .py .go .rs …)
117
- │
118
- ▼
119
- Tree-sitter AST parser
120
- │
121
- ├──► Named chunks (functions, classes, methods)
122
- │ │
123
- │ ├──► BM25 inverted index ──┐
124
- │ └──► Ollama embeddings ──┤── RRF fusion ──► Ranked results
125
- │ │
126
- └──► Dependency graph ────────────────┘
127
- (importedBy / imports) ▼
128
- MCP tools for agents
129
- ```
88
+ **Recall@5 = 1.00** across every project: the correct answer is always in the top 5. **MRR = 0.95**: the correct answer is rank 1 or 2 on average.
130
89
 
131
90
  ---
132
91
 
133
92
  ## Getting started
134
93
 
94
+ **Requirements:** Node.js v18+ · Ollama (optional, for semantic search)
95
+
135
96
  ### 1. Install
136
97
 
137
98
  ```bash
@@ -139,9 +100,27 @@ npm install graph-indexer --save-dev
139
100
  npx graph-indexer init
140
101
  ```
141
102
 
142
- `init` auto-detects your IDE (Claude, Cursor, VS Code), adds npm scripts to `package.json`, and updates `.gitignore`.
103
+ `init` auto-detects your IDE (Claude, Cursor, VS Code), adds npm scripts to `package.json`, updates `.gitignore`, and opens an interactive language selector:
143
104
 
144
- ### 2. Index your repository
105
+ ```
106
+ ⚙️ Select languages (Arrows/Tab: move, Space: toggle, Enter: confirm):
107
+
108
+ ❯ ◯ TypeScript / TSX .ts, .tsx
109
+ ◯ JavaScript .js, .jsx, .mjs, .cjs
110
+ ◯ Python .py
111
+ ◯ Go .go
112
+ ◯ Rust .rs
113
+ ◯ PHP .php
114
+ ◯ Java .java
115
+ ◯ Kotlin .kt, .kts
116
+ ◯ C# .cs
117
+ ◯ Ruby .rb
118
+ ◯ CSS / SCSS .css, .scss
119
+ ```
120
+
121
+ Navigate with **↑ ↓**, toggle with **Space**, confirm with **Enter**. Leaving all unselected enables every language. Your selection is saved to `.graph-indexer.json`. Pass `--all-languages` to skip the prompt.
122
+
123
+ ### 2. Index your codebase
145
124
 
146
125
  **With semantic search (recommended):**
147
126
  ```bash
@@ -155,30 +134,32 @@ npm run mcp:index
155
134
  INDEXER_EMBEDDINGS=off npm run mcp:index
156
135
  ```
157
136
 
158
- Lexical-only still achieves Recall@5 = 1.00 for the test suites above.
137
+ Lexical-only still achieves Recall@5 = 1.00 on all benchmarks above.
159
138
 
160
139
  ### 3. Configure your IDE
161
140
 
141
+ `init` writes the config automatically. Manual examples:
142
+
162
143
  **Claude Desktop** (`~/Library/Application Support/Claude/claude_desktop_config.json`):
163
144
  ```json
164
145
  {
165
146
  "mcpServers": {
166
147
  "graph-indexer": {
167
- "command": "node",
168
- "args": ["/path/to/project/node_modules/graph-indexer/mcp-server.mjs"],
148
+ "command": "npm",
149
+ "args": ["run", "--prefix", "/path/to/project", "mcp:start"],
169
150
  "env": { "MCP_PROJECT_ROOT": "/path/to/project" }
170
151
  }
171
152
  }
172
153
  }
173
154
  ```
174
155
 
175
- **Cursor** (`.cursor/mcp.json`):
156
+ **Cursor** (`.cursor/mcp.json`) and **Claude Code** (`.mcp.json`):
176
157
  ```json
177
158
  {
178
159
  "mcpServers": {
179
160
  "graph-indexer": {
180
- "command": "node",
181
- "args": ["${workspaceFolder}/node_modules/graph-indexer/mcp-server.mjs"],
161
+ "command": "npm",
162
+ "args": ["run", "mcp:start"],
182
163
  "env": { "MCP_PROJECT_ROOT": "${workspaceFolder}" }
183
164
  }
184
165
  }
@@ -191,8 +172,9 @@ Lexical-only still achieves Recall@5 = 1.00 for the test suites above.
191
172
  "servers": {
192
173
  "graph-indexer": {
193
174
  "type": "stdio",
194
- "command": "node",
195
- "args": ["${workspaceFolder}/node_modules/graph-indexer/mcp-server.mjs"]
175
+ "command": "npm",
176
+ "args": ["run", "mcp:start"],
177
+ "env": { "MCP_PROJECT_ROOT": "${workspaceFolder}" }
196
178
  }
197
179
  }
198
180
  }
@@ -200,7 +182,7 @@ Lexical-only still achieves Recall@5 = 1.00 for the test suites above.
200
182
 
201
183
  ### 4. Add the agent system prompt
202
184
 
203
- Copy the contents of [PROMPT.md](./PROMPT.md) into your AI agent's system prompt. This instructs the agent to use graph-indexer tools instead of reading files directly.
185
+ Copy [PROMPT.md](./PROMPT.md) into your AI agent's system prompt to instruct it to use graph-indexer tools instead of reading files directly.
204
186
 
205
187
  ---
206
188
 
@@ -208,96 +190,88 @@ Copy the contents of [PROMPT.md](./PROMPT.md) into your AI agent's system prompt
208
190
 
209
191
  ### `search_code`
210
192
 
211
- Searches the index by natural language query. Returns signature cards for all results, then code bodies according to the `detail` level.
193
+ Hybrid BM25 + vector search over the index. The primary entry point for agents.
212
194
 
213
195
  | Parameter | Type | Default | Description |
214
196
  | :--- | :--- | :--- | :--- |
215
197
  | `query` | string | required | Natural language description of the logic to find |
216
198
  | `exact_tokens` | string? | — | Exact symbol name — guarantees rank-1 placement |
217
- | `detail` | `"signatures"` \| `"smart"` \| `"full"` | `"smart"` | Controls how much code is returned |
199
+ | `detail` | `"signatures"` \| `"smart"` \| `"full"` | `"smart"` | How much code to return per result |
218
200
  | `top_k` | number | `5` | Results to return (1–20) |
219
- | `include_topology` | boolean | `true` | Include Deps / Used by / Calls in each card |
220
- | `min_score` | number | `0.3` | Minimum score threshold |
221
- | `token_budget` | number? | auto | Token budget for code bodies |
201
+ | `include_topology` | boolean | `true` | Include imports / used-by / calls |
202
+ | `min_score` | number | `0.3` | Minimum relevance threshold |
203
+ | `token_budget` | number? | auto | Cap total tokens returned |
222
204
 
223
205
  **`detail` levels:**
224
206
 
225
- | Value | Per-result cost | What you get |
207
+ | Value | Cost | Returns |
226
208
  | :--- | ---: | :--- |
227
- | `"signatures"` | ~20 tokens | Name, type, params, return type, topology only |
228
- | `"smart"` (default) | ~150 tokens | Signature + only lines relevant to the query |
229
- | `"full"` | ~300 tokens | Signature + complete source body |
230
-
231
- Start with `"signatures"`, escalate only when you need implementation details.
209
+ | `"signatures"` | ~20 tok | Name, type, params, return type, topology |
210
+ | `"smart"` (default) | ~150 tok | Signature + lines relevant to the query |
211
+ | `"full"` | ~300 tok | Signature + complete source body |
232
212
 
233
213
  ---
234
214
 
235
215
  ### `resolve_symbol`
236
216
 
237
- Looks up any symbol by exact name in O(1) — no search needed.
217
+ O(1) lookup by exact name — no search ranking needed.
238
218
 
239
219
  | Parameter | Type | Description |
240
220
  | :--- | :--- | :--- |
241
221
  | `symbol` | string | Exact function, class, or type name (e.g. `"validateToken"`) |
242
222
 
243
- Returns: definition location, type signature, docstring, type references, and cross-file topology. Faster than `search_code` for known names.
244
-
245
223
  ---
246
224
 
247
225
  ### `get_chunk`
248
226
 
249
- Returns the full source code of one chunk by its ID from search results.
227
+ Returns the source code of a single chunk by its ID.
250
228
 
251
229
  | Parameter | Type | Default | Description |
252
230
  | :--- | :--- | :--- | :--- |
253
- | `chunk_id` | string | required | ID shown in `search_code` results |
254
- | `view` | `"full"` \| `"signature"` | `"full"` | `"full"` = complete body; `"signature"` = first line only |
231
+ | `chunk_id` | string | required | ID from `search_code` results |
232
+ | `view` | `"full"` \| `"signature"` | `"full"` | Full body or first line only |
255
233
 
256
234
  ---
257
235
 
258
236
  ### `get_chunk_summary`
259
237
 
260
- Returns the interface of a chunk (signature + docstring + calls) without the body. About 50 tokens vs ~300 for the full body.
238
+ Signature + docstring + calls without the body. ~50 tokens vs ~300 for full.
261
239
 
262
240
  | Parameter | Type | Description |
263
241
  | :--- | :--- | :--- |
264
242
  | `chunk_id` | string | ID from `search_code` results |
265
243
 
266
- Use when you need to understand what a function does without reading its implementation.
267
-
268
244
  ---
269
245
 
270
246
  ### `get_file_skeleton`
271
247
 
272
- Returns all top-level exports and definitions in a file with line numbers — no bodies.
248
+ All top-level exports and definitions in a file with line numbers — no bodies.
273
249
 
274
250
  | Parameter | Type | Description |
275
251
  | :--- | :--- | :--- |
276
252
  | `file_path` | string | Relative path (e.g. `src/utils/auth.ts`) |
277
253
 
278
- Useful for understanding what a file exports before deciding which chunk to read.
279
-
280
254
  ---
281
255
 
282
256
  ### `get_call_graph`
283
257
 
284
- Finds all chunks that call a specific function by name.
258
+ Every chunk across the repo that calls a specific function.
285
259
 
286
260
  | Parameter | Type | Description |
287
261
  | :--- | :--- | :--- |
288
262
  | `target_function` | string | Exact function name (e.g. `"validateToken"`) |
289
263
 
290
- Always call this before modifying an exported function — it shows every call site repo-wide.
264
+ Call this before modifying any exported function to find all affected call sites.
291
265
 
292
266
  ---
293
267
 
294
268
  ### `get_repo_map`
295
269
 
296
- Returns a compact symbol map of the entire codebase ordered by importance (PageRank over the dependency graph — most-imported files first). Useful for orienting in an unfamiliar codebase in ~1,500 tokens.
270
+ Compact symbol map ordered by PageRank (most-imported files first). Orients agents in an unfamiliar codebase in ~1,500 tokens.
297
271
 
298
272
  | Parameter | Type | Default | Description |
299
273
  | :--- | :--- | :--- | :--- |
300
- | `path_filter` | string? | — | Only show files whose path contains this string |
274
+ | `path_filter` | string? | — | Limit to files whose path contains this string |
301
275
  | `max_files` | number | `80` | Maximum files to include |
302
276
  | `sort_by` | `"importance"` \| `"path"` | `"importance"` | Sort order |
303
277
 
@@ -305,13 +279,13 @@ Returns a compact symbol map of the entire codebase ordered by importance (PageR
305
279
 
306
280
  ### `list_index_stats`
307
281
 
308
- Returns index health: chunk count, file count, symbol table size, vector entry count, search mode (hybrid or lexical-only), daemon status, and index age.
282
+ Index health snapshot: chunk count, file count, symbol table size, vector count, search mode, daemon status, index age.
309
283
 
310
284
  ---
311
285
 
312
286
  ### `graph://dependencies/{file_path}` (resource)
313
287
 
314
- Returns the full bidirectional dependency topology for a file: what it imports and what imports it.
288
+ Full bidirectional dependency topology for a file: what it imports and what imports it.
315
289
 
316
290
  ---
317
291
 
@@ -321,68 +295,114 @@ Returns the full bidirectional dependency topology for a file: what it imports a
321
295
 
322
296
  | Variable | Default | Description |
323
297
  | :--- | :--- | :--- |
324
- | `MCP_PROJECT_ROOT` | `process.cwd()` | Project root for import resolution |
298
+ | `MCP_PROJECT_ROOT` | `process.cwd()` | Project root directory |
325
299
  | `OLLAMA_HOST` | `http://localhost:11434` | Ollama API endpoint |
326
300
  | `INDEXER_EMBEDDINGS` | — | Set to `off` to disable vector embeddings |
327
301
 
328
- ### Custom Ollama host
302
+ ### Language selection
329
303
 
330
- ```bash
331
- # Non-default port
332
- OLLAMA_HOST=http://localhost:11435 npm run mcp:index
304
+ `init` saves your selection to `.graph-indexer.json` in the project root:
305
+
306
+ ```json
307
+ { "languages": ["typescript", "javascript", "python"] }
308
+ ```
333
309
 
334
- # Remote host
335
- OLLAMA_HOST=http://192.168.1.100:11434 npm run mcp:index
310
+ Valid keys: `typescript`, `javascript`, `python`, `go`, `rust`, `php`, `java`, `kotlin`, `csharp`, `ruby`, `css`.
311
+
312
+ Update at any time by re-running `init`:
313
+
314
+ ```bash
315
+ npx graph-indexer init # Re-run interactive selector
316
+ npx graph-indexer init --all-languages # Enable all, no prompt
336
317
  ```
337
318
 
319
+ If `.graph-indexer.json` is absent or has no `languages` key, all installed parsers are used.
320
+
338
321
  ---
339
322
 
340
323
  ## Supported languages
341
324
 
342
- | Language | Extensions | Chunk types extracted |
325
+ | Language | Extensions | Chunks extracted |
343
326
  | :--- | :--- | :--- |
344
327
  | TypeScript / TSX | `.ts`, `.tsx` | functions, classes, methods, exports |
345
328
  | JavaScript | `.js`, `.jsx`, `.mjs`, `.cjs` | functions, classes, expressions |
346
- | Python | `.py` | function_definition, class_definition |
347
- | Go | `.go` | function_declaration, method, type |
329
+ | Python | `.py` | functions, classes |
330
+ | Go | `.go` | functions, methods, types |
348
331
  | Rust | `.rs` | fn, struct, enum, trait, impl |
349
332
  | Java | `.java` | class, method, interface, constructor, enum |
350
333
  | Kotlin | `.kt`, `.kts` | function, class, object, companion |
351
334
  | C# | `.cs` | class, method, interface, property, enum |
352
335
  | Ruby | `.rb` | method, class, module |
353
336
  | PHP | `.php` | function, class |
354
- | CSS / SCSS | `.css`, `.scss` | rule_set |
337
+ | CSS / SCSS | `.css`, `.scss` | rule sets |
355
338
 
356
339
  ---
357
340
 
358
- ## Requirements
341
+ ## How it works
342
+
343
+ ### Indexing
344
+
345
+ When you run `npm run mcp:index`:
346
+
347
+ 1. **Parses** every source file with Tree-sitter — no regex, exact AST boundaries.
348
+ 2. **Extracts chunks**: named functions, classes, methods, and exports, each with name, docstring, parameters, return type, call sites, and type references.
349
+ 3. **Builds a dependency graph**: bidirectional import map so each chunk knows what it imports and what imports it.
350
+ 4. **Creates two indexes**: a BM25 inverted index (lexical) and per-chunk float32 embeddings via Ollama `nomic-embed-text` (optional).
351
+ 5. **Serializes to disk**: `code-index.json` + `code-index.embeddings.bin`.
352
+
353
+ A background watcher daemon re-indexes changed files automatically.
359
354
 
360
- | | |
361
- | :--- | :--- |
362
- | **Node.js** | v18+ (ES Modules) |
363
- | **Ollama** | Optional — enables semantic search. Pull `nomic-embed-text`. |
364
- | **Disk** | `code-index.json` + `code-index.embeddings.bin` (both gitignored by default) |
355
+ ### Search pipeline
356
+
357
+ ```
358
+ Source files
359
+ │
360
+ ▼
361
+ Tree-sitter AST
362
+ │
363
+ ├──► Chunks ──► BM25 inverted index ──┐
364
+ │ └──► Float32 embeddings ──┤── RRF fusion ──► Ranked results ──► MCP tools
365
+ │ │
366
+ └──► Dependency graph ────────────────┘
367
+ ```
368
+
369
+ 1. **BM25 lexical search** — O(query_terms) over the inverted index.
370
+ 2. **Vector search** — cosine similarity over pre-normalized float32 embeddings.
371
+ 3. **RRF fusion** merges both lists (lexical 1.5×, vector 1.0×).
372
+ 4. **Score boosts**: exact name match +2.0×, suffix match +1.4×, path token +1.4×, test files demoted.
373
+
374
+ ---
375
+
376
+ ## Best practices
377
+
378
+ **Write descriptive docstrings.** Docstrings are embedded alongside code — better documentation directly improves semantic search quality.
379
+
380
+ **Use specific function names.** Names like `fetchAndCacheUserProfile` trigger the exact-name boost (2.0×) and reliably appear as rank-1 results when queried by name.
381
+
382
+ **Export named functions, not anonymous ones.** Anonymous default exports can't be targeted by `resolve_symbol` and rank lower in name-boost scoring.
383
+
384
+ **Avoid catch-all utility files.** Files that mix unrelated utilities produce weaker per-chunk signals. One responsibility per module indexes more cleanly.
365
385
 
366
386
  ---
367
387
 
368
388
  ## Troubleshooting
369
389
 
370
- **Index takes too long to build**
390
+ **Index takes too long**
371
391
  - Verify Ollama is running: `curl http://localhost:11434/api/tags`
372
- - Fall back to lexical-only: `INDEXER_EMBEDDINGS=off npm run mcp:index`
392
+ - Use lexical-only mode: `INDEXER_EMBEDDINGS=off npm run mcp:index`
373
393
 
374
394
  **Search returns irrelevant results**
375
395
  - Use `exact_tokens` for known symbol names
376
- - Verify the chunk is indexed with `list_index_stats()`
396
+ - Check coverage with `list_index_stats()`
377
397
  - Increase `top_k` to see more candidates
378
398
 
379
399
  **MCP server won't connect**
380
400
  - Verify `MCP_PROJECT_ROOT` points to the indexed directory
381
- - Check that `code-index.json` exists: `ls -la code-index.json`
382
- - Test the server manually: `node mcp-server.mjs`
401
+ - Check `code-index.json` exists: `ls -la code-index.json`
402
+ - Test manually: `npm run mcp:start`
383
403
 
384
404
  **Results are stale after file changes**
385
- - The background daemon auto-updates; if changes aren't reflected, run `npm run mcp:index` manually
405
+ - The daemon auto-updates; if stale, run `npm run mcp:index` manually
386
406
  - Check daemon status with `list_index_stats()`
387
407
 
388
408
  ---
@@ -400,18 +420,6 @@ npm run mcp:start # start MCP server
400
420
 
401
421
  ---
402
422
 
403
- ## Best practices for better search quality
404
-
405
- **Write descriptive docstrings.** The indexer embeds docstrings alongside code. Better documentation directly improves semantic search quality.
406
-
407
- **Use specific function names.** Functions with exact, descriptive names (e.g. `fetchAndCacheUserProfile`) hit the exact-name boost (2.0×) and return as rank-1 results when queried by name.
408
-
409
- **Export named functions, not anonymous ones.** Anonymous default exports cannot be targeted by `resolve_symbol` and score lower in name-boost ranking.
410
-
411
- **Avoid catch-all utility files.** Large files that mix unrelated utilities produce weaker per-chunk signals. One responsibility per module indexes more cleanly.
412
-
413
- ---
414
-
415
423
  ## License
416
424
 
417
425
  Released under the [MIT License](LICENSE). Copyright (c) 2026 MaquinaTech.