graph-indexer 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/PROMPT.md ADDED
@@ -0,0 +1,191 @@
1
+ # System Instructions โ€” graph-indexer MCP
2
+
3
+ You are connected to **graph-indexer**, a high-precision, in-memory AST code indexer exposed as an MCP server. This codebase has been fully indexed: every function, class, method, and component is a searchable chunk with its bidirectional dependency graph pre-computed.
4
+
5
+ > **Core Principle:** You have a surgical search tool. Use it like a scalpel, not a shovel. Never read files blindly. Every action starts with `search_code`.
6
+
7
+ ---
8
+
9
+ ## ๐Ÿ›‘ MANDATORY RULES โ€” Never Violate These
10
+
11
+ ### Rule 1: search_code FIRST, ALWAYS
12
+ - **NEVER** read a file, list a directory, or glob for files as a first step.
13
+ - **ALWAYS** call `search_code` before touching any file.
14
+ - Only read a file directly when `search_code` has already returned the exact `file_path` and you need surrounding context beyond the returned chunk.
15
+
16
+ ### Rule 2: One Tool Per Intent
17
+ - `search_code` โ†’ discover *what* exists and *where*
18
+ - `get_chunk("id")` โ†’ read the *full body* of one specific chunk
19
+ - `get_file_skeleton("path")` โ†’ inspect a *file's structure* without reading all code
20
+ - `get_call_graph("fnName")` โ†’ find *every caller* before refactoring
21
+ - `graph://dependencies/path` โ†’ inspect *import topology* of a file
22
+
23
+ Do not mix intents. Do not call `search_code` when you already have the chunk ID โ€” use `get_chunk` instead.
24
+
25
+ ### Rule 3: Topology Before Editing
26
+ Before modifying any function, component, or interface:
27
+ 1. Call `get_call_graph("functionName")` to find all callers across the repo.
28
+ 2. Check the `โฌ†๏ธ Used by` field in the `search_code` response.
29
+ 3. Only then proceed with the edit โ€” and update all callers if the signature changes.
30
+
31
+ ### Rule 4: Respect Token Budget
32
+ - Default `top_k = 5`. Do not raise it above 10 without a specific reason.
33
+ - Use `token_budget` when you need more complete code bodies; omit it when signature cards suffice.
34
+ - Prefer `get_chunk("id")` for deep dives over raising `top_k`.
35
+
36
+ ---
37
+
38
+ ## ๐Ÿ”ง Tool Reference
39
+
40
+ ### `search_code` โ€” Primary Search Tool
41
+
42
+ ```
43
+ search_code(
44
+ query: string, // REQUIRED. Describe the concept or logic in natural language.
45
+ exact_tokens?: string, // OPTIONAL. Exact symbol name(s) to boost. Critical for known names.
46
+ top_k?: number, // Default 5. Max 20.
47
+ min_score?: number, // Default 0.3. Lower to 0.0 to include all lexical results.
48
+ include_topology?: bool, // Default true. Always keep true โ€” topology is critical context.
49
+ token_budget?: number, // Optional. Estimated tokens for code bodies (1 tok โ‰ˆ 4 chars).
50
+ )
51
+ ```
52
+
53
+ **When to use `exact_tokens`:**
54
+ - You know the exact name: `exact_tokens: "AuthProvider"` โ†’ guaranteed rank-1
55
+ - Searching for a specific hook, function, or constant by name
56
+ - When the semantic query alone might return related but not exact results
57
+
58
+ **Query writing guide:**
59
+ | Intent | Good Query | Bad Query |
60
+ |--------|-----------|-----------|
61
+ | Find auth middleware | `"JWT token validation middleware"` | `"auth"` |
62
+ | Find a specific hook | `"hook for managing trip itinerary state"` | `"useTripItineraries"` |
63
+ | Find store actions | `"zustand store for notification state management"` | `"notifications"` |
64
+ | Find UI component | `"animated button with ripple press feedback"` | `"button component"` |
65
+ | Find error handler | `"global error boundary and crash reporting"` | `"error"` |
66
+
67
+ **Reading the response:**
68
+ ```
69
+ #1 ยท functionName [node_type]
70
+ ๐Ÿ“„ src/path/file.ts:12โ€“45 ยท ID: `abc123` ยท RRF: 0.0312
71
+ ๐Ÿ’ฌ First line of JSDoc docstring...
72
+ โฌ‡๏ธ Deps: src/utils/jwt.ts [getJWT, decodeToken] | src/db/session.ts [getSession]
73
+ โฌ†๏ธ Used by: src/routes/api.ts, src/app.ts
74
+ ๐Ÿ”— Calls: validateToken, refreshSession, logEvent
75
+ โ†ฉ๏ธ Expand body: get_chunk("abc123")
76
+ ```
77
+
78
+ - `RRF` is the relevance score (higher = more relevant)
79
+ - `Deps` shows what this file imports (with key symbols)
80
+ - `Used by` shows what imports this file โ€” READ THIS before editing
81
+ - `Calls` shows what functions this chunk calls internally
82
+ - **Always prefer `get_chunk("id")` over reading the whole file**
83
+
84
+ ---
85
+
86
+ ### `get_chunk` โ€” Full Code Retrieval
87
+
88
+ Use after `search_code` returns an `ID`. Returns full source body + topology.
89
+
90
+ ```
91
+ get_chunk(chunk_id: "abc123def456")
92
+ ```
93
+
94
+ Returns: complete source code, exact line range, all imports/exporters, call list.
95
+
96
+ ---
97
+
98
+ ### `get_file_skeleton` โ€” File Structure Overview
99
+
100
+ Use when you need to understand what a *file* contains without reading all code:
101
+
102
+ ```
103
+ get_file_skeleton(file_path: "src/stores/authStore.ts")
104
+ ```
105
+
106
+ Returns: list of all functions, classes, hooks, and types in the file โ€” names + line numbers only.
107
+ Costs ~50 tokens instead of thousands for the full file.
108
+
109
+ ---
110
+
111
+ ### `get_call_graph` โ€” Impact Analysis Before Refactoring
112
+
113
+ Use **before** modifying any exported function:
114
+
115
+ ```
116
+ get_call_graph(target_function: "validateToken")
117
+ ```
118
+
119
+ Returns: every chunk across the repo that calls this function, with file paths and line numbers.
120
+ If it returns callers, you MUST update them when changing the signature.
121
+
122
+ ---
123
+
124
+ ### `graph://dependencies/{file_path}` โ€” Topology Resource
125
+
126
+ Read the full import graph of a file in one call:
127
+
128
+ ```
129
+ URI: graph://dependencies/src/contexts/AuthContext.tsx
130
+ ```
131
+
132
+ Returns: all files this file imports, and all files that import this file. Use for understanding a module's place in the architecture before a major refactor.
133
+
134
+ ---
135
+
136
+ ## ๐Ÿ”„ Standard Workflows
137
+
138
+ ### Workflow A: Answer a Question About the Code
139
+ 1. `search_code(query: "...")` โ†’ identify the relevant chunk
140
+ 2. `get_chunk("id")` โ†’ read the full code if needed
141
+ 3. Answer from the returned context. Do not read other files speculatively.
142
+
143
+ ### Workflow B: Implement a New Feature
144
+ 1. `search_code` โ†’ find where to add the new code (nearest component/module)
145
+ 2. `get_file_skeleton` โ†’ understand the file structure before editing
146
+ 3. `get_call_graph` โ†’ check if any shared utilities you'll use are already called elsewhere
147
+ 4. Implement in the correct file. Do not create new files unless no existing one fits.
148
+
149
+ ### Workflow C: Refactor an Existing Function
150
+ 1. `search_code(exact_tokens: "functionName")` โ†’ locate the function
151
+ 2. `get_call_graph("functionName")` โ†’ find ALL callers
152
+ 3. `get_chunk("id")` โ†’ read the current implementation
153
+ 4. Modify the function AND update all callers identified in step 2.
154
+
155
+ ### Workflow D: Debug an Error
156
+ 1. Extract the exact error message / function name from the stack trace
157
+ 2. `search_code(query: "<error description>", exact_tokens: "<functionName>")` โ†’ find the source
158
+ 3. `get_chunk("id")` โ†’ read the code
159
+ 4. `get_call_graph("functionName")` โ†’ trace upstream if the bug is in a caller
160
+ 5. Fix and propagate.
161
+
162
+ ### Workflow E: Understand a File's Role
163
+ 1. `get_file_skeleton("src/path/file.ts")` โ†’ see all exports
164
+ 2. `graph://dependencies/src/path/file.ts` โ†’ see what it depends on and what depends on it
165
+ 3. Use this information to answer architecture questions without reading the full file.
166
+
167
+ ---
168
+
169
+ ## โš ๏ธ What NOT To Do
170
+
171
+ | Forbidden Action | Correct Alternative |
172
+ |-----------------|---------------------|
173
+ | `readFile("src/components/Button.tsx")` without search first | `search_code("button component with press animation")` |
174
+ | `listDir("src/stores")` to find a store | `search_code("zustand store for auth state")` |
175
+ | `search_code(top_k: 20)` to "see everything" | Search with a specific query; raise top_k only if 5 results miss the target |
176
+ | Editing a function without checking callers | `get_call_graph("fnName")` first |
177
+ | Re-running `search_code` with the same query | Refine the query or use `exact_tokens` |
178
+ | Reading a full file to understand its structure | `get_file_skeleton("path")` |
179
+
180
+ ---
181
+
182
+ ## ๐Ÿ“ Mental Model
183
+
184
+ Think of `graph-indexer` as a **surgical map of the codebase**:
185
+ - Every function is a GPS coordinate (chunk ID + file:line)
186
+ - `search_code` is the navigator โ€” it gives you the route
187
+ - `get_chunk` is the zoom-in โ€” it shows the details of one location
188
+ - `get_call_graph` is the traffic analysis โ€” it shows who depends on the road you're about to repave
189
+ - Topology (`โฌ‡๏ธ Deps` / `โฌ†๏ธ Used by`) is the city map โ€” shows how locations connect
190
+
191
+ You have a map. Never wander blindly.
package/README.md ADDED
@@ -0,0 +1,512 @@
1
+ <h1 align="center">graph-indexer</h1>
2
+
3
+ <p align="center">
4
+ <strong>Zero-DB ยท Air-Gapped ยท AST-Precision ยท Hybrid RRF Search</strong><br>
5
+ <em>The production-grade MCP code indexer for AI coding agents โ€” surgical context retrieval that cuts token costs by up to 90%.</em>
6
+ </p>
7
+
8
+ <p align="center">
9
+ <a href="https://www.npmjs.com/package/graph-indexer"><img src="https://img.shields.io/npm/v/graph-indexer?color=blue&style=flat-square" alt="npm version"></a>
10
+ <a href="LICENSE"><img src="https://img.shields.io/badge/License-GPL--3.0-blue.svg?style=flat-square" alt="License: GPL-3.0"></a>
11
+ <a href="https://nodejs.org"><img src="https://img.shields.io/badge/Node.js-%3E%3D18-green?style=flat-square" alt="Node.js 18+"></a>
12
+ </p>
13
+
14
+ ---
15
+
16
+ ## The Problem: AI Agents Are Context-Blind and Wasteful
17
+
18
+ When you use an AI coding agent (Claude, Cursor, GitHub Copilot) on a real codebase, it operates with **brute-force context**: it reads entire filesโ€”often 500โ€“1000 linesโ€”just to find one function. This has three compounding costs:
19
+
20
+ 1. **Token burn:** A single `readFile` on an 800-line file consumes ~10,000 tokens of context window. At GPT-4 pricing, that's money per keystroke.
21
+ 2. **Lost in the Middle:** LLMs reliably lose accuracy when the relevant code is buried in a large context. More code โ‰  better answers.
22
+ 3. **Topology blindness:** File reads give you code, not structure. The agent can't see that `fileA.ts` calls a function from `fileB.ts` unless it reads both. Refactors break silently.
23
+
24
+ ## The Solution: Surgical, AST-Precise Retrieval
25
+
26
+ `graph-indexer` is a local MCP server that gives your AI agent a **structural map** of the entire codebase instead of raw file access. It works in three steps:
27
+
28
+ 1. **Parse**: Tree-sitter builds an AST of every source file. Every function, class, method, and interface is extracted as a discrete, named chunk with exact line numbers.
29
+ 2. **Index**: Each chunk is indexed in two parallel structures โ€” a TF-IDF inverted index for lexical search and a `Float32Array` vector matrix for semantic (embedding) search.
30
+ 3. **Serve**: The MCP server exposes tools that let the agent retrieve the exact chunk it needs in a single call, receiving ~100 tokens of surgical context instead of ~10,000 tokens of a full file.
31
+
32
+ The result is **up to 90% reduction in token consumption** with higher code accuracy because the LLM processes only focused, relevant context.
33
+
34
+ ---
35
+
36
+ ## Architecture
37
+
38
+ ```mermaid
39
+ flowchart LR
40
+ subgraph INPUT["Input Layer"]
41
+ FS["Source Files\n(.ts .js .py .rs .go .css)"]
42
+ GIT[".gitignore Filter\n(ignore library)"]
43
+ end
44
+
45
+ subgraph PARSE["AST Layer โ€” Tree-sitter"]
46
+ AST["Language Grammar\nParser (6 languages)"]
47
+ CHUNKS["Semantic Chunk\nExtractor\n(functions ยท classes ยท impls)"]
48
+ DEPS["Bidirectional Import\nExtractor"]
49
+ end
50
+
51
+ subgraph INDEX["Dual Indexing Layer"]
52
+ EMBED["Ollama\nnomic-embed-text\nโ†’ Float32Array matrix"]
53
+ TFIDF["TF-IDF Inverted\nIndex (sublinear tf)"]
54
+ GRAPH["Dependency Graph\nimportedBy โ†” imports"]
55
+ end
56
+
57
+ subgraph SEARCH["Hybrid RRF Engine"]
58
+ VEC["Vector Search\n(cosine ยท flat/HNSW)"]
59
+ LEX["Lexical Search\n(TF-IDF ยท CamelCase-aware)"]
60
+ RRF["Reciprocal Rank\nFusion K=60"]
61
+ BOOST["Exact-Name\nBoost"]
62
+ end
63
+
64
+ subgraph MCP["MCP Protocol Layer (stdio)"]
65
+ SC["search_code\ntok-budget aware"]
66
+ GC["get_chunk"]
67
+ GFS["get_file_skeleton"]
68
+ GCG["get_call_graph"]
69
+ RES["graph:// resources"]
70
+ end
71
+
72
+ FS --> GIT --> AST --> CHUNKS & DEPS
73
+ CHUNKS --> EMBED & TFIDF
74
+ DEPS --> GRAPH
75
+ EMBED --> VEC
76
+ TFIDF --> LEX
77
+ VEC & LEX --> RRF --> BOOST --> SC & GC
78
+ GRAPH --> RES & GFS & GCG
79
+ ```
80
+
81
+ ---
82
+
83
+ ## Why This Beats Standard RAG
84
+
85
+ | Feature | **graph-indexer** | Standard RAG (ChromaDB + text chunks) |
86
+ | :--- | :--- | :--- |
87
+ | **Chunking** | AST-precise: entire functions/classes, never split mid-logic | Naive token-count splits break code in half |
88
+ | **Infrastructure** | Zero-DB โ€” pure in-memory `Map` + `Float32Array` | External vector DB (Chroma, Pinecone, Weaviate) |
89
+ | **Privacy** | 100% air-gapped (local Ollama or lexical-only mode) | Often requires cloud embedding APIs |
90
+ | **Search quality** | Hybrid RRF fusing dense vectors + sparse TF-IDF | Dense-only or BM25-only |
91
+ | **Dependency context** | Bidirectional AST topology for every result | None |
92
+ | **Fault tolerance** | Graceful degradation to pure TF-IDF if Ollama is offline | Hard failure if embedding service is unavailable |
93
+ | **Latency** | Sub-millisecond per query (pure V8 `Float32Array`) | Network RTT + DB overhead |
94
+ | **Languages** | JS/TS/Python/Rust/Go/PHP/CSS | Language-agnostic (no topology) |
95
+
96
+ ---
97
+
98
+ ## Quick Start
99
+
100
+ ### 1. Install
101
+
102
+ ```bash
103
+ npm install graph-indexer --save-dev
104
+ ```
105
+
106
+ Add scripts to your project's `package.json`:
107
+
108
+ ```json
109
+ "scripts": {
110
+ "mcp:index": "idx-index --repo .",
111
+ "mcp:watch": "idx-watch",
112
+ "mcp:start": "idx-mcp"
113
+ }
114
+ ```
115
+
116
+ ### 2. Index Your Repository
117
+
118
+ ```bash
119
+ npm run mcp:index
120
+ ```
121
+
122
+ This scans every source file, builds the AST, generates embeddings via Ollama (if available), and writes `code-index.json` + `code-index.embeddings.bin`.
123
+
124
+ ### 3. Start the MCP Server
125
+
126
+ ```bash
127
+ npm run mcp:start
128
+ ```
129
+
130
+ Point your MCP client at this process. The server spawns the file watcher daemon automatically.
131
+
132
+ ### 4. (Optional) Run the File Watcher Manually
133
+
134
+ For persistent incremental updates in a separate terminal:
135
+
136
+ ```bash
137
+ npm run mcp:watch
138
+ ```
139
+
140
+ ---
141
+
142
+ ## System Requirements
143
+
144
+ | Requirement | Details |
145
+ | :--- | :--- |
146
+ | **Node.js** | v18+ (ES Modules) |
147
+ | **Ollama** | Optional. Runs locally for embedding generation. Pull `nomic-embed-text` to enable semantic search. |
148
+ | **Disk** | `code-index.json` (metadata) + `code-index.embeddings.bin` (binary float32 vectors). Both are gitignored by default. |
149
+
150
+ ### Ollama Setup (Optional but Recommended)
151
+
152
+ ```bash
153
+ # Install Ollama: https://ollama.ai
154
+ ollama pull nomic-embed-text
155
+ npm run mcp:index # now with semantic vectors
156
+ ```
157
+
158
+ ### Lexical-Only Mode (No Ollama Required)
159
+
160
+ ```bash
161
+ INDEXER_EMBEDDINGS=off npm run mcp:index
162
+ ```
163
+
164
+ Recall@3 remains 100% for exact-name queries. Semantic recall (concept-to-code) degrades to lexical-only; still effective for most programming tasks.
165
+
166
+ ### Custom Ollama Host
167
+
168
+ ```bash
169
+ # Non-standard port
170
+ OLLAMA_HOST=http://localhost:11435 npm run mcp:index
171
+
172
+ # Remote server
173
+ OLLAMA_HOST=http://192.168.1.100:11434 npm run mcp:index
174
+ ```
175
+
176
+ ---
177
+
178
+ ## IDE / Client Configuration
179
+
180
+ ### Claude Desktop (`claude_desktop_config.json`)
181
+
182
+ ```json
183
+ {
184
+ "mcpServers": {
185
+ "graph-indexer": {
186
+ "command": "node",
187
+ "args": ["/absolute/path/to/your/project/node_modules/graph-indexer/mcp-server.mjs"],
188
+ "env": {
189
+ "MCP_PROJECT_ROOT": "/absolute/path/to/your/project",
190
+ "OLLAMA_HOST": "http://localhost:11434"
191
+ }
192
+ }
193
+ }
194
+ }
195
+ ```
196
+
197
+ ### VS Code (`.vscode/mcp.json`)
198
+
199
+ ```json
200
+ {
201
+ "servers": {
202
+ "graph-indexer": {
203
+ "type": "stdio",
204
+ "command": "node",
205
+ "args": ["${workspaceFolder}/node_modules/graph-indexer/mcp-server.mjs"],
206
+ "env": {
207
+ "MCP_PROJECT_ROOT": "${workspaceFolder}",
208
+ "OLLAMA_HOST": "http://localhost:11434"
209
+ }
210
+ }
211
+ }
212
+ }
213
+ ```
214
+
215
+ ### Cursor (`.cursor/mcp.json`)
216
+
217
+ ```json
218
+ {
219
+ "mcpServers": {
220
+ "graph-indexer": {
221
+ "command": "node",
222
+ "args": ["${workspaceFolder}/node_modules/graph-indexer/mcp-server.mjs"],
223
+ "env": {
224
+ "MCP_PROJECT_ROOT": "${workspaceFolder}"
225
+ }
226
+ }
227
+ }
228
+ }
229
+ ```
230
+
231
+ ---
232
+
233
+ ## Agent System Prompt
234
+
235
+ Copy the contents of [`PROMPT.md`](./PROMPT.md) into your agent's system instructions. This document instructs the agent on:
236
+ - When to call which tool (strict priority order)
237
+ - How to write effective `query` and `exact_tokens` parameters
238
+ - Workflow patterns for Q&A, implementation, refactoring, and debugging
239
+ - What NOT to do (file reads without search first, raising `top_k` speculatively, etc.)
240
+
241
+ ---
242
+
243
+ ## MCP Tools Reference
244
+
245
+ ### `search_code`
246
+
247
+ Hybrid semantic + lexical search. Returns compact signature cards for all results, then fills remaining token budget with code bodies.
248
+
249
+ | Parameter | Type | Default | Description |
250
+ | :--- | :--- | :--- | :--- |
251
+ | `query` | `string` | โ€” | Natural language description of the logic to find |
252
+ | `exact_tokens` | `string?` | โ€” | Exact symbol name to guarantee rank-1 placement |
253
+ | `top_k` | `number` | `5` | Number of results (1โ€“20) |
254
+ | `min_score` | `number` | `0.3` | Minimum cosine similarity threshold |
255
+ | `token_budget` | `number?` | 1500 chars | Estimated token budget for code bodies |
256
+ | `include_topology` | `boolean` | `true` | Include `โฌ‡๏ธ Deps` / `โฌ†๏ธ Used by` / `๐Ÿ”— Calls` in output |
257
+
258
+ **Example response:**
259
+ ```
260
+ #1 ยท validateToken [function_declaration]
261
+ ๐Ÿ“„ src/utils/jwt.ts:14โ€“42 ยท ID: `a3f9c1b2` ยท RRF: 0.0321
262
+ ๐Ÿ’ฌ Validates and decodes a JWT access token. Throws on expiry.
263
+ โฌ‡๏ธ Deps: src/config/env.ts [JWT_SECRET] | src/utils/errors.ts [AuthError]
264
+ โฌ†๏ธ Used by: src/middleware/auth.ts, src/routes/api.ts
265
+ ๐Ÿ”— Calls: verify, decodePayload, throwIfExpired
266
+ โ†ฉ๏ธ Expand body: get_chunk("a3f9c1b2")
267
+ ```
268
+
269
+ ---
270
+
271
+ ### `get_chunk`
272
+
273
+ Retrieves the full source body of one chunk by ID. Use this after `search_code` instead of reading the whole file.
274
+
275
+ | Parameter | Type | Description |
276
+ | :--- | :--- | :--- |
277
+ | `chunk_id` | `string` | The ID shown in `search_code` results |
278
+
279
+ ---
280
+
281
+ ### `get_file_skeleton`
282
+
283
+ Returns only the names and line ranges of all functions/classes in a file โ€” no code bodies. ~50 tokens instead of thousands.
284
+
285
+ | Parameter | Type | Description |
286
+ | :--- | :--- | :--- |
287
+ | `file_path` | `string` | Relative path (e.g. `src/components/Button.tsx`) |
288
+
289
+ ---
290
+
291
+ ### `get_call_graph`
292
+
293
+ Finds every chunk in the repo that calls a specific function by name. Critical before any signature change.
294
+
295
+ | Parameter | Type | Description |
296
+ | :--- | :--- | :--- |
297
+ | `target_function` | `string` | Exact function name (e.g. `validateToken`) |
298
+
299
+ ---
300
+
301
+ ### `graph://dependencies/{file_path}` Resource
302
+
303
+ Fetches the full bidirectional dependency topology for a file without consuming any search quota.
304
+
305
+ ```
306
+ URI: graph://dependencies/src/middleware/auth.ts
307
+ ```
308
+
309
+ Returns which files this file imports, and which files import it.
310
+
311
+ ---
312
+
313
+ ## Supported Languages
314
+
315
+ | Language | Extensions | Chunk Types | Import Resolution |
316
+ | :--- | :--- | :--- | :--- |
317
+ | TypeScript / TSX | `.ts`, `.tsx` | functions, classes, methods, exports | Absolute & relative paths |
318
+ | JavaScript | `.js`, `.jsx`, `.mjs`, `.cjs` | functions, classes, expressions | Absolute & relative paths |
319
+ | Python | `.py` | function_definition, class_definition | Relative imports (`.`, `..`) โ†’ file paths |
320
+ | Rust | `.rs` | fn, struct, enum, trait, impl | `crate::` path resolution |
321
+ | Go | `.go` | function_declaration, method, type | Import spec resolution |
322
+ | PHP | `.php` | function, class | include/require paths |
323
+ | CSS / SCSS | `.css`, `.scss` | rule_set | โ€” |
324
+
325
+ ---
326
+
327
+ ## Performance & Evaluation
328
+
329
+ Validate indexer quality on your own codebase:
330
+
331
+ ```bash
332
+ npm run test
333
+ # or with custom Ollama:
334
+ OLLAMA_HOST=http://localhost:11435 npm run test
335
+ ```
336
+
337
+ ### Benchmark Results
338
+
339
+ | Scenario | Recall@3 | Recall@5 | Latency (p50) |
340
+ | :--- | :--- | :--- | :--- |
341
+ | graph-indexer own codebase (34 chunks, lexical-only) | **100%** | 100% | 0.07 ms |
342
+ | graph-indexer own codebase (34 chunks, with Ollama) | **75%** | 100% | 0.14 ms |
343
+ | React Native project (311 chunks, with Ollama) | **58%** | 75% | 1.02 ms |
344
+
345
+ > The React Native project numbers are without any JSDoc in the source files. Adding docstrings to exported functions (see Best Practices below) brings recall to **100% @3**.
346
+
347
+ ### Vector Search Performance
348
+
349
+ | Corpus size | Pure JS flat scan | HNSW (auto-activated โ‰ฅ5k) |
350
+ | :--- | :--- | :--- |
351
+ | 1,000 chunks (dim=768) | 0.9 ms/query | โ€” |
352
+ | 5,000 chunks (dim=768) | 4.5 ms/query | 1.2 ms/query |
353
+ | 20,000 chunks (dim=768) | 30.5 ms/query | 3.1 ms/query |
354
+
355
+ ---
356
+
357
+ ## Best Practices for 100% Recall
358
+
359
+ ### 1. Add JSDoc/TSDoc to Exported Functions
360
+
361
+ The indexer embeds your docstrings alongside code. Docstrings dramatically improve semantic search:
362
+
363
+ ```typescript
364
+ /**
365
+ * Validates and decodes a JWT access token.
366
+ * Throws AuthError if the token is expired or malformed.
367
+ *
368
+ * @param token - Raw JWT string from Authorization header
369
+ * @returns Decoded payload with userId, role, and expiry
370
+ */
371
+ export function validateToken(token: string): TokenPayload { ... }
372
+ ```
373
+
374
+ ### 2. Use Descriptive, Specific Names
375
+
376
+ The exact-name boost gives a free rank-1 slot to any chunk whose name exactly matches `exact_tokens`. Descriptive names make this reliable:
377
+
378
+ ```typescript
379
+ // โœ… Exact-name searchable
380
+ export const useNotificationPermissionStatus = () => { ... };
381
+ export function calculateTripDurationInDays(start: Date, end: Date) { ... }
382
+
383
+ // โŒ Generic โ€” impossible to target without semantic search
384
+ export const util = () => { ... };
385
+ export function process(data: any) { ... }
386
+ ```
387
+
388
+ ### 3. Expand Minimal Utility Files
389
+
390
+ Files with fewer than ~5 indexable lines produce zero or one chunk. Add error handling and validation to make them indexable:
391
+
392
+ ```typescript
393
+ // โŒ 2 lines โ€” may not be indexed
394
+ export const Context = createContext(null);
395
+ export const useCtx = () => useContext(Context);
396
+
397
+ // โœ… Indexable with meaningful docstring + validation
398
+ /**
399
+ * Context for accessing authentication state throughout the component tree.
400
+ */
401
+ export const AuthContext = createContext<AuthState | null>(null);
402
+
403
+ /**
404
+ * Hook for consuming AuthContext. Throws if used outside AuthProvider.
405
+ */
406
+ export function useAuth(): AuthState {
407
+ const ctx = useContext(AuthContext);
408
+ if (!ctx) throw new Error('useAuth must be used inside <AuthProvider>');
409
+ return ctx;
410
+ }
411
+ ```
412
+
413
+ ### 4. Keep Imports Explicit and Organized
414
+
415
+ The topology graph follows import paths. Barrel re-exports from `index.ts` break dependency resolution โ€” prefer direct imports:
416
+
417
+ ```typescript
418
+ // โœ… Topology-friendly
419
+ import { useAuthStore } from '@/stores/authStore';
420
+ import { validateEmail } from '@/utils/validators';
421
+
422
+ // โŒ Breaks topology โ€” indexer can't resolve what's actually imported
423
+ import { useAuthStore, validateEmail } from '@/index';
424
+ ```
425
+
426
+ ---
427
+
428
+ ## File Structure
429
+
430
+ ```
431
+ graph-indexer/
432
+ โ”œโ”€โ”€ core-engine.mjs # In-memory index: vector matrix, TF-IDF, RRF search, save/load
433
+ โ”œโ”€โ”€ parser-utils.mjs # Tree-sitter AST parsing, 6 language grammars, Ollama embeddings
434
+ โ”œโ”€โ”€ indexer.mjs # CLI bootstrap: scan repo โ†’ parse โ†’ embed โ†’ write index
435
+ โ”œโ”€โ”€ watch-daemon.mjs # Chokidar daemon: incremental real-time index updates
436
+ โ”œโ”€โ”€ mcp-server.mjs # MCP stdio server: exposes tools + graph:// resources
437
+ โ”œโ”€โ”€ PROMPT.md # Agent system prompt โ€” paste into your IDE's agent instructions
438
+ โ”œโ”€โ”€ tests/
439
+ โ”‚ โ”œโ”€โ”€ eval-harness.mjs # Recall@k + latency evaluation against the indexer's own codebase
440
+ โ”‚ โ””โ”€โ”€ react-native-test.mjs # Real-world quality test on a 311-chunk React Native project
441
+ โ””โ”€โ”€ package.json
442
+ ```
443
+
444
+ ---
445
+
446
+ ## Mathematical Foundation
447
+
448
+ ### Sublinear TF Scaling
449
+
450
+ Term frequencies use sublinear scaling to prevent common tokens (`return`, `const`) from burying business-logic terms:
451
+
452
+ $$w(t,d) = 1 + \log(f_{t,d})$$
453
+
454
+ ### Reciprocal Rank Fusion
455
+
456
+ Vector and lexical results are merged by rank position, not raw incompatible scores:
457
+
458
+ $$\text{RRF}(d) = \sum_{i \in \{\text{vec, lex}\}} \frac{1}{K + \text{rank}_i(d)}, \quad K = 60$$
459
+
460
+ ### Exact-Name Boost
461
+
462
+ Any chunk whose name exactly matches `exact_tokens` receives an additive bonus equal to the maximum single-list RRF contribution:
463
+
464
+ $$\Delta\text{score} = \frac{1}{K + 1}$$
465
+
466
+ This guarantees rank-1 placement for any exact symbol name query.
467
+
468
+ ---
469
+
470
+ ## How the Index Files Work
471
+
472
+ | File | Purpose | Typical Size |
473
+ | :--- | :--- | :--- |
474
+ | `code-index.json` | Chunk metadata (names, file paths, docstrings, calls, graph) | ~200 KB / 1k chunks |
475
+ | `code-index.embeddings.bin` | Binary `float32` embedding vectors (4.8ร— smaller than JSON) | ~3 MB / 1k chunks @768-dim |
476
+
477
+ Both files are written atomically via a `tmp โ†’ rename` pattern to prevent corruption on crash. Both should be added to `.gitignore`.
478
+
479
+ ---
480
+
481
+ ## Environment Variables
482
+
483
+ | Variable | Default | Description |
484
+ | :--- | :--- | :--- |
485
+ | `OLLAMA_HOST` | `http://localhost:11434` | Ollama API base URL |
486
+ | `INDEXER_EMBEDDINGS` | โ€” | Set to `off` to disable embeddings entirely |
487
+ | `MCP_PROJECT_ROOT` | `process.cwd()` | Absolute path to the project being indexed |
488
+
489
+ ---
490
+
491
+ ## License & Intellectual Property
492
+
493
+ Released under the [GNU General Public License v3.0](LICENSE).
494
+
495
+ This license ensures:
496
+ - **Copyleft protection**: Any software that incorporates this code must also be released under GPL-3.0.
497
+ - **Attribution**: The original author (MaquinaTech) must be credited.
498
+ - **Source availability**: Any distributed modifications must include source code.
499
+
500
+ Commercial use is permitted but requires GPL-3.0 compliance. For proprietary licensing inquiries, contact the author.
501
+
502
+ ---
503
+
504
+ ## Author
505
+
506
+ Built and maintained by **MaquinaTech**.
507
+
508
+ - **GitHub:** [github.com/MaquinaTech](https://github.com/MaquinaTech)
509
+ - **NPM:** [npmjs.com/package/graph-indexer](https://www.npmjs.com/package/graph-indexer)
510
+ - **Issues:** [github.com/MaquinaTech/graph-indexer/issues](https://github.com/MaquinaTech/graph-indexer/issues)
511
+
512
+ Issues, pull requests, and feature discussions are welcome.