graph-indexer 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +677 -0
- package/PROMPT.md +191 -0
- package/README.md +512 -0
- package/core-engine.mjs +405 -0
- package/indexer.mjs +180 -0
- package/mcp-server.mjs +295 -0
- package/package.json +81 -0
- package/parser-utils.mjs +458 -0
- package/watch-daemon.mjs +136 -0
package/PROMPT.md
ADDED
|
@@ -0,0 +1,191 @@
|
|
|
1
|
+
# System Instructions โ graph-indexer MCP
|
|
2
|
+
|
|
3
|
+
You are connected to **graph-indexer**, a high-precision, in-memory AST code indexer exposed as an MCP server. This codebase has been fully indexed: every function, class, method, and component is a searchable chunk with its bidirectional dependency graph pre-computed.
|
|
4
|
+
|
|
5
|
+
> **Core Principle:** You have a surgical search tool. Use it like a scalpel, not a shovel. Never read files blindly. Every action starts with `search_code`.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## ๐ MANDATORY RULES โ Never Violate These
|
|
10
|
+
|
|
11
|
+
### Rule 1: search_code FIRST, ALWAYS
|
|
12
|
+
- **NEVER** read a file, list a directory, or glob for files as a first step.
|
|
13
|
+
- **ALWAYS** call `search_code` before touching any file.
|
|
14
|
+
- Only read a file directly when `search_code` has already returned the exact `file_path` and you need surrounding context beyond the returned chunk.
|
|
15
|
+
|
|
16
|
+
### Rule 2: One Tool Per Intent
|
|
17
|
+
- `search_code` โ discover *what* exists and *where*
|
|
18
|
+
- `get_chunk("id")` โ read the *full body* of one specific chunk
|
|
19
|
+
- `get_file_skeleton("path")` โ inspect a *file's structure* without reading all code
|
|
20
|
+
- `get_call_graph("fnName")` โ find *every caller* before refactoring
|
|
21
|
+
- `graph://dependencies/path` โ inspect *import topology* of a file
|
|
22
|
+
|
|
23
|
+
Do not mix intents. Do not call `search_code` when you already have the chunk ID โ use `get_chunk` instead.
|
|
24
|
+
|
|
25
|
+
### Rule 3: Topology Before Editing
|
|
26
|
+
Before modifying any function, component, or interface:
|
|
27
|
+
1. Call `get_call_graph("functionName")` to find all callers across the repo.
|
|
28
|
+
2. Check the `โฌ๏ธ Used by` field in the `search_code` response.
|
|
29
|
+
3. Only then proceed with the edit โ and update all callers if the signature changes.
|
|
30
|
+
|
|
31
|
+
### Rule 4: Respect Token Budget
|
|
32
|
+
- Default `top_k = 5`. Do not raise it above 10 without a specific reason.
|
|
33
|
+
- Use `token_budget` when you need more complete code bodies; omit it when signature cards suffice.
|
|
34
|
+
- Prefer `get_chunk("id")` for deep dives over raising `top_k`.
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## ๐ง Tool Reference
|
|
39
|
+
|
|
40
|
+
### `search_code` โ Primary Search Tool
|
|
41
|
+
|
|
42
|
+
```
|
|
43
|
+
search_code(
|
|
44
|
+
query: string, // REQUIRED. Describe the concept or logic in natural language.
|
|
45
|
+
exact_tokens?: string, // OPTIONAL. Exact symbol name(s) to boost. Critical for known names.
|
|
46
|
+
top_k?: number, // Default 5. Max 20.
|
|
47
|
+
min_score?: number, // Default 0.3. Lower to 0.0 to include all lexical results.
|
|
48
|
+
include_topology?: bool, // Default true. Always keep true โ topology is critical context.
|
|
49
|
+
token_budget?: number, // Optional. Estimated tokens for code bodies (1 tok โ 4 chars).
|
|
50
|
+
)
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
**When to use `exact_tokens`:**
|
|
54
|
+
- You know the exact name: `exact_tokens: "AuthProvider"` โ guaranteed rank-1
|
|
55
|
+
- Searching for a specific hook, function, or constant by name
|
|
56
|
+
- When the semantic query alone might return related but not exact results
|
|
57
|
+
|
|
58
|
+
**Query writing guide:**
|
|
59
|
+
| Intent | Good Query | Bad Query |
|
|
60
|
+
|--------|-----------|-----------|
|
|
61
|
+
| Find auth middleware | `"JWT token validation middleware"` | `"auth"` |
|
|
62
|
+
| Find a specific hook | `"hook for managing trip itinerary state"` | `"useTripItineraries"` |
|
|
63
|
+
| Find store actions | `"zustand store for notification state management"` | `"notifications"` |
|
|
64
|
+
| Find UI component | `"animated button with ripple press feedback"` | `"button component"` |
|
|
65
|
+
| Find error handler | `"global error boundary and crash reporting"` | `"error"` |
|
|
66
|
+
|
|
67
|
+
**Reading the response:**
|
|
68
|
+
```
|
|
69
|
+
#1 ยท functionName [node_type]
|
|
70
|
+
๐ src/path/file.ts:12โ45 ยท ID: `abc123` ยท RRF: 0.0312
|
|
71
|
+
๐ฌ First line of JSDoc docstring...
|
|
72
|
+
โฌ๏ธ Deps: src/utils/jwt.ts [getJWT, decodeToken] | src/db/session.ts [getSession]
|
|
73
|
+
โฌ๏ธ Used by: src/routes/api.ts, src/app.ts
|
|
74
|
+
๐ Calls: validateToken, refreshSession, logEvent
|
|
75
|
+
โฉ๏ธ Expand body: get_chunk("abc123")
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
- `RRF` is the relevance score (higher = more relevant)
|
|
79
|
+
- `Deps` shows what this file imports (with key symbols)
|
|
80
|
+
- `Used by` shows what imports this file โ READ THIS before editing
|
|
81
|
+
- `Calls` shows what functions this chunk calls internally
|
|
82
|
+
- **Always prefer `get_chunk("id")` over reading the whole file**
|
|
83
|
+
|
|
84
|
+
---
|
|
85
|
+
|
|
86
|
+
### `get_chunk` โ Full Code Retrieval
|
|
87
|
+
|
|
88
|
+
Use after `search_code` returns an `ID`. Returns full source body + topology.
|
|
89
|
+
|
|
90
|
+
```
|
|
91
|
+
get_chunk(chunk_id: "abc123def456")
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
Returns: complete source code, exact line range, all imports/exporters, call list.
|
|
95
|
+
|
|
96
|
+
---
|
|
97
|
+
|
|
98
|
+
### `get_file_skeleton` โ File Structure Overview
|
|
99
|
+
|
|
100
|
+
Use when you need to understand what a *file* contains without reading all code:
|
|
101
|
+
|
|
102
|
+
```
|
|
103
|
+
get_file_skeleton(file_path: "src/stores/authStore.ts")
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
Returns: list of all functions, classes, hooks, and types in the file โ names + line numbers only.
|
|
107
|
+
Costs ~50 tokens instead of thousands for the full file.
|
|
108
|
+
|
|
109
|
+
---
|
|
110
|
+
|
|
111
|
+
### `get_call_graph` โ Impact Analysis Before Refactoring
|
|
112
|
+
|
|
113
|
+
Use **before** modifying any exported function:
|
|
114
|
+
|
|
115
|
+
```
|
|
116
|
+
get_call_graph(target_function: "validateToken")
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
Returns: every chunk across the repo that calls this function, with file paths and line numbers.
|
|
120
|
+
If it returns callers, you MUST update them when changing the signature.
|
|
121
|
+
|
|
122
|
+
---
|
|
123
|
+
|
|
124
|
+
### `graph://dependencies/{file_path}` โ Topology Resource
|
|
125
|
+
|
|
126
|
+
Read the full import graph of a file in one call:
|
|
127
|
+
|
|
128
|
+
```
|
|
129
|
+
URI: graph://dependencies/src/contexts/AuthContext.tsx
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
Returns: all files this file imports, and all files that import this file. Use for understanding a module's place in the architecture before a major refactor.
|
|
133
|
+
|
|
134
|
+
---
|
|
135
|
+
|
|
136
|
+
## ๐ Standard Workflows
|
|
137
|
+
|
|
138
|
+
### Workflow A: Answer a Question About the Code
|
|
139
|
+
1. `search_code(query: "...")` โ identify the relevant chunk
|
|
140
|
+
2. `get_chunk("id")` โ read the full code if needed
|
|
141
|
+
3. Answer from the returned context. Do not read other files speculatively.
|
|
142
|
+
|
|
143
|
+
### Workflow B: Implement a New Feature
|
|
144
|
+
1. `search_code` โ find where to add the new code (nearest component/module)
|
|
145
|
+
2. `get_file_skeleton` โ understand the file structure before editing
|
|
146
|
+
3. `get_call_graph` โ check if any shared utilities you'll use are already called elsewhere
|
|
147
|
+
4. Implement in the correct file. Do not create new files unless no existing one fits.
|
|
148
|
+
|
|
149
|
+
### Workflow C: Refactor an Existing Function
|
|
150
|
+
1. `search_code(exact_tokens: "functionName")` โ locate the function
|
|
151
|
+
2. `get_call_graph("functionName")` โ find ALL callers
|
|
152
|
+
3. `get_chunk("id")` โ read the current implementation
|
|
153
|
+
4. Modify the function AND update all callers identified in step 2.
|
|
154
|
+
|
|
155
|
+
### Workflow D: Debug an Error
|
|
156
|
+
1. Extract the exact error message / function name from the stack trace
|
|
157
|
+
2. `search_code(query: "<error description>", exact_tokens: "<functionName>")` โ find the source
|
|
158
|
+
3. `get_chunk("id")` โ read the code
|
|
159
|
+
4. `get_call_graph("functionName")` โ trace upstream if the bug is in a caller
|
|
160
|
+
5. Fix and propagate.
|
|
161
|
+
|
|
162
|
+
### Workflow E: Understand a File's Role
|
|
163
|
+
1. `get_file_skeleton("src/path/file.ts")` โ see all exports
|
|
164
|
+
2. `graph://dependencies/src/path/file.ts` โ see what it depends on and what depends on it
|
|
165
|
+
3. Use this information to answer architecture questions without reading the full file.
|
|
166
|
+
|
|
167
|
+
---
|
|
168
|
+
|
|
169
|
+
## โ ๏ธ What NOT To Do
|
|
170
|
+
|
|
171
|
+
| Forbidden Action | Correct Alternative |
|
|
172
|
+
|-----------------|---------------------|
|
|
173
|
+
| `readFile("src/components/Button.tsx")` without search first | `search_code("button component with press animation")` |
|
|
174
|
+
| `listDir("src/stores")` to find a store | `search_code("zustand store for auth state")` |
|
|
175
|
+
| `search_code(top_k: 20)` to "see everything" | Search with a specific query; raise top_k only if 5 results miss the target |
|
|
176
|
+
| Editing a function without checking callers | `get_call_graph("fnName")` first |
|
|
177
|
+
| Re-running `search_code` with the same query | Refine the query or use `exact_tokens` |
|
|
178
|
+
| Reading a full file to understand its structure | `get_file_skeleton("path")` |
|
|
179
|
+
|
|
180
|
+
---
|
|
181
|
+
|
|
182
|
+
## ๐ Mental Model
|
|
183
|
+
|
|
184
|
+
Think of `graph-indexer` as a **surgical map of the codebase**:
|
|
185
|
+
- Every function is a GPS coordinate (chunk ID + file:line)
|
|
186
|
+
- `search_code` is the navigator โ it gives you the route
|
|
187
|
+
- `get_chunk` is the zoom-in โ it shows the details of one location
|
|
188
|
+
- `get_call_graph` is the traffic analysis โ it shows who depends on the road you're about to repave
|
|
189
|
+
- Topology (`โฌ๏ธ Deps` / `โฌ๏ธ Used by`) is the city map โ shows how locations connect
|
|
190
|
+
|
|
191
|
+
You have a map. Never wander blindly.
|
package/README.md
ADDED
|
@@ -0,0 +1,512 @@
|
|
|
1
|
+
<h1 align="center">graph-indexer</h1>
|
|
2
|
+
|
|
3
|
+
<p align="center">
|
|
4
|
+
<strong>Zero-DB ยท Air-Gapped ยท AST-Precision ยท Hybrid RRF Search</strong><br>
|
|
5
|
+
<em>The production-grade MCP code indexer for AI coding agents โ surgical context retrieval that cuts token costs by up to 90%.</em>
|
|
6
|
+
</p>
|
|
7
|
+
|
|
8
|
+
<p align="center">
|
|
9
|
+
<a href="https://www.npmjs.com/package/graph-indexer"><img src="https://img.shields.io/npm/v/graph-indexer?color=blue&style=flat-square" alt="npm version"></a>
|
|
10
|
+
<a href="LICENSE"><img src="https://img.shields.io/badge/License-GPL--3.0-blue.svg?style=flat-square" alt="License: GPL-3.0"></a>
|
|
11
|
+
<a href="https://nodejs.org"><img src="https://img.shields.io/badge/Node.js-%3E%3D18-green?style=flat-square" alt="Node.js 18+"></a>
|
|
12
|
+
</p>
|
|
13
|
+
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
## The Problem: AI Agents Are Context-Blind and Wasteful
|
|
17
|
+
|
|
18
|
+
When you use an AI coding agent (Claude, Cursor, GitHub Copilot) on a real codebase, it operates with **brute-force context**: it reads entire filesโoften 500โ1000 linesโjust to find one function. This has three compounding costs:
|
|
19
|
+
|
|
20
|
+
1. **Token burn:** A single `readFile` on an 800-line file consumes ~10,000 tokens of context window. At GPT-4 pricing, that's money per keystroke.
|
|
21
|
+
2. **Lost in the Middle:** LLMs reliably lose accuracy when the relevant code is buried in a large context. More code โ better answers.
|
|
22
|
+
3. **Topology blindness:** File reads give you code, not structure. The agent can't see that `fileA.ts` calls a function from `fileB.ts` unless it reads both. Refactors break silently.
|
|
23
|
+
|
|
24
|
+
## The Solution: Surgical, AST-Precise Retrieval
|
|
25
|
+
|
|
26
|
+
`graph-indexer` is a local MCP server that gives your AI agent a **structural map** of the entire codebase instead of raw file access. It works in three steps:
|
|
27
|
+
|
|
28
|
+
1. **Parse**: Tree-sitter builds an AST of every source file. Every function, class, method, and interface is extracted as a discrete, named chunk with exact line numbers.
|
|
29
|
+
2. **Index**: Each chunk is indexed in two parallel structures โ a TF-IDF inverted index for lexical search and a `Float32Array` vector matrix for semantic (embedding) search.
|
|
30
|
+
3. **Serve**: The MCP server exposes tools that let the agent retrieve the exact chunk it needs in a single call, receiving ~100 tokens of surgical context instead of ~10,000 tokens of a full file.
|
|
31
|
+
|
|
32
|
+
The result is **up to 90% reduction in token consumption** with higher code accuracy because the LLM processes only focused, relevant context.
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
## Architecture
|
|
37
|
+
|
|
38
|
+
```mermaid
|
|
39
|
+
flowchart LR
|
|
40
|
+
subgraph INPUT["Input Layer"]
|
|
41
|
+
FS["Source Files\n(.ts .js .py .rs .go .css)"]
|
|
42
|
+
GIT[".gitignore Filter\n(ignore library)"]
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
subgraph PARSE["AST Layer โ Tree-sitter"]
|
|
46
|
+
AST["Language Grammar\nParser (6 languages)"]
|
|
47
|
+
CHUNKS["Semantic Chunk\nExtractor\n(functions ยท classes ยท impls)"]
|
|
48
|
+
DEPS["Bidirectional Import\nExtractor"]
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
subgraph INDEX["Dual Indexing Layer"]
|
|
52
|
+
EMBED["Ollama\nnomic-embed-text\nโ Float32Array matrix"]
|
|
53
|
+
TFIDF["TF-IDF Inverted\nIndex (sublinear tf)"]
|
|
54
|
+
GRAPH["Dependency Graph\nimportedBy โ imports"]
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
subgraph SEARCH["Hybrid RRF Engine"]
|
|
58
|
+
VEC["Vector Search\n(cosine ยท flat/HNSW)"]
|
|
59
|
+
LEX["Lexical Search\n(TF-IDF ยท CamelCase-aware)"]
|
|
60
|
+
RRF["Reciprocal Rank\nFusion K=60"]
|
|
61
|
+
BOOST["Exact-Name\nBoost"]
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
subgraph MCP["MCP Protocol Layer (stdio)"]
|
|
65
|
+
SC["search_code\ntok-budget aware"]
|
|
66
|
+
GC["get_chunk"]
|
|
67
|
+
GFS["get_file_skeleton"]
|
|
68
|
+
GCG["get_call_graph"]
|
|
69
|
+
RES["graph:// resources"]
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
FS --> GIT --> AST --> CHUNKS & DEPS
|
|
73
|
+
CHUNKS --> EMBED & TFIDF
|
|
74
|
+
DEPS --> GRAPH
|
|
75
|
+
EMBED --> VEC
|
|
76
|
+
TFIDF --> LEX
|
|
77
|
+
VEC & LEX --> RRF --> BOOST --> SC & GC
|
|
78
|
+
GRAPH --> RES & GFS & GCG
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
---
|
|
82
|
+
|
|
83
|
+
## Why This Beats Standard RAG
|
|
84
|
+
|
|
85
|
+
| Feature | **graph-indexer** | Standard RAG (ChromaDB + text chunks) |
|
|
86
|
+
| :--- | :--- | :--- |
|
|
87
|
+
| **Chunking** | AST-precise: entire functions/classes, never split mid-logic | Naive token-count splits break code in half |
|
|
88
|
+
| **Infrastructure** | Zero-DB โ pure in-memory `Map` + `Float32Array` | External vector DB (Chroma, Pinecone, Weaviate) |
|
|
89
|
+
| **Privacy** | 100% air-gapped (local Ollama or lexical-only mode) | Often requires cloud embedding APIs |
|
|
90
|
+
| **Search quality** | Hybrid RRF fusing dense vectors + sparse TF-IDF | Dense-only or BM25-only |
|
|
91
|
+
| **Dependency context** | Bidirectional AST topology for every result | None |
|
|
92
|
+
| **Fault tolerance** | Graceful degradation to pure TF-IDF if Ollama is offline | Hard failure if embedding service is unavailable |
|
|
93
|
+
| **Latency** | Sub-millisecond per query (pure V8 `Float32Array`) | Network RTT + DB overhead |
|
|
94
|
+
| **Languages** | JS/TS/Python/Rust/Go/PHP/CSS | Language-agnostic (no topology) |
|
|
95
|
+
|
|
96
|
+
---
|
|
97
|
+
|
|
98
|
+
## Quick Start
|
|
99
|
+
|
|
100
|
+
### 1. Install
|
|
101
|
+
|
|
102
|
+
```bash
|
|
103
|
+
npm install graph-indexer --save-dev
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
Add scripts to your project's `package.json`:
|
|
107
|
+
|
|
108
|
+
```json
|
|
109
|
+
"scripts": {
|
|
110
|
+
"mcp:index": "idx-index --repo .",
|
|
111
|
+
"mcp:watch": "idx-watch",
|
|
112
|
+
"mcp:start": "idx-mcp"
|
|
113
|
+
}
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
### 2. Index Your Repository
|
|
117
|
+
|
|
118
|
+
```bash
|
|
119
|
+
npm run mcp:index
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
This scans every source file, builds the AST, generates embeddings via Ollama (if available), and writes `code-index.json` + `code-index.embeddings.bin`.
|
|
123
|
+
|
|
124
|
+
### 3. Start the MCP Server
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
npm run mcp:start
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
Point your MCP client at this process. The server spawns the file watcher daemon automatically.
|
|
131
|
+
|
|
132
|
+
### 4. (Optional) Run the File Watcher Manually
|
|
133
|
+
|
|
134
|
+
For persistent incremental updates in a separate terminal:
|
|
135
|
+
|
|
136
|
+
```bash
|
|
137
|
+
npm run mcp:watch
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## System Requirements
|
|
143
|
+
|
|
144
|
+
| Requirement | Details |
|
|
145
|
+
| :--- | :--- |
|
|
146
|
+
| **Node.js** | v18+ (ES Modules) |
|
|
147
|
+
| **Ollama** | Optional. Runs locally for embedding generation. Pull `nomic-embed-text` to enable semantic search. |
|
|
148
|
+
| **Disk** | `code-index.json` (metadata) + `code-index.embeddings.bin` (binary float32 vectors). Both are gitignored by default. |
|
|
149
|
+
|
|
150
|
+
### Ollama Setup (Optional but Recommended)
|
|
151
|
+
|
|
152
|
+
```bash
|
|
153
|
+
# Install Ollama: https://ollama.ai
|
|
154
|
+
ollama pull nomic-embed-text
|
|
155
|
+
npm run mcp:index # now with semantic vectors
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
### Lexical-Only Mode (No Ollama Required)
|
|
159
|
+
|
|
160
|
+
```bash
|
|
161
|
+
INDEXER_EMBEDDINGS=off npm run mcp:index
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
Recall@3 remains 100% for exact-name queries. Semantic recall (concept-to-code) degrades to lexical-only; still effective for most programming tasks.
|
|
165
|
+
|
|
166
|
+
### Custom Ollama Host
|
|
167
|
+
|
|
168
|
+
```bash
|
|
169
|
+
# Non-standard port
|
|
170
|
+
OLLAMA_HOST=http://localhost:11435 npm run mcp:index
|
|
171
|
+
|
|
172
|
+
# Remote server
|
|
173
|
+
OLLAMA_HOST=http://192.168.1.100:11434 npm run mcp:index
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
## IDE / Client Configuration
|
|
179
|
+
|
|
180
|
+
### Claude Desktop (`claude_desktop_config.json`)
|
|
181
|
+
|
|
182
|
+
```json
|
|
183
|
+
{
|
|
184
|
+
"mcpServers": {
|
|
185
|
+
"graph-indexer": {
|
|
186
|
+
"command": "node",
|
|
187
|
+
"args": ["/absolute/path/to/your/project/node_modules/graph-indexer/mcp-server.mjs"],
|
|
188
|
+
"env": {
|
|
189
|
+
"MCP_PROJECT_ROOT": "/absolute/path/to/your/project",
|
|
190
|
+
"OLLAMA_HOST": "http://localhost:11434"
|
|
191
|
+
}
|
|
192
|
+
}
|
|
193
|
+
}
|
|
194
|
+
}
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
### VS Code (`.vscode/mcp.json`)
|
|
198
|
+
|
|
199
|
+
```json
|
|
200
|
+
{
|
|
201
|
+
"servers": {
|
|
202
|
+
"graph-indexer": {
|
|
203
|
+
"type": "stdio",
|
|
204
|
+
"command": "node",
|
|
205
|
+
"args": ["${workspaceFolder}/node_modules/graph-indexer/mcp-server.mjs"],
|
|
206
|
+
"env": {
|
|
207
|
+
"MCP_PROJECT_ROOT": "${workspaceFolder}",
|
|
208
|
+
"OLLAMA_HOST": "http://localhost:11434"
|
|
209
|
+
}
|
|
210
|
+
}
|
|
211
|
+
}
|
|
212
|
+
}
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
### Cursor (`.cursor/mcp.json`)
|
|
216
|
+
|
|
217
|
+
```json
|
|
218
|
+
{
|
|
219
|
+
"mcpServers": {
|
|
220
|
+
"graph-indexer": {
|
|
221
|
+
"command": "node",
|
|
222
|
+
"args": ["${workspaceFolder}/node_modules/graph-indexer/mcp-server.mjs"],
|
|
223
|
+
"env": {
|
|
224
|
+
"MCP_PROJECT_ROOT": "${workspaceFolder}"
|
|
225
|
+
}
|
|
226
|
+
}
|
|
227
|
+
}
|
|
228
|
+
}
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
---
|
|
232
|
+
|
|
233
|
+
## Agent System Prompt
|
|
234
|
+
|
|
235
|
+
Copy the contents of [`PROMPT.md`](./PROMPT.md) into your agent's system instructions. This document instructs the agent on:
|
|
236
|
+
- When to call which tool (strict priority order)
|
|
237
|
+
- How to write effective `query` and `exact_tokens` parameters
|
|
238
|
+
- Workflow patterns for Q&A, implementation, refactoring, and debugging
|
|
239
|
+
- What NOT to do (file reads without search first, raising `top_k` speculatively, etc.)
|
|
240
|
+
|
|
241
|
+
---
|
|
242
|
+
|
|
243
|
+
## MCP Tools Reference
|
|
244
|
+
|
|
245
|
+
### `search_code`
|
|
246
|
+
|
|
247
|
+
Hybrid semantic + lexical search. Returns compact signature cards for all results, then fills remaining token budget with code bodies.
|
|
248
|
+
|
|
249
|
+
| Parameter | Type | Default | Description |
|
|
250
|
+
| :--- | :--- | :--- | :--- |
|
|
251
|
+
| `query` | `string` | โ | Natural language description of the logic to find |
|
|
252
|
+
| `exact_tokens` | `string?` | โ | Exact symbol name to guarantee rank-1 placement |
|
|
253
|
+
| `top_k` | `number` | `5` | Number of results (1โ20) |
|
|
254
|
+
| `min_score` | `number` | `0.3` | Minimum cosine similarity threshold |
|
|
255
|
+
| `token_budget` | `number?` | 1500 chars | Estimated token budget for code bodies |
|
|
256
|
+
| `include_topology` | `boolean` | `true` | Include `โฌ๏ธ Deps` / `โฌ๏ธ Used by` / `๐ Calls` in output |
|
|
257
|
+
|
|
258
|
+
**Example response:**
|
|
259
|
+
```
|
|
260
|
+
#1 ยท validateToken [function_declaration]
|
|
261
|
+
๐ src/utils/jwt.ts:14โ42 ยท ID: `a3f9c1b2` ยท RRF: 0.0321
|
|
262
|
+
๐ฌ Validates and decodes a JWT access token. Throws on expiry.
|
|
263
|
+
โฌ๏ธ Deps: src/config/env.ts [JWT_SECRET] | src/utils/errors.ts [AuthError]
|
|
264
|
+
โฌ๏ธ Used by: src/middleware/auth.ts, src/routes/api.ts
|
|
265
|
+
๐ Calls: verify, decodePayload, throwIfExpired
|
|
266
|
+
โฉ๏ธ Expand body: get_chunk("a3f9c1b2")
|
|
267
|
+
```
|
|
268
|
+
|
|
269
|
+
---
|
|
270
|
+
|
|
271
|
+
### `get_chunk`
|
|
272
|
+
|
|
273
|
+
Retrieves the full source body of one chunk by ID. Use this after `search_code` instead of reading the whole file.
|
|
274
|
+
|
|
275
|
+
| Parameter | Type | Description |
|
|
276
|
+
| :--- | :--- | :--- |
|
|
277
|
+
| `chunk_id` | `string` | The ID shown in `search_code` results |
|
|
278
|
+
|
|
279
|
+
---
|
|
280
|
+
|
|
281
|
+
### `get_file_skeleton`
|
|
282
|
+
|
|
283
|
+
Returns only the names and line ranges of all functions/classes in a file โ no code bodies. ~50 tokens instead of thousands.
|
|
284
|
+
|
|
285
|
+
| Parameter | Type | Description |
|
|
286
|
+
| :--- | :--- | :--- |
|
|
287
|
+
| `file_path` | `string` | Relative path (e.g. `src/components/Button.tsx`) |
|
|
288
|
+
|
|
289
|
+
---
|
|
290
|
+
|
|
291
|
+
### `get_call_graph`
|
|
292
|
+
|
|
293
|
+
Finds every chunk in the repo that calls a specific function by name. Critical before any signature change.
|
|
294
|
+
|
|
295
|
+
| Parameter | Type | Description |
|
|
296
|
+
| :--- | :--- | :--- |
|
|
297
|
+
| `target_function` | `string` | Exact function name (e.g. `validateToken`) |
|
|
298
|
+
|
|
299
|
+
---
|
|
300
|
+
|
|
301
|
+
### `graph://dependencies/{file_path}` Resource
|
|
302
|
+
|
|
303
|
+
Fetches the full bidirectional dependency topology for a file without consuming any search quota.
|
|
304
|
+
|
|
305
|
+
```
|
|
306
|
+
URI: graph://dependencies/src/middleware/auth.ts
|
|
307
|
+
```
|
|
308
|
+
|
|
309
|
+
Returns which files this file imports, and which files import it.
|
|
310
|
+
|
|
311
|
+
---
|
|
312
|
+
|
|
313
|
+
## Supported Languages
|
|
314
|
+
|
|
315
|
+
| Language | Extensions | Chunk Types | Import Resolution |
|
|
316
|
+
| :--- | :--- | :--- | :--- |
|
|
317
|
+
| TypeScript / TSX | `.ts`, `.tsx` | functions, classes, methods, exports | Absolute & relative paths |
|
|
318
|
+
| JavaScript | `.js`, `.jsx`, `.mjs`, `.cjs` | functions, classes, expressions | Absolute & relative paths |
|
|
319
|
+
| Python | `.py` | function_definition, class_definition | Relative imports (`.`, `..`) โ file paths |
|
|
320
|
+
| Rust | `.rs` | fn, struct, enum, trait, impl | `crate::` path resolution |
|
|
321
|
+
| Go | `.go` | function_declaration, method, type | Import spec resolution |
|
|
322
|
+
| PHP | `.php` | function, class | include/require paths |
|
|
323
|
+
| CSS / SCSS | `.css`, `.scss` | rule_set | โ |
|
|
324
|
+
|
|
325
|
+
---
|
|
326
|
+
|
|
327
|
+
## Performance & Evaluation
|
|
328
|
+
|
|
329
|
+
Validate indexer quality on your own codebase:
|
|
330
|
+
|
|
331
|
+
```bash
|
|
332
|
+
npm run test
|
|
333
|
+
# or with custom Ollama:
|
|
334
|
+
OLLAMA_HOST=http://localhost:11435 npm run test
|
|
335
|
+
```
|
|
336
|
+
|
|
337
|
+
### Benchmark Results
|
|
338
|
+
|
|
339
|
+
| Scenario | Recall@3 | Recall@5 | Latency (p50) |
|
|
340
|
+
| :--- | :--- | :--- | :--- |
|
|
341
|
+
| graph-indexer own codebase (34 chunks, lexical-only) | **100%** | 100% | 0.07 ms |
|
|
342
|
+
| graph-indexer own codebase (34 chunks, with Ollama) | **75%** | 100% | 0.14 ms |
|
|
343
|
+
| React Native project (311 chunks, with Ollama) | **58%** | 75% | 1.02 ms |
|
|
344
|
+
|
|
345
|
+
> The React Native project numbers are without any JSDoc in the source files. Adding docstrings to exported functions (see Best Practices below) brings recall to **100% @3**.
|
|
346
|
+
|
|
347
|
+
### Vector Search Performance
|
|
348
|
+
|
|
349
|
+
| Corpus size | Pure JS flat scan | HNSW (auto-activated โฅ5k) |
|
|
350
|
+
| :--- | :--- | :--- |
|
|
351
|
+
| 1,000 chunks (dim=768) | 0.9 ms/query | โ |
|
|
352
|
+
| 5,000 chunks (dim=768) | 4.5 ms/query | 1.2 ms/query |
|
|
353
|
+
| 20,000 chunks (dim=768) | 30.5 ms/query | 3.1 ms/query |
|
|
354
|
+
|
|
355
|
+
---
|
|
356
|
+
|
|
357
|
+
## Best Practices for 100% Recall
|
|
358
|
+
|
|
359
|
+
### 1. Add JSDoc/TSDoc to Exported Functions
|
|
360
|
+
|
|
361
|
+
The indexer embeds your docstrings alongside code. Docstrings dramatically improve semantic search:
|
|
362
|
+
|
|
363
|
+
```typescript
|
|
364
|
+
/**
|
|
365
|
+
* Validates and decodes a JWT access token.
|
|
366
|
+
* Throws AuthError if the token is expired or malformed.
|
|
367
|
+
*
|
|
368
|
+
* @param token - Raw JWT string from Authorization header
|
|
369
|
+
* @returns Decoded payload with userId, role, and expiry
|
|
370
|
+
*/
|
|
371
|
+
export function validateToken(token: string): TokenPayload { ... }
|
|
372
|
+
```
|
|
373
|
+
|
|
374
|
+
### 2. Use Descriptive, Specific Names
|
|
375
|
+
|
|
376
|
+
The exact-name boost gives a free rank-1 slot to any chunk whose name exactly matches `exact_tokens`. Descriptive names make this reliable:
|
|
377
|
+
|
|
378
|
+
```typescript
|
|
379
|
+
// โ
Exact-name searchable
|
|
380
|
+
export const useNotificationPermissionStatus = () => { ... };
|
|
381
|
+
export function calculateTripDurationInDays(start: Date, end: Date) { ... }
|
|
382
|
+
|
|
383
|
+
// โ Generic โ impossible to target without semantic search
|
|
384
|
+
export const util = () => { ... };
|
|
385
|
+
export function process(data: any) { ... }
|
|
386
|
+
```
|
|
387
|
+
|
|
388
|
+
### 3. Expand Minimal Utility Files
|
|
389
|
+
|
|
390
|
+
Files with fewer than ~5 indexable lines produce zero or one chunk. Add error handling and validation to make them indexable:
|
|
391
|
+
|
|
392
|
+
```typescript
|
|
393
|
+
// โ 2 lines โ may not be indexed
|
|
394
|
+
export const Context = createContext(null);
|
|
395
|
+
export const useCtx = () => useContext(Context);
|
|
396
|
+
|
|
397
|
+
// โ
Indexable with meaningful docstring + validation
|
|
398
|
+
/**
|
|
399
|
+
* Context for accessing authentication state throughout the component tree.
|
|
400
|
+
*/
|
|
401
|
+
export const AuthContext = createContext<AuthState | null>(null);
|
|
402
|
+
|
|
403
|
+
/**
|
|
404
|
+
* Hook for consuming AuthContext. Throws if used outside AuthProvider.
|
|
405
|
+
*/
|
|
406
|
+
export function useAuth(): AuthState {
|
|
407
|
+
const ctx = useContext(AuthContext);
|
|
408
|
+
if (!ctx) throw new Error('useAuth must be used inside <AuthProvider>');
|
|
409
|
+
return ctx;
|
|
410
|
+
}
|
|
411
|
+
```
|
|
412
|
+
|
|
413
|
+
### 4. Keep Imports Explicit and Organized
|
|
414
|
+
|
|
415
|
+
The topology graph follows import paths. Barrel re-exports from `index.ts` break dependency resolution โ prefer direct imports:
|
|
416
|
+
|
|
417
|
+
```typescript
|
|
418
|
+
// โ
Topology-friendly
|
|
419
|
+
import { useAuthStore } from '@/stores/authStore';
|
|
420
|
+
import { validateEmail } from '@/utils/validators';
|
|
421
|
+
|
|
422
|
+
// โ Breaks topology โ indexer can't resolve what's actually imported
|
|
423
|
+
import { useAuthStore, validateEmail } from '@/index';
|
|
424
|
+
```
|
|
425
|
+
|
|
426
|
+
---
|
|
427
|
+
|
|
428
|
+
## File Structure
|
|
429
|
+
|
|
430
|
+
```
|
|
431
|
+
graph-indexer/
|
|
432
|
+
โโโ core-engine.mjs # In-memory index: vector matrix, TF-IDF, RRF search, save/load
|
|
433
|
+
โโโ parser-utils.mjs # Tree-sitter AST parsing, 6 language grammars, Ollama embeddings
|
|
434
|
+
โโโ indexer.mjs # CLI bootstrap: scan repo โ parse โ embed โ write index
|
|
435
|
+
โโโ watch-daemon.mjs # Chokidar daemon: incremental real-time index updates
|
|
436
|
+
โโโ mcp-server.mjs # MCP stdio server: exposes tools + graph:// resources
|
|
437
|
+
โโโ PROMPT.md # Agent system prompt โ paste into your IDE's agent instructions
|
|
438
|
+
โโโ tests/
|
|
439
|
+
โ โโโ eval-harness.mjs # Recall@k + latency evaluation against the indexer's own codebase
|
|
440
|
+
โ โโโ react-native-test.mjs # Real-world quality test on a 311-chunk React Native project
|
|
441
|
+
โโโ package.json
|
|
442
|
+
```
|
|
443
|
+
|
|
444
|
+
---
|
|
445
|
+
|
|
446
|
+
## Mathematical Foundation
|
|
447
|
+
|
|
448
|
+
### Sublinear TF Scaling
|
|
449
|
+
|
|
450
|
+
Term frequencies use sublinear scaling to prevent common tokens (`return`, `const`) from burying business-logic terms:
|
|
451
|
+
|
|
452
|
+
$$w(t,d) = 1 + \log(f_{t,d})$$
|
|
453
|
+
|
|
454
|
+
### Reciprocal Rank Fusion
|
|
455
|
+
|
|
456
|
+
Vector and lexical results are merged by rank position, not raw incompatible scores:
|
|
457
|
+
|
|
458
|
+
$$\text{RRF}(d) = \sum_{i \in \{\text{vec, lex}\}} \frac{1}{K + \text{rank}_i(d)}, \quad K = 60$$
|
|
459
|
+
|
|
460
|
+
### Exact-Name Boost
|
|
461
|
+
|
|
462
|
+
Any chunk whose name exactly matches `exact_tokens` receives an additive bonus equal to the maximum single-list RRF contribution:
|
|
463
|
+
|
|
464
|
+
$$\Delta\text{score} = \frac{1}{K + 1}$$
|
|
465
|
+
|
|
466
|
+
This guarantees rank-1 placement for any exact symbol name query.
|
|
467
|
+
|
|
468
|
+
---
|
|
469
|
+
|
|
470
|
+
## How the Index Files Work
|
|
471
|
+
|
|
472
|
+
| File | Purpose | Typical Size |
|
|
473
|
+
| :--- | :--- | :--- |
|
|
474
|
+
| `code-index.json` | Chunk metadata (names, file paths, docstrings, calls, graph) | ~200 KB / 1k chunks |
|
|
475
|
+
| `code-index.embeddings.bin` | Binary `float32` embedding vectors (4.8ร smaller than JSON) | ~3 MB / 1k chunks @768-dim |
|
|
476
|
+
|
|
477
|
+
Both files are written atomically via a `tmp โ rename` pattern to prevent corruption on crash. Both should be added to `.gitignore`.
|
|
478
|
+
|
|
479
|
+
---
|
|
480
|
+
|
|
481
|
+
## Environment Variables
|
|
482
|
+
|
|
483
|
+
| Variable | Default | Description |
|
|
484
|
+
| :--- | :--- | :--- |
|
|
485
|
+
| `OLLAMA_HOST` | `http://localhost:11434` | Ollama API base URL |
|
|
486
|
+
| `INDEXER_EMBEDDINGS` | โ | Set to `off` to disable embeddings entirely |
|
|
487
|
+
| `MCP_PROJECT_ROOT` | `process.cwd()` | Absolute path to the project being indexed |
|
|
488
|
+
|
|
489
|
+
---
|
|
490
|
+
|
|
491
|
+
## License & Intellectual Property
|
|
492
|
+
|
|
493
|
+
Released under the [GNU General Public License v3.0](LICENSE).
|
|
494
|
+
|
|
495
|
+
This license ensures:
|
|
496
|
+
- **Copyleft protection**: Any software that incorporates this code must also be released under GPL-3.0.
|
|
497
|
+
- **Attribution**: The original author (MaquinaTech) must be credited.
|
|
498
|
+
- **Source availability**: Any distributed modifications must include source code.
|
|
499
|
+
|
|
500
|
+
Commercial use is permitted but requires GPL-3.0 compliance. For proprietary licensing inquiries, contact the author.
|
|
501
|
+
|
|
502
|
+
---
|
|
503
|
+
|
|
504
|
+
## Author
|
|
505
|
+
|
|
506
|
+
Built and maintained by **MaquinaTech**.
|
|
507
|
+
|
|
508
|
+
- **GitHub:** [github.com/MaquinaTech](https://github.com/MaquinaTech)
|
|
509
|
+
- **NPM:** [npmjs.com/package/graph-indexer](https://www.npmjs.com/package/graph-indexer)
|
|
510
|
+
- **Issues:** [github.com/MaquinaTech/graph-indexer/issues](https://github.com/MaquinaTech/graph-indexer/issues)
|
|
511
|
+
|
|
512
|
+
Issues, pull requests, and feature discussions are welcome.
|