npm - graph-indexer - Versions diffs - 1.0.0 → 1.0.2 - Mend

graph-indexer 1.0.0 → 1.0.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (10) hide show

package/README.md CHANGED Viewed

@@ -1,512 +1,421 @@
-<h1 align="center">graph-indexer</h1>
+# graph-indexer
 <p align="center">
-  <strong>Zero-DB · Air-Gapped · AST-Precision · Hybrid RRF Search</strong><br>
-  <em>The production-grade MCP code indexer for AI coding agents — surgical context retrieval that cuts token costs by up to 90%.</em>
+  <strong>Zero-Database · Air-Gapped · AST-Precise · Hybrid Search</strong><br>
+  <em>An MCP server that lets AI coding agents search your codebase by concept instead of reading files — reducing context usage by 65–88% per query.</em>
 </p>
 <p align="center">
   <a href="https://www.npmjs.com/package/graph-indexer"><img src="https://img.shields.io/npm/v/graph-indexer?color=blue&style=flat-square" alt="npm version"></a>
-  <a href="LICENSE"><img src="https://img.shields.io/badge/License-GPL--3.0-blue.svg?style=flat-square" alt="License: GPL-3.0"></a>
+  <a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-yellow.svg?style=flat-square" alt="License: MIT"></a>
   <a href="https://nodejs.org"><img src="https://img.shields.io/badge/Node.js-%3E%3D18-green?style=flat-square" alt="Node.js 18+"></a>
+  <a href="#results"><img src="https://img.shields.io/badge/Recall@5-100%25-brightgreen?style=flat-square" alt="Recall@5 100%"></a>
 </p>
 ---
-## The Problem: AI Agents Are Context-Blind and Wasteful
+## What it does
-When you use an AI coding agent (Claude, Cursor, GitHub Copilot) on a real codebase, it operates with **brute-force context**: it reads entire files—often 500–1000 lines—just to find one function. This has three compounding costs:
+graph-indexer pre-indexes your codebase into an AST-precise search index and exposes it as an MCP server. Instead of reading full files, AI agents call `search_code("payment validation")` and get back the exact functions that match — with their type signatures, docstrings, dependency graph, and call sites — in a fraction of the tokens.
-1. **Token burn:** A single `readFile` on an 800-line file consumes ~10,000 tokens of context window. At GPT-4 pricing, that's money per keystroke.
-2. **Lost in the Middle:** LLMs reliably lose accuracy when the relevant code is buried in a large context. More code ≠ better answers.
-3. **Topology blindness:** File reads give you code, not structure. The agent can't see that `fileA.ts` calls a function from `fileB.ts` unless it reads both. Refactors break silently.
+It runs entirely on your machine. No external database, no cloud APIs, no telemetry.
-## The Solution: Surgical, AST-Precise Retrieval
+---
+## Why it matters
-`graph-indexer` is a local MCP server that gives your AI agent a **structural map** of the entire codebase instead of raw file access. It works in three steps:
+When an AI agent works on a real codebase without graph-indexer, it reads files to find code:
-1. **Parse**: Tree-sitter builds an AST of every source file. Every function, class, method, and interface is extracted as a discrete, named chunk with exact line numbers.
-2. **Index**: Each chunk is indexed in two parallel structures — a TF-IDF inverted index for lexical search and a `Float32Array` vector matrix for semantic (embedding) search.
-3. **Serve**: The MCP server exposes tools that let the agent retrieve the exact chunk it needs in a single call, receiving ~100 tokens of surgical context instead of ~10,000 tokens of a full file.
+```
+Agent task: "Add error handling to the payment processing flow"
+Without graph-indexer:
+  readFile("src/payments/service.ts")   → 8,400 tokens
+  readFile("src/payments/handlers.ts")  → 6,200 tokens
+  readFile("src/types/payment.ts")      → 1,800 tokens
+  Total context consumed:               ~16,400 tokens for 3 files
+With graph-indexer:
+  search_code("payment processing")     → 650 tokens (5 exact chunks)
+  get_chunk("chunk_id")                 → 280 tokens (full function body)
+  Total context consumed:               ~930 tokens
+```
-The result is **up to 90% reduction in token consumption** with higher code accuracy because the LLM processes only focused, relevant context.
+The difference compounds across a session. Agents that read files also lack topology — they can't see which other functions call the one they're modifying, so they miss side effects.
 ---
-## Architecture
-```mermaid
-flowchart LR
-    subgraph INPUT["Input Layer"]
-        FS["Source Files\n(.ts .js .py .rs .go .css)"]
-        GIT[".gitignore Filter\n(ignore library)"]
-    end
-    subgraph PARSE["AST Layer — Tree-sitter"]
-        AST["Language Grammar\nParser (6 languages)"]
-        CHUNKS["Semantic Chunk\nExtractor\n(functions · classes · impls)"]
-        DEPS["Bidirectional Import\nExtractor"]
-    end
-    subgraph INDEX["Dual Indexing Layer"]
-        EMBED["Ollama\nnomic-embed-text\n→ Float32Array matrix"]
-        TFIDF["TF-IDF Inverted\nIndex (sublinear tf)"]
-        GRAPH["Dependency Graph\nimportedBy ↔ imports"]
-    end
-    subgraph SEARCH["Hybrid RRF Engine"]
-        VEC["Vector Search\n(cosine · flat/HNSW)"]
-        LEX["Lexical Search\n(TF-IDF · CamelCase-aware)"]
-        RRF["Reciprocal Rank\nFusion  K=60"]
-        BOOST["Exact-Name\nBoost"]
-    end
-    subgraph MCP["MCP Protocol Layer (stdio)"]
-        SC["search_code\ntok-budget aware"]
-        GC["get_chunk"]
-        GFS["get_file_skeleton"]
-        GCG["get_call_graph"]
-        RES["graph:// resources"]
-    end
-    FS --> GIT --> AST --> CHUNKS & DEPS
-    CHUNKS --> EMBED & TFIDF
-    DEPS --> GRAPH
-    EMBED --> VEC
-    TFIDF --> LEX
-    VEC & LEX --> RRF --> BOOST --> SC & GC
-    GRAPH --> RES & GFS & GCG
-```
+## Results
----
+Measured across 5 production open-source codebases (7,503 total AST chunks, 65 ground-truth queries). Token savings compare the chunks returned per query against reading the full source files they came from.
-## Why This Beats Standard RAG
+### Token usage per query: with vs. without graph-indexer
-| Feature | **graph-indexer** | Standard RAG (ChromaDB + text chunks) |
-| :--- | :--- | :--- |
-| **Chunking** | AST-precise: entire functions/classes, never split mid-logic | Naive token-count splits break code in half |
-| **Infrastructure** | Zero-DB — pure in-memory `Map` + `Float32Array` | External vector DB (Chroma, Pinecone, Weaviate) |
-| **Privacy** | 100% air-gapped (local Ollama or lexical-only mode) | Often requires cloud embedding APIs |
-| **Search quality** | Hybrid RRF fusing dense vectors + sparse TF-IDF | Dense-only or BM25-only |
-| **Dependency context** | Bidirectional AST topology for every result | None |
-| **Fault tolerance** | Graceful degradation to pure TF-IDF if Ollama is offline | Hard failure if embedding service is unavailable |
-| **Latency** | Sub-millisecond per query (pure V8 `Float32Array`) | Network RTT + DB overhead |
-| **Languages** | JS/TS/Python/Rust/Go/PHP/CSS | Language-agnostic (no topology) |
+| Project | Language | Codebase size | Without (full files) | With graph-indexer | Savings | Latency |
+| :--- | :--- | ---: | ---: | ---: | ---: | ---: |
+| Axios v1.6.0 | JavaScript | 111k tokens | ~2,290 tok/query | ~655 tok/query | **71.4%** | 0.2 ms |
+| Express 4.18.2 | JavaScript | 145k tokens | ~7,330 tok/query | ~850 tok/query | **88.4%** | 0.2 ms |
+| NestJS v10.4.9 | TypeScript | 700k tokens | ~2,535 tok/query | ~900 tok/query | **64.5%** | 0.7 ms |
+| FastAPI 0.103.0 | Python | 859k tokens | ~4,030 tok/query | ~685 tok/query | **83.0%** | 1.0 ms |
+| Gin v1.9.1 | Go | 131k tokens | ~2,530 tok/query | ~450 tok/query | **82.2%** | 0.4 ms |
+| **Mean** | | | | | **77.9%** | **0.5 ms** |
----
+> "Full files" = reading the source files that contain the top-5 retrieved chunks. "With graph-indexer" = tokens in the returned chunk code snippets only. Measured at 4 chars/token.
-## Quick Start
+**Key observations:**
+- NestJS has 700k tokens of source — larger than most LLMs' full context window. graph-indexer reduces each query to ~900 tokens.
+- Express shows 88.4% savings because its core router logic is spread across large files with much non-relevant code.
+- Latency is sub-millisecond for all suites at their real corpus sizes.
-### 1. Install
+### Search quality
-```bash
-npm install graph-indexer --save-dev
-```
+| Project | Chunks | Recall@1 | Recall@3 | Recall@5 | MRR | Queries |
+| :--- | ---: | ---: | ---: | ---: | ---: | ---: |
+| Axios v1.6.0 | 435 | 0.86 | 1.00 | **1.00** | 0.92 | 14 |
+| Express 4.18.2 | 389 | 0.93 | 0.93 | **1.00** | 0.95 | 14 |
+| NestJS v10.4.9 | 2,019 | 1.00 | 1.00 | **1.00** | **1.00** | 15 |
+| FastAPI 0.103.0 | 3,572 | 0.93 | 1.00 | **1.00** | 0.96 | 14 |
+| Gin v1.9.1 | 1,088 | 0.85 | 1.00 | **1.00** | 0.92 | 13 |
+| **Mean** | **7,503** | **0.91** | **0.99** | **1.00** | **0.95** | **70** |
-Add scripts to your project's `package.json`:
+- **Recall@5 = 1.00** across every project and language: the correct answer is always in the top 5 results.
+- **MRR = 0.95** overall: on average, the correct answer is the first or second result.
+- Queries span three difficulty levels: easy (exact symbol names), medium (multi-token descriptions), hard (semantic concepts with vocabulary mismatch).
-```json
-"scripts": {
-  "mcp:index": "idx-index --repo .",
-  "mcp:watch": "idx-watch",
-  "mcp:start": "idx-mcp"
-}
-```
+---
-### 2. Index Your Repository
+## How it works
-```bash
-npm run mcp:index
-```
+### Indexing
-This scans every source file, builds the AST, generates embeddings via Ollama (if available), and writes `code-index.json` + `code-index.embeddings.bin`.
+When you run `npm run mcp:index`, graph-indexer:
-### 3. Start the MCP Server
+1. **Parses** every source file with Tree-sitter to build an AST — no regex, exact code boundaries.
+2. **Extracts chunks**: named functions, classes, methods, and exports. Each chunk includes its name, docstring, parameters, return type, call sites, and type references.
+3. **Builds a dependency graph**: bidirectional import map so each chunk knows what it imports and what imports it.
+4. **Creates two search indexes**:
+   - A BM25 inverted index (lexical search, O(1) per-token lookup).
+   - A vector embedding per chunk via Ollama `nomic-embed-text` (optional; falls back to lexical-only if Ollama is unavailable).
+5. **Serializes to disk**: `code-index.json` (metadata + BM25 index) and `code-index.embeddings.bin` (binary float32 vectors).
-```bash
-npm run mcp:start
-```
+A background file watcher daemon (`watch-daemon.mjs`) re-indexes changed files automatically after the initial index is built.
-Point your MCP client at this process. The server spawns the file watcher daemon automatically.
+### Search
-### 4. (Optional) Run the File Watcher Manually
+When an agent calls `search_code("payment validation middleware")`:
-For persistent incremental updates in a separate terminal:
+1. **Lexical search** over the BM25 inverted index — returns candidates with BM25 scores in O(query_terms) time.
+2. **Vector search** over pre-normalized float32 embeddings — returns semantically similar chunks.
+3. **Reciprocal Rank Fusion (RRF)** merges both ranked lists with weights (lexical 1.5×, vector 1.0×).
+4. **Scoring adjustments**: exact name match +2.0×, snake_case suffix match +1.4×, path token match +1.4×, test/example files demoted.
+5. **Returns** top-k chunks with: code snippet, type signature, docstring, call graph, and dependency topology — all in one response.
-```bash
-npm run mcp:watch
+### Architecture
+```
+Source files (.ts .js .py .go .rs …)
+     │
+     ▼
+Tree-sitter AST parser
+     │
+     ├──► Named chunks (functions, classes, methods)
+     │         │
+     │         ├──► BM25 inverted index  ──┐
+     │         └──► Ollama embeddings    ──┤── RRF fusion ──► Ranked results
+     │                                     │
+     └──► Dependency graph ────────────────┘
+               (importedBy / imports)           ▼
+                                          MCP tools for agents
 ```
 ---
-## System Requirements
+## Getting started
-| Requirement | Details |
-| :--- | :--- |
-| **Node.js** | v18+ (ES Modules) |
-| **Ollama** | Optional. Runs locally for embedding generation. Pull `nomic-embed-text` to enable semantic search. |
-| **Disk** | `code-index.json` (metadata) + `code-index.embeddings.bin` (binary float32 vectors). Both are gitignored by default. |
-### Ollama Setup (Optional but Recommended)
+### 1. Install
 ```bash
-# Install Ollama: https://ollama.ai
-ollama pull nomic-embed-text
-npm run mcp:index   # now with semantic vectors
+npm install graph-indexer --save-dev
+npx graph-indexer init
 ```
-### Lexical-Only Mode (No Ollama Required)
+`init` auto-detects your IDE (Claude, Cursor, VS Code), adds npm scripts to `package.json`, and updates `.gitignore`.
+### 2. Index your repository
+**With semantic search (recommended):**
 ```bash
-INDEXER_EMBEDDINGS=off npm run mcp:index
+# Install Ollama: https://ollama.ai
+ollama pull nomic-embed-text
+npm run mcp:index
 ```
-Recall@3 remains 100% for exact-name queries. Semantic recall (concept-to-code) degrades to lexical-only; still effective for most programming tasks.
-### Custom Ollama Host
+**Lexical-only (no Ollama required):**
 ```bash
-# Non-standard port
-OLLAMA_HOST=http://localhost:11435 npm run mcp:index
-# Remote server
-OLLAMA_HOST=http://192.168.1.100:11434 npm run mcp:index
+INDEXER_EMBEDDINGS=off npm run mcp:index
 ```
----
-## IDE / Client Configuration
+Lexical-only still achieves Recall@5 = 1.00 for the test suites above.
-### Claude Desktop (`claude_desktop_config.json`)
+### 3. Configure your IDE
+**Claude Desktop** (`~/Library/Application Support/Claude/claude_desktop_config.json`):
 ```json
 {
   "mcpServers": {
     "graph-indexer": {
       "command": "node",
-      "args": ["/absolute/path/to/your/project/node_modules/graph-indexer/mcp-server.mjs"],
-      "env": {
-        "MCP_PROJECT_ROOT": "/absolute/path/to/your/project",
-        "OLLAMA_HOST": "http://localhost:11434"
-      }
+      "args": ["/path/to/project/node_modules/graph-indexer/mcp-server.mjs"],
+      "env": { "MCP_PROJECT_ROOT": "/path/to/project" }
     }
   }
 }
 ```
-### VS Code (`.vscode/mcp.json`)
+**Cursor** (`.cursor/mcp.json`):
 ```json
 {
-  "servers": {
+  "mcpServers": {
     "graph-indexer": {
-      "type": "stdio",
       "command": "node",
       "args": ["${workspaceFolder}/node_modules/graph-indexer/mcp-server.mjs"],
-      "env": {
-        "MCP_PROJECT_ROOT": "${workspaceFolder}",
-        "OLLAMA_HOST": "http://localhost:11434"
-      }
+      "env": { "MCP_PROJECT_ROOT": "${workspaceFolder}" }
     }
   }
 }
 ```
-### Cursor (`.cursor/mcp.json`)
+**VS Code** (`.vscode/mcp.json`):
 ```json
 {
-  "mcpServers": {
+  "servers": {
     "graph-indexer": {
+      "type": "stdio",
       "command": "node",
-      "args": ["${workspaceFolder}/node_modules/graph-indexer/mcp-server.mjs"],
-      "env": {
-        "MCP_PROJECT_ROOT": "${workspaceFolder}"
-      }
+      "args": ["${workspaceFolder}/node_modules/graph-indexer/mcp-server.mjs"]
     }
   }
 }
 ```
----
-## Agent System Prompt
+### 4. Add the agent system prompt
-Copy the contents of [`PROMPT.md`](./PROMPT.md) into your agent's system instructions. This document instructs the agent on:
-- When to call which tool (strict priority order)
-- How to write effective `query` and `exact_tokens` parameters
-- Workflow patterns for Q&A, implementation, refactoring, and debugging
-- What NOT to do (file reads without search first, raising `top_k` speculatively, etc.)
+Copy the contents of [PROMPT.md](./PROMPT.md) into your AI agent's system prompt. This instructs the agent to use graph-indexer tools instead of reading files directly.
 ---
-## MCP Tools Reference
+## MCP tools
 ### `search_code`
-Hybrid semantic + lexical search. Returns compact signature cards for all results, then fills remaining token budget with code bodies.
+Searches the index by natural language query. Returns signature cards for all results, then code bodies according to the `detail` level.
 | Parameter | Type | Default | Description |
 | :--- | :--- | :--- | :--- |
-| `query` | `string` | — | Natural language description of the logic to find |
-| `exact_tokens` | `string?` | — | Exact symbol name to guarantee rank-1 placement |
-| `top_k` | `number` | `5` | Number of results (1–20) |
-| `min_score` | `number` | `0.3` | Minimum cosine similarity threshold |
-| `token_budget` | `number?` | 1500 chars | Estimated token budget for code bodies |
-| `include_topology` | `boolean` | `true` | Include `⬇️ Deps` / `⬆️ Used by` / `🔗 Calls` in output |
-**Example response:**
-```
-#1 · validateToken [function_declaration]
-📄 src/utils/jwt.ts:14–42 · ID: `a3f9c1b2` · RRF: 0.0321
-💬 Validates and decodes a JWT access token. Throws on expiry.
-⬇️  Deps:    src/config/env.ts [JWT_SECRET] | src/utils/errors.ts [AuthError]
-⬆️  Used by: src/middleware/auth.ts, src/routes/api.ts
-🔗 Calls:   verify, decodePayload, throwIfExpired
-↩️  Expand body: get_chunk("a3f9c1b2")
-```
+| `query` | string | required | Natural language description of the logic to find |
+| `exact_tokens` | string? | — | Exact symbol name — guarantees rank-1 placement |
+| `detail` | `"signatures"` \| `"smart"` \| `"full"` | `"smart"` | Controls how much code is returned |
+| `top_k` | number | `5` | Results to return (1–20) |
+| `include_topology` | boolean | `true` | Include Deps / Used by / Calls in each card |
+| `min_score` | number | `0.3` | Minimum score threshold |
+| `token_budget` | number? | auto | Token budget for code bodies |
----
+**`detail` levels:**
-### `get_chunk`
+| Value | Per-result cost | What you get |
+| :--- | ---: | :--- |
+| `"signatures"` | ~20 tokens | Name, type, params, return type, topology only |
+| `"smart"` (default) | ~150 tokens | Signature + only lines relevant to the query |
+| `"full"` | ~300 tokens | Signature + complete source body |
-Retrieves the full source body of one chunk by ID. Use this after `search_code` instead of reading the whole file.
-| Parameter | Type | Description |
-| :--- | :--- | :--- |
-| `chunk_id` | `string` | The ID shown in `search_code` results |
+Start with `"signatures"`, escalate only when you need implementation details.
 ---
-### `get_file_skeleton`
+### `resolve_symbol`
-Returns only the names and line ranges of all functions/classes in a file — no code bodies. ~50 tokens instead of thousands.
+Looks up any symbol by exact name in O(1) — no search needed.
 | Parameter | Type | Description |
 | :--- | :--- | :--- |
-| `file_path` | `string` | Relative path (e.g. `src/components/Button.tsx`) |
+| `symbol` | string | Exact function, class, or type name (e.g. `"validateToken"`) |
+Returns: definition location, type signature, docstring, type references, and cross-file topology. Faster than `search_code` for known names.
 ---
-### `get_call_graph`
+### `get_chunk`
-Finds every chunk in the repo that calls a specific function by name. Critical before any signature change.
+Returns the full source code of one chunk by its ID from search results.
-| Parameter | Type | Description |
-| :--- | :--- | :--- |
-| `target_function` | `string` | Exact function name (e.g. `validateToken`) |
+| Parameter | Type | Default | Description |
+| :--- | :--- | :--- | :--- |
+| `chunk_id` | string | required | ID shown in `search_code` results |
+| `view` | `"full"` \| `"signature"` | `"full"` | `"full"` = complete body; `"signature"` = first line only |
 ---
-### `graph://dependencies/{file_path}` Resource
+### `get_chunk_summary`
-Fetches the full bidirectional dependency topology for a file without consuming any search quota.
+Returns the interface of a chunk (signature + docstring + calls) without the body. About 50 tokens vs ~300 for the full body.
-```
-URI: graph://dependencies/src/middleware/auth.ts
-```
+| Parameter | Type | Description |
+| :--- | :--- | :--- |
+| `chunk_id` | string | ID from `search_code` results |
-Returns which files this file imports, and which files import it.
+Use when you need to understand what a function does without reading its implementation.
 ---
-## Supported Languages
-| Language | Extensions | Chunk Types | Import Resolution |
-| :--- | :--- | :--- | :--- |
-| TypeScript / TSX | `.ts`, `.tsx` | functions, classes, methods, exports | Absolute & relative paths |
-| JavaScript | `.js`, `.jsx`, `.mjs`, `.cjs` | functions, classes, expressions | Absolute & relative paths |
-| Python | `.py` | function_definition, class_definition | Relative imports (`.`, `..`) → file paths |
-| Rust | `.rs` | fn, struct, enum, trait, impl | `crate::` path resolution |
-| Go | `.go` | function_declaration, method, type | Import spec resolution |
-| PHP | `.php` | function, class | include/require paths |
-| CSS / SCSS | `.css`, `.scss` | rule_set | — |
----
-## Performance & Evaluation
+### `get_file_skeleton`
-Validate indexer quality on your own codebase:
+Returns all top-level exports and definitions in a file with line numbers — no bodies.
-```bash
-npm run test
-# or with custom Ollama:
-OLLAMA_HOST=http://localhost:11435 npm run test
-```
+| Parameter | Type | Description |
+| :--- | :--- | :--- |
+| `file_path` | string | Relative path (e.g. `src/utils/auth.ts`) |
-### Benchmark Results
+Useful for understanding what a file exports before deciding which chunk to read.
-| Scenario | Recall@3 | Recall@5 | Latency (p50) |
-| :--- | :--- | :--- | :--- |
-| graph-indexer own codebase (34 chunks, lexical-only) | **100%** | 100% | 0.07 ms |
-| graph-indexer own codebase (34 chunks, with Ollama) | **75%** | 100% | 0.14 ms |
-| React Native project (311 chunks, with Ollama) | **58%** | 75% | 1.02 ms |
+---
-> The React Native project numbers are without any JSDoc in the source files. Adding docstrings to exported functions (see Best Practices below) brings recall to **100% @3**.
+### `get_call_graph`
-### Vector Search Performance
+Finds all chunks that call a specific function by name.
-| Corpus size | Pure JS flat scan | HNSW (auto-activated ≥5k) |
+| Parameter | Type | Description |
 | :--- | :--- | :--- |
-| 1,000 chunks (dim=768) | 0.9 ms/query | — |
-| 5,000 chunks (dim=768) | 4.5 ms/query | 1.2 ms/query |
-| 20,000 chunks (dim=768) | 30.5 ms/query | 3.1 ms/query |
+| `target_function` | string | Exact function name (e.g. `"validateToken"`) |
----
+Always call this before modifying an exported function — it shows every call site repo-wide.
-## Best Practices for 100% Recall
+---
-### 1. Add JSDoc/TSDoc to Exported Functions
+### `get_repo_map`
-The indexer embeds your docstrings alongside code. Docstrings dramatically improve semantic search:
+Returns a compact symbol map of the entire codebase ordered by importance (PageRank over the dependency graph — most-imported files first). Useful for orienting in an unfamiliar codebase in ~1,500 tokens.
-```typescript
-/**
- * Validates and decodes a JWT access token.
- * Throws AuthError if the token is expired or malformed.
- *
- * @param token - Raw JWT string from Authorization header
- * @returns Decoded payload with userId, role, and expiry
- */
-export function validateToken(token: string): TokenPayload { ... }
-```
+| Parameter | Type | Default | Description |
+| :--- | :--- | :--- | :--- |
+| `path_filter` | string? | — | Only show files whose path contains this string |
+| `max_files` | number | `80` | Maximum files to include |
+| `sort_by` | `"importance"` \| `"path"` | `"importance"` | Sort order |
-### 2. Use Descriptive, Specific Names
+---
-The exact-name boost gives a free rank-1 slot to any chunk whose name exactly matches `exact_tokens`. Descriptive names make this reliable:
+### `list_index_stats`
-```typescript
-// ✅ Exact-name searchable
-export const useNotificationPermissionStatus = () => { ... };
-export function calculateTripDurationInDays(start: Date, end: Date) { ... }
+Returns index health: chunk count, file count, symbol table size, vector entry count, search mode (hybrid or lexical-only), daemon status, and index age.
-// ❌ Generic — impossible to target without semantic search
-export const util = () => { ... };
-export function process(data: any) { ... }
-```
+---
-### 3. Expand Minimal Utility Files
+### `graph://dependencies/{file_path}` (resource)
-Files with fewer than ~5 indexable lines produce zero or one chunk. Add error handling and validation to make them indexable:
+Returns the full bidirectional dependency topology for a file: what it imports and what imports it.
-```typescript
-// ❌ 2 lines — may not be indexed
-export const Context = createContext(null);
-export const useCtx = () => useContext(Context);
+---
-// ✅ Indexable with meaningful docstring + validation
-/**
- * Context for accessing authentication state throughout the component tree.
- */
-export const AuthContext = createContext<AuthState | null>(null);
+## Configuration
-/**
- * Hook for consuming AuthContext. Throws if used outside AuthProvider.
- */
-export function useAuth(): AuthState {
-  const ctx = useContext(AuthContext);
-  if (!ctx) throw new Error('useAuth must be used inside <AuthProvider>');
-  return ctx;
-}
-```
+### Environment variables
-### 4. Keep Imports Explicit and Organized
+| Variable | Default | Description |
+| :--- | :--- | :--- |
+| `MCP_PROJECT_ROOT` | `process.cwd()` | Project root for import resolution |
+| `OLLAMA_HOST` | `http://localhost:11434` | Ollama API endpoint |
+| `INDEXER_EMBEDDINGS` | — | Set to `off` to disable vector embeddings |
-The topology graph follows import paths. Barrel re-exports from `index.ts` break dependency resolution — prefer direct imports:
+### Custom Ollama host
-```typescript
-// ✅ Topology-friendly
-import { useAuthStore } from '@/stores/authStore';
-import { validateEmail } from '@/utils/validators';
+```bash
+# Non-default port
+OLLAMA_HOST=http://localhost:11435 npm run mcp:index
-// ❌ Breaks topology — indexer can't resolve what's actually imported
-import { useAuthStore, validateEmail } from '@/index';
+# Remote host
+OLLAMA_HOST=http://192.168.1.100:11434 npm run mcp:index
 ```
 ---
-## File Structure
+## Supported languages
-```
-graph-indexer/
-├── core-engine.mjs        # In-memory index: vector matrix, TF-IDF, RRF search, save/load
-├── parser-utils.mjs       # Tree-sitter AST parsing, 6 language grammars, Ollama embeddings
-├── indexer.mjs            # CLI bootstrap: scan repo → parse → embed → write index
-├── watch-daemon.mjs       # Chokidar daemon: incremental real-time index updates
-├── mcp-server.mjs         # MCP stdio server: exposes tools + graph:// resources
-├── PROMPT.md              # Agent system prompt — paste into your IDE's agent instructions
-├── tests/
-│   ├── eval-harness.mjs   # Recall@k + latency evaluation against the indexer's own codebase
-│   └── react-native-test.mjs  # Real-world quality test on a 311-chunk React Native project
-└── package.json
-```
+| Language | Extensions | Chunk types extracted |
+| :--- | :--- | :--- |
+| TypeScript / TSX | `.ts`, `.tsx` | functions, classes, methods, exports |
+| JavaScript | `.js`, `.jsx`, `.mjs`, `.cjs` | functions, classes, expressions |
+| Python | `.py` | function_definition, class_definition |
+| Go | `.go` | function_declaration, method, type |
+| Rust | `.rs` | fn, struct, enum, trait, impl |
+| Java | `.java` | class, method, interface, constructor, enum |
+| Kotlin | `.kt`, `.kts` | function, class, object, companion |
+| C# | `.cs` | class, method, interface, property, enum |
+| Ruby | `.rb` | method, class, module |
+| PHP | `.php` | function, class |
+| CSS / SCSS | `.css`, `.scss` | rule_set |
 ---
-## Mathematical Foundation
-### Sublinear TF Scaling
+## Requirements
-Term frequencies use sublinear scaling to prevent common tokens (`return`, `const`) from burying business-logic terms:
-$$w(t,d) = 1 + \log(f_{t,d})$$
-### Reciprocal Rank Fusion
+| | |
+| :--- | :--- |
+| **Node.js** | v18+ (ES Modules) |
+| **Ollama** | Optional — enables semantic search. Pull `nomic-embed-text`. |
+| **Disk** | `code-index.json` + `code-index.embeddings.bin` (both gitignored by default) |
-Vector and lexical results are merged by rank position, not raw incompatible scores:
+---
-$$\text{RRF}(d) = \sum_{i \in \{\text{vec, lex}\}} \frac{1}{K + \text{rank}_i(d)}, \quad K = 60$$
+## Troubleshooting
-### Exact-Name Boost
+**Index takes too long to build**
+- Verify Ollama is running: `curl http://localhost:11434/api/tags`
+- Fall back to lexical-only: `INDEXER_EMBEDDINGS=off npm run mcp:index`
-Any chunk whose name exactly matches `exact_tokens` receives an additive bonus equal to the maximum single-list RRF contribution:
+**Search returns irrelevant results**
+- Use `exact_tokens` for known symbol names
+- Verify the chunk is indexed with `list_index_stats()`
+- Increase `top_k` to see more candidates
-$$\Delta\text{score} = \frac{1}{K + 1}$$
+**MCP server won't connect**
+- Verify `MCP_PROJECT_ROOT` points to the indexed directory
+- Check that `code-index.json` exists: `ls -la code-index.json`
+- Test the server manually: `node mcp-server.mjs`
-This guarantees rank-1 placement for any exact symbol name query.
+**Results are stale after file changes**
+- The background daemon auto-updates; if changes aren't reflected, run `npm run mcp:index` manually
+- Check daemon status with `list_index_stats()`
 ---
-## How the Index Files Work
-| File | Purpose | Typical Size |
-| :--- | :--- | :--- |
-| `code-index.json` | Chunk metadata (names, file paths, docstrings, calls, graph) | ~200 KB / 1k chunks |
-| `code-index.embeddings.bin` | Binary `float32` embedding vectors (4.8× smaller than JSON) | ~3 MB / 1k chunks @768-dim |
+## Development
-Both files are written atomically via a `tmp → rename` pattern to prevent corruption on crash. Both should be added to `.gitignore`.
+```bash
+git clone https://github.com/MaquinaTech/graph-indexer.git
+cd graph-indexer
+npm install
+npm run mcp:index   # index this repo
+npm run test        # run test suite (lexical-only)
+npm run mcp:start   # start MCP server
+```
 ---
-## Environment Variables
+## Best practices for better search quality
-| Variable | Default | Description |
-| :--- | :--- | :--- |
-| `OLLAMA_HOST` | `http://localhost:11434` | Ollama API base URL |
-| `INDEXER_EMBEDDINGS` | — | Set to `off` to disable embeddings entirely |
-| `MCP_PROJECT_ROOT` | `process.cwd()` | Absolute path to the project being indexed |
+**Write descriptive docstrings.** The indexer embeds docstrings alongside code. Better documentation directly improves semantic search quality.
----
+**Use specific function names.** Functions with exact, descriptive names (e.g. `fetchAndCacheUserProfile`) hit the exact-name boost (2.0×) and return as rank-1 results when queried by name.
-## License & Intellectual Property
+**Export named functions, not anonymous ones.** Anonymous default exports cannot be targeted by `resolve_symbol` and score lower in name-boost ranking.
-Released under the [GNU General Public License v3.0](LICENSE).
-This license ensures:
-- **Copyleft protection**: Any software that incorporates this code must also be released under GPL-3.0.
-- **Attribution**: The original author (MaquinaTech) must be credited.
-- **Source availability**: Any distributed modifications must include source code.
-Commercial use is permitted but requires GPL-3.0 compliance. For proprietary licensing inquiries, contact the author.
+**Avoid catch-all utility files.** Large files that mix unrelated utilities produce weaker per-chunk signals. One responsibility per module indexes more cleanly.
 ---
-## Author
+## License
-Built and maintained by **MaquinaTech**.
+Released under the [MIT License](LICENSE). Copyright (c) 2026 MaquinaTech.
-- **GitHub:** [github.com/MaquinaTech](https://github.com/MaquinaTech)
-- **NPM:** [npmjs.com/package/graph-indexer](https://www.npmjs.com/package/graph-indexer)
-- **Issues:** [github.com/MaquinaTech/graph-indexer/issues](https://github.com/MaquinaTech/graph-indexer/issues)
+---
-Issues, pull requests, and feature discussions are welcome.
+Built by [MaquinaTech](https://github.com/MaquinaTech) · [Issues](https://github.com/MaquinaTech/graph-indexer/issues) · [npm](https://www.npmjs.com/package/graph-indexer)