graph-indexer 1.2.0 → 2.1.0

package/README.md

@@ -1,17 +1,12 @@
 <div align="center">
-  <!-- Reemplaza la URL de abajo con la ruta a tu nuevo logo profesional en formato SVG o PNG transparente -->
   <img src="https://raw.githubusercontent.com/MaquinaTech/graph-indexer/main/assets/logo.jpg" alt="Graph Indexer Logo" width="250" />
 
   <h1>Graph Indexer</h1>
 
   <p>
-    <strong>Zero-Database · Air-Gapped · AST-Precise · Hybrid Search · Monorepo-Scale</strong>
-  </p>
-  <p>
-    <em>An MCP server that lets AI coding agents search your codebase by concept instead of reading files, cutting context usage by 65–88% per query.</em>
+    <em>A local MCP server that gives AI coding agents an AST-precise, topology-aware index of your codebase.</em>
   </p>
 
-  <!-- Badges -->
   <a href="https://www.npmjs.com/package/graph-indexer"><img src="https://img.shields.io/npm/v/graph-indexer?color=007acc&style=for-the-badge" alt="npm version"></a>
   <a href="https://www.npmjs.com/package/graph-indexer"><img src="https://img.shields.io/npm/dt/graph-indexer?color=4caf50&style=for-the-badge" alt="NPM Downloads"></a>
   <a href="https://nodejs.org"><img src="https://img.shields.io/badge/Node.js-%3E%3D18-brightgreen?style=for-the-badge&logo=nodedotjs" alt="Node.js 18+"></a>
@@ -20,683 +15,351 @@
 
 <br />
 
----
-
-## Table of Contents
-
-- [What it does](#what-it-does)
-- [Results](#results)
-- [Getting started](#getting-started)
-- [Storage backends](#storage-backends)
-- [Semantic enrichment](#semantic-enrichment)
-- [MCP tools](#mcp-tools)
-- [Configuration](#configuration)
-- [Supported languages](#supported-languages)
-- [How it works](#how-it-works)
-- [Best practices](#best-practices)
-- [Troubleshooting](#troubleshooting)
-- [Development](#development)
-- [Security](#security)
-- [License](#license)
-
----
-
 ## What it does
 
-graph-indexer pre-indexes your codebase into an AST-precise search index and exposes it as an MCP server. Instead of reading full files, AI agents call `search_code("payment validation")` and get back the exact functions that match — with type signatures, docstrings, decorators, inheritance edges, dependency graph, and call sites — in a fraction of the tokens.
-
-It runs entirely on your machine. No external database, no cloud APIs, no telemetry. Two design choices make it suitable for any repository size:
+Graph Indexer is a local [Model Context Protocol](https://modelcontextprotocol.io) server that builds an AST-precise index of your repository with Tree-sitter and serves it to AI coding agents. Instead of grepping text or embedding whole files, it indexes *semantic chunks* — functions, classes, methods — and the call graph and import topology that connect them, so an agent can find the right symbol, see every caller and dependency that would be affected by a change, and resolve references across files. It runs entirely on your machine: the default search path is lexical (BM25 + morphological stemming) and needs no model, no daemon, and no network. Dense vector embeddings, LLM enrichment, and an LLM reranker are all available but off by default — you opt into each one when you have a measured reason to.
 
-- A **default in-memory engine** that is instant and dependency-free, and an optional **disk-backed SQLite backend** that holds the index on disk so retrieval RAM stays flat on enterprise monorepos that would otherwise exhaust Node's heap.
-- Optional **local-LLM semantic enrichment** that teaches the index what each core component *does*, so conceptual queries match code that shares none of their words.
+**Why it matters for AI coding agents:**
 
-**Token comparison:**
+- **Up to 98.2% Token Savings:** Delivers the exact AST chunk needed instead of dumping 1M token context files.
+- **Blast radius before every edit.** `get_call_graph` and `find_references` surface every caller, subclass, and dependency a change would touch, so the agent reasons about impact on code it never opened.
+- **Private by default.** Everything runs locally; the default path makes zero network calls and needs no model — your code never leaves your machine. (The very first index build fetches only the Tree-sitter grammar(s) for languages actually in your repo — see [Language grammars](#language-grammars) — cached from then on; `--sealed` enforces zero network hard.)
+- **One command, any language.** Guided setup wires your editors in seconds, and the lexical engine indexes 14 languages — installing each one's grammar on first use, not bundled, so a fresh `npm install graph-indexer` stays small.
 
-```
-Agent task: "Add error handling to the payment processing flow"
-
-Without graph-indexer:
-  readFile("src/payments/service.ts")   →  8,400 tokens
-  readFile("src/payments/handlers.ts")  →  6,200 tokens
-  readFile("src/types/payment.ts")      →  1,800 tokens
-  Total:                                ~ 16,400 tokens
-
-With graph-indexer:
-  search_code("payment processing")     →    650 tokens  (5 exact chunks)
-  get_chunk("chunk_id")                 →    280 tokens  (full function body)
-  Total:                                ~    930 tokens
-```
+## 📦 Prerequisites
 
-The difference compounds across a session. Agents that read files also lack topology — they can't see which other functions call the one they're modifying, so they miss side effects.
+* **Node.js** 18+ (22+ recommended if you hit >15k chunks for automatic SQLite scaling).
+* **OS:** Agnostic. macOS Apple Silicon required only for the optional MLX GPU acceleration.
+* **Network (first index build only):** installs the Tree-sitter grammar(s) for languages in your repo (a few MB each) — see [Language grammars](#language-grammars). Cached after; fully offline from then on, or pre-install before going air-gapped.
+* **Optional:** Ollama for embeddings, enrichment, and reranking (see [Ollama setup](#ollama-setup)).
+* **Optional:** Python 3.10+ for the MLX embedder (macOS Apple Silicon only).
 
----
+## Quick start
 
-## Results
-
-Measured across 5 production open-source codebases (8,296 AST chunks, 100 ground-truth queries:
-69 **symbolic** — the developer names or paraphrases the symbol — and 31 **semantic** — agent-style
-behavioural descriptions sharing few or no words with the code).
-
-* **Average Token Savings:** 79.0%
-* **Strict Success@5 (Hybrid + Rerank):** 82%
-* **Strict MRR (Symbolic queries):** 0.84
-
-### Token savings per query
-
-| Project | Language | Chunks | Recall@5 (loose) | Savings | 
-| :--- | :--- | ---: | ---: | ---: |
-| Axios v1.6.0 | JavaScript | 450 | 0.89 | **69.7%** |
-| Express 4.18.2 | JavaScript | 389 | 0.95 | **88.7%** |
-| NestJS v10.4.9 | TypeScript | 2,675 | 0.95 | **67.5%** |
-| FastAPI 0.103.0 | Python | 3,694 | 0.76 | **85.5%** |
-| Gin v1.9.1 | Go | 1,088 | 0.94 | **83.7%** |
-| **Mean** | | **8,296** | **0.90** | **79.0%** |
-
-> Savings = tokens in the returned chunk snippets vs the full source files containing the top-5
-> results, at 4 chars/token. Search latency is sub-millisecond lexical; the hybrid channel adds
-> one local embedding call plus a streamed vector scan (a few ms at this corpus size).
-
-### Search quality (strict)
-
-Strict scoring: a result counts **only** if its symbol name (or its class, for method chunks)
-exactly equals the ground truth — no substring matching, no file-path credit. Hybrid =
-BM25 + `nomic-embed-text` vectors with `qwen2.5-coder:1.5b` enrichment; rerank adds the
-opt-in `qwen2.5-coder:7b` judge on natural-language queries.
-
-| Project | Strict success@5 | **Rank-1 (strict)** | MRR (strict) |
-| :--- | ---: | ---: | ---: |
-| Axios v1.6.0 | 0.74 | 0.68 | 0.72 |
-| Express 4.18.2 | 0.90 | 0.62 | 0.73 |
-| NestJS v10.4.9 | 0.67 | 0.57 | 0.60 |
-| FastAPI 0.103.0 | 0.81 | 0.67 | 0.74 |
-| Gin v1.9.1 | 1.00 | 0.78 | 0.84 |
-| **Mean (hybrid + rerank)** | **0.82** | **0.66** | **0.73** |
-
-Split by query style and configuration (the numbers that matter for agent workflows):
-
-| Channel | Rank-1 | MRR | Success@5 |
-| :--- | ---: | ---: | ---: |
-| **Symbolic** (name-lookup, 69q), hybrid | **0.80** | **0.84** | — |
-| **Semantic** (behavioural, 31q), hybrid + rerank | **0.35** | **0.47** | **0.65** |
-| Semantic, hybrid (no rerank) | 0.23 | 0.37 | 0.55 |
-| Semantic, lexical-only (no Ollama) | 0.19 | 0.29 | 0.48 |
-
-Read the semantic rows with their denominator in mind: under *strict* scoring several of those
-queries are unwinnable in-repo (the expected symbol is an anonymous default export, or is
-re-exported from a dependency whose implementation isn't in the codebase), and most strict
-"misses" still land the right file in the top-2 — which the loose channel (0.89) credits and an
-agent can use. Symbol-naming queries — the bulk of real agent traffic — find the exact chunk
-first try 80% of the time and in the top ranks (MRR 0.84) almost always.
-
-Ranking favours exact symbol matches even for short, high-signal names — `res.json`,
-`req.get`, `app.all` — by gating the name boost on corpus **document frequency** instead of a
-blunt length cutoff (with singular/plural equivalence, so `BackgroundTask` finds
-`BackgroundTasks`). Natural-language queries are detected and ranked **vector-first** (with the
-name/path boosts gated on semantic agreement), so behavioural questions aren't drowned by
-keyword noise. Test, spec, example and sandbox chunks are demoted in both channels.
-
-**Backend parity is exact**: every rank-assigning sort uses deterministic tie-breaking and both
-stores funnel vector candidates through one shared finalizer, so the in-memory engine and the
-SQLite store return **identical top-5 chunk ids for 100/100 benchmark queries** — verified, not
-approximate.
-
-Reproduce every view:
+### 1. Run the Interactive Indexer
+Go to your project root (works for Python, Go, Rust, TS/JS, C#, and 9 more languages) and run:
 
 ```bash
-npm run test:setup     # clone the 5 fixture repos
-npm run test           # loose hit-rate + token savings + indexing stats
-npm run test:eval      # strict symbol-level accuracy, lexical channel
-OLLAMA_HOST=http://localhost:11434 node test/evaluate.mjs --embeddings              # hybrid
-OLLAMA_HOST=http://localhost:11434 node test/evaluate.mjs --embeddings --use-sqlite # parity
-OLLAMA_HOST=http://localhost:11434 RERANK_MODEL=qwen2.5-coder:7b \
-  node test/evaluate.mjs --embeddings --rerank                                      # + judge
+npx graph-indexer /path/to/your/repo
 ```
 
-> The strict harness ([test/evaluate.mjs](test/evaluate.mjs)) exists specifically to keep
-> these numbers honest and prevent the tool from being tuned to a friendly benchmark — it
-> reports the inflation gap (loose minus strict) per suite so regressions in real precision
-> can't hide behind a permissive hit-rate, and it splits the symbolic and semantic channels
-> so neither inflates the other.
-
----
-
-## Getting started
-
-**Requirements:** Node.js v18+ · Ollama (optional, for semantic search and enrichment) · Node.js v22.5+ for the optional SQLite backend
+That runs the [guided setup](#guided-setup) against that repo, which leaves it ready to use. It is idempotent — re-run it whenever you add a language or change a setting; it merges into what you already have and never clobbers another tool's config. Every generated file lands in `.graph-indexer/` (git-ignored), so your repo root stays clean.
 
-### 1. Install
+### 2. Connect Your Agent
 
-```bash
-npm install graph-indexer --save-dev
-npx graph-indexer init
-```
-
-`init` auto-detects your IDE (Claude, Cursor, VS Code), adds npm scripts to `package.json`, updates `.gitignore`, and opens an interactive language selector:
+**If you ran the guided setup and selected your agent in step 4, this is already done — skip ahead.**
 
-```
-⚙️  Select languages (Arrows/Tab: move, Space: toggle, Enter: confirm):
-
-  ❯ ◯ TypeScript / TSX         .ts, .tsx
-    ◯ JavaScript               .js, .jsx, .mjs, .cjs
-    ◯ Python                   .py
-    ◯ Go                       .go
-    ◯ Rust                     .rs
-    ◯ PHP                      .php
-    ◯ Java                     .java
-    ◯ Kotlin                   .kt, .kts
-    ◯ C#                       .cs
-    ◯ Ruby                     .rb
-    ◯ CSS / SCSS               .css, .scss
-```
+The guided setup writes the MCP config automatically (with absolute paths that survive GUI launches) for **Claude Code** (`.mcp.json`), **Claude Desktop**, **Cursor** (`.cursor/mcp.json`), and **VS Code** (`.vscode/mcp.json`). Re-run `npx graph-indexer /path/to/your/repo` at any time to add or refresh a client.
 
-Navigate with **↑ ↓**, toggle with **Space**, confirm with **Enter**. Leaving all unselected enables every language. Your selection is saved to `.graph-indexer.json`. Pass `--all-languages` to skip the prompt.
+<details>
+<summary>Manual wiring (if you skipped setup or use a client not listed above)</summary>
 
-### 2. Index your codebase
+#### Claude Code — CLI
 
-**With semantic search (recommended):**
 ```bash
-# Install Ollama: https://ollama.ai
-ollama pull nomic-embed-text
-npm run mcp:index
+claude mcp add graph-indexer -- npx -y -p graph-indexer idx-mcp --repo /path/to/your/repo
 ```
 
-**Lexical-only (no Ollama required):**
-```bash
-INDEXER_EMBEDDINGS=off npm run mcp:index
-```
-
-Lexical-only still scores loose recall@5 = 0.90 and strict symbolic MRR 0.81 on the benchmarks above — only the semantic (behavioural-query) channel needs embeddings.
-
-**On a large monorepo, keep retrieval RAM flat with the SQLite backend:**
-```bash
-npm run mcp:index -- --use-sqlite
-```
-
-**Sharpen conceptual recall with local-LLM enrichment:**
-```bash
-ollama pull qwen2.5-coder:1.5b
-npm run mcp:index -- --llm-enrichment
-```
-
-The two options compose (`--use-sqlite --llm-enrichment`) and can be made permanent in `.graph-indexer.json` — see [Configuration](#configuration).
+#### Cursor — `.cursor/mcp.json`
 
-### 3. Configure your IDE
-
-`init` writes the config automatically. Manual examples:
-
-**Claude Desktop** (`~/Library/Application Support/Claude/claude_desktop_config.json`):
 ```json
 {
   "mcpServers": {
     "graph-indexer": {
-      "command": "npm",
-      "args": ["run", "--prefix", "/path/to/project", "mcp:start"],
-      "env": { "MCP_PROJECT_ROOT": "/path/to/project" }
+      "command": "npx",
+      "args": ["-y", "-p", "graph-indexer", "idx-mcp", "--repo", "/path/to/your/repo"]
     }
   }
 }
 ```
 
-**Cursor** (`.cursor/mcp.json`) and **Claude Code** (`.mcp.json`):
+#### VS Code — `.vscode/mcp.json`
+
 ```json
 {
-  "mcpServers": {
+  "servers": {
     "graph-indexer": {
-      "command": "npm",
-      "args": ["run", "mcp:start"],
-      "env": { "MCP_PROJECT_ROOT": "${workspaceFolder}" }
+      "command": "npx",
+      "args": ["-y", "-p", "graph-indexer", "idx-mcp", "--repo", "/path/to/your/repo"]
     }
   }
 }
 ```
 
-**VS Code** (`.vscode/mcp.json`):
+#### Claude Desktop — `claude_desktop_config.json`
+
 ```json
 {
-  "servers": {
+  "mcpServers": {
     "graph-indexer": {
-      "type": "stdio",
-      "command": "npm",
-      "args": ["run", "mcp:start"],
-      "env": { "MCP_PROJECT_ROOT": "${workspaceFolder}" }
+      "command": "npx",
+      "args": ["-y", "-p", "graph-indexer", "idx-mcp", "--repo", "/path/to/your/repo"]
     }
   }
 }
 ```
 
-The server reads the backend from `.graph-indexer.json`, so the same launch config works for both the in-memory and SQLite indexes.
-
-### 4. Add the agent system prompt
-
-Copy [PROMPT.md](./PROMPT.md) into your AI agent's system prompt to instruct it to use graph-indexer tools instead of reading files directly.
-
----
-
-## Storage backends
-
-The same index, the same tools, the same ranking — two ways to hold the data. The MCP tools are written against a storage contract and never see which backend is active.
-
-| Backend | Default | Resident RAM* | Dependencies | Best for |
-| :--- | :--- | ---: | :--- | :--- |
-| **In-memory** | ✅ | ~586 MB | none | Single packages and services; instant cold start. |
-| **SQLite** | `--use-sqlite` | ~79 MB | none — uses Node's built-in `node:sqlite` | Monorepos / 1M+ LOC where holding every chunk in the heap would OOM. |
-
-> *Resident set serving the **same** 50,000-chunk corpus, measured in isolated processes by
-> [test/scale.mjs](test/scale.mjs). The in-memory engine keeps every chunk and the full inverted
-> index in the heap, so its footprint grows with the codebase; the SQLite store keeps only the
-> small file-level dependency graph resident and reads chunks, posting lists and vectors from disk
-> on demand, so its footprint stays flat regardless of corpus size — an **87% smaller** resident
-> set here, and the gap widens as the repo grows.
-
-The SQLite backend adds **no external dependency**: `node:sqlite` ships inside Node (v22.5+). Lexical BM25 reads from an indexed `postings` table, symbols and call edges from indexed columns, and vectors live in the shared `code-index.embeddings.bin` — point reads are `pread` on demand. Because both backends feed the same fusion-and-boost ranker with deterministic tie-breaking, switching is purely an operational choice with **zero** quality trade-off (identical top-5 ids on the full benchmark).
-
-### Vector search that stays fast at monorepo scale
-
-Below 10k vectors the semantic channel runs an exact streaming scan of the bin (~20 ms worst
-case). Above it, a **binary-quantized sketch** takes over: each vector's sign bits (768 dims →
-96 bytes, 0.1 % of the float data) are kept in RAM; a query does a Hamming-distance pass over
-packed Uint32 words, then exact-cosine rescores only the top candidates from disk. Measured on
-synthetic corpora (warm):
-
-| Corpus | Exact scan | Sketch | Speedup | Sketch RAM |
-| ---: | ---: | ---: | ---: | ---: |
-| 50,000 vectors | 104 ms | **5–11 ms** | ~20× | ~9 MB |
-| 200,000 vectors | 519 ms | **11 ms** | ~36× | ~35 MB |
-
-The sketch recovers the exact top-10 (10/10 in validation) because it rescores 2× the candidate
-budget with true cosine before ranking. It is **append-aware**: daemon updates extend it by
-scanning only the unseen tail of the bin, and a full re-index is detected by fingerprint and
-triggers a rebuild. `test/scale.mjs` asserts hybrid queries stay interactive (<60 ms) at 50k
-chunks on both backends.
-
-```bash
-npm run mcp:index -- --use-sqlite     # writes code-index.db
-```
-
-### Live updates on both backends
-
-The watch daemon keeps **whichever backend is configured** fresh, incrementally:
-
-- **In-memory** — the daemon rewrites the JSON snapshot; a running MCP server watches the file and reloads it automatically.
-- **SQLite** — each file save becomes one WAL transaction (`applyFileUpdate`): the file's old chunks, postings and call edges are replaced with exact BM25 bookkeeping, and new vectors are *appended* to the embeddings bin (O(changed chunks), never a full rewrite). Running MCP servers detect the commit via `PRAGMA data_version` on their next query and refresh themselves — no re-indexing, no restarts.
-
-Daemon startup is also O(changed files): files older than the index artifact are skipped during the initial scan, so edits made while the daemon was down are picked up without re-parsing the whole repo.
-
----
-
-## Semantic enrichment
-
-Embeddings match text proximity, not intent. A query like *"payment gateway webhook bottleneck"* misses the function that handles it if that function never spells out those words. Enrichment closes that gap.
-
-With `--llm-enrichment`, the indexer routes every substantive **production-source chunk** (tests, specs and example trees are excluded — agents search for implementations) through a local LLM via Ollama, producing per chunk:
-
-- a one-line **summary** in developer vocabulary, and
-- a set of **concept tags** (e.g. `authentication, JWT, middleware`).
-
-These ride **three** retrieval paths: the tags join the chunk's BM25 lexical document as high-IDF domain terms; the summary leads the code-payload embedding; and — decisive for natural-language queries — each enriched chunk gets a **second, summary-only vector**. A one-line behavioural query embeds far closer to a one-line summary than to hundreds of characters of code, so conceptual searches hit code that shares none of their words.
-
-Enrichment is **incremental**: results are cached in `code-index.enrichment.json` keyed by content hash, so a re-index only sends new or changed code to the LLM (the embedding cache keys account for enrichment too, so nothing is re-embedded needlessly). Each run enriches up to `maxChunks` new chunks, ordered by file centrality (PageRank), and coverage accumulates across runs. The watch daemon re-attaches cached enrichment on every file save and live-enriches changed chunks. Generation is best-effort — if the model is unreachable the index is built without enrichment rather than failing.
-
-```bash
-ollama pull qwen2.5-coder:1.5b
-npm run mcp:index -- --llm-enrichment
-```
-
-The model, the file-selection ratio and the per-run cap are all configurable (see [Configuration](#configuration)). Enrichment is fully optional and leaves the default index byte-for-byte unchanged when disabled.
-
-### LLM reranking (query time, opt-in)
-
-Fusion resolves most queries, but a natural-language query can end in a near-tie between the
-semantically right chunk and a lexically similar neighbour — a gap no static boost can close.
-With `rerank.enabled` (or `search_code(..., rerank: true)`), the fused top-8 is shown to a local
-LLM judge — one line per candidate — which reorders them. Measured effect on the strict semantic
-channel: **rank-1 0.23 → 0.35, MRR 0.37 → 0.47, success@5 0.55 → 0.65**, with the symbolic
-channel untouched (the judge only fires on natural-language queries, never on symbol lookups or
-`exact_tokens` calls). Cost: one generation call (~1–2 s with `qwen2.5-coder:7b`). Best-effort —
-any model failure preserves the original order.
-
----
-
-## MCP tools
-
-### `search_code`
-
-Hybrid BM25 + vector search over the index. The primary entry point for agents.
-
-| Parameter | Type | Default | Description |
-| :--- | :--- | :--- | :--- |
-| `query` | string | required | Natural language description of the logic to find |
-| `exact_tokens` | string? | — | Exact symbol name — guarantees rank-1 placement |
-| `detail` | `"signatures"` \| `"smart"` \| `"full"` | `"smart"` | How much code to return per result |
-| `top_k` | number | `5` | Results to return (1–20) |
-| `include_topology` | boolean | `true` | Include imports / used-by / calls |
-| `min_score` | number | `0.3` | Minimum relevance threshold |
-| `token_budget` | number? | auto | Cap total tokens returned |
-| `rerank` | boolean? | config | LLM-judge rerank for natural-language queries (see `rerank.*` config) |
+> **Note:** The `-p graph-indexer idx-mcp` form is required — `npx graph-indexer` runs the setup wizard, not the MCP server. If `npx` isn't on your GUI PATH (common on macOS when launching editors from Finder/Dock), run the guided setup instead — it writes absolute paths that always work.
 
-**`detail` levels:**
+</details>
 
-| Value | Cost | Returns |
-| :--- | ---: | :--- |
-| `"signatures"` | ~20 tok | Name, type, params, return type, topology |
-| `"smart"` (default) | ~150 tok | Signature + lines relevant to the query |
-| `"full"` | ~300 tok | Signature + complete source body |
+Once connected, the agent can call `search_code`. A query like `search_code("rate limiting middleware")` returns ranked semantic chunks, not whole files:
 
-For purely conceptual queries with no lexical overlap, `"smart"` falls back to a structural skeleton — control-flow lines and call sites — so the agent always gets meaningful signal about what the code does, never a blind truncation.
-
----
-
-### `resolve_symbol`
-
-O(1) lookup by exact name — no search ranking needed.
-
-| Parameter | Type | Description |
-| :--- | :--- | :--- |
-| `symbol` | string | Exact function, class, or type name (e.g. `"validateToken"`) |
-
----
-
-### `get_chunk`
-
-Returns the source code of a single chunk by its ID.
-
-| Parameter | Type | Default | Description |
-| :--- | :--- | :--- | :--- |
-| `chunk_id` | string | required | ID from `search_code` results |
-| `view` | `"full"` \| `"signature"` | `"full"` | Full body or first line only |
-
----
-
-### `get_chunk_summary`
-
-Signature + docstring + calls without the body. ~50 tokens vs ~300 for full.
-
-| Parameter | Type | Default | Description |
-| :--- | :--- | :--- | :--- |
-| `chunk_id` | string | required | ID from `search_code` results |
-| `expand_calls` | boolean | `false` | Resolve each outgoing call's signature inline (~150 tok) instead of issuing a follow-up tool call per dependency |
-
----
-
-### `get_file_skeleton`
-
-All top-level exports and definitions in a file with line numbers — no bodies.
-
-| Parameter | Type | Description |
-| :--- | :--- | :--- |
-| `file_path` | string | Relative path (e.g. `src/utils/auth.ts`) |
-
----
-
-### `get_call_graph`
-
-Every chunk across the repo that calls a specific function.
-
-| Parameter | Type | Description |
-| :--- | :--- | :--- |
-| `target_function` | string | Exact function name (e.g. `"validateToken"`) |
-
-Call this before modifying any exported function to find all affected call sites.
-
----
-
-### `get_repo_map`
-
-Compact symbol map ordered by PageRank (most-imported files first). Orients agents in an unfamiliar codebase in ~1,500 tokens.
-
-| Parameter | Type | Default | Description |
-| :--- | :--- | :--- | :--- |
-| `path_filter` | string? | — | Limit to files whose path contains this string |
-| `max_files` | number | `80` | Maximum files to include |
-| `sort_by` | `"importance"` \| `"path"` | `"importance"` | Sort order |
-
----
-
-### `list_index_stats`
-
-Index health snapshot: storage backend, chunk count, file count, symbol table size, vector count, search mode, daemon status, index age.
-
----
-
-### `graph://dependencies/{file_path}` (resource)
-
-Full bidirectional dependency topology for a file: what it imports and what imports it.
-
----
-
-## Configuration
-
-### `.graph-indexer.json`
-
-All persistent settings live in one file at the project root, written by `init` and read by the indexer, watcher and server:
-
-```json
-{
-  "languages": ["typescript", "javascript", "python"],
-  "storage": "sqlite",
-  "enrichment": {
-    "enabled": true,
-    "model": "qwen2.5-coder:1.5b",
-    "coreRatio": 0.15,
-    "maxChunks": 400
+```jsonc
+[
+  {
+    "score": 8.41,
+    "chunk": {
+      "id": "src/middleware/rateLimit.ts:14",
+      "file_path": "src/middleware/rateLimit.ts",
+      "name": "rateLimiter",
+      "node_type": "function_declaration",
+      "start_line": 14, "end_line": 47,
+      "calls": ["tokenBucket", "getClientKey"],
+      "class_context": ""
+    }
   }
-}
+]
 ```
 
-| Key | Default | Description |
-| :--- | :--- | :--- |
-| `languages` | all installed | Parsers to load. Valid keys: `typescript`, `javascript`, `python`, `go`, `rust`, `php`, `java`, `kotlin`, `csharp`, `ruby`, `css`. |
-| `storage` | `"memory"` | `"memory"` (in-heap, zero-dependency) or `"sqlite"` (disk-backed). |
-| `ollamaHost` | `"http://localhost:11434"` | Ollama endpoint (also settable via `OLLAMA_HOST`). |
-| `embedModel` | `"nomic-embed-text"` | Embedding model (index and queries must use the same one). |
-| `enrichment.enabled` | `false` | Run local-LLM enrichment during indexing. |
-| `enrichment.model` | `"qwen2.5-coder:1.5b"` | Ollama model used for generation. |
-| `enrichment.coreRatio` | `1.0` | Share of production files eligible (by PageRank). `1.0` = all; tests/examples are always excluded. |
-| `enrichment.maxChunks` | `500` | Cap on **new** LLM calls per index run — the cache accumulates coverage across runs. |
-| `enrichment.concurrency` | `12` | Parallel Ollama requests during enrichment. |
-| `rerank.enabled` | `false` | LLM-judge reranking of natural-language queries (+50% semantic rank-1, ~1–2 s per NL query). |
-| `rerank.model` | `"qwen2.5-coder:7b"` | Judge model. Quality matters: 7B measured a large gain where 1.5B measured ~none. |
-| `rerank.topM` | `8` | Fused results shown to the judge (8 measured better than 10). |
-
-### CLI flags
-
-Flags override the config file for a single run:
-
-| Flag | Equivalent |
-| :--- | :--- |
-| `--repo <dir>` | Index a directory other than the cwd |
-| `--use-sqlite` | `"storage": "sqlite"` |
-| `--llm-enrichment` | `"enrichment.enabled": true` |
-| `--enrich-model <name>` | `"enrichment.model"` |
-| `--enrich-max <n>` | `"enrichment.maxChunks"` for this run |
-| `--enrich-concurrency <n>` | `"enrichment.concurrency"` for this run |
-
-### Environment variables
-
-| Variable | Default | Description |
-| :--- | :--- | :--- |
-| `MCP_PROJECT_ROOT` | `process.cwd()` | Project root directory |
-| `OLLAMA_HOST` | `http://localhost:11434` | Ollama API endpoint |
-| `INDEXER_EMBEDDINGS` | — | Set to `off` to disable vector embeddings |
-
-Re-run `init` at any time to change languages, or `init --all-languages` to enable every installed parser without the prompt.
+The agent can then call `get_call_graph("rateLimiter")` to see what calls it (the blast radius) before changing it.
 
----
+## Guided setup
 
-## Supported languages
+`npx graph-indexer <path>` runs an interactive wizard that leaves the repo ready to use. It is **idempotent** — re-run it whenever you add a language or change a setting; it merges into what you already have and never clobbers another tool's config. Every generated file lands in `.graph-indexer/` (git-ignored), so your repo root stays clean.
 
-| Language | Extensions | Chunks extracted |
-| :--- | :--- | :--- |
-| TypeScript / TSX | `.ts`, `.tsx` | functions, classes, methods, exports |
-| JavaScript | `.js`, `.jsx`, `.mjs`, `.cjs` | functions, classes, expressions |
-| Python | `.py` | functions, classes |
-| Go | `.go` | functions, methods, types |
-| Rust | `.rs` | fn, struct, enum, trait, impl |
-| Java | `.java` | class, method, interface, constructor, enum |
-| Kotlin | `.kt`, `.kts` | function, class, object, companion |
-| C# | `.cs` | class, method, interface, property, enum |
-| Ruby | `.rb` | method, class, module |
-| PHP | `.php` | function, class |
-| CSS / SCSS | `.css`, `.scss` | rule sets |
+Before the steps, it scans the repo to pre-select your stack — languages from a bounded file walk, frameworks from your manifests (`package.json`, `pyproject.toml`, `composer.json`, `pom.xml`, `Gemfile`, `*.csproj`, …) — and, if it finds a pre-v1.4 layout, tidies those stray artifacts into `.graph-indexer/` first. Then six steps:
 
-Oversized "god classes" are split automatically: a class longer than ~200 lines is indexed as a compact skeleton header plus one independently searchable chunk per method, so a single `get_chunk` never blows the token budget and every method stays reachable by search.
+| Step | You choose | What happens |
+|------|------------|--------------|
+| **1 · Languages** | Languages to index | Detected languages are pre-checked — press Enter to accept. Selecting none indexes **all** supported languages. |
+| **2 · Frameworks** | Prompt add-ons | Filtered to your languages and pre-checked from detection. Sharpens the agent prompt for React, Express/NestJS, FastAPI/Django, Spring, Rails, Laravel/Symfony, ASP.NET, or Android. |
+| **3 · Search engine & LLM** | Retrieval engine | **Press Enter for the recommended default** — lexical search, no LLM, no network. Everything heavier is opt-in (see below). |
+| **4 · Agents & IDEs** | Your coding tools | Multi-select which agents to wire: Claude Code/Desktop, Cursor, VS Code/Copilot, Windsurf, Cline/Roo, JetBrains Junie, Codex (AGENTS.md), Gemini CLI. Pre-checked from your saved choice, else what's detected, else all. Drives steps 5 **and** 7 — deselect a tool and graph-indexer generates nothing for it. |
+| **5 · MCP server wiring** | *(automatic)* | Wires the MCP server for each **selected** editor (VS Code, Cursor, Claude Desktop, Claude Code). **Merge-safe**: your other MCP servers and keys are preserved. |
+| **6 · Project files & daemon** | *(automatic)* | Adds `mcp:*` npm scripts (index + daemon control), a managed `.gitignore` block, and `.graph-indexer/config.json` (which remembers your agent selection). |
+| **7 · Agent instructions** | *(automatic)* | Always writes the canonical layered prompt (`GRAPH_INDEXER_PROMPT.md`) + a `GRAPH_INDEXER_DOMAIN.md` template for your own rules. Then, for each **selected** agent: `@`-imports (no duplication) in `CLAUDE.md` and `GEMINI.md`; rule files for Cursor, Windsurf, and Cline/Roo; and managed blocks in `.github/copilot-instructions.md`, `.junie/guidelines.md`, and `AGENTS.md` (Codex/Zed/Jules) — preserving anything already there. |
 
----
-
-## How it works
+It finishes with a grouped summary (created / updated / kept / skipped / warnings), offers to **build the index now**, and prints your next steps.
 
-### Architecture
+**Most repos need nothing past step 3's default** — the lexical engine works in any language, installing that language's grammar on the first index build. Step 3 only branches when you decline the defaults:
 
-graph-indexer is organised around a small set of cohesive modules with a strict separation between *retrieval math*, *storage*, and *transport*:
+- **Storage** — `auto` (in-memory, promoting to SQLite past ~15k chunks), or force in-memory / SQLite.
+- **Embeddings** — `off` (default), `auto`, **Ollama**, **local** (in-process MiniLM, ~25 MB on first index), or **Apple Metal (MLX)** on macOS. Ollama is probed only when you actually pick it; choosing MLX offers to provision its Python venv on the spot.
+- **Enrichment / reranker** — off by default. A measured per-language note is shown before the reranker prompt (it helps Go/Python, regresses JS/TS). Model pickers are provider-aware, and if you select MLX the wizard verifies `mlx_lm.server` end-to-end — installing `mlx-lm`, starting the server, and pre-loading the model — so your first index can't fail against a server that isn't up.
 
-```
-config.mjs        Resolves CLI flags > env > .graph-indexer.json into one config.
-parser-utils.mjs  Tree-sitter parsing, chunk extraction, Ollama embeddings.
-search-core.mjs   Shared retrieval math: tokenisation, BM25, RRF fusion + boosts
-                  (query-adaptive NL weighting), PageRank, embedding cache keys.
-                  The numbers are measured once here and reused.
-core-engine.mjs   MemoryGraphIndex — the default in-heap store — plus the
-                  embeddings-bin codecs (write/read/append/scan) and the
-                  binary-quantized vector sketch (Hamming prefilter + rescore).
-sqlite-store.mjs  SqliteGraphStore — the disk-backed store (node:sqlite) with
-                  per-file incremental writes and data_version live refresh.
-storage.mjs       createStore(config) — picks a backend; documents the contract
-                  both implement (searchHybrid, getChunk, applyFileUpdate, …).
-enrichment.mjs    Optional LLM summaries + concept tags, cached by content hash.
-mcp-tools.mjs     The eight tools, written against the storage contract only.
-mcp-server.mjs    Thin bootstrap: config → store → tools → stdio.
-indexer.mjs       Bootstrap indexer.   watch-daemon.mjs  Incremental updates.
-```
+### Non-interactive & preview
 
-Because both stores call the identical `fuseAndRank` from `search-core`, the in-memory and SQLite backends are rank-consistent by construction, and a single change to the ranking math applies everywhere.
+| Flag | Effect |
+|------|--------|
+| `--yes`, `-y` | Accept all detected/default selections — no prompts (CI). |
+| `--dry-run` | Print every file action without writing anything. |
+| `--all-languages` | Index every supported language (implies `--yes`). |
+| `--help`, `-h` | Show usage. |
 
-### Indexing
+Setup also runs non-interactively whenever stdin isn't a TTY, so piping into it behaves like `--yes`.
 
-When you run `npm run mcp:index`:
+## How it works
 
-1. **Parses** every source file with Tree-sitter — no regex, exact AST boundaries.
-2. **Extracts chunks**: named functions, classes, methods, and exports — including Go/Rust `type`/`struct` declarations by their real name — each with docstring, parameters, return type, call sites, type references, **decorators/annotations** (`@Controller`, `@Injectable`, `@app.route`), and **inheritance edges** (`extends`/`implements`/base classes).
-3. **Builds a dependency graph**: bidirectional import map so each chunk knows what it imports and what imports it.
-4. **(Optional) Enriches** production-source chunks with LLM summaries + concept tags (cache-first; only new code pays an LLM call).
-5. **Creates two indexes**: a BM25 inverted index (lexical) and per-chunk float32 embeddings via Ollama `nomic-embed-text` (optional) — two vectors per enriched chunk (code payload + summary-only).
-6. **Persists** to the configured backend: `code-index.json` + `code-index.embeddings.bin`, or `code-index.db` + the same embeddings binary.
+- **AST indexing.** Tree-sitter parses each file into a syntax tree, and the indexer extracts one *chunk* per top-level definition (function, class, method, struct, …) with its name, parameters, line range, call sites, and enclosing class. A "god class" is split so its methods become their own chunks. Supported languages: **TypeScript/JavaScript** (`.ts .tsx .js .jsx .mjs .cjs`), **Python**, **Go**, **Rust**, **Java**, **Kotlin**, **C#**, **C**, **Ruby**, **PHP**, **Bash**, and **Swift**, plus **CSS/SCSS**.
+- **Retrieval.** A hybrid ranker fuses a lexical channel (BM25 with camelCase splitting and language-agnostic Porter stemming) with an optional dense-vector channel (local embeddings) via Reciprocal Rank Fusion. With embeddings off, only the lexical channel runs — and it is the default.
+- **Call graph.** `get_call_graph` returns the callers and callees of a symbol — the *blast radius* of a change — so an agent can see what it might break before editing code it never read.
+- **Backend parity.** The in-memory and SQLite backends share the same ranking core, so they return identical top-5 results for the same query (enforced by `test/sqlite.mjs`).
+
+### Language grammars
+
+Tree-sitter grammars aren't bundled with the package — a fresh `npm install graph-indexer` doesn't download all 14 languages' native bindings, just the ~5 MB core engine. Instead, the first `idx-index` run installs the grammar(s) for whichever languages you actually selected (or, un-narrowed, whatever's detected) into `.graph-indexer/node_modules`, and reuses that cache on every later run — no re-download, no re-build. This works the same way whether graph-indexer is a project dependency, a global install, or run ad hoc via `npx`, because the install target is always your repo's own `.graph-indexer/` directory, not graph-indexer's own location. A platform-incompatible grammar (e.g. a missing native toolchain) is skipped on its own — other languages still install and index normally, the same graceful degradation `npm`'s `optionalDependencies` used to give for free. Under `--sealed`, auto-install is disabled entirely: an unresolvable grammar fails the run with the exact `npm install --prefix .graph-indexer <pkg>@<version>` command to run first, rather than silently shipping a partial index.
+
+### MCP tools
+
+| Tool | Returns |
+|------|---------|
+| `search_code` | Ranked semantic chunks for a natural-language or symbol query. |
+| `get_chunk` | The full source of one chunk by id. |
+| `get_chunk_summary` | A compact summary of a chunk (signature, calls, context). |
+| `resolve_symbol` | Exact, case-insensitive symbol lookup by name. |
+| `get_file_skeleton` | The top-level structure (symbols + signatures) of a file. |
+| `get_call_graph` | Callers and callees of a symbol — the blast radius. |
+| `find_references` | Where a symbol is used: callers, subclasses, and type references. On a `--resolver scip` index, adds a precise cross-file **`resolved`-tier** dimension that disambiguates same-named symbols and recovers references the name heuristics miss. |
+| `find_routes` | HTTP routes mapped to their handler chunks (NestJS, FastAPI/Flask, Spring, Express/Koa). |
+| `get_subgraph` | A bounded connected subgraph around a seed symbol — its callees, high-confidence callers, and type/inheritance users, in one call. |
+| `get_repo_map` | A high-level map of the repository's modules and topology. With `--symbol-graph`, also lists the most-central symbols (PageRank). |
+| `list_index_stats` | Index health: chunk/file/symbol/vector counts and the active config. |
+| `tests_for` | The test/spec chunks that exercise a symbol (call or reference it) — which tests to run or update before changing it. |
+| `explain_symbol` | One-call overview of a symbol: signature, callees, callers (blast radius), subclasses/type users, routes it handles, tests, git recency/co-change, and (with `--symbol-graph`) its symbol-centrality rank. |
+| `impact_of_edit` | The precise blast radius of a change: pass the symbols/files you're about to edit → transitively-affected code, the routes that reach it, the tests to run, and git co-change. Most precise with `--symbol-graph`; `precision: 'strict'` follows only provably-unambiguous (`resolved`) edges (best on a `--resolver precise` or `--resolver scip` index). |
+| `trace_taint` | Security: trace untrusted data from a source (request body/query/params, argv/env, stdin) to a dangerous sink (eval/exec, SQL, fs/path, HTML, outbound request) across the call graph — injection-class risks (rce/sqli/xss/path/ssrf) with the source→sink path. JS/TS, Python, Java, Go; heuristic finder, not a verifier. |
+| `find_tainted_sinks` | Security orientation: every dangerous sink grouped by category, each flagged with whether an untrusted source reaches it — map the attack surface, then `trace_taint` a specific one. |
 
-A background watcher daemon applies changed files incrementally to **either** backend (JSON
-snapshot rewrites for in-memory, per-file WAL transactions for SQLite). It respects
-`.gitignore` and skips `node_modules`, build output, and dot-directories, so it never traverses
-(or exhausts OS file-watcher limits on) dependency trees.
+## Configuration
 
-### Search pipeline
+Everything beyond the lexical default is opt-in. The server, indexer, and daemon all print their **effective configuration** at startup (storage backend, model names, which optional features are on), and emit a visible warning whenever an opt-in feature has a known trade-off — nothing is enabled silently.
+
+### Headline trade-offs
+
+| Option | Default | When to enable | Cost |
+|--------|---------|----------------|------|
+| `--embeddings` | off | Larger repos where recall matters; lifts success@5 | Requires Ollama or the in-process MiniLM model; slower indexing |
+| `--embed-model qwen3-embedding:4b` | `nomic-embed-text` | A code-specialized embedder — lifts agent-style recall on Go (and some code-heavy repos); measure first | slower indexing; requires Ollama; neutral on JS/Python in our tests |
+| `--enrichment` | off | Only useful paired with `--rerank`; alone it regresses | slowest indexing |
+| `--rerank` | off | Go/Python repos with weak semantic recall; regresses JS repos | Requires an Ollama 7B model; adds query latency |
+| `--rerank --rerank-provider cross-encoder` | `generative` | Want a reranker's semantic lift with **no LLM/daemon** — air-gapped, deterministic, fast | Optional `@huggingface/transformers`; smaller lift than the 7B judge |
+| `--use-sqlite` | `auto` | Repos past ~15k chunks or memory-constrained environments | Slightly higher query latency; needs Node 22+ |
+
+For most repos, the default (lexical + stemming, no embeddings) is the right starting point. Enable embeddings when you notice the agent missing chunks it should find. Enable the reranker only on Go or Python repos after measuring whether it helps — it is known to regress JavaScript repositories.
+
+**Reranker providers.** The reranker reorders the over-fetched candidate pool on natural-language queries; three providers are available (`--rerank-provider`):
+
+- **`generative`** (default) — a local LLM judge (Ollama 7B). The strongest reranker in our benchmarks (agent-style semantic rank-1 0.19 → 0.42 across the core suites; no symbolic regression), but it needs a running model and its output is non-deterministic.
+- **`cross-encoder`** — a local MS-MARCO cross-encoder (`Xenova/ms-marco-MiniLM-L-6-v2`, via the optional `@huggingface/transformers`). It scores each (query, candidate) pair and sorts. **Air-gapped, deterministic, and fast** (no Ollama/daemon; the model downloads once). It captures part of the lift — agent-style semantic rank-1 0.19 → 0.26, and notably Go (gin) 0.20 → 0.40 and Python (fastapi) 0.14 → 0.29 — but less than the 7B judge, and like all rerankers it is mixed on JavaScript. Pick it when you want a reranker without an LLM in the loop. Measure on your repo: `npm run test:eval -- --rerank --rerank-provider cross-encoder`.
+- **`late-interaction`** (B4) — a **ColBERT-style multi-vector MaxSim** reranker. It encodes the query into several vectors (the whole query + content sub-units, via the same local embedder) and scores `Σ_qi max_dj cos(qi, dj)` against each candidate's stored document sub-vectors — **no storage blow-up** (it reuses the base/summary/window vectors `--embeddings` already writes), air-gapped, deterministic, parity-safe. **Requires `--embeddings`** (a no-op without it). Measured at production retrieval depth: **helps Go** (gin semantic rank-1 0.00 → 0.40, overall 0.68 → 0.77, no symbolic regression) and **neutral on JavaScript** (express unchanged) — the "helps Go/Python" half of the reranker profile without the JS regression. Approximates true token-level ColBERT with a sentence embedder. Measure on your repo: `npm run test:eval -- --embeddings --rerank --rerank-provider late-interaction`.
+
+### All CLI flags
+
+| Flag | Default | Effect |
+|------|---------|--------|
+| `--repo <path>` | current directory | Repository to index / serve. |
+| `--embeddings` | off | Enable the dense-vector channel. |
+| `--embed-model <model>` | `nomic-embed-text` | Ollama embedding model (e.g. `qwen3-embedding:4b`). |
+| `--embed-provider <auto\|ollama\|local\|mlx\|off>` | `auto` | Force the embedding backend (see [Embedder backends](#embedder-backends)). |
+| `--mlx-embed-model <model>` | `mlx-community/all-MiniLM-L6-v2-4bit` | Model loaded by the MLX (Apple Metal) embedder. |
+| `--use-sqlite` | `auto` | Force the disk-backed SQLite backend. |
+| `--enrichment` | off | Enable LLM enrichment of central chunks. |
+| `--enrich-model <model>` | `qwen2.5-coder:1.5b` | Model used for enrichment. |
+| `--enrich-max <n>` | 500 | Cap on new LLM calls per index run. |
+| `--enrich-concurrency <n>` | 4 | Parallel Ollama requests during enrichment. |
+| `--rerank` | off | Enable the reranker (reorders the over-fetched pool on NL queries). |
+| `--rerank-provider <generative\|cross-encoder\|late-interaction>` | `generative` | `generative` = local LLM judge; `cross-encoder` = local air-gapped MS-MARCO cross-encoder (optional `@huggingface/transformers`, no Ollama); `late-interaction` = ColBERT-style multi-vector MaxSim over the stored doc vectors (needs `--embeddings`; air-gapped, deterministic, no storage blow-up; measured helps-Go / neutral-JS at production depth). |
+| `--interprocedural` | off | Index-time inter-procedural receiver-type fixpoint: propagate return types along factory call chains so multi-hop receivers resolve in `get_call_graph` / `find_references`. Air-gapped, deterministic; default index byte-identical; search ranking unaffected. |
+| `--symbol-graph` | off | Persist a resolved chunk→chunk symbol graph (edges with kind + confidence) at index time, exposed via `getEdges`. `findCallers`/`findReferers` read it (identical sets, fall back to a scan). Also computes symbol-level centrality (confidence-weighted PageRank over the edges), surfaced by `explain_symbol` and `get_repo_map`. Air-gapped, deterministic; default index byte-identical; search ranking unaffected. |
+| `--taint` | off | Precompute and serialize the taint flow set (C2) at index time, so `trace_taint` / `find_tainted_sinks` answer instantly instead of walking the call graph per query. Computed against the symbol-graph edges when `--symbol-graph` is on (cross-function precision), else `chunk.calls`. Both backends serve byte-identical flows; default index byte-identical; air-gapped. |
+| `--ranker <rrf\|learned>` | `rrf` | Search ranker (D3). `rrf` (default) = the RRF + boost-ladder fusion. `learned` = an opt-in, zero-dependency post-fusion linear re-rank over the RRF score + symbol centrality + resolved/SCIP in-degree + git (best paired with `--symbol-graph`). **Measured ≈neutral on the benchmark** — RRF is the default; the learned ranker is provided for users to retrain on their own repo (`bench/train-ranker.mjs`). Default path byte-identical. |
+| `--learned-sparse` | off | Learn a corpus co-occurrence (PMI) association model (B3) at index time and serialize it, adding a **learned-sparse vocabulary-expansion** channel: NL queries expand into a weighted term set (literal terms + learned associates) fused via RRF, to close lexical-gap misses stemming can't. Zero new dependency, air-gapped, NL-asymmetric (symbolic lookups byte-identical), parity-safe. **Measured net-neutral on the strict benchmark** (does not beat lexical — like BM25F/AST); shipped opt-in for repos with genuine query/code vocabulary mismatch. Default index byte-identical. |
+| `--resolver <heuristic\|precise\|scip>` | `heuristic` | Resolver provider for the symbol graph. `precise` lifts provably-unambiguous edges (sole definition, or a type-pinned receiver) from `high` to a `resolved` tier. `scip` ingests a locally-generated SCIP index (`--scip-index`) and resolves edges by *real cross-file bindings* — promoting confirmed edges to `resolved` **and suppressing the wrong-target fan-out** an ambiguous same-name reference would otherwise emit, **and** adding a precise cross-file **`resolved`-tier** dimension to `find_references` (disambiguates same-named symbols, recovers references the name heuristics miss). Both power `impact_of_edit(precision: 'strict')`. Only meaningful with `--symbol-graph`. |
+| `--scip-index <path>` | — | Path to a SCIP index generated locally by your own toolchain (`scip-typescript`, `scip-python`, `scip-java`, `rust-analyzer --scip`), consumed by `--resolver scip`. Air-gapped (a local file read — sealed-compatible). Files the index doesn't cover fall back to heuristic confidence (never worse); coverage is printed at index time. Accepts the `.scip` protobuf or a SCIP-shaped JSON. |
+| `--sealed [strict\|local]` | off | **Sealed mode** — air-gapped *enforced and verifiable*. `strict` = zero network egress (lexical-only); `local` = loopback only (local LLM/embedders OK, nothing off-box). Fail-closed: refuses to start if any enabled feature would egress beyond the tier, and installs a deny-by-default runtime egress guard. Bare `--sealed` = `strict`. |
+| `--attest` | — | Print the deterministic sealed-mode manifest (tier, providers, egress posture) and exit — an auditor/CI artifact. Add `--sign-key <path>` to print a **cryptographically signed** envelope instead (Ed25519/RSA/EC via `node:crypto`; air-gapped). |
+| `--gen-attestation-key <prefix>` | — | Write an Ed25519 attestation keypair (`<prefix>.key` private, `<prefix>.pub` public) and exit. |
+| `--verify-attestation <file> --pub-key <path>` | — | Verify a signed attestation's signature and report whether it still matches the current effective config (policy drift → non-zero exit). |
+| `--no-git-signals` | (signals on) | Skip collecting local git churn/recency/co-change. |
+| `--git-rank-boost <0..1>` | 0 | Opt-in weight for git recency/churn in ranking (0 = ranking unchanged). |
+| `--llm-provider <ollama\|mlx>` | `ollama` | LLM backend for enrichment, reranking, and HyDE. `mlx` routes calls to a local `mlx_lm.server`. |
+| `--mlx-lm-host <url>` | `http://localhost:8080` | Endpoint for the `mlx_lm.server` when `--llm-provider mlx` is set. |
+
+### All environment variables
+
+| Variable | Default | Effect |
+|----------|---------|--------|
+| `MCP_PROJECT_ROOT` | current directory | Repository root when `--repo` is not given. |
+| `OLLAMA_HOST` | `http://localhost:11434` | Ollama endpoint for embeddings/enrichment/rerank. |
+| `INDEXER_EMBEDDINGS` | `off` | `on` enables embeddings; `off` always wins over `--embeddings`. |
+| `EMBED_MODEL` | `nomic-embed-text` | Ollama embedding model (overrides config; overridden by `--embed-model`). |
+| `INDEXER_EMBED_PROVIDER` | `auto` | `auto` \| `ollama` \| `local` \| `mlx` \| `off`. |
+| `INDEXER_MLX_EMBED_MODEL` | `mlx-community/all-MiniLM-L6-v2-4bit` | Model loaded by the MLX embedder (overridden by `--mlx-embed-model`). |
+| `INDEXER_MLX_BATCH_SIZE` | 32 | Texts per batch sent to the MLX subprocess (raise on large unified memory). |
+| `INDEXER_STORAGE` | `auto` | `auto` \| `memory` \| `sqlite`. |
+| `ENRICH_MODEL` | (unset) | Naming a model enables enrichment and selects it. |
+| `RERANK_MODEL` | (unset) | Naming a model enables the (generative) reranker and selects it. |
+| `RERANK_PROVIDER` | `generative` | `generative` (LLM judge), `cross-encoder` (local air-gapped), or `late-interaction` (ColBERT-style MaxSim, needs embeddings); setting it enables reranking. |
+| `RERANK_CROSS_ENCODER_MODEL` | `Xenova/ms-marco-MiniLM-L-6-v2` | Cross-encoder model id (downloaded on first use). |
+| `INDEXER_GIT_SIGNALS` | (on) | Set to `off` to skip git-signal collection. |
+| `INDEXER_INTERPROCEDURAL` | (off) | `on` enables the index-time inter-procedural receiver-type fixpoint (`--interprocedural`). |
+| `INDEXER_SYMBOL_GRAPH` | (off) | `on` builds the persistent resolved symbol graph (`--symbol-graph`). |
+| `INDEXER_TAINT` | (off) | `on` precomputes + serializes the taint flow set at index time (`--taint`). |
+| `INDEXER_RANKER` | `rrf` | `learned` enables the opt-in post-fusion learned re-rank (`--ranker`). |
+| `INDEXER_LEARNED_SPARSE` | (off) | `on` learns + serializes the corpus association model for the learned-sparse expansion channel (`--learned-sparse`). |
+| `INDEXER_RESOLVER` | `heuristic` | `precise` enables the precise resolver; `scip` enables the SCIP cross-file resolver (`resolved` edge tier) for the symbol graph (`--resolver`). |
+| `INDEXER_SCIP_INDEX` | (unset) | Path to a locally-generated SCIP index for `--resolver scip` (`--scip-index`). |
+| `INDEXER_SEALED` | (off) | `strict` (or `on`) / `local` enables sealed mode (`--sealed`): fail-closed validation + a runtime egress guard. |
+| `INDEXER_GIT_RANK_BOOST` | 0 | Opt-in git recency/churn ranking weight (0..1). |
+| `INDEXER_LLM_PROVIDER` | `ollama` | LLM backend for enrichment, reranking, and HyDE: `ollama` or `mlx`. |
+| `INDEXER_MLX_LM_HOST` | `http://localhost:8080` | Endpoint for the `mlx_lm.server` when `INDEXER_LLM_PROVIDER=mlx`. |
+| `INDEXER_EMBED_CONCURRENCY` | 4 | Parallel embedding batches; lower to 1 for large models on modest hardware. |
+| `INDEXER_EMBED_TIMEOUT_MS` | 120000 | Per-batch embedding timeout; raise for very large models. |
+
+When embeddings are enabled, the provider is selected in this order, and every fallback is logged (never silent): Ollama with `EMBED_MODEL` if set and reachable → Ollama with `nomic-embed-text` → the in-process MiniLM model (optional `@huggingface/transformers`) → lexical-only with a warning.
+
+### Embedder backends
+
+Set the backend with `--embed-provider` (or `INDEXER_EMBED_PROVIDER`). `mlx` is a native Python embedder that runs the Apple Metal GPU as a reused subprocess — faster in-process vectors than the bundled Xenova path, with no running Ollama daemon. The in-process (`local`) and `mlx` backends produce all-MiniLM-L6-v2-family **384-dim** vectors; the default `auto`→Ollama path uses `nomic-embed-text` (768-dim).
+
+| Backend | `--embed-provider` | Platform | Throughput¹ | Setup |
+|---------|--------------------|----------|-------------|-------|
+| Auto (Ollama → local → off) | `auto` (default) | any | varies | none |
+| In-process (Xenova) | `local` | any | ~18 ch/s | `npm i @huggingface/transformers` |
+| **Apple Metal (MLX)** | **`mlx`** | **macOS Apple Silicon** | **~42 ch/s** | `npm run embed:setup:mlx` |
+| Ollama daemon | `ollama` | any | ~14 ch/s² | `ollama serve` + `ollama pull` |
+
+¹ **Throughput is hardware-dependent.** These figures are the median of 3 cold builds on an **Apple M2 Mac mini (24 GB)** indexing the `express-js` fixture (389 chunks): `node bench/cell.mjs express-js <E0|E0_MLX|O0>`. Expect different numbers on other chips; reproduce on your own machine. Throughput also varies with system load, model warm state, and corpus size — larger repos amortize cold-start better (e.g. Ollama/nomic reaches ~32 ch/s on the larger `gin` corpus).
+
+² On Apple Silicon, Ollama already uses the Metal GPU internally (via llama.cpp). The `mlx` provider's advantage comes from a smaller model (all-MiniLM-L6-v2-4bit, 22M params, 384-dim) and no HTTP round-trip, not from GPU vs CPU.
+
+The index records which provider/model built it (in the `code-index.embeddings.bin.meta.json` sidecar) and queries with the same one, so vectors never get mixed across models. Switching providers or the MLX model between builds is detected and triggers a clean re-embed.
+
+**Code-specialized embedding models.** The default `nomic-embed-text` is a general-purpose embedder. Models trained on code — `qwen3-embedding`, `nomic-embed-code`, `jina-embeddings-v2-base-code` — embed code semantics more faithfully and can lift agent-style (natural-language) recall. Select one with `--embed-model <name>` (Ollama); it is stamped into the index meta and queried with the same model so vectors never get mixed. **Measure before adopting** — in our benchmarks `qwen3-embedding` lifted Go (gin) semantic rank-1 0.20 → 0.40 but was neutral on Python (django) and JavaScript (express), and it indexes substantially slower than `nomic-embed-text`. Run `npm run test:eval -- --embeddings` on your own repo to confirm the trade-off. We also evaluated *re-weighting the vector channel upward* for these stronger embedders and found **no measurable recall@5 gain** (it regressed rank-1/symbolic at higher weights) — the existing default weighting already extracts the benefit, so there is no separate "code weight" knob. `jina-embeddings-v2-base-code` remains opt-in only: it helped some fixtures, regressed others, and indexes far slower, so it is never a default.
+
+**macOS Apple Silicon (recommended):**
 
+```bash
+npm run embed:setup:mlx                                  # one-time: creates embedders/venv-mlx + installs MLX
+npx idx-index --repo . --embeddings --embed-provider mlx
 ```
-Source files
-     │
-     ▼
-Tree-sitter AST
-     │
-     ├──► Chunks ──► BM25 inverted index ──┐
-     │         └──► Float32 embeddings  ──┤── RRF fusion ──► Ranked results ──► MCP tools
-     │                                     │   (search-core)
-     └──► Dependency graph ────────────────┘
-```
-
-1. **BM25 lexical search** — O(query_terms) over the inverted index / `postings` table.
-2. **Vector search** — exact cosine below 10k vectors; binary-sketch Hamming prefilter +
-   exact rescore above it (5–11 ms at 50k+, see [Storage backends](#storage-backends)). Both
-   code and summary vectors compete; hits fold onto their chunk through one shared finalizer.
-3. **RRF fusion** merges both lists — lexical-led (1.5×/1.0×) for keyword queries,
-   vector-led (1.0×/1.6×) for natural-language ones.
-4. **Score boosts**: exact name match +2.0× (df-gated, plural-aware), suffix match +1.4×,
-   path token +1.4×, test/example files demoted; on NL queries, boosts require semantic
-   agreement (the chunk must also be a vector candidate).
-5. **(Opt-in) LLM rerank** of the fused top-8 on natural-language queries.
-
----
-
-## Best practices
 
-**Write descriptive docstrings.** Docstrings are embedded alongside code — better documentation directly improves semantic search quality.
+`embed:setup:mlx` provisions a **dedicated Python virtualenv** under `embedders/venv-mlx` (so MLX's deps never touch your system Python) and is idempotent — re-running it is a no-op once ready. The interactive setup (`npx graph-indexer`) offers MLX directly and can run this step for you. The first index downloads the model (~90 MB) into the Hugging Face cache; later builds reuse it.
 
-**Use specific function names.** Names like `fetchAndCacheUserProfile` trigger the exact-name boost (2.0×) and reliably appear as rank-1 results when queried by name.
+Choose a different MLX model with `--mlx-embed-model <id>` (or `INDEXER_MLX_EMBED_MODEL`); it must be an `mlx_embeddings`-compatible sentence model, and the default `mlx-community/all-MiniLM-L6-v2-4bit` is the proven option. Tune batch size with `INDEXER_MLX_BATCH_SIZE` (default 32; lower to 16 on 8 GB machines, raise to 64 on M-series Max/Ultra). `mlx` is macOS-only; requesting it elsewhere fails fast with the alternative to use.
 
-**Export named functions, not anonymous ones.** Anonymous default exports can't be targeted by `resolve_symbol` and rank lower in name-boost scoring.
+## Benchmarks
 
-**Avoid catch-all utility files.** Files that mix unrelated utilities produce weaker per-chunk signals. One responsibility per module indexes more cleanly.
+Every number here is produced by the eval harness on cold, isolated builds — never hand-edited — with **strict scoring** (exact symbol match, no file-path fallback) against a **held-out** split (~20–25% per language, never used to tune). Reproduce it all with `npm run test:eval`.
 
-**Reach for the SQLite backend when the heap gets tight.** If indexing a large monorepo pushes Node toward its memory limit, switch to `--use-sqlite`; retrieval quality is identical and resident memory stops scaling with the codebase.
+What the measurements say, across 18 real-world fixtures spanning every supported language:
 
----
+- **The zero-dependency lexical default is the right starting point.** It wins the held-out metric outright on 2 fixtures, and 6 more need only a cheap in-process embedder — so **8 of 18 run with no Ollama and no network**. Most repos never need more.
+- **Symbolic lookups are strong** (mean rank-1 ≈ 0.70). Behavioural, natural-language queries are harder (≈ 0.30 overall (held-out semantic rank-1 0.40 with stemming)) — that's where the optional embedding and reranker channels earn their keep.
+- **Optional features are measured, not assumed.** Dense embeddings lift recall on larger repos; the LLM reranker is language- and repo-dependent (it lifts 8 fixtures but taxes JavaScript and Spring); enrichment helps only where proven (just rust). Setup surfaces each trade-off before you enable it.
+- **Backend parity:** the in-memory and SQLite backends return byte-identical top-5 on all 18 fixtures (gated by `test/sqlite.mjs`).
 
-## Troubleshooting
+Per-fixture best configs, 3× spreads, and copy-paste enable flags live in **[docs/benchmarks/BENCH_PER_FIXTURE.md](docs/benchmarks/BENCH_PER_FIXTURE.md)**.
 
-**Index takes too long**
-- Verify Ollama is running: `curl http://localhost:11434/api/tags`
-- Use lexical-only mode: `INDEXER_EMBEDDINGS=off npm run mcp:index`
+### Structural coverage by language
 
-**Indexing a huge repo runs out of memory**
-- Build with `--use-sqlite` so chunks live on disk instead of the heap
-- Confirm the active backend with `list_index_stats()`
+`get_call_graph` and `find_references` are richest for typed languages. Every verdict is confirmed by invocation on the real index, not by reading field counts:
 
-**`--use-sqlite` reports node:sqlite is unavailable**
-- The SQLite backend needs Node v22.5+; upgrade Node, or use the default in-memory backend
+| Capability | Strong | Limited |
+|---|---|---|
+| **Call graph** (callers/callees) | resolves on every supported language | Java/Spring is class-granular; SCSS effectively none |
+| **Caller precision** | receiver-aware: TS/JS, Python, C#, Swift, PHP, Java (call-site level — but its call graph is class-granular) | name-only: Go, Rust, Kotlin, Ruby, Bash, C |
+| **Typed `find_references`** | precise: TS/JS, Python · field-precise: C# | heuristic: Java/PHP/Kotlin/Swift/Rust/Go/C · empty: Ruby, Bash, SCSS, dynamic JS/Python |
 
-**Search returns irrelevant results**
-- Use `exact_tokens` for known symbol names
-- For conceptual queries, index with `--llm-enrichment` and embeddings enabled
-- Check coverage with `list_index_stats()`; increase `top_k` to see more candidates
+AST chunking and lexical search cover **every** supported language; only the typed cross-reference channel narrows for dynamic ones.
 
-**MCP server won't connect**
-- Verify `MCP_PROJECT_ROOT` points to the indexed directory
-- Check the index exists: `ls -la code-index.json` (or `code-index.db`)
-- Test manually: `npm run mcp:start`
+**Multi-hop receivers (`--interprocedural`).** Caller precision relies on resolving a call's receiver type. Intra-procedurally, graph-indexer already resolves `const r = makeRepo(); r.save()` one hop (reading `makeRepo`'s recorded return type). The opt-in `--interprocedural` flag adds an index-time fixpoint that propagates return types *along factory call chains*, so a receiver whose type comes from a multi-hop or unannotated factory (`makeRepoIndirect()` → `makeRepo()` → `new OrderRepo()`) still resolves to its concrete class — promoting those callers from *name-only* to *high-confidence* in `get_call_graph` / `find_references`. It is air-gapped and deterministic, the result is serialized into the index (both backends identical), and it leaves search ranking and the default index byte-identical. Note: the watch daemon updates one file at a time and does not re-run the whole-program pass, so cross-file chains refresh on the next full `idx-index` (degrading to single-hop until then, never worse).
 
-**Results are stale after file changes**
-- The daemon updates both backends live and running servers refresh automatically; check `list_index_stats()` for daemon status
-- If the daemon isn't running, restart the MCP server (it spawns the daemon) or run `npm run mcp:index` manually
+### Where it's weakest (honestly)
 
----
+- Behavioural, natural-language recall on the default path (rank-1 ≈ 0.30 overall; held-out semantic rank-1 0.40 — the embedding/reranker channel closes it further) — closing it needs the embedding/reranker channel, not lexical tuning.
+- The hardest fixtures stay below 0.65 held-out success@5 even with the full stack (rails 0.50, rust 0.61).
+- Java/Spring `get_call_graph` reports at class granularity (the god-class split only fires ≥200 lines); SCSS has no meaningful call graph.
+- LLM enrichment without reranking regresses precision — enable it only paired with `--rerank`.
 
-## Development
+## Reproducing the benchmarks
 
 ```bash
-git clone https://github.com/MaquinaTech/graph-indexer.git
-cd graph-indexer
-npm install
-npm run mcp:index   # index this repo
-npm run mcp:start   # start MCP server
+npm run test:all                   # full unit + integration suite
+npm run test:setup                 # index the benchmark fixtures
+npm run test:eval                  # default lexical path — prints the HELD-OUT block
+npm run test:eval -- --suite css   # any single fixture (18 authored suites)
+npm run test:eval -- --embeddings  # hybrid eval (requires Ollama)
+npm run test:eval -- --verbose     # per-query breakdown incl. file-rank column
+node test/sqlite.mjs               # backend-parity gate (memory ↔ SQLite top-5)
+node bench/cell.mjs <fixture> L1   # cold rebuild + score one fixture
 ```
 
-### Tests
-
-| Command | Scope |
-| :--- | :--- |
-| `npm run test` | Integration suite: indexing + loose hit-rate + token savings across 5 fixtures |
-| `npm run test:eval` | Strict symbol-level accuracy (rank-1, precision, nDCG) |
-| `npm run test:unit` | Pure helpers, chunk splitting |
-| `npm run test:sqlite` | SQLite round-trip + rank consistency vs the in-memory engine |
-| `npm run test:enrich` | HyDE enrichment, including the lexically-disjoint rank-1 flip |
-| `npm run test:mcp` | End-to-end MCP server over stdio |
-| `npm run test:scale` | Mock 50k-chunk corpus proving SQLite RAM stays bounded |
-| `npm run test:all` | Every dependency-free suite above, no Ollama required |
+The harness is [test/evaluate.mjs](test/evaluate.mjs); deeper reproduction notes live in [docs/benchmarks/BENCH_BASELINE.md](docs/benchmarks/BENCH_BASELINE.md) and [docs/benchmarks/BENCH_FULL_SUITE.md](docs/benchmarks/BENCH_FULL_SUITE.md).
 
 ---
 
-## Security
-
-graph-indexer runs locally and is air-gapped by default — its only outbound calls
-are to a local Ollama endpoint for embeddings and optional enrichment (skipped entirely with
-`INDEXER_EMBEDDINGS=off` and without `--llm-enrichment`). It never executes the code it indexes,
-and the index artifacts contain source snippets, so keep them git-ignored (as `init` configures).
-
-See [SECURITY.md](SECURITY.md) for the full threat model and how to report a
-vulnerability.
-
----
-
-## License
-
-Released under the [MIT License](LICENSE). Copyright (c) 2026 MaquinaTech.
-
----
-
-Built by [MaquinaTech](https://github.com/MaquinaTech) · [Issues](https://github.com/MaquinaTech/graph-indexer/issues) · [npm](https://www.npmjs.com/package/graph-indexer)
+<div align="center">
+  <sub>MIT licensed · runs entirely on your machine · see <a href="SECURITY.md">SECURITY.md</a> and <a href="TRADEMARK.md">TRADEMARK.md</a></sub>
+</div>