opencode-codebase-index 0.20.0 → 0.21.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,1329 +1,235 @@
1
- # opencode-codebase-index
1
+ # open-codebase-index
2
2
 
3
- [![npm version](https://img.shields.io/npm/v/opencode-codebase-index.svg)](https://www.npmjs.com/package/opencode-codebase-index)
3
+ [![npm version](https://img.shields.io/npm/v/open-codebase-index.svg)](https://www.npmjs.com/package/open-codebase-index)
4
4
  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
5
- [![Downloads](https://img.shields.io/npm/dm/opencode-codebase-index.svg)](https://www.npmjs.com/package/opencode-codebase-index)
5
+ [![Downloads](https://img.shields.io/npm/dm/open-codebase-index.svg)](https://www.npmjs.com/package/open-codebase-index)
6
6
  [![Build Status](https://img.shields.io/github/actions/workflow/status/Helweg/opencode-codebase-index/ci.yml?branch=main)](https://github.com/Helweg/opencode-codebase-index/actions)
7
7
  [![Node.js](https://img.shields.io/badge/node-%3E%3D20-brightgreen.svg)](https://nodejs.org/)
8
8
 
9
- > **Stop grepping for concepts. Start searching for meaning.**
9
+ > Search a codebase by meaning, then follow the result into definitions, callers, and dependency paths.
10
10
 
11
- **opencode-codebase-index** brings semantic understanding to your [OpenCode](https://opencode.ai), [Jcode](https://github.com/1jehuang/jcode), [Pi](https://pi.dev), Codex, and MCP-compatible workflows. Instead of guessing function names or grepping for keywords, ask your codebase questions in plain English.
11
+ `open-codebase-index` is a local semantic code index for OpenCode, Jcode, Pi, Codex, Claude Code, and other MCP clients. It combines embeddings, BM25 keyword search, branch-aware filtering, symbol lookup, and a call graph behind agent-friendly tools.
12
12
 
13
- ## šŸ“Œ Quick Navigation
13
+ New installs should use `open-codebase-index` and `open-codebase-index-mcp`. The legacy package `opencode-codebase-index` and `opencode-codebase-index-mcp` remain supported aliases.
14
14
 
15
- - [⚔ Quick Start](#-quick-start)
16
- - [🧠 Jcode](#-jcode)
17
- - [🄧 Pi Package](#-pi-package)
18
- - [🧩 Codex Plugin](#-codex-plugin)
19
- - [🧩 Claude Code Plugin](#-claude-code-plugin)
20
- - [🌐 MCP Server (Cursor, Claude Code, Windsurf, etc.)](#-mcp-server-cursor-claude-code-windsurf-etc)
21
- - [šŸŽÆ When to Use What](#-when-to-use-what)
22
- - [🧭 OMO CodeGraph Compatibility](#-omo-codegraph-compatibility)
23
- - [🧰 Tools Available](#-tools-available)
24
- - [šŸŽ® Slash Commands](#-slash-commands)
25
- - [šŸ“š Knowledge Base](#-knowledge-base)
26
- - [šŸ”„ Reranking](#-reranking)
27
- - [āš™ļø Configuration](#ļø-configuration)
28
- - [šŸ¤ Contributing](#-contributing)
15
+ ## Highlights
29
16
 
30
- ## šŸ‘‹ Choose Your Path
17
+ - **Semantic and hybrid retrieval** for questions where you do not know the identifier.
18
+ - **Low-token discovery** through `codebase_context` and `codebase_peek`.
19
+ - **Definition and graph navigation** through `implementation_lookup`, `call_graph`, and `call_graph_path`.
20
+ - **Incremental, branch-aware indexing** with file watching and content-hash reuse.
21
+ - **Local storage** backed by SQLite, usearch vectors, and a BM25 inverted index.
22
+ - **Multiple embedding providers**: Ollama, GitHub Copilot, OpenAI, Google, or a custom OpenAI-compatible endpoint.
23
+ - **Native parsing** for TypeScript/TSX, JavaScript/JSX, Python, Rust, Swift, Go, Java, C#, Ruby, C/C++, Metal, PHP, Apex, Bash, Zig, GDScript, MATLAB, JSON, TOML, YAML, Markdown, and HTML, with text fallback.
31
24
 
32
- - **I want to try it now** → go to [Quick Start](#-quick-start)
33
- - **I use Jcode** → go to [Jcode](#-jcode)
34
- - **I use Pi** → go to [Pi Package](#-pi-package)
35
- - **I use Cursor/Claude Code/Windsurf** → go to [MCP Server setup](#-mcp-server-cursor-claude-code-windsurf-etc)
36
- - **I’m comparing tools and workflows** → go to [When to Use What](#-when-to-use-what)
37
- - **I’m tuning behavior/cost/performance** → go to [Configuration](#ļø-configuration)
38
- - **I want to contribute** → go to [Contributing](#-contributing)
39
-
40
- ## šŸš€ Why Use This?
41
-
42
- - 🧠 **Semantic Search**: Finds "user authentication" logic even if the function is named `check_creds`.
43
- - ⚔ **Blazing Fast Indexing**: Powered by a Rust native module using `tree-sitter` and `usearch`. Incremental updates take milliseconds.
44
- - 🌿 **Branch-Aware**: Seamlessly handles git branch switches — reuses embeddings, filters stale results.
45
- - šŸ”’ **Privacy Focused**: Your vector index is stored locally in your project.
46
- - šŸ”Œ **Model Agnostic**: Works out-of-the-box with GitHub Copilot, OpenAI, Gemini, or local Ollama models.
47
- - 🧠 **Jcode Host**: One global MCP configuration follows each Jcode session into its active repository.
48
- - 🄧 **Pi Package**: First-class Pi extension and skill package with native tools.
49
- - 🌐 **MCP Server**: Use with Cursor, Claude Code, Windsurf, or any MCP-compatible client — index once, search from anywhere.
50
-
51
- ## ⚔ Quick Start
25
+ ## Quick start with OpenCode
52
26
 
53
27
  Requires Node.js 20 or newer.
54
28
 
55
- 1. **Install the plugin**
56
- ```bash
57
- npm install opencode-codebase-index
58
- ```
59
-
60
- 2. **Add to `opencode.json`**
61
- ```json
62
- {
63
- "plugin": ["opencode-codebase-index"]
64
- }
65
- ```
66
-
67
- 3. **Index your codebase**
68
- Run `/index` or ask the agent to index your codebase. This only needs to be done once — subsequent updates are incremental.
69
-
70
- **Recommended check:** run `/status` after the first index so you can confirm the detected provider/model before you start searching.
71
-
72
- 4. **Start Searching**
73
- Ask:
74
- > "Find the function that handles credit card validation errors"
75
-
76
- ## 🧠 Jcode
77
-
78
- Jcode v0.56.0 and newer starts non-shared MCP servers in each session's working directory. Configure the server once in `~/.jcode/mcp.json` and it will index the repository where each Jcode session is running.
29
+ 1. Install the package:
79
30
 
80
- ```json
81
- {
82
- "servers": {
83
- "codebase-index": {
84
- "command": "npx",
85
- "args": [
86
- "-y",
87
- "--package",
88
- "opencode-codebase-index@latest",
89
- "opencode-codebase-index-mcp",
90
- "--host",
91
- "jcode"
92
- ],
93
- "env": {},
94
- "shared": false
95
- }
96
- }
97
- }
98
- ```
99
-
100
- Do not add a fixed `--project` argument. Jcode supplies the active session directory as the MCP process working directory. `shared: false` gives every repository session its own indexer process and prevents cross-repository state from being shared accidentally.
101
-
102
- Jcode uses the neutral `.codebase-index/` project storage and falls back to existing OpenCode state when present. Restart Jcode after changing `~/.jcode/mcp.json`, then use `index_codebase`, `index_status`, `codebase_peek`, or `codebase_search`.
103
-
104
- The explicit `@latest` keeps `npx` on the published package even when Jcode is opened inside an `opencode-codebase-index` source checkout. For local development of this package instead, run `npm run build:ts && npm run dev:link-mcp` first.
105
-
106
- ## 🄧 Pi Package
107
-
108
- Install as a Pi package to get first-class `codebase_search`, `index_codebase`, call graph, PR impact, and knowledge-base tools plus the `codebase-search` skill.
109
-
110
- ```bash
111
- pi install npm:opencode-codebase-index
112
- # or, for local development:
113
- pi install ./path/to/opencode-codebase-index
114
- ```
115
-
116
- Pi uses the neutral `.codebase-index/` project storage and falls back to existing OpenCode state when present.
117
-
118
- The Pi extension injects lightweight routing guidance and exposes `codebase_context` as its preferred first repository tool. It routes conceptual discovery, known-symbol definitions, and dependency paths before broad shell search or file reads.
119
-
120
- ## 🧩 Codex Plugin
121
- Install once for Codex threads and get skill guidance plus MCP tools in one manifest.
122
-
123
- 1. **Add this repo as a marketplace source**
124
- ```bash
125
- codex plugin marketplace add Helweg/opencode-codebase-index
126
- ```
127
- 2. **Install the plugin**
128
31
  ```bash
129
- codex plugin add codebase-index@helweg-plugins
32
+ npm install open-codebase-index
130
33
  ```
131
- 3. **Restart or open a new thread** in the target workspace.
132
- 4. Use MCP tools (`index_codebase`, `index_status`, `codebase_search`, etc.) and the `codebase-search` skill guidance.
133
-
134
- The plugin includes:
135
- - `skills/` guidance for local workflows
136
- - `hooks/hooks.json` lightweight session-start guidance
137
- - `.mcp.json` running the published `opencode-codebase-index` CLI via `npx … --host codex`, so a git marketplace install works without a local build
138
- - `.agents/plugins/marketplace.json` so this repo can act as a Codex marketplace source
139
-
140
- For local plugin development from this checkout, build and link the local MCP bin once:
141
-
142
- ```bash
143
- npm run build:ts
144
- npm run dev:link-mcp
145
- ```
146
-
147
- After that, the normal `.mcp.json` command also works when Codex starts the plugin from this repository.
148
-
149
- The native Codex plugin is important: its session hook and `codebase-search` skill tell Codex to use `index_status` and `codebase_context` before shell exploration. A bare MCP configuration does not provide the same selection reliability. Current Codex `exec` releases may cancel MCP calls in non-interactive mode while waiting for an app-tool approval; use an interactive Codex thread to approve the tool call. This is a Codex client limitation, not an MCP server failure.
150
34
 
151
- ## 🧩 Claude Code Plugin
152
- Install once for Claude Code sessions and get skill guidance plus MCP tools in one manifest.
35
+ Legacy installs continue to work with:
153
36
 
154
- 1. **Add this repo as a marketplace source**
155
37
  ```bash
156
- /plugin marketplace add Helweg/opencode-codebase-index
157
- ```
158
- 2. **Install the plugin**
159
- ```bash
160
- /plugin install codebase-index@helweg-plugins
161
- ```
162
- 3. **Restart or open a new session** in the target workspace.
163
- 4. Use MCP tools (`index_codebase`, `index_status`, `codebase_search`, etc.) and the `codebase-search` skill guidance.
164
-
165
- The plugin includes:
166
- - `skills/` guidance for local workflows
167
- - inline `mcpServers` (in `.claude-plugin/plugin.json`) running the published `opencode-codebase-index` CLI via `npx … --host claude`, so a git marketplace install works without a local build
168
- - `.claude-plugin/marketplace.json` so this repo can act as a Claude Code marketplace source
169
-
170
- ### Provider selection notes
171
-
172
- - **Default auto-detect order:** Ollama → GitHub Copilot → OpenAI → Google
173
- - **Ollama** is the preferred zero-cost local option and works especially well for large repos:
174
-
175
- ```bash
176
- ollama pull nomic-embed-text
177
- ```
178
-
179
- ```json
180
- {
181
- "embeddingProvider": "ollama"
182
- }
183
- ```
184
-
185
- - **GitHub Copilot** is a good default if OpenCode already has Copilot auth and you prefer hosted embeddings.
186
- - **OpenAI** is a good hosted option when you want predictable API behavior and standard cloud setup.
187
- - **Google** is available if you prefer Gemini-hosted embeddings.
188
- - If `/status` reports provider or compatibility problems, follow that guidance before using `/index force`.
189
-
190
- ## 🌐 MCP Server (Cursor, Claude Code, Windsurf, etc.)
191
-
192
- Use the same semantic search from any MCP-compatible client. Index once, search from anywhere.
193
-
194
- 1. **Install dependencies**
195
- ```bash
196
- npm install opencode-codebase-index @modelcontextprotocol/sdk zod
38
+ npm install opencode-codebase-index
197
39
  ```
198
40
 
199
- 2. **Configure your MCP client**
41
+ 2. Add it to `opencode.json`:
200
42
 
201
- **Cursor** (`.cursor/mcp.json`):
202
43
  ```json
203
44
  {
204
- "mcpServers": {
205
- "codebase-index": {
206
- "command": "npx",
207
- "args": ["-y", "--package", "opencode-codebase-index", "opencode-codebase-index-mcp", "--project", "/path/to/your/project"]
208
- }
209
- }
45
+ "plugin": ["open-codebase-index"]
210
46
  }
211
47
  ```
212
48
 
213
- **Claude Code** (`claude_desktop_config.json`):
49
+ Legacy alias:
50
+
214
51
  ```json
215
52
  {
216
- "mcpServers": {
217
- "codebase-index": {
218
- "command": "npx",
219
- "args": ["-y", "--package", "opencode-codebase-index", "opencode-codebase-index-mcp", "--project", "/path/to/your/project"]
220
- }
221
- }
53
+ "plugin": ["opencode-codebase-index"]
222
54
  }
223
55
  ```
224
56
 
225
- 3. **CLI options**
226
- ```bash
227
- npx -y --package opencode-codebase-index opencode-codebase-index-mcp --project /path/to/repo
228
- npx -y --package opencode-codebase-index opencode-codebase-index-mcp --config /path/to/config
229
- npx -y --package opencode-codebase-index opencode-codebase-index-mcp
230
- ```
231
-
232
- The MCP server exposes all 13 tools (`codebase_context`, `codebase_search`, `codebase_peek`, `find_similar`, `implementation_lookup`, `call_graph`, `call_graph_path`, `pr_impact`, `index_codebase`, `index_status`, `index_health_check`, `index_metrics`, `index_logs`) and 5 prompts (`search`, `find`, `definition`, `index`, `status`). Native OpenCode and Pi integrations expose the same `codebase_context` entry point.
233
-
234
- The tools carry self-routing descriptions so clients can choose the lightweight path without relying on separate documentation:
235
-
236
- 1. `codebase_context` as the preferred single entry point for repository questions
237
- 2. `index_status` when index readiness is unknown
238
- 3. `codebase_peek` for direct low-token conceptual discovery
239
- 4. `implementation_lookup` for direct known-symbol definition lookup
240
- 5. `codebase_search` only when full semantic content is needed
241
- 6. `grep` for exact identifiers or exhaustive matches
242
- 7. `call_graph` / `call_graph_path` for direct graph queries
243
-
244
- `codebase_context` accepts a `tokenBudget` from 128 to 4000 tokens, defaulting to 1200. The hard cap is counted with the `cl100k_base` tokenizer, including multilingual and emoji text. It returns deterministic location evidence rather than source bodies, removes overlapping same-file results, diversifies evidence across files, and distinguishes duplicates, result-limit exclusions, and token-budget omissions. Increase the budget only when broader location coverage is useful. Use `implementation_lookup`, `codebase_search`, or a targeted file read for the exact source after selecting a location.
245
-
246
- The server also publishes this workflow through the standard MCP initialization `instructions` field. Client behavior remains client-controlled: an MCP server can describe and recommend its tools, but cannot force an agent host to read server instructions or invoke a tool before filesystem search. Clients that ignore MCP instructions still receive the routing guidance in each tool description.
247
-
248
- The MCP dependencies (`@modelcontextprotocol/sdk`, `zod`) ship with the package so published `npx --package opencode-codebase-index` launches work in clean MCP clients.
249
-
250
- If you are testing the MCP command from inside this repository checkout and see `opencode-codebase-index-mcp: command not found`, run `npm run build:ts && npm run dev:link-mcp`. That adds the local bin shim expected by `npx` without changing the published MCP config.
251
-
252
- ## šŸ” See It In Action
253
-
254
- **Scenario**: You're new to a codebase and need to fix a bug in the payment flow.
255
-
256
- **Without Plugin (grep)**:
257
- - `grep "payment" .` → 500 results (too many)
258
- - `grep "card" .` → 200 results (mostly UI)
259
- - `grep "stripe" .` → 50 results (maybe?)
260
-
261
- **With `opencode-codebase-index`**:
262
- You ask: *"Where is the payment validation logic?"*
263
-
264
- Plugin returns:
265
- ```text
266
- src/services/billing.ts:45 (Class PaymentValidator)
267
- src/utils/stripe.ts:12 (Function validateCardToken)
268
- src/api/checkout.ts:89 (Route handler for /pay)
269
- ```
270
-
271
- ## šŸŽÆ When to Use What
272
-
273
- | Scenario | Tool | Why |
274
- |----------|------|-----|
275
- | Don't know the function name | `codebase_context` | Routes conceptual questions to a bounded, low-token evidence pack |
276
- | Exploring unfamiliar codebase | `codebase_context` | Available natively in OpenCode and Pi, and through MCP for other clients |
277
- | Just need to find locations | `codebase_peek` (or `codebase_context`) | Returns metadata only, saves ~90% tokens |
278
- | Need the authoritative definition site | `implementation_lookup` | Prioritizes real implementation definitions over docs/tests |
279
- | Understand code flow | `call_graph` | Find callers/callees of any function |
280
- | Trace dependency paths | `call_graph_path` | Find the shortest known call path between two symbols |
281
- | Know exact identifier | `grep` | Faster, finds all occurrences |
282
- | Need ALL matches | `grep` | Semantic returns top N only |
283
- | Mixed discovery + precision | `/find` (hybrid) | Best of both worlds |
284
-
285
- **Rule of thumb**: `codebase_context` to route discovery first. Then `Read` to examine exact content and `grep` for precision. For symbol-definition questions, use `implementation_lookup` first.
286
-
287
- ## 🧭 OMO CodeGraph Compatibility
288
-
289
- Recent OMO releases include a built-in CodeGraph MCP and make it part of the default agent workflow. This does **not** replace `opencode-codebase-index`; the two tools answer different first questions.
290
-
291
- | Need | Prefer | Why |
292
- |------|--------|-----|
293
- | Find code by intent, behavior, or natural language | `codebase_peek` / `codebase_search` | Semantic + hybrid retrieval works when you do not know exact names |
294
- | Jump to the likely implementation site | `implementation_lookup` | Definition-oriented ranking prefers source over tests/docs |
295
- | Find similar implementations or duplicate patterns | `find_similar` | Embedding similarity compares code shape and meaning |
296
- | Follow callers, callees, imports, inheritance, or implementations | OMO CodeGraph or `call_graph` | Structural graph tools are best for dependency topology |
297
- | Find a shortest known relationship chain | `call_graph_path` | Uses this plugin's indexed call edges to connect two symbols |
298
- | Include external docs, examples, or API references in discovery | `add_knowledge_base` + `codebase_search` | Knowledge bases are indexed into the same retrieval store |
299
-
300
- Recommended OMO workflow:
301
-
302
- 1. Start broad with `codebase_context` when the prompt is conceptual, such as "where is auth enforced?" or "payment validation flow".
303
- 2. Use `implementation_lookup` once you have a symbol or concept that should resolve to a definition.
304
- 3. Use OMO CodeGraph, `call_graph`, or `call_graph_path` after locating the relevant symbol to check blast radius and dependency flow.
305
- 4. Keep `grep` for exact identifiers and exhaustive text matches.
306
-
307
- If OMO reports an uninitialized CodeGraph workspace, follow its `codegraph init` guidance. That setup is independent from this plugin's index under `.opencode/index/`, so `/index` and `codegraph init` may both be useful in the same repository.
57
+ 3. Run `/status`, then `/index`.
58
+ 4. Ask a repository question, for example:
308
59
 
309
- ## šŸ“Š Token Usage
60
+ > Where is authentication state validated before an API request?
310
61
 
311
- In our testing across open-source codebases (axios, express), we observed **up to 90% reduction in token usage** for conceptual queries like *"find the error handling middleware"*.
62
+ The first index creates embeddings. Later runs reuse unchanged content and process only relevant changes.
312
63
 
313
- ### Why It Saves Tokens
64
+ ## Choose your host
314
65
 
315
- - **Without plugin**: Agent explores files, reads code, backtracks, explores more
316
- - **With plugin**: Semantic search returns relevant code immediately → less exploration
66
+ | Host | Recommended integration | Storage |
67
+ |---|---|---|
68
+ | OpenCode | Native plugin | `.opencode/` |
69
+ | Jcode | Per-session MCP server | `.codebase-index/` |
70
+ | Pi | Pi package | `.codebase-index/` |
71
+ | Codex | Marketplace plugin with MCP and skill guidance | `.codebase-index/` |
72
+ | Claude Code | Marketplace plugin with MCP and skill guidance | `.claude/` |
73
+ | Cursor, Windsurf, other MCP clients | `open-codebase-index-mcp` (legacy alias: `opencode-codebase-index-mcp`) | Selected by `--host`; default is OpenCode-compatible |
317
74
 
318
- ### Key Takeaways
75
+ See [Installation and host setup](docs/installation.md) for complete instructions.
319
76
 
320
- 1. **Significant savings possible**: Up to 90% reduction in the best cases
321
- 2. **Results vary**: Savings depend on query type, codebase structure, and agent behavior
322
- 3. **Best for discovery**: Conceptual queries benefit most; exact identifier lookups should use grep
323
- 4. **Complements existing tools**: Provides a faster initial signal, doesn't replace grep/explore
77
+ ## Recommended workflow
324
78
 
325
- ### When the Plugin Helps Most
79
+ 1. **Check readiness** with `index_status` or `/status`.
80
+ 2. **Index when needed** with `index_codebase` or `/index`.
81
+ 3. **Start repository discovery** with `codebase_context`.
82
+ 4. **Use `codebase_peek`** when you only need likely locations.
83
+ 5. **Use `implementation_lookup`** for a known symbol or definition question.
84
+ 6. **Use `codebase_search`** when you need full matching source content.
85
+ 7. **Use `grep`** for exact identifiers or exhaustive text matches.
86
+ 8. **Use call-graph tools** for callers, callees, and dependency paths.
326
87
 
327
- - **Conceptual queries**: "Where is the authentication logic?" (no keywords to grep for)
328
- - **Unfamiliar codebases**: You don't know what to search for yet
329
- - **Large codebases**: Semantic search scales better than exhaustive exploration
88
+ ### Which search tool should I use?
330
89
 
331
- ## šŸ› ļø How It Works
332
-
333
- ```mermaid
334
- graph TD
335
- subgraph Indexing
336
- A[Source Code] -->|Tree-sitter| B[Semantic Chunks]
337
- B -->|Embedding Model| C[Vectors]
338
- C -->|uSearch| D[(Vector Store)]
339
- C -->|SQLite| G[(Embeddings DB)]
340
- B -->|BM25| E[(Inverted Index)]
341
- B -->|Branch Catalog| G
342
- end
343
-
344
- subgraph Searching
345
- Q[User Query] -->|Embedding Model| V[Query Vector]
346
- V -->|Cosine Similarity| D
347
- Q -->|BM25| E
348
- D --> F[Hybrid Fusion RRF/Weighted]
349
- E --> F
350
- F --> X[Deterministic Rerank]
351
- G -->|Branch + Metadata Filters| X
352
- X --> R[Ranked Results]
353
- end
354
- ```
355
-
356
- 1. **Parsing**: We use `tree-sitter` to intelligently parse your code into meaningful blocks (functions, classes, interfaces). JSDoc comments and docstrings are automatically included with their associated code.
357
-
358
- **Supported Languages (Tree-sitter semantic parsing)**: TypeScript, JavaScript, Python, Rust, Swift, Go, Java, C#, Ruby, PHP, Apex, Bash, C, C++, Metal, JSON, TOML, YAML, Zig, GDScript, MATLAB†
359
-
360
- Swift support uses [`tree-sitter-swift` 0.7.3](https://github.com/alex-pinkus/tree-sitter-swift/tree/0.7.3), distributed under the MIT license. It provides Tree-sitter syntax analysis for chunks, symbols, and calls, but does not replace SourceKit semantic analysis.
361
-
362
- † MATLAB (`.m`) is opt-in — see below.
363
-
364
- **Additional Supported Formats (line-based chunking)**: TXT, HTML, HTM, Markdown, Shell scripts
365
-
366
- #### Verified PHP 8.x compatibility
367
-
368
- PHP support uses [`tree-sitter-php` 0.24.2](https://github.com/tree-sitter/tree-sitter-php/releases/tag/v0.24.2), distributed under the MIT license. The regression fixture [`php-8-features.php`](tests/fixtures/call-graph/php-8-features.php) and the Rust and Vitest suites cover the following matrix:
369
-
370
- | Version | Verified syntax |
90
+ | Need | Tool |
371
91
  |---|---|
372
- | PHP 8.0 | attributes, named arguments, promoted properties, union types, the nullsafe operator, and `match` expressions |
373
- | PHP 8.1 | enums with constants, `readonly` properties, intersection types, and first-class callables |
374
- | PHP 8.2 | `readonly` classes and DNF types |
375
- | PHP 8.3 | typed constants and dynamic class-constant access |
376
- | PHP 8.4 | property hooks, asymmetric visibility, and unparenthesized chaining on `new` |
377
- | PHP 8.5 | attributes on constants and the pipe operator `|>` |
378
-
379
- Semantic parsing verifies chunk names and types for functions, classes, interfaces, traits, and `enum_declaration`. Methods remain inside their containing chunk rather than becoming standalone chunks.
380
-
381
- The call graph distinguishes invocations from callable references. Standalone `foo(...)`, `$object->method(...)`, and `Type::method(...)` references do not create edges, while the same callables used as direct or parenthesized `|>` operands are recorded because the pipe invokes them. Coverage also includes qualified and relative `namespace\foo()` calls, nullsafe and static calls, constructors, named arguments, `match` bodies, property hooks, and enum methods.
382
-
383
- The graph remains limited to statically resolvable named targets. Dynamic calls such as `$callable()`, `$object->$method()`, and `new $class()`, `include` and `require` expressions, and `extends` and `implements` relationships are outside this contribution.
384
-
385
- PHP 8.5 support remains partial in upstream grammar 0.24.2. Final promoted properties and `clone($object, [...])` still produce `ERROR` nodes, so this project does not claim support for those two forms.
386
-
387
- **Default File Patterns**:
388
- ```
389
- **/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs} **/*.{py,pyi}
390
- **/*.{go,rs,java,cs,kt,scala} **/*.{c,cpp,cc,cxx,h,hpp,hxx}
391
- **/*.{rb,php,inc,swift} **/*.{vue,svelte,astro}
392
- **/*.{sql,graphql,proto} **/*.{yaml,yml,toml}
393
- **/*.{md,mdx} **/*.{sh,bash,zsh}
394
- **/*.{txt,html,htm} **/*.{cls,trigger}
395
- **/*.zig **/*.gd
396
- **/*.metal
397
- ```
398
-
399
- Use `include` to replace defaults, or `additionalInclude` to extend (e.g. `"**/*.pdf"`, `"**/*.csv"`).
400
-
401
- †**MATLAB opt-in**: `.m` is excluded from defaults because it conflicts with the Objective-C extension used on Apple codebases. To enable MATLAB discovery, add to your global config (`~/.config/opencode/codebase-index.json`):
402
- ```json
403
- { "additionalInclude": ["**/*.m"] }
404
- ```
92
+ | Route a repository question to a bounded evidence pack | `codebase_context` |
93
+ | Find likely files and symbols without source bodies | `codebase_peek` |
94
+ | Retrieve full matching code | `codebase_search` |
95
+ | Find an authoritative definition | `implementation_lookup` |
96
+ | Find analogous implementations or duplicates | `find_similar` |
97
+ | Find direct callers or callees | `call_graph` |
98
+ | Find a path between two symbols | `call_graph_path` |
99
+ | Analyze a branch or pull request blast radius | `pr_impact` |
405
100
 
406
- **Max File Size**: Default 1MB (1048576 bytes). Configure via `indexing.maxFileSize` (bytes).
407
- 2. **Chunking**: Large blocks are split with overlapping windows to preserve context across chunk boundaries.
408
- 3. **Embedding**: These blocks are converted into vector representations using your configured AI provider.
409
- 4. **Storage**: Embeddings are stored in SQLite (deduplicated by content hash) and vectors in `usearch` with F16 quantization for 50% memory savings. A branch catalog tracks which chunks exist on each branch.
410
- 5. **Hybrid Search**: Combines semantic similarity (vectors) with BM25 keyword matching, applies branch and hard metadata scopes before reranking, fuses candidates (`rrf` default, `weighted` fallback), then applies deterministic local intent ranking and optional external reranking.
101
+ See [Tools and commands](docs/tools.md) for host availability, tool details, MCP prompts, and slash commands.
411
102
 
412
- **Performance characteristics:**
413
- - **Incremental indexing**: ~50ms check time — only re-embeds changed files
414
- - **Smart chunking**: Understands code structure to keep functions whole, with overlap for context
415
- - **Native speed**: Core logic written in Rust for maximum performance
416
- - **Memory efficient**: F16 vector quantization reduces index size by 50%
417
- - **Branch-aware**: Automatically tracks which chunks exist on each git branch
418
- - **Provider validation**: Detects embedding provider/model changes and requires rebuild to prevent garbage results
103
+ ## How it works
419
104
 
420
- ## 🌿 Branch-Aware Indexing
421
-
422
- The plugin automatically detects git branches and optimizes indexing across branch switches.
423
-
424
- ### How It Works
425
-
426
- When you switch branches, code changes but embeddings for unchanged content remain the same. The plugin:
427
-
428
- 1. **Stores embeddings by content hash**: Embeddings are deduplicated across branches
429
- 2. **Tracks branch membership**: A lightweight catalog tracks which chunks exist on each branch
430
- 3. **Filters search results**: Queries only return results relevant to the current branch
431
-
432
- ### Benefits
433
-
434
- | Scenario | Without Branch Awareness | With Branch Awareness |
435
- |----------|-------------------------|----------------------|
436
- | Switch to feature branch | Re-index everything | Instant — reuse existing embeddings |
437
- | Return to main | Re-index everything | Instant — catalog already exists |
438
- | Search on branch | May return stale results | Only returns current branch's code |
439
-
440
- ### Automatic Behavior
441
-
442
- - **Branch detection**: Automatically reads from `.git/HEAD`
443
- - **Re-indexing on switch**: Triggers when you switch branches (via file watcher)
444
- - **Legacy migration**: Automatically migrates old indexes on first run
445
- - **Garbage collection**: Health check removes orphaned embeddings and chunks
446
-
447
- ### Storage Structure
448
-
449
- ```
450
- .opencode/index/
451
- ā”œā”€ā”€ codebase.db # SQLite: embeddings, chunks, branch catalog, symbols, call edges
452
- ā”œā”€ā”€ vectors.usearch # Vector index (uSearch)
453
- ā”œā”€ā”€ inverted-index.json # BM25 keyword index
454
- └── file-hashes.json # File change detection
105
+ ```text
106
+ source files
107
+ │
108
+ ā”œā”€ file discovery and git-aware change detection
109
+ ā”œā”€ tree-sitter parsing and semantic chunking
110
+ ā”œā”€ embedding generation and content-hash reuse
111
+ ā–¼
112
+ SQLite metadata + usearch vectors + BM25 index
113
+ │
114
+ ā”œā”€ semantic candidates
115
+ ā”œā”€ keyword candidates
116
+ ā”œā”€ branch and request filters
117
+ ā”œā”€ deterministic fusion and ranking
118
+ ā–¼
119
+ locations, source results, definitions, and call-graph evidence
455
120
  ```
456
121
 
457
- ### File Exclusions
458
-
459
- The following files/folders are excluded from indexing by default:
460
-
461
- - **Hidden files/folders**: Files starting with `.` (e.g., `.github`, `.vscode`, `.env`)
462
- - **Build folders**: Folders containing "build" in their name (e.g., `build`, `mingwBuildDebug`, `cmake-build-debug`)
463
- - **Default excludes**: `node_modules`, `dist`, `vendor`, `__pycache__`, `target`, `coverage`, etc.
464
-
465
- ## 🧰 Tools Available
466
-
467
- The plugin exposes these tools to the OpenCode agent:
468
-
469
- `codebase_context` is MCP-server-only.
470
-
471
- ### `codebase_context`
472
- *MCP-only entrypoint for combined routing*
473
- **Preferred first tool for repository questions.** Routes to the lowest-token indexed operation that matches the query: conceptual discovery, definition lookup, callers/callees, or symbol-to-symbol paths.
474
- - **Use for**: New questions about behavior, locating symbols, or tracing direct call relationships.
475
- - **Example**: `"Where is the payment validation logic?"`
476
- - **Workflow**: If the query is conceptual, it may return locations first. For exact behavior text, follow with `codebase_search`.
477
-
478
- ### `codebase_search`
479
- **Behavioral semantic retrieval with full content.** Searches code by describing behavior.
480
- - **Use for**: Discovery when you already want full matching snippets and are ready to inspect implementation text.
481
- - **Example**: `"find the middleware that sanitizes input"`
482
- - **Ranking path**: hybrid retrieval → branch/directory/file-type/chunk/blame scope → fusion (`search.fusionStrategy`) → deterministic local intent rerank (`search.rerankTopN`) → optional external rerank within local evidence classes → minimum-score filter
483
- - **Blame filters**: when `indexing.gitBlame.enabled` is `true`, filter with `blameAuthor`, `blameSha`, or `blameSince`.
484
-
485
- The local ranker is deterministic and does not call another model. Exact symbol names receive a strong NFKC and case-normalized match signal. Definition and implementation questions prefer authoritative declarations over imports, export wrappers, tests, fixtures, docs, and generated or vendor files. Explicit test, docs, config, and call-flow wording instead promotes the requested evidence class. Natural-language conceptual queries keep retrieval score as the dominant signal, while nested duplicate chunks are removed and remaining evidence is spread across relevant files. True score ties retain input order, then candidate id as a final deterministic fallback.
486
-
487
- When an external reranker is enabled, hard directory, file-type, chunk-type, and blame scopes are enforced before any request. The external service receives only candidates already inside that scope and only the candidate's exact indexed line range, without extra surrounding source. Local intent classification, exact-name promotion, duplicate suppression, and evidence-class ordering remain local.
122
+ The TypeScript layer handles host integration, configuration, indexing orchestration, providers, ranking, and tools. The Rust NAPI module handles parsing, vector storage, SQLite operations, BM25 indexing, hashing, and call extraction.
488
123
 
489
- **Writing good queries:**
124
+ Read [Architecture](ARCHITECTURE.md) for the detailed data flow and design decisions.
490
125
 
491
- | āœ… Good queries (describe behavior) | āŒ Bad queries (too vague) |
492
- |-------------------------------------|---------------------------|
493
- | "function that validates email format" | "email" |
494
- | "error handling for failed API calls" | "error" |
495
- | "middleware that checks authentication" | "auth middleware" |
496
- | "code that calculates shipping costs" | "shipping" |
497
- | "where user permissions are checked" | "permissions" |
126
+ ## Embedding providers
498
127
 
499
- ### `codebase_peek`
500
- **Token-efficient discovery.** Returns only metadata (file, line, name, type) without code content.
501
- - **Use for**: Finding WHERE code is before deciding what to read. Saves ~90% tokens vs `codebase_search`.
502
- - **Ranking path**: same hybrid ranking path as `codebase_search` (metadata-only output)
503
- - **Example output**:
504
- ```
505
- [1] function "validatePayment" at src/billing.ts:45-67 (score: 0.92)
506
- abc1234 | Jane Doe | 2025-03-14 | billing: validate payment state
507
- [2] class "PaymentProcessor" at src/processor.ts:12-89 (score: 0.87)
128
+ With `embeddingProvider: "auto"`, providers are tried in this order:
508
129
 
509
- Use Read tool to examine specific files.
510
- ```
511
- - **Workflow**: `codebase_peek` → find locations → `Read` specific files
130
+ 1. Ollama
131
+ 2. GitHub Copilot
132
+ 3. OpenAI
133
+ 4. Google
512
134
 
513
- ### `implementation_lookup`
514
- **Definition-first lookup.** Jumps to the authoritative definition site for a symbol or natural-language definition query.
515
- - **Use for**: "Where is X defined?", symbol-definition requests, and cases where you want the implementation site rather than all usages.
516
- - **Behavior**: Prefers real implementation files over tests, docs, examples, and fixtures.
517
- - **Fallback**: If nothing authoritative is found, use `codebase_search` for broader discovery.
518
-
519
- ### `find_similar`
520
- Find code similar to a provided snippet.
521
- - **Use for**: Duplicate detection, refactor prep, pattern mining.
522
- - **Ranking path**: semantic retrieval only + deterministic rerank (no BM25, no RRF).
523
-
524
- ### `index_codebase`
525
- Manually trigger indexing.
526
- - **Use for**: Forcing a re-index or checking stats.
527
- - **Parameters**: `force` (rebuild all), `estimateOnly` (check costs), `verbose` (show skipped files and parse failures).
528
-
529
- ### `index_status`
530
- Checks if the index is ready and healthy.
531
- - **Recommended workflow**: run this after `/index` to confirm the detected provider/model and whether the index is ready to search.
532
-
533
- ### `index_health_check`
534
- Maintenance tool to remove stale entries from deleted files and orphaned embeddings/chunks from the database.
535
-
536
- ### `index_metrics`
537
- Returns collected metrics about indexing and search performance. Operational metrics require `debug.enabled` and `debug.metrics` to be `true`.
538
- - **Metrics include**: Files indexed, chunks created, cache hit rate, search timing breakdown, GC stats, embedding API call stats.
539
- - **Privacy-safe effectiveness metrics**: Set top-level `effectivenessMetrics.enabled` to `true`. This path does not enable debug logging. It records fixed route/host/outcome counters, bounded aggregate histograms, and bounded per-route outcome, result-count, latency, and returned-token histograms. The per-route views make it possible to compare route reliability and cost without retaining request data. Counters are disabled by default, memory-only, fixed-cardinality, and process-lifetime across Indexer replacement and configuration-watcher refresh. One process-wide collector aggregates opted-in calls without project or repository identity dimensions. It never retains queries, response text, source, symbols, paths, repository names, user identity, or stable identifiers.
540
- - **Effectiveness-only privacy boundary**: With only `effectivenessMetrics.enabled` set, `debug.enabled` remains `false`, `index_logs` remains disabled, operational debug metrics remain off, and no query, path, source, secret, or response text is emitted by metrics output or written to index files. Only the bounded aggregate snapshot is returned by `index_metrics`.
541
- - **Debug logging is separate**: Explicitly enabling `debug.enabled` with `debug.logSearch` preserves the existing diagnostic behavior and may retain raw queries and repository details in memory. Do not enable debug search logs when only privacy-safe aggregates are wanted.
542
- - **Reset**: Pass `reset: true` to clear both operational and process-wide effectiveness metrics before returning the new zeroed snapshot. Process exit also clears effectiveness metrics because they are never persisted.
543
-
544
- ### `index_logs`
545
- Returns recent debug logs with optional filtering.
546
- - **Parameters**: `category` (optional: `search`, `embedding`, `cache`, `gc`, `branch`), `level` (optional: `error`, `warn`, `info`, `debug`), `limit` (default: 50).
547
-
548
- ### `call_graph`
549
-
550
- Query the call graph to find callers or callees of a function/method. Automatically built during indexing for TypeScript, JavaScript, Python, Go, Rust, Swift, PHP, Apex, Zig, GDScript, MATLAB, Bash, C, C++, and Metal.
551
-
552
- For Swift, resolution remains name-based, so overloads, extension duplicates, and protocol dispatch can remain ambiguous. Syntax alone cannot always distinguish a superclass from a protocol conformance or an enum raw-value type from a protocol. Constructor classification uses an uppercase ASCII initial heuristic while retaining the exact name for resolution. `tree-sitter-swift` 0.7.3 also has limitations around `sending`, abbreviated collection constructors such as `[Int]()`, and some generic calls after `self`, `super`, optional chaining, or chained calls. The index does not infer SourceKit semantics for these cases.
553
-
554
- - **Use for**: Understanding code flow, tracing dependencies, impact analysis.
555
- - **Parameters**: `name` (function name), `direction` (`callers` or `callees`), `filePath` (optional duplicate-name disambiguator), `symbolId` (optional backward-compatible escape hatch), `relationshipType` (optional: `Call`, `MethodCall`, `Constructor`, `Import`, `Inherits`, `Implements`). Unique names resolve automatically; ambiguous names return bounded candidate locations.
556
- - **Example**: Find who calls `validateToken` → `call_graph(name="validateToken", direction="callers")`
557
-
558
- ### `call_graph_path`
559
-
560
- Find the shortest known call-graph path between two symbols. Use it after `codebase_peek`, `implementation_lookup`, or `call_graph` identifies the important source and target names.
561
-
562
- - **Use for**: Blast-radius checks, dependency-chain discovery, explaining how one subsystem reaches another.
563
- - **Parameters**: `from` (source symbol name), `to` (target symbol name), `fromFilePath` and `toFilePath` (optional duplicate-name disambiguators), `maxDepth` (optional, default `10`).
564
- - **Example**: Trace how `createOrder` reaches `chargeCard` → `call_graph_path(from="createOrder", to="chargeCard")`
565
-
566
- ### `pr_impact`
567
- Analyzes a PR's changed files to determine impact scope within the codebase.
568
- - **Use for**: Understanding which symbols are affected by a PR, their call-graph reach, risk level, and community/cluster detection.
569
- - **Parameters**: `checkConflicts` (optional, default `false`) — when `true`, detects overlapping concurrent PRs sharing affected symbols and returns `conflictingPRs`.
570
-
571
- ### `index_visualize`
572
- Generate a self-contained temporal call graph view for browser-based exploration.
573
-
574
- - **Use for**: Onboarding, architecture walkthroughs, and understanding what moved recently before drilling into call relationships.
575
- - **What it shows**: Recent change lenses, module overview, symbol exploration, hotspots, and cycles.
576
- - **Concepts**: Modules are path-based code areas such as `src/tools` or `native`; symbols are indexed functions, classes, methods, or similar named code units; edges are caller/callee relationships.
577
- - **Parameters**: `directory` (optional folder filter), `maxNodes` (default `5000`), `includeOrphans` (include disconnected symbols).
578
- - **Output**: Writes a temporary HTML file you can open in any browser.
579
- - **Example**: `index_visualize(directory="src/tools", maxNodes=1500)`
580
-
581
- CLI shortcut after building locally:
135
+ Ollama is the simplest local option:
582
136
 
583
137
  ```bash
584
- npm run build
585
- npm run visualize
586
- npm run visualize -- native
587
- npm run visualize -- src/tools max=1000
588
- npm run visualize -- src/indexer orphans
589
- ```
590
-
591
-
592
- ### `add_knowledge_base`
593
- Add a folder as a knowledge base to be indexed alongside project code.
594
- - **Use for**: Indexing external documentation, API references, example programs.
595
- - **Parameters**: `path` (folder path, absolute or relative), `reindex` (optional, default `true`).
596
- - **Restrictions**: System directories (`/etc`, `/proc`, `/sys`, `/dev`) and sensitive home directories (`.ssh`, `.gnupg`, `.aws`, `.docker`, `.kube`) are blocked. Symlinks are resolved before validation.
597
- - **Example**: `add_knowledge_base(path="/path/to/docs")`
598
-
599
- ### `list_knowledge_bases`
600
- List all configured knowledge base folders and their status.
601
-
602
- ### `remove_knowledge_base`
603
- Remove a knowledge base folder from the index.
604
- - **Parameters**: `path` (folder path to remove), `reindex` (optional, default `false`).
605
- - **Example**: `remove_knowledge_base(path="/path/to/docs")`
606
-
607
- ## šŸŽ® Slash Commands
608
-
609
- The plugin automatically registers these slash commands:
610
-
611
- | Command | Description |
612
- | ------- | ----------- |
613
- | `/definition <query>` | **Definition Lookup**. Finds the authoritative implementation site for a symbol or concept. |
614
- | `/peek <query>` | **Quick Semantic Lookup**. Returns likely locations only, without full code content. |
615
- | `/reindex` | **Full Rebuild**. Rebuilds the codebase index from scratch. |
616
- | `/search <query>` | **Pure Semantic Search**. Best for "How does X work?" |
617
- | `/find <query>` | **Hybrid Search**. Combines semantic search + grep. Best for "Find usage of X". |
618
- | `/call-graph <query>` | **Call Graph Trace**. Find callers/callees to understand execution flow. |
619
- | `/pr-impact <PR number or branch>` | **PR Impact Analysis**. Analyze changed files, affected symbols, communities, hub nodes, and risk. |
620
- | `/visualize [directory|max=N|orphans]` | **Temporal Call Graph Visualization**. Open recent changes, module overview, symbol exploration, hotspots, and cycles in a browser-ready HTML file. |
621
- | `/index` | **Update Index**. Runs incremental indexing by default; use `/index force` for a full rebuild. |
622
- | `/status` | **Check Status**. Shows if indexed, chunk count, and provider info. |
623
-
624
- ## šŸ“š Knowledge Base
625
-
626
- The plugin can index **external documentation** alongside your project code. The indexed codebase includes:
627
-
628
- - **Project Source Code** — all code files in the current workspace
629
- - **API References** — hardware API docs, library documentation
630
- - **Usage Guides** — tutorials, how-to guides
631
- - **Example Programs** — code samples, demo projects
632
-
633
- ### Adding Knowledge Base Folders
634
-
635
- Use the built-in tools to add documentation folders:
636
-
637
- ```
638
- add_knowledge_base(path="/path/to/api-docs")
639
- add_knowledge_base(path="/path/to/examples")
640
- ```
641
-
642
- The folder will be indexed into the **same database** as your project code. All searches automatically include both sources.
643
-
644
- ### Managing Knowledge Bases
645
-
646
- ```
647
- list_knowledge_bases # Show configured knowledge bases
648
- remove_knowledge_base(path="/path/to/api-docs") # Remove a knowledge base
649
- ```
650
-
651
- ### Configuration Example
652
-
653
- Project-level config (`.opencode/codebase-index.json`):
654
- ```json
655
- {
656
- "knowledgeBases": [
657
- "/home/user/docs/esp-idf",
658
- "/home/user/docs/arduino"
659
- ]
660
- }
138
+ ollama pull nomic-embed-text
661
139
  ```
662
140
 
663
- Global-level config (`~/.config/opencode/codebase-index.json`):
664
141
  ```json
665
142
  {
666
- "embeddingProvider": "custom",
667
- "customProvider": {
668
- "baseUrl": "{env:EMBED_BASE_URL}",
669
- "model": "BAAI/bge-m3",
670
- "dimensions": 1024,
671
- "apiKey": "{env:EMBED_API_KEY}"
672
- }
143
+ "embeddingProvider": "ollama"
673
144
  }
674
145
  ```
675
146
 
676
- Config merging: Global config is the base, project config overrides. Knowledge bases from both levels are merged.
677
-
678
- ### Syncing Changes
147
+ A custom OpenAI-compatible embeddings endpoint is also supported. Provider, indexing, search, reranking, include/exclude, knowledge-base, storage, and debug settings are documented in [Configuration](docs/configuration.md).
679
148
 
680
- - **Project code**: Auto-synced via file watcher (real-time)
681
- - **Knowledge base folders**: Manual sync — run `/index force` after changes
149
+ ## Configuration example
682
150
 
683
- ## šŸ”„ Reranking
684
-
685
- The plugin supports **API-based reranking** for improved search result quality. Reranking uses a cross-encoder model to rescore the top search results.
686
-
687
- ### Enable Reranking
688
-
689
- Add to your config (`.opencode/codebase-index.json` or global config):
151
+ OpenCode project config lives at `.opencode/codebase-index.json`. Codex, Pi, and Jcode use `.codebase-index/config.json`; Claude uses `.claude/codebase-index.json`.
690
152
 
691
153
  ```json
692
154
  {
693
- "reranker": {
694
- "enabled": true,
695
- "baseUrl": "https://api.cohere.ai/v1",
696
- "model": "rerank-v3.5",
697
- "apiKey": "{env:RERANK_API_KEY}",
698
- "topN": 20
699
- }
700
- }
701
- ```
702
-
703
- ### Reranker Options
704
-
705
- | Option | Default | Description |
706
- |--------|---------|-------------|
707
- | `enabled` | `false` | Enable reranking |
708
- | `baseUrl` | - | Rerank API endpoint |
709
- | `model` | - | Reranking model name |
710
- | `apiKey` | - | API key (use `{env:VAR}` for security) |
711
- | `topN` | `20` | Number of top results to rerank |
712
- | `timeoutMs` | `30000` | Request timeout |
713
-
714
- ### How It Works
715
-
716
- ```
717
- Query → Embedding Search → BM25 Search → Fusion → Reranking → Results
718
- ```
719
-
720
- 1. **Embedding Search**: Semantic similarity via vector search
721
- 2. **BM25 Search**: Keyword matching via inverted index
722
- 3. **Fusion**: Combine semantic + keyword results (RRF or weighted)
723
- 4. **Reranking**: Cross-encoder rescores top N results via API
724
- 5. **Results**: Final ranked results
725
-
726
- ### Supported Reranking APIs
727
-
728
- Any OpenAI-compatible reranking endpoint. Examples:
729
- - **SiliconFlow**: `BAAI/bge-reranker-v2-m3`
730
- - **Cohere**: `rerank-english-v3.0`
731
- - **Local models**: Any server implementing `/v1/rerank` format
732
-
733
- ## āš™ļø Configuration
734
-
735
- ### Storage Paths (OpenCode + Codex + Claude + Pi)
736
- OpenCode default (existing behavior):
737
- - project config: `.opencode/codebase-index.json`
738
- - project index: `.opencode/index`
739
- - global config: `~/.config/opencode/codebase-index.json`
740
- - global index: `~/.opencode/global-index`
741
-
742
- Codex/Pi host mode (neutral default):
743
- - project config: `.codebase-index/config.json`
744
- - project index: `.codebase-index/index`
745
- - global config: `~/.config/codebase-index/config.json`
746
- - global index: `~/.codebase-index/global-index`
747
-
748
- Claude Code host mode (`--host claude`):
749
- - project config: `.claude/codebase-index.json`
750
- - project index: `.claude/index`
751
- - global config: `~/.claude/codebase-index.json`
752
- - global index: `~/.claude/global-index`
753
-
754
- Codex, Claude Code, and Pi read legacy OpenCode paths when host-native paths are absent, so existing state continues to work.
755
-
756
- Zero-config by default (uses `auto` mode). Customize in `.opencode/codebase-index.json`:
757
-
758
- ### Full Configuration Example
759
-
760
- ```json
761
- {
762
- // === Embedding Provider ===
763
- "embeddingProvider": "custom", // auto | github-copilot | openai | google | ollama | custom
764
- "scope": "project", // project (per-repo) | global (shared)
765
-
766
- // === Custom Embedding API (when embeddingProvider is "custom") ===
767
- "customProvider": {
768
- "baseUrl": "{env:EMBED_BASE_URL}",
769
- "model": "BAAI/bge-m3",
770
- "dimensions": 1024,
771
- "apiKey": "{env:EMBED_API_KEY}",
772
- "maxTokens": 8192, // Max tokens per input text
773
- "timeoutMs": 30000, // Request timeout (ms)
774
- "concurrency": 3, // Max concurrent requests
775
- "requestIntervalMs": 1000, // Min delay between requests (ms)
776
- "maxBatchSize": 64 // Max inputs per /embeddings request
777
- },
778
-
779
- // === File Patterns ===
780
- "include": [ // Override default include patterns
781
- "**/*.{ts,js,py,go,rs}"
782
- ],
783
- "exclude": [ // Override default exclude patterns
784
- "**/node_modules/**"
785
- ],
786
- "additionalInclude": [ // Extend defaults (not replace)
787
- "**/*.{txt,html,htm}",
788
- "**/*.pdf"
789
- ],
790
-
791
- // === Knowledge Bases ===
792
- "knowledgeBases": [ // External docs to index alongside code
793
- "/home/user/docs/esp-idf",
794
- "/home/user/docs/arduino"
795
- ],
796
-
797
- // === Indexing ===
155
+ "embeddingProvider": "auto",
156
+ "scope": "project",
798
157
  "indexing": {
799
- "autoIndex": false, // Explicit opt-in for automatic indexing
800
- "autoIndexWaitMs": 10000, // First-retrieval wait bound (0-60000ms)
801
- "autoIndexMaxRetries": 5, // Transient interprocess lock retries (0-10)
802
- "autoIndexRetryDelayMs": 100, // Initial exponential lock retry delay
803
- "watchFiles": true, // Re-index on file changes
804
- "pauseBackgroundIndexingOnBattery": false, // Defer background indexing on macOS battery power
805
- "maxFileSize": 1048576, // Max file size in bytes (default: 1MB)
806
- "maxChunksPerFile": 100, // Max chunks per file
807
- "semanticOnly": false, // Only index functions/classes (skip blocks)
808
- "retries": 3, // Embedding API retry attempts
809
- "retryDelayMs": 1000, // Delay between retries (ms)
810
- "autoGc": true, // Auto garbage collection
811
- "gcIntervalDays": 7, // GC interval (days)
812
- "gcOrphanThreshold": 100, // GC trigger threshold
813
- "requireProjectMarker": true, // Require .git/package.json to index
814
- "maxDepth": 5, // Max directory depth (-1=unlimited, 0=root only)
815
- "maxFilesPerDirectory": 100, // Max files per directory (smallest first)
816
- "fallbackToTextOnMaxChunks": true // Fallback to text chunking on maxChunksPerFile
158
+ "autoIndex": false,
159
+ "watchFiles": true,
160
+ "requireProjectMarker": true,
161
+ "semanticOnly": false
817
162
  },
818
-
819
- // === Search ===
820
163
  "search": {
821
- "maxResults": 20, // Max results to return
822
- "minScore": 0.1, // Min similarity score (0-1)
823
- "hybridWeight": 0.5, // Keyword (1.0) vs semantic (0.0)
824
- "fusionStrategy": "rrf", // rrf | weighted
825
- "rrfK": 60, // RRF smoothing constant
826
- "rerankTopN": 20, // Deterministic rerank depth
827
- "contextLines": 0, // Extra lines before/after match
828
- "routingHints": true, // Runtime nudges for local discovery/definition queries
829
- "routingGraphHandoffHints": false, // Add opt-in graph/OMO CodeGraph handoff wording
830
- "routingHintRole": "system" // system | developer (message role used for hints)
831
- },
832
- "reranker": {
833
- "enabled": false,
834
- "provider": "cohere",
835
- "model": "rerank-v3.5",
836
- "apiKey": "{env:RERANK_API_KEY}",
837
- "topN": 15,
838
- "timeoutMs": 10000
839
- },
840
- "debug": {
841
- "enabled": false, // Enable debug logging
842
- "logLevel": "info", // error | warn | info | debug
843
- "logSearch": true, // Log search operations
844
- "logEmbedding": true, // Log embedding API calls
845
- "logCache": true, // Log cache hits/misses
846
- "logGc": true, // Log garbage collection
847
- "logBranch": true, // Log branch detection and switches to index_logs (no stdout output)
848
- "metrics": false // Enable operational metrics collection
849
- },
850
- "effectivenessMetrics": {
851
- "enabled": false // Opt in without enabling debug logs
164
+ "maxResults": 20,
165
+ "minScore": 0.1,
166
+ "fusionStrategy": "rrf",
167
+ "rerankTopN": 20
852
168
  }
853
169
  }
854
170
  ```
855
171
 
856
- String values in `codebase-index.json` can reference environment variables with `{env:VAR_NAME}` when the placeholder is the entire string value. Variable names must match `[A-Z_][A-Z0-9_]*`. This is useful for secrets such as custom provider API keys so they do not need to be committed to the config file.
172
+ Only specify values you want to override. See [Configuration](docs/configuration.md) for defaults and host-specific paths.
857
173
 
858
- ```json
859
- {
860
- "embeddingProvider": "custom",
861
- "customProvider": {
862
- "baseUrl": "{env:EMBED_BASE_URL}",
863
- "model": "nomic-embed-text",
864
- "dimensions": 768,
865
- "apiKey": "{env:EMBED_API_KEY}"
866
- }
867
- }
868
- ```
869
-
870
- ### Options Reference
871
-
872
- `indexing.autoIndex` remains disabled by default because indexing can invoke a paid embedding provider. When explicitly enabled, MCP hosts start one process-scoped job before accepting tool work where practical, skip a healthy current index, retry transient multiprocess lock contention within the configured bound, and expose sanitized state/progress through `index_status`. If no readable index exists, retrieval tools wait up to `autoIndexWaitMs`; after that they report that indexing is still running or failed instead of returning misleading empty search results. Home-directory and project-marker protections still apply.
873
-
874
- | Option | Default | Description |
875
- |--------|---------|-------------|
876
- | `embeddingProvider` | `"auto"` | Which AI to use: `auto`, `github-copilot`, `openai`, `google`, `ollama`, `custom` |
877
- | `scope` | `"project"` | `project` = index per repo, `global` = shared index across repos |
878
- | `include` | (defaults) | Override the default include patterns (replaces defaults) |
879
- | `exclude` | (defaults) | Override the default exclude patterns (replaces defaults) |
880
- | `additionalInclude` | `[]` | Additional file patterns to include (extends defaults, e.g. `"**/*.txt"`, `"**/*.html"`) |
881
- | `knowledgeBases` | `[]` | External directories to index as knowledge bases (absolute or relative paths) |
882
- | **indexing** | | |
883
- | `autoIndex` | `false` | Explicitly opt in to automatic startup/first-use indexing. When disabled, retrieval never starts indexing or paid embedding work. |
884
- | `autoIndexWaitMs` | `10000` | Maximum time (0-60000ms) a first retrieval waits for an enabled automatic index job before returning actionable in-progress status. |
885
- | `autoIndexMaxRetries` | `5` | Maximum transient interprocess lock retries (0-10) for background automatic indexing. |
886
- | `autoIndexRetryDelayMs` | `100` | Initial exponential lock retry delay in milliseconds (10-10000). |
887
- | `watchFiles` | `true` | Re-index when files change |
888
- | `pauseBackgroundIndexingOnBattery` | `false` | On macOS, defer automatic startup and watcher-triggered indexing while using battery power, then run one pending incremental update after AC power returns. Manual `index_codebase` requests remain available. This option has no effect on other platforms. |
889
- | `maxFileSize` | `1048576` | Skip files larger than this (bytes). Default: 1MB |
890
- | `maxChunksPerFile` | `100` | Maximum chunks to index per file (controls token costs for large files) |
891
- | `semanticOnly` | `false` | When `true`, only index semantic nodes (functions, classes) and skip generic blocks |
892
- | `retries` | `3` | Number of retry attempts for failed embedding API calls |
893
- | `retryDelayMs` | `1000` | Delay between retries in milliseconds |
894
- | `autoGc` | `true` | Automatically run garbage collection to remove orphaned embeddings/chunks |
895
- | `gcIntervalDays` | `7` | Run GC on initialization if last GC was more than N days ago |
896
- | `gcOrphanThreshold` | `100` | Run GC after indexing if orphan count exceeds this threshold |
897
- | `requireProjectMarker` | `true` | Require a project marker (`.git`, `package.json`, etc.) to enable file watching and auto-indexing. Prevents accidentally indexing large directories like home. Set to `false` to index any directory. |
898
- | `maxDepth` | `5` | Max directory traversal depth. `-1` = unlimited, `0` = only files in root dir, `1` = one level of subdirectories, etc. |
899
- | `maxFilesPerDirectory` | `100` | Max files to index per directory. Always picks the smallest files first. |
900
- | `fallbackToTextOnMaxChunks` | `true` | When a file exceeds `maxChunksPerFile`, fallback to text-based (line-by-line) chunking instead of skipping the rest of the file. |
901
- | `gitBlame.enabled` | `false` | Annotate changed chunks with `git blame` commit SHA, author, author email, commit timestamp, and summary. Enables `blameAuthor`, `blameSha`, and `blameSince` filters. |
902
- | **search** | | |
903
- | `maxResults` | `20` | Maximum results to return |
904
- | `minScore` | `0.1` | Minimum similarity score (0-1). Lower = more results |
905
- | `hybridWeight` | `0.5` | Balance between keyword (1.0) and semantic (0.0) search |
906
- | `fusionStrategy` | `"rrf"` | Hybrid fusion mode: `"rrf"` (rank-based reciprocal rank fusion) or `"weighted"` (legacy score blending fallback) |
907
- | `rrfK` | `60` | RRF smoothing constant. Higher values flatten rank impact, lower values prioritize top-ranked candidates more strongly |
908
- | `rerankTopN` | `20` | Deterministic rerank depth cap. Applies lightweight name/path/chunk-type rerank to top-N only |
909
- | `contextLines` | `0` | Extra lines to include before/after each match |
910
- | `routingHints` | `true` | Inject lightweight runtime hints for local conceptual discovery, broad repository coding tasks, and definition lookups. Set to `false` to disable plugin-side routing nudges. |
911
- | `routingGraphHandoffHints` | `false` | When `true`, conceptual discovery hints also say to use graph tools (including OMO CodeGraph) after semantic discovery identifies relevant symbols. |
912
- | `routingHintRole` | `"system"` | Message role used when injecting routing hints: `"system"` (default) or `"developer"`. |
913
- | **reranker** | | Optional second-stage model reranker for the top candidate pool |
914
- | `enabled` | `false` | Turn external reranking on/off |
915
- | `provider` | `"custom"` | Hosted shortcuts: `cohere`, `jina`, or `custom` |
916
- | `model` | — | Reranker model name required when enabled |
917
- | `baseUrl` | provider default | Override reranker endpoint base URL. `cohere` → `https://api.cohere.ai/v1`, `jina` → `https://api.jina.ai/v1` |
918
- | `apiKey` | — | API key for hosted reranker providers |
919
- | `topN` | `15` | Number of top candidates to send to the external reranker |
920
- | `timeoutMs` | `10000` | Timeout for external rerank requests |
921
- | **debug** | | |
922
- | `enabled` | `false` | Enable debug logging and metrics collection |
923
- | `logLevel` | `"info"` | Log level: `error`, `warn`, `info`, `debug` |
924
- | `logSearch` | `true` | Log search operations with timing breakdown |
925
- | `logEmbedding` | `true` | Log embedding API calls (success, error, rate-limit) |
926
- | `logCache` | `true` | Log cache hits and misses |
927
- | `logGc` | `true` | Log garbage collection operations |
928
- | `debug.logBranch` | `true` | Record branch detection and switches in `index_logs` when `debug.enabled` is also `true`; never prints routine branch changes to stdout |
929
- | `metrics` | `false` | Enable metrics collection (indexing stats, search timing, cache performance) |
930
- | **effectivenessMetrics** | | |
931
- | `enabled` | `false` | Independently opt in to memory-only, fixed-cardinality repository-tool effectiveness counters. Does not enable debug logs; stores no queries, response text, code, symbols, paths, repo names, user identity, or stable identifiers. |
932
-
933
- ### Recovery warnings in debug logs
174
+ ## Branch-aware indexing
934
175
 
935
- When debug logging is enabled, the indexer now emits warn-level recovery messages if persisted cache state cannot be read safely.
176
+ The index stores reusable content by hash and maintains branch catalogs for chunks and symbols. On a branch switch, unchanged content can be reused while results remain scoped to the active branch. Linked worktrees without a local project config share the main checkout's portable project index; adding a worktree-local config creates an isolated index boundary.
936
177
 
937
- - Corrupted or unreadable `file-hashes.json` causes the in-memory file hash cache to be reset.
938
- - Corrupted or unreadable `failed-batches.json` causes persisted retry batches to be skipped for that run.
178
+ ## Knowledge bases and reranking
939
179
 
940
- These warnings improve observability but do **not** change the recovery behavior: the indexer still falls back to a safe reset/skip path instead of crashing. If these warnings recur, remove the affected file under `.opencode/index/` (or the global index directory) and rebuild with `/index force`.
180
+ OpenCode and Pi can index additional directories as knowledge bases. Configure them with `knowledgeBases` or use the host-native knowledge-base tools where available.
941
181
 
942
- ### Retrieval ranking behavior
182
+ Optional external reranking supports Cohere, Jina, and custom compatible endpoints. Local filtering and evidence classes are applied before external candidates are submitted.
943
183
 
944
- - `codebase_search` and `codebase_peek` use the hybrid path: semantic + keyword retrieval → fusion (`fusionStrategy`) → deterministic rerank (`rerankTopN`) → optional external reranker (`reranker`) → filtering.
945
- - When `search.routingHints` is enabled (default), the plugin adds a tiny, one-shot runtime hint for matching local conceptual discovery, broad repository coding tasks such as fixing or investigating code, and definition queries. A hint is emitted at most once per user message, so tool-call loops do not repeatedly add it. Conceptual and broad-task prompts are nudged toward `codebase_context` first, with `codebase_peek` / `codebase_search` for targeted follow-up, while definition questions are nudged toward `implementation_lookup`. Exact identifier and unrelated operational tasks are left alone. Set `search.routingGraphHandoffHints` to `true` to add opt-in graph/OMO CodeGraph handoff wording, and set `search.routingHintRole` to `"developer"` if your client/runtime expects developer-role guidance instead of system-role guidance.
946
- - `find_similar` stays semantic-only: semantic retrieval + deterministic rerank only (no keyword retrieval, no RRF).
947
- - For compatibility rollbacks, set `search.fusionStrategy` to `"weighted"` to use the legacy weighted fusion path.
948
- - When enabled, the external reranker sees path metadata plus a bounded on-disk code snippet for each candidate so it can distinguish real implementations from docs/tests more reliably.
949
- - Retrieval benchmark artifacts are separated by role:
950
- - baseline (versioned): `benchmarks/baselines/retrieval-baseline.json`
951
- - latest candidate run (generated): `benchmark-results/retrieval-candidate.json`
184
+ See [Configuration](docs/configuration.md) for examples and privacy considerations.
952
185
 
953
- ## šŸ“ Evaluation Harness
954
-
955
- This repository includes a first-class eval system for retrieval quality with versioned golden sets, compare mode, parameter sweeps, CI budgets, and run artifacts.
956
-
957
- ### Commands
958
-
959
- ```bash
960
- npm run eval
961
- npm run eval:ci
962
- npm run eval:ci:ollama
963
- npm run eval:compare -- --against benchmarks/baselines/eval-baseline-summary.json
964
- npm run eval:effectiveness
965
- ```
186
+ ## Troubleshooting
966
187
 
967
- CI usage split:
188
+ Start with:
968
189
 
969
- - `npm run eval:smoke`: harness smoke check with local mock embeddings (used in main CI)
970
- - `npm run eval:ci`: real quality gate against baseline/budget (for scheduled/manual quality workflow)
971
- - `npm run eval:effectiveness`: deterministic offline synthetic-fixture formatting report. Every route receives the same capped ranked result list and final-response token budget. Evidence is credited only when its literal marker is visible in returned text. The baseline emits only exact matching source lines, performs no arbitrary complete reads, makes no network calls, and writes `benchmarks/baselines/privacy-safe-effectiveness.json`.
190
+ 1. `/status` or `index_status`
191
+ 2. `index_health_check`
192
+ 3. a normal `/index` retry
193
+ 4. a forced rebuild only when status reports incompatibility or corruption
972
194
 
973
- The effectiveness report includes median/p95 `cl100k_base` token counts and final-text evidence recall. Context and peek are metadata-oriented in this benchmark, so they receive no content-evidence credit unless the marker is actually visible in their response. The exact-search snippet baseline uses oracle markers and excludes discovery cost. The report does not measure retrieval quality, latency, end-to-end agent success, causal impact, or production-repository performance.
195
+ Common provider, native module, stale index, branch, and performance issues are covered in [Troubleshooting](TROUBLESHOOTING.md).
974
196
 
975
- For `eval-quality.yml`, the default CI path uses **GitHub Models** with the workflow `GITHUB_TOKEN` plus `models: read`, so you do not need a separate OpenAI API key just to run the scheduled gate.
197
+ ## Evaluation and performance
976
198
 
977
- That default GitHub Models path uses `benchmarks/budgets/github-models.json`, which applies stable absolute thresholds instead of the stricter baseline-regression budget used for explicit external providers.
199
+ The repository includes reproducible retrieval datasets, latency and quality budgets, baseline comparison, and cross-repository benchmarking tools.
978
200
 
979
- Optional override secrets for another OpenAI-compatible endpoint:
201
+ - [Evaluation harness](docs/evaluation.md)
202
+ - [Cross-repository benchmarking](docs/benchmarking-cross-repo.md)
980
203
 
981
- - `EVAL_EMBED_BASE_URL`
982
- - `EVAL_EMBED_API_KEY`
983
- - `EVAL_EMBED_MODEL` (optional, default `text-embedding-3-small`)
984
- - `EVAL_EMBED_DIMENSIONS` (optional, default `1536`)
204
+ Performance depends on repository size, parser coverage, provider latency, embedding cache reuse, and the selected indexing limits. Prefer measured evaluation over fixed marketing claims.
985
205
 
986
- If you override the provider, set both `EVAL_EMBED_BASE_URL` and `EVAL_EMBED_API_KEY`. Otherwise the workflow falls back to GitHub Models automatically. Override providers continue to use the baseline-driven budget in `benchmarks/budgets/default.json`.
987
-
988
- No OpenAI API access? Use Ollama quality gate locally:
989
-
990
- - Config: `.github/eval-ollama-config.json`
991
- - Script: `npm run eval:ci:ollama`
992
-
993
- Prerequisites: Ollama installed, `ollama serve` running on `127.0.0.1:11434`, and `nomic-embed-text` pulled.
994
-
995
- Examples:
206
+ ## Development
996
207
 
997
208
  ```bash
998
- # Run against small golden set
999
- npm run eval -- --dataset benchmarks/golden/small.json
1000
-
1001
- # Compare against baseline
1002
- npm run eval:compare -- --against benchmarks/baselines/eval-baseline-summary.json --dataset benchmarks/golden/medium.json
1003
-
1004
- # Sweep retrieval parameters
1005
- npm run eval -- --dataset benchmarks/golden/small.json --sweepFusionStrategy rrf,weighted --sweepHybridWeight 0.3,0.5,0.7 --sweepRrfK 30,60 --sweepRerankTopN 10,20
1006
- ```
1007
-
1008
- ### What it reports
1009
-
1010
- - Hit@1, Hit@3, Hit@5, Hit@10
1011
- - MRR@10, nDCG@10
1012
- - Latency p50/p95/p99
1013
- - Token estimates, embedding call counts, estimated embedding cost
1014
- - Failure buckets (`wrong-file`, `wrong-symbol`, `docs-tests-outranking-source`, `no-relevant-hit-top-k`)
1015
-
1016
- ### Artifacts
1017
-
1018
- Each run writes:
1019
-
1020
- `benchmarks/results/<timestamp>/`
1021
-
1022
- - `summary.json`
1023
- - `summary.md`
1024
- - `per-query.json`
1025
- - `compare.json` (when baseline/sweep used)
1026
-
1027
- ### Golden sets and budgets
1028
-
1029
- - Golden datasets:
1030
- - `benchmarks/golden/small.json`
1031
- - `benchmarks/golden/medium.json`
1032
- - `benchmarks/golden/large.json`
1033
- - CI budgets:
1034
- - `benchmarks/budgets/github-models.json` for the default GitHub Models workflow path
1035
- - `benchmarks/budgets/default.json` for explicit external provider overrides with baseline comparison
1036
-
1037
- Full docs: `docs/evaluation.md`
1038
-
1039
- ### Cross-repo benchmark results snapshot
1040
-
1041
- Recent representative runs (plugin vs `ripgrep` vs `ast-grep`) on two medium repos:
1042
-
1043
- Methodology for the snapshot below:
1044
-
1045
- - Dataset: auto-generated cross-repo golden sets for `axios` + `express`
1046
- - Repeats: **20** per mode
1047
- - Aggregation: **median** metric per tool (then averaged across repos)
1048
- - Reindex behavior: when enabled, index reset applies on repeat #1 only; subsequent repeats measure warm-index query behavior
1049
- - Sampling note: repository parsing can be capped; benchmark reports include truncation metadata
1050
- - ast-grep scope note: sg metrics are computed on its compatible query subset (`definition`, `keyword-heavy`) with scoped denominators shown in run reports
1051
-
1052
- #### Without reindex (`--no-reindex`, default)
1053
-
1054
- | Metric | Plugin | ripgrep | ast-grep (5/10 queries) |
1055
- |---|---:|---:|---:|
1056
- | Hit@5 | 50% | 5% | 100% |
1057
- | MRR@10 | 0.48 | 0.04 | 0.90 |
1058
- | nDCG@10 | 0.48 | 0.08 | 0.93 |
1059
- | Latency p50 (ms) | 17.5 | 36.9 | 66.6 |
1060
- | Latency p95 (ms) | 30.9 | 44.1 | 70.7 |
1061
-
1062
- #### With reindex (`--reindex`)
1063
-
1064
- | Metric | Plugin | ripgrep | ast-grep (5/10 queries) |
1065
- |---|---:|---:|---:|
1066
- | Hit@5 | 50% | 5% | 100% |
1067
- | MRR@10 | 0.48 | 0.04 | 0.98 |
1068
- | nDCG@10 | 0.48 | 0.07 | 0.98 |
1069
- | Latency p50 (ms) | 17.1 | 35.9 | 69.1 |
1070
- | Latency p95 (ms) | 30.4 | 43.7 | 75.1 |
1071
-
1072
- ast-grep metrics are computed on its compatible query subset only (`definition` + `keyword-heavy`, 5/10 queries per repo). Plugin and ripgrep are scored on all 10 queries.
1073
-
1074
- Interpretation:
1075
-
1076
- - ast-grep dominates on its scoped subset (structural definition queries), but only handles 50% of query types. Plugin handles all query types including natural language.
1077
- - Plugin leads on rank-sensitive quality (MRR/nDCG) vs ripgrep across all query types.
1078
- - ripgrep remains a useful speed-oriented lexical baseline but has significantly lower retrieval relevance for intent-style queries.
1079
- - Plugin is the fastest tool at p50 (~17ms), ahead of ripgrep (~36ms) and ast-grep (~67ms).
1080
- - Reported numbers are rounded to avoid false precision; use report artifacts for full per-repeat audit trails.
1081
-
1082
- For reproducible setup and commands (including with/without reindex), see:
1083
-
1084
- - `docs/benchmarking-cross-repo.md`
1085
-
1086
- ### Embedding Providers
1087
- The plugin automatically detects available credentials in this order:
1088
- 1. **GitHub Copilot** (Free if you have it)
1089
- 2. **OpenAI** (Standard Embeddings)
1090
- 3. **Google** (Gemini Embeddings)
1091
- 4. **Ollama** (Local/Private - requires `nomic-embed-text`)
1092
-
1093
- You can also use **Custom** to connect any OpenAI-compatible embedding endpoint (llama.cpp, vLLM, text-embeddings-inference, LiteLLM, etc.).
1094
-
1095
- ### Rate Limits by Provider
1096
-
1097
- Each provider has different rate limits. The plugin automatically adjusts concurrency and delays:
1098
-
1099
- | Provider | Concurrency | Delay | Best For |
1100
- |----------|-------------|-------|----------|
1101
- | **GitHub Copilot** | 1 | 4s | Small codebases (<1k files) |
1102
- | **OpenAI** | 3 | 500ms | Medium codebases |
1103
- | **Google** | 5 | 200ms | Medium-large codebases |
1104
- | **Ollama** | 5 | None | Large codebases (10k+ files) |
1105
- | **Custom** | 3 | 1s | Any OpenAI-compatible endpoint |
1106
-
1107
- **For large codebases**, use Ollama locally to avoid rate limits:
1108
-
1109
- ```bash
1110
- # Install the embedding model
1111
- ollama pull nomic-embed-text
1112
- ```
1113
-
1114
- ```json
1115
- // .opencode/codebase-index.json
1116
- {
1117
- "embeddingProvider": "ollama"
1118
- }
1119
- ```
1120
-
1121
- The built-in `ollama` provider uses Ollama's native API. It discovers installed embedding-capable models and reads their vector dimensions and context length from `/api/show`.
1122
-
1123
- For the built-in Ollama path, the plugin budgets `nomic-embed-text` against an observed effective input limit of about **2048 tokens**, not the model's higher advertised theoretical context. This keeps batching and chunk text generation aligned with real Ollama embedding runtime behavior.
1124
-
1125
- To select another installed embedding model, configure it directly:
1126
-
1127
- ```json
1128
- {
1129
- "embeddingProvider": "ollama",
1130
- "embeddingModel": "qwen3-embedding:0.6b"
1131
- }
1132
- ```
1133
-
1134
- When `embeddingModel` is omitted, auto-detection selects the first installed model that advertises embedding capability. Set `OLLAMA_HOST` only to a trusted Ollama server because source text is sent to that endpoint for embedding.
1135
-
1136
- ## šŸ“ˆ Performance
1137
-
1138
- The plugin is built for speed with a Rust native module (`tree-sitter`, `usearch`, SQLite). In practice, indexing and retrieval remain fast enough for interactive use on medium/large repositories.
1139
-
1140
- - Typical query latency: ~800-1000ms (mostly embedding API time)
1141
- - Incremental indexing: only changed files are re-embedded
1142
- - Batch DB operations: significant write-speed improvements for large indexes
1143
-
1144
- For reproducible measurements on your machine, run: `npx tsx benchmarks/run.ts`.
1145
-
1146
- ## šŸŽÆ Choosing a Provider
1147
-
1148
- Quick recommendation:
1149
-
1150
- - **Want local + private + fast indexing** → use **Ollama**
1151
- - **Already have Copilot and a smaller repo** → use **GitHub Copilot**
1152
- - **General cloud setup** → use **OpenAI** or **Google**
1153
- - **Custom/OpenAI-compatible endpoint** → use **custom** provider
1154
-
1155
- ### Provider Comparison
1156
-
1157
- | Provider | Speed | Cost | Privacy | Best For |
1158
- |----------|-------|------|---------|----------|
1159
- | **Ollama** | Fastest | Free | Full | Large codebases, privacy-sensitive |
1160
- | **GitHub Copilot** | Slow (rate limited) | Free* | Cloud | Small codebases, existing subscribers |
1161
- | **OpenAI** | Medium | ~$0.0001/1K tokens | Cloud | General use |
1162
- | **Google** | Fast | Free tier available | Cloud | Medium-large codebases |
1163
- | **Custom** | Varies | Varies | Varies | Self-hosted or third-party endpoints |
1164
-
1165
- *Requires active Copilot subscription
1166
-
1167
- ### Setup by Provider
1168
-
1169
- Set the provider in `.opencode/codebase-index.json`:
1170
-
1171
- ```json
1172
- { "embeddingProvider": "ollama" }
1173
- ```
1174
-
1175
- Credentials (if required) are read from environment variables (for example `OPENAI_API_KEY` or `GOOGLE_API_KEY`).
1176
-
1177
- **Custom (OpenAI-compatible)**
1178
- Works with any server that implements the OpenAI `/v1/embeddings` API format (llama.cpp, vLLM, text-embeddings-inference, LiteLLM, etc.).
1179
- ```json
1180
- {
1181
- "embeddingProvider": "custom",
1182
- "customProvider": {
1183
- "baseUrl": "{env:EMBED_BASE_URL}",
1184
- "model": "nomic-embed-text",
1185
- "dimensions": 768,
1186
- "apiKey": "{env:EMBED_API_KEY}",
1187
- "maxTokens": 8192,
1188
- "timeoutMs": 30000,
1189
- "maxBatchSize": 64
1190
- }
1191
- }
1192
- ```
1193
- Required fields: `baseUrl`, `model`, `dimensions` (positive integer). Optional: `apiKey`, `maxTokens`, `timeoutMs` (default: 30000), `maxBatchSize` (or `max_batch_size`) to cap inputs per `/embeddings` request for servers like text-embeddings-inference. `{env:VAR_NAME}` placeholders are resolved before config validation for fields that are actually used and throw if the referenced environment variable is missing or malformed.
1194
-
1195
- **Ollama through an OpenAI-compatible proxy**
1196
- Use the `custom` provider only when Ollama is exposed through an OpenAI-compatible proxy or when you need to override metadata manually:
1197
-
1198
- ```json
1199
- {
1200
- "embeddingProvider": "custom",
1201
- "customProvider": {
1202
- "baseUrl": "http://127.0.0.1:11434/v1",
1203
- "model": "qwen3-embedding:0.6b",
1204
- "dimensions": 1024,
1205
- "apiKey": "ollama"
1206
- }
1207
- }
1208
- ```
1209
-
1210
- Notes:
1211
- - The plugin appends `/embeddings`, so `baseUrl` should be `http://127.0.0.1:11434/v1`, not just `http://127.0.0.1:11434`.
1212
- - Ollama ignores the API key, but some OpenAI-compatible clients expect one, so a placeholder like `"ollama"` is fine.
1213
- - Make sure `dimensions` matches the actual model output. The built-in `ollama` provider discovers this value automatically and is preferred for direct Ollama connections.
1214
-
1215
- ## āš ļø Tradeoffs
1216
-
1217
- Be aware of these characteristics:
1218
-
1219
- | Aspect | Reality |
1220
- |--------|---------|
1221
- | **Search latency** | ~800-1000ms per query (embedding API call) |
1222
- | **First index** | Takes time depending on codebase size (e.g., ~30s for 500 chunks) |
1223
- | **Requires API** | Needs an embedding provider (Copilot, OpenAI, Google, or local Ollama) |
1224
- | **Token costs** | Uses embedding tokens (free with Copilot, minimal with others) |
1225
- | **Best for** | Discovery and exploration, not exhaustive matching |
1226
-
1227
- ## šŸ’» Local Development
1228
-
1229
- 1. **Build**:
1230
- ```bash
1231
- npm run build
1232
- ```
1233
-
1234
- 2. **Register in Test Project** (use `file://` URL in `opencode.json`):
1235
- ```json
1236
- {
1237
- "plugin": [
1238
- "file:///path/to/opencode-codebase-index"
1239
- ]
1240
- }
1241
- ```
1242
-
1243
- This loads directly from your source directory, so changes take effect after rebuilding.
1244
-
1245
- ## šŸ¤ Contributing
1246
-
1247
- For contribution workflow, standards, and release-label requirements, see [`CONTRIBUTING.md`](./CONTRIBUTING.md).
1248
-
1249
- If you want to add support for a new language, see [`docs/adding-language-support.md`](./docs/adding-language-support.md) for the full Rust + TypeScript checklist.
1250
-
1251
- Quick path:
1252
-
1253
- 1. Fork + branch
1254
- 2. Implement + tests
1255
- 3. Run checks: `npm run build && npm run typecheck && npm run lint && npm run test:run`
1256
- 4. Open PR with a release category label
1257
-
1258
- ### Release process (structured + complete notes)
1259
-
1260
- To ensure release notes reflect all merged work, this repo uses a draft-release workflow.
1261
-
1262
- 1. **Label every PR** with at least one semantic label:
1263
- - `feature`, `bug`, `performance`, `documentation`, `dependencies`, `refactor`, `test`, `chore`
1264
- - and (when relevant) `semver:major`, `semver:minor`, or `semver:patch`
1265
- - PRs are validated by CI (`Release Label Check`) and fail if no release category label is present
1266
- 2. **Let Release Drafter build the draft notes** automatically from merged PRs on `main`.
1267
- 3. **Before publishing**:
1268
- - compare `git log --oneline vX.Y.Z..HEAD` (or the previous release tag range) against the draft release notes so the release summary covers the full shipped delta, not just the current `CHANGELOG.md` `Unreleased` section
1269
- - copy/finalize relevant highlights into `CHANGELOG.md`
1270
- - bump `package.json` version
1271
- - run: `npm run build && npm run typecheck && npm run lint && npm run test:run`
1272
- 4. **Publish release** from the draft (or via `gh release create` after reviewing draft content).
1273
-
1274
- PRs labeled `skip-changelog` are intentionally excluded from release notes.
1275
-
1276
- ### Project Structure
1277
-
1278
- ```
1279
- ā”œā”€ā”€ src/
1280
- │ ā”œā”€ā”€ index.ts # Plugin entry point
1281
- │ ā”œā”€ā”€ mcp-server.ts # MCP server (Cursor, Claude Code, Windsurf)
1282
- │ ā”œā”€ā”€ cli.ts # CLI entry for MCP stdio transport
1283
- │ ā”œā”€ā”€ config/ # Configuration schema
1284
- │ ā”œā”€ā”€ embeddings/ # Provider detection and API calls
1285
- │ ā”œā”€ā”€ indexer/ # Core indexing logic + inverted index
1286
- │ ā”œā”€ā”€ git/ # Git utilities (branch detection)
1287
- │ ā”œā”€ā”€ tools/ # OpenCode tool definitions
1288
- │ ā”œā”€ā”€ utils/ # File collection, cost estimation
1289
- │ ā”œā”€ā”€ native/ # Rust native module wrapper
1290
- │ └── watcher/ # File/git change watcher
1291
- ā”œā”€ā”€ native/
1292
- │ └── src/ # Rust: tree-sitter, usearch, xxhash, SQLite
1293
- ā”œā”€ā”€ tests/ # Unit tests (vitest)
1294
- ā”œā”€ā”€ commands/ # Slash command definitions
1295
- ā”œā”€ā”€ skill/ # Agent skill guidance
1296
- └── .github/workflows/ # CI/CD (test, build, publish)
209
+ npm ci
210
+ npm run build
211
+ npm run typecheck
212
+ npm run lint
213
+ npm run test:run
1297
214
  ```
1298
215
 
1299
- ### Native Module
216
+ Native changes require Rust and `npm run build:native`. See [Contributing](CONTRIBUTING.md), [Architecture](ARCHITECTURE.md), and [Adding language support](docs/adding-language-support.md).
1300
217
 
1301
- The Rust native module handles performance-critical operations:
1302
- - **tree-sitter**: Language-aware code parsing with JSDoc/docstring extraction
1303
- - **usearch**: High-performance vector similarity search with F16 quantization
1304
- - **SQLite**: Persistent storage for embeddings, chunks, branch catalog, symbols, and call edges
1305
- - **BM25 inverted index**: Fast keyword search for hybrid retrieval
1306
- - **Call graph extraction**: Tree-sitter query-based extraction of function calls, method calls, constructors, and imports (TypeScript/JavaScript, Python, Go, Rust, Swift, PHP, Apex, Zig, GDScript, MATLAB, Bash, C, C++, Metal)
1307
- - **xxhash**: Fast content hashing for change detection
218
+ ## Documentation
1308
219
 
1309
- Rebuild with: `npm run build:native` (requires Rust toolchain)
1310
-
1311
- ### Platform Support
1312
-
1313
- Pre-built native binaries are published for:
1314
-
1315
- | Platform | Architecture | SIMD Acceleration |
1316
- |----------|-------------|--------------------|
1317
- | macOS | x86_64 | āœ… simsimd |
1318
- | macOS | ARM64 (Apple Silicon) | āœ… simsimd |
1319
- | Linux | x86_64 (GNU) | āœ… simsimd |
1320
- | Linux | ARM64 (GNU) | āœ… simsimd |
1321
- | Windows | x86_64 (MSVC) | āŒ scalar fallback |
1322
-
1323
- Windows builds use scalar distance functions instead of SIMD — functionally identical, marginally slower for very large indexes. This is due to MSVC lacking support for certain AVX-512 intrinsics used by simsimd.
220
+ - [Installation and host setup](docs/installation.md)
221
+ - [Tools and commands](docs/tools.md)
222
+ - [Configuration](docs/configuration.md)
223
+ - [Troubleshooting](TROUBLESHOOTING.md)
224
+ - [Architecture](ARCHITECTURE.md)
225
+ - [Evaluation](docs/evaluation.md)
226
+ - [Cross-repository benchmarking](docs/benchmarking-cross-repo.md)
227
+ - [Adding language support](docs/adding-language-support.md)
228
+ - [Future `open-codebase-index` rename plan](docs/rename-to-open-codebase-index.md)
229
+ - [Contributing](CONTRIBUTING.md)
230
+ - [Security](SECURITY.md)
231
+ - [Changelog](CHANGELOG.md)
1324
232
 
1325
233
  ## License
1326
234
 
1327
- MIT
1328
-
1329
- Notices for third-party components distributed with the package are available in [`THIRD_PARTY_LICENSES.md`](./THIRD_PARTY_LICENSES.md).
235
+ MIT. See [LICENSE](LICENSE).