opencode-codebase-index 0.19.1 → 0.20.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -6,1322 +6,213 @@
6
6
  [![Build Status](https://img.shields.io/github/actions/workflow/status/Helweg/opencode-codebase-index/ci.yml?branch=main)](https://github.com/Helweg/opencode-codebase-index/actions)
7
7
  [![Node.js](https://img.shields.io/badge/node-%3E%3D20-brightgreen.svg)](https://nodejs.org/)
8
8
 
9
- > **Stop grepping for concepts. Start searching for meaning.**
9
+ > Search a codebase by meaning, then follow the result into definitions, callers, and dependency paths.
10
10
 
11
- **opencode-codebase-index** brings semantic understanding to your [OpenCode](https://opencode.ai), [Jcode](https://github.com/1jehuang/jcode), [Pi](https://pi.dev), Codex, and MCP-compatible workflows. Instead of guessing function names or grepping for keywords, ask your codebase questions in plain English.
11
+ `opencode-codebase-index` is a local semantic code index for OpenCode, Jcode, Pi, Codex, Claude Code, and other MCP clients. It combines embeddings, BM25 keyword search, branch-aware filtering, symbol lookup, and a call graph behind agent-friendly tools.
12
12
 
13
- ## šŸ“Œ Quick Navigation
13
+ ## Highlights
14
14
 
15
- - [⚔ Quick Start](#-quick-start)
16
- - [🧠 Jcode](#-jcode)
17
- - [🄧 Pi Package](#-pi-package)
18
- - [🧩 Codex Plugin](#-codex-plugin)
19
- - [🧩 Claude Code Plugin](#-claude-code-plugin)
20
- - [🌐 MCP Server (Cursor, Claude Code, Windsurf, etc.)](#-mcp-server-cursor-claude-code-windsurf-etc)
21
- - [šŸŽÆ When to Use What](#-when-to-use-what)
22
- - [🧭 OMO CodeGraph Compatibility](#-omo-codegraph-compatibility)
23
- - [🧰 Tools Available](#-tools-available)
24
- - [šŸŽ® Slash Commands](#-slash-commands)
25
- - [šŸ“š Knowledge Base](#-knowledge-base)
26
- - [šŸ”„ Reranking](#-reranking)
27
- - [āš™ļø Configuration](#ļø-configuration)
28
- - [šŸ¤ Contributing](#-contributing)
15
+ - **Semantic and hybrid retrieval** for questions where you do not know the identifier.
16
+ - **Low-token discovery** through `codebase_context` and `codebase_peek`.
17
+ - **Definition and graph navigation** through `implementation_lookup`, `call_graph`, and `call_graph_path`.
18
+ - **Incremental, branch-aware indexing** with file watching and content-hash reuse.
19
+ - **Local storage** backed by SQLite, usearch vectors, and a BM25 inverted index.
20
+ - **Multiple embedding providers**: Ollama, GitHub Copilot, OpenAI, Google, or a custom OpenAI-compatible endpoint.
21
+ - **Native parsing** for TypeScript/TSX, JavaScript/JSX, Python, Rust, Swift, Go, Java, C#, Ruby, C/C++, Metal, PHP, Apex, Bash, Zig, GDScript, MATLAB, JSON, TOML, YAML, Markdown, and HTML, with text fallback.
29
22
 
30
- ## šŸ‘‹ Choose Your Path
31
-
32
- - **I want to try it now** → go to [Quick Start](#-quick-start)
33
- - **I use Jcode** → go to [Jcode](#-jcode)
34
- - **I use Pi** → go to [Pi Package](#-pi-package)
35
- - **I use Cursor/Claude Code/Windsurf** → go to [MCP Server setup](#-mcp-server-cursor-claude-code-windsurf-etc)
36
- - **I’m comparing tools and workflows** → go to [When to Use What](#-when-to-use-what)
37
- - **I’m tuning behavior/cost/performance** → go to [Configuration](#ļø-configuration)
38
- - **I want to contribute** → go to [Contributing](#-contributing)
39
-
40
- ## šŸš€ Why Use This?
41
-
42
- - 🧠 **Semantic Search**: Finds "user authentication" logic even if the function is named `check_creds`.
43
- - ⚔ **Blazing Fast Indexing**: Powered by a Rust native module using `tree-sitter` and `usearch`. Incremental updates take milliseconds.
44
- - 🌿 **Branch-Aware**: Seamlessly handles git branch switches — reuses embeddings, filters stale results.
45
- - šŸ”’ **Privacy Focused**: Your vector index is stored locally in your project.
46
- - šŸ”Œ **Model Agnostic**: Works out-of-the-box with GitHub Copilot, OpenAI, Gemini, or local Ollama models.
47
- - 🧠 **Jcode Host**: One global MCP configuration follows each Jcode session into its active repository.
48
- - 🄧 **Pi Package**: First-class Pi extension and skill package with native tools.
49
- - 🌐 **MCP Server**: Use with Cursor, Claude Code, Windsurf, or any MCP-compatible client — index once, search from anywhere.
50
-
51
- ## ⚔ Quick Start
23
+ ## Quick start with OpenCode
52
24
 
53
25
  Requires Node.js 20 or newer.
54
26
 
55
- 1. **Install the plugin**
56
- ```bash
57
- npm install opencode-codebase-index
58
- ```
59
-
60
- 2. **Add to `opencode.json`**
61
- ```json
62
- {
63
- "plugin": ["opencode-codebase-index"]
64
- }
65
- ```
66
-
67
- 3. **Index your codebase**
68
- Run `/index` or ask the agent to index your codebase. This only needs to be done once — subsequent updates are incremental.
69
-
70
- **Recommended check:** run `/status` after the first index so you can confirm the detected provider/model before you start searching.
71
-
72
- 4. **Start Searching**
73
- Ask:
74
- > "Find the function that handles credit card validation errors"
75
-
76
- ## 🧠 Jcode
77
-
78
- Jcode v0.56.0 and newer starts non-shared MCP servers in each session's working directory. Configure the server once in `~/.jcode/mcp.json` and it will index the repository where each Jcode session is running.
79
-
80
- ```json
81
- {
82
- "servers": {
83
- "codebase-index": {
84
- "command": "npx",
85
- "args": [
86
- "-y",
87
- "--package",
88
- "opencode-codebase-index@latest",
89
- "opencode-codebase-index-mcp",
90
- "--host",
91
- "jcode"
92
- ],
93
- "env": {},
94
- "shared": false
95
- }
96
- }
97
- }
98
- ```
99
-
100
- Do not add a fixed `--project` argument. Jcode supplies the active session directory as the MCP process working directory. `shared: false` gives every repository session its own indexer process and prevents cross-repository state from being shared accidentally.
101
-
102
- Jcode uses the neutral `.codebase-index/` project storage and falls back to existing OpenCode state when present. Restart Jcode after changing `~/.jcode/mcp.json`, then use `index_codebase`, `index_status`, `codebase_peek`, or `codebase_search`.
103
-
104
- The explicit `@latest` keeps `npx` on the published package even when Jcode is opened inside an `opencode-codebase-index` source checkout. For local development of this package instead, run `npm run build:ts && npm run dev:link-mcp` first.
105
-
106
- ## 🄧 Pi Package
107
-
108
- Install as a Pi package to get first-class `codebase_search`, `index_codebase`, call graph, PR impact, and knowledge-base tools plus the `codebase-search` skill.
109
-
110
- ```bash
111
- pi install npm:opencode-codebase-index
112
- # or, for local development:
113
- pi install ./path/to/opencode-codebase-index
114
- ```
115
-
116
- Pi uses the neutral `.codebase-index/` project storage and falls back to existing OpenCode state when present.
27
+ 1. Install the package:
117
28
 
118
- The Pi extension injects lightweight routing guidance and exposes `codebase_context` as its preferred first repository tool. It routes conceptual discovery, known-symbol definitions, and dependency paths before broad shell search or file reads.
119
-
120
- ## 🧩 Codex Plugin
121
- Install once for Codex threads and get skill guidance plus MCP tools in one manifest.
122
-
123
- 1. **Add this repo as a marketplace source**
124
- ```bash
125
- codex plugin marketplace add Helweg/opencode-codebase-index
126
- ```
127
- 2. **Install the plugin**
128
29
  ```bash
129
- codex plugin add codebase-index@helweg-plugins
130
- ```
131
- 3. **Restart or open a new thread** in the target workspace.
132
- 4. Use MCP tools (`index_codebase`, `index_status`, `codebase_search`, etc.) and the `codebase-search` skill guidance.
133
-
134
- The plugin includes:
135
- - `skills/` guidance for local workflows
136
- - `hooks/hooks.json` lightweight session-start guidance
137
- - `.mcp.json` running the published `opencode-codebase-index` CLI via `npx … --host codex`, so a git marketplace install works without a local build
138
- - `.agents/plugins/marketplace.json` so this repo can act as a Codex marketplace source
139
-
140
- For local plugin development from this checkout, build and link the local MCP bin once:
141
-
142
- ```bash
143
- npm run build:ts
144
- npm run dev:link-mcp
145
- ```
146
-
147
- After that, the normal `.mcp.json` command also works when Codex starts the plugin from this repository.
148
-
149
- The native Codex plugin is important: its session hook and `codebase-search` skill tell Codex to use `index_status` and `codebase_context` before shell exploration. A bare MCP configuration does not provide the same selection reliability. Current Codex `exec` releases may cancel MCP calls in non-interactive mode while waiting for an app-tool approval; use an interactive Codex thread to approve the tool call. This is a Codex client limitation, not an MCP server failure.
150
-
151
- ## 🧩 Claude Code Plugin
152
- Install once for Claude Code sessions and get skill guidance plus MCP tools in one manifest.
153
-
154
- 1. **Add this repo as a marketplace source**
155
- ```bash
156
- /plugin marketplace add Helweg/opencode-codebase-index
157
- ```
158
- 2. **Install the plugin**
159
- ```bash
160
- /plugin install codebase-index@helweg-plugins
161
- ```
162
- 3. **Restart or open a new session** in the target workspace.
163
- 4. Use MCP tools (`index_codebase`, `index_status`, `codebase_search`, etc.) and the `codebase-search` skill guidance.
164
-
165
- The plugin includes:
166
- - `skills/` guidance for local workflows
167
- - inline `mcpServers` (in `.claude-plugin/plugin.json`) running the published `opencode-codebase-index` CLI via `npx … --host claude`, so a git marketplace install works without a local build
168
- - `.claude-plugin/marketplace.json` so this repo can act as a Claude Code marketplace source
169
-
170
- ### Provider selection notes
171
-
172
- - **Default auto-detect order:** Ollama → GitHub Copilot → OpenAI → Google
173
- - **Ollama** is the preferred zero-cost local option and works especially well for large repos:
174
-
175
- ```bash
176
- ollama pull nomic-embed-text
177
- ```
178
-
179
- ```json
180
- {
181
- "embeddingProvider": "ollama"
182
- }
183
- ```
184
-
185
- - **GitHub Copilot** is a good default if OpenCode already has Copilot auth and you prefer hosted embeddings.
186
- - **OpenAI** is a good hosted option when you want predictable API behavior and standard cloud setup.
187
- - **Google** is available if you prefer Gemini-hosted embeddings.
188
- - If `/status` reports provider or compatibility problems, follow that guidance before using `/index force`.
189
-
190
- ## 🌐 MCP Server (Cursor, Claude Code, Windsurf, etc.)
191
-
192
- Use the same semantic search from any MCP-compatible client. Index once, search from anywhere.
193
-
194
- 1. **Install dependencies**
195
- ```bash
196
- npm install opencode-codebase-index @modelcontextprotocol/sdk zod
30
+ npm install opencode-codebase-index
197
31
  ```
198
32
 
199
- 2. **Configure your MCP client**
33
+ 2. Add it to `opencode.json`:
200
34
 
201
- **Cursor** (`.cursor/mcp.json`):
202
35
  ```json
203
36
  {
204
- "mcpServers": {
205
- "codebase-index": {
206
- "command": "npx",
207
- "args": ["-y", "--package", "opencode-codebase-index", "opencode-codebase-index-mcp", "--project", "/path/to/your/project"]
208
- }
209
- }
210
- }
211
- ```
212
-
213
- **Claude Code** (`claude_desktop_config.json`):
214
- ```json
215
- {
216
- "mcpServers": {
217
- "codebase-index": {
218
- "command": "npx",
219
- "args": ["-y", "--package", "opencode-codebase-index", "opencode-codebase-index-mcp", "--project", "/path/to/your/project"]
220
- }
221
- }
37
+ "plugin": ["opencode-codebase-index"]
222
38
  }
223
39
  ```
224
40
 
225
- 3. **CLI options**
226
- ```bash
227
- npx -y --package opencode-codebase-index opencode-codebase-index-mcp --project /path/to/repo
228
- npx -y --package opencode-codebase-index opencode-codebase-index-mcp --config /path/to/config
229
- npx -y --package opencode-codebase-index opencode-codebase-index-mcp
230
- ```
231
-
232
- The MCP server exposes all 13 tools (`codebase_context`, `codebase_search`, `codebase_peek`, `find_similar`, `implementation_lookup`, `call_graph`, `call_graph_path`, `pr_impact`, `index_codebase`, `index_status`, `index_health_check`, `index_metrics`, `index_logs`) and 5 prompts (`search`, `find`, `definition`, `index`, `status`). Native OpenCode and Pi integrations expose the same `codebase_context` entry point.
233
-
234
- The tools carry self-routing descriptions so clients can choose the lightweight path without relying on separate documentation:
235
-
236
- 1. `codebase_context` as the preferred single entry point for repository questions
237
- 2. `index_status` when index readiness is unknown
238
- 3. `codebase_peek` for direct low-token conceptual discovery
239
- 4. `implementation_lookup` for direct known-symbol definition lookup
240
- 5. `codebase_search` only when full semantic content is needed
241
- 6. `grep` for exact identifiers or exhaustive matches
242
- 7. `call_graph` / `call_graph_path` for direct graph queries
243
-
244
- `codebase_context` accepts a `tokenBudget` from 128 to 4000 tokens, defaulting to 1200. The hard cap is counted with the `cl100k_base` tokenizer, including multilingual and emoji text. It returns deterministic location evidence rather than source bodies, removes overlapping same-file results, diversifies evidence across files, and distinguishes duplicates, result-limit exclusions, and token-budget omissions. Increase the budget only when broader location coverage is useful. Use `implementation_lookup`, `codebase_search`, or a targeted file read for the exact source after selecting a location.
245
-
246
- The server also publishes this workflow through the standard MCP initialization `instructions` field. Client behavior remains client-controlled: an MCP server can describe and recommend its tools, but cannot force an agent host to read server instructions or invoke a tool before filesystem search. Clients that ignore MCP instructions still receive the routing guidance in each tool description.
247
-
248
- The MCP dependencies (`@modelcontextprotocol/sdk`, `zod`) ship with the package so published `npx --package opencode-codebase-index` launches work in clean MCP clients.
249
-
250
- If you are testing the MCP command from inside this repository checkout and see `opencode-codebase-index-mcp: command not found`, run `npm run build:ts && npm run dev:link-mcp`. That adds the local bin shim expected by `npx` without changing the published MCP config.
251
-
252
- ## šŸ” See It In Action
253
-
254
- **Scenario**: You're new to a codebase and need to fix a bug in the payment flow.
255
-
256
- **Without Plugin (grep)**:
257
- - `grep "payment" .` → 500 results (too many)
258
- - `grep "card" .` → 200 results (mostly UI)
259
- - `grep "stripe" .` → 50 results (maybe?)
260
-
261
- **With `opencode-codebase-index`**:
262
- You ask: *"Where is the payment validation logic?"*
263
-
264
- Plugin returns:
265
- ```text
266
- src/services/billing.ts:45 (Class PaymentValidator)
267
- src/utils/stripe.ts:12 (Function validateCardToken)
268
- src/api/checkout.ts:89 (Route handler for /pay)
269
- ```
270
-
271
- ## šŸŽÆ When to Use What
272
-
273
- | Scenario | Tool | Why |
274
- |----------|------|-----|
275
- | Don't know the function name | `codebase_context` | Routes conceptual questions to a bounded, low-token evidence pack |
276
- | Exploring unfamiliar codebase | `codebase_context` | Available natively in OpenCode and Pi, and through MCP for other clients |
277
- | Just need to find locations | `codebase_peek` (or `codebase_context`) | Returns metadata only, saves ~90% tokens |
278
- | Need the authoritative definition site | `implementation_lookup` | Prioritizes real implementation definitions over docs/tests |
279
- | Understand code flow | `call_graph` | Find callers/callees of any function |
280
- | Trace dependency paths | `call_graph_path` | Find the shortest known call path between two symbols |
281
- | Know exact identifier | `grep` | Faster, finds all occurrences |
282
- | Need ALL matches | `grep` | Semantic returns top N only |
283
- | Mixed discovery + precision | `/find` (hybrid) | Best of both worlds |
41
+ 3. Run `/status`, then `/index`.
42
+ 4. Ask a repository question, for example:
284
43
 
285
- **Rule of thumb**: `codebase_context` to route discovery first. Then `Read` to examine exact content and `grep` for precision. For symbol-definition questions, use `implementation_lookup` first.
44
+ > Where is authentication state validated before an API request?
286
45
 
287
- ## 🧭 OMO CodeGraph Compatibility
46
+ The first index creates embeddings. Later runs reuse unchanged content and process only relevant changes.
288
47
 
289
- Recent OMO releases include a built-in CodeGraph MCP and make it part of the default agent workflow. This does **not** replace `opencode-codebase-index`; the two tools answer different first questions.
48
+ ## Choose your host
290
49
 
291
- | Need | Prefer | Why |
292
- |------|--------|-----|
293
- | Find code by intent, behavior, or natural language | `codebase_peek` / `codebase_search` | Semantic + hybrid retrieval works when you do not know exact names |
294
- | Jump to the likely implementation site | `implementation_lookup` | Definition-oriented ranking prefers source over tests/docs |
295
- | Find similar implementations or duplicate patterns | `find_similar` | Embedding similarity compares code shape and meaning |
296
- | Follow callers, callees, imports, inheritance, or implementations | OMO CodeGraph or `call_graph` | Structural graph tools are best for dependency topology |
297
- | Find a shortest known relationship chain | `call_graph_path` | Uses this plugin's indexed call edges to connect two symbols |
298
- | Include external docs, examples, or API references in discovery | `add_knowledge_base` + `codebase_search` | Knowledge bases are indexed into the same retrieval store |
50
+ | Host | Recommended integration | Storage |
51
+ |---|---|---|
52
+ | OpenCode | Native plugin | `.opencode/` |
53
+ | Jcode | Per-session MCP server | `.codebase-index/` |
54
+ | Pi | Pi package | `.codebase-index/` |
55
+ | Codex | Marketplace plugin with MCP and skill guidance | `.codebase-index/` |
56
+ | Claude Code | Marketplace plugin with MCP and skill guidance | `.claude/` |
57
+ | Cursor, Windsurf, other MCP clients | `opencode-codebase-index-mcp` | Selected by `--host`; default is OpenCode-compatible |
299
58
 
300
- Recommended OMO workflow:
59
+ See [Installation and host setup](docs/installation.md) for complete instructions.
301
60
 
302
- 1. Start broad with `codebase_context` when the prompt is conceptual, such as "where is auth enforced?" or "payment validation flow".
303
- 2. Use `implementation_lookup` once you have a symbol or concept that should resolve to a definition.
304
- 3. Use OMO CodeGraph, `call_graph`, or `call_graph_path` after locating the relevant symbol to check blast radius and dependency flow.
305
- 4. Keep `grep` for exact identifiers and exhaustive text matches.
61
+ ## Recommended workflow
306
62
 
307
- If OMO reports an uninitialized CodeGraph workspace, follow its `codegraph init` guidance. That setup is independent from this plugin's index under `.opencode/index/`, so `/index` and `codegraph init` may both be useful in the same repository.
63
+ 1. **Check readiness** with `index_status` or `/status`.
64
+ 2. **Index when needed** with `index_codebase` or `/index`.
65
+ 3. **Start repository discovery** with `codebase_context`.
66
+ 4. **Use `codebase_peek`** when you only need likely locations.
67
+ 5. **Use `implementation_lookup`** for a known symbol or definition question.
68
+ 6. **Use `codebase_search`** when you need full matching source content.
69
+ 7. **Use `grep`** for exact identifiers or exhaustive text matches.
70
+ 8. **Use call-graph tools** for callers, callees, and dependency paths.
308
71
 
309
- ## šŸ“Š Token Usage
72
+ ### Which search tool should I use?
310
73
 
311
- In our testing across open-source codebases (axios, express), we observed **up to 90% reduction in token usage** for conceptual queries like *"find the error handling middleware"*.
312
-
313
- ### Why It Saves Tokens
314
-
315
- - **Without plugin**: Agent explores files, reads code, backtracks, explores more
316
- - **With plugin**: Semantic search returns relevant code immediately → less exploration
317
-
318
- ### Key Takeaways
319
-
320
- 1. **Significant savings possible**: Up to 90% reduction in the best cases
321
- 2. **Results vary**: Savings depend on query type, codebase structure, and agent behavior
322
- 3. **Best for discovery**: Conceptual queries benefit most; exact identifier lookups should use grep
323
- 4. **Complements existing tools**: Provides a faster initial signal, doesn't replace grep/explore
324
-
325
- ### When the Plugin Helps Most
326
-
327
- - **Conceptual queries**: "Where is the authentication logic?" (no keywords to grep for)
328
- - **Unfamiliar codebases**: You don't know what to search for yet
329
- - **Large codebases**: Semantic search scales better than exhaustive exploration
330
-
331
- ## šŸ› ļø How It Works
332
-
333
- ```mermaid
334
- graph TD
335
- subgraph Indexing
336
- A[Source Code] -->|Tree-sitter| B[Semantic Chunks]
337
- B -->|Embedding Model| C[Vectors]
338
- C -->|uSearch| D[(Vector Store)]
339
- C -->|SQLite| G[(Embeddings DB)]
340
- B -->|BM25| E[(Inverted Index)]
341
- B -->|Branch Catalog| G
342
- end
343
-
344
- subgraph Searching
345
- Q[User Query] -->|Embedding Model| V[Query Vector]
346
- V -->|Cosine Similarity| D
347
- Q -->|BM25| E
348
- D --> F[Hybrid Fusion RRF/Weighted]
349
- E --> F
350
- F --> X[Deterministic Rerank]
351
- G -->|Branch + Metadata Filters| X
352
- X --> R[Ranked Results]
353
- end
354
- ```
355
-
356
- 1. **Parsing**: We use `tree-sitter` to intelligently parse your code into meaningful blocks (functions, classes, interfaces). JSDoc comments and docstrings are automatically included with their associated code.
357
-
358
- **Supported Languages (Tree-sitter semantic parsing)**: TypeScript, JavaScript, Python, Rust, Swift, Go, Java, C#, Ruby, PHP, Apex, Bash, C, C++, Metal, JSON, TOML, YAML, Zig, GDScript, MATLAB†
359
-
360
- Swift support uses [`tree-sitter-swift` 0.7.3](https://github.com/alex-pinkus/tree-sitter-swift/tree/0.7.3), distributed under the MIT license. It provides Tree-sitter syntax analysis for chunks, symbols, and calls, but does not replace SourceKit semantic analysis.
361
-
362
- † MATLAB (`.m`) is opt-in — see below.
363
-
364
- **Additional Supported Formats (line-based chunking)**: TXT, HTML, HTM, Markdown, Shell scripts
365
-
366
- #### Verified PHP 8.x compatibility
367
-
368
- PHP support uses [`tree-sitter-php` 0.24.2](https://github.com/tree-sitter/tree-sitter-php/releases/tag/v0.24.2), distributed under the MIT license. The regression fixture [`php-8-features.php`](tests/fixtures/call-graph/php-8-features.php) and the Rust and Vitest suites cover the following matrix:
369
-
370
- | Version | Verified syntax |
74
+ | Need | Tool |
371
75
  |---|---|
372
- | PHP 8.0 | attributes, named arguments, promoted properties, union types, the nullsafe operator, and `match` expressions |
373
- | PHP 8.1 | enums with constants, `readonly` properties, intersection types, and first-class callables |
374
- | PHP 8.2 | `readonly` classes and DNF types |
375
- | PHP 8.3 | typed constants and dynamic class-constant access |
376
- | PHP 8.4 | property hooks, asymmetric visibility, and unparenthesized chaining on `new` |
377
- | PHP 8.5 | attributes on constants and the pipe operator `|>` |
378
-
379
- Semantic parsing verifies chunk names and types for functions, classes, interfaces, traits, and `enum_declaration`. Methods remain inside their containing chunk rather than becoming standalone chunks.
380
-
381
- The call graph distinguishes invocations from callable references. Standalone `foo(...)`, `$object->method(...)`, and `Type::method(...)` references do not create edges, while the same callables used as direct or parenthesized `|>` operands are recorded because the pipe invokes them. Coverage also includes qualified and relative `namespace\foo()` calls, nullsafe and static calls, constructors, named arguments, `match` bodies, property hooks, and enum methods.
382
-
383
- The graph remains limited to statically resolvable named targets. Dynamic calls such as `$callable()`, `$object->$method()`, and `new $class()`, `include` and `require` expressions, and `extends` and `implements` relationships are outside this contribution.
384
-
385
- PHP 8.5 support remains partial in upstream grammar 0.24.2. Final promoted properties and `clone($object, [...])` still produce `ERROR` nodes, so this project does not claim support for those two forms.
76
+ | Route a repository question to a bounded evidence pack | `codebase_context` |
77
+ | Find likely files and symbols without source bodies | `codebase_peek` |
78
+ | Retrieve full matching code | `codebase_search` |
79
+ | Find an authoritative definition | `implementation_lookup` |
80
+ | Find analogous implementations or duplicates | `find_similar` |
81
+ | Find direct callers or callees | `call_graph` |
82
+ | Find a path between two symbols | `call_graph_path` |
83
+ | Analyze a branch or pull request blast radius | `pr_impact` |
386
84
 
387
- **Default File Patterns**:
388
- ```
389
- **/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs} **/*.{py,pyi}
390
- **/*.{go,rs,java,cs,kt,scala} **/*.{c,cpp,cc,cxx,h,hpp,hxx}
391
- **/*.{rb,php,inc,swift} **/*.{vue,svelte,astro}
392
- **/*.{sql,graphql,proto} **/*.{yaml,yml,toml}
393
- **/*.{md,mdx} **/*.{sh,bash,zsh}
394
- **/*.{txt,html,htm} **/*.{cls,trigger}
395
- **/*.zig **/*.gd
396
- **/*.metal
397
- ```
398
-
399
- Use `include` to replace defaults, or `additionalInclude` to extend (e.g. `"**/*.pdf"`, `"**/*.csv"`).
400
-
401
- †**MATLAB opt-in**: `.m` is excluded from defaults because it conflicts with the Objective-C extension used on Apple codebases. To enable MATLAB discovery, add to your global config (`~/.config/opencode/codebase-index.json`):
402
- ```json
403
- { "additionalInclude": ["**/*.m"] }
404
- ```
405
-
406
- **Max File Size**: Default 1MB (1048576 bytes). Configure via `indexing.maxFileSize` (bytes).
407
- 2. **Chunking**: Large blocks are split with overlapping windows to preserve context across chunk boundaries.
408
- 3. **Embedding**: These blocks are converted into vector representations using your configured AI provider.
409
- 4. **Storage**: Embeddings are stored in SQLite (deduplicated by content hash) and vectors in `usearch` with F16 quantization for 50% memory savings. A branch catalog tracks which chunks exist on each branch.
410
- 5. **Hybrid Search**: Combines semantic similarity (vectors) with BM25 keyword matching, applies branch and hard metadata scopes before reranking, fuses candidates (`rrf` default, `weighted` fallback), then applies deterministic local intent ranking and optional external reranking.
411
-
412
- **Performance characteristics:**
413
- - **Incremental indexing**: ~50ms check time — only re-embeds changed files
414
- - **Smart chunking**: Understands code structure to keep functions whole, with overlap for context
415
- - **Native speed**: Core logic written in Rust for maximum performance
416
- - **Memory efficient**: F16 vector quantization reduces index size by 50%
417
- - **Branch-aware**: Automatically tracks which chunks exist on each git branch
418
- - **Provider validation**: Detects embedding provider/model changes and requires rebuild to prevent garbage results
419
-
420
- ## 🌿 Branch-Aware Indexing
421
-
422
- The plugin automatically detects git branches and optimizes indexing across branch switches.
423
-
424
- ### How It Works
85
+ See [Tools and commands](docs/tools.md) for host availability, tool details, MCP prompts, and slash commands.
425
86
 
426
- When you switch branches, code changes but embeddings for unchanged content remain the same. The plugin:
87
+ ## How it works
427
88
 
428
- 1. **Stores embeddings by content hash**: Embeddings are deduplicated across branches
429
- 2. **Tracks branch membership**: A lightweight catalog tracks which chunks exist on each branch
430
- 3. **Filters search results**: Queries only return results relevant to the current branch
431
-
432
- ### Benefits
433
-
434
- | Scenario | Without Branch Awareness | With Branch Awareness |
435
- |----------|-------------------------|----------------------|
436
- | Switch to feature branch | Re-index everything | Instant — reuse existing embeddings |
437
- | Return to main | Re-index everything | Instant — catalog already exists |
438
- | Search on branch | May return stale results | Only returns current branch's code |
439
-
440
- ### Automatic Behavior
441
-
442
- - **Branch detection**: Automatically reads from `.git/HEAD`
443
- - **Re-indexing on switch**: Triggers when you switch branches (via file watcher)
444
- - **Legacy migration**: Automatically migrates old indexes on first run
445
- - **Garbage collection**: Health check removes orphaned embeddings and chunks
446
-
447
- ### Storage Structure
448
-
449
- ```
450
- .opencode/index/
451
- ā”œā”€ā”€ codebase.db # SQLite: embeddings, chunks, branch catalog, symbols, call edges
452
- ā”œā”€ā”€ vectors.usearch # Vector index (uSearch)
453
- ā”œā”€ā”€ inverted-index.json # BM25 keyword index
454
- └── file-hashes.json # File change detection
89
+ ```text
90
+ source files
91
+ │
92
+ ā”œā”€ file discovery and git-aware change detection
93
+ ā”œā”€ tree-sitter parsing and semantic chunking
94
+ ā”œā”€ embedding generation and content-hash reuse
95
+ ā–¼
96
+ SQLite metadata + usearch vectors + BM25 index
97
+ │
98
+ ā”œā”€ semantic candidates
99
+ ā”œā”€ keyword candidates
100
+ ā”œā”€ branch and request filters
101
+ ā”œā”€ deterministic fusion and ranking
102
+ ā–¼
103
+ locations, source results, definitions, and call-graph evidence
455
104
  ```
456
105
 
457
- ### File Exclusions
458
-
459
- The following files/folders are excluded from indexing by default:
460
-
461
- - **Hidden files/folders**: Files starting with `.` (e.g., `.github`, `.vscode`, `.env`)
462
- - **Build folders**: Folders containing "build" in their name (e.g., `build`, `mingwBuildDebug`, `cmake-build-debug`)
463
- - **Default excludes**: `node_modules`, `dist`, `vendor`, `__pycache__`, `target`, `coverage`, etc.
464
-
465
- ## 🧰 Tools Available
106
+ The TypeScript layer handles host integration, configuration, indexing orchestration, providers, ranking, and tools. The Rust NAPI module handles parsing, vector storage, SQLite operations, BM25 indexing, hashing, and call extraction.
466
107
 
467
- The plugin exposes these tools to the OpenCode agent:
108
+ Read [Architecture](ARCHITECTURE.md) for the detailed data flow and design decisions.
468
109
 
469
- `codebase_context` is MCP-server-only.
110
+ ## Embedding providers
470
111
 
471
- ### `codebase_context`
472
- *MCP-only entrypoint for combined routing*
473
- **Preferred first tool for repository questions.** Routes to the lowest-token indexed operation that matches the query: conceptual discovery, definition lookup, callers/callees, or symbol-to-symbol paths.
474
- - **Use for**: New questions about behavior, locating symbols, or tracing direct call relationships.
475
- - **Example**: `"Where is the payment validation logic?"`
476
- - **Workflow**: If the query is conceptual, it may return locations first. For exact behavior text, follow with `codebase_search`.
112
+ With `embeddingProvider: "auto"`, providers are tried in this order:
477
113
 
478
- ### `codebase_search`
479
- **Behavioral semantic retrieval with full content.** Searches code by describing behavior.
480
- - **Use for**: Discovery when you already want full matching snippets and are ready to inspect implementation text.
481
- - **Example**: `"find the middleware that sanitizes input"`
482
- - **Ranking path**: hybrid retrieval → branch/directory/file-type/chunk/blame scope → fusion (`search.fusionStrategy`) → deterministic local intent rerank (`search.rerankTopN`) → optional external rerank within local evidence classes → minimum-score filter
483
- - **Blame filters**: when `indexing.gitBlame.enabled` is `true`, filter with `blameAuthor`, `blameSha`, or `blameSince`.
114
+ 1. Ollama
115
+ 2. GitHub Copilot
116
+ 3. OpenAI
117
+ 4. Google
484
118
 
485
- The local ranker is deterministic and does not call another model. Exact symbol names receive a strong NFKC and case-normalized match signal. Definition and implementation questions prefer authoritative declarations over imports, export wrappers, tests, fixtures, docs, and generated or vendor files. Explicit test, docs, config, and call-flow wording instead promotes the requested evidence class. Natural-language conceptual queries keep retrieval score as the dominant signal, while nested duplicate chunks are removed and remaining evidence is spread across relevant files. True score ties retain input order, then candidate id as a final deterministic fallback.
486
-
487
- When an external reranker is enabled, hard directory, file-type, chunk-type, and blame scopes are enforced before any request. The external service receives only candidates already inside that scope and only the candidate's exact indexed line range, without extra surrounding source. Local intent classification, exact-name promotion, duplicate suppression, and evidence-class ordering remain local.
488
-
489
- **Writing good queries:**
490
-
491
- | āœ… Good queries (describe behavior) | āŒ Bad queries (too vague) |
492
- |-------------------------------------|---------------------------|
493
- | "function that validates email format" | "email" |
494
- | "error handling for failed API calls" | "error" |
495
- | "middleware that checks authentication" | "auth middleware" |
496
- | "code that calculates shipping costs" | "shipping" |
497
- | "where user permissions are checked" | "permissions" |
498
-
499
- ### `codebase_peek`
500
- **Token-efficient discovery.** Returns only metadata (file, line, name, type) without code content.
501
- - **Use for**: Finding WHERE code is before deciding what to read. Saves ~90% tokens vs `codebase_search`.
502
- - **Ranking path**: same hybrid ranking path as `codebase_search` (metadata-only output)
503
- - **Example output**:
504
- ```
505
- [1] function "validatePayment" at src/billing.ts:45-67 (score: 0.92)
506
- abc1234 | Jane Doe | 2025-03-14 | billing: validate payment state
507
- [2] class "PaymentProcessor" at src/processor.ts:12-89 (score: 0.87)
508
-
509
- Use Read tool to examine specific files.
510
- ```
511
- - **Workflow**: `codebase_peek` → find locations → `Read` specific files
512
-
513
- ### `implementation_lookup`
514
- **Definition-first lookup.** Jumps to the authoritative definition site for a symbol or natural-language definition query.
515
- - **Use for**: "Where is X defined?", symbol-definition requests, and cases where you want the implementation site rather than all usages.
516
- - **Behavior**: Prefers real implementation files over tests, docs, examples, and fixtures.
517
- - **Fallback**: If nothing authoritative is found, use `codebase_search` for broader discovery.
518
-
519
- ### `find_similar`
520
- Find code similar to a provided snippet.
521
- - **Use for**: Duplicate detection, refactor prep, pattern mining.
522
- - **Ranking path**: semantic retrieval only + deterministic rerank (no BM25, no RRF).
523
-
524
- ### `index_codebase`
525
- Manually trigger indexing.
526
- - **Use for**: Forcing a re-index or checking stats.
527
- - **Parameters**: `force` (rebuild all), `estimateOnly` (check costs), `verbose` (show skipped files and parse failures).
528
-
529
- ### `index_status`
530
- Checks if the index is ready and healthy.
531
- - **Recommended workflow**: run this after `/index` to confirm the detected provider/model and whether the index is ready to search.
532
-
533
- ### `index_health_check`
534
- Maintenance tool to remove stale entries from deleted files and orphaned embeddings/chunks from the database.
535
-
536
- ### `index_metrics`
537
- Returns collected metrics about indexing and search performance. Operational metrics require `debug.enabled` and `debug.metrics` to be `true`.
538
- - **Metrics include**: Files indexed, chunks created, cache hit rate, search timing breakdown, GC stats, embedding API call stats.
539
- - **Privacy-safe effectiveness metrics**: Set top-level `effectivenessMetrics.enabled` to `true`. This path does not enable debug logging. It records only fixed route/host/outcome counters and bounded result-count, latency, token-budget, and returned-token histograms. Counters are disabled by default, memory-only, fixed-cardinality, and process-lifetime across Indexer replacement and configuration-watcher refresh. One process-wide collector aggregates opted-in calls without project or repository identity dimensions. It never retains queries, response text, source, symbols, paths, repository names, user identity, or stable identifiers.
540
- - **Effectiveness-only privacy boundary**: With only `effectivenessMetrics.enabled` set, `debug.enabled` remains `false`, `index_logs` remains disabled, operational debug metrics remain off, and no query, path, source, secret, or response text is emitted by metrics output or written to index files. Only the bounded aggregate snapshot is returned by `index_metrics`.
541
- - **Debug logging is separate**: Explicitly enabling `debug.enabled` with `debug.logSearch` preserves the existing diagnostic behavior and may retain raw queries and repository details in memory. Do not enable debug search logs when only privacy-safe aggregates are wanted.
542
- - **Reset**: Pass `reset: true` to clear both operational and process-wide effectiveness metrics before returning the new zeroed snapshot. Process exit also clears effectiveness metrics because they are never persisted.
543
-
544
- ### `index_logs`
545
- Returns recent debug logs with optional filtering.
546
- - **Parameters**: `category` (optional: `search`, `embedding`, `cache`, `gc`, `branch`), `level` (optional: `error`, `warn`, `info`, `debug`), `limit` (default: 50).
547
-
548
- ### `call_graph`
549
-
550
- Query the call graph to find callers or callees of a function/method. Automatically built during indexing for TypeScript, JavaScript, Python, Go, Rust, Swift, PHP, Apex, Zig, GDScript, MATLAB, Bash, C, C++, and Metal.
551
-
552
- For Swift, resolution remains name-based, so overloads, extension duplicates, and protocol dispatch can remain ambiguous. Syntax alone cannot always distinguish a superclass from a protocol conformance or an enum raw-value type from a protocol. Constructor classification uses an uppercase ASCII initial heuristic while retaining the exact name for resolution. `tree-sitter-swift` 0.7.3 also has limitations around `sending`, abbreviated collection constructors such as `[Int]()`, and some generic calls after `self`, `super`, optional chaining, or chained calls. The index does not infer SourceKit semantics for these cases.
553
-
554
- - **Use for**: Understanding code flow, tracing dependencies, impact analysis.
555
- - **Parameters**: `name` (function name), `direction` (`callers` or `callees`), `filePath` (optional duplicate-name disambiguator), `symbolId` (optional backward-compatible escape hatch), `relationshipType` (optional: `Call`, `MethodCall`, `Constructor`, `Import`, `Inherits`, `Implements`). Unique names resolve automatically; ambiguous names return bounded candidate locations.
556
- - **Example**: Find who calls `validateToken` → `call_graph(name="validateToken", direction="callers")`
557
-
558
- ### `call_graph_path`
559
-
560
- Find the shortest known call-graph path between two symbols. Use it after `codebase_peek`, `implementation_lookup`, or `call_graph` identifies the important source and target names.
561
-
562
- - **Use for**: Blast-radius checks, dependency-chain discovery, explaining how one subsystem reaches another.
563
- - **Parameters**: `from` (source symbol name), `to` (target symbol name), `fromFilePath` and `toFilePath` (optional duplicate-name disambiguators), `maxDepth` (optional, default `10`).
564
- - **Example**: Trace how `createOrder` reaches `chargeCard` → `call_graph_path(from="createOrder", to="chargeCard")`
565
-
566
- ### `pr_impact`
567
- Analyzes a PR's changed files to determine impact scope within the codebase.
568
- - **Use for**: Understanding which symbols are affected by a PR, their call-graph reach, risk level, and community/cluster detection.
569
- - **Parameters**: `checkConflicts` (optional, default `false`) — when `true`, detects overlapping concurrent PRs sharing affected symbols and returns `conflictingPRs`.
570
-
571
- ### `index_visualize`
572
- Generate a self-contained temporal call graph view for browser-based exploration.
573
-
574
- - **Use for**: Onboarding, architecture walkthroughs, and understanding what moved recently before drilling into call relationships.
575
- - **What it shows**: Recent change lenses, module overview, symbol exploration, hotspots, and cycles.
576
- - **Concepts**: Modules are path-based code areas such as `src/tools` or `native`; symbols are indexed functions, classes, methods, or similar named code units; edges are caller/callee relationships.
577
- - **Parameters**: `directory` (optional folder filter), `maxNodes` (default `5000`), `includeOrphans` (include disconnected symbols).
578
- - **Output**: Writes a temporary HTML file you can open in any browser.
579
- - **Example**: `index_visualize(directory="src/tools", maxNodes=1500)`
580
-
581
- CLI shortcut after building locally:
119
+ Ollama is the simplest local option:
582
120
 
583
121
  ```bash
584
- npm run build
585
- npm run visualize
586
- npm run visualize -- native
587
- npm run visualize -- src/tools max=1000
588
- npm run visualize -- src/indexer orphans
589
- ```
590
-
591
-
592
- ### `add_knowledge_base`
593
- Add a folder as a knowledge base to be indexed alongside project code.
594
- - **Use for**: Indexing external documentation, API references, example programs.
595
- - **Parameters**: `path` (folder path, absolute or relative), `reindex` (optional, default `true`).
596
- - **Restrictions**: System directories (`/etc`, `/proc`, `/sys`, `/dev`) and sensitive home directories (`.ssh`, `.gnupg`, `.aws`, `.docker`, `.kube`) are blocked. Symlinks are resolved before validation.
597
- - **Example**: `add_knowledge_base(path="/path/to/docs")`
598
-
599
- ### `list_knowledge_bases`
600
- List all configured knowledge base folders and their status.
601
-
602
- ### `remove_knowledge_base`
603
- Remove a knowledge base folder from the index.
604
- - **Parameters**: `path` (folder path to remove), `reindex` (optional, default `false`).
605
- - **Example**: `remove_knowledge_base(path="/path/to/docs")`
606
-
607
- ## šŸŽ® Slash Commands
608
-
609
- The plugin automatically registers these slash commands:
610
-
611
- | Command | Description |
612
- | ------- | ----------- |
613
- | `/definition <query>` | **Definition Lookup**. Finds the authoritative implementation site for a symbol or concept. |
614
- | `/peek <query>` | **Quick Semantic Lookup**. Returns likely locations only, without full code content. |
615
- | `/reindex` | **Full Rebuild**. Rebuilds the codebase index from scratch. |
616
- | `/search <query>` | **Pure Semantic Search**. Best for "How does X work?" |
617
- | `/find <query>` | **Hybrid Search**. Combines semantic search + grep. Best for "Find usage of X". |
618
- | `/call-graph <query>` | **Call Graph Trace**. Find callers/callees to understand execution flow. |
619
- | `/pr-impact <PR number or branch>` | **PR Impact Analysis**. Analyze changed files, affected symbols, communities, hub nodes, and risk. |
620
- | `/visualize [directory|max=N|orphans]` | **Temporal Call Graph Visualization**. Open recent changes, module overview, symbol exploration, hotspots, and cycles in a browser-ready HTML file. |
621
- | `/index` | **Update Index**. Runs incremental indexing by default; use `/index force` for a full rebuild. |
622
- | `/status` | **Check Status**. Shows if indexed, chunk count, and provider info. |
623
-
624
- ## šŸ“š Knowledge Base
625
-
626
- The plugin can index **external documentation** alongside your project code. The indexed codebase includes:
627
-
628
- - **Project Source Code** — all code files in the current workspace
629
- - **API References** — hardware API docs, library documentation
630
- - **Usage Guides** — tutorials, how-to guides
631
- - **Example Programs** — code samples, demo projects
632
-
633
- ### Adding Knowledge Base Folders
634
-
635
- Use the built-in tools to add documentation folders:
636
-
637
- ```
638
- add_knowledge_base(path="/path/to/api-docs")
639
- add_knowledge_base(path="/path/to/examples")
640
- ```
641
-
642
- The folder will be indexed into the **same database** as your project code. All searches automatically include both sources.
643
-
644
- ### Managing Knowledge Bases
645
-
646
- ```
647
- list_knowledge_bases # Show configured knowledge bases
648
- remove_knowledge_base(path="/path/to/api-docs") # Remove a knowledge base
649
- ```
650
-
651
- ### Configuration Example
652
-
653
- Project-level config (`.opencode/codebase-index.json`):
654
- ```json
655
- {
656
- "knowledgeBases": [
657
- "/home/user/docs/esp-idf",
658
- "/home/user/docs/arduino"
659
- ]
660
- }
661
- ```
662
-
663
- Global-level config (`~/.config/opencode/codebase-index.json`):
664
- ```json
665
- {
666
- "embeddingProvider": "custom",
667
- "customProvider": {
668
- "baseUrl": "{env:EMBED_BASE_URL}",
669
- "model": "BAAI/bge-m3",
670
- "dimensions": 1024,
671
- "apiKey": "{env:EMBED_API_KEY}"
672
- }
673
- }
122
+ ollama pull nomic-embed-text
674
123
  ```
675
124
 
676
- Config merging: Global config is the base, project config overrides. Knowledge bases from both levels are merged.
677
-
678
- ### Syncing Changes
679
-
680
- - **Project code**: Auto-synced via file watcher (real-time)
681
- - **Knowledge base folders**: Manual sync — run `/index force` after changes
682
-
683
- ## šŸ”„ Reranking
684
-
685
- The plugin supports **API-based reranking** for improved search result quality. Reranking uses a cross-encoder model to rescore the top search results.
686
-
687
- ### Enable Reranking
688
-
689
- Add to your config (`.opencode/codebase-index.json` or global config):
690
-
691
125
  ```json
692
126
  {
693
- "reranker": {
694
- "enabled": true,
695
- "baseUrl": "https://api.cohere.ai/v1",
696
- "model": "rerank-v3.5",
697
- "apiKey": "{env:RERANK_API_KEY}",
698
- "topN": 20
699
- }
127
+ "embeddingProvider": "ollama"
700
128
  }
701
129
  ```
702
130
 
703
- ### Reranker Options
704
-
705
- | Option | Default | Description |
706
- |--------|---------|-------------|
707
- | `enabled` | `false` | Enable reranking |
708
- | `baseUrl` | - | Rerank API endpoint |
709
- | `model` | - | Reranking model name |
710
- | `apiKey` | - | API key (use `{env:VAR}` for security) |
711
- | `topN` | `20` | Number of top results to rerank |
712
- | `timeoutMs` | `30000` | Request timeout |
713
-
714
- ### How It Works
715
-
716
- ```
717
- Query → Embedding Search → BM25 Search → Fusion → Reranking → Results
718
- ```
719
-
720
- 1. **Embedding Search**: Semantic similarity via vector search
721
- 2. **BM25 Search**: Keyword matching via inverted index
722
- 3. **Fusion**: Combine semantic + keyword results (RRF or weighted)
723
- 4. **Reranking**: Cross-encoder rescores top N results via API
724
- 5. **Results**: Final ranked results
725
-
726
- ### Supported Reranking APIs
727
-
728
- Any OpenAI-compatible reranking endpoint. Examples:
729
- - **SiliconFlow**: `BAAI/bge-reranker-v2-m3`
730
- - **Cohere**: `rerank-english-v3.0`
731
- - **Local models**: Any server implementing `/v1/rerank` format
732
-
733
- ## āš™ļø Configuration
734
-
735
- ### Storage Paths (OpenCode + Codex + Claude + Pi)
736
- OpenCode default (existing behavior):
737
- - project config: `.opencode/codebase-index.json`
738
- - project index: `.opencode/index`
739
- - global config: `~/.config/opencode/codebase-index.json`
740
- - global index: `~/.opencode/global-index`
131
+ A custom OpenAI-compatible embeddings endpoint is also supported. Provider, indexing, search, reranking, include/exclude, knowledge-base, storage, and debug settings are documented in [Configuration](docs/configuration.md).
741
132
 
742
- Codex/Pi host mode (neutral default):
743
- - project config: `.codebase-index/config.json`
744
- - project index: `.codebase-index/index`
745
- - global config: `~/.config/codebase-index/config.json`
746
- - global index: `~/.codebase-index/global-index`
133
+ ## Configuration example
747
134
 
748
- Claude Code host mode (`--host claude`):
749
- - project config: `.claude/codebase-index.json`
750
- - project index: `.claude/index`
751
- - global config: `~/.claude/codebase-index.json`
752
- - global index: `~/.claude/global-index`
753
-
754
- Codex, Claude Code, and Pi read legacy OpenCode paths when host-native paths are absent, so existing state continues to work.
755
-
756
- Zero-config by default (uses `auto` mode). Customize in `.opencode/codebase-index.json`:
757
-
758
- ### Full Configuration Example
135
+ OpenCode project config lives at `.opencode/codebase-index.json`. Codex, Pi, and Jcode use `.codebase-index/config.json`; Claude uses `.claude/codebase-index.json`.
759
136
 
760
137
  ```json
761
138
  {
762
- // === Embedding Provider ===
763
- "embeddingProvider": "custom", // auto | github-copilot | openai | google | ollama | custom
764
- "scope": "project", // project (per-repo) | global (shared)
765
-
766
- // === Custom Embedding API (when embeddingProvider is "custom") ===
767
- "customProvider": {
768
- "baseUrl": "{env:EMBED_BASE_URL}",
769
- "model": "BAAI/bge-m3",
770
- "dimensions": 1024,
771
- "apiKey": "{env:EMBED_API_KEY}",
772
- "maxTokens": 8192, // Max tokens per input text
773
- "timeoutMs": 30000, // Request timeout (ms)
774
- "concurrency": 3, // Max concurrent requests
775
- "requestIntervalMs": 1000, // Min delay between requests (ms)
776
- "maxBatchSize": 64 // Max inputs per /embeddings request
777
- },
778
-
779
- // === File Patterns ===
780
- "include": [ // Override default include patterns
781
- "**/*.{ts,js,py,go,rs}"
782
- ],
783
- "exclude": [ // Override default exclude patterns
784
- "**/node_modules/**"
785
- ],
786
- "additionalInclude": [ // Extend defaults (not replace)
787
- "**/*.{txt,html,htm}",
788
- "**/*.pdf"
789
- ],
790
-
791
- // === Knowledge Bases ===
792
- "knowledgeBases": [ // External docs to index alongside code
793
- "/home/user/docs/esp-idf",
794
- "/home/user/docs/arduino"
795
- ],
796
-
797
- // === Indexing ===
139
+ "embeddingProvider": "auto",
140
+ "scope": "project",
798
141
  "indexing": {
799
- "autoIndex": false, // Explicit opt-in for automatic indexing
800
- "autoIndexWaitMs": 10000, // First-retrieval wait bound (0-60000ms)
801
- "autoIndexMaxRetries": 5, // Transient interprocess lock retries (0-10)
802
- "autoIndexRetryDelayMs": 100, // Initial exponential lock retry delay
803
- "watchFiles": true, // Re-index on file changes
804
- "maxFileSize": 1048576, // Max file size in bytes (default: 1MB)
805
- "maxChunksPerFile": 100, // Max chunks per file
806
- "semanticOnly": false, // Only index functions/classes (skip blocks)
807
- "retries": 3, // Embedding API retry attempts
808
- "retryDelayMs": 1000, // Delay between retries (ms)
809
- "autoGc": true, // Auto garbage collection
810
- "gcIntervalDays": 7, // GC interval (days)
811
- "gcOrphanThreshold": 100, // GC trigger threshold
812
- "requireProjectMarker": true, // Require .git/package.json to index
813
- "maxDepth": 5, // Max directory depth (-1=unlimited, 0=root only)
814
- "maxFilesPerDirectory": 100, // Max files per directory (smallest first)
815
- "fallbackToTextOnMaxChunks": true // Fallback to text chunking on maxChunksPerFile
142
+ "autoIndex": false,
143
+ "watchFiles": true,
144
+ "requireProjectMarker": true,
145
+ "semanticOnly": false
816
146
  },
817
-
818
- // === Search ===
819
147
  "search": {
820
- "maxResults": 20, // Max results to return
821
- "minScore": 0.1, // Min similarity score (0-1)
822
- "hybridWeight": 0.5, // Keyword (1.0) vs semantic (0.0)
823
- "fusionStrategy": "rrf", // rrf | weighted
824
- "rrfK": 60, // RRF smoothing constant
825
- "rerankTopN": 20, // Deterministic rerank depth
826
- "contextLines": 0, // Extra lines before/after match
827
- "routingHints": true, // Runtime nudges for local discovery/definition queries
828
- "routingGraphHandoffHints": false, // Add opt-in graph/OMO CodeGraph handoff wording
829
- "routingHintRole": "system" // system | developer (message role used for hints)
830
- },
831
- "reranker": {
832
- "enabled": false,
833
- "provider": "cohere",
834
- "model": "rerank-v3.5",
835
- "apiKey": "{env:RERANK_API_KEY}",
836
- "topN": 15,
837
- "timeoutMs": 10000
838
- },
839
- "debug": {
840
- "enabled": false, // Enable debug logging
841
- "logLevel": "info", // error | warn | info | debug
842
- "logSearch": true, // Log search operations
843
- "logEmbedding": true, // Log embedding API calls
844
- "logCache": true, // Log cache hits/misses
845
- "logGc": true, // Log garbage collection
846
- "logBranch": true, // Log branch detection
847
- "metrics": false // Enable operational metrics collection
848
- },
849
- "effectivenessMetrics": {
850
- "enabled": false // Opt in without enabling debug logs
148
+ "maxResults": 20,
149
+ "minScore": 0.1,
150
+ "fusionStrategy": "rrf",
151
+ "rerankTopN": 20
851
152
  }
852
153
  }
853
154
  ```
854
155
 
855
- String values in `codebase-index.json` can reference environment variables with `{env:VAR_NAME}` when the placeholder is the entire string value. Variable names must match `[A-Z_][A-Z0-9_]*`. This is useful for secrets such as custom provider API keys so they do not need to be committed to the config file.
856
-
857
- ```json
858
- {
859
- "embeddingProvider": "custom",
860
- "customProvider": {
861
- "baseUrl": "{env:EMBED_BASE_URL}",
862
- "model": "nomic-embed-text",
863
- "dimensions": 768,
864
- "apiKey": "{env:EMBED_API_KEY}"
865
- }
866
- }
867
- ```
868
-
869
- ### Options Reference
870
-
871
- `indexing.autoIndex` remains disabled by default because indexing can invoke a paid embedding provider. When explicitly enabled, MCP hosts start one process-scoped job before accepting tool work where practical, skip a healthy current index, retry transient multiprocess lock contention within the configured bound, and expose sanitized state/progress through `index_status`. If no readable index exists, retrieval tools wait up to `autoIndexWaitMs`; after that they report that indexing is still running or failed instead of returning misleading empty search results. Home-directory and project-marker protections still apply.
872
-
873
- | Option | Default | Description |
874
- |--------|---------|-------------|
875
- | `embeddingProvider` | `"auto"` | Which AI to use: `auto`, `github-copilot`, `openai`, `google`, `ollama`, `custom` |
876
- | `scope` | `"project"` | `project` = index per repo, `global` = shared index across repos |
877
- | `include` | (defaults) | Override the default include patterns (replaces defaults) |
878
- | `exclude` | (defaults) | Override the default exclude patterns (replaces defaults) |
879
- | `additionalInclude` | `[]` | Additional file patterns to include (extends defaults, e.g. `"**/*.txt"`, `"**/*.html"`) |
880
- | `knowledgeBases` | `[]` | External directories to index as knowledge bases (absolute or relative paths) |
881
- | **indexing** | | |
882
- | `autoIndex` | `false` | Explicitly opt in to automatic startup/first-use indexing. When disabled, retrieval never starts indexing or paid embedding work. |
883
- | `autoIndexWaitMs` | `10000` | Maximum time (0-60000ms) a first retrieval waits for an enabled automatic index job before returning actionable in-progress status. |
884
- | `autoIndexMaxRetries` | `5` | Maximum transient interprocess lock retries (0-10) for background automatic indexing. |
885
- | `autoIndexRetryDelayMs` | `100` | Initial exponential lock retry delay in milliseconds (10-10000). |
886
- | `watchFiles` | `true` | Re-index when files change |
887
- | `maxFileSize` | `1048576` | Skip files larger than this (bytes). Default: 1MB |
888
- | `maxChunksPerFile` | `100` | Maximum chunks to index per file (controls token costs for large files) |
889
- | `semanticOnly` | `false` | When `true`, only index semantic nodes (functions, classes) and skip generic blocks |
890
- | `retries` | `3` | Number of retry attempts for failed embedding API calls |
891
- | `retryDelayMs` | `1000` | Delay between retries in milliseconds |
892
- | `autoGc` | `true` | Automatically run garbage collection to remove orphaned embeddings/chunks |
893
- | `gcIntervalDays` | `7` | Run GC on initialization if last GC was more than N days ago |
894
- | `gcOrphanThreshold` | `100` | Run GC after indexing if orphan count exceeds this threshold |
895
- | `requireProjectMarker` | `true` | Require a project marker (`.git`, `package.json`, etc.) to enable file watching and auto-indexing. Prevents accidentally indexing large directories like home. Set to `false` to index any directory. |
896
- | `maxDepth` | `5` | Max directory traversal depth. `-1` = unlimited, `0` = only files in root dir, `1` = one level of subdirectories, etc. |
897
- | `maxFilesPerDirectory` | `100` | Max files to index per directory. Always picks the smallest files first. |
898
- | `fallbackToTextOnMaxChunks` | `true` | When a file exceeds `maxChunksPerFile`, fallback to text-based (line-by-line) chunking instead of skipping the rest of the file. |
899
- | `gitBlame.enabled` | `false` | Annotate changed chunks with `git blame` commit SHA, author, author email, commit timestamp, and summary. Enables `blameAuthor`, `blameSha`, and `blameSince` filters. |
900
- | **search** | | |
901
- | `maxResults` | `20` | Maximum results to return |
902
- | `minScore` | `0.1` | Minimum similarity score (0-1). Lower = more results |
903
- | `hybridWeight` | `0.5` | Balance between keyword (1.0) and semantic (0.0) search |
904
- | `fusionStrategy` | `"rrf"` | Hybrid fusion mode: `"rrf"` (rank-based reciprocal rank fusion) or `"weighted"` (legacy score blending fallback) |
905
- | `rrfK` | `60` | RRF smoothing constant. Higher values flatten rank impact, lower values prioritize top-ranked candidates more strongly |
906
- | `rerankTopN` | `20` | Deterministic rerank depth cap. Applies lightweight name/path/chunk-type rerank to top-N only |
907
- | `contextLines` | `0` | Extra lines to include before/after each match |
908
- | `routingHints` | `true` | Inject lightweight runtime hints for local conceptual discovery and definition lookups. Set to `false` to disable plugin-side routing nudges. |
909
- | `routingGraphHandoffHints` | `false` | When `true`, conceptual discovery hints also say to use graph tools (including OMO CodeGraph) after semantic discovery identifies relevant symbols. |
910
- | `routingHintRole` | `"system"` | Message role used when injecting routing hints: `"system"` (default) or `"developer"`. |
911
- | **reranker** | | Optional second-stage model reranker for the top candidate pool |
912
- | `enabled` | `false` | Turn external reranking on/off |
913
- | `provider` | `"custom"` | Hosted shortcuts: `cohere`, `jina`, or `custom` |
914
- | `model` | — | Reranker model name required when enabled |
915
- | `baseUrl` | provider default | Override reranker endpoint base URL. `cohere` → `https://api.cohere.ai/v1`, `jina` → `https://api.jina.ai/v1` |
916
- | `apiKey` | — | API key for hosted reranker providers |
917
- | `topN` | `15` | Number of top candidates to send to the external reranker |
918
- | `timeoutMs` | `10000` | Timeout for external rerank requests |
919
- | **debug** | | |
920
- | `enabled` | `false` | Enable debug logging and metrics collection |
921
- | `logLevel` | `"info"` | Log level: `error`, `warn`, `info`, `debug` |
922
- | `logSearch` | `true` | Log search operations with timing breakdown |
923
- | `logEmbedding` | `true` | Log embedding API calls (success, error, rate-limit) |
924
- | `logCache` | `true` | Log cache hits and misses |
925
- | `logGc` | `true` | Log garbage collection operations |
926
- | `logBranch` | `true` | Log branch detection and switches |
927
- | `metrics` | `false` | Enable metrics collection (indexing stats, search timing, cache performance) |
928
- | **effectivenessMetrics** | | |
929
- | `enabled` | `false` | Independently opt in to memory-only, fixed-cardinality repository-tool effectiveness counters. Does not enable debug logs; stores no queries, response text, code, symbols, paths, repo names, user identity, or stable identifiers. |
930
-
931
- ### Recovery warnings in debug logs
932
-
933
- When debug logging is enabled, the indexer now emits warn-level recovery messages if persisted cache state cannot be read safely.
934
-
935
- - Corrupted or unreadable `file-hashes.json` causes the in-memory file hash cache to be reset.
936
- - Corrupted or unreadable `failed-batches.json` causes persisted retry batches to be skipped for that run.
937
-
938
- These warnings improve observability but do **not** change the recovery behavior: the indexer still falls back to a safe reset/skip path instead of crashing. If these warnings recur, remove the affected file under `.opencode/index/` (or the global index directory) and rebuild with `/index force`.
939
-
940
- ### Retrieval ranking behavior
941
-
942
- - `codebase_search` and `codebase_peek` use the hybrid path: semantic + keyword retrieval → fusion (`fusionStrategy`) → deterministic rerank (`rerankTopN`) → optional external reranker (`reranker`) → filtering.
943
- - When `search.routingHints` is enabled (default), the plugin adds tiny per-turn runtime hints for local conceptual discovery and definition queries. Conceptual discovery is nudged toward `codebase_peek` / `codebase_search`, while definition questions are nudged toward `implementation_lookup`. Exact identifier and unrelated operational tasks are left alone. Set `search.routingGraphHandoffHints` to `true` to add opt-in graph/OMO CodeGraph handoff wording, and set `search.routingHintRole` to `"developer"` if your client/runtime expects developer-role guidance instead of system-role guidance.
944
- - `find_similar` stays semantic-only: semantic retrieval + deterministic rerank only (no keyword retrieval, no RRF).
945
- - For compatibility rollbacks, set `search.fusionStrategy` to `"weighted"` to use the legacy weighted fusion path.
946
- - When enabled, the external reranker sees path metadata plus a bounded on-disk code snippet for each candidate so it can distinguish real implementations from docs/tests more reliably.
947
- - Retrieval benchmark artifacts are separated by role:
948
- - baseline (versioned): `benchmarks/baselines/retrieval-baseline.json`
949
- - latest candidate run (generated): `benchmark-results/retrieval-candidate.json`
950
-
951
- ## šŸ“ Evaluation Harness
952
-
953
- This repository includes a first-class eval system for retrieval quality with versioned golden sets, compare mode, parameter sweeps, CI budgets, and run artifacts.
954
-
955
- ### Commands
956
-
957
- ```bash
958
- npm run eval
959
- npm run eval:ci
960
- npm run eval:ci:ollama
961
- npm run eval:compare -- --against benchmarks/baselines/eval-baseline-summary.json
962
- npm run eval:effectiveness
963
- ```
964
-
965
- CI usage split:
966
-
967
- - `npm run eval:smoke`: harness smoke check with local mock embeddings (used in main CI)
968
- - `npm run eval:ci`: real quality gate against baseline/budget (for scheduled/manual quality workflow)
969
- - `npm run eval:effectiveness`: deterministic offline synthetic-fixture formatting report. Every route receives the same capped ranked result list and final-response token budget. Evidence is credited only when its literal marker is visible in returned text. The baseline emits only exact matching source lines, performs no arbitrary complete reads, makes no network calls, and writes `benchmarks/baselines/privacy-safe-effectiveness.json`.
970
-
971
- The effectiveness report includes median/p95 `cl100k_base` token counts and final-text evidence recall. Context and peek are metadata-oriented in this benchmark, so they receive no content-evidence credit unless the marker is actually visible in their response. The exact-search snippet baseline uses oracle markers and excludes discovery cost. The report does not measure retrieval quality, latency, end-to-end agent success, causal impact, or production-repository performance.
972
-
973
- For `eval-quality.yml`, the default CI path uses **GitHub Models** with the workflow `GITHUB_TOKEN` plus `models: read`, so you do not need a separate OpenAI API key just to run the scheduled gate.
974
-
975
- That default GitHub Models path uses `benchmarks/budgets/github-models.json`, which applies stable absolute thresholds instead of the stricter baseline-regression budget used for explicit external providers.
976
-
977
- Optional override secrets for another OpenAI-compatible endpoint:
978
-
979
- - `EVAL_EMBED_BASE_URL`
980
- - `EVAL_EMBED_API_KEY`
981
- - `EVAL_EMBED_MODEL` (optional, default `text-embedding-3-small`)
982
- - `EVAL_EMBED_DIMENSIONS` (optional, default `1536`)
983
-
984
- If you override the provider, set both `EVAL_EMBED_BASE_URL` and `EVAL_EMBED_API_KEY`. Otherwise the workflow falls back to GitHub Models automatically. Override providers continue to use the baseline-driven budget in `benchmarks/budgets/default.json`.
985
-
986
- No OpenAI API access? Use Ollama quality gate locally:
987
-
988
- - Config: `.github/eval-ollama-config.json`
989
- - Script: `npm run eval:ci:ollama`
990
-
991
- Prerequisites: Ollama installed, `ollama serve` running on `127.0.0.1:11434`, and `nomic-embed-text` pulled.
992
-
993
- Examples:
994
-
995
- ```bash
996
- # Run against small golden set
997
- npm run eval -- --dataset benchmarks/golden/small.json
998
-
999
- # Compare against baseline
1000
- npm run eval:compare -- --against benchmarks/baselines/eval-baseline-summary.json --dataset benchmarks/golden/medium.json
1001
-
1002
- # Sweep retrieval parameters
1003
- npm run eval -- --dataset benchmarks/golden/small.json --sweepFusionStrategy rrf,weighted --sweepHybridWeight 0.3,0.5,0.7 --sweepRrfK 30,60 --sweepRerankTopN 10,20
1004
- ```
1005
-
1006
- ### What it reports
1007
-
1008
- - Hit@1, Hit@3, Hit@5, Hit@10
1009
- - MRR@10, nDCG@10
1010
- - Latency p50/p95/p99
1011
- - Token estimates, embedding call counts, estimated embedding cost
1012
- - Failure buckets (`wrong-file`, `wrong-symbol`, `docs-tests-outranking-source`, `no-relevant-hit-top-k`)
1013
-
1014
- ### Artifacts
1015
-
1016
- Each run writes:
156
+ Only specify values you want to override. See [Configuration](docs/configuration.md) for defaults and host-specific paths.
1017
157
 
1018
- `benchmarks/results/<timestamp>/`
158
+ ## Branch-aware indexing
1019
159
 
1020
- - `summary.json`
1021
- - `summary.md`
1022
- - `per-query.json`
1023
- - `compare.json` (when baseline/sweep used)
160
+ The index stores reusable content by hash and maintains branch catalogs for chunks and symbols. On a branch switch, unchanged content can be reused while results remain scoped to the active branch. Project indexes are kept separate for linked worktrees, while configuration may fall back to the main repository.
1024
161
 
1025
- ### Golden sets and budgets
162
+ ## Knowledge bases and reranking
1026
163
 
1027
- - Golden datasets:
1028
- - `benchmarks/golden/small.json`
1029
- - `benchmarks/golden/medium.json`
1030
- - `benchmarks/golden/large.json`
1031
- - CI budgets:
1032
- - `benchmarks/budgets/github-models.json` for the default GitHub Models workflow path
1033
- - `benchmarks/budgets/default.json` for explicit external provider overrides with baseline comparison
164
+ OpenCode and Pi can index additional directories as knowledge bases. Configure them with `knowledgeBases` or use the host-native knowledge-base tools where available.
1034
165
 
1035
- Full docs: `docs/evaluation.md`
166
+ Optional external reranking supports Cohere, Jina, and custom compatible endpoints. Local filtering and evidence classes are applied before external candidates are submitted.
1036
167
 
1037
- ### Cross-repo benchmark results snapshot
168
+ See [Configuration](docs/configuration.md) for examples and privacy considerations.
1038
169
 
1039
- Recent representative runs (plugin vs `ripgrep` vs `ast-grep`) on two medium repos:
170
+ ## Troubleshooting
1040
171
 
1041
- Methodology for the snapshot below:
172
+ Start with:
1042
173
 
1043
- - Dataset: auto-generated cross-repo golden sets for `axios` + `express`
1044
- - Repeats: **20** per mode
1045
- - Aggregation: **median** metric per tool (then averaged across repos)
1046
- - Reindex behavior: when enabled, index reset applies on repeat #1 only; subsequent repeats measure warm-index query behavior
1047
- - Sampling note: repository parsing can be capped; benchmark reports include truncation metadata
1048
- - ast-grep scope note: sg metrics are computed on its compatible query subset (`definition`, `keyword-heavy`) with scoped denominators shown in run reports
174
+ 1. `/status` or `index_status`
175
+ 2. `index_health_check`
176
+ 3. a normal `/index` retry
177
+ 4. a forced rebuild only when status reports incompatibility or corruption
1049
178
 
1050
- #### Without reindex (`--no-reindex`, default)
179
+ Common provider, native module, stale index, branch, and performance issues are covered in [Troubleshooting](TROUBLESHOOTING.md).
1051
180
 
1052
- | Metric | Plugin | ripgrep | ast-grep (5/10 queries) |
1053
- |---|---:|---:|---:|
1054
- | Hit@5 | 50% | 5% | 100% |
1055
- | MRR@10 | 0.48 | 0.04 | 0.90 |
1056
- | nDCG@10 | 0.48 | 0.08 | 0.93 |
1057
- | Latency p50 (ms) | 17.5 | 36.9 | 66.6 |
1058
- | Latency p95 (ms) | 30.9 | 44.1 | 70.7 |
181
+ ## Evaluation and performance
1059
182
 
1060
- #### With reindex (`--reindex`)
183
+ The repository includes reproducible retrieval datasets, latency and quality budgets, baseline comparison, and cross-repository benchmarking tools.
1061
184
 
1062
- | Metric | Plugin | ripgrep | ast-grep (5/10 queries) |
1063
- |---|---:|---:|---:|
1064
- | Hit@5 | 50% | 5% | 100% |
1065
- | MRR@10 | 0.48 | 0.04 | 0.98 |
1066
- | nDCG@10 | 0.48 | 0.07 | 0.98 |
1067
- | Latency p50 (ms) | 17.1 | 35.9 | 69.1 |
1068
- | Latency p95 (ms) | 30.4 | 43.7 | 75.1 |
185
+ - [Evaluation harness](docs/evaluation.md)
186
+ - [Cross-repository benchmarking](docs/benchmarking-cross-repo.md)
1069
187
 
1070
- ast-grep metrics are computed on its compatible query subset only (`definition` + `keyword-heavy`, 5/10 queries per repo). Plugin and ripgrep are scored on all 10 queries.
188
+ Performance depends on repository size, parser coverage, provider latency, embedding cache reuse, and the selected indexing limits. Prefer measured evaluation over fixed marketing claims.
1071
189
 
1072
- Interpretation:
1073
-
1074
- - ast-grep dominates on its scoped subset (structural definition queries), but only handles 50% of query types. Plugin handles all query types including natural language.
1075
- - Plugin leads on rank-sensitive quality (MRR/nDCG) vs ripgrep across all query types.
1076
- - ripgrep remains a useful speed-oriented lexical baseline but has significantly lower retrieval relevance for intent-style queries.
1077
- - Plugin is the fastest tool at p50 (~17ms), ahead of ripgrep (~36ms) and ast-grep (~67ms).
1078
- - Reported numbers are rounded to avoid false precision; use report artifacts for full per-repeat audit trails.
1079
-
1080
- For reproducible setup and commands (including with/without reindex), see:
1081
-
1082
- - `docs/benchmarking-cross-repo.md`
1083
-
1084
- ### Embedding Providers
1085
- The plugin automatically detects available credentials in this order:
1086
- 1. **GitHub Copilot** (Free if you have it)
1087
- 2. **OpenAI** (Standard Embeddings)
1088
- 3. **Google** (Gemini Embeddings)
1089
- 4. **Ollama** (Local/Private - requires `nomic-embed-text`)
1090
-
1091
- You can also use **Custom** to connect any OpenAI-compatible embedding endpoint (llama.cpp, vLLM, text-embeddings-inference, LiteLLM, etc.).
1092
-
1093
- ### Rate Limits by Provider
1094
-
1095
- Each provider has different rate limits. The plugin automatically adjusts concurrency and delays:
1096
-
1097
- | Provider | Concurrency | Delay | Best For |
1098
- |----------|-------------|-------|----------|
1099
- | **GitHub Copilot** | 1 | 4s | Small codebases (<1k files) |
1100
- | **OpenAI** | 3 | 500ms | Medium codebases |
1101
- | **Google** | 5 | 200ms | Medium-large codebases |
1102
- | **Ollama** | 5 | None | Large codebases (10k+ files) |
1103
- | **Custom** | 3 | 1s | Any OpenAI-compatible endpoint |
1104
-
1105
- **For large codebases**, use Ollama locally to avoid rate limits:
190
+ ## Development
1106
191
 
1107
192
  ```bash
1108
- # Install the embedding model
1109
- ollama pull nomic-embed-text
1110
- ```
1111
-
1112
- ```json
1113
- // .opencode/codebase-index.json
1114
- {
1115
- "embeddingProvider": "ollama"
1116
- }
1117
- ```
1118
-
1119
- The built-in `ollama` provider uses Ollama's native API. It discovers installed embedding-capable models and reads their vector dimensions and context length from `/api/show`.
1120
-
1121
- For the built-in Ollama path, the plugin budgets `nomic-embed-text` against an observed effective input limit of about **2048 tokens**, not the model's higher advertised theoretical context. This keeps batching and chunk text generation aligned with real Ollama embedding runtime behavior.
1122
-
1123
- To select another installed embedding model, configure it directly:
1124
-
1125
- ```json
1126
- {
1127
- "embeddingProvider": "ollama",
1128
- "embeddingModel": "qwen3-embedding:0.6b"
1129
- }
1130
- ```
1131
-
1132
- When `embeddingModel` is omitted, auto-detection selects the first installed model that advertises embedding capability. Set `OLLAMA_HOST` only to a trusted Ollama server because source text is sent to that endpoint for embedding.
1133
-
1134
- ## šŸ“ˆ Performance
1135
-
1136
- The plugin is built for speed with a Rust native module (`tree-sitter`, `usearch`, SQLite). In practice, indexing and retrieval remain fast enough for interactive use on medium/large repositories.
1137
-
1138
- - Typical query latency: ~800-1000ms (mostly embedding API time)
1139
- - Incremental indexing: only changed files are re-embedded
1140
- - Batch DB operations: significant write-speed improvements for large indexes
1141
-
1142
- For reproducible measurements on your machine, run: `npx tsx benchmarks/run.ts`.
1143
-
1144
- ## šŸŽÆ Choosing a Provider
1145
-
1146
- Quick recommendation:
1147
-
1148
- - **Want local + private + fast indexing** → use **Ollama**
1149
- - **Already have Copilot and a smaller repo** → use **GitHub Copilot**
1150
- - **General cloud setup** → use **OpenAI** or **Google**
1151
- - **Custom/OpenAI-compatible endpoint** → use **custom** provider
1152
-
1153
- ### Provider Comparison
1154
-
1155
- | Provider | Speed | Cost | Privacy | Best For |
1156
- |----------|-------|------|---------|----------|
1157
- | **Ollama** | Fastest | Free | Full | Large codebases, privacy-sensitive |
1158
- | **GitHub Copilot** | Slow (rate limited) | Free* | Cloud | Small codebases, existing subscribers |
1159
- | **OpenAI** | Medium | ~$0.0001/1K tokens | Cloud | General use |
1160
- | **Google** | Fast | Free tier available | Cloud | Medium-large codebases |
1161
- | **Custom** | Varies | Varies | Varies | Self-hosted or third-party endpoints |
1162
-
1163
- *Requires active Copilot subscription
1164
-
1165
- ### Setup by Provider
1166
-
1167
- Set the provider in `.opencode/codebase-index.json`:
1168
-
1169
- ```json
1170
- { "embeddingProvider": "ollama" }
1171
- ```
1172
-
1173
- Credentials (if required) are read from environment variables (for example `OPENAI_API_KEY` or `GOOGLE_API_KEY`).
1174
-
1175
- **Custom (OpenAI-compatible)**
1176
- Works with any server that implements the OpenAI `/v1/embeddings` API format (llama.cpp, vLLM, text-embeddings-inference, LiteLLM, etc.).
1177
- ```json
1178
- {
1179
- "embeddingProvider": "custom",
1180
- "customProvider": {
1181
- "baseUrl": "{env:EMBED_BASE_URL}",
1182
- "model": "nomic-embed-text",
1183
- "dimensions": 768,
1184
- "apiKey": "{env:EMBED_API_KEY}",
1185
- "maxTokens": 8192,
1186
- "timeoutMs": 30000,
1187
- "maxBatchSize": 64
1188
- }
1189
- }
1190
- ```
1191
- Required fields: `baseUrl`, `model`, `dimensions` (positive integer). Optional: `apiKey`, `maxTokens`, `timeoutMs` (default: 30000), `maxBatchSize` (or `max_batch_size`) to cap inputs per `/embeddings` request for servers like text-embeddings-inference. `{env:VAR_NAME}` placeholders are resolved before config validation for fields that are actually used and throw if the referenced environment variable is missing or malformed.
1192
-
1193
- **Ollama through an OpenAI-compatible proxy**
1194
- Use the `custom` provider only when Ollama is exposed through an OpenAI-compatible proxy or when you need to override metadata manually:
1195
-
1196
- ```json
1197
- {
1198
- "embeddingProvider": "custom",
1199
- "customProvider": {
1200
- "baseUrl": "http://127.0.0.1:11434/v1",
1201
- "model": "qwen3-embedding:0.6b",
1202
- "dimensions": 1024,
1203
- "apiKey": "ollama"
1204
- }
1205
- }
1206
- ```
1207
-
1208
- Notes:
1209
- - The plugin appends `/embeddings`, so `baseUrl` should be `http://127.0.0.1:11434/v1`, not just `http://127.0.0.1:11434`.
1210
- - Ollama ignores the API key, but some OpenAI-compatible clients expect one, so a placeholder like `"ollama"` is fine.
1211
- - Make sure `dimensions` matches the actual model output. The built-in `ollama` provider discovers this value automatically and is preferred for direct Ollama connections.
1212
-
1213
- ## āš ļø Tradeoffs
1214
-
1215
- Be aware of these characteristics:
1216
-
1217
- | Aspect | Reality |
1218
- |--------|---------|
1219
- | **Search latency** | ~800-1000ms per query (embedding API call) |
1220
- | **First index** | Takes time depending on codebase size (e.g., ~30s for 500 chunks) |
1221
- | **Requires API** | Needs an embedding provider (Copilot, OpenAI, Google, or local Ollama) |
1222
- | **Token costs** | Uses embedding tokens (free with Copilot, minimal with others) |
1223
- | **Best for** | Discovery and exploration, not exhaustive matching |
1224
-
1225
- ## šŸ’» Local Development
1226
-
1227
- 1. **Build**:
1228
- ```bash
1229
- npm run build
1230
- ```
1231
-
1232
- 2. **Register in Test Project** (use `file://` URL in `opencode.json`):
1233
- ```json
1234
- {
1235
- "plugin": [
1236
- "file:///path/to/opencode-codebase-index"
1237
- ]
1238
- }
1239
- ```
1240
-
1241
- This loads directly from your source directory, so changes take effect after rebuilding.
1242
-
1243
- ## šŸ¤ Contributing
1244
-
1245
- For contribution workflow, standards, and release-label requirements, see [`CONTRIBUTING.md`](./CONTRIBUTING.md).
1246
-
1247
- If you want to add support for a new language, see [`docs/adding-language-support.md`](./docs/adding-language-support.md) for the full Rust + TypeScript checklist.
1248
-
1249
- Quick path:
1250
-
1251
- 1. Fork + branch
1252
- 2. Implement + tests
1253
- 3. Run checks: `npm run build && npm run typecheck && npm run lint && npm run test:run`
1254
- 4. Open PR with a release category label
1255
-
1256
- ### Release process (structured + complete notes)
1257
-
1258
- To ensure release notes reflect all merged work, this repo uses a draft-release workflow.
1259
-
1260
- 1. **Label every PR** with at least one semantic label:
1261
- - `feature`, `bug`, `performance`, `documentation`, `dependencies`, `refactor`, `test`, `chore`
1262
- - and (when relevant) `semver:major`, `semver:minor`, or `semver:patch`
1263
- - PRs are validated by CI (`Release Label Check`) and fail if no release category label is present
1264
- 2. **Let Release Drafter build the draft notes** automatically from merged PRs on `main`.
1265
- 3. **Before publishing**:
1266
- - compare `git log --oneline vX.Y.Z..HEAD` (or the previous release tag range) against the draft release notes so the release summary covers the full shipped delta, not just the current `CHANGELOG.md` `Unreleased` section
1267
- - copy/finalize relevant highlights into `CHANGELOG.md`
1268
- - bump `package.json` version
1269
- - run: `npm run build && npm run typecheck && npm run lint && npm run test:run`
1270
- 4. **Publish release** from the draft (or via `gh release create` after reviewing draft content).
1271
-
1272
- PRs labeled `skip-changelog` are intentionally excluded from release notes.
1273
-
1274
- ### Project Structure
1275
-
1276
- ```
1277
- ā”œā”€ā”€ src/
1278
- │ ā”œā”€ā”€ index.ts # Plugin entry point
1279
- │ ā”œā”€ā”€ mcp-server.ts # MCP server (Cursor, Claude Code, Windsurf)
1280
- │ ā”œā”€ā”€ cli.ts # CLI entry for MCP stdio transport
1281
- │ ā”œā”€ā”€ config/ # Configuration schema
1282
- │ ā”œā”€ā”€ embeddings/ # Provider detection and API calls
1283
- │ ā”œā”€ā”€ indexer/ # Core indexing logic + inverted index
1284
- │ ā”œā”€ā”€ git/ # Git utilities (branch detection)
1285
- │ ā”œā”€ā”€ tools/ # OpenCode tool definitions
1286
- │ ā”œā”€ā”€ utils/ # File collection, cost estimation
1287
- │ ā”œā”€ā”€ native/ # Rust native module wrapper
1288
- │ └── watcher/ # File/git change watcher
1289
- ā”œā”€ā”€ native/
1290
- │ └── src/ # Rust: tree-sitter, usearch, xxhash, SQLite
1291
- ā”œā”€ā”€ tests/ # Unit tests (vitest)
1292
- ā”œā”€ā”€ commands/ # Slash command definitions
1293
- ā”œā”€ā”€ skill/ # Agent skill guidance
1294
- └── .github/workflows/ # CI/CD (test, build, publish)
193
+ npm ci
194
+ npm run build
195
+ npm run typecheck
196
+ npm run lint
197
+ npm run test:run
1295
198
  ```
1296
199
 
1297
- ### Native Module
200
+ Native changes require Rust and `npm run build:native`. See [Contributing](CONTRIBUTING.md), [Architecture](ARCHITECTURE.md), and [Adding language support](docs/adding-language-support.md).
1298
201
 
1299
- The Rust native module handles performance-critical operations:
1300
- - **tree-sitter**: Language-aware code parsing with JSDoc/docstring extraction
1301
- - **usearch**: High-performance vector similarity search with F16 quantization
1302
- - **SQLite**: Persistent storage for embeddings, chunks, branch catalog, symbols, and call edges
1303
- - **BM25 inverted index**: Fast keyword search for hybrid retrieval
1304
- - **Call graph extraction**: Tree-sitter query-based extraction of function calls, method calls, constructors, and imports (TypeScript/JavaScript, Python, Go, Rust, Swift, PHP, Apex, Zig, GDScript, MATLAB, Bash, C, C++, Metal)
1305
- - **xxhash**: Fast content hashing for change detection
202
+ ## Documentation
1306
203
 
1307
- Rebuild with: `npm run build:native` (requires Rust toolchain)
1308
-
1309
- ### Platform Support
1310
-
1311
- Pre-built native binaries are published for:
1312
-
1313
- | Platform | Architecture | SIMD Acceleration |
1314
- |----------|-------------|--------------------|
1315
- | macOS | x86_64 | āœ… simsimd |
1316
- | macOS | ARM64 (Apple Silicon) | āœ… simsimd |
1317
- | Linux | x86_64 (GNU) | āœ… simsimd |
1318
- | Linux | ARM64 (GNU) | āœ… simsimd |
1319
- | Windows | x86_64 (MSVC) | āŒ scalar fallback |
1320
-
1321
- Windows builds use scalar distance functions instead of SIMD — functionally identical, marginally slower for very large indexes. This is due to MSVC lacking support for certain AVX-512 intrinsics used by simsimd.
204
+ - [Installation and host setup](docs/installation.md)
205
+ - [Tools and commands](docs/tools.md)
206
+ - [Configuration](docs/configuration.md)
207
+ - [Troubleshooting](TROUBLESHOOTING.md)
208
+ - [Architecture](ARCHITECTURE.md)
209
+ - [Evaluation](docs/evaluation.md)
210
+ - [Cross-repository benchmarking](docs/benchmarking-cross-repo.md)
211
+ - [Adding language support](docs/adding-language-support.md)
212
+ - [Contributing](CONTRIBUTING.md)
213
+ - [Security](SECURITY.md)
214
+ - [Changelog](CHANGELOG.md)
1322
215
 
1323
216
  ## License
1324
217
 
1325
- MIT
1326
-
1327
- Notices for third-party components distributed with the package are available in [`THIRD_PARTY_LICENSES.md`](./THIRD_PARTY_LICENSES.md).
218
+ MIT. See [LICENSE](LICENSE).