opencode-codebase-index 0.5.2 → 0.6.0

package/README.md

@@ -10,6 +10,24 @@
 
 **opencode-codebase-index** brings semantic understanding to your [OpenCode](https://opencode.ai) workflow — and now to any MCP-compatible client like Cursor, Claude Code, and Windsurf. Instead of guessing function names or grepping for keywords, ask your codebase questions in plain English.
 
+## 📌 Quick Navigation
+
+- [⚡ Quick Start](#-quick-start)
+- [🌐 MCP Server (Cursor, Claude Code, Windsurf, etc.)](#-mcp-server-cursor-claude-code-windsurf-etc)
+- [🎯 When to Use What](#-when-to-use-what)
+- [🧰 Tools Available](#-tools-available)
+- [🎮 Slash Commands](#-slash-commands)
+- [⚙️ Configuration](#️-configuration)
+- [🤝 Contributing](#-contributing)
+
+## 👋 Choose Your Path
+
+- **I want to try it now** → go to [Quick Start](#-quick-start)
+- **I use Cursor/Claude Code/Windsurf** → go to [MCP Server setup](#-mcp-server-cursor-claude-code-windsurf-etc)
+- **I’m comparing tools and workflows** → go to [When to Use What](#-when-to-use-what)
+- **I’m tuning behavior/cost/performance** → go to [Configuration](#️-configuration)
+- **I want to contribute** → go to [Contributing](#-contributing)
+
 ## 🚀 Why Use This?
 
 - 🧠 **Semantic Search**: Finds "user authentication" logic even if the function is named `check_creds`.
@@ -168,7 +186,7 @@ graph TD
 
 1. **Parsing**: We use `tree-sitter` to intelligently parse your code into meaningful blocks (functions, classes, interfaces). JSDoc comments and docstrings are automatically included with their associated code.
 
-**Supported Languages**: TypeScript, JavaScript, Python, Rust, Go, Java, C#, Ruby, Bash, C, C++, JSON, TOML, YAML
+**Supported Languages**: TypeScript, JavaScript, Python, Rust, Go, Java, C#, Ruby, PHP, Bash, C, C++, JSON, TOML, YAML
 2. **Chunking**: Large blocks are split with overlapping windows to preserve context across chunk boundaries.
 3. **Embedding**: These blocks are converted into vector representations using your configured AI provider.
 4. **Storage**: Embeddings are stored in SQLite (deduplicated by content hash) and vectors in `usearch` with F16 quantization for 50% memory savings. A branch catalog tracks which chunks exist on each branch.
@@ -290,6 +308,7 @@ The plugin automatically registers these slash commands:
 | ------- | ----------- |
 | `/search <query>` | **Pure Semantic Search**. Best for "How does X work?" |
 | `/find <query>` | **Hybrid Search**. Combines semantic search + grep. Best for "Find usage of X". |
+| `/call-graph <query>` | **Call Graph Trace**. Find callers/callees to understand execution flow. |
 | `/index` | **Update Index**. Forces a refresh of the codebase index. |
 | `/status` | **Check Status**. Shows if indexed, chunk count, and provider info. |
 
@@ -365,7 +384,7 @@ Zero-config by default (uses `auto` mode). Customize in `.opencode/codebase-inde
 | `logBranch` | `true` | Log branch detection and switches |
 | `metrics` | `false` | Enable metrics collection (indexing stats, search timing, cache performance) |
 
-### Retrieval ranking behavior (Phase 1)
+### Retrieval ranking behavior
 
 - `codebase_search` and `codebase_peek` use the hybrid path: semantic + keyword retrieval → fusion (`fusionStrategy`) → deterministic rerank (`rerankTopN`) → filtering.
 - `find_similar` stays semantic-only: semantic retrieval + deterministic rerank only (no keyword retrieval, no RRF).
@@ -374,6 +393,135 @@ Zero-config by default (uses `auto` mode). Customize in `.opencode/codebase-inde
   - baseline (versioned): `benchmarks/baselines/retrieval-baseline.json`
   - latest candidate run (generated): `benchmark-results/retrieval-candidate.json`
 
+## 📏 Evaluation Harness
+
+This repository includes a first-class eval system for retrieval quality with versioned golden sets, compare mode, parameter sweeps, CI budgets, and run artifacts.
+
+### Commands
+
+```bash
+npm run eval
+npm run eval:ci
+npm run eval:ci:ollama
+npm run eval:compare -- --against benchmarks/baselines/eval-baseline-summary.json
+```
+
+CI usage split:
+
+- `npm run eval:smoke`: harness smoke check with local mock embeddings (used in main CI)
+- `npm run eval:ci`: real quality gate against baseline/budget (for scheduled/manual quality workflow)
+
+For `eval-quality.yml`, the default CI path uses **GitHub Models** with the workflow `GITHUB_TOKEN` plus `models: read`, so you do not need a separate OpenAI API key just to run the scheduled gate.
+
+That default GitHub Models path uses `benchmarks/budgets/github-models.json`, which applies stable absolute thresholds instead of the stricter baseline-regression budget used for explicit external providers.
+
+Optional override secrets for another OpenAI-compatible endpoint:
+
+- `EVAL_EMBED_BASE_URL`
+- `EVAL_EMBED_API_KEY`
+- `EVAL_EMBED_MODEL` (optional, default `text-embedding-3-small`)
+- `EVAL_EMBED_DIMENSIONS` (optional, default `1536`)
+
+If you override the provider, set both `EVAL_EMBED_BASE_URL` and `EVAL_EMBED_API_KEY`. Otherwise the workflow falls back to GitHub Models automatically. Override providers continue to use the baseline-driven budget in `benchmarks/budgets/default.json`.
+
+No OpenAI API access? Use Ollama quality gate locally:
+
+- Config: `.github/eval-ollama-config.json`
+- Script: `npm run eval:ci:ollama`
+
+Prerequisites: Ollama installed, `ollama serve` running on `127.0.0.1:11434`, and `nomic-embed-text` pulled.
+
+Examples:
+
+```bash
+# Run against small golden set
+npm run eval -- --dataset benchmarks/golden/small.json
+
+# Compare against baseline
+npm run eval:compare -- --against benchmarks/baselines/eval-baseline-summary.json --dataset benchmarks/golden/medium.json
+
+# Sweep retrieval parameters
+npm run eval -- --dataset benchmarks/golden/small.json --sweepFusionStrategy rrf,weighted --sweepHybridWeight 0.3,0.5,0.7 --sweepRrfK 30,60 --sweepRerankTopN 10,20
+```
+
+### What it reports
+
+- Hit@1, Hit@3, Hit@5, Hit@10
+- MRR@10, nDCG@10
+- Latency p50/p95/p99
+- Token estimates, embedding call counts, estimated embedding cost
+- Failure buckets (`wrong-file`, `wrong-symbol`, `docs-tests-outranking-source`, `no-relevant-hit-top-k`)
+
+### Artifacts
+
+Each run writes:
+
+`benchmarks/results/<timestamp>/`
+
+- `summary.json`
+- `summary.md`
+- `per-query.json`
+- `compare.json` (when baseline/sweep used)
+
+### Golden sets and budgets
+
+- Golden datasets:
+  - `benchmarks/golden/small.json`
+  - `benchmarks/golden/medium.json`
+  - `benchmarks/golden/large.json`
+- CI budgets:
+  - `benchmarks/budgets/github-models.json` for the default GitHub Models workflow path
+  - `benchmarks/budgets/default.json` for explicit external provider overrides with baseline comparison
+
+Full docs: `docs/evaluation.md`
+
+### Cross-repo benchmark results snapshot
+
+Recent representative runs (plugin vs `ripgrep` vs `ast-grep`) on two medium repos:
+
+Methodology for the snapshot below:
+
+- Dataset: auto-generated cross-repo golden sets for `axios` + `express`
+- Repeats: **20** per mode
+- Aggregation: **median** metric per tool (then averaged across repos)
+- Reindex behavior: when enabled, index reset applies on repeat #1 only; subsequent repeats measure warm-index query behavior
+- Sampling note: repository parsing can be capped; benchmark reports include truncation metadata
+- ast-grep scope note: sg metrics are computed on its compatible query subset (`definition`, `keyword-heavy`) with scoped denominators shown in run reports
+
+#### Without reindex (`--no-reindex`, default)
+
+| Metric | Plugin | ripgrep | ast-grep (5/10 queries) |
+|---|---:|---:|---:|
+| Hit@5 | 50% | 5% | 100% |
+| MRR@10 | 0.48 | 0.04 | 0.90 |
+| nDCG@10 | 0.48 | 0.08 | 0.93 |
+| Latency p50 (ms) | 17.5 | 36.9 | 66.6 |
+| Latency p95 (ms) | 30.9 | 44.1 | 70.7 |
+
+#### With reindex (`--reindex`)
+
+| Metric | Plugin | ripgrep | ast-grep (5/10 queries) |
+|---|---:|---:|---:|
+| Hit@5 | 50% | 5% | 100% |
+| MRR@10 | 0.48 | 0.04 | 0.98 |
+| nDCG@10 | 0.48 | 0.07 | 0.98 |
+| Latency p50 (ms) | 17.1 | 35.9 | 69.1 |
+| Latency p95 (ms) | 30.4 | 43.7 | 75.1 |
+
+ast-grep metrics are computed on its compatible query subset only (`definition` + `keyword-heavy`, 5/10 queries per repo). Plugin and ripgrep are scored on all 10 queries.
+
+Interpretation:
+
+- ast-grep dominates on its scoped subset (structural definition queries), but only handles 50% of query types. Plugin handles all query types including natural language.
+- Plugin leads on rank-sensitive quality (MRR/nDCG) vs ripgrep across all query types.
+- ripgrep remains a useful speed-oriented lexical baseline but has significantly lower retrieval relevance for intent-style queries.
+- Plugin is the fastest tool at p50 (~17ms), ahead of ripgrep (~36ms) and ast-grep (~67ms).
+- Reported numbers are rounded to avoid false precision; use report artifacts for full per-repeat audit trails.
+
+For reproducible setup and commands (including with/without reindex), see:
+
+- `docs/benchmarking-cross-repo.md`
+
 ### Embedding Providers
 The plugin automatically detects available credentials in this order:
 1. **GitHub Copilot** (Free if you have it)
@@ -411,79 +559,22 @@ ollama pull nomic-embed-text
 
 ## 📈 Performance
 
-The plugin is built for speed with a Rust native module. Here are typical performance numbers (Apple M1):
-
-### Parsing (tree-sitter)
-
-| Files | Chunks | Time |
-|-------|--------|------|
-| 100 | 1,200 | ~7ms |
-| 500 | 6,000 | ~32ms |
+The plugin is built for speed with a Rust native module (`tree-sitter`, `usearch`, SQLite). In practice, indexing and retrieval remain fast enough for interactive use on medium/large repositories.
 
-### Vector Search (usearch)
+- Typical query latency: ~800-1000ms (mostly embedding API time)
+- Incremental indexing: only changed files are re-embedded
+- Batch DB operations: significant write-speed improvements for large indexes
 
-| Index Size | Search Time | Throughput |
-|------------|-------------|------------|
-| 1,000 vectors | 0.7ms | 1,400 ops/sec |
-| 5,000 vectors | 1.2ms | 850 ops/sec |
-| 10,000 vectors | 1.3ms | 780 ops/sec |
-
-### Database Operations (SQLite with batch)
-
-| Operation | 1,000 items | 10,000 items |
-|-----------|-------------|--------------|
-| Insert chunks | 4ms | 44ms |
-| Add to branch | 2ms | 22ms |
-| Check embedding exists | <0.01ms | <0.01ms |
-
-### Batch vs Sequential Performance
-
-Batch operations provide significant speedups:
-
-| Operation | Sequential | Batch | Speedup |
-|-----------|------------|-------|---------|
-| Insert 1,000 chunks | 38ms | 4ms | **~10x** |
-| Add 1,000 to branch | 29ms | 2ms | **~14x** |
-| Insert 1,000 embeddings | 59ms | 40ms | **~1.5x** |
-
-Run benchmarks yourself: `npx tsx benchmarks/run.ts`
+For reproducible measurements on your machine, run: `npx tsx benchmarks/run.ts`.
 
 ## 🎯 Choosing a Provider
 
-Use this decision tree to pick the right embedding provider:
+Quick recommendation:
 
-```
-                    ┌─────────────────────────┐
-                    │ Do you have Copilot?    │
-                    └───────────┬─────────────┘
-                          ┌─────┴─────┐
-                         YES          NO
-                          │            │
-              ┌───────────▼───────┐    │
-              │ Codebase < 1k     │    │
-              │ files?            │    │
-              └─────────┬─────────┘    │
-                  ┌─────┴─────┐        │
-                 YES          NO       │
-                  │            │       │
-                  ▼            │       │
-           ┌──────────┐        │       │
-           │ Copilot  │        │       │
-           │ (free)   │        │       │
-           └──────────┘        │       │
-                               ▼       ▼
-                    ┌─────────────────────────┐
-                    │ Need fastest indexing?  │
-                    └───────────┬─────────────┘
-                          ┌─────┴─────┐
-                         YES          NO
-                          │            │
-                          ▼            ▼
-                   ┌──────────┐ ┌──────────────┐
-                   │ Ollama   │ │ OpenAI or    │
-                   │ (local)  │ │ Google       │
-                   └──────────┘ └──────────────┘
-```
+- **Want local + private + fast indexing** → use **Ollama**
+- **Already have Copilot and a smaller repo** → use **GitHub Copilot**
+- **General cloud setup** → use **OpenAI** or **Google**
+- **Custom/OpenAI-compatible endpoint** → use **custom** provider
 
 ### Provider Comparison
 
@@ -499,35 +590,13 @@ Use this decision tree to pick the right embedding provider:
 
 ### Setup by Provider
 
-**Ollama (Recommended for large codebases)**
-```bash
-ollama pull nomic-embed-text
-```
-```json
-{ "embeddingProvider": "ollama" }
-```
-
-**OpenAI**
-```bash
-export OPENAI_API_KEY=sk-...
-```
-```json
-{ "embeddingProvider": "openai" }
-```
+Set the provider in `.opencode/codebase-index.json`:
 
-**Google**
-```bash
-export GOOGLE_API_KEY=...
-```
 ```json
-{ "embeddingProvider": "google" }
+{ "embeddingProvider": "ollama" }
 ```
 
-**GitHub Copilot**
-No setup needed if you have an active Copilot subscription.
-```json
-{ "embeddingProvider": "github-copilot" }
-```
+Credentials (if required) are read from environment variables (for example `OPENAI_API_KEY` or `GOOGLE_API_KEY`).
 
 **Custom (OpenAI-compatible)**
 Works with any server that implements the OpenAI `/v1/embeddings` API format (llama.cpp, vLLM, text-embeddings-inference, LiteLLM, etc.).
@@ -578,14 +647,33 @@ Be aware of these characteristics:
 
 ## 🤝 Contributing
 
-1. Fork the repository
-2. Create a feature branch: `git checkout -b feature/my-feature`
-3. Make your changes and add tests
-4. Run checks: `npm run build && npm run test:run && npm run lint`
-5. Commit: `git commit -m "feat: add my feature"`
-6. Push and open a pull request
+For contribution workflow, standards, and release-label requirements, see [`CONTRIBUTING.md`](./CONTRIBUTING.md).
+
+If you want to add support for a new language, see [`docs/adding-language-support.md`](./docs/adding-language-support.md) for the full Rust + TypeScript checklist.
+
+Quick path:
+
+1. Fork + branch
+2. Implement + tests
+3. Run checks: `npm run build && npm run typecheck && npm run lint && npm run test:run`
+4. Open PR with a release category label
+
+### Release process (structured + complete notes)
+
+To ensure release notes reflect all merged work, this repo uses a draft-release workflow.
+
+1. **Label every PR** with at least one semantic label:
+   - `feature`, `bug`, `performance`, `documentation`, `dependencies`, `refactor`, `test`, `chore`
+   - and (when relevant) `semver:major`, `semver:minor`, or `semver:patch`
+   - PRs are validated by CI (`Release Label Check`) and fail if no release category label is present
+2. **Let Release Drafter build the draft notes** automatically from merged PRs on `main`.
+3. **Before publishing**:
+   - copy/finalize relevant highlights into `CHANGELOG.md`
+   - bump `package.json` version
+   - run: `npm run build && npm run typecheck && npm run lint && npm run test:run`
+4. **Publish release** from the draft (or via `gh release create` after reviewing draft content).
 
-CI will automatically run tests and type checking on your PR.
+PRs labeled `skip-changelog` are intentionally excluded from release notes.
 
 ### Project Structure
 

package/commands/definition.md

@@ -0,0 +1,24 @@
+---
+description: Find where a symbol is defined in the codebase
+---
+
+Find the authoritative definition of a symbol in the codebase.
+
+User input: $ARGUMENTS
+
+The input is a symbol name or description of what to find the definition of.
+
+Look for optional parameters:
+- `limit=N` or "top N" → set limit
+- `dir=X` or "in folder X" → set directory filter
+- File extensions like ".ts", ".py" → set fileType
+
+Call `implementation_lookup` with the parsed arguments.
+
+Examples:
+- `/definition validateToken` → query="validateToken"
+- `/definition payment handler` → query="payment handler"
+- `/definition createSystem dir=packages/react` → query="createSystem", directory="packages/react"
+
+This prioritizes real implementation files over tests, docs, and examples.
+If no definition is found, suggest using `codebase_search` for broader discovery.