opencode-rag-plugin 1.2.11 → 1.3.2

package/ReadMe.md

@@ -1,322 +1,330 @@
 # OpenCodeRAG
 
-Local-first RAG plugin for OpenCode — semantic code search powered by
-embeddings and vector similarity.
-
-**Published on npm as [`opencode-rag-plugin`](https://www.npmjs.com/package/opencode-rag-plugin).**
+> A RAG (Retrieval-Augmented Generation) plugin for [OpenCode](https://opencode.ai) that adds **semantic code search** powered by locally-hosted embedding models.
+
+**Published on npm: [`opencode-rag-plugin`](https://www.npmjs.com/package/opencode-rag-plugin)**
+
+> ⚠️ **Do not confuse with `opencode-rag`** — that is a discontinued project by a different author.
+
+OpenCodeRAG reduces token usage by replacing expensive file-read tool calls with targeted, vector-similarity-based chunk retrieval. Large codebases and files on a local dedicated GPU benefit most in terms of performance, but even a modern CPU handles most workloads without a GPU.
+
+---
+
+## Table of Contents
+
+- [Features](#features)
+- [Prerequisites](#prerequisites)
+- [Quick Start](#quick-start)
+- [Installation](#installation)
+  - [Global Installation (Recommended)](#global-installation-recommended)
+  - [Per-Workspace Setup](#per-workspace-setup)
+  - [Manual Installation](#manual-installation)
+  - [Uninstallation](#uninstallation)
+- [Configuration](#configuration)
+  - [Embedding Providers](#embedding-providers)
+  - [Logging](#logging)
+- [Usage](#usage)
+  - [CLI](#cli)
+  - [OpenCode Plugin](#opencode-plugin)
+- [Architecture](#architecture)
+- [Tech Stack](#tech-stack)
+- [Chunking Language Support](#chunking-language-support)
+  - [Adding Custom Chunkers](#adding-custom-chunkers)
+- [Data Model](#data-model)
+- [Vector Store](#vector-store)
+- [Development](#development)
+- [Privacy](#privacy)
+
+---
 
 ## Features
 
-- **AST-aware chunking** — splits code into functions, classes, methods using
-  tree-sitter for 16 languages, plus regex-based chunking for 4 markup/config/doc
-  formats (Markdown, Razor, .sln, LaTeX). Falls back to line-based chunking for
-  unrecognized formats.
-- **Incremental indexing** — manifest-backed indexing skips unchanged files,
-  removes deleted entries, and updates only changed files.
-- **Watch mode** — `index --watch` re-indexes on file changes with debounced,
-  serialized passes.
-- **Pluggable chunkers** — add custom language chunkers via config or programmatic API.
-- **Configurable embeddings** — Ollama (default) or OpenAI-compatible providers.
-  Batch embedding with configurable batch size.
-- **Local vector store** — LanceDB with L2 distance scoring, memory mode for
-  testing.
-- **CLI** — index, query, clear, status commands.
-- **OpenCode plugin** — exposes a chunk retrieval tool and suggests relevant files after each user message via the `chat.message` hook.
+| Feature | Description |
+|---------|-------------|
+| **AST-aware chunking** | Splits code into functions, classes, and methods via tree-sitter for 16 languages; regex-based for Markdown, Razor, `.sln`, and LaTeX; line-based fallback for everything else |
+| **Incremental indexing** | Skips unchanged files, removes deleted entries, re-indexes only what changed |
+| **Watch mode** | `index --watch` debounces file changes and re-indexes automatically |
+| **Pluggable chunkers** | Add custom language chunkers via config file or programmatic API |
+| **Configurable embeddings** | Ollama (default, fully local) or any OpenAI-compatible provider; configurable batch size |
+| **Hybrid search** | TF×IDF keyword index fused with vector similarity — better precision on identifiers and function names |
+| **Local vector store** | LanceDB with L2 distance scoring; in-memory mode for testing |
+| **CLI** | `index`, `query`, `clear`, `status` commands |
+| **OpenCode integration** | Auto-injects relevant code chunks into each message; suggests related files; exposes a retrieval tool the agent can call directly |
 
-## Architecture
+---
 
-```
-Workspace Files
-       │
-       ▼
-┌──────────────┐
-│   Chunker    │  AST-based (tree-sitter) or line-based fallback
-└──────┬───────┘
-       │ chunks
-       ▼
-┌──────────────┐
-│   Embedder   │  Ollama / OpenAI-compatible API
-└──────┬───────┘
-       │ vectors
-       ▼
-┌──────────────┐
-│  VectorStore │  LanceDB (local files or memory:// for tests)
-└──────┬───────┘
-       │ + manifest.json
-       ▼
-┌──────────────┐
-│ Indexer/Retr.│  incremental index or query/search
-└──────┬───────┘
-       │ results
-       ▼
-   LLM Context
-```
+## Prerequisites
 
-## Tech Stack
+- **[OpenCode](https://opencode.ai)** installed and configured
+- **Node.js v22+** (required for native ESM and `fetch`)
+- **[Ollama](https://ollama.ai)** running locally with an embedding model (default: `nomic-embed-text`)
+  — or any OpenAI-compatible embedding API
 
-| Layer       | Technology                                          |
-| ----------- | --------------------------------------------------- |
-| Runtime     | Node.js v22 + tsx (ESM)                             |
-| Language    | TypeScript 5.8                                      |
-| Chunking    | web-tree-sitter (WASM) + tree-sitter-wasm grammars  |
-| Embeddings  | Ollama / OpenAI-compatible (native fetch)           |
-| Vector DB   | LanceDB (`@lancedb/lancedb`)                        |
-| CLI         | commander                                           |
-| Tests       | Node built-in test runner (`node --test`)           |
-| Package mgr | npm (with `--legacy-peer-deps`)                     |
+---
 
-## Installation
+## Quick Start
+
+```bash
+# 1. Clone the repository
+git clone https://github.com/your-org/OpenCodeRAG.git
+cd OpenCodeRAG
 
-### Global installation (recommended)
+# 2. Run the install script
+./install.sh          # Linux/macOS
+.\install.ps1         # Windows
 
-Clone the repository and run the install script:
+# 3. Go to your project and initialize
+cd /path/to/your/project
+opencode-rag init
 
-**Windows:**
-```powershell
-.\install.ps1
+# 4. Index the workspace
+opencode-rag index
+
+# 5. Restart OpenCode — the plugin is active
 ```
 
-**Linux/macOS:**
+---
+
+## Installation
+
+### Global Installation (Recommended)
+
+Clone the repository and run the platform-specific install script:
+
 ```bash
+# Linux/macOS
 ./install.sh
+
+# Windows
+.\install.ps1
 ```
 
-This will:
+The script will:
 1. Build the plugin from source (`npm run build`)
 2. Install it into OpenCode's runtime (`~/.opencode/node_modules/`)
 3. Register it in the global OpenCode config (`~/.config/opencode/opencode.jsonc`)
 
-After installation, restart OpenCode and the plugin is ready.
+**Restart OpenCode after installation.**
 
-### Per-workspace setup
+---
+
+### Per-Workspace Setup
 
 ```bash
 cd your-workspace
 opencode-rag init
 ```
 
-`opencode-rag init` bootstraps the current workspace by creating:
+Bootstraps the current directory with all required files and a sample configuration.
+Add `--skip-install` to generate the files only, without installing dependencies.
 
-- `opencode-rag.json`
-- `.opencode/.gitignore`
-- `.opencode/opencode.json`
-- `.opencode/package.json`
-- `.opencode/plugins/rag-plugin.js` (workspace-local plugin fallback)
-- `.opencode/node_modules/` with the workspace-local plugin dependencies
+---
 
-Add `--skip-install` if you only want the files without installing dependencies.
+### Manual Installation
 
-### Uninstallation
+If you prefer to build and install without the script:
 
-To completely remove OpenCodeRAG from your system, run the uninstall script:
+```bash
+npm run build
+npm pack
 
-**Windows:**
-```powershell
-.\install.ps1 uninstall
+# Install into OpenCode's runtime
+npm install --prefix ~/.opencode/ opencode-rag-plugin-1.2.0.tgz
+npm install --prefix ~/.config/opencode/ opencode-rag-plugin-1.2.0.tgz
+
+# Then add "opencode-rag-plugin" to the plugins array in:
+# ~/.config/opencode/opencode.jsonc
 ```
 
-**Linux/macOS:**
+---
+
+### Uninstallation
+
 ```bash
+# Linux/macOS
 ./install.sh uninstall
+
+# Windows
+.\install.ps1 uninstall
 ```
 
-This will remove:
-- CLI wrapper from `~/.local/bin/`
-- Plugin installations from `~/.config/opencode/node_modules/`
-- Plugin installations from `~/.opencode/node_modules/`
-- .tgz package files
-- Plugin entries from OpenCode configuration files
-- Legacy workspace-local plugin files
+**Restart OpenCode after uninstalling.**
 
-After uninstallation, restart OpenCode if it is running.
+> **Automatic dependencies:** `apache-arrow` (LanceDB peer dependency) and `tree-sitter-wasm` (pre-built WASM grammars) are installed automatically.
 
-### Dependencies
+---
 
-### Dependencies
+## Configuration
 
-- **Node.js v22+** for native ESM and fetch support
-- **apache-arrow** — peer dependency for LanceDB (auto-installed)
-- **tree-sitter-wasm** — ships pre-built WASM grammars for all supported languages
+Create `opencode-rag.json` in your project root (auto-detected), or point to it explicitly with `--config ./path/to/config.json`.
 
-## Configuration
+The repository's own [`opencode-rag.json`](./opencode-rag.json) is a complete, annotated example covering all available options.
 
-Create `opencode-rag.json` in the project root (auto-detected) or pass via
-`--config`. The repository's own [`opencode-rag.json`](./opencode-rag.json) serves
-as a complete example with all available options.
+**Partial overrides are supported:** only the keys you set are applied — everything else falls back to defaults. Sections are deep-merged.
 
-Config files support partial overrides — missing keys fall back to defaults.
-Deep merging is applied per section.
+---
 
 ### Embedding Providers
 
-| Provider | `baseUrl` example                 | Notes                        |
-| -------- | --------------------------------- | ---------------------------- |
-| ollama   | `http://localhost:11434/api`      | Default. No apiKey required. Proxy is disabled when `embedding.proxy.url` is empty. |
-| openai   | `https://api.openai.com/v1`       | Requires apiKey.             |
+| Provider | `baseUrl` | Notes |
+|----------|-----------|-------|
+| `ollama` | `http://localhost:11434/api` | **Default.** No API key required. |
+| `openai` | `https://api.openai.com/v1` | Requires `apiKey`. Sends all texts in one batch request. |
+
+- `embedding.timeoutMs` defaults to `30000` ms. Increase it if your local model has a slow cold start.
+- Ollama sends one request per batch to `/api/embed`. Set `embedding.proxy.url` to route through a standard HTTP proxy.
+
+---
+
+### Logging
+
+```json
+{
+  "logging": {
+    "level": "info",
+    "logFilePath": "./.opencode/opencode-rag.log"
+  }
+}
+```
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `level` | `"info"` | `"debug"`, `"info"`, or `"error"` |
+| `logFilePath` | `"./.opencode/opencode-rag.log"` | Relative paths resolve against the workspace root |
 
-`embedding.timeoutMs` defaults to 30000 ms. Increase it if your local model has a slow cold start.
+The `LOG_FILE_PATH` environment variable is a fallback when no config value is set. Config takes precedence over the env var.
 
-OpenAI provider sends all texts in a single request. Ollama sends one request
-per request to `/api/embed`. Set `embedding.proxy.url` to use the standard
-proxy-aware HTTP path instead of the direct socket path.
+---
 
 ## Usage
 
-This extension consists of two main interfaces:
-1. **CLI** — for manual indexing and querying from the terminal
-2. **OpenCode plugin** — for automatic retrieval and file suggestions within the chat interface
+OpenCodeRAG provides two interfaces:
+
+| Interface | Best for |
+|-----------|----------|
+| **CLI** | Manual indexing, searching, and diagnostics |
+| **OpenCode plugin** | Automatic retrieval and file suggestions while chatting |
+
+---
 
 ### CLI
 
 ```bash
-# Index the workspace incrementally
+# Index the workspace (incremental by default)
 opencode-rag index
 
-# Force full re-index (clears existing data first)
+# Force a complete re-index
 opencode-rag index --force
 
-# Watch workspace and incrementally re-index on changes
+# Watch for changes and re-index automatically
 opencode-rag index --watch
 
 # Semantic search
 opencode-rag query "How is authentication handled?"
-
-# Limit results
 opencode-rag query "error handling" --top-k 5
 
-# Show indexing stats
+# Show index statistics
 opencode-rag status
 
-# Example output:
-#   Indexed chunks:    1247
-#   Store path:        /home/user/project/.opencode/rag_db
-#   Embedding provider: ollama
-#   Embedding model:   nomic-embed-text
-#   Manifest status:   ok
-#   Manifest entries:  42
-#   Last indexed:      2026-05-28 10:45:02
-#   Up-to-date files:  42
-#   Pending files:     0
-#   Watch mode:        off
-
 # Clear all indexed data
 opencode-rag clear
 
-# Use custom config
+# Use a custom config file
 opencode-rag index --config ./my-config.json
 ```
 
-`index` is incremental by default. A sidecar manifest is stored at
-`<vectorStore.path>/manifest.json` and tracks file hashes, chunk counts, and the
-last successful index timestamp. If the manifest is missing or corrupt while the
-vector store already contains data, the next index pass clears and rebuilds the
-store to avoid duplicates.
+**Example `status` output:**
+```
+Indexed chunks:      1247
+Store path:          /home/user/project/.opencode/rag_db
+Embedding provider:  ollama
+Embedding model:     nomic-embed-text
+Manifest status:     ok
+Manifest entries:    42
+Last indexed:        2026-05-28 10:45:02
+Up-to-date files:    42
+Pending files:       0
+Keyword index:       enabled (1274 chunks)
+```
+
+#### How Incremental Indexing Works
+
+A manifest is stored at `<vectorStore.path>/manifest.json` and tracks file hashes, chunk counts, and the last successful index timestamp.
 
-### Watch workflow
+- Only **changed files** are re-embedded; **deleted files** are removed from the store.
+- If the manifest is missing or corrupt while the store already has data, the next pass **clears and rebuilds** the entire store to prevent duplicates.
 
-Start a watch session:
+#### Watch Mode
 
 ```bash
 opencode-rag index --watch
 ```
 
-The initial pass indexes the workspace, then watches for file changes. On each
-`add`, `change`, `unlink`, or `unlinkDir` event, the watch debounces (300 ms)
-and triggers a new incremental pass. If a pass is already running, the re-index
-queues one follow-up pass and runs it as soon as the current pass finishes.
+- Runs a full initial index pass, then listens for `add`, `change`, `unlink`, and `unlinkDir` events.
+- Changes are debounced (300 ms) before triggering an incremental pass.
+- If a pass is already running, one follow-up pass is queued and starts immediately after.
+- The watcher ignores excluded directories, the vector store path, and the manifest file itself.
+- Press `Ctrl+C` to stop.
 
-The watcher ignores excluded directories, the vector store path, and the
-manifest file itself. Press `Ctrl+C` to stop.
+---
 
 ### OpenCode Plugin
 
-The plugin registers:
+Once installed, the plugin registers two hooks:
+
+| Hook | What it does |
+|------|-------------|
+| `opencode-rag-context` | A retrieval tool the agent can call directly to fetch code chunks with file paths and line ranges |
+| `chat.message` | After each user message, retrieves relevant files and appends them to the message |
 
-1. **`opencode-rag-context`** — a custom retrieval tool for chunk-level evidence
-2. **`chat.message`** — after each user message, automatically retrieves relevant indexed files and appends a compact suggestion list to the message text
+#### Auto-Injection
 
-#### Chat Message File Suggestions
+After each message you send, the plugin:
 
-After you send a message, the plugin:
-1. Extracts the user's message text
+1. Extracts your message text
 2. Runs semantic retrieval against the indexed workspace
-3. Groups results by file, sorts by best chunk score, and formats a compact file list:
+3. **High-confidence results** (relevance ≥ `autoInject.minScore`, default `0.75`): injects the actual code chunks directly into the message — the agent gets context without a tool-call round-trip:
    ```
-   src/plugin.ts (typescript, lines 10-42)
-   src/core/config.ts (typescript, lines 66-145)
+   ---
+   **Auto-retrieved code context** _(context: 2 chunks, 1 file, relevance 0.88–0.92)_
+   ---
+   [src/auth.ts:12-30] (typescript, score: 0.92)
+   ```typescript
+   function login() { ... }
+   ```
+   ---
+   ```
+4. **No high-confidence results**: falls back to a compact file list (max 10 files):
+   ```
+   src/plugin.ts (typescript, lines 10-42, relevance 0.87)
+   src/core/config.ts (typescript, lines 66-145, relevance 0.72)
    ```
-4. Appends the list (max 10 files) to your message text
-
-Only file paths, language, and line ranges are shown — no scores or code snippets. This gives the agent lightweight hints about which files are relevant without inflating the context window.
-
-**Config:**
-
-| Option | Default | Description |
-| ------ | ------- | ----------- |
-| `openCode.overrideRead` | `false` | Set to `true` to restore the legacy RAG-backed `read` tool (deprecated) |
-| `openCode.maxContextChunks` | `5` | Maximum chunks per retrieval (affects `opencode-rag-context` tool output) |
-| `retrieval.topK` | `10` | Number of chunks fetched per query (controls chat.message file suggestion breadth) |
-
-Errors during retrieval are silently caught — a failed search won't break the
-chat.
-
-#### Install from source
-
-After cloning and installing dependencies:
-
-```bash
-# Option 1: Install globally via the install script (recommended)
-./install.sh        # Linux/macOS
-.\install.ps1       # Windows
-
-# Option 2: Build and pack manually, then register globally
-npm run build
-npm pack
-npm install --prefix ~/.opencode/ opencode-rag-plugin-1.2.0.tgz
-npm install --prefix ~/.config/opencode/ opencode-rag-plugin-1.2.0.tgz
-# Add "opencode-rag-plugin" to the plugin array in ~/.config/opencode/opencode.jsonc
-
-# Option 3: Bootstrap workspace only (uses npm version)
-opencode-rag init
-
-# Uninstall (removes all global copies)
-./install.sh uninstall     # Linux/macOS
-.\install.ps1 uninstall    # Windows
-```
-
-The plugin auto-detects configuration from `opencode-rag.json` or
-`.opencode/opencode-rag.json` or `.opencode/rag.json` in the project root.
-
-`opencode-rag init` creates a workspace-local plugin file at `.opencode/plugins/rag-plugin.js`
-that re-exports from `node_modules/`, serving as a fallback when global loading fails.
-No `plugin` entry is required in `.opencode/opencode.json`.
 
-Restart OpenCode after changing plugin files or plugin configuration.
+This eliminates a tool-call round-trip for roughly 70% of code-related messages. The agent can still call `opencode-rag-context` for more targeted or additional context.
 
-### Logging
+> **Note:** Retrieval errors are silently caught — a failed search will never break the chat.
 
-Logging is configured under the `logging` key:
+---
 
-```json
-{
-  "logging": {
-    "level": "info",
-    "logFilePath": "./.opencode/opencode-rag.log"
-  }
-}
-```
+#### Plugin Configuration Reference
 
-| Option       | Default                        | Description                                  |
-| ------------ | ------------------------------ | -------------------------------------------- |
-| `level`      | `"info"`                       | Log level: `"debug"`, `"info"`, or `"error"` |
-| `logFilePath` | `"./.opencode/opencode-rag.log"` | Path to the log file (relative paths are resolved against the workspace directory) |
+| Option | Default | Description |
+|--------|---------|-------------|
+| `openCode.readOverride` | `false` | When `true`, registers a RAG-backed `read` tool that shadows OpenCode's built-in file reader — the agent gets indexed chunks instead of full file contents |
+| `openCode.maxContextChunks` | `5` | Maximum chunks returned by `opencode-rag-context` |
+| `openCode.autoInject.enabled` | `true` | Enable or disable auto-injection |
+| `openCode.autoInject.minScore` | `0.75` | Minimum relevance score for auto-injection (0–1) |
+| `openCode.autoInject.maxChunks` | `3` | Maximum chunks to inject per message |
+| `openCode.autoInject.maxTokens` | `2000` | Token budget for injected content (~4 chars/token) |
+| `retrieval.topK` | `10` | Chunks fetched per query (controls file suggestion breadth) |
+| `retrieval.hybridSearch.enabled` | `true` | Enable or disable hybrid TF×IDF + vector search |
+| `retrieval.hybridSearch.keywordWeight` | `0.4` | Keyword weight in hybrid fusion (0–1; higher = more keyword influence) |
 
-The resolved log file path also falls back to the `LOG_FILE_PATH` environment variable when the config value is not set. Config takes precedence over the env var when both are provided.
+---
 
-#### AGENTS.md hints for using the plugin
+#### AGENTS.md Snippet
 
-Add a section like this to the target workspace's `AGENTS.md` so the agent
-knows how to use the plugin correctly:
+Add this section to your workspace's `AGENTS.md` so the agent knows how to use the plugin:
 
 ```markdown
 ## OpenCodeRAG Plugin
@@ -324,98 +332,121 @@ knows how to use the plugin correctly:
 This workspace has OpenCodeRAG installed for semantic code retrieval.
 
 ### `opencode-rag-context` tool
-Before planning, editing, or answering, use this tool to retrieve relevant code
-chunks with file paths, line ranges, and surrounding implementation.
+Before planning, editing, or answering questions, use this tool to retrieve relevant
+code chunks with file paths, line ranges, and surrounding implementation.
 - `query` (required) — narrow, specific search, e.g. `"authentication middleware setup"`
 - `pathHints` (optional) — up to 10 path filters, e.g. `["src/auth/"]`
 - `languageHints` (optional) — up to 10 language filters, e.g. `["typescript"]`
-- `topK` (optional) — result count (1-25, default 10)
+- `topK` (optional) — result count (1–25, default 10)
 
 ### File suggestions
 After each user message, a `chat.message` hook appends up to 10 relevant file
-suggestions to the message. Look for lines like
-`src/file.ts (typescript, lines 10-42)` at the bottom of user input.
+suggestions. Look for lines like `src/file.ts (typescript, lines 10-42)` at the
+bottom of user input.
 
 ### Indexing
-- The plugin auto-indexes changed files in the background (debounced 5s)
-- If no results come back, the workspace may not be indexed yet —
-  run `opencode-rag index` from the terminal (or `npx opencode-rag-plugin`)  
-- Tiny files (under 1 KB), excluded extensions, and excluded directories
-  (`node_modules`, `.git`, `.opencode`, `dist`, etc.) are silently skipped
+- Changed files are auto-indexed in the background (debounced 5 s).
+- If searches return no results, the workspace may not be indexed yet —
+  run `opencode-rag index` from the terminal (or `npx opencode-rag-plugin`).
+- Files under 1 KB, excluded extensions, and excluded directories
+  (`node_modules`, `.git`, `.opencode`, `dist`, etc.) are silently skipped.
 ```
 
-The plugin registers itself in the system prompt via the
-`experimental.chat.system.transform` hook, so compliant agents will see a
-reminder about the `opencode-rag-context` tool in their system instructions.
+The plugin also injects a reminder about `opencode-rag-context` into the agent's system prompt via `experimental.chat.system.transform`.
 
-## Data Model
+---
 
-```typescript
-interface Chunk {
-  id: string;
-  content: string;
-  embedding?: number[];
-  metadata: {
-    filePath: string;
-    startLine: number;
-    endLine: number;
-    language: string;
-  };
-}
+## Architecture
 
-interface SearchResult {
-  chunk: Chunk;
-  score: number;   // 1 / (1 + L2_distance), range [0, 1]
-}
 ```
+Workspace Files
+      │
+      ▼
+┌──────────────┐
+│   Chunker    │  AST-based (tree-sitter) or line-based fallback
+└──────┬───────┘
+       │ chunks
+       ▼
+┌──────────────┐
+│   Embedder   │  Ollama / OpenAI-compatible API
+└──────┬───────┘
+       │ vectors
+       ▼
+┌──────────────┐
+│  VectorStore │  LanceDB (local files or memory:// for tests)
+└──────┬───────┘
+       │ + manifest.json
+       ▼
+┌──────────────┐
+│ Indexer/Retr.│  incremental index or query/search
+└──────┬───────┘
+       │ results
+       ▼
+   LLM Context
+```
+
+---
+
+## Tech Stack
 
-## Chunking
-
-| Language   | Strategy                       | Captures                                  |
-| ---------- | ------------------------------ | ----------------------------------------- |
-| TypeScript | AST (tree-sitter)              | functions, methods, classes, interfaces   |
-| Python     | AST (tree-sitter)              | functions, classes, decorated definitions |
-| Java       | AST (tree-sitter)              | methods, classes, interfaces, enums       |
-| Go         | AST (tree-sitter)              | functions, methods, type declarations     |
-| C          | AST (tree-sitter)              | functions, structs, enums, unions, typedefs |
-| C++        | AST (tree-sitter)              | functions, classes, structs, enums, namespaces, templates |
-| C#         | AST (tree-sitter)              | classes, interfaces, structs, enums, methods, namespaces, records |
-| JavaScript | AST (tree-sitter)              | functions, classes, arrow functions, exports |
-| JSON       | AST (tree-sitter)              | key-value pairs                           |
-| XML        | AST (tree-sitter)              | elements (1 chunk per root element)       |
-| HTML       | AST (tree-sitter)              | `<script>` / `<style>` blocks             |
-| CSS        | AST (tree-sitter)              | rule sets, at-rules, media, keyframes     |
-| Razor      | Regex (brace matching)         | `@code` / `@functions` blocks, template regions |
-| Markdown   | Regex heading split            | h1/h2 sections + trailing content         |
-| Solution   | Regex (section boundary)       | project entries and global sections       |
-| Rust       | AST (tree-sitter)              | functions, structs, enums, traits, impl blocks, modules, types |
-| Ruby       | AST (tree-sitter)              | methods, classes, modules, singleton methods |
-| Kotlin     | AST (tree-sitter)              | functions, classes, interfaces, objects, properties |
-| Swift      | AST (tree-sitter)              | functions, classes, structs, enums, protocols, extensions, variables |
-| LaTeX      | Regex section split            | chapter/section/subsection/subsubsection boundaries |
-| PDF        | Paragraph-based (text extraction) | groups small paragraphs, splits oversized |
-| Word (docx) | Paragraph-based (text extraction) | extracts raw text via mammoth, groups small paragraphs, splits oversized |
-| Word (doc) | Paragraph-based (text extraction) | extracts raw text via word-extractor, groups small paragraphs, splits oversized |
-| Excel (xls/xlsx) | Row-batch (text extraction) | extracts CSV per sheet via @e965/xlsx, splits by sheet then by row batches |
-| (other)    | Line-based (100 lines/chunk)   | raw text blocks                           |
-
-Custom chunkers can be added without modifying the project source code. Two
-registration paths are supported:
-
-### Config file
-
-Add a `chunkers` array to `opencode-rag.json`:
+| Layer | Technology |
+|-------|------------|
+| Runtime | Node.js v22 + tsx (ESM) |
+| Language | TypeScript 5.8 |
+| Chunking | web-tree-sitter (WASM) + tree-sitter-wasm grammars |
+| Embeddings | Ollama / OpenAI-compatible (native fetch) |
+| Vector DB | LanceDB (`@lancedb/lancedb`) |
+| CLI | commander |
+| Tests | Node built-in test runner (`node --test`) |
+| Package manager | npm (with `--legacy-peer-deps`) |
+
+---
+
+## Chunking Language Support
+
+| Language | Strategy | Captures |
+|----------|----------|---------|
+| TypeScript | AST (tree-sitter) | functions, methods, classes, interfaces |
+| JavaScript | AST (tree-sitter) | functions, classes, arrow functions, exports |
+| Python | AST (tree-sitter) | functions, classes, decorated definitions |
+| Java | AST (tree-sitter) | methods, classes, interfaces, enums |
+| Go | AST (tree-sitter) | functions, methods, type declarations |
+| Rust | AST (tree-sitter) | functions, structs, enums, traits, impl blocks, modules, types |
+| C | AST (tree-sitter) | functions, structs, enums, unions, typedefs |
+| C++ | AST (tree-sitter) | functions, classes, structs, enums, namespaces, templates |
+| C# | AST (tree-sitter) | classes, interfaces, structs, enums, methods, namespaces, records |
+| Ruby | AST (tree-sitter) | methods, classes, modules, singleton methods |
+| Kotlin | AST (tree-sitter) | functions, classes, interfaces, objects, properties |
+| Swift | AST (tree-sitter) | functions, classes, structs, enums, protocols, extensions, variables |
+| JSON | AST (tree-sitter) | key-value pairs |
+| XML | AST (tree-sitter) | elements (1 chunk per root element) |
+| HTML | AST (tree-sitter) | `<script>` / `<style>` blocks |
+| CSS | AST (tree-sitter) | rule sets, at-rules, media, keyframes |
+| Razor | Regex (brace matching) | `@code` / `@functions` blocks, template regions |
+| Markdown | Regex heading split | h1/h2 sections + trailing content |
+| Solution (`.sln`) | Regex (section boundary) | project entries and global sections |
+| LaTeX | Regex section split | chapter/section/subsection/subsubsection boundaries |
+| PDF | Paragraph-based | groups small paragraphs, splits oversized ones |
+| Word (`.docx`) | Paragraph-based | extracts via mammoth; groups small paragraphs, splits oversized |
+| Word (`.doc`) | Paragraph-based | extracts via word-extractor; groups small paragraphs, splits oversized |
+| Excel (`.xls`/`.xlsx`) | Row-batch | extracts CSV per sheet via `@e965/xlsx`; splits by sheet, then by row batch |
+| *(other)* | Line-based (100 lines/chunk) | raw text blocks |
+
+### Adding Custom Chunkers
+
+Custom chunkers can be registered without modifying the project source.
+
+**Via config file** — add to `opencode-rag.json`:
 
 ```json
 {
   "chunkers": [
-    { "module": "./path/to/rust-chunker.js", "extensions": [".rs"] }
+    { "module": "./path/to/my-chunker.js", "extensions": [".xyz"] }
   ]
 }
 ```
 
-The module path is resolved relative to the config file. The loaded module must
-export (as default or named) an object implementing the `Chunker` interface:
+The module path is resolved relative to the config file. The module must export (as default or named) an object implementing the `Chunker` interface:
 
 ```typescript
 interface Chunker {
@@ -425,29 +456,56 @@ interface Chunker {
 }
 ```
 
-### Programmatic
+**Via programmatic API:**
 
 ```typescript
 import { registerChunker } from "opencode-rag-plugin/library";
-registerChunker(myChunker, [".rs"]);
+registerChunker(myChunker, [".xyz"]); // second arg overrides fileExtensions
+```
+
+> If a built-in chunker already handles the requested extension, the new registration is skipped and a warning is emitted.
+
+---
+
+## Data Model
+
+```typescript
+interface Chunk {
+  id: string;
+  content: string;
+  embedding?: number[];
+  metadata: {
+    filePath: string;
+    startLine: number;
+    endLine: number;
+    language: string;
+  };
+}
+
+interface SearchResult {
+  chunk: Chunk;
+  score: number; // 1 / (1 + L2_distance), range [0, 1]
+}
 ```
 
-The optional second argument overrides the chunker's `fileExtensions`. If a
-built-in chunker already covers the requested extension, the new registration is
-skipped and a warning is emitted.
+---
 
 ## Vector Store
 
-LanceDB stores chunks in a `chunks` table with columns: `id`, `content`,
-`embedding` (vector), `filePath`, `startLine`, `endLine`, `language`.
+LanceDB stores chunks in a `chunks` table with columns: `id`, `content`, `embedding` (vector), `filePath`, `startLine`, `endLine`, `language`.
+
+| Mode | Description |
+|------|-------------|
+| **Disk** | Files written to `vectorStore.path` (default: `.opencode/rag_db`) |
+| **Memory** | `memory://` URI — for tests only; data is lost when the process exits |
 
-- **Disk mode**: files in `vectorStore.path` (default `.opencode/rag_db`)
-- **Memory mode**: `memory://` URI — for tests only, data lost on close
-- **Manifest sidecar**: `manifest.json` in the store directory tracks indexed
-  files for incremental updates
+Additional details:
 - Schema is auto-inferred from a seed row on first table creation
-- L2 distance search, score = `1 / (1 + distance)`
-- Stored file paths are normalized to absolute forward-slash paths
+- L2 distance search; score = `1 / (1 + distance)`
+- File paths are normalized to absolute forward-slash paths
+- `manifest.json` in the store directory tracks indexed files for incremental updates
+
+---
 
 ## Development
 
@@ -458,34 +516,14 @@ npm run typecheck
 # Run all tests
 npm test
 
-# Run specific test file
+# Run a specific test file
 node --import tsx --test src/__tests__/chunker/fallback.test.ts
 ```
 
-Project structure:
-```
-  src/
-    core/          — interfaces.ts, config.ts
-    chunker/       — grammar.ts, base.ts, language chunkers, fallback.ts, factory.ts, uuid.ts
-    embedder/      — ollama.ts, openai.ts, factory.ts
-    vectorstore/   — lancedb.ts
-    retriever/     — retriever.ts
-    types/         — opencode-plugin.d.ts
-    indexer.ts     — incremental indexing + watch scheduling
-    watcher.ts     — background indexer (chokidar + debounced scheduler + periodic timer)
-    cli.ts, plugin.ts, plugin-entry.ts, index.ts
-    __tests__/     — mirrors the module structure
-```
-
-Test framework is Node's built-in runner (`node:test`) with `tsx` for TypeScript
-imports. No test library dependencies.
-
-## Limitations
+Tests use Node's built-in runner (`node:test`) with `tsx` for TypeScript imports. No external test library is required.
 
-- Embedding model dimension is auto-probed at startup; falls back to 384 if probing fails.
-- 21 built-in chunkers (AST for 16, regex for 4, PDF text for 1) + configurable fallback
+---
 
 ## Privacy
 
-All processing is local. Embeddings are generated via local Ollama by default.
-No data leaves the machine unless configured to use a remote embedding API.
+All processing runs locally. By default, embeddings are generated by Ollama on your own machine. **No data leaves your machine** unless you configure a remote embedding API.