opencode-rag-plugin 1.3.5 → 1.3.6

package/ReadMe.md

@@ -1,68 +1,40 @@
 # OpenCodeRAG
 
-> A RAG (Retrieval-Augmented Generation) plugin for [OpenCode](https://opencode.ai) that adds **semantic code search** powered by locally-hosted embedding models.
+OpenCodeRAG is a RAG (Retrieval-Augmented Generation) plugin for semantic code search powered by locally-hosted embedding models. 
+It features a seamless integration with [OpenCode](https://opencode.ai) and offers a CLI interface for external usage.
 
-**Published on npm: [`opencode-rag-plugin`](https://www.npmjs.com/package/opencode-rag-plugin)**
+[![npm version](https://img.shields.io/npm/v/opencode-rag-plugin.svg)](https://www.npmjs.com/package/opencode-rag-plugin)
 
-> ⚠️ **Do not confuse with `opencode-rag`** — that is a discontinued project by a different author.
+> ⚠️ **Note:** Don't confuse this with the npm-package `opencode-rag`, which is a discontinued project of a different author. Ensure you are using **`opencode-rag-plugin`**.
 
-OpenCodeRAG reduces token usage by replacing expensive file-read tool calls with targeted, vector-similarity-based chunk retrieval. Large codebases and files on a local dedicated GPU benefit most in terms of performance, but even a modern CPU handles most workloads without a GPU.
+**Why Using OpenCodeRAG?**  
+OpenCodeRAG aims to reduce LLM token usage by replacing expensive full-file reads with targeted, vector-similarity-based chunk retrieval. 
+Large codebases benefit massively in performance. It runs perfectly on a local dedicated GPU, but modern CPUs handle most workloads just fine. 
+When using only locally hosted embedding models and LLMs, your codebase also remains 100% private.
 
 ---
 
 ## Table of Contents
 
-- [Features](#features)
-- [Prerequisites](#prerequisites)
-- [Quick Start](#quick-start)
-- [Installation](#installation)
-  - [Global Installation (Recommended)](#global-installation-recommended)
-  - [Per-Workspace Setup](#per-workspace-setup)
-  - [Manual Installation](#manual-installation)
-  - [Uninstallation](#uninstallation)
-- [Configuration](#configuration)
-  - [Embedding Providers](#embedding-providers)
-  - [Logging](#logging)
-- [Usage](#usage)
-  - [CLI](#cli)
-  - [OpenCode Plugin](#opencode-plugin)
-- [Architecture](#architecture)
-- [Tech Stack](#tech-stack)
-- [Chunking Language Support](#chunking-language-support)
-  - [Adding Custom Chunkers](#adding-custom-chunkers)
-- [Data Model](#data-model)
-- [Vector Store](#vector-store)
-- [Development](#development)
-- [Privacy](#privacy)
+- [Quick Start](#-quick-start)
+- [How it Works](#-how-it-works)
+- [OpenCode Integration](#-opencode-integration)
+- [Configuration](#-configuration)
+- [CLI Usage (Standalone)](#-cli-interface)
+- [Supported Languages](#-supported-languages)
+- [Developer Guide & Architecture](#-developer-guide--architecture)
+- [Privacy & Security](#-privacy--security)
 
 ---
 
-## Features
-
-| Feature | Description |
-|---------|-------------|
-| **AST-aware chunking** | Splits code into functions, classes, and methods via tree-sitter for 16 languages; regex-based for Markdown, Razor, `.sln`, and LaTeX; line-based fallback for everything else |
-| **Incremental indexing** | Skips unchanged files, removes deleted entries, re-indexes only what changed |
-| **Watch mode** | `index --watch` debounces file changes and re-indexes automatically |
-| **Pluggable chunkers** | Add custom language chunkers via config file or programmatic API |
-| **Configurable embeddings** | Ollama (default, fully local) or any OpenAI-compatible provider; configurable batch size |
-| **Hybrid search** | TF×IDF keyword index fused with vector similarity — better precision on identifiers and function names |
-| **Local vector store** | LanceDB with L2 distance scoring; in-memory mode for testing |
-| **CLI** | `index`, `query`, `clear`, `status` commands |
-| **OpenCode integration** | Auto-injects relevant code chunks into each message; suggests related files; exposes a retrieval tool the agent can call directly |
-
----
-
-## Prerequisites
-
-- **[OpenCode](https://opencode.ai)** installed and configured
-- **Node.js v22+** (required for native ESM and `fetch`)
-- **[Ollama](https://ollama.ai)** running locally with an embedding model (default: `nomic-embed-text`)
-  — or any OpenAI-compatible embedding API
+## Quick Start
 
----
+### Prerequisites
+- **Node.js v22+** (required for native ESM and `fetch`).
+- **[Ollama](https://ollama.ai)** running locally (default model: `embeddinggemma`) OR an OpenAI-compatible API.
+- **[OpenCode](https://opencode.ai)** if you want to use the automated agent features.
 
-## Quick Start
+### Installation
 
 ```bash
 # 1. Clone the repository
@@ -73,371 +45,171 @@ cd OpenCodeRAG
 ./install.sh          # Linux/macOS
 .\install.ps1         # Windows
 
-# 3. Go to your project and initialize
+# 3. Initialize in your project
 cd /path/to/your/project
 opencode-rag init
 
-# 4. Index the workspace
-opencode-rag index
-
-# 5. Restart OpenCode — the plugin is active
-```
-
----
-
-## Installation
-
-### Global Installation (Recommended)
-
-Clone the repository and run the platform-specific install script:
-
-```bash
-# Linux/macOS
-./install.sh
-
-# Windows
-.\install.ps1
-```
-
-The script will:
-1. Build the plugin from source (`npm run build`)
-2. Install it into OpenCode's runtime (`~/.opencode/node_modules/`)
-3. Register it in the global OpenCode config (`~/.config/opencode/opencode.jsonc`)
-
-**Restart OpenCode after installation.** 
-
----
+# 4. Edit configuration file (optional), e.g.
+nano opencoderag.json
 
-### Per-Workspace Setup
-
-```bash
-cd your-workspace
-opencode-rag init
+# 4. Index your workspace
+opencode-rag index
 ```
 
-Bootstraps the current directory with all required files and a sample configuration.
-Add `--skip-install` to generate the files only, without installing dependencies.
+*(To uninstall, simply run `./install.sh uninstall` or `.\install.ps1 uninstall`)*
 
 ---
 
-### Manual Installation
+## How it Works
 
-If you prefer to build and install without the script:
-
-```bash
-npm run build
-npm pack
-
-# Install into OpenCode's runtime
-npm install --prefix ~/.opencode/ opencode-rag-plugin-1.2.0.tgz
-npm install --prefix ~/.config/opencode/ opencode-rag-plugin-1.2.0.tgz
-
-# Then add "opencode-rag-plugin" to the plugins array in:
-# ~/.config/opencode/opencode.jsonc
-```
+OpenCodeRAG intelligently processes your query in OpenCode and your codebase for high-precision retrieval:
+1. **AST-aware chunking:** Intelligently splits code into functions, methods, and classes using `tree-sitter`.
+2. **Local Vector Database:** Uses LanceDB to store embeddings locally without any cloud dependencies.
+3. **Hybrid Search:** Combines TF×IDF keyword matching with vector similarity for superior precision on identifiers.
+4. **Incremental Indexing:** Only updates files that have changed, ensuring that the vector database stays up-to-date.
 
 ---
 
-### Uninstallation
+## OpenCode Integration
 
-```bash
-# Linux/macOS
-./install.sh uninstall
+When are using OpenCode, the plugin enhances your agent with two main features:
 
-# Windows
-.\install.ps1 uninstall
-```
-
-**Restart OpenCode after uninstalling.**
+### 1. Auto-Injection (Background Context)
+After every message you send, the plugin effectively searches your vector-indexed codebase:
+- **High-confidence results (score ≥ 0.75):** Actual code chunks are injected directly into your prompt, giving the agent instant context without a tool-call round-trip.
+- **Lower-confidence results:** A compact list of suggested files is appended instead (e.g., `src/plugin.ts (lines 10-42)`).
 
-> **Automatic dependencies:** `apache-arrow` (LanceDB peer dependency) and `tree-sitter-wasm` (pre-built WASM grammars) are installed automatically.
+### 2. Direct Tool Call (`opencode-rag-context`)
+Any OpenCode agent can actively invoke the RAG tool to fetch specific code chunks, filter by exact file paths, or target specific programming languages.
 
 ---
 
 ## Configuration
 
-Create `opencode-rag.json` in your project root (auto-detected), or point to it explicitly with `--config ./path/to/config.json`.
-
-The repository's own [`opencode-rag.json`](./opencode-rag.json) is a complete, annotated example covering all available options.
-
-**Partial overrides are supported:** only the keys you set are applied — everything else falls back to defaults. Sections are deep-merged.
-
----
+Running `opencode-rag init` creates the config file `opencode-rag.json` in your project root. You only need to define the values you want to override.
 
 ### Embedding Providers
+```json
+{
+  "embedding": {
+    "provider": "ollama",
+    "baseUrl": "http://localhost:11434/api",
+    "timeoutMs": 30000
+  }
+}
+```
+* **Ollama (Default):** Local, private, no API key needed. *(Increase `timeoutMs` if your model has a slow cold start)*.
+* **OpenAI:** Set `provider` to `"openai"`, change `baseUrl` to `"https://api.openai.com/v1"`, and provide your `apiKey`.
 
-| Provider | `baseUrl` | Notes |
-|----------|-----------|-------|
-| `ollama` | `http://localhost:11434/api` | **Default.** No API key required. |
-| `openai` | `https://api.openai.com/v1` | Requires `apiKey`. Sends all texts in one batch request. |
-
-- `embedding.timeoutMs` defaults to `30000` ms. Increase it if your local model has a slow cold start.
-- Ollama sends one request per batch to `/api/embed`. Set `embedding.proxy.url` to route through a standard HTTP proxy.
+### Plugin Settings
 
----
+| Option | Default | Description |
+|--------|---------|-------------|
+| `openCode.readOverride` | `false` | If `true`, overrides OpenCode's default file reader to append RAG chunks as supplementary context. |
+| `openCode.autoInject.enabled` | `true` | Turn background auto-injection on/off. |
+| `openCode.autoInject.minScore` | `0.75` | Minimum relevance score to inject actual code (0–1). |
+| `retrieval.topK` | `10` | Default number of chunks fetched per query. |
+| `retrieval.hybridSearch.enabled` | `true` | Enables combined TF×IDF + vector search. |
 
-### Logging
+<details>
+<summary>View Logging Configuration</summary>
 
 ```json
 {
   "logging": {
-    "level": "info",
+    "level": "info", // Options: "debug", "info", or "error"
     "logFilePath": "./.opencode/opencode-rag.log"
   }
 }
 ```
+</details>
 
-| Option | Default | Description |
-|--------|---------|-------------|
-| `level` | `"info"` | `"debug"`, `"info"`, or `"error"` |
-| `logFilePath` | `"./.opencode/opencode-rag.log"` | Relative paths resolve against the workspace root |
+### `AGENTS.md` Setup
+To ensure the OpenCode agent knows how to leverage the plugin, add this snippet to your workspace's `AGENTS.md`:
 
-The `LOG_FILE_PATH` environment variable is a fallback when no config value is set. Config takes precedence over the env var.
+<details>
+<summary>Click to view the AGENTS.md snippet</summary>
 
----
+```markdown
+## OpenCodeRAG Plugin
+
+This workspace has OpenCodeRAG installed for semantic code retrieval.
 
-## Usage
+### `opencode-rag-context` tool
+Before planning, editing, or answering questions, use this tool to retrieve relevant code chunks with file paths, line ranges, and surrounding implementation.
+- `query` (required) — narrow, specific search, e.g. `"authentication middleware setup"`
+- `pathHints` (optional) — up to 10 path filters, e.g. `["src/auth/"]`
+- `languageHints` (optional) — up to 10 language filters, e.g. `["typescript"]`
+- `topK` (optional) — result count (1–25, default 10)
 
-OpenCodeRAG provides two interfaces:
+### File suggestions
+After each user message, a `chat.message` hook appends up to 10 relevant file suggestions. Look for lines like `src/file.ts (typescript, lines 10-42)` at the bottom of user input.
 
-| Interface | Best for |
-|-----------|----------|
-| **CLI** | Manual indexing, searching, and diagnostics |
-| **OpenCode plugin** | Automatic retrieval and file suggestions while chatting |
+### Indexing
+- Changed files are auto-indexed in the background.
+- If searches return no results, run `opencode-rag index` in the terminal.
+```
+</details>
 
 ---
 
-### CLI
+## CLI Interface
+
+The CLI interface (`opencode-rag`) provides full access to build, manage, and search your project's vector database, even without opening OpenCode.
+This is mainly intended for testing/debugging purposes, but can also be integrated into your own applications or scripts.
 
 ```bash
 # Index the workspace (incremental by default)
 opencode-rag index
 
-# Force a complete re-index
-opencode-rag index --force
-
-# Watch for changes and re-index automatically
+# Watch for file changes and re-index automatically
 opencode-rag index --watch
 
-# Semantic search
-opencode-rag query "How is authentication handled?"
-opencode-rag query "error handling" --top-k 5
+# Force a complete rebuild of the index
+opencode-rag index --force
 
-# Show index statistics
-opencode-rag status
+# Semantic search directly from the terminal
+opencode-rag query "How is authentication handled?" --top-k 5
 
-# Clear all indexed data
+# Show index statistics or clear data
+opencode-rag status
 opencode-rag clear
-
-# Use a custom config file
-opencode-rag index --config ./my-config.json
-```
-
-**Example `status` output:**
-```
-Indexed chunks:      1247
-Store path:          /home/user/project/.opencode/rag_db
-Embedding provider:  ollama
-Embedding model:     nomic-embed-text
-Manifest status:     ok
-Manifest entries:    42
-Last indexed:        2026-05-28 10:45:02
-Up-to-date files:    42
-Pending files:       0
-Keyword index:       enabled (1274 chunks)
-```
-
-#### How Incremental Indexing Works
-
-A manifest is stored at `<vectorStore.path>/manifest.json` and tracks file hashes, chunk counts, and the last successful index timestamp.
-
-- Only **changed files** are re-embedded; **deleted files** are removed from the store.
-- If the manifest is missing or corrupt while the store already has data, the next pass **clears and rebuilds** the entire store to prevent duplicates.
-
-#### Watch Mode
-
-```bash
-opencode-rag index --watch
 ```
 
-- Runs a full initial index pass, then listens for `add`, `change`, `unlink`, and `unlinkDir` events.
-- Changes are debounced (300 ms) before triggering an incremental pass.
-- If a pass is already running, one follow-up pass is queued and starts immediately after.
-- The watcher ignores excluded directories, the vector store path, and the manifest file itself.
-- Press `Ctrl+C` to stop.
-
 ---
 
-### OpenCode Plugin
-
-Once installed, the plugin registers two hooks:
-
-| Hook | What it does |
-|------|-------------|
-| `opencode-rag-context` | A retrieval tool the agent can call directly to fetch code chunks with file paths and line ranges |
-| `chat.message` | After each user message, retrieves relevant files and appends them to the message |
-
-#### Auto-Injection
+## Supported Languages
 
-After each message you send, the plugin:
+OpenCodeRAG uses advanced AST (Abstract Syntax Tree) parsing via `tree-sitter` for major languages, falling back to regex or line-based chunking for others.
 
-1. Extracts your message text
-2. Runs semantic retrieval against the indexed workspace
-3. **High-confidence results** (relevance ≥ `autoInject.minScore`, default `0.75`): injects the actual code chunks directly into the message — the agent gets context without a tool-call round-trip:
-   ```
-   ---
-   **Auto-retrieved code context** _(context: 2 chunks, 1 file, relevance 0.88–0.92)_
-   ---
-   [src/auth.ts:12-30] (typescript, score: 0.92)
-   ```typescript
-   function login() { ... }
-   ```
-   ---
-   ```
-4. **No high-confidence results**: falls back to a compact file list (max 10 files):
-   ```
-   src/plugin.ts (typescript, lines 10-42, relevance 0.87)
-   src/core/config.ts (typescript, lines 66-145, relevance 0.72)
-   ```
-
-This eliminates a tool-call round-trip for roughly 70% of code-related messages. The agent can still call `opencode-rag-context` for more targeted or additional context.
-
-> **Note:** Retrieval errors are silently caught — a failed search will never break the chat.
+* **AST Support:** TypeScript, JavaScript, Python, Java, Go, Rust, C, C++, C#, Ruby, Kotlin, Swift, JSON, XML, HTML, CSS.
+* **Regex / Structure Support:** Markdown, Razor, `.sln`, LaTeX, PDF, Word (`.docx`/`.doc`), Excel.
+* **Fallback:** Raw text blocks (100 lines/chunk).
 
 ---
 
-#### Plugin Configuration Reference
-
-| Option | Default | Description |
-|--------|---------|-------------|
-| `openCode.readOverride` | `false` | When `true`, registers a RAG-backed `read` tool that shadows OpenCode's built-in file reader — returns full file contents with relevant RAG chunks appended as supplementary context |
-| `openCode.maxContextChunks` | `5` | Maximum chunks returned by `opencode-rag-context` |
-| `openCode.autoInject.enabled` | `true` | Enable or disable auto-injection |
-| `openCode.autoInject.minScore` | `0.75` | Minimum relevance score for auto-injection (0–1) |
-| `openCode.autoInject.maxChunks` | `3` | Maximum chunks to inject per message |
-| `openCode.autoInject.maxTokens` | `2000` | Token budget for injected content (~4 chars/token) |
-| `retrieval.topK` | `10` | Chunks fetched per query (controls file suggestion breadth) |
-| `retrieval.hybridSearch.enabled` | `true` | Enable or disable hybrid TF×IDF + vector search |
-| `retrieval.hybridSearch.keywordWeight` | `0.4` | Keyword weight in hybrid fusion (0–1; higher = more keyword influence) |
-
----
-
-#### AGENTS.md Snippet
-
-Add this section to your workspace's `AGENTS.md` so the agent knows how to use the plugin:
-
-```markdown
-## OpenCodeRAG Plugin
-
-This workspace has OpenCodeRAG installed for semantic code retrieval.
-
-### `opencode-rag-context` tool
-Before planning, editing, or answering questions, use this tool to retrieve relevant
-code chunks with file paths, line ranges, and surrounding implementation.
-- `query` (required) — narrow, specific search, e.g. `"authentication middleware setup"`
-- `pathHints` (optional) — up to 10 path filters, e.g. `["src/auth/"]`
-- `languageHints` (optional) — up to 10 language filters, e.g. `["typescript"]`
-- `topK` (optional) — result count (1–25, default 10)
-
-### File suggestions
-After each user message, a `chat.message` hook appends up to 10 relevant file
-suggestions. Look for lines like `src/file.ts (typescript, lines 10-42)` at the
-bottom of user input.
-
-### Indexing
-- Changed files are auto-indexed in the background (debounced 5 s).
-- If searches return no results, the workspace may not be indexed yet —
-  run `opencode-rag index` from the terminal (or `npx opencode-rag-plugin`).
-- Files under 1 KB, excluded extensions, and excluded directories
-  (`node_modules`, `.git`, `.opencode`, `dist`, etc.) are silently skipped.
-```
-
-The plugin also injects a reminder about `opencode-rag-context` into the agent's system prompt via `experimental.chat.system.transform`.
-
----
+## Developer Guide
 
-## Architecture
+Developers are welcome! If you fixed a bug or implemented a new feature, just submit your pull request to this repository.
 
+### Architecture Flow
+```text
+Workspace Files ──> Chunker (tree-sitter/regex) ──> Chunks
+                      │
+Chunks ──> Embedder (Ollama/OpenAI) ──> Vectors
+                      │
+Vectors ──> VectorStore (LanceDB) <──> Indexer / Retriever ──> CLI / Agent Context
 ```
-Workspace Files
-      │
-      ▼
-┌──────────────┐
-│   Chunker    │  AST-based (tree-sitter) or line-based fallback
-└──────┬───────┘
-       │ chunks
-       ▼
-┌──────────────┐
-│   Embedder   │  Ollama / OpenAI-compatible API
-└──────┬───────┘
-       │ vectors
-       ▼
-┌──────────────┐
-│  VectorStore │  LanceDB (local files or memory:// for tests)
-└──────┬───────┘
-       │ + manifest.json
-       ▼
-┌──────────────┐
-│ Indexer/Retr.│  incremental index or query/search
-└──────┬───────┘
-       │ results
-       ▼
-   LLM Context
-```
-
----
-
-## Tech Stack
-
-| Layer | Technology |
-|-------|------------|
-| Runtime | Node.js v22 + tsx (ESM) |
-| Language | TypeScript 5.8 |
-| Chunking | web-tree-sitter (WASM) + tree-sitter-wasm grammars |
-| Embeddings | Ollama / OpenAI-compatible (native fetch) |
-| Vector DB | LanceDB (`@lancedb/lancedb`) |
-| CLI | commander |
-| Tests | Node built-in test runner (`node --test`) |
-| Package manager | npm (with `--legacy-peer-deps`) |
 
----
+### Tech Stack
+- **Runtime:** Node.js v22 + `tsx` (ESM), TypeScript 5.8
+- **Chunking:** `web-tree-sitter` (WASM) + grammars
+- **Vector DB:** LanceDB (`@lancedb/lancedb`) stored locally (`.opencode/rag_db`)
+- **Tests:** Node built-in test runner (`npm test`)
 
-## Chunking Language Support
-
-| Language | Strategy | Captures |
-|----------|----------|---------|
-| TypeScript | AST (tree-sitter) | functions, methods, classes, interfaces |
-| JavaScript | AST (tree-sitter) | functions, classes, arrow functions, exports |
-| Python | AST (tree-sitter) | functions, classes, decorated definitions |
-| Java | AST (tree-sitter) | methods, classes, interfaces, enums |
-| Go | AST (tree-sitter) | functions, methods, type declarations |
-| Rust | AST (tree-sitter) | functions, structs, enums, traits, impl blocks, modules, types |
-| C | AST (tree-sitter) | functions, structs, enums, unions, typedefs |
-| C++ | AST (tree-sitter) | functions, classes, structs, enums, namespaces, templates |
-| C# | AST (tree-sitter) | classes, interfaces, structs, enums, methods, namespaces, records |
-| Ruby | AST (tree-sitter) | methods, classes, modules, singleton methods |
-| Kotlin | AST (tree-sitter) | functions, classes, interfaces, objects, properties |
-| Swift | AST (tree-sitter) | functions, classes, structs, enums, protocols, extensions, variables |
-| JSON | AST (tree-sitter) | key-value pairs |
-| XML | AST (tree-sitter) | elements (1 chunk per root element) |
-| HTML | AST (tree-sitter) | `<script>` / `<style>` blocks |
-| CSS | AST (tree-sitter) | rule sets, at-rules, media, keyframes |
-| Razor | Regex (brace matching) | `@code` / `@functions` blocks, template regions |
-| Markdown | Regex heading split | h1/h2 sections + trailing content |
-| Solution (`.sln`) | Regex (section boundary) | project entries and global sections |
-| LaTeX | Regex section split | chapter/section/subsection/subsubsection boundaries |
-| PDF | Paragraph-based | groups small paragraphs, splits oversized ones |
-| Word (`.docx`) | Paragraph-based | extracts via mammoth; groups small paragraphs, splits oversized |
-| Word (`.doc`) | Paragraph-based | extracts via word-extractor; groups small paragraphs, splits oversized |
-| Excel (`.xls`/`.xlsx`) | Row-batch | extracts CSV per sheet via `@e965/xlsx`; splits by sheet, then by row batch |
-| *(other)* | Line-based (100 lines/chunk) | raw text blocks |
 
 ### Adding Custom Chunkers
-
-Custom chunkers can be registered without modifying the project source.
-
-**Via config file** — add to `opencode-rag.json`:
-
+You can easily inject custom chunkers without modifying the source code via `opencode-rag.json`:
 ```json
 {
   "chunkers": [
@@ -446,84 +218,10 @@ Custom chunkers can be registered without modifying the project source.
 }
 ```
 
-The module path is resolved relative to the config file. The module must export (as default or named) an object implementing the `Chunker` interface:
-
-```typescript
-interface Chunker {
-  readonly language: string;
-  readonly fileExtensions?: string[];
-  chunk(filePath: string, content: string): Promise<Chunk[]>;
-}
-```
-
-**Via programmatic API:**
-
-```typescript
-import { registerChunker } from "opencode-rag-plugin/library";
-registerChunker(myChunker, [".xyz"]); // second arg overrides fileExtensions
-```
-
-> If a built-in chunker already handles the requested extension, the new registration is skipped and a warning is emitted.
-
----
-
-## Data Model
-
-```typescript
-interface Chunk {
-  id: string;
-  content: string;
-  embedding?: number[];
-  metadata: {
-    filePath: string;
-    startLine: number;
-    endLine: number;
-    language: string;
-  };
-}
-
-interface SearchResult {
-  chunk: Chunk;
-  score: number; // 1 / (1 + L2_distance), range [0, 1]
-}
-```
-
 ---
 
-## Vector Store
-
-LanceDB stores chunks in a `chunks` table with columns: `id`, `content`, `embedding` (vector), `filePath`, `startLine`, `endLine`, `language`.
-
-| Mode | Description |
-|------|-------------|
-| **Disk** | Files written to `vectorStore.path` (default: `.opencode/rag_db`) |
-| **Memory** | `memory://` URI — for tests only; data is lost when the process exits |
-
-Additional details:
-- Schema is auto-inferred from a seed row on first table creation
-- L2 distance search; score = `1 / (1 + distance)`
-- File paths are normalized to absolute forward-slash paths
-- `manifest.json` in the store directory tracks indexed files for incremental updates
-
----
-
-## Development
-
-```bash
-# TypeScript typecheck
-npm run typecheck
-
-# Run all tests
-npm test
-
-# Run a specific test file
-node --import tsx --test src/__tests__/chunker/fallback.test.ts
-```
-
-Tests use Node's built-in runner (`node:test`) with `tsx` for TypeScript imports. No external test library is required.
-
----
+## Privacy & Security
 
-## Privacy
+**100% Local Processing by Default.**  
+Whether you use it via CLI or as an agent plugin, OpenCodeRAG honors privacy strictly. Embeddings are generated locally by Ollama, and the vector database (LanceDB) is stored right in your project's directory. **No source code or embeddings ever leave your machine** unless you explicitly configure a third-party API like OpenAI.
 
-All processing runs locally. By default, embeddings are generated by Ollama on your own machine. **No data leaves your machine** unless you configure a remote embedding API.

package/dist/chunker/pdf.js

@@ -1,13 +1,34 @@
 import { uuid } from "./uuid.js";
+import { createRequire } from "node:module";
+import { pathToFileURL } from "node:url";
+import path from "node:path";
 const MAX_CHUNK_CHARS = 4000;
 const MIN_GROUP_CHARS = 300;
 const PARAGRAPH_SPLIT = /\n\s*\n/;
+function getStandardFontsUrl() {
+    const require = createRequire(import.meta.url);
+    const pkgJson = require.resolve("pdfjs-dist/package.json");
+    const fontsDir = path.join(path.dirname(pkgJson), "standard_fonts");
+    const url = pathToFileURL(fontsDir).href;
+    return url.endsWith("/") ? url : url + "/";
+}
 async function createPdfDocument(buffer) {
-    const { DOMMatrix } = await import("canvas");
-    globalThis.DOMMatrix ??= DOMMatrix;
-    globalThis.DOMMatrixReadOnly ??= DOMMatrix;
+    try {
+        const { DOMMatrix } = await import("canvas");
+        globalThis.DOMMatrix ??= DOMMatrix;
+        globalThis.DOMMatrixReadOnly ??= DOMMatrix;
+    }
+    catch {
+        const { default: CSSMatrix } = await import("@thednp/dommatrix");
+        globalThis.DOMMatrix ??= CSSMatrix;
+        globalThis.DOMMatrixReadOnly ??= CSSMatrix;
+    }
     const { getDocument } = await import("pdfjs-dist/legacy/build/pdf.mjs");
-    const loadingTask = getDocument({ data: new Uint8Array(buffer) });
+    const loadingTask = getDocument({
+        data: new Uint8Array(buffer),
+        standardFontDataUrl: getStandardFontsUrl(),
+        verbosity: 0,
+    });
     return loadingTask.promise;
 }
 export async function extractPdfText(buffer) {

package/dist/chunker/pdf.js.map

@@ -1 +1 @@
-{"version":3,"file":"pdf.js","sourceRoot":"","sources":["../../src/chunker/pdf.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,IAAI,EAAE,MAAM,WAAW,CAAC;AAEjC,MAAM,eAAe,GAAG,IAAI,CAAC;AAC7B,MAAM,eAAe,GAAG,GAAG,CAAC;AAE5B,MAAM,eAAe,GAAG,SAAS,CAAC;AAElC,KAAK,UAAU,iBAAiB,CAAC,MAAc;IAC7C,MAAM,EAAE,SAAS,EAAE,GAAG,MAAM,MAAM,CAAC,QAAQ,CAAC,CAAC;IAC7C,UAAU,CAAC,SAAS,KAAK,SAAmD,CAAC;IAC7E,UAAU,CAAC,iBAAiB,KAAK,SAA2D,CAAC;IAE7F,MAAM,EAAE,WAAW,EAAE,GAAG,MAAM,MAAM,CAAC,iCAAiC,CAAC,CAAC;IACxE,MAAM,WAAW,GAAG,WAAW,CAAC,EAAE,IAAI,EAAE,IAAI,UAAU,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC;IAClE,OAAO,WAAW,CAAC,OAAO,CAAC;AAC7B,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,cAAc,CAAC,MAAc;IACjD,MAAM,GAAG,GAAG,MAAM,iBAAiB,CAAC,MAAM,CAAC,CAAC;IAC5C,MAAM,KAAK,GAAa,EAAE,CAAC;IAE3B,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,IAAI,GAAG,CAAC,QAAQ,EAAE,CAAC,EAAE,EAAE,CAAC;QACvC,MAAM,IAAI,GAAG,MAAM,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;QAClC,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,cAAc,EAAE,CAAC;QAC5C,MAAM,SAAS,GAAG,OAAO,CAAC,KAAK,CAAC,MAAM,CACpC,CAAC,IAAI,EAAE,EAAE,CAAC,OAAO,IAAI,KAAK,QAAQ,IAAI,IAAI,KAAK,IAAI,IAAI,KAAK,IAAI,IAAI,CAChD,CAAC;QACvB,MAAM,OAAO,GAAG,SAAS,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;QAClD,KAAK,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC;IAChC,CAAC;IAED,OAAO,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;AAC5B,CAAC;AAED,MAAM,OAAO,UAAU;IACZ,QAAQ,GAAG,KAAK,CAAC;IACjB,cAAc,GAAG,CAAC,MAAM,CAAC,CAAC;IAEnC,KAAK,CAAC,KAAK,CAAC,QAAgB,EAAE,OAAe;QAC3C,IAAI,OAAO,CAAC,IAAI,EAAE,CAAC,MAAM,KAAK,CAAC;YAAE,OAAO,EAAE,CAAC;QAE3C,MAAM,UAAU,GAAG,OAAO,CAAC,KAAK,CAAC,eAAe,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;QACrF,IAAI,UAAU,CAAC,MAAM,KAAK,CAAC;YAAE,OAAO,EAAE,CAAC;QAEvC,MAAM,MAAM,GAAY,EAAE,CAAC;QAC3B,IAAI,YAAY,GAAa,EAAE,CAAC;QAChC,IAAI,WAAW,GAAG,CAAC,CAAC;QACpB,IAAI,cAAc,GAAG,CAAC,CAAC;QAEvB,SAAS,KAAK;YACZ,MAAM,IAAI,GAAG,YAAY,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,CAAC;YAC9C,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC;gBAAE,OAAO;YAC9B,MAAM,CAAC,IAAI,CAAC;gBACV,EAAE,EAAE,IAAI,EAAE;gBACV,OAAO,EAAE,IAAI;gBACb,QAAQ,EAAE;oBACR,QAAQ;oBACR,SAAS,EAAE,cAAc,GAAG,YAAY,CAAC,MAAM,GAAG,CAAC;oBACnD,OAAO,EAAE,cAAc;oBACvB,QAAQ,EAAE,KAAK;iBAChB;aACF,CAAC,CAAC;YACH,YAAY,GAAG,EAAE,CAAC;YAClB,WAAW,GAAG,CAAC,CAAC;QAClB,CAAC;QAED,KAAK,MAAM,IAAI,IAAI,UAAU,EAAE,CAAC;YAC9B,cAAc,EAAE,CAAC;YACjB,MAAM,OAAO,GAAG,IAAI,CAAC,MAAM,CAAC;YAE5B,IAAI,OAAO,GAAG,eAAe,EAAE,CAAC;gBAC9B,IAAI,YAAY,CAAC,MAAM,GAAG,CAAC;oBAAE,KAAK,EAAE,CAAC;gBACrC,MAAM,CAAC,IAAI,CAAC;oBACV,EAAE,EAAE,IAAI,EAAE;oBACV,OAAO,EAAE,IAAI;oBACb,QAAQ,EAAE;wBACR,QAAQ;wBACR,SAAS,EAAE,cAAc;wBACzB,OAAO,EAAE,cAAc;wBACvB,QAAQ,EAAE,KAAK;qBAChB;iBACF,CAAC,CAAC;gBACH,SAAS;YACX,CAAC;YAED,IAAI,YAAY,CAAC,MAAM,GAAG,CAAC,IAAI,WAAW,GAAG,OAAO,GAAG,eAAe,EAAE,CAAC;gBACvE,KAAK,EAAE,CAAC;YACV,CAAC;YAED,YAAY,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;YACxB,WAAW,IAAI,OAAO,CAAC;YAEvB,IAAI,WAAW,IAAI,eAAe,IAAI,YAAY,CAAC,MAAM,IAAI,CAAC,EAAE,CAAC;gBAC/D,MAAM,kBAAkB,GACtB,cAAc,GAAG,UAAU,CAAC,MAAM;oBAClC,UAAU,CAAC,cAAc,CAAE,CAAC,MAAM,GAAG,eAAe,CAAC;gBACvD,IAAI,CAAC,kBAAkB,EAAE,CAAC;oBACxB,KAAK,EAAE,CAAC;gBACV,CAAC;YACH,CAAC;QACH,CAAC;QAED,IAAI,YAAY,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAC5B,KAAK,EAAE,CAAC;QACV,CAAC;QAED,OAAO,MAAM,CAAC;IAChB,CAAC;CACF;AAED,MAAM,CAAC,MAAM,UAAU,GAAG,IAAI,UAAU,EAAE,CAAC"}
+{"version":3,"file":"pdf.js","sourceRoot":"","sources":["../../src/chunker/pdf.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,IAAI,EAAE,MAAM,WAAW,CAAC;AACjC,OAAO,EAAE,aAAa,EAAE,MAAM,aAAa,CAAC;AAC5C,OAAO,EAAE,aAAa,EAAE,MAAM,UAAU,CAAC;AACzC,OAAO,IAAI,MAAM,WAAW,CAAC;AAE7B,MAAM,eAAe,GAAG,IAAI,CAAC;AAC7B,MAAM,eAAe,GAAG,GAAG,CAAC;AAE5B,MAAM,eAAe,GAAG,SAAS,CAAC;AAElC,SAAS,mBAAmB;IAC1B,MAAM,OAAO,GAAG,aAAa,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;IAC/C,MAAM,OAAO,GAAG,OAAO,CAAC,OAAO,CAAC,yBAAyB,CAAC,CAAC;IAC3D,MAAM,QAAQ,GAAG,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,OAAO,CAAC,EAAE,gBAAgB,CAAC,CAAC;IACpE,MAAM,GAAG,GAAG,aAAa,CAAC,QAAQ,CAAC,CAAC,IAAI,CAAC;IACzC,OAAO,GAAG,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,GAAG,GAAG,GAAG,CAAC;AAC7C,CAAC;AAED,KAAK,UAAU,iBAAiB,CAAC,MAAc;IAC7C,IAAI,CAAC;QACH,MAAM,EAAE,SAAS,EAAE,GAAG,MAAM,MAAM,CAAC,QAAQ,CAAC,CAAC;QAC7C,UAAU,CAAC,SAAS,KAAK,SAAmD,CAAC;QAC7E,UAAU,CAAC,iBAAiB,KAAK,SAA2D,CAAC;IAC/F,CAAC;IAAC,MAAM,CAAC;QACP,MAAM,EAAE,OAAO,EAAE,SAAS,EAAE,GAAG,MAAM,MAAM,CAAC,mBAAmB,CAAC,CAAC;QACjE,UAAU,CAAC,SAAS,KAAK,SAAmD,CAAC;QAC7E,UAAU,CAAC,iBAAiB,KAAK,SAA2D,CAAC;IAC/F,CAAC;IAED,MAAM,EAAE,WAAW,EAAE,GAAG,MAAM,MAAM,CAAC,iCAAiC,CAAC,CAAC;IACxE,MAAM,WAAW,GAAG,WAAW,CAAC;QAC9B,IAAI,EAAE,IAAI,UAAU,CAAC,MAAM,CAAC;QAC5B,mBAAmB,EAAE,mBAAmB,EAAE;QAC1C,SAAS,EAAE,CAAC;KACb,CAAC,CAAC;IACH,OAAO,WAAW,CAAC,OAAO,CAAC;AAC7B,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,cAAc,CAAC,MAAc;IACjD,MAAM,GAAG,GAAG,MAAM,iBAAiB,CAAC,MAAM,CAAC,CAAC;IAC5C,MAAM,KAAK,GAAa,EAAE,CAAC;IAE3B,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,IAAI,GAAG,CAAC,QAAQ,EAAE,CAAC,EAAE,EAAE,CAAC;QACvC,MAAM,IAAI,GAAG,MAAM,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;QAClC,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,cAAc,EAAE,CAAC;QAC5C,MAAM,SAAS,GAAG,OAAO,CAAC,KAAK,CAAC,MAAM,CACpC,CAAC,IAAI,EAAE,EAAE,CAAC,OAAO,IAAI,KAAK,QAAQ,IAAI,IAAI,KAAK,IAAI,IAAI,KAAK,IAAI,IAAI,CAChD,CAAC;QACvB,MAAM,OAAO,GAAG,SAAS,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;QAClD,KAAK,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC;IAChC,CAAC;IAED,OAAO,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;AAC5B,CAAC;AAED,MAAM,OAAO,UAAU;IACZ,QAAQ,GAAG,KAAK,CAAC;IACjB,cAAc,GAAG,CAAC,MAAM,CAAC,CAAC;IAEnC,KAAK,CAAC,KAAK,CAAC,QAAgB,EAAE,OAAe;QAC3C,IAAI,OAAO,CAAC,IAAI,EAAE,CAAC,MAAM,KAAK,CAAC;YAAE,OAAO,EAAE,CAAC;QAE3C,MAAM,UAAU,GAAG,OAAO,CAAC,KAAK,CAAC,eAAe,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;QACrF,IAAI,UAAU,CAAC,MAAM,KAAK,CAAC;YAAE,OAAO,EAAE,CAAC;QAEvC,MAAM,MAAM,GAAY,EAAE,CAAC;QAC3B,IAAI,YAAY,GAAa,EAAE,CAAC;QAChC,IAAI,WAAW,GAAG,CAAC,CAAC;QACpB,IAAI,cAAc,GAAG,CAAC,CAAC;QAEvB,SAAS,KAAK;YACZ,MAAM,IAAI,GAAG,YAAY,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,CAAC;YAC9C,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC;gBAAE,OAAO;YAC9B,MAAM,CAAC,IAAI,CAAC;gBACV,EAAE,EAAE,IAAI,EAAE;gBACV,OAAO,EAAE,IAAI;gBACb,QAAQ,EAAE;oBACR,QAAQ;oBACR,SAAS,EAAE,cAAc,GAAG,YAAY,CAAC,MAAM,GAAG,CAAC;oBACnD,OAAO,EAAE,cAAc;oBACvB,QAAQ,EAAE,KAAK;iBAChB;aACF,CAAC,CAAC;YACH,YAAY,GAAG,EAAE,CAAC;YAClB,WAAW,GAAG,CAAC,CAAC;QAClB,CAAC;QAED,KAAK,MAAM,IAAI,IAAI,UAAU,EAAE,CAAC;YAC9B,cAAc,EAAE,CAAC;YACjB,MAAM,OAAO,GAAG,IAAI,CAAC,MAAM,CAAC;YAE5B,IAAI,OAAO,GAAG,eAAe,EAAE,CAAC;gBAC9B,IAAI,YAAY,CAAC,MAAM,GAAG,CAAC;oBAAE,KAAK,EAAE,CAAC;gBACrC,MAAM,CAAC,IAAI,CAAC;oBACV,EAAE,EAAE,IAAI,EAAE;oBACV,OAAO,EAAE,IAAI;oBACb,QAAQ,EAAE;wBACR,QAAQ;wBACR,SAAS,EAAE,cAAc;wBACzB,OAAO,EAAE,cAAc;wBACvB,QAAQ,EAAE,KAAK;qBAChB;iBACF,CAAC,CAAC;gBACH,SAAS;YACX,CAAC;YAED,IAAI,YAAY,CAAC,MAAM,GAAG,CAAC,IAAI,WAAW,GAAG,OAAO,GAAG,eAAe,EAAE,CAAC;gBACvE,KAAK,EAAE,CAAC;YACV,CAAC;YAED,YAAY,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;YACxB,WAAW,IAAI,OAAO,CAAC;YAEvB,IAAI,WAAW,IAAI,eAAe,IAAI,YAAY,CAAC,MAAM,IAAI,CAAC,EAAE,CAAC;gBAC/D,MAAM,kBAAkB,GACtB,cAAc,GAAG,UAAU,CAAC,MAAM;oBAClC,UAAU,CAAC,cAAc,CAAE,CAAC,MAAM,GAAG,eAAe,CAAC;gBACvD,IAAI,CAAC,kBAAkB,EAAE,CAAC;oBACxB,KAAK,EAAE,CAAC;gBACV,CAAC;YACH,CAAC;QACH,CAAC;QAED,IAAI,YAAY,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAC5B,KAAK,EAAE,CAAC;QACV,CAAC;QAED,OAAO,MAAM,CAAC;IAChB,CAAC;CACF;AAED,MAAM,CAAC,MAAM,UAAU,GAAG,IAAI,UAAU,EAAE,CAAC"}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "opencode-rag-plugin",
-  "version": "1.3.5",
+  "version": "1.3.6",
   "description": "OpenCode plugin for local-first RAG-based semantic code search",
   "type": "module",
   "main": "./dist/plugin-entry.js",
@@ -72,6 +72,7 @@
     "@e965/xlsx": "^0.20.3",
     "@lancedb/lancedb": "^0.29.0",
     "apache-arrow": "18.1.0",
+    "@thednp/dommatrix": "^3.0.4",
     "canvas": "^3.2.3",
     "chokidar": "^4.0.3",
     "commander": "^13.1.0",