codesearch-mcp 0.0.18-alpha → 0.1.0-alpha

package/CHANGELOG.md

@@ -5,6 +5,29 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 ## [Unreleased]
 
+## [0.1.0-alpha] - 2026-07-12
+
+### Added
+
+- **`infinity` embedding provider** — a first-class local backend (`embedding.provider = infinity`) that serves both `/embeddings` and `/rerank` from a single [Infinity](https://github.com/michaelfeil/infinity) container (e.g. `jinaai/jina-embeddings-v2-base-code` + `Alibaba-NLP/gte-reranker-modernbert-base`), so local reranking works out of the box — something Ollama does not provide. Reuses the OpenAI-compatible HTTP path and adds only container-readiness behavior. Selecting `infinity` in `rag-setup` optionally provisions a reranker model and auto-wires `reranker.*` to the same server. Managed provisioning vs. an external/already-running server is decided by a `/health` probe (no config flag) — the same "already running?" probe now also defaults `rag-setup`'s "start Docker?" prompt to no for a reachable Ollama. The managed container prefers an NVIDIA GPU and falls back to CPU, and is recreated when the configured model set changes (Infinity binds models at startup). The managed container also gets `restart: unless-stopped` (survives host reboot) and disables Infinity's model warmup on CPU (a multi-minute benchmark at every boot; kept on for GPU where it is cheap). `jinaai/jina-embeddings-v2-base-code` (768 dims, the Infinity default) and `BAAI/bge-m3` (1024) are added to the model registry so their dimension resolves without a probe.
+- **Reranking is `openai-compatible`-only** — `reranker.provider` no longer accepts `ollama` (Ollama has no rerank API — the previous `/api/rerank` path always 404'd and silently fell back to vector order); the only valid backend is an OpenAI-compatible `/rerank` server (Infinity, TEI, vLLM). The default provider is now `openai-compatible`, and an existing `reranker.provider: 'ollama'` is migrated automatically on read. The embedding provider stays independent of the rerank server, so Ollama embeddings + an Infinity reranker is a valid combination.
+- **Pluggable embedding backends** — the embedder is now selected by `embedding.provider` via a factory, with three implementations: `ollama` (native `/api/embed`, the **backward-compatible default**), `openai-compatible` (one code path covering **OpenAI cloud, Infinity, TEI, vLLM, LM Studio, LocalAI** via a configurable `embedding.baseURL`), and `voyage` (native schema with mandatory `input_type`). To run a faster local backend or a cloud embedder you now edit config, not code.
+- **Batch embedding with retry** — `IEmbedder` gains `embedBatch(texts, inputType?)`; indexing sends one request per batch (`embedding.batchSize`, default 32) instead of one call per chunk, with bounded exponential backoff on HTTP 429/5xx. Makes cloud/Infinity indexing of a whole monorepo practical.
+- **Query/document input-type asymmetry** — `embed`/`embedBatch` take an optional `inputType` (`query` | `document`); search embeds as `query`, indexing as `document`. Voyage forwards it as `input_type`; Ollama and the OpenAI-compatible `/embeddings` endpoint ignore it (OpenAI has no such field).
+- **Flexible vector dimension** — the `code_chunk.embedding` column width now derives from the active model (`embedding.dimensions` override → model registry → one-shot runtime probe) instead of a hardcoded 1024, capped at ≤ 2000 (pgvector HNSW limit); larger cloud models reduce via the Matryoshka `dimensions`/`output_dimension` request parameter.
+- **`index_meta` source-of-truth table + mismatch guard** — a single-row table records `{ provider, model, dimensions }`. `search_codebase` (before embedding a query) and `rag-mcp` (at start) refuse with a clear "run `rag-index --reindex`" error when the configured model disagrees with the indexed one, instead of returning noise from a foreign vector space. A dimension change drops+recreates `code_chunk` at the new width on re-index; a same-dimension model change clears its rows.
+- **API-key secrets via `.env`** — keys live in `~/.config/codesearch-mcp/.env` (chmod 600), loaded into the environment at startup before config is read (a real env var wins over the file), so the client-spawned `rag-mcp` sees them too. Config values support an `env:VAR_NAME` reference; a missing referenced variable fails with a clear error. The dashboard shows `✔ set via <VAR>` for `env:` refs and masks raw keys.
+- **Auto-pull of local Ollama models with live progress** — `rag-index` now pulls the configured Ollama model if it is not present locally (only for `provider: ollama`; a no-op for cloud providers), so switching to (or back to) a local model that was never pulled just works without re-running `rag-setup`. The pull streams progress: a live single-line bar on a TTY (`downloading [██████░░░░] 62% (245.7/393.2 MB)`), de-duplicated status lines otherwise. `rag-setup`'s in-container pull now renders Ollama's native progress bar (allocates a TTY via `docker exec -t` when interactive) and pulls the model you actually chose (previously hardcoded to `bge-m3`).
+- **Cloud providers skip the Ollama container** — when a non-Ollama provider is selected, `rag-setup`'s Docker helper starts only the database service (`docker compose up -d db`) and skips the Ollama wait + model pull, instead of bringing up an unused Ollama container.
+- **Owner-symbol breadcrumb in search results** — each `search_codebase` result now shows a `Symbol:` line naming the declaration the chunk belongs to (`IndexerService.ingestFile`, `collectFiles`, …), so the enclosing class/method is visible without opening the file. Computed at index time into a new nullable `code_chunk.ownerPath` column (added via `ADD COLUMN IF NOT EXISTS`): the deepest declaration containing the chunk's start line wins, so a method beats its enclosing class. Method/constructor/accessor ranges are extracted purely for this breadcrumb and are **not** written to `code_symbol`, so `find_symbol` is unchanged. Module-scope chunks (imports) and rows indexed before this release show no breadcrumb; re-index to populate. No embedding-dimension change.
+- **AST-aware chunking** — TS/JS files are now chunked along top-level declaration boundaries (functions, classes, interfaces, types, exported variables) instead of at fixed character offsets, so a declaration is no longer split across chunks and search-hit line ranges map to coherent units of code. Declarations larger than `indexing.chunkSize` and the gaps between them (imports, top-level code) are sub-split by the recursive splitter, so no chunk exceeds the size limit and no source line is dropped. Reuses the symbol ranges already produced during indexing — no extra parse. New config key `indexing.chunkStrategy` (`ast` \| `character`, default `ast`); set it to `character` to restore the previous fixed-size behavior. Schema files (`.prisma`/`.graphql`/`.gql`) and files without symbol analysis keep the character splitter. Takes effect on the next re-index; no embedding-dimension or DB-schema change.
+
+### Changed (breaking)
+
+- **Config restructure** — the `ollama.*` section is replaced by `embedding.*` (`provider`, `baseURL`, `apiKey`, `model`, `dimensions`, `batchSize`); `reranker.*` gains `provider`/`baseURL`/`apiKey` mirroring the embedding taxonomy (Infinity can serve both from one server). **Legacy `ollama.host`/`ollama.model` configs are migrated to `embedding.*` automatically on read**, so existing installs keep working without manual edits.
+- **Query translation removed** — the `IEmbedder.translate` method and the `search.autoTranslate` / `search.translateModel` config keys are gone; queries are embedded as-is. The keys are silently ignored if still present.
+- **Re-index required on model/dimension change** — switching the embedding model, provider, or dimension now requires `rag-index --reindex` (a dimension change recreates `code_chunk`). Staying on Ollama + `bge-m3` keeps dimension 1024, backfills `index_meta`, and forces no re-index. The mismatch guard makes any disagreement loud rather than silently corrupting results.
+
 ## [0.0.18-alpha] - 2026-07-04
 
 ### Added

package/README.md

@@ -35,7 +35,7 @@ rag-setup
 
 Walks you through:
 - **Database** — enter connection details; optionally starts Docker containers automatically
-- **Ollama** — configure API URL
+- **Embedding backend** — pick a provider (`ollama` / `infinity` / `openai-compatible` / `voyage`), its model, and an API key if the provider needs one (see [Embedding providers](#embedding-providers))
 - **Logging** — choose log level and directory (defaults: `error` level, `~/.config/codesearch-mcp/logs/`)
 
 Config is saved to `~/.config/codesearch-mcp/config.json`.
@@ -118,7 +118,9 @@ After each indexing run an interactive `project-graph.html` is written to the wo
 Pass `include_graph_context: true` to `search_codebase` to see direct imports and dependents inline with each result — without extra `get_dependencies` / `get_dependents` calls:
 
 ```
-**File:** src/services/InvoiceService.ts  **Line:** 45
+**File:** src/services/InvoiceService.ts
+**Symbol:** InvoiceService.create
+**Line:** 45
 ...code...
 **Relevance:** 0.32
 **Imports:** src/repositories/InvoiceRepo.ts · src/dto/CreateInvoiceDto.ts
@@ -127,6 +129,8 @@ Pass `include_graph_context: true` to `search_codebase` to see direct imports an
 
 Useful for immediately understanding a file's role in the dependency graph.
 
+Each result also shows a **Symbol** breadcrumb — the declaration the chunk belongs to (`InvoiceService.create`, `collectFiles`, …) — so you can see the enclosing class/method without opening the file. It is omitted for module-scope chunks and for projects indexed before this feature (re-index to populate).
+
 ## Excluding files
 
 Create `.ragignore` in a project root — same syntax as `.gitignore`:
@@ -172,15 +176,80 @@ rag-config reset <key>               # remove key (reverts to default)
 | `db.password` | — | PostgreSQL password |
 | `db.name` | `rag_<os-username>` | Database name — auto-created on first run |
 | `workspaceRoot` | — | Root of your projects directory |
-| `ollama.host` | `http://localhost:11434` | Ollama API URL |
-| `ollama.model` | `bge-m3` | Embedding model |
+| `embedding.provider` | `ollama` | Embedding backend: `ollama` \| `infinity` \| `openai-compatible` \| `voyage` |
+| `embedding.baseURL` | per-provider | Backend base URL (ollama `http://localhost:11434`, OpenAI `https://api.openai.com/v1`, Infinity/TEI/vLLM `http://localhost:7997`, Voyage `https://api.voyageai.com/v1`) |
+| `embedding.apiKey` | `''` | API key, or `env:VAR_NAME` to read from `.env`/environment. Empty for local servers |
+| `embedding.model` | `bge-m3` | Embedding model |
+| `embedding.dimensions` | (derived) | Vector width override; required for models whose native width exceeds 2000 (Matryoshka). Otherwise derived from model → registry → runtime probe |
+| `embedding.batchSize` | `32` | Texts per batch embedding request |
 | `search.similarityThreshold` | `0.6` | Max cosine distance — raise if search returns nothing |
-| `search.autoTranslate` | `false` | Translate non-English queries to English before embedding |
-| `search.translateModel` | `llama3.2` | Ollama model for translation (requires `ollama pull llama3.2`) |
+| `reranker.enabled` | `false` | Enable cross-encoder reranking; requires `reranker.model` |
+| `reranker.provider` | `openai-compatible` | Rerank backend — `openai-compatible` `/rerank` only (Infinity, TEI, vLLM). Ollama has no rerank API |
+| `reranker.baseURL` | `embedding.baseURL` | Rerank server base URL (Infinity serves embed + rerank from one server) |
+| `reranker.model` | `''` | Reranker model name (e.g. `qllama/bge-reranker-v2-m3`) |
 | `indexing.astBatchSize` | `3000` | Max files per ts-morph batch — lower if indexing large projects causes OOM |
+| `indexing.chunkStrategy` | `ast` | How TS/JS files are chunked: `ast` = boundaries aligned to functions/classes, `character` = fixed-size splitter. Re-index to apply |
 | `logging.level` | `error` | Log level written to file: `error` \| `warn` \| `info` \| `debug` |
 | `logging.dir` | `~/.config/codesearch-mcp/logs` | Base log directory, or `"false"` to disable |
 
+## Embedding providers
+
+`embedding.provider` selects the backend. All four produce vectors the same way; they differ in where they run and which models they offer.
+
+### Choosing a provider
+
+| Provider | When to use | Needs |
+|----------|-------------|-------|
+| `ollama` | Default. Local, zero-setup, general-purpose (`bge-m3`); auto-pulls the model. | Ollama running (Docker or native) |
+| `infinity` | Local, best quality **plus a real reranker** in one container; GPU or CPU. | Docker (managed) or a running Infinity server |
+| `openai-compatible` | OpenAI cloud, or any OpenAI-style server (TEI, vLLM, LM Studio, LocalAI). | `baseURL` (+ `apiKey` for cloud) |
+| `voyage` | Voyage AI cloud, code-specialized embeddings. | Voyage `apiKey` |
+
+Easiest path: `rag-setup` asks for the provider, model, and any API key. Haven't run setup? It's the place to start. To change things later, edit config directly (below).
+
+### Switching provider or changing the model
+
+Any change to the provider, model, or dimension requires a **re-index** — the mismatch guard blocks search until you do it. The index is a rebuildable cache, so there's no data migration.
+
+```bash
+# Switch provider (example: OpenAI cloud)
+rag-config set embedding.provider openai-compatible
+rag-config set embedding.baseURL  https://api.openai.com/v1
+rag-config set embedding.model    text-embedding-3-small
+rag-config set embedding.apiKey   env:OPENAI_API_KEY     # key lives in .env, not config.json
+rag-index --reindex
+
+# Change only the model on the current provider
+rag-config set embedding.model <model-name>
+rag-index --reindex
+```
+
+Per-provider notes:
+
+- **ollama** — `rag-index` auto-pulls the model if it isn't present locally (`bge-m3`, `nomic-embed-text`, `mxbai-embed-large`, …). No API key. Change model: `rag-config set embedding.model nomic-embed-text`.
+- **infinity** — the model set is bound at container startup, so changing `embedding.model` (or the reranker model) **recreates** the managed container on the next `rag-index`. Defaults: `jinaai/jina-embeddings-v2-base-code` + reranker `Alibaba-NLP/gte-reranker-modernbert-base`.
+- **openai-compatible** — point `embedding.baseURL` at the server; set `embedding.apiKey` (as `env:VAR`) for cloud, leave empty for local servers (TEI/vLLM/LM Studio). E.g. `text-embedding-3-small`, `text-embedding-3-large`.
+- **voyage** — set `embedding.apiKey`; e.g. `voyage-code-3`, `voyage-3`.
+
+The vector **dimension** is derived automatically (model registry → one-shot probe); set `embedding.dimensions` only to force a Matryoshka-reduced width for models whose native size exceeds 2000.
+
+### Infinity — local embeddings + reranking in one container
+
+Select `infinity` in `rag-setup` (or set `embedding.provider = infinity`) to run [Infinity](https://github.com/michaelfeil/infinity), which serves both `/embeddings` and `/rerank` from a single container — so you get a real cross-encoder reranker locally, which Ollama does not offer. During setup you're asked for an embedding model (default `jinaai/jina-embeddings-v2-base-code`, 768 dims) and an optional reranker model (default `Alibaba-NLP/gte-reranker-modernbert-base`); when a reranker is given, `reranker.*` is auto-wired to the same server.
+
+Managed vs. external is decided by a `/health` probe at `embedding.baseURL` (default `http://localhost:7997`) — no config flag:
+
+- **Managed** — if nothing answers the probe, `rag-setup` (and `rag-index`) bring up the bundled `infinity` Docker service with your model set. It **prefers an NVIDIA GPU and falls back to CPU** automatically (GPU layered via `docker-compose.infinity-gpu.yml` only when `nvidia-smi` is present). Because Infinity binds its models at startup, changing `embedding.model`/reranker model **recreates** the container.
+- **External** — if a server already answers (your own Infinity, remote host, CI), it is used as-is and no Docker command runs. Point `embedding.baseURL` at it.
+
+The same "already running?" probe now also applies to Ollama: a reachable backend defaults `rag-setup`'s "start Docker?" prompt to no.
+
+### Secrets & external embedders
+
+API keys live in `~/.config/codesearch-mcp/.env` (chmod 600), **not** in `config.json`. Reference them with `env:VAR_NAME`, e.g. `embedding.apiKey = env:OPENAI_API_KEY` with `OPENAI_API_KEY=sk-…` in `.env`. The file is loaded into the environment at startup (a real environment variable wins over the file), so all binaries — including the client-spawned `rag-mcp` — see it identically.
+
+Switching the embedding model, provider, or dimension is applied by **re-indexing** (`rag-index --reindex`) — the index is a rebuildable cache, so there is no data migration. A single-row `index_meta` table records which model built the index; `search_codebase` and `rag-mcp` refuse with a clear "re-index required" error when the configured model disagrees, rather than returning noise from a foreign vector space. Staying on Ollama + `bge-m3` keeps dimension 1024 and forces no re-index.
+
 ## License
 
 MIT

package/dist/app.js

@@ -34,17 +34,53 @@ var __importStar = (this && this.__importStar) || (function () {
     };
 })();
 Object.defineProperty(exports, "__esModule", { value: true });
+const fs = __importStar(require("fs"));
 const path = __importStar(require("path"));
 const help_1 = require("./src/help");
 const container_1 = require("./src/container");
 const config_1 = require("./src/config");
+const config_store_1 = require("./src/config-store");
 const IndexCommand_1 = require("./src/commands/IndexCommand");
 const McpCommand_1 = require("./src/commands/McpCommand");
 const BackfillCommand_1 = require("./src/commands/BackfillCommand");
 const ConfigCommand_1 = require("./src/commands/ConfigCommand");
 const WebCommand_1 = require("./src/commands/WebCommand");
 const setup_1 = require("./src/setup");
+/**
+ * Load `~/.config/codesearch-mcp/.env` into `process.env` before config is read.
+ * All five binaries share this file — `rag-mcp` in particular is spawned by the
+ * MCP client and does not inherit shell exports. A real `process.env` value
+ * always wins over the file. Minimal `KEY=VALUE` parser (no `dotenv` dep):
+ * skips blanks and `#` comments, strips one layer of surrounding quotes.
+ */
+function loadDotEnv(file) {
+    let text;
+    try {
+        text = fs.readFileSync(file, 'utf-8');
+    }
+    catch {
+        return; // absent / unreadable → no-op
+    }
+    for (const rawLine of text.split(/\r?\n/)) {
+        const line = rawLine.trim();
+        if (!line || line.startsWith('#'))
+            continue;
+        const eq = line.indexOf('=');
+        if (eq === -1)
+            continue;
+        const key = line.slice(0, eq).trim();
+        if (!key)
+            continue;
+        let val = line.slice(eq + 1).trim();
+        if (val.length >= 2 && ((val[0] === '"' && val.at(-1) === '"') || (val[0] === "'" && val.at(-1) === "'"))) {
+            val = val.slice(1, -1);
+        }
+        if (!(key in process.env))
+            process.env[key] = val; // real env wins
+    }
+}
 async function main() {
+    loadDotEnv(config_store_1.ENV_FILE);
     const argv1 = path.basename(process.argv[1] ?? '');
     const isIndex = argv1 === 'rag-index' || process.argv[2] === 'index' || process.argv[2] === 'rag-index';
     const isMcp = argv1 === 'rag-mcp' || process.argv[2] === 'mcp' || process.argv[2] === 'rag-mcp';

package/dist/src/commands/IndexCommand.js

@@ -14,11 +14,24 @@ class IndexCommand {
         this.argv1 = argv1;
     }
     async execute() {
-        const { db, indexer, searcher, config, discovery } = this.container;
+        const { db, indexer, searcher, config, discovery, embedder } = this.container;
         const args = process.argv.slice(this.argv1 === 'rag-index' ? 2 : 3);
         const force = args.includes('--reindex');
         const forceSelect = args.includes('--select');
         await db.initDbWithSchema();
+        // Ensure the local model is pulled (Ollama only — no-op for cloud), then
+        // resolve the active embedding dimension and reconcile it with index_meta
+        // (recreate code_chunk on a dimension change, clear rows on a model change)
+        // before scanning. A clear error here beats a mid-run vector width mismatch.
+        try {
+            await embedder.ensureModelAvailable?.();
+            const meta = await indexer.prepareEmbeddingSchema();
+            console.log(`Embedding: ${meta.provider} / ${meta.model} (${meta.dimensions} dims)`);
+        }
+        catch (err) {
+            console.error(`\nEmbedding setup failed: ${err.message}`);
+            return;
+        }
         const controller = new AbortController();
         const teardown = (0, interrupt_1.setupInterrupt)(controller);
         try {

package/dist/src/commands/McpCommand.js

@@ -7,14 +7,18 @@ class McpCommand {
         this.container = container;
     }
     async execute() {
-        const { db, mcp, logger } = this.container;
+        const { db, mcp, searcher, logger } = this.container;
         let startupError;
         try {
             await db.initDb();
+            // Refuse to serve searches embedded into a foreign vector space — surface
+            // the mismatch through the existing startup-error plumbing so every tool
+            // call reports it clearly.
+            await searcher.assertIndexMatches();
         }
         catch (err) {
             startupError = err instanceof Error ? err : new Error(String(err));
-            logger.error('McpCommand: database connection failed', { reason: startupError.message });
+            logger.error('McpCommand: startup check failed', { reason: startupError.message });
         }
         try {
             await mcp.start(startupError);

package/dist/src/config-store.d.ts

@@ -2,6 +2,7 @@ import { z } from 'zod';
 export declare const CONFIG_DIR: string;
 export declare const CONFIG_FILE: string;
 export declare const SCHEMA_FILE: string;
+export declare const ENV_FILE: string;
 declare const jsonSchema: z.ZodObject<{
     db: z.ZodOptional<z.ZodObject<{
         host: z.ZodOptional<z.ZodString>;
@@ -10,25 +11,41 @@ declare const jsonSchema: z.ZodObject<{
         password: z.ZodOptional<z.ZodString>;
         name: z.ZodOptional<z.ZodString>;
     }, z.core.$strip>>;
-    ollama: z.ZodOptional<z.ZodObject<{
-        host: z.ZodOptional<z.ZodString>;
+    embedding: z.ZodOptional<z.ZodObject<{
+        provider: z.ZodOptional<z.ZodEnum<{
+            ollama: "ollama";
+            "openai-compatible": "openai-compatible";
+            voyage: "voyage";
+            infinity: "infinity";
+        }>>;
+        baseURL: z.ZodOptional<z.ZodString>;
+        apiKey: z.ZodOptional<z.ZodString>;
         model: z.ZodOptional<z.ZodString>;
+        dimensions: z.ZodOptional<z.ZodNumber>;
+        batchSize: z.ZodOptional<z.ZodNumber>;
     }, z.core.$strip>>;
     indexing: z.ZodOptional<z.ZodObject<{
         chunkSize: z.ZodOptional<z.ZodNumber>;
         chunkOverlap: z.ZodOptional<z.ZodNumber>;
         embedBatchSize: z.ZodOptional<z.ZodNumber>;
         astBatchSize: z.ZodOptional<z.ZodNumber>;
+        chunkStrategy: z.ZodOptional<z.ZodEnum<{
+            ast: "ast";
+            character: "character";
+        }>>;
     }, z.core.$strip>>;
     search: z.ZodOptional<z.ZodObject<{
         similarityThreshold: z.ZodOptional<z.ZodNumber>;
         defaultLimit: z.ZodOptional<z.ZodNumber>;
         maxLimit: z.ZodOptional<z.ZodNumber>;
-        autoTranslate: z.ZodOptional<z.ZodBoolean>;
-        translateModel: z.ZodOptional<z.ZodString>;
     }, z.core.$strip>>;
     reranker: z.ZodOptional<z.ZodObject<{
         enabled: z.ZodOptional<z.ZodBoolean>;
+        provider: z.ZodOptional<z.ZodEnum<{
+            "openai-compatible": "openai-compatible";
+        }>>;
+        baseURL: z.ZodOptional<z.ZodString>;
+        apiKey: z.ZodOptional<z.ZodString>;
         model: z.ZodOptional<z.ZodString>;
         url: z.ZodOptional<z.ZodString>;
         overfetch: z.ZodOptional<z.ZodNumber>;
@@ -52,29 +69,50 @@ export declare const ALL_KEYS: string[];
  * `resolvedSchema` in `src/config.ts`, whose `.default(...)` calls reference
  * these values — so the runtime default and the dashboard-displayed default
  * cannot diverge. `as const` keeps the literal types so those `.default()`
- * lookups stay type-safe. Keys omitted here have no static default (required
- * fields like db.host, or values derived at runtime like db.name → rag_<user>
- * and reranker.url → ${ollama.host}/api/rerank).
+ * lookups stay type-safe. Keys omitted here have no static default: required
+ * fields (db.host/user/password) and values derived at runtime — db.name →
+ * rag_<user>, and the provider-dependent URLs (embedding.baseURL,
+ * reranker.baseURL, reranker.url), which are computed in the `Config`
+ * constructor per provider rather than through a schema `.default(...)`.
  */
 export declare const CONFIG_DEFAULTS: {
     readonly 'db.port': 5433;
-    readonly 'ollama.host': "http://localhost:11434";
-    readonly 'ollama.model': string;
+    readonly 'embedding.provider': "ollama";
+    readonly 'embedding.apiKey': "";
+    readonly 'embedding.model': "bge-m3";
+    readonly 'embedding.batchSize': 32;
     readonly 'indexing.chunkSize': 1500;
     readonly 'indexing.chunkOverlap': 200;
     readonly 'indexing.embedBatchSize': 5;
     readonly 'indexing.astBatchSize': 3000;
+    readonly 'indexing.chunkStrategy': "ast";
     readonly 'search.similarityThreshold': 0.6;
     readonly 'search.defaultLimit': 5;
     readonly 'search.maxLimit': 20;
-    readonly 'search.autoTranslate': false;
-    readonly 'search.translateModel': "llama3.2";
     readonly 'reranker.enabled': false;
+    readonly 'reranker.provider': "openai-compatible";
+    readonly 'reranker.apiKey': "";
     readonly 'reranker.model': "";
     readonly 'reranker.overfetch': 3;
     readonly 'logging.level': "error";
 };
 export declare function readConfigJson(): JsonConfig | null;
+/**
+ * Migrate a legacy raw config object to the current shape, in place of a
+ * versioned migration system. Pure — returns a new object, never mutates input.
+ *
+ * - Legacy `ollama.host`/`ollama.model` → `embedding.{baseURL,model}` with
+ *   `provider: 'ollama'` (only when `embedding` is not already present).
+ * - Removed translation keys (`search.autoTranslate`, `search.translateModel`)
+ *   are dropped so they neither validate-fail nor take effect.
+ * - Legacy `reranker.provider: 'ollama'` (a dead value — Ollama has no rerank
+ *   API) is rewritten to `openai-compatible`, and a stale `/api/rerank` URL is
+ *   dropped so it re-derives to `/rerank`. Otherwise the tightened enum would
+ *   fail validation on existing configs and lock the user out.
+ *
+ * `ollama` itself is left for the schema to strip. Exported for unit testing.
+ */
+export declare function migrateRawConfig(raw: unknown): unknown;
 /**
  * Validate a parsed config object against the Zod schema. Unknown keys (e.g.
  * the "$schema" editor reference) are stripped. Throws with a caller-friendly

package/dist/src/config-store.js

@@ -33,8 +33,9 @@ var __importStar = (this && this.__importStar) || (function () {
     };
 })();
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.CONFIG_DEFAULTS = exports.ALL_KEYS = exports.SCHEMA_FILE = exports.CONFIG_FILE = exports.CONFIG_DIR = void 0;
+exports.CONFIG_DEFAULTS = exports.ALL_KEYS = exports.ENV_FILE = exports.SCHEMA_FILE = exports.CONFIG_FILE = exports.CONFIG_DIR = void 0;
 exports.readConfigJson = readConfigJson;
+exports.migrateRawConfig = migrateRawConfig;
 exports.parseConfig = parseConfig;
 exports.buildJsonSchema = buildJsonSchema;
 exports.writeConfigJson = writeConfigJson;
@@ -50,6 +51,7 @@ const constants_1 = require("./constants");
 exports.CONFIG_DIR = path.join(os.homedir(), '.config', 'codesearch-mcp');
 exports.CONFIG_FILE = path.join(exports.CONFIG_DIR, 'config.json');
 exports.SCHEMA_FILE = path.join(exports.CONFIG_DIR, 'config.schema.json');
+exports.ENV_FILE = path.join(exports.CONFIG_DIR, '.env');
 // Relative $schema reference written into config.json so editors resolve the
 // sibling schema file for autocomplete / inline validation.
 const SCHEMA_REF = './config.schema.json';
@@ -63,27 +65,33 @@ const jsonSchema = zod_1.z.object({
         password: zod_1.z.string().describe('PostgreSQL password').optional(),
         name: zod_1.z.string().describe('Database name; defaults to rag_<os-username>, auto-created on first run').optional(),
     }).optional(),
-    ollama: zod_1.z.object({
-        host: zod_1.z.string().describe('Ollama API URL (default http://localhost:11434)').optional(),
+    embedding: zod_1.z.object({
+        provider: zod_1.z.enum(['ollama', 'openai-compatible', 'voyage', 'infinity']).describe('Embedding backend (default ollama)').optional(),
+        baseURL: zod_1.z.string().describe('Backend base URL. ollama: http://localhost:11434 · OpenAI: https://api.openai.com/v1 · Infinity/TEI/vLLM: http://localhost:7997 · Voyage: https://api.voyageai.com/v1').optional(),
+        apiKey: zod_1.z.string().describe('API key. Use env:VAR_NAME to read from ~/.config/codesearch-mcp/.env or the environment (recommended). Empty for local servers.').optional(),
         model: zod_1.z.string().describe('Embedding model (default bge-m3)').optional(),
+        dimensions: zod_1.z.number().int().describe('Vector width override; required for models whose native width exceeds 2000 (Matryoshka reduction). Omit to derive from the model/registry/probe.').optional(),
+        batchSize: zod_1.z.number().int().describe('Texts per batch embedding request (default 32)').optional(),
     }).optional(),
     indexing: zod_1.z.object({
         chunkSize: zod_1.z.number().int().describe('Max chars per chunk (default 1500)').optional(),
         chunkOverlap: zod_1.z.number().int().describe('Overlap between chunks (default 200)').optional(),
-        embedBatchSize: zod_1.z.number().int().describe('Parallel embeddings per batch (default 5)').optional(),
+        embedBatchSize: zod_1.z.number().int().describe('Rows per DB-insert batch when writing chunks (default 5). Embedding request size is embedding.batchSize.').optional(),
         astBatchSize: zod_1.z.number().int().describe('Max files per ts-morph batch; larger projects skip dependency resolution (default 3000)').optional(),
+        chunkStrategy: zod_1.z.enum(['ast', 'character']).describe('How TS/JS files are chunked: ast = symbol-aligned boundaries, character = fixed-size recursive splitter (default ast)').optional(),
     }).optional(),
     search: zod_1.z.object({
         similarityThreshold: zod_1.z.number().min(0).max(1).describe('Max cosine distance 0–1; lower = stricter (default 0.6)').optional(),
         defaultLimit: zod_1.z.number().int().describe('Default result count (default 5)').optional(),
         maxLimit: zod_1.z.number().int().describe('Hard cap on results (default 20)').optional(),
-        autoTranslate: zod_1.z.boolean().describe('Translate non-English queries before embedding').optional(),
-        translateModel: zod_1.z.string().describe('Ollama model for translation (default llama3.2)').optional(),
     }).optional(),
     reranker: zod_1.z.object({
         enabled: zod_1.z.boolean().describe('Enable cross-encoder reranking; requires reranker.model').optional(),
+        provider: zod_1.z.enum(['openai-compatible']).describe('Rerank backend. Only openai-compatible /rerank servers (Infinity, TEI, vLLM) — Ollama has no rerank API.').optional(),
+        baseURL: zod_1.z.string().describe('Rerank server base URL (defaults to embedding.baseURL — Infinity serves both).').optional(),
+        apiKey: zod_1.z.string().describe('API key, or env:VAR_NAME. Empty for local servers.').optional(),
         model: zod_1.z.string().describe('Reranker model name (e.g. qllama/bge-reranker-v2-m3)').optional(),
-        url: zod_1.z.string().describe('Full URL of the rerank endpoint (default ${ollama.host}/api/rerank)').optional(),
+        url: zod_1.z.string().describe('Full URL of the rerank endpoint (default derived from provider + baseURL)').optional(),
         overfetch: zod_1.z.number().int().min(1).max(20).describe('Candidate multiplier before reranking (limit × N, default 3)').optional(),
     }).optional(),
     logging: zod_1.z.object({
@@ -98,18 +106,24 @@ const FIELD_TYPES = {
     'db.user': 'string',
     'db.password': 'string',
     'db.name': 'string',
-    'ollama.host': 'string',
-    'ollama.model': 'string',
+    'embedding.provider': 'string',
+    'embedding.baseURL': 'string',
+    'embedding.apiKey': 'string',
+    'embedding.model': 'string',
+    'embedding.dimensions': 'number',
+    'embedding.batchSize': 'number',
     'indexing.chunkSize': 'number',
     'indexing.chunkOverlap': 'number',
     'indexing.embedBatchSize': 'number',
     'indexing.astBatchSize': 'number',
+    'indexing.chunkStrategy': 'string',
     'search.similarityThreshold': 'number',
     'search.defaultLimit': 'number',
     'search.maxLimit': 'number',
-    'search.autoTranslate': 'boolean',
-    'search.translateModel': 'string',
     'reranker.enabled': 'boolean',
+    'reranker.provider': 'string',
+    'reranker.baseURL': 'string',
+    'reranker.apiKey': 'string',
     'reranker.model': 'string',
     'reranker.url': 'string',
     'reranker.overfetch': 'number',
@@ -124,24 +138,29 @@ exports.ALL_KEYS = Object.keys(FIELD_TYPES);
  * `resolvedSchema` in `src/config.ts`, whose `.default(...)` calls reference
  * these values — so the runtime default and the dashboard-displayed default
  * cannot diverge. `as const` keeps the literal types so those `.default()`
- * lookups stay type-safe. Keys omitted here have no static default (required
- * fields like db.host, or values derived at runtime like db.name → rag_<user>
- * and reranker.url → ${ollama.host}/api/rerank).
+ * lookups stay type-safe. Keys omitted here have no static default: required
+ * fields (db.host/user/password) and values derived at runtime — db.name →
+ * rag_<user>, and the provider-dependent URLs (embedding.baseURL,
+ * reranker.baseURL, reranker.url), which are computed in the `Config`
+ * constructor per provider rather than through a schema `.default(...)`.
  */
 exports.CONFIG_DEFAULTS = {
     'db.port': 5433,
-    'ollama.host': 'http://localhost:11434',
-    'ollama.model': constants_1.DEFAULT_EMBED_MODEL,
+    'embedding.provider': 'ollama',
+    'embedding.apiKey': '',
+    'embedding.model': constants_1.DEFAULT_EMBED_MODEL,
+    'embedding.batchSize': 32,
     'indexing.chunkSize': 1500,
     'indexing.chunkOverlap': 200,
     'indexing.embedBatchSize': 5,
     'indexing.astBatchSize': 3000,
+    'indexing.chunkStrategy': 'ast',
     'search.similarityThreshold': 0.6,
     'search.defaultLimit': 5,
     'search.maxLimit': 20,
-    'search.autoTranslate': false,
-    'search.translateModel': 'llama3.2',
     'reranker.enabled': false,
+    'reranker.provider': 'openai-compatible',
+    'reranker.apiKey': '',
     'reranker.model': '',
     'reranker.overfetch': 3,
     'logging.level': 'error',
@@ -164,7 +183,48 @@ function readConfigJson() {
         throw new Error(`Invalid JSON in ${exports.CONFIG_FILE}: ${e.message}\n` +
             `Fix the file manually or delete it and run: rag-setup`);
     }
-    return parseConfig(parsed, exports.CONFIG_FILE);
+    return parseConfig(migrateRawConfig(parsed), exports.CONFIG_FILE);
+}
+/**
+ * Migrate a legacy raw config object to the current shape, in place of a
+ * versioned migration system. Pure — returns a new object, never mutates input.
+ *
+ * - Legacy `ollama.host`/`ollama.model` → `embedding.{baseURL,model}` with
+ *   `provider: 'ollama'` (only when `embedding` is not already present).
+ * - Removed translation keys (`search.autoTranslate`, `search.translateModel`)
+ *   are dropped so they neither validate-fail nor take effect.
+ * - Legacy `reranker.provider: 'ollama'` (a dead value — Ollama has no rerank
+ *   API) is rewritten to `openai-compatible`, and a stale `/api/rerank` URL is
+ *   dropped so it re-derives to `/rerank`. Otherwise the tightened enum would
+ *   fail validation on existing configs and lock the user out.
+ *
+ * `ollama` itself is left for the schema to strip. Exported for unit testing.
+ */
+function migrateRawConfig(raw) {
+    if (raw == null || typeof raw !== 'object')
+        return raw;
+    const obj = JSON.parse(JSON.stringify(raw));
+    const legacyOllama = obj.ollama;
+    if (!obj.embedding && legacyOllama && typeof legacyOllama === 'object') {
+        const embedding = { provider: 'ollama' };
+        if (typeof legacyOllama.host === 'string')
+            embedding.baseURL = legacyOllama.host;
+        if (typeof legacyOllama.model === 'string')
+            embedding.model = legacyOllama.model;
+        obj.embedding = embedding;
+    }
+    delete obj.ollama;
+    if (obj.search && typeof obj.search === 'object') {
+        delete obj.search.autoTranslate;
+        delete obj.search.translateModel;
+    }
+    if (obj.reranker && typeof obj.reranker === 'object') {
+        if (obj.reranker.provider === 'ollama')
+            obj.reranker.provider = 'openai-compatible';
+        if (typeof obj.reranker.url === 'string' && /\/api\/rerank\/?(\?.*)?$/.test(obj.reranker.url))
+            delete obj.reranker.url;
+    }
+    return obj;
 }
 /**
  * Validate a parsed config object against the Zod schema. Unknown keys (e.g.

package/dist/src/config.d.ts

@@ -1,6 +1,14 @@
 export declare class ConfigError extends Error {
     constructor(message: string);
 }
+/**
+ * Resolve a config value that may reference an environment variable. A value of
+ * the form `env:VAR_NAME` resolves to `process.env.VAR_NAME`; any other string
+ * (including empty) is returned verbatim. A referenced-but-unset variable throws
+ * a clear error naming the variable and the `.env` file. Pure aside from the
+ * injected env map. See `src/app.ts` for how `.env` is loaded into the process.
+ */
+export declare function resolveEnvRef(value: string, env?: NodeJS.ProcessEnv): string;
 export declare class Config {
     private readonly data;
     get db(): {
@@ -10,26 +18,42 @@ export declare class Config {
         password: string;
         database: string;
     };
-    get ollama(): {
-        host: string;
+    /**
+     * Embedding config. `apiKey` is returned **raw** (possibly an `env:VAR` ref) —
+     * callers resolve it lazily via `resolveEnvRef` at the moment they build a
+     * request, so an unset referenced variable fails only when the key is actually
+     * needed (during embed/search) rather than crashing container construction and
+     * locking the user out of `rag-web`/`rag-mcp`.
+     */
+    get embedding(): {
+        provider: "ollama" | "openai-compatible" | "voyage" | "infinity";
+        baseURL: string;
+        apiKey: string;
+        model: string;
+        dimensions: number | null;
+        batchSize: number;
+    };
+    /** Reranker config. `apiKey` is raw — resolve lazily with `resolveEnvRef`. */
+    get reranker(): {
+        enabled: boolean;
+        provider: "openai-compatible";
+        baseURL: string;
+        apiKey: string;
         model: string;
-        rerankerModel: string;
-        rerankerUrl: string;
+        url: string;
+        overfetch: number;
     };
     get indexer(): {
         chunkSize: number;
         chunkOverlap: number;
         batchSize: number;
         astBatchSize: number;
+        chunkStrategy: "ast" | "character";
     };
     get search(): {
         similarityThreshold: number;
         defaultLimit: number;
         maxLimit: number;
-        autoTranslate: boolean;
-        translateModel: string;
-        rerankerEnabled: boolean;
-        rerankerOverfetch: number;
     };
     get logging(): {
         level: "error" | "debug" | "info" | "warn";