open-context-engine 0.1.3 → 0.1.5

package/README.md

@@ -37,7 +37,7 @@ Measured with the optional batch rerank API. 40 source-derived tasks, each asked
 
 Requires **macOS, Linux, or Windows**, Node.js 22.14+, Python 3.10+, Git, and configured embedding/reranking services. Go repositories also need Go 1.22+.
 
-Native Windows support is available in version **0.1.3** and later. Run the installation commands in PowerShell.
+Native Windows support is available in version **0.1.3** and later. Run the installation commands in PowerShell. Version **0.1.4** adds automatic worker sharing across MCP sessions to prevent index-lock conflicts. Version **0.1.5** raises the default retrieval output budget to 8,000 tokens; pass `budget: 4000` when a smaller response is sufficient.
 
 Install from npm, then expand your client's guide. Model settings are shared across clients on the same machine.
 

package/README.zh-CN.md

@@ -37,7 +37,7 @@ OpenContextEngine 是一个**可自行部署、面向 AI 编程助手的代码
 
 需要 **macOS、Linux 或 Windows**、Node.js 22.14+、Python 3.10+、Git，以及已配置好的向量和重排服务。分析 Go 仓库还需要 Go 1.22+。
 
-**0.1.3** 起支持原生 Windows，安装命令可在 PowerShell 中运行。
+**0.1.3** 起支持原生 Windows，安装命令可在 PowerShell 中运行。**0.1.4** 起支持多个 MCP 会话自动共享 worker，避免索引锁冲突。**0.1.5** 起默认检索输出预算提高到 8,000 tokens；需要更精简的结果时，可显式传入 `budget: 4000`。
 
 通过 npm 安装，展开你所用客户端的教程即可。同一台机器、同一用户下的多个客户端可以共用模型配置。下方链接的详细技术文档目前为英文。
 

package/docs/QUICKSTART.md

@@ -32,7 +32,7 @@ open-context-engine setup --python "C:\Program Files\Python312\python.exe"
 
 Use native absolute project paths such as `C:\Users\you\project` in MCP calls. Generated MCP JSON escapes backslashes automatically.
 
-For internal builds, maintainers can create an archive with `npm pack` and install it with `npm install -g /path/to/open-context-engine-0.1.3.tgz`. See the [release checklist](https://github.com/AnnaSuSu/OpenContextEngine/blob/main/docs/RELEASING.md).
+For internal builds, maintainers can create an archive with `npm pack` and install it with `npm install -g /path/to/open-context-engine-0.1.5.tgz`. See the [release checklist](https://github.com/AnnaSuSu/OpenContextEngine/blob/main/docs/RELEASING.md).
 
 The CLI and package are named `open-context-engine`. The previous `opencontextengine` command remains an alias. Existing configuration and cache directories keep their paths, so saved keys and indexes are reused.
 
@@ -60,6 +60,20 @@ The setup prompt initially suggests `1024` dimensions; replace it with your endp
 
 The embedding service must implement `POST /v1/embeddings`. By default, the reranker uses the ordinary `/rerank` API: requests contain `model`, `query`, `documents`, and `top_n`; responses must return every requested document in `results`, with its original `index` and a finite `relevance_score` between 0 and 1. Results may arrive in relevance order. Set the reranker base URL to the part before `/rerank`: for example, `https://provider.example/v1`, `/v2`, or `https://provider.example` for an unversioned endpoint.
 
+For Alibaba Cloud Model Studio, use its OpenAI-compatible embedding endpoint and select the DashScope rerank format:
+
+```dotenv
+EMBEDDING_BASE_URL=https://YOUR_WORKSPACE.cn-beijing.maas.aliyuncs.com/compatible-mode/v1
+EMBEDDING_MODEL=qwen3.7-text-embedding
+OCE_EMBEDDING_DIMENSIONS=1024
+OCE_EMBEDDING_BATCH_SIZE=20
+RERANK_BASE_URL=https://YOUR_WORKSPACE.cn-beijing.maas.aliyuncs.com/api/v1
+RERANK_MODEL=qwen3.7-text-rerank
+OCE_RERANK_API=dashscope
+```
+
+Supply your API key through `EMBEDDING_API_KEY` and `RERANK_API_KEY`, then run setup to save these settings. DashScope mode calls `/services/rerank/text-rerank/text-rerank`, nests the request under `input` and `parameters`, and reads `output.results`. The embedding model defaults to 1024 dimensions and accepts up to 20 texts per request; `OCE_EMBEDDING_BATCH_SIZE` limits index-building batches (default 64, range 1–64). Query embedding uses at most five texts per MCP search. Clear any previous SSH transport overrides when switching to direct HTTPS endpoints. Changing the embedding provider or model rebuilds the index; existing embeddings are retained for reuse with their original configuration.
+
 HTTPS is the default. For an explicitly trusted remote HTTP deployment, set `OCE_ALLOW_HTTP=1` before running `open-context-engine setup`; setup saves this choice in the shared configuration. HTTP transmits API keys and source text without encryption. Local model endpoints remain prohibited. Set `OCE_EMBEDDING_DIMENSIONS` to the service's actual output size (for example, `2560`); changing the provider or dimensions creates a new index generation and does not mix incompatible cached vectors.
 
 OpenContextEngine groups the needed pairs by query, reuses scores within each search, and makes at most two concurrent rerank requests by default. Optional `OCE_RERANK_CONCURRENCY` (1–8, default 2) and `OCE_RERANK_MAX_DOCUMENTS` (1–1,024, default 128) control concurrency and documents per request. It requests all scores and rejects missing, duplicate, or invalid result indices; errors are surfaced without silently switching endpoints.
@@ -106,7 +120,9 @@ If your client already has `open-context-engine` on `PATH`, this shorter equival
 
 Use your client's equivalent configuration format. Configuration and dependency errors go to stderr; MCP stdout is reserved for protocol messages.
 
-Without `--root`, the server uses **automatic workspace mode**. Your agent supplies the absolute project directory in `directory_path` on each tool call. No indexing starts until a project is requested; first access starts a repository worker and background indexing. Later calls reuse it. One MCP session can search multiple projects, each with an independent worker and persistent index. Workers stay active until the client disconnects, when all are stopped.
+Without `--root`, the server uses **automatic workspace mode**. Your agent supplies the absolute project directory in `directory_path` on each tool call. No indexing starts until a project is requested; first access starts a repository worker and background indexing. Later calls reuse it, including calls from other MCP processes under the same local user. One MCP session can search multiple projects, each with an independent worker and persistent index.
+
+Version **0.1.4** and later automatically share one worker per index directory across compatible clients. Closing a client releases only its lease; a worker exits after all leases expire or are released, no searches remain in flight, and it has been idle for 30 seconds. Clients renew their leases automatically, and reconnect if the worker exits.
 
 The server does not infer your editor's project from its own launch directory. Its tool instructions tell the agent to use the project path supplied by the host, or inspect the current project directory. Missing, relative, or invalid paths return an error. Symbolic links to the same directory share a worker. Supply the same project root consistently, rather than a different subdirectory on each call.
 
@@ -120,7 +136,7 @@ To pin the server to one project instead, append `"--root", "/absolute/path/to/y
 
 | Tool | Purpose |
 | --- | --- |
-| `search_code` | Pass `directory_path` and describe the behavior in `query`. Returns source paths, line numbers, and relevant code; default budget: 4,000 tokens. |
+| `search_code` | Pass `directory_path` and describe the behavior in `query`. Returns source paths, line numbers, and relevant code; default budget: 8,000 tokens. Pass `budget: 4000` for a smaller response. |
 | `index_status` | Pass `directory_path` to inspect indexing progress, active generation, vector reuse, and the latest update error. First access also starts that project's index. |
 
 ### Client setup notes
@@ -158,7 +174,15 @@ Saved files are checked every second by default (`OCE_POLL_SECONDS=1`), with a 3
 
 Search actively checks source hashes before retrieval and again before returning. It waits up to 30 seconds for synchronization (`freshnessWaitMs`, maximum 120 seconds). Failed updates, timeouts, or edits during retrieval produce explicit errors. Unsaved editor buffers are not indexed.
 
-State is stored in `~/.cache/opencontextengine/<repository-path-hash>/`. Override it with `--state /outside/repository/index`: automatic mode creates a separate path-hash subdirectory for each project; fixed `--root` mode uses that exact state directory. Existing installations automatically reuse their previous cache location. One worker may write to a state directory at a time. Stop that worker and remove the directory to delete stored source and embeddings.
+Python, JavaScript, TypeScript, and Go files with syntax errors (including unfilled templates) are indexed as plain text with their original paths and line numbers. Healthy files keep their structural analysis; no structural relations are inferred for degraded files. The index remains `ready`, while `index_status` reports `generation.degradedFiles` and `generation.parseDiagnostics` (path, language, error type, line, column, and fallback mode). Search responses include a degraded-file count and MCP displays a short notice. Diagnostics persist across restarts and disappear when the file is repaired or deleted. Model, toolchain, storage, and source-integrity failures still fail explicitly; stale source is never substituted.
+
+State is stored in `~/.cache/opencontextengine/<repository-path-hash>/`. Override it with `--state /outside/repository/index`: automatic mode creates a separate path-hash subdirectory for each project; fixed `--root` mode uses that exact state directory. Existing installations automatically reuse their previous cache location. One worker may write to a state directory at a time. Stop the worker and remove the directory to delete stored source and embeddings.
+
+Automatic mode discovers the writer through an authenticated loopback handshake; concurrent starts keep the existing writer lock intact. Its `worker.json` connection record contains a local access token and is restricted to the current user (POSIX file permissions or a Windows file ACL). Do not share this file.
+
+Configuration and runtime compatibility include model credentials, endpoints, language options, and worker source. Incompatible clients receive an explicit error instead of silently adopting another configuration; close the existing clients before changing settings, or select a separate `--state` directory. An older or manually started worker cannot be adopted automatically: use `--connect` or stop its owning clients before switching to automatic sharing.
+
+Searches are serialized with a bounded queue of 16 active/waiting requests and a 30-second queue wait; saturation returns a retryable busy error.
 
 When model weights change under the same name, increment `OCE_EMBEDDING_REVISION`. A different provider, model name, or dimension count also invalidates vector reuse. Other models need separate compatibility and quality validation.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "open-context-engine",
-  "version": "0.1.3",
+  "version": "0.1.5",
   "type": "module",
   "engines": {
     "node": ">=22.14"
@@ -43,6 +43,7 @@
     "src/environment.mjs",
     "src/client.mjs",
     "src/service.mjs",
+    "src/shared-service.mjs",
     "src/workspaces.mjs",
     "src/mcp.mjs",
     "src/runtime.mjs",
@@ -80,6 +81,6 @@
     "access": "public",
     "registry": "https://registry.npmjs.org/"
   },
-  "readme": "<div align=\"center\">\n  <picture>\n    <source media=\"(prefers-color-scheme: dark)\" srcset=\"assets/brand/logo-lockup-dark.svg\">\n    <img src=\"assets/brand/logo-lockup.svg\" alt=\"OpenContextEngine\" width=\"660\">\n  </picture>\n  <p><strong>Precise code context for AI coding agents.</strong></p>\n  <p>Find related code across files. Follow its connections. Give your agent the evidence it needs.</p>\n  <p>\n    <a href=\"docs/BENCHMARKS.md#seven-method-comparison\"><img src=\"https://img.shields.io/badge/dev_evidence_coverage-94.79%25-23875b?style=flat-square\" alt=\"Development evidence coverage: 94.79%\"></a>\n    <a href=\"docs/BENCHMARKS.md#seven-method-comparison\"><img src=\"https://img.shields.io/badge/median_retrieval-1.73_s-23875b?style=flat-square\" alt=\"Median retrieval: 1.73 seconds\"></a>\n    <a href=\"docs/BENCHMARKS.md#engineering-validation\"><img src=\"https://img.shields.io/badge/verified_tests-107-23875b?style=flat-square\" alt=\"107 verified tests\"></a>\n    <a href=\"docs/QUICKSTART.md\"><img src=\"https://img.shields.io/badge/MCP-stdio-193c34?style=flat-square\" alt=\"MCP over stdio\"></a>\n  </p>\n  <p><a href=\"#quick-start\">Quick start</a> · <a href=\"docs/BENCHMARKS.md\">Benchmarks</a> · <a href=\"docs/QUICKSTART.md\">MCP setup</a> · <strong>English</strong> | <a href=\"README.zh-CN.md\">简体中文</a></p>\n</div>\n\nOpenContextEngine is a **self-hostable code context engine for AI coding agents**. Connect it to your agent through MCP to help it explore an unfamiliar codebase, locate implementations, and find the related code needed for a fix or feature.\n\nIt indexes your working directory and follows saved changes. Given a natural-language task, it combines semantic and keyword search, code relationships, and reranking to return relevant source snippets with file paths and line numbers, within a fixed context budget.\n\n## Why OpenContextEngine\n\n- **Search beyond exact words.** Describe a behavior; retrieve its implementation and connected code across files.\n- **Understand code structure.** Python, TypeScript, JavaScript, and Go adapters, plus text fallback for other languages, configuration, and scripts.\n- **Stay current as you edit.** Saved changes, file deletions, and branch switches sync automatically. Unchanged embeddings are reused; incomplete updates never replace a complete index.\n- **Work across projects through MCP.** Your agent supplies the project path; indexes start on demand and are reused. `search_code` retrieves evidence; `index_status` reports synchronization.\n\n## Measured results\n\n![Required evidence coverage and observed query time](assets/benchmarks/method-comparison.svg)\n\n**94.79% required evidence coverage · 1.73 s median retrieval · 69/80 queries with complete evidence.** Seven engines, four repositories, the same 4,000-token output budget. OpenContextEngine retained the most required evidence in this internal development evaluation.\n\nMeasured with the optional batch rerank API. 40 source-derived tasks, each asked in Chinese and English. Coverage measures source evidence, not coding-agent success. Timings reflect native retrieval for open tools and SDK client calls for ACE. [Full comparison, configurations, and per-query results →](docs/eval/METHOD_COMPARISON.md)\n\n## Quick start\n\nRequires **macOS, Linux, or Windows**, Node.js 22.14+, Python 3.10+, Git, and configured embedding/reranking services. Go repositories also need Go 1.22+.\n\nNative Windows support is available in version **0.1.3** and later. Run the installation commands in PowerShell.\n\nInstall from npm, then expand your client's guide. Model settings are shared across clients on the same machine.\n\n<details open>\n<summary><strong>Codex — install, connect, and search</strong></summary>\n\n**1. Install the CLI**\n\nWith Codex CLI already installed, run:\n\n```sh\nnpm install -g open-context-engine\n```\n\n**2. Configure your models**\n\n```sh\nopen-context-engine setup\n```\n\nEnter your embedding and reranking base URLs, API keys, model names, and embedding dimensions. Setup installs isolated Python dependencies and saves your settings. The default reranker uses the ordinary `/rerank` API. [Endpoint examples →](docs/QUICKSTART.md#2-shared-model-configuration)\n\n**3. Add the MCP server**\n\n```sh\ncodex mcp add open-context-engine -- open-context-engine mcp\ncodex mcp get open-context-engine\n```\n\nThe second command checks the saved configuration. These commands assume `open-context-engine` is on the client's `PATH`. For the desktop app or source installations, use the [absolute-path configuration](docs/QUICKSTART.md#client-setup-notes). Restart an already-running Codex client after adding the server.\n\n**4. Search your project**\n\n```sh\ncd /path/to/your-project\ncodex\n```\n\nAsk:\n\n> Use open-context-engine's search_code tool to explain this project's main functionality. Include the entry points and relevant file paths and line numbers.\n\nCodex supplies the project's absolute path as `directory_path`. The first request starts indexing; if it is still building, ask Codex to check `index_status` and retry when ready. Later searches reuse the index, and saved changes update automatically.\n\n</details>\n\n<details>\n<summary><strong>Claude Code — install, connect, and search</strong></summary>\n\n**1. Install the CLI**\n\nWith Claude Code already installed, run:\n\n```sh\nnpm install -g open-context-engine\n```\n\n**2. Configure your models**\n\n```sh\nopen-context-engine setup\n```\n\nEnter your embedding and reranking base URLs, API keys, model names, and embedding dimensions. Setup installs isolated Python dependencies and saves your settings. If you already completed setup for Codex, reuse those settings and skip this step. [Endpoint examples →](docs/QUICKSTART.md#2-shared-model-configuration)\n\n**3. Add the MCP server**\n\n```sh\nclaude mcp add --transport stdio --scope user open-context-engine -- open-context-engine mcp\n```\n\nUser scope makes the server available across your projects. For a shared project configuration, run the command from that project and replace `--scope user` with `--scope project`. These commands assume `open-context-engine` is on the client's `PATH`; see [client setup notes](docs/QUICKSTART.md#client-setup-notes) for absolute paths. Restart an already-running Claude Code session after adding the server.\n\n**4. Search your project**\n\n```sh\ncd /path/to/your-project\nclaude\n```\n\nRun `/mcp` to check the connection, then ask:\n\n> Use open-context-engine's search_code tool to explain this project's main functionality. Include the entry points and relevant file paths and line numbers.\n\nClaude supplies the project's absolute path as `directory_path`. The first request starts indexing; if it is still building, ask Claude to check `index_status` and retry when ready. Later searches reuse the index, and saved changes update automatically.\n\n</details>\n\n<details>\n<summary>Other MCP clients</summary>\n\n[Install and run setup](docs/QUICKSTART.md#1-install), then use the configuration printed by `open-context-engine mcp-config` in your client's supported format. For clients that accept `mcpServers` JSON and can find the installed command on `PATH`:\n\n```json\n{\n  \"mcpServers\": {\n    \"open-context-engine\": {\n      \"command\": \"open-context-engine\",\n      \"args\": [\"mcp\"]\n    }\n  }\n}\n```\n\nYour agent supplies the current project's absolute path as `directory_path`. To pin one project, add `\"--root\", \"/absolute/path/to/your-repository\"` to `args`.\n\n</details>\n\n[Model configuration, troubleshooting, and update behavior →](docs/QUICKSTART.md)\n\n## Explore\n\n[Benchmark report](https://github.com/AnnaSuSu/OpenContextEngine/blob/main/docs/BENCHMARKS.md) · [Raw evaluations](https://github.com/AnnaSuSu/OpenContextEngine/tree/main/docs/eval/results) · [Retrieval engine](https://github.com/AnnaSuSu/OpenContextEngine/tree/main/src/retrieval)\n\n## License\n\n[MIT](LICENSE) © 2026 AnnaSuSu.\n\n## Acknowledgments\n\nThanks to the [LINUX DO](https://linux.do/) community.\n",
+  "readme": "<div align=\"center\">\n  <picture>\n    <source media=\"(prefers-color-scheme: dark)\" srcset=\"assets/brand/logo-lockup-dark.svg\">\n    <img src=\"assets/brand/logo-lockup.svg\" alt=\"OpenContextEngine\" width=\"660\">\n  </picture>\n  <p><strong>Precise code context for AI coding agents.</strong></p>\n  <p>Find related code across files. Follow its connections. Give your agent the evidence it needs.</p>\n  <p>\n    <a href=\"docs/BENCHMARKS.md#seven-method-comparison\"><img src=\"https://img.shields.io/badge/dev_evidence_coverage-94.79%25-23875b?style=flat-square\" alt=\"Development evidence coverage: 94.79%\"></a>\n    <a href=\"docs/BENCHMARKS.md#seven-method-comparison\"><img src=\"https://img.shields.io/badge/median_retrieval-1.73_s-23875b?style=flat-square\" alt=\"Median retrieval: 1.73 seconds\"></a>\n    <a href=\"docs/BENCHMARKS.md#engineering-validation\"><img src=\"https://img.shields.io/badge/verified_tests-107-23875b?style=flat-square\" alt=\"107 verified tests\"></a>\n    <a href=\"docs/QUICKSTART.md\"><img src=\"https://img.shields.io/badge/MCP-stdio-193c34?style=flat-square\" alt=\"MCP over stdio\"></a>\n  </p>\n  <p><a href=\"#quick-start\">Quick start</a> · <a href=\"docs/BENCHMARKS.md\">Benchmarks</a> · <a href=\"docs/QUICKSTART.md\">MCP setup</a> · <strong>English</strong> | <a href=\"README.zh-CN.md\">简体中文</a></p>\n</div>\n\nOpenContextEngine is a **self-hostable code context engine for AI coding agents**. Connect it to your agent through MCP to help it explore an unfamiliar codebase, locate implementations, and find the related code needed for a fix or feature.\n\nIt indexes your working directory and follows saved changes. Given a natural-language task, it combines semantic and keyword search, code relationships, and reranking to return relevant source snippets with file paths and line numbers, within a fixed context budget.\n\n## Why OpenContextEngine\n\n- **Search beyond exact words.** Describe a behavior; retrieve its implementation and connected code across files.\n- **Understand code structure.** Python, TypeScript, JavaScript, and Go adapters, plus text fallback for other languages, configuration, and scripts.\n- **Stay current as you edit.** Saved changes, file deletions, and branch switches sync automatically. Unchanged embeddings are reused; incomplete updates never replace a complete index.\n- **Work across projects through MCP.** Your agent supplies the project path; indexes start on demand and are reused. `search_code` retrieves evidence; `index_status` reports synchronization.\n\n## Measured results\n\n![Required evidence coverage and observed query time](assets/benchmarks/method-comparison.svg)\n\n**94.79% required evidence coverage · 1.73 s median retrieval · 69/80 queries with complete evidence.** Seven engines, four repositories, the same 4,000-token output budget. OpenContextEngine retained the most required evidence in this internal development evaluation.\n\nMeasured with the optional batch rerank API. 40 source-derived tasks, each asked in Chinese and English. Coverage measures source evidence, not coding-agent success. Timings reflect native retrieval for open tools and SDK client calls for ACE. [Full comparison, configurations, and per-query results →](docs/eval/METHOD_COMPARISON.md)\n\n## Quick start\n\nRequires **macOS, Linux, or Windows**, Node.js 22.14+, Python 3.10+, Git, and configured embedding/reranking services. Go repositories also need Go 1.22+.\n\nNative Windows support is available in version **0.1.3** and later. Run the installation commands in PowerShell. Version **0.1.4** adds automatic worker sharing across MCP sessions to prevent index-lock conflicts. Version **0.1.5** raises the default retrieval output budget to 8,000 tokens; pass `budget: 4000` when a smaller response is sufficient.\n\nInstall from npm, then expand your client's guide. Model settings are shared across clients on the same machine.\n\n<details open>\n<summary><strong>Codex — install, connect, and search</strong></summary>\n\n**1. Install the CLI**\n\nWith Codex CLI already installed, run:\n\n```sh\nnpm install -g open-context-engine\n```\n\n**2. Configure your models**\n\n```sh\nopen-context-engine setup\n```\n\nEnter your embedding and reranking base URLs, API keys, model names, and embedding dimensions. Setup installs isolated Python dependencies and saves your settings. The default reranker uses the ordinary `/rerank` API. [Endpoint examples →](docs/QUICKSTART.md#2-shared-model-configuration)\n\n**3. Add the MCP server**\n\n```sh\ncodex mcp add open-context-engine -- open-context-engine mcp\ncodex mcp get open-context-engine\n```\n\nThe second command checks the saved configuration. These commands assume `open-context-engine` is on the client's `PATH`. For the desktop app or source installations, use the [absolute-path configuration](docs/QUICKSTART.md#client-setup-notes). Restart an already-running Codex client after adding the server.\n\n**4. Search your project**\n\n```sh\ncd /path/to/your-project\ncodex\n```\n\nAsk:\n\n> Use open-context-engine's search_code tool to explain this project's main functionality. Include the entry points and relevant file paths and line numbers.\n\nCodex supplies the project's absolute path as `directory_path`. The first request starts indexing; if it is still building, ask Codex to check `index_status` and retry when ready. Later searches reuse the index, and saved changes update automatically.\n\n</details>\n\n<details>\n<summary><strong>Claude Code — install, connect, and search</strong></summary>\n\n**1. Install the CLI**\n\nWith Claude Code already installed, run:\n\n```sh\nnpm install -g open-context-engine\n```\n\n**2. Configure your models**\n\n```sh\nopen-context-engine setup\n```\n\nEnter your embedding and reranking base URLs, API keys, model names, and embedding dimensions. Setup installs isolated Python dependencies and saves your settings. If you already completed setup for Codex, reuse those settings and skip this step. [Endpoint examples →](docs/QUICKSTART.md#2-shared-model-configuration)\n\n**3. Add the MCP server**\n\n```sh\nclaude mcp add --transport stdio --scope user open-context-engine -- open-context-engine mcp\n```\n\nUser scope makes the server available across your projects. For a shared project configuration, run the command from that project and replace `--scope user` with `--scope project`. These commands assume `open-context-engine` is on the client's `PATH`; see [client setup notes](docs/QUICKSTART.md#client-setup-notes) for absolute paths. Restart an already-running Claude Code session after adding the server.\n\n**4. Search your project**\n\n```sh\ncd /path/to/your-project\nclaude\n```\n\nRun `/mcp` to check the connection, then ask:\n\n> Use open-context-engine's search_code tool to explain this project's main functionality. Include the entry points and relevant file paths and line numbers.\n\nClaude supplies the project's absolute path as `directory_path`. The first request starts indexing; if it is still building, ask Claude to check `index_status` and retry when ready. Later searches reuse the index, and saved changes update automatically.\n\n</details>\n\n<details>\n<summary>Other MCP clients</summary>\n\n[Install and run setup](docs/QUICKSTART.md#1-install), then use the configuration printed by `open-context-engine mcp-config` in your client's supported format. For clients that accept `mcpServers` JSON and can find the installed command on `PATH`:\n\n```json\n{\n  \"mcpServers\": {\n    \"open-context-engine\": {\n      \"command\": \"open-context-engine\",\n      \"args\": [\"mcp\"]\n    }\n  }\n}\n```\n\nYour agent supplies the current project's absolute path as `directory_path`. To pin one project, add `\"--root\", \"/absolute/path/to/your-repository\"` to `args`.\n\n</details>\n\n[Model configuration, troubleshooting, and update behavior →](docs/QUICKSTART.md)\n\n## Explore\n\n[Benchmark report](https://github.com/AnnaSuSu/OpenContextEngine/blob/main/docs/BENCHMARKS.md) · [Raw evaluations](https://github.com/AnnaSuSu/OpenContextEngine/tree/main/docs/eval/results) · [Retrieval engine](https://github.com/AnnaSuSu/OpenContextEngine/tree/main/src/retrieval)\n\n## License\n\n[MIT](LICENSE) © 2026 AnnaSuSu.\n\n## Acknowledgments\n\nThanks to the [LINUX DO](https://linux.do/) community.\n",
   "readmeFilename": "README.md"
 }

package/scripts/retrieval-server.py

@@ -1,7 +1,9 @@
 """Authenticated, persistent retrieval worker; model inference stays remote."""
 import hashlib
+import errno
 from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
 import json
+import os
 from pathlib import Path
 import re
 import secrets
@@ -14,8 +16,11 @@ import numpy as np
 
 ROOT = Path(__file__).resolve().parents[1]
 sys.path.insert(0, str(ROOT / 'src' / 'retrieval'))
-from routed import RoutedEngine, VERSION
+from evidence import EvidenceEngine, VERSION
+from planning import plan_query
 from live import LiveIndex, IndexUnavailable
+from writer_lock import WriterBusy
+from shared_worker import SharedWorker
 
 
 class LoopbackHTTPServer(ThreadingHTTPServer):
@@ -25,29 +30,25 @@ class LoopbackHTTPServer(ThreadingHTTPServer):
         self.server_name, self.server_port = self.server_address
 
 
-def plan_query(query):
-    parts = [part.strip() for part in re.split(r'[:：;；，]|,\s+(?=how|which|why|where|what)|\s+and\s+(?=how|which|why|where|what)', query, flags=re.I) if len(part.strip()) > 7]
-    facets = parts if 1 < len(parts) <= 4 else [query]
-    return {'intent':query,'facets':[{'question':part,'terms':re.findall(r'[A-Za-z][A-Za-z0-9_]*',part)} for part in facets]}
-
 
 def serve(config):
     initialized = time.monotonic()
     live = LiveIndex(config).start() if config.get('root') else None
+    shared = SharedWorker(config) if config.get('shared') and live else None
     state = Path(config['state'])
     if live:
         index, retrieval = None, None
     else:
         units = json.loads((state / 'units.json').read_text())
         index = json.loads((state / 'metadata.json').read_text())
-        retrieval = RoutedEngine(units, np.load(state/'vectors.npy'), config['embeddingUrl'], config['reranker'], config['embeddingKey'])
+        retrieval = EvidenceEngine(units, np.load(state/'vectors.npy'), config['embeddingUrl'], config['reranker'], config['embeddingKey'])
     health = {'status':'ready','engine':VERSION,'index':index,'queryCache':False,
         'initializationMs':round((time.monotonic()-initialized)*1000),
         'sourceSha256':{name:hashlib.sha256((ROOT/name).read_bytes()).hexdigest() for name in
-            ['src/retrieval/engine.py','src/retrieval/batched.py','src/retrieval/routed.py','src/retrieval/reranker.py','scripts/retrieval-server.py',
+            ['src/retrieval/engine.py','src/retrieval/batched.py','src/retrieval/routed.py','src/retrieval/evidence.py','src/retrieval/planning.py','src/retrieval/reranker.py','scripts/retrieval-server.py',
              'src/retrieval/languages/__init__.py','src/retrieval/languages/schema.py',
              'src/retrieval/languages/text.py','src/retrieval/languages/files.py',
-             'src/retrieval/languages/python.py','src/retrieval/languages/go.py','src/retrieval/languages/go_ast.go','src/retrieval/languages/go_types.go',
+             'src/retrieval/languages/python.py','src/retrieval/languages/python_calls.py','src/retrieval/languages/go.py','src/retrieval/languages/go_ast.go','src/retrieval/languages/go_types.go',
              'src/retrieval/languages/typescript.py',
              'src/retrieval/languages/typescript.mjs','package.json','package-lock.json']
             if name != 'package-lock.json' or (ROOT/name).is_file()}}
@@ -56,6 +57,7 @@ def serve(config):
         with urlopen(model_base+'/healthz',timeout=10) as response:
             health['reranker'] = json.load(response)
     lock = threading.Lock()
+    slots = threading.BoundedSemaphore(16)
 
     class Handler(BaseHTTPRequestHandler):
         def log_message(self, *args):
@@ -79,6 +81,19 @@ def serve(config):
             self.reply(404,{'error':'Not found'})
 
         def do_POST(self):
+            if shared and self.path in ('/lease', '/release'):
+                if not secrets.compare_digest(self.headers.get('Authorization', ''), 'Bearer ' + config['serviceKey']):
+                    return self.reply(401, {'error': 'Unauthorized'})
+                try:
+                    length = int(self.headers.get('Content-Length', '0'))
+                    if not 1 <= length <= 4096:
+                        raise ValueError('Invalid body size')
+                    body = json.loads(self.rfile.read(length))
+                    if not isinstance(body, dict):
+                        raise ValueError('Invalid lease')
+                except (ValueError, TypeError):
+                    return self.reply(422, {'error': 'Invalid lease request'})
+                return self.reply(*shared.lease(body, release=self.path == '/release'))
             if self.path!='/search':
                 return self.reply(404,{'error':'Not found'})
             if not secrets.compare_digest(self.headers.get('Authorization',''), 'Bearer '+config['serviceKey']):
@@ -90,7 +105,7 @@ def serve(config):
                 body = json.loads(self.rfile.read(length))
                 if not isinstance(body,dict) or set(body)-{'query','budget','trace','freshnessWaitMs'}:
                     raise ValueError('Unknown fields')
-                query,budget = body.get('query'),body.get('budget',4000)
+                query,budget = body.get('query'),body.get('budget',8000)
                 wait_ms = body.get('freshnessWaitMs', 30000)
                 if type(wait_ms) is not int or not 0 <= wait_ms <= 120000:
                     raise ValueError('Invalid freshness wait')
@@ -99,8 +114,16 @@ def serve(config):
             except (ValueError,TypeError):
                 return self.reply(422,{'error':'Expected query text and a token budget between 256 and 8000'})
             start = time.monotonic()
-            if not lock.acquire(timeout=5):
-                return self.reply(429,{'error':'Retrieval worker busy'})
+            if not slots.acquire(blocking=False):
+                return self.reply(429, {'error': 'Retrieval queue is full; retry later'})
+            if shared and not shared.enter():
+                slots.release()
+                return self.reply(503, {'error': 'Worker is stopping'})
+            if not lock.acquire(timeout=30):
+                slots.release()
+                if shared:
+                    shared.leave()
+                return self.reply(429, {'error': 'Retrieval queue wait exceeded 30 seconds; retry later'})
             try:
                 queued = round((time.monotonic()-start)*1000)
                 generation = live.current(wait_ms/1000) if live else None
@@ -115,7 +138,8 @@ def serve(config):
                     'retrievalMs':debug['elapsedMs'],'queueMs':queued,
                     'serverElapsedMs':round((time.monotonic()-start)*1000),'queryCache':False,
                     'index': {'mode':'live','identity':generation.identity,'freshness':'verified-after-search',
-                              'completedAt':generation.info['completedAt']} if live else {'mode':'frozen'}}
+                              'completedAt':generation.info['completedAt'],
+                              'degradedFiles':generation.info.get('degradedFiles', 0)} if live else {'mode':'frozen'}}
                 if body.get('trace'):
                     response['diagnostics'] = debug
                 self.reply(200,response)
@@ -126,18 +150,43 @@ def serve(config):
                 self.reply(502,{'error':'Retrieval or model request failed'})
             finally:
                 lock.release()
+                slots.release()
+                if shared:
+                    shared.leave()
 
     server = LoopbackHTTPServer(('127.0.0.1',config.get('port',23505)),Handler)
     server.daemon_threads = True
-    print(json.dumps({'listening':f'http://127.0.0.1:{server.server_port}',
-                      'health':{'status':'running','mode':'live'} if live else health}),flush=True)
+    if shared:
+        shared.publish(server.server_port)
+        shared.monitor(server)
+    try:
+        print(json.dumps({'listening':f'http://127.0.0.1:{server.server_port}',
+                          'health':{'status':'running','mode':'live'} if live else health}),flush=True)
+    except OSError as error:
+        # Another client may already have attached through discovery when the
+        # launcher disappears. Its closed startup pipe must not kill that writer.
+        if not shared or error.errno not in (errno.EPIPE, errno.EINVAL):
+            raise
+    if shared:
+        # Shared workers outlive their launching MCP process and its stdio pipes.
+        sys.stdout = open(os.devnull, 'w')
+        sys.stderr = open(os.devnull, 'w')
     try:
         server.serve_forever()
     finally:
         server.server_close()
+        if shared:
+            shared.close()
         if live:
             live.close()
 
 
 if __name__ == '__main__':
-    serve(json.loads(sys.stdin.readline()))
+    try:
+        serve(json.loads(sys.stdin.readline()))
+    except Exception as error:
+        code = 'INDEX_LOCKED' if isinstance(error, WriterBusy) else 'STARTUP_FAILED'
+        # Only known-safe diagnostics cross the startup protocol; never serialize config or keys.
+        message = str(error) if isinstance(error, WriterBusy) else type(error).__name__
+        print(json.dumps({'startupError': {'code': code, 'message': message}}), flush=True)
+        sys.exit(1)

package/src/client.mjs

@@ -10,9 +10,9 @@ export function clientConfig(environment=process.env) {
   return {baseUrl:url.href.replace(/\/$/,''),apiKey};
 }
 
-export async function search(query,{budget=4000,trace=false,freshnessWaitMs=30000,config=clientConfig(),signal}={}) {
+export async function search(query,{budget=8000,trace=false,freshnessWaitMs=30000,config=clientConfig(),signal}={}) {
   const started=performance.now();
-  const timeout = AbortSignal.timeout(freshnessWaitMs + 30000);
+  const timeout = AbortSignal.timeout(freshnessWaitMs + 60000);
   const response=await fetch(`${config.baseUrl}/search`,{method:'POST',redirect:'error',signal:signal ? AbortSignal.any([signal,timeout]) : timeout,
     headers:{'content-type':'application/json',authorization:`Bearer ${config.apiKey}`},body:JSON.stringify({query,budget,trace,freshnessWaitMs})});
   if (!response.ok) {
@@ -20,6 +20,10 @@ export async function search(query,{budget=4000,trace=false,freshnessWaitMs=3000
       const body = await response.json();
       throw new Error(`Index unavailable: ${body.error || 'update pending'}`);
     }
+    if (response.status === 429) {
+      const body = await response.json();
+      throw new Error(`Retrieval busy: ${body.error || 'retry later'}`);
+    }
     throw new Error(`Retrieval HTTP ${response.status}`);
   }
   const result=await response.json();

package/src/config.mjs

@@ -12,7 +12,7 @@ export const configKeys = new Set([
   'RERANK_BASE_URL','RERANK_API_KEY','RERANK_MODEL','RERANK_REMOTE_RUNTIME_URL','RERANK_REMOTE_HOST',
   'RERANK_SSH_TUNNEL_URL','RERANK_SSH_REMOTE',
   'OCE_ALLOW_HTTP','OCE_RERANK_API','OCE_RERANK_CONCURRENCY','OCE_RERANK_MAX_DOCUMENTS','OCE_EMBEDDING_DIMENSIONS',
-  'OCE_EMBEDDING_REVISION','OCE_PYTHON','OCE_GO_BINARY','OCE_LANGUAGE_OPTIONS','OCE_POLL_SECONDS',
+  'OCE_EMBEDDING_REVISION','OCE_EMBEDDING_BATCH_SIZE','OCE_PYTHON','OCE_GO_BINARY','OCE_LANGUAGE_OPTIONS','OCE_POLL_SECONDS',
   'OCE_DEBOUNCE_SECONDS','OCE_API_KEY','OCE_BASE_URL','TIKTOKEN_CACHE_DIR',
 ]);
 export function configDirectory(environment = process.env) {

package/src/eval/remote-models.mjs

@@ -20,7 +20,7 @@ export function remoteRerankerConfig(env) {
     throw new Error('A user-approved remote reranker is required. No local model will be installed or started.');
   }
   const api = env.OCE_RERANK_API ?? 'rerank';
-  if (!['rerank', 'rerank-batch'].includes(api)) throw new Error('OCE_RERANK_API must be rerank or rerank-batch');
+  if (!['rerank', 'rerank-batch', 'dashscope'].includes(api)) throw new Error('OCE_RERANK_API must be rerank, rerank-batch or dashscope');
   const integer = (name, fallback, maximum) => {
     const value = Number(env[name] ?? fallback);
     if (!Number.isInteger(value) || value < 1 || value > maximum) throw new Error(`${name} must be an integer from 1 to ${maximum}`);

package/src/mcp.mjs

@@ -15,7 +15,7 @@ export function createMcpServer(config, {resolveConfig, automatic = false} = {})
   }
   const pathSchema = z.string().min(1).describe('Absolute path to the project directory. Required in automatic workspace mode.');
   const directoryPath = automatic ? pathSchema : pathSchema.optional();
-  const server = new McpServer({name:'open-context-engine',version:'0.1.3'}, {
+  const server = new McpServer({name:'open-context-engine',version:'0.1.5'}, {
     instructions:workspaceInstructions + 'Search for source evidence. Results include source paths and line numbers. '
       + 'Search waits for saved file changes to be indexed. If an update is pending or fails, inspect index_status and retry after it completes. '
       + 'Read target files again before editing, because code may change after a search.',
@@ -26,7 +26,7 @@ export function createMcpServer(config, {resolveConfig, automatic = false} = {})
     description:'Find code implementing a task or behavior in a repository. Returns source evidence under a token budget. '
       + workspaceInstructions
       + 'Uses saved working-tree content, including uncommitted changes; rejects stale results when synchronization fails.',
-    inputSchema:{directory_path:directoryPath,query:z.string().trim().min(1).max(8192),budget:z.number().int().min(256).max(8000).default(4000),
+    inputSchema:{directory_path:directoryPath,query:z.string().trim().min(1).max(8192),budget:z.number().int().min(256).max(8000).default(8000),
       freshnessWaitMs:z.number().int().min(0).max(120000).default(30000)},
     annotations,
   }, async ({directory_path,query,budget,freshnessWaitMs}, extra) => {
@@ -34,7 +34,9 @@ export function createMcpServer(config, {resolveConfig, automatic = false} = {})
       const selected = await selectConfig(directory_path);
       const result = await search(query,{budget,freshnessWaitMs,config:selected,signal:extra.signal});
       if (result.index?.mode !== 'live') throw new Error('This service uses a frozen index; connect to a service started with --root');
-      return {content:[{type:'text',text:result.context || 'No matching source context.'}],structuredContent:result};
+      const warning = result.index.degradedFiles
+        ? `Note: ${result.index.degradedFiles} file(s) indexed as plain text after syntax errors; inspect index_status for paths and locations.\n\n` : '';
+      return {content:[{type:'text',text:warning + (result.context || 'No matching source context.')}],structuredContent:result};
     } catch (error) {
       return {isError:true,content:[{type:'text',text:error.message}]};
     }

package/src/retrieval/batched.py

@@ -209,7 +209,7 @@ class BatchedEngine(Engine):
                     'selectionAnchor':uid,'contextOnly':item not in retained})
         raw = '\n'.join(self.render(self.units[uid]) for uid in selected)
         end = time.monotonic()
-        return raw,{'version':getattr(self, 'version', VERSION),'elapsedMs':round((end-start)*1000),'tokens':len(self.encoding.encode(raw)),
+        return raw,{'version':getattr(self, 'version', VERSION),'elapsedMs':round((end-start)*1000),'tokens':len(self.encoding.encode_ordinary(raw)),
             'candidateCount':len(candidates),'rerankedCount':len(retained),'expandedCount':len(expanded),
             'retention':retention,
             'modelRequests':{'embedding':1,'rerank':sum(w['requests'] for w in waves)},

package/src/retrieval/cascade.py

@@ -130,7 +130,7 @@ class CascadeEngine(Engine):
         raw, selected = self.pack(retained, scores, affinity, budget)
         finished = time.monotonic()
         return raw, {'version': VERSION, 'elapsedMs': round((finished-start)*1000),
-            'tokens': len(self.encoding.encode(raw)), 'candidateCount': candidate_count,
+            'tokens': len(self.encoding.encode_ordinary(raw)), 'candidateCount': candidate_count,
             'rerankedCount': len(retained), 'expandedCount': len(expanded),
             'modelRequests': {'embedding': 1, 'rerank': 1}, 'queryCache': False,
             'timingMs': {'embedding': round((embedded_at-start)*1000), 'recall': round((recalled_at-embedded_at)*1000),

package/src/retrieval/engine.py

@@ -56,7 +56,7 @@ class Engine:
         for u in units:
             for target in u['edges']:
                 self.incoming[target].append(u['id'])
-        self.costs = [len(self.encoding.encode(self.render(u))) + 2 for u in units]
+        self.costs = [len(self.encoding.encode_ordinary(self.render(u))) + 2 for u in units]
 
     @staticmethod
     def render(u):
@@ -153,7 +153,7 @@ class Engine:
             trace.append({'id': uid, 'overall': overall.get(uid, 0), 'facets': values.tolist(),
                           'tokens': self.costs[uid], 'gain': gain, 'graphExpanded': uid in expanded})
         raw = '\n'.join(self.render(self.units[uid]) for uid in selected)
-        return raw, {'elapsedMs': round((time.monotonic()-start)*1000), 'tokens': len(self.encoding.encode(raw)),
+        return raw, {'elapsedMs': round((time.monotonic()-start)*1000), 'tokens': len(self.encoding.encode_ordinary(raw)),
             'candidateCount': len(candidates), 'rerankedCount': len(retained), 'expandedCount': len(expanded),
             'selected': trace, 'plan': plan}
 

package/src/retrieval/entities.py

@@ -175,7 +175,7 @@ class EntityEngine(Engine):
         assembled, expanded_scores, provenance = self.complete(candidates,scores,dense)
         raw,selected = self.pack_entities(assembled,expanded_scores,dense,unit_dense,budget)
         finished = time.monotonic()
-        return raw,{'version':VERSION,'elapsedMs':round((finished-start)*1000),'tokens':len(self.encoding.encode(raw)),
+        return raw,{'version':VERSION,'elapsedMs':round((finished-start)*1000),'tokens':len(self.encoding.encode_ordinary(raw)),
             'queryCache':False,'modelRequests':{'embedding':1,'rerank':1},'candidateCount':len(candidates),
             'expandedCount':len(provenance),'assembledCount':len(assembled),'policy':POLICY,'plan':plan,
             'timingMs':{'embedding':round((embedded-start)*1000),'recall':round((recalled-embedded)*1000),