open-context-engine 0.1.5 → 0.1.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/README.zh-CN.md +1 -1
- package/docs/QUICKSTART.md +3 -3
- package/package.json +2 -2
- package/scripts/retrieval-server.py +2 -1
- package/src/client.mjs +3 -1
- package/src/mcp.mjs +31 -4
- package/src/retrieval/live.py +20 -2
package/README.md
CHANGED
|
@@ -37,7 +37,7 @@ Measured with the optional batch rerank API. 40 source-derived tasks, each asked
|
|
|
37
37
|
|
|
38
38
|
Requires **macOS, Linux, or Windows**, Node.js 22.14+, Python 3.10+, Git, and configured embedding/reranking services. Go repositories also need Go 1.22+.
|
|
39
39
|
|
|
40
|
-
Native Windows support is available in version **0.1.3** and later. Run the installation commands in PowerShell. Version **0.1.4** adds automatic worker sharing across MCP sessions to prevent index-lock conflicts. Version **0.1.5** raises the default retrieval output budget to 8,000 tokens; pass `budget: 4000` when a smaller response is sufficient.
|
|
40
|
+
Native Windows support is available in version **0.1.3** and later. Run the installation commands in PowerShell. Version **0.1.4** adds automatic worker sharing across MCP sessions to prevent index-lock conflicts. Version **0.1.5** raises the default retrieval output budget to 8,000 tokens; pass `budget: 4000` when a smaller response is sufficient. Version **0.1.6** reports pending indexing as progress, retries once, and keeps actual indexing failures explicit.
|
|
41
41
|
|
|
42
42
|
Install from npm, then expand your client's guide. Model settings are shared across clients on the same machine.
|
|
43
43
|
|
package/README.zh-CN.md
CHANGED
|
@@ -37,7 +37,7 @@ OpenContextEngine 是一个**可自行部署、面向 AI 编程助手的代码
|
|
|
37
37
|
|
|
38
38
|
需要 **macOS、Linux 或 Windows**、Node.js 22.14+、Python 3.10+、Git,以及已配置好的向量和重排服务。分析 Go 仓库还需要 Go 1.22+。
|
|
39
39
|
|
|
40
|
-
**0.1.3** 起支持原生 Windows,安装命令可在 PowerShell 中运行。**0.1.4** 起支持多个 MCP 会话自动共享 worker,避免索引锁冲突。**0.1.5** 起默认检索输出预算提高到 8,000 tokens;需要更精简的结果时,可显式传入 `budget: 4000`。
|
|
40
|
+
**0.1.3** 起支持原生 Windows,安装命令可在 PowerShell 中运行。**0.1.4** 起支持多个 MCP 会话自动共享 worker,避免索引锁冲突。**0.1.5** 起默认检索输出预算提高到 8,000 tokens;需要更精简的结果时,可显式传入 `budget: 4000`。 **0.1.6** 起索引更新中会显示进度并自动重试一次,实际索引失败仍明确报错。
|
|
41
41
|
|
|
42
42
|
通过 npm 安装,展开你所用客户端的教程即可。同一台机器、同一用户下的多个客户端可以共用模型配置。下方链接的详细技术文档目前为英文。
|
|
43
43
|
|
package/docs/QUICKSTART.md
CHANGED
|
@@ -32,7 +32,7 @@ open-context-engine setup --python "C:\Program Files\Python312\python.exe"
|
|
|
32
32
|
|
|
33
33
|
Use native absolute project paths such as `C:\Users\you\project` in MCP calls. Generated MCP JSON escapes backslashes automatically.
|
|
34
34
|
|
|
35
|
-
For internal builds, maintainers can create an archive with `npm pack` and install it with `npm install -g /path/to/open-context-engine-0.1.
|
|
35
|
+
For internal builds, maintainers can create an archive with `npm pack` and install it with `npm install -g /path/to/open-context-engine-0.1.6.tgz`. See the [release checklist](https://github.com/AnnaSuSu/OpenContextEngine/blob/main/docs/RELEASING.md).
|
|
36
36
|
|
|
37
37
|
The CLI and package are named `open-context-engine`. The previous `opencontextengine` command remains an alias. Existing configuration and cache directories keep their paths, so saved keys and indexes are reused.
|
|
38
38
|
|
|
@@ -163,7 +163,7 @@ Quote paths that contain spaces. If the generated configuration includes `env` (
|
|
|
163
163
|
|
|
164
164
|
- **Command not found:** use the absolute paths above and confirm the Node executable still exists after a Node upgrade.
|
|
165
165
|
- **Tools missing:** check `codex mcp get open-context-engine` for the saved Codex entry, or `/mcp` inside Claude Code for connection status; restart the client after configuration changes.
|
|
166
|
-
- **
|
|
166
|
+
- **Search reports "Index is updating":** synchronization is still running in the background; the search has not run yet. MCP retries a pending update once, then returns a normal `status: "updating"` response with `retryable: true` if more time is needed. This is not an empty search result. Ask the agent to call `index_status` for the same project and retry `search_code` once ready. Progress shows the current stage and, during embedding, completed/total documents requiring new vectors for this update (cached vectors are excluded). Large repositories can take longer than a single tool call's timeout. If status reports an error, resolve that error before retrying.
|
|
167
167
|
- **Model configuration or authentication errors:** rerun `open-context-engine setup`. `open-context-engine doctor` checks local configuration and dependencies; an actual search checks access to the model APIs. If using SSH forwarding, keep the tunnel running.
|
|
168
168
|
|
|
169
169
|
Client references: [Codex MCP](https://developers.openai.com/codex/mcp) · [Claude Code MCP](https://code.claude.com/docs/en/mcp).
|
|
@@ -172,7 +172,7 @@ Client references: [Codex MCP](https://developers.openai.com/codex/mcp) · [Clau
|
|
|
172
172
|
|
|
173
173
|
Saved files are checked every second by default (`OCE_POLL_SECONDS=1`), with a 300 ms debounce (`OCE_DEBOUNCE_SECONDS=0.3`). New files, deletions, renames, and branch changes update the index automatically. Embeddings are reused by model identity and actual input content. Structural analysis conservatively refreshes the affected language group to update references in unchanged files.
|
|
174
174
|
|
|
175
|
-
Search actively checks source hashes before retrieval and again before returning.
|
|
175
|
+
Search actively checks source hashes before retrieval and again before returning. Each HTTP search waits up to 30 seconds for synchronization by default (`freshnessWaitMs`, maximum 120 seconds). MCP retries `INDEX_UPDATING` once with an additional wait of `min(freshnessWaitMs, 10000)` milliseconds, giving a default synchronization wait of up to 40 seconds across both attempts; `freshnessWaitMs: 0` disables waiting and retry. HTTP pending responses remain 503 and include `code: "INDEX_UPDATING"`, `retryable: true`, and index progress. MCP presents this specific condition as a normal updating status. Failed updates and edits during retrieval still produce explicit errors. Unsaved editor buffers are not indexed.
|
|
176
176
|
|
|
177
177
|
Python, JavaScript, TypeScript, and Go files with syntax errors (including unfilled templates) are indexed as plain text with their original paths and line numbers. Healthy files keep their structural analysis; no structural relations are inferred for degraded files. The index remains `ready`, while `index_status` reports `generation.degradedFiles` and `generation.parseDiagnostics` (path, language, error type, line, column, and fallback mode). Search responses include a degraded-file count and MCP displays a short notice. Diagnostics persist across restarts and disappear when the file is repaired or deleted. Model, toolchain, storage, and source-integrity failures still fail explicitly; stale source is never substituted.
|
|
178
178
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "open-context-engine",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.6",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"engines": {
|
|
6
6
|
"node": ">=22.14"
|
|
@@ -81,6 +81,6 @@
|
|
|
81
81
|
"access": "public",
|
|
82
82
|
"registry": "https://registry.npmjs.org/"
|
|
83
83
|
},
|
|
84
|
-
"readme": "<div align=\"center\">\n <picture>\n <source media=\"(prefers-color-scheme: dark)\" srcset=\"assets/brand/logo-lockup-dark.svg\">\n <img src=\"assets/brand/logo-lockup.svg\" alt=\"OpenContextEngine\" width=\"660\">\n </picture>\n <p><strong>Precise code context for AI coding agents.</strong></p>\n <p>Find related code across files. Follow its connections. Give your agent the evidence it needs.</p>\n <p>\n <a href=\"docs/BENCHMARKS.md#seven-method-comparison\"><img src=\"https://img.shields.io/badge/dev_evidence_coverage-94.79%25-23875b?style=flat-square\" alt=\"Development evidence coverage: 94.79%\"></a>\n <a href=\"docs/BENCHMARKS.md#seven-method-comparison\"><img src=\"https://img.shields.io/badge/median_retrieval-1.73_s-23875b?style=flat-square\" alt=\"Median retrieval: 1.73 seconds\"></a>\n <a href=\"docs/BENCHMARKS.md#engineering-validation\"><img src=\"https://img.shields.io/badge/verified_tests-107-23875b?style=flat-square\" alt=\"107 verified tests\"></a>\n <a href=\"docs/QUICKSTART.md\"><img src=\"https://img.shields.io/badge/MCP-stdio-193c34?style=flat-square\" alt=\"MCP over stdio\"></a>\n </p>\n <p><a href=\"#quick-start\">Quick start</a> · <a href=\"docs/BENCHMARKS.md\">Benchmarks</a> · <a href=\"docs/QUICKSTART.md\">MCP setup</a> · <strong>English</strong> | <a href=\"README.zh-CN.md\">简体中文</a></p>\n</div>\n\nOpenContextEngine is a **self-hostable code context engine for AI coding agents**. Connect it to your agent through MCP to help it explore an unfamiliar codebase, locate implementations, and find the related code needed for a fix or feature.\n\nIt indexes your working directory and follows saved changes. Given a natural-language task, it combines semantic and keyword search, code relationships, and reranking to return relevant source snippets with file paths and line numbers, within a fixed context budget.\n\n## Why OpenContextEngine\n\n- **Search beyond exact words.** Describe a behavior; retrieve its implementation and connected code across files.\n- **Understand code structure.** Python, TypeScript, JavaScript, and Go adapters, plus text fallback for other languages, configuration, and scripts.\n- **Stay current as you edit.** Saved changes, file deletions, and branch switches sync automatically. Unchanged embeddings are reused; incomplete updates never replace a complete index.\n- **Work across projects through MCP.** Your agent supplies the project path; indexes start on demand and are reused. `search_code` retrieves evidence; `index_status` reports synchronization.\n\n## Measured results\n\n\n\n**94.79% required evidence coverage · 1.73 s median retrieval · 69/80 queries with complete evidence.** Seven engines, four repositories, the same 4,000-token output budget. OpenContextEngine retained the most required evidence in this internal development evaluation.\n\nMeasured with the optional batch rerank API. 40 source-derived tasks, each asked in Chinese and English. Coverage measures source evidence, not coding-agent success. Timings reflect native retrieval for open tools and SDK client calls for ACE. [Full comparison, configurations, and per-query results →](docs/eval/METHOD_COMPARISON.md)\n\n## Quick start\n\nRequires **macOS, Linux, or Windows**, Node.js 22.14+, Python 3.10+, Git, and configured embedding/reranking services. Go repositories also need Go 1.22+.\n\nNative Windows support is available in version **0.1.3** and later. Run the installation commands in PowerShell. Version **0.1.4** adds automatic worker sharing across MCP sessions to prevent index-lock conflicts. Version **0.1.5** raises the default retrieval output budget to 8,000 tokens; pass `budget: 4000` when a smaller response is sufficient.\n\nInstall from npm, then expand your client's guide. Model settings are shared across clients on the same machine.\n\n<details open>\n<summary><strong>Codex — install, connect, and search</strong></summary>\n\n**1. Install the CLI**\n\nWith Codex CLI already installed, run:\n\n```sh\nnpm install -g open-context-engine\n```\n\n**2. Configure your models**\n\n```sh\nopen-context-engine setup\n```\n\nEnter your embedding and reranking base URLs, API keys, model names, and embedding dimensions. Setup installs isolated Python dependencies and saves your settings. The default reranker uses the ordinary `/rerank` API. [Endpoint examples →](docs/QUICKSTART.md#2-shared-model-configuration)\n\n**3. Add the MCP server**\n\n```sh\ncodex mcp add open-context-engine -- open-context-engine mcp\ncodex mcp get open-context-engine\n```\n\nThe second command checks the saved configuration. These commands assume `open-context-engine` is on the client's `PATH`. For the desktop app or source installations, use the [absolute-path configuration](docs/QUICKSTART.md#client-setup-notes). Restart an already-running Codex client after adding the server.\n\n**4. Search your project**\n\n```sh\ncd /path/to/your-project\ncodex\n```\n\nAsk:\n\n> Use open-context-engine's search_code tool to explain this project's main functionality. Include the entry points and relevant file paths and line numbers.\n\nCodex supplies the project's absolute path as `directory_path`. The first request starts indexing; if it is still building, ask Codex to check `index_status` and retry when ready. Later searches reuse the index, and saved changes update automatically.\n\n</details>\n\n<details>\n<summary><strong>Claude Code — install, connect, and search</strong></summary>\n\n**1. Install the CLI**\n\nWith Claude Code already installed, run:\n\n```sh\nnpm install -g open-context-engine\n```\n\n**2. Configure your models**\n\n```sh\nopen-context-engine setup\n```\n\nEnter your embedding and reranking base URLs, API keys, model names, and embedding dimensions. Setup installs isolated Python dependencies and saves your settings. If you already completed setup for Codex, reuse those settings and skip this step. [Endpoint examples →](docs/QUICKSTART.md#2-shared-model-configuration)\n\n**3. Add the MCP server**\n\n```sh\nclaude mcp add --transport stdio --scope user open-context-engine -- open-context-engine mcp\n```\n\nUser scope makes the server available across your projects. For a shared project configuration, run the command from that project and replace `--scope user` with `--scope project`. These commands assume `open-context-engine` is on the client's `PATH`; see [client setup notes](docs/QUICKSTART.md#client-setup-notes) for absolute paths. Restart an already-running Claude Code session after adding the server.\n\n**4. Search your project**\n\n```sh\ncd /path/to/your-project\nclaude\n```\n\nRun `/mcp` to check the connection, then ask:\n\n> Use open-context-engine's search_code tool to explain this project's main functionality. Include the entry points and relevant file paths and line numbers.\n\nClaude supplies the project's absolute path as `directory_path`. The first request starts indexing; if it is still building, ask Claude to check `index_status` and retry when ready. Later searches reuse the index, and saved changes update automatically.\n\n</details>\n\n<details>\n<summary>Other MCP clients</summary>\n\n[Install and run setup](docs/QUICKSTART.md#1-install), then use the configuration printed by `open-context-engine mcp-config` in your client's supported format. For clients that accept `mcpServers` JSON and can find the installed command on `PATH`:\n\n```json\n{\n \"mcpServers\": {\n \"open-context-engine\": {\n \"command\": \"open-context-engine\",\n \"args\": [\"mcp\"]\n }\n }\n}\n```\n\nYour agent supplies the current project's absolute path as `directory_path`. To pin one project, add `\"--root\", \"/absolute/path/to/your-repository\"` to `args`.\n\n</details>\n\n[Model configuration, troubleshooting, and update behavior →](docs/QUICKSTART.md)\n\n## Explore\n\n[Benchmark report](https://github.com/AnnaSuSu/OpenContextEngine/blob/main/docs/BENCHMARKS.md) · [Raw evaluations](https://github.com/AnnaSuSu/OpenContextEngine/tree/main/docs/eval/results) · [Retrieval engine](https://github.com/AnnaSuSu/OpenContextEngine/tree/main/src/retrieval)\n\n## License\n\n[MIT](LICENSE) © 2026 AnnaSuSu.\n\n## Acknowledgments\n\nThanks to the [LINUX DO](https://linux.do/) community.\n",
|
|
84
|
+
"readme": "<div align=\"center\">\n <picture>\n <source media=\"(prefers-color-scheme: dark)\" srcset=\"assets/brand/logo-lockup-dark.svg\">\n <img src=\"assets/brand/logo-lockup.svg\" alt=\"OpenContextEngine\" width=\"660\">\n </picture>\n <p><strong>Precise code context for AI coding agents.</strong></p>\n <p>Find related code across files. Follow its connections. Give your agent the evidence it needs.</p>\n <p>\n <a href=\"docs/BENCHMARKS.md#seven-method-comparison\"><img src=\"https://img.shields.io/badge/dev_evidence_coverage-94.79%25-23875b?style=flat-square\" alt=\"Development evidence coverage: 94.79%\"></a>\n <a href=\"docs/BENCHMARKS.md#seven-method-comparison\"><img src=\"https://img.shields.io/badge/median_retrieval-1.73_s-23875b?style=flat-square\" alt=\"Median retrieval: 1.73 seconds\"></a>\n <a href=\"docs/BENCHMARKS.md#engineering-validation\"><img src=\"https://img.shields.io/badge/verified_tests-107-23875b?style=flat-square\" alt=\"107 verified tests\"></a>\n <a href=\"docs/QUICKSTART.md\"><img src=\"https://img.shields.io/badge/MCP-stdio-193c34?style=flat-square\" alt=\"MCP over stdio\"></a>\n </p>\n <p><a href=\"#quick-start\">Quick start</a> · <a href=\"docs/BENCHMARKS.md\">Benchmarks</a> · <a href=\"docs/QUICKSTART.md\">MCP setup</a> · <strong>English</strong> | <a href=\"README.zh-CN.md\">简体中文</a></p>\n</div>\n\nOpenContextEngine is a **self-hostable code context engine for AI coding agents**. Connect it to your agent through MCP to help it explore an unfamiliar codebase, locate implementations, and find the related code needed for a fix or feature.\n\nIt indexes your working directory and follows saved changes. Given a natural-language task, it combines semantic and keyword search, code relationships, and reranking to return relevant source snippets with file paths and line numbers, within a fixed context budget.\n\n## Why OpenContextEngine\n\n- **Search beyond exact words.** Describe a behavior; retrieve its implementation and connected code across files.\n- **Understand code structure.** Python, TypeScript, JavaScript, and Go adapters, plus text fallback for other languages, configuration, and scripts.\n- **Stay current as you edit.** Saved changes, file deletions, and branch switches sync automatically. Unchanged embeddings are reused; incomplete updates never replace a complete index.\n- **Work across projects through MCP.** Your agent supplies the project path; indexes start on demand and are reused. `search_code` retrieves evidence; `index_status` reports synchronization.\n\n## Measured results\n\n\n\n**94.79% required evidence coverage · 1.73 s median retrieval · 69/80 queries with complete evidence.** Seven engines, four repositories, the same 4,000-token output budget. OpenContextEngine retained the most required evidence in this internal development evaluation.\n\nMeasured with the optional batch rerank API. 40 source-derived tasks, each asked in Chinese and English. Coverage measures source evidence, not coding-agent success. Timings reflect native retrieval for open tools and SDK client calls for ACE. [Full comparison, configurations, and per-query results →](docs/eval/METHOD_COMPARISON.md)\n\n## Quick start\n\nRequires **macOS, Linux, or Windows**, Node.js 22.14+, Python 3.10+, Git, and configured embedding/reranking services. Go repositories also need Go 1.22+.\n\nNative Windows support is available in version **0.1.3** and later. Run the installation commands in PowerShell. Version **0.1.4** adds automatic worker sharing across MCP sessions to prevent index-lock conflicts. Version **0.1.5** raises the default retrieval output budget to 8,000 tokens; pass `budget: 4000` when a smaller response is sufficient. Version **0.1.6** reports pending indexing as progress, retries once, and keeps actual indexing failures explicit.\n\nInstall from npm, then expand your client's guide. Model settings are shared across clients on the same machine.\n\n<details open>\n<summary><strong>Codex — install, connect, and search</strong></summary>\n\n**1. Install the CLI**\n\nWith Codex CLI already installed, run:\n\n```sh\nnpm install -g open-context-engine\n```\n\n**2. Configure your models**\n\n```sh\nopen-context-engine setup\n```\n\nEnter your embedding and reranking base URLs, API keys, model names, and embedding dimensions. Setup installs isolated Python dependencies and saves your settings. The default reranker uses the ordinary `/rerank` API. [Endpoint examples →](docs/QUICKSTART.md#2-shared-model-configuration)\n\n**3. Add the MCP server**\n\n```sh\ncodex mcp add open-context-engine -- open-context-engine mcp\ncodex mcp get open-context-engine\n```\n\nThe second command checks the saved configuration. These commands assume `open-context-engine` is on the client's `PATH`. For the desktop app or source installations, use the [absolute-path configuration](docs/QUICKSTART.md#client-setup-notes). Restart an already-running Codex client after adding the server.\n\n**4. Search your project**\n\n```sh\ncd /path/to/your-project\ncodex\n```\n\nAsk:\n\n> Use open-context-engine's search_code tool to explain this project's main functionality. Include the entry points and relevant file paths and line numbers.\n\nCodex supplies the project's absolute path as `directory_path`. The first request starts indexing; if it is still building, ask Codex to check `index_status` and retry when ready. Later searches reuse the index, and saved changes update automatically.\n\n</details>\n\n<details>\n<summary><strong>Claude Code — install, connect, and search</strong></summary>\n\n**1. Install the CLI**\n\nWith Claude Code already installed, run:\n\n```sh\nnpm install -g open-context-engine\n```\n\n**2. Configure your models**\n\n```sh\nopen-context-engine setup\n```\n\nEnter your embedding and reranking base URLs, API keys, model names, and embedding dimensions. Setup installs isolated Python dependencies and saves your settings. If you already completed setup for Codex, reuse those settings and skip this step. [Endpoint examples →](docs/QUICKSTART.md#2-shared-model-configuration)\n\n**3. Add the MCP server**\n\n```sh\nclaude mcp add --transport stdio --scope user open-context-engine -- open-context-engine mcp\n```\n\nUser scope makes the server available across your projects. For a shared project configuration, run the command from that project and replace `--scope user` with `--scope project`. These commands assume `open-context-engine` is on the client's `PATH`; see [client setup notes](docs/QUICKSTART.md#client-setup-notes) for absolute paths. Restart an already-running Claude Code session after adding the server.\n\n**4. Search your project**\n\n```sh\ncd /path/to/your-project\nclaude\n```\n\nRun `/mcp` to check the connection, then ask:\n\n> Use open-context-engine's search_code tool to explain this project's main functionality. Include the entry points and relevant file paths and line numbers.\n\nClaude supplies the project's absolute path as `directory_path`. The first request starts indexing; if it is still building, ask Claude to check `index_status` and retry when ready. Later searches reuse the index, and saved changes update automatically.\n\n</details>\n\n<details>\n<summary>Other MCP clients</summary>\n\n[Install and run setup](docs/QUICKSTART.md#1-install), then use the configuration printed by `open-context-engine mcp-config` in your client's supported format. For clients that accept `mcpServers` JSON and can find the installed command on `PATH`:\n\n```json\n{\n \"mcpServers\": {\n \"open-context-engine\": {\n \"command\": \"open-context-engine\",\n \"args\": [\"mcp\"]\n }\n }\n}\n```\n\nYour agent supplies the current project's absolute path as `directory_path`. To pin one project, add `\"--root\", \"/absolute/path/to/your-repository\"` to `args`.\n\n</details>\n\n[Model configuration, troubleshooting, and update behavior →](docs/QUICKSTART.md)\n\n## Explore\n\n[Benchmark report](https://github.com/AnnaSuSu/OpenContextEngine/blob/main/docs/BENCHMARKS.md) · [Raw evaluations](https://github.com/AnnaSuSu/OpenContextEngine/tree/main/docs/eval/results) · [Retrieval engine](https://github.com/AnnaSuSu/OpenContextEngine/tree/main/src/retrieval)\n\n## License\n\n[MIT](LICENSE) © 2026 AnnaSuSu.\n\n## Acknowledgments\n\nThanks to the [LINUX DO](https://linux.do/) community.\n",
|
|
85
85
|
"readmeFilename": "README.md"
|
|
86
86
|
}
|
|
@@ -144,7 +144,8 @@ def serve(config):
|
|
|
144
144
|
response['diagnostics'] = debug
|
|
145
145
|
self.reply(200,response)
|
|
146
146
|
except IndexUnavailable as error:
|
|
147
|
-
self.reply(503,{'error':str(error),'
|
|
147
|
+
self.reply(503,{'error':str(error),'code':error.code,
|
|
148
|
+
'retryable':error.code == 'INDEX_UPDATING','index':live.status()})
|
|
148
149
|
except Exception as error:
|
|
149
150
|
print(json.dumps({'event':'search-failed','type':type(error).__name__}),flush=True)
|
|
150
151
|
self.reply(502,{'error':'Retrieval or model request failed'})
|
package/src/client.mjs
CHANGED
|
@@ -18,7 +18,9 @@ export async function search(query,{budget=8000,trace=false,freshnessWaitMs=3000
|
|
|
18
18
|
if (!response.ok) {
|
|
19
19
|
if (response.status === 503) {
|
|
20
20
|
const body = await response.json();
|
|
21
|
-
throw new Error(`Index unavailable: ${body.error || 'update pending'}`)
|
|
21
|
+
throw Object.assign(new Error(`Index unavailable: ${body.error || 'update pending'}`), {
|
|
22
|
+
code:body.code, index:body.index,
|
|
23
|
+
});
|
|
22
24
|
}
|
|
23
25
|
if (response.status === 429) {
|
|
24
26
|
const body = await response.json();
|
package/src/mcp.mjs
CHANGED
|
@@ -2,6 +2,24 @@ import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
|
|
|
2
2
|
import { z } from 'zod';
|
|
3
3
|
import { search, indexStatus } from './client.mjs';
|
|
4
4
|
|
|
5
|
+
function updatingResult(error) {
|
|
6
|
+
const progress = error.index?.progress;
|
|
7
|
+
let detail = '';
|
|
8
|
+
if (progress?.stage === 'embedding') {
|
|
9
|
+
detail = ` Embedding documents: ${progress.completedDocuments}/${progress.totalDocuments} completed for this update.`;
|
|
10
|
+
} else if (progress?.stage === 'parsing') {
|
|
11
|
+
detail = ` Parsing ${progress.files} files.`;
|
|
12
|
+
} else if (progress?.stage === 'finalizing') {
|
|
13
|
+
detail = ' Finalizing the searchable index.';
|
|
14
|
+
} else if (progress?.stage === 'source_changed') {
|
|
15
|
+
detail = ' Saved files changed during indexing; synchronizing the latest version.';
|
|
16
|
+
}
|
|
17
|
+
return {isError:false,
|
|
18
|
+
content:[{type:'text',text:'Index is updating; search has not run yet.' + detail
|
|
19
|
+
+ ' Background synchronization continues. Check index_status for this project and retry search_code once ready.'}],
|
|
20
|
+
structuredContent:{status:'updating',code:'INDEX_UPDATING',retryable:true,index:error.index}};
|
|
21
|
+
}
|
|
22
|
+
|
|
5
23
|
export function createMcpServer(config, {resolveConfig, automatic = false} = {}) {
|
|
6
24
|
const workspaceInstructions = automatic
|
|
7
25
|
? 'Pass directory_path as the absolute directory of the project the user is working on with every tool call. '
|
|
@@ -15,9 +33,9 @@ export function createMcpServer(config, {resolveConfig, automatic = false} = {})
|
|
|
15
33
|
}
|
|
16
34
|
const pathSchema = z.string().min(1).describe('Absolute path to the project directory. Required in automatic workspace mode.');
|
|
17
35
|
const directoryPath = automatic ? pathSchema : pathSchema.optional();
|
|
18
|
-
const server = new McpServer({name:'open-context-engine',version:'0.1.
|
|
36
|
+
const server = new McpServer({name:'open-context-engine',version:'0.1.6'}, {
|
|
19
37
|
instructions:workspaceInstructions + 'Search for source evidence. Results include source paths and line numbers. '
|
|
20
|
-
+ 'Search waits for saved file changes
|
|
38
|
+
+ 'Search waits for saved file changes and retries a pending update once. A status of updating means search has not run; inspect index_status and retry once ready. '
|
|
21
39
|
+ 'Read target files again before editing, because code may change after a search.',
|
|
22
40
|
});
|
|
23
41
|
const annotations = {readOnlyHint:true,destructiveHint:false,idempotentHint:true,openWorldHint:false};
|
|
@@ -27,17 +45,26 @@ export function createMcpServer(config, {resolveConfig, automatic = false} = {})
|
|
|
27
45
|
+ workspaceInstructions
|
|
28
46
|
+ 'Uses saved working-tree content, including uncommitted changes; rejects stale results when synchronization fails.',
|
|
29
47
|
inputSchema:{directory_path:directoryPath,query:z.string().trim().min(1).max(8192),budget:z.number().int().min(256).max(8000).default(8000),
|
|
30
|
-
freshnessWaitMs:z.number().int().min(0).max(120000).default(30000)
|
|
48
|
+
freshnessWaitMs:z.number().int().min(0).max(120000).default(30000)
|
|
49
|
+
.describe('Initial synchronization wait. Pending updates are retried once for up to 10 additional seconds; 0 disables waiting and retry.')},
|
|
31
50
|
annotations,
|
|
32
51
|
}, async ({directory_path,query,budget,freshnessWaitMs}, extra) => {
|
|
33
52
|
try {
|
|
34
53
|
const selected = await selectConfig(directory_path);
|
|
35
|
-
|
|
54
|
+
let result;
|
|
55
|
+
try {
|
|
56
|
+
result = await search(query,{budget,freshnessWaitMs,config:selected,signal:extra.signal});
|
|
57
|
+
} catch (error) {
|
|
58
|
+
extra.signal.throwIfAborted();
|
|
59
|
+
if (error.code !== 'INDEX_UPDATING' || freshnessWaitMs === 0) throw error;
|
|
60
|
+
result = await search(query,{budget,freshnessWaitMs:Math.min(freshnessWaitMs,10000),config:selected,signal:extra.signal});
|
|
61
|
+
}
|
|
36
62
|
if (result.index?.mode !== 'live') throw new Error('This service uses a frozen index; connect to a service started with --root');
|
|
37
63
|
const warning = result.index.degradedFiles
|
|
38
64
|
? `Note: ${result.index.degradedFiles} file(s) indexed as plain text after syntax errors; inspect index_status for paths and locations.\n\n` : '';
|
|
39
65
|
return {content:[{type:'text',text:warning + (result.context || 'No matching source context.')}],structuredContent:result};
|
|
40
66
|
} catch (error) {
|
|
67
|
+
if (error.code === 'INDEX_UPDATING') return updatingResult(error);
|
|
41
68
|
return {isError:true,content:[{type:'text',text:error.message}]};
|
|
42
69
|
}
|
|
43
70
|
});
|
package/src/retrieval/live.py
CHANGED
|
@@ -28,6 +28,10 @@ def digest(value):
|
|
|
28
28
|
class IndexUnavailable(Exception):
|
|
29
29
|
"""The requested working tree has no complete, current searchable generation."""
|
|
30
30
|
|
|
31
|
+
def __init__(self, message, *, code='INDEX_UNAVAILABLE'):
|
|
32
|
+
super().__init__(message)
|
|
33
|
+
self.code = code
|
|
34
|
+
|
|
31
35
|
|
|
32
36
|
class SourceChanged(Exception):
|
|
33
37
|
pass
|
|
@@ -73,6 +77,7 @@ class LiveIndex:
|
|
|
73
77
|
self.generation = None
|
|
74
78
|
self.error = None
|
|
75
79
|
self.phase = 'starting'
|
|
80
|
+
self.progress = None
|
|
76
81
|
self.parse_cache = {}
|
|
77
82
|
self.state.mkdir(parents=True, exist_ok=True, mode=0o700)
|
|
78
83
|
self.lock_file = acquire_writer_lock(self.state/'writer.lock')
|
|
@@ -109,7 +114,12 @@ class LiveIndex:
|
|
|
109
114
|
with self.condition:
|
|
110
115
|
return {'status': self.phase, 'mode': 'live', 'root': str(self.root),
|
|
111
116
|
'generation': self.generation.info if self.generation else None,
|
|
112
|
-
'error': self.error, 'pollSeconds': self.poll
|
|
117
|
+
'error': self.error, 'pollSeconds': self.poll,
|
|
118
|
+
'progress': dict(self.progress) if self.progress else None}
|
|
119
|
+
|
|
120
|
+
def _progress(self, stage, **counts):
|
|
121
|
+
with self.condition:
|
|
122
|
+
self.progress = {'stage': stage, **counts}
|
|
113
123
|
|
|
114
124
|
def current(self, timeout=30):
|
|
115
125
|
deadline = time.monotonic() + timeout
|
|
@@ -125,7 +135,7 @@ class LiveIndex:
|
|
|
125
135
|
raise IndexUnavailable('Index update failed: ' + self.error['type'])
|
|
126
136
|
remaining = deadline - time.monotonic()
|
|
127
137
|
if remaining <= 0:
|
|
128
|
-
raise IndexUnavailable('Index
|
|
138
|
+
raise IndexUnavailable('Index is updating; search has not run yet', code='INDEX_UPDATING')
|
|
129
139
|
self.condition.wait(min(remaining, self.poll))
|
|
130
140
|
raise IndexUnavailable('Index is stopping')
|
|
131
141
|
|
|
@@ -167,6 +177,7 @@ class LiveIndex:
|
|
|
167
177
|
|
|
168
178
|
def _build(self, snapshot, identity):
|
|
169
179
|
start = time.monotonic()
|
|
180
|
+
self._progress('parsing', files=len(snapshot['files']))
|
|
170
181
|
report = {}
|
|
171
182
|
units = source_units(self.root, snapshot['files'], language_options=self.options,
|
|
172
183
|
cache=self.parse_cache, report=report)
|
|
@@ -184,6 +195,7 @@ class LiveIndex:
|
|
|
184
195
|
continue
|
|
185
196
|
missing[key] = text
|
|
186
197
|
entries = list(missing.items())
|
|
198
|
+
self._progress('embedding', completedDocuments=0, totalDocuments=len(entries))
|
|
187
199
|
for offset in range(0, len(entries), self.batch_size):
|
|
188
200
|
if self.stop_event.is_set():
|
|
189
201
|
raise SourceChanged()
|
|
@@ -201,6 +213,8 @@ class LiveIndex:
|
|
|
201
213
|
vectors[key] = vector
|
|
202
214
|
db.execute('INSERT OR REPLACE INTO vectors VALUES (?, ?)', (key, vector.tobytes()))
|
|
203
215
|
db.commit() # Reuse successful batches even after a concurrent edit.
|
|
216
|
+
self._progress('embedding', completedDocuments=offset+len(batch), totalDocuments=len(entries))
|
|
217
|
+
self._progress('finalizing')
|
|
204
218
|
matrix = np.asarray([vectors[key] for key in keys], dtype=np.float32).reshape(len(keys), self.dimensions)
|
|
205
219
|
engine = self._engine(units, matrix)
|
|
206
220
|
if self.stop_event.is_set() or self.scan() != snapshot:
|
|
@@ -243,6 +257,7 @@ class LiveIndex:
|
|
|
243
257
|
if self.generation and self.generation.identity == identity:
|
|
244
258
|
with self.condition:
|
|
245
259
|
self.phase, self.error = 'ready', None
|
|
260
|
+
self.progress = None
|
|
246
261
|
self.stop_event.wait(self.poll)
|
|
247
262
|
continue
|
|
248
263
|
if failed_identity == identity and time.monotonic() < retry_at:
|
|
@@ -250,6 +265,7 @@ class LiveIndex:
|
|
|
250
265
|
continue
|
|
251
266
|
with self.condition:
|
|
252
267
|
self.phase, self.error = 'updating', None
|
|
268
|
+
self.progress = {'stage': 'checking', 'files': len(snapshot['files'])}
|
|
253
269
|
if self.stop_event.wait(self.debounce):
|
|
254
270
|
break
|
|
255
271
|
if self.scan() != snapshot:
|
|
@@ -259,9 +275,11 @@ class LiveIndex:
|
|
|
259
275
|
raise SourceChanged()
|
|
260
276
|
with self.condition:
|
|
261
277
|
self.generation, self.phase, self.error = generation, 'ready', None
|
|
278
|
+
self.progress = None
|
|
262
279
|
failed_identity = None
|
|
263
280
|
self.condition.notify_all()
|
|
264
281
|
except SourceChanged:
|
|
282
|
+
self._progress('source_changed')
|
|
265
283
|
continue
|
|
266
284
|
except Exception as error:
|
|
267
285
|
failed_identity, retry_at = identity, time.monotonic() + max(2, self.poll)
|