beacon-mfg-mcp 1.3.1 → 1.3.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +19 -2
- package/RELEASE.md +28 -4
- package/bin/beacon-mfg-mcp.js +6 -0
- package/package.json +1 -1
- package/server.py +268 -91
package/README.md
CHANGED
|
@@ -46,8 +46,18 @@ npm i -g beacon-mfg-mcp # 或不安装直接用 npx beacon-mfg-mcp
|
|
|
46
46
|
```
|
|
47
47
|
|
|
48
48
|
```jsonc
|
|
49
|
-
{
|
|
49
|
+
{
|
|
50
|
+
"mcpServers": {
|
|
51
|
+
"beacon-mfg": {
|
|
52
|
+
"command": "beacon-mfg-mcp",
|
|
53
|
+
"args": [],
|
|
54
|
+
"env": { "PYTHONUTF8": "1", "PYTHONIOENCODING": "utf-8" }
|
|
55
|
+
}
|
|
56
|
+
}
|
|
57
|
+
}
|
|
50
58
|
```
|
|
59
|
+
> 启动器(`bin/beacon-mfg-mcp.js`)已默认注入上述两个变量;此处写出仅为显式/可覆盖。
|
|
60
|
+
> 二者保证 python 子进程以 UTF-8 输出,否则中文 Windows 上工具描述/结果会乱码。
|
|
51
61
|
|
|
52
62
|
不填 `BEACON_REPO` 时默认走 `https://beacon-mfg.pages.dev`,即开即用。
|
|
53
63
|
|
|
@@ -92,6 +102,13 @@ npm i -g beacon-mfg-mcp # 或不安装直接用 npx beacon-mfg-mcp
|
|
|
92
102
|
HTTP 模式带正确 `User-Agent` 与 `ETag` 304 缓存(与 App 同款策略),并落盘 `~/.cache/beacon-mcp-cache`,
|
|
93
103
|
重复查询零传输。CF 部署版手机号可能是全号——隐私敏感场景请用方式 A。
|
|
94
104
|
|
|
105
|
+
> **CDN 兜底镜像(v1.3.2 新增)**:默认源 `beacon-mfg.pages.dev` 不可达(抖动/被墙)时,
|
|
106
|
+
> MCP 会**自动回退**到 GitHub 兜底镜像拉 `manifest.json` 与分片——`cdn.jsdelivr.net/gh/eiry16/beacon-mfg@main`
|
|
107
|
+
> 与 `raw.githubusercontent.com/eiry16/beacon-mfg/main`(与 Android App 同策略)。因此「免 clone 检索」
|
|
108
|
+
> 在单一 CDN 失效时仍可工作。如需自定义/追加镜像,设 `BEACON_MIRRORS`(逗号分隔的基址列表)。
|
|
109
|
+
> 注意:能力卡(`get_capability_card`)不在 git 内、仅存于 CF/R2,镜像兜底覆盖的是 manifest + 分片 + 档案这类主数据;
|
|
110
|
+
> 在中国大陆等直连 GitHub 困难的网络,给 MCP 进程设 `HTTPS_PROXY=http://127.0.0.1:7890`(或你的代理)可让镜像链路走代理。
|
|
111
|
+
|
|
95
112
|
## 远程 HTTP 端点(Streamable HTTP)
|
|
96
113
|
|
|
97
114
|
`server.py` 本体是 **stdio**;`http_server.py` 是它的 **HTTP 传输适配层**(复用同一份 `_dispatch`/`TOOLS`,
|
|
@@ -169,7 +186,7 @@ MCP 服务有两种发布渠道(详见 `mcp/RELEASE.md`):
|
|
|
169
186
|
|
|
170
187
|
- **GitHub Release(已配 CI)**:推送 `mcp-v*` 标签即由 `.github/workflows/mcp-release.yml`
|
|
171
188
|
自动打包 `mcp/` 目录为 `beacon-mfg-mcp-mcp-vX.Y.Z.tar.gz` 并创建 Release。
|
|
172
|
-
- **npm 包(已发布 ✅)**:`beacon-mfg-mcp` 已发布到 npm 公共仓库,当前版本 **`1.3.
|
|
189
|
+
- **npm 包(已发布 ✅)**:`beacon-mfg-mcp` 已发布到 npm 公共仓库,当前版本 **`1.3.2`**
|
|
173
190
|
(<https://www.npmjs.com/package/beacon-mfg-mcp>)。第三方可直接 `npm i -g beacon-mfg-mcp`
|
|
174
191
|
或 `npx beacon-mfg-mcp`,客户端配置 `"command": "beacon-mfg-mcp"` 即可,无需 clone、无需填路径。
|
|
175
192
|
**1.3.0 起 npm 包自带 rfq-kernel**,安装 → MCP 握手 → `tools/list` 暴露 6 个 tool 且全部可用
|
package/RELEASE.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Beacon-MFG 只读 MCP 服务 · v1.3.
|
|
1
|
+
# Beacon-MFG 只读 MCP 服务 · v1.3.3
|
|
2
2
|
|
|
3
3
|
让任意支持 MCP 的主流 agent(Claude Desktop / Cline / Continue / WorkBuddy 等)能够
|
|
4
4
|
**检索与调用**已发布到 Cloudflare Pages(或 GitHub)的灯塔工厂供应商数据,
|
|
@@ -21,7 +21,29 @@ MCP 只做**只读检索**。后端数据采集、英文翻译、库维护等流
|
|
|
21
21
|
后三个构成「采购寻源」多轮会话链路 `start_sourcing → answer_sourcing → refine_sourcing`,
|
|
22
22
|
依赖仓库平级的 `skills/rfq-kernel` 桥接模块。
|
|
23
23
|
|
|
24
|
-
## 本次更新(v1.3.
|
|
24
|
+
## 本次更新(v1.3.3)
|
|
25
|
+
- **本地检索免提交、免重启即时可见**:这是 v1.3.1「worktree 兜底」的收尾。v1.3.1 只让**单文件读**走了工作树,
|
|
26
|
+
但关键词/城市检索的**召回**走的是**批量读** `_read_many_text()`——它在 git 模式下恒走 `git archive HEAD`
|
|
27
|
+
(已提交快照),于是**改完数据、重建索引后,关键词检索仍读旧索引、必须 commit 才可见**。
|
|
28
|
+
本次把批量读也改为「**工作树优先 → `git archive HEAD` 兜底 → HTTP**」。
|
|
29
|
+
- **进程内 + 磁盘缓存按文件新鲜度失效**:新增 `_wt_mtime` / `_cache_stale`,各 loader
|
|
30
|
+
(manifest / 别名 / 国标名表 / cap 索引 / city / 索引桶 / fp 分片 / 能力词)改为
|
|
31
|
+
**按工作树文件 mtime 判新**——派生层一重建,MCP 立刻重读,**无需重启进程**;
|
|
32
|
+
磁盘索引缓存(`idx-*`)加 **mtime 印记**(`repr` 精确往返),同秒重写也能识别。
|
|
33
|
+
- **可选 TTL**:`BEACON_CACHE_TTL`(秒,默认 `0`=关闭)可让进程内缓存周期性刷新;
|
|
34
|
+
非工作树模式(`BEACON_WORKTREE=0`)保持旧的「提交后 / 重启后生效」行为不变。
|
|
35
|
+
- 已用**真实注入实验**验证:单进程、不 commit、只 sleep,`data/gb` 新记录经本地重建后
|
|
36
|
+
关键词检索由 `0` 命中变 `1` 命中。
|
|
37
|
+
|
|
38
|
+
## 本次更新(v1.3.2,已发布)
|
|
39
|
+
- **npm 启动器锁定 UTF-8**:`bin/beacon-mfg-mcp.js` 拉起 python 时注入 `PYTHONUTF8=1` / `PYTHONIOENCODING=utf-8`,
|
|
40
|
+
修复中文 Windows 上 python 按 GBK 写 stdout、被客户端按 UTF-8 解码导致的工具描述/结果乱码。
|
|
41
|
+
- **HTTP 兜底镜像**:默认源 `beacon-mfg.pages.dev` 不可达时,自动回退到 GitHub 镜像
|
|
42
|
+
`cdn.jsdelivr.net/gh/eiry16/beacon-mfg@main` 与 `raw.githubusercontent.com/eiry16/beacon-mfg/main`
|
|
43
|
+
(与 Android App 同策略),保证「免 clone 检索」在单一 CDN 失效时仍可工作;可用 `BEACON_MIRRORS` 追加。
|
|
44
|
+
- **serverInfo 版本对齐**:服务端自报版本改为读 `package.json`,不再硬编码,消除与 npm 包版本漂移。
|
|
45
|
+
|
|
46
|
+
## 本次更新(v1.3.1,已发布)
|
|
25
47
|
- **npm 包自带 rfq-kernel**:`prepack` 会把 `skills/rfq-kernel` 打进包内,因此 `npm i -g beacon-mfg-mcp`
|
|
26
48
|
安装后 6 个 tool 全部可用(早于此版本的 npm 包不含该桥接,3 个寻源 tool 会降级)。
|
|
27
49
|
GitHub Release tarball 同步包含 `rfq-kernel/`。
|
|
@@ -32,7 +54,7 @@ MCP 只做**只读检索**。后端数据采集、英文翻译、库维护等流
|
|
|
32
54
|
## 安装(三选一)
|
|
33
55
|
|
|
34
56
|
### 1. 从 GitHub Release 下载(推荐,零依赖)
|
|
35
|
-
下载本 Release 的 `beacon-mfg-mcp-mcp-v1.3.
|
|
57
|
+
下载本 Release 的 `beacon-mfg-mcp-mcp-v1.3.3.tar.gz`,解压后:
|
|
36
58
|
```jsonc
|
|
37
59
|
{
|
|
38
60
|
"mcpServers": {
|
|
@@ -55,7 +77,7 @@ npx beacon-mfg-mcp
|
|
|
55
77
|
```jsonc
|
|
56
78
|
{
|
|
57
79
|
"mcpServers": {
|
|
58
|
-
"beacon-mfg": { "command": "beacon-mfg-mcp", "args": [] }
|
|
80
|
+
"beacon-mfg": { "command": "beacon-mfg-mcp", "args": [], "env": { "PYTHONUTF8": "1", "PYTHONIOENCODING": "utf-8" } }
|
|
59
81
|
}
|
|
60
82
|
}
|
|
61
83
|
```
|
|
@@ -70,6 +92,8 @@ git clone https://github.com/eiry16/beacon-mfg.git
|
|
|
70
92
|
- `BEACON_REPO`:本地 git 仓库路径;设了就用 `git show HEAD:` + 工作树 读(推荐:离线 + 掩码态隐私安全)。
|
|
71
93
|
- `BEACON_SOURCE`:HTTP 基址,默认 `https://beacon-mfg.pages.dev`(可改自定义域名)。
|
|
72
94
|
- `BEACON_WORKTREE`:本地模式是否优先读工作树(默认 `1`;设 `0` 恢复「只读已提交快照」)。
|
|
95
|
+
- `BEACON_MIRRORS`:HTTP 模式追加的兜底镜像基址(逗号分隔);默认源不可达时除内置的 jsDelivr/raw 外,可在此追加自有镜像。
|
|
96
|
+
- `HTTPS_PROXY` / `HTTP_PROXY`:python 子进程的代理;在中国大陆等直连 GitHub 困难的网络,设 `HTTPS_PROXY=http://127.0.0.1:7890` 可让镜像链路走代理。
|
|
73
97
|
|
|
74
98
|
## 许可
|
|
75
99
|
MIT
|
package/bin/beacon-mfg-mcp.js
CHANGED
|
@@ -26,5 +26,11 @@ if (!chosen) {
|
|
|
26
26
|
|
|
27
27
|
const child = spawnSync(chosen, [serverPy, ...process.argv.slice(2)], {
|
|
28
28
|
stdio: 'inherit',
|
|
29
|
+
// 显式锁定 UTF-8:中文 Windows 上 python 默认按 GBK 写 stdout,
|
|
30
|
+
// MCP 客户端按 UTF-8 解码会乱码。从启动器层面保证子进程用 UTF-8 输出。
|
|
31
|
+
env: Object.assign({}, process.env, {
|
|
32
|
+
PYTHONUTF8: '1',
|
|
33
|
+
PYTHONIOENCODING: 'utf-8',
|
|
34
|
+
}),
|
|
29
35
|
});
|
|
30
36
|
process.exit(child.status === null ? 1 : child.status);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "beacon-mfg-mcp",
|
|
3
|
-
"version": "1.3.
|
|
3
|
+
"version": "1.3.3",
|
|
4
4
|
"description": "Read-only MCP server exposing the BeaconMFG灯塔工厂 supplier directory (search vendors, get vendor detail, get capability card, multi-turn sourcing) to any MCP-compatible agent. No API key required.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"mcp",
|
package/server.py
CHANGED
|
@@ -208,6 +208,90 @@ _alias_cache: Optional[Dict[str, Any]] = None
|
|
|
208
208
|
_gb_name_cache: Optional[Dict[str, str]] = None
|
|
209
209
|
|
|
210
210
|
|
|
211
|
+
# ───────────────────────────────────────────────────────────────
|
|
212
|
+
# 派生层缓存失效(让「后台上传 → 立刻可见」无需提交 / 无需重启)
|
|
213
|
+
#
|
|
214
|
+
# BEACON_WORKTREE=1(默认)下,postfetch 会把 fp / 倒排索引 / 清单等派生层重建到
|
|
215
|
+
# 工作树。下面这组机制让 MCP 在不提交、不重启的前提下读到新版本:
|
|
216
|
+
# - 磁盘索引缓存(idx-*)按「HEAD sha + 工作树 mtime」判新(见 _index_text)
|
|
217
|
+
# - 进程内缓存(manifest/alias/cap/gb_name/fp_index/桶/分片)按文件 mtime 判新
|
|
218
|
+
# (见 _cache_stale),或按 BEACON_CACHE_TTL(秒)周期刷新。
|
|
219
|
+
# - 批量读(_read_many_text,被 _candidate_shards / _records_from_paths 使用)
|
|
220
|
+
# 改为**工作树优先**、git archive HEAD 兜底 —— 否则这两条关键词 / 城市检索的
|
|
221
|
+
# 关键路径会绕过上面的失效机制,永远读到已提交快照。
|
|
222
|
+
# 非工作树模式(BEACON_WORKTREE=0)或 HTTP 模式:恢复旧的「提交后 / 重启后生效」。
|
|
223
|
+
BEACON_CACHE_TTL = float(os.environ.get("BEACON_CACHE_TTL", "0") or "0")
|
|
224
|
+
|
|
225
|
+
_mtime_cache: Dict[str, tuple] = {} # relpath -> (checked_at, mtime),2s 内复用避免热路径狂 stat
|
|
226
|
+
|
|
227
|
+
|
|
228
|
+
def _wt_mtime(relpath: str) -> Optional[float]:
|
|
229
|
+
"""工作树文件 mtime(秒)。非工作树模式或文件不存在 → None(不按 mtime 判新)。"""
|
|
230
|
+
if not REPO or os.environ.get("BEACON_WORKTREE", "1") == "0":
|
|
231
|
+
return None
|
|
232
|
+
now = time.time()
|
|
233
|
+
c = _mtime_cache.get(relpath)
|
|
234
|
+
if c is not None and (now - c[0]) < 2.0:
|
|
235
|
+
return c[1]
|
|
236
|
+
p = os.path.join(REPO, relpath)
|
|
237
|
+
try:
|
|
238
|
+
m = os.stat(p).st_mtime
|
|
239
|
+
except OSError:
|
|
240
|
+
m = None
|
|
241
|
+
_mtime_cache[relpath] = (now, m)
|
|
242
|
+
return m
|
|
243
|
+
|
|
244
|
+
|
|
245
|
+
def _wt_mtime_any(relpaths) -> Optional[float]:
|
|
246
|
+
"""多文件取最大 mtime(任一不存在则忽略),全不存在 → None。"""
|
|
247
|
+
mts = [m for m in (_wt_mtime(r) for r in relpaths) if m is not None]
|
|
248
|
+
return max(mts) if mts else None
|
|
249
|
+
|
|
250
|
+
|
|
251
|
+
def _cache_stale(slot: tuple, relpath: Optional[str]) -> bool:
|
|
252
|
+
"""进程内缓存是否该失效。slot = (value, loaded_at, sig_mtime)。
|
|
253
|
+
value 为空 / 超 BEACON_CACHE_TTL / 工作树 mtime 变化(postfetch 重建)→ 失效。
|
|
254
|
+
relpath 可为单路径或路径序列(取最大 mtime)。非工作树模式(mtime 为 None)则只受
|
|
255
|
+
TTL 与首次加载约束,即旧行为。
|
|
256
|
+
"""
|
|
257
|
+
if slot is None or slot[0] is None:
|
|
258
|
+
return True
|
|
259
|
+
if BEACON_CACHE_TTL > 0 and (time.time() - slot[1]) > BEACON_CACHE_TTL:
|
|
260
|
+
return True
|
|
261
|
+
if relpath is not None:
|
|
262
|
+
m = _wt_mtime_any(relpath) if isinstance(relpath, (list, tuple, set)) else _wt_mtime(relpath)
|
|
263
|
+
if m is not None and slot[2] != m:
|
|
264
|
+
return True
|
|
265
|
+
return False
|
|
266
|
+
|
|
267
|
+
|
|
268
|
+
# 版本号单一来源:直接读同目录 package.json,避免 MCP 内部版本与发布包版本漂移。
|
|
269
|
+
def _pkg_version() -> str:
|
|
270
|
+
try:
|
|
271
|
+
p = os.path.join(os.path.dirname(os.path.abspath(__file__)), "package.json")
|
|
272
|
+
with open(p, "r", encoding="utf-8") as f:
|
|
273
|
+
return json.load(f).get("version", "1.3.3")
|
|
274
|
+
except Exception:
|
|
275
|
+
return "1.3.3"
|
|
276
|
+
|
|
277
|
+
|
|
278
|
+
# HTTP 基址列表(含兜底镜像)。默认部署 Cloudflare Pages 不可达时,自动回退到
|
|
279
|
+
# GitHub 兜底镜像(与 Android App 同策略:pages.dev 主源 + jsDelivr / raw 镜像),
|
|
280
|
+
# 保证「免 clone 检索」在单一 CDN 抖动/被墙时仍可工作。可用 BEACON_MIRRORS 追加自定义镜像。
|
|
281
|
+
def _http_bases() -> List[str]:
|
|
282
|
+
bases = [BEACON_SOURCE]
|
|
283
|
+
if BEACON_SOURCE == DEFAULT_SOURCE:
|
|
284
|
+
bases.append("https://cdn.jsdelivr.net/gh/eiry16/beacon-mfg@main")
|
|
285
|
+
bases.append("https://raw.githubusercontent.com/eiry16/beacon-mfg/main")
|
|
286
|
+
m = os.environ.get("BEACON_MIRRORS")
|
|
287
|
+
if m:
|
|
288
|
+
for x in m.split(","):
|
|
289
|
+
x = x.strip().rstrip("/")
|
|
290
|
+
if x:
|
|
291
|
+
bases.append(x)
|
|
292
|
+
return bases
|
|
293
|
+
|
|
294
|
+
|
|
211
295
|
# --------------------------------------------------------------------------- #
|
|
212
296
|
# 使用量埋点(设计见 docs/MCP_USAGE_AUDIT.md §3.4 / §4 / §14)
|
|
213
297
|
# --------------------------------------------------------------------------- #
|
|
@@ -362,48 +446,59 @@ def _git_show(relpath: str) -> Optional[str]:
|
|
|
362
446
|
return None
|
|
363
447
|
|
|
364
448
|
|
|
449
|
+
# 单基址 HTTP 超时(秒)。多镜像回退时每个基址各自计时,避免单一 CDN 卡死拖垮整体。
|
|
450
|
+
_HTTP_TIMEOUT = 15
|
|
451
|
+
|
|
452
|
+
|
|
365
453
|
def _http_get(relpath: str) -> Tuple[Optional[str], Optional[str]]:
|
|
366
|
-
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
454
|
+
"""统一的 HTTP 读取:依次尝试 _http_bases() 中的基址(主源 + 兜底镜像),
|
|
455
|
+
任一成功即返回;全部失败则用本地缓存兜底(保证离线可用)。"""
|
|
456
|
+
cached_body = None
|
|
457
|
+
cached_etag = None
|
|
458
|
+
for base in _http_bases():
|
|
459
|
+
url = base + "/" + relpath
|
|
460
|
+
cap = _cache_path(url)
|
|
461
|
+
etag = None
|
|
462
|
+
if os.path.exists(cap):
|
|
463
|
+
try:
|
|
464
|
+
with open(cap, "r", encoding="utf-8") as f:
|
|
465
|
+
blob = json.load(f)
|
|
466
|
+
etag = blob.get("etag")
|
|
467
|
+
cached_body = blob.get("body")
|
|
468
|
+
cached_etag = etag
|
|
469
|
+
except Exception:
|
|
470
|
+
etag = None
|
|
471
|
+
headers = {"User-Agent": UA, "Accept": "*/*"}
|
|
472
|
+
# 埋点头:服务端据此填 tool / tokens / hits 列。收不到也没关系 ——
|
|
473
|
+
# worker 那边**照样计数**,只是这三列为空(文档 §3.4)。
|
|
370
474
|
try:
|
|
371
|
-
|
|
372
|
-
blob = json.load(f)
|
|
373
|
-
etag = blob.get("etag")
|
|
475
|
+
headers.update(_beacon_headers())
|
|
374
476
|
except Exception:
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
try:
|
|
380
|
-
headers.update(_beacon_headers())
|
|
381
|
-
except Exception:
|
|
382
|
-
pass
|
|
383
|
-
req = urllib.request.Request(url, headers=headers)
|
|
384
|
-
if etag:
|
|
385
|
-
req.add_header("If-None-Match", etag)
|
|
386
|
-
try:
|
|
387
|
-
resp = urllib.request.urlopen(req, timeout=30)
|
|
388
|
-
body = resp.read().decode("utf-8")
|
|
389
|
-
new_etag = resp.headers.get("ETag")
|
|
477
|
+
pass
|
|
478
|
+
req = urllib.request.Request(url, headers=headers)
|
|
479
|
+
if etag:
|
|
480
|
+
req.add_header("If-None-Match", etag)
|
|
390
481
|
try:
|
|
391
|
-
|
|
392
|
-
|
|
482
|
+
resp = urllib.request.urlopen(req, timeout=_HTTP_TIMEOUT)
|
|
483
|
+
body = resp.read().decode("utf-8")
|
|
484
|
+
new_etag = resp.headers.get("ETag")
|
|
485
|
+
try:
|
|
486
|
+
with open(cap, "w", encoding="utf-8") as f:
|
|
487
|
+
json.dump({"etag": new_etag, "body": body}, f)
|
|
488
|
+
except Exception:
|
|
489
|
+
pass
|
|
490
|
+
return body, new_etag
|
|
491
|
+
except urllib.error.HTTPError as e:
|
|
492
|
+
if e.code == 304 and cached_body is not None:
|
|
493
|
+
return cached_body, etag
|
|
494
|
+
continue
|
|
393
495
|
except Exception:
|
|
394
|
-
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
return None, None
|
|
401
|
-
except Exception:
|
|
402
|
-
# 网络不通时若本地有缓存也返回,保证离线可用
|
|
403
|
-
if os.path.exists(cap):
|
|
404
|
-
with open(cap, "r", encoding="utf-8") as f:
|
|
405
|
-
return json.load(f).get("body"), etag
|
|
406
|
-
return None, None
|
|
496
|
+
# 当前基址不可达:尝试下一个兜底镜像;全部失败后在末尾用本地缓存兜底
|
|
497
|
+
continue
|
|
498
|
+
# 全部基址失败:若有任何本地缓存也返回,保证离线可用
|
|
499
|
+
if cached_body is not None:
|
|
500
|
+
return cached_body, cached_etag
|
|
501
|
+
return None, None
|
|
407
502
|
|
|
408
503
|
|
|
409
504
|
def _worktree_read(relpath: str) -> Optional[str]:
|
|
@@ -467,12 +562,12 @@ def fetch_text(relpath: str) -> Optional[str]:
|
|
|
467
562
|
# --------------------------------------------------------------------------- #
|
|
468
563
|
def load_manifest() -> Dict[str, Any]:
|
|
469
564
|
global _manifest_cache
|
|
470
|
-
if _manifest_cache
|
|
565
|
+
if _cache_stale(_manifest_cache, "data/manifest.json"):
|
|
471
566
|
txt = fetch_text("data/manifest.json")
|
|
472
567
|
if not txt:
|
|
473
568
|
raise RuntimeError("无法取得 manifest.json(检查 BEACON_SOURCE / BEACON_REPO / 网络)")
|
|
474
|
-
_manifest_cache = json.loads(txt)
|
|
475
|
-
return _manifest_cache
|
|
569
|
+
_manifest_cache = (json.loads(txt), time.time(), _wt_mtime("data/manifest.json"))
|
|
570
|
+
return _manifest_cache[0]
|
|
476
571
|
|
|
477
572
|
|
|
478
573
|
def _shards_of_type(t: str) -> List[Dict[str, Any]]:
|
|
@@ -513,13 +608,14 @@ def _load_alias_file(rel: str) -> Dict[str, Any]:
|
|
|
513
608
|
def load_gb_alias() -> Dict[str, Any]:
|
|
514
609
|
"""词 → 条目。两层表合并:gb-alias.json(数据推导)+ gb-alias-curated.json(人工策展)。"""
|
|
515
610
|
global _alias_cache
|
|
516
|
-
|
|
611
|
+
_alias_relpaths = ("data/gb-alias.json", "data/gb-alias-curated.json")
|
|
612
|
+
if _cache_stale(_alias_cache, _alias_relpaths):
|
|
517
613
|
merged: Dict[str, Any] = {}
|
|
518
|
-
for rel in
|
|
614
|
+
for rel in _alias_relpaths:
|
|
519
615
|
for k, v in _load_alias_file(rel).items():
|
|
520
616
|
merged.setdefault(str(k).lower(), v)
|
|
521
|
-
_alias_cache = merged
|
|
522
|
-
return _alias_cache
|
|
617
|
+
_alias_cache = (merged, time.time(), _wt_mtime_any(_alias_relpaths))
|
|
618
|
+
return _alias_cache[0]
|
|
523
619
|
|
|
524
620
|
|
|
525
621
|
def _alias_entries(v: Any) -> List[Dict[str, Any]]:
|
|
@@ -546,7 +642,7 @@ def _alias_entries(v: Any) -> List[Dict[str, Any]]:
|
|
|
546
642
|
def _gb_names() -> Dict[str, str]:
|
|
547
643
|
"""国标码 → 中文名。取不到(离线/文件缺失)就返回空表,不影响主链路。"""
|
|
548
644
|
global _gb_name_cache
|
|
549
|
-
if _gb_name_cache
|
|
645
|
+
if _cache_stale(_gb_name_cache, "data/gb4754-full.json"):
|
|
550
646
|
names: Dict[str, str] = {}
|
|
551
647
|
txt = fetch_text("data/gb4754-full.json")
|
|
552
648
|
if txt:
|
|
@@ -561,8 +657,8 @@ def _gb_names() -> Dict[str, str]:
|
|
|
561
657
|
if key in ("classes", "groups", "divisions") and isinstance(body, dict):
|
|
562
658
|
for k, v in body.items():
|
|
563
659
|
names[str(k)] = str(v.get("name", v)) if isinstance(v, dict) else str(v)
|
|
564
|
-
_gb_name_cache = names
|
|
565
|
-
return _gb_name_cache
|
|
660
|
+
_gb_name_cache = (names, time.time(), _wt_mtime("data/gb4754-full.json"))
|
|
661
|
+
return _gb_name_cache[0]
|
|
566
662
|
|
|
567
663
|
|
|
568
664
|
def alias_codes(q: str, max_codes: int = 6) -> List[Dict[str, Any]]:
|
|
@@ -624,13 +720,14 @@ def _load_cap_index() -> Optional[Dict[str, Any]]:
|
|
|
624
720
|
"""skills/registry/index/cap.json —— 发布产物(与 city.json 同构):
|
|
625
721
|
{terms:{code:词面串}, shards:{code:{shard_name:count}}}。"""
|
|
626
722
|
global _cap_index_cache
|
|
627
|
-
if _cap_index_cache
|
|
723
|
+
if _cache_stale(_cap_index_cache, "skills/registry/index/cap.json"):
|
|
628
724
|
txt = _index_text("skills/registry/index/cap.json")
|
|
629
725
|
try:
|
|
630
|
-
_cap_index_cache = json.loads(txt) if txt else {}
|
|
726
|
+
_cap_index_cache = (json.loads(txt) if txt else {},
|
|
727
|
+
time.time(), _wt_mtime("skills/registry/index/cap.json"))
|
|
631
728
|
except Exception:
|
|
632
|
-
_cap_index_cache = {}
|
|
633
|
-
return _cap_index_cache
|
|
729
|
+
_cap_index_cache = ({}, time.time(), _wt_mtime("skills/registry/index/cap.json"))
|
|
730
|
+
return _cap_index_cache[0]
|
|
634
731
|
|
|
635
732
|
|
|
636
733
|
def _cap_term_index() -> Dict[str, str]:
|
|
@@ -829,6 +926,11 @@ def _build_fp_index() -> List[Dict[str, Any]]:
|
|
|
829
926
|
cp = _index_cache_path()
|
|
830
927
|
wt = os.environ.get("BEACON_WORKTREE", "1") != "0"
|
|
831
928
|
key = (cp or "mem") + ("|wt" if wt else "")
|
|
929
|
+
if wt:
|
|
930
|
+
# 工作树模式:manifest 由 postfetch 每次重建(mtime 随之变),把它并进键 →
|
|
931
|
+
# fp 分片被重建后 fp_index 自动翻新,无需提交 / 重启。
|
|
932
|
+
mm = _wt_mtime("data/manifest.json")
|
|
933
|
+
key += "|m%.0f" % (mm if mm is not None else 0)
|
|
832
934
|
if _fp_index is not None and _fp_index_key == key:
|
|
833
935
|
return _fp_index
|
|
834
936
|
if cp and not wt and os.path.exists(cp):
|
|
@@ -888,7 +990,7 @@ def _build_fp_index() -> List[Dict[str, Any]]:
|
|
|
888
990
|
return recs
|
|
889
991
|
|
|
890
992
|
|
|
891
|
-
|
|
993
|
+
_CAP_TERMS_SLOT: Optional[tuple] = None
|
|
892
994
|
|
|
893
995
|
|
|
894
996
|
def _cap_term_of(code: str) -> str:
|
|
@@ -901,7 +1003,8 @@ def _cap_term_of(code: str) -> str:
|
|
|
901
1003
|
取不到时退回键本身:`caps_index` 允许出现码表之外的裸值(材料「不锈钢」、
|
|
902
1004
|
品类「正餐」),它们本身就是可检索词,丢掉等于让这类查询瞎掉。
|
|
903
1005
|
"""
|
|
904
|
-
|
|
1006
|
+
global _CAP_TERMS_SLOT
|
|
1007
|
+
if _cache_stale(_CAP_TERMS_SLOT, "skills/registry/index/cap.json"):
|
|
905
1008
|
terms: Dict[str, Any] = {}
|
|
906
1009
|
try:
|
|
907
1010
|
txt = _index_text("skills/registry/index/cap.json")
|
|
@@ -909,8 +1012,8 @@ def _cap_term_of(code: str) -> str:
|
|
|
909
1012
|
terms = (json.loads(txt).get("terms") or {})
|
|
910
1013
|
except Exception:
|
|
911
1014
|
terms = {}
|
|
912
|
-
|
|
913
|
-
return str(
|
|
1015
|
+
_CAP_TERMS_SLOT = (terms, time.time(), _wt_mtime("skills/registry/index/cap.json"))
|
|
1016
|
+
return str(_CAP_TERMS_SLOT[0].get(code) or code or "")
|
|
914
1017
|
|
|
915
1018
|
|
|
916
1019
|
def _hay(rec: Dict[str, Any]) -> str:
|
|
@@ -954,8 +1057,8 @@ def _rec_summary(rec: Dict[str, Any]) -> Dict[str, Any]:
|
|
|
954
1057
|
INDEX_DIR = "skills/registry/index"
|
|
955
1058
|
INDEX_VERSION = 3 # 倒排 key 为分片路径;v2 用国标码会漏掉 gb=null 的记录
|
|
956
1059
|
_index_meta_cache: Optional[Dict[str, Any]] = None
|
|
957
|
-
_bucket_cache: Dict[str,
|
|
958
|
-
_shard_rec_cache: Dict[str,
|
|
1060
|
+
_bucket_cache: Dict[str, tuple] = {}
|
|
1061
|
+
_shard_rec_cache: Dict[str, tuple] = {}
|
|
959
1062
|
|
|
960
1063
|
_RE_CJK = re.compile("[\u4e00-\u9fff\u3400-\u4dbf]+")
|
|
961
1064
|
_RE_WORD = re.compile(r"[a-z0-9]+")
|
|
@@ -998,13 +1101,18 @@ def _grams(text: str, query_mode: bool = False) -> List[str]:
|
|
|
998
1101
|
def _index_text(relpath: str) -> Optional[str]:
|
|
999
1102
|
"""读索引文件。git 模式按 HEAD sha 落盘缓存(桶均 11KB),
|
|
1000
1103
|
缓存后跨进程重复查询几乎零成本。
|
|
1104
|
+
|
|
1105
|
+
BEACON_WORKTREE=1(默认):磁盘缓存额外按工作树文件 mtime 判新 —— postfetch
|
|
1106
|
+
重建索引后 mtime 变化,下次查询即重读工作树版本,无需提交 HEAD 即可让关键词 /
|
|
1107
|
+
城市检索看到新数据(修 DSH 发现的「不提交就永远看不到」坑)。非工作树模式保持
|
|
1108
|
+
旧的「HEAD 变化才刷新」行为。
|
|
1001
1109
|
"""
|
|
1002
1110
|
cp = None
|
|
1003
1111
|
if REPO:
|
|
1004
1112
|
h = _head_sha() or "nosha"
|
|
1005
1113
|
cp = os.path.join(CACHE_DIR, "idx-" +
|
|
1006
1114
|
hashlib.sha1((h + "|" + relpath).encode("utf-8")).hexdigest() + ".json")
|
|
1007
|
-
if os.path.exists(cp):
|
|
1115
|
+
if os.path.exists(cp) and _index_sig_ok(cp, relpath):
|
|
1008
1116
|
try:
|
|
1009
1117
|
with open(cp, "r", encoding="utf-8") as f:
|
|
1010
1118
|
return f.read()
|
|
@@ -1016,11 +1124,43 @@ def _index_text(relpath: str) -> Optional[str]:
|
|
|
1016
1124
|
os.makedirs(CACHE_DIR, exist_ok=True)
|
|
1017
1125
|
with open(cp, "w", encoding="utf-8") as f:
|
|
1018
1126
|
f.write(txt)
|
|
1127
|
+
_write_index_sig(cp, relpath)
|
|
1019
1128
|
except Exception:
|
|
1020
1129
|
pass
|
|
1021
1130
|
return txt
|
|
1022
1131
|
|
|
1023
1132
|
|
|
1133
|
+
def _index_sig_ok(cp: str, relpath: str) -> bool:
|
|
1134
|
+
"""工作树模式下,磁盘缓存里的 mtime 印记是否与当前工作树文件一致。
|
|
1135
|
+
|
|
1136
|
+
印记用 repr 精确往返(浮点不丢精度),避免 postfetch 在同一秒内重写索引文件时
|
|
1137
|
+
截断判成「未变」而误服陈旧缓存。
|
|
1138
|
+
"""
|
|
1139
|
+
sig = _wt_mtime(relpath)
|
|
1140
|
+
if sig is None:
|
|
1141
|
+
return True # 非工作树:不按 mtime 判新
|
|
1142
|
+
sf = cp + ".sig"
|
|
1143
|
+
try:
|
|
1144
|
+
with open(sf, "r", encoding="utf-8") as f:
|
|
1145
|
+
return float(f.read().strip()) == sig
|
|
1146
|
+
except Exception:
|
|
1147
|
+
return False # 缺印记 / 损坏 → 重建
|
|
1148
|
+
|
|
1149
|
+
|
|
1150
|
+
def _write_index_sig(cp: str, relpath: str) -> None:
|
|
1151
|
+
sig = _wt_mtime(relpath)
|
|
1152
|
+
sf = cp + ".sig"
|
|
1153
|
+
try:
|
|
1154
|
+
if sig is None:
|
|
1155
|
+
if os.path.exists(sf):
|
|
1156
|
+
os.remove(sf)
|
|
1157
|
+
return
|
|
1158
|
+
with open(sf, "w", encoding="utf-8") as f:
|
|
1159
|
+
f.write(repr(sig))
|
|
1160
|
+
except Exception:
|
|
1161
|
+
pass
|
|
1162
|
+
|
|
1163
|
+
|
|
1024
1164
|
def _git_read_many(paths: List[str]) -> Dict[str, str]:
|
|
1025
1165
|
"""git 模式:一次 `git archive` 批量取出多个文件。
|
|
1026
1166
|
|
|
@@ -1050,10 +1190,29 @@ def _git_read_many(paths: List[str]) -> Dict[str, str]:
|
|
|
1050
1190
|
|
|
1051
1191
|
|
|
1052
1192
|
def _read_many_text(paths: List[str]) -> Dict[str, str]:
|
|
1053
|
-
"""
|
|
1193
|
+
"""批量取文本:工作树优先 → git archive 兜底;HTTP 模式线程池并发。
|
|
1194
|
+
|
|
1195
|
+
BEACON_WORKTREE=1(默认):**必须工作树优先**。`_candidate_shards`(索引桶)与
|
|
1196
|
+
`_records_from_paths`(fp 分片)都走这里 —— 若沿用「git archive HEAD」就会永远
|
|
1197
|
+
读到**已提交快照**,postfetch 刚重建、尚未提交的索引 / 分片完全看不见(DSH 实测
|
|
1198
|
+
「关键词 / 城市检索不提交就搜不到」的真凶)。工作树缺的文件再走 archive 已提交
|
|
1199
|
+
快照兜底,最后 HTTP。非工作树模式(BEACON_WORKTREE=0)保持旧行为。
|
|
1200
|
+
"""
|
|
1054
1201
|
if not paths:
|
|
1055
1202
|
return {}
|
|
1056
1203
|
if REPO:
|
|
1204
|
+
if os.environ.get("BEACON_WORKTREE", "1") != "0":
|
|
1205
|
+
out: Dict[str, str] = {}
|
|
1206
|
+
missing: List[str] = []
|
|
1207
|
+
for p in paths:
|
|
1208
|
+
w = _worktree_read(p)
|
|
1209
|
+
if w is not None:
|
|
1210
|
+
out[p] = w
|
|
1211
|
+
else:
|
|
1212
|
+
missing.append(p)
|
|
1213
|
+
if missing: # 工作树缺失/半成品 → 已提交快照兜底
|
|
1214
|
+
out.update(_git_read_many(missing))
|
|
1215
|
+
return out
|
|
1057
1216
|
got = _git_read_many(paths)
|
|
1058
1217
|
if len(got) >= max(1, len(paths) // 2): # archive 正常覆盖
|
|
1059
1218
|
return got
|
|
@@ -1067,16 +1226,17 @@ def _read_many_text(paths: List[str]) -> Dict[str, str]:
|
|
|
1067
1226
|
|
|
1068
1227
|
def _index_meta() -> Optional[Dict[str, Any]]:
|
|
1069
1228
|
global _index_meta_cache
|
|
1070
|
-
if _index_meta_cache
|
|
1071
|
-
|
|
1072
|
-
|
|
1073
|
-
|
|
1074
|
-
|
|
1075
|
-
|
|
1076
|
-
|
|
1077
|
-
|
|
1078
|
-
|
|
1079
|
-
|
|
1229
|
+
if _cache_stale(_index_meta_cache, INDEX_DIR + "/meta.json"):
|
|
1230
|
+
txt = _index_text(INDEX_DIR + "/meta.json")
|
|
1231
|
+
if not txt:
|
|
1232
|
+
_index_meta_cache = (None, time.time(), _wt_mtime(INDEX_DIR + "/meta.json"))
|
|
1233
|
+
else:
|
|
1234
|
+
try:
|
|
1235
|
+
_index_meta_cache = (json.loads(txt), time.time(),
|
|
1236
|
+
_wt_mtime(INDEX_DIR + "/meta.json"))
|
|
1237
|
+
except Exception:
|
|
1238
|
+
_index_meta_cache = (None, time.time(), _wt_mtime(INDEX_DIR + "/meta.json"))
|
|
1239
|
+
return _index_meta_cache[0]
|
|
1080
1240
|
|
|
1081
1241
|
|
|
1082
1242
|
def _index_fresh() -> bool:
|
|
@@ -1117,17 +1277,16 @@ _city_names_cache: Optional[set] = None
|
|
|
1117
1277
|
def _city_names() -> set:
|
|
1118
1278
|
"""倒排索引里出现过的城市名集合:地区词不作为相关性证据(只作筛选偏好)。"""
|
|
1119
1279
|
global _city_names_cache
|
|
1120
|
-
if _city_names_cache
|
|
1121
|
-
|
|
1122
|
-
|
|
1123
|
-
|
|
1124
|
-
|
|
1125
|
-
|
|
1126
|
-
|
|
1127
|
-
|
|
1128
|
-
|
|
1129
|
-
_city_names_cache
|
|
1130
|
-
return names
|
|
1280
|
+
if _cache_stale(_city_names_cache, INDEX_DIR + "/city.json"):
|
|
1281
|
+
names: set = set()
|
|
1282
|
+
txt = _index_text(INDEX_DIR + "/city.json")
|
|
1283
|
+
if txt:
|
|
1284
|
+
try:
|
|
1285
|
+
names = set(json.loads(txt).keys())
|
|
1286
|
+
except Exception:
|
|
1287
|
+
names = set()
|
|
1288
|
+
_city_names_cache = (names, time.time(), _wt_mtime(INDEX_DIR + "/city.json"))
|
|
1289
|
+
return _city_names_cache[0]
|
|
1131
1290
|
|
|
1132
1291
|
|
|
1133
1292
|
def _is_cjk(s: str) -> bool:
|
|
@@ -1164,16 +1323,23 @@ def _collapse_prefixes(toks: set) -> set:
|
|
|
1164
1323
|
|
|
1165
1324
|
|
|
1166
1325
|
def _bucket_postings(gram: str, buckets: int) -> Dict[str, Any]:
|
|
1167
|
-
"""取倒排索引里某词的 postings {分片路径: 命中条数};缺失/异常返回空 dict。
|
|
1326
|
+
"""取倒排索引里某词的 postings {分片路径: 命中条数};缺失/异常返回空 dict。
|
|
1327
|
+
|
|
1328
|
+
桶缓存按工作树 mtime 判新(见 _wt_mtime):postfetch 重建索引后,对应桶文件
|
|
1329
|
+
mtime 变化即自动重读,无需重启。
|
|
1330
|
+
"""
|
|
1168
1331
|
rel = _bucket_rel(gram, buckets)
|
|
1169
|
-
|
|
1170
|
-
|
|
1332
|
+
m = _wt_mtime(rel)
|
|
1333
|
+
slot = _bucket_cache.get(rel)
|
|
1334
|
+
if slot is not None and (m is None or slot[0] == m):
|
|
1335
|
+
b = slot[1]
|
|
1336
|
+
else:
|
|
1171
1337
|
txt = _index_text(rel)
|
|
1172
1338
|
try:
|
|
1173
1339
|
b = json.loads(txt) if txt else {}
|
|
1174
1340
|
except Exception:
|
|
1175
1341
|
b = {}
|
|
1176
|
-
_bucket_cache[rel] = b
|
|
1342
|
+
_bucket_cache[rel] = (m, b)
|
|
1177
1343
|
post = b.get(gram) if isinstance(b, dict) else None
|
|
1178
1344
|
return post if isinstance(post, dict) else {}
|
|
1179
1345
|
|
|
@@ -1316,13 +1482,22 @@ def _candidate_shards(tokens: List[str], city: str) -> Optional[List[str]]:
|
|
|
1316
1482
|
|
|
1317
1483
|
|
|
1318
1484
|
def _records_from_paths(paths: List[str]) -> List[Dict[str, Any]]:
|
|
1319
|
-
"""批量读取若干 fp 分片的记录,并在进程内按路径缓存(重复查询近乎零成本)。
|
|
1485
|
+
"""批量读取若干 fp 分片的记录,并在进程内按路径缓存(重复查询近乎零成本)。
|
|
1486
|
+
|
|
1487
|
+
缓存按工作树 mtime 判新(见 _wt_mtime):postfetch 重建 fp 分片后,对应分片 mtime
|
|
1488
|
+
变化即重读,无需重启;非工作树模式保持旧行为(按路径存在性判新)。
|
|
1489
|
+
"""
|
|
1320
1490
|
out: List[Dict[str, Any]] = []
|
|
1321
1491
|
if not paths:
|
|
1322
1492
|
return out
|
|
1323
|
-
todo = [
|
|
1324
|
-
for p in
|
|
1325
|
-
|
|
1493
|
+
todo: List[str] = []
|
|
1494
|
+
for p in paths:
|
|
1495
|
+
m = _wt_mtime(p)
|
|
1496
|
+
slot = _shard_rec_cache.get(p)
|
|
1497
|
+
if slot is not None and (m is None or slot[0] == m):
|
|
1498
|
+
continue
|
|
1499
|
+
todo.append(p)
|
|
1500
|
+
_shard_rec_cache[p] = (m, [])
|
|
1326
1501
|
for _path, txt in _read_many_text(todo).items():
|
|
1327
1502
|
recs: List[Dict[str, Any]] = []
|
|
1328
1503
|
for line in txt.splitlines():
|
|
@@ -1333,9 +1508,11 @@ def _records_from_paths(paths: List[str]) -> List[Dict[str, Any]]:
|
|
|
1333
1508
|
recs.append(json.loads(line))
|
|
1334
1509
|
except Exception:
|
|
1335
1510
|
continue
|
|
1336
|
-
_shard_rec_cache[_path] = recs
|
|
1511
|
+
_shard_rec_cache[_path] = (_wt_mtime(_path), recs)
|
|
1337
1512
|
for p in paths:
|
|
1338
|
-
|
|
1513
|
+
slot = _shard_rec_cache.get(p)
|
|
1514
|
+
if slot:
|
|
1515
|
+
out.extend(slot[1])
|
|
1339
1516
|
return out
|
|
1340
1517
|
|
|
1341
1518
|
|
|
@@ -1935,7 +2112,7 @@ def main() -> None:
|
|
|
1935
2112
|
"result": {
|
|
1936
2113
|
"protocolVersion": "2024-11-05",
|
|
1937
2114
|
"capabilities": {"tools": {}},
|
|
1938
|
-
"serverInfo": {"name": "beacon-mfg-readonly", "version":
|
|
2115
|
+
"serverInfo": {"name": "beacon-mfg-readonly", "version": _pkg_version()},
|
|
1939
2116
|
},
|
|
1940
2117
|
})
|
|
1941
2118
|
elif method == "notifications/initialized":
|