weflow-cli 1.12.0 → 1.13.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +217 -1
- package/OPERATIONS.md +254 -19
- package/README.md +4 -0
- package/bin/weflow-cli.ts +260 -51
- package/dist/bin/weflow-cli.js +255 -32
- package/dist/bin/weflow-cli.js.map +1 -1
- package/dist/src/core/dbPathService.d.ts.map +1 -1
- package/dist/src/core/dbPathService.js +5 -0
- package/dist/src/core/dbPathService.js.map +1 -1
- package/dist/src/core/ntCore.d.ts +12 -1
- package/dist/src/core/ntCore.d.ts.map +1 -1
- package/dist/src/core/ntCore.js +23 -3
- package/dist/src/core/ntCore.js.map +1 -1
- package/dist/src/core/sqlcipherCore.d.ts +8 -0
- package/dist/src/core/sqlcipherCore.d.ts.map +1 -1
- package/dist/src/core/sqlcipherCore.js +46 -11
- package/dist/src/core/sqlcipherCore.js.map +1 -1
- package/dist/src/services/configService.d.ts +3 -0
- package/dist/src/services/configService.d.ts.map +1 -1
- package/dist/src/services/configService.js +6 -3
- package/dist/src/services/configService.js.map +1 -1
- package/dist/src/services/diagnosticReport.d.ts +80 -0
- package/dist/src/services/diagnosticReport.d.ts.map +1 -0
- package/dist/src/services/diagnosticReport.js +191 -0
- package/dist/src/services/diagnosticReport.js.map +1 -0
- package/dist/src/services/wechatClient.d.ts +9 -3
- package/dist/src/services/wechatClient.d.ts.map +1 -1
- package/dist/src/services/wechatClient.js +55 -22
- package/dist/src/services/wechatClient.js.map +1 -1
- package/docs/DECISIONS.md +347 -1
- package/docs/NEXT_ACTIONS.md +7 -1
- package/docs/POSITIONING.md +68 -0
- package/docs/PROJECT_STATE.md +29 -8
- package/package.json +88 -86
- package/scripts/_utils.py +120 -4
- package/scripts/biz_daily.py +118 -11
- package/scripts/daily_task.ps1 +132 -0
- package/scripts/export_chat_html.py +12 -1
- package/scripts/install_tasks.cmd +32 -0
- package/scripts/install_tasks.ps1 +91 -0
- package/scripts/jev_client.py +335 -3
- package/scripts/nt_decrypt.py +79 -14
- package/scripts/pipeline.py +21 -0
- package/scripts/repro_customer_init.cmd +86 -0
- package/scripts/semantic_search.py +286 -24
- package/src/core/dbPathService.ts +5 -0
- package/src/core/ntCore.ts +34 -3
- package/src/core/sqlcipherCore.ts +47 -11
- package/src/services/configService.ts +12 -3
- package/src/services/diagnosticReport.ts +237 -0
- package/src/services/wechatClient.ts +418 -380
package/docs/DECISIONS.md
CHANGED
|
@@ -1406,6 +1406,347 @@ which is what the gate is for.
|
|
|
1406
1406
|
semantics are not reliable here), as is any automatic labelling: there is no gold standard for "is this
|
|
1407
1407
|
draft right", so the feature records what it judged and says so rather than claiming calibration.
|
|
1408
1408
|
|
|
1409
|
+
## D-126: 排障输出必须**判定过**且**可整段复制**(`check --report`),脱敏是硬约束
|
|
1410
|
+
|
|
1411
|
+
**Status:** Active(起因:同一个根因在 2026-08~10 被报成了四种不同问题 —— issue #3「无法通过步骤4获取key」、
|
|
1412
|
+
#9「密钥解不开了」、#10「读不到近期记录」,以及 2026-10-18 那封「sessions 找不到会话」的邮件)
|
|
1413
|
+
|
|
1414
|
+
**决定。** `weflow-cli check --report` 输出一段给人(和 AI)读的文本,包含环境、微信客户端、配置三项事实,
|
|
1415
|
+
以及**结论**:每条结论是「卡在哪一步 + 下一步运行什么」。三条约束:
|
|
1416
|
+
|
|
1417
|
+
1. **判定,不是罗列。** 只把 `sqlcipher3: false` 打出来,用户仍然不知道这意味着什么;报告必须说
|
|
1418
|
+
「缺 sqlcipher3,运行 `pip install sqlcipher3`」,并且**不要把"缺 sqlcipher3"报成"Python 不可用"**
|
|
1419
|
+
(那会让人去重装 Python,问题不变 —— 有测试钉住这两条不能混)。
|
|
1420
|
+
2. **脱敏是硬约束。** 报告里出现的东西只有:布尔状态、版本号、路径(用户名换成 `<用户>`)、
|
|
1421
|
+
命令名、依赖名。**密钥与口令的值连读都不读**(只传 `messageKeyConfigured: boolean`),
|
|
1422
|
+
聊天内容从不进入;`wxid` 与密钥形态字符串由 `redactForReport` 兜底替换。原因是这份文本的
|
|
1423
|
+
目的地是公开 issue / 第三方 AI —— AGENTS.md 明令禁止把这些放进去。
|
|
1424
|
+
3. **纯函数生成。** `renderDiagnosticReport(input)` 不读文件、不碰配置,所以「结论对不对」「脱敏漏没漏」
|
|
1425
|
+
都能用单元测试钉;`--json`(给机器)与 `--report`(给人)共用**同一份探测结果**,避免两个分支
|
|
1426
|
+
出现「JSON 说可用、报告说不可用」。
|
|
1427
|
+
|
|
1428
|
+
**没做的(明写)**:不自动上传、不联网、不写入文件;不做通用 PII 扫描(过度打码会把
|
|
1429
|
+
"路径里有中文目录"这种真线索删掉)。报告不会替代日志 —— 具体命令的报错原文仍要用户附上,
|
|
1430
|
+
报告末尾也是这么写的。
|
|
1431
|
+
|
|
1432
|
+
## D-125: 运行时版本的第二观测:命令行没 `--user-lib-dir` 时看**实际映射的 `Weixin.dll`**
|
|
1433
|
+
|
|
1434
|
+
**Status:** Active(2026-10-18 本机实测:4.1.15.13 → 4.1.15.53 后主进程不再带该参数)
|
|
1435
|
+
|
|
1436
|
+
**Decision.** D-107 的判据保留(命令行 `--user-lib-dir` 是**首选**:那是客户端自己说的),
|
|
1437
|
+
但**它不再是唯一来源**。命令行取不到时,取 `Weixin.exe` 进程实际映射的 **`Weixin.dll` 所在目录**;
|
|
1438
|
+
两者都没有 ⇒ `version: null` + `note`,仍然**不**拿"磁盘上只有一个版本目录"去推断。
|
|
1439
|
+
`note` 必须写清版本来自哪个观测(`命令行里没有 --user-lib-dir,版本取自进程实际加载的 Weixin.dll`)。
|
|
1440
|
+
|
|
1441
|
+
**Reason.【实测】** 本机升到 **4.1.15.53** 后,`Weixin.exe` 主进程的命令行**只有一个可执行路径**
|
|
1442
|
+
(`"D:\WeiXin\Weixin.exe"`,没有任何开关),而 4.1.15.13 时至少有一个进程带 `--user-lib-dir`。
|
|
1443
|
+
结果是 D-107 的判据在**新版本上恒返回"版本未知"**——`check` 看不出用户装的是哪一版,
|
|
1444
|
+
而这正是它存在的理由(排障第一个要问的)。模块表给出的是同一件事的**另一种观测**:
|
|
1445
|
+
`Weixin.dll` 此刻确实映射在 `D:\WeiXin\4.1.15.53\Weixin.dll`。改后 `check --json` 报
|
|
1446
|
+
`version: 4.1.15.53`、`loadedDirs: [D:\WeiXin\4.1.15.53]`,并附一句来源说明。
|
|
1447
|
+
|
|
1448
|
+
**两条判据为什么要分开记来源。** 命令行与模块表量的是**同一个量**,不是两个独立证据:
|
|
1449
|
+
命令行说 A、模块表也说 A 时,把它们混在一起去重会得到"两个目录指向不同位置"的假象,
|
|
1450
|
+
版本号反而会丢。所以顺序是**先①后②**,且只在①为空时才看②;`note` 记录实际用的是哪个。
|
|
1451
|
+
|
|
1452
|
+
**边界(不猜的地方一个没少)。**
|
|
1453
|
+
- 模块表要**同用户、活进程**才读得到;读不到(权限/进程刚起)就是 `null`,不退回磁盘推断。
|
|
1454
|
+
- 模块表出现**多个**目录时照样不挑一个报版本(与命令行那条同一规则)。
|
|
1455
|
+
- 仍不读 `Weixin.exe` 里硬编码的版本字面量(D-107 的理由没变)。
|
|
1456
|
+
- 旁证一条,**不拿来推版本**:`--client_version` 的 token 从 `0xF2541F0D`(.13)变成
|
|
1457
|
+
`0xF2541F35`(.53)。这与 PROJECT_STATE 里 2026-10-11 解出的**低 16 位**规则一致
|
|
1458
|
+
(byte3 = 第四段 `0x35`=53;byte2 − 0x10 = 第三段 `0x1F−0x10`=15),是第三个吻合的样本;
|
|
1459
|
+
但高 16 位仍是 `F254`、三个样本都是 `4.1.x`,**主次版本分不开**,所以代码照旧不拿 token
|
|
1460
|
+
推版本号(模块表已经直接给出目录名,没有理由再走一条半解码的路)。
|
|
1461
|
+
|
|
1462
|
+
## D-124: `judgeProvider=chat` 必须尊重 `aiEngine` —— 本地引擎的用户不许被静默送出境外
|
|
1463
|
+
|
|
1464
|
+
**Status:** Active(2026-10-18;7 条单测覆盖分派,**未**对真实 Ollama/LM Studio 跑过端到端)
|
|
1465
|
+
|
|
1466
|
+
**Decision.** `chat` 通路先读 `aiEngine` 再决定往哪发:
|
|
1467
|
+
`local`/`ollama` → `_utils.detect_local_engine()` 找到的本地服务(`provider='chat-local'`);
|
|
1468
|
+
`claude` → **明确拒绝**(这条通路只支持 OpenAI 兼容端点,要用云端判断请改 `judgeProvider=openrouter`);
|
|
1469
|
+
`deepseek` 或未配置 → 维持原样(`aiBaseUrl` 中转站或直连 DeepSeek)。
|
|
1470
|
+
**本地服务检测不到时返回 `None`,绝不退回云端**,只打一行 WARN 说明这批判断会走 LLM 路径。
|
|
1471
|
+
|
|
1472
|
+
**Reason.** D-120 引入 `judgeProvider=chat` 时,那条分支无条件读 `deepseekApiKey`。
|
|
1473
|
+
它的语义是"用**配置里的**对话模型",而"配置里的"到底指谁只有 `aiEngine` 说得清。
|
|
1474
|
+
所以对 `aiEngine=local` 的用户,一开 `judgeProvider=chat` 就把文章正文发去了 DeepSeek ——
|
|
1475
|
+
**界面上什么都没变**,这不是多花点钱,是用户明确选了不出境之后又被送出境。
|
|
1476
|
+
这与仓库既有的边界("云端 AI 要保留显式权限边界")冲突,属于必须改的缺陷而不是偏好。
|
|
1477
|
+
|
|
1478
|
+
**Consequences / limits.**
|
|
1479
|
+
- 本地引擎的 `chat()` 只回字符串、没有 usage:`ChatJevClient` 在 `engine` 路径上把 `usage.model`
|
|
1480
|
+
记为引擎自己的 `model`,契约校验(`_check_choice`/`_check_score`)与云端逐字同一条。
|
|
1481
|
+
- `claude` 不自动换成 `openrouter`:换厂商要用户自己按 key 与地区限制决定(D-120 的实测结论是
|
|
1482
|
+
OpenRouter 的地区限制逐模型生效,不能替用户默认)。
|
|
1483
|
+
- 单测用假 engine 验证分派与契约;**真实本地服务的时延与 JSON 稳定性未测**,
|
|
1484
|
+
所以 OPERATIONS 里写的是"可用",不写"已验证质量"。
|
|
1485
|
+
|
|
1486
|
+
## D-123: 重排的池子**不降级**(干净 A/B 否掉了"用池 20 换 5 倍速度"),并记一次混淆教训
|
|
1487
|
+
|
|
1488
|
+
**Status:** Active(2026-10-18 实测:五组 × 40 题;每组同题同种子)
|
|
1489
|
+
|
|
1490
|
+
**Decision.** `RERANK_CANDIDATES` 保持 **100**。重排那句 LLM 调用是单次查询 10 秒里的 ~9 秒,
|
|
1491
|
+
但它是全流程**最大的质量杠杆**,把那 9 秒当作"可优化开销"是错的。
|
|
1492
|
+
|
|
1493
|
+
**Reason.【实测】** 五组结果(同 40 题、同种子):
|
|
1494
|
+
|
|
1495
|
+
| 组 | 池 | 重排机制 | hit@1 | recall@5 | recall@10 | MRR | nDCG |
|
|
1496
|
+
| --- | --- | --- | --- | --- | --- | --- | --- |
|
|
1497
|
+
| A | — | **不重排** | 0.550 | 0.650 | 0.750 | 0.594 | 0.630 |
|
|
1498
|
+
| D | 20 | llm | 0.675 | 0.800 | 0.825 | 0.733 | 0.756 |
|
|
1499
|
+
| **E** | **100** | llm | **0.825** | **0.925** | **0.950** | **0.863** | **0.884** |
|
|
1500
|
+
| B | 20 | llm(契约先失败) | 0.675 | 0.775 | 0.825 | 0.726 | 0.750 |
|
|
1501
|
+
| C | 100 | 契约(DeepSeek 满足) | 0.800 | 0.925 | 0.950 | 0.849 | 0.873 |
|
|
1502
|
+
|
|
1503
|
+
- **重排本身值很多**:A → E 是 hit@1 0.55 → 0.825、recall@5 0.65 → 0.925、nDCG 0.63 → 0.884。
|
|
1504
|
+
- **同一机制下池 100 全面胜出**(D → E:+0.15 / +0.125 / +0.128),所以**不能拿池子换速度**。
|
|
1505
|
+
- **B≈D、C≈E**:两种重排机制(noul 契约 / JSON 打分)在**同池**下几乎等价 —— 决定因素只有池子。
|
|
1506
|
+
|
|
1507
|
+
**这次差点犯的错(值得单记):** 第一轮 A/B 里 B=`llm`、C=`typesafe`,我差点把 B→C 的差距
|
|
1508
|
+
读成"池子从 20 到 100 的收益"。实际那个差距**同时混了两个变量**:C 的契约恰好被
|
|
1509
|
+
`judgeProvider=chat` 的 DeepSeek 满足(`provider` 字段写的是契约名,不是谁在服务),而 B 的契约
|
|
1510
|
+
概率和 0.94 被判失败、退了 llm。
|
|
1511
|
+
⇒ **对照实验里,先把机制钉住再比参数**;状态字段要能区分"意图"与"事实"(`transport` 因此而生,见 D-122)。
|
|
1512
|
+
另外我此前对用户说过"池 100 下重排没有可测增益"——那是把"契约段失败的 WARN"当成了"没重排",
|
|
1513
|
+
**同一类误读**;本次 A vs E 证明重排的增益很大。
|
|
1514
|
+
|
|
1515
|
+
**Consequences.**
|
|
1516
|
+
- 查询延迟(~10 s)以重排为主,**这是买质量的钱,不要动**。真要提速只能换更快的模型/更小的提示
|
|
1517
|
+
(清洗摘录已做过),而不是砍候选。
|
|
1518
|
+
- `--rerank-pool` 仍可手动调小(离线批处理、赶时间时),但**默认值不许改回 20**;
|
|
1519
|
+
`PoolTests` 钉着默认值,改它要带新数据。
|
|
1520
|
+
- 记录的口径更新:`OPERATIONS.md` 的"重排效果"一节换成上表(旧数字是池 20 时代,
|
|
1521
|
+
"hit@1 0.55 → 0.65"那组已经不适用)。
|
|
1522
|
+
|
|
1523
|
+
## D-122: 检索的失败必须**显式**:`vector.used` / `vector.fallback` 进返回,`--mmr` 不再能崩
|
|
1524
|
+
|
|
1525
|
+
**Status:** Active(2026-10-18 代码审阅时读出来的两处,当天修 + 8 条测试 + 变异检查;全量 1067 绿)
|
|
1526
|
+
|
|
1527
|
+
**Decision.** `search()` 里那个**裸 `except: pass`** 换成 `except Exception as error`,并把
|
|
1528
|
+
"这次到底走没走向量、没走是什么原因"写进返回的 `vector` 字段(`{used, fallback}`);
|
|
1529
|
+
`vectors` 在 `try` 之前初始化为 `None`,fallback 里的 MMR 判空;每线条数在 `search()` 里
|
|
1530
|
+
**顺手算一次**(`_counts_from_meta`),`main()` 不再重复解析 meta。
|
|
1531
|
+
|
|
1532
|
+
**Reason.** 三件事是一起读出来的,性质不同:
|
|
1533
|
+
|
|
1534
|
+
1. **静默降级是最贵的一类失效。** 向量路径任何异常(embedding 上游失败、numpy 不可用、meta 坏掉)
|
|
1535
|
+
都会被那个裸 `except` 变成"只搜关键词",而返回的 envelope 里照样写着 `hybrid: true` ——
|
|
1536
|
+
**两种质量完全不同的结果在输出里长得一模一样**。这还让所有"快慢/好坏"的对比都可能落在
|
|
1537
|
+
不同路径上(2026-10-18 我在性能普查里就差点基于它下结论)。
|
|
1538
|
+
2. **一个真 bug**:`vectors` **只在 `try` 里赋值**,而 fallback 里的 `mmr_select(results, vectors, …)`
|
|
1539
|
+
用它。异常发生在 `np.load` 之前时(`require_numpy()` 就炸),`--mmr` 会 **UnboundLocalError**。
|
|
1540
|
+
3. **一处 1.2% 的浪费 + 一个漂移风险**:`search()` 已经解析了 meta,`main()` 又调 `line_counts()`
|
|
1541
|
+
解析一遍(实测 69 + 56 ms)。两份解析同一份文件迟早会漂,而漂的表现是"条数对不上但谁都不报错"。
|
|
1542
|
+
|
|
1543
|
+
**Consequences.**
|
|
1544
|
+
- 裸 `except` 还会吞掉 `KeyboardInterrupt` —— 一个跑 30 分钟以上的日报里,Ctrl+C 可能不生效。
|
|
1545
|
+
这次只改了最要紧的这一处,**其余 78 处没动**(数量上够做一条静态守卫,见 D-123 之后再说)。
|
|
1546
|
+
- **`--mmr` 在向量失败时现在直接跳过 MMR**(它需要矩阵),而不是崩;`fallback` 里会写明原因。
|
|
1547
|
+
- 测试纪律(这次最值得记的一条):第一版回归测试让 `get_embeddings` 抛错——**那时 `vectors` 已经赋过值**,
|
|
1548
|
+
把 `vectors = None` 拿掉测试照样全绿。**变异检查当场证明了它是"绿着但没盯住东西"**,
|
|
1549
|
+
于是补了一条在 `require_numpy()` 处失败的用例,再变异就红了。凡是要钉"未绑定/未初始化"这类问题,
|
|
1550
|
+
失败点必须落在**赋值之前**。
|
|
1551
|
+
- `vector` 是**只增字段**,`hybrid` 的含义不变("融合开关是开的"),两者一起读才能判断这次的真实路径。
|
|
1552
|
+
- **`rerank.transport` 是同一类修正**:`provider` 只说明"走的是 Jev 那套契约",不说明**谁在服务**。
|
|
1553
|
+
2026-10-18 的 A/B 里就撞上了:`judgeProvider=chat` 时契约由 DeepSeek 满足,状态里却写着
|
|
1554
|
+
`provider=typesafe`,差一点被读成"原生通了"(同一批查询的另一半退了 `llm`)。
|
|
1555
|
+
|
|
1556
|
+
⇒ 凡是"状态里记了某件事"的字段,要问一句**它记的是意图还是事实**;记意图的字段在换通路之后
|
|
1557
|
+
会安静地撒谎(同 D-034 记 `last_model` 的动机:实际服务的模型要与请求的别名分开)。
|
|
1558
|
+
|
|
1559
|
+
## D-121: 第三条腿是"原样定位"(FTS5/trigram),不是又一个相似度打分
|
|
1560
|
+
|
|
1561
|
+
**Status:** Active(2026-10-17 本机实测:8 次探针定形状;真语料 15,995 条建索引 10.7 s / 96.9 MB;
|
|
1562
|
+
6 条真查询;离线测试 19 条)
|
|
1563
|
+
|
|
1564
|
+
**Decision.** 新增 `search --phrase "<文本>"` 与 `search-index --phrase`。用 SQLite **FTS5 + `trigram`**
|
|
1565
|
+
建一条**精确短语**索引(`output/.phrase_index/phrase.sqlite`),查询按**字面量**匹配,结果按
|
|
1566
|
+
(账号/会话 × 日期)分组给出——即"**哪个号在哪天原样写过这句话**"。索引**从 `meta.json` 派生**
|
|
1567
|
+
(语义索引的记录清单 + `load_record_text` 取读者看到的那份正文),不自己重收集语料。
|
|
1568
|
+
|
|
1569
|
+
**Reason.【实测】前两条腿都是"相似",答不了"原样"**:bigram+BM25 按词项计分、向量按语义近似。
|
|
1570
|
+
专有名词、代码名(`std::vector`)、政策原文、引文核对这类问句要的是**原样**,把"查不到原样"
|
|
1571
|
+
混进相似度排序里,用户看不出区别——那是比排错更坏的失效方式。四个实测结论决定了形状:
|
|
1572
|
+
|
|
1573
|
+
1. **默认分词 `unicode61` 对中文完全不可用**(整句一个 token:`沙箱逃逸` 命中 **0**)⇒ 必须 `trigram`。
|
|
1574
|
+
2. **trigram 是连续子串,不是词项 AND**:`光伏补贴政策` 只命中**原样连续**的那篇,而
|
|
1575
|
+
`补贴政策` 命中"的补贴政策""补贴政策解读"。这正是"原样定位"要的语义,**也正是它不能替关键词腿的原因**
|
|
1576
|
+
——那条腿要的是"分开出现也能排上分"(实测 `光伏补贴政策` 在 1 万篇语料上是 0 命中)。
|
|
1577
|
+
3. **trigram 有 3 字下限**:`光伏`(2 字)恒空 ⇒ 短查询必须**明说"查不了"**,绝不能回报"0 命中"
|
|
1578
|
+
(把"查不了"说成"没有"是这条腿最坏的失效方式)。
|
|
1579
|
+
4. **查询里的标点会被当语法**:`PM2.5` → `syntax error near "."`、`GPT-4` → `no such column: 4`。
|
|
1580
|
+
整串加引号当字面量(内部 `"` 翻倍)后,`PM2.5` / `C++` / `GPT-4` 全部正常,且 ASCII 大小写不敏感。
|
|
1581
|
+
|
|
1582
|
+
**"每篇文章一个表"被否掉(同一次实测)**:15,995 条 × FTS5 每表约 5 张影子表 ≈ **9.6 万张表**;
|
|
1583
|
+
更要紧的是 `bm25()` 的**语料级统计**(df / avgdl)在单文档表上退化 —— 每个表只有 1 行,
|
|
1584
|
+
排名失去意义;查询还得 N 次 MATCH 再自己合并名次,等于把倒排索引唯一的好处丢掉。
|
|
1585
|
+
正解是**一条表装整个语料**(`rowid` 挂我们的记录 id,另存 line/label/date/title 几个 UNINDEXED 列),
|
|
1586
|
+
这也是参考项目自己的做法(它的 architecture.md 写的就是一条 FTS 索引)。
|
|
1587
|
+
|
|
1588
|
+
**为什么从 `meta.json` 派生**:再写一套记录枚举,两个索引的语料迟早分叉,而"两个索引覆盖的不是同一批"
|
|
1589
|
+
这种错**不会报**,只会让结果看起来少。代价是它依赖语义索引存在(没有就报错并给命令)。
|
|
1590
|
+
|
|
1591
|
+
**真语料实测**(15,995 条):`沙箱逃逸` 2 条(跨两个号)、`PM2.5` **58 条**(含聊天线,
|
|
1592
|
+
"孙天乐 2026-10-13 16:53 …那个 PM2.5 的")、`腾讯混元` 7 条(跨 7 个号)、`分布式光伏` 1 条;
|
|
1593
|
+
`std::vector` 与 `光伏补贴政策` 都是 **0 条** —— 与"原样"的预期一致(语料里真没这么写过)。
|
|
1594
|
+
|
|
1595
|
+
**Consequences.**
|
|
1596
|
+
- **纯本地**:不联网、不花钱、**不需要 key、不需要 `--yes`**(确认门是为出境设的)。CLI 预览里
|
|
1597
|
+
写明了 `mayUseCloudEmbedding: false` —— 别让人以为"本地检索也要花 vector 的钱"。
|
|
1598
|
+
- 索引是**派生**产物(96.9 MB / 全量重建 10.7 s),**它自己不排期**:语义索引变了(新文章入库)
|
|
1599
|
+
它就得跟着重建。这与"索引新鲜度"是同一个未决问题,不是两个。
|
|
1600
|
+
- `--from` 与向量那条路**同一口径**(文件名子串):所以命中的 `label` 可能与筛选词不同
|
|
1601
|
+
(号名常出现在标题里)。要精确按号,同一条 WHERE 里已经包含 `label` 精确匹配。
|
|
1602
|
+
- **明说边界:查不到原样 ≠ 没写过**(同义改写、全角/半角、正文里的断行都会漏)。这条腿回答
|
|
1603
|
+
"原样",语义那两条腿回答"意思上"——三者的分工要写在使用处,而不是让用户自己猜。
|
|
1604
|
+
- **未测的已知限制**:全角/半角不互认(`PM2.5` 查不到 `PM2.5`);大小写折叠只对 ASCII。
|
|
1605
|
+
|
|
1606
|
+
## D-120: 决策模型的三条通路:原生 / 国内对话模型 / OpenRouter —— 默认原生,国内那条是给被地区封锁的机器的
|
|
1607
|
+
|
|
1608
|
+
**Status:** Active(2026-10-17 本机实测:key 有效性、8 次真实调用、契约用仓库自己的 `_check_choice`
|
|
1609
|
+
验过;离线测试 34 条;`judgeProvider=chat` 已在本机打开)
|
|
1610
|
+
|
|
1611
|
+
**Decision.** 新增配置 `judgeProvider`,取值三条:
|
|
1612
|
+
|
|
1613
|
+
| 取值 | 走哪 | 需要什么 | 本机可用性【实测】 |
|
|
1614
|
+
| --- | --- | --- | --- |
|
|
1615
|
+
| `native`(**默认**) | `api.typesafe.ai` 的 `systemone` | `typesafeApiKey` | ❌ **HTTP 451 地区封锁** |
|
|
1616
|
+
| **`chat`**(推荐给本机) | 配置里**已有的**对话模型:`aiBaseUrl` 中转站 / `aiModel` / `deepseekApiKey` | 无需新厂商、新 key | ✅ `deepseek-chat` **1.0 s/篇**、契约通过 |
|
|
1617
|
+
| `openrouter` | OpenRouter 的 chat/completions | `openrouterApiKey` + `openrouterJudgeModel` | ⚠️ 契约通过但**该账号 0 额度**,调用被 402 拒 |
|
|
1618
|
+
|
|
1619
|
+
外加 `openrouterApiKey`(加密落盘)与 `openrouterJudgeModel`(默认 `typesafe/jev-router`)。
|
|
1620
|
+
`create_client()` 按 `judgeProvider` 分派,两条 chat 路共用同一个 `ChatJevClient`(`base_url` 可换),
|
|
1621
|
+
它与 `JevClient` 是**同一个鸭子类型**,所以**一处开关替掉整层**:日报判断、检索重排、草稿回复、
|
|
1622
|
+
欠话台账、路由选词共用它。**默认 `native`,行为与引入之前完全一致。**
|
|
1623
|
+
|
|
1624
|
+
**Reason.【实测】**
|
|
1625
|
+
- **原生这条路在本机是死的**:`api.typesafe.ai` 返回 HTTP 451 "not available in your region"。
|
|
1626
|
+
症状不是报错而是**静默退化**:最近 40 个完整天 12,542 篇里 **0 篇**带 `relevanceScore`、
|
|
1627
|
+
`decisionModel` 全空;同层的重排/欠话/草稿跟着一起退化。
|
|
1628
|
+
- **"中国地区可用的类 Jev"就在配置里**:拿同一份提示词与同一套校验问三个国内模型,**契约全过**——
|
|
1629
|
+
`deepseek-chat`(1.0 s、1649/69 tokens)、`deepseek-reasoner`(7.2 s)、`qwen-plus`(2.3 s)。
|
|
1630
|
+
`qwen-turbo` 撞的是账号速率限制(429),不是地区。所以这条路**不需要新厂商、新 key**,
|
|
1631
|
+
一句话 `config set judgeProvider chat` 就打开;成本按 tokens 估是 **¥1/天量级**(500 篇,见下)。
|
|
1632
|
+
- **⚠️ 契约一致 ≠ 判断一致**:同一篇文章 `deepseek-chat` 给 relevance=1 / 收录概率 0.15,
|
|
1633
|
+
`qwen-plus` 给 **2 / 0.85**。下游阈值(0.5/1.5、0.50)是对着 Jev 拍的,所以换判断器要当成
|
|
1634
|
+
**换了一把尺子**——这也是 `relevanceScore`/`includeScore`/`usage.model` 都要落盘的用处。
|
|
1635
|
+
- **实际服务的模型可能与请求的别名不同【实测】**:请求 `deepseek-chat`,`usage.model` 回的是
|
|
1636
|
+
**`deepseek-flash`**。这正是 D-034 那条"落盘实际服务的模型"在起作用——不然"线上跑的不是测过的那个"
|
|
1637
|
+
永远不可见。
|
|
1638
|
+
- **地区限制是逐模型的 —— "能调一个"不等于"能调所有"**:OpenRouter 上
|
|
1639
|
+
`typesafe/jev-router` 与 `:free` 模型能通,而 `openai/gpt-4o-mini` 返回
|
|
1640
|
+
**403 "This model is not available in your region."**(失败在路由的 "Gate Endpoints with Geo Restrictions")。
|
|
1641
|
+
换模型必须在本机重测一次——这条规矩是被那次 403 换来的。
|
|
1642
|
+
- **契约改由我们自己保证,这是换通路真正的代价**:原生是服务端直接给结构化答案
|
|
1643
|
+
(`{probabilities, choice}` / `{score}` / `{noul}`),chat 模型要**被要求**才会给,所以提示词写死
|
|
1644
|
+
输出形状、客户端逐条验。四条概率校验复用原生那套 `_check_choice`,另加 `_check_score`
|
|
1645
|
+
与 `_check_noul`——**noul 是概率不是布尔**,这个坑仓库踩过(把浮点当布尔读,每问都读成"否")。
|
|
1646
|
+
- **OpenRouter 的成本【实测】与路由的不确定性**:一次调用路由到 `x-ai/grok-4.7`,花 **$0.014**,
|
|
1647
|
+
其中 `reasoning_tokens` **1535**(比正文还长);另一次路由到 `z-ai/glm-5.3-flash`。
|
|
1648
|
+
**路由到谁每次可能不同**,所以只能按最坏估:**一天 500 篇 ≈ $7/天**,
|
|
1649
|
+
而且 `max_tokens` **管不住**它(实测设 600、实际返回 1598)。
|
|
1650
|
+
- **那个 OpenRouter key 的账号没有额度【实测】**:`/credits` 报 `total_credits: 0`,请求被 402 拒
|
|
1651
|
+
("you can only afford 6666 tokens")。所以 **OpenRouter 那条现在打不通**,接好了但没打开。
|
|
1652
|
+
|
|
1653
|
+
**Consequences.**
|
|
1654
|
+
- 一个开关同时改五处消费者,所以默认必须是 `native`;打开前要知道成本量级
|
|
1655
|
+
(`chat` ≈ ¥1/天,`openrouter` 最坏 ≈ $7/天)。
|
|
1656
|
+
- 四种失败的处置完全不同,所以报错分开说:**402=额度**、**403+"region"=地区(换模型才有用,不是换 key)**、
|
|
1657
|
+
**401=鉴权**、**空正文=推理吃光预算(调大 `OPENROUTER_MAX_TOKENS`)**。
|
|
1658
|
+
混在一起会让人去换 key,而那条路上换 key 一点用都没有。
|
|
1659
|
+
- 契约漂移**当场抛** `JevError`(不放行、不给默认值):失效方式若是"形状变了但看着正常",
|
|
1660
|
+
下游读到的就是错的判断。
|
|
1661
|
+
- **不许做的推断**:不能说任两条通路的判断一致——原生在本机调不通**无法对拍**,而两条 chat 路之间
|
|
1662
|
+
实测就已经不一致(见上)。能声明的只有"**契约成立、分数落盘、谁判的可查**"。
|
|
1663
|
+
- **判断质量仍未定标**:换了尺子之后,"这个 relevance 值对不对"没有金标准可校。要做的是
|
|
1664
|
+
`quality_eval.py sample` 抽 50 篇人工标一次(欠着的旧账),而不是拿三家模型互相投票当答案。
|
|
1665
|
+
- 另一条教训(与代码无关但值得留):那封自称给 $25 额度的推广邮件里,产品(Drex / nace.ai)
|
|
1666
|
+
在 OpenRouter 的 **466 个模型里不存在**,账号里也**没有**积分。核实靠官方接口,不靠邮件。
|
|
1667
|
+
|
|
1668
|
+
## D-119: 日报的排期形状:跑"昨天"、允许补跑、抓之前并发预热正文缓存
|
|
1669
|
+
|
|
1670
|
+
**Status:** Active(2026-10-16 落地;本机任务 `WeFlow Daily` 每日 08:20 跑 `pipeline.py --yesterday
|
|
1671
|
+
--prefetch-bodies`,并用 `Start-ScheduledTask` 实跑过一次)
|
|
1672
|
+
|
|
1673
|
+
**Decision.** 四件事一起定,它们是同一个问题的四个面:
|
|
1674
|
+
1. 计划任务跑**昨天**(新旗标 `--yesterday`),不是"今天";
|
|
1675
|
+
2. 用 `Register-ScheduledTask` + `-StartWhenAvailable` 注册,**不用** `schtasks /create`;
|
|
1676
|
+
3. 抓取之前先**并发预热正文缓存**(新旗标 `--prefetch-bodies [n]`,**默认关**);
|
|
1677
|
+
4. 任务的动作指向 `scripts/pipeline.py`,**不指向** `weflow-cli` 这个 npm 包。
|
|
1678
|
+
|
|
1679
|
+
**Reason.**
|
|
1680
|
+
- **根因是"没人跑",所以这不是优化而是补缺。** 【实测】本机 2026-10-16 复核 372 个日期目录:
|
|
1681
|
+
**13 天有网页/图片**(`08-24`~`09-05` 的 12 天 + 本次刚产出的 `10-15`)+ **358 天只有 `.articles.json`**。
|
|
1682
|
+
即**完整日报这条线从 2026-09-05 起暗了 40 天**:`10-07`~`10-13` 那些"看着有"的天都是回填形态
|
|
1683
|
+
(README 自己写着"未下载图片、未调用模型",没有 `index.html`),而 `10-14`~`10-16` 连目录都不存在。
|
|
1684
|
+
计划任务里除了两个 watcher(且都已 Disabled)没有任何跑日报的任务——与 D-011 那条"不跑就不会发生"
|
|
1685
|
+
完全一致。**这里更正一处旧口径**:09-28 曾被当成"最后完整的一天",其实它也是回填形态;
|
|
1686
|
+
"有 `.articles.json` + README"正是 **D-060** 警告过的"文件存在 ≠ 这天完整"。
|
|
1687
|
+
- **为什么是"昨天"**:公众号发到 24:00 才停,所以**当晚跑的那一天永远不全**,而且以后没人回头补它;
|
|
1688
|
+
反过来"跑昨天"这个动作**与时刻无关** —— 00:00 之后任何时间跑,读到的都是同一天的完整数据。
|
|
1689
|
+
于是最常见的漏跑(机器在睡觉)只要任务允许补跑,醒来补上就**结果完全相同**。
|
|
1690
|
+
【实测】22:26 手动触发 `--yesterday`,它产出的是 `2026-10-15`。
|
|
1691
|
+
- **为什么不是 `schtasks /create`**:它无法表达"错过就尽量补跑"(那在任务 XML 里是
|
|
1692
|
+
`<StartWhenAvailable>`)。没有它,机器 08:20 在睡觉 = 这天**静默**没有日报。本机现成的
|
|
1693
|
+
反例就在眼前:`PilgrimIntelDaily` 是 `StartWhenAvailable=True` 且 `LastTaskResult=0`,
|
|
1694
|
+
而另两条任务的 `LastTaskResult=2`(**找不到文件**)—— 仓库那次搬迁之后没人看过它们的上次结果。
|
|
1695
|
+
- **为什么指向 `pipeline.py` 而不是 CLI**:npm 全局那个包是**发布版**、滞后于仓库;更要紧的是
|
|
1696
|
+
它一旦是构建产物,"改完代码没 `npm run build`"就变成"任务跑的还是旧逻辑",属于静默失败。
|
|
1697
|
+
python 层没有构建步骤,是同一份源码。
|
|
1698
|
+
- **并发预热的取舍**:抓取阶段串行且未命中缓存时 `sleep` 8~12 秒(节流是为了少打扰上游),
|
|
1699
|
+
450 篇约 75 分钟;而**命中缓存既不联网也不 sleep**。预热把这段换成"先按 8 并发抓一遍",
|
|
1700
|
+
对上游的**请求次数完全不变**,只是从"一个多小时里慢慢发"变成"更短时间里发完"(瞬时并发更高)。
|
|
1701
|
+
正因为它改变了"日报故意串行节流"这条既有边界(D-108 记过),所以**默认关、只能显式打开**。
|
|
1702
|
+
【实测 2026-10-16 首次真跑(523 篇的那天)】预热 **516 篇 10 分钟**(前 6 分钟约 75 篇/分钟,
|
|
1703
|
+
最后 8 篇慢的拖了 4 分钟),整天 `biz_daily` **15 分 49 秒**(22:26:21 → 22:42:10);同样篇数
|
|
1704
|
+
串行 + 节流要一个多小时。也就是说这一步的收益是**约 90 分钟 → 16 分钟**。
|
|
1705
|
+
- **为什么不用 `backfill_articles --prefetch-all` 来预热**:那个开关会**产出同一天的文件**
|
|
1706
|
+
(md/`.articles.json`/README),与 `daily` 抢同名,而两条路径的产物形态并不相同。写过、测过、
|
|
1707
|
+
**撤掉了**:预热这事该由"要用这份缓存的那一方"自己做(`biz_daily.prefetch_bodies`),
|
|
1708
|
+
否则就是给一个不冲突的需求加一个会冲突的工具。
|
|
1709
|
+
- **别把缓存读成检索收益**:让索引吃完整正文实测**没有增益**(D-118),这份缓存只服务展示路径
|
|
1710
|
+
(`search --with-text`)与日报自己的耗时。
|
|
1711
|
+
|
|
1712
|
+
**Consequences.**
|
|
1713
|
+
- 任务动作里**没有** `--yes`(确认门在 CLI 层,python 层没有)。也就是说**能改这个仓库的人**
|
|
1714
|
+
就能让日报在 08:20 跑起来并调云端模型:单用户机器上可接受,换机器要重新评估。
|
|
1715
|
+
- 与 D-011 不冲突:那条兜底是"不带日期时先补昨天",`--yesterday` 是显式指定,不走兜底。
|
|
1716
|
+
- `--date` 与 `--yesterday` 同时给会**当场报错退出**,而不是静默选一个(两个答案并存时"哪一天"
|
|
1717
|
+
没有唯一解)。
|
|
1718
|
+
- 排期这一节的运行方法写在 `OPERATIONS.md`「让它每天自己跑」,含怎么读 `LastTaskResult`
|
|
1719
|
+
(`267009` = `0x41301` = **正在运行**,不是错误)。
|
|
1720
|
+
|
|
1721
|
+
## D-118: 索引**不**吃完整正文 —— 我提过改,量完撤回了(自造题裁判不了这件事)
|
|
1722
|
+
|
|
1723
|
+
**Status:** Active(2026-10-16 实测;102 条自造题 + 一次 40 篇的离线重嵌)
|
|
1724
|
+
|
|
1725
|
+
**Decision.** 建索引继续吃**日报 md 那份 500 字摘录**(`collect_articles` 里 `body[:500]`);
|
|
1726
|
+
正文缓存只服务**展示路径**(`--with-text` / `--fetch-missing`)。`best_article_body` 这个"取更长那份"
|
|
1727
|
+
的函数**只给展示路径用**,索引不碰它 —— 并有一条**反向**测试(`IndexBodyTests`)钉住这件事。
|
|
1728
|
+
|
|
1729
|
+
**Reason 【实测】: 加厚正文对检索没有可测的增益。**
|
|
1730
|
+
- 背景:日报 md 里正文中位 **259 汉字**,而日报**自己抓到的**完整正文缓存里是 **2,063 汉字**;
|
|
1731
|
+
索引嵌的是前者。于是很自然地怀疑"素材薄 ⇒ 检索差",并打算重建一次索引、让单条预算从 500 提到 2,000。
|
|
1732
|
+
- 做法(不动索引、不花钱重建):挑出"当前文本薄、但本地已有更厚缓存"的文章,只用完整正文**重嵌一遍**
|
|
1733
|
+
(旧中位 229 字 → 新中位 2,020 字),在**同一批目标**上比新旧名次(102 条题)。
|
|
1734
|
+
- 结果:唯一**中立**的一组(标题 —— 两份文本里都有它)**持平**:recall@1/@5/@10 = 35/39/40 → 33/40/40。
|
|
1735
|
+
两组碎片题各自跟着**自己的偏置**走、方向相反:碎片取自薄文本时 16/21/29 → **7/16/18**(大跌),
|
|
1736
|
+
取自厚文本时 2/4/8 → **4/7/8**(微升)。逐题 17 好 / 31 差 / 54 平。
|
|
1737
|
+
- **方法上的结论比数字本身重要**:题是从**被索引的那份文本**里抠出来的,所以**自造题裁判不了"索引吃哪份文本"**
|
|
1738
|
+
—— 换了索引文本,题也跟着换。要裁判它需要一份与索引无关的手写问题(`--queries-file` 已经支持),
|
|
1739
|
+
而且问题要问到"正文里才有、标题和摘要里没有"的东西。这个电池**这次没做**。
|
|
1740
|
+
- 【未知】因此**不能**说"完整正文对检索有害";能说的只是"**没有可测的增益**",
|
|
1741
|
+
而它不足以为此付一次完整重建(≈¥2.8、20 分钟)外加索引内部不再一致(有缓存的厚、没缓存的薄)。
|
|
1742
|
+
|
|
1743
|
+
**Consequences.**
|
|
1744
|
+
- 批量预填正文缓存(`backfill_articles --prefetch-all`)**不再为索引而做**。为展示而预填是另一回事,
|
|
1745
|
+
但 9,163 篇 × 约 4 小时买不到索引收益,所以**不做全量预填**;展示走 `--fetch-missing` 按需抓一次 + 永久缓存。
|
|
1746
|
+
- 撤销 2026-10-16 我提出的"让索引用完整正文"这条建议(同 D-110 那次"砍综合新闻号",
|
|
1747
|
+
也是我先提、量完自己撤回)。D-109/D-111 里的召回数字都是在**薄正文**索引上量的,**不需要重测**。
|
|
1748
|
+
- `IndexBodyTests` 反向钉住这条:它红了 = 有人又改了索引吃哪份文本,那时必须带上新的证据(手写问题电池)。
|
|
1749
|
+
|
|
1409
1750
|
## D-117: MMR 做了,但**默认关** —— 它压掉的"重复"里有相当一部分是有用的
|
|
1410
1751
|
|
|
1411
1752
|
**Status:** Active(2026-10-16 实测;同批 40 题看排序,另 10 题 × 5 条走判官看准确率)
|
|
@@ -1722,7 +2063,12 @@ region`,于是"重排"只在 stderr 打一行 WARN(CLI 还不转发子进程
|
|
|
1722
2063
|
|
|
1723
2064
|
**决定。** `check` / `health_check` 新增一项**微信客户端版本**【实测】:版本取自**活进程命令行的
|
|
1724
2065
|
`--user-lib-dir`**(`check --json` 多一个 `wechatClient` 块:`running` / `version` / `loadedDirs` /
|
|
1725
|
-
`note` / `unusableVersionDirs
|
|
2066
|
+
`note` / `unusableVersionDirs`)。
|
|
2067
|
+
|
|
2068
|
+
> **2026-10-18 修订(D-125):** 4.1.15.53 起主进程命令行**不再带** `--user-lib-dir`。命令行仍是首选,
|
|
2069
|
+
> 取不到时改看进程实际映射的 `Weixin.dll` 所在目录;两者都没有才报未知。下面三条判据不变。
|
|
2070
|
+
|
|
2071
|
+
三条判据写死:
|
|
1726
2072
|
|
|
1727
2073
|
1. **不读 `Weixin.exe` 里的版本字面量。**【实测·报告】它确实硬编码着、而且在本机**恰好与版本目录
|
|
1728
2074
|
同名** —— 但它只是**兜底**(控制流显示只有主路径加载失败才会用到它;主路径读
|
package/docs/NEXT_ACTIONS.md
CHANGED
|
@@ -31,7 +31,13 @@
|
|
|
31
31
|
抓取失败与"正文本来就短"从此分开记(`.articles.json` 的 `fetchFailed`,那天判为没做完、重跑只重试失败的篇)——
|
|
32
32
|
它直接对应本节验收里"失败后重跑可继续补齐,不会将部分日期或部分文章误标为完整"那一条。
|
|
33
33
|
判据与代价写在 D-108;顺带量到一件事:**接力是部分的**——日报对每篇有 url 的都抓正文,回填只抓"正文过短"的,
|
|
34
|
-
2026-10-13 那天 485 篇里只有 145 篇进了缓存(详见 D-108
|
|
34
|
+
2026-10-13 那天 485 篇里只有 145 篇进了缓存(详见 D-108)。
|
|
35
|
+
|
|
36
|
+
> **2026-10-16 补完:显式预取开关做了,但不在回填侧。** `daily --prefetch-bodies [n]`(默认关)在抓取阶段
|
|
37
|
+
> **之前**并发预热这份缓存,于是抓取阶段全是缓存命中、故意的 8–12 秒/篇节流不再触发(D-119)。
|
|
38
|
+
> 写过一个 `backfill_articles --prefetch-all`,**撤掉了**——它会产出同一天的文件、与 `daily` 抢同名。
|
|
39
|
+
> 同时量到一件反直觉的事并据此否掉了自己的提议:**让索引吃完整正文没有增益**(D-118),
|
|
40
|
+
> 所以这份缓存只服务展示路径(`search --with-text`)与日报耗时,**不服务检索**。
|
|
35
41
|
|
|
36
42
|
### 目标
|
|
37
43
|
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
# 这个项目靠什么活着(定位与护城河)
|
|
2
|
+
|
|
3
|
+
> 写于 2026-10-19,起因是微信 4.1.15 把「问 AI 总结文件和公众号文章」做进了客户端。
|
|
4
|
+
> 这个问题值得认真回答,因为答案决定了后面所有功能该不该做:**如果价值只是"用 AI 把微信里
|
|
5
|
+
> 已有的内容处理一遍",那微信自己会做,而且做得更顺(右键就有)。**
|
|
6
|
+
|
|
7
|
+
## 一句话
|
|
8
|
+
|
|
9
|
+
**不是"更好用的微信",是"能被带走、能被追问、能穿越时间的个人历史"。**
|
|
10
|
+
|
|
11
|
+
判据只有一个:**能不能回答微信回答不了的问题。** 答不了,就只是给微信套了一层 AI,别人没有理由换。
|
|
12
|
+
|
|
13
|
+
## 微信最近做了什么(2026-09-15,4.1.15 Win/Mac)
|
|
14
|
+
|
|
15
|
+
官方只承认两条,实测多出四条([IT之家](https://www.ithome.com/1/003/287.htm)):
|
|
16
|
+
|
|
17
|
+
| 更新 | 影响 |
|
|
18
|
+
| --- | --- |
|
|
19
|
+
| **文件和公众号文章支持「问 AI」** | ⚠️ 直接撞在我们"AI 读内容"的一半上 |
|
|
20
|
+
| 双人语音/视频通话可邀请他人加入 | 通话功能,与数据无关 |
|
|
21
|
+
| 群通话可邀请不在群里的人 | 同上 |
|
|
22
|
+
| 侧边栏新增「发现」 | UI 收敛 |
|
|
23
|
+
| 群免打扰可设「仍通知」(@我/@所有人/群公告 + 4 个成员) | 消息管理 |
|
|
24
|
+
| 聊天旁三栏浏览区,可独立窗口/吸附 | 阅读体验 |
|
|
25
|
+
|
|
26
|
+
`.53` / `.56` 这类构建**没有公开更新日志**(官网日志仍停在 4.1.15 / 09-15,GitHub 归档只到 `.53`)。
|
|
27
|
+
从安装目录看,变动集中在 `RadiumWMPF.bin`(412 MB,小程序/浏览器内核)、`WeChatOcr.bin`、
|
|
28
|
+
`XFilesOfficeReader`、`WePlayer` 这些组件——热修通常动内核,不动产品形态。
|
|
29
|
+
|
|
30
|
+
**要盯的方向**:微信正在把 AI 往"单文件/单文章"上做。它越做,越说明"跨内容、跨时间"那层是空白。
|
|
31
|
+
|
|
32
|
+
## 五条护城河(按"腾讯为什么不会做"排序)
|
|
33
|
+
|
|
34
|
+
| 护城河 | 微信不做/做不到的原因 | 我们的证据(可复验) |
|
|
35
|
+
| --- | --- | --- |
|
|
36
|
+
| **数据出口** | 微信从不提供导出聊天/朋友圈/收藏;"迁移"只到手机。4.1.15 六条更新里**没有一条**开放数据 | `export` 输出 HTML/md/json/xlsx + 图片与文件本体;issue #11 已修 |
|
|
37
|
+
| **纵向时间** | 微信的搜索只服务"找那条消息",不做"这个人三个月的变化""这件事我答应过几次" | 欠话台账、关系温度(`bonds`)、承诺追踪(`todos`)、日报归档(372 天) |
|
|
38
|
+
| **跨库联合** | 微信是产品不是平台:不会读你的 Obsidian、代码仓库、订阅源 | 语义索引 ~16k 条(聊天 5.2k + 文章 10.7k),实测 recall@10 **0.95**;原样定位(FTS5);Vault 概念页 |
|
|
39
|
+
| **本地/隐私** | 微信的 AI 必走它自己的云;承诺"不出本机"等于砍掉它的商业模型 | 支持 Ollama/LM Studio;`judgeProvider=chat` 必须尊重 `aiEngine=local`(D-124) |
|
|
40
|
+
| **可溯源** | 微信的「问 AI」给你一段总结,**点不回原句** | 每条命中带 `id`/`sourceFile`/原文链接,能回到那两个字节 |
|
|
41
|
+
|
|
42
|
+
## 会被立刻吃掉的功能(别把力气花在这里)
|
|
43
|
+
|
|
44
|
+
- 单篇文章/单个文件摘要 —— 4.1.15 已做;
|
|
45
|
+
- 会话内关键词搜索 —— 微信自带(虽然不全);
|
|
46
|
+
- 通用"和 AI 聊微信记录" —— 门槛极低,人人能做;
|
|
47
|
+
- 任何"把微信 UI 做得更顺手"的尝试。
|
|
48
|
+
|
|
49
|
+
## 四个能验的问题(产品自测题)
|
|
50
|
+
|
|
51
|
+
只要这四类问题答得比微信好,这个项目就成立:
|
|
52
|
+
|
|
53
|
+
1. **"我三个月前答应过谁什么,兑现了吗"** —— 跨时间 + 承诺追踪。
|
|
54
|
+
2. **"我和谁的关系在变淡,还救得回来吗"** —— 纵向聚合;微信只给当下。
|
|
55
|
+
3. **"这个号今年发了几篇,观点怎么变的"** —— 跨文章统计;微信只给单篇。
|
|
56
|
+
4. **"那句话他原样写过吗,哪天"** —— 原样定位;微信搜不到三个月前的群聊。
|
|
57
|
+
|
|
58
|
+
## 因此,产品的三个"不做"
|
|
59
|
+
|
|
60
|
+
1. **不追微信的新功能**(右键问 AI、发现页、三栏阅读)——那是它的产品节奏,追不上也没意义。
|
|
61
|
+
2. **不做纯 UI 替代品** —— 用户没有理由因为"更好看的聊天窗口"而迁移。
|
|
62
|
+
3. **不把 AI 摘要当成卖点** —— 摘要人人能做;**能溯源、能跨时间、能带走**才是。
|
|
63
|
+
|
|
64
|
+
## 这个判断会过期的情形
|
|
65
|
+
|
|
66
|
+
如果哪天微信真的开放了"导出全部聊天记录"或"本地数据 API",第一条护城河会塌一半——
|
|
67
|
+
到那时要立刻重估:**是不是该转向"分析层"(纵向/跨库/证据链),而不是"读取层"**。
|
|
68
|
+
这件事目前没有任何迹象,但要盯住更新日志里出现 `export` / `开放平台` / `本地数据库` 这类字眼。
|
package/docs/PROJECT_STATE.md
CHANGED
|
@@ -6,6 +6,8 @@
|
|
|
6
6
|
|
|
7
7
|
WeFlow CLI is a local-first command-line tool and MCP server for user-authorized WeChat data workflows: local chat queries and exports, official-account daily reading, favorites, knowledge workflows, and an optional local assistant.
|
|
8
8
|
|
|
9
|
+
**Why it exists next to WeChat itself** is written down in [POSITIONING.md](POSITIONING.md): the value is not "AI over the content WeChat already shows" (4.1.15 added exactly that for files and articles) but the layer WeChat does not provide - data that can leave the client, cross-content/cross-time queries, local inference, and answers traceable to the original message.
|
|
10
|
+
|
|
9
11
|
## Current Baseline
|
|
10
12
|
|
|
11
13
|
- Source package version: take it from `package.json` -> `version` rather than from this file - a hardcoded copy drifts silently, and `package-lock.json` must always match it. The npm registry may remain on an earlier version until a maintainer publishes a release.
|
|
@@ -19,10 +21,19 @@ WeFlow CLI is a local-first command-line tool and MCP server for user-authorized
|
|
|
19
21
|
- npm package contents: what ships is `package.json` -> `files`, and it is an allowlist of **directories and globs** - a file added just outside a listed pattern is dropped with nothing going red. `test/pack-contents.test.ts` asserts the positive side against a real `npm pack --dry-run --json`: every tracked file under `scripts/` ships except the dev-only `.ts`/`.mjs` ones (the `.mjs` is a verification tool that needs a GPU browser), every local image the two READMEs reference ships, and no `dist/` file lacks a matching source (`tsc` never prunes, so a renamed source used to keep shipping - four such files were removed on 2026-10-11). The negative side (nothing from `output/` or `models/`, no keys) is the CI `release-consistency` job. Both halves mattered on 2026-10-11: the published `1.9.0` tarball was missing `scripts/data/wechat_emoji.json` (the emoji fallback map `scripts/wechat_emoji.py` reads) and `logo.png`, so npm installs rendered a batch of face codes as literal text and the package page showed a broken logo.
|
|
20
22
|
- Downstream data contract: local JSON export is the first compatibility boundary for `she-love-me` and future consumers; see `docs/DATA_CONTRACT.md`.
|
|
21
23
|
- Supported focus: Windows WeChat 4.x; Linux WeChat 4.x has an NT database path; macOS requires user-provided local access credentials for initialization.
|
|
22
|
-
The version actually verified on this machine is **4.1.15.
|
|
23
|
-
process's `--user-lib-dir
|
|
24
|
-
|
|
25
|
-
|
|
24
|
+
The version actually verified on this machine is **4.1.15.56** (2026-10-19; before it 4.1.15.53 on 2026-10-18, and
|
|
25
|
+
4.1.15.13 before that) - read from the live process's `--user-lib-dir` when it is there, and otherwise from the
|
|
26
|
+
`Weixin.dll` that process actually has mapped (4.1.15.53+ carries **no** switches at all - D-125), **not** from the
|
|
27
|
+
version literal inside `Weixin.exe` (that one is only a fallback for when the primary load path fails; leftover
|
|
28
|
+
version *directories* are flagged, see `docs/CONTACT_DB_SCHEMA.md`, which also records which ones are usable).
|
|
29
|
+
**Every upgrade kept the derived keys working**, verified each time by reading real data: 4.1.15.53 on 2026-10-18
|
|
30
|
+
(from 4.1.15.13) and **4.1.15.56 on 2026-10-19** - after that one, `sessions`, `messages`, `contacts`,
|
|
31
|
+
`daily-stats` (official-account DB), `fav` (favorites DB) and `sns stats` (4,164 cached posts) all returned data
|
|
32
|
+
with the keys derived under the older client. 4.1.15.56 was obtained through the client's own updater (it is not on
|
|
33
|
+
the public download page or the GitHub version archives, which still served 4.1.15.53 on 2026-10-19), so a user on
|
|
34
|
+
it is not on a different product line. The customer-reported `init` failure (`NT scan failed (exit
|
|
35
|
+
9009)`) is therefore **not** a client-version problem; the `9009` message itself was verified
|
|
36
|
+
end-to-end against a real process that exits `9009`.
|
|
26
37
|
|
|
27
38
|
## Verified Capabilities
|
|
28
39
|
|
|
@@ -30,9 +41,9 @@ WeFlow CLI is a local-first command-line tool and MCP server for user-authorized
|
|
|
30
41
|
| --- | --- | --- |
|
|
31
42
|
| Local chat data | Query sessions, contacts, messages, favorites, Moments cache, and exports. HTML chat exports decode NT compressed media records, locally cached WeChat 4.x V2 image containers, entity-escaped emoji XML, and remote emoticons; app cards preserve links and embed available公众号/Bilibili covers. Media matching prefers server-message resources, exact local-message-ID plus timestamp pairs, and a unique local-ID fallback; high-bit encoded image rows are resolved through the same conversation cache. | `sessions`, `contacts`, `messages`, `export`, `fav`, `sns` |
|
|
32
43
|
| Initialization | Verify and reuse existing local database access by default; refresh only when needed. Missing-key tests can run without changing saved configuration. Data-root discovery asks the **running client** first (its command line carries `--wechat-files-path`; the other two routes guess from the folder name) and reports which source won; a hint still has to pass the same structural check. | `init`, `init --refresh`, `init --test-missing-keys`, `config forget-keys`, `check` |
|
|
33
|
-
| Environment / diagnosis | `check` (and the scheduled `health_check`) reports the **WeChat build actually loaded**, read from the live process's `--user-lib-dir` rather than
|
|
44
|
+
| Environment / diagnosis | `check` (and the scheduled `health_check`) reports the **WeChat build actually loaded**, read from the live process's `--user-lib-dir` and - since 4.1.15.53 launches the main process with no switches (D-125) - falling back to the `Weixin.dll` that process actually has mapped, rather than to the version literal inside `Weixin.exe` (that one is only a fallback - D-107). It flags version directories that cannot work (no `Weixin.dll`). When the client is not running it says *unknown*, and a missing field counts as a failure so a stale build cannot pass. Verified on this machine: `4.1.15.53` loaded (2026-10-18, after updating from 4.1.15.13), `4.1.15.11\` (one file) flagged as a leftover. `--report` renders the same facts as a **paste-ready, redacted verdict** (environment, client, key presence, then "what is broken and the next command"), so a user can hand an AI or an issue something better than a description of the symptom (D-126). | `check`, `check --json`, `check --report`, `scripts/health_check.py` |
|
|
34
45
|
| Relationship temperature | Who is going quiet, who is heating up, and who is reachable **only** here. Reads the local message database directly: read-only, no model, no network (D-062). Three groups - `cooling` (substantial history then silence, ranked by messages x silent days), `heating` (last 7 days at least twice the long-run daily rate), `only-wechat` (no phone/email/meeting-up ever appears in the history). Only 1:1 chats, only text messages, and only accounts whose **id shape** is a person: anything with `@` (`@chatroom`/`@openim`/`@weclaw`), `filehelper` or `gh_` is out - ranking by volume alone produced no information, and the first irreplaceability rule was inverted (it is trivially true of bots, so it put a coffee-shop bot first). Accounts that look like a person but whose name reads as a shop are **flagged, not dropped**, and every exclusion is printed with its reason. Deliberately **not** in the assistant or MCP tool tables: its output is an inference about third parties. | `bonds` |
|
|
35
|
-
| Official-account daily | Filter configured sources, preserve source categories, backfill incomplete yesterday output before an unqualified today run, fetch articles, classify them, create summaries, generate a local HTML reader, and synchronize reader favorites into local files. Topic (6-way), relevance (3-level) and "does this belong in today's report" (yes/no) are decided by TypeSafe's Jev when `typesafeApiKey` is configured - concurrently, before the serial summary loop (12 articles in 3.3s at 6 workers against ~12s one at a time), and by the previous LLM-parse path otherwise; `--classifier llm` forces the latter. The report's admission rule is applied on both loader paths and can be bypassed with `--include-all`. | `daily`, `daily favorites`, `daily-stats`, `daily-server` |
|
|
46
|
+
| Official-account daily | Filter configured sources, preserve source categories, backfill incomplete yesterday output before an unqualified today run, fetch articles, classify them, create summaries, generate a local HTML reader, and synchronize reader favorites into local files. Topic (6-way), relevance (3-level) and "does this belong in today's report" (yes/no) are decided by TypeSafe's Jev when `typesafeApiKey` is configured - concurrently, before the serial summary loop (12 articles in 3.3s at 6 workers against ~12s one at a time), and by the previous LLM-parse path otherwise; `--classifier llm` forces the latter. The report's admission rule is applied on both loader paths and can be bypassed with `--include-all`. Two flags exist for the unattended case: **`--yesterday`** (a scheduled run wants *the previous day*, since accounts publish until midnight, and "yesterday" is time-insensitive - the same data whenever the machine wakes up) and **`--prefetch-bodies [n]`** (default off; warms the body cache concurrently *before* the serial, deliberately throttled fetch phase, which then runs entirely from cache - the upstream request count is unchanged, only the window it is spread over). | `daily`, `daily favorites`, `daily-stats`, `daily-server` |
|
|
36
47
|
| Knowledge workflows | Wiki compilation (`compile_wiki` aggregates `[[concept]]` links into Vault pages; **the article corpus produced none as of 2026-09-26** (the 1,633 article notes then in the Vault carried only their own topic link and `## 关联网络` path links, measured over the whole corpus), so an article-side concept-extraction step was missing; `article-notes` (added the same day) is that step, and it works: 30 article notes produced 110 concepts and 10 concept pages on this machine, which are now the first knowledge pages the assistant can search (`vault search`, `search_knowledge`). The corpus has grown since - 7,119 concept cards, and 22,374 concept pages across both lines as of 2026-09-28. The degeneracy reported in that trial was an artefact of a 120-card sample: with all 710 September articles carded, the topic field splits into 学术/新闻/政治 (62/33/25 of 108 pages), so the earlier "the classifier puts everything in one bucket" judgement is withdrawn. Two upstream noise sources showed up in that trial. The first - WeChat reader UI text inside article summaries - is **fixed**, and the backfill this line used to defer has since been run: measured over the corpus it was 1,484 of 1,633 notes (the corpus has grown to 34,092 since), the cleaner runs on both write paths, and `create_reading_notes --clean-summaries` took the existing notes to **0**; the Vault's `Sources/WeChat` copies were cleaned on 2026-09-28 as well (D-055), so the residue now survives only in `output/biz-daily`, which is the archive. The second is **measured but not fixed**: of 224 notes whose titles read as news, 40 are labelled `学术` - that classifier belongs to the daily pipeline and changing its criteria without labels would be a guess. Fed by both sources: `output/biz-daily` articles and, since 2026-09-26, `output/chat-notes` conversation cards produced by the confirmed `chat-notes` command - which, since 2026-10-01, carries voice messages as **text** rather than a bare `[语音]`: it reads the local faster-whisper transcript cache (content-addressed, `output/.voice-cache` - one definition, in `wechat_voice`) and labels each one `[语音·本机转写·未校对]`, because a machine transcript of someone's speech may be wrong and the model must not quote it as fact. Measured on the 30-day window: **54 of 125** voice messages carry text that used to be a placeholder; the ones still missing are counted and printed, and `--transcribe-voice` fills them locally (the cache is the progress, so an interrupted run resumes); **the graph is also kept free of duplicate nodes by `wiki compile --merge-duplicates`** (74 groups merged, 80 pages removed, and 98 links that pointed at the removed names still resolve through `aliases` - see D-053); **most of the article line's pages were never written by a model** - `wiki compile --pages-from-cards` assembles a page from material the cards already carry (the definition is the card's own `desc`, the related concepts are the other concepts from the same article, the source is that article) and marks it `summary_by: card`. That label is deliberate and is not the same as "no model was involved": the `desc` line was written by `article_notes`, so the quality is that line's quality, and a few read as meta-descriptions of where the term appeared rather than as definitions. 17,383 pages at ¥0; the model-based route was priced at ¥31 and rejected, because every one of those concepts is mentioned by exactly one article (D-054); **the pages accumulate only if `wiki compile --refresh-sources` is run** - `wiki compile` itself only builds pages for concepts that do not have one, so an existing page never gains an edge (measured: one run added 3,164 edges across 324 article-line pages; on 2026-10-01, after the whole
|
|
37
48
|
pre-2026-03-03 window had been carded, another run added **8,551** edges across **3,506** pages - verified against a
|
|
38
49
|
snapshot of all 48,426 pages: the file count changed, the frontmatter that changed is **0**, and the files that lost
|
|
@@ -129,7 +140,7 @@ few weeks. Each row is an index entry - the measurements and the reasoning live
|
|
|
129
140
|
| --- | --- | --- |
|
|
130
141
|
| The **semantic (vector) search path** (D-105) | **Partly closed on 2026-10-13**: the index now exists (built with a 7-day chat / 3-article window - 749 records, inside DashScope's free quota) and the vector path has been exercised for real, per line (`indexCounts` 383 + 366 = 749). Still open: the **default 90/30 window** was never built (it exceeds the free quota; measured cost ≈ ¥1.3-2.2), and the **rerank layer cannot run on this machine** - it calls Typesafe (`api.typesafe.ai`) and answers `HTTP 451: Typesafe is not available in your region`, so the default path here is the un-reranked one. | Two things, both cheap but not done here: (a) decide whether the degraded search should stay byte-identical to a successful un-reranked one - `rerank` returns only the list, so nothing in the JSON says it failed; (b) build the 90/30 window if the small one proves useful. |
|
|
131
142
|
| `update\update.data`'s **internal format** (D-107) | The report reverse-engineered the loader but not this file, and the file does not exist here. When it does exist the loaded build may not be the one the command line names, so `check` **states that** instead of guessing. | A machine where the file exists. |
|
|
132
|
-
| The **version token ↔ dotted version** mapping (D-107) | The **low 16 bits are decoded** (2026-10-11): byte 3 is the fourth component, byte 2 is the third `+ 0x10` -
|
|
143
|
+
| The **version token ↔ dotted version** mapping (D-107) | The **low 16 bits are decoded** (2026-10-11): byte 3 is the fourth component, byte 2 is the third `+ 0x10` - three pairs now agree (`0xF2541F0D`→4.1.15.13, `0xF2541F35`→**4.1.15.53**, 2026-10-18, plus the older pair). The high 16 bits are `F254` on all of them, and all three versions are `4.1.x`, so major/minor cannot be separated (n=3: "all fit", not "solved"). | More (build, token) pairs, or the encoder. `update\download` and `update\patch` are empty on this machine, and the client's `log/*.xlog` (53 files, 373 MB) is **not plain text**: grep for the token there returns 0 matches, which is a **false negative** rather than evidence, and single-byte-XOR zlib/zstd does not unlock it either. The falsifiable prediction to test: `4.1.15.11 → 0xF2541F0B`. |
|
|
133
144
|
| The MMKV stores under `%APPDATA%\Tencent\xwechat\radium\mmkv\` (D-101) | Their format holds (crc32 precondition passes; one is empty), but the only key we can construct does not open them, and the `config/` root key is random and was never captured. | **Deliberately not attempted**: the remaining route is injecting into the client process, which this repo does not do (D-101). |
|
|
134
145
|
| **Why** the data root uses hard links (CONTACT_DB_SCHEMA §6.2) | The effect is measured (395 groups, 407 MB double-counted if you sum sizes); the cause is the report's `hardlink.dat` inference (multi-account / multi-version reuse) and has not been established. The file itself was read on 2026-10-11: it is **32 hex characters** (a 16-byte value), one copy on this machine, older than both installed client versions - so it is a marker or seed, not the link inventory (16 bytes cannot describe 395 groups). | Finding what consumes it. One testable guess, not yet tried: use those 16 bytes as an AES-128 key for the `radium\mmkv\` stores, judged by the D-101 oracle (crc32 consistent **and** the whole record sequence walks). The stores are on this machine; the tooling is not. |
|
|
135
146
|
| Issue #7 (HTML emoji export) | Open pending reporter verification. | The reporter's confirmation. |
|
|
@@ -186,9 +197,15 @@ Documentation was synchronized with the current source baseline on 2026-09-19. C
|
|
|
186
197
|
- **The LLM is no longer asked to classify when the decision model already has.** The summary prompt carried `【主题】`/`【相关度】` plus the six-category criteria and three-level definitions whoever was judging, so on the Jev path those were generated and discarded: 446 characters of prompt per article (57% of the prompt skeleton, ~9% of total input since the body dominates) and 10-15 output tokens. `SUMMARY_ONLY_PROMPT` drops exactly those sections and a test asserts the **summary/tag/concept requirements are byte-identical** between the two prompts, which is what keeps the change from altering the output rather than just its cost. The switch is **per article**: `--classifier llm` and any article whose judgement failed still get the full prompt, because those need the LLM's own topic and relevance - so the fallback path stays byte-identical per D-031. Savings are estimated from prompt size and output fields; neither client exposes billed usage.
|
|
187
198
|
- **Judging no longer requires generating.** `daily --no-summary` keeps the decision model for topic/relevance/inclusion and skips every LLM call inside the daily step, so a machine with no DeepSeek key can still classify. Two things make it honest rather than convenient: the LLM-key checks had to be relaxed in **both** callers (`biz_daily` and `pipeline`), because they encode "Phase 2 needs an LLM key" - a claim that stops being true once nothing is generated; and the artifacts must not imply a summary that does not exist, so the md omits the `## AI 摘要` section entirely and `.articles.json` records `summary: ""` (an empty heading reads as a failed generation, the platform digest reads as our summary - both would be claims about something absent). Tags fall back to `[topic]`, the convention that already covers "no tags were extracted". The scope is deliberately the daily step only: the pipeline's later steps (action suggestions, wiki compile, AI report) still call an LLM and say so at runtime; a fully LLM-free run combines it with `--skip-classify --skip-wiki --skip-ai-report`. Verified with a deliberately invalid DeepSeek key: the run completes with no 401, so nothing reached the LLM.
|
|
188
199
|
- **The decision model's free period ends 2026-09-25**, after which input tokens are billed (output has been free during the promotion). Anything expensive to re-measure should be measured before then, or the cost of a re-run should be assumed. The client only ever sees token counts, not a billed amount, so cost stays an estimate from tokens times price - the same position the reference implementation (`browser-use` × TypeSafe) documents. Verified against the live API on 2026-09-23: `usage` is exactly `{input_tokens, output_tokens, model}`. There **is** a route that reports money - the same model is served by OpenRouter as `typesafe/jev-1.13` and that response carries `usage.cost` - so "what does a daily actually cost" is answerable, but only by adding a second vendor to the request path, which has not been done.
|
|
200
|
+
- **The second vendor now exists, and the region block is measured rather than assumed (D-120, 2026-10-17).** `judgeProvider` selects among **three** transports for the whole decision layer (`native` default / **`chat`** / `openrouter`) - the same client serves the daily's topic/relevance judgement, search reranking, reply debt, drafted replies and route-card term selection, so one switch moves all of them. `chat` is the one that matters here: it reuses the conversational model the summary path already uses (`aiBaseUrl` relay / `aiModel` / `deepseekApiKey`), so **no new vendor and no new key**, and it was measured on one real article with the same prompt and the same validators - **`deepseek-chat` passes the Jev contract in 1.0 s** (1,649 prompt + 69 completion tokens), `deepseek-reasoner` in 7.2 s, `qwen-plus` in 2.3 s, while `qwen-turbo` was refused by that account's own rate limit (429, not a region block). This is now the configured path on this machine. Two corrections to the paragraph above: the OpenRouter slug is **`typesafe/jev-router`**, not `typesafe/jev-1.13` (gone from the 466-model list), and **the region filter is per model, not per site** - `typesafe/jev-router` and `:free` models pass from here while `openai/gpt-4o-mini` returns `403 This model is not available in your region`. **A passing contract is not a passing judgement**: on that same article `deepseek-chat` returned relevance 1 / inclusion 0.15 and `qwen-plus` returned 2 / 0.85, and the report's thresholds (0.5 / 1.5 / 0.50) were set against Jev - so a judge swap is a ruler swap, which is exactly why the raw scores and `usage.model` are stored. That paid off immediately: a request for `deepseek-chat` was served by **`deepseek-flash`**. OpenRouter cost is measured too (**$0.014** for one article on a call routed to `x-ai/grok-4.7`, 1,535 of 1,598 completion tokens being reasoning; the router picks a different upstream per call, so budget the worst case, ~$7 for a 500-article day, and `max_tokens` does not cap it) - and **that path cannot run on the key tested here at all**, since `/credits` reports `total_credits: 0` and calls are refused with 402. The contract now lives on our side for both chat paths, so the prompt plus four validators are in `jev_client.py` and drift raises `JevError` instead of yielding a plausible-looking default; 34 offline tests pin the shape, the key-name mapping (`prompt_tokens`→`input_tokens`, which decides whether cost reporting silently reads 0) and the four distinct failures. What is **not** verified: whether these values are *right* - there is still no gold standard, and `quality_eval.py sample` (50 articles, hand-labelled) remains the open item.
|
|
201
|
+
|
|
189
202
|
- **The decision-model path has never completed a production daily run.** Measured on the archive: of 2203 articles, **4** carry `relevanceScore` (2026-09-05, from a `--date` re-run while the feature was being built) and **none** carries `includeScore`; the last complete day is 2026-09-05, and 2026-09-22's directory exists but is empty because that run stopped at 105/403 fetches (the failure 1.7.0's fetch cache was built for). So "classification now uses Jev" describes the code and a 60-article comparison, not a day that has happened. Every threshold quoted above (relevance `0.5`/`1.5`, inclusion `0.50`) is still the guess it was written as, and the switch is unexercised at day scale. `quality_eval.py sample` drives the same production call over a 150-article pool, which is the closest it has come; the daily itself is the thing that has not run.
|
|
190
203
|
- Relevance in the daily report is split into 高/中/低 at cut points that are **provisional** (0.5 and 1.5 on a `[0,2]` score). The raw score is stored as `relevanceScore` so recalibrating does not require re-running a day. On a 6-article trial with the shipped criteria, nothing reached 高.
|
|
191
204
|
- Search reranking is fail-soft by design: no key, a network failure or an overloaded service (`HTTP 529` was observed live) all return the first-stage order with a warning. Its one real hazard is candidate numbering - a model that answers question `c3` against candidate `c5` sinks the best hit with no error at all - so every candidate is labelled in the state and a separate single-choice question cross-checks the argmax, warning on disagreement. Verified on real data before shipping, but only on one planted query.
|
|
205
|
+
- **Literal-string location is now a separate leg from similarity, and it is deliberately local (D-121, 2026-10-17).** `search --phrase` finds a phrase **verbatim** via SQLite's bundled **FTS5 + `trigram`** and reports hits grouped by account × date ("which account wrote this, when") - the questions the other two legs cannot answer at all (a code name, a proper noun, a policy phrase, a quotation check), because BM25 scores terms and vectors score meaning. Measured on the shipped corpus (15,995 records, built in 10.7 s into 96.9 MB): `沙箱逃逸` 2 hits across two accounts, `PM2.5` **58** (including the chat line), `腾讯混元` 7, `分布式光伏` 1, and `std::vector`/`光伏补贴政策` **0** - the last two being the honest reading of "verbatim". Four facts were measured rather than assumed: the default `unicode61` tokenizer is **useless for Chinese** (`沙箱逃逸` matched 0), `trigram` matches contiguous substrings rather than a term AND (so it cannot replace the keyword leg), it has a **3-character floor** (short queries return an explicit `tooShort`, never "0 hits"), and punctuation in a query is parsed as **syntax** (`PM2.5` errored until the string was quoted as a literal). The index is **derived** from the semantic index's `meta.json` rather than re-enumerating the database - two enumerations drift, and "the two indexes do not cover the same records" is a failure that reports nothing. It runs with no key, no network and no confirmation gate, and **it is not scheduled**: the semantic index changing means rebuilding it, which is the same open freshness problem, not a second one.
|
|
206
|
+
- **Reranking is the largest quality lever in the search path, and the candidate pool is what carries it (D-123, measured 2026-10-18).** Same 40 queries, same seed: no rerank `hit@1 0.550 / recall@5 0.650 / nDCG 0.630`; pool 20 `0.675 / 0.800 / 0.756`; **pool 100 (the default) `0.825 / 0.925 / 0.884`**. So the ~9 s the rerank costs per query buys the largest measurable gain in the system, and cutting the pool to 20 for a 5x speed-up would cost ~0.15 on every metric - the "optimisation" is refused by the data rather than by taste. Two mechanisms (the noul contract vs the JSON-scores fallback) measured **equivalent at the same pool** (0.800 vs 0.825 hit@1, identical recall), which is what made a clean pool comparison possible after a first, confounded one. Two instrumentation defects surfaced in the process and are fixed: `search()`'s bare `except: pass` silently turned any vector-path failure into a keyword-only search while still reporting `hybrid: true` (now `vector: {used, fallback}`), and `RERANK_STATE.provider` named the *contract* rather than who served it - with `judgeProvider=chat` the Jev contract is satisfied by DeepSeek, so `provider: typesafe` read as "the native API worked" when it had not (now `transport` records the client identity). The lesson is written down: **pin the mechanism before comparing a parameter**.
|
|
207
|
+
- **The `chat` judge path now follows `aiEngine`, so choosing a local engine is no longer overridden (D-124, 2026-10-18).** D-120's `judgeProvider=chat` branch read `deepseekApiKey` unconditionally, so a user with `aiEngine=local` who turned the judge on sent article text to DeepSeek **while nothing in the UI changed** - a privacy boundary broken by a transport choice, not a cost question. `local`/`ollama` now routes through `detect_local_engine()` (`provider='chat-local'`, the same contract validators), an undetected local service returns `None` with a WARN rather than falling back to the cloud, and `claude` is refused explicitly instead of being swapped for another vendor. Seven tests cover the dispatch with a stub engine; **a live Ollama/LM Studio run has not been done**, so the claim is "dispatches correctly and satisfies the contract", not "the local judge is good".
|
|
208
|
+
- **`init` now distinguishes "finished" from "ran", and Windows `9009` carries its own explanation (2026-10-18).** A real report ended with `NT scan failed (exit 9009)`, "✓ 已沿用配置中的密钥", "初始化完成! (4.x 模式)" and "密钥: 已安全保存", then `sessions` was empty on every run. The chain: `getPythonCommand()` falls back to the **bare name `python`** when no candidate passes its probe (runnable *and* `import sqlcipher3`), which on Windows is frequently the Microsoft Store placeholder that exits `9009`; the scan therefore never ran, so "the existing key" had never been checked against this database, and the closing summary was unconditional. The exit code is now translated into the requirement it actually encodes, and the summary is gated on whether **this run** obtained a key (memory-scan match or passphrase derivation) - otherwise it prints a failed-init heading and the three steps that fix it. The underlying fallback in `getPythonCommand()` was deliberately left alone: other commands do work on a plain `python`, and turning the fallback into a hard failure would break them to improve one message.
|
|
192
209
|
- `build_index` had never indexed any article (`glob(' marriage*.md')` matched 0 of 188 files), so semantic search covered chat messages only. The glob is fixed; **the article corpus will enter the index on the next rebuild**, which spends embedding calls, so it is a deliberate action rather than a side effect.
|
|
193
210
|
- Every config key a script reads is declared in `CliConfig`, so `config set` can set it; `test/config_keys_declared_test.py` enforces this with a justification-required exception list. Two keys were found by that audit: `dashscopeApiKey` (read by two scripts, absent from the schema entirely) and `favPassphrase` (a first-class encrypted secret that the allowlist refused). Both are settable now, and `dashscopeApiKey` became encrypted.
|
|
194
211
|
- `bizKey` / `bizSalt` are read by three scripts and written by nothing, so that branch is unreachable. **This entry used to add that `biz_message_0.db` "is always opened via the key derived from `favPassphrase`", and that half was never checked against the three callers**: only `biz_daily.py` had the fallback - `chat_stats.py` and `mcp_bridge.py` errored out with "missing key" and pointed the user at `config set bizKey`, a key that is not in the CLI's writable allowlist, so following their own advice reached `INVALID_CONFIG_KEY`. `biz_daily.py` additionally computed the same key **twice in a row** (a `bizKey` pass immediately overwritten by a passphrase pass), which is what a fork in the logic leaves behind. **Fixed on 2026-10-01 rather than left recorded**: the derivation now exists once, in `_utils.biz_message_db()`, all three call it, and `test/biz_message_db_test.py` asserts that no other script reads `bizKey` - a pattern check, so a script added later is covered too. **No configuration surface was added**: the dead branch was deleted rather than given a knob to turn, which is exactly what the original note was protecting against. Verified by running the path that used to fail (`daily-stats` now returns per-account counts; a daily dry run opens the database).
|
|
@@ -279,7 +296,11 @@ inter-article throttle never fires on a cache hit (the sleep lives in the daily'
|
|
|
279
296
|
function). The pipelines themselves stay separate for the reasons below - different outputs (text vs
|
|
280
297
|
HTML+images), different fetch policy (concurrent, no throttle here vs serial and deliberately throttled there) -
|
|
281
298
|
and each side keeps its own `--no-fetch-cache` for a forced refetch (same name, same meaning: skip the read,
|
|
282
|
-
still write the result back).
|
|
299
|
+
still write the result back). **Who warms that cache and who reads it:** since 2026-10-16 `daily
|
|
300
|
+
--prefetch-bodies` warms it concurrently before the fetch phase (so the phase is cache hits and the throttle
|
|
301
|
+
never fires), and the backfill warms it as a side effect; **the semantic index does not read it at all** -
|
|
302
|
+
feeding full bodies to the embedding was measured that day and showed no gain (D-118), so the cache serves the
|
|
303
|
+
display path (`search --with-text`) and the daily's running time, not retrieval. Neither of the two
|
|
283
304
|
image-carrying outputs is needed for the knowledge base, and the difference is not marginal: the daily path stores
|
|
284
305
|
`output/biz-daily/<date>/` (article HTML plus images, ~1.1 MB/article) *and* an identical copy under
|
|
285
306
|
`wechat-vault/Sources/WeChat/<date>/` (`pipeline.py:133`), so the same 35,463 articles would cost ~65 GB the daily
|