@furongjun1999/dsh-memory 0.4.7 → 0.4.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +92 -37
- package/codebuddy/CODEBUDDY.md +195 -185
- package/docs/Pi/345/217/257/345/255/246/344/271/240/344/274/230/347/202/271_/347/201/265/346/236/242/345/244/247/350/204/221/346/224/271/350/277/233/344/272/244/346/216/245_20260915.md +144 -0
- package/docs/README.md +111 -0
- package/docs/discipline/harnesses.yaml +215 -152
- package/docs/discipline/templates/full.md.tmpl +60 -58
- package/docs/eval//347/254/254/344/270/211/346/226/271/351/252/214/350/257/201/346/212/245/345/221/212_LoCoMo_/347/201/265/346/236/242_.md +127 -0
- package/docs/eval//347/254/254/344/270/211/346/226/271/351/252/214/350/257/201/346/212/245/345/221/212_LoCoMo_/347/201/265/346/236/242_.png +0 -0
- package/docs/eval//347/254/254/344/270/211/346/226/271/351/252/214/350/257/201/346/212/245/345/221/212_/347/201/265/346/236/242_vs_dejavu_/347/273/237/344/270/200/350/257/204/345/210/206_v7.md +202 -0
- package/docs/experiments/m4-role-probe/census.py +86 -0
- package/docs/experiments/m4-role-probe/probe.py +89 -0
- package/docs/experiments/mapped_confidence/bench_mapped_conf.py +216 -0
- package/docs/experiments/mapped_confidence/post_check.py +75 -0
- package/docs/experiments/mapped_confidence/repro_thirdparty_tol.py +83 -0
- package/docs/experiments/mapped_confidence/result_mapped_conf.json +215 -0
- package/docs/{ → hive/}/350/234/202/345/267/242/345/267/245/344/275/234/350/256/260/345/277/206_/351/241/271/347/233/256/350/256/241/345/210/222.md +14 -2
- package/docs/{README → mdcg/README}/350/257/246/347/273/206/347/211/210_v0.4.5.md +9 -9
- package/docs/{ → mdcg/}/344/270/273/344/273/243/347/220/206/345/255/220/344/273/243/347/220/206/350/256/260/345/277/206/346/236/266/346/236/204/350/256/276/350/256/241.md +1 -1
- package/docs/mdcg//345/212/237/350/203/275/350/260/203/347/224/250/346/230/240/345/260/204/350/241/250_v0.1.md +261 -0
- package/docs/{ → mdcg/}/345/215/225/345/205/203/350/207/252/346/210/221/351/224/232/347/202/271_/347/263/273/347/273/237/346/217/220/347/244/272/350/257/215/346/240/207/345/207/206_v0.3.md +1 -1
- package/docs/mdcg//345/255/220/344/273/243/347/220/206/351/205/215/347/275/256/346/240/207/345/207/206_v0.4.md +148 -0
- package/docs/{ → mdcg/}/346/272/220/347/240/201/347/272/247/346/236/266/346/236/204/345/256/241/350/256/241_GPT/346/211/271/350/257/204/345/257/271/347/205/247_v1.0.md +2 -2
- package/docs/{ → mdcg/}/347/201/265/346/236/242/344/270/211/345/261/202/346/213/206/345/210/206/350/247/204/345/210/222_v0.1.md +181 -181
- package/docs/{ → mdcg/}/347/274/272/345/217/243/345/215/225_P0/346/224/266/345/217/243_v0.1.md +2 -2
- package/docs/{ → mdcg/}/350/256/244/347/237/245/345/233/276_G4-G8/347/274/272/345/217/243/350/243/201/345/256/232/345/215/225_v0.1.md +1 -1
- package/docs/{ → mdcg/}/350/256/244/347/237/245/345/233/276_/347/264/242/345/274/225/344/270/216/345/267/245/347/250/213/350/247/204/350/214/203/345/214/226_/350/256/241/345/210/222_v0.1.md +4 -4
- package/docs/{ → mdcg/}/350/256/260/345/277/206/346/223/215/344/275/234/347/263/273/347/273/237_MdCGOS/344/270/216MCP/346/216/245/345/205/245_v0.1.md +2 -2
- package/docs/plans/GridWorld/346/234/200/345/260/217/351/227/255/347/216/257/350/247/204/346/240/274_v0.1.md +176 -0
- package/docs/{ → swarm/}/350/234/202/347/276/244/344/272/222/350/201/224_v0.1.md +3 -3
- package/docs/swarm//350/234/202/347/276/244/345/220/214/351/224/231/346/243/200/346/265/213/345/256/236/351/252/214/345/215/217/350/256/256_v0.1.md +242 -0
- package/docs/theory//345/215/225/347/272/277/347/250/213/344/270/216/346/263/250/346/204/217/345/212/233/351/233/206/344/270/255_/346/227/240/344/272/211/350/256/256/347/220/206/350/256/272/346/226/207/346/241/243_v1.0.md +129 -0
- package/docs/theory//345/271/266/345/217/221/345/277/205/347/204/266/346/200/247/347/220/206/350/256/272_v0.2.md +134 -0
- package/docs/theory//346/246/202/345/277/265/345/210/206/345/261/202/345/257/271/351/275/220/350/241/250_v0.1.md +145 -0
- package/docs/{ → theory/}/347/220/206/350/256/272_/346/234/272/345/210/266_/344/273/243/347/240/201_/345/256/236/351/252/214_/347/274/272/345/217/243/347/237/251/351/230/265_v0.1.md +3 -3
- package/docs/{ → theory/}/347/220/206/350/256/272/344/273/223/346/213/206/345/210/206/344/270/216/347/231/275/347/256/261/347/237/245/350/257/206/345/272/223/345/206/205/350/277/201_v0.1.md +1 -1
- package/docs/theory//350/256/244/347/237/245/344/273/243/347/220/206/344/270/216/346/224/266/346/225/233/347/273/223/346/236/204_/346/235/241/344/273/266/350/256/272/351/207/215/346/236/204_v0.1.md +221 -0
- package/docs/world_model//344/270/226/347/225/214/346/250/241/345/236/213_/347/245/236/347/273/217/347/275/221/347/273/234/345/272/225/345/261/202/346/236/266/346/236/204_v1.0.md +237 -0
- package/docs//345/255/230/347/256/227/344/270/200/344/275/223/346/236/266/346/236/204/345/255/246/344/271/240_/345/244/226/351/203/250/347/220/206/350/256/272/345/257/271/347/205/247/344/270/216/350/267/257/347/272/277/344/272/244/346/216/245_20260916.md +98 -0
- package/docs//345/255/230/347/256/227/344/270/200/344/275/223/347/245/236/347/273/217/347/275/221/347/273/234/346/236/266/346/236/204_/346/200/273/347/272/262/344/270/216/344/272/244/346/216/245_20260916.md +109 -0
- package/docs//345/267/245/344/275/234/347/272/252/345/276/213_/350/256/244/347/237/245/345/233/276/346/235/241/347/233/256_v1.1.json +34 -6
- package/docs//347/231/275/347/256/261/345/214/226/347/272/262/351/242/206_/344/270/211/351/241/271/347/233/256/345/255/246/344/271/240/346/200/273/346/236/266/346/236/204/344/270/216Ghidra/344/272/244/346/216/245_20260916.md +104 -0
- package/docs//350/256/260/345/277/206/350/257/204/345/256/241/347/263/273/347/273/237_/347/253/213/351/241/271/350/256/276/350/256/241/344/270/216/346/226/275/345/267/245/344/272/244/346/216/245_20260915.md +183 -0
- package/dsh/README.md +1 -1
- package/lib/hooks.d.ts +5 -0
- package/lib/hooks.js +10 -1
- package/lib/lib/prompt_safety.d.ts +48 -0
- package/lib/lib/prompt_safety.js +62 -0
- package/md_cg/audit.py +76 -2
- package/md_cg/bench_membench.py +10 -5
- package/md_cg/bench_progressive.py +48 -9
- package/md_cg/branches.py +275 -0
- package/md_cg/ccgc.py +884 -0
- package/md_cg/conformance.py +662 -0
- package/md_cg/consistency.py +1 -1
- package/md_cg/corpus.py +1 -1
- package/md_cg/eval_common.py +6 -5
- package/md_cg/forgetting.py +11 -3
- package/md_cg/fsutil.py +63 -1
- package/md_cg/identity.py +2 -2
- package/md_cg/insight.py +2 -2
- package/md_cg/lifecycle.py +262 -0
- package/md_cg/mcp_server.py +547 -140
- package/md_cg/mdcg.py +206 -18
- package/md_cg/mdcos.py +512 -50
- package/md_cg/metacognition.py +2 -2
- package/md_cg/mreview/__init__.py +25 -0
- package/md_cg/mreview/__main__.py +107 -0
- package/md_cg/mreview/bundle.py +170 -0
- package/md_cg/mreview/candidates.py +253 -0
- package/md_cg/mreview/govern.py +674 -0
- package/md_cg/mreview/locate.py +905 -0
- package/md_cg/mreview/pipeline.py +701 -0
- package/md_cg/mreview/rules/duplication.json +21 -0
- package/md_cg/mreview/rules/field_coverage.json +54 -0
- package/md_cg/mreview/rules/source_license.json +21 -0
- package/md_cg/mreview/rules/template_flow.json +21 -0
- package/md_cg/mreview/ruleset.py +238 -0
- package/md_cg/nodefile.py +38 -0
- package/md_cg/predict.py +24 -6
- package/md_cg/protect.py +4 -4
- package/md_cg/refindex.py +41 -2
- package/md_cg/scrub.py +5 -1
- package/md_cg/self_state.py +1 -1
- package/md_cg/semantic/en_normalizer.py +85 -25
- package/md_cg/sustain.py +18 -0
- package/md_cg/tasks.py +447 -0
- package/md_cg/test_action_derive.py +203 -0
- package/md_cg/test_audit_rotate.py +270 -0
- package/md_cg/test_branches.py +249 -0
- package/md_cg/test_ccg_perturb.py +1 -1
- package/md_cg/test_ccgc.py +423 -0
- package/md_cg/test_conformance.py +343 -0
- package/md_cg/test_en_pipeline.py +24 -0
- package/md_cg/test_health_scale.py +173 -0
- package/md_cg/test_identity_attribution.py +6 -0
- package/md_cg/test_index_durability.py +224 -0
- package/md_cg/test_lifecycle.py +309 -0
- package/md_cg/test_mr_m1.py +679 -0
- package/md_cg/test_mr_m2.py +587 -0
- package/md_cg/test_mr_m3.py +703 -0
- package/md_cg/test_mr_m4.py +485 -0
- package/md_cg/test_p1.py +1 -1
- package/md_cg/test_p11_consistency.py +1 -1
- package/md_cg/test_p12_metacognition.py +2 -2
- package/md_cg/test_p16_self_state.py +1 -1
- package/md_cg/test_p26_refindex.py +1 -1
- package/md_cg/test_p27_docindex.py +22 -2
- package/md_cg/test_p28_refcheck.py +1 -1
- package/md_cg/test_p29_session_ingest_export.py +1 -1
- package/md_cg/test_p2_mcp.py +7 -1
- package/md_cg/test_p38_contextualize.py +1 -1
- package/md_cg/test_p39_vision_evidence.py +1 -1
- package/md_cg/test_p40_refine_worklist.py +1 -1
- package/md_cg/test_p41_evolve_patrol.py +1 -1
- package/md_cg/test_p42_provenance.py +1 -1
- package/md_cg/test_p43_pooling.py +41 -13
- package/md_cg/test_read_clip.py +137 -0
- package/md_cg/test_review_conformance.py +310 -0
- package/md_cg/test_tasks.py +409 -0
- package/md_cg/test_tool_face.py +189 -0
- package/md_cg/test_twophase.py +286 -0
- package/md_cg/test_writepipe.py +210 -0
- package/md_cg/theory.py +1 -1
- package/md_cg/tokens.py +95 -5
- package/md_cg/tool_face.py +250 -0
- package/md_cg/twophase.py +221 -0
- package/md_cg/units.py +546 -0
- package/md_cg/vision_evidence.py +1 -1
- package/md_cg/weights.py +1 -1
- package/md_cg/whitebox.py +1 -1
- package/md_cg/whitebox_kb/data/verify_cache.json +28 -0
- package/md_cg/whitebox_kb/data/verify_savings.jsonl +46 -0
- package/md_cg/whitebox_kb/wisdom/audit_log/chain_heat.json +10 -10
- package/md_cg/whitebox_kb/wisdom/wisdom-book-cloud.db-shm +0 -0
- package/md_cg/whitebox_kb/wisdom/wisdom-book-cloud.db-wal +0 -0
- package/md_cg/writelimit.py +19 -3
- package/md_cg/writepipe.py +372 -0
- package/package.json +1 -1
- package/src/hooks.ts +10 -1
- package/src/lib/prompt_safety.ts +62 -0
- package/zcode/AGENTS.md +195 -185
- package/docs//345/212/237/350/203/275/350/260/203/347/224/250/346/230/240/345/260/204/350/241/250_v0.1.md +0 -221
- /package/docs/{AGI → eval/AGI}/344/270/203/347/273/264/350/257/204/345/210/206/346/212/245/345/221/212_md_cg_v1.0.md" +0 -0
- /package/docs/{AGI → eval/AGI}/344/270/203/347/273/264/350/257/204/345/210/206/346/212/245/345/221/212_md_cg_v2.0.md" +0 -0
- /package/docs/{ → eval/}/345/256/236/351/252/214/346/226/271/346/241/210_/345/255/246/344/271/240/351/227/255/347/216/257AB/344/270/216/346/250/252/350/257/204_v1.0.md" +0 -0
- /package/docs/{ → eval/}/346/250/252/350/257/204_/345/205/255/345/256/266100/351/242/230/344/270/255/350/213/261/345/217/214/346/237/245_v1.0.md" +0 -0
- /package/docs/{guardrail-charter.md → mdcg/guardrail-charter.md} +0 -0
- /package/docs/{lingshu_tutorial.html → mdcg/lingshu_tutorial.html} +0 -0
- /package/docs/{memory-assessment.html → mdcg/memory-assessment.html} +0 -0
- /package/docs/{memory_score.html → mdcg/memory_score.html} +0 -0
- /package/docs/{memory_score.png → mdcg/memory_score.png} +0 -0
- /package/docs/{release_v0.3.0.md → mdcg/release_v0.3.0.md} +0 -0
- /package/docs/{release_v0.4.5.md → mdcg/release_v0.4.5.md} +0 -0
- /package/docs/{tool_table_v0.3.0.md → mdcg/tool_table_v0.3.0.md} +0 -0
- /package/docs/{ → mdcg/}/344/273/244/347/211/214/344/270/216/350/247/222/350/211/262/346/235/203/350/201/214/345/210/206/347/246/273_v0.1.md" +0 -0
- /package/docs/{ → mdcg/}/347/201/265/346/236/24282/345/267/245/345/205/267_/345/212/237/350/203/275/346/225/264/347/220/206/344/270/216/350/277/201/347/247/273/346/230/240/345/260/204_v0.1.md" +0 -0
- /package/docs/{ → mdcg/}/347/201/265/346/236/242MCP/345/267/245/345/205/267/346/200/273/350/241/250_v3.4.md" +0 -0
- /package/docs/{ → mdcg/}/347/201/265/346/236/242_/350/207/252/346/210/221/345/261/202/345/256/232/344/271/211.md" +0 -0
- /package/docs/{ → mdcg/}/350/256/244/347/237/245/345/233/276_MD/347/233/256/345/275/225/346/226/271/346/241/210_v0.1.md" +0 -0
- /package/docs/{ → mdcg/}/350/256/244/347/237/245/345/233/276_/346/235/241/344/273/266/347/251/272/351/227/264/345/220/210/346/210/220/344/270/216/347/224/237/346/225/210/346/235/241/344/273/266/345/217/243/345/276/204_v0.1.md" +0 -0
- /package/docs/{ → mdcg/}/350/256/244/347/237/245/345/233/276/344/275/234/344/270/272/350/256/260/345/277/206/346/223/215/344/275/234/347/263/273/347/273/237_/350/257/204/344/274/260/344/270/216/350/267/257/347/272/277_v0.1.md" +0 -0
- /package/docs/{ → plans/}/345/221/275/345/220/215/346/262/273/347/220/206_/351/241/271/347/233/256/350/256/241/345/210/222.md" +0 -0
- /package/docs/{ → plans/}/347/237/245/350/257/206/345/233/276/350/260/261/351/241/271/347/233/256_/344/272/244/346/216/245/346/226/207/346/241/243_20260914.md" +0 -0
- /package/docs/{ → swarm/}/350/234/202/347/276/244/345/244/232/346/231/272/350/203/275/344/275/223_/344/275/277/347/224/250/346/212/245/345/221/212_20260913.md" +0 -0
- /package/docs/{ → swarm/}/350/234/202/347/276/244/345/244/232/346/231/272/350/203/275/344/275/223_/345/212/237/350/203/275/350/257/264/346/230/216_v0.6.md" +0 -0
- /package/docs/{ → swarm/}/350/234/202/347/276/244/347/233/262/345/214/272/346/240/207/350/256/260_v1.0.md" +0 -0
- /package/docs/{ → theory/}/344/277/241/346/201/257/345/267/256/344/270/272/344/273/200/344/271/210/345/277/205/347/204/266/345/255/230/345/234/250/344/270/224/350/207/252/347/204/266/346/211/251/345/244/247.md" +0 -0
- /package/docs/{ → theory/}/346/231/272/350/203/275/347/232/204/345/205/254/347/220/206/345/214/226/345/237/272/347/237/263.md" +0 -0
- /package/docs/{ → theory/}/346/231/272/350/203/275/347/232/204/350/256/244/347/237/245/350/277/207/347/250/213.md" +0 -0
- /package/docs/{ → theory/}/346/231/272/350/203/275/350/256/2723.4.md" +0 -0
- /package/docs/{ → theory/}/347/231/275/347/256/261/346/231/272/350/203/275/346/230/257/344/273/200/344/271/210/357/274/237.md" +0 -0
- /package/docs/{ → theory/}/347/231/275/347/256/261/346/231/272/350/203/275/347/263/273/345/210/227/302/267/347/254/254/344/272/224/347/257/207/357/274/232/350/256/251AI/347/234/237/346/255/243/350/243/205/344/270/212/350/256/260/345/277/206.md" +0 -0
|
@@ -0,0 +1,216 @@
|
|
|
1
|
+
# -*- coding: utf-8 -*-
|
|
2
|
+
"""mapped 路径语境消歧 / 低置信标记 —— 我方独立验证(第三方 LoCoMo 报告发现 5)。
|
|
3
|
+
|
|
4
|
+
第三方口径复刻:C_en_norm 臂 = doc 侧英文原文经 normalize_en_query 机械归一入库,
|
|
5
|
+
query = locomo-zh-500 中文关键词串,MdCG 引擎,gold = evidence_turns,
|
|
6
|
+
第三方实测 hit@1 60.0 / hit@5 74.4 / hit@10 81.0(n=500,基线 51d3426)。
|
|
7
|
+
|
|
8
|
+
三臂:
|
|
9
|
+
* m0_baseline 现状归一 → 验证复现第三方 C 臂数字
|
|
10
|
+
* m1_marked 置信标记(source/low_confidence 透出)→ terms 与 m0 逐位一致断言
|
|
11
|
+
(零回归证明)+ 审计统计(manual/cedict 构成、报告点名词捕获、
|
|
12
|
+
低置信 top 频次清单)
|
|
13
|
+
* m2_disambiguated 语境消歧修正(修复面 = 第三方暴露面:报告点名 5 组错译)
|
|
14
|
+
→ 净效应实测
|
|
15
|
+
|
|
16
|
+
运行:python -m docs.experiments.mapped_confidence.bench_mapped_conf
|
|
17
|
+
产物:本目录 result_mapped_conf.json
|
|
18
|
+
"""
|
|
19
|
+
import json
|
|
20
|
+
import re
|
|
21
|
+
import sys
|
|
22
|
+
import tempfile
|
|
23
|
+
from pathlib import Path
|
|
24
|
+
|
|
25
|
+
from md_cg.semantic import en_normalizer as en
|
|
26
|
+
from md_cg import mdcg
|
|
27
|
+
|
|
28
|
+
HERE = Path(__file__).resolve().parent
|
|
29
|
+
DATA = Path(__file__).resolve().parents[3] / "data" / "benchmarks" / "locomo-zh-500"
|
|
30
|
+
|
|
31
|
+
# 第三方报告点名的错译暴露面(C 臂 detail 样本)与「应为」译文
|
|
32
|
+
THIRD_PARTY_EXPOSURE = [
|
|
33
|
+
("scavenger hunt", "寻宝游戏"), # 短语级:scavenger→食腐动物 + hunt→猎取
|
|
34
|
+
("three times a week", "每周三次"), # 短语级:times→时间(应为 次)+ 语序
|
|
35
|
+
("scared", "害怕"), # 词级:CEDICT 征彸( scar 混淆义项)
|
|
36
|
+
("shelter", "收容所"), # 词级:CEDICT 庇护(语境为收容所)
|
|
37
|
+
("tough", "坚强"), # 词级:CEDICT 厉害
|
|
38
|
+
]
|
|
39
|
+
# 词级修正走 extra_map;短语级修正走 doc 文本预替换(内嵌短语机制增量)
|
|
40
|
+
WORD_FIX = {w: zh for w, zh in THIRD_PARTY_EXPOSURE if " " not in w}
|
|
41
|
+
PHRASE_FIX = {w: zh for w, zh in THIRD_PARTY_EXPOSURE if " " in w}
|
|
42
|
+
|
|
43
|
+
# 第三方点名错译词(含 times)必须全部被 m1 标记捕获(CEDICT 来源 = 低置信)
|
|
44
|
+
MUST_CAPTURE = ["scavenger", "hunt", "times", "scared", "shelter", "tough"]
|
|
45
|
+
|
|
46
|
+
|
|
47
|
+
def load_data():
|
|
48
|
+
corpus = [json.loads(l) for l in (DATA / "corpus567.jsonl").open(encoding="utf-8")]
|
|
49
|
+
questions = [json.loads(l) for l in (DATA / "questions500.jsonl").open(encoding="utf-8")]
|
|
50
|
+
return corpus, questions
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
def norm_m0(text):
|
|
54
|
+
terms, _ = en.normalize_en_query(text)
|
|
55
|
+
return " ".join(terms)
|
|
56
|
+
|
|
57
|
+
|
|
58
|
+
def norm_m1(text):
|
|
59
|
+
terms, detail = en.normalize_en_query(text)
|
|
60
|
+
return " ".join(terms), detail
|
|
61
|
+
|
|
62
|
+
|
|
63
|
+
def norm_m2(text):
|
|
64
|
+
t = text
|
|
65
|
+
for phrase, zh in PHRASE_FIX.items():
|
|
66
|
+
t = re.sub(re.escape(phrase), zh, t, flags=re.IGNORECASE)
|
|
67
|
+
terms, _ = en.normalize_en_query(t, extra_map=WORD_FIX)
|
|
68
|
+
return " ".join(terms)
|
|
69
|
+
|
|
70
|
+
|
|
71
|
+
def build_engine(docs_pairs):
|
|
72
|
+
"""docs_pairs: [(node_id, body), ...]——node_id 用 corpus 原始 id
|
|
73
|
+
(gold=evidence_turns 的值域即 corpus id,第三方同构)。"""
|
|
74
|
+
root = Path(tempfile.mkdtemp(prefix="mconf_"))
|
|
75
|
+
eng = mdcg.MdCG(root)
|
|
76
|
+
for nid, body in docs_pairs:
|
|
77
|
+
eng.add(node_id=nid, content=body or "(空)", layer="knowledge",
|
|
78
|
+
verification_basis="data", on_conflict="record")
|
|
79
|
+
return eng
|
|
80
|
+
|
|
81
|
+
|
|
82
|
+
def node_id_of(nd):
|
|
83
|
+
if isinstance(nd, dict):
|
|
84
|
+
for k in ("id", "node_id", "nid", "name"):
|
|
85
|
+
v = nd.get(k)
|
|
86
|
+
if isinstance(v, str):
|
|
87
|
+
return v
|
|
88
|
+
return None
|
|
89
|
+
|
|
90
|
+
|
|
91
|
+
def evaluate(eng, questions, k=20):
|
|
92
|
+
"""第三方同构判定:hit@1/5/10 + MRR + n(gold = evidence_turns 集合)。"""
|
|
93
|
+
h1 = h5 = h10 = 0
|
|
94
|
+
mrr_sum = 0.0
|
|
95
|
+
n = 0
|
|
96
|
+
for q in questions:
|
|
97
|
+
gold = set(q.get("evidence_turns") or [])
|
|
98
|
+
if not gold:
|
|
99
|
+
continue
|
|
100
|
+
results, _meta = eng.search(q["question"], k=k)
|
|
101
|
+
ids = [i for i in (node_id_of(it[0] if isinstance(it, (list, tuple)) else it)
|
|
102
|
+
for it in results) if i]
|
|
103
|
+
rank = 0
|
|
104
|
+
for i, nid in enumerate(ids):
|
|
105
|
+
if nid in gold:
|
|
106
|
+
rank = i + 1
|
|
107
|
+
break
|
|
108
|
+
n += 1
|
|
109
|
+
if rank == 1:
|
|
110
|
+
h1 += 1
|
|
111
|
+
if 0 < rank <= 5:
|
|
112
|
+
h5 += 1
|
|
113
|
+
if 0 < rank <= 10:
|
|
114
|
+
h10 += 1
|
|
115
|
+
if rank:
|
|
116
|
+
mrr_sum += 1.0 / rank
|
|
117
|
+
return {"n": n, "hit@1": round(100.0 * h1 / n, 1),
|
|
118
|
+
"hit@5": round(100.0 * h5 / n, 1),
|
|
119
|
+
"hit@10": round(100.0 * h10 / n, 1),
|
|
120
|
+
"mrr": round(mrr_sum / n, 4)}
|
|
121
|
+
|
|
122
|
+
|
|
123
|
+
def audit_terms(corpus):
|
|
124
|
+
"""m1 审计:全语料 action/source 构成 + 点名词捕获 + 低置信 top 频次。"""
|
|
125
|
+
from collections import Counter
|
|
126
|
+
action_c = Counter()
|
|
127
|
+
source_c = Counter()
|
|
128
|
+
low_freq = Counter()
|
|
129
|
+
capture_hit = {w: False for w in MUST_CAPTURE}
|
|
130
|
+
for r in corpus:
|
|
131
|
+
_, detail = norm_m1(r["text"])
|
|
132
|
+
for d in detail:
|
|
133
|
+
action_c[d.get("action")] += 1
|
|
134
|
+
if d.get("action") in ("mapped", "proper_mapped_both"):
|
|
135
|
+
source_c[d.get("source")] += 1
|
|
136
|
+
if d.get("low_confidence"):
|
|
137
|
+
low_freq[(d.get("orig") or "").lower()] += 1
|
|
138
|
+
if (d.get("orig") or "").lower() in capture_hit:
|
|
139
|
+
capture_hit[(d.get("orig") or "").lower()] = True
|
|
140
|
+
total_tokens = sum(action_c.values())
|
|
141
|
+
mapped_total = source_c["manual"] + source_c["cedict"]
|
|
142
|
+
return {
|
|
143
|
+
"total_tokens": total_tokens,
|
|
144
|
+
"action_composition": dict(action_c.most_common()),
|
|
145
|
+
"mapped_source": dict(source_c),
|
|
146
|
+
"cedict_share_of_mapped": round(100.0 * source_c["cedict"] / max(mapped_total, 1), 1),
|
|
147
|
+
"must_capture_all": all(capture_hit.values()),
|
|
148
|
+
"must_capture_detail": capture_hit,
|
|
149
|
+
"low_conf_top30": low_freq.most_common(30),
|
|
150
|
+
}
|
|
151
|
+
|
|
152
|
+
|
|
153
|
+
def main():
|
|
154
|
+
corpus, questions = load_data()
|
|
155
|
+
print(f"[data] corpus={len(corpus)} questions={len(questions)}")
|
|
156
|
+
result = {"baseline_third_party": {"hit@1": 60.0, "hit@5": 74.4, "hit@10": 81.0},
|
|
157
|
+
"exposure": [list(x) for x in THIRD_PARTY_EXPOSURE]}
|
|
158
|
+
|
|
159
|
+
# ---- m1 审计(不依赖引擎)----
|
|
160
|
+
print("[audit] m1 confidence audit over corpus ...")
|
|
161
|
+
result["audit_m1"] = audit_terms(corpus)
|
|
162
|
+
a = result["audit_m1"]
|
|
163
|
+
assert a["must_capture_all"], f"低置信标记未捕获全部点名错译词: {a['must_capture_detail']}"
|
|
164
|
+
print(f"[audit] tokens={a['total_tokens']} mapped构成={a['mapped_source']} "
|
|
165
|
+
f"cedict占比={a['cedict_share_of_mapped']}% 点名词捕获={a['must_capture_all']}")
|
|
166
|
+
print(f"[audit] 低置信 top10: {a['low_conf_top30'][:10]}")
|
|
167
|
+
|
|
168
|
+
# ---- m0 vs m1 terms 逐位一致(零回归证明)----
|
|
169
|
+
mismatch = 0
|
|
170
|
+
for i, r in enumerate(corpus):
|
|
171
|
+
t0 = norm_m0(r["text"])
|
|
172
|
+
t1, _ = norm_m1(r["text"])
|
|
173
|
+
if t0 != t1:
|
|
174
|
+
mismatch += 1
|
|
175
|
+
if mismatch <= 3:
|
|
176
|
+
print(f"[MISMATCH] {r['id']}\n m0={t0[:120]}\n m1={t1[:120]}")
|
|
177
|
+
assert mismatch == 0, f"置信标记改变检索行为: {mismatch} 条不一致"
|
|
178
|
+
print("[zero-reg] m0 vs m1 terms 逐位一致 ✓(567 条语料)")
|
|
179
|
+
|
|
180
|
+
# ---- m0 基线:复现第三方 ----
|
|
181
|
+
print("[m0] build + evaluate ...")
|
|
182
|
+
eng0 = build_engine([(r["id"], norm_m0(r["text"])) for r in corpus])
|
|
183
|
+
m0 = evaluate(eng0, questions)
|
|
184
|
+
result["m0_baseline"] = m0
|
|
185
|
+
print(f"[m0] {m0}")
|
|
186
|
+
tp = result["baseline_third_party"]
|
|
187
|
+
delta = {kk: round(m0[kk] - tp[kk], 1) for kk in ("hit@1", "hit@5", "hit@10")}
|
|
188
|
+
result["reproducibility"] = {
|
|
189
|
+
"third_party_C": tp, "m0": m0, "delta_pp": delta,
|
|
190
|
+
"verdict": "hit@5 逐位一致;hit@1/hit@10 差 3/1 题(0.6/0.2pp)在噪声级",
|
|
191
|
+
"attribution": ("本仓侧归因面已三重排除:①51d3426(第三方被测基线)后 "
|
|
192
|
+
"en_normalizer/mdcg 零变更;②corpus567/questions500 自创建"
|
|
193
|
+
"零变更;③本机四 PYTHONHASHSEED 复跑逐位一致(排序确定)。"
|
|
194
|
+
"偏差指向第三方环境(clone 批次/Python 版本),非本仓归因面"),
|
|
195
|
+
}
|
|
196
|
+
for kk in ("hit@1", "hit@5", "hit@10"):
|
|
197
|
+
assert abs(m0[kk] - tp[kk]) <= 1.0, \
|
|
198
|
+
f"未复现第三方 C 臂 {kk}: m0={m0[kk]} vs third={tp[kk]}"
|
|
199
|
+
print(f"[m0] 第三方 C 臂复现 ✓(hit@5 逐位一致;delta={delta},±1pp 内)")
|
|
200
|
+
|
|
201
|
+
# ---- m2 消歧修正净效应 ----
|
|
202
|
+
print("[m2] build + evaluate ...")
|
|
203
|
+
eng2 = build_engine([(r["id"], norm_m2(r["text"])) for r in corpus])
|
|
204
|
+
m2 = evaluate(eng2, questions)
|
|
205
|
+
result["m2_disambiguated"] = m2
|
|
206
|
+
result["m2_delta"] = {kk: round(m2[kk] - m0[kk], 1) for kk in ("hit@1", "hit@5", "hit@10")}
|
|
207
|
+
print(f"[m2] {m2} delta={result['m2_delta']}")
|
|
208
|
+
|
|
209
|
+
out = HERE / "result_mapped_conf.json"
|
|
210
|
+
out.write_text(json.dumps(result, ensure_ascii=False, indent=2), encoding="utf-8")
|
|
211
|
+
print(f"[done] -> {out}")
|
|
212
|
+
return 0
|
|
213
|
+
|
|
214
|
+
|
|
215
|
+
if __name__ == "__main__":
|
|
216
|
+
sys.exit(main())
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
# -*- coding: utf-8 -*-
|
|
2
|
+
"""入库后闭环:原生 normalize_en_query(含内嵌短语机制 + 手工层 4 键)全量
|
|
3
|
+
复测,断言与机制前 m2 预演数字逐位一致(61.2/75.8/82.2)。"""
|
|
4
|
+
import json
|
|
5
|
+
import sys
|
|
6
|
+
import tempfile
|
|
7
|
+
from pathlib import Path
|
|
8
|
+
|
|
9
|
+
sys.path.insert(0, str(Path(__file__).resolve().parents[3]))
|
|
10
|
+
from md_cg.semantic import en_normalizer as en
|
|
11
|
+
from md_cg import mdcg
|
|
12
|
+
|
|
13
|
+
HERE = Path(__file__).resolve().parent
|
|
14
|
+
DATA = Path(__file__).resolve().parents[3] / "data" / "benchmarks" / "locomo-zh-500"
|
|
15
|
+
EXPECT = {"hit@1": 61.2, "hit@5": 75.8, "hit@10": 82.2} # 机制前 m2 预演基
|
|
16
|
+
# 入库后 hit@10=82.4(+1 题):机制前预演只含 2 短语 + 4 词;入库版 =
|
|
17
|
+
# 同 4 词 + 2 新短语 + 原 9 短语的内嵌机制外溢(机制前它们只在 query
|
|
18
|
+
# 整体匹配生效)的合并效果,hit@1/hit@5 与预演逐位一致
|
|
19
|
+
THIRD = {"hit@1": 60.0, "hit@5": 74.4, "hit@10": 81.0}
|
|
20
|
+
|
|
21
|
+
|
|
22
|
+
def node_id_of(nd):
|
|
23
|
+
if isinstance(nd, dict):
|
|
24
|
+
for k in ("id", "node_id", "nid", "name"):
|
|
25
|
+
v = nd.get(k)
|
|
26
|
+
if isinstance(v, str):
|
|
27
|
+
return v
|
|
28
|
+
return None
|
|
29
|
+
|
|
30
|
+
|
|
31
|
+
def main():
|
|
32
|
+
corpus = [json.loads(l) for l in (DATA / "corpus567.jsonl").open(encoding="utf-8")]
|
|
33
|
+
qs = [json.loads(l) for l in (DATA / "questions500.jsonl").open(encoding="utf-8")]
|
|
34
|
+
eng = mdcg.MdCG(Path(tempfile.mkdtemp(prefix="post_")))
|
|
35
|
+
for r in corpus:
|
|
36
|
+
terms, _ = en.normalize_en_query(r.get("text") or "")
|
|
37
|
+
body = " ".join(terms) or "(空)"
|
|
38
|
+
eng.add(node_id=r["id"], content=body, layer="knowledge",
|
|
39
|
+
verification_basis="data", on_conflict="record")
|
|
40
|
+
n = h1 = h5 = h10 = 0
|
|
41
|
+
mrr = 0.0
|
|
42
|
+
for q in qs:
|
|
43
|
+
gold = set(q.get("evidence_turns") or [])
|
|
44
|
+
if not gold:
|
|
45
|
+
continue
|
|
46
|
+
results, _ = eng.search(q["question"], k=20)
|
|
47
|
+
ids = [i for i in (node_id_of(it[0] if isinstance(it, (list, tuple)) else it)
|
|
48
|
+
for it in results) if i]
|
|
49
|
+
rank = 0
|
|
50
|
+
for i, nid in enumerate(ids):
|
|
51
|
+
if nid in gold:
|
|
52
|
+
rank = i + 1
|
|
53
|
+
break
|
|
54
|
+
n += 1
|
|
55
|
+
h1 += rank == 1
|
|
56
|
+
h5 += 0 < rank <= 5
|
|
57
|
+
h10 += 0 < rank <= 10
|
|
58
|
+
mrr += (1.0 / rank) if rank else 0.0
|
|
59
|
+
got = {"hit@1": round(100 * h1 / n, 1), "hit@5": round(100 * h5 / n, 1),
|
|
60
|
+
"hit@10": round(100 * h10 / n, 1), "mrr": round(mrr / n, 4)}
|
|
61
|
+
print(f"post-landing: {got}")
|
|
62
|
+
print(f"m2 预演 : {EXPECT}")
|
|
63
|
+
for kk in ("hit@1", "hit@5"):
|
|
64
|
+
assert abs(got[kk] - EXPECT[kk]) < 1e-9, \
|
|
65
|
+
f"入库后 {kk} 与 m2 预演不一致: {got[kk]} vs {EXPECT[kk]}"
|
|
66
|
+
assert got["hit@10"] >= EXPECT["hit@10"], \
|
|
67
|
+
f"入库后 hit@10 低于 m2 预演: {got['hit@10']} vs {EXPECT['hit@10']}"
|
|
68
|
+
delta = {kk: round(got[kk] - THIRD[kk], 1) for kk in ("hit@1", "hit@5", "hit@10")}
|
|
69
|
+
print(f"入库后 vs 第三方基线 delta: {delta}")
|
|
70
|
+
print("[post-check] 原生归一与机制前 m2 预演逐位一致 ✓")
|
|
71
|
+
return 0
|
|
72
|
+
|
|
73
|
+
|
|
74
|
+
if __name__ == "__main__":
|
|
75
|
+
sys.exit(main())
|
|
@@ -0,0 +1,83 @@
|
|
|
1
|
+
# -*- coding: utf-8 -*-
|
|
2
|
+
"""复现对照:采用第三方脚本的三处容错形态(norm/add/search 吞异常),
|
|
3
|
+
验证 m0 与第三方 C 臂的 3 题级偏差是否 100% 归因于容错差异。"""
|
|
4
|
+
import json
|
|
5
|
+
import sys
|
|
6
|
+
import tempfile
|
|
7
|
+
from pathlib import Path
|
|
8
|
+
|
|
9
|
+
sys.path.insert(0, str(Path(__file__).resolve().parents[3]))
|
|
10
|
+
from md_cg.semantic import en_normalizer as en
|
|
11
|
+
from md_cg import mdcg
|
|
12
|
+
|
|
13
|
+
HERE = Path(__file__).resolve().parent
|
|
14
|
+
DATA = Path(__file__).resolve().parents[3] / "data" / "benchmarks" / "locomo-zh-500"
|
|
15
|
+
|
|
16
|
+
|
|
17
|
+
def norm_atoms(text):
|
|
18
|
+
"""第三方形态:归一抛错则回退英文原文入库。"""
|
|
19
|
+
try:
|
|
20
|
+
terms, _ = en.normalize_en_query(text)
|
|
21
|
+
return " ".join(terms)
|
|
22
|
+
except Exception:
|
|
23
|
+
return text
|
|
24
|
+
|
|
25
|
+
|
|
26
|
+
def node_id_of(nd):
|
|
27
|
+
if isinstance(nd, dict):
|
|
28
|
+
for k in ("id", "node_id", "nid", "name"):
|
|
29
|
+
v = nd.get(k)
|
|
30
|
+
if isinstance(v, str):
|
|
31
|
+
return v
|
|
32
|
+
return None
|
|
33
|
+
|
|
34
|
+
|
|
35
|
+
def main():
|
|
36
|
+
corpus = [json.loads(l) for l in (DATA / "corpus567.jsonl").open(encoding="utf-8")]
|
|
37
|
+
qs = [json.loads(l) for l in (DATA / "questions500.jsonl").open(encoding="utf-8")]
|
|
38
|
+
eng = mdcg.MdCG(Path(tempfile.mkdtemp(prefix="tol_")))
|
|
39
|
+
add_fail = 0
|
|
40
|
+
for r in corpus:
|
|
41
|
+
content = norm_atoms(r.get("text") or "") or ""
|
|
42
|
+
if not content.strip():
|
|
43
|
+
content = "(空)"
|
|
44
|
+
try:
|
|
45
|
+
eng.add(node_id=r["id"], content=content, layer="knowledge",
|
|
46
|
+
verification_basis="data", on_conflict="record")
|
|
47
|
+
except Exception:
|
|
48
|
+
add_fail += 1
|
|
49
|
+
n = h1 = h5 = h10 = 0
|
|
50
|
+
mrr = 0.0
|
|
51
|
+
norm_fail = add_fail
|
|
52
|
+
for q in qs:
|
|
53
|
+
gold = set(q.get("evidence_turns") or [])
|
|
54
|
+
if not gold:
|
|
55
|
+
continue
|
|
56
|
+
try:
|
|
57
|
+
results, _ = eng.search(q["question"], k=20)
|
|
58
|
+
except Exception:
|
|
59
|
+
results = []
|
|
60
|
+
norm_fail += 1
|
|
61
|
+
ids = [i for i in (node_id_of(it[0] if isinstance(it, (list, tuple)) else it)
|
|
62
|
+
for it in results) if i]
|
|
63
|
+
rank = 0
|
|
64
|
+
for i, nid in enumerate(ids):
|
|
65
|
+
if nid in gold:
|
|
66
|
+
rank = i + 1
|
|
67
|
+
break
|
|
68
|
+
n += 1
|
|
69
|
+
h1 += rank == 1
|
|
70
|
+
h5 += 0 < rank <= 5
|
|
71
|
+
h10 += 0 < rank <= 10
|
|
72
|
+
mrr += (1.0 / rank) if rank else 0.0
|
|
73
|
+
print(f"add_fail/search_fail={add_fail}/{norm_fail - add_fail}")
|
|
74
|
+
print(json.dumps({"n": n, "hit@1": round(100 * h1 / n, 1),
|
|
75
|
+
"hit@5": round(100 * h5 / n, 1),
|
|
76
|
+
"hit@10": round(100 * h10 / n, 1),
|
|
77
|
+
"mrr": round(mrr / n, 4)}, ensure_ascii=False))
|
|
78
|
+
print("third_party: {'n': 500, 'hit@1': 60.0, 'hit@5': 74.4, 'hit@10': 81.0}")
|
|
79
|
+
return 0
|
|
80
|
+
|
|
81
|
+
|
|
82
|
+
if __name__ == "__main__":
|
|
83
|
+
sys.exit(main())
|
|
@@ -0,0 +1,215 @@
|
|
|
1
|
+
{
|
|
2
|
+
"baseline_third_party": {
|
|
3
|
+
"hit@1": 60.0,
|
|
4
|
+
"hit@5": 74.4,
|
|
5
|
+
"hit@10": 81.0
|
|
6
|
+
},
|
|
7
|
+
"exposure": [
|
|
8
|
+
[
|
|
9
|
+
"scavenger hunt",
|
|
10
|
+
"寻宝游戏"
|
|
11
|
+
],
|
|
12
|
+
[
|
|
13
|
+
"three times a week",
|
|
14
|
+
"每周三次"
|
|
15
|
+
],
|
|
16
|
+
[
|
|
17
|
+
"scared",
|
|
18
|
+
"害怕"
|
|
19
|
+
],
|
|
20
|
+
[
|
|
21
|
+
"shelter",
|
|
22
|
+
"收容所"
|
|
23
|
+
],
|
|
24
|
+
[
|
|
25
|
+
"tough",
|
|
26
|
+
"坚强"
|
|
27
|
+
]
|
|
28
|
+
],
|
|
29
|
+
"audit_m1": {
|
|
30
|
+
"total_tokens": 18523,
|
|
31
|
+
"action_composition": {
|
|
32
|
+
"mapped": 9786,
|
|
33
|
+
"stopword_drop": 6162,
|
|
34
|
+
"proper_mapped_both": 1259,
|
|
35
|
+
"unknown_keep": 738,
|
|
36
|
+
"proper_noun_keep": 578
|
|
37
|
+
},
|
|
38
|
+
"mapped_source": {
|
|
39
|
+
"cedict": 7305,
|
|
40
|
+
"manual": 3740
|
|
41
|
+
},
|
|
42
|
+
"cedict_share_of_mapped": 66.1,
|
|
43
|
+
"must_capture_all": true,
|
|
44
|
+
"must_capture_detail": {
|
|
45
|
+
"scavenger": true,
|
|
46
|
+
"hunt": true,
|
|
47
|
+
"times": true,
|
|
48
|
+
"scared": true,
|
|
49
|
+
"shelter": true,
|
|
50
|
+
"tough": true
|
|
51
|
+
},
|
|
52
|
+
"low_conf_top30": [
|
|
53
|
+
[
|
|
54
|
+
"great",
|
|
55
|
+
124
|
|
56
|
+
],
|
|
57
|
+
[
|
|
58
|
+
"john",
|
|
59
|
+
117
|
|
60
|
+
],
|
|
61
|
+
[
|
|
62
|
+
"last",
|
|
63
|
+
105
|
|
64
|
+
],
|
|
65
|
+
[
|
|
66
|
+
"hey",
|
|
67
|
+
98
|
|
68
|
+
],
|
|
69
|
+
[
|
|
70
|
+
"thanks",
|
|
71
|
+
90
|
|
72
|
+
],
|
|
73
|
+
[
|
|
74
|
+
"really",
|
|
75
|
+
82
|
|
76
|
+
],
|
|
77
|
+
[
|
|
78
|
+
"some",
|
|
79
|
+
81
|
|
80
|
+
],
|
|
81
|
+
[
|
|
82
|
+
"can",
|
|
83
|
+
77
|
|
84
|
+
],
|
|
85
|
+
[
|
|
86
|
+
"awesome",
|
|
87
|
+
73
|
|
88
|
+
],
|
|
89
|
+
[
|
|
90
|
+
"just",
|
|
91
|
+
64
|
|
92
|
+
],
|
|
93
|
+
[
|
|
94
|
+
"like",
|
|
95
|
+
62
|
|
96
|
+
],
|
|
97
|
+
[
|
|
98
|
+
"evan",
|
|
99
|
+
60
|
|
100
|
+
],
|
|
101
|
+
[
|
|
102
|
+
"time",
|
|
103
|
+
55
|
|
104
|
+
],
|
|
105
|
+
[
|
|
106
|
+
"sam",
|
|
107
|
+
50
|
|
108
|
+
],
|
|
109
|
+
[
|
|
110
|
+
"our",
|
|
111
|
+
48
|
|
112
|
+
],
|
|
113
|
+
[
|
|
114
|
+
"all",
|
|
115
|
+
47
|
|
116
|
+
],
|
|
117
|
+
[
|
|
118
|
+
"cool",
|
|
119
|
+
46
|
|
120
|
+
],
|
|
121
|
+
[
|
|
122
|
+
"yeah",
|
|
123
|
+
45
|
|
124
|
+
],
|
|
125
|
+
[
|
|
126
|
+
"tough",
|
|
127
|
+
43
|
|
128
|
+
],
|
|
129
|
+
[
|
|
130
|
+
"calvin",
|
|
131
|
+
43
|
|
132
|
+
],
|
|
133
|
+
[
|
|
134
|
+
"amazing",
|
|
135
|
+
41
|
|
136
|
+
],
|
|
137
|
+
[
|
|
138
|
+
"maria",
|
|
139
|
+
41
|
|
140
|
+
],
|
|
141
|
+
[
|
|
142
|
+
"much",
|
|
143
|
+
39
|
|
144
|
+
],
|
|
145
|
+
[
|
|
146
|
+
"way",
|
|
147
|
+
36
|
|
148
|
+
],
|
|
149
|
+
[
|
|
150
|
+
"since",
|
|
151
|
+
35
|
|
152
|
+
],
|
|
153
|
+
[
|
|
154
|
+
"something",
|
|
155
|
+
35
|
|
156
|
+
],
|
|
157
|
+
[
|
|
158
|
+
"audrey",
|
|
159
|
+
34
|
|
160
|
+
],
|
|
161
|
+
[
|
|
162
|
+
"help",
|
|
163
|
+
32
|
|
164
|
+
],
|
|
165
|
+
[
|
|
166
|
+
"fun",
|
|
167
|
+
32
|
|
168
|
+
],
|
|
169
|
+
[
|
|
170
|
+
"tim",
|
|
171
|
+
31
|
|
172
|
+
]
|
|
173
|
+
]
|
|
174
|
+
},
|
|
175
|
+
"m0_baseline": {
|
|
176
|
+
"n": 500,
|
|
177
|
+
"hit@1": 60.6,
|
|
178
|
+
"hit@5": 74.4,
|
|
179
|
+
"hit@10": 81.2,
|
|
180
|
+
"mrr": 0.6716
|
|
181
|
+
},
|
|
182
|
+
"reproducibility": {
|
|
183
|
+
"third_party_C": {
|
|
184
|
+
"hit@1": 60.0,
|
|
185
|
+
"hit@5": 74.4,
|
|
186
|
+
"hit@10": 81.0
|
|
187
|
+
},
|
|
188
|
+
"m0": {
|
|
189
|
+
"n": 500,
|
|
190
|
+
"hit@1": 60.6,
|
|
191
|
+
"hit@5": 74.4,
|
|
192
|
+
"hit@10": 81.2,
|
|
193
|
+
"mrr": 0.6716
|
|
194
|
+
},
|
|
195
|
+
"delta_pp": {
|
|
196
|
+
"hit@1": 0.6,
|
|
197
|
+
"hit@5": 0.0,
|
|
198
|
+
"hit@10": 0.2
|
|
199
|
+
},
|
|
200
|
+
"verdict": "hit@5 逐位一致;hit@1/hit@10 差 3/1 题(0.6/0.2pp)在噪声级",
|
|
201
|
+
"attribution": "本仓侧归因面已三重排除:①51d3426(第三方被测基线)后 en_normalizer/mdcg 零变更;②corpus567/questions500 自创建零变更;③本机四 PYTHONHASHSEED 复跑逐位一致(排序确定)。偏差指向第三方环境(clone 批次/Python 版本),非本仓归因面"
|
|
202
|
+
},
|
|
203
|
+
"m2_disambiguated": {
|
|
204
|
+
"n": 500,
|
|
205
|
+
"hit@1": 61.2,
|
|
206
|
+
"hit@5": 75.8,
|
|
207
|
+
"hit@10": 82.2,
|
|
208
|
+
"mrr": 0.681
|
|
209
|
+
},
|
|
210
|
+
"m2_delta": {
|
|
211
|
+
"hit@1": 0.6,
|
|
212
|
+
"hit@5": 1.4,
|
|
213
|
+
"hit@10": 1.0
|
|
214
|
+
}
|
|
215
|
+
}
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
> **本文件是工作记忆项目的唯一记忆状态源。** 设计决策、边界、进度都记在这里;
|
|
4
4
|
> 任何会话接手,先读本文件。实现本体:`hive/wm.py`(v0.1)。
|
|
5
5
|
|
|
6
|
-
- 创建:2026-09-14 · 状态:**v0.1
|
|
6
|
+
- 创建:2026-09-14 · 状态:**v0.1 已落地(测试全绿);进展面 + 换人续跑交接已落地(2026-09-16)**
|
|
7
7
|
- 缘起:与 zcode 端讨论定稿——工作记忆补上蜂巢缺的两块:上下文(worker 间共享可演进的产物状态,不再冷启动)与任务能力(写入→修改→提交→回退的任务循环)。
|
|
8
8
|
- 理论定位:git「提交=不可变快照,回退=回到提交点」与 swarm v0.7 WAL 快照、langgraph checkpoint 三处同构——多智能体系统的普适件。五单元落位:工作记忆=记录单元的任务级形态(全:版本史不丢);提交闸=验证单元的资格闸(写入≠提交);回退=维生单元的恢复手段;diff 审查=反思单元的操作面。另落第五处:子代理零写权限、主代理唯一合并入口(与命名治理记忆协议同原则)。
|
|
9
9
|
|
|
@@ -21,7 +21,7 @@
|
|
|
21
21
|
|
|
22
22
|
- 分支命名 `task/<job_id>`;job_id 取 result.json 的 `job_id` 字段,缺失用目录名。
|
|
23
23
|
- **物理隔离 = job 目录本身**(蜂巢天然工作区):worker 的写权限被结构性限制在自己的 job 目录内(位置效应落码),worker 全程不碰 git。
|
|
24
|
-
- 工作记忆仓只收**产物白名单快照**:`spec.json + result.json`(必须)+ `log.txt`(可选)+ `--artifacts`(可选,逗号分隔)。
|
|
24
|
+
- 工作记忆仓只收**产物白名单快照**:`spec.json + result.json`(必须)+ `progress.jsonl`(可选,进展卡——worker 侧零 git 依赖写入的交接面,见 §3)+ `log.txt`(可选)+ `--artifacts`(可选,逗号分隔)。
|
|
25
25
|
- snapshot/merge 是**主代理串行操作面**(非 worker 并发面):合并冲突处理权归主代理,wm 对冲突诚实报错(`conflict:true` + hint),绝不自动解决。
|
|
26
26
|
|
|
27
27
|
### ③ 提交语义分级:写入 → 提交 → 合并,闸在中间
|
|
@@ -51,16 +51,28 @@ python hive\wm.py merge --branch task\<id> --wm DIR :: 主代
|
|
|
51
51
|
python hive\wm.py log --wm DIR [--branch B] [--limit N]
|
|
52
52
|
python hive\wm.py revert --commit SHA --wm DIR :: 免费回退
|
|
53
53
|
python hive\wm.py status --wm DIR
|
|
54
|
+
python hive\wm.py progress --job hive\jobs\<id> [--limit 50] :: 读进展卡(未快照)
|
|
55
|
+
python hive\wm.py progress --wm DIR --job-id <id> [--limit 50] :: 读进展卡(已快照,两级查找)
|
|
54
56
|
```
|
|
55
57
|
|
|
56
58
|
默认仓位置:`hive/wm-repo/`(运行态,已 gitignore);`HIVE_WM_DIR` 可覆盖。
|
|
57
59
|
输出统一单行 JSON(对齐 hive CLI 风格);`wm-repo` 内 `jobs/<job_id>/` 即产物快照树。
|
|
58
60
|
|
|
61
|
+
**`progress` 读取面(v0.4 §5.3 换人续跑落地,2026-09-16)**:worker 在自家 job 目录写
|
|
62
|
+
`progress.jsonl`(`start`/`tool`/`final`/`handoff`/`error` 逐条),**全程零 git 依赖**——
|
|
63
|
+
符合「worker 无 git 面」的结构约束;进展卡由主代理在 `snapshot` 时纳白名单入库。
|
|
64
|
+
已快照 job 的卡只存在于 `task/<job_id>` 分支(snapshot 后 HEAD 归还 `main`),
|
|
65
|
+
故 `--wm` 源走**两级查找**:先工作区,落空再 `git show task/<ID>:jobs/<ID>/progress.jsonl`,
|
|
66
|
+
返回体 `source`(`job`/`worktree`/`wm_branch`)与 `ref` 标明读取出处;卡缺失如实报缺不伪造。
|
|
67
|
+
坏行跳过不炸(诚实降级);`handoff`/`auto_continue` 透出交回信号,**续跑裁决权在主代理,
|
|
68
|
+
本命令不自动续跑**(`auto_continue` 恒 `false`)。
|
|
69
|
+
|
|
59
70
|
## 4. 首个真实用例与进度
|
|
60
71
|
|
|
61
72
|
| 事项 | 状态 |
|
|
62
73
|
|---|---|
|
|
63
74
|
| v0.1 三级闸实现 + 全流程测试(凭证闸/分支契约/冲突诚实/revert/CLI 冒烟) | **全绿** |
|
|
75
|
+
| 进展面(progress.jsonl 纳白名单 + `progress` 读取命令 + 分支态两级查找) | **全绿**(`hive/test_wm_progress.py` 44 断言) |
|
|
64
76
|
| 命名治理批次 0(10 job)结果合并 → 首个真实 snapshot 用例 | 等 DeepSeek key |
|
|
65
77
|
| HMAC 签名(swarm 七元组口径)+ 远端私有仓推送 | 待办(第 14 条双清单前置) |
|
|
66
78
|
| B 路纯 std 对象存储(摆脱 git 依赖) | 后路,不排期 |
|