eduevidence 6.2.0 → 6.3.0

package/docs/reproducibility.md

@@ -0,0 +1,138 @@
+# EduEvidence 复现指南
+
+> **复现范围声明（v5.2.0 起，R6 口径收敛）**：可复现承诺分两层，不得混谈。
+>
+> 1. **确定性层（完全可复现）**：schema 校验、置信度计算（规则化公式）、pre-verdict gate、
+>    meta-analysis/robustness 数学、报告烘焙与哈希链——同一输入在同一版本上逐字节可复现；
+>    本指南的三步走覆盖这一层。
+> 2. **LLM 层（有边界可复现）**：检索、抽取、挑战、裁决等依赖外部 LLM 的阶段不是逐字节
+>    确定的。其复现条件 = 模型版本 + temperature + prompt 版本 三者同时固定；当前仓库通过
+>    run manifest 记录这三项（如 `benchmarks/empirical/run-empirical-01/manifest.json` 的
+>    `temperature: 0.0`），但不承诺跨模型供应商的结果一致。
+>
+> 引擎对"证据是否真实"的立场：所有示例包在 `result.json.meta.data_origin` 与报告头徽章中
+> 如实标注数据来源（`real` / `manual_curated` / `synthetic` / `hybrid`）；引用的 DOI 经
+> `scripts/audit_dois.py` 对照 Crossref/DataCite 注册表核验（报告：
+> `benchmarks/doi-audit/report.md`）。标注 `synthetic` 的演示包不构成任何实证声明。
+
+本指南确保任何人在全新环境中都能复现 EduEvidence 确定性层的校验、测试与 Benchmark 结果。全部命令以仓库根目录 `<repo-root>`（即 pyproject.toml 所在目录）为工作目录执行。
+
+## 〇、V2 Research Engine 复现
+
+- 引擎模块 `engine/`（stdlib-only）可独立复现：`python3 -m pytest tests/test_v2_*.py`。
+- 图状态复现：`graph/HEAD` + `revisions/rev-N` 快照 + `manifest.json` 的 before/after 哈希链；同一输入 bundle 提交产生确定性哈希。
+- V1 历史产物（`examples/ai-coding-assistant/`、旧 `runs/`）不可变；`eduevidence migrate-v1 --pack <dir>` 生成新 V2 Project 且不改动源包。
+- Shared Research Library 与 Project 图同用不可变 revision 模型；快照导入记录 `library_revision/library_entity_id/content_hash/imported_at`。
+- 测试覆盖：V1 兼容基线、图原子性/孤儿 revision/HEAD 镜像分歧修复、迁移不变量、模式推荐、能力规划、synthesis 独立计数、Tribunal 政策、Gap 推导、数据集隐私门、全周期单 revision 提交、决策 diff、CLI 薄分发。
+
+## 一、环境要求
+
+- **Python**：≥ 3.10（pyproject.toml 中 `requires-python = ">=3.10"`）；
+- **pytest**：≥ 7.0（dev 依赖，安装时自动带入）；
+- 无其它强制运行时依赖：EduEvidence 本体是文档+Schema+脚本，Mode A 平台原生模式下不需要 daemon / CLI / Agent MCP；
+- 操作系统：macOS / Linux / Windows 均可；建议使用 venv 隔离环境。
+
+### 1.1 创建虚拟环境（推荐）
+
+```bash
+cd <repo-root>
+python3.11 -m venv .venv
+source .venv/bin/activate   # Windows: .venv\Scripts\activate
+```
+
+## 二、安装步骤
+
+在虚拟环境内安装本项目（含 dev 依赖）：
+
+```bash
+pip install -e '.[dev]'
+```
+
+- `-e`（editable）安装使脚本与 schema 变更即时生效，无需重装；
+- `.[dev]` 安装 `pytest>=7.0`；
+- 安装成功后可用 `python -c "import sys; print(sys.version)"` 确认版本 ≥ 3.10。
+
+## 三、运行测试
+
+```bash
+pytest
+```
+
+- 测试自动发现 `tests/` 下 `test_*.py` 文件（见 pyproject.toml `[tool.pytest.ini_options]`）；
+- 覆盖范围：schema 校验器、confidence 规则化计算、Complexity Gate 判级、benchmark 输出格式等；
+- 预期结果：全部测试通过（`addopts = "-q"` 输出简洁）。
+
+## 四、运行 Schema 验证
+
+校验某份数据是否符合对应 Schema（mandatory 字段缺失会标记为 UNSUPPORTED）：
+
+```bash
+python scripts/validate_schema.py \
+  --schema schemas/evidence.schema.json \
+  --data examples/ai-coding-assistant/evidence.jsonl
+```
+
+- `--schema`：指定任一顶层 Schema（schemas/ 下 37 个契约，跨 V1–V4 代际，如 education-frame / evidence / methodology / verdict / intervention / evaluation / report-result 等）；
+- `--data`：指定要校验的 JSON 或 JSONL 数据文件；
+- 主 Demo 示例数据位于 `examples/ai-coding-assistant/evidence.jsonl`，验证通过时应输出每条 evidence 的校验状态与 SUPPORTED/UNSUPPORTED 统计；
+- 合格线：**校验通过率 100%**，任一条未通过即非零退出码。
+
+## 五、运行 Benchmark
+
+逐题运行五条基线（B0–B4）并生成结果：
+
+```bash
+python scripts/benchmark.py --questions benchmarks/questions.jsonl
+```
+
+- `--questions`：第一版 30 题（S×10 / M×10 / L×10），每题字段为 `id`、`level`（S/M/L）、`domain`、`question`、`expected_outcomes`、`notes`（与 `validate_questions()` 校验一致）；人工金标注位于 `benchmarks/annotations/gold-<id>.json`；
+- 运行产物：每题一个 JSON，写入 `benchmarks/results/`；
+- 核心指标（Citation Support Precision、Unsupported Claim Rate、Contradiction Discovery Rate、Outcome Separation Accuracy、Scope Calibration、Intervention Evidence Alignment）由 `benchmarks/evaluator/` 对照 `benchmarks/annotations/` 计算；
+- 可选参数（如 `--ablation` 跑 A1–A7、`--repeat 5` 测稳定性）以 `python scripts/benchmark.py --help` 为准。
+
+## 六、输出产物：Research & Decision Pack
+
+每次端到端运行（Demo 或 L 级题目）产出一份双层 Research & Decision Pack：
+
+- **Visual Brief**：用于快速浏览 Decision、Outcome Separation、Evidence Tribunal、Evidence-to-Action 与关键来源。
+- **Full Report**：由当前研究内容规划为 **5–7 个动态章节**，而不是固定章节模板。
+
+完整报告必须覆盖以下语义模块；这些模块可以按研究叙事合并进同一章节，但不得遗漏：
+
+| 模块 | 内容 |
+|------|------|
+| decision | 最终决策、Confidence、可说/不可说与主要风险 |
+| scope | Education Research Frame、研究边界与决策标准 |
+| retrieval | 检索策略、来源覆盖与证据选择 |
+| outcomes | Outcome Separation：任务表现、学习、保持、迁移与风险 |
+| evidence | Claim-Level Evidence 与 Evidence Matrix |
+| quality | Methodology / Evidence Quality Audit |
+| conflicts | Skeptic / Counter-Evidence / Conflict Analysis |
+| trace | Evidence Tribunal + Claim → Evidence → Source 追溯 |
+| applicability | 适用性、外推边界与必要条件 |
+| intervention | 最小可验证干预、Guardrails 与 Stop Conditions |
+| evaluation | Baseline / Post-test / Retention / Transfer 评价方案 |
+| sources | Sources、Provenance 与附录 |
+
+Pack 的证据链始终可以从 Decision / Claim 反查到 Evidence，再追溯到 Source 与 source location；章节如何合并不改变这一追溯关系。
+
+## 七、常见问题（FAQ）
+
+| 现象 | 处理 |
+|------|------|
+| `pytest` 无测试收集 | 确认在仓库根目录运行且依赖已安装 |
+| validate_schema 报非零退出 | 逐条查看 UNSUPPORTED 记录，补齐 mandatory 字段 |
+| benchmark 输出缺失 | 检查 `benchmarks/results/` 目录可写；确认 `benchmarks/questions.jsonl` 字段完整 |
+| pip install 找不到 `[dev]` | 确认工作目录是根目录（pyproject.toml 存在） |
+| Python 版本低于 3.10 | 升级解释器或使用 pyenv 安装 ≥3.10 |
+
+## 八、端到端复现三步走
+
+```bash
+pip install -e '.[dev]'   # 1. 安装
+pytest                    # 2. 验证环境
+python scripts/validate_schema.py --schema schemas/evidence.schema.json --data examples/ai-coding-assistant/evidence.jsonl
+python scripts/benchmark.py --questions benchmarks/questions.jsonl   # 3. 复现示例与基准
+```
+
+三步全部成功即视为复现完成。

package/domains/_neutral/copy/few_shots.json

@@ -0,0 +1,21 @@
+{
+  "domain": "_neutral",
+  "states": {
+    "adopt": {
+      "zh": "",
+      "en": ""
+    },
+    "pilot": {
+      "zh": "",
+      "en": ""
+    },
+    "reject": {
+      "zh": "",
+      "en": ""
+    },
+    "insufficient_evidence": {
+      "zh": "",
+      "en": ""
+    }
+  }
+}

package/domains/_neutral/copy/framing_lexicon.json

@@ -0,0 +1,19 @@
+{
+  "domain": "_neutral",
+  "frame_enums": {
+    "zh": {},
+    "en": {}
+  },
+  "frame_fields": {
+    "zh": {},
+    "en": {}
+  },
+  "frame_subfields": {
+    "zh": {},
+    "en": {}
+  },
+  "evidence_detail_labels": {
+    "zh": {},
+    "en": {}
+  }
+}

package/domains/_neutral/copy/module_labels.json

@@ -0,0 +1,5 @@
+{
+  "domain": "_neutral",
+  "purpose": "Neutral UI labels index; companions module_labels_*.json hold the actual keys.",
+  "labels": {}
+}

package/domains/_neutral/copy/module_labels_footer.json

@@ -0,0 +1,102 @@
+{
+  "domain": "_neutral",
+  "purpose": "Neutral UI labels — module_labels_footer.",
+  "labels": {
+    "tribunal_decision": {
+      "zh": "决策",
+      "en": "Decision"
+    },
+    "tribunal_confidence": {
+      "zh": "置信度",
+      "en": "Confidence"
+    },
+    "tribunal_can": {
+      "zh": "可以主张",
+      "en": "Can claim"
+    },
+    "tribunal_uncertain": {
+      "zh": "尚不能主张",
+      "en": "Cannot yet claim"
+    },
+    "tribunal_cannot": {
+      "zh": "被反驳的主张",
+      "en": "Contradicted claims"
+    },
+    "tribunal_missing": {
+      "zh": "缺失证据",
+      "en": "Missing evidence"
+    },
+    "tribunal_flow": {
+      "zh": "EvidenceFlow 协议",
+      "en": "EvidenceFlow Protocol"
+    },
+    "tribunal_figure": {
+      "zh": "裁决信息图",
+      "en": "Tribunal infographic"
+    },
+    "header_evidence": {
+      "zh": "证据 ",
+      "en": "Evidence: "
+    },
+    "header_sources": {
+      "zh": "来源 ",
+      "en": "Sources: "
+    },
+    "header_mode": {
+      "zh": "模式：",
+      "en": "Mode: "
+    },
+    "header_generated": {
+      "zh": "生成时间：",
+      "en": "Generated: "
+    },
+    "header_evidence_suffix": {
+      "zh": " 条",
+      "en": ""
+    },
+    "header_sources_suffix": {
+      "zh": " 个",
+      "en": ""
+    },
+    "footer_schema": {
+      "zh": "Schema",
+      "en": "Schema"
+    },
+    "footer_claims": {
+      "zh": "Claim Binding",
+      "en": "Claim Binding"
+    },
+    "footer_numbers": {
+      "zh": "Numeric Consistency",
+      "en": "Numeric Consistency"
+    },
+    "footer_bilingual": {
+      "zh": "Bilingual Structure",
+      "en": "Bilingual Structure"
+    },
+    "footer_language": {
+      "zh": "语言人话化",
+      "en": "Human Language"
+    },
+    "footer_no_false_precision": {
+      "zh": "无伪精度",
+      "en": "False Precision"
+    },
+    "footer_lieflat_bound": {
+      "zh": "Lieflat 数据溯源",
+      "en": "Lieflat Data Bound"
+    },
+    "footer_no_axis_distortion": {
+      "zh": "坐标轴无失真",
+      "en": "Axis Distortion"
+    },
+    "footer_colorblind_safe": {
+      "zh": "色盲安全",
+      "en": "Colorblind Safe"
+    },
+    "footer": {
+      "zh": "EduEvidence 证据报告 · {integrity} · 单文件离线可打开 · 数据源：result.json",
+      "en": "EduEvidence Evidence Report · {integrity} · single-file offline · source: result.json"
+    }
+  }
+}

package/domains/_neutral/copy/module_labels_modules.json

@@ -0,0 +1,204 @@
+{
+  "domain": "_neutral",
+  "purpose": "Neutral UI module/visual labels (part 2).",
+  "labels": {
+    "svg_balance_desc": {
+      "zh": "各结果类型的正向 / 负向 / 零效应证据条数（基于 effect_direction，不等同于 Claim 是否被支持）。",
+      "en": "Positive / negative / null effect counts per outcome (based on effect_direction, not claim support)."
+    },
+    "svg_balance_title": {
+      "zh": "各结果类型证据效应分布",
+      "en": "Outcome evidence effect balance"
+    },
+    "svg_benchmark_desc": {
+      "zh": "各基线的引用支持精度（Citation support precision）对比；无基准数据时不绘制。",
+      "en": "Citation support precision per baseline; not drawn when no baseline data exists."
+    },
+    "svg_benchmark_title": {
+      "zh": "基准对比：各基线引用支持精度",
+      "en": "Benchmark: citation support precision by baseline"
+    },
+    "svg_figure1_desc": {
+      "zh": "各结果类型的正向 / 负向 / 零效应证据条数；计数轴整数刻度，不随主题变化。来源：EduEvidence result.json。",
+      "en": "Counts of positive / negative / null effects per outcome type with an integer count axis, theme-independent. Source: EduEvidence result.json."
+    },
+    "svg_figure1_title": {
+      "zh": "各结果类型效应方向分布（出版级学术图）",
+      "en": "Effect direction by outcome type (publication figure)"
+    },
+    "svg_tribunal_desc": {
+      "zh": "可以主张与不可主张的证据 ID 与建议决策徽章；完整主张文本见下方裁决卡片。",
+      "en": "Evidence IDs for claims that can and cannot be claimed, plus the recommended action badge; full claim text is in the tribunal cards below."
+    },
+    "svg_tribunal_title": {
+      "zh": "证据裁决信息图",
+      "en": "Evidence Tribunal infographic"
+    },
+    "svg_workflow_title": {
+      "zh": "EvidenceFlow 协议流程",
+      "en": "EvidenceFlow Protocol"
+    },
+    "outcome_separation_title": {
+      "zh": "结果分离",
+      "en": "Outcome separation"
+    },
+    "outcome_sep_note": {
+      "zh": "将不同结果类型分开裁决。",
+      "en": "Outcomes are adjudicated separately."
+    },
+    "outcome_group_task": {
+      "zh": "过程 / 近端",
+      "en": "Proximal / process"
+    },
+    "outcome_group_learning": {
+      "zh": "目标结果",
+      "en": "Target outcomes"
+    },
+    "outcome_group_risk": {
+      "zh": "风险",
+      "en": "Risk"
+    },
+    "outcome_group_other": {
+      "zh": "其他结果",
+      "en": "Other outcomes"
+    },
+    "method_guard": {
+      "zh": "构念护栏",
+      "en": "Construct guard"
+    },
+    "applicability": {
+      "zh": [
+        "适用于谁",
+        "适用范围",
+        "适用结果",
+        "适用条件",
+        "目标人群",
+        "目标情境"
+      ],
+      "en": [
+        "Suitable for",
+        "Scope",
+        "Outcomes",
+        "Conditions",
+        "Target population",
+        "Target context"
+      ]
+    },
+    "intervention_lesaners": {
+      "zh": "目标人群",
+      "en": "Target population"
+    },
+    "intervention_population": {
+      "zh": "目标人群",
+      "en": "Target population"
+    },
+    "intervention_duration": {
+      "zh": "试点时长",
+      "en": "Pilot duration"
+    },
+    "intervention_policy": {
+      "zh": "使用规则",
+      "en": "Usage policy"
+    },
+    "intervention_rule": {
+      "zh": "规则",
+      "en": "Rule"
+    },
+    "intervention_activities": {
+      "zh": "活动",
+      "en": "Activities"
+    },
+    "intervention_check": {
+      "zh": "结果检查",
+      "en": "Outcome check"
+    },
+    "intervention_stop": {
+      "zh": "停止条件",
+      "en": "Stop conditions"
+    },
+    "intervention_timeline": {
+      "zh": "干预时间线信息图",
+      "en": "Intervention timeline infographic"
+    },
+    "evaluation_measures": {
+      "zh": [
+        "基线",
+        "后测",
+        "持续",
+        "扩展"
+      ],
+      "en": [
+        "Baseline",
+        "Post",
+        "Durability",
+        "Scale-out"
+      ]
+    },
+    "evaluation_metrics": {
+      "zh": [
+        "过程指标",
+        "效果指标",
+        "风险指标"
+      ],
+      "en": [
+        "Process metrics",
+        "Effect metrics",
+        "Risk metrics"
+      ]
+    },
+    "evaluation_question": {
+      "zh": "研究问题",
+      "en": "Research question"
+    },
+    "evaluation_threshold": {
+      "zh": "成功阈值",
+      "en": "Success threshold"
+    },
+    "evaluation_plan": {
+      "zh": "分析计划",
+      "en": "Analysis plan"
+    },
+    "evaluation_figure": {
+      "zh": "评价设计信息图",
+      "en": "Evaluation design infographic"
+    },
+    "svg_intervention_title": {
+      "zh": "干预时间线",
+      "en": "Intervention timeline"
+    },
+    "svg_intervention_desc": {
+      "zh": "各试点阶段的短名称与活动数量。",
+      "en": "Short phase names and activity counts."
+    },
+    "svg_evaluation_title": {
+      "zh": "评价设计流程",
+      "en": "Evaluation design flow"
+    },
+    "svg_evaluation_desc": {
+      "zh": "基线、后测、持续与扩展的评价流程。",
+      "en": "Evaluation flow across baseline, post, durability and scale-out."
+    },
+    "svg_workflow_desc": {
+      "zh": "从问题框架到裁决与适用性的完整流程。",
+      "en": "Research flow from framing to adjudication and applicability."
+    },
+    "decision_kpi": {
+      "zh": [
+        "决策",
+        "置信度",
+        "证据最充分的结果",
+        "最不确定的结果",
+        "主要风险",
+        "来源数量"
+      ],
+      "en": [
+        "Decision",
+        "Confidence",
+        "Best-supported outcome",
+        "Most uncertain outcome",
+        "Main risk",
+        "Sources"
+      ]
+    }
+  }
+}

package/domains/_neutral/copy/module_labels_nav.json

@@ -0,0 +1,126 @@
+{
+  "domain": "_neutral",
+  "purpose": "Neutral UI labels — module_labels_nav.",
+  "labels": {
+    "theme_label": {
+      "zh": "主题",
+      "en": "Theme"
+    },
+    "lang_label": {
+      "zh": "语言",
+      "en": "Language"
+    },
+    "zh": {
+      "zh": "中文",
+      "en": "中文"
+    },
+    "en": {
+      "zh": "EN",
+      "en": "EN"
+    },
+    "visual_brief": {
+      "zh": "可视化摘要",
+      "en": "Visual Brief"
+    },
+    "full_report": {
+      "zh": "完整报告",
+      "en": "Full Report"
+    },
+    "toolbar_brief": {
+      "zh": "摘要",
+      "en": "Brief"
+    },
+    "toolbar_full": {
+      "zh": "完整报告",
+      "en": "Full report"
+    },
+    "toolbar_print": {
+      "zh": "打印",
+      "en": "Print"
+    },
+    "brief_nav_aria": {
+      "zh": "摘要导航",
+      "en": "Brief navigation"
+    },
+    "contents": {
+      "zh": "目录",
+      "en": "Contents"
+    },
+    "collapse_contents": {
+      "zh": "收起目录",
+      "en": "Collapse contents"
+    },
+    "expand_contents": {
+      "zh": "展开目录",
+      "en": "Expand contents"
+    },
+    "expand_evidence": {
+      "zh": "查看完整证据",
+      "en": "View full evidence"
+    },
+    "expand_methodology": {
+      "zh": "查看审计依据",
+      "en": "View audit rationale"
+    },
+    "expand_source": {
+      "zh": "查看来源与溯源",
+      "en": "View source & provenance"
+    },
+    "expand_details": {
+      "zh": "展开完整说明",
+      "en": "Expand full explanation"
+    },
+    "what_this_means": {
+      "zh": "这意味着什么",
+      "en": "What this means"
+    },
+    "original_title": {
+      "zh": "原文标题",
+      "en": "Original title"
+    },
+    "original_text": {
+      "zh": "原文",
+      "en": "Original text"
+    },
+    "full_report_intro": {
+      "zh": "结论前置：全部可追溯证据与方法学细节都在这里，关键论证位置穿插有意义的可视化，每个数字都能回查到 result.json。",
+      "en": "Conclusions first: every traceable piece of evidence and method note lives here, with visuals only at points where they add meaning. Every number traces back to result.json."
+    },
+    "raw_tag_title": {
+      "zh": "原始标识",
+      "en": "raw id"
+    },
+    "colon": {
+      "zh": "：",
+      "en": ": "
+    },
+    "trace_claim_sep": {
+      "zh": "：",
+      "en": ": "
+    },
+    "lang_switcher_aria": {
+      "zh": "语言切换 / Language switch",
+      "en": "语言切换 / Language switch"
+    },
+    "theme_switcher_aria": {
+      "zh": "主题 / Theme",
+      "en": "主题 / Theme"
+    },
+    "no_data": {
+      "zh": "无数据。",
+      "en": "No data."
+    },
+    "cite_doi_ok": {
+      "zh": "DOI 可验证",
+      "en": "DOI verified"
+    },
+    "benchmark_note": {
+      "zh": "result.json 未携带 benchmark.baselines，本图不绘制；基准表现见独立基准报告。",
+      "en": "result.json carries no benchmark.baselines, so this visual is omitted; see the standalone benchmark report."
+    },
+    "method_target": {
+      "zh": "审查目标",
+      "en": "Audit target"
+    }
+  }
+}