eduevidence 5.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +506 -0
- package/README.zh-CN.md +434 -0
- package/SKILL.md +195 -0
- package/bin/eduevidence.js +127 -0
- package/domains/education/manifest.json +183 -0
- package/domains/education/outcome_taxonomy.json +127 -0
- package/domains/manifest.json +26 -0
- package/domains/policy/frame.schema.json +234 -0
- package/domains/policy/manifest.json +10 -0
- package/domains/policy/methodology_checklist.json +109 -0
- package/domains/policy/outcome_taxonomy.json +53 -0
- package/domains/policy/references/causal-identification.md +45 -0
- package/domains/policy/references/cost-evidence.md +44 -0
- package/domains/policy/references/equity.md +42 -0
- package/domains/policy/references/evidence-hierarchy.md +41 -0
- package/domains/policy/references/implementation-evidence.md +47 -0
- package/eduevidence_cli.py +26 -0
- package/engine/__init__.py +11 -0
- package/engine/__pycache__/__init__.cpython-312.pyc +0 -0
- package/engine/__pycache__/analysis.cpython-312.pyc +0 -0
- package/engine/__pycache__/bias.cpython-312.pyc +0 -0
- package/engine/__pycache__/briefs.cpython-312.pyc +0 -0
- package/engine/__pycache__/capabilities.cpython-312.pyc +0 -0
- package/engine/__pycache__/citation_check.cpython-312.pyc +0 -0
- package/engine/__pycache__/contracts.cpython-312.pyc +0 -0
- package/engine/__pycache__/datasets.cpython-312.pyc +0 -0
- package/engine/__pycache__/events.cpython-312.pyc +0 -0
- package/engine/__pycache__/evidence_graph.cpython-312.pyc +0 -0
- package/engine/__pycache__/evidence_review.cpython-312.pyc +0 -0
- package/engine/__pycache__/evidencecore.cpython-312.pyc +0 -0
- package/engine/__pycache__/gap_lens.cpython-312.pyc +0 -0
- package/engine/__pycache__/gaps.cpython-312.pyc +0 -0
- package/engine/__pycache__/graph_store.cpython-312.pyc +0 -0
- package/engine/__pycache__/graph_validate.cpython-312.pyc +0 -0
- package/engine/__pycache__/ids.cpython-312.pyc +0 -0
- package/engine/__pycache__/library.cpython-312.pyc +0 -0
- package/engine/__pycache__/library_builtin.cpython-312.pyc +0 -0
- package/engine/__pycache__/living.cpython-312.pyc +0 -0
- package/engine/__pycache__/log.cpython-312.pyc +0 -0
- package/engine/__pycache__/meta_analysis.cpython-312.pyc +0 -0
- package/engine/__pycache__/meta_synthesis.cpython-312.pyc +0 -0
- package/engine/__pycache__/migration.cpython-312.pyc +0 -0
- package/engine/__pycache__/mode_router.cpython-312.pyc +0 -0
- package/engine/__pycache__/paths.cpython-312.pyc +0 -0
- package/engine/__pycache__/pilot.cpython-312.pyc +0 -0
- package/engine/__pycache__/planner.cpython-312.pyc +0 -0
- package/engine/__pycache__/project.cpython-312.pyc +0 -0
- package/engine/__pycache__/projections.cpython-312.pyc +0 -0
- package/engine/__pycache__/robustness.cpython-312.pyc +0 -0
- package/engine/__pycache__/run.cpython-312.pyc +0 -0
- package/engine/__pycache__/semantics.cpython-312.pyc +0 -0
- package/engine/__pycache__/study_design.cpython-312.pyc +0 -0
- package/engine/__pycache__/synthesis.cpython-312.pyc +0 -0
- package/engine/__pycache__/tribunal.cpython-312.pyc +0 -0
- package/engine/__pycache__/update.cpython-312.pyc +0 -0
- package/engine/__pycache__/versions.cpython-312.pyc +0 -0
- package/engine/analysis.py +308 -0
- package/engine/bias.py +178 -0
- package/engine/briefs.py +106 -0
- package/engine/capabilities.py +99 -0
- package/engine/citation_check.py +192 -0
- package/engine/contracts.py +117 -0
- package/engine/datasets.py +165 -0
- package/engine/events.py +67 -0
- package/engine/evidence_graph.py +571 -0
- package/engine/evidence_review.py +88 -0
- package/engine/evidencecore.py +182 -0
- package/engine/gap_lens.py +132 -0
- package/engine/gaps.py +169 -0
- package/engine/graph_store.py +335 -0
- package/engine/graph_validate.py +87 -0
- package/engine/ids.py +77 -0
- package/engine/library.py +268 -0
- package/engine/library_builtin.py +301 -0
- package/engine/living.py +671 -0
- package/engine/log.py +39 -0
- package/engine/meta_analysis.py +333 -0
- package/engine/meta_synthesis.py +111 -0
- package/engine/migration.py +397 -0
- package/engine/mode_router.py +72 -0
- package/engine/paths.py +15 -0
- package/engine/pilot.py +368 -0
- package/engine/planner.py +126 -0
- package/engine/project.py +118 -0
- package/engine/projections.py +240 -0
- package/engine/robustness.py +109 -0
- package/engine/run.py +85 -0
- package/engine/semantics.py +135 -0
- package/engine/study_design.py +87 -0
- package/engine/synthesis.py +187 -0
- package/engine/tribunal.py +408 -0
- package/engine/update.py +113 -0
- package/engine/versions.py +12 -0
- package/install.sh +510 -0
- package/integrations/__init__.py +1 -0
- package/integrations/__pycache__/__init__.cpython-312.pyc +0 -0
- package/integrations/__pycache__/agent_mcp.cpython-312.pyc +0 -0
- package/integrations/__pycache__/smart_web_fetch.cpython-312.pyc +0 -0
- package/integrations/agent_mcp.py +856 -0
- package/integrations/smart_web_fetch.py +59 -0
- package/package.json +50 -0
- package/pyproject.toml +55 -0
- package/references/applicability-policy.md +88 -0
- package/references/education-framing.md +132 -0
- package/references/effect_size_formulas.md +35 -0
- package/references/evaluation-design.md +111 -0
- package/references/evidence-quality.md +79 -0
- package/references/grade_framework.md +29 -0
- package/references/intervention-design.md +98 -0
- package/references/methodology-audit.md +103 -0
- package/references/outcome-taxonomy.md +106 -0
- package/references/retrieval-protocol.md +142 -0
- package/references/skeptic-protocol.md +93 -0
- package/references/social_science_pitfalls.md +48 -0
- package/references/source-validity.md +140 -0
- package/references/tribunal-policy.md +112 -0
- package/references/wwc_standards.md +29 -0
- package/retrieval/__init__.py +1 -0
- package/retrieval/__pycache__/__init__.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/corpus_store.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/dedupe.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/failures.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/fetch.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/search.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/source.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/validate.cpython-312.pyc +0 -0
- package/retrieval/corpus_store.py +181 -0
- package/retrieval/dedupe.py +127 -0
- package/retrieval/failures.py +90 -0
- package/retrieval/fetch.py +435 -0
- package/retrieval/search.py +493 -0
- package/retrieval/source.py +160 -0
- package/retrieval/validate.py +257 -0
- package/schemas/agent-mcp-approval.schema.json +57 -0
- package/schemas/chart-spec.schema.json +88 -0
- package/schemas/cross-model-review.schema.json +28 -0
- package/schemas/education-frame.schema.json +106 -0
- package/schemas/evaluation.schema.json +35 -0
- package/schemas/evidence.schema.json +81 -0
- package/schemas/fetch-result.schema.json +119 -0
- package/schemas/intervention.schema.json +46 -0
- package/schemas/methodology.schema.json +102 -0
- package/schemas/report-result.schema.json +381 -0
- package/schemas/report-spec.schema.json +130 -0
- package/schemas/source.schema.json +311 -0
- package/schemas/v2/analysis-plan.schema.json +28 -0
- package/schemas/v2/analysis-run.schema.json +33 -0
- package/schemas/v2/claim.schema.json +26 -0
- package/schemas/v2/dataset-asset.schema.json +40 -0
- package/schemas/v2/decision-snapshot.schema.json +53 -0
- package/schemas/v2/evidence-link.schema.json +38 -0
- package/schemas/v2/finding.schema.json +47 -0
- package/schemas/v2/graph-revision.schema.json +30 -0
- package/schemas/v2/knowledge-gap.schema.json +40 -0
- package/schemas/v2/methodology-audit.schema.json +30 -0
- package/schemas/v2/outcome.schema.json +18 -0
- package/schemas/v2/project.schema.json +31 -0
- package/schemas/v2/research-intent.schema.json +24 -0
- package/schemas/v2/run.schema.json +43 -0
- package/schemas/v2/source.schema.json +24 -0
- package/schemas/v2/study-design.schema.json +67 -0
- package/schemas/v2/study.schema.json +37 -0
- package/schemas/v3/pilot-outcome.schema.json +132 -0
- package/schemas/v3/run-manifest.schema.json +193 -0
- package/schemas/v3/synthesis.schema.json +86 -0
- package/schemas/v4/drift-report.schema.json +66 -0
- package/schemas/v4/evidence-library.schema.json +42 -0
- package/schemas/v4/living-subscription.schema.json +55 -0
- package/schemas/v4/meta-analysis.schema.json +152 -0
- package/schemas/verdict.schema.json +56 -0
- package/scripts/__init__.py +0 -0
- package/scripts/__pycache__/__init__.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark_evaluator.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark_judge.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark_routing.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark_v2.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark_v3.cpython-312.pyc +0 -0
- package/scripts/__pycache__/build_result.cpython-312.pyc +0 -0
- package/scripts/__pycache__/claim_audit.cpython-312.pyc +0 -0
- package/scripts/__pycache__/complexity_gate.cpython-312.pyc +0 -0
- package/scripts/__pycache__/compute_confidence.cpython-312.pyc +0 -0
- package/scripts/__pycache__/dashboard_server.cpython-312.pyc +0 -0
- package/scripts/__pycache__/did_regression.cpython-312.pyc +0 -0
- package/scripts/__pycache__/effect_calculator.cpython-312.pyc +0 -0
- package/scripts/__pycache__/evidence_matrix.cpython-312.pyc +0 -0
- package/scripts/__pycache__/evidence_score.cpython-312.pyc +0 -0
- package/scripts/__pycache__/evidence_semantics.cpython-312.pyc +0 -0
- package/scripts/__pycache__/fetch_benchmark.cpython-312.pyc +0 -0
- package/scripts/__pycache__/lint_report_layout.cpython-312.pyc +0 -0
- package/scripts/__pycache__/orchestrator.cpython-312.pyc +0 -0
- package/scripts/__pycache__/pre_verdict_gate.cpython-312.pyc +0 -0
- package/scripts/__pycache__/recompute_demo_quality.cpython-312.pyc +0 -0
- package/scripts/__pycache__/render_report.cpython-312.pyc +0 -0
- package/scripts/__pycache__/render_report_html.cpython-312.pyc +0 -0
- package/scripts/__pycache__/run_workspace.cpython-312.pyc +0 -0
- package/scripts/__pycache__/skill_lint.cpython-312.pyc +0 -0
- package/scripts/__pycache__/startup_probe.cpython-312.pyc +0 -0
- package/scripts/__pycache__/sync_killer_demo_report.cpython-312.pyc +0 -0
- package/scripts/__pycache__/test_adversarial_empirical.cpython-312-pytest-9.0.2.pyc +0 -0
- package/scripts/__pycache__/test_adversarial_empirical.cpython-312-pytest-9.1.1.pyc +0 -0
- package/scripts/__pycache__/validate_schema.cpython-312.pyc +0 -0
- package/scripts/audit_dois.py +292 -0
- package/scripts/bake_pack.sh +37 -0
- package/scripts/benchmark.py +183 -0
- package/scripts/benchmark_evaluator.py +371 -0
- package/scripts/benchmark_judge.py +535 -0
- package/scripts/benchmark_routing.py +120 -0
- package/scripts/benchmark_v2.py +304 -0
- package/scripts/benchmark_v3.py +552 -0
- package/scripts/build_esl_artifacts.py +1921 -0
- package/scripts/build_evidence_library.py +307 -0
- package/scripts/build_killer_demo.py +295 -0
- package/scripts/build_result.py +311 -0
- package/scripts/check_version_consistency.py +96 -0
- package/scripts/citation_check.py +123 -0
- package/scripts/claim_audit.py +157 -0
- package/scripts/complexity_gate.py +180 -0
- package/scripts/compute_confidence.py +176 -0
- package/scripts/dashboard_server.py +536 -0
- package/scripts/did_regression.py +315 -0
- package/scripts/effect_calculator.py +99 -0
- package/scripts/enrich_projects_human_and_lieflat.py +315 -0
- package/scripts/evidence_matrix.py +129 -0
- package/scripts/evidence_score.py +234 -0
- package/scripts/evidence_semantics.py +87 -0
- package/scripts/fetch_benchmark.py +177 -0
- package/scripts/generate_metrics.py +99 -0
- package/scripts/generate_new_projects.py +686 -0
- package/scripts/generate_promo.py +192 -0
- package/scripts/lint_report_layout.py +182 -0
- package/scripts/orchestrator.py +1456 -0
- package/scripts/pre_verdict_gate.py +513 -0
- package/scripts/quickstart.py +121 -0
- package/scripts/rebake_all_5themes.py +88 -0
- package/scripts/recompute_demo_quality.py +205 -0
- package/scripts/render_report.py +270 -0
- package/scripts/render_report_html.py +356 -0
- package/scripts/retraction_watch.py +110 -0
- package/scripts/run_workspace.py +337 -0
- package/scripts/serve_web.py +54 -0
- package/scripts/skill_lint.py +150 -0
- package/scripts/startup_probe.py +265 -0
- package/scripts/sync_killer_demo_report.py +270 -0
- package/scripts/test_adversarial_empirical.py +541 -0
- package/scripts/validate_schema.py +256 -0
- package/skill/agents/education-planner.md +80 -0
- package/skill/agents/evaluation-designer.md +74 -0
- package/skill/agents/evidence-analyst.md +106 -0
- package/skill/agents/evidence-judge.md +111 -0
- package/skill/agents/evidence-retriever.md +80 -0
- package/skill/agents/intervention-designer.md +82 -0
- package/skill/agents/method-reviewer.md +104 -0
- package/skill/agents/skeptic.md +89 -0
- package/skill/sub-skills/aihot-trend-analysis/SKILL.md +31 -0
- package/skill/sub-skills/contradiction-analysis/SKILL.md +17 -0
- package/skill/sub-skills/data-analysis/SKILL.md +23 -0
- package/skill/sub-skills/ethics-review/SKILL.md +25 -0
- package/skill/sub-skills/evidence-extraction/SKILL.md +19 -0
- package/skill/sub-skills/evidence-review/SKILL.md +18 -0
- package/skill/sub-skills/gap-analysis/SKILL.md +25 -0
- package/skill/sub-skills/literature-review/SKILL.md +21 -0
- package/skill/sub-skills/methodology-audit/SKILL.md +20 -0
- package/skill/sub-skills/report-generation/SKILL.md +51 -0
- package/skill/sub-skills/research-planning/SKILL.md +21 -0
- package/skill/sub-skills/study-design/SKILL.md +16 -0
- package/skill/task-briefs/adjudicate.md +17 -0
- package/skill/task-briefs/audit.md +15 -0
- package/skill/task-briefs/challenge.md +15 -0
- package/skill/task-briefs/evaluate.md +13 -0
- package/skill/task-briefs/extract.md +16 -0
- package/skill/task-briefs/frame.md +17 -0
- package/skill/task-briefs/intervene.md +14 -0
- package/skill/task-briefs/present.md +16 -0
- package/skill/task-briefs/retrieve.md +15 -0
- package/visualization/eduevidence-report/assets/base.css +337 -0
- package/visualization/eduevidence-report/motion/motion.css +157 -0
- package/visualization/eduevidence-report/motion/motion.js +107 -0
- package/visualization/eduevidence-report/references/bilingual-style.md +7 -0
- package/visualization/eduevidence-report/references/component-catalog.md +145 -0
- package/visualization/eduevidence-report/references/evidence-expansion.md +65 -0
- package/visualization/eduevidence-report/references/full-report-outline.md +86 -0
- package/visualization/eduevidence-report/references/layout-constraints.md +63 -0
- package/visualization/eduevidence-report/references/lieflat-composition.md +79 -0
- package/visualization/eduevidence-report/references/motion-system.md +31 -0
- package/visualization/eduevidence-report/schemas/adapter-envelope.schema.json +22 -0
- package/visualization/eduevidence-report/schemas/visual-layout.schema.json +87 -0
- package/visualization/eduevidence-report/scripts/__pycache__/adapter_contract.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/build_artifact_manifest.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/build_charts.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/build_figures.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/build_infographics.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/build_report.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/charts_data.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/lieflat_engine.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/zh_labels.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/adapter_contract.py +72 -0
- package/visualization/eduevidence-report/scripts/build_artifact_manifest.py +70 -0
- package/visualization/eduevidence-report/scripts/build_charts.py +283 -0
- package/visualization/eduevidence-report/scripts/build_figures.py +515 -0
- package/visualization/eduevidence-report/scripts/build_infographics.py +268 -0
- package/visualization/eduevidence-report/scripts/build_report.py +3211 -0
- package/visualization/eduevidence-report/scripts/charts_data.py +617 -0
- package/visualization/eduevidence-report/scripts/check_mobile_layout.js +220 -0
- package/visualization/eduevidence-report/scripts/lieflat_engine.py +852 -0
- package/visualization/eduevidence-report/scripts/zh_labels.py +245 -0
- package/visualization/eduevidence-report/themes/academic.css +94 -0
- package/visualization/eduevidence-report/themes/claude.css +96 -0
- package/visualization/eduevidence-report/themes/datalab-dark.css +147 -0
- package/visualization/eduevidence-report/themes/datalab.css +151 -0
- package/visualization/eduevidence-report/themes/presentation.css +140 -0
package/README.zh-CN.md
ADDED
|
@@ -0,0 +1,434 @@
|
|
|
1
|
+
# EduEvidence
|
|
2
|
+
|
|
3
|
+
> **🌐 [English](README.md) | 中文**
|
|
4
|
+
|
|
5
|
+
## EduEvidence Research Engine — Evidence-Based Education Decision Skill
|
|
6
|
+
|
|
7
|
+
> **From Education Questions to Evidence-Based Decisions.**
|
|
8
|
+
> **从教育问题,到有证据支撑的教育决策。**
|
|
9
|
+
|
|
10
|
+
EduEvidence 面向教师、教学研究者与教学管理者,把"是否采用某教学法、课程设计、评估方式或 AI 教学工具"从经验判断转化为**可追溯、可质疑、可验证的证据决策流程**——教学法、课程、评估、学习干预与 AI 工具均是应用域,引擎方法学教育通用。
|
|
11
|
+
|
|
12
|
+
- ⚖️ 不是替教师生成答案,而是帮助教师知道:证据支持什么、不能支持什么、适用于谁、应该怎样试点并验证。
|
|
13
|
+
- 🧪 基于真实研究(示例包含 CHI 2023 / PNAS 2025 / ACL 2025 / Springer 2024 的实证证据),不做无来源断言。
|
|
14
|
+
- 🚦 最终输出不是"允许/禁止"的二元结论,而是 **ADOPT / PILOT / REJECT / INSUFFICIENT EVIDENCE** 四态决策 + 可落地的教学干预与评价方案。
|
|
15
|
+
|
|
16
|
+

|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## 快速安装
|
|
21
|
+
|
|
22
|
+
**方式一:npm(推荐 — 安装为 Agent Skill)**
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
npm install -g eduevidence
|
|
26
|
+
eduevidence skill # 交互式选择宿主(默认)
|
|
27
|
+
eduevidence skill --list-hosts # 或先查看全部宿主与落点
|
|
28
|
+
# 非交互:eduevidence skill --host cursor
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
**方式二:curl 一键安装**
|
|
32
|
+
|
|
33
|
+
> ⚠️ 供应链提示(E7):curl 直跑等于执行远端脚本。更稳妥的方式是**方式二 clone 后
|
|
34
|
+
> 先审阅再安装**;若坚持直跑,建议把 URL 固定到具体 commit 并先下载审阅。
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
bash -c "$(curl -fsSL https://raw.githubusercontent.com/37chengshan/eduevidence/main/install.sh)"
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
脚本自动 clone 仓库到 `./eduevidence` 并完成安装(venv + 依赖 + 自检 + 测试)。
|
|
41
|
+
副作用披露:会在 `~/.eduevidence/env` 写入 `AGENT_MCP_INSTALLED=1` 声明;
|
|
42
|
+
`--skill` 模式覆盖已有 skill 前自动备份。
|
|
43
|
+
|
|
44
|
+
**方式三:git clone**
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
git clone https://github.com/37chengshan/eduevidence.git
|
|
48
|
+
cd eduevidence
|
|
49
|
+
bash install.sh # 一键:venv + 依赖 + 自检 + 测试
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
安装后直接打开示例报告:
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
open examples/ai-coding-assistant/EduEvidence_Report.html
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
> 需要 Python 3.10+;核心零第三方依赖。学术图 PNG/PDF 导出可选装 matplotlib。
|
|
59
|
+
> 安装完成后脚本会提示为项目点 star(仅提示,不会自动执行任何 GitHub 操作)。
|
|
60
|
+
> ⚠️ 管道执行远程脚本以当前用户权限运行,建议先审阅 [install.sh](install.sh) 或用 git clone。
|
|
61
|
+
|
|
62
|
+
---
|
|
63
|
+
|
|
64
|
+
## 安装为 Skill(AI Agent 用户)
|
|
65
|
+
|
|
66
|
+
> EduEvidence 本体是一个 **AI Agent Skill**(SKILL.md + skill/agents/ + references/ + schemas/ + scripts/ + retrieval/ + integrations/ + visualization/)。
|
|
67
|
+
> 安装后,你的宿主 Agent(Claude Code / OMP / Codex / OpenCode / Kimi / ZCode / OpenClaw / Harness / Grok / Copilot / Cline …)就能在收到教学决策类问题时自动装载本 Skill。
|
|
68
|
+
|
|
69
|
+
```bash
|
|
70
|
+
npm install -g eduevidence
|
|
71
|
+
eduevidence skill # 交互式选择宿主(默认)
|
|
72
|
+
eduevidence skill --list-hosts
|
|
73
|
+
eduevidence skill --dry-run
|
|
74
|
+
eduevidence skill --host cursor # 可选:跳过菜单,直接指定宿主
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
或从 git clone 目录:
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
bash install.sh --skill # 交互式选择安装到哪个 Agent
|
|
81
|
+
bash install.sh --list-hosts # 查看支持的 Agent 与 Skill 落点
|
|
82
|
+
bash install.sh --skill --host claude
|
|
83
|
+
bash install.sh --skill --dry-run # 只预览不写入
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
也可以不经 clone 直接远程执行(⚠️ 供应链提示见上:优先 clone + 审阅;直跑请固定 commit):
|
|
87
|
+
|
|
88
|
+
```bash
|
|
89
|
+
bash -c "$(curl -fsSL https://raw.githubusercontent.com/37chengshan/eduevidence/main/install.sh)"
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
安装前脚本会自动备份目标目录中已有的 skill(`cp -r` 到 `.bak-<时间戳>`);`--dry-run` 只预览不写入。
|
|
93
|
+
|
|
94
|
+
### 支持的 Agent 与配置落点
|
|
95
|
+
|
|
96
|
+
| Agent | 探测路径 | Skill 安装落点 |
|
|
97
|
+
|---|---|---|
|
|
98
|
+
| Cursor | `~/.cursor` | `~/.cursor/skills/eduevidence/` |
|
|
99
|
+
| Claude Code | `~/.claude` | `~/.claude/skills/eduevidence/`(无用户级配置则装到项目 `.claude/skills/`)|
|
|
100
|
+
| Codex | `~/.codex` 或 `codex` 命令 | `~/.agents/skills/`(兼容 `~/.codex/skills/`、`~/.codex/prompts/`)|
|
|
101
|
+
| OMP | `~/.omp` | `~/.omp/agent/skills/eduevidence/` |
|
|
102
|
+
| OpenCode | `~/.config/opencode` | `~/.config/opencode/skills/eduevidence/` |
|
|
103
|
+
| Kimi Code | `$KIMI_CODE_HOME` 或 `~/.kimi-code` | `~/.kimi-code/skills/eduevidence/` |
|
|
104
|
+
| ZCode | `~/.zcode` | `~/.zcode/skills/eduevidence/` |
|
|
105
|
+
| OpenClaw | `~/.openclaw` | `~/.openclaw/skills/eduevidence/` |
|
|
106
|
+
| Harness | `~/.harness` | `~/.harness/skills/eduevidence/` |
|
|
107
|
+
| Grok | `~/.grok` | `~/.grok/skills/eduevidence/` |
|
|
108
|
+
| GitHub Copilot CLI | `~/.copilot` | `~/.copilot/skills/eduevidence/` |
|
|
109
|
+
| Cline | `~/.cline` 或 `~/.config/cline` | `~/.cline/skills/eduevidence/` |
|
|
110
|
+
|
|
111
|
+
交互菜单中:选择 `all` 安装到全部 Agent;`custom` 手动指定 skill 目录;`local` 只装本地(venv + pytest + 自检)。
|
|
112
|
+
|
|
113
|
+
### 方式三:通用提示词(未列出的 Agent)
|
|
114
|
+
|
|
115
|
+
你的 Agent 不在列表里?把下面这段提示词**原样**交给任意支持 skill / 自定义指令的 AI,它会读取 [安装指南](docs/install-guide.md) 自行完成安装:
|
|
116
|
+
|
|
117
|
+
```text
|
|
118
|
+
请按照 https://github.com/37chengshan/eduevidence/blob/main/docs/install-guide.md
|
|
119
|
+
的安装指南,为我把 EduEvidence 安装为 skill:先读取该文档,按第 2 节的落点表
|
|
120
|
+
把 SKILL.md、skill/、references/、schemas/、scripts/、retrieval/、integrations/、
|
|
121
|
+
visualization/ 复制到我的 skill 目录(或按我的装载机制导入),然后按第 3 节
|
|
122
|
+
完成验证(SKILL.md 可读 + 脚本可运行 + 示例报告可渲染)。
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
## What Problem We Solve
|
|
126
|
+
|
|
127
|
+
普通 AI 面对教育问题通常执行:
|
|
128
|
+
|
|
129
|
+
```text
|
|
130
|
+
问题 → 搜索若干材料 → 总结观点 → 给出建议
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
EduEvidence 执行:
|
|
134
|
+
|
|
135
|
+
```text
|
|
136
|
+
教学问题
|
|
137
|
+
→ Education Research Framing(学习者/干预/对照/Outcome/场景)
|
|
138
|
+
→ 文献与证据检索(支持证据 + 独立反方证据)
|
|
139
|
+
→ Claim-Level Evidence Extraction
|
|
140
|
+
→ Skeptic 反证协议 + Method Reviewer 方法学审查
|
|
141
|
+
→ Evidence Tribunal(证据裁决)
|
|
142
|
+
→ Applicability Analysis(适用性)
|
|
143
|
+
→ Decision: ADOPT / PILOT / REJECT / INSUFFICIENT EVIDENCE
|
|
144
|
+
→ Teaching Intervention(最小可验证试点)
|
|
145
|
+
→ Evaluation Plan(效果评价)
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
最终回答六个问题:
|
|
149
|
+
|
|
150
|
+
1. 当前证据到底支持什么?
|
|
151
|
+
2. 当前证据不能支持什么?
|
|
152
|
+
3. 为什么不同研究会得到不同结果?
|
|
153
|
+
4. 对哪类学生、什么课程、什么条件适用?
|
|
154
|
+
5. 如果学校真的要用,怎样低风险落地?
|
|
155
|
+
6. 实施后如何验证它到底有没有效果?
|
|
156
|
+
|
|
157
|
+
## 30-second Demo
|
|
158
|
+
|
|
159
|
+
> 主 Demo:**大一 C 语言课程是否应该允许学生使用生成式 AI 编程助手?**
|
|
160
|
+
|
|
161
|
+
| 时间 | 阶段 |
|
|
162
|
+
|---|---|
|
|
163
|
+
| 0–20s | 输入教学问题 |
|
|
164
|
+
| 20–45s | Education Research Frame |
|
|
165
|
+
| 45–75s | Evidence Retrieval |
|
|
166
|
+
| 75–110s | Evidence Matrix |
|
|
167
|
+
| 110–135s | Methodology + Skeptic |
|
|
168
|
+
| 135–155s | Evidence Tribunal |
|
|
169
|
+
| 155–170s | Teaching Intervention + Evaluation |
|
|
170
|
+
| 170–180s | Benchmark |
|
|
171
|
+
|
|
172
|
+
完整示例包见 [`examples/ai-coding-assistant/`](examples/ai-coding-assistant/)。
|
|
173
|
+
|
|
174
|
+
## Why Education Evidence Is Hard
|
|
175
|
+
|
|
176
|
+
教育研究证据有几个天然陷阱,EduEvidence 的核心创新就是把应对这些陷阱的环节标准化:
|
|
177
|
+
|
|
178
|
+
- **Outcome Separation**:`代码完成更快 ≠ 真正学会编程`;`短期成绩提高 ≠ 长期保持提高`;`AI 协助完成任务 ≠ 无 AI 环境下能够迁移`。
|
|
179
|
+
- **Counter-Evidence Search**:不能只验证用户的最初假设,必须独立寻找 null / negative / contradictory 证据、AI dependency、novelty effect、self-selection bias 等。
|
|
180
|
+
- **Evidence Tribunal**:不是简单把正反论文列在一起,而是判断哪些研究更可信、冲突来自样本/测量/课程/工具还是实验设计、目前最多能得出什么结论。
|
|
181
|
+
- **Evidence-to-Action Bridge**:不能停在"研究显示……",必须连到适用性判断、教学决策、试点干预与评价设计。
|
|
182
|
+
|
|
183
|
+
## How EduEvidence Works
|
|
184
|
+
|
|
185
|
+
```text
|
|
186
|
+
┌─────────────────────────────────────┐
|
|
187
|
+
│ EduEvidence │
|
|
188
|
+
│ 教育领域知识 + 决策 + 干预 + 评价 │
|
|
189
|
+
└────────────────┬────────────────────┘
|
|
190
|
+
│
|
|
191
|
+
┌────────────────▼────────────────────┐
|
|
192
|
+
│ EvidenceFlow Protocol │
|
|
193
|
+
│ Frame / Retrieve / Extract / │
|
|
194
|
+
│ Challenge / Audit / Adjudicate │
|
|
195
|
+
└────────────────┬────────────────────┘
|
|
196
|
+
│
|
|
197
|
+
┌────────┴────────┐
|
|
198
|
+
▼ ▼
|
|
199
|
+
Platform Native Agent MCP
|
|
200
|
+
Execution Mode Enhanced Mode
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
完整工作流(9 步):
|
|
204
|
+
|
|
205
|
+
```text
|
|
206
|
+
1. Frame 构建 EducationResearchFrame
|
|
207
|
+
2. Retrieve 文献与证据检索(支持证据 + 独立反方证据)
|
|
208
|
+
3. Extract 抽取 Claim-Level Evidence(绑定 Outcome)
|
|
209
|
+
4. Challenge Skeptic 反证协议(固定 9 项检查)
|
|
210
|
+
5. Audit Method Reviewer 方法学审查(15 项清单)
|
|
211
|
+
6. Adjudicate Evidence Tribunal 证据裁决(Evidence Matrix + Verdict)
|
|
212
|
+
7. Applicability 适用性分析
|
|
213
|
+
8. Intervene Teaching Intervention 设计(最小可验证试点)
|
|
214
|
+
9. Evaluate Evaluation Plan 设计
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
每一步产出经过 JSON Schema 校验(`schemas/`),确定性逻辑由 `scripts/` 提供,教育方法论在 `references/` 中独立成文。
|
|
218
|
+
|
|
219
|
+
## Outcome Separation
|
|
220
|
+
|
|
221
|
+
EduEvidence 强制区分 20 类 Outcome(`references/outcome-taxonomy.md`):
|
|
222
|
+
|
|
223
|
+
```text
|
|
224
|
+
学习效果: Knowledge Gain / Concept Understanding / Retention / Transfer / Independent Problem Solving
|
|
225
|
+
任务表现: Completion Time / Accuracy / Code Quality / Assignment Score
|
|
226
|
+
学习过程: Engagement / Motivation / Cognitive Load / Help-Seeking / Metacognition
|
|
227
|
+
风险指标: AI Dependency / Over-reliance / Reduced Effort / Reduced Transfer / Academic Integrity Risk / False Confidence
|
|
228
|
+
```
|
|
229
|
+
|
|
230
|
+
主 Demo 的高光点正是这种区分:Kazemitabaar et al. (CHI 2023) 中 AI 代码助手使任务完成率提升 1.15×、得分提升 1.8×,但一周后的保持测试差异不显著——**任务表现 ≠ 学习效果**。
|
|
231
|
+
|
|
232
|
+
## Evidence Tribunal
|
|
233
|
+
|
|
234
|
+
`references/tribunal-policy.md` 定义了裁决规则:输入 Frame + Evidence Matrix + Skeptic Findings + Method Reviews,输出 EducationVerdict(`schemas/verdict.schema.json`),包括:
|
|
235
|
+
|
|
236
|
+
- supported / uncertain / contradicted claims
|
|
237
|
+
- 冲突来源分析(样本 / 测量 / 课程 / 工具 / 实验设计)
|
|
238
|
+
- Can Claim / Cannot Claim 边界
|
|
239
|
+
- 四态决策 + Confidence(规则化计算,不由模型自由生成)
|
|
240
|
+
|
|
241
|
+

|
|
242
|
+
|
|
243
|
+
## From Evidence to Action
|
|
244
|
+
|
|
245
|
+
证据必须连接到真实教学现场(`references/applicability-policy.md`、`intervention-design.md`、`evaluation-design.md`):
|
|
246
|
+
|
|
247
|
+
- **Applicability**:For whom? For which course? For which outcome? Under what conditions? With what AI usage policy?
|
|
248
|
+
- **Intervention**:永远是"最小可验证试点",禁止直接全面部署;含 AI 使用规则、教师/学生角色、反思要求、停止条件。
|
|
249
|
+
- **Evaluation**:任何 PILOT/ADOPT 建议必须附评价方案;区分基线/后测/保持测试/迁移测试;区分任务表现指标与学习指标。
|
|
250
|
+
|
|
251
|
+
## Benchmark
|
|
252
|
+
|
|
253
|
+
第一版 30 个教育研究问题(`benchmarks/questions.jsonl`),S×10 / M×10 / L×10;其中 15 题为主域"高校 AI 辅助教学",30 题全部含人工金标注(`benchmarks/annotations/gold-Q01..Q30`)。
|
|
254
|
+
|
|
255
|
+
基线设计:
|
|
256
|
+
|
|
257
|
+
```text
|
|
258
|
+
B0 Direct LLM
|
|
259
|
+
B1 Search + LLM
|
|
260
|
+
B2 Standard Research Agent
|
|
261
|
+
B3 EduEvidence Single-Agent ← 证明教育方法论价值(B2 vs B3)
|
|
262
|
+
B4 EduEvidence + Agent MCP ← 证明多 Agent 增强价值(B3 vs B4)
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
核心指标:Citation Support Precision / Unsupported Claim Rate / Contradiction Discovery Rate / Outcome Separation Accuracy / Scope Calibration / Intervention Evidence Alignment。详见 `docs/benchmark.md`。
|
|
266
|
+
|
|
267
|
+
> ⚠️ `benchmarks/results/` 仍为 **harness validation(deterministic simulation,标注 SIMULATED)**,仅证明评测框架可运行,不是真实模型性能。**首轮 Layer B 实证运行已启动**(B2 vs B3,10 题 × 3 次,`omp` 驱动 `deepseek-v4-flash`——见 `benchmarks/empirical/run-empirical-01`,报告见 `benchmarks/empirical/v3-report.md`)。指标为 gold-based 启发式(`method: heuristic`),结果仍受模型与题目集限制,运行结果复核前不作定论性效果宣称。
|
|
268
|
+
|
|
269
|
+
## Example: AI Coding Assistant
|
|
270
|
+
|
|
271
|
+
> **大学一年级 C 语言课程是否应该允许学生使用生成式 AI 编程助手?**
|
|
272
|
+
|
|
273
|
+
`examples/ai-coding-assistant/` 完整展示了从问题到决策的全过程:
|
|
274
|
+
|
|
275
|
+
- **证据**(7 条,均绑定真实来源):任务表现提升(Kazemitabaar 2023)、无护栏访问损害独立考试表现 -17%(Bastani 2025, PNAS)、护栏设计消除负效应(Bastani 2025)、形成性反馈写作证据(Marzuki 2024)。
|
|
276
|
+
- **决策**:**PILOT** —— 任务表现证据强,但大学编程课程的直接学习效应证据缺失,无护栏风险已被证实。
|
|
277
|
+
- **干预**:4 阶段试点(Independent Foundation → Explain Don't Solve → Structured Collaboration → Transfer Check)。
|
|
278
|
+
- **评价**:无 AI 基线/后测/期末考试保持/无 AI 迁移任务 + AI 依赖风险指标。
|
|
279
|
+
|
|
280
|
+
另外两个示例:AI 写作助手(`examples/ai-writing-assistant/`)、高数 AI Tutor(`examples/ai-tutor/`)——证明 Skill 不是为一个问题写死。
|
|
281
|
+
|
|
282
|
+
## Visualization: Bilingual HTML Report + Infographics + Academic Figures
|
|
283
|
+
|
|
284
|
+
After research completes, `result.json` is rendered by three deterministic Python adapters. The adapters use the standard library; the optional Web Studio chart enhancement has a separate browser dependency.
|
|
285
|
+
|
|
286
|
+
```text
|
|
287
|
+
result.json + result.zh.json
|
|
288
|
+
├─ build_charts.py → chart_specs.json (ECharts option data; no ECharts runtime bundled)
|
|
289
|
+
├─ build_infographics.py → infographics.json (hand-authored SVGs)
|
|
290
|
+
├─ build_figures.py → figures/ (publication figures: figure_data.json + SVG/PNG/PDF)
|
|
291
|
+
└─ build_report.py → EduEvidence_Report.html (single-file bilingual report + report_spec.json)
|
|
292
|
+
```
|
|
293
|
+
|
|
294
|
+
**EduEvidence_Report.html (main deliverable)**:
|
|
295
|
+
|
|
296
|
+
- **Bilingual switch**: Chinese by default, one click to EN; data remains isomorphic.
|
|
297
|
+
- **Executive summary narrative**: question → evidence → action, with traceable source sections.
|
|
298
|
+
- **Two-page layout**: Visual Brief + Full Report (AI-planned 5–7 dynamic chapters, not a fixed template).
|
|
299
|
+
- **Five styles (chosen at generation time, no in-HTML switcher)**: claude / academic / datalab / datalab-dark / presentation.
|
|
300
|
+
- **Static-first**: decision, matrix, tribunal, intervention and sources remain readable without JavaScript; ECharts is an optional enhancement.
|
|
301
|
+
- **Integrity gate**: chart numbers are checked against result.json item by item; publishing is blocked with `REPORT_INVALID` on mismatch.
|
|
302
|
+
|
|
303
|
+
**Local Web Studio** (`python3 scripts/dashboard_server.py --port 8765`) has exactly three read-only views: Dashboard, Report Browser and Data Visualization. It loads ECharts 5.4.3 from jsDelivr for interactive charts; the submission package does not bundle that runtime, so Web interactivity requires network access. The baked report's static HTML/SVG remains the offline artifact.
|
|
304
|
+
|
|
305
|
+
> Open the example directly: `examples/ai-coding-assistant/EduEvidence_Report.html`
|
|
306
|
+
|
|
307
|
+
## Architecture
|
|
308
|
+
|
|
309
|
+
仓库是一个完整的 **Skill 包**:`SKILL.md` 是入口,其余目录按"Skill 运行必需 → 质量保障 → 演示"分层。详见 [`docs/architecture.md`](docs/architecture.md):
|
|
310
|
+
|
|
311
|
+
```text
|
|
312
|
+
EduEvidence/ (= 一个 Skill 包)
|
|
313
|
+
│
|
|
314
|
+
├─ SKILL.md ← Skill 入口:When to Use / Inputs / 9 步 Workflow / 输出契约
|
|
315
|
+
│
|
|
316
|
+
├─ Skill 本体(运行必需)
|
|
317
|
+
│ ├─ skill/agents/ 8 个角色协议(Planner / Retriever / Analyst / Skeptic /
|
|
318
|
+
│ │ Method Reviewer / Judge / Intervention Designer / Evaluation Designer)
|
|
319
|
+
│ ├─ references/ 11 个教育方法论文档(证据质量 / 反证协议 / 裁决规则 / 干预设计…)
|
|
320
|
+
│ ├─ schemas/ 33 个 JSON Schema 数据契约(13 顶层 + 17 v2 + 3 v3,每步输出的校验门)
|
|
321
|
+
│ ├─ scripts/ 17 个确定性逻辑脚本(评分 / 矩阵 / 审计 / 置信度 / Orchestrator / 启动探测)
|
|
322
|
+
│ ├─ retrieval/ 检索与抓取层(fetch / validate / dedupe / failures)
|
|
323
|
+
│ ├─ integrations/ Agent MCP 增强层 + Smart Web Fetch 集成
|
|
324
|
+
│ └─ visualization/ 结果呈现层(ECharts / 信息图 / 学术图 / 双语 HTML Composer)
|
|
325
|
+
│
|
|
326
|
+
├─ 质量保障
|
|
327
|
+
│ ├─ tests/ pytest 测试矩阵(752 个测试函数,73 个文件,见 docs/metrics.json)
|
|
328
|
+
│ └─ benchmarks/ 30 题 + 30 份金标注 + B0–B4 评测框架
|
|
329
|
+
│
|
|
330
|
+
└─ 演示与分发
|
|
331
|
+
├─ examples/ 3 个完整 Research & Decision Pack(含双语 HTML 报告)
|
|
332
|
+
├─ docs/ 架构 / 方法论 / Benchmark / Demo / 复现指南
|
|
333
|
+
├─ install.sh 一键安装(本地 / 多 Agent Skill)+ 自检
|
|
334
|
+
├─ pyproject.toml 打包元数据(核心零第三方依赖)
|
|
335
|
+
└─ README(.en).md 双语说明
|
|
336
|
+
```
|
|
337
|
+
|
|
338
|
+
> Skill 包设计原则:**运行所需的最小集是 `SKILL.md + skill/ + references/ + schemas/ + scripts/`**;`retrieval/`、`integrations/`、`visualization/` 是让 Skill 真正"可运行、可呈现"的执行层;`tests/`、`benchmarks/`、`examples/`、`docs/` 是可信度与上手保障,不影响 Skill 本体。
|
|
339
|
+
|
|
340
|
+
### SCP / Platform Native Mode
|
|
341
|
+
|
|
342
|
+
EduEvidence 可完全脱离 Agent MCP 独立运行(无需任何外部服务):
|
|
343
|
+
|
|
344
|
+
- 不依赖本地 daemon
|
|
345
|
+
- 不依赖某一个 CLI
|
|
346
|
+
- 不依赖 Agent MCP
|
|
347
|
+
- SKILL.md 可单独理解,核心工作流可完整执行
|
|
348
|
+
- 所有 Schema / 方法 / 输出契约独立存在
|
|
349
|
+
|
|
350
|
+
### Agent MCP Enhanced Mode
|
|
351
|
+
|
|
352
|
+
Agent MCP 是**性能与可靠性增强层,不是 EduEvidence 成立的前提**(`docs/methodology.md` 的 Complexity Gate):
|
|
353
|
+
|
|
354
|
+
- S 级任务:单 Agent 直接执行,0 spawn
|
|
355
|
+
- M 级任务:Primary Analysis + Independent Check
|
|
356
|
+
- L 级任务:8 角色工作流(Planner / Retriever / Analyst / Skeptic / Method Reviewer / Judge / Intervention Designer / Evaluation Designer)
|
|
357
|
+
|
|
358
|
+
> 角色数量 ≠ 必须启动的 Agent 数量。Platform Native Mode 由单 Agent 串行执行角色协议。
|
|
359
|
+
|
|
360
|
+
> 🔒 Agent MCP 原则:**Scan first. Recommend second. Ask the user. Execute only after explicit confirmation.** 未经用户确认不得 spawn;用户拒绝则回退 Native。
|
|
361
|
+
|
|
362
|
+

|
|
363
|
+
|
|
364
|
+
## Usage
|
|
365
|
+
|
|
366
|
+
```bash
|
|
367
|
+
# 1. 验证数据符合 Schema 契约
|
|
368
|
+
python3 scripts/validate_schema.py --schema schemas/evidence.schema.json \
|
|
369
|
+
--data examples/ai-coding-assistant/evidence.jsonl
|
|
370
|
+
|
|
371
|
+
# 2. 计算证据质量分与 Confidence
|
|
372
|
+
python3 scripts/evidence_score.py examples/ai-coding-assistant/evidence.jsonl
|
|
373
|
+
|
|
374
|
+
# 3. 生成 Evidence Matrix(主产品界面之一)
|
|
375
|
+
python3 scripts/evidence_matrix.py examples/ai-coding-assistant/evidence.jsonl
|
|
376
|
+
|
|
377
|
+
# 4. 运行 Citation Audit(Claim-证据追溯)
|
|
378
|
+
python3 scripts/claim_audit.py --claims claims.jsonl --evidence evidence.jsonl
|
|
379
|
+
|
|
380
|
+
# 5. 渲染 Research & Decision Pack(Markdown)
|
|
381
|
+
python3 scripts/render_report.py \
|
|
382
|
+
--frame examples/ai-coding-assistant/frame.json \
|
|
383
|
+
--evidence examples/ai-coding-assistant/evidence.jsonl \
|
|
384
|
+
--methodology examples/ai-coding-assistant/methodology.json \
|
|
385
|
+
--verdict examples/ai-coding-assistant/verdict.json \
|
|
386
|
+
--intervention examples/ai-coding-assistant/intervention.json \
|
|
387
|
+
--evaluation examples/ai-coding-assistant/evaluation.json \
|
|
388
|
+
--out REPORT.md
|
|
389
|
+
|
|
390
|
+
# 6. 渲染单文件双语 HTML 报告(主产物)
|
|
391
|
+
python3 visualization/eduevidence-report/scripts/build_report.py \
|
|
392
|
+
--result examples/ai-coding-assistant/result.json \
|
|
393
|
+
--out examples/ai-coding-assistant/EduEvidence_Report.html
|
|
394
|
+
|
|
395
|
+
# 7. 校验 Benchmark 题目集
|
|
396
|
+
python3 scripts/benchmark.py --questions benchmarks/questions.jsonl
|
|
397
|
+
|
|
398
|
+
# 8. 运行测试
|
|
399
|
+
pytest
|
|
400
|
+
```
|
|
401
|
+
|
|
402
|
+
> 真实使用中,Skill 由 Agent 读取 SKILL.md 执行 9 步工作流;`scripts/` 保证结构化数据的确定性校验,`visualization/` 保证展示层的确定性渲染,`examples/` 是完整运行示例。
|
|
403
|
+
|
|
404
|
+
## Methodology
|
|
405
|
+
|
|
406
|
+
- 教育证据质量框架:五维 0–2 分(D1 研究设计 / D2 样本质量 / D3 测量效度 / D4 时间强度 / D5 直接性),总分 0–10(`references/evidence-quality.md`)。
|
|
407
|
+
- 方法学审查 15 项清单,最高优先级规则:**任务完成表现不能自动等价为学习效果**(`references/methodology-audit.md`)。
|
|
408
|
+
- Confidence 规则化计算:`Evidence Quality + Consistency + Directness + Evidence Count - Conflict Penalty - Unsupported Penalty` → High / Moderate / Low / Insufficient(`scripts/evidence_score.py`)。
|
|
409
|
+
- 失败处理:INSUFFICIENT_SOURCES / UNSUPPORTED_CLAIM / CONFLICT_UNRESOLVED / SCOPE_MISMATCH / METHODOLOGY_TOO_WEAK / NEEDS_USER_CONTEXT / TOOL_FAILURE —— 失败时禁止强行生成高确定性建议。
|
|
410
|
+
|
|
411
|
+
## Limitations
|
|
412
|
+
|
|
413
|
+
- Benchmark 基于真实文献的可检索证据;模型实际运行结果需按 `docs/benchmark.md` 的 B0–B4 基线采集。
|
|
414
|
+
- 搜索与抽取依赖可用检索资源;`TOOL_FAILURE` 时不编造来源。
|
|
415
|
+
- EduEvidence 是教学决策辅助,**不代替教师或学校最终决策**;涉及高风险评价、学生处分、个体心理判断、学生重大教育机会时不自动决策。
|
|
416
|
+
|
|
417
|
+
## Roadmap
|
|
418
|
+
|
|
419
|
+
**已完成(仅 harness / 仿真,标注 SIMULATED,非实证):**
|
|
420
|
+
|
|
421
|
+
- [x] Benchmark v2 harness / simulation —— `benchmarks/results/` 是确定性仿真,证明评测框架可运行,**不是**真实模型性能(见上方 [Benchmark](#benchmark) 的 ⚠️ 说明)。
|
|
422
|
+
- [x] Skill 核心与管线:9 步协议(Research Core 6 + Decision Extension 3)、13 个顶层 JSON Schema、确定性脚本、8 角色协议(原计划 Phase 0–6)。
|
|
423
|
+
- [x] Evidence-to-Action:适用性 / 四态决策 / 干预 / 评价设计。
|
|
424
|
+
- [x] 产品 UI:单文件双语 HTML 报告 + 信息图 + 学术图(原计划 Phase 8)。
|
|
425
|
+
|
|
426
|
+
**待做(尚未完成,不做已完成宣称):**
|
|
427
|
+
|
|
428
|
+
- [ ] **Empirical Benchmark** —— 首轮已启动(B2 vs B3,10 题 × 3 次,`omp` / `deepseek-v4-flash`,→ `benchmarks/empirical/v3-report.md`);30 题全覆盖、**B3 vs B4**、重复运行、方差报告与 gold annotation / 独立评审计分仍在计划中(见 `docs/benchmark.md`)。
|
|
429
|
+
- [ ] **HTML 可访问性** —— 语言切换同步 `<html lang>`、theme/lang 按钮 `aria-pressed`、表格筛选控件 label、SVG 图双语 title/desc、source link 安全 scheme 白名单。
|
|
430
|
+
- [ ] **纵向决策闭环** —— 打通 **PILOT → 真实数据 → 再裁决**:试点结果回填项目证据图,产出更新后的决策(Full Research Cycle 端到端真实数据跑通)。
|
|
431
|
+
|
|
432
|
+
## License
|
|
433
|
+
|
|
434
|
+
MIT — 见 [LICENSE](LICENSE)。
|
package/SKILL.md
ADDED
|
@@ -0,0 +1,195 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: eduevidence
|
|
3
|
+
description: "Evidence-based decisions for ANY educational & social science question — whether, when, and how to adopt a teaching method, curriculum change, AI tool, or policy intervention. Orchestrates modular sub-skills for literature review, methodology bias audit (WWC 5.0 / GRADE), SSOT causal evidence graph construction, gap lens discovery, pre-registered trial design, and empirical DID data re-injection."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# EduEvidence 5.2 — Universal Evidence-Based Decision Skill
|
|
7
|
+
|
|
8
|
+
> **AI4SS Track | 科艺融合 · 通用智能**
|
|
9
|
+
> **From Empirical Questions to Evidence-Based Decisions & Causal Trial Loops.**
|
|
10
|
+
|
|
11
|
+
EduEvidence delivers the **EduEvidence Research Engine** — a persistent system transforming research questions into verifiable decisions. State lives in a unified **Project Workspace** (Project / Run / Revision / DecisionSnapshot) and a **Single Source of Truth (SSOT) Evidence Graph** (engine/evidence_graph.py), linking Papers, Quantitative Effect Sizes (g), Claims, Social Science Risks, and Shared Research Library knowledge.
|
|
12
|
+
|
|
13
|
+
The canonical protocol below is the single authoritative definition (docs/architecture.md); every stage output is schema-gated (schemas/, 37 contracts across V1–V4 generations — counts are machine-audited in docs/metrics.json).
|
|
14
|
+
|
|
15
|
+
**Schema 版本口径**:schemas/ 顶层 13 个 = V1 契约(evidence.schema.json 当前修订 1.1、education-frame / verdict 等);schemas/v2/ 17 个 = V2 契约(evidence-link / research-intent / study / graph-revision / project 等)。文档与代理配置一律以此口径命名。
|
|
16
|
+
|
|
17
|
+
## 🧭 双研究模式(Research Mode)与执行层
|
|
18
|
+
|
|
19
|
+
- **Evidence Review**(二手证据综述)与 **Full Research Cycle**(综述 → 知识缺口 → 研究设计 → 本地数据 → 分析 → 图更新 → 再裁决)两种 Research Mode。
|
|
20
|
+
- **Project Workspace + Evidence Graph** 为不可变 revision 模型;result.json / HTML / Markdown 都是投影,不是事实库。
|
|
21
|
+
- **Shared Research Library**:已验证外部事实(Source/Study/Finding/Audit)跨项目快照复用;研究事实可复用,解释(Claim/EvidenceLink/Applicability/Decision)必须项目本地。
|
|
22
|
+
- 冻结科学规则:**No new study design without evidence grounding** — 任何新研究设计必须引用显式、有证据奠基的 KnowledgeGap ID。
|
|
23
|
+
- 唯一事实来源与防重规则:**Single Canonical Project per Research Question** — 同一研究问题严格保持单实例,禁止重复建立同名主题目录;更新时采用不可变 Revision 升级机制。
|
|
24
|
+
- **执行层双模式**:Mode A **Platform Native**(纯 SKILL 零依赖交付)/ Mode B **Agent MCP** Enhanced(可选增强:启动检测 → 推荐 → 用户授权 → safe_spawn;未启用自动降级 Native)。
|
|
25
|
+
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
## 📜 Canonical Protocol — 9 Steps = Research Core 6 + Decision Extension 3
|
|
29
|
+
|
|
30
|
+
```text
|
|
31
|
+
Research Core(6 阶段,证据纪律核心):
|
|
32
|
+
Frame → Retrieve → Extract → Challenge → Audit → Adjudicate
|
|
33
|
+
|
|
34
|
+
Decision Extension(3 阶段,证据到行动):
|
|
35
|
+
Applicability → Intervene → Evaluate
|
|
36
|
+
|
|
37
|
+
端到端 9 步:
|
|
38
|
+
Frame → Retrieve → Extract → Challenge → Audit → Adjudicate
|
|
39
|
+
→ Applicability → Intervene → Evaluate
|
|
40
|
+
|
|
41
|
+
Fetch / Validate = Retrieve 内部强制 gate(RULE 2:snippet ≠ 证据内容)
|
|
42
|
+
Present = 最终呈现层(不计入 9 步协议)
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
| # | 阶段 | 输出契约(schemas/) |
|
|
46
|
+
|---|------|----------------------|
|
|
47
|
+
| 1 | Frame | education-frame.schema.json |
|
|
48
|
+
| 2 | Retrieve(内含 Fetch/Validate gate) | source.schema.json / fetch-result.schema.json |
|
|
49
|
+
| 3 | Extract | evidence.schema.json |
|
|
50
|
+
| 4 | Challenge | evidence.schema.json(反方/未发现/confounder) |
|
|
51
|
+
| 5 | Audit | methodology.schema.json(task_vs_learning_guard) |
|
|
52
|
+
| 6 | Adjudicate | verdict.schema.json(规则化置信度 + Pre-Verdict Gate) |
|
|
53
|
+
| 7 | Applicability | applicability-policy.md(For whom / which course / which outcome / conditions) |
|
|
54
|
+
| 8 | Intervene | intervention.schema.json(最小可验证 PILOT + Stop Conditions) |
|
|
55
|
+
| 9 | Evaluate | evaluation.schema.json(基/后/保持/迁移 + 成功阈值) |
|
|
56
|
+
| Present | 呈现层 | result.json / result.zh.json → 烘焙主题报告 |
|
|
57
|
+
|
|
58
|
+
**Present 主题显示名(生成时五选一,烘焙定主题,最终 HTML 仅中英文切换)**:
|
|
59
|
+
|
|
60
|
+
```text
|
|
61
|
+
├─ Claude Research [Light]
|
|
62
|
+
├─ Academic Paper [Light]
|
|
63
|
+
├─ DataLab [Light]
|
|
64
|
+
├─ DataLab [Dark]
|
|
65
|
+
└─ Presentation / Judge [Dark]
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
---
|
|
69
|
+
|
|
70
|
+
## 🗺️ Sub-Skill Map(12 个子 skill 归位)
|
|
71
|
+
|
|
72
|
+
| 阶段 | 实现组件 |
|
|
73
|
+
|------|----------|
|
|
74
|
+
| 1 Frame | skill/sub-skills/research-planning + Complexity Gate(S/M/L,scripts/complexity_gate.py) |
|
|
75
|
+
| 2 Retrieve | skill/sub-skills/literature-review + skill/sub-skills/aihot-trend-analysis(retrieval/fetch.py + validate.py 为内部 gate) |
|
|
76
|
+
| 3 Extract | skill/sub-skills/evidence-extraction |
|
|
77
|
+
| 4 Challenge | skill/sub-skills/contradiction-analysis |
|
|
78
|
+
| 5 Audit | skill/sub-skills/methodology-audit(WWC 5.0 / GRADE) |
|
|
79
|
+
| 6 Adjudicate | skill/sub-skills/evidence-review + scripts/pre_verdict_gate.py + scripts/compute_confidence.py(meta-analysis:DerSimonian-Laird、Q、I²、Egger) |
|
|
80
|
+
| 7 Applicability | references/applicability-policy.md |
|
|
81
|
+
| 8 Intervene | references/intervention-design.md + skill/agents/intervention-designer.md |
|
|
82
|
+
| 9 Evaluate | references/evaluation-design.md + skill/agents/evaluation-designer.md |
|
|
83
|
+
| Present | skill/sub-skills/report-generation + visualization/eduevidence-report(静态报告与三适配器;Lieflat 仅作开发期图表参考,不是提交包运行时) |
|
|
84
|
+
| Full Research Cycle 扩展 | skill/sub-skills/gap-analysis → skill/sub-skills/study-design → skill/sub-skills/data-analysis(DID 回注) |
|
|
85
|
+
| 横切 | skill/sub-skills/ethics-review(IRB 合规) |
|
|
86
|
+
|
|
87
|
+
**Full Research Cycle = Decision-to-Outcome Loop**:证据综述 → 知识缺口(Gap)→ 预注册研究设计 → 本地课堂/田野数据 → DID/OLS 分析 → Evidence Graph 更新 → 再裁决。Grounding Rule:不基于证据奠基的 KnowledgeGap ID,禁止产生新研究设计。
|
|
88
|
+
|
|
89
|
+
**Complexity Gate(S/M/L)**:S(Quick Fact)快速路径;M(Standard Review)八角色全走;L(Deep Causal Cycle)标准路径 + 系统化搜索 + 独立双审 + 干预/评价强制产出。默认门控 M,向上从严。
|
|
90
|
+
|
|
91
|
+
---
|
|
92
|
+
|
|
93
|
+
## 🎯 Flagship Scenario: AI Coding Assistants in CS1
|
|
94
|
+
|
|
95
|
+
**Question**: *Should first-year university C/Python students be allowed to use generative AI coding assistants?*
|
|
96
|
+
|
|
97
|
+
`examples/ai-coding-assistant-evidence/` — every source is a real study with a
|
|
98
|
+
registry-verified DOI (Crossref/DataCite audit trail: `benchmarks/doi-audit/report.md`):
|
|
99
|
+
|
|
100
|
+
- **Real evidence base (8 sources)**: Bastani et al. PNAS 2025 field RCT,
|
|
101
|
+
Kazemitabaar et al. CHI 2023 novice RCT, Peng et al. 2023 professional-developer RCT
|
|
102
|
+
(preprint), plus benchmark/usability studies (JSS 2023, ICER 2022, UIST 2022, Koli 2023).
|
|
103
|
+
- **Honest synthesis**: task performance improves reliably during AI access; unguarded
|
|
104
|
+
access harmed independent exam performance by −17% (PNAS 2025); guardrail design
|
|
105
|
+
(hints-not-answers) mitigates the harm.
|
|
106
|
+
- **Tribunal Verdict**: **PILOT** (restricted 4-phase fading pilot) with engine-computed
|
|
107
|
+
confidence Moderate (0.586, policy 2026-08-12.v2) over 8 independent studies.
|
|
108
|
+
- **Provenance badge**: data_origin=`manual_curated`. Other demo packs are badged
|
|
109
|
+
`synthetic` and must not be cited as evidence.
|
|
110
|
+
- **Causal Closed Loop**: 12-week quasi-experimental DID trial design + Classroom CSV data injection.
|
|
111
|
+
|
|
112
|
+
---
|
|
113
|
+
|
|
114
|
+
## 🖥️ Local Web Studio(3 页,无 Agent 派遣)
|
|
115
|
+
|
|
116
|
+
```bash
|
|
117
|
+
python3 scripts/dashboard_server.py --port 8765
|
|
118
|
+
# http://127.0.0.1:8765/
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
静态前端(web/)+ 轻量 API 服务,直接读取 examples/<project_id>/ 下的 skill 工件:
|
|
122
|
+
|
|
123
|
+
1. **仪表盘 (Dashboard)** — 跨课题 KPI、效应量对比、课题资产矩阵。
|
|
124
|
+
2. **报告浏览 (Report Browser)** — 列出有 EduEvidence_Report.html 的课题;若存在 reports-5themes/EduEvidence_Report_<theme>.html 则提供 5 主题烘焙变体选择。报告仅在生成时定主题,最终 HTML 只保留中英文切换,不做运行时换肤。
|
|
125
|
+
3. **数据可视化 (Data Visualization)** — 按课题选择,查看森林图、效应量分布、结果维度与 SSOT 证据图谱。
|
|
126
|
+
|
|
127
|
+
页面是 **契约驱动**:新课题只要产出 result.json / evidence_graph.json 即自动出现,无硬编码 demo 数据。
|
|
128
|
+
|
|
129
|
+
---
|
|
130
|
+
|
|
131
|
+
## 📦 Visualization Data Contract(Skill ↔ Web 同步)
|
|
132
|
+
|
|
133
|
+
| 工件 | 产出阶段 | 消费页面 |
|
|
134
|
+
| --- | --- | --- |
|
|
135
|
+
| result.json → forest_plot_data | Extract + Adjudicate | 森林图 |
|
|
136
|
+
| result.json → evidence(数值 effect_size) | Extract | 效应量分布 |
|
|
137
|
+
| result.json → outcome_mapping | Extract + Audit | 结果维度 |
|
|
138
|
+
| evidence_graph.json → export_echarts_graph() | Adjudicate(evidence-review) | SSOT 证据图谱 |
|
|
139
|
+
| EduEvidence_Report.html + reports-5themes/*.html | Present(visualization/eduevidence-report/scripts/build_report.py) | 报告浏览(烘焙变体) |
|
|
140
|
+
| result.json → meta.question + decision | Present | 仪表盘 / 可视化标题与裁决 |
|
|
141
|
+
|
|
142
|
+
关键 forest_plot_data 字段:study_label、outcome_dimension、effect_size(Hedges g)、ci_lower/ci_upper、sample_size、direction、wwc_rating。
|
|
143
|
+
|
|
144
|
+
---
|
|
145
|
+
|
|
146
|
+
## 🛡️ 科学可信度强制(失败关闭)
|
|
147
|
+
|
|
148
|
+
- **DID 失败关闭**:`scripts/did_regression.py` 对奇异/共线设计、空 2×2 格、零方差、
|
|
149
|
+
饱和模型等不可估计输入返回 `status="error"` + 稳定 `error_code`,且
|
|
150
|
+
`did_coefficient` / `standard_error` / `p_value` / `ci_95` / `hedges_g` 全部为
|
|
151
|
+
`null`,绝不伪造 `SE=1.0` 或虚假 p 值。
|
|
152
|
+
- **非 cluster 显式标注**:普通 DID 结果带 `inference_status="non_cluster_warning"`;
|
|
153
|
+
DID/准实验(QED)永远不得标注 `Meets Standards Without Reservations`。
|
|
154
|
+
- **无直接学习证据不得 ADOPT**:裁决要求 learning/独立迁移结果(直接性 directness=2)
|
|
155
|
+
才允许 High+支持 → ADOPT;任务表现、程序效率、主观体验只能 PILOT/INSUFFICIENT。
|
|
156
|
+
- **无伪精度**:缺 CI 的森林图点只画点、不画误差线并标记 "CI not reported";
|
|
157
|
+
meta 合并不使用默认 `se=0.20`,无精度证据的条目记录排除原因。
|
|
158
|
+
|
|
159
|
+
## 📦 三适配器统一契约(Python 标准库)
|
|
160
|
+
|
|
161
|
+
三个可视化适配器共享同一 envelope 契约,适配器本身只使用 Python 标准库;ECharts 是 Web Studio 的可选浏览器运行时,不属于适配器或提交包:
|
|
162
|
+
统一 CLI:`--result <result.json> --out <out.json> [--lang zh|en]`:
|
|
163
|
+
|
|
164
|
+
```bash
|
|
165
|
+
python3 visualization/eduevidence-report/scripts/build_charts.py \
|
|
166
|
+
--result examples/ai-coding-assistant/result.json --out /tmp/charts.json
|
|
167
|
+
python3 visualization/eduevidence-report/scripts/build_infographics.py \
|
|
168
|
+
--result examples/ai-coding-assistant/result.json --out /tmp/infographics.json
|
|
169
|
+
python3 visualization/eduevidence-report/scripts/build_figures.py \
|
|
170
|
+
--result examples/ai-coding-assistant/result.json --out /tmp/figures.json --theme okabe_ito
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
输出 envelope:`adapter` / `contract_version` / `source_ref` / `source_sha256`(provenance)
|
|
174
|
+
/ `locale` / `data`。报告渲染器 `build_report.py` 与 CLI 共享同一核心函数。
|
|
175
|
+
(`build_figures.py --out-dir` 为兼容参数,迁移后移除。)
|
|
176
|
+
|
|
177
|
+
## 🛠️ CLI Quick Commands
|
|
178
|
+
|
|
179
|
+
```bash
|
|
180
|
+
# 1. 3 页 Local Web Studio
|
|
181
|
+
python3 scripts/dashboard_server.py --port 8765
|
|
182
|
+
|
|
183
|
+
# 2. 学术与实时趋势检索
|
|
184
|
+
python3 -m retrieval.search "AI coding assistants learning transfer"
|
|
185
|
+
|
|
186
|
+
# 3. 田野数据 DID 回归(数据契约回写见 skill/sub-skills/data-analysis;
|
|
187
|
+
# 示例用全研究周期夹具的 40 行匿名 DID 数据,无 PII)
|
|
188
|
+
python3 scripts/did_regression.py examples/full-research-cycle-fixture/data.csv
|
|
189
|
+
|
|
190
|
+
# 4. 效应量计算器
|
|
191
|
+
python3 scripts/effect_calculator.py --mean1 78.5 --sd1 10.2 --n1 90 --mean2 72.1 --sd2 11.0 --n2 90
|
|
192
|
+
|
|
193
|
+
# 5. Skill 一致性检查
|
|
194
|
+
python3 scripts/skill_lint.py
|
|
195
|
+
```
|