eduevidence 5.2.0 → 6.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CONTRIBUTING.md +105 -0
- package/README.md +142 -75
- package/README.zh-CN.md +73 -30
- package/SKILL.md +397 -131
- package/agents/openai.yaml +4 -0
- package/assets/readme/controlled-execution.svg +34 -0
- package/assets/readme/landing-tour.gif +0 -0
- package/assets/readme/logo.png +0 -0
- package/assets/readme/research-workflow.svg +56 -0
- package/assets/readme/studio-graph.png +0 -0
- package/assets/readme/studio-overview.png +0 -0
- package/assets/readme/studio-reports.png +0 -0
- package/assets/readme/studio-tour.gif +0 -0
- package/autoevolve/config.yaml +17 -0
- package/autoevolve/program.md +25 -0
- package/autoevolve/protected.manifest.yaml +34 -0
- package/benchmarks/adversarial/cases.jsonl +7 -0
- package/benchmarks/evidence-library.json +5268 -0
- package/benchmarks/partitions.json +8 -0
- package/bin/eduevidence.js +2 -1
- package/docs/architecture.md +496 -0
- package/docs/autoresearch-evolution-plan.md +2903 -0
- package/docs/autoresearch-implementation-status.md +101 -0
- package/docs/demo-storyboard.md +20 -0
- package/docs/demo-workplace-ai.md +92 -0
- package/docs/demo.md +32 -0
- package/docs/install-guide.md +150 -0
- package/docs/orchestration-role-model.md +1254 -0
- package/docs/release-closeout/README.md +17 -0
- package/docs/release-closeout/frontend-acceptance.md +23 -0
- package/docs/release-closeout/issues.md +19 -0
- package/docs/release-closeout/verification.md +28 -0
- package/docs/release-contract.md +108 -0
- package/docs/research-studio-guide.zh-CN.md +166 -0
- package/docs/sciverse-api.md +125 -0
- package/eduevidence_cli.py +29 -13
- package/engine/_resources.py +13 -0
- package/engine/autoevolve/__init__.py +3 -0
- package/engine/autoevolve/agent_view.py +167 -0
- package/engine/autoevolve/core.py +357 -0
- package/engine/autoevolve/events.py +11 -0
- package/engine/autoevolve/git_workspace.py +77 -0
- package/engine/autoevolve/projection.py +23 -0
- package/engine/autoevolve/runner.py +413 -0
- package/engine/autoevolve/trust.py +146 -0
- package/engine/autoresearch/__init__.py +6 -0
- package/engine/autoresearch/commit.py +132 -0
- package/engine/autoresearch/contracts.py +126 -0
- package/engine/autoresearch/controller.py +207 -0
- package/engine/autoresearch/events.py +12 -0
- package/engine/autoresearch/gap_priority.py +168 -0
- package/engine/autoresearch/projection.py +30 -0
- package/engine/autoresearch/research_memory.py +59 -0
- package/engine/autoresearch/saturation.py +91 -0
- package/engine/briefs.py +2 -1
- package/engine/capabilities.py +1 -0
- package/engine/contracts.py +3 -1
- package/engine/decision_policy.py +96 -0
- package/engine/evidence_graph.py +14 -10
- package/engine/evidencecore.py +7 -5
- package/engine/gaps.py +132 -73
- package/engine/ids.py +2 -0
- package/engine/judge_pack.py +65 -0
- package/engine/library.py +6 -2
- package/engine/library_builtin.py +3 -1
- package/engine/living.py +36 -5
- package/engine/meta_synthesis.py +3 -1
- package/engine/migration.py +88 -3
- package/engine/orchestration.py +460 -0
- package/engine/paths.py +2 -0
- package/engine/pilot.py +36 -33
- package/engine/project.py +2 -2
- package/engine/research_service.py +113 -0
- package/engine/studio_read_model.py +400 -0
- package/engine/taxonomy.py +211 -0
- package/engine/tribunal.py +44 -33
- package/engine/update.py +1 -0
- package/engine/versions.py +1 -1
- package/engine/worker_result.py +109 -0
- package/engine/workflows.py +70 -0
- package/examples/ai-coding-assistant-evidence/EduEvidence_Report.html +2934 -0
- package/examples/ai-coding-assistant-evidence/artifact_manifest.json +15 -0
- package/examples/ai-coding-assistant-evidence/citation_check.json +79 -0
- package/examples/ai-coding-assistant-evidence/claims.jsonl +12 -0
- package/examples/ai-coding-assistant-evidence/evaluation.json +35 -0
- package/examples/ai-coding-assistant-evidence/evidence.jsonl +12 -0
- package/examples/ai-coding-assistant-evidence/final_verdict.json +107 -0
- package/examples/ai-coding-assistant-evidence/frame.json +48 -0
- package/examples/ai-coding-assistant-evidence/gate_report.json +101 -0
- package/examples/ai-coding-assistant-evidence/intervention.json +51 -0
- package/examples/ai-coding-assistant-evidence/methodology.json +36 -0
- package/examples/ai-coding-assistant-evidence/raw_verdict.json +86 -0
- package/examples/ai-coding-assistant-evidence/report_spec.json +230 -0
- package/examples/ai-coding-assistant-evidence/reports-5themes/EduEvidence_Report_academic.html +2934 -0
- package/examples/ai-coding-assistant-evidence/reports-5themes/EduEvidence_Report_claude.html +2934 -0
- package/examples/ai-coding-assistant-evidence/reports-5themes/EduEvidence_Report_datalab-dark.html +2934 -0
- package/examples/ai-coding-assistant-evidence/reports-5themes/EduEvidence_Report_datalab.html +2934 -0
- package/examples/ai-coding-assistant-evidence/reports-5themes/EduEvidence_Report_presentation.html +2934 -0
- package/examples/ai-coding-assistant-evidence/reports-5themes/report_academic.html +2934 -0
- package/examples/ai-coding-assistant-evidence/reports-5themes/report_claude.html +2934 -0
- package/examples/ai-coding-assistant-evidence/reports-5themes/report_datalab-dark.html +2934 -0
- package/examples/ai-coding-assistant-evidence/reports-5themes/report_datalab.html +2934 -0
- package/examples/ai-coding-assistant-evidence/reports-5themes/report_presentation.html +2934 -0
- package/examples/ai-coding-assistant-evidence/result.json +1457 -0
- package/examples/ai-coding-assistant-evidence/result.zh.json +1457 -0
- package/examples/ai-coding-assistant-evidence/skeptic.json +72 -0
- package/examples/ai-coding-assistant-evidence/sources.jsonl +8 -0
- package/examples/ai-coding-assistant-evidence/verdict.json +107 -0
- package/examples/spaced-retrieval-practice/applicability.json +14 -0
- package/examples/spaced-retrieval-practice/artifact_manifest.json +15 -0
- package/examples/spaced-retrieval-practice/claims.jsonl +3 -0
- package/examples/spaced-retrieval-practice/evidence.jsonl +6 -0
- package/examples/spaced-retrieval-practice/final_verdict.json +93 -0
- package/examples/spaced-retrieval-practice/frame.json +58 -0
- package/examples/spaced-retrieval-practice/gate_report.json +101 -0
- package/examples/spaced-retrieval-practice/methodology.json +78 -0
- package/examples/spaced-retrieval-practice/report_spec.json +212 -0
- package/examples/spaced-retrieval-practice/reports-5themes/EduEvidence_Report_academic.html +2728 -0
- package/examples/spaced-retrieval-practice/reports-5themes/EduEvidence_Report_claude.html +2728 -0
- package/examples/spaced-retrieval-practice/reports-5themes/EduEvidence_Report_datalab-dark.html +2728 -0
- package/examples/spaced-retrieval-practice/reports-5themes/EduEvidence_Report_datalab.html +2728 -0
- package/examples/spaced-retrieval-practice/reports-5themes/EduEvidence_Report_presentation.html +2728 -0
- package/examples/spaced-retrieval-practice/reports-5themes/report_academic.html +2728 -0
- package/examples/spaced-retrieval-practice/reports-5themes/report_claude.html +2728 -0
- package/examples/spaced-retrieval-practice/reports-5themes/report_datalab-dark.html +2728 -0
- package/examples/spaced-retrieval-practice/reports-5themes/report_datalab.html +2728 -0
- package/examples/spaced-retrieval-practice/reports-5themes/report_presentation.html +2728 -0
- package/examples/spaced-retrieval-practice/result.json +942 -0
- package/examples/spaced-retrieval-practice/result.zh.json +942 -0
- package/examples/spaced-retrieval-practice/skeptic.json +70 -0
- package/examples/spaced-retrieval-practice/sources.jsonl +7 -0
- package/examples/spaced-retrieval-practice/verdict.json +93 -0
- package/examples/workplace-ai-assistant/artifact_manifest.json +15 -0
- package/examples/workplace-ai-assistant/claims.jsonl +4 -0
- package/examples/workplace-ai-assistant/evaluation.json +19 -0
- package/examples/workplace-ai-assistant/evidence.jsonl +4 -0
- package/examples/workplace-ai-assistant/evidence_graph.json +444 -0
- package/examples/workplace-ai-assistant/final_verdict.json +78 -0
- package/examples/workplace-ai-assistant/frame.json +41 -0
- package/examples/workplace-ai-assistant/gate_report.json +101 -0
- package/examples/workplace-ai-assistant/intervention.json +27 -0
- package/examples/workplace-ai-assistant/legacy-link-check.json +16 -0
- package/examples/workplace-ai-assistant/methodology.json +60 -0
- package/examples/workplace-ai-assistant/report_spec.json +224 -0
- package/examples/workplace-ai-assistant/reports-5themes/EduEvidence_Report_academic.html +2814 -0
- package/examples/workplace-ai-assistant/reports-5themes/EduEvidence_Report_claude.html +2814 -0
- package/examples/workplace-ai-assistant/reports-5themes/EduEvidence_Report_datalab-dark.html +2814 -0
- package/examples/workplace-ai-assistant/reports-5themes/EduEvidence_Report_datalab.html +2814 -0
- package/examples/workplace-ai-assistant/reports-5themes/EduEvidence_Report_presentation.html +2814 -0
- package/examples/workplace-ai-assistant/reports-5themes/report_academic.html +2814 -0
- package/examples/workplace-ai-assistant/reports-5themes/report_claude.html +2814 -0
- package/examples/workplace-ai-assistant/reports-5themes/report_datalab-dark.html +2814 -0
- package/examples/workplace-ai-assistant/reports-5themes/report_datalab.html +2814 -0
- package/examples/workplace-ai-assistant/reports-5themes/report_presentation.html +2814 -0
- package/examples/workplace-ai-assistant/result.json +615 -0
- package/examples/workplace-ai-assistant/result.zh.json +615 -0
- package/examples/workplace-ai-assistant/search_log.json +19 -0
- package/examples/workplace-ai-assistant/skeptic.json +72 -0
- package/examples/workplace-ai-assistant/sources.jsonl +3 -0
- package/examples/workplace-ai-assistant/validation_result.json +9 -0
- package/examples/workplace-ai-assistant/verdict.json +78 -0
- package/install.sh +7 -7
- package/integrations/agent_mcp.py +2 -2
- package/integrations/orchestration_dispatch.py +146 -0
- package/package.json +46 -3
- package/pyproject.toml +14 -22
- package/references/autoresearch.md +30 -0
- package/references/evaluation-policy.md +24 -0
- package/references/orchestration.md +22 -0
- package/references/report-copy-style.md +67 -0
- package/references/retrieval-compliance.md +75 -0
- package/references/retrieval-protocol.md +20 -0
- package/references/scientific-invariants.md +19 -0
- package/retrieval/audit.py +178 -0
- package/retrieval/fetch.py +96 -0
- package/retrieval/sciverse.py +398 -0
- package/retrieval/search.py +47 -7
- package/schemas/applicability.schema.json +94 -0
- package/schemas/chart-spec.schema.json +10 -3
- package/schemas/evidence.schema.json +316 -43
- package/schemas/fetch-result.schema.json +2 -1
- package/schemas/intervention.schema.json +106 -21
- package/schemas/report-result.schema.json +12 -4
- package/schemas/report-spec.schema.json +98 -100
- package/schemas/skeptic.schema.json +86 -0
- package/schemas/source.schema.json +21 -2
- package/schemas/v2/finding.schema.json +5 -1
- package/schemas/v2/methodology-audit.schema.json +5 -1
- package/schemas/v2/outcome.schema.json +28 -5
- package/schemas/v2/project.schema.json +2 -2
- package/schemas/v2/run.schema.json +1 -1
- package/schemas/v2/study.schema.json +5 -1
- package/schemas/vNext/autoevolve-session.schema.json +34 -0
- package/schemas/vNext/eval-snapshot.schema.json +77 -0
- package/schemas/vNext/execution-plan.schema.json +50 -0
- package/schemas/vNext/gap-priority.schema.json +54 -0
- package/schemas/vNext/negative-search-record.schema.json +68 -0
- package/schemas/vNext/research-iteration.schema.json +87 -0
- package/schemas/vNext/research-strategy.schema.json +62 -0
- package/schemas/vNext/skill-experiment.schema.json +90 -0
- package/schemas/vNext/task-spec.schema.json +156 -0
- package/schemas/vNext/worker-result.schema.json +60 -0
- package/schemas/verdict.schema.json +164 -28
- package/scripts/benchmark_judge.py +2 -2
- package/scripts/benchmark_v3.py +26 -43
- package/scripts/build_esl_artifacts.py +4 -4
- package/scripts/build_evidence_library.py +2 -2
- package/scripts/build_gh_pages.py +98 -0
- package/scripts/build_readme_diagrams.py +72 -0
- package/scripts/build_report_variants.py +101 -0
- package/scripts/build_result.py +74 -9
- package/scripts/check_autoresearch_invariants.py +95 -0
- package/scripts/check_package_parity.py +85 -0
- package/scripts/check_protocol_alignment.py +375 -0
- package/scripts/check_versioned_schemas.py +254 -0
- package/scripts/claim_audit.py +13 -8
- package/scripts/compute_confidence.py +10 -0
- package/scripts/daily_evolve.py +30 -0
- package/scripts/dashboard_server.py +130 -101
- package/scripts/did_regression.py +17 -32
- package/scripts/enrich_projects_human_and_lieflat.py +1 -1
- package/scripts/evidence_score.py +5 -2
- package/scripts/generate_metrics.py +4 -3
- package/scripts/generate_new_projects.py +5 -5
- package/scripts/orchestrator.py +286 -36
- package/scripts/pre_verdict_gate.py +224 -26
- package/scripts/quickstart.py +18 -2
- package/scripts/rebake_all_5themes.py +1 -2
- package/scripts/research_auto_cli.py +475 -0
- package/scripts/run_workspace.py +24 -8
- package/scripts/search_provenance.py +64 -0
- package/scripts/serve_web.py +9 -10
- package/scripts/skill_lint.py +1 -1
- package/scripts/skill_payload.py +81 -0
- package/scripts/test_adversarial_empirical.py +26 -19
- package/scripts/validate_schema.py +46 -2
- package/scripts/vnext_cli.py +133 -0
- package/setup.py +12 -0
- package/skill/agents/evaluation-designer.md +20 -4
- package/skill/agents/evidence-analyst.md +19 -3
- package/skill/agents/evidence-judge.md +50 -2
- package/skill/agents/evidence-retriever.md +20 -3
- package/skill/agents/intervention-designer.md +20 -4
- package/skill/agents/method-reviewer.md +18 -2
- package/skill/agents/{education-planner.md → research-planner.md} +19 -3
- package/skill/agents/skeptic.md +18 -2
- package/skill/roles/registry.yaml +45 -0
- package/skill/sub-skills/aihot-trend-analysis/SKILL.md +28 -9
- package/skill/sub-skills/contradiction-analysis/SKILL.md +31 -11
- package/skill/sub-skills/data-analysis/SKILL.md +34 -15
- package/skill/sub-skills/ethics-review/SKILL.md +33 -10
- package/skill/sub-skills/evidence-extraction/SKILL.md +29 -11
- package/skill/sub-skills/evidence-review/SKILL.md +31 -12
- package/skill/sub-skills/gap-analysis/SKILL.md +31 -9
- package/skill/sub-skills/literature-review/SKILL.md +35 -14
- package/skill/sub-skills/methodology-audit/SKILL.md +29 -12
- package/skill/sub-skills/report-generation/SKILL.md +40 -6
- package/skill/sub-skills/research-planning/SKILL.md +41 -14
- package/skill/sub-skills/study-design/SKILL.md +30 -9
- package/skill/task-briefs/adjudicate.md +32 -7
- package/skill/task-briefs/applicability.md +38 -0
- package/skill/task-briefs/audit.md +32 -7
- package/skill/task-briefs/challenge.md +34 -5
- package/skill/task-briefs/evaluate.md +30 -5
- package/skill/task-briefs/extract.md +31 -8
- package/skill/task-briefs/frame.md +39 -10
- package/skill/task-briefs/intervene.md +32 -6
- package/skill/task-briefs/present.md +32 -8
- package/skill/task-briefs/projection.md +37 -0
- package/skill/task-briefs/retrieve.md +36 -6
- package/skill/workflows/decision-and-pilot.md +85 -0
- package/skill/workflows/evaluate-and-update.md +93 -0
- package/skill/workflows/evidence-review.md +117 -0
- package/visualization/eduevidence-report/assets/base.css +2 -2
- package/visualization/eduevidence-report/assets/reader.css +752 -0
- package/visualization/eduevidence-report/assets/reader.js +132 -0
- package/visualization/eduevidence-report/references/chart-selection-catalog.md +109 -0
- package/visualization/eduevidence-report/references/lieflat-composition.md +3 -1
- package/visualization/eduevidence-report/scripts/build_figures.py +25 -3
- package/visualization/eduevidence-report/scripts/build_infographics.py +5 -1
- package/visualization/eduevidence-report/scripts/build_report.py +561 -121
- package/visualization/eduevidence-report/scripts/charts_data.py +2 -0
- package/visualization/eduevidence-report/scripts/lieflat_engine.py +371 -136
- package/visualization/eduevidence-report/scripts/zh_labels.py +80 -1
- package/visualization/eduevidence-report/themes/academic.css +1 -1
- package/visualization/eduevidence-report/themes/claude.css +1 -1
- package/visualization/eduevidence-report/themes/datalab-dark.css +2 -2
- package/visualization/eduevidence-report/themes/datalab.css +2 -2
- package/visualization/eduevidence-report/themes/presentation.css +2 -2
- package/web/README.md +18 -0
- package/web/architecture.html +14885 -0
- package/web/index.html +53 -0
- package/web/studio/THIRD_PARTY_LICENSES.txt +146 -0
- package/web/studio/assets/index-B8tkF44Q.css +1 -0
- package/web/studio/assets/index-CQ6Keoyc.js +230 -0
- package/web/studio/config.json +1 -0
- package/web/studio/index.html +14 -0
- package/engine/__pycache__/__init__.cpython-312.pyc +0 -0
- package/engine/__pycache__/analysis.cpython-312.pyc +0 -0
- package/engine/__pycache__/bias.cpython-312.pyc +0 -0
- package/engine/__pycache__/briefs.cpython-312.pyc +0 -0
- package/engine/__pycache__/capabilities.cpython-312.pyc +0 -0
- package/engine/__pycache__/citation_check.cpython-312.pyc +0 -0
- package/engine/__pycache__/contracts.cpython-312.pyc +0 -0
- package/engine/__pycache__/datasets.cpython-312.pyc +0 -0
- package/engine/__pycache__/events.cpython-312.pyc +0 -0
- package/engine/__pycache__/evidence_graph.cpython-312.pyc +0 -0
- package/engine/__pycache__/evidence_review.cpython-312.pyc +0 -0
- package/engine/__pycache__/evidencecore.cpython-312.pyc +0 -0
- package/engine/__pycache__/gap_lens.cpython-312.pyc +0 -0
- package/engine/__pycache__/gaps.cpython-312.pyc +0 -0
- package/engine/__pycache__/graph_store.cpython-312.pyc +0 -0
- package/engine/__pycache__/graph_validate.cpython-312.pyc +0 -0
- package/engine/__pycache__/ids.cpython-312.pyc +0 -0
- package/engine/__pycache__/library.cpython-312.pyc +0 -0
- package/engine/__pycache__/library_builtin.cpython-312.pyc +0 -0
- package/engine/__pycache__/living.cpython-312.pyc +0 -0
- package/engine/__pycache__/log.cpython-312.pyc +0 -0
- package/engine/__pycache__/meta_analysis.cpython-312.pyc +0 -0
- package/engine/__pycache__/meta_synthesis.cpython-312.pyc +0 -0
- package/engine/__pycache__/migration.cpython-312.pyc +0 -0
- package/engine/__pycache__/mode_router.cpython-312.pyc +0 -0
- package/engine/__pycache__/paths.cpython-312.pyc +0 -0
- package/engine/__pycache__/pilot.cpython-312.pyc +0 -0
- package/engine/__pycache__/planner.cpython-312.pyc +0 -0
- package/engine/__pycache__/project.cpython-312.pyc +0 -0
- package/engine/__pycache__/projections.cpython-312.pyc +0 -0
- package/engine/__pycache__/robustness.cpython-312.pyc +0 -0
- package/engine/__pycache__/run.cpython-312.pyc +0 -0
- package/engine/__pycache__/semantics.cpython-312.pyc +0 -0
- package/engine/__pycache__/study_design.cpython-312.pyc +0 -0
- package/engine/__pycache__/synthesis.cpython-312.pyc +0 -0
- package/engine/__pycache__/tribunal.cpython-312.pyc +0 -0
- package/engine/__pycache__/update.cpython-312.pyc +0 -0
- package/engine/__pycache__/versions.cpython-312.pyc +0 -0
- package/integrations/__pycache__/__init__.cpython-312.pyc +0 -0
- package/integrations/__pycache__/agent_mcp.cpython-312.pyc +0 -0
- package/integrations/__pycache__/smart_web_fetch.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/__init__.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/corpus_store.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/dedupe.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/failures.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/fetch.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/search.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/source.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/validate.cpython-312.pyc +0 -0
- package/scripts/__pycache__/__init__.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark_evaluator.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark_judge.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark_routing.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark_v2.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark_v3.cpython-312.pyc +0 -0
- package/scripts/__pycache__/build_result.cpython-312.pyc +0 -0
- package/scripts/__pycache__/claim_audit.cpython-312.pyc +0 -0
- package/scripts/__pycache__/complexity_gate.cpython-312.pyc +0 -0
- package/scripts/__pycache__/compute_confidence.cpython-312.pyc +0 -0
- package/scripts/__pycache__/dashboard_server.cpython-312.pyc +0 -0
- package/scripts/__pycache__/did_regression.cpython-312.pyc +0 -0
- package/scripts/__pycache__/effect_calculator.cpython-312.pyc +0 -0
- package/scripts/__pycache__/evidence_matrix.cpython-312.pyc +0 -0
- package/scripts/__pycache__/evidence_score.cpython-312.pyc +0 -0
- package/scripts/__pycache__/evidence_semantics.cpython-312.pyc +0 -0
- package/scripts/__pycache__/fetch_benchmark.cpython-312.pyc +0 -0
- package/scripts/__pycache__/lint_report_layout.cpython-312.pyc +0 -0
- package/scripts/__pycache__/orchestrator.cpython-312.pyc +0 -0
- package/scripts/__pycache__/pre_verdict_gate.cpython-312.pyc +0 -0
- package/scripts/__pycache__/recompute_demo_quality.cpython-312.pyc +0 -0
- package/scripts/__pycache__/render_report.cpython-312.pyc +0 -0
- package/scripts/__pycache__/render_report_html.cpython-312.pyc +0 -0
- package/scripts/__pycache__/run_workspace.cpython-312.pyc +0 -0
- package/scripts/__pycache__/skill_lint.cpython-312.pyc +0 -0
- package/scripts/__pycache__/startup_probe.cpython-312.pyc +0 -0
- package/scripts/__pycache__/sync_killer_demo_report.cpython-312.pyc +0 -0
- package/scripts/__pycache__/test_adversarial_empirical.cpython-312-pytest-9.0.2.pyc +0 -0
- package/scripts/__pycache__/test_adversarial_empirical.cpython-312-pytest-9.1.1.pyc +0 -0
- package/scripts/__pycache__/validate_schema.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/adapter_contract.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/build_artifact_manifest.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/build_charts.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/build_figures.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/build_infographics.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/build_report.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/charts_data.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/lieflat_engine.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/zh_labels.cpython-312.pyc +0 -0
package/SKILL.md
CHANGED
|
@@ -1,195 +1,461 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: eduevidence
|
|
3
|
-
description: "
|
|
3
|
+
description: "Decision-grade evidence synthesis for education and applied social science intervention decisions. Use when a user needs to determine whether, when, for whom, or how to adopt, pilot, evaluate, or revise a teaching method, curriculum change, AI tool, program, or policy intervention. Run an auditable evidence-to-decision workflow spanning systematic retrieval, counter-evidence challenge, methodological quality and evidence-certainty appraisal, provenance-traceable evidence graphs, applicability boundaries, evidence-grounded gap detection, preregistration-ready study or pilot design, empirical evidence re-injection, and decision revision."
|
|
4
4
|
---
|
|
5
|
+
# EduEvidence 6.2 — Decision-Grade Evidence Engine
|
|
6
|
+
> **AI4SS Track | Art–Science Integration · General Intelligence**
|
|
7
|
+
> **From empirical questions to decision-grade evidence and evidence-to-action loops.**
|
|
8
|
+
|
|
9
|
+
## 1. Mission
|
|
10
|
+
Use EduEvidence to turn an education or applied social science decision question into an **action boundary** that can be traced, challenged, appraised, piloted, evaluated, and revised.
|
|
11
|
+
Do not stop at “what does the literature say?” Determine:
|
|
12
|
+
- **Whether** the evidence supports action.
|
|
13
|
+
- **When** the intervention is likely to work.
|
|
14
|
+
- **For whom** the evidence is applicable.
|
|
15
|
+
- **How** the intervention should be piloted, evaluated, or revised.
|
|
16
|
+
- **What evidence would change the decision** if current certainty is insufficient.
|
|
17
|
+
Keep the public workflow deliberately lean:
|
|
18
|
+
```text
|
|
19
|
+
Evidence Review → Decision & Pilot → Evaluate & Update
|
|
20
|
+
```
|
|
21
|
+
Internally, route work through one architecture only:
|
|
22
|
+
```text
|
|
23
|
+
Research intent → Workflow → Capability DAG → Scientific gate → Artifact
|
|
24
|
+
→ Evidence Graph revision → Decision snapshot → Projection
|
|
25
|
+
```
|
|
26
|
+
Treat roles, models, Agent MCP, retrievers, scripts, and HTML reports as **execution adapters or projections**, not as competing workflow architectures.
|
|
5
27
|
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
> **AI4SS Track | 科艺融合 · 通用智能**
|
|
9
|
-
> **From Empirical Questions to Evidence-Based Decisions & Causal Trial Loops.**
|
|
28
|
+
---
|
|
10
29
|
|
|
11
|
-
|
|
30
|
+
## 2. Route the Request to a Workflow
|
|
31
|
+
Load exactly one primary workflow for the current task:
|
|
32
|
+
- `skill/workflows/evidence-review.md` — use for evidence synthesis and decision appraisal from existing research.
|
|
33
|
+
- `skill/workflows/decision-and-pilot.md` — use when the evidence must be converted into an actionable pilot or intervention plan.
|
|
34
|
+
- `skill/workflows/evaluate-and-update.md` — use when empirical data or new evidence must update the Evidence Graph and decision.
|
|
35
|
+
Treat `skill/sub-skills/` as **internal capability recipes**. Do not expose them as separate user-facing entry points unless the runtime explicitly requires it.
|
|
12
36
|
|
|
13
|
-
|
|
37
|
+
---
|
|
14
38
|
|
|
15
|
-
|
|
39
|
+
## 3. Select a Research Mode
|
|
40
|
+
### Mode 1 — Evidence Review
|
|
41
|
+
Use when the user primarily needs a defensible synthesis of existing evidence and an evidence-bounded decision.
|
|
42
|
+
Typical flow:
|
|
43
|
+
```text
|
|
44
|
+
Frame → Retrieve → Extract → Challenge → Audit → Adjudicate → Applicability
|
|
45
|
+
```
|
|
46
|
+
### Mode 2 — Full Research Cycle
|
|
47
|
+
Use when the task must continue beyond synthesis into new evidence generation.
|
|
48
|
+
Typical flow:
|
|
49
|
+
```text
|
|
50
|
+
Evidence Review
|
|
51
|
+
→ Knowledge Gap
|
|
52
|
+
→ Study / Pilot Design
|
|
53
|
+
→ Local or Field Data
|
|
54
|
+
→ Analysis
|
|
55
|
+
→ Evidence Graph Revision
|
|
56
|
+
→ Decision Revision
|
|
57
|
+
```
|
|
58
|
+
Never design a new study merely because evidence is weak. A new study or pilot must be grounded in an explicit, evidence-supported `KnowledgeGap`.
|
|
16
59
|
|
|
17
|
-
|
|
60
|
+
---
|
|
18
61
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
62
|
+
## 4. Preserve Canonical State and Provenance
|
|
63
|
+
Use the unified Project Workspace as the authoritative state model:
|
|
64
|
+
```text
|
|
65
|
+
Project / Run / Revision / DecisionSnapshot
|
|
66
|
+
```
|
|
67
|
+
Use the provenance-traceable **Evidence Graph** as the authoritative evidence structure.
|
|
68
|
+
Treat `result.json`, Markdown reports, HTML reports, charts, and dashboards as projections. Do not treat presentation artifacts as the source of truth.
|
|
69
|
+
Apply the following invariants:
|
|
70
|
+
1. **Single Canonical Project per Research Question**
|
|
71
|
+
Keep one canonical project for the same research question. Do not create duplicate topic directories for the same question; create an immutable Revision instead.
|
|
72
|
+
2. **Immutable Revisions**
|
|
73
|
+
Never overwrite the provenance history of prior evidence or decisions. Add a new revision and bind new artifacts to it.
|
|
74
|
+
3. **Shared Facts, Local Interpretation**
|
|
75
|
+
Verified external research facts such as `Source`, `Study`, `Finding`, and `Audit` may be reused across project snapshots. Keep `Claim`, `EvidenceLink`, `Applicability`, and `Decision` project-local.
|
|
76
|
+
4. **Evidence-Grounded Research Design**
|
|
77
|
+
Do not create a new study design unless it cites an explicit evidence-grounded `KnowledgeGap` identifier.
|
|
25
78
|
|
|
26
79
|
---
|
|
27
80
|
|
|
28
|
-
##
|
|
29
|
-
|
|
81
|
+
## 5. Choose the Execution Layer
|
|
82
|
+
### Mode A — Platform Native
|
|
83
|
+
Prefer Platform Native when the environment does not provide Agent MCP or when a portable, zero-MCP Skill execution is required.
|
|
84
|
+
Preserve the same workflow, schemas, scientific gates, provenance rules, and decision semantics even when enhanced adapters are unavailable.
|
|
85
|
+
### Mode B — Agent MCP Enhanced
|
|
86
|
+
Use Agent MCP only as an optional execution enhancement:
|
|
30
87
|
```text
|
|
31
|
-
|
|
32
|
-
|
|
88
|
+
Detect → Recommend → Obtain user authorization → safe_spawn
|
|
89
|
+
```
|
|
90
|
+
If Agent MCP is unavailable or not authorized, degrade cleanly to Platform Native. Never make MCP availability a prerequisite for scientific correctness.
|
|
33
91
|
|
|
34
|
-
|
|
35
|
-
Applicability → Intervene → Evaluate
|
|
92
|
+
---
|
|
36
93
|
|
|
37
|
-
|
|
94
|
+
## 6. Execute the Canonical 9-Step Protocol
|
|
95
|
+
Treat `docs/architecture.md` as the canonical protocol definition and validate stage outputs against the schemas in `schemas/`.
|
|
96
|
+
### Research Core — Evidence Discipline
|
|
97
|
+
```text
|
|
38
98
|
Frame → Retrieve → Extract → Challenge → Audit → Adjudicate
|
|
39
|
-
→ Applicability → Intervene → Evaluate
|
|
40
|
-
|
|
41
|
-
Fetch / Validate = Retrieve 内部强制 gate(RULE 2:snippet ≠ 证据内容)
|
|
42
|
-
Present = 最终呈现层(不计入 9 步协议)
|
|
43
99
|
```
|
|
100
|
+
### Decision Extension — Evidence to Action
|
|
101
|
+
```text
|
|
102
|
+
Applicability → Intervene → Evaluate
|
|
103
|
+
```
|
|
104
|
+
### End-to-End Protocol
|
|
105
|
+
| # | Stage | Primary Contract | Required Duty |
|
|
106
|
+
|---|---|---|---|
|
|
107
|
+
| 1 | **Frame** | `education-frame.schema.json` | Define the intervention, target population, comparison or counterfactual, context, and outcome dimensions. |
|
|
108
|
+
| 2 | **Retrieve** | `source.schema.json`, `fetch-result.schema.json` | Retrieve candidate evidence, fetch the underlying source, and validate provenance. Treat snippets as discovery aids, never as evidence content. |
|
|
109
|
+
| 3 | **Extract** | `evidence.schema.json` | Extract findings, effect sizes, confidence intervals, sample sizes, outcomes, population characteristics, and study design information when available. |
|
|
110
|
+
| 4 | **Challenge** | `evidence.schema.json` | Search explicitly for null findings, negative findings, contradictory evidence, alternative explanations, and confounders. |
|
|
111
|
+
| 5 | **Audit** | `methodology.schema.json` | Appraise study quality and evidence certainty. Apply WWC 5.0 criteria where relevant to education-study design; use GRADE-informed certainty assessment at the body-of-evidence level where appropriate. |
|
|
112
|
+
| 6 | **Adjudicate** | `verdict.schema.json` | Integrate evidence and emit a bounded decision: `ADOPT`, `PILOT`, `DO_NOT_ADOPT`, or `INSUFFICIENT`. Pass the Pre-Verdict Gate before finalizing. |
|
|
113
|
+
| 7 | **Applicability** | `references/applicability-policy.md` | State who the evidence applies to, in which contexts, for which outcomes, and under what conditions. |
|
|
114
|
+
| 8 | **Intervene** | `intervention.schema.json` | Design the minimum viable intervention or pilot and define explicit success, failure, and stop conditions. |
|
|
115
|
+
| 9 | **Evaluate** | `evaluation.schema.json` | Define baseline, post-intervention, retention/maintenance, transfer, and decision-update logic. |
|
|
116
|
+
Treat **Present** as a projection layer, not a protocol stage.
|
|
44
117
|
|
|
45
|
-
|
|
46
|
-
|---|------|----------------------|
|
|
47
|
-
| 1 | Frame | education-frame.schema.json |
|
|
48
|
-
| 2 | Retrieve(内含 Fetch/Validate gate) | source.schema.json / fetch-result.schema.json |
|
|
49
|
-
| 3 | Extract | evidence.schema.json |
|
|
50
|
-
| 4 | Challenge | evidence.schema.json(反方/未发现/confounder) |
|
|
51
|
-
| 5 | Audit | methodology.schema.json(task_vs_learning_guard) |
|
|
52
|
-
| 6 | Adjudicate | verdict.schema.json(规则化置信度 + Pre-Verdict Gate) |
|
|
53
|
-
| 7 | Applicability | applicability-policy.md(For whom / which course / which outcome / conditions) |
|
|
54
|
-
| 8 | Intervene | intervention.schema.json(最小可验证 PILOT + Stop Conditions) |
|
|
55
|
-
| 9 | Evaluate | evaluation.schema.json(基/后/保持/迁移 + 成功阈值) |
|
|
56
|
-
| Present | 呈现层 | result.json / result.zh.json → 烘焙主题报告 |
|
|
57
|
-
|
|
58
|
-
**Present 主题显示名(生成时五选一,烘焙定主题,最终 HTML 仅中英文切换)**:
|
|
118
|
+
---
|
|
59
119
|
|
|
120
|
+
## 7. Enforce Scientific Gates
|
|
121
|
+
### Gate A — Fetch Before Evidence Use
|
|
122
|
+
Do not treat search-result snippets, abstracts without sufficient context, or generated summaries as direct evidence when the underlying source can be validated.
|
|
123
|
+
Require:
|
|
60
124
|
```text
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
125
|
+
Retrieve → Fetch → Validate → Extract
|
|
126
|
+
```
|
|
127
|
+
Record provenance or an explicit retrieval limitation.
|
|
128
|
+
### Gate B — Counter-Evidence Before Verdict
|
|
129
|
+
Before adjudication, actively search for evidence that could weaken or reverse the initial interpretation.
|
|
130
|
+
Include, where relevant:
|
|
131
|
+
- Null or negative findings.
|
|
132
|
+
- Replication failures.
|
|
133
|
+
- Boundary conditions.
|
|
134
|
+
- Population or context mismatch.
|
|
135
|
+
- Outcome substitution.
|
|
136
|
+
- Confounding or selection bias.
|
|
137
|
+
- Publication-bias signals.
|
|
138
|
+
### Gate C — No Direct Learning Evidence, No ADOPT
|
|
139
|
+
For education interventions, do not issue `ADOPT` based only on task speed, task completion, productivity, usability, preference, or subjective experience.
|
|
140
|
+
Require direct evidence on learning, retention, independent transfer, or another explicitly decision-relevant outcome before `ADOPT` can be considered.
|
|
141
|
+
If direct evidence is missing, bound the decision to `PILOT`, `INSUFFICIENT`, or `DO_NOT_ADOPT` as justified by the evidence.
|
|
142
|
+
### Gate D — No False Precision
|
|
143
|
+
Never invent missing uncertainty statistics.
|
|
144
|
+
- If a confidence interval is not reported, do not fabricate one.
|
|
145
|
+
- Do not assume a default standard error such as `SE=0.20` for meta-analysis.
|
|
146
|
+
- Record why an observation cannot be pooled when precision information is unavailable.
|
|
147
|
+
- In forest plots, display an effect point without an error bar and mark `CI not reported` when appropriate.
|
|
148
|
+
### Gate E — Causal Estimation Must Fail Closed
|
|
149
|
+
For DID, OLS, or quasi-experimental analysis, fail closed when the design is not estimable.
|
|
150
|
+
For singular or collinear designs, empty treatment/time cells, zero-variance variables, saturated models, or other invalid inputs:
|
|
151
|
+
```json
|
|
152
|
+
{
|
|
153
|
+
"status": "error",
|
|
154
|
+
"did_coefficient": null,
|
|
155
|
+
"standard_error": null,
|
|
156
|
+
"p_value": null,
|
|
157
|
+
"ci_95": null,
|
|
158
|
+
"hedges_g": null
|
|
159
|
+
}
|
|
66
160
|
```
|
|
161
|
+
Do not fabricate fallback statistics.
|
|
162
|
+
For non-clustered DID inference, emit:
|
|
163
|
+
```text
|
|
164
|
+
inference_status = "non_cluster_warning"
|
|
165
|
+
```
|
|
166
|
+
Do not label DID or other quasi-experimental evidence as `Meets Standards Without Reservations` solely because a regression executed successfully.
|
|
167
|
+
### Gate F — No Study Design Without a Grounded Gap
|
|
168
|
+
Require every generated intervention study, trial, or quasi-experiment to cite the `KnowledgeGap` that justifies it.
|
|
169
|
+
A valid gap must arise from the evidence state, such as:
|
|
170
|
+
- Missing population evidence.
|
|
171
|
+
- Missing transfer or retention outcomes.
|
|
172
|
+
- Conflicting findings.
|
|
173
|
+
- Weak causal identification.
|
|
174
|
+
- Context mismatch.
|
|
175
|
+
- Insufficient precision.
|
|
176
|
+
- Unresolved mechanism or implementation uncertainty.
|
|
177
|
+
Do not generate generic “future research” ideas detached from the Evidence Graph.
|
|
67
178
|
|
|
68
179
|
---
|
|
69
180
|
|
|
70
|
-
##
|
|
71
|
-
|
|
72
|
-
|
|
|
73
|
-
|
|
74
|
-
|
|
|
75
|
-
|
|
|
76
|
-
|
|
|
77
|
-
|
|
|
78
|
-
|
|
|
79
|
-
|
|
|
80
|
-
|
|
|
81
|
-
|
|
|
82
|
-
|
|
|
83
|
-
| Present | skill/sub-skills/report-generation + visualization/eduevidence-report
|
|
84
|
-
| Full Research Cycle
|
|
85
|
-
|
|
|
181
|
+
## 8. Use the Capability Map
|
|
182
|
+
Route stages to reusable implementation capabilities as follows:
|
|
183
|
+
| Stage | Implementation Capability |
|
|
184
|
+
|---|---|
|
|
185
|
+
| Frame | `skill/sub-skills/research-planning` + `scripts/complexity_gate.py` |
|
|
186
|
+
| Retrieve | `skill/sub-skills/literature-review` + `skill/sub-skills/aihot-trend-analysis` |
|
|
187
|
+
| Extract | `skill/sub-skills/evidence-extraction` |
|
|
188
|
+
| Challenge | `skill/sub-skills/contradiction-analysis` |
|
|
189
|
+
| Audit | `skill/sub-skills/methodology-audit` |
|
|
190
|
+
| Adjudicate | `skill/sub-skills/evidence-review` + `scripts/pre_verdict_gate.py` + `scripts/compute_confidence.py` |
|
|
191
|
+
| Applicability | `references/applicability-policy.md` |
|
|
192
|
+
| Intervene | `references/intervention-design.md` + `skill/agents/intervention-designer.md` |
|
|
193
|
+
| Evaluate | `references/evaluation-design.md` + `skill/agents/evaluation-designer.md` |
|
|
194
|
+
| Present | `skill/sub-skills/report-generation` + `visualization/eduevidence-report` |
|
|
195
|
+
| Full Research Cycle extension | `skill/sub-skills/gap-analysis` → `study-design` → `data-analysis` |
|
|
196
|
+
| Cross-cutting | `skill/sub-skills/ethics-review` |
|
|
197
|
+
Do not describe these capabilities as independent user-facing Skills unless packaging or runtime compatibility specifically requires it.
|
|
86
198
|
|
|
87
|
-
|
|
199
|
+
---
|
|
88
200
|
|
|
89
|
-
|
|
201
|
+
## 9. Apply the Complexity Gate
|
|
202
|
+
Classify the task before execution:
|
|
203
|
+
### S — Quick Evidence Check
|
|
204
|
+
Use for narrowly scoped factual or evidence checks. Preserve provenance and uncertainty, but do not force a full deep-review pipeline when unnecessary.
|
|
205
|
+
### M — Standard Review
|
|
206
|
+
Use as the default for intervention decisions. Run the standard evidence workflow with Challenge, Audit, Adjudicate, and Applicability gates.
|
|
207
|
+
### L — Deep Research Cycle
|
|
208
|
+
Use when the decision is high-impact, disputed, methodologically complex, or expected to lead to a real-world intervention.
|
|
209
|
+
Require stronger search coverage, independent challenge/review, explicit applicability analysis, and—when the evidence justifies it—pilot and evaluation outputs.
|
|
210
|
+
When uncertain, escalate upward rather than skipping scientific safeguards.
|
|
90
211
|
|
|
91
212
|
---
|
|
92
213
|
|
|
93
|
-
##
|
|
94
|
-
|
|
95
|
-
**
|
|
214
|
+
## 10. Build Decision-Grade Evidence, Not a Literature Summary
|
|
215
|
+
For every substantial decision output, distinguish at least the following layers:
|
|
216
|
+
1. **What the studies found.**
|
|
217
|
+
2. **How trustworthy those findings are.**
|
|
218
|
+
3. **Where findings agree or conflict.**
|
|
219
|
+
4. **Whether the evidence applies to the target population and context.**
|
|
220
|
+
5. **What decision the evidence currently supports.**
|
|
221
|
+
6. **What the evidence does not support.**
|
|
222
|
+
7. **What new evidence would be most decision-changing.**
|
|
223
|
+
8. **How a pilot or study could generate that evidence.**
|
|
224
|
+
9. **How new empirical evidence would revise the graph and decision.**
|
|
225
|
+
Prefer bounded language over universal claims.
|
|
96
226
|
|
|
97
|
-
|
|
98
|
-
registry-verified DOI (Crossref/DataCite audit trail: `benchmarks/doi-audit/report.md`):
|
|
227
|
+
---
|
|
99
228
|
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
-
|
|
229
|
+
## 11. Preserve Evidence Graph Traceability
|
|
230
|
+
Bind every material decision claim to traceable evidence objects.
|
|
231
|
+
Maintain a clear path such as:
|
|
232
|
+
```text
|
|
233
|
+
Source → Study → Finding → Audit → Claim / Counterclaim
|
|
234
|
+
→ EvidenceLink → Applicability → DecisionSnapshot
|
|
235
|
+
```
|
|
236
|
+
When evidence changes, create a new Evidence Graph revision and a new DecisionSnapshot rather than silently mutating the previous conclusion.
|
|
237
|
+
Use `data_origin` or equivalent provenance metadata to distinguish:
|
|
238
|
+
- `manual_curated`
|
|
239
|
+
- validated retrieved evidence
|
|
240
|
+
- local empirical data
|
|
241
|
+
- synthetic fixtures or demos
|
|
242
|
+
Never allow synthetic demo data to appear as real empirical support.
|
|
111
243
|
|
|
112
244
|
---
|
|
113
245
|
|
|
114
|
-
##
|
|
246
|
+
## 12. Generate Evidence-Grounded Gaps
|
|
247
|
+
Treat a `KnowledgeGap` as an evidence object, not as brainstorming text.
|
|
248
|
+
A gap should specify:
|
|
249
|
+
- The unresolved claim or decision boundary.
|
|
250
|
+
- Which evidence is missing, conflicting, indirect, or weak.
|
|
251
|
+
- Why resolving the gap could change the decision.
|
|
252
|
+
- The minimum useful evidence required to reduce uncertainty.
|
|
253
|
+
Only then route the gap into `study-design` or `decision-and-pilot`.
|
|
115
254
|
|
|
116
|
-
|
|
117
|
-
python3 scripts/dashboard_server.py --port 8765
|
|
118
|
-
# http://127.0.0.1:8765/
|
|
119
|
-
```
|
|
255
|
+
---
|
|
120
256
|
|
|
121
|
-
|
|
257
|
+
## 13. Design Preregistration-Ready Studies and Pilots
|
|
258
|
+
Generate **preregistration-ready**, not falsely “preregistered,” designs unless an actual registration has occurred.
|
|
259
|
+
At minimum, define:
|
|
260
|
+
- Research question and grounded `KnowledgeGap`.
|
|
261
|
+
- Population and eligibility criteria.
|
|
262
|
+
- Intervention and comparison.
|
|
263
|
+
- Primary and secondary outcomes.
|
|
264
|
+
- Timing and follow-up windows.
|
|
265
|
+
- Assignment or identification strategy.
|
|
266
|
+
- Exclusion and missing-data rules.
|
|
267
|
+
- Analysis plan.
|
|
268
|
+
- Success thresholds.
|
|
269
|
+
- Stop conditions.
|
|
270
|
+
- Ethics, privacy, and IRB considerations when applicable.
|
|
271
|
+
- Decision-update rule specifying how results alter the Evidence Graph or verdict.
|
|
122
272
|
|
|
123
|
-
|
|
124
|
-
2. **报告浏览 (Report Browser)** — 列出有 EduEvidence_Report.html 的课题;若存在 reports-5themes/EduEvidence_Report_<theme>.html 则提供 5 主题烘焙变体选择。报告仅在生成时定主题,最终 HTML 只保留中英文切换,不做运行时换肤。
|
|
125
|
-
3. **数据可视化 (Data Visualization)** — 按课题选择,查看森林图、效应量分布、结果维度与 SSOT 证据图谱。
|
|
273
|
+
---
|
|
126
274
|
|
|
127
|
-
|
|
275
|
+
## 14. Re-Inject Empirical Evidence
|
|
276
|
+
Treat DID as one concrete implementation, not as the system's conceptual boundary.
|
|
277
|
+
When valid local or field data are supplied:
|
|
278
|
+
```text
|
|
279
|
+
Validate data
|
|
280
|
+
→ Execute appropriate empirical analysis
|
|
281
|
+
→ Record assumptions and limitations
|
|
282
|
+
→ Create new Finding / Audit objects
|
|
283
|
+
→ Create Evidence Graph revision
|
|
284
|
+
→ Re-run applicability and adjudication
|
|
285
|
+
→ Create new DecisionSnapshot
|
|
286
|
+
```
|
|
287
|
+
Use DID, OLS, RCT analysis, pre/post comparisons, regression discontinuity, interrupted time series, matching, panel methods, or other designs only when justified by the data-generating process and identification assumptions.
|
|
288
|
+
Never imply that statistical significance alone establishes a decision.
|
|
128
289
|
|
|
129
290
|
---
|
|
130
291
|
|
|
131
|
-
##
|
|
292
|
+
## 15. Use the Flagship Scenario as a Demonstration, Not a Universal Template
|
|
293
|
+
Use `examples/ai-coding-assistant-evidence/` as the flagship demonstration of the full evidence-to-decision loop.
|
|
294
|
+
Flagship question:
|
|
295
|
+
> **Should first-year university C/Python students be allowed to use generative AI coding assistants?**
|
|
296
|
+
The example demonstrates:
|
|
297
|
+
- A real, registry-verified evidence base.
|
|
298
|
+
- Structured extraction and counter-evidence challenge.
|
|
299
|
+
- Separation of task performance from independent learning outcomes.
|
|
300
|
+
- A bounded `PILOT` verdict rather than unconditional adoption.
|
|
301
|
+
- Provenance labels that distinguish curated evidence from synthetic fixtures.
|
|
302
|
+
- A quasi-experimental classroom evaluation path.
|
|
303
|
+
- Empirical evidence re-injection followed by decision revision.
|
|
304
|
+
Do not generalize the flagship verdict to unrelated populations, courses, tools, or policy contexts.
|
|
132
305
|
|
|
133
|
-
|
|
134
|
-
| --- | --- | --- |
|
|
135
|
-
| result.json → forest_plot_data | Extract + Adjudicate | 森林图 |
|
|
136
|
-
| result.json → evidence(数值 effect_size) | Extract | 效应量分布 |
|
|
137
|
-
| result.json → outcome_mapping | Extract + Audit | 结果维度 |
|
|
138
|
-
| evidence_graph.json → export_echarts_graph() | Adjudicate(evidence-review) | SSOT 证据图谱 |
|
|
139
|
-
| EduEvidence_Report.html + reports-5themes/*.html | Present(visualization/eduevidence-report/scripts/build_report.py) | 报告浏览(烘焙变体) |
|
|
140
|
-
| result.json → meta.question + decision | Present | 仪表盘 / 可视化标题与裁决 |
|
|
306
|
+
The four-state output is reachable on both ends, and each public case records which end it reached:
|
|
141
307
|
|
|
142
|
-
|
|
308
|
+
- `examples/ai-coding-assistant-evidence/` - `PILOT` / Moderate / 0.586. Primary evidence stops at task performance, so the decision is bounded.
|
|
309
|
+
- `examples/spaced-retrieval-practice/` - `ADOPT` / High / 0.893. Retention and transfer, the primary outcomes, carry direct evidence at directness 2.
|
|
310
|
+
- `examples/workplace-ai-assistant/` - `PILOT` / Moderate / 0.578, using the policy domain contract.
|
|
311
|
+
|
|
312
|
+
A verdict never awards itself an action: the Pre-Verdict Gate re-derives primary-outcome directness from the evidence corpus and caps an unsupported `ADOPT` to `PILOT`. Never present a case as ADOPT without that derivation passing.
|
|
143
313
|
|
|
144
314
|
---
|
|
145
315
|
|
|
146
|
-
##
|
|
316
|
+
## 16. Keep Presentation as a Projection Layer
|
|
317
|
+
Generate `result.json` and `result.zh.json` as structured presentation inputs when the workflow requires them.
|
|
318
|
+
Support baked report themes as projections:
|
|
319
|
+
- Claude Research — Light
|
|
320
|
+
- Academic Paper — Light
|
|
321
|
+
- DataLab — Light
|
|
322
|
+
- DataLab — Dark
|
|
323
|
+
- Presentation / Judge — Dark
|
|
324
|
+
Bake the selected theme at generation time. Keep only language switching in the final report unless the runtime explicitly supports another behavior.
|
|
325
|
+
Do not let presentation-layer state modify the canonical Evidence Graph.
|
|
147
326
|
|
|
148
|
-
|
|
149
|
-
饱和模型等不可估计输入返回 `status="error"` + 稳定 `error_code`,且
|
|
150
|
-
`did_coefficient` / `standard_error` / `p_value` / `ci_95` / `hedges_g` 全部为
|
|
151
|
-
`null`,绝不伪造 `SE=1.0` 或虚假 p 值。
|
|
152
|
-
- **非 cluster 显式标注**:普通 DID 结果带 `inference_status="non_cluster_warning"`;
|
|
153
|
-
DID/准实验(QED)永远不得标注 `Meets Standards Without Reservations`。
|
|
154
|
-
- **无直接学习证据不得 ADOPT**:裁决要求 learning/独立迁移结果(直接性 directness=2)
|
|
155
|
-
才允许 High+支持 → ADOPT;任务表现、程序效率、主观体验只能 PILOT/INSUFFICIENT。
|
|
156
|
-
- **无伪精度**:缺 CI 的森林图点只画点、不画误差线并标记 "CI not reported";
|
|
157
|
-
meta 合并不使用默认 `se=0.20`,无精度证据的条目记录排除原因。
|
|
327
|
+
---
|
|
158
328
|
|
|
159
|
-
##
|
|
329
|
+
## 17. Respect the Visualization Contract
|
|
330
|
+
When visualization assets are generated, derive them from canonical project artifacts rather than hard-coded demo values.
|
|
331
|
+
| View | Contract Source |
|
|
332
|
+
|---|---|
|
|
333
|
+
| Dashboard question and verdict | `result.json → meta.question + decision` |
|
|
334
|
+
| Forest plot | `result.json → forest_plot_data` |
|
|
335
|
+
| Effect-size distribution | `result.json → evidence` |
|
|
336
|
+
| Outcome mapping | `result.json → outcome_mapping` |
|
|
337
|
+
| Evidence graph | `evidence_graph.json → export_echarts_graph()` |
|
|
338
|
+
| Report variants | `EduEvidence_Report.html` + `reports-5themes/*.html` |
|
|
339
|
+
For `forest_plot_data`, preserve fields such as:
|
|
340
|
+
```text
|
|
341
|
+
study_label
|
|
342
|
+
outcome_dimension
|
|
343
|
+
effect_size
|
|
344
|
+
ci_lower
|
|
345
|
+
ci_upper
|
|
346
|
+
sample_size
|
|
347
|
+
direction
|
|
348
|
+
wwc_rating
|
|
349
|
+
```
|
|
350
|
+
Do not invent values required only for visualization.
|
|
160
351
|
|
|
161
|
-
|
|
162
|
-
统一 CLI:`--result <result.json> --out <out.json> [--lang zh|en]`:
|
|
352
|
+
---
|
|
163
353
|
|
|
354
|
+
## 18. Use Unified Visualization Adapter Envelopes
|
|
355
|
+
Keep visualization adapters contract-compatible.
|
|
356
|
+
Expected envelope fields:
|
|
357
|
+
```text
|
|
358
|
+
adapter
|
|
359
|
+
contract_version
|
|
360
|
+
source_ref
|
|
361
|
+
source_sha256
|
|
362
|
+
locale
|
|
363
|
+
data
|
|
364
|
+
```
|
|
365
|
+
Example commands:
|
|
164
366
|
```bash
|
|
165
367
|
python3 visualization/eduevidence-report/scripts/build_charts.py \
|
|
166
|
-
|
|
368
|
+
--result examples/ai-coding-assistant-evidence/result.json \
|
|
369
|
+
--out /tmp/charts.json
|
|
167
370
|
python3 visualization/eduevidence-report/scripts/build_infographics.py \
|
|
168
|
-
|
|
371
|
+
--result examples/ai-coding-assistant-evidence/result.json \
|
|
372
|
+
--out /tmp/infographics.json
|
|
169
373
|
python3 visualization/eduevidence-report/scripts/build_figures.py \
|
|
170
|
-
|
|
374
|
+
--result examples/ai-coding-assistant-evidence/result.json \
|
|
375
|
+
--out /tmp/figures.json \
|
|
376
|
+
--theme okabe_ito
|
|
171
377
|
```
|
|
378
|
+
Treat ECharts option data as an adapter format. Legacy browser enhancement is optional; the read-only Research Studio uses bundled React/SVG rendering and does not require a remote ECharts runtime.
|
|
172
379
|
|
|
173
|
-
|
|
174
|
-
/ `locale` / `data`。报告渲染器 `build_report.py` 与 CLI 共享同一核心函数。
|
|
175
|
-
(`build_figures.py --out-dir` 为兼容参数,迁移后移除。)
|
|
380
|
+
---
|
|
176
381
|
|
|
177
|
-
##
|
|
382
|
+
## 19. Use Research Studio Only as an Inspection Surface
|
|
383
|
+
Start the read-only local research workspace:
|
|
384
|
+
```bash
|
|
385
|
+
python3 scripts/dashboard_server.py --host 127.0.0.1 --port 8765
|
|
386
|
+
```
|
|
387
|
+
Open `http://127.0.0.1:8765/studio/`. Inspect local Projects, recorded Runs, staging/committed artifacts, Evidence Graph relationships, revision ancestry and generated reports. Keep project research iterations separate from Skill Autoevolve experiments.
|
|
388
|
+
Do not route scientific reasoning or research mutations through the console. It cannot create studies, dispatch workers, stop runs, approve pilots or write canonical research state. Search, filtering, language and appearance controls change browser state only.
|
|
389
|
+
Display missing artifacts, partial uncertainty and stale DecisionSnapshots explicitly. Never invent a conclusion, scientific gate result, progress percentage or pooled effect to fill a screen. Only show report variants that actually exist.
|
|
390
|
+
Use `scripts/build_report_variants.py` during build to generate the five existing report identities from complete bilingual inputs. Preserve independent offline HTML, source provenance and scientific content across themes.
|
|
391
|
+
Ship the prebuilt `web/studio/` resources with the Skill; Node is a development dependency only. Export public examples, never private `EDUEVIDENCE_HOME` research state, to static hosting. See `docs/research-studio-guide.zh-CN.md` for workflow and delivery contracts.
|
|
392
|
+
|
|
393
|
+
---
|
|
178
394
|
|
|
395
|
+
## 20. Use CLI Utilities When Deterministic Execution Helps
|
|
179
396
|
```bash
|
|
180
|
-
#
|
|
397
|
+
# Start the local Web Studio
|
|
181
398
|
python3 scripts/dashboard_server.py --port 8765
|
|
182
|
-
|
|
183
|
-
# 2. 学术与实时趋势检索
|
|
399
|
+
# Search academic and current evidence
|
|
184
400
|
python3 -m retrieval.search "AI coding assistants learning transfer"
|
|
401
|
+
# Run the DID fixture / empirical analysis path
|
|
402
|
+
python3 scripts/did_regression.py <your.csv> # needs treat / post / outcome columns
|
|
403
|
+
# column names are matched case-insensitively; see scripts/did_regression.py
|
|
404
|
+
# Compute an effect size
|
|
405
|
+
python3 scripts/effect_calculator.py \
|
|
406
|
+
--mean1 78.5 --sd1 10.2 --n1 90 \
|
|
407
|
+
--mean2 72.1 --sd2 11.0 --n2 90
|
|
408
|
+
# Check Skill consistency
|
|
409
|
+
python3 scripts/skill_lint.py
|
|
410
|
+
```
|
|
411
|
+
Prefer deterministic scripts for calculations, validation, schema checks, and reproducible transformations. Do not replace a deterministic computation with invented model arithmetic when a validated script is available.
|
|
185
412
|
|
|
186
|
-
|
|
187
|
-
# 示例用全研究周期夹具的 40 行匿名 DID 数据,无 PII)
|
|
188
|
-
python3 scripts/did_regression.py examples/full-research-cycle-fixture/data.csv
|
|
413
|
+
---
|
|
189
414
|
|
|
190
|
-
|
|
191
|
-
|
|
415
|
+
## 21. Final Output Requirements
|
|
416
|
+
Before presenting a final decision-grade result, verify that the output makes the following explicit when relevant:
|
|
417
|
+
- Canonical research question.
|
|
418
|
+
- Target population and context.
|
|
419
|
+
- Intervention and comparison.
|
|
420
|
+
- Decision-relevant outcomes.
|
|
421
|
+
- Evidence provenance.
|
|
422
|
+
- Supporting evidence.
|
|
423
|
+
- Counter-evidence and null evidence.
|
|
424
|
+
- Methodological limitations.
|
|
425
|
+
- Evidence certainty.
|
|
426
|
+
- Applicability boundary.
|
|
427
|
+
- Current verdict.
|
|
428
|
+
- Conditions that would change the verdict.
|
|
429
|
+
- Evidence-grounded gaps.
|
|
430
|
+
- Pilot/study design, when justified.
|
|
431
|
+
- Evaluation and stop conditions, when justified.
|
|
432
|
+
- Decision revision after new evidence, when applicable.
|
|
433
|
+
Prefer a scientifically bounded answer over a confident but weakly supported answer.
|
|
192
434
|
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
435
|
+
---
|
|
436
|
+
|
|
437
|
+
## 22. Core Behavioral Principle
|
|
438
|
+
Optimize for **decision integrity**, not answer confidence.
|
|
439
|
+
The central question is not:
|
|
440
|
+
> “Can the system produce a recommendation?”
|
|
441
|
+
It is:
|
|
442
|
+
> **“Is the available evidence strong, direct, applicable, and traceable enough to justify this action — and what evidence should change the decision next?”**
|
|
443
|
+
|
|
444
|
+
---
|
|
445
|
+
|
|
446
|
+
## Appendix — Repository Tooling Contract
|
|
447
|
+
|
|
448
|
+
This appendix keeps the deterministic repository gates (skill lint / version / metrics / release contract) stable while the English Skill text above remains the primary operating contract.
|
|
449
|
+
|
|
450
|
+
- **Progressive disclosure**: load the root SKILL first; load a workflow only when the task matches it; load a capability recipe (under `skill/sub-skills/`) or a methodology/domain profile only when the workflow calls for it. Do not dump the entire research archive into one agent context.
|
|
451
|
+
- **EduEvidence Research Engine** — the runnable engine delivered as a Skill package (`engine/`, `retrieval/`, `scripts/`, `schemas/`).
|
|
452
|
+
- **Shared Research Library** — verified external facts (`Source` / `Study` / `Finding` / `Audit`) may be reused across project snapshots; interpretive objects (`Claim` / `EvidenceLink` / `Applicability` / `Decision`) stay project-local.
|
|
453
|
+
- **No new study design without evidence grounding** — every study or pilot must cite an explicit, evidence-supported `KnowledgeGap` identifier.
|
|
454
|
+
- **Schema 版本口径**:schemas/ 顶层 13 个 = V1 契约(evidence.schema.json 当前修订 1.1、education-frame / verdict 等);schemas/v2/ 17 个 = V2 契约(evidence-link / research-intent / study / graph-revision / project 等)。文档与代理配置一律以此口径命名。
|
|
455
|
+
- Five baked report themes (presentation systems, not science):
|
|
456
|
+
|
|
457
|
+
├─ Claude Research [Light]
|
|
458
|
+
├─ Academic Paper [Light]
|
|
459
|
+
├─ DataLab [Light]
|
|
460
|
+
├─ DataLab [Dark]
|
|
461
|
+
└─ Presentation / Judge [Dark]
|