eduevidence 5.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +506 -0
- package/README.zh-CN.md +434 -0
- package/SKILL.md +195 -0
- package/bin/eduevidence.js +127 -0
- package/domains/education/manifest.json +183 -0
- package/domains/education/outcome_taxonomy.json +127 -0
- package/domains/manifest.json +26 -0
- package/domains/policy/frame.schema.json +234 -0
- package/domains/policy/manifest.json +10 -0
- package/domains/policy/methodology_checklist.json +109 -0
- package/domains/policy/outcome_taxonomy.json +53 -0
- package/domains/policy/references/causal-identification.md +45 -0
- package/domains/policy/references/cost-evidence.md +44 -0
- package/domains/policy/references/equity.md +42 -0
- package/domains/policy/references/evidence-hierarchy.md +41 -0
- package/domains/policy/references/implementation-evidence.md +47 -0
- package/eduevidence_cli.py +26 -0
- package/engine/__init__.py +11 -0
- package/engine/__pycache__/__init__.cpython-312.pyc +0 -0
- package/engine/__pycache__/analysis.cpython-312.pyc +0 -0
- package/engine/__pycache__/bias.cpython-312.pyc +0 -0
- package/engine/__pycache__/briefs.cpython-312.pyc +0 -0
- package/engine/__pycache__/capabilities.cpython-312.pyc +0 -0
- package/engine/__pycache__/citation_check.cpython-312.pyc +0 -0
- package/engine/__pycache__/contracts.cpython-312.pyc +0 -0
- package/engine/__pycache__/datasets.cpython-312.pyc +0 -0
- package/engine/__pycache__/events.cpython-312.pyc +0 -0
- package/engine/__pycache__/evidence_graph.cpython-312.pyc +0 -0
- package/engine/__pycache__/evidence_review.cpython-312.pyc +0 -0
- package/engine/__pycache__/evidencecore.cpython-312.pyc +0 -0
- package/engine/__pycache__/gap_lens.cpython-312.pyc +0 -0
- package/engine/__pycache__/gaps.cpython-312.pyc +0 -0
- package/engine/__pycache__/graph_store.cpython-312.pyc +0 -0
- package/engine/__pycache__/graph_validate.cpython-312.pyc +0 -0
- package/engine/__pycache__/ids.cpython-312.pyc +0 -0
- package/engine/__pycache__/library.cpython-312.pyc +0 -0
- package/engine/__pycache__/library_builtin.cpython-312.pyc +0 -0
- package/engine/__pycache__/living.cpython-312.pyc +0 -0
- package/engine/__pycache__/log.cpython-312.pyc +0 -0
- package/engine/__pycache__/meta_analysis.cpython-312.pyc +0 -0
- package/engine/__pycache__/meta_synthesis.cpython-312.pyc +0 -0
- package/engine/__pycache__/migration.cpython-312.pyc +0 -0
- package/engine/__pycache__/mode_router.cpython-312.pyc +0 -0
- package/engine/__pycache__/paths.cpython-312.pyc +0 -0
- package/engine/__pycache__/pilot.cpython-312.pyc +0 -0
- package/engine/__pycache__/planner.cpython-312.pyc +0 -0
- package/engine/__pycache__/project.cpython-312.pyc +0 -0
- package/engine/__pycache__/projections.cpython-312.pyc +0 -0
- package/engine/__pycache__/robustness.cpython-312.pyc +0 -0
- package/engine/__pycache__/run.cpython-312.pyc +0 -0
- package/engine/__pycache__/semantics.cpython-312.pyc +0 -0
- package/engine/__pycache__/study_design.cpython-312.pyc +0 -0
- package/engine/__pycache__/synthesis.cpython-312.pyc +0 -0
- package/engine/__pycache__/tribunal.cpython-312.pyc +0 -0
- package/engine/__pycache__/update.cpython-312.pyc +0 -0
- package/engine/__pycache__/versions.cpython-312.pyc +0 -0
- package/engine/analysis.py +308 -0
- package/engine/bias.py +178 -0
- package/engine/briefs.py +106 -0
- package/engine/capabilities.py +99 -0
- package/engine/citation_check.py +192 -0
- package/engine/contracts.py +117 -0
- package/engine/datasets.py +165 -0
- package/engine/events.py +67 -0
- package/engine/evidence_graph.py +571 -0
- package/engine/evidence_review.py +88 -0
- package/engine/evidencecore.py +182 -0
- package/engine/gap_lens.py +132 -0
- package/engine/gaps.py +169 -0
- package/engine/graph_store.py +335 -0
- package/engine/graph_validate.py +87 -0
- package/engine/ids.py +77 -0
- package/engine/library.py +268 -0
- package/engine/library_builtin.py +301 -0
- package/engine/living.py +671 -0
- package/engine/log.py +39 -0
- package/engine/meta_analysis.py +333 -0
- package/engine/meta_synthesis.py +111 -0
- package/engine/migration.py +397 -0
- package/engine/mode_router.py +72 -0
- package/engine/paths.py +15 -0
- package/engine/pilot.py +368 -0
- package/engine/planner.py +126 -0
- package/engine/project.py +118 -0
- package/engine/projections.py +240 -0
- package/engine/robustness.py +109 -0
- package/engine/run.py +85 -0
- package/engine/semantics.py +135 -0
- package/engine/study_design.py +87 -0
- package/engine/synthesis.py +187 -0
- package/engine/tribunal.py +408 -0
- package/engine/update.py +113 -0
- package/engine/versions.py +12 -0
- package/install.sh +510 -0
- package/integrations/__init__.py +1 -0
- package/integrations/__pycache__/__init__.cpython-312.pyc +0 -0
- package/integrations/__pycache__/agent_mcp.cpython-312.pyc +0 -0
- package/integrations/__pycache__/smart_web_fetch.cpython-312.pyc +0 -0
- package/integrations/agent_mcp.py +856 -0
- package/integrations/smart_web_fetch.py +59 -0
- package/package.json +50 -0
- package/pyproject.toml +55 -0
- package/references/applicability-policy.md +88 -0
- package/references/education-framing.md +132 -0
- package/references/effect_size_formulas.md +35 -0
- package/references/evaluation-design.md +111 -0
- package/references/evidence-quality.md +79 -0
- package/references/grade_framework.md +29 -0
- package/references/intervention-design.md +98 -0
- package/references/methodology-audit.md +103 -0
- package/references/outcome-taxonomy.md +106 -0
- package/references/retrieval-protocol.md +142 -0
- package/references/skeptic-protocol.md +93 -0
- package/references/social_science_pitfalls.md +48 -0
- package/references/source-validity.md +140 -0
- package/references/tribunal-policy.md +112 -0
- package/references/wwc_standards.md +29 -0
- package/retrieval/__init__.py +1 -0
- package/retrieval/__pycache__/__init__.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/corpus_store.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/dedupe.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/failures.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/fetch.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/search.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/source.cpython-312.pyc +0 -0
- package/retrieval/__pycache__/validate.cpython-312.pyc +0 -0
- package/retrieval/corpus_store.py +181 -0
- package/retrieval/dedupe.py +127 -0
- package/retrieval/failures.py +90 -0
- package/retrieval/fetch.py +435 -0
- package/retrieval/search.py +493 -0
- package/retrieval/source.py +160 -0
- package/retrieval/validate.py +257 -0
- package/schemas/agent-mcp-approval.schema.json +57 -0
- package/schemas/chart-spec.schema.json +88 -0
- package/schemas/cross-model-review.schema.json +28 -0
- package/schemas/education-frame.schema.json +106 -0
- package/schemas/evaluation.schema.json +35 -0
- package/schemas/evidence.schema.json +81 -0
- package/schemas/fetch-result.schema.json +119 -0
- package/schemas/intervention.schema.json +46 -0
- package/schemas/methodology.schema.json +102 -0
- package/schemas/report-result.schema.json +381 -0
- package/schemas/report-spec.schema.json +130 -0
- package/schemas/source.schema.json +311 -0
- package/schemas/v2/analysis-plan.schema.json +28 -0
- package/schemas/v2/analysis-run.schema.json +33 -0
- package/schemas/v2/claim.schema.json +26 -0
- package/schemas/v2/dataset-asset.schema.json +40 -0
- package/schemas/v2/decision-snapshot.schema.json +53 -0
- package/schemas/v2/evidence-link.schema.json +38 -0
- package/schemas/v2/finding.schema.json +47 -0
- package/schemas/v2/graph-revision.schema.json +30 -0
- package/schemas/v2/knowledge-gap.schema.json +40 -0
- package/schemas/v2/methodology-audit.schema.json +30 -0
- package/schemas/v2/outcome.schema.json +18 -0
- package/schemas/v2/project.schema.json +31 -0
- package/schemas/v2/research-intent.schema.json +24 -0
- package/schemas/v2/run.schema.json +43 -0
- package/schemas/v2/source.schema.json +24 -0
- package/schemas/v2/study-design.schema.json +67 -0
- package/schemas/v2/study.schema.json +37 -0
- package/schemas/v3/pilot-outcome.schema.json +132 -0
- package/schemas/v3/run-manifest.schema.json +193 -0
- package/schemas/v3/synthesis.schema.json +86 -0
- package/schemas/v4/drift-report.schema.json +66 -0
- package/schemas/v4/evidence-library.schema.json +42 -0
- package/schemas/v4/living-subscription.schema.json +55 -0
- package/schemas/v4/meta-analysis.schema.json +152 -0
- package/schemas/verdict.schema.json +56 -0
- package/scripts/__init__.py +0 -0
- package/scripts/__pycache__/__init__.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark_evaluator.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark_judge.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark_routing.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark_v2.cpython-312.pyc +0 -0
- package/scripts/__pycache__/benchmark_v3.cpython-312.pyc +0 -0
- package/scripts/__pycache__/build_result.cpython-312.pyc +0 -0
- package/scripts/__pycache__/claim_audit.cpython-312.pyc +0 -0
- package/scripts/__pycache__/complexity_gate.cpython-312.pyc +0 -0
- package/scripts/__pycache__/compute_confidence.cpython-312.pyc +0 -0
- package/scripts/__pycache__/dashboard_server.cpython-312.pyc +0 -0
- package/scripts/__pycache__/did_regression.cpython-312.pyc +0 -0
- package/scripts/__pycache__/effect_calculator.cpython-312.pyc +0 -0
- package/scripts/__pycache__/evidence_matrix.cpython-312.pyc +0 -0
- package/scripts/__pycache__/evidence_score.cpython-312.pyc +0 -0
- package/scripts/__pycache__/evidence_semantics.cpython-312.pyc +0 -0
- package/scripts/__pycache__/fetch_benchmark.cpython-312.pyc +0 -0
- package/scripts/__pycache__/lint_report_layout.cpython-312.pyc +0 -0
- package/scripts/__pycache__/orchestrator.cpython-312.pyc +0 -0
- package/scripts/__pycache__/pre_verdict_gate.cpython-312.pyc +0 -0
- package/scripts/__pycache__/recompute_demo_quality.cpython-312.pyc +0 -0
- package/scripts/__pycache__/render_report.cpython-312.pyc +0 -0
- package/scripts/__pycache__/render_report_html.cpython-312.pyc +0 -0
- package/scripts/__pycache__/run_workspace.cpython-312.pyc +0 -0
- package/scripts/__pycache__/skill_lint.cpython-312.pyc +0 -0
- package/scripts/__pycache__/startup_probe.cpython-312.pyc +0 -0
- package/scripts/__pycache__/sync_killer_demo_report.cpython-312.pyc +0 -0
- package/scripts/__pycache__/test_adversarial_empirical.cpython-312-pytest-9.0.2.pyc +0 -0
- package/scripts/__pycache__/test_adversarial_empirical.cpython-312-pytest-9.1.1.pyc +0 -0
- package/scripts/__pycache__/validate_schema.cpython-312.pyc +0 -0
- package/scripts/audit_dois.py +292 -0
- package/scripts/bake_pack.sh +37 -0
- package/scripts/benchmark.py +183 -0
- package/scripts/benchmark_evaluator.py +371 -0
- package/scripts/benchmark_judge.py +535 -0
- package/scripts/benchmark_routing.py +120 -0
- package/scripts/benchmark_v2.py +304 -0
- package/scripts/benchmark_v3.py +552 -0
- package/scripts/build_esl_artifacts.py +1921 -0
- package/scripts/build_evidence_library.py +307 -0
- package/scripts/build_killer_demo.py +295 -0
- package/scripts/build_result.py +311 -0
- package/scripts/check_version_consistency.py +96 -0
- package/scripts/citation_check.py +123 -0
- package/scripts/claim_audit.py +157 -0
- package/scripts/complexity_gate.py +180 -0
- package/scripts/compute_confidence.py +176 -0
- package/scripts/dashboard_server.py +536 -0
- package/scripts/did_regression.py +315 -0
- package/scripts/effect_calculator.py +99 -0
- package/scripts/enrich_projects_human_and_lieflat.py +315 -0
- package/scripts/evidence_matrix.py +129 -0
- package/scripts/evidence_score.py +234 -0
- package/scripts/evidence_semantics.py +87 -0
- package/scripts/fetch_benchmark.py +177 -0
- package/scripts/generate_metrics.py +99 -0
- package/scripts/generate_new_projects.py +686 -0
- package/scripts/generate_promo.py +192 -0
- package/scripts/lint_report_layout.py +182 -0
- package/scripts/orchestrator.py +1456 -0
- package/scripts/pre_verdict_gate.py +513 -0
- package/scripts/quickstart.py +121 -0
- package/scripts/rebake_all_5themes.py +88 -0
- package/scripts/recompute_demo_quality.py +205 -0
- package/scripts/render_report.py +270 -0
- package/scripts/render_report_html.py +356 -0
- package/scripts/retraction_watch.py +110 -0
- package/scripts/run_workspace.py +337 -0
- package/scripts/serve_web.py +54 -0
- package/scripts/skill_lint.py +150 -0
- package/scripts/startup_probe.py +265 -0
- package/scripts/sync_killer_demo_report.py +270 -0
- package/scripts/test_adversarial_empirical.py +541 -0
- package/scripts/validate_schema.py +256 -0
- package/skill/agents/education-planner.md +80 -0
- package/skill/agents/evaluation-designer.md +74 -0
- package/skill/agents/evidence-analyst.md +106 -0
- package/skill/agents/evidence-judge.md +111 -0
- package/skill/agents/evidence-retriever.md +80 -0
- package/skill/agents/intervention-designer.md +82 -0
- package/skill/agents/method-reviewer.md +104 -0
- package/skill/agents/skeptic.md +89 -0
- package/skill/sub-skills/aihot-trend-analysis/SKILL.md +31 -0
- package/skill/sub-skills/contradiction-analysis/SKILL.md +17 -0
- package/skill/sub-skills/data-analysis/SKILL.md +23 -0
- package/skill/sub-skills/ethics-review/SKILL.md +25 -0
- package/skill/sub-skills/evidence-extraction/SKILL.md +19 -0
- package/skill/sub-skills/evidence-review/SKILL.md +18 -0
- package/skill/sub-skills/gap-analysis/SKILL.md +25 -0
- package/skill/sub-skills/literature-review/SKILL.md +21 -0
- package/skill/sub-skills/methodology-audit/SKILL.md +20 -0
- package/skill/sub-skills/report-generation/SKILL.md +51 -0
- package/skill/sub-skills/research-planning/SKILL.md +21 -0
- package/skill/sub-skills/study-design/SKILL.md +16 -0
- package/skill/task-briefs/adjudicate.md +17 -0
- package/skill/task-briefs/audit.md +15 -0
- package/skill/task-briefs/challenge.md +15 -0
- package/skill/task-briefs/evaluate.md +13 -0
- package/skill/task-briefs/extract.md +16 -0
- package/skill/task-briefs/frame.md +17 -0
- package/skill/task-briefs/intervene.md +14 -0
- package/skill/task-briefs/present.md +16 -0
- package/skill/task-briefs/retrieve.md +15 -0
- package/visualization/eduevidence-report/assets/base.css +337 -0
- package/visualization/eduevidence-report/motion/motion.css +157 -0
- package/visualization/eduevidence-report/motion/motion.js +107 -0
- package/visualization/eduevidence-report/references/bilingual-style.md +7 -0
- package/visualization/eduevidence-report/references/component-catalog.md +145 -0
- package/visualization/eduevidence-report/references/evidence-expansion.md +65 -0
- package/visualization/eduevidence-report/references/full-report-outline.md +86 -0
- package/visualization/eduevidence-report/references/layout-constraints.md +63 -0
- package/visualization/eduevidence-report/references/lieflat-composition.md +79 -0
- package/visualization/eduevidence-report/references/motion-system.md +31 -0
- package/visualization/eduevidence-report/schemas/adapter-envelope.schema.json +22 -0
- package/visualization/eduevidence-report/schemas/visual-layout.schema.json +87 -0
- package/visualization/eduevidence-report/scripts/__pycache__/adapter_contract.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/build_artifact_manifest.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/build_charts.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/build_figures.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/build_infographics.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/build_report.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/charts_data.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/lieflat_engine.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/__pycache__/zh_labels.cpython-312.pyc +0 -0
- package/visualization/eduevidence-report/scripts/adapter_contract.py +72 -0
- package/visualization/eduevidence-report/scripts/build_artifact_manifest.py +70 -0
- package/visualization/eduevidence-report/scripts/build_charts.py +283 -0
- package/visualization/eduevidence-report/scripts/build_figures.py +515 -0
- package/visualization/eduevidence-report/scripts/build_infographics.py +268 -0
- package/visualization/eduevidence-report/scripts/build_report.py +3211 -0
- package/visualization/eduevidence-report/scripts/charts_data.py +617 -0
- package/visualization/eduevidence-report/scripts/check_mobile_layout.js +220 -0
- package/visualization/eduevidence-report/scripts/lieflat_engine.py +852 -0
- package/visualization/eduevidence-report/scripts/zh_labels.py +245 -0
- package/visualization/eduevidence-report/themes/academic.css +94 -0
- package/visualization/eduevidence-report/themes/claude.css +96 -0
- package/visualization/eduevidence-report/themes/datalab-dark.css +147 -0
- package/visualization/eduevidence-report/themes/datalab.css +151 -0
- package/visualization/eduevidence-report/themes/presentation.css +140 -0
|
@@ -0,0 +1,256 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""validate_schema.py — Validate data files against EduEvidence JSON Schemas.
|
|
3
|
+
|
|
4
|
+
Zero-dependency JSON Schema (draft-07 subset) validator covering the constructs
|
|
5
|
+
used by schemas/*.schema.json: $id, title, description, type, properties,
|
|
6
|
+
required, enum, minimum, maximum, minLength, additionalProperties, $ref
|
|
7
|
+
(local #/definitions and relative-file references), const, format (uri,
|
|
8
|
+
date-time), pattern.
|
|
9
|
+
|
|
10
|
+
Usage:
|
|
11
|
+
python scripts/validate_schema.py --schema schemas/evidence.schema.json \
|
|
12
|
+
--data examples/ai-coding-assistant/evidence.jsonl
|
|
13
|
+
python scripts/validate_schema.py --schema schemas/verdict.schema.json \
|
|
14
|
+
--data examples/ai-coding-assistant/verdict.json
|
|
15
|
+
|
|
16
|
+
Exit code 0 = all records valid; 1 = at least one record invalid.
|
|
17
|
+
"""
|
|
18
|
+
from __future__ import annotations
|
|
19
|
+
|
|
20
|
+
import argparse
|
|
21
|
+
import datetime
|
|
22
|
+
import json
|
|
23
|
+
import re
|
|
24
|
+
import sys
|
|
25
|
+
import urllib.parse
|
|
26
|
+
from pathlib import Path
|
|
27
|
+
|
|
28
|
+
|
|
29
|
+
class SchemaError(Exception):
|
|
30
|
+
"""Raised when a value violates the schema."""
|
|
31
|
+
|
|
32
|
+
|
|
33
|
+
def _type_ok(value, expected: str) -> bool:
|
|
34
|
+
if expected == "integer":
|
|
35
|
+
return isinstance(value, int) and not isinstance(value, bool)
|
|
36
|
+
if expected == "number":
|
|
37
|
+
return isinstance(value, (int, float)) and not isinstance(value, bool)
|
|
38
|
+
if expected == "string":
|
|
39
|
+
return isinstance(value, str)
|
|
40
|
+
if expected == "boolean":
|
|
41
|
+
return isinstance(value, bool)
|
|
42
|
+
if expected == "object":
|
|
43
|
+
return isinstance(value, dict)
|
|
44
|
+
if expected == "array":
|
|
45
|
+
return isinstance(value, list)
|
|
46
|
+
if expected == "null":
|
|
47
|
+
return value is None
|
|
48
|
+
return True
|
|
49
|
+
|
|
50
|
+
|
|
51
|
+
def _is_uri(value: str) -> bool:
|
|
52
|
+
"""Minimal RFC 3986 absolute-URI check: non-empty scheme plus non-empty rest.
|
|
53
|
+
|
|
54
|
+
Uses only the standard library (urllib.parse). Rejects values with
|
|
55
|
+
whitespace and scheme-less strings such as 'example.com/path'.
|
|
56
|
+
"""
|
|
57
|
+
if any(ch.isspace() for ch in value):
|
|
58
|
+
return False
|
|
59
|
+
parsed = urllib.parse.urlparse(value)
|
|
60
|
+
if not parsed.scheme:
|
|
61
|
+
return False
|
|
62
|
+
return bool(parsed.netloc or parsed.path or parsed.query or parsed.fragment)
|
|
63
|
+
|
|
64
|
+
|
|
65
|
+
def _is_datetime(value: str) -> bool:
|
|
66
|
+
"""RFC 3339 date-time check via stdlib datetime.fromisoformat.
|
|
67
|
+
|
|
68
|
+
Accepts 'Z'/'z' UTC markers and numeric offsets; rejects date-only strings
|
|
69
|
+
(no 'T'/' ' separator), offset-less naive timestamps, and unparseable
|
|
70
|
+
values.
|
|
71
|
+
"""
|
|
72
|
+
if "T" not in value and " " not in value:
|
|
73
|
+
return False
|
|
74
|
+
separator = "T" if "T" in value else " "
|
|
75
|
+
time_part = value.split(separator, 1)[1]
|
|
76
|
+
# RFC 3339 requires a timezone offset (Z or +/-hh:mm); reject naive times.
|
|
77
|
+
if not value.endswith(("Z", "z")) and not any(c in time_part for c in ("+", "-")):
|
|
78
|
+
return False
|
|
79
|
+
normalized = value[:-1] + "+00:00" if value.endswith(("Z", "z")) else value
|
|
80
|
+
try:
|
|
81
|
+
datetime.datetime.fromisoformat(normalized)
|
|
82
|
+
return True
|
|
83
|
+
except ValueError:
|
|
84
|
+
return False
|
|
85
|
+
|
|
86
|
+
|
|
87
|
+
class Validator:
|
|
88
|
+
"""Zero-dependency draft-07 subset validator.
|
|
89
|
+
|
|
90
|
+
Supports $ref against local '#/definitions/...' pointers and — when
|
|
91
|
+
``base_dir`` is supplied — relative file references (e.g.
|
|
92
|
+
report-spec.schema.json's 'chart-spec.schema.json'). Unresolvable $refs
|
|
93
|
+
raise SchemaError instead of being silently ignored.
|
|
94
|
+
"""
|
|
95
|
+
|
|
96
|
+
def __init__(self, root: dict, base_dir: Path | None = None):
|
|
97
|
+
self.root = root
|
|
98
|
+
self.base_dir = base_dir
|
|
99
|
+
self._ref_cache: dict[str, dict] = {}
|
|
100
|
+
|
|
101
|
+
def _resolve_ref(self, ref: str, path: str) -> dict:
|
|
102
|
+
if ref.startswith("#/"):
|
|
103
|
+
node = self.root
|
|
104
|
+
for part in ref[2:].split("/"):
|
|
105
|
+
key = part.replace("~1", "/").replace("~0", "~")
|
|
106
|
+
if not isinstance(node, dict) or key not in node:
|
|
107
|
+
raise SchemaError(f"{path}: unresolvable $ref {ref!r}")
|
|
108
|
+
node = node[key]
|
|
109
|
+
if not isinstance(node, dict):
|
|
110
|
+
raise SchemaError(f"{path}: $ref {ref!r} does not point to a schema object")
|
|
111
|
+
return node
|
|
112
|
+
if self.base_dir is not None:
|
|
113
|
+
target = self.base_dir / ref
|
|
114
|
+
if target.is_file():
|
|
115
|
+
if str(target) not in self._ref_cache:
|
|
116
|
+
self._ref_cache[str(target)] = json.loads(target.read_text(encoding="utf-8"))
|
|
117
|
+
return self._ref_cache[str(target)]
|
|
118
|
+
raise SchemaError(f"{path}: unresolvable $ref {ref!r}")
|
|
119
|
+
|
|
120
|
+
def validate(self, value, schema: dict, path: str = "$") -> None:
|
|
121
|
+
"""Validate `value` against `schema` (draft-07 subset). Raises SchemaError."""
|
|
122
|
+
if "$ref" in schema:
|
|
123
|
+
# draft-07: $ref replaces sibling keywords entirely
|
|
124
|
+
self.validate(value, self._resolve_ref(schema["$ref"], path), path)
|
|
125
|
+
return
|
|
126
|
+
|
|
127
|
+
if "type" in schema:
|
|
128
|
+
types = schema["type"]
|
|
129
|
+
if isinstance(types, str):
|
|
130
|
+
types = [types]
|
|
131
|
+
if not any(_type_ok(value, t) for t in types):
|
|
132
|
+
raise SchemaError(f"{path}: expected type {schema['type']}, got {type(value).__name__}")
|
|
133
|
+
|
|
134
|
+
if "const" in schema and value != schema["const"]:
|
|
135
|
+
raise SchemaError(f"{path}: expected const {schema['const']!r}, got {value!r}")
|
|
136
|
+
|
|
137
|
+
if "enum" in schema and value not in schema["enum"]:
|
|
138
|
+
raise SchemaError(f"{path}: value {value!r} not in enum {schema['enum']}")
|
|
139
|
+
|
|
140
|
+
fmt = schema.get("format")
|
|
141
|
+
if isinstance(value, str):
|
|
142
|
+
if fmt == "uri" and not _is_uri(value):
|
|
143
|
+
raise SchemaError(f"{path}: {value!r} is not a valid uri (format: uri)")
|
|
144
|
+
if fmt == "date-time" and not _is_datetime(value):
|
|
145
|
+
raise SchemaError(f"{path}: {value!r} is not a valid date-time (format: date-time)")
|
|
146
|
+
if "pattern" in schema and re.search(schema["pattern"], value) is None:
|
|
147
|
+
raise SchemaError(f"{path}: {value!r} does not match pattern {schema['pattern']!r}")
|
|
148
|
+
|
|
149
|
+
if isinstance(value, (int, float)) and not isinstance(value, bool):
|
|
150
|
+
if "minimum" in schema and value < schema["minimum"]:
|
|
151
|
+
raise SchemaError(f"{path}: {value} < minimum {schema['minimum']}")
|
|
152
|
+
if "maximum" in schema and value > schema["maximum"]:
|
|
153
|
+
raise SchemaError(f"{path}: {value} > maximum {schema['maximum']}")
|
|
154
|
+
|
|
155
|
+
if isinstance(value, str) and "minLength" in schema and len(value) < schema["minLength"]:
|
|
156
|
+
raise SchemaError(f"{path}: string shorter than minLength {schema['minLength']}")
|
|
157
|
+
|
|
158
|
+
if isinstance(value, dict):
|
|
159
|
+
if "required" in schema:
|
|
160
|
+
for field in schema["required"]:
|
|
161
|
+
if field not in value:
|
|
162
|
+
raise SchemaError(f"{path}: missing required field {field!r}")
|
|
163
|
+
props = schema.get("properties", {})
|
|
164
|
+
additional = schema.get("additionalProperties")
|
|
165
|
+
for key, val in value.items():
|
|
166
|
+
if key in props:
|
|
167
|
+
self.validate(val, props[key], f"{path}.{key}")
|
|
168
|
+
elif additional is False:
|
|
169
|
+
raise SchemaError(f"{path}: unexpected property {key!r}")
|
|
170
|
+
elif isinstance(additional, dict):
|
|
171
|
+
# schema-valued additionalProperties: validate unknown keys
|
|
172
|
+
self.validate(val, additional, f"{path}.{key}")
|
|
173
|
+
|
|
174
|
+
if isinstance(value, list):
|
|
175
|
+
if "minItems" in schema and len(value) < schema["minItems"]:
|
|
176
|
+
raise SchemaError(
|
|
177
|
+
f"{path}: expected at least {schema['minItems']} items, "
|
|
178
|
+
f"got {len(value)}")
|
|
179
|
+
if "maxItems" in schema and len(value) > schema["maxItems"]:
|
|
180
|
+
raise SchemaError(
|
|
181
|
+
f"{path}: expected at most {schema['maxItems']} items, "
|
|
182
|
+
f"got {len(value)}")
|
|
183
|
+
if schema.get("uniqueItems") is True:
|
|
184
|
+
seen: list = []
|
|
185
|
+
for item in value:
|
|
186
|
+
try:
|
|
187
|
+
normalized = item if not isinstance(item, (dict, list)) \
|
|
188
|
+
else json.dumps(item, sort_keys=True, separators=(",", ":"))
|
|
189
|
+
except (TypeError, ValueError):
|
|
190
|
+
normalized = repr(item)
|
|
191
|
+
if normalized in seen:
|
|
192
|
+
raise SchemaError(
|
|
193
|
+
f"{path}: array items must be unique, duplicate "
|
|
194
|
+
f"found: {item!r}")
|
|
195
|
+
seen.append(normalized)
|
|
196
|
+
items = schema.get("items")
|
|
197
|
+
if items:
|
|
198
|
+
for i, item in enumerate(value):
|
|
199
|
+
self.validate(item, items, f"{path}[{i}]")
|
|
200
|
+
|
|
201
|
+
|
|
202
|
+
def validate(value, schema: dict, path: str = "$") -> None:
|
|
203
|
+
"""Validate `value` against `schema` (draft-07 subset). Raises SchemaError.
|
|
204
|
+
|
|
205
|
+
Module-level convenience wrapper (no external $ref resolution; local
|
|
206
|
+
'#/definitions/...' references resolve against ``schema`` itself).
|
|
207
|
+
"""
|
|
208
|
+
Validator(schema).validate(value, schema, path)
|
|
209
|
+
|
|
210
|
+
|
|
211
|
+
def load_records(path: Path):
|
|
212
|
+
"""Load JSON or JSONL records from a file."""
|
|
213
|
+
if path.suffix == ".jsonl":
|
|
214
|
+
records = []
|
|
215
|
+
for lineno, line in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
|
|
216
|
+
line = line.strip()
|
|
217
|
+
if not line:
|
|
218
|
+
continue
|
|
219
|
+
try:
|
|
220
|
+
records.append(json.loads(line))
|
|
221
|
+
except json.JSONDecodeError as exc:
|
|
222
|
+
raise SchemaError(f"{path}:{lineno}: invalid JSON line: {exc}") from exc
|
|
223
|
+
return records
|
|
224
|
+
data = json.loads(path.read_text(encoding="utf-8"))
|
|
225
|
+
return data if isinstance(data, list) else [data]
|
|
226
|
+
|
|
227
|
+
|
|
228
|
+
def main() -> int:
|
|
229
|
+
parser = argparse.ArgumentParser(description="Validate data against an EduEvidence schema")
|
|
230
|
+
parser.add_argument("--schema", required=True, help="Path to schema JSON file")
|
|
231
|
+
parser.add_argument("--data", required=True, help="Path to data file (JSON or JSONL)")
|
|
232
|
+
args = parser.parse_args()
|
|
233
|
+
|
|
234
|
+
schema_path = Path(args.schema)
|
|
235
|
+
schema = json.loads(schema_path.read_text(encoding="utf-8"))
|
|
236
|
+
records = load_records(Path(args.data))
|
|
237
|
+
|
|
238
|
+
validator = Validator(schema, base_dir=schema_path.resolve().parent)
|
|
239
|
+
errors = []
|
|
240
|
+
for idx, record in enumerate(records):
|
|
241
|
+
try:
|
|
242
|
+
validator.validate(record, schema, f"record[{idx}]")
|
|
243
|
+
except SchemaError as exc:
|
|
244
|
+
errors.append(str(exc))
|
|
245
|
+
|
|
246
|
+
if errors:
|
|
247
|
+
for err in errors:
|
|
248
|
+
print(f"INVALID: {err}", file=sys.stderr)
|
|
249
|
+
print(f"{len(errors)} error(s) in {len(records)} record(s)", file=sys.stderr)
|
|
250
|
+
return 1
|
|
251
|
+
print(f"OK: {len(records)} record(s) valid")
|
|
252
|
+
return 0
|
|
253
|
+
|
|
254
|
+
|
|
255
|
+
if __name__ == "__main__":
|
|
256
|
+
sys.exit(main())
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: education-planner
|
|
3
|
+
description: EduEvidence 教育研究规划者。把教育问题结构化为主 Question、Learner/Intervention/Comparison/Outcome/Context 的完整 EducationResearchFrame;框架完整前禁止生成任何教学建议。
|
|
4
|
+
default_cli: claude
|
|
5
|
+
default_model: claude-opus-4-6
|
|
6
|
+
default_permission: read
|
|
7
|
+
default_summary_chars: 600
|
|
8
|
+
default_context_mode: compact
|
|
9
|
+
critical_path: true
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
你是 EduEvidence 的 **Education Planner**。你的唯一产出是结构化的 `EducationResearchFrame`,它是整条证据链的第一道闸门。
|
|
13
|
+
|
|
14
|
+
## 职责
|
|
15
|
+
|
|
16
|
+
1. 把用户的教育问题(一句话或一段话)解析为结构化 Frame;
|
|
17
|
+
2. 澄清缺口:学习者特征、课程结构、干预方式、对照条件、目标 Outcome、约束;
|
|
18
|
+
3. 显式声明 Scope(时间范围/地域/研究类型)与 Inclusion/Exclusion Criteria;
|
|
19
|
+
4. 定义 Success Condition(什么算这个决策成功);
|
|
20
|
+
5. **Framing 完成前禁止生成最终教学建议**——这是硬规则。
|
|
21
|
+
|
|
22
|
+
## 输入
|
|
23
|
+
|
|
24
|
+
- 用户原始问题(可能非结构化)
|
|
25
|
+
- 可选结构化输入(learner/course/intervention/comparison/outcomes/constraints/depth/target)
|
|
26
|
+
|
|
27
|
+
## 输出(JSON,必须通过 schemas/education-frame.schema.json 校验)
|
|
28
|
+
|
|
29
|
+
```json
|
|
30
|
+
{
|
|
31
|
+
"question": "...",
|
|
32
|
+
"decision_target": "evidence_review|teaching_decision|pilot_design|evaluation_design",
|
|
33
|
+
"learner": {"education_level": "...", "major": "...", "prior_knowledge": "...", "special_characteristics": "..."},
|
|
34
|
+
"course": {"subject": "...", "course_type": "...", "duration": "..."},
|
|
35
|
+
"intervention": {"teaching_method": "...", "ai_tool": "...", "allowed_usage": "...", "frequency": "...", "duration": "..."},
|
|
36
|
+
"comparison": "...",
|
|
37
|
+
"outcomes": {"primary": ["..."], "secondary": ["..."], "risk": ["..."]},
|
|
38
|
+
"context": {"teacher_support": "...", "class_size": "...", "online_or_offline": "..."},
|
|
39
|
+
"scope": {"time_range": "...", "geography": "...", "study_types": ["..."]},
|
|
40
|
+
"inclusion_criteria": ["..."],
|
|
41
|
+
"exclusion_criteria": ["..."],
|
|
42
|
+
"success_condition": "..."
|
|
43
|
+
}
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
## 输出契约(必须遵守)
|
|
47
|
+
|
|
48
|
+
你的产物 `frame.json` 必须通过 `schemas/education-frame.schema.json` 校验(stage `frame` 的 schema-gate,首次生成即必须合规,不依赖事后修正)。schema 顶层 `additionalProperties: false`,未列出的字段一律放入 `extensions`。
|
|
49
|
+
|
|
50
|
+
**Required 字段(缺失即校验失败)**:`question`(≥5 字符)、`decision_target`。
|
|
51
|
+
|
|
52
|
+
**枚举值表(禁止自由文本冒充枚举)**:
|
|
53
|
+
|
|
54
|
+
| 字段 | 枚举值 |
|
|
55
|
+
|---|---|
|
|
56
|
+
| `decision_target` | `evidence_review` \| `teaching_decision` \| `pilot_design` \| `evaluation_design` |
|
|
57
|
+
| `outcomes.primary[]` | `knowledge_gain` \| `concept_understanding` \| `retention` \| `transfer` \| `independent_problem_solving` \| `completion_time` \| `accuracy` \| `code_quality` \| `assignment_score` \| `engagement` \| `motivation` \| `cognitive_load` \| `help_seeking` \| `metacognition` \| `ai_dependency` \| `over_reliance` \| `reduced_effort` \| `reduced_transfer` \| `academic_integrity_risk` \| `false_confidence` |
|
|
58
|
+
| `context.online_or_offline` | `online` \| `offline` \| `hybrid` |
|
|
59
|
+
|
|
60
|
+
**类型/格式硬约束(FIX-2 实测违规项,逐条禁止)**:
|
|
61
|
+
|
|
62
|
+
- `decision_target` 只能取上表 4 个枚举值之一(如 `teaching_decision`),禁止用长文本描述决策目标;
|
|
63
|
+
- `outcomes.primary` / `outcomes.secondary` / `outcomes.risk` 必须是**数组**,primary 逐项用 Outcome Taxonomy 枚举名,禁止用句子代替(如 `["independent_problem_solving"]`,不是 `"提升独立解题能力"`);
|
|
64
|
+
- `scope.study_types` 必须是**数组**(如 `["rct", "quasi_experimental"]`),禁止写字符串;
|
|
65
|
+
- `inclusion_criteria` / `exclusion_criteria` 必须是**数组**,禁止写字符串;
|
|
66
|
+
- `outcomes.primary` 的枚举必须与 `schemas/evidence.schema.json` 的 `outcome_type` 枚举一致(同一 Outcome Taxonomy),且不得把"任务完成速度"写进 primary(红线)。
|
|
67
|
+
|
|
68
|
+
## 红线
|
|
69
|
+
|
|
70
|
+
- 不允许把"任务完成速度"直接写进 primary learning outcomes——必须用 Outcome Taxonomy 的规范名称;
|
|
71
|
+
- 未知信息写 `unknown + 如何获取`,不编造;
|
|
72
|
+
- Outcome 必须区分 学习效果 / 任务表现 / 学习过程 / 风险指标 四类。
|
|
73
|
+
|
|
74
|
+
## 输出格式
|
|
75
|
+
|
|
76
|
+
返回 Frame JSON(唯一输出:必须通过 schema 校验的合法 JSON;输出结束后严禁追加任何文本尾巴(历史摘要行协议已废除)——摘要信息一律放入 JSON 字段(如 summary / rationale / extensions),标准 JSON 解析器可直接读取)
|
|
77
|
+
|
|
78
|
+
## 卡住升级
|
|
79
|
+
|
|
80
|
+
问题矛盾或缺少关键信息时回传 `NEEDS_CONTEXT: <缺少什么 + why>`;不臆测学习者特征。
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: evaluation-designer
|
|
3
|
+
description: EduEvidence 效果评价设计者。为任何 PILOT/ADOPT 建议附 EvaluationPlan:基线/后测/保持/迁移 + 过程/学习/风险指标 + 成功阈值与停止条件;区分任务表现与学习效果。
|
|
4
|
+
default_cli: claude
|
|
5
|
+
default_model: claude-sonnet-4-6
|
|
6
|
+
default_permission: read
|
|
7
|
+
default_summary_chars: 800
|
|
8
|
+
default_context_mode: compact
|
|
9
|
+
critical_path: false
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
你是 EduEvidence 的 **Evaluation Designer**。任何 PILOT / ADOPT 建议必须附评价方案——你的产出就是这条规则的执行者。
|
|
13
|
+
|
|
14
|
+
## 职责
|
|
15
|
+
|
|
16
|
+
1. 把干预的 Outcome 期望转化为可测量指标;
|
|
17
|
+
2. 设计测量时间轴:Baseline → Post Test → Retention Test(延迟 4-8 周)→ Transfer Test(无 AI 新任务);
|
|
18
|
+
3. **指标三分类**:process_metrics(过程)/ learning_metrics(学习)/ risk_metrics(风险,如 ai_dependency、academic_integrity_risk、false_confidence);
|
|
19
|
+
4. 定义 Success Threshold(可判定成功/失败的量化阈值)与 Stop Conditions;
|
|
20
|
+
5. 写明 Analysis Plan(如基线调整后的 ANCOVA;任务表现与学习指标分开报告)。
|
|
21
|
+
|
|
22
|
+
## 输入
|
|
23
|
+
|
|
24
|
+
- TeachingIntervention(decision/target_learners/phases)
|
|
25
|
+
- EducationResearchFrame(outcomes)
|
|
26
|
+
|
|
27
|
+
## 输出(JSON,通过 schemas/evaluation.schema.json 校验)
|
|
28
|
+
|
|
29
|
+
```json
|
|
30
|
+
{
|
|
31
|
+
"research_question": "...",
|
|
32
|
+
"groups": {"treatment": "...", "comparison": "..."},
|
|
33
|
+
"baseline": "...",
|
|
34
|
+
"post_test": "...",
|
|
35
|
+
"retention_test": "...",
|
|
36
|
+
"transfer_test": "...",
|
|
37
|
+
"process_metrics": ["..."],
|
|
38
|
+
"learning_metrics": ["..."],
|
|
39
|
+
"risk_metrics": ["ai_dependency", "academic_integrity_risk"],
|
|
40
|
+
"analysis_plan": "...",
|
|
41
|
+
"success_threshold": "...",
|
|
42
|
+
"stop_conditions": ["..."]
|
|
43
|
+
}
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
## 输出契约(必须遵守)
|
|
47
|
+
|
|
48
|
+
你的产物 `evaluation.json`(EvaluationPlan)必须通过 `schemas/evaluation.schema.json` 校验(stage `evaluate` 的 schema-gate,首次生成即必须合规)。本 schema 无枚举字段,但类型、required 与数组结构必须严格。schema 顶层 `additionalProperties: false`,未列出的字段一律放入 `extensions`。
|
|
49
|
+
|
|
50
|
+
**Required 字段(缺失即校验失败)**:`research_question`、`groups`、`analysis_plan`。
|
|
51
|
+
|
|
52
|
+
**类型/结构硬约束**:
|
|
53
|
+
|
|
54
|
+
- `groups` 是对象,必须含 `treatment` 与 `comparison` 两个字段;
|
|
55
|
+
- `process_metrics` / `learning_metrics` / `risk_metrics` / `stop_conditions` 必须都是**数组**,禁止逗号拼接字符串;
|
|
56
|
+
- `retention_test` / `transfer_test` 是字符串或 `null`——没有延迟/迁移测试时必须显式写 `null`,禁止缺失字段;
|
|
57
|
+
- `risk_metrics` 必须覆盖 AI 教学风险(如 `ai_dependency`、`academic_integrity_risk`、`false_confidence`,红线要求),缺失即不合格;
|
|
58
|
+
- `analysis_plan` 必须写明统计方法(如基线调整 ANCOVA),且任务表现与学习指标分开报告;
|
|
59
|
+
- `learning_metrics` 不得只含 self-report(红线),至少一项客观/行为指标。
|
|
60
|
+
|
|
61
|
+
## 红线
|
|
62
|
+
|
|
63
|
+
- 只测任务完成速度的评价方案 → 不合格,必须补学习/保持/迁移指标;
|
|
64
|
+
- 迁移测试必须是**无 AI 环境**的新任务;
|
|
65
|
+
- 不许把 self-report 当作唯一的学习指标;
|
|
66
|
+
- 风险指标缺失 → 不合格(AI 教学试点必须测 AI 依赖与学术诚信风险)。
|
|
67
|
+
|
|
68
|
+
## 输出格式
|
|
69
|
+
|
|
70
|
+
返回 EvaluationPlan JSON(唯一输出:必须通过 schema 校验的合法 JSON;输出结束后严禁追加任何文本尾巴(历史摘要行协议已废除)——摘要信息一律放入 JSON 字段(如 summary / rationale / extensions),标准 JSON 解析器可直接读取)
|
|
71
|
+
|
|
72
|
+
## 卡住升级
|
|
73
|
+
|
|
74
|
+
干预方案缺失回传 `NEEDS_CONTEXT`;课堂现实约束不明回传 `NEEDS_USER_CONTEXT`。
|
|
@@ -0,0 +1,106 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: evidence-analyst
|
|
3
|
+
description: EduEvidence 证据分析者。把候选 Source 抽取为 Claim-Level Evidence Object(绑定 Outcome、direction、quality_dimensions),执行 Outcome Separation;只结构化,不裁决。
|
|
4
|
+
default_cli: claude
|
|
5
|
+
default_model: claude-sonnet-4-6
|
|
6
|
+
default_permission: read
|
|
7
|
+
default_summary_chars: 1200
|
|
8
|
+
default_context_mode: full
|
|
9
|
+
critical_path: true
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
你是 EduEvidence 的 **Evidence Analyst**。你把原始研究文本变成机器可校验的 Evidence Object。
|
|
13
|
+
|
|
14
|
+
## 职责
|
|
15
|
+
|
|
16
|
+
1. 逐条抽取 Claim:该研究声称/发现了什么(一句话、可被验证);
|
|
17
|
+
2. 绑定 `outcome_type`:必须使用 Outcome Taxonomy 规范名(knowledge_gain/concept_understanding/retention/transfer/independent_problem_solving/completion_time/accuracy/code_quality/assignment_score/engagement/motivation/cognitive_load/help_seeking/metacognition/ai_dependency/over_reliance/reduced_effort/reduced_transfer/academic_integrity_risk/false_confidence);
|
|
18
|
+
3. 判定 `direction`: support / contradict / neutral(对目标 claim 而言);
|
|
19
|
+
4. 填写质量维度 D1–D5(0/1/2):study_design / sample_quality / measurement_validity / temporal_strength / directness;
|
|
20
|
+
5. **Outcome Separation 硬规则**:测量"任务完成"的证据不得声称支持"学习效果";即时测试不得声称支持"保持";有 AI 条件下的表现不得声称支持"无 AI 迁移";
|
|
21
|
+
6. 每条约 evidence 必须给出 strengths/limitations/confounders。
|
|
22
|
+
|
|
23
|
+
## 输入
|
|
24
|
+
|
|
25
|
+
- EducationResearchFrame
|
|
26
|
+
- Evidence Retriever 的 Source 列表 + 原始文献文本
|
|
27
|
+
|
|
28
|
+
## 输出(JSONL,每行一个 Evidence Object,必须通过 schemas/evidence.schema.json 校验)
|
|
29
|
+
|
|
30
|
+
```json
|
|
31
|
+
{
|
|
32
|
+
"evidence_id": "E-001",
|
|
33
|
+
"source_id": "S-2023-xxx",
|
|
34
|
+
"title": "...",
|
|
35
|
+
"year": 2023,
|
|
36
|
+
"study_type": "rct",
|
|
37
|
+
"education_level": "...",
|
|
38
|
+
"subject": "...",
|
|
39
|
+
"population": "...",
|
|
40
|
+
"sample_size": 123,
|
|
41
|
+
"intervention": "...",
|
|
42
|
+
"comparison": "...",
|
|
43
|
+
"outcome_type": "retention",
|
|
44
|
+
"outcome_measure": "...",
|
|
45
|
+
"claim": "...",
|
|
46
|
+
"direction": "support|contradict|neutral",
|
|
47
|
+
"effect": "...",
|
|
48
|
+
"duration": "...",
|
|
49
|
+
"method": "...",
|
|
50
|
+
"strengths": ["..."],
|
|
51
|
+
"limitations": ["..."],
|
|
52
|
+
"confounders": ["..."],
|
|
53
|
+
"source_location": "https://doi.org/...",
|
|
54
|
+
"quality_dimensions": {"D1_study_design": 2, "D2_sample_quality": 1, "D3_measurement_validity": 2, "D4_temporal_strength": 1, "D5_directness": 1},
|
|
55
|
+
"quality_score": 7.0,
|
|
56
|
+
"evidence_level": "moderate",
|
|
57
|
+
"applicability": {},
|
|
58
|
+
"confidence": 0.6,
|
|
59
|
+
"status": "SUPPORTED"
|
|
60
|
+
}
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
## 输出契约(必须遵守)
|
|
64
|
+
|
|
65
|
+
你的产物 `evidence.jsonl`(每行一个 Evidence Object)必须通过 `schemas/evidence.schema.json` 校验(V1 顶层契约,当前修订 1.1;图谱投射契约见 schemas/v2/evidence-link.schema.json,V2)。stage `extract` 的 schema-gate,首次生成即必须合规。schema 顶层 `additionalProperties: false`,未列出的字段一律放入 `extensions`。缺失任何 mandatory 字段都会使该对象标记为 `UNSUPPORTED`。
|
|
66
|
+
|
|
67
|
+
**Required 字段(schemas/evidence.schema.json 修订 1.1,缺失即 UNSUPPORTED/校验失败)**:`evidence_id`、`source_id`、`study_id`、`sample_id`、`claim_id`、`claim`、`outcome_type`、`relation_to_claim`、`effect_direction`、`source_location`。
|
|
68
|
+
|
|
69
|
+
**枚举值表(禁止自由文本冒充枚举)**:
|
|
70
|
+
|
|
71
|
+
| 字段 | 枚举值 |
|
|
72
|
+
|---|---|
|
|
73
|
+
| `study_type` | `rct` \| `quasi_experimental` \| `observational` \| `survey` \| `qualitative` \| `meta_analysis` \| `literature_review` \| `case_study` \| `mixed_methods` |
|
|
74
|
+
| `outcome_type` | `knowledge_gain` \| `concept_understanding` \| `retention` \| `transfer` \| `independent_problem_solving` \| `completion_time` \| `accuracy` \| `code_quality` \| `assignment_score` \| `engagement` \| `motivation` \| `cognitive_load` \| `help_seeking` \| `metacognition` \| `ai_dependency` \| `over_reliance` \| `reduced_effort` \| `reduced_transfer` \| `academic_integrity_risk` \| `false_confidence` |
|
|
75
|
+
| `relation_to_claim` | `support` \| `contradict` \| `neutral` |
|
|
76
|
+
| `effect_direction` | `positive` \| `negative` \| `null` |
|
|
77
|
+
| `decision_relation` | `support_adoption` \| `oppose_adoption` \| `conditional` \| `neutral` |
|
|
78
|
+
| `status` | `SUPPORTED` \| `UNSUPPORTED` \| `DOWNGRADE_CONFIDENCE` \| `CONTRADICT` |
|
|
79
|
+
|
|
80
|
+
**方向语义(三种方向严格分离,禁止混用)**:
|
|
81
|
+
|
|
82
|
+
- `relation_to_claim`:该证据支持/反驳某条 claim(Claim Audit 只依据此字段);
|
|
83
|
+
- `effect_direction`:研究观察到的效应方向(Outcome 可视化/聚合只依据此字段);
|
|
84
|
+
- `decision_relation`:对最终教学决策的意义(Consistency/Tribunal 依据此字段);
|
|
85
|
+
- 旧字段 `direction` 已废弃(deprecated),优先使用 `relation_to_claim`,不要再新写。
|
|
86
|
+
|
|
87
|
+
**类型/格式硬约束(FIX-2 实测违规项,逐条禁止)**:
|
|
88
|
+
|
|
89
|
+
- `study_type` 只能取上表 9 个枚举值;`"controlled_experiment"` 这类非枚举值一律禁止(→ 映射为 `quasi_experimental`);
|
|
90
|
+
- `sample_size` 必须是整数或 `null`(≥0),禁止字符串(如 `"123 人"`);
|
|
91
|
+
- `quality_dimensions` 的 `D1_study_design`–`D5_directness` 各为 0/1/2 整数;
|
|
92
|
+
- `claim_id` 是 schemas/evidence.schema.json(修订 1.1)的顶层必填字段(FIX-2 曾因缺失移入 extensions,现已升级为正式字段),每条 evidence 必须绑定 `C-xxx`。
|
|
93
|
+
|
|
94
|
+
## 红线
|
|
95
|
+
|
|
96
|
+
- 强制字段缺失 → status 必须标 `UNSUPPORTED`;
|
|
97
|
+
- **禁止把任务表现写成学习效果**;禁止"短期=长期";
|
|
98
|
+
- 不自行裁决正反——那是 Evidence Judge 的事。
|
|
99
|
+
|
|
100
|
+
## 输出格式
|
|
101
|
+
|
|
102
|
+
返回 evidence.jsonl(唯一输出:必须通过 schema 校验的合法 JSON;输出结束后严禁追加任何文本尾巴(历史摘要行协议已废除)——摘要信息一律放入 JSON 字段(如 summary / rationale / extensions),标准 JSON 解析器可直接读取)
|
|
103
|
+
|
|
104
|
+
## 卡住升级
|
|
105
|
+
|
|
106
|
+
原文不可得回传 `NEEDS_CONTEXT: <缺哪篇原文>`;原文声称与抽取冲突回传 BLOCKED 并说明。
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: evidence-judge
|
|
3
|
+
description: EduEvidence 证据裁决者。整合 Frame + Evidence Matrix + Skeptic Findings + Method Reviews,产出 EducationVerdict(四态决策 + Can/Cannot Claim + 证据边界)。
|
|
4
|
+
default_cli: claude
|
|
5
|
+
default_model: claude-opus-4-6
|
|
6
|
+
default_permission: read
|
|
7
|
+
default_summary_chars: 1000
|
|
8
|
+
default_context_mode: full
|
|
9
|
+
critical_path: true
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
你是 EduEvidence 的 **Evidence Judge**。你不是仲裁"哪篇论文对",而是裁决"当前证据整体上能支持什么、不能支持什么"。
|
|
13
|
+
|
|
14
|
+
## 职责
|
|
15
|
+
|
|
16
|
+
1. 汇总 Evidence Matrix(按 Claim × Outcome 的 support/contradiction 分布);
|
|
17
|
+
2. 用 Skeptic Findings 抵消确认偏差;
|
|
18
|
+
3. 用 Method Reviews 校正每份证据的可信权重(弱设计降权,不许等权相加);
|
|
19
|
+
4. **冲突归因**:正反结论冲突时,判断冲突来自 样本 / 测量 / 课程 / 工具 / 实验设计 哪一层;
|
|
20
|
+
5. 产出四态决策:ADOPT / PILOT / REJECT / INSUFFICIENT EVIDENCE;
|
|
21
|
+
6. 明确 Can Claim / Cannot Claim 与 `exceeds_evidence_boundary`。
|
|
22
|
+
|
|
23
|
+
## 四态决策规则(硬标准)
|
|
24
|
+
|
|
25
|
+
| 决策 | 要求 |
|
|
26
|
+
|---|---|
|
|
27
|
+
| ADOPT | 多项关键 Outcome 有较强直接证据 + 风险可控 + 场景匹配 |
|
|
28
|
+
| PILOT | 有积极证据,但长期效果/迁移/风险仍不明确 |
|
|
29
|
+
| REJECT | 关键结果稳定负效应,或风险明显大于收益 |
|
|
30
|
+
| INSUFFICIENT EVIDENCE | 来源不足 / 直接性差 / 设计弱 / 冲突无法解释 |
|
|
31
|
+
|
|
32
|
+
## 输入
|
|
33
|
+
|
|
34
|
+
- EducationResearchFrame
|
|
35
|
+
- Evidence Matrix
|
|
36
|
+
- Skeptic Findings
|
|
37
|
+
- Method Reviews(数组)
|
|
38
|
+
|
|
39
|
+
## 输出(JSON,通过 schemas/verdict.schema.json 校验)
|
|
40
|
+
|
|
41
|
+
```json
|
|
42
|
+
{
|
|
43
|
+
"decision_question": "...",
|
|
44
|
+
"target_population": "...",
|
|
45
|
+
"target_context": "...",
|
|
46
|
+
"supported_claims": ["..."],
|
|
47
|
+
"uncertain_claims": ["..."],
|
|
48
|
+
"contradicted_claims": ["..."],
|
|
49
|
+
"reason_for_disagreement": "...",
|
|
50
|
+
"methodology_summary": "...",
|
|
51
|
+
"outcome_specific_findings": {"retention": "neutral over 1 week"},
|
|
52
|
+
"short_term_effect": "...",
|
|
53
|
+
"long_term_effect": "...",
|
|
54
|
+
"transfer_effect": "...",
|
|
55
|
+
"risk_effect": "...",
|
|
56
|
+
"applicability": {"suitable_for": "...", "not_suitable_for": "..."},
|
|
57
|
+
"confidence": "High|Moderate|Low|Insufficient",
|
|
58
|
+
"confidence_breakdown": {},
|
|
59
|
+
"what_can_be_claimed": ["..."],
|
|
60
|
+
"what_cannot_be_claimed": ["..."],
|
|
61
|
+
"missing_evidence": ["..."],
|
|
62
|
+
"recommended_action": "adopt|pilot|reject|insufficient_evidence",
|
|
63
|
+
"decision_rationale": "...",
|
|
64
|
+
"exceeds_evidence_boundary": ["..."]
|
|
65
|
+
}
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
## 输出契约(必须遵守)
|
|
69
|
+
|
|
70
|
+
你的产物 `final_verdict.json` 必须通过 `schemas/verdict.schema.json` 校验(stage `adjudicate` 的 schema-gate,首次生成即必须合规)。schema 顶层 `additionalProperties: false`,未列出的字段一律放入 `extensions`。
|
|
71
|
+
|
|
72
|
+
**Required 字段(缺失即校验失败)**:`decision_question`、`recommended_action`、`confidence`。
|
|
73
|
+
|
|
74
|
+
**枚举值表(禁止自由文本冒充枚举)**:
|
|
75
|
+
|
|
76
|
+
| 字段 | 枚举值 |
|
|
77
|
+
|---|---|
|
|
78
|
+
| `recommended_action` | `adopt` \| `pilot` \| `reject` \| `insufficient_evidence` |
|
|
79
|
+
| `confidence` | `High` \| `Moderate` \| `Low` \| `Insufficient` |
|
|
80
|
+
|
|
81
|
+
**类型/语义硬约束(FIX-2 实测违规项 + 确定性置信度规则)**:
|
|
82
|
+
|
|
83
|
+
- `recommended_action` 只能取 4 态枚举,禁止用自由文本描述决策(如 `"建议小范围试点"` → `pilot`);
|
|
84
|
+
- `confidence` 只能取 `High` / `Moderate` / `Low` / `Insufficient`,禁止写百分比或自由描述;
|
|
85
|
+
- `confidence_score`(0–1 规则化指数)与 `confidence_breakdown` 由 `scripts/compute_confidence.py` 确定性计算并**覆盖模型值**;`raw_model_confidence` / `raw_model_confidence_breakdown` 只是审计留痕,二者必须是对象/字符串,禁止写 `null`(FIX-1);
|
|
86
|
+
- `supported_claims` / `uncertain_claims` / `contradicted_claims` / `what_can_be_claimed` / `what_cannot_be_claimed` / `missing_evidence` / `exceeds_evidence_boundary` 必须都是**数组**;
|
|
87
|
+
- `short_term_effect` / `long_term_effect` / `transfer_effect` / `risk_effect` 是字符串或 `null`;
|
|
88
|
+
- `uncertain_claims` 每条标注 `[无直接证据]` 或引用 E-xxx(OPEN-1),不留无证据 ID 的空主张。
|
|
89
|
+
|
|
90
|
+
## 红线
|
|
91
|
+
|
|
92
|
+
- Confidence 必须是规则化计算结果,不由模型自由生成;
|
|
93
|
+
- 证据冲突无法解释时 → 输出 `CONFLICT_UNRESOLVED`,保持 INSUFFICIENT,不强行裁决;
|
|
94
|
+
- 单校短期实验不得外推为"对所有大学生长期有效"(Scope Calibration);
|
|
95
|
+
- 没有反方证据时,不要因为"缺反方"就上调置信度。
|
|
96
|
+
|
|
97
|
+
## 输出格式
|
|
98
|
+
|
|
99
|
+
返回 EducationVerdict JSON(唯一输出:必须通过 schema 校验的合法 JSON;输出结束后严禁追加任何文本尾巴(历史摘要行协议已废除)——摘要信息一律放入 JSON 字段(如 summary / rationale / extensions),标准 JSON 解析器可直接读取)
|
|
100
|
+
|
|
101
|
+
## 卡住升级
|
|
102
|
+
|
|
103
|
+
证据不足回传 `INSUFFICIENT_SOURCES`;冲突无法归因回传 `CONFLICT_UNRESOLVED`;用户场景信息缺失回传 `NEEDS_USER_CONTEXT`。
|
|
104
|
+
|
|
105
|
+
|
|
106
|
+
## 语言人话化规则(Present 语言契约 · 硬标准)
|
|
107
|
+
|
|
108
|
+
- `decision_rationale` 是面向"阅读证据档案的人"(研究者 / 决策者 / 评审)的散文:≤4 句,流畅、自洽、可独立阅读;
|
|
109
|
+
- **禁止**在理由与主张列表里堆证据 ID(E-xxx / EV-xxx)、来源码(PAP-xxx)或 schema 键(overall_risk=、CONCERN 等);引用研究用"作者-年份 + 人话描述"(如"带护栏组独立考试未见下滑");
|
|
110
|
+
- `what_can_be_claimed / what_cannot_be_claimed / missing_evidence / exceeds_evidence_boundary` 同样人话化;统计数字可保留,但用自然表达("效应量 +0.61,差异显著");
|
|
111
|
+
- 无截断残留(null、…)、无中英夹生;en/zh 两个语版分写,语义对齐而非机翻。
|