eduevidence 6.0.0 → 6.2.0

package/docs/sciverse-api.md

@@ -0,0 +1,125 @@
+# Sciverse 检索通道（API 契约存档）
+
+本条记录 EduEvidence 接入 Sciverse 开放平台所用的最小契约，依据官方 `openapi.yaml` **v0.14.2**（`opendatalab/Sciverse-Agent-Tools`）整理。目的是让检索链在**没有网络**时也能被复核：字段名、错误码、限制与语义都在这里，代码变更需同步更新本文件。
+
+## 1. 基本信息
+
+| 项 | 值 |
+|---|---|
+| Base URL | `https://api.sciverse.space` |
+| 鉴权 | `Authorization: Bearer <SCIVERSE_API_TOKEN>`（HTTP Bearer） |
+| Token 来源 | 控制台 Tokens 页；同账号可通用于 Sciverse / DianShi / SeqStudio 已开通能力 |
+| 实现 | `retrieval/sciverse.py`（stdlib-only） |
+| 通道类型 | key-based 学术通道，在 `MultiSearchRouter.academic_key_providers` 中优先于零配置学术通道 |
+| 无 token 行为 | 通道静默失活（`SCIVERSE_UNAVAILABLE`），零配置通道继续工作 |
+
+## 2. 使用的四个端点
+
+### 2.1 `POST /meta-search` — 结构化元数据检索
+
+用于 Source 级命中：标题、作者、摘要、期刊、年份、DOI。
+
+- 请求（所用子集）：`collection`（papers/authors/sources）、`query`（BM25）、`filters_advanced[]`（`{field, operator, value}`）、`page`、`page_size`（≤50）。
+- 响应：`results[]`（注意是 `results`，不是 `hits`）、`total_count`、`page`、`page_size`、`total_pages`、`next_cursor`。
+- 关键字段：`unique_id`（元数据全局唯一 ID，**始终存在**）、`doc_id`（全文内容哈希 sha256，**仅当存在全文**）、`is_content_accessible`、`doi`、`author[].name`、`publication_published_year`、`publication_venue_name_unified`。
+- 过滤操作符：`FILTER_OP_EQ/NE/GT/GTE/LT/LTE/IN/NIN/CONTAINS/MATCH/MATCH_PHRASE`；`doi` 用 `EQ`（服务端去 `doi.org` 前缀并转小写后精确匹配）。
+- 排序与加权：`sort_by_year`（`auto` 在带 query 时保留相关性排序）、`freshness_boost` / `impact_boost` / `language_affinity`（`NONE|MILD|STRONG`，仅在 query 非空时生效；加权生效时**不支持深翻页**）。
+
+### 2.2 `POST /agentic-search` — 自然语言语义检索（RAG）
+
+用于 chunk 级**定位子**：
+
+- 请求：`query`（1–200 字最佳）、`top_k`（1–100）、`mode`（`fast` ~200ms / `balanced` ~600ms / `quality` ~2–4s）、可选 `filters`、`source_types`（`web` / `pdf`）。
+- 响应：`hits[]`，每条含 `chunk_id`、`doc_id`、`score`、`title`、`offset`（**Unicode 码点**，可直接作为 `/content` 的 offset）、`page_no`、`source_type`、`chunk`、`abstract`。
+- **限制**：`balanced` 模式服务端约截断至 50 条；同一篇论文最多返回约 3 个 chunk，因此高 `top_k` 需要足够多的不同论文。
+- **软过滤语义**：`filters` 在召回阶段与语义检索同时下推，但 chunk 侧元数据缺失的文档不会被排除（按年份过滤时，缺年份的 chunk 仍可能返回）。结论表述必须写"近似范围"；需要严格范围时改用 `meta-search` 的结构化过滤并核对返回记录。
+- 唯一的硬过滤字段是 `doc_id`（命中绝不越出给定集合；去重后上限默认 1000）。
+
+### 2.3 `GET /content` — 按码点区间读原文
+
+把 chunk 定位子扩展为可抽取的正文：
+
+- 参数：`doc_id`（必填）、`offset`（默认 0）、`limit`（默认 4096，服务端上限 524288，超出静默钳制）。
+- **必须显式传 `offset`**：省略时服务端返回整篇全文并忽略 `limit`。
+- 响应：`text`、`bytes_returned`（UTF-8 字节数，仅供参考）、`next_offset`（下一段起始码点，翻页用它而非字节数）、`more`。
+- 单位：`offset` / `limit` 均以 **Unicode 码点**计，与 Python `len` 一致，不是字节。
+
+### 2.4 `POST /meta-paper-relations` — 引用 / 被引 / 相关工作
+
+用于引文链审计（`SearchQuery.purpose = citation_chain`）：
+
+- 请求：`unique_id`（**不是 doc_id**）、`relation`（`CITATIONS` 被引 / `REFERENCES` 参考文献 / `RELATED_WORKS`）、`page`、`page_size`（≤200）。
+- 响应：`items[]`（`id` / `id_type` / `title`）、`total_count`、`page`、`page_size`、`total_pages`。
+- 方向语义：`CITATIONS` 是"谁引用了我"，`REFERENCES` 是"我引用了谁"，两者相反。
+- **上限**：关系数超 10000 返回 `429`；`page × page_size` 超 10000 返回 `400`。两种情况改用 `meta-search` 的 `references_unique_id` 反查（支持深翻页与任意排序）。
+- `total_count` 只统计库内命中，与论文自身 `citation_count` 可能有约 ±1% 差异。
+
+## 3. 错误处理与状态映射
+
+`retrieval/sciverse.py` 把 HTTP 与网络异常统一映射为**定型状态**，绝不把异常抛进检索管道：
+
+| HTTP / 情形 | 状态 | 管道行为 |
+|---|---|---|
+| 200 | `ok` | 正常解析 |
+| 401 / 403 | `SCIVERSE_UNAUTHORIZED` | 记录尝试，切换其他通道 |
+| 400 / 404 / 429 | `SCIVERSE_BAD_REQUEST` | 同上（429 视为配额/上限耗尽） |
+| 502 / 503 / 5xx | `SCIVERSE_UPSTREAM_ERROR` | 同上 |
+| 超时 / DNS / TLS | `SCIVERSE_NETWORK_ERROR` | 同上 |
+| 未配置 token | `SCIVERSE_UNAVAILABLE` | 通道失活，不产生请求 |
+
+错误信息只保留服务端 `ApiError.message` 或状态描述，**不携带 Authorization 头或 token 片段**。
+
+## 4. 在证据链中的位置（RULE 2 的机器化）
+
+```text
+/meta-search      → Source 级命中（DOI → https://doi.org/<doi>；无 DOI 则标记 needs_manual_location）
+/agentic-search   → chunk 定位子（doc_id + offset）→ 写入 chunks.jsonl，标注 discovery_only_requires_content_fetch
+/content          → 按定位读原文 → 经 retrieval/validate.py 校验门 → 才可进入 Extract
+/meta-paper-relations → 引文链记录（写入审计导出，供筛选与饱和判断）
+```
+
+要点：**chunk 不是证据**。它是发现线索；只有 `/content` 读到的正文通过校验门（长度、错误页、登录页、验证码、标题匹配等）之后，才允许被抽取为 Evidence Object。这条纪律由 `retrieval/fetch.py::fetch_sciverse_content()` 实现，产出与 `FetchResult` 同形，下游零改动。
+
+引用目标永远是论文本身（DOI / `unique_id`），**不是 Sciverse** —— 它是读取路径，不是来源。
+
+## 5. 配置
+
+```bash
+export SCIVERSE_API_TOKEN=sv-...        # 必需；未设置时通道失活
+# 可选：指向测试环境
+export SCIVERSE_BASE_URL=https://api.sciverse.space
+```
+
+宿主侧如已安装官方 Skill / MCP，可作为**补充**而非替代（本仓库的实现不依赖它）：
+
+```bash
+npx skills add https://sciverse.space
+# 或 MCP server
+npm install -g sciverse-mcp-server
+export SCIVERSE_API_TOKEN=sv-...
+```
+
+## 6. 验证
+
+```bash
+# 契约与降级行为（离线，mock HTTP）
+python -m pytest tests/test_sciverse_channel.py -q
+
+# 真实连通冒烟（需 token）
+python - <<'PY'
+from retrieval import sciverse as s
+print('available:', s.available())
+print('meta:', s.meta_search('generative AI coding assistants learning outcomes', limit=3).status)
+r = s.agentic_search('unguarded GPT-4 access harms independent exam performance', top_k=3)
+recs = s.chunk_records(r)
+print('chunks:', len(recs))
+if recs:
+    c = s.read_content(recs[0]['doc_id'], offset=recs[0]['offset'], limit=600)
+    print('content:', c.status, len(c.data.get('text') or ''))
+PY
+```
+
+## 7. 合规
+
+配额、限速、缓存与署名规则统一见 `references/retrieval-compliance.md`（§3 Sciverse 附加约定）。规范漂移时以官方 `openapi.yaml` 为准，并同步更新本文件与 `references/retrieval-protocol.md`。
+

package/eduevidence_cli.py

@@ -8,6 +8,16 @@ only two intercepted command domains:
     eduevidence evolve ...
 """
 import sys
+
+MIN_PYTHON = (3, 10)
+
+if sys.version_info < MIN_PYTHON:
+    import platform
+    sys.stderr.write(
+        f"EduEvidence requires Python {MIN_PYTHON[0]}.{MIN_PYTHON[1]} or newer; "
+        f"this interpreter is {platform.python_version()}.\n"
+        "Install a supported Python and re-run, or set PYTHON to one.\n")
+    raise SystemExit(2)
 from pathlib import Path
 
 ROOT = Path(__file__).resolve().parent

package/engine/decision_policy.py

@@ -0,0 +1,96 @@
+"""Single source of truth for the ADOPT direct-evidence gate.
+
+The four-state decision (ADOPT / PILOT / REJECT / INSUFFICIENT EVIDENCE) is
+computed by two layers that must never drift apart:
+
+  * engine/tribunal.py - V2 Evidence Graph adjudication
+  * scripts/pre_verdict_gate.py - V1 run/example-pack gate enforcement
+
+Before this module existed the rule lived only inside the tribunal, so the V1
+gate could not enforce it and a hand-written verdict could carry any action it
+liked. Both layers now resolve the 'which outcome categories count for this
+domain' question here, and the V1 gate additionally re-derives direct-evidence
+presence from the pack's own evidence records.
+
+Stdlib only, consistent with the "Native Core" policy of engine/.
+"""
+from __future__ import annotations
+
+#: Category buckets that count as a domain's PRIMARY effect for the ADOPT gate.
+#: The gate asks 'is there direct evidence on the outcome this decision is
+#: actually about?' - for education that is a learning outcome (task
+#: performance and process measures never qualify); for policy it is the
+#: policy-effectiveness / cost class. Every entry must name a category the
+#: domain registry declares, which check_protocol_alignment.py enforces.
+PRIMARY_EFFECT_CATEGORIES: dict[str, tuple[str, ...]] = {
+    "education": ("learning",),
+    "policy": ("effectiveness", "cost"),
+}
+
+#: Directness (0-2) at which an evidence link may carry an ADOPT claim.
+ADOPT_DIRECTNESS = 2
+
+#: Confidence band required before ADOPT is possible at all.
+ADOPT_REQUIRED_LABEL = "High"
+
+
+def primary_effect_categories(domain: str) -> tuple[str, ...]:
+    """Categories that satisfy the ADOPT direct-evidence gate for a domain."""
+    from engine.taxonomy import categories as taxonomy_categories
+
+    declared = PRIMARY_EFFECT_CATEGORIES.get(domain)
+    if declared:
+        known = taxonomy_categories(domain)
+        missing = [c for c in declared if c not in known]
+        if missing:
+            raise ValueError(
+                'domain ' + repr(domain) + ' ADOPT gate references undeclared '
+                'categories ' + repr(missing) + '; declared: ' + repr(sorted(known)))
+        return declared
+    known = taxonomy_categories(domain)
+    if not known:
+        raise ValueError('domain ' + repr(domain) + ' declares no outcome categories')
+    return (next(iter(known)),)
+
+
+def outcome_category(domain: str, value: str, primary: tuple[str, ...]) -> str | None:
+    """Resolve an outcome value to its category, or None when unresolvable.
+
+    The outcomes table stores CATEGORY buckets (learning / task_performance /
+    process / risk, plus each domain's own buckets), while V1 packs store raw
+    taxonomy tokens. Accept a category directly and, for a token, resolve it
+    through the registry. An unknown value returns None so callers fail
+    closed instead of silently treating it as decision-grade evidence.
+    """
+    from engine.taxonomy import TaxonomyError, category_of
+
+    if value in primary:
+        return value
+    try:
+        return category_of(domain, value)
+    except (TaxonomyError, ValueError):
+        return None
+
+
+def decision_action(*, confidence_label: str, decisive_relations: dict[str, str],
+                    has_direct_primary_evidence: bool) -> str:
+    """Gate-enforced decision action (uppercase four-state).
+
+    REJECT requires usable direct opposition evidence (an independent Study
+    folded to oppose_adoption). Low/Insufficient can never yield ADOPT.
+    ADOPT additionally requires direct evidence on the domain's PRIMARY
+    outcome category: High + decisive support WITHOUT such evidence downgrades
+    to PILOT - task performance and procedural efficiency are not
+    decision-grade effects. Moderate + decisive support -> PILOT; otherwise
+    INSUFFICIENT_EVIDENCE.
+    """
+    has_oppose = any(r == "oppose_adoption" for r in decisive_relations.values())
+    has_support = any(r == "support_adoption" for r in decisive_relations.values())
+    if has_oppose:
+        return "REJECT"
+    if (confidence_label == ADOPT_REQUIRED_LABEL and has_support
+            and has_direct_primary_evidence):
+        return "ADOPT"
+    if confidence_label in ("High", "Moderate") and has_support:
+        return "PILOT"
+    return "INSUFFICIENT_EVIDENCE"

package/engine/evidence_graph.py

@@ -54,7 +54,11 @@ class EvidenceNode:
     outcome_dimension: str = OutcomeDimension.GENERAL_MEASURE
     claim_id: Optional[str] = None
     outcome_id: Optional[str] = None
-    effect_size: Dict[str, Any] = field(default_factory=lambda: {"metric": "Hedges g", "value": 0.0, "ci_lower": 0.0, "ci_upper": 0.0, "p_value": 0.05})
+    # A missing effect must stay missing: the old default fabricated a
+    # g = 0.0 with a p = 0.05, which would serialise as a real (and
+    # false) "no effect" result. Callers that need a number must supply
+    # one; the extractors already suppress charts when it is absent.
+    effect_size: Optional[Dict[str, Any]] = None
     sample_size: int = 0
     sample_description: str = ""
     study_design: str = "Quasi-Experimental"  # RCT, Quasi-Experimental DID, Meta-Analysis, Observational
@@ -292,9 +296,9 @@ class EvidenceGraph:
         for ev in self.evidence.values():
             paper = self.papers.get(ev.paper_id)
             study_label = f"{paper.authors[0] if paper and paper.authors else ev.paper_id} ({paper.year if paper else ''})"
-            effect_val = ev.effect_size.get("value", 0.0)
-            ci_l = ev.effect_size.get("ci_lower")
-            ci_u = ev.effect_size.get("ci_upper")
+            effect_val = (ev.effect_size or {}).get("value") or 0.0
+            ci_l = (ev.effect_size or {}).get("ci_lower")
+            ci_u = (ev.effect_size or {}).get("ci_upper")
             has_ci = ci_l is not None and ci_u is not None and float(ci_u) >= float(ci_l)
             points.append({
                 "evidence_id": ev.evidence_id,
@@ -327,13 +331,13 @@ class EvidenceGraph:
             precision_counts = {"reported_ci": 0, "derived_from_sample_size": 0}
             excluded_no_precision = 0
             for n in nodes:
-                eff = n.effect_size.get("value")
+                eff = (n.effect_size or {}).get("value")
                 if eff is None or math.isnan(float(eff)) or math.isinf(float(eff)):
                     continue
 
                 # Statistical variance derivation (Borenstein et al. 2009)
-                ci_l = n.effect_size.get("ci_lower")
-                ci_u = n.effect_size.get("ci_upper")
+                ci_l = (n.effect_size or {}).get("ci_lower")
+                ci_u = (n.effect_size or {}).get("ci_upper")
                 if ci_l is not None and ci_u is not None and float(ci_u) > float(ci_l):
                     se = (float(ci_u) - float(ci_l)) / (2.0 * 1.95996)
                     precision_counts["reported_ci"] += 1
@@ -422,7 +426,7 @@ class EvidenceGraph:
                 "quote": p.summary,
             })
         for ev in self.evidence.values():
-            effect_val = ev.effect_size.get("value", 0.0)
+            effect_val = (ev.effect_size or {}).get("value") or 0.0
             symbol_size = max(18, min(45, int(18 + abs(effect_val) * 20)))
             nodes.append({
                 "id": ev.evidence_id,
@@ -433,8 +437,8 @@ class EvidenceGraph:
                 "dimension": ev.outcome_dimension,
                 "direction": ev.direction,
                 "effect_size": effect_val,
-                "ci_lower": ev.effect_size.get("ci_lower", "N/A"),
-                "ci_upper": ev.effect_size.get("ci_upper", "N/A"),
+                "ci_lower": (ev.effect_size or {}).get("ci_lower", "N/A"),
+                "ci_upper": (ev.effect_size or {}).get("ci_upper", "N/A"),
                 "sample_size": ev.sample_size,
                 "wwc_rating": ev.wwc_rating,
                 "quote": ev.key_quote,

package/engine/gaps.py

@@ -21,10 +21,32 @@ from engine.graph_store import GraphStore
 from engine.ids import new_local_id
 from engine.synthesis import ClaimSynthesis
 
-_RETENTION_TYPES = {"retention", "long_term", "learning_retention"}
-_TRANSFER_TYPES = {"transfer", "transfer_learning", "far_transfer"}
-_TASK_PERFORMANCE = {"task_performance", "assignment_score", "task_completion"}
-_LEARNING = {"learning"}
+#: Gap kind -> the outcome categories that count as covering it. Categories are
+#: read from the domain registry, so a policy run classifies its outcomes with
+#: the policy buckets (effectiveness / cost / equity / feasibility / risk)
+#: instead of being measured against education vocabulary.
+GAP_KIND_CATEGORIES: dict[str, tuple[str, ...]] = {
+    "learning": ("learning",),
+    "retention": ("learning",),
+    "transfer": ("learning",),
+    "task_performance": ("task_performance",),
+    "process": ("process",),
+    "risk": ("risk",),
+    "effectiveness": ("effectiveness",),
+    "cost": ("cost",),
+    "equity": ("equity",),
+    "feasibility": ("feasibility",),
+}
+
+#: Legacy aliases accepted when a frame names its requested outcomes with older
+#: vocabulary; they resolve to a gap kind above.
+GAP_KIND_ALIASES: dict[str, str] = {
+    "long_term": "retention", "learning_retention": "retention",
+    "transfer_learning": "transfer", "far_transfer": "transfer",
+    "assignment_score": "task_performance", "task_completion": "task_performance",
+    "policy_effectiveness": "effectiveness", "cost_effectiveness": "cost",
+    "implementation_risk": "risk",
+}
 
 
 def _autoresearch_key(
@@ -97,26 +119,24 @@ def derive_gaps(*, store: GraphStore,
             req_type = str(req.get("outcome_type", "")).lower()
         else:
             req_name, req_type = str(req).lower(), ""
-        if req_type in _RETENTION_TYPES or req_name in _RETENTION_TYPES:
-            return "retention", req.get("name", "") if isinstance(req, dict) else str(req)
-        if req_type in _TRANSFER_TYPES or req_name in _TRANSFER_TYPES:
-            return "transfer", req.get("name", "") if isinstance(req, dict) else str(req)
-        if req_type in _TASK_PERFORMANCE or req_name in _TASK_PERFORMANCE:
-            return "task_performance", req.get("name", "") if isinstance(req, dict) else str(req)
-        if req_type in _LEARNING or req_name in _LEARNING:
-            return "learning", req.get("name", "") if isinstance(req, dict) else str(req)
-        return "other", req.get("name", "") if isinstance(req, dict) else str(req)
+        label = req.get("name", "") if isinstance(req, dict) else str(req)
+        kind = GAP_KIND_ALIASES.get(req_type) or GAP_KIND_ALIASES.get(req_name)
+        if kind is None:
+            kind = req_type if req_type in GAP_KIND_CATEGORIES else req_name
+        if kind in GAP_KIND_CATEGORIES:
+            return kind, label
+        return "other", label
 
     def covered_for_kind(kind: str) -> bool:
-        if kind == "retention":
-            return bool(covered_types & _RETENTION_TYPES)
-        if kind == "transfer":
-            return bool(covered_types & _TRANSFER_TYPES)
-        if kind == "task_performance":
-            return bool(covered_types & _TASK_PERFORMANCE)
-        if kind == "learning":
-            return bool(covered_types & _LEARNING)
-        return False
+        """True when the graph already carries an outcome for this gap kind.
+
+        ``covered_types`` holds the category buckets stored on outcomes, which
+        is why this compares categories rather than V1 tokens.
+        """
+        expected = GAP_KIND_CATEGORIES.get(kind)
+        if not expected:
+            return False
+        return bool(covered_types & set(expected))
 
     seen: set[tuple[str, str]] = set()
     for req in requested:

package/engine/ids.py

@@ -17,6 +17,8 @@ keep stable Project-local IDs across edits; version/revision metadata records
 change instead of silently re-identifying.
 """
 
+from __future__ import annotations
+
 from datetime import datetime, timezone
 import hashlib
 import secrets

package/engine/library.py

@@ -8,6 +8,8 @@ changes an existing Project's conclusions — only an explicit import/sync
 advances the Project graph.
 """
 
+from __future__ import annotations
+
 import hashlib
 import json
 import os
@@ -197,8 +199,10 @@ class ResearchLibrary:
             outcomes.append({
                 "outcome_id": oid,
                 "name": f.get("measure", oid),
-                "outcome_type": (f.get("extensions") or {}).get(
-                    "outcome_type", "learning"),
+                # No silent default: an unlabelled finding must not be filed as a
+                # learning outcome; that is how task-performance evidence used
+                # to reach the ADOPT gate on the V2 path.
+                "outcome_type": (f.get("extensions") or {}).get("outcome_type", ""),
                 "extensions": {
                     "auto_created_from_library_import": True,
                     "library_revision": lib_rev,

package/engine/living.py

@@ -227,6 +227,26 @@ def set_subscription_status(project: ProjectWorkspace, subscription_id: str,
     return subscription
 
 
+def _validated_outcome_type(value, domain: str) -> str:
+    """Outcome token from a refresh payload, validated against the registry.
+
+    A missing or unknown token raises: the living path used to default to
+    a learning outcome, which silently promoted task-performance evidence.
+    """
+    from engine.taxonomy import category_of, tokens as taxonomy_tokens
+
+    token = str(value or "").strip()
+    if not token:
+        raise ValueError(
+            "living evidence record must declare outcome_type; the engine "
+            "will not guess a category")
+    if token not in taxonomy_tokens(domain):
+        raise ValueError(
+            f"outcome_type {token!r} is not registered for domain {domain!r}")
+    category_of(domain, token)  # fail closed on a malformed taxonomy
+    return token
+
+
 def refresh(project: ProjectWorkspace, subscription_id: str, *,
             new_evidence: list[dict] | None = None,
             retriever: Callable[[dict], list[dict]] | None = None) -> dict:
@@ -299,8 +319,10 @@ def refresh(project: ProjectWorkspace, subscription_id: str, *,
 
     # ---- normalize + validate fresh evidence ---------------------------
     next_revision = store.active_revision() + 1
+    domain = str((project.manifest() or {}).get("domain") or "education")
     mutation, new_hashes, finding_ids, summary_parts = _build_mutation(
-        project, store, subscription, fresh_packets, claims, next_revision)
+        project, store, subscription, fresh_packets, claims, next_revision,
+        domain=domain)
 
     revision = store.commit(
         run_id=new_run_id(),
@@ -370,7 +392,7 @@ def _existing_drift_ids(project: ProjectWorkspace) -> set[str]:
 
 
 def _build_mutation(project, store, subscription, fresh_packets, claims,
-                    next_revision) -> tuple[GraphMutation, set[str], set[str], list[str]]:
+                    next_revision, domain: str = "education") -> tuple[GraphMutation, set[str], set[str], list[str]]:
     """Normalize + validate each fresh record and assemble one GraphMutation."""
     existing = {t: {row[_ID_KEY[t]] for row in store.read_table(t)}
                 for t in _GRAPH_TABLES}
@@ -418,6 +440,8 @@ def _build_mutation(project, store, subscription, fresh_packets, claims,
             source_upserted = True
             upserts["sources"].append(src)
 
+        # Outcome tokens are validated against the project domain registry
+        # rather than defaulted; see _validated_outcome_type().
         # --- outcome (optional; reused when the id already exists) -------
         outcome_upserted = False
         outcome = None
@@ -439,7 +463,7 @@ def _build_mutation(project, store, subscription, fresh_packets, claims,
             outcome = {
                 "outcome_id": out_id,
                 "name": outcome_pkt.get("name") or out_id[len("OUT-"):],
-                "outcome_type": outcome_pkt.get("outcome_type", "learning"),
+                "outcome_type": _validated_outcome_type(outcome_pkt.get("outcome_type"), domain),
                 "extensions": outcome_pkt.get("extensions") or {},
             }
             existing["outcomes"].add(out_id)
@@ -503,7 +527,13 @@ def _build_mutation(project, store, subscription, fresh_packets, claims,
                 f"{label}: relation_to_claim must be one of "
                 f"{sorted(_RELATION_TO_IMPLICATION)}, got {relation!r}")
         link.setdefault("decision_implication", _RELATION_TO_IMPLICATION[relation])
-        link.setdefault("directness", 2)
+        # directness decides whether this link can carry an ADOPT claim.
+        # Defaulting it to 2 (direct) would let an unclassified refresh
+        # assert direct evidence it never established, so it is required.
+        if "directness" not in link:
+            raise ValueError(
+                f"{label}: directness is required for a living-evidence link "
+                "(2 = direct evidence for the claim; supply it explicitly)")
         link.setdefault("applicability", {"scope_match": "direct"})
         link.setdefault("reasoning_note",
                         f"living evidence refresh: {subscription['subscription_id']}")

package/engine/migration.py

@@ -30,6 +30,15 @@ from engine.versions import (
 
 OUTCOME_TYPES = ("learning", "task_performance", "process", "risk")
 
+#: V1 D5 Directness (0/1/2) -> V2 link directness + applicability scope_match.
+#: Directness decides whether a link can carry an ADOPT claim, so a flat
+#: hard-coded value silently capped every migrated pack at PILOT.
+_D5_TO_DIRECTNESS = {
+    2: (2, "direct"),
+    1: (1, "partial"),
+    0: (1, "mismatch"),
+}
+
 _CLAIM_TO_IMPLICATION = {
     "support": "support_adoption",
     "contradict": "oppose_adoption",
@@ -75,6 +84,71 @@ def _map_decision_implication(ev: dict) -> str:
     return _CLAIM_TO_IMPLICATION[_map_relation(ev.get("relation_to_claim") or ev.get("direction"))]
 
 
+def _v1_effect_estimate(ev: dict) -> dict | None:
+    """Effect magnitude from a V1 record, or None when it recorded none.
+
+    V1 kept numbers either on a top-level effect_size field or inside
+    extensions.raw_result; both are real sources, and absence stays None.
+    The V2 contract is narrow (metric + raw_text required, no extra keys), so
+    a V1 magnitude is normalized instead of copied verbatim: ci_lower/ci_upper
+    become ci_low/ci_high and keys the V2 contract does not declare are kept in
+    the raw_text so nothing recorded is silently lost.
+    """
+    value = ev.get("effect_size")
+    if isinstance(value, dict) and value.get("value") is not None:
+        return _normalize_effect_estimate(value)
+    if isinstance(value, (int, float)):
+        return {"value": float(value), "source": "v1_effect_size"}
+    raw = (ev.get("extensions") or {}).get("raw_result")
+    if isinstance(raw, dict) and raw.get("value") is not None:
+        return _normalize_effect_estimate(raw)
+    return None
+
+
+def _normalize_effect_estimate(raw: dict) -> dict:
+    """Project a V1 magnitude onto the V2 effect_estimate contract."""
+    known = ("metric", "value", "unit", "ci_low", "ci_high", "p_value")
+    out: dict = {}
+    for key in known:
+        if key in raw and raw[key] is not None:
+            out[key] = raw[key]
+    for legacy, canonical in (("ci_lower", "ci_low"), ("ci_upper", "ci_high")):
+        if canonical not in out and raw.get(legacy) is not None:
+            out[canonical] = raw[legacy]
+    out.setdefault("metric", str(raw.get("source") or "v1_effect_size"))
+    extra = {k: v for k, v in raw.items()
+             if k not in known and k not in ("ci_lower", "ci_upper")
+             and v is not None}
+    text = raw.get("raw_text")
+    if not text:
+        text = json.dumps(extra, ensure_ascii=False, sort_keys=True) if extra else ""
+    out["raw_text"] = str(text)
+    return out
+
+
+def _v1_directness(ev: dict) -> tuple[int, str, bool]:
+    """Carry V1 D5 Directness across as (link directness, scope_match, recorded).
+
+    D5 is a 0/1/2 axis in the V1 evidence contract and the V2 ADOPT gate reads
+    the link's directness, so the mapping has to be explicit:
+
+        2 -> directness 2 / scope_match "direct"
+        1 -> directness 1 / scope_match "partial"
+        0 -> directness 1 / scope_match "mismatch"  (0 can never gate an ADOPT)
+
+    A missing or non-integer D5 is not silently promoted: it migrates as
+    directness 1 / scope_match "partial" and the caller records a downgrade.
+    """
+    dims = ev.get("quality_dimensions")
+    raw = dims.get("D5_directness") if isinstance(dims, dict) else None
+    if isinstance(raw, bool) or not isinstance(raw, int):
+        return 1, "partial", False
+    mapped = _D5_TO_DIRECTNESS.get(raw)
+    if mapped is None:
+        return 1, "partial", False
+    return mapped[0], mapped[1], True
+
+
 def migrate_v1_pack(pack_dir: Path, *, home: Path,
                     title: str | None = None) -> MigrationResult:
     """Import a V1 pack directory into a new V2 Project graph.
@@ -273,7 +347,10 @@ def migrate_v1_pack(pack_dir: Path, *, home: Path,
             "measure": ev.get("outcome_type") or "outcome",
             "timepoint": None,
             "effect_direction": _map_effect_direction(ev.get("effect_direction")),
-            "effect_estimate": None,
+            # Carry the magnitude across the hop instead of dropping it:
+            # a migrated pack used to report 100% not_extractable, which
+            # reads as "no evidence" rather than "not migrated".
+            "effect_estimate": _v1_effect_estimate(ev),
             "raw_result_text": ev.get("claim") or "unavailable",
             "source_locator": ev.get("source_location") or "unavailable",
             "extensions": {"v1_legacy": True},
@@ -281,6 +358,14 @@ def migrate_v1_pack(pack_dir: Path, *, home: Path,
         claim_id = claim_by_evidence.get(ev["evidence_id"], f"CLM-{ev['evidence_id']}")
         link_id = f"LNK-{ev['evidence_id']}"
 
+        directness, scope_match, d5_recorded = _v1_directness(ev)
+        if not d5_recorded:
+            report["downgrades"].append({
+                "evidence_id": ev["evidence_id"],
+                "from": "V1 quality_dimensions.D5_directness (absent or unreadable)",
+                "to": f"directness={directness}, scope_match={scope_match!r}",
+            })
+
         links.append({
 
             "evidence_link_id": link_id,
@@ -289,8 +374,8 @@ def migrate_v1_pack(pack_dir: Path, *, home: Path,
             "relation_to_claim": _map_relation(
                 ev.get("relation_to_claim") or ev.get("direction")),
             "decision_implication": _map_decision_implication(ev),
-            "directness": 1,
-            "applicability": {"scope_match": "direct"},
+            "directness": directness,
+            "applicability": {"scope_match": scope_match},
             "reasoning_note": "migrated from V1 Evidence Object",
             "created_in_revision": 1,
             "extensions": {"v1_legacy": True},