eduevidence 5.2.0 → 6.2.0

package/references/report-copy-style.md

@@ -0,0 +1,67 @@
+# Report Copy Style — 读者向文案规范
+
+本规范约束**报告里面向读者的文字**（第一屏决策叙事、裁决列表、章节引导语、图表标题与说明）。它服务于一个判断标准：**读者能否一次读懂，且能核对**。规则由 `skill/agents/evidence-judge.md`（写作前）与 `visualization/eduevidence-report/scripts/build_report.py` 的语言门禁（交付后）双向执行。
+
+## 1. 谁写什么
+
+| 内容 | 作者 | 说明 |
+|---|---|---|
+| 决策叙事四件套（`strongest_support` / `key_uncertainty` / `main_risk` / `next_action`） | **裁决角色（模型）撰写** | 渲染器只呈现，缺字段就报缺，禁止用列表片段拼装句子 |
+| `decision_rationale` | 裁决角色 | ≤4 句，可独立阅读 |
+| 章节引导语、图例、按钮、兜底句 | 渲染器（固定文案表） | 不随研究内容变化 |
+| 证据与主张原文 | 抽取/裁决角色 | 渲染器不改写 |
+
+**为什么写死这条**：报告第一屏曾由渲染器从 `what_can_be_claimed[0]` 之类的片段拼出，读起来像拼装而非撰写，且掩盖了「该字段没人产出」这一事实。撰写责任在模型，呈现责任在渲染器。
+
+## 2. 语言
+
+- 面向**非本领域的决策者**：不假设读者熟悉术语。
+- 一句话一个意思；先结论、后依据。
+- 中文用中文，英文用英文；两种语言各自成篇，不做逐字直译。
+- 允许保留原文的例外：AI、RCT、DOI、WWC、GRADE、论文原标题（须标注「原文标题」）。
+
+## 3. 字数上限
+
+| 字段 | 上限 | 约合 |
+|---|---|---|
+| `strongest_support` | 60 字 | 2 句 |
+| `key_uncertainty` | 70 字 | 2–3 句 |
+| `main_risk` | 60 字 | 2 句 |
+| `next_action` | 80 字 | 2–3 句 |
+| `decision_rationale` | 160 字 | ≤4 句 |
+
+英文字数上限按等义折算（约为中文字数 ÷ 3 个单词）。
+
+## 4. 术语对照
+
+同一概念全文只用一个说法：
+
+| 内部字段 | 中文 | 英文 |
+|---|---|---|
+| `outcome` / `outcome_type` | 结果 | outcome |
+| `claim` | 主张 | claim |
+| `evidence` | 证据 | evidence |
+| `relation_to_claim` | 与主张的关系 | relation to claim |
+| `effect_direction` | 效应方向 | effect direction |
+| `directness` | 直接性 | directness |
+| `task_performance` | 任务表现 | task performance |
+| `retention` | 保持 | retention |
+| `transfer` | 迁移 | transfer |
+| `applicability` | 适用性 | applicability |
+
+## 5. 禁止
+
+- 内部字段名与存储标识（`effect_direction`、`first_programming_course_...`）出现在叙述句里；它们只能出现在「原始标识」提示或溯源展开区。
+- 证据 ID 堆砌（`E-001、E-006`）出现在决策叙事里；引用研究用「作者-年份 + 人话描述」。
+- 图表标题写公式（如 `position = (positive − negative) ÷ count`）；用自然语言说明图形含义。
+- 半句截断、`null` 残留、中英夹生。
+- 用兜底句掩盖缺失：字段没产出就如实说没产出。
+
+## 6. 门禁如何执行
+
+- `check_language_parallel()`：叙述字段必须为对应语言、双语不得完全相同、不得含内部键名。
+- 原始标识检查：`research_frame` 与决策字段中的多词串若含未注册的 `snake_case`，记为缺陷。
+- 字数检查：四件套超过上限即失败。
+- 一致性检查：`result.json` 与 `result.zh.json` 的证据在 id/来源/研究/样本/方向/关系六个结构字段上必须逐条相等（自由文本可不同）。
+
+相关：`visualization/eduevidence-report/references/bilingual-style.md`（双语渲染约定）、`references/scientific-invariants.md`（科学不变量）。

package/references/retrieval-compliance.md

@@ -0,0 +1,75 @@
+# Retrieval Compliance Policy（检索合规政策）
+
+本条政策约束 EduEvidence 的一切外部检索与抓取行为。它服务于两件事：**研究的可复现性**（每条来源都能被第三方重新定位）与**对他方服务与作者的尊重**（不越权、不滥用、不掩盖出处）。违反本条政策的检索结果不得进入证据链。
+
+## 1. 定位与适用范围
+
+适用于 `retrieval/` 下的全部通道：
+
+| 通道 | 类型 | 是否需 key |
+|---|---|---|
+| OpenAlex / Semantic Scholar / CrossRef | 学术元数据 API | 否 |
+| AIHot | 行业动态源 | 否 |
+| AgentSearch / ArXiv | 预印本检索 | 否 |
+| DuckDuckGo | 通用网页回退 | 否 |
+| Sciverse | 学术检索（含全文） | 是（`SCIVERSE_API_TOKEN`） |
+| Tavily / Brave | 商业搜索 API | 是 |
+| 抓取链（builtin / jina_reader / defuddle / markdown_new / raw_html） | 正文读取 | 否 |
+
+## 2. 抓取前：robots 与访问边界
+
+- **尊重 robots.txt**：目标站点在 robots.txt 中禁止抓取的路径不得作为抓取目标；需要该内容时改用其官方 API 或元数据记录，并在筛选中注明"仅元数据"。
+- **不绕过访问控制**：登录墙、付费墙、验证码页面一律视为不可读内容。抓取链会在校验门中把这些页面判为无效（`is_login_page` / `is_captcha_page`），此时正确做法是回到检索阶段寻找开放版本（预印本、机构库、作者主页），**而不是尝试绕过**。
+- **不伪造可读性**：`FETCH_FAILED` / `FETCH_PARTIAL` 的内容不得作为证据；不得由模型根据标题或摘要"补写"正文。
+- **私有地址不外发**：指向本机/私网的目标只允许本地读取，绝不提交给第三方清洗服务（`retrieval/fetch.py` 的 `LOCAL_PROVIDERS` 约束）。
+
+## 3. 限速与重试
+
+- 检索按查询批次串行执行，每次 provider 尝试最多重试 1 次（`AuditedSearchExecutor(max_retries=1)`）；失败即记录并切换通道，不做无限重试。
+- 单次运行的检索预算由 `SearchPlan.provider_budget` 限定；S/M/L 分级只影响预算，不影响协议。
+- 超时：检索 12 秒、抓取 20 秒；超时按失败处理并进入降级链。
+- 商用 API 通道按其配额与速率限制使用；配额耗尽是运营问题，不构成放宽其他通道约束的理由。
+
+### Sciverse 通道附加约定
+
+- 单次 `agentic-search` 的 `top_k` 上限 100，且同一篇论文最多返回约 3 个 chunk；`balanced` 模式服务端约截断至 50 条。
+- `filters` 为**软过滤**语义：chunk 侧元数据缺失的文档不会被排除。按年份等条件过滤时，结论表述必须写"近似范围"；需要严格范围时改用 `meta-search` 的结构化字段并核对返回记录。
+- `offset` / `limit` 以 **Unicode 码点**计（与 Python `len` 一致），不是字节；翻页使用返回的 `next_offset`。
+- 引用必须回指论文本身（DOI 或 `unique_id`），**不得把 Sciverse 记为引用目标**——它是读取路径，不是来源。
+
+## 4. 署名与引用
+
+- 引用目标永远是被引文献本身（DOI / 正式 URL / 数据库标识），绝不是检索或清洗通道（`r.jina.ai`、`markdown.new`、Sciverse 等）。
+- 作者、年份、标题、期刊按原文记录，不改写、不翻译、不合并同名作者。
+- 使用的每个来源都要能给出可核验的 `source_location`；没有位置的记录标 `needs_manual_location` 进入人工筛选。
+- 撤稿与更正：已引 DOI 通过 `scripts/retraction_watch.py` 定期核查，命中即移除其支撑作用并重新裁决。
+
+## 5. 缓存与留存
+
+- 抓取正文在 run workspace 的 `fetch/` 下保存（raw + clean + provenance + fallback_chain），用于复现与审计；该目录随 run 生命周期管理。
+- 检索审计（`search-provenance.json` / `search-attempts.jsonl` / `source-screening.csv` / `exclusion-log.csv`）与 run 同寿命，用于证明"检索确实发生过、范围如何"。
+- 不长期镜像第三方全文；需要长期复用时保存定位信息与哈希，而非内容副本。
+- 私有项目、用户上传数据与本地运行历史一律不进入公共产物（提交包由 `scripts/skill_payload.py` 的显式白名单构建）。
+
+## 6. 凭据处理
+
+- API key 只从环境变量读取（`SCIVERSE_API_TOKEN` / `TAVILY_API_KEY` / `BRAVE_API_KEY`）；不写入仓库、不写入产物、不进日志。
+- 错误信息只保留状态与简短描述，不携带 Authorization 头或 token 片段。
+- 缺少 key 时通道静默失活并如实上报状态，不影响零配置通道与科学门。
+
+## 7. 失败与例外
+
+| 情形 | 处理 |
+|---|---|
+| 站点禁止抓取 | 退回元数据记录并标注；不得绕过。 |
+| 付费墙 | 寻找开放版本；找不到则记 `needs_manual_location`。 |
+| 配额耗尽 | 记录该次尝试；切换零配置通道继续。 |
+| 合规与时效冲突 | 以合规为准；宁可结论标注"证据不足"。 |
+
+## 8. 交叉引用
+
+- 检索协议：`references/retrieval-protocol.md`（查询构造、来源分级、饱和规则）
+- 来源有效性：`references/source-validity.md`
+- 抓取与校验实现：`retrieval/fetch.py`、`retrieval/validate.py`
+- Sciverse 通道：`docs/sciverse-api.md`
+

package/references/retrieval-protocol.md

@@ -68,6 +68,24 @@ source-validity.md）后才能进入 Evidence Extraction。
 | RP-03 | 厂商/行业声明（如 Copilot 官方博客、AI 产品宣传页）**一律不得**作为独立证据，即使域名是 `.edu` / `.gov`（需核查内容是否厂商资助）。 |
 | RP-04 | 无法回溯到原始来源的二手转述，不得进入 Evidence Matrix。 |
 
+### 3.1 Sciverse 通道使用规则（key-based 学术通道）
+
+Sciverse（`retrieval/sciverse.py`，需 `SCIVERSE_API_TOKEN`）提供引用级学术检索与全文定位。使用时必须遵守以下语义，否则结论口径会被静默夸大：
+
+| 编号 | 规则 |
+| --- | --- |
+| RP-SV-01 | `/agentic-search` 返回的 chunk 是**定位子**（`doc_id` + Unicode 码点 `offset`），必须经 `/content` 读出正文并通过 `retrieval/validate.py` 校验门，才可进入 Extract（RULE 2 的机器化）。定位写入 `chunks.jsonl`，标注 `discovery_only_requires_content_fetch`。 |
+| RP-SV-02 | `filters` 是**软过滤**：chunk 元数据缺失的文档不会被排除。按年份等条件过滤时，结论与筛选表必须写"近似范围"；需要严格范围时改用 `/meta-search` 的结构化过滤并逐条核对返回记录。 |
+| RP-SV-03 | `offset` / `limit` 以 **Unicode 码点**计（与 Python `len` 一致）；翻页使用返回的 `next_offset`，不得用 `bytes_returned` 推算。 |
+| RP-SV-04 | 调用 `/content` 必须显式传 `offset`（省略会返回整篇全文并忽略 `limit`）。 |
+| RP-SV-05 | 同一篇论文最多返回约 3 个 chunk，`balanced` 模式服务端约截断至 50 条；高 `top_k` 需要足够多的不同论文，不得用同一篇的多个 chunk 充当多项独立证据。 |
+| RP-SV-06 | 引用目标永远是论文本身（DOI / `unique_id`），**不得把 Sciverse 记为来源或抓取渠道**。 |
+| RP-SV-07 | 无 DOI、无 URL 的记录标 `needs_manual_location` 进入人工筛选，**禁止伪造定位**。 |
+| RP-SV-08 | 引文链（`/meta-paper-relations`）用于饱和判断与滚雪球检索，对应 `SearchQuery.purpose = citation_chain`，其方向语义（CITATIONS 被引 / REFERENCES 参考文献）必须在记录中保留。 |
+| RP-SV-09 | 通道不可用（无 token / 401 / 429 / 5xx / 网络失败）时按定型状态记录并切换其他通道；不得因配额耗尽放宽证据标准。 |
+
+端点契约与限制见 `docs/sciverse-api.md`；配额、缓存与署名见 `references/retrieval-compliance.md`。
+
 ## 4. 检索轮次与饱和规则
 
 ### 4.1 最小轮次
@@ -140,3 +158,5 @@ Skeptic 的 9 项固定任务（skeptic-protocol.md）需要对应的独立查
 | RP-09 | snippet 与摘要不得直接作为证据内容（RULE 2）；检索阶段产物只能是线索。 |
 | RP-10 | 厂商声明与二手转述不得作为独立证据（RP-03 / RP-04）。 |
 | RP-11 | 达到饱和规则或数量下限后仍不足的，如实输出 `INSUFFICIENT_SOURCES`，禁止降低纳入标准凑数。 |
+
+来源合规（robots、限速、paywall、署名与凭据）统一见 `references/retrieval-compliance.md`；Sciverse 通道附加约定见本文 §3.1。

package/references/scientific-invariants.md

@@ -0,0 +1,19 @@
+# EduEvidence Scientific Invariants
+
+These rules are protected scientific constraints. Autoresearch may optimize the research process, but must never optimize the evidence set toward a preferred conclusion.
+
+1. **Optimize for decision integrity, not answer confidence.**
+2. **Optimize the research process, never the conclusion.**
+3. **Every iteration must improve the evidence state, improve the research system, or teach why an attempted path failed.**
+4. Validated evidence is append-only regardless of whether it supports, contradicts, is null, or is neutral toward the current decision.
+5. Search snippets are discovery metadata, never direct evidence.
+6. Task performance is not learning; direct learning/retention/independent-transfer evidence is required before ADOPT for education interventions.
+7. Missing uncertainty statistics must never be fabricated.
+8. Causal estimators fail closed on invalid or unidentified designs.
+9. StudyDesign requires an explicit evidence-grounded KnowledgeGap.
+10. Negative search results describe the bounded search scope only; never infer that evidence does not exist globally.
+11. Evidence Autoresearch may update project research state but may not modify Skill/repository code.
+12. Skill Autoresearch may modify approved mutable repository surfaces but may not touch real user research state.
+13. Holdout/gold/evaluator/schema/scientific-invariant surfaces are protected from automatic mutation.
+14. Canonical project state has a single writer. Subagents return staging artifacts only.
+15. No autonomous human-subject study launch, policy deployment, main-branch merge, or release.

package/retrieval/audit.py

@@ -0,0 +1,178 @@
+"""Auditable, bounded search execution for evidence retrieval.
+
+Provider implementations remain deliberately separate. This module records
+the query intent, every provider attempt, screening decisions and saturation
+stop reason so a search result is never mistaken for unobserved provenance.
+"""
+from __future__ import annotations
+
+import csv
+import json
+import time
+from dataclasses import asdict, dataclass, field
+from pathlib import Path
+from typing import Any, Iterable
+
+from retrieval.search import SearchHit
+from retrieval.source import parse_doi_from_url, title_fingerprint
+
+
+@dataclass(frozen=True)
+class SearchQuery:
+    query_id: str
+    query: str
+    purpose: str  # core | expansion | counter_evidence | citation_chain
+
+
+@dataclass(frozen=True)
+class SearchPlan:
+    question: str
+    domain: str
+    concepts: tuple[str, ...]
+    synonyms: tuple[str, ...]
+    inclusion_criteria: tuple[str, ...]
+    exclusion_criteria: tuple[str, ...]
+    queries: tuple[SearchQuery, ...]
+    provider_budget: int = 10
+    policy_version: str = "2026.09"
+
+    @classmethod
+    def from_question(cls, question: str, *, domain: str = "education",
+                      concepts: Iterable[str] = (), synonyms: Iterable[str] = ()) -> "SearchPlan":
+        terms = tuple(dict.fromkeys(x.strip() for x in (*concepts, *synonyms) if x.strip()))
+        core = " ".join(terms) or question
+        return cls(question, domain, tuple(concepts), tuple(synonyms), (), (), (
+            SearchQuery("Q1", core, "core"),
+            SearchQuery("Q2", f"{core} systematic review OR meta-analysis", "expansion"),
+            SearchQuery("Q3", f"{core} null negative harm bias limitation", "counter_evidence"),
+        ))
+
+
+@dataclass
+class SearchAttempt:
+    attempt_id: str
+    query_id: str
+    provider: str
+    started_at: float
+    ended_at: float
+    status: str
+    result_count: int
+    error: str = ""
+    retry_index: int = 0
+
+    def to_dict(self) -> dict[str, Any]:
+        value = asdict(self)
+        value["latency_ms"] = round((self.ended_at - self.started_at) * 1000, 2)
+        return value
+
+
+def _hit_key(hit: SearchHit) -> tuple[str, str, str]:
+    doi = (hit.doi or parse_doi_from_url(hit.url) or "").lower()
+    return doi, hit.url.rstrip("/").lower(), title_fingerprint(hit.title)
+
+
+def dedupe_hits(hits: Iterable[SearchHit]) -> list[SearchHit]:
+    """Deduplicate DOI, canonical URL and normalized title, retaining priority."""
+    seen: set[tuple[str, str]] = set()
+    kept: list[SearchHit] = []
+    for hit in sorted(hits, key=lambda item: (item.score, item.citation_count or 0), reverse=True):
+        doi, url, title = _hit_key(hit)
+        keys = [("doi", doi), ("url", url), ("title", title)]
+        nonempty = [key for key in keys if key[1]]
+        if any(key in seen for key in nonempty):
+            continue
+        seen.update(nonempty)
+        kept.append(hit)
+    return kept
+
+
+class AuditedSearchExecutor:
+    """Runs an explicit plan with bounded retries and durable audit exports."""
+
+    def __init__(self, providers: Iterable[Any], *, max_retries: int = 1):
+        self.providers = list(providers)
+        self.max_retries = max_retries
+
+    def execute(self, plan: SearchPlan, output_dir: Path, *, limit: int = 10) -> list[SearchHit]:
+        output_dir = Path(output_dir)
+        output_dir.mkdir(parents=True, exist_ok=True)
+        attempts: list[SearchAttempt] = []
+        hits: list[SearchHit] = []
+        attempted: set[tuple[str, str]] = set()
+        counter_hits = 0
+        for query in plan.queries:
+            for provider in self.providers:
+                name = getattr(provider, "name", provider.__class__.__name__)
+                fingerprint = (name, query.query)
+                if fingerprint in attempted:
+                    continue
+                attempted.add(fingerprint)
+                for retry in range(self.max_retries + 1):
+                    started = time.time()
+                    try:
+                        result = provider.search(query.query, limit=limit)
+                        ended = time.time()
+                        attempts.append(SearchAttempt(
+                            f"A-{len(attempts) + 1:04d}", query.query_id, name, started, ended,
+                            "success", len(result), retry_index=retry,
+                        ))
+                        hits.extend(result)
+                        if query.purpose == "counter_evidence":
+                            counter_hits += len(result)
+                        break
+                    except Exception as exc:  # failures are recorded, never swallowed
+                        ended = time.time()
+                        attempts.append(SearchAttempt(
+                            f"A-{len(attempts) + 1:04d}", query.query_id, name, started, ended,
+                            "failed", 0, error=f"{type(exc).__name__}: {exc}", retry_index=retry,
+                        ))
+        unique = dedupe_hits(hits)
+        self._write_exports(output_dir, plan, attempts, unique, counter_hits)
+        return unique[:limit]
+
+    @staticmethod
+    def _write_exports(output_dir: Path, plan: SearchPlan, attempts: list[SearchAttempt],
+                       hits: list[SearchHit], counter_hits: int) -> None:
+        (output_dir / "search-provenance.json").write_text(json.dumps({
+            "plan": {**asdict(plan), "queries": [asdict(q) for q in plan.queries]},
+            "search_completed": True,
+            "counter_evidence_queries_executed": sum(q.purpose == "counter_evidence" for q in plan.queries),
+            "counter_evidence_hit_count": counter_hits,
+            "stop_reason": "planned_queries_completed",
+        }, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
+        (output_dir / "search-attempts.jsonl").write_text(
+            "".join(json.dumps(item.to_dict(), ensure_ascii=False) + "\n" for item in attempts), encoding="utf-8")
+        # Sciverse chunk locators are a working set, never evidence: /content
+        # must expand them and the fetch/validate gate must pass first (RULE 2).
+        chunk_records = [
+            {"chunk_id": hit.chunk_id, "doc_id": hit.doc_id, "offset": hit.offset,
+             "title": hit.title, "doi": hit.doi, "provider": hit.provider,
+             "score": hit.score, "locator_state": "discovery_only_requires_content_fetch"}
+            for hit in hits
+            if getattr(hit, "doc_id", None)
+        ]
+        if chunk_records:
+            (output_dir / "chunks.jsonl").write_text(
+                "".join(json.dumps(record, ensure_ascii=False) + "\n" for record in chunk_records),
+                encoding="utf-8")
+        with (output_dir / "source-screening.csv").open("w", newline="", encoding="utf-8") as fh:
+            writer = csv.DictWriter(fh, fieldnames=["title", "doi", "url", "provider", "year",
+                                                    "doc_id", "chunk_id", "offset", "screening_status", "reason"])
+            writer.writeheader()
+            for hit in hits:
+                doi = hit.doi or parse_doi_from_url(hit.url) or ""
+                url = hit.url
+                # Never fabricate a location for a DOI-less record; the explicit
+                # marker routes it to manual screening instead.
+                if not url and not doi:
+                    url = "needs_manual_location"
+                writer.writerow({"title": hit.title, "doi": doi, "url": url,
+                                 "provider": hit.provider, "year": hit.year or "",
+                                 "doc_id": getattr(hit, "doc_id", None) or "",
+                                 "chunk_id": getattr(hit, "chunk_id", None) or "",
+                                 "offset": getattr(hit, "offset", None) if getattr(hit, "offset", None) is not None else "",
+                                 "screening_status": "candidate",
+                                 "reason": "discovery metadata only; fetch and validation required before evidence extraction"})
+        with (output_dir / "exclusion-log.csv").open("w", newline="", encoding="utf-8") as fh:
+            writer = csv.DictWriter(fh, fieldnames=["identifier", "reason"])
+            writer.writeheader()

package/retrieval/fetch.py

@@ -210,6 +210,98 @@ def _fetch_raw_html(url: str, timeout: int) -> tuple[int, str, str]:
     return _http_get(url, timeout=timeout)
 
 
+def fetch_sciverse_content(
+    doc_id: str,
+    *,
+    offset: int = 0,
+    limit: int = 4096,
+    canonical_url: str | None = None,
+    expect_title: str | None = None,
+) -> dict[str, Any]:
+    """Read a Sciverse full-text slice through the same FetchResult contract.
+
+    This is the machine-enforced half of RULE 2: a ``/agentic-search`` chunk is
+    a locator (doc_id + code-point offset), and only the text returned here —
+    after passing the Fetch Validation Gate — may enter Evidence Extraction.
+
+    ``offset``/``limit`` count Unicode code points, exactly like Python ``len``.
+    The locator is recorded inside ``extensions`` because the FetchResult
+    contract is closed (``additionalProperties: false``). ``canonical_url``
+    carries the source's own citation pointer when one exists; the locator URL
+    is used otherwise and is a provenance pointer, never a citation target.
+    """
+    from retrieval.sciverse import (
+        STATUS_OK,
+        STATUS_UNAVAILABLE,
+        read_content,
+    )
+
+    fetched_at = datetime.now(timezone.utc).isoformat()
+    resolved_url = canonical_url or f"https://sciverse.space/doc/{doc_id}"
+    response = read_content(doc_id, offset=offset, limit=limit)
+
+    if response.status == STATUS_UNAVAILABLE:
+        return FetchResult(
+            original_url=resolved_url,
+            fetch_provider="sciverse_content",
+            fetch_status="FETCH_FAILED",
+            fetched_at=fetched_at,
+            validation={"passed": False,
+                        "checks": {"http_success": False, "body_length_ok": False},
+                        "issues": ["SCIVERSE_UNAVAILABLE: no API token configured"]},
+            extensions={"sciverse": {"doc_id": doc_id, "offset": offset,
+                                      "status": response.status, "error": response.error}},
+        ).to_dict()
+
+    if response.status != STATUS_OK:
+        return FetchResult(
+            original_url=resolved_url,
+            fetch_provider="sciverse_content",
+            fetch_status="FETCH_FAILED",
+            fetched_at=fetched_at,
+            validation={"passed": False,
+                        "checks": {"http_success": False, "body_length_ok": False},
+                        "issues": [f"{response.status}: {response.error}".strip(": ")]},
+            extensions={"sciverse": {"doc_id": doc_id, "offset": offset,
+                                      "status": response.status,
+                                      "http_status": response.http_status,
+                                      "error": response.error}},
+        ).to_dict()
+
+    content = str(response.data.get("text") or "")
+    next_offset = response.data.get("next_offset")
+    more = bool(response.data.get("more"))
+    candidate = FetchResult(
+        original_url=resolved_url,
+        resolved_url=resolved_url,
+        fetch_method="smart_web_fetch",
+        fetch_provider="sciverse_content",
+        fetch_status="FETCH_VALID" if content.strip() else "FETCH_FAILED",
+        fetched_at=fetched_at,
+        raw_size=len(content.encode("utf-8")),
+        content=content,
+        extensions={"sciverse": {
+            "doc_id": doc_id,
+            "offset": offset,
+            "limit": limit,
+            "next_offset": next_offset,
+            "more": more,
+            "location_unit": "unicode_code_point",
+        }},
+    )
+    candidate.clean_size = candidate.raw_size
+    candidate.content_length = candidate.raw_size
+    candidate.content_hash = _hash(content) if content else ""
+    # A Sciverse locator URL is not an http(s) target: the scheme/URL-match and
+    # private-target checks do not apply; length and error-page checks do.
+    candidate.validation = validate_fetch_result(
+        candidate.to_dict(), expect_title=expect_title,
+    )
+    if not candidate.validation.get("passed") and candidate.fetch_status == "FETCH_VALID":
+        candidate.fetch_status = "FETCH_PARTIAL"
+    return candidate.to_dict()
+
+
 _PROVIDER_FETCHERS: dict[str, Callable[[str, int], tuple[int, str, str]]] = {
     "builtin": _fetch_builtin,
     "jina_reader": _fetch_jina_reader,
@@ -299,6 +391,9 @@ class FetchResult:
     fallback_chain: list[str] = field(default_factory=list)
     content: str = ""
     validation: dict[str, Any] = field(default_factory=dict)
+    #: Structured extension container (schema: fetch-result.extensions).
+    #: Sciverse locators live here because the top level is closed.
+    extensions: dict[str, Any] = field(default_factory=dict)
 
     def to_dict(self) -> dict[str, Any]:
         return {
@@ -317,6 +412,7 @@ class FetchResult:
             "fallback_chain": self.fallback_chain,
             "content": self.content if self.fetch_status != "FETCH_FAILED" else "",
             "validation": self.validation,
+            "extensions": self.extensions,
         }