agent-comparability 0.5.2__tar.gz → 0.5.4__tar.gz

{agent_comparability-0.5.2/src/agent_comparability.egg-info → agent_comparability-0.5.4}/PKG-INFO

@@ -1,8 +1,12 @@
 Metadata-Version: 2.4
 Name: agent-comparability
-Version: 0.5.2
+Version: 0.5.4
 Summary: Agent evaluation comparability semantics: spec + conformance suite + thin reference implementation
 License: Apache-2.0
+Project-URL: Homepage, https://github.com/Rethymus/agent-comparability
+Project-URL: Repository, https://github.com/Rethymus/agent-comparability
+Project-URL: Issues, https://github.com/Rethymus/agent-comparability/issues
+Project-URL: Changelog, https://github.com/Rethymus/agent-comparability/blob/main/CHANGELOG.md
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
@@ -25,6 +29,7 @@ Dynamic: license-file
 [![CI](https://github.com/Rethymus/agent-comparability/actions/workflows/ci.yml/badge.svg)](https://github.com/Rethymus/agent-comparability/actions/workflows/ci.yml)
 [![License: Apache-2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
 [![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-3776AB?logo=python&logoColor=white)](pyproject.toml)
+[![PyPI](https://img.shields.io/pypi/v/agent-comparability)](https://pypi.org/project/agent-comparability/)
 
 **English** | [简体中文](README.zh-CN.md)
 
@@ -74,6 +79,12 @@ Agent Comparability is a framework-independent **specification** ([spec/SPEC.md]
 a frozen **conformance suite**, and a thin **reference implementation** that adjudicates
 *comparability* of agent evaluation records — never the scores themselves.
 
+**Does it actually refuse unsupported claims?** See
+[experiments/scenarios_001/RECORD.md](experiments/scenarios_001/RECORD.md): four real-pipeline
+scenarios (real `.eval` logs → converter → core) covering all four verdicts — including the
+confounded comparison and the missing-trajectory case, both correctly **refused** rather than
+packaged as supported conclusions.
+
 **Who is this for**: developers comparing agent/prompt/harness evaluation results,
 benchmark authors documenting what conclusions their data supports, and teams building
 reliability pipelines on top of existing eval frameworks.
@@ -97,6 +108,8 @@ Three failure modes silently corrupt agent-evaluation conclusions:
 | **Evidence gaps** | Missing traces quietly inflate success rates; rankings reverse across harnesses |
 
 These boundaries should be **machine-checkable**, not vibes. That is what this project ships.
+How this relates to benchmark-rigor and transcript-audit work (verified literature anchors):
+see [docs/positioning.md](docs/positioning.md).
 
 ## 3. What Agent Comparability Does
 
@@ -153,10 +166,30 @@ pip install -e .[suite]                 # + suite basics (pyyaml, jsonschema)
 pip install -e .[suite,inspect,pydantic]  # FULL conformance run — required by run_suite.py
 ```
 
-Requires Python ≥ 3.10. No Docker, no model keys.
+Requires Python ≥ 3.10. No Docker, no model keys. Note: the core package supports 3.10+;
+the optional `inspect`/`pydantic` extras pull dependencies that require Python ≥ 3.11 —
+on 3.10 pip resolves older versions of them.
 
 ## 7. Quickstart
 
+**Command line** (offline, zero extra dependencies — input contract below):
+
+```bash
+python -m agent_comparability examples/demo-case.json --claim score_comparison
+# → CDR JSON on stdout (exit 0). INADMISSIBLE / PARTIAL / INCONCLUSIVE are verdicts,
+#   not program failures. Invalid input (bad JSON / missing or mistyped fields) →
+#   exit 2 with a path-precise diagnostic — never silently converted into a verdict.
+#   -o cdr.json writes to a file instead. The demo case is SYNTHETIC/illustrative.
+```
+
+Input contract (`{"id": str, "records": [{"task_id", "harness", "model", "reward":
+number, "reward_status", ...optional}]}`); unknown record fields are ignored and
+recorded in `unknown_conditions` (SPEC §5). Add `--manifest out.manifest.json` to
+also emit an Evidence Manifest documenting artifact digests and evidence trust
+states (never auto-upgraded; see [docs/evidence-manifest.md](docs/evidence-manifest.md)).
+
+**Python API**:
+
 ```python
 from agent_comparability import adjudicate
 
@@ -222,7 +255,7 @@ CDR — actual output:
     "rec:t1/m1/h1",
     "rec:t1/m1/h2"
   ],
-  "policy_version": "cmp/0.3.1",
+  "policy_version": "cmp/0.4.0",
   "limitations": [
     "摘要一致仅证明未同步修改，不构成抗恶意篡改（单写信任域）"
   ]
@@ -281,10 +314,14 @@ Both wrap the *same* core (no semantic duplication). Inspect scorer uses
 The Pydantic evaluator checks the **task's real output** (`ctx.output`) against
 frozen expectations — it never re-adjudicates inside the evaluator — and the
 adapter's exit code derives from the full 22-pair `report` (cases + failures).
+For **real Inspect EvalLog files** (not frozen cases) use the converter:
+`python -m agent_comparability.inspect_log run.eval --claim … --status …`
+(official APIs, explicit mapping policy, no silent guessing — see
+[docs/inspect-log.md](docs/inspect-log.md)).
 
 ## 12. Specification & Versioning
 
-Three distinct lines: software tag (e.g. `v0.5.0`), specification `cmp/0.3.1`
+Three distinct lines: software tag (e.g. `v0.5.0`), specification `cmp/0.4.0`
 (inside SPEC), and `agent_comparability.__version__`. Judgment-table changes
 require a spec major bump (SPEC §5). See [CHANGELOG](CHANGELOG.md) for the full tag lineage
 (v0.3.1–v0.3.6 are deprecated candidates kept immutably for the record).
@@ -318,13 +355,13 @@ No — no network, no credentials, no Docker. It only ever sees the records you
 
 **Which versions should I depend on?**
 Package `v0.5.0+` (version lines unified and CI-enforced since then); specification
-`cmp/0.3.1` evolves independently — see [§12](#12-specification--versioning).
+`cmp/0.4.0` evolves independently — see [§12](#12-specification--versioning).
 
 ## Project status
 
 | Component | Status |
 |---|---|
-| Specification `cmp/0.3.1` | Stable — frozen conformance set anchors it |
+| Specification `cmp/0.4.0` | Stable — frozen conformance set anchors it |
 | Claim types | `score_comparison`, `causal`, `descriptive_registration` (+ unknown → `INCONCLUSIVE`) |
 | Conformance cases | 22 frozen pairs (A + B sets), three separated metrics, all green in CI |
 | Adapters | Inspect AI ✓ · Pydantic Evals ✓ |

{agent_comparability-0.5.2 → agent_comparability-0.5.4}/README.md

@@ -7,6 +7,7 @@
 [![CI](https://github.com/Rethymus/agent-comparability/actions/workflows/ci.yml/badge.svg)](https://github.com/Rethymus/agent-comparability/actions/workflows/ci.yml)
 [![License: Apache-2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
 [![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-3776AB?logo=python&logoColor=white)](pyproject.toml)
+[![PyPI](https://img.shields.io/pypi/v/agent-comparability)](https://pypi.org/project/agent-comparability/)
 
 **English** | [简体中文](README.zh-CN.md)
 
@@ -56,6 +57,12 @@ Agent Comparability is a framework-independent **specification** ([spec/SPEC.md]
 a frozen **conformance suite**, and a thin **reference implementation** that adjudicates
 *comparability* of agent evaluation records — never the scores themselves.
 
+**Does it actually refuse unsupported claims?** See
+[experiments/scenarios_001/RECORD.md](experiments/scenarios_001/RECORD.md): four real-pipeline
+scenarios (real `.eval` logs → converter → core) covering all four verdicts — including the
+confounded comparison and the missing-trajectory case, both correctly **refused** rather than
+packaged as supported conclusions.
+
 **Who is this for**: developers comparing agent/prompt/harness evaluation results,
 benchmark authors documenting what conclusions their data supports, and teams building
 reliability pipelines on top of existing eval frameworks.
@@ -79,6 +86,8 @@ Three failure modes silently corrupt agent-evaluation conclusions:
 | **Evidence gaps** | Missing traces quietly inflate success rates; rankings reverse across harnesses |
 
 These boundaries should be **machine-checkable**, not vibes. That is what this project ships.
+How this relates to benchmark-rigor and transcript-audit work (verified literature anchors):
+see [docs/positioning.md](docs/positioning.md).
 
 ## 3. What Agent Comparability Does
 
@@ -135,10 +144,30 @@ pip install -e .[suite]                 # + suite basics (pyyaml, jsonschema)
 pip install -e .[suite,inspect,pydantic]  # FULL conformance run — required by run_suite.py
 ```
 
-Requires Python ≥ 3.10. No Docker, no model keys.
+Requires Python ≥ 3.10. No Docker, no model keys. Note: the core package supports 3.10+;
+the optional `inspect`/`pydantic` extras pull dependencies that require Python ≥ 3.11 —
+on 3.10 pip resolves older versions of them.
 
 ## 7. Quickstart
 
+**Command line** (offline, zero extra dependencies — input contract below):
+
+```bash
+python -m agent_comparability examples/demo-case.json --claim score_comparison
+# → CDR JSON on stdout (exit 0). INADMISSIBLE / PARTIAL / INCONCLUSIVE are verdicts,
+#   not program failures. Invalid input (bad JSON / missing or mistyped fields) →
+#   exit 2 with a path-precise diagnostic — never silently converted into a verdict.
+#   -o cdr.json writes to a file instead. The demo case is SYNTHETIC/illustrative.
+```
+
+Input contract (`{"id": str, "records": [{"task_id", "harness", "model", "reward":
+number, "reward_status", ...optional}]}`); unknown record fields are ignored and
+recorded in `unknown_conditions` (SPEC §5). Add `--manifest out.manifest.json` to
+also emit an Evidence Manifest documenting artifact digests and evidence trust
+states (never auto-upgraded; see [docs/evidence-manifest.md](docs/evidence-manifest.md)).
+
+**Python API**:
+
 ```python
 from agent_comparability import adjudicate
 
@@ -204,7 +233,7 @@ CDR — actual output:
     "rec:t1/m1/h1",
     "rec:t1/m1/h2"
   ],
-  "policy_version": "cmp/0.3.1",
+  "policy_version": "cmp/0.4.0",
   "limitations": [
     "摘要一致仅证明未同步修改，不构成抗恶意篡改（单写信任域）"
   ]
@@ -263,10 +292,14 @@ Both wrap the *same* core (no semantic duplication). Inspect scorer uses
 The Pydantic evaluator checks the **task's real output** (`ctx.output`) against
 frozen expectations — it never re-adjudicates inside the evaluator — and the
 adapter's exit code derives from the full 22-pair `report` (cases + failures).
+For **real Inspect EvalLog files** (not frozen cases) use the converter:
+`python -m agent_comparability.inspect_log run.eval --claim … --status …`
+(official APIs, explicit mapping policy, no silent guessing — see
+[docs/inspect-log.md](docs/inspect-log.md)).
 
 ## 12. Specification & Versioning
 
-Three distinct lines: software tag (e.g. `v0.5.0`), specification `cmp/0.3.1`
+Three distinct lines: software tag (e.g. `v0.5.0`), specification `cmp/0.4.0`
 (inside SPEC), and `agent_comparability.__version__`. Judgment-table changes
 require a spec major bump (SPEC §5). See [CHANGELOG](CHANGELOG.md) for the full tag lineage
 (v0.3.1–v0.3.6 are deprecated candidates kept immutably for the record).
@@ -300,13 +333,13 @@ No — no network, no credentials, no Docker. It only ever sees the records you
 
 **Which versions should I depend on?**
 Package `v0.5.0+` (version lines unified and CI-enforced since then); specification
-`cmp/0.3.1` evolves independently — see [§12](#12-specification--versioning).
+`cmp/0.4.0` evolves independently — see [§12](#12-specification--versioning).
 
 ## Project status
 
 | Component | Status |
 |---|---|
-| Specification `cmp/0.3.1` | Stable — frozen conformance set anchors it |
+| Specification `cmp/0.4.0` | Stable — frozen conformance set anchors it |
 | Claim types | `score_comparison`, `causal`, `descriptive_registration` (+ unknown → `INCONCLUSIVE`) |
 | Conformance cases | 22 frozen pairs (A + B sets), three separated metrics, all green in CI |
 | Adapters | Inspect AI ✓ · Pydantic Evals ✓ |

{agent_comparability-0.5.2 → agent_comparability-0.5.4}/pyproject.toml

@@ -1,23 +1,29 @@
-[build-system]
-requires = ["setuptools>=68"]
-build-backend = "setuptools.build_meta"
-
-[project]
-name = "agent-comparability"
-version = "0.5.2"
-description = "Agent evaluation comparability semantics: spec + conformance suite + thin reference implementation"
-readme = "README.md"
-requires-python = ">=3.10"
-license = {text = "Apache-2.0"}
-dependencies = []
-
-[project.optional-dependencies]
-suite = ["pyyaml", "jsonschema"]
-inspect = ["inspect-ai"]
-pydantic = ["pydantic-evals"]
-
-[tool.setuptools.packages.find]
-where = ["src"]
-
-[tool.setuptools.package-data]
-agent_comparability = ["data/*"]
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "agent-comparability"
+version = "0.5.4"
+description = "Agent evaluation comparability semantics: spec + conformance suite + thin reference implementation"
+readme = "README.md"
+requires-python = ">=3.10"
+license = {text = "Apache-2.0"}
+dependencies = []
+
+[project.urls]
+Homepage = "https://github.com/Rethymus/agent-comparability"
+Repository = "https://github.com/Rethymus/agent-comparability"
+Issues = "https://github.com/Rethymus/agent-comparability/issues"
+Changelog = "https://github.com/Rethymus/agent-comparability/blob/main/CHANGELOG.md"
+
+[project.optional-dependencies]
+suite = ["pyyaml", "jsonschema"]
+inspect = ["inspect-ai"]
+pydantic = ["pydantic-evals"]
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+agent_comparability = ["data/*"]

agent_comparability-0.5.4/src/agent_comparability/__init__.py

@@ -0,0 +1,3 @@
+"""Agent Comparability — cmp/0.4.0 参考实现。"""
+from .core import CDR, adjudicate, POLICY_VERSION
+__version__ = "0.5.4"

agent_comparability-0.5.4/src/agent_comparability/__main__.py

@@ -0,0 +1,131 @@
+"""离线 JSON → CDR 命令行闭环（#16，零额外运行时依赖）。
+
+用法：
+    python -m agent_comparability input.json --claim score_comparison [-o cdr.json]
+
+输入契约（= adjudicate 既有输入形状）：
+    {"id": str,
+     "records": [{"task_id": str, "harness": str, "model": str,
+                  "reward": number, "reward_status": str,
+                  ...可选 trajectory_available/score_scale/repetition_count}]}
+
+退出码契约：
+    0 = 产出 CDR（ADMISSIBLE/INADMISSIBLE/PARTIAL/INCONCLUSIVE 均为裁决成功，非程序失败）
+    2 = 输入错误（文件不可读/JSON 语法/契约字段缺失或类型错）——诊断含出错路径；
+        无效输入绝不允许被转换为貌似成功的裁决
+
+本模块只做契约存在性与类型检查；一切判定语义由 adjudicate() 负责（不重新实现判定表，
+不改规范语义）。未知字段按 SPEC §5 忽略并记入 CDR 的 unknown_conditions，不在此报错。
+"""
+import argparse
+import json
+import sys
+
+from . import adjudicate
+
+REQUIRED_STR = ("task_id", "harness", "model", "reward_status")
+
+
+class ContractError(Exception):
+    """输入违反契约：消息含出错路径，如 records[2].reward。"""
+
+
+def _is_number(v):
+    return isinstance(v, (int, float)) and not isinstance(v, bool)
+
+
+def _is_int(v):
+    return isinstance(v, int) and not isinstance(v, bool)
+
+
+def validate_case(case):
+    if not isinstance(case, dict):
+        raise ContractError("输入顶层必须是 JSON 对象，实际是 " + type(case).__name__)
+    if "id" not in case:
+        raise ContractError("缺少必填字段 id")
+    if not isinstance(case["id"], str):
+        raise ContractError("id 必须是字符串，实际是 " + type(case["id"]).__name__)
+    if "records" not in case:
+        raise ContractError("缺少必填字段 records")
+    if not isinstance(case["records"], list) or not case["records"]:
+        raise ContractError("records 必须是非空数组")
+    for i, r in enumerate(case["records"]):
+        where = f"records[{i}]"
+        if not isinstance(r, dict):
+            raise ContractError(f"{where} 必须是 JSON 对象，实际是 " + type(r).__name__)
+        for k in REQUIRED_STR:
+            if k not in r:
+                raise ContractError(f"{where}.{k} 缺失（必填字段）")
+            if not isinstance(r[k], str):
+                raise ContractError(f"{where}.{k} 必须是字符串，实际是 " + type(r[k]).__name__)
+        if "reward" not in r:
+            raise ContractError(f"{where}.reward 缺失（必填字段）")
+        if not _is_number(r["reward"]):
+            raise ContractError(f"{where}.reward 必须是数值，实际是 " + type(r["reward"]).__name__)
+        if "trajectory_available" in r and not isinstance(r["trajectory_available"], bool):
+            raise ContractError(f"{where}.trajectory_available 必须是布尔值")
+        if "score_scale" in r and not isinstance(r["score_scale"], dict):
+            raise ContractError(f"{where}.score_scale 必须是对象 {{range, unit, verifier}}")
+        if "repetition_count" in r and (not _is_int(r["repetition_count"]) or r["repetition_count"] < 1):
+            raise ContractError(f"{where}.repetition_count 必须是 >=1 的整数")
+
+
+def main(argv=None) -> int:
+    p = argparse.ArgumentParser(
+        prog="python -m agent_comparability",
+        description="离线评测记录 JSON → 结构化 CDR（Comparison Decision Record）")
+    p.add_argument("input", help="输入 JSON 文件路径（契约见模块文档）")
+    p.add_argument("--claim", required=True,
+                   help="请求结论类型：score_comparison / causal / descriptive_registration"
+                        "（未支持类型 → INCONCLUSIVE，不猜测）")
+    p.add_argument("-o", "--output", help="CDR 输出文件路径（缺省打印到 stdout）")
+    p.add_argument("--manifest", help="同时输出 Evidence Manifest（#18）：输入文件作为证据制品"
+                                      "实算 sha256，全部记录 artifact_observed，不含任何独立验证声明")
+    args = p.parse_args(argv)
+
+    try:
+        raw = open(args.input, encoding="utf-8").read()
+    except OSError as e:
+        print(f"error: 无法读取输入文件 {args.input}: {e.strerror or e}", file=sys.stderr)
+        return 2
+    try:
+        case = json.loads(raw)
+    except json.JSONDecodeError as e:
+        print(f"error: JSON 解析错误 {args.input}（第 {e.lineno} 行第 {e.colno} 列）: {e.msg}", file=sys.stderr)
+        return 2
+    try:
+        validate_case(case)
+    except ContractError as e:
+        print(f"error: 输入违反契约 {args.input}: {e}", file=sys.stderr)
+        return 2
+
+    cdr = adjudicate(case, args.claim).to_dict()
+    out = json.dumps(cdr, ensure_ascii=False, indent=2)
+    if args.output:
+        try:
+            with open(args.output, "w", encoding="utf-8") as f:
+                f.write(out + "\n")
+        except OSError as e:
+            print(f"error: 无法写入输出文件 {args.output}: {e.strerror or e}", file=sys.stderr)
+            return 2
+    else:
+        print(out)
+    if args.manifest:
+        from .evidence import build_manifest
+        # 离线 JSON 的诚实默认：输入文件即证据制品（读后实算摘要）；无来源端验证能力，
+        # 无独立验证声明——绝不把 source_claimed 冒充为已验证。
+        m = build_manifest(case, source_kind="offline_json",
+                           artifacts={i: args.input for i in range(len(case["records"]))},
+                           requested_claim=args.claim,
+                           unsupported_capabilities=["source_side_verification_unavailable"])
+        try:
+            with open(args.manifest, "w", encoding="utf-8") as f:
+                f.write(json.dumps(m, ensure_ascii=False, indent=2) + "\n")
+        except OSError as e:
+            print(f"error: 无法写入 manifest 文件 {args.manifest}: {e.strerror or e}", file=sys.stderr)
+            return 2
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

{agent_comparability-0.5.2 → agent_comparability-0.5.4}/src/agent_comparability/core.py

@@ -1,16 +1,20 @@
-"""比较判定核心库 cmp/core（cmp/0.3.1 规范参考实现，第十九轮升级）。
+"""比较判定核心库 cmp/core（cmp/0.4.0 规范参考实现）。
 
+相对 0.3.1 的判定表分支修正（#27 裁决方向 B，SPEC §5 允许 minor 新增分支）：
+  ⑤score_comparison 的 INADMISSIBLE 触发移除 same_model——单变量对称性
+    （§4 行 2：仅模型不同 → ADMISSIBLE 并排）；模型-only 变化由
+    score_scale_consistent / zero_semantics_uniform 独立把关（D2 冻结对证明）。
 相对 0.2 的规范符合性修复（依据 SPEC 0.3.1；B 集冻结预期先于本修改存在）：
   ①同任务性检查入 score 判定优先级 2（B3 暴露 0.2 缺失）
   ②评分尺度一致性检查（含 0.3.1 隐式默认规则）（B4/B5 暴露 0.2 缺失）
   ③claim 路由补 registration/descriptive 关键词（B2-B5 暴露误判）
   ④policy_version 升 cmp/0.3.1（消费方按 SPEC §5 重判旧 CDR）
-判定优先级（SPEC §3.3）：INCONCLUSIVE(缺trace) > INADMISSIBLE(任务/模型/混淆/尺度) > PARTIAL(零语义) > ADMISSIBLE。
+判定优先级（SPEC §3.3）：INCONCLUSIVE(缺trace) > INADMISSIBLE(任务/混淆/尺度) > PARTIAL(零语义) > ADMISSIBLE。
 """
 
 from dataclasses import dataclass, field, asdict
 
-POLICY_VERSION = "cmp/0.3.1"
+POLICY_VERSION = "cmp/0.4.0"
 TAMPER_LIMITATION = "摘要一致仅证明未同步修改，不构成抗恶意篡改（单写信任域）"
 DEFAULT_SCALE = {"range": "0..1", "unit": "fraction", "verifier": "source"}
 KNOWN_FIELDS = {"task_id", "harness", "model", "reward", "reward_status",
@@ -92,9 +96,10 @@ def adjudicate(case: dict, requested_claim: str) -> CDR:
             cdr.admissibility = "INCONCLUSIVE"
             cdr.supported_claims.append("score_transcription(source_verifier_attested)")
             cdr.unsupported_claims.append("success_rate(含证据缺失记录)")
-        elif len(tasks) > 1 or len(models) > 1 or confounded or "score_scale_consistent" in cdr.violated_conditions:
+        elif len(tasks) > 1 or confounded or "score_scale_consistent" in cdr.violated_conditions:
+            # 0.4：same_model 移出硬触发（#27 单变量对称性）；尺度/零语义条件独立把关
             cdr.admissibility = "INADMISSIBLE"
-            cdr.unsupported_claims.append("score_comparison(任务/模型/尺度条件冲突)")
+            cdr.unsupported_claims.append("score_comparison(任务/混淆/尺度条件冲突)")
         elif any(v.startswith("zero_semantics_uniform") for v in cdr.violated_conditions):
             cdr.admissibility = "PARTIAL"
             cdr.supported_claims.append("per_status_stratified_rate(verified_zero/unresolved_zero 分列)")

{agent_comparability-0.5.2 → agent_comparability-0.5.4}/src/agent_comparability/data/SPEC.md

@@ -1,4 +1,4 @@
-# Agent Evaluation Comparability Specification — cmp/0.3.1 (Release Candidate)
+# Agent Evaluation Comparability Specification — cmp/0.4.0 (Release Candidate)
 
 规范状态：候选规范（非行业标准、无第三方认证）。本文是判定语义的单一事实来源；参考实现（`scripts/cmp/core.py`）与一切第三方实现均以本文为准，冲突时以本文裁决。可脱离任何源码阅读。
 
@@ -32,7 +32,7 @@
 | 类型 | 语义 |
 |---|---|
 | `causal` | 断言某变量（模型/Harness）**导致**了分数差异 |
-| `score_comparison` | 请求对连续分数做（并排或汇总）比较 |
+| `score_comparison` | 请求对连续分数做（并排或汇总）比较。历史冻结案例集使用早期写法 `score_compare`；参考实现通过下划线归一化与关键词匹配接受两者（实现细节，非规范语义），新代码应一律使用 `score_comparison`。 |
 | `descriptive_registration` | 仅登记条件与分数，不作比较结论 |
 | 其他 | **未支持类型** |
 
@@ -67,7 +67,10 @@
 
 **`score_comparison`**（按优先级，先命中先裁决）：
 1. 违反 `trace_available` → `INCONCLUSIVE`（来源评分转录保留于 `unknown_conditions`，禁止删除）
-2. 违反 `single_variable_isolation` **或** `same_task_id` 或 `same_model` → `INADMISSIBLE`
+2. 违反 `single_variable_isolation` **或** `same_task_id` → `INADMISSIBLE`
+   （**cmp/0.4 修正（#27）**：`same_model` 不再单独触发 INADMISSIBLE——单变量对称性，与
+   §4 行 1/行 2 一致；模型-only 变化由 `score_scale_consistent`/`zero_semantics_uniform`
+   独立把关。0.3.x 行为见版本记录。）
 3. 违反 `score_scale_consistent` → `INADMISSIBLE`
 4. 违反 `zero_semantics_uniform` → `PARTIAL`（支持按 verified/unresolved 分列的率；禁止合并 fail 率）
 5. 其余 → `ADMISSIBLE`（同尺度连续分数并排）
@@ -95,7 +98,7 @@
 
 - **未知字段**：Record 中规范未定义的字段**必须忽略**（不得报错、不得猜测），并在 `unknown_conditions` 记 `ignored_fields:[...]`。
 - **未支持 Claim**：见 §2 —— INCONCLUSIVE + `unrecognized_claim_type`，禁止近似匹配。
-- **Policy 版本**：CDR 必须携带 `policy_version`。消费方遇到 `policy_version` 不认识时**必须丢弃该 CDR 并重新判定**，不得部分信任。本规范为 `cmp/0.3.1`（0.3 的澄清版：仅补隐式默认尺度规则，判定表未变）。
+- **Policy 版本**：CDR 必须携带 `policy_version`。消费方遇到 `policy_version` 不认识时**必须丢弃该 CDR 并重新判定**，不得部分信任。本规范为 `cmp/0.4.0`（0.3→0.4：判定表分支修正——score_comparison 的 INADMISSIBLE 触发移除 `same_model`，实现单变量对称性，裁决记录见 #27；D 冻结对（D1/D2）先于实现冻结）。0.3.1 为 0.3 的澄清版（仅补隐式默认尺度规则）。
 - **规范不兼容变更**：升 minor（0.3→0.4）可新增判定分支；改判定表必须升 major 并迁移案例。
 
 ---

agent_comparability-0.5.4/src/agent_comparability/data/cases-0.4.json

@@ -0,0 +1,16 @@
+{
+  "policy_version": "cmp/0.4.0",
+  "frozen_note": "D 集（#27 裁决方向 B，先于实现冻结）：单变量对称性——模型-only 变化遵循 SPEC §4 行 2。A/B 集与既有预期零改动；D2 证明尺度条件独立把关未被放松。",
+  "cases": [
+    {"id": "D1", "category": "model-only-variation",
+     "records": [
+       {"task_id": "t-model-only", "harness": "h1", "model": "model-a", "reward": 0.8, "reward_status": "verified"},
+       {"task_id": "t-model-only", "harness": "h1", "model": "model-b", "reward": 0.6, "reward_status": "verified"}],
+     "claims": {"score_comparison": "ADMISSIBLE", "causal": "INADMISSIBLE"}},
+    {"id": "D2", "category": "model-plus-scale-violation",
+     "records": [
+       {"task_id": "t-model-scale", "harness": "h1", "model": "model-a", "reward": 0.8, "reward_status": "verified"},
+       {"task_id": "t-model-scale", "harness": "h1", "model": "model-b", "reward": 62.0, "reward_status": "verified"}],
+     "claims": {"score_comparison": "INADMISSIBLE", "descriptive_registration": "ADMISSIBLE"}}
+  ]
+}

{agent_comparability-0.5.2 → agent_comparability-0.5.4}/src/agent_comparability/data/cdr.schema.json

@@ -74,7 +74,7 @@
     },
     "policy_version": {
       "type": "string",
-      "pattern": "^cmp/0\\.3(\\.1)?$"
+      "pattern": "^cmp/0\\.[34](\\.\\d+)?$"
     },
     "limitations": {
       "type": "array",

agent_comparability-0.5.4/src/agent_comparability/evidence.py

@@ -0,0 +1,109 @@
+"""Evidence Manifest（#18）——CDR 外部的来源与证据可信边界。
+
+三态（**证据制品级**，与 SPEC §1 的记录级评分信任四层是不同维度，绝不进入
+adjudicate、不改变任何 CDR 字段或判定结果）：
+
+    source_claimed < artifact_observed < independently_verified
+
+升级只能由显式证据触发：
+  - artifact_observed ⟺ 制品路径存在且被实际读取且 sha256 已实算记录；
+  - independently_verified ⟺ 在 artifact_observed 之上存在显式 verification 条目
+    （method/operator/at 三者齐备）；
+  - 无制品 → 地板 source_claimed + 警告；有 verification 无制品 → 不一致，不升级。
+不存在任何隐式跃迁（防"证据不足被包装成可信结论"）。
+"""
+import hashlib
+
+from . import POLICY_VERSION
+
+MANIFEST_VERSION = "evidence/0.1"
+STATES = ("source_claimed", "artifact_observed", "independently_verified")
+
+
+def _digest(path):
+    h = hashlib.sha256()
+    try:
+        with open(path, "rb") as f:
+            for chunk in iter(lambda: f.read(65536), b""):
+                h.update(chunk)
+    except OSError:
+        return None
+    return "sha256:" + h.hexdigest()
+
+
+def build_manifest(case, source_kind, source_version=None, artifacts=None,
+                   verifications=None, requested_claim=None,
+                   unsupported_capabilities=None):
+    """构建 manifest。artifacts: {record_index: 制品路径}；verifications: {record_index:
+    {method, operator, at}}。状态只由提供的证据决定——本函数绝不主动升级。"""
+    artifacts = artifacts or {}
+    verifications = verifications or {}
+    warnings = []
+    records = []
+    for i, r in enumerate(case["records"]):
+        state, digest, art_ref, reason = "source_claimed", None, None, None
+        if i in artifacts:
+            art_ref = artifacts[i]
+            digest = _digest(art_ref)
+            if digest is None:
+                warnings.append(f"records[{i}]:artifact_missing({art_ref} 不可读——保持 source_claimed)")
+                reason = f"artifact_unreadable:{art_ref}"
+            else:
+                state = "artifact_observed"
+        else:
+            warnings.append(f"records[{i}]:artifact_missing(未提供制品——保持 source_claimed)")
+            reason = "no_artifact_provided"
+        verification = None
+        if i in verifications:
+            v = verifications[i]
+            valid = isinstance(v, dict) and {"method", "operator", "at"} <= set(v)
+            if state != "artifact_observed":
+                warnings.append(f"records[{i}]:verification_without_artifact("
+                                f"无法独立验证从未观察过的制品——不升级)")
+            elif not valid:
+                warnings.append(f"records[{i}]:verification_entry_invalid(需 method/operator/at)")
+            else:
+                state = "independently_verified"
+                verification = {"method": v["method"], "operator": v["operator"], "at": v["at"]}
+        records.append({
+            "record_id": f"records[{i}]",
+            "task_identity": {"task_id": r["task_id"], "model": r["model"], "harness": r["harness"]},
+            "artifact_ref": art_ref,
+            "file_digest": digest,
+            "verification_state": state,
+            "verification": verification,
+            "unverified_reason": reason,
+            "provenance_note": (f"state={state}; artifact="
+                                f"{'read+sha256' if digest else 'none'}; source={source_kind}"),
+        })
+    return {
+        "manifest_version": MANIFEST_VERSION,
+        "policy_version": POLICY_VERSION,  # 引用锚定：标识判定所用规范版本，不参与判定
+        "source_kind": source_kind,
+        "source_version": source_version,  # None = 不可取得（诚实记录，不猜测）
+        "case_id": case["id"],
+        "requested_claim": requested_claim,
+        "records": records,
+        "mapping_warnings": warnings,
+        "unsupported_capabilities": unsupported_capabilities or [],
+    }
+
+
+def verify_manifest(manifest):
+    """重算校验（供审计与后续转换器复用）：重读 artifact 实算摘要比对记录值。
+    返回问题列表；空列表 = 干净。只验证已声明观察过的制品——不产生新状态。"""
+    problems = []
+    for e in manifest.get("records", []):
+        rid, ref, dig = e.get("record_id"), e.get("artifact_ref"), e.get("file_digest")
+        if e.get("verification_state") in ("artifact_observed", "independently_verified"):
+            if not ref or not dig:
+                problems.append(f"{rid}:structural(声明已观察但缺 artifact_ref/file_digest)")
+                continue
+            now = _digest(ref)
+            if now is None:
+                problems.append(f"{rid}:artifact_missing_on_reverify({ref})")
+            elif now != dig:
+                problems.append(f"{rid}:digest_mismatch({ref})")
+        if e.get("verification_state") == "independently_verified" and not e.get("verification"):
+            problems.append(f"{rid}:structural(verified 态缺 verification 条目)")
+    return problems