proofbundle 0.4.1__tar.gz → 0.6.0__tar.gz

{proofbundle-0.4.1/src/proofbundle.egg-info → proofbundle-0.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: proofbundle
-Version: 0.4.1
+Version: 0.6.0
 Summary: Emit and verify portable cryptographic evidence bundles, offline: Ed25519 + RFC 6962 Merkle + optional SD-JWT.
 Author: Konrad Gruszka
 License: MIT
@@ -27,6 +27,8 @@ Provides-Extra: sdjwt
 Provides-Extra: eval
 Requires-Dist: rfc8785>=0.1.4; extra == "eval"
 Provides-Extra: adapters
+Provides-Extra: inspect
+Requires-Dist: inspect_ai<0.4,>=0.3.100; extra == "inspect"
 Provides-Extra: dev
 Requires-Dist: pytest>=7; extra == "dev"
 Requires-Dist: ruff>=0.5; extra == "dev"
@@ -35,6 +37,8 @@ Requires-Dist: mypy>=1.8; extra == "dev"
 Requires-Dist: build>=1; extra == "dev"
 Requires-Dist: hypothesis>=6; extra == "dev"
 Requires-Dist: rfc8785>=0.1.4; extra == "dev"
+Requires-Dist: sd-jwt>=0.10; extra == "dev"
+Requires-Dist: inspect_ai<0.4,>=0.3.100; extra == "dev"
 Dynamic: license-file
 
 <div align="center">
@@ -51,17 +55,19 @@ signed and anchored in a tamper-evident log — and optionally carries a
 selectively disclosable credential. Pure Python, no server, no daemon, one JSON file.**
 
 [![CI](https://github.com/b7n0de/proofbundle/actions/workflows/ci.yml/badge.svg)](https://github.com/b7n0de/proofbundle/actions/workflows/ci.yml)
-[![PyPI](https://img.shields.io/pypi/v/proofbundle.svg?color=D6248A)](https://pypi.org/project/proofbundle/)
-[![Python](https://img.shields.io/pypi/pyversions/proofbundle.svg?color=D6248A)](https://pypi.org/project/proofbundle/)
+[![PyPI](https://img.shields.io/pypi/v/proofbundle.svg?color=D6248A&cacheSeconds=3600)](https://pypi.org/project/proofbundle/)
+[![Python](https://img.shields.io/pypi/pyversions/proofbundle.svg?color=D6248A&cacheSeconds=3600)](https://pypi.org/project/proofbundle/)
+[![Downloads](https://static.pepy.tech/badge/proofbundle)](https://pepy.tech/project/proofbundle)
 [![License: MIT](https://img.shields.io/badge/license-MIT-D6248A.svg)](LICENSE)
 [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
 [![SLSA build provenance](https://img.shields.io/badge/SLSA-build_provenance-D6248A.svg)](https://slsa.dev)
+[![PyPI attestations](https://img.shields.io/badge/PyPI-attestations_(PEP_740)-D6248A.svg)](https://pypi.org/project/proofbundle/)
 
 </div>
 
 **At a glance:** `proofbundle emit` signs and anchors a payload; `proofbundle
 verify` checks one self-contained `bundle.json` with three offline cryptographic
-checks → `OK` or `FAILED`. No network, no daemon, no own crypto. 50 tests.
+checks → `OK` or `FAILED`. No network, no daemon, no own crypto. 63 tests.
 
 ## Contents
 
@@ -282,15 +288,44 @@ commitments — it does **not** prove the evaluation was well designed or that t
 itself is correct. Those are human judgements; what it removes is the need to simply
 trust the number.
 
+### A verification layer for trustworthy eval logs
+
+The UK AISI inspect_ai team names an open gap ([arXiv:2507.06893](https://arxiv.org/abs/2507.06893)):
+a database of trustworthy evaluation results with proper provenance tracking. proofbundle is the
+missing **signature + selective-disclosure layer** for exactly that — complementary to metadata
+aggregation (Every Eval Ever) and documentation taxonomies (Eval Factsheets), not a competitor.
+See [INTEROP.md](INTEROP.md) for how it maps to OpenSSF Model Signing, CycloneDX ML-BOM, and in-toto.
+
+- **Two framework adapters** — `pip install "proofbundle[inspect]"` reads a UK AISI
+  [inspect_ai](https://github.com/UKGovernmentBEIS/inspect_ai) eval log via the stable `read_eval_log`
+  API (lazy import). `proofbundle.adapters.from_lm_eval_results` reads a real EleutherAI
+  [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) `results_*.json` (the
+  genuine `acc,none` filter-suffix format) and captures run provenance — no framework import either way.
+- **in-toto Statement v1** — `proofbundle.intoto.to_intoto_statement(claim, root_b64=…)`
+  emits the receipt as an in-toto statement with a self-hosted predicate type. The subject
+  digest is an *honest salted commitment* under a custom key, never `sha256` (see
+  [PREDICATE.md](PREDICATE.md)).
+- **SD-JWT issuance** (RFC 9901) — `proofbundle.sdjwt_issue.issue_sd_jwt(claim, signer,
+  root_b64=…, exact_score=…)` issues the receipt so a holder can disclose `passed` +
+  `threshold` while **withholding the exact score** and the identifier openings. The signed
+  bundle payload is the source of truth; the SD-JWT is a derived, bundle-bound view, verified
+  by proofbundle's own verifier **and** the `sd-jwt-python` reference.
+
+Every release ships **PEP 740 attestations** (Trusted Publishing) + an SLSA build-provenance
+attestation — see [SECURITY.md](SECURITY.md).
+
 ## Roadmap
 
 - **v0.1** — the offline verifier plus a real example bundle.
 - **v0.2** — the emitter: `emit_bundle` / `proofbundle emit`.
 - **v0.3** — external RFC 6962 conformance vectors + real Sigstore Rekor interop.
-- **v0.4 (current release)** — the eval-receipt emitter (`emit_eval_receipt` /
-  `proofbundle emit-eval`), salted commitments, issuer binding, file-based adapters.
-- **v0.5** — selective disclosure of the exact score via SD-JWT **issuance** (the issuer
-  reveals identifier + salt on demand) and full SD-JWT VC conformance.
+- **v0.4** — the eval-receipt emitter (`emit_eval_receipt` / `proofbundle emit-eval`),
+  salted commitments, issuer binding.
+- **v0.5** — inspect_ai adapter (stable API), in-toto Statement v1 view, SD-JWT **issuance** (RFC 9901).
+- **v0.6 (current release)** — a second eval adapter (lm-evaluation-harness, real format + provenance),
+  INTEROP.md, CITATION.cff, PEP 740 attestations documented.
+- **Deferred** (explicitly not yet built) — SD-JWT VC conformance + `vct` metadata,
+  Key-Binding JWT, status lists / revocation, an official in-toto PR, DSSE / a full in-toto client.
 
 ## Contributing
 

{proofbundle-0.4.1 → proofbundle-0.6.0}/README.md

@@ -12,17 +12,19 @@ signed and anchored in a tamper-evident log — and optionally carries a
 selectively disclosable credential. Pure Python, no server, no daemon, one JSON file.**
 
 [![CI](https://github.com/b7n0de/proofbundle/actions/workflows/ci.yml/badge.svg)](https://github.com/b7n0de/proofbundle/actions/workflows/ci.yml)
-[![PyPI](https://img.shields.io/pypi/v/proofbundle.svg?color=D6248A)](https://pypi.org/project/proofbundle/)
-[![Python](https://img.shields.io/pypi/pyversions/proofbundle.svg?color=D6248A)](https://pypi.org/project/proofbundle/)
+[![PyPI](https://img.shields.io/pypi/v/proofbundle.svg?color=D6248A&cacheSeconds=3600)](https://pypi.org/project/proofbundle/)
+[![Python](https://img.shields.io/pypi/pyversions/proofbundle.svg?color=D6248A&cacheSeconds=3600)](https://pypi.org/project/proofbundle/)
+[![Downloads](https://static.pepy.tech/badge/proofbundle)](https://pepy.tech/project/proofbundle)
 [![License: MIT](https://img.shields.io/badge/license-MIT-D6248A.svg)](LICENSE)
 [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
 [![SLSA build provenance](https://img.shields.io/badge/SLSA-build_provenance-D6248A.svg)](https://slsa.dev)
+[![PyPI attestations](https://img.shields.io/badge/PyPI-attestations_(PEP_740)-D6248A.svg)](https://pypi.org/project/proofbundle/)
 
 </div>
 
 **At a glance:** `proofbundle emit` signs and anchors a payload; `proofbundle
 verify` checks one self-contained `bundle.json` with three offline cryptographic
-checks → `OK` or `FAILED`. No network, no daemon, no own crypto. 50 tests.
+checks → `OK` or `FAILED`. No network, no daemon, no own crypto. 63 tests.
 
 ## Contents
 
@@ -243,15 +245,44 @@ commitments — it does **not** prove the evaluation was well designed or that t
 itself is correct. Those are human judgements; what it removes is the need to simply
 trust the number.
 
+### A verification layer for trustworthy eval logs
+
+The UK AISI inspect_ai team names an open gap ([arXiv:2507.06893](https://arxiv.org/abs/2507.06893)):
+a database of trustworthy evaluation results with proper provenance tracking. proofbundle is the
+missing **signature + selective-disclosure layer** for exactly that — complementary to metadata
+aggregation (Every Eval Ever) and documentation taxonomies (Eval Factsheets), not a competitor.
+See [INTEROP.md](INTEROP.md) for how it maps to OpenSSF Model Signing, CycloneDX ML-BOM, and in-toto.
+
+- **Two framework adapters** — `pip install "proofbundle[inspect]"` reads a UK AISI
+  [inspect_ai](https://github.com/UKGovernmentBEIS/inspect_ai) eval log via the stable `read_eval_log`
+  API (lazy import). `proofbundle.adapters.from_lm_eval_results` reads a real EleutherAI
+  [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) `results_*.json` (the
+  genuine `acc,none` filter-suffix format) and captures run provenance — no framework import either way.
+- **in-toto Statement v1** — `proofbundle.intoto.to_intoto_statement(claim, root_b64=…)`
+  emits the receipt as an in-toto statement with a self-hosted predicate type. The subject
+  digest is an *honest salted commitment* under a custom key, never `sha256` (see
+  [PREDICATE.md](PREDICATE.md)).
+- **SD-JWT issuance** (RFC 9901) — `proofbundle.sdjwt_issue.issue_sd_jwt(claim, signer,
+  root_b64=…, exact_score=…)` issues the receipt so a holder can disclose `passed` +
+  `threshold` while **withholding the exact score** and the identifier openings. The signed
+  bundle payload is the source of truth; the SD-JWT is a derived, bundle-bound view, verified
+  by proofbundle's own verifier **and** the `sd-jwt-python` reference.
+
+Every release ships **PEP 740 attestations** (Trusted Publishing) + an SLSA build-provenance
+attestation — see [SECURITY.md](SECURITY.md).
+
 ## Roadmap
 
 - **v0.1** — the offline verifier plus a real example bundle.
 - **v0.2** — the emitter: `emit_bundle` / `proofbundle emit`.
 - **v0.3** — external RFC 6962 conformance vectors + real Sigstore Rekor interop.
-- **v0.4 (current release)** — the eval-receipt emitter (`emit_eval_receipt` /
-  `proofbundle emit-eval`), salted commitments, issuer binding, file-based adapters.
-- **v0.5** — selective disclosure of the exact score via SD-JWT **issuance** (the issuer
-  reveals identifier + salt on demand) and full SD-JWT VC conformance.
+- **v0.4** — the eval-receipt emitter (`emit_eval_receipt` / `proofbundle emit-eval`),
+  salted commitments, issuer binding.
+- **v0.5** — inspect_ai adapter (stable API), in-toto Statement v1 view, SD-JWT **issuance** (RFC 9901).
+- **v0.6 (current release)** — a second eval adapter (lm-evaluation-harness, real format + provenance),
+  INTEROP.md, CITATION.cff, PEP 740 attestations documented.
+- **Deferred** (explicitly not yet built) — SD-JWT VC conformance + `vct` metadata,
+  Key-Binding JWT, status lists / revocation, an official in-toto PR, DSSE / a full in-toto client.
 
 ## Contributing
 

{proofbundle-0.4.1 → proofbundle-0.6.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "proofbundle"
-version = "0.4.1"
+version = "0.6.0"
 description = "Emit and verify portable cryptographic evidence bundles, offline: Ed25519 + RFC 6962 Merkle + optional SD-JWT."
 readme = "README.md"
 requires-python = ">=3.9"
@@ -43,9 +43,14 @@ sdjwt = []
 # path (verify_bundle / decode_eval_claim) never canonicalizes — it checks stored bytes — so the
 # verifier stays dependency-free. `pip install proofbundle[eval]` adds emit-side canonicalization.
 eval = ["rfc8785>=0.1.4"]
-# Framework adapters read exported result JSON only (no framework import) → pure stdlib today.
+# The lm-eval adapter reads exported results.json (no import) → pure stdlib.
 adapters = []
-dev = ["pytest>=7", "ruff>=0.5", "jsonschema>=4", "mypy>=1.8", "build>=1", "hypothesis>=6", "rfc8785>=0.1.4"]
+# The inspect_ai adapter uses the STABLE read_eval_log API (lazy import). Pinned with an UPPER bound:
+# the .eval format + pydantic schema change between versions (inspect_ai issue 834), and the fixture
+# test is bound to this range. `pip install "proofbundle[inspect]"`.
+inspect = ["inspect_ai>=0.3.100,<0.4"]
+dev = ["pytest>=7", "ruff>=0.5", "jsonschema>=4", "mypy>=1.8", "build>=1", "hypothesis>=6",
+       "rfc8785>=0.1.4", "sd-jwt>=0.10", "inspect_ai>=0.3.100,<0.4"]
 
 [project.urls]
 Homepage = "https://b7n0de.com"

{proofbundle-0.4.1 → proofbundle-0.6.0}/src/proofbundle/__init__.py

@@ -13,7 +13,7 @@ from .emit import emit_bundle, generate_signer
 from .errors import Check, ProofBundleError, VerificationResult
 from .merkle import verify_consistency, verify_inclusion
 
-__version__ = "0.4.1"
+__version__ = "0.6.0"
 
 __all__ = [
     "__version__",

proofbundle-0.6.0/src/proofbundle/adapters/inspect_ai.py

@@ -0,0 +1,65 @@
+"""Adapter for UK AISI inspect_ai eval logs — via the STABLE API, optional extra `proofbundle[inspect]`.
+
+Unlike the v0.4 file-based reader, this uses the stable `inspect_ai.log.read_eval_log(path,
+header_only=True)` API (the `.eval` on-disk format + its pydantic schema change between versions, see
+inspect_ai issue 834; the stable API is robust). inspect_ai is imported LAZILY inside the function, so
+the proofbundle core stays dependency-free — only `pip install "proofbundle[inspect]"` pulls it.
+
+Object model (inspect_ai): `log.eval.task` is the suite; `log.results.scores` is a list of EvalScore;
+`EvalScore.metrics` is a dict name→EvalMetric; `EvalMetric.value` is the number. threshold, comparator
+and thus `passed` are set by proofbundle, NOT read from the log. model_id/dataset_id become salted
+commitments (never plaintext in the payload).
+"""
+from __future__ import annotations
+
+from typing import Optional
+
+from ..evalclaim import build_eval_claim
+
+
+class InspectAdapterError(RuntimeError):
+    """Raised when inspect_ai is missing or the log lacks the expected structure (no bare AttributeError)."""
+
+
+def from_inspect_ai_log(path, metric: str, *, comparator: str, threshold: str, timestamp: str,
+                        model_salt: Optional[bytes] = None, dataset_salt: Optional[bytes] = None):
+    """Read an inspect_ai eval log via the stable API and build an eval claim for `metric`.
+
+    Returns (claim, salts). Raises InspectAdapterError if inspect_ai is unavailable or the log is
+    missing the expected attributes — a clear error instead of an opaque AttributeError.
+    """
+    try:
+        from inspect_ai.log import read_eval_log  # noqa: PLC0415 — lazy: keeps the core dependency-free
+    except ImportError as e:
+        raise InspectAdapterError(
+            "inspect_ai is required for this adapter — install with: pip install \"proofbundle[inspect]\"") from e
+
+    try:
+        log = read_eval_log(str(path), header_only=True)
+    except Exception as e:  # noqa: BLE001 — surface any read/parse failure as a clear adapter error
+        raise InspectAdapterError(f"could not read inspect_ai log {path!r}: {e}") from e
+
+    ev = getattr(log, "eval", None)
+    results = getattr(log, "results", None)
+    if ev is None or results is None:
+        raise InspectAdapterError("inspect_ai log missing .eval or .results (empty or malformed log)")
+
+    value = None
+    for score in (getattr(results, "scores", None) or []):
+        metrics = getattr(score, "metrics", None) or {}
+        if metric in metrics:
+            value = getattr(metrics[metric], "value", None)
+            break
+    if value is None:
+        raise InspectAdapterError(f"metric {metric!r} not found in any score.metrics of the log")
+
+    suite = str(getattr(ev, "task", "inspect_ai"))
+    model_id = str(getattr(ev, "model", "unknown"))
+    dataset = getattr(ev, "dataset", None)
+    dataset_id = str(getattr(dataset, "name", None) or suite)
+    return build_eval_claim(
+        suite=suite, suite_version=str(getattr(ev, "task_version", "1")),
+        metric=metric, comparator=comparator, threshold=threshold, score=repr(value),
+        n=int(getattr(results, "total_samples", 0) or 0),
+        model_id=model_id, dataset_id=dataset_id, issuer="", timestamp=timestamp,
+        model_salt=model_salt, dataset_salt=dataset_salt)

proofbundle-0.6.0/src/proofbundle/adapters/lm_eval.py

@@ -0,0 +1,76 @@
+"""Adapter for EleutherAI lm-evaluation-harness results_*.json (file-based, NO lm_eval import).
+
+Parses the exported result JSON only — no runtime dependency on lm_eval, no runner rebuild.
+
+Real 0.4.x format (validated against a genuine harness run, see tests/fixtures/lm_eval_arc_easy_real.json):
+the metric keys carry a *filter suffix*, e.g. `"acc,none"`, and the standard error is a **sibling** key
+`"acc_stderr,none"` (not nested). So a caller asking for metric `"acc"` is matched against `"acc,none"`
+(or `"acc,<filter>"`). Provenance (git_hash, harness/task version, n-shot) is copied into the receipt's
+optional `provenance` field so a verifier can trace exactly which run produced it.
+"""
+from __future__ import annotations
+
+import json
+from pathlib import Path
+from typing import Optional
+
+from ..evalclaim import build_eval_claim
+
+
+def _find_metric(res: dict, metric: str):
+    """Return (value, stderr, matched_key) for `metric`, handling the `metric,<filter>` suffix format.
+
+    Prefers an exact `metric` key, then `metric,none`, then any `metric,<filter>`. The stderr sibling is
+    `metric_stderr,<same filter>`."""
+    if metric in res:                       # bare key (older/simple exports)
+        stderr = res.get(f"{metric}_stderr")
+        return res[metric], stderr, metric
+    if f"{metric},none" in res:
+        return res[f"{metric},none"], res.get(f"{metric}_stderr,none"), f"{metric},none"
+    for key in res:                         # any filter, e.g. metric,custom-filter
+        if key == metric or (key.startswith(f"{metric},") and not key.startswith(f"{metric}_stderr")):
+            flt = key.split(",", 1)[1] if "," in key else "none"
+            return res[key], res.get(f"{metric}_stderr,{flt}"), key
+    return None, None, None
+
+
+def from_lm_eval_results(path, task: str, metric: str, *, comparator: str, threshold: str,
+                         timestamp: str, model_salt: Optional[bytes] = None,
+                         dataset_salt: Optional[bytes] = None):
+    """Read an lm-evaluation-harness results_*.json and build an eval claim for `task`/`metric`.
+
+    `metric` is the bare name (e.g. "acc"); the real key may be "acc,none". The score is read as a STRING
+    to avoid float canonicalization issues. Returns (claim, salts).
+    """
+    data = json.loads(Path(path).read_text(encoding="utf-8"))
+    res = data.get("results", {}).get(task)
+    if res is None:
+        raise ValueError(f"task not found in results: {task!r}")
+    value, stderr, matched = _find_metric(res, metric)
+    if value is None:
+        raise ValueError(f"metric {metric!r} not found in results[{task!r}] "
+                         f"(available: {sorted(k for k in res if ',' in k)})")
+    score = value if isinstance(value, str) else repr(value)
+
+    n_samples = data.get("n-samples", {}).get(task, {})
+    n = int(n_samples.get("effective") or n_samples.get("original") or res.get("sample_len") or 0)
+    cfg = data.get("config", {})
+    model_id = str(cfg.get("model_name") or cfg.get("model") or "unknown")
+    if cfg.get("model_args"):
+        model_id = f"{model_id}::{cfg['model_args']}"   # include args so the commitment pins the exact model
+
+    provenance = {"harness": "lm-evaluation-harness", "matched_metric_key": matched}
+    if data.get("git_hash"):
+        provenance["git_hash"] = str(data["git_hash"])
+    if data.get("versions", {}).get(task) is not None:
+        provenance["task_version"] = str(data["versions"][task])
+    if data.get("n-shot", {}).get(task) is not None:
+        provenance["n_shot"] = str(data["n-shot"][task])
+    if stderr is not None:
+        provenance["stderr"] = repr(stderr) if not isinstance(stderr, str) else stderr
+
+    return build_eval_claim(
+        suite=task, suite_version=str(data.get("versions", {}).get(task, "lm-eval")),
+        metric=metric, comparator=comparator, threshold=threshold, score=str(score), n=n,
+        model_id=model_id, dataset_id=task, issuer="", timestamp=timestamp,
+        provenance=provenance, model_salt=model_salt, dataset_salt=dataset_salt)

{proofbundle-0.4.1 → proofbundle-0.6.0}/src/proofbundle/evalclaim.py

@@ -37,7 +37,7 @@ _MAX_SAFE_INT = 2 ** 53 - 1
 # The exact key set of an eval claim; decode/validate reject anything else.
 _REQUIRED = {"schema", "suite", "suite_version", "metric", "comparator", "threshold",
              "passed", "n", "model_id_commit", "dataset_id_commit", "commit_alg", "issuer", "timestamp"}
-_OPTIONAL = {"context_binding", "ci95", "multiple_testing", "prereg_sha256"}
+_OPTIONAL = {"context_binding", "ci95", "multiple_testing", "prereg_sha256", "provenance"}
 
 __all__ = [
     "EVAL_CLAIM_SCHEMA", "COMMIT_ALG", "canonicalize", "build_eval_claim",
@@ -126,7 +126,7 @@ def build_eval_claim(*, suite: str, suite_version: str, metric: str, comparator:
                      threshold: str, score: str, n: int, model_id: str, dataset_id: str,
                      issuer: str, timestamp: str, context_binding: Optional[str] = None,
                      ci95: Optional[Sequence[str]] = None, multiple_testing: Optional[str] = None,
-                     prereg_sha256: Optional[str] = None,
+                     prereg_sha256: Optional[str] = None, provenance: Optional[dict] = None,
                      model_salt: Optional[bytes] = None, dataset_salt: Optional[bytes] = None):
     """Build a valid eval claim from raw values. Computes `passed` ITSELF from the comparator
     (never trusts the caller), creates salted commitments, and returns (claim, salts) with the
@@ -163,6 +163,8 @@ def build_eval_claim(*, suite: str, suite_version: str, metric: str, comparator:
         claim["multiple_testing"] = multiple_testing
     if prereg_sha256 is not None:
         claim["prereg_sha256"] = prereg_sha256
+    if provenance is not None:
+        claim["provenance"] = provenance
     _reject_non_jcs(claim)
     return claim, {"model_salt": m_salt, "dataset_salt": d_salt}
 

proofbundle-0.6.0/src/proofbundle/intoto.py

@@ -0,0 +1,63 @@
+"""in-toto Statement v1 view of an eval receipt (self-hosted predicate type).
+
+A self-hosted `predicateType` URI is fully in-toto-spec-conform and the right choice for a solo v0.x
+(no official in-toto/attestation PR needed). See PREDICATE.md.
+
+HONESTY (important): the `subject.digest` here is a SALTED COMMITMENT to the model identifier, NOT the
+content hash of an artifact. Placing it under the standard `sha256` key would suggest an artifact hash
+and mislead generic in-toto verifiers. in-toto permits arbitrary digest keys, so we use a unique custom
+key `proofbundleModelCommitV1`; the `subject.name` is the descriptive `model-id-commitment`; and the
+predicate mirrors the note in `subject_digest_note`. Full artifact digests come only once a model artifact
+exists (deferred, see the roadmap).
+"""
+from __future__ import annotations
+
+from typing import Optional
+
+STATEMENT_TYPE = "https://in-toto.io/Statement/v1"
+PREDICATE_TYPE = "https://b7n0de.com/proofbundle/eval-receipt/v0.1"
+VERIFIER_ID = "https://b7n0de.com/proofbundle"
+MODEL_COMMIT_DIGEST_KEY = "proofbundleModelCommitV1"
+
+_SUBJECT_DIGEST_NOTE = (
+    "subject.digest is a salted commitment to the model identifier (key "
+    f"{MODEL_COMMIT_DIGEST_KEY}), NOT an artifact content hash — do not treat it as sha256.")
+
+
+def _commit_hex(commit: str) -> str:
+    """Extract the hex of a `sha256:<hex>` salted commitment (the value that goes into the digest)."""
+    return commit.split(":", 1)[1] if ":" in commit else commit
+
+
+def to_intoto_statement(claim: dict, *, root_b64: Optional[str] = None,
+                        harness: Optional[dict] = None) -> dict:
+    """Build an in-toto Statement v1 whose predicate is the eval receipt.
+
+    `root_b64` (from the signed bundle's merkle root) binds the statement to the receipt. `harness`
+    (e.g. {"name": "inspect_ai", "version": "0.3.217"}) is optional. The subject digest is the model
+    commitment under a custom key (never `sha256`).
+    """
+    statement = {
+        "_type": STATEMENT_TYPE,
+        "subject": [{
+            "name": "model-id-commitment",
+            "digest": {MODEL_COMMIT_DIGEST_KEY: _commit_hex(claim["model_id_commit"])},
+        }],
+        "predicateType": PREDICATE_TYPE,
+        "predicate": {
+            "verifier": {"id": VERIFIER_ID},
+            "evaluatedAt": claim["timestamp"],
+            "suite": claim["suite"],
+            "claims": [{
+                "metric": claim["metric"], "comparator": claim["comparator"],
+                "threshold": claim["threshold"], "passed": claim["passed"],
+            }],
+            "datasetCommit": claim.get("dataset_id_commit"),
+            "subject_digest_note": _SUBJECT_DIGEST_NOTE,
+        },
+    }
+    if harness:
+        statement["predicate"]["harness"] = harness
+    if root_b64:
+        statement["predicate"]["receipt"] = {"schema": "proofbundle/v0.1", "root_b64": root_b64}
+    return statement

proofbundle-0.6.0/src/proofbundle/sdjwt_issue.py

@@ -0,0 +1,119 @@
+"""SD-JWT issuance per RFC 9901 — the differentiation feature (v0.5).
+
+Issue an eval receipt so a holder can disclose `passed` + `threshold` while WITHHOLDING the exact score
+and the identifier openings. The existing verifier (proofbundle.sdjwt) stays; this adds issuance.
+
+Source of truth: the signed canonical bundle payload (evalclaim) is the ONLY truth. This SD-JWT is a
+derived view — its always-open claims are copied bit-exact from that payload, and it binds the bundle
+anchor via `receipt.root_b64`. Sign the SD-JWT with the SAME Ed25519 key that signed the bundle (matching
+the `issuer` field). A holder cannot lift a claim under a different key.
+
+Always-open (plaintext JWT claims, NEVER a disclosure): passed, threshold, comparator, suite, issuer,
+receipt.root_b64. Selectively-disclosable (via `_sd` + disclosures): the exact metric value, ci95, and
+the identifier-commitment openings (identifier + salt).
+
+RFC 9901 §4.2.4.1 digest byte-chain (the subtle, load-bearing detail): for each disclosable field, a
+CSPRNG salt of ≥128 bit (base64url); the disclosure is base64url(UTF-8(JSON array [salt, name, value]));
+the digest placed in `_sd` is **base64url(SHA-256(ASCII bytes of the base64url-ENCODED disclosure
+string)))** — hashed over the ENCODED string, NOT over the JSON bytes. `_sd_alg` = "sha-256" at the top
+level. The JWT is signed with EdDSA. Compact form is tilde-separated: JWT~disclosure1~...~ (trailing ~).
+"""
+from __future__ import annotations
+
+import base64
+import hashlib
+import json
+import os
+from typing import Optional, Sequence
+
+from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
+from cryptography.hazmat.primitives.serialization import Encoding, PublicFormat
+
+SD_ALG = "sha-256"
+_SALT_BYTES = 16  # 128 bit
+
+
+def _b64url(data: bytes) -> str:
+    return base64.urlsafe_b64encode(data).rstrip(b"=").decode("ascii")
+
+
+def _make_disclosure(name: str, value, salt_b64: str) -> tuple[str, str]:
+    """Return (disclosure_b64url, digest_b64url) per RFC 9901 §4.2.4.1.
+
+    The digest hashes the ASCII bytes of the base64url-ENCODED disclosure string (not the JSON bytes)."""
+    disclosure_json = json.dumps([salt_b64, name, value])            # array [salt, name, value]
+    disclosure_b64 = _b64url(disclosure_json.encode("utf-8"))
+    digest = _b64url(hashlib.sha256(disclosure_b64.encode("ascii")).digest())
+    return disclosure_b64, digest
+
+
+def issue_sd_jwt(claim: dict, signer: Ed25519PrivateKey, *, root_b64: str,
+                 exact_score: Optional[str] = None, ci95: Optional[Sequence[str]] = None,
+                 model_id_opening: Optional[Sequence] = None,
+                 dataset_id_opening: Optional[Sequence] = None) -> str:
+    """Issue a compact SD-JWT for the eval claim, signed with `signer` (must match claim['issuer']).
+
+    Openings are (identifier, salt_hex) pairs the issuer may later reveal; `exact_score`/`ci95` are the
+    withheld numeric detail. All extras are selectively-disclosable; the pass/threshold facts are open.
+    """
+    always_open = {
+        "passed": claim["passed"], "threshold": claim["threshold"],
+        "comparator": claim["comparator"], "suite": claim["suite"],
+        "issuer": claim["issuer"], "receipt": {"root_b64": root_b64},
+    }
+    disclosures: list[str] = []
+    sd_digests: list[str] = []
+
+    def _add(name: str, value):
+        d, dig = _make_disclosure(name, value, _b64url(os.urandom(_SALT_BYTES)))
+        disclosures.append(d)
+        sd_digests.append(dig)
+
+    if exact_score is not None:
+        _add("exact_score", exact_score)
+    if ci95 is not None:
+        _add("ci95", list(ci95))
+    if model_id_opening is not None:
+        _add("model_id_opening", list(model_id_opening))
+    if dataset_id_opening is not None:
+        _add("dataset_id_opening", list(dataset_id_opening))
+
+    payload = dict(always_open)
+    if sd_digests:
+        payload["_sd"] = sd_digests
+        payload["_sd_alg"] = SD_ALG
+
+    header = {"alg": "EdDSA", "typ": "sd-jwt"}
+    signing_input = _b64url(json.dumps(header).encode("utf-8")) + "." + _b64url(json.dumps(payload).encode("utf-8"))
+    signature = signer.sign(signing_input.encode("ascii"))
+    jwt = signing_input + "." + _b64url(signature)
+
+    # compact: JWT ~ disclosure1 ~ ... ~ (trailing tilde, no key-binding JWT in v0.5)
+    return "~".join([jwt, *disclosures]) + "~"
+
+
+def issuer_matches(claim: dict, signer: Ed25519PrivateKey) -> bool:
+    """True iff the claim's issuer fingerprint equals the signer's public key (bundle↔SD-JWT same key)."""
+    raw = signer.public_key().public_bytes(Encoding.Raw, PublicFormat.Raw)
+    return claim.get("issuer") == "ed25519:" + base64.b64encode(raw).decode("ascii")
+
+
+def _jwt_payload(compact: str) -> dict:
+    """Decode the always-open JWT payload of a compact SD-JWT (the part before the first '~')."""
+    jwt = compact.split("~", 1)[0]
+    payload_b64 = jwt.split(".")[1]
+    padded = payload_b64 + "=" * (-len(payload_b64) % 4)
+    return json.loads(base64.urlsafe_b64decode(padded).decode("utf-8"))
+
+
+def check_binds_bundle(compact: str, claim: dict, root_b64: str) -> bool:
+    """No-Fake binding: the SD-JWT's always-open claims MUST match the signed bundle payload bit-exact and
+    bind its merkle root. A derived SD-JWT that diverges from its bundle source of truth is rejected."""
+    try:
+        p = _jwt_payload(compact)
+    except (ValueError, KeyError, IndexError):
+        return False
+    return (p.get("passed") == claim["passed"] and p.get("threshold") == claim["threshold"]
+            and p.get("comparator") == claim["comparator"] and p.get("suite") == claim["suite"]
+            and p.get("issuer") == claim["issuer"]
+            and (p.get("receipt") or {}).get("root_b64") == root_b64)

{proofbundle-0.4.1 → proofbundle-0.6.0/src/proofbundle.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: proofbundle
-Version: 0.4.1
+Version: 0.6.0
 Summary: Emit and verify portable cryptographic evidence bundles, offline: Ed25519 + RFC 6962 Merkle + optional SD-JWT.
 Author: Konrad Gruszka
 License: MIT
@@ -27,6 +27,8 @@ Provides-Extra: sdjwt
 Provides-Extra: eval
 Requires-Dist: rfc8785>=0.1.4; extra == "eval"
 Provides-Extra: adapters
+Provides-Extra: inspect
+Requires-Dist: inspect_ai<0.4,>=0.3.100; extra == "inspect"
 Provides-Extra: dev
 Requires-Dist: pytest>=7; extra == "dev"
 Requires-Dist: ruff>=0.5; extra == "dev"
@@ -35,6 +37,8 @@ Requires-Dist: mypy>=1.8; extra == "dev"
 Requires-Dist: build>=1; extra == "dev"
 Requires-Dist: hypothesis>=6; extra == "dev"
 Requires-Dist: rfc8785>=0.1.4; extra == "dev"
+Requires-Dist: sd-jwt>=0.10; extra == "dev"
+Requires-Dist: inspect_ai<0.4,>=0.3.100; extra == "dev"
 Dynamic: license-file
 
 <div align="center">
@@ -51,17 +55,19 @@ signed and anchored in a tamper-evident log — and optionally carries a
 selectively disclosable credential. Pure Python, no server, no daemon, one JSON file.**
 
 [![CI](https://github.com/b7n0de/proofbundle/actions/workflows/ci.yml/badge.svg)](https://github.com/b7n0de/proofbundle/actions/workflows/ci.yml)
-[![PyPI](https://img.shields.io/pypi/v/proofbundle.svg?color=D6248A)](https://pypi.org/project/proofbundle/)
-[![Python](https://img.shields.io/pypi/pyversions/proofbundle.svg?color=D6248A)](https://pypi.org/project/proofbundle/)
+[![PyPI](https://img.shields.io/pypi/v/proofbundle.svg?color=D6248A&cacheSeconds=3600)](https://pypi.org/project/proofbundle/)
+[![Python](https://img.shields.io/pypi/pyversions/proofbundle.svg?color=D6248A&cacheSeconds=3600)](https://pypi.org/project/proofbundle/)
+[![Downloads](https://static.pepy.tech/badge/proofbundle)](https://pepy.tech/project/proofbundle)
 [![License: MIT](https://img.shields.io/badge/license-MIT-D6248A.svg)](LICENSE)
 [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
 [![SLSA build provenance](https://img.shields.io/badge/SLSA-build_provenance-D6248A.svg)](https://slsa.dev)
+[![PyPI attestations](https://img.shields.io/badge/PyPI-attestations_(PEP_740)-D6248A.svg)](https://pypi.org/project/proofbundle/)
 
 </div>
 
 **At a glance:** `proofbundle emit` signs and anchors a payload; `proofbundle
 verify` checks one self-contained `bundle.json` with three offline cryptographic
-checks → `OK` or `FAILED`. No network, no daemon, no own crypto. 50 tests.
+checks → `OK` or `FAILED`. No network, no daemon, no own crypto. 63 tests.
 
 ## Contents
 
@@ -282,15 +288,44 @@ commitments — it does **not** prove the evaluation was well designed or that t
 itself is correct. Those are human judgements; what it removes is the need to simply
 trust the number.
 
+### A verification layer for trustworthy eval logs
+
+The UK AISI inspect_ai team names an open gap ([arXiv:2507.06893](https://arxiv.org/abs/2507.06893)):
+a database of trustworthy evaluation results with proper provenance tracking. proofbundle is the
+missing **signature + selective-disclosure layer** for exactly that — complementary to metadata
+aggregation (Every Eval Ever) and documentation taxonomies (Eval Factsheets), not a competitor.
+See [INTEROP.md](INTEROP.md) for how it maps to OpenSSF Model Signing, CycloneDX ML-BOM, and in-toto.
+
+- **Two framework adapters** — `pip install "proofbundle[inspect]"` reads a UK AISI
+  [inspect_ai](https://github.com/UKGovernmentBEIS/inspect_ai) eval log via the stable `read_eval_log`
+  API (lazy import). `proofbundle.adapters.from_lm_eval_results` reads a real EleutherAI
+  [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) `results_*.json` (the
+  genuine `acc,none` filter-suffix format) and captures run provenance — no framework import either way.
+- **in-toto Statement v1** — `proofbundle.intoto.to_intoto_statement(claim, root_b64=…)`
+  emits the receipt as an in-toto statement with a self-hosted predicate type. The subject
+  digest is an *honest salted commitment* under a custom key, never `sha256` (see
+  [PREDICATE.md](PREDICATE.md)).
+- **SD-JWT issuance** (RFC 9901) — `proofbundle.sdjwt_issue.issue_sd_jwt(claim, signer,
+  root_b64=…, exact_score=…)` issues the receipt so a holder can disclose `passed` +
+  `threshold` while **withholding the exact score** and the identifier openings. The signed
+  bundle payload is the source of truth; the SD-JWT is a derived, bundle-bound view, verified
+  by proofbundle's own verifier **and** the `sd-jwt-python` reference.
+
+Every release ships **PEP 740 attestations** (Trusted Publishing) + an SLSA build-provenance
+attestation — see [SECURITY.md](SECURITY.md).
+
 ## Roadmap
 
 - **v0.1** — the offline verifier plus a real example bundle.
 - **v0.2** — the emitter: `emit_bundle` / `proofbundle emit`.
 - **v0.3** — external RFC 6962 conformance vectors + real Sigstore Rekor interop.
-- **v0.4 (current release)** — the eval-receipt emitter (`emit_eval_receipt` /
-  `proofbundle emit-eval`), salted commitments, issuer binding, file-based adapters.
-- **v0.5** — selective disclosure of the exact score via SD-JWT **issuance** (the issuer
-  reveals identifier + salt on demand) and full SD-JWT VC conformance.
+- **v0.4** — the eval-receipt emitter (`emit_eval_receipt` / `proofbundle emit-eval`),
+  salted commitments, issuer binding.
+- **v0.5** — inspect_ai adapter (stable API), in-toto Statement v1 view, SD-JWT **issuance** (RFC 9901).
+- **v0.6 (current release)** — a second eval adapter (lm-evaluation-harness, real format + provenance),
+  INTEROP.md, CITATION.cff, PEP 740 attestations documented.
+- **Deferred** (explicitly not yet built) — SD-JWT VC conformance + `vct` metadata,
+  Key-Binding JWT, status lists / revocation, an official in-toto PR, DSSE / a full in-toto client.
 
 ## Contributing
 

{proofbundle-0.4.1 → proofbundle-0.6.0}/src/proofbundle.egg-info/SOURCES.txt

@@ -7,9 +7,11 @@ src/proofbundle/cli.py
 src/proofbundle/emit.py
 src/proofbundle/errors.py
 src/proofbundle/evalclaim.py
+src/proofbundle/intoto.py
 src/proofbundle/merkle.py
 src/proofbundle/py.typed
 src/proofbundle/sdjwt.py
+src/proofbundle/sdjwt_issue.py
 src/proofbundle/signature.py
 src/proofbundle.egg-info/PKG-INFO
 src/proofbundle.egg-info/SOURCES.txt
@@ -27,10 +29,12 @@ tests/test_cli_eval.py
 tests/test_emit.py
 tests/test_eval_claim_schema.py
 tests/test_evalclaim.py
+tests/test_intoto.py
 tests/test_merkle.py
 tests/test_merkle_property.py
 tests/test_rekor_interop.py
 tests/test_rfc6962_external_vectors.py
 tests/test_schema.py
+tests/test_sdjwt_issue.py
 tests/test_sdjwt_reference.py
 tests/test_signature.py

{proofbundle-0.4.1 → proofbundle-0.6.0}/src/proofbundle.egg-info/requires.txt

@@ -10,8 +10,13 @@ mypy>=1.8
 build>=1
 hypothesis>=6
 rfc8785>=0.1.4
+sd-jwt>=0.10
+inspect_ai<0.4,>=0.3.100
 
 [eval]
 rfc8785>=0.1.4
 
+[inspect]
+inspect_ai<0.4,>=0.3.100
+
 [sdjwt]

proofbundle-0.6.0/tests/test_adapters.py

@@ -0,0 +1,56 @@
+"""Adapters map real exported eval JSON to a valid claim (file-based, no framework import)."""
+import unittest
+from pathlib import Path
+
+from proofbundle.adapters import from_inspect_ai_log, from_lm_eval_results
+
+FX = Path(__file__).resolve().parent / "fixtures"
+TS = "2026-07-01T12:00:00Z"
+
+
+class TestAdapters(unittest.TestCase):
+    def test_lm_eval_real_acc_none_format(self):
+        # REAL lm-evaluation-harness 0.4.12 export: metric key is "acc,none", stderr sibling "acc_stderr,none".
+        claim, salts = from_lm_eval_results(FX / "lm_eval_arc_easy_real.json", "arc_easy", "acc",
+                                            comparator=">=", threshold="0.30", timestamp=TS,
+                                            model_salt=b"0" * 16, dataset_salt=b"1" * 16)
+        self.assertEqual(claim["suite"], "arc_easy")
+        self.assertTrue(claim["passed"])                       # acc 0.5 >= 0.30
+        self.assertEqual(claim["provenance"]["matched_metric_key"], "acc,none")  # suffix handled
+        self.assertIn("git_hash", claim["provenance"])         # provenance captured
+        self.assertEqual(claim["provenance"]["n_shot"], "0")
+        self.assertIn("stderr", claim["provenance"])           # sibling stderr, not nested
+
+    def test_lm_eval_missing_metric_lists_available(self):
+        with self.assertRaises(ValueError):
+            from_lm_eval_results(FX / "lm_eval_arc_easy_real.json", "arc_easy", "nonexistent",
+                                 comparator=">=", threshold="0.5", timestamp=TS,
+                                 model_salt=b"0" * 16, dataset_salt=b"1" * 16)
+
+    def test_inspect_ai_stable_api(self):
+        # Real .eval log fixture, read via the stable inspect_ai.log.read_eval_log API (proofbundle[inspect]).
+        try:
+            import inspect_ai.log  # noqa: F401
+        except ImportError:
+            self.skipTest("inspect_ai not installed (pip install proofbundle[inspect])")
+        claim, salts = from_inspect_ai_log(FX / "inspect_logs" / "safety_refusal_demo.eval", "accuracy",
+                                           comparator=">=", threshold="0.00", timestamp=TS,
+                                           model_salt=b"0" * 16, dataset_salt=b"1" * 16)
+        self.assertEqual(claim["suite"], "safety_refusal_demo")
+        self.assertTrue(claim["passed"])                    # accuracy 0.0 >= 0.00
+        self.assertNotIn("mockllm/model", str(claim))       # model id only as salted commitment
+
+    def test_inspect_ai_missing_metric_clear_error(self):
+        from proofbundle.adapters.inspect_ai import InspectAdapterError
+        try:
+            import inspect_ai.log  # noqa: F401
+        except ImportError:
+            self.skipTest("inspect_ai not installed")
+        with self.assertRaises(InspectAdapterError):
+            from_inspect_ai_log(FX / "inspect_logs" / "safety_refusal_demo.eval", "nonexistent_metric",
+                                comparator=">=", threshold="0.5", timestamp=TS,
+                                model_salt=b"0" * 16, dataset_salt=b"1" * 16)
+
+
+if __name__ == "__main__":
+    unittest.main()

proofbundle-0.6.0/tests/test_intoto.py

@@ -0,0 +1,63 @@
+"""in-toto Statement v1 view of an eval receipt — structurally valid + honest salted-commitment digest."""
+import json
+import unittest
+from pathlib import Path
+
+try:
+    import jsonschema
+except ImportError:
+    jsonschema = None
+
+from proofbundle.emit import generate_signer
+from proofbundle.evalclaim import build_eval_claim, issuer_fingerprint
+from proofbundle.intoto import MODEL_COMMIT_DIGEST_KEY, PREDICATE_TYPE, to_intoto_statement
+
+ROOT = Path(__file__).resolve().parents[1]
+TS = "2026-07-01T12:00:00Z"
+
+
+def _claim():
+    signer = generate_signer()
+    claim, _ = build_eval_claim(
+        suite="safety-refusal", suite_version="v1", metric="accuracy", comparator=">=",
+        threshold="0.65", score="0.92", n=500, model_id="acme/model-x", dataset_id="acme/set",
+        issuer=issuer_fingerprint(signer), timestamp=TS, model_salt=b"0" * 16, dataset_salt=b"1" * 16)
+    return claim
+
+
+class TestInToto(unittest.TestCase):
+    def test_structure(self):
+        stmt = to_intoto_statement(_claim(), root_b64="cm9vdA==",
+                                   harness={"name": "inspect_ai", "version": "0.3.217"})
+        self.assertEqual(stmt["_type"], "https://in-toto.io/Statement/v1")
+        self.assertEqual(stmt["predicateType"], PREDICATE_TYPE)
+        self.assertEqual(len(stmt["subject"]), 1)
+        self.assertIn("digest", stmt["subject"][0])
+        # honest custom digest key, NOT sha256 (would mislead generic verifiers about an artifact hash)
+        self.assertIn(MODEL_COMMIT_DIGEST_KEY, stmt["subject"][0]["digest"])
+        self.assertNotIn("sha256", stmt["subject"][0]["digest"])
+        self.assertIn("salted commitment", stmt["predicate"]["subject_digest_note"])
+        self.assertEqual(stmt["predicate"]["receipt"]["root_b64"], "cm9vdA==")
+
+    def test_digest_is_commit_hex(self):
+        claim = _claim()
+        stmt = to_intoto_statement(claim)
+        expected_hex = claim["model_id_commit"].split(":", 1)[1]
+        self.assertEqual(stmt["subject"][0]["digest"][MODEL_COMMIT_DIGEST_KEY], expected_hex)
+
+    @unittest.skipIf(jsonschema is None, "jsonschema not installed (pip install proofbundle[dev])")
+    def test_validates_against_official_intoto_v1_schema(self):
+        schema = json.loads((ROOT / "schemas" / "in_toto_statement_v1.schema.json").read_text(encoding="utf-8"))
+        stmt = to_intoto_statement(_claim(), root_b64="cm9vdA==")
+        jsonschema.validate(instance=stmt, schema=schema)  # raises if invalid
+
+    @unittest.skipIf(jsonschema is None, "jsonschema not installed")
+    def test_schema_rejects_missing_subject(self):
+        schema = json.loads((ROOT / "schemas" / "in_toto_statement_v1.schema.json").read_text(encoding="utf-8"))
+        bad = {"_type": "https://in-toto.io/Statement/v1", "predicateType": "x", "subject": []}
+        with self.assertRaises(jsonschema.ValidationError):
+            jsonschema.validate(instance=bad, schema=schema)
+
+
+if __name__ == "__main__":
+    unittest.main()

proofbundle-0.6.0/tests/test_sdjwt_issue.py

@@ -0,0 +1,98 @@
+"""SD-JWT issuance (v0.5, RFC 9901) — own verifier + reference interop + red-tests. No-Fake."""
+import base64
+import json
+import unittest
+from pathlib import Path
+
+from cryptography.hazmat.primitives.serialization import Encoding, PublicFormat
+
+from proofbundle.emit import generate_signer
+from proofbundle.evalclaim import build_eval_claim, issuer_fingerprint
+from proofbundle.sdjwt import verify_sd_jwt
+from proofbundle.sdjwt_issue import (
+    _make_disclosure,
+    check_binds_bundle,
+    issue_sd_jwt,
+)
+
+FX = Path(__file__).resolve().parent / "fixtures"
+TS = "2026-07-01T12:00:00Z"
+ROOT_B64 = "cm9vdA=="
+
+
+def _claim(signer):
+    claim, _ = build_eval_claim(suite="safety", suite_version="v1", metric="accuracy", comparator=">=",
+        threshold="0.65", score="0.92", n=500, model_id="acme/model-x", dataset_id="acme/set",
+        issuer=issuer_fingerprint(signer), timestamp=TS, model_salt=b"0" * 16, dataset_salt=b"1" * 16)
+    return claim
+
+
+def _raw_pub(signer):
+    return signer.public_key().public_bytes(Encoding.Raw, PublicFormat.Raw)
+
+
+class TestSdJwtIssue(unittest.TestCase):
+    def test_own_verifier_accepts(self):
+        signer = generate_signer()
+        compact = issue_sd_jwt(_claim(signer), signer, root_b64=ROOT_B64, exact_score="0.92", ci95=["0.90", "0.94"])
+        res = verify_sd_jwt(compact, _raw_pub(signer))
+        self.assertTrue(res["structure_ok"], res)
+        self.assertTrue(res["sig_ok"], res)
+
+    def test_reference_verifier_accepts(self):
+        try:
+            from jwcrypto.jwk import JWK
+            from sd_jwt.verifier import SDJWTVerifier
+        except ImportError:
+            self.skipTest("sd-jwt-python not installed (dev extra)")
+        signer = generate_signer()
+        compact = issue_sd_jwt(_claim(signer), signer, root_b64=ROOT_B64, exact_score="0.92")
+        jwk = JWK(kty="OKP", crv="Ed25519", x=base64.urlsafe_b64encode(_raw_pub(signer)).rstrip(b"=").decode())
+        payload = SDJWTVerifier(compact, lambda *_a, **_k: jwk).get_verified_payload()
+        self.assertEqual(payload["passed"], True)          # always-open
+        self.assertEqual(payload["exact_score"], "0.92")   # selectively disclosed
+
+    def test_digest_byte_chain_vector(self):
+        # RFC 9901 §4.2.4.1: digest over the base64url-ENCODED disclosure string, not the JSON bytes.
+        v = json.loads((FX / "sdjwt_disclosure_vector.json").read_text(encoding="utf-8"))
+        d_b64, dig = _make_disclosure(v["name"], v["value"], v["salt_b64url"])
+        self.assertEqual(d_b64, v["disclosure_b64url"])
+        self.assertEqual(dig, v["expected_digest_b64url"])
+
+    def test_always_open_vs_selective(self):
+        signer = generate_signer()
+        compact = issue_sd_jwt(_claim(signer), signer, root_b64=ROOT_B64, exact_score="0.92")
+        jwt_payload = json.loads(base64.urlsafe_b64decode(
+            compact.split("~")[0].split(".")[1] + "==").decode("utf-8"))
+        # passed/threshold are plaintext; exact_score is NOT (only its digest is in _sd)
+        self.assertEqual(jwt_payload["passed"], True)
+        self.assertIn("threshold", jwt_payload)
+        self.assertNotIn("exact_score", jwt_payload)
+        self.assertIn("_sd", jwt_payload)
+
+    def test_binds_bundle(self):
+        signer = generate_signer()
+        claim = _claim(signer)
+        compact = issue_sd_jwt(claim, signer, root_b64=ROOT_B64, exact_score="0.92")
+        self.assertTrue(check_binds_bundle(compact, claim, ROOT_B64))
+
+    def test_divergence_red(self):  # SD-JWT claims diverge from bundle → rejected
+        signer = generate_signer()
+        claim = _claim(signer)
+        compact = issue_sd_jwt(claim, signer, root_b64=ROOT_B64, exact_score="0.92")
+        diverged = dict(claim, passed=False)            # bundle says passed=False, SD-JWT says True
+        self.assertFalse(check_binds_bundle(compact, diverged, ROOT_B64))
+        self.assertFalse(check_binds_bundle(compact, claim, "d3Jvbmc="))   # wrong root
+
+    def test_tamper_disclosure_red(self):  # tampered disclosure → digest mismatch → own verifier fails
+        signer = generate_signer()
+        compact = issue_sd_jwt(_claim(signer), signer, root_b64=ROOT_B64, exact_score="0.92")
+        jwt, *disc = compact.rstrip("~").split("~")
+        tampered_d, _ = _make_disclosure("exact_score", "0.99", "AAAAAAAAAAAAAAAAAAAAAA")  # not committed in _sd
+        tampered = "~".join([jwt, tampered_d]) + "~"
+        res = verify_sd_jwt(tampered, _raw_pub(signer))
+        self.assertFalse(res.get("structure_ok") and res.get("sig_ok") and "1 disclosure" in res.get("detail", ""))
+
+
+if __name__ == "__main__":
+    unittest.main()

proofbundle-0.4.1/src/proofbundle/adapters/inspect_ai.py

@@ -1,36 +0,0 @@
-"""Adapter for UK AISI inspect_ai eval-log JSON (file-based, no framework import)."""
-from __future__ import annotations
-
-import json
-from pathlib import Path
-from typing import Optional
-
-from ..evalclaim import build_eval_claim
-
-
-def from_inspect_ai_log(path, metric: str, *, comparator: str, threshold: str, timestamp: str,
-                        model_salt: Optional[bytes] = None, dataset_salt: Optional[bytes] = None):
-    """Read an inspect_ai eval-log JSON and build an eval claim.
-
-    Expects: {"eval": {"task": ..., "model": ..., "dataset": {"name": ...}},
-    "results": {"total_samples": n, "scores": [{"metrics": {metric: {"value": <number>}}}]}}.
-    Returns (claim, salts).
-    """
-    data = json.loads(Path(path).read_text(encoding="utf-8"))
-    ev = data.get("eval", {})
-    scores = data.get("results", {}).get("scores", [])
-    value = None
-    for s in scores:
-        m = s.get("metrics", {})
-        if metric in m:
-            value = m[metric].get("value")
-            break
-    if value is None:
-        raise ValueError(f"metric {metric!r} not found in inspect_ai scores")
-    n = int(data.get("results", {}).get("total_samples") or 0)
-    return build_eval_claim(
-        suite=str(ev.get("task", "inspect_ai")), suite_version=str(ev.get("task_version", "1")),
-        metric=metric, comparator=comparator, threshold=threshold, score=repr(value), n=n,
-        model_id=str(ev.get("model", "unknown")),
-        dataset_id=str(ev.get("dataset", {}).get("name", ev.get("task", "unknown"))),
-        issuer="", timestamp=timestamp, model_salt=model_salt, dataset_salt=dataset_salt)

proofbundle-0.4.1/src/proofbundle/adapters/lm_eval.py

@@ -1,32 +0,0 @@
-"""Adapter for EleutherAI lm-evaluation-harness results.json (file-based, no framework import)."""
-from __future__ import annotations
-
-import json
-from pathlib import Path
-from typing import Optional
-
-from ..evalclaim import build_eval_claim
-
-
-def from_lm_eval_results(path, task: str, metric: str, *, comparator: str, threshold: str,
-                         timestamp: str, model_salt: Optional[bytes] = None,
-                         dataset_salt: Optional[bytes] = None):
-    """Read an lm-evaluation-harness results.json and build an eval claim for `task`/`metric`.
-
-    Expects the standard shape: {"results": {task: {metric: <number>, ...}, ...},
-    "n-samples": {task: {"effective": n}}, "config"/"model_name": ...}. The score is read as a
-    STRING to avoid float canonicalization issues. Returns (claim, salts).
-    """
-    data = json.loads(Path(path).read_text(encoding="utf-8"))
-    res = data.get("results", {}).get(task)
-    if res is None or metric not in res:
-        raise ValueError(f"task/metric not found in results: {task}/{metric}")
-    score = repr(res[metric]) if not isinstance(res[metric], str) else res[metric]
-    n = int(data.get("n-samples", {}).get(task, {}).get("effective")
-            or data.get("n-samples", {}).get(task, {}).get("original") or 0)
-    model_id = str(data.get("model_name") or data.get("config", {}).get("model") or "unknown")
-    return build_eval_claim(
-        suite=task, suite_version=str(data.get("config", {}).get("model_source", "lm-eval")),
-        metric=metric, comparator=comparator, threshold=threshold, score=str(score), n=n,
-        model_id=model_id, dataset_id=task, issuer="", timestamp=timestamp,
-        model_salt=model_salt, dataset_salt=dataset_salt)

proofbundle-0.4.1/tests/test_adapters.py

@@ -1,32 +0,0 @@
-"""Adapters map real exported eval JSON to a valid claim (file-based, no framework import)."""
-import unittest
-from pathlib import Path
-
-from proofbundle.adapters import from_inspect_ai_log, from_lm_eval_results
-
-FX = Path(__file__).resolve().parent / "fixtures"
-TS = "2026-07-01T12:00:00Z"
-
-
-class TestAdapters(unittest.TestCase):
-    def test_lm_eval(self):
-        claim, salts = from_lm_eval_results(FX / "lm_eval_results.json", "hellaswag", "acc",
-                                            comparator=">=", threshold="0.70", timestamp=TS,
-                                            model_salt=b"0" * 16, dataset_salt=b"1" * 16)
-        self.assertEqual(claim["suite"], "hellaswag")
-        self.assertEqual(claim["threshold"], "0.70")
-        self.assertTrue(claim["passed"])              # 0.7534 >= 0.70
-        self.assertNotIn("acme/model-x", str(claim))  # id only as salted commitment
-        self.assertEqual(claim["n"], 10042)
-
-    def test_inspect_ai(self):
-        claim, salts = from_inspect_ai_log(FX / "inspect_ai_log.json", "accuracy",
-                                           comparator=">=", threshold="0.80", timestamp=TS,
-                                           model_salt=b"0" * 16, dataset_salt=b"1" * 16)
-        self.assertEqual(claim["suite"], "safety_refusal")
-        self.assertTrue(claim["passed"])              # 0.92 >= 0.80
-        self.assertEqual(claim["n"], 500)
-
-
-if __name__ == "__main__":
-    unittest.main()