proofbundle 1.7.0__tar.gz → 1.9.0__tar.gz

proofbundle-1.9.0/PKG-INFO

@@ -0,0 +1,190 @@
+Metadata-Version: 2.4
+Name: proofbundle
+Version: 1.9.0
+Summary: Emit and verify portable cryptographic evidence bundles, offline: Ed25519 + RFC 6962 Merkle + optional SD-JWT.
+Author: Konrad Gruszka
+License: MIT
+Project-URL: Homepage, https://b7n0de.com
+Project-URL: Repository, https://github.com/b7n0de/proofbundle
+Project-URL: Issues, https://github.com/b7n0de/proofbundle/issues
+Project-URL: Changelog, https://github.com/b7n0de/proofbundle/blob/main/CHANGELOG.md
+Project-URL: Documentation, https://github.com/b7n0de/proofbundle#readme
+Keywords: cryptography,merkle,transparency-log,ed25519,sd-jwt,verifiable-credentials,attestation,provenance,rfc6962
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Security :: Cryptography
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: cryptography>=42
+Provides-Extra: sdjwt
+Provides-Extra: eval
+Requires-Dist: rfc8785>=0.1.4; extra == "eval"
+Provides-Extra: adapters
+Provides-Extra: pq
+Requires-Dist: cryptography>=48; extra == "pq"
+Provides-Extra: pytest
+Requires-Dist: pytest>=7; extra == "pytest"
+Provides-Extra: inspect
+Requires-Dist: inspect_ai<0.4,>=0.3.112; extra == "inspect"
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Requires-Dist: ruff>=0.5; extra == "dev"
+Requires-Dist: jsonschema>=4; extra == "dev"
+Requires-Dist: mypy>=1.8; extra == "dev"
+Requires-Dist: build>=1; extra == "dev"
+Requires-Dist: hypothesis>=6; extra == "dev"
+Requires-Dist: rfc8785>=0.1.4; extra == "dev"
+Requires-Dist: sd-jwt>=0.10; extra == "dev"
+Requires-Dist: inspect_ai<0.4,>=0.3.112; extra == "dev"
+Dynamic: license-file
+
+<div align="center">
+
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="assets/b7n0de-logo-dark.svg">
+  <img alt="b7n0de, Verified AI Work" src="assets/b7n0de-logo.svg" height="60">
+</picture>
+
+<h1>proofbundle</h1>
+
+**Turn an AI eval result into one portable, offline-verifiable receipt.**
+It proves *who signed these exact bytes* and *that nothing changed since* — not that the number is
+true. Ed25519 + RFC 6962 Merkle, one file, no server, no network.
+
+[![CI](https://github.com/b7n0de/proofbundle/actions/workflows/ci.yml/badge.svg)](https://github.com/b7n0de/proofbundle/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/license-MIT-D6248A.svg)](LICENSE)
+[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+[![Mutation tested](https://img.shields.io/badge/tests-mutation_gated-D6248A.svg)](scripts/mutation_check.py)
+<!-- PyPI / Downloads / SLSA / PEP 740 badges are enabled on the first PyPI release — see RELEASE.md. -->
+
+</div>
+
+## The problem
+
+Every AI eval number you read — a safety benchmark, a capability score, a leaderboard entry — is an
+**unverifiable claim**. You trust the lab. There's no portable way to check, offline, that a result
+was signed by a stated party, hasn't been altered, and covers the samples it claims.
+
+proofbundle is that check. It's a small MIT-licensed Python tool (a compact, auditable trusted core,
+depends only on [`cryptography`](https://cryptography.io)) that turns a result into a signed
+receipt anyone can verify from a single file — and it's honest about the line it does not cross.
+
+## 60-second try (offline, no setup)
+
+```bash
+pip install "proofbundle[eval]"
+proofbundle demo
+```
+
+You'll see an honest receipt verify `=> OK`, then six independent tampers each verify `FAILED`, then
+a swapped sample get caught — all in memory. The command exits non-zero if any tamper slips through,
+so it's also a self-test. Full walkthrough: **[docs/DEMO.md](docs/DEMO.md)**.
+
+```bash
+# your own receipt, from a signed payload:
+proofbundle emit --payload-file result.json --new-key signer.key --out receipt.json
+proofbundle verify receipt.json        # exit 0 = OK, 1 = failed, 2 = malformed
+```
+
+## What a receipt proves — and what it doesn't
+
+| ✅ It proves | ❌ It does **not** prove |
+|---|---|
+| These exact bytes were signed by this key (**authorship**) | That the number is **true** |
+| Nothing changed since signing (**integrity**, Ed25519 + RFC 6962) | That the **issuer is honest** |
+| The result is attributable to a stated issuer | That the **eval was well-designed** |
+| A threshold was met while hiding the model/dataset (salted commitments) | That there was **no cherry-picking** — unless pre-registered |
+| Optionally: individual samples, offline-auditable (per-sample Merkle) | That the **computation was correct** — that needs a TEE or independent reproduction |
+
+This boundary is the point, not a weakness. A receipt makes a claim **attributable, tamper-evident,
+and — with pre-registration and per-sample auditing — bounded and spot-checkable**. Full detail:
+**[THREAT_MODEL.md](THREAT_MODEL.md)**.
+
+## How it fits together
+
+```mermaid
+flowchart LR
+    H["eval harness<br/>inspect_ai · lm-eval · promptfoo · pytest"] --> A["adapter → signed claim<br/>salted commitments · provenance · samples root"]
+    A --> R["receipt<br/>one portable file"]
+    R --> V{{"proofbundle verify — offline"}}
+    V --> C["signature · Merkle inclusion · SD-JWT/KB ·<br/>witness quorum · status list · sample openings"]
+    C --> OK(["=> OK / FAILED"])
+    style V fill:#D6248A,stroke:#D6248A,color:#fff
+    style OK fill:#D6248A,stroke:#D6248A,color:#fff
+```
+
+## What's in the box
+
+- **Core** — Ed25519 signature + RFC 6962 / 9162 Merkle inclusion, verified fully offline. Checks a
+  real [Sigstore Rekor](https://docs.sigstore.dev/) proof, so correctness isn't self-referential.
+- **Eval receipts** — a signed claim (`metric ⋈ threshold`, `n`, salted model/dataset commitments,
+  assurance level, provenance) from your run. See [EVAL_CLAIM.md](EVAL_CLAIM.md).
+- **Selective disclosure** — SD-JWT ([RFC 9901](https://datatracker.ietf.org/doc/rfc9901/)) with Key
+  Binding: prove a threshold while withholding the exact score.
+- **Transparency-log interop** — C2SP `tlog-checkpoint` / cosignature / `.tlog-proof`, with
+  post-quantum **ML-DSA-44** witness cosignatures. Optional Token-Status-List revocation snapshots.
+- **Per-sample audit** — commit to every sample; an auditor challenges random indices (with a fresh
+  nonce or a **public randomness beacon**, v1.9) and openings must bind to the signed root. Catches
+  1% sample-doctoring with 95% confidence at 300 samples, regardless of run size.
+- **Pre-registration** — `proofbundle prereg <plan>` commits to the protocol before the run, so
+  best-of-many publishing becomes visible.
+- **Integrations** — opt-in inspect_ai end-of-task hook and pytest plugin (emit only when
+  `PROOFBUNDLE_EMIT=1` / `--proofbundle`), plus a Hugging Face Community Evals bridge. See
+  [INTEGRATIONS.md](INTEGRATIONS.md).
+
+## Docs
+
+| For… | Read |
+|---|---|
+| Skeptics (why not SHA-256 / Sigstore / trust the issuer) | [docs/FAQ.md](docs/FAQ.md) |
+| Reviewers (30-minute adversarial audit path) | [docs/REVIEWERS.md](docs/REVIEWERS.md) |
+| Where every trust anchor comes from | [docs/TRUST_ANCHORS.md](docs/TRUST_ANCHORS.md) |
+| The demos, tier by tier | [docs/DEMO.md](docs/DEMO.md) |
+| The normative format + verification order | [SPEC.md](SPEC.md) |
+| Honest comparison to Rekor / in-toto / OMS / ValiChord | [INTEROP.md](INTEROP.md) |
+| Regulatory mapping (and what to never claim) | [COMPLIANCE.md](COMPLIANCE.md) |
+| Funders / role fit | [docs/PROJECT_BRIEF.md](docs/PROJECT_BRIEF.md) |
+
+## Install
+
+```bash
+pip install proofbundle                 # core: offline verify + plain emit (dependency-free)
+pip install "proofbundle[eval]"          # + eval receipts, prereg, and the demo (adds rfc8785 JCS)
+pip install "proofbundle[eval]"         # emit eval receipts (adds an RFC 8785 canonicalizer)
+pip install "proofbundle[inspect]"      # inspect_ai adapter + hook
+pip install "proofbundle[pq]"           # verify ML-DSA-44 (post-quantum) witness cosignatures
+```
+
+Requires Python 3.10+. The verify path never rolls its own crypto — Ed25519 comes from
+`cryptography`; Merkle hashing is RFC 6962.
+
+## Status & scope
+
+Beta, SemVer-committed, 299 tests + a CI mutation gate + property-based parser fuzzing. Correctness
+is anchored to external RFC 6962 vectors and a real Rekor proof, not just its own bundles. It is
+**not** a log service, a full in-toto client, a TEE, a consensus network, or a compliance product
+by itself — it is the small, offline, standards-native receipt layer between them. Security policy:
+[SECURITY.md](SECURITY.md).
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md) and the [Code of Conduct](CODE_OF_CONDUCT.md). Good first
+issues are labeled [`good-first-issue`](https://github.com/b7n0de/proofbundle/labels/good-first-issue);
+security findings go through [SECURITY.md](SECURITY.md). The verifier core aims to stay small,
+dependency-light, and correct.
+
+## License
+
+MIT — see [LICENSE](LICENSE).
+
+---
+
+<p align="center"><sub>proofbundle is part of <b>b7n0de</b>, Verified AI Work · <a href="https://b7n0de.com">b7n0de.com</a></sub></p>

proofbundle-1.9.0/README.md

@@ -0,0 +1,142 @@
+<div align="center">
+
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="assets/b7n0de-logo-dark.svg">
+  <img alt="b7n0de, Verified AI Work" src="assets/b7n0de-logo.svg" height="60">
+</picture>
+
+<h1>proofbundle</h1>
+
+**Turn an AI eval result into one portable, offline-verifiable receipt.**
+It proves *who signed these exact bytes* and *that nothing changed since* — not that the number is
+true. Ed25519 + RFC 6962 Merkle, one file, no server, no network.
+
+[![CI](https://github.com/b7n0de/proofbundle/actions/workflows/ci.yml/badge.svg)](https://github.com/b7n0de/proofbundle/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/license-MIT-D6248A.svg)](LICENSE)
+[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+[![Mutation tested](https://img.shields.io/badge/tests-mutation_gated-D6248A.svg)](scripts/mutation_check.py)
+<!-- PyPI / Downloads / SLSA / PEP 740 badges are enabled on the first PyPI release — see RELEASE.md. -->
+
+</div>
+
+## The problem
+
+Every AI eval number you read — a safety benchmark, a capability score, a leaderboard entry — is an
+**unverifiable claim**. You trust the lab. There's no portable way to check, offline, that a result
+was signed by a stated party, hasn't been altered, and covers the samples it claims.
+
+proofbundle is that check. It's a small MIT-licensed Python tool (a compact, auditable trusted core,
+depends only on [`cryptography`](https://cryptography.io)) that turns a result into a signed
+receipt anyone can verify from a single file — and it's honest about the line it does not cross.
+
+## 60-second try (offline, no setup)
+
+```bash
+pip install "proofbundle[eval]"
+proofbundle demo
+```
+
+You'll see an honest receipt verify `=> OK`, then six independent tampers each verify `FAILED`, then
+a swapped sample get caught — all in memory. The command exits non-zero if any tamper slips through,
+so it's also a self-test. Full walkthrough: **[docs/DEMO.md](docs/DEMO.md)**.
+
+```bash
+# your own receipt, from a signed payload:
+proofbundle emit --payload-file result.json --new-key signer.key --out receipt.json
+proofbundle verify receipt.json        # exit 0 = OK, 1 = failed, 2 = malformed
+```
+
+## What a receipt proves — and what it doesn't
+
+| ✅ It proves | ❌ It does **not** prove |
+|---|---|
+| These exact bytes were signed by this key (**authorship**) | That the number is **true** |
+| Nothing changed since signing (**integrity**, Ed25519 + RFC 6962) | That the **issuer is honest** |
+| The result is attributable to a stated issuer | That the **eval was well-designed** |
+| A threshold was met while hiding the model/dataset (salted commitments) | That there was **no cherry-picking** — unless pre-registered |
+| Optionally: individual samples, offline-auditable (per-sample Merkle) | That the **computation was correct** — that needs a TEE or independent reproduction |
+
+This boundary is the point, not a weakness. A receipt makes a claim **attributable, tamper-evident,
+and — with pre-registration and per-sample auditing — bounded and spot-checkable**. Full detail:
+**[THREAT_MODEL.md](THREAT_MODEL.md)**.
+
+## How it fits together
+
+```mermaid
+flowchart LR
+    H["eval harness<br/>inspect_ai · lm-eval · promptfoo · pytest"] --> A["adapter → signed claim<br/>salted commitments · provenance · samples root"]
+    A --> R["receipt<br/>one portable file"]
+    R --> V{{"proofbundle verify — offline"}}
+    V --> C["signature · Merkle inclusion · SD-JWT/KB ·<br/>witness quorum · status list · sample openings"]
+    C --> OK(["=> OK / FAILED"])
+    style V fill:#D6248A,stroke:#D6248A,color:#fff
+    style OK fill:#D6248A,stroke:#D6248A,color:#fff
+```
+
+## What's in the box
+
+- **Core** — Ed25519 signature + RFC 6962 / 9162 Merkle inclusion, verified fully offline. Checks a
+  real [Sigstore Rekor](https://docs.sigstore.dev/) proof, so correctness isn't self-referential.
+- **Eval receipts** — a signed claim (`metric ⋈ threshold`, `n`, salted model/dataset commitments,
+  assurance level, provenance) from your run. See [EVAL_CLAIM.md](EVAL_CLAIM.md).
+- **Selective disclosure** — SD-JWT ([RFC 9901](https://datatracker.ietf.org/doc/rfc9901/)) with Key
+  Binding: prove a threshold while withholding the exact score.
+- **Transparency-log interop** — C2SP `tlog-checkpoint` / cosignature / `.tlog-proof`, with
+  post-quantum **ML-DSA-44** witness cosignatures. Optional Token-Status-List revocation snapshots.
+- **Per-sample audit** — commit to every sample; an auditor challenges random indices (with a fresh
+  nonce or a **public randomness beacon**, v1.9) and openings must bind to the signed root. Catches
+  1% sample-doctoring with 95% confidence at 300 samples, regardless of run size.
+- **Pre-registration** — `proofbundle prereg <plan>` commits to the protocol before the run, so
+  best-of-many publishing becomes visible.
+- **Integrations** — opt-in inspect_ai end-of-task hook and pytest plugin (emit only when
+  `PROOFBUNDLE_EMIT=1` / `--proofbundle`), plus a Hugging Face Community Evals bridge. See
+  [INTEGRATIONS.md](INTEGRATIONS.md).
+
+## Docs
+
+| For… | Read |
+|---|---|
+| Skeptics (why not SHA-256 / Sigstore / trust the issuer) | [docs/FAQ.md](docs/FAQ.md) |
+| Reviewers (30-minute adversarial audit path) | [docs/REVIEWERS.md](docs/REVIEWERS.md) |
+| Where every trust anchor comes from | [docs/TRUST_ANCHORS.md](docs/TRUST_ANCHORS.md) |
+| The demos, tier by tier | [docs/DEMO.md](docs/DEMO.md) |
+| The normative format + verification order | [SPEC.md](SPEC.md) |
+| Honest comparison to Rekor / in-toto / OMS / ValiChord | [INTEROP.md](INTEROP.md) |
+| Regulatory mapping (and what to never claim) | [COMPLIANCE.md](COMPLIANCE.md) |
+| Funders / role fit | [docs/PROJECT_BRIEF.md](docs/PROJECT_BRIEF.md) |
+
+## Install
+
+```bash
+pip install proofbundle                 # core: offline verify + plain emit (dependency-free)
+pip install "proofbundle[eval]"          # + eval receipts, prereg, and the demo (adds rfc8785 JCS)
+pip install "proofbundle[eval]"         # emit eval receipts (adds an RFC 8785 canonicalizer)
+pip install "proofbundle[inspect]"      # inspect_ai adapter + hook
+pip install "proofbundle[pq]"           # verify ML-DSA-44 (post-quantum) witness cosignatures
+```
+
+Requires Python 3.10+. The verify path never rolls its own crypto — Ed25519 comes from
+`cryptography`; Merkle hashing is RFC 6962.
+
+## Status & scope
+
+Beta, SemVer-committed, 299 tests + a CI mutation gate + property-based parser fuzzing. Correctness
+is anchored to external RFC 6962 vectors and a real Rekor proof, not just its own bundles. It is
+**not** a log service, a full in-toto client, a TEE, a consensus network, or a compliance product
+by itself — it is the small, offline, standards-native receipt layer between them. Security policy:
+[SECURITY.md](SECURITY.md).
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md) and the [Code of Conduct](CODE_OF_CONDUCT.md). Good first
+issues are labeled [`good-first-issue`](https://github.com/b7n0de/proofbundle/labels/good-first-issue);
+security findings go through [SECURITY.md](SECURITY.md). The verifier core aims to stay small,
+dependency-light, and correct.
+
+## License
+
+MIT — see [LICENSE](LICENSE).
+
+---
+
+<p align="center"><sub>proofbundle is part of <b>b7n0de</b>, Verified AI Work · <a href="https://b7n0de.com">b7n0de.com</a></sub></p>

{proofbundle-1.7.0 → proofbundle-1.9.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "proofbundle"
-version = "1.7.0"
+version = "1.9.0"
 description = "Emit and verify portable cryptographic evidence bundles, offline: Ed25519 + RFC 6962 Merkle + optional SD-JWT."
 readme = "README.md"
 requires-python = ">=3.10"

{proofbundle-1.7.0 → proofbundle-1.9.0}/src/proofbundle/__init__.py

@@ -13,7 +13,7 @@ from __future__ import annotations
 
 from typing import TYPE_CHECKING
 
-__version__ = "1.7.0"
+__version__ = "1.9.0"
 
 __all__ = [
     "__version__",
@@ -36,6 +36,9 @@ __all__ = [
     "sample_opening",
     "verify_sample_opening",
     "audit_challenge",
+    "prereg_hash",
+    "verify_prereg",
+    "beacon_audit_challenge",
     "VerificationResult",
     "Check",
     "ProofBundleError",
@@ -59,6 +62,9 @@ _LAZY = {
     "sample_opening": ".persample",
     "verify_sample_opening": ".persample",
     "audit_challenge": ".persample",
+    "prereg_hash": ".prereg",
+    "verify_prereg": ".prereg",
+    "beacon_audit_challenge": ".beacon",
 }
 
 if TYPE_CHECKING:  # static analysers + IDEs see the real names/types; runtime stays lazy
@@ -70,6 +76,8 @@ if TYPE_CHECKING:  # static analysers + IDEs see the real names/types; runtime s
     from .hf_evals import receipt_token, verify_receipt_token
     from .persample import (audit_challenge, build_sample_tree, sample_opening,
                             verify_sample_opening)
+    from .beacon import beacon_audit_challenge
+    from .prereg import prereg_hash, verify_prereg
     from .statuslist import verify_status_snapshot
     from .tlogproof import verify_tlog_proof
     from .merkle import verify_consistency, verify_inclusion

proofbundle-1.9.0/src/proofbundle/adapters/_provenance.py

@@ -0,0 +1,63 @@
+"""Shared provenance helpers for adapters (v1.8).
+
+The external review found that run-id and config-hash were missing from nearly every adapter and
+that the two flagship adapters took the timestamp from the caller rather than the eval log. These
+helpers close that: each adapter now records, where the framework exposes it, a stable RUN id, a
+CONFIG hash, and the LOG-NATIVE timestamp — so a receipt is traceable back to the exact run.
+
+Design notes (verified against framework source, 2026-07):
+  - No framework ships a canonical config hash, so we compute our own. Config is an in-memory
+    object re-serialized non-deterministically, so it MUST be canonicalized before hashing —
+    RFC 8785 JCS via the same `rfc8785` extra the emit path already needs. If that extra is
+    absent we fall back to a deterministic `json.dumps(sort_keys=True)` and LABEL the hash
+    algorithm accordingly, so a verifier is never misled about how the hash was formed.
+  - The hash is over the config's JSON, prefixed with a domain tag, hex sha256. It is provenance
+    metadata (traceability), NOT a security commitment — it is not salted and reveals structure;
+    it exists so two receipts from the same config are linkable and a changed config is visible.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+from typing import Optional
+
+_CONFIG_DOMAIN = b"proofbundle/v1.8/config-hash\x00"
+
+
+def config_hash(config) -> Optional[str]:
+    """Return ``"<alg>:<hex>"`` over the canonical JSON of a config object, or None if it is
+    empty/None. ``<alg>`` is ``sha256-jcs`` when RFC 8785 is available, else ``sha256-sortkeys``
+    (both deterministic; the label tells a verifier which normalization produced the hex)."""
+    if config is None or config == {} or config == []:
+        return None
+    try:
+        import rfc8785  # noqa: PLC0415 — same optional dep as the emit path
+        canonical = rfc8785.dumps(config)
+        alg = "sha256-jcs"
+    except (ImportError, ValueError, TypeError):
+        # rfc8785 rejects non-JCS-able values (e.g. floats it deems unsafe); fall back to a
+        # deterministic stdlib serialization and label it so the difference is never hidden.
+        try:
+            canonical = json.dumps(config, sort_keys=True, separators=(",", ":"),
+                                   ensure_ascii=False).encode("utf-8")
+        except (TypeError, ValueError):
+            return None
+        alg = "sha256-sortkeys"
+    return f"{alg}:{hashlib.sha256(_CONFIG_DOMAIN + canonical).hexdigest()}"
+
+
+def add_provenance(provenance: dict, *, run_id=None, config=None, log_timestamp=None,
+                   config_hash_value: Optional[str] = None) -> dict:
+    """Merge the standard traceability fields into a provenance dict, skipping absent ones.
+
+    ``config_hash_value`` lets a caller pass a precomputed hash (e.g. over already-canonical
+    material) instead of a config object; otherwise ``config`` is hashed here."""
+    if run_id:
+        provenance["run_id"] = str(run_id)
+    if log_timestamp is not None:
+        provenance["run_timestamp"] = str(log_timestamp)
+    ch = config_hash_value if config_hash_value is not None else config_hash(config)
+    if ch:
+        provenance["config_hash"] = ch
+    return provenance

{proofbundle-1.7.0 → proofbundle-1.9.0}/src/proofbundle/adapters/inspect_ai.py

@@ -90,6 +90,15 @@ def from_inspect_ai_log(path, metric: str, *, comparator: str, threshold: str, t
     if tv is not None:
         provenance["task_version"] = str(tv)
 
+    # v1.8 (external review): run-id + config-hash + LOG-NATIVE timestamp so a receipt traces back
+    # to the exact run. inspect_ai: eval.run_id (unique run id), eval.created (UTC datetime string),
+    # eval.task_args (the config material — no native config hash exists, so we compute one).
+    from ._provenance import add_provenance  # noqa: PLC0415
+    task_args = getattr(ev, "task_args", None)
+    add_provenance(provenance, run_id=getattr(ev, "run_id", None),
+                   config=task_args if isinstance(task_args, dict) else None,
+                   log_timestamp=getattr(ev, "created", None))
+
     return build_eval_claim(
         suite=suite, suite_version=str(getattr(ev, "task_version", "1")),
         metric=metric, comparator=comparator, threshold=threshold, score=_score_str(value),

{proofbundle-1.7.0 → proofbundle-1.9.0}/src/proofbundle/adapters/lm_eval.py

@@ -71,6 +71,16 @@ def from_lm_eval_results(path, task: str, metric: str, *, comparator: str, thres
     if stderr is not None:
         provenance["stderr"] = repr(stderr) if not isinstance(stderr, str) else stderr
 
+    # v1.8 (external review): config-hash + LOG-NATIVE timestamp. lm-eval has no dedicated run-id;
+    # its `date` is a Unix float (distinct from the ISO filename stamp). `config` is the run config
+    # block (model/args/seeds/gen_kwargs); no native hash exists, so we compute one.
+    from ._provenance import add_provenance  # noqa: PLC0415
+    add_provenance(provenance, config=cfg if isinstance(cfg, dict) and cfg else None,
+                   log_timestamp=data.get("date"))
+    task_hashes = data.get("task_hashes", {})
+    if isinstance(task_hashes, dict) and task_hashes.get(task):
+        provenance["task_hash"] = str(task_hashes[task])   # lm-eval's native per-task sample hash
+
     return build_eval_claim(
         suite=task, suite_version=str(data.get("versions", {}).get(task, "lm-eval")),
         metric=metric, comparator=comparator, threshold=threshold, score=str(score), n=n,

{proofbundle-1.7.0 → proofbundle-1.9.0}/src/proofbundle/adapters/promptfoo.py

@@ -124,6 +124,13 @@ def from_promptfoo_results(path, *, comparator: str, threshold: str, timestamp:
     if summary.get("timestamp"):
         provenance["run_timestamp"] = str(summary["timestamp"])
 
+    # v1.8 (external review): a uniform run_id key across adapters + a config-hash over the FULL
+    # resolved suite config (providers/prompts/tests/…), not just the tests-derived dataset id.
+    from ._provenance import add_provenance  # noqa: PLC0415
+    add_provenance(provenance, run_id=eval_id,
+                   config=config if isinstance(config, dict) and config else None,
+                   log_timestamp=metadata.get("evaluationCreatedAt"))
+
     return build_eval_claim(
         suite=suite, suite_version=f"promptfoo-summary-v{version}",
         metric="pass_rate", comparator=comparator, threshold=threshold, score=score, n=n,

proofbundle-1.9.0/src/proofbundle/beacon.py

@@ -0,0 +1,109 @@
+"""Public-randomness audit challenges (v1.9) — non-interactive, publicly re-derivable.
+
+The per-sample audit (SPEC §7g) has three challenge modes. Two shipped in v1.5: an *auditor
+nonce* (interactive, grinding-impossible) and *self-challenge* (a documented-grindable sanity
+check). This module formalizes the third — a **public randomness beacon** — so an audit needs no
+live auditor and anyone can re-derive the same challenged indices from published data.
+
+Why a beacon RESISTS grinding without a live auditor, AND WHAT IT ASSUMES: the producer signs the receipt (with its
+``samples_root``) at time T. A beacon pulse from a round whose emission time is *after* T did not
+exist when the producer committed, so the producer cannot have chosen the samples to fit the
+challenge. This is the RFC 3797 pattern ("derive selections from pre-specified future public
+randomness") applied to sample auditing. Established beacons: the drand League of Entropy
+(``randomness`` = 32 bytes per round) and the NIST Interoperable Randomness Beacon
+(``outputValue`` = 64 bytes per pulse).
+
+Offline-first: proofbundle never fetches. The relying party obtains the pulse out of band (or it
+is bundled) and passes its raw bytes here. The returned ``AuditRequest`` records the beacon id
+and round so a third party can fetch the *same* pulse and re-run ``audit_challenge`` to the
+identical indices — the audit is reproducible without trusting the auditor.
+
+Soundness caveats, stated honestly (the beacon mode is grinding-resistant ONLY under these, otherwise it
+is no stronger than the documented-grindable self-challenge, so a relying party MUST check them):
+  1. Beacon signature: this module does NOT verify the beacon's own signature (drand pulses are BLS-signed;
+     NIST pulses are RSA-signed) — that is a separate trust anchor the relying party validates with the
+     beacon's public key out of band, like every anchor in docs/TRUST_ANCHORS.md.
+  2. Ordering depends on a SELF-DECLARED, UNVERIFIED timestamp. The "the round emitted after time T" argument
+     uses the receipt's own ``timestamp`` field, which the producer writes and signs but which nothing here
+     proves is the true commit time. A dishonest producer can BACKDATE ``timestamp``, wait for a round R whose
+     randomness is already public, grind sample trees against R, and sign a receipt claiming a timestamp before
+     R's emission. Ed25519 only proves the false timestamp was signed, not that it is true. So the relying party
+     must corroborate the ordering from an INDEPENDENT source (a transparency-log inclusion time for the receipt,
+     a witnessed/notarized timestamp, or a pre-registered round id chosen before the run) — not from the receipt's
+     own timestamp alone. Without independent corroboration the beacon mode does NOT close producer-side grinding.
+  3. Round independence: the round id must be fixed BEFORE the samples are committed (a future round), not chosen
+     by the producer after seeing published randomness; this module records the round but cannot enforce that it
+     was pre-committed.
+"""
+
+from __future__ import annotations
+
+import hashlib
+from typing import List
+
+from .errors import BundleFormatError
+from .persample import audit_challenge
+
+__all__ = ["AuditRequest", "beacon_nonce", "beacon_audit_challenge"]
+
+# Beacon randomness lengths we accept (raw bytes). drand = 32, NIST = 64. Other lengths are
+# allowed too (any >=16 bytes of published randomness), but these are the named, verified ones.
+_MIN_PULSE_BYTES = 16
+_BEACON_DOMAIN = b"proofbundle/v1.9/beacon-nonce\x00"
+
+
+class AuditRequest:
+    """A reproducible audit challenge derived from a public beacon pulse.
+
+    Everything a third party needs to RE-derive the same indices: the beacon id, the round, and
+    the resulting indices. ``as_dict`` is JSON-serializable for publishing alongside the receipt.
+    """
+
+    __slots__ = ("beacon", "round", "n", "k", "indices")
+
+    def __init__(self, beacon: str, round_: int, n: int, k: int, indices: List[int]):
+        self.beacon = beacon
+        self.round = round_
+        self.n = n
+        self.k = k
+        self.indices = indices
+
+    def as_dict(self) -> dict:
+        return {"beacon": self.beacon, "round": self.round, "n": self.n, "k": self.k,
+                "indices": list(self.indices)}
+
+    def __repr__(self) -> str:  # pragma: no cover - cosmetic
+        return f"AuditRequest(beacon={self.beacon!r}, round={self.round}, k={self.k}, n={self.n})"
+
+
+def beacon_nonce(pulse_randomness: bytes, beacon: str, round_: int) -> bytes:
+    """Derive the ``audit_challenge`` nonce from a beacon pulse, binding the beacon id + round.
+
+    The nonce is ``SHA-256(domain ‖ beacon ‖ 0x00 ‖ u64(round) ‖ pulse_randomness)`` — so two
+    different beacons or rounds never collide to the same challenge, and the nonce is a fixed
+    32 bytes regardless of the beacon's own randomness length.
+    """
+    if not isinstance(pulse_randomness, (bytes, bytearray)) or len(pulse_randomness) < _MIN_PULSE_BYTES:
+        raise BundleFormatError(
+            f"beacon pulse randomness must be at least {_MIN_PULSE_BYTES} bytes")
+    if not beacon or "\x00" in beacon:
+        raise BundleFormatError("beacon id must be a non-empty string without NUL")
+    if isinstance(round_, bool) or not isinstance(round_, int) or round_ < 0 or round_ >= 2**64:
+        raise BundleFormatError("beacon round must be a u64 (0 <= round < 2**64)")
+    return hashlib.sha256(_BEACON_DOMAIN + beacon.encode("utf-8") + b"\x00"
+                          + round_.to_bytes(8, "big") + bytes(pulse_randomness)).digest()
+
+
+def beacon_audit_challenge(root, n: int, k: int, *, pulse_randomness: bytes, beacon: str,
+                           round_: int) -> AuditRequest:
+    """Derive a reproducible per-sample audit challenge from a public beacon pulse.
+
+    ``root``/``n``/``k`` are the receipt's signed samples root, committed count, and the number
+    of samples to challenge. ``pulse_randomness`` is the raw randomness of a beacon pulse whose
+    round emits AFTER the receipt's signed timestamp (the relying party checks that). Returns an
+    :class:`AuditRequest` recording the beacon id + round + indices, so the challenge is
+    publicly re-derivable — no live auditor, no trust in who ran it.
+    """
+    nonce = beacon_nonce(pulse_randomness, beacon, round_)
+    indices = audit_challenge(root, n, k, nonce)
+    return AuditRequest(beacon=beacon, round_=round_, n=n, k=k, indices=indices)