proofbundle 0.9.0__tar.gz → 1.0.0__tar.gz

{proofbundle-0.9.0/src/proofbundle.egg-info → proofbundle-1.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: proofbundle
-Version: 0.9.0
+Version: 1.0.0
 Summary: Emit and verify portable cryptographic evidence bundles, offline: Ed25519 + RFC 6962 Merkle + optional SD-JWT.
 Author: Konrad Gruszka
 License: MIT
@@ -27,8 +27,10 @@ Provides-Extra: sdjwt
 Provides-Extra: eval
 Requires-Dist: rfc8785>=0.1.4; extra == "eval"
 Provides-Extra: adapters
+Provides-Extra: pytest
+Requires-Dist: pytest>=7; extra == "pytest"
 Provides-Extra: inspect
-Requires-Dist: inspect_ai<0.4,>=0.3.100; python_version >= "3.10" and extra == "inspect"
+Requires-Dist: inspect_ai<0.4,>=0.3.112; python_version >= "3.10" and extra == "inspect"
 Provides-Extra: dev
 Requires-Dist: pytest>=7; extra == "dev"
 Requires-Dist: ruff>=0.5; extra == "dev"
@@ -38,7 +40,7 @@ Requires-Dist: build>=1; extra == "dev"
 Requires-Dist: hypothesis>=6; extra == "dev"
 Requires-Dist: rfc8785>=0.1.4; extra == "dev"
 Requires-Dist: sd-jwt>=0.10; extra == "dev"
-Requires-Dist: inspect_ai<0.4,>=0.3.100; python_version >= "3.10" and extra == "dev"
+Requires-Dist: inspect_ai<0.4,>=0.3.112; python_version >= "3.10" and extra == "dev"
 Dynamic: license-file
 
 <div align="center">
@@ -71,7 +73,7 @@ no server, no network.**
 
 **At a glance:** `proofbundle emit` signs and anchors a payload; `proofbundle
 verify` checks one self-contained `bundle.json` with three offline cryptographic
-checks → `OK` or `FAILED`. No network, no daemon, no own crypto. 96 tests.
+checks → `OK` or `FAILED`. No network, no daemon, no own crypto. 102 tests.
 
 ## Contents
 
@@ -81,6 +83,7 @@ checks → `OK` or `FAILED`. No network, no daemon, no own crypto. 96 tests.
 - [Install](#install)
 - [Quickstart](#quickstart)
 - [Demo](#demo--a-real-eval-log-to-a-verified-receipt-offline)
+- [Integrations](#integrations--a-signed-receipt-of-your-eval-or-test-run-automatically-v10)
 - [Interoperability](#interoperability)
 - [Bundle format](#bundle-format-proofbundlev01)
 - [Eval receipts](#eval-receipts)
@@ -226,6 +229,25 @@ that the *artifact* is signed and offline-verifiable, with model and dataset kep
 See [`examples/inspect_receipt.py`](examples/inspect_receipt.py) and
 [`examples/lm_eval_receipt.py`](examples/lm_eval_receipt.py).
 
+## Integrations — a signed receipt of your eval or test run, automatically (v1.0)
+
+Since v1.0, proofbundle can **auto-emit** a signed receipt of an **inspect_ai eval** or a **pytest run** via
+each framework's native plugin API — installed and ready, but strictly **opt-in** (it emits only when you set
+`PROOFBUNDLE_EMIT=1` or pass a flag; never silently, never failing your run):
+
+```bash
+pip install "proofbundle[inspect,eval]" && PROOFBUNDLE_EMIT=1 inspect eval task.py --model mockllm/model
+pip install "proofbundle[pytest,eval]"  && PROOFBUNDLE_EMIT=1 pytest
+```
+
+The distinguishing angle is exactly this opt-in **auto-emit of an Ed25519-signed receipt via the framework's
+own plugin** (an inspect_ai end-of-task hook + a pytest11 plugin), on top of the standards stack. Named
+fairly: [ai-audit-trail](https://pypi.org/project/ai-audit-trail/) records *runtime* agent Decision Receipts
+(FastAPI/LangChain, ISO 42001), a different layer; [ValiChord](https://github.com/topeuph-ai/ValiChord)
+builds attestation bundles from inspect_ai logs *post-hoc* (its v1 library is JCS + SHA-256 Merkle + HMAC
+challenge-response, **not digitally signed** — signatures are v2 scope). See
+[INTEGRATIONS.md](INTEGRATIONS.md) (+ a prepared composite GitHub Action under [`action/`](action/action.yml)).
+
 ## Interoperability
 
 proofbundle uses the same RFC 6962 / RFC 9162 Merkle primitive as
@@ -336,9 +358,9 @@ computation is the domain of TEE approaches such as
 fairly: [Every Eval Ever](https://github.com/evaleval/every_eval_ever) standardizes eval *metadata* but
 adds no cryptography (proofbundle ships an EEE→receipt converter);
 [OpenSSF Model Signing](https://github.com/ossf/model-signing-spec) signs *model weights*, not eval
-results; [ValiChord](https://github.com/topeuph-ai/ValiChord) provides blind peer consensus and an
-attested log on a Holochain network (its v1 attestation library uses a simple SHA-256 Merkle tree, no
-signature, no SD-JWT, no in-toto). proofbundle is the lightweight, **standards-native** piece between them:
+results; [ValiChord](https://github.com/topeuph-ai/ValiChord) plans blind peer consensus and a
+Holochain attested log (v2 scope); its current v1 attestation library uses a simple SHA-256 Merkle tree with
+no digital signature, no SD-JWT, no in-toto. proofbundle is the lightweight, **standards-native** piece between them:
 a portable receipt a third party verifies offline, with selective disclosure so an auditor can prove a
 threshold was met without revealing the model or the data. See [INTEROP.md](INTEROP.md).
 
@@ -384,8 +406,10 @@ attestation — see [SECURITY.md](SECURITY.md).
   verifier robustness + CI on Python 3.9 after a holistic review.
 - **v0.8** — an offline `make demo` (real eval log -> signed receipt -> verified),
   a sharpened honesty guardrail (authenticity/integrity, not computation-correctness), and outreach drafts.
-- **v0.9 (current release)** — the standards moat: a DSSE-signed in-toto `test-result` export, a C2SP
-  tlog-checkpoint over the RFC 6962 root, an Every Eval Ever converter, and standards-native repositioning.
+- **v0.9** — the standards moat: a DSSE-signed in-toto `test-result` export, a C2SP tlog-checkpoint over
+  the RFC 6962 root, an Every Eval Ever converter, and standards-native repositioning.
+- **v1.0 (current release)** — distribution: opt-in framework integrations that auto-emit a signed receipt
+  of an inspect_ai eval (end-of-task hook) or a pytest run (pytest11 plugin), plus a composite GitHub Action.
 - **Deferred** (explicitly not yet built) — SD-JWT VC conformance + `vct` metadata,
   Key-Binding JWT, status lists / revocation, an official in-toto PR, DSSE / a full in-toto client.
 

{proofbundle-0.9.0 → proofbundle-1.0.0}/README.md

@@ -28,7 +28,7 @@ no server, no network.**
 
 **At a glance:** `proofbundle emit` signs and anchors a payload; `proofbundle
 verify` checks one self-contained `bundle.json` with three offline cryptographic
-checks → `OK` or `FAILED`. No network, no daemon, no own crypto. 96 tests.
+checks → `OK` or `FAILED`. No network, no daemon, no own crypto. 102 tests.
 
 ## Contents
 
@@ -38,6 +38,7 @@ checks → `OK` or `FAILED`. No network, no daemon, no own crypto. 96 tests.
 - [Install](#install)
 - [Quickstart](#quickstart)
 - [Demo](#demo--a-real-eval-log-to-a-verified-receipt-offline)
+- [Integrations](#integrations--a-signed-receipt-of-your-eval-or-test-run-automatically-v10)
 - [Interoperability](#interoperability)
 - [Bundle format](#bundle-format-proofbundlev01)
 - [Eval receipts](#eval-receipts)
@@ -183,6 +184,25 @@ that the *artifact* is signed and offline-verifiable, with model and dataset kep
 See [`examples/inspect_receipt.py`](examples/inspect_receipt.py) and
 [`examples/lm_eval_receipt.py`](examples/lm_eval_receipt.py).
 
+## Integrations — a signed receipt of your eval or test run, automatically (v1.0)
+
+Since v1.0, proofbundle can **auto-emit** a signed receipt of an **inspect_ai eval** or a **pytest run** via
+each framework's native plugin API — installed and ready, but strictly **opt-in** (it emits only when you set
+`PROOFBUNDLE_EMIT=1` or pass a flag; never silently, never failing your run):
+
+```bash
+pip install "proofbundle[inspect,eval]" && PROOFBUNDLE_EMIT=1 inspect eval task.py --model mockllm/model
+pip install "proofbundle[pytest,eval]"  && PROOFBUNDLE_EMIT=1 pytest
+```
+
+The distinguishing angle is exactly this opt-in **auto-emit of an Ed25519-signed receipt via the framework's
+own plugin** (an inspect_ai end-of-task hook + a pytest11 plugin), on top of the standards stack. Named
+fairly: [ai-audit-trail](https://pypi.org/project/ai-audit-trail/) records *runtime* agent Decision Receipts
+(FastAPI/LangChain, ISO 42001), a different layer; [ValiChord](https://github.com/topeuph-ai/ValiChord)
+builds attestation bundles from inspect_ai logs *post-hoc* (its v1 library is JCS + SHA-256 Merkle + HMAC
+challenge-response, **not digitally signed** — signatures are v2 scope). See
+[INTEGRATIONS.md](INTEGRATIONS.md) (+ a prepared composite GitHub Action under [`action/`](action/action.yml)).
+
 ## Interoperability
 
 proofbundle uses the same RFC 6962 / RFC 9162 Merkle primitive as
@@ -293,9 +313,9 @@ computation is the domain of TEE approaches such as
 fairly: [Every Eval Ever](https://github.com/evaleval/every_eval_ever) standardizes eval *metadata* but
 adds no cryptography (proofbundle ships an EEE→receipt converter);
 [OpenSSF Model Signing](https://github.com/ossf/model-signing-spec) signs *model weights*, not eval
-results; [ValiChord](https://github.com/topeuph-ai/ValiChord) provides blind peer consensus and an
-attested log on a Holochain network (its v1 attestation library uses a simple SHA-256 Merkle tree, no
-signature, no SD-JWT, no in-toto). proofbundle is the lightweight, **standards-native** piece between them:
+results; [ValiChord](https://github.com/topeuph-ai/ValiChord) plans blind peer consensus and a
+Holochain attested log (v2 scope); its current v1 attestation library uses a simple SHA-256 Merkle tree with
+no digital signature, no SD-JWT, no in-toto. proofbundle is the lightweight, **standards-native** piece between them:
 a portable receipt a third party verifies offline, with selective disclosure so an auditor can prove a
 threshold was met without revealing the model or the data. See [INTEROP.md](INTEROP.md).
 
@@ -341,8 +361,10 @@ attestation — see [SECURITY.md](SECURITY.md).
   verifier robustness + CI on Python 3.9 after a holistic review.
 - **v0.8** — an offline `make demo` (real eval log -> signed receipt -> verified),
   a sharpened honesty guardrail (authenticity/integrity, not computation-correctness), and outreach drafts.
-- **v0.9 (current release)** — the standards moat: a DSSE-signed in-toto `test-result` export, a C2SP
-  tlog-checkpoint over the RFC 6962 root, an Every Eval Ever converter, and standards-native repositioning.
+- **v0.9** — the standards moat: a DSSE-signed in-toto `test-result` export, a C2SP tlog-checkpoint over
+  the RFC 6962 root, an Every Eval Ever converter, and standards-native repositioning.
+- **v1.0 (current release)** — distribution: opt-in framework integrations that auto-emit a signed receipt
+  of an inspect_ai eval (end-of-task hook) or a pytest run (pytest11 plugin), plus a composite GitHub Action.
 - **Deferred** (explicitly not yet built) — SD-JWT VC conformance + `vct` metadata,
   Key-Binding JWT, status lists / revocation, an official in-toto PR, DSSE / a full in-toto client.
 

{proofbundle-0.9.0 → proofbundle-1.0.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "proofbundle"
-version = "0.9.0"
+version = "1.0.0"
 description = "Emit and verify portable cryptographic evidence bundles, offline: Ed25519 + RFC 6962 Merkle + optional SD-JWT."
 readme = "README.md"
 requires-python = ">=3.9"
@@ -45,13 +45,16 @@ sdjwt = []
 eval = ["rfc8785>=0.1.4"]
 # The lm-eval adapter reads exported results.json (no import) → pure stdlib.
 adapters = []
+# The pytest plugin (opt-in test-run receipt via the pytest11 entry-point) — pytest is an optional extra,
+# never a core dependency. Floor 7.0: the hooks/stats API is signature-stable across pytest 7/8/9.
+pytest = ["pytest>=7"]
 # The inspect_ai adapter uses the STABLE read_eval_log API (lazy import). Pinned with an UPPER bound:
 # the .eval format + pydantic schema change between versions (inspect_ai issue 834), and the fixture
 # test is bound to this range. inspect_ai requires Python >= 3.10, so the marker gates it out on 3.9
 # (base + [eval]/[sdjwt] still work on 3.9; the inspect adapter test skips there). Fixes the red 3.9 CI.
-inspect = ['inspect_ai>=0.3.100,<0.4; python_version >= "3.10"']
+inspect = ['inspect_ai>=0.3.112,<0.4; python_version >= "3.10"']
 dev = ["pytest>=7", "ruff>=0.5", "jsonschema>=4", "mypy>=1.8", "build>=1", "hypothesis>=6",
-       "rfc8785>=0.1.4", "sd-jwt>=0.10", 'inspect_ai>=0.3.100,<0.4; python_version >= "3.10"']
+       "rfc8785>=0.1.4", "sd-jwt>=0.10", 'inspect_ai>=0.3.112,<0.4; python_version >= "3.10"']
 
 [project.urls]
 Homepage = "https://b7n0de.com"
@@ -63,6 +66,12 @@ Documentation = "https://github.com/b7n0de/proofbundle#readme"
 [project.scripts]
 proofbundle = "proofbundle.cli:main"
 
+# Framework integrations (opt-in auto-emit; gated on PROOFBUNDLE_EMIT / a flag, never silent).
+[project.entry-points.inspect_ai]
+proofbundle = "proofbundle._inspect_registry"
+[project.entry-points.pytest11]
+proofbundle = "proofbundle.pytest_plugin"
+
 [tool.setuptools.packages.find]
 where = ["src"]
 

proofbundle-1.0.0/src/proofbundle/__init__.py

@@ -0,0 +1,56 @@
+"""proofbundle — emit and verify portable, offline cryptographic evidence bundles for AI eval receipts.
+
+Verify, fully offline and in pure Python, that a payload was Ed25519 signed and anchored under an RFC 6962
+Merkle root, with optional SD-JWT selective disclosure — plus opt-in framework integrations that auto-emit a
+signed receipt of an inspect_ai eval or a pytest run.
+
+The public API is loaded LAZILY (PEP 562): ``import proofbundle`` — and, via the entry points, loading the
+pytest plugin / inspect_ai hook — does NOT pull the crypto core until a name like ``verify_bundle`` is
+actually used. ``from proofbundle import verify_bundle`` works exactly as before; it just imports the backing
+module on first access. This keeps the framework integrations light at startup.
+"""
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+__version__ = "1.0.0"
+
+__all__ = [
+    "__version__",
+    "SCHEMA",
+    "verify_bundle",
+    "load_bundle",
+    "emit_bundle",
+    "generate_signer",
+    "verify_inclusion",
+    "verify_consistency",
+    "VerificationResult",
+    "Check",
+    "ProofBundleError",
+]
+
+# name → backing submodule (relative). Loaded on first attribute access.
+_LAZY = {
+    "SCHEMA": ".bundle", "load_bundle": ".bundle", "verify_bundle": ".bundle",
+    "emit_bundle": ".emit", "generate_signer": ".emit",
+    "Check": ".errors", "ProofBundleError": ".errors", "VerificationResult": ".errors",
+    "verify_consistency": ".merkle", "verify_inclusion": ".merkle",
+}
+
+if TYPE_CHECKING:  # static analysers + IDEs see the real names/types; runtime stays lazy
+    from .bundle import SCHEMA, load_bundle, verify_bundle
+    from .emit import emit_bundle, generate_signer
+    from .errors import Check, ProofBundleError, VerificationResult
+    from .merkle import verify_consistency, verify_inclusion
+
+
+def __getattr__(name: str):
+    module = _LAZY.get(name)
+    if module is None:
+        raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+    import importlib  # noqa: PLC0415
+    return getattr(importlib.import_module(module, __name__), name)
+
+
+def __dir__():
+    return sorted(__all__)

proofbundle-1.0.0/src/proofbundle/_inspect_registry.py

@@ -0,0 +1,3 @@
+"""Entry-point target for the inspect_ai hook group. Importing this module registers ProofbundleHooks as an
+import side-effect. Kept intentionally minimal (no crypto) so inspect's startup discovery stays fast."""
+from .inspect_hook import ProofbundleHooks  # noqa: F401 — import side-effect registers the hook

proofbundle-1.0.0/src/proofbundle/_integration.py

@@ -0,0 +1,84 @@
+"""Shared opt-in helper for the framework integrations (inspect_ai hook, pytest plugin) — v1.0.
+
+THE TOP RULE (opt-in safety): an integration must NEVER silently write a file or alter a host run. It emits
+a receipt ONLY when the user explicitly turns it on — the ``PROOFBUNDLE_EMIT=1`` environment variable, or a
+framework flag that maps to it. A security tool that surprises you loses trust. Every function here is a
+no-op unless emission is enabled, catches its own errors (an integration must never fail the host run), and
+imports the crypto lazily (this module is only imported from inside a hook body, never at framework startup).
+
+Configuration (all optional, all env):
+  PROOFBUNDLE_EMIT       "1" to enable emission (the master opt-in). Anything else = disabled.
+  PROOFBUNDLE_KEY        path to a 32-byte raw Ed25519 seed to sign with. If unset, an EPHEMERAL key is
+                         generated (a warning is printed; the receipt is self-verifiable but not tied to a
+                         durable identity).
+  PROOFBUNDLE_OUT        output path: a file, or a directory (the default file name is written into it).
+                         Default: the default file name in the current directory.
+  PROOFBUNDLE_METRIC     which metric to bind (else the integration's first/most-relevant metric).
+  PROOFBUNDLE_COMPARATOR ">=" | ">" | "<=" | "<"  (default ">=").
+  PROOFBUNDLE_THRESHOLD  decimal string (default "0") — the pass/fail threshold to assert.
+"""
+from __future__ import annotations
+
+import os
+from pathlib import Path
+from typing import Optional
+
+DEFAULT_COMPARATOR = ">="
+DEFAULT_THRESHOLD = "0"
+
+
+def emit_enabled(flag: bool = False) -> bool:
+    """The master opt-in gate. True only if PROOFBUNDLE_EMIT == "1" OR an explicit framework flag is set."""
+    return flag or os.environ.get("PROOFBUNDLE_EMIT") == "1"
+
+
+def emit_config() -> dict:
+    """Read the (metric, comparator, threshold) emission config from the environment, with safe defaults."""
+    return {
+        "metric": os.environ.get("PROOFBUNDLE_METRIC"),
+        "comparator": os.environ.get("PROOFBUNDLE_COMPARATOR") or DEFAULT_COMPARATOR,
+        "threshold": os.environ.get("PROOFBUNDLE_THRESHOLD") or DEFAULT_THRESHOLD,
+    }
+
+
+def _resolve_signer():
+    """Return (signer, is_ephemeral). Loads PROOFBUNDLE_KEY if set, else generates an ephemeral key."""
+    from .emit import generate_signer, load_signer  # noqa: PLC0415 — lazy: only on actual emit
+    key_path = os.environ.get("PROOFBUNDLE_KEY")
+    if key_path:
+        return load_signer(key_path), False
+    return generate_signer(), True
+
+
+def _output_path(default_name: str) -> Path:
+    """Resolve the output file path from PROOFBUNDLE_OUT (file or directory) or the default name in cwd."""
+    out = os.environ.get("PROOFBUNDLE_OUT")
+    if not out:
+        return Path.cwd() / default_name
+    p = Path(out)
+    if p.is_dir() or out.endswith(os.sep):
+        return p / default_name
+    return p
+
+
+def emit_claim_receipt(claim: dict, default_name: str) -> Optional[str]:
+    """Sign ``claim`` into an eval receipt and write it to the resolved output path. Returns the path, or
+    None on any failure (an integration must never raise into the host run). Assumes emission is enabled
+    (the caller checks ``emit_enabled`` first)."""
+    try:
+        from .evalclaim import emit_eval_receipt  # noqa: PLC0415 — lazy
+        import json  # noqa: PLC0415
+
+        signer, ephemeral = _resolve_signer()
+        if ephemeral:
+            print("[proofbundle] PROOFBUNDLE_KEY not set — signing with an EPHEMERAL key "
+                  "(receipt is self-verifiable but not bound to a durable identity).")
+        bundle = emit_eval_receipt(claim, signer)
+        out = _output_path(default_name)
+        out.parent.mkdir(parents=True, exist_ok=True)
+        out.write_text(json.dumps(bundle, indent=2), encoding="utf-8")
+        print(f"[proofbundle] wrote signed eval receipt → {out}")
+        return str(out)
+    except Exception as e:  # noqa: BLE001 — never let emission break the host run
+        print(f"[proofbundle] receipt emission skipped ({type(e).__name__}: {e})")
+        return None

{proofbundle-0.9.0 → proofbundle-1.0.0}/src/proofbundle/adapters/inspect_ai.py

@@ -21,23 +21,42 @@ class InspectAdapterError(RuntimeError):
     """Raised when inspect_ai is missing or the log lacks the expected structure (no bare AttributeError)."""
 
 
+def _score_str(value) -> str:
+    """Render a metric value as a PLAIN decimal string (no scientific notation) that build_eval_claim
+    accepts. ``repr(float)`` emits '1e-05'/'1e+20' for very small/large values, which the claim's decimal
+    pattern rejects — so numbers are formatted fixed-point (like the pytest plugin's ``_fmt``)."""
+    if isinstance(value, bool):
+        return "1" if value else "0"
+    if isinstance(value, int):
+        return str(value)
+    if isinstance(value, float):
+        if value != value or value in (float("inf"), float("-inf")):
+            raise InspectAdapterError("metric value must be finite")
+        return format(value, ".12f").rstrip("0").rstrip(".") or "0"
+    return str(value)
+
+
 def from_inspect_ai_log(path, metric: str, *, comparator: str, threshold: str, timestamp: str,
                         model_salt: Optional[bytes] = None, dataset_salt: Optional[bytes] = None):
     """Read an inspect_ai eval log via the stable API and build an eval claim for `metric`.
 
-    Returns (claim, salts). Raises InspectAdapterError if inspect_ai is unavailable or the log is
-    missing the expected attributes — a clear error instead of an opaque AttributeError.
+    ``path`` may be a path/str to a ``.eval`` log OR an already-loaded EvalLog object (e.g. the inspect_ai
+    hook's ``data.log``). Returns (claim, salts). Raises InspectAdapterError if inspect_ai is unavailable
+    or the log is missing the expected attributes — a clear error instead of an opaque AttributeError.
     """
-    try:
-        from inspect_ai.log import read_eval_log  # noqa: PLC0415 — lazy: keeps the core dependency-free
-    except ImportError as e:
-        raise InspectAdapterError(
-            "inspect_ai is required for this adapter — install with: pip install \"proofbundle[inspect]\"") from e
-
-    try:
-        log = read_eval_log(str(path), header_only=True)
-    except Exception as e:  # noqa: BLE001 — surface any read/parse failure as a clear adapter error
-        raise InspectAdapterError(f"could not read inspect_ai log {path!r}: {e}") from e
+    # An already-loaded EvalLog (has .eval + .results) is used directly — no re-read from disk.
+    if hasattr(path, "eval") and hasattr(path, "results"):
+        log = path
+    else:
+        try:
+            from inspect_ai.log import read_eval_log  # noqa: PLC0415 — lazy: keeps the core dependency-free
+        except ImportError as e:
+            raise InspectAdapterError(
+                "inspect_ai is required for this adapter — install with: pip install \"proofbundle[inspect]\"") from e
+        try:
+            log = read_eval_log(str(path), header_only=True)
+        except Exception as e:  # noqa: BLE001 — surface any read/parse failure as a clear adapter error
+            raise InspectAdapterError(f"could not read inspect_ai log {path!r}: {e}") from e
 
     ev = getattr(log, "eval", None)
     results = getattr(log, "results", None)
@@ -73,7 +92,7 @@ def from_inspect_ai_log(path, metric: str, *, comparator: str, threshold: str, t
 
     return build_eval_claim(
         suite=suite, suite_version=str(getattr(ev, "task_version", "1")),
-        metric=metric, comparator=comparator, threshold=threshold, score=repr(value),
+        metric=metric, comparator=comparator, threshold=threshold, score=_score_str(value),
         n=int(getattr(results, "total_samples", 0) or 0),
         model_id=model_id, dataset_id=dataset_id, issuer="", timestamp=timestamp,
         provenance=provenance, model_salt=model_salt, dataset_salt=dataset_salt)

proofbundle-1.0.0/src/proofbundle/inspect_hook.py

@@ -0,0 +1,63 @@
+"""inspect_ai end-of-task hook — opt-in auto-emit of a signed eval receipt (v1.0).
+
+Registered via the ``inspect_ai`` entry-point group (see ``_inspect_registry``). When a task ends, and ONLY
+when the user has opted in (``PROOFBUNDLE_EMIT=1``), this builds a signed proofbundle receipt from the eval
+log and writes it out — never silently, never failing the eval. The heavy crypto is imported lazily inside
+the hook body, so merely installing proofbundle does not slow inspect's startup.
+
+Requires ``inspect_ai>=0.3.112`` (the generic lifecycle hooks + ``on_task_end``). Reuses the existing
+``from_inspect_ai_log`` adapter — ``data.log`` is already an EvalLog, so no re-read of the .eval file for a
+normal ``eval()``. For an ``eval_set()`` the log may be header-only; then we fall back to reading the log
+from ``data.log.location`` to recover the results.
+"""
+from __future__ import annotations
+
+import os
+
+from inspect_ai.hooks import Hooks, hooks
+
+
+def _emit_enabled() -> bool:
+    # kept inline + cheap: enabled() is consulted before every hook invocation (potentially per sample)
+    return os.environ.get("PROOFBUNDLE_EMIT") == "1"
+
+
+def _first_metric(log) -> str:
+    """The first score's first metric name from an EvalLog, as a sensible default binding target."""
+    results = getattr(log, "results", None)
+    for score in (getattr(results, "scores", None) or []):
+        metrics = getattr(score, "metrics", None) or {}
+        for name in metrics:
+            return name
+    return "accuracy"
+
+
+@hooks(name="proofbundle_hooks", description="Emit a signed proofbundle eval receipt at end of task (opt-in via PROOFBUNDLE_EMIT)")
+class ProofbundleHooks(Hooks):
+    def enabled(self) -> bool:
+        return _emit_enabled()
+
+    async def on_task_end(self, data) -> None:
+        if not _emit_enabled():
+            return
+        try:
+            from datetime import datetime, timezone  # noqa: PLC0415
+
+            from ._integration import emit_claim_receipt, emit_config  # noqa: PLC0415
+            from .adapters.inspect_ai import from_inspect_ai_log  # noqa: PLC0415
+
+            log = data.log
+            # header-only (eval_set) fallback: if results are missing, re-read the full log from its location
+            if getattr(log, "results", None) is None and getattr(log, "location", None):
+                from inspect_ai.log import read_eval_log  # noqa: PLC0415
+                log = read_eval_log(str(log.location))
+
+            cfg = emit_config()
+            metric = cfg["metric"] or _first_metric(log)
+            claim, _ = from_inspect_ai_log(log, metric, comparator=cfg["comparator"],
+                                           threshold=cfg["threshold"],
+                                           timestamp=datetime.now(timezone.utc).isoformat())
+            eval_id = getattr(data, "eval_id", None) or "eval"
+            emit_claim_receipt(claim, f"proofbundle_receipt_{eval_id}.json")
+        except Exception as e:  # noqa: BLE001 — an integration must never fail the host eval
+            print(f"[proofbundle] inspect_ai receipt emission skipped ({type(e).__name__}: {e})")

proofbundle-1.0.0/src/proofbundle/pytest_plugin.py

@@ -0,0 +1,67 @@
+"""pytest plugin — opt-in auto-emit of a signed receipt of the test run (v1.0).
+
+Registered via the ``pytest11`` entry-point, so pytest loads it automatically at startup. It emits a signed
+proofbundle receipt of the run ONLY when the user opts in — the ``--proofbundle`` flag or ``PROOFBUNDLE_EMIT=1``
+— never on a normal run, never failing the run. This module must import cheaply (it loads at every pytest
+startup), so the crypto is imported lazily inside the terminal-summary hook, not at module top.
+
+Counts come from ``terminalreporter.stats`` (the canonical count source): a dict outcome→list. Only the
+pass/fail outcomes (passed/failed/error) form the denominator; skipped/xfailed/xpassed are recorded in
+provenance. ``error`` (collect/setup/teardown failures) is kept separate from ``failed`` (call-phase).
+"""
+from __future__ import annotations
+
+
+def pytest_addoption(parser):
+    group = parser.getgroup("proofbundle", "proofbundle signed test-run receipt")
+    group.addoption("--proofbundle", action="store_true", dest="proofbundle", default=False,
+                    help="emit a signed proofbundle receipt of the test run (also enabled by PROOFBUNDLE_EMIT=1)")
+
+
+def _fmt(x: float) -> str:
+    return format(x, ".6f").rstrip("0").rstrip(".") or "0"
+
+
+def pytest_terminal_summary(terminalreporter, exitstatus, config):
+    flag = bool(config.getoption("proofbundle", default=False))
+    from ._integration import emit_enabled  # noqa: PLC0415
+    if not emit_enabled(flag):
+        return
+    try:
+        from datetime import datetime, timezone  # noqa: PLC0415
+
+        from ._integration import emit_claim_receipt, emit_config  # noqa: PLC0415
+        from .evalclaim import build_eval_claim  # noqa: PLC0415
+
+        stats = terminalreporter.stats
+        counts = {k: len(stats.get(k, [])) for k in
+                  ("passed", "failed", "error", "skipped", "xfailed", "xpassed")}
+
+        # Count UNIQUE tests by node id, not the sum of reports: a single test can produce both a call-phase
+        # report (passed/failed) AND a separate setup/teardown 'error' report, so summing over-counts. A test
+        # is a clean pass only if it passed AND has no failure/error report; ran = distinct nodes with an
+        # outcome. This keeps the signed pass_rate + n honest for fixture-teardown-erroring suites.
+        def _ids(key):
+            return {getattr(r, "nodeid", id(r)) for r in stats.get(key, [])}
+        passed_ids, failed_ids, error_ids = _ids("passed"), _ids("failed"), _ids("error")
+        ran_ids = passed_ids | failed_ids | error_ids
+        clean_passed = passed_ids - failed_ids - error_ids
+        ran = len(ran_ids)
+        if ran == 0:
+            print("[proofbundle] no pass/fail tests to attest — receipt skipped")
+            return
+        pass_rate = len(clean_passed) / ran
+
+        cfg = emit_config()
+        rootname = getattr(getattr(config, "rootpath", None), "name", None) or "pytest"
+        provenance = {"harness": "pytest", "exit_status": int(exitstatus), "tests_ran": ran,
+                      "tests_passed": len(clean_passed), **{f"n_{k}": v for k, v in counts.items()}}
+        claim, _ = build_eval_claim(
+            suite="pytest", suite_version=str(getattr(__import__("pytest"), "__version__", "unknown")),
+            metric=cfg["metric"] or "pass_rate", comparator=cfg["comparator"], threshold=cfg["threshold"],
+            score=_fmt(pass_rate), n=ran, model_id=str(getattr(config, "rootpath", rootname)),
+            dataset_id="pytest-suite", issuer="", timestamp=datetime.now(timezone.utc).isoformat(),
+            provenance=provenance)
+        emit_claim_receipt(claim, "proofbundle_pytest_receipt.json")
+    except Exception as e:  # noqa: BLE001 — an integration must never fail the host test run
+        print(f"[proofbundle] pytest receipt emission skipped ({type(e).__name__}: {e})")

{proofbundle-0.9.0 → proofbundle-1.0.0/src/proofbundle.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: proofbundle
-Version: 0.9.0
+Version: 1.0.0
 Summary: Emit and verify portable cryptographic evidence bundles, offline: Ed25519 + RFC 6962 Merkle + optional SD-JWT.
 Author: Konrad Gruszka
 License: MIT
@@ -27,8 +27,10 @@ Provides-Extra: sdjwt
 Provides-Extra: eval
 Requires-Dist: rfc8785>=0.1.4; extra == "eval"
 Provides-Extra: adapters
+Provides-Extra: pytest
+Requires-Dist: pytest>=7; extra == "pytest"
 Provides-Extra: inspect
-Requires-Dist: inspect_ai<0.4,>=0.3.100; python_version >= "3.10" and extra == "inspect"
+Requires-Dist: inspect_ai<0.4,>=0.3.112; python_version >= "3.10" and extra == "inspect"
 Provides-Extra: dev
 Requires-Dist: pytest>=7; extra == "dev"
 Requires-Dist: ruff>=0.5; extra == "dev"
@@ -38,7 +40,7 @@ Requires-Dist: build>=1; extra == "dev"
 Requires-Dist: hypothesis>=6; extra == "dev"
 Requires-Dist: rfc8785>=0.1.4; extra == "dev"
 Requires-Dist: sd-jwt>=0.10; extra == "dev"
-Requires-Dist: inspect_ai<0.4,>=0.3.100; python_version >= "3.10" and extra == "dev"
+Requires-Dist: inspect_ai<0.4,>=0.3.112; python_version >= "3.10" and extra == "dev"
 Dynamic: license-file
 
 <div align="center">
@@ -71,7 +73,7 @@ no server, no network.**
 
 **At a glance:** `proofbundle emit` signs and anchors a payload; `proofbundle
 verify` checks one self-contained `bundle.json` with three offline cryptographic
-checks → `OK` or `FAILED`. No network, no daemon, no own crypto. 96 tests.
+checks → `OK` or `FAILED`. No network, no daemon, no own crypto. 102 tests.
 
 ## Contents
 
@@ -81,6 +83,7 @@ checks → `OK` or `FAILED`. No network, no daemon, no own crypto. 96 tests.
 - [Install](#install)
 - [Quickstart](#quickstart)
 - [Demo](#demo--a-real-eval-log-to-a-verified-receipt-offline)
+- [Integrations](#integrations--a-signed-receipt-of-your-eval-or-test-run-automatically-v10)
 - [Interoperability](#interoperability)
 - [Bundle format](#bundle-format-proofbundlev01)
 - [Eval receipts](#eval-receipts)
@@ -226,6 +229,25 @@ that the *artifact* is signed and offline-verifiable, with model and dataset kep
 See [`examples/inspect_receipt.py`](examples/inspect_receipt.py) and
 [`examples/lm_eval_receipt.py`](examples/lm_eval_receipt.py).
 
+## Integrations — a signed receipt of your eval or test run, automatically (v1.0)
+
+Since v1.0, proofbundle can **auto-emit** a signed receipt of an **inspect_ai eval** or a **pytest run** via
+each framework's native plugin API — installed and ready, but strictly **opt-in** (it emits only when you set
+`PROOFBUNDLE_EMIT=1` or pass a flag; never silently, never failing your run):
+
+```bash
+pip install "proofbundle[inspect,eval]" && PROOFBUNDLE_EMIT=1 inspect eval task.py --model mockllm/model
+pip install "proofbundle[pytest,eval]"  && PROOFBUNDLE_EMIT=1 pytest
+```
+
+The distinguishing angle is exactly this opt-in **auto-emit of an Ed25519-signed receipt via the framework's
+own plugin** (an inspect_ai end-of-task hook + a pytest11 plugin), on top of the standards stack. Named
+fairly: [ai-audit-trail](https://pypi.org/project/ai-audit-trail/) records *runtime* agent Decision Receipts
+(FastAPI/LangChain, ISO 42001), a different layer; [ValiChord](https://github.com/topeuph-ai/ValiChord)
+builds attestation bundles from inspect_ai logs *post-hoc* (its v1 library is JCS + SHA-256 Merkle + HMAC
+challenge-response, **not digitally signed** — signatures are v2 scope). See
+[INTEGRATIONS.md](INTEGRATIONS.md) (+ a prepared composite GitHub Action under [`action/`](action/action.yml)).
+
 ## Interoperability
 
 proofbundle uses the same RFC 6962 / RFC 9162 Merkle primitive as
@@ -336,9 +358,9 @@ computation is the domain of TEE approaches such as
 fairly: [Every Eval Ever](https://github.com/evaleval/every_eval_ever) standardizes eval *metadata* but
 adds no cryptography (proofbundle ships an EEE→receipt converter);
 [OpenSSF Model Signing](https://github.com/ossf/model-signing-spec) signs *model weights*, not eval
-results; [ValiChord](https://github.com/topeuph-ai/ValiChord) provides blind peer consensus and an
-attested log on a Holochain network (its v1 attestation library uses a simple SHA-256 Merkle tree, no
-signature, no SD-JWT, no in-toto). proofbundle is the lightweight, **standards-native** piece between them:
+results; [ValiChord](https://github.com/topeuph-ai/ValiChord) plans blind peer consensus and a
+Holochain attested log (v2 scope); its current v1 attestation library uses a simple SHA-256 Merkle tree with
+no digital signature, no SD-JWT, no in-toto. proofbundle is the lightweight, **standards-native** piece between them:
 a portable receipt a third party verifies offline, with selective disclosure so an auditor can prove a
 threshold was met without revealing the model or the data. See [INTEROP.md](INTEROP.md).
 
@@ -384,8 +406,10 @@ attestation — see [SECURITY.md](SECURITY.md).
   verifier robustness + CI on Python 3.9 after a holistic review.
 - **v0.8** — an offline `make demo` (real eval log -> signed receipt -> verified),
   a sharpened honesty guardrail (authenticity/integrity, not computation-correctness), and outreach drafts.
-- **v0.9 (current release)** — the standards moat: a DSSE-signed in-toto `test-result` export, a C2SP
-  tlog-checkpoint over the RFC 6962 root, an Every Eval Ever converter, and standards-native repositioning.
+- **v0.9** — the standards moat: a DSSE-signed in-toto `test-result` export, a C2SP tlog-checkpoint over
+  the RFC 6962 root, an Every Eval Ever converter, and standards-native repositioning.
+- **v1.0 (current release)** — distribution: opt-in framework integrations that auto-emit a signed receipt
+  of an inspect_ai eval (end-of-task hook) or a pytest run (pytest11 plugin), plus a composite GitHub Action.
 - **Deferred** (explicitly not yet built) — SD-JWT VC conformance + `vct` metadata,
   Key-Binding JWT, status lists / revocation, an official in-toto PR, DSSE / a full in-toto client.
 

{proofbundle-0.9.0 → proofbundle-1.0.0}/src/proofbundle.egg-info/SOURCES.txt

@@ -2,6 +2,8 @@ LICENSE
 README.md
 pyproject.toml
 src/proofbundle/__init__.py
+src/proofbundle/_inspect_registry.py
+src/proofbundle/_integration.py
 src/proofbundle/bundle.py
 src/proofbundle/checkpoint.py
 src/proofbundle/cli.py
@@ -10,9 +12,11 @@ src/proofbundle/eee_eval_schema.json
 src/proofbundle/emit.py
 src/proofbundle/errors.py
 src/proofbundle/evalclaim.py
+src/proofbundle/inspect_hook.py
 src/proofbundle/intoto.py
 src/proofbundle/merkle.py
 src/proofbundle/py.typed
+src/proofbundle/pytest_plugin.py
 src/proofbundle/sdjwt.py
 src/proofbundle/sdjwt_issue.py
 src/proofbundle/signature.py
@@ -37,10 +41,12 @@ tests/test_emit.py
 tests/test_eval_claim_schema.py
 tests/test_evalclaim.py
 tests/test_examples.py
+tests/test_inspect_hook.py
 tests/test_intoto.py
 tests/test_intoto_dsse.py
 tests/test_merkle.py
 tests/test_merkle_property.py
+tests/test_pytest_plugin.py
 tests/test_rekor_interop.py
 tests/test_rfc6962_external_vectors.py
 tests/test_schema.py

proofbundle-1.0.0/src/proofbundle.egg-info/entry_points.txt

@@ -0,0 +1,8 @@
+[console_scripts]
+proofbundle = proofbundle.cli:main
+
+[inspect_ai]
+proofbundle = proofbundle._inspect_registry
+
+[pytest11]
+proofbundle = proofbundle.pytest_plugin

{proofbundle-0.9.0 → proofbundle-1.0.0}/src/proofbundle.egg-info/requires.txt

@@ -13,7 +13,7 @@ rfc8785>=0.1.4
 sd-jwt>=0.10
 
 [dev:python_version >= "3.10"]
-inspect_ai<0.4,>=0.3.100
+inspect_ai<0.4,>=0.3.112
 
 [eval]
 rfc8785>=0.1.4
@@ -21,6 +21,9 @@ rfc8785>=0.1.4
 [inspect]
 
 [inspect:python_version >= "3.10"]
-inspect_ai<0.4,>=0.3.100
+inspect_ai<0.4,>=0.3.112
+
+[pytest]
+pytest>=7
 
 [sdjwt]

proofbundle-1.0.0/tests/test_inspect_hook.py

@@ -0,0 +1,57 @@
+"""inspect_ai hook (v1.0): opt-in safety + signed receipt from a real EvalLog (data.log)."""
+import asyncio
+import json
+import os
+import types
+import unittest
+from pathlib import Path
+from tempfile import TemporaryDirectory
+
+FX = Path(__file__).resolve().parent / "fixtures" / "inspect_logs" / "safety_refusal_demo.eval"
+
+
+class TestInspectHook(unittest.TestCase):
+    def setUp(self):
+        try:
+            from inspect_ai.log import read_eval_log  # noqa: F401
+        except ImportError:
+            self.skipTest("inspect_ai not installed (pip install proofbundle[inspect])")
+
+    def _data(self):
+        from inspect_ai.log import read_eval_log
+        log = read_eval_log(str(FX), header_only=True)
+        return types.SimpleNamespace(log=log, eval_id="demo", run_id="r", eval_set_id=None)
+
+    def test_opt_in_off_no_receipt(self):
+        from proofbundle.inspect_hook import ProofbundleHooks
+        with TemporaryDirectory() as d:
+            os.environ.pop("PROOFBUNDLE_EMIT", None)
+            os.environ["PROOFBUNDLE_OUT"] = d
+            asyncio.run(ProofbundleHooks().on_task_end(self._data()))
+            self.assertEqual(list(Path(d).glob("*.json")), [])
+
+    def test_opt_in_emits_verifiable_receipt(self):
+        from proofbundle import verify_bundle
+        from proofbundle.evalclaim import decode_eval_claim
+        from proofbundle.inspect_hook import ProofbundleHooks
+        with TemporaryDirectory() as d:
+            os.environ["PROOFBUNDLE_EMIT"] = "1"
+            os.environ["PROOFBUNDLE_OUT"] = d
+            os.environ["PROOFBUNDLE_THRESHOLD"] = "0"
+            asyncio.run(ProofbundleHooks().on_task_end(self._data()))
+            files = list(Path(d).glob("*.json"))
+            os.environ.pop("PROOFBUNDLE_EMIT", None)
+            self.assertEqual(len(files), 1)
+            b = json.loads(files[0].read_text())
+            self.assertTrue(verify_bundle(b).ok)
+            dec = decode_eval_claim(b)
+            self.assertEqual(dec["suite"], "safety_refusal_demo")
+            self.assertNotIn("mockllm", json.dumps(dec))        # model stays a salted commitment
+
+    def test_enabled_reflects_env(self):
+        from proofbundle.inspect_hook import ProofbundleHooks
+        os.environ["PROOFBUNDLE_EMIT"] = "1"
+        self.assertTrue(ProofbundleHooks().enabled())
+        os.environ["PROOFBUNDLE_EMIT"] = "0"
+        self.assertFalse(ProofbundleHooks().enabled())
+        os.environ.pop("PROOFBUNDLE_EMIT", None)

proofbundle-1.0.0/tests/test_pytest_plugin.py

@@ -0,0 +1,62 @@
+"""pytest plugin (v1.0): opt-in safety + signed receipt of the test run from terminalreporter.stats."""
+import json
+import types
+import unittest
+from pathlib import Path
+from tempfile import TemporaryDirectory
+
+from proofbundle import verify_bundle
+from proofbundle.evalclaim import decode_eval_claim
+from proofbundle.pytest_plugin import pytest_terminal_summary
+
+
+def _reporter(passed=3, failed=0, error=0):
+    stats = {}
+    if passed:
+        stats["passed"] = [object()] * passed
+    if failed:
+        stats["failed"] = [object()] * failed
+    if error:
+        stats["error"] = [object()] * error
+    return types.SimpleNamespace(stats=stats)
+
+
+def _config(flag, rootpath="myproj"):
+    return types.SimpleNamespace(getoption=lambda name, default=False: flag,
+                                 rootpath=types.SimpleNamespace(name=rootpath, __str__=lambda self: rootpath))
+
+
+class TestPytestPlugin(unittest.TestCase):
+    def test_opt_in_off_no_receipt(self):
+        with TemporaryDirectory() as d:
+            import os
+            os.environ.pop("PROOFBUNDLE_EMIT", None)
+            os.environ["PROOFBUNDLE_OUT"] = d
+            pytest_terminal_summary(_reporter(), 0, _config(False))
+            self.assertEqual([f for f in Path(d).iterdir()], [])
+
+    def test_opt_in_flag_emits_receipt(self):
+        with TemporaryDirectory() as d:
+            import os
+            os.environ["PROOFBUNDLE_OUT"] = d
+            os.environ["PROOFBUNDLE_THRESHOLD"] = "0.5"
+            os.environ.pop("PROOFBUNDLE_EMIT", None)
+            pytest_terminal_summary(_reporter(passed=3, failed=1), 1, _config(True))
+            files = list(Path(d).glob("*.json"))
+            self.assertEqual(len(files), 1)
+            b = json.loads(files[0].read_text())
+            self.assertTrue(verify_bundle(b).ok)
+            d2 = decode_eval_claim(b)
+            self.assertEqual(d2["suite"], "pytest")
+            self.assertEqual(d2["metric"], "pass_rate")
+            self.assertTrue(d2["passed"])                       # 3/4 = 0.75 >= 0.5
+            self.assertEqual(d2["provenance"]["n_failed"], 1)
+
+    def test_no_tests_no_receipt(self):
+        with TemporaryDirectory() as d:
+            import os
+            os.environ["PROOFBUNDLE_EMIT"] = "1"
+            os.environ["PROOFBUNDLE_OUT"] = d
+            pytest_terminal_summary(_reporter(passed=0), 5, _config(False))
+            self.assertEqual(list(Path(d).glob("*.json")), [])
+            os.environ.pop("PROOFBUNDLE_EMIT", None)

proofbundle-0.9.0/src/proofbundle/__init__.py

@@ -1,30 +0,0 @@
-"""proofbundle, an offline verifier for portable cryptographic evidence bundles.
-
-Verify, fully offline and in pure Python, that a payload was Ed25519 signed and
-anchored under an RFC 6962 Merkle root, with optional SD-JWT selective
-disclosure. The verification half of a signed, third-party-verifiable evidence
-receipt.
-"""
-
-from __future__ import annotations
-
-from .bundle import SCHEMA, load_bundle, verify_bundle
-from .emit import emit_bundle, generate_signer
-from .errors import Check, ProofBundleError, VerificationResult
-from .merkle import verify_consistency, verify_inclusion
-
-__version__ = "0.9.0"
-
-__all__ = [
-    "__version__",
-    "SCHEMA",
-    "verify_bundle",
-    "load_bundle",
-    "emit_bundle",
-    "generate_signer",
-    "verify_inclusion",
-    "verify_consistency",
-    "VerificationResult",
-    "Check",
-    "ProofBundleError",
-]

proofbundle-0.9.0/src/proofbundle.egg-info/entry_points.txt

@@ -1,2 +0,0 @@
-[console_scripts]
-proofbundle = proofbundle.cli:main