symbola 0.1.0__tar.gz

symbola-0.1.0/PKG-INFO

@@ -0,0 +1,115 @@
+Metadata-Version: 2.4
+Name: symbola
+Version: 0.1.0
+Summary: Spec packs for coding agents: checkable specs, verifiable evidence.
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://symbola.sh
+Project-URL: Documentation, https://kausalflow.github.io/symbola/
+Project-URL: Source, https://github.com/kausalflow/symbola
+Project-URL: Issues, https://github.com/kausalflow/symbola/issues
+Keywords: specification,contracts,coding-agents,conformance,attestation
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Requires-Dist: pydantic>=2.6
+Requires-Dist: hypothesis>=6.100
+Requires-Dist: typer>=0.12
+Requires-Dist: cryptography>=42
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Provides-Extra: docs
+Requires-Dist: zensical>=0.0.71; extra == "docs"
+
+# Symbolon
+
+Trustable contracts between agents. A contract is a precise, content-addressed promise; evidence that someone kept it can be checked by anyone. Software capabilities are the first vertical: a contract specifies a capability, and any implementation, hand-written or AI-generated, is checked against it.
+
+This repository covers **MVP steps 1 and 2**: the contract format with `init`, `validate`, and `test`, and signed attestations with `attest` and `verify`, run in a container or process sandbox.
+
+## Install
+
+```bash
+uv tool install git+https://github.com/kausalflow/symbola   # or: pipx install git+https://github.com/kausalflow/symbola
+symbola --help
+```
+
+The package and command are `symbola` (after [symbola.sh](https://symbola.sh)). `pip install symbolon` is an unrelated project.
+
+## Quick start
+
+From a clone of this repository:
+
+```bash
+pip install -e ".[dev]"
+
+symbola init acme/shout                      # scaffold a contract that already validates
+symbola validate contracts/acme/shout/versions/1      # check it and print its digest
+
+symbola validate contracts/core/date-parse/versions/1
+symbola test contracts/core/date-parse/versions/1 examples/impls/date-parse-strptime   # fails: too lenient
+symbola test contracts/core/date-parse/versions/1 examples/impls/date-parse-strict     # passes
+
+symbola keygen                                                                 # this runner's signing key
+symbola attest contracts/core/date-parse/versions/1 examples/impls/date-parse-strict -o att.json
+symbola verify att.json --trust ~/.config/symbolon/runner.pub \
+  --contract contracts/core/date-parse/versions/1 --impl examples/impls/date-parse-strict --rerun
+```
+
+## Commands
+
+| Command | What it does |
+| --- | --- |
+| `symbola init <namespace/name>` | Scaffold a contract that passes `validate` out of the box |
+| `symbola validate <dir>` | Check structure, schema drift, examples, reference, spec vacuity and mutants; print the digest |
+| `symbola schema <dir>` | Regenerate `schema/*.json` from `schema.py` |
+| `symbola test <dir> <impl.py>` | Run the contract's examples and properties against an implementation |
+| `symbola digest <dir>` | Print the contract's content digest |
+| `symbola impl-init <dir> --contract <dir>` | Write an implementation manifest pinned to the contract's digest |
+| `symbola keygen` | Create this runner's Ed25519 signing key |
+| `symbola attest <contract> <impl>` | Run the spec in a sandbox; write a signed in-toto attestation |
+| `symbola verify <attestation>` | Check signature, trust, digests; `--rerun` reproduces it locally |
+| `symbola diff <contract> <impl> <impl>...` | Run several builds on one input pool; report where they disagree, the ambiguity estimate, and the obligations each split traces to |
+| `symbola extract <source> --pack <ns/product>` | Have a coding agent (Claude Code by default) draft a spec pack from an existing product; check it against the original |
+| `symbola replicate <pack>` | Hand a pack to outside coding agents (Claude Code by default), then attest, holdout-test and diff the builds they make |
+| `symbola pack validate <dir>` | Check a spec pack's brief, assumptions and obligations; report obligations by status |
+| `symbola mutants <dir>` | Show which checks kill which mutants, and the contract's kill rate |
+| `symbola canary plant <dir> <out>` / `canary check` | Give a builder a copy with one contradictory example; catch builds fitted to the examples |
+| `symbola flag <ref> <reason>` | Report a contradiction in the spec instead of coding around it |
+
+## Seed contracts
+
+| Contract | Capability |
+| --- | --- |
+| `core/date-parse@1` | Parse a date written in ISO, en-US, en-GB, or de-DE conventions |
+| `core/csv-to-records@1`, `@2` | Parse RFC 4180 CSV with a header into records (v2 also rejects malformed quoting) |
+| `core/unit-convert@1` | Convert length, mass, and temperature with exact factors |
+
+The contract format is specified in [docs/CONTRACT_FORMAT.md](docs/CONTRACT_FORMAT.md), implementations and attestations in [docs/ATTESTATION.md](docs/ATTESTATION.md), and spec packs in [docs/PACK_FORMAT.md](docs/PACK_FORMAT.md).
+
+## Documentation
+
+The docs site is built with [Zensical](https://zensical.org) from `docs/` and published to GitHub Pages on every push to `main`. Each pull request gets a preview at `pr-preview/pr-<number>/`, linked in a comment on the PR.
+
+```bash
+pip install -e ".[docs]"
+zensical serve      # http://localhost:8000
+```
+
+## Tests
+
+```bash
+pytest
+```
+
+Sandbox tests run for each sandbox available on the machine: container tests need a Docker daemon, process tests need unprivileged user namespaces. Others are skipped.
+
+## Releasing
+
+Bump `__version__` in `src/symbola/__init__.py`, merge, then publish a GitHub release tagged `v<version>` (for example `v0.1.0`). The `Publish to PyPI` workflow builds the package and uploads it to PyPI as `symbola` through trusted publishing.

symbola-0.1.0/README.md

@@ -0,0 +1,86 @@
+# Symbolon
+
+Trustable contracts between agents. A contract is a precise, content-addressed promise; evidence that someone kept it can be checked by anyone. Software capabilities are the first vertical: a contract specifies a capability, and any implementation, hand-written or AI-generated, is checked against it.
+
+This repository covers **MVP steps 1 and 2**: the contract format with `init`, `validate`, and `test`, and signed attestations with `attest` and `verify`, run in a container or process sandbox.
+
+## Install
+
+```bash
+uv tool install git+https://github.com/kausalflow/symbola   # or: pipx install git+https://github.com/kausalflow/symbola
+symbola --help
+```
+
+The package and command are `symbola` (after [symbola.sh](https://symbola.sh)). `pip install symbolon` is an unrelated project.
+
+## Quick start
+
+From a clone of this repository:
+
+```bash
+pip install -e ".[dev]"
+
+symbola init acme/shout                      # scaffold a contract that already validates
+symbola validate contracts/acme/shout/versions/1      # check it and print its digest
+
+symbola validate contracts/core/date-parse/versions/1
+symbola test contracts/core/date-parse/versions/1 examples/impls/date-parse-strptime   # fails: too lenient
+symbola test contracts/core/date-parse/versions/1 examples/impls/date-parse-strict     # passes
+
+symbola keygen                                                                 # this runner's signing key
+symbola attest contracts/core/date-parse/versions/1 examples/impls/date-parse-strict -o att.json
+symbola verify att.json --trust ~/.config/symbolon/runner.pub \
+  --contract contracts/core/date-parse/versions/1 --impl examples/impls/date-parse-strict --rerun
+```
+
+## Commands
+
+| Command | What it does |
+| --- | --- |
+| `symbola init <namespace/name>` | Scaffold a contract that passes `validate` out of the box |
+| `symbola validate <dir>` | Check structure, schema drift, examples, reference, spec vacuity and mutants; print the digest |
+| `symbola schema <dir>` | Regenerate `schema/*.json` from `schema.py` |
+| `symbola test <dir> <impl.py>` | Run the contract's examples and properties against an implementation |
+| `symbola digest <dir>` | Print the contract's content digest |
+| `symbola impl-init <dir> --contract <dir>` | Write an implementation manifest pinned to the contract's digest |
+| `symbola keygen` | Create this runner's Ed25519 signing key |
+| `symbola attest <contract> <impl>` | Run the spec in a sandbox; write a signed in-toto attestation |
+| `symbola verify <attestation>` | Check signature, trust, digests; `--rerun` reproduces it locally |
+| `symbola diff <contract> <impl> <impl>...` | Run several builds on one input pool; report where they disagree, the ambiguity estimate, and the obligations each split traces to |
+| `symbola extract <source> --pack <ns/product>` | Have a coding agent (Claude Code by default) draft a spec pack from an existing product; check it against the original |
+| `symbola replicate <pack>` | Hand a pack to outside coding agents (Claude Code by default), then attest, holdout-test and diff the builds they make |
+| `symbola pack validate <dir>` | Check a spec pack's brief, assumptions and obligations; report obligations by status |
+| `symbola mutants <dir>` | Show which checks kill which mutants, and the contract's kill rate |
+| `symbola canary plant <dir> <out>` / `canary check` | Give a builder a copy with one contradictory example; catch builds fitted to the examples |
+| `symbola flag <ref> <reason>` | Report a contradiction in the spec instead of coding around it |
+
+## Seed contracts
+
+| Contract | Capability |
+| --- | --- |
+| `core/date-parse@1` | Parse a date written in ISO, en-US, en-GB, or de-DE conventions |
+| `core/csv-to-records@1`, `@2` | Parse RFC 4180 CSV with a header into records (v2 also rejects malformed quoting) |
+| `core/unit-convert@1` | Convert length, mass, and temperature with exact factors |
+
+The contract format is specified in [docs/CONTRACT_FORMAT.md](docs/CONTRACT_FORMAT.md), implementations and attestations in [docs/ATTESTATION.md](docs/ATTESTATION.md), and spec packs in [docs/PACK_FORMAT.md](docs/PACK_FORMAT.md).
+
+## Documentation
+
+The docs site is built with [Zensical](https://zensical.org) from `docs/` and published to GitHub Pages on every push to `main`. Each pull request gets a preview at `pr-preview/pr-<number>/`, linked in a comment on the PR.
+
+```bash
+pip install -e ".[docs]"
+zensical serve      # http://localhost:8000
+```
+
+## Tests
+
+```bash
+pytest
+```
+
+Sandbox tests run for each sandbox available on the machine: container tests need a Docker daemon, process tests need unprivileged user namespaces. Others are skipped.
+
+## Releasing
+
+Bump `__version__` in `src/symbola/__init__.py`, merge, then publish a GitHub release tagged `v<version>` (for example `v0.1.0`). The `Publish to PyPI` workflow builds the package and uploads it to PyPI as `symbola` through trusted publishing.

symbola-0.1.0/pyproject.toml

@@ -0,0 +1,52 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+# The PyPI name "symbolon" belongs to an unrelated project, so the package,
+# module and command are "symbola" (after symbola.sh).
+name = "symbola"
+dynamic = ["version"]
+description = "Spec packs for coding agents: checkable specs, verifiable evidence."
+readme = "README.md"
+requires-python = ">=3.11"
+license = "Apache-2.0"
+keywords = ["specification", "contracts", "coding-agents", "conformance", "attestation"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Environment :: Console",
+    "Intended Audience :: Developers",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Software Development :: Testing",
+]
+dependencies = [
+    "pydantic>=2.6",
+    "hypothesis>=6.100",
+    "typer>=0.12",
+    "cryptography>=42",
+]
+
+[project.optional-dependencies]
+dev = ["pytest>=8"]
+docs = ["zensical>=0.0.71"]
+
+[project.urls]
+Homepage = "https://symbola.sh"
+Documentation = "https://kausalflow.github.io/symbola/"
+Source = "https://github.com/kausalflow/symbola"
+Issues = "https://github.com/kausalflow/symbola/issues"
+
+[project.scripts]
+symbola = "symbola.cli:app"
+
+[tool.setuptools.dynamic]
+version = { attr = "symbola.__version__" }
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

symbola-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

symbola-0.1.0/src/symbola/__init__.py

@@ -0,0 +1,3 @@
+"""Symbolon: trustable contracts between agents."""
+
+__version__ = "0.1.0"

symbola-0.1.0/src/symbola/agent.py

@@ -0,0 +1,132 @@
+"""Hand work to an external coding agent.
+
+Symbolon does not build software or read code itself. Commands that need that,
+`extract` and `replicate`, run an outside coding agent (Claude Code by default)
+as a subprocess and check what it leaves behind.
+
+An agent is any command that takes a prompt on stdin, works in its current
+directory, and exits. The prompt is also written to a file, for agents that
+would rather read one. A command may name these placeholders, which are
+substituted after splitting:
+
+    {prompt_file}   the prompt file
+    {workdir}       the directory the agent works in
+
+and it gets the same values as SYMBOLON_PROMPT_FILE and SYMBOLON_WORKDIR in its
+environment. Directories the agent must read outside its workdir are passed to
+Claude Code as --add-dir, and to any agent as SYMBOLON_EXTRA_DIRS.
+"""
+
+from __future__ import annotations
+
+import os
+import shlex
+import subprocess
+import time
+from dataclasses import dataclass, field
+from pathlib import Path
+
+AGENT_ENV = "SYMBOLON_AGENT"
+DEFAULT_AGENT = "claude -p --permission-mode acceptEdits --allowedTools Read,Write,Edit,Glob,Grep,Bash"
+PROMPT_DIR = ".symbolon"
+PROMPT_FILE = "prompt.md"
+
+
+class AgentError(Exception):
+    pass
+
+
+@dataclass(frozen=True)
+class AgentCommand:
+    command: str
+    name: str = ""
+
+    def __post_init__(self) -> None:
+        if not self.argv:
+            raise AgentError("the agent command is empty")
+        if not self.name:
+            object.__setattr__(self, "name", Path(self.argv[0]).name)
+
+    @property
+    def argv(self) -> list[str]:
+        return shlex.split(self.command)
+
+    @property
+    def is_claude_code(self) -> bool:
+        return Path(self.argv[0]).name == "claude"
+
+
+def resolve_agent(option: str | None = None) -> AgentCommand:
+    """The agent to run: the --agent option, else $SYMBOLON_AGENT, else Claude Code."""
+    return AgentCommand(option or os.environ.get(AGENT_ENV) or DEFAULT_AGENT)
+
+
+def parse_agents(options: list[str] | None) -> list[AgentCommand]:
+    """Agents given as repeated `--agent NAME=COMMAND` (or a bare COMMAND); default as resolve_agent."""
+    if not options:
+        return [resolve_agent()]
+    agents = []
+    for opt in options:
+        name, sep, command = opt.partition("=")
+        # A bare command can contain '=' in its flags; only a plain word before it is a name.
+        if sep and name and " " not in name and "-" != name[0]:
+            agents.append(AgentCommand(command, name))
+        else:
+            agents.append(AgentCommand(opt))
+    names = [a.name for a in agents]
+    dupes = sorted({n for n in names if names.count(n) > 1})
+    if dupes:
+        raise AgentError(f"agent names must be unique; give them as NAME=COMMAND ({', '.join(dupes)})")
+    return agents
+
+
+@dataclass
+class AgentRun:
+    returncode: int
+    stdout: str
+    stderr: str
+    seconds: float
+    command: list[str] = field(default_factory=list)
+
+    @property
+    def ok(self) -> bool:
+        return self.returncode == 0
+
+    def log(self) -> str:
+        return (f"$ {shlex.join(self.command)}\n# exit {self.returncode} after {self.seconds:.1f}s\n"
+                f"\n## stdout\n{self.stdout}\n## stderr\n{self.stderr}")
+
+
+def run_agent(agent: AgentCommand, prompt: str, workdir: Path, *, timeout: int = 3600,
+              prompt_file: Path | None = None, extra_dirs: list[Path] = ()) -> AgentRun:
+    """Run the agent in workdir with the prompt on stdin and wait for it to exit."""
+    workdir = Path(workdir).resolve()
+    workdir.mkdir(parents=True, exist_ok=True)
+    prompt_file = Path(prompt_file).resolve() if prompt_file else workdir / PROMPT_DIR / PROMPT_FILE
+    prompt_file.parent.mkdir(parents=True, exist_ok=True)
+    prompt_file.write_text(prompt)
+    values = {"prompt_file": str(prompt_file), "workdir": str(workdir)}
+    argv = [_substitute(arg, values) for arg in agent.argv]
+    if "/" in argv[0] and not Path(argv[0]).is_absolute():
+        argv[0] = str(Path(argv[0]).resolve())  # ./my-agent.sh means here, not in workdir
+    dirs = [str(Path(d).resolve()) for d in extra_dirs]
+    if agent.is_claude_code:
+        for d in dirs:
+            argv += ["--add-dir", d]
+    env = {**os.environ, "SYMBOLON_PROMPT_FILE": values["prompt_file"],
+           "SYMBOLON_WORKDIR": values["workdir"], "SYMBOLON_EXTRA_DIRS": os.pathsep.join(dirs)}
+    start = time.monotonic()
+    try:
+        proc = subprocess.run(argv, input=prompt, capture_output=True, text=True,
+                              cwd=workdir, env=env, timeout=timeout)
+    except FileNotFoundError:
+        raise AgentError(f"agent command not found: {argv[0]} (set --agent or ${AGENT_ENV})") from None
+    except subprocess.TimeoutExpired as e:
+        raise AgentError(f"agent {agent.name} timed out after {timeout}s") from e
+    return AgentRun(proc.returncode, proc.stdout, proc.stderr, time.monotonic() - start, argv)
+
+
+def _substitute(arg: str, values: dict[str, str]) -> str:
+    for key, value in values.items():
+        arg = arg.replace("{" + key + "}", value)
+    return arg

symbola-0.1.0/src/symbola/attest.py

@@ -0,0 +1,192 @@
+"""Signed attestations: an in-toto Test Result statement in a DSSE envelope.
+
+The statement's subject is the implementation digest. The contract digest
+and the environment digest are configuration entries. Symbolon-specific
+details (seed, budget, failures, resources, sandbox, runner key) live under
+`predicate.symbolon`, which in-toto consumers that only know the Test Result
+predicate can ignore. See docs/ATTESTATION.md.
+"""
+
+from __future__ import annotations
+
+import base64
+import json
+from dataclasses import dataclass, field
+from datetime import datetime, timezone
+from pathlib import Path
+
+import symbola
+
+from . import keys
+from .digest import contract_digest, hex_of, implementation_digest
+from .implementation import check_lockfile, load_impl_manifest
+from .manifest import load_manifest
+from .sandbox import Limits, Sandbox, Workspace, environment_digest
+
+STATEMENT_TYPE = "https://in-toto.io/Statement/v1"
+PREDICATE_TYPE = "https://in-toto.io/attestation/test-result/v0.1"
+
+
+class AttestError(Exception):
+    pass
+
+
+def _now() -> str:
+    return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
+
+
+def _test_names(result: dict) -> tuple[list[str], list[str], dict[str, str]]:
+    passed, failed, details = [], [], {}
+    for kind, items in (("example", result["examples"]), ("property", result["properties"])):
+        for c in items:
+            name = f"{kind}:{c['name'].removeprefix('example ')}"
+            (passed if c["ok"] else failed).append(name)
+            if not c["ok"]:
+                details[name] = c["detail"]
+    return passed, failed, details
+
+
+def run_attestation(contract_dir: Path, impl_dir: Path, sandbox: Sandbox, key) -> dict:
+    """Run the contract's spec against an implementation in `sandbox`; return a signed envelope."""
+    contract_dir, impl_dir = Path(contract_dir).resolve(), Path(impl_dir).resolve()
+    manifest = load_manifest(contract_dir)
+    try:
+        impl_manifest = load_impl_manifest(impl_dir)
+        check_lockfile(impl_dir)
+    except Exception as e:
+        raise AttestError(f"implementation manifest: {e}") from None
+
+    ws = Workspace.create(contract_dir, impl_dir)
+    try:
+        c_digest = contract_digest(ws.contract)
+        i_digest = implementation_digest(ws.impl)
+        if impl_manifest.implementation.contract != c_digest:
+            raise AttestError(
+                f"implementation targets {impl_manifest.implementation.contract}, "
+                f"but this contract is {c_digest}"
+            )
+        section = impl_manifest.implementation
+        if section.language == "python" and not (ws.impl / section.entrypoint).is_file():
+            raise AttestError(f"entrypoint {section.entrypoint} not found in the implementation")
+
+        network = bool(manifest.requires.network)
+        env = sandbox.environment()
+        started = _now()
+        result = sandbox.run(
+            ws, network=network, lockfile=(ws.impl / "requirements.lock").exists(),
+            limits=Limits(timeout_seconds=manifest.spec.timeout_seconds),
+        )
+        finished = _now()
+    finally:
+        ws.cleanup()
+
+    if "error" in result:
+        raise AttestError(f"implementation could not be loaded: {result['error']}")
+
+    passed, failed, details = _test_names(result)
+    public = key.public_key()
+    statement = {
+        "_type": STATEMENT_TYPE,
+        "subject": [{
+            "name": f"implementation-of:{manifest.alias}",
+            "digest": {"sha256": hex_of(i_digest)},
+        }],
+        "predicateType": PREDICATE_TYPE,
+        "predicate": {
+            "result": "PASSED" if result["passed"] else "FAILED",
+            "configuration": [
+                {"name": f"contract:{manifest.alias}", "digest": {"sha256": hex_of(c_digest)}},
+                {"name": "environment", "digest": {"sha256": hex_of(environment_digest(env))}},
+            ],
+            "passedTests": passed,
+            "warnedTests": [],
+            "failedTests": failed,
+            "symbolon": {
+                "version": symbola.__version__,
+                "evidence": "observed" if network else "reproducible",
+                "spec": {"seed": manifest.spec.seed, "max_examples": manifest.spec.max_examples},
+                "failures": details,
+                "resources": result.get("resources", {}),
+                "sandbox": {**sandbox.isolation(network), "environment": env},
+                "implementation": impl_manifest.model_dump(exclude_none=True),
+                "runner": {
+                    "keyid": keys.keyid(public),
+                    "publicKey": base64.b64encode(keys.raw_public(public)).decode(),
+                },
+                "startedOn": started,
+                "finishedOn": finished,
+            },
+        },
+    }
+    payload = json.dumps(statement, sort_keys=True, separators=(",", ":")).encode()
+    return keys.sign_envelope(payload, key)
+
+
+@dataclass
+class Verification:
+    statement: dict
+    keyid: str
+    trusted: bool | None
+    problems: list[str] = field(default_factory=list)
+
+    @property
+    def ok(self) -> bool:
+        return not self.problems and self.trusted is not False
+
+    @property
+    def predicate(self) -> dict:
+        return self.statement["predicate"]
+
+
+def verify_attestation(envelope: dict, *, trusted_keys: list[Path] = (),
+                       contract_dir: Path | None = None, impl_dir: Path | None = None) -> Verification:
+    """Check the signature, and optionally the trust set and digests against local copies."""
+    try:
+        claimed = json.loads(base64.b64decode(envelope["payload"]))
+        runner = claimed["predicate"]["symbolon"]["runner"]
+        public = keys.public_from_b64(runner["publicKey"])
+        payload = keys.verify_envelope(envelope, public)
+    except Exception as e:
+        raise AttestError(f"not a valid Symbolon attestation: {e}") from None
+    statement = json.loads(payload)
+    kid = keys.keyid(public)
+    problems = []
+    if runner["keyid"] != kid:
+        problems.append("runner keyid does not match the runner public key")
+    if statement.get("_type") != STATEMENT_TYPE or statement.get("predicateType") != PREDICATE_TYPE:
+        problems.append("unexpected statement or predicate type")
+
+    trusted = None
+    if trusted_keys:
+        trusted = kid in {keys.keyid(keys.load_public(p)) for p in trusted_keys}
+
+    config = {c["name"].split(":", 1)[0]: c for c in statement["predicate"]["configuration"]}
+    if contract_dir is not None:
+        actual = hex_of(contract_digest(contract_dir))
+        if config["contract"]["digest"]["sha256"] != actual:
+            problems.append(f"contract digest differs: attested {config['contract']['digest']['sha256'][:12]}…, local {actual[:12]}…")
+    if impl_dir is not None:
+        actual = hex_of(implementation_digest(impl_dir))
+        if statement["subject"][0]["digest"]["sha256"] != actual:
+            problems.append(f"implementation digest differs: attested {statement['subject'][0]['digest']['sha256'][:12]}…, local {actual[:12]}…")
+    return Verification(statement, kid, trusted, problems)
+
+
+def compare_runs(original: dict, rerun: dict) -> list[str]:
+    """Differences in per-test outcomes between two statements' predicates."""
+    a, b = original["predicate"], rerun["predicate"]
+    diffs = []
+    if a["result"] != b["result"]:
+        diffs.append(f"result {a['result']} vs {b['result']}")
+    for label, key in (("passed", "passedTests"), ("failed", "failedTests")):
+        only_a = sorted(set(a[key]) - set(b[key]))
+        only_b = sorted(set(b[key]) - set(a[key]))
+        if only_a:
+            diffs.append(f"{label} only originally: {', '.join(only_a)}")
+        if only_b:
+            diffs.append(f"{label} only on rerun: {', '.join(only_b)}")
+    return diffs
+
+
+def statement_of(envelope: dict) -> dict:
+    return json.loads(base64.b64decode(envelope["payload"]))