verdictkit 0.1.0__tar.gz

verdictkit-0.1.0/PKG-INFO

@@ -0,0 +1,191 @@
+Metadata-Version: 2.4
+Name: verdictkit
+Version: 0.1.0
+Summary: Non-custodial verification for agent work: did the agent do what it said?
+Author: agentbuilt
+License: MIT
+Project-URL: Homepage, https://github.com/mattedwardseo/verdictkit
+Project-URL: Documentation, https://github.com/mattedwardseo/verdictkit#readme
+Project-URL: Repository, https://github.com/mattedwardseo/verdictkit
+Project-URL: Issues, https://github.com/mattedwardseo/verdictkit/issues
+Keywords: agents,verification,ai-safety,auditing,deterministic-checks
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Testing
+Classifier: Topic :: Security
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: requests>=2.28
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Requires-Dist: build; extra == "dev"
+
+# verdictkit
+
+Non-custodial verification for agent work. Answers one question: **did the
+agent do what it said it would?**
+
+The product is the *verification layer*, not custody: a machine-readable
+contract (intended action + checkable success criteria), deterministic
+checks against the world, and a rubric verdict — `PASS` / `FLAG` / `FAIL`
+— with evidence. No money moves through it, so there is no money-transmitter
+licensing wall. That is the whole strategic point.
+
+**Status: 0.1.0, launch-ready, NOT published to PyPI.** Do not run the
+publish command without approval.
+
+## Quickstart
+
+```bash
+pip install verdictkit        # not on PyPI yet; use: pip install .
+```
+
+### Local verification (free, unlimited, no key, no network)
+
+```python
+import verdictkit as vk
+
+contract = vk.define_contract(
+    task_id="research-042",
+    agent_id="agent-7f3a",
+    intended_action="Summarize the Q3 incident reports into incidents.md",
+    success_criteria=[
+        {"id": "c1", "type": "file_exists",
+         "path": "incidents.md",
+         "description": "summary file was written"},
+        {"id": "c2", "type": "pattern_count_gte",
+         "path": "incidents.md",
+         "pattern": r"^## Incident",
+         "min_count": 3,
+         "description": "covers at least 3 incidents"},
+    ],
+    unverifiable_claims=["tone is executive-appropriate"],  # -> FLAG, honestly
+)
+
+result = vk.verify_run(contract, base_path="./output")
+print(result["verdict"])   # PASS | FLAG | FAIL
+print(result["reason"])    # evidence, always
+```
+
+Or from the shell:
+
+```bash
+verdictkit verify contract.json --base-path ./output -v
+# exit code: 0 = PASS, 1 = FAIL, 2 = FLAG, 3 = usage/IO error
+```
+
+### Hosted verification (reputation ledger + re-verification)
+
+Local mode checks *your* machine. The hosted tier lets a third party trust
+the verdict: you capture evidence locally, the server **re-runs the
+deterministic checks against the evidence** — it never trusts the agent's
+claims alone.
+
+```python
+import verdictkit as vk
+
+client = vk.Client(api_key="vk_live_...")          # base_url=... to override
+contract_id = client.create_contract(contract)      # -> "vc_..."
+bundle = vk.collect_evidence(contract, base_path="./output")
+verdict = client.submit_run(contract_id, bundle)    # server-side verdict dict
+
+client.get_verdict(contract_id)                     # latest verdict
+client.list_verdicts(agent_id="agent-7f3a")         # reputation ledger
+```
+
+Errors are clean: `vk.AuthError` on bad credentials (401/403),
+`vk.VerdictkitError` on network trouble, timeouts, or bad responses.
+
+## API reference
+
+| Symbol | Kind | Description |
+|---|---|---|
+| `vk.define_contract(task_id, agent_id, intended_action, success_criteria, scope=None, unverifiable_claims=None, principal="unknown")` | function | Build a validated contract dict. Raises `ValueError` on unknown check types or criteria missing `id`/`type`. |
+| `vk.verify_run(contract, base_path=None)` | function | Run all checks locally, return verdict dict. No key, no network. `base_path` resolves relative criterion paths. |
+| `vk.Verdict` | class | Constants `PASS`, `FLAG`, `FAIL`. |
+| `vk.collect_evidence(contract, base_path=None)` | function | Capture a content-addressed evidence bundle (file bytes + sha256, dir listings, parsed JSON, sqlite row counts). Local only, no network. |
+| `vk.verify_bundle_hash(bundle)` | function | Recompute `bundle_hash`; `False` means the bundle was tampered with. |
+| `vk.Client(api_key, base_url=..., timeout=30.0)` | class | Hosted API client. `create_contract(contract) -> contract_id`, `submit_run(contract_id, evidence_bundle) -> verdict dict`, `get_verdict(contract_id) -> dict`, `list_verdicts(agent_id=None, limit=100) -> list`. Raises `ValueError` without an API key. |
+| `vk.VerdictkitError` | exception | Base error: network, timeout, bad server response. |
+| `vk.AuthError(VerdictkitError)` | exception | 401/403 from the API. |
+
+### Verdict shape
+
+```python
+{"contract_id": ..., "task_id": ..., "agent_id": ...,
+ "verdict": "PASS" | "FLAG" | "FAIL",
+ "reason": "human-readable evidence summary",
+ "check_results": [{"check_id": ..., "type": ..., "status": "pass"|"fail"|"error",
+                    "evidence": ...}, ...],
+ "unverifiable_claims": [...]}
+```
+
+### Check types
+
+`file_exists` · `file_nonempty` · `file_contains` (regex) ·
+`pattern_count_gte` · `dir_exists` · `json_valid` · `line_count_gte` ·
+`sqlite_table_has_rows` · `path_absent` (scope guard)
+
+## The rubric
+
+- **FAIL** — any check fails. Fail dominates everything else.
+- **FLAG** — no failures, but something couldn't be verified: a check
+  errored, a claim was declared unverifiable, or there was nothing
+  checkable at all.
+- **PASS** — every check passed cleanly, and nothing was left unverifiable.
+
+Design rule: a check that cannot run is an *error*, never a pass. A run
+with nothing checkable can at best be *flagged*. Absence of evidence is
+never evidence of success.
+
+## Trust model
+
+1. **Contracts are machine-readable promises.** What the agent intended to
+   do, written before the run, with deterministic success criteria.
+2. **Evidence is content-addressed.** `collect_evidence` captures file
+   bytes with their sha256, directory listings, and query results. The
+   bundle carries a `bundle_hash` over canonical JSON — the server
+   recomputes every hash; tampering breaks them.
+3. **The server re-runs checks; it never trusts claims.** Hosted
+   verification executes the same deterministic check engine against the
+   bundle's content. Claims without evidence stay in
+   `unverifiable_claims` and can only produce FLAG.
+4. **Truncation is honest.** Artifacts are capped (256 KiB file content,
+   2000 dir entries). Truncated artifacts are marked, and checks the
+   server cannot fully re-run become FLAG — never PASS.
+
+## What verification is NOT
+
+- **Not a guarantee of quality.** PASS means the deterministic criteria
+  were met — the file exists, has ≥3 sections, parses as JSON. It says
+  nothing about whether the writing is good, the code is correct, or the
+  summary is accurate. Put anything subjective in `unverifiable_claims`
+  and accept the FLAG honestly.
+- **Not an oracle.** Checks only see what they can read: files, dirs,
+  sqlite. They cannot verify "the user is happy" or "the bug is fixed"
+  unless that was operationalized into a checkable artifact.
+- **Not custody, not escrow.** No money moves through verdictkit; it
+  attests to work done, it does not hold funds or release payment.
+- **Not tamper-proof on a compromised machine.** Local verification trusts
+  the local filesystem. The hosted tier raises the bar (content-addressed
+  bundles, server-side re-runs, reputation ledger) but a fully
+  compromised agent host can still fake the inputs. Verification makes
+  lying *auditable*, not impossible.
+
+## Development
+
+```bash
+pip install -e ".[dev]"   # dev extras: pytest, build
+python3 -m pytest         # 59 tests, all local, no network
+python3 -m build          # wheel/sdist -- DO NOT upload without approval
+```
+
+## License
+
+MIT. See `LICENSE`.

verdictkit-0.1.0/README.md

@@ -0,0 +1,163 @@
+# verdictkit
+
+Non-custodial verification for agent work. Answers one question: **did the
+agent do what it said it would?**
+
+The product is the *verification layer*, not custody: a machine-readable
+contract (intended action + checkable success criteria), deterministic
+checks against the world, and a rubric verdict — `PASS` / `FLAG` / `FAIL`
+— with evidence. No money moves through it, so there is no money-transmitter
+licensing wall. That is the whole strategic point.
+
+**Status: 0.1.0, launch-ready, NOT published to PyPI.** Do not run the
+publish command without approval.
+
+## Quickstart
+
+```bash
+pip install verdictkit        # not on PyPI yet; use: pip install .
+```
+
+### Local verification (free, unlimited, no key, no network)
+
+```python
+import verdictkit as vk
+
+contract = vk.define_contract(
+    task_id="research-042",
+    agent_id="agent-7f3a",
+    intended_action="Summarize the Q3 incident reports into incidents.md",
+    success_criteria=[
+        {"id": "c1", "type": "file_exists",
+         "path": "incidents.md",
+         "description": "summary file was written"},
+        {"id": "c2", "type": "pattern_count_gte",
+         "path": "incidents.md",
+         "pattern": r"^## Incident",
+         "min_count": 3,
+         "description": "covers at least 3 incidents"},
+    ],
+    unverifiable_claims=["tone is executive-appropriate"],  # -> FLAG, honestly
+)
+
+result = vk.verify_run(contract, base_path="./output")
+print(result["verdict"])   # PASS | FLAG | FAIL
+print(result["reason"])    # evidence, always
+```
+
+Or from the shell:
+
+```bash
+verdictkit verify contract.json --base-path ./output -v
+# exit code: 0 = PASS, 1 = FAIL, 2 = FLAG, 3 = usage/IO error
+```
+
+### Hosted verification (reputation ledger + re-verification)
+
+Local mode checks *your* machine. The hosted tier lets a third party trust
+the verdict: you capture evidence locally, the server **re-runs the
+deterministic checks against the evidence** — it never trusts the agent's
+claims alone.
+
+```python
+import verdictkit as vk
+
+client = vk.Client(api_key="vk_live_...")          # base_url=... to override
+contract_id = client.create_contract(contract)      # -> "vc_..."
+bundle = vk.collect_evidence(contract, base_path="./output")
+verdict = client.submit_run(contract_id, bundle)    # server-side verdict dict
+
+client.get_verdict(contract_id)                     # latest verdict
+client.list_verdicts(agent_id="agent-7f3a")         # reputation ledger
+```
+
+Errors are clean: `vk.AuthError` on bad credentials (401/403),
+`vk.VerdictkitError` on network trouble, timeouts, or bad responses.
+
+## API reference
+
+| Symbol | Kind | Description |
+|---|---|---|
+| `vk.define_contract(task_id, agent_id, intended_action, success_criteria, scope=None, unverifiable_claims=None, principal="unknown")` | function | Build a validated contract dict. Raises `ValueError` on unknown check types or criteria missing `id`/`type`. |
+| `vk.verify_run(contract, base_path=None)` | function | Run all checks locally, return verdict dict. No key, no network. `base_path` resolves relative criterion paths. |
+| `vk.Verdict` | class | Constants `PASS`, `FLAG`, `FAIL`. |
+| `vk.collect_evidence(contract, base_path=None)` | function | Capture a content-addressed evidence bundle (file bytes + sha256, dir listings, parsed JSON, sqlite row counts). Local only, no network. |
+| `vk.verify_bundle_hash(bundle)` | function | Recompute `bundle_hash`; `False` means the bundle was tampered with. |
+| `vk.Client(api_key, base_url=..., timeout=30.0)` | class | Hosted API client. `create_contract(contract) -> contract_id`, `submit_run(contract_id, evidence_bundle) -> verdict dict`, `get_verdict(contract_id) -> dict`, `list_verdicts(agent_id=None, limit=100) -> list`. Raises `ValueError` without an API key. |
+| `vk.VerdictkitError` | exception | Base error: network, timeout, bad server response. |
+| `vk.AuthError(VerdictkitError)` | exception | 401/403 from the API. |
+
+### Verdict shape
+
+```python
+{"contract_id": ..., "task_id": ..., "agent_id": ...,
+ "verdict": "PASS" | "FLAG" | "FAIL",
+ "reason": "human-readable evidence summary",
+ "check_results": [{"check_id": ..., "type": ..., "status": "pass"|"fail"|"error",
+                    "evidence": ...}, ...],
+ "unverifiable_claims": [...]}
+```
+
+### Check types
+
+`file_exists` · `file_nonempty` · `file_contains` (regex) ·
+`pattern_count_gte` · `dir_exists` · `json_valid` · `line_count_gte` ·
+`sqlite_table_has_rows` · `path_absent` (scope guard)
+
+## The rubric
+
+- **FAIL** — any check fails. Fail dominates everything else.
+- **FLAG** — no failures, but something couldn't be verified: a check
+  errored, a claim was declared unverifiable, or there was nothing
+  checkable at all.
+- **PASS** — every check passed cleanly, and nothing was left unverifiable.
+
+Design rule: a check that cannot run is an *error*, never a pass. A run
+with nothing checkable can at best be *flagged*. Absence of evidence is
+never evidence of success.
+
+## Trust model
+
+1. **Contracts are machine-readable promises.** What the agent intended to
+   do, written before the run, with deterministic success criteria.
+2. **Evidence is content-addressed.** `collect_evidence` captures file
+   bytes with their sha256, directory listings, and query results. The
+   bundle carries a `bundle_hash` over canonical JSON — the server
+   recomputes every hash; tampering breaks them.
+3. **The server re-runs checks; it never trusts claims.** Hosted
+   verification executes the same deterministic check engine against the
+   bundle's content. Claims without evidence stay in
+   `unverifiable_claims` and can only produce FLAG.
+4. **Truncation is honest.** Artifacts are capped (256 KiB file content,
+   2000 dir entries). Truncated artifacts are marked, and checks the
+   server cannot fully re-run become FLAG — never PASS.
+
+## What verification is NOT
+
+- **Not a guarantee of quality.** PASS means the deterministic criteria
+  were met — the file exists, has ≥3 sections, parses as JSON. It says
+  nothing about whether the writing is good, the code is correct, or the
+  summary is accurate. Put anything subjective in `unverifiable_claims`
+  and accept the FLAG honestly.
+- **Not an oracle.** Checks only see what they can read: files, dirs,
+  sqlite. They cannot verify "the user is happy" or "the bug is fixed"
+  unless that was operationalized into a checkable artifact.
+- **Not custody, not escrow.** No money moves through verdictkit; it
+  attests to work done, it does not hold funds or release payment.
+- **Not tamper-proof on a compromised machine.** Local verification trusts
+  the local filesystem. The hosted tier raises the bar (content-addressed
+  bundles, server-side re-runs, reputation ledger) but a fully
+  compromised agent host can still fake the inputs. Verification makes
+  lying *auditable*, not impossible.
+
+## Development
+
+```bash
+pip install -e ".[dev]"   # dev extras: pytest, build
+python3 -m pytest         # 59 tests, all local, no network
+python3 -m build          # wheel/sdist -- DO NOT upload without approval
+```
+
+## License
+
+MIT. See `LICENSE`.

verdictkit-0.1.0/pyproject.toml

@@ -0,0 +1,47 @@
+[build-system]
+requires = ["setuptools>=61"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "verdictkit"
+version = "0.1.0"
+description = "Non-custodial verification for agent work: did the agent do what it said?"
+readme = "README.md"
+requires-python = ">=3.10"
+license = { text = "MIT" }
+authors = [{ name = "agentbuilt" }]
+keywords = ["agents", "verification", "ai-safety", "auditing", "deterministic-checks"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Software Development :: Testing",
+    "Topic :: Security",
+]
+dependencies = [
+    "requests>=2.28",
+]
+
+[project.optional-dependencies]
+dev = ["pytest>=7", "build"]
+# PLACEHOLDER URLs -- replace with the real repo/docs once they exist.
+[project.urls]
+Homepage = "https://github.com/mattedwardseo/verdictkit"
+Documentation = "https://github.com/mattedwardseo/verdictkit#readme"
+Repository = "https://github.com/mattedwardseo/verdictkit"
+Issues = "https://github.com/mattedwardseo/verdictkit/issues"
+
+[project.scripts]
+verdictkit = "verdictkit.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["verdictkit*"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

verdictkit-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

verdictkit-0.1.0/tests/test_checks.py

@@ -0,0 +1,157 @@
+"""All 9 check types: pass, fail, and error paths."""
+import json
+import os
+import sqlite3
+
+import pytest
+
+from verdictkit.checks import run_check
+
+
+def crit(cid, ctype, **kw):
+    return {"id": cid, "type": ctype, **kw}
+
+
+# -- file_exists -----------------------------------------------------------
+def test_file_exists_pass_fail(tmp_path):
+    f = tmp_path / "a.txt"
+    f.write_text("hi")
+    assert run_check(crit("c", "file_exists", path=str(f)))["status"] == "pass"
+    assert run_check(crit("c", "file_exists", path=str(tmp_path / "nope")))["status"] == "fail"
+
+
+# -- file_nonempty ---------------------------------------------------------
+def test_file_nonempty_pass_fail(tmp_path):
+    full = tmp_path / "full.txt"
+    full.write_text("x")
+    assert run_check(crit("c", "file_nonempty", path=str(full)))["status"] == "pass"
+    empty = tmp_path / "empty.txt"
+    empty.write_text("")
+    assert run_check(crit("c", "file_nonempty", path=str(empty)))["status"] == "fail"
+    assert run_check(crit("c", "file_nonempty", path=str(tmp_path / "nope")))["status"] == "fail"
+
+
+# -- file_contains ---------------------------------------------------------
+def test_file_contains_pass_fail_and_bad_regex(tmp_path):
+    f = tmp_path / "doc.md"
+    f.write_text("## Incident one\nbody\n")
+    assert run_check(crit("c", "file_contains", path=str(f),
+                          pattern=r"^## Incident"))["status"] == "pass"
+    assert run_check(crit("c", "file_contains", path=str(f),
+                          pattern=r"nope"))["status"] == "fail"
+    r = run_check(crit("c", "file_contains", path=str(f), pattern=r"([unclosed"))
+    assert r["status"] == "error" and "bad regex" in r["evidence"]
+    r = run_check(crit("c", "file_contains", path=str(tmp_path / "nope"),
+                       pattern=r"x"))
+    assert r["status"] == "fail"  # unreadable file is fail, not pass
+
+
+# -- dir_exists ------------------------------------------------------------
+def test_dir_exists_pass_fail(tmp_path):
+    d = tmp_path / "sub"
+    d.mkdir()
+    assert run_check(crit("c", "dir_exists", path=str(d)))["status"] == "pass"
+    assert run_check(crit("c", "dir_exists", path=str(tmp_path / "nope")))["status"] == "fail"
+    f = tmp_path / "file.txt"
+    f.write_text("x")
+    assert run_check(crit("c", "dir_exists", path=str(f)))["status"] == "fail"  # file != dir
+
+
+# -- json_valid ------------------------------------------------------------
+def test_json_valid_pass_fail(tmp_path):
+    good = tmp_path / "good.json"
+    good.write_text(json.dumps({"a": 1}))
+    assert run_check(crit("c", "json_valid", path=str(good)))["status"] == "pass"
+    bad = tmp_path / "bad.json"
+    bad.write_text("{not json")
+    assert run_check(crit("c", "json_valid", path=str(bad)))["status"] == "fail"
+    assert run_check(crit("c", "json_valid", path=str(tmp_path / "nope")))["status"] == "fail"
+
+
+# -- line_count_gte --------------------------------------------------------
+def test_line_count_gte_pass_fail(tmp_path):
+    f = tmp_path / "lines.txt"
+    f.write_text("one\ntwo\nthree\n")
+    assert run_check(crit("c", "line_count_gte", path=str(f), min_lines=3))["status"] == "pass"
+    assert run_check(crit("c", "line_count_gte", path=str(f), min_lines=4))["status"] == "fail"
+    assert run_check(crit("c", "line_count_gte", path=str(tmp_path / "nope")))["status"] == "fail"
+
+
+# -- sqlite_table_has_rows --------------------------------------------------
+@pytest.fixture
+def db_path(tmp_path):
+    p = str(tmp_path / "test.db")
+    con = sqlite3.connect(p)
+    con.execute("CREATE TABLE events (id INTEGER)")
+    con.executemany("INSERT INTO events VALUES (?)", [(1,), (2,)])
+    con.execute("CREATE TABLE empty_t (id INTEGER)")
+    con.commit()
+    con.close()
+    return p
+
+
+def test_sqlite_table_has_rows_pass_fail(db_path):
+    assert run_check(crit("c", "sqlite_table_has_rows", db=db_path,
+                          table="events", min_rows=2))["status"] == "pass"
+    assert run_check(crit("c", "sqlite_table_has_rows", db=db_path,
+                          table="events", min_rows=3))["status"] == "fail"
+    assert run_check(crit("c", "sqlite_table_has_rows", db=db_path,
+                          table="empty_t"))["status"] == "fail"
+
+
+def test_sqlite_errors(tmp_path):
+    r = run_check(crit("c", "sqlite_table_has_rows", db=str(tmp_path / "x.db"),
+                       table="events"))
+    assert r["status"] == "error"  # missing db -> error, never pass
+    r = run_check(crit("c", "sqlite_table_has_rows", db="x",
+                       table="events; DROP TABLE events;--"))
+    assert r["status"] == "error" and "unsafe table name" in r["evidence"]
+
+
+# -- path_absent -----------------------------------------------------------
+def test_path_absent_pass_fail(tmp_path):
+    f = tmp_path / "present.txt"
+    f.write_text("x")
+    assert run_check(crit("c", "path_absent", path=str(f)))["status"] == "fail"
+    assert run_check(crit("c", "path_absent", path=str(tmp_path / "gone")))["status"] == "pass"
+
+
+# -- pattern_count_gte -----------------------------------------------------
+def test_pattern_count_gte_pass_fail_and_bad_regex(tmp_path):
+    f = tmp_path / "doc.md"
+    f.write_text("## A\n## B\n## C\n")
+    assert run_check(crit("c", "pattern_count_gte", path=str(f),
+                          pattern=r"^## ", min_count=3))["status"] == "pass"
+    assert run_check(crit("c", "pattern_count_gte", path=str(f),
+                          pattern=r"^## ", min_count=4))["status"] == "fail"
+    r = run_check(crit("c", "pattern_count_gte", path=str(f),
+                       pattern=r"([bad", min_count=1))
+    assert r["status"] == "error" and "bad regex" in r["evidence"]
+    r = run_check(crit("c", "pattern_count_gte", path=str(tmp_path / "nope"),
+                       pattern=r"x"))
+    assert r["status"] == "fail"
+
+
+# -- unknown type ----------------------------------------------------------
+def test_unknown_type_is_error():
+    r = run_check({"id": "c", "type": "does_not_exist"})
+    assert r["status"] == "error"
+
+
+def test_check_result_shape(tmp_path):
+    f = tmp_path / "a.txt"
+    f.write_text("hi")
+    r = run_check(crit("c1", "file_exists", path=str(f)))
+    assert set(r) == {"check_id", "type", "status", "evidence"}
+    assert r["check_id"] == "c1" and r["type"] == "file_exists"
+
+
+def test_env_var_expansion(tmp_path):
+    os.environ["VK_TEST_DIR"] = str(tmp_path)
+    try:
+        f = tmp_path / "env.txt"
+        f.write_text("hi")
+        r = run_check(crit("c", "file_exists", path="$VK_TEST_DIR/env.txt"))
+        assert r["status"] == "pass"
+    finally:
+        del os.environ["VK_TEST_DIR"]

verdictkit-0.1.0/tests/test_cli.py

@@ -0,0 +1,87 @@
+"""CLI and public API surface."""
+import json
+
+import pytest
+
+import verdictkit as vk
+from verdictkit.cli import main
+
+
+def test_public_surface():
+    for name in ["define_contract", "verify_run", "Verdict",
+                 "collect_evidence", "verify_bundle_hash",
+                 "Client", "VerdictkitError", "AuthError"]:
+        assert hasattr(vk, name), name
+        assert name in vk.__all__
+    assert vk.__version__ == "0.1.0"
+
+
+def write_contract(path, tmp_path, filename="contract.json"):
+    p = tmp_path / filename
+    p.write_text(json.dumps(path))
+    return str(p)
+
+
+def _contract_dict(out_file):
+    return {
+        "contract_id": "vc_test",
+        "task_id": "t",
+        "agent_id": "a",
+        "created_at": "2026-09-29T00:00:00+00:00",
+        "intended_action": "write file",
+        "scope": {},
+        "success_criteria": [
+            {"id": "c1", "type": "file_nonempty", "path": out_file,
+             "description": "written"}
+        ],
+        "unverifiable_claims": [],
+    }
+
+
+def test_cli_verify_pass(tmp_path, capsys):
+    f = tmp_path / "out.txt"
+    f.write_text("hello")
+    p = write_contract(_contract_dict(str(f)), tmp_path)
+    code = main(["verify", p])
+    assert code == 0
+    out = capsys.readouterr().out
+    assert "verdict: PASS" in out
+
+
+def test_cli_verify_fail(tmp_path, capsys):
+    p = write_contract(_contract_dict(str(tmp_path / "missing.txt")), tmp_path)
+    code = main(["verify", p])
+    assert code == 1
+    assert "verdict: FAIL" in capsys.readouterr().out
+
+
+def test_cli_verify_flag(tmp_path, capsys):
+    c = _contract_dict(str(tmp_path / "missing.txt"))
+    c["success_criteria"] = []
+    p = write_contract(c, tmp_path)
+    code = main(["verify", p])
+    assert code == 2
+    assert "verdict: FLAG" in capsys.readouterr().out
+
+
+def test_cli_bad_file(tmp_path, capsys):
+    code = main(["verify", str(tmp_path / "nope.json")])
+    assert code == 3
+    assert "error" in capsys.readouterr().err
+
+
+def test_cli_verbose(tmp_path, capsys):
+    f = tmp_path / "out.txt"
+    f.write_text("hello")
+    p = write_contract(_contract_dict(str(f)), tmp_path)
+    code = main(["verify", p, "--base-path", str(tmp_path), "-v"])
+    assert code == 0
+    out = capsys.readouterr().out
+    assert "[ pass]" in out and "c1" in out
+
+
+def test_cli_version(capsys):
+    with pytest.raises(SystemExit) as e:
+        main(["--version"])
+    assert e.value.code == 0
+    assert "0.1.0" in capsys.readouterr().out