fleetproof 0.1.0__tar.gz

fleetproof-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Gathrazio
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

fleetproof-0.1.0/PKG-INFO

@@ -0,0 +1,231 @@
+Metadata-Version: 2.4
+Name: fleetproof
+Version: 0.1.0
+Summary: Independent, out-of-band verification for agent fleets.
+Author: Gathrazio
+License: MIT
+Project-URL: Homepage, https://github.com/Gathrazio/fleetproof
+Project-URL: Repository, https://github.com/Gathrazio/fleetproof
+Keywords: agents,verification,claude-code,audit,run-log
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Software Development :: Quality Assurance
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Dynamic: license-file
+
+# FleetProof
+
+Independent, out-of-band verification for agent fleets — *what did my agents actually do, and what did they claim was done when it wasn't?*
+
+FleetProof is a Claude Code plugin plus a small stdlib-only Python package. It
+records what your agents do to a durable run log, and — critically — grades that
+work with a checker that runs in a separate process from the agent that did
+it. An agent may author the checks; it never executes and grades its own work.
+
+## Why this exists
+
+The failure mode this targets is the false "done": an agent reports a task
+complete when it isn't. A recent characterization study
+([arXiv:2606.09863](https://arxiv.org/abs/2606.09863)) measured it directly:
+45–48% of failures in single-control tau2-bench domains are exactly this —
+the agent confidently claims success. Among self-assessing coding-agent
+trajectories on AppWorld, it's 75.8%. And in the study's one *dual-control*
+setting — where a second, independent control point contradicts the agent's
+claims — false success collapses to 3%.
+
+FleetProof is built on that comparison: it gives any repo a second, independent
+control point. (To be precise: the 3% is a measured property of that
+benchmark setting, not a measured result of installing this tool — the study
+motivates the design; it didn't test it.)
+
+The word doing the work there is *independent*. A verifier that is the same agent
+(or a sub-agent prompted by it) inherits the same blind spots and the same
+incentive to declare victory. The only way to escape that is to move verification
+out of the agent's own process and make it deterministic. That is the entire
+design of FleetProof:
+
+- the agent writes the run records and authors `checks.json`;
+- a different process — the plugin's Stop hook, a `type: "command"` hook,
+  never an LLM — executes those checks and grades them;
+- if a blocking check fails, the hook returns `{"decision": "block", ...}` and
+  Claude Code refuses to let the agent stop on the false "done".
+
+No language model sits in the grading path. Grading is comparison.
+
+> Verifying one machine is free, forever. A hosted team tier — shared
+> evidence chains, approval queues, compliance export — is coming:
+> [join the waitlist](https://forms.gle/FcuBTYyoV4x2z8Hm8).
+
+## What's in the box
+
+| Piece | What it does |
+|---|---|
+| `@recorded` / `record()` | Durable per-invocation run records under `.fleetproof/runs/`. |
+| `fleetproof check` | The independent checker: runs each declared check in its own subprocess, records the verdict. |
+| Stop hook | Runs the checker when an agent claims done; blocks on a blocking-check failure. |
+| PostToolUse hook | Accretes a per-tool evidence trail into the run log. |
+| `fleetproof report` | One self-contained HTML file: per run, claimed-done vs. independently-verified. |
+
+Runtime dependencies: none (Python standard library only). A tool whose job is
+being trustworthy should add as little dependency and supply-chain surface as it can.
+
+## Install (each line is one command in Claude Code)
+
+```
+/plugin marketplace add Gathrazio/fleetproof
+/plugin install fleetproof@fleetproof
+```
+
+Then install the Python package so the CLI and hooks can run:
+
+```
+pip install fleetproof
+```
+
+Requires Python 3.10+. The plugin's hook commands invoke `python`, so `python`
+must be on your `PATH` (see *Scope and limitations*).
+
+## 10-minute quickstart
+
+1. Install the plugin and the package (three commands above).
+2. Declare what "done" means for your repo:
+   ```
+   fleetproof init
+   ```
+   This writes a starter `.fleetproof/checks.json`. Edit it — each entry is
+   `{ "id", "run", "expect", "block" }`:
+   ```json
+   {
+     "checks": [
+       { "id": "tests-pass", "run": "python -m pytest -q", "expect": "exit0", "block": true },
+       { "id": "artifact-built", "expect": { "file_exists": "dist/app.zip" }, "block": true }
+     ]
+   }
+   ```
+   `expect` is one of `"exit0"`, `{ "exit": N }`, `{ "regex": "..." }`, or
+   `{ "file_exists": "path" }`. `block: true` gates "done"; `block: false` is advisory.
+3. Run any Claude Code agent task as usual. When it claims done, the Stop hook
+   runs `fleetproof check` in its own process. If a blocking check fails, Claude is
+   blocked from stopping and told exactly which check disagreed.
+4. See what actually happened:
+   ```
+   fleetproof list                 # runs, newest first
+   fleetproof report               # writes .fleetproof/fleetproof-report.html
+   ```
+   The report marks each run *verified*, *contradicted* (claimed done, but a
+   blocking check failed), or *unverified* (no out-of-band verdict on record).
+
+You can also run the checker yourself at any time — `fleetproof check` — or on
+demand via the bundled `/fleetproof:verify-fleet` skill.
+
+## The check-spec format
+
+`.fleetproof/checks.json` is deliberately small and declarative. It is a contract
+you (or an agent, under your review) write *before* work is graded, kept in the
+repo where it is versioned and diffable. JSON, not YAML, on purpose: there is no
+third-party parser in the runtime.
+
+An agent is welcome to propose or edit checks. The guarantee FleetProof makes is
+narrower and firmer than "the agent verified its work": it is that *something
+other than the agent* ran the checks and recorded the result.
+
+## Writing richer checks
+
+The grading vocabulary is deliberately small — exit codes, a regex, a file
+existing — but `run` is an arbitrary command, so the *predicate* can be any
+program. The pattern: encode "done" as a script that exits non-zero when it
+isn't, and let FleetProof gate on the exit code.
+
+```json
+{
+  "checks": [
+    { "id": "dataset-valid",
+      "run": "python scripts/validate_dataset.py out/rows.csv --min-rows 1000",
+      "expect": "exit0", "block": true,
+      "description": "Output CSV exists, parses, has >=1000 rows, no nulls in key columns." },
+    { "id": "citations-resolve",
+      "run": "python scripts/check_links.py report/draft.md",
+      "expect": "exit0", "block": true },
+    { "id": "full-suite-advisory",
+      "run": "python -m pytest tests/slow -q",
+      "expect": "exit0", "block": false,
+      "description": "Slow suite is advisory: recorded, never gates the stop." }
+  ]
+}
+```
+
+Anything a program can decide, a check can gate: schema conformance, row
+counts, API health, diffs, wordcounts, link resolution. FleetProof's claim is
+never that the predicate language is rich — it's that *whatever predicate you
+choose runs outside the agent's process*.
+
+Guidance that keeps the gate honest: keep blocking checks fast and
+deterministic (flaky checks flap the gate; slow ones tax every stop) and demote
+heavy suites to `block: false`. For fleets doing varied tasks in one repo,
+the working convention is to have the agent author task-specific checks at task
+start — authoring is a feature — and let the spec-drift flag make any later
+revision loud. Per-task check scoping as a first-class mechanism is on the
+roadmap.
+
+## Scope and limitations (v0.1)
+
+This is an early release. Honest boundaries:
+
+- **`python` must be on `PATH`.** The plugin hooks invoke the checker via
+  `python "${CLAUDE_PLUGIN_ROOT}/scripts/..."`. Environments where the interpreter
+  is only reachable as `python3`, or not on `PATH`, need a shim — any wrapper on
+  `PATH` named `python` works, e.g. `sudo ln -s $(which python3) /usr/local/bin/python`.
+- **Checks are shell commands.** Their portability is your responsibility — a
+  check that shells out to `grep` won't behave identically on every OS. And
+  because a repo's `checks.json` runs shell commands on the Stop event, treat a
+  cloned repo's `checks.json` with the same trust you give its `Makefile` or
+  pre-commit config: it is executable code. FleetProof adds no network surface of
+  its own, and first-run consent-per-spec-hash is on the roadmap.
+- **The gate is only as good as the checks.** FleetProof enforces that an
+  independent process runs your checks; it cannot know whether your checks capture
+  what "done" really means. Weak checks give false confidence.
+- **An agent can weaken its own checks.** Authoring `checks.json` is a feature, so
+  nothing stops an agent from editing it mid-session to slip the gate. FleetProof
+  does not (yet) forbid spec edits; instead it records the SHA-256 of the spec on
+  every verdict and flags mid-session *spec drift* loudly — in the Stop-gate
+  output, the CLI, and the report — when a verdict's spec hash differs from the
+  session's first. Drift does not by itself block a passing verdict in v0.1; spec
+  pinning and consent-on-change are on the roadmap.
+- **Fail-open when unconfigured.** With no `.fleetproof/checks.json`, the Stop hook
+  does nothing (and says so on stderr) rather than blocking every task. A missing
+  spec is not a passing grade — it is an absent one.
+- **Blocking can repeat.** If a blocking check keeps failing, the Stop hook keeps
+  blocking. That is intended (don't stop on a false done), but it means a check
+  that can never pass needs operator intervention.
+
+## FleetProof for teams (coming — waitlist open)
+
+The plugin verifies one machine. The hosted tier turns those local verdicts into
+shared, durable evidence chains, team approval queues for gating agent
+work a human signs off on, and a compliance export a non-engineer can
+inspect — relevant as EU AI Act Article 14 human-oversight obligations become
+enforceable (2026-08-02).
+
+**[Join the waitlist →](https://forms.gle/FcuBTYyoV4x2z8Hm8)** — and tell us
+which piece you'd want first.
+
+The free plugin never sends anything anywhere: no telemetry, no network calls in
+the runtime, your run data stays on your disk. The hosted tier is opt-in and
+separate.
+
+## Development
+
+```
+pip install -e ".[dev]"
+python -m pytest -q
+```
+
+## License
+
+MIT. Author: Gathrazio.

fleetproof-0.1.0/README.md

@@ -0,0 +1,210 @@
+# FleetProof
+
+Independent, out-of-band verification for agent fleets — *what did my agents actually do, and what did they claim was done when it wasn't?*
+
+FleetProof is a Claude Code plugin plus a small stdlib-only Python package. It
+records what your agents do to a durable run log, and — critically — grades that
+work with a checker that runs in a separate process from the agent that did
+it. An agent may author the checks; it never executes and grades its own work.
+
+## Why this exists
+
+The failure mode this targets is the false "done": an agent reports a task
+complete when it isn't. A recent characterization study
+([arXiv:2606.09863](https://arxiv.org/abs/2606.09863)) measured it directly:
+45–48% of failures in single-control tau2-bench domains are exactly this —
+the agent confidently claims success. Among self-assessing coding-agent
+trajectories on AppWorld, it's 75.8%. And in the study's one *dual-control*
+setting — where a second, independent control point contradicts the agent's
+claims — false success collapses to 3%.
+
+FleetProof is built on that comparison: it gives any repo a second, independent
+control point. (To be precise: the 3% is a measured property of that
+benchmark setting, not a measured result of installing this tool — the study
+motivates the design; it didn't test it.)
+
+The word doing the work there is *independent*. A verifier that is the same agent
+(or a sub-agent prompted by it) inherits the same blind spots and the same
+incentive to declare victory. The only way to escape that is to move verification
+out of the agent's own process and make it deterministic. That is the entire
+design of FleetProof:
+
+- the agent writes the run records and authors `checks.json`;
+- a different process — the plugin's Stop hook, a `type: "command"` hook,
+  never an LLM — executes those checks and grades them;
+- if a blocking check fails, the hook returns `{"decision": "block", ...}` and
+  Claude Code refuses to let the agent stop on the false "done".
+
+No language model sits in the grading path. Grading is comparison.
+
+> Verifying one machine is free, forever. A hosted team tier — shared
+> evidence chains, approval queues, compliance export — is coming:
+> [join the waitlist](https://forms.gle/FcuBTYyoV4x2z8Hm8).
+
+## What's in the box
+
+| Piece | What it does |
+|---|---|
+| `@recorded` / `record()` | Durable per-invocation run records under `.fleetproof/runs/`. |
+| `fleetproof check` | The independent checker: runs each declared check in its own subprocess, records the verdict. |
+| Stop hook | Runs the checker when an agent claims done; blocks on a blocking-check failure. |
+| PostToolUse hook | Accretes a per-tool evidence trail into the run log. |
+| `fleetproof report` | One self-contained HTML file: per run, claimed-done vs. independently-verified. |
+
+Runtime dependencies: none (Python standard library only). A tool whose job is
+being trustworthy should add as little dependency and supply-chain surface as it can.
+
+## Install (each line is one command in Claude Code)
+
+```
+/plugin marketplace add Gathrazio/fleetproof
+/plugin install fleetproof@fleetproof
+```
+
+Then install the Python package so the CLI and hooks can run:
+
+```
+pip install fleetproof
+```
+
+Requires Python 3.10+. The plugin's hook commands invoke `python`, so `python`
+must be on your `PATH` (see *Scope and limitations*).
+
+## 10-minute quickstart
+
+1. Install the plugin and the package (three commands above).
+2. Declare what "done" means for your repo:
+   ```
+   fleetproof init
+   ```
+   This writes a starter `.fleetproof/checks.json`. Edit it — each entry is
+   `{ "id", "run", "expect", "block" }`:
+   ```json
+   {
+     "checks": [
+       { "id": "tests-pass", "run": "python -m pytest -q", "expect": "exit0", "block": true },
+       { "id": "artifact-built", "expect": { "file_exists": "dist/app.zip" }, "block": true }
+     ]
+   }
+   ```
+   `expect` is one of `"exit0"`, `{ "exit": N }`, `{ "regex": "..." }`, or
+   `{ "file_exists": "path" }`. `block: true` gates "done"; `block: false` is advisory.
+3. Run any Claude Code agent task as usual. When it claims done, the Stop hook
+   runs `fleetproof check` in its own process. If a blocking check fails, Claude is
+   blocked from stopping and told exactly which check disagreed.
+4. See what actually happened:
+   ```
+   fleetproof list                 # runs, newest first
+   fleetproof report               # writes .fleetproof/fleetproof-report.html
+   ```
+   The report marks each run *verified*, *contradicted* (claimed done, but a
+   blocking check failed), or *unverified* (no out-of-band verdict on record).
+
+You can also run the checker yourself at any time — `fleetproof check` — or on
+demand via the bundled `/fleetproof:verify-fleet` skill.
+
+## The check-spec format
+
+`.fleetproof/checks.json` is deliberately small and declarative. It is a contract
+you (or an agent, under your review) write *before* work is graded, kept in the
+repo where it is versioned and diffable. JSON, not YAML, on purpose: there is no
+third-party parser in the runtime.
+
+An agent is welcome to propose or edit checks. The guarantee FleetProof makes is
+narrower and firmer than "the agent verified its work": it is that *something
+other than the agent* ran the checks and recorded the result.
+
+## Writing richer checks
+
+The grading vocabulary is deliberately small — exit codes, a regex, a file
+existing — but `run` is an arbitrary command, so the *predicate* can be any
+program. The pattern: encode "done" as a script that exits non-zero when it
+isn't, and let FleetProof gate on the exit code.
+
+```json
+{
+  "checks": [
+    { "id": "dataset-valid",
+      "run": "python scripts/validate_dataset.py out/rows.csv --min-rows 1000",
+      "expect": "exit0", "block": true,
+      "description": "Output CSV exists, parses, has >=1000 rows, no nulls in key columns." },
+    { "id": "citations-resolve",
+      "run": "python scripts/check_links.py report/draft.md",
+      "expect": "exit0", "block": true },
+    { "id": "full-suite-advisory",
+      "run": "python -m pytest tests/slow -q",
+      "expect": "exit0", "block": false,
+      "description": "Slow suite is advisory: recorded, never gates the stop." }
+  ]
+}
+```
+
+Anything a program can decide, a check can gate: schema conformance, row
+counts, API health, diffs, wordcounts, link resolution. FleetProof's claim is
+never that the predicate language is rich — it's that *whatever predicate you
+choose runs outside the agent's process*.
+
+Guidance that keeps the gate honest: keep blocking checks fast and
+deterministic (flaky checks flap the gate; slow ones tax every stop) and demote
+heavy suites to `block: false`. For fleets doing varied tasks in one repo,
+the working convention is to have the agent author task-specific checks at task
+start — authoring is a feature — and let the spec-drift flag make any later
+revision loud. Per-task check scoping as a first-class mechanism is on the
+roadmap.
+
+## Scope and limitations (v0.1)
+
+This is an early release. Honest boundaries:
+
+- **`python` must be on `PATH`.** The plugin hooks invoke the checker via
+  `python "${CLAUDE_PLUGIN_ROOT}/scripts/..."`. Environments where the interpreter
+  is only reachable as `python3`, or not on `PATH`, need a shim — any wrapper on
+  `PATH` named `python` works, e.g. `sudo ln -s $(which python3) /usr/local/bin/python`.
+- **Checks are shell commands.** Their portability is your responsibility — a
+  check that shells out to `grep` won't behave identically on every OS. And
+  because a repo's `checks.json` runs shell commands on the Stop event, treat a
+  cloned repo's `checks.json` with the same trust you give its `Makefile` or
+  pre-commit config: it is executable code. FleetProof adds no network surface of
+  its own, and first-run consent-per-spec-hash is on the roadmap.
+- **The gate is only as good as the checks.** FleetProof enforces that an
+  independent process runs your checks; it cannot know whether your checks capture
+  what "done" really means. Weak checks give false confidence.
+- **An agent can weaken its own checks.** Authoring `checks.json` is a feature, so
+  nothing stops an agent from editing it mid-session to slip the gate. FleetProof
+  does not (yet) forbid spec edits; instead it records the SHA-256 of the spec on
+  every verdict and flags mid-session *spec drift* loudly — in the Stop-gate
+  output, the CLI, and the report — when a verdict's spec hash differs from the
+  session's first. Drift does not by itself block a passing verdict in v0.1; spec
+  pinning and consent-on-change are on the roadmap.
+- **Fail-open when unconfigured.** With no `.fleetproof/checks.json`, the Stop hook
+  does nothing (and says so on stderr) rather than blocking every task. A missing
+  spec is not a passing grade — it is an absent one.
+- **Blocking can repeat.** If a blocking check keeps failing, the Stop hook keeps
+  blocking. That is intended (don't stop on a false done), but it means a check
+  that can never pass needs operator intervention.
+
+## FleetProof for teams (coming — waitlist open)
+
+The plugin verifies one machine. The hosted tier turns those local verdicts into
+shared, durable evidence chains, team approval queues for gating agent
+work a human signs off on, and a compliance export a non-engineer can
+inspect — relevant as EU AI Act Article 14 human-oversight obligations become
+enforceable (2026-08-02).
+
+**[Join the waitlist →](https://forms.gle/FcuBTYyoV4x2z8Hm8)** — and tell us
+which piece you'd want first.
+
+The free plugin never sends anything anywhere: no telemetry, no network calls in
+the runtime, your run data stays on your disk. The hosted tier is opt-in and
+separate.
+
+## Development
+
+```
+pip install -e ".[dev]"
+python -m pytest -q
+```
+
+## License
+
+MIT. Author: Gathrazio.

fleetproof-0.1.0/pyproject.toml

@@ -0,0 +1,40 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "fleetproof"
+version = "0.1.0"
+description = "Independent, out-of-band verification for agent fleets."
+readme = "README.md"
+requires-python = ">=3.10"
+license = { text = "MIT" }
+authors = [{ name = "Gathrazio" }]
+keywords = ["agents", "verification", "claude-code", "audit", "run-log"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Topic :: Software Development :: Quality Assurance",
+]
+# Runtime is stdlib-only on purpose: a verification tool should add as little
+# dependency (and supply-chain) surface as it can get away with.
+dependencies = []
+
+[project.optional-dependencies]
+dev = ["pytest>=7"]
+
+[project.urls]
+Homepage = "https://github.com/Gathrazio/fleetproof"
+Repository = "https://github.com/Gathrazio/fleetproof"
+
+[project.scripts]
+fleetproof = "fleetproof.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+pythonpath = ["src"]

fleetproof-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

fleetproof-0.1.0/src/fleetproof/__init__.py

@@ -0,0 +1,46 @@
+"""FleetProof — independent, out-of-band verification for agent fleets.
+
+The agent may author the checks; it never executes and grades its own work.
+"""
+
+from __future__ import annotations
+
+__version__ = "0.1.0"
+
+from .runlog import (
+    RunRecord,
+    SubInvocation,
+    child_run_id,
+    current_run_id,
+    list_run_records,
+    load_run,
+    project_root,
+    record,
+    recorded,
+    runs_dir,
+    set_runs_dir,
+)
+from .checks import Check, CheckSpecError, load_checks
+from .checker import CheckReport, CheckResult, run_check, run_checks
+
+__all__ = [
+    "__version__",
+    "recorded",
+    "record",
+    "current_run_id",
+    "child_run_id",
+    "project_root",
+    "runs_dir",
+    "set_runs_dir",
+    "list_run_records",
+    "load_run",
+    "RunRecord",
+    "SubInvocation",
+    "Check",
+    "CheckSpecError",
+    "load_checks",
+    "CheckReport",
+    "CheckResult",
+    "run_check",
+    "run_checks",
+]

fleetproof-0.1.0/src/fleetproof/__main__.py

@@ -0,0 +1,6 @@
+"""Allow ``python -m fleetproof`` as an alternative to the console script."""
+
+from .cli import main
+
+if __name__ == "__main__":
+    raise SystemExit(main())