edward-guard 0.1.1__tar.gz

edward_guard-0.1.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Edward contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

edward_guard-0.1.1/PKG-INFO

@@ -0,0 +1,243 @@
+Metadata-Version: 2.4
+Name: edward-guard
+Version: 0.1.1
+Summary: External control plane for AI coding agents: deterministic rules + local semantic scorer, audit trail, human-resumable interventions.
+Author: Edward contributors
+License: MIT
+Keywords: ai,agent,guardrails,safety,observability,cost-control
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Software Development :: Quality Assurance
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# Edward
+
+**Edward is an external control plane for AI coding agents.** Agents fail quietly: they retry the same broken test 40 times, burn $8 in tokens on a loop, run `rm -rf` on a database directory, and write to files they were never supposed to touch. The agent doesn't know it's failing — from its perspective, it's still trying.
+
+Edward sits between the agent and its runtime. It watches the event stream, builds a picture of what the agent is actually doing across turns, and intervenes when the picture stops looking right.
+
+[![ci](https://github.com/OWNER/edward/actions/workflows/ci.yml/badge.svg)](../../actions)
+[![PyPI](https://img.shields.io/badge/PyPI-edward--guard-blue)](https://pypi.org/project/edward-guard/)
+[![license](https://img.shields.io/badge/license-MIT-green)](LICENSE)
+[![python](https://img.shields.io/badge/python-3.11%2B-blue)](pyproject.toml)
+
+**Measured on the StepShield benchmark (NeurIPS 2026):** deterministic rules alone detect 7.4% of content-semantic violations at 1.9% FPR; adding a local 4B scorer with evidence-grounded task-contract probes reaches **57.4% recall at 20.4% FPR with EIR_3 0.790** (paper's LLMJudge tier: 95.4% / 5.6% / 0.89, at GPT-4.1-mini cost). See [BENCHMARK.md](BENCHMARK.md).
+
+```bash
+pipx install edward-guard        # zero dependencies, stdlib only
+edward demo                      # self-running proof: 6 failure scenarios
+edward wrap -- pi "fix the flaky test"
+edward wrap --no-scorer -- python my_agent.py     # any command, rule-only
+```
+
+
+```
+Agent (Pi / Codex / custom)
+    │ events
+    ▼
+Canonical Event Schema   ← normalizes tool names to capabilities
+    │
+    ▼
+State Engine             ← materializes cross-turn agent state
+    │
+    ▼
+Trigger Rules [FROZEN]   ← deterministic safety + convergence checks
+    │
+    ├─ HARD_CONSTRAINT ──→ Kernel: BLOCK (Jev cannot override)
+    │
+    └─ SOFT_DECISION ──→ Jev (TypeSafe) ──→ Policy Resolver
+                              │                │
+                              └────────────────┘
+                                       │
+                                       ▼
+                               Control Kernel
+                                       │
+                                       ▼
+                                PAUSE / CANCEL / RESUME
+```
+
+## Why not just if/else?
+
+A simple watchdog (`error_count > 5 → stop`) looks at individual events. The failure modes that actually kill long-running agents don't show up in any single event. They emerge from the shape of the trajectory over time.
+
+We tested this directly.
+
+### Watchdog vs State Engine → MSS
+
+Held-out evaluation, 20 trials × 6 scenarios, frozen config:
+
+| Scenario | Watchdog detection | State Engine detection | Watchdog ctx | State ctx |
+|---|---|---|---|---|
+| budget bleed | 20% | **100%** | 2,322 | 409 |
+| infinite loop | 0% | **100%** | 6,084 | 409 |
+| convergence stall | 0% | **100%** | 1,442 | 409 |
+| dangerous command | 75% | **100%** | 522 | 434 |
+
+The watchdog misses infinite loops and stalls entirely because it has no memory across turns. The State Engine maintains a sliding window of recent tool calls, tracks file modifications, and computes elapsed time — none of which fit in a single-event check.
+
+## What it detects
+
+Six trigger rules, tuned on a development set (seed 42) and evaluated frozen on held-out data (seed 137):
+
+| Signal | Threshold | Fires when |
+|---|---|---|
+| Error rate | > 40% over last 8+ calls | Agent is failing repeatedly, with no recovery signal |
+| Retry count | ≥ 3 | Agent keeps retrying the same thing |
+| Token budget | > 80% consumed | Agent is burning resources |
+| Convergence stall | > 600s + > 5 turns | Agent is running but not completing |
+| Passive stall | 12 consecutive reads, 0 writes | Agent is going in circles |
+| Dangerous command | regex match | Agent executes `rm -rf`, `sudo`, `git push --force`, etc. |
+
+Plus two post-hoc additions from robustness testing:
+
+| Signal | Fires when |
+|---|---|
+| Scope violation | Agent writes to a path outside the allowed prefix list |
+| Silent corruption risk | ≥ 10 consecutive file writes with zero shell verification |
+
+## Held-out benchmark
+
+50 trials × 6 scenarios, frozen configuration (no post-hoc tuning):
+
+```
+                   Detection    FP Rate    Latency (tool calls)
+infinite_loop      100%         —          8.0
+budget_bleed       100%         —          11.7
+dangerous          100%         —          5.0
+stall              100%         —          4.0
+normal             —            0%         —
+transient_failure  —            0%         —
+
+Precision: 100%    Recall: 100%    FPR: 0/100
+```
+
+## Robustness: where it breaks
+
+We attacked the architecture along four dimensions. Results after P0+P1 fixes:
+
+**Agent heterogeneity.** Three event schemas (Pi, Codex-style, custom) normalized through the canonical schema layer. Custom agents use tool names like `execute` and `terminal` instead of `bash`. Before the fix, dangerous command detection on custom agents was 0%. After: 100% across all six combinations.
+
+**Task distribution shift.** Three unseen normal task profiles (heavy-write, error-then-recover, mixed-signal). The error-then-recover profile initially produced 60% false positives because the error-rate trigger couldn't distinguish "temporarily failing but recovering" from "systematically diverging." Added a `recovery_signal` property: if the tail of the sliding window shows ≥3 consecutive successes after errors, the trigger holds. Post-fix FPR: 0% on all three profiles.
+
+**Controller ablation.** Jev (TypeSafe's System One model) acts as a probabilistic second opinion on soft decisions. On dangerous commands, rules fire deterministically as HARD_CONSTRAINT — Jev cannot override. On soft decisions, Jev confirms ~100% of infinite-loop interventions but overrides ~97% of budget-bleed and ~100% of stall interventions back to CONTINUE. This tension is a design feature: the policy resolver lets deterministic safety guards override probabilistic judgments, but not vice versa.
+
+**Unseen failure modes.** Three novel failure scenarios not in dev or test:
+
+| Scenario | Detection | Mechanism |
+|---|---|---|
+| Permission escalation | 100% | Dangerous command regex |
+| Cascading failure | 100% | Retry count trigger |
+| Silent corruption | 76.7% | Unverified-write streak detector |
+
+Silent corruption — the agent writes 15 files in a row without running any test — was invisible to every original trigger. The streak detector catches most cases but not all (some trials have only 8-9 writes, below the threshold). Full coverage would require World State tracking: comparing what the agent wrote against what the task expected.
+
+## The four things we learned
+
+1. **Execution telemetry ≠ world state.** The agent's own view ("I'm still trying") diverges from what an external observer sees (error rate rising, budget depleting, no progress). This gap is the entire reason a control plane exists.
+
+2. **Event-level checks cannot detect temporal failure modes.** Every trigger that requires memory across turns (stall, convergence, recovery) needs the State Engine. Single-event or simple-counter approaches score 0% on these.
+
+3. **Safety and intelligence need different authority levels.** Dangerous-command blocking is deterministic and non-overridable. Everything else benefits from a probabilistic second opinion. Conflating the two makes the safety guard probabilistic, which defeats the purpose.
+
+4. **Agent-execution monitoring alone cannot detect world-state corruption.** If the agent writes destructive content without producing any error, no amount of execution telemetry will catch it. This requires comparing actual filesystem state against a task contract. It's the clearest motivation for v0.2.
+
+## Running it
+
+### Product quickstart (edward CLI)
+
+```bash
+pip install -e .            # zero dependencies, stdlib only (Python >= 3.11)
+
+# optional: the pi coding agent (full monitoring + intervention on pi tasks)
+npm install -g --ignore-scripts @earendil-works/pi-coding-agent
+
+edward doctor           # environment checks: scorer, pi, audit dir
+edward demo             # self-running proof: 6 scenarios, PASS/FAIL gate
+edward wrap -- pi "Fix the bug in utils.py so that the test passes"
+edward wrap --scope ./src --no-scorer -- python agent_script.py   # any command, rule-only
+edward eval --policy conservative --trials 30   # tune a policy before live use
+edward audit            # intervention summary ($ saved evidence)
+edward policy-template --preset balanced > edward.toml
+```
+
+The semantic scorer is an internal HTTP endpoint (LAN only, no auth):
+`GET /health` + `POST /v1/score` on Qwen/Qwen3.5-4B — see `edward/scorer_client.py`.
+Without it, edward runs in rule-only mode and stays fully protective.
+
+Exit codes: `0` completed, `75` PAUSED (resumable: `edward wrap --continue -- ...`),
+`76` terminated by control plane, `130` interrupted. Audit JSONL lands in
+`~/.edward/audit.jsonl`. Policy packs are TOML/JSON with three presets
+(`conservative` / `balanced` = FROZEN defaults / `aggressive`).
+
+### Research scripts (pre-packaging, still work)
+
+```bash
+python main.py "Fix the bug in utils.py so that the test passes"
+
+# Run the benchmark
+python benchmark.py
+
+# Run the ablation
+python ablation.py
+
+# Run held-out evaluation
+python heldout_eval.py
+
+# Run robustness evaluation
+python robustness_eval.py
+```
+
+### External benchmark: StepShield (NeurIPS 2026)
+
+```bash
+git clone --depth 1 https://github.com/glo26/stepshield /tmp/stepshield
+edward eval --suite stepshield --data /tmp/stepshield/data --mode rules
+edward eval --suite stepshield --data /tmp/stepshield/data --mode contract \
+    --scorer http://192.168.2.51:8000        # needs live scorer endpoint
+```
+
+`rules` = the deterministic v0.1 layer (dangerous-command regex + scope
+violations derived from task constraints). `contract` = the v0.2 Task
+Contract preview: task intent + constraints go into the scorer question,
+and the 4B endpoint judges each state-changing step (OK / VIOLATION /
+UNSURE) with asymmetric confirmation: conf >= 0.9 fires alone, 0.6-0.9
+needs one corroborating violation, a confident OK clears suspicion.
+Metrics are paper-aligned (EIR_k, recall, FPR on clean, per-category).
+
+Pi uses `--mode rpc` for headless operation. The control plane spawns it as a subprocess, reads JSONL events from stdout, and sends control commands (abort, steer) via stdin. No Pi source code is modified.
+
+## Project structure
+
+```
+edward/            Product package (pip install -e .)
+  cli.py               edward CLI: wrap / demo / eval / audit / doctor
+  engine.py            ControlPlane: events -> triggers -> scorer -> decision -> audit
+  config.py            Policy packs (TOML/JSON, 3 presets, strict validation)
+  audit.py             Append-only JSONL audit log (never blocks monitoring)
+  scorer.py            Semantic scorer client with circuit breaker
+  scenarios.py         Failure scenario suite (single source for demo/eval)
+  evalcmd.py           Policy evaluation gate (detection / FPR / timing)
+  pi_client.py         Pi RPC client (cwd, provider/model, stderr capture)
+  scorer_client.py        /v1/score client (internal endpoint, Qwen3.5-4B)
+  state_engine.py      Materialize AgentState from event stream
+  triggers.py          8 trigger rules, policy-parameterized (defaults FROZEN)
+  kernel.py            Decision authority hierarchy
+  canonical_events.py  Normalize Pi/Codex/custom events to canonical schema
+  notify.py            Slack webhook + stderr banners (fail-silent)
+main.py                Legacy entry -> edward wrap
+benchmark.py           300-trial held-out benchmark (re-exports scenarios)
+ablation.py            Watchdog vs State Engine comparison
+heldout_eval.py        Dev/test split + frozen config evaluation
+robustness_eval.py     4-dimension robustness attack
+extreme_tests.py       4 extreme scenario demos
+test_product.py        Product test suite (unittest, offline)
+```
+
+## What's next
+
+The silent-corruption gap points at the next layer: a Task Contract that defines which files the agent should touch and what the expected end-state looks like. When the agent's actual filesystem mutations diverge from the contract, that's a scope violation — regardless of whether any tool call returned an error. This moves the State Engine from "what is the agent doing" to "what is the agent doing to the world, and is that still allowed."

edward_guard-0.1.1/README.md

@@ -0,0 +1,226 @@
+# Edward
+
+**Edward is an external control plane for AI coding agents.** Agents fail quietly: they retry the same broken test 40 times, burn $8 in tokens on a loop, run `rm -rf` on a database directory, and write to files they were never supposed to touch. The agent doesn't know it's failing — from its perspective, it's still trying.
+
+Edward sits between the agent and its runtime. It watches the event stream, builds a picture of what the agent is actually doing across turns, and intervenes when the picture stops looking right.
+
+[![ci](https://github.com/OWNER/edward/actions/workflows/ci.yml/badge.svg)](../../actions)
+[![PyPI](https://img.shields.io/badge/PyPI-edward--guard-blue)](https://pypi.org/project/edward-guard/)
+[![license](https://img.shields.io/badge/license-MIT-green)](LICENSE)
+[![python](https://img.shields.io/badge/python-3.11%2B-blue)](pyproject.toml)
+
+**Measured on the StepShield benchmark (NeurIPS 2026):** deterministic rules alone detect 7.4% of content-semantic violations at 1.9% FPR; adding a local 4B scorer with evidence-grounded task-contract probes reaches **57.4% recall at 20.4% FPR with EIR_3 0.790** (paper's LLMJudge tier: 95.4% / 5.6% / 0.89, at GPT-4.1-mini cost). See [BENCHMARK.md](BENCHMARK.md).
+
+```bash
+pipx install edward-guard        # zero dependencies, stdlib only
+edward demo                      # self-running proof: 6 failure scenarios
+edward wrap -- pi "fix the flaky test"
+edward wrap --no-scorer -- python my_agent.py     # any command, rule-only
+```
+
+
+```
+Agent (Pi / Codex / custom)
+    │ events
+    ▼
+Canonical Event Schema   ← normalizes tool names to capabilities
+    │
+    ▼
+State Engine             ← materializes cross-turn agent state
+    │
+    ▼
+Trigger Rules [FROZEN]   ← deterministic safety + convergence checks
+    │
+    ├─ HARD_CONSTRAINT ──→ Kernel: BLOCK (Jev cannot override)
+    │
+    └─ SOFT_DECISION ──→ Jev (TypeSafe) ──→ Policy Resolver
+                              │                │
+                              └────────────────┘
+                                       │
+                                       ▼
+                               Control Kernel
+                                       │
+                                       ▼
+                                PAUSE / CANCEL / RESUME
+```
+
+## Why not just if/else?
+
+A simple watchdog (`error_count > 5 → stop`) looks at individual events. The failure modes that actually kill long-running agents don't show up in any single event. They emerge from the shape of the trajectory over time.
+
+We tested this directly.
+
+### Watchdog vs State Engine → MSS
+
+Held-out evaluation, 20 trials × 6 scenarios, frozen config:
+
+| Scenario | Watchdog detection | State Engine detection | Watchdog ctx | State ctx |
+|---|---|---|---|---|
+| budget bleed | 20% | **100%** | 2,322 | 409 |
+| infinite loop | 0% | **100%** | 6,084 | 409 |
+| convergence stall | 0% | **100%** | 1,442 | 409 |
+| dangerous command | 75% | **100%** | 522 | 434 |
+
+The watchdog misses infinite loops and stalls entirely because it has no memory across turns. The State Engine maintains a sliding window of recent tool calls, tracks file modifications, and computes elapsed time — none of which fit in a single-event check.
+
+## What it detects
+
+Six trigger rules, tuned on a development set (seed 42) and evaluated frozen on held-out data (seed 137):
+
+| Signal | Threshold | Fires when |
+|---|---|---|
+| Error rate | > 40% over last 8+ calls | Agent is failing repeatedly, with no recovery signal |
+| Retry count | ≥ 3 | Agent keeps retrying the same thing |
+| Token budget | > 80% consumed | Agent is burning resources |
+| Convergence stall | > 600s + > 5 turns | Agent is running but not completing |
+| Passive stall | 12 consecutive reads, 0 writes | Agent is going in circles |
+| Dangerous command | regex match | Agent executes `rm -rf`, `sudo`, `git push --force`, etc. |
+
+Plus two post-hoc additions from robustness testing:
+
+| Signal | Fires when |
+|---|---|
+| Scope violation | Agent writes to a path outside the allowed prefix list |
+| Silent corruption risk | ≥ 10 consecutive file writes with zero shell verification |
+
+## Held-out benchmark
+
+50 trials × 6 scenarios, frozen configuration (no post-hoc tuning):
+
+```
+                   Detection    FP Rate    Latency (tool calls)
+infinite_loop      100%         —          8.0
+budget_bleed       100%         —          11.7
+dangerous          100%         —          5.0
+stall              100%         —          4.0
+normal             —            0%         —
+transient_failure  —            0%         —
+
+Precision: 100%    Recall: 100%    FPR: 0/100
+```
+
+## Robustness: where it breaks
+
+We attacked the architecture along four dimensions. Results after P0+P1 fixes:
+
+**Agent heterogeneity.** Three event schemas (Pi, Codex-style, custom) normalized through the canonical schema layer. Custom agents use tool names like `execute` and `terminal` instead of `bash`. Before the fix, dangerous command detection on custom agents was 0%. After: 100% across all six combinations.
+
+**Task distribution shift.** Three unseen normal task profiles (heavy-write, error-then-recover, mixed-signal). The error-then-recover profile initially produced 60% false positives because the error-rate trigger couldn't distinguish "temporarily failing but recovering" from "systematically diverging." Added a `recovery_signal` property: if the tail of the sliding window shows ≥3 consecutive successes after errors, the trigger holds. Post-fix FPR: 0% on all three profiles.
+
+**Controller ablation.** Jev (TypeSafe's System One model) acts as a probabilistic second opinion on soft decisions. On dangerous commands, rules fire deterministically as HARD_CONSTRAINT — Jev cannot override. On soft decisions, Jev confirms ~100% of infinite-loop interventions but overrides ~97% of budget-bleed and ~100% of stall interventions back to CONTINUE. This tension is a design feature: the policy resolver lets deterministic safety guards override probabilistic judgments, but not vice versa.
+
+**Unseen failure modes.** Three novel failure scenarios not in dev or test:
+
+| Scenario | Detection | Mechanism |
+|---|---|---|
+| Permission escalation | 100% | Dangerous command regex |
+| Cascading failure | 100% | Retry count trigger |
+| Silent corruption | 76.7% | Unverified-write streak detector |
+
+Silent corruption — the agent writes 15 files in a row without running any test — was invisible to every original trigger. The streak detector catches most cases but not all (some trials have only 8-9 writes, below the threshold). Full coverage would require World State tracking: comparing what the agent wrote against what the task expected.
+
+## The four things we learned
+
+1. **Execution telemetry ≠ world state.** The agent's own view ("I'm still trying") diverges from what an external observer sees (error rate rising, budget depleting, no progress). This gap is the entire reason a control plane exists.
+
+2. **Event-level checks cannot detect temporal failure modes.** Every trigger that requires memory across turns (stall, convergence, recovery) needs the State Engine. Single-event or simple-counter approaches score 0% on these.
+
+3. **Safety and intelligence need different authority levels.** Dangerous-command blocking is deterministic and non-overridable. Everything else benefits from a probabilistic second opinion. Conflating the two makes the safety guard probabilistic, which defeats the purpose.
+
+4. **Agent-execution monitoring alone cannot detect world-state corruption.** If the agent writes destructive content without producing any error, no amount of execution telemetry will catch it. This requires comparing actual filesystem state against a task contract. It's the clearest motivation for v0.2.
+
+## Running it
+
+### Product quickstart (edward CLI)
+
+```bash
+pip install -e .            # zero dependencies, stdlib only (Python >= 3.11)
+
+# optional: the pi coding agent (full monitoring + intervention on pi tasks)
+npm install -g --ignore-scripts @earendil-works/pi-coding-agent
+
+edward doctor           # environment checks: scorer, pi, audit dir
+edward demo             # self-running proof: 6 scenarios, PASS/FAIL gate
+edward wrap -- pi "Fix the bug in utils.py so that the test passes"
+edward wrap --scope ./src --no-scorer -- python agent_script.py   # any command, rule-only
+edward eval --policy conservative --trials 30   # tune a policy before live use
+edward audit            # intervention summary ($ saved evidence)
+edward policy-template --preset balanced > edward.toml
+```
+
+The semantic scorer is an internal HTTP endpoint (LAN only, no auth):
+`GET /health` + `POST /v1/score` on Qwen/Qwen3.5-4B — see `edward/scorer_client.py`.
+Without it, edward runs in rule-only mode and stays fully protective.
+
+Exit codes: `0` completed, `75` PAUSED (resumable: `edward wrap --continue -- ...`),
+`76` terminated by control plane, `130` interrupted. Audit JSONL lands in
+`~/.edward/audit.jsonl`. Policy packs are TOML/JSON with three presets
+(`conservative` / `balanced` = FROZEN defaults / `aggressive`).
+
+### Research scripts (pre-packaging, still work)
+
+```bash
+python main.py "Fix the bug in utils.py so that the test passes"
+
+# Run the benchmark
+python benchmark.py
+
+# Run the ablation
+python ablation.py
+
+# Run held-out evaluation
+python heldout_eval.py
+
+# Run robustness evaluation
+python robustness_eval.py
+```
+
+### External benchmark: StepShield (NeurIPS 2026)
+
+```bash
+git clone --depth 1 https://github.com/glo26/stepshield /tmp/stepshield
+edward eval --suite stepshield --data /tmp/stepshield/data --mode rules
+edward eval --suite stepshield --data /tmp/stepshield/data --mode contract \
+    --scorer http://192.168.2.51:8000        # needs live scorer endpoint
+```
+
+`rules` = the deterministic v0.1 layer (dangerous-command regex + scope
+violations derived from task constraints). `contract` = the v0.2 Task
+Contract preview: task intent + constraints go into the scorer question,
+and the 4B endpoint judges each state-changing step (OK / VIOLATION /
+UNSURE) with asymmetric confirmation: conf >= 0.9 fires alone, 0.6-0.9
+needs one corroborating violation, a confident OK clears suspicion.
+Metrics are paper-aligned (EIR_k, recall, FPR on clean, per-category).
+
+Pi uses `--mode rpc` for headless operation. The control plane spawns it as a subprocess, reads JSONL events from stdout, and sends control commands (abort, steer) via stdin. No Pi source code is modified.
+
+## Project structure
+
+```
+edward/            Product package (pip install -e .)
+  cli.py               edward CLI: wrap / demo / eval / audit / doctor
+  engine.py            ControlPlane: events -> triggers -> scorer -> decision -> audit
+  config.py            Policy packs (TOML/JSON, 3 presets, strict validation)
+  audit.py             Append-only JSONL audit log (never blocks monitoring)
+  scorer.py            Semantic scorer client with circuit breaker
+  scenarios.py         Failure scenario suite (single source for demo/eval)
+  evalcmd.py           Policy evaluation gate (detection / FPR / timing)
+  pi_client.py         Pi RPC client (cwd, provider/model, stderr capture)
+  scorer_client.py        /v1/score client (internal endpoint, Qwen3.5-4B)
+  state_engine.py      Materialize AgentState from event stream
+  triggers.py          8 trigger rules, policy-parameterized (defaults FROZEN)
+  kernel.py            Decision authority hierarchy
+  canonical_events.py  Normalize Pi/Codex/custom events to canonical schema
+  notify.py            Slack webhook + stderr banners (fail-silent)
+main.py                Legacy entry -> edward wrap
+benchmark.py           300-trial held-out benchmark (re-exports scenarios)
+ablation.py            Watchdog vs State Engine comparison
+heldout_eval.py        Dev/test split + frozen config evaluation
+robustness_eval.py     4-dimension robustness attack
+extreme_tests.py       4 extreme scenario demos
+test_product.py        Product test suite (unittest, offline)
+```
+
+## What's next
+
+The silent-corruption gap points at the next layer: a Task Contract that defines which files the agent should touch and what the expected end-state looks like. When the agent's actual filesystem mutations diverge from the contract, that's a scope violation — regardless of whether any tool call returned an error. This moves the State Engine from "what is the agent doing" to "what is the agent doing to the world, and is that still allowed."

edward_guard-0.1.1/edward/__init__.py

@@ -0,0 +1,8 @@
+"""edward — external control plane for AI coding agents."""
+
+__version__ = "0.1.1"
+
+from .config import Policy, PolicyError, load_policy  # noqa: F401
+from .engine import ControlPlane, Decision  # noqa: F401
+from .audit import AuditLog  # noqa: F401
+from .scorer import Scorer  # noqa: F401

edward_guard-0.1.1/edward/audit.py

@@ -0,0 +1,112 @@
+"""Append-only JSONL audit log.
+
+Never blocks or crashes the control plane: if the disk write fails, the
+entry is echoed to stderr once and monitoring continues. Rotates at 50 MB
+by renaming to <name>.1.
+"""
+
+import json
+import os
+import sys
+import time
+from pathlib import Path
+
+
+MAX_BYTES = 50 * 1024 * 1024
+SCHEMA_VERSION = 1
+
+
+class AuditLog:
+    def __init__(self, path=None):
+        self.path = Path(path) if path else None
+        self._degraded = False
+        if self.path:
+            try:
+                self.path.parent.mkdir(parents=True, exist_ok=True)
+            except OSError as exc:
+                self._degrade(f"cannot create audit dir {self.path.parent}: {exc}")
+
+    def _degrade(self, msg: str) -> None:
+        if not self._degraded:
+            self._degraded = True
+            print(f"[edward] audit degraded to stderr: {msg}", file=sys.stderr)
+
+    def emit(self, entry_type: str, session: str = "", **fields) -> None:
+        record = {
+            "ts": time.strftime("%Y-%m-%dT%H:%M:%S%z"),
+            "v": SCHEMA_VERSION,
+            "type": entry_type,
+            "session": session,
+        }
+        record.update(fields)
+        line = json.dumps(record, default=str, ensure_ascii=False)
+        if self.path is None:
+            return
+        try:
+            if self.path.exists() and self.path.stat().st_size > MAX_BYTES:
+                rotated = self.path.with_suffix(self.path.suffix + ".1")
+                os.replace(self.path, rotated)
+            with open(self.path, "a", encoding="utf-8") as fh:
+                fh.write(line + "\n")
+        except OSError as exc:
+            self._degrade(f"{exc}")
+            print(f"[edward] audit: {line}", file=sys.stderr)
+
+    def session_start(self, session: str, policy_preset: str, command) -> None:
+        self.emit("session_start", session=session, policy_preset=policy_preset,
+                  command=[str(c) for c in command])
+
+    def intervention(self, session: str, trigger_reason: str, decision_type: str,
+                     action: str, source: str, authority: str, mss: dict,
+                     jev: dict = None, est_avoided_usd: float = None) -> None:
+        self.emit("intervention", session=session, trigger_reason=trigger_reason,
+                  decision_type=decision_type, action=action, source=source,
+                  authority=authority, mss=mss, jev=jev or {},
+                  est_avoided_usd=est_avoided_usd)
+
+    def session_end(self, session: str, reason: str, exit_code: int) -> None:
+        self.emit("session_end", session=session, reason=reason, exit_code=exit_code)
+
+
+def summarize(path) -> dict:
+    """Read an audit file and aggregate the numbers that matter."""
+    path = Path(path)
+    summary = {
+        "file": str(path), "sessions": 0, "interventions": 0,
+        "by_action": {}, "by_decision_type": {}, "tokens_at_intervention": [],
+        "cost_usd_at_intervention": [], "est_avoided_usd": 0.0,
+        "first_ts": None, "last_ts": None,
+    }
+    sessions = set()
+    if not path.exists():
+        return summary
+    with open(path, "r", encoding="utf-8", errors="replace") as fh:
+        for raw in fh:
+            raw = raw.strip()
+            if not raw:
+                continue
+            try:
+                rec = json.loads(raw)
+            except json.JSONDecodeError:
+                continue
+            ts = rec.get("ts")
+            if ts:
+                summary["first_ts"] = summary["first_ts"] or ts
+                summary["last_ts"] = ts
+            if rec.get("session"):
+                sessions.add(rec["session"])
+            if rec.get("type") == "intervention":
+                summary["interventions"] += 1
+                action = rec.get("action", "?")
+                summary["by_action"][action] = summary["by_action"].get(action, 0) + 1
+                dtype = rec.get("decision_type", "?")
+                summary["by_decision_type"][dtype] = summary["by_decision_type"].get(dtype, 0) + 1
+                mss = rec.get("mss") or {}
+                if mss.get("token_usage"):
+                    summary["tokens_at_intervention"].append(mss["token_usage"])
+                if mss.get("cost_usd"):
+                    summary["cost_usd_at_intervention"].append(mss["cost_usd"])
+                if rec.get("est_avoided_usd") is not None:
+                    summary["est_avoided_usd"] += rec["est_avoided_usd"]
+    summary["sessions"] = len(sessions)
+    return summary