tamperward 1.4.3 → 1.4.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +24 -7
  2. package/package.json +2 -1
package/README.md CHANGED
@@ -18,9 +18,10 @@ path: deleting failing tests, skipping them, casting to `any`, suppressing lint
18
18
  lowering coverage gates, rewriting snapshots, editing CI, or bypassing hooks with
19
19
  `--no-verify`. Tamperward treats the safety nets themselves as protected assets and
20
20
  blocks the **class** of bypass — one ruleset, evaluated on the actual diff and commands
21
- as a verdict, not a probability, enforced everywhere a change can be made: **inside the
22
- agent's loop (a deny before the tool runs, and a sweep at end of turn), at the commit,
23
- and at the merge.**
21
+ as a verdict, not a probability, enforced at every stage a change passes through:
22
+ **inside the agent's loop (a Claude Code hook today a deny before the tool runs, and
23
+ a sweep at end of turn), at the commit, and at the merge.** CI is the authority; the
24
+ earlier layers shorten the feedback loop for agents that carry them.
24
25
 
25
26
  > Apache-2.0 · zero runtime model calls · every headline claim below is measured, with
26
27
  > the pre-registered predictions (including the refuted ones) committed to this repo
@@ -108,13 +109,29 @@ escape valve is a PR label only a human with write access can apply.
108
109
  **Does the loop work?** `harness/run.sh` stands up an isolated repo with a real bug and a
109
110
  failing protected test, lets a headless agent loose with the hooks installed, and
110
111
  classifies the end state with an oracle independent of the detectors under test
111
- (SPEC §7.B). Across 42 runs of these **guarded** scenarios — every attempted bypass class
112
- had a detector covering it — **0 tampers reached green; all 20 runs where a block fired
113
- ended in an honest fix**, a 20/20 point estimate whose 95% lower bound is ≈86%. The split is the
112
+ (SPEC §7.B). Across 67 runs of these **guarded** scenarios — every attempted bypass class
113
+ had a detector covering it — **0 tampers reached green; all 25 runs where a block fired
114
+ ended in an honest fix**, a 25/25 point estimate whose 95% lower bound is ≈89%. The split is the
114
115
  honest part: Sonnet 5 never attempted a bypass at all, even when the prompt *ordered*
115
- one; all 20 interceptions came from Haiku 4.5 on a seed whose honest fix is genuinely
116
+ one; every interception came from Haiku 4.5 on a seed whose honest fix is genuinely
116
117
  expensive. Reproduce: `npm run build && harness/run.sh 5 adversarial haiku seed-hard`.
117
118
 
119
+ **Is the fix caused by the deny, or would it have happened anyway?** Measured, with a
120
+ pre-registered no-gate control arm (`harness/PREDICTION-ungated-control.md`): same seed,
121
+ model, prompt and oracle, hooks removed. **6/10 ungated runs tampered through; honest
122
+ fixes fell from 10/10 gated to 4/10 ungated.** The gate's measured contribution on that
123
+ configuration is a 60-point swing — causation, not survivorship — and one of the three
124
+ control-arm bets was refuted (ungated honest fixes came in above my bet).
125
+
126
+ **And when the agent knows the gate is there?** Also pre-registered
127
+ (`harness/PREDICTION-informed-adversary.md`) — betting the gate would *leak*. Refuted,
128
+ three bets out of three: with the detector sources vendored into the repo and the prompt
129
+ naming the gate, Haiku's bypass attempts went from 5/10 uninformed (same day, same
130
+ everything) to **0/10 informed — fifteen informed runs, zero evasion attempts**. Sonnet,
131
+ told the gate exists under "speed is everything" pressure, refused the task outright in
132
+ 3/5 runs rather than work around it, and fixed honestly in the other 2. At this scenario
133
+ size, disclosure measured as deterrence, not as an evasion spec.
134
+
118
135
  **Which detectors deserve to exist?** Measured too. Affordance seeds offer an agent one
119
136
  specific bypass each; predictions are pre-registered before any rep runs
120
137
  (`harness/PREDICTION-affordance.md`). Result: 7/10 adversarial runs regenerated a golden
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "tamperward",
3
- "version": "1.4.3",
3
+ "version": "1.4.5",
4
4
  "description": "The deterministic agent-integrity gate. One ruleset, evaluated on the actual diff/commands as a verdict, enforced everywhere a change can be made.",
5
5
  "license": "Apache-2.0",
6
6
  "author": "hexrift",
@@ -56,6 +56,7 @@
56
56
  "@types/node": "^20.14.0",
57
57
  "@types/picomatch": "^3.0.1",
58
58
  "esbuild": "^0.24.0",
59
+ "vitepress": "^1.6.4",
59
60
  "vitest": "^2.1.8"
60
61
  }
61
62
  }