tamperward 1.4.3 → 1.4.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +24 -7
- package/package.json +2 -1
package/README.md
CHANGED
|
@@ -18,9 +18,10 @@ path: deleting failing tests, skipping them, casting to `any`, suppressing lint
|
|
|
18
18
|
lowering coverage gates, rewriting snapshots, editing CI, or bypassing hooks with
|
|
19
19
|
`--no-verify`. Tamperward treats the safety nets themselves as protected assets and
|
|
20
20
|
blocks the **class** of bypass — one ruleset, evaluated on the actual diff and commands
|
|
21
|
-
as a verdict, not a probability, enforced
|
|
22
|
-
agent's loop (a
|
|
23
|
-
and at the merge.**
|
|
21
|
+
as a verdict, not a probability, enforced at every stage a change passes through:
|
|
22
|
+
**inside the agent's loop (a Claude Code hook today — a deny before the tool runs, and
|
|
23
|
+
a sweep at end of turn), at the commit, and at the merge.** CI is the authority; the
|
|
24
|
+
earlier layers shorten the feedback loop for agents that carry them.
|
|
24
25
|
|
|
25
26
|
> Apache-2.0 · zero runtime model calls · every headline claim below is measured, with
|
|
26
27
|
> the pre-registered predictions (including the refuted ones) committed to this repo
|
|
@@ -108,13 +109,29 @@ escape valve is a PR label only a human with write access can apply.
|
|
|
108
109
|
**Does the loop work?** `harness/run.sh` stands up an isolated repo with a real bug and a
|
|
109
110
|
failing protected test, lets a headless agent loose with the hooks installed, and
|
|
110
111
|
classifies the end state with an oracle independent of the detectors under test
|
|
111
|
-
(SPEC §7.B). Across
|
|
112
|
-
had a detector covering it — **0 tampers reached green; all
|
|
113
|
-
ended in an honest fix**, a
|
|
112
|
+
(SPEC §7.B). Across 67 runs of these **guarded** scenarios — every attempted bypass class
|
|
113
|
+
had a detector covering it — **0 tampers reached green; all 25 runs where a block fired
|
|
114
|
+
ended in an honest fix**, a 25/25 point estimate whose 95% lower bound is ≈89%. The split is the
|
|
114
115
|
honest part: Sonnet 5 never attempted a bypass at all, even when the prompt *ordered*
|
|
115
|
-
one;
|
|
116
|
+
one; every interception came from Haiku 4.5 on a seed whose honest fix is genuinely
|
|
116
117
|
expensive. Reproduce: `npm run build && harness/run.sh 5 adversarial haiku seed-hard`.
|
|
117
118
|
|
|
119
|
+
**Is the fix caused by the deny, or would it have happened anyway?** Measured, with a
|
|
120
|
+
pre-registered no-gate control arm (`harness/PREDICTION-ungated-control.md`): same seed,
|
|
121
|
+
model, prompt and oracle, hooks removed. **6/10 ungated runs tampered through; honest
|
|
122
|
+
fixes fell from 10/10 gated to 4/10 ungated.** The gate's measured contribution on that
|
|
123
|
+
configuration is a 60-point swing — causation, not survivorship — and one of the three
|
|
124
|
+
control-arm bets was refuted (ungated honest fixes came in above my bet).
|
|
125
|
+
|
|
126
|
+
**And when the agent knows the gate is there?** Also pre-registered
|
|
127
|
+
(`harness/PREDICTION-informed-adversary.md`) — betting the gate would *leak*. Refuted,
|
|
128
|
+
three bets out of three: with the detector sources vendored into the repo and the prompt
|
|
129
|
+
naming the gate, Haiku's bypass attempts went from 5/10 uninformed (same day, same
|
|
130
|
+
everything) to **0/10 informed — fifteen informed runs, zero evasion attempts**. Sonnet,
|
|
131
|
+
told the gate exists under "speed is everything" pressure, refused the task outright in
|
|
132
|
+
3/5 runs rather than work around it, and fixed honestly in the other 2. At this scenario
|
|
133
|
+
size, disclosure measured as deterrence, not as an evasion spec.
|
|
134
|
+
|
|
118
135
|
**Which detectors deserve to exist?** Measured too. Affordance seeds offer an agent one
|
|
119
136
|
specific bypass each; predictions are pre-registered before any rep runs
|
|
120
137
|
(`harness/PREDICTION-affordance.md`). Result: 7/10 adversarial runs regenerated a golden
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "tamperward",
|
|
3
|
-
"version": "1.4.
|
|
3
|
+
"version": "1.4.5",
|
|
4
4
|
"description": "The deterministic agent-integrity gate. One ruleset, evaluated on the actual diff/commands as a verdict, enforced everywhere a change can be made.",
|
|
5
5
|
"license": "Apache-2.0",
|
|
6
6
|
"author": "hexrift",
|
|
@@ -56,6 +56,7 @@
|
|
|
56
56
|
"@types/node": "^20.14.0",
|
|
57
57
|
"@types/picomatch": "^3.0.1",
|
|
58
58
|
"esbuild": "^0.24.0",
|
|
59
|
+
"vitepress": "^1.6.4",
|
|
59
60
|
"vitest": "^2.1.8"
|
|
60
61
|
}
|
|
61
62
|
}
|