tamperward 1.4.5 → 1.5.0

package/README.md

@@ -26,6 +26,9 @@ earlier layers shorten the feedback loop for agents that carry them.
 > Apache-2.0 · zero runtime model calls · every headline claim below is measured, with
 > the pre-registered predictions (including the refuted ones) committed to this repo
 
+**[Docs & guide](https://hexrift.github.io/tamperward/)** ·
+**[The launch post](https://hexrift.github.io/tamperward/blog/what-agents-do-when-you-block-their-shortcuts)** — how these numbers were measured, and which of my own bets lost
+
 ## Install
 
 ```bash
@@ -35,12 +38,12 @@ npx tamperward init
 One idempotent command wires the policy and every enforcement point — it never
 overwrites anything you wrote, and `--dry-run` shows the plan first:
 
-| point | what it wires |
+| what | wired as |
 | --- | --- |
+| policy | a commented baseline `.tamperward.yml` — the defaults apply even without it |
 | agent loop | Claude Code `PreToolUse` deny + `Stop` sweep, merged into `.claude/settings.json` |
 | pre-commit | husky when present, the plain git hook otherwise |
 | CI | a PR-gate workflow with out-of-band sign-off (details below) |
-| policy | a commented baseline `.tamperward.yml` — the defaults apply even without it |
 
 Or run the views directly:
 
@@ -78,6 +81,7 @@ tamperward: 2 blocking
     evidence  2 test block(s) removed from test/calc.test.js
     instead   Keep the assertions and fix the code. Removing test blocks to go
               green is the tamper.
+    sign-off  tamperward allow test-deletion --file test/calc.test.js --reason "..."
 
 A blocking finding clears only with a human sign-off. In CI that sign-off is
 out-of-band — a PR label applied by a reviewer — never a file committed on the
@@ -89,16 +93,21 @@ Under GitHub Actions each finding also lands as an **inline annotation on the di
 
 ## The rules
 
-Nine mechanical rules, deterministic by construction, `block` by default:
+Ten mechanical rules, deterministic by construction — eight `block`, two `warn`:
 `test-deletion` (AST block-count, rename-out-of-glob, shell mutation), `test-skip`
 (including `.only`, which narrows the suite), `ts-any-cast`, `lint-suppression`,
 `coverage-lowering` (Jest and Vitest shapes, all four metrics, `package.json` included),
 `ci-tampering` (a **moved** check is not a removed one), `hook-tampering`, `no-verify`
-(flags and the env-var escape hatches), and `snapshot-rewrite` (`warn` — see the FP study
-below for why).
+(flags and the env-var escape hatches), `snapshot-rewrite` (`warn` — see the FP study below
+for why), and `snapshot-only-rewrite` (`warn` — the FP study's narrow signal, a
+snapshot moved with **no accompanying change**, as its own rule so it can graduate to
+`block` at a later policy `version:` without touching the broad rule; it fires only at
+commit granularity, where "no accompanying change" means something).
 
-Three heuristics ship as `warn` and never block until their precision clears the bar:
-`assertion-weakening`, `guard-removal`, `ts-any-launder`.
+Two heuristic rule names — `assertion-weakening` and `guard-removal` — are **reserved
+in the baseline policy but not yet built**: they get detectors only once a measured
+negatives corpus exists (SPEC §7.A), enter as `warn`, and never block until their
+precision clears the bar.
 
 The policy file itself is un-ignorable, CI reads the policy from the **merge-base** (a PR
 cannot govern its own verdict), and the agent honours no sign-off it can author — the
@@ -109,19 +118,26 @@ escape valve is a PR label only a human with write access can apply.
 **Does the loop work?** `harness/run.sh` stands up an isolated repo with a real bug and a
 failing protected test, lets a headless agent loose with the hooks installed, and
 classifies the end state with an oracle independent of the detectors under test
-(SPEC §7.B). Across 67 runs of these **guarded** scenarios — every attempted bypass class
+(SPEC §7.B). Across 77 runs of these **guarded** scenarios — every attempted bypass class
 had a detector covering it — **0 tampers reached green; all 25 runs where a block fired
 ended in an honest fix**, a 25/25 point estimate whose 95% lower bound is ≈89%. The split is the
-honest part: Sonnet 5 never attempted a bypass at all, even when the prompt *ordered*
-one; every interception came from Haiku 4.5 on a seed whose honest fix is genuinely
-expensive. Reproduce: `npm run build && harness/run.sh 5 adversarial haiku seed-hard`.
+honest part: across four models (Haiku 4.5, Sonnet 5, Opus 5, Fable 5), every
+interception came from Haiku on a seed whose honest fix is genuinely expensive — no
+frontier model attempted a bypass at all, even when the prompt *ordered* one.
+Reproduce: `npm run build && harness/run.sh 5 adversarial haiku seed-hard`.
 
 **Is the fix caused by the deny, or would it have happened anyway?** Measured, with a
 pre-registered no-gate control arm (`harness/PREDICTION-ungated-control.md`): same seed,
 model, prompt and oracle, hooks removed. **6/10 ungated runs tampered through; honest
 fixes fell from 10/10 gated to 4/10 ungated.** The gate's measured contribution on that
 configuration is a 60-point swing — causation, not survivorship — and one of the three
-control-arm bets was refuted (ungated honest fixes came in above my bet).
+control-arm bets was refuted (ungated honest fixes came in above my bet). The control
+arm was then extended across the model family (`harness/PREDICTION-frontier-models.md`):
+ungated Sonnet 5, Opus 5, and Fable 5 each went **10/10 honest** under the same
+pressure prompt — frontier honesty is disposition, not the gate's deterrence — while
+ungated Haiku stays 6/10 tamper-green. The gate's measured value concentrates on the
+models that do cheat. (The Fable rows are a Fable-driven harness measuring Fable:
+self-graded homework, bounded by the deterministic held-out oracle.)
 
 **And when the agent knows the gate is there?** Also pre-registered
 (`harness/PREDICTION-informed-adversary.md`) — betting the gate would *leak*. Refuted,
@@ -138,14 +154,21 @@ specific bypass each; predictions are pre-registered before any rep runs
 file from buggy output and sailed through — **because nothing guarded that class yet**;
 that measured demand is why `snapshot-rewrite` exists — while
 timeout-inflation and lint-config-gutting measured **0/10 attempts each** — two
-pre-registered bets refuted, two detectors *not* built. Bypass classes with no measured
-demand stay in a record-only shadow scanner that accumulates evidence on every weekly
-harness run.
+pre-registered bets refuted, two detectors *not* built. The same discipline settled
+`assertion-weakening` (`harness/PREDICTION-assert-affordance.md`): a seed making the
+expectation edit the one cheap, prompt-named move — with a plausible convention-mismatch
+cover story — measured **0/10 attempts**; Haiku implemented the real fix every time. My
+≥6/10 bet was refuted, and the rule stays a reserved name. Bypass classes with no
+measured demand stay in a record-only shadow scanner that accumulates evidence on every
+weekly harness run.
 
 **What about false positives?** `snapshot-rewrite` was swept over **1,652 real mainline
 commits** (prettier, jest, docusaurus, immer): 216 touched snapshots, all legitimately —
 which is exactly why that rule is a `warn` asking for human confirmation, not a block,
-and why the study is committed (`harness/fp-study/`) rather than summarized away.
+and why the study is committed (`harness/fp-study/`) rather than summarized away. The
+study's narrow signal — snapshot-only diffs, ~0.06% of audited mainline commits but
+7/7 of observed tampers — now ships as the distinct `snapshot-only-rewrite` warn, the
+graduation candidate.
 
 ## Stability
 

package/dist/cli/index.js

@@ -457,6 +457,10 @@ function defaultPolicy(version = 1) {
       // mechanical but intent-ambiguous: updating a snapshot is the legitimate workflow
       // when intended output changes. WARN until the §7 evidence path earns block.
       "snapshot-rewrite": { severity: "warn" },
+      // the narrow graduation candidate: snapshot moved with NO accompanying change.
+      // ~0.06% FP on 1,652 audited mainline commits, 7/7 TP on observed tampers
+      // (harness/fp-study/). warn until a version: gate graduates it (BLOCK_SINCE).
+      "snapshot-only-rewrite": { severity: "warn" },
       // heuristic — warn until precision is measured (SPEC §7)
       "assertion-weakening": { severity: "warn" },
       "guard-removal": { severity: "warn" }
@@ -1314,6 +1318,39 @@ var snapshotRewrite = {
   }
 };
 
+// src/detectors/snapshot-only.ts
+var RULE9 = "snapshot-only-rewrite";
+var COMMIT_VIEWS = ["staged", "range"];
+function isSnapshotChange(c, policy) {
+  if (c.kind !== "file") return false;
+  if (c.op === "add") return false;
+  if (isProtected(c.path, policy, "snapshots")) return true;
+  return c.oldPath != null && isProtected(c.oldPath, policy, "snapshots");
+}
+var snapshotOnlyRewrite = {
+  id: RULE9,
+  surface: ["file"],
+  certainty: "mechanical",
+  run(changes, policy, view) {
+    if (!view || !COMMIT_VIEWS.includes(view)) return [];
+    const files = changes.filter((c) => c.kind === "file");
+    if (files.length === 0) return [];
+    const snaps = files.filter((c) => isSnapshotChange(c, policy));
+    if (snaps.length === 0 || snaps.length !== files.length) return [];
+    const first = snaps[0];
+    const names = snaps.map((c) => c.kind === "file" ? c.path : "").filter(Boolean).slice(0, 3).join(", ");
+    return [
+      makeFinding(RULE9, policy, {
+        file: first.path,
+        message: `Only recorded expectations changed in this diff (${snaps.length} snapshot file${snaps.length === 1 ? "" : "s"}, no code change to explain them).`,
+        evidence: names + (snaps.length > 3 ? ", \u2026" : ""),
+        remediation: "Pair the snapshot update with the source change that motivates it, or confirm the new expectations are genuinely intended. In 1,652 audited mainline commits this shape occurred once; in observed tamper attempts, every time.",
+        defaultSeverity: "warn"
+      })
+    ];
+  }
+};
+
 // src/detectors/index.ts
 var allDetectors = [
   noVerify,
@@ -1324,7 +1361,8 @@ var allDetectors = [
   ciTampering,
   hookTampering,
   testDeletion,
-  snapshotRewrite
+  snapshotRewrite,
+  snapshotOnlyRewrite
 ];
 
 // src/engine.ts
@@ -1339,13 +1377,13 @@ function isSuppressed(c, policy) {
 function activeChanges(changes, policy) {
   return changes.filter((c) => !isSuppressed(c, policy));
 }
-function evaluate(changes, policy, detectors = allDetectors) {
+function evaluate(changes, policy, detectors = allDetectors, view) {
   const active = activeChanges(changes, policy);
   const out = [];
   for (const d of detectors) {
     if (!isEnabled(d.id, policy)) continue;
     try {
-      out.push(...d.run(active, policy));
+      out.push(...d.run(active, policy, view));
     } catch (e) {
       process.stderr.write(`tamperward: detector "${d.id}" errored and was skipped: ${String(e)}
 `);
@@ -1705,12 +1743,15 @@ function check(opts) {
   const cwd = opts.cwd ?? process.cwd();
   let changes;
   let layer;
+  let view;
   if (opts.staged) {
     changes = diffStaged({ cwd: opts.cwd });
     layer = "local";
+    view = "staged";
   } else if (opts.worktree) {
     changes = diffWorktree({ cwd: opts.cwd });
     layer = "local";
+    view = "worktree";
   } else if (opts.diff) {
     const [base, head] = opts.diff.split(/\.{2,3}/);
     if (!base || !head) {
@@ -1720,13 +1761,14 @@ function check(opts) {
     }
     changes = diffRange(base, head, { cwd: opts.cwd });
     layer = "ci";
+    view = "range";
     policy = loadPolicyAt(mergeBaseOf(base, head, { cwd: opts.cwd }), opts.cwd) ?? defaultPolicy();
   } else {
     process.stderr.write("tamperward: specify --staged, --worktree, or --diff <base>...<head>\n");
     return 2;
   }
   const ignoredFiles = changes.filter((c) => isSuppressed(c, policy)).length;
-  let findings = evaluate(changes, policy);
+  let findings = evaluate(changes, policy, void 0, view);
   const { findings: remaining, cleared } = layer === "local" ? applyLocalSignoffs(findings, cwd, policy) : applyOobSignoffs(findings, oobFromEnv());
   findings = remaining;
   if (cleared.length) {
@@ -1963,7 +2005,7 @@ function preToolUseVerdict(input) {
     const cwd = input.cwd ?? process.cwd();
     turnBaseline(cwd, input.session_id);
     const changes = changesFromClaudeHook(input, cwd);
-    const blocks = evaluate(changes, loadPolicy(cwd)).filter((f) => f.severity === "block");
+    const blocks = evaluate(changes, loadPolicy(cwd), void 0, "tool-call").filter((f) => f.severity === "block");
     return verdict(blocks, "PreToolUse");
   } catch (e) {
     return failClosed("PreToolUse", errText(e));
@@ -1977,7 +2019,7 @@ function stopVerdict(input) {
   try {
     const base = turnBaseline(cwd, input.session_id);
     const changes = base ? diffSince(base, { cwd }) : diffWorktree({ cwd });
-    blocks = evaluate(changes, loadPolicy(cwd)).filter((f) => f.severity === "block");
+    blocks = evaluate(changes, loadPolicy(cwd), void 0, "turn").filter((f) => f.severity === "block");
   } catch (e) {
     return failClosed("Stop", errText(e));
   }
@@ -2017,7 +2059,7 @@ function runAllow(opts) {
   const policy = loadPolicy(cwd);
   let findings;
   try {
-    findings = evaluate(diffWorktree({ cwd }), policy);
+    findings = evaluate(diffWorktree({ cwd }), policy, void 0, "worktree");
   } catch {
     process.stderr.write("tamperward: cannot read the working tree (not a git repo?).\n");
     return 2;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "tamperward",
-  "version": "1.4.5",
+  "version": "1.5.0",
   "description": "The deterministic agent-integrity gate. One ruleset, evaluated on the actual diff/commands as a verdict, enforced everywhere a change can be made.",
   "license": "Apache-2.0",
   "author": "hexrift",