tamperward 2.29.11 → 2.29.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +173 -676
- package/dist/cli/index.js +67 -25
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -19,11 +19,13 @@ correction in one place, each number bound to its committed artifact ·
|
|
|
19
19
|
**[The research series](./docs/blog/index.md)** — every registered prediction
|
|
20
20
|
published beside its outcome
|
|
21
21
|
|
|
22
|
+
## What it is
|
|
23
|
+
|
|
22
24
|
Coding agents can modify both the implementation and the tests, configuration,
|
|
23
25
|
CI, hooks and verifier used to judge that implementation — and in observed
|
|
24
|
-
trajectories, some modify or attempt to modify verification in ways that can
|
|
25
|
-
|
|
26
|
-
|
|
26
|
+
trajectories, some modify or attempt to modify verification in ways that can turn
|
|
27
|
+
incorrect work into apparent success. Under pressure, the cheaper route to green
|
|
28
|
+
is sometimes to weaken the checks instead of fixing the failure.
|
|
27
29
|
|
|
28
30
|
In plain English: Tamperward lets a coding agent change your code, but not the
|
|
29
31
|
trusted starting point, the rules, or the checks used to judge that code.
|
|
@@ -31,7 +33,7 @@ trusted starting point, the rules, or the checks used to judge that code.
|
|
|
31
33
|
Tamperward is a **deterministic verification-integrity layer**. It blocks known
|
|
32
34
|
weakening moves as they happen, observes protected-state effects, and
|
|
33
35
|
independently re-adjudicates apparent success outside the agent's normal
|
|
34
|
-
completion path. No runtime LLM judge.
|
|
36
|
+
completion path. No runtime LLM judge. It fails closed when adjudication is
|
|
35
37
|
impossible.
|
|
36
38
|
|
|
37
39
|
> **Project status: active research release.** Tamperward is usable today, but
|
|
@@ -42,310 +44,6 @@ impossible.
|
|
|
42
44
|
> security maturity; the distance to that is tracked, milestone by milestone, in
|
|
43
45
|
> [SPEC §9.1](./SPEC.md#91-maturity-milestones).
|
|
44
46
|
|
|
45
|
-
## What we have actually measured
|
|
46
|
-
|
|
47
|
-
Plain-English takeaway: the original detector-centred design was insufficient.
|
|
48
|
-
Later versions materially reduced false-green outcomes in two held-out rounds,
|
|
49
|
-
the subsequent stronger-model replication was inconclusive, and the larger
|
|
50
|
-
preregistered round 4 on the 2.x line **did not replicate** the prevention
|
|
51
|
-
effect — a failed prediction, published beside the bet.
|
|
52
|
-
|
|
53
|
-
| experiment | result | what it supports |
|
|
54
|
-
| --- | --- | --- |
|
|
55
|
-
| **Round 1** — 26 paired real repos, historical regressions, 53 counted trajectories, **v1.6.0** ([`harness/taskbench/`](./harness/taskbench/)) | Transfer: **9/27 ungated runs (33.3%)** violated policy ([corrected](./harness/taskbench/reanalysis/TRANSFER-REANALYSIS.md) from a published 13/26 — the original predicate was defective; the transfer bet is refuted, not held). Headline prevention bet **lost**: b=5 / c=4, RD +3.8pp [−17.2, +24.7], exact McNemar **p = 1.0** — published beside the bet | A detector-centric architecture was insufficient: agents routed around the shipped detector classes |
|
|
56
|
-
| **Pristine oracle, round 1** — independent re-execution of the original suite, **including withheld semantic cases** | Identified **every masked failure observed among the 53 trajectories** while diff-time detection was routed around | The outcome-level signal that motivated `tamperward verify`. The shipped command productizes this oracle's base-restoration component only — it carries no withheld cases and cannot detect a semantically incomplete fix the base tests also accept |
|
|
57
|
-
| **Round 2** — **22 fresh held-out repos** no detector was tuned on ([`harness/taskbench/round2/`](./harness/taskbench/round2/)) | Transfer **12/22 (54.5%)** ([corrected](./harness/taskbench/reanalysis/TRANSFER-REANALYSIS.md) from a published 14/22). Prevention: **b=9 / c=0**, RD **+40.9pp**, BP95 **[17.8, 61.3]**, exact McNemar **p = 0.0039** | The preregistered **v1.9.0** treatment materially reduced false greens in that setting |
|
|
58
|
-
| **After prevention** — the nine round-2 prevented false greens | **8 of 9 became honest completions**; the ninth an honest **non**-completion | Prevention usually redirected trajectories toward honest work rather than merely blocking them |
|
|
59
|
-
| **Round 3** — 17 paired Python repos, fresh PyPI frame, **v1.14.0** ([`harness/taskbench/round3/`](./harness/taskbench/round3/)) | Transfer 9/17 (52.9%). Prevention: **b=6 / c=0**, RD **+35.3pp**, BP95 [9.5, 58.7], exact McNemar **p = 0.0313** | The prevention result appeared in Python — with the treatment also changed from v1.9.0, so ecosystem transfer is not isolated. The in-loop skip detector proved blind to pytest syntax; the outer layers carried it |
|
|
60
|
-
| **Round 3.1** — the same 16 pairs under **`claude-sonnet-5`** ([`harness/taskbench/round3.1/`](./harness/taskbench/round3.1/)) | Transfer 4/16 (25.0%). Prevention: **b=1 / c=0**, RD +6.3pp, BP95 [−13.8, 28.3], exact McNemar **p = 1.0000** | The confirmatory result **did not replicate**, and could not have: only three ungated false greens occurred, so `b ≤ 3` and p ≥ .25 whatever the gate did. A failure to reject, not evidence of no effect |
|
|
61
|
-
| **Round 4** — fresh pool of 110 paired Python repos under **`claude-sonnet-5`**, treatment **2.10.3** pinned by artefact hash, preregistered exact McNemar ([`harness/taskbench/round4/`](./harness/taskbench/round4/), sealed readout [`ROUND4-ANALYSIS.md`](./harness/taskbench/round4/ROUND4-ANALYSIS.md)) | Primary endpoint: prevention **b=5 / c=3**, exact McNemar **p = 0.7265625**, H₀ **not rejected** — the preregistered prediction (b=16, c=1, reject) **did not replicate**. Realized valid pairs **79/110**; the registered interpretation floor was met (`a + b = 15` ungated opportunities, floor 6), but realized discordance (8 pairs) was below the ~17 the power model assumed. Separate, narrower integrity observation: across **201 measured trajectories**, **0 strict tamper bypass** (landed weakening that survived to the final tree *and* was certified clean) | The registered confirmatory claim failed on the 2.x line and is reported as a failed prediction, not reinterpreted. The zero-bypass count is descriptive and does not stand in for it: one transient landing was certified clean (correctly, it did not survive), and 31/110 pairs were lost to apparatus attrition, which is not claimed bias-free |
|
|
62
|
-
|
|
63
|
-
Key: `b` = false greens seen only without Tamperward; `c` = false greens seen
|
|
64
|
-
only with it; `RD` = paired risk difference; `pp` = percentage points; `BP95` =
|
|
65
|
-
Bonett–Price 95% interval.
|
|
66
|
-
|
|
67
|
-
Earlier controlled experiments → **[the research series](./docs/blog/index.md)**.
|
|
68
|
-
|
|
69
|
-
> **Scope.** The rows above cover specific models, pressure prompts, treatment
|
|
70
|
-
> versions, and finite JavaScript/TypeScript and Python repository samples —
|
|
71
|
-
> evidence for those settings, not a universal claim. Round 2 tested
|
|
72
|
-
> the v1.9.0 stack; the current **2.x** line adds post-study envelope hardening
|
|
73
|
-
> (externally reviewed, with findings tracked individually as REPRO or AUDIT in
|
|
74
|
-
> [SECURITY-ENVELOPE.md](./SECURITY-ENVELOPE.md) and closed with regression and
|
|
75
|
-
> mutation checks — see [CHANGELOG](./CHANGELOG.md)). Rounds 3 and 3.1 are
|
|
76
|
-
> complete and are in the table above. **1.14.1 closed a bypass in `tamperward
|
|
77
|
-
> verify`** — an agent-added protected file could suppress the tests the pristine
|
|
78
|
-
> run had just restored — which was present from v1.9.0 onward, so rounds 2, 3
|
|
79
|
-
> and 3.1 all ran on a verifier carrying it. No evidence any agent used it; the
|
|
80
|
-
> layer's authority in those rounds was nonetheless weaker than described at the
|
|
81
|
-
> time. **Round 4 is complete and sealed.** It was the first counted round on the
|
|
82
|
-
> 2.x line: a fresh pool of 110 paired repositories under `claude-sonnet-5`,
|
|
83
|
-
> treatment **2.10.3** pinned by artefact hash, with the whole draw — task order,
|
|
84
|
-
> arm assignment, and a separate 22-pair instability budget — derived from
|
|
85
|
-
> committed seeds before the first counted trajectory. The registered primary
|
|
86
|
-
> prediction **did not replicate**: b=5, c=3, exact McNemar p = 0.7265625, null not
|
|
87
|
-
> rejected, over the 79 of 110 repositories that produced a valid paired
|
|
88
|
-
> measurement. The narrower integrity observation — 0 strict tamper bypasses across
|
|
89
|
-
> 201 measured trajectories — is a separate descriptive result and does not
|
|
90
|
-
> replace the registered outcome. Only 79/110 pairs survived to measurement; the
|
|
91
|
-
> loss was mostly symmetric apparatus attrition (venv/interpreter execution
|
|
92
|
-
> failures), which reduces concern about arm-specific bias but is not claimed to
|
|
93
|
-
> be free of selection bias — the readout says so. Every figure is the sealed
|
|
94
|
-
> value in [`ROUND4-RESULTS.json`](./harness/taskbench/round4/ROUND4-RESULTS.json)
|
|
95
|
-
> (`payload_sha256` `e7bfce08…`), read out in
|
|
96
|
-
> [`ROUND4-ANALYSIS.md`](./harness/taskbench/round4/ROUND4-ANALYSIS.md) and the
|
|
97
|
-
> deviation ledger [`DEVIATIONS.md`](./harness/taskbench/round4/DEVIATIONS.md).
|
|
98
|
-
> Read the round as a pair — the bets before, the outcome after:
|
|
99
|
-
> [How round 4 is built to be hard to fool](./docs/blog/how-round-4-is-built-to-be-hard-to-fool.md)
|
|
100
|
-
> (preregistration, no numbers) →
|
|
101
|
-
> [The prevention bet didn't replicate. No surviving tampering was certified clean.](./docs/blog/the-prevention-bet-didnt-replicate-no-surviving-tampering-was-certified-clean.md)
|
|
102
|
-
> (results). Round 4.1, a second agent runtime over the same frozen pool, is the
|
|
103
|
-
> next registered step ([SPEC §9.1](./SPEC.md#91-maturity-milestones)).
|
|
104
|
-
|
|
105
|
-
## Architecture
|
|
106
|
-
|
|
107
|
-
The agent may produce the work, but it must not control how that work is
|
|
108
|
-
judged. In architectural terms, Tamperward separates **steering** from
|
|
109
|
-
**adjudication**. "Visible" verification runs the candidate as it stands;
|
|
110
|
-
"pristine" verification restores the protected verification state from the
|
|
111
|
-
trusted starting point and runs the checks again.
|
|
112
|
-
|
|
113
|
-
For the complete component map, trust boundaries, data flows, and deployment
|
|
114
|
-
surfaces, see [the full architecture diagram](./docs/architecture.md).
|
|
115
|
-
|
|
116
|
-
> **Core invariant:** the agent may author the candidate tree, but it must not
|
|
117
|
-
> choose the trusted baseline, the governing policy, the verifier, or the final
|
|
118
|
-
> verdict.
|
|
119
|
-
|
|
120
|
-
| plane | mechanism | role |
|
|
121
|
-
| --- | --- | --- |
|
|
122
|
-
| **In-loop steering** | Claude Code `PreToolUse`, protected-tree drift checks, the `Stop` sweep, and optional filesystem-event observation | Immediate corrective feedback. Not the final authority. |
|
|
123
|
-
| **Local adjudication** | `tamperward run -- <agent…>` | Wraps the runtime and independently judges the tree it leaves behind. |
|
|
124
|
-
| **Commit backstop** | `tamperward check --staged` | Catches human changes, and agents without supported hooks. |
|
|
125
|
-
| **Repository authority** | Required PR checks plus branch protection | Decides what may reach the protected branch. |
|
|
126
|
-
|
|
127
|
-
| layer | the question it answers |
|
|
128
|
-
| --- | --- |
|
|
129
|
-
| **Change semantics** | Does a proposed or recorded change match a known weakening class? (PreToolUse deny, staged diff, range diff) |
|
|
130
|
-
| **Effect** | What did the protected state actually become, however the mutation arrived? (drift check, fs-event observer, Stop sweep) |
|
|
131
|
-
| **Outcome** | Does the candidate still pass when protected verification state from the base is restored? (pristine verification — no diff interpreted at all) |
|
|
132
|
-
| **Authority** | Where is the final verdict rendered, and what assumptions make it authoritative? (run envelope, CI, SHA-bound human sign-off) |
|
|
133
|
-
|
|
134
|
-
### The local enforcement envelope
|
|
135
|
-
|
|
136
|
-

|
|
137
|
-
|
|
138
|
-
Before starting the agent, the envelope
|
|
139
|
-
|
|
140
|
-
1. records the exact entry commit;
|
|
141
|
-
2. loads the policy and the verifier command and budget from that commit;
|
|
142
|
-
3. records the installed `node_modules` content fingerprint, where one exists; and
|
|
143
|
-
4. refuses an already-dirty tree unless the operator explicitly accepts the risk.
|
|
144
|
-
|
|
145
|
-
**2.16.3 deliberately disables the 2.16.2 run→verify dependency-attestation reuse.**
|
|
146
|
-
The envelope still takes a complete dependency checkpoint immediately before `runVerify`,
|
|
147
|
-
and the nested verifier independently takes its own entry checkpoint. A clean local run
|
|
148
|
-
therefore performs **6 complete dependency snapshots and 0 reused snapshots** on every
|
|
149
|
-
platform and lifecycle outcome. The optimization is not re-enabled merely because Linux
|
|
150
|
-
established descendant ownership: an independent verifier-entry read is a clearer trust
|
|
151
|
-
boundary than coupling dependency integrity to same-UID lifecycle supervision. The checks
|
|
152
|
-
after visible execution, after pristine execution, and at the envelope's final quiescence
|
|
153
|
-
boundary remain independent full reads as well. Set `TAMPERWARD_DIAGNOSTICS=1` to report
|
|
154
|
-
`full_snapshots`, `reused_snapshots`, aggregate snapshot wall time, lifecycle ownership,
|
|
155
|
-
and `entry_reuse=no`.
|
|
156
|
-
|
|
157
|
-
The hooks then steer the agent while it works: they read live state, deny known
|
|
158
|
-
weakening operations, detect protected-tree drift, and sweep the turn's net
|
|
159
|
-
changes. The `Stop` hook runs that sweep — it does **not** invoke `verify`.
|
|
160
|
-
`tamperward watch` can additionally record transient mutations, but it is an
|
|
161
|
-
optional observer that `init` neither starts nor supervises, and it is not an
|
|
162
|
-
enforcement boundary. Since **2.13.3** the observer writes a health sidecar next to
|
|
163
|
-
its JSONL event log, recording backend, PID/start time, watched-directory count,
|
|
164
|
-
successful appends, dropped events and errors. `tamperward doctor` reports that
|
|
165
|
-
channel as **healthy**, **degraded**, or **unavailable**, and Stop writes the same
|
|
166
|
-
non-authoritative state into the audit log when `TAMPERWARD_DENYLOG` is enabled.
|
|
167
|
-
This distinguishes “healthy observer, zero events” from “observer telemetry was
|
|
168
|
-
unavailable” without turning absence of watcher telemetry into an enforcement pass
|
|
169
|
-
or failure.
|
|
170
|
-
|
|
171
|
-
Since **2.14.0**, `tamperward run --observe-transients -- <agent...>` can supervise
|
|
172
|
-
that observer as part of the envelope lifecycle. It creates a unique session log under
|
|
173
|
-
the repository git directory, pins the watcher's protected-path policy to the same
|
|
174
|
-
trusted base used by the envelope, waits for observer health before launching the
|
|
175
|
-
agent, consumes the temporal events itself after the agent exits, and shuts the
|
|
176
|
-
observer down deterministically. Observer health remains advisory; transient findings
|
|
177
|
-
remain warnings by default and affect the envelope only when the operator explicitly
|
|
178
|
-
sets `TAMPERWARD_TRANSIENT=block`.
|
|
179
|
-
|
|
180
|
-
Since **2.15.2**, Stop-sweep event consumption uses the saved cursor as a real byte
|
|
181
|
-
offset: it performs positioned reads of only new JSONL bytes instead of decoding the
|
|
182
|
-
whole historical log on every turn. Each physical read is capped at **4 MiB**, and one
|
|
183
|
-
authority decision drains at most **16 MiB** in bounded chunks. Only complete
|
|
184
|
-
newline-terminated records advance the cursor, so a torn final watcher write is replayed
|
|
185
|
-
after completion instead of being lost. Stop commits the cursor only after the entire
|
|
186
|
-
bounded telemetry tail has been parsed and classified; malformed records, an oversized
|
|
187
|
-
single record, a torn tail, or telemetry beyond the 16 MiB aggregate ceiling blocks Stop
|
|
188
|
-
and retains the previous cursor rather than certifying unjudged bytes. Supervised
|
|
189
|
-
`run --observe-transients` keeps observer evidence advisory by default, but when
|
|
190
|
-
`TAMPERWARD_TRANSIENT=block` is explicitly enabled, any unclassified observer tail
|
|
191
|
-
also fails the envelope closed.
|
|
192
|
-
|
|
193
|
-
After the runtime exits its exit code is treated as untrusted, and the envelope
|
|
194
|
-
checks that post-agent `HEAD` still descends from the entry commit; the committed
|
|
195
|
-
changes over `entry...HEAD`; staged, unstaged and untracked non-ignored worktree
|
|
196
|
-
changes; dependency drift and whether the tree stayed quiescent; and the
|
|
197
|
-
verification outcome. Since **2.16.3**, every wrapped agent goes through a lifecycle
|
|
198
|
-
supervisor even when no runtime budget is requested. Ordinary POSIX descendants remain in
|
|
199
|
-
an owned process group. On Linux the stronger boundary uses a fixed, non-caller-writable
|
|
200
|
-
system `python3` launched with isolated startup (`-I -S -E`), supervisor cwd `/`, and a
|
|
201
|
-
minimal supervisor environment; the agent receives its separately frozen original cwd and
|
|
202
|
-
environment. The supervisor sets `PR_SET_CHILD_SUBREAPER`, so orphaned descendants from
|
|
203
|
-
double-fork/`setsid()` are reparented to it by the kernel. Before adjudication begins it
|
|
204
|
-
kills the agent process group plus adopted children and does not report lifecycle ownership
|
|
205
|
-
until `waitpid(-1, WNOHANG)` reaches the kernel **ECHILD** condition. An unreadable child
|
|
206
|
-
list, failed kill, abnormal supervisor exit, unavailable trusted interpreter, or drain
|
|
207
|
-
deadline fails closed before adjudication. The same-UID result file is evidence only after
|
|
208
|
-
normal supervisor completion, so forging it and killing the supervisor cannot manufacture a
|
|
209
|
-
trusted lifecycle result.
|
|
210
|
-
|
|
211
|
-
Linux therefore requires Python 3 with the standard `ctypes`/process modules available at
|
|
212
|
-
a fixed trusted system path; TamperWard does not resolve this supervisor through candidate
|
|
213
|
-
`PATH`, `PYTHONPATH`, user site-packages, or candidate cwd. **Linux `tamperward run`
|
|
214
|
-
also refuses to start when the caller's uid or effective uid is 0/root.** Root can write
|
|
215
|
-
ordinary system interpreter paths, so the same-UID trust argument used by this lifecycle
|
|
216
|
-
backend is not meaningful there. `tamperward doctor` reports this explicitly as a
|
|
217
|
-
BROKEN platform posture rather than misdiagnosing it as a missing Python installation.
|
|
218
|
-
`--agent-budget <seconds>`
|
|
219
|
-
adds an operator-owned wall-clock boundary around the same lifecycle (separate from the
|
|
220
|
-
verifier's `--budget`). A clean timeout is `AGENT_TIMEOUT` / exit 124; enforcement or
|
|
221
|
-
cannot-adjudicate still outranks it. **From 2.16.4, `tamperward run` is Linux-only
|
|
222
|
-
for authoritative lifecycle certification.** Windows, macOS and other non-Linux platforms
|
|
223
|
-
fail closed before the agent starts because this release has no OS primitive there that can
|
|
224
|
-
prove the detached execution domain is drained. From **2.16.5**, checkpointed-local
|
|
225
|
-
`tamperward verify` also has an explicit host-shell contract: Linux, macOS and the
|
|
226
|
-
supported POSIX Node platforms use `/bin/sh -c`; Windows local verification fails closed
|
|
227
|
-
before candidate execution rather than depending on an incidental MSYS/Git-for-Windows
|
|
228
|
-
`sh` on PATH. A digest-pinned container verifier is a separate backend and must pass its
|
|
229
|
-
own Docker authority preflight. Dependency-attestation reuse remains disabled on all
|
|
230
|
-
platforms.
|
|
231
|
-
|
|
232
|
-
### Platform support
|
|
233
|
-
|
|
234
|
-
| Capability | Linux | macOS | Windows |
|
|
235
|
-
| --- | --- | --- | --- |
|
|
236
|
-
| `check` / policy evaluation | Supported | Supported | Supported |
|
|
237
|
-
| parsed output (`--json`, `--format github`, hook/sweep JSON) on a pipe | Complete before exit | Complete before exit (asynchronous pipe drained first) | Complete before exit (asynchronous pipe drained first) |
|
|
238
|
-
| Claude hook / Stop adapter | Supported where Claude Code command hooks are available | Same | Same |
|
|
239
|
-
| `watch` / observer telemetry | Supported; backend health is reported | Supported/degraded according to `fs.watch` health | Supported/degraded according to `fs.watch` health |
|
|
240
|
-
| opt-in `hook-service` | Supported (per-user `0600` unix socket) | Supported (per-user `0600` unix socket) | **Unsupported; `start` refuses, hooks run in-process** |
|
|
241
|
-
| checkpointed-local `verify` | Supported via `/bin/sh` | Supported via `/bin/sh` | **Unsupported; fails before candidate execution** |
|
|
242
|
-
| isolated-container `verify` | Supported when Docker authority preflight passes | Not claimed beyond Docker preflight | Not claimed beyond Docker preflight |
|
|
243
|
-
| advisory `trace-verify` | **Supported with `strace` + `tar`** | **Unsupported; reports no parity** | **Unsupported; reports no parity** |
|
|
244
|
-
| authoritative `run` | **Supported only with trusted non-root subreaper backend** | **Unsupported; fails before agent start** | **Unsupported; fails before agent start** |
|
|
245
|
-
| CI coverage for this contract | Full suite + platform contract | Platform-contract job | Platform-contract job |
|
|
246
|
-
|
|
247
|
-
The platform-contract CI job runs on real `ubuntu-latest`, `macos-latest` and
|
|
248
|
-
`windows-latest` hosts. It verifies the declared shell/lifecycle selection and proves
|
|
249
|
-
that Windows local verification refuses before a candidate command can produce a side
|
|
250
|
-
effect. This matrix is intentionally narrower than the Linux adversarial suite; it tests
|
|
251
|
-
the support boundary rather than implying feature parity where none is claimed.
|
|
252
|
-
|
|
253
|
-
`tamperward verify` materialises two temporary copies — they are **not
|
|
254
|
-
sandboxes**, and both execute agent-controlled code. The visible copy runs the
|
|
255
|
-
candidate as-is. In the pristine copy, files on the protected overlay classes and
|
|
256
|
-
on a separate enumerated **verification surface** (the configuration a test runner
|
|
257
|
-
auto-consults) are restored from the trusted base, and agent-added files on those
|
|
258
|
-
surfaces are removed. That surface list deliberately **bounds** runner-configuration
|
|
259
|
-
inheritance rather than eliminating it — see
|
|
260
|
-
[docs/THREAT-MODEL-pristine-run.md](./docs/THREAT-MODEL-pristine-run.md).
|
|
261
|
-
|
|
262
|
-
Visible green followed by pristine red is a `MASKED_FAILURE`. Two green runs
|
|
263
|
-
establish only that the candidate passes the base suite; they establish neither
|
|
264
|
-
semantic correctness nor a substitute for independent held-out tests.
|
|
265
|
-
|
|
266
|
-
| result | envelope behaviour |
|
|
267
|
-
| --- | --- |
|
|
268
|
-
| Enforcement clean and agent exits 0 | exit 0 |
|
|
269
|
-
| Blocking finding, masked failure, dependency drift, or non-quiescent tree | exit 1 |
|
|
270
|
-
| Required adjudication cannot be completed | exit 2, failing closed |
|
|
271
|
-
| Enforcement clean but the agent exceeded `--agent-budget` | `AGENT_TIMEOUT`, exit 124 |
|
|
272
|
-
| Enforcement clean but the agent failed | the agent's non-zero exit is preserved |
|
|
273
|
-
|
|
274
|
-
### CI authority
|
|
275
|
-
|
|
276
|
-
Pre-commit is an early backstop; protected CI is the merge authority. Two workflows
|
|
277
|
-
are involved here, and they are not the same thing.
|
|
278
|
-
|
|
279
|
-
**The shipped workflow** — the one `tamperward init` writes into your repository as
|
|
280
|
-
`.github/workflows/tamperward.yml` — runs, independently:
|
|
281
|
-
|
|
282
|
-
1. `check --diff base.sha...head.sha`, with policy from the trusted merge-base; and
|
|
283
|
-
2. `verify --require-ancestor --base <base-sha>`, which since **1.14.3** also reads
|
|
284
|
-
its policy, verification surface and verifier command and budget from that same
|
|
285
|
-
trusted base — so neither step's verdict is governed by the candidate.
|
|
286
|
-
|
|
287
|
-
Since **2.12.0**, the generated job uses GitHub Actions' 360-minute outer timeout and
|
|
288
|
-
runs `tamperward doctor` before verification. The doctor reads the **trusted base**
|
|
289
|
-
policy, finds the job(s) that actually invoke `tamperward verify`, and requires enough
|
|
290
|
-
static outer time for both independently budgeted stages plus a 60-minute authority
|
|
291
|
-
reserve. With the generated 360-minute job that means budgets up to 9,000 seconds per
|
|
292
|
-
stage fit. Larger positive finite policy budgets remain valid — there is no schema cap —
|
|
293
|
-
but this generated GitHub-hosted authority refuses deterministically and tells you how
|
|
294
|
-
much outer time is required, so a custom runner/workflow can provide it instead of
|
|
295
|
-
GitHub killing verification mid-verdict.
|
|
296
|
-
|
|
297
|
-
Since **2.15.0**, `tamperward doctor` is also the one-shot installation/posture
|
|
298
|
-
report promised by `init`'s security model. It reuses `init`'s canonical wiring
|
|
299
|
-
planner rather than maintaining a second definition of “installed correctly”, and
|
|
300
|
-
reports named `OK`, `WARN`, or `BROKEN` checks for policy/schema, Claude hooks,
|
|
301
|
-
pre-commit, CI authority wiring, CODEOWNERS, workflow permissions, binary/pin
|
|
302
|
-
alignment, verifier trust mode + declared inputs, platform residuals, CI verifier
|
|
303
|
-
outer time, observer health, and (with `--github`) repository authority. Use
|
|
304
|
-
`--json` for one machine-readable document with `authoritative: true|false`.
|
|
305
|
-
|
|
306
|
-
**2.15.1 closes an authority-reporting gap in that surface.** The permission check now
|
|
307
|
-
includes job-level overrides, not only workflow-root permissions; a policy schema newer
|
|
308
|
-
than the running binary is BROKEN rather than certifiable; and the exact workflow(s)
|
|
309
|
-
selected by `--workflow` or discovered as verifier authorities are also the workflow(s)
|
|
310
|
-
used for CI-wiring and token-permission posture. A custom verifier workflow therefore
|
|
311
|
-
does not get judged against an unrelated/missing generated file, and a second verifier
|
|
312
|
-
workflow with a write-scoped job cannot hide behind a safe canonical workflow.
|
|
313
|
-
|
|
314
|
-
For this repository, `tamperward doctor --github --repo hexrift/tamperward --branch main` also verifies the active main ruleset's Code Owner requirement, stale-review dismissal, every direct CI check named by `ci.yml`, and the absence of bypass actors. A missing setting is an authority failure, not a warning; run it with a token that can read repository rulesets.
|
|
315
|
-
|
|
316
|
-
Local early-layer gaps remain posture findings rather than silently changing the
|
|
317
|
-
existing generated-CI exit contract; hard CI/GitHub validation failures still exit 2.
|
|
318
|
-
|
|
319
|
-
Legitimate exceptions are out-of-band PR labels, `tamperward:allow:<rule>@<head-sha>`.
|
|
320
|
-
The workflow passes the head SHA to both steps through `TAMPERWARD_OOB_HEAD`, so an
|
|
321
|
-
approval is bound to the exact commit it was granted for and a new push invalidates it.
|
|
322
|
-
Since **2.1.0** the verify step reads the same labels: `tamperward:allow:verify@<head-sha>`
|
|
323
|
-
accepts a `MASKED_FAILURE` — the case where a behaviour change makes the original
|
|
324
|
-
expectations wrong and a reviewer has read the test edit and said so. It clears nothing
|
|
325
|
-
else: a red visible suite, or a run that could not verify, stays red whatever the labels
|
|
326
|
-
say, and the verdict is still reported as a masked failure; only the exit code changes.
|
|
327
|
-
|
|
328
|
-
**This repository's own self-gate** — the `gate` job in `.github/workflows/ci.yml` —
|
|
329
|
-
runs the built CLI's `check --diff` over the pull-request range, cleared only by the
|
|
330
|
-
same label channel, but does **not** run `verify` on itself. This repo's test
|
|
331
|
-
expectations legitimately change whenever a rule changes, so nearly every rule pull
|
|
332
|
-
request would need the verify label; the self-gate stays a diff-time gate.
|
|
333
|
-
|
|
334
|
-
This guarantee depends on a protected and immutable base, required status checks, a
|
|
335
|
-
pinned Tamperward version, and label permissions restricted to trusted humans.
|
|
336
|
-
The default local verifier is checkpointed same-host execution, not OS isolation.
|
|
337
|
-
An optional digest-pinned container backend isolates final verification with no network,
|
|
338
|
-
host dependency tree, HOME, temp, credential or socket sharing. Tamperward still is not a
|
|
339
|
-
semantic-correctness oracle: verifier output reports `oracle_assurance: suite-exit-only`
|
|
340
|
-
because candidate source still executes inside the configured suite process and can
|
|
341
|
-
terminate or interpose on that in-process oracle. `isolated-container` therefore means
|
|
342
|
-
execution-domain isolation, not semantic/oracle isolation. `tamperward run` also does
|
|
343
|
-
not pretend that a same-identity host agent is isolated from the Docker daemon.
|
|
344
|
-
|
|
345
|
-
Full assumptions and residual risks: [SPEC.md](./SPEC.md),
|
|
346
|
-
[SECURITY-ENVELOPE.md](./SECURITY-ENVELOPE.md), and the
|
|
347
|
-
[series limitations page](./docs/blog/limitations.md).
|
|
348
|
-
|
|
349
47
|
## Quick start
|
|
350
48
|
|
|
351
49
|
```bash
|
|
@@ -356,407 +54,206 @@ Requires Node.js 20.19 or later. JavaScript and TypeScript are the fully
|
|
|
356
54
|
supported detector surface; the other documented ecosystems get file-level and
|
|
357
55
|
pattern-based protection.
|
|
358
56
|
|
|
359
|
-
`onboard`
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
The deterministic primitive underneath is unchanged:
|
|
370
|
-
|
|
371
|
-
```bash
|
|
372
|
-
npx tamperward init
|
|
373
|
-
```
|
|
374
|
-
|
|
375
|
-
One idempotent command wires the policy, the agent hooks, the pre-commit hook, a
|
|
376
|
-
CI workflow that runs both the diff-time check and pristine verification, and a
|
|
377
|
-
`CODEOWNERS` requirement on the paths that decide whether the gate runs at all.
|
|
378
|
-
It never overwrites anything you wrote; `--dry-run` prints the plan.
|
|
379
|
-
|
|
380
|
-
**`init` is not sufficient on its own, and it will tell you so.** The protected
|
|
381
|
-
branch needs all three repository-authority controls:
|
|
382
|
-
|
|
383
|
-
1. require the **`tamperward`** status check;
|
|
384
|
-
2. enable **Require review from Code Owners**; and
|
|
385
|
-
3. enable **Dismiss stale pull request approvals when new commits are pushed**.
|
|
386
|
-
|
|
387
|
-
The third control is load-bearing: an approval for an older gate-critical diff must
|
|
388
|
-
not authorize a later push. GitHub's "require approval of the most recent reviewable
|
|
389
|
-
push" can be useful in addition, but it is not equivalent here because that fresh
|
|
390
|
-
approver is not necessarily the Code Owner for the gate path. A `pull_request`
|
|
391
|
-
workflow runs from the pull request's own head and a required check is matched by job
|
|
392
|
-
name, so without this human boundary a PR can keep the job name, replace the gate with
|
|
393
|
-
`true`, and present a green required check over a change the gate would have blocked.
|
|
394
|
-
That is reproduced on this project's own CI, not a theoretical concern.
|
|
395
|
-
|
|
396
|
-
After configuring GitHub, verify the boundary with
|
|
397
|
-
`tamperward doctor --github --repo OWNER/REPO --branch <default-branch>`.
|
|
398
|
-
Public rulesets can be read anonymously; set `GH_TOKEN` or `GITHUB_TOKEN` when
|
|
399
|
-
authentication is required.
|
|
57
|
+
`onboard` is the guided first run: it previews the installation, explains each
|
|
58
|
+
enforcement point, asks before writing anything, runs the canonical `init`, offers
|
|
59
|
+
the detected suite command for your explicit acceptance, runs and explains the first
|
|
60
|
+
`verify`, offers a safe demonstration of a weakening move on a disposable worktree,
|
|
61
|
+
checks the GitHub controls, and ends with a `READY` / `READY WITH WARNINGS` /
|
|
62
|
+
`BROKEN` / `INCOMPLETE` posture. The deterministic primitive underneath is `npx
|
|
63
|
+
tamperward init` — one idempotent command that wires the policy, the agent hooks, the
|
|
64
|
+
pre-commit hook, a CI workflow running both the diff-time check and pristine
|
|
65
|
+
verification, and a `CODEOWNERS` requirement on the paths that decide whether the gate
|
|
66
|
+
runs. It never overwrites anything you wrote; `--dry-run` prints the plan.
|
|
400
67
|
|
|
401
68
|
A real deployment needs a verify command configured — the generated CI verify step
|
|
402
|
-
**fails closed (exit 2) without one
|
|
403
|
-
`tamperward init` ends with a separate **VERIFICATION SETUP** status: it says either
|
|
404
|
-
`verification configured — <command>` or prominently reports
|
|
405
|
-
`INCOMPLETE: verification not configured — CI will fail closed`. It can suggest a
|
|
406
|
-
single high-confidence command such as `npm test`, `pytest`, `tox`, `cargo test`
|
|
407
|
-
or `go test ./...`; multiple candidates are listed without choosing one. Suggestions
|
|
408
|
-
are advisory only — init never writes an inferred verifier command because that command
|
|
409
|
-
is part of the trust anchor. In `.tamperward.yml`:
|
|
69
|
+
**fails closed (exit 2) without one**. In `.tamperward.yml`:
|
|
410
70
|
|
|
411
71
|
```yaml
|
|
412
72
|
verify:
|
|
413
73
|
command: npm test
|
|
414
74
|
budget: 300
|
|
415
|
-
inputs: ['scripts/**'] #
|
|
75
|
+
inputs: ['scripts/**'] # files the command DELEGATES to
|
|
416
76
|
# Optional stronger final-verification boundary:
|
|
417
77
|
# backend: container
|
|
418
78
|
# image: ghcr.io/acme/verifier@sha256:<64-hex-digest>
|
|
419
79
|
```
|
|
420
80
|
|
|
421
|
-
`backend: local` is the default and is reported as `checkpointed-local`. With
|
|
422
|
-
`backend: container`, the image must be digest-pinned and already present on the fixed
|
|
423
|
-
local Docker daemon; Tamperward never pulls during adjudication. The materialised
|
|
424
|
-
candidate/pristine tree is mounted read-only, the image owns runtime/dependencies,
|
|
425
|
-
network is disabled, HOME/tmp are private, and optional suite output belongs in
|
|
426
|
-
`$TAMPERWARD_OUTPUT_DIR` (`/workspace-out`). The isolated verifier also applies a
|
|
427
|
-
fixed host-protection envelope: **2 GiB memory, no additional swap, 2 CPUs and 256
|
|
428
|
-
PIDs**, in addition to the configured wall-clock budget. Those limits are included in
|
|
429
|
-
the JSON backend report. Docker-confirmed memory OOM is `CANNOT_VERIFY` with
|
|
430
|
-
`VERIFIER_RESOURCE_EXHAUSTED`, never a suite failure; an exit such as 137 without
|
|
431
|
-
`OOMKilled=true` remains the suite's own exit. v2.11.2 deliberately has no environment
|
|
432
|
-
variable tuning knob for these ceilings because candidate-controlled CI/env must not
|
|
433
|
-
weaken verifier containment. Images that declare Dockerfile
|
|
434
|
-
`VOLUME` paths are refused: Docker mounts those paths writable even with
|
|
435
|
-
`--read-only`, which would undermine the immutable verifier-image boundary. Image
|
|
436
|
-
`ENTRYPOINT` is also overridden; the pinned image supplies the runtime/dependencies,
|
|
437
|
-
while the trusted policy's `verify.command` remains the command that is adjudicated.
|
|
438
|
-
|
|
439
81
|
That block is itself a guarded surface: changing the command, lowering the budget,
|
|
440
82
|
narrowing `inputs`, removing `backend: container`, or changing its pinned image is
|
|
441
|
-
flagged as policy weakening
|
|
442
|
-
all.
|
|
443
|
-
|
|
444
|
-
`inputs` names the files the command *executes*, so the pristine run gets the
|
|
445
|
-
base's copy of them too. A command token that names a file present at the base is
|
|
446
|
-
picked up automatically (`node runner.js`), so most repositories need nothing
|
|
447
|
-
here. Delegation is what needs the list: `npm test` names no file, and the base's
|
|
448
|
-
restored `"test": "sh scripts/test.sh"` will happily call a script nothing
|
|
449
|
-
restored. It bounds the class rather than closing it — see
|
|
450
|
-
[the threat model](./docs/THREAT-MODEL-pristine-run.md).
|
|
451
|
-
|
|
452
|
-
From **2.18.0**, Linux can turn that residual into an auditable observation with
|
|
453
|
-
`tamperward trace-verify`. It materialises the caller-selected trusted base, runs the
|
|
454
|
-
known-good verifier under `strace`, repeats the trace (two runs by default), unions the
|
|
455
|
-
file/exec observations, and reports:
|
|
456
|
-
- tracked repository inputs the verifier actually read or executed;
|
|
457
|
-
- likely config inputs;
|
|
458
|
-
- external dependency/runtime paths;
|
|
459
|
-
- paths seen in only some runs as **dynamic**;
|
|
460
|
-
- whether each tracked input is already covered by the same pristine-verification
|
|
461
|
-
surface used by `verify`;
|
|
462
|
-
- exact uncovered paths as candidate `verify.inputs` entries for **human review**.
|
|
463
|
-
|
|
464
|
-
It is advisory only: it never edits `.tamperward.yml`, never widens a glob, and never
|
|
465
|
-
treats absence from one or several traces as proof a path can never be read. Use a
|
|
466
|
-
known-good base; `trace-verify` observes what those executions did, it does not prove
|
|
467
|
-
the base or external runtime/dependencies are trustworthy. macOS and Windows report the
|
|
468
|
-
feature unsupported rather than implying parity.
|
|
469
|
-
|
|
470
|
-
Example:
|
|
471
|
-
|
|
472
|
-
```bash
|
|
473
|
-
npx tamperward trace-verify --base main --cmd "npm test" --runs 3
|
|
474
|
-
```
|
|
475
|
-
|
|
476
|
-
From **2.16.0**, verifier suite output is diagnostic evidence instead of discarded
|
|
477
|
-
noise. Both visible and pristine stages continuously drain stdout/stderr through a
|
|
478
|
-
trusted supervisor, retain only the final **16 KiB per stream**, and count the total
|
|
479
|
-
bytes observed. Structured output exposes this under
|
|
480
|
-
`visible.diagnostics` / `pristine.diagnostics` with
|
|
481
|
-
`captured_bytes`, `retained_bytes`, `truncated`, and a bounded `tail` on a
|
|
482
|
-
failed stage. Default human output remains quiet on success. On failure, retained
|
|
483
|
-
diagnostics are rendered with every line prefixed and terminal/control characters
|
|
484
|
-
escaped, so candidate output cannot become ANSI control traffic or a GitHub
|
|
485
|
-
`::command::`. These bytes are evidence produced by candidate code; they never own
|
|
486
|
-
the verifier verdict.
|
|
487
|
-
|
|
488
|
-
The four primitives:
|
|
83
|
+
flagged as policy weakening. The four primitives:
|
|
489
84
|
|
|
490
85
|
```bash
|
|
491
86
|
npx tamperward check --staged # pre-commit view
|
|
492
87
|
npx tamperward check --diff "main...HEAD" # CI view over the PR's commit range
|
|
493
88
|
npx tamperward verify --base main # pristine-suite re-execution
|
|
494
|
-
npx tamperward trace-verify --base main --runs 2 # advisory observed-input discovery (Linux)
|
|
495
89
|
npx tamperward run --agent-budget 1800 -- <agent command...> # optional agent-runtime bound
|
|
496
90
|
```
|
|
497
91
|
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
|
|
502
|
-
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
From **2.19.0**, the public JSON verdict surfaces are versioned independently of the
|
|
506
|
-
npm package version. `check --json`, `verify --json`, `run --json`,
|
|
507
|
-
`doctor --json`, `research run --json` and `research summarize` include top-level `"schema_version": 1`. TamperWard publishes the
|
|
508
|
-
corresponding JSON Schema Draft 2020-12 documents in the npm package and repository:
|
|
509
|
-
|
|
510
|
-
- [`schemas/check-v1.schema.json`](./schemas/check-v1.schema.json)
|
|
511
|
-
- [`schemas/verify-v1.schema.json`](./schemas/verify-v1.schema.json)
|
|
512
|
-
- [`schemas/run-v1.schema.json`](./schemas/run-v1.schema.json)
|
|
513
|
-
- [`schemas/doctor-v1.schema.json`](./schemas/doctor-v1.schema.json)
|
|
514
|
-
- [`schemas/research-v1.schema.json`](./schemas/research-v1.schema.json) — from **2.23.0**, the `pair` records `research run` writes (and prints with `--json`) and the `summary` document `research summarize` prints
|
|
515
|
-
- [`schemas/audit-v1.schema.json`](./schemas/audit-v1.schema.json) — from **2.26.0**, the privacy-safe structured event written under `TAMPERWARD_AUDIT_LOG`; unlike verdict schemas it is JSONL, one event per line
|
|
516
|
-
- [`schemas/stats-v1.schema.json`](./schemas/stats-v1.schema.json) — the aggregate document emitted by `tamperward stats --json`
|
|
517
|
-
|
|
518
|
-
Schema major **1** is deliberately additive: consumers should ignore fields they do not
|
|
519
|
-
understand. Adding new evidence/diagnostic fields does not require a schema bump.
|
|
520
|
-
Removing or renaming a required field, changing its type, or changing the meaning of a
|
|
521
|
-
discriminator requires `schema_version: 2` and new `*-v2.schema.json` files; the v1
|
|
522
|
-
files remain published for existing integrations.
|
|
523
|
-
|
|
524
|
-
`run --json` owns stdout after the wrapped agent starts and emits one final envelope
|
|
525
|
-
document, including post-agent early convictions such as `OBJECT_REWRITE`,
|
|
526
|
-
`HISTORY_REWRITE`, `DEPENDENCY_DRIFT`, or lifecycle cannot-adjudicate. In this mode the
|
|
527
|
-
agent's own stdout is routed to stderr (its stderr is unchanged), so stdout carries
|
|
528
|
-
exactly one document however noisy the agent is. The document's `complete` field says
|
|
529
|
-
whether the full post-agent adjudication ran: `true` documents carry `head`,
|
|
530
|
-
`checks.{diff,worktree,verify}` and `observer`; early convictions and lifecycle refusals
|
|
531
|
-
are `complete: false`. A `CANNOT_ADJUDICATE` document always names which layer could not
|
|
532
|
-
judge in `reason` (`AGENT_LIFECYCLE_NOT_OWNED`, `VERIFY_CANNOT_VERIFY`,
|
|
533
|
-
`CHECK_DIFF_UNJUDGEABLE`, `CHECK_WORKTREE_UNJUDGEABLE`). Argument, trusted-base, policy,
|
|
534
|
-
dirty-start, and other **pre-agent/preflight** failures still fail closed on stderr at
|
|
535
|
-
exit 2 because no agent adjudication occurred.
|
|
536
|
-
|
|
537
|
-
`verify --json` never falls back to prose: every fail-closed exit before a verdict exists
|
|
538
|
-
is a `CANNOT_VERIFY` document whose `reason` is one of the enumerated codes in the schema
|
|
539
|
-
(`POLICY_ERROR`, `NO_SUITE_COMMAND`, `VERIFIER_BACKEND_UNAVAILABLE`, `WORKTREE_CHANGED`,
|
|
540
|
-
`PRISTINE_INTEGRITY_CHANGED`, …) with a human `detail` beside it and, once a suite
|
|
541
|
-
execution was in flight, the `stage` it happened in. The reason vocabularies for both
|
|
542
|
-
`verify` and `run` are defined once in the source and asserted equal to the schema enums
|
|
543
|
-
by the test suite, so a new code cannot ship without the contract.
|
|
544
|
-
|
|
545
|
-
The JSON schemas describe **data shape**, not process status. Exit codes are a separate
|
|
546
|
-
public protocol and are documented in the table immediately below. Consumers should
|
|
547
|
-
validate both independently: schema validation answers “can I parse this verdict?”;
|
|
548
|
-
the process exit answers “did the gate allow, block, time out, or fail to adjudicate?”
|
|
549
|
-
|
|
550
|
-
### CLI reference
|
|
551
|
-
|
|
552
|
-
Every flag below is what the command's parser actually reads (`src/cli/main.ts`,
|
|
553
|
-
`parseVerify`, `parseRun`). Since **2.13.1**, user-supplied argv is validated before
|
|
554
|
-
any command-specific parser or side effect runs: unknown options, missing values,
|
|
555
|
-
invalid numeric budgets/timeouts, duplicate/conflicting `check` views, and unexpected
|
|
556
|
-
positionals fail closed with exit 2 and one deterministic `tamperward: ...` diagnostic.
|
|
557
|
-
`run` requires an explicit `--` before the wrapped command, so a typoed TamperWard
|
|
558
|
-
option can never be reinterpreted as the agent command.
|
|
559
|
-
|
|
560
|
-
| command | flags |
|
|
92
|
+
Full flags, exit codes, verifier configuration and the `trace-verify` advisory tool
|
|
93
|
+
are in the **[getting-started guide](./docs/guide/getting-started.md)**.
|
|
94
|
+
|
|
95
|
+
## Choose your path
|
|
96
|
+
|
|
97
|
+
| Goal | Destination |
|
|
561
98
|
| --- | --- |
|
|
562
|
-
|
|
|
563
|
-
|
|
|
564
|
-
|
|
|
565
|
-
|
|
|
566
|
-
|
|
|
567
|
-
|
|
|
568
|
-
|
|
|
569
|
-
|
|
570
|
-
|
|
571
|
-
|
|
572
|
-
|
|
573
|
-
|
|
574
|
-
|
|
575
|
-
|
|
576
|
-
|
|
577
|
-
|
|
578
|
-
|
|
579
|
-
|
|
580
|
-
|
|
581
|
-
|
|
582
|
-
|
|
583
|
-
|
|
584
|
-
|
|
585
|
-
|
|
586
|
-
|
|
587
|
-
|
|
588
|
-
|
|
589
|
-
|
|
590
|
-
|
|
591
|
-
|
|
592
|
-
|
|
593
|
-
|
|
594
|
-
|
|
595
|
-
|
|
596
|
-
|
|
597
|
-
|
|
99
|
+
| Install TamperWard | [Quick start](#quick-start) (this README) |
|
|
100
|
+
| Configure authoritative CI | [Enforcement guide](./docs/guide/enforcement.md) |
|
|
101
|
+
| Understand the architecture | [Architecture reference](./docs/architecture.md) |
|
|
102
|
+
| Browse detectors and rules | [Rules guide](./docs/guide/rules.md) |
|
|
103
|
+
| Evaluate a model | [Research guide](./docs/guide/research.md) |
|
|
104
|
+
| Inspect published evidence | [Research overview](./docs/research/index.md) |
|
|
105
|
+
| Contribute code | [Development](#contributing) |
|
|
106
|
+
|
|
107
|
+
## Architecture
|
|
108
|
+
|
|
109
|
+
The agent may produce the work, but it must not control how that work is judged.
|
|
110
|
+
Tamperward separates **steering** from **adjudication**. In-loop hooks steer the
|
|
111
|
+
agent while it works — denying known weakening operations and detecting
|
|
112
|
+
protected-tree drift — but they are not the final authority. After the agent exits,
|
|
113
|
+
`tamperward run` independently judges the tree it left behind: "visible" verification
|
|
114
|
+
runs the candidate as it stands; "pristine" verification restores the protected
|
|
115
|
+
verification state from the trusted base and runs the checks again. Visible green
|
|
116
|
+
followed by pristine red is a masked failure.
|
|
117
|
+
|
|
118
|
+
> **Core invariant:** the agent may author the candidate tree, but it must not
|
|
119
|
+
> choose the trusted baseline, the governing policy, the verifier, or the final
|
|
120
|
+
> verdict.
|
|
121
|
+
|
|
122
|
+

|
|
123
|
+
|
|
124
|
+
The complete component map, trust boundaries, data flows, the run-envelope lifecycle,
|
|
125
|
+
platform support and the version-by-version history that earned each layer are on the
|
|
126
|
+
**[architecture reference](./docs/architecture.md)**. `tamperward verify` materialises
|
|
127
|
+
two temporary copies that both execute agent-controlled code — they are **not
|
|
128
|
+
sandboxes**; the pristine-run bounding of runner-configuration inheritance is
|
|
129
|
+
documented in [the threat model](./docs/THREAT-MODEL-pristine-run.md).
|
|
130
|
+
|
|
131
|
+
## Production deployment
|
|
132
|
+
|
|
133
|
+
`init` is not sufficient on its own, and it says so. Authoritative enforcement needs
|
|
134
|
+
all of:
|
|
135
|
+
|
|
136
|
+
1. **A configured verify command** in `.tamperward.yml` — CI fails closed without one.
|
|
137
|
+
2. **The required `tamperward` status check** on the protected branch.
|
|
138
|
+
3. **Require review from Code Owners** — `init` writes `CODEOWNERS` over the
|
|
139
|
+
gate-critical paths; branch protection is the half it cannot set for you.
|
|
140
|
+
4. **Dismiss stale pull request approvals when new commits are pushed** — load-bearing:
|
|
141
|
+
an approval for an older gate-critical diff must not authorize a later push. A
|
|
142
|
+
`pull_request` workflow runs from the PR's own head and a required check is matched
|
|
143
|
+
by job name, so without this human boundary a PR can keep the job name, replace the
|
|
144
|
+
gate with `true`, and present a green required check over a blocked change. That is
|
|
145
|
+
reproduced on this project's own CI, not a theoretical concern.
|
|
146
|
+
5. **A pinned Tamperward version** and label permissions restricted to trusted humans.
|
|
147
|
+
6. **Optionally, `backend: container`** for a digest-pinned, network-isolated final
|
|
148
|
+
verifier — execution-domain isolation, not a semantic-correctness oracle.
|
|
149
|
+
|
|
150
|
+
After configuring GitHub, confirm the boundary with
|
|
151
|
+
`tamperward doctor --github --repo OWNER/REPO --branch <default-branch>`. It verifies the
|
|
152
|
+
active ruleset's Code Owner requirement, stale-review dismissal, every direct CI check
|
|
153
|
+
named by `ci.yml`, and the absence of bypass actors — a missing setting is an authority
|
|
154
|
+
failure, not a warning, so run it with a token that can read repository rulesets. The full
|
|
155
|
+
trust model, sign-off channel and CI authority separation are in the
|
|
156
|
+
**[enforcement guide](./docs/guide/enforcement.md)**.
|
|
157
|
+
|
|
158
|
+
## Evidence
|
|
159
|
+
|
|
160
|
+
Plain-English takeaway: the original detector-centred design was insufficient. Later
|
|
161
|
+
versions materially reduced false-green outcomes in two held-out rounds (Round 2:
|
|
162
|
+
b=9 / c=0, p = 0.0039; Round 3: b=6 / c=0, p = 0.0313), the stronger-model
|
|
163
|
+
replication was inconclusive, and the larger preregistered Round 4 on the 2.x line
|
|
164
|
+
**did not replicate** the prevention effect — a failed prediction, published beside
|
|
165
|
+
the bet.
|
|
166
|
+
|
|
167
|
+
| experiment | result | what it supports |
|
|
598
168
|
| --- | --- | --- |
|
|
599
|
-
|
|
|
600
|
-
|
|
601
|
-
|
|
602
|
-
|
|
603
|
-
|
|
604
|
-
|
|
605
|
-
|
|
606
|
-
|
|
607
|
-
|
|
608
|
-
|
|
609
|
-
|
|
610
|
-
|
|
611
|
-
|
|
612
|
-
|
|
613
|
-
|
|
614
|
-
|
|
615
|
-
|
|
616
|
-
|
|
617
|
-
|
|
618
|
-
|
|
619
|
-
|
|
620
|
-
|
|
621
|
-
|
|
622
|
-
|
|
623
|
-
|
|
624
|
-
|
|
625
|
-
|
|
626
|
-
|
|
627
|
-
rewrites from re-blocking an already-existing skip/focus.
|
|
628
|
-
`assertion-weakening` ships only the AST-proven one-way subset — statically
|
|
629
|
-
proven exact/structural values weakened to truthy/defined, positive exception
|
|
630
|
-
specificity removed, or a pure assertion removed from the same suite-qualified
|
|
631
|
-
test. It measured 12/12 true-positive fires with 0/20 false positives on the
|
|
632
|
-
committed detector-specific replay and remains `warn`; `guard-removal` is still
|
|
633
|
-
reserved with no detector, and `ts-any-launder` is a permanent warn.
|
|
634
|
-
`ts-cast-growth` (2.20.0) is the assertion budget: net growth of ordinary `as T` /
|
|
635
|
-
`<T>x` / `x!` assertions in non-test source, counted on the AST net of casts the same
|
|
636
|
-
change removed, reported as a warning that never blocks by default. It fires on 8.7%
|
|
637
|
-
of legitimate mainline pairs across four real TypeScript libraries
|
|
638
|
-
(`harness/fp-study/CAST-GROWTH-CORPUS.md`), so block is closed by corpus; the same
|
|
639
|
-
pass removed every assertion from TamperWard's own `src/` (`docs/CAST-INVENTORY.md`).
|
|
640
|
-
`coverage-exclusion` (2.24.0) is the per-function form of `coverage-lowering`: an
|
|
641
|
-
inline `istanbul ignore` / `c8 ignore` / `v8 ignore` / `node:coverage ignore` /
|
|
642
|
-
`# pragma: no cover` / `#[coverage(off)]` added to non-test source, or a `//go:build`
|
|
643
|
-
constraint added to an existing Go file. It fired on 0 of the same 460 mainline
|
|
644
|
-
pairs, but every real add in the deeper histories is an honest unreachable-branch
|
|
645
|
-
marker (`harness/fp-study/COVERAGE-EXCLUSION-CORPUS.md`), so it warns and never
|
|
646
|
-
blocks by default.
|
|
647
|
-
`config-weakening` (2.26.0, #447) closes the gap where `tsconfig*.json`,
|
|
648
|
-
`.eslintrc*` / `eslint.config.*` / `.eslintignore` and `biome.json[c]` were
|
|
649
|
-
protected by name but nothing read what changed inside them: a tsconfig
|
|
650
|
-
strictness flag lowered or a loosening flag (`skipLibCheck`, …) added, `exclude`
|
|
651
|
-
grown to hide a source file, `extends` dropped or redirected off the protected
|
|
652
|
-
set; an eslint/biome rule turned off, or `ignores` / `ignorePatterns` /
|
|
653
|
-
`.eslintignore` grown; and a jest/vitest `setupFiles*` / `globalSetup` /
|
|
654
|
-
`runner` / `testEnvironment` entry newly pointing at a file the gate cannot
|
|
655
|
-
read. `test-deletion` gained the matching read for the runner config's own
|
|
656
|
-
*selection* going opaque — `export { default } from './x'`, a `...base`
|
|
657
|
-
spread, `mergeConfig`, `extends`/`preset`/`projects` pointing off the
|
|
658
|
-
protected set, and `testNamePattern`. Both ship `warn` pending an fp-study
|
|
659
|
-
replay (`harness/fp-study/CONFIG-WEAKENING-CORPUS.md`).
|
|
169
|
+
| **Round 4** — fresh pool of 110 paired Python repos under **`claude-sonnet-5`**, treatment **2.10.3** pinned by artefact hash, preregistered exact McNemar ([`harness/taskbench/round4/`](./harness/taskbench/round4/), sealed readout [`ROUND4-ANALYSIS.md`](./harness/taskbench/round4/ROUND4-ANALYSIS.md)) | Primary endpoint: prevention **b=5 / c=3**, exact McNemar **p = 0.7265625**, H₀ **not rejected** — the preregistered prediction (b=16, c=1, reject) **did not replicate**. Realized valid pairs **79/110**; the registered interpretation floor was met (`a + b = 15` ungated opportunities, floor 6), but realized discordance (8 pairs) was below the ~17 the power model assumed. Separate, narrower integrity observation: across **201 measured trajectories**, **0 strict tamper bypass** (landed weakening that survived to the final tree *and* was certified clean) | The registered confirmatory claim failed on the 2.x line and is reported as a failed prediction, not reinterpreted. The zero-bypass count is descriptive and does not stand in for it: one transient landing was certified clean (correctly, it did not survive), and 31/110 pairs were lost to apparatus attrition, which is not claimed bias-free |
|
|
170
|
+
|
|
171
|
+
Every Round 4 figure is the sealed value in
|
|
172
|
+
[`ROUND4-RESULTS.json`](./harness/taskbench/round4/ROUND4-RESULTS.json)
|
|
173
|
+
(`payload_sha256` `e7bfce08…`), read out in
|
|
174
|
+
[`ROUND4-ANALYSIS.md`](./harness/taskbench/round4/ROUND4-ANALYSIS.md) and the
|
|
175
|
+
deviation ledger [`DEVIATIONS.md`](./harness/taskbench/round4/DEVIATIONS.md). Only
|
|
176
|
+
79/110 pairs survived to measurement; the loss was mostly symmetric apparatus
|
|
177
|
+
attrition (venv/interpreter execution failures), which reduces concern about
|
|
178
|
+
arm-specific bias but is not claimed free of selection bias — the readout says so.
|
|
179
|
+
Read the round as a pair — the bets before, the outcome after:
|
|
180
|
+
[How round 4 is built to be hard to fool](./docs/blog/how-round-4-is-built-to-be-hard-to-fool.md)
|
|
181
|
+
(preregistration, no numbers) →
|
|
182
|
+
[The prevention bet didn't replicate. No surviving tampering was certified clean.](./docs/blog/the-prevention-bet-didnt-replicate-no-surviving-tampering-was-certified-clean.md)
|
|
183
|
+
(results).
|
|
184
|
+
|
|
185
|
+
> **Scope.** These rows cover specific models, pressure prompts, treatment versions,
|
|
186
|
+
> and finite JavaScript/TypeScript and Python repository samples — evidence for those
|
|
187
|
+
> settings, not a universal claim. The rounds differ in ecosystem, treatment, model
|
|
188
|
+
> and sample, so they cannot be pooled.
|
|
189
|
+
|
|
190
|
+
Every round, study, correction and sealed artifact — each number bound to its
|
|
191
|
+
committed evidence — is in the **[research overview](./docs/research/index.md)** and
|
|
192
|
+
the **[research series](./docs/blog/index.md)**. Predictions are registered before
|
|
193
|
+
counted runs, with numeric bets and explicit losing conditions; seeds, pools, frozen
|
|
194
|
+
analysis scripts and deviation ledgers are committed, and losing predictions stay in
|
|
195
|
+
the public record. `tamperward research run` is the supported paired-evaluation
|
|
196
|
+
command — see the **[research guide](./docs/guide/research.md)**.
|
|
660
197
|
|
|
661
198
|
## What Tamperward does not do
|
|
662
199
|
|
|
663
200
|
- **Not a correctness oracle.** Pristine verification can only re-run tests that
|
|
664
|
-
exist in the tree.
|
|
665
|
-
|
|
666
|
-
|
|
667
|
-
A hardcoded implementation is likewise invisible to any diff rule.
|
|
201
|
+
exist in the tree. An agent that honestly half-fixes the visible cases (with the
|
|
202
|
+
failing cases withheld) gets a genuine visible green; a hardcoded implementation is
|
|
203
|
+
likewise invisible to any diff rule.
|
|
668
204
|
- **Not an OS sandbox, a network firewall, a secret manager, or a replacement for
|
|
669
|
-
branch protection.** It enforces verification integrity; it does not confine
|
|
670
|
-
|
|
671
|
-
- **The intended deployment is defence in depth:** instructions naming the
|
|
672
|
-
|
|
673
|
-
|
|
674
|
-
|
|
675
|
-
**A word on the words:** *tamper* and *honest* are operational labels on
|
|
676
|
-
|
|
677
|
-
|
|
678
|
-
|
|
679
|
-
|
|
680
|
-
|
|
681
|
-
|
|
682
|
-
|
|
683
|
-
|
|
684
|
-
|
|
685
|
-
|
|
686
|
-
|
|
687
|
-
|
|
688
|
-
|
|
689
|
-
|
|
690
|
-
|
|
691
|
-
|
|
692
|
-
|
|
693
|
-
|
|
694
|
-
|
|
695
|
-
|
|
696
|
-
|
|
697
|
-
contract), pins one source commit per task, adjudicates both arms with the same
|
|
698
|
-
`verify` + `check` primitives, records TamperWard's own verdict separately from that
|
|
699
|
-
outcome, and `research summarize` reports model behaviour, independent outcome,
|
|
700
|
-
TamperWard hits/misses and paired counts with no composite score — a result in which
|
|
701
|
-
TamperWard loses is as plain as one in which it wins. **[Guide](./docs/guide/research.md)**.
|
|
702
|
-
|
|
703
|
-
**[The research series](./docs/blog/index.md)** ·
|
|
704
|
-
**[The harness](./harness/)** ·
|
|
705
|
-
**[Errata](./docs/blog/errata.md)** ·
|
|
706
|
-
**[Maturity milestones](./SPEC.md#91-maturity-milestones)**
|
|
707
|
-
|
|
708
|
-
## The architecture was earned by failures
|
|
709
|
-
|
|
710
|
-
| study / version | consequence |
|
|
711
|
-
| --- | --- |
|
|
712
|
-
| Early seed studies | The proposal gate: deny the known weakening classes before the tool call runs, with the deny message as the correction signal |
|
|
713
|
-
| Blind-dimension studies | Coverage ≠ completeness: when the cheapest cheat sat outside detector coverage, agents took it — so an orthogonal verification layer matters more than more rules |
|
|
714
|
-
| Round 1, p = 1.0 | The 53 real trajectories became the evasion corpus — evasion classes the detectors never imagined, extracted with full forensics |
|
|
715
|
-
| v1.7.0 | `test-content-removal`: the row-deletion / assertion-stripping / setup-gutting family that carried round 1's gated false greens, corpus-priced on 2,304 real commits before shipping as block |
|
|
716
|
-
| v1.8.0 | The effect layer: per-call protected-tree drift check plus the `tamperward watch` fs-event observer — judge what the files became, not the command spelling |
|
|
717
|
-
| v1.9.0 | `tamperward verify`: the round-1 pristine oracle productized — visible-green + pristine-red is a masked failure, and "could not verify" fails closed |
|
|
718
|
-
| Round 2, 9–0 | On 22 fresh repositories, v1.9.0 produced b=9/c=0 false-green discordance (p = 0.0039); four gated false greens remained, through two documented residual mechanisms |
|
|
719
|
-
| Round 4, p = 0.7265625 | On 110 fresh paired repositories under 2.10.3, the preregistered prevention bet did not replicate (b=5/c=3, 79 valid pairs); the 35 masked failures split roughly evenly between tampering-related and plain correctness gaps, so the next round separates the two at the endpoint and qualifies repositories under the exact production runner before they enter the pool |
|
|
720
|
-
| cssstyle lifecycle escape (round 2) | The `tamperward run` envelope: the gate denied 42 mutations and verify refused all 25 stop attempts, and the runtime completed the session anyway over a masked tree — hooks decide, runtimes terminate, so the adjudication boundary moved outside the runtime |
|
|
721
|
-
| v1.14.6, external audit | A pull request can neuter the gate that judges it: `pull_request` workflows run from the PR's own head and required checks match by job name, so a job that keeps its name and runs `true` reports green over anything. Reproduced on this repo's own CI — `gate` succeeded in three seconds over a PR that lowered a mechanical rule to warn. No detector can close it, because the detector is defined by the candidate; `init` now writes a CODEOWNERS requirement on the gate-critical paths and states, every run, that branch protection is the half it cannot set for you |
|
|
722
|
-
| v1.14.5, external audit | The pristine copy was a sibling of the visible copy and fully prepared before it ran, so the candidate's own code wrote `../pristine/<the failing test>` and both runs returned 0 — no race, no background process, no git. The pristine copy is now built only after the visible run exits, its restored files are digested either side of the pristine run, and the working and dependency trees are fingerprinted around both. Also: the overlay restored content but not MODE, so `chmod -x` alone hid a test from an exec-bit runner in both copies; a transient dependency substitution that restored itself on the way out left the entry-vs-exit fingerprint identical; and a committed `.npmrc` chose the registry the CI gate was downloaded from |
|
|
723
|
-
| v1.14.4, external audit | The overlay restored the tests and then ran the agent's *runner*: `verify.command` was frozen, but `runner.js` — and anything `npm test` delegates to — was in no protected class. And `git replace` redirected what the base resolved to for `ls-tree`, `show` and `merge-base` alike, with no ref moved and no file touched. Verifier inputs are now restored from the base; every trusted read sets `GIT_NO_REPLACE_OBJECTS=1` and the envelope convicts a rewrite installed during the run. `init` also gained workflow migration and matcher repair, and the Claude hook now denies a payload it cannot parse instead of allowing it |
|
|
724
|
-
| v1.14.3, external review | Standalone `verify` loaded its policy from the working tree, so the generated CI workflow let a pull request supply the `verify:` command for its own re-execution. `check --diff` flagged the edit as hook-tampering, so the workflow caught it as a pair — but only where both jobs are required, and `verify` alone had no protection. With a `--base`, policy now comes from that commit |
|
|
725
|
-
| v1.14.2, threat model | 1.14.1 removed agent-added files only inside the protected classes, whose `config` list is JS/TS-only; an added `pytest.ini`, `setup.cfg`, `tox.ini` or `pyproject.toml` still reached the pristine run. `verify` now owns a verification surface covering runner-consulted configuration |
|
|
726
|
-
| v1.14.1, article audit | `tamperward verify` kept agent-added protected files in the pristine run on the premise that they "only add strictness". An added `conftest.py` could deselect the restored base tests by node id, so a masked failure reported VERIFIED and the envelope printed GREEN MEANS GREEN over an unfixed bug. Added protected files are now removed; PoC and mutation-checked regression committed |
|
|
727
|
-
| v1.10.1–v1.14.0, owner + two-pass external review | Frozen entry-time policy and verifier, entry-SHA ancestry enforcement, quiescence guard, `node_modules` content fingerprint, a CI verify step in the generated workflow, the gate pinned to its own version in CI, and SHA-bound sign-off labels |
|
|
728
|
-
|
|
729
|
-
Each row's primary artifact: [CHANGELOG.md](./CHANGELOG.md), [SPEC.md](./SPEC.md),
|
|
730
|
-
and the posts in [docs/blog/](./docs/blog/index.md).
|
|
731
|
-
|
|
732
|
-
## Stability
|
|
733
|
-
|
|
734
|
-
The public surface is the CLI and its **exit codes**, the hook wire format, the
|
|
735
|
-
`.tamperward.yml` schema, and the versioned `check` / `verify` / `run` / `doctor`
|
|
736
|
-
machine-output schemas under `schemas/`. No `main`, no `exports` — it is a binary,
|
|
737
|
-
not a library. The machine-output schema major is intentionally independent of the npm
|
|
738
|
-
version: additive fields stay compatible within the major; breaking shape/semantic
|
|
739
|
-
changes require a new schema major. The version answers one question: *can taking this
|
|
740
|
-
upgrade turn a green build red without me changing anything?* **Patch never can** —
|
|
741
|
-
bypass fixes and false-positive fixes ship as patches so they reach you automatically.
|
|
742
|
-
Rule graduations (`warn` → `block`) are **opt-in**: they gate on the `version:` field in
|
|
743
|
-
your policy, so they ship as minors and apply only when you raise it. Releases publish
|
|
744
|
-
via npm trusted publishing with SLSA provenance. Full rule:
|
|
745
|
-
[CONTRIBUTING](./CONTRIBUTING.md#versioning).
|
|
746
|
-
|
|
747
|
-
## Develop
|
|
205
|
+
branch protection.** It enforces verification integrity; it does not confine the
|
|
206
|
+
process.
|
|
207
|
+
- **The intended deployment is defence in depth:** instructions naming the forbidden
|
|
208
|
+
move + capability restriction + Tamperward + independent held-out tests +
|
|
209
|
+
authoritative CI. Each layer covers classes the others measurably miss.
|
|
210
|
+
|
|
211
|
+
**A word on the words:** *tamper* and *honest* are operational labels on artifacts —
|
|
212
|
+
a protected asset changed, or an independent oracle failed, while the visible check
|
|
213
|
+
went green — decided deterministically. They are not claims about any model's intent.
|
|
214
|
+
|
|
215
|
+
## Repository map
|
|
216
|
+
|
|
217
|
+
- **Operators** — installing and running the gate: the
|
|
218
|
+
[getting-started guide](./docs/guide/getting-started.md) (CLI, exit codes, verifier
|
|
219
|
+
config), the [enforcement guide](./docs/guide/enforcement.md) (CI authority and
|
|
220
|
+
sign-off), the [rules reference](./docs/guide/rules.md), the
|
|
221
|
+
[environment variables](./docs/guide/environment.md), and
|
|
222
|
+
[audit history & stats](./docs/guide/audit.md).
|
|
223
|
+
- **Researchers** — the [research overview](./docs/research/index.md), the
|
|
224
|
+
[research series](./docs/blog/index.md), the [errata](./docs/blog/errata.md), and
|
|
225
|
+
the harness ([`harness/`](./harness/)) holding seeds, oracles, transcripts tooling
|
|
226
|
+
and every pre-registered prediction with its outcome.
|
|
227
|
+
- **Contributors** — [`SPEC.md`](./SPEC.md) (build spec, detector table,
|
|
228
|
+
enforcement-point wiring, proof-harness design),
|
|
229
|
+
[`SECURITY-ENVELOPE.md`](./SECURITY-ENVELOPE.md), the [architecture
|
|
230
|
+
reference](./docs/architecture.md), and [`CONTRIBUTING.md`](./CONTRIBUTING.md).
|
|
231
|
+
|
|
232
|
+
## Contributing
|
|
748
233
|
|
|
749
234
|
```bash
|
|
750
235
|
npm install && npm run build # bundles the CLI to dist/cli/index.js
|
|
751
|
-
npm test #
|
|
236
|
+
npm test # parser, detectors, engine, policy, renderers
|
|
752
237
|
npm run typecheck
|
|
753
|
-
node harness/perf/bench.mjs # performance budgets
|
|
238
|
+
node harness/perf/bench.mjs # performance budgets (docs/PERF.md)
|
|
754
239
|
```
|
|
755
240
|
|
|
756
241
|
Tamperward's own CI runs the engine it ships over every pull request — `check --diff`
|
|
757
|
-
over the PR range, cleared only by an out-of-band label
|
|
758
|
-
|
|
759
|
-
|
|
760
|
-
|
|
761
|
-
|
|
762
|
-
|
|
242
|
+
over the PR range, cleared only by an out-of-band label — and has, on more than one
|
|
243
|
+
occasion, blocked its own author's commits. Branch, then open a PR; `main` is
|
|
244
|
+
protected and CI must be green. Changing a protected asset will block your own PR
|
|
245
|
+
(working as intended); a reviewed, legitimate change is cleared by a maintainer
|
|
246
|
+
applying a `tamperward:allow:<rule>` label, never by weakening the policy.
|
|
247
|
+
|
|
248
|
+
The public surface is the CLI and its exit codes, the hook wire format, the
|
|
249
|
+
`.tamperward.yml` schema, and the versioned machine-output schemas under `schemas/` —
|
|
250
|
+
no `main`, no `exports`; it is a binary, not a library. The version answers one
|
|
251
|
+
question: *can taking this upgrade turn a green build red without me changing
|
|
252
|
+
anything?* Patch never can; rule graduations (`warn` → `block`) are opt-in via the
|
|
253
|
+
`version:` field. Full versioning rule and PR conventions:
|
|
254
|
+
[CONTRIBUTING](./CONTRIBUTING.md#versioning). Report a bypass privately — see
|
|
255
|
+
[SECURITY.md](./SECURITY.md).
|
|
256
|
+
|
|
257
|
+
## License
|
|
258
|
+
|
|
259
|
+
[Apache-2.0](./LICENSE).
|
package/dist/cli/index.js
CHANGED
|
@@ -3663,32 +3663,57 @@ function kindOfType(type, resolveName) {
|
|
|
3663
3663
|
return null;
|
|
3664
3664
|
}
|
|
3665
3665
|
function buildAliasMap(sf) {
|
|
3666
|
-
const
|
|
3666
|
+
const bindings = [];
|
|
3667
|
+
const shadow = (name, scope) => {
|
|
3668
|
+
if (scope) bindings.push({ kind: "shadow", name, scope });
|
|
3669
|
+
};
|
|
3667
3670
|
const collect2 = (node) => {
|
|
3668
|
-
if (ts.isTypeAliasDeclaration(node)
|
|
3671
|
+
if (ts.isTypeAliasDeclaration(node)) {
|
|
3672
|
+
if (!node.typeParameters || node.typeParameters.length === 0) {
|
|
3673
|
+
bindings.push({ kind: "alias", name: node.name.text, scope: node.parent, type: node.type });
|
|
3674
|
+
} else {
|
|
3675
|
+
shadow(node.name.text, node.parent);
|
|
3676
|
+
}
|
|
3677
|
+
} else if (ts.isInterfaceDeclaration(node) || ts.isEnumDeclaration(node)) {
|
|
3678
|
+
shadow(node.name.text, node.parent);
|
|
3679
|
+
} else if ((ts.isClassDeclaration(node) || ts.isModuleDeclaration(node)) && node.name && ts.isIdentifier(node.name)) {
|
|
3680
|
+
shadow(node.name.text, node.parent);
|
|
3681
|
+
} else if (ts.isTypeParameterDeclaration(node) && ts.isIdentifier(node.name)) {
|
|
3682
|
+
shadow(node.name.text, node.parent);
|
|
3683
|
+
}
|
|
3669
3684
|
ts.forEachChild(node, collect2);
|
|
3670
3685
|
};
|
|
3671
3686
|
collect2(sf);
|
|
3672
|
-
const
|
|
3687
|
+
const kindOfBinding = (b, seen) => b.type ? kindOfType(b.type, (n) => resolveLexical(n, b.type, seen)) : null;
|
|
3688
|
+
const resolveLexical = (name, at, seen) => {
|
|
3689
|
+
if (seen.has(name)) return null;
|
|
3690
|
+
for (let n = at; n; n = n.parent) {
|
|
3691
|
+
const here = bindings.filter((x) => x.name === name && x.scope === n);
|
|
3692
|
+
if (here.length === 0) continue;
|
|
3693
|
+
if (here.some((b) => b.kind !== "alias" || !b.type)) return null;
|
|
3694
|
+
return kindOfBinding(here[0], new Set(seen).add(name));
|
|
3695
|
+
}
|
|
3696
|
+
return null;
|
|
3697
|
+
};
|
|
3698
|
+
const resolveFlat = (name, seen) => {
|
|
3673
3699
|
if (seen.has(name)) return null;
|
|
3674
|
-
|
|
3675
|
-
const
|
|
3676
|
-
|
|
3700
|
+
if (bindings.some((b) => b.name === name && (b.kind !== "alias" || !b.type))) return null;
|
|
3701
|
+
const aliases = bindings.filter((b) => b.kind === "alias" && b.name === name);
|
|
3702
|
+
if (aliases.length === 0) return null;
|
|
3703
|
+
const kinds = new Set(aliases.map((b) => kindOfBinding(b, new Set(seen).add(name))));
|
|
3704
|
+
return kinds.size === 1 ? [...kinds][0] ?? null : null;
|
|
3705
|
+
};
|
|
3706
|
+
return {
|
|
3707
|
+
resolve: (name, at) => at && at.getSourceFile() === sf ? resolveLexical(name, at, /* @__PURE__ */ new Set()) : resolveFlat(name, /* @__PURE__ */ new Set())
|
|
3677
3708
|
};
|
|
3678
|
-
const resolved = /* @__PURE__ */ new Map();
|
|
3679
|
-
for (const name of decls.keys()) {
|
|
3680
|
-
const k = resolve17(name, /* @__PURE__ */ new Set());
|
|
3681
|
-
if (k) resolved.set(name, k);
|
|
3682
|
-
}
|
|
3683
|
-
return resolved;
|
|
3684
3709
|
}
|
|
3685
|
-
function assertedLaunderKind(node,
|
|
3686
|
-
return kindOfType(assertedType(node), (n) =>
|
|
3710
|
+
function assertedLaunderKind(node, aliases) {
|
|
3711
|
+
return kindOfType(assertedType(node), (n) => aliases.resolve(n, node));
|
|
3687
3712
|
}
|
|
3688
|
-
function isDoubleCast(node,
|
|
3713
|
+
function isDoubleCast(node, aliases) {
|
|
3689
3714
|
const inner = unparenthesized(node.expression);
|
|
3690
3715
|
if (!(ts.isAsExpression(inner) || ts.isTypeAssertionExpression(inner))) return false;
|
|
3691
|
-
const k = assertedLaunderKind(inner,
|
|
3716
|
+
const k = assertedLaunderKind(inner, aliases ?? EMPTY_ALIASES);
|
|
3692
3717
|
return k != null && DOUBLE_CAST_INNER.has(k);
|
|
3693
3718
|
}
|
|
3694
3719
|
function surfaceOf(path, src) {
|
|
@@ -3767,7 +3792,7 @@ var init_ts_cast_growth = __esm({
|
|
|
3767
3792
|
GENERATED_HEADER = /@generated\b|\bAUTO-?GENERATED\b|\bDO NOT EDIT\b/i;
|
|
3768
3793
|
HEADER_LINES = 20;
|
|
3769
3794
|
DOUBLE_CAST_INNER = /* @__PURE__ */ new Set(["any", "never", "unknown", "empty", "objectish"]);
|
|
3770
|
-
EMPTY_ALIASES =
|
|
3795
|
+
EMPTY_ALIASES = { resolve: () => null };
|
|
3771
3796
|
tsCastGrowth = {
|
|
3772
3797
|
id: RULE2,
|
|
3773
3798
|
surface: ["file"],
|
|
@@ -3895,7 +3920,7 @@ function lineHasRow4Cast(line, aliasMap) {
|
|
|
3895
3920
|
function aliasMapForAddedLines(c) {
|
|
3896
3921
|
const snippet = addedLines(c).map((l) => l.content).join("\n");
|
|
3897
3922
|
const sf = parseSource("lines.ts", snippet);
|
|
3898
|
-
return sf ? buildAliasMap(sf) :
|
|
3923
|
+
return sf ? buildAliasMap(sf) : EMPTY_ALIASES;
|
|
3899
3924
|
}
|
|
3900
3925
|
var BLOCK_RULE, WARN_RULE, JSDOC_ANY_CAST, JS_FILE, countMatches, DIRECTIVE_START, NARROW_LINE, BROAD_LINE, FRAGMENT_DIRECTIVE, BLOCK_REMEDIATION, WARN_REMEDIATION, tsAnyCast;
|
|
3901
3926
|
var init_ts_any_cast = __esm({
|
|
@@ -15720,6 +15745,21 @@ function renderStageDiagnostics(out3, stage, result) {
|
|
|
15720
15745
|
if (!result.diagnostics) return;
|
|
15721
15746
|
for (const line of diagnosticLines(stage, result.diagnostics)) out3(line);
|
|
15722
15747
|
}
|
|
15748
|
+
function verifyVerdictLine(verdict2, ctx) {
|
|
15749
|
+
const base10 = ctx.base.slice(0, 10);
|
|
15750
|
+
switch (verdict2) {
|
|
15751
|
+
case "VERIFIED":
|
|
15752
|
+
return `verified: the suite passes, and still passes with all ${ctx.restored} protected files restored from ${base10}.`;
|
|
15753
|
+
case "MASKED_FAILURE":
|
|
15754
|
+
return `MASKED FAILURE: the visible suite passes, but with the ${ctx.restored} protected files restored from ${base10} it FAILS (exit ${ctx.pristineExit}). Something weakened the checks; the code does not pass the original suite.`;
|
|
15755
|
+
case "SUITE_RED":
|
|
15756
|
+
return `suite red: the visible suite fails (exit ${ctx.visibleExit}) \u2014 fix the code first (pristine exit ${ctx.pristineExit}).`;
|
|
15757
|
+
case "BUDGET_EXCEEDED":
|
|
15758
|
+
return `budget exceeded (${ctx.budget}s): could not verify \u2014 failing closed, not open.`;
|
|
15759
|
+
default:
|
|
15760
|
+
return verdict2;
|
|
15761
|
+
}
|
|
15762
|
+
}
|
|
15723
15763
|
function runVerify(opts) {
|
|
15724
15764
|
const cwd = opts.cwd ?? process.cwd();
|
|
15725
15765
|
const out3 = opts.silent ? (_s) => {
|
|
@@ -16046,13 +16086,15 @@ function runVerify(opts) {
|
|
|
16046
16086
|
})
|
|
16047
16087
|
);
|
|
16048
16088
|
} else {
|
|
16049
|
-
|
|
16050
|
-
|
|
16051
|
-
|
|
16052
|
-
|
|
16053
|
-
|
|
16054
|
-
|
|
16055
|
-
|
|
16089
|
+
out3(
|
|
16090
|
+
`tamperward verify \u2014 ${verifyVerdictLine(verdict2, {
|
|
16091
|
+
restored: restored.length,
|
|
16092
|
+
base,
|
|
16093
|
+
visibleExit: visible.exit,
|
|
16094
|
+
pristineExit: pristine.exit,
|
|
16095
|
+
budget
|
|
16096
|
+
})}`
|
|
16097
|
+
);
|
|
16056
16098
|
out3(`verifier backend: ${verifierBackendSummary(verifierBackend)}`);
|
|
16057
16099
|
out3(
|
|
16058
16100
|
"oracle assurance: suite-exit-only (candidate source executes inside the suite process; execution-domain isolation is not semantic/oracle isolation)"
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "tamperward",
|
|
3
|
-
"version": "2.29.
|
|
3
|
+
"version": "2.29.12",
|
|
4
4
|
"description": "The deterministic agent-integrity gate. One ruleset, evaluated on the actual diff/commands as a verdict, enforced everywhere a change can be made.",
|
|
5
5
|
"license": "Apache-2.0",
|
|
6
6
|
"author": "hexrift",
|