reeve 0.5.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +106 -1
- data/README.md +113 -6
- data/lib/generators/reeve/install/templates/create_audit_entries.rb.tt +41 -3
- data/lib/generators/reeve/install/templates/initializer.rb.tt +20 -0
- data/lib/generators/reeve/upgrade/templates/add_ledger_chain_to_reeve_audit_entries.rb.tt +51 -0
- data/lib/generators/reeve/upgrade/upgrade_generator.rb +40 -2
- data/lib/reeve/audit/chain/canonical.rb +129 -0
- data/lib/reeve/audit/chain/head.rb +94 -0
- data/lib/reeve/audit/chain/keyring.rb +53 -0
- data/lib/reeve/audit/chain/normalizer.rb +88 -0
- data/lib/reeve/audit/chain/report.rb +134 -0
- data/lib/reeve/audit/chain/verification.rb +110 -0
- data/lib/reeve/audit/chain/verifier.rb +160 -0
- data/lib/reeve/audit/chain/walk.rb +123 -0
- data/lib/reeve/audit/chain.rb +168 -0
- data/lib/reeve/audit/chain_record.rb +61 -0
- data/lib/reeve/audit/isolated_recorder.rb +5 -0
- data/lib/reeve/audit/recorder.rb +32 -19
- data/lib/reeve/audit.rb +24 -4
- data/lib/reeve/configuration.rb +66 -2
- data/lib/reeve/errors.rb +15 -0
- data/lib/reeve/ledger_key.rb +47 -0
- data/lib/reeve/railtie.rb +15 -0
- data/lib/reeve/tasks.rb +62 -0
- data/lib/reeve/testing/checks/contract_version.rb +2 -1
- data/lib/reeve/testing/checks/ledger_chain.rb +116 -0
- data/lib/reeve/testing/checks.rb +29 -4
- data/lib/reeve/testing/compliance_assertions.rb +3 -2
- data/lib/reeve/testing/compliance_suite.rb +2 -2
- data/lib/reeve/testing/ledger.rb +11 -1
- data/lib/reeve/testing/report.rb +8 -0
- data/lib/reeve/testing.rb +1 -1
- data/lib/reeve/version.rb +1 -1
- data/lib/reeve.rb +2 -0
- metadata +17 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 03d0a3695140d0703272da744486caa3ab21530eddb0d48d275977e3b83047c0
|
|
4
|
+
data.tar.gz: aa8988dd9d7fc1dfb40b5c7a67f1f0a9f70d5ba72350b7669ba32afb2121f140
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 116dbf0359f8666ffa50cb70a9a85aa7a037d76390305b3315fc4ecc37233fe83a5faabb14103c0ae999ae69f5cc5fa020813cd7da3ade8ae86f837639604c98
|
|
7
|
+
data.tar.gz: 4ae861908807ec1bc3b3363442edda7cf6936d60d01e39b5aa03f6ae91cd7e0daad49ac8a154a53db277894c866cf9adfdb2e8776d99071b20d6d4704647f4b0
|
data/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,111 @@ All notable changes are recorded here. This project follows [Semantic
|
|
|
4
4
|
Versioning](https://semver.org), with one rule specific to what it does — see
|
|
5
5
|
[Versioning policy](#versioning-policy).
|
|
6
6
|
|
|
7
|
+
## [0.6.0] - 2026-10-10
|
|
8
|
+
|
|
9
|
+
The ledger has been append-only at the library level since 0.1.0: no API updates or deletes
|
|
10
|
+
an entry, and the generated migration documents the grant that enforces the rest. Nothing
|
|
11
|
+
*detected* a change made outside the library. A DBA, a stolen application credential or a
|
|
12
|
+
well-meant data fix could edit, delete or reorder rows, and the ledger read exactly as
|
|
13
|
+
before. Now every row is chained to the one before it with a keyed digest, and
|
|
14
|
+
`bin/rails reeve:verify_ledger` reports every place the chain no longer holds.
|
|
15
|
+
|
|
16
|
+
**Audit-entry contract `2` → `3`, and a migration is required.** The new columns are
|
|
17
|
+
nullable, so the shape change is additive; the bump is MAJOR by the contract's own rule,
|
|
18
|
+
because what an absence means changed. On a contract 1–2 row a NULL digest means "written
|
|
19
|
+
before chaining"; on a contract-3 row it means the row was written unchained, which
|
|
20
|
+
verification reports. Upgrading is three steps:
|
|
21
|
+
|
|
22
|
+
```bash
|
|
23
|
+
bin/rails generate reeve:upgrade
|
|
24
|
+
bin/rails db:migrate
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
then add the line the generator prints to `config/initializers/reeve.rb`:
|
|
28
|
+
|
|
29
|
+
```ruby
|
|
30
|
+
config.ledger_key = Rails.application.key_generator.generate_key("reeve audit ledger", 32)
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
Existing rows are not touched. They are reported as "not covered" from then on, and are
|
|
34
|
+
exactly as trustworthy as they were before.
|
|
35
|
+
|
|
36
|
+
What this detects and what it does not are set out in the README, under "Tamper-evident,
|
|
37
|
+
within stated limits". In short, it does **not** detect a change made by someone holding
|
|
38
|
+
the key; the newest rows deleted by someone who also rewinds the chain record's head; the
|
|
39
|
+
whole ledger deleted (reported as *no chain*, never as a pass); or anything about rows
|
|
40
|
+
written before the upgrade. It detects; it does not prevent.
|
|
41
|
+
|
|
42
|
+
Behaviour changes worth reading before upgrading:
|
|
43
|
+
|
|
44
|
+
- **A ledger key is required.** Without one, every guarded call fails with
|
|
45
|
+
`Reeve::AuditWriteError` (cause `Reeve::LedgerKeyError`, naming the setting). Under
|
|
46
|
+
`audit_failure_mode = :warn` the call goes through and the row is written *unchained*, a
|
|
47
|
+
warning is logged on every such write, and verification reports the row.
|
|
48
|
+
- **Rotating `secret_key_base` rotates the ledger key**, if you keep the generated line. Put
|
|
49
|
+
the old derived key in `config.retired_ledger_keys`, or entries it signed become
|
|
50
|
+
unverifiable — reported as such, never as tampered. A dedicated key avoids the coupling.
|
|
51
|
+
- **Writes serialize on the chain record.** About 0.5 ms more per ledger write on SQLite and
|
|
52
|
+
about 1 ms on PostgreSQL. Under the default recorder inside a transaction the host opened,
|
|
53
|
+
the lock is held until that transaction ends; `IsolatedRecorder` holds it for one insert.
|
|
54
|
+
- **Rails 7.0 with SQLite:** sqlite3 1.x waits for the write lock while holding Ruby's GVL,
|
|
55
|
+
so concurrent guarded calls in one process can fail closed after the busy timeout. Rails
|
|
56
|
+
7.1+ or a server database avoids it. The limit predates this release; the chain's longer
|
|
57
|
+
write makes it likelier.
|
|
58
|
+
- **The compliance suites run one more check, `LedgerChain`.** It writes an entry of its own
|
|
59
|
+
and fails without a key, on a broken or unverifiable chain, or on unchained entries. It is
|
|
60
|
+
skipped, saying so, when there is no compliance tool to write with or no ledger loaded.
|
|
61
|
+
The test environment needs the key line too; the generated one works there unchanged.
|
|
62
|
+
- **Breaking for custom recorders** that write to `Reeve::Audit::Entry`: rows must be written
|
|
63
|
+
through `Reeve::Audit::Chain#append!` (or `#append_unchained!` to keep running without a
|
|
64
|
+
key), or verification reports them as unchained.
|
|
65
|
+
- **A table still at contract 2** now fails with `Reeve::ConfigurationError` naming
|
|
66
|
+
`reeve:upgrade`, instead of a column error from inside the write.
|
|
67
|
+
- **Known limit, MySQL:** a number in a tool's arguments beyond double precision is stored
|
|
68
|
+
rounded by MySQL's `JSON` type, and its row then fails to verify. A false alarm, never a
|
|
69
|
+
false pass.
|
|
70
|
+
- **During a rolling deploy**, servers still on 0.5.0 keep writing contract-2 rows. They
|
|
71
|
+
are counted as not covered, not reported as tampering.
|
|
72
|
+
|
|
73
|
+
### Added
|
|
74
|
+
|
|
75
|
+
- **The ledger chain** (`Reeve::Audit::Chain`). Every entry carries a sequence number, the
|
|
76
|
+
16-character fingerprint of the key that signed it, and an HMAC-SHA256 digest over its
|
|
77
|
+
canonical form and its predecessor's digest. The digest is computed over the values the
|
|
78
|
+
database stores — defaults, casts and JSON normalization included — so a row verifies
|
|
79
|
+
after a dump and restore with every primary key renumbered.
|
|
80
|
+
- **`Reeve::Audit::Chain.verify`** (optionally with `keys:`, for an auditor's own copy of the
|
|
81
|
+
key) and **`bin/rails reeve:verify_ledger`**, registered by a Railtie that loads only when
|
|
82
|
+
Rails is present. Exit status `0` intact, `1` broken, `2` could not verify; `LIMIT=` caps
|
|
83
|
+
the breaks listed, never the totals; `FORMAT=json` writes every break. Every line that
|
|
84
|
+
says *intact* or *pass* states the covered range and the uncovered count beside it.
|
|
85
|
+
`Reeve::Tasks.verify_ledger` is the same check without Rake.
|
|
86
|
+
- **`config.ledger_key`** and **`config.retired_ledger_keys`**, validated at assignment (32
|
|
87
|
+
bytes minimum) and shown only as fingerprints in `to_h`, `inspect`, logs and reports.
|
|
88
|
+
- **`Checks::LedgerChain`**, in both compliance suites.
|
|
89
|
+
- **`reeve:upgrade`**'s contract-3 step, which prints the initializer line, the rotation
|
|
90
|
+
warning and how to build the new index concurrently on a large PostgreSQL ledger. A step
|
|
91
|
+
now counts as applied only when its columns and its tables both exist.
|
|
92
|
+
|
|
93
|
+
### Fixed
|
|
94
|
+
|
|
95
|
+
- **A per-tool compliance check with no tools to check is skipped, not passed.** With no
|
|
96
|
+
tool registered, `compliance_tools` set to `[]`, or `tools: []`, `Reeve::Checks.run`
|
|
97
|
+
returned an empty `Report` — which reads as passed — so the RSpec shared group and
|
|
98
|
+
`ComplianceAssertions` showed all six per-tool checks green over zero tools. Each now
|
|
99
|
+
returns one skipped result, "no compliance tools to check, so GuardDeclared established
|
|
100
|
+
nothing — the registry is empty and no endpoint inventory routes a tool" (or `the run was
|
|
101
|
+
given tools: []`, or `compliance_tools is []`), shown as pending in RSpec and as a skip
|
|
102
|
+
in Minitest. Over an empty registry `run_all` used to report only the ledger-wide checks
|
|
103
|
+
(two in 0.5.0); it now also reports the six per-tool skips. Nothing that ran over at
|
|
104
|
+
least one tool changes.
|
|
105
|
+
- **The framework-free gate fails when a check had nothing to check.** The README's
|
|
106
|
+
`abort report.to_s unless report.passed?` stayed green over zero tools, because a skip is
|
|
107
|
+
not a failure. The documented gate is now
|
|
108
|
+
`abort report.to_s if report.failed? || report.nothing_checked?`, using the new
|
|
109
|
+
`Report#nothing_checked?`. A skip for a missing inventory does not trip it.
|
|
110
|
+
|
|
111
|
+
|
|
7
112
|
## [0.5.0] - 2026-10-01
|
|
8
113
|
|
|
9
114
|
Reeve protects the tools routed through it, and only those. A server that registers its
|
|
@@ -412,4 +517,4 @@ is for:
|
|
|
412
517
|
- Adding a nullable ledger column is minor and leaves the audit-entry contract version
|
|
413
518
|
alone. Removing or renaming a column, or changing what a value means, is major and bumps
|
|
414
519
|
it.
|
|
415
|
-
- The audit-entry contract version is `
|
|
520
|
+
- The audit-entry contract version is `3`.
|
data/README.md
CHANGED
|
@@ -7,7 +7,7 @@ A Ruby gem that makes it safe for a Rails application to expose MCP tools to AI
|
|
|
7
7
|
declarative per-record authorization, an append-only audit ledger, and a testing kit that
|
|
8
8
|
proves both hold.
|
|
9
9
|
|
|
10
|
-
> **Status: 0.
|
|
10
|
+
> **Status: 0.6.0.** Read the known limitations in
|
|
11
11
|
> [CHANGELOG.md](CHANGELOG.md) before adopting it — particularly the one about a
|
|
12
12
|
> transaction wrapped around an invocation, if your application wraps requests in one.
|
|
13
13
|
|
|
@@ -52,7 +52,10 @@ end
|
|
|
52
52
|
```
|
|
53
53
|
|
|
54
54
|
Until that resolver is set, every guarded call denies with `no_principal`. That is the
|
|
55
|
-
intended behaviour: the library is safe before it is configured.
|
|
55
|
+
intended behaviour: the library is safe before it is configured. The initializer also
|
|
56
|
+
derives the key that chains the audit ledger from `secret_key_base`, so that works in
|
|
57
|
+
every environment with nothing to set up — see [Tamper-evident, within stated
|
|
58
|
+
limits](#tamper-evident-within-stated-limits).
|
|
56
59
|
|
|
57
60
|
**2. Guard a tool.**
|
|
58
61
|
|
|
@@ -186,6 +189,108 @@ transaction holds the write lock; `IsolatedRecorder.available?` lets you branch
|
|
|
186
189
|
develop on one and deploy on the other. The default recorder warns when it detects the
|
|
187
190
|
case rather than failing silently.
|
|
188
191
|
|
|
192
|
+
## Tamper-evident, within stated limits
|
|
193
|
+
|
|
194
|
+
Every row Reeve writes is linked to the one before it by a keyed digest (HMAC-SHA256), with
|
|
195
|
+
a key the database does not hold. Someone who can write to the ledger table but does not
|
|
196
|
+
hold the key can still change it, but the change shows when the chain is checked:
|
|
197
|
+
|
|
198
|
+
```bash
|
|
199
|
+
bin/rails reeve:verify_ledger
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
```text
|
|
203
|
+
reeve ledger: INTACT — sequences 1–48,211 verified (head at start: 48,211); 5,000 earlier entries not covered (5,000 when the chain started)
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
It exits `0` when the chain is intact, `1` when it is broken, and `2` when it could not
|
|
207
|
+
verify (no chain, no key, or rows signed by a key it does not hold). Run it on a schedule
|
|
208
|
+
and alert on anything but `0`. `LIMIT=` caps the breaks it lists, never the totals;
|
|
209
|
+
`FORMAT=json` writes every break. The same answer from Ruby, or with an auditor's own copy
|
|
210
|
+
of the key:
|
|
211
|
+
|
|
212
|
+
```ruby
|
|
213
|
+
result = Reeve::Audit::Chain.verify
|
|
214
|
+
result.status # => :intact, :broken, :unverifiable or :no_chain
|
|
215
|
+
result.covered # => 1..48211
|
|
216
|
+
puts result # the report above, with every break listed under it
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
In CI the compliance suite you already run includes `LedgerChain`, in both frameworks:
|
|
220
|
+
it writes entries of its own and fails on a broken chain, a missing key, or entries that
|
|
221
|
+
were not chained.
|
|
222
|
+
|
|
223
|
+
**The key.** The generated initializer derives it from `secret_key_base`:
|
|
224
|
+
|
|
225
|
+
```ruby
|
|
226
|
+
config.ledger_key = Rails.application.key_generator.generate_key("reeve audit ledger", 32)
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
Rotating `secret_key_base` therefore rotates the ledger key. Keep the old one, or the
|
|
230
|
+
entries it signed become *unverifiable* — reported as such, never as tampered:
|
|
231
|
+
|
|
232
|
+
```ruby
|
|
233
|
+
config.retired_ledger_keys = [old_key] # verify with these; never sign with them
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
A dedicated key decouples the two: `config.ledger_key = Rails.application.credentials.reeve_ledger_key`.
|
|
237
|
+
Keys must be at least 32 bytes and are only ever printed as 16-character fingerprints.
|
|
238
|
+
Without a key, every guarded call fails closed; under `audit_failure_mode = :warn` it is
|
|
239
|
+
recorded unchained instead, logged each time, and reported by verification.
|
|
240
|
+
|
|
241
|
+
**Detected**, provided the key has not leaked:
|
|
242
|
+
|
|
243
|
+
- a chained row whose recorded content was changed;
|
|
244
|
+
- a chained row removed from anywhere before the newest one;
|
|
245
|
+
- chained rows swapped, renumbered or moved, or a row inserted that Reeve did not write —
|
|
246
|
+
including one numbered past the head;
|
|
247
|
+
- the newest rows removed while the chain record was left alone;
|
|
248
|
+
- the chain record removed while chained rows remain.
|
|
249
|
+
|
|
250
|
+
**Not detected** — stated plainly, because these are the limits of the guarantee:
|
|
251
|
+
|
|
252
|
+
1. **A leaked key.** Anyone holding it can rewrite the chain from any point forward. By
|
|
253
|
+
default that is anyone holding `secret_key_base`.
|
|
254
|
+
2. **The newest rows deleted by someone who also rewinds the chain record's head.** Only a
|
|
255
|
+
checkpoint held outside the database could catch that; Reeve does not ship one.
|
|
256
|
+
3. **The whole ledger deleted, chain record included.** Reported as *no chain*, never as a
|
|
257
|
+
pass — but indistinguishable from a ledger that was never chained.
|
|
258
|
+
4. **Rows written before you upgraded.** They are counted as "not covered" and are exactly
|
|
259
|
+
as trustworthy as they were before.
|
|
260
|
+
|
|
261
|
+
And it prevents nothing: it detects a change after the fact. The `GRANT INSERT, SELECT` in
|
|
262
|
+
the generated migration is what prevents one. The chain record needs `UPDATE` as well,
|
|
263
|
+
which is why a role allowed to rewind it is limit 2.
|
|
264
|
+
|
|
265
|
+
**What it costs.** About half a millisecond per ledger write on SQLite and about one on
|
|
266
|
+
PostgreSQL. Every chained write locks the chain record, so writes serialize. Under the
|
|
267
|
+
default recorder inside a transaction your application opened, that lock is held until the
|
|
268
|
+
transaction ends, and every other guarded call waits behind it — up to the database's own
|
|
269
|
+
lock wait: indefinitely on PostgreSQL (`lock_timeout` is 0 by default), 50 seconds on MySQL
|
|
270
|
+
(`innodb_lock_wait_timeout`), and ActiveRecord's `timeout` on SQLite (5 seconds in a
|
|
271
|
+
generated `database.yml`). `IsolatedRecorder` holds it for the length of one insert. On
|
|
272
|
+
**Rails 7.0 with SQLite**, a thread waiting for SQLite's write lock holds Ruby's GVL
|
|
273
|
+
(sqlite3 1.x), so concurrent guarded calls in one process can fail closed after the busy
|
|
274
|
+
timeout. Rails 7.1+ or a server database avoids it.
|
|
275
|
+
|
|
276
|
+
**Your own recorder.** A recorder that writes to Reeve's table chains its rows through the
|
|
277
|
+
same door the built-in ones use, or verification reports them as unchained:
|
|
278
|
+
|
|
279
|
+
```ruby
|
|
280
|
+
Reeve::Audit::Chain.new(entry_class: Reeve::Audit::Entry).append!(row)
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
**Upgrading from 0.5.0:**
|
|
284
|
+
|
|
285
|
+
```bash
|
|
286
|
+
bin/rails generate reeve:upgrade
|
|
287
|
+
bin/rails db:migrate
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
Then add the `config.ledger_key` line the generator prints. Existing rows are not touched;
|
|
291
|
+
the next guarded call starts the chain. Servers still on 0.5.0 during a rolling deploy keep
|
|
292
|
+
writing unchained rows, which are counted as not covered rather than reported as tampering.
|
|
293
|
+
|
|
189
294
|
## Provable in CI, in whichever framework you already use
|
|
190
295
|
|
|
191
296
|
All the logic lives in framework-neutral checks. RSpec and Minitest are thin front-ends
|
|
@@ -223,11 +328,13 @@ script, or a boot-time assertion in staging:
|
|
|
223
328
|
require "reeve/testing"
|
|
224
329
|
|
|
225
330
|
report = Reeve::Checks.run_all(principals: [alice, bob])
|
|
226
|
-
abort report.to_s
|
|
331
|
+
abort report.to_s if report.failed? || report.nothing_checked?
|
|
227
332
|
```
|
|
228
333
|
|
|
229
334
|
`alice` and `bob` are two fixture principals with disjoint records — that disjointness is
|
|
230
|
-
what makes a shared record identifier proof of a leak.
|
|
335
|
+
what makes a shared record identifier proof of a leak. `passed?` only means nothing failed,
|
|
336
|
+
so a run over no tools at all passes; `nothing_checked?` is what stops this gate when the
|
|
337
|
+
registry is empty, since a script has no pending test to show it.
|
|
231
338
|
|
|
232
339
|
**The suite walks every tool that included `Reeve::Guard`, not only the ones that declared
|
|
233
340
|
a guard.** A tool that forgot `guard_with` is the one worth finding, and it is absent from
|
|
@@ -257,7 +364,7 @@ namespace :reeve do
|
|
|
257
364
|
require "reeve/testing"
|
|
258
365
|
|
|
259
366
|
report = Reeve::Checks.run_all(principals: Reeve::Testing.compliance_principals)
|
|
260
|
-
abort report.to_s
|
|
367
|
+
abort report.to_s if report.failed? || report.nothing_checked?
|
|
261
368
|
puts report
|
|
262
369
|
end
|
|
263
370
|
end
|
|
@@ -470,7 +577,7 @@ ActiveRecord unless they are not — a plain object with an `id` works.
|
|
|
470
577
|
## What you have not gained
|
|
471
578
|
|
|
472
579
|
Rate limiting, prompt-injection defence, cost control, and connection authentication are
|
|
473
|
-
out of scope. This library governs *what an authenticated agent may touch* and *what it
|
|
580
|
+
out of scope, and the ledger's chain detects tampering rather than preventing it. This library governs *what an authenticated agent may touch* and *what it
|
|
474
581
|
touched*. Keep your existing auth.
|
|
475
582
|
|
|
476
583
|
Ruby 3.0+, Rails 7.0+. No runtime dependencies. The fast-mcp adapter needs Ruby 3.1+,
|
|
@@ -13,12 +13,23 @@
|
|
|
13
13
|
# REVOKE ALL ON reeve_audit_entries FROM app_role;
|
|
14
14
|
# GRANT INSERT, SELECT ON reeve_audit_entries TO app_role;
|
|
15
15
|
# GRANT USAGE ON SEQUENCE reeve_audit_entries_id_seq TO app_role;
|
|
16
|
+
# -- the chain record: its head moves on every write
|
|
17
|
+
# GRANT INSERT, SELECT, UPDATE ON reeve_audit_chains TO app_role;
|
|
18
|
+
# GRANT USAGE ON SEQUENCE reeve_audit_chains_id_seq TO app_role;
|
|
19
|
+
#
|
|
20
|
+
# Every row is chained to the one before it with a keyed digest (audit-entry contract 3),
|
|
21
|
+
# so a row changed, removed or reordered outside reeve is reported by
|
|
22
|
+
# `bin/rails reeve:verify_ledger`. That is tamper-evidence, not tamper-proofing: it
|
|
23
|
+
# detects a change after the fact and prevents nothing — the grant above is what
|
|
24
|
+
# prevents. It does not detect a change made by someone holding the ledger key, nor the
|
|
25
|
+
# newest rows being deleted by a role that can also rewind the head in
|
|
26
|
+
# reeve_audit_chains; that role needs UPDATE there, which is why the chain table is
|
|
27
|
+
# granted separately from the ledger.
|
|
16
28
|
#
|
|
17
29
|
# Also explicitly out of scope for now, so you can plan around it:
|
|
18
30
|
# * no retention, rotation or archival — this table grows until you prune it, and
|
|
19
|
-
# pruning is your policy, run by a role that is allowed to delete
|
|
20
|
-
#
|
|
21
|
-
# nullable column, which the audit-entry contract's versioning already permits.
|
|
31
|
+
# pruning is your policy, run by a role that is allowed to delete. Deleting rows
|
|
32
|
+
# breaks the chain by design.
|
|
22
33
|
#
|
|
23
34
|
# `invocation_id` is a UUID stored as a string rather than a native `uuid` column,
|
|
24
35
|
# because reeve supports every database ActiveRecord does and only PostgreSQL has the
|
|
@@ -61,6 +72,13 @@ class CreateReeveAuditEntries < ActiveRecord::Migration[7.0]
|
|
|
61
72
|
# it means "the caller passed none". Stamping each row keeps an export readable
|
|
62
73
|
# across an upgrade instead of collapsing that distinction.
|
|
63
74
|
t.integer :contract_version, null: false
|
|
75
|
+
|
|
76
|
+
# The chain (contract 3). Nullable: a row written before the chain began has no
|
|
77
|
+
# honest value for these, and nothing is backfilled into an append-only table.
|
|
78
|
+
# `chain_key_id` is the key's fingerprint, never the key.
|
|
79
|
+
t.bigint :chain_sequence
|
|
80
|
+
t.string :chain_key_id, limit: 16
|
|
81
|
+
t.string :chain_digest, limit: 64
|
|
64
82
|
end
|
|
65
83
|
|
|
66
84
|
add_index :reeve_audit_entries, :invocation_id, unique: true
|
|
@@ -73,5 +91,25 @@ class CreateReeveAuditEntries < ActiveRecord::Migration[7.0]
|
|
|
73
91
|
add_index :reeve_audit_entries, %i[tool_name occurred_at]
|
|
74
92
|
add_index :reeve_audit_entries, %i[agent_id occurred_at]
|
|
75
93
|
add_index :reeve_audit_entries, %i[outcome occurred_at]
|
|
94
|
+
|
|
95
|
+
# One position, one row. NULLs are distinct, so unchained rows never collide; the
|
|
96
|
+
# index also orders the walk `reeve:verify_ledger` makes.
|
|
97
|
+
add_index :reeve_audit_entries, :chain_sequence, unique: true
|
|
98
|
+
|
|
99
|
+
# The chain record: one row. The genesis columns are written once, when the first
|
|
100
|
+
# chained entry is; the head moves with every chained write, in the same transaction
|
|
101
|
+
# as the row it points at.
|
|
102
|
+
create_table :reeve_audit_chains do |t|
|
|
103
|
+
t.string :name, null: false
|
|
104
|
+
t.datetime :genesis_at, null: false, precision: 6
|
|
105
|
+
t.bigint :pre_chain_count, null: false
|
|
106
|
+
t.bigint :pre_chain_unchained_count, null: false
|
|
107
|
+
t.string :genesis_key_id, null: false, limit: 16
|
|
108
|
+
t.string :genesis_digest, null: false, limit: 64
|
|
109
|
+
t.bigint :head_sequence, null: false
|
|
110
|
+
t.string :head_digest, null: false, limit: 64
|
|
111
|
+
end
|
|
112
|
+
|
|
113
|
+
add_index :reeve_audit_chains, :name, unique: true
|
|
76
114
|
end
|
|
77
115
|
end
|
|
@@ -72,6 +72,26 @@ Reeve.configure do |config|
|
|
|
72
72
|
#
|
|
73
73
|
# Left unset, which resolves to Reeve::Audit::Recorder.
|
|
74
74
|
|
|
75
|
+
# ---------------------------------------------------------------------------
|
|
76
|
+
# The key that chains the audit ledger, so a row changed, removed or reordered
|
|
77
|
+
# outside reeve is reported by `bin/rails reeve:verify_ledger`. It must live
|
|
78
|
+
# outside the database: someone who can write the table but does not hold the
|
|
79
|
+
# key cannot forge the chain.
|
|
80
|
+
#
|
|
81
|
+
# Derived from secret_key_base, it works in every environment — test included —
|
|
82
|
+
# with no new secret to provision. Without a key, every guarded call fails
|
|
83
|
+
# closed (or, under audit_failure_mode :warn, is recorded unchained).
|
|
84
|
+
config.ledger_key = Rails.application.key_generator.generate_key("reeve audit ledger", 32)
|
|
85
|
+
|
|
86
|
+
# Rotating secret_key_base rotates this key too. Keep the old one here, or the
|
|
87
|
+
# entries it signed become unverifiable (reported as such, never as tampered):
|
|
88
|
+
#
|
|
89
|
+
# config.retired_ledger_keys = [old_key]
|
|
90
|
+
#
|
|
91
|
+
# Or use a dedicated key, so secret_key_base can rotate on its own schedule:
|
|
92
|
+
#
|
|
93
|
+
# config.ledger_key = Rails.application.credentials.reeve_ledger_key
|
|
94
|
+
|
|
75
95
|
# Cap on how many record identifiers one entry stores. The true count is
|
|
76
96
|
# always recorded, and the entry says when the list was truncated.
|
|
77
97
|
config.max_recorded_ids = 1000
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# Audit-entry contract 2 → 3. Generated by `bin/rails generate reeve:upgrade`.
|
|
4
|
+
#
|
|
5
|
+
# Chains every row reeve writes from here on to the one before it with a keyed digest, so a
|
|
6
|
+
# row changed, removed or reordered outside reeve is reported by
|
|
7
|
+
# `bin/rails reeve:verify_ledger`.
|
|
8
|
+
#
|
|
9
|
+
# Rows already in the table are not touched: there is no honest digest to give a row
|
|
10
|
+
# nobody was there to sign, and nothing is backfilled into an append-only table. They are
|
|
11
|
+
# reported as "not covered" from now on — exactly as trustworthy as they were before this
|
|
12
|
+
# migration, and no more. The three columns are nullable for that reason.
|
|
13
|
+
#
|
|
14
|
+
# The chain needs a key, held outside the database. Add this to
|
|
15
|
+
# config/initializers/reeve.rb, or every guarded call fails closed after you migrate:
|
|
16
|
+
#
|
|
17
|
+
# config.ledger_key = Rails.application.key_generator.generate_key("reeve audit ledger", 32)
|
|
18
|
+
#
|
|
19
|
+
# The chain record's head moves on every write, so the role your application connects as
|
|
20
|
+
# needs UPDATE on reeve_audit_chains (the ledger itself stays INSERT, SELECT):
|
|
21
|
+
#
|
|
22
|
+
# GRANT INSERT, SELECT, UPDATE ON reeve_audit_chains TO app_role;
|
|
23
|
+
#
|
|
24
|
+
# A role holding that UPDATE can rewind the head, and a rewound head hides the deletion of
|
|
25
|
+
# the newest rows. Tamper-evidence detects a change after the fact; the grants are what
|
|
26
|
+
# prevent one.
|
|
27
|
+
#
|
|
28
|
+
# On a very large ledger, the unique index below scans the table once. On PostgreSQL that
|
|
29
|
+
# scan blocks writes for its duration; to avoid it, add `disable_ddl_transaction!` to this
|
|
30
|
+
# class and `algorithm: :concurrently` to the add_index call.
|
|
31
|
+
class AddLedgerChainToReeveAuditEntries < ActiveRecord::Migration[7.0]
|
|
32
|
+
def change
|
|
33
|
+
add_column :reeve_audit_entries, :chain_sequence, :bigint
|
|
34
|
+
add_column :reeve_audit_entries, :chain_key_id, :string, limit: 16
|
|
35
|
+
add_column :reeve_audit_entries, :chain_digest, :string, limit: 64
|
|
36
|
+
add_index :reeve_audit_entries, :chain_sequence, unique: true
|
|
37
|
+
|
|
38
|
+
create_table :reeve_audit_chains do |t|
|
|
39
|
+
t.string :name, null: false
|
|
40
|
+
t.datetime :genesis_at, null: false, precision: 6
|
|
41
|
+
t.bigint :pre_chain_count, null: false
|
|
42
|
+
t.bigint :pre_chain_unchained_count, null: false
|
|
43
|
+
t.string :genesis_key_id, null: false, limit: 16
|
|
44
|
+
t.string :genesis_digest, null: false, limit: 64
|
|
45
|
+
t.bigint :head_sequence, null: false
|
|
46
|
+
t.string :head_digest, null: false, limit: 64
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
add_index :reeve_audit_chains, :name, unique: true
|
|
50
|
+
end
|
|
51
|
+
end
|
|
@@ -33,15 +33,29 @@ module Reeve
|
|
|
33
33
|
# contracts/audit-entry.md, which exists so that stays hypothetical: an append-only
|
|
34
34
|
# ledger cannot be backfilled, so a column that is not additive has no honest value
|
|
35
35
|
# to give the rows already written.
|
|
36
|
+
# `creates` names any table a step builds. A step is applied only when every column it
|
|
37
|
+
# adds and every table it creates exist: a ledger with one half and not the other is
|
|
38
|
+
# half-migrated, and calling it current would leave the host with no way forward.
|
|
36
39
|
STEPS = [
|
|
37
40
|
{
|
|
38
41
|
contract: 2,
|
|
39
42
|
adds: %w[contract_version],
|
|
43
|
+
creates: [],
|
|
40
44
|
template: "add_contract_version_to_reeve_audit_entries.rb.tt",
|
|
41
45
|
destination: "add_contract_version_to_reeve_audit_entries.rb"
|
|
46
|
+
},
|
|
47
|
+
{
|
|
48
|
+
contract: 3,
|
|
49
|
+
adds: %w[chain_sequence chain_key_id chain_digest],
|
|
50
|
+
creates: %w[reeve_audit_chains],
|
|
51
|
+
template: "add_ledger_chain_to_reeve_audit_entries.rb.tt",
|
|
52
|
+
destination: "add_ledger_chain_to_reeve_audit_entries.rb"
|
|
42
53
|
}
|
|
43
54
|
].freeze
|
|
44
55
|
|
|
56
|
+
KEY_LINE = "config.ledger_key = " \
|
|
57
|
+
'Rails.application.key_generator.generate_key("reeve audit ledger", 32)'
|
|
58
|
+
|
|
45
59
|
source_root File.expand_path("templates", __dir__)
|
|
46
60
|
|
|
47
61
|
desc "Brings an existing reeve audit ledger up to the current audit-entry contract."
|
|
@@ -93,7 +107,7 @@ module Reeve
|
|
|
93
107
|
end
|
|
94
108
|
|
|
95
109
|
def pending_message
|
|
96
|
-
<<~NEXT
|
|
110
|
+
message = <<~NEXT
|
|
97
111
|
|
|
98
112
|
#{@emitted.size} migration#{'s' if @emitted.size != 1} written, taking the ledger
|
|
99
113
|
to audit-entry contract #{@emitted.last[:contract]}. Run:
|
|
@@ -103,13 +117,37 @@ module Reeve
|
|
|
103
117
|
Existing rows are stamped with the contract they were actually written under,
|
|
104
118
|
not with the current one.
|
|
105
119
|
NEXT
|
|
120
|
+
message += chain_message if @emitted.any? { |step| step[:contract] == 3 }
|
|
121
|
+
message
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
# Contract 3 changes what the application must configure, not only the table: without
|
|
125
|
+
# a key, every guarded call fails closed once the migration runs. The host is reading
|
|
126
|
+
# this output at exactly the moment they need to know that.
|
|
127
|
+
def chain_message
|
|
128
|
+
<<~CHAIN
|
|
129
|
+
|
|
130
|
+
The ledger is now chained (contract 3). Add this line to config/initializers/reeve.rb
|
|
131
|
+
before you deploy, or every guarded call fails closed:
|
|
132
|
+
|
|
133
|
+
#{KEY_LINE}
|
|
134
|
+
|
|
135
|
+
It derives the key from secret_key_base, so rotating secret_key_base rotates the
|
|
136
|
+
ledger key: keep the old derived key in config.retired_ledger_keys, or entries it
|
|
137
|
+
signed become unverifiable. A dedicated key avoids that coupling.
|
|
138
|
+
|
|
139
|
+
Very large ledger on PostgreSQL? The migration builds a unique index that blocks
|
|
140
|
+
writes while it scans; the migration's header shows how to build it with
|
|
141
|
+
algorithm: :concurrently instead.
|
|
142
|
+
CHAIN
|
|
106
143
|
end
|
|
107
144
|
|
|
108
145
|
# Asked of the table rather than of a version the host recorded: the table is the
|
|
109
146
|
# thing being changed, and it is the only party that cannot be out of date about
|
|
110
147
|
# itself.
|
|
111
148
|
def applied?(step)
|
|
112
|
-
(step[:adds] - existing_columns).empty?
|
|
149
|
+
(step[:adds] - existing_columns).empty? &&
|
|
150
|
+
step[:creates].all? { |table| connection.table_exists?(table) }
|
|
113
151
|
end
|
|
114
152
|
|
|
115
153
|
def existing_columns
|
|
@@ -0,0 +1,129 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "bigdecimal"
|
|
4
|
+
require "json"
|
|
5
|
+
require "openssl"
|
|
6
|
+
|
|
7
|
+
module Reeve
|
|
8
|
+
module Audit
|
|
9
|
+
class Chain
|
|
10
|
+
# The exact text a digest is computed over (specs/002 FR-002, FR-003; research R1, R2
|
|
11
|
+
# step 2; data-model "Canonical entry").
|
|
12
|
+
#
|
|
13
|
+
# Computed once when a row is written and again, from what the database returns,
|
|
14
|
+
# when it is verified — so everything here exists to make those two readings produce
|
|
15
|
+
# identical text:
|
|
16
|
+
#
|
|
17
|
+
# * object keys sorted at every depth, so a database that reorders JSON keys
|
|
18
|
+
# (PostgreSQL jsonb, MySQL JSON) changes nothing;
|
|
19
|
+
# * one spelling per number, so `1.0e+20` written and `100000000000000000000` read
|
|
20
|
+
# back are the same number;
|
|
21
|
+
# * times as integer microseconds since the epoch, so neither the time zone nor the
|
|
22
|
+
# driver's Time class matters;
|
|
23
|
+
# * strings, booleans and nil typed, so "1" and 1 never digest alike.
|
|
24
|
+
#
|
|
25
|
+
# The field list is written out rather than read from the table — a check that
|
|
26
|
+
# derives its expectation from the thing it checks checks nothing — and is keyed by
|
|
27
|
+
# contract version, so a future column joins the digest for rows written under that
|
|
28
|
+
# contract only. The tags keep an entry digest and a genesis digest from ever being
|
|
29
|
+
# interchangeable, and let a future canonical form be told apart from this one.
|
|
30
|
+
module Canonical
|
|
31
|
+
ENTRY_TAG = "reeve-ledger-entry-v1"
|
|
32
|
+
GENESIS_TAG = "reeve-ledger-genesis-v1"
|
|
33
|
+
|
|
34
|
+
ENTRY_FIELDS = {
|
|
35
|
+
3 => %w[
|
|
36
|
+
chain_sequence chain_key_id contract_version
|
|
37
|
+
invocation_id occurred_at agent_id agent_name principal_type principal_id
|
|
38
|
+
tool_name arguments outcome rule detail record_type record_ids record_count
|
|
39
|
+
truncated derived guard duration_ms metadata
|
|
40
|
+
].freeze
|
|
41
|
+
}.freeze
|
|
42
|
+
|
|
43
|
+
GENESIS_FIELDS = %w[genesis_at pre_chain_count pre_chain_unchained_count
|
|
44
|
+
genesis_key_id].freeze
|
|
45
|
+
|
|
46
|
+
TIME_FIELDS = %w[occurred_at genesis_at].freeze
|
|
47
|
+
|
|
48
|
+
module_function
|
|
49
|
+
|
|
50
|
+
def entry_digest(secret, row, previous_digest:)
|
|
51
|
+
hmac(secret, ENTRY_TAG, entry_message_body(row, previous_digest))
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
def genesis_digest(secret, genesis)
|
|
55
|
+
pairs = GENESIS_FIELDS.map { |field| [field, value(genesis, field)] }
|
|
56
|
+
hmac(secret, GENESIS_TAG, dump(pairs))
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
def entry_message_body(row, previous_digest)
|
|
60
|
+
version = value(row, "contract_version")
|
|
61
|
+
fields = ENTRY_FIELDS.fetch(version) do
|
|
62
|
+
raise ArgumentError,
|
|
63
|
+
"there is no canonical field list for audit-entry contract " \
|
|
64
|
+
"#{version.inspect}; only contract " \
|
|
65
|
+
"#{ENTRY_FIELDS.keys.join(', ')} rows are chained"
|
|
66
|
+
end
|
|
67
|
+
|
|
68
|
+
pairs = fields.map { |field| [field, value(row, field)] }
|
|
69
|
+
dump([["previous_digest", previous_digest], *pairs])
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
def microseconds(time)
|
|
73
|
+
(time.to_r * 1_000_000).floor
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
def dump(value)
|
|
77
|
+
case value
|
|
78
|
+
when Array then "[#{value.map { |item| dump(item) }.join(',')}]"
|
|
79
|
+
when Hash then object(value)
|
|
80
|
+
else scalar(value)
|
|
81
|
+
end
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
def scalar(value)
|
|
85
|
+
case value
|
|
86
|
+
when nil, true, false then value.nil? ? "null" : value.to_s
|
|
87
|
+
when Integer, Float, BigDecimal then number(value)
|
|
88
|
+
when String, Symbol then JSON.generate(value.to_s)
|
|
89
|
+
else
|
|
90
|
+
raise ArgumentError,
|
|
91
|
+
"#{value.class} has no canonical form; normalize it before it is digested"
|
|
92
|
+
end
|
|
93
|
+
end
|
|
94
|
+
|
|
95
|
+
def object(hash)
|
|
96
|
+
pairs = hash.map { |key, item| [key.to_s, item] }.sort_by(&:first)
|
|
97
|
+
"{#{pairs.map { |key, item| "#{JSON.generate(key)}:#{dump(item)}" }.join(',')}}"
|
|
98
|
+
end
|
|
99
|
+
|
|
100
|
+
# The shortest exact decimal: no exponent, no trailing zeros, no "-0".
|
|
101
|
+
def number(value)
|
|
102
|
+
if value.is_a?(Float) && !value.finite?
|
|
103
|
+
raise ArgumentError,
|
|
104
|
+
"a non-finite number has no canonical form; normalize it before it is digested"
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
decimal = value.is_a?(Float) ? BigDecimal(value.to_s) : BigDecimal(value)
|
|
108
|
+
return "0" if decimal.zero?
|
|
109
|
+
return decimal.to_i.to_s if decimal.frac.zero?
|
|
110
|
+
|
|
111
|
+
decimal.to_s("F")
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
def value(row, field)
|
|
115
|
+
raw = if row.is_a?(Hash)
|
|
116
|
+
row.key?(field) ? row[field] : row[field.to_sym]
|
|
117
|
+
else
|
|
118
|
+
row[field]
|
|
119
|
+
end
|
|
120
|
+
TIME_FIELDS.include?(field) && !raw.nil? ? microseconds(raw) : raw
|
|
121
|
+
end
|
|
122
|
+
|
|
123
|
+
def hmac(secret, tag, body)
|
|
124
|
+
OpenSSL::HMAC.hexdigest("SHA256", secret, "#{tag}\n#{body}")
|
|
125
|
+
end
|
|
126
|
+
end
|
|
127
|
+
end
|
|
128
|
+
end
|
|
129
|
+
end
|