reeve 0.7.0 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (45) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +117 -0
  3. data/README.md +236 -9
  4. data/lib/generators/reeve/install/templates/create_audit_entries.rb.tt +29 -4
  5. data/lib/generators/reeve/install/templates/initializer.rb.tt +10 -0
  6. data/lib/generators/reeve/upgrade/templates/add_ledger_retention_to_reeve_audit_entries.rb.tt +37 -0
  7. data/lib/generators/reeve/upgrade/upgrade_generator.rb +10 -1
  8. data/lib/generators/reeve/upgrade/upgrade_notes.rb +15 -4
  9. data/lib/reeve/audit/chain/anchoring.rb +91 -0
  10. data/lib/reeve/audit/chain/canonical.rb +17 -1
  11. data/lib/reeve/audit/chain/checkpoint.rb +228 -0
  12. data/lib/reeve/audit/chain/checkpoint_check.rb +156 -0
  13. data/lib/reeve/audit/chain/entrypoints.rb +29 -0
  14. data/lib/reeve/audit/chain/epochs.rb +10 -4
  15. data/lib/reeve/audit/chain/head.rb +26 -1
  16. data/lib/reeve/audit/chain/keyring.rb +6 -0
  17. data/lib/reeve/audit/chain/report.rb +188 -31
  18. data/lib/reeve/audit/chain/retention/archive.rb +98 -0
  19. data/lib/reeve/audit/chain/retention/archive_check.rb +179 -0
  20. data/lib/reeve/audit/chain/retention/check.rb +94 -0
  21. data/lib/reeve/audit/chain/retention/connection.rb +65 -0
  22. data/lib/reeve/audit/chain/retention/digest.rb +29 -0
  23. data/lib/reeve/audit/chain/retention/plan.rb +100 -0
  24. data/lib/reeve/audit/chain/retention/preflight.rb +48 -0
  25. data/lib/reeve/audit/chain/retention/purge.rb +155 -0
  26. data/lib/reeve/audit/chain/retention/purge_result.rb +134 -0
  27. data/lib/reeve/audit/chain/retention/records.rb +23 -0
  28. data/lib/reeve/audit/chain/verification.rb +31 -9
  29. data/lib/reeve/audit/chain/verifier.rb +64 -19
  30. data/lib/reeve/audit/chain/walk.rb +9 -3
  31. data/lib/reeve/audit/chain.rb +12 -3
  32. data/lib/reeve/audit/chain_record.rb +3 -2
  33. data/lib/reeve/audit/file_anchor.rb +49 -0
  34. data/lib/reeve/audit/retention_record.rb +24 -0
  35. data/lib/reeve/audit.rb +16 -0
  36. data/lib/reeve/configuration.rb +25 -0
  37. data/lib/reeve/errors.rb +15 -0
  38. data/lib/reeve/railtie.rb +2 -2
  39. data/lib/reeve/tasks/descriptions.rb +22 -0
  40. data/lib/reeve/tasks/purge_options.rb +54 -0
  41. data/lib/reeve/tasks.rb +82 -14
  42. data/lib/reeve/testing/checks/ledger_chain.rb +56 -2
  43. data/lib/reeve/testing/ledger.rb +19 -3
  44. data/lib/reeve/version.rb +1 -1
  45. metadata +21 -2
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 4a50e8fe479452d3b685dcd79b3022881c76567651444c6204e8db3d601b07c8
4
- data.tar.gz: fe9974e2c035ce7c958f75d44a847d4b951f9cd828820aba9468310e2a143acd
3
+ metadata.gz: 5fec548cc5008459901b118b5bd79b3a3c47899554199c7a8a686ca3e9528cc8
4
+ data.tar.gz: 277700a1ad87207e90b693e96cfb2536ffec41459d420bd9d93d9fe08a1ebd68
5
5
  SHA512:
6
- metadata.gz: aa5ff3ffce5221ee1b8baa64461865842faed31d432a0977051f6f3ec8be1846474174d27e19037a1a6b6901af5c6329fd38ac390e2da3fc664f7dfe9b7214ee
7
- data.tar.gz: d986d9bd16574387f5529e983e932c1e740068a9b4a833a7220cf03dee30b3e134a5fd101a0ee41d2091bf3fa6660cc7ac67d52d720039bc149e32e6ec1dd33a
6
+ metadata.gz: 077330b44a698212477ea31db93ade9d0242c8c1a85ac546363a85a296ca5faaadb3865b058720ac74fb9e1a71e2e0afa72c42eb0687b78707cd5588cbad3394
7
+ data.tar.gz: f6b915ea4c1fb5ade7a2d07c71cc199b5446eca9dcfb8beecbdc233f1d7574e754b026de3a6ecd309fa82f9205af99b65a65c0182e8e5ba5fe31a7c57585fab8
data/CHANGELOG.md CHANGED
@@ -4,6 +4,121 @@ All notable changes are recorded here. This project follows [Semantic
4
4
  Versioning](https://semver.org), with one rule specific to what it does — see
5
5
  [Versioning policy](#versioning-policy).
6
6
 
7
+ ## [0.8.0] - 2026-10-25
8
+
9
+ Two additions to the tamper-evident ledger: **anchoring**, checkpoints of the chain kept
10
+ outside the database, so deleting the newest entries or the whole ledger is caught; and
11
+ **retention**, purging the oldest entries up to a checkpoint without breaking verification.
12
+ No audit-entry contract change: it stays `4`.
13
+
14
+ ### Upgrading from 0.7.0
15
+
16
+ 1. `bundle update reeve`, then `bin/rails generate reeve:upgrade && bin/rails db:migrate`. It
17
+ adds one table, `reeve_audit_retentions`; nothing in the ledger table changes.
18
+ 2. Grant the application's database role `SELECT` on `reeve_audit_retentions`.
19
+ 3. If anything reads the text output of `bin/rails reeve:verify_ledger`, switch it to
20
+ `FORMAT=json`: every summary line now ends with its anchoring (`; not anchored` until you
21
+ set one up). Existing JSON keys are unchanged.
22
+
23
+ Anchoring and retention are both opt-in; the README shows how to set each up.
24
+
25
+ ### Anchoring
26
+
27
+ The chain's own records live in the same database as the entries, so two kinds of tampering
28
+ went unseen: deleting the newest entries and rewinding the head to match, and deleting the
29
+ whole ledger, which read as "no chain". A new epoch nobody had written under yet could also
30
+ be deleted without trace. Now reeve can make a **checkpoint**, one signed line saying how far
31
+ the chain reached, and hand it to an **anchor**: storage you choose, outside the database.
32
+ Verification checks the ledger against every checkpoint.
33
+
34
+ **No contract change.** The audit-entry contract stays `4`. Nothing changes for a ledger
35
+ write. (Retention, below, adds one table.)
36
+
37
+ #### Added
38
+
39
+ - `bin/rails reeve:checkpoint_ledger` (`Reeve::Tasks.checkpoint_ledger`,
40
+ `Reeve::Audit::Chain.checkpoint!`): makes a checkpoint, publishes it, and prints it. Exit `0`
41
+ made, `1` refused (no key, no chain yet) or the anchor failed; the line is printed either way
42
+ so it can be stored by hand.
43
+ - `config.ledger_anchor`: any object with `publish(line)` and `checkpoints`. reeve ships
44
+ `Reeve::Audit::FileAnchor`, a file it only ever appends to. It is a building block: on the
45
+ database host it protects nothing. Point it at storage the database's users cannot change,
46
+ such as a mount backed by an object-locked bucket, and use one per environment.
47
+ - `reeve:verify_ledger` checks against the anchor's checkpoints when one is configured.
48
+ `CHECKPOINTS=path` reads a file of checkpoint lines instead, `CHECKPOINTS=` (empty) uses
49
+ none, and `FROM=latest` walks only what came after the newest checkpoint. A checkpoint line
50
+ that does not read is exit `2`, naming its position.
51
+ - New breaks, all exit `1`: `checkpoint_not_reached`, `checkpoint_mismatch`,
52
+ `checkpoint_altered`. A checkpoint signed by a key not configured is unverifiable (exit `2`).
53
+ With a checkpoint, an emptied ledger is BROKEN, not NO CHAIN.
54
+ - `reeve:rotate_ledger_key` makes a checkpoint after opening an epoch when an anchor is
55
+ configured, so the new epoch is pinned before anything is written under it.
56
+ - `Reeve::Checks::LedgerChain.new(anchored: true)`: also makes a checkpoint, reads it back
57
+ from the anchor and verifies against it. Run it as a test of its own (the README shows how);
58
+ the suite's own `LedgerChain` never reads the anchor, since checkpoints earlier runs left
59
+ name test databases that are gone.
60
+
61
+ #### Changed
62
+
63
+ - **Every summary line that follows a walk now ends with its anchoring**: `; not anchored`,
64
+ or `; anchored at #48,000 (checkpoint 2026-10-25 14:00 UTC), 211 entries after it`. This
65
+ applies whether or not you configure an anchor. If something reads the text report, read
66
+ `FORMAT=json` instead; its existing keys are unchanged, and it gains `checkpoints`,
67
+ `anchored_at`, `beyond_anchor`, `walked_from` and `breaks[].checkpoint`.
68
+
69
+ #### Limits, stated plainly
70
+
71
+ - Entries written after the latest checkpoint can still be removed without trace. The summary
72
+ line counts the entries still there after it, not ones deleted; how often you checkpoint is
73
+ the size of that window.
74
+ - A checkpoint is only as safe as its store. One an attacker can delete or overwrite protects
75
+ nothing, and reeve cannot make a store append-only.
76
+ - A leaked key can still rewrite the chain from any point, but not a checkpoint made before
77
+ the leak: rewriting an entry it saw changes the digest it names.
78
+ - Restoring a backup older than a checkpoint reads as BROKEN, because the restored ledger lost
79
+ entries that checkpoint saw. A database reset on purpose needs a new anchor file.
80
+ - A checkpoint's time is the producing host's clock, for people. Verification never uses it.
81
+
82
+ ### Retention
83
+
84
+ The ledger could only grow, and deleting old rows by hand broke the chain and read as
85
+ tampering. Now `bin/rails reeve:purge_ledger` removes the oldest entries up to a checkpoint
86
+ your anchor holds, writes an archive of them first, and leaves a signed **retention record**, so
87
+ the rest still verifies and the purge reads as a purge.
88
+
89
+ **One migration, no contract change.** `bin/rails generate reeve:upgrade` adds the table
90
+ `reeve_audit_retentions`; the audit-entry contract stays `4`. Grant the application's role
91
+ `SELECT` on it. Nothing is purged until you run the task.
92
+
93
+ **The constitution's append-only rule now has one exception** (1.1.0): application code still
94
+ cannot update or delete an entry; only this operator-run purge can, and only behind a verified,
95
+ anchored checkpoint, on a database role the application does not use, after writing a signed
96
+ record.
97
+
98
+ - `config.ledger_retention_connection`: an abstract ActiveRecord class connected to the ledger's
99
+ database as a role with `DELETE`; the purge refuses another database, or the application's
100
+ own user (SQLite has no roles, so there it warns).
101
+ - `bin/rails reeve:purge_ledger`: `OLDER_THAN=365d` or `THROUGH=sequence`; exactly one of
102
+ `ARCHIVE=path` (a new file each run) and `NO_ARCHIVE=1`; nothing removed without `CONFIRM=1`.
103
+ Exit `0` for a dry run, a purge, or nothing to purge (the first line says which); `1` refused,
104
+ with the reason and the fix. It requires the whole ledger to verify INTACT first. A purge
105
+ stopped midway leaves rows "awaiting purge" (INTACT, not a break); any rerun finishes it first.
106
+ The README shows the role, the grants and the `database.yml` entry.
107
+ - `bin/rails reeve:verify_archive ARCHIVE=path`: verifies an archive on its own, from its start to
108
+ its boundary checkpoint. It needs the application (for the table's column types), not the
109
+ purged rows.
110
+ - Verification starts after the latest purge, names the purged range on every summary line,
111
+ reports checkpoints older than it as purged, and gains `retention_altered` (BROKEN) and the JSON
112
+ keys `retention` and `awaiting_purge`.
113
+
114
+ Keep every ledger key: epoch and retention records stay in the ledger and are checked under
115
+ their own keys, and archives need the keys of the rows they hold.
116
+
117
+ Limits, in plain words: what a purge removed without an archive is gone for good, and the record
118
+ proves only that it existed and verified. An archive is a file you must keep safe, as you do the
119
+ anchor. A row edited inside the purged range while the purge runs is not reported by the ledger,
120
+ because it is being removed; the archive holds what was read.
121
+
7
122
  ## [0.7.0] - 2026-10-23
8
123
 
9
124
  0.6.0 let a ledger be verified across a key rotation, provided the old key was kept as a
@@ -690,4 +805,6 @@ is for:
690
805
  alone — unless what its absence means, or what an existing value covers, differs between
691
806
  rows written before and after it, which is major (contracts 3 and 4 are both examples).
692
807
  Removing or renaming a column, or changing what a value means, is major and bumps it.
808
+ - A new table beside the ledger that leaves its rows unchanged is minor and leaves the
809
+ contract alone (`reeve_audit_retentions`, 0.8.0).
693
810
  - The audit-entry contract version is `4`.
data/README.md CHANGED
@@ -7,7 +7,7 @@ A Ruby gem that makes it safe for a Rails application to expose MCP tools to AI
7
7
  declarative per-record authorization, an append-only audit ledger, and a testing kit that
8
8
  proves both hold.
9
9
 
10
- > **Status: 0.7.0.** Read the known limitations in
10
+ > **Status: 0.8.0.** Read the known limitations in
11
11
  > [CHANGELOG.md](CHANGELOG.md) before adopting it — particularly the one about a
12
12
  > transaction wrapped around an invocation, if your application wraps requests in one.
13
13
 
@@ -200,7 +200,7 @@ bin/rails reeve:verify_ledger
200
200
  ```
201
201
 
202
202
  ```text
203
- reeve ledger: INTACT — sequences 1–48,211 verified (head at start: 48,211); 5,000 earlier entries not covered (5,000 when the chain started)
203
+ reeve ledger: INTACT — sequences 1–48,211 verified (head at start: 48,211); 5,000 earlier entries not covered (5,000 when the chain started); not anchored
204
204
  ```
205
205
 
206
206
  It exits `0` when the chain is intact, `1` when it is broken, and `2` when it could not
@@ -218,7 +218,8 @@ puts result # the report above, with every break listed under it
218
218
 
219
219
  In CI the compliance suite you already run includes `LedgerChain`, in both frameworks:
220
220
  it writes entries of its own and fails on a broken chain, a missing key, or entries that
221
- were not chained.
221
+ were not chained. "Not anchored" on the summary line is explained under **Anchoring the
222
+ chain** below.
222
223
 
223
224
  **The key.** The generated initializer derives it from `secret_key_base`:
224
225
 
@@ -256,6 +257,216 @@ rotation.epoch # => 1
256
257
  Verification then lists each epoch under its summary line, with the key fingerprint, where
257
258
  it begins, and whether it verified. The CHANGELOG has the full procedure.
258
259
 
260
+ **Anchoring the chain.** The chain's own records live in the same database as the entries, so
261
+ someone who deletes the newest entries and rewinds the head to match, or deletes the whole
262
+ ledger, leaves nothing behind that verification can see. A *checkpoint* is a second witness:
263
+ one signed line saying how far the chain reached (its genesis, its head, its latest key
264
+ epoch), kept where the database's users cannot change it. It holds digests and counts only,
265
+ never entry content or a key.
266
+
267
+ Point an anchor at that storage. The line goes at the bottom of
268
+ `config/initializers/reeve.rb`, after `require "reeve/audit"` (which defines `FileAnchor`),
269
+ outside the `Reeve.configure` block, hence `Reeve.config`:
270
+
271
+ ```ruby
272
+ require "reeve/audit"
273
+
274
+ Reeve.config.ledger_anchor =
275
+ Reeve::Audit::FileAnchor.new("/mnt/ledger-anchor/#{Rails.env}.checkpoints")
276
+ ```
277
+
278
+ Use one anchor per environment, on storage outside the database host, such as a mount backed
279
+ by an object-locked bucket; a file next to the database protects nothing. The hosts that make
280
+ checkpoints need to write the file (it is created on the first one), and the hosts that verify
281
+ need to read it. Any object with `publish(line)` and `checkpoints` works in its place.
282
+
283
+ Then make a checkpoint on a schedule, from one host (hourly, say), and keep verifying as
284
+ before. Alert on anything but `0` from either task. `reeve:checkpoint_ledger` exits `1` when
285
+ it refused (no key, or no chain until the first guarded call) or the anchor failed, and still
286
+ prints the line so it can be stored by hand. Two runs at once are safe.
287
+
288
+ ```bash
289
+ bin/rails reeve:checkpoint_ledger # exit 0 made and published
290
+ bin/rails reeve:verify_ledger # checks the ledger against every checkpoint
291
+ ```
292
+
293
+ ```text
294
+ reeve ledger: INTACT — sequences 1–48,211 verified (head at start: 48,211); 5,000 earlier entries not covered (5,000 when the chain started); anchored at #48,000 (checkpoint 2026-10-25 14:00 UTC), 211 entries after it
295
+ ```
296
+
297
+ Every checkpoint is checked: a ledger that no longer reaches one, holds another entry where it
298
+ saw one, or lost an epoch record it named, is BROKEN (exit `1`), and so is an emptied ledger,
299
+ which without a checkpoint reads only as "no chain". The newest entries deleted with the head
300
+ rewound read like this:
301
+
302
+ ```text
303
+ reeve ledger: BROKEN — 1 break in sequences 1–46,211; 5,000 earlier entries not covered (5,000 when the chain started); anchored at #44,000 (checkpoint 2026-10-25 13:00 UTC), 2,211 entries after it
304
+ totals: checkpoint_not_reached 1
305
+ checkpoint #48,000 checkpoint_not_reached the head is at 46,211; entries the checkpoint saw were removed
306
+ ```
307
+
308
+ What it does not cover: entries written after the latest checkpoint can still be removed
309
+ without trace. "211 entries after it" counts the entries still there past the checkpoint; it
310
+ cannot see ones that were deleted, so how often you checkpoint is the size of that window.
311
+ `reeve:rotate_ledger_key` makes a checkpoint itself after opening an epoch. Writes never touch
312
+ the anchor.
313
+
314
+ *Restores and resets.* Restoring a backup older than a checkpoint reads as BROKEN, truthfully:
315
+ the restored ledger lost entries that checkpoint saw. Resetting a database on purpose (a
316
+ staging rebuild) is the same, so start a new anchor file for the new ledger: change the path
317
+ (`staging-2026-10-25.checkpoints`, say), keep the old file as the record of the old ledger, and
318
+ make a checkpoint. Until then the run reads "not anchored". `CHECKPOINTS=` (empty) verifies
319
+ without any checkpoint, for a one-off check, never as the fix.
320
+
321
+ *Other sources, large ledgers.* `CHECKPOINTS=path` verifies against a file of checkpoint lines
322
+ instead of the anchor, say an auditor's copy. `FROM=latest` walks only what came after the
323
+ newest checkpoint: it first checks that checkpoint (its signature, the genesis and epoch it
324
+ names, and that its entry still has the digest it saw), then verifies every entry after it.
325
+ Entries before it are not re-verified by that run, and the summary says how many; keep a full
326
+ run on a slower schedule. With no checkpoint to start from it exits `2`. From Ruby:
327
+
328
+ ```ruby
329
+ Reeve::Audit::Chain.checkpoint!.checkpoint.to_line # => "{\"epoch\":1,…}"
330
+ Reeve::Audit::Chain.verify(checkpoints: lines).anchored_at # the highest checkpoint that held
331
+ ```
332
+
333
+ *In CI.* The `LedgerChain` check the compliance suite runs never reads the anchor, so
334
+ configuring one does not change it: a test database is emptied between runs, and old
335
+ checkpoints name ledgers that are gone. To prove the anchor works, run the anchored variant as
336
+ a test of its own, with the test environment's anchor pointed at a scratch file such as
337
+ `Rails.root.join("tmp/ledger.checkpoints")`. It makes a checkpoint, reads it back through the
338
+ anchor, and verifies against that one alone:
339
+
340
+ ```ruby
341
+ # RSpec
342
+ it "anchors the ledger" do
343
+ result = Reeve::Checks::LedgerChain.new(anchored: true).call
344
+ expect(result).to be_passed, result.message
345
+ end
346
+
347
+ # Minitest
348
+ def test_anchors_the_ledger
349
+ result = Reeve::Checks::LedgerChain.new(anchored: true).call
350
+ assert result.passed?, result.message
351
+ end
352
+ ```
353
+
354
+ **Purging old entries.** The ledger grows until you purge it. Deleting rows by hand breaks
355
+ the chain and is reported as tampering, so reeve purges for you: the oldest entries, up to a
356
+ checkpoint your anchor holds, after verifying the whole ledger INTACT, leaving a signed
357
+ *retention record* so the rest still verifies and the purge reads as a purge. It also removes
358
+ rows written before the chain began, and nothing else. It stops at a checkpoint, so with
359
+ `OLDER_THAN=365d` a few entries just over a year old can remain until the next run.
360
+
361
+ *Setting it up, once.* Run `bin/rails generate reeve:upgrade && bin/rails db:migrate` (it adds
362
+ `reeve_audit_retentions`), then create a database role only the purge uses; the application's
363
+ role never gains `DELETE`. On PostgreSQL (the migration's header shows the same grants):
364
+
365
+ ```sql
366
+ CREATE ROLE reeve_retention LOGIN PASSWORD '…';
367
+ GRANT SELECT ON reeve_audit_entries, reeve_audit_chains, reeve_audit_chain_epochs,
368
+ reeve_audit_retentions TO reeve_retention;
369
+ GRANT DELETE ON reeve_audit_entries TO reeve_retention;
370
+ GRANT INSERT ON reeve_audit_retentions TO reeve_retention;
371
+ GRANT USAGE ON SEQUENCE reeve_audit_retentions_id_seq TO reeve_retention;
372
+ GRANT SELECT ON reeve_audit_retentions TO app_role; -- the application only reads it
373
+ ```
374
+
375
+ Add the role to `config/database.yml` as a second database entry pointing at the same
376
+ database, and a class that connects through it:
377
+
378
+ ```yaml
379
+ production:
380
+ primary:
381
+ <<: *default
382
+ database: app_production
383
+ ledger_retention:
384
+ <<: *default
385
+ database: app_production # the same database
386
+ username: reeve_retention
387
+ password: <%= ENV["REEVE_RETENTION_PASSWORD"] %>
388
+ database_tasks: false # never migrated through this entry
389
+ ```
390
+
391
+ ```ruby
392
+ # app/models/ledger_retention_record.rb
393
+ class LedgerRetentionRecord < ActiveRecord::Base
394
+ self.abstract_class = true
395
+ connects_to database: { writing: :ledger_retention }
396
+ end
397
+
398
+ # config/initializers/reeve.rb, after require "reeve/audit"
399
+ Reeve.config.ledger_retention_connection = LedgerRetentionRecord
400
+ ```
401
+
402
+ The purge refuses a connection to another database, or one that logs in as the application's
403
+ own user. On SQLite, which has no roles, it can only warn. The task needs the ledger key, as
404
+ every chain task does.
405
+
406
+ *Running it.* Monthly, from one host, a dry run first (no `CONFIRM`), then the purge. The
407
+ archive path must be new each run: an existing archive is never overwritten.
408
+
409
+ ```bash
410
+ bin/rails reeve:purge_ledger OLDER_THAN=365d ARCHIVE=/mnt/archive/ledger-2026-10.jsonl # dry run
411
+ bin/rails reeve:purge_ledger OLDER_THAN=365d ARCHIVE=/mnt/archive/ledger-2026-10.jsonl CONFIRM=1
412
+ ```
413
+
414
+ ```text
415
+ reeve ledger purge: WOULD PURGE — sequences 1–6,000 through checkpoint #6,000 (2025-10-01 14:00 UTC): 6,000 chained entries and 5,000 pre-chain rows; archive to /mnt/archive/ledger-2026-10.jsonl; run again with CONFIRM=1
416
+ ```
417
+
418
+ In crontab, escape the date's `%`: `ledger-$(date +\%Y-\%m).jsonl`. `THROUGH=6000` names a
419
+ checkpoint's sequence instead of an age. Exactly one of `ARCHIVE=path` and `NO_ARCHIVE=1` is
420
+ required; with neither the task refuses, with both it raises. Exit `0` covers a dry run, a
421
+ purge, and nothing old enough (the first line says which: `WOULD PURGE`, `PURGED`, `NOTHING TO
422
+ PURGE`); exit `1` is a refusal, which removes nothing and names the fix:
423
+
424
+ | Refusal says | Fix |
425
+ |---|---|
426
+ | no anchor configured | set `config.ledger_anchor` and schedule checkpoints (above) |
427
+ | no retention connection / another database / the application's own database user | `config.ledger_retention_connection` and the role above |
428
+ | the ledger does not verify intact | read `bin/rails reeve:verify_ledger`; nothing is purged while anything is broken or unverifiable |
429
+ | no checkpoint the anchor holds is at that sequence | use one of the two it names, or `OLDER_THAN` |
430
+ | the archive … already exists, or cannot be written | a new path, in a directory the task can write |
431
+
432
+ Any other failure (a lost connection, a full disk) raises, exit `1` from Rake, like any task.
433
+
434
+ *If it stops midway.* The archive is written in full and the retention record committed
435
+ before any row is deleted. If the host dies during the deletes, verification reads INTACT,
436
+ exit `0`, with `; 120 entries awaiting purge` beside the purged range, which is not a break.
437
+ Run the purge again, the same command or any other: it first finishes the interrupted purge
438
+ (`PURGED — … (resumed; 120 rows removed this run)`), whatever boundary the new run names, and
439
+ skips the archive, which already exists. Run it once more to purge further.
440
+
441
+ *The archive.* Every purged row with every column, so the range can be checked without the
442
+ ledger's rows:
443
+
444
+ ```bash
445
+ bin/rails reeve:verify_archive ARCHIVE=/mnt/archive/ledger-2026-10.jsonl # reeve archive: INTACT — sequences 1–6,000 verified, …
446
+ ```
447
+
448
+ It runs inside the application (it needs the entries table's columns, from any database with
449
+ the schema, not the purged rows), with the keys that signed the archived rows configured as
450
+ `ledger_key` or `retired_ledger_keys`; the anchor is not needed. Each archive covers the range
451
+ its purge removed, named in its first line, so name files by the month they were made. Keep
452
+ archives where they cannot be changed, as you do the anchor. `NO_ARCHIVE=1` deletes without
453
+ one, for data you must not keep; the record and the output then say no archive was written.
454
+
455
+ *Keys.* Keep every key you ever signed with. A purge removes entries, not epoch records or
456
+ retention records, and verification still checks those under their own keys; archives need
457
+ the keys of the rows they hold.
458
+
459
+ After a purge, every summary line names what was purged before its coverage; with
460
+ `FORMAT=json`, `retention` and `awaiting_purge` hold the same:
461
+
462
+ ```text
463
+ reeve ledger: INTACT — sequences 6,001–10,000 verified (head at start: 10,000); 1–6,000 purged (retention record 0, 2026-10-25); 0 earlier entries not covered (5,000 when the chain started, 5,000 purged); anchored at #10,000 (checkpoint 2026-10-25 14:00 UTC), 0 entries after it
464
+ ```
465
+
466
+ Checkpoints older than the boundary read as purged. Deleting entries a retention record does
467
+ not cover, or editing or deleting a record, is BROKEN (`retention_altered` or `missing`), as
468
+ before.
469
+
259
470
  A dedicated key decouples the two: `config.ledger_key = Rails.application.credentials.reeve_ledger_key`.
260
471
  Keys must be at least 32 bytes and are only ever printed as 16-character fingerprints.
261
472
  Without a key, every guarded call fails closed; under `audit_failure_mode = :warn` it is
@@ -271,22 +482,32 @@ recorded unchained instead, logged each time, and reported by verification.
271
482
  - the chain record removed while chained rows remain;
272
483
  - an entry whose key id or key epoch was changed, or that names an epoch with no record, and
273
484
  an epoch record that was changed, or removed while entries name it (one forged under a key
274
- you do not hold reads as unverifiable, never intact).
485
+ you do not hold reads as unverifiable, never intact);
486
+ - with checkpoints: any entry the latest checkpoint saw removed, even with the head rewound;
487
+ the whole ledger deleted or replaced; an epoch record a checkpoint named, removed or changed;
488
+ a checkpoint edited, or forged without a key you hold.
275
489
 
276
490
  **Not detected** — stated plainly, because these are the limits of the guarantee:
277
491
 
278
492
  1. **A leaked key.** Anyone holding it can rewrite the chain from any point forward. By
279
- default that is anyone holding `secret_key_base`.
280
- 2. **The newest rows deleted by someone who also rewinds the chain record's head.** Only a
281
- checkpoint held outside the database could catch that; Reeve does not ship one.
282
- 3. **The whole ledger deleted, chain record included.** Reported as *no chain*, never as a
283
- pass — but indistinguishable from a ledger that was never chained.
493
+ default that is anyone holding `secret_key_base`. Not past a checkpoint made before the
494
+ leak, though: rewriting an entry it saw changes the digest it names.
495
+ 2. **The newest rows deleted by someone who also rewinds the chain record's head**, unless a
496
+ checkpoint saw them. Rows after the latest checkpoint stay exposed; the summary line counts
497
+ them.
498
+ 3. **The whole ledger deleted, chain record included**, without a checkpoint. Reported as
499
+ *no chain*, never as a pass, but indistinguishable from a ledger that was never chained.
500
+ And checkpoints are only as safe as their store: one an attacker can delete protects nothing.
284
501
  4. **Rows written before you upgraded.** They are counted as "not covered" and are exactly
285
502
  as trustworthy as they were before. Likewise, entries without a key epoch — chained by
286
503
  0.6.0, before or during the upgrade — are checked as 0.6.0 checked them: a rewritten key id
287
504
  on one of them reads as unverifiable, not as tampering.
288
505
  5. **A key you no longer hold.** Entries it signed cannot be checked, by anyone. Epochs make
289
506
  that visible and name the epoch; they do not undo it.
507
+ 6. **What a purge removed.** Without an archive, nobody can read it again; the retention record
508
+ proves only that it existed, verified, and was removed on purpose. A row edited inside the
509
+ range while the purge runs is not reported by the ledger, since it is being removed; the
510
+ archive holds what was read.
290
511
 
291
512
  And it prevents nothing: it detects a change after the fact. The `GRANT INSERT, SELECT` in
292
513
  the generated migration is what prevents one. The chain record needs `UPDATE` as well,
@@ -310,6 +531,12 @@ same door the built-in ones use, or verification reports them as unchained:
310
531
  Reeve::Audit::Chain.new(entry_class: Reeve::Audit::Entry).append!(row)
311
532
  ```
312
533
 
534
+ **Upgrading from 0.7.0:** `bin/rails generate reeve:upgrade && bin/rails db:migrate` adds
535
+ `reeve_audit_retentions`; grant the application's role `SELECT` on it. Nothing is purged until
536
+ you run the task, and anchoring is opt-in. Every summary line now ends with its anchoring
537
+ (`; not anchored` until you set one up), so anything that matches the whole text line should
538
+ read `FORMAT=json` instead, whose existing keys are unchanged.
539
+
313
540
  **Upgrading from 0.6.0:** `bin/rails generate reeve:upgrade && bin/rails db:migrate` before
314
541
  the new code serves traffic, and grant `INSERT, SELECT` on `reeve_audit_chain_epochs`. The next guarded call opens epoch 0;
315
542
  earlier entries are listed as "before epoch 0" and checked as before.
@@ -19,6 +19,15 @@
19
19
  # -- the key epochs: written once each, never updated
20
20
  # GRANT INSERT, SELECT ON reeve_audit_chain_epochs TO app_role;
21
21
  # GRANT USAGE ON SEQUENCE reeve_audit_chain_epochs_id_seq TO app_role;
22
+ # -- retention records: the application only reads them
23
+ # GRANT SELECT ON reeve_audit_retentions TO app_role;
24
+ #
25
+ # -- the retention role, used only by `bin/rails reeve:purge_ledger`
26
+ # GRANT SELECT ON reeve_audit_entries, reeve_audit_chains, reeve_audit_chain_epochs,
27
+ # reeve_audit_retentions TO retention_role;
28
+ # GRANT DELETE ON reeve_audit_entries TO retention_role;
29
+ # GRANT INSERT ON reeve_audit_retentions TO retention_role;
30
+ # GRANT USAGE ON SEQUENCE reeve_audit_retentions_id_seq TO retention_role;
22
31
  #
23
32
  # Every row is chained to the one before it with a keyed digest, and names the key epoch
24
33
  # that signed it (audit-entry contract 4),
@@ -30,10 +39,10 @@
30
39
  # reeve_audit_chains; that role needs UPDATE there, which is why the chain table is
31
40
  # granted separately from the ledger.
32
41
  #
33
- # Also explicitly out of scope for now, so you can plan around it:
34
- # * no retention, rotation or archival — this table grows until you prune it, and
35
- # pruning is your policy, run by a role that is allowed to delete. Deleting rows
36
- # breaks the chain by design.
42
+ # The table grows until you purge it. Deleting rows by hand breaks the chain by design and
43
+ # is reported as tampering; `bin/rails reeve:purge_ledger` removes the oldest entries up to
44
+ # a checkpoint, archives them, and leaves a signed record in reeve_audit_retentions so the
45
+ # rest still verifies. It runs on the retention role above, never the application's.
37
46
  #
38
47
  # `invocation_id` is a UUID stored as a string rather than a native `uuid` column,
39
48
  # because reeve supports every database ActiveRecord does and only PostgreSQL has the
@@ -133,5 +142,21 @@ class CreateReeveAuditEntries < ActiveRecord::Migration[7.0]
133
142
 
134
143
  add_index :reeve_audit_chain_epochs, :epoch, unique: true
135
144
  add_index :reeve_audit_chain_epochs, :key_id, unique: true
145
+
146
+ create_table :reeve_audit_retentions do |t|
147
+ t.integer :purge_number, null: false
148
+ t.bigint :through_sequence, null: false
149
+ t.string :through_digest, null: false, limit: 64
150
+ t.bigint :previous_through, null: false
151
+ t.bigint :removed_chained, null: false
152
+ t.bigint :removed_pre_chain, null: false
153
+ t.boolean :archived, null: false
154
+ t.string :archive_sha256, limit: 64
155
+ t.datetime :purged_at, null: false, precision: 6
156
+ t.string :key_id, null: false, limit: 16
157
+ t.string :retention_digest, null: false, limit: 64
158
+ end
159
+
160
+ add_index :reeve_audit_retentions, :purge_number, unique: true
136
161
  end
137
162
  end
@@ -102,3 +102,13 @@ end
102
102
 
103
103
  # The ledger itself is opt-in, so the core stays loadable without ActiveRecord.
104
104
  require "reeve/audit"
105
+
106
+ # Checkpoints of the chain, kept where the database's users cannot change them,
107
+ # catch what the chain alone cannot: the newest entries removed with the head
108
+ # rewound, or the whole ledger deleted. Run `bin/rails reeve:checkpoint_ledger`
109
+ # on a schedule; `reeve:verify_ledger` then checks against them. One anchor per
110
+ # environment, on storage outside the database host — a local file protects
111
+ # nothing:
112
+ #
113
+ # Reeve.config.ledger_anchor =
114
+ # Reeve::Audit::FileAnchor.new("/mnt/ledger-anchor/#{Rails.env}.checkpoints")
@@ -0,0 +1,37 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Ledger retention. Generated by `bin/rails generate reeve:upgrade`. The audit-entry contract
4
+ # stays 4: no column of reeve_audit_entries changes.
5
+ #
6
+ # Adds reeve_audit_retentions, where `bin/rails reeve:purge_ledger` records each purge: the
7
+ # checkpoint it purged up to, how many rows it removed, and the archive it wrote, signed with
8
+ # the ledger key. Verification reads it to tell a purge from a deletion. Nothing is purged
9
+ # until you run the task.
10
+ #
11
+ # The application role only reads the new table. The retention role, which only the purge
12
+ # uses, needs DELETE on the ledger and INSERT here:
13
+ #
14
+ # GRANT SELECT ON reeve_audit_retentions TO app_role;
15
+ # GRANT SELECT ON reeve_audit_entries, reeve_audit_chains, reeve_audit_chain_epochs,
16
+ # reeve_audit_retentions TO retention_role;
17
+ # GRANT DELETE ON reeve_audit_entries TO retention_role;
18
+ # GRANT INSERT ON reeve_audit_retentions TO retention_role;
19
+ class AddLedgerRetentionToReeveAuditEntries < ActiveRecord::Migration[7.0]
20
+ def change
21
+ create_table :reeve_audit_retentions do |t|
22
+ t.integer :purge_number, null: false
23
+ t.bigint :through_sequence, null: false
24
+ t.string :through_digest, null: false, limit: 64
25
+ t.bigint :previous_through, null: false
26
+ t.bigint :removed_chained, null: false
27
+ t.bigint :removed_pre_chain, null: false
28
+ t.boolean :archived, null: false
29
+ t.string :archive_sha256, limit: 64
30
+ t.datetime :purged_at, null: false, precision: 6
31
+ t.string :key_id, null: false, limit: 16
32
+ t.string :retention_digest, null: false, limit: 64
33
+ end
34
+
35
+ add_index :reeve_audit_retentions, :purge_number, unique: true
36
+ end
37
+ end
@@ -58,6 +58,15 @@ module Reeve
58
58
  creates: %w[reeve_audit_chain_epochs],
59
59
  template: "add_key_epochs_to_reeve_audit_entries.rb.tt",
60
60
  destination: "add_key_epochs_to_reeve_audit_entries.rb"
61
+ },
62
+ # Retention (specs/005) adds a table and no column, so the contract stays 4.
63
+ {
64
+ contract: 4,
65
+ note: :retention,
66
+ adds: [],
67
+ creates: %w[reeve_audit_retentions],
68
+ template: "add_ledger_retention_to_reeve_audit_entries.rb.tt",
69
+ destination: "add_ledger_retention_to_reeve_audit_entries.rb"
61
70
  }
62
71
  ].freeze
63
72
 
@@ -124,7 +133,7 @@ module Reeve
124
133
  Existing rows are stamped with the contract they were actually written under,
125
134
  not with the current one.
126
135
  NEXT
127
- message + UpgradeNotes.for(@emitted.map { |step| step[:contract] })
136
+ message + UpgradeNotes.for(@emitted)
128
137
  end
129
138
 
130
139
  # Asked of the table rather than of a version the host recorded: the table is the
@@ -38,13 +38,24 @@ module Reeve
38
38
  The application role needs INSERT and SELECT on reeve_audit_chain_epochs.
39
39
  EPOCHS
40
40
 
41
- BY_CONTRACT = { 3 => CHAIN, 4 => EPOCHS }.freeze
41
+ # Retention changes nothing until the host runs a purge, but the purge needs a role the
42
+ # application does not have, and the host should hear that before it needs one.
43
+ RETENTION = <<~RETENTION
44
+
45
+ The ledger can now be purged (reeve_audit_retentions): bin/rails reeve:purge_ledger
46
+ removes the oldest entries up to a checkpoint and records the purge, so the rest still
47
+ verifies. Nothing is purged until you run it. The application role needs SELECT on
48
+ the new table; the purge runs on a retention role of its own (see the migration).
49
+ RETENTION
50
+
51
+ # Keyed by the step's note, or by its contract when it has none.
52
+ BY_STEP = { 3 => CHAIN, 4 => EPOCHS, retention: RETENTION }.freeze
42
53
 
43
54
  module_function
44
55
 
45
- # The notes for the contracts just written, oldest first.
46
- def for(contracts)
47
- contracts.filter_map { |contract| BY_CONTRACT[contract] }.join
56
+ # The notes for the steps just written, oldest first.
57
+ def for(steps)
58
+ steps.filter_map { |step| BY_STEP[step.fetch(:note, step[:contract])] }.join
48
59
  end
49
60
  end
50
61
  end