standard_audit 0.12.1 → 0.13.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a6da3bdedaecde5d44a57afe47372bf419b9290ace08a2312d1c522df5a67561
-  data.tar.gz: 283e8c2839ddce46fa2ec6468423c75380256f51f05a0a52844763d5454b7da3
+  metadata.gz: 7ca3b859a3eb6993caa5c57a9a43621f67a44871c5ee3f05b2ea5d5ca79860b4
+  data.tar.gz: 7a85141df78963fa16c7b223da7a7d48270c368538815f26d8e097a09487c0ff
 SHA512:
-  metadata.gz: 96cd2995bbaaab665bc485770604f37b0b4188cf052a30639b804882a70cc276fad7e10b7ee21c65fa0294018687e06cea921ae7e387f4666a3f7298d29bbd09
-  data.tar.gz: 810d2becd35d3e0a5325786b86415c48318b4c23d7c8df46742916053cff6eaf5f97e3cba1e94b8ab479437751ba94177c28b05bc4c12837208a975698788aa0
+  metadata.gz: b74d1bbbd38c3580fb36f4db24f40f7fd01c9f6f807035e3aee8e0a3979692f0c4dc8175cdb53799291d578fdf0e8ca80add04e6df1b09b819425717088abea1
+  data.tar.gz: 32dd1f53798d592fe517966dd4677abf42ce5ab7dd650176f56dfcd873025a43a3a81a048e276da37fdf90954cc68d3746fb9355e9c1e40559d123c87ab605af

data/CHANGELOG.md

@@ -7,6 +7,213 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.13.1] - 2026-09-25
+
+Fixes the checksum so it survives PostgreSQL `jsonb`
+([fundbright/delivery-ops#689](https://github.com/fundbright/delivery-ops/issues/689)).
+Through 0.13.0, the row digest hashed `metadata.to_json` in the order the
+Ruby hash was built. `jsonb` stores keys in its own order (shortest first,
+then bytewise). So any row whose metadata had more than one key failed
+`verify_chain` with `digest_mismatch`, unless its keys happened to be
+written in that order. In fundbright production that is 17,448 of 25,910
+rows (67%). It, not concurrency, was the main cause behind
+fundbright/delivery-ops#433; the 0.8.0 recovery search rescues 24 of those
+rows. Every host on `jsonb` is affected: fundbright, sidekick, jumpdrive,
+luminality and nutripod. The gem's suite runs on SQLite, which keeps JSON as
+text in insertion order, so it never saw the bug.
+
+### ⚠️ Deploy before 2026-10-01T00:00:00Z
+
+The new checksum switches on **by the clock, not by deploy**. Rows whose
+`created_at` is at or after `StandardAudit::CANONICAL_CHECKSUM_CUTOVER`
+(**2026-10-01T00:00:00Z**) are signed with the canonical checksum and
+verified strictly with it. **Every process that writes audit rows must run
+0.13.1 before then.** A web worker or job runner still on an older gem after
+the cutover writes legacy-hashed rows with post-cutover timestamps. Those
+rows fail verification as `:digest_mismatch`, and the only honest fix
+afterwards is to explain them.
+
+If your rollout will slip, set `config.canonical_checksum_since` to a later
+time **before 2026-10-01**, in a `configure(baseline: true)` block. Never move
+it once the time has passed: verification recomputes the same decision from
+each row's stored `created_at`, so moving it re-judges rows under the other
+algorithm.
+
+### Upgrade steps
+
+1. Bump to 0.13.1 and deploy **before 2026-10-01T00:00:00Z** (or move
+   `config.canonical_checksum_since`, as described above). **No migration or
+   new column is needed.**
+2. Regenerate Sorbet RBIs where the app uses Tapioca. This release adds
+   `StandardAudit::Checksum` and new `verify_chain` keywords.
+3. After the cutover, run `verify_chain` (or `rake standard_audit:verify`)
+   and read the new counts; see "What `verify_chain` reports" below. Rows
+   created after the cutover should all verify. Record the
+   `legacy_unverifiable` count and alert if it ever grows.
+4. **Decide the policy for legacy rows that can't be reconstructed.** This
+   is a human decision (#689 options a to c). The gem reports these rows; it
+   does not make the call.
+
+**Requires Rails 8.1** (`activerecord`, `activejob`, `activesupport` `>= 8.1`,
+was `>= 8.0`). Every consumer app runs 8.1; 8.0 was never exercised in CI.
+
+### Changed
+
+- **Canonical checksum for rows created at or after the cutover.** It is
+  the SHA-256 of canonical JSON
+  `{"fields": {…}, "previous_checksum": …, "v": 2}`:
+  - JSON values (metadata, or any Hash/Array field) are round-tripped exactly
+    as the column stores them.
+  - Object keys are sorted bytewise at every depth.
+  - Integral floats hash as integers, and times as UTC ISO 8601 with
+    microseconds.
+  - Strings are escaped at the byte level.
+  - nil is distinct from `""`.
+
+  Every input to the digest was audited. `CHECKSUM_FIELDS` is unchanged, and
+  `metadata` is the only JSON field in the shipped schema. The parent is
+  covered. The canonical form also closes a field-shifting ambiguity in the
+  legacy form, where `|` inside a value could move content between adjacent
+  fields with the same digest.
+- **Rows created before the cutover keep the legacy digest byte for byte**
+  (`StandardAudit::Checksum.legacy_digest`). They are written and verified
+  exactly as before.
+- The algorithm is chosen from `created_at` on every write path: `create`,
+  the batched `insert_all!` path, and `backfill_checksums!`. Verification and
+  `relink_checksums!` make the same choice. `created_at` is fixed before the
+  checksum is computed, so the writer and the verifier read the same stored
+  value. `created_at` is not itself hashed; see "Alert if it grows" below.
+- `compute_checksum_value` (instance and class) picks the algorithm from
+  `created_at`. `version:` forces one (`Checksum::LEGACY` / `CANONICAL`).
+- **`verify_chain` returns `reordered:`, `legacy_unverifiable:` and
+  `unverifiable:`**, and takes `key_order_search_limit:` (default 720) and
+  `fail_on_legacy_unverifiable:` (default false). Existing keys and reasons
+  are unchanged.
+- `rake standard_audit:verify` prints the cutover, the new counts and a tally
+  per reason. It takes `FAIL_ON_LEGACY_UNVERIFIABLE=1` and
+  `KEY_ORDER_SEARCH_LIMIT=<n>`.
+
+### What `verify_chain` reports
+
+**Rows created at or after the cutover** are checked with the canonical
+digest only, against the declared parent, else the preceding row, else the
+recovery search. A mismatch is `:digest_mismatch`. These rows are never
+classified as legacy. Chain linkage holds across the cutover: the first
+canonical row's parent is the last legacy row's checksum.
+
+**Rows created before the cutover** are first checked with the legacy digest
+exactly as in 0.13.0: stored order, against the declared parent, else the
+preceding row, else the recovery search. A row that still doesn't reproduce
+goes through these steps:
+
+1. **Key-order reconstruction.** The insertion order can't be read back,
+   because `jsonb`'s order depends only on the key set. It can be searched,
+   though: an ordering of the stored keys (at every depth) that reproduces the
+   digest the row has held since it was written is a witness, in the same
+   sense as the parent search. A row whose values were edited reproduces no
+   ordering.
+   - Such rows count in `reordered` and are valid.
+   - The search is bounded by `key_order_search_limit` orderings per row.
+   - Orders learned from earlier rows with the same keys are tried first, so
+     most rows cost one hash.
+   - Parents tried: the declared parent, or else the preceding row and "no
+     parent".
+2. **`:digest_mismatch`** if the search covered every ordering against the
+   parent the row declares, or if the row has no JSON object with more than
+   one key (key order can't be why it fails).
+3. **`:missing_parent`** if the declared parent is gone.
+4. Otherwise **`:legacy_key_order_unverifiable`**: the row has a multi-key
+   object and every other check passed. It is listed in `unverifiable` and
+   counted in `legacy_unverifiable`.
+
+**`valid` semantics.** `valid` is still `failures.empty?`. A
+`:legacy_key_order_unverifiable` row **does not make the chain invalid on its
+own**, because it means "cannot be proven either way": an edited legacy row
+looks exactly like one whose key order was lost. It is never silent. It is
+reported separately with a count, and `fail_on_legacy_unverifiable: true`
+turns these rows into failures.
+
+**Alert if it grows.** After the cutover, no new legacy row can be written,
+so under honest operation `legacy_unverifiable` never grows. Growth means a
+pre-cutover row was edited, or a row's `created_at` was moved back across
+the cutover.
+
+How many fundbright rows reconstruction rescues depends on their key counts
+and on whether they carry `previous_checksum`. That hasn't been measured on
+production data yet.
+
+### Added
+
+- `StandardAudit::CANONICAL_CHECKSUM_CUTOVER` (frozen,
+  `Time.utc(2026, 10, 1)`) and `config.canonical_checksum_since`, which
+  defaults to it.
+- `StandardAudit::Checksum` (`algorithm_for`, `digest`, `legacy_digest`,
+  `canonical_digest`, `canonical_json`) and
+  `StandardAudit::Checksum::KeyOrderSearch`.
+- **A PostgreSQL CI leg** (`test (postgres)`: `postgres:16-alpine`, `jsonb`
+  metadata) runs the full suite. The SQLite matrix stays. The dummy app uses
+  `DATABASE_URL` when it is set. New specs cover:
+  - both sides of the cutover, a row just before and just after it, the
+    config override, and the batch and backfill paths;
+  - linkage across the cutover;
+  - the `jsonb` regression. On Postgres the specs also assert that the stored
+    order really changed and that the legacy digest fails on it.
+
+### Not included: re-sealing legacy rows
+
+No tool re-signs legacy rows with the canonical digest. This is deliberate,
+and a follow-up if you want one. Re-sealing replaces an attestation made at
+write time with one made today. It needs an operator to attest to the rows
+first, and it has to keep the chain intact, because each row's successor
+hashed the row's *old* checksum as its parent. A safe version would need to:
+
+1. keep the original checksum (for example in a `legacy_checksum` column) so
+   the successor link can still be checked;
+2. record who re-sealed which rows and when (a `resealed_at` stamp plus an
+   audit event);
+3. re-seal only rows classified `:legacy_key_order_unverifiable`, never a
+   `:digest_mismatch`;
+4. have `verify_chain` report re-sealed rows separately, so a re-seal never
+   reads as original evidence.
+
+That is a schema change plus a policy decision, so it is left for a separate
+release. `backfill_checksums!` is still only for rows that never had a
+checksum; don't use it for this.
+
+## [0.13.0] - 2026-09-24
+
+The Phase 4 release. It removes what 0.12 deprecated, the empty engine
+routing, and the gaps the five app adoptions of 0.12 ran into.
+
+### Removed (breaking)
+
+- **`standard_audit:add_checksums` generator** (deprecated in 0.12.0). It was the 0.2 → 0.3 upgrade path. Every install since 0.3 creates `checksum`; hosts older than 0.8 run `standard_audit:add_previous_checksum`.
+- **`isolate_namespace StandardAudit` and the empty `config/routes.rb`.** The engine has no routes, controllers or views, and no host mounts it. `AuditLog` sets its own `table_name`, so table naming is unchanged. Side effects: `StandardAudit::Engine.routes` no longer holds an (empty) isolated route set, and `StandardAudit.table_name_prefix` / `railtie_namespace` are no longer set by isolation. The gem uses neither.
+
+### Added
+
+- **`config.error_reporter = ->(error, context) { … }`**, the destination for every error the gem swallows: `record(raise: false)`, subscriber writes, raising `before_checksum` hooks, and `audit!` write failures under the default policy. nil (the default) keeps `Rails.error.report(error, handled: true, context:)`, so nothing changes unless you set it. Apps that don't forward `Rails.error` to their tracker (jumpdrive-web, nutripod-web) can report straight to Sentry instead of wrapping calls in their own rescue. A raising reporter is logged and ignored. `audit_write_error_handler` still takes precedence for `audit!`.
+- **`entry[:via]` in `before_write`**: `:direct` (`record` without a block, `record_audit`, `audit!`), `:notification` (the ActiveSupport::Notifications subscriber, including `record` with a block), or `:rails_event`. A hook can now tell a direct write from a subscriber write. Not persisted. `StandardAudit::VIA` lists the values.
+- **`hooks:` option on the `"a standard_audit baseline"` shared example**: an Integer (exact `before_checksum` hook count) or an Array of Symbol hook names. The mutation example clears the hooks before the reset, so it fails when a hook lives outside the baseline block.
+
+### Fixed
+
+- **Upgrade generators number the migration after the host's newest migration.** `add_anonymized_at`, `add_previous_checksum` and `install` stamped `Time.now`, which sorts before future-dated host migrations. They now use the later of now and one second after the newest migration in the target directory (a real timestamp, unlike ActiveRecord's `+1`, which can produce `…235960`).
+- **README `before_write` example.** It showed a PII guard in `before_write` next to a masking `metadata_builder`. `before_write` runs after the builder, so the guard only ever saw masked values. The README now documents the order (resolvers → `metadata_builder` → `before_write` → dereference → redact → persist) and shows guard-then-mask inside `before_write`.
+
+### Upgrade notes (0.12.x → 0.13.0)
+
+Grepped `origin/main` of sidekick-web, jumpdrive-web (control-plane), fundbright-web, luminality-web and nutripod-web on 2026-09-24.
+
+**Required host changes: none.** No app runs `add_checksums`, mounts `StandardAudit::Engine`, or uses its route helpers. Regenerate Sorbet RBIs (`bin/tapioca gem standard_audit`, `bin/tapioca dsl`) where the app uses Tapioca.
+
+**Optional cleanups this release enables:**
+- sidekick-web `spec/initializers/standard_audit_baseline_spec.rb:49-53` ("still carries the two classification hooks after a reset"): replace with `hooks: 2` on the `it_behaves_like` call.
+- jumpdrive-web `control-plane/app/services/mcp/server.rb:98-104` (`audit_tool` rescue → `ErrorReporting.notify`): replace with `StandardAudit.record(..., raise: false)` plus a `config.error_reporter` that calls `ErrorReporting.notify`. Or keep it, since it also adds `component:` context.
+- nutripod-web `app/controllers/concerns/audit_auth_failure.rb:92-96`: the rescue reports through `Rails.error` itself, so it can become `record(..., raise: false)` with `config.audit_error_context_key = :audit_event`, like the other apps. Add a `config.error_reporter` if nutripod-web does not forward `Rails.error` to Sentry.
+- fundbright-web `AuditWritePolicy` (`before_write`) can use `entry[:via]` if its PII guard should skip gem-published subscriber payloads.
+- Any `before_write` that iterates every `entry` key now also sees `:via`.
+
 ## [0.12.1] - 2026-09-24
 
 ### Upgrade steps

data/README.md

@@ -102,8 +102,8 @@ When `actor` is omitted, it falls back to the configured `current_actor_resolver
 
 Where a missing audit row must never break the request — logging an
 authentication failure, say — pass `raise: false`. A failed write is logged,
-reported to `Rails.error` as handled (context
-`{ <audit_error_context_key> => event_type, source: "StandardAudit.record" }`),
+reported through `config.error_reporter` (default: `Rails.error`, as handled;
+context `{ <audit_error_context_key> => event_type, source: "StandardAudit.record" }`),
 and `record` returns nil:
 
 ```ruby
@@ -116,11 +116,30 @@ StandardAudit.record("auth.token_invalid",
 The default is `raise: true` (unchanged). In block form the option only
 governs the audit write; errors from your block always propagate.
 
-`raise: false` does not talk to Sentry (or any other tracker) itself. It calls
-`Rails.error.report`, so a swallowed failure reaches your error tracker only if
-something subscribes to `Rails.error`. `sentry-rails` registers that subscriber
-for you. With a hand-rolled Sentry setup, register one
-(`Rails.error.subscribe(...)`), or these failures show up only in the log.
+#### Where swallowed failures go: `config.error_reporter`
+
+By default the gem reports every error it swallows with
+`Rails.error.report(error, handled: true, context:)`. That covers a failed
+`record(raise: false)`, a failed subscriber write, a raising `before_checksum`
+hook, and a failed `audit!` write under the default policy. It reaches your
+error tracker only if something subscribes to `Rails.error`. `sentry-rails`
+registers that subscriber for you. If your app never forwards `Rails.error`
+(a hand-rolled Sentry setup), point the gem straight at the tracker (0.13.0+):
+
+```ruby
+config.error_reporter = ->(error, context) { Sentry.capture_exception(error, extra: context) }
+```
+
+It receives the error and the context Hash (keyed by
+`audit_error_context_key`) and **replaces** the `Rails.error` call. If you want
+both, call `Rails.error.report` from it too. A reporter that raises is logged
+and ignored, so it can never turn a swallowed audit failure into a raised one.
+`audit_write_error_handler`, when set, still takes precedence for `audit!`
+write failures.
+
+**Replace your host code with** `config.error_reporter`. It supersedes any
+rescue-and-report wrapper kept around `record(raise: false)` or `audit!` only
+because the app does not forward `Rails.error` to its tracker.
 
 **Replace your host code with** `raise: false`. It supersedes the
 `AuditAuthFailure#record_auth_failure` rescue-and-report wrapper
@@ -172,18 +191,40 @@ never ran on direct `record` calls.)
 ```ruby
 config.before_write = ->(entry) {
   # entry: { event_type:, actor:, target:, scope:, metadata:,
-  #          request_id:, ip_address:, user_agent:, session_id: }
-  AuditMetadataPii.verify!(entry[:metadata]) if StandardAudit::Operation::Audit.verify?
-  if (surface = Current.audit_surface).present?
-    entry[:metadata] = entry[:metadata].merge(surface: surface)
-  end
+  #          request_id:, ip_address:, user_agent:, session_id:, via: }
+  metadata = entry[:metadata]
+  AuditMetadataPii.verify!(metadata) if StandardAudit::Operation::Audit.verify? && entry[:via] == :direct
+  metadata = AuditMetadataPii.mask(metadata)
+  metadata = metadata.merge(surface: Current.audit_surface) if Current.audit_surface.present?
+  entry[:metadata] = metadata
 }
 ```
 
-It runs after the `Current` resolvers and `metadata_builder`, and **before**
-dereferencing and `sensitive_keys` redaction, so anything it injects is still
-filtered. Mutate `entry` in place; the return value is ignored. Raising aborts
-the write: direct callers see the error, the subscribers rescue and report it.
+**Order, per write:** `Current` resolvers → `metadata_builder` → `before_write`
+→ record dereferencing → `sensitive_keys` redaction → persist (or buffer, or
+enqueue). So:
+
+- `before_write` **sees the builder's output, not the raw metadata.** If your
+  `metadata_builder` masks or rewrites values, a guard in `before_write` checks
+  the masked values and can never fire. Keep the builder to idempotent
+  injection (like `engine_scope`), and do guard-then-mask in `before_write`, in
+  that order, as above. Before 0.13 this README showed a guard in
+  `before_write` next to a masking builder, which does not work.
+- Anything `before_write` injects is still dereferenced and redacted.
+
+Mutate `entry` in place; the return value is ignored. Raising aborts the write:
+direct callers see the error, the subscribers rescue and report it.
+
+**`entry[:via]`** (0.13.0+) names the entry point, so a hook can treat direct
+writes differently from subscriber writes. For example, it can apply a guard
+only to rows your own code writes and not to payloads a gem publishes. It is
+not persisted.
+
+| `entry[:via]` | Written by |
+|---|---|
+| `:direct` | `StandardAudit.record` (no block), `Auditable#record_audit`, `Operation#audit!` |
+| `:notification` | the ActiveSupport::Notifications subscriber (`subscribe_to` patterns), including `StandardAudit.record` **with a block** |
+| `:rails_event` | the `Rails.event` subscriber (Rails 8.1+) |
 
 **Hooks run once per row, batched writes included.** `before_write` and
 `before_checksum` run for every row inside `StandardAudit.batch` too (at flush
@@ -424,7 +465,8 @@ RSpec.describe "StandardAudit configuration baseline" do
     catalogue: -> { AuditCatalogue::ACTIONS },
     sensitive_keys: %i[source_payload],
     sensitive_key_patterns: [/secret/i],
-    present: %i[metadata_builder before_write current_scope_resolver]
+    present: %i[metadata_builder before_write current_scope_resolver],
+    hooks: 2 # before_checksum hooks: a count, or %i[backfill_scope] by name
 end
 ```
 
@@ -433,6 +475,13 @@ restored after a mutation plus `reset_configuration!`. Behaviour held in
 lambdas can only be checked for presence; keep an app-specific example for
 anything whose *result* matters.
 
+`hooks:` (0.13.0+) covers `before_checksum` hooks, which a reset drops unless
+the baseline re-adds them. Pass an Integer to require exactly that many, or an
+Array of Symbol hook names (`config.before_checksum :name`) to require each one.
+The mutation example clears the hooks before the reset, so a hook registered
+outside the baseline block fails it. This replaces hand-written "still carries
+the N hooks after a reset" examples.
+
 **Replace your host code with** the shared example. It supersedes the bulk of
 each app's `spec/initializers/standard_audit_baseline_spec.rb` (or
 `spec/config/…`). The `Current.account` / `Current.session&.id` resolver
@@ -527,9 +576,14 @@ StandardAudit.configure(baseline: true) do |config|
   config.current_scope_resolver = -> { Current.organisation }
 
   # -- before_write --
-  # Runs on every write path, before redaction. See "One write path".
+  # Runs on every write path, after metadata_builder and before redaction.
+  # entry[:via] is :direct, :notification or :rails_event. See "One write path".
   config.before_write = ->(entry) { entry[:metadata] = entry[:metadata].merge("surface" => Current.surface) }
 
+  # -- Error reporting --
+  # Where swallowed audit failures go. nil (default) = Rails.error.report.
+  # config.error_reporter = ->(error, context) { Sentry.capture_exception(error, extra: context) }
+
   # -- Async Processing --
   # Offload audit log creation to ActiveJob.
   config.async = false
@@ -546,6 +600,13 @@ StandardAudit.configure(baseline: true) do |config|
   # Defaults from STANDARD_AUDIT_RETENTION_DAYS (see Retention below); set here
   # to override per app. Leave unset for infinite retention.
   config.retention_days = 90
+
+  # -- Checksum cutover --
+  # Rows created at/after this time get the canonical (key-order-independent)
+  # checksum. Default StandardAudit::CANONICAL_CHECKSUM_CUTOVER (2026-10-01Z).
+  # Only move it if your rollout slips, only to a future time, and never after
+  # it has passed. See "Checksum algorithm versions".
+  # config.canonical_checksum_since = Time.utc(2026, 10, 15)
 end
 ```
 
@@ -736,7 +797,8 @@ reproduce its own digest. Since 0.12.0 its stored `checksum` is left untouched
 
 ```ruby
 StandardAudit::AuditLog.verify_chain
-# => { valid: true, verified: 5576, recovered: 0, redacted: 1, failures: [] }
+# => { valid: true, verified: 5576, recovered: 0, reordered: 0, redacted: 1,
+#      legacy_unverifiable: 0, unverifiable: [], failures: [] }
 ```
 
 A redacted row's declared parent is still checked, so deleting the row before
@@ -832,7 +894,8 @@ own digest, so editing it invalidates the row.
 
 ```ruby
 result = StandardAudit::AuditLog.verify_chain
-# => { valid: true, verified: 5577, recovered: 0, redacted: 0, failures: [] }
+# => { valid: true, verified: 5577, recovered: 0, reordered: 0, redacted: 0,
+#      legacy_unverifiable: 0, unverifiable: [], failures: [] }
 ```
 
 `redacted` counts rows anonymized by `anonymize_actor!` — see "Anonymization
@@ -848,8 +911,13 @@ and the checksum chain".
   `created_at`, so rows whose two timestamps disagree can leave a hole rather
   than a pruned start, and a hole is reported — truthfully, since rows really
   are missing.
+  Pre-cutover rows whose metadata key order is lost are listed separately in
+  `unverifiable` (`reason: :legacy_key_order_unverifiable`) — see "Checksum
+  algorithm versions".
 - `recovered` counts rows with no `previous_checksum` whose parent had to be
   found by searching back through recent digests — see below.
+- `reordered`, `legacy_unverifiable` and `unverifiable` concern pre-cutover
+  rows only — see "Checksum algorithm versions".
 - `verify_chain(scope: org)` skips the missing-parent check, because the log is
   global and a scoped row's parent usually belongs to another scope.
 
@@ -860,6 +928,105 @@ previous strict-line reading did not actually detect insertions either — it
 reported every concurrent append as tampering, which on one production log meant
 67% of rows red and any real signal lost in the noise.
 
+### Checksum algorithm versions
+
+Up to 0.13.0 the digest hashed `metadata.to_json` in the order the Ruby hash
+was built. PostgreSQL `jsonb` (and MySQL `JSON`) store object keys in their own
+order — shortest first, then bytewise — so a row whose metadata had more than
+one key could only be verified if its keys happened to be written in that
+order. On one production log that was 67% of rows
+([fundbright/delivery-ops#689](https://github.com/fundbright/delivery-ops/issues/689)).
+SQLite keeps JSON as text in insertion order, so the gem's own suite never saw
+it; CI now runs the suite on PostgreSQL too.
+
+There are two algorithms, and **which one a row uses is decided by its
+`created_at`** — nothing extra is stored and no migration is needed:
+
+| Rows created | Algorithm | Digest |
+|--------------|-----------|--------|
+| before `config.canonical_checksum_since` | legacy | `SHA256("<parent>\|field=value\|…")`, Hash values via `to_json` in Ruby key order. Kept byte-for-byte so these rows verify as they were signed. |
+| at or after it | canonical | `SHA256` of canonical JSON `{"fields": {…}, "previous_checksum": …, "v": 2}`: JSON values round-tripped exactly as the column stores them, object keys sorted bytewise at every depth, integral floats as integers, times as UTC ISO 8601 (µs), nil distinct from `""`, no separator ambiguity. |
+
+`config.canonical_checksum_since` defaults to
+`StandardAudit::CANONICAL_CHECKSUM_CUTOVER`, **2026-10-01T00:00:00Z**. The
+switch happens by the clock, not by deploy:
+
+> **Deploy 0.13.1 or later before the cutover.** A process still running an
+> older gem after it writes legacy-hashed rows with post-cutover
+> `created_at`s, and those fail strict canonical verification as
+> `:digest_mismatch`. If your rollout will slip, set
+> `config.canonical_checksum_since` to a later time **before** the default
+> passes — and never move it once it has passed, because verification
+> recomputes the same decision from each row's stored `created_at`.
+
+```ruby
+# config/initializers/standard_audit.rb — only if the rollout slips
+StandardAudit.configure(baseline: true) do |config|
+  config.canonical_checksum_since = Time.utc(2026, 10, 15)
+end
+```
+
+Rows written between upgrading and the cutover are still legacy-hashed, so a
+multi-key row written then can still end up unverifiable (below). Deploying
+early does not change that by itself. To switch sooner, set
+`canonical_checksum_since` to a time that is still in the future and after
+every process runs 0.13.1. Never set it to a time that has already passed,
+because rows written since then would be re-judged under the other algorithm.
+
+**How `verify_chain` judges each row:**
+
+- **At or after the cutover:** the canonical digest only — declared parent,
+  else the preceding row, else the recovery search. A mismatch is
+  `:digest_mismatch`; these rows are never classified as legacy.
+- **Before the cutover:** the legacy digest, exactly as before (declared
+  parent, preceding row, recovery search). A row that still does not
+  reproduce is then:
+  1. **Reconstructed.** The insertion order the legacy digest hashed cannot be
+     read back — jsonb's order depends only on the key set — but it can be
+     *searched*. If some ordering of the stored keys (at every depth), hashed
+     with the row's parent, reproduces the digest the row has held since it
+     was written, that is a witness in the same sense as the parent recovery
+     search: a row whose values were edited reproduces no ordering. Such rows
+     count in `reordered` and are valid. At most `key_order_search_limit:`
+     orderings per row (default 720, six keys in one object); orders learned
+     from earlier rows with the same keys are tried first, so most rows cost
+     one hash. Parents tried: the declared one, or else the preceding row and
+     "no parent" (not the 256-row window).
+  2. **`:digest_mismatch`** if that search was exhaustive against the parent
+     the row *declares* (no key order explains it), or if the row has no JSON
+     object with more than one key (key order cannot be why it fails).
+  3. **`:missing_parent`** if its declared parent is gone.
+  4. Otherwise **`:legacy_key_order_unverifiable`**.
+
+**What `valid` means.** `valid` is `failures.empty?`. A
+`:legacy_key_order_unverifiable` row does **not** make the chain invalid on its
+own: it means "cannot be proven either way" — an edited legacy row looks
+exactly like one whose key order was lost — and the policy for such rows is
+the host's. It is never silent, though: `legacy_unverifiable` counts them and
+`unverifiable` lists them (same shape as a failure).
+
+```ruby
+StandardAudit::AuditLog.verify_chain
+# => { valid: true, verified: 25910, recovered: 24, reordered: <n>, redacted: 0,
+#      legacy_unverifiable: <m>, unverifiable: [...], failures: [] }
+
+StandardAudit::AuditLog.verify_chain(fail_on_legacy_unverifiable: true)
+# => the same rows reported in failures, valid: false while any exist
+```
+
+After the cutover no new legacy row can be written, so under honest operation
+`legacy_unverifiable` never grows. **Alert if it does:** it means a pre-cutover
+row was edited, or a row's `created_at` was moved back across the cutover
+(`created_at` is not itself hashed).
+
+**Cost.** A canonical row costs one JSON round trip and one hash. A legacy row
+that needs the search costs up to `limit × 2` hashes the first time a key set
+is seen and usually one after that. Raise the limit (e.g. 5040 for seven keys)
+if your events carry wider metadata.
+
+**Re-sealing legacy rows is not provided.** See the 0.13.1 CHANGELOG for why
+and for what a safe version would need.
+
 ### Rows written before 0.8.0
 
 They have no `previous_checksum`. Verification falls back to the preceding row
@@ -896,13 +1063,20 @@ It is for rows that never had a checksum at all (pre-feature data).
 | `standard_audit:install` | `audit_logs` migration + initializer (new installs) |
 | `standard_audit:add_previous_checksum` | Adds `previous_checksum` (upgrading from < 0.8) |
 | `standard_audit:add_anonymized_at` | Adds `anonymized_at` (upgrading from < 0.12) |
-| `standard_audit:add_checksums` | **Deprecated** (0.12.0; to be removed). The 0.2 → 0.3 upgrade path; warns when run |
+
+The upgrade generators number their migration one second after the newest
+migration already in `db/migrate` when that is later than now (0.13.0+), so the
+new migration sorts after future-dated host migrations instead of before them.
+Before 0.13.0 they stamped the current time. (`standard_audit:add_checksums`,
+the 0.2 → 0.3 upgrade path, was removed in 0.13.0.)
 
 ## Rake Tasks
 
 ```bash
 # Verify chain integrity (exits non-zero on failures)
 rake standard_audit:verify
+# ...treating unreconstructable pre-cutover rows as failures too
+FAIL_ON_LEGACY_UNVERIFIABLE=1 rake standard_audit:verify
 
 # Record the parent digest each existing row was signed against
 rake standard_audit:relink_checksums
@@ -950,6 +1124,10 @@ For PostgreSQL, edit the generated migration to use `jsonb` instead of `json`:
 t.jsonb :metadata, default: {}
 ```
 
+`jsonb` and MySQL `JSON` reorder object keys. Rows created since the
+canonical-checksum cutover do not depend on key order; earlier rows may — see
+"Checksum algorithm versions".
+
 ## Best Practices
 
 **What to audit**: Authentication events, data mutations, permission changes, financial transactions, admin actions, data exports, and API access from external services.