standard_ledger 0.2.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +170 -1
- data/README.md +102 -4
- data/lib/standard_ledger/config.rb +3 -2
- data/lib/standard_ledger/engine.rb +7 -0
- data/lib/standard_ledger/entry.rb +99 -10
- data/lib/standard_ledger/errors.rb +10 -0
- data/lib/standard_ledger/jobs/projection_job.rb +94 -0
- data/lib/standard_ledger/modes/async.rb +141 -0
- data/lib/standard_ledger/modes/matview.rb +35 -4
- data/lib/standard_ledger/modes/trigger.rb +50 -0
- data/lib/standard_ledger/projector.rb +279 -2
- data/lib/standard_ledger/version.rb +1 -1
- data/lib/standard_ledger.rb +38 -15
- data/lib/tasks/standard_ledger.rake +60 -0
- metadata +10 -5
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 411ad3db9a71ca8adddecc84d407dabddbd167c7ed00e534e975a700b3a2e8f2
|
|
4
|
+
data.tar.gz: aeb938a4e1fb16c6df67fbb76a93a5687c40116be040df5da44e65541ce4d282
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 2d0cd11b66c8d22b76cf9444e8840d92c35ccd81d4eba224e8123b208aa5c706503bc9cf6e9bbf1dc7bff564df4728adac79eeeacc254a976efb8cfa1fd0907a
|
|
7
|
+
data.tar.gz: 5f352fe00a36e01ef43aa4d8efb197a32314c43f0b31657342996dc91bcf422933e4076b851813bd6c6fd7ab69a52d18f53e9650fbced76292b9deca52b6ce0b
|
data/CHANGELOG.md
CHANGED
|
@@ -8,6 +8,174 @@ project adheres to [Semantic Versioning](https://semver.org/).
|
|
|
8
8
|
|
|
9
9
|
Nothing yet.
|
|
10
10
|
|
|
11
|
+
## [0.4.0] - 2026-05-07
|
|
12
|
+
|
|
13
|
+
### Added
|
|
14
|
+
- New `:manual` projection mode. Records the projection contract
|
|
15
|
+
(target + projector class) without installing any callback —
|
|
16
|
+
intended for AASM/state-machine entries whose interesting lifecycle
|
|
17
|
+
event is a transition rather than `after_create`. Hosts invoke the
|
|
18
|
+
projector explicitly from operation code; the gem keeps the
|
|
19
|
+
contract introspectable (`standard_ledger_projections`) and
|
|
20
|
+
log-replayable via `StandardLedger.rebuild!`. Requires
|
|
21
|
+
`via: ProjectorClass`; rejects blocks, locks, and `permissive:`.
|
|
22
|
+
- New `allow_destroy:` keyword on `ledger_entry`. When `true`, an
|
|
23
|
+
immutable entry permits `destroy` (including `dependent: :destroy`
|
|
24
|
+
cascades from a parent record) while still blocking `save`/`update`.
|
|
25
|
+
The default is `false` — preserves the strict journal contract.
|
|
26
|
+
Use this when an owning record's destroy cascade needs to reap
|
|
27
|
+
events for sandbox tear-down or GDPR erasure.
|
|
28
|
+
- New `counters:` shortcut for `:inline` projections. A
|
|
29
|
+
`kind => column` Hash that synthesises one
|
|
30
|
+
`on(kind) { |t, _| t.class.increment_counter(col, t.id) }` per
|
|
31
|
+
entry. Direct UPDATE (the class-method form) is intentional — it
|
|
32
|
+
invalidates the SQL query cache for the target table, keeping
|
|
33
|
+
multiple sibling-entry creates inside a single transaction (e.g.
|
|
34
|
+
via `accepts_nested_attributes_for`) from losing updates against
|
|
35
|
+
stale cached reads. Block form remains available for non-counter
|
|
36
|
+
projections.
|
|
37
|
+
- New `rebuild_sql:` keyword on `:trigger` mode. Equivalent to the
|
|
38
|
+
block-DSL `rebuild_sql "..."` clause, callable without a block.
|
|
39
|
+
- Partial unique indexes are now accepted by the idempotency-index
|
|
40
|
+
validator when their predicate is the canonical
|
|
41
|
+
`<idempotency_key> IS NOT NULL` shape. Other predicates still raise
|
|
42
|
+
`MissingIdempotencyIndex` with a clearer error message.
|
|
43
|
+
- New `StandardLedger::RefreshInsideTransaction` error. Raised when
|
|
44
|
+
`StandardLedger.refresh!(view, concurrently: true)` is called
|
|
45
|
+
inside an open transaction — Postgres rejects
|
|
46
|
+
`REFRESH MATERIALIZED VIEW CONCURRENTLY` inside transaction blocks,
|
|
47
|
+
and the gem now catches this at the boundary instead of letting
|
|
48
|
+
`PG::ActiveSqlTransaction` escape mid-call. The non-concurrent
|
|
49
|
+
form is still permitted by Postgres inside transactions and is
|
|
50
|
+
unaffected. No SQL is issued and no `.refreshed`/`.failed` event
|
|
51
|
+
fires when the guard rejects.
|
|
52
|
+
- `docs/MIGRATION_GUIDE.md` covering the five real-world adoption
|
|
53
|
+
paths (counter caches, custom inline logic, bespoke jobs, existing
|
|
54
|
+
Postgres triggers, AASM state machines) and the cascade-delete /
|
|
55
|
+
refresh-in-transaction edge cases.
|
|
56
|
+
|
|
57
|
+
### Documentation
|
|
58
|
+
- `ledger_entry`'s YARD now notes that a single-symbol `scope:` is
|
|
59
|
+
normalised to a flat array on the stored config; assertions in
|
|
60
|
+
host specs should compare against `[:foo]`, not `:foo`.
|
|
61
|
+
|
|
62
|
+
## [0.3.0] - 2026-05-05
|
|
63
|
+
|
|
64
|
+
### Added
|
|
65
|
+
- `:async` projection mode + `StandardLedger::ProjectionJob`. Used when
|
|
66
|
+
the projection is too expensive or stateful for the entry's transaction
|
|
67
|
+
(jsonb rebuild, multi-row aggregate) — the canonical example is
|
|
68
|
+
nutripod's `Order#payable_balance` / `Order#fulfillable_balance` jsonb
|
|
69
|
+
columns. The strategy installs an `after_create_commit` callback that
|
|
70
|
+
enqueues `StandardLedger::ProjectionJob` per (entry, projection) pair;
|
|
71
|
+
the job resolves the target via the entry's `belongs_to`, wraps
|
|
72
|
+
`target.with_lock { projector.apply(target, entry) }`, and fires the
|
|
73
|
+
same `<prefix>.projection.applied` / `<prefix>.projection.failed`
|
|
74
|
+
events as `:inline` (with an additional `attempt:` key drawn from
|
|
75
|
+
ActiveJob's `executions` accessor).
|
|
76
|
+
- Class-form only: `:async` projections must declare `via: ProjectorClass`.
|
|
77
|
+
The projector's `apply(target, entry)` should recompute from the log
|
|
78
|
+
inside `with_lock` rather than apply a delta — async retries can run
|
|
79
|
+
the projector more than once, so block-form per-kind handlers
|
|
80
|
+
(incrementing counters) are silently corrupting under retry. The
|
|
81
|
+
registration path rejects block forms with a clear ArgumentError;
|
|
82
|
+
`lock:` and `permissive: true` are also rejected (`with_lock` is
|
|
83
|
+
unconditional and there are no per-kind handlers).
|
|
84
|
+
- Retries: `Config#default_async_retries` (default 3 total attempts —
|
|
85
|
+
one initial run + two retries, matching ActiveJob's `retry_on
|
|
86
|
+
attempts:` semantics) caps the attempt count. The job uses a
|
|
87
|
+
hand-rolled `rescue_from(StandardError)` that reads the cap at
|
|
88
|
+
perform time so reconfiguration in tests / hosts takes effect
|
|
89
|
+
immediately, with `discard_on StandardLedger::Error` so programmer
|
|
90
|
+
errors (missing definition, renamed association) skip the retry
|
|
91
|
+
budget entirely. Each failure emits its own
|
|
92
|
+
`<prefix>.projection.failed` event with the current `attempt` so
|
|
93
|
+
subscribers see the full retry history.
|
|
94
|
+
- `Config#default_async_job` — hosts can swap the default
|
|
95
|
+
`StandardLedger::ProjectionJob` for their own subclass (per-mode
|
|
96
|
+
queue routing, custom retry policies). The strategy reads it at
|
|
97
|
+
enqueue time.
|
|
98
|
+
- `with_modes(EntryClass => :inline)` interop: when the override map
|
|
99
|
+
forces an entry class to `:inline`, the strategy short-circuits the
|
|
100
|
+
enqueue and runs `target.with_lock { projector.apply(target, entry) }`
|
|
101
|
+
synchronously inside the block. Useful in unit specs that want
|
|
102
|
+
end-to-end coverage without standing up a job runner. Notifications
|
|
103
|
+
fire with `mode: :async, attempt: 1` so subscribers can't tell the
|
|
104
|
+
difference.
|
|
105
|
+
- `StandardLedger.rebuild!` extends to `:async` projections — same
|
|
106
|
+
per-target rebuild semantics as `:inline` (delegates to
|
|
107
|
+
`definition.projector_class.new.rebuild(target)`). The mode
|
|
108
|
+
difference is only in the after-create path, not the rebuild path.
|
|
109
|
+
- `:trigger` projection mode. The host owns the database trigger
|
|
110
|
+
(created in a Rails migration); the gem **does not create or manage
|
|
111
|
+
triggers** — that's deliberate, because giving a Ruby DSL the power
|
|
112
|
+
to install/replace triggers is a deploy footgun (silent re-creation
|
|
113
|
+
on `db:schema:load` against a non-empty DB), and triggers are
|
|
114
|
+
versioned by `db/schema.rb` like any other DDL. The gem only records
|
|
115
|
+
the trigger's name and the equivalent rebuild SQL.
|
|
116
|
+
- `projects_onto :assoc, mode: :trigger, trigger_name: "..." do
|
|
117
|
+
rebuild_sql "..." end` declares a trigger projection. The
|
|
118
|
+
`trigger_name:` keyword is required (a non-empty string); the
|
|
119
|
+
block must call `rebuild_sql "..."` exactly once with a SQL
|
|
120
|
+
string containing the `:target_id` placeholder. Registration
|
|
121
|
+
rejects `via:`, `lock:`, and `permissive:` (none are meaningful
|
|
122
|
+
for `:trigger` mode — the trigger is the contract). `Definition`
|
|
123
|
+
gains a `trigger_name` field, populated only for `:trigger` mode;
|
|
124
|
+
the rebuild SQL is stored in the existing `recompute_sql` slot
|
|
125
|
+
(shared with `:sql` mode — both modes use it the same way).
|
|
126
|
+
- `StandardLedger::Modes::Trigger` strategy — `install!` is a no-op
|
|
127
|
+
marker (trigger projections fire from the database, not Ruby, so
|
|
128
|
+
no `after_create` callback is wired). The strategy class exists
|
|
129
|
+
only to keep `Projector#install_mode_callbacks_for`'s dispatch
|
|
130
|
+
table uniform across modes and to mark the entry class as having
|
|
131
|
+
at least one `:trigger` projection registered.
|
|
132
|
+
- `StandardLedger.rebuild!(EntryClass)` extends to `:trigger`
|
|
133
|
+
projections: the same SQL recompute path as `:sql` mode runs the
|
|
134
|
+
recorded `rebuild_sql` against each target the log references,
|
|
135
|
+
binding `:target_id` to each target's id. `target:` /
|
|
136
|
+
`target_class:` / no-arg scoping works the same way.
|
|
137
|
+
`<prefix>.projection.rebuilt` fires per target with `mode:
|
|
138
|
+
:trigger`. The gem does NOT verify or recreate the trigger
|
|
139
|
+
during `rebuild!` — `standard_ledger:doctor` is the deploy-time
|
|
140
|
+
check.
|
|
141
|
+
- `standard_ledger:doctor` rake task. Iterates every registered
|
|
142
|
+
`:trigger` projection across all loaded entry classes (discovered
|
|
143
|
+
by walking `ActiveRecord::Base.descendants` for classes that
|
|
144
|
+
include `StandardLedger::Projector` and have at least one `:trigger`
|
|
145
|
+
projection). For each, queries `pg_trigger` to confirm the named
|
|
146
|
+
trigger exists in the connected schema. Reports missing triggers
|
|
147
|
+
on stderr with a clear remediation message and exits 1; prints a
|
|
148
|
+
success message and exits 0 otherwise. **Postgres-only** — the
|
|
149
|
+
task queries `pg_trigger` directly and will raise on a non-Postgres
|
|
150
|
+
connection. SQLite has no comparable per-statement trigger
|
|
151
|
+
introspection that fits this gem's contract; the only adopter
|
|
152
|
+
today is nutripod-web (Postgres). The task is auto-loaded via
|
|
153
|
+
`Engine.rake_tasks` so `bin/rails -T standard_ledger` shows it
|
|
154
|
+
immediately after the gem is installed.
|
|
155
|
+
- Integration spec for `:trigger` mode
|
|
156
|
+
(`spec/standard_ledger/trigger_integration_spec.rb`) covers DSL
|
|
157
|
+
registration validation (missing `trigger_name`, missing block,
|
|
158
|
+
empty block, `via:` / `lock:` / `permissive:` rejection,
|
|
159
|
+
`:target_id` placeholder enforcement, double `rebuild_sql` call),
|
|
160
|
+
the strategy's no-callback contract (creating an entry does NOT
|
|
161
|
+
mutate the target via Ruby — that's the trigger's job in
|
|
162
|
+
production), and `StandardLedger.rebuild!` for both single-target
|
|
163
|
+
and walk-the-log scoping with the `<prefix>.projection.rebuilt`
|
|
164
|
+
event firing with `mode: :trigger`.
|
|
165
|
+
- Doctor task spec
|
|
166
|
+
(`spec/standard_ledger/tasks/doctor_spec.rb`) mocks the
|
|
167
|
+
`connection.exec_query` against `pg_trigger` to exercise the
|
|
168
|
+
task's three behaviours (success, failure with exit 1 + stderr
|
|
169
|
+
message, ignoring entry classes without `:trigger` projections)
|
|
170
|
+
without requiring a real Postgres database.
|
|
171
|
+
- Integration spec for `:async` mode
|
|
172
|
+
(`spec/standard_ledger/async_integration_spec.rb`) covers DSL
|
|
173
|
+
registration validation, after-commit enqueue, multi-target fan-out,
|
|
174
|
+
nil-FK skip (both at enqueue and at perform), the `with_modes`
|
|
175
|
+
inline override, the `default_async_job` swap, retry-on-failure with
|
|
176
|
+
attempt-counter telemetry, the `discard_on StandardLedger::Error`
|
|
177
|
+
programmer-error path, and the rebuild path.
|
|
178
|
+
|
|
11
179
|
## [0.2.0] - 2026-05-05
|
|
12
180
|
|
|
13
181
|
### Added
|
|
@@ -255,6 +423,7 @@ roadmap.
|
|
|
255
423
|
and `:trigger` (host-owned, gem records rebuild SQL).
|
|
256
424
|
- `standard_ledger:doctor` rake task (verifies trigger presence, etc.).
|
|
257
425
|
|
|
258
|
-
[Unreleased]: https://github.com/rarebit-one/standard_ledger/compare/v0.
|
|
426
|
+
[Unreleased]: https://github.com/rarebit-one/standard_ledger/compare/v0.3.0...HEAD
|
|
427
|
+
[0.3.0]: https://github.com/rarebit-one/standard_ledger/compare/v0.2.0...v0.3.0
|
|
259
428
|
[0.2.0]: https://github.com/rarebit-one/standard_ledger/compare/v0.1.0...v0.2.0
|
|
260
429
|
[0.1.0]: https://github.com/rarebit-one/standard_ledger/releases/tag/v0.1.0
|
data/README.md
CHANGED
|
@@ -2,10 +2,12 @@
|
|
|
2
2
|
|
|
3
3
|
Immutable journal entries with declarative aggregate projections for Rails apps.
|
|
4
4
|
|
|
5
|
-
> **Status: v0.
|
|
6
|
-
>
|
|
7
|
-
>
|
|
8
|
-
>
|
|
5
|
+
> **Status: v0.3.0** — feature-complete across all five projection modes
|
|
6
|
+
> (`:inline`, `:async`, `:sql`, `:matview`, `:trigger`) plus
|
|
7
|
+
> `StandardLedger.rebuild!` log-replay, `StandardLedger.refresh!` ad-hoc
|
|
8
|
+
> matview refresh, and the `standard_ledger:doctor` rake task. Ready for
|
|
9
|
+
> adoption in luminality-web, fundbright-web, sidekick-web, and
|
|
10
|
+
> nutripod-web. See [`standard_ledger-design.md`](https://github.com/rarebit-one/standard_ledger/blob/main/standard_ledger-design.md)
|
|
9
11
|
> for the full design and rollout plan.
|
|
10
12
|
|
|
11
13
|
## What it is
|
|
@@ -101,6 +103,50 @@ mid-loop are not unwound. Block-form (delta) projections raise
|
|
|
101
103
|
`NotRebuildable` because they cannot be reconstructed from the log
|
|
102
104
|
without a host-supplied recompute path.
|
|
103
105
|
|
|
106
|
+
For projections too expensive or stateful to run inside the entry's
|
|
107
|
+
transaction (jsonb rebuild, multi-row aggregate), use `mode: :async` —
|
|
108
|
+
the strategy enqueues `StandardLedger::ProjectionJob` from
|
|
109
|
+
`after_create_commit`, and the job runs `target.with_lock { projector.apply(target, entry) }`
|
|
110
|
+
on the configured ActiveJob backend:
|
|
111
|
+
|
|
112
|
+
```ruby
|
|
113
|
+
class Orders::FulfillableProjector < StandardLedger::Projection
|
|
114
|
+
# Recompute the jsonb balance from the full log inside with_lock.
|
|
115
|
+
# `:async` projectors must be retry-safe — async retries can run
|
|
116
|
+
# `apply` more than once, so block-form per-kind handlers
|
|
117
|
+
# (incrementing counters) are rejected at registration time.
|
|
118
|
+
def apply(order, _entry)
|
|
119
|
+
order.update!(
|
|
120
|
+
fulfillable_balance: order.fulfillment_records.group(:key).sum(:amount)
|
|
121
|
+
)
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
def rebuild(order)
|
|
125
|
+
apply(order, nil)
|
|
126
|
+
end
|
|
127
|
+
end
|
|
128
|
+
|
|
129
|
+
class FulfillmentRecord < ApplicationRecord
|
|
130
|
+
include StandardLedger::Entry
|
|
131
|
+
include StandardLedger::Projector
|
|
132
|
+
|
|
133
|
+
belongs_to :order
|
|
134
|
+
|
|
135
|
+
ledger_entry kind: :action, idempotency_key: :external_ref, scope: :organisation_id
|
|
136
|
+
|
|
137
|
+
projects_onto :order, mode: :async, via: Orders::FulfillableProjector
|
|
138
|
+
end
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
Retries are capped by `Config#default_async_retries` (default 3); the
|
|
142
|
+
job emits `<prefix>.projection.applied` and `<prefix>.projection.failed`
|
|
143
|
+
events with an additional `attempt:` key so subscribers can tell
|
|
144
|
+
first-try success from retry success. Tests can force async projections
|
|
145
|
+
to run inline via `StandardLedger.with_modes(FulfillmentRecord => :inline) { ... }`
|
|
146
|
+
— the strategy short-circuits the enqueue and runs the projector
|
|
147
|
+
synchronously inside `with_lock`, so end-to-end coverage works without a
|
|
148
|
+
job runner.
|
|
149
|
+
|
|
104
150
|
For projections expressible as a single `UPDATE` over an aggregate of the
|
|
105
151
|
log, use `mode: :sql` — no Ruby-side handlers, no AR object loads, just
|
|
106
152
|
a recompute statement that runs in the entry's `after_create`:
|
|
@@ -132,6 +178,58 @@ SQL is the entire contract — `:sql` projections are naturally
|
|
|
132
178
|
rebuildable: `StandardLedger.rebuild!` runs the same statement against
|
|
133
179
|
every target the log references.
|
|
134
180
|
|
|
181
|
+
When the host **already has** a database trigger that updates the
|
|
182
|
+
projection target on every entry INSERT, register it with `mode: :trigger`
|
|
183
|
+
so the gem records the trigger's name and the equivalent rebuild SQL —
|
|
184
|
+
without taking ownership of the trigger DDL. The host writes the trigger
|
|
185
|
+
in a Rails migration; the gem only consumes the metadata.
|
|
186
|
+
|
|
187
|
+
```ruby
|
|
188
|
+
class InventoryRecord < ApplicationRecord
|
|
189
|
+
include StandardLedger::Entry
|
|
190
|
+
include StandardLedger::Projector
|
|
191
|
+
|
|
192
|
+
belongs_to :sku
|
|
193
|
+
|
|
194
|
+
ledger_entry kind: :action, idempotency_key: :serial_no, scope: :organisation_id
|
|
195
|
+
|
|
196
|
+
projects_onto :sku, mode: :trigger,
|
|
197
|
+
trigger_name: "inventory_records_apply_to_skus" do
|
|
198
|
+
rebuild_sql <<~SQL
|
|
199
|
+
UPDATE skus SET
|
|
200
|
+
total_count = c.total_count,
|
|
201
|
+
reserved_count = c.reserved_count,
|
|
202
|
+
free_count = c.total_count - c.reserved_count
|
|
203
|
+
FROM (
|
|
204
|
+
SELECT sku_id,
|
|
205
|
+
COUNT(*) FILTER (WHERE action IN ('grant','adjust_in')) AS total_count,
|
|
206
|
+
COUNT(*) FILTER (WHERE action = 'reserve') AS reserved_count
|
|
207
|
+
FROM inventory_records
|
|
208
|
+
WHERE sku_id = :target_id
|
|
209
|
+
GROUP BY sku_id
|
|
210
|
+
) c
|
|
211
|
+
WHERE skus.id = :target_id AND skus.id = c.sku_id
|
|
212
|
+
SQL
|
|
213
|
+
end
|
|
214
|
+
end
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
The trigger continues to fire on every `INSERT` (the host owns the DDL);
|
|
218
|
+
the gem records the trigger name + rebuild SQL for two purposes:
|
|
219
|
+
|
|
220
|
+
- `StandardLedger.rebuild!(InventoryRecord, target: sku)` runs the
|
|
221
|
+
recorded `rebuild_sql` with `:target_id` bound to each target's id.
|
|
222
|
+
- `bin/rails standard_ledger:doctor` verifies that every registered
|
|
223
|
+
`:trigger` projection's named trigger exists in the connected schema
|
|
224
|
+
(queries `pg_trigger`). Run this as a deploy-time check — migration
|
|
225
|
+
drift surfaces immediately rather than at runtime. **Postgres-only**;
|
|
226
|
+
the task raises on non-Postgres connections.
|
|
227
|
+
|
|
228
|
+
Registration rejects `via:`, `lock:`, and `permissive:` (none are
|
|
229
|
+
meaningful when the trigger itself is the contract). The `trigger_name:`
|
|
230
|
+
keyword is required; the block must call `rebuild_sql "..."` exactly
|
|
231
|
+
once with a SQL string containing the `:target_id` placeholder.
|
|
232
|
+
|
|
135
233
|
Refresh a `:matview` projection ad-hoc when the host needs immediate
|
|
136
234
|
read-your-write semantics (e.g. at the end of a draw operation, before
|
|
137
235
|
the next scheduled refresh would otherwise show stale counts):
|
|
@@ -11,8 +11,9 @@ module StandardLedger
|
|
|
11
11
|
# `c.default_async_job = Orders::FulfillableProjectionJob`.
|
|
12
12
|
attr_accessor :default_async_job
|
|
13
13
|
|
|
14
|
-
#
|
|
15
|
-
# Default: 3.
|
|
14
|
+
# Total attempts (including the first) for an `:async` projection before
|
|
15
|
+
# the failure is propagated. Default: 3 (one initial run + two retries).
|
|
16
|
+
# Matches ActiveJob's `retry_on attempts:` semantics.
|
|
16
17
|
attr_accessor :default_async_retries
|
|
17
18
|
|
|
18
19
|
# Scheduler backend for `:matview` refresh jobs. One of
|
|
@@ -15,5 +15,12 @@ module StandardLedger
|
|
|
15
15
|
# no-op — the gem emits events but ships no internal subscribers; hosts
|
|
16
16
|
# subscribe directly via ActiveSupport::Notifications.subscribe.
|
|
17
17
|
end
|
|
18
|
+
|
|
19
|
+
# Engines auto-discover `lib/tasks/*.rake` in most Rails versions, but
|
|
20
|
+
# we register explicitly for defence-in-depth. Hosts get
|
|
21
|
+
# `standard_ledger:doctor` available under `bin/rails -T standard_ledger`.
|
|
22
|
+
rake_tasks do
|
|
23
|
+
load File.expand_path("../tasks/standard_ledger.rake", __dir__)
|
|
24
|
+
end
|
|
18
25
|
end
|
|
19
26
|
end
|
|
@@ -35,15 +35,29 @@ module StandardLedger
|
|
|
35
35
|
# guards against duplicate inserts. `nil` means the entry is not
|
|
36
36
|
# idempotent — explicitly opt-in to that.
|
|
37
37
|
# @param scope [Symbol, Array<Symbol>, nil] additional columns the
|
|
38
|
-
# idempotency index is scoped by (e.g. `:organisation_id`).
|
|
39
|
-
#
|
|
40
|
-
#
|
|
41
|
-
|
|
38
|
+
# idempotency index is scoped by (e.g. `:organisation_id`). Always
|
|
39
|
+
# normalised to a flat array on the stored config so downstream
|
|
40
|
+
# reads don't need to handle both shapes — assertions in host specs
|
|
41
|
+
# should compare against `[:foo]`, not `:foo`.
|
|
42
|
+
# @param immutable [Boolean] when true (default), `save`/`update`
|
|
43
|
+
# raise after the row is persisted. Also blocks `destroy` unless
|
|
44
|
+
# `allow_destroy: true` is set.
|
|
45
|
+
# @param allow_destroy [Boolean] when true, `destroy` (including
|
|
46
|
+
# `dependent: :destroy` cascades from a parent record) is permitted
|
|
47
|
+
# even on `immutable: true` entries. Use this when an owning record
|
|
48
|
+
# declares `has_many :events, dependent: :destroy` and you want the
|
|
49
|
+
# cascade to work for cleanup paths (sandbox tear-down, GDPR
|
|
50
|
+
# erasure, etc.) while still blocking app-code mutations to
|
|
51
|
+
# persisted entries. Defaults to `false` — keeping the strict
|
|
52
|
+
# journal contract.
|
|
53
|
+
def ledger_entry(kind: :kind, idempotency_key: nil, scope: nil,
|
|
54
|
+
immutable: true, allow_destroy: false)
|
|
42
55
|
self.standard_ledger_entry_config = {
|
|
43
56
|
kind: kind,
|
|
44
57
|
idempotency_key: idempotency_key,
|
|
45
58
|
scope: Array(scope).compact,
|
|
46
|
-
immutable: immutable
|
|
59
|
+
immutable: immutable,
|
|
60
|
+
allow_destroy: allow_destroy
|
|
47
61
|
}
|
|
48
62
|
self.standard_ledger_idempotency_index_validated = false
|
|
49
63
|
end
|
|
@@ -97,14 +111,28 @@ module StandardLedger
|
|
|
97
111
|
indexes = connection.indexes(table_name)
|
|
98
112
|
|
|
99
113
|
match = indexes.any? do |index|
|
|
100
|
-
|
|
114
|
+
next false unless index.unique
|
|
115
|
+
next false unless index.columns.map(&:to_s).to_set == required
|
|
116
|
+
|
|
117
|
+
# Full-table unique indexes are always valid. Partial indexes are
|
|
118
|
+
# accepted only when the predicate is the canonical
|
|
119
|
+
# `<idempotency_key> IS NOT NULL` shape — that's the common
|
|
120
|
+
# real-world pattern (e.g. an event table whose serial number is
|
|
121
|
+
# optional but unique-per-scope when present), and it preserves
|
|
122
|
+
# the gem's idempotency contract: rows with a non-null key are
|
|
123
|
+
# deduped; rows without one are explicitly opting out.
|
|
124
|
+
standard_ledger_index_predicate_acceptable?(index, config[:idempotency_key])
|
|
101
125
|
end
|
|
102
126
|
|
|
103
127
|
unless match
|
|
104
128
|
raise StandardLedger::MissingIdempotencyIndex,
|
|
105
129
|
"#{name} declares idempotency_key: #{config[:idempotency_key].inspect} " \
|
|
106
130
|
"with scope: #{config[:scope].inspect} but no matching unique index " \
|
|
107
|
-
"covers exactly #{required.to_a.sort.inspect} on `#{table_name}`."
|
|
131
|
+
"covers exactly #{required.to_a.sort.inspect} on `#{table_name}`. " \
|
|
132
|
+
"If a matching partial index exists, its WHERE predicate must be " \
|
|
133
|
+
"`#{config[:idempotency_key]} IS NOT NULL` (other predicates aren't " \
|
|
134
|
+
"automatically validated — opt out of the check by setting " \
|
|
135
|
+
"idempotency_key: nil and enforcing uniqueness another way)."
|
|
108
136
|
end
|
|
109
137
|
|
|
110
138
|
self.standard_ledger_idempotency_index_validated = true
|
|
@@ -112,6 +140,29 @@ module StandardLedger
|
|
|
112
140
|
|
|
113
141
|
private
|
|
114
142
|
|
|
143
|
+
# Match a partial-index predicate of the form `<col> IS NOT NULL`
|
|
144
|
+
# (with optional whitespace and optional table/schema qualification
|
|
145
|
+
# on the column reference). Full-table indexes (no predicate) always
|
|
146
|
+
# qualify. This is conservative: predicates outside this shape can
|
|
147
|
+
# still be perfectly valid for the host's idempotency intent, but
|
|
148
|
+
# we'd need a real SQL parser to decide that — better to raise and
|
|
149
|
+
# let the host either restructure their index or opt out via
|
|
150
|
+
# `idempotency_key: nil`.
|
|
151
|
+
def standard_ledger_index_predicate_acceptable?(index, idempotency_key)
|
|
152
|
+
predicate = index.where
|
|
153
|
+
return true if predicate.nil? || predicate.to_s.strip.empty?
|
|
154
|
+
|
|
155
|
+
col = idempotency_key.to_s
|
|
156
|
+
# Postgres wraps the index predicate in parentheses when it returns
|
|
157
|
+
# it via pg_indexes (e.g. `(idempotency_key IS NOT NULL)`) — strip
|
|
158
|
+
# those along with the per-adapter quoting characters before
|
|
159
|
+
# matching. SQLite returns the raw expression, so the same strip is
|
|
160
|
+
# a no-op there. The regex tolerates whitespace around the column
|
|
161
|
+
# and operator and accepts an optional table-qualifier prefix.
|
|
162
|
+
normalised = predicate.to_s.gsub(/["`\[\]()]/, "").strip
|
|
163
|
+
normalised.match?(/\A([\w]+\.)?#{Regexp.escape(col)}\s+IS\s+NOT\s+NULL\z/i)
|
|
164
|
+
end
|
|
165
|
+
|
|
115
166
|
def find_existing_standard_ledger_entry(attributes)
|
|
116
167
|
return nil if attributes.nil?
|
|
117
168
|
|
|
@@ -165,9 +216,11 @@ module StandardLedger
|
|
|
165
216
|
# plain Ruby classes that include Entry for testing the DSL surface
|
|
166
217
|
# get the macro registration without the callback. AR's `readonly?`
|
|
167
218
|
# path covers save/update on persisted rows; this catch-all stops
|
|
168
|
-
# `destroy` for the AR case
|
|
219
|
+
# `destroy` for the AR case unless the entry opts out via
|
|
220
|
+
# `allow_destroy: true` (typically because an owning record's
|
|
221
|
+
# `dependent: :destroy` cascade needs to reap them on cleanup).
|
|
169
222
|
if respond_to?(:before_destroy)
|
|
170
|
-
before_destroy :standard_ledger_raise_readonly, if: :
|
|
223
|
+
before_destroy :standard_ledger_raise_readonly, if: :standard_ledger_destroy_blocked?
|
|
171
224
|
end
|
|
172
225
|
|
|
173
226
|
# Emit `<namespace>.entry.created` after the row is durably committed
|
|
@@ -187,16 +240,40 @@ module StandardLedger
|
|
|
187
240
|
!!@_standard_ledger_idempotent
|
|
188
241
|
end
|
|
189
242
|
|
|
190
|
-
# AR consults `readonly?` from `save`/`update` paths; raising
|
|
243
|
+
# AR consults `readonly?` from `save`/`update`/`destroy` paths; raising
|
|
191
244
|
# ReadOnlyRecord here matches the ActiveRecord contract for persisted
|
|
192
245
|
# immutable rows. New, unpersisted instances stay writable so the
|
|
193
246
|
# initial INSERT can land.
|
|
247
|
+
#
|
|
248
|
+
# When `allow_destroy: true` is set, `#destroy` toggles
|
|
249
|
+
# `@_standard_ledger_destroying` so `readonly?` returns false for the
|
|
250
|
+
# duration of the destroy call (and the duration of any cascade
|
|
251
|
+
# destroys that fire from its `dependent: :destroy` associations).
|
|
252
|
+
# The save/update path is unaffected — those still raise on
|
|
253
|
+
# persisted rows.
|
|
194
254
|
def readonly?
|
|
195
255
|
return super unless standard_ledger_immutable?
|
|
256
|
+
return false if @_standard_ledger_destroying
|
|
196
257
|
|
|
197
258
|
!new_record?
|
|
198
259
|
end
|
|
199
260
|
|
|
261
|
+
# Wrap `destroy` so it can bypass the `readonly?` guard when the
|
|
262
|
+
# entry has opted in via `allow_destroy: true`. This applies to
|
|
263
|
+
# `destroy`, `destroy!`, and `dependent: :destroy` cascades from a
|
|
264
|
+
# parent record (all routes call through `#destroy`).
|
|
265
|
+
def destroy
|
|
266
|
+
return super unless self.class.respond_to?(:standard_ledger_entry_config)
|
|
267
|
+
|
|
268
|
+
config = self.class.standard_ledger_entry_config
|
|
269
|
+
return super if config.nil? || !config[:immutable] || !config[:allow_destroy]
|
|
270
|
+
|
|
271
|
+
@_standard_ledger_destroying = true
|
|
272
|
+
super
|
|
273
|
+
ensure
|
|
274
|
+
@_standard_ledger_destroying = false
|
|
275
|
+
end
|
|
276
|
+
|
|
200
277
|
# Returns the entry's belongs_to targets keyed by association name.
|
|
201
278
|
# Used by the `entry.created` notification payload and by
|
|
202
279
|
# `StandardLedger.post`'s telemetry. Skips polymorphic and missing
|
|
@@ -232,6 +309,18 @@ module StandardLedger
|
|
|
232
309
|
!config.nil? && config[:immutable]
|
|
233
310
|
end
|
|
234
311
|
|
|
312
|
+
# Destroys are blocked when the entry is `immutable: true` AND the user
|
|
313
|
+
# has not opted out via `allow_destroy: true`. Split out so the
|
|
314
|
+
# before_destroy guard can be conditional independently of the
|
|
315
|
+
# save/update `readonly?` path.
|
|
316
|
+
def standard_ledger_destroy_blocked?
|
|
317
|
+
config = self.class.standard_ledger_entry_config
|
|
318
|
+
return false if config.nil?
|
|
319
|
+
return false unless config[:immutable]
|
|
320
|
+
|
|
321
|
+
!config[:allow_destroy]
|
|
322
|
+
end
|
|
323
|
+
|
|
235
324
|
def standard_ledger_raise_readonly
|
|
236
325
|
raise ActiveRecord::ReadOnlyRecord
|
|
237
326
|
end
|
|
@@ -30,4 +30,14 @@ module StandardLedger
|
|
|
30
30
|
super("Enqueued #{enqueued.size} projections; #{failed.size} failed to enqueue")
|
|
31
31
|
end
|
|
32
32
|
end
|
|
33
|
+
|
|
34
|
+
# Raised when `StandardLedger.refresh!(view, concurrently: true)` is called
|
|
35
|
+
# inside an open transaction. PostgreSQL rejects
|
|
36
|
+
# `REFRESH MATERIALIZED VIEW CONCURRENTLY` inside transaction blocks; the
|
|
37
|
+
# gem catches this at the boundary so the failure is a clear,
|
|
38
|
+
# gem-attributable error instead of a raw `PG::ActiveSqlTransaction`.
|
|
39
|
+
# Callers wanting read-your-write semantics inside an operation should
|
|
40
|
+
# wrap the call in `connection.add_transaction_record { ... }` to defer
|
|
41
|
+
# to after-commit, or move the refresh outside the transaction block.
|
|
42
|
+
class RefreshInsideTransaction < Error; end
|
|
33
43
|
end
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
require "active_job"
|
|
2
|
+
|
|
3
|
+
module StandardLedger
|
|
4
|
+
# ActiveJob class that runs a single `:async`-mode projection after the
|
|
5
|
+
# entry's outer transaction has committed. Enqueued by
|
|
6
|
+
# `StandardLedger::Modes::Async` from an `after_create_commit` callback.
|
|
7
|
+
#
|
|
8
|
+
# The job resolves the projection definition by `target_association` (the
|
|
9
|
+
# only stable handle — the Definition struct doesn't serialize cleanly
|
|
10
|
+
# through ActiveJob), looks up the target via the entry's `belongs_to`
|
|
11
|
+
# setter, and runs `target.with_lock { projector_class.new.apply(target,
|
|
12
|
+
# entry) }`. The projector must be class-form (`via: ProjectorClass`) and
|
|
13
|
+
# should recompute the aggregate from the log inside `apply` for retry
|
|
14
|
+
# safety — async projections can run more than once when the job retries.
|
|
15
|
+
#
|
|
16
|
+
# Retries are configurable per-projection via subclassing this job and
|
|
17
|
+
# overriding `retry_on`, or globally via `Config#default_async_retries`
|
|
18
|
+
# (default 3). When retries are exhausted, ActiveJob's dead-letter behavior
|
|
19
|
+
# takes over — the job still emits `<prefix>.projection.failed` on every
|
|
20
|
+
# attempt so subscribers see the full retry history.
|
|
21
|
+
#
|
|
22
|
+
# Notification payloads include `attempt:` (drawn from ActiveJob's
|
|
23
|
+
# `executions` accessor — 1 on first attempt, increments per retry) so
|
|
24
|
+
# subscribers can distinguish first-try success from retry-success.
|
|
25
|
+
class ProjectionJob < ::ActiveJob::Base
|
|
26
|
+
# Hand-rolled retry path so the attempt cap reads
|
|
27
|
+
# `Config#default_async_retries` at perform time. ActiveJob's
|
|
28
|
+
# `retry_on attempts:` requires a constant Integer (or `:unlimited`)
|
|
29
|
+
# and is captured at class-definition time — that's incompatible with
|
|
30
|
+
# the gem's pattern of letting hosts reconfigure `default_async_retries`
|
|
31
|
+
# in their initializer (or specs flipping it temporarily). We rescue
|
|
32
|
+
# `StandardError` and re-enqueue manually until the cap is reached.
|
|
33
|
+
rescue_from(StandardError) do |error|
|
|
34
|
+
attempts = StandardLedger.config.default_async_retries
|
|
35
|
+
if executions < attempts
|
|
36
|
+
# Compute a polynomial backoff inline. Mirrors ActiveJob's
|
|
37
|
+
# `:polynomially_longer` algorithm (`executions**4 + 2`) without
|
|
38
|
+
# depending on its private `determine_delay` API.
|
|
39
|
+
delay = (executions**4) + 2
|
|
40
|
+
retry_job(wait: delay, error: error)
|
|
41
|
+
else
|
|
42
|
+
raise error
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
# Discard programmer errors immediately — they're deterministic and
|
|
47
|
+
# retrying just burns the budget. `StandardLedger::Error` is raised by
|
|
48
|
+
# `#perform` on missing/renamed projection definitions and similar
|
|
49
|
+
# bookkeeping mistakes; the next attempt would raise the same error.
|
|
50
|
+
# Declared AFTER the `rescue_from(StandardError)` block above because
|
|
51
|
+
# `ActiveSupport::Rescuable` searches handlers from the most-recently-
|
|
52
|
+
# registered first — so this more-specific `StandardLedger::Error`
|
|
53
|
+
# handler wins over the catch-all retry path for its matching errors.
|
|
54
|
+
discard_on StandardLedger::Error
|
|
55
|
+
|
|
56
|
+
def perform(entry, target_association)
|
|
57
|
+
definition = entry.class.standard_ledger_projections.find { |d|
|
|
58
|
+
d.mode == :async && d.target_association == target_association.to_sym
|
|
59
|
+
}
|
|
60
|
+
if definition.nil?
|
|
61
|
+
raise StandardLedger::Error,
|
|
62
|
+
"no :async projection #{target_association.inspect} on #{entry.class.name}"
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
target = entry.public_send(definition.target_association)
|
|
66
|
+
return if target.nil?
|
|
67
|
+
|
|
68
|
+
prefix = StandardLedger.config.notification_namespace
|
|
69
|
+
started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
70
|
+
|
|
71
|
+
begin
|
|
72
|
+
target.with_lock do
|
|
73
|
+
definition.projector_class.new.apply(target, entry)
|
|
74
|
+
end
|
|
75
|
+
rescue StandardError => e
|
|
76
|
+
duration_ms = (Process.clock_gettime(Process::CLOCK_MONOTONIC) - started) * 1000.0
|
|
77
|
+
StandardLedger::EventEmitter.emit(
|
|
78
|
+
"#{prefix}.projection.failed",
|
|
79
|
+
entry: entry, target: target, projection: definition.target_association,
|
|
80
|
+
mode: :async, error: e, duration_ms: duration_ms,
|
|
81
|
+
attempt: executions
|
|
82
|
+
)
|
|
83
|
+
raise
|
|
84
|
+
end
|
|
85
|
+
|
|
86
|
+
duration_ms = (Process.clock_gettime(Process::CLOCK_MONOTONIC) - started) * 1000.0
|
|
87
|
+
StandardLedger::EventEmitter.emit(
|
|
88
|
+
"#{prefix}.projection.applied",
|
|
89
|
+
entry: entry, target: target, projection: definition.target_association,
|
|
90
|
+
mode: :async, duration_ms: duration_ms, attempt: executions
|
|
91
|
+
)
|
|
92
|
+
end
|
|
93
|
+
end
|
|
94
|
+
end
|