standard_ledger 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +119 -1
- data/README.md +102 -4
- data/lib/standard_ledger/config.rb +3 -2
- data/lib/standard_ledger/engine.rb +7 -0
- data/lib/standard_ledger/jobs/projection_job.rb +94 -0
- data/lib/standard_ledger/modes/async.rb +141 -0
- data/lib/standard_ledger/modes/trigger.rb +50 -0
- data/lib/standard_ledger/projector.rb +168 -2
- data/lib/standard_ledger/version.rb +1 -1
- data/lib/standard_ledger.rb +37 -15
- data/lib/tasks/standard_ledger.rake +60 -0
- metadata +10 -5
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 276066f9976675bcd6ed7220d0df2d72518c355eab43036807cf53c1c1da7e00
|
|
4
|
+
data.tar.gz: 01de25809973bb72ed2414f32f5c6c9398f45f2d44fed75bfdb1743c9beaaedc
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: bc772c009801e0bf6ab2e3568e95aeff7305413fce076742d89dfb62388cada99a168659fc0745807ca661c31bd30cc33529b105c4f7bd163a91090d3b5603ab
|
|
7
|
+
data.tar.gz: 1002bacfe4f93410a2d06ffda6dec585328c0c248d518f3e0e12148a9bd5effb451f02dbb1967569b3e2d30b3d8c71bcc4abba53efa7e41d218f451ee21d659c
|
data/CHANGELOG.md
CHANGED
|
@@ -8,6 +8,123 @@ project adheres to [Semantic Versioning](https://semver.org/).
|
|
|
8
8
|
|
|
9
9
|
Nothing yet.
|
|
10
10
|
|
|
11
|
+
## [0.3.0] - 2026-05-05
|
|
12
|
+
|
|
13
|
+
### Added
|
|
14
|
+
- `:async` projection mode + `StandardLedger::ProjectionJob`. Used when
|
|
15
|
+
the projection is too expensive or stateful for the entry's transaction
|
|
16
|
+
(jsonb rebuild, multi-row aggregate) — the canonical example is
|
|
17
|
+
nutripod's `Order#payable_balance` / `Order#fulfillable_balance` jsonb
|
|
18
|
+
columns. The strategy installs an `after_create_commit` callback that
|
|
19
|
+
enqueues `StandardLedger::ProjectionJob` per (entry, projection) pair;
|
|
20
|
+
the job resolves the target via the entry's `belongs_to`, wraps
|
|
21
|
+
`target.with_lock { projector.apply(target, entry) }`, and fires the
|
|
22
|
+
same `<prefix>.projection.applied` / `<prefix>.projection.failed`
|
|
23
|
+
events as `:inline` (with an additional `attempt:` key drawn from
|
|
24
|
+
ActiveJob's `executions` accessor).
|
|
25
|
+
- Class-form only: `:async` projections must declare `via: ProjectorClass`.
|
|
26
|
+
The projector's `apply(target, entry)` should recompute from the log
|
|
27
|
+
inside `with_lock` rather than apply a delta — async retries can run
|
|
28
|
+
the projector more than once, so block-form per-kind handlers
|
|
29
|
+
(incrementing counters) are silently corrupting under retry. The
|
|
30
|
+
registration path rejects block forms with a clear ArgumentError;
|
|
31
|
+
`lock:` and `permissive: true` are also rejected (`with_lock` is
|
|
32
|
+
unconditional and there are no per-kind handlers).
|
|
33
|
+
- Retries: `Config#default_async_retries` (default 3 total attempts —
|
|
34
|
+
one initial run + two retries, matching ActiveJob's `retry_on
|
|
35
|
+
attempts:` semantics) caps the attempt count. The job uses a
|
|
36
|
+
hand-rolled `rescue_from(StandardError)` that reads the cap at
|
|
37
|
+
perform time so reconfiguration in tests / hosts takes effect
|
|
38
|
+
immediately, with `discard_on StandardLedger::Error` so programmer
|
|
39
|
+
errors (missing definition, renamed association) skip the retry
|
|
40
|
+
budget entirely. Each failure emits its own
|
|
41
|
+
`<prefix>.projection.failed` event with the current `attempt` so
|
|
42
|
+
subscribers see the full retry history.
|
|
43
|
+
- `Config#default_async_job` — hosts can swap the default
|
|
44
|
+
`StandardLedger::ProjectionJob` for their own subclass (per-mode
|
|
45
|
+
queue routing, custom retry policies). The strategy reads it at
|
|
46
|
+
enqueue time.
|
|
47
|
+
- `with_modes(EntryClass => :inline)` interop: when the override map
|
|
48
|
+
forces an entry class to `:inline`, the strategy short-circuits the
|
|
49
|
+
enqueue and runs `target.with_lock { projector.apply(target, entry) }`
|
|
50
|
+
synchronously inside the block. Useful in unit specs that want
|
|
51
|
+
end-to-end coverage without standing up a job runner. Notifications
|
|
52
|
+
fire with `mode: :async, attempt: 1` so subscribers can't tell the
|
|
53
|
+
difference.
|
|
54
|
+
- `StandardLedger.rebuild!` extends to `:async` projections — same
|
|
55
|
+
per-target rebuild semantics as `:inline` (delegates to
|
|
56
|
+
`definition.projector_class.new.rebuild(target)`). The mode
|
|
57
|
+
difference is only in the after-create path, not the rebuild path.
|
|
58
|
+
- `:trigger` projection mode. The host owns the database trigger
|
|
59
|
+
(created in a Rails migration); the gem **does not create or manage
|
|
60
|
+
triggers** — that's deliberate, because giving a Ruby DSL the power
|
|
61
|
+
to install/replace triggers is a deploy footgun (silent re-creation
|
|
62
|
+
on `db:schema:load` against a non-empty DB), and triggers are
|
|
63
|
+
versioned by `db/schema.rb` like any other DDL. The gem only records
|
|
64
|
+
the trigger's name and the equivalent rebuild SQL.
|
|
65
|
+
- `projects_onto :assoc, mode: :trigger, trigger_name: "..." do
|
|
66
|
+
rebuild_sql "..." end` declares a trigger projection. The
|
|
67
|
+
`trigger_name:` keyword is required (a non-empty string); the
|
|
68
|
+
block must call `rebuild_sql "..."` exactly once with a SQL
|
|
69
|
+
string containing the `:target_id` placeholder. Registration
|
|
70
|
+
rejects `via:`, `lock:`, and `permissive:` (none are meaningful
|
|
71
|
+
for `:trigger` mode — the trigger is the contract). `Definition`
|
|
72
|
+
gains a `trigger_name` field, populated only for `:trigger` mode;
|
|
73
|
+
the rebuild SQL is stored in the existing `recompute_sql` slot
|
|
74
|
+
(shared with `:sql` mode — both modes use it the same way).
|
|
75
|
+
- `StandardLedger::Modes::Trigger` strategy — `install!` is a no-op
|
|
76
|
+
marker (trigger projections fire from the database, not Ruby, so
|
|
77
|
+
no `after_create` callback is wired). The strategy class exists
|
|
78
|
+
only to keep `Projector#install_mode_callbacks_for`'s dispatch
|
|
79
|
+
table uniform across modes and to mark the entry class as having
|
|
80
|
+
at least one `:trigger` projection registered.
|
|
81
|
+
- `StandardLedger.rebuild!(EntryClass)` extends to `:trigger`
|
|
82
|
+
projections: the same SQL recompute path as `:sql` mode runs the
|
|
83
|
+
recorded `rebuild_sql` against each target the log references,
|
|
84
|
+
binding `:target_id` to each target's id. `target:` /
|
|
85
|
+
`target_class:` / no-arg scoping works the same way.
|
|
86
|
+
`<prefix>.projection.rebuilt` fires per target with `mode:
|
|
87
|
+
:trigger`. The gem does NOT verify or recreate the trigger
|
|
88
|
+
during `rebuild!` — `standard_ledger:doctor` is the deploy-time
|
|
89
|
+
check.
|
|
90
|
+
- `standard_ledger:doctor` rake task. Iterates every registered
|
|
91
|
+
`:trigger` projection across all loaded entry classes (discovered
|
|
92
|
+
by walking `ActiveRecord::Base.descendants` for classes that
|
|
93
|
+
include `StandardLedger::Projector` and have at least one `:trigger`
|
|
94
|
+
projection). For each, queries `pg_trigger` to confirm the named
|
|
95
|
+
trigger exists in the connected schema. Reports missing triggers
|
|
96
|
+
on stderr with a clear remediation message and exits 1; prints a
|
|
97
|
+
success message and exits 0 otherwise. **Postgres-only** — the
|
|
98
|
+
task queries `pg_trigger` directly and will raise on a non-Postgres
|
|
99
|
+
connection. SQLite has no comparable per-statement trigger
|
|
100
|
+
introspection that fits this gem's contract; the only adopter
|
|
101
|
+
today is nutripod-web (Postgres). The task is auto-loaded via
|
|
102
|
+
`Engine.rake_tasks` so `bin/rails -T standard_ledger` shows it
|
|
103
|
+
immediately after the gem is installed.
|
|
104
|
+
- Integration spec for `:trigger` mode
|
|
105
|
+
(`spec/standard_ledger/trigger_integration_spec.rb`) covers DSL
|
|
106
|
+
registration validation (missing `trigger_name`, missing block,
|
|
107
|
+
empty block, `via:` / `lock:` / `permissive:` rejection,
|
|
108
|
+
`:target_id` placeholder enforcement, double `rebuild_sql` call),
|
|
109
|
+
the strategy's no-callback contract (creating an entry does NOT
|
|
110
|
+
mutate the target via Ruby — that's the trigger's job in
|
|
111
|
+
production), and `StandardLedger.rebuild!` for both single-target
|
|
112
|
+
and walk-the-log scoping with the `<prefix>.projection.rebuilt`
|
|
113
|
+
event firing with `mode: :trigger`.
|
|
114
|
+
- Doctor task spec
|
|
115
|
+
(`spec/standard_ledger/tasks/doctor_spec.rb`) mocks the
|
|
116
|
+
`connection.exec_query` against `pg_trigger` to exercise the
|
|
117
|
+
task's three behaviours (success, failure with exit 1 + stderr
|
|
118
|
+
message, ignoring entry classes without `:trigger` projections)
|
|
119
|
+
without requiring a real Postgres database.
|
|
120
|
+
- Integration spec for `:async` mode
|
|
121
|
+
(`spec/standard_ledger/async_integration_spec.rb`) covers DSL
|
|
122
|
+
registration validation, after-commit enqueue, multi-target fan-out,
|
|
123
|
+
nil-FK skip (both at enqueue and at perform), the `with_modes`
|
|
124
|
+
inline override, the `default_async_job` swap, retry-on-failure with
|
|
125
|
+
attempt-counter telemetry, the `discard_on StandardLedger::Error`
|
|
126
|
+
programmer-error path, and the rebuild path.
|
|
127
|
+
|
|
11
128
|
## [0.2.0] - 2026-05-05
|
|
12
129
|
|
|
13
130
|
### Added
|
|
@@ -255,6 +372,7 @@ roadmap.
|
|
|
255
372
|
and `:trigger` (host-owned, gem records rebuild SQL).
|
|
256
373
|
- `standard_ledger:doctor` rake task (verifies trigger presence, etc.).
|
|
257
374
|
|
|
258
|
-
[Unreleased]: https://github.com/rarebit-one/standard_ledger/compare/v0.
|
|
375
|
+
[Unreleased]: https://github.com/rarebit-one/standard_ledger/compare/v0.3.0...HEAD
|
|
376
|
+
[0.3.0]: https://github.com/rarebit-one/standard_ledger/compare/v0.2.0...v0.3.0
|
|
259
377
|
[0.2.0]: https://github.com/rarebit-one/standard_ledger/compare/v0.1.0...v0.2.0
|
|
260
378
|
[0.1.0]: https://github.com/rarebit-one/standard_ledger/releases/tag/v0.1.0
|
data/README.md
CHANGED
|
@@ -2,10 +2,12 @@
|
|
|
2
2
|
|
|
3
3
|
Immutable journal entries with declarative aggregate projections for Rails apps.
|
|
4
4
|
|
|
5
|
-
> **Status: v0.
|
|
6
|
-
>
|
|
7
|
-
>
|
|
8
|
-
>
|
|
5
|
+
> **Status: v0.3.0** — feature-complete across all five projection modes
|
|
6
|
+
> (`:inline`, `:async`, `:sql`, `:matview`, `:trigger`) plus
|
|
7
|
+
> `StandardLedger.rebuild!` log-replay, `StandardLedger.refresh!` ad-hoc
|
|
8
|
+
> matview refresh, and the `standard_ledger:doctor` rake task. Ready for
|
|
9
|
+
> adoption in luminality-web, fundbright-web, sidekick-web, and
|
|
10
|
+
> nutripod-web. See [`standard_ledger-design.md`](https://github.com/rarebit-one/standard_ledger/blob/main/standard_ledger-design.md)
|
|
9
11
|
> for the full design and rollout plan.
|
|
10
12
|
|
|
11
13
|
## What it is
|
|
@@ -101,6 +103,50 @@ mid-loop are not unwound. Block-form (delta) projections raise
|
|
|
101
103
|
`NotRebuildable` because they cannot be reconstructed from the log
|
|
102
104
|
without a host-supplied recompute path.
|
|
103
105
|
|
|
106
|
+
For projections too expensive or stateful to run inside the entry's
|
|
107
|
+
transaction (jsonb rebuild, multi-row aggregate), use `mode: :async` —
|
|
108
|
+
the strategy enqueues `StandardLedger::ProjectionJob` from
|
|
109
|
+
`after_create_commit`, and the job runs `target.with_lock { projector.apply(target, entry) }`
|
|
110
|
+
on the configured ActiveJob backend:
|
|
111
|
+
|
|
112
|
+
```ruby
|
|
113
|
+
class Orders::FulfillableProjector < StandardLedger::Projection
|
|
114
|
+
# Recompute the jsonb balance from the full log inside with_lock.
|
|
115
|
+
# `:async` projectors must be retry-safe — async retries can run
|
|
116
|
+
# `apply` more than once, so block-form per-kind handlers
|
|
117
|
+
# (incrementing counters) are rejected at registration time.
|
|
118
|
+
def apply(order, _entry)
|
|
119
|
+
order.update!(
|
|
120
|
+
fulfillable_balance: order.fulfillment_records.group(:key).sum(:amount)
|
|
121
|
+
)
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
def rebuild(order)
|
|
125
|
+
apply(order, nil)
|
|
126
|
+
end
|
|
127
|
+
end
|
|
128
|
+
|
|
129
|
+
class FulfillmentRecord < ApplicationRecord
|
|
130
|
+
include StandardLedger::Entry
|
|
131
|
+
include StandardLedger::Projector
|
|
132
|
+
|
|
133
|
+
belongs_to :order
|
|
134
|
+
|
|
135
|
+
ledger_entry kind: :action, idempotency_key: :external_ref, scope: :organisation_id
|
|
136
|
+
|
|
137
|
+
projects_onto :order, mode: :async, via: Orders::FulfillableProjector
|
|
138
|
+
end
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
Retries are capped by `Config#default_async_retries` (default 3); the
|
|
142
|
+
job emits `<prefix>.projection.applied` and `<prefix>.projection.failed`
|
|
143
|
+
events with an additional `attempt:` key so subscribers can tell
|
|
144
|
+
first-try success from retry success. Tests can force async projections
|
|
145
|
+
to run inline via `StandardLedger.with_modes(FulfillmentRecord => :inline) { ... }`
|
|
146
|
+
— the strategy short-circuits the enqueue and runs the projector
|
|
147
|
+
synchronously inside `with_lock`, so end-to-end coverage works without a
|
|
148
|
+
job runner.
|
|
149
|
+
|
|
104
150
|
For projections expressible as a single `UPDATE` over an aggregate of the
|
|
105
151
|
log, use `mode: :sql` — no Ruby-side handlers, no AR object loads, just
|
|
106
152
|
a recompute statement that runs in the entry's `after_create`:
|
|
@@ -132,6 +178,58 @@ SQL is the entire contract — `:sql` projections are naturally
|
|
|
132
178
|
rebuildable: `StandardLedger.rebuild!` runs the same statement against
|
|
133
179
|
every target the log references.
|
|
134
180
|
|
|
181
|
+
When the host **already has** a database trigger that updates the
|
|
182
|
+
projection target on every entry INSERT, register it with `mode: :trigger`
|
|
183
|
+
so the gem records the trigger's name and the equivalent rebuild SQL —
|
|
184
|
+
without taking ownership of the trigger DDL. The host writes the trigger
|
|
185
|
+
in a Rails migration; the gem only consumes the metadata.
|
|
186
|
+
|
|
187
|
+
```ruby
|
|
188
|
+
class InventoryRecord < ApplicationRecord
|
|
189
|
+
include StandardLedger::Entry
|
|
190
|
+
include StandardLedger::Projector
|
|
191
|
+
|
|
192
|
+
belongs_to :sku
|
|
193
|
+
|
|
194
|
+
ledger_entry kind: :action, idempotency_key: :serial_no, scope: :organisation_id
|
|
195
|
+
|
|
196
|
+
projects_onto :sku, mode: :trigger,
|
|
197
|
+
trigger_name: "inventory_records_apply_to_skus" do
|
|
198
|
+
rebuild_sql <<~SQL
|
|
199
|
+
UPDATE skus SET
|
|
200
|
+
total_count = c.total_count,
|
|
201
|
+
reserved_count = c.reserved_count,
|
|
202
|
+
free_count = c.total_count - c.reserved_count
|
|
203
|
+
FROM (
|
|
204
|
+
SELECT sku_id,
|
|
205
|
+
COUNT(*) FILTER (WHERE action IN ('grant','adjust_in')) AS total_count,
|
|
206
|
+
COUNT(*) FILTER (WHERE action = 'reserve') AS reserved_count
|
|
207
|
+
FROM inventory_records
|
|
208
|
+
WHERE sku_id = :target_id
|
|
209
|
+
GROUP BY sku_id
|
|
210
|
+
) c
|
|
211
|
+
WHERE skus.id = :target_id AND skus.id = c.sku_id
|
|
212
|
+
SQL
|
|
213
|
+
end
|
|
214
|
+
end
|
|
215
|
+
```
|
|
216
|
+
|
|
217
|
+
The trigger continues to fire on every `INSERT` (the host owns the DDL);
|
|
218
|
+
the gem records the trigger name + rebuild SQL for two purposes:
|
|
219
|
+
|
|
220
|
+
- `StandardLedger.rebuild!(InventoryRecord, target: sku)` runs the
|
|
221
|
+
recorded `rebuild_sql` with `:target_id` bound to each target's id.
|
|
222
|
+
- `bin/rails standard_ledger:doctor` verifies that every registered
|
|
223
|
+
`:trigger` projection's named trigger exists in the connected schema
|
|
224
|
+
(queries `pg_trigger`). Run this as a deploy-time check — migration
|
|
225
|
+
drift surfaces immediately rather than at runtime. **Postgres-only**;
|
|
226
|
+
the task raises on non-Postgres connections.
|
|
227
|
+
|
|
228
|
+
Registration rejects `via:`, `lock:`, and `permissive:` (none are
|
|
229
|
+
meaningful when the trigger itself is the contract). The `trigger_name:`
|
|
230
|
+
keyword is required; the block must call `rebuild_sql "..."` exactly
|
|
231
|
+
once with a SQL string containing the `:target_id` placeholder.
|
|
232
|
+
|
|
135
233
|
Refresh a `:matview` projection ad-hoc when the host needs immediate
|
|
136
234
|
read-your-write semantics (e.g. at the end of a draw operation, before
|
|
137
235
|
the next scheduled refresh would otherwise show stale counts):
|
|
@@ -11,8 +11,9 @@ module StandardLedger
|
|
|
11
11
|
# `c.default_async_job = Orders::FulfillableProjectionJob`.
|
|
12
12
|
attr_accessor :default_async_job
|
|
13
13
|
|
|
14
|
-
#
|
|
15
|
-
# Default: 3.
|
|
14
|
+
# Total attempts (including the first) for an `:async` projection before
|
|
15
|
+
# the failure is propagated. Default: 3 (one initial run + two retries).
|
|
16
|
+
# Matches ActiveJob's `retry_on attempts:` semantics.
|
|
16
17
|
attr_accessor :default_async_retries
|
|
17
18
|
|
|
18
19
|
# Scheduler backend for `:matview` refresh jobs. One of
|
|
@@ -15,5 +15,12 @@ module StandardLedger
|
|
|
15
15
|
# no-op — the gem emits events but ships no internal subscribers; hosts
|
|
16
16
|
# subscribe directly via ActiveSupport::Notifications.subscribe.
|
|
17
17
|
end
|
|
18
|
+
|
|
19
|
+
# Engines auto-discover `lib/tasks/*.rake` in most Rails versions, but
|
|
20
|
+
# we register explicitly for defence-in-depth. Hosts get
|
|
21
|
+
# `standard_ledger:doctor` available under `bin/rails -T standard_ledger`.
|
|
22
|
+
rake_tasks do
|
|
23
|
+
load File.expand_path("../tasks/standard_ledger.rake", __dir__)
|
|
24
|
+
end
|
|
18
25
|
end
|
|
19
26
|
end
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
require "active_job"
|
|
2
|
+
|
|
3
|
+
module StandardLedger
|
|
4
|
+
# ActiveJob class that runs a single `:async`-mode projection after the
|
|
5
|
+
# entry's outer transaction has committed. Enqueued by
|
|
6
|
+
# `StandardLedger::Modes::Async` from an `after_create_commit` callback.
|
|
7
|
+
#
|
|
8
|
+
# The job resolves the projection definition by `target_association` (the
|
|
9
|
+
# only stable handle — the Definition struct doesn't serialize cleanly
|
|
10
|
+
# through ActiveJob), looks up the target via the entry's `belongs_to`
|
|
11
|
+
# setter, and runs `target.with_lock { projector_class.new.apply(target,
|
|
12
|
+
# entry) }`. The projector must be class-form (`via: ProjectorClass`) and
|
|
13
|
+
# should recompute the aggregate from the log inside `apply` for retry
|
|
14
|
+
# safety — async projections can run more than once when the job retries.
|
|
15
|
+
#
|
|
16
|
+
# Retries are configurable per-projection via subclassing this job and
|
|
17
|
+
# overriding `retry_on`, or globally via `Config#default_async_retries`
|
|
18
|
+
# (default 3). When retries are exhausted, ActiveJob's dead-letter behavior
|
|
19
|
+
# takes over — the job still emits `<prefix>.projection.failed` on every
|
|
20
|
+
# attempt so subscribers see the full retry history.
|
|
21
|
+
#
|
|
22
|
+
# Notification payloads include `attempt:` (drawn from ActiveJob's
|
|
23
|
+
# `executions` accessor — 1 on first attempt, increments per retry) so
|
|
24
|
+
# subscribers can distinguish first-try success from retry-success.
|
|
25
|
+
class ProjectionJob < ::ActiveJob::Base
|
|
26
|
+
# Hand-rolled retry path so the attempt cap reads
|
|
27
|
+
# `Config#default_async_retries` at perform time. ActiveJob's
|
|
28
|
+
# `retry_on attempts:` requires a constant Integer (or `:unlimited`)
|
|
29
|
+
# and is captured at class-definition time — that's incompatible with
|
|
30
|
+
# the gem's pattern of letting hosts reconfigure `default_async_retries`
|
|
31
|
+
# in their initializer (or specs flipping it temporarily). We rescue
|
|
32
|
+
# `StandardError` and re-enqueue manually until the cap is reached.
|
|
33
|
+
rescue_from(StandardError) do |error|
|
|
34
|
+
attempts = StandardLedger.config.default_async_retries
|
|
35
|
+
if executions < attempts
|
|
36
|
+
# Compute a polynomial backoff inline. Mirrors ActiveJob's
|
|
37
|
+
# `:polynomially_longer` algorithm (`executions**4 + 2`) without
|
|
38
|
+
# depending on its private `determine_delay` API.
|
|
39
|
+
delay = (executions**4) + 2
|
|
40
|
+
retry_job(wait: delay, error: error)
|
|
41
|
+
else
|
|
42
|
+
raise error
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
# Discard programmer errors immediately — they're deterministic and
|
|
47
|
+
# retrying just burns the budget. `StandardLedger::Error` is raised by
|
|
48
|
+
# `#perform` on missing/renamed projection definitions and similar
|
|
49
|
+
# bookkeeping mistakes; the next attempt would raise the same error.
|
|
50
|
+
# Declared AFTER the `rescue_from(StandardError)` block above because
|
|
51
|
+
# `ActiveSupport::Rescuable` searches handlers from the most-recently-
|
|
52
|
+
# registered first — so this more-specific `StandardLedger::Error`
|
|
53
|
+
# handler wins over the catch-all retry path for its matching errors.
|
|
54
|
+
discard_on StandardLedger::Error
|
|
55
|
+
|
|
56
|
+
def perform(entry, target_association)
|
|
57
|
+
definition = entry.class.standard_ledger_projections.find { |d|
|
|
58
|
+
d.mode == :async && d.target_association == target_association.to_sym
|
|
59
|
+
}
|
|
60
|
+
if definition.nil?
|
|
61
|
+
raise StandardLedger::Error,
|
|
62
|
+
"no :async projection #{target_association.inspect} on #{entry.class.name}"
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
target = entry.public_send(definition.target_association)
|
|
66
|
+
return if target.nil?
|
|
67
|
+
|
|
68
|
+
prefix = StandardLedger.config.notification_namespace
|
|
69
|
+
started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
70
|
+
|
|
71
|
+
begin
|
|
72
|
+
target.with_lock do
|
|
73
|
+
definition.projector_class.new.apply(target, entry)
|
|
74
|
+
end
|
|
75
|
+
rescue StandardError => e
|
|
76
|
+
duration_ms = (Process.clock_gettime(Process::CLOCK_MONOTONIC) - started) * 1000.0
|
|
77
|
+
StandardLedger::EventEmitter.emit(
|
|
78
|
+
"#{prefix}.projection.failed",
|
|
79
|
+
entry: entry, target: target, projection: definition.target_association,
|
|
80
|
+
mode: :async, error: e, duration_ms: duration_ms,
|
|
81
|
+
attempt: executions
|
|
82
|
+
)
|
|
83
|
+
raise
|
|
84
|
+
end
|
|
85
|
+
|
|
86
|
+
duration_ms = (Process.clock_gettime(Process::CLOCK_MONOTONIC) - started) * 1000.0
|
|
87
|
+
StandardLedger::EventEmitter.emit(
|
|
88
|
+
"#{prefix}.projection.applied",
|
|
89
|
+
entry: entry, target: target, projection: definition.target_association,
|
|
90
|
+
mode: :async, duration_ms: duration_ms, attempt: executions
|
|
91
|
+
)
|
|
92
|
+
end
|
|
93
|
+
end
|
|
94
|
+
end
|
|
@@ -0,0 +1,141 @@
|
|
|
1
|
+
module StandardLedger
|
|
2
|
+
module Modes
|
|
3
|
+
# `:async` mode: applies the projection in a background job enqueued
|
|
4
|
+
# from the entry's `after_create_commit` callback. The job runs after
|
|
5
|
+
# the outer transaction has committed, so the entry is durable before
|
|
6
|
+
# the projection runs — and a job failure does NOT roll back the entry.
|
|
7
|
+
#
|
|
8
|
+
# Used when the projection is too expensive or stateful for the entry's
|
|
9
|
+
# transaction (jsonb rebuild, multi-row aggregate). The canonical
|
|
10
|
+
# example is nutripod's `Order#payable_balance` / `Order#fulfillable_balance`
|
|
11
|
+
# jsonb columns, which need to be recomputed from every PaymentRecord /
|
|
12
|
+
# FulfillmentRecord against the order — work that's safe to defer past
|
|
13
|
+
# the originating transaction.
|
|
14
|
+
#
|
|
15
|
+
# Class-form only: `:async` projections must declare `via: ProjectorClass`,
|
|
16
|
+
# whose `apply(target, entry)` should recompute from the log inside
|
|
17
|
+
# `with_lock` rather than apply a delta. Block-form per-kind handlers
|
|
18
|
+
# aren't safe under retry — incrementing a counter twice on retry is a
|
|
19
|
+
# silent data corruption bug — so block-form is rejected at registration
|
|
20
|
+
# time (see `Projector#projects_onto`).
|
|
21
|
+
#
|
|
22
|
+
# The strategy installs an `after_create_commit` callback once per entry
|
|
23
|
+
# class. The callback walks every `:async`-mode projection registered on
|
|
24
|
+
# the class and enqueues `StandardLedger::ProjectionJob` per (entry,
|
|
25
|
+
# projection) pair, honoring the optional `if:` guard.
|
|
26
|
+
#
|
|
27
|
+
# ## with_modes interop
|
|
28
|
+
#
|
|
29
|
+
# `StandardLedger.with_modes(EntryClass => :inline) { ... }` forces async
|
|
30
|
+
# projections to run synchronously inside the block — useful in unit
|
|
31
|
+
# specs that want end-to-end coverage without standing up a job runner.
|
|
32
|
+
# Inline-override mode skips the enqueue and runs `target.with_lock {
|
|
33
|
+
# projector.apply(target, entry) }` directly. The override is read in
|
|
34
|
+
# `#call`; specs can capture the empty job queue via `have_enqueued_job`
|
|
35
|
+
# / `perform_enqueued_jobs` and still observe the projection's effects.
|
|
36
|
+
class Async
|
|
37
|
+
# Install the `after_create_commit` callback on `entry_class` exactly
|
|
38
|
+
# once. Subsequent calls (e.g. when a second `:async` projection is
|
|
39
|
+
# added later in the class body) are no-ops — the same callback
|
|
40
|
+
# handles all `:async` projections registered on the class.
|
|
41
|
+
#
|
|
42
|
+
# @param entry_class [Class] the host entry class.
|
|
43
|
+
# @return [void]
|
|
44
|
+
# @raise [ArgumentError] when `entry_class` is not ActiveRecord-backed
|
|
45
|
+
# (no `after_create_commit` hook available). `:async` mode requires
|
|
46
|
+
# AR transactional callbacks; non-AR entry classes can't dispatch
|
|
47
|
+
# the post-commit enqueue.
|
|
48
|
+
def self.install!(entry_class)
|
|
49
|
+
return if entry_class.instance_variable_get(:@_standard_ledger_async_installed)
|
|
50
|
+
|
|
51
|
+
unless entry_class.respond_to?(:after_create_commit)
|
|
52
|
+
raise ArgumentError,
|
|
53
|
+
"Modes::Async requires an ActiveRecord-backed entry class on " \
|
|
54
|
+
"#{entry_class.name || entry_class.inspect} — the entry class must inherit " \
|
|
55
|
+
"from ActiveRecord::Base. Use :inline mode for plain-Ruby entry classes."
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
entry_class.after_create_commit { StandardLedger::Modes::Async.new.call(self) }
|
|
59
|
+
entry_class.instance_variable_set(:@_standard_ledger_async_installed, true)
|
|
60
|
+
end
|
|
61
|
+
|
|
62
|
+
# Enqueue `ProjectionJob` for every `:async` projection registered on
|
|
63
|
+
# the entry's class. Called from the `after_create_commit` callback
|
|
64
|
+
# installed by `.install!`.
|
|
65
|
+
#
|
|
66
|
+
# Honors the optional `if:` guard (skips enqueue when guard returns
|
|
67
|
+
# false) and `StandardLedger.mode_override_for(entry_class)` (when set
|
|
68
|
+
# to `:inline`, runs the projection synchronously inside `with_lock`
|
|
69
|
+
# instead of enqueueing).
|
|
70
|
+
#
|
|
71
|
+
# A nil target at enqueue time skips silently — the entry's FK was
|
|
72
|
+
# unset for this projection's association, so there's nothing to
|
|
73
|
+
# project onto. (The job has its own nil-target guard; this short-
|
|
74
|
+
# circuit just avoids the wasted enqueue.)
|
|
75
|
+
#
|
|
76
|
+
# @param entry [ActiveRecord::Base] the just-committed entry.
|
|
77
|
+
# @return [void]
|
|
78
|
+
def call(entry)
|
|
79
|
+
definitions = async_definitions_for(entry.class)
|
|
80
|
+
return if definitions.empty?
|
|
81
|
+
|
|
82
|
+
override = StandardLedger.mode_override_for(entry.class)
|
|
83
|
+
|
|
84
|
+
definitions.each do |definition|
|
|
85
|
+
next if definition.guard && !entry.instance_exec(&definition.guard)
|
|
86
|
+
|
|
87
|
+
if override == :inline
|
|
88
|
+
run_inline(entry, definition)
|
|
89
|
+
else
|
|
90
|
+
target = entry.public_send(definition.target_association)
|
|
91
|
+
next if target.nil?
|
|
92
|
+
|
|
93
|
+
job_class = StandardLedger.config.default_async_job || StandardLedger::ProjectionJob
|
|
94
|
+
job_class.perform_later(entry, definition.target_association.to_s)
|
|
95
|
+
end
|
|
96
|
+
end
|
|
97
|
+
end
|
|
98
|
+
|
|
99
|
+
private
|
|
100
|
+
|
|
101
|
+
def async_definitions_for(entry_class)
|
|
102
|
+
return [] unless entry_class.respond_to?(:standard_ledger_projections_for)
|
|
103
|
+
|
|
104
|
+
entry_class.standard_ledger_projections_for(:async)
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
# `with_modes(EntryClass => :inline)` short-circuit: run the projector
|
|
108
|
+
# synchronously inside `with_lock`, mirroring the job's behavior
|
|
109
|
+
# without the enqueue. Skips silently when the target is nil so the
|
|
110
|
+
# nil-FK contract matches the enqueue path.
|
|
111
|
+
def run_inline(entry, definition)
|
|
112
|
+
target = entry.public_send(definition.target_association)
|
|
113
|
+
return if target.nil?
|
|
114
|
+
|
|
115
|
+
prefix = StandardLedger.config.notification_namespace
|
|
116
|
+
started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
|
117
|
+
|
|
118
|
+
begin
|
|
119
|
+
target.with_lock do
|
|
120
|
+
definition.projector_class.new.apply(target, entry)
|
|
121
|
+
end
|
|
122
|
+
rescue StandardError => e
|
|
123
|
+
duration_ms = (Process.clock_gettime(Process::CLOCK_MONOTONIC) - started) * 1000.0
|
|
124
|
+
StandardLedger::EventEmitter.emit(
|
|
125
|
+
"#{prefix}.projection.failed",
|
|
126
|
+
entry: entry, target: target, projection: definition.target_association,
|
|
127
|
+
mode: :async, error: e, duration_ms: duration_ms, attempt: 1
|
|
128
|
+
)
|
|
129
|
+
raise
|
|
130
|
+
end
|
|
131
|
+
|
|
132
|
+
duration_ms = (Process.clock_gettime(Process::CLOCK_MONOTONIC) - started) * 1000.0
|
|
133
|
+
StandardLedger::EventEmitter.emit(
|
|
134
|
+
"#{prefix}.projection.applied",
|
|
135
|
+
entry: entry, target: target, projection: definition.target_association,
|
|
136
|
+
mode: :async, duration_ms: duration_ms, attempt: 1
|
|
137
|
+
)
|
|
138
|
+
end
|
|
139
|
+
end
|
|
140
|
+
end
|
|
141
|
+
end
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
module StandardLedger
|
|
2
|
+
module Modes
|
|
3
|
+
# `:trigger` mode: the host owns a database trigger (created in a Rails
|
|
4
|
+
# migration) that updates the projection target's columns on every
|
|
5
|
+
# entry INSERT. The gem records the trigger's name and the equivalent
|
|
6
|
+
# rebuild SQL, but does **not** create or manage the trigger itself —
|
|
7
|
+
# giving a Ruby DSL the power to install/replace triggers is a deploy
|
|
8
|
+
# footgun (silent re-creation on `db:schema:load` against a non-empty
|
|
9
|
+
# DB), and triggers are versioned by `db/schema.rb` like any other DDL.
|
|
10
|
+
#
|
|
11
|
+
# Two consumers read the recorded metadata:
|
|
12
|
+
#
|
|
13
|
+
# - `StandardLedger.rebuild!` runs the rebuild SQL when invoked,
|
|
14
|
+
# binding `:target_id` to each target the log references. The same
|
|
15
|
+
# recompute path as `:sql` mode — the only difference is that the
|
|
16
|
+
# after-create application is performed by the database trigger
|
|
17
|
+
# rather than a Ruby callback.
|
|
18
|
+
# - The `standard_ledger:doctor` rake task verifies that the named
|
|
19
|
+
# trigger exists in the connected schema. Migration drift (a missing
|
|
20
|
+
# or renamed trigger) is caught at deploy time, not at runtime.
|
|
21
|
+
#
|
|
22
|
+
# Unlike `:inline` and `:sql`, this strategy does NOT install an
|
|
23
|
+
# `after_create` callback — the trigger fires from the database. The
|
|
24
|
+
# `install!` no-op only marks the entry class as having at least one
|
|
25
|
+
# `:trigger` projection registered (mirrors the `Modes::Matview` shape).
|
|
26
|
+
#
|
|
27
|
+
# See `standard_ledger-design.md` §5.3.4 for the full contract.
|
|
28
|
+
class Trigger
|
|
29
|
+
# Mark the entry class as having at least one `:trigger` projection
|
|
30
|
+
# registered. The actual trigger metadata lives on the entry class's
|
|
31
|
+
# `standard_ledger_projections` array; this method exists only to
|
|
32
|
+
# mirror the `Modes::*.install!` shape so `Projector#install_mode_callbacks_for`
|
|
33
|
+
# can dispatch uniformly across modes.
|
|
34
|
+
#
|
|
35
|
+
# No `after_create` callback is installed — the trigger runs in the
|
|
36
|
+
# database, not Ruby. Idempotent across multiple `:trigger`
|
|
37
|
+
# projections on the same entry class.
|
|
38
|
+
#
|
|
39
|
+
# @param entry_class [Class] the host entry class (unused — kept for
|
|
40
|
+
# parity with the other strategy classes).
|
|
41
|
+
# @return [void]
|
|
42
|
+
def self.install!(_entry_class)
|
|
43
|
+
# Intentionally empty: trigger projections fire from the DB, so
|
|
44
|
+
# there's no Ruby-side callback to wire. The DSL has already
|
|
45
|
+
# captured `trigger_name` and `recompute_sql` on the Definition;
|
|
46
|
+
# `rebuild!` and the `doctor` rake task consume those directly.
|
|
47
|
+
end
|
|
48
|
+
end
|
|
49
|
+
end
|
|
50
|
+
end
|
|
@@ -34,7 +34,7 @@ module StandardLedger
|
|
|
34
34
|
# projections; they're `nil` for every other mode.
|
|
35
35
|
Definition = Struct.new(
|
|
36
36
|
:target_association, :mode, :projector_class, :handlers, :guard, :lock, :permissive,
|
|
37
|
-
:recompute_sql, :view, :refresh_options, :options,
|
|
37
|
+
:recompute_sql, :trigger_name, :view, :refresh_options, :options,
|
|
38
38
|
keyword_init: true
|
|
39
39
|
)
|
|
40
40
|
|
|
@@ -62,12 +62,142 @@ module StandardLedger
|
|
|
62
62
|
# metadata, e.g. `{ every: 5.minutes, concurrently: true }`. The
|
|
63
63
|
# gem records this on the Definition for hosts to read when wiring
|
|
64
64
|
# their scheduler; the gem does NOT auto-schedule.
|
|
65
|
+
# @param trigger_name [String, Symbol, nil] for `mode: :trigger` only —
|
|
66
|
+
# the name of the host-owned database trigger. Required when mode is
|
|
67
|
+
# `:trigger`; ignored otherwise. The gem does NOT create or manage
|
|
68
|
+
# the trigger; it records the name so `standard_ledger:doctor` can
|
|
69
|
+
# verify the trigger's presence in the connected schema.
|
|
65
70
|
# @yield optional block-DSL form: register per-kind handlers via
|
|
66
71
|
# `on(:kind) { |target, entry| ... }`. Not allowed for `mode: :matview`.
|
|
67
72
|
# @return [Definition] the registered projection.
|
|
68
|
-
def projects_onto(target_association, mode:, via: nil, if: nil, lock: nil, permissive: false, view: nil, refresh: nil, **options, &block)
|
|
73
|
+
def projects_onto(target_association, mode:, via: nil, if: nil, lock: nil, permissive: false, view: nil, refresh: nil, trigger_name: nil, **options, &block)
|
|
69
74
|
guard = binding.local_variable_get(:if) # `if:` is a reserved keyword
|
|
70
75
|
|
|
76
|
+
if mode == :async
|
|
77
|
+
if block
|
|
78
|
+
raise ArgumentError,
|
|
79
|
+
"projects_onto :#{target_association} mode: :async does not accept a block — " \
|
|
80
|
+
"use `via: ProjectorClass` whose `apply(target, entry)` recomputes from the log " \
|
|
81
|
+
"inside `with_lock` for retry-safety. Block-form per-kind handlers aren't safe " \
|
|
82
|
+
"under async retry."
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
if via.nil?
|
|
86
|
+
raise ArgumentError,
|
|
87
|
+
"projects_onto :#{target_association} mode: :async requires `via: ProjectorClass` " \
|
|
88
|
+
"(class-form only — see the registration error message for block-form for the why)"
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
unless lock.nil?
|
|
92
|
+
raise ArgumentError,
|
|
93
|
+
"projects_onto :#{target_association} got `lock:` with mode: :async; " \
|
|
94
|
+
":async always wraps the projector in `target.with_lock` for retry-safety, " \
|
|
95
|
+
"so the option is redundant. Drop it."
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
if permissive
|
|
99
|
+
raise ArgumentError,
|
|
100
|
+
"projects_onto :#{target_association} got `permissive: true` with mode: :async; " \
|
|
101
|
+
"`permissive:` is only meaningful with the block form, and `:async` mode rejects " \
|
|
102
|
+
"block-form for retry-safety reasons"
|
|
103
|
+
end
|
|
104
|
+
|
|
105
|
+
definition = Definition.new(
|
|
106
|
+
target_association: target_association,
|
|
107
|
+
mode: mode,
|
|
108
|
+
projector_class: via,
|
|
109
|
+
handlers: {},
|
|
110
|
+
guard: guard,
|
|
111
|
+
lock: lock,
|
|
112
|
+
permissive: permissive,
|
|
113
|
+
recompute_sql: nil,
|
|
114
|
+
trigger_name: nil,
|
|
115
|
+
view: nil,
|
|
116
|
+
refresh_options: nil,
|
|
117
|
+
options: options
|
|
118
|
+
)
|
|
119
|
+
|
|
120
|
+
self.standard_ledger_projections = standard_ledger_projections + [ definition ]
|
|
121
|
+
install_mode_callbacks_for(definition)
|
|
122
|
+
return definition
|
|
123
|
+
end
|
|
124
|
+
|
|
125
|
+
if mode == :trigger
|
|
126
|
+
if via
|
|
127
|
+
raise ArgumentError,
|
|
128
|
+
"projects_onto :#{target_association} got `via:` with mode: :trigger; " \
|
|
129
|
+
"the trigger is the contract — there is no projector class. The host " \
|
|
130
|
+
"owns the trigger via a Rails migration; the gem only records the " \
|
|
131
|
+
"trigger's name and rebuild SQL"
|
|
132
|
+
end
|
|
133
|
+
|
|
134
|
+
unless lock.nil?
|
|
135
|
+
raise ArgumentError,
|
|
136
|
+
"projects_onto :#{target_association} got `lock:` with mode: :trigger; " \
|
|
137
|
+
"`lock:` is not supported by :trigger mode — the database trigger handles " \
|
|
138
|
+
"concurrency atomically"
|
|
139
|
+
end
|
|
140
|
+
|
|
141
|
+
if permissive
|
|
142
|
+
raise ArgumentError,
|
|
143
|
+
"projects_onto :#{target_association} got `permissive:` with mode: :trigger; " \
|
|
144
|
+
"`permissive:` is not supported by :trigger mode — the trigger doesn't " \
|
|
145
|
+
"dispatch through per-kind handlers"
|
|
146
|
+
end
|
|
147
|
+
|
|
148
|
+
if guard
|
|
149
|
+
raise ArgumentError,
|
|
150
|
+
"projects_onto :#{target_association} got `if:` with mode: :trigger; " \
|
|
151
|
+
"`if:` is not supported by :trigger mode — the database trigger fires " \
|
|
152
|
+
"unconditionally from the DB on INSERT, so a Ruby-side guard would " \
|
|
153
|
+
"silently never run"
|
|
154
|
+
end
|
|
155
|
+
|
|
156
|
+
if trigger_name.nil? || trigger_name.to_s.empty?
|
|
157
|
+
raise ArgumentError,
|
|
158
|
+
"projects_onto :#{target_association} requires `trigger_name: \"...\"` for mode: :trigger; " \
|
|
159
|
+
"the gem records the trigger's name so `standard_ledger:doctor` can verify its presence"
|
|
160
|
+
end
|
|
161
|
+
|
|
162
|
+
unless block
|
|
163
|
+
raise ArgumentError,
|
|
164
|
+
"projects_onto :#{target_association} requires a block with `rebuild_sql \"...\"` for mode: :trigger"
|
|
165
|
+
end
|
|
166
|
+
|
|
167
|
+
dsl = TriggerDsl.new
|
|
168
|
+
dsl.instance_eval(&block)
|
|
169
|
+
|
|
170
|
+
if dsl.rebuild_sql_text.nil?
|
|
171
|
+
raise ArgumentError,
|
|
172
|
+
"projects_onto :#{target_association} block is empty; mode: :trigger requires a `rebuild_sql \"...\"` clause"
|
|
173
|
+
end
|
|
174
|
+
|
|
175
|
+
unless dsl.rebuild_sql_text.include?(":target_id")
|
|
176
|
+
raise ArgumentError,
|
|
177
|
+
"projects_onto :#{target_association} rebuild SQL must include the `:target_id` placeholder; " \
|
|
178
|
+
"it's bound to each target's id when `StandardLedger.rebuild!` walks the log"
|
|
179
|
+
end
|
|
180
|
+
|
|
181
|
+
definition = Definition.new(
|
|
182
|
+
target_association: target_association,
|
|
183
|
+
mode: mode,
|
|
184
|
+
projector_class: nil,
|
|
185
|
+
handlers: {},
|
|
186
|
+
guard: guard,
|
|
187
|
+
lock: lock,
|
|
188
|
+
permissive: permissive,
|
|
189
|
+
recompute_sql: dsl.rebuild_sql_text,
|
|
190
|
+
trigger_name: trigger_name.to_s,
|
|
191
|
+
view: nil,
|
|
192
|
+
refresh_options: nil,
|
|
193
|
+
options: options
|
|
194
|
+
)
|
|
195
|
+
|
|
196
|
+
self.standard_ledger_projections = standard_ledger_projections + [ definition ]
|
|
197
|
+
install_mode_callbacks_for(definition)
|
|
198
|
+
return definition
|
|
199
|
+
end
|
|
200
|
+
|
|
71
201
|
if mode == :sql
|
|
72
202
|
if via
|
|
73
203
|
raise ArgumentError,
|
|
@@ -118,6 +248,7 @@ module StandardLedger
|
|
|
118
248
|
lock: lock,
|
|
119
249
|
permissive: permissive,
|
|
120
250
|
recompute_sql: dsl.recompute_sql,
|
|
251
|
+
trigger_name: nil,
|
|
121
252
|
view: nil,
|
|
122
253
|
refresh_options: nil,
|
|
123
254
|
options: options
|
|
@@ -150,6 +281,7 @@ module StandardLedger
|
|
|
150
281
|
lock: lock,
|
|
151
282
|
permissive: permissive,
|
|
152
283
|
recompute_sql: nil,
|
|
284
|
+
trigger_name: nil,
|
|
153
285
|
view: view.to_s,
|
|
154
286
|
refresh_options: refresh || {},
|
|
155
287
|
options: options
|
|
@@ -200,6 +332,7 @@ module StandardLedger
|
|
|
200
332
|
lock: lock,
|
|
201
333
|
permissive: permissive,
|
|
202
334
|
recompute_sql: nil,
|
|
335
|
+
trigger_name: nil,
|
|
203
336
|
view: nil,
|
|
204
337
|
refresh_options: nil,
|
|
205
338
|
options: options
|
|
@@ -222,10 +355,14 @@ module StandardLedger
|
|
|
222
355
|
case definition.mode
|
|
223
356
|
when :inline
|
|
224
357
|
StandardLedger::Modes::Inline.install!(self)
|
|
358
|
+
when :async
|
|
359
|
+
StandardLedger::Modes::Async.install!(self)
|
|
225
360
|
when :sql
|
|
226
361
|
StandardLedger::Modes::Sql.install!(self)
|
|
227
362
|
when :matview
|
|
228
363
|
StandardLedger::Modes::Matview.install!(self)
|
|
364
|
+
when :trigger
|
|
365
|
+
StandardLedger::Modes::Trigger.install!(self)
|
|
229
366
|
end
|
|
230
367
|
end
|
|
231
368
|
|
|
@@ -275,6 +412,12 @@ module StandardLedger
|
|
|
275
412
|
"the recompute SQL runs through `Modes::Sql#call` directly with no per-kind dispatch"
|
|
276
413
|
end
|
|
277
414
|
|
|
415
|
+
if definition.mode == :trigger
|
|
416
|
+
raise Error,
|
|
417
|
+
"apply_projection! is not supported for mode: :trigger; " \
|
|
418
|
+
"the database trigger fires from the DB on INSERT, not from Ruby"
|
|
419
|
+
end
|
|
420
|
+
|
|
278
421
|
return false if definition.guard && !instance_exec(&definition.guard)
|
|
279
422
|
|
|
280
423
|
target = public_send(definition.target_association)
|
|
@@ -357,5 +500,28 @@ module StandardLedger
|
|
|
357
500
|
@recompute_sql = sql
|
|
358
501
|
end
|
|
359
502
|
end
|
|
503
|
+
|
|
504
|
+
# Internal collector for the `:trigger`-mode block-DSL form. Captures
|
|
505
|
+
# the `rebuild_sql "..."` clause's SQL string. The trigger itself is
|
|
506
|
+
# owned by the host (created in a Rails migration); the gem only
|
|
507
|
+
# records this rebuild SQL so `StandardLedger.rebuild!` can recompute
|
|
508
|
+
# the projection from the log when invoked. Like `:sql` mode, the SQL
|
|
509
|
+
# must be expressible as a single statement with `:target_id` bound
|
|
510
|
+
# from each target's id at rebuild time.
|
|
511
|
+
class TriggerDsl
|
|
512
|
+
attr_reader :rebuild_sql_text
|
|
513
|
+
|
|
514
|
+
def rebuild_sql(sql)
|
|
515
|
+
unless sql.is_a?(String)
|
|
516
|
+
raise ArgumentError, "rebuild_sql requires a SQL string; got #{sql.class}"
|
|
517
|
+
end
|
|
518
|
+
unless @rebuild_sql_text.nil?
|
|
519
|
+
raise ArgumentError,
|
|
520
|
+
"rebuild_sql called more than once in the same projects_onto block; " \
|
|
521
|
+
":trigger mode supports exactly one rebuild_sql clause per projection"
|
|
522
|
+
end
|
|
523
|
+
@rebuild_sql_text = sql
|
|
524
|
+
end
|
|
525
|
+
end
|
|
360
526
|
end
|
|
361
527
|
end
|
data/lib/standard_ledger.rb
CHANGED
|
@@ -14,7 +14,10 @@ require "standard_ledger/projector"
|
|
|
14
14
|
require "standard_ledger/modes/inline"
|
|
15
15
|
require "standard_ledger/modes/sql"
|
|
16
16
|
require "standard_ledger/modes/matview"
|
|
17
|
+
require "standard_ledger/modes/trigger"
|
|
18
|
+
require "standard_ledger/modes/async"
|
|
17
19
|
require "standard_ledger/jobs/matview_refresh_job"
|
|
20
|
+
require "standard_ledger/jobs/projection_job"
|
|
18
21
|
require "standard_ledger/engine" if defined?(::Rails::Engine)
|
|
19
22
|
|
|
20
23
|
# StandardLedger captures the recurring "immutable journal entry → N
|
|
@@ -197,9 +200,18 @@ module StandardLedger
|
|
|
197
200
|
# refresh *is* rebuild. Postgres has no partial-refresh primitive,
|
|
198
201
|
# so `target:` / `target_class:` scope arguments are ignored for
|
|
199
202
|
# `:matview` projections and the full view is always refreshed.
|
|
200
|
-
# - `:
|
|
201
|
-
#
|
|
202
|
-
#
|
|
203
|
+
# - `:sql` and `:trigger` projections rebuild by running their
|
|
204
|
+
# recorded rebuild SQL with `:target_id` bound to each target's
|
|
205
|
+
# id. For `:trigger`, the database trigger fires on entry INSERT;
|
|
206
|
+
# `rebuild!` runs the same logical recompute against each target
|
|
207
|
+
# the log references. The gem does NOT verify or recreate the
|
|
208
|
+
# trigger here — `standard_ledger:doctor` is the deploy-time
|
|
209
|
+
# check for trigger presence.
|
|
210
|
+
# - `:async` projections rebuild via the same per-target semantics as
|
|
211
|
+
# `:inline` (delegates to `definition.projector_class.new.rebuild(target)`).
|
|
212
|
+
# The mode difference is only in the after-create path (in-transaction
|
|
213
|
+
# vs. post-commit job), not in the rebuild path, which always runs
|
|
214
|
+
# synchronously.
|
|
203
215
|
#
|
|
204
216
|
# Atomicity: each (target, projection) pair runs in its own
|
|
205
217
|
# transaction. A failure mid-loop is **not** rolled back — earlier
|
|
@@ -281,7 +293,7 @@ module StandardLedger
|
|
|
281
293
|
validate_rebuildable_projector!(entry_class, definition)
|
|
282
294
|
|
|
283
295
|
each_rebuild_target(entry_class, definition, target: target, batch_size: batch_size) do |t|
|
|
284
|
-
if definition.mode == :sql
|
|
296
|
+
if definition.mode == :sql || definition.mode == :trigger
|
|
285
297
|
rebuild_one_sql(entry_class, definition, t)
|
|
286
298
|
else
|
|
287
299
|
rebuild_one(entry_class, definition, t)
|
|
@@ -420,13 +432,20 @@ module StandardLedger
|
|
|
420
432
|
reflection.klass
|
|
421
433
|
end
|
|
422
434
|
|
|
423
|
-
#
|
|
424
|
-
# log-replay path
|
|
425
|
-
#
|
|
435
|
+
# All five projection modes (`:inline`, `:async`, `:sql`, `:matview`,
|
|
436
|
+
# `:trigger`) implement a log-replay path through this method.
|
|
437
|
+
#
|
|
438
|
+
# `:async` and `:inline` share the same per-target rebuild semantics —
|
|
439
|
+
# both delegate to `definition.projector_class.new.rebuild(target)`.
|
|
440
|
+
# The difference between the two modes is only in the after-create
|
|
441
|
+
# path (in-transaction vs. post-commit job), not in the rebuild path,
|
|
442
|
+
# which always runs synchronously.
|
|
426
443
|
def validate_rebuildable_mode!(entry_class, definition)
|
|
427
444
|
return if definition.mode == :inline
|
|
445
|
+
return if definition.mode == :async
|
|
428
446
|
return if definition.mode == :sql
|
|
429
447
|
return if definition.mode == :matview
|
|
448
|
+
return if definition.mode == :trigger
|
|
430
449
|
|
|
431
450
|
raise StandardLedger::Error,
|
|
432
451
|
"rebuild! does not yet support mode: #{definition.mode.inspect} " \
|
|
@@ -463,10 +482,12 @@ module StandardLedger
|
|
|
463
482
|
# rebuildable should extract a `Projection` subclass and implement
|
|
464
483
|
# `rebuild(target)`.
|
|
465
484
|
def validate_rebuildable_projector!(entry_class, definition)
|
|
466
|
-
# `:sql`
|
|
467
|
-
# no projector class is required
|
|
468
|
-
# registration). Skip the class-form
|
|
485
|
+
# `:sql` and `:trigger` modes carry their rebuild path in the
|
|
486
|
+
# recompute / rebuild SQL itself — no projector class is required
|
|
487
|
+
# (and `via:` is rejected at registration). Skip the class-form
|
|
488
|
+
# preflight checks below.
|
|
469
489
|
return if definition.mode == :sql
|
|
490
|
+
return if definition.mode == :trigger
|
|
470
491
|
|
|
471
492
|
if definition.projector_class.nil?
|
|
472
493
|
raise StandardLedger::NotRebuildable,
|
|
@@ -535,11 +556,12 @@ module StandardLedger
|
|
|
535
556
|
)
|
|
536
557
|
end
|
|
537
558
|
|
|
538
|
-
# `:sql` mode rebuild path: run the
|
|
539
|
-
#
|
|
540
|
-
#
|
|
541
|
-
#
|
|
542
|
-
#
|
|
559
|
+
# `:sql` / `:trigger` mode rebuild path: run the recorded recompute
|
|
560
|
+
# SQL bound to this target's id. For `:sql` mode this is the same
|
|
561
|
+
# statement the `after_create` callback runs; for `:trigger` mode
|
|
562
|
+
# it's the rebuild SQL the host registered (the database trigger
|
|
563
|
+
# itself owns the after-INSERT path). Either way there's no
|
|
564
|
+
# projector class to invoke — the SQL is the entire contract.
|
|
543
565
|
def rebuild_one_sql(entry_class, definition, target)
|
|
544
566
|
target.class.transaction do
|
|
545
567
|
sql = ActiveRecord::Base.sanitize_sql_array([ definition.recompute_sql, { target_id: target.id } ])
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
namespace :standard_ledger do
|
|
2
|
+
# PostgreSQL-only: queries `pg_trigger` directly. The only mode that
|
|
3
|
+
# registers a trigger today is `:trigger`, and the only adopter is
|
|
4
|
+
# nutripod-web (Postgres). SQLite has no comparable per-statement
|
|
5
|
+
# trigger introspection that makes sense for this gem's contract — if
|
|
6
|
+
# a non-Postgres host adopts `:trigger` mode in the future, they'll
|
|
7
|
+
# need to extend this task with a connection-adapter dispatch. The
|
|
8
|
+
# task is loud about its Postgres assumption: a `pg_trigger` query
|
|
9
|
+
# against a non-Postgres connection will raise, which is preferable
|
|
10
|
+
# to silently passing.
|
|
11
|
+
desc "Verify that every :trigger projection's trigger exists in the database (Postgres-only)"
|
|
12
|
+
task doctor: :environment do
|
|
13
|
+
require "standard_ledger"
|
|
14
|
+
|
|
15
|
+
# Discover entry classes by walking ActiveRecord descendants for
|
|
16
|
+
# ones that include `StandardLedger::Projector` and have at least
|
|
17
|
+
# one `:trigger` projection registered. The host's eager loading
|
|
18
|
+
# (Rails default in production / when explicitly invoked in dev)
|
|
19
|
+
# ensures all entry classes are loaded before this iterates.
|
|
20
|
+
entry_classes = ActiveRecord::Base.descendants.select { |klass|
|
|
21
|
+
klass.respond_to?(:standard_ledger_projections) &&
|
|
22
|
+
klass.standard_ledger_projections.any? { |d| d.mode == :trigger }
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
missing = []
|
|
26
|
+
entry_classes.each do |klass|
|
|
27
|
+
klass.standard_ledger_projections_for(:trigger).each do |definition|
|
|
28
|
+
# `pg_trigger.tgname` is the trigger's name as known to Postgres,
|
|
29
|
+
# but trigger names are scoped per-table — two tables can each
|
|
30
|
+
# have a trigger called e.g. `update_counts`. Join `pg_trigger`
|
|
31
|
+
# to `pg_class` and filter by the entry class's table name so the
|
|
32
|
+
# doctor reports presence on the *correct* table, not "anywhere
|
|
33
|
+
# in the schema". Use `klass.connection` (rather than
|
|
34
|
+
# `ActiveRecord::Base.connection`) so multi-DB setups query the
|
|
35
|
+
# connection that owns the entry class's table.
|
|
36
|
+
result = klass.connection.exec_query(
|
|
37
|
+
"SELECT 1 FROM pg_trigger t " \
|
|
38
|
+
"JOIN pg_class c ON c.oid = t.tgrelid " \
|
|
39
|
+
"WHERE t.tgname = $1 AND c.relname = $2 " \
|
|
40
|
+
"LIMIT 1",
|
|
41
|
+
"standard_ledger:doctor",
|
|
42
|
+
[ definition.trigger_name, klass.table_name ]
|
|
43
|
+
)
|
|
44
|
+
if result.rows.empty?
|
|
45
|
+
missing << " #{klass.name}##{definition.target_association}: trigger #{definition.trigger_name.inspect} not found"
|
|
46
|
+
end
|
|
47
|
+
end
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
if missing.empty?
|
|
51
|
+
puts "All :trigger projections have their triggers present."
|
|
52
|
+
else
|
|
53
|
+
warn "Missing triggers detected:"
|
|
54
|
+
missing.each { |line| warn line }
|
|
55
|
+
warn ""
|
|
56
|
+
warn "Run the migration that creates the trigger, or check that the trigger name in `projects_onto` matches the actual trigger name in the schema."
|
|
57
|
+
exit 1
|
|
58
|
+
end
|
|
59
|
+
end
|
|
60
|
+
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: standard_ledger
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.3.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Jaryl Sim
|
|
@@ -122,10 +122,11 @@ dependencies:
|
|
|
122
122
|
- !ruby/object:Gem::Version
|
|
123
123
|
version: '0'
|
|
124
124
|
description: StandardLedger captures the recurring 'append-only entry → N projection
|
|
125
|
-
updates' pattern as a declarative DSL on host ActiveRecord models. Supports
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
125
|
+
updates' pattern as a declarative DSL on host ActiveRecord models. Supports five
|
|
126
|
+
projection modes — :inline, :async, :sql, :matview, :trigger — plus a deterministic
|
|
127
|
+
rebuild path from the entry log, ad-hoc materialized view refresh, and a doctor
|
|
128
|
+
rake task that verifies host-owned trigger presence. Enforces idempotency-by-unique-index
|
|
129
|
+
and immutability at the entry level.
|
|
129
130
|
email:
|
|
130
131
|
- code@jaryl.dev
|
|
131
132
|
executables: []
|
|
@@ -145,9 +146,12 @@ files:
|
|
|
145
146
|
- lib/standard_ledger/errors.rb
|
|
146
147
|
- lib/standard_ledger/event_emitter.rb
|
|
147
148
|
- lib/standard_ledger/jobs/matview_refresh_job.rb
|
|
149
|
+
- lib/standard_ledger/jobs/projection_job.rb
|
|
150
|
+
- lib/standard_ledger/modes/async.rb
|
|
148
151
|
- lib/standard_ledger/modes/inline.rb
|
|
149
152
|
- lib/standard_ledger/modes/matview.rb
|
|
150
153
|
- lib/standard_ledger/modes/sql.rb
|
|
154
|
+
- lib/standard_ledger/modes/trigger.rb
|
|
151
155
|
- lib/standard_ledger/projection.rb
|
|
152
156
|
- lib/standard_ledger/projector.rb
|
|
153
157
|
- lib/standard_ledger/result.rb
|
|
@@ -155,6 +159,7 @@ files:
|
|
|
155
159
|
- lib/standard_ledger/rspec/helpers.rb
|
|
156
160
|
- lib/standard_ledger/rspec/matchers.rb
|
|
157
161
|
- lib/standard_ledger/version.rb
|
|
162
|
+
- lib/tasks/standard_ledger.rake
|
|
158
163
|
homepage: https://github.com/rarebit-one/standard_ledger
|
|
159
164
|
licenses:
|
|
160
165
|
- MIT
|