orfeas_lyra 0.8.0 → 0.9.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (85) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +869 -1
  3. data/README.md +216 -29
  4. data/Rakefile +11 -3
  5. data/app/controllers/lyra/application_controller.rb +48 -0
  6. data/app/controllers/lyra/dashboard_controller.rb +2 -1
  7. data/app/views/lyra/dashboard/discrepancies.html.erb +64 -35
  8. data/app/views/lyra/dashboard/projections.html.erb +0 -4
  9. data/app/views/lyra/dashboard/verification.html.erb +8 -2
  10. data/app/workflows/es_async_mode_workflow.rb +24 -60
  11. data/app/workflows/es_sync_mode_workflow.rb +25 -50
  12. data/app/workflows/hijack_mode_workflow.rb +18 -33
  13. data/app/workflows/lifecycle_workflow.rb +1 -1
  14. data/app/workflows/monitor_mode_workflow.rb +12 -22
  15. data/config/routes.rb +4 -2
  16. data/lib/lyra/access_log.rb +121 -0
  17. data/lib/lyra/advisory_lock.rb +45 -0
  18. data/lib/lyra/aggregate.rb +25 -13
  19. data/lib/lyra/associations/event_aware.rb +3 -4
  20. data/lib/lyra/associations/habtm_links.rb +255 -0
  21. data/lib/lyra/bypass_events.rb +221 -0
  22. data/lib/lyra/command.rb +3 -0
  23. data/lib/lyra/command_handler.rb +168 -42
  24. data/lib/lyra/configuration.rb +224 -8
  25. data/lib/lyra/consistency/read_your_writes.rb +74 -23
  26. data/lib/lyra/correlation.rb +89 -0
  27. data/lib/lyra/domain_events.rb +161 -0
  28. data/lib/lyra/dual_view.rb +27 -6
  29. data/lib/lyra/dual_view_sampler.rb +33 -0
  30. data/lib/lyra/encrypted_attributes.rb +101 -0
  31. data/lib/lyra/engine.rb +75 -5
  32. data/lib/lyra/erasure.rb +316 -0
  33. data/lib/lyra/event.rb +120 -2
  34. data/lib/lyra/event_flow.rb +16 -6
  35. data/lib/lyra/event_serializer.rb +45 -0
  36. data/lib/lyra/event_store_adapter.rb +58 -62
  37. data/lib/lyra/genesis.rb +189 -0
  38. data/lib/lyra/id_generator.rb +42 -10
  39. data/lib/lyra/interceptors/association_interceptor.rb +123 -6
  40. data/lib/lyra/interceptors/crud_interceptor.rb +429 -72
  41. data/lib/lyra/interceptors/lazy_reads.rb +46 -0
  42. data/lib/lyra/mode_sync.rb +125 -0
  43. data/lib/lyra/mode_transition.rb +386 -0
  44. data/lib/lyra/model_selection.rb +160 -0
  45. data/lib/lyra/optimistic_locking.rb +88 -0
  46. data/lib/lyra/optional_dependency.rb +24 -0
  47. data/lib/lyra/own_connection.rb +54 -0
  48. data/lib/lyra/privacy/adapters/pam.rb +151 -0
  49. data/lib/lyra/privacy/gdpr_compliance.rb +50 -0
  50. data/lib/lyra/privacy/interface.rb +262 -0
  51. data/lib/lyra/privacy/pii_detector.rb +8 -10
  52. data/lib/lyra/privacy/pii_masker.rb +30 -13
  53. data/lib/lyra/privacy/policy_integration.rb +36 -141
  54. data/lib/lyra/projection.rb +30 -7
  55. data/lib/lyra/projections/association_conditions.rb +81 -0
  56. data/lib/lyra/projections/async_checkpoint.rb +57 -0
  57. data/lib/lyra/projections/async_projection_job.rb +71 -17
  58. data/lib/lyra/projections/cached_joins.rb +192 -0
  59. data/lib/lyra/projections/cached_projection.rb +278 -162
  60. data/lib/lyra/projections/cached_relation.rb +666 -85
  61. data/lib/lyra/projections/deferred_relation.rb +120 -0
  62. data/lib/lyra/projections/event_store_reader.rb +27 -4
  63. data/lib/lyra/projections/lazy_projection.rb +259 -0
  64. data/lib/lyra/projections/model_projection.rb +32 -34
  65. data/lib/lyra/projections/rebuild.rb +58 -29
  66. data/lib/lyra/projections/reference_links.rb +309 -0
  67. data/lib/lyra/purpose_bound_reads.rb +197 -0
  68. data/lib/lyra/repair.rb +114 -0
  69. data/lib/lyra/retention.rb +180 -0
  70. data/lib/lyra/schema/event_class_registrar.rb +2 -1
  71. data/lib/lyra/schema/validator.rb +14 -5
  72. data/lib/lyra/sti_stream_migration.rb +70 -0
  73. data/lib/lyra/strict_data_access.rb +225 -175
  74. data/lib/lyra/temporal.rb +128 -0
  75. data/lib/lyra/verification/bypass_workflow.rb +170 -0
  76. data/lib/lyra/verification/crud_lifecycle_workflow.rb +32 -6
  77. data/lib/lyra/verification/workflow_generator.rb +73 -179
  78. data/lib/lyra/version.rb +1 -1
  79. data/lib/lyra.rb +113 -17
  80. data/lib/tasks/lyra_mode.rake +89 -0
  81. data/lib/tasks/lyra_projections.rake +43 -0
  82. data/lib/tasks/lyra_schema.rake +4 -0
  83. data/lib/tasks/lyra_workflows.rake +28 -27
  84. metadata +36 -5
  85. data/lib/lyra/event_mapper.rb +0 -106
data/CHANGELOG.md CHANGED
@@ -7,9 +7,877 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.9.1] - 2026-11-02
11
+
12
+ Lyra's changes since 0.9.0; PAM DSL and PetriFlow change only in their gem descriptions.
13
+
14
+ ### Released
15
+ - The gems on RubyGems (`orfeas_lyra`, `orfeas_petri_flow`, `orfeas_pam_dsl`), built from tag
16
+ `v0.9.1`
17
+
18
+ ### Added
19
+ - **Writes are grouped into causal chains by default** (`config.correlate_writes`, on). Before,
20
+ Lyra carried correlation and causation only where the application called
21
+ `Lyra::Correlation.with_id`/`Lyra::Causation.with_id`, which no example did: every event was its
22
+ own chain. Now:
23
+ - every event-producing write runs in a write scope (`around_save`/`around_destroy`, first in
24
+ the chain) that fixes its event id before its callbacks run, so a write made inside them
25
+ records that event as its `causation_id`; a record saving itself inside its own callbacks is
26
+ caused by the outer save;
27
+ - the outermost write opens the correlation when none is in scope, its own event id, as the
28
+ event store does for a root event;
29
+ - `Lyra::Correlation::Middleware`, installed by the engine, gives the writes of one HTTP request
30
+ one correlation, the request id; a correlation already in scope is kept;
31
+ - `Lyra::Correlation::Job`, included into `ActiveJob::Base` by the engine, gives the writes of
32
+ one job the correlation in scope where the job was created, or the job id without one, and a
33
+ write in scope there as their cause. Both are taken when the job is created, so a job deferred
34
+ to after the commit (`enqueue_after_transaction_commit`) keeps them, and travel in its payload
35
+ (`lyra_correlation_id`, `lyra_causation_id`), so a retry does too. A job created inside a
36
+ transaction and sent at once (the Rails default) could outlive a rollback of its cause; its
37
+ cause is kept when it runs only if that event is in the log, so no job names a cause the log
38
+ lacks (one that runs before the enqueuing transaction commits loses the link).
39
+ Monitor, Hijack and event-sourcing events alike (`DomainEvents.build` takes the pre-assigned
40
+ `event_id:`, given once per write). The Olist replay runs each step as one request would
41
+ (`test/write_correlation_test.rb`).
42
+
43
+ ### Changed
44
+ - **ORFEAS is expanded as the words its letters stand for**: Object-Relational Framework for
45
+ Event-Aware Systems (README, `docs/ARCHITECTURE.md` and its citation, gem descriptions). It
46
+ read "Object-Relational to Event-Sourcing Architecture", which has no F.
47
+
48
+ ### Fixed
49
+ - **Writes that skip callbacks name who made them.** Events of `update_columns`, `touch`,
50
+ `delete`, `update_all`, `delete_all`, `insert_all`, `upsert_all` and `dependent: :nullify`, and
51
+ `has_and_belongs_to_many` links, carried correlation and causation but neither `user_id` nor
52
+ `request_id`, so the same request's writes named a user only if they ran callbacks. They now
53
+ take both from `Current`, as the documentation already said. Events written before this keep no
54
+ actor; their correlation still ties them to the request's other writes.
55
+ - **`AsyncProjectionJob` warns again when its event is not in the log.** The warning sat in a
56
+ rescue of `RubyEventStore::EventNotFound`, which `read.event` never raises (it returns nil;
57
+ `read.event!` raises), so it was never logged. The job still did nothing for a missing event.
58
+
59
+ ## [0.9.0] - 2026-10-25
60
+
61
+ Everything since 0.8.0, the version the PAM article describes.
62
+
63
+ ### Released
64
+ - The gems on RubyGems (`orfeas_lyra`, `orfeas_petri_flow`, `orfeas_pam_dsl`), built from tag
65
+ `v0.9.0` (2026-10-19); the entries marked *after the tag* are not in them
66
+ - The replication package: https://doi.org/10.5281/zenodo.17770613
67
+ - The benchmark results: https://doi.org/10.5281/zenodo.23640846
68
+
69
+ ### Added
70
+ - **`config.models` takes namespace patterns, `:all` and a denylist** (`Lyra::ModelSelection`):
71
+ `config.models = %w[Spree::* Order]`, `config.models = :all`,
72
+ `config.models = :all, except: %w[Session]`. Patterns and `:all` enrol only eligible models
73
+ (concrete, named, with a table and a single-column primary key, base classes of an STI
74
+ hierarchy), after eager-loading the application's code. The event store's tables, Lyra's own and
75
+ those of Solid Cache/Queue/Cable, Active Storage, Action Text, Action Mailbox and PaperTrail are
76
+ never enrolled; naming one fails the boot. A pattern matching nothing warns; `:all` outside
77
+ Monitor mode warns. On a Solidus application `:all` enrols 90 models, excludes 28 and leaves 48
78
+ abstract classes and STI subclasses to their bases (`test/model_selection_test.rb`).
79
+ - **`ReadYourWrites.with_guaranteed_read(each_write: true)`**: each write in the block is projected
80
+ as it is made, so reads later in the same block see it (by default the block projects when it
81
+ ends). For code that writes and reads back in one unit of work, as Solidus's checkout does
82
+ (a replay of 2,000 Olist orders through Solidus under ES-Async matched the oracle for 135 orders,
83
+ and for 1,992, as plain ActiveRecord does, with each step in such a block).
84
+ `test/projections/read_your_writes_async_test.rb`. The controller concern takes it per
85
+ controller, inherited by subclasses, so an application is adapted from an initializer:
86
+ `Spree::BaseController.include Lyra::Consistency::ControllerConcern` and
87
+ `Spree::BaseController.lyra_read_your_writes each_write: true`
88
+ (`test/consistency/controller_concern_test.rb`). `docs/API_REFERENCE.md` named the concern
89
+ `ReadYourWrites::ControllerConcern`, which does not exist; it is `Consistency::ControllerConcern`.
90
+ - **`config.lock_aggregate_before_append`** (default `true`): Hijack, ES-Sync and ES-Async lock
91
+ the aggregate before appending. Two writers of one record could append in one order and write the
92
+ row in the other, leaving the row at a value its log does not replay to; seen in the wild, 2
93
+ rounds of 120 with 8 threads on one record, and reproduced deterministically
94
+ (`test/concurrent_same_record_test.rb`). An update or destroy now locks the record's row before
95
+ its event is appended, held to the end of the transaction (assumption L of Theorem 5), in the
96
+ strength the write takes anyway (PostgreSQL `FOR NO KEY UPDATE` for an update, `FOR UPDATE` for
97
+ a destroy), so a write takes the locks plain ActiveRecord takes, in the same order, and adds no
98
+ deadlock. A first version took an advisory lock on the stream before the row lock; it deadlocked
99
+ against a writer that holds the row first (`record.lock!`, `with_lock`) and then updates it,
100
+ where plain ActiveRecord only makes the update wait. The advisory lock remains for a record with
101
+ no row. `test/aggregate_lock_deadlock_test.rb`: writes to several records in one order never
102
+ deadlock, opposite orders deadlock exactly as plain ActiveRecord does, a row lock held first
103
+ does not deadlock a plain writer. SQLite needs no lock: it serialises writers from the first
104
+ write, which is the append (`test/sqlite_same_record_test.rb`). ES-Async writers take the
105
+ advisory lock on the stream (they do not write the row, the projection job does, so they never
106
+ wait for the row's lock); it makes a record's events commit in append order, which the job's
107
+ checkpoint relies on (below). Creates, Monitor, ES-Lazy and ES-NoProj are unchanged
108
+ (`test/aggregate_lock_statements_test.rb`). Cost: an update issues three more SQL statements
109
+ (the lock, then the savepoint pair the append now needs), so Hijack and ES-Sync updates go from
110
+ 5 to 8 statements and ES-Async's from 4 to 7, and a destroy one more; on one contended record
111
+ Hijack's throughput falls from about 90 to about 52 ops/s. Set `false` for the previous
112
+ behaviour and its numbers.
113
+ - **ES-Async's projection job applies only the events after the stream's checkpoint**
114
+ (`Lyra::Projections::AsyncCheckpoint`, table `lyra_async_checkpoints`, created on first use)
115
+ instead of replaying the record's whole stream, so a record's jobs cost what its history is
116
+ long, not its square. Exact because the lock above makes a record's commits follow its
117
+ appends; with the lock off, or without a usable checkpoint (its event gone), the job replays
118
+ the whole stream. A checkpoint older than the row only re-applies events the row already has,
119
+ which is harmless (`test/projections/async_checkpoint_test.rb`).
120
+ - `test/projections/cached_projection_concurrency_test.rb`: readers and writers of one record under
121
+ ES-NoProj see only states the stream had, never go back, and converge.
122
+ - **Reference links for ES-NoProj** (`Lyra::Projections::ReferenceLinks`, `config.reference_links`,
123
+ default `:auto`). An event that sets a `belongs_to` foreign key is also linked into the stream
124
+ `Model.foreign_key$value`, the way Rails Event Store links events by metadata. A lookup by that
125
+ key, including the dependent-association lookups Active Record makes on destroy, reads the
126
+ stream for candidates and keeps those whose current state still matches, so moved and destroyed
127
+ children drop out and the answer is exact, without rebuilding every record of the model.
128
+ `:auto` writes links in ES-NoProj only, so no other mode pays; `true` and `false` force it.
129
+ Events stored while linking was off are linked before a model's first lookup in a process, from
130
+ a checkpoint kept in the store (`$lyra_reference_links$Model`). On the Aegean testbed this made
131
+ an ES-NoProj delete about 30 times faster at 16 threads and about 250 times at 64.
132
+ - **`config.monitor_append_failure`** (opt-in; default `:log`, unchanged). With `:fail_write`,
133
+ Monitor fails a write whose event cannot be stored, raising `EventStoreUnavailableError` and
134
+ rolling the write back (bulk writes included), as Hijack and the event-sourcing modes always
135
+ do. Row and event already commit together; the setting decides only what a failed append
136
+ does. Validated on assignment.
137
+ - **Trace conformance test** (`test/verification/trace_conformance_test.rb`). It records the
138
+ steps Lyra takes on real writes in every mode (event built, applied to the aggregate, stored;
139
+ row written; commit) and replays each trace token by token on a PetriFlow net: the paper's
140
+ core net with the row write placed as each mode places it. Every trace must be a firing
141
+ sequence ending in the net's final place, each operation must build exactly one event of its
142
+ own type, and a trace in another order is rejected. It found that Hijack and the
143
+ event-sourcing modes apply the event to the aggregate before storing it, the order the formal
144
+ model now uses.
145
+ - **`LYRA_DISABLE_PETRI_FLOW=true`** — runs Lyra as without petri_flow, as
146
+ `LYRA_DISABLE_PAM_DSL=true` does for PAM DSL.
147
+ - **`lyra:workflows:generate OUTPUT_DIR=... REPORTS_DIR=...`** — write the workflow files and
148
+ the reports to other directories than `app/workflows/` and `reports/`.
149
+ - **Dashboard authorization** (`config.dashboard_authorization`) — a proc run on the engine's
150
+ controller before every action (`instance_exec`, and given the controller when it takes an
151
+ argument); truthy allows, falsy answers 403. Unset, the dashboard is open in development and
152
+ test only and refused everywhere else with a log line, since its privacy pages show a data
153
+ subject's personal data. Example: `->(controller) { controller.current_user&.admin? }`.
154
+ - **`mode=` and `projection_mode=` validate** — one of `Configuration::MODES` and the new
155
+ `Configuration::PROJECTION_MODES`; a string is taken as its symbol, anything else raises
156
+ `ArgumentError` naming the valid values.
157
+ - **`lyra:erase MAX_COPIES=n`** — passed to `Lyra::Erasure.erase!` as `max_copies:`; the task
158
+ also prints how many values were left as shared.
159
+ - **The blog example runs again** (`examples/blog_app`) — rebuilt for Rails 8.1,
160
+ RailsEventStore 3 and PostgreSQL, with `bin/setup` and a passing test suite.
161
+ - **ES-NoProj runs Solidus** — the Olist replay passes under `projection_mode :disabled`
162
+ (6 of 6 orders end as Olist records them). Association queries (`variant.prices.find_by`,
163
+ `.where`, `.find_or_create_by!`) run on the records the event store holds for the owner, and
164
+ `:through`, polymorphic (`as:`) and scoped associations are read correctly: a `:through`
165
+ association was filtered on a foreign key its target does not have. Calculations and plucks on
166
+ an association (`payment.refunds.sum(:amount)`) read the events instead of the empty table.
167
+ Building through an association or a `where` takes its conditions, as ActiveRecord's
168
+ `scope_for_create` does. `CachedRelation` gains `group` with grouped `count`/`sum`/`average`/
169
+ `minimum`/`maximum`, `extending`, class attributes (`discard_column`), records as condition
170
+ values (mapped to their ids) and the relation readers ActiveRecord's calculations ask for.
171
+ - **Erasure driven by the policy, opt-in** (`Lyra::Retention`, `config.retention_executor`,
172
+ `config.retention_anchors`, `bin/rails lyra:retention:apply [DRY_RUN=1] [MODELS=]`,
173
+ `Lyra::RetentionJob`) — applies the policy's retention rules: for each monitored model with a
174
+ rule, a record past its period gets the rule's `on_expiry` — `:anonymize` (Lyra::Erasure),
175
+ `:hard_delete` (erase, then destroy through the write path), `:soft_delete` (`deleted_at` /
176
+ `discarded_at` through the write path; skipped without one), `:archive` (skipped: no archive
177
+ is named). Attributes with a shorter period of their own are erased first; rule conditions get
178
+ the record; already-erased attributes are skipped, so runs are idempotent. The period runs
179
+ from `created_at` or the column named per model. Off by default; a dry run works either way.
180
+ The privacy interface gains `Policy#retention_rule`.
181
+ - **Erasure finds copies by value, and optionally everywhere** — `Lyra::Erasure` now also
182
+ replaces an erased value wherever the record's events copy it under another name (a payload's
183
+ `payer_email`). `everywhere: true` (`EVERYWHERE=1`) searches the whole log for the values:
184
+ other records' events are scrubbed, their rows get the value replaced where they hold it, and
185
+ each gets an `ErasureApplied`; `Result#copies` lists them. A value held by more than
186
+ `max_copies` other records (default 10) is shared by many people, not a copy, and is left
187
+ (`Result#shared_values` counts them): on the Olist replay a placeholder address shared by
188
+ every customer took all 1,954 addresses with it. Only direct identifiers (the policy's email,
189
+ phone, identifier, card, payment-token, IP, credential types) are searched for in other
190
+ records: a name, city or postal code can be another customer's by coincidence (erasing one
191
+ Olist customer's postal-code prefix took two other customers' addresses). Erasure writes rows with one `UPDATE` by id,
192
+ so a record the application marks read-only (a Solidus address once an order uses it) is
193
+ erased too.
194
+ - **Purpose-bound `pluck` and `pick`** — checked by the declared attributes they name (also
195
+ inside SQL fragments), with the model (`"Registration$*"`) as subject, so a purpose whose
196
+ consent the policy requires refuses them.
197
+ - **Purpose-bound reads** (`Lyra::PurposeBoundReads`, `Lyra.with_purpose`, `lyra_purpose` for
198
+ controllers and jobs, `config.reads_without_purpose`) — a read of a monitored model with a
199
+ privacy policy, made within a declared purpose, is checked through the policy's
200
+ `validate_access!` with the record as subject: every declared attribute the query loaded must be
201
+ allowed for the purpose, so loading more than the purpose needs is refused (data minimisation).
202
+ No setting turns it on: the purpose does. Reads with no purpose follow
203
+ `config.reads_without_purpose`: `:allow` (default), `:audit` (logged, and recorded as an audited
204
+ access with purpose `none` when the access log is on) or `:deny` (`PurposeRequiredError`).
205
+ Lyra's own reads (projections, bypass snapshots, Genesis, DualView, mode checks, repair,
206
+ erasure) are exempt; reads that load no model are not checked. Every monitored model gains an
207
+ `after_find`, which returns at once when no purpose is in scope and reads without one are
208
+ allowed.
209
+ - **Erasure executor** (`Lyra::Erasure.erase!`, `bin/rails lyra:erase MODEL= ID= REASON=
210
+ [FIELDS=]`, `Lyra::Events::ErasureApplied`) — erases one record's personal attributes (declared
211
+ by its policy or listed by its events' privacy stamps, or exactly `fields:`) from the row and
212
+ from every event in its stream, overwriting each event in place (same id, position, time):
213
+ attributes, both sides of each change, and payload keys with the attribute's name. Appends
214
+ `ErasureApplied` (fields, reason, who; never a value), which replay skips; the stream still
215
+ replays to the anonymized row. Replacement is nil where allowed, `"erased:<id>"` for a NOT NULL
216
+ string, else the column default. Works in every mode, with strict data access on and inside a
217
+ purpose. On request only; not policy-driven, and crypto-shredding remains future work.
218
+ - **Event store failure policies, named** (`Lyra::EventStoreUnavailableError`, `Lyra.append_events`;
219
+ FEATURE_GAP_PLAN F1) — every event Lyra writes now goes through `Lyra.append_events`, which
220
+ raises `EventStoreUnavailableError` (the store's error as its `cause`, the stream in its message)
221
+ when the store fails. The policy is fixed by the mode: **fail-closed** in Hijack and every
222
+ event-sourcing mode (the write fails with the error and rolls back), **log-and-continue** in
223
+ Monitor (the write stands; the log line names the stream and points to `lyra:repair`). No
224
+ in-memory retry queue: the store shares the application's database, a queue dies with its
225
+ process, and a retried event could land behind a later one in its stream.
226
+ - **Repair after lost events** (`Lyra::Repair`, `bin/rails lyra:repair [DRY_RUN=1] [MODELS=...]`) —
227
+ brings the event log back in line with the tables after Monitor lost events: an `Imported` event
228
+ for a row with no events (or one its stream says was destroyed), an `Updated` event with the
229
+ differing columns for a row that differs from its events, a `Destroyed` event for events with no
230
+ row. Each record is re-checked and repaired under a lock on its row; repair events carry
231
+ metadata `source: "lyra_repair"` and the problem repaired. Refused in Hijack and event sourcing,
232
+ where the events are authoritative (rebuild the tables instead). `ModeTransition.record_ids` and
233
+ `ModeTransition.discrepancy` expose the check it uses.
234
+ - **Access log** (`config.record_access_events`, default false; `Lyra::AccessLog`,
235
+ `Lyra::Events::DataAccessed`, `Lyra::Events::DataAccessDenied`; FEATURE_GAP_PLAN F6) — when on,
236
+ every access the privacy policy validates (`validate_access!`) is recorded in the subject's
237
+ access stream (`Lyra::DataAccess$<Model>$<id>`): `DataAccessed` when it went ahead (outcome
238
+ `granted`, or `audited` with the violations audit mode let through), `DataAccessDenied` when
239
+ strict mode refused it. Each carries the policy, purpose, legal basis, field names (never
240
+ values) and outcome. Who accessed goes in the metadata: `user_id` and `ip_address` from the
241
+ application's `Current.user` / `Current.ip_address`, request and correlation ids, and whatever
242
+ `config.access_metadata_proc` returns for the access (for consoles, jobs and API tokens that
243
+ `Current` does not know; a failing proc is logged and skipped). Every access is
244
+ recorded, with no sampling; an access the store cannot record does not go ahead. Nothing is
245
+ recorded in disabled mode, and the record's own stream is untouched. `AccessLog.for(subject)`
246
+ reads them back. Off by default because reads can outnumber writes by orders of magnitude; the
247
+ Aegean testbed turns it on with `LYRA_RECORD_ACCESS=1`, which also makes the benchmark's reads
248
+ validate access, and records it in its benchmark reports. The Article 30 register stays
249
+ PAM's `article_30_report`, generated from the declared policy.
250
+ - **Privacy stamps on events** (`config.annotate_privacy`, default false; `Lyra::Privacy.stamp`,
251
+ `Lyra::Privacy.stamp_of`; FEATURE_GAP_PLAN F5) — when on, every event that carries an
252
+ attribute the model's privacy policy declares is stamped as it is built with
253
+ `metadata[:privacy]`: the policy's name and, per attribute, its type, sensitivity, allowed
254
+ purposes, retention period (ISO 8601) and the contexts it has a transformation for — never a
255
+ value. Every declared attribute the event carries is annotated: a create's, import's or
256
+ destroy's attributes, and an update's changes together with the whole row Monitor also records
257
+ (annotating only the changes left 28,374 of an Olist replay's events carrying personal data
258
+ unannotated). All four places events are built stamp them: Monitor, the command handler (Hijack,
259
+ event sourcing, and so domain events), bypass events and Genesis imports. The stamp records the
260
+ policy as it was when the data was written, unlike `annotations_for`, which reads today's
261
+ policy; `stamp_of(event)` reads it back with string keys. Off by default because it adds work
262
+ to every write (no SQL statements); the Aegean testbed turns it on with
263
+ `LYRA_ANNOTATE_PRIVACY=1` and records it in its benchmark reports. The privacy `Annotation`
264
+ gains a `transformations` member.
265
+ - **ModeSync: one mode per application** (`Lyra::ModeSync`, `config.mode_sync`,
266
+ `config.mode_sync_interval`) — the latest switch recorded in `lyra_mode_transitions` is the
267
+ application's mode; every process adopts a newer one within `mode_sync_interval` seconds
268
+ (default 5), checking before each web request (Rack middleware), each background job and each
269
+ write to a monitored model, never inside an open transaction. A runtime switch
270
+ (`ModeTransition.to!`) used to change only the process that made it. `to!` now logs what the
271
+ switch reaches. On wherever the Mode Transition Safety gate is (not in the test environment);
272
+ the raw `config.mode =` setter is left alone. the migration guide (`docs/MIGRATION_GUIDE.md`, Switching modes) documents the
273
+ deploy-time rule.
274
+ - **Mode Transition Safety** (`Lyra::ModeTransition`, `rake lyra:mode:check`; FEATURE_GAP_PLAN F3)
275
+ — a mode switch that changes the authoritative store is allowed only when rows and events
276
+ agree for every monitored record: escalating from Disabled or Monitor to Hijack or event
277
+ sourcing (rows that predate Lyra are imported by Genesis first), and leaving ES-NoProj,
278
+ ES-Lazy or ES-Async for a mode that reads tables (`rebuild: true` rebuilds them first). The
279
+ check compares every row and every stream in batches, reading the real table, then
280
+ re-checks what changed while it ran, and stores a clean result as a certificate.
281
+ `ModeTransition.to!` and the `enable_*!` helpers accept a fresh certificate after
282
+ re-checking only what changed since, else run the full check, and raise `Refused` with the
283
+ discrepancies; `force: true` overrides. At boot, a switch from the mode the application last
284
+ ran in needs a certificate (`LYRA_FORCE_MODE_TRANSITION=1` overrides; rake processes are
285
+ exempt). `config.mode_transition_gate`: nil (default) gates everywhere but the test
286
+ environment; `config.mode = ...` stays the raw, ungated setter.
287
+ - **Sampled DualView verification** (`config.dual_view_sample_rate`, default 0.0, off;
288
+ `config.dual_view_discrepancy_handler`) — after a write commits, a sampled share of writes
289
+ is compared with its events; a discrepancy is logged and passed to the handler, and the write
290
+ is never failed. Skipped in ES-NoProj, ES-Lazy and ES-Async, whose tables lag by design.
291
+ - **Domain events** (`monitor_with_lyra domain_events: [...]`, `Lyra::DomainEvents`;
292
+ FEATURE_GAP_PLAN F8) — rules name a write's event after what it means instead of the CRUD
293
+ operation: `{ name: "PaymentCompleted", on: %i[create update], if: ->(payment, changes) { ... } }`.
294
+ The first matching rule names the write's own event, which keeps Lyra's envelope (model, id,
295
+ operation, attributes, changes), so replay, projections, DualView and `Lyra.state_at` work
296
+ unchanged; a write no rule matches keeps its CRUD event. `payload:` adds fields for the
297
+ event's consumers under `data[:payload]`; `class:` uses an event class of your own (a
298
+ `RubyEventStore::Event` subclass) instead of a generated `Lyra::Events::<name>`;
299
+ `also: true` emits an additional event from the same write, in the same stream and
300
+ transaction, marked `replay: false` and pointing to the write's own event through its
301
+ `causation_id`. Works in Monitor, Hijack and every event-sourcing mode; writes that skip
302
+ callbacks are still recorded as CRUD bypass events. Generated event classes are registered
303
+ at boot. A rule whose block raises fails the write where events are the record (Hijack,
304
+ event sourcing) and is logged in Monitor. Commands now carry their `record`.
305
+ - **Configuration surface** (FEATURE_GAP_PLAN F2): `config.models = %w[Order Payment]` (or a
306
+ hash of name => options) declares the monitored models by name in the initializer; once the
307
+ application's code has loaded, each is given `monitor_with_lyra`, with no model file edited,
308
+ and a name that does not resolve fails the boot. `config.verify_mapping!` verifies the
309
+ CRUD-to-event mapping at boot (`Lyra.verify_mapping!`: the PetriFlow lifecycle, mode and
310
+ bypass nets through `CrudVerifier`, and a table and primary key for every monitored model)
311
+ and stops the boot with `Lyra::MappingVerificationError` naming each failed check; called
312
+ after boot it runs at once. `config.privacy_policy = :name` is the default policy for
313
+ monitored models that name none of their own.
314
+ - **Point-in-time reconstruction** (`Lyra.state_at(model, id, time)`, `Model.as_of(time)`,
315
+ `Lyra::Temporal`) — a record's attributes at a given time, rebuilt by replaying its stream up
316
+ to the last event stored at or before then: `nil` if it did not exist yet or had been
317
+ destroyed. `Model.as_of(time).find(id)` / `.find_by_id(id)` return read-only records and
318
+ `.all` every record that existed then. For a row imported by Genesis, a time before its
319
+ import raises `Lyra::Temporal::HistoryNotRecorded` instead of guessing (a time before its own
320
+ `created_at` is `nil`). Works in every mode that records events, Monitor included
321
+ (FEATURE_GAP_PLAN F7).
322
+ - **ES-NoProj joins, evaluated in memory** (`Lyra::Projections::CachedJoins`) — `joins`,
323
+ `left_joins`/`left_outer_joins`, `where.missing` and `where.associated` on `belongs_to`,
324
+ `has_many` and `has_one` associations, with hash conditions on the joined table
325
+ (`where(programs: { available: true })`, by table or association name), used to raise
326
+ `UnsupportedQuery`. They are now evaluated as SQL would: a joined relation holds rows of a
327
+ record and its partners; an inner join drops records without a partner; a `has_many` join
328
+ repeats the record per partner (so `count` agrees, and `distinct` removes the repeats); a
329
+ missing left-join partner behaves like NULL, including under `where.not`. Partners come from
330
+ the joined model's streams if it is an ES-NoProj model, otherwise from one SQL query on its
331
+ table. Still refused: SQL-string, nested, `:through`, polymorphic and scoped joins, a
332
+ condition on a table not joined first, and `merge` with an arbitrary relation (so Solidus
333
+ still needs ES-Lazy). `where` with no arguments now returns the where-chain, as in
334
+ ActiveRecord.
335
+ - **Privacy provider interface** (`Lyra::Privacy`) — `Policy` (declared attributes and their
336
+ `Annotation`s), `Detector` (name-based PII heuristic) and `Provider`, with null defaults when
337
+ no provider is installed. PAM is an adapter (`Lyra::Privacy::Adapters::Pam`), loaded when
338
+ `pam_dsl` is present. `Lyra::Privacy.annotations_for(event)` returns the annotation of every
339
+ declared attribute an event touched. The foundation for `config.privacy_policy=` and PII
340
+ annotation in event metadata (`FEATURE_GAP_PLAN.md`, F2 and F5). From `thesis-restructure`
341
+ (1adadfe); the testbed binds `Registration` and `PaymentTransaction` to its `:epay_data`
342
+ policy (d0df542), which adds nothing to the write path.
343
+ - **`Lyra::IdGenerator.reserves_safely?(model)`** — true for UUID keys and for integer keys
344
+ backed by a PostgreSQL sequence. It uses the cached sequence lookup, so it adds no query per
345
+ write once a table's sequence is known.
346
+ - **Bypass events for bulk writes** (`Lyra::BypassEvents`) — `update_all`, `delete_all`,
347
+ `insert`/`insert_all(!)`, `upsert`/`upsert_all` and `dependent: :nullify` on a monitored model
348
+ now publish one event per affected row, whatever the `strict_data_access` setting
349
+ (`update_column(s)`, `touch` and `delete` already did). `update_all` publishes "updated" per
350
+ row that changed; `delete_all` "destroyed" per row; inserts "created" per row; `upsert_all`
351
+ finds existing rows by its conflict key (primary key, `unique_by:` columns or index name) and
352
+ publishes "updated" for those, "created" for the rest. The caller's `returning:` result is
353
+ unchanged. Brought over from the `thesis-restructure` branch (3ba39bb, 975dc6e), keeping
354
+ master's instance methods (events built after the write).
355
+ - **`Lyra.projection_write { ... }`** — Lyra's own read-model writes: skips strict data access
356
+ and publishes no bypass events. `ModelProjection` and `Rebuild` use it; use it for writes that
357
+ must not reach the event stream, such as seeding or wiping a table.
358
+ - **Genesis: event streams for rows that predate Lyra** (`Lyra::Genesis`,
359
+ `config.genesis`, `rake lyra:genesis`) — a row written before Lyra was enabled had no
360
+ stream, so its first `Updated` event started a stream with no head, and DualView,
361
+ `Rebuild` and ES-NoProj reads saw the record wrongly or not at all. On a model's first
362
+ use in a process (its first write, or in ES-NoProj its first read or aggregate), every
363
+ row without a stream now gets one `Imported` event holding the row as it is. `Imported`
364
+ is replayed like `Created` everywhere state is rebuilt. Later uses cost nothing; when
365
+ every row already has a stream the check is one query and takes no lock. The import
366
+ runs under a per-model advisory lock on a connection of its own, so a long caller
367
+ transaction does not block other first uses and a caller's rollback does not undo it.
368
+ `config.genesis`: `:auto` (default, event-sourcing mode only), `true` (also Monitor and
369
+ Hijack) or `false`. `rake lyra:genesis MODEL=...` imports ahead of time for large tables.
370
+ - **ES-NoProj SQL aggregates computed from streams only, with SQL semantics** —
371
+ `count(column)`, `sum`, `average`, `minimum`, `maximum` and `calculate` on an ES-NoProj
372
+ relation run over the records rebuilt from streams (rows that predate Lyra included, via
373
+ Genesis). `sum` used to go through `to_f` (losing decimal precision) and now keeps the
374
+ column's type, returning 0 over no rows; `average` returns a `BigDecimal` for integer and
375
+ decimal columns, as ActiveRecord does; `count(column)` counts non-NULL values, where it
376
+ used `present?` and dropped `""` and `false`. An SQL expression (`sum("price * qty")`)
377
+ raises `UnsupportedQuery` instead of being guessed. Known difference: `average` keeps
378
+ every digit of the quotient, while PostgreSQL rounds a numeric `AVG` to about 16
379
+ significant digits.
380
+ - **ES-Lazy: event sourcing that projects on read** (`projection_mode :lazy`, in
381
+ event-sourcing mode; `Lyra::Projections::LazyProjection`, `Lyra::Interceptors::LazyReads`)
382
+ — writes store events only, as in ES-NoProj; before any database read, the tables are
383
+ brought up to date from the event log, and the read runs as real SQL, so joins, merged
384
+ relations, SQL fragments and aggregates all work. The log stays the only source of
385
+ truth; the tables are a cache that `Rebuild` can recreate. Events are applied in the
386
+ log's global order under a PostgreSQL advisory lock, against a checkpoint that also
387
+ tracks ids still in flight (filled by replaying the record when they commit, forgotten
388
+ after 5 minutes). Read-your-writes consistent. Needs no migration: the checkpoint table
389
+ is created on first use. It runs Solidus, which ES-NoProj cannot: on 20 real Olist
390
+ orders it matches plain ActiveRecord with DualView clean. Cost: every read first checks
391
+ the log (about half ES-Sync's throughput in the replays).
392
+ - **`Lyra::EventSerializer`** — a JSON serializer for RailsEventStore that keeps time
393
+ precision. Ruby's `JSON` writes `Time` with `Time#to_s`, dropping fractional seconds, so
394
+ apps passing `serializer: JSON` stored `09:51:15.304` as `"09:51:15 UTC"`, and an update
395
+ from `.304` to `.352` as no change at all. `Lyra::EventSerializer` writes times as ISO 8601
396
+ with microseconds and `BigDecimal` as its exact decimal string; `load` is plain
397
+ `JSON.parse`. Opt in with
398
+ `EventRepository.new(serializer: Lyra::EventSerializer)`. Lyra's default client (RES's
399
+ YAML serializer) was not affected. Found by replaying the BPI Challenge 2017 log
400
+ (`examples/bpi2017_loan_app`), whose timestamps carry milliseconds.
401
+ ### Changed
402
+ - **Documentation** (*after the tag*) — `docs/PERFORMANCE.md` gives the 0.9.0 measurement (three runs, with
403
+ controls for PaperTrail and for the lock on the aggregate), the SQL statements per write under the
404
+ lock, and the published runs (https://doi.org/10.5281/zenodo.23640846); the install instructions
405
+ take the gems from RubyGems (`"~> 0.9"`); the case studies, benchmark harness and formal-model
406
+ scripts are referred to as the replication package (https://doi.org/10.5281/zenodo.17770613).
407
+ Comments no longer cite issue numbers; `lib/` changes in comments only.
408
+ - **The Rakefile's testbed tasks** (*after the tag*) say where the testbed is when it is absent, and `stats:all`
409
+ skips it.
410
+ - **ES-NoProj reads**: a stale cache entry is brought up to date from the events after its stamp
411
+ instead of a full replay; cache entries have a process-local copy in front of the shared cache;
412
+ an association's records are read when first needed, and a reference-link lookup intersects
413
+ every indexed key and takes candidate ids from the stream table; `Model.new` reads no record.
414
+ A replay of 640 Olist orders through Solidus took 855 s where it took 2,229 s, with the same
415
+ end state.
416
+ - **`ModeTransition.to!` links before switching into ES-NoProj.** Entering `event_sourcing`
417
+ with `projection_mode: :disabled` first runs a full reference-link pass over the monitored
418
+ models, so lookups are complete from the first one.
419
+ - **Documentation regrouped.** `docs/ADOPTION.md` and `docs/MODE_TRANSITIONS.md` are now
420
+ sections of `docs/MIGRATION_GUIDE.md` (Adopting Lyra in an existing application; Switching
421
+ modes); `docs/ORFEAS_FRAMEWORK_OVERVIEW.md` is the opening of `docs/ARCHITECTURE.md` (About
422
+ ORFEAS); `docs/PROJECTIONS_DATA_ACCESS_MONITORING.md` is split between `ARCHITECTURE.md`
423
+ (projections, write-side monitoring) and `docs/PRIVACY_COMPLIANCE.md` (read-side monitoring).
424
+ Claims were checked against the code on the way. `docs/PERFORMANCE.md` no longer presents the
425
+ withdrawn July 2026 figures: it explains each mode's cost in SQL statements per write, and
426
+ the throughput is being re-measured.
427
+ - **The workflow generator's mode nets name what Lyra does.** Their steps were labelled with
428
+ methods that do not exist (`CommandHandler.validate`, `CommandHandler.execute`,
429
+ `Projection.apply`, `EventStore.append_to_stream`) or with `Aggregate.apply(command)`, and
430
+ the Monitor net called the event store through `Lyra::Event.publish`. Each step now names the
431
+ method Lyra calls, in the order the trace conformance test observes; the generator no longer
432
+ claims to introspect the implementation (the mode nets are written by hand). The shipped
433
+ `app/workflows/*_mode_workflow.rb` files are regenerated.
434
+ - **Ruby 4.0 or later is required** (`required_ruby_version >= 4.0`, was `>= 3.4.5`); CI tests
435
+ Ruby 4.0. The project runs 4.0.7.
436
+ - **Article 30 register (PAM)** — `pam_dsl:report:article_30` states only what the policy declares (data subjects, recipients, transfers, security measures) and lists the rest as not declared; see `gems/pam_dsl/CHANGELOG.md`.
437
+ - **PII detection (PAM)** — Lyra's detector fallback (`Lyra::Privacy::PIIDetector`, `PolicyIntegration`) uses PAM's merged dictionary: more personal columns found (street, postal code, IP columns, card digits, account tokens), foreign keys and generic `*_name` columns no longer flagged. See `gems/pam_dsl/CHANGELOG.md`.
438
+ - **Hijack fails a write whose event cannot be stored with `EventStoreUnavailableError`**, as event
439
+ sourcing does, instead of a refused save (`save` returning false with the store's message on
440
+ `errors[:base]`). A store failure is not a validation error; `save` now raises it, as it raises
441
+ database errors.
442
+ - **ES-Async can run out of line in tests** — `async_projections_inline` now decides: unset
443
+ (the new default, `nil`) projects inline in the test environment only, as before; `true`
444
+ always; `false` never, so a test with an `:async` job adapter exercises the real background
445
+ path. The test environment used to force inline projection whatever the setting said, so no
446
+ test could run ES-Async as it runs in production. `Configuration#async_projections_inline?`
447
+ gives the effective value.
448
+ - **ES-NoProj cache: entries stamped with their stream's last event** (`CachedProjection`) — each
449
+ record's cached rebuild carries the id of the last event it was built from and is used only
450
+ while that is still the stream's last event, so it is exactly right or detectably out of date.
451
+ Collection reads (`all`, `where`, `find_by` on non-key attributes, `count`) are assembled from
452
+ those entries: one query for every stream's last event, one bulk cache read, and a replay of
453
+ only the streams that changed. They used to be cached as whole collections that every write
454
+ threw away, so after any write the next collection read replayed every stream of the model.
455
+ A `dependent: :nullify` delete, which must find the children by attribute, cost 1.7-3.5 s in
456
+ the Aegean smoke runs; it now costs 0.1-0.4 s (that cell: 10-16 → 49-101 ops/s). ES-NoProj
457
+ SQL statements per write: 12 → 11 per create, 9 → 8 per update (a write no longer deletes
458
+ collection keys; it reads its stream's last event). Cache entries are versioned `v2`; old
459
+ ones are ignored.
460
+ - **`Lyra.privacy_features_available?`** asks the installed privacy provider instead of
461
+ checking for `pam_dsl` directly. `PIIDetector` and `PolicyIntegration` load always and work
462
+ through the interface; `PIIMasker` and `GDPRCompliance` stay PAM-only.
463
+ - **Hijack creates reserve an id only where it cannot collide** (`reserves_safely?`). Elsewhere
464
+ (an integer key without a PostgreSQL sequence, e.g. SQLite or MySQL) hijack falls back to the
465
+ `pending-<hex>` placeholder with a one-time warning, rather than guessing an id the database
466
+ may also hand out. On PostgreSQL nothing changes, and SQL statements per write are unchanged
467
+ in every mode.
468
+ - **Bulk writes on monitored models publish events** (see Added). Code that relied on
469
+ `update_all`/`delete_all`/`insert_all` leaving the event stream untouched should wrap them in
470
+ `Lyra.projection_write`. Bulk-write guards are on `ActiveRecord::Relation`
471
+ (`Lyra::StrictDataAccessRelation`); `Lyra::StrictDataAccessClassMethods` is removed.
472
+ - **ES-NoProj bulk writes are events.** With projections disabled the records exist only in
473
+ the stream, so `CachedRelation#delete_all`/`update_all` publish the events that are the write
474
+ (a `delete_all`d record used to reappear on the next read). `upsert_all` raises
475
+ `ArgumentError` there: its `ON CONFLICT` would check the table, not the stream. Cost: a
476
+ `dependent: :nullify` delete must find the children, which in ES-NoProj rebuilds every stream
477
+ of the child model.
478
+ - Instance bypass events (`update_column(s)`, `touch`, `delete`) go through
479
+ `BypassEvents.publish`, which also invalidates the ES-NoProj read cache.
480
+ - **ES-NoProj answers exactly or raises** (`Lyra::Projections::CachedRelation`, new
481
+ `Lyra::Projections::UnsupportedQuery`) — reading from the event store, `CachedRelation`
482
+ used to answer queries it could not evaluate anyway, with the wrong records: a
483
+ SQL-string `where` kept every record, unrecognised conditions (`!=`, ranges written as
484
+ SQL, ORs) were dropped, `joins` and `where.missing`/`where.associated` were ignored, a
485
+ condition on an unknown column matched nothing, and a scope that failed, carried
486
+ `order`/`limit`/`group` and the like, or was not a scope at all, returned the whole set.
487
+ Each now raises `UnsupportedQuery`, naming the query and the alternatives. Hash
488
+ conditions, ordering, limits, loading hints and scopes made of hash conditions work as
489
+ before. This found the Aegean benchmark's own cleanup, `where("email LIKE ?")` under
490
+ ES-NoProj, silently matching every registration.
491
+ - **RailsEventStore 3** — `lyra.gemspec` now requires `rails_event_store ~> 3.0` (3.2.0 in the
492
+ lockfile), on Ruby 4.0.7 and Rails 8.1.4 (8.1.3.1 and RES 3.1.0 until 2026-10-19). RES 3 moved
493
+ the ActiveRecord repository to
494
+ `RubyEventStore::ActiveRecord::EventRepository` and the event base class and errors to
495
+ `RubyEventStore`; `Lyra::Event`, `Lyra::Aggregate` and the default client follow. Apps
496
+ that build their own client must switch from `RailsEventStoreActiveRecord::EventRepository`.
497
+ - **`IdGenerator` caches the PostgreSQL sequence name** — `pg_get_serial_sequence` ran on
498
+ every reserved ID, a round trip whose answer never changes. Since hijack mode now reserves
499
+ its IDs here too, that was one extra SQL statement on every hijack-mode create (two
500
+ instead of one). The name is now looked up once per table and cached; only `nextval`
501
+ runs per ID.
502
+
503
+ ### Removed
504
+ Dead code, none of it on the write path. Initializers that set `config.event_backend`, or models
505
+ that pass `command_handler:`, must drop it: the former now raises `NoMethodError`, the latter is
506
+ ignored like any unknown option.
507
+ - **`Lyra::EventMapper` and `Lyra::AuditMapper`** (`lib/lyra/event_mapper.rb`). The interceptor
508
+ and the command handler build their events themselves (`CrudInterceptor#publish_event`,
509
+ `CommandHandler#create_event`, both through `Lyra::DomainEvents.build`); nothing called the
510
+ mappers.
511
+ - **`Lyra::EventStoreAdapter`, `RailsEventStoreAdapter`, `CustomEventStoreAdapter` and
512
+ `config.event_backend`**, read only by `EventStoreAdapter.build`. Lyra uses the RailsEventStore
513
+ client in `config.event_store` directly; `Lyra.append_events` and `EventStoreUnavailableError`
514
+ stay.
515
+ - **The `command_handler` model option** (`monitor_with_lyra command_handler:`,
516
+ `ModelConfiguration#command_handler`) and its dashboard line. It was stored and shown, but the
517
+ write path always uses `Lyra::CommandHandler`.
518
+
519
+ ### Fixed
520
+ - **`trace_conformance_test.rb` on a fresh database** (*after the tag*) — the recorder took the commit of Lyra's own
521
+ checkpoint table, created on a connection of its own, for part of the write; it now records only
522
+ the write's connection. Test only.
523
+ - **A thread that wrote through Lyra kept a database connection until it ended.** Lyra
524
+ called `Model.connection` (Rails' `lease_connection`) in its write, lock and read paths: the
525
+ aggregate lock, the id generator, Genesis, ES-Async's job, ES-NoProj's cache and reference links,
526
+ ES-Lazy's catch-up, and its maintenance calls (DualView, Rebuild, mode checks, erasure). Since
527
+ Rails 7.2 that keeps the connection for the thread, even inside `with_connection`, which then does
528
+ not return it. So in Hijack and every event-sourcing mode, a thread of the application's own (a
529
+ job runner, a worker pool) ended still holding one, and the pool got it back only by reaping: the
530
+ first thread to find the pool full reset every dead thread's connection, one by one, while the
531
+ others waited. In the benchmark's 64-thread cells such a wait now and then passed the 5 s checkout
532
+ timeout (0.3 to 3 operations per trial of 12,800 in Hijack, Hijack+PT and ES-Sync; part of
533
+ ES-NoProj's 14 to 18), and the contention benchmark reset each round's connections inside its
534
+ timed window. Lyra now uses `with_connection` throughout: inside a transaction it is the
535
+ transaction's own connection, outside one it is returned after use. `OwnConnection`'s thread
536
+ returns whatever its work leased. `test/connection_release_test.rb`: in every mode, a thread that
537
+ writes and reads, bare or in `with_connection`, ends without a connection, and `lib/` has no
538
+ `Model.connection` call left.
539
+ - **Genesis and the reference-link catch-up assumed a second connection.** Inside an open
540
+ transaction both ran their work on a thread that checked out a connection of its own and waited
541
+ for it. Under Rails' transactional tests the pool's connection is pinned, every checkout returns
542
+ the caller's own, locked connection, and the wait never ended: the Aegean testbed suite in
543
+ ES-NoProj deadlocked at its first indexed lookup. Under load each caller took a second
544
+ connection, and at 64 threads the pool ran out (5 s checkout timeouts, about 0.1% of ES-NoProj's
545
+ operations in the benchmark's 64-thread cells). `Lyra::OwnConnection` now runs such work: on a
546
+ connection of its own when one can be had, otherwise inline in the caller's transaction, with
547
+ the model recorded as linked or imported only once that transaction commits (after a rollback
548
+ the work runs again on the next use). The catch-up runs once per model and process; concurrent
549
+ first lookups wait for it without a second connection. `test/projections/reference_links_test.rb`:
550
+ a pinned connection, and concurrent first lookups inside transactions.
551
+ - **`has_and_belongs_to_many` links were not recorded.** The join table has no model of its own
552
+ (Active Record writes it through an anonymous class), so `item.tags << tag` wrote the join row
553
+ and no event recorded it: a rebuild lost the association and DualView did not compare it. Every
554
+ insert into and delete from a HABTM join table is now an `AssociationLinked` or
555
+ `AssociationUnlinked` event on the owner's stream, linked into each monitored associated
556
+ record's stream (`Lyra::Associations::HabtmLinks`), in every mode. DualView compares each HABTM
557
+ association's links with the join table, `Rebuild` restores join tables from the events, and
558
+ Genesis records the links pre-existing rows had. Under ES-NoProj, HABTM writes failed before
559
+ reaching Lyra (`CachedRelation` lacked `where_values_hash`); they now work and read back
560
+ (`test/habtm_links_test.rb`, every mode).
561
+ - **Optimistic locking: a stale update silently won under event sourcing.** Active Record checks
562
+ `lock_version` in the UPDATE or DELETE that writes the row; ES-Sync, ES-Async, ES-NoProj and
563
+ ES-Lazy skip that statement, so a stale `update` or `destroy` overwrote the newer write (a lost
564
+ update), and `touch` raised `StaleObjectError` on every call under ES-NoProj, where no row
565
+ matches. No mode's events recorded the version's increment, so the log replayed to a
566
+ `lock_version` the row did not have (DualView: `lock_version: { crud: 1, event_sourced: 0 }`).
567
+ Every write's event now records the increment, `touch` included; the event-sourcing modes check
568
+ the version against the log before appending, under the lock on the aggregate (taken for such a
569
+ model in every event-sourcing mode), and raise `ActiveRecord::StaleObjectError` as Active Record
570
+ does (`Lyra::OptimisticLocking`). A record that expects a newer version than the log's passes,
571
+ so streams written before the change keep working (`test/optimistic_locking_test.rb`, every
572
+ mode, concurrent writers included).
573
+ - **Encrypted attributes (Active Record `encrypts`) reached the event log in clear** while their
574
+ row held ciphertext: events are built from attributes Active Record has already decrypted.
575
+ `Lyra.append_events`, through which every event passes, now stores those attributes' values (in
576
+ `attributes` and `changes`) as the ciphertext Active Record writes to the column
577
+ (`Lyra::EncryptedAttributes`); projections, rebuilds and DualView decrypt them, and records
578
+ instantiated from events are decrypted by Active Record. Events written before keep their clear
579
+ values and still read; rows written from them are encrypted. No mode leaks the value, no value is
580
+ encrypted twice (`test/encrypted_attributes_test.rb`).
581
+ - **ES-Lazy stopped projecting after the event store was emptied or restored without its
582
+ checkpoint table**: the checkpoint then lay past the newest event, and every new event looked
583
+ applied. A checkpoint past the newest event is now recognised, warned about and reset (applying
584
+ is idempotent). It made two tests fail in some orders, since the test suite reloads the event
585
+ tables but not the checkpoint table (`test/projections/lazy_projection_test.rb`).
586
+ - **Single-table inheritance: a record's events were filed under its concrete class's stream**
587
+ (`Spree::Calculator::Shipping::FlatRate$1`), and anything given the base class read
588
+ `Spree::Calculator$1` and found none: DualView reported the row as missing from the log, and
589
+ under ES-NoProj, ES-Lazy or a rebuild the base class saw an empty stream. A record's stream is
590
+ now named after the class that owns the table (`Lyra.stream_name`, `Lyra.stream_prefix`), at
591
+ every one of the 21 places that built it; reference links and Lyra's caches follow the table.
592
+ Under ES-NoProj a subclass reads only its own records and its subclasses', as ActiveRecord's
593
+ type condition does; events naming a subclass of a monitored model resolve to it
594
+ (`Configuration#monitored_model_named`, used by ES-Lazy's catch-up and erasure). Event types
595
+ still name the concrete class. **Upgrade**: `bin/rails lyra:streams:migrate_sti`
596
+ (`Lyra::StiStreamMigration`) links events written before into the base streams, once; it
597
+ reports a base stream that already holds newer events instead of linking out of order
598
+ (`test/sti_streams_test.rb`, every mode).
599
+ - **ES-Async projected inline could leave the row behind the log.** With
600
+ `async_projections_inline` (the test environment's default) a write applied only its own event,
601
+ inside its transaction and under no lock: of two writers of one record, the one that projected
602
+ last could leave the row at its older value. Inline now runs what `AsyncProjectionJob` runs, in
603
+ the writer's thread once its transaction has committed, under the per-stream lock. The job path
604
+ was not affected (`test/concurrent_same_record_test.rb`).
605
+ - **ES-NoProj: a named scope called on an association was ignored** (`order.payments.completed`
606
+ returned every payment) and its body loaded every record of the model; records an association
607
+ loaded did not point back at their owner instance.
608
+ - **Documentation: `ReadYourWrites` projects a block's writes when the block ends**, so a read
609
+ after the block sees them and a read inside it does not; `docs/API_REFERENCE.md` said or
610
+ implied the latter.
611
+ - **ES-Lazy: concurrent first reads no longer fail creating the checkpoint table.** Two threads
612
+ that both found `lyra_projection_checkpoints` missing both created it, and the second `CREATE`
613
+ failed the write that triggered it (one write of the full BPI 2017 replay at two threads).
614
+ `Lyra.create_own_table` now defines the table in one transaction and ignores a failed create
615
+ once the table exists.
616
+ - **Leaving ES-Lazy skipped its catch-up** — `ModeTransition.to!` from `event_sourcing/lazy`
617
+ calls `LazyProjection.catch_up!(force: true)`. Without `force:`, `catch_up!` returns 0 outside
618
+ ES-Lazy, and the mode had already changed by the time the transition ran it, so the tables
619
+ could be left behind the log.
620
+ - **Locks on databases other than PostgreSQL** — Genesis, ES-Lazy catch-up and ES-Async
621
+ projection take their transaction-level advisory lock through `Lyra::AdvisoryLock.xact_lock`.
622
+ On SQLite (which serialises writers) it does nothing; on other adapters it logs once per
623
+ purpose that the step runs unlocked and is safe in a single process only. Before, those
624
+ adapters ran unlocked silently.
625
+ - **`upsert_all` under ES-NoProj raised `ArgumentError`**; it now raises
626
+ `Lyra::Projections::UnsupportedQuery`, like every other query ES-NoProj cannot answer, naming
627
+ the reason (the `ON CONFLICT` check runs against the table, not the event stream) and the
628
+ per-record alternative.
629
+ - **`DualView` reported a record for a stream of only `replay: false` events** — such a stream
630
+ describes no record, so `exists` is now false.
631
+ - **`lyra.rb` required ActiveRecord and ActiveJob implicitly**, relying on the host to load them
632
+ first; loading the gem after only `require "rails"` raised `NameError`. It now requires both.
633
+ - **`Lyra::Projection.subscribe_to` raised `RubyEventStore::InvalidHandler`** — RailsEventStore
634
+ 3.1 subscribers must respond to `call`. A projection class now does (`call(event)` hands the
635
+ event to a new instance's `handle`), and `subscribe_to` returns the unsubscribe procs.
636
+ `handle` takes the `apply_<name>` method from `event.event_type`, so an event read back as a
637
+ plain `RubyEventStore::Event` reaches its handler too.
638
+ - **Nested `with_guaranteed_read` lost the outer block's writes** — the inner block reset the
639
+ list of pending writes, then cleared it. An inner block now projects its own writes when it
640
+ ends and restores the outer list; if it raises, its unprojected writes pass to the outer
641
+ block. Every write is projected by the time the outermost block ends.
642
+ - **A process that only reads got plain `RubyEventStore::Event`s** — in a lazily loaded process
643
+ (console, rake, development) no event class was registered, so events came back without
644
+ Lyra's readers (`operation`, `attributes`, `changes`, ...). `Lyra::Events` now defines the
645
+ class on first reference for a name a monitored model or the stored schema writes, loading
646
+ the model from the name's stem (`PostCreated` → `Post`, `SpreeOrderCreated` →
647
+ `Spree::Order`); other names still raise `NameError`. A model with a custom `event_prefix` or
648
+ `event_mapping` that is not loaded resolves only through the schema store.
649
+ - **petri_flow was not optional** — `app/workflows/*.rb` subclass `PetriFlow::Workflow`, so
650
+ without the gem eager loading (production boot, `lyra:mode:*`, `lyra:repair`,
651
+ `lyra:schema:*`) raised `NameError`. Without petri_flow the engine neither autoloads nor
652
+ eager-loads `app/workflows`.
653
+ - **`lyra:workflows:generate MODE=...` wrote a class its file did not name** — `MODE=monitor`
654
+ defined `MonitorModeWorkflowWorkflow`, which Zeitwerk could not load. A mode's file and class
655
+ are now the same whether one mode or all are generated (`monitor_mode_workflow.rb` defines
656
+ `MonitorModeWorkflow`, `es_sync_mode_workflow.rb` `EsSyncModeWorkflow`;
657
+ `WorkflowGenerator.workflow_file_basename(mode)`, `.workflow_class_name(mode)`).
658
+ - **`already initialized constant Lyra::VERSION` in a checkout holding a nested copy of the
659
+ engine** — Bundler's default gemspec search also loaded the nested copy's `lyra.gemspec`. The root `Gemfile`
660
+ uses `gemspec glob: "lyra.gemspec"`, and the example apps' Gemfiles `glob: "lyra.gemspec"`.
661
+ - **Hijack and event-sourcing events lost who wrote them** — the command handler wrote only
662
+ the causal chain, so `user_id`, `request_id`, the user action and `config.metadata_proc`
663
+ reached Monitor events alone. Every mode now carries the same attribution metadata
664
+ (`record.lyra_event_metadata(operation)`), plus `source: "lyra_command_handler"`.
665
+ - **`ReadYourWrites` had no effect under ES-Async** — only the ES-Sync path recorded writes
666
+ for it. Writes inside `with_guaranteed_read` are now projected when the block ends; the job
667
+ is still enqueued and converges on the same row.
668
+ - **Aggregates never loaded their history** — `GenericAggregate.load` raised `NoMethodError`
669
+ (no model class for its stream name) and the command handler swallowed it. A model's own
670
+ `aggregate_class` is now loaded with its stream's history for updates and destroys; the
671
+ default `GenericAggregate`, which decides nothing from history, starts empty and reads no
672
+ stream, so its writes issue no extra SQL. A programming error while loading now fails the
673
+ command instead of being swallowed.
674
+ - **`AuditProjection.audit_trail` always gave `user_id: nil`** — it read the user from the
675
+ event data, which no longer holds the metadata; it reads the event's metadata now.
676
+ - **DualView reported every destroyed record as `exists_mismatch`** — a destroyed record
677
+ whose row is gone now compares clean (`event_sourced_state` gains `destroyed:`).
678
+ - **The engine's root redirect was hard-coded to `/lyra/dashboard`** — it is now relative
679
+ to wherever the engine is mounted.
680
+ - **`strict_schema` stopped the boot on any schema drift** — including info-level changes
681
+ such as an added column. It now refuses to boot only on a breaking change (model or column
682
+ removed, column type or event name changed) and logs the rest.
683
+ - **`PIIMasker.mask_events` was broken** — it built events with keywords
684
+ `RubyEventStore::Event` does not accept. It returns masked copies now (same class, id and
685
+ metadata; attributes and changes masked).
686
+ - **`Erasure::Result#shared_values` was sometimes `[]`, sometimes an Integer** — always an
687
+ Integer count now.
688
+ - **`EventFlow#crud_to_event_mapping` found nothing for a class** — it accepts a class or its
689
+ name, and string operations and ids (as the dashboard passes them); `data_lineage` likewise.
690
+ - **A test left a guard on `Article#body=`** (`multi_mode_integration_test`), failing 11 later
691
+ tests under some seeds; it now removes it.
692
+ - **ES-NoProj answers more queries exactly instead of refusing them** — scopes with `order` or
693
+ `limit` (their conditions, then the order and limit); `relation.or(other)` over the same model
694
+ (the union); and `where` fragments of `col OP ?` terms joined all by AND or all by OR, with OP
695
+ one of `= != <> < <= > >= LIKE ILIKE` (values cast to the column type, NULL never matching, LIKE
696
+ patterns with `%`, `_` and `\` escapes). Anything else is still refused. `order("col DESC")`
697
+ strings and Arel orderings are read, and an ordering it cannot read now raises: it used to
698
+ compare as equal and leave the result silently unordered. The testbed suite under ES-NoProj
699
+ goes from 20 failures to none (the domain-events tests also created their records before
700
+ switching mode, so under ES-NoProj the parents existed only as events).
701
+ - **ES-NoProj lost `update_columns` and `touch`** — in the events-only store the `UPDATE` matched no
702
+ row and returned false, so no event was published and the change was lost, against Bypass
703
+ Coverage ("the events are the write"). The event is now published whatever the row count.
704
+ - **ES-Lazy lost a callback-bypassing write made right after a write** — a write appends events
705
+ only, so the record's row did not exist until the next read; `update_columns`, `update_all`
706
+ or `delete_all` on it matched nothing, recorded no event, and the change was lost.
707
+ `BypassEvents.atomically` now brings the tables up to date first, as a read does.
708
+ - **Lyra's own tables vanished with a rolled-back transaction** — `lyra_projection_checkpoints`
709
+ and `lyra_mode_transitions` were created inside whatever transaction was open, and remembered
710
+ as created; when it rolled back (a failed request, every transactional test) the table was
711
+ gone and every later use failed. Under ES-Lazy the testbed suite failed 395 tests this way.
712
+ They are now created on a connection of their own when a transaction is open
713
+ (`Lyra.create_own_table`).
714
+ - **ES-NoProj ignored `select` and hid purpose refusals as "not found"** — `CachedRelation#select`
715
+ returned whole records, and the record builder swallowed a policy error, dropping the record.
716
+ Query results are now narrowed to the selected columns and checked when returned
717
+ (`CachedRelationDelivery`), as are the class-level `find`, `find_by` and `find_by!`.
718
+ - **The all-modes runner skipped ES-Lazy** — `lyra:test:all_modes` and the PAM-less run now cover
719
+ all seven configurations.
720
+ - **Lyra's rake tasks ran twice, and host apps got the development checkout's maintenance tasks** — Rails
721
+ loads every `lib/tasks/*.rake` under an engine's root, and the engine also loaded its three
722
+ task files explicitly, so each `lyra:mode:*`, `lyra:projections:*` and `lyra:schema:*` task
723
+ ran twice (`lyra:mode:check` ran its full check twice), and `gems:*`, `public:*`, `stats:*` and
724
+ the other maintenance tasks appeared in every host app. The engine now loads `lyra_*.rake` only,
725
+ once.
726
+ - **`lyra:mode:check` could certify a switch having checked nothing** — monitored models register
727
+ as their classes load, which a rake task in development does not do, so the check saw no
728
+ models, found no discrepancy and certified the switch. The mode, repair, projection and schema
729
+ tasks now load the application's models first; check and repair refuse when none are monitored.
730
+ - **Renamed events were skipped on replay** — `Rebuild`, the ES-NoProj cache and the aggregate
731
+ decided what an event did from the end of its name (`…Created`, `…Updated`, `…Destroyed`).
732
+ An event renamed with `event_mapping` (or, now, a domain event) was skipped: `Rebuild` left
733
+ the record's row empty and ES-NoProj could not read it. Every replay site now dispatches on
734
+ the operation each event records in its data (`Lyra::Event.operation_of`); only events too
735
+ old to carry one fall back to the name.
736
+ - **Event sourcing lost writes whose event could not be stored** — `lyra_store_events` logged a
737
+ failed append and carried on (unless `strict_projections`, a setting about projections, was
738
+ on): ES-NoProj reported a write that existed nowhere, and ES-Sync projected a row from an
739
+ event that was never stored. In Hijack and the event-sourcing modes a write whose event
740
+ cannot be stored now fails and is rolled back, bypass writes included
741
+ (`BypassEvents.required?`, `BypassEvents.atomically`: the write and its events share a
742
+ transaction). Monitor, where the table stays authoritative, still logs the failure and keeps
743
+ the write.
744
+ - **A failed event-sourced create made the next insert on its thread vanish** — the
745
+ skip-insert signal was cleared only at the end of a successful finalize, so after any
746
+ failure the next insert of any model on that thread (a Puma thread, say) skipped its row while
747
+ reporting success. It is now cleared as soon as the record's own INSERT has run, and
748
+ finalize clears its state on every exit.
749
+ - **ES-NoProj evaluated association conditions record by record, and some wrongly** —
750
+ `where(program: x)` loaded each cached record's association (one query per record: about
751
+ 10,000 for one `count` in the mode-comparison benchmark) and compared objects;
752
+ `find_by(program: x)` compared against a `program` attribute the record does not have and
753
+ silently found nothing; `where(program_id: record)` matched nothing. Conditions are now
754
+ rewritten as ActiveRecord does (`Lyra::Projections::AssociationConditions`): a `belongs_to`
755
+ becomes its foreign key (polymorphic: type and id), and a record given as a column value
756
+ becomes its id. A `has_many`/`has_one` condition, which ActiveRecord accepts only with a
757
+ join, raises `UnsupportedQuery`, as does a polymorphic condition over several types.
758
+ - **ES-NoProj could serve out-of-date records from its cache** — a write warmed its record's
759
+ entry inside the write's transaction, so a rollback left a record cached that never existed;
760
+ two processes filling one key could leave a stale entry for up to an hour; and
761
+ `where`/`find_by` results were cached for five minutes and never invalidated, so they could
762
+ return records a write had already changed. Stamped entries (see Changed) make all three
763
+ impossible: a stale entry is never used.
764
+ - **An explicit id was replaced** — `CommandHandler#handle_create` generated an id even when the
765
+ application set one; it now keeps it, in the event and in the row. From `thesis-restructure`
766
+ (526908e), applied to master's own Hijack fix (cd3ea89).
767
+ - **ES-NoProj `insert_all` ran, then raised** — through `CachedRelation`'s scope branch the
768
+ insert executed and the caller then got `UnsupportedQuery` ("not a scope"). Inserts and
769
+ upserts now go to the table relation and return its result.
770
+ - **Genesis left a cached "not found" in place on Solid Cache** — an ES-NoProj lookup
771
+ made before a record was imported (in an earlier process, or before the event store
772
+ was reset) caches `nil` for it. Genesis cleared the cache with
773
+ `CachedProjection.invalidate_all`, which cannot delete per-record entries on a store
774
+ without prefix deletion (Solid Cache, the Rails 8 default), so the imported record stayed
775
+ "not found" until the entry expired, up to an hour. Genesis now drops each imported
776
+ record's entry by id. Found by the Aegean benchmark harness, which empties the event
777
+ store between modes.
778
+ - **`CachedRelation#delete_all`/`destroy_all`/`update_all` wiped the whole table**
779
+ (`Lyra::Projections::CachedRelation`, from `stack/latest-ruby-rails-res`) — the
780
+ `method_missing` scope fallback rebuilt them from `model_class.unscoped` and ran them
781
+ before checking the result, so `Model.where(id: ids).delete_all` under ES-NoProj issued
782
+ a bare `DELETE FROM` the whole table (found when the Aegean benchmark's ES-NoProj
783
+ cleanup emptied `registrations` mid-sweep). They now act only on the relation's own
784
+ records. `where(column: [a, b])` (an `IN` with several values) is no longer silently
785
+ dropped.
786
+ - **Projection wrote through the read-from-events override** (`Lyra::Projections::ModelProjection`)
787
+ — with `projection_mode :disabled` (ES-NoProj) a monitored model's `where`/`all` answer
788
+ from the event store. Projection's update and destroy go through
789
+ `model_class.where(...)`, so they acted on event-store records instead of rows: a
790
+ record destroyed in the log is absent there, so its row was never deleted.
791
+ `Rebuild` (`rake lyra:projections:rebuild`) and `AsyncProjectionJob` both project
792
+ through this path in any mode. Projection now always addresses the table.
793
+ - **`invalidate_all` raised on Solid Cache, and keyed `find_by` replayed every stream**
794
+ (`Lyra::Projections::CachedProjection`) — `invalidate_all` guarded prefix deletion with
795
+ `respond_to?(:delete_matched)`, which every cache store answers true; Solid Cache, the
796
+ Rails 8 default, then raised `NotImplementedError`. The raise is now rescued and the
797
+ fallback used. `find_by` took its single-stream path only when the primary key was the
798
+ sole condition, so `find_by(id: 5, email: x)` replayed every stream of the model; any
799
+ lookup that pins the primary key now reads that one stream and checks the remaining
800
+ conditions against the record it yields.
801
+ - **ES-Async projected a record's events out of order** (`Lyra::Projections::AsyncProjectionJob`)
802
+ — each job applied its one event, and a worker pool runs jobs concurrently, so an
803
+ earlier update's job could land after a later one's and roll the row back; a late
804
+ create job could even resurrect a destroyed record. The Olist replay through Solidus
805
+ left 2 of 100 orders at `confirm` whose events ended at `complete`. Each job now
806
+ brings its record up to date with the whole stream (`Rebuild.replay_record`, in place),
807
+ under a PostgreSQL advisory lock held per stream, so jobs converge in any order. Cost:
808
+ each job re-reads its stream, so projections lag more and ES-Async's read-after-write
809
+ window widens (Olist, 100 orders: 151 -> 183 such failures).
810
+ - **Hijack and event-sourcing modes built events before the model's own callbacks**
811
+ (`Lyra::Interceptors::CrudInterceptor`, new `WriteHooks`) — the create/update/destroy
812
+ commands ran in `before_*` callbacks registered on `ActiveRecord::Base`, so ahead of
813
+ every model's own `before_*` callbacks. Values those set were missing from the event:
814
+ in Hijack mode the event disagreed with the row; under event sourcing, where the row is
815
+ projected from the event, they were lost from the table, and DualView could not tell
816
+ (both sides agreed). Replaying Olist orders through Solidus 4.7 under ES-Sync lost every
817
+ order's `guest_token` and every payment's `number`. The commands now run in
818
+ `WriteHooks`, prepended to `ActiveRecord::Persistence`: after all `before_*` callbacks,
819
+ just before the SQL write. A failed command returns false (save returns false, the
820
+ transaction rolls back) instead of throwing `:abort`. PaperTrail is still switched off
821
+ first. SQL statements per write are unchanged in every mode, in order and count.
822
+ - **Hijack creates re-assigned every attribute** (`Lyra::Interceptors::CrudInterceptor`) —
823
+ after the create command, the interceptor called `assign_attributes` with all of the
824
+ record's attributes, although only the reserved ID had changed. That fails on models
825
+ that guard a writer: Solidus makes `StockItem#count_on_hand=` unusable so stock moves
826
+ only through `set_count_on_hand`, and Hijack mode could not create a variant. Only the
827
+ ID is assigned now.
828
+ - **Writes that bypass callbacks went missing from the event log** (`Lyra::StrictDataAccess`)
829
+ — three holes, found by the Olist replay through Solidus 4.7, where DualView then
830
+ disagreed with the table for 17 of 20 orders and every shipment:
831
+ - *Assign, then `update_columns`* published nothing. The old value was read from the
832
+ in-memory attribute, which already held the new value, so no change was seen
833
+ (Solidus's `Shipment#persist_amounts`). The old value now comes from
834
+ `attribute_in_database` and the new one from the record after the write, which also
835
+ covers the timestamps `touch: true` sets.
836
+ - *`touch`* was not intercepted at all (Solidus records completion with
837
+ `touch(:completed_at)`). It now publishes a bypass event; it is not a strict-mode
838
+ violation, since Rails itself calls it for `belongs_to ... touch: true`.
839
+ - *`update_column`* published its event twice (Lyra's override and Rails'
840
+ `update_columns` underneath both published), and the override dropped Rails 8's
841
+ `touch:` keyword, so callers passing it raised `ArgumentError`.
842
+ - **Hijack and event-sourcing modes failed on namespaced models** (`Lyra::CommandHandler`) —
843
+ the event class name was built from the model name without stripping `::`, so
844
+ `Spree::Price` produced the invalid constant `Spree::PriceCreated` and every create
845
+ failed. The failure surfaced only as a parent record left unsaved, with no exception
846
+ (`Spree::Product.create!` returned an unsaved product). Every other event path already
847
+ stripped the separator; this one now does too. Found by the Olist replay through
848
+ Solidus 4.7, where it, not STI or polymorphism, kept Solidus out of these modes.
849
+ - **Optional gems that failed to load were reported as missing** (`lyra.rb`,
850
+ `Lyra::OptionalDependency`) — PAM DSL and PetriFlow were loaded inside a bare
851
+ `rescue LoadError`, so a gem that was installed but could not load (PetriFlow without
852
+ `rexml`) was treated as absent and left half-defined; eager-loading `app/workflows`
853
+ then failed with `uninitialized constant PetriFlow::Workflow`. Only a gem that is itself
854
+ not installed now counts as unavailable; any other load failure raises. PetriFlow now
855
+ declares `rexml` (see `gems/petri_flow/CHANGELOG.md`).
856
+ - **DualView compared times at whole seconds** (`Lyra::DualView`) — `normalize_value` turned
857
+ every time into `iso8601` with no fraction, so `15:49:11.420` in the table and
858
+ `15:49:11 UTC` in a lossy event compared equal. That is how the sub-second loss fixed by
859
+ `Lyra::EventSerializer` passed dual-view checks under Monitor mode. Times now compare at
860
+ microseconds, the resolution of a PostgreSQL timestamp.
861
+ - **Hijack mode filed `Created` under a placeholder stream** (`Lyra::CommandHandler`) — for
862
+ integer primary keys, hijack mode stored the Created event under `"pending-<hex>"` and let
863
+ the database assign the ID afterwards. Nothing linked the two: later events went to the
864
+ real `Model$<id>` stream, so every record's history began with an update and rebuilding it
865
+ from events (dual view, projection rebuild) found no attributes. Hijack now reserves the
866
+ ID up front through `IdGenerator`, as event-sourcing mode does, and the row is inserted
867
+ under the ID the event carries. Hijack is now covered by the event-correctness
868
+ integration tests it had been left out of.
869
+ - **ES-Async lost projections silently** (`Lyra::Projections::AsyncProjectionJob`) — the job
870
+ was enqueued inside the transaction that stores its event, so a worker could run it
871
+ before the commit, find no event, complete, and never project the row (no error, no
872
+ retry). Replaying 40 BPI 2017 applications left 37 missing. The job now sets
873
+ `enqueue_after_transaction_commit`, so it runs only once its event is visible and is
874
+ never enqueued after a rollback. Eventual consistency itself is unchanged. The test dummy
875
+ app now loads `active_job/railtie`.
876
+
10
877
  ## [0.8.0] - 2026-08-09
11
878
 
12
- The version the PAM article (Requirements Engineering, 2026) describes.
879
+ The version the PAM article (Requirements Engineering, 2026) describes, released from
880
+ lyra-engine commit `b863d2e`. Not published to RubyGems until 2026-10-10.
13
881
 
14
882
  ### Added
15
883
  - **Projection rebuild from the event log** (`Lyra::Projections::Rebuild`) - Reconstruct