orfeas_lyra 0.9.1 → 0.9.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +115 -1
- data/README.md +9 -2
- data/app/controllers/lyra/application_controller.rb +19 -0
- data/app/controllers/lyra/dashboard_controller.rb +16 -16
- data/app/controllers/lyra/flow_controller.rb +5 -4
- data/lib/lyra/bypass_events.rb +22 -3
- data/lib/lyra/consistency/read_your_writes.rb +23 -1
- data/lib/lyra/erasure.rb +29 -10
- data/lib/lyra/interceptors/crud_interceptor.rb +44 -9
- data/lib/lyra/mode_transition.rb +19 -3
- data/lib/lyra/privacy/gdpr_compliance.rb +57 -19
- data/lib/lyra/strict_data_access.rb +15 -4
- data/lib/lyra/version.rb +1 -1
- data/lib/tasks/lyra_mode.rake +5 -2
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: e172591829893991e3e30e9b33d23ef49eb7b6435e63b97532c12ad24c22492d
|
|
4
|
+
data.tar.gz: c8686bfb2b92fa8d6f584e1fbb0ed2e392cfe91d5b90e365f6148115c8485dd6
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 613d62bce3b4d58c286e9105cb607aab4eed680ce75407c906490a6b4a5800cb282c1541847f330b14c36403b2e196d540b67e84b56271ee7a02bcf058b0ac1d
|
|
7
|
+
data.tar.gz: 5ef783bb318bc64aafb0cbf3dcb033641fa8ccfb56811303791d0b3f7956a945f06c4f3960e99c551e2e5f9a0877ac4c0ef38530538fd61b241b0079feb85836
|
data/CHANGELOG.md
CHANGED
|
@@ -7,13 +7,127 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.9.3] - 2026-11-04
|
|
11
|
+
|
|
12
|
+
**Security update.** A thorough review of Lyra's code against the claims made for it found the
|
|
13
|
+
defects below. None of them touches the gradual migration itself: the modes, the interception
|
|
14
|
+
of writes, the mapping of operations to events, the checked transitions between modes, and the
|
|
15
|
+
results that rest on them are unchanged. The defects are in the dashboard, the GDPR helpers, the
|
|
16
|
+
hand-over from PaperTrail and the operator tasks. Upgrade if the dashboard is open to anyone, or if
|
|
17
|
+
the Art. 15/20 export, the retention check, `lyra:erase` or the policy generator is used. PAM DSL
|
|
18
|
+
moves to 0.9.3 for its policy generator, and PetriFlow to 0.9.3 for the way it runs Graphviz
|
|
19
|
+
(below).
|
|
20
|
+
|
|
21
|
+
### Security
|
|
22
|
+
- **The dashboard resolves only monitored models.** The routes that take a model
|
|
23
|
+
(`dashboard/compare`, `discrepancies`, `model_overview`, `audit_trail`, `model_heatmap`,
|
|
24
|
+
`event_list`, `flow/event_chain`, `flow/visualization`) called `constantize` on the request
|
|
25
|
+
parameter and returned the record's attributes. Any user the dashboard admitted could therefore
|
|
26
|
+
read the rows of any Active Record model, monitored or not, including password digests and
|
|
27
|
+
tokens. They now answer 404 for a model Lyra does not monitor. List sizes, day ranges and graph
|
|
28
|
+
depth are clamped (`limit` at most 1,000, 500 or 100 by route, `days` at most 90, `depth` at most
|
|
29
|
+
3). As before, the dashboard is closed outside development and test unless
|
|
30
|
+
`config.dashboard_authorization` admits the request.
|
|
31
|
+
- **The Art. 15/20 export gathers the subject's data, not the subject's edits.**
|
|
32
|
+
`Lyra::Privacy::GDPRCompliance` selected every event whose metadata named the subject as its
|
|
33
|
+
*actor*. A staff member's export therefore held the records they had edited, which is other
|
|
34
|
+
people's data, while a subject's own records written by staff could be missed. It now gathers
|
|
35
|
+
the events of the subject's own record, of records that refer to the subject by foreign key
|
|
36
|
+
(`<subject_type>_id`, e.g. `user_id`), and of events that name the subject explicitly. The
|
|
37
|
+
dashboard's subject timeline does the same; who did what remains the `user_actions` view.
|
|
38
|
+
- **PetriFlow runs Graphviz without a shell.** `VerificationRunner` (its PNG, PDF and SVG
|
|
39
|
+
images) and `Visualization::Graphviz#render` built a shell command from the report directory,
|
|
40
|
+
the workflow's name, the output path and the layout, so a path or name with shell syntax in it
|
|
41
|
+
would have run as a command. They now pass Graphviz its arguments one by one, and `render`
|
|
42
|
+
accepts only Graphviz's layout engines (`Graphviz::LAYOUTS`) and plain format names. The
|
|
43
|
+
values come from the developer, not from a request; a static analysis (Brakeman) flagged both.
|
|
44
|
+
|
|
45
|
+
### Fixed
|
|
46
|
+
- **PaperTrail is switched off per model, not per request.** Where Lyra takes over a write
|
|
47
|
+
(Hijack and the event-sourcing modes), it set `PaperTrail.request.enabled = false`. That turned
|
|
48
|
+
versioning off for every model for the rest of the request or job thread and never turned it
|
|
49
|
+
back on, so writes to versioned models Lyra does not monitor lost their versions, with no event
|
|
50
|
+
in their place. Lyra now calls `PaperTrail.request.disable_model` for the model it takes over.
|
|
51
|
+
- **The retention check reads the policies.** `GDPRCompliance#retention_compliance_check` ignored
|
|
52
|
+
the policies' `retention` blocks in favour of hard-coded periods for models named Payment,
|
|
53
|
+
Invoice, Student and Enrollment, and raised `TypeError` when `config.retention_policy` was a
|
|
54
|
+
single duration such as `7.years`. It now takes each monitored model's period from its policy,
|
|
55
|
+
over the configured default. `retention_compliance_with_policy` called a method that did not
|
|
56
|
+
exist.
|
|
57
|
+
- **Erasures and mode switches name who made them.** `rake lyra:erase` passes `OPERATOR=` as
|
|
58
|
+
`erased_by`, falling back to `LYRA_OPERATOR`, then the operating-system user and process; before,
|
|
59
|
+
it recorded no one. Every mode switch records the same in a new `actor` column of
|
|
60
|
+
`lyra_mode_transitions`, added on first use to an existing table, and logs it. A switch to
|
|
61
|
+
Disabled turns purpose checks and the access log off, so it is now attributable.
|
|
62
|
+
- **Erasure is limited to personal fields.** `Lyra::Erasure.erase!` (and `lyra:erase FIELDS=`)
|
|
63
|
+
erased any column it was named, rewriting history in place under an Art. 17 reason. It now
|
|
64
|
+
refuses a column that neither the record's policy declares personal nor the events' privacy
|
|
65
|
+
stamps list, unless called with `any_fields: true` (`ANY_FIELDS=1`).
|
|
66
|
+
- **Bulk writes lock the rows they write.** `update_all` and `delete_all` (and the writes routed
|
|
67
|
+
through them: `update_counters`, `touch_all`, `delete_by`, `dependent: :delete_all`) took their
|
|
68
|
+
snapshot with a plain read, so under READ COMMITTED a row another transaction changed or
|
|
69
|
+
inserted between the snapshot and the write could be written without an event. They now lock
|
|
70
|
+
the matched rows (`FOR UPDATE`, through a subquery of keys so DISTINCT and joins work) and write
|
|
71
|
+
exactly those, inside a transaction in every mode that keeps tables. A failed append rolls back
|
|
72
|
+
only its savepoint, so in Monitor the write still stands.
|
|
73
|
+
- **A destroyed record is not written again.** Two writers of one record, a destroy and an
|
|
74
|
+
update, could both pass: the second found no row to lock, fell back to the advisory lock and
|
|
75
|
+
appended after the Destroyed event, leaving a stream that brought the record back
|
|
76
|
+
(`Lyra.state_at`, DualView). An update or destroy of a record whose stream ends in a destroy now
|
|
77
|
+
raises `Lyra::RecordDestroyedError` (a `RecordNotFound`). To check it, ES-NoProj and ES-Lazy now
|
|
78
|
+
take the lock on the aggregate for updates and destroys, as the other event-sourcing modes did.
|
|
79
|
+
- **PAM DSL 0.9.3: the policy generator no longer chooses legal bases.** It wrote one for each
|
|
80
|
+
purpose it drafted: contract for billing, which tax law often makes a legal obligation;
|
|
81
|
+
legitimate interests for analytics, which a public authority cannot use for its tasks; consent
|
|
82
|
+
without a consent block. A drafted purpose now reads `basis :undetermined` with the usual
|
|
83
|
+
candidate and its caveats as a comment. Access under it is refused
|
|
84
|
+
(`PamDsl::UndeterminedBasisError`), and the Article 30 register lists it as a gap, until the
|
|
85
|
+
controller decides.
|
|
86
|
+
|
|
87
|
+
### Documented
|
|
88
|
+
- **The dashboard is a development tool.** Opening it in production with
|
|
89
|
+
`config.dashboard_authorization` is at the deployer's risk: whoever the rule admits reads every
|
|
90
|
+
data subject's records and history, with no declared purpose and no access-log entry, and the
|
|
91
|
+
rule covers the whole dashboard. What bringing it to production would take is listed in the
|
|
92
|
+
README (Dashboard) as a future feature.
|
|
93
|
+
|
|
94
|
+
### Dependencies
|
|
95
|
+
- **The development lockfile is updated against published advisories** (bundler-audit,
|
|
96
|
+
2026-11-04): `rack`, `nokogiri`, `loofah`, `rails-html-sanitizer`, `crass` and `sqlite3` (tests
|
|
97
|
+
only). The gems' own dependency declarations are unchanged: none pins a vulnerable version, and
|
|
98
|
+
an application resolves its own.
|
|
99
|
+
|
|
100
|
+
### Known limitations (unchanged)
|
|
101
|
+
- PAM DSL keeps consent records in memory, per process. Persistent consent storage is future
|
|
102
|
+
work; an application that relies on consent must store it. Documented in the PAM DSL README
|
|
103
|
+
("Runtime consent store"), on `ConsentStore`, and in `docs/PRIVACY_COMPLIANCE.md`.
|
|
104
|
+
|
|
105
|
+
## [0.9.2] - 2026-11-03
|
|
106
|
+
|
|
107
|
+
One fix in Lyra; PAM DSL and PetriFlow are unchanged (0.9.1).
|
|
108
|
+
|
|
109
|
+
### Released
|
|
110
|
+
- `orfeas_lyra` on RubyGems, built from tag `v0.9.2` and pushed on 2026-11-03
|
|
111
|
+
|
|
112
|
+
### Fixed
|
|
113
|
+
- **ES-Async read-your-writes no longer races a projection job.** With read-your-writes
|
|
114
|
+
(`each_write: true`), the writer projected its own event without the per-stream lock that
|
|
115
|
+
`AsyncProjectionJob` holds. A job that had read the record's stream before that event
|
|
116
|
+
committed could write the row afterwards, putting it back to the older value until the newer
|
|
117
|
+
event's job ran: a reader in between saw the record step back (an application's suite, run
|
|
118
|
+
with real background projection, read a refund as "approved" just after it was "processed").
|
|
119
|
+
`Lyra::Consistency::ReadYourWrites` now takes the job's lock before projecting, held to the
|
|
120
|
+
end of the writer's transaction. Final states converged before as well. Plain ES-Async (no
|
|
121
|
+
read-your-writes) was not affected. `test/concurrent_same_record_test.rb` reproduces the
|
|
122
|
+
interleaving.
|
|
123
|
+
|
|
10
124
|
## [0.9.1] - 2026-11-02
|
|
11
125
|
|
|
12
126
|
Lyra's changes since 0.9.0; PAM DSL and PetriFlow change only in their gem descriptions.
|
|
13
127
|
|
|
14
128
|
### Released
|
|
15
129
|
- The gems on RubyGems (`orfeas_lyra`, `orfeas_petri_flow`, `orfeas_pam_dsl`), built from tag
|
|
16
|
-
`v0.9.1`
|
|
130
|
+
`v0.9.1` and pushed on 2026-11-02
|
|
17
131
|
|
|
18
132
|
### Added
|
|
19
133
|
- **Writes are grouped into causal chains by default** (`config.correlate_writes`, on). Before,
|
data/README.md
CHANGED
|
@@ -52,7 +52,8 @@ seven configurations:
|
|
|
52
52
|
- PostgreSQL. Lyra depends on the `pg` gem; the advisory locks behind Genesis
|
|
53
53
|
and ES-Lazy, and id reservation in Hijack mode, need PostgreSQL.
|
|
54
54
|
|
|
55
|
-
The
|
|
55
|
+
The gems are on RubyGems: `orfeas_lyra` at 0.9.2 (3 November 2026), `orfeas_pam_dsl` and
|
|
56
|
+
`orfeas_petri_flow` at 0.9.1 (2 November 2026). Their names differ from the files to require:
|
|
56
57
|
|
|
57
58
|
```ruby
|
|
58
59
|
# Gemfile
|
|
@@ -88,7 +89,13 @@ Beyond the modes above (see the [API Reference](docs/API_REFERENCE.md) for each)
|
|
|
88
89
|
- **Dashboard**: `mount Lyra::Engine, at: "/lyra"`. Outside development and
|
|
89
90
|
test it answers 403 until `config.dashboard_authorization` (a proc run on the
|
|
90
91
|
controller, e.g. `->(controller) { controller.current_user&.admin? }`) says
|
|
91
|
-
who may use it.
|
|
92
|
+
who may use it. The dashboard is a development tool. In production, whoever
|
|
93
|
+
the rule admits reads every data subject's records and history, with no
|
|
94
|
+
declared purpose and no access-log entry, and the rule covers the whole
|
|
95
|
+
dashboard, not single pages. Bringing it to production is a future feature:
|
|
96
|
+
per-page and per-role authorization, a declared purpose and an access-log
|
|
97
|
+
entry for every subject shown or exported, masking by sensitivity, retention
|
|
98
|
+
for what it records, and a security review of its routes.
|
|
92
99
|
|
|
93
100
|
---
|
|
94
101
|
|
|
@@ -9,6 +9,10 @@ module Lyra
|
|
|
9
9
|
# Lyra::Configuration#dashboard_authorization.
|
|
10
10
|
before_action :authorize_lyra_dashboard!
|
|
11
11
|
|
|
12
|
+
# A model_class parameter that names no monitored model answers 404.
|
|
13
|
+
class UnknownModel < StandardError; end
|
|
14
|
+
rescue_from(UnknownModel) { head :not_found }
|
|
15
|
+
|
|
12
16
|
helper_method :pam_dsl_available?, :user_tracking_configured?, :petri_flow_available?
|
|
13
17
|
|
|
14
18
|
# Check if PAM DSL is available for privacy features
|
|
@@ -57,6 +61,21 @@ module Lyra
|
|
|
57
61
|
head :forbidden unless allowed
|
|
58
62
|
end
|
|
59
63
|
|
|
64
|
+
# The monitored model a request names, or 404. The routes that take a model
|
|
65
|
+
# read its records and return their attributes, so only models Lyra monitors
|
|
66
|
+
# may be named: resolving any constant (as 0.9.2 and earlier did) let a
|
|
67
|
+
# dashboard user read any table, unmonitored ones included.
|
|
68
|
+
def monitored_model!(name)
|
|
69
|
+
Lyra.config.monitored_model_named(name) || raise(UnknownModel, name.to_s)
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
# An integer parameter, clamped to 1..max; the default when absent. Bounds
|
|
73
|
+
# the work a single dashboard request can ask for.
|
|
74
|
+
def bounded_param(key, default:, max:)
|
|
75
|
+
value = params[key].presence&.to_i || default
|
|
76
|
+
value.clamp(1, max)
|
|
77
|
+
end
|
|
78
|
+
|
|
60
79
|
def lyra_dashboard_open_by_default?
|
|
61
80
|
return true if Rails.env.development? || Rails.env.test?
|
|
62
81
|
|
|
@@ -11,14 +11,14 @@ module Lyra
|
|
|
11
11
|
|
|
12
12
|
# GET /lyra/dashboard/model/:model_class
|
|
13
13
|
def model_overview
|
|
14
|
-
@model_class = params[:model_class]
|
|
14
|
+
@model_class = monitored_model!(params[:model_class])
|
|
15
15
|
@records_count = @model_class.count
|
|
16
16
|
@events_count = count_events_for_model(@model_class)
|
|
17
17
|
end
|
|
18
18
|
|
|
19
19
|
# GET /lyra/dashboard/compare/:model_class/:id
|
|
20
20
|
def compare
|
|
21
|
-
model_class = params[:model_class]
|
|
21
|
+
model_class = monitored_model!(params[:model_class])
|
|
22
22
|
model_id = params[:id]
|
|
23
23
|
|
|
24
24
|
@comparison = DualView.new(model_class, model_id).compare
|
|
@@ -34,7 +34,7 @@ module Lyra
|
|
|
34
34
|
|
|
35
35
|
# GET /lyra/dashboard/discrepancies/:model_class
|
|
36
36
|
def discrepancies
|
|
37
|
-
@model_class = params[:model_class]
|
|
37
|
+
@model_class = monitored_model!(params[:model_class])
|
|
38
38
|
@discrepancies = DualView.find_discrepancies(@model_class)
|
|
39
39
|
# find_discrepancies returns only the records that differ
|
|
40
40
|
@records_checked = @model_class.count
|
|
@@ -120,7 +120,7 @@ module Lyra
|
|
|
120
120
|
|
|
121
121
|
# If a model is selected, load available records for the picker
|
|
122
122
|
if @selected_model.present?
|
|
123
|
-
@model_class = @selected_model
|
|
123
|
+
@model_class = monitored_model!(@selected_model)
|
|
124
124
|
@available_records = fetch_available_records(@model_class, @search_query)
|
|
125
125
|
end
|
|
126
126
|
|
|
@@ -132,7 +132,7 @@ module Lyra
|
|
|
132
132
|
|
|
133
133
|
# GET /lyra/dashboard/audit_trail/:model_class/:id
|
|
134
134
|
def audit_trail_for_record
|
|
135
|
-
@model_class = params[:model_class]
|
|
135
|
+
@model_class = monitored_model!(params[:model_class])
|
|
136
136
|
@model_id = params[:id]
|
|
137
137
|
@record = @model_class.find_by(id: @model_id)
|
|
138
138
|
@audit_entries = AuditProjection.audit_trail(@model_class, @model_id)
|
|
@@ -146,7 +146,7 @@ module Lyra
|
|
|
146
146
|
|
|
147
147
|
# GET /lyra/dashboard/visualizations/timeline
|
|
148
148
|
def timeline
|
|
149
|
-
events = fetch_recent_events(limit:
|
|
149
|
+
events = fetch_recent_events(limit: bounded_param(:limit, default: 100, max: 1000))
|
|
150
150
|
timeline = Visualization::Timeline.new(events)
|
|
151
151
|
|
|
152
152
|
render json: timeline.to_data
|
|
@@ -158,7 +158,7 @@ module Lyra
|
|
|
158
158
|
events = fetch_filtered_events(
|
|
159
159
|
model_class: params[:model_class],
|
|
160
160
|
operation: params[:operation],
|
|
161
|
-
limit:
|
|
161
|
+
limit: bounded_param(:limit, default: 50, max: 500)
|
|
162
162
|
)
|
|
163
163
|
graph = Visualization::EventGraph.new(events)
|
|
164
164
|
|
|
@@ -170,7 +170,7 @@ module Lyra
|
|
|
170
170
|
def entity_graph
|
|
171
171
|
model_class = params[:model_class]
|
|
172
172
|
model_id = params[:id]
|
|
173
|
-
depth =
|
|
173
|
+
depth = bounded_param(:depth, default: 1, max: 3) # How many related entities to include
|
|
174
174
|
|
|
175
175
|
events = fetch_entity_events(model_class, model_id, depth: depth)
|
|
176
176
|
graph = Visualization::EventGraph.new(events)
|
|
@@ -184,10 +184,10 @@ module Lyra
|
|
|
184
184
|
# Returns list of entities with event counts for the picker UI
|
|
185
185
|
def event_list
|
|
186
186
|
model_filter = params[:model_class]
|
|
187
|
-
limit =
|
|
187
|
+
limit = bounded_param(:limit, default: 20, max: 100)
|
|
188
188
|
|
|
189
189
|
entities = []
|
|
190
|
-
models = model_filter.present? ? [model_filter
|
|
190
|
+
models = model_filter.present? ? [monitored_model!(model_filter)] : Lyra.config.monitored_models
|
|
191
191
|
|
|
192
192
|
models.each do |model_class|
|
|
193
193
|
model_class.order(updated_at: :desc).limit(limit).each do |record|
|
|
@@ -224,7 +224,7 @@ module Lyra
|
|
|
224
224
|
|
|
225
225
|
# GET /lyra/dashboard/visualizations/heatmap
|
|
226
226
|
def heatmap
|
|
227
|
-
events = fetch_events_for_period(days:
|
|
227
|
+
events = fetch_events_for_period(days: bounded_param(:days, default: 7, max: 90))
|
|
228
228
|
heatmap = Visualization::ActivityHeatmap.new(events)
|
|
229
229
|
|
|
230
230
|
render json: heatmap.to_data
|
|
@@ -232,7 +232,7 @@ module Lyra
|
|
|
232
232
|
|
|
233
233
|
# GET /lyra/visualizations/event_graph (HTML view)
|
|
234
234
|
def event_graph_view
|
|
235
|
-
@limit =
|
|
235
|
+
@limit = bounded_param(:limit, default: 50, max: 500)
|
|
236
236
|
events = fetch_recent_events(limit: @limit)
|
|
237
237
|
graph = Visualization::EventGraph.new(events)
|
|
238
238
|
@graph_data = graph.to_data
|
|
@@ -241,7 +241,7 @@ module Lyra
|
|
|
241
241
|
|
|
242
242
|
# GET /lyra/visualizations/heatmap (HTML view)
|
|
243
243
|
def heatmap_view
|
|
244
|
-
@days =
|
|
244
|
+
@days = bounded_param(:days, default: 7, max: 90)
|
|
245
245
|
events = fetch_events_for_period(days: @days)
|
|
246
246
|
heatmap = Visualization::ActivityHeatmap.new(events)
|
|
247
247
|
@heatmap_data = heatmap.to_data
|
|
@@ -362,8 +362,8 @@ module Lyra
|
|
|
362
362
|
|
|
363
363
|
# GET /lyra/dashboard/visualizations/model_heatmap/:model_class
|
|
364
364
|
def model_heatmap
|
|
365
|
-
model_class = params[:model_class]
|
|
366
|
-
events = fetch_events_for_model(model_class, days:
|
|
365
|
+
model_class = monitored_model!(params[:model_class])
|
|
366
|
+
events = fetch_events_for_model(model_class, days: bounded_param(:days, default: 7, max: 90))
|
|
367
367
|
heatmap = Visualization::ActivityHeatmap.new(events)
|
|
368
368
|
|
|
369
369
|
render json: {
|
|
@@ -475,7 +475,7 @@ module Lyra
|
|
|
475
475
|
|
|
476
476
|
def fetch_filtered_events(model_class: nil, operation: nil, limit: 50)
|
|
477
477
|
events = []
|
|
478
|
-
models = model_class.present? ? [model_class
|
|
478
|
+
models = model_class.present? ? [monitored_model!(model_class)] : Lyra.config.monitored_models
|
|
479
479
|
|
|
480
480
|
models.each do |klass|
|
|
481
481
|
klass.order(updated_at: :desc).limit(limit).each do |record|
|
|
@@ -39,7 +39,7 @@ module Lyra
|
|
|
39
39
|
|
|
40
40
|
# GET /lyra/flow/event_chain/:model_class/:model_id
|
|
41
41
|
def event_chain
|
|
42
|
-
model_class = params[:model_class]
|
|
42
|
+
model_class = monitored_model!(params[:model_class])
|
|
43
43
|
model_id = params[:model_id]
|
|
44
44
|
|
|
45
45
|
flow = EventFlow.new
|
|
@@ -89,7 +89,7 @@ module Lyra
|
|
|
89
89
|
|
|
90
90
|
# GET /lyra/flow/visualization/:model_class/:model_id
|
|
91
91
|
def visualization
|
|
92
|
-
model_class = params[:model_class]
|
|
92
|
+
model_class = monitored_model!(params[:model_class])
|
|
93
93
|
model_id = params[:model_id]
|
|
94
94
|
format = params[:format] || 'json'
|
|
95
95
|
|
|
@@ -212,9 +212,10 @@ module Lyra
|
|
|
212
212
|
events = Lyra.config.event_store.read.to_a
|
|
213
213
|
|
|
214
214
|
if subject_id && subject_type
|
|
215
|
+
# The subject's own record. What a user did is user_actions; their
|
|
216
|
+
# edits of other records are not part of their timeline.
|
|
215
217
|
events = events.select do |event|
|
|
216
|
-
event.
|
|
217
|
-
(event_model_class(event) == subject_type && event_model_id(event) == subject_id.to_i)
|
|
218
|
+
event_model_class(event).to_s == subject_type.to_s && event_model_id(event).to_s == subject_id.to_s
|
|
218
219
|
end
|
|
219
220
|
end
|
|
220
221
|
|
data/lib/lyra/bypass_events.rb
CHANGED
|
@@ -35,7 +35,19 @@ module Lyra
|
|
|
35
35
|
# matched nothing, recorded no event, and the change was lost.
|
|
36
36
|
def atomically(model_class, &block)
|
|
37
37
|
Lyra::Projections::LazyProjection.before_read(model_class)
|
|
38
|
-
required? ? model_class.transaction(&block) : yield
|
|
38
|
+
required? || locks_rows? ? model_class.transaction(&block) : yield
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
# Whether a bulk write locks the rows it matches and writes exactly
|
|
42
|
+
# those (snapshot(lock: true), then write_snapshotted): wherever the
|
|
43
|
+
# rows live in a table, so in every mode but the events-only store.
|
|
44
|
+
# From 0.9.3: before, the snapshot was a plain read, and under READ
|
|
45
|
+
# COMMITTED a row another transaction changed or inserted between the
|
|
46
|
+
# snapshot and the write could be written without an event. A failed
|
|
47
|
+
# append inside the transaction rolls back only its own savepoint, so in
|
|
48
|
+
# Monitor the write still stands when its event fails.
|
|
49
|
+
def locks_rows?
|
|
50
|
+
!(Lyra.config.event_sourcing_mode? && Lyra.config.projection_mode == :disabled)
|
|
39
51
|
end
|
|
40
52
|
|
|
41
53
|
# Report a failed publish: raise when the events are required (see
|
|
@@ -90,9 +102,16 @@ module Lyra
|
|
|
90
102
|
|
|
91
103
|
# Attributes of every row the relation matches, keyed by primary key.
|
|
92
104
|
# Taken before a bulk write so the events can carry old values.
|
|
93
|
-
def snapshot(model_class, relation)
|
|
105
|
+
def snapshot(model_class, relation, lock: false)
|
|
94
106
|
pk = model_class.primary_key
|
|
95
|
-
|
|
107
|
+
scope = relation
|
|
108
|
+
if lock
|
|
109
|
+
# Lock through a subquery of keys, so a relation with DISTINCT,
|
|
110
|
+
# GROUP BY or joins, which FOR UPDATE refuses, can still be locked.
|
|
111
|
+
keys = relation.reselect(model_class.arel_table[pk])
|
|
112
|
+
scope = model_class.unscoped.where(pk => keys).lock
|
|
113
|
+
end
|
|
114
|
+
rows = Lyra::PurposeBoundReads.internal { scope.to_a }
|
|
96
115
|
rows.to_h { |record| [record.public_send(pk), record.attributes] }
|
|
97
116
|
end
|
|
98
117
|
|
|
@@ -89,12 +89,34 @@ module Lyra
|
|
|
89
89
|
(Thread.current[:lyra_pending_writes] || []).each { |write| project(write) }
|
|
90
90
|
end
|
|
91
91
|
|
|
92
|
+
# Projects under the lock AsyncProjectionJob holds per stream, taken
|
|
93
|
+
# in the writer's transaction (or one of its own) and held to its end.
|
|
94
|
+
# A job that had read the stream before this write committed would
|
|
95
|
+
# otherwise write the row after this projection, putting it back to
|
|
96
|
+
# the older value until the newer event's job ran. With the lock, this
|
|
97
|
+
# projection waits for a job in flight; a job that starts after it
|
|
98
|
+
# reads a stream that already holds this write's event.
|
|
92
99
|
def project(write)
|
|
93
|
-
|
|
100
|
+
model_class = write[:model_class]
|
|
101
|
+
model_class.transaction do
|
|
102
|
+
lock_stream(model_class, write[:result])
|
|
103
|
+
Lyra::Projections::ModelProjection.project(model_class, write[:operation], write[:result])
|
|
104
|
+
end
|
|
94
105
|
rescue => e
|
|
95
106
|
Rails.logger.error("Lyra: Failed to ensure projection - #{e.message}")
|
|
96
107
|
raise if Lyra.config.strict_projections
|
|
97
108
|
end
|
|
109
|
+
|
|
110
|
+
# The same key AsyncProjectionJob#lock_stream! locks: the record's
|
|
111
|
+
# stream name, from the model id its event carries.
|
|
112
|
+
def lock_stream(model_class, result)
|
|
113
|
+
data = result.events&.first&.data
|
|
114
|
+
model_id = data && (data[:model_id] || data["model_id"])
|
|
115
|
+
return unless model_id
|
|
116
|
+
|
|
117
|
+
stream = Lyra.stream_name(model_class, model_id)
|
|
118
|
+
model_class.with_connection { |c| Lyra::AdvisoryLock.xact_lock(c, stream, purpose: "read-your-writes projection") }
|
|
119
|
+
end
|
|
98
120
|
end
|
|
99
121
|
end
|
|
100
122
|
|
data/lib/lyra/erasure.rb
CHANGED
|
@@ -14,7 +14,10 @@ module Lyra
|
|
|
14
14
|
#
|
|
15
15
|
# Personal fields are those the record's policy declares plus those the
|
|
16
16
|
# events' privacy stamps (config.annotate_privacy) list, or exactly the
|
|
17
|
-
# +fields:+ given
|
|
17
|
+
# +fields:+ given, which must be among those unless +any_fields: true+
|
|
18
|
+
# (from 0.9.3: an erasure rewrites history in place, so naming any column,
|
|
19
|
+
# a payment's status, say, would rewrite non-personal history under an
|
|
20
|
+
# Art. 17 reason). In the record's events a value is erased wherever it
|
|
18
21
|
# appears: under the attribute's own name (attributes, both sides of each
|
|
19
22
|
# change, payload keys), and under any other name, matched by value (a
|
|
20
23
|
# domain-event payload that copies the email as payer_email).
|
|
@@ -53,10 +56,17 @@ module Lyra
|
|
|
53
56
|
keyword_init: true)
|
|
54
57
|
|
|
55
58
|
class << self
|
|
56
|
-
def erase!(model, id, reason:, fields: nil, erased_by: nil, everywhere: false, max_copies: MAX_COPIES
|
|
59
|
+
def erase!(model, id, reason:, fields: nil, erased_by: nil, everywhere: false, max_copies: MAX_COPIES,
|
|
60
|
+
any_fields: false)
|
|
57
61
|
stream = Lyra.stream_name(model, id)
|
|
58
62
|
events = Lyra.config.event_store.read.stream(stream).to_a
|
|
59
|
-
|
|
63
|
+
declared = personal_fields(model, events)
|
|
64
|
+
personal = (fields || declared).map(&:to_s).uniq
|
|
65
|
+
undeclared = any_fields || fields.nil? ? [] : personal - declared
|
|
66
|
+
if undeclared.any?
|
|
67
|
+
raise Unsupported, "#{model.name} #{id}: #{undeclared.join(', ')} not declared personal by its policy " \
|
|
68
|
+
"or the events' privacy stamps; pass any_fields: true to erase them anyway"
|
|
69
|
+
end
|
|
60
70
|
raise Unsupported, "#{model.name} #{id}: no personal fields to erase" if personal.empty?
|
|
61
71
|
|
|
62
72
|
replacements = personal.to_h { |field| [field, replacement(model, id, field)] }
|
|
@@ -71,8 +81,8 @@ module Lyra
|
|
|
71
81
|
resolve = ->(key, value) { resolve_in(model, id, replacements, originals, key, value) }
|
|
72
82
|
rewritten = events.filter_map { |event| rewrite(event, resolve) }
|
|
73
83
|
Lyra.config.event_store.overwrite(rewritten) if rewritten.any?
|
|
74
|
-
row_erased = erase_row(model, id, replacements)
|
|
75
|
-
record_erasure(model, id, stream, personal, reason, erased_by)
|
|
84
|
+
row_erased, row_copies = erase_row(model, id, replacements, originals)
|
|
85
|
+
record_erasure(model, id, stream, personal | row_copies, reason, erased_by)
|
|
76
86
|
if everywhere
|
|
77
87
|
copies, shared = erase_copies(model, id, stream, originals, replacements, reason, erased_by, max_copies)
|
|
78
88
|
end
|
|
@@ -175,15 +185,24 @@ module Lyra
|
|
|
175
185
|
end
|
|
176
186
|
end
|
|
177
187
|
|
|
178
|
-
|
|
179
|
-
|
|
188
|
+
# The erased fields, and any other column of the row that still holds
|
|
189
|
+
# one of their values (a username that is the email): its events were
|
|
190
|
+
# rewritten with that column's replacement, so the row gets it too and
|
|
191
|
+
# still equals its replay. Returns [erased?, the copied columns].
|
|
192
|
+
def erase_row(model, id, replacements, originals)
|
|
193
|
+
return [false, []] if events_only?
|
|
180
194
|
|
|
181
195
|
row = model.unscoped.lock.find_by(model.primary_key => id)
|
|
182
|
-
return false unless row
|
|
196
|
+
return [false, []] unless row
|
|
183
197
|
|
|
184
198
|
columns = replacements.slice(*row.attribute_names)
|
|
185
|
-
|
|
186
|
-
|
|
199
|
+
copies = row.attributes.each_with_object({}) do |(column, value), acc|
|
|
200
|
+
next if columns.key?(column) || !value.is_a?(String) || !originals.key?(value)
|
|
201
|
+
|
|
202
|
+
acc[column] = replacement_for(model, id, column, replacements[originals[value]])
|
|
203
|
+
end
|
|
204
|
+
write_columns(model, id, columns.merge(copies))
|
|
205
|
+
[true, copies.keys]
|
|
187
206
|
end
|
|
188
207
|
|
|
189
208
|
# everywhere: events of other streams holding the erased values, and
|
|
@@ -1,4 +1,9 @@
|
|
|
1
1
|
module Lyra
|
|
2
|
+
# An update or destroy of a record whose event stream says it was destroyed
|
|
3
|
+
# (see CrudInterceptor#lyra_lock_aggregate_unless_destroyed!). A
|
|
4
|
+
# RecordNotFound, as for any write to a record that is gone.
|
|
5
|
+
class RecordDestroyedError < ActiveRecord::RecordNotFound; end
|
|
6
|
+
|
|
2
7
|
module Interceptors
|
|
3
8
|
# Where hijack and event-sourcing modes take over a write.
|
|
4
9
|
#
|
|
@@ -481,11 +486,37 @@ module Lyra
|
|
|
481
486
|
es_async = lyra_event_sourcing_mode? && Lyra.config.projection_mode == :async
|
|
482
487
|
es_sync = lyra_event_sourcing_mode? && Lyra.config.projection_mode == :sync
|
|
483
488
|
return unless lyra_hijack_mode? || es_sync || es_async || checks_version
|
|
484
|
-
return if (lyra_hijack_mode? || es_sync) && lyra_lock_own_row(operation)
|
|
489
|
+
return :row if (lyra_hijack_mode? || es_sync) && lyra_lock_own_row(operation)
|
|
485
490
|
|
|
486
491
|
self.class.with_connection do |connection|
|
|
487
492
|
Lyra::AdvisoryLock.xact_lock(connection, "lyra_aggregate/#{lyra_stream_name}", purpose: "aggregate write")
|
|
488
493
|
end
|
|
494
|
+
:advisory
|
|
495
|
+
end
|
|
496
|
+
|
|
497
|
+
# The lock on the aggregate, then a refusal if the record's stream says
|
|
498
|
+
# it was destroyed (from 0.9.3). A destroy and an update of one record
|
|
499
|
+
# could both pass before: the second writer found no row to lock, fell
|
|
500
|
+
# back to the advisory lock and appended after the Destroyed event,
|
|
501
|
+
# leaving a stream that brings the record back (state_at, DualView).
|
|
502
|
+
# Where the record's row was found and locked it exists, so there is
|
|
503
|
+
# nothing to check. Elsewhere the stream's last event is read under the
|
|
504
|
+
# advisory lock, which this takes in ES-NoProj and ES-Lazy too, for
|
|
505
|
+
# updates and destroys only. With lock_aggregate_before_append off, as
|
|
506
|
+
# in the benchmarks' no-lock controls, nothing is checked.
|
|
507
|
+
def lyra_lock_aggregate_unless_destroyed!(operation = :update)
|
|
508
|
+
locked = lyra_lock_aggregate!(operation)
|
|
509
|
+
return if locked == :row || id.nil? || !Lyra.config.lock_aggregate_before_append
|
|
510
|
+
|
|
511
|
+
unless locked == :advisory
|
|
512
|
+
self.class.with_connection do |connection|
|
|
513
|
+
Lyra::AdvisoryLock.xact_lock(connection, "lyra_aggregate/#{lyra_stream_name}", purpose: "aggregate write")
|
|
514
|
+
end
|
|
515
|
+
end
|
|
516
|
+
last = Lyra.config.event_store.read.stream(lyra_stream_name).backward.limit(1).to_a.first
|
|
517
|
+
return unless last && Lyra::Event.operation_of(last) == :destroyed
|
|
518
|
+
|
|
519
|
+
raise Lyra::RecordDestroyedError, "#{self.class.name} #{id} was destroyed; its #{operation} is refused"
|
|
489
520
|
end
|
|
490
521
|
|
|
491
522
|
# True when the record's row was found and locked.
|
|
@@ -510,7 +541,7 @@ module Lyra
|
|
|
510
541
|
def lyra_hijack_update
|
|
511
542
|
@lyra_hijacked = true
|
|
512
543
|
lyra_disable_paper_trail!
|
|
513
|
-
|
|
544
|
+
lyra_lock_aggregate_unless_destroyed!
|
|
514
545
|
|
|
515
546
|
# Active Record checks and increments lock_version in the row write
|
|
516
547
|
# that follows; the event records the increment it makes.
|
|
@@ -529,7 +560,7 @@ module Lyra
|
|
|
529
560
|
def lyra_hijack_destroy
|
|
530
561
|
@lyra_hijacked = true
|
|
531
562
|
lyra_disable_paper_trail!
|
|
532
|
-
|
|
563
|
+
lyra_lock_aggregate_unless_destroyed!(:destroy)
|
|
533
564
|
|
|
534
565
|
command = Lyra::Commands::DestroyCommand.new(self.class, id)
|
|
535
566
|
command.record = self
|
|
@@ -575,7 +606,7 @@ module Lyra
|
|
|
575
606
|
def lyra_prepare_event_source_update
|
|
576
607
|
@lyra_hijacked = true
|
|
577
608
|
lyra_disable_paper_trail!
|
|
578
|
-
|
|
609
|
+
lyra_lock_aggregate_unless_destroyed!
|
|
579
610
|
# Active Record checks and increments lock_version only when it writes
|
|
580
611
|
# the row, that is when something changed.
|
|
581
612
|
lock_was = lyra_take_lock_version!("update") if has_changes_to_save?
|
|
@@ -643,7 +674,7 @@ module Lyra
|
|
|
643
674
|
def lyra_prepare_event_source_destroy
|
|
644
675
|
@lyra_hijacked = true
|
|
645
676
|
lyra_disable_paper_trail!
|
|
646
|
-
|
|
677
|
+
lyra_lock_aggregate_unless_destroyed!(:destroy)
|
|
647
678
|
Lyra::OptimisticLocking.check!(self, "destroy")
|
|
648
679
|
|
|
649
680
|
command = Lyra::Commands::DestroyCommand.new(self.class, id)
|
|
@@ -800,10 +831,14 @@ module Lyra
|
|
|
800
831
|
def lyra_disable_paper_trail!
|
|
801
832
|
return unless defined?(PaperTrail)
|
|
802
833
|
|
|
803
|
-
# PaperTrail
|
|
804
|
-
#
|
|
805
|
-
|
|
806
|
-
|
|
834
|
+
# PaperTrail's request store: switch versioning off for this model
|
|
835
|
+
# only. Before 0.9.3 this set PaperTrail.request.enabled = false,
|
|
836
|
+
# which switched it off for every model for the rest of the request
|
|
837
|
+
# or job thread and was never restored, so writes to versioned models
|
|
838
|
+
# Lyra does not monitor lost their versions with no event in their
|
|
839
|
+
# place.
|
|
840
|
+
if PaperTrail.respond_to?(:request) && PaperTrail.request.respond_to?(:disable_model)
|
|
841
|
+
PaperTrail.request.disable_model(self.class)
|
|
807
842
|
elsif respond_to?(:paper_trail) && paper_trail.respond_to?(:enabled=)
|
|
808
843
|
# Older PaperTrail versions use instance-level enabled
|
|
809
844
|
paper_trail.enabled = false
|
data/lib/lyra/mode_transition.rb
CHANGED
|
@@ -321,9 +321,22 @@ module Lyra
|
|
|
321
321
|
return nil unless table_ready?
|
|
322
322
|
|
|
323
323
|
sql(:select_value,
|
|
324
|
-
"INSERT INTO #{TABLE} (kind, from_config, to_config, position, checked, created_at) VALUES " \
|
|
325
|
-
"('applied', NULL, #{q(to)}, #{Integer(event_position)}, 0, #{q(Time.current.utc)}) RETURNING id"
|
|
326
|
-
).to_i
|
|
324
|
+
"INSERT INTO #{TABLE} (kind, from_config, to_config, position, checked, created_at, actor) VALUES " \
|
|
325
|
+
"('applied', NULL, #{q(to)}, #{Integer(event_position)}, 0, #{q(Time.current.utc)}, #{q(actor)}) RETURNING id"
|
|
326
|
+
).to_i.tap { |id| Rails.logger&.warn("[Lyra] Mode switched to #{to} by #{actor} (lyra_mode_transitions #{id})") }
|
|
327
|
+
end
|
|
328
|
+
|
|
329
|
+
# Who is making a change: the given operator, else the signed-in user
|
|
330
|
+
# (Current.user), else LYRA_OPERATOR, else the operating-system user and
|
|
331
|
+
# process of a task or console. Recorded with every mode switch and, from
|
|
332
|
+
# lyra:erase, every erasure (0.9.3; before, neither named anyone).
|
|
333
|
+
public def actor(operator = nil)
|
|
334
|
+
return operator.to_s if operator.present?
|
|
335
|
+
|
|
336
|
+
user = defined?(::Current) && ::Current.respond_to?(:user) ? ::Current.user : nil
|
|
337
|
+
return "user:#{user.respond_to?(:id) ? user.id : user}" if user
|
|
338
|
+
|
|
339
|
+
ENV["LYRA_OPERATOR"].presence || ["process", ENV["USER"], Process.pid].compact.join(":")
|
|
327
340
|
end
|
|
328
341
|
|
|
329
342
|
# Say what a switch reaches: with ModeSync, every process within its
|
|
@@ -358,8 +371,11 @@ module Lyra
|
|
|
358
371
|
t.bigint :position, null: false, default: 0
|
|
359
372
|
t.integer :checked, null: false, default: 0
|
|
360
373
|
t.datetime :created_at, null: false
|
|
374
|
+
t.string :actor
|
|
361
375
|
end
|
|
362
376
|
end
|
|
377
|
+
# Tables created before 0.9.3 have no actor column.
|
|
378
|
+
connection.add_column(TABLE, :actor, :string) unless connection.column_exists?(TABLE, :actor)
|
|
363
379
|
end
|
|
364
380
|
@table_ready = true
|
|
365
381
|
rescue ActiveRecord::NoDatabaseError, ActiveRecord::ConnectionNotEstablished
|
|
@@ -37,7 +37,7 @@ module Lyra
|
|
|
37
37
|
model_class_extractor: method(:extract_model_class),
|
|
38
38
|
model_id_extractor: method(:extract_model_id),
|
|
39
39
|
changes_extractor: method(:extract_changes),
|
|
40
|
-
retention_policy:
|
|
40
|
+
retention_policy: retention_policy
|
|
41
41
|
)
|
|
42
42
|
end
|
|
43
43
|
|
|
@@ -92,15 +92,34 @@ module Lyra
|
|
|
92
92
|
end
|
|
93
93
|
end
|
|
94
94
|
|
|
95
|
-
#
|
|
95
|
+
# Whether an event holds the subject's data: an event of the subject's
|
|
96
|
+
# own record, or of a record that refers to the subject through its
|
|
97
|
+
# foreign key (a Registration's user_id, for subject type User).
|
|
98
|
+
#
|
|
99
|
+
# Who *made* a change is not whose data it is: before 0.9.3 an event
|
|
100
|
+
# whose metadata named the subject as its actor (metadata[:user_id])
|
|
101
|
+
# was exported too, so a staff member's export held every record they
|
|
102
|
+
# had edited, other people's data, while a subject's own records written
|
|
103
|
+
# by staff could be missed.
|
|
96
104
|
def event_relates_to_subject?(event, subject_id, subject_type)
|
|
97
|
-
metadata = event.respond_to?(:metadata) ? event.metadata : {}
|
|
98
105
|
data = event.respond_to?(:data) ? event.data : {}
|
|
106
|
+
id = subject_id.to_s
|
|
107
|
+
return true if extract_model_class(event).to_s == subject_type.to_s && extract_model_id(event).to_s == id
|
|
108
|
+
|
|
109
|
+
explicit = data[:subject_id] || data["subject_id"]
|
|
110
|
+
if explicit
|
|
111
|
+
type = data[:subject_type] || data["subject_type"]
|
|
112
|
+
return explicit.to_s == id && (type.nil? || type.to_s == subject_type.to_s)
|
|
113
|
+
end
|
|
114
|
+
|
|
115
|
+
attributes = extract_attributes(event)
|
|
116
|
+
value = attributes[subject_foreign_key(subject_type)] || attributes[subject_foreign_key(subject_type).to_sym]
|
|
117
|
+
!value.nil? && value.to_s == id
|
|
118
|
+
end
|
|
99
119
|
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
(extract_model_class(event) == subject_type && extract_model_id(event) == subject_id)
|
|
120
|
+
# The foreign key a record uses to refer to a subject of +subject_type+.
|
|
121
|
+
def subject_foreign_key(subject_type)
|
|
122
|
+
"#{subject_type.to_s.demodulize.underscore}_id"
|
|
104
123
|
end
|
|
105
124
|
|
|
106
125
|
# Lyra-specific extractors
|
|
@@ -147,14 +166,32 @@ module Lyra
|
|
|
147
166
|
{}
|
|
148
167
|
end
|
|
149
168
|
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
169
|
+
# The retention periods the compliance check applies, as PAM's check
|
|
170
|
+
# expects them ({ "Model" => { duration: d }, default: { duration: d } }):
|
|
171
|
+
# each monitored model's period from its privacy policy's retention block,
|
|
172
|
+
# over config.retention_policy, which may be a single duration (7.years) or
|
|
173
|
+
# such a hash.
|
|
174
|
+
#
|
|
175
|
+
# Before 0.9.3 the policies' retention blocks were ignored in favour of
|
|
176
|
+
# hard-coded periods for models named Payment, Invoice, Student and
|
|
177
|
+
# Enrollment, and a single duration in config.retention_policy raised
|
|
178
|
+
# TypeError.
|
|
179
|
+
def retention_policy
|
|
180
|
+
configured = Lyra.config.retention_policy
|
|
181
|
+
policy =
|
|
182
|
+
case configured
|
|
183
|
+
when nil then { default: { duration: 7.years } }
|
|
184
|
+
when Hash then configured.transform_values { |v| v.is_a?(Hash) ? v : { duration: v } }
|
|
185
|
+
else { default: { duration: configured } }
|
|
186
|
+
end
|
|
187
|
+
|
|
188
|
+
Lyra.config.monitored_models.each do |model|
|
|
189
|
+
duration = Lyra::Privacy.policy_for(model).retention_for(model)
|
|
190
|
+
policy[model.name] = { duration: duration } if duration.is_a?(ActiveSupport::Duration)
|
|
191
|
+
rescue StandardError
|
|
192
|
+
next
|
|
193
|
+
end
|
|
194
|
+
policy
|
|
158
195
|
end
|
|
159
196
|
end
|
|
160
197
|
|
|
@@ -175,9 +212,9 @@ module Lyra
|
|
|
175
212
|
# Returns nil for models without defined retention (infinite/manual)
|
|
176
213
|
def retention_compliance_with_policy(policy_name)
|
|
177
214
|
integration = PolicyIntegration.new(policy_name)
|
|
178
|
-
events =
|
|
215
|
+
events = Lyra.config.event_store.read.to_a
|
|
179
216
|
|
|
180
|
-
events.group_by { |e| e
|
|
217
|
+
events.group_by { |e| extract_model_class(e) }.map do |model_class, model_events|
|
|
181
218
|
retention_duration = integration.retention_duration(model_class)
|
|
182
219
|
|
|
183
220
|
# nil means infinite/manual retention - always compliant
|
|
@@ -192,7 +229,8 @@ module Lyra
|
|
|
192
229
|
}
|
|
193
230
|
else
|
|
194
231
|
expired = model_events.select do |event|
|
|
195
|
-
|
|
232
|
+
timestamp = extract_timestamp(event)
|
|
233
|
+
timestamp && timestamp < (Time.current - retention_duration)
|
|
196
234
|
end
|
|
197
235
|
|
|
198
236
|
{
|
|
@@ -201,7 +239,7 @@ module Lyra
|
|
|
201
239
|
expired_events: expired.count,
|
|
202
240
|
retention_period: retention_duration,
|
|
203
241
|
compliance_status: expired.empty? ? :compliant : :requires_action,
|
|
204
|
-
expired_event_ids: expired.map(
|
|
242
|
+
expired_event_ids: expired.map { |e| e.respond_to?(:event_id) ? e.event_id : e.object_id }
|
|
205
243
|
}
|
|
206
244
|
end
|
|
207
245
|
end
|
|
@@ -170,8 +170,9 @@ module Lyra
|
|
|
170
170
|
return super unless Lyra::BypassEvents.enabled_for?(klass)
|
|
171
171
|
|
|
172
172
|
Lyra::BypassEvents.atomically(klass) do
|
|
173
|
-
|
|
174
|
-
|
|
173
|
+
lock = Lyra::BypassEvents.locks_rows?
|
|
174
|
+
before = Lyra::BypassEvents.snapshot(klass, self, lock: lock)
|
|
175
|
+
result = lock ? write_snapshotted(before) { |rows| rows.update_all(updates) } : super
|
|
175
176
|
safely_publish_bulk("update_all") { Lyra::BypassEvents.publish_updates(klass, before, source: "update_all") }
|
|
176
177
|
result
|
|
177
178
|
end
|
|
@@ -186,8 +187,9 @@ module Lyra
|
|
|
186
187
|
return super unless Lyra::BypassEvents.enabled_for?(klass)
|
|
187
188
|
|
|
188
189
|
Lyra::BypassEvents.atomically(klass) do
|
|
189
|
-
|
|
190
|
-
|
|
190
|
+
lock = Lyra::BypassEvents.locks_rows?
|
|
191
|
+
before = Lyra::BypassEvents.snapshot(klass, self, lock: lock)
|
|
192
|
+
result = lock ? write_snapshotted(before) { |rows| rows.delete_all } : super
|
|
191
193
|
safely_publish_bulk("delete_all") { Lyra::BypassEvents.publish_destroys(klass, before, source: "delete_all") }
|
|
192
194
|
result
|
|
193
195
|
end
|
|
@@ -303,6 +305,15 @@ module Lyra
|
|
|
303
305
|
raise StrictDataAccessViolation.new(method_name, klass)
|
|
304
306
|
end
|
|
305
307
|
|
|
308
|
+
# The bulk write, on exactly the rows the locked snapshot holds: no row
|
|
309
|
+
# another transaction inserted since is written without its event. Runs as
|
|
310
|
+
# Lyra's own write, so it is not intercepted a second time.
|
|
311
|
+
def write_snapshotted(before)
|
|
312
|
+
return 0 if before.empty?
|
|
313
|
+
|
|
314
|
+
Lyra.projection_write { yield where(klass.primary_key => before.keys) }
|
|
315
|
+
end
|
|
316
|
+
|
|
306
317
|
def safely_publish_bulk(source)
|
|
307
318
|
yield
|
|
308
319
|
rescue => e
|
data/lib/lyra/version.rb
CHANGED
data/lib/tasks/lyra_mode.rake
CHANGED
|
@@ -51,7 +51,8 @@ namespace :lyra do
|
|
|
51
51
|
abort e.message
|
|
52
52
|
end
|
|
53
53
|
desc "Erase one record's personal data from its row and its events (Art. 17) " \
|
|
54
|
-
"(MODEL=Registration ID=5 REASON='Art. 17 request #12' [FIELDS=email,phone]
|
|
54
|
+
"(MODEL=Registration ID=5 REASON='Art. 17 request #12' [OPERATOR=jdoe] [FIELDS=email,phone] " \
|
|
55
|
+
"[ANY_FIELDS=1: FIELDS may name columns the policy does not declare personal] [EVERYWHERE=1] " \
|
|
55
56
|
"[MAX_COPIES=10: with EVERYWHERE, a value held by more other records is left as shared])"
|
|
56
57
|
task erase: :environment do
|
|
57
58
|
model = ENV.fetch("MODEL").constantize
|
|
@@ -62,7 +63,9 @@ namespace :lyra do
|
|
|
62
63
|
end
|
|
63
64
|
result = Lyra::Erasure.erase!(model, ENV.fetch("ID"), reason: ENV.fetch("REASON"),
|
|
64
65
|
fields: ENV["FIELDS"]&.split(",")&.map(&:strip),
|
|
65
|
-
everywhere: ENV["EVERYWHERE"].present?, max_copies: max_copies
|
|
66
|
+
everywhere: ENV["EVERYWHERE"].present?, max_copies: max_copies,
|
|
67
|
+
any_fields: ENV["ANY_FIELDS"].present?,
|
|
68
|
+
erased_by: Lyra::ModeTransition.actor(ENV["OPERATOR"]))
|
|
66
69
|
puts "Erased #{result.fields.join(', ')} of #{result.model} #{result.id}: " \
|
|
67
70
|
"#{result.events_rewritten} events rewritten, row #{result.row_erased ? 'erased' : 'absent'}."
|
|
68
71
|
puts "Copies erased: #{result.copies.join(', ')}" if result.copies.any?
|