paper_trail_diff 0.17.1 → 0.18.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +661 -0
- data/QUICKSTART.md +12 -14
- data/README.md +32 -30
- data/docs/architecture.md +65 -29
- data/docs/performance.md +161 -46
- data/docs/reference.md +495 -394
- data/docs/testing.md +9 -15
- data/lib/paper_trail_diff/activity_belongs_to_event_applier.rb +42 -9
- data/lib/paper_trail_diff/activity_boundary.rb +15 -6
- data/lib/paper_trail_diff/activity_child_candidate_loader.rb +91 -18
- data/lib/paper_trail_diff/activity_collection_event_applier.rb +14 -15
- data/lib/paper_trail_diff/activity_collection_record_updater.rb +18 -8
- data/lib/paper_trail_diff/activity_collection_route_updater.rb +12 -25
- data/lib/paper_trail_diff/activity_event.rb +53 -34
- data/lib/paper_trail_diff/activity_event_record_normalizer.rb +12 -26
- data/lib/paper_trail_diff/activity_event_record_resolver.rb +45 -32
- data/lib/paper_trail_diff/activity_event_snapshot_refresher.rb +26 -33
- data/lib/paper_trail_diff/activity_history.rb +28 -34
- data/lib/paper_trail_diff/activity_relationship.rb +3 -26
- data/lib/paper_trail_diff/activity_root_snapshot_refresher.rb +14 -8
- data/lib/paper_trail_diff/activity_snapshot_delta.rb +2 -3
- data/lib/paper_trail_diff/activity_snapshot_sequence.rb +92 -25
- data/lib/paper_trail_diff/activity_timeline_builder.rb +128 -77
- data/lib/paper_trail_diff/activity_transaction_applier.rb +49 -0
- data/lib/paper_trail_diff/activity_tree_filter.rb +70 -57
- data/lib/paper_trail_diff/activity_tree_places.rb +76 -0
- data/lib/paper_trail_diff/activity_version_collector.rb +45 -32
- data/lib/paper_trail_diff/activity_view.rb +2 -0
- data/lib/paper_trail_diff/analysis.rb +6 -5
- data/lib/paper_trail_diff/analysis_batch.rb +21 -17
- data/lib/paper_trail_diff/association_activity_edge.rb +99 -85
- data/lib/paper_trail_diff/association_activity_selection.rb +46 -14
- data/lib/paper_trail_diff/association_discovery.rb +4 -4
- data/lib/paper_trail_diff/association_key.rb +10 -18
- data/lib/paper_trail_diff/association_traversal.rb +72 -92
- data/lib/paper_trail_diff/batch_boundary_resolver.rb +1 -7
- data/lib/paper_trail_diff/batch_preload.rb +2 -3
- data/lib/paper_trail_diff/batched_root_analyses.rb +60 -25
- data/lib/paper_trail_diff/batched_root_analyzer.rb +5 -5
- data/lib/paper_trail_diff/batched_root_versions.rb +7 -30
- data/lib/paper_trail_diff/branch_snapshot_refresher.rb +19 -14
- data/lib/paper_trail_diff/candidate_table.rb +160 -0
- data/lib/paper_trail_diff/change_attribution.rb +15 -23
- data/lib/paper_trail_diff/collection_identity_index.rb +68 -24
- data/lib/paper_trail_diff/collection_transition.rb +3 -3
- data/lib/paper_trail_diff/comparison_batch.rb +13 -6
- data/lib/paper_trail_diff/database_time.rb +85 -0
- data/lib/paper_trail_diff/deleted_analysis_batch.rb +22 -23
- data/lib/paper_trail_diff/deleted_root_filter.rb +10 -6
- data/lib/paper_trail_diff/diagnostics.rb +26 -86
- data/lib/paper_trail_diff/endpoint.rb +37 -5
- data/lib/paper_trail_diff/engine.rb +1 -1
- data/lib/paper_trail_diff/errors.rb +0 -6
- data/lib/paper_trail_diff/exact_column_type.rb +96 -0
- data/lib/paper_trail_diff/historical_snapshot_store.rb +88 -83
- data/lib/paper_trail_diff/instrumentation.rb +0 -8
- data/lib/paper_trail_diff/live_endpoint_batch_loader.rb +1 -4
- data/lib/paper_trail_diff/live_graph_collector.rb +1 -3
- data/lib/paper_trail_diff/nested_comparator.rb +42 -10
- data/lib/paper_trail_diff/paper_trail_adapter.rb +55 -98
- data/lib/paper_trail_diff/prepared_activity_index.rb +214 -182
- data/lib/paper_trail_diff/prepared_child_candidates.rb +13 -3
- data/lib/paper_trail_diff/prepared_edge_loader.rb +11 -49
- data/lib/paper_trail_diff/prepared_history.rb +27 -177
- data/lib/paper_trail_diff/prepared_history_loader.rb +39 -57
- data/lib/paper_trail_diff/prepared_record_index.rb +221 -147
- data/lib/paper_trail_diff/prepared_snapshot_builder.rb +47 -42
- data/lib/paper_trail_diff/prepared_state_association_reader.rb +85 -62
- data/lib/paper_trail_diff/recorded_changes.rb +34 -10
- data/lib/paper_trail_diff/recorded_owner.rb +67 -0
- data/lib/paper_trail_diff/root_version_selection.rb +6 -2
- data/lib/paper_trail_diff/scope_condition.rb +120 -80
- data/lib/paper_trail_diff/scope_ranking.rb +14 -32
- data/lib/paper_trail_diff/scoped_analysis.rb +11 -6
- data/lib/paper_trail_diff/scoped_analysis_builder.rb +17 -11
- data/lib/paper_trail_diff/scoped_root_selection.rb +91 -85
- data/lib/paper_trail_diff/selection_window.rb +31 -0
- data/lib/paper_trail_diff/snapshot.rb +60 -58
- data/lib/paper_trail_diff/snapshot_members.rb +115 -0
- data/lib/paper_trail_diff/snapshot_normalizer.rb +0 -11
- data/lib/paper_trail_diff/step.rb +1 -5
- data/lib/paper_trail_diff/stored_form.rb +146 -0
- data/lib/paper_trail_diff/stored_json_values.rb +8 -5
- data/lib/paper_trail_diff/stored_time_precision.rb +67 -33
- data/lib/paper_trail_diff/stored_values.rb +35 -3
- data/lib/paper_trail_diff/support.rb +185 -35
- data/lib/paper_trail_diff/time_of_day.rb +13 -2
- data/lib/paper_trail_diff/time_range.rb +85 -11
- data/lib/paper_trail_diff/time_version_range.rb +5 -1
- data/lib/paper_trail_diff/timeline_builder.rb +5 -11
- data/lib/paper_trail_diff/timeline_range.rb +56 -19
- data/lib/paper_trail_diff/timeline_snapshot_provider.rb +31 -7
- data/lib/paper_trail_diff/total_order.rb +19 -5
- data/lib/paper_trail_diff/transaction_anchors.rb +138 -0
- data/lib/paper_trail_diff/transaction_cut.rb +67 -327
- data/lib/paper_trail_diff/transaction_ends.rb +195 -0
- data/lib/paper_trail_diff/traversal_preparer.rb +12 -24
- data/lib/paper_trail_diff/{approximate_paths.rb → unreadable_paths.rb} +8 -9
- data/lib/paper_trail_diff/value_objects.rb +8 -6
- data/lib/paper_trail_diff/version.rb +1 -1
- data/lib/paper_trail_diff/version_association_candidate_scope.rb +12 -10
- data/lib/paper_trail_diff/version_boundary.rb +0 -29
- data/lib/paper_trail_diff/version_query.rb +49 -1
- data/lib/paper_trail_diff/version_scope_filter.rb +74 -5
- data/lib/paper_trail_diff/version_sequence_diagnostics.rb +4 -31
- data/lib/paper_trail_diff/version_table.rb +8 -1
- data/lib/paper_trail_diff.rb +56 -110
- data/sig/generated/paper_trail_diff/activity_belongs_to_event_applier.rbs +24 -2
- data/sig/generated/paper_trail_diff/activity_boundary.rbs +6 -0
- data/sig/generated/paper_trail_diff/activity_child_candidate_loader.rbs +46 -5
- data/sig/generated/paper_trail_diff/activity_collection_event_applier.rbs +7 -4
- data/sig/generated/paper_trail_diff/activity_collection_record_updater.rbs +11 -2
- data/sig/generated/paper_trail_diff/activity_collection_route_updater.rbs +8 -6
- data/sig/generated/paper_trail_diff/activity_event.rbs +27 -18
- data/sig/generated/paper_trail_diff/activity_event_record_normalizer.rbs +7 -11
- data/sig/generated/paper_trail_diff/activity_event_record_resolver.rbs +26 -8
- data/sig/generated/paper_trail_diff/activity_event_snapshot_refresher.rbs +10 -2
- data/sig/generated/paper_trail_diff/activity_history.rbs +23 -26
- data/sig/generated/paper_trail_diff/activity_relationship.rbs +0 -6
- data/sig/generated/paper_trail_diff/activity_root_snapshot_refresher.rbs +5 -0
- data/sig/generated/paper_trail_diff/activity_snapshot_sequence.rbs +53 -10
- data/sig/generated/paper_trail_diff/activity_timeline_builder.rbs +64 -38
- data/sig/generated/paper_trail_diff/activity_transaction_applier.rbs +28 -0
- data/sig/generated/paper_trail_diff/activity_tree_filter.rbs +46 -38
- data/sig/generated/paper_trail_diff/activity_tree_places.rbs +45 -0
- data/sig/generated/paper_trail_diff/activity_version_collector.rbs +21 -11
- data/sig/generated/paper_trail_diff/analysis.rbs +6 -5
- data/sig/generated/paper_trail_diff/analysis_batch.rbs +8 -6
- data/sig/generated/paper_trail_diff/association_activity_edge.rbs +58 -45
- data/sig/generated/paper_trail_diff/association_activity_selection.rbs +20 -3
- data/sig/generated/paper_trail_diff/association_discovery.rbs +2 -2
- data/sig/generated/paper_trail_diff/association_key.rbs +2 -11
- data/sig/generated/paper_trail_diff/association_traversal.rbs +43 -44
- data/sig/generated/paper_trail_diff/batch_boundary_resolver.rbs +0 -2
- data/sig/generated/paper_trail_diff/batched_root_analyses.rbs +40 -17
- data/sig/generated/paper_trail_diff/batched_root_analyzer.rbs +2 -2
- data/sig/generated/paper_trail_diff/batched_root_versions.rbs +2 -11
- data/sig/generated/paper_trail_diff/branch_snapshot_refresher.rbs +12 -8
- data/sig/generated/paper_trail_diff/candidate_table.rbs +97 -0
- data/sig/generated/paper_trail_diff/change_attribution.rbs +9 -16
- data/sig/generated/paper_trail_diff/collection_identity_index.rbs +33 -13
- data/sig/generated/paper_trail_diff/collection_transition.rbs +2 -2
- data/sig/generated/paper_trail_diff/comparison_batch.rbs +5 -0
- data/sig/generated/paper_trail_diff/database_time.rbs +48 -0
- data/sig/generated/paper_trail_diff/deleted_analysis_batch.rbs +13 -15
- data/sig/generated/paper_trail_diff/deleted_root_filter.rbs +4 -0
- data/sig/generated/paper_trail_diff/diagnostics.rbs +13 -31
- data/sig/generated/paper_trail_diff/endpoint.rbs +14 -1
- data/sig/generated/paper_trail_diff/errors.rbs +0 -8
- data/sig/generated/paper_trail_diff/exact_column_type.rbs +51 -0
- data/sig/generated/paper_trail_diff/historical_snapshot_store.rbs +51 -31
- data/sig/generated/paper_trail_diff/instrumentation.rbs +0 -5
- data/sig/generated/paper_trail_diff/nested_comparator.rbs +21 -0
- data/sig/generated/paper_trail_diff/paper_trail_adapter.rbs +18 -29
- data/sig/generated/paper_trail_diff/prepared_activity_index.rbs +138 -94
- data/sig/generated/paper_trail_diff/prepared_child_candidates.rbs +10 -2
- data/sig/generated/paper_trail_diff/prepared_edge_loader.rbs +4 -21
- data/sig/generated/paper_trail_diff/prepared_history.rbs +22 -79
- data/sig/generated/paper_trail_diff/prepared_history_loader.rbs +24 -32
- data/sig/generated/paper_trail_diff/prepared_record_index.rbs +137 -63
- data/sig/generated/paper_trail_diff/prepared_snapshot_builder.rbs +22 -13
- data/sig/generated/paper_trail_diff/prepared_state_association_reader.rbs +41 -28
- data/sig/generated/paper_trail_diff/recorded_changes.rbs +19 -3
- data/sig/generated/paper_trail_diff/recorded_owner.rbs +39 -0
- data/sig/generated/paper_trail_diff/root_version_selection.rbs +4 -1
- data/sig/generated/paper_trail_diff/scope_condition.rbs +65 -52
- data/sig/generated/paper_trail_diff/scope_ranking.rbs +10 -23
- data/sig/generated/paper_trail_diff/scoped_analysis.rbs +11 -6
- data/sig/generated/paper_trail_diff/scoped_analysis_builder.rbs +7 -2
- data/sig/generated/paper_trail_diff/scoped_root_selection.rbs +49 -43
- data/sig/generated/paper_trail_diff/selection_window.rbs +19 -0
- data/sig/generated/paper_trail_diff/snapshot.rbs +37 -38
- data/sig/generated/paper_trail_diff/snapshot_members.rbs +58 -0
- data/sig/generated/paper_trail_diff/snapshot_normalizer.rbs +0 -4
- data/sig/generated/paper_trail_diff/stored_form.rbs +63 -0
- data/sig/generated/paper_trail_diff/stored_json_values.rbs +0 -2
- data/sig/generated/paper_trail_diff/stored_time_precision.rbs +33 -20
- data/sig/generated/paper_trail_diff/stored_values.rbs +18 -2
- data/sig/generated/paper_trail_diff/support.rbs +96 -16
- data/sig/generated/paper_trail_diff/time_of_day.rbs +4 -0
- data/sig/generated/paper_trail_diff/time_range.rbs +49 -4
- data/sig/generated/paper_trail_diff/time_version_range.rbs +3 -0
- data/sig/generated/paper_trail_diff/timeline_builder.rbs +3 -6
- data/sig/generated/paper_trail_diff/timeline_range.rbs +29 -8
- data/sig/generated/paper_trail_diff/timeline_snapshot_provider.rbs +17 -4
- data/sig/generated/paper_trail_diff/total_order.rbs +9 -3
- data/sig/generated/paper_trail_diff/transaction_anchors.rbs +80 -0
- data/sig/generated/paper_trail_diff/transaction_cut.rbs +43 -176
- data/sig/generated/paper_trail_diff/transaction_ends.rbs +126 -0
- data/sig/generated/paper_trail_diff/traversal_preparer.rbs +6 -11
- data/sig/generated/paper_trail_diff/{approximate_paths.rbs → unreadable_paths.rbs} +6 -7
- data/sig/generated/paper_trail_diff/value_objects.rbs +4 -2
- data/sig/generated/paper_trail_diff/version_association_candidate_scope.rbs +11 -5
- data/sig/generated/paper_trail_diff/version_boundary.rbs +0 -14
- data/sig/generated/paper_trail_diff/version_query.rbs +26 -0
- data/sig/generated/paper_trail_diff/version_scope_filter.rbs +28 -2
- data/sig/generated/paper_trail_diff/version_sequence_diagnostics.rbs +4 -15
- data/sig/generated/paper_trail_diff/version_table.rbs +4 -0
- data/sig/generated/paper_trail_diff.rbs +22 -23
- metadata +28 -20
- data/lib/paper_trail_diff/activity_target_candidate_loader.rb +0 -123
- data/lib/paper_trail_diff/fallback_paths.rb +0 -35
- data/lib/paper_trail_diff/habtm_transaction_owners.rb +0 -70
- data/lib/paper_trail_diff/historical_association_reifier.rb +0 -161
- data/lib/paper_trail_diff/historical_filter.rb +0 -210
- data/lib/paper_trail_diff/own_window.rb +0 -35
- data/lib/paper_trail_diff/prepared_association_reifier.rb +0 -51
- data/sig/generated/paper_trail_diff/activity_target_candidate_loader.rbs +0 -74
- data/sig/generated/paper_trail_diff/fallback_paths.rbs +0 -22
- data/sig/generated/paper_trail_diff/habtm_transaction_owners.rbs +0 -41
- data/sig/generated/paper_trail_diff/historical_association_reifier.rbs +0 -58
- data/sig/generated/paper_trail_diff/historical_filter.rbs +0 -96
- data/sig/generated/paper_trail_diff/own_window.rbs +0 -22
- data/sig/generated/paper_trail_diff/prepared_association_reifier.rbs +0 -27
data/QUICKSTART.md
CHANGED
|
@@ -150,11 +150,11 @@ Read the changes between each pair of article versions:
|
|
|
150
150
|
|
|
151
151
|
<!-- executable:quickstart-timeline -->
|
|
152
152
|
```ruby
|
|
153
|
-
steps = PaperTrailDiff.
|
|
153
|
+
steps = PaperTrailDiff.analyze(
|
|
154
154
|
article,
|
|
155
155
|
from: draft_version,
|
|
156
156
|
to: final_version
|
|
157
|
-
)
|
|
157
|
+
).timeline
|
|
158
158
|
|
|
159
159
|
steps.map do |step|
|
|
160
160
|
change = step.diff.attributes["title"]
|
|
@@ -168,7 +168,7 @@ visible_steps = steps.reject(&:empty?)
|
|
|
168
168
|
Each item in `steps` has a `diff`, just like the comparison above. Steps with
|
|
169
169
|
no reported changes are kept; `reject(&:empty?)` removes them for display.
|
|
170
170
|
|
|
171
|
-
Without `from:` and `to:`, `PaperTrailDiff.
|
|
171
|
+
Without `from:` and `to:`, `PaperTrailDiff.analyze(article)` reads the whole
|
|
172
172
|
history through the current saved article, including the change to
|
|
173
173
|
`"Archived"`. Pass a version as `to:` when the result should stay the same after
|
|
174
174
|
later edits.
|
|
@@ -180,7 +180,7 @@ Use `within:` to choose changes by their PaperTrail version's `created_at`:
|
|
|
180
180
|
<!-- executable:quickstart-time-range -->
|
|
181
181
|
```ruby
|
|
182
182
|
window = draft_version.created_at...final_version.created_at
|
|
183
|
-
ranged_steps = PaperTrailDiff.
|
|
183
|
+
ranged_steps = PaperTrailDiff.analyze(article, within: window).timeline
|
|
184
184
|
|
|
185
185
|
ranged_steps.map do |step|
|
|
186
186
|
change = step.diff.attributes["title"]
|
|
@@ -225,11 +225,10 @@ For one article, use a window ending now:
|
|
|
225
225
|
|
|
226
226
|
<!-- executable:quickstart-current-window -->
|
|
227
227
|
```ruby
|
|
228
|
-
current_steps = PaperTrailDiff.
|
|
228
|
+
current_steps = PaperTrailDiff.analyze(article, within: 1.day.ago..Time.current).timeline
|
|
229
229
|
```
|
|
230
230
|
|
|
231
|
-
The same `within:` option works on `
|
|
232
|
-
`analyze_many`, and `analyze_scope`.
|
|
231
|
+
The same `within:` option works on `analyze_many` and `analyze_scope`.
|
|
233
232
|
|
|
234
233
|
## 6. Choose a method
|
|
235
234
|
|
|
@@ -237,10 +236,8 @@ The same `within:` option works on `activity_timeline`, `analyze`,
|
|
|
237
236
|
| --- | --- |
|
|
238
237
|
| Net difference between two versions, or a version and the current record | `compare(from, to)` |
|
|
239
238
|
| The same for a list of `{ from:, to: }` pairs, one per record | `compare_many(pairs)` |
|
|
240
|
-
|
|
|
241
|
-
|
|
|
242
|
-
| Net difference and timeline together | `analyze(record)` |
|
|
243
|
-
| The same, plus the activity timeline | `analyze(record, activity: true)` |
|
|
239
|
+
| Net difference and each change to one record, version by version | `analyze(record)`, then `diff` and `timeline` |
|
|
240
|
+
| The same, plus each change to its chosen associations, with who made it | `analyze(record, associations: ..., activity: true)`, then `activity_timeline` |
|
|
244
241
|
| Analyses for an array of records | `analyze_many(records)` |
|
|
245
242
|
| Analyses for the records a relation selects, over a time window | `analyze_scope(scope, within:, limit:)` |
|
|
246
243
|
| Recent activity through now, for the records a relation selects | `report(scope, since:, limit:)` |
|
|
@@ -357,12 +354,13 @@ To show the comment creation as its own activity step, without updating the arti
|
|
|
357
354
|
|
|
358
355
|
<!-- executable:quickstart-activity -->
|
|
359
356
|
```ruby
|
|
360
|
-
steps = PaperTrailDiff.
|
|
357
|
+
steps = PaperTrailDiff.analyze(
|
|
361
358
|
article,
|
|
362
359
|
from: before,
|
|
363
360
|
to: current,
|
|
364
|
-
associations: ["comments"]
|
|
365
|
-
|
|
361
|
+
associations: ["comments"],
|
|
362
|
+
activity: true
|
|
363
|
+
).activity_timeline
|
|
366
364
|
|
|
367
365
|
steps.reject(&:empty?).each do |step|
|
|
368
366
|
source = step.source_boundary
|
data/README.md
CHANGED
|
@@ -61,8 +61,8 @@ historical states.
|
|
|
61
61
|
PaperTrail's `version.changeset` describes one recorded change. `compare`
|
|
62
62
|
describes the net difference between the two states you choose. If a field
|
|
63
63
|
changes and then changes back, there is no net difference for that field. To
|
|
64
|
-
see every change instead, use `PaperTrailDiff.
|
|
65
|
-
the article's whole history through its current saved values.
|
|
64
|
+
see every change instead, use `PaperTrailDiff.analyze(article).timeline`, which
|
|
65
|
+
covers the article's whole history through its current saved values.
|
|
66
66
|
|
|
67
67
|
An attribute appears in `diff.attributes` only when it changed. `diff.empty?`
|
|
68
68
|
tells you whether anything changed. A PaperTrail `create` version has no earlier
|
|
@@ -71,7 +71,8 @@ record (`reify` returns `nil`), so comparing it with the current record reports
|
|
|
71
71
|
|
|
72
72
|
To include comments, add `associations: [:comments]`. If comments have replies,
|
|
73
73
|
`associations: ["comments.replies"]` includes those too. Associated models must
|
|
74
|
-
also use `has_paper_trail
|
|
74
|
+
also use `has_paper_trail` and declare the `belongs_to` side of a `has_many` or
|
|
75
|
+
`has_one`, and association tracking must be enabled.
|
|
75
76
|
|
|
76
77
|
For nested results, `each_change` lets you read changes without walking every
|
|
77
78
|
association yourself:
|
|
@@ -101,9 +102,9 @@ PaperTrail.request(whodunnit: "alice") do
|
|
|
101
102
|
end
|
|
102
103
|
end
|
|
103
104
|
|
|
104
|
-
steps = PaperTrailDiff.
|
|
105
|
-
article, associations: [:comments], group: :transaction, snapshots: true
|
|
106
|
-
)
|
|
105
|
+
steps = PaperTrailDiff.analyze(
|
|
106
|
+
article, associations: [:comments], activity: true, group: :transaction, snapshots: true
|
|
107
|
+
).activity_timeline
|
|
107
108
|
|
|
108
109
|
feed = steps.reject(&:empty?).map do |step|
|
|
109
110
|
{
|
|
@@ -122,10 +123,9 @@ and `item_id`. Its `recorded_at` is the version's `created_at`.
|
|
|
122
123
|
Use this source information for attribution: the version used to read the
|
|
123
124
|
resulting state, exposed as `to_boundary`, may have been saved by someone else.
|
|
124
125
|
A change saved without a version, such as with `update_columns`, appears in the
|
|
125
|
-
step for that record's previous recorded change. To
|
|
126
|
-
version recorded,
|
|
127
|
-
|
|
128
|
-
[what each event recorded](docs/reference.md#what-each-event-recorded-recorded_changes-true).
|
|
126
|
+
step for that record's previous recorded change. To read only what each
|
|
127
|
+
version recorded, read `step.source_boundary.recorded_changes`; see
|
|
128
|
+
[what each event recorded](docs/reference.md#what-each-event-recorded).
|
|
129
129
|
|
|
130
130
|
`group: :transaction` combines changes from one database transaction. Its source
|
|
131
131
|
information comes from the first event in that group, and `step.sources` lists
|
|
@@ -164,6 +164,10 @@ report.unreachable # Model names and IDs of selected records no longer in the da
|
|
|
164
164
|
report.deleted # Reports for those records, or nil when destroy history is missing
|
|
165
165
|
```
|
|
166
166
|
|
|
167
|
+
With `associations:`, `report` combines each save of a record and its
|
|
168
|
+
associated records into one step, as `group: :transaction` does above. Pass
|
|
169
|
+
`group: nil` for a step per version.
|
|
170
|
+
|
|
167
171
|
`report.analyses` is a hash of reports for records that still exist. Each key is
|
|
168
172
|
`[model_name, id_as_a_string]`. Each value has the net `diff` and an
|
|
169
173
|
`activity_timeline`. `report.roots` returns the selected ActiveRecord objects,
|
|
@@ -208,10 +212,10 @@ Three selection rules matter:
|
|
|
208
212
|
or deliberately raise the ceiling.
|
|
209
213
|
|
|
210
214
|
Use the relation's `order`, `offset`, and `limit` to choose a page, for example
|
|
211
|
-
`Article.order(:id).limit(20)`.
|
|
215
|
+
`Article.order(:id).limit(20)`. `report.total` counts the records on every page, for
|
|
216
|
+
"page 2 of 6". Joined rows can repeat a record; each record is
|
|
212
217
|
analyzed once. See the [reporting reference](docs/reference.md#which-records-are-selected)
|
|
213
|
-
for pagination
|
|
214
|
-
`historical_filter:`.
|
|
218
|
+
for pagination and deleted records.
|
|
215
219
|
|
|
216
220
|
For a fixed past window, use `analyze_scope(Article.all, within: start_time...end_time,
|
|
217
221
|
activity: true, limit: 100)`. Without `activity: true`, `analyze_scope` compares
|
|
@@ -260,10 +264,11 @@ article's new author, names the step's first event. `row.source` is the
|
|
|
260
264
|
`ActivityBoundary` itself.
|
|
261
265
|
|
|
262
266
|
A write made without a version, such as `update_columns`, shows up in the
|
|
263
|
-
step of the record's previous change.
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
`
|
|
267
|
+
step of the record's previous change. Its rows tell it apart: the part of a
|
|
268
|
+
change PaperTrail did not record gets a row of its own, with
|
|
269
|
+
`row.unversioned?` true and no `version_id`, `event`, `whodunnit`, or
|
|
270
|
+
`recorded_at`, so it is never credited to whoever saved next. To list only
|
|
271
|
+
what versions recorded:
|
|
267
272
|
|
|
268
273
|
```ruby
|
|
269
274
|
recorded = report.change_rows.transform_values { |rows| rows.reject(&:unversioned?) }
|
|
@@ -281,7 +286,7 @@ author:
|
|
|
281
286
|
```ruby
|
|
282
287
|
report = PaperTrailDiff.report(
|
|
283
288
|
Article.all, since: 1.day.ago, limit: 100, associations: [:comments],
|
|
284
|
-
snapshots: true
|
|
289
|
+
snapshots: true
|
|
285
290
|
)
|
|
286
291
|
|
|
287
292
|
changes = report.net_changes { |row| row.source.whodunnit }
|
|
@@ -300,16 +305,14 @@ activity step. Labels, grouping, and formatting are left to your application.
|
|
|
300
305
|
| --- | --- |
|
|
301
306
|
| Net difference between two versions, or a version and the current record | `compare(from, to)` |
|
|
302
307
|
| The same for a list of `{ from:, to: }` pairs, one per record | `compare_many(pairs)` |
|
|
303
|
-
|
|
|
304
|
-
|
|
|
305
|
-
| Net difference and timeline together | `analyze(record)` |
|
|
306
|
-
| The same, plus the activity timeline | `analyze(record, activity: true)` |
|
|
308
|
+
| Net difference and each change to one record, version by version | `analyze(record)`, then `diff` and `timeline` |
|
|
309
|
+
| The same, plus each change to its chosen associations, with who made it | `analyze(record, associations: ..., activity: true)`, then `activity_timeline` |
|
|
307
310
|
| Analyses for an array of records | `analyze_many(records)` |
|
|
308
311
|
| Analyses for the records a relation selects, over a time window | `analyze_scope(scope, within:, limit:)` |
|
|
309
312
|
| Recent activity through now, for the records a relation selects | `report(scope, since:, limit:)` |
|
|
310
313
|
|
|
311
|
-
`
|
|
312
|
-
|
|
314
|
+
`analyze` covers the whole history through the current saved record by
|
|
315
|
+
default. Narrow them with `from:` and `to:`
|
|
313
316
|
(versions) or `within:` (a time range). See
|
|
314
317
|
[choosing a range](docs/reference.md#choosing-a-range).
|
|
315
318
|
|
|
@@ -326,12 +329,11 @@ Scopes and default scopes made of plain conditions, such as
|
|
|
326
329
|
primary key, such as `-> { order(:position, :id).limit(3) }`, on a direct
|
|
327
330
|
association or a `has_many :through`. History cannot read a few scopes, such
|
|
328
331
|
as a `limit` ordered by a string, a scope that takes the owner, or SQL text.
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
Remove it from `associations:`, or
|
|
332
|
-
|
|
333
|
-
|
|
334
|
-
Callbacks and callable historical filters can also issue queries. See the
|
|
332
|
+
Who was in such an association at a past point cannot be told, so every
|
|
333
|
+
method that reads history raises `UnsupportedAssociationError` naming the
|
|
334
|
+
association. Remove it from `associations:`, or rewrite its scope. See
|
|
335
|
+
[association scopes](docs/reference.md#association-scopes).
|
|
336
|
+
Callbacks can also issue queries. See the
|
|
335
337
|
[performance guide](docs/performance.md) for measured behavior and limitations.
|
|
336
338
|
|
|
337
339
|
On a large version table, add these indexes, or every `since:` or `within:`
|
data/docs/architecture.md
CHANGED
|
@@ -5,10 +5,12 @@ rebuilds history and what that costs. You do not need it to use the gem.
|
|
|
5
5
|
|
|
6
6
|
## Two ways to rebuild a timeline
|
|
7
7
|
|
|
8
|
-
`
|
|
9
|
-
|
|
8
|
+
`analyze` returns the same root-record steps for the same range with
|
|
9
|
+
`activity: true` or without, except where a transaction crosses a window's
|
|
10
|
+
edge: without it, each root version is reported where it was saved, and with
|
|
11
|
+
it, activity on associations reports a transaction in the window it ends in. They build them differently:
|
|
10
12
|
|
|
11
|
-
- `
|
|
13
|
+
- Without `activity:`, `analyze` rebuilds the root record and every chosen association separately
|
|
12
14
|
at each root version. Its cost grows with *root versions × size of the chosen
|
|
13
15
|
associations*. It does not depend on how often the associations changed.
|
|
14
16
|
- `analyze(activity: true)` rebuilds the state once, then applies each recorded
|
|
@@ -18,7 +20,7 @@ the same range. They build them differently:
|
|
|
18
20
|
Neither is always faster. Choose by the output you need. If performance matters,
|
|
19
21
|
measure on real history with the [instrumentation events](reference.md#instrumentation).
|
|
20
22
|
As a rough guide, `analyze(activity: true)` suits wide associations with modest
|
|
21
|
-
activity, and `
|
|
23
|
+
activity, and `analyze` without it suits a few root versions with many child events
|
|
22
24
|
between them.
|
|
23
25
|
|
|
24
26
|
## Loading history
|
|
@@ -62,31 +64,65 @@ previous snapshot forward and replace only the parts an event changed, so an
|
|
|
62
64
|
edit to one comment does not copy the whole article again. Events in one
|
|
63
65
|
PT-AT transaction are applied together.
|
|
64
66
|
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
67
|
+
A transaction takes effect where it saved its last version, of any record.
|
|
68
|
+
`RecordedTransactionEnds` reads that version from the version table, for all
|
|
69
|
+
the versions a history holds at once, since which of a transaction's records
|
|
70
|
+
a timeline holds depends on where it is read from. `TransactionEnds` places
|
|
71
|
+
every version of a transaction at that last one, and a timeline replays its events in that order, so two
|
|
72
|
+
transactions that interleaved their versions in one tree are read one after
|
|
73
|
+
the other, in the order they ended. The record index reads a record at a
|
|
74
|
+
boundary as it stood before every transaction that ends at or after the
|
|
75
|
+
boundary's own. A transaction thus changes no snapshot while it is open, and
|
|
76
|
+
a change is reported with its own transaction's, under its own version. A
|
|
77
|
+
snapshot read for a timeline whose transactions interleaved is cached apart
|
|
78
|
+
from one read without.
|
|
79
|
+
|
|
80
|
+
A window reports the transactions that end inside it. Its history is first
|
|
81
|
+
read over the window itself. `CrossingTransactions` then says how far back
|
|
82
|
+
its transactions reach: where the earliest transaction open at its start
|
|
83
|
+
began, whatever records it saved. When one was open, the root's versions and
|
|
84
|
+
the history are read once more from there, the whole of it is replayed, and
|
|
85
|
+
the steps of the transactions that end inside the window are reported. A
|
|
86
|
+
window is not read past its end. A transaction that saves inside it and ends
|
|
87
|
+
later is left to the later window.
|
|
88
|
+
|
|
89
|
+
A range of versions, and a whole history read without a window, is read the
|
|
90
|
+
same way. It starts before its first version's transaction
|
|
91
|
+
(`TransactionCut.range_start`), and its history is read from where a
|
|
92
|
+
transaction open there began (`TransactionCut.crossing_start`): what another
|
|
93
|
+
transaction saved before the start is part of the starting state if that
|
|
94
|
+
transaction ended first, and is reported in its own step if it ended later.
|
|
95
|
+
It ends where its last version's transaction takes effect
|
|
96
|
+
(`TransactionCut.range_end`), so what another transaction committed before
|
|
97
|
+
then is read, and the steps of transactions that end after it are left out.
|
|
98
|
+
`compare` reads each version the same way, so one version reads alike as the
|
|
99
|
+
end of one range, the start of the next, and an endpoint of `compare`. A
|
|
100
|
+
destroyed root's tree is read up to where its destruction takes effect, and
|
|
101
|
+
nothing saved there by a transaction that ended later is the root's. Without
|
|
102
|
+
associations no other record is read, so each of the record's versions takes
|
|
103
|
+
effect where it was saved: row locks keep two transactions from interleaving
|
|
104
|
+
one record's versions.
|
|
105
|
+
|
|
106
|
+
## What history cannot read
|
|
107
|
+
|
|
108
|
+
Every association is read from the loaded history, or not at all. These
|
|
109
|
+
shapes are refused, since who was in one at a past point cannot be told from
|
|
110
|
+
stored versions:
|
|
72
111
|
|
|
73
112
|
- associations with a scope the gem cannot read as plain conditions and an
|
|
74
113
|
exact ranking, including owner-dependent scopes and a `limit` without a
|
|
75
114
|
total integer order;
|
|
76
|
-
- targets with such a default scope
|
|
77
|
-
target,
|
|
115
|
+
- targets with such a default scope. A polymorphic `belongs_to` names its
|
|
116
|
+
target's class only in each record, so one that points at such a class is
|
|
117
|
+
refused when that record is read;
|
|
78
118
|
- composite foreign keys.
|
|
79
119
|
|
|
80
|
-
|
|
81
|
-
`allow_approximate: true`; see [costly associations](reference.md#costly-associations).
|
|
120
|
+
See [association scopes](reference.md#association-scopes).
|
|
82
121
|
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
history is read. Even at the root, a transaction in which another record with a
|
|
88
|
-
HABTM of the same name saved a version holds both records' members;
|
|
89
|
-
`HabtmTransactionOwners` finds those transactions, and reading one raises.
|
|
122
|
+
A `has_and_belongs_to_many` is refused before any history is read: its join
|
|
123
|
+
rows have no versions. The reference shows how to
|
|
124
|
+
[replace one](reference.md#replacing-a-has_and_belongs_to_many) with a
|
|
125
|
+
`has_many :through` a versioned join model.
|
|
90
126
|
|
|
91
127
|
## Database indexes
|
|
92
128
|
|
|
@@ -125,10 +161,10 @@ Keep PT-AT's own index on `version_associations` (`foreign_key_name`,
|
|
|
125
161
|
- Repeated events in one very wide collection still copy that collection's
|
|
126
162
|
list of records for each step. The comparison work stays small, but memory
|
|
127
163
|
traffic grows with *events × collection size*.
|
|
128
|
-
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
takes one statement per
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
164
|
+
- A read with associations asks its version table where each transaction
|
|
165
|
+
it holds ends, and a `within:` or `since:` window asks which transactions
|
|
166
|
+
were open at its start (see [time windows](reference.md#time-windows)).
|
|
167
|
+
The first takes one statement per 500 transactions; the second reads the
|
|
168
|
+
versions saved since the window began, which no index of the README
|
|
169
|
+
serves. See [performance](performance.md#what-still-grows) for the cost on
|
|
170
|
+
a large table.
|
data/docs/performance.md
CHANGED
|
@@ -26,9 +26,7 @@ for one association, at 1, 5, and 20 records:
|
|
|
26
26
|
| Polymorphic `belongs_to` | 17 / 17 / 17 | 0 |
|
|
27
27
|
| `has_many :through` with `source_type:` | 21 / 21 / 21 | 0 |
|
|
28
28
|
| A `has_many` with `order(:id).limit(1)` | 14 / 18 / 33 | 0 |
|
|
29
|
-
| A scope that takes the owner, read with PT-AT | 25 / 73 / 253 | 9 / 45 / 180 |
|
|
30
29
|
| None, with a querying `after_initialize` | 10 / 26 / 86 | 2 / 10 / 40 |
|
|
31
|
-
| None, with a callable `historical_filter:` that queries | 11 / 15 / 30 | 2 / 10 / 40 |
|
|
32
30
|
|
|
33
31
|
The last four rows grow with the number of records, for the reasons in
|
|
34
32
|
[what costs a query per record](#what-costs-a-query-per-record).
|
|
@@ -57,6 +55,12 @@ Flat query counts do not make a report's cost flat:
|
|
|
57
55
|
[history stored as JSON](reference.md#history-stored-as-json)). PaperTrail
|
|
58
56
|
reads every stored version with the one serializer, so switching means
|
|
59
57
|
converting the versions already saved.
|
|
58
|
+
- **Grouping by transaction costs little.** `report` with `associations:`
|
|
59
|
+
groups by default. On the `association_window` benchmark (articles created
|
|
60
|
+
with two comments in one transaction, then edited with one of them in
|
|
61
|
+
another), 2,000 articles took 2.6 to 3.2 s grouped and 2.6 to 3.4 s with
|
|
62
|
+
`group: nil`, three samples each, with the same statements, about 4% more
|
|
63
|
+
allocations, and slightly less retained memory.
|
|
60
64
|
- **Memory grows with the number of records.** A `report` of 1,000 articles
|
|
61
65
|
with their comments, replies, and author peaked at about 100 KB per
|
|
62
66
|
article. `limit:` is a safety ceiling, not a page size, so to bound memory,
|
|
@@ -65,30 +69,87 @@ Flat query counts do not make a report's cost flat:
|
|
|
65
69
|
- **Work on a wide collection grows with its size times its history.** Each
|
|
66
70
|
step of an activity timeline copies the collection's list of records, so
|
|
67
71
|
an activity timeline over `n` comments that changed `m` times does about
|
|
68
|
-
`n × m` work: 800 comments edited 1,000 times took about
|
|
72
|
+
`n × m` work: 800 comments edited 1,000 times took about 0.4 s, and 3,200
|
|
73
|
+
edited 3,200 times about 1.8 s. A transaction that saves several records
|
|
74
|
+
costs the same for each: its versions are applied to the records they
|
|
75
|
+
saved at the event after its last, so 1,000 comments edited two at a time
|
|
76
|
+
in 1,000 transactions took 1.9 s, and edited one at a time along with
|
|
77
|
+
their article, 1.0 s. A record that joins a collection or leaves it costs
|
|
78
|
+
a little more than an edit: 1,000 comments created one at a time on one
|
|
79
|
+
article took about 0.8 s, and 3,000 about 1.8 s. Records nested under a
|
|
80
|
+
wide collection cost no more: 2,000 replies created one at a time under
|
|
81
|
+
2,000 comments took about 1.4 s. Members
|
|
69
82
|
saved in one transaction share one snapshot: a `report` on 100 articles,
|
|
70
83
|
each created with 100 comments in one transaction and edited with all of
|
|
71
84
|
them in another, took about 4.6 s. `timeline` reads every member at each
|
|
72
85
|
version of the root instead: 300 comments over 2,001 versions of their
|
|
73
86
|
article took about 7.5 s.
|
|
74
|
-
-
|
|
75
|
-
|
|
87
|
+
- **A range that ended in the past does not read what was created since.**
|
|
88
|
+
A child whose first version is a `create` saved after the range's end is
|
|
89
|
+
left out when the children are found, with its row and versions: with 2,000
|
|
90
|
+
comments added to an article after the range, `compare`, `timeline`, and a
|
|
91
|
+
window that a later version of the article ends each took 10 to 13 ms, as
|
|
92
|
+
with none. A window with `activity: true` on a record that has not been
|
|
93
|
+
saved since is the exception: it closes on the record as it is now, so it
|
|
94
|
+
replays everything up to now and reports what fell inside the window, which
|
|
95
|
+
took about 0.8 s with those 2,000 comments.
|
|
96
|
+
- **A record that a path reaches twice has its branch read again at each
|
|
97
|
+
of its events.** An article read with `associations: ["author.articles"]`
|
|
98
|
+
is the root and one of its author's articles, and under
|
|
99
|
+
`articles.comments.article` an article is a member of a collection and the
|
|
100
|
+
target of a `belongs_to` below it. An event on a record of such a model,
|
|
101
|
+
or on any record when the tree holds a polymorphic `belongs_to`, which can
|
|
102
|
+
point at a record of any model, is not applied to the previous snapshot in
|
|
103
|
+
place: the branches that hold the record are read again, or the whole
|
|
104
|
+
snapshot for the root, so the event costs the size of what is read. An
|
|
105
|
+
article with `author.articles` and 400 events on the author's articles
|
|
106
|
+
took 0.3 s when the author had 10 articles, 0.5 s with 100, and 3.4 s with
|
|
107
|
+
1,000; with `associations: ["author"]`, 0.1 s for each.
|
|
108
|
+
- **Change rows decode what each version recorded.** Activity keeps each
|
|
109
|
+
version until its rows are read, then decodes its `object_changes` to
|
|
110
|
+
credit each change to the version that recorded it. On SQLite, a `report`
|
|
111
|
+
of 2,000 articles with three comments each (16,000 versions) took 5.6 s
|
|
112
|
+
with or without them and held 77 MB against 55 MB; listing its 32,000
|
|
113
|
+
change rows then took 3.0 s against 0.4 s. `recorded_changes: false`
|
|
114
|
+
skips both, and credits a field's whole change to its record's last
|
|
115
|
+
version.
|
|
116
|
+
- **`snapshots: true` costs memory.** Ask for it only when you use the
|
|
117
|
+
snapshots. Records that did not change are shared
|
|
76
118
|
between steps, so the cost is modest: an activity `report` of 10,000
|
|
77
119
|
articles with their 57,000 comments (114,000 versions, 49,000 steps) took
|
|
78
120
|
about 40 s and held 372 MB, and 411 MB with snapshots, at the same
|
|
79
121
|
time within run-to-run noise. Folding its rows with `net_changes` took
|
|
80
122
|
another 17 s, and listing them with `change_rows` 13 s.
|
|
81
|
-
- **
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
123
|
+
- **A window with associations looks for the transactions open at its
|
|
124
|
+
start.** A window reports the transactions that end inside it, so its
|
|
125
|
+
history is asked where the earliest transaction open at its start began,
|
|
126
|
+
whatever records it saved. That takes one statement per version table the
|
|
127
|
+
tree's models use, or per version table of the application when a path
|
|
128
|
+
passes through a polymorphic `belongs_to`, and one per 500 transactions its
|
|
129
|
+
history holds, to read where each ends. It is asked before the window is
|
|
130
|
+
read, so the history is read once, from where the earliest began, and
|
|
131
|
+
`analyze_scope` and `report` ask it once for record selection and every
|
|
132
|
+
record of the model. A report on 100 articles with their comments and an
|
|
133
|
+
unrelated transaction open at the window's start ran 19 statements. The
|
|
134
|
+
statement for the start reads the versions saved since the window began
|
|
135
|
+
that were not the first of their transaction, and no index of the README
|
|
136
|
+
serves that. On SQLite, in a table of a million versions with the README's
|
|
137
|
+
indexes and PT-AT's on `transaction_id`, it took 0.06 s for a window that
|
|
138
|
+
began 1% of the table ago, 0.18 s for one that began half of it ago, and
|
|
139
|
+
0.27 s for one that began near its start. An index on `created_at` alone
|
|
140
|
+
took the first to 0.006 s and left the others as they were. A read without associations runs none of these
|
|
141
|
+
statements, with a window or not.
|
|
142
|
+
- **A read with associations reads where its transactions end**, with
|
|
143
|
+
`activity: true` or without. A
|
|
144
|
+
transaction takes effect at the last version it saved, of any record, so a
|
|
145
|
+
history asks its version table for the last version of each transaction it
|
|
146
|
+
holds: one statement per 500 transactions, and one more when some of those
|
|
147
|
+
versions are of records it does not hold. Each reads by `transaction_id`,
|
|
148
|
+
which PT-AT indexes: in the same table, the ends of 10,000 transactions
|
|
149
|
+
took 0.11 s in 20 statements, and 0.19 s in 40 when none of their last
|
|
150
|
+
versions was held. With several version tables, one more
|
|
151
|
+
statement per table per 500 finds the numbers two tables both gave a
|
|
152
|
+
transaction.
|
|
92
153
|
- **`version_scope:` on activity reads every version in the range**, as
|
|
93
154
|
activity without one does. With associations, it takes one more statement
|
|
94
155
|
per 500 of the transactions its history reads, to find those the filter
|
|
@@ -103,40 +164,95 @@ records that changed: with 5 million versions on PostgreSQL, about 230 ms per
|
|
|
103
164
|
report instead of 2 ms. [Database indexes](architecture.md#database-indexes)
|
|
104
165
|
explains why.
|
|
105
166
|
|
|
167
|
+
Selecting a report's records does not grow with the model's own table. The
|
|
168
|
+
records changed in the window and deleted since are found with `NOT EXISTS`,
|
|
169
|
+
which every database plans as an anti-join. With 2 million articles on
|
|
170
|
+
PostgreSQL 17 and 2,000 of them edited in the window, that statement took 3 ms
|
|
171
|
+
and the `report` 1.4 s. Asked with `NOT IN`, which PostgreSQL reads the whole
|
|
172
|
+
table for, it took 14.8 s of a 16 s report, and 9.7 s of 11 s with 500,000
|
|
173
|
+
articles.
|
|
174
|
+
|
|
106
175
|
## What costs a query per record
|
|
107
176
|
|
|
108
|
-
- **A path PT-AT reads.** History cannot read a few scopes, such as one that
|
|
109
|
-
takes the owner, SQL text, or a `limit` ordered by a string. PT-AT reads
|
|
110
|
-
those, one record at a time and only approximately, so every method refuses
|
|
111
|
-
them unless you pass `allow_approximate: true`. See
|
|
112
|
-
[costly associations](reference.md#costly-associations).
|
|
113
177
|
- **A limited association on the current record.** Active Record's `preload`
|
|
114
178
|
applies a `limit` or `offset` across all the records it loads, so each
|
|
115
179
|
record's limited association is loaded on its own: one query per record.
|
|
116
180
|
- **Your own code.** Model callbacks that query, such as an `after_initialize`,
|
|
117
|
-
custom attribute types
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
`
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
181
|
+
and custom attribute types run for each state they are given.
|
|
182
|
+
|
|
183
|
+
## Selecting records when key types differ
|
|
184
|
+
|
|
185
|
+
A version table whose `item_id` does not share the model's key type, such as
|
|
186
|
+
a string `item_id` beside an integer key, cannot be compared with the key in
|
|
187
|
+
SQL. The changed ids are then read first, and the relation reads its records
|
|
188
|
+
among them in pages of up to 2,000, each page listing every id, so the
|
|
189
|
+
relation keeps its own order, `limit`, and `offset`. On SQL Server, more than 500 matching
|
|
190
|
+
records are first written to a temporary table, in statements of 1,000 rows,
|
|
191
|
+
and each page selects from it instead of listing them, since SQL Server
|
|
192
|
+
compiles a statement with its ids written in afresh. The table is dropped
|
|
193
|
+
when the selection ends. A read that prevents writes, as a replica read
|
|
194
|
+
through Rails' automatic role switching does, lists the ids instead, since
|
|
195
|
+
Active Record counts the table's statements as writes. On a local SQL Server 2022 container, with the plan
|
|
196
|
+
cache cleared before each run:
|
|
197
|
+
|
|
198
|
+
| Matching records | Listing every id | Temporary table |
|
|
199
|
+
| --- | --- | --- |
|
|
200
|
+
| 10,000, selection only | 5.0 s | 0.6 s |
|
|
201
|
+
| 20,000, selection only | 29.7 s | 1.1 s |
|
|
202
|
+
| 40,000, selection only | not run | 2.2 s |
|
|
203
|
+
|
|
204
|
+
Each is one run.
|
|
205
|
+
Other databases still list the ids on each page, which they parse quickly.
|
|
206
|
+
|
|
207
|
+
## Selecting records with associations on SQL Server
|
|
208
|
+
|
|
209
|
+
`analyze_scope` and `report` with `associations:` select a record when any
|
|
210
|
+
of several subqueries names it: its own versions in the window, or the
|
|
211
|
+
versions of a record on one of its paths. A single statement that joins
|
|
212
|
+
them with `OR`, as other databases run, SQL Server planned as one: it
|
|
213
|
+
walked the records in order and ran an association's subquery again for
|
|
214
|
+
each, though each subquery alone took under 0.2 s. On SQL Server the gem
|
|
215
|
+
therefore fills a temporary table from the subqueries, one statement each,
|
|
216
|
+
and every page selects from it. A read that prevents writes, as a replica
|
|
217
|
+
read through Rails' automatic role switching does, still runs the single
|
|
218
|
+
statement, since Active Record counts the table's statements as writes.
|
|
219
|
+
|
|
220
|
+
Selecting the orders changed in a window, from 1,600 orders with their line
|
|
221
|
+
items and 28,729 versions, `associations: [:line_items]`, on a local SQL
|
|
222
|
+
Server 2019 container with the plan cache cleared before each run:
|
|
223
|
+
|
|
224
|
+
| Window | Orders selected | One statement | Temporary table |
|
|
225
|
+
| --- | --- | --- | --- |
|
|
226
|
+
| A week | 540 | 35.1 s | 0.43 s |
|
|
227
|
+
| A month | 1,201 | 20.0 s | 0.36 s |
|
|
228
|
+
|
|
229
|
+
The one-statement times are one run each; the temporary-table times are
|
|
230
|
+
the median of three. The rest of the report is unchanged: there, a page of
|
|
231
|
+
100 of the week's orders, chosen with the relation's `order`, `limit`, and
|
|
232
|
+
`offset`, took 0.9 to 1.3 s in all, counting the 540 for `total` included.
|
|
233
|
+
|
|
234
|
+
## Selecting records whose children had no version
|
|
235
|
+
|
|
236
|
+
A `has_many` or `has_one` child with no version yet, such as a row from
|
|
237
|
+
before PaperTrail, holds the owner it leaves only in the stored state of its
|
|
238
|
+
first version. `analyze_scope` and `report` read those first versions
|
|
239
|
+
since the window's start for the children a path reaches, one statement per
|
|
240
|
+
source of changes below each such edge, and decode each version's state in
|
|
241
|
+
Ruby. The cost grows with the children whose first version falls in the
|
|
242
|
+
window.
|
|
243
|
+
|
|
244
|
+
A page of 25 from 3,000 orders with 15,000 line items saved without
|
|
245
|
+
versions, after 3,000 of the items got their first version in the window
|
|
246
|
+
(half moved to another order), `report(..., associations: [:line_items])`,
|
|
247
|
+
median of five runs, plan cache cleared before each on SQL Server:
|
|
248
|
+
|
|
249
|
+
| Database | Before | After |
|
|
250
|
+
| --- | --- | --- |
|
|
251
|
+
| SQLite | 0.31 s | 0.42 s |
|
|
252
|
+
| SQL Server 2019 (local container) | 0.60 s | 0.74 s |
|
|
253
|
+
|
|
254
|
+
Once those rows have versions, each later move is found from
|
|
255
|
+
PaperTrail-AssociationTracking's rows, and no state is decoded.
|
|
140
256
|
|
|
141
257
|
## Large reports
|
|
142
258
|
|
|
@@ -157,8 +273,7 @@ mise exec -- bundle exec ruby benchmark/reporting.rb --case many_roots --samples
|
|
|
157
273
|
|
|
158
274
|
`reconstruction.rb` prints the queries, `reify` calls, and model instances
|
|
159
275
|
one `report` makes for a scenario and a number of records; the table above
|
|
160
|
-
comes from its scenarios.
|
|
161
|
-
`allow_approximate: true`.
|
|
276
|
+
comes from its scenarios.
|
|
162
277
|
`reporting.rb` times `analyze_scope` over articles with no associations and,
|
|
163
278
|
as `association_window`, a `report` over articles with their comments read
|
|
164
279
|
through PT-AT, each in a fresh process, and records memory; `--output` keeps
|