paper_trail_diff 0.14.0 → 0.16.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (150) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +501 -0
  3. data/QUICKSTART.md +21 -4
  4. data/README.md +59 -56
  5. data/docs/architecture.md +32 -11
  6. data/docs/performance.md +125 -391
  7. data/docs/reference.md +470 -117
  8. data/docs/testing.md +83 -7
  9. data/lib/paper_trail_diff/activity_boundary.rb +83 -26
  10. data/lib/paper_trail_diff/activity_child_candidate_loader.rb +2 -1
  11. data/lib/paper_trail_diff/activity_collection_event_applier.rb +33 -1
  12. data/lib/paper_trail_diff/activity_event.rb +41 -10
  13. data/lib/paper_trail_diff/activity_event_route_finder.rb +31 -0
  14. data/lib/paper_trail_diff/activity_event_snapshot_refresher.rb +12 -1
  15. data/lib/paper_trail_diff/activity_history.rb +34 -9
  16. data/lib/paper_trail_diff/activity_range.rb +18 -3
  17. data/lib/paper_trail_diff/activity_root_snapshot_refresher.rb +43 -25
  18. data/lib/paper_trail_diff/activity_snapshot_sequence.rb +76 -37
  19. data/lib/paper_trail_diff/activity_target_candidate_loader.rb +5 -5
  20. data/lib/paper_trail_diff/activity_timeline_builder.rb +272 -111
  21. data/lib/paper_trail_diff/activity_transaction_grouper.rb +2 -1
  22. data/lib/paper_trail_diff/activity_tree_filter.rb +114 -0
  23. data/lib/paper_trail_diff/activity_version_collector.rb +7 -5
  24. data/lib/paper_trail_diff/activity_view.rb +69 -0
  25. data/lib/paper_trail_diff/analysis.rb +37 -6
  26. data/lib/paper_trail_diff/analysis_batch.rb +25 -31
  27. data/lib/paper_trail_diff/approximate_paths.rb +48 -0
  28. data/lib/paper_trail_diff/association_activity_edge.rb +6 -6
  29. data/lib/paper_trail_diff/association_discovery.rb +9 -1
  30. data/lib/paper_trail_diff/association_key.rb +56 -0
  31. data/lib/paper_trail_diff/association_traversal.rb +13 -4
  32. data/lib/paper_trail_diff/batch_boundary_resolver.rb +3 -4
  33. data/lib/paper_trail_diff/batch_preload.rb +53 -0
  34. data/lib/paper_trail_diff/batched_root_analyzer.rb +18 -14
  35. data/lib/paper_trail_diff/batched_root_versions.rb +38 -14
  36. data/lib/paper_trail_diff/branch_snapshot_refresher.rb +19 -16
  37. data/lib/paper_trail_diff/change_row.rb +133 -0
  38. data/lib/paper_trail_diff/collection_comparator.rb +5 -32
  39. data/lib/paper_trail_diff/configuration.rb +35 -12
  40. data/lib/paper_trail_diff/deleted_analysis_batch.rb +23 -19
  41. data/lib/paper_trail_diff/deleted_root_filter.rb +101 -0
  42. data/lib/paper_trail_diff/deprecation.rb +61 -0
  43. data/lib/paper_trail_diff/diagnostics.rb +86 -99
  44. data/lib/paper_trail_diff/endpoint.rb +26 -3
  45. data/lib/paper_trail_diff/errors.rb +3 -1
  46. data/lib/paper_trail_diff/fallback_paths.rb +38 -0
  47. data/lib/paper_trail_diff/historical_association_reifier.rb +2 -4
  48. data/lib/paper_trail_diff/historical_filter.rb +19 -10
  49. data/lib/paper_trail_diff/historical_snapshot_store.rb +57 -67
  50. data/lib/paper_trail_diff/live_endpoint_batch_loader.rb +1 -35
  51. data/lib/paper_trail_diff/nested_comparator.rb +19 -18
  52. data/lib/paper_trail_diff/paper_trail_adapter.rb +198 -166
  53. data/lib/paper_trail_diff/preloaded_endpoint_batch_loader.rb +13 -1
  54. data/lib/paper_trail_diff/prepared_activity_index.rb +17 -13
  55. data/lib/paper_trail_diff/prepared_edge_loader.rb +2 -1
  56. data/lib/paper_trail_diff/prepared_history.rb +15 -1
  57. data/lib/paper_trail_diff/prepared_history_loader.rb +2 -3
  58. data/lib/paper_trail_diff/prepared_record_index.rb +42 -7
  59. data/lib/paper_trail_diff/read_snapshot.rb +77 -0
  60. data/lib/paper_trail_diff/root_version_selection.rb +4 -2
  61. data/lib/paper_trail_diff/scope_condition.rb +157 -32
  62. data/lib/paper_trail_diff/scope_ranking.rb +23 -14
  63. data/lib/paper_trail_diff/scoped_analysis.rb +11 -0
  64. data/lib/paper_trail_diff/scoped_analysis_builder.rb +29 -11
  65. data/lib/paper_trail_diff/scoped_root_selection.rb +29 -16
  66. data/lib/paper_trail_diff/snapshot.rb +16 -1
  67. data/lib/paper_trail_diff/support.rb +29 -2
  68. data/lib/paper_trail_diff/time_range.rb +39 -16
  69. data/lib/paper_trail_diff/timeline_builder.rb +13 -7
  70. data/lib/paper_trail_diff/timeline_range.rb +9 -1
  71. data/lib/paper_trail_diff/timeline_snapshot_provider.rb +24 -17
  72. data/lib/paper_trail_diff/transaction_cut.rb +240 -0
  73. data/lib/paper_trail_diff/traversal_preparer.rb +134 -29
  74. data/lib/paper_trail_diff/version.rb +1 -1
  75. data/lib/paper_trail_diff/version_boundary.rb +41 -10
  76. data/lib/paper_trail_diff/version_query.rb +34 -10
  77. data/lib/paper_trail_diff/version_scope_filter.rb +9 -10
  78. data/lib/paper_trail_diff/version_sequence_diagnostics.rb +35 -17
  79. data/lib/paper_trail_diff/version_table.rb +60 -0
  80. data/lib/paper_trail_diff.rb +101 -57
  81. data/sig/generated/paper_trail_diff/activity_boundary.rbs +54 -19
  82. data/sig/generated/paper_trail_diff/activity_collection_event_applier.rbs +13 -0
  83. data/sig/generated/paper_trail_diff/activity_event.rbs +25 -4
  84. data/sig/generated/paper_trail_diff/activity_event_route_finder.rbs +14 -0
  85. data/sig/generated/paper_trail_diff/activity_event_snapshot_refresher.rbs +6 -0
  86. data/sig/generated/paper_trail_diff/activity_history.rbs +21 -6
  87. data/sig/generated/paper_trail_diff/activity_range.rbs +9 -2
  88. data/sig/generated/paper_trail_diff/activity_root_snapshot_refresher.rbs +21 -10
  89. data/sig/generated/paper_trail_diff/activity_snapshot_sequence.rbs +48 -9
  90. data/sig/generated/paper_trail_diff/activity_timeline_builder.rbs +138 -30
  91. data/sig/generated/paper_trail_diff/activity_tree_filter.rbs +73 -0
  92. data/sig/generated/paper_trail_diff/activity_version_collector.rbs +2 -0
  93. data/sig/generated/paper_trail_diff/activity_view.rbs +42 -0
  94. data/sig/generated/paper_trail_diff/analysis.rbs +24 -5
  95. data/sig/generated/paper_trail_diff/analysis_batch.rbs +8 -15
  96. data/sig/generated/paper_trail_diff/approximate_paths.rbs +23 -0
  97. data/sig/generated/paper_trail_diff/association_discovery.rbs +5 -0
  98. data/sig/generated/paper_trail_diff/association_key.rbs +29 -0
  99. data/sig/generated/paper_trail_diff/association_traversal.rbs +4 -0
  100. data/sig/generated/paper_trail_diff/batch_preload.rbs +29 -0
  101. data/sig/generated/paper_trail_diff/batched_root_analyzer.rbs +13 -10
  102. data/sig/generated/paper_trail_diff/batched_root_versions.rbs +14 -4
  103. data/sig/generated/paper_trail_diff/branch_snapshot_refresher.rbs +6 -0
  104. data/sig/generated/paper_trail_diff/change_row.rbs +86 -0
  105. data/sig/generated/paper_trail_diff/collection_comparator.rbs +2 -16
  106. data/sig/generated/paper_trail_diff/configuration.rbs +19 -3
  107. data/sig/generated/paper_trail_diff/deleted_analysis_batch.rbs +11 -8
  108. data/sig/generated/paper_trail_diff/deleted_root_filter.rbs +56 -0
  109. data/sig/generated/paper_trail_diff/deprecation.rbs +35 -0
  110. data/sig/generated/paper_trail_diff/diagnostics.rbs +37 -30
  111. data/sig/generated/paper_trail_diff/endpoint.rbs +11 -2
  112. data/sig/generated/paper_trail_diff/errors.rbs +3 -1
  113. data/sig/generated/paper_trail_diff/fallback_paths.rbs +22 -0
  114. data/sig/generated/paper_trail_diff/historical_filter.rbs +12 -2
  115. data/sig/generated/paper_trail_diff/historical_snapshot_store.rbs +33 -28
  116. data/sig/generated/paper_trail_diff/live_endpoint_batch_loader.rbs +0 -15
  117. data/sig/generated/paper_trail_diff/nested_comparator.rbs +10 -7
  118. data/sig/generated/paper_trail_diff/paper_trail_adapter.rbs +64 -52
  119. data/sig/generated/paper_trail_diff/preloaded_endpoint_batch_loader.rbs +9 -1
  120. data/sig/generated/paper_trail_diff/prepared_activity_index.rbs +11 -4
  121. data/sig/generated/paper_trail_diff/prepared_history.rbs +10 -0
  122. data/sig/generated/paper_trail_diff/prepared_record_index.rbs +20 -2
  123. data/sig/generated/paper_trail_diff/read_snapshot.rbs +43 -0
  124. data/sig/generated/paper_trail_diff/root_version_selection.rbs +2 -0
  125. data/sig/generated/paper_trail_diff/scope_condition.rbs +81 -15
  126. data/sig/generated/paper_trail_diff/scope_ranking.rbs +14 -6
  127. data/sig/generated/paper_trail_diff/scoped_analysis.rbs +6 -0
  128. data/sig/generated/paper_trail_diff/scoped_analysis_builder.rbs +14 -2
  129. data/sig/generated/paper_trail_diff/scoped_root_selection.rbs +16 -7
  130. data/sig/generated/paper_trail_diff/snapshot.rbs +8 -0
  131. data/sig/generated/paper_trail_diff/support.rbs +15 -0
  132. data/sig/generated/paper_trail_diff/time_range.rbs +24 -9
  133. data/sig/generated/paper_trail_diff/timeline_builder.rbs +6 -2
  134. data/sig/generated/paper_trail_diff/timeline_range.rbs +2 -0
  135. data/sig/generated/paper_trail_diff/timeline_snapshot_provider.rbs +17 -7
  136. data/sig/generated/paper_trail_diff/transaction_cut.rbs +126 -0
  137. data/sig/generated/paper_trail_diff/traversal_preparer.rbs +66 -8
  138. data/sig/generated/paper_trail_diff/version_boundary.rbs +22 -3
  139. data/sig/generated/paper_trail_diff/version_query.rbs +18 -2
  140. data/sig/generated/paper_trail_diff/version_scope_filter.rbs +4 -5
  141. data/sig/generated/paper_trail_diff/version_sequence_diagnostics.rbs +19 -9
  142. data/sig/generated/paper_trail_diff/version_table.rbs +30 -0
  143. data/sig/generated/paper_trail_diff.rbs +29 -26
  144. metadata +49 -11
  145. data/lib/paper_trail_diff/activity_grouping.rb +0 -38
  146. data/lib/paper_trail_diff/batch_safety.rb +0 -63
  147. data/lib/paper_trail_diff/time_activity_timeline_builder.rb +0 -223
  148. data/sig/generated/paper_trail_diff/activity_grouping.rbs +0 -22
  149. data/sig/generated/paper_trail_diff/batch_safety.rbs +0 -30
  150. data/sig/generated/paper_trail_diff/time_activity_timeline_builder.rbs +0 -92
data/README.md CHANGED
@@ -122,13 +122,14 @@ and `item_id`. Its `recorded_at` is the version's `created_at`.
122
122
  Use this source information for attribution: the version used to read the
123
123
  resulting state, exposed as `to_boundary`, may have been saved by someone else.
124
124
  A change saved without a version, such as with `update_columns`, appears in the
125
- step for that record's previous recorded change. To read only what a version
126
- recorded, use `PaperTrailDiff.recorded_changes(version)`; see
127
- [what a version recorded](docs/reference.md#what-a-version-recorded).
125
+ step for that record's previous recorded change. To also read only what each
126
+ version recorded, pass `recorded_changes: true` and read
127
+ `step.source_boundary.recorded_changes`; see
128
+ [what each event recorded](docs/reference.md#what-each-event-recorded-recorded_changes-true).
128
129
 
129
130
  `group: :transaction` combines changes from one database transaction. Its source
130
- information comes from the first event in that group. Omit `group:` if you need
131
- to attribute each version separately.
131
+ information comes from the first event in that group, and `step.sources` lists
132
+ every event in it, each with its own author.
132
133
 
133
134
  `snapshots: true` also keeps copies of the compared records' data, including
134
135
  unchanged fields. These read-only copies are called snapshots. Access them as
@@ -159,7 +160,7 @@ rows = report.analyses.map do |identity, analysis|
159
160
  }
160
161
  end
161
162
 
162
- report.unreachable # Model names and IDs for records no longer in the database
163
+ report.unreachable # Model names and IDs of selected records no longer in the database
163
164
  report.deleted # Reports for those records, or nil when destroy history is missing
164
165
  ```
165
166
 
@@ -195,8 +196,11 @@ Three selection rules matter:
195
196
  `associations: [:comments]`, editing only a comment selects its article, and
196
197
  the article's report shows the edit.
197
198
  - A relation such as `Article.where(status: "open")` checks today's values.
198
- It cannot establish whether deleted articles matched that condition. Deleted
199
- records are therefore listed separately.
199
+ Deleted records have none, so they are listed separately. With
200
+ `include_deleted: true`, the gem tests each against the relation as it was
201
+ when destroyed and keeps only those that match, so a report on one account's
202
+ records holds only that account's deletions. That needs plain conditions on
203
+ the model's own columns; a join or SQL text raises `ConfigurationError`.
200
204
  - `limit: 100` is a safety ceiling, not pagination. Selecting too many records
201
205
  raises `BatchLimitExceededError`. With `include_deleted: true`, both existing
202
206
  and deleted records count toward it. Use a narrower relation or time window,
@@ -215,9 +219,9 @@ own.
215
219
 
216
220
  ## 4. Turn a report into rows
217
221
 
218
- For a history table or an export, flatten a report into one row per changed
219
- field. Each row names the record, the change, and who made it. This example
220
- requires the association setup in the quickstart:
222
+ For a history table or an export, `change_rows` flattens a report into one row
223
+ per change. Each row names the record, the change, and the version it came
224
+ from. This example requires the association setup in the quickstart:
221
225
 
222
226
  <!-- executable:recipe-summary -->
223
227
  ```ruby
@@ -225,48 +229,35 @@ report = PaperTrailDiff.report(
225
229
  Article.all, since: 1.day.ago, limit: 100,
226
230
  associations: [:comments], include_deleted: true
227
231
  )
228
- analyses = report.analyses.merge(report.deleted.compact)
229
-
230
- rows = analyses.flat_map do |root, analysis|
231
- analysis.activity_timeline.flat_map do |step|
232
- source = step.source_boundary
233
- step.diff.each_entry.filter_map do |entry|
234
- next unless entry.attribute # skip entries for whole records and associations
235
-
236
- from, to, change =
237
- if entry.change?
238
- [entry.value.from, entry.value.to, :changed]
239
- elsif entry.state == :after
240
- [nil, entry.value, :added]
241
- else
242
- [entry.value, nil, :removed]
243
- end
244
- next if from.nil? && to.nil? # a blank field of an added or removed record
245
-
246
- record = entry.record # nil for the article itself
247
-
248
- {
249
- root: root,
250
- association: entry.association_path.join("."), # "" for the article itself
251
- record: record ? [record.type, record.id.to_s] : root,
252
- change: change, attribute: entry.attribute, from: from, to: to,
253
- actor: source.whodunnit, event: source.event, recorded_at: source.recorded_at
254
- }
255
- end
256
- end
232
+
233
+ rows = report.change_rows.flat_map do |root, changes|
234
+ changes.map { |row| row.to_h.merge(root: root) }
257
235
  end
258
236
  ```
259
237
 
260
- `change` is `:changed` for an edited field. It is `:added` or `:removed` for
261
- each field of a record that appeared or disappeared: the article was created
262
- or deleted, or a comment joined or left it. A comment moved from another
263
- article is also `:added`; its `event` is then `"update"` rather than
264
- `"create"`. `actor`, `event`, and `recorded_at` describe the step's source
265
- version. With `group: :transaction`, that is the transaction's first event.
266
-
267
- Rows follow the report's record order, with deleted records last, then each
268
- record's steps in time order. Labels, grouping, and formatting are left to your
269
- application.
238
+ `report.change_rows` is a hash keyed like `report.analyses`, with deleted
239
+ records last. Each value lists that record's rows in time order. Each row
240
+ covers one of two things:
241
+
242
+ - **A field**: `attribute` names it, and `change` is `:changed` for an edit.
243
+ For a record that appeared or disappeared, each of its fields gets an
244
+ `:added` or `:removed` row, holding its value in `to` or `from`. Blank
245
+ fields of such a record get no row.
246
+ - **A whole record**: `attribute` is `nil`, and `change` is `:added` or
247
+ `:removed`. The record was created or deleted, a comment joined or left the
248
+ article, or a `belongs_to` pointed somewhere new, which gives a `:removed`
249
+ row for the old record and an `:added` row for the new one. A comment moved
250
+ from another article is `:added` too; its `event` is then `"update"`.
251
+
252
+ `record` is the record the row is about, or `nil` for the article itself, and
253
+ `association_path` is how it was reached, such as `["comments"]`. `to_h` adds
254
+ the step's source version: `version_id`, `event`, `whodunnit`, `recorded_at`,
255
+ and `transaction_id`. With `group: :transaction`, that is the transaction's
256
+ first event; `step.sources` lists every event. `row.source` is the
257
+ `ActivityBoundary` itself.
258
+
259
+ `change_rows` is also on an `Analysis` built with `activity: true` and on each
260
+ activity step. Labels, grouping, and formatting are left to your application.
270
261
 
271
262
  ## Choose a method
272
263
 
@@ -298,15 +289,27 @@ Scopes and default scopes made of plain conditions, such as
298
289
  `-> { where(archived: false) }`, are batched too, and so is a `limit` or
299
290
  `offset` ordered by integer, boolean, date, or datetime columns down to the
300
291
  primary key, such as `-> { order(:position, :id).limit(3) }`, on a direct
301
- association or a `has_many :through`. Some associations still need per-record
302
- queries, such as a `limit` ordered by a string or scopes that take
303
- the owner.
304
- Batch methods reject those by default with `UnsupportedBatchPathError`. The
305
- error names the association. Remove it from `associations:`, or pass
306
- `allow_expensive: true` if you accept that cost.
292
+ association or a `has_many :through`. History cannot read a few scopes, such
293
+ as a `limit` ordered by a string, a scope that takes the owner, or SQL text.
294
+ PT-AT reads those instead, one record at a time and only approximately, so
295
+ every method raises `UnsupportedAssociationError` naming the association.
296
+ Remove it from `associations:`, or pass `allow_approximate: true` to accept
297
+ PT-AT's answer and its cost. See
298
+ [costly associations](docs/reference.md#costly-associations).
307
299
  Callbacks and callable historical filters can also issue queries. See the
308
300
  [performance guide](docs/performance.md) for measured behavior and limitations.
309
301
 
302
+ On a large version table, add these indexes, or every `since:` or `within:`
303
+ report reads all of the model's versions. See
304
+ [database indexes](docs/architecture.md#database-indexes) for why.
305
+
306
+ ```ruby
307
+ add_index :versions, %i[item_type item_id created_at id],
308
+ name: "index_versions_on_item_and_timeline"
309
+ add_index :versions, %i[item_type created_at item_id],
310
+ name: "index_versions_on_type_and_time"
311
+ ```
312
+
310
313
  - `updated_at` is ignored. Passing `ignore:` replaces the default list.
311
314
  - Records passed as current state must be saved and have no unsaved changes.
312
315
  They are reloaded from the database.
data/docs/architecture.md CHANGED
@@ -66,17 +66,19 @@ PT-AT transaction are applied together.
66
66
 
67
67
  These association shapes cannot be read from the loaded history. The gem asks
68
68
  PT-AT to rebuild them at each point instead, which costs queries per record and
69
- resolves association membership by timestamp only:
69
+ resolves association membership by timestamp only. PT-AT starts from the
70
+ association's current members and swaps in their older versions, without
71
+ checking a scope against them, so its membership is approximate:
70
72
 
71
73
  - associations with a scope the gem cannot read as plain conditions and an
72
74
  exact ranking, including owner-dependent scopes and a `limit` without a
73
75
  total integer order;
74
- - targets with such a default scope;
75
- - targets that are not versioned;
76
+ - targets with such a default scope, including a polymorphic `belongs_to`
77
+ target, whose class and scope are known only from each record;
76
78
  - composite foreign keys.
77
79
 
78
- Batch methods reject the scoped cases unless you pass
79
- `allow_expensive: true`; see [costly associations](reference.md#costly-associations).
80
+ Every method that reads history rejects the scoped cases unless you pass
81
+ `allow_approximate: true`; see [costly associations](reference.md#costly-associations).
80
82
 
81
83
  HABTM membership comes from the snapshots PT-AT records with each transaction,
82
84
  so it is only known at root versions saved in a transaction. Those snapshots
@@ -88,18 +90,32 @@ HABTM of the same name saved a version holds both records' members;
88
90
 
89
91
  ## Database indexes
90
92
 
91
- For large histories, add a composite index that matches how the gem reads
92
- versions:
93
+ For large histories, add two indexes to each version table, one for each way
94
+ the gem reads versions:
93
95
 
94
96
  ```ruby
95
97
  add_index :versions, %i[item_type item_id created_at id],
96
98
  name: "index_versions_on_item_and_timeline"
99
+ add_index :versions, %i[item_type created_at item_id],
100
+ name: "index_versions_on_type_and_time"
97
101
  ```
98
102
 
99
- It helps most with repeated `within:` queries, which filter by `item_type`,
100
- `item_id`, and `created_at`, then order by time and ID. Keep PT-AT's own index
101
- on `version_associations` (`foreign_key_name`, `foreign_key_id`,
102
- `foreign_type`).
103
+ The first serves reading one record's history: it filters by `item_type` and
104
+ `item_id`, then by time, in time and ID order. It covers PaperTrail's own
105
+ `(item_type, item_id)` index, which you can then drop.
106
+
107
+ The second serves finding which records changed in a window, as `report`,
108
+ `analyze_scope`, and the other batch methods do with `since:` or `within:`.
109
+ They filter by `item_type` and `created_at` alone, which the first index cannot
110
+ narrow, so without the second the database reads every version of the model.
111
+ On PostgreSQL with 5 million article versions, a report on five articles
112
+ changed in the last hour took about 230 ms without it and 2 ms with it.
113
+ Ending the index with `item_id` lets the database answer from the index alone,
114
+ and SQLite, which keeps no table statistics unless you run `ANALYZE`, picks it
115
+ only then.
116
+
117
+ Keep PT-AT's own index on `version_associations` (`foreign_key_name`,
118
+ `foreign_key_id`, `foreign_type`).
103
119
 
104
120
  ## Remaining costs
105
121
 
@@ -109,3 +125,8 @@ on `version_associations` (`foreign_key_name`, `foreign_key_id`,
109
125
  - Repeated events in one very wide collection still copy that collection's
110
126
  list of records for each step. The comparison work stays small, but memory
111
127
  traffic grows with *events × collection size*.
128
+ - With PT-AT installed, a `within:` or `since:` window looks up each root's
129
+ versions on either side of both edges, to move an edge that falls inside a
130
+ transaction (see [time windows](reference.md#time-windows)). These are
131
+ four indexed lookups per root, sent 200 roots per query, whatever the
132
+ length of the root's history: about 0.2 ms per root on SQLite.