paper_trail_diff 0.14.0 → 0.16.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (150) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +501 -0
  3. data/QUICKSTART.md +21 -4
  4. data/README.md +59 -56
  5. data/docs/architecture.md +32 -11
  6. data/docs/performance.md +125 -391
  7. data/docs/reference.md +470 -117
  8. data/docs/testing.md +83 -7
  9. data/lib/paper_trail_diff/activity_boundary.rb +83 -26
  10. data/lib/paper_trail_diff/activity_child_candidate_loader.rb +2 -1
  11. data/lib/paper_trail_diff/activity_collection_event_applier.rb +33 -1
  12. data/lib/paper_trail_diff/activity_event.rb +41 -10
  13. data/lib/paper_trail_diff/activity_event_route_finder.rb +31 -0
  14. data/lib/paper_trail_diff/activity_event_snapshot_refresher.rb +12 -1
  15. data/lib/paper_trail_diff/activity_history.rb +34 -9
  16. data/lib/paper_trail_diff/activity_range.rb +18 -3
  17. data/lib/paper_trail_diff/activity_root_snapshot_refresher.rb +43 -25
  18. data/lib/paper_trail_diff/activity_snapshot_sequence.rb +76 -37
  19. data/lib/paper_trail_diff/activity_target_candidate_loader.rb +5 -5
  20. data/lib/paper_trail_diff/activity_timeline_builder.rb +272 -111
  21. data/lib/paper_trail_diff/activity_transaction_grouper.rb +2 -1
  22. data/lib/paper_trail_diff/activity_tree_filter.rb +114 -0
  23. data/lib/paper_trail_diff/activity_version_collector.rb +7 -5
  24. data/lib/paper_trail_diff/activity_view.rb +69 -0
  25. data/lib/paper_trail_diff/analysis.rb +37 -6
  26. data/lib/paper_trail_diff/analysis_batch.rb +25 -31
  27. data/lib/paper_trail_diff/approximate_paths.rb +48 -0
  28. data/lib/paper_trail_diff/association_activity_edge.rb +6 -6
  29. data/lib/paper_trail_diff/association_discovery.rb +9 -1
  30. data/lib/paper_trail_diff/association_key.rb +56 -0
  31. data/lib/paper_trail_diff/association_traversal.rb +13 -4
  32. data/lib/paper_trail_diff/batch_boundary_resolver.rb +3 -4
  33. data/lib/paper_trail_diff/batch_preload.rb +53 -0
  34. data/lib/paper_trail_diff/batched_root_analyzer.rb +18 -14
  35. data/lib/paper_trail_diff/batched_root_versions.rb +38 -14
  36. data/lib/paper_trail_diff/branch_snapshot_refresher.rb +19 -16
  37. data/lib/paper_trail_diff/change_row.rb +133 -0
  38. data/lib/paper_trail_diff/collection_comparator.rb +5 -32
  39. data/lib/paper_trail_diff/configuration.rb +35 -12
  40. data/lib/paper_trail_diff/deleted_analysis_batch.rb +23 -19
  41. data/lib/paper_trail_diff/deleted_root_filter.rb +101 -0
  42. data/lib/paper_trail_diff/deprecation.rb +61 -0
  43. data/lib/paper_trail_diff/diagnostics.rb +86 -99
  44. data/lib/paper_trail_diff/endpoint.rb +26 -3
  45. data/lib/paper_trail_diff/errors.rb +3 -1
  46. data/lib/paper_trail_diff/fallback_paths.rb +38 -0
  47. data/lib/paper_trail_diff/historical_association_reifier.rb +2 -4
  48. data/lib/paper_trail_diff/historical_filter.rb +19 -10
  49. data/lib/paper_trail_diff/historical_snapshot_store.rb +57 -67
  50. data/lib/paper_trail_diff/live_endpoint_batch_loader.rb +1 -35
  51. data/lib/paper_trail_diff/nested_comparator.rb +19 -18
  52. data/lib/paper_trail_diff/paper_trail_adapter.rb +198 -166
  53. data/lib/paper_trail_diff/preloaded_endpoint_batch_loader.rb +13 -1
  54. data/lib/paper_trail_diff/prepared_activity_index.rb +17 -13
  55. data/lib/paper_trail_diff/prepared_edge_loader.rb +2 -1
  56. data/lib/paper_trail_diff/prepared_history.rb +15 -1
  57. data/lib/paper_trail_diff/prepared_history_loader.rb +2 -3
  58. data/lib/paper_trail_diff/prepared_record_index.rb +42 -7
  59. data/lib/paper_trail_diff/read_snapshot.rb +77 -0
  60. data/lib/paper_trail_diff/root_version_selection.rb +4 -2
  61. data/lib/paper_trail_diff/scope_condition.rb +157 -32
  62. data/lib/paper_trail_diff/scope_ranking.rb +23 -14
  63. data/lib/paper_trail_diff/scoped_analysis.rb +11 -0
  64. data/lib/paper_trail_diff/scoped_analysis_builder.rb +29 -11
  65. data/lib/paper_trail_diff/scoped_root_selection.rb +29 -16
  66. data/lib/paper_trail_diff/snapshot.rb +16 -1
  67. data/lib/paper_trail_diff/support.rb +29 -2
  68. data/lib/paper_trail_diff/time_range.rb +39 -16
  69. data/lib/paper_trail_diff/timeline_builder.rb +13 -7
  70. data/lib/paper_trail_diff/timeline_range.rb +9 -1
  71. data/lib/paper_trail_diff/timeline_snapshot_provider.rb +24 -17
  72. data/lib/paper_trail_diff/transaction_cut.rb +240 -0
  73. data/lib/paper_trail_diff/traversal_preparer.rb +134 -29
  74. data/lib/paper_trail_diff/version.rb +1 -1
  75. data/lib/paper_trail_diff/version_boundary.rb +41 -10
  76. data/lib/paper_trail_diff/version_query.rb +34 -10
  77. data/lib/paper_trail_diff/version_scope_filter.rb +9 -10
  78. data/lib/paper_trail_diff/version_sequence_diagnostics.rb +35 -17
  79. data/lib/paper_trail_diff/version_table.rb +60 -0
  80. data/lib/paper_trail_diff.rb +101 -57
  81. data/sig/generated/paper_trail_diff/activity_boundary.rbs +54 -19
  82. data/sig/generated/paper_trail_diff/activity_collection_event_applier.rbs +13 -0
  83. data/sig/generated/paper_trail_diff/activity_event.rbs +25 -4
  84. data/sig/generated/paper_trail_diff/activity_event_route_finder.rbs +14 -0
  85. data/sig/generated/paper_trail_diff/activity_event_snapshot_refresher.rbs +6 -0
  86. data/sig/generated/paper_trail_diff/activity_history.rbs +21 -6
  87. data/sig/generated/paper_trail_diff/activity_range.rbs +9 -2
  88. data/sig/generated/paper_trail_diff/activity_root_snapshot_refresher.rbs +21 -10
  89. data/sig/generated/paper_trail_diff/activity_snapshot_sequence.rbs +48 -9
  90. data/sig/generated/paper_trail_diff/activity_timeline_builder.rbs +138 -30
  91. data/sig/generated/paper_trail_diff/activity_tree_filter.rbs +73 -0
  92. data/sig/generated/paper_trail_diff/activity_version_collector.rbs +2 -0
  93. data/sig/generated/paper_trail_diff/activity_view.rbs +42 -0
  94. data/sig/generated/paper_trail_diff/analysis.rbs +24 -5
  95. data/sig/generated/paper_trail_diff/analysis_batch.rbs +8 -15
  96. data/sig/generated/paper_trail_diff/approximate_paths.rbs +23 -0
  97. data/sig/generated/paper_trail_diff/association_discovery.rbs +5 -0
  98. data/sig/generated/paper_trail_diff/association_key.rbs +29 -0
  99. data/sig/generated/paper_trail_diff/association_traversal.rbs +4 -0
  100. data/sig/generated/paper_trail_diff/batch_preload.rbs +29 -0
  101. data/sig/generated/paper_trail_diff/batched_root_analyzer.rbs +13 -10
  102. data/sig/generated/paper_trail_diff/batched_root_versions.rbs +14 -4
  103. data/sig/generated/paper_trail_diff/branch_snapshot_refresher.rbs +6 -0
  104. data/sig/generated/paper_trail_diff/change_row.rbs +86 -0
  105. data/sig/generated/paper_trail_diff/collection_comparator.rbs +2 -16
  106. data/sig/generated/paper_trail_diff/configuration.rbs +19 -3
  107. data/sig/generated/paper_trail_diff/deleted_analysis_batch.rbs +11 -8
  108. data/sig/generated/paper_trail_diff/deleted_root_filter.rbs +56 -0
  109. data/sig/generated/paper_trail_diff/deprecation.rbs +35 -0
  110. data/sig/generated/paper_trail_diff/diagnostics.rbs +37 -30
  111. data/sig/generated/paper_trail_diff/endpoint.rbs +11 -2
  112. data/sig/generated/paper_trail_diff/errors.rbs +3 -1
  113. data/sig/generated/paper_trail_diff/fallback_paths.rbs +22 -0
  114. data/sig/generated/paper_trail_diff/historical_filter.rbs +12 -2
  115. data/sig/generated/paper_trail_diff/historical_snapshot_store.rbs +33 -28
  116. data/sig/generated/paper_trail_diff/live_endpoint_batch_loader.rbs +0 -15
  117. data/sig/generated/paper_trail_diff/nested_comparator.rbs +10 -7
  118. data/sig/generated/paper_trail_diff/paper_trail_adapter.rbs +64 -52
  119. data/sig/generated/paper_trail_diff/preloaded_endpoint_batch_loader.rbs +9 -1
  120. data/sig/generated/paper_trail_diff/prepared_activity_index.rbs +11 -4
  121. data/sig/generated/paper_trail_diff/prepared_history.rbs +10 -0
  122. data/sig/generated/paper_trail_diff/prepared_record_index.rbs +20 -2
  123. data/sig/generated/paper_trail_diff/read_snapshot.rbs +43 -0
  124. data/sig/generated/paper_trail_diff/root_version_selection.rbs +2 -0
  125. data/sig/generated/paper_trail_diff/scope_condition.rbs +81 -15
  126. data/sig/generated/paper_trail_diff/scope_ranking.rbs +14 -6
  127. data/sig/generated/paper_trail_diff/scoped_analysis.rbs +6 -0
  128. data/sig/generated/paper_trail_diff/scoped_analysis_builder.rbs +14 -2
  129. data/sig/generated/paper_trail_diff/scoped_root_selection.rbs +16 -7
  130. data/sig/generated/paper_trail_diff/snapshot.rbs +8 -0
  131. data/sig/generated/paper_trail_diff/support.rbs +15 -0
  132. data/sig/generated/paper_trail_diff/time_range.rbs +24 -9
  133. data/sig/generated/paper_trail_diff/timeline_builder.rbs +6 -2
  134. data/sig/generated/paper_trail_diff/timeline_range.rbs +2 -0
  135. data/sig/generated/paper_trail_diff/timeline_snapshot_provider.rbs +17 -7
  136. data/sig/generated/paper_trail_diff/transaction_cut.rbs +126 -0
  137. data/sig/generated/paper_trail_diff/traversal_preparer.rbs +66 -8
  138. data/sig/generated/paper_trail_diff/version_boundary.rbs +22 -3
  139. data/sig/generated/paper_trail_diff/version_query.rbs +18 -2
  140. data/sig/generated/paper_trail_diff/version_scope_filter.rbs +4 -5
  141. data/sig/generated/paper_trail_diff/version_sequence_diagnostics.rbs +19 -9
  142. data/sig/generated/paper_trail_diff/version_table.rbs +30 -0
  143. data/sig/generated/paper_trail_diff.rbs +29 -26
  144. metadata +49 -11
  145. data/lib/paper_trail_diff/activity_grouping.rb +0 -38
  146. data/lib/paper_trail_diff/batch_safety.rb +0 -63
  147. data/lib/paper_trail_diff/time_activity_timeline_builder.rb +0 -223
  148. data/sig/generated/paper_trail_diff/activity_grouping.rbs +0 -22
  149. data/sig/generated/paper_trail_diff/batch_safety.rbs +0 -30
  150. data/sig/generated/paper_trail_diff/time_activity_timeline_builder.rbs +0 -92
data/docs/reference.md CHANGED
@@ -61,7 +61,7 @@ Take an article created as `"Draft"`, then updated to `"Published"`, then to
61
61
  only in the table
62
62
  ```
63
63
 
64
- Four things follow from this:
64
+ Six things follow from this:
65
65
 
66
66
  - **A `create` version stands for "no record yet".** Its `reify` returns `nil`.
67
67
  Comparing it with a later state reports the record as added
@@ -76,9 +76,24 @@ Four things follow from this:
76
76
  changes the values the record's next version stores, or its current row. The
77
77
  gem reads the result of a change from there, so the write appears in the step
78
78
  for that record's previous recorded change and is attributed to its author.
79
- Ending a range at a version leaves out only the writes after it.
79
+ Ending a range at a version leaves out only the writes after it. A column
80
+ that `has_paper_trail only:` or `ignore:` leaves out of `object_changes` is
81
+ read the same way, so a key saved with a version is that version's change.
82
+ Rails updates `counter_cache` columns like this, so a parent's count changes
83
+ in the step of the parent's previous change; pass the column in `ignore:` to
84
+ leave it out.
80
85
  [`recorded_changes`](#what-a-version-recorded) reads what a version itself
81
- recorded.
86
+ recorded, and activity steps can [include it](#what-each-event-recorded-recorded_changes-true).
87
+ - **A record created without a version looks as if it always existed.** A
88
+ row written with versioning off, or imported with `insert_all`, has no
89
+ `create` version, so history never reports it as added: its earliest
90
+ version's stored values are its state from before history began, and an
91
+ association includes it there. Rows from before PaperTrail was installed
92
+ are the same, which is usually what you want.
93
+ - **Pruned history is gone.** With `has_paper_trail limit:`, PaperTrail
94
+ deletes a record's oldest versions, and the earliest one left becomes its
95
+ start: states before it cannot be read, and an older version passed in is
96
+ no longer there to pass.
82
97
 
83
98
  ## Choosing a method
84
99
 
@@ -126,8 +141,15 @@ PaperTrailDiff.timeline(article, within: 1.week.ago..Time.current)
126
141
 
127
142
  - `from:` defaults to `:first`, the record's earliest version.
128
143
  - `to:` defaults to the record itself, meaning its current state. Pass a
129
- version instead for a result that never changes when the record is edited
130
- again.
144
+ version instead to anchor the result there. It does not change when the
145
+ record is edited again, unless a transaction that stamped changes before that
146
+ version commits after it; see [version order](#version-order).
147
+ - A version stands for the state before its transaction, since PT-AT rebuilds
148
+ associations as they were before it. Child changes saved in the same
149
+ transaction, such as the update behind a `touch: true` parent version, fall
150
+ after a range that ends there and inside a range that starts there. So a
151
+ history split at that version reports each such change exactly once, in the
152
+ later part.
131
153
  - Versions must belong to the record and be given in time order.
132
154
  - A record with no versions has an empty history, not an error.
133
155
 
@@ -145,10 +167,61 @@ window = Time.zone.parse("2026-08-01")...Time.zone.parse("2026-09-01")
145
167
  steps = PaperTrailDiff.timeline(article, within: window)
146
168
  ```
147
169
 
148
- To show what the window's last change produced, the gem reads the next version
149
- after the window. If there is none, nothing has been recorded since, so it reads
150
- the current record instead. The step's `to_boundary.kind` is then `:current`.
151
- This works the same for a window that ends now and one that ended long ago.
170
+ A `Date` bound is a whole day in `Time.zone`, so `Date.today..Date.today` is
171
+ all of today and `august...september` stops before September 1. A string is a
172
+ time in `Time.zone` unless it names an offset. Without `Time.zone`, both are
173
+ read in UTC, never in the server's own zone.
174
+
175
+ A window edge that falls inside a transaction moves back to where that
176
+ transaction began, so the whole transaction is in the later window, as it is
177
+ when a range [starts or ends at a version](#choosing-a-range). This applies to
178
+ transactions that saved a version of the root record, such as a comment edit
179
+ and the `touch: true` article version saved with it. So back-to-back windows,
180
+ or a `report` with `since:` after an earlier report, show each change exactly
181
+ once, credited to the transaction that made it. Each root moves its own edges,
182
+ so in a batch two roots can start a few moments apart.
183
+
184
+ `analysis.window` is the window an `Analysis` covers after any such move, as a
185
+ Ruby range. It is `nil` when `within:` or `since:` was not given. Every step of
186
+ the analysis is inside it, so to check a step against the window, use
187
+ `analysis.window` rather than the range you passed: a step at the start of a
188
+ moved window begins before the time you asked for.
189
+
190
+ ```ruby
191
+ analysis = PaperTrailDiff.analyze_many(articles, within: window, activity: true).values.first
192
+ analysis.window # => window, or one that starts earlier
193
+ ```
194
+
195
+ Each move emits a `moved_window.paper_trail_diff` notification whose payload
196
+ has `item_type`, `item_id`, the `requested` range, and the moved `window`. In
197
+ history saved by your application, an edge moves back by the time one
198
+ transaction took, usually well under a second. A move of minutes or days
199
+ means a transaction ran that long, or that the versions' times were changed
200
+ after they were saved, as a test might do; see
201
+ [writing history in tests](#writing-history-in-tests).
202
+
203
+ A closed window can still change. PaperTrail stamps a version's `created_at`
204
+ when it builds the version, inside its transaction, and the transaction commits
205
+ later. A transaction that stamped a change inside the window and committed
206
+ after the window was read adds that change to it. Wait longer than your longest
207
+ transaction after the window ends before you rely on it, so that every
208
+ transaction that could have written in the window has committed. A report
209
+ through now is final only for changes older than that delay. The gem cannot see
210
+ open transactions, so it cannot tell when a window has settled.
211
+
212
+ To show what the window's last change produced, the gem reads the state just
213
+ before the next change after the window:
214
+
215
+ - With `activity_timeline`, or `analyze` with `activity: true`, that is the next
216
+ version of the record or of any record on a requested association path. If
217
+ none was recorded after the window, it reads the current record instead,
218
+ and the last step's `to_boundary.kind` is `:current`. So a window that ended
219
+ long ago ends at its own end, even when only a comment changed since.
220
+ - `timeline`, and `analyze` without `activity:`, compare the record's own
221
+ versions. The last step runs to the record's next version, or to the
222
+ current record when it has none, and an association change made in between
223
+ is part of that step, credited to its author. Use `activity: true` to keep
224
+ a past window's associations as they were at its end.
152
225
 
153
226
  Reading the current record has one consequence: it includes changes PaperTrail
154
227
  never recorded, such as an `update_columns` call. Like any
@@ -158,8 +231,10 @@ in the step for that record's last recorded change.
158
231
  If the window's last change destroyed the record, the window closes on that
159
232
  deletion instead. See [a destroyed record](#a-destroyed-record).
160
233
 
161
- `close_on: :current` asks for the behavior above explicitly. It is the default
162
- now and is still accepted.
234
+ `close_on: :current` is deprecated and has no effect; passing it warns through
235
+ `PaperTrailDiff.deprecator`. Remove it. In a Rails app, register the deprecator
236
+ with `Rails.application.deprecators[:paper_trail_diff] = PaperTrailDiff.deprecator`
237
+ to handle its warnings as the app's own.
163
238
 
164
239
  `timeline` returns no steps for a window with no versions of the root record.
165
240
  `activity_timeline` still reports chosen associations that changed in such a
@@ -177,7 +252,7 @@ PaperTrailDiff.timeline(article, within: window, version_scope: user_edits)
177
252
  ```
178
253
 
179
254
  It is accepted by `timeline`, `activity_timeline`, `analyze`, `analyze_many`,
180
- and `analyze_scope`.
255
+ `analyze_scope`, and `report`.
181
256
 
182
257
  - Each reported change is compared with the version right after it, even if
183
258
  that version is filtered out. So each step shows exactly what that one change
@@ -191,6 +266,11 @@ and `analyze_scope`.
191
266
  - It looks at the root record's versions only. Someone who only edited comments
192
267
  has no root versions to select. To find one person's changes, including to
193
268
  associations, see [one person's changes](#one-persons-changes).
269
+ - In `analyze_scope` and `report`, it also chooses which records are selected:
270
+ a record's own versions select it only when the filter keeps one of them in
271
+ the window. A record whose versions there it all drops would have nothing to
272
+ report, so it is left out. A change on a requested association still selects
273
+ a record, whoever made it.
194
274
  - `activity_timeline` uses the filter to decide where the range starts and
195
275
  ends, but still lists every step inside it. Dropping one would move its
196
276
  change into a neighboring step and credit the wrong person.
@@ -233,8 +313,9 @@ diff.to_h
233
313
  deleted.
234
314
  - A current record must be saved, not destroyed, and have no unsaved changes.
235
315
  The gem reloads it, so in-memory edits and cached associations are ignored.
236
- - Options: `associations:`, `ignore:`, and `reload_live_endpoints:` (see
237
- [reusing loaded records](#reusing-loaded-records)).
316
+ - Options: `associations:`, `ignore:`, and `allow_approximate:` (see
317
+ [costly associations](#costly-associations)). `reload_live_endpoints:` is
318
+ deprecated (see [reusing loaded records](#reusing-loaded-records)).
238
319
 
239
320
  ## compare_many
240
321
 
@@ -258,31 +339,26 @@ diffs.fetch(["Order", orders.first.id.to_s]) # => PaperTrailDiff::Diff
258
339
  record, so `{ from: :first, to: :first }` raises.
259
340
  - A record with no versions gets an empty `Diff`.
260
341
  - Each record can appear in only one pair, because results are keyed by record.
261
- - Options: `associations:`, `ignore:`, `reload_live_endpoints:`, and
262
- `allow_expensive:` (see [costly associations](#costly-associations)).
342
+ - Options: `associations:`, `ignore:`, and `allow_approximate:` (see
343
+ [costly associations](#costly-associations)).
263
344
 
264
345
  If you look up versions yourself, do it in bulk. Looking up one record at a
265
346
  time brings back the per-record queries this method avoids.
266
347
 
267
348
  ### Reusing loaded records
268
349
 
269
- `compare`, `compare_many`, and `activity_timeline` reload current records by
270
- default. If you already loaded them with every requested association, you can
271
- skip the reload:
272
-
273
- ```ruby
274
- orders = Order.where(id: order_ids).preload(line_items: :product).to_a
275
-
276
- diffs = PaperTrailDiff.compare_many(
277
- orders.map { |order| { from: :first, to: order } },
278
- associations: ["line_items.product"],
279
- reload_live_endpoints: false
280
- )
281
- ```
282
-
283
- The values then come from your objects, so keeping them fresh is up to you. A
284
- requested association that is not loaded raises `UnloadedAssociationError`
285
- instead of quietly running a query per record.
350
+ Every method reloads the current records it compares with, together with the
351
+ requested associations, in a query for the records and one per association
352
+ level. `reload_live_endpoints: false` on `compare`, `compare_many`, and
353
+ `activity_timeline` skips that reload and reads your loaded objects instead.
354
+ It is deprecated and will be removed: it saves those few queries per call (2
355
+ for 500 articles with their comments), and the values then come from objects
356
+ that may be stale. Until it is removed it still works,
357
+ with a warning. A requested association that is not loaded raises
358
+ `UnloadedAssociationError`, and one with a `limit` or `offset`, directly or in
359
+ a part it goes through, or a scope that takes the owner, is read again for
360
+ each record, because Active Record's `preload` applies a limit to all the
361
+ records it loads together.
286
362
 
287
363
  ## timeline
288
364
 
@@ -314,7 +390,7 @@ steps.reject(&:empty?) # hide steps with no reported changes
314
390
  `item_type`, `item_id`, and `recorded_at`. `to_h(metadata: true)` adds author,
315
391
  event, transaction ID, and record to each boundary.
316
392
  - Options: `from:`, `to:`, `within:`, `associations:`, `ignore:`,
317
- `version_scope:`, `close_on:`.
393
+ `version_scope:`, `allow_approximate:`.
318
394
 
319
395
  ## activity_timeline
320
396
 
@@ -340,14 +416,21 @@ end
340
416
  `:destroyed` (after a deletion). They also have `event`, `whodunnit`,
341
417
  `recorded_at`, `item_type`, `item_id`, `transaction_id`, and `record`, and
342
418
  the predicates `version?`, `current?`, and `destroyed?`.
419
+ - `step.sources` lists the events whose changes the step shows, oldest first.
420
+ That is just `source_boundary`, unless the step is a
421
+ [transaction group](#one-step-per-transaction-group-transaction).
422
+ - A record's versions are steps only while it is in the tree. A comment edited
423
+ on another article and moved to this one later shows up when it moves, not
424
+ when it was edited there.
343
425
  - `step.to_h` returns `{ from:, to:, diff: }`. `to_h(metadata: true)` adds
344
- author, event, transaction ID, and record. `to_h(snapshots: true)` adds the
345
- retained snapshots.
426
+ author, event, transaction ID, and record, and a `sources` list.
427
+ `to_h(snapshots: true)` adds the retained snapshots.
346
428
  - A child's change becomes visible at the next recorded event, or at the
347
429
  current record. You do not need to save the article after each comment change.
348
430
  - Options: `from:`, `to:`, `within:`, `associations:`, `ignore:`,
349
- `version_scope:`, `close_on:`, `reload_live_endpoints:`, `snapshots:`,
350
- `group:`.
431
+ `version_scope:`, `snapshots:`, `group:`, `recorded_changes:`,
432
+ `allow_approximate:`. `reload_live_endpoints:` is deprecated (see
433
+ [reusing loaded records](#reusing-loaded-records)).
351
434
 
352
435
  ### One person's changes
353
436
 
@@ -390,15 +473,23 @@ PaperTrailDiff.activity_timeline(article, associations: [:comments], group: :tra
390
473
  - A combined step compares the state before the transaction with the state after
391
474
  it. A field set and then restored inside the transaction is not reported.
392
475
  - Its `source_boundary` is the first event in the transaction. That event's
393
- author is not necessarily the author of every grouped change.
476
+ author is not necessarily the author of every grouped change. `step.sources`
477
+ lists every event in the transaction, each with its own author, except
478
+ those of records that were outside the tree before and after it.
394
479
  - Versions without a `transaction_id` are never grouped.
480
+ - A savepoint that rolls back can split one transaction into more than one
481
+ group. PT-AT clears the transaction id when the savepoint rolls back, so the
482
+ versions the transaction saves afterwards form a separate group. Releasing
483
+ the savepoint keeps one group.
395
484
  - Also accepted by `analyze`, `analyze_many`, and `analyze_scope` with
396
485
  `activity: true`, and by `report`.
397
486
 
398
487
  Without grouping, a step inside a transaction can look empty. A child's new
399
488
  value is only visible at the next recorded event, and inside a transaction that
400
489
  event is often another part of the same save. Nothing is lost; the change
401
- appears one step later.
490
+ appears one step later. When the range ends at the version that closes that
491
+ save, there is no later step, so the change is
492
+ [outside the range](#choosing-a-range).
402
493
 
403
494
  ### Keeping snapshots: `snapshots: true`
404
495
 
@@ -419,6 +510,49 @@ step.from_snapshot.associations["comments"].records.first.attributes["author"]
419
510
  - Also accepted by `analyze`, `analyze_many`, and `analyze_scope` with
420
511
  `activity: true`, and by `report`.
421
512
 
513
+ ### What each event recorded: `recorded_changes: true`
514
+
515
+ A step's `diff` compares the state before and after an event. To also see what
516
+ PaperTrail [stored with each version](#what-a-version-recorded), such as for an
517
+ audit log, pass `recorded_changes: true`:
518
+
519
+ <!-- executable:reference-step-sources -->
520
+ ```ruby
521
+ steps = PaperTrailDiff.activity_timeline(
522
+ article, associations: [:comments], group: :transaction, recorded_changes: true
523
+ )
524
+
525
+ steps.reject(&:empty?).flat_map(&:sources).map do |source|
526
+ {
527
+ version_id: source.version_id,
528
+ record: source.record.to_h, # { type: "Comment", id: 7 }
529
+ event: source.event,
530
+ actor: source.whodunnit,
531
+ recorded_at: source.recorded_at,
532
+ transaction_id: source.transaction_id,
533
+ changes: source.recorded_changes # { "body" => <from "Old" to "New"> }
534
+ }
535
+ end
536
+ ```
537
+
538
+ - Each boundary's `recorded_changes` is what
539
+ `PaperTrailDiff.recorded_changes(version)` returns for its version: `nil`
540
+ when nothing was recorded, and always `nil` for the current record. The
541
+ versions are already loaded, so this adds no queries, but decoding each
542
+ version's changes takes time: about a third more for a `report` of 200
543
+ articles with comments, with history stored as YAML. Pass it only when you
544
+ use them.
545
+ - Without the option, `recorded_changes` raises `ConfigurationError` instead of
546
+ returning `nil`, so a missing option cannot look like a version that recorded
547
+ nothing.
548
+ - `to_h(metadata: true)` includes them on each boundary and source.
549
+ - Names in `ignore:`, or in `ignore: { all: }`, are left out. So are names
550
+ ignored on the root path, `"$"`, for the root's own versions. Names ignored
551
+ on an association path are not left out, because a child's version does not
552
+ say which path reached it.
553
+ - Also accepted by `analyze`, `analyze_many`, and `analyze_scope` with
554
+ `activity: true`, and by `report`.
555
+
422
556
  ### A destroyed record
423
557
 
424
558
  When the range ends at the record's `destroy` version, the activity timeline
@@ -452,6 +586,27 @@ and members were recorded there, reading the root's membership at that version
452
586
  raises `IncompleteAssociationHistoryError` instead of mixing both records'
453
587
  members. Save each HABTM owner's versions in a transaction of its own.
454
588
 
589
+ PT-AT records membership only when the owner saves a version, so the gem knows
590
+ it only at root versions. A change made between two root versions is reported
591
+ in the first step after the earlier one, and credited to that step's source:
592
+ usually the earlier root version, which someone else may have saved. A member
593
+ created between them is reported in the step that starts with its creation
594
+ instead. A change attached before a checkpoint is therefore credited to
595
+ whoever saved the version before the checkpoint, not to the checkpoint's
596
+ author. To credit a change to the person who made it, save a checkpoint just
597
+ before the change as well as after it: the step between the two checkpoints is
598
+ theirs.
599
+
600
+ PT-AT works out the membership a version records from the instance that saves
601
+ it: the instance's current members, with the changes that instance made undone,
602
+ so that the version holds the membership from before its own event. It never
603
+ forgets those changes, not even on `reload`. Only the first version an instance
604
+ saves after changing a HABTM is right; each later save from it records the
605
+ membership from before that change again. The gem then reports the change late,
606
+ or reports a member that never left as removed and added again. Load the record
607
+ again, with `find`, before saving it a second time, and save checkpoints from a
608
+ freshly loaded record, as the [checkpoint recipe](#saving-a-checkpoint) does.
609
+
455
610
  ## analyze
456
611
 
457
612
  `analyze(record)` returns an `Analysis` with the net `diff` and the `timeline`
@@ -465,14 +620,34 @@ analysis.timeline # the same steps timeline returns
465
620
  analysis.activity_timeline # nil unless activity: true
466
621
  analysis.from_snapshot # the full state at the start (nil before creation)
467
622
  analysis.to_snapshot # the full state at the end
623
+ analysis.window # the time window it covers, or nil without within:
468
624
  analysis.to_h(snapshots: true, metadata: true)
469
625
  ```
470
626
 
471
627
  - It accepts the same ranges as `timeline`.
472
- - `activity: true` also builds the activity timeline. `snapshots:` and `group:`
473
- apply to the activity timeline and require `activity: true`.
628
+ - `from_snapshot` is the record as it was when the range began: before the
629
+ first change in a window, or before the change of the version you passed as
630
+ `from:`. With `version_scope:`, a window begins at the first change the
631
+ filter keeps. It is `nil` when the record was created inside the range; the
632
+ state right after it was created is then the first activity step's
633
+ `to_snapshot`, which `snapshots: true` keeps.
634
+ - To read the record at one version, with its values typed and its
635
+ associations, take `from_snapshot` from a range that starts there:
636
+ `PaperTrailDiff.analyze(article, from: version, to: article).from_snapshot`.
637
+ Don't decode `version.object` yourself; the gem casts what PaperTrail
638
+ stores, which a `json` column or an enum leaves in another form.
639
+ - When nothing in the range is reported, the analysis is empty: an empty
640
+ `diff` and timelines, and `nil` for `from_snapshot` and `to_snapshot`. Here
641
+ `nil` means no state was read, not that the record did not exist; reading
642
+ one would rebuild the record's whole graph, a cost per record in a batch.
643
+ Use `compare` with a version and the record to see a state that did not
644
+ change.
645
+ - `activity: true` also builds the activity timeline. `snapshots:`, `group:`,
646
+ and `recorded_changes:` apply to the activity timeline and require
647
+ `activity: true`.
474
648
  - Options: `from:`, `to:`, `within:`, `associations:`, `ignore:`, `activity:`,
475
- `version_scope:`, `close_on:`, `snapshots:`, `group:`.
649
+ `version_scope:`, `snapshots:`, `group:`, `recorded_changes:`,
650
+ `allow_approximate:`.
476
651
 
477
652
  ## Reports for many records
478
653
 
@@ -496,10 +671,11 @@ results.fetch(["Order", order.id.to_s]).diff
496
671
 
497
672
  - Without `within:`, each record's whole history through its current state is
498
673
  analyzed.
499
- - A record with nothing to report gets an empty `Analysis`.
674
+ - A record with nothing to report gets an empty `Analysis`, whose snapshots
675
+ are `nil` (see [analyze](#analyze)).
500
676
  - Records must be saved and unique.
501
677
  - Options: `within:`, `associations:`, `ignore:`, `activity:`, `version_scope:`,
502
- `close_on:`, `snapshots:`, `group:`, `allow_expensive:`.
678
+ `snapshots:`, `group:`, `recorded_changes:`, `allow_approximate:`.
503
679
 
504
680
  ### analyze_scope
505
681
 
@@ -536,9 +712,9 @@ recently":
536
712
  report = PaperTrailDiff.report(Article.all, since: 1.day.ago, limit: 100)
537
713
  ```
538
714
 
539
- It accepts `associations:`, `ignore:`, `snapshots:`, `group:`,
540
- `include_deleted:`, `historical_filter:`, `max_filter_versions:`, and
541
- `allow_expensive:`.
715
+ It accepts `associations:`, `ignore:`, `version_scope:`, `snapshots:`, `group:`,
716
+ `recorded_changes:`, `include_deleted:`, `historical_filter:`, `max_filter_versions:`,
717
+ and `allow_approximate:`.
542
718
 
543
719
  ### Which records are selected
544
720
 
@@ -549,7 +725,9 @@ It accepts `associations:`, `ignore:`, `snapshots:`, `group:`,
549
725
  both. Nested paths count at every level, and a `has_many :through` counts its
550
726
  join rows too. Without `activity: true`, an analysis compares a record's own
551
727
  versions and cannot show a change to its associations alone, so only a
552
- record's own version selects it.
728
+ record's own version selects it. With
729
+ [`version_scope:`](#reporting-only-some-changes-version_scope), only the
730
+ record's own versions that the filter keeps select it.
553
731
  - **An analysis can still be empty.** A record is selected by any version on its
554
732
  paths, but its analysis compares only the state you requested. A join row
555
733
  whose role changed, or a version that touched only ignored columns, selects a
@@ -572,15 +750,29 @@ It accepts `associations:`, `ignore:`, `snapshots:`, `group:`,
572
750
 
573
751
  ### Deleted records
574
752
 
575
- A deleted record cannot be tested against a relation's conditions, because it
576
- has no row. Records that changed in the window but no longer exist are listed in
577
- `unreachable`, including those selected by a change to their associations. It
578
- is not paginated and does not mean those records matched the relation.
579
-
580
- `include_deleted: true` also analyzes them from their `destroy` versions and
581
- returns the results in `deleted`, keyed the same way. A value is `nil` when no
582
- `destroy` version was recorded. With this option, existing and deleted records
583
- together count toward `limit:`.
753
+ A relation's conditions read the table, and a deleted record has no row.
754
+ Records that changed in the window but no longer exist are listed in
755
+ `unreachable`, including those selected by a change to their associations,
756
+ when the state each held when it was destroyed meets the relation's
757
+ conditions. That test needs conditions it can read, as `include_deleted:`
758
+ below does; for any other relation, such as one with SQL text or a join,
759
+ `unreachable` lists every such record of the model, whatever the relation. It
760
+ is not paginated.
761
+
762
+ `include_deleted: true` tests each of them against the relation's conditions,
763
+ as it stood when it was destroyed, and keeps only those that match. So a report
764
+ on `Article.where(account_id: account.id)` returns that account's deletions,
765
+ not every account's. The relation's conditions must be plain conditions on the
766
+ model's own columns, as a [readable association scope's](#costly-associations)
767
+ must, ranges included; anything else, such as a join or SQL text, raises
768
+ `ConfigurationError`. `Article.none` keeps none. Its order, limit, and offset
769
+ are ignored here, since they choose a page of existing rows. A record whose
770
+ destroy PaperTrail did not record has no state to test and stays listed.
771
+
772
+ It then analyzes the kept records from their `destroy` versions and returns the
773
+ results in `deleted`, keyed the same way. A value is `nil` when no `destroy`
774
+ version was recorded. With this option, existing and deleted records together
775
+ count toward `limit:`.
584
776
 
585
777
  ```ruby
586
778
  missing_history = report.deleted.select { |_identity, analysis| analysis.nil? }.keys
@@ -607,7 +799,10 @@ report = PaperTrailDiff.report(
607
799
 
608
800
  The filter checks the record as it was when the window started and after each
609
801
  change made inside it. A record matches if any of those states match. With a
610
- hash, every key must be equal (`==`). Values are read from the stored history
802
+ hash, every key must be equal (`==`) once both sides are cast to the
803
+ attribute's type, as `where` casts its values: `{ status: :published }` matches
804
+ an enum, and a date or time matches whether the history stored it as YAML or
805
+ as a string in a `json` column. Values are read from the stored history
611
806
  without calling `reify`, and unknown or unrecorded attributes raise an error
612
807
  instead of matching `nil`. For other checks, pass a callable such as
613
808
  `->(article) { article.title.start_with?("Pub") }`; it receives each state as a
@@ -615,14 +810,18 @@ model instance and may run queries of its own.
615
810
 
616
811
  - It looks only at the record itself, not its associations, and does not check
617
812
  whether a value was held for the whole window.
618
- - The relation still filters current rows. Matching deleted records are listed
813
+ - Only the history of records the relation selects is read, and the relation's
814
+ order, `limit`, and `offset` then page the ones that match. Deleted records,
815
+ which a relation cannot test, are read too, and those that match are listed
619
816
  in `unreachable`.
620
- - Every matching record counts toward `limit:`, including deleted ones and ones
621
- the relation later excludes. Matching stops as soon as the limit is exceeded.
817
+ - Matching records the relation selects count toward `limit:`, and matching
818
+ stops as soon as the limit is exceeded. Matching deleted records count only
819
+ with `include_deleted: true`, as without a filter.
622
820
  - `max_filter_versions:` limits how many versions the filter loads, including
623
- the one version after the window it reads for each record. The default is
624
- 10,000. Going over raises `FilterLimitExceededError` before the extra batch is
625
- checked. It does not limit queries inside a callable.
821
+ the one version after the window it reads for each record, and the versions
822
+ of deleted records. The default is 10,000. Going over raises
823
+ `FilterLimitExceededError` before the extra batch is checked. It does not
824
+ limit queries inside a callable.
626
825
  - `version_scope:` narrows which changes are reported, not which states are
627
826
  tested here.
628
827
  - It is accepted by `analyze_scope` and `report`, which select records from a
@@ -642,12 +841,20 @@ default_scope { where.not(hidden: true) }
642
841
 
643
842
  The gem reads such a scope once and checks each record's stored values against
644
843
  it at every point, so a record that moves into or out of the scope is added or
645
- removed at the version that moved it. It reads equality, `IN`, `nil`, their
646
- `where.not` forms, and `and`/`or` combinations of them on integer, boolean,
647
- and enum columns. String and text columns are read only on SQLite and
648
- PostgreSQL, and only without a column collation: MySQL and SQL Server compare
649
- strings without regard to case or trailing spaces by default, which the gem
650
- cannot repeat.
844
+ removed at the version that moved it. It reads equality, `IN`, their
845
+ `where.not` forms, and `and`/`or`/`not` combinations of them on integer,
846
+ boolean, and enum columns, with SQL's rules for NULL. A `nil` test
847
+ (`IS NULL`, `IS NOT NULL`) is read on a column of any type, so a soft-delete
848
+ scope such as `where(deleted_at: nil)` is read too. String and text columns
849
+ are read only on SQLite and PostgreSQL, and only without a column collation:
850
+ MySQL and SQL Server compare strings without regard to case or trailing spaces
851
+ by default, which the gem cannot repeat, and neither is PostgreSQL's
852
+ `citext`.
853
+
854
+ Ranges, such as `where(position: 2..)` or `where(created_at: 1.day.ago...)`,
855
+ and Arel's `gt`, `gteq`, `lt`, and `lteq`, are read on the columns an order
856
+ can sort by, below: integer, integer enum, date, and datetime columns, with
857
+ the same conditions on a datetime's precision.
651
858
 
652
859
  A `limit` or `offset`, and the order of a `has_one`, are read the same way
653
860
  when the order is total and the gem can repeat it:
@@ -685,14 +892,24 @@ join records fills two places, as in the query. A `has_one :through` whose
685
892
  own scope or whose parts have a limit, an offset, or a `has_one` order is not
686
893
  read.
687
894
 
688
- A few associations still need queries per record: a `limit` or `offset` whose
689
- order is not total or sorts by another column type (strings follow a
690
- collation) or by a datetime history cannot rank exactly, scopes that take the
691
- owner (`->(article) { ... }`), scopes written as SQL strings or ranges, scopes
692
- that join or select, and conditions on other column types or other tables. Batch methods raise `UnsupportedBatchPathError`
693
- for these by default and name the association. Remove it from
694
- `associations:`, or pass `allow_expensive: true` to accept the per-record
695
- cost. Single-record methods always allow them.
895
+ The gem cannot read the rest: a `limit` or `offset` whose order is not total or
896
+ sorts by another column type (strings follow a collation) or by a datetime
897
+ history cannot rank exactly, scopes that take the owner
898
+ (`->(article) { ... }`), scopes written as SQL strings, ranges on other column
899
+ types, scopes that join or select, and conditions on other column types or
900
+ other tables, such as a string column on MySQL. For these it asks PT-AT, which costs queries per
901
+ record and is approximate: PT-AT rebuilds an association from its current
902
+ members, swapping in their older versions, and never checks the scope against
903
+ them. A record that entered the scope since is reported as a member it was
904
+ not, so a comment moved into `-> { where("position > 2") }`, or restored from
905
+ a soft delete, shows as changed rather than added.
906
+
907
+ So every method that reads history raises `UnsupportedAssociationError` for
908
+ such a path and names it. Remove it from `associations:`, rewrite its scope as
909
+ plain conditions on the target's own columns, or pass `allow_approximate:
910
+ true` to accept PT-AT's answer, which also accepts its cost of queries per
911
+ record in batch methods. A comparison of two current records reads no history
912
+ and needs no opt-in. `allow_expensive:` is deprecated and has no effect.
696
913
 
697
914
  A polymorphic `belongs_to` is loaded together too, with one query per target
698
915
  class. Its classes are known only from the records, though, so batch methods
@@ -713,6 +930,10 @@ paths; the parents in a path are included automatically:
713
930
  diff = PaperTrailDiff.compare(from, to, associations: [:author, "comments.replies"])
714
931
  ```
715
932
 
933
+ `associations:` also takes what `includes` takes: a single name
934
+ (`associations: :comments`) or hashes of names to what to include below them
935
+ (`associations: [:author, { comments: :replies }]`).
936
+
716
937
  Supported: `belongs_to`, `has_one`, `has_many` (including `:through`), and
717
938
  `has_and_belongs_to_many`. Associated models must use `has_paper_trail`, and
718
939
  historical states need PT-AT installed and enabled
@@ -786,6 +1007,13 @@ comments.changed.first.associations["replies"].added
786
1007
  member history cannot show. Replace it with a `has_many :through` a
787
1008
  versioned join model, which can be scoped. `association_paths` leaves it
788
1009
  out.
1010
+ - An association keyed on a column other than a primary key, such as
1011
+ `has_many :comments, primary_key: :legacy_id`, raises
1012
+ `UnsupportedAssociationError` up front in every method, and so does a
1013
+ `has_many :through` with such a part. PT-AT records which record a child
1014
+ pointed at by that record's ID, so history read that way would show other
1015
+ records' children. `association_paths` leaves it out. Naming the primary key
1016
+ itself, `primary_key: :id`, is accepted.
789
1017
  - A root that uses single-table inheritance gets the associations of the
790
1018
  class it had at each version, so you can request one that only a subclass
791
1019
  declares. Without an `item_subtype` column, PaperTrail stores only the base
@@ -822,13 +1050,21 @@ Without `synchronize_version_creation_timestamp: false`, the checkpoint takes th
822
1050
  record's old `updated_at` as its time and lands in the past. Save one record per
823
1051
  transaction: HABTM membership cannot be read from a transaction in which
824
1052
  another record with the same HABTM also saved a version (see the
825
- [HABTM limitation](#habtm-limitation)).
1053
+ [HABTM limitation](#habtm-limitation)). A checkpoint records membership as it
1054
+ is when saved, including a member attached or created earlier in the same
1055
+ transaction. A change it captures is credited to the step before it, so its
1056
+ author is whoever saved the version before the checkpoint, unless that is a
1057
+ checkpoint the same person saved just before the change. A member created in
1058
+ the checkpoint's transaction is reported in the step of its creation instead,
1059
+ or, when the range ends at the checkpoint, in the step before it. Don't save
1060
+ the instance that made the HABTM change again; it records the old membership.
826
1061
 
827
1062
  ### Files from ActiveStorage
828
1063
 
829
1064
  `has_one_attached` and `has_many_attached` point to ActiveStorage's own models,
830
1065
  which PaperTrail does not version. Comparing them historically raises
831
- `UnversionedAssociationError` rather than claiming nothing changed. Versioning
1066
+ `UnversionedAssociationError` rather than claiming nothing changed, and
1067
+ `association_paths` leaves them out. Versioning
832
1068
  ActiveStorage's models does not help either, because `has_one_attached` is a
833
1069
  scoped `has_one`.
834
1070
 
@@ -916,6 +1152,7 @@ for numeric IDs).
916
1152
  | `Step` | one `timeline` step |
917
1153
  | `ActivityStep` | one `activity_timeline` step |
918
1154
  | `ActivityBoundary` | one end of a step |
1155
+ | `ChangeRow` | one row from `change_rows` |
919
1156
  | `Analysis` | result of `analyze` |
920
1157
  | `ScopedAnalysis` | result of `analyze_scope` and `report` |
921
1158
  | `TraversalEntry`, `RecordReference` | entries from `each_change` and `each_entry` |
@@ -965,6 +1202,38 @@ end
965
1202
  Both return an `Enumerator` without a block. They never query the database. A
966
1203
  record reachable through two requested paths appears at both.
967
1204
 
1205
+ ### Change rows
1206
+
1207
+ For a table or an export, `change_rows` flattens activity into one row per
1208
+ change, each attributed to its step's source version:
1209
+
1210
+ ```ruby
1211
+ steps.last.change_rows # one activity step
1212
+ analysis.change_rows # an Analysis built with activity: true
1213
+ report.change_rows # { identity => rows }, deleted records last
1214
+
1215
+ row = report.change_rows.values.first.first
1216
+ row.change # :changed, :added, or :removed
1217
+ row.attribute # "body", or nil for a row about a whole record
1218
+ row.from # nil for :added
1219
+ row.to # nil for :removed
1220
+ row.record # a RecordReference, or nil for the root
1221
+ row.association_path # ["comments"], or [] for the root
1222
+ row.source # the step's source_boundary
1223
+ row.to_h # the above, plus version_id, event, whodunnit, recorded_at,
1224
+ # and transaction_id
1225
+ ```
1226
+
1227
+ - An edited field is one `:changed` row. A record that appeared or disappeared
1228
+ gets an `:added` or `:removed` row with `attribute` `nil`, then a row for each
1229
+ field that has a value and for each record it holds.
1230
+ - A replaced `belongs_to` or `has_one` target gives the old record `:removed`
1231
+ rows, then the new one `:added` rows.
1232
+ - Rows follow the diff's walking order, and an analysis's rows follow its steps
1233
+ in time order.
1234
+ - `Analysis#change_rows` raises `ConfigurationError` without `activity: true`.
1235
+ `report.change_rows` leaves out a deleted record without destroy history.
1236
+
968
1237
  ## JSON columns
969
1238
 
970
1239
  A `json` or `jsonb` column is compared as one value. `nested_changes` reports
@@ -984,8 +1253,10 @@ PaperTrailDiff.nested_changes(from_value, to_value) # also accepts two values
984
1253
  inside arrays are compared whole.
985
1254
  - A missing key is `PaperTrailDiff::NestedComparator::ABSENT`, not `nil`, because
986
1255
  `{"a": null}` and `{}` are different.
987
- - A value that is not JSON on both sides returns `{}`; the attribute diff still
988
- has the whole change.
1256
+ - Two arrays at the top, as a `json` column can hold, return one `ArrayChange`
1257
+ at the empty path: `{ [] => <added [3], removed [2]> }`.
1258
+ - A value that is not JSON on both sides, or an object facing an array, returns
1259
+ `{}`; the attribute diff still has the whole change.
989
1260
  - Keys such as `:a` and `"a"` in one object raise `AmbiguousNestedKeyError`.
990
1261
 
991
1262
  ## What a version recorded
@@ -1025,17 +1296,18 @@ Other details:
1025
1296
  saves a query per version, and a deleted record's values keep their types,
1026
1297
  such as an enum's label rather than the integer PaperTrail stored.
1027
1298
 
1028
- Steps name their source version by ID. Load the versions in one query rather
1029
- than one per step; a model with its own version class needs that class here:
1299
+ For activity steps, pass `recorded_changes: true` instead of loading the
1300
+ versions again. Each step's sources then carry what their versions recorded,
1301
+ with no extra queries; see
1302
+ [what each event recorded](#what-each-event-recorded-recorded_changes-true):
1030
1303
 
1031
1304
  <!-- executable:reference-recorded-changes -->
1032
1305
  ```ruby
1033
- ids = steps.map { |step| step.source_boundary.version_id }
1034
- recorded = PaperTrail::Version.where(id: ids).to_h do |version|
1035
- [version.id, PaperTrailDiff.recorded_changes(version)]
1036
- end
1306
+ steps = PaperTrailDiff.activity_timeline(
1307
+ article, associations: [:comments], recorded_changes: true
1308
+ )
1037
1309
 
1038
- recorded.fetch(steps.last.source_boundary.version_id)
1310
+ steps.last.source_boundary.recorded_changes
1039
1311
  ```
1040
1312
 
1041
1313
  ## Checking your setup
@@ -1053,16 +1325,23 @@ and `cycle`. Cycles are marked and not followed.
1053
1325
 
1054
1326
  Only associations a request accepts are listed, so each path can be passed to
1055
1327
  `associations:`. The PaperTrail versions association, a `has_many :through`
1056
- that goes through a HABTM, a scoped HABTM, and a HABTM below the root are left
1057
- out. A
1058
- `has_many :through` Active Record cannot follow, such as one through a
1059
- polymorphic `belongs_to`, raises Active Record's own error from any model
1060
- discovery reaches, as requesting it does.
1061
-
1062
- Check two versions for known problems before relying on association history:
1328
+ that goes through a HABTM, a scoped HABTM, a HABTM below the root, an
1329
+ association keyed on a column other than a primary key, one whose ends keep
1330
+ their versions in different tables, and one whose target, or a model its
1331
+ `has_many :through` goes through, has no `has_paper_trail` are left out. A path
1332
+ history can read only approximately, such as one with a scope written as SQL
1333
+ text, is listed, but requesting it also takes `allow_approximate: true` (see
1334
+ [costly associations](#costly-associations)). A `has_many :through` Active
1335
+ Record cannot follow, such as one through a polymorphic `belongs_to`, raises
1336
+ Active Record's own error from any model discovery reaches, as requesting it
1337
+ does.
1338
+
1339
+ Check a comparison's endpoints for known problems before relying on
1340
+ association history. They are what `compare` takes: two versions, or a version
1341
+ and the current record.
1063
1342
 
1064
1343
  ```ruby
1065
- report = PaperTrailDiff.diagnose(from_version, to_version, associations: [:tags, "comments.replies"])
1344
+ report = PaperTrailDiff.diagnose(from_version, article, associations: [:tags, "comments.replies"])
1066
1345
 
1067
1346
  report.ok? # no errors among the checks that ran
1068
1347
  report.errors.map(&:code)
@@ -1070,7 +1349,12 @@ report.warnings.map(&:code)
1070
1349
  ```
1071
1350
 
1072
1351
  Association checks run only for requested associations. The version-order check
1073
- always runs. `ok?` does not prove the history is complete.
1352
+ always runs. Every reason `compare` would refuse the same request is an error,
1353
+ with the message `compare` would raise, so `ok?` is false whenever `compare`
1354
+ would refuse: an unknown or unsupported association, missing association
1355
+ tracking, an unversioned target, or a path read only approximately. Pass
1356
+ `allow_approximate: true`, as you would to `compare`, and such a path is a
1357
+ warning instead. `ok?` does not prove the history is complete.
1074
1358
 
1075
1359
  ## Models, schemas, and edge cases
1076
1360
 
@@ -1080,14 +1364,27 @@ always runs. `ok?` does not prove the history is complete.
1080
1364
  - **Non-integer primary keys**, such as UUIDs, are supported, and so is a string
1081
1365
  `versions.item_id` column.
1082
1366
  - **Custom version classes** (`has_paper_trail versions: { class_name: }`) are
1083
- supported.
1367
+ supported. For association history, a model and each association you ask
1368
+ for must keep their versions, and their PT-AT version associations, in the
1369
+ same tables. PT-AT reads an association's history from its owner's tables
1370
+ and numbers transactions within one table, so it cannot follow an
1371
+ association into a different version table. Such an association raises
1372
+ `UnsupportedAssociationError` naming both tables, and `association_paths`
1373
+ leaves it out. One custom table shared by the models works like
1374
+ PaperTrail's own.
1375
+ - **A scoped versions association** (`has_paper_trail versions: { scope: }`)
1376
+ does not change what is read: every method reads all of a record's
1377
+ versions, because a history replayed without some of them would credit
1378
+ their changes to the next version kept. To report only some changes, pass
1379
+ [`version_scope:`](#reporting-only-some-changes-version_scope).
1084
1380
 
1085
1381
  ### Version order
1086
1382
 
1087
1383
  Versions are ordered by `created_at`, then by ID when timestamps are equal. That
1088
- works when IDs increase with each insert, as autoincrement IDs do. UUID version
1089
- IDs do not, so two UUID versions with the same timestamp cannot be ordered; the
1090
- gem raises `AmbiguousVersionOrderError` naming both instead of guessing.
1384
+ works when IDs increase with each insert, as autoincrement IDs do, in one
1385
+ table. UUID version IDs do not, and models with their own version tables count
1386
+ IDs separately, so two such versions with the same timestamp cannot be ordered;
1387
+ the gem raises `AmbiguousVersionOrderError` naming both instead of guessing.
1091
1388
  Timestamps stored to the second, as on a MySQL `datetime` column, make ties
1092
1389
  common. `diagnose` reports this as an error.
1093
1390
 
@@ -1095,13 +1392,27 @@ common. `diagnose` reports this as an error.
1095
1392
  backwards, or servers whose clocks disagree, can record a later change with an
1096
1393
  earlier time. Keep server clocks synchronized.
1097
1394
 
1395
+ Versions are ordered by when they were stamped, not by when their transactions
1396
+ committed. A transaction stamps its versions while it is open, so one that
1397
+ commits last can still sort first. The gem cannot see commit order. A report
1398
+ can therefore show a state the database never held, and a result for an earlier
1399
+ version or a closed window can change once that transaction commits.
1400
+ [Time windows](#time-windows) explains how to settle a report.
1401
+
1098
1402
  Association changes between versions with the same timestamp are placed using
1099
1403
  the same order. A few association shapes fall back to PT-AT's own reifier, which
1100
- uses timestamps only: scoped associations, unversioned targets, and the others
1101
- listed in the [architecture notes](architecture.md). For those, `diagnose` warns
1102
- about tied timestamps, and a comparison between two tied versions emits the
1103
- `ambiguous_association_boundary.paper_trail_diff` notification. Recording
1104
- timestamps below one second avoids the problem.
1404
+ uses timestamps only: a path history reads only approximately, which takes
1405
+ `allow_approximate: true`, a composite foreign key, and a polymorphic
1406
+ `belongs_to` whose target's default scope history cannot read. The
1407
+ [architecture notes](architecture.md) list them. When a request includes such a
1408
+ path, or any polymorphic `belongs_to`, whose targets are known only per record,
1409
+ `diagnose` warns about tied timestamps and a comparison between two tied
1410
+ versions emits the `ambiguous_association_boundary.paper_trail_diff`
1411
+ notification. Recording timestamps below one second avoids the problem. On SQL Server, PT-AT compares
1412
+ those timestamps only to about 3 milliseconds (a `datetime` tick, 1/300 s),
1413
+ because the adapter writes the times in its queries that way, so for these
1414
+ shapes two versions saved closer together than that may be read in the wrong
1415
+ order.
1105
1416
 
1106
1417
  ### History stored as JSON
1107
1418
 
@@ -1109,7 +1420,35 @@ PaperTrail's JSON serializer, and a `json` or `jsonb` `object` column, store
1109
1420
  times to milliseconds by default (`ActiveSupport::JSON::Encoding.time_precision`).
1110
1421
  For those histories the gem compares all times at that precision, so an
1111
1422
  unchanged timestamp is not reported as changed. A change smaller than that
1112
- precision was never recorded and cannot be reported.
1423
+ precision was never recorded and cannot be reported. A `json` or `jsonb`
1424
+ column also returns times and dates as strings, which PaperTrail leaves as
1425
+ they are; the gem casts them back to the attribute's type.
1426
+
1427
+ ### Writing history in tests
1428
+
1429
+ To give versions fixed times in a test, save them at those times with
1430
+ `travel_to`:
1431
+
1432
+ ```ruby
1433
+ travel_to(Time.zone.local(2026, 8, 15, 9, 1)) do
1434
+ PaperTrail.request(whodunnit: "alice") { article.update!(title: "Published") }
1435
+ end
1436
+ ```
1437
+
1438
+ If you change `created_at` after saving instead, change every version saved
1439
+ in the same transaction, not only the ones you check. PT-AT gives all of
1440
+ them the same `transaction_id`, including versions of other models saved by
1441
+ callbacks. A version left at the time the test ran makes that transaction
1442
+ seem to last from your chosen time until today. A window edge in between
1443
+ then moves back to the transaction's start (see [Time windows](#time-windows)),
1444
+ and the window can lose the changes the test expects.
1445
+
1446
+ ```ruby
1447
+ def set_version_time(version, time)
1448
+ same = version.transaction_id && { transaction_id: version.transaction_id }
1449
+ PaperTrail::Version.where(same || { id: version.id }).update_all(created_at: time)
1450
+ end
1451
+ ```
1113
1452
 
1114
1453
  ## Instrumentation
1115
1454
 
@@ -1123,6 +1462,7 @@ subscribe to:
1123
1462
  - `load_live_endpoints.paper_trail_diff`
1124
1463
  - `prepare_history.paper_trail_diff`
1125
1464
  - `ambiguous_association_boundary.paper_trail_diff`
1465
+ - `moved_window.paper_trail_diff`, when a [window edge moves](#time-windows)
1126
1466
 
1127
1467
  ```ruby
1128
1468
  ActiveSupport::Notifications.subscribe(/\.paper_trail_diff\z/) do |name, start, finish, _id, payload|
@@ -1130,8 +1470,8 @@ ActiveSupport::Notifications.subscribe(/\.paper_trail_diff\z/) do |name, start,
1130
1470
  end
1131
1471
  ```
1132
1472
 
1133
- Payloads contain counts, model names, association paths, and options, never
1134
- record data. To see the SQL for one call, subscribe to `sql.active_record`
1473
+ Payloads contain counts, model names, ids, times, association paths, and
1474
+ options, never record data. To see the SQL for one call, subscribe to `sql.active_record`
1135
1475
  around it with `ActiveSupport::Notifications.subscribed`.
1136
1476
 
1137
1477
  ## Errors
@@ -1141,9 +1481,9 @@ All errors inherit from `PaperTrailDiff::Error`.
1141
1481
  | Error | Raised when |
1142
1482
  | --- | --- |
1143
1483
  | `ConfigurationError` | An option is invalid |
1144
- | `UnsupportedBatchPathError` | A batch requests a [costly association](#costly-associations) without `allow_expensive: true` |
1484
+ | `UnsupportedBatchPathError` | No longer raised; kept until `allow_expensive:` is removed |
1145
1485
  | `InvalidEndpointError` | A record is unsaved, destroyed, or has unsaved changes, or `recorded_changes` is given something other than a version |
1146
- | `UnloadedAssociationError` | `reload_live_endpoints: false` and an association is not loaded |
1486
+ | `UnloadedAssociationError` | The deprecated `reload_live_endpoints: false`, and an association is not loaded |
1147
1487
  | `VersionMismatchError` | Two endpoints belong to different records |
1148
1488
  | `ReversedEndpointsError` | Two versions are given newest first |
1149
1489
  | `InvalidTimelineRangeError` | A range is invalid, for example `within:` with `from:`, or `:last` |
@@ -1154,8 +1494,8 @@ All errors inherit from `PaperTrailDiff::Error`.
1154
1494
  | `FilterLimitExceededError` | `historical_filter:` needs more than `max_filter_versions:` |
1155
1495
  | `AssociationTrackingUnavailableError` | Association history is requested without PT-AT |
1156
1496
  | `UnknownAssociationError` | An association name does not exist |
1157
- | `UnsupportedAssociationError` | An association's macro is not supported, a path continues past a polymorphic `belongs_to`, a `has_many :through` goes through a HABTM, or a HABTM is scoped or below the root |
1158
- | `UnversionedAssociationError` | An association's target model has no `has_paper_trail`, including the class a polymorphic `belongs_to` names |
1497
+ | `UnsupportedAssociationError` | An association's macro is not supported, a path continues past a polymorphic `belongs_to`, a `has_many :through` goes through a HABTM, a HABTM is scoped or below the root, an association is keyed on a column other than a primary key, or history cannot read a [costly association](#costly-associations) and `allow_approximate: true` was not passed |
1498
+ | `UnversionedAssociationError` | An association's target model, or a model a `has_many :through` goes through, has no `has_paper_trail`, including the class a polymorphic `belongs_to` names |
1159
1499
  | `IncompleteAssociationHistoryError` | PT-AT did not record enough to rebuild an association, such as HABTM without a transaction or in a transaction another record with that HABTM also saved a version in, or a polymorphic type names no class |
1160
1500
  | `UnsupportedLiveActivityError` | A HABTM activity range ends at the current record |
1161
1501
  | `AmbiguousNestedKeyError` | `nested_changes` finds keys like `:a` and `"a"` together |
@@ -1170,8 +1510,21 @@ Results can only be as complete as the history PaperTrail and PT-AT recorded.
1170
1510
  [how PaperTrail stores history](#how-papertrail-stores-history).
1171
1511
  - Deleted versions, schema changes, and serializer changes can affect how old
1172
1512
  versions read.
1173
- - Several queries against the current record are not one database snapshot.
1174
- Use a transaction with a suitable isolation level if that matters.
1513
+ - Each call reads its history and live rows in one snapshot on PostgreSQL and
1514
+ MySQL (through mysql2 or trilogy), and on adapters built on them such as
1515
+ PostGIS, so a commit that lands during the call is seen by none of its
1516
+ statements. The snapshot covers the database of the record's model and its
1517
+ versions; an associated model on another database is read outside it. A
1518
+ call made inside a transaction of your own
1519
+ keeps that transaction's isolation level. To make the whole call see one
1520
+ snapshot, start that transaction at `REPEATABLE READ` or stricter. SQLite and
1521
+ SQL Server take no snapshot, so a commit during a call can appear in some of
1522
+ its statements and not others. A long call holds its snapshot until it ends:
1523
+ on PostgreSQL that can keep VACUUM from reclaiming old row versions, and on
1524
+ MySQL it keeps undo history.
1525
+ - A savepoint that rolls back can split one transaction into several
1526
+ transaction groups; see
1527
+ [one step per transaction group](#one-step-per-transaction-group-transaction).
1175
1528
  - HABTM membership history exists only where PT-AT recorded it in a
1176
1529
  transaction, and only for the root record without a scope: a scoped HABTM,
1177
1530
  or one below the root, raises `UnsupportedAssociationError`, and a transaction that another record with