paper_trail_diff 0.13.1 → 0.15.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (181) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +630 -0
  3. data/QUICKSTART.md +30 -10
  4. data/README.md +91 -13
  5. data/docs/architecture.md +62 -18
  6. data/docs/performance.md +116 -298
  7. data/docs/reference.md +554 -102
  8. data/docs/testing.md +93 -14
  9. data/lib/paper_trail_diff/activity_belongs_to_event_applier.rb +19 -9
  10. data/lib/paper_trail_diff/activity_boundary.rb +83 -26
  11. data/lib/paper_trail_diff/activity_child_candidate_loader.rb +2 -1
  12. data/lib/paper_trail_diff/activity_collection_event_applier.rb +11 -1
  13. data/lib/paper_trail_diff/activity_collection_record_updater.rb +16 -2
  14. data/lib/paper_trail_diff/activity_event.rb +41 -10
  15. data/lib/paper_trail_diff/activity_event_record_resolver.rb +20 -1
  16. data/lib/paper_trail_diff/activity_event_route_finder.rb +2 -1
  17. data/lib/paper_trail_diff/activity_event_snapshot_refresher.rb +1 -1
  18. data/lib/paper_trail_diff/activity_history.rb +34 -9
  19. data/lib/paper_trail_diff/activity_range.rb +32 -5
  20. data/lib/paper_trail_diff/activity_relationship.rb +12 -0
  21. data/lib/paper_trail_diff/activity_root_snapshot_refresher.rb +5 -2
  22. data/lib/paper_trail_diff/activity_snapshot_sequence.rb +76 -37
  23. data/lib/paper_trail_diff/activity_target_candidate_loader.rb +131 -0
  24. data/lib/paper_trail_diff/activity_timeline_builder.rb +281 -120
  25. data/lib/paper_trail_diff/activity_transaction_grouper.rb +2 -1
  26. data/lib/paper_trail_diff/activity_tree_filter.rb +114 -0
  27. data/lib/paper_trail_diff/activity_version_collector.rb +37 -24
  28. data/lib/paper_trail_diff/activity_view.rb +69 -0
  29. data/lib/paper_trail_diff/analysis.rb +13 -1
  30. data/lib/paper_trail_diff/analysis_batch.rb +24 -30
  31. data/lib/paper_trail_diff/approximate_paths.rb +48 -0
  32. data/lib/paper_trail_diff/association_activity_edge.rb +236 -0
  33. data/lib/paper_trail_diff/association_activity_selection.rb +197 -0
  34. data/lib/paper_trail_diff/association_discovery.rb +14 -11
  35. data/lib/paper_trail_diff/association_key.rb +56 -0
  36. data/lib/paper_trail_diff/association_traversal.rb +180 -47
  37. data/lib/paper_trail_diff/batch_boundary_resolver.rb +3 -4
  38. data/lib/paper_trail_diff/batch_preload.rb +53 -0
  39. data/lib/paper_trail_diff/batched_root_analyzer.rb +12 -14
  40. data/lib/paper_trail_diff/batched_root_versions.rb +53 -47
  41. data/lib/paper_trail_diff/change_row.rb +133 -0
  42. data/lib/paper_trail_diff/collection_comparator.rb +5 -32
  43. data/lib/paper_trail_diff/comparison_batch.rb +37 -10
  44. data/lib/paper_trail_diff/configuration.rb +35 -12
  45. data/lib/paper_trail_diff/deleted_analysis_batch.rb +45 -27
  46. data/lib/paper_trail_diff/deleted_root_filter.rb +79 -0
  47. data/lib/paper_trail_diff/deprecation.rb +61 -0
  48. data/lib/paper_trail_diff/diagnostics.rb +105 -102
  49. data/lib/paper_trail_diff/direct_record_state_loader.rb +10 -1
  50. data/lib/paper_trail_diff/endpoint.rb +26 -3
  51. data/lib/paper_trail_diff/errors.rb +5 -2
  52. data/lib/paper_trail_diff/fallback_paths.rb +38 -0
  53. data/lib/paper_trail_diff/habtm_transaction_owners.rb +68 -0
  54. data/lib/paper_trail_diff/historical_association_reifier.rb +34 -6
  55. data/lib/paper_trail_diff/historical_filter.rb +19 -10
  56. data/lib/paper_trail_diff/historical_snapshot_store.rb +63 -76
  57. data/lib/paper_trail_diff/live_endpoint_batch_loader.rb +12 -13
  58. data/lib/paper_trail_diff/live_graph_collector.rb +1 -1
  59. data/lib/paper_trail_diff/nested_comparator.rb +19 -18
  60. data/lib/paper_trail_diff/paper_trail_adapter.rb +136 -125
  61. data/lib/paper_trail_diff/preloaded_endpoint_batch_loader.rb +14 -2
  62. data/lib/paper_trail_diff/prepared_activity_index.rb +110 -36
  63. data/lib/paper_trail_diff/prepared_edge_loader.rb +31 -46
  64. data/lib/paper_trail_diff/prepared_history.rb +70 -53
  65. data/lib/paper_trail_diff/prepared_history_loader.rb +34 -43
  66. data/lib/paper_trail_diff/prepared_record_index.rb +64 -7
  67. data/lib/paper_trail_diff/prepared_snapshot_builder.rb +1 -1
  68. data/lib/paper_trail_diff/prepared_state_association_reader.rb +119 -29
  69. data/lib/paper_trail_diff/recorded_changes.rb +83 -0
  70. data/lib/paper_trail_diff/recorded_class.rb +53 -0
  71. data/lib/paper_trail_diff/root_version_selection.rb +4 -2
  72. data/lib/paper_trail_diff/scope_condition.rb +386 -0
  73. data/lib/paper_trail_diff/scope_ranking.rb +206 -0
  74. data/lib/paper_trail_diff/scoped_analysis.rb +11 -0
  75. data/lib/paper_trail_diff/scoped_analysis_builder.rb +20 -11
  76. data/lib/paper_trail_diff/scoped_root_selection.rb +105 -49
  77. data/lib/paper_trail_diff/snapshot.rb +16 -1
  78. data/lib/paper_trail_diff/snapshot_normalizer.rb +10 -3
  79. data/lib/paper_trail_diff/support.rb +38 -6
  80. data/lib/paper_trail_diff/time_range.rb +37 -18
  81. data/lib/paper_trail_diff/time_version_range.rb +5 -5
  82. data/lib/paper_trail_diff/timeline_builder.rb +7 -7
  83. data/lib/paper_trail_diff/timeline_range.rb +9 -1
  84. data/lib/paper_trail_diff/timeline_snapshot_provider.rb +24 -17
  85. data/lib/paper_trail_diff/transaction_cut.rb +181 -0
  86. data/lib/paper_trail_diff/traversal_preparer.rb +134 -29
  87. data/lib/paper_trail_diff/version.rb +1 -1
  88. data/lib/paper_trail_diff/version_association_candidate_scope.rb +13 -3
  89. data/lib/paper_trail_diff/version_boundary.rb +37 -8
  90. data/lib/paper_trail_diff/version_query.rb +34 -10
  91. data/lib/paper_trail_diff/version_range.rb +8 -15
  92. data/lib/paper_trail_diff/version_scope_filter.rb +20 -13
  93. data/lib/paper_trail_diff/version_sequence_diagnostics.rb +31 -16
  94. data/lib/paper_trail_diff/version_table.rb +60 -0
  95. data/lib/paper_trail_diff.rb +126 -55
  96. data/sig/generated/paper_trail_diff/activity_belongs_to_event_applier.rbs +10 -2
  97. data/sig/generated/paper_trail_diff/activity_boundary.rbs +54 -19
  98. data/sig/generated/paper_trail_diff/activity_collection_event_applier.rbs +5 -0
  99. data/sig/generated/paper_trail_diff/activity_collection_record_updater.rbs +5 -0
  100. data/sig/generated/paper_trail_diff/activity_event.rbs +25 -4
  101. data/sig/generated/paper_trail_diff/activity_event_record_resolver.rbs +7 -0
  102. data/sig/generated/paper_trail_diff/activity_history.rbs +21 -6
  103. data/sig/generated/paper_trail_diff/activity_range.rbs +14 -3
  104. data/sig/generated/paper_trail_diff/activity_relationship.rbs +3 -0
  105. data/sig/generated/paper_trail_diff/activity_root_snapshot_refresher.rbs +2 -0
  106. data/sig/generated/paper_trail_diff/activity_snapshot_sequence.rbs +48 -9
  107. data/sig/generated/paper_trail_diff/activity_target_candidate_loader.rbs +79 -0
  108. data/sig/generated/paper_trail_diff/activity_timeline_builder.rbs +138 -28
  109. data/sig/generated/paper_trail_diff/activity_tree_filter.rbs +73 -0
  110. data/sig/generated/paper_trail_diff/activity_version_collector.rbs +12 -4
  111. data/sig/generated/paper_trail_diff/activity_view.rbs +42 -0
  112. data/sig/generated/paper_trail_diff/analysis.rbs +8 -1
  113. data/sig/generated/paper_trail_diff/analysis_batch.rbs +8 -15
  114. data/sig/generated/paper_trail_diff/approximate_paths.rbs +23 -0
  115. data/sig/generated/paper_trail_diff/association_activity_edge.rbs +148 -0
  116. data/sig/generated/paper_trail_diff/association_activity_selection.rbs +112 -0
  117. data/sig/generated/paper_trail_diff/association_discovery.rbs +8 -5
  118. data/sig/generated/paper_trail_diff/association_key.rbs +29 -0
  119. data/sig/generated/paper_trail_diff/association_traversal.rbs +88 -9
  120. data/sig/generated/paper_trail_diff/batch_preload.rbs +29 -0
  121. data/sig/generated/paper_trail_diff/batched_root_analyzer.rbs +9 -10
  122. data/sig/generated/paper_trail_diff/batched_root_versions.rbs +14 -10
  123. data/sig/generated/paper_trail_diff/change_row.rbs +86 -0
  124. data/sig/generated/paper_trail_diff/collection_comparator.rbs +2 -16
  125. data/sig/generated/paper_trail_diff/comparison_batch.rbs +22 -4
  126. data/sig/generated/paper_trail_diff/configuration.rbs +19 -3
  127. data/sig/generated/paper_trail_diff/deleted_analysis_batch.rbs +20 -8
  128. data/sig/generated/paper_trail_diff/deleted_root_filter.rbs +48 -0
  129. data/sig/generated/paper_trail_diff/deprecation.rbs +35 -0
  130. data/sig/generated/paper_trail_diff/diagnostics.rbs +44 -32
  131. data/sig/generated/paper_trail_diff/direct_record_state_loader.rbs +6 -0
  132. data/sig/generated/paper_trail_diff/endpoint.rbs +11 -2
  133. data/sig/generated/paper_trail_diff/errors.rbs +5 -2
  134. data/sig/generated/paper_trail_diff/fallback_paths.rbs +22 -0
  135. data/sig/generated/paper_trail_diff/habtm_transaction_owners.rbs +40 -0
  136. data/sig/generated/paper_trail_diff/historical_association_reifier.rbs +13 -0
  137. data/sig/generated/paper_trail_diff/historical_filter.rbs +12 -2
  138. data/sig/generated/paper_trail_diff/historical_snapshot_store.rbs +36 -30
  139. data/sig/generated/paper_trail_diff/live_endpoint_batch_loader.rbs +5 -3
  140. data/sig/generated/paper_trail_diff/nested_comparator.rbs +10 -7
  141. data/sig/generated/paper_trail_diff/paper_trail_adapter.rbs +55 -48
  142. data/sig/generated/paper_trail_diff/preloaded_endpoint_batch_loader.rbs +9 -1
  143. data/sig/generated/paper_trail_diff/prepared_activity_index.rbs +52 -10
  144. data/sig/generated/paper_trail_diff/prepared_edge_loader.rbs +4 -12
  145. data/sig/generated/paper_trail_diff/prepared_history.rbs +40 -24
  146. data/sig/generated/paper_trail_diff/prepared_history_loader.rbs +15 -13
  147. data/sig/generated/paper_trail_diff/prepared_record_index.rbs +34 -2
  148. data/sig/generated/paper_trail_diff/prepared_state_association_reader.rbs +56 -11
  149. data/sig/generated/paper_trail_diff/recorded_changes.rbs +47 -0
  150. data/sig/generated/paper_trail_diff/recorded_class.rbs +22 -0
  151. data/sig/generated/paper_trail_diff/root_version_selection.rbs +2 -0
  152. data/sig/generated/paper_trail_diff/scope_condition.rbs +219 -0
  153. data/sig/generated/paper_trail_diff/scope_ranking.rbs +125 -0
  154. data/sig/generated/paper_trail_diff/scoped_analysis.rbs +6 -0
  155. data/sig/generated/paper_trail_diff/scoped_analysis_builder.rbs +8 -3
  156. data/sig/generated/paper_trail_diff/scoped_root_selection.rbs +49 -11
  157. data/sig/generated/paper_trail_diff/snapshot.rbs +8 -0
  158. data/sig/generated/paper_trail_diff/snapshot_normalizer.rbs +6 -0
  159. data/sig/generated/paper_trail_diff/support.rbs +20 -2
  160. data/sig/generated/paper_trail_diff/time_range.rbs +20 -2
  161. data/sig/generated/paper_trail_diff/time_version_range.rbs +4 -4
  162. data/sig/generated/paper_trail_diff/timeline_builder.rbs +2 -2
  163. data/sig/generated/paper_trail_diff/timeline_range.rbs +2 -0
  164. data/sig/generated/paper_trail_diff/timeline_snapshot_provider.rbs +17 -7
  165. data/sig/generated/paper_trail_diff/transaction_cut.rbs +94 -0
  166. data/sig/generated/paper_trail_diff/traversal_preparer.rbs +66 -8
  167. data/sig/generated/paper_trail_diff/version_association_candidate_scope.rbs +7 -0
  168. data/sig/generated/paper_trail_diff/version_boundary.rbs +20 -3
  169. data/sig/generated/paper_trail_diff/version_query.rbs +18 -2
  170. data/sig/generated/paper_trail_diff/version_range.rbs +4 -2
  171. data/sig/generated/paper_trail_diff/version_scope_filter.rbs +7 -4
  172. data/sig/generated/paper_trail_diff/version_sequence_diagnostics.rbs +16 -9
  173. data/sig/generated/paper_trail_diff/version_table.rbs +30 -0
  174. data/sig/generated/paper_trail_diff.rbs +47 -25
  175. metadata +62 -10
  176. data/lib/paper_trail_diff/activity_grouping.rb +0 -38
  177. data/lib/paper_trail_diff/batch_safety.rb +0 -55
  178. data/lib/paper_trail_diff/time_activity_timeline_builder.rb +0 -224
  179. data/sig/generated/paper_trail_diff/activity_grouping.rbs +0 -22
  180. data/sig/generated/paper_trail_diff/batch_safety.rbs +0 -24
  181. data/sig/generated/paper_trail_diff/time_activity_timeline_builder.rbs +0 -92
data/QUICKSTART.md CHANGED
@@ -37,6 +37,19 @@ table. `--with-changes` adds `object_changes`, which PaperTrail uses for
37
37
  for each event. This gem can also compare the full earlier records saved in
38
38
  PaperTrail's `object` column.
39
39
 
40
+ PaperTrail stores earlier records as YAML, and Rails reads back only the YAML
41
+ classes you allow. Without this, reading the first version raises
42
+ `Psych::DisallowedClass`. Add PaperTrail's list to `config/application.rb`,
43
+ inside `class Application`:
44
+
45
+ <!-- executable:quickstart-config -->
46
+ ```ruby
47
+ config.active_record.yaml_column_permitted_classes = [
48
+ ActiveRecord::Type::Time::Value, ActiveSupport::TimeWithZone, ActiveSupport::TimeZone,
49
+ BigDecimal, Date, Symbol, Time
50
+ ]
51
+ ```
52
+
40
53
  ## 2. Version a model
41
54
 
42
55
  Create an article model and tell PaperTrail to track its changes:
@@ -203,9 +216,10 @@ report.analyses.each_value { |analysis| analysis.diff.to_h }
203
216
  are selected, the call raises instead of returning a partial report. Omit
204
217
  `include_deleted:` if you only need articles that still exist.
205
218
 
206
- An article needs its own PaperTrail version in the time window to be selected.
207
- Editing a comment alone does not select its article. A relation's `where`
208
- conditions check current values; they cannot check a deleted article's values.
219
+ An article is selected when it has a PaperTrail version in the time window. With
220
+ `associations: [:comments]`, editing a comment alone also selects its article. A
221
+ relation's `where` conditions check current values; they cannot check a deleted
222
+ article's values.
209
223
 
210
224
  For one article, use a window ending now:
211
225
 
@@ -232,10 +246,11 @@ The same `within:` option works on `activity_timeline`, `analyze`,
232
246
  | Recent activity through now, for the records a relation selects | `report(scope, since:, limit:)` |
233
247
 
234
248
  Use the batch methods for many records so they can share database queries.
235
- Some scoped associations and `has_many :through` histories still need queries
236
- for each record. Batch calls reject those with `UnsupportedBatchPathError`.
237
- The error names the association. `allow_expensive: true` permits that extra work;
238
- see the [performance guide](docs/performance.md) before using it in a large report.
249
+ A few scoped associations, such as one whose scope is SQL text, can be read
250
+ only with PT-AT, approximately and one record at a time. Every method rejects
251
+ them with `UnsupportedAssociationError`, naming the association; passing
252
+ `allow_approximate: true` accepts both costs. See
253
+ [costly associations](docs/reference.md#costly-associations).
239
254
 
240
255
  When associations are included, `timeline` compares their values at the article's
241
256
  versions. `activity_timeline` also reports the comments' own version events, even
@@ -374,9 +389,11 @@ window to select that article.
374
389
  change. Pass `article` to read the current saved values.
375
390
  - PaperTrail's creation version returns `nil` from `reify`. There was no record
376
391
  before creation; the diff reports the record being added.
377
- - A range ending at the current article includes changes PaperTrail never
378
- recorded, such as `update_columns`. Pass a version as `to:` to stop at
379
- recorded history.
392
+ - Changes PaperTrail never recorded, such as `update_columns`, still show up:
393
+ in the step for the article's previous recorded change, credited to its
394
+ author. Passing a version as `to:` leaves out only the ones made after it.
395
+ `PaperTrailDiff.recorded_changes(version)` reads just what a version
396
+ recorded.
380
397
  - `ignore:` replaces the default list. Include `updated_at` yourself when using
381
398
  a custom list and you still want it ignored.
382
399
  - Historical associations require PT-AT to be installed, loaded, migrated, and
@@ -385,6 +402,9 @@ window to select that article.
385
402
  comments does not automatically include every association on each comment.
386
403
  - Historical output cannot contain data that PaperTrail or PT-AT did not record
387
404
  or can no longer reconstruct.
405
+ - A report with `since:` or `within:` gets slow as the version table grows
406
+ unless you add the
407
+ [two version indexes](docs/architecture.md#database-indexes).
388
408
 
389
409
  For association support, performance, error handling, ignore rules, and all
390
410
  available result methods, see the
data/README.md CHANGED
@@ -121,10 +121,15 @@ responsible for that change, including its `whodunnit`, `event`, `item_type`,
121
121
  and `item_id`. Its `recorded_at` is the version's `created_at`.
122
122
  Use this source information for attribution: the version used to read the
123
123
  resulting state, exposed as `to_boundary`, may have been saved by someone else.
124
+ A change saved without a version, such as with `update_columns`, appears in the
125
+ step for that record's previous recorded change. To also read only what each
126
+ version recorded, pass `recorded_changes: true` and read
127
+ `step.source_boundary.recorded_changes`; see
128
+ [what each event recorded](docs/reference.md#what-each-event-recorded-recorded_changes-true).
124
129
 
125
130
  `group: :transaction` combines changes from one database transaction. Its source
126
- information comes from the first event in that group. Omit `group:` if you need
127
- to attribute each version separately.
131
+ information comes from the first event in that group, and `step.sources` lists
132
+ every event in it, each with its own author.
128
133
 
129
134
  `snapshots: true` also keeps copies of the compared records' data, including
130
135
  unchanged fields. These read-only copies are called snapshots. Access them as
@@ -172,19 +177,30 @@ unavailable = report.deleted.select { |_identity, analysis| analysis.nil? }.keys
172
177
  deleted_activity = report.deleted.transform_values do |analysis|
173
178
  analysis&.activity_timeline
174
179
  end
180
+ last_state = report.deleted.transform_values do |analysis|
181
+ analysis&.activity_timeline&.last&.diff&.record_presence_change&.from
182
+ end
175
183
  ```
176
184
 
177
185
  The results are already computed; you do not need to query or reify each deleted
178
- record. Omit `include_deleted:` if you only need records that still exist.
186
+ record. `last_state` is each record just before its deletion, which the final
187
+ step removes. Use it to show what a deleted record was, such as its title:
188
+ `analysis.from_snapshot` is the record when the window began, so it is `nil` for
189
+ one created inside the window. Omit `include_deleted:` if you only need records
190
+ that still exist.
179
191
 
180
192
  Three selection rules matter:
181
193
 
182
- - The record itself needs a PaperTrail version in the time window. For example,
183
- editing only a comment does not select its article. Comment activity is included
184
- for selected articles when you request `associations: [:comments]`.
194
+ - A record is selected when it, or a record on one of the `associations:` you
195
+ request, has a PaperTrail version in the time window. With
196
+ `associations: [:comments]`, editing only a comment selects its article, and
197
+ the article's report shows the edit.
185
198
  - A relation such as `Article.where(status: "open")` checks today's values.
186
- It cannot establish whether deleted articles matched that condition. Deleted
187
- records are therefore listed separately.
199
+ Deleted records have none, so they are listed separately. With
200
+ `include_deleted: true`, the gem tests each against the relation as it was
201
+ when destroyed and keeps only those that match, so a report on one account's
202
+ records holds only that account's deletions. That needs plain conditions on
203
+ the model's own columns; a join or SQL text raises `ConfigurationError`.
188
204
  - `limit: 100` is a safety ceiling, not pagination. Selecting too many records
189
205
  raises `BatchLimitExceededError`. With `include_deleted: true`, both existing
190
206
  and deleted records count toward it. Use a narrower relation or time window,
@@ -197,7 +213,51 @@ for pagination, deleted records, and selecting by past values with
197
213
  `historical_filter:`.
198
214
 
199
215
  For a fixed past window, use `analyze_scope(Article.all, within: start_time...end_time,
200
- limit: 100)`.
216
+ activity: true, limit: 100)`. Without `activity: true`, `analyze_scope` compares
217
+ each record's own versions, so it selects only records with a version of their
218
+ own.
219
+
220
+ ## 4. Turn a report into rows
221
+
222
+ For a history table or an export, `change_rows` flattens a report into one row
223
+ per change. Each row names the record, the change, and the version it came
224
+ from. This example requires the association setup in the quickstart:
225
+
226
+ <!-- executable:recipe-summary -->
227
+ ```ruby
228
+ report = PaperTrailDiff.report(
229
+ Article.all, since: 1.day.ago, limit: 100,
230
+ associations: [:comments], include_deleted: true
231
+ )
232
+
233
+ rows = report.change_rows.flat_map do |root, changes|
234
+ changes.map { |row| row.to_h.merge(root: root) }
235
+ end
236
+ ```
237
+
238
+ `report.change_rows` is a hash keyed like `report.analyses`, with deleted
239
+ records last. Each value lists that record's rows in time order. Each row
240
+ covers one of two things:
241
+
242
+ - **A field**: `attribute` names it, and `change` is `:changed` for an edit.
243
+ For a record that appeared or disappeared, each of its fields gets an
244
+ `:added` or `:removed` row, holding its value in `to` or `from`. Blank
245
+ fields of such a record get no row.
246
+ - **A whole record**: `attribute` is `nil`, and `change` is `:added` or
247
+ `:removed`. The record was created or deleted, a comment joined or left the
248
+ article, or a `belongs_to` pointed somewhere new, which gives a `:removed`
249
+ row for the old record and an `:added` row for the new one. A comment moved
250
+ from another article is `:added` too; its `event` is then `"update"`.
251
+
252
+ `record` is the record the row is about, or `nil` for the article itself, and
253
+ `association_path` is how it was reached, such as `["comments"]`. `to_h` adds
254
+ the step's source version: `version_id`, `event`, `whodunnit`, `recorded_at`,
255
+ and `transaction_id`. With `group: :transaction`, that is the transaction's
256
+ first event; `step.sources` lists every event. `row.source` is the
257
+ `ActivityBoundary` itself.
258
+
259
+ `change_rows` is also on an `Analysis` built with `activity: true` and on each
260
+ activity step. Labels, grouping, and formatting are left to your application.
201
261
 
202
262
  ## Choose a method
203
263
 
@@ -225,13 +285,31 @@ method inside a loop. They share history loading across records. Ordinary
225
285
  record and direct-association histories can be read without creating a model
226
286
  instance for every historical state.
227
287
 
228
- Some associations still need per-record queries. Batch methods reject scoped
229
- associations and certain `has_many :through` histories by default with
230
- `UnsupportedBatchPathError`. The error names the association. Remove it from
231
- `associations:`, or pass `allow_expensive: true` if you accept that cost.
288
+ Scopes and default scopes made of plain conditions, such as
289
+ `-> { where(archived: false) }`, are batched too, and so is a `limit` or
290
+ `offset` ordered by integer, boolean, date, or datetime columns down to the
291
+ primary key, such as `-> { order(:position, :id).limit(3) }`, on a direct
292
+ association or a `has_many :through`. History cannot read a few scopes, such
293
+ as a `limit` ordered by a string, a scope that takes the owner, or SQL text.
294
+ PT-AT reads those instead, one record at a time and only approximately, so
295
+ every method raises `UnsupportedAssociationError` naming the association.
296
+ Remove it from `associations:`, or pass `allow_approximate: true` to accept
297
+ PT-AT's answer and its cost. See
298
+ [costly associations](docs/reference.md#costly-associations).
232
299
  Callbacks and callable historical filters can also issue queries. See the
233
300
  [performance guide](docs/performance.md) for measured behavior and limitations.
234
301
 
302
+ On a large version table, add these indexes, or every `since:` or `within:`
303
+ report reads all of the model's versions. See
304
+ [database indexes](docs/architecture.md#database-indexes) for why.
305
+
306
+ ```ruby
307
+ add_index :versions, %i[item_type item_id created_at id],
308
+ name: "index_versions_on_item_and_timeline"
309
+ add_index :versions, %i[item_type created_at item_id],
310
+ name: "index_versions_on_type_and_time"
311
+ ```
312
+
235
313
  - `updated_at` is ignored. Passing `ignore:` replaces the default list.
236
314
  - Records passed as current state must be saved and have no unsaved changes.
237
315
  They are reloaded from the database.
data/docs/architecture.md CHANGED
@@ -36,9 +36,26 @@ only the association paths you request:
36
36
 
37
37
  From that, the gem builds each state without querying again. Plain recorded
38
38
  attributes are read straight from the stored version, without creating a model
39
- instance. Models with initialization callbacks, custom attribute types or
40
- defaults, default scopes, or encrypted attributes are rebuilt with PaperTrail's
41
- `reify` instead, so their behavior is kept.
39
+ instance, including the members of a `has_many :through` and the target of a
40
+ polymorphic `belongs_to`. Models with initialization callbacks, custom
41
+ attribute types or defaults, default scopes that assign attributes, or
42
+ encrypted attributes are rebuilt with PaperTrail's `reify` instead, so their
43
+ behavior is kept.
44
+
45
+ An association scope or a target's default scope is read once into a
46
+ `ScopeCondition`: the `where` conditions of the relation it builds, kept only
47
+ when each is an equality, `IN`, or `nil` test, or a negation of one, on the
48
+ target's own column of a type Ruby compares as the database does. A record
49
+ belongs to the association at a point when its key matches and its state
50
+ there meets the condition. A `limit`, `offset`, or `has_one` order is read
51
+ into a `ScopeRanking` when its order is total and every key compares in Ruby
52
+ as the database sorts it, including where each adapter puts NULL; the records
53
+ that meet the condition are then sorted, offset, and limited as the
54
+ association's query would. A `has_many :through` reads its parts' conditions
55
+ only, at any depth, as its query joins them, and ranks by its own order the
56
+ rows the join yields, a record reached twice counting twice. Activity reads a scoped branch again at each event
57
+ on it rather than patching the previous snapshot, since any edit can move a
58
+ record into or out of the scope, and with a ranking, move another record too.
42
59
 
43
60
  Each rebuilt state is an immutable snapshot. Activity timelines carry the
44
61
  previous snapshot forward and replace only the parts an event changed, so an
@@ -49,34 +66,56 @@ PT-AT transaction are applied together.
49
66
 
50
67
  These association shapes cannot be read from the loaded history. The gem asks
51
68
  PT-AT to rebuild them at each point instead, which costs queries per record and
52
- resolves association membership by timestamp only:
69
+ resolves association membership by timestamp only. PT-AT starts from the
70
+ association's current members and swaps in their older versions, without
71
+ checking a scope against them, so its membership is approximate:
53
72
 
54
- - associations with a scope, including `limit` and owner-dependent scopes;
55
- - targets with a default scope;
56
- - targets that are not versioned;
57
- - composite foreign keys;
58
- - `has_many :through` associations whose source is not a `belongs_to`.
73
+ - associations with a scope the gem cannot read as plain conditions and an
74
+ exact ranking, including owner-dependent scopes and a `limit` without a
75
+ total integer order;
76
+ - targets with such a default scope, including a polymorphic `belongs_to`
77
+ target, whose class and scope are known only from each record;
78
+ - composite foreign keys.
59
79
 
60
- Batch methods reject the scoped and through cases unless you pass
61
- `allow_expensive: true`; see [costly associations](reference.md#costly-associations).
80
+ Every method that reads history rejects the scoped cases unless you pass
81
+ `allow_approximate: true`; see [costly associations](reference.md#costly-associations).
62
82
 
63
83
  HABTM membership comes from the snapshots PT-AT records with each transaction,
64
- so it is only known at root versions saved in a transaction.
84
+ so it is only known at root versions saved in a transaction. Those snapshots
85
+ name the transaction and the association but not the owner, so they give only
86
+ the root's own membership, and a HABTM below the root is rejected before any
87
+ history is read. Even at the root, a transaction in which another record with a
88
+ HABTM of the same name saved a version holds both records' members;
89
+ `HabtmTransactionOwners` finds those transactions, and reading one raises.
65
90
 
66
91
  ## Database indexes
67
92
 
68
- For large histories, add a composite index that matches how the gem reads
69
- versions:
93
+ For large histories, add two indexes to each version table, one for each way
94
+ the gem reads versions:
70
95
 
71
96
  ```ruby
72
97
  add_index :versions, %i[item_type item_id created_at id],
73
98
  name: "index_versions_on_item_and_timeline"
99
+ add_index :versions, %i[item_type created_at item_id],
100
+ name: "index_versions_on_type_and_time"
74
101
  ```
75
102
 
76
- It helps most with repeated `within:` queries, which filter by `item_type`,
77
- `item_id`, and `created_at`, then order by time and ID. Keep PT-AT's own index
78
- on `version_associations` (`foreign_key_name`, `foreign_key_id`,
79
- `foreign_type`).
103
+ The first serves reading one record's history: it filters by `item_type` and
104
+ `item_id`, then by time, in time and ID order. It covers PaperTrail's own
105
+ `(item_type, item_id)` index, which you can then drop.
106
+
107
+ The second serves finding which records changed in a window, as `report`,
108
+ `analyze_scope`, and the other batch methods do with `since:` or `within:`.
109
+ They filter by `item_type` and `created_at` alone, which the first index cannot
110
+ narrow, so without the second the database reads every version of the model.
111
+ On PostgreSQL with 5 million article versions, a report on five articles
112
+ changed in the last hour took about 230 ms without it and 2 ms with it.
113
+ Ending the index with `item_id` lets the database answer from the index alone,
114
+ and SQLite, which keeps no table statistics unless you run `ANALYZE`, picks it
115
+ only then.
116
+
117
+ Keep PT-AT's own index on `version_associations` (`foreign_key_name`,
118
+ `foreign_key_id`, `foreign_type`).
80
119
 
81
120
  ## Remaining costs
82
121
 
@@ -86,3 +125,8 @@ on `version_associations` (`foreign_key_name`, `foreign_key_id`,
86
125
  - Repeated events in one very wide collection still copy that collection's
87
126
  list of records for each step. The comparison work stays small, but memory
88
127
  traffic grows with *events × collection size*.
128
+ - With PT-AT installed, a `within:` or `since:` window looks up each root's
129
+ versions on either side of both edges, to move an edge that falls inside a
130
+ transaction (see [time windows](reference.md#time-windows)). These are
131
+ four indexed lookups per root, sent 200 roots per query, whatever the
132
+ length of the root's history: about 0.2 ms per root on SQLite.