paper_trail_diff 0.16.0 → 0.17.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +566 -1
- data/README.md +42 -7
- data/docs/architecture.md +7 -5
- data/docs/performance.md +86 -38
- data/docs/reference.md +309 -92
- data/docs/testing.md +240 -32
- data/lib/paper_trail_diff/activity_belongs_to_event_applier.rb +2 -16
- data/lib/paper_trail_diff/activity_boundary.rb +55 -34
- data/lib/paper_trail_diff/activity_child_candidate_loader.rb +46 -57
- data/lib/paper_trail_diff/activity_collection_event_applier.rb +3 -13
- data/lib/paper_trail_diff/activity_collection_route_updater.rb +1 -1
- data/lib/paper_trail_diff/activity_event.rb +33 -21
- data/lib/paper_trail_diff/activity_event_record_resolver.rb +16 -116
- data/lib/paper_trail_diff/activity_event_route_finder.rb +29 -29
- data/lib/paper_trail_diff/activity_event_snapshot_refresher.rb +2 -7
- data/lib/paper_trail_diff/activity_history.rb +90 -49
- data/lib/paper_trail_diff/activity_range.rb +9 -35
- data/lib/paper_trail_diff/activity_root_snapshot_refresher.rb +74 -40
- data/lib/paper_trail_diff/activity_root_steps.rb +10 -13
- data/lib/paper_trail_diff/activity_scope_filter.rb +77 -0
- data/lib/paper_trail_diff/activity_snapshot_delta.rb +12 -15
- data/lib/paper_trail_diff/activity_snapshot_sequence.rb +37 -15
- data/lib/paper_trail_diff/activity_target_candidate_loader.rb +35 -43
- data/lib/paper_trail_diff/activity_timeline_builder.rb +194 -249
- data/lib/paper_trail_diff/activity_transaction_grouper.rb +3 -1
- data/lib/paper_trail_diff/activity_tree_filter.rb +54 -13
- data/lib/paper_trail_diff/activity_version_collector.rb +72 -138
- data/lib/paper_trail_diff/activity_view.rb +22 -11
- data/lib/paper_trail_diff/analysis.rb +19 -5
- data/lib/paper_trail_diff/analysis_batch.rb +16 -58
- data/lib/paper_trail_diff/approximate_paths.rb +4 -8
- data/lib/paper_trail_diff/association_activity_edge.rb +5 -24
- data/lib/paper_trail_diff/association_activity_selection.rb +10 -18
- data/lib/paper_trail_diff/association_discovery.rb +1 -6
- data/lib/paper_trail_diff/association_key.rb +9 -6
- data/lib/paper_trail_diff/association_traversal.rb +49 -6
- data/lib/paper_trail_diff/batch_boundary_resolver.rb +33 -42
- data/lib/paper_trail_diff/batch_preload.rb +2 -12
- data/lib/paper_trail_diff/batched_root_analyses.rb +72 -0
- data/lib/paper_trail_diff/batched_root_analyzer.rb +16 -14
- data/lib/paper_trail_diff/batched_root_versions.rb +25 -20
- data/lib/paper_trail_diff/branch_snapshot_refresher.rb +27 -31
- data/lib/paper_trail_diff/change_attribution.rb +314 -0
- data/lib/paper_trail_diff/change_row.rb +30 -10
- data/lib/paper_trail_diff/comparison_batch.rb +3 -4
- data/lib/paper_trail_diff/configuration.rb +1 -1
- data/lib/paper_trail_diff/deleted_analysis_batch.rb +22 -37
- data/lib/paper_trail_diff/deleted_root_filter.rb +3 -6
- data/lib/paper_trail_diff/deprecation.rb +4 -43
- data/lib/paper_trail_diff/diagnostics.rb +1 -2
- data/lib/paper_trail_diff/direct_record_state_loader.rb +8 -1
- data/lib/paper_trail_diff/endpoint.rb +78 -44
- data/lib/paper_trail_diff/errors.rb +2 -9
- data/lib/paper_trail_diff/fallback_paths.rb +2 -5
- data/lib/paper_trail_diff/habtm_transaction_owners.rb +5 -3
- data/lib/paper_trail_diff/historical_association_reifier.rb +3 -1
- data/lib/paper_trail_diff/historical_filter.rb +40 -15
- data/lib/paper_trail_diff/historical_snapshot_store.rb +80 -38
- data/lib/paper_trail_diff/instrumentation.rb +0 -8
- data/lib/paper_trail_diff/live_endpoint_batch_loader.rb +11 -7
- data/lib/paper_trail_diff/live_endpoint_provider.rb +7 -23
- data/lib/paper_trail_diff/live_graph_collector.rb +3 -3
- data/lib/paper_trail_diff/nested_comparator.rb +46 -19
- data/lib/paper_trail_diff/net_change.rb +276 -0
- data/lib/paper_trail_diff/own_window.rb +35 -0
- data/lib/paper_trail_diff/paper_trail_adapter.rb +40 -45
- data/lib/paper_trail_diff/prepared_activity_index.rb +170 -42
- data/lib/paper_trail_diff/prepared_child_candidates.rb +58 -0
- data/lib/paper_trail_diff/prepared_edge_loader.rb +43 -107
- data/lib/paper_trail_diff/prepared_history.rb +20 -31
- data/lib/paper_trail_diff/prepared_history_loader.rb +46 -9
- data/lib/paper_trail_diff/prepared_record_index.rb +137 -47
- data/lib/paper_trail_diff/prepared_snapshot_builder.rb +16 -0
- data/lib/paper_trail_diff/prepared_state_association_reader.rb +17 -19
- data/lib/paper_trail_diff/read_snapshot.rb +41 -19
- data/lib/paper_trail_diff/recorded_changes.rb +10 -11
- data/lib/paper_trail_diff/recorded_changes_reader.rb +60 -0
- data/lib/paper_trail_diff/recorded_class.rb +14 -19
- data/lib/paper_trail_diff/recorded_columns.rb +33 -0
- data/lib/paper_trail_diff/root_version_plan.rb +20 -22
- data/lib/paper_trail_diff/root_version_selection.rb +39 -29
- data/lib/paper_trail_diff/scope_condition.rb +7 -17
- data/lib/paper_trail_diff/scope_ranking.rb +16 -22
- data/lib/paper_trail_diff/scoped_analysis.rb +27 -4
- data/lib/paper_trail_diff/scoped_analysis_builder.rb +4 -3
- data/lib/paper_trail_diff/scoped_root_selection.rb +60 -41
- data/lib/paper_trail_diff/snapshot_normalizer.rb +18 -3
- data/lib/paper_trail_diff/stored_json_values.rb +52 -0
- data/lib/paper_trail_diff/stored_time_precision.rb +30 -18
- data/lib/paper_trail_diff/stored_values.rb +20 -0
- data/lib/paper_trail_diff/support.rb +185 -35
- data/lib/paper_trail_diff/time_of_day.rb +39 -0
- data/lib/paper_trail_diff/time_range.rb +37 -20
- data/lib/paper_trail_diff/time_version_range.rb +5 -10
- data/lib/paper_trail_diff/timeline_builder.rb +3 -3
- data/lib/paper_trail_diff/timeline_range.rb +18 -12
- data/lib/paper_trail_diff/timeline_snapshot_provider.rb +1 -1
- data/lib/paper_trail_diff/total_order.rb +55 -0
- data/lib/paper_trail_diff/transaction_cut.rb +192 -91
- data/lib/paper_trail_diff/traversal_preparer.rb +5 -8
- data/lib/paper_trail_diff/value_objects.rb +8 -4
- data/lib/paper_trail_diff/version.rb +1 -1
- data/lib/paper_trail_diff/version_association_candidate_scope.rb +1 -12
- data/lib/paper_trail_diff/version_boundary.rb +37 -21
- data/lib/paper_trail_diff/version_query.rb +25 -25
- data/lib/paper_trail_diff/version_range.rb +12 -16
- data/lib/paper_trail_diff/version_sequence_diagnostics.rb +32 -0
- data/lib/paper_trail_diff/version_table.rb +12 -8
- data/lib/paper_trail_diff.rb +28 -42
- data/sig/generated/paper_trail_diff/activity_belongs_to_event_applier.rbs +0 -8
- data/sig/generated/paper_trail_diff/activity_boundary.rbs +39 -17
- data/sig/generated/paper_trail_diff/activity_child_candidate_loader.rbs +19 -22
- data/sig/generated/paper_trail_diff/activity_collection_event_applier.rbs +0 -5
- data/sig/generated/paper_trail_diff/activity_event.rbs +25 -9
- data/sig/generated/paper_trail_diff/activity_event_record_resolver.rbs +6 -34
- data/sig/generated/paper_trail_diff/activity_event_route_finder.rbs +11 -4
- data/sig/generated/paper_trail_diff/activity_event_snapshot_refresher.rbs +0 -3
- data/sig/generated/paper_trail_diff/activity_history.rbs +54 -19
- data/sig/generated/paper_trail_diff/activity_range.rbs +6 -20
- data/sig/generated/paper_trail_diff/activity_root_snapshot_refresher.rbs +36 -2
- data/sig/generated/paper_trail_diff/activity_root_steps.rbs +6 -6
- data/sig/generated/paper_trail_diff/activity_scope_filter.rbs +51 -0
- data/sig/generated/paper_trail_diff/activity_snapshot_delta.rbs +5 -3
- data/sig/generated/paper_trail_diff/activity_snapshot_sequence.rbs +21 -3
- data/sig/generated/paper_trail_diff/activity_target_candidate_loader.rbs +19 -24
- data/sig/generated/paper_trail_diff/activity_timeline_builder.rbs +121 -127
- data/sig/generated/paper_trail_diff/activity_tree_filter.rbs +41 -8
- data/sig/generated/paper_trail_diff/activity_version_collector.rbs +42 -35
- data/sig/generated/paper_trail_diff/activity_view.rbs +16 -8
- data/sig/generated/paper_trail_diff/analysis.rbs +15 -6
- data/sig/generated/paper_trail_diff/analysis_batch.rbs +6 -24
- data/sig/generated/paper_trail_diff/association_activity_edge.rbs +0 -13
- data/sig/generated/paper_trail_diff/association_activity_selection.rbs +6 -9
- data/sig/generated/paper_trail_diff/association_discovery.rbs +0 -3
- data/sig/generated/paper_trail_diff/association_key.rbs +5 -0
- data/sig/generated/paper_trail_diff/association_traversal.rbs +26 -2
- data/sig/generated/paper_trail_diff/batch_boundary_resolver.rbs +17 -17
- data/sig/generated/paper_trail_diff/batch_preload.rbs +0 -4
- data/sig/generated/paper_trail_diff/batched_root_analyses.rbs +47 -0
- data/sig/generated/paper_trail_diff/batched_root_analyzer.rbs +9 -8
- data/sig/generated/paper_trail_diff/batched_root_versions.rbs +10 -6
- data/sig/generated/paper_trail_diff/branch_snapshot_refresher.rbs +11 -5
- data/sig/generated/paper_trail_diff/change_attribution.rbs +185 -0
- data/sig/generated/paper_trail_diff/change_row.rbs +22 -7
- data/sig/generated/paper_trail_diff/deleted_analysis_batch.rbs +11 -12
- data/sig/generated/paper_trail_diff/deleted_root_filter.rbs +0 -2
- data/sig/generated/paper_trail_diff/deprecation.rbs +4 -24
- data/sig/generated/paper_trail_diff/direct_record_state_loader.rbs +4 -0
- data/sig/generated/paper_trail_diff/endpoint.rbs +40 -16
- data/sig/generated/paper_trail_diff/errors.rbs +2 -11
- data/sig/generated/paper_trail_diff/habtm_transaction_owners.rbs +2 -1
- data/sig/generated/paper_trail_diff/historical_filter.rbs +20 -9
- data/sig/generated/paper_trail_diff/historical_snapshot_store.rbs +40 -10
- data/sig/generated/paper_trail_diff/instrumentation.rbs +0 -3
- data/sig/generated/paper_trail_diff/live_endpoint_batch_loader.rbs +3 -0
- data/sig/generated/paper_trail_diff/live_endpoint_provider.rbs +5 -8
- data/sig/generated/paper_trail_diff/live_graph_collector.rbs +2 -2
- data/sig/generated/paper_trail_diff/nested_comparator.rbs +20 -4
- data/sig/generated/paper_trail_diff/net_change.rbs +168 -0
- data/sig/generated/paper_trail_diff/own_window.rbs +22 -0
- data/sig/generated/paper_trail_diff/paper_trail_adapter.rbs +9 -6
- data/sig/generated/paper_trail_diff/prepared_activity_index.rbs +103 -14
- data/sig/generated/paper_trail_diff/prepared_child_candidates.rbs +39 -0
- data/sig/generated/paper_trail_diff/prepared_edge_loader.rbs +18 -35
- data/sig/generated/paper_trail_diff/prepared_history.rbs +9 -12
- data/sig/generated/paper_trail_diff/prepared_history_loader.rbs +28 -3
- data/sig/generated/paper_trail_diff/prepared_record_index.rbs +68 -14
- data/sig/generated/paper_trail_diff/prepared_snapshot_builder.rbs +12 -0
- data/sig/generated/paper_trail_diff/prepared_state_association_reader.rbs +6 -7
- data/sig/generated/paper_trail_diff/read_snapshot.rbs +27 -8
- data/sig/generated/paper_trail_diff/recorded_changes.rbs +4 -2
- data/sig/generated/paper_trail_diff/recorded_changes_reader.rbs +41 -0
- data/sig/generated/paper_trail_diff/recorded_class.rbs +7 -6
- data/sig/generated/paper_trail_diff/recorded_columns.rbs +21 -0
- data/sig/generated/paper_trail_diff/root_version_plan.rbs +8 -4
- data/sig/generated/paper_trail_diff/root_version_selection.rbs +27 -13
- data/sig/generated/paper_trail_diff/scope_condition.rbs +0 -2
- data/sig/generated/paper_trail_diff/scope_ranking.rbs +10 -10
- data/sig/generated/paper_trail_diff/scoped_analysis.rbs +19 -0
- data/sig/generated/paper_trail_diff/scoped_analysis_builder.rbs +2 -2
- data/sig/generated/paper_trail_diff/scoped_root_selection.rbs +37 -14
- data/sig/generated/paper_trail_diff/snapshot_normalizer.rbs +7 -0
- data/sig/generated/paper_trail_diff/stored_json_values.rbs +35 -0
- data/sig/generated/paper_trail_diff/stored_time_precision.rbs +19 -5
- data/sig/generated/paper_trail_diff/stored_values.rbs +13 -0
- data/sig/generated/paper_trail_diff/support.rbs +120 -23
- data/sig/generated/paper_trail_diff/time_of_day.rbs +21 -0
- data/sig/generated/paper_trail_diff/time_range.rbs +15 -8
- data/sig/generated/paper_trail_diff/time_version_range.rbs +2 -5
- data/sig/generated/paper_trail_diff/timeline_builder.rbs +2 -2
- data/sig/generated/paper_trail_diff/timeline_range.rbs +12 -7
- data/sig/generated/paper_trail_diff/total_order.rbs +32 -0
- data/sig/generated/paper_trail_diff/transaction_cut.rbs +111 -52
- data/sig/generated/paper_trail_diff/traversal_preparer.rbs +2 -2
- data/sig/generated/paper_trail_diff/value_objects.rbs +5 -1
- data/sig/generated/paper_trail_diff/version_association_candidate_scope.rbs +0 -7
- data/sig/generated/paper_trail_diff/version_boundary.rbs +15 -8
- data/sig/generated/paper_trail_diff/version_query.rbs +17 -10
- data/sig/generated/paper_trail_diff/version_range.rbs +6 -9
- data/sig/generated/paper_trail_diff/version_sequence_diagnostics.rbs +13 -0
- data/sig/generated/paper_trail_diff/version_table.rbs +3 -0
- data/sig/generated/paper_trail_diff.rbs +16 -16
- metadata +28 -6
- data/lib/paper_trail_diff/preloaded_endpoint_batch_loader.rb +0 -60
- data/sig/generated/paper_trail_diff/preloaded_endpoint_batch_loader.rbs +0 -32
data/docs/architecture.md
CHANGED
|
@@ -125,8 +125,10 @@ Keep PT-AT's own index on `version_associations` (`foreign_key_name`,
|
|
|
125
125
|
- Repeated events in one very wide collection still copy that collection's
|
|
126
126
|
list of records for each step. The comparison work stays small, but memory
|
|
127
127
|
traffic grows with *events × collection size*.
|
|
128
|
-
- With PT-AT installed, a `within:` or `since:` window
|
|
129
|
-
versions
|
|
130
|
-
transaction (see [time windows](reference.md#time-windows)).
|
|
131
|
-
|
|
132
|
-
|
|
128
|
+
- With PT-AT installed, a `within:` or `since:` window reads each root's
|
|
129
|
+
versions nearest each edge, to move an edge that falls inside a
|
|
130
|
+
transaction (see [time windows](reference.md#time-windows)). Each edge
|
|
131
|
+
takes one statement per 250 roots and one or two per 500 transactions they
|
|
132
|
+
find, whatever the length of their histories; a `report` window has no end,
|
|
133
|
+
so only its start is read. See
|
|
134
|
+
[performance](performance.md) for the cost on a large table.
|
data/docs/performance.md
CHANGED
|
@@ -18,24 +18,26 @@ for one association, at 1, 5, and 20 records:
|
|
|
18
18
|
|
|
19
19
|
| Association | Queries | `reify` calls |
|
|
20
20
|
| --- | --- | --- |
|
|
21
|
-
| None |
|
|
22
|
-
| `has_many` |
|
|
23
|
-
| `belongs_to` |
|
|
24
|
-
| A `has_many` scope of plain conditions, or a default scope |
|
|
25
|
-
| `has_many :through` |
|
|
26
|
-
| Polymorphic `belongs_to` |
|
|
27
|
-
| `has_many :through` with `source_type:` |
|
|
28
|
-
| A `has_many` with `order(:id).limit(1)` |
|
|
29
|
-
| A scope that takes the owner, read with PT-AT |
|
|
30
|
-
| None, with a querying `after_initialize` |
|
|
31
|
-
| None, with a callable `historical_filter:` that queries |
|
|
21
|
+
| None | 6 / 6 / 6 | 0 |
|
|
22
|
+
| `has_many` | 13 / 13 / 13 | 0 |
|
|
23
|
+
| `belongs_to` | 13 / 13 / 13 | 0 |
|
|
24
|
+
| A `has_many` scope of plain conditions, or a default scope | 13 / 13 / 13 | 0 |
|
|
25
|
+
| `has_many :through` | 19 / 19 / 19 | 0 |
|
|
26
|
+
| Polymorphic `belongs_to` | 17 / 17 / 17 | 0 |
|
|
27
|
+
| `has_many :through` with `source_type:` | 21 / 21 / 21 | 0 |
|
|
28
|
+
| A `has_many` with `order(:id).limit(1)` | 14 / 18 / 33 | 0 |
|
|
29
|
+
| A scope that takes the owner, read with PT-AT | 25 / 73 / 253 | 9 / 45 / 180 |
|
|
30
|
+
| None, with a querying `after_initialize` | 10 / 26 / 86 | 2 / 10 / 40 |
|
|
31
|
+
| None, with a callable `historical_filter:` that queries | 11 / 15 / 30 | 2 / 10 / 40 |
|
|
32
32
|
|
|
33
33
|
The last four rows grow with the number of records, for the reasons in
|
|
34
34
|
[what costs a query per record](#what-costs-a-query-per-record).
|
|
35
35
|
|
|
36
36
|
The counts leave out transaction statements. On PostgreSQL and MySQL each call
|
|
37
37
|
also runs in one snapshot transaction, which adds two or three statements per
|
|
38
|
-
call whatever its size.
|
|
38
|
+
call whatever its size. On SQL Server a call first checks whether the
|
|
39
|
+
database allows snapshot isolation, in one statement, and when it does, runs
|
|
40
|
+
in a snapshot transaction too. See [limitations](reference.md#limitations).
|
|
39
41
|
|
|
40
42
|
## What still grows
|
|
41
43
|
|
|
@@ -44,24 +46,53 @@ Flat query counts do not make a report's cost flat:
|
|
|
44
46
|
- **Ruby work grows with the history you report on.** Every version in the
|
|
45
47
|
window is decoded and compared: an `analyze_scope` of articles with five
|
|
46
48
|
updates each took about 1.5 ms per article, and one article with 5,000
|
|
47
|
-
updates took 1.2 s.
|
|
49
|
+
updates took 1.2 s. A `report` of 1,000 articles with their comments, from
|
|
50
|
+
a table of 114,000 versions, took about 4 ms an article on SQLite,
|
|
51
|
+
PostgreSQL, and MySQL, and 5.5 ms on SQL Server, most of it in Ruby.
|
|
52
|
+
- **Versions stored as JSON read about a third faster than YAML.** A `report`
|
|
53
|
+
on 500 articles with 31 versions each took 6.8 s with PaperTrail's default
|
|
54
|
+
YAML serializer and 4.5 s with its JSON one, or 9.8 s and 6.7 s with their
|
|
55
|
+
comments, and peaked at up to a fifth less memory. JSON keeps times
|
|
56
|
+
to milliseconds and stores a few values differently (see
|
|
57
|
+
[history stored as JSON](reference.md#history-stored-as-json)). PaperTrail
|
|
58
|
+
reads every stored version with the one serializer, so switching means
|
|
59
|
+
converting the versions already saved.
|
|
48
60
|
- **Memory grows with the number of records.** A `report` of 1,000 articles
|
|
49
61
|
with their comments, replies, and author peaked at about 100 KB per
|
|
50
62
|
article. `limit:` is a safety ceiling, not a page size, so to bound memory,
|
|
51
63
|
page with the relation (`Article.where(...).order(:id).limit(200)`), or
|
|
52
64
|
report on shorter windows.
|
|
53
|
-
- **
|
|
54
|
-
|
|
55
|
-
activity timeline over `n` comments that changed `m` times does about
|
|
56
|
-
`n × m` work: 800 comments edited 1,000 times took about 3 s.
|
|
57
|
-
|
|
65
|
+
- **Work on a wide collection grows with its size times its history.** Each
|
|
66
|
+
step of an activity timeline copies the collection's list of records, so
|
|
67
|
+
an activity timeline over `n` comments that changed `m` times does about
|
|
68
|
+
`n × m` work: 800 comments edited 1,000 times took about 3 s. Members
|
|
69
|
+
saved in one transaction share one snapshot: a `report` on 100 articles,
|
|
70
|
+
each created with 100 comments in one transaction and edited with all of
|
|
71
|
+
them in another, took about 4.6 s. `timeline` reads every member at each
|
|
72
|
+
version of the root instead: 300 comments over 2,001 versions of their
|
|
73
|
+
article took about 7.5 s.
|
|
58
74
|
- **`snapshots: true` and `recorded_changes: true` cost memory and time.** Ask
|
|
59
|
-
for them only when you use them.
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
75
|
+
for them only when you use them. Records that did not change are shared
|
|
76
|
+
between steps, so the cost is modest: an activity `report` of 10,000
|
|
77
|
+
articles with their 57,000 comments (114,000 versions, 49,000 steps) took
|
|
78
|
+
about 40 s and held 372 MB, and 411 MB with snapshots, at the same
|
|
79
|
+
time within run-to-run noise. Folding its rows with `net_changes` took
|
|
80
|
+
another 17 s, and listing them with `change_rows` 13 s.
|
|
81
|
+
- **Each edge of a window takes one statement per 250 records, and one or two
|
|
82
|
+
per 500 transactions it finds.** A
|
|
83
|
+
window from `within:` or `since:` looks for transactions that cross either
|
|
84
|
+
edge, and with associations also for those that saved a record before it
|
|
85
|
+
and an associated record after it. A `report` window has no end, so only
|
|
86
|
+
its start is read. With associations, it also takes one statement per 500
|
|
87
|
+
of the transactions its history reads, to find one that saved associated
|
|
88
|
+
records on both sides of its start; when it finds one, it reads the
|
|
89
|
+
window's history again from where that transaction began. On 1.4 million
|
|
90
|
+
versions, both edges of a window over 10,800 records took about 6 s on SQL
|
|
91
|
+
Server and 1.5 s on PostgreSQL and MySQL.
|
|
92
|
+
- **`version_scope:` on activity reads every version in the range**, as
|
|
93
|
+
activity without one does. With associations, it takes one more statement
|
|
94
|
+
per 500 of the transactions its history reads, to find those the filter
|
|
95
|
+
keeps a version of.
|
|
65
96
|
|
|
66
97
|
## Database indexes
|
|
67
98
|
|
|
@@ -82,8 +113,6 @@ explains why.
|
|
|
82
113
|
- **A limited association on the current record.** Active Record's `preload`
|
|
83
114
|
applies a `limit` or `offset` across all the records it loads, so each
|
|
84
115
|
record's limited association is loaded on its own: one query per record.
|
|
85
|
-
The deprecated `reload_live_endpoints: false` does the same for one you
|
|
86
|
-
preloaded.
|
|
87
116
|
- **Your own code.** Model callbacks that query, such as an `after_initialize`,
|
|
88
117
|
custom attribute types, and a callable `historical_filter:` run for each
|
|
89
118
|
state they are given.
|
|
@@ -98,7 +127,16 @@ queries. `max_filter_versions:` (10,000 by default) caps the versions it loads.
|
|
|
98
127
|
Either way, every version of every candidate in the window is decoded: with
|
|
99
128
|
1,000 candidates and no match, a hash and a callable each took about 0.8 s.
|
|
100
129
|
Finding the relation's records among the candidates takes up to two queries
|
|
101
|
-
per
|
|
130
|
+
per 500 of them, and with PT-AT, finding where their windows move takes one
|
|
131
|
+
statement per 250 of them and one or two per 500 transactions, for each edge.
|
|
132
|
+
|
|
133
|
+
The relation then reads the records that matched in pages of up to 2,000, and
|
|
134
|
+
each page lists every matching id, so the relation keeps its own order,
|
|
135
|
+
`limit`, and `offset`. A version table whose `item_id` does not share the key's
|
|
136
|
+
type selects records the same way. SQL Server compiles each such statement
|
|
137
|
+
afresh: with 10,000 matching records, each page took about 0.7 s, 5 s of a
|
|
138
|
+
12 s report, and with about 20,000 a page can outlast the adapter's default
|
|
139
|
+
timeout of 5 seconds. Narrow the window, or page with the relation.
|
|
102
140
|
|
|
103
141
|
## Large reports
|
|
104
142
|
|
|
@@ -121,16 +159,26 @@ mise exec -- bundle exec ruby benchmark/reporting.rb --case many_roots --samples
|
|
|
121
159
|
one `report` makes for a scenario and a number of records; the table above
|
|
122
160
|
comes from its scenarios. `PTD_PROBE_ALLOW_APPROXIMATE=1` passes
|
|
123
161
|
`allow_approximate: true`.
|
|
124
|
-
`reporting.rb` times `analyze_scope` over articles with no associations,
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
162
|
+
`reporting.rb` times `analyze_scope` over articles with no associations and,
|
|
163
|
+
as `association_window`, a `report` over articles with their comments read
|
|
164
|
+
through PT-AT, each in a fresh process, and records memory; `--output` keeps
|
|
165
|
+
the raw samples. Both use an in-memory SQLite database, never your
|
|
166
|
+
application's, and read time from the [test clock](testing.md#test-clock), so
|
|
167
|
+
a host clock that steps backwards cannot reorder the versions they seed.
|
|
168
|
+
Compare counts as well as times, and run the correctness suites separately.
|
|
169
|
+
No CI job checks timings, but one spec bounds how a report's SQL grows with
|
|
170
|
+
its window (see [SQL work](testing.md#sql-work)).
|
|
171
|
+
|
|
172
|
+
The query counts in the first table and the `timeline` of 300 comments were
|
|
173
|
+
measured on 2026-11-03, the counts after `report`'s window lost its end; the
|
|
174
|
+
serializers and the snapshot and net change costs on 2026-11-04; the other
|
|
175
|
+
numbers on 2026-10-30 at revision `6c3d30b`. Both used Ruby 4.0.1, Active
|
|
176
|
+
Record 8.1.3.1, PaperTrail 17.0.0, and SQLite 3.53.2, on WSL2 on an AMD
|
|
177
|
+
Ryzen 7 7800X3D. The per-database `report` costs were measured on 2026-11-04
|
|
178
|
+
with Ruby 4.0.9 in Docker, on SQLite, PostgreSQL 17.11, MySQL 8.4.13, and SQL
|
|
179
|
+
Server 2022, each holding the same rows. The memory, wide-collection,
|
|
180
|
+
serializer, snapshot, per-database, and index figures come from one-off
|
|
181
|
+
scripts rather than these two. The benchmark's
|
|
182
|
+
raw samples are in
|
|
135
183
|
[`benchmark/results/reporting-2026-10-30.json`](../benchmark/results/reporting-2026-10-30.json),
|
|
136
184
|
next to earlier runs.
|