upkeep-rails 0.2.5-aarch64-linux-gnu

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (81) hide show
  1. checksums.yaml +7 -0
  2. data/LICENSE.txt +21 -0
  3. data/README.md +244 -0
  4. data/docs/drafts/turbo-streams-is-cache-invalidation-you-write-by-hand.md +240 -0
  5. data/docs/how-it-works.md +329 -0
  6. data/docs/plans/sqlglot-active-record-migration.md +211 -0
  7. data/docs/plans/turbo-frame-subscription-composition.md +90 -0
  8. data/docs/spikes/SQLGLOT_VS_AREL_FINDINGS.md +136 -0
  9. data/docs/spikes/query-source-analysis-comparison/FINDINGS.md +169 -0
  10. data/docs/spikes/sqlglot-first-active-record/README.md +26 -0
  11. data/docs/spikes/sqlglot-query-analysis/FINDINGS.md +110 -0
  12. data/docs/spikes/sqlglot-query-analysis/PULSE_1802_FINDINGS.md +95 -0
  13. data/docs/spikes/sqlglot-semantic-bindings/README.md +29 -0
  14. data/lib/generators/upkeep/install/install_generator.rb +192 -0
  15. data/lib/generators/upkeep/install/templates/create_upkeep_subscriptions.rb.erb +49 -0
  16. data/lib/generators/upkeep/install/templates/subscription.js +288 -0
  17. data/lib/generators/upkeep/install/templates/upkeep.rb +65 -0
  18. data/lib/upkeep/active_record_query.rb +248 -0
  19. data/lib/upkeep/capture/request.rb +150 -0
  20. data/lib/upkeep/dag/subscription_shape.rb +244 -0
  21. data/lib/upkeep/dag.rb +454 -0
  22. data/lib/upkeep/delivery/action_cable_adapter.rb +48 -0
  23. data/lib/upkeep/delivery/async_dispatcher.rb +102 -0
  24. data/lib/upkeep/delivery/broadcast_transport.rb +89 -0
  25. data/lib/upkeep/delivery/transport.rb +194 -0
  26. data/lib/upkeep/delivery/turbo_streams.rb +339 -0
  27. data/lib/upkeep/delivery.rb +7 -0
  28. data/lib/upkeep/dependencies.rb +600 -0
  29. data/lib/upkeep/herb/developer_report.rb +135 -0
  30. data/lib/upkeep/herb/manifest_cache.rb +83 -0
  31. data/lib/upkeep/herb/manifest_diff.rb +183 -0
  32. data/lib/upkeep/herb/source_instrumenter.rb +149 -0
  33. data/lib/upkeep/herb/template_manifest.rb +548 -0
  34. data/lib/upkeep/invalidation/collection_append.rb +84 -0
  35. data/lib/upkeep/invalidation/collection_member_replace.rb +78 -0
  36. data/lib/upkeep/invalidation/collection_prepend.rb +84 -0
  37. data/lib/upkeep/invalidation/collection_remove.rb +57 -0
  38. data/lib/upkeep/invalidation/planner.rb +411 -0
  39. data/lib/upkeep/invalidation.rb +7 -0
  40. data/lib/upkeep/rails/action_view_capture.rb +1007 -0
  41. data/lib/upkeep/rails/activation_token.rb +55 -0
  42. data/lib/upkeep/rails/cable/channel.rb +165 -0
  43. data/lib/upkeep/rails/cable/subscriber_identity.rb +361 -0
  44. data/lib/upkeep/rails/cable.rb +4 -0
  45. data/lib/upkeep/rails/client_subscription.rb +65 -0
  46. data/lib/upkeep/rails/cluster_guard.rb +57 -0
  47. data/lib/upkeep/rails/configuration.rb +252 -0
  48. data/lib/upkeep/rails/controller_runtime.rb +187 -0
  49. data/lib/upkeep/rails/install.rb +28 -0
  50. data/lib/upkeep/rails/job_runtime.rb +43 -0
  51. data/lib/upkeep/rails/railtie.rb +44 -0
  52. data/lib/upkeep/rails/replay.rb +244 -0
  53. data/lib/upkeep/rails/testing.rb +259 -0
  54. data/lib/upkeep/rails.rb +466 -0
  55. data/lib/upkeep/replay.rb +462 -0
  56. data/lib/upkeep/runtime.rb +1276 -0
  57. data/lib/upkeep/shared_streams.rb +86 -0
  58. data/lib/upkeep/sql_dependency_analysis.rb +553 -0
  59. data/lib/upkeep/sqlglot/libsqlglot_rust.so +0 -0
  60. data/lib/upkeep/sqlglot/native.rb +121 -0
  61. data/lib/upkeep/sqlglot/native_library.rb +23 -0
  62. data/lib/upkeep/sqlglot.rb +367 -0
  63. data/lib/upkeep/subscriptions/active_record_store.rb +398 -0
  64. data/lib/upkeep/subscriptions/active_record_subscription_persistence.rb +411 -0
  65. data/lib/upkeep/subscriptions/active_registry.rb +80 -0
  66. data/lib/upkeep/subscriptions/base_store.rb +110 -0
  67. data/lib/upkeep/subscriptions/json_snapshot.rb +98 -0
  68. data/lib/upkeep/subscriptions/layered_reverse_index.rb +125 -0
  69. data/lib/upkeep/subscriptions/lookup_instrumentation.rb +32 -0
  70. data/lib/upkeep/subscriptions/persistent_reverse_index.rb +228 -0
  71. data/lib/upkeep/subscriptions/registrar.rb +36 -0
  72. data/lib/upkeep/subscriptions/reverse_index.rb +313 -0
  73. data/lib/upkeep/subscriptions/shape.rb +117 -0
  74. data/lib/upkeep/subscriptions/store.rb +349 -0
  75. data/lib/upkeep/subscriptions.rb +7 -0
  76. data/lib/upkeep/targeting.rb +146 -0
  77. data/lib/upkeep/version.rb +5 -0
  78. data/lib/upkeep-rails.rb +3 -0
  79. data/lib/upkeep.rb +15 -0
  80. data/upkeep-rails.gemspec +66 -0
  81. metadata +327 -0
@@ -0,0 +1,329 @@
1
+ # How Upkeep Works
2
+
3
+ This document explains the runtime model behind Upkeep Rails. It intentionally
4
+ does not cover installation or common configuration; use the
5
+ [README](../README.md) for that public API surface.
6
+
7
+ ## Rendered Pages
8
+
9
+ A rendered page is a successful HTML GET that Upkeep can keep fresh. The
10
+ request runs normally through Rails. Upkeep observes the controller, Action View
11
+ rendering, Active Record reads, request inputs, and identity inputs used by the
12
+ response.
13
+
14
+ Upkeep only captures successful HTML responses. Non-HTML responses, redirects,
15
+ failed responses, and explicit non-page interactions continue to behave like
16
+ ordinary Rails responses.
17
+
18
+ ## Frames
19
+
20
+ A frame is a rendered page, template, partial, collection render site, or
21
+ fragment with a stable delivery target.
22
+
23
+ Frames let Upkeep refresh a specific part of the page instead of replaying the
24
+ whole response when a narrower update is proven safe. A page frame is the broad
25
+ fallback. A render-site or fragment frame is narrower.
26
+
27
+ Upkeep instruments Action View templates and adds internal `data-upkeep-*`
28
+ markers for page roots, fragment roots, and safe collection render-site
29
+ containers. Normal templates do not need to call helper APIs directly.
30
+
31
+ The `upkeep_frame` helper is an advanced escape hatch for generated or
32
+ helper-built boundaries that cannot be derived from template source. Ordinary
33
+ ERB and partial collections should not need it.
34
+
35
+ ### Controller Work and Render Regions
36
+
37
+ Dependencies inherit the frame that is active when the read occurs. A query
38
+ inside a partial or `upkeep_frame` belongs to that render region. A query in a
39
+ controller callback or action runs before rendering, so its narrowest sound
40
+ boundary is the page:
41
+
42
+ ```ruby
43
+ before_action do
44
+ @show_reminder = !current_user.time_logs.where(log_date: Date.current).exists?
45
+ end
46
+ ```
47
+
48
+ Rails does not record which later template expression consumes an instance
49
+ variable. Ruby may also pass or access that value dynamically, so Upkeep cannot
50
+ reliably infer that `@show_reminder` affects only one element. A matching
51
+ `TimeLog` change can therefore select a page replay.
52
+
53
+ When controller work exists only to render one section, execute it inside that
54
+ section instead:
55
+
56
+ ```erb
57
+ <%= upkeep_frame "layout/time-log-reminder" do %>
58
+ <div data-upkeep-render-site="layout/time-log-reminder">
59
+ <% if time_log_reminder_data %>
60
+ <%= render "shared/time_log_reminder" %>
61
+ <% end %>
62
+ </div>
63
+ <% end %>
64
+ ```
65
+
66
+ The helper or partial can perform the query while this frame is active. Upkeep
67
+ then associates the dependency with the reminder region instead of the page.
68
+ The region must always render a stable target, including when its conditional
69
+ content is empty, so it can transition in either direction.
70
+
71
+ This is a granularity rule, not a refusal: controller-level reads remain live
72
+ through a conservative page replay. Use an explicit region only when that
73
+ broader replay is undesirable and the application knows the true presentation
74
+ boundary.
75
+
76
+ ## Surfaces
77
+
78
+ A surface is the set of facts about future writes that would make a frame
79
+ stale.
80
+
81
+ For Active Record, Upkeep derives surfaces from observed record attributes,
82
+ rendered collections, and relation shape where Rails exposes structural Arel
83
+ queries. A rendered collection of open cards ordered by position produces a
84
+ surface tied to the cards table, the columns that decide membership and order,
85
+ and the records rendered in that collection.
86
+
87
+ When a write commits, Upkeep compares the write facts with registered surfaces.
88
+ Only frames whose surfaces can be affected are selected for delivery.
89
+
90
+ Controller requests and Active Job executions both establish a change-capture
91
+ boundary. After the boundary completes, Upkeep plans and delivers updates for
92
+ the committed Active Record changes. Sidekiq jobs using the Active Job adapter
93
+ are covered by this lifecycle; direct Sidekiq workers are not.
94
+
95
+ ## Identity Boundaries
96
+
97
+ An identity boundary is state that decides who may receive a live update.
98
+
99
+ Upkeep records observed CurrentAttributes, Warden, session, cookie, and request
100
+ reads for replay and sharing. It does not infer subscriber identity by naming
101
+ convention. The app declares which render-time value maps to which
102
+ subscribe-time ActionCable value.
103
+
104
+ The safety rule is simple: if rendered output depends on a non-public identity,
105
+ only subscribers proving the same identity may receive that output. If Upkeep
106
+ cannot identify the boundary, it refuses live registration rather than sending
107
+ viewer-specific HTML to the wrong browser.
108
+
109
+ Absent identities are public. For example, if a logged-out page reads a nil
110
+ viewer, that nil value can be treated as anonymous-public instead of
111
+ subscriber-specific.
112
+
113
+ ## Subscriptions
114
+
115
+ A subscription is the browser's live connection back to the captured page.
116
+
117
+ Upkeep injects a body-scoped `<upkeep-subscription-source>` marker into
118
+ successful HTML responses. The generated browser bootstrap upgrades that marker
119
+ into a Turbo stream source, subscribes over ActionCable, and lets Turbo process
120
+ received stream payloads.
121
+
122
+ The server stores a replayable subscription graph for the rendered page. The
123
+ graph contains frames, dependencies, target metadata, replay recipes, request
124
+ inputs, and identity information needed to plan later updates.
125
+
126
+ Turbo Frames are graph scopes inside that page subscription. A frame response
127
+ replaces the matching scope while the shell and sibling scopes remain intact.
128
+ The browser connects the composed subscription before disconnecting the
129
+ previous one, so a frame visit never leaves the visible page without its full
130
+ dependency graph.
131
+
132
+ ## Proven Delivery
133
+
134
+ Proven delivery means Upkeep only emits the narrowest Turbo operation it can
135
+ justify.
136
+
137
+ Depending on the proof available, delivery may use:
138
+
139
+ - `append`
140
+ - `prepend`
141
+ - `remove`
142
+ - `replace`
143
+ - `update`
144
+ - Turbo page `refresh`
145
+
146
+ Render-site replays use Turbo Stream `update method="morph"` against the real
147
+ HTML element Upkeep marked as the render site. The stream template is the
148
+ render site's children, so `update` preserves the legal container element and
149
+ swaps its contents.
150
+
151
+ Page-level fallbacks use Turbo Stream `refresh method="morph"
152
+ scroll="preserve"` instead of replacing `<html>` or writing a new document from
153
+ JavaScript.
154
+
155
+ When a change was committed while handling a GET or HEAD request, the refresh
156
+ tag also carries that request's Turbo id as `request-id` (from
157
+ `Turbo.current_request_id` or the `X-Turbo-Request-Id` header). Turbo 8's client
158
+ ignores refreshes for its own recent requests, so view tracking cannot refresh
159
+ the viewer who caused it into a self-refresh loop. Mutations omit the request id
160
+ so the originating tab still refreshes when its response does not render every
161
+ affected region. Writes from jobs or the console also refresh everyone.
162
+
163
+ ## Deoptimization
164
+
165
+ A deoptimization means Upkeep can still prove correctness, but not the cheapest
166
+ operation.
167
+
168
+ For example, a collection member update might not have enough proof for a
169
+ single member `replace`, but the enclosing render site might still be safe to
170
+ rerender. In that case, the page remains live and Upkeep falls back to the
171
+ broader proven target.
172
+
173
+ Planning and delivery telemetry record deoptimization reasons so benchmarks and
174
+ tests can separate safety fallbacks from true refusals.
175
+
176
+ ## Refused Boundaries
177
+
178
+ A refused boundary means Upkeep cannot prove correctness.
179
+
180
+ If Upkeep cannot answer which future write facts can make a rendered result
181
+ stale, which target can be replayed or patched, or which identity inputs decide
182
+ sharing, it refuses the live boundary.
183
+
184
+ This is intentional. A boundary that cannot be proven should behave like
185
+ ordinary Rails HTML instead of registering a broad or unsafe live dependency.
186
+
187
+ Refusal is different from deoptimization:
188
+
189
+ - refusal: Upkeep cannot prove correctness, so the boundary is not live
190
+ - deoptimization: Upkeep can prove correctness through a broader target, so the
191
+ boundary remains live
192
+
193
+ ## What Upkeep Observes
194
+
195
+ Render structure:
196
+
197
+ - Rails-resolved page templates
198
+ - partial and object partial renders
199
+ - Action View-instrumented collection render sites and child fragments
200
+ - polymorphic `render @records` collection shorthand when runtime rendering
201
+ confirms a collection
202
+ - `tag.*` and `content_tag` containers lowered by Herb into ordinary template
203
+ structure
204
+ - single-root fragment targets and legal render-site container targets
205
+
206
+ Template parsing:
207
+
208
+ - Upkeep plans narrow source-derived targets only from templates that pass
209
+ Herb's strict parser.
210
+ - If strict parsing fails but Herb can recover with `strict: false`, Upkeep
211
+ reports the strict parser diagnostics as warnings and may still add broad
212
+ page or fragment root markers.
213
+ - Recovered render sites are diagnostic only. Fix strict warnings before
214
+ expecting narrow collection updates from that template.
215
+
216
+ Data dependencies:
217
+
218
+ - Active Record attribute reads
219
+ - Active Record relation collection renders
220
+ - Active Record callback writes
221
+ - supported bulk `update_all` and `delete_all` writes
222
+ - relation table and column coverage derived from Arel where Rails exposes a
223
+ structural query shape
224
+
225
+ Identity and ambient inputs:
226
+
227
+ - `ActiveSupport::CurrentAttributes` reads
228
+ - Warden and Devise user reads through Warden
229
+ - session and cookie reads
230
+ - request values such as host, path, params, user agent, and remote IP
231
+ - declared Upkeep identities that map observed render-time values to
232
+ ActionCable subscribe-time values
233
+
234
+ ## What Upkeep Cannot Capture
235
+
236
+ Upkeep captures reactive facts, not arbitrary Ruby execution. A boundary is
237
+ capturable only when Upkeep can prove the future write facts that affect it,
238
+ the target that can be replayed or patched, and the identity inputs that decide
239
+ whether it can be shared.
240
+
241
+ These surfaces are not capturable today:
242
+
243
+ | Surface | Why it is not capturable | Runtime behavior |
244
+ | --- | --- | --- |
245
+ | Opaque Active Record relations: raw SQL predicates, raw joins, raw `from` sources, unknown table aliases, opaque order expressions, or opaque pluck columns. | Rails no longer exposes enough structure to prove table, column, predicate, and lifecycle coverage. | Upkeep refuses the live boundary instead of widening to an unsafe dependency. |
246
+ | Controller queries that are never rendered as a collection boundary. | There is no DOM collection surface where membership can be appended, removed, prepended, or replaced. | The page can still render normally. Scalar relation output may be tracked as a page-level dependency, but it does not unlock collection stream planning. |
247
+ | Reads from external stores or process state: Redis, HTTP APIs, files, global variables, class variables, singleton caches, background thread state, or service memoization. | Active Record commit facts cannot select these reads, and Upkeep has no source adapter for their lifecycle. | They are not live dependencies. If another observed dependency causes a replay, normal Rails code may read the new value during that replay. |
248
+ | Writes outside observed Active Record paths: direct connection SQL, writes in another datastore, or side effects that do not emit Upkeep change facts. | Upkeep cannot match a future change to an existing surface without a write fact. | No refresh is scheduled from that write. |
249
+ | Replay inputs that cannot be rebuilt: arbitrary objects, procs, IO handles, open clients, or values that only exist in one Ruby process. | A captured target must be replayable later, often in a different request context. | Non-replayable values block the narrow replay path until represented as stable data. |
250
+ | Patch targets Upkeep cannot identify in rendered HTML. | Delivery needs a stable page, render-site, fragment, or member target. | Upkeep uses the narrowest proven target. If no safe target exists, the boundary is refused. |
251
+
252
+ ## Query Shapes
253
+
254
+ Collection dependencies are accepted only with proven column coverage. Opaque
255
+ predicates or table-only sources are refused instead of widening into broad
256
+ invalidation.
257
+
258
+ Controller materialization is supported when the rendered value keeps a
259
+ structural relation proof:
260
+
261
+ ```ruby
262
+ def index
263
+ @cards = Card.where(status: "open").order(:position).to_a
264
+ end
265
+ ```
266
+
267
+ ```erb
268
+ <%= render partial: "cards/card", collection: @cards, as: :card %>
269
+ ```
270
+
271
+ Upkeep attaches the collection dependency to the rendered collection boundary,
272
+ not to every controller query. A materialized relation that is never rendered as
273
+ a collection is not a lifecycle dependency by itself.
274
+
275
+ Scalar relation output is tracked as a page-level query dependency:
276
+
277
+ ```ruby
278
+ @tag_names = Tag.where(active: true).pluck(:name)
279
+ ```
280
+
281
+ Simple plucked columns are live and can select a page replay when they change.
282
+ They are not collection dependencies, so they do not participate in
283
+ append/remove/prepend planning.
284
+
285
+ ## Testing Model
286
+
287
+ Use `Upkeep::Rails::Testing` for app-level assertions around subscription
288
+ registration and delivery.
289
+
290
+ Structure tests around behavior, not store internals:
291
+
292
+ - Most request and system tests can run against the memory store. Memory has
293
+ the same public lifecycle as ActiveRecord: registration is fetchable
294
+ immediately, lookup visibility starts on activation, touch updates liveness,
295
+ unregister and prune remove lookup entries, and delivery uses the same
296
+ planner surface.
297
+ - Keep a smaller ActiveRecord-backed integration slice for production-only
298
+ concerns: generated migration shape, schema validation, durable rows,
299
+ reload and rehydration, async persistence, and cross-process lookup.
300
+ - Do not assert implementation details that are unique to one store unless the
301
+ test is explicitly about that implementation. For app behavior, assert the
302
+ marker, activation, streams, broadcasts, and rendered bytes.
303
+
304
+ Useful helpers:
305
+
306
+ - `assert_upkeep_subscription_registered`
307
+ - `upkeep_subscription`
308
+ - `upkeep_stream_names`
309
+ - `activate_upkeep_subscription!`
310
+ - `capture_upkeep_broadcasts`
311
+ - `drain_upkeep_delivery!`
312
+ - `capture_upkeep_change_facts`
313
+ - `upkeep_match_report`
314
+
315
+ Use `capture_upkeep_broadcasts` when an app test needs to assert rendered
316
+ Turbo Stream payloads without depending on the host app's Action Cable test
317
+ adapter. The helper captures Upkeep delivery after planning and rendering, but
318
+ before the transport broadcasts.
319
+
320
+ Use `capture_upkeep_change_facts` and `upkeep_match_report` when debugging an
321
+ invalidation miss. Capture the committed facts produced by the request, then
322
+ dry-run them against the current subscription store. The report returns the
323
+ candidate count, matched count, miss reason, and render targets without
324
+ broadcasting.
325
+
326
+ For structural subscription debugging, call `subscription.explain` or
327
+ `Upkeep::Rails.subscriptions.explain(subscription.id)`. Explanations summarize
328
+ the dependency tables and attributes, identity, frame count, lookup keys, and
329
+ metadata without requiring store-specific instance-variable inspection.
@@ -0,0 +1,211 @@
1
+ # SQLGlot Active Record query-analysis migration
2
+
3
+ ## Goal
4
+
5
+ Make SQLGlot the only production decoder for Active Record relation
6
+ dependencies in the next Upkeep release. Use the former Arel oracle to freeze a
7
+ parity corpus, then delete the oracle once the SQLGlot, raw-SQL, and adapter
8
+ corpora prove the same dependencies.
9
+
10
+ “Drop Arel” means Upkeep production code does not call `Relation#arel`, inspect
11
+ Arel nodes, or select a structured/unstructured policy. Active Record may still
12
+ use Arel internally to generate `Relation#to_sql`.
13
+
14
+ ## API boundary
15
+
16
+ Upkeep owns a small Ruby wrapper over the released `sql-glot-rust` C ABI. Its
17
+ names, arguments, and result shapes follow the native API:
18
+
19
+ 1. Wrap the existing parse and generation API:
20
+ - `Upkeep::SQLGlot.parse(sql, dialect:)`
21
+ - `Upkeep::SQLGlot.generate(statement, dialect:)`
22
+ - `Upkeep::SQLGlot.transpile(sql, from:, to:)`
23
+ 2. Project the released semantic API:
24
+ - `MappingSchema`
25
+ - `qualify_columns(statement, schema)`
26
+ - `build_scope(statement)`
27
+ - `lineage(column, statement, schema, config)`
28
+ 3. Preserve Rust/Python scope and lineage fields. Do not flatten child scope
29
+ collections or introduce an Upkeep-shaped SQLGlot response.
30
+
31
+ There is deliberately no combined `Upkeep::SQLGlot.analyze` API.
32
+
33
+ Upkeep-specific lowering remains separate from the sibling API:
34
+
35
+ ```text
36
+ Upkeep::SQLGlot.parse + SQL schema
37
+
38
+
39
+ Upkeep::SQLGlot::MappingSchema
40
+ Upkeep::SQLGlot.qualify_columns
41
+ Upkeep::SQLGlot.build_scope
42
+
43
+
44
+ Upkeep::SQLDependencyAnalysis
45
+
46
+
47
+ Upkeep::ActiveRecordQuery::Result
48
+ ```
49
+
50
+ `SQLDependencyAnalysis` owns physical dependency sources, referenced columns,
51
+ simple predicate DNF, equality edges, query shape, and conservative warnings.
52
+ `ActiveRecordQuery` owns model table, primary key, adapter dialect, schema
53
+ extraction, and the internal result contract.
54
+
55
+ This release is intentionally breaking. There are no existing users to migrate,
56
+ so no compatibility decoder, cache migration, top-level `Sqlglot` namespace,
57
+ or mixed installation is supported. Installing the release requires a full
58
+ bundle/gem reinstall so Bundler selects the matching Upkeep platform gem.
59
+
60
+ ## Native SQLGlot packaging
61
+
62
+ `upkeep-rails` depends on `ffi` and binds `sql-glot-rust` v0.10.26 directly.
63
+ The Ruby boundary lives under `Upkeep::SQLGlot`; there is no external
64
+ `sqlglot` Ruby gem dependency and no Upkeep-specific Rust crate or C ABI.
65
+
66
+ There is one logical gem. Each
67
+ release consists of five platform-specific `upkeep-rails` artifacts:
68
+
69
+ - `x86_64-linux-gnu`;
70
+ - `aarch64-linux-gnu`;
71
+ - `x86_64-darwin`;
72
+ - `arm64-darwin`; and
73
+ - `x64-mingw-ucrt`.
74
+
75
+ The released v0.10.26 tag is built in release CI and its unmodified shared
76
+ library is included in each artifact. Rust sources are build inputs and are not
77
+ shipped in the gem, so installation never requires Rust. There is deliberately
78
+ no generic source artifact.
79
+
80
+ The v0.10.26 scope builder preserves CTE references as scope sources. Upkeep
81
+ does not treat a scope source as a physical table; the AST lowerer resolves the
82
+ logical CTE to its physical child and scope validation checks every
83
+ schema-backed source.
84
+
85
+ Active Record SQL types remain available in the Ruby `MappingSchema`. The
86
+ native dependency schema represents every column type as `UNKNOWN` because
87
+ qualification and lineage inspect table and column identity, not column type.
88
+ Database-specific types therefore cannot prevent dependency extraction, and
89
+ Upkeep does not maintain a parallel SQL type catalog.
90
+
91
+ `qualify_columns` expands wildcards by design. Upkeep restores wildcard
92
+ projection nodes before dependency lowering so `SELECT table.*` does not add
93
+ every model attribute to collection invalidation. Qualification in predicates,
94
+ joins, grouping, ordering, and explicit projections is retained.
95
+
96
+ ## Implementation stages
97
+
98
+ ### 1. Establish SQLGlot as an internal runtime boundary
99
+
100
+ - Bind the released Rust library through `ffi`.
101
+ - Add adapter-to-dialect mapping.
102
+ - Convert Active Record schema metadata into SQLGlot-compatible table/column
103
+ metadata, retaining SQL types where available.
104
+ - Fail closed with an actionable SQL analysis error; never silently subscribe
105
+ only to the primary table after an unknown joined source.
106
+
107
+ ### 2. Productize dependency lowering
108
+
109
+ - Move the SQL-first spike into `Upkeep::SQLDependencyAnalysis`.
110
+ - Accept SQL AST, dialect, and schema only—no relation or model objects.
111
+ - Keep SQL constructs inside this decoder and lower them into a generic graph:
112
+ physical tables, columns, equality edges, predicate groups, and query shape.
113
+ - Treat CTEs and derived sources as logical sources backed by physical tables.
114
+ - Record ambiguity as conservative coverage or an unsupported analysis, never
115
+ as a missing dependency.
116
+
117
+ ### 3. Switch the Active Record runtime path
118
+
119
+ - Make `ActiveRecordQuery.analyze` parse `relation.to_sql` with SQLGlot.
120
+ - Preserve `ActiveRecordQuery::Result` so invalidation/replay consumers do not
121
+ need a simultaneous rewrite.
122
+ - Give write observation a separate `analyze_for_write` entry point. It may
123
+ conservatively describe the model table when semantic predicate analysis
124
+ fails; this is not a collection-query decoder or runtime fallback.
125
+ - Update opaque-query guidance so it no longer asks users to rewrite SQL as
126
+ Arel.
127
+
128
+ ### 4. Move Arel to an oracle
129
+
130
+ - Move the current collector to test support and rename it
131
+ `ArelQueryAnalysisOracle`.
132
+ - Production `lib/` must contain no Arel node dispatch and no
133
+ `Relation#arel` call.
134
+ - Compare the oracle with SQLGlot for relations the oracle can prove.
135
+ - SQLGlot may report additional safe dependencies. It must not omit any table,
136
+ column, predicate, or query-shape restriction proven by the oracle.
137
+ - Raw SQL cases that the oracle rejects are asserted directly against SQLGlot.
138
+
139
+ ### 5. Verification and release hardening
140
+
141
+ - Cover existing query-analysis tests and Pulse #1802 edge cases.
142
+ - Add raw predicate, raw join, alias/self-join, correlated subquery, CTE,
143
+ function/operator, `IN`, null, negation, OR-DNF, group/having/distinct,
144
+ limit/offset, and parse-error cases.
145
+ - Run the full Upkeep suite.
146
+ - Benchmark parsing separately from invalidation fan-out. This milestone does
147
+ not add an Upkeep-owned analysis or schema cache.
148
+ - Package the released native library for every supported platform.
149
+
150
+ ## Oracle exit criteria
151
+
152
+ Arel can be deleted after:
153
+
154
+ - the supported adapter corpus has no known dependency false negatives;
155
+ - SQLGlot covers every relation the oracle proves;
156
+ - raw SQL coverage has dedicated assertions;
157
+ - unsupported SQL fails with query, dialect, and parser diagnostics;
158
+ - production code has no oracle switch or fallback;
159
+ - analysis stays outside invalidation fan-out and meets the agreed performance
160
+ budget.
161
+
162
+ ## Current milestone
163
+
164
+ This milestone is complete in the 0.2.0 release branch: `Upkeep::SQLGlot` is
165
+ the only production SQL decoder, the former oracle outputs are frozen as
166
+ ordinary expected values, and the Arel collector has been deleted. Remaining
167
+ Arel usage in tests constructs Active Record input queries only; no Upkeep
168
+ analyzer inspects Arel.
169
+
170
+ ## Execution status
171
+
172
+ Completed:
173
+
174
+ - [x] Added `Upkeep::SQLGlot` as the production runtime parser.
175
+ - [x] Added generic SQL AST dependency lowering in
176
+ `Upkeep::SQLDependencyAnalysis`.
177
+ - [x] Switched `ActiveRecordQuery.analyze` from `Relation#arel` to
178
+ `Upkeep::SQLGlot.parse(relation.to_sql, dialect:)`.
179
+ - [x] Kept schema access on Active Record's existing schema cache; Upkeep owns
180
+ no analysis or schema cache.
181
+ - [x] Added a separate conservative write-analysis path without introducing a
182
+ collection-query fallback.
183
+ - [x] Used `ArelQueryAnalysisOracle` to freeze the parity corpus, then deleted
184
+ the collector and its test-support require.
185
+ - [x] Added direct raw-SQL coverage, including correlated subqueries, CTEs,
186
+ set operations, PostgreSQL operators, and MySQL/SQLite functions.
187
+ - [x] Bound parse, generate, transpile, `MappingSchema`, qualification, scope,
188
+ and lineage directly to `sql-glot-rust` v0.10.26.
189
+ - [x] Decoupled dependency schemas from database type parsing while retaining
190
+ the original SQL types at the Ruby boundary.
191
+ - [x] Preserved Rust scope and lineage fields and locked the corrected CTE
192
+ scope-source behavior in a binding test.
193
+ - [x] Added five platform-specific `upkeep-rails` artifacts for macOS, Linux,
194
+ and Windows, with no source gem or install-time Cargo build.
195
+ - [x] Added an adapter corpus for PostgreSQL, MySQL, and SQLite.
196
+ - [x] Added a warm semantic-analysis performance gate with a 2 ms CI budget;
197
+ the local 10,000-iteration mean is approximately 146 µs.
198
+ - [x] Verified the full suite and the focused performance gate.
199
+ - [x] Removed compatibility and mixed-installation handling; this release
200
+ requires a full reinstall.
201
+
202
+ Upstream and ecosystem follow-up:
203
+
204
+ - [x] Open the semantic Ruby bindings in `sql-glot-ruby`.
205
+ - [x] Merge the CTE source-overwrite fix in `sql-glot-rust`; it shipped in
206
+ v0.10.24 through
207
+ [`protegrity/sql-glot-rust#26`](https://github.com/protegrity/sql-glot-rust/pull/26).
208
+ - [x] Merge and release the Rust semantic C ABI in v0.10.25.
209
+ - [x] Delete the Upkeep-specific Rust bridge and external Ruby SQLGlot
210
+ dependency.
211
+ - [ ] Continue the Ruby wrapper PRs as optional ecosystem contributions.
@@ -0,0 +1,90 @@
1
+ # Turbo Frame subscription composition
2
+
3
+ ## Goal
4
+
5
+ Keep one authoritative Upkeep subscription for the visible page across Turbo
6
+ Frame navigation.
7
+
8
+ A frame response is not a page response. Turbo extracts only the matching
9
+ `<turbo-frame>`, and frame endpoints may omit the application shell and sibling
10
+ frames. Replacing the page subscription with the response capture would
11
+ therefore lose dependencies that are still visible. Keeping both captures would
12
+ retain stale dependencies and duplicate invalidation fan-out.
13
+
14
+ ## Model
15
+
16
+ `turbo_frame_tag` establishes a generic render scope in the dependency graph:
17
+
18
+ ```text
19
+ page
20
+ ├── shell
21
+ ├── turbo_frame:filters
22
+ │ └── current frame dependencies
23
+ └── turbo_frame:details
24
+ └── current frame dependencies
25
+ ```
26
+
27
+ For a Turbo Frame GET, the browser sends the current subscription identity and
28
+ Turbo's target frame id. Upkeep captures the response, finds the matching render
29
+ scope in both graphs, and creates a new pending subscription from:
30
+
31
+ ```text
32
+ old page graph - old target subtree + response target subtree
33
+ ```
34
+
35
+ The old subscription remains active while Action Cable connects the candidate.
36
+ After `connected`, the client installs the candidate as current and
37
+ unsubscribes the old subscription. Rejected or superseded candidates never
38
+ replace the active page graph.
39
+
40
+ ## Invariants
41
+
42
+ - There is one current page subscription per browser document.
43
+ - A page with no reactive full render keeps a dormant coordinator; its first
44
+ reactive frame establishes the subscription.
45
+ - A frame transition changes only the matching render scope.
46
+ - Shell and sibling dependencies survive frame navigation.
47
+ - Replaced dependencies leave the reverse index when the old cable subscription
48
+ disconnects.
49
+ - The server accepts a base subscription only with a valid activation token and
50
+ the same derived subscriber identity.
51
+ - A response cannot become current unless its Action Cable subscription
52
+ connects.
53
+ - Concurrent frame responses are ordered by the browser transition generation;
54
+ a late response cannot restore an older graph.
55
+ - Expired or structurally incompatible base graphs request a full Turbo render
56
+ instead of discarding already-active scopes.
57
+
58
+ ## Protocol
59
+
60
+ Before a frame fetch, the client adds these headers when a current subscription
61
+ exists:
62
+
63
+ - `X-Upkeep-Subscription-Id`
64
+ - `X-Upkeep-Subscription-Token`
65
+
66
+ After composing and registering the replacement, the response adds:
67
+
68
+ - `X-Upkeep-Subscription-Id`
69
+ - `X-Upkeep-Subscription-Token`
70
+ - `X-Upkeep-Subscription-Channel`
71
+ - `X-Upkeep-Subscription-Stream`
72
+
73
+ `turbo:frame-render` is the commit point. The custom subscription source reads
74
+ the response headers and starts the atomic cable transition. Full-page Turbo
75
+ Drive visits continue to replace the body-scoped source element normally.
76
+
77
+ ## Implementation
78
+
79
+ - [x] Capture `turbo_frame_tag` blocks as `turbo_frame` graph nodes with
80
+ controller replay recipes.
81
+ - [x] Add generic DAG subtree replacement and recorder composition.
82
+ - [x] Validate and compose frame registrations against the active page
83
+ subscription.
84
+ - [x] Return candidate subscription metadata in response headers.
85
+ - [x] Keep the body source stable during frame visits and atomically switch its
86
+ Action Cable subscription.
87
+ - [x] Cover graph composition, replay/targeting, request protocol, race/rejection
88
+ behavior, and a real Turbo Frame navigation followed by live invalidation.
89
+ - [x] Regenerate Pulse's installed client, run the full Upkeep proof suite, and
90
+ repeat the Pulse browser scenario without a page reload.