upkeep-rails 0.1.9 → 0.1.12

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (35) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +105 -195
  3. data/docs/drafts/turbo-streams-is-cache-invalidation-you-write-by-hand.md +240 -0
  4. data/docs/handoffs/_archive/2026-06-10-main.md +47 -0
  5. data/docs/how-it-works.md +8 -0
  6. data/lib/generators/upkeep/install/install_generator.rb +59 -0
  7. data/lib/generators/upkeep/install/templates/subscription.js +6 -5
  8. data/lib/generators/upkeep/install/templates/upkeep.rb +8 -7
  9. data/lib/upkeep/delivery/turbo_streams.rb +40 -15
  10. data/lib/upkeep/dependencies.rb +55 -5
  11. data/lib/upkeep/invalidation/planner.rb +48 -10
  12. data/lib/upkeep/rails/cable/channel.rb +27 -5
  13. data/lib/upkeep/rails/cable/subscriber_identity.rb +21 -1
  14. data/lib/upkeep/rails/client_subscription.rb +12 -12
  15. data/lib/upkeep/rails/cluster_guard.rb +57 -0
  16. data/lib/upkeep/rails/configuration.rb +9 -16
  17. data/lib/upkeep/rails/controller_runtime.rb +17 -0
  18. data/lib/upkeep/rails/railtie.rb +1 -10
  19. data/lib/upkeep/rails/testing.rb +1 -1
  20. data/lib/upkeep/rails.rb +58 -17
  21. data/lib/upkeep/runtime.rb +39 -2
  22. data/lib/upkeep/shared_streams.rb +17 -3
  23. data/lib/upkeep/subscriptions/active_record_store.rb +53 -62
  24. data/lib/upkeep/subscriptions/active_record_subscription_persistence.rb +6 -3
  25. data/lib/upkeep/subscriptions/active_registry.rb +0 -7
  26. data/lib/upkeep/subscriptions/base_store.rb +106 -0
  27. data/lib/upkeep/subscriptions/layered_reverse_index.rb +9 -13
  28. data/lib/upkeep/subscriptions/lookup_instrumentation.rb +32 -0
  29. data/lib/upkeep/subscriptions/persistent_reverse_index.rb +4 -1
  30. data/lib/upkeep/subscriptions/store.rb +38 -64
  31. data/lib/upkeep/version.rb +1 -1
  32. data/upkeep-rails.gemspec +0 -1
  33. metadata +7 -24
  34. data/lib/upkeep/rails/delivery_job.rb +0 -29
  35. data/lib/upkeep/subscriptions/async_durable_writer.rb +0 -131
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: a4b31fcfa6f392fc224821b3fdd7c0ec4b98474cebaa775aacd504172c1c930e
4
- data.tar.gz: a421c17550f3065d13dad7502984436a065a8d1ede670252f0b2a2187a88a785
3
+ metadata.gz: dc9e5c1e0ea0a632a8fea8240f0b36da735af2b8b652fa1d8f7ad40d6794f64e
4
+ data.tar.gz: d695df991c56c9f5a4f371cb76fbfc90b63c34404b4f3f272a8b809130395dfb
5
5
  SHA512:
6
- metadata.gz: 33947d347624c6e61888db6096f71f36b8abc893fa9cbe3a8a9643c9ef91a33eed220c643086ae65dfdcea6290b5881d44d7986feb8bbc1265906673cc4b9b8b
7
- data.tar.gz: b358287f543e6bb3ec9e0a254dd453d2ae134575cbfd93a2710f2b58a24ac4d1c4b7b2784498505baede37ce8027ae28340384025bac8b0fa6acc7169abd7934
6
+ metadata.gz: df24407ec23c61979e8f8960794a6b3ae6c7f82aacb5902780731e664e925cab32e81f48cfc3208fe9bf9d505c871169d8e5774ce5888309fb0041574adcb941
7
+ data.tar.gz: 2afe28057f1d6f565900fb55f3c3d6cfba2a8fe73b817c343cd95349014ea7a7fb04664b3a83a84c21bed606f2379e2f4287f5f19537d0692aab25ac018dcda0
data/README.md CHANGED
@@ -1,93 +1,38 @@
1
1
  # Upkeep Rails
2
2
 
3
- ## 0. Quick Intro
3
+ Upkeep automatically syncs record changes to browsers showing affected data. When you save a record, it figures out which rendered pages depend on it and sends Turbo Streams to the right subscribers.
4
4
 
5
- Upkeep Rails keeps ordinary Rails pages fresh when the data, request inputs, or
6
- identity values they used change.
7
-
8
- A successful HTML GET renders through Rails as usual. Upkeep records the
9
- templates, records, relations, request values, and identity values that shaped
10
- the response. Later, an Active Record commit emits facts about what changed.
11
- Upkeep matches those facts to affected rendered frames and delivers ordinary
12
- Turbo Stream updates over ActionCable.
13
-
14
- The design goal is Rails-shaped DX: controllers load state, views render ERB,
15
- models commit writes, and Upkeep derives the reactive boundary from the Rails
16
- surfaces it observes. There is no query catalog and no `watch` or `track` DSL.
17
-
18
- For the deeper runtime model, see [How Upkeep Works](docs/how-it-works.md).
19
-
20
- ## 1. Upkeep vs Vanilla Turbo
21
-
22
- With vanilla Turbo Streams, write paths often need to name the UI they refresh:
5
+ Instead of naming stream targets in your controller:
23
6
 
24
7
  ```ruby
25
- # app/controllers/cards_controller.rb
26
8
  class CardsController < ApplicationController
27
9
  def create
28
- @board = Board.find(params[:board_id])
29
- @card = @board.cards.new(card_params)
30
-
31
- if @card.save
32
- @open_card_count = @board.cards.open.count
33
-
34
- respond_to do |format|
35
- format.turbo_stream
36
- end
37
- else
38
- render :new, status: :unprocessable_entity
39
- end
10
+ @card = Card.create!(card_params)
11
+ respond_to { |f| f.turbo_stream }
40
12
  end
41
13
  end
42
14
  ```
43
15
 
44
16
  ```erb
45
- <%# app/views/cards/create.turbo_stream.erb %>
46
- <%= turbo_stream.append "cards",
47
- partial: "cards/card",
48
- locals: { card: @card } %>
49
-
50
- <%= turbo_stream.update "open_card_count", @open_card_count %>
17
+ <%= turbo_stream.append "cards", partial: "card", locals: { card: @card } %>
51
18
  ```
52
19
 
53
- That follows the standard Turbo Stream shape and avoids a page visit for the
54
- submitting browser. The tradeoff is that the write path is still coupled to the
55
- current UI. Adding another dependent page, sidebar, filter, or counter usually
56
- means revisiting stream templates, controller assignments, callbacks, or
57
- broadcasts.
58
-
59
- With Upkeep, the controller can acknowledge the successful write without naming
60
- stream targets:
20
+ You just save the record:
61
21
 
62
22
  ```ruby
63
23
  class CardsController < ApplicationController
64
24
  def create
65
- board = Board.find(params[:board_id])
66
- board.cards.create!(card_params)
67
-
25
+ Card.create!(card_params)
68
26
  head :no_content
69
27
  end
70
28
  end
71
29
  ```
72
30
 
73
- The submitting Turbo request gets a successful empty response, so it does not
74
- perform a page visit. The GET that rendered the page already recorded the
75
- rendered dependencies. When the commit lands, Upkeep selects affected
76
- subscribers and sends Turbo Streams to the browsers that need them. Validation
77
- and error rendering stay ordinary application code; the live update path does
78
- not need to name DOM targets.
31
+ Upkeep observed which records the initial GET read, so it knows the page depends on the card table. When the create commits, it broadcasts to the right browsers automatically.
79
32
 
80
- | Concern | Vanilla Turbo | Upkeep |
81
- | --- | --- | --- |
82
- | Write path | Names stream targets, partials, counters, or pages. | Commits domain changes. |
83
- | Read path | Ordinary Rails render. | Ordinary Rails render, captured during HTML GETs. |
84
- | Browser update | Turbo Streams or page refresh. | Turbo Streams or page refresh. |
85
- | Boundary | App declares it in stream templates, callbacks, or broadcasts. | Upkeep derives it from rendered Rails surfaces when it can prove safety. |
86
- | Unsafe shape | App decides how broad to broadcast. | Upkeep raises or warns and refuses the live boundary. |
87
-
88
- ## 2. Install
33
+ ## Install
89
34
 
90
- Add the gem:
35
+ Add the gem to your Gemfile:
91
36
 
92
37
  ```ruby
93
38
  gem "upkeep-rails"
@@ -100,68 +45,37 @@ bin/rails generate upkeep:install
100
45
  bin/rails db:migrate
101
46
  ```
102
47
 
103
- The generator creates subscription tables, writes
104
- `config/initializers/upkeep.rb`, mounts ActionCable when needed, pins Turbo and
105
- ActionCable for importmap apps, and imports the browser bootstrap from
106
- `app/javascript/application.js`.
107
-
108
- The browser bootstrap is vendored into the host app at
109
- `app/javascript/upkeep/subscription.js`. After upgrading `upkeep-rails`, rerun
110
- the installer or compare that file with the generated template.
48
+ This creates the subscription tables, writes an initializer, mounts ActionCable if needed, and imports the browser client.
111
49
 
112
- Requirements: Ruby 3.2+, Rails 7.1+, and Turbo 2.0+.
50
+ Requirements: Ruby 3.2+, Rails 7.1+, Turbo 2.0+.
113
51
 
114
- ## 3. Configure Runtime
52
+ ## Configure runtime
115
53
 
116
- The generated initializer is the normal place to configure Upkeep:
54
+ The installer creates `config/initializers/upkeep.rb`. Most projects work with the defaults, but you can customize:
117
55
 
118
56
  ```ruby
119
- # config/initializers/upkeep.rb
120
57
  Upkeep::Rails.configure do |config|
121
- app_config = Rails.application.config.upkeep
122
-
123
- config.enabled = app_config.fetch(:enabled, true)
124
- config.subscription_store = app_config.fetch(:subscription_store, Rails.env.test? ? :memory : :active_record)
125
- config.delivery_adapter = app_config.fetch(:delivery_adapter, Rails.env.production? ? :active_job : :async)
126
- config.delivery_queue = app_config.fetch(:delivery_queue, :upkeep_realtime)
58
+ config.subscription_store = :active_record # :memory for tests
59
+ config.deliver_inline = false # true for tests/console
60
+ config.refused_boundary_behavior = :warn # :raise in strict mode
127
61
  end
128
62
  ```
129
63
 
130
- Use `:active_record` for durable subscription storage in production. The
131
- generated migration creates the required tables. Use `:memory` for most request
132
- and system tests, and keep at least one app or CI path on `:active_record` when
133
- you want to exercise durable rows, schema checks, reload, and cross-process
134
- lookup.
135
-
136
- Production apps should use Active Job for delivery so planning, rerendering,
137
- and broadcasting do not run in the writer's request:
64
+ **Subscription store:** Use `:memory` for test suites. Use `:active_record` for production—it stores subscriptions durably across restarts.
138
65
 
139
- ```ruby
140
- Upkeep::Rails.configure do |config|
141
- config.delivery_adapter = Rails.env.production? ? :active_job : :async
142
- config.delivery_queue = :upkeep_realtime
143
- end
144
- ```
66
+ **Delivery mode:** Upkeep broadcasts from a background dispatcher in the same process that performed the write. For tests and console sessions where you want synchronous behavior, set `deliver_inline = true`.
145
67
 
146
- Configure the app's Active Job backend normally, such as Solid Queue, Sidekiq,
147
- or GoodJob. ActionCable still needs a shared adapter in multi-process
148
- deployments because a job worker may not be the process holding the browser's
149
- WebSocket. Redis, Solid Cable, and PostgreSQL are shared ActionCable adapters.
68
+ **Subscription lifecycle:** Subscriptions clean themselves up. A clean disconnect deletes the subscription immediately; abandoned subscriptions (crashed browsers, dropped connections) are trimmed opportunistically in small batches once they go untouched for `config.subscription_ttl` (default 24 hours); and connected pages send a liveness heartbeat every 20 minutes, so a long-lived open tab is never pruned. Adjust `subscription_ttl` to control how long abandoned subscriptions may linger.
150
69
 
151
- For local debugging, set `config.delivery_adapter = :inline` to run delivery
152
- immediately.
70
+ **Multi-process deployments:** If you run multiple Puma workers, you need a cross-process cable adapter so broadcasts reach all processes. We recommend `solid_cable`.
153
71
 
154
- ## 4. Configure Identity
72
+ ## Configure identity
155
73
 
156
- Pages that depend on a user, account, tenant, locale, or other viewer-specific
157
- value need an explicit identity bridge.
74
+ If your pages depend on the current user, account, or any viewer-specific value, tell Upkeep how to match that between render time and subscription time.
158
75
 
159
- The render side tells Upkeep which value the HTML render read. The subscribe
160
- side tells Upkeep how the ActionCable connection proves the same value when the
161
- browser subscribes.
76
+ First, declare the identity in the initializer:
162
77
 
163
78
  ```ruby
164
- # config/initializers/upkeep.rb
165
79
  Upkeep::Rails.configure do |config|
166
80
  config.identify :viewer, current: ["Current", :user] do
167
81
  subscribe { |connection| connection.current_user }
@@ -169,10 +83,9 @@ Upkeep::Rails.configure do |config|
169
83
  end
170
84
  ```
171
85
 
172
- The matching cable connection must expose that identity:
86
+ Then make sure your cable connection exposes it:
173
87
 
174
88
  ```ruby
175
- # app/channels/application_cable/connection.rb
176
89
  module ApplicationCable
177
90
  class Connection < ActionCable::Connection::Base
178
91
  identified_by :current_user
@@ -184,128 +97,125 @@ module ApplicationCable
184
97
  end
185
98
  ```
186
99
 
187
- Choose the source keyword from the API your render path actually reads:
100
+ Pick the declaration that matches how your render path reads the identity:
188
101
 
189
- | Render path reads | Declare | Subscribe side returns |
102
+ | Render path | Configuration | Subscribe returns |
190
103
  | --- | --- | --- |
191
- | `Current.user` | `current: ["Current", :user]` | the same user, usually `connection.current_user` |
192
- | Devise or Warden user reads | `warden: :user` | the same Devise user, usually `connection.current_user` |
104
+ | `Current.user` | `current: ["Current", :user]` | `connection.current_user` |
105
+ | Devise/Warden | `warden: :user` | `connection.current_user` |
193
106
  | `session[:user_id]` | `session: :user_id` | `connection.session[:user_id]` |
194
107
  | `cookies[:account_id]` | `cookie: :account_id` | `connection.cookies[:account_id]` |
195
108
 
196
- By default, `nil` means a declared identity boundary is absent. That keeps
197
- logged-out pages anonymous-public even when a layout checks `Current.user` or
198
- `session[:user_id]`. If your app has another "not signed in" sentinel, declare
199
- it:
109
+ For logged-out pages where `nil` is a valid identity, declare how to recognize "absent":
200
110
 
201
111
  ```ruby
202
- Upkeep::Rails.configure do |config|
203
- config.identify :viewer, session: :user_id do
204
- absent_if { |value| value.nil? || value == false }
205
- subscribe { |connection| connection.session[:user_id] }
206
- end
112
+ config.identify :viewer, session: :user_id do
113
+ absent_if { |value| value.nil? || value == false }
114
+ subscribe { |connection| connection.session[:user_id] }
207
115
  end
208
116
  ```
209
117
 
210
- If a page reads an undeclared non-absent `CurrentAttributes` or Warden identity,
211
- Upkeep refuses live registration instead of guessing who may receive updates.
118
+ If your render path reads an undeclared identity value (like a `CurrentAttributes` field), Upkeep will refuse to register that page for live updates rather than guess who should receive the broadcast.
212
119
 
213
- ## 5. Opaque Values, DX, and Refactors
120
+ ## When Upkeep can't track your data
214
121
 
215
- Upkeep only registers live boundaries it can prove. It needs to know which
216
- future write facts can make a rendered result stale, which target can be
217
- rerendered or patched, and which observed identity inputs decide whether the
218
- result can be shared.
122
+ Upkeep needs to understand your queries to know which records affect which renders. It can't work with raw SQL strings, opaque joins, or objects that aren't tied back to the database schema.
219
123
 
220
- `Opaque` means application code used something real, but Rails did not expose
221
- enough structure for Upkeep to answer those questions. Common examples are raw
222
- SQL predicates, raw joins, raw `from` sources, unknown table aliases, opaque
223
- order expressions, and render locals that cannot be rebuilt later.
124
+ These queries will cause Upkeep to refuse the page:
224
125
 
225
- The DX is intentionally fail-fast:
126
+ ```ruby
127
+ # Raw SQL predicates
128
+ Story.where("score >= 0")
129
+ Story.order("created_at DESC")
226
130
 
227
- - development and test raise by default
228
- - production warns and skips live registration by default
229
- - `refused_boundary.upkeep` is emitted for instrumentation
230
- - Upkeep does not widen to a broad unsafe dependency
131
+ # Opaque joins
132
+ User.joins("INNER JOIN posts ON ...")
133
+
134
+ # From sources that aren't table names
135
+ Story.from("(SELECT * FROM stories WHERE ...)")
136
+ ```
231
137
 
232
- You can choose the behavior explicitly:
138
+ Rewrite them using Rails or Arel:
139
+
140
+ ```ruby
141
+ Story.where(Story.arel_table[:score].gteq(0))
142
+ Story.order(created_at: :desc)
143
+ ```
144
+
145
+ By default, Upkeep raises an error in development and test, and logs a warning in production (but still renders the page). You can change this behavior:
233
146
 
234
147
  ```ruby
235
148
  Upkeep::Rails.configure do |config|
236
- config.refused_boundary_behavior = :raise # or :warn
149
+ config.refused_boundary_behavior = :warn # or :raise
237
150
  end
238
151
  ```
239
152
 
240
- Most refactors are normal Rails cleanup. Prefer hash conditions and symbolic
241
- orders when they express the query:
153
+ For queries that genuinely can't be made structural—like full-text search backed by raw `tsvector`—opt the request out entirely instead:
242
154
 
243
155
  ```ruby
244
- # Before: opaque SQL string order
245
- Story.order("stories.created_at DESC")
156
+ class StoriesController < ApplicationController
157
+ def index
158
+ @stories = params[:query].present? ? Story.search(params[:query]) : Story.all
159
+ end
246
160
 
247
- # After: structural Rails order
248
- Story.order(created_at: :desc)
161
+ private
162
+
163
+ def upkeep_reactive_request?
164
+ return false if params[:query].present?
165
+ super
166
+ end
167
+ end
249
168
  ```
250
169
 
251
- Use Arel when the query needs an operator or correlated condition that hash
252
- syntax cannot express:
170
+ The request still renders normally. Upkeep just doesn't register subscriptions or analyze the boundary. The unfiltered view (without a query param) stays reactive.
253
171
 
254
- ```ruby
255
- # Before: opaque SQL string predicate
256
- Story.where("score >= 0")
172
+ **General rule:** if Rails or Arel can describe the table, columns, predicates, and sort order, Upkeep can track it. If the query logic lives inside a SQL string, Upkeep will refuse the page.
257
173
 
258
- # After: structural Arel predicate
259
- Story.where(Story.arel_table[:score].gteq(0))
260
- ```
174
+ ## How Upkeep compares to Turbo Streams
175
+
176
+ With vanilla Turbo Streams, the write action names which parts of the UI to refresh. You maintain the list of targets as your UI evolves:
261
177
 
262
178
  ```ruby
263
- # Before: opaque correlated SQL
264
- HiddenStory.where(Arel.sql("hidden_stories.story_id = stories.id"))
179
+ class CardsController < ApplicationController
180
+ def create
181
+ @board = Board.find(params[:board_id])
182
+ @card = @board.cards.create!(card_params)
183
+
184
+ respond_to do |format|
185
+ format.turbo_stream
186
+ end
187
+ end
188
+ end
189
+ ```
265
190
 
266
- # After: structural correlated predicate
267
- HiddenStory.where(
268
- HiddenStory.arel_table[:story_id].eq(Story.arel_table[:id])
269
- )
191
+ ```erb
192
+ <%= turbo_stream.append "cards", partial: "card", locals: { card: @card } %>
193
+ <%= turbo_stream.update "board_open_count", @board.open_count %>
270
194
  ```
271
195
 
272
- For replay values, keep frame locals and render options to records, relations,
273
- arrays, hashes, literals, and observed request, session, or cookie values. Avoid
274
- passing procs, IO handles, open clients, or process-local objects into live
275
- render boundaries.
196
+ This couples the write path to the current UI. When you add a sidebar showing the same board, or a filter that depends on card status, you revisit the controller and stream template.
276
197
 
277
- Some boundaries are intractable on purpose. A full-text search backed by raw
278
- `tsvector`/`tsquery` SQL has no structural column coverage to prove, and
279
- rewriting it would defeat the search. When a request should not be made
280
- reactive at all, opt it out instead of refusing a boundary mid-render. Override
281
- `upkeep_reactive_request?` in the controller and return `false` for those
282
- requests:
198
+ With Upkeep, the write path just saves the record. The subscription comes from the read:
283
199
 
284
200
  ```ruby
285
- # app/controllers/stories_controller.rb
286
- def index
287
- @stories = params[:query].present? ? Story.search(params[:query]) : Story.recent
201
+ class CardsController < ApplicationController
202
+ def create
203
+ Board.find(params[:board_id]).cards.create!(card_params)
204
+ head :no_content
205
+ end
288
206
  end
207
+ ```
289
208
 
290
- private
209
+ Upkeep observed which records your initial GET read, so it knows which renders care about this card. It broadcasts to those subscribers automatically. You add a sidebar or filter without touching the write path.
291
210
 
292
- # Search results use a raw full-text scope Upkeep cannot prove; render them
293
- # normally but do not register them for live refresh.
294
- def upkeep_reactive_request?
295
- return false if params[:query].present?
211
+ The tradeoff: Turbo Streams let you control exactly what gets broadcast and how it's rendered (you might patch a counter without rerendering the list). Upkeep derives updates from renders it observed, so it's narrower but also more opinionated about safety.
296
212
 
297
- super
298
- end
299
- ```
213
+ | Concern | Turbo Streams | Upkeep |
214
+ | --- | --- | --- |
215
+ | Write path | Declares targets and templates | Just commits the record |
216
+ | Boundary discovery | You maintain it | Upkeep infers from renders |
217
+ | Unsafe broadcasts | You decide how broad | Upkeep refuses unsafe patterns |
218
+ | Render changes | Update the stream template | Happens automatically |
219
+ | Full control | Yes | No, but less coupling |
300
220
 
301
- An opted-out request still runs the action and renders the page; Upkeep simply
302
- records no subscription, injects no source, and analyzes no boundary — so an
303
- opaque relation on that request neither raises nor warns. The unfiltered page
304
- (no `query`) stays reactive. Reach for this only when the boundary is
305
- genuinely unprovable; prefer the structural refactors above whenever the shape
306
- *can* be made explicit.
307
-
308
- The rule of thumb: when Rails and Arel can describe the table, column,
309
- predicate, order, and value shape, Upkeep can usually reason about it. When the
310
- shape is hidden inside a string or arbitrary Ruby object, Upkeep refuses the
311
- live boundary and tells you where to make the shape explicit.
221
+ See [How Upkeep Works](docs/how-it-works.md) for details on the runtime model.