upkeep-rails 0.1.0

data/docs/cost-model-roadmap.md

@@ -0,0 +1,703 @@
+# Upkeep Rails Cost Model Roadmap
+
+This roadmap tracks the work needed to make `upkeep-rails` competitive on the
+runtime surfaces where it can win.
+
+Last updated: 2026-05-21.
+
+Current code state:
+
+- Base runtime/docs commit: `474fb9c` (`Instrument capture and document Upkeep Rails`).
+- Current pass adds gated request/action profiling, operation-scoped capture
+  metrics, mutation action/delivery timings, client/server phase correlation,
+  benchmark phase labels, lightweight subscription-shape tracing, static
+  template metadata caching, shared-stream signature memoization, and queued
+  dependency flushing.
+- Latest verification: `bundle exec rake test` passed with `186` runs and
+  `1172` assertions.
+- Latest benchmark data is from the 2026-05-21 identity-free feed compare
+  reports below.
+
+The target is not "cheaper than hand-written Turbo Streams for every case."
+Hand-written `append`, `prepend`, `remove`, `replace`, and `update` operations
+are cheaper when the app already knows the exact stream, target, and operation.
+Upkeep's target is the space where apps otherwise fall back to
+`broadcast_refreshes_to`, broad fanout, or duplicated manual stream rules.
+
+## Work Lanes
+
+Keep the work split into three lanes:
+
+- Runtime behavior: change capture, dependency indexing, invalidation planning,
+  stream operation proofs, subscription lifecycle, store/dispatcher/broadcast
+  boundaries, and delivery semantics.
+- Deoptimization diagnostics: explanations for cases where the runtime falls
+  back from a cheaper proven operation to a broader conservative operation.
+- Benchmarks and telemetry: timing, payload, fanout, and comparison reports used
+  to validate claims. This lane should measure the runtime; it should not drive
+  runtime behavior.
+
+## Current Runtime Shape
+
+The current Rails runtime observes Action View render frames, Active Record
+attribute reads, Active Record collection renders, and request identity
+surfaces. It stores subscription graphs, plans invalidations through a reverse
+index, rerenders selected targets, and delivers Turbo Stream HTML over
+ActionCable.
+
+The current implementation has these cost boundaries:
+
+- Change events carry explicit lifecycle type plus table, model, id, changed
+  attributes, and normalized old/new values for row-level writes.
+- Collection invalidation lookup is keyed by proven table/column pairs and then
+  matched against structurally visible predicate values where available.
+- Delivery emits rendered HTML in Turbo Stream envelopes. It does not currently
+  emit keyed HTML diff ops.
+- Collection `append`, `prepend`, `remove`, and stable member `replace` exist
+  as guarded operation optimizations. Member updates that leave the relation or
+  change rendered order still fall back to render-site replacement.
+- Subscription rows and reverse-index entries persist. Registration is
+  live-first: the active in-process registry is updated synchronously, while
+  durable Active Record rows are written by a coalesced writer. Channel
+  subscribe does not write liveness metadata; explicit touch/prune maintenance
+  owns stale-row cleanup.
+- Production subscription storage is explicit: `:active_record` is the supported
+  queryable reverse-index store, while `:memory` is an explicit development/test
+  store. There is no runtime fallback from ActiveRecord to memory.
+- The dispatcher and broadcast bus are separate from subscription storage. The
+  current dispatcher is process-local async batching; a future ActiveJob
+  dispatcher may use Solid Queue, Redis-backed queues, or any app queue adapter.
+  Broadcast delivery still goes through ActionCable and needs a shared
+  ActionCable adapter in multi-worker deployments.
+- Planning and Turbo Stream batch building report selected actions,
+  deoptimization reasons, render counts, render duration, payload bytes, and
+  envelope counts. Benchmarking and broader telemetry stay separate from
+  runtime proof work.
+- Identity, request, session, and cookie dependencies are recorded on the graph
+  for replay and sharing, but they are not indexed as lifecycle invalidation
+  lookup rows.
+- Request capture, subscription registration, and subscription shape compilation
+  are separate runtime boundaries. `Capture::Request` owns the rendered response
+  and recorder, `Subscriptions::Registrar` owns identity/registration, and
+  `Subscriptions::ShapeCache` owns reusable identity-free subscription shapes.
+- Request capture now reports server-side phases for pending delivery, action
+  capture, response body handling, request signature, registration, client
+  marker injection, and post-action delivery.
+- Subscription-shape identity is a DAG/runtime concept. `DAG::SubscriptionShape`
+  owns the reusable structural signature, and `Runtime::Recorder` traces the
+  shape terms as frames/dependencies/containment are recorded. The trace stores
+  seen term keys rather than full component snapshots, and dependency reads are
+  queued by owner before being flushed into the DAG. Normal capture hits use an
+  order-sensitive rolling trace digest; this can create false cache misses if
+  record order differs, but it cannot create unsafe sharing.
+- Repeated identity-free anonymous pages share compiled subscription shape and
+  reverse-index entry templates. The initial HTML response is not shared.
+
+## Current Measured State
+
+Latest identity-free shared report:
+
+- Report: `benchmark/results/identity-free-feed-compare-20260521113822.md`
+- Shape cache: `199` hits, `1` miss, `0` bypasses.
+- Upkeep setup p95: `58ms`; Turbo setup p95: `38.05ms`.
+- Upkeep page render p95: `47ms`; Turbo page render p95: `24.05ms`.
+- Upkeep WebSocket connect p95: `9ms`; Turbo WebSocket connect p95: `11ms`.
+- Upkeep subscribe call p95: `6ms`; Turbo subscribe call p95: `2ms`.
+- Upkeep subscribe ack p95: `5ms`; Turbo subscribe ack p95: `1.05ms`.
+- Upkeep write POST p95: `58.90ms`; Turbo write POST p95: `58.69ms`.
+- Upkeep update-settled p95: `191ms`; Turbo update-settled p95: `342.10ms`.
+- Upkeep setup page server p95: `34.28ms`; Turbo setup page server p95:
+  `10.12ms`.
+- Upkeep write request server p95: `15.74ms`; Turbo write request server p95:
+  `34.74ms`.
+- Upkeep `GET /feed` request-capture total p95: `33.081ms`; action p95:
+  `14.896ms`; view p95: `15.456ms`; template p95: `11.104ms`; collection
+  render p95: `4.362ms`; SQL p95: `0.191ms`; registration p95: `3.814ms`.
+- Upkeep recorder profile inside `GET /feed`: dependency p95 `3.916ms`, frame
+  p95 `1.364ms`, shape trace p95 `3.588ms`.
+- Upkeep shape miss total: `4.324ms`; shape hit key p95: `0.031ms`; shape hit
+  total p95: `0.049ms`.
+- Upkeep server-side subscribe total p95: `1.884ms`.
+
+Latest identity-free single-subscriber report:
+
+- Report: `benchmark/results/identity-free-feed-compare-20260521114026.md`
+- Shape cache: `1` miss, `0` hits, `0` bypasses.
+- Upkeep setup p95: `140ms`; Turbo setup p95: `104ms`.
+- Upkeep page render p95: `91ms`; Turbo page render p95: `73ms`.
+- Upkeep WebSocket connect p95: `38ms`; Turbo WebSocket connect p95: `17ms`.
+- Upkeep subscribe call p95: `10ms`; Turbo subscribe call p95: `14ms`.
+- Upkeep subscribe ack p95: `10ms`; Turbo subscribe ack p95: `13ms`.
+- Upkeep write POST p95: `38.85ms`; Turbo write POST p95: `66.37ms`.
+- Upkeep update-settled p95: `90ms`; Turbo update-settled p95: `96ms`.
+- Upkeep setup page server p95: `83.88ms`; Turbo setup page server p95:
+  `66.02ms`.
+- Upkeep write request server p95: `23.78ms`; Turbo write request server p95:
+  `32.72ms`.
+- Upkeep `GET /feed` request-capture total p95: `82.921ms`; action p95:
+  `70.594ms`; view p95: `55.578ms`; template p95: `42.352ms`; collection
+  render p95: `13.226ms`; SQL p95: `1.577ms`; registration p95: `12.129ms`.
+- Upkeep recorder profile inside `GET /feed`: dependency p95 `5.783ms`, frame
+  p95 `2.418ms`, shape trace p95 `4.264ms`.
+- Upkeep shape miss total: `4.221ms`; server-side subscribe total:
+  `2.085ms`.
+- The single-subscriber run is intentionally tracked as fixed-cost evidence,
+  but the one-sample p95 is noisy. Use it to find cost centers, not to claim a
+  stable product-level percentile.
+
+Next cost-model actions:
+
+- Continue structural reductions in `GET /feed` capture/render work. The latest
+  pass cut shared recorder shape trace from `8.107ms` to `3.588ms`, frame work
+  from `3.521ms` to `1.364ms`, and collection render capture from `9.356ms` to
+  `4.362ms`. The remaining shared setup gap is now mostly Action View template
+  capture plus page request overhead, not SQL, shape hits, response-body
+  handling, or subscribe activation.
+- Keep request-capture profiling gated by listeners so production capture does
+  not pay profiler clock overhead. Benchmark data should use these fields for
+  diagnosis, not as always-on runtime work.
+- Preserve implicit anonymous sharing. Developers should not declare a special
+  anonymous mode; the runtime should prove the absence of identity dependencies,
+  record why it refused sharing when identity is observed, and keep identity
+  diagnostics separate from benchmark-only telemetry.
+- Use client/server timing correlation to separate real Rails work from k6 or
+  network scheduling delay. The current shared setup gap is mainly setup page
+  server time: Upkeep `34.28ms` versus Turbo `10.12ms`.
+- Keep write POST phase telemetry, because shared update economics are strong
+  but write latency should stay flat as fanout packing changes.
+- Run 500/1000-subscriber shared reports to validate fanout packing after the
+  200-subscriber one-render path is stable.
+- Keep refused-boundary and live-deoptimization reporting separate. Runtime
+  refusal and delivery deopt telemetry exist; benchmark aggregation and
+  source-level suggestions still need to be made complete.
+- Keep shared-response HTML reuse out of the core path; share subscription
+  structure and update renders, not initial response bytes.
+
+## Phase 1: Normalize Change Events
+
+Goal: make every Active Record write produce a lifecycle event that can support
+precise invalidation.
+
+Work:
+
+- Emit explicit `create`, `update`, and `destroy` event types.
+- Include `old_values`, `new_values`, and per-attribute change pairs.
+- Preserve `changed_attributes` for existing planner compatibility.
+- Keep bulk `update_all` and `delete_all` events explicit and conservative.
+- Add tests for create, update, destroy, bulk update, and bulk delete event
+  shape.
+
+Exit criteria:
+
+- Destroy events can select collection dependencies.
+- Future predicate matching can evaluate old and new row values without
+  reaching back into deleted records.
+- Existing invalidation tests continue to pass.
+
+## Phase 2: Predicate-Aware Collection Invalidation
+
+Goal: reduce over-selection for collection dependencies.
+
+Status: structural predicate metadata has landed, opaque collection predicates
+are refused instead of broadening, and reverse-index keys now use proven
+collection table+column coverage instead of table-only collection lookup.
+
+Work:
+
+- Persist normalized predicate metadata from structural Arel analysis.
+- Index exact equality predicates where values are structurally visible.
+- Match update events against both old and new values so moved rows invalidate
+  source and destination collections.
+- Keep collection reverse-index lookup keyed by proven table+column pairs.
+
+Exit criteria:
+
+- Updating an unrelated row in the same table no longer selects unrelated
+  equality-filtered collections.
+- Moving a row between filter buckets selects both affected render sites.
+- Opaque predicates refuse the reactive boundary instead of registering broad
+  invalidation.
+
+## Phase 3: Provable Stream Operations
+
+Goal: emit cheaper operations only when the runtime can prove them.
+
+Status: create, destroy, and stable member-update operations have landed.
+Ordering and membership-changing updates still intentionally use render-site
+replacement until a separate move/removal proof exists.
+
+Work:
+
+- Keep `append` gated by relation shape and replay proof.
+- Add `prepend` for creates when ordering proves the new row belongs first.
+- Add `remove` for destroys when the prior rendered member target is known.
+- Add member `replace` for updates when the row was already rendered, still
+  belongs to the relation after the write, and replay proves the prior rendered
+  member order is unchanged.
+- Fall back to render-site replacement whenever proof conditions fail.
+
+Exit criteria:
+
+- Operation planning is cheaper than full render-site replacement for common
+  create/destroy/member-update paths.
+- Tests cover success and fallback cases for each operation.
+- Member updates that change relation membership or ordering use render-site
+  replacement until a cheaper move operation has its own proof.
+
+## Phase 4: Subscription Lifecycle And Backpressure
+
+Goal: bound retained work as clients navigate, disconnect, and churn.
+
+Status: production storage now fails fast to the ActiveRecord store. Memory
+storage is explicit development/test configuration only. ActiveRecord storage
+has been split into an active registry, persistent reverse index, durable
+subscription persistence, layered lookup, JSON snapshot codec, and async durable
+writer. The hot registration path now indexes live subscriptions in-process and
+enqueues durable writes without issuing SQL in the request thread. Channel
+unsubscribe now unregisters live subscriptions, cancels queued durable writes,
+removes persisted rows for inflight work, and deletes reverse-index entries
+incrementally instead of rebuilding the index.
+
+Measured gate after the cleanup:
+
+- `benchmark/store_perf.rb`: ActiveRecord live registration is 0.047ms/op with
+  zero SQL, durable drain is 534ms for 1,000 subscriptions, and cold fetch of
+  100 subscriptions is 17.9ms.
+- `matrix/cold_connect_churn_chat` upkeep-only gate
+  (`20260514231017`): setup p95 is 1,531.8ms, login p95 is 819.3ms, page p95
+  is 761.0ms, WebSocket connect p95 is 758.8ms, subscribe ack p95 is 7ms, and
+  subscription registration p95 is 0.17ms.
+- Typed reverse-index rows (`20260514231954`) keep setup p95 in the same range
+  at 1,511ms while lowering real persistence batch p95 to 546.6ms.
+- Lifecycle cancellation (`20260514232743`) lowers cold churn setup p95 to
+  1,097.75ms, leaves subscription and reverse-index table counts at zero after
+  disconnect, and keeps real persistence batch p95 at 137.6ms.
+
+Work:
+
+- Keep subscription storage, dispatching, and ActionCable broadcast bus
+  configuration as separate contracts.
+- Keep last-seen timestamps as explicit maintenance writes, not subscribe-time
+  writes.
+- Add expiry pruning for stale subscription and reverse-index rows.
+- Deduplicate replacement subscriptions for the same subscriber and page where
+  the old graph is no longer useful.
+- Keep per-delivery retry and queue bounds visible in metrics.
+- Move durable write work out of the Puma process or behind an app queue adapter
+  before claiming high-churn multi-worker support.
+
+Exit criteria:
+
+- Reverse-index size does not grow unbounded under navigation churn.
+- Channel disconnect cleanup is covered by unit tests and the cold churn gate.
+- Expiry can be run safely from app jobs or maintenance tasks.
+
+## Phase 5: Deoptimization Diagnostics
+
+Goal: keep the deoptimization surface small and actionable without mixing
+benchmarking into the runtime path.
+
+Status: collection operation fallbacks report deoptimization reasons in planner
+notifications and Turbo Stream reports. Opaque Active Record collection
+boundaries now raise in development/test and warn/refuse subscription
+registration when configured for production-style warnings. Request-capture
+telemetry reports refusal/deoptimization context, and the README now documents
+what is not capturable, why, and the developer experience for each class.
+Refused-boundary benchmark aggregation, source locations, and complete
+refactor suggestions are still pending.
+
+Work:
+
+- Keep successful cheap operations free of diagnostic noise.
+- Report why a create/update/destroy deoptimized to render-site replacement.
+- Keep only "live deopts" where correctness is already proven and Upkeep merely
+  lost a cheaper operation.
+- Refuse boundaries when dependency, identity, replay, or DOM target proof is
+  missing and no safe enclosing target exists.
+- Remove predicate/table broad fallback from collection subscription capture;
+  keep table coverage only for write-event description where no subscription is
+  registered from it.
+- Include a stable reason and a concrete refactor suggestion with every refused
+  boundary.
+- Keep transport retry/backpressure reporting in the lifecycle lane, not the
+  invalidation deoptimization surface.
+
+Exit criteria:
+
+- A report can distinguish "optimized operation proved" from "deoptimized to
+  replacement because proof failed."
+- A report can distinguish "live deopt" from "refused boundary."
+- Deoptimization names are stable enough for app logs and benchmark summaries.
+
+## Phase 6: Benchmarks And Telemetry
+
+Goal: make the cost model inspectable in production and benchmarks.
+
+Status: planner counts, payload bytes, envelope counts, render counts, render
+duration, render-group attribution, shape-cache timing, server-side subscribe
+timing, and request-capture phase timing have landed in the identity-free feed
+reports. Reverse-index lookup timing, mutation write phase timing, client/server
+setup correlation, broadcast timing, and broader benchmark summaries are still
+pending.
+
+Work:
+
+- Instrument reverse-index lookup time and candidate count.
+- Instrument mutation write phases: database write, invalidation planning,
+  stream build, ActionCable enqueue, and response return.
+- Correlate client setup phases with server timings: page response, WebSocket
+  open, cable confirmation, and subscription confirmation.
+- Instrument broadcast time and end-to-end dispatch time.
+- Add a multi-worker benchmark gate that uses the ActiveRecord subscription
+  store, a shared ActionCable adapter, and enough writer VUs to exercise
+  delivery across workers.
+- Add benchmark summaries that compare Upkeep against Turbo refresh and against
+  hand-written Turbo stream operations separately.
+
+Exit criteria:
+
+- A benchmark run can say where time went without reading logs.
+- Upkeep claims are split by baseline: refresh replacement, manual stream
+  replacement, and identity-partitioned rendering.
+
+## Large-Scale Direction: Proven Incremental Rendering
+
+Goal: make Upkeep a deterministic incremental rendering system, not an
+automatic Turbo wrapper with heuristic fallbacks.
+
+The large-scale rule is: prove, compile, or refuse/deopt visibly. Production
+behavior should not depend on "probably safe" invalidation, hidden broad
+fallback, or local-only state that can miss another worker's subscriptions.
+
+The parallel Herb roadmap is the template-structure track for this direction:
+Herb should provide source-derived manifests, stable frame/render-site
+addresses, manifest fingerprints, diff classification, and developer guidance.
+Rails runtime observation still owns the data and identity surfaces a subscriber
+actually read.
+
+### Strict Mode
+
+Strict mode is the production posture for large installations.
+
+- Keep live deoptimizations only when Upkeep has already proven the write
+  affects the subscriber and can safely re-render a known enclosing target.
+- Refuse reactive boundaries when table sources, predicates, identity inputs,
+  replay inputs, or DOM targets cannot be proven.
+- Raise refused boundaries in development/test so apps learn about unsupported
+  shapes before production.
+- Warn and refuse registration in production for unsafe boundaries. Production
+  warning must not silently widen into broad reactivity.
+- Treat manifest/runtime mismatches as named refused-boundary or live-deopt
+  reasons such as missing manifest, stale fingerprint, helper-hidden
+  collection, multi-root partial, or missing render site.
+- Keep the development/debug path available, but make production claims against
+  strict behavior only.
+
+Exit criteria:
+
+- Development/test raises for unsafe reactive boundaries.
+- Production warns and marks unsafe boundaries non-reactive.
+- Benchmark reports separate strict successful proofs from deoptimized or
+  non-reactive frames.
+
+### Deoptimization Policy
+
+Use two buckets:
+
+- Live deopt: correctness is proven, but a cheaper Turbo Stream operation could
+  not be proven.
+- Refused boundary: correctness is not proven, so Upkeep must not register the
+  boundary implicitly.
+
+Live deopt examples:
+
+```ruby
+Card.where(board_id: board.id).order(:position)
+```
+
+If a create cannot be proven as append or prepend, replace the render site and
+report `collection_create_position_unproven`.
+
+```ruby
+Card.where(board_id: board.id).order(:title)
+```
+
+If an update may move the member, replace the render site and report
+`collection_member_replace_unproven`.
+
+```ruby
+Card.transaction do
+  first.update!(title: "A")
+  second.update!(title: "B")
+end
+```
+
+If multiple changes cannot be reduced to deterministic member operations,
+replace the render site and report `collection_multi_change_fallback`.
+
+Refused-boundary examples and guidance:
+
+```ruby
+Card.where("status = ?", "open")
+```
+
+Reason: opaque predicate. Refactor to structural Active Record or Arel:
+
+```ruby
+Card.where(status: "open")
+```
+
+```ruby
+Card.joins("INNER JOIN authors ON authors.id = cards.author_id")
+```
+
+Reason: opaque table source. Refactor to an association or structural Arel
+join:
+
+```ruby
+Card.joins(:author)
+```
+
+```erb
+<%= card_list(@cards) %>
+```
+
+Reason: helper-hidden collection. Refactor to Rails collection rendering or a
+future explicit supported boundary:
+
+```erb
+<%= render partial: "cards/card", collection: @cards, as: :card %>
+```
+
+```erb
+<li id="<%= dom_id(card) %>"><%= card.title %></li>
+<li>extra</li>
+```
+
+Reason: multi-root member partial. Refactor member partials to a single root
+when member-level operations are expected. If the parent render site is known,
+Upkeep may keep the boundary live by replacing the render site; otherwise it
+refuses the boundary.
+
+Exit criteria:
+
+- Every refused boundary includes reason, source location where available, and
+  at least one refactor suggestion.
+- Production reports count refused boundaries separately.
+- Benchmarks exclude refused boundaries from strict performance claims.
+
+### Herb Manifest Structure Layer
+
+Large-scale Upkeep needs one template-structure authority. The Herb worktree is
+moving in that direction: a `TemplateManifest` describes parse status, root
+shape, render sites, helper-lowered HTML, frontend tag targets, and a stable
+fingerprint.
+
+- Use Herb as the compile-time structural layer for Rails views.
+- Keep Rails runtime observation as the source of data reads, identity reads,
+  and actual subscriber ownership.
+- Apply source instrumentation before ActionView compiles supported ERB
+  templates, then retire post-render marker inference when equivalence is
+  proven.
+- Attach manifest path and fingerprint provenance to page, fragment, and
+  render-site frames.
+- Check selected targets against manifest entries and report concrete
+  deoptimization reasons when runtime shape does not match source topology.
+- Use `Herb.diff`-style classification to distinguish content-only manifest
+  refreshes, topology rebuilds, and unsafe full rebuilds.
+- Turn manifest coverage and fallback reasons into developer guidance so apps
+  can improve strict-mode eligibility intentionally.
+
+Exit criteria:
+
+- Herb manifests discover every render site and single-root fragment used by
+  maintained benchmark apps.
+- Runtime frames and selected targets either match manifest provenance or carry
+  stable deoptimization reasons.
+- ActionView integration produces equivalent HTML except for expected Upkeep
+  markers.
+- Herb becomes the single template structure path after the production gates
+  pass; duplicate non-Herb marker/replay paths are removed.
+
+### Typed Subscription Indexes
+
+The ActiveRecord store should keep durable state inspectable before it moves all
+lookup access onto fully typed columns.
+
+Status: the in-memory and persisted reverse-index key set now avoids table-only
+collection candidates, avoids any-id attribute rows for record-specific
+dependencies, and no longer indexes identity/request dependencies that
+lifecycle events cannot select. Persisted recorder snapshots use versioned
+JSON-compatible payloads instead of opaque binary Ruby object snapshots, so rows
+can be inspected without object deserialization. Durable subscription rows now
+keep replay, frame, and identity graph data; lifecycle dependency payloads live
+in typed reverse-index rows. Reverse-index rows group owner ids for the same
+subscription/lookup/dependency tuple.
+
+Measured typed-row benefit:
+
+- `benchmark/store_perf.rb`: cold persistent attribute lookup dropped from
+  38.8ms/op to 23.2ms/op, and cold collection lookup dropped from 38.7ms/op to
+  22.7ms/op. Durable drain is effectively flat in the synthetic store benchmark
+  because row count is unchanged.
+- `matrix/cold_connect_churn_chat` upkeep-only gate (`20260514231954`): real
+  persistence batch p95 is 546.6ms; setup p95 remains 1,511ms, so this is a
+  stale/multi-worker lookup optimization rather than a cold-admission fix.
+
+Work:
+
+- Add database indexes that match the typed planner access pattern instead of
+  querying only by lookup digest.
+- Keep JSON snapshots only for replay/debug payloads and collection metadata
+  that cannot be represented structurally yet.
+- Extend typed rows for future dependency keys: lifecycle, target id,
+  subscriber/identity digest, and target kind.
+
+Exit criteria:
+
+- A row-level write can query the reverse index through typed indexed columns.
+- Persisted lookup does not need to unmarshal lookup-key or lifecycle dependency
+  snapshots to prove a match.
+
+### Store Versioning And Worker Coherence
+
+Multi-worker correctness needs a shared coherence signal.
+
+- Add a monotonic subscription-store version that advances when subscriptions
+  or index rows are added, pruned, or materially changed.
+- Let each worker track the version covered by its active in-process registry.
+- Use the active registry without a warm-path `COUNT(*)` only when the local
+  version is current.
+- Fetch deltas or fall back to persisted lookup when the worker is stale.
+
+Exit criteria:
+
+- Warm same-worker planning avoids repeated count queries.
+- Cross-worker planning still sees subscriptions registered by another worker.
+- Benchmarks can report active, persistent, and stale-worker lookup modes.
+
+### Compiled Subscription Plans
+
+Subscription capture should produce compact execution artifacts.
+
+- Compile the render graph into dependency rows, replay recipes, target
+  addresses, identity signatures, and operation-proof inputs at registration
+  time.
+- Reference source-derived manifest entries from replay recipes where Herb
+  proves the frame or render-site address.
+- Avoid repeatedly interpreting large recorder snapshots during planning.
+- Store graph snapshots as debug/replay evidence, not the primary planning
+  data structure.
+- Keep compiled artifacts deterministic and rebuildable from captured evidence.
+
+Exit criteria:
+
+- Planning uses compiled dependency/target artifacts for common paths.
+- Recorder graph hydration is not required for every invalidation lookup.
+
+### Exact Collection Delta Proofs
+
+Large scale depends on narrow operations, not render-site replacement.
+
+- Extend collection proofs beyond append, prepend, remove, and stable member
+  replace.
+- Prove membership moves using old/new values, relation predicates, and order
+  expressions.
+- Emit deterministic Turbo Stream operations when relation shape proves the
+  exact delta.
+- Refuse or deopt visibly when limit, offset, grouping, custom SQL, or
+  unobservable order semantics make the delta unprovable.
+
+Exit criteria:
+
+- Common create/update/destroy paths on ordered filtered collections avoid full
+  render-site replacement.
+- Move and membership-change proofs have success and deoptimization tests.
+
+### Content-Addressed Render Dedup
+
+Fanout should scale with distinct output, not raw subscriber count.
+
+Status: delivery groups planned targets by action, target, identity signature,
+sharing signature, replay recipe signature, and deoptimization reason before
+rendering. Public fragments with equivalent replay inputs now render once for
+multiple subscribers; shared render-site streams already use the same grouping
+boundary.
+
+- Key rendered payloads by target recipe, identity signature, data version, and
+  operation kind.
+- Share rendered Turbo Stream payloads across subscribers with equivalent
+  target, identity, and data inputs.
+- Cache only proof-valid render outputs; do not share when identity or ambient
+  dependencies differ.
+- Report subscriber-renders saved by content-addressed dedup in benchmark
+  output.
+
+Exit criteria:
+
+- A write to a shared/public fragment renders once for many subscribers.
+- Identity-specific output produces separate payloads only when visible bytes
+  or identity dependencies differ.
+
+### Partitioned Indexes
+
+The reverse index should not behave like one global table at large scale.
+
+- Partition by tenant, stream namespace, app-defined shard key, or canonical
+  subscriber scope where Rails can prove the boundary.
+- Carry partition keys through subscription capture, dependency rows, and
+  lifecycle events.
+- Reject or deopt boundaries where the partition key is not structurally
+  observable.
+
+Exit criteria:
+
+- A tenant-scoped write queries a bounded partition.
+- Benchmarks can compare global lookup versus partitioned lookup cost.
+
+### Deterministic Coalescing
+
+Batching should be semantics-preserving, not timing-based guesswork.
+
+- Coalesce work by exact target key, data version, operation kind, and
+  supersession rules.
+- Drop earlier work only when a later operation deterministically supersedes it
+  for the same target.
+- Preserve ordering when operations are not provably commutative or
+  superseding.
+
+Exit criteria:
+
+- Bursty writes to the same target produce bounded delivery work.
+- Coalescing reports why work was merged, superseded, or preserved.
+
+### ActiveJob Dispatcher
+
+Queueing should isolate latency after correctness is already deterministic.
+
+- Add a dispatcher boundary that can enqueue delivery through ActiveJob.
+- Let host apps use Solid Queue, Redis-backed queues, or another configured
+  queue adapter without changing the subscription store.
+- Keep subscription store, dispatcher, and ActionCable broadcast bus separate.
+- Build fork/process lifecycle hooks before claiming multi-worker support.
+
+Exit criteria:
+
+- The request thread can hand off delivery work to ActiveJob.
+- Queue retry/backpressure is reported separately from invalidation proof and
+  render cost.
+
+## Benchmark Positioning
+
+The benchmark matrix should keep these baselines separate:
+
+- Turbo refresh: Upkeep should aim to reduce page GET fanout and full-page
+  rerender cost.
+- Turbo append/prepend/remove: Turbo should be expected to win when the stream
+  operation is manually known.
+- Identity-specific output: compare correctness and per-identity render cost,
+  not only payload latency.
+
+The first production milestone is a warm-subscription filtered collection page
+where Upkeep beats Turbo refresh with explicit change events, predicate-aware
+invalidation, stale subscription cleanup, and cost attribution.