yrby-rails 0.5.0 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 805c1e286dbc5d7fd802c9b6a1a70fce29b9ea10b6ba1e45966c5c29507cc4c2
4
- data.tar.gz: e5939ca7311f642d1685ad785f8ea8b312d6ae20f59661139f481b95a1253fb2
3
+ metadata.gz: fdd96efe432c38bca8b5b8eb31c69cea768361f2ddd89d239819820b1a483bd1
4
+ data.tar.gz: 8b7b3571f7aa632468f10eb4c53f6399ec107660b66a4bc5e020111bca038f13
5
5
  SHA512:
6
- metadata.gz: aa51adae8ea5fc9259453730d24a609c7faf990ceef1a6c4bce18a49123d6222a1402695bcb9d44e3e53dd634ab560522ee03f705c892d7dfd188121205cfbb8
7
- data.tar.gz: '09ada40a8d1e1989a7a514e8b06ea5128d0fcb3cee79a59c61430681634b54d6f5ee7db717da3c9b94ad07137ef7d49367ace024f5d5ac0cf377925375ea0bdf'
6
+ metadata.gz: 0aba33eed57da270c02632e475792bc8839e6831a26d9b9e48a9a8103ecfc5d88a47c4068a24477df89e98485dce24ef3d51eaa15c1496780ee3a0ac692d1e0a
7
+ data.tar.gz: e62c580f3c82b1c7ace0a6e8ecd62b416f1abfecf7f8b9f8f1a43e441f29b5d9efadffdb228575ba4ef7d8999179fbfb3bd5c3879089e9747f472f8750f99084
data/CHANGELOG-rails.md CHANGED
@@ -1,17 +1,59 @@
1
- # Changelog yrby-rails
1
+ # Changelog: yrby-rails
2
2
 
3
3
  All notable changes to the `yrby-rails` gem (formerly `yrby-actioncable`) are
4
4
  documented here. The
5
5
  format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and
6
6
  this project aims to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [0.6.0] - 2026-08-11
9
+
10
+ ### Changed
11
+
12
+ - **Causal gaps are now accepted.** A causally-incomplete update, one whose
13
+ causally-prior update the store hasn't seen, is recorded and acked like any
14
+ other (ack-on-durable) instead of being rejected with a resync request,
15
+ and served onward like any other state (a peer parks a pending struct
16
+ exactly as the server does). The gap heals through the ack loop: the
17
+ missing dependency is an update its own sender still holds unacked and
18
+ keeps retransmitting, and join or reconnect handshakes let any client
19
+ that holds it supply it. The write path no longer rebuilds the document
20
+ per update:
21
+ it appends, relays, and acks, so a lost-ack retry records again (replay
22
+ converges; CRDT apply is idempotent).
23
+
24
+ This tightens the store contract: `on_load` must preserve pending
25
+ (`encode_state_as_update` or a replayed raw append log), compaction must
26
+ never fold a pending update into a gap-free snapshot and drop the raw
27
+ row (the bundled `Y::Document` quarantines pending rows and folds only
28
+ clean ones), and `on_change` must tolerate duplicate deltas. An acked
29
+ update that leaves durable storage before it integrates is a silent
30
+ data loss.
31
+
32
+ ### Changed (Y::Document)
33
+
34
+ - Compaction folds past an open gap. A batch holding a causal gap still
35
+ compacts everything integrable: the fold captures every struct that
36
+ integrates, rows independent of the gap included, and only the gap
37
+ tail survives as quarantined raw rows. A healed gap folds out at the
38
+ next compaction. Rows are judged per row against the folded state, so
39
+ a row that causally builds on the gap quarantines with it and an
40
+ acked update never leaves the table before its content is durably in
41
+ state.
42
+
43
+ ### Added
44
+
45
+ - `on_gap` channel hook: fires with the document key at join/serve time
46
+ whenever the loaded document still holds a causal gap, for metrics on
47
+ unhealed gaps (which no longer surface as resync traffic). An open gap is
48
+ also logged at `info`.
49
+
8
50
  ## [0.5.0] - 2026-08-05
9
51
 
10
52
  ### Added
11
53
 
12
54
  - `Y::EncryptedDocument` / `Y::EncryptedDocumentUpdate`: document storage
13
55
  encrypted with Active Record encryption (`state` and update payloads),
14
- on the same tables the class you access through decides the
56
+ on the same tables; the class you access through decides the
15
57
  cryptography, the way `ActionText::EncryptedRichText` does. Point a
16
58
  channel's `on_load`/`on_change` (or a record association) at
17
59
  `Y::EncryptedDocument` and configure the app's encryption keys. Keep
@@ -55,7 +97,6 @@ this project aims to follow [Semantic Versioning](https://semver.org/spec/v2.0.0
55
97
  - `rails generate yrby:install`: a `DocumentChannel` speaking the
56
98
  y-websocket protocol over the gem-owned storage, plus the storage
57
99
  migration.
58
-
59
100
  ## [0.3.1] - 2026-07-01
60
101
 
61
102
  ### Removed
@@ -80,7 +121,7 @@ Fixes from a full source review:
80
121
  - **A lost-ack retry now re-broadcasts.** If the original attempt recorded the
81
122
  update and then crashed (or the pub/sub broadcast failed) before
82
123
  distributing, the retry was previously settled as `:applied` without
83
- re-broadcasting live subscribers stayed stale until their next full resync,
124
+ re-broadcasting; live subscribers stayed stale until their next full resync,
84
125
  and nothing else could reach them. The retry now re-broadcasts before acking;
85
126
  idempotent CRDT apply makes the duplicate free for every receiver.
86
127
  - **A missing document key now fails closed.** Under a transport that doesn't
@@ -101,10 +142,10 @@ Fixes from a full source review:
101
142
  ### Changed
102
143
  - Raised the `yrby` floor to `>= 0.3.0`. That release makes
103
144
  `Doc#handle_sync_message` answer `SyncStep1` with integrated-only (gap-free)
104
- state it no longer serves un-integrable pending structs, which previously
145
+ state: it no longer serves un-integrable pending structs, which previously
105
146
  poisoned peers and drove endless resync traffic. The sync channel serves its
106
147
  SyncStep2 response through that method, so with an older core a poisoned server
107
- store would still hand the gap to clients. No code change here pinning the
148
+ store would still hand the gap to clients. No code change here; pinning the
108
149
  floor makes gap-free serving self-enforcing instead of dependent on the app
109
150
  updating the core gem.
110
151
 
@@ -116,7 +157,7 @@ Fixes from a full source review:
116
157
  record-before-distribute on `update_advances?` (`return :applied unless
117
158
  doc.update_advances?(update)`), so with an older core a lost-ack retry of a
118
159
  deletion the server had already integrated was re-recorded and re-broadcast
119
- each time. No code change here pinning the floor just makes the gem's
160
+ each time. No code change here; pinning the floor just makes the gem's
120
161
  exactly-once durable-recording guarantee self-enforcing instead of dependent on
121
162
  the app updating the core gem.
122
163
 
@@ -124,7 +165,7 @@ Fixes from a full source review:
124
165
 
125
166
  ### Changed
126
167
  - **Internal:** ActionCable stream-name prefix `y_ruby:` → `yrby:`.
127
- Server-internal (broadcast + `stream_from` both use it) no public API or
168
+ Server-internal (broadcast + `stream_from` both use it), no public API or
128
169
  client-facing wire change. Depends on `yrby >= 0.2.1`.
129
170
 
130
171
  ## [0.2.0] - 2026-06-28
data/README.md CHANGED
@@ -2,13 +2,43 @@
2
2
 
3
3
  [![CI](https://github.com/jpcamara/yrby/actions/workflows/ci.yml/badge.svg)](https://github.com/jpcamara/yrby/actions/workflows/ci.yml)
4
4
 
5
- Collaborative editing for Rails, backed by [y-crdt](https://github.com/y-crdt/y-crdt)
6
- (the Rust library behind Y.js). Your Rails server speaks the y-websocket sync
7
- protocol directly, so there's no separate Node process hosting the Y.js
8
- documents. Pronounced "yer-bee".
5
+ yrby (pronounced "yer-bee") makes Rails a real Yjs backend. It binds
6
+ [y-crdt](https://github.com/y-crdt/y-crdt), the Rust engine behind Y.js, into
7
+ Ruby, and builds the rest of the stack around it: a sync server for Action
8
+ Cable and AnyCable, a browser provider, and server-side reading and rendering
9
+ of the documents. Real-time collaboration in a Rails app with no Node process
10
+ anywhere in the path.
9
11
 
10
12
  ![Two people typing on separate lines of the same document, each keystroke synced through a Rails server, seen from a third browser with labeled carets](docs/images/collab.gif)
11
13
 
14
+ On the server, `yrby-rails` implements the full y-websocket protocol
15
+ (document sync plus presence) as a channel concern. Its delivery contract is
16
+ stricter than the usual Yjs servers: every update is ack-tracked and durably
17
+ recorded before it is acknowledged or broadcast to anyone. Replaying your
18
+ store always rebuilds the document, across any number of processes.
19
+ ([Delivery guarantees](#delivery-guarantees))
20
+
21
+ In the browser, `yrby-client`'s `ActionCableProvider` connects anything that
22
+ speaks Yjs. The demo app runs four rich text editors, and CI drives each one
23
+ in real Chrome: Tiptap, [Lexxy](https://www.npmjs.com/package/lexxy-realtime),
24
+ Rhino Editor, and CodeMirror. The same channel also syncs Yjs shapes with no
25
+ editor at all: a whiteboard on a `Y.Map`, a kanban board on a `Y.Array`, a
26
+ co-filled form. ([Editors](#editors))
27
+
28
+ In Ruby, the documents are readable without a browser. `Doc#read_text` and
29
+ `Doc#read_map` reconstruct contents for search, validation, and exports.
30
+ `Y::Tiptap` and `Y::Lexxy` render a document to HTML byte-identical to the
31
+ editor's own serializer, take rules for your app's custom nodes, and drop
32
+ straight into ActionText. ([Rendering to HTML](#rendering-to-html))
33
+
34
+ Underneath, the core is built for a production Rails deployment. A `Doc` is
35
+ thread-safe across Puma and ActionCable threads. Native CRDT work runs with
36
+ the GVL released, so it parallelizes on MRI. Incoming frames are validated
37
+ before anything processes them, and multi-process and AnyCable setups are
38
+ tested end to end. ([Thread Safety](#thread-safety))
39
+
40
+ The whole server side of a collaborative document is one channel:
41
+
12
42
  ```ruby
13
43
  class DocumentChannel < ApplicationCable::Channel
14
44
  include Y::ActionCable
@@ -21,31 +51,36 @@ class DocumentChannel < ApplicationCable::Channel
21
51
  end
22
52
  ```
23
53
 
24
- On the browser, use the `ActionCableProvider` from the
25
- [`yrby-client`](https://www.npmjs.com/package/yrby-client) npm package.
26
- Integrates with any editor that includes Y.js support, such as Tiptap, ProseMirror
27
- and [Lexxy](https://www.npmjs.com/package/lexxy-realtime).
28
-
29
- ## Usage
30
-
31
- Install the gem and npm package:
54
+ Install the gem and the npm package:
32
55
 
33
56
  ```
34
- gem install yrby-actioncable # depends on yrby
57
+ gem install yrby-rails # depends on yrby
35
58
  npm install yrby-client
36
59
  ```
37
60
 
38
- ## What you get
39
-
40
- - A thread-safe Ruby `Doc` you can share across Ruby threads/fibers, and native CRDT work
41
- runs with the GVL released.
42
- - The y-websocket protocol (document sync plus awareness/presence) as a
43
- one-include ActionCable concern.
44
- - Authoritative record-before-distribute semantics: each document change can be
45
- recorded durably before it goes out to anyone.
46
- - Optional server-side reads: `Doc#read_text` and `Doc#read_map` reconstruct a
47
- document's contents in Ruby - no Node process - for search, exports, validation,
48
- or server-side rendering.
61
+ ## Contents
62
+
63
+ - [Scope](#scope)
64
+ - [Durability and delivery](#durability-and-delivery)
65
+ - [What about yrb?](#what-about-yrb)
66
+ - [Testing](#testing)
67
+ - [Install](#install)
68
+ - [Docs](#docs)
69
+ - [Editors](#editors)
70
+ - [Usage](#usage)
71
+ - [Doc (Low-Level Document Sync)](#doc-low-level-document-sync)
72
+ - [Reading document contents](#reading-document-contents)
73
+ - [Pending structs and gap-free state](#pending-structs-and-gap-free-state)
74
+ - [Rendering to HTML](#rendering-to-html)
75
+ - [Protocol codec (module functions)](#protocol-codec-module-functions)
76
+ - [ActionCable Integration](#actioncable-integration)
77
+ - [Thread Safety](#thread-safety)
78
+ - [Parallelism (GVL release)](#parallelism-gvl-release)
79
+ - [Message Type Constants](#message-type-constants)
80
+ - [Sync Flow](#sync-flow)
81
+ - [Development](#development)
82
+ - [License](#license)
83
+ - [Acknowledgments](#acknowledgments)
49
84
 
50
85
  ## Scope
51
86
 
@@ -63,17 +98,16 @@ guarantees, correctness, and thread safety.
63
98
 
64
99
  Towards that goal, `yrby` adds opinionated defaults on top of normal Yjs syncing:
65
100
 
66
- - Built-in update acknowledgement: the `ActionCableProvider` in `yrby-client` will continue to
67
- send updates until an ack is received from the server. [`yrby-actioncable`](https://rubygems.org/gems/yrby-actioncable)
68
- only sends an ack when applying an update is successful. The goal is at-least-once delivery,
69
- and because CRDTs are idempotent a duplicate update is effectively a no-op.
70
- - Gap detection in document updates: before applying an update and sending an ack to the client,
71
- `yrby` checks whether the update results in any causal gap. Ie, an update comes through
72
- which depends on a previous update that is not yet present in the document. This can result in
73
- a document stuck with "pending" updates, which will _never_ apply if the missing update is not sent.
74
- To avoid this, `yrby` does not apply the update, and starts a new y-protocol sync with the client.
75
- That will cause the client to synchronize its document with the server, sending through any updates
76
- that may have been missed
101
+ - Built-in update acknowledgement: the `ActionCableProvider` in `yrby-client` keeps
102
+ sending an update until the server acks it, and [`yrby-rails`](https://rubygems.org/gems/yrby-rails)
103
+ only acks once the update is durably recorded. That gives you at-least-once
104
+ delivery, and because CRDT updates are idempotent a duplicate is a no-op.
105
+ - Gap awareness: an update can arrive before another update it depends on (a
106
+ "causal gap"). `yrby` records and acks it like any other, and the document
107
+ heals on its own once the missing update arrives; its sender keeps
108
+ retransmitting it until it is acked. `Doc#pending?` and the `on_gap` hook
109
+ tell you when a document is waiting on a missing update.
110
+ ([Causal gaps](#causal-gaps))
77
111
 
78
112
  ## What about [yrb](https://github.com/y-crdt/yrb)?
79
113
 
@@ -147,8 +181,8 @@ editor's own serializer. Each page is a working integration to copy from:
147
181
  | [Rhino Editor](https://github.com/KonnorRogers/rhino-editor) (Tiptap 3) | `@tiptap/extension-collaboration` + `-caret` | [`rhino.js`](examples/actioncable-demo/frontend/src/rhino.js) |
148
182
  | [CodeMirror 6](https://codemirror.net) | `y-codemirror.next` | [`codemirror.js`](examples/actioncable-demo/frontend/src/codemirror.js) |
149
183
 
150
- The demo also syncs plain Yjs shapes with no editor at all a whiteboard
151
- on a `Y.Map`, a kanban board on a `Y.Array`, a co-filled form over the
184
+ The demo also syncs plain Yjs shapes with no editor at all (a whiteboard
185
+ on a `Y.Map`, a kanban board on a `Y.Array`, a co-filled form) over the
152
186
  same channel. The demo README's "Using this in your own app" section has
153
187
  the integration recipe, and its `NoteMaterializer` shows how to render a
154
188
  document to ActionText server-side with `Y::Tiptap` or `Y::Lexxy`.
@@ -173,17 +207,19 @@ doc.compacted_state_update # => full update, gap-free (excludes pending)
173
207
  # Applying updates
174
208
  doc.apply_update(update_bytes) # apply raw V1 update
175
209
  doc.pending? # => true if holding un-integrable pending structs
210
+ doc.update_ready?(update) # => true if update would integrate cleanly (no gap)
211
+ doc.update_advances?(update) # => true if update moves integrated state forward
176
212
 
177
213
  # Sync protocol
178
214
  doc.sync_step1 # => SyncStep1 message (this doc's state vector)
179
215
  doc.handle_sync_message(data) # => [msg_type, sync_type, response]; answers a
180
- # peer's SyncStep1 with an integrated-only
181
- # SyncStep2 (never serves pending structs)
216
+ # peer's SyncStep1 with full state (lossless,
217
+ # pending included, like Y.js)
182
218
  ```
183
219
 
184
220
  ### Reading document contents
185
221
 
186
- Reconstruct a document server-side search, exports, emails, SSR with no
222
+ Reconstruct a document server-side (search, exports, emails, SSR) with no
187
223
  Node process:
188
224
 
189
225
  ```ruby
@@ -199,16 +235,18 @@ update), yrs parks it as a **pending** struct: the integrated state vector stays
199
235
  empty, but the pending block is held as a recovery buffer and heals if the
200
236
  missing dependency later arrives. `Doc#pending?` reports this.
201
237
 
202
- Pending structs are *not* document state, so they must not cross the sync
203
- boundary a peer that receives one can't integrate it and gets stuck. Two
204
- guarantees keep serving safe:
238
+ Pending structs travel like any other state. `handle_sync_message` answers
239
+ `SyncStep1` with the doc's full state, pending included, just like Y.js's
240
+ `encodeStateAsUpdate`: a peer parks the pending struct the same way this doc
241
+ did and heals it the same way. The one place pending must not go is a
242
+ compacted snapshot:
205
243
 
206
- - `handle_sync_message` answers `SyncStep1` with **integrated-only** state, so a
207
- server never serves a struct it can't integrate itself (this is automatic).
208
- - `Doc#compacted_state_update` gives you the same gap-free full-state update for
209
- when you persist or hand off state yourself. It's non-destructive (the doc
210
- keeps its pending), while `encode_state_as_update` stays lossless so you can
211
- still preserve the raw pending bytes for recovery.
244
+ - `Doc#compacted_state_update` returns a gap-free full-state update for
245
+ compaction. Folding a log into one blob would otherwise freeze an
246
+ un-integrable struct into the base state forever. It's non-destructive: the
247
+ doc keeps its pending.
248
+ - `encode_state_as_update` stays lossless, so persistence and serving keep
249
+ the raw pending bytes and the gap can still heal.
212
250
 
213
251
  ### Rendering to HTML
214
252
 
@@ -231,7 +269,7 @@ tiptap.to_html("content") # or another XML root
231
269
  The output matches Tiptap's own `getHTML()`, checked byte-for-byte in the tests
232
270
  against a document captured from a real editor. It follows
233
271
  [`tiptap-php`](https://github.com/ueberdosis/tiptap-php) and reads both name
234
- styles editors use Tiptap's `bulletList`/`bold` and prosemirror-schema-basic's
272
+ styles editors use: Tiptap's `bulletList`/`bold` and prosemirror-schema-basic's
235
273
  `bullet_list`/`strong`.
236
274
 
237
275
  It covers paragraphs, headings, blockquotes, bullet/ordered/task lists, code
@@ -241,8 +279,8 @@ as semantic `<table><tbody>`, without the column-width styling Tiptap's editor
241
279
  view adds.
242
280
 
243
281
  The support is layered like the Lexical side: `Y::ProseMirror` covers core
244
- ProseMirror natively prosemirror-schema-basic plus the prosemirror-tables
245
- family and Tiptap's extension nodes (task lists, mentions, the details
282
+ ProseMirror natively (prosemirror-schema-basic plus the prosemirror-tables
283
+ family) and Tiptap's extension nodes (task lists, mentions, the details
246
284
  family) are `Y::Tiptap`'s rule set (`Y::Tiptap::NODES`), built on the
247
285
  extension API below. Marks stay in the base: mark rendering (nesting order,
248
286
  `textStyle` CSS, `code` exclusivity) runs through native text-run machinery
@@ -259,8 +297,8 @@ lexxy.to_html("notepad") # or another XML root
259
297
 
260
298
  The HTML is identical to what a `lexxy-editor` submits to Rails (its `value`).
261
299
  The tests check this byte-for-byte against a document captured from a real
262
- editor. Stock Lexical has no canonical serializer every editor configures
263
- its own so the editor-specific class carries the editor's name, and
300
+ editor. Stock Lexical has no canonical serializer (every editor configures
301
+ its own), so the editor-specific class carries the editor's name, and
264
302
  `Y::Lexical` is the core-Lexical base: paragraphs, headings, quotes, code,
265
303
  lists, tables, links, and the full text-format model, for any other Lexical
266
304
  editor to extend with rules.
@@ -273,14 +311,14 @@ mentions both emit `<action-text-attachment>` elements that ActionText can
273
311
  re-render).
274
312
 
275
313
  Internally that support is layered: `Y::Lexical` covers core Lexical
276
- structure natively, and everything Lexxy adds its node types (attachments,
314
+ structure natively, and everything Lexxy adds, its node types (attachments,
277
315
  galleries) and its decorations of core nodes (the table wrapper, header-cell
278
- styling, nested-list classes) is `Y::Lexxy`'s rule set
316
+ styling, nested-list classes), is `Y::Lexxy`'s rule set
279
317
  (`Y::Lexxy::NODES`), built on the extension API below. The gem's own Lexxy
280
318
  support is the API's first consumer: an app rule for one of those types
281
319
  simply replaces it.
282
320
 
283
- In both renderers an unknown node keeps its content text and nested blocks
321
+ In both renderers an unknown node keeps its content: text and nested blocks
284
322
  fall back to readable markup rather than disappearing.
285
323
 
286
324
  #### Custom nodes and marks
@@ -290,7 +328,7 @@ their own node types. Both renderers take rules for them. A rule is checked
290
328
  before the built-in schema, so it can add a node type or replace how a
291
329
  built-in renders.
292
330
 
293
- Rules register in a block one `rules.node` call per type. A declarative
331
+ Rules register in a block, one `rules.node` call per type. A declarative
294
332
  rule is markup as data, rendered natively:
295
333
 
296
334
  ```ruby
@@ -304,14 +342,14 @@ end
304
342
  `tag` names the element. `attrs` values are templates: a string is a literal,
305
343
  a symbol reads that attribute off the node, an array concatenates both kinds;
306
344
  an attribute that resolves empty is left out. `text` (same template form)
307
- emits literal text content. `contains` declares what lives inside the node `:inline` (formatted text,
308
- the default), `:blocks` (child block nodes a container), or `:none` (a
345
+ emits literal text content. `contains` declares what lives inside the node: `:inline` (formatted text,
346
+ the default), `:blocks` (child block nodes, a container), or `:none` (a
309
347
  leaf). `void: true` skips the closing tag.
310
348
 
311
349
  You don't have to guess any of those names or shapes. Editors store types
312
350
  and attributes under names you'd never predict (Rhino's strike mark is
313
351
  `rhino-strike`; Lexical prefixes its own props `__`), so ask a real
314
- document instead make one in your editor using your custom node, then:
352
+ document instead: make one in your editor using your custom node, then:
315
353
 
316
354
  ```ruby
317
355
  Y::Tiptap.new(doc).node_types
@@ -336,25 +374,25 @@ lexical = Y::Lexical.new(doc) do |rules|
336
374
  end
337
375
  ```
338
376
 
339
- The block gets the node's type, its stored attributes, `node.content` the
340
- children, already rendered to HTML and `node.child_types`, the node's
377
+ The block gets the node's type, its stored attributes, `node.content` (the
378
+ children, already rendered to HTML), and `node.child_types`, the node's
341
379
  element/block children by type, in document order. `child_types` answers the
342
380
  structural questions attributes can't: how many images a gallery holds, or
343
381
  whether a list item carries a nested list. Whatever the block returns is
344
382
  spliced into the output as-is: it's trusted HTML, so escape any values you
345
- interpolate. To set the content mode for a callback, give the node both
383
+ interpolate. To set the content mode for a callback, give the node both:
346
384
  `rules.node "embed", contains: :blocks do |node| ... end`.
347
385
 
348
386
  Callbacks never run while the document is locked. The render finishes first
349
387
  (inside one read transaction, GVL released), then the blocks run and their
350
- output is spliced in so a callback can safely read or even write the same
388
+ output is spliced in, so a callback can safely read or even write the same
351
389
  doc. With no callback rules, `to_html` skips the splicing entirely.
352
390
 
353
391
  Blocks are the escape hatch for everything the declarative form can't say,
354
392
  and they're proven sufficient: `Y::Lexxy` and `Y::Tiptap` are themselves
355
- built on this API (`lib/y/lexxy.rb`, `lib/y/tiptap.rb`) simple nodes as
393
+ built on this API (`lib/y/lexxy.rb`, `lib/y/tiptap.rb`): simple nodes as
356
394
  declarative hashes, everything with logic as plain methods mapped by node
357
- type (a `Method` responds to `call` like any lambda) and the fixture tests
395
+ type (a `Method` responds to `call` like any lambda), and the fixture tests
358
396
  hold their output byte-identical to a live editor's.
359
397
 
360
398
  The ProseMirror side also takes custom marks:
@@ -371,7 +409,7 @@ for a built-in mark name (`"bold"`) replaces its built-in tag.
371
409
 
372
410
  ##### Worked examples
373
411
 
374
- A video-embed node from an app's Tiptap extension a type the pinned schema
412
+ A video-embed node from an app's Tiptap extension, a type the pinned schema
375
413
  has never heard of:
376
414
 
377
415
  ```ruby
@@ -398,7 +436,7 @@ tiptap = Y::Tiptap.new(doc) do |rules|
398
436
  end
399
437
  ```
400
438
 
401
- Overriding a shipped rule rendering Lexxy uploads as real image markup
439
+ Overriding a shipped rule: rendering Lexxy uploads as real image markup
402
440
  instead of the `<action-text-attachment>` elements ActionText re-renders:
403
441
 
404
442
  ```ruby
@@ -414,7 +452,7 @@ lexxy = Y::Lexxy.new(doc) do |rules|
414
452
  end
415
453
  ```
416
454
 
417
- Markup that depends on structure `node.child_types` lists the node's
455
+ Markup that depends on structure: `node.child_types` lists the node's
418
456
  element/block children in document order, so a layout container can size
419
457
  itself by its column count while the columns themselves stay declarative:
420
458
 
@@ -427,7 +465,7 @@ tiptap = Y::Tiptap.new(doc) do |rules|
427
465
  end
428
466
  ```
429
467
 
430
- Content-aware overrides dropping the empty paragraphs an editor keeps
468
+ Content-aware overrides: dropping the empty paragraphs an editor keeps
431
469
  around the cursor, since `node.content` arrives already rendered:
432
470
 
433
471
  ```ruby
@@ -438,10 +476,10 @@ lexical = Y::Lexical.new(doc) do |rules|
438
476
  end
439
477
  ```
440
478
 
441
- For a larger reference, the gem's own editor schemas ship this way see
479
+ For a larger reference, the gem's own editor schemas ship this way; see
442
480
  `Y::Lexxy::NODES` in `lib/y/lexxy.rb` (declarative hashes for the simple
443
- nodes, a plain method per node that needs logic galleries, list items,
444
- header cells, both attachment types mapped with `method(:name)`) and
481
+ nodes, a plain method per node that needs logic (galleries, list items,
482
+ header cells, both attachment types) mapped with `method(:name)`) and
445
483
  `Y::Tiptap::NODES` in `lib/y/tiptap.rb` (task lists, mentions, the details
446
484
  family).
447
485
 
@@ -449,7 +487,7 @@ family).
449
487
 
450
488
  Classifying and unwrapping wire frames is stateless, so it's exposed as
451
489
  `Y` module functions rather than a class. The server never holds presence
452
- or document state to route a frame presence lives in the browser clients, and
490
+ or document state to route a frame; presence lives in the browser clients, and
453
491
  the server only relays awareness frames opaquely.
454
492
 
455
493
  ```ruby
@@ -470,31 +508,31 @@ bin/rails db:migrate
470
508
  The models ship in the gem, the way Action Text owns
471
509
  `ActionText::RichText`:
472
510
 
473
- - **`Y::Document`** one row per document, addressed two ways: by `key`
474
- (what a channel addresses one opaque, unique string, sometimes
511
+ - **`Y::Document`**: one row per document, addressed two ways: by `key`
512
+ (what a channel addresses; one opaque, unique string, sometimes
475
513
  app-supplied, never parsed) and, optionally, by polymorphic `record` +
476
514
  `name` (which model attribute it backs; `name` is the attribute name,
477
- `"body"` one document per attribute per record, the
515
+ `"body"`; one document per attribute per record, the
478
516
  ActionText::RichText scheme). Key-only documents leave the binding nil.
479
517
  Either side can arrive first: `Y::Document.for(record, name)` finds or
480
518
  creates the binding, derives a readable key (`post/1/body`), and adopts
481
519
  a key-only row already holding that key, so a channel writing first and
482
520
  a binding created later converge on one document. The row also holds
483
- the merged `state` snapshot CRDT state only; derived data (rendered
521
+ the merged `state` snapshot, CRDT state only; derived data (rendered
484
522
  HTML, search text) is the application's job, typically in the channel's
485
523
  on_change. `.load_state(key)` / `.append(key, update)` are the store
486
524
  calls the generated channel uses.
487
- - **`Y::DocumentUpdate`** the uncompacted tail: one delta per row,
525
+ - **`Y::DocumentUpdate`**: the uncompacted tail, one delta per row,
488
526
  compacted into `state` and deleted once the tail reaches `compact_every`
489
527
  (default 64). Loading reads the snapshot plus the current tail; an
490
528
  empty tail returns `state` directly. Compaction serializes on a
491
- per-document row lock and skips causally-gapped rows they're
529
+ per-document row lock and skips causally-gapped rows; they're
492
530
  quarantined until they heal rather than compacted into state or
493
531
  deleted. Destroying a document deletes its updates with it.
494
532
 
495
533
  Encrypted storage: `Y::EncryptedDocument` stores `state` and update
496
534
  payloads through Active Record encryption on the same tables, the way
497
- `ActionText::EncryptedRichText` does point the channel's
535
+ `ActionText::EncryptedRichText` does. Point the channel's
498
536
  `on_load`/`on_change` at it instead and configure your app's encryption
499
537
  keys. Use one access path per document: rows written encrypted read back
500
538
  as ciphertext through the plain classes.
@@ -508,8 +546,7 @@ answered, and they can point at anything.
508
546
 
509
547
  `include Y::ActionCable` (from the `yrby-rails` gem) is the channel
510
548
  integration: the y-websocket protocol (document sync +
511
- awareness/presence) over ActionCable. (`include Y::ActionCable::Sync`
512
- keeps working and has the same effect.)
549
+ awareness/presence) over ActionCable.
513
550
 
514
551
  ```ruby
515
552
  # app/channels/document_channel.rb
@@ -537,8 +574,8 @@ end
537
574
  ```
538
575
 
539
576
  The concern is store-backed. A handshake is answered from `on_load`; document
540
- changes are checked against that durable state, recorded through `on_change`,
541
- then broadcast. Nothing authoritative is kept in ActionCable process memory, so
577
+ changes are recorded through `on_change`, then broadcast. Nothing
578
+ authoritative is kept in ActionCable process memory, so
542
579
  AnyCable RPC workers, Puma workers, and separate dynos can all handle messages
543
580
  for the same document as long as they share the same store and cable adapter.
544
581
 
@@ -556,16 +593,18 @@ no single client can relay garbage that breaks the others in a room.
556
593
 
557
594
  #### Delivery guarantees
558
595
 
559
- The contract is the same at every scale one process, or hundreds across many
596
+ The contract is the same at every scale: one process, or hundreds across many
560
597
  servers:
561
598
 
562
599
  - **The document always converges.** CRDT updates are commutative and
563
600
  idempotent, so out-of-order, duplicate, or concurrent delivery all converge to
564
601
  the same correct document. This needs no coordination and holds everywhere.
565
- - **The durable log never goes gappy.** An update is recorded only once its
566
- causal dependencies are already in the store (checked against `on_load`); a
567
- causally-incomplete update triggers a resync instead, so the log always
568
- rebuilds cleanly.
602
+ - **An acked update is durable, even one that arrived out of order.** An
603
+ update with a missing dependency is recorded and acked like any other, and
604
+ parks as pending in the document. That missing dependency is an update some
605
+ client still holds unacked, so that client keeps retransmitting it until
606
+ the server records it, and the gap closes. The ack loop is the guarantee.
607
+ See [Causal gaps](#causal-gaps).
569
608
  - **`on_change` is at-least-once, and the durable guarantee is that replaying the
570
609
  log reconstructs the document.** Every update triggers `on_change` before it's acked or
571
610
  broadcast (record-before-distribute). If exactly-once updates matter for you, **you
@@ -575,13 +614,13 @@ servers:
575
614
  There is no negative-ack: the client simply never receives the ack, keeps the
576
615
  update pending, and retransmits on its timer/reconnect. This is built for
577
616
  *transient* failures (the store is briefly down → a retry lands). A block that
578
- raises *deterministically* a validation that always fails for this edit
617
+ raises *deterministically* (a validation that always fails for this edit)
579
618
  will be retried forever, since nothing tells the client to stop. Enforce hard
580
619
  rejections before the edit reaches `on_change` (channel authorization in
581
620
  `subscribed`), not by raising inside it.
582
621
  - **An over-cap frame is dropped the same silent way.** A frame larger than
583
- `max_frame_bytes` (default 8 MiB) is dropped before decoding no ack, no
584
- broadcast to bound the work a client can force. For a genuine document
622
+ `max_frame_bytes` (default 8 MiB) is dropped before decoding (no ack, no
623
+ broadcast) to bound the work a client can force. For a genuine document
585
624
  update that means the same implicit rejection as above: unacked, retransmitted
586
625
  forever. Normal typing never approaches the cap, but a large paste, an embedded
587
626
  image, or a big initial `SyncStep2` can. The drop is logged (`warn` for
@@ -591,6 +630,85 @@ servers:
591
630
  genuinely-too-big content upstream rather than relying on the cap to reject it
592
631
  gracefully.
593
632
 
633
+ #### Causal gaps
634
+
635
+ Yjs updates can arrive out of order: an update can reach the server before
636
+ another update it depends on. yrby treats that as normal. The update is
637
+ recorded and acked like any other, parks as a pending struct in the document,
638
+ and integrates on its own the moment the missing dependency lands. The write
639
+ path never rebuilds the document; it appends, relays, and acks, so a gapped
640
+ update costs the same as any other.
641
+
642
+ Serving is lossless too, like any Yjs server. `handle_sync_message` serves
643
+ full state, pending included, so a peer parks the same pending struct and
644
+ heals it the same way. Healing needs no special machinery: the missing
645
+ dependency is an update its sender still holds unacked, and at-least-once
646
+ retransmission delivers it. Only compaction excludes pending
647
+ (`compacted_state_update`), because folding a log must not freeze an
648
+ un-integrable struct into the base state.
649
+
650
+ The bundled `Y::Document` store handles all of this. If you write your own
651
+ store, keep two things in mind:
652
+
653
+ **1. Load losslessly, and tolerate duplicates.** `on_load` should return
654
+ state that preserves pending: `encode_state_as_update`, or a replay of the
655
+ raw append log. Don't compact with `compacted_state_update` while
656
+ `doc.pending?`; that strips the pending struct and the acked edit inside it.
657
+ (`Y::Document` quarantines pending rows for exactly this reason.) A lost ack
658
+ also means a client resends an update the store already has. Replay converges
659
+ anyway, because CRDT apply is idempotent, so deduping is optional. If log
660
+ size matters, dedup by content hash:
661
+
662
+ ```ruby
663
+ class DocumentStore
664
+ # append tolerates duplicates: a re-delivered update upserts to a no-op.
665
+ def append(key, update)
666
+ Revision.upsert({ doc_key: key, update_hash: Digest::SHA256.hexdigest(update), update: update },
667
+ unique_by: %i[doc_key update_hash])
668
+ end
669
+
670
+ # load is lossless: replay the raw log so a pending struct is preserved and
671
+ # heals when its dependency arrives.
672
+ def load(key)
673
+ updates = Revision.where(doc_key: key).order(:id).pluck(:update)
674
+ return nil if updates.empty?
675
+
676
+ doc = Y::Doc.new
677
+ updates.each { |u| doc.apply_update(u) }
678
+ doc.encode_state_as_update # lossless: keeps pending
679
+ end
680
+
681
+ # optional compaction: only when there is no open gap, or you would drop it.
682
+ def compact(key)
683
+ doc = Y::Doc.new
684
+ Revision.where(doc_key: key).order(:id).pluck(:update).each { |u| doc.apply_update(u) }
685
+ return if doc.pending? # a gap is open; compacting now would drop it
686
+ # ... replace the log with a single revision holding doc.compacted_state_update ...
687
+ end
688
+ end
689
+ ```
690
+
691
+ **2. Watch for gaps that never heal.** An open gap is quiet: the edit sits
692
+ as pending, invisible in the document, until its dependency arrives. Normally
693
+ that resolves itself. The sender retransmits the missing update until it is
694
+ acked, and every join or reconnect handshake has the client send everything
695
+ the server hasn't integrated, so any client holding the dependency supplies
696
+ it just by connecting. The gap worth alerting on is one no live client can
697
+ supply, and that is what the `on_gap` hook surfaces. It fires with the
698
+ document key whenever a document is loaded to serve state and a gap is still
699
+ open. Use it to emit a metric (a pending-document count, or the age of the
700
+ oldest open gap) so a stuck gap is visible. Gaps are also logged at `info`,
701
+ and errors raised in the hook are swallowed so observability can never break
702
+ frame handling.
703
+
704
+ ```ruby
705
+ class DocumentChannel < ApplicationCable::Channel
706
+ include Y::ActionCable
707
+
708
+ on_gap { |key| StatsD.increment("yrby.gap", tags: ["doc:#{key}"]) }
709
+ end
710
+ ```
711
+
594
712
  #### Multi-process deployments
595
713
 
596
714
  Most Rails apps run several processes, and any of them might serve a given document.
@@ -650,12 +768,86 @@ duplicate record replays to the same document.
650
768
  The demo wires `on_change` to a durable Postgres-backed log by default, and checks
651
769
  end to end that the log alone rebuilds the document.
652
770
 
771
+ #### Ephemeral documents (no database)
772
+
773
+ `on_load` and `on_change` are plain blocks, and nothing requires them to touch
774
+ a database. For documents that don't need to outlive their session (a
775
+ scratchpad, live form state, a draft you only persist on submit) the store
776
+ can be connection state that travels with each request:
777
+
778
+ ```ruby
779
+ class ScratchpadChannel < ApplicationCable::Channel
780
+ include Y::ActionCable
781
+
782
+ on_load { |key| @doc_state }
783
+
784
+ on_change do |key, update|
785
+ doc = Y::Doc.new
786
+ doc.apply_update(@doc_state) if @doc_state
787
+ doc.apply_update(update)
788
+ @doc_state = doc.compacted_state_update
789
+ end
790
+
791
+ def subscribed = sync_subscribed(params[:id])
792
+ def receive(data) = sync_receive(data, params[:id])
793
+ end
794
+ ```
795
+
796
+ On AnyCable the channel object doesn't survive between messages, so an
797
+ instance variable won't hold. Declare the store as channel state instead
798
+ (`state_attr_accessor` comes from anycable-rails) and Base64 it, because that
799
+ state is serialized as JSON into each RPC exchange with `anycable-go`:
800
+
801
+ ```ruby
802
+ class ScratchpadChannel < ApplicationCable::Channel
803
+ include Y::ActionCable
804
+
805
+ state_attr_accessor :doc_state
806
+
807
+ on_load { |key| doc_state && Base64.strict_decode64(doc_state) }
808
+
809
+ on_change do |key, update|
810
+ doc = Y::Doc.new
811
+ doc.apply_update(Base64.strict_decode64(doc_state)) if doc_state
812
+ doc.apply_update(update)
813
+ self.doc_state = Base64.strict_encode64(doc.compacted_state_update)
814
+ end
815
+
816
+ def subscribed = sync_subscribed(params[:id])
817
+ def receive(data) = sync_receive(data, params[:id])
818
+ end
819
+ ```
820
+
821
+ Both hooks run in the channel instance (`instance_exec`), so they can use
822
+ anything the channel can, and `sync_receive` rebuilds the document from
823
+ `on_load` on every update, which is what lets the store live on the
824
+ connection. On Action Cable the channel instance lasts as long as the
825
+ connection, so an instance variable is the whole store. Merging into
826
+ `compacted_state_update` keeps it one blob instead of a growing update log.
827
+
828
+ The store is per connection, which shapes what this fits. A single writer gets
829
+ the full delivery contract with no database anywhere. With several people
830
+ editing at once, one client's update can depend on edits its own connection
831
+ has never seen; that update records as pending, and the next handshake with
832
+ that client (which always holds the full document) supplies the missing state
833
+ and heals it. The document still converges; heavy concurrent editing just
834
+ parks more pending between handshakes than a shared store would. On
835
+ AnyCable, keep the payload in mind too: the blob travels with every message,
836
+ so that variant suits small documents, not long manuscripts.
837
+
838
+ Durability is the connection plus the browsers. A reconnecting client re-seeds
839
+ an empty server through the ordinary sync handshake, so the document survives
840
+ server restarts as long as some client still has it. For ephemeral documents
841
+ shared across clients on a single-process deployment, the same two hooks over
842
+ a class-level `Concurrent::Map` work instead; that version stops being
843
+ coherent the moment you scale past one process.
844
+
653
845
  #### Reliable delivery (acks)
654
846
 
655
847
  yrby document delivery is ack-tracked. Browser document updates carry an
656
- `"id"`, and the server replies `{ "ack": <id> }` once `on_change` has succesfully fired.
657
- A causally-gapped update is not acked; the server sends a resync request, and
658
- the client keeps the update queued until it lands.
848
+ `"id"`, and the server replies `{ "ack": <id> }` once `on_change` has
849
+ successfully fired. Every decodable document update is recorded and acked,
850
+ including one that arrives out of order.
659
851
 
660
852
  ```
661
853
  client -> server { "update": "<base64 update>", "id": 42 }
@@ -669,13 +861,13 @@ one `{ ack: id }` cumulatively confirms everything up to it. Because CRDT apply
669
861
  is idempotent, a resend that already landed is a harmless no-op that just
670
862
  re-acks. Awareness stays ephemeral and is not acked.
671
863
 
672
- Presence (cursors, selections) is owned by the browser clients the server
864
+ Presence (cursors, selections) is owned by the browser clients; the server
673
865
  never sets or holds presence state, it only relays awareness frames opaquely.
674
866
  See `yrby-client` for the client-side awareness API.
675
867
 
676
868
  ## Thread Safety
677
869
 
678
- A `Doc` is safe to share across Ruby threads used concurrently from Puma
870
+ A `Doc` is safe to share across Ruby threads, used concurrently from Puma
679
871
  workers, ActionCable connection threads, or background jobs without external
680
872
  locking.
681
873
 
@@ -6,13 +6,13 @@
6
6
  # Apps can supply their own ("room-42"), so nothing
7
7
  # parses meaning out of a key.
8
8
  # record + name which model attribute the document backs, where name
9
- # is the attribute name ("body") — optional, one
9
+ # is the attribute name ("body"). Optional, one
10
10
  # document per attribute per record, the same scheme as
11
11
  # ActionText::RichText.
12
12
  #
13
13
  # When a binding exists and no key was supplied, the key derives as
14
- # post/1/body. Either side can arrive first a channel can write under a
15
- # key before any binding exists so `.for` adopts a key-only row whose
14
+ # post/1/body. Either side can arrive first (a channel can write under a
15
+ # key before any binding exists), so `.for` adopts a key-only row whose
16
16
  # key matches the derived one, converging both on one row.
17
17
  #
18
18
  # `state` holds the merged snapshot; the update rows are the uncompacted
@@ -50,12 +50,12 @@ class Y::Document < ActiveRecord::Base
50
50
  # The document bound to a record's attribute, created on first use.
51
51
  # Find first (after the first call, every call is a read), then adopt:
52
52
  # if a channel already appended under the key this binding derives, a
53
- # key-only row exists whose key is taken claiming it converges the two
53
+ # key-only row exists whose key is taken; claiming it converges the two
54
54
  # identities where a plain insert would collide on the key index.
55
55
  #
56
56
  # The insert can still lose a race it can't see: a channel creates the
57
57
  # key-only row after adopt looked and before the insert lands, so
58
- # create_or_find_by! collides on the key index and its internal
58
+ # create_or_find_by! collides on the key index, and its internal
59
59
  # retry, which looks up by record + name, misses the key-only row and
60
60
  # raises RecordNotFound. One more pass adopts the row that won.
61
61
  def for(record, name)
@@ -111,9 +111,9 @@ class Y::Document < ActiveRecord::Base
111
111
  # The merged document: state plus the whole tail. The tail is read first
112
112
  # and the snapshot re-read after it, both straight from the database: a
113
113
  # compaction committing between the two reads then hands us rows already
114
- # folded into the fresh snapshot an idempotent double-apply where
114
+ # folded into the fresh snapshot, an idempotent double-apply, where
115
115
  # the reverse order could pair a pre-compaction snapshot with an empty
116
- # tail and omit committed changes. Quarantined rows are applied too
116
+ # tail and omit committed changes. Quarantined rows are applied too:
117
117
  # the output goes through compacted_state_update, which is gap-free by
118
118
  # construction, so an unhealed gap contributes nothing while a gap
119
119
  # healed by a newer tail row is served immediately instead of waiting
@@ -133,26 +133,51 @@ class Y::Document < ActiveRecord::Base
133
133
  # compactions; a delta landing mid-compaction isn't in `rows`, so it
134
134
  # survives the delete and compacts next time.
135
135
  #
136
- # A causally-gapped batch is never compacted whole and never deleted:
137
- # state would silently exclude the gap, destroying the only healable
138
- # copy. If the clean rows alone merge gap-free, they compact and only
139
- # the gap is quarantined (marked pending); rows that causally build on
140
- # quarantined content quarantine with it.
136
+ # A gapped batch still compacts everything integrable: the fold's
137
+ # compacted_state_update captures every struct that integrates, so
138
+ # rows independent of the gap land in state no matter how they
139
+ # interleave with it. Only the gap tail survives as raw rows, judged
140
+ # per row against the folded state: a row the new state could not
141
+ # integrate cleanly carries the gap (or builds on it) and is
142
+ # quarantined (marked pending); a row that is ready and adds nothing
143
+ # is fully captured and deleted. An acked update never leaves the
144
+ # table before its content is durably in state.
141
145
  def compact!
142
146
  with_lock do
143
147
  rows = updates.pluck(:id, :payload, :pending)
144
148
  next if rows.empty?
149
+ next if compact_rows(rows)
145
150
 
146
- unless compact_rows(rows)
147
- clean = rows.reject { |_, _, pending| pending }
148
- remainder = compact_rows(clean) ? rows - clean : rows
149
- updates.where(id: remainder.map(&:first)).update_all(pending: true)
150
- end
151
+ compact_around_gap(rows)
151
152
  end
152
153
  end
153
154
 
154
155
  private
155
156
 
157
+ # The gapped-batch path: fold everything integrable into state, then
158
+ # judge each row against the folded state. A row the folded state
159
+ # cannot integrate cleanly carries the gap (or builds on it) and is
160
+ # quarantined; a row that is ready and adds nothing is fully captured
161
+ # and deleted.
162
+ def compact_around_gap(rows)
163
+ folded = Y::Doc.new
164
+ folded.apply_update(state) if state
165
+ rows.each { |_, payload, _| folded.apply_update(payload) }
166
+ new_state = folded.compacted_state_update
167
+ # Skip the write when nothing integrated beyond the current state
168
+ # (compacted encoding is deterministic, so equal structs mean equal
169
+ # bytes); a gap-only batch leaves state untouched.
170
+ update!(state: new_state) unless new_state == (state || Y::Doc.new.compacted_state_update)
171
+
172
+ base = Y::Doc.new
173
+ base.apply_update(new_state)
174
+ tail, captured = rows.partition do |_, payload, _|
175
+ !base.update_ready?(payload) || base.update_advances?(payload)
176
+ end
177
+ updates.where(id: captured.map(&:first)).delete_all
178
+ updates.where(id: tail.map(&:first)).update_all(pending: true)
179
+ end
180
+
156
181
  # Compact state + the given rows if the merge is gap-free: writes state,
157
182
  # deletes the rows, returns true. Leaves everything untouched and returns
158
183
  # false on a gap.
@@ -169,7 +194,7 @@ class Y::Document < ActiveRecord::Base
169
194
  true
170
195
  end
171
196
 
172
- # Derives post/1/body from the polymorphic record_type Rails stores
197
+ # Derives post/1/body from the polymorphic record_type. Rails stores
173
198
  # the polymorphic_name there, which is base_class-derived, so STI
174
199
  # subclasses share a key. Namespaces keep their slash
175
200
  # (admin/post/1/body); flattening would collide Admin::Post with
@@ -2,7 +2,7 @@
2
2
 
3
3
  # A document whose stored bytes are encrypted with Active Record encryption
4
4
  # (state here, update payloads via Y::EncryptedDocumentUpdate). Same tables
5
- # as Y::Document the class you access through decides the cryptography,
5
+ # as Y::Document; the class you access through decides the cryptography,
6
6
  # the way ActionText::EncryptedRichText does for rich text. Keep one access
7
7
  # path per document: rows written encrypted read back as ciphertext through
8
8
  # the plain classes.
@@ -29,7 +29,7 @@ class DocumentChannel < ApplicationCable::Channel
29
29
  # Everyone is denied until you fill this in. Wire it to your app's auth:
30
30
  # identify current_user on the cable connection, then check they may read
31
31
  # and write this document. Don't lean on on_change raising for access
32
- # control that path exists for store failures.
32
+ # control; that path exists for store failures.
33
33
  def authorized?(_document_key)
34
34
  false
35
35
  end
@@ -5,15 +5,15 @@ require "rails/generators/active_record"
5
5
 
6
6
  module Yrby
7
7
  module Generators
8
- # `bin/rails generate yrby:tables` the migration for the gem-owned
8
+ # `bin/rails generate yrby:tables`: the migration for the gem-owned
9
9
  # document models (Y::Document + Y::DocumentUpdate). Invoked by
10
10
  # yrby:install, and by other gems building on the same storage.
11
11
  #
12
12
  # Template notes (kept here, not in the emitted migration): state is
13
- # 4.gigabytes - 1 (longblob on MySQL a compacted snapshot is the whole
13
+ # 4.gigabytes - 1 (longblob on MySQL; a compacted snapshot is the whole
14
14
  # document; a 16 MB cap would break compaction) and payload is
15
15
  # 16.megabytes - 1 (one update can carry a big paste or a client's
16
- # accumulated offline edits the 64 KB default blob is too small).
16
+ # accumulated offline edits; the 64 KB default blob is too small).
17
17
  # The partial unique index's WHERE only keeps
18
18
  # key-only rows out of the index: uniqueness holds without it, since
19
19
  # unique indexes treat NULLs as distinct on every supported database,
@@ -16,7 +16,7 @@ module Y::ActionCable # rubocop:disable Style/ClassAndModuleChildren
16
16
  #
17
17
  # Example:
18
18
  # class DocumentChannel < ApplicationCable::Channel
19
- # include Y::ActionCable::Sync
19
+ # include Y::ActionCable
20
20
  #
21
21
  # on_load { |key| Document.find_by(key: key)&.content }
22
22
  # # on_change runs in the channel instance's context, so instance methods
@@ -35,9 +35,10 @@ module Y::ActionCable # rubocop:disable Style/ClassAndModuleChildren
35
35
  # There is no unsubscribe hook: the server keeps no per-connection document or
36
36
  # presence state, so a disconnect needs no server-side cleanup.
37
37
  #
38
- # The concern is store-backed: every document update is validated against
39
- # `on_load`, recorded through `on_change`, and only then broadcast.
40
- # No authoritative document state is kept in ActionCable process memory.
38
+ # The concern is store-backed: every document update is recorded through
39
+ # `on_change` before it is broadcast or acked, and documents rebuild from
40
+ # `on_load` whenever state is served. No authoritative document state is
41
+ # kept in ActionCable process memory.
41
42
  module Sync
42
43
  # Frame kinds we act on, from Y.message_kind. Its other codes (0 for a
43
44
  # drop: malformed/truncated/multi-message/unknown, and 4 for an awareness
@@ -91,6 +92,19 @@ module Y::ActionCable # rubocop:disable Style/ClassAndModuleChildren
91
92
 
92
93
  superclass.respond_to?(:max_frame_bytes) ? superclass.max_frame_bytes : DEFAULT_MAX_FRAME_BYTES
93
94
  end
95
+
96
+ # Optional observability hook, fired at join/serve time whenever the
97
+ # loaded document still holds a causal gap (a pending struct). Called
98
+ # with (key) in the channel instance's context (instance_exec). Use it
99
+ # to emit a metric (pending-document count, gap age) so an unhealed gap
100
+ # is visible. Errors in the hook are swallowed so observability can
101
+ # never break frame handling.
102
+ def on_gap(&block)
103
+ @on_gap = block if block
104
+ return @on_gap if defined?(@on_gap) && @on_gap
105
+
106
+ superclass.respond_to?(:on_gap) ? superclass.on_gap : nil
107
+ end
94
108
  end
95
109
 
96
110
  # Call from `subscribed`. Streams broadcasts for this document and
@@ -104,7 +118,15 @@ module Y::ActionCable # rubocop:disable Style/ClassAndModuleChildren
104
118
  # client path to ephemeral presence rather than the durable document stream.
105
119
  stream_from sync_stream_name
106
120
  stream_from sync_awareness_stream_name, whisper: true if respond_to?(:whispers_to)
107
- sync_transmit(sync_load_doc.sync_step1)
121
+
122
+ # The opening handshake is also the gap-repair prompt: sending our SyncStep1
123
+ # asks the joining client for everything beyond our integrated state, which
124
+ # is exactly the missing dependency an open gap is waiting on. If a live
125
+ # client has it, the join heals the gap. We only surface it (on_gap); the
126
+ # handshake below already does the soliciting.
127
+ doc = sync_load_doc
128
+ sync_transmit(doc.sync_step1)
129
+ sync_observe_gap if doc.pending?
108
130
  end
109
131
 
110
132
  # Call from `receive`. Applies the client's message, replies directly
@@ -112,9 +134,10 @@ module Y::ActionCable # rubocop:disable Style/ClassAndModuleChildren
112
134
  # to the other subscribers.
113
135
  #
114
136
  # Reliable delivery: document updates carry an "id", and the server replies
115
- # `{ "ack" => id }` once the update has been durably recorded. A
116
- # causally-gapped update is not acked; it gets a resync instead, so the
117
- # client retransmits until the update lands.
137
+ # `{ "ack" => id }` once the update has been durably recorded. A causally-
138
+ # incomplete update is recorded and acked like any other; it stays pending
139
+ # (durable, invisible in the document) until its missing dependency
140
+ # arrives.
118
141
  def sync_receive(data, key = nil)
119
142
  # Pass `key` (params[:id]) when your transport doesn't keep the channel
120
143
  # instance alive across actions. Under AnyCable each RPC command gets a
@@ -156,23 +179,14 @@ module Y::ActionCable # rubocop:disable Style/ClassAndModuleChildren
156
179
 
157
180
  private
158
181
 
159
- # Ask this connection's client to resync: re-send SyncStep1 carrying the
160
- # server's current (gap-free) state vector. The client replies SyncStep2
161
- # with everything the server is missing, delivered as one causally-complete
162
- # delta, which heals the gap that triggered the resync.
163
- def sync_request_resync(doc)
164
- sync_transmit(doc.sync_step1)
165
- end
166
-
167
182
  # Reliable delivery: acknowledge an accepted update back to the sending
168
183
  # connection. An ack-aware client tags each outgoing update with an "id"
169
- # and retains it until the matching `{ "ack" => id }` returns, retransmitting
170
- # on a timer or reconnect; idempotent CRDT apply makes resends free. Acks
171
- # are sent only after the update has been durably recorded, or when a retry
172
- # is already present in the durable store.
184
+ # and retains it until the matching `{ "ack" => id }` returns,
185
+ # retransmitting on a timer or reconnect; applying a CRDT update twice is
186
+ # safe. Acks are sent only after the update has been durably recorded.
173
187
  def sync_send_ack(id, outcome)
174
188
  return if id.nil?
175
- return unless %i[recorded applied].include?(outcome)
189
+ return unless outcome == :recorded
176
190
 
177
191
  # The braces are required: a bare hash would bind to transmit's `via:`
178
192
  # keyword instead of its positional data argument.
@@ -223,6 +237,36 @@ module Y::ActionCable # rubocop:disable Style/ClassAndModuleChildren
223
237
  end
224
238
  end
225
239
 
240
+ # A causal gap was observed at join/serve time: the update is durable but
241
+ # its content stays invisible in the document until its missing dependency
242
+ # arrives and heals it. Healing is quiet, so make the open gap findable: log at info,
243
+ # and fire the on_gap hook so the app can emit a metric (pending-document
244
+ # count, gap age). Errors in the hook are swallowed; observability must
245
+ # never break frame handling.
246
+ def sync_observe_gap
247
+ logger.info do
248
+ parts = ["key=#{@sync_key.inspect}"]
249
+ parts << sync_log_context_safe
250
+ "[yrby] causal gap present (pending until its dependency arrives): #{parts.compact.join(" ")}"
251
+ end
252
+
253
+ return unless (hook = self.class.on_gap)
254
+
255
+ begin
256
+ instance_exec(@sync_key, &hook)
257
+ rescue StandardError => e
258
+ logger.error { "[yrby] on_gap hook raised (#{e.class}); continuing: key=#{@sync_key.inspect}" }
259
+ end
260
+ end
261
+
262
+ # sync_log_context, guarded: a broken context hook must surface in the log,
263
+ # not take down frame handling.
264
+ def sync_log_context_safe
265
+ sync_log_context
266
+ rescue StandardError => e
267
+ "log-context-error=#{e.class}"
268
+ end
269
+
226
270
  # This concern acks updates as durably recorded, so it must have both a
227
271
  # loader (to rebuild the doc and detect causal gaps) and a recorder (to
228
272
  # actually persist before acking). Fail closed rather than silently acking
@@ -243,7 +287,7 @@ module Y::ActionCable # rubocop:disable Style/ClassAndModuleChildren
243
287
  # Fail closed when no document key is set (typically: AnyCable rebuilt the
244
288
  # channel instance and the app forgot to pass `key` to sync_receive).
245
289
  # Proceeding would record under nil, broadcast to a stream nobody
246
- # subscribes to, and still ack the client believes the edit was
290
+ # subscribes to, and still ack; the client believes the edit was
247
291
  # delivered when it reached no one.
248
292
  def sync_validate_key!
249
293
  return unless @sync_key.nil? || @sync_key.empty?
@@ -259,45 +303,26 @@ module Y::ActionCable # rubocop:disable Style/ClassAndModuleChildren
259
303
  # before relay and then broadcast, and awareness is relayed best-effort.
260
304
  # Echoing back to the sender is harmless, since the CRDT apply is idempotent.
261
305
  #
262
- # Returns an outcome symbol for the reliable-delivery ack: :recorded when a
263
- # document update was durably recorded and relayed, :gap when it was
264
- # rejected for a resync, :noop for everything else.
306
+ # Returns an outcome symbol for the reliable-delivery ack: :recorded when
307
+ # a document update was durably recorded and relayed (a lost-ack retry
308
+ # records again; the store tolerates duplicates), :noop for everything
309
+ # else.
265
310
  def sync_handle_frame(encoded, bytes)
266
311
  sync_validate_required_hooks!
267
312
  sync_validate_key!
268
313
 
269
314
  case Y.message_kind(bytes)
270
315
  when MSG_KIND_SYNC_STEP1
271
- result = sync_load_doc.handle_sync_message(bytes)
272
- sync_transmit(result[2])
316
+ doc = sync_load_doc
317
+ result = doc.handle_sync_message(bytes)
318
+ sync_transmit(result[2]) # full state, pending included
319
+ sync_observe_gap if doc.pending?
273
320
  :noop
274
321
  when MSG_KIND_UPDATE
275
322
  update = Y.update_from_message(bytes)
276
323
  return :noop unless update
277
324
 
278
- # Rebuild from the store (O(history) per update; snapshot in on_load if
279
- # that cost bites).
280
- doc = sync_load_doc
281
-
282
- # Don't record a causally-incomplete update; resync instead so the gap
283
- # heals as one complete delta.
284
- unless doc.update_ready?(update)
285
- sync_request_resync(doc)
286
- return :gap
287
- end
288
-
289
- # A lost-ack retry: already recorded, so skip on_change — but DO
290
- # re-broadcast. If the first attempt died between record and broadcast,
291
- # this retry is the only path left to the live subscribers. Duplicate
292
- # broadcasts are free (CRDT apply is idempotent).
293
- unless doc.update_advances?(update)
294
- sync_distribute(encoded)
295
- return :applied
296
- end
297
-
298
- sync_record_change(update) # record before relay
299
- sync_distribute(encoded)
300
- :recorded
325
+ sync_handle_document_update(update, encoded)
301
326
  when MSG_KIND_AWARENESS
302
327
  sync_distribute(encoded)
303
328
  :noop
@@ -306,6 +331,21 @@ module Y::ActionCable # rubocop:disable Style/ClassAndModuleChildren
306
331
  end
307
332
  end
308
333
 
334
+ # Ack-on-durable, with no doc rebuild and no gap check on the write
335
+ # path: record (before relay), relay, ack. A causally-incomplete update is
336
+ # recorded as a pending struct like any other edit and served onward like
337
+ # one (a peer parks and heals it the same way this doc does). The gap
338
+ # heals when its missing dependency arrives: its own sender retransmits
339
+ # it until acked, and every join or reconnect handshake has the client
340
+ # send everything beyond the server's integrated state. An open gap is
341
+ # surfaced by on_gap at join/serve time. on_change must tolerate
342
+ # duplicates: a lost-ack retry records again.
343
+ def sync_handle_document_update(update, encoded)
344
+ sync_record_change(update) # record before relay
345
+ sync_distribute(encoded)
346
+ :recorded
347
+ end
348
+
309
349
  # Build a fresh document from the durable store (on_load). Callers validate
310
350
  # the hooks first, so on_load is present; a nil state means a fresh document.
311
351
  def sync_load_doc
@@ -2,6 +2,6 @@
2
2
 
3
3
  module Yrby
4
4
  module Rails
5
- VERSION = "0.5.0"
5
+ VERSION = "0.6.0"
6
6
  end
7
7
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: yrby-rails
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.5.0
4
+ version: 0.6.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - JP Camara
@@ -57,14 +57,14 @@ dependencies:
57
57
  requirements:
58
58
  - - ">="
59
59
  - !ruby/object:Gem::Version
60
- version: 0.3.1
60
+ version: 0.7.0
61
61
  type: :runtime
62
62
  prerelease: false
63
63
  version_requirements: !ruby/object:Gem::Requirement
64
64
  requirements:
65
65
  - - ">="
66
66
  - !ruby/object:Gem::Version
67
- version: 0.3.1
67
+ version: 0.7.0
68
68
  - !ruby/object:Gem::Dependency
69
69
  name: actioncable
70
70
  requirement: !ruby/object:Gem::Requirement