yrby-rails 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md ADDED
@@ -0,0 +1,753 @@
1
+ # yrby
2
+
3
+ [![CI](https://github.com/jpcamara/yrby/actions/workflows/ci.yml/badge.svg)](https://github.com/jpcamara/yrby/actions/workflows/ci.yml)
4
+
5
+ Collaborative editing for Rails, backed by [y-crdt](https://github.com/y-crdt/y-crdt)
6
+ (the Rust library behind Y.js). Your Rails server speaks the y-websocket sync
7
+ protocol directly, so there's no separate Node process hosting the Y.js
8
+ documents. Pronounced "yer-bee".
9
+
10
+ ![Two people typing on separate lines of the same document, each keystroke synced through a Rails server, seen from a third browser with labeled carets](docs/images/collab.gif)
11
+
12
+ ```ruby
13
+ class DocumentChannel < ApplicationCable::Channel
14
+ include Y::ActionCable
15
+
16
+ on_load { |key| Y::Document.load_state(key) }
17
+ on_change { |key, update| Y::Document.append(key, update) }
18
+
19
+ def subscribed = sync_subscribed(params[:id])
20
+ def receive(data) = sync_receive(data, params[:id])
21
+ end
22
+ ```
23
+
24
+ On the browser, use the `ActionCableProvider` from the
25
+ [`yrby-client`](https://www.npmjs.com/package/yrby-client) npm package.
26
+ Integrates with any editor that includes Y.js support, such as Tiptap, ProseMirror
27
+ and [Lexxy](https://www.npmjs.com/package/lexxy-realtime).
28
+
29
+ ## Usage
30
+
31
+ Install the gem and npm package:
32
+
33
+ ```
34
+ gem install yrby-actioncable # depends on yrby
35
+ npm install yrby-client
36
+ ```
37
+
38
+ ## What you get
39
+
40
+ - A thread-safe Ruby `Doc` you can share across Ruby threads/fibers, and native CRDT work
41
+ runs with the GVL released.
42
+ - The y-websocket protocol (document sync plus awareness/presence) as a
43
+ one-include ActionCable concern.
44
+ - Authoritative record-before-distribute semantics: each document change can be
45
+ recorded durably before it goes out to anyone.
46
+ - Optional server-side reads: `Doc#read_text` and `Doc#read_map` reconstruct a
47
+ document's contents in Ruby - no Node process - for search, exports, validation,
48
+ or server-side rendering.
49
+
50
+ ## Scope
51
+
52
+ `yrby` binds just the part of `y-crdt` you need to *sync and persist* collaborative
53
+ documents - a `Doc`, awareness, and the y-websocket protocol primitives. By default
54
+ the Ruby side treats a document as opaque CRDT state: it applies updates, answers
55
+ sync handshakes, and records deltas without reaching into the contents - the browser
56
+ editor owns the document's shape. When you do need to look inside, `Doc#read_text`
57
+ and `Doc#read_map` reconstruct it server-side, in Ruby.
58
+
59
+ ## Durability and delivery
60
+
61
+ The surface is intentionally small, but the focus is durability, resiliency, delivery
62
+ guarantees, correctness, and thread safety.
63
+
64
+ Towards that goal, `yrby` adds opinionated defaults on top of normal Yjs syncing:
65
+
66
+ - Built-in update acknowledgement: the `ActionCableProvider` in `yrby-client` will continue to
67
+ send updates until an ack is received from the server. [`yrby-actioncable`](https://rubygems.org/gems/yrby-actioncable)
68
+ only sends an ack when applying an update is successful. The goal is at-least-once delivery,
69
+ and because CRDTs are idempotent a duplicate update is effectively a no-op.
70
+ - Gap detection in document updates: before applying an update and sending an ack to the client,
71
+ `yrby` checks whether the update results in any causal gap. Ie, an update comes through
72
+ which depends on a previous update that is not yet present in the document. This can result in
73
+ a document stuck with "pending" updates, which will _never_ apply if the missing update is not sent.
74
+ To avoid this, `yrby` does not apply the update, and starts a new y-protocol sync with the client.
75
+ That will cause the client to synchronize its document with the server, sending through any updates
76
+ that may have been missed
77
+
78
+ ## What about [yrb](https://github.com/y-crdt/yrb)?
79
+
80
+ `yrb` has a much larger interface that gives you most of the Yjs type system -
81
+ shared text, arrays, maps, XML - to build and query documents in Ruby. It was a great
82
+ inspiration for my use of Yjs in Ruby/Rails, and I originally considered building
83
+ on top of it. There are a few reasons I went with `yrby` instead:
84
+
85
+ - `yrb` is largely unmaintained. It was built as an experiment for GitLab, and the original
86
+ author mostly moved onto other projects.
87
+ - [It isn't thread-safe](https://github.com/y-crdt/yrb/issues/72). It segfaults in a threaded
88
+ environment (such as ActionCable...)
89
+ - It's a much larger set of features to maintain, which most people don't need. The vast
90
+ majority of people manipulate Y.js documents in the browser, not from a server-side language.
91
+
92
+ ## Testing
93
+
94
+ Ruby and Rust unit tests cover the core. CI also runs the npm client tests and a
95
+ Rails demo smoke slice against the real ActionCable stack. The demo includes
96
+ heavier local suites for hostile input, crash recovery, multi-browser editing,
97
+ AnyCable, and load testing. The benchmark note below is from a single laptop.
98
+ Issues and PRs are welcome.
99
+
100
+ ## Install
101
+
102
+ ```ruby
103
+ # Core CRDT + protocol primitives:
104
+ gem "yrby"
105
+
106
+ # For the Rails side (the sync channel, document models, the generator).
107
+ # Formerly yrby-actioncable; that name stops at 0.3.1.
108
+ gem "yrby-rails"
109
+ ```
110
+
111
+ Requires Ruby 3.4 or newer. The release workflow builds precompiled gems for
112
+ Ruby 3.4 and 4.0 across the supported Ruby platforms, with native smoke tests
113
+ on Linux x86_64 and macOS arm64. Installing from a matching platform gem needs
114
+ no Rust; a source build needs [Rust](https://rustup.rs).
115
+
116
+ To work on the gem itself:
117
+
118
+ ```bash
119
+ git clone https://github.com/jpcamara/yrby
120
+ cd yrby
121
+ bundle install
122
+ bundle exec rake compile test
123
+ ```
124
+
125
+ The rest of the dev setup, plus the demo, is in [CONTRIBUTING.md](CONTRIBUTING.md).
126
+
127
+ ## Docs
128
+
129
+ - The ActionCable concern and a quickstart are [below](#actioncable-integration).
130
+ - [`examples/actioncable-demo`](examples/actioncable-demo): a runnable Rails +
131
+ Tiptap app with collaborative cursors, the AnyCable setup, a Postgres store,
132
+ and the test/load suites.
133
+ - [CHANGELOG.md](CHANGELOG.md) and [CONTRIBUTING.md](CONTRIBUTING.md).
134
+
135
+ ## Editors
136
+
137
+ yrby syncs opaque Yjs updates, so it works with any editor that has a Yjs
138
+ binding. The demo app runs four, and CI drives each one in real Chrome:
139
+ concurrent typing with every keystroke accounted for, remote cursors,
140
+ local-only undo, and byte parity between the server-side renderers and the
141
+ editor's own serializer. Each page is a working integration to copy from:
142
+
143
+ | Editor | Yjs binding | Demo code |
144
+ |---|---|---|
145
+ | [Tiptap](https://tiptap.dev) (v2) | `@tiptap/extension-collaboration` | [`app.js`](examples/actioncable-demo/frontend/src/app.js) |
146
+ | [Lexxy](https://github.com/basecamp/lexxy) (Lexical) | [`lexxy-realtime`](https://www.npmjs.com/package/lexxy-realtime) | [`lexxy.js`](examples/actioncable-demo/frontend/src/lexxy.js) |
147
+ | [Rhino Editor](https://github.com/KonnorRogers/rhino-editor) (Tiptap 3) | `@tiptap/extension-collaboration` + `-caret` | [`rhino.js`](examples/actioncable-demo/frontend/src/rhino.js) |
148
+ | [CodeMirror 6](https://codemirror.net) | `y-codemirror.next` | [`codemirror.js`](examples/actioncable-demo/frontend/src/codemirror.js) |
149
+
150
+ The demo also syncs plain Yjs shapes with no editor at all — a whiteboard
151
+ on a `Y.Map`, a kanban board on a `Y.Array`, a co-filled form — over the
152
+ same channel. The demo README's "Using this in your own app" section has
153
+ the integration recipe, and its `NoteMaterializer` shows how to render a
154
+ document to ActionText server-side with `Y::Tiptap` or `Y::Lexxy`.
155
+
156
+ ## Usage
157
+
158
+ ### Doc (Low-Level Document Sync)
159
+
160
+ ```ruby
161
+ require "y"
162
+
163
+ # Create docs
164
+ doc = Y::Doc.new # random client ID
165
+ doc = Y::Doc.new(12345) # specific client ID (used for CRDT identity)
166
+
167
+ # Encoding
168
+ doc.encode_state_vector # => current state vector
169
+ doc.encode_state_as_update # => full update (lossless: keeps pending)
170
+ doc.encode_state_as_update(sv) # => update diff against state vector
171
+ doc.compacted_state_update # => full update, gap-free (excludes pending)
172
+
173
+ # Applying updates
174
+ doc.apply_update(update_bytes) # apply raw V1 update
175
+ doc.pending? # => true if holding un-integrable pending structs
176
+
177
+ # Sync protocol
178
+ doc.sync_step1 # => SyncStep1 message (this doc's state vector)
179
+ doc.handle_sync_message(data) # => [msg_type, sync_type, response]; answers a
180
+ # peer's SyncStep1 with an integrated-only
181
+ # SyncStep2 (never serves pending structs)
182
+ ```
183
+
184
+ ### Reading document contents
185
+
186
+ Reconstruct a document server-side — search, exports, emails, SSR — with no
187
+ Node process:
188
+
189
+ ```ruby
190
+ doc.read_text("prosemirror") # => plain text of a Y.Text root, or nil
191
+ doc.read_xml("root") # => text of an XML root, one block per line
192
+ doc.read_map("state") # => a Y.Map root as a JSON string; JSON.parse it
193
+ ```
194
+
195
+ ### Pending structs and gap-free state
196
+
197
+ If a doc applies an update whose causally-prior update is missing (a "gappy"
198
+ update), yrs parks it as a **pending** struct: the integrated state vector stays
199
+ empty, but the pending block is held as a recovery buffer and heals if the
200
+ missing dependency later arrives. `Doc#pending?` reports this.
201
+
202
+ Pending structs are *not* document state, so they must not cross the sync
203
+ boundary — a peer that receives one can't integrate it and gets stuck. Two
204
+ guarantees keep serving safe:
205
+
206
+ - `handle_sync_message` answers `SyncStep1` with **integrated-only** state, so a
207
+ server never serves a struct it can't integrate itself (this is automatic).
208
+ - `Doc#compacted_state_update` gives you the same gap-free full-state update for
209
+ when you persist or hand off state yourself. It's non-destructive (the doc
210
+ keeps its pending), while `encode_state_as_update` stays lossless so you can
211
+ still preserve the raw pending bytes for recovery.
212
+
213
+ ### Rendering to HTML
214
+
215
+ Schema-pinned renderers turn a collaborative document into HTML on the
216
+ server, with no Node process or headless editor. Each is an editor-specific
217
+ class (byte-for-byte with that editor's own serializer) built on a core base
218
+ any other editor extends with rules: `Y::Tiptap` on `Y::ProseMirror` for
219
+ ProseMirror documents, and `Y::Lexxy` (the
220
+ [Lexxy](https://github.com/basecamp/lexxy) editor) on `Y::Lexical`. Each
221
+ returns `nil` for a root that belongs to the other schema.
222
+
223
+ #### `Y::Tiptap` (and `Y::ProseMirror`, its base)
224
+
225
+ ```ruby
226
+ tiptap = Y::Tiptap.new(doc)
227
+ tiptap.to_html # the "default" fragment (Tiptap's default root)
228
+ tiptap.to_html("content") # or another XML root
229
+ ```
230
+
231
+ The output matches Tiptap's own `getHTML()`, checked byte-for-byte in the tests
232
+ against a document captured from a real editor. It follows
233
+ [`tiptap-php`](https://github.com/ueberdosis/tiptap-php) and reads both name
234
+ styles editors use — Tiptap's `bulletList`/`bold` and prosemirror-schema-basic's
235
+ `bullet_list`/`strong`.
236
+
237
+ It covers paragraphs, headings, blockquotes, bullet/ordered/task lists, code
238
+ blocks, links, images, mentions, details, hard breaks, horizontal rules,
239
+ tables, text styles (color, font family), and every text mark. A table renders
240
+ as semantic `<table><tbody>`, without the column-width styling Tiptap's editor
241
+ view adds.
242
+
243
+ The support is layered like the Lexical side: `Y::ProseMirror` covers core
244
+ ProseMirror natively — prosemirror-schema-basic plus the prosemirror-tables
245
+ family — and Tiptap's extension nodes (task lists, mentions, the details
246
+ family) are `Y::Tiptap`'s rule set (`Y::Tiptap::NODES`), built on the
247
+ extension API below. Marks stay in the base: mark rendering (nesting order,
248
+ `textStyle` CSS, `code` exclusivity) runs through native text-run machinery
249
+ that node rules don't reach, so `Y::ProseMirror` renders Tiptap's mark set
250
+ as-is and `rules.mark` overrides individual marks.
251
+
252
+ #### `Y::Lexxy` (and `Y::Lexical`, its base)
253
+
254
+ ```ruby
255
+ lexxy = Y::Lexxy.new(doc)
256
+ lexxy.to_html # the "root" fragment (Lexical's default root name)
257
+ lexxy.to_html("notepad") # or another XML root
258
+ ```
259
+
260
+ The HTML is identical to what a `lexxy-editor` submits to Rails (its `value`).
261
+ The tests check this byte-for-byte against a document captured from a real
262
+ editor. Stock Lexical has no canonical serializer — every editor configures
263
+ its own — so the editor-specific class carries the editor's name, and
264
+ `Y::Lexical` is the core-Lexical base: paragraphs, headings, quotes, code,
265
+ lists, tables, links, and the full text-format model, for any other Lexical
266
+ editor to extend with rules.
267
+
268
+ It handles the whole Lexxy 0.9.x node set: paragraphs, headings, every text
269
+ format and their combinations, links, the four list types and nesting,
270
+ blockquotes, code blocks, tabs and soft breaks, horizontal rules, tables with
271
+ header cells, image galleries, and ActionText attachments (uploads and
272
+ mentions both emit `<action-text-attachment>` elements that ActionText can
273
+ re-render).
274
+
275
+ Internally that support is layered: `Y::Lexical` covers core Lexical
276
+ structure natively, and everything Lexxy adds — its node types (attachments,
277
+ galleries) and its decorations of core nodes (the table wrapper, header-cell
278
+ styling, nested-list classes) — is `Y::Lexxy`'s rule set
279
+ (`Y::Lexxy::NODES`), built on the extension API below. The gem's own Lexxy
280
+ support is the API's first consumer: an app rule for one of those types
281
+ simply replaces it.
282
+
283
+ In both renderers an unknown node keeps its content — text and nested blocks
284
+ fall back to readable markup rather than disappearing.
285
+
286
+ #### Custom nodes and marks
287
+
288
+ The built-in schemas are pinned to what Tiptap and Lexxy ship, but apps add
289
+ their own node types. Both renderers take rules for them. A rule is checked
290
+ before the built-in schema, so it can add a node type or replace how a
291
+ built-in renders.
292
+
293
+ Rules register in a block — one `rules.node` call per type. A declarative
294
+ rule is markup as data, rendered natively:
295
+
296
+ ```ruby
297
+ tiptap = Y::Tiptap.new(doc) do |rules|
298
+ rules.node "callout", tag: "aside",
299
+ attrs: { "class" => ["callout callout--", :kind] },
300
+ contains: :blocks
301
+ end
302
+ ```
303
+
304
+ `tag` names the element. `attrs` values are templates: a string is a literal,
305
+ a symbol reads that attribute off the node, an array concatenates both kinds;
306
+ an attribute that resolves empty is left out. `text` (same template form)
307
+ emits literal text content. `contains` declares what lives inside the node — `:inline` (formatted text,
308
+ the default), `:blocks` (child block nodes — a container), or `:none` (a
309
+ leaf). `void: true` skips the closing tag.
310
+
311
+ You don't have to guess any of those names or shapes. Editors store types
312
+ and attributes under names you'd never predict (Rhino's strike mark is
313
+ `rhino-strike`; Lexical prefixes its own props `__`), so ask a real
314
+ document instead — make one in your editor using your custom node, then:
315
+
316
+ ```ruby
317
+ Y::Tiptap.new(doc).node_types
318
+ # => { "callout" => { "count" => 2, "attrs" => ["kind"],
319
+ # "children" => ["paragraph"], "text" => false,
320
+ # "handled" => nil },
321
+ # "paragraph" => { ..., "handled" => "builtin" } }
322
+ ```
323
+
324
+ `handled` nil marks the types that still need a rule; `attrs` are the stored
325
+ names your templates and blocks will read; `children` plus `text` is how you
326
+ pick `contains:` (child block types → `:blocks`; text → `:inline`).
327
+
328
+ When markup-as-data isn't enough, give the node a block:
329
+
330
+ ```ruby
331
+ lexical = Y::Lexical.new(doc) do |rules|
332
+ rules.node "video_embed" do |node|
333
+ src = ERB::Util.html_escape(node.attrs["__src"])
334
+ %(<video controls src="#{src}"></video>)
335
+ end
336
+ end
337
+ ```
338
+
339
+ The block gets the node's type, its stored attributes, `node.content` — the
340
+ children, already rendered to HTML — and `node.child_types`, the node's
341
+ element/block children by type, in document order. `child_types` answers the
342
+ structural questions attributes can't: how many images a gallery holds, or
343
+ whether a list item carries a nested list. Whatever the block returns is
344
+ spliced into the output as-is: it's trusted HTML, so escape any values you
345
+ interpolate. To set the content mode for a callback, give the node both —
346
+ `rules.node "embed", contains: :blocks do |node| ... end`.
347
+
348
+ Callbacks never run while the document is locked. The render finishes first
349
+ (inside one read transaction, GVL released), then the blocks run and their
350
+ output is spliced in — so a callback can safely read or even write the same
351
+ doc. With no callback rules, `to_html` skips the splicing entirely.
352
+
353
+ Blocks are the escape hatch for everything the declarative form can't say,
354
+ and they're proven sufficient: `Y::Lexxy` and `Y::Tiptap` are themselves
355
+ built on this API (`lib/y/lexxy.rb`, `lib/y/tiptap.rb`) — simple nodes as
356
+ declarative hashes, everything with logic as plain methods mapped by node
357
+ type (a `Method` responds to `call` like any lambda) — and the fixture tests
358
+ hold their output byte-identical to a live editor's.
359
+
360
+ The ProseMirror side also takes custom marks:
361
+
362
+ ```ruby
363
+ tiptap = Y::Tiptap.new(doc) do |rules|
364
+ rules.mark "comment", tag: "span", attrs: { "data-comment-id" => :id }
365
+ end
366
+ ```
367
+
368
+ Symbol refs resolve against the mark's own attributes. A custom mark wraps
369
+ outside every built-in mark; several on one run nest alphabetically. A rule
370
+ for a built-in mark name (`"bold"`) replaces its built-in tag.
371
+
372
+ ##### Worked examples
373
+
374
+ A video-embed node from an app's Tiptap extension — a type the pinned schema
375
+ has never heard of:
376
+
377
+ ```ruby
378
+ tiptap = Y::Tiptap.new(doc) do |rules|
379
+ rules.node "videoEmbed" do |node|
380
+ src = ERB::Util.html_escape(node.attrs["src"])
381
+ title = ERB::Util.html_escape(node.attrs["title"] || "Video")
382
+ %(<figure class="video"><iframe src="#{src}" title="#{title}" allowfullscreen></iframe></figure>)
383
+ end
384
+ end
385
+ ```
386
+
387
+ Resolving mentions against the database. Blocks run after the document read
388
+ has finished, so hitting ActiveRecord (or the doc itself) inside one is safe:
389
+
390
+ ```ruby
391
+ tiptap = Y::Tiptap.new(doc) do |rules|
392
+ rules.node "mention" do |node|
393
+ user = User.find_by(id: node.attrs["id"])
394
+ next "<span>@unknown</span>" unless user
395
+
396
+ %(<a class="mention" href="/users/#{user.id}">@#{ERB::Util.html_escape(user.handle)}</a>)
397
+ end
398
+ end
399
+ ```
400
+
401
+ Overriding a shipped rule — rendering Lexxy uploads as real image markup
402
+ instead of the `<action-text-attachment>` elements ActionText re-renders:
403
+
404
+ ```ruby
405
+ lexxy = Y::Lexxy.new(doc) do |rules|
406
+ rules.node "action_text_attachment" do |node|
407
+ src = ERB::Util.html_escape(node.attrs["src"])
408
+ alt = ERB::Util.html_escape(node.attrs["altText"].to_s)
409
+ caption = node.attrs["caption"].to_s
410
+ html = %(<img src="#{src}" alt="#{alt}" loading="lazy">)
411
+ html += "<figcaption>#{ERB::Util.html_escape(caption)}</figcaption>" unless caption.empty?
412
+ "<figure>#{html}</figure>"
413
+ end
414
+ end
415
+ ```
416
+
417
+ Markup that depends on structure — `node.child_types` lists the node's
418
+ element/block children in document order, so a layout container can size
419
+ itself by its column count while the columns themselves stay declarative:
420
+
421
+ ```ruby
422
+ tiptap = Y::Tiptap.new(doc) do |rules|
423
+ rules.node "columns", contains: :blocks do |node|
424
+ %(<div class="columns columns--#{node.child_types.length}">#{node.content}</div>)
425
+ end
426
+ rules.node "column", tag: "div", attrs: { "class" => "column" }, contains: :blocks
427
+ end
428
+ ```
429
+
430
+ Content-aware overrides — dropping the empty paragraphs an editor keeps
431
+ around the cursor, since `node.content` arrives already rendered:
432
+
433
+ ```ruby
434
+ lexical = Y::Lexical.new(doc) do |rules|
435
+ rules.node "paragraph" do |node|
436
+ node.content.empty? ? "" : "<p>#{node.content}</p>"
437
+ end
438
+ end
439
+ ```
440
+
441
+ For a larger reference, the gem's own editor schemas ship this way — see
442
+ `Y::Lexxy::NODES` in `lib/y/lexxy.rb` (declarative hashes for the simple
443
+ nodes, a plain method per node that needs logic — galleries, list items,
444
+ header cells, both attachment types — mapped with `method(:name)`) and
445
+ `Y::Tiptap::NODES` in `lib/y/tiptap.rb` (task lists, mentions, the details
446
+ family).
447
+
448
+ ### Protocol codec (module functions)
449
+
450
+ Classifying and unwrapping wire frames is stateless, so it's exposed as
451
+ `Y` module functions rather than a class. The server never holds presence
452
+ or document state to route a frame — presence lives in the browser clients, and
453
+ the server only relays awareness frames opaquely.
454
+
455
+ ```ruby
456
+ Y.message_kind(frame) # => 0 drop / 1 step1 / 2 update / 3 awareness / 4 query
457
+ Y.update_from_message(frame) # => the document delta carried by a frame, or nil
458
+ Y.wrap_update(update_bytes) # => wrap a raw doc update as a sync Update frame
459
+ ```
460
+
461
+ ### ActionCable Integration
462
+
463
+ In a Rails app, one generator creates the channel and the migration:
464
+
465
+ ```bash
466
+ bin/rails generate yrby:install
467
+ bin/rails db:migrate
468
+ ```
469
+
470
+ The models ship in the gem, the way Action Text owns
471
+ `ActionText::RichText`:
472
+
473
+ - **`Y::Document`** — one row per document, addressed two ways: by `key`
474
+ (what a channel addresses — one opaque, unique string, sometimes
475
+ app-supplied, never parsed) and, optionally, by polymorphic `record` +
476
+ `name` (which model attribute it backs; `name` is the attribute name,
477
+ `"body"` — one document per attribute per record, the
478
+ ActionText::RichText scheme). Key-only documents leave the binding nil.
479
+ Either side can arrive first: `Y::Document.for(record, name)` finds or
480
+ creates the binding, derives a readable key (`post/1/body`), and adopts
481
+ a key-only row already holding that key, so a channel writing first and
482
+ a binding created later converge on one document. The row also holds
483
+ the merged `state` snapshot — CRDT state only; derived data (rendered
484
+ HTML, search text) is the application's job, typically in the channel's
485
+ on_change. `.load_state(key)` / `.append(key, update)` are the store
486
+ calls the generated channel uses.
487
+ - **`Y::DocumentUpdate`** — the uncompacted tail: one delta per row,
488
+ compacted into `state` and deleted once the tail reaches `compact_every`
489
+ (default 64). Loading reads the snapshot plus the current tail; an
490
+ empty tail returns `state` directly. Compaction serializes on a
491
+ per-document row lock and skips causally-gapped rows — they're
492
+ quarantined until they heal rather than compacted into state or
493
+ deleted. Destroying a document deletes its updates with it.
494
+
495
+ The migration creates `y_documents` and `y_document_updates`. To rename
496
+ them, edit the generated migration and point `Y::Document.table_name` /
497
+ `Y::DocumentUpdate.table_name` at the new names in an initializer.
498
+
499
+ Storage is swappable: the channel only needs `on_load` and `on_change`
500
+ answered, and they can point at anything.
501
+
502
+ `include Y::ActionCable` (from the `yrby-rails` gem) is the channel
503
+ integration: the y-websocket protocol (document sync +
504
+ awareness/presence) over ActionCable. (`include Y::ActionCable::Sync`
505
+ keeps working and has the same effect.)
506
+
507
+ ```ruby
508
+ # app/channels/document_channel.rb
509
+ class DocumentChannel < ApplicationCable::Channel
510
+ include Y::ActionCable
511
+
512
+ on_load { |key| Y::Document.load_state(key) } # rebuild from storage
513
+ on_change { |key, update| Y::Document.append(key, update) } # record, then broadcast
514
+
515
+ def subscribed
516
+ return reject unless authorized?(params[:id])
517
+
518
+ sync_subscribed params[:id]
519
+ end
520
+
521
+ def receive(data)
522
+ sync_receive(data, params[:id])
523
+ end
524
+
525
+ private
526
+
527
+ # Everyone is denied until you wire this to your app's auth.
528
+ def authorized?(_document_key) = false
529
+ end
530
+ ```
531
+
532
+ The concern is store-backed. A handshake is answered from `on_load`; document
533
+ changes are checked against that durable state, recorded through `on_change`,
534
+ then broadcast. Nothing authoritative is kept in ActionCable process memory, so
535
+ AnyCable RPC workers, Puma workers, and separate dynos can all handle messages
536
+ for the same document as long as they share the same store and cable adapter.
537
+
538
+ `on_load` and `on_change` are required. If either is missing, the channel fails
539
+ before it can acknowledge or broadcast edits. Presence is ephemeral:
540
+ awareness frames are relayed, and `yrby-client` sends a best-effort
541
+ presence-removal frame on disconnect/pagehide, with the client-side awareness
542
+ timeout as the fallback for abrupt disconnects.
543
+
544
+ Incoming frames are validated as a single well-formed protocol message before
545
+ anything processes or relays them. Malformed, truncated, multi-message,
546
+ oversized, or unknown frames are dropped. A bad frame can't crash the process: a
547
+ Rust panic is caught at the FFI boundary and re-raised as a Ruby exception. And
548
+ no single client can relay garbage that breaks the others in a room.
549
+
550
+ #### Delivery guarantees
551
+
552
+ The contract is the same at every scale — one process, or hundreds across many
553
+ servers:
554
+
555
+ - **The document always converges.** CRDT updates are commutative and
556
+ idempotent, so out-of-order, duplicate, or concurrent delivery all converge to
557
+ the same correct document. This needs no coordination and holds everywhere.
558
+ - **The durable log never goes gappy.** An update is recorded only once its
559
+ causal dependencies are already in the store (checked against `on_load`); a
560
+ causally-incomplete update triggers a resync instead, so the log always
561
+ rebuilds cleanly.
562
+ - **`on_change` is at-least-once, and the durable guarantee is that replaying the
563
+ log reconstructs the document.** Every update triggers `on_change` before it's acked or
564
+ broadcast (record-before-distribute). If exactly-once updates matter for you, **you
565
+ must make `on_change` idempotent**. But remember that the CRDT can handle duplicates.
566
+ - **A raising `on_change` rejects the update implicitly.** If the block raises,
567
+ the update is neither acked nor broadcast (record-before-distribute stops both).
568
+ There is no negative-ack: the client simply never receives the ack, keeps the
569
+ update pending, and retransmits on its timer/reconnect. This is built for
570
+ *transient* failures (the store is briefly down → a retry lands). A block that
571
+ raises *deterministically* — a validation that always fails for this edit —
572
+ will be retried forever, since nothing tells the client to stop. Enforce hard
573
+ rejections before the edit reaches `on_change` (channel authorization in
574
+ `subscribed`), not by raising inside it.
575
+ - **An over-cap frame is dropped the same silent way.** A frame larger than
576
+ `max_frame_bytes` (default 8 MiB) is dropped before decoding — no ack, no
577
+ broadcast — to bound the work a client can force. For a genuine document
578
+ update that means the same implicit rejection as above: unacked, retransmitted
579
+ forever. Normal typing never approaches the cap, but a large paste, an embedded
580
+ image, or a big initial `SyncStep2` can. The drop is logged (`warn` for
581
+ over-cap, `debug` for undecodable) with the document key and update id so it's
582
+ findable; override `sync_log_context` on the channel to add a user/connection
583
+ id. Size the cap for your largest expected payload, and reject
584
+ genuinely-too-big content upstream rather than relying on the cap to reject it
585
+ gracefully.
586
+
587
+ #### Multi-process deployments
588
+
589
+ Most Rails apps run several processes, and any of them might serve a given document.
590
+ Two pieces keep them in step.
591
+
592
+ Broadcasts cross processes through the Action Cable adapter, so it needs to something
593
+ like `redis` or `solid_cable`, not `async`. With that in place, a change
594
+ on one process reaches clients on all of them.
595
+
596
+ Every process rebuilds document state from the durable store through `on_load`.
597
+ Because changes are recorded before broadcast, record-before-distribute holds
598
+ across processes: whichever process receives a change records it to the shared
599
+ store before anyone, anywhere, sees it.
600
+
601
+ `bun multiprocess.mjs` in the demo runs clients across two processes and checks
602
+ convergence, fresh reads on both, presence across processes, and one shared log.
603
+
604
+ ##### AnyCable
605
+
606
+ `yrby` fully supports AnyCable.
607
+
608
+ The demo checks this against a real anycable-go + RPC server
609
+ (`frontend/anycable_probe.mjs`, `anycable_concurrent.mjs`): liveness, the
610
+ yrby client provider, cross-process reads, and concurrent convergence.
611
+
612
+ ##### Demo
613
+
614
+ [`examples/actioncable-demo`](examples/actioncable-demo) is a full Rails + Tiptap
615
+ app using the yrby provider, with end-to-end tests.
616
+
617
+ #### Record Before Distribute
618
+
619
+ Every document change is handed to the `on_change` handler before broadcasting.
620
+ It is up to you to durably record it:
621
+
622
+ ```ruby
623
+ class DocumentChannel < ApplicationCable::Channel
624
+ include Y::ActionCable
625
+
626
+ # ...
627
+
628
+ on_change do |key, update|
629
+ # Synchronous, durable write. `update` is the exact CRDT delta.
630
+ AuditLog.append!(key, update) # raise to REJECT the change
631
+ end
632
+
633
+ # ...
634
+ end
635
+ ```
636
+
637
+ If the recorder raises (say the store is down), the change is rejected: not
638
+ applied, not sent to anyone. The cost is a synchronous durable write on the path
639
+ of every change. There's no in-gem per-document lock; concurrent writes to one
640
+ document can both record (at-least-once), and since CRDT apply is idempotent a
641
+ duplicate record replays to the same document.
642
+
643
+ The demo wires `on_change` to a durable Postgres-backed log by default, and checks
644
+ end to end that the log alone rebuilds the document.
645
+
646
+ #### Reliable delivery (acks)
647
+
648
+ yrby document delivery is ack-tracked. Browser document updates carry an
649
+ `"id"`, and the server replies `{ "ack": <id> }` once `on_change` has succesfully fired.
650
+ A causally-gapped update is not acked; the server sends a resync request, and
651
+ the client keeps the update queued until it lands.
652
+
653
+ ```
654
+ client -> server { "update": "<base64 update>", "id": 42 }
655
+ server -> client { "ack": 42 } # update accepted; safe to forget
656
+ ```
657
+
658
+ `yrby-client`'s `ActionCableProvider` handles this automatically. It keeps
659
+ the unacknowledged local document tail in a queue and sends the merged tail as a
660
+ single causally-complete delta. The id is the highest sequence in the batch, so
661
+ one `{ ack: id }` cumulatively confirms everything up to it. Because CRDT apply
662
+ is idempotent, a resend that already landed is a harmless no-op that just
663
+ re-acks. Awareness stays ephemeral and is not acked.
664
+
665
+ Presence (cursors, selections) is owned by the browser clients — the server
666
+ never sets or holds presence state, it only relays awareness frames opaquely.
667
+ See `yrby-client` for the client-side awareness API.
668
+
669
+ ## Thread Safety
670
+
671
+ A `Doc` is safe to share across Ruby threads — used concurrently from Puma
672
+ workers, ActionCable connection threads, or background jobs without external
673
+ locking.
674
+
675
+ `test/thread_safety_test.rb` runs shared docs, the full sync handshake, and
676
+ fan-in sync across 8 threads at once, and checks the interleaving doesn't change
677
+ convergence.
678
+
679
+ ### Parallelism (GVL release)
680
+
681
+ Every method that does real CRDT work (applying updates, encoding state,
682
+ handling sync messages) releases Ruby's Global VM Lock
683
+ (`rb_thread_call_without_gvl`) while the native code runs. That buys two things.
684
+
685
+ CRDT work runs in parallel across Ruby threads on MRI, not just
686
+ JRuby/TruffleRuby. `bench/parallelism_bench.rb` measures over 2x wall-clock
687
+ speedup applying a ~900 KB update concurrently; native code that held the GVL
688
+ couldn't beat serial time.
689
+
690
+ A slow operation also can't stall the VM. A thread applying a large update holds
691
+ the doc's write lock without holding the GVL, so other Ruby threads keep running
692
+ instead of queuing behind it.
693
+
694
+ Each method has the same shape: copy Ruby byte strings first, drop the GVL, do
695
+ the yrs work while taking and releasing native locks entirely inside the
696
+ closure, take the GVL back, then build Ruby objects. No Ruby API is touched
697
+ without the GVL, and no native lock is held while reacquiring it, so the lock
698
+ order can't deadlock. Panics in native code are caught and re-raised as Ruby
699
+ exceptions.
700
+
701
+ ## Message Type Constants
702
+
703
+ ```ruby
704
+ Y::MSG_SYNC # 0 - Document sync messages
705
+ Y::MSG_AWARENESS # 1 - User presence data
706
+
707
+ Y::MSG_SYNC_STEP1 # 0 - State vector request
708
+ Y::MSG_SYNC_STEP2 # 1 - Update response
709
+ Y::MSG_SYNC_UPDATE # 2 - Incremental update
710
+ ```
711
+
712
+ ## Sync Flow
713
+
714
+ ```
715
+ Client A Server
716
+ | |
717
+ |-------- connect() ------------->|
718
+ | (SyncStep1 + Awareness) |
719
+ | |
720
+ |<--- handle_sync_message resp ---|
721
+ | (SyncStep2) |
722
+ | |
723
+ | (Document synchronized!) |
724
+ | |
725
+ |<------- updates ----------------|
726
+ |-------- updates --------------->|
727
+ ```
728
+
729
+ ## Development
730
+
731
+ ```bash
732
+ # Setup
733
+ bundle install
734
+
735
+ # Build extension
736
+ rake compile
737
+
738
+ # Run tests
739
+ rake test
740
+
741
+ # Clean build artifacts
742
+ rake clean
743
+ ```
744
+
745
+ ## License
746
+
747
+ MIT License
748
+
749
+ ## Acknowledgments
750
+
751
+ - [y-crdt/yrs](https://github.com/y-crdt/y-crdt) - The Rust implementation of Y.js
752
+ - [Magnus](https://github.com/matsadler/magnus) - Ruby bindings for Rust
753
+ - [rb-sys](https://github.com/oxidize-rb/rb-sys) - Rust extensions for Ruby