yrby 0.7.1-aarch64-linux → 0.8.0-aarch64-linux

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -2,60 +2,87 @@
2
2
 
3
3
  [![CI](https://github.com/jpcamara/yrby/actions/workflows/ci.yml/badge.svg)](https://github.com/jpcamara/yrby/actions/workflows/ci.yml)
4
4
 
5
- yrby (pronounced "yer-bee") makes Rails a real Yjs backend. It binds
5
+ yrby (pronounced "yer-bee") makes Rails a Yjs backend. It binds
6
6
  [y-crdt](https://github.com/y-crdt/y-crdt), the Rust engine behind Y.js, into
7
- Ruby, and builds the rest of the stack around it: a sync server for Action
7
+ Ruby, and on top of that it builds a sync server for Action
8
8
  Cable and AnyCable, a browser provider, and server-side reading and rendering
9
- of the documents. Real-time collaboration in a Rails app with no Node process
10
- anywhere in the path.
9
+ of the documents. You get real-time collaboration in a Rails app with no Node
10
+ process to run.
11
11
 
12
12
  ![Two people typing on separate lines of the same document, each keystroke synced through a Rails server, seen from a third browser with labeled carets](docs/images/collab.gif)
13
13
 
14
- On the server, `yrby-rails` implements the full y-websocket protocol
15
- (document sync plus presence) as a channel concern. Its delivery contract is
16
- stricter than the usual Yjs servers: every update is ack-tracked and durably
17
- recorded before it is acknowledged or broadcast to anyone. Replaying your
18
- store always rebuilds the document, across any number of processes.
19
- ([Delivery guarantees](#delivery-guarantees))
14
+ On the server, `yrby-rails` implements the y-websocket protocol (document sync
15
+ and presence) as a channel concern. The server writes every update to your
16
+ store before it acknowledges the update or sends it to other clients, which
17
+ most Yjs servers don't do. Replaying the store rebuilds the document, however
18
+ many processes you run. ([Delivery guarantees](#delivery-guarantees))
20
19
 
21
20
  In the browser, `yrby-client`'s `ActionCableProvider` connects anything that
22
21
  speaks Yjs. The demo app runs four rich text editors, and CI drives each one
23
22
  in real Chrome: Tiptap, [Lexxy](https://www.npmjs.com/package/lexxy-realtime),
24
23
  Rhino Editor, and CodeMirror. The same channel also syncs Yjs shapes with no
25
24
  editor at all: a whiteboard on a `Y.Map`, a kanban board on a `Y.Array`, a
26
- co-filled form. ([Editors](#editors))
25
+ form filled in together. ([Editors](#editors))
27
26
 
28
27
  In Ruby, the documents are readable without a browser. `Doc#read_text` and
29
- `Doc#read_map` reconstruct contents for search, validation, and exports.
30
- `Y::Tiptap` and `Y::Lexxy` render a document to HTML byte-identical to the
31
- editor's own serializer, take rules for your app's custom nodes, and drop
32
- straight into ActionText. ([Rendering to HTML](#rendering-to-html))
28
+ `Doc#read_map` rebuild the contents for search, validation, and exports.
29
+ `Y::Tiptap` and `Y::Lexxy` render a document to HTML that matches the editor's
30
+ own serializer byte for byte. You can add rules for your app's custom nodes,
31
+ and you can save the output directly to ActionText.
32
+ ([Rendering to HTML](#rendering-to-html))
33
33
 
34
34
  Underneath, the core is built for a production Rails deployment. A `Doc` is
35
35
  thread-safe across Puma and ActionCable threads. Native CRDT work runs with
36
- the GVL released, so it parallelizes on MRI. Incoming frames are validated
36
+ the GVL released, so it runs in parallel on MRI. Incoming frames are validated
37
37
  before anything processes them, and multi-process and AnyCable setups are
38
38
  tested end to end. ([Thread Safety](#thread-safety))
39
39
 
40
- The whole server side of a collaborative document is one channel:
40
+ You add one view helper, on a page where the user can already edit the record:
41
41
 
42
- ```ruby
43
- class DocumentChannel < ApplicationCable::Channel
44
- include Y::ActionCable
42
+ ```erb
43
+ <%= collaborative_document_tag @post, :body %>
44
+ ```
45
45
 
46
- on_load { |key| Y::Document.load_state(key) }
47
- on_change { |key, update| Y::Document.append(key, update) }
46
+ The helper renders a signed grant for that record and attribute, much as
47
+ `turbo_stream_from` signs its stream names. The client subscribes with the
48
+ grant and the attribute name. The gem's `Y::DocumentChannel` verifies the
49
+ grant, then saves every change as `Y::Document` rows before acknowledging it,
50
+ so you don't have to write a channel yourself.
48
51
 
49
- def subscribed = sync_subscribed(params[:id])
50
- def receive(data) = sync_receive(data, params[:id])
51
- end
52
+ The tag renders an element that connects by itself once you import it. When the
53
+ document has synced, your listener receives it and passes it to your editor
54
+ binding:
55
+
56
+ ```js
57
+ import "yrby-client/element"
58
+
59
+ document.addEventListener("yrby:synced", ({ target, detail }) => {
60
+ const editor = bindYourEditor(target, detail.doc, detail.provider)
61
+ detail.signal.addEventListener("abort", () => editor.destroy(), { once: true })
62
+ })
63
+ ```
64
+
65
+ If the user leaves the page with edits still unsent, the session keeps them and
66
+ finishes delivering. Your binding cleans up when the abort signal fires, and if
67
+ nothing is pending, the next visit loads the document from Rails. The [client
68
+ lifecycle and recovery contract](packages/client/README.md#document-sessions)
69
+ has the details.
70
+
71
+ The document is stored as rows in your database, and you can read it back in
72
+ Ruby:
73
+
74
+ ```ruby
75
+ doc = post.collaborative_document(:body).y_doc
76
+ doc.read_text("content") # or Y::Lexxy.new(doc).to_html for rich text
52
77
  ```
53
78
 
54
79
  Install the gem and the npm package:
55
80
 
56
81
  ```
57
82
  gem install yrby-rails # depends on yrby
58
- npm install yrby-client
83
+ npm install yrby-client yjs y-protocols @rails/actioncable
84
+
85
+ bin/rails generate yrby:install && bin/rails db:migrate
59
86
  ```
60
87
 
61
88
  ## Contents
@@ -84,51 +111,54 @@ npm install yrby-client
84
111
 
85
112
  ## Scope
86
113
 
87
- `yrby` binds just the part of `y-crdt` you need to *sync and persist* collaborative
88
- documents - a `Doc`, awareness, and the y-websocket protocol primitives. By default
89
- the Ruby side treats a document as opaque CRDT state: it applies updates, answers
90
- sync handshakes, and records deltas without reaching into the contents - the browser
91
- editor owns the document's shape. When you do need to look inside, `Doc#read_text`
92
- and `Doc#read_map` reconstruct it server-side, in Ruby.
114
+ `yrby` binds the parts of `y-crdt` you need to sync and persist collaborative
115
+ documents, which are a `Doc`, awareness, and the y-websocket protocol
116
+ primitives. By default the Ruby side treats a document as opaque CRDT state. It
117
+ applies updates, answers sync handshakes, and records deltas without reading
118
+ the contents, and the browser editor decides what shape the document has. When
119
+ you do need to look inside, `Doc#read_text` and `Doc#read_map` rebuild it in
120
+ Ruby.
93
121
 
94
122
  ## Durability and delivery
95
123
 
96
- The surface is intentionally small, but the focus is durability, resiliency, delivery
124
+ The API is small, and most of the work went into durability, resiliency, delivery
97
125
  guarantees, correctness, and thread safety.
98
126
 
99
- Towards that goal, `yrby` adds opinionated defaults on top of normal Yjs syncing:
100
-
101
- - Built-in update acknowledgement: the `ActionCableProvider` in `yrby-client` keeps
102
- sending an update until the server acks it, and [`yrby-rails`](https://rubygems.org/gems/yrby-rails)
103
- only acks once the update is durably recorded. That gives you at-least-once
104
- delivery, and because CRDT updates are idempotent a duplicate is a no-op.
105
- - Gap awareness: an update can arrive before another update it depends on (a
106
- "causal gap"). `yrby` records and acks it like any other, and the document
107
- heals on its own once the missing update arrives; its sender keeps
108
- retransmitting it until it is acked. `Doc#pending?` and the `on_gap` hook
127
+ `yrby` adds two opinionated defaults on top of normal Yjs syncing:
128
+
129
+ - yrby acknowledges updates. The `ActionCableProvider` in `yrby-client` keeps
130
+ resending an update until the server acks it, and
131
+ [`yrby-rails`](https://rubygems.org/gems/yrby-rails) only acks an update
132
+ after it's durably recorded. Together that gives you at-least-once delivery,
133
+ and because CRDT updates are idempotent, a duplicate does nothing.
134
+ - yrby expects causal gaps, where an update arrives before another update it
135
+ depends on, and it records and acks that update like any other. The sender of
136
+ the missing update keeps retransmitting it until it's acked, and the gap
137
+ closes when it arrives. `Doc#pending?` and the `on_gap` hook
109
138
  tell you when a document is waiting on a missing update.
110
139
  ([Causal gaps](#causal-gaps))
111
140
 
112
141
  ## What about [yrb](https://github.com/y-crdt/yrb)?
113
142
 
114
- `yrb` has a much larger interface that gives you most of the Yjs type system -
115
- shared text, arrays, maps, XML - to build and query documents in Ruby. It was a great
116
- inspiration for my use of Yjs in Ruby/Rails, and I originally considered building
117
- on top of it. There are a few reasons I went with `yrby` instead:
143
+ `yrb` has a much larger interface, with most of the Yjs type system (shared
144
+ text, arrays, maps, XML) for building and querying documents in Ruby. It was a
145
+ big inspiration for me in using Yjs from Ruby and Rails, and I considered
146
+ building on top of it. I went with `yrby` for a few reasons:
118
147
 
119
- - `yrb` is largely unmaintained. It was built as an experiment for GitLab, and the original
120
- author mostly moved onto other projects.
121
- - [It isn't thread-safe](https://github.com/y-crdt/yrb/issues/72). It segfaults in a threaded
122
- environment (such as ActionCable...)
123
- - It's a much larger set of features to maintain, which most people don't need. The vast
124
- majority of people manipulate Y.js documents in the browser, not from a server-side language.
148
+ - `yrb` started as an experiment for GitLab, and its original author has mostly
149
+ moved on to other projects, so it's largely unmaintained.
150
+ - [It isn't thread-safe](https://github.com/y-crdt/yrb/issues/72), and it
151
+ segfaults in threaded environments like ActionCable.
152
+ - It's a much larger set of features to maintain, and most people don't need
153
+ them. The vast majority of Y.js documents are only ever manipulated in the
154
+ browser.
125
155
 
126
156
  ## Testing
127
157
 
128
158
  Ruby and Rust unit tests cover the core. CI also runs the npm client tests and a
129
159
  Rails demo smoke slice against the real ActionCable stack. The demo includes
130
160
  heavier local suites for hostile input, crash recovery, multi-browser editing,
131
- AnyCable, and load testing. The benchmark note below is from a single laptop.
161
+ AnyCable, and load testing. The benchmark number below is from a single laptop.
132
162
  Issues and PRs are welcome.
133
163
 
134
164
  ## Install
@@ -142,10 +172,11 @@ gem "yrby"
142
172
  gem "yrby-rails"
143
173
  ```
144
174
 
145
- Requires Ruby 3.4 or newer. The release workflow builds precompiled gems for
146
- Ruby 3.4 and 4.0 across the supported Ruby platforms, with native smoke tests
147
- on Linux x86_64 and macOS arm64. Installing from a matching platform gem needs
148
- no Rust; a source build needs [Rust](https://rustup.rs).
175
+ yrby needs Ruby 3.4 or newer. Releases include precompiled gems for Ruby 3.4
176
+ and 4.0 on the supported Ruby platforms, and the release workflow smoke-tests
177
+ the native builds on Linux x86_64 and macOS arm64. You only need
178
+ [Rust](https://rustup.rs) when no precompiled gem matches your platform and the
179
+ gem builds from source.
149
180
 
150
181
  To work on the gem itself:
151
182
 
@@ -163,16 +194,17 @@ The rest of the dev setup, plus the demo, is in [CONTRIBUTING.md](CONTRIBUTING.m
163
194
  - The ActionCable concern and a quickstart are [below](#actioncable-integration).
164
195
  - [`examples/actioncable-demo`](examples/actioncable-demo): a runnable Rails +
165
196
  Tiptap app with collaborative cursors, the AnyCable setup, a Postgres store,
166
- and the test/load suites.
197
+ and the test and load suites.
167
198
  - [CHANGELOG.md](CHANGELOG.md) and [CONTRIBUTING.md](CONTRIBUTING.md).
168
199
 
169
200
  ## Editors
170
201
 
171
- yrby syncs opaque Yjs updates, so it works with any editor that has a Yjs
172
- binding. The demo app runs four, and CI drives each one in real Chrome:
202
+ yrby syncs opaque Yjs updates, so any editor with a Yjs binding works. The demo
203
+ app runs four of them, and CI drives each one in real Chrome to check
173
204
  concurrent typing with every keystroke accounted for, remote cursors,
174
- local-only undo, and byte parity between the server-side renderers and the
175
- editor's own serializer. Each page is a working integration to copy from:
205
+ local-only undo, and that the server-side renderers produce the same HTML as
206
+ the editor's own serializer. Each demo page is a working integration you can
207
+ copy from:
176
208
 
177
209
  | Editor | Yjs binding | Demo code |
178
210
  |---|---|---|
@@ -181,11 +213,11 @@ editor's own serializer. Each page is a working integration to copy from:
181
213
  | [Rhino Editor](https://github.com/KonnorRogers/rhino-editor) (Tiptap 3) | `@tiptap/extension-collaboration` + `-caret` | [`rhino.js`](examples/actioncable-demo/frontend/src/rhino.js) |
182
214
  | [CodeMirror 6](https://codemirror.net) | `y-codemirror.next` | [`codemirror.js`](examples/actioncable-demo/frontend/src/codemirror.js) |
183
215
 
184
- The demo also syncs plain Yjs shapes with no editor at all (a whiteboard
185
- on a `Y.Map`, a kanban board on a `Y.Array`, a co-filled form) over the
186
- same channel. The demo README's "Using this in your own app" section has
187
- the integration recipe, and its `NoteMaterializer` shows how to render a
188
- document to ActionText server-side with `Y::Tiptap` or `Y::Lexxy`.
216
+ The demo also syncs plain Yjs shapes with no editor (a whiteboard on a
217
+ `Y.Map`, a kanban board on a `Y.Array`, a form filled in together) over the
218
+ same channel. The demo README's "Using this in your own app" section has the
219
+ integration recipe, and its `NoteMaterializer` shows how to render a document
220
+ to ActionText on the server with `Y::Tiptap` or `Y::Lexxy`.
189
221
 
190
222
  ## Usage
191
223
 
@@ -219,43 +251,46 @@ doc.handle_sync_message(data) # => [msg_type, sync_type, response]; answers
219
251
 
220
252
  ### Reading document contents
221
253
 
222
- Reconstruct a document server-side (search, exports, emails, SSR) with no
254
+ Rebuild a document on the server for search, exports, emails, or SSR, with no
223
255
  Node process:
224
256
 
225
257
  ```ruby
226
258
  doc.read_text("prosemirror") # => plain text of a Y.Text root, or nil
227
259
  doc.read_xml("root") # => text of an XML root, one block per line
228
260
  doc.read_map("state") # => a Y.Map root as a JSON string; JSON.parse it
261
+ doc.read_array("cards") # => a Y.Array root as a JSON string; JSON.parse it
229
262
  ```
230
263
 
231
264
  ### Pending structs and gap-free state
232
265
 
233
266
  If a doc applies an update whose causally-prior update is missing (a "gappy"
234
- update), yrs parks it as a **pending** struct: the integrated state vector stays
235
- empty, but the pending block is held as a recovery buffer and heals if the
236
- missing dependency later arrives. `Doc#pending?` reports this.
267
+ update), yrs holds it as a **pending** struct and the integrated state vector
268
+ doesn't move. yrs keeps the pending block as a recovery buffer and integrates
269
+ it if the missing dependency arrives later. `Doc#pending?` tells you when a doc
270
+ is in this state.
237
271
 
238
- Pending structs travel like any other state. `handle_sync_message` answers
239
- `SyncStep1` with the doc's full state, pending included, just like Y.js's
240
- `encodeStateAsUpdate`: a peer parks the pending struct the same way this doc
241
- did and heals it the same way. The one place pending must not go is a
242
- compacted snapshot:
272
+ `handle_sync_message` answers `SyncStep1` with the doc's full state, pending
273
+ structs included, which is what Y.js's `encodeStateAsUpdate` does too. A peer
274
+ that receives it holds the pending struct and integrates it the same way this
275
+ doc would. The one place pending structs must be left out is a compacted
276
+ snapshot:
243
277
 
244
278
  - `Doc#compacted_state_update` returns a gap-free full-state update for
245
279
  compaction. Folding a log into one blob would otherwise freeze an
246
- un-integrable struct into the base state forever. It's non-destructive: the
247
- doc keeps its pending.
248
- - `encode_state_as_update` stays lossless, so persistence and serving keep
249
- the raw pending bytes and the gap can still heal.
280
+ un-integrable struct into the base state permanently. The call doesn't modify
281
+ the doc, which keeps its pending structs.
282
+ - `encode_state_as_update` is lossless, so persistence and serving keep the
283
+ raw pending bytes and the gap can still close.
250
284
 
251
285
  ### Rendering to HTML
252
286
 
253
- Schema-pinned renderers turn a collaborative document into HTML on the
254
- server, with no Node process or headless editor. Each is an editor-specific
255
- class (byte-for-byte with that editor's own serializer) built on a core base
256
- any other editor extends with rules: `Y::Tiptap` on `Y::ProseMirror` for
257
- ProseMirror documents, and `Y::Lexxy` (the
258
- [Lexxy](https://github.com/basecamp/lexxy) editor) on `Y::Lexical`. Each
287
+ The renderers turn a collaborative document into HTML on the server with no
288
+ Node process or headless editor. Each renderer is a class for one editor, and
289
+ its output matches that editor's own serializer. `Y::Tiptap` renders
290
+ ProseMirror documents and is built on `Y::ProseMirror`, and `Y::Lexxy` renders
291
+ documents from the [Lexxy](https://github.com/basecamp/lexxy) editor and is
292
+ built on `Y::Lexical`. To support another editor on one of those engines,
293
+ extend the matching base class with rules. Each renderer
259
294
  returns `nil` for a root that belongs to the other schema.
260
295
 
261
296
  #### `Y::Tiptap` (and `Y::ProseMirror`, its base)
@@ -266,26 +301,27 @@ tiptap.to_html # the "default" fragment (Tiptap's default root)
266
301
  tiptap.to_html("content") # or another XML root
267
302
  ```
268
303
 
269
- The output matches Tiptap's own `getHTML()`, checked byte-for-byte in the tests
270
- against a document captured from a real editor. It follows
271
- [`tiptap-php`](https://github.com/ueberdosis/tiptap-php) and reads both name
272
- styles editors use: Tiptap's `bulletList`/`bold` and prosemirror-schema-basic's
273
- `bullet_list`/`strong`.
274
-
275
- It covers paragraphs, headings, blockquotes, bullet/ordered/task lists, code
276
- blocks, links, images, mentions, details, hard breaks, horizontal rules,
277
- tables, text styles (color, font family), and every text mark. A table renders
278
- as semantic `<table><tbody>`, without the column-width styling Tiptap's editor
279
- view adds.
280
-
281
- The support is layered like the Lexical side: `Y::ProseMirror` covers core
282
- ProseMirror natively (prosemirror-schema-basic plus the prosemirror-tables
283
- family) and Tiptap's extension nodes (task lists, mentions, the details
284
- family) are `Y::Tiptap`'s rule set (`Y::Tiptap::NODES`), built on the
285
- extension API below. Marks stay in the base: mark rendering (nesting order,
286
- `textStyle` CSS, `code` exclusivity) runs through native text-run machinery
287
- that node rules don't reach, so `Y::ProseMirror` renders Tiptap's mark set
288
- as-is and `rules.mark` overrides individual marks.
304
+ The output matches Tiptap's own `getHTML()`, and the tests check it against a
305
+ document captured from a real editor. The implementation follows
306
+ [`tiptap-php`](https://github.com/ueberdosis/tiptap-php) and accepts both
307
+ naming styles editors use, Tiptap's `bulletList` and `bold` as well as
308
+ prosemirror-schema-basic's `bullet_list` and `strong`.
309
+
310
+ It covers paragraphs, headings, blockquotes, bullet, ordered, and task lists,
311
+ code blocks, links, images, mentions, details, hard breaks, horizontal rules,
312
+ tables, text styles (color and font family), and every text mark. Tables render
313
+ as a plain `<table><tbody>` without the column-width styling that Tiptap's
314
+ editor view adds.
315
+
316
+ Support is split into two layers, as on the Lexical side. `Y::ProseMirror`
317
+ handles core ProseMirror natively, meaning prosemirror-schema-basic plus the
318
+ prosemirror-tables family. Tiptap's extension nodes (task lists, mentions, the
319
+ details family) are a rule set, `Y::Tiptap::NODES`, written with the extension
320
+ API described below. Marks are handled in the base class. Mark rendering deals
321
+ with nesting order, the CSS on `textStyle`, and the exclusivity of `code`, and
322
+ it runs through native text-run code that node rules can't reach, so
323
+ `Y::ProseMirror` renders Tiptap's mark set as is. You can override individual
324
+ marks with `rules.mark`.
289
325
 
290
326
  #### `Y::Lexxy` (and `Y::Lexical`, its base)
291
327
 
@@ -295,41 +331,41 @@ lexxy.to_html # the "root" fragment (Lexical's default root name)
295
331
  lexxy.to_html("notepad") # or another XML root
296
332
  ```
297
333
 
298
- The HTML is identical to what a `lexxy-editor` submits to Rails (its `value`).
299
- The tests check this byte-for-byte against a document captured from a real
300
- editor. Stock Lexical has no canonical serializer (every editor configures
301
- its own), so the editor-specific class carries the editor's name, and
302
- `Y::Lexical` is the core-Lexical base: paragraphs, headings, quotes, code,
303
- lists, tables, links, and the full text-format model, for any other Lexical
304
- editor to extend with rules.
334
+ The HTML is identical to the `value` a `lexxy-editor` submits to Rails, and the
335
+ tests check it against a document captured from a real editor. The class is
336
+ named after the editor because stock Lexical has no canonical serializer, and
337
+ every editor configures its own. `Y::Lexical` is the core Lexical base
338
+ (paragraphs, headings, quotes, code, lists, tables, links, and the full
339
+ text-format model), and other Lexical editors extend it with rules.
305
340
 
306
- It handles the whole Lexxy 0.9.x node set: paragraphs, headings, every text
307
- format and their combinations, links, the four list types and nesting,
341
+ It handles every node in the Lexxy 0.9.x set: paragraphs, headings, every text
342
+ format and their combinations, links, the four list types with nesting,
308
343
  blockquotes, code blocks, tabs and soft breaks, horizontal rules, tables with
309
- header cells, image galleries, and ActionText attachments (uploads and
310
- mentions both emit `<action-text-attachment>` elements that ActionText can
311
- re-render).
344
+ header cells, image galleries, and ActionText attachments. Uploads and mentions
345
+ both render as `<action-text-attachment>` elements, which ActionText can
346
+ re-render.
312
347
 
313
- Internally that support is layered: `Y::Lexical` covers core Lexical
314
- structure natively, and everything Lexxy adds, its node types (attachments,
315
- galleries) and its decorations of core nodes (the table wrapper, header-cell
316
- styling, nested-list classes), is `Y::Lexxy`'s rule set
317
- (`Y::Lexxy::NODES`), built on the extension API below. The gem's own Lexxy
318
- support is the API's first consumer: an app rule for one of those types
319
- simply replaces it.
348
+ The Lexical support has the same two layers. `Y::Lexical` handles core Lexical
349
+ structure natively, and everything Lexxy adds is in `Y::Lexxy`'s rule set,
350
+ `Y::Lexxy::NODES`, written with the extension API below. That includes Lexxy's
351
+ own node types (attachments, galleries) and its decorations of core nodes (the
352
+ table wrapper, header-cell styling, nested-list classes). Since the gem's Lexxy
353
+ support was the first thing written on that API, an app rule for one of those
354
+ types replaces the built-in one.
320
355
 
321
- In both renderers an unknown node keeps its content: text and nested blocks
322
- fall back to readable markup rather than disappearing.
356
+ In both renderers, an unknown node still renders its text and nested blocks as
357
+ readable markup.
323
358
 
324
359
  #### Custom nodes and marks
325
360
 
326
- The built-in schemas are pinned to what Tiptap and Lexxy ship, but apps add
327
- their own node types. Both renderers take rules for them. A rule is checked
328
- before the built-in schema, so it can add a node type or replace how a
329
- built-in renders.
361
+ The built-in schemas match what Tiptap and Lexxy ship. When your app adds its
362
+ own node types, both renderers take rules for them. The renderer checks rules
363
+ before the built-in schema, so a rule can add a node type or change how a
364
+ built-in one renders.
330
365
 
331
- Rules register in a block, one `rules.node` call per type. A declarative
332
- rule is markup as data, rendered natively:
366
+ You register rules in a block with one `rules.node` call per type. A
367
+ declarative rule describes the markup as a tag, attributes, and a content mode,
368
+ and the renderer emits it natively:
333
369
 
334
370
  ```ruby
335
371
  tiptap = Y::Tiptap.new(doc) do |rules|
@@ -339,17 +375,18 @@ tiptap = Y::Tiptap.new(doc) do |rules|
339
375
  end
340
376
  ```
341
377
 
342
- `tag` names the element. `attrs` values are templates: a string is a literal,
343
- a symbol reads that attribute off the node, an array concatenates both kinds;
344
- an attribute that resolves empty is left out. `text` (same template form)
345
- emits literal text content. `contains` declares what lives inside the node: `:inline` (formatted text,
346
- the default), `:blocks` (child block nodes, a container), or `:none` (a
347
- leaf). `void: true` skips the closing tag.
378
+ `tag` names the element. The values in `attrs` are templates, where a string is
379
+ a literal, a symbol reads that attribute off the node, and an array
380
+ concatenates a mix of the two. Attributes that resolve to an empty value are
381
+ left out. `text` takes the same template form and emits literal text content.
382
+ `contains` says what goes inside the node, which is `:inline` for formatted
383
+ text (the default), `:blocks` for child block nodes, or `:none` for a leaf.
384
+ `void: true` skips the closing tag.
348
385
 
349
- You don't have to guess any of those names or shapes. Editors store types
350
- and attributes under names you'd never predict (Rhino's strike mark is
351
- `rhino-strike`; Lexical prefixes its own props `__`), so ask a real
352
- document instead: make one in your editor using your custom node, then:
386
+ Editors store types and attributes under names you wouldn't predict. Rhino's
387
+ strike mark is `rhino-strike`, for example, and Lexical prefixes its own props
388
+ with `__`. To find the real names and shapes, make a document in your editor
389
+ that uses your custom node, then run:
353
390
 
354
391
  ```ruby
355
392
  Y::Tiptap.new(doc).node_types
@@ -359,11 +396,12 @@ Y::Tiptap.new(doc).node_types
359
396
  # "paragraph" => { ..., "handled" => "builtin" } }
360
397
  ```
361
398
 
362
- `handled` nil marks the types that still need a rule; `attrs` are the stored
363
- names your templates and blocks will read; `children` plus `text` is how you
364
- pick `contains:` (child block types → `:blocks`; text → `:inline`).
399
+ A type with `handled` set to nil still needs a rule. `attrs` lists the stored
400
+ attribute names your templates and blocks will read, and `children` and `text`
401
+ tell you which `contains:` to pick. Child block types mean `:blocks`, and text
402
+ means `:inline`.
365
403
 
366
- When markup-as-data isn't enough, give the node a block:
404
+ When a declarative rule can't express the markup, give the node a block:
367
405
 
368
406
  ```ruby
369
407
  lexical = Y::Lexical.new(doc) do |rules|
@@ -374,26 +412,26 @@ lexical = Y::Lexical.new(doc) do |rules|
374
412
  end
375
413
  ```
376
414
 
377
- The block gets the node's type, its stored attributes, `node.content` (the
378
- children, already rendered to HTML), and `node.child_types`, the node's
379
- element/block children by type, in document order. `child_types` answers the
380
- structural questions attributes can't: how many images a gallery holds, or
381
- whether a list item carries a nested list. Whatever the block returns is
382
- spliced into the output as-is: it's trusted HTML, so escape any values you
383
- interpolate. To set the content mode for a callback, give the node both:
415
+ The block receives the node's type and stored attributes, plus `node.content`,
416
+ which is the node's children already rendered to HTML. `node.child_types` lists
417
+ the node's element and block children by type, in document order, so you can
418
+ answer questions the attributes can't, like how many images a gallery holds or
419
+ whether a list item has a nested list. The renderer inserts whatever the block
420
+ returns into the output as is and treats it as trusted HTML, so escape any
421
+ values you interpolate. To set the content mode for a block rule, pass both:
384
422
  `rules.node "embed", contains: :blocks do |node| ... end`.
385
423
 
386
- Callbacks never run while the document is locked. The render finishes first
387
- (inside one read transaction, GVL released), then the blocks run and their
388
- output is spliced in, so a callback can safely read or even write the same
389
- doc. With no callback rules, `to_html` skips the splicing entirely.
424
+ Blocks never run while the document is locked. The renderer finishes the whole
425
+ render inside one read transaction with the GVL released, and only then runs
426
+ the blocks and inserts their output, so a block can safely read the same doc or
427
+ even write to it. If no rule has a block, `to_html` skips that last step.
390
428
 
391
- Blocks are the escape hatch for everything the declarative form can't say,
392
- and they're proven sufficient: `Y::Lexxy` and `Y::Tiptap` are themselves
393
- built on this API (`lib/y/lexxy.rb`, `lib/y/tiptap.rb`): simple nodes as
394
- declarative hashes, everything with logic as plain methods mapped by node
395
- type (a `Method` responds to `call` like any lambda), and the fixture tests
396
- hold their output byte-identical to a live editor's.
429
+ Blocks cover whatever the declarative form can't express. `Y::Lexxy` and
430
+ `Y::Tiptap` are both built on this API (`lib/y/lexxy.rb`, `lib/y/tiptap.rb`),
431
+ so it already handles two complete editor schemas. Their simple nodes are
432
+ declarative hashes, and each node with logic is a plain method mapped by node
433
+ type (a `Method` responds to `call` like any lambda). The fixture tests check
434
+ that output against a live editor's.
397
435
 
398
436
  The ProseMirror side also takes custom marks:
399
437
 
@@ -403,14 +441,15 @@ tiptap = Y::Tiptap.new(doc) do |rules|
403
441
  end
404
442
  ```
405
443
 
406
- Symbol refs resolve against the mark's own attributes. A custom mark wraps
407
- outside every built-in mark; several on one run nest alphabetically. A rule
408
- for a built-in mark name (`"bold"`) replaces its built-in tag.
444
+ Symbols read from the mark's own attributes. Custom marks wrap outside all the
445
+ built-in marks, and when several custom marks apply to one run of text, they
446
+ nest alphabetically by name. A rule for a built-in mark name such as `"bold"`
447
+ replaces that mark's tag.
409
448
 
410
449
  ##### Worked examples
411
450
 
412
- A video-embed node from an app's Tiptap extension, a type the pinned schema
413
- has never heard of:
451
+ Here's a video-embed node from an app's Tiptap extension, which the built-in
452
+ schema doesn't know about:
414
453
 
415
454
  ```ruby
416
455
  tiptap = Y::Tiptap.new(doc) do |rules|
@@ -422,8 +461,8 @@ tiptap = Y::Tiptap.new(doc) do |rules|
422
461
  end
423
462
  ```
424
463
 
425
- Resolving mentions against the database. Blocks run after the document read
426
- has finished, so hitting ActiveRecord (or the doc itself) inside one is safe:
464
+ This one resolves mentions against the database. Blocks run after the document
465
+ read has finished, so a block can safely query ActiveRecord or the doc itself:
427
466
 
428
467
  ```ruby
429
468
  tiptap = Y::Tiptap.new(doc) do |rules|
@@ -436,8 +475,9 @@ tiptap = Y::Tiptap.new(doc) do |rules|
436
475
  end
437
476
  ```
438
477
 
439
- Overriding a shipped rule: rendering Lexxy uploads as real image markup
440
- instead of the `<action-text-attachment>` elements ActionText re-renders:
478
+ This one overrides a shipped rule. The shipped rule emits
479
+ `<action-text-attachment>` elements for ActionText to re-render, and the
480
+ override renders Lexxy uploads as `<figure>` and `<img>` markup:
441
481
 
442
482
  ```ruby
443
483
  lexxy = Y::Lexxy.new(doc) do |rules|
@@ -452,9 +492,10 @@ lexxy = Y::Lexxy.new(doc) do |rules|
452
492
  end
453
493
  ```
454
494
 
455
- Markup that depends on structure: `node.child_types` lists the node's
456
- element/block children in document order, so a layout container can size
457
- itself by its column count while the columns themselves stay declarative:
495
+ Some markup depends on structure. Because `node.child_types` lists the node's
496
+ element and block children in document order, a layout container can size
497
+ itself by its column count, and the columns themselves can still use
498
+ declarative rules:
458
499
 
459
500
  ```ruby
460
501
  tiptap = Y::Tiptap.new(doc) do |rules|
@@ -465,8 +506,9 @@ tiptap = Y::Tiptap.new(doc) do |rules|
465
506
  end
466
507
  ```
467
508
 
468
- Content-aware overrides: dropping the empty paragraphs an editor keeps
469
- around the cursor, since `node.content` arrives already rendered:
509
+ A block can also look at the rendered content. This one drops the empty
510
+ paragraphs an editor keeps around the cursor, which works because
511
+ `node.content` is already rendered:
470
512
 
471
513
  ```ruby
472
514
  lexical = Y::Lexical.new(doc) do |rules|
@@ -476,19 +518,20 @@ lexical = Y::Lexical.new(doc) do |rules|
476
518
  end
477
519
  ```
478
520
 
479
- For a larger reference, the gem's own editor schemas ship this way; see
480
- `Y::Lexxy::NODES` in `lib/y/lexxy.rb` (declarative hashes for the simple
481
- nodes, a plain method per node that needs logic (galleries, list items,
482
- header cells, both attachment types) mapped with `method(:name)`) and
521
+ For a larger reference, look at the gem's own editor schemas, which are written
522
+ the same way. See `Y::Lexxy::NODES` in `lib/y/lexxy.rb` (declarative hashes for
523
+ the simple nodes, and a plain method mapped with `method(:name)` for each node
524
+ that needs logic, which means galleries, list items, header cells, and both
525
+ attachment types) and
483
526
  `Y::Tiptap::NODES` in `lib/y/tiptap.rb` (task lists, mentions, the details
484
527
  family).
485
528
 
486
529
  ### Protocol codec (module functions)
487
530
 
488
- Classifying and unwrapping wire frames is stateless, so it's exposed as
489
- `Y` module functions rather than a class. The server never holds presence
490
- or document state to route a frame; presence lives in the browser clients, and
491
- the server only relays awareness frames opaquely.
531
+ Classifying and unwrapping wire frames needs no state, so these are module
532
+ functions on `Y` and you don't construct anything. The server routes a frame
533
+ without holding any presence or document state. The browser clients keep
534
+ presence, and the server relays awareness frames without reading them.
492
535
 
493
536
  ```ruby
494
537
  Y.message_kind(frame) # => 0 drop / 1 step1 / 2 update / 3 awareness / 4 query
@@ -498,55 +541,170 @@ Y.wrap_update(update_bytes) # => wrap a raw doc update as a sync Update frame
498
541
 
499
542
  ### ActionCable Integration
500
543
 
501
- In a Rails app, one generator creates the channel and the migration:
544
+ In a Rails app, one generator creates the storage migration. The models and
545
+ `Y::DocumentChannel` come with the gem:
502
546
 
503
547
  ```bash
504
548
  bin/rails generate yrby:install
505
549
  bin/rails db:migrate
506
550
  ```
507
551
 
508
- The models ship in the gem, the way Action Text owns
509
- `ActionText::RichText`:
510
-
511
- - **`Y::Document`**: one row per document, addressed two ways: by `key`
512
- (what a channel addresses; one opaque, unique string, sometimes
513
- app-supplied, never parsed) and, optionally, by polymorphic `record` +
514
- `name` (which model attribute it backs; `name` is the attribute name,
515
- `"body"`; one document per attribute per record, the
516
- ActionText::RichText scheme). Key-only documents leave the binding nil.
517
- Either side can arrive first: `Y::Document.for(record, name)` finds or
518
- creates the binding, derives a readable key (`post/1/body`), and adopts
519
- a key-only row already holding that key, so a channel writing first and
520
- a binding created later converge on one document. The row also holds
521
- the merged `state` snapshot, CRDT state only; derived data (rendered
522
- HTML, search text) is the application's job, typically in the channel's
523
- on_change. `.load_state(key)` / `.append(key, update)` are the store
524
- calls the generated channel uses.
525
- - **`Y::DocumentUpdate`**: the uncompacted tail, one delta per row,
526
- compacted into `state` and deleted once the tail reaches `compact_every`
527
- (default 64). Loading reads the snapshot plus the current tail; an
528
- empty tail returns `state` directly. Compaction serializes on a
529
- per-document row lock and skips causally-gapped rows; they're
530
- quarantined until they heal rather than compacted into state or
531
- deleted. Destroying a document deletes its updates with it.
532
-
533
- Encrypted storage: `Y::EncryptedDocument` stores `state` and update
534
- payloads through Active Record encryption on the same tables, the way
535
- `ActionText::EncryptedRichText` does. Point the channel's
536
- `on_load`/`on_change` at it instead and configure your app's encryption
537
- keys. Use one access path per document: rows written encrypted read back
538
- as ciphertext through the plain classes.
539
-
540
- The migration creates `y_documents` and `y_document_updates`. To rename
541
- them, edit the generated migration and point `Y::Document.table_name` /
552
+ As with Action Text's `ActionText::RichText`, the models are defined in the
553
+ gem:
554
+
555
+ - **`Y::Document`** stores one row per document, and you can find a row in two
556
+ ways. Channels use `key`, a single opaque, unique string that the app
557
+ sometimes supplies and that nothing parses. Optionally, a row also has a
558
+ polymorphic `record` and a `name` that say which model attribute the document
559
+ backs, where `name` is the attribute name, such as `"body"`. Each record gets
560
+ one document per attribute, the same scheme ActionText::RichText uses, and
561
+ key-only documents leave `record` and `name` nil. Either side can come first.
562
+ `Y::Document.for(record, name)` finds or creates the record binding, derives
563
+ a readable key (`post/1/body`), and adopts any key-only row that already has
564
+ that key, so a channel that writes first and a binding created later end up
565
+ on the same document. The row also holds the merged `state` snapshot, which
566
+ contains only CRDT state. Derived data such as rendered HTML or search text
567
+ is up to your application, and the usual place for it is the channel's
568
+ `on_change`. By default the channel concern calls `.load_state(key)` and
569
+ `.append(key, update)` to read and write the store.
570
+ - **`Y::DocumentUpdate`** holds the uncompacted tail, one delta per row. When
571
+ the tail reaches `compact_every` (default 64), yrby compacts it into `state`
572
+ and deletes those rows. A load reads the snapshot plus the current tail, and
573
+ returns `state` directly when the tail is empty. Compactions of one document
574
+ run one at a time under a per-document row lock, and they skip rows with an
575
+ open causal gap, leaving them as they are until the gap closes. Destroying a
576
+ document also deletes its updates.
577
+
578
+ For encrypted storage, `Y::EncryptedDocument` writes `state` and update
579
+ payloads through Active Record encryption on the same tables, as
580
+ `ActionText::EncryptedRichText` does. You declare it on the model, and the
581
+ model decides whether an attribute is encrypted, so a page or client can't
582
+ change it:
583
+
584
+ ```ruby
585
+ class Post < ApplicationRecord
586
+ has_collaborative_document :body, encrypted: true
587
+ end
588
+ ```
589
+
590
+ `Y::DocumentChannel` reads that declaration and routes every load and append
591
+ for the attribute through the encrypted class, while attributes without it use
592
+ plain `Y::Document`. In your own channel, point `on_load` and `on_change` at
593
+ `Y::EncryptedDocument`. In both cases you need to configure your app's
594
+ encryption keys and use one access path per document, because the plain classes
595
+ read encrypted rows back as ciphertext.
596
+
597
+ `post.collaborative_document(:body)` returns a bound
598
+ `Y::Collaborative::Attribute` with `load_state`, `append(update)`, `key`, and
599
+ `y_doc`. `y_doc` builds a fresh native `Y::Doc` you can read and render in
600
+ Ruby. For built-in row operations like compaction, call
601
+ `post.collaborative_document(:body).document_row.compact!`. The channel and
602
+ your application code go through the same accessor, which handles encryption
603
+ for both.
604
+
605
+ An attribute uses the key its document row was stored under. Before a row
606
+ exists, the key is `Y::Document.key_for(record, name)`, and computing it
607
+ doesn't create a row. Grants keep their existing scope and lifetime.
608
+
609
+ To check the current user's permissions in addition to the signed grant, give
610
+ the shipped channel an `authorize_document` block. It runs in channel context,
611
+ so it can use `current_user` or any other identifiers your Action Cable
612
+ connection provides:
613
+
614
+ ```ruby
615
+ # config/initializers/yrby.rb
616
+ Rails.application.config.to_prepare do
617
+ Y::DocumentChannel.authorize_document do |record, name|
618
+ current_user.present? && record.editable_by?(current_user, attribute: name)
619
+ end
620
+ end
621
+ ```
622
+
623
+ `editable_by?` is a policy method in your application, and yrby doesn't provide
624
+ it. The block receives a freshly loaded record and the attribute name as a
625
+ string. If it returns false or nil, the channel rejects the subscription before
626
+ opening a stream or serving any state. An invalid grant is rejected before the
627
+ block is called. Without a block, a valid grant is enough, so the view that
628
+ renders the helper must still require edit permission. If the block raises, the
629
+ channel rejects the subscription and the error goes to your normal error
630
+ handling.
631
+
632
+ The policy runs once, when the client subscribes, and after that the open
633
+ subscription authorizes each message. That's how Action Cable is meant to work,
634
+ and it means a keystroke or cursor move doesn't cost a record load plus your
635
+ own queries. The tradeoff is that if you revoke a permission mid-session, it
636
+ takes effect the next time that client subscribes. You can shorten that window
637
+ in two ways. Every new subscription checks grant expiry, so a short
638
+ `expires_in:` on the tag limits it, as long as you also give the element a
639
+ `refresh:` URL (see below). If your application has to cut off access
640
+ immediately, stop the subscription yourself when the permission changes.
641
+
642
+ Both transports remember the decision for the life of the subscription. Action
643
+ Cable keeps the same channel instance around. AnyCable builds a new one for
644
+ each command, so yrby declares the authorized document as channel state and
645
+ sends it with each RPC, and because anycable-go holds that state, a client
646
+ can't forge it. A frame that arrives without an authorized subscription is
647
+ rejected even if its grant is valid.
648
+
649
+ Every channel does its authorization in one method, `authorized?`, which the
650
+ concern calls when a client subscribes, before it opens a stream or serves any
651
+ state. In a channel you write, you define `authorized?` yourself and it
652
+ receives the document key. The shipped `Y::DocumentChannel` defines it to run
653
+ the `authorize_document` block with the record and the attribute name, or to
654
+ accept any valid grant when there's no block. A subclass can override
655
+ `authorized?` directly and read the located record from `record`.
656
+
657
+ ### Grant lifetime and refresh
658
+
659
+ A grant lasts as long as GlobalID's signed-id default, which is one month under
660
+ Rails, and `expires_in:` on the tag shortens it. The grant is part of the
661
+ rendered page, though, and Action Cable resubscribes with it after every
662
+ network drop. If the grant expires before the user is done editing, the editor
663
+ gets blocked at the next reconnect. To avoid that, pair `expires_in:` with
664
+ `refresh:`, a URL the element fetches when a subscription is rejected:
665
+
666
+ ```erb
667
+ <%= collaborative_document_tag @post, :body, expires_in: 10.minutes,
668
+ refresh: grant_post_path(@post) %>
669
+ ```
670
+
671
+ ```ruby
672
+ # config/routes.rb: resources :posts do get :grant, on: :member end
673
+ # app/controllers/posts_controller.rb
674
+ def grant
675
+ @post = current_user.posts.find(params[:id]) # your own authorization, again
676
+ render json: { grant: @post.collaborative_sgid(:body, expires_in: 10.minutes) }
677
+ end
678
+ ```
679
+
680
+ That action grants write access, so its check must be at least as strict as the
681
+ page that renders the tag. If it skips authorization, anyone who can reach the
682
+ URL gets a grant, and a short `expires_in:` protects nothing.
683
+
684
+ When a subscription is rejected, the element fetches that URL with the session
685
+ cookie, and the action runs your authorization again. If the response is
686
+ `{ "grant": ... }`, the same session resubscribes with the new grant and keeps
687
+ its document and pending edits. Any other response, a non-2xx status, or a second
688
+ rejection blocks the session as before, and so does a refresh that takes
689
+ longer than 15 seconds. The element doesn't renew grants on a timer, so it
690
+ won't interrupt a healthy open subscription. Every reconnect after expiry is a
691
+ fresh permission check, which is why you'd want a short lifetime in the first
692
+ place.
693
+
694
+ For room-keyed collaboration or other custom channel behavior, generate a
695
+ channel with `bin/rails generate yrby:install --channel` and implement its
696
+ `authorized?(key)`. Custom channels can still use both storage hooks.
697
+
698
+ The migration creates `y_documents` and `y_document_updates`. To rename them,
699
+ edit the generated migration and point `Y::Document.table_name` and
542
700
  `Y::DocumentUpdate.table_name` at the new names in an initializer.
543
701
 
544
- Storage is swappable: the channel only needs `on_load` and `on_change`
702
+ You can swap the storage. The channel only needs `on_load` and `on_change`
545
703
  answered, and they can point at anything.
546
704
 
547
705
  `include Y::ActionCable` (from the `yrby-rails` gem) is the channel
548
- integration: the y-websocket protocol (document sync +
549
- awareness/presence) over ActionCable.
706
+ integration. It implements the y-websocket protocol over ActionCable, covering
707
+ document sync, awareness, and presence.
550
708
 
551
709
  ```ruby
552
710
  # app/channels/document_channel.rb
@@ -557,8 +715,6 @@ class DocumentChannel < ApplicationCable::Channel
557
715
  on_change { |key, update| Y::Document.append(key, update) } # record, then broadcast
558
716
 
559
717
  def subscribed
560
- return reject unless authorized?(params[:id])
561
-
562
718
  sync_subscribed params[:id]
563
719
  end
564
720
 
@@ -568,80 +724,112 @@ class DocumentChannel < ApplicationCable::Channel
568
724
 
569
725
  private
570
726
 
571
- # Everyone is denied until you wire this to your app's auth.
727
+ # This denies everyone until you wire it to your app's auth.
728
+ # sync_subscribed rejects the subscription unless this returns true.
572
729
  def authorized?(_document_key) = false
573
730
  end
574
731
  ```
575
732
 
576
- The concern is store-backed. A handshake is answered from `on_load`; document
577
- changes are recorded through `on_change`, then broadcast. Nothing
578
- authoritative is kept in ActionCable process memory, so
579
- AnyCable RPC workers, Puma workers, and separate dynos can all handle messages
580
- for the same document as long as they share the same store and cable adapter.
733
+ For documents that belong to a record, `Y::Collaborative` (which the engine
734
+ includes into Active Record) provides the token flow `authorized?` needs. The
735
+ page creates a signed GlobalID scoped to one attribute, and the channel looks
736
+ up the record from it, so the page decides which document the client gets.
737
+
738
+ ```erb
739
+ <%# the view picks the document and signs it %>
740
+ <%= tag.div data: { grant: post.collaborative_sgid(:body) } %>
741
+ ```
742
+
743
+ ```ruby
744
+ # The channel looks up the record from the token. This isn't memoized
745
+ # because AnyCable builds a fresh channel for each command, and a cached
746
+ # record would go stale.
747
+ def authorized?(_key) = record.present? && record.editable_by?(current_user)
748
+ def record = Y::Collaborative.locate(params[:grant], :body)
749
+ ```
581
750
 
582
- `on_load` and `on_change` are required. If either is missing, the channel fails
583
- before it can acknowledge or broadcast edits. Presence is ephemeral:
584
- awareness frames are relayed, and `yrby-client` sends a best-effort
585
- presence-removal frame on disconnect/pagehide, with the client-side awareness
586
- timeout as the fallback for abrupt disconnects.
751
+ A token created for `:body` only verifies under `:body`'s purpose
752
+ (`"yrby/body"`), and a tampered, expired, or wrong-attribute token returns no
753
+ record.
754
+
755
+ The concern reads and writes through your store. It answers handshakes from
756
+ `on_load`, and it passes document changes through `on_change` before
757
+ broadcasting them. The ActionCable process doesn't keep authoritative state in
758
+ memory, so AnyCable RPC workers, Puma workers, and separate dynos can all
759
+ handle messages for the same document, as long as they share a store and a
760
+ cable adapter.
761
+
762
+ If the yrby-rails models are installed and you declare neither hook, `on_load`
763
+ and `on_change` both use `Y::Document` storage. To use a different store,
764
+ declare both hooks so reads and writes go to the same place. Once a pair is
765
+ configured explicitly, a subclass may override either hook. Outside a
766
+ yrby-rails app there's no default, and until you declare both hooks, the
767
+ channel fails before it can acknowledge or broadcast an edit.
768
+
769
+ Presence is ephemeral. The server relays awareness frames, and `yrby-client`
770
+ sends a best-effort presence-removal frame on disconnect and on `pagehide`. If
771
+ a client can't send that frame, the client-side awareness timeout removes its
772
+ presence.
587
773
 
588
774
  Incoming frames are validated as a single well-formed protocol message before
589
775
  anything processes or relays them. Malformed, truncated, multi-message,
590
- oversized, or unknown frames are dropped. A bad frame can't crash the process: a
591
- Rust panic is caught at the FFI boundary and re-raised as a Ruby exception. And
592
- no single client can relay garbage that breaks the others in a room.
776
+ oversized, and unknown frames are dropped, so a bad frame can't crash the
777
+ process and one client can't relay garbage that breaks everyone else in the
778
+ room. A Rust panic is caught at the FFI boundary and re-raised as a Ruby
779
+ exception.
593
780
 
594
781
  #### Delivery guarantees
595
782
 
596
- The contract is the same at every scale: one process, or hundreds across many
597
- servers:
783
+ These guarantees are the same whether you run one process or hundreds of them
784
+ across many servers:
598
785
 
599
786
  - **The document always converges.** CRDT updates are commutative and
600
- idempotent, so out-of-order, duplicate, or concurrent delivery all converge to
601
- the same correct document. This needs no coordination and holds everywhere.
602
- - **An acked update is durable, even one that arrived out of order.** An
603
- update with a missing dependency is recorded and acked like any other, and
604
- parks as pending in the document. That missing dependency is an update some
605
- client still holds unacked, so that client keeps retransmitting it until
606
- the server records it, and the gap closes. The ack loop is the guarantee.
607
- See [Causal gaps](#causal-gaps).
608
- - **`on_change` is at-least-once, and the durable guarantee is that replaying the
609
- log reconstructs the document.** Every update triggers `on_change` before it's acked or
610
- broadcast (record-before-distribute). If exactly-once updates matter for you, **you
611
- must make `on_change` idempotent**. But remember that the CRDT can handle duplicates.
787
+ idempotent, so out-of-order, duplicate, and concurrent delivery all produce
788
+ the same correct document in every deployment, with no coordination.
789
+ - **An acked update is durable, including one that arrived out of order.** The
790
+ server records and acks an update with a missing dependency like any other,
791
+ and the update waits as pending in the document. Some client still holds the
792
+ missing update unacked and keeps retransmitting it until the server records
793
+ it, and then the gap closes. See [Causal gaps](#causal-gaps).
794
+ - **`on_change` runs at least once, and replaying the log always rebuilds the
795
+ document.** yrby calls `on_change` for every update before acking or
796
+ broadcasting it. If you need exactly-once behavior, make `on_change`
797
+ idempotent. The CRDT handles duplicates either way.
612
798
  - **A raising `on_change` rejects the update implicitly.** If the block raises,
613
- the update is neither acked nor broadcast (record-before-distribute stops both).
614
- There is no negative-ack: the client simply never receives the ack, keeps the
615
- update pending, and retransmits on its timer/reconnect. This is built for
616
- *transient* failures (the store is briefly down → a retry lands). A block that
617
- raises *deterministically* (a validation that always fails for this edit)
618
- will be retried forever, since nothing tells the client to stop. Enforce hard
619
- rejections before the edit reaches `on_change` (channel authorization in
620
- `subscribed`), not by raising inside it.
799
+ the server doesn't ack or broadcast the update, and it doesn't send a
800
+ negative ack either. The client keeps the update pending and retransmits it
801
+ on its timer or on reconnect. That works for transient failures, such as a
802
+ store that was briefly down, where a retry succeeds. A block that raises
803
+ every time (say, a validation that always fails for this edit) gets retried
804
+ forever, because nothing tells the client to stop. Put hard rejections in the
805
+ channel's authorization at subscribe time, before an edit can reach
806
+ `on_change`, and don't raise inside the hook for them.
621
807
  - **An over-cap frame is dropped the same silent way.** A frame larger than
622
- `max_frame_bytes` (default 8 MiB) is dropped before decoding (no ack, no
623
- broadcast) to bound the work a client can force. For a genuine document
624
- update that means the same implicit rejection as above: unacked, retransmitted
625
- forever. Normal typing never approaches the cap, but a large paste, an embedded
626
- image, or a big initial `SyncStep2` can. The drop is logged (`warn` for
627
- over-cap, `debug` for undecodable) with the document key and update id so it's
628
- findable; override `sync_log_context` on the channel to add a user/connection
629
- id. Size the cap for your largest expected payload, and reject
630
- genuinely-too-big content upstream rather than relying on the cap to reject it
631
- gracefully.
808
+ `max_frame_bytes` (default 8 MiB) is dropped before decoding, with no ack and
809
+ no broadcast, which limits how much work a client can force on the server. A
810
+ real document update that hits the cap gets the same implicit rejection as
811
+ above, so it's never acked and is retransmitted forever. Normal typing won't
812
+ come near the cap, but a large paste, an embedded image, or a big initial
813
+ `SyncStep2` can. The server logs each drop (`warn` for over-cap, `debug` for
814
+ undecodable) with the document key and update id so you can find it, and you
815
+ can override `sync_log_context` on the channel to add a user or connection
816
+ id. Size the cap for your largest expected payload, and reject content that's
817
+ too big before it reaches the channel. The cap is a last line of defense, and
818
+ the client never sees an error from it.
632
819
 
633
820
  #### Causal gaps
634
821
 
635
- Yjs updates can arrive out of order: an update can reach the server before
636
- another update it depends on. yrby treats that as normal. The update is
637
- recorded and acked like any other, parks as a pending struct in the document,
638
- and integrates on its own the moment the missing dependency lands. The write
639
- path never rebuilds the document; it appends, relays, and acks, so a gapped
640
- update costs the same as any other.
641
-
642
- Serving is lossless too, like any Yjs server. `handle_sync_message` serves
643
- full state, pending included, so a peer parks the same pending struct and
644
- heals it the same way. Healing needs no special machinery: the missing
822
+ Yjs updates can arrive out of order, so an update can reach the server before
823
+ the update it depends on. yrby treats that as normal. It records and acks the
824
+ update like any other, the update waits as a pending struct in the document,
825
+ and it integrates once the missing dependency arrives. The write path appends,
826
+ relays, and acks without rebuilding the document, so an update with a gap costs
827
+ the same as any other.
828
+
829
+ Serving is lossless too, as on any Yjs server. `handle_sync_message` serves
830
+ full state with pending structs included, so a peer holds the same pending
831
+ struct and integrates it the same way. Closing the gap takes no special
832
+ machinery, because the missing
645
833
  dependency is an update its sender still holds unacked, and at-least-once
646
834
  retransmission delivers it. Only compaction excludes pending
647
835
  (`compacted_state_update`), because folding a log must not freeze an
@@ -650,14 +838,14 @@ un-integrable struct into the base state.
650
838
  The bundled `Y::Document` store handles all of this. If you write your own
651
839
  store, keep two things in mind:
652
840
 
653
- **1. Load losslessly, and tolerate duplicates.** `on_load` should return
654
- state that preserves pending: `encode_state_as_update`, or a replay of the
841
+ **1. Load losslessly, and tolerate duplicates.** `on_load` should return state
842
+ that keeps pending updates, either `encode_state_as_update` or a replay of the
655
843
  raw append log. Don't compact with `compacted_state_update` while
656
- `doc.pending?`; that strips the pending struct and the acked edit inside it.
657
- (`Y::Document` quarantines pending rows for exactly this reason.) A lost ack
658
- also means a client resends an update the store already has. Replay converges
659
- anyway, because CRDT apply is idempotent, so deduping is optional. If log
660
- size matters, dedup by content hash:
844
+ `doc.pending?` is true, because that strips the pending struct and loses the
845
+ acked edit inside it. `Y::Document` keeps pending rows out of compaction for
846
+ this reason. When an ack gets lost, the client resends an update the store
847
+ already has. Replay still converges because CRDT apply is idempotent, so
848
+ deduping is optional. If log size matters, dedup by content hash:
661
849
 
662
850
  ```ruby
663
851
  class DocumentStore
@@ -688,17 +876,18 @@ class DocumentStore
688
876
  end
689
877
  ```
690
878
 
691
- **2. Watch for gaps that never heal.** An open gap is quiet: the edit sits
692
- as pending, invisible in the document, until its dependency arrives. Normally
693
- that resolves itself. The sender retransmits the missing update until it is
694
- acked, and every join or reconnect handshake has the client send everything
695
- the server hasn't integrated, so any client holding the dependency supplies
696
- it just by connecting. The gap worth alerting on is one no live client can
697
- supply, and that is what the `on_gap` hook surfaces. It fires with the
698
- document key whenever a document is loaded to serve state and a gap is still
699
- open. Use it to emit a metric (a pending-document count, or the age of the
700
- oldest open gap) so a stuck gap is visible. Gaps are also logged at `info`,
701
- and errors raised in the hook are swallowed so observability can never break
879
+ **2. Watch for gaps that don't close.** An open gap is easy to miss, because
880
+ the pending edit doesn't show up in the document until its dependency arrives.
881
+ Usually the gap closes by itself. The sender retransmits the missing update
882
+ until it's acked, and on every join or reconnect handshake the client sends
883
+ everything the server hasn't integrated, so any client holding the dependency
884
+ supplies it by connecting. The gap worth alerting on is one that no live client
885
+ can supply, and that's what the `on_gap` hook is for. It fires with the
886
+ document key whenever a document is loaded to serve state while a gap is open.
887
+ Use it to emit a metric, such as a pending-document count or the age of the
888
+ oldest open gap, so you can see a gap that isn't closing. yrby also logs gaps
889
+ at `info`. Errors raised in the hook are swallowed, so a broken metrics call
890
+ can't break
702
891
  frame handling.
703
892
 
704
893
  ```ruby
@@ -711,28 +900,32 @@ end
711
900
 
712
901
  #### Multi-process deployments
713
902
 
714
- Most Rails apps run several processes, and any of them might serve a given document.
715
- Two pieces keep them in step.
903
+ Most Rails apps run several processes, and any of them might end up serving a
904
+ given document. Two things keep them consistent.
716
905
 
717
- Broadcasts cross processes through the Action Cable adapter, so it needs to something
718
- like `redis` or `solid_cable`, not `async`. With that in place, a change
906
+ Broadcasts go between processes through the Action Cable adapter, so use
907
+ `redis`, `solid_cable`, or another cross-process adapter, since `async` only
908
+ works inside one process. With one of those adapters in place, a change
719
909
  on one process reaches clients on all of them.
720
910
 
721
911
  Every process rebuilds document state from the durable store through `on_load`.
722
- Because changes are recorded before broadcast, record-before-distribute holds
723
- across processes: whichever process receives a change records it to the shared
724
- store before anyone, anywhere, sees it.
912
+ Changes are still recorded before they're broadcast, so whichever process
913
+ receives a change writes it to the shared store before any client on any
914
+ process sees it.
725
915
 
726
- `bun multiprocess.mjs` in the demo runs clients across two processes and checks
727
- convergence, fresh reads on both, presence across processes, and one shared log.
916
+ In the demo, `bun multiprocess.mjs` runs clients across two processes and
917
+ checks that the documents converge, that fresh reads work on both processes,
918
+ that presence reaches clients on the other process, and that both processes
919
+ write to one shared log.
728
920
 
729
921
  ##### AnyCable
730
922
 
731
- `yrby` fully supports AnyCable.
923
+ `yrby` supports AnyCable end to end.
732
924
 
733
- The demo checks this against a real anycable-go + RPC server
734
- (`frontend/anycable_probe.mjs`, `anycable_concurrent.mjs`): liveness, the
735
- yrby client provider, cross-process reads, and concurrent convergence.
925
+ The demo tests it against a real anycable-go server and RPC server in
926
+ `frontend/anycable_probe.mjs` and `anycable_concurrent.mjs`, covering liveness,
927
+ the yrby client provider, cross-process reads, and convergence under concurrent
928
+ editing.
736
929
 
737
930
  ##### Demo
738
931
 
@@ -741,8 +934,8 @@ app using the yrby provider, with end-to-end tests.
741
934
 
742
935
  #### Record Before Distribute
743
936
 
744
- Every document change is handed to the `on_change` handler before broadcasting.
745
- It is up to you to durably record it:
937
+ Every document change goes through your `on_change` handler before it's
938
+ broadcast, and the handler is where you record it durably:
746
939
 
747
940
  ```ruby
748
941
  class DocumentChannel < ApplicationCable::Channel
@@ -759,21 +952,21 @@ class DocumentChannel < ApplicationCable::Channel
759
952
  end
760
953
  ```
761
954
 
762
- If the recorder raises (say the store is down), the change is rejected: not
763
- applied, not sent to anyone. The cost is a synchronous durable write on the path
764
- of every change. There's no in-gem per-document lock; concurrent writes to one
765
- document can both record (at-least-once), and since CRDT apply is idempotent a
766
- duplicate record replays to the same document.
955
+ If the handler raises (say the store is down), the change is rejected, so it
956
+ isn't applied or sent to anyone. The cost is a synchronous durable write on the
957
+ path of every change. The gem doesn't take a per-document lock, so two
958
+ concurrent writes to one document can both record (at least once). Because CRDT
959
+ apply is idempotent, a duplicate record replays to the same document.
767
960
 
768
- The demo wires `on_change` to a durable Postgres-backed log by default, and checks
769
- end to end that the log alone rebuilds the document.
961
+ The demo wires `on_change` to a durable Postgres-backed log by default, and
962
+ checks end to end that the log alone rebuilds the document.
770
963
 
771
964
  #### Ephemeral documents (no database)
772
965
 
773
- `on_load` and `on_change` are plain blocks, and nothing requires them to touch
774
- a database. For documents that don't need to outlive their session (a
775
- scratchpad, live form state, a draft you only persist on submit) the store
776
- can be connection state that travels with each request:
966
+ `on_load` and `on_change` are plain blocks, and they don't have to touch a
967
+ database. Some documents only need to last for a session, such as a scratchpad,
968
+ live form state, or a draft you save on submit. For those, the store can be
969
+ connection state that's sent with each request:
777
970
 
778
971
  ```ruby
779
972
  class ScratchpadChannel < ApplicationCable::Channel
@@ -790,6 +983,11 @@ class ScratchpadChannel < ApplicationCable::Channel
790
983
 
791
984
  def subscribed = sync_subscribed(params[:id])
792
985
  def receive(data) = sync_receive(data, params[:id])
986
+
987
+ private
988
+
989
+ # Each subscriber gets its own scratchpad on this connection.
990
+ def authorized?(_key) = true
793
991
  end
794
992
  ```
795
993
 
@@ -815,65 +1013,69 @@ class ScratchpadChannel < ApplicationCable::Channel
815
1013
 
816
1014
  def subscribed = sync_subscribed(params[:id])
817
1015
  def receive(data) = sync_receive(data, params[:id])
1016
+
1017
+ private
1018
+
1019
+ def authorized?(_key) = true # per-connection scratchpad, see above
818
1020
  end
819
1021
  ```
820
1022
 
821
- Both hooks run in the channel instance (`instance_exec`), so they can use
822
- anything the channel can, and `sync_receive` rebuilds the document from
823
- `on_load` on every update, which is what lets the store live on the
824
- connection. On Action Cable the channel instance lasts as long as the
825
- connection, so an instance variable is the whole store. Merging into
826
- `compacted_state_update` keeps it one blob instead of a growing update log.
827
-
828
- The store is per connection, which shapes what this fits. A single writer gets
829
- the full delivery contract with no database anywhere. With several people
830
- editing at once, one client's update can depend on edits its own connection
831
- has never seen; that update records as pending, and the next handshake with
832
- that client (which always holds the full document) supplies the missing state
833
- and heals it. The document still converges; heavy concurrent editing just
834
- parks more pending between handshakes than a shared store would. On
835
- AnyCable, keep the payload in mind too: the blob travels with every message,
836
- so that variant suits small documents, not long manuscripts.
1023
+ Both hooks run in the channel instance through `instance_exec`, so they can use
1024
+ anything the channel can. `sync_receive` rebuilds the document from `on_load`
1025
+ on every update, which is why the store can sit on the connection. On Action
1026
+ Cable the channel instance lasts as long as the connection, so an instance
1027
+ variable is all the store you need. Merging with `compacted_state_update` keeps
1028
+ the store at one blob so it doesn't grow into an update log.
1029
+
1030
+ Because the store is per connection, this pattern has limits. A single writer
1031
+ gets the full delivery contract without any database. When several people edit
1032
+ at the same time, one client's update can depend on edits its own connection
1033
+ has never seen. That update is recorded as pending, and the next handshake with
1034
+ that client (which always has the full document) supplies the missing state and
1035
+ closes the gap. The document still converges, but heavy concurrent editing
1036
+ leaves more pending between handshakes than a shared store would. On AnyCable,
1037
+ watch the payload size as well. The blob is sent with every message, so this
1038
+ only makes sense for small documents.
837
1039
 
838
1040
  Durability is the connection plus the browsers. A reconnecting client re-seeds
839
- an empty server through the ordinary sync handshake, so the document survives
840
- server restarts as long as some client still has it. For ephemeral documents
841
- shared across clients on a single-process deployment, the same two hooks over
842
- a class-level `Concurrent::Map` work instead; that version stops being
843
- coherent the moment you scale past one process.
1041
+ an empty server through the ordinary sync handshake, so the document survives a
1042
+ server restart as long as some client still has it. If an ephemeral document
1043
+ needs to be shared across clients on a single-process deployment, point the
1044
+ same two hooks at a class-level `Concurrent::Map`. That version is no longer
1045
+ coherent once you run more than one process.
844
1046
 
845
1047
  #### Reliable delivery (acks)
846
1048
 
847
1049
  yrby document delivery is ack-tracked. Browser document updates carry an
848
- `"id"`, and the server replies `{ "ack": <id> }` once `on_change` has
849
- successfully fired. Every decodable document update is recorded and acked,
850
- including one that arrives out of order.
1050
+ `"id"`, and the server replies `{ "ack": <id> }` after `on_change` succeeds.
1051
+ The server records and acks every decodable document update, including one that
1052
+ arrives out of order.
851
1053
 
852
1054
  ```
853
1055
  client -> server { "update": "<base64 update>", "id": 42 }
854
1056
  server -> client { "ack": 42 } # update accepted; safe to forget
855
1057
  ```
856
1058
 
857
- `yrby-client`'s `ActionCableProvider` handles this automatically. It keeps
858
- the unacknowledged local document tail in a queue and sends the merged tail as a
859
- single causally-complete delta. The id is the highest sequence in the batch, so
860
- one `{ ack: id }` cumulatively confirms everything up to it. Because CRDT apply
861
- is idempotent, a resend that already landed is a harmless no-op that just
862
- re-acks. Awareness stays ephemeral and is not acked.
1059
+ `yrby-client`'s `ActionCableProvider` handles this for you. It queues the
1060
+ unacknowledged tail of local document updates and sends it merged into one
1061
+ causally complete delta, using the highest sequence in the batch as the id, so
1062
+ one `{ ack: id }` confirms everything up to it. Because CRDT apply is
1063
+ idempotent, resending an update the server already has does no harm, and the
1064
+ server acks it again. Awareness is ephemeral and isn't acked.
863
1065
 
864
- Presence (cursors, selections) is owned by the browser clients; the server
865
- never sets or holds presence state, it only relays awareness frames opaquely.
866
- See `yrby-client` for the client-side awareness API.
1066
+ The browser clients manage presence (cursors, selections). The server doesn't
1067
+ set or hold presence state, and it relays awareness frames without reading
1068
+ them. See `yrby-client` for the client-side awareness API.
867
1069
 
868
1070
  ## Thread Safety
869
1071
 
870
- A `Doc` is safe to share across Ruby threads, used concurrently from Puma
871
- workers, ActionCable connection threads, or background jobs without external
872
- locking.
1072
+ You can share a `Doc` across Ruby threads. Puma workers, ActionCable connection
1073
+ threads, and background jobs can all use the same one concurrently, with no
1074
+ locking in your code.
873
1075
 
874
1076
  `test/thread_safety_test.rb` runs shared docs, the full sync handshake, and
875
- fan-in sync across 8 threads at once, and checks the interleaving doesn't change
876
- convergence.
1077
+ fan-in sync across 8 concurrent threads and checks that the interleaving
1078
+ doesn't change convergence.
877
1079
 
878
1080
  ### Parallelism (GVL release)
879
1081
 
@@ -881,21 +1083,21 @@ Every method that does real CRDT work (applying updates, encoding state,
881
1083
  handling sync messages) releases Ruby's Global VM Lock
882
1084
  (`rb_thread_call_without_gvl`) while the native code runs. That buys two things.
883
1085
 
884
- CRDT work runs in parallel across Ruby threads on MRI, not just
885
- JRuby/TruffleRuby. `bench/parallelism_bench.rb` measures over 2x wall-clock
886
- speedup applying a ~900 KB update concurrently; native code that held the GVL
887
- couldn't beat serial time.
888
-
889
- A slow operation also can't stall the VM. A thread applying a large update holds
890
- the doc's write lock without holding the GVL, so other Ruby threads keep running
891
- instead of queuing behind it.
892
-
893
- Each method has the same shape: copy Ruby byte strings first, drop the GVL, do
894
- the yrs work while taking and releasing native locks entirely inside the
895
- closure, take the GVL back, then build Ruby objects. No Ruby API is touched
896
- without the GVL, and no native lock is held while reacquiring it, so the lock
897
- order can't deadlock. Panics in native code are caught and re-raised as Ruby
898
- exceptions.
1086
+ First, CRDT work runs in parallel across Ruby threads on MRI, with no need for
1087
+ JRuby or TruffleRuby. `bench/parallelism_bench.rb` measures more than a 2x
1088
+ wall-clock speedup when applying a roughly 900 KB update concurrently. Native
1089
+ code that held the GVL couldn't beat serial time.
1090
+
1091
+ Second, a slow operation can't stall the VM. A thread applying a large update
1092
+ holds the doc's write lock but not the GVL, so other Ruby threads keep running
1093
+ while it works.
1094
+
1095
+ Each of those methods copies the Ruby byte strings, releases the GVL, does the
1096
+ yrs work (taking and releasing the native locks inside that closure), takes the
1097
+ GVL back, and then builds the Ruby objects. Ruby APIs are only called while
1098
+ holding the GVL, and no native lock is held while reacquiring it, so the lock
1099
+ order can't deadlock. A panic in native code is caught and re-raised as a Ruby
1100
+ exception.
899
1101
 
900
1102
  ## Message Type Constants
901
1103