yrby 0.7.1-aarch64-linux → 0.8.1-aarch64-linux

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.md CHANGED
@@ -2,60 +2,87 @@
2
2
 
3
3
  [![CI](https://github.com/jpcamara/yrby/actions/workflows/ci.yml/badge.svg)](https://github.com/jpcamara/yrby/actions/workflows/ci.yml)
4
4
 
5
- yrby (pronounced "yer-bee") makes Rails a real Yjs backend. It binds
5
+ yrby (pronounced "yer-bee") makes Rails a Yjs backend. It binds
6
6
  [y-crdt](https://github.com/y-crdt/y-crdt), the Rust engine behind Y.js, into
7
- Ruby, and builds the rest of the stack around it: a sync server for Action
7
+ Ruby, and on top of that it builds a sync server for Action
8
8
  Cable and AnyCable, a browser provider, and server-side reading and rendering
9
- of the documents. Real-time collaboration in a Rails app with no Node process
10
- anywhere in the path.
9
+ of the documents. You get real-time collaboration in a Rails app with no Node
10
+ process to run.
11
11
 
12
12
  ![Two people typing on separate lines of the same document, each keystroke synced through a Rails server, seen from a third browser with labeled carets](docs/images/collab.gif)
13
13
 
14
- On the server, `yrby-rails` implements the full y-websocket protocol
15
- (document sync plus presence) as a channel concern. Its delivery contract is
16
- stricter than the usual Yjs servers: every update is ack-tracked and durably
17
- recorded before it is acknowledged or broadcast to anyone. Replaying your
18
- store always rebuilds the document, across any number of processes.
19
- ([Delivery guarantees](#delivery-guarantees))
14
+ On the server, `yrby-rails` implements the y-websocket protocol (document sync
15
+ and presence) as a channel concern. The server writes every update to your
16
+ store before it acknowledges the update or sends it to other clients, which
17
+ most Yjs servers don't do. Replaying the store rebuilds the document, however
18
+ many processes you run. ([Delivery guarantees](#delivery-guarantees))
20
19
 
21
20
  In the browser, `yrby-client`'s `ActionCableProvider` connects anything that
22
21
  speaks Yjs. The demo app runs four rich text editors, and CI drives each one
23
22
  in real Chrome: Tiptap, [Lexxy](https://www.npmjs.com/package/lexxy-realtime),
24
23
  Rhino Editor, and CodeMirror. The same channel also syncs Yjs shapes with no
25
24
  editor at all: a whiteboard on a `Y.Map`, a kanban board on a `Y.Array`, a
26
- co-filled form. ([Editors](#editors))
25
+ form filled in together. ([Editors](#editors))
27
26
 
28
27
  In Ruby, the documents are readable without a browser. `Doc#read_text` and
29
- `Doc#read_map` reconstruct contents for search, validation, and exports.
30
- `Y::Tiptap` and `Y::Lexxy` render a document to HTML byte-identical to the
31
- editor's own serializer, take rules for your app's custom nodes, and drop
32
- straight into ActionText. ([Rendering to HTML](#rendering-to-html))
28
+ `Doc#read_map` rebuild the contents for search, validation, and exports.
29
+ `Y::Tiptap` and `Y::Lexxy` render a document to HTML that matches the editor's
30
+ own serializer byte for byte. You can add rules for your app's custom nodes,
31
+ and you can save the output directly to ActionText.
32
+ ([Rendering to HTML](#rendering-to-html))
33
33
 
34
34
  Underneath, the core is built for a production Rails deployment. A `Doc` is
35
35
  thread-safe across Puma and ActionCable threads. Native CRDT work runs with
36
- the GVL released, so it parallelizes on MRI. Incoming frames are validated
36
+ the GVL released, so it runs in parallel on MRI. Incoming frames are validated
37
37
  before anything processes them, and multi-process and AnyCable setups are
38
38
  tested end to end. ([Thread Safety](#thread-safety))
39
39
 
40
- The whole server side of a collaborative document is one channel:
40
+ You add one view helper, on a page where the user can already edit the record:
41
41
 
42
- ```ruby
43
- class DocumentChannel < ApplicationCable::Channel
44
- include Y::ActionCable
42
+ ```erb
43
+ <%= collaborative_document_tag @post, :body %>
44
+ ```
45
45
 
46
- on_load { |key| Y::Document.load_state(key) }
47
- on_change { |key, update| Y::Document.append(key, update) }
46
+ The helper renders a signed grant for that record and attribute, much as
47
+ `turbo_stream_from` signs its stream names. The client subscribes with the
48
+ grant and the attribute name. The gem's `Y::DocumentChannel` verifies the
49
+ grant, then saves every change as `Y::Document` rows before acknowledging it,
50
+ so you don't have to write a channel yourself.
48
51
 
49
- def subscribed = sync_subscribed(params[:id])
50
- def receive(data) = sync_receive(data, params[:id])
51
- end
52
+ The tag renders an element that connects by itself once you import it. When the
53
+ document has synced, your listener receives it and passes it to your editor
54
+ binding:
55
+
56
+ ```js
57
+ import "yrby-client/element"
58
+
59
+ document.addEventListener("yrby:synced", ({ target, detail }) => {
60
+ const editor = bindYourEditor(target, detail.doc, detail.provider)
61
+ detail.signal.addEventListener("abort", () => editor.destroy(), { once: true })
62
+ })
63
+ ```
64
+
65
+ If the user leaves the page with edits still unsent, the session keeps them and
66
+ finishes delivering. Your binding cleans up when the abort signal fires, and if
67
+ nothing is pending, the next visit loads the document from Rails. The [client
68
+ lifecycle and recovery contract](packages/client/README.md#document-sessions)
69
+ has the details.
70
+
71
+ The document is stored as rows in your database, and you can read it back in
72
+ Ruby:
73
+
74
+ ```ruby
75
+ doc = post.collaborative_document(:body).y_doc
76
+ doc.read_text("content") # or Y::Lexxy.new(doc).to_html for rich text
52
77
  ```
53
78
 
54
79
  Install the gem and the npm package:
55
80
 
56
81
  ```
57
82
  gem install yrby-rails # depends on yrby
58
- npm install yrby-client
83
+ npm install yrby-client yjs y-protocols @rails/actioncable
84
+
85
+ bin/rails generate yrby:install && bin/rails db:migrate
59
86
  ```
60
87
 
61
88
  ## Contents
@@ -84,51 +111,54 @@ npm install yrby-client
84
111
 
85
112
  ## Scope
86
113
 
87
- `yrby` binds just the part of `y-crdt` you need to *sync and persist* collaborative
88
- documents - a `Doc`, awareness, and the y-websocket protocol primitives. By default
89
- the Ruby side treats a document as opaque CRDT state: it applies updates, answers
90
- sync handshakes, and records deltas without reaching into the contents - the browser
91
- editor owns the document's shape. When you do need to look inside, `Doc#read_text`
92
- and `Doc#read_map` reconstruct it server-side, in Ruby.
114
+ `yrby` binds the parts of `y-crdt` you need to sync and persist collaborative
115
+ documents, which are a `Doc`, awareness, and the y-websocket protocol
116
+ primitives. By default the Ruby side treats a document as opaque CRDT state. It
117
+ applies updates, answers sync handshakes, and records deltas without reading
118
+ the contents, and the browser editor decides what shape the document has. When
119
+ you do need to look inside, `Doc#read_text` and `Doc#read_map` rebuild it in
120
+ Ruby.
93
121
 
94
122
  ## Durability and delivery
95
123
 
96
- The surface is intentionally small, but the focus is durability, resiliency, delivery
124
+ The API is small, and most of the work went into durability, resiliency, delivery
97
125
  guarantees, correctness, and thread safety.
98
126
 
99
- Towards that goal, `yrby` adds opinionated defaults on top of normal Yjs syncing:
100
-
101
- - Built-in update acknowledgement: the `ActionCableProvider` in `yrby-client` keeps
102
- sending an update until the server acks it, and [`yrby-rails`](https://rubygems.org/gems/yrby-rails)
103
- only acks once the update is durably recorded. That gives you at-least-once
104
- delivery, and because CRDT updates are idempotent a duplicate is a no-op.
105
- - Gap awareness: an update can arrive before another update it depends on (a
106
- "causal gap"). `yrby` records and acks it like any other, and the document
107
- heals on its own once the missing update arrives; its sender keeps
108
- retransmitting it until it is acked. `Doc#pending?` and the `on_gap` hook
127
+ `yrby` adds two opinionated defaults on top of normal Yjs syncing:
128
+
129
+ - yrby acknowledges updates. The `ActionCableProvider` in `yrby-client` keeps
130
+ resending an update until the server acks it, and
131
+ [`yrby-rails`](https://rubygems.org/gems/yrby-rails) only acks an update
132
+ after it's durably recorded. Together that gives you at-least-once delivery,
133
+ and because CRDT updates are idempotent, a duplicate does nothing.
134
+ - yrby expects causal gaps, where an update arrives before another update it
135
+ depends on, and it records and acks that update like any other. The sender of
136
+ the missing update keeps retransmitting it until it's acked, and the gap
137
+ closes when it arrives. `Doc#pending?` and the `on_gap` hook
109
138
  tell you when a document is waiting on a missing update.
110
139
  ([Causal gaps](#causal-gaps))
111
140
 
112
141
  ## What about [yrb](https://github.com/y-crdt/yrb)?
113
142
 
114
- `yrb` has a much larger interface that gives you most of the Yjs type system -
115
- shared text, arrays, maps, XML - to build and query documents in Ruby. It was a great
116
- inspiration for my use of Yjs in Ruby/Rails, and I originally considered building
117
- on top of it. There are a few reasons I went with `yrby` instead:
143
+ `yrb` has a much larger interface, with most of the Yjs type system (shared
144
+ text, arrays, maps, XML) for building and querying documents in Ruby. It was a
145
+ big inspiration for me in using Yjs from Ruby and Rails, and I considered
146
+ building on top of it. I went with `yrby` for a few reasons:
118
147
 
119
- - `yrb` is largely unmaintained. It was built as an experiment for GitLab, and the original
120
- author mostly moved onto other projects.
121
- - [It isn't thread-safe](https://github.com/y-crdt/yrb/issues/72). It segfaults in a threaded
122
- environment (such as ActionCable...)
123
- - It's a much larger set of features to maintain, which most people don't need. The vast
124
- majority of people manipulate Y.js documents in the browser, not from a server-side language.
148
+ - `yrb` started as an experiment for GitLab, and its original author has mostly
149
+ moved on to other projects, so it's largely unmaintained.
150
+ - [It isn't thread-safe](https://github.com/y-crdt/yrb/issues/72), and it
151
+ segfaults in threaded environments like ActionCable.
152
+ - It's a much larger set of features to maintain, and most people don't need
153
+ them. The vast majority of Y.js documents are only ever manipulated in the
154
+ browser.
125
155
 
126
156
  ## Testing
127
157
 
128
158
  Ruby and Rust unit tests cover the core. CI also runs the npm client tests and a
129
159
  Rails demo smoke slice against the real ActionCable stack. The demo includes
130
160
  heavier local suites for hostile input, crash recovery, multi-browser editing,
131
- AnyCable, and load testing. The benchmark note below is from a single laptop.
161
+ AnyCable, and load testing. The benchmark number below is from a single laptop.
132
162
  Issues and PRs are welcome.
133
163
 
134
164
  ## Install
@@ -142,10 +172,11 @@ gem "yrby"
142
172
  gem "yrby-rails"
143
173
  ```
144
174
 
145
- Requires Ruby 3.4 or newer. The release workflow builds precompiled gems for
146
- Ruby 3.4 and 4.0 across the supported Ruby platforms, with native smoke tests
147
- on Linux x86_64 and macOS arm64. Installing from a matching platform gem needs
148
- no Rust; a source build needs [Rust](https://rustup.rs).
175
+ yrby needs Ruby 3.4 or newer. Releases include precompiled gems for Ruby 3.4
176
+ and 4.0 on the supported Ruby platforms, and the release workflow smoke-tests
177
+ the native builds on Linux x86_64 and macOS arm64. You only need
178
+ [Rust](https://rustup.rs) when no precompiled gem matches your platform and the
179
+ gem builds from source.
149
180
 
150
181
  To work on the gem itself:
151
182
 
@@ -163,16 +194,17 @@ The rest of the dev setup, plus the demo, is in [CONTRIBUTING.md](CONTRIBUTING.m
163
194
  - The ActionCable concern and a quickstart are [below](#actioncable-integration).
164
195
  - [`examples/actioncable-demo`](examples/actioncable-demo): a runnable Rails +
165
196
  Tiptap app with collaborative cursors, the AnyCable setup, a Postgres store,
166
- and the test/load suites.
197
+ and the test and load suites.
167
198
  - [CHANGELOG.md](CHANGELOG.md) and [CONTRIBUTING.md](CONTRIBUTING.md).
168
199
 
169
200
  ## Editors
170
201
 
171
- yrby syncs opaque Yjs updates, so it works with any editor that has a Yjs
172
- binding. The demo app runs four, and CI drives each one in real Chrome:
202
+ yrby syncs opaque Yjs updates, so any editor with a Yjs binding works. The demo
203
+ app runs four of them, and CI drives each one in real Chrome to check
173
204
  concurrent typing with every keystroke accounted for, remote cursors,
174
- local-only undo, and byte parity between the server-side renderers and the
175
- editor's own serializer. Each page is a working integration to copy from:
205
+ local-only undo, and that the server-side renderers produce the same HTML as
206
+ the editor's own serializer. Each demo page is a working integration you can
207
+ copy from:
176
208
 
177
209
  | Editor | Yjs binding | Demo code |
178
210
  |---|---|---|
@@ -181,11 +213,11 @@ editor's own serializer. Each page is a working integration to copy from:
181
213
  | [Rhino Editor](https://github.com/KonnorRogers/rhino-editor) (Tiptap 3) | `@tiptap/extension-collaboration` + `-caret` | [`rhino.js`](examples/actioncable-demo/frontend/src/rhino.js) |
182
214
  | [CodeMirror 6](https://codemirror.net) | `y-codemirror.next` | [`codemirror.js`](examples/actioncable-demo/frontend/src/codemirror.js) |
183
215
 
184
- The demo also syncs plain Yjs shapes with no editor at all (a whiteboard
185
- on a `Y.Map`, a kanban board on a `Y.Array`, a co-filled form) over the
186
- same channel. The demo README's "Using this in your own app" section has
187
- the integration recipe, and its `NoteMaterializer` shows how to render a
188
- document to ActionText server-side with `Y::Tiptap` or `Y::Lexxy`.
216
+ The demo also syncs plain Yjs shapes with no editor (a whiteboard on a
217
+ `Y.Map`, a kanban board on a `Y.Array`, a form filled in together) over the
218
+ same channel. The demo README's "Using this in your own app" section has the
219
+ integration recipe, and its `NoteMaterializer` shows how to render a document
220
+ to ActionText on the server with `Y::Tiptap` or `Y::Lexxy`.
189
221
 
190
222
  ## Usage
191
223
 
@@ -219,43 +251,46 @@ doc.handle_sync_message(data) # => [msg_type, sync_type, response]; answers
219
251
 
220
252
  ### Reading document contents
221
253
 
222
- Reconstruct a document server-side (search, exports, emails, SSR) with no
254
+ Rebuild a document on the server for search, exports, emails, or SSR, with no
223
255
  Node process:
224
256
 
225
257
  ```ruby
226
258
  doc.read_text("prosemirror") # => plain text of a Y.Text root, or nil
227
259
  doc.read_xml("root") # => text of an XML root, one block per line
228
260
  doc.read_map("state") # => a Y.Map root as a JSON string; JSON.parse it
261
+ doc.read_array("cards") # => a Y.Array root as a JSON string; JSON.parse it
229
262
  ```
230
263
 
231
264
  ### Pending structs and gap-free state
232
265
 
233
266
  If a doc applies an update whose causally-prior update is missing (a "gappy"
234
- update), yrs parks it as a **pending** struct: the integrated state vector stays
235
- empty, but the pending block is held as a recovery buffer and heals if the
236
- missing dependency later arrives. `Doc#pending?` reports this.
267
+ update), yrs holds it as a **pending** struct and the integrated state vector
268
+ doesn't move. yrs keeps the pending block as a recovery buffer and integrates
269
+ it if the missing dependency arrives later. `Doc#pending?` tells you when a doc
270
+ is in this state.
237
271
 
238
- Pending structs travel like any other state. `handle_sync_message` answers
239
- `SyncStep1` with the doc's full state, pending included, just like Y.js's
240
- `encodeStateAsUpdate`: a peer parks the pending struct the same way this doc
241
- did and heals it the same way. The one place pending must not go is a
242
- compacted snapshot:
272
+ `handle_sync_message` answers `SyncStep1` with the doc's full state, pending
273
+ structs included, which is what Y.js's `encodeStateAsUpdate` does too. A peer
274
+ that receives it holds the pending struct and integrates it the same way this
275
+ doc would. The one place pending structs must be left out is a compacted
276
+ snapshot:
243
277
 
244
278
  - `Doc#compacted_state_update` returns a gap-free full-state update for
245
279
  compaction. Folding a log into one blob would otherwise freeze an
246
- un-integrable struct into the base state forever. It's non-destructive: the
247
- doc keeps its pending.
248
- - `encode_state_as_update` stays lossless, so persistence and serving keep
249
- the raw pending bytes and the gap can still heal.
280
+ un-integrable struct into the base state permanently. The call doesn't modify
281
+ the doc, which keeps its pending structs.
282
+ - `encode_state_as_update` is lossless, so persistence and serving keep the
283
+ raw pending bytes and the gap can still close.
250
284
 
251
285
  ### Rendering to HTML
252
286
 
253
- Schema-pinned renderers turn a collaborative document into HTML on the
254
- server, with no Node process or headless editor. Each is an editor-specific
255
- class (byte-for-byte with that editor's own serializer) built on a core base
256
- any other editor extends with rules: `Y::Tiptap` on `Y::ProseMirror` for
257
- ProseMirror documents, and `Y::Lexxy` (the
258
- [Lexxy](https://github.com/basecamp/lexxy) editor) on `Y::Lexical`. Each
287
+ The renderers turn a collaborative document into HTML on the server with no
288
+ Node process or headless editor. Each renderer is a class for one editor, and
289
+ its output matches that editor's own serializer. `Y::Tiptap` renders
290
+ ProseMirror documents and is built on `Y::ProseMirror`, and `Y::Lexxy` renders
291
+ documents from the [Lexxy](https://github.com/basecamp/lexxy) editor and is
292
+ built on `Y::Lexical`. To support another editor on one of those engines,
293
+ extend the matching base class with rules. Each renderer
259
294
  returns `nil` for a root that belongs to the other schema.
260
295
 
261
296
  #### `Y::Tiptap` (and `Y::ProseMirror`, its base)
@@ -266,26 +301,27 @@ tiptap.to_html # the "default" fragment (Tiptap's default root)
266
301
  tiptap.to_html("content") # or another XML root
267
302
  ```
268
303
 
269
- The output matches Tiptap's own `getHTML()`, checked byte-for-byte in the tests
270
- against a document captured from a real editor. It follows
271
- [`tiptap-php`](https://github.com/ueberdosis/tiptap-php) and reads both name
272
- styles editors use: Tiptap's `bulletList`/`bold` and prosemirror-schema-basic's
273
- `bullet_list`/`strong`.
274
-
275
- It covers paragraphs, headings, blockquotes, bullet/ordered/task lists, code
276
- blocks, links, images, mentions, details, hard breaks, horizontal rules,
277
- tables, text styles (color, font family), and every text mark. A table renders
278
- as semantic `<table><tbody>`, without the column-width styling Tiptap's editor
279
- view adds.
280
-
281
- The support is layered like the Lexical side: `Y::ProseMirror` covers core
282
- ProseMirror natively (prosemirror-schema-basic plus the prosemirror-tables
283
- family) and Tiptap's extension nodes (task lists, mentions, the details
284
- family) are `Y::Tiptap`'s rule set (`Y::Tiptap::NODES`), built on the
285
- extension API below. Marks stay in the base: mark rendering (nesting order,
286
- `textStyle` CSS, `code` exclusivity) runs through native text-run machinery
287
- that node rules don't reach, so `Y::ProseMirror` renders Tiptap's mark set
288
- as-is and `rules.mark` overrides individual marks.
304
+ The output matches Tiptap's own `getHTML()`, and the tests check it against a
305
+ document captured from a real editor. The implementation follows
306
+ [`tiptap-php`](https://github.com/ueberdosis/tiptap-php) and accepts both
307
+ naming styles editors use, Tiptap's `bulletList` and `bold` as well as
308
+ prosemirror-schema-basic's `bullet_list` and `strong`.
309
+
310
+ It covers paragraphs, headings, blockquotes, bullet, ordered, and task lists,
311
+ code blocks, links, images, mentions, details, hard breaks, horizontal rules,
312
+ tables, text styles (color and font family), and every text mark. Tables render
313
+ as a plain `<table><tbody>` without the column-width styling that Tiptap's
314
+ editor view adds.
315
+
316
+ Support is split into two layers, as on the Lexical side. `Y::ProseMirror`
317
+ handles core ProseMirror natively, meaning prosemirror-schema-basic plus the
318
+ prosemirror-tables family. Tiptap's extension nodes (task lists, mentions, the
319
+ details family) are a rule set, `Y::Tiptap::NODES`, written with the extension
320
+ API described below. Marks are handled in the base class. Mark rendering deals
321
+ with nesting order, the CSS on `textStyle`, and the exclusivity of `code`, and
322
+ it runs through native text-run code that node rules can't reach, so
323
+ `Y::ProseMirror` renders Tiptap's mark set as is. You can override individual
324
+ marks with `rules.mark`.
289
325
 
290
326
  #### `Y::Lexxy` (and `Y::Lexical`, its base)
291
327
 
@@ -295,41 +331,41 @@ lexxy.to_html # the "root" fragment (Lexical's default root name)
295
331
  lexxy.to_html("notepad") # or another XML root
296
332
  ```
297
333
 
298
- The HTML is identical to what a `lexxy-editor` submits to Rails (its `value`).
299
- The tests check this byte-for-byte against a document captured from a real
300
- editor. Stock Lexical has no canonical serializer (every editor configures
301
- its own), so the editor-specific class carries the editor's name, and
302
- `Y::Lexical` is the core-Lexical base: paragraphs, headings, quotes, code,
303
- lists, tables, links, and the full text-format model, for any other Lexical
304
- editor to extend with rules.
305
-
306
- It handles the whole Lexxy 0.9.x node set: paragraphs, headings, every text
307
- format and their combinations, links, the four list types and nesting,
308
- blockquotes, code blocks, tabs and soft breaks, horizontal rules, tables with
309
- header cells, image galleries, and ActionText attachments (uploads and
310
- mentions both emit `<action-text-attachment>` elements that ActionText can
311
- re-render).
312
-
313
- Internally that support is layered: `Y::Lexical` covers core Lexical
314
- structure natively, and everything Lexxy adds, its node types (attachments,
315
- galleries) and its decorations of core nodes (the table wrapper, header-cell
316
- styling, nested-list classes), is `Y::Lexxy`'s rule set
317
- (`Y::Lexxy::NODES`), built on the extension API below. The gem's own Lexxy
318
- support is the API's first consumer: an app rule for one of those types
319
- simply replaces it.
320
-
321
- In both renderers an unknown node keeps its content: text and nested blocks
322
- fall back to readable markup rather than disappearing.
334
+ The HTML is identical to the `value` a `lexxy-editor` submits to Rails, and the
335
+ tests check it against a document captured from a real editor. The class is
336
+ named after the editor because stock Lexical has no canonical serializer, and
337
+ every editor configures its own. `Y::Lexical` is the core Lexical base
338
+ (paragraphs, headings, quotes, code, lists, tables, links, and the full
339
+ text-format model), and other Lexical editors extend it with rules.
340
+
341
+ It handles every node in Lexxy 1.0, which has the same set as 0.9.x:
342
+ paragraphs, headings, every text format and their combinations, links, the
343
+ four list types with nesting, blockquotes, code blocks, tabs and soft breaks,
344
+ horizontal rules, tables with header cells, image galleries, and ActionText
345
+ attachments. Uploads and mentions both render as `<action-text-attachment>`
346
+ elements, which ActionText can re-render.
347
+
348
+ The Lexical support has the same two layers. `Y::Lexical` handles core Lexical
349
+ structure natively, and everything Lexxy adds is in `Y::Lexxy`'s rule set,
350
+ `Y::Lexxy::NODES`, written with the extension API below. That includes Lexxy's
351
+ own node types (attachments, galleries) and its decorations of core nodes (the
352
+ table wrapper, header-cell styling, nested-list classes). Since the gem's Lexxy
353
+ support was the first thing written on that API, an app rule for one of those
354
+ types replaces the built-in one.
355
+
356
+ In both renderers, an unknown node still renders its text and nested blocks as
357
+ readable markup.
323
358
 
324
359
  #### Custom nodes and marks
325
360
 
326
- The built-in schemas are pinned to what Tiptap and Lexxy ship, but apps add
327
- their own node types. Both renderers take rules for them. A rule is checked
328
- before the built-in schema, so it can add a node type or replace how a
329
- built-in renders.
361
+ The built-in schemas match what Tiptap and Lexxy ship. When your app adds its
362
+ own node types, both renderers take rules for them. The renderer checks rules
363
+ before the built-in schema, so a rule can add a node type or change how a
364
+ built-in one renders.
330
365
 
331
- Rules register in a block, one `rules.node` call per type. A declarative
332
- rule is markup as data, rendered natively:
366
+ You register rules in a block with one `rules.node` call per type. A
367
+ declarative rule describes the markup as a tag, attributes, and a content mode,
368
+ and the renderer emits it natively:
333
369
 
334
370
  ```ruby
335
371
  tiptap = Y::Tiptap.new(doc) do |rules|
@@ -339,17 +375,18 @@ tiptap = Y::Tiptap.new(doc) do |rules|
339
375
  end
340
376
  ```
341
377
 
342
- `tag` names the element. `attrs` values are templates: a string is a literal,
343
- a symbol reads that attribute off the node, an array concatenates both kinds;
344
- an attribute that resolves empty is left out. `text` (same template form)
345
- emits literal text content. `contains` declares what lives inside the node: `:inline` (formatted text,
346
- the default), `:blocks` (child block nodes, a container), or `:none` (a
347
- leaf). `void: true` skips the closing tag.
378
+ `tag` names the element. The values in `attrs` are templates, where a string is
379
+ a literal, a symbol reads that attribute off the node, and an array
380
+ concatenates a mix of the two. Attributes that resolve to an empty value are
381
+ left out. `text` takes the same template form and emits literal text content.
382
+ `contains` says what goes inside the node, which is `:inline` for formatted
383
+ text (the default), `:blocks` for child block nodes, or `:none` for a leaf.
384
+ `void: true` skips the closing tag.
348
385
 
349
- You don't have to guess any of those names or shapes. Editors store types
350
- and attributes under names you'd never predict (Rhino's strike mark is
351
- `rhino-strike`; Lexical prefixes its own props `__`), so ask a real
352
- document instead: make one in your editor using your custom node, then:
386
+ Editors store types and attributes under names you wouldn't predict. Rhino's
387
+ strike mark is `rhino-strike`, for example, and Lexical prefixes its own props
388
+ with `__`. To find the real names and shapes, make a document in your editor
389
+ that uses your custom node, then run:
353
390
 
354
391
  ```ruby
355
392
  Y::Tiptap.new(doc).node_types
@@ -359,11 +396,19 @@ Y::Tiptap.new(doc).node_types
359
396
  # "paragraph" => { ..., "handled" => "builtin" } }
360
397
  ```
361
398
 
362
- `handled` nil marks the types that still need a rule; `attrs` are the stored
363
- names your templates and blocks will read; `children` plus `text` is how you
364
- pick `contains:` (child block types → `:blocks`; text → `:inline`).
399
+ A type with `handled` set to nil still needs a rule. `attrs` lists the stored
400
+ attribute names your templates and blocks will read, and `children` and `text`
401
+ tell you which `contains:` to pick. Child block types mean `:blocks`, and text
402
+ means `:inline`.
403
+
404
+ `unknown_types` lists the types whose `handled` is nil. `to_html` still
405
+ renders them as well as it can. An unknown container or inline wrapper renders
406
+ its children without its own markup, so its text stays. A node that keeps its
407
+ content only in its attributes renders nothing. Either way, the node's markup is
408
+ missing from the HTML, and nothing raises. When you add editor nodes, add a test
409
+ that renders a real document and asserts `unknown_types` is empty.
365
410
 
366
- When markup-as-data isn't enough, give the node a block:
411
+ When a declarative rule can't express the markup, give the node a block:
367
412
 
368
413
  ```ruby
369
414
  lexical = Y::Lexical.new(doc) do |rules|
@@ -374,26 +419,26 @@ lexical = Y::Lexical.new(doc) do |rules|
374
419
  end
375
420
  ```
376
421
 
377
- The block gets the node's type, its stored attributes, `node.content` (the
378
- children, already rendered to HTML), and `node.child_types`, the node's
379
- element/block children by type, in document order. `child_types` answers the
380
- structural questions attributes can't: how many images a gallery holds, or
381
- whether a list item carries a nested list. Whatever the block returns is
382
- spliced into the output as-is: it's trusted HTML, so escape any values you
383
- interpolate. To set the content mode for a callback, give the node both:
422
+ The block receives the node's type and stored attributes, plus `node.content`,
423
+ which is the node's children already rendered to HTML. `node.child_types` lists
424
+ the node's element and block children by type, in document order, so you can
425
+ answer questions the attributes can't, like how many images a gallery holds or
426
+ whether a list item has a nested list. The renderer inserts whatever the block
427
+ returns into the output as is and treats it as trusted HTML, so escape any
428
+ values you interpolate. To set the content mode for a block rule, pass both:
384
429
  `rules.node "embed", contains: :blocks do |node| ... end`.
385
430
 
386
- Callbacks never run while the document is locked. The render finishes first
387
- (inside one read transaction, GVL released), then the blocks run and their
388
- output is spliced in, so a callback can safely read or even write the same
389
- doc. With no callback rules, `to_html` skips the splicing entirely.
431
+ Blocks never run while the document is locked. The renderer finishes the whole
432
+ render inside one read transaction with the GVL released, and only then runs
433
+ the blocks and inserts their output, so a block can safely read the same doc or
434
+ even write to it. If no rule has a block, `to_html` skips that last step.
390
435
 
391
- Blocks are the escape hatch for everything the declarative form can't say,
392
- and they're proven sufficient: `Y::Lexxy` and `Y::Tiptap` are themselves
393
- built on this API (`lib/y/lexxy.rb`, `lib/y/tiptap.rb`): simple nodes as
394
- declarative hashes, everything with logic as plain methods mapped by node
395
- type (a `Method` responds to `call` like any lambda), and the fixture tests
396
- hold their output byte-identical to a live editor's.
436
+ Blocks cover whatever the declarative form can't express. `Y::Lexxy` and
437
+ `Y::Tiptap` are both built on this API (`lib/y/lexxy.rb`, `lib/y/tiptap.rb`),
438
+ so it already handles two complete editor schemas. Their simple nodes are
439
+ declarative hashes, and each node with logic is a plain method mapped by node
440
+ type (a `Method` responds to `call` like any lambda). The fixture tests check
441
+ that output against a live editor's.
397
442
 
398
443
  The ProseMirror side also takes custom marks:
399
444
 
@@ -403,14 +448,15 @@ tiptap = Y::Tiptap.new(doc) do |rules|
403
448
  end
404
449
  ```
405
450
 
406
- Symbol refs resolve against the mark's own attributes. A custom mark wraps
407
- outside every built-in mark; several on one run nest alphabetically. A rule
408
- for a built-in mark name (`"bold"`) replaces its built-in tag.
451
+ Symbols read from the mark's own attributes. Custom marks wrap outside all the
452
+ built-in marks, and when several custom marks apply to one run of text, they
453
+ nest alphabetically by name. A rule for a built-in mark name such as `"bold"`
454
+ replaces that mark's tag.
409
455
 
410
456
  ##### Worked examples
411
457
 
412
- A video-embed node from an app's Tiptap extension, a type the pinned schema
413
- has never heard of:
458
+ Here's a video-embed node from an app's Tiptap extension, which the built-in
459
+ schema doesn't know about:
414
460
 
415
461
  ```ruby
416
462
  tiptap = Y::Tiptap.new(doc) do |rules|
@@ -422,8 +468,8 @@ tiptap = Y::Tiptap.new(doc) do |rules|
422
468
  end
423
469
  ```
424
470
 
425
- Resolving mentions against the database. Blocks run after the document read
426
- has finished, so hitting ActiveRecord (or the doc itself) inside one is safe:
471
+ This one resolves mentions against the database. Blocks run after the document
472
+ read has finished, so a block can safely query ActiveRecord or the doc itself:
427
473
 
428
474
  ```ruby
429
475
  tiptap = Y::Tiptap.new(doc) do |rules|
@@ -436,8 +482,9 @@ tiptap = Y::Tiptap.new(doc) do |rules|
436
482
  end
437
483
  ```
438
484
 
439
- Overriding a shipped rule: rendering Lexxy uploads as real image markup
440
- instead of the `<action-text-attachment>` elements ActionText re-renders:
485
+ This one overrides a shipped rule. The shipped rule emits
486
+ `<action-text-attachment>` elements for ActionText to re-render, and the
487
+ override renders Lexxy uploads as `<figure>` and `<img>` markup:
441
488
 
442
489
  ```ruby
443
490
  lexxy = Y::Lexxy.new(doc) do |rules|
@@ -452,9 +499,10 @@ lexxy = Y::Lexxy.new(doc) do |rules|
452
499
  end
453
500
  ```
454
501
 
455
- Markup that depends on structure: `node.child_types` lists the node's
456
- element/block children in document order, so a layout container can size
457
- itself by its column count while the columns themselves stay declarative:
502
+ Some markup depends on structure. Because `node.child_types` lists the node's
503
+ element and block children in document order, a layout container can size
504
+ itself by its column count, and the columns themselves can still use
505
+ declarative rules:
458
506
 
459
507
  ```ruby
460
508
  tiptap = Y::Tiptap.new(doc) do |rules|
@@ -465,8 +513,9 @@ tiptap = Y::Tiptap.new(doc) do |rules|
465
513
  end
466
514
  ```
467
515
 
468
- Content-aware overrides: dropping the empty paragraphs an editor keeps
469
- around the cursor, since `node.content` arrives already rendered:
516
+ A block can also look at the rendered content. This one drops the empty
517
+ paragraphs an editor keeps around the cursor, which works because
518
+ `node.content` is already rendered:
470
519
 
471
520
  ```ruby
472
521
  lexical = Y::Lexical.new(doc) do |rules|
@@ -476,19 +525,20 @@ lexical = Y::Lexical.new(doc) do |rules|
476
525
  end
477
526
  ```
478
527
 
479
- For a larger reference, the gem's own editor schemas ship this way; see
480
- `Y::Lexxy::NODES` in `lib/y/lexxy.rb` (declarative hashes for the simple
481
- nodes, a plain method per node that needs logic (galleries, list items,
482
- header cells, both attachment types) mapped with `method(:name)`) and
528
+ For a larger reference, look at the gem's own editor schemas, which are written
529
+ the same way. See `Y::Lexxy::NODES` in `lib/y/lexxy.rb` (declarative hashes for
530
+ the simple nodes, and a plain method mapped with `method(:name)` for each node
531
+ that needs logic, which means galleries, list items, header cells, and both
532
+ attachment types) and
483
533
  `Y::Tiptap::NODES` in `lib/y/tiptap.rb` (task lists, mentions, the details
484
534
  family).
485
535
 
486
536
  ### Protocol codec (module functions)
487
537
 
488
- Classifying and unwrapping wire frames is stateless, so it's exposed as
489
- `Y` module functions rather than a class. The server never holds presence
490
- or document state to route a frame; presence lives in the browser clients, and
491
- the server only relays awareness frames opaquely.
538
+ Classifying and unwrapping wire frames needs no state, so these are module
539
+ functions on `Y` and you don't construct anything. The server routes a frame
540
+ without holding any presence or document state. The browser clients keep
541
+ presence, and the server relays awareness frames without reading them.
492
542
 
493
543
  ```ruby
494
544
  Y.message_kind(frame) # => 0 drop / 1 step1 / 2 update / 3 awareness / 4 query
@@ -498,55 +548,170 @@ Y.wrap_update(update_bytes) # => wrap a raw doc update as a sync Update frame
498
548
 
499
549
  ### ActionCable Integration
500
550
 
501
- In a Rails app, one generator creates the channel and the migration:
551
+ In a Rails app, one generator creates the storage migration. The models and
552
+ `Y::DocumentChannel` come with the gem:
502
553
 
503
554
  ```bash
504
555
  bin/rails generate yrby:install
505
556
  bin/rails db:migrate
506
557
  ```
507
558
 
508
- The models ship in the gem, the way Action Text owns
509
- `ActionText::RichText`:
510
-
511
- - **`Y::Document`**: one row per document, addressed two ways: by `key`
512
- (what a channel addresses; one opaque, unique string, sometimes
513
- app-supplied, never parsed) and, optionally, by polymorphic `record` +
514
- `name` (which model attribute it backs; `name` is the attribute name,
515
- `"body"`; one document per attribute per record, the
516
- ActionText::RichText scheme). Key-only documents leave the binding nil.
517
- Either side can arrive first: `Y::Document.for(record, name)` finds or
518
- creates the binding, derives a readable key (`post/1/body`), and adopts
519
- a key-only row already holding that key, so a channel writing first and
520
- a binding created later converge on one document. The row also holds
521
- the merged `state` snapshot, CRDT state only; derived data (rendered
522
- HTML, search text) is the application's job, typically in the channel's
523
- on_change. `.load_state(key)` / `.append(key, update)` are the store
524
- calls the generated channel uses.
525
- - **`Y::DocumentUpdate`**: the uncompacted tail, one delta per row,
526
- compacted into `state` and deleted once the tail reaches `compact_every`
527
- (default 64). Loading reads the snapshot plus the current tail; an
528
- empty tail returns `state` directly. Compaction serializes on a
529
- per-document row lock and skips causally-gapped rows; they're
530
- quarantined until they heal rather than compacted into state or
531
- deleted. Destroying a document deletes its updates with it.
532
-
533
- Encrypted storage: `Y::EncryptedDocument` stores `state` and update
534
- payloads through Active Record encryption on the same tables, the way
535
- `ActionText::EncryptedRichText` does. Point the channel's
536
- `on_load`/`on_change` at it instead and configure your app's encryption
537
- keys. Use one access path per document: rows written encrypted read back
538
- as ciphertext through the plain classes.
539
-
540
- The migration creates `y_documents` and `y_document_updates`. To rename
541
- them, edit the generated migration and point `Y::Document.table_name` /
559
+ As with Action Text's `ActionText::RichText`, the models are defined in the
560
+ gem:
561
+
562
+ - **`Y::Document`** stores one row per document, and you can find a row in two
563
+ ways. Channels use `key`, a single opaque, unique string that the app
564
+ sometimes supplies and that nothing parses. Optionally, a row also has a
565
+ polymorphic `record` and a `name` that say which model attribute the document
566
+ backs, where `name` is the attribute name, such as `"body"`. Each record gets
567
+ one document per attribute, the same scheme ActionText::RichText uses, and
568
+ key-only documents leave `record` and `name` nil. Either side can come first.
569
+ `Y::Document.for(record, name)` finds or creates the record binding, derives
570
+ a readable key (`post/1/body`), and adopts any key-only row that already has
571
+ that key, so a channel that writes first and a binding created later end up
572
+ on the same document. The row also holds the merged `state` snapshot, which
573
+ contains only CRDT state. Derived data such as rendered HTML or search text
574
+ is up to your application, and the usual place for it is the channel's
575
+ `on_change`. By default the channel concern calls `.load_state(key)` and
576
+ `.append(key, update)` to read and write the store.
577
+ - **`Y::DocumentUpdate`** holds the uncompacted tail, one delta per row. When
578
+ the tail reaches `compact_every` (default 64), yrby compacts it into `state`
579
+ and deletes those rows. A load reads the snapshot plus the current tail, and
580
+ returns `state` directly when the tail is empty. Compactions of one document
581
+ run one at a time under a per-document row lock, and they skip rows with an
582
+ open causal gap, leaving them as they are until the gap closes. Destroying a
583
+ document also deletes its updates.
584
+
585
+ For encrypted storage, `Y::EncryptedDocument` writes `state` and update
586
+ payloads through Active Record encryption on the same tables, as
587
+ `ActionText::EncryptedRichText` does. You declare it on the model, and the
588
+ model decides whether an attribute is encrypted, so a page or client can't
589
+ change it:
590
+
591
+ ```ruby
592
+ class Post < ApplicationRecord
593
+ has_collaborative_document :body, encrypted: true
594
+ end
595
+ ```
596
+
597
+ `Y::DocumentChannel` reads that declaration and routes every load and append
598
+ for the attribute through the encrypted class, while attributes without it use
599
+ plain `Y::Document`. In your own channel, point `on_load` and `on_change` at
600
+ `Y::EncryptedDocument`. In both cases you need to configure your app's
601
+ encryption keys and use one access path per document, because the plain classes
602
+ read encrypted rows back as ciphertext.
603
+
604
+ `post.collaborative_document(:body)` returns a bound
605
+ `Y::Collaborative::Attribute` with `load_state`, `append(update)`, `key`, and
606
+ `y_doc`. `y_doc` builds a fresh native `Y::Doc` you can read and render in
607
+ Ruby. For built-in row operations like compaction, call
608
+ `post.collaborative_document(:body).document_row.compact!`. The channel and
609
+ your application code go through the same accessor, which handles encryption
610
+ for both.
611
+
612
+ An attribute uses the key its document row was stored under. Before a row
613
+ exists, the key is `Y::Document.key_for(record, name)`, and computing it
614
+ doesn't create a row. Grants keep their existing scope and lifetime.
615
+
616
+ To check the current user's permissions in addition to the signed grant, give
617
+ the shipped channel an `authorize_document` block. It runs in channel context,
618
+ so it can use `current_user` or any other identifiers your Action Cable
619
+ connection provides:
620
+
621
+ ```ruby
622
+ # config/initializers/yrby.rb
623
+ Rails.application.config.to_prepare do
624
+ Y::DocumentChannel.authorize_document do |record, name|
625
+ current_user.present? && record.editable_by?(current_user, attribute: name)
626
+ end
627
+ end
628
+ ```
629
+
630
+ `editable_by?` is a policy method in your application, and yrby doesn't provide
631
+ it. The block receives a freshly loaded record and the attribute name as a
632
+ string. If it returns false or nil, the channel rejects the subscription before
633
+ opening a stream or serving any state. An invalid grant is rejected before the
634
+ block is called. Without a block, a valid grant is enough, so the view that
635
+ renders the helper must still require edit permission. If the block raises, the
636
+ channel rejects the subscription and the error goes to your normal error
637
+ handling.
638
+
639
+ The policy runs once, when the client subscribes, and after that the open
640
+ subscription authorizes each message. That's how Action Cable is meant to work,
641
+ and it means a keystroke or cursor move doesn't cost a record load plus your
642
+ own queries. The tradeoff is that if you revoke a permission mid-session, it
643
+ takes effect the next time that client subscribes. You can shorten that window
644
+ in two ways. Every new subscription checks grant expiry, so a short
645
+ `expires_in:` on the tag limits it, as long as you also give the element a
646
+ `refresh:` URL (see below). If your application has to cut off access
647
+ immediately, stop the subscription yourself when the permission changes.
648
+
649
+ Both transports remember the decision for the life of the subscription. Action
650
+ Cable keeps the same channel instance around. AnyCable builds a new one for
651
+ each command, so yrby declares the authorized document as channel state and
652
+ sends it with each RPC, and because anycable-go holds that state, a client
653
+ can't forge it. A frame that arrives without an authorized subscription is
654
+ rejected even if its grant is valid.
655
+
656
+ Every channel does its authorization in one method, `authorized?`, which the
657
+ concern calls when a client subscribes, before it opens a stream or serves any
658
+ state. In a channel you write, you define `authorized?` yourself and it
659
+ receives the document key. The shipped `Y::DocumentChannel` defines it to run
660
+ the `authorize_document` block with the record and the attribute name, or to
661
+ accept any valid grant when there's no block. A subclass can override
662
+ `authorized?` directly and read the located record from `record`.
663
+
664
+ ### Grant lifetime and refresh
665
+
666
+ A grant lasts as long as GlobalID's signed-id default, which is one month under
667
+ Rails, and `expires_in:` on the tag shortens it. The grant is part of the
668
+ rendered page, though, and Action Cable resubscribes with it after every
669
+ network drop. If the grant expires before the user is done editing, the editor
670
+ gets blocked at the next reconnect. To avoid that, pair `expires_in:` with
671
+ `refresh:`, a URL the element fetches when a subscription is rejected:
672
+
673
+ ```erb
674
+ <%= collaborative_document_tag @post, :body, expires_in: 10.minutes,
675
+ refresh: grant_post_path(@post) %>
676
+ ```
677
+
678
+ ```ruby
679
+ # config/routes.rb: resources :posts do get :grant, on: :member end
680
+ # app/controllers/posts_controller.rb
681
+ def grant
682
+ @post = current_user.posts.find(params[:id]) # your own authorization, again
683
+ render json: { grant: @post.collaborative_sgid(:body, expires_in: 10.minutes) }
684
+ end
685
+ ```
686
+
687
+ That action grants write access, so its check must be at least as strict as the
688
+ page that renders the tag. If it skips authorization, anyone who can reach the
689
+ URL gets a grant, and a short `expires_in:` protects nothing.
690
+
691
+ When a subscription is rejected, the element fetches that URL with the session
692
+ cookie, and the action runs your authorization again. If the response is
693
+ `{ "grant": ... }`, the same session resubscribes with the new grant and keeps
694
+ its document and pending edits. Any other response, a non-2xx status, or a second
695
+ rejection blocks the session as before, and so does a refresh that takes
696
+ longer than 15 seconds. The element doesn't renew grants on a timer, so it
697
+ won't interrupt a healthy open subscription. Every reconnect after expiry is a
698
+ fresh permission check, which is why you'd want a short lifetime in the first
699
+ place.
700
+
701
+ For room-keyed collaboration or other custom channel behavior, generate a
702
+ channel with `bin/rails generate yrby:install --channel` and implement its
703
+ `authorized?(key)`. Custom channels can still use both storage hooks.
704
+
705
+ The migration creates `y_documents` and `y_document_updates`. To rename them,
706
+ edit the generated migration and point `Y::Document.table_name` and
542
707
  `Y::DocumentUpdate.table_name` at the new names in an initializer.
543
708
 
544
- Storage is swappable: the channel only needs `on_load` and `on_change`
709
+ You can swap the storage. The channel only needs `on_load` and `on_change`
545
710
  answered, and they can point at anything.
546
711
 
547
712
  `include Y::ActionCable` (from the `yrby-rails` gem) is the channel
548
- integration: the y-websocket protocol (document sync +
549
- awareness/presence) over ActionCable.
713
+ integration. It implements the y-websocket protocol over ActionCable, covering
714
+ document sync, awareness, and presence.
550
715
 
551
716
  ```ruby
552
717
  # app/channels/document_channel.rb
@@ -557,8 +722,6 @@ class DocumentChannel < ApplicationCable::Channel
557
722
  on_change { |key, update| Y::Document.append(key, update) } # record, then broadcast
558
723
 
559
724
  def subscribed
560
- return reject unless authorized?(params[:id])
561
-
562
725
  sync_subscribed params[:id]
563
726
  end
564
727
 
@@ -568,80 +731,112 @@ class DocumentChannel < ApplicationCable::Channel
568
731
 
569
732
  private
570
733
 
571
- # Everyone is denied until you wire this to your app's auth.
734
+ # This denies everyone until you wire it to your app's auth.
735
+ # sync_subscribed rejects the subscription unless this returns true.
572
736
  def authorized?(_document_key) = false
573
737
  end
574
738
  ```
575
739
 
576
- The concern is store-backed. A handshake is answered from `on_load`; document
577
- changes are recorded through `on_change`, then broadcast. Nothing
578
- authoritative is kept in ActionCable process memory, so
579
- AnyCable RPC workers, Puma workers, and separate dynos can all handle messages
580
- for the same document as long as they share the same store and cable adapter.
740
+ For documents that belong to a record, `Y::Collaborative` (which the engine
741
+ includes into Active Record) provides the token flow `authorized?` needs. The
742
+ page creates a signed GlobalID scoped to one attribute, and the channel looks
743
+ up the record from it, so the page decides which document the client gets.
581
744
 
582
- `on_load` and `on_change` are required. If either is missing, the channel fails
583
- before it can acknowledge or broadcast edits. Presence is ephemeral:
584
- awareness frames are relayed, and `yrby-client` sends a best-effort
585
- presence-removal frame on disconnect/pagehide, with the client-side awareness
586
- timeout as the fallback for abrupt disconnects.
745
+ ```erb
746
+ <%# the view picks the document and signs it %>
747
+ <%= tag.div data: { grant: post.collaborative_sgid(:body) } %>
748
+ ```
749
+
750
+ ```ruby
751
+ # The channel looks up the record from the token. This isn't memoized
752
+ # because AnyCable builds a fresh channel for each command, and a cached
753
+ # record would go stale.
754
+ def authorized?(_key) = record.present? && record.editable_by?(current_user)
755
+ def record = Y::Collaborative.locate(params[:grant], :body)
756
+ ```
757
+
758
+ A token created for `:body` only verifies under `:body`'s purpose
759
+ (`"yrby/body"`), and a tampered, expired, or wrong-attribute token returns no
760
+ record.
761
+
762
+ The concern reads and writes through your store. It answers handshakes from
763
+ `on_load`, and it passes document changes through `on_change` before
764
+ broadcasting them. The ActionCable process doesn't keep authoritative state in
765
+ memory, so AnyCable RPC workers, Puma workers, and separate dynos can all
766
+ handle messages for the same document, as long as they share a store and a
767
+ cable adapter.
768
+
769
+ If the yrby-rails models are installed and you declare neither hook, `on_load`
770
+ and `on_change` both use `Y::Document` storage. To use a different store,
771
+ declare both hooks so reads and writes go to the same place. Once a pair is
772
+ configured explicitly, a subclass may override either hook. Outside a
773
+ yrby-rails app there's no default, and until you declare both hooks, the
774
+ channel fails before it can acknowledge or broadcast an edit.
775
+
776
+ Presence is ephemeral. The server relays awareness frames, and `yrby-client`
777
+ sends a best-effort presence-removal frame on disconnect and on `pagehide`. If
778
+ a client can't send that frame, the client-side awareness timeout removes its
779
+ presence.
587
780
 
588
781
  Incoming frames are validated as a single well-formed protocol message before
589
782
  anything processes or relays them. Malformed, truncated, multi-message,
590
- oversized, or unknown frames are dropped. A bad frame can't crash the process: a
591
- Rust panic is caught at the FFI boundary and re-raised as a Ruby exception. And
592
- no single client can relay garbage that breaks the others in a room.
783
+ oversized, and unknown frames are dropped, so a bad frame can't crash the
784
+ process and one client can't relay garbage that breaks everyone else in the
785
+ room. A Rust panic is caught at the FFI boundary and re-raised as a Ruby
786
+ exception.
593
787
 
594
788
  #### Delivery guarantees
595
789
 
596
- The contract is the same at every scale: one process, or hundreds across many
597
- servers:
790
+ These guarantees are the same whether you run one process or hundreds of them
791
+ across many servers:
598
792
 
599
793
  - **The document always converges.** CRDT updates are commutative and
600
- idempotent, so out-of-order, duplicate, or concurrent delivery all converge to
601
- the same correct document. This needs no coordination and holds everywhere.
602
- - **An acked update is durable, even one that arrived out of order.** An
603
- update with a missing dependency is recorded and acked like any other, and
604
- parks as pending in the document. That missing dependency is an update some
605
- client still holds unacked, so that client keeps retransmitting it until
606
- the server records it, and the gap closes. The ack loop is the guarantee.
607
- See [Causal gaps](#causal-gaps).
608
- - **`on_change` is at-least-once, and the durable guarantee is that replaying the
609
- log reconstructs the document.** Every update triggers `on_change` before it's acked or
610
- broadcast (record-before-distribute). If exactly-once updates matter for you, **you
611
- must make `on_change` idempotent**. But remember that the CRDT can handle duplicates.
794
+ idempotent, so out-of-order, duplicate, and concurrent delivery all produce
795
+ the same correct document in every deployment, with no coordination.
796
+ - **An acked update is durable, including one that arrived out of order.** The
797
+ server records and acks an update with a missing dependency like any other,
798
+ and the update waits as pending in the document. Some client still holds the
799
+ missing update unacked and keeps retransmitting it until the server records
800
+ it, and then the gap closes. See [Causal gaps](#causal-gaps).
801
+ - **`on_change` runs at least once, and replaying the log always rebuilds the
802
+ document.** yrby calls `on_change` for every update before acking or
803
+ broadcasting it. If you need exactly-once behavior, make `on_change`
804
+ idempotent. The CRDT handles duplicates either way.
612
805
  - **A raising `on_change` rejects the update implicitly.** If the block raises,
613
- the update is neither acked nor broadcast (record-before-distribute stops both).
614
- There is no negative-ack: the client simply never receives the ack, keeps the
615
- update pending, and retransmits on its timer/reconnect. This is built for
616
- *transient* failures (the store is briefly down → a retry lands). A block that
617
- raises *deterministically* (a validation that always fails for this edit)
618
- will be retried forever, since nothing tells the client to stop. Enforce hard
619
- rejections before the edit reaches `on_change` (channel authorization in
620
- `subscribed`), not by raising inside it.
806
+ the server doesn't ack or broadcast the update, and it doesn't send a
807
+ negative ack either. The client keeps the update pending and retransmits it
808
+ on its timer or on reconnect. That works for transient failures, such as a
809
+ store that was briefly down, where a retry succeeds. A block that raises
810
+ every time (say, a validation that always fails for this edit) gets retried
811
+ forever, because nothing tells the client to stop. Put hard rejections in the
812
+ channel's authorization at subscribe time, before an edit can reach
813
+ `on_change`, and don't raise inside the hook for them.
621
814
  - **An over-cap frame is dropped the same silent way.** A frame larger than
622
- `max_frame_bytes` (default 8 MiB) is dropped before decoding (no ack, no
623
- broadcast) to bound the work a client can force. For a genuine document
624
- update that means the same implicit rejection as above: unacked, retransmitted
625
- forever. Normal typing never approaches the cap, but a large paste, an embedded
626
- image, or a big initial `SyncStep2` can. The drop is logged (`warn` for
627
- over-cap, `debug` for undecodable) with the document key and update id so it's
628
- findable; override `sync_log_context` on the channel to add a user/connection
629
- id. Size the cap for your largest expected payload, and reject
630
- genuinely-too-big content upstream rather than relying on the cap to reject it
631
- gracefully.
815
+ `max_frame_bytes` (default 8 MiB) is dropped before decoding, with no ack and
816
+ no broadcast, which limits how much work a client can force on the server. A
817
+ real document update that hits the cap gets the same implicit rejection as
818
+ above, so it's never acked and is retransmitted forever. Normal typing won't
819
+ come near the cap, but a large paste, an embedded image, or a big initial
820
+ `SyncStep2` can. The server logs each drop (`warn` for over-cap, `debug` for
821
+ undecodable) with the document key and update id so you can find it, and you
822
+ can override `sync_log_context` on the channel to add a user or connection
823
+ id. Size the cap for your largest expected payload, and reject content that's
824
+ too big before it reaches the channel. The cap is a last line of defense, and
825
+ the client never sees an error from it.
632
826
 
633
827
  #### Causal gaps
634
828
 
635
- Yjs updates can arrive out of order: an update can reach the server before
636
- another update it depends on. yrby treats that as normal. The update is
637
- recorded and acked like any other, parks as a pending struct in the document,
638
- and integrates on its own the moment the missing dependency lands. The write
639
- path never rebuilds the document; it appends, relays, and acks, so a gapped
640
- update costs the same as any other.
641
-
642
- Serving is lossless too, like any Yjs server. `handle_sync_message` serves
643
- full state, pending included, so a peer parks the same pending struct and
644
- heals it the same way. Healing needs no special machinery: the missing
829
+ Yjs updates can arrive out of order, so an update can reach the server before
830
+ the update it depends on. yrby treats that as normal. It records and acks the
831
+ update like any other, the update waits as a pending struct in the document,
832
+ and it integrates once the missing dependency arrives. The write path appends,
833
+ relays, and acks without rebuilding the document, so an update with a gap costs
834
+ the same as any other.
835
+
836
+ Serving is lossless too, as on any Yjs server. `handle_sync_message` serves
837
+ full state with pending structs included, so a peer holds the same pending
838
+ struct and integrates it the same way. Closing the gap takes no special
839
+ machinery, because the missing
645
840
  dependency is an update its sender still holds unacked, and at-least-once
646
841
  retransmission delivers it. Only compaction excludes pending
647
842
  (`compacted_state_update`), because folding a log must not freeze an
@@ -650,14 +845,14 @@ un-integrable struct into the base state.
650
845
  The bundled `Y::Document` store handles all of this. If you write your own
651
846
  store, keep two things in mind:
652
847
 
653
- **1. Load losslessly, and tolerate duplicates.** `on_load` should return
654
- state that preserves pending: `encode_state_as_update`, or a replay of the
848
+ **1. Load losslessly, and tolerate duplicates.** `on_load` should return state
849
+ that keeps pending updates, either `encode_state_as_update` or a replay of the
655
850
  raw append log. Don't compact with `compacted_state_update` while
656
- `doc.pending?`; that strips the pending struct and the acked edit inside it.
657
- (`Y::Document` quarantines pending rows for exactly this reason.) A lost ack
658
- also means a client resends an update the store already has. Replay converges
659
- anyway, because CRDT apply is idempotent, so deduping is optional. If log
660
- size matters, dedup by content hash:
851
+ `doc.pending?` is true, because that strips the pending struct and loses the
852
+ acked edit inside it. `Y::Document` keeps pending rows out of compaction for
853
+ this reason. When an ack gets lost, the client resends an update the store
854
+ already has. Replay still converges because CRDT apply is idempotent, so
855
+ deduping is optional. If log size matters, dedup by content hash:
661
856
 
662
857
  ```ruby
663
858
  class DocumentStore
@@ -688,17 +883,18 @@ class DocumentStore
688
883
  end
689
884
  ```
690
885
 
691
- **2. Watch for gaps that never heal.** An open gap is quiet: the edit sits
692
- as pending, invisible in the document, until its dependency arrives. Normally
693
- that resolves itself. The sender retransmits the missing update until it is
694
- acked, and every join or reconnect handshake has the client send everything
695
- the server hasn't integrated, so any client holding the dependency supplies
696
- it just by connecting. The gap worth alerting on is one no live client can
697
- supply, and that is what the `on_gap` hook surfaces. It fires with the
698
- document key whenever a document is loaded to serve state and a gap is still
699
- open. Use it to emit a metric (a pending-document count, or the age of the
700
- oldest open gap) so a stuck gap is visible. Gaps are also logged at `info`,
701
- and errors raised in the hook are swallowed so observability can never break
886
+ **2. Watch for gaps that don't close.** An open gap is easy to miss, because
887
+ the pending edit doesn't show up in the document until its dependency arrives.
888
+ Usually the gap closes by itself. The sender retransmits the missing update
889
+ until it's acked, and on every join or reconnect handshake the client sends
890
+ everything the server hasn't integrated, so any client holding the dependency
891
+ supplies it by connecting. The gap worth alerting on is one that no live client
892
+ can supply, and that's what the `on_gap` hook is for. It fires with the
893
+ document key whenever a document is loaded to serve state while a gap is open.
894
+ Use it to emit a metric, such as a pending-document count or the age of the
895
+ oldest open gap, so you can see a gap that isn't closing. yrby also logs gaps
896
+ at `info`. Errors raised in the hook are swallowed, so a broken metrics call
897
+ can't break
702
898
  frame handling.
703
899
 
704
900
  ```ruby
@@ -711,28 +907,32 @@ end
711
907
 
712
908
  #### Multi-process deployments
713
909
 
714
- Most Rails apps run several processes, and any of them might serve a given document.
715
- Two pieces keep them in step.
910
+ Most Rails apps run several processes, and any of them might end up serving a
911
+ given document. Two things keep them consistent.
716
912
 
717
- Broadcasts cross processes through the Action Cable adapter, so it needs to something
718
- like `redis` or `solid_cable`, not `async`. With that in place, a change
913
+ Broadcasts go between processes through the Action Cable adapter, so use
914
+ `redis`, `solid_cable`, or another cross-process adapter, since `async` only
915
+ works inside one process. With one of those adapters in place, a change
719
916
  on one process reaches clients on all of them.
720
917
 
721
918
  Every process rebuilds document state from the durable store through `on_load`.
722
- Because changes are recorded before broadcast, record-before-distribute holds
723
- across processes: whichever process receives a change records it to the shared
724
- store before anyone, anywhere, sees it.
919
+ Changes are still recorded before they're broadcast, so whichever process
920
+ receives a change writes it to the shared store before any client on any
921
+ process sees it.
725
922
 
726
- `bun multiprocess.mjs` in the demo runs clients across two processes and checks
727
- convergence, fresh reads on both, presence across processes, and one shared log.
923
+ In the demo, `bun multiprocess.mjs` runs clients across two processes and
924
+ checks that the documents converge, that fresh reads work on both processes,
925
+ that presence reaches clients on the other process, and that both processes
926
+ write to one shared log.
728
927
 
729
928
  ##### AnyCable
730
929
 
731
- `yrby` fully supports AnyCable.
930
+ `yrby` supports AnyCable end to end.
732
931
 
733
- The demo checks this against a real anycable-go + RPC server
734
- (`frontend/anycable_probe.mjs`, `anycable_concurrent.mjs`): liveness, the
735
- yrby client provider, cross-process reads, and concurrent convergence.
932
+ The demo tests it against a real anycable-go server and RPC server in
933
+ `frontend/anycable_probe.mjs` and `anycable_concurrent.mjs`, covering liveness,
934
+ the yrby client provider, cross-process reads, and convergence under concurrent
935
+ editing.
736
936
 
737
937
  ##### Demo
738
938
 
@@ -741,8 +941,8 @@ app using the yrby provider, with end-to-end tests.
741
941
 
742
942
  #### Record Before Distribute
743
943
 
744
- Every document change is handed to the `on_change` handler before broadcasting.
745
- It is up to you to durably record it:
944
+ Every document change goes through your `on_change` handler before it's
945
+ broadcast, and the handler is where you record it durably:
746
946
 
747
947
  ```ruby
748
948
  class DocumentChannel < ApplicationCable::Channel
@@ -759,21 +959,21 @@ class DocumentChannel < ApplicationCable::Channel
759
959
  end
760
960
  ```
761
961
 
762
- If the recorder raises (say the store is down), the change is rejected: not
763
- applied, not sent to anyone. The cost is a synchronous durable write on the path
764
- of every change. There's no in-gem per-document lock; concurrent writes to one
765
- document can both record (at-least-once), and since CRDT apply is idempotent a
766
- duplicate record replays to the same document.
962
+ If the handler raises (say the store is down), the change is rejected, so it
963
+ isn't applied or sent to anyone. The cost is a synchronous durable write on the
964
+ path of every change. The gem doesn't take a per-document lock, so two
965
+ concurrent writes to one document can both record (at least once). Because CRDT
966
+ apply is idempotent, a duplicate record replays to the same document.
767
967
 
768
- The demo wires `on_change` to a durable Postgres-backed log by default, and checks
769
- end to end that the log alone rebuilds the document.
968
+ The demo wires `on_change` to a durable Postgres-backed log by default, and
969
+ checks end to end that the log alone rebuilds the document.
770
970
 
771
971
  #### Ephemeral documents (no database)
772
972
 
773
- `on_load` and `on_change` are plain blocks, and nothing requires them to touch
774
- a database. For documents that don't need to outlive their session (a
775
- scratchpad, live form state, a draft you only persist on submit) the store
776
- can be connection state that travels with each request:
973
+ `on_load` and `on_change` are plain blocks, and they don't have to touch a
974
+ database. Some documents only need to last for a session, such as a scratchpad,
975
+ live form state, or a draft you save on submit. For those, the store can be
976
+ connection state that's sent with each request:
777
977
 
778
978
  ```ruby
779
979
  class ScratchpadChannel < ApplicationCable::Channel
@@ -790,6 +990,11 @@ class ScratchpadChannel < ApplicationCable::Channel
790
990
 
791
991
  def subscribed = sync_subscribed(params[:id])
792
992
  def receive(data) = sync_receive(data, params[:id])
993
+
994
+ private
995
+
996
+ # Each subscriber gets its own scratchpad on this connection.
997
+ def authorized?(_key) = true
793
998
  end
794
999
  ```
795
1000
 
@@ -815,65 +1020,69 @@ class ScratchpadChannel < ApplicationCable::Channel
815
1020
 
816
1021
  def subscribed = sync_subscribed(params[:id])
817
1022
  def receive(data) = sync_receive(data, params[:id])
1023
+
1024
+ private
1025
+
1026
+ def authorized?(_key) = true # per-connection scratchpad, see above
818
1027
  end
819
1028
  ```
820
1029
 
821
- Both hooks run in the channel instance (`instance_exec`), so they can use
822
- anything the channel can, and `sync_receive` rebuilds the document from
823
- `on_load` on every update, which is what lets the store live on the
824
- connection. On Action Cable the channel instance lasts as long as the
825
- connection, so an instance variable is the whole store. Merging into
826
- `compacted_state_update` keeps it one blob instead of a growing update log.
827
-
828
- The store is per connection, which shapes what this fits. A single writer gets
829
- the full delivery contract with no database anywhere. With several people
830
- editing at once, one client's update can depend on edits its own connection
831
- has never seen; that update records as pending, and the next handshake with
832
- that client (which always holds the full document) supplies the missing state
833
- and heals it. The document still converges; heavy concurrent editing just
834
- parks more pending between handshakes than a shared store would. On
835
- AnyCable, keep the payload in mind too: the blob travels with every message,
836
- so that variant suits small documents, not long manuscripts.
1030
+ Both hooks run in the channel instance through `instance_exec`, so they can use
1031
+ anything the channel can. `sync_receive` rebuilds the document from `on_load`
1032
+ on every update, which is why the store can sit on the connection. On Action
1033
+ Cable the channel instance lasts as long as the connection, so an instance
1034
+ variable is all the store you need. Merging with `compacted_state_update` keeps
1035
+ the store at one blob so it doesn't grow into an update log.
1036
+
1037
+ Because the store is per connection, this pattern has limits. A single writer
1038
+ gets the full delivery contract without any database. When several people edit
1039
+ at the same time, one client's update can depend on edits its own connection
1040
+ has never seen. That update is recorded as pending, and the next handshake with
1041
+ that client (which always has the full document) supplies the missing state and
1042
+ closes the gap. The document still converges, but heavy concurrent editing
1043
+ leaves more pending between handshakes than a shared store would. On AnyCable,
1044
+ watch the payload size as well. The blob is sent with every message, so this
1045
+ only makes sense for small documents.
837
1046
 
838
1047
  Durability is the connection plus the browsers. A reconnecting client re-seeds
839
- an empty server through the ordinary sync handshake, so the document survives
840
- server restarts as long as some client still has it. For ephemeral documents
841
- shared across clients on a single-process deployment, the same two hooks over
842
- a class-level `Concurrent::Map` work instead; that version stops being
843
- coherent the moment you scale past one process.
1048
+ an empty server through the ordinary sync handshake, so the document survives a
1049
+ server restart as long as some client still has it. If an ephemeral document
1050
+ needs to be shared across clients on a single-process deployment, point the
1051
+ same two hooks at a class-level `Concurrent::Map`. That version is no longer
1052
+ coherent once you run more than one process.
844
1053
 
845
1054
  #### Reliable delivery (acks)
846
1055
 
847
1056
  yrby document delivery is ack-tracked. Browser document updates carry an
848
- `"id"`, and the server replies `{ "ack": <id> }` once `on_change` has
849
- successfully fired. Every decodable document update is recorded and acked,
850
- including one that arrives out of order.
1057
+ `"id"`, and the server replies `{ "ack": <id> }` after `on_change` succeeds.
1058
+ The server records and acks every decodable document update, including one that
1059
+ arrives out of order.
851
1060
 
852
1061
  ```
853
1062
  client -> server { "update": "<base64 update>", "id": 42 }
854
1063
  server -> client { "ack": 42 } # update accepted; safe to forget
855
1064
  ```
856
1065
 
857
- `yrby-client`'s `ActionCableProvider` handles this automatically. It keeps
858
- the unacknowledged local document tail in a queue and sends the merged tail as a
859
- single causally-complete delta. The id is the highest sequence in the batch, so
860
- one `{ ack: id }` cumulatively confirms everything up to it. Because CRDT apply
861
- is idempotent, a resend that already landed is a harmless no-op that just
862
- re-acks. Awareness stays ephemeral and is not acked.
1066
+ `yrby-client`'s `ActionCableProvider` handles this for you. It queues the
1067
+ unacknowledged tail of local document updates and sends it merged into one
1068
+ causally complete delta, using the highest sequence in the batch as the id, so
1069
+ one `{ ack: id }` confirms everything up to it. Because CRDT apply is
1070
+ idempotent, resending an update the server already has does no harm, and the
1071
+ server acks it again. Awareness is ephemeral and isn't acked.
863
1072
 
864
- Presence (cursors, selections) is owned by the browser clients; the server
865
- never sets or holds presence state, it only relays awareness frames opaquely.
866
- See `yrby-client` for the client-side awareness API.
1073
+ The browser clients manage presence (cursors, selections). The server doesn't
1074
+ set or hold presence state, and it relays awareness frames without reading
1075
+ them. See `yrby-client` for the client-side awareness API.
867
1076
 
868
1077
  ## Thread Safety
869
1078
 
870
- A `Doc` is safe to share across Ruby threads, used concurrently from Puma
871
- workers, ActionCable connection threads, or background jobs without external
872
- locking.
1079
+ You can share a `Doc` across Ruby threads. Puma workers, ActionCable connection
1080
+ threads, and background jobs can all use the same one concurrently, with no
1081
+ locking in your code.
873
1082
 
874
1083
  `test/thread_safety_test.rb` runs shared docs, the full sync handshake, and
875
- fan-in sync across 8 threads at once, and checks the interleaving doesn't change
876
- convergence.
1084
+ fan-in sync across 8 concurrent threads and checks that the interleaving
1085
+ doesn't change convergence.
877
1086
 
878
1087
  ### Parallelism (GVL release)
879
1088
 
@@ -881,21 +1090,21 @@ Every method that does real CRDT work (applying updates, encoding state,
881
1090
  handling sync messages) releases Ruby's Global VM Lock
882
1091
  (`rb_thread_call_without_gvl`) while the native code runs. That buys two things.
883
1092
 
884
- CRDT work runs in parallel across Ruby threads on MRI, not just
885
- JRuby/TruffleRuby. `bench/parallelism_bench.rb` measures over 2x wall-clock
886
- speedup applying a ~900 KB update concurrently; native code that held the GVL
887
- couldn't beat serial time.
888
-
889
- A slow operation also can't stall the VM. A thread applying a large update holds
890
- the doc's write lock without holding the GVL, so other Ruby threads keep running
891
- instead of queuing behind it.
892
-
893
- Each method has the same shape: copy Ruby byte strings first, drop the GVL, do
894
- the yrs work while taking and releasing native locks entirely inside the
895
- closure, take the GVL back, then build Ruby objects. No Ruby API is touched
896
- without the GVL, and no native lock is held while reacquiring it, so the lock
897
- order can't deadlock. Panics in native code are caught and re-raised as Ruby
898
- exceptions.
1093
+ First, CRDT work runs in parallel across Ruby threads on MRI, with no need for
1094
+ JRuby or TruffleRuby. `bench/parallelism_bench.rb` measures more than a 2x
1095
+ wall-clock speedup when applying a roughly 900 KB update concurrently. Native
1096
+ code that held the GVL couldn't beat serial time.
1097
+
1098
+ Second, a slow operation can't stall the VM. A thread applying a large update
1099
+ holds the doc's write lock but not the GVL, so other Ruby threads keep running
1100
+ while it works.
1101
+
1102
+ Each of those methods copies the Ruby byte strings, releases the GVL, does the
1103
+ yrs work (taking and releasing the native locks inside that closure), takes the
1104
+ GVL back, and then builds the Ruby objects. Ruby APIs are only called while
1105
+ holding the GVL, and no native lock is held while reacquiring it, so the lock
1106
+ order can't deadlock. A panic in native code is caught and re-raised as a Ruby
1107
+ exception.
899
1108
 
900
1109
  ## Message Type Constants
901
1110