yrby 0.6.0-aarch64-linux → 0.7.0-aarch64-linux

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: d8ac7194fe7f383fbea760aae137ae6848f6291687eff4a334dc318d1600f467
4
- data.tar.gz: 98197e70d3928419eac8a590b50bff1a336ae53c62f2eda17d2b9147e1280cb0
3
+ metadata.gz: 7b56482181dafbce1bac53dedf886c50be3e59440275bded5598dae9ce0879fa
4
+ data.tar.gz: 2c107975f0a63abe25fd26a04acab155a908aa2a8795ef4c44c0ad9bacee20cf
5
5
  SHA512:
6
- metadata.gz: 9aa048cce5443d92b7192174da78525019535142323033b85ffe6788258097a9ccbca492c567d8b456491a130f067605b521e83b3b3f1e7858785d268b7158af
7
- data.tar.gz: fbaa5ffe1493344b10c012d9ec4284f85dc9734a7386a438ae40672733c48f0082d3c9d0d5e1556881597f08e10add4e360ffc40700c6864789320c4c22e5f28
6
+ metadata.gz: ad63b30fc03e05f50321a925cb8714852922ebffb13847d65a36e96db5165694c66e599bff27a610b1813788ecb5c7cb729c78ee1375a63cb584512e46a6b250
7
+ data.tar.gz: '039e1cda7052a5688470361c574ff8621989c55c9a9a9a4b4d12a9168a72a2498712996b75e61383465b3df830d467aef41bf734d20a4029f393afa8125a27bf'
data/CHANGELOG.md CHANGED
@@ -4,21 +4,66 @@ All notable changes to this project are documented here. The format is based on
4
4
  [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project aims
5
5
  to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6
6
 
7
+ ## [0.7.0] - 2026-08-11
8
+
9
+ ### Added
10
+
11
+ - `Y::Decoder` ships in the core gem and loads with `require "y"`. It was
12
+ scaffolded as a separate `yrby-decoder` gem, but it is 66 lines of pure
13
+ Ruby over `Doc#read_text` / `read_xml`, requires the native core either
14
+ way, and the separate gem was never published, so the split left the
15
+ module in no gem at all. The `yrby-decoder` name is retired unused.
16
+
17
+ ### Changed
18
+
19
+ - `Doc#handle_sync_message` answers a SyncStep1 with the doc's full state,
20
+ pending included, matching Y.js's `encodeStateAsUpdate`. It previously
21
+ served integrated-only state. A peer parks a served pending struct the
22
+ same way the doc did and heals it when the missing dependency arrives
23
+ from its sender's ack-driven retransmit. `compacted_state_update` still
24
+ excludes pending, so compaction cannot freeze a gap into a snapshot.
25
+
26
+ ### Fixed
27
+
28
+ - **`Doc#update_advances?` no longer misreads a gappy merged update carrying
29
+ novel content as a no-op.** A crafted frame can hide an internal gap behind
30
+ a Skip block while its post-gap blocks still integrate (yrs plants a Skip
31
+ hole in the store). That moves neither the doc's public state vector nor
32
+ pending, so the probe comparison reported genuinely novel content as
33
+ "doesn't advance"; `update_ready?` accepted the frame and the doc applied
34
+ it, but it was never recorded to the durable log or broadcast. Any
35
+ insertion past the update's own (Skip-capped) `state_vector()` now
36
+ conservatively reports as advancing. Not reachable through standard Yjs
37
+ providers (a client's own updates and diffs are gap-free), so this closes
38
+ a hostile-input hole, not a real-world regression.
39
+
40
+ ## [0.6.1] - 2026-08-04
41
+
42
+ ### Fixed
43
+
44
+ - `Y::Lexxy` emits the attachment tag the node was created with instead of
45
+ a hardcoded `<action-text-attachment>`. Lexxy makes the tag configurable
46
+ (`Lexxy.configure`'s `attachmentTagName`, paired with
47
+ `ActionText::Attachment.tag_name` in Rails), and each attachment node
48
+ stores its tag, so a custom-tag app's rendered HTML now matches its
49
+ editor. A stored value that doesn't look like a tag name falls back to
50
+ the default; documents from before the tag was stored render unchanged.
51
+
7
52
  ## [0.6.0] - 2026-07-11
8
53
 
9
54
  ### Added
10
55
 
11
56
  - **Custom render rules for `Y::Lexical` and `Y::ProseMirror`.** Both
12
- renderers now take a block registering rules per node type
57
+ renderers now take a block registering rules per node type:
13
58
  `rules.node "callout", tag: "aside"` for markup-as-data, a Ruby block for
14
- logic plus `nodes:`/`marks:` keywords as the equivalent data form, to
59
+ logic, plus `nodes:`/`marks:` keywords as the equivalent data form, to
15
60
  render node types the pinned schemas don't know or to override how a
16
61
  built-in renders.
17
62
  Declarative rules (`tag`/`attrs`/`text`/`contains`, with templates mixing
18
63
  literals and attribute references) render natively at full speed. Callback
19
64
  rules run a Ruby block per node, receiving its type, attributes,
20
65
  already-rendered children, and `child_types` (its element/block children by
21
- type the structural facts behind gallery counts and nested-list classes).
66
+ type, the structural facts behind gallery counts and nested-list classes).
22
67
  The block runs after the document read has finished, never while the doc is
23
68
  locked, so it can safely read or write the same doc. Blocks are proven
24
69
  sufficient for whole schemas: the gem's own editor schemas (`Y::Lexxy`,
@@ -26,13 +71,13 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
26
71
  output byte-identical to a live editor's. With no callback rules the
27
72
  render path is unchanged, byte for byte. See "Custom nodes and marks" in
28
73
  the README.
29
- - **`Y::Lexical#node_types` / `Y::ProseMirror#node_types` schema
74
+ - **`Y::Lexical#node_types` / `Y::ProseMirror#node_types`: schema
30
75
  discovery.** Ask a real document which node types it holds and what they
31
76
  look like: counts, attribute names as stored, child types, whether text
32
77
  runs appear, and whether a builtin or one of your rules already handles
33
78
  each ("handled" nil marks what still needs a rule). Editors store names
34
79
  you'd never guess; this is how you find them.
35
- - `Y::RenderRules.escape_text` / `escape_attr` the exact escaping the
80
+ - `Y::RenderRules.escape_text` / `escape_attr`: the exact escaping the
36
81
  native renderers use, for blocks that build markup from stored values
37
82
  (ERB's `html_escape` also rewrites apostrophes, which breaks byte parity
38
83
  with editor output).
@@ -42,15 +87,15 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
42
87
  - **Lexical rendering is now two classes: `Y::Lexical` (core Lexical) and
43
88
  `Y::Lexxy` (core plus the Lexxy schema as render rules).** Stock Lexical
44
89
  has no canonical serializer, so the editor-specific class carries the
45
- editor's name `Y::Lexxy.new(doc).to_html` is the byte-parity call for
90
+ editor's name; `Y::Lexxy.new(doc).to_html` is the byte-parity call for
46
91
  Lexxy/Rails apps, and `Y::Lexical` is the base any other Lexical editor
47
92
  extends with its own rules. The
48
- native side renders core structure paragraphs, headings, quotes, code,
93
+ native side renders core structure (paragraphs, headings, quotes, code,
49
94
  lists, tables, links, the full text-format model. Lexxy's own node types
50
95
  (attachments, galleries, `early_escape_code`, `horizontal_divider`) and its
51
96
  decorations of core nodes (the table figure wrapper, header-cell styling,
52
97
  the nested-list-item class) are rules applied beneath the app's, on the
53
- same extension API the gem's Lexxy support is the API's first consumer.
98
+ same extension API; the gem's Lexxy support is the API's first consumer.
54
99
  Output is unchanged: the fixture tests still hold `to_html` byte-identical
55
100
  to a live editor's serialized value, now through the extension path. An
56
101
  unknown Lexical container also degrades better: its block children render
@@ -59,8 +104,8 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
59
104
  ProseMirror) and `Y::Tiptap` (core plus Tiptap's extension nodes as render
60
105
  rules).** `Y::Tiptap.new(doc).to_html` is the byte-parity call for Tiptap
61
106
  apps. The native side renders prosemirror-schema-basic plus the
62
- prosemirror-tables family; Tiptap's extension nodes task lists, mentions,
63
- the details family are `Y::Tiptap::NODES` rules. Marks stay native in the
107
+ prosemirror-tables family; Tiptap's extension nodes (task lists, mentions,
108
+ the details family) are `Y::Tiptap::NODES` rules. Marks stay native in the
64
109
  base class: mark rendering (nesting order, `textStyle` CSS, `code`
65
110
  exclusivity) runs through text-run machinery node rules don't reach, so
66
111
  `Y::ProseMirror` still renders Tiptap's full mark set and `rules.mark`
@@ -71,7 +116,7 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
71
116
 
72
117
  ### Added
73
118
 
74
- - **`Y::Lexical` render Lexical/Lexxy documents to HTML.**
119
+ - **`Y::Lexical`: render Lexical/Lexxy documents to HTML.**
75
120
  `Y::Lexical.new(doc).to_html` turns a Lexxy document into HTML on the server,
76
121
  with no Node process or headless editor. The output is identical to the HTML
77
122
  a `lexxy-editor` submits to Rails; the tests check it byte-for-byte against a
@@ -89,12 +134,12 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
89
134
 
90
135
  ### Added
91
136
 
92
- - **`Y::ProseMirror` render ProseMirror/Tiptap documents to HTML.**
137
+ - **`Y::ProseMirror`: render ProseMirror/Tiptap documents to HTML.**
93
138
  `Y::ProseMirror.new(doc).to_html` turns a Tiptap document into HTML on the
94
139
  server, with no Node process or headless editor. The output matches Tiptap's
95
140
  own `getHTML()`; the tests check it byte-for-byte against a document captured
96
141
  from a real editor. It follows `ueberdosis/tiptap-php` and reads both name
97
- styles editors use Tiptap's `bulletList`/`bold` and prosemirror-schema-basic's
142
+ styles editors use: Tiptap's `bulletList`/`bold` and prosemirror-schema-basic's
98
143
  `bullet_list`/`strong`. Covers paragraphs, headings, blockquotes,
99
144
  bullet/ordered/task lists, code blocks, links, images, mentions, details,
100
145
  hard breaks, horizontal rules, tables, text styles (color, font family), and
@@ -110,8 +155,8 @@ Fixes from a full source review.
110
155
 
111
156
  - **`Doc#update_ready?` is now exact.** It previously checked only the
112
157
  per-client clock lower bound, but yrs's real integration gate also requires
113
- every block referenced by an item's origin / right-origin / parent which
114
- routinely belong to *other* clients and post-Skip blocks in a merged update
158
+ every block referenced by an item's origin / right-origin / parent (which
159
+ routinely belong to *other* clients) and post-Skip blocks in a merged update
115
160
  sit above the lower bound. An update could pass the clock check yet park as
116
161
  pending; downstream, `update_advances?` then misread the parked update as an
117
162
  already-applied retry (pending doesn't move a state vector) and the sync
@@ -135,7 +180,7 @@ Fixes from a full source review.
135
180
  ### Changed
136
181
 
137
182
  - `update_advances?` skips its full-document probe when the update carries
138
- blocks beyond the doc's state vector (a novel update trivially advances)
183
+ blocks beyond the doc's state vector (a novel update trivially advances);
139
184
  the common case no longer pays O(doc) per frame.
140
185
  - The gem no longer packages the `yrby-decoder` gem's files (they ship in that
141
186
  gem; the duplicate copy could shadow a newer standalone release), and now
@@ -146,7 +191,7 @@ Fixes from a full source review.
146
191
  ### Fixed
147
192
 
148
193
  - **Sync no longer serves un-integrable pending structs.** When a doc holds a
149
- *pending* struct (a gappy update whose causally-prior update is missing e.g.
194
+ *pending* struct (a gappy update whose causally-prior update is missing, e.g.
150
195
  legacy data recorded before the `update_ready?` gate existed), its integrated
151
196
  state vector is empty but `encode_state_as_update` merges the pending bytes back
152
197
  in. Answering a peer's `SyncStep1` with that state handed the peer content it
@@ -154,18 +199,18 @@ Fixes from a full source review.
154
199
  non-empty-content mismatch drove endless resync traffic (observed as a browser
155
200
  re-sending frames several times a second). `handle_sync_message` now answers
156
201
  `SyncStep1` with **integrated-only** state, so a server never serves a struct it
157
- can't integrate itself. Neutralizes existing poisoned server state on deploy
202
+ can't integrate itself. Neutralizes existing poisoned server state on deploy;
158
203
  no migration needed. The server's own pending is untouched and still heals if
159
204
  the missing dependency later arrives (only then does the content become
160
205
  visible in sync). Live delta relay (`Update` frames) is unchanged.
161
206
 
162
207
  ### Added
163
208
 
164
- - `Doc#pending?` true if the doc holds un-integrable pending structs or a
209
+ - `Doc#pending?`: true if the doc holds un-integrable pending structs or a
165
210
  pending delete set (content waiting on a missing causally-prior update).
166
- - `Doc#compacted_state_update` like `encode_state_as_update` (full state) but
211
+ - `Doc#compacted_state_update`: like `encode_state_as_update` (full state) but
167
212
  **gap-free**: excludes pending structs/delete set. Use it when persisting or
168
- serving state other peers will apply. Non-destructive the doc keeps its
213
+ serving state other peers will apply. Non-destructive; the doc keeps its
169
214
  pending (so it can still heal), and `encode_state_as_update` stays lossless for
170
215
  raw-update recovery.
171
216
 
@@ -183,13 +228,13 @@ Fixes from a full source review.
183
228
  before vs. after a trial apply on an isolated probe: a genuinely new deletion
184
229
  changes it (`true`); an already-applied retry re-encodes identically (`false`).
185
230
  Insert/format-only updates keep the cheaper state-vector path, so only
186
- delete-bearing frames a minority pay for the exact comparison. The exactly-
231
+ delete-bearing frames (a minority) pay for the exact comparison. The exactly-
187
232
  once guarantee is unchanged in the safe direction: a real deletion is never
188
233
  dropped.
189
234
 
190
235
  This lets `yrby-actioncable` (and any caller gating `on_change` on
191
- `update_advances?`) settle a duplicate pure-delete frame as `:applied` acked,
192
- but not stored or relayed so apps no longer need an app-level
236
+ `update_advances?`) settle a duplicate pure-delete frame as `:applied` (acked,
237
+ but not stored or relayed), so apps no longer need an app-level
193
238
  encode-and-compare guard around their durable writes.
194
239
 
195
240
  ## [0.2.2] - 2026-06-30
@@ -200,7 +245,7 @@ Fixes from a full source review.
200
245
  embeds child blocks (list items, table cells, nested lists) as `Y.XmlText`
201
246
  embeds that `get_string` silently drops, so lists and tables previously came
202
247
  back empty. `read_xml` now walks the embeds: text runs build a line, inline
203
- children (links) join it, and nested block children flush and recurse so a
248
+ children (links) join it, and nested block children flush and recurse, so a
204
249
  document with headings, formatted text, links, bullet/numbered/check/nested
205
250
  lists, blockquotes, code blocks and tables extracts every piece of text.
206
251
  Lexical decorator elements (horizontal rule, image) are skipped instead of
@@ -211,17 +256,17 @@ Fixes from a full source review.
211
256
 
212
257
  ### Changed
213
258
  - **Internal:** renamed the native extension crate `y_ruby` → `yrby` (now loads
214
- from `lib/y/yrby.bundle`). No public API change `require "y"` and `Y::Doc`
259
+ from `lib/y/yrby.bundle`). No public API change: `require "y"` and `Y::Doc`
215
260
  are unchanged.
216
261
 
217
262
  ## [0.2.0] - 2026-06-28
218
263
 
219
264
  First release. The public Ruby interface is the top-level module **`Y`**
220
- (`Y::Doc`, `Y::Error`, `Y::VERSION`), loaded with `require "y"` mirroring the
265
+ (`Y::Doc`, `Y::Error`, `Y::VERSION`), loaded with `require "y"`, mirroring the
221
266
  `y-rb` gem's `Y::Doc` interface.
222
267
 
223
268
  ### Added
224
- - Native `Doc#read_text` and `Doc#read_map` readers reconstruct plain text and
269
+ - Native `Doc#read_text` and `Doc#read_map` readers: reconstruct plain text and
225
270
  a JSON map from the stored CRDT state in-process, server-side, with no Node or
226
271
  subprocess.
227
272
 
data/README.md CHANGED
@@ -2,48 +2,85 @@
2
2
 
3
3
  [![CI](https://github.com/jpcamara/yrby/actions/workflows/ci.yml/badge.svg)](https://github.com/jpcamara/yrby/actions/workflows/ci.yml)
4
4
 
5
- Collaborative editing for Rails, backed by [y-crdt](https://github.com/y-crdt/y-crdt)
6
- (the Rust library behind Y.js). Your Rails server speaks the y-websocket sync
7
- protocol directly, so there's no separate Node process hosting the Y.js
8
- documents.
5
+ yrby (pronounced "yer-bee") makes Rails a real Yjs backend. It binds
6
+ [y-crdt](https://github.com/y-crdt/y-crdt), the Rust engine behind Y.js, into
7
+ Ruby, and builds the rest of the stack around it: a sync server for Action
8
+ Cable and AnyCable, a browser provider, and server-side reading and rendering
9
+ of the documents. Real-time collaboration in a Rails app with no Node process
10
+ anywhere in the path.
11
+
12
+ ![Two people typing on separate lines of the same document, each keystroke synced through a Rails server, seen from a third browser with labeled carets](docs/images/collab.gif)
13
+
14
+ On the server, `yrby-rails` implements the full y-websocket protocol
15
+ (document sync plus presence) as a channel concern. Its delivery contract is
16
+ stricter than the usual Yjs servers: every update is ack-tracked and durably
17
+ recorded before it is acknowledged or broadcast to anyone. Replaying your
18
+ store always rebuilds the document, across any number of processes.
19
+ ([Delivery guarantees](#delivery-guarantees))
20
+
21
+ In the browser, `yrby-client`'s `ActionCableProvider` connects anything that
22
+ speaks Yjs. The demo app runs four rich text editors, and CI drives each one
23
+ in real Chrome: Tiptap, [Lexxy](https://www.npmjs.com/package/lexxy-realtime),
24
+ Rhino Editor, and CodeMirror. The same channel also syncs Yjs shapes with no
25
+ editor at all: a whiteboard on a `Y.Map`, a kanban board on a `Y.Array`, a
26
+ co-filled form. ([Editors](#editors))
27
+
28
+ In Ruby, the documents are readable without a browser. `Doc#read_text` and
29
+ `Doc#read_map` reconstruct contents for search, validation, and exports.
30
+ `Y::Tiptap` and `Y::Lexxy` render a document to HTML byte-identical to the
31
+ editor's own serializer, take rules for your app's custom nodes, and drop
32
+ straight into ActionText. ([Rendering to HTML](#rendering-to-html))
33
+
34
+ Underneath, the core is built for a production Rails deployment. A `Doc` is
35
+ thread-safe across Puma and ActionCable threads. Native CRDT work runs with
36
+ the GVL released, so it parallelizes on MRI. Incoming frames are validated
37
+ before anything processes them, and multi-process and AnyCable setups are
38
+ tested end to end. ([Thread Safety](#thread-safety))
39
+
40
+ The whole server side of a collaborative document is one channel:
9
41
 
10
42
  ```ruby
11
43
  class DocumentChannel < ApplicationCable::Channel
12
- include Y::ActionCable::Sync
44
+ include Y::ActionCable
13
45
 
14
- on_load { |key| MyStore.load(key) }
15
- on_change { |key, update| MyStore.append(key, update) }
46
+ on_load { |key| Y::Document.load_state(key) }
47
+ on_change { |key, update| Y::Document.append(key, update) }
16
48
 
17
49
  def subscribed = sync_subscribed(params[:id])
18
50
  def receive(data) = sync_receive(data, params[:id])
19
51
  end
20
52
  ```
21
53
 
22
- On the browser, use the `ActionCableProvider` from the
23
- [`yrby-client`](https://www.npmjs.com/package/yrby-client) npm package.
24
- Integrates with any editor that includes Y.js support, such as Tiptap, ProseMirror
25
- and [Lexxy](https://www.npmjs.com/package/lexxy-realtime).
26
-
27
- ## Usage
28
-
29
- Install the gem and npm package:
54
+ Install the gem and the npm package:
30
55
 
31
56
  ```
32
- gem install yrby-actioncable # depends on yrby
57
+ gem install yrby-rails # depends on yrby
33
58
  npm install yrby-client
34
59
  ```
35
60
 
36
- ## What you get
37
-
38
- - A thread-safe Ruby `Doc` you can share across Ruby threads/fibers, and native CRDT work
39
- runs with the GVL released.
40
- - The y-websocket protocol (document sync plus awareness/presence) as a
41
- one-include ActionCable concern.
42
- - Authoritative record-before-distribute semantics: each document change can be
43
- recorded durably before it goes out to anyone.
44
- - Optional server-side reads: `Doc#read_text` and `Doc#read_map` reconstruct a
45
- document's contents in Ruby - no Node process - for search, exports, validation,
46
- or server-side rendering.
61
+ ## Contents
62
+
63
+ - [Scope](#scope)
64
+ - [Durability and delivery](#durability-and-delivery)
65
+ - [What about yrb?](#what-about-yrb)
66
+ - [Testing](#testing)
67
+ - [Install](#install)
68
+ - [Docs](#docs)
69
+ - [Editors](#editors)
70
+ - [Usage](#usage)
71
+ - [Doc (Low-Level Document Sync)](#doc-low-level-document-sync)
72
+ - [Reading document contents](#reading-document-contents)
73
+ - [Pending structs and gap-free state](#pending-structs-and-gap-free-state)
74
+ - [Rendering to HTML](#rendering-to-html)
75
+ - [Protocol codec (module functions)](#protocol-codec-module-functions)
76
+ - [ActionCable Integration](#actioncable-integration)
77
+ - [Thread Safety](#thread-safety)
78
+ - [Parallelism (GVL release)](#parallelism-gvl-release)
79
+ - [Message Type Constants](#message-type-constants)
80
+ - [Sync Flow](#sync-flow)
81
+ - [Development](#development)
82
+ - [License](#license)
83
+ - [Acknowledgments](#acknowledgments)
47
84
 
48
85
  ## Scope
49
86
 
@@ -59,19 +96,18 @@ and `Doc#read_map` reconstruct it server-side, in Ruby.
59
96
  The surface is intentionally small, but the focus is durability, resiliency, delivery
60
97
  guarantees, correctness, and thread safety.
61
98
 
62
- Towards that goal, `yrby` adds capabilities that stand out even in the Yjs ecosystem:
99
+ Towards that goal, `yrby` adds opinionated defaults on top of normal Yjs syncing:
63
100
 
64
- - Built-in update acknowledgement: the `ActionCableProvider` in `yrby-client` will continue to
65
- send updates until an ack is received from the server. [`yrby-actioncable`](https://rubygems.org/gems/yrby-actioncable)
66
- only sends an ack when applying an update is successful. The goal is at-least-once delivery,
67
- and because CRDTs are idempotent a duplicate update is effectively a no-op.
68
- - Gap detection in document updates: before applying an update and sending an ack to the client,
69
- `yrby` checks whether the update results in any causal gap. Ie, an update comes through
70
- which depends on a previous update that is not yet present in the document. This can result in
71
- a document stuck with "pending" updates, which will _never_ apply if the missing update is not sent.
72
- To avoid this, `yrby` does not apply the update, and starts a new y-protocol sync with the client.
73
- That will cause the client to synchronize its document with the server, sending through any updates
74
- that may have been missed
101
+ - Built-in update acknowledgement: the `ActionCableProvider` in `yrby-client` keeps
102
+ sending an update until the server acks it, and [`yrby-rails`](https://rubygems.org/gems/yrby-rails)
103
+ only acks once the update is durably recorded. That gives you at-least-once
104
+ delivery, and because CRDT updates are idempotent a duplicate is a no-op.
105
+ - Gap awareness: an update can arrive before another update it depends on (a
106
+ "causal gap"). `yrby` records and acks it like any other, and the document
107
+ heals on its own once the missing update arrives; its sender keeps
108
+ retransmitting it until it is acked. `Doc#pending?` and the `on_gap` hook
109
+ tell you when a document is waiting on a missing update.
110
+ ([Causal gaps](#causal-gaps))
75
111
 
76
112
  ## What about [yrb](https://github.com/y-crdt/yrb)?
77
113
 
@@ -101,8 +137,9 @@ Issues and PRs are welcome.
101
137
  # Core CRDT + protocol primitives:
102
138
  gem "yrby"
103
139
 
104
- # For the Rails/ActionCable server concern (Y::ActionCable::Sync):
105
- gem "yrby-actioncable"
140
+ # For the Rails side (the sync channel, document models, the generator).
141
+ # Formerly yrby-actioncable; that name stops at 0.3.1.
142
+ gem "yrby-rails"
106
143
  ```
107
144
 
108
145
  Requires Ruby 3.4 or newer. The release workflow builds precompiled gems for
@@ -144,8 +181,8 @@ editor's own serializer. Each page is a working integration to copy from:
144
181
  | [Rhino Editor](https://github.com/KonnorRogers/rhino-editor) (Tiptap 3) | `@tiptap/extension-collaboration` + `-caret` | [`rhino.js`](examples/actioncable-demo/frontend/src/rhino.js) |
145
182
  | [CodeMirror 6](https://codemirror.net) | `y-codemirror.next` | [`codemirror.js`](examples/actioncable-demo/frontend/src/codemirror.js) |
146
183
 
147
- The demo also syncs plain Yjs shapes with no editor at all a whiteboard
148
- on a `Y.Map`, a kanban board on a `Y.Array`, a co-filled form over the
184
+ The demo also syncs plain Yjs shapes with no editor at all (a whiteboard
185
+ on a `Y.Map`, a kanban board on a `Y.Array`, a co-filled form) over the
149
186
  same channel. The demo README's "Using this in your own app" section has
150
187
  the integration recipe, and its `NoteMaterializer` shows how to render a
151
188
  document to ActionText server-side with `Y::Tiptap` or `Y::Lexxy`.
@@ -170,17 +207,19 @@ doc.compacted_state_update # => full update, gap-free (excludes pending)
170
207
  # Applying updates
171
208
  doc.apply_update(update_bytes) # apply raw V1 update
172
209
  doc.pending? # => true if holding un-integrable pending structs
210
+ doc.update_ready?(update) # => true if update would integrate cleanly (no gap)
211
+ doc.update_advances?(update) # => true if update moves integrated state forward
173
212
 
174
213
  # Sync protocol
175
214
  doc.sync_step1 # => SyncStep1 message (this doc's state vector)
176
215
  doc.handle_sync_message(data) # => [msg_type, sync_type, response]; answers a
177
- # peer's SyncStep1 with an integrated-only
178
- # SyncStep2 (never serves pending structs)
216
+ # peer's SyncStep1 with full state (lossless,
217
+ # pending included, like Y.js)
179
218
  ```
180
219
 
181
220
  ### Reading document contents
182
221
 
183
- Reconstruct a document server-side search, exports, emails, SSR with no
222
+ Reconstruct a document server-side (search, exports, emails, SSR) with no
184
223
  Node process:
185
224
 
186
225
  ```ruby
@@ -196,16 +235,18 @@ update), yrs parks it as a **pending** struct: the integrated state vector stays
196
235
  empty, but the pending block is held as a recovery buffer and heals if the
197
236
  missing dependency later arrives. `Doc#pending?` reports this.
198
237
 
199
- Pending structs are *not* document state, so they must not cross the sync
200
- boundary a peer that receives one can't integrate it and gets stuck. Two
201
- guarantees keep serving safe:
238
+ Pending structs travel like any other state. `handle_sync_message` answers
239
+ `SyncStep1` with the doc's full state, pending included, just like Y.js's
240
+ `encodeStateAsUpdate`: a peer parks the pending struct the same way this doc
241
+ did and heals it the same way. The one place pending must not go is a
242
+ compacted snapshot:
202
243
 
203
- - `handle_sync_message` answers `SyncStep1` with **integrated-only** state, so a
204
- server never serves a struct it can't integrate itself (this is automatic).
205
- - `Doc#compacted_state_update` gives you the same gap-free full-state update for
206
- when you persist or hand off state yourself. It's non-destructive (the doc
207
- keeps its pending), while `encode_state_as_update` stays lossless so you can
208
- still preserve the raw pending bytes for recovery.
244
+ - `Doc#compacted_state_update` returns a gap-free full-state update for
245
+ compaction. Folding a log into one blob would otherwise freeze an
246
+ un-integrable struct into the base state forever. It's non-destructive: the
247
+ doc keeps its pending.
248
+ - `encode_state_as_update` stays lossless, so persistence and serving keep
249
+ the raw pending bytes and the gap can still heal.
209
250
 
210
251
  ### Rendering to HTML
211
252
 
@@ -228,7 +269,7 @@ tiptap.to_html("content") # or another XML root
228
269
  The output matches Tiptap's own `getHTML()`, checked byte-for-byte in the tests
229
270
  against a document captured from a real editor. It follows
230
271
  [`tiptap-php`](https://github.com/ueberdosis/tiptap-php) and reads both name
231
- styles editors use Tiptap's `bulletList`/`bold` and prosemirror-schema-basic's
272
+ styles editors use: Tiptap's `bulletList`/`bold` and prosemirror-schema-basic's
232
273
  `bullet_list`/`strong`.
233
274
 
234
275
  It covers paragraphs, headings, blockquotes, bullet/ordered/task lists, code
@@ -238,8 +279,8 @@ as semantic `<table><tbody>`, without the column-width styling Tiptap's editor
238
279
  view adds.
239
280
 
240
281
  The support is layered like the Lexical side: `Y::ProseMirror` covers core
241
- ProseMirror natively prosemirror-schema-basic plus the prosemirror-tables
242
- family and Tiptap's extension nodes (task lists, mentions, the details
282
+ ProseMirror natively (prosemirror-schema-basic plus the prosemirror-tables
283
+ family) and Tiptap's extension nodes (task lists, mentions, the details
243
284
  family) are `Y::Tiptap`'s rule set (`Y::Tiptap::NODES`), built on the
244
285
  extension API below. Marks stay in the base: mark rendering (nesting order,
245
286
  `textStyle` CSS, `code` exclusivity) runs through native text-run machinery
@@ -256,8 +297,8 @@ lexxy.to_html("notepad") # or another XML root
256
297
 
257
298
  The HTML is identical to what a `lexxy-editor` submits to Rails (its `value`).
258
299
  The tests check this byte-for-byte against a document captured from a real
259
- editor. Stock Lexical has no canonical serializer every editor configures
260
- its own so the editor-specific class carries the editor's name, and
300
+ editor. Stock Lexical has no canonical serializer (every editor configures
301
+ its own), so the editor-specific class carries the editor's name, and
261
302
  `Y::Lexical` is the core-Lexical base: paragraphs, headings, quotes, code,
262
303
  lists, tables, links, and the full text-format model, for any other Lexical
263
304
  editor to extend with rules.
@@ -270,14 +311,14 @@ mentions both emit `<action-text-attachment>` elements that ActionText can
270
311
  re-render).
271
312
 
272
313
  Internally that support is layered: `Y::Lexical` covers core Lexical
273
- structure natively, and everything Lexxy adds its node types (attachments,
314
+ structure natively, and everything Lexxy adds, its node types (attachments,
274
315
  galleries) and its decorations of core nodes (the table wrapper, header-cell
275
- styling, nested-list classes) is `Y::Lexxy`'s rule set
316
+ styling, nested-list classes), is `Y::Lexxy`'s rule set
276
317
  (`Y::Lexxy::NODES`), built on the extension API below. The gem's own Lexxy
277
318
  support is the API's first consumer: an app rule for one of those types
278
319
  simply replaces it.
279
320
 
280
- In both renderers an unknown node keeps its content text and nested blocks
321
+ In both renderers an unknown node keeps its content: text and nested blocks
281
322
  fall back to readable markup rather than disappearing.
282
323
 
283
324
  #### Custom nodes and marks
@@ -287,7 +328,7 @@ their own node types. Both renderers take rules for them. A rule is checked
287
328
  before the built-in schema, so it can add a node type or replace how a
288
329
  built-in renders.
289
330
 
290
- Rules register in a block one `rules.node` call per type. A declarative
331
+ Rules register in a block, one `rules.node` call per type. A declarative
291
332
  rule is markup as data, rendered natively:
292
333
 
293
334
  ```ruby
@@ -301,14 +342,14 @@ end
301
342
  `tag` names the element. `attrs` values are templates: a string is a literal,
302
343
  a symbol reads that attribute off the node, an array concatenates both kinds;
303
344
  an attribute that resolves empty is left out. `text` (same template form)
304
- emits literal text content. `contains` declares what lives inside the node `:inline` (formatted text,
305
- the default), `:blocks` (child block nodes a container), or `:none` (a
345
+ emits literal text content. `contains` declares what lives inside the node: `:inline` (formatted text,
346
+ the default), `:blocks` (child block nodes, a container), or `:none` (a
306
347
  leaf). `void: true` skips the closing tag.
307
348
 
308
349
  You don't have to guess any of those names or shapes. Editors store types
309
350
  and attributes under names you'd never predict (Rhino's strike mark is
310
351
  `rhino-strike`; Lexical prefixes its own props `__`), so ask a real
311
- document instead make one in your editor using your custom node, then:
352
+ document instead: make one in your editor using your custom node, then:
312
353
 
313
354
  ```ruby
314
355
  Y::Tiptap.new(doc).node_types
@@ -333,25 +374,25 @@ lexical = Y::Lexical.new(doc) do |rules|
333
374
  end
334
375
  ```
335
376
 
336
- The block gets the node's type, its stored attributes, `node.content` the
337
- children, already rendered to HTML and `node.child_types`, the node's
377
+ The block gets the node's type, its stored attributes, `node.content` (the
378
+ children, already rendered to HTML), and `node.child_types`, the node's
338
379
  element/block children by type, in document order. `child_types` answers the
339
380
  structural questions attributes can't: how many images a gallery holds, or
340
381
  whether a list item carries a nested list. Whatever the block returns is
341
382
  spliced into the output as-is: it's trusted HTML, so escape any values you
342
- interpolate. To set the content mode for a callback, give the node both
383
+ interpolate. To set the content mode for a callback, give the node both:
343
384
  `rules.node "embed", contains: :blocks do |node| ... end`.
344
385
 
345
386
  Callbacks never run while the document is locked. The render finishes first
346
387
  (inside one read transaction, GVL released), then the blocks run and their
347
- output is spliced in so a callback can safely read or even write the same
388
+ output is spliced in, so a callback can safely read or even write the same
348
389
  doc. With no callback rules, `to_html` skips the splicing entirely.
349
390
 
350
391
  Blocks are the escape hatch for everything the declarative form can't say,
351
392
  and they're proven sufficient: `Y::Lexxy` and `Y::Tiptap` are themselves
352
- built on this API (`lib/y/lexxy.rb`, `lib/y/tiptap.rb`) simple nodes as
393
+ built on this API (`lib/y/lexxy.rb`, `lib/y/tiptap.rb`): simple nodes as
353
394
  declarative hashes, everything with logic as plain methods mapped by node
354
- type (a `Method` responds to `call` like any lambda) and the fixture tests
395
+ type (a `Method` responds to `call` like any lambda), and the fixture tests
355
396
  hold their output byte-identical to a live editor's.
356
397
 
357
398
  The ProseMirror side also takes custom marks:
@@ -368,7 +409,7 @@ for a built-in mark name (`"bold"`) replaces its built-in tag.
368
409
 
369
410
  ##### Worked examples
370
411
 
371
- A video-embed node from an app's Tiptap extension a type the pinned schema
412
+ A video-embed node from an app's Tiptap extension, a type the pinned schema
372
413
  has never heard of:
373
414
 
374
415
  ```ruby
@@ -395,7 +436,7 @@ tiptap = Y::Tiptap.new(doc) do |rules|
395
436
  end
396
437
  ```
397
438
 
398
- Overriding a shipped rule rendering Lexxy uploads as real image markup
439
+ Overriding a shipped rule: rendering Lexxy uploads as real image markup
399
440
  instead of the `<action-text-attachment>` elements ActionText re-renders:
400
441
 
401
442
  ```ruby
@@ -411,7 +452,7 @@ lexxy = Y::Lexxy.new(doc) do |rules|
411
452
  end
412
453
  ```
413
454
 
414
- Markup that depends on structure `node.child_types` lists the node's
455
+ Markup that depends on structure: `node.child_types` lists the node's
415
456
  element/block children in document order, so a layout container can size
416
457
  itself by its column count while the columns themselves stay declarative:
417
458
 
@@ -424,7 +465,7 @@ tiptap = Y::Tiptap.new(doc) do |rules|
424
465
  end
425
466
  ```
426
467
 
427
- Content-aware overrides dropping the empty paragraphs an editor keeps
468
+ Content-aware overrides: dropping the empty paragraphs an editor keeps
428
469
  around the cursor, since `node.content` arrives already rendered:
429
470
 
430
471
  ```ruby
@@ -435,10 +476,10 @@ lexical = Y::Lexical.new(doc) do |rules|
435
476
  end
436
477
  ```
437
478
 
438
- For a larger reference, the gem's own editor schemas ship this way see
479
+ For a larger reference, the gem's own editor schemas ship this way; see
439
480
  `Y::Lexxy::NODES` in `lib/y/lexxy.rb` (declarative hashes for the simple
440
- nodes, a plain method per node that needs logic galleries, list items,
441
- header cells, both attachment types mapped with `method(:name)`) and
481
+ nodes, a plain method per node that needs logic (galleries, list items,
482
+ header cells, both attachment types) mapped with `method(:name)`) and
442
483
  `Y::Tiptap::NODES` in `lib/y/tiptap.rb` (task lists, mentions, the details
443
484
  family).
444
485
 
@@ -446,7 +487,7 @@ family).
446
487
 
447
488
  Classifying and unwrapping wire frames is stateless, so it's exposed as
448
489
  `Y` module functions rather than a class. The server never holds presence
449
- or document state to route a frame presence lives in the browser clients, and
490
+ or document state to route a frame; presence lives in the browser clients, and
450
491
  the server only relays awareness frames opaquely.
451
492
 
452
493
  ```ruby
@@ -457,31 +498,84 @@ Y.wrap_update(update_bytes) # => wrap a raw doc update as a sync Update frame
457
498
 
458
499
  ### ActionCable Integration
459
500
 
460
- `Y::ActionCable::Sync` (from the `yrby-actioncable` gem) is a channel
461
- concern that implements the full y-websocket protocol (document sync +
462
- awareness/presence) over ActionCable:
501
+ In a Rails app, one generator creates the channel and the migration:
502
+
503
+ ```bash
504
+ bin/rails generate yrby:install
505
+ bin/rails db:migrate
506
+ ```
507
+
508
+ The models ship in the gem, the way Action Text owns
509
+ `ActionText::RichText`:
510
+
511
+ - **`Y::Document`**: one row per document, addressed two ways: by `key`
512
+ (what a channel addresses; one opaque, unique string, sometimes
513
+ app-supplied, never parsed) and, optionally, by polymorphic `record` +
514
+ `name` (which model attribute it backs; `name` is the attribute name,
515
+ `"body"`; one document per attribute per record, the
516
+ ActionText::RichText scheme). Key-only documents leave the binding nil.
517
+ Either side can arrive first: `Y::Document.for(record, name)` finds or
518
+ creates the binding, derives a readable key (`post/1/body`), and adopts
519
+ a key-only row already holding that key, so a channel writing first and
520
+ a binding created later converge on one document. The row also holds
521
+ the merged `state` snapshot, CRDT state only; derived data (rendered
522
+ HTML, search text) is the application's job, typically in the channel's
523
+ on_change. `.load_state(key)` / `.append(key, update)` are the store
524
+ calls the generated channel uses.
525
+ - **`Y::DocumentUpdate`**: the uncompacted tail, one delta per row,
526
+ compacted into `state` and deleted once the tail reaches `compact_every`
527
+ (default 64). Loading reads the snapshot plus the current tail; an
528
+ empty tail returns `state` directly. Compaction serializes on a
529
+ per-document row lock and skips causally-gapped rows; they're
530
+ quarantined until they heal rather than compacted into state or
531
+ deleted. Destroying a document deletes its updates with it.
532
+
533
+ Encrypted storage: `Y::EncryptedDocument` stores `state` and update
534
+ payloads through Active Record encryption on the same tables, the way
535
+ `ActionText::EncryptedRichText` does. Point the channel's
536
+ `on_load`/`on_change` at it instead and configure your app's encryption
537
+ keys. Use one access path per document: rows written encrypted read back
538
+ as ciphertext through the plain classes.
539
+
540
+ The migration creates `y_documents` and `y_document_updates`. To rename
541
+ them, edit the generated migration and point `Y::Document.table_name` /
542
+ `Y::DocumentUpdate.table_name` at the new names in an initializer.
543
+
544
+ Storage is swappable: the channel only needs `on_load` and `on_change`
545
+ answered, and they can point at anything.
546
+
547
+ `include Y::ActionCable` (from the `yrby-rails` gem) is the channel
548
+ integration: the y-websocket protocol (document sync +
549
+ awareness/presence) over ActionCable.
463
550
 
464
551
  ```ruby
465
552
  # app/channels/document_channel.rb
466
553
  class DocumentChannel < ApplicationCable::Channel
467
- include Y::ActionCable::Sync
554
+ include Y::ActionCable
468
555
 
469
- on_load { |key| MyStore.load(key) } # source of truth
470
- on_change { |key, update| MyStore.append(key, update) } # durable record
556
+ on_load { |key| Y::Document.load_state(key) } # rebuild from storage
557
+ on_change { |key, update| Y::Document.append(key, update) } # record, then broadcast
471
558
 
472
559
  def subscribed
560
+ return reject unless authorized?(params[:id])
561
+
473
562
  sync_subscribed params[:id]
474
563
  end
475
564
 
476
565
  def receive(data)
477
566
  sync_receive(data, params[:id])
478
567
  end
568
+
569
+ private
570
+
571
+ # Everyone is denied until you wire this to your app's auth.
572
+ def authorized?(_document_key) = false
479
573
  end
480
574
  ```
481
575
 
482
576
  The concern is store-backed. A handshake is answered from `on_load`; document
483
- changes are checked against that durable state, recorded through `on_change`,
484
- then broadcast. Nothing authoritative is kept in ActionCable process memory, so
577
+ changes are recorded through `on_change`, then broadcast. Nothing
578
+ authoritative is kept in ActionCable process memory, so
485
579
  AnyCable RPC workers, Puma workers, and separate dynos can all handle messages
486
580
  for the same document as long as they share the same store and cable adapter.
487
581
 
@@ -499,16 +593,18 @@ no single client can relay garbage that breaks the others in a room.
499
593
 
500
594
  #### Delivery guarantees
501
595
 
502
- The contract is the same at every scale one process, or hundreds across many
596
+ The contract is the same at every scale: one process, or hundreds across many
503
597
  servers:
504
598
 
505
599
  - **The document always converges.** CRDT updates are commutative and
506
600
  idempotent, so out-of-order, duplicate, or concurrent delivery all converge to
507
601
  the same correct document. This needs no coordination and holds everywhere.
508
- - **The durable log never goes gappy.** An update is recorded only once its
509
- causal dependencies are already in the store (checked against `on_load`); a
510
- causally-incomplete update triggers a resync instead, so the log always
511
- rebuilds cleanly.
602
+ - **An acked update is durable, even one that arrived out of order.** An
603
+ update with a missing dependency is recorded and acked like any other, and
604
+ parks as pending in the document. That missing dependency is an update some
605
+ client still holds unacked, so that client keeps retransmitting it until
606
+ the server records it, and the gap closes. The ack loop is the guarantee.
607
+ See [Causal gaps](#causal-gaps).
512
608
  - **`on_change` is at-least-once, and the durable guarantee is that replaying the
513
609
  log reconstructs the document.** Every update triggers `on_change` before it's acked or
514
610
  broadcast (record-before-distribute). If exactly-once updates matter for you, **you
@@ -518,13 +614,13 @@ servers:
518
614
  There is no negative-ack: the client simply never receives the ack, keeps the
519
615
  update pending, and retransmits on its timer/reconnect. This is built for
520
616
  *transient* failures (the store is briefly down → a retry lands). A block that
521
- raises *deterministically* a validation that always fails for this edit
617
+ raises *deterministically* (a validation that always fails for this edit)
522
618
  will be retried forever, since nothing tells the client to stop. Enforce hard
523
619
  rejections before the edit reaches `on_change` (channel authorization in
524
620
  `subscribed`), not by raising inside it.
525
621
  - **An over-cap frame is dropped the same silent way.** A frame larger than
526
- `max_frame_bytes` (default 8 MiB) is dropped before decoding no ack, no
527
- broadcast to bound the work a client can force. For a genuine document
622
+ `max_frame_bytes` (default 8 MiB) is dropped before decoding (no ack, no
623
+ broadcast) to bound the work a client can force. For a genuine document
528
624
  update that means the same implicit rejection as above: unacked, retransmitted
529
625
  forever. Normal typing never approaches the cap, but a large paste, an embedded
530
626
  image, or a big initial `SyncStep2` can. The drop is logged (`warn` for
@@ -534,6 +630,85 @@ servers:
534
630
  genuinely-too-big content upstream rather than relying on the cap to reject it
535
631
  gracefully.
536
632
 
633
+ #### Causal gaps
634
+
635
+ Yjs updates can arrive out of order: an update can reach the server before
636
+ another update it depends on. yrby treats that as normal. The update is
637
+ recorded and acked like any other, parks as a pending struct in the document,
638
+ and integrates on its own the moment the missing dependency lands. The write
639
+ path never rebuilds the document; it appends, relays, and acks, so a gapped
640
+ update costs the same as any other.
641
+
642
+ Serving is lossless too, like any Yjs server. `handle_sync_message` serves
643
+ full state, pending included, so a peer parks the same pending struct and
644
+ heals it the same way. Healing needs no special machinery: the missing
645
+ dependency is an update its sender still holds unacked, and at-least-once
646
+ retransmission delivers it. Only compaction excludes pending
647
+ (`compacted_state_update`), because folding a log must not freeze an
648
+ un-integrable struct into the base state.
649
+
650
+ The bundled `Y::Document` store handles all of this. If you write your own
651
+ store, keep two things in mind:
652
+
653
+ **1. Load losslessly, and tolerate duplicates.** `on_load` should return
654
+ state that preserves pending: `encode_state_as_update`, or a replay of the
655
+ raw append log. Don't compact with `compacted_state_update` while
656
+ `doc.pending?`; that strips the pending struct and the acked edit inside it.
657
+ (`Y::Document` quarantines pending rows for exactly this reason.) A lost ack
658
+ also means a client resends an update the store already has. Replay converges
659
+ anyway, because CRDT apply is idempotent, so deduping is optional. If log
660
+ size matters, dedup by content hash:
661
+
662
+ ```ruby
663
+ class DocumentStore
664
+ # append tolerates duplicates: a re-delivered update upserts to a no-op.
665
+ def append(key, update)
666
+ Revision.upsert({ doc_key: key, update_hash: Digest::SHA256.hexdigest(update), update: update },
667
+ unique_by: %i[doc_key update_hash])
668
+ end
669
+
670
+ # load is lossless: replay the raw log so a pending struct is preserved and
671
+ # heals when its dependency arrives.
672
+ def load(key)
673
+ updates = Revision.where(doc_key: key).order(:id).pluck(:update)
674
+ return nil if updates.empty?
675
+
676
+ doc = Y::Doc.new
677
+ updates.each { |u| doc.apply_update(u) }
678
+ doc.encode_state_as_update # lossless: keeps pending
679
+ end
680
+
681
+ # optional compaction: only when there is no open gap, or you would drop it.
682
+ def compact(key)
683
+ doc = Y::Doc.new
684
+ Revision.where(doc_key: key).order(:id).pluck(:update).each { |u| doc.apply_update(u) }
685
+ return if doc.pending? # a gap is open; compacting now would drop it
686
+ # ... replace the log with a single revision holding doc.compacted_state_update ...
687
+ end
688
+ end
689
+ ```
690
+
691
+ **2. Watch for gaps that never heal.** An open gap is quiet: the edit sits
692
+ as pending, invisible in the document, until its dependency arrives. Normally
693
+ that resolves itself. The sender retransmits the missing update until it is
694
+ acked, and every join or reconnect handshake has the client send everything
695
+ the server hasn't integrated, so any client holding the dependency supplies
696
+ it just by connecting. The gap worth alerting on is one no live client can
697
+ supply, and that is what the `on_gap` hook surfaces. It fires with the
698
+ document key whenever a document is loaded to serve state and a gap is still
699
+ open. Use it to emit a metric (a pending-document count, or the age of the
700
+ oldest open gap) so a stuck gap is visible. Gaps are also logged at `info`,
701
+ and errors raised in the hook are swallowed so observability can never break
702
+ frame handling.
703
+
704
+ ```ruby
705
+ class DocumentChannel < ApplicationCable::Channel
706
+ include Y::ActionCable
707
+
708
+ on_gap { |key| StatsD.increment("yrby.gap", tags: ["doc:#{key}"]) }
709
+ end
710
+ ```
711
+
537
712
  #### Multi-process deployments
538
713
 
539
714
  Most Rails apps run several processes, and any of them might serve a given document.
@@ -571,7 +746,7 @@ It is up to you to durably record it:
571
746
 
572
747
  ```ruby
573
748
  class DocumentChannel < ApplicationCable::Channel
574
- include Y::ActionCable::Sync
749
+ include Y::ActionCable
575
750
 
576
751
  # ...
577
752
 
@@ -593,12 +768,86 @@ duplicate record replays to the same document.
593
768
  The demo wires `on_change` to a durable Postgres-backed log by default, and checks
594
769
  end to end that the log alone rebuilds the document.
595
770
 
771
+ #### Ephemeral documents (no database)
772
+
773
+ `on_load` and `on_change` are plain blocks, and nothing requires them to touch
774
+ a database. For documents that don't need to outlive their session (a
775
+ scratchpad, live form state, a draft you only persist on submit) the store
776
+ can be connection state that travels with each request:
777
+
778
+ ```ruby
779
+ class ScratchpadChannel < ApplicationCable::Channel
780
+ include Y::ActionCable
781
+
782
+ on_load { |key| @doc_state }
783
+
784
+ on_change do |key, update|
785
+ doc = Y::Doc.new
786
+ doc.apply_update(@doc_state) if @doc_state
787
+ doc.apply_update(update)
788
+ @doc_state = doc.compacted_state_update
789
+ end
790
+
791
+ def subscribed = sync_subscribed(params[:id])
792
+ def receive(data) = sync_receive(data, params[:id])
793
+ end
794
+ ```
795
+
796
+ On AnyCable the channel object doesn't survive between messages, so an
797
+ instance variable won't hold. Declare the store as channel state instead
798
+ (`state_attr_accessor` comes from anycable-rails) and Base64 it, because that
799
+ state is serialized as JSON into each RPC exchange with `anycable-go`:
800
+
801
+ ```ruby
802
+ class ScratchpadChannel < ApplicationCable::Channel
803
+ include Y::ActionCable
804
+
805
+ state_attr_accessor :doc_state
806
+
807
+ on_load { |key| doc_state && Base64.strict_decode64(doc_state) }
808
+
809
+ on_change do |key, update|
810
+ doc = Y::Doc.new
811
+ doc.apply_update(Base64.strict_decode64(doc_state)) if doc_state
812
+ doc.apply_update(update)
813
+ self.doc_state = Base64.strict_encode64(doc.compacted_state_update)
814
+ end
815
+
816
+ def subscribed = sync_subscribed(params[:id])
817
+ def receive(data) = sync_receive(data, params[:id])
818
+ end
819
+ ```
820
+
821
+ Both hooks run in the channel instance (`instance_exec`), so they can use
822
+ anything the channel can, and `sync_receive` rebuilds the document from
823
+ `on_load` on every update, which is what lets the store live on the
824
+ connection. On Action Cable the channel instance lasts as long as the
825
+ connection, so an instance variable is the whole store. Merging into
826
+ `compacted_state_update` keeps it one blob instead of a growing update log.
827
+
828
+ The store is per connection, which shapes what this fits. A single writer gets
829
+ the full delivery contract with no database anywhere. With several people
830
+ editing at once, one client's update can depend on edits its own connection
831
+ has never seen; that update records as pending, and the next handshake with
832
+ that client (which always holds the full document) supplies the missing state
833
+ and heals it. The document still converges; heavy concurrent editing just
834
+ parks more pending between handshakes than a shared store would. On
835
+ AnyCable, keep the payload in mind too: the blob travels with every message,
836
+ so that variant suits small documents, not long manuscripts.
837
+
838
+ Durability is the connection plus the browsers. A reconnecting client re-seeds
839
+ an empty server through the ordinary sync handshake, so the document survives
840
+ server restarts as long as some client still has it. For ephemeral documents
841
+ shared across clients on a single-process deployment, the same two hooks over
842
+ a class-level `Concurrent::Map` work instead; that version stops being
843
+ coherent the moment you scale past one process.
844
+
596
845
  #### Reliable delivery (acks)
597
846
 
598
847
  yrby document delivery is ack-tracked. Browser document updates carry an
599
- `"id"`, and the server replies `{ "ack": <id> }` once `on_change` has succesfully fired.
600
- A causally-gapped update is not acked; the server sends a resync request, and
601
- the client keeps the update queued until it lands.
848
+ `"id"`, and the server replies `{ "ack": <id> }` once `on_change` has
849
+ successfully fired. Every decodable document update is recorded and acked,
850
+ including one that arrives out of order.
602
851
 
603
852
  ```
604
853
  client -> server { "update": "<base64 update>", "id": 42 }
@@ -612,13 +861,13 @@ one `{ ack: id }` cumulatively confirms everything up to it. Because CRDT apply
612
861
  is idempotent, a resend that already landed is a harmless no-op that just
613
862
  re-acks. Awareness stays ephemeral and is not acked.
614
863
 
615
- Presence (cursors, selections) is owned by the browser clients the server
864
+ Presence (cursors, selections) is owned by the browser clients; the server
616
865
  never sets or holds presence state, it only relays awareness frames opaquely.
617
866
  See `yrby-client` for the client-side awareness API.
618
867
 
619
868
  ## Thread Safety
620
869
 
621
- A `Doc` is safe to share across Ruby threads used concurrently from Puma
870
+ A `Doc` is safe to share across Ruby threads, used concurrently from Puma
622
871
  workers, ActionCable connection threads, or background jobs without external
623
872
  locking.
624
873
 
@@ -0,0 +1,44 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "rails/generators"
4
+ require "generators/yrby/tables/tables_generator"
5
+
6
+ module Yrby
7
+ module Generators
8
+ # `bin/rails generate yrby:install` — a DocumentChannel speaking the
9
+ # y-websocket protocol over the gem's document storage, plus the storage
10
+ # migration (via yrby:tables). The models ship in the gem; only the
11
+ # migration lands in the app.
12
+ class InstallGenerator < ::Rails::Generators::Base
13
+ source_root File.expand_path("templates", __dir__)
14
+
15
+ def create_channel
16
+ template "document_channel.rb", "app/channels/document_channel.rb"
17
+ end
18
+
19
+ def create_tables
20
+ invoke "yrby:tables"
21
+ end
22
+
23
+ def show_next_steps
24
+ say <<~NEXT
25
+
26
+ Next steps:
27
+
28
+ 1. Authorize document access: implement `authorized?` in
29
+ app/channels/document_channel.rb (it denies everyone until you do).
30
+ 2. bin/rails db:migrate
31
+ 3. Install the yrby-client npm package and connect an editor:
32
+
33
+ import { ActionCableProvider } from "yrby-client"
34
+ const provider = new ActionCableProvider(doc, consumer,
35
+ "DocumentChannel", { id: documentId })
36
+ provider.connect()
37
+
38
+ The README's Editors section links working integrations for
39
+ Tiptap, Lexxy, Rhino Editor, and CodeMirror.
40
+ NEXT
41
+ end
42
+ end
43
+ end
44
+ end
@@ -0,0 +1,36 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Collaborative documents over Action Cable: one channel speaking the
4
+ # y-websocket protocol (sync plus presence). Storage is Y::Document +
5
+ # Y::DocumentUpdate; a document is created on its first change and its
6
+ # history goes with it when it is destroyed. Point on_load/on_change
7
+ # elsewhere to swap storage.
8
+ class DocumentChannel < ApplicationCable::Channel
9
+ include Y::ActionCable
10
+
11
+ # Rebuild a document from durable storage (nil means a brand-new document).
12
+ on_load { |key| Y::Document.load_state(key) }
13
+
14
+ # Record each CRDT delta durably. Runs before the change is acknowledged
15
+ # or broadcast; if this raises, the change is neither acked nor relayed,
16
+ # and yrby-client retries it.
17
+ on_change { |key, update| Y::Document.append(key, update) }
18
+
19
+ def subscribed
20
+ return reject unless authorized?(params[:id])
21
+
22
+ sync_subscribed(params[:id])
23
+ end
24
+
25
+ def receive(data) = sync_receive(data, params[:id])
26
+
27
+ private
28
+
29
+ # Everyone is denied until you fill this in. Wire it to your app's auth:
30
+ # identify current_user on the cable connection, then check they may read
31
+ # and write this document. Don't lean on on_change raising for access
32
+ # control; that path exists for store failures.
33
+ def authorized?(_document_key)
34
+ false
35
+ end
36
+ end
@@ -0,0 +1,41 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "rails/generators"
4
+ require "rails/generators/active_record"
5
+
6
+ module Yrby
7
+ module Generators
8
+ # `bin/rails generate yrby:tables`: the migration for the gem-owned
9
+ # document models (Y::Document + Y::DocumentUpdate). Invoked by
10
+ # yrby:install, and by other gems building on the same storage.
11
+ #
12
+ # Template notes (kept here, not in the emitted migration): state is
13
+ # 4.gigabytes - 1 (longblob on MySQL; a compacted snapshot is the whole
14
+ # document; a 16 MB cap would break compaction) and payload is
15
+ # 16.megabytes - 1 (one update can carry a big paste or a client's
16
+ # accumulated offline edits; the 64 KB default blob is too small).
17
+ # The partial unique index's WHERE only keeps
18
+ # key-only rows out of the index: uniqueness holds without it, since
19
+ # unique indexes treat NULLs as distinct on every supported database,
20
+ # and MySQL drops the predicate harmlessly. y_document_updates indexes
21
+ # (document_id, pending) instead of bare document_id: the prefix
22
+ # serves the tail and foreign-key lookups, and the pair serves the
23
+ # clean-row count every append runs.
24
+ class TablesGenerator < ::Rails::Generators::Base
25
+ include ActiveRecord::Generators::Migration
26
+
27
+ source_root File.expand_path("templates", __dir__)
28
+
29
+ def create_migration_file
30
+ migration_template "create_y_tables.rb",
31
+ File.join(db_migrate_path, "create_y_tables.rb")
32
+ end
33
+
34
+ private
35
+
36
+ def migration_version
37
+ "[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
38
+ end
39
+ end
40
+ end
41
+ end
@@ -0,0 +1,24 @@
1
+ # frozen_string_literal: true
2
+
3
+ class CreateYTables < ActiveRecord::Migration<%= migration_version %>
4
+ def change
5
+ create_table :y_documents do |t|
6
+ t.string :key, null: false, index: { unique: true }
7
+ t.references :record, polymorphic: true, null: true, index: false
8
+ t.string :name
9
+ t.binary :state, limit: 4.gigabytes - 1
10
+ t.timestamps
11
+ t.index %i[record_type record_id name], unique: true,
12
+ where: "record_type IS NOT NULL",
13
+ name: "index_y_documents_on_record_and_name"
14
+ end
15
+
16
+ create_table :y_document_updates do |t|
17
+ t.references :document, null: false, foreign_key: { to_table: :y_documents }, index: false
18
+ t.binary :payload, null: false, limit: 16.megabytes - 1
19
+ t.boolean :pending, null: false, default: false
20
+ t.datetime :created_at, null: false
21
+ t.index %i[document_id pending]
22
+ end
23
+ end
24
+ end
data/lib/y/3.4/yrby.so CHANGED
Binary file
data/lib/y/4.0/yrby.so CHANGED
Binary file
data/lib/y/decoder.rb ADDED
@@ -0,0 +1,64 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "y"
4
+
5
+ module Y
6
+ # Plain-text reconstruction of a stored Yjs document, for search indexing
7
+ # and previews. It reads the text out of the shared type the editor uses
8
+ # (Lexical's `Y.XmlText`, plain `Y.Text`, or ProseMirror's `Y.XmlFragment`)
9
+ # in-process, on the same native extension as `Doc`: no Node, no
10
+ # subprocess.
11
+ #
12
+ # state = doc.encode_state_as_update # opaque CRDT bytes from the store
13
+ # Y::Decoder.text(state) # => "hello world"
14
+ # Y::Decoder.preview(state, 280) # => "hello world…"
15
+ #
16
+ # For full-fidelity HTML, `Y::Lexxy` and `Y::Tiptap` render the document
17
+ # with the editor's own semantics; this module is the cheap plain-text
18
+ # path.
19
+ module Decoder
20
+ class Error < Y::Error; end
21
+
22
+ module_function
23
+
24
+ # Plain text of the document. `field` pins the root key (Lexical: the editor
25
+ # id; ProseMirror: "default"); omit it to use the document's sole root.
26
+ def text(state, field: nil)
27
+ field ||= Y::Doc.new.tap { |d| d.apply_update(state) }.root_names.first
28
+ return "" unless field
29
+
30
+ # A plain `Y.Text` root (a simple shared-text editor) reads straight out.
31
+ # (A yrs root's type is fixed by its first typed access, so each reader
32
+ # gets a fresh doc to try a different shared type against the same state.)
33
+ direct = load(state).read_text(field)
34
+ return normalize(direct) if direct && !direct.strip.empty?
35
+
36
+ # Lexical (each block a sibling `Y.XmlText`) and ProseMirror (blocks are
37
+ # `Y.XmlElement`s) both come back from read_xml as block-per-line markup;
38
+ # strip any element tags to plain text.
39
+ markup = load(state).read_xml(field)
40
+ markup ? normalize(strip_tags(markup)) : ""
41
+ end
42
+
43
+ # A compact, single-line preview for list UIs.
44
+ def preview(state, limit: 280, field: nil)
45
+ body = text(state, field: field).gsub(/\s+/, " ").strip
46
+ body.length > limit ? "#{body[0, limit].rstrip}…" : body
47
+ end
48
+
49
+ def load(state)
50
+ Y::Doc.new.tap { |doc| doc.apply_update(state) }
51
+ end
52
+
53
+ def strip_tags(markup)
54
+ markup.gsub(/<[^>]*>/, " ")
55
+ end
56
+
57
+ def normalize(text)
58
+ text.gsub(/[ \t]+/, " ") # collapse runs of spaces/tabs
59
+ .gsub(/ *\n */, "\n") # trim spaces left around block separators
60
+ .gsub(/\n{3,}/, "\n\n") # cap blank-line runs
61
+ .strip
62
+ end
63
+ end
64
+ end
data/lib/y/lexxy.rb CHANGED
@@ -52,24 +52,44 @@ module Y
52
52
  # and presence mirror Lexxy's exportDOM (nulls omitted, `previewable`
53
53
  # only when true, `presentation="gallery"` always).
54
54
  def self.upload(node)
55
- out = +"<action-text-attachment"
55
+ tag = attachment_tag(node)
56
+ out = "<#{tag}"
56
57
  out << attachment_attr(node, "sgid", "sgid")
57
58
  out << %( previewable="true") if node.attrs["previewable"] == true
58
59
  [%w[url src], %w[alt altText], %w[caption caption],
59
60
  %w[content-type contentType], %w[filename fileName],
60
61
  %w[filesize fileSize], %w[width width], %w[height height]]
61
62
  .each { |html_name, stored| out << attachment_attr(node, html_name, stored) }
62
- %(#{out} presentation="gallery"></action-text-attachment>)
63
+ %(#{out} presentation="gallery"></#{tag}>)
63
64
  end
64
65
 
65
66
  # A content attachment (mention, embed): `content` carries the escaped
66
67
  # inner HTML; `plainText` is not exported.
67
68
  def self.mention(node)
68
- out = +"<action-text-attachment"
69
+ tag = attachment_tag(node)
70
+ out = "<#{tag}"
69
71
  out << attachment_attr(node, "sgid", "sgid")
70
72
  out << attachment_attr(node, "content", "innerHtml")
71
73
  out << attachment_attr(node, "content-type", "contentType")
72
- "#{out}></action-text-attachment>"
74
+ "#{out}></#{tag}>"
75
+ end
76
+
77
+ # Lexxy's attachment tag is configurable (`Lexxy.configure`'s
78
+ # attachmentTagName, paired with ActionText::Attachment.tag_name on the
79
+ # Rails side), and each attachment node stores the tag it was created
80
+ # with. Emit the stored tag. The value is stored document data, not
81
+ # markup, so anything that doesn't look like a tag name falls back to
82
+ # ActionText's default.
83
+ def self.attachment_tag(node)
84
+ tag = node.attrs["tagName"].to_s
85
+ tag.match?(/\A[a-zA-Z][a-zA-Z0-9-]*\z/) ? tag : "action-text-attachment"
86
+ end
87
+
88
+ # An in-flight upload placeholder. Only the uploader's browser can
89
+ # complete it, so it is never finished content -- materialized output
90
+ # (Action Text bodies, search text) must not carry it.
91
+ def self.pending_upload(_node)
92
+ ""
73
93
  end
74
94
 
75
95
  # A stored nil (unset) is skipped; a stored empty string still emits.
@@ -90,6 +110,7 @@ module Y
90
110
  "tablecell" => { contains: :blocks, render: method(:table_cell) },
91
111
  "listitem" => { contains: :blocks, render: method(:list_item) },
92
112
  "action_text_attachment" => method(:upload),
113
+ "action_text_attachment_upload" => method(:pending_upload),
93
114
  "custom_action_text_attachment" => method(:mention)
94
115
  }.freeze
95
116
 
data/lib/y/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Y
4
- VERSION = "0.6.0"
4
+ VERSION = "0.7.0"
5
5
  end
data/lib/y.rb CHANGED
@@ -15,6 +15,7 @@ end
15
15
  require_relative "y/rendering"
16
16
  require_relative "y/lexxy"
17
17
  require_relative "y/tiptap"
18
+ require_relative "y/decoder"
18
19
 
19
20
  module Y
20
21
  # Doc, Error, and the protocol module functions are defined in the Rust
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: yrby
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.6.0
4
+ version: 0.7.0
5
5
  platform: aarch64-linux
6
6
  authors:
7
7
  - JP Camara
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2026-07-12 00:00:00.000000000 Z
11
+ date: 2026-08-12 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: minitest
@@ -55,7 +55,7 @@ dependencies:
55
55
  description: 'yrby is a thread-safe Ruby binding over the Rust y-crdt (yrs) library:
56
56
  CRDT documents, awareness/presence, and the y-websocket sync protocol primitives,
57
57
  with the GVL released during native work so documents sync in parallel. The ActionCable/Rails
58
- integration lives in the companion yrby-actioncable gem.'
58
+ integration lives in the companion yrby-rails gem.'
59
59
  email:
60
60
  - johnpcamara@gmail.com
61
61
  executables: []
@@ -65,9 +65,14 @@ files:
65
65
  - CHANGELOG.md
66
66
  - LICENSE
67
67
  - README.md
68
+ - lib/generators/yrby/install/install_generator.rb
69
+ - lib/generators/yrby/install/templates/document_channel.rb
70
+ - lib/generators/yrby/tables/tables_generator.rb
71
+ - lib/generators/yrby/tables/templates/create_y_tables.rb
68
72
  - lib/y.rb
69
73
  - lib/y/3.4/yrby.so
70
74
  - lib/y/4.0/yrby.so
75
+ - lib/y/decoder.rb
71
76
  - lib/y/lexxy.rb
72
77
  - lib/y/rendering.rb
73
78
  - lib/y/tiptap.rb