yrby 0.7.1 → 0.8.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +88 -0
- data/Cargo.lock +3 -3
- data/README.md +593 -384
- data/ext/yrby/crates/html-core/Cargo.toml +1 -1
- data/ext/yrby/crates/html-core/src/lib.rs +13 -13
- data/ext/yrby/crates/lexical-html/Cargo.toml +2 -2
- data/ext/yrby/crates/lexical-html/src/lib.rs +90 -45
- data/ext/yrby/crates/prosemirror-html/Cargo.toml +2 -2
- data/ext/yrby/crates/prosemirror-html/src/lib.rs +44 -35
- data/ext/yrby/src/lib.rs +27 -12
- data/ext/yrby/src/protocol.rs +17 -17
- data/ext/yrby/src/read.rs +137 -26
- data/lib/generators/yrby/install/install_generator.rb +24 -13
- data/lib/generators/yrby/install/templates/document_channel.rb +4 -7
- data/lib/generators/yrby/tables/tables_generator.rb +7 -4
- data/lib/generators/yrby/tables/templates/create_y_tables.rb +1 -1
- data/lib/y/collaborative/attribute.rb +53 -0
- data/lib/y/collaborative/helper.rb +48 -0
- data/lib/y/collaborative.rb +98 -0
- data/lib/y/lexxy.rb +4 -4
- data/lib/y/rendering.rb +39 -15
- data/lib/y/tiptap.rb +3 -3
- data/lib/y/version.rb +1 -1
- metadata +4 -9
- data/ext/yrby/target/debug/build/clang-sys-e3ae45bd384f74c3/out/common.rs +0 -355
- data/ext/yrby/target/debug/build/clang-sys-e3ae45bd384f74c3/out/dynamic.rs +0 -276
- data/ext/yrby/target/debug/build/clang-sys-e3ae45bd384f74c3/out/macros.rs +0 -49
- data/ext/yrby/target/debug/build/rb-sys-4407948463231c4f/out/bindings-0.9.128-mri-arm64-darwin23-3.4.7.rs +0 -8934
- data/ext/yrby/target/debug/build/rb-sys-dbeea42737529c2d/out/bindings-0.9.128-mri-arm64-darwin23-3.4.7.rs +0 -8934
- data/ext/yrby/target/debug/build/serde-58ea0ee887cc2602/out/private.rs +0 -6
- data/ext/yrby/target/debug/build/serde_core-41f407c21c1f205e/out/private.rs +0 -5
- data/ext/yrby/target/debug/build/thiserror-0f1416a82ff26f22/out/private.rs +0 -5
data/README.md
CHANGED
|
@@ -2,60 +2,87 @@
|
|
|
2
2
|
|
|
3
3
|
[](https://github.com/jpcamara/yrby/actions/workflows/ci.yml)
|
|
4
4
|
|
|
5
|
-
yrby (pronounced "yer-bee") makes Rails a
|
|
5
|
+
yrby (pronounced "yer-bee") makes Rails a Yjs backend. It binds
|
|
6
6
|
[y-crdt](https://github.com/y-crdt/y-crdt), the Rust engine behind Y.js, into
|
|
7
|
-
Ruby, and
|
|
7
|
+
Ruby, and on top of that it builds a sync server for Action
|
|
8
8
|
Cable and AnyCable, a browser provider, and server-side reading and rendering
|
|
9
|
-
of the documents.
|
|
10
|
-
|
|
9
|
+
of the documents. You get real-time collaboration in a Rails app with no Node
|
|
10
|
+
process to run.
|
|
11
11
|
|
|
12
12
|

|
|
13
13
|
|
|
14
|
-
On the server, `yrby-rails` implements the
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
([Delivery guarantees](#delivery-guarantees))
|
|
14
|
+
On the server, `yrby-rails` implements the y-websocket protocol (document sync
|
|
15
|
+
and presence) as a channel concern. The server writes every update to your
|
|
16
|
+
store before it acknowledges the update or sends it to other clients, which
|
|
17
|
+
most Yjs servers don't do. Replaying the store rebuilds the document, however
|
|
18
|
+
many processes you run. ([Delivery guarantees](#delivery-guarantees))
|
|
20
19
|
|
|
21
20
|
In the browser, `yrby-client`'s `ActionCableProvider` connects anything that
|
|
22
21
|
speaks Yjs. The demo app runs four rich text editors, and CI drives each one
|
|
23
22
|
in real Chrome: Tiptap, [Lexxy](https://www.npmjs.com/package/lexxy-realtime),
|
|
24
23
|
Rhino Editor, and CodeMirror. The same channel also syncs Yjs shapes with no
|
|
25
24
|
editor at all: a whiteboard on a `Y.Map`, a kanban board on a `Y.Array`, a
|
|
26
|
-
|
|
25
|
+
form filled in together. ([Editors](#editors))
|
|
27
26
|
|
|
28
27
|
In Ruby, the documents are readable without a browser. `Doc#read_text` and
|
|
29
|
-
`Doc#read_map`
|
|
30
|
-
`Y::Tiptap` and `Y::Lexxy` render a document to HTML
|
|
31
|
-
|
|
32
|
-
|
|
28
|
+
`Doc#read_map` rebuild the contents for search, validation, and exports.
|
|
29
|
+
`Y::Tiptap` and `Y::Lexxy` render a document to HTML that matches the editor's
|
|
30
|
+
own serializer byte for byte. You can add rules for your app's custom nodes,
|
|
31
|
+
and you can save the output directly to ActionText.
|
|
32
|
+
([Rendering to HTML](#rendering-to-html))
|
|
33
33
|
|
|
34
34
|
Underneath, the core is built for a production Rails deployment. A `Doc` is
|
|
35
35
|
thread-safe across Puma and ActionCable threads. Native CRDT work runs with
|
|
36
|
-
the GVL released, so it
|
|
36
|
+
the GVL released, so it runs in parallel on MRI. Incoming frames are validated
|
|
37
37
|
before anything processes them, and multi-process and AnyCable setups are
|
|
38
38
|
tested end to end. ([Thread Safety](#thread-safety))
|
|
39
39
|
|
|
40
|
-
|
|
40
|
+
You add one view helper, on a page where the user can already edit the record:
|
|
41
41
|
|
|
42
|
-
```
|
|
43
|
-
|
|
44
|
-
|
|
42
|
+
```erb
|
|
43
|
+
<%= collaborative_document_tag @post, :body %>
|
|
44
|
+
```
|
|
45
45
|
|
|
46
|
-
|
|
47
|
-
|
|
46
|
+
The helper renders a signed grant for that record and attribute, much as
|
|
47
|
+
`turbo_stream_from` signs its stream names. The client subscribes with the
|
|
48
|
+
grant and the attribute name. The gem's `Y::DocumentChannel` verifies the
|
|
49
|
+
grant, then saves every change as `Y::Document` rows before acknowledging it,
|
|
50
|
+
so you don't have to write a channel yourself.
|
|
48
51
|
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
+
The tag renders an element that connects by itself once you import it. When the
|
|
53
|
+
document has synced, your listener receives it and passes it to your editor
|
|
54
|
+
binding:
|
|
55
|
+
|
|
56
|
+
```js
|
|
57
|
+
import "yrby-client/element"
|
|
58
|
+
|
|
59
|
+
document.addEventListener("yrby:synced", ({ target, detail }) => {
|
|
60
|
+
const editor = bindYourEditor(target, detail.doc, detail.provider)
|
|
61
|
+
detail.signal.addEventListener("abort", () => editor.destroy(), { once: true })
|
|
62
|
+
})
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
If the user leaves the page with edits still unsent, the session keeps them and
|
|
66
|
+
finishes delivering. Your binding cleans up when the abort signal fires, and if
|
|
67
|
+
nothing is pending, the next visit loads the document from Rails. The [client
|
|
68
|
+
lifecycle and recovery contract](packages/client/README.md#document-sessions)
|
|
69
|
+
has the details.
|
|
70
|
+
|
|
71
|
+
The document is stored as rows in your database, and you can read it back in
|
|
72
|
+
Ruby:
|
|
73
|
+
|
|
74
|
+
```ruby
|
|
75
|
+
doc = post.collaborative_document(:body).y_doc
|
|
76
|
+
doc.read_text("content") # or Y::Lexxy.new(doc).to_html for rich text
|
|
52
77
|
```
|
|
53
78
|
|
|
54
79
|
Install the gem and the npm package:
|
|
55
80
|
|
|
56
81
|
```
|
|
57
82
|
gem install yrby-rails # depends on yrby
|
|
58
|
-
npm install yrby-client
|
|
83
|
+
npm install yrby-client yjs y-protocols @rails/actioncable
|
|
84
|
+
|
|
85
|
+
bin/rails generate yrby:install && bin/rails db:migrate
|
|
59
86
|
```
|
|
60
87
|
|
|
61
88
|
## Contents
|
|
@@ -84,51 +111,54 @@ npm install yrby-client
|
|
|
84
111
|
|
|
85
112
|
## Scope
|
|
86
113
|
|
|
87
|
-
`yrby` binds
|
|
88
|
-
documents
|
|
89
|
-
the Ruby side treats a document as opaque CRDT state
|
|
90
|
-
sync handshakes, and records deltas without
|
|
91
|
-
|
|
92
|
-
and `Doc#read_map`
|
|
114
|
+
`yrby` binds the parts of `y-crdt` you need to sync and persist collaborative
|
|
115
|
+
documents, which are a `Doc`, awareness, and the y-websocket protocol
|
|
116
|
+
primitives. By default the Ruby side treats a document as opaque CRDT state. It
|
|
117
|
+
applies updates, answers sync handshakes, and records deltas without reading
|
|
118
|
+
the contents, and the browser editor decides what shape the document has. When
|
|
119
|
+
you do need to look inside, `Doc#read_text` and `Doc#read_map` rebuild it in
|
|
120
|
+
Ruby.
|
|
93
121
|
|
|
94
122
|
## Durability and delivery
|
|
95
123
|
|
|
96
|
-
The
|
|
124
|
+
The API is small, and most of the work went into durability, resiliency, delivery
|
|
97
125
|
guarantees, correctness, and thread safety.
|
|
98
126
|
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
-
|
|
102
|
-
|
|
103
|
-
only acks
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
retransmitting it until it
|
|
127
|
+
`yrby` adds two opinionated defaults on top of normal Yjs syncing:
|
|
128
|
+
|
|
129
|
+
- yrby acknowledges updates. The `ActionCableProvider` in `yrby-client` keeps
|
|
130
|
+
resending an update until the server acks it, and
|
|
131
|
+
[`yrby-rails`](https://rubygems.org/gems/yrby-rails) only acks an update
|
|
132
|
+
after it's durably recorded. Together that gives you at-least-once delivery,
|
|
133
|
+
and because CRDT updates are idempotent, a duplicate does nothing.
|
|
134
|
+
- yrby expects causal gaps, where an update arrives before another update it
|
|
135
|
+
depends on, and it records and acks that update like any other. The sender of
|
|
136
|
+
the missing update keeps retransmitting it until it's acked, and the gap
|
|
137
|
+
closes when it arrives. `Doc#pending?` and the `on_gap` hook
|
|
109
138
|
tell you when a document is waiting on a missing update.
|
|
110
139
|
([Causal gaps](#causal-gaps))
|
|
111
140
|
|
|
112
141
|
## What about [yrb](https://github.com/y-crdt/yrb)?
|
|
113
142
|
|
|
114
|
-
`yrb` has a much larger interface
|
|
115
|
-
|
|
116
|
-
inspiration for
|
|
117
|
-
on top of it.
|
|
143
|
+
`yrb` has a much larger interface, with most of the Yjs type system (shared
|
|
144
|
+
text, arrays, maps, XML) for building and querying documents in Ruby. It was a
|
|
145
|
+
big inspiration for me in using Yjs from Ruby and Rails, and I considered
|
|
146
|
+
building on top of it. I went with `yrby` for a few reasons:
|
|
118
147
|
|
|
119
|
-
- `yrb`
|
|
120
|
-
|
|
121
|
-
- [It isn't thread-safe](https://github.com/y-crdt/yrb/issues/72)
|
|
122
|
-
|
|
123
|
-
- It's a much larger set of features to maintain,
|
|
124
|
-
majority of
|
|
148
|
+
- `yrb` started as an experiment for GitLab, and its original author has mostly
|
|
149
|
+
moved on to other projects, so it's largely unmaintained.
|
|
150
|
+
- [It isn't thread-safe](https://github.com/y-crdt/yrb/issues/72), and it
|
|
151
|
+
segfaults in threaded environments like ActionCable.
|
|
152
|
+
- It's a much larger set of features to maintain, and most people don't need
|
|
153
|
+
them. The vast majority of Y.js documents are only ever manipulated in the
|
|
154
|
+
browser.
|
|
125
155
|
|
|
126
156
|
## Testing
|
|
127
157
|
|
|
128
158
|
Ruby and Rust unit tests cover the core. CI also runs the npm client tests and a
|
|
129
159
|
Rails demo smoke slice against the real ActionCable stack. The demo includes
|
|
130
160
|
heavier local suites for hostile input, crash recovery, multi-browser editing,
|
|
131
|
-
AnyCable, and load testing. The benchmark
|
|
161
|
+
AnyCable, and load testing. The benchmark number below is from a single laptop.
|
|
132
162
|
Issues and PRs are welcome.
|
|
133
163
|
|
|
134
164
|
## Install
|
|
@@ -142,10 +172,11 @@ gem "yrby"
|
|
|
142
172
|
gem "yrby-rails"
|
|
143
173
|
```
|
|
144
174
|
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
on Linux x86_64 and macOS arm64.
|
|
148
|
-
|
|
175
|
+
yrby needs Ruby 3.4 or newer. Releases include precompiled gems for Ruby 3.4
|
|
176
|
+
and 4.0 on the supported Ruby platforms, and the release workflow smoke-tests
|
|
177
|
+
the native builds on Linux x86_64 and macOS arm64. You only need
|
|
178
|
+
[Rust](https://rustup.rs) when no precompiled gem matches your platform and the
|
|
179
|
+
gem builds from source.
|
|
149
180
|
|
|
150
181
|
To work on the gem itself:
|
|
151
182
|
|
|
@@ -163,16 +194,17 @@ The rest of the dev setup, plus the demo, is in [CONTRIBUTING.md](CONTRIBUTING.m
|
|
|
163
194
|
- The ActionCable concern and a quickstart are [below](#actioncable-integration).
|
|
164
195
|
- [`examples/actioncable-demo`](examples/actioncable-demo): a runnable Rails +
|
|
165
196
|
Tiptap app with collaborative cursors, the AnyCable setup, a Postgres store,
|
|
166
|
-
and the test
|
|
197
|
+
and the test and load suites.
|
|
167
198
|
- [CHANGELOG.md](CHANGELOG.md) and [CONTRIBUTING.md](CONTRIBUTING.md).
|
|
168
199
|
|
|
169
200
|
## Editors
|
|
170
201
|
|
|
171
|
-
yrby syncs opaque Yjs updates, so
|
|
172
|
-
|
|
202
|
+
yrby syncs opaque Yjs updates, so any editor with a Yjs binding works. The demo
|
|
203
|
+
app runs four of them, and CI drives each one in real Chrome to check
|
|
173
204
|
concurrent typing with every keystroke accounted for, remote cursors,
|
|
174
|
-
local-only undo, and
|
|
175
|
-
editor's own serializer. Each page is a working integration
|
|
205
|
+
local-only undo, and that the server-side renderers produce the same HTML as
|
|
206
|
+
the editor's own serializer. Each demo page is a working integration you can
|
|
207
|
+
copy from:
|
|
176
208
|
|
|
177
209
|
| Editor | Yjs binding | Demo code |
|
|
178
210
|
|---|---|---|
|
|
@@ -181,11 +213,11 @@ editor's own serializer. Each page is a working integration to copy from:
|
|
|
181
213
|
| [Rhino Editor](https://github.com/KonnorRogers/rhino-editor) (Tiptap 3) | `@tiptap/extension-collaboration` + `-caret` | [`rhino.js`](examples/actioncable-demo/frontend/src/rhino.js) |
|
|
182
214
|
| [CodeMirror 6](https://codemirror.net) | `y-codemirror.next` | [`codemirror.js`](examples/actioncable-demo/frontend/src/codemirror.js) |
|
|
183
215
|
|
|
184
|
-
The demo also syncs plain Yjs shapes with no editor
|
|
185
|
-
|
|
186
|
-
same channel. The demo README's "Using this in your own app" section has
|
|
187
|
-
|
|
188
|
-
|
|
216
|
+
The demo also syncs plain Yjs shapes with no editor (a whiteboard on a
|
|
217
|
+
`Y.Map`, a kanban board on a `Y.Array`, a form filled in together) over the
|
|
218
|
+
same channel. The demo README's "Using this in your own app" section has the
|
|
219
|
+
integration recipe, and its `NoteMaterializer` shows how to render a document
|
|
220
|
+
to ActionText on the server with `Y::Tiptap` or `Y::Lexxy`.
|
|
189
221
|
|
|
190
222
|
## Usage
|
|
191
223
|
|
|
@@ -219,43 +251,46 @@ doc.handle_sync_message(data) # => [msg_type, sync_type, response]; answers
|
|
|
219
251
|
|
|
220
252
|
### Reading document contents
|
|
221
253
|
|
|
222
|
-
|
|
254
|
+
Rebuild a document on the server for search, exports, emails, or SSR, with no
|
|
223
255
|
Node process:
|
|
224
256
|
|
|
225
257
|
```ruby
|
|
226
258
|
doc.read_text("prosemirror") # => plain text of a Y.Text root, or nil
|
|
227
259
|
doc.read_xml("root") # => text of an XML root, one block per line
|
|
228
260
|
doc.read_map("state") # => a Y.Map root as a JSON string; JSON.parse it
|
|
261
|
+
doc.read_array("cards") # => a Y.Array root as a JSON string; JSON.parse it
|
|
229
262
|
```
|
|
230
263
|
|
|
231
264
|
### Pending structs and gap-free state
|
|
232
265
|
|
|
233
266
|
If a doc applies an update whose causally-prior update is missing (a "gappy"
|
|
234
|
-
update), yrs
|
|
235
|
-
|
|
236
|
-
missing dependency later
|
|
267
|
+
update), yrs holds it as a **pending** struct and the integrated state vector
|
|
268
|
+
doesn't move. yrs keeps the pending block as a recovery buffer and integrates
|
|
269
|
+
it if the missing dependency arrives later. `Doc#pending?` tells you when a doc
|
|
270
|
+
is in this state.
|
|
237
271
|
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
272
|
+
`handle_sync_message` answers `SyncStep1` with the doc's full state, pending
|
|
273
|
+
structs included, which is what Y.js's `encodeStateAsUpdate` does too. A peer
|
|
274
|
+
that receives it holds the pending struct and integrates it the same way this
|
|
275
|
+
doc would. The one place pending structs must be left out is a compacted
|
|
276
|
+
snapshot:
|
|
243
277
|
|
|
244
278
|
- `Doc#compacted_state_update` returns a gap-free full-state update for
|
|
245
279
|
compaction. Folding a log into one blob would otherwise freeze an
|
|
246
|
-
un-integrable struct into the base state
|
|
247
|
-
doc keeps its pending.
|
|
248
|
-
- `encode_state_as_update`
|
|
249
|
-
|
|
280
|
+
un-integrable struct into the base state permanently. The call doesn't modify
|
|
281
|
+
the doc, which keeps its pending structs.
|
|
282
|
+
- `encode_state_as_update` is lossless, so persistence and serving keep the
|
|
283
|
+
raw pending bytes and the gap can still close.
|
|
250
284
|
|
|
251
285
|
### Rendering to HTML
|
|
252
286
|
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
287
|
+
The renderers turn a collaborative document into HTML on the server with no
|
|
288
|
+
Node process or headless editor. Each renderer is a class for one editor, and
|
|
289
|
+
its output matches that editor's own serializer. `Y::Tiptap` renders
|
|
290
|
+
ProseMirror documents and is built on `Y::ProseMirror`, and `Y::Lexxy` renders
|
|
291
|
+
documents from the [Lexxy](https://github.com/basecamp/lexxy) editor and is
|
|
292
|
+
built on `Y::Lexical`. To support another editor on one of those engines,
|
|
293
|
+
extend the matching base class with rules. Each renderer
|
|
259
294
|
returns `nil` for a root that belongs to the other schema.
|
|
260
295
|
|
|
261
296
|
#### `Y::Tiptap` (and `Y::ProseMirror`, its base)
|
|
@@ -266,26 +301,27 @@ tiptap.to_html # the "default" fragment (Tiptap's default root)
|
|
|
266
301
|
tiptap.to_html("content") # or another XML root
|
|
267
302
|
```
|
|
268
303
|
|
|
269
|
-
The output matches Tiptap's own `getHTML()`,
|
|
270
|
-
|
|
271
|
-
[`tiptap-php`](https://github.com/ueberdosis/tiptap-php) and
|
|
272
|
-
styles editors use
|
|
273
|
-
`bullet_list
|
|
274
|
-
|
|
275
|
-
It covers paragraphs, headings, blockquotes, bullet
|
|
276
|
-
blocks, links, images, mentions, details, hard breaks, horizontal rules,
|
|
277
|
-
tables, text styles (color
|
|
278
|
-
as
|
|
279
|
-
view adds.
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
ProseMirror natively
|
|
283
|
-
family
|
|
284
|
-
family) are
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
that node rules
|
|
288
|
-
as
|
|
304
|
+
The output matches Tiptap's own `getHTML()`, and the tests check it against a
|
|
305
|
+
document captured from a real editor. The implementation follows
|
|
306
|
+
[`tiptap-php`](https://github.com/ueberdosis/tiptap-php) and accepts both
|
|
307
|
+
naming styles editors use, Tiptap's `bulletList` and `bold` as well as
|
|
308
|
+
prosemirror-schema-basic's `bullet_list` and `strong`.
|
|
309
|
+
|
|
310
|
+
It covers paragraphs, headings, blockquotes, bullet, ordered, and task lists,
|
|
311
|
+
code blocks, links, images, mentions, details, hard breaks, horizontal rules,
|
|
312
|
+
tables, text styles (color and font family), and every text mark. Tables render
|
|
313
|
+
as a plain `<table><tbody>` without the column-width styling that Tiptap's
|
|
314
|
+
editor view adds.
|
|
315
|
+
|
|
316
|
+
Support is split into two layers, as on the Lexical side. `Y::ProseMirror`
|
|
317
|
+
handles core ProseMirror natively, meaning prosemirror-schema-basic plus the
|
|
318
|
+
prosemirror-tables family. Tiptap's extension nodes (task lists, mentions, the
|
|
319
|
+
details family) are a rule set, `Y::Tiptap::NODES`, written with the extension
|
|
320
|
+
API described below. Marks are handled in the base class. Mark rendering deals
|
|
321
|
+
with nesting order, the CSS on `textStyle`, and the exclusivity of `code`, and
|
|
322
|
+
it runs through native text-run code that node rules can't reach, so
|
|
323
|
+
`Y::ProseMirror` renders Tiptap's mark set as is. You can override individual
|
|
324
|
+
marks with `rules.mark`.
|
|
289
325
|
|
|
290
326
|
#### `Y::Lexxy` (and `Y::Lexical`, its base)
|
|
291
327
|
|
|
@@ -295,41 +331,41 @@ lexxy.to_html # the "root" fragment (Lexical's default root name)
|
|
|
295
331
|
lexxy.to_html("notepad") # or another XML root
|
|
296
332
|
```
|
|
297
333
|
|
|
298
|
-
The HTML is identical to
|
|
299
|
-
|
|
300
|
-
editor
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
galleries) and its decorations of core nodes (the
|
|
316
|
-
styling, nested-list classes)
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
fall back to readable markup rather than disappearing.
|
|
334
|
+
The HTML is identical to the `value` a `lexxy-editor` submits to Rails, and the
|
|
335
|
+
tests check it against a document captured from a real editor. The class is
|
|
336
|
+
named after the editor because stock Lexical has no canonical serializer, and
|
|
337
|
+
every editor configures its own. `Y::Lexical` is the core Lexical base
|
|
338
|
+
(paragraphs, headings, quotes, code, lists, tables, links, and the full
|
|
339
|
+
text-format model), and other Lexical editors extend it with rules.
|
|
340
|
+
|
|
341
|
+
It handles every node in Lexxy 1.0, which has the same set as 0.9.x:
|
|
342
|
+
paragraphs, headings, every text format and their combinations, links, the
|
|
343
|
+
four list types with nesting, blockquotes, code blocks, tabs and soft breaks,
|
|
344
|
+
horizontal rules, tables with header cells, image galleries, and ActionText
|
|
345
|
+
attachments. Uploads and mentions both render as `<action-text-attachment>`
|
|
346
|
+
elements, which ActionText can re-render.
|
|
347
|
+
|
|
348
|
+
The Lexical support has the same two layers. `Y::Lexical` handles core Lexical
|
|
349
|
+
structure natively, and everything Lexxy adds is in `Y::Lexxy`'s rule set,
|
|
350
|
+
`Y::Lexxy::NODES`, written with the extension API below. That includes Lexxy's
|
|
351
|
+
own node types (attachments, galleries) and its decorations of core nodes (the
|
|
352
|
+
table wrapper, header-cell styling, nested-list classes). Since the gem's Lexxy
|
|
353
|
+
support was the first thing written on that API, an app rule for one of those
|
|
354
|
+
types replaces the built-in one.
|
|
355
|
+
|
|
356
|
+
In both renderers, an unknown node still renders its text and nested blocks as
|
|
357
|
+
readable markup.
|
|
323
358
|
|
|
324
359
|
#### Custom nodes and marks
|
|
325
360
|
|
|
326
|
-
The built-in schemas
|
|
327
|
-
|
|
328
|
-
before the built-in schema, so
|
|
329
|
-
built-in renders.
|
|
361
|
+
The built-in schemas match what Tiptap and Lexxy ship. When your app adds its
|
|
362
|
+
own node types, both renderers take rules for them. The renderer checks rules
|
|
363
|
+
before the built-in schema, so a rule can add a node type or change how a
|
|
364
|
+
built-in one renders.
|
|
330
365
|
|
|
331
|
-
|
|
332
|
-
rule
|
|
366
|
+
You register rules in a block with one `rules.node` call per type. A
|
|
367
|
+
declarative rule describes the markup as a tag, attributes, and a content mode,
|
|
368
|
+
and the renderer emits it natively:
|
|
333
369
|
|
|
334
370
|
```ruby
|
|
335
371
|
tiptap = Y::Tiptap.new(doc) do |rules|
|
|
@@ -339,17 +375,18 @@ tiptap = Y::Tiptap.new(doc) do |rules|
|
|
|
339
375
|
end
|
|
340
376
|
```
|
|
341
377
|
|
|
342
|
-
`tag` names the element. `attrs`
|
|
343
|
-
a symbol reads that attribute off the node, an array
|
|
344
|
-
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
|
|
378
|
+
`tag` names the element. The values in `attrs` are templates, where a string is
|
|
379
|
+
a literal, a symbol reads that attribute off the node, and an array
|
|
380
|
+
concatenates a mix of the two. Attributes that resolve to an empty value are
|
|
381
|
+
left out. `text` takes the same template form and emits literal text content.
|
|
382
|
+
`contains` says what goes inside the node, which is `:inline` for formatted
|
|
383
|
+
text (the default), `:blocks` for child block nodes, or `:none` for a leaf.
|
|
384
|
+
`void: true` skips the closing tag.
|
|
348
385
|
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
`
|
|
352
|
-
|
|
386
|
+
Editors store types and attributes under names you wouldn't predict. Rhino's
|
|
387
|
+
strike mark is `rhino-strike`, for example, and Lexical prefixes its own props
|
|
388
|
+
with `__`. To find the real names and shapes, make a document in your editor
|
|
389
|
+
that uses your custom node, then run:
|
|
353
390
|
|
|
354
391
|
```ruby
|
|
355
392
|
Y::Tiptap.new(doc).node_types
|
|
@@ -359,11 +396,19 @@ Y::Tiptap.new(doc).node_types
|
|
|
359
396
|
# "paragraph" => { ..., "handled" => "builtin" } }
|
|
360
397
|
```
|
|
361
398
|
|
|
362
|
-
`handled`
|
|
363
|
-
names your templates and blocks will read
|
|
364
|
-
|
|
399
|
+
A type with `handled` set to nil still needs a rule. `attrs` lists the stored
|
|
400
|
+
attribute names your templates and blocks will read, and `children` and `text`
|
|
401
|
+
tell you which `contains:` to pick. Child block types mean `:blocks`, and text
|
|
402
|
+
means `:inline`.
|
|
403
|
+
|
|
404
|
+
`unknown_types` lists the types whose `handled` is nil. `to_html` still
|
|
405
|
+
renders them as well as it can. An unknown container or inline wrapper renders
|
|
406
|
+
its children without its own markup, so its text stays. A node that keeps its
|
|
407
|
+
content only in its attributes renders nothing. Either way, the node's markup is
|
|
408
|
+
missing from the HTML, and nothing raises. When you add editor nodes, add a test
|
|
409
|
+
that renders a real document and asserts `unknown_types` is empty.
|
|
365
410
|
|
|
366
|
-
When
|
|
411
|
+
When a declarative rule can't express the markup, give the node a block:
|
|
367
412
|
|
|
368
413
|
```ruby
|
|
369
414
|
lexical = Y::Lexical.new(doc) do |rules|
|
|
@@ -374,26 +419,26 @@ lexical = Y::Lexical.new(doc) do |rules|
|
|
|
374
419
|
end
|
|
375
420
|
```
|
|
376
421
|
|
|
377
|
-
The block
|
|
378
|
-
children
|
|
379
|
-
element
|
|
380
|
-
|
|
381
|
-
whether a list item
|
|
382
|
-
|
|
383
|
-
interpolate. To set the content mode for a
|
|
422
|
+
The block receives the node's type and stored attributes, plus `node.content`,
|
|
423
|
+
which is the node's children already rendered to HTML. `node.child_types` lists
|
|
424
|
+
the node's element and block children by type, in document order, so you can
|
|
425
|
+
answer questions the attributes can't, like how many images a gallery holds or
|
|
426
|
+
whether a list item has a nested list. The renderer inserts whatever the block
|
|
427
|
+
returns into the output as is and treats it as trusted HTML, so escape any
|
|
428
|
+
values you interpolate. To set the content mode for a block rule, pass both:
|
|
384
429
|
`rules.node "embed", contains: :blocks do |node| ... end`.
|
|
385
430
|
|
|
386
|
-
|
|
387
|
-
|
|
388
|
-
|
|
389
|
-
|
|
431
|
+
Blocks never run while the document is locked. The renderer finishes the whole
|
|
432
|
+
render inside one read transaction with the GVL released, and only then runs
|
|
433
|
+
the blocks and inserts their output, so a block can safely read the same doc or
|
|
434
|
+
even write to it. If no rule has a block, `to_html` skips that last step.
|
|
390
435
|
|
|
391
|
-
Blocks
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
declarative hashes,
|
|
395
|
-
type (a `Method` responds to `call` like any lambda)
|
|
396
|
-
|
|
436
|
+
Blocks cover whatever the declarative form can't express. `Y::Lexxy` and
|
|
437
|
+
`Y::Tiptap` are both built on this API (`lib/y/lexxy.rb`, `lib/y/tiptap.rb`),
|
|
438
|
+
so it already handles two complete editor schemas. Their simple nodes are
|
|
439
|
+
declarative hashes, and each node with logic is a plain method mapped by node
|
|
440
|
+
type (a `Method` responds to `call` like any lambda). The fixture tests check
|
|
441
|
+
that output against a live editor's.
|
|
397
442
|
|
|
398
443
|
The ProseMirror side also takes custom marks:
|
|
399
444
|
|
|
@@ -403,14 +448,15 @@ tiptap = Y::Tiptap.new(doc) do |rules|
|
|
|
403
448
|
end
|
|
404
449
|
```
|
|
405
450
|
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
for a built-in mark name
|
|
451
|
+
Symbols read from the mark's own attributes. Custom marks wrap outside all the
|
|
452
|
+
built-in marks, and when several custom marks apply to one run of text, they
|
|
453
|
+
nest alphabetically by name. A rule for a built-in mark name such as `"bold"`
|
|
454
|
+
replaces that mark's tag.
|
|
409
455
|
|
|
410
456
|
##### Worked examples
|
|
411
457
|
|
|
412
|
-
|
|
413
|
-
|
|
458
|
+
Here's a video-embed node from an app's Tiptap extension, which the built-in
|
|
459
|
+
schema doesn't know about:
|
|
414
460
|
|
|
415
461
|
```ruby
|
|
416
462
|
tiptap = Y::Tiptap.new(doc) do |rules|
|
|
@@ -422,8 +468,8 @@ tiptap = Y::Tiptap.new(doc) do |rules|
|
|
|
422
468
|
end
|
|
423
469
|
```
|
|
424
470
|
|
|
425
|
-
|
|
426
|
-
has finished, so
|
|
471
|
+
This one resolves mentions against the database. Blocks run after the document
|
|
472
|
+
read has finished, so a block can safely query ActiveRecord or the doc itself:
|
|
427
473
|
|
|
428
474
|
```ruby
|
|
429
475
|
tiptap = Y::Tiptap.new(doc) do |rules|
|
|
@@ -436,8 +482,9 @@ tiptap = Y::Tiptap.new(doc) do |rules|
|
|
|
436
482
|
end
|
|
437
483
|
```
|
|
438
484
|
|
|
439
|
-
|
|
440
|
-
|
|
485
|
+
This one overrides a shipped rule. The shipped rule emits
|
|
486
|
+
`<action-text-attachment>` elements for ActionText to re-render, and the
|
|
487
|
+
override renders Lexxy uploads as `<figure>` and `<img>` markup:
|
|
441
488
|
|
|
442
489
|
```ruby
|
|
443
490
|
lexxy = Y::Lexxy.new(doc) do |rules|
|
|
@@ -452,9 +499,10 @@ lexxy = Y::Lexxy.new(doc) do |rules|
|
|
|
452
499
|
end
|
|
453
500
|
```
|
|
454
501
|
|
|
455
|
-
|
|
456
|
-
element
|
|
457
|
-
itself by its column count
|
|
502
|
+
Some markup depends on structure. Because `node.child_types` lists the node's
|
|
503
|
+
element and block children in document order, a layout container can size
|
|
504
|
+
itself by its column count, and the columns themselves can still use
|
|
505
|
+
declarative rules:
|
|
458
506
|
|
|
459
507
|
```ruby
|
|
460
508
|
tiptap = Y::Tiptap.new(doc) do |rules|
|
|
@@ -465,8 +513,9 @@ tiptap = Y::Tiptap.new(doc) do |rules|
|
|
|
465
513
|
end
|
|
466
514
|
```
|
|
467
515
|
|
|
468
|
-
|
|
469
|
-
around the cursor,
|
|
516
|
+
A block can also look at the rendered content. This one drops the empty
|
|
517
|
+
paragraphs an editor keeps around the cursor, which works because
|
|
518
|
+
`node.content` is already rendered:
|
|
470
519
|
|
|
471
520
|
```ruby
|
|
472
521
|
lexical = Y::Lexical.new(doc) do |rules|
|
|
@@ -476,19 +525,20 @@ lexical = Y::Lexical.new(doc) do |rules|
|
|
|
476
525
|
end
|
|
477
526
|
```
|
|
478
527
|
|
|
479
|
-
For a larger reference, the gem's own editor schemas
|
|
480
|
-
`Y::Lexxy::NODES` in `lib/y/lexxy.rb` (declarative hashes for
|
|
481
|
-
nodes, a plain method
|
|
482
|
-
|
|
528
|
+
For a larger reference, look at the gem's own editor schemas, which are written
|
|
529
|
+
the same way. See `Y::Lexxy::NODES` in `lib/y/lexxy.rb` (declarative hashes for
|
|
530
|
+
the simple nodes, and a plain method mapped with `method(:name)` for each node
|
|
531
|
+
that needs logic, which means galleries, list items, header cells, and both
|
|
532
|
+
attachment types) and
|
|
483
533
|
`Y::Tiptap::NODES` in `lib/y/tiptap.rb` (task lists, mentions, the details
|
|
484
534
|
family).
|
|
485
535
|
|
|
486
536
|
### Protocol codec (module functions)
|
|
487
537
|
|
|
488
|
-
Classifying and unwrapping wire frames
|
|
489
|
-
`Y`
|
|
490
|
-
|
|
491
|
-
the server
|
|
538
|
+
Classifying and unwrapping wire frames needs no state, so these are module
|
|
539
|
+
functions on `Y` and you don't construct anything. The server routes a frame
|
|
540
|
+
without holding any presence or document state. The browser clients keep
|
|
541
|
+
presence, and the server relays awareness frames without reading them.
|
|
492
542
|
|
|
493
543
|
```ruby
|
|
494
544
|
Y.message_kind(frame) # => 0 drop / 1 step1 / 2 update / 3 awareness / 4 query
|
|
@@ -498,55 +548,170 @@ Y.wrap_update(update_bytes) # => wrap a raw doc update as a sync Update frame
|
|
|
498
548
|
|
|
499
549
|
### ActionCable Integration
|
|
500
550
|
|
|
501
|
-
In a Rails app, one generator creates the
|
|
551
|
+
In a Rails app, one generator creates the storage migration. The models and
|
|
552
|
+
`Y::DocumentChannel` come with the gem:
|
|
502
553
|
|
|
503
554
|
```bash
|
|
504
555
|
bin/rails generate yrby:install
|
|
505
556
|
bin/rails db:migrate
|
|
506
557
|
```
|
|
507
558
|
|
|
508
|
-
|
|
509
|
-
|
|
510
|
-
|
|
511
|
-
- **`Y::Document
|
|
512
|
-
|
|
513
|
-
|
|
514
|
-
`
|
|
515
|
-
`
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
a key
|
|
520
|
-
a
|
|
521
|
-
the merged `state` snapshot,
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
(default 64)
|
|
528
|
-
|
|
529
|
-
|
|
530
|
-
|
|
531
|
-
|
|
532
|
-
|
|
533
|
-
|
|
534
|
-
|
|
535
|
-
|
|
536
|
-
`
|
|
537
|
-
|
|
538
|
-
|
|
539
|
-
|
|
540
|
-
|
|
541
|
-
|
|
559
|
+
As with Action Text's `ActionText::RichText`, the models are defined in the
|
|
560
|
+
gem:
|
|
561
|
+
|
|
562
|
+
- **`Y::Document`** stores one row per document, and you can find a row in two
|
|
563
|
+
ways. Channels use `key`, a single opaque, unique string that the app
|
|
564
|
+
sometimes supplies and that nothing parses. Optionally, a row also has a
|
|
565
|
+
polymorphic `record` and a `name` that say which model attribute the document
|
|
566
|
+
backs, where `name` is the attribute name, such as `"body"`. Each record gets
|
|
567
|
+
one document per attribute, the same scheme ActionText::RichText uses, and
|
|
568
|
+
key-only documents leave `record` and `name` nil. Either side can come first.
|
|
569
|
+
`Y::Document.for(record, name)` finds or creates the record binding, derives
|
|
570
|
+
a readable key (`post/1/body`), and adopts any key-only row that already has
|
|
571
|
+
that key, so a channel that writes first and a binding created later end up
|
|
572
|
+
on the same document. The row also holds the merged `state` snapshot, which
|
|
573
|
+
contains only CRDT state. Derived data such as rendered HTML or search text
|
|
574
|
+
is up to your application, and the usual place for it is the channel's
|
|
575
|
+
`on_change`. By default the channel concern calls `.load_state(key)` and
|
|
576
|
+
`.append(key, update)` to read and write the store.
|
|
577
|
+
- **`Y::DocumentUpdate`** holds the uncompacted tail, one delta per row. When
|
|
578
|
+
the tail reaches `compact_every` (default 64), yrby compacts it into `state`
|
|
579
|
+
and deletes those rows. A load reads the snapshot plus the current tail, and
|
|
580
|
+
returns `state` directly when the tail is empty. Compactions of one document
|
|
581
|
+
run one at a time under a per-document row lock, and they skip rows with an
|
|
582
|
+
open causal gap, leaving them as they are until the gap closes. Destroying a
|
|
583
|
+
document also deletes its updates.
|
|
584
|
+
|
|
585
|
+
For encrypted storage, `Y::EncryptedDocument` writes `state` and update
|
|
586
|
+
payloads through Active Record encryption on the same tables, as
|
|
587
|
+
`ActionText::EncryptedRichText` does. You declare it on the model, and the
|
|
588
|
+
model decides whether an attribute is encrypted, so a page or client can't
|
|
589
|
+
change it:
|
|
590
|
+
|
|
591
|
+
```ruby
|
|
592
|
+
class Post < ApplicationRecord
|
|
593
|
+
has_collaborative_document :body, encrypted: true
|
|
594
|
+
end
|
|
595
|
+
```
|
|
596
|
+
|
|
597
|
+
`Y::DocumentChannel` reads that declaration and routes every load and append
|
|
598
|
+
for the attribute through the encrypted class, while attributes without it use
|
|
599
|
+
plain `Y::Document`. In your own channel, point `on_load` and `on_change` at
|
|
600
|
+
`Y::EncryptedDocument`. In both cases you need to configure your app's
|
|
601
|
+
encryption keys and use one access path per document, because the plain classes
|
|
602
|
+
read encrypted rows back as ciphertext.
|
|
603
|
+
|
|
604
|
+
`post.collaborative_document(:body)` returns a bound
|
|
605
|
+
`Y::Collaborative::Attribute` with `load_state`, `append(update)`, `key`, and
|
|
606
|
+
`y_doc`. `y_doc` builds a fresh native `Y::Doc` you can read and render in
|
|
607
|
+
Ruby. For built-in row operations like compaction, call
|
|
608
|
+
`post.collaborative_document(:body).document_row.compact!`. The channel and
|
|
609
|
+
your application code go through the same accessor, which handles encryption
|
|
610
|
+
for both.
|
|
611
|
+
|
|
612
|
+
An attribute uses the key its document row was stored under. Before a row
|
|
613
|
+
exists, the key is `Y::Document.key_for(record, name)`, and computing it
|
|
614
|
+
doesn't create a row. Grants keep their existing scope and lifetime.
|
|
615
|
+
|
|
616
|
+
To check the current user's permissions in addition to the signed grant, give
|
|
617
|
+
the shipped channel an `authorize_document` block. It runs in channel context,
|
|
618
|
+
so it can use `current_user` or any other identifiers your Action Cable
|
|
619
|
+
connection provides:
|
|
620
|
+
|
|
621
|
+
```ruby
|
|
622
|
+
# config/initializers/yrby.rb
|
|
623
|
+
Rails.application.config.to_prepare do
|
|
624
|
+
Y::DocumentChannel.authorize_document do |record, name|
|
|
625
|
+
current_user.present? && record.editable_by?(current_user, attribute: name)
|
|
626
|
+
end
|
|
627
|
+
end
|
|
628
|
+
```
|
|
629
|
+
|
|
630
|
+
`editable_by?` is a policy method in your application, and yrby doesn't provide
|
|
631
|
+
it. The block receives a freshly loaded record and the attribute name as a
|
|
632
|
+
string. If it returns false or nil, the channel rejects the subscription before
|
|
633
|
+
opening a stream or serving any state. An invalid grant is rejected before the
|
|
634
|
+
block is called. Without a block, a valid grant is enough, so the view that
|
|
635
|
+
renders the helper must still require edit permission. If the block raises, the
|
|
636
|
+
channel rejects the subscription and the error goes to your normal error
|
|
637
|
+
handling.
|
|
638
|
+
|
|
639
|
+
The policy runs once, when the client subscribes, and after that the open
|
|
640
|
+
subscription authorizes each message. That's how Action Cable is meant to work,
|
|
641
|
+
and it means a keystroke or cursor move doesn't cost a record load plus your
|
|
642
|
+
own queries. The tradeoff is that if you revoke a permission mid-session, it
|
|
643
|
+
takes effect the next time that client subscribes. You can shorten that window
|
|
644
|
+
in two ways. Every new subscription checks grant expiry, so a short
|
|
645
|
+
`expires_in:` on the tag limits it, as long as you also give the element a
|
|
646
|
+
`refresh:` URL (see below). If your application has to cut off access
|
|
647
|
+
immediately, stop the subscription yourself when the permission changes.
|
|
648
|
+
|
|
649
|
+
Both transports remember the decision for the life of the subscription. Action
|
|
650
|
+
Cable keeps the same channel instance around. AnyCable builds a new one for
|
|
651
|
+
each command, so yrby declares the authorized document as channel state and
|
|
652
|
+
sends it with each RPC, and because anycable-go holds that state, a client
|
|
653
|
+
can't forge it. A frame that arrives without an authorized subscription is
|
|
654
|
+
rejected even if its grant is valid.
|
|
655
|
+
|
|
656
|
+
Every channel does its authorization in one method, `authorized?`, which the
|
|
657
|
+
concern calls when a client subscribes, before it opens a stream or serves any
|
|
658
|
+
state. In a channel you write, you define `authorized?` yourself and it
|
|
659
|
+
receives the document key. The shipped `Y::DocumentChannel` defines it to run
|
|
660
|
+
the `authorize_document` block with the record and the attribute name, or to
|
|
661
|
+
accept any valid grant when there's no block. A subclass can override
|
|
662
|
+
`authorized?` directly and read the located record from `record`.
|
|
663
|
+
|
|
664
|
+
### Grant lifetime and refresh
|
|
665
|
+
|
|
666
|
+
A grant lasts as long as GlobalID's signed-id default, which is one month under
|
|
667
|
+
Rails, and `expires_in:` on the tag shortens it. The grant is part of the
|
|
668
|
+
rendered page, though, and Action Cable resubscribes with it after every
|
|
669
|
+
network drop. If the grant expires before the user is done editing, the editor
|
|
670
|
+
gets blocked at the next reconnect. To avoid that, pair `expires_in:` with
|
|
671
|
+
`refresh:`, a URL the element fetches when a subscription is rejected:
|
|
672
|
+
|
|
673
|
+
```erb
|
|
674
|
+
<%= collaborative_document_tag @post, :body, expires_in: 10.minutes,
|
|
675
|
+
refresh: grant_post_path(@post) %>
|
|
676
|
+
```
|
|
677
|
+
|
|
678
|
+
```ruby
|
|
679
|
+
# config/routes.rb: resources :posts do get :grant, on: :member end
|
|
680
|
+
# app/controllers/posts_controller.rb
|
|
681
|
+
def grant
|
|
682
|
+
@post = current_user.posts.find(params[:id]) # your own authorization, again
|
|
683
|
+
render json: { grant: @post.collaborative_sgid(:body, expires_in: 10.minutes) }
|
|
684
|
+
end
|
|
685
|
+
```
|
|
686
|
+
|
|
687
|
+
That action grants write access, so its check must be at least as strict as the
|
|
688
|
+
page that renders the tag. If it skips authorization, anyone who can reach the
|
|
689
|
+
URL gets a grant, and a short `expires_in:` protects nothing.
|
|
690
|
+
|
|
691
|
+
When a subscription is rejected, the element fetches that URL with the session
|
|
692
|
+
cookie, and the action runs your authorization again. If the response is
|
|
693
|
+
`{ "grant": ... }`, the same session resubscribes with the new grant and keeps
|
|
694
|
+
its document and pending edits. Any other response, a non-2xx status, or a second
|
|
695
|
+
rejection blocks the session as before, and so does a refresh that takes
|
|
696
|
+
longer than 15 seconds. The element doesn't renew grants on a timer, so it
|
|
697
|
+
won't interrupt a healthy open subscription. Every reconnect after expiry is a
|
|
698
|
+
fresh permission check, which is why you'd want a short lifetime in the first
|
|
699
|
+
place.
|
|
700
|
+
|
|
701
|
+
For room-keyed collaboration or other custom channel behavior, generate a
|
|
702
|
+
channel with `bin/rails generate yrby:install --channel` and implement its
|
|
703
|
+
`authorized?(key)`. Custom channels can still use both storage hooks.
|
|
704
|
+
|
|
705
|
+
The migration creates `y_documents` and `y_document_updates`. To rename them,
|
|
706
|
+
edit the generated migration and point `Y::Document.table_name` and
|
|
542
707
|
`Y::DocumentUpdate.table_name` at the new names in an initializer.
|
|
543
708
|
|
|
544
|
-
|
|
709
|
+
You can swap the storage. The channel only needs `on_load` and `on_change`
|
|
545
710
|
answered, and they can point at anything.
|
|
546
711
|
|
|
547
712
|
`include Y::ActionCable` (from the `yrby-rails` gem) is the channel
|
|
548
|
-
integration
|
|
549
|
-
awareness
|
|
713
|
+
integration. It implements the y-websocket protocol over ActionCable, covering
|
|
714
|
+
document sync, awareness, and presence.
|
|
550
715
|
|
|
551
716
|
```ruby
|
|
552
717
|
# app/channels/document_channel.rb
|
|
@@ -557,8 +722,6 @@ class DocumentChannel < ApplicationCable::Channel
|
|
|
557
722
|
on_change { |key, update| Y::Document.append(key, update) } # record, then broadcast
|
|
558
723
|
|
|
559
724
|
def subscribed
|
|
560
|
-
return reject unless authorized?(params[:id])
|
|
561
|
-
|
|
562
725
|
sync_subscribed params[:id]
|
|
563
726
|
end
|
|
564
727
|
|
|
@@ -568,80 +731,112 @@ class DocumentChannel < ApplicationCable::Channel
|
|
|
568
731
|
|
|
569
732
|
private
|
|
570
733
|
|
|
571
|
-
#
|
|
734
|
+
# This denies everyone until you wire it to your app's auth.
|
|
735
|
+
# sync_subscribed rejects the subscription unless this returns true.
|
|
572
736
|
def authorized?(_document_key) = false
|
|
573
737
|
end
|
|
574
738
|
```
|
|
575
739
|
|
|
576
|
-
|
|
577
|
-
|
|
578
|
-
|
|
579
|
-
|
|
580
|
-
for the same document as long as they share the same store and cable adapter.
|
|
740
|
+
For documents that belong to a record, `Y::Collaborative` (which the engine
|
|
741
|
+
includes into Active Record) provides the token flow `authorized?` needs. The
|
|
742
|
+
page creates a signed GlobalID scoped to one attribute, and the channel looks
|
|
743
|
+
up the record from it, so the page decides which document the client gets.
|
|
581
744
|
|
|
582
|
-
|
|
583
|
-
|
|
584
|
-
|
|
585
|
-
|
|
586
|
-
|
|
745
|
+
```erb
|
|
746
|
+
<%# the view picks the document and signs it %>
|
|
747
|
+
<%= tag.div data: { grant: post.collaborative_sgid(:body) } %>
|
|
748
|
+
```
|
|
749
|
+
|
|
750
|
+
```ruby
|
|
751
|
+
# The channel looks up the record from the token. This isn't memoized
|
|
752
|
+
# because AnyCable builds a fresh channel for each command, and a cached
|
|
753
|
+
# record would go stale.
|
|
754
|
+
def authorized?(_key) = record.present? && record.editable_by?(current_user)
|
|
755
|
+
def record = Y::Collaborative.locate(params[:grant], :body)
|
|
756
|
+
```
|
|
757
|
+
|
|
758
|
+
A token created for `:body` only verifies under `:body`'s purpose
|
|
759
|
+
(`"yrby/body"`), and a tampered, expired, or wrong-attribute token returns no
|
|
760
|
+
record.
|
|
761
|
+
|
|
762
|
+
The concern reads and writes through your store. It answers handshakes from
|
|
763
|
+
`on_load`, and it passes document changes through `on_change` before
|
|
764
|
+
broadcasting them. The ActionCable process doesn't keep authoritative state in
|
|
765
|
+
memory, so AnyCable RPC workers, Puma workers, and separate dynos can all
|
|
766
|
+
handle messages for the same document, as long as they share a store and a
|
|
767
|
+
cable adapter.
|
|
768
|
+
|
|
769
|
+
If the yrby-rails models are installed and you declare neither hook, `on_load`
|
|
770
|
+
and `on_change` both use `Y::Document` storage. To use a different store,
|
|
771
|
+
declare both hooks so reads and writes go to the same place. Once a pair is
|
|
772
|
+
configured explicitly, a subclass may override either hook. Outside a
|
|
773
|
+
yrby-rails app there's no default, and until you declare both hooks, the
|
|
774
|
+
channel fails before it can acknowledge or broadcast an edit.
|
|
775
|
+
|
|
776
|
+
Presence is ephemeral. The server relays awareness frames, and `yrby-client`
|
|
777
|
+
sends a best-effort presence-removal frame on disconnect and on `pagehide`. If
|
|
778
|
+
a client can't send that frame, the client-side awareness timeout removes its
|
|
779
|
+
presence.
|
|
587
780
|
|
|
588
781
|
Incoming frames are validated as a single well-formed protocol message before
|
|
589
782
|
anything processes or relays them. Malformed, truncated, multi-message,
|
|
590
|
-
oversized,
|
|
591
|
-
|
|
592
|
-
|
|
783
|
+
oversized, and unknown frames are dropped, so a bad frame can't crash the
|
|
784
|
+
process and one client can't relay garbage that breaks everyone else in the
|
|
785
|
+
room. A Rust panic is caught at the FFI boundary and re-raised as a Ruby
|
|
786
|
+
exception.
|
|
593
787
|
|
|
594
788
|
#### Delivery guarantees
|
|
595
789
|
|
|
596
|
-
|
|
597
|
-
servers:
|
|
790
|
+
These guarantees are the same whether you run one process or hundreds of them
|
|
791
|
+
across many servers:
|
|
598
792
|
|
|
599
793
|
- **The document always converges.** CRDT updates are commutative and
|
|
600
|
-
idempotent, so out-of-order, duplicate,
|
|
601
|
-
the same correct document
|
|
602
|
-
- **An acked update is durable,
|
|
603
|
-
update with a missing dependency
|
|
604
|
-
|
|
605
|
-
|
|
606
|
-
|
|
607
|
-
|
|
608
|
-
|
|
609
|
-
|
|
610
|
-
|
|
611
|
-
must make `on_change` idempotent**. But remember that the CRDT can handle duplicates.
|
|
794
|
+
idempotent, so out-of-order, duplicate, and concurrent delivery all produce
|
|
795
|
+
the same correct document in every deployment, with no coordination.
|
|
796
|
+
- **An acked update is durable, including one that arrived out of order.** The
|
|
797
|
+
server records and acks an update with a missing dependency like any other,
|
|
798
|
+
and the update waits as pending in the document. Some client still holds the
|
|
799
|
+
missing update unacked and keeps retransmitting it until the server records
|
|
800
|
+
it, and then the gap closes. See [Causal gaps](#causal-gaps).
|
|
801
|
+
- **`on_change` runs at least once, and replaying the log always rebuilds the
|
|
802
|
+
document.** yrby calls `on_change` for every update before acking or
|
|
803
|
+
broadcasting it. If you need exactly-once behavior, make `on_change`
|
|
804
|
+
idempotent. The CRDT handles duplicates either way.
|
|
612
805
|
- **A raising `on_change` rejects the update implicitly.** If the block raises,
|
|
613
|
-
the
|
|
614
|
-
|
|
615
|
-
|
|
616
|
-
|
|
617
|
-
|
|
618
|
-
|
|
619
|
-
|
|
620
|
-
`
|
|
806
|
+
the server doesn't ack or broadcast the update, and it doesn't send a
|
|
807
|
+
negative ack either. The client keeps the update pending and retransmits it
|
|
808
|
+
on its timer or on reconnect. That works for transient failures, such as a
|
|
809
|
+
store that was briefly down, where a retry succeeds. A block that raises
|
|
810
|
+
every time (say, a validation that always fails for this edit) gets retried
|
|
811
|
+
forever, because nothing tells the client to stop. Put hard rejections in the
|
|
812
|
+
channel's authorization at subscribe time, before an edit can reach
|
|
813
|
+
`on_change`, and don't raise inside the hook for them.
|
|
621
814
|
- **An over-cap frame is dropped the same silent way.** A frame larger than
|
|
622
|
-
`max_frame_bytes` (default 8 MiB) is dropped before decoding
|
|
623
|
-
broadcast
|
|
624
|
-
update that
|
|
625
|
-
|
|
626
|
-
|
|
627
|
-
|
|
628
|
-
|
|
629
|
-
|
|
630
|
-
|
|
631
|
-
|
|
815
|
+
`max_frame_bytes` (default 8 MiB) is dropped before decoding, with no ack and
|
|
816
|
+
no broadcast, which limits how much work a client can force on the server. A
|
|
817
|
+
real document update that hits the cap gets the same implicit rejection as
|
|
818
|
+
above, so it's never acked and is retransmitted forever. Normal typing won't
|
|
819
|
+
come near the cap, but a large paste, an embedded image, or a big initial
|
|
820
|
+
`SyncStep2` can. The server logs each drop (`warn` for over-cap, `debug` for
|
|
821
|
+
undecodable) with the document key and update id so you can find it, and you
|
|
822
|
+
can override `sync_log_context` on the channel to add a user or connection
|
|
823
|
+
id. Size the cap for your largest expected payload, and reject content that's
|
|
824
|
+
too big before it reaches the channel. The cap is a last line of defense, and
|
|
825
|
+
the client never sees an error from it.
|
|
632
826
|
|
|
633
827
|
#### Causal gaps
|
|
634
828
|
|
|
635
|
-
Yjs updates can arrive out of order
|
|
636
|
-
|
|
637
|
-
|
|
638
|
-
and integrates
|
|
639
|
-
|
|
640
|
-
|
|
641
|
-
|
|
642
|
-
Serving is lossless too,
|
|
643
|
-
full state
|
|
644
|
-
|
|
829
|
+
Yjs updates can arrive out of order, so an update can reach the server before
|
|
830
|
+
the update it depends on. yrby treats that as normal. It records and acks the
|
|
831
|
+
update like any other, the update waits as a pending struct in the document,
|
|
832
|
+
and it integrates once the missing dependency arrives. The write path appends,
|
|
833
|
+
relays, and acks without rebuilding the document, so an update with a gap costs
|
|
834
|
+
the same as any other.
|
|
835
|
+
|
|
836
|
+
Serving is lossless too, as on any Yjs server. `handle_sync_message` serves
|
|
837
|
+
full state with pending structs included, so a peer holds the same pending
|
|
838
|
+
struct and integrates it the same way. Closing the gap takes no special
|
|
839
|
+
machinery, because the missing
|
|
645
840
|
dependency is an update its sender still holds unacked, and at-least-once
|
|
646
841
|
retransmission delivers it. Only compaction excludes pending
|
|
647
842
|
(`compacted_state_update`), because folding a log must not freeze an
|
|
@@ -650,14 +845,14 @@ un-integrable struct into the base state.
|
|
|
650
845
|
The bundled `Y::Document` store handles all of this. If you write your own
|
|
651
846
|
store, keep two things in mind:
|
|
652
847
|
|
|
653
|
-
**1. Load losslessly, and tolerate duplicates.** `on_load` should return
|
|
654
|
-
|
|
848
|
+
**1. Load losslessly, and tolerate duplicates.** `on_load` should return state
|
|
849
|
+
that keeps pending updates, either `encode_state_as_update` or a replay of the
|
|
655
850
|
raw append log. Don't compact with `compacted_state_update` while
|
|
656
|
-
`doc.pending
|
|
657
|
-
|
|
658
|
-
|
|
659
|
-
|
|
660
|
-
size matters, dedup by content hash:
|
|
851
|
+
`doc.pending?` is true, because that strips the pending struct and loses the
|
|
852
|
+
acked edit inside it. `Y::Document` keeps pending rows out of compaction for
|
|
853
|
+
this reason. When an ack gets lost, the client resends an update the store
|
|
854
|
+
already has. Replay still converges because CRDT apply is idempotent, so
|
|
855
|
+
deduping is optional. If log size matters, dedup by content hash:
|
|
661
856
|
|
|
662
857
|
```ruby
|
|
663
858
|
class DocumentStore
|
|
@@ -688,17 +883,18 @@ class DocumentStore
|
|
|
688
883
|
end
|
|
689
884
|
```
|
|
690
885
|
|
|
691
|
-
**2. Watch for gaps that
|
|
692
|
-
|
|
693
|
-
|
|
694
|
-
acked, and every join or reconnect handshake
|
|
695
|
-
the server hasn't integrated, so any client holding the dependency
|
|
696
|
-
it
|
|
697
|
-
supply, and that
|
|
698
|
-
document key whenever a document is loaded to serve state
|
|
699
|
-
|
|
700
|
-
oldest open gap
|
|
701
|
-
|
|
886
|
+
**2. Watch for gaps that don't close.** An open gap is easy to miss, because
|
|
887
|
+
the pending edit doesn't show up in the document until its dependency arrives.
|
|
888
|
+
Usually the gap closes by itself. The sender retransmits the missing update
|
|
889
|
+
until it's acked, and on every join or reconnect handshake the client sends
|
|
890
|
+
everything the server hasn't integrated, so any client holding the dependency
|
|
891
|
+
supplies it by connecting. The gap worth alerting on is one that no live client
|
|
892
|
+
can supply, and that's what the `on_gap` hook is for. It fires with the
|
|
893
|
+
document key whenever a document is loaded to serve state while a gap is open.
|
|
894
|
+
Use it to emit a metric, such as a pending-document count or the age of the
|
|
895
|
+
oldest open gap, so you can see a gap that isn't closing. yrby also logs gaps
|
|
896
|
+
at `info`. Errors raised in the hook are swallowed, so a broken metrics call
|
|
897
|
+
can't break
|
|
702
898
|
frame handling.
|
|
703
899
|
|
|
704
900
|
```ruby
|
|
@@ -711,28 +907,32 @@ end
|
|
|
711
907
|
|
|
712
908
|
#### Multi-process deployments
|
|
713
909
|
|
|
714
|
-
Most Rails apps run several processes, and any of them might
|
|
715
|
-
Two
|
|
910
|
+
Most Rails apps run several processes, and any of them might end up serving a
|
|
911
|
+
given document. Two things keep them consistent.
|
|
716
912
|
|
|
717
|
-
Broadcasts
|
|
718
|
-
|
|
913
|
+
Broadcasts go between processes through the Action Cable adapter, so use
|
|
914
|
+
`redis`, `solid_cable`, or another cross-process adapter, since `async` only
|
|
915
|
+
works inside one process. With one of those adapters in place, a change
|
|
719
916
|
on one process reaches clients on all of them.
|
|
720
917
|
|
|
721
918
|
Every process rebuilds document state from the durable store through `on_load`.
|
|
722
|
-
|
|
723
|
-
|
|
724
|
-
|
|
919
|
+
Changes are still recorded before they're broadcast, so whichever process
|
|
920
|
+
receives a change writes it to the shared store before any client on any
|
|
921
|
+
process sees it.
|
|
725
922
|
|
|
726
|
-
`bun multiprocess.mjs`
|
|
727
|
-
|
|
923
|
+
In the demo, `bun multiprocess.mjs` runs clients across two processes and
|
|
924
|
+
checks that the documents converge, that fresh reads work on both processes,
|
|
925
|
+
that presence reaches clients on the other process, and that both processes
|
|
926
|
+
write to one shared log.
|
|
728
927
|
|
|
729
928
|
##### AnyCable
|
|
730
929
|
|
|
731
|
-
`yrby`
|
|
930
|
+
`yrby` supports AnyCable end to end.
|
|
732
931
|
|
|
733
|
-
The demo
|
|
734
|
-
|
|
735
|
-
yrby client provider, cross-process reads, and concurrent
|
|
932
|
+
The demo tests it against a real anycable-go server and RPC server in
|
|
933
|
+
`frontend/anycable_probe.mjs` and `anycable_concurrent.mjs`, covering liveness,
|
|
934
|
+
the yrby client provider, cross-process reads, and convergence under concurrent
|
|
935
|
+
editing.
|
|
736
936
|
|
|
737
937
|
##### Demo
|
|
738
938
|
|
|
@@ -741,8 +941,8 @@ app using the yrby provider, with end-to-end tests.
|
|
|
741
941
|
|
|
742
942
|
#### Record Before Distribute
|
|
743
943
|
|
|
744
|
-
Every document change
|
|
745
|
-
|
|
944
|
+
Every document change goes through your `on_change` handler before it's
|
|
945
|
+
broadcast, and the handler is where you record it durably:
|
|
746
946
|
|
|
747
947
|
```ruby
|
|
748
948
|
class DocumentChannel < ApplicationCable::Channel
|
|
@@ -759,21 +959,21 @@ class DocumentChannel < ApplicationCable::Channel
|
|
|
759
959
|
end
|
|
760
960
|
```
|
|
761
961
|
|
|
762
|
-
If the
|
|
763
|
-
applied
|
|
764
|
-
of every change.
|
|
765
|
-
document can both record (at
|
|
766
|
-
duplicate record replays to the same document.
|
|
962
|
+
If the handler raises (say the store is down), the change is rejected, so it
|
|
963
|
+
isn't applied or sent to anyone. The cost is a synchronous durable write on the
|
|
964
|
+
path of every change. The gem doesn't take a per-document lock, so two
|
|
965
|
+
concurrent writes to one document can both record (at least once). Because CRDT
|
|
966
|
+
apply is idempotent, a duplicate record replays to the same document.
|
|
767
967
|
|
|
768
|
-
The demo wires `on_change` to a durable Postgres-backed log by default, and
|
|
769
|
-
end to end that the log alone rebuilds the document.
|
|
968
|
+
The demo wires `on_change` to a durable Postgres-backed log by default, and
|
|
969
|
+
checks end to end that the log alone rebuilds the document.
|
|
770
970
|
|
|
771
971
|
#### Ephemeral documents (no database)
|
|
772
972
|
|
|
773
|
-
`on_load` and `on_change` are plain blocks, and
|
|
774
|
-
|
|
775
|
-
|
|
776
|
-
|
|
973
|
+
`on_load` and `on_change` are plain blocks, and they don't have to touch a
|
|
974
|
+
database. Some documents only need to last for a session, such as a scratchpad,
|
|
975
|
+
live form state, or a draft you save on submit. For those, the store can be
|
|
976
|
+
connection state that's sent with each request:
|
|
777
977
|
|
|
778
978
|
```ruby
|
|
779
979
|
class ScratchpadChannel < ApplicationCable::Channel
|
|
@@ -790,6 +990,11 @@ class ScratchpadChannel < ApplicationCable::Channel
|
|
|
790
990
|
|
|
791
991
|
def subscribed = sync_subscribed(params[:id])
|
|
792
992
|
def receive(data) = sync_receive(data, params[:id])
|
|
993
|
+
|
|
994
|
+
private
|
|
995
|
+
|
|
996
|
+
# Each subscriber gets its own scratchpad on this connection.
|
|
997
|
+
def authorized?(_key) = true
|
|
793
998
|
end
|
|
794
999
|
```
|
|
795
1000
|
|
|
@@ -815,65 +1020,69 @@ class ScratchpadChannel < ApplicationCable::Channel
|
|
|
815
1020
|
|
|
816
1021
|
def subscribed = sync_subscribed(params[:id])
|
|
817
1022
|
def receive(data) = sync_receive(data, params[:id])
|
|
1023
|
+
|
|
1024
|
+
private
|
|
1025
|
+
|
|
1026
|
+
def authorized?(_key) = true # per-connection scratchpad, see above
|
|
818
1027
|
end
|
|
819
1028
|
```
|
|
820
1029
|
|
|
821
|
-
Both hooks run in the channel instance
|
|
822
|
-
anything the channel can
|
|
823
|
-
|
|
824
|
-
|
|
825
|
-
|
|
826
|
-
|
|
827
|
-
|
|
828
|
-
|
|
829
|
-
the full delivery contract
|
|
830
|
-
|
|
831
|
-
has never seen
|
|
832
|
-
that client (which always
|
|
833
|
-
|
|
834
|
-
|
|
835
|
-
|
|
836
|
-
|
|
1030
|
+
Both hooks run in the channel instance through `instance_exec`, so they can use
|
|
1031
|
+
anything the channel can. `sync_receive` rebuilds the document from `on_load`
|
|
1032
|
+
on every update, which is why the store can sit on the connection. On Action
|
|
1033
|
+
Cable the channel instance lasts as long as the connection, so an instance
|
|
1034
|
+
variable is all the store you need. Merging with `compacted_state_update` keeps
|
|
1035
|
+
the store at one blob so it doesn't grow into an update log.
|
|
1036
|
+
|
|
1037
|
+
Because the store is per connection, this pattern has limits. A single writer
|
|
1038
|
+
gets the full delivery contract without any database. When several people edit
|
|
1039
|
+
at the same time, one client's update can depend on edits its own connection
|
|
1040
|
+
has never seen. That update is recorded as pending, and the next handshake with
|
|
1041
|
+
that client (which always has the full document) supplies the missing state and
|
|
1042
|
+
closes the gap. The document still converges, but heavy concurrent editing
|
|
1043
|
+
leaves more pending between handshakes than a shared store would. On AnyCable,
|
|
1044
|
+
watch the payload size as well. The blob is sent with every message, so this
|
|
1045
|
+
only makes sense for small documents.
|
|
837
1046
|
|
|
838
1047
|
Durability is the connection plus the browsers. A reconnecting client re-seeds
|
|
839
|
-
an empty server through the ordinary sync handshake, so the document survives
|
|
840
|
-
server
|
|
841
|
-
shared across clients on a single-process deployment, the
|
|
842
|
-
a class-level `Concurrent::Map
|
|
843
|
-
coherent
|
|
1048
|
+
an empty server through the ordinary sync handshake, so the document survives a
|
|
1049
|
+
server restart as long as some client still has it. If an ephemeral document
|
|
1050
|
+
needs to be shared across clients on a single-process deployment, point the
|
|
1051
|
+
same two hooks at a class-level `Concurrent::Map`. That version is no longer
|
|
1052
|
+
coherent once you run more than one process.
|
|
844
1053
|
|
|
845
1054
|
#### Reliable delivery (acks)
|
|
846
1055
|
|
|
847
1056
|
yrby document delivery is ack-tracked. Browser document updates carry an
|
|
848
|
-
`"id"`, and the server replies `{ "ack": <id> }`
|
|
849
|
-
|
|
850
|
-
|
|
1057
|
+
`"id"`, and the server replies `{ "ack": <id> }` after `on_change` succeeds.
|
|
1058
|
+
The server records and acks every decodable document update, including one that
|
|
1059
|
+
arrives out of order.
|
|
851
1060
|
|
|
852
1061
|
```
|
|
853
1062
|
client -> server { "update": "<base64 update>", "id": 42 }
|
|
854
1063
|
server -> client { "ack": 42 } # update accepted; safe to forget
|
|
855
1064
|
```
|
|
856
1065
|
|
|
857
|
-
`yrby-client`'s `ActionCableProvider` handles this
|
|
858
|
-
|
|
859
|
-
|
|
860
|
-
one `{ ack: id }`
|
|
861
|
-
|
|
862
|
-
|
|
1066
|
+
`yrby-client`'s `ActionCableProvider` handles this for you. It queues the
|
|
1067
|
+
unacknowledged tail of local document updates and sends it merged into one
|
|
1068
|
+
causally complete delta, using the highest sequence in the batch as the id, so
|
|
1069
|
+
one `{ ack: id }` confirms everything up to it. Because CRDT apply is
|
|
1070
|
+
idempotent, resending an update the server already has does no harm, and the
|
|
1071
|
+
server acks it again. Awareness is ephemeral and isn't acked.
|
|
863
1072
|
|
|
864
|
-
|
|
865
|
-
|
|
866
|
-
See `yrby-client` for the client-side awareness API.
|
|
1073
|
+
The browser clients manage presence (cursors, selections). The server doesn't
|
|
1074
|
+
set or hold presence state, and it relays awareness frames without reading
|
|
1075
|
+
them. See `yrby-client` for the client-side awareness API.
|
|
867
1076
|
|
|
868
1077
|
## Thread Safety
|
|
869
1078
|
|
|
870
|
-
|
|
871
|
-
|
|
872
|
-
locking.
|
|
1079
|
+
You can share a `Doc` across Ruby threads. Puma workers, ActionCable connection
|
|
1080
|
+
threads, and background jobs can all use the same one concurrently, with no
|
|
1081
|
+
locking in your code.
|
|
873
1082
|
|
|
874
1083
|
`test/thread_safety_test.rb` runs shared docs, the full sync handshake, and
|
|
875
|
-
fan-in sync across 8 threads
|
|
876
|
-
convergence.
|
|
1084
|
+
fan-in sync across 8 concurrent threads and checks that the interleaving
|
|
1085
|
+
doesn't change convergence.
|
|
877
1086
|
|
|
878
1087
|
### Parallelism (GVL release)
|
|
879
1088
|
|
|
@@ -881,21 +1090,21 @@ Every method that does real CRDT work (applying updates, encoding state,
|
|
|
881
1090
|
handling sync messages) releases Ruby's Global VM Lock
|
|
882
1091
|
(`rb_thread_call_without_gvl`) while the native code runs. That buys two things.
|
|
883
1092
|
|
|
884
|
-
CRDT work runs in parallel across Ruby threads on MRI,
|
|
885
|
-
JRuby
|
|
886
|
-
speedup applying a
|
|
887
|
-
couldn't beat serial time.
|
|
888
|
-
|
|
889
|
-
|
|
890
|
-
the doc's write lock
|
|
891
|
-
|
|
892
|
-
|
|
893
|
-
Each
|
|
894
|
-
|
|
895
|
-
|
|
896
|
-
|
|
897
|
-
order can't deadlock.
|
|
898
|
-
|
|
1093
|
+
First, CRDT work runs in parallel across Ruby threads on MRI, with no need for
|
|
1094
|
+
JRuby or TruffleRuby. `bench/parallelism_bench.rb` measures more than a 2x
|
|
1095
|
+
wall-clock speedup when applying a roughly 900 KB update concurrently. Native
|
|
1096
|
+
code that held the GVL couldn't beat serial time.
|
|
1097
|
+
|
|
1098
|
+
Second, a slow operation can't stall the VM. A thread applying a large update
|
|
1099
|
+
holds the doc's write lock but not the GVL, so other Ruby threads keep running
|
|
1100
|
+
while it works.
|
|
1101
|
+
|
|
1102
|
+
Each of those methods copies the Ruby byte strings, releases the GVL, does the
|
|
1103
|
+
yrs work (taking and releasing the native locks inside that closure), takes the
|
|
1104
|
+
GVL back, and then builds the Ruby objects. Ruby APIs are only called while
|
|
1105
|
+
holding the GVL, and no native lock is held while reacquiring it, so the lock
|
|
1106
|
+
order can't deadlock. A panic in native code is caught and re-raised as a Ruby
|
|
1107
|
+
exception.
|
|
899
1108
|
|
|
900
1109
|
## Message Type Constants
|
|
901
1110
|
|