yrby 0.7.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +71 -0
- data/Cargo.lock +29 -29
- data/README.md +582 -380
- data/ext/yrby/Cargo.toml +3 -3
- data/ext/yrby/crates/html-core/Cargo.toml +5 -4
- data/ext/yrby/crates/html-core/src/lib.rs +16 -20
- data/ext/yrby/crates/lexical-html/Cargo.toml +5 -4
- data/ext/yrby/crates/lexical-html/src/lib.rs +57 -48
- data/ext/yrby/crates/prosemirror-html/Cargo.toml +5 -4
- data/ext/yrby/crates/prosemirror-html/src/lib.rs +151 -104
- data/ext/yrby/src/lib.rs +30 -15
- data/ext/yrby/src/protocol.rs +17 -17
- data/ext/yrby/src/read.rs +137 -26
- data/lib/generators/yrby/install/install_generator.rb +24 -13
- data/lib/generators/yrby/install/templates/document_channel.rb +4 -7
- data/lib/generators/yrby/tables/tables_generator.rb +7 -4
- data/lib/generators/yrby/tables/templates/create_y_tables.rb +1 -1
- data/lib/y/collaborative/attribute.rb +53 -0
- data/lib/y/collaborative/helper.rb +48 -0
- data/lib/y/collaborative.rb +98 -0
- data/lib/y/lexxy.rb +2 -2
- data/lib/y/rendering.rb +15 -15
- data/lib/y/tiptap.rb +3 -3
- data/lib/y/version.rb +1 -1
- data/lib/y.rb +1 -1
- metadata +4 -9
- data/ext/yrby/target/debug/build/clang-sys-e3ae45bd384f74c3/out/common.rs +0 -355
- data/ext/yrby/target/debug/build/clang-sys-e3ae45bd384f74c3/out/dynamic.rs +0 -276
- data/ext/yrby/target/debug/build/clang-sys-e3ae45bd384f74c3/out/macros.rs +0 -49
- data/ext/yrby/target/debug/build/rb-sys-4407948463231c4f/out/bindings-0.9.128-mri-arm64-darwin23-3.4.7.rs +0 -8934
- data/ext/yrby/target/debug/build/rb-sys-dbeea42737529c2d/out/bindings-0.9.128-mri-arm64-darwin23-3.4.7.rs +0 -8934
- data/ext/yrby/target/debug/build/serde-58ea0ee887cc2602/out/private.rs +0 -6
- data/ext/yrby/target/debug/build/serde_core-41f407c21c1f205e/out/private.rs +0 -5
- data/ext/yrby/target/debug/build/thiserror-0f1416a82ff26f22/out/private.rs +0 -5
data/README.md
CHANGED
|
@@ -2,60 +2,87 @@
|
|
|
2
2
|
|
|
3
3
|
[](https://github.com/jpcamara/yrby/actions/workflows/ci.yml)
|
|
4
4
|
|
|
5
|
-
yrby (pronounced "yer-bee") makes Rails a
|
|
5
|
+
yrby (pronounced "yer-bee") makes Rails a Yjs backend. It binds
|
|
6
6
|
[y-crdt](https://github.com/y-crdt/y-crdt), the Rust engine behind Y.js, into
|
|
7
|
-
Ruby, and
|
|
7
|
+
Ruby, and on top of that it builds a sync server for Action
|
|
8
8
|
Cable and AnyCable, a browser provider, and server-side reading and rendering
|
|
9
|
-
of the documents.
|
|
10
|
-
|
|
9
|
+
of the documents. You get real-time collaboration in a Rails app with no Node
|
|
10
|
+
process to run.
|
|
11
11
|
|
|
12
12
|

|
|
13
13
|
|
|
14
|
-
On the server, `yrby-rails` implements the
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
([Delivery guarantees](#delivery-guarantees))
|
|
14
|
+
On the server, `yrby-rails` implements the y-websocket protocol (document sync
|
|
15
|
+
and presence) as a channel concern. The server writes every update to your
|
|
16
|
+
store before it acknowledges the update or sends it to other clients, which
|
|
17
|
+
most Yjs servers don't do. Replaying the store rebuilds the document, however
|
|
18
|
+
many processes you run. ([Delivery guarantees](#delivery-guarantees))
|
|
20
19
|
|
|
21
20
|
In the browser, `yrby-client`'s `ActionCableProvider` connects anything that
|
|
22
21
|
speaks Yjs. The demo app runs four rich text editors, and CI drives each one
|
|
23
22
|
in real Chrome: Tiptap, [Lexxy](https://www.npmjs.com/package/lexxy-realtime),
|
|
24
23
|
Rhino Editor, and CodeMirror. The same channel also syncs Yjs shapes with no
|
|
25
24
|
editor at all: a whiteboard on a `Y.Map`, a kanban board on a `Y.Array`, a
|
|
26
|
-
|
|
25
|
+
form filled in together. ([Editors](#editors))
|
|
27
26
|
|
|
28
27
|
In Ruby, the documents are readable without a browser. `Doc#read_text` and
|
|
29
|
-
`Doc#read_map`
|
|
30
|
-
`Y::Tiptap` and `Y::Lexxy` render a document to HTML
|
|
31
|
-
|
|
32
|
-
|
|
28
|
+
`Doc#read_map` rebuild the contents for search, validation, and exports.
|
|
29
|
+
`Y::Tiptap` and `Y::Lexxy` render a document to HTML that matches the editor's
|
|
30
|
+
own serializer byte for byte. You can add rules for your app's custom nodes,
|
|
31
|
+
and you can save the output directly to ActionText.
|
|
32
|
+
([Rendering to HTML](#rendering-to-html))
|
|
33
33
|
|
|
34
34
|
Underneath, the core is built for a production Rails deployment. A `Doc` is
|
|
35
35
|
thread-safe across Puma and ActionCable threads. Native CRDT work runs with
|
|
36
|
-
the GVL released, so it
|
|
36
|
+
the GVL released, so it runs in parallel on MRI. Incoming frames are validated
|
|
37
37
|
before anything processes them, and multi-process and AnyCable setups are
|
|
38
38
|
tested end to end. ([Thread Safety](#thread-safety))
|
|
39
39
|
|
|
40
|
-
|
|
40
|
+
You add one view helper, on a page where the user can already edit the record:
|
|
41
41
|
|
|
42
|
-
```
|
|
43
|
-
|
|
44
|
-
|
|
42
|
+
```erb
|
|
43
|
+
<%= collaborative_document_tag @post, :body %>
|
|
44
|
+
```
|
|
45
45
|
|
|
46
|
-
|
|
47
|
-
|
|
46
|
+
The helper renders a signed grant for that record and attribute, much as
|
|
47
|
+
`turbo_stream_from` signs its stream names. The client subscribes with the
|
|
48
|
+
grant and the attribute name. The gem's `Y::DocumentChannel` verifies the
|
|
49
|
+
grant, then saves every change as `Y::Document` rows before acknowledging it,
|
|
50
|
+
so you don't have to write a channel yourself.
|
|
48
51
|
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
+
The tag renders an element that connects by itself once you import it. When the
|
|
53
|
+
document has synced, your listener receives it and passes it to your editor
|
|
54
|
+
binding:
|
|
55
|
+
|
|
56
|
+
```js
|
|
57
|
+
import "yrby-client/element"
|
|
58
|
+
|
|
59
|
+
document.addEventListener("yrby:synced", ({ target, detail }) => {
|
|
60
|
+
const editor = bindYourEditor(target, detail.doc, detail.provider)
|
|
61
|
+
detail.signal.addEventListener("abort", () => editor.destroy(), { once: true })
|
|
62
|
+
})
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
If the user leaves the page with edits still unsent, the session keeps them and
|
|
66
|
+
finishes delivering. Your binding cleans up when the abort signal fires, and if
|
|
67
|
+
nothing is pending, the next visit loads the document from Rails. The [client
|
|
68
|
+
lifecycle and recovery contract](packages/client/README.md#document-sessions)
|
|
69
|
+
has the details.
|
|
70
|
+
|
|
71
|
+
The document is stored as rows in your database, and you can read it back in
|
|
72
|
+
Ruby:
|
|
73
|
+
|
|
74
|
+
```ruby
|
|
75
|
+
doc = post.collaborative_document(:body).y_doc
|
|
76
|
+
doc.read_text("content") # or Y::Lexxy.new(doc).to_html for rich text
|
|
52
77
|
```
|
|
53
78
|
|
|
54
79
|
Install the gem and the npm package:
|
|
55
80
|
|
|
56
81
|
```
|
|
57
82
|
gem install yrby-rails # depends on yrby
|
|
58
|
-
npm install yrby-client
|
|
83
|
+
npm install yrby-client yjs y-protocols @rails/actioncable
|
|
84
|
+
|
|
85
|
+
bin/rails generate yrby:install && bin/rails db:migrate
|
|
59
86
|
```
|
|
60
87
|
|
|
61
88
|
## Contents
|
|
@@ -84,51 +111,54 @@ npm install yrby-client
|
|
|
84
111
|
|
|
85
112
|
## Scope
|
|
86
113
|
|
|
87
|
-
`yrby` binds
|
|
88
|
-
documents
|
|
89
|
-
the Ruby side treats a document as opaque CRDT state
|
|
90
|
-
sync handshakes, and records deltas without
|
|
91
|
-
|
|
92
|
-
and `Doc#read_map`
|
|
114
|
+
`yrby` binds the parts of `y-crdt` you need to sync and persist collaborative
|
|
115
|
+
documents, which are a `Doc`, awareness, and the y-websocket protocol
|
|
116
|
+
primitives. By default the Ruby side treats a document as opaque CRDT state. It
|
|
117
|
+
applies updates, answers sync handshakes, and records deltas without reading
|
|
118
|
+
the contents, and the browser editor decides what shape the document has. When
|
|
119
|
+
you do need to look inside, `Doc#read_text` and `Doc#read_map` rebuild it in
|
|
120
|
+
Ruby.
|
|
93
121
|
|
|
94
122
|
## Durability and delivery
|
|
95
123
|
|
|
96
|
-
The
|
|
124
|
+
The API is small, and most of the work went into durability, resiliency, delivery
|
|
97
125
|
guarantees, correctness, and thread safety.
|
|
98
126
|
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
-
|
|
102
|
-
|
|
103
|
-
only acks
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
retransmitting it until it
|
|
127
|
+
`yrby` adds two opinionated defaults on top of normal Yjs syncing:
|
|
128
|
+
|
|
129
|
+
- yrby acknowledges updates. The `ActionCableProvider` in `yrby-client` keeps
|
|
130
|
+
resending an update until the server acks it, and
|
|
131
|
+
[`yrby-rails`](https://rubygems.org/gems/yrby-rails) only acks an update
|
|
132
|
+
after it's durably recorded. Together that gives you at-least-once delivery,
|
|
133
|
+
and because CRDT updates are idempotent, a duplicate does nothing.
|
|
134
|
+
- yrby expects causal gaps, where an update arrives before another update it
|
|
135
|
+
depends on, and it records and acks that update like any other. The sender of
|
|
136
|
+
the missing update keeps retransmitting it until it's acked, and the gap
|
|
137
|
+
closes when it arrives. `Doc#pending?` and the `on_gap` hook
|
|
109
138
|
tell you when a document is waiting on a missing update.
|
|
110
139
|
([Causal gaps](#causal-gaps))
|
|
111
140
|
|
|
112
141
|
## What about [yrb](https://github.com/y-crdt/yrb)?
|
|
113
142
|
|
|
114
|
-
`yrb` has a much larger interface
|
|
115
|
-
|
|
116
|
-
inspiration for
|
|
117
|
-
on top of it.
|
|
143
|
+
`yrb` has a much larger interface, with most of the Yjs type system (shared
|
|
144
|
+
text, arrays, maps, XML) for building and querying documents in Ruby. It was a
|
|
145
|
+
big inspiration for me in using Yjs from Ruby and Rails, and I considered
|
|
146
|
+
building on top of it. I went with `yrby` for a few reasons:
|
|
118
147
|
|
|
119
|
-
- `yrb`
|
|
120
|
-
|
|
121
|
-
- [It isn't thread-safe](https://github.com/y-crdt/yrb/issues/72)
|
|
122
|
-
|
|
123
|
-
- It's a much larger set of features to maintain,
|
|
124
|
-
majority of
|
|
148
|
+
- `yrb` started as an experiment for GitLab, and its original author has mostly
|
|
149
|
+
moved on to other projects, so it's largely unmaintained.
|
|
150
|
+
- [It isn't thread-safe](https://github.com/y-crdt/yrb/issues/72), and it
|
|
151
|
+
segfaults in threaded environments like ActionCable.
|
|
152
|
+
- It's a much larger set of features to maintain, and most people don't need
|
|
153
|
+
them. The vast majority of Y.js documents are only ever manipulated in the
|
|
154
|
+
browser.
|
|
125
155
|
|
|
126
156
|
## Testing
|
|
127
157
|
|
|
128
158
|
Ruby and Rust unit tests cover the core. CI also runs the npm client tests and a
|
|
129
159
|
Rails demo smoke slice against the real ActionCable stack. The demo includes
|
|
130
160
|
heavier local suites for hostile input, crash recovery, multi-browser editing,
|
|
131
|
-
AnyCable, and load testing. The benchmark
|
|
161
|
+
AnyCable, and load testing. The benchmark number below is from a single laptop.
|
|
132
162
|
Issues and PRs are welcome.
|
|
133
163
|
|
|
134
164
|
## Install
|
|
@@ -142,10 +172,11 @@ gem "yrby"
|
|
|
142
172
|
gem "yrby-rails"
|
|
143
173
|
```
|
|
144
174
|
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
on Linux x86_64 and macOS arm64.
|
|
148
|
-
|
|
175
|
+
yrby needs Ruby 3.4 or newer. Releases include precompiled gems for Ruby 3.4
|
|
176
|
+
and 4.0 on the supported Ruby platforms, and the release workflow smoke-tests
|
|
177
|
+
the native builds on Linux x86_64 and macOS arm64. You only need
|
|
178
|
+
[Rust](https://rustup.rs) when no precompiled gem matches your platform and the
|
|
179
|
+
gem builds from source.
|
|
149
180
|
|
|
150
181
|
To work on the gem itself:
|
|
151
182
|
|
|
@@ -163,16 +194,17 @@ The rest of the dev setup, plus the demo, is in [CONTRIBUTING.md](CONTRIBUTING.m
|
|
|
163
194
|
- The ActionCable concern and a quickstart are [below](#actioncable-integration).
|
|
164
195
|
- [`examples/actioncable-demo`](examples/actioncable-demo): a runnable Rails +
|
|
165
196
|
Tiptap app with collaborative cursors, the AnyCable setup, a Postgres store,
|
|
166
|
-
and the test
|
|
197
|
+
and the test and load suites.
|
|
167
198
|
- [CHANGELOG.md](CHANGELOG.md) and [CONTRIBUTING.md](CONTRIBUTING.md).
|
|
168
199
|
|
|
169
200
|
## Editors
|
|
170
201
|
|
|
171
|
-
yrby syncs opaque Yjs updates, so
|
|
172
|
-
|
|
202
|
+
yrby syncs opaque Yjs updates, so any editor with a Yjs binding works. The demo
|
|
203
|
+
app runs four of them, and CI drives each one in real Chrome to check
|
|
173
204
|
concurrent typing with every keystroke accounted for, remote cursors,
|
|
174
|
-
local-only undo, and
|
|
175
|
-
editor's own serializer. Each page is a working integration
|
|
205
|
+
local-only undo, and that the server-side renderers produce the same HTML as
|
|
206
|
+
the editor's own serializer. Each demo page is a working integration you can
|
|
207
|
+
copy from:
|
|
176
208
|
|
|
177
209
|
| Editor | Yjs binding | Demo code |
|
|
178
210
|
|---|---|---|
|
|
@@ -181,11 +213,11 @@ editor's own serializer. Each page is a working integration to copy from:
|
|
|
181
213
|
| [Rhino Editor](https://github.com/KonnorRogers/rhino-editor) (Tiptap 3) | `@tiptap/extension-collaboration` + `-caret` | [`rhino.js`](examples/actioncable-demo/frontend/src/rhino.js) |
|
|
182
214
|
| [CodeMirror 6](https://codemirror.net) | `y-codemirror.next` | [`codemirror.js`](examples/actioncable-demo/frontend/src/codemirror.js) |
|
|
183
215
|
|
|
184
|
-
The demo also syncs plain Yjs shapes with no editor
|
|
185
|
-
|
|
186
|
-
same channel. The demo README's "Using this in your own app" section has
|
|
187
|
-
|
|
188
|
-
|
|
216
|
+
The demo also syncs plain Yjs shapes with no editor (a whiteboard on a
|
|
217
|
+
`Y.Map`, a kanban board on a `Y.Array`, a form filled in together) over the
|
|
218
|
+
same channel. The demo README's "Using this in your own app" section has the
|
|
219
|
+
integration recipe, and its `NoteMaterializer` shows how to render a document
|
|
220
|
+
to ActionText on the server with `Y::Tiptap` or `Y::Lexxy`.
|
|
189
221
|
|
|
190
222
|
## Usage
|
|
191
223
|
|
|
@@ -219,43 +251,46 @@ doc.handle_sync_message(data) # => [msg_type, sync_type, response]; answers
|
|
|
219
251
|
|
|
220
252
|
### Reading document contents
|
|
221
253
|
|
|
222
|
-
|
|
254
|
+
Rebuild a document on the server for search, exports, emails, or SSR, with no
|
|
223
255
|
Node process:
|
|
224
256
|
|
|
225
257
|
```ruby
|
|
226
258
|
doc.read_text("prosemirror") # => plain text of a Y.Text root, or nil
|
|
227
259
|
doc.read_xml("root") # => text of an XML root, one block per line
|
|
228
260
|
doc.read_map("state") # => a Y.Map root as a JSON string; JSON.parse it
|
|
261
|
+
doc.read_array("cards") # => a Y.Array root as a JSON string; JSON.parse it
|
|
229
262
|
```
|
|
230
263
|
|
|
231
264
|
### Pending structs and gap-free state
|
|
232
265
|
|
|
233
266
|
If a doc applies an update whose causally-prior update is missing (a "gappy"
|
|
234
|
-
update), yrs
|
|
235
|
-
|
|
236
|
-
missing dependency later
|
|
267
|
+
update), yrs holds it as a **pending** struct and the integrated state vector
|
|
268
|
+
doesn't move. yrs keeps the pending block as a recovery buffer and integrates
|
|
269
|
+
it if the missing dependency arrives later. `Doc#pending?` tells you when a doc
|
|
270
|
+
is in this state.
|
|
237
271
|
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
272
|
+
`handle_sync_message` answers `SyncStep1` with the doc's full state, pending
|
|
273
|
+
structs included, which is what Y.js's `encodeStateAsUpdate` does too. A peer
|
|
274
|
+
that receives it holds the pending struct and integrates it the same way this
|
|
275
|
+
doc would. The one place pending structs must be left out is a compacted
|
|
276
|
+
snapshot:
|
|
243
277
|
|
|
244
278
|
- `Doc#compacted_state_update` returns a gap-free full-state update for
|
|
245
279
|
compaction. Folding a log into one blob would otherwise freeze an
|
|
246
|
-
un-integrable struct into the base state
|
|
247
|
-
doc keeps its pending.
|
|
248
|
-
- `encode_state_as_update`
|
|
249
|
-
|
|
280
|
+
un-integrable struct into the base state permanently. The call doesn't modify
|
|
281
|
+
the doc, which keeps its pending structs.
|
|
282
|
+
- `encode_state_as_update` is lossless, so persistence and serving keep the
|
|
283
|
+
raw pending bytes and the gap can still close.
|
|
250
284
|
|
|
251
285
|
### Rendering to HTML
|
|
252
286
|
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
287
|
+
The renderers turn a collaborative document into HTML on the server with no
|
|
288
|
+
Node process or headless editor. Each renderer is a class for one editor, and
|
|
289
|
+
its output matches that editor's own serializer. `Y::Tiptap` renders
|
|
290
|
+
ProseMirror documents and is built on `Y::ProseMirror`, and `Y::Lexxy` renders
|
|
291
|
+
documents from the [Lexxy](https://github.com/basecamp/lexxy) editor and is
|
|
292
|
+
built on `Y::Lexical`. To support another editor on one of those engines,
|
|
293
|
+
extend the matching base class with rules. Each renderer
|
|
259
294
|
returns `nil` for a root that belongs to the other schema.
|
|
260
295
|
|
|
261
296
|
#### `Y::Tiptap` (and `Y::ProseMirror`, its base)
|
|
@@ -266,26 +301,27 @@ tiptap.to_html # the "default" fragment (Tiptap's default root)
|
|
|
266
301
|
tiptap.to_html("content") # or another XML root
|
|
267
302
|
```
|
|
268
303
|
|
|
269
|
-
The output matches Tiptap's own `getHTML()`,
|
|
270
|
-
|
|
271
|
-
[`tiptap-php`](https://github.com/ueberdosis/tiptap-php) and
|
|
272
|
-
styles editors use
|
|
273
|
-
`bullet_list
|
|
274
|
-
|
|
275
|
-
It covers paragraphs, headings, blockquotes, bullet
|
|
276
|
-
blocks, links, images, mentions, details, hard breaks, horizontal rules,
|
|
277
|
-
tables, text styles (color
|
|
278
|
-
as
|
|
279
|
-
view adds.
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
ProseMirror natively
|
|
283
|
-
family
|
|
284
|
-
family) are
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
that node rules
|
|
288
|
-
as
|
|
304
|
+
The output matches Tiptap's own `getHTML()`, and the tests check it against a
|
|
305
|
+
document captured from a real editor. The implementation follows
|
|
306
|
+
[`tiptap-php`](https://github.com/ueberdosis/tiptap-php) and accepts both
|
|
307
|
+
naming styles editors use, Tiptap's `bulletList` and `bold` as well as
|
|
308
|
+
prosemirror-schema-basic's `bullet_list` and `strong`.
|
|
309
|
+
|
|
310
|
+
It covers paragraphs, headings, blockquotes, bullet, ordered, and task lists,
|
|
311
|
+
code blocks, links, images, mentions, details, hard breaks, horizontal rules,
|
|
312
|
+
tables, text styles (color and font family), and every text mark. Tables render
|
|
313
|
+
as a plain `<table><tbody>` without the column-width styling that Tiptap's
|
|
314
|
+
editor view adds.
|
|
315
|
+
|
|
316
|
+
Support is split into two layers, as on the Lexical side. `Y::ProseMirror`
|
|
317
|
+
handles core ProseMirror natively, meaning prosemirror-schema-basic plus the
|
|
318
|
+
prosemirror-tables family. Tiptap's extension nodes (task lists, mentions, the
|
|
319
|
+
details family) are a rule set, `Y::Tiptap::NODES`, written with the extension
|
|
320
|
+
API described below. Marks are handled in the base class. Mark rendering deals
|
|
321
|
+
with nesting order, the CSS on `textStyle`, and the exclusivity of `code`, and
|
|
322
|
+
it runs through native text-run code that node rules can't reach, so
|
|
323
|
+
`Y::ProseMirror` renders Tiptap's mark set as is. You can override individual
|
|
324
|
+
marks with `rules.mark`.
|
|
289
325
|
|
|
290
326
|
#### `Y::Lexxy` (and `Y::Lexical`, its base)
|
|
291
327
|
|
|
@@ -295,41 +331,41 @@ lexxy.to_html # the "root" fragment (Lexical's default root name)
|
|
|
295
331
|
lexxy.to_html("notepad") # or another XML root
|
|
296
332
|
```
|
|
297
333
|
|
|
298
|
-
The HTML is identical to
|
|
299
|
-
|
|
300
|
-
editor
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
editor to extend with rules.
|
|
334
|
+
The HTML is identical to the `value` a `lexxy-editor` submits to Rails, and the
|
|
335
|
+
tests check it against a document captured from a real editor. The class is
|
|
336
|
+
named after the editor because stock Lexical has no canonical serializer, and
|
|
337
|
+
every editor configures its own. `Y::Lexical` is the core Lexical base
|
|
338
|
+
(paragraphs, headings, quotes, code, lists, tables, links, and the full
|
|
339
|
+
text-format model), and other Lexical editors extend it with rules.
|
|
305
340
|
|
|
306
|
-
It handles the
|
|
307
|
-
format and their combinations, links, the four list types
|
|
341
|
+
It handles every node in the Lexxy 0.9.x set: paragraphs, headings, every text
|
|
342
|
+
format and their combinations, links, the four list types with nesting,
|
|
308
343
|
blockquotes, code blocks, tabs and soft breaks, horizontal rules, tables with
|
|
309
|
-
header cells, image galleries, and ActionText attachments
|
|
310
|
-
|
|
311
|
-
re-render
|
|
344
|
+
header cells, image galleries, and ActionText attachments. Uploads and mentions
|
|
345
|
+
both render as `<action-text-attachment>` elements, which ActionText can
|
|
346
|
+
re-render.
|
|
312
347
|
|
|
313
|
-
|
|
314
|
-
structure natively, and everything Lexxy adds
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
support
|
|
319
|
-
|
|
348
|
+
The Lexical support has the same two layers. `Y::Lexical` handles core Lexical
|
|
349
|
+
structure natively, and everything Lexxy adds is in `Y::Lexxy`'s rule set,
|
|
350
|
+
`Y::Lexxy::NODES`, written with the extension API below. That includes Lexxy's
|
|
351
|
+
own node types (attachments, galleries) and its decorations of core nodes (the
|
|
352
|
+
table wrapper, header-cell styling, nested-list classes). Since the gem's Lexxy
|
|
353
|
+
support was the first thing written on that API, an app rule for one of those
|
|
354
|
+
types replaces the built-in one.
|
|
320
355
|
|
|
321
|
-
In both renderers an unknown node
|
|
322
|
-
|
|
356
|
+
In both renderers, an unknown node still renders its text and nested blocks as
|
|
357
|
+
readable markup.
|
|
323
358
|
|
|
324
359
|
#### Custom nodes and marks
|
|
325
360
|
|
|
326
|
-
The built-in schemas
|
|
327
|
-
|
|
328
|
-
before the built-in schema, so
|
|
329
|
-
built-in renders.
|
|
361
|
+
The built-in schemas match what Tiptap and Lexxy ship. When your app adds its
|
|
362
|
+
own node types, both renderers take rules for them. The renderer checks rules
|
|
363
|
+
before the built-in schema, so a rule can add a node type or change how a
|
|
364
|
+
built-in one renders.
|
|
330
365
|
|
|
331
|
-
|
|
332
|
-
rule
|
|
366
|
+
You register rules in a block with one `rules.node` call per type. A
|
|
367
|
+
declarative rule describes the markup as a tag, attributes, and a content mode,
|
|
368
|
+
and the renderer emits it natively:
|
|
333
369
|
|
|
334
370
|
```ruby
|
|
335
371
|
tiptap = Y::Tiptap.new(doc) do |rules|
|
|
@@ -339,17 +375,18 @@ tiptap = Y::Tiptap.new(doc) do |rules|
|
|
|
339
375
|
end
|
|
340
376
|
```
|
|
341
377
|
|
|
342
|
-
`tag` names the element. `attrs`
|
|
343
|
-
a symbol reads that attribute off the node, an array
|
|
344
|
-
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
|
|
378
|
+
`tag` names the element. The values in `attrs` are templates, where a string is
|
|
379
|
+
a literal, a symbol reads that attribute off the node, and an array
|
|
380
|
+
concatenates a mix of the two. Attributes that resolve to an empty value are
|
|
381
|
+
left out. `text` takes the same template form and emits literal text content.
|
|
382
|
+
`contains` says what goes inside the node, which is `:inline` for formatted
|
|
383
|
+
text (the default), `:blocks` for child block nodes, or `:none` for a leaf.
|
|
384
|
+
`void: true` skips the closing tag.
|
|
348
385
|
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
`
|
|
352
|
-
|
|
386
|
+
Editors store types and attributes under names you wouldn't predict. Rhino's
|
|
387
|
+
strike mark is `rhino-strike`, for example, and Lexical prefixes its own props
|
|
388
|
+
with `__`. To find the real names and shapes, make a document in your editor
|
|
389
|
+
that uses your custom node, then run:
|
|
353
390
|
|
|
354
391
|
```ruby
|
|
355
392
|
Y::Tiptap.new(doc).node_types
|
|
@@ -359,11 +396,12 @@ Y::Tiptap.new(doc).node_types
|
|
|
359
396
|
# "paragraph" => { ..., "handled" => "builtin" } }
|
|
360
397
|
```
|
|
361
398
|
|
|
362
|
-
`handled`
|
|
363
|
-
names your templates and blocks will read
|
|
364
|
-
|
|
399
|
+
A type with `handled` set to nil still needs a rule. `attrs` lists the stored
|
|
400
|
+
attribute names your templates and blocks will read, and `children` and `text`
|
|
401
|
+
tell you which `contains:` to pick. Child block types mean `:blocks`, and text
|
|
402
|
+
means `:inline`.
|
|
365
403
|
|
|
366
|
-
When
|
|
404
|
+
When a declarative rule can't express the markup, give the node a block:
|
|
367
405
|
|
|
368
406
|
```ruby
|
|
369
407
|
lexical = Y::Lexical.new(doc) do |rules|
|
|
@@ -374,26 +412,26 @@ lexical = Y::Lexical.new(doc) do |rules|
|
|
|
374
412
|
end
|
|
375
413
|
```
|
|
376
414
|
|
|
377
|
-
The block
|
|
378
|
-
children
|
|
379
|
-
element
|
|
380
|
-
|
|
381
|
-
whether a list item
|
|
382
|
-
|
|
383
|
-
interpolate. To set the content mode for a
|
|
415
|
+
The block receives the node's type and stored attributes, plus `node.content`,
|
|
416
|
+
which is the node's children already rendered to HTML. `node.child_types` lists
|
|
417
|
+
the node's element and block children by type, in document order, so you can
|
|
418
|
+
answer questions the attributes can't, like how many images a gallery holds or
|
|
419
|
+
whether a list item has a nested list. The renderer inserts whatever the block
|
|
420
|
+
returns into the output as is and treats it as trusted HTML, so escape any
|
|
421
|
+
values you interpolate. To set the content mode for a block rule, pass both:
|
|
384
422
|
`rules.node "embed", contains: :blocks do |node| ... end`.
|
|
385
423
|
|
|
386
|
-
|
|
387
|
-
|
|
388
|
-
|
|
389
|
-
|
|
424
|
+
Blocks never run while the document is locked. The renderer finishes the whole
|
|
425
|
+
render inside one read transaction with the GVL released, and only then runs
|
|
426
|
+
the blocks and inserts their output, so a block can safely read the same doc or
|
|
427
|
+
even write to it. If no rule has a block, `to_html` skips that last step.
|
|
390
428
|
|
|
391
|
-
Blocks
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
declarative hashes,
|
|
395
|
-
type (a `Method` responds to `call` like any lambda)
|
|
396
|
-
|
|
429
|
+
Blocks cover whatever the declarative form can't express. `Y::Lexxy` and
|
|
430
|
+
`Y::Tiptap` are both built on this API (`lib/y/lexxy.rb`, `lib/y/tiptap.rb`),
|
|
431
|
+
so it already handles two complete editor schemas. Their simple nodes are
|
|
432
|
+
declarative hashes, and each node with logic is a plain method mapped by node
|
|
433
|
+
type (a `Method` responds to `call` like any lambda). The fixture tests check
|
|
434
|
+
that output against a live editor's.
|
|
397
435
|
|
|
398
436
|
The ProseMirror side also takes custom marks:
|
|
399
437
|
|
|
@@ -403,14 +441,15 @@ tiptap = Y::Tiptap.new(doc) do |rules|
|
|
|
403
441
|
end
|
|
404
442
|
```
|
|
405
443
|
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
for a built-in mark name
|
|
444
|
+
Symbols read from the mark's own attributes. Custom marks wrap outside all the
|
|
445
|
+
built-in marks, and when several custom marks apply to one run of text, they
|
|
446
|
+
nest alphabetically by name. A rule for a built-in mark name such as `"bold"`
|
|
447
|
+
replaces that mark's tag.
|
|
409
448
|
|
|
410
449
|
##### Worked examples
|
|
411
450
|
|
|
412
|
-
|
|
413
|
-
|
|
451
|
+
Here's a video-embed node from an app's Tiptap extension, which the built-in
|
|
452
|
+
schema doesn't know about:
|
|
414
453
|
|
|
415
454
|
```ruby
|
|
416
455
|
tiptap = Y::Tiptap.new(doc) do |rules|
|
|
@@ -422,8 +461,8 @@ tiptap = Y::Tiptap.new(doc) do |rules|
|
|
|
422
461
|
end
|
|
423
462
|
```
|
|
424
463
|
|
|
425
|
-
|
|
426
|
-
has finished, so
|
|
464
|
+
This one resolves mentions against the database. Blocks run after the document
|
|
465
|
+
read has finished, so a block can safely query ActiveRecord or the doc itself:
|
|
427
466
|
|
|
428
467
|
```ruby
|
|
429
468
|
tiptap = Y::Tiptap.new(doc) do |rules|
|
|
@@ -436,8 +475,9 @@ tiptap = Y::Tiptap.new(doc) do |rules|
|
|
|
436
475
|
end
|
|
437
476
|
```
|
|
438
477
|
|
|
439
|
-
|
|
440
|
-
|
|
478
|
+
This one overrides a shipped rule. The shipped rule emits
|
|
479
|
+
`<action-text-attachment>` elements for ActionText to re-render, and the
|
|
480
|
+
override renders Lexxy uploads as `<figure>` and `<img>` markup:
|
|
441
481
|
|
|
442
482
|
```ruby
|
|
443
483
|
lexxy = Y::Lexxy.new(doc) do |rules|
|
|
@@ -452,9 +492,10 @@ lexxy = Y::Lexxy.new(doc) do |rules|
|
|
|
452
492
|
end
|
|
453
493
|
```
|
|
454
494
|
|
|
455
|
-
|
|
456
|
-
element
|
|
457
|
-
itself by its column count
|
|
495
|
+
Some markup depends on structure. Because `node.child_types` lists the node's
|
|
496
|
+
element and block children in document order, a layout container can size
|
|
497
|
+
itself by its column count, and the columns themselves can still use
|
|
498
|
+
declarative rules:
|
|
458
499
|
|
|
459
500
|
```ruby
|
|
460
501
|
tiptap = Y::Tiptap.new(doc) do |rules|
|
|
@@ -465,8 +506,9 @@ tiptap = Y::Tiptap.new(doc) do |rules|
|
|
|
465
506
|
end
|
|
466
507
|
```
|
|
467
508
|
|
|
468
|
-
|
|
469
|
-
around the cursor,
|
|
509
|
+
A block can also look at the rendered content. This one drops the empty
|
|
510
|
+
paragraphs an editor keeps around the cursor, which works because
|
|
511
|
+
`node.content` is already rendered:
|
|
470
512
|
|
|
471
513
|
```ruby
|
|
472
514
|
lexical = Y::Lexical.new(doc) do |rules|
|
|
@@ -476,19 +518,20 @@ lexical = Y::Lexical.new(doc) do |rules|
|
|
|
476
518
|
end
|
|
477
519
|
```
|
|
478
520
|
|
|
479
|
-
For a larger reference, the gem's own editor schemas
|
|
480
|
-
`Y::Lexxy::NODES` in `lib/y/lexxy.rb` (declarative hashes for
|
|
481
|
-
nodes, a plain method
|
|
482
|
-
|
|
521
|
+
For a larger reference, look at the gem's own editor schemas, which are written
|
|
522
|
+
the same way. See `Y::Lexxy::NODES` in `lib/y/lexxy.rb` (declarative hashes for
|
|
523
|
+
the simple nodes, and a plain method mapped with `method(:name)` for each node
|
|
524
|
+
that needs logic, which means galleries, list items, header cells, and both
|
|
525
|
+
attachment types) and
|
|
483
526
|
`Y::Tiptap::NODES` in `lib/y/tiptap.rb` (task lists, mentions, the details
|
|
484
527
|
family).
|
|
485
528
|
|
|
486
529
|
### Protocol codec (module functions)
|
|
487
530
|
|
|
488
|
-
Classifying and unwrapping wire frames
|
|
489
|
-
`Y`
|
|
490
|
-
|
|
491
|
-
the server
|
|
531
|
+
Classifying and unwrapping wire frames needs no state, so these are module
|
|
532
|
+
functions on `Y` and you don't construct anything. The server routes a frame
|
|
533
|
+
without holding any presence or document state. The browser clients keep
|
|
534
|
+
presence, and the server relays awareness frames without reading them.
|
|
492
535
|
|
|
493
536
|
```ruby
|
|
494
537
|
Y.message_kind(frame) # => 0 drop / 1 step1 / 2 update / 3 awareness / 4 query
|
|
@@ -498,55 +541,170 @@ Y.wrap_update(update_bytes) # => wrap a raw doc update as a sync Update frame
|
|
|
498
541
|
|
|
499
542
|
### ActionCable Integration
|
|
500
543
|
|
|
501
|
-
In a Rails app, one generator creates the
|
|
544
|
+
In a Rails app, one generator creates the storage migration. The models and
|
|
545
|
+
`Y::DocumentChannel` come with the gem:
|
|
502
546
|
|
|
503
547
|
```bash
|
|
504
548
|
bin/rails generate yrby:install
|
|
505
549
|
bin/rails db:migrate
|
|
506
550
|
```
|
|
507
551
|
|
|
508
|
-
|
|
509
|
-
|
|
510
|
-
|
|
511
|
-
- **`Y::Document
|
|
512
|
-
|
|
513
|
-
|
|
514
|
-
`
|
|
515
|
-
`
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
a key
|
|
520
|
-
a
|
|
521
|
-
the merged `state` snapshot,
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
527
|
-
(default 64)
|
|
528
|
-
|
|
529
|
-
|
|
530
|
-
|
|
531
|
-
|
|
532
|
-
|
|
533
|
-
|
|
534
|
-
|
|
535
|
-
|
|
536
|
-
`
|
|
537
|
-
|
|
538
|
-
|
|
539
|
-
|
|
540
|
-
|
|
541
|
-
|
|
552
|
+
As with Action Text's `ActionText::RichText`, the models are defined in the
|
|
553
|
+
gem:
|
|
554
|
+
|
|
555
|
+
- **`Y::Document`** stores one row per document, and you can find a row in two
|
|
556
|
+
ways. Channels use `key`, a single opaque, unique string that the app
|
|
557
|
+
sometimes supplies and that nothing parses. Optionally, a row also has a
|
|
558
|
+
polymorphic `record` and a `name` that say which model attribute the document
|
|
559
|
+
backs, where `name` is the attribute name, such as `"body"`. Each record gets
|
|
560
|
+
one document per attribute, the same scheme ActionText::RichText uses, and
|
|
561
|
+
key-only documents leave `record` and `name` nil. Either side can come first.
|
|
562
|
+
`Y::Document.for(record, name)` finds or creates the record binding, derives
|
|
563
|
+
a readable key (`post/1/body`), and adopts any key-only row that already has
|
|
564
|
+
that key, so a channel that writes first and a binding created later end up
|
|
565
|
+
on the same document. The row also holds the merged `state` snapshot, which
|
|
566
|
+
contains only CRDT state. Derived data such as rendered HTML or search text
|
|
567
|
+
is up to your application, and the usual place for it is the channel's
|
|
568
|
+
`on_change`. By default the channel concern calls `.load_state(key)` and
|
|
569
|
+
`.append(key, update)` to read and write the store.
|
|
570
|
+
- **`Y::DocumentUpdate`** holds the uncompacted tail, one delta per row. When
|
|
571
|
+
the tail reaches `compact_every` (default 64), yrby compacts it into `state`
|
|
572
|
+
and deletes those rows. A load reads the snapshot plus the current tail, and
|
|
573
|
+
returns `state` directly when the tail is empty. Compactions of one document
|
|
574
|
+
run one at a time under a per-document row lock, and they skip rows with an
|
|
575
|
+
open causal gap, leaving them as they are until the gap closes. Destroying a
|
|
576
|
+
document also deletes its updates.
|
|
577
|
+
|
|
578
|
+
For encrypted storage, `Y::EncryptedDocument` writes `state` and update
|
|
579
|
+
payloads through Active Record encryption on the same tables, as
|
|
580
|
+
`ActionText::EncryptedRichText` does. You declare it on the model, and the
|
|
581
|
+
model decides whether an attribute is encrypted, so a page or client can't
|
|
582
|
+
change it:
|
|
583
|
+
|
|
584
|
+
```ruby
|
|
585
|
+
class Post < ApplicationRecord
|
|
586
|
+
has_collaborative_document :body, encrypted: true
|
|
587
|
+
end
|
|
588
|
+
```
|
|
589
|
+
|
|
590
|
+
`Y::DocumentChannel` reads that declaration and routes every load and append
|
|
591
|
+
for the attribute through the encrypted class, while attributes without it use
|
|
592
|
+
plain `Y::Document`. In your own channel, point `on_load` and `on_change` at
|
|
593
|
+
`Y::EncryptedDocument`. In both cases you need to configure your app's
|
|
594
|
+
encryption keys and use one access path per document, because the plain classes
|
|
595
|
+
read encrypted rows back as ciphertext.
|
|
596
|
+
|
|
597
|
+
`post.collaborative_document(:body)` returns a bound
|
|
598
|
+
`Y::Collaborative::Attribute` with `load_state`, `append(update)`, `key`, and
|
|
599
|
+
`y_doc`. `y_doc` builds a fresh native `Y::Doc` you can read and render in
|
|
600
|
+
Ruby. For built-in row operations like compaction, call
|
|
601
|
+
`post.collaborative_document(:body).document_row.compact!`. The channel and
|
|
602
|
+
your application code go through the same accessor, which handles encryption
|
|
603
|
+
for both.
|
|
604
|
+
|
|
605
|
+
An attribute uses the key its document row was stored under. Before a row
|
|
606
|
+
exists, the key is `Y::Document.key_for(record, name)`, and computing it
|
|
607
|
+
doesn't create a row. Grants keep their existing scope and lifetime.
|
|
608
|
+
|
|
609
|
+
To check the current user's permissions in addition to the signed grant, give
|
|
610
|
+
the shipped channel an `authorize_document` block. It runs in channel context,
|
|
611
|
+
so it can use `current_user` or any other identifiers your Action Cable
|
|
612
|
+
connection provides:
|
|
613
|
+
|
|
614
|
+
```ruby
|
|
615
|
+
# config/initializers/yrby.rb
|
|
616
|
+
Rails.application.config.to_prepare do
|
|
617
|
+
Y::DocumentChannel.authorize_document do |record, name|
|
|
618
|
+
current_user.present? && record.editable_by?(current_user, attribute: name)
|
|
619
|
+
end
|
|
620
|
+
end
|
|
621
|
+
```
|
|
622
|
+
|
|
623
|
+
`editable_by?` is a policy method in your application, and yrby doesn't provide
|
|
624
|
+
it. The block receives a freshly loaded record and the attribute name as a
|
|
625
|
+
string. If it returns false or nil, the channel rejects the subscription before
|
|
626
|
+
opening a stream or serving any state. An invalid grant is rejected before the
|
|
627
|
+
block is called. Without a block, a valid grant is enough, so the view that
|
|
628
|
+
renders the helper must still require edit permission. If the block raises, the
|
|
629
|
+
channel rejects the subscription and the error goes to your normal error
|
|
630
|
+
handling.
|
|
631
|
+
|
|
632
|
+
The policy runs once, when the client subscribes, and after that the open
|
|
633
|
+
subscription authorizes each message. That's how Action Cable is meant to work,
|
|
634
|
+
and it means a keystroke or cursor move doesn't cost a record load plus your
|
|
635
|
+
own queries. The tradeoff is that if you revoke a permission mid-session, it
|
|
636
|
+
takes effect the next time that client subscribes. You can shorten that window
|
|
637
|
+
in two ways. Every new subscription checks grant expiry, so a short
|
|
638
|
+
`expires_in:` on the tag limits it, as long as you also give the element a
|
|
639
|
+
`refresh:` URL (see below). If your application has to cut off access
|
|
640
|
+
immediately, stop the subscription yourself when the permission changes.
|
|
641
|
+
|
|
642
|
+
Both transports remember the decision for the life of the subscription. Action
|
|
643
|
+
Cable keeps the same channel instance around. AnyCable builds a new one for
|
|
644
|
+
each command, so yrby declares the authorized document as channel state and
|
|
645
|
+
sends it with each RPC, and because anycable-go holds that state, a client
|
|
646
|
+
can't forge it. A frame that arrives without an authorized subscription is
|
|
647
|
+
rejected even if its grant is valid.
|
|
648
|
+
|
|
649
|
+
Every channel does its authorization in one method, `authorized?`, which the
|
|
650
|
+
concern calls when a client subscribes, before it opens a stream or serves any
|
|
651
|
+
state. In a channel you write, you define `authorized?` yourself and it
|
|
652
|
+
receives the document key. The shipped `Y::DocumentChannel` defines it to run
|
|
653
|
+
the `authorize_document` block with the record and the attribute name, or to
|
|
654
|
+
accept any valid grant when there's no block. A subclass can override
|
|
655
|
+
`authorized?` directly and read the located record from `record`.
|
|
656
|
+
|
|
657
|
+
### Grant lifetime and refresh
|
|
658
|
+
|
|
659
|
+
A grant lasts as long as GlobalID's signed-id default, which is one month under
|
|
660
|
+
Rails, and `expires_in:` on the tag shortens it. The grant is part of the
|
|
661
|
+
rendered page, though, and Action Cable resubscribes with it after every
|
|
662
|
+
network drop. If the grant expires before the user is done editing, the editor
|
|
663
|
+
gets blocked at the next reconnect. To avoid that, pair `expires_in:` with
|
|
664
|
+
`refresh:`, a URL the element fetches when a subscription is rejected:
|
|
665
|
+
|
|
666
|
+
```erb
|
|
667
|
+
<%= collaborative_document_tag @post, :body, expires_in: 10.minutes,
|
|
668
|
+
refresh: grant_post_path(@post) %>
|
|
669
|
+
```
|
|
670
|
+
|
|
671
|
+
```ruby
|
|
672
|
+
# config/routes.rb: resources :posts do get :grant, on: :member end
|
|
673
|
+
# app/controllers/posts_controller.rb
|
|
674
|
+
def grant
|
|
675
|
+
@post = current_user.posts.find(params[:id]) # your own authorization, again
|
|
676
|
+
render json: { grant: @post.collaborative_sgid(:body, expires_in: 10.minutes) }
|
|
677
|
+
end
|
|
678
|
+
```
|
|
679
|
+
|
|
680
|
+
That action grants write access, so its check must be at least as strict as the
|
|
681
|
+
page that renders the tag. If it skips authorization, anyone who can reach the
|
|
682
|
+
URL gets a grant, and a short `expires_in:` protects nothing.
|
|
683
|
+
|
|
684
|
+
When a subscription is rejected, the element fetches that URL with the session
|
|
685
|
+
cookie, and the action runs your authorization again. If the response is
|
|
686
|
+
`{ "grant": ... }`, the same session resubscribes with the new grant and keeps
|
|
687
|
+
its document and pending edits. Any other response, a non-2xx status, or a second
|
|
688
|
+
rejection blocks the session as before, and so does a refresh that takes
|
|
689
|
+
longer than 15 seconds. The element doesn't renew grants on a timer, so it
|
|
690
|
+
won't interrupt a healthy open subscription. Every reconnect after expiry is a
|
|
691
|
+
fresh permission check, which is why you'd want a short lifetime in the first
|
|
692
|
+
place.
|
|
693
|
+
|
|
694
|
+
For room-keyed collaboration or other custom channel behavior, generate a
|
|
695
|
+
channel with `bin/rails generate yrby:install --channel` and implement its
|
|
696
|
+
`authorized?(key)`. Custom channels can still use both storage hooks.
|
|
697
|
+
|
|
698
|
+
The migration creates `y_documents` and `y_document_updates`. To rename them,
|
|
699
|
+
edit the generated migration and point `Y::Document.table_name` and
|
|
542
700
|
`Y::DocumentUpdate.table_name` at the new names in an initializer.
|
|
543
701
|
|
|
544
|
-
|
|
702
|
+
You can swap the storage. The channel only needs `on_load` and `on_change`
|
|
545
703
|
answered, and they can point at anything.
|
|
546
704
|
|
|
547
705
|
`include Y::ActionCable` (from the `yrby-rails` gem) is the channel
|
|
548
|
-
integration
|
|
549
|
-
awareness
|
|
706
|
+
integration. It implements the y-websocket protocol over ActionCable, covering
|
|
707
|
+
document sync, awareness, and presence.
|
|
550
708
|
|
|
551
709
|
```ruby
|
|
552
710
|
# app/channels/document_channel.rb
|
|
@@ -557,8 +715,6 @@ class DocumentChannel < ApplicationCable::Channel
|
|
|
557
715
|
on_change { |key, update| Y::Document.append(key, update) } # record, then broadcast
|
|
558
716
|
|
|
559
717
|
def subscribed
|
|
560
|
-
return reject unless authorized?(params[:id])
|
|
561
|
-
|
|
562
718
|
sync_subscribed params[:id]
|
|
563
719
|
end
|
|
564
720
|
|
|
@@ -568,80 +724,112 @@ class DocumentChannel < ApplicationCable::Channel
|
|
|
568
724
|
|
|
569
725
|
private
|
|
570
726
|
|
|
571
|
-
#
|
|
727
|
+
# This denies everyone until you wire it to your app's auth.
|
|
728
|
+
# sync_subscribed rejects the subscription unless this returns true.
|
|
572
729
|
def authorized?(_document_key) = false
|
|
573
730
|
end
|
|
574
731
|
```
|
|
575
732
|
|
|
576
|
-
|
|
577
|
-
|
|
578
|
-
|
|
579
|
-
|
|
580
|
-
|
|
733
|
+
For documents that belong to a record, `Y::Collaborative` (which the engine
|
|
734
|
+
includes into Active Record) provides the token flow `authorized?` needs. The
|
|
735
|
+
page creates a signed GlobalID scoped to one attribute, and the channel looks
|
|
736
|
+
up the record from it, so the page decides which document the client gets.
|
|
737
|
+
|
|
738
|
+
```erb
|
|
739
|
+
<%# the view picks the document and signs it %>
|
|
740
|
+
<%= tag.div data: { grant: post.collaborative_sgid(:body) } %>
|
|
741
|
+
```
|
|
742
|
+
|
|
743
|
+
```ruby
|
|
744
|
+
# The channel looks up the record from the token. This isn't memoized
|
|
745
|
+
# because AnyCable builds a fresh channel for each command, and a cached
|
|
746
|
+
# record would go stale.
|
|
747
|
+
def authorized?(_key) = record.present? && record.editable_by?(current_user)
|
|
748
|
+
def record = Y::Collaborative.locate(params[:grant], :body)
|
|
749
|
+
```
|
|
581
750
|
|
|
582
|
-
|
|
583
|
-
|
|
584
|
-
|
|
585
|
-
|
|
586
|
-
|
|
751
|
+
A token created for `:body` only verifies under `:body`'s purpose
|
|
752
|
+
(`"yrby/body"`), and a tampered, expired, or wrong-attribute token returns no
|
|
753
|
+
record.
|
|
754
|
+
|
|
755
|
+
The concern reads and writes through your store. It answers handshakes from
|
|
756
|
+
`on_load`, and it passes document changes through `on_change` before
|
|
757
|
+
broadcasting them. The ActionCable process doesn't keep authoritative state in
|
|
758
|
+
memory, so AnyCable RPC workers, Puma workers, and separate dynos can all
|
|
759
|
+
handle messages for the same document, as long as they share a store and a
|
|
760
|
+
cable adapter.
|
|
761
|
+
|
|
762
|
+
If the yrby-rails models are installed and you declare neither hook, `on_load`
|
|
763
|
+
and `on_change` both use `Y::Document` storage. To use a different store,
|
|
764
|
+
declare both hooks so reads and writes go to the same place. Once a pair is
|
|
765
|
+
configured explicitly, a subclass may override either hook. Outside a
|
|
766
|
+
yrby-rails app there's no default, and until you declare both hooks, the
|
|
767
|
+
channel fails before it can acknowledge or broadcast an edit.
|
|
768
|
+
|
|
769
|
+
Presence is ephemeral. The server relays awareness frames, and `yrby-client`
|
|
770
|
+
sends a best-effort presence-removal frame on disconnect and on `pagehide`. If
|
|
771
|
+
a client can't send that frame, the client-side awareness timeout removes its
|
|
772
|
+
presence.
|
|
587
773
|
|
|
588
774
|
Incoming frames are validated as a single well-formed protocol message before
|
|
589
775
|
anything processes or relays them. Malformed, truncated, multi-message,
|
|
590
|
-
oversized,
|
|
591
|
-
|
|
592
|
-
|
|
776
|
+
oversized, and unknown frames are dropped, so a bad frame can't crash the
|
|
777
|
+
process and one client can't relay garbage that breaks everyone else in the
|
|
778
|
+
room. A Rust panic is caught at the FFI boundary and re-raised as a Ruby
|
|
779
|
+
exception.
|
|
593
780
|
|
|
594
781
|
#### Delivery guarantees
|
|
595
782
|
|
|
596
|
-
|
|
597
|
-
servers:
|
|
783
|
+
These guarantees are the same whether you run one process or hundreds of them
|
|
784
|
+
across many servers:
|
|
598
785
|
|
|
599
786
|
- **The document always converges.** CRDT updates are commutative and
|
|
600
|
-
idempotent, so out-of-order, duplicate,
|
|
601
|
-
the same correct document
|
|
602
|
-
- **An acked update is durable,
|
|
603
|
-
update with a missing dependency
|
|
604
|
-
|
|
605
|
-
|
|
606
|
-
|
|
607
|
-
|
|
608
|
-
|
|
609
|
-
|
|
610
|
-
|
|
611
|
-
must make `on_change` idempotent**. But remember that the CRDT can handle duplicates.
|
|
787
|
+
idempotent, so out-of-order, duplicate, and concurrent delivery all produce
|
|
788
|
+
the same correct document in every deployment, with no coordination.
|
|
789
|
+
- **An acked update is durable, including one that arrived out of order.** The
|
|
790
|
+
server records and acks an update with a missing dependency like any other,
|
|
791
|
+
and the update waits as pending in the document. Some client still holds the
|
|
792
|
+
missing update unacked and keeps retransmitting it until the server records
|
|
793
|
+
it, and then the gap closes. See [Causal gaps](#causal-gaps).
|
|
794
|
+
- **`on_change` runs at least once, and replaying the log always rebuilds the
|
|
795
|
+
document.** yrby calls `on_change` for every update before acking or
|
|
796
|
+
broadcasting it. If you need exactly-once behavior, make `on_change`
|
|
797
|
+
idempotent. The CRDT handles duplicates either way.
|
|
612
798
|
- **A raising `on_change` rejects the update implicitly.** If the block raises,
|
|
613
|
-
the
|
|
614
|
-
|
|
615
|
-
|
|
616
|
-
|
|
617
|
-
|
|
618
|
-
|
|
619
|
-
|
|
620
|
-
`
|
|
799
|
+
the server doesn't ack or broadcast the update, and it doesn't send a
|
|
800
|
+
negative ack either. The client keeps the update pending and retransmits it
|
|
801
|
+
on its timer or on reconnect. That works for transient failures, such as a
|
|
802
|
+
store that was briefly down, where a retry succeeds. A block that raises
|
|
803
|
+
every time (say, a validation that always fails for this edit) gets retried
|
|
804
|
+
forever, because nothing tells the client to stop. Put hard rejections in the
|
|
805
|
+
channel's authorization at subscribe time, before an edit can reach
|
|
806
|
+
`on_change`, and don't raise inside the hook for them.
|
|
621
807
|
- **An over-cap frame is dropped the same silent way.** A frame larger than
|
|
622
|
-
`max_frame_bytes` (default 8 MiB) is dropped before decoding
|
|
623
|
-
broadcast
|
|
624
|
-
update that
|
|
625
|
-
|
|
626
|
-
|
|
627
|
-
|
|
628
|
-
|
|
629
|
-
|
|
630
|
-
|
|
631
|
-
|
|
808
|
+
`max_frame_bytes` (default 8 MiB) is dropped before decoding, with no ack and
|
|
809
|
+
no broadcast, which limits how much work a client can force on the server. A
|
|
810
|
+
real document update that hits the cap gets the same implicit rejection as
|
|
811
|
+
above, so it's never acked and is retransmitted forever. Normal typing won't
|
|
812
|
+
come near the cap, but a large paste, an embedded image, or a big initial
|
|
813
|
+
`SyncStep2` can. The server logs each drop (`warn` for over-cap, `debug` for
|
|
814
|
+
undecodable) with the document key and update id so you can find it, and you
|
|
815
|
+
can override `sync_log_context` on the channel to add a user or connection
|
|
816
|
+
id. Size the cap for your largest expected payload, and reject content that's
|
|
817
|
+
too big before it reaches the channel. The cap is a last line of defense, and
|
|
818
|
+
the client never sees an error from it.
|
|
632
819
|
|
|
633
820
|
#### Causal gaps
|
|
634
821
|
|
|
635
|
-
Yjs updates can arrive out of order
|
|
636
|
-
|
|
637
|
-
|
|
638
|
-
and integrates
|
|
639
|
-
|
|
640
|
-
|
|
641
|
-
|
|
642
|
-
Serving is lossless too,
|
|
643
|
-
full state
|
|
644
|
-
|
|
822
|
+
Yjs updates can arrive out of order, so an update can reach the server before
|
|
823
|
+
the update it depends on. yrby treats that as normal. It records and acks the
|
|
824
|
+
update like any other, the update waits as a pending struct in the document,
|
|
825
|
+
and it integrates once the missing dependency arrives. The write path appends,
|
|
826
|
+
relays, and acks without rebuilding the document, so an update with a gap costs
|
|
827
|
+
the same as any other.
|
|
828
|
+
|
|
829
|
+
Serving is lossless too, as on any Yjs server. `handle_sync_message` serves
|
|
830
|
+
full state with pending structs included, so a peer holds the same pending
|
|
831
|
+
struct and integrates it the same way. Closing the gap takes no special
|
|
832
|
+
machinery, because the missing
|
|
645
833
|
dependency is an update its sender still holds unacked, and at-least-once
|
|
646
834
|
retransmission delivers it. Only compaction excludes pending
|
|
647
835
|
(`compacted_state_update`), because folding a log must not freeze an
|
|
@@ -650,14 +838,14 @@ un-integrable struct into the base state.
|
|
|
650
838
|
The bundled `Y::Document` store handles all of this. If you write your own
|
|
651
839
|
store, keep two things in mind:
|
|
652
840
|
|
|
653
|
-
**1. Load losslessly, and tolerate duplicates.** `on_load` should return
|
|
654
|
-
|
|
841
|
+
**1. Load losslessly, and tolerate duplicates.** `on_load` should return state
|
|
842
|
+
that keeps pending updates, either `encode_state_as_update` or a replay of the
|
|
655
843
|
raw append log. Don't compact with `compacted_state_update` while
|
|
656
|
-
`doc.pending
|
|
657
|
-
|
|
658
|
-
|
|
659
|
-
|
|
660
|
-
size matters, dedup by content hash:
|
|
844
|
+
`doc.pending?` is true, because that strips the pending struct and loses the
|
|
845
|
+
acked edit inside it. `Y::Document` keeps pending rows out of compaction for
|
|
846
|
+
this reason. When an ack gets lost, the client resends an update the store
|
|
847
|
+
already has. Replay still converges because CRDT apply is idempotent, so
|
|
848
|
+
deduping is optional. If log size matters, dedup by content hash:
|
|
661
849
|
|
|
662
850
|
```ruby
|
|
663
851
|
class DocumentStore
|
|
@@ -688,17 +876,18 @@ class DocumentStore
|
|
|
688
876
|
end
|
|
689
877
|
```
|
|
690
878
|
|
|
691
|
-
**2. Watch for gaps that
|
|
692
|
-
|
|
693
|
-
|
|
694
|
-
acked, and every join or reconnect handshake
|
|
695
|
-
the server hasn't integrated, so any client holding the dependency
|
|
696
|
-
it
|
|
697
|
-
supply, and that
|
|
698
|
-
document key whenever a document is loaded to serve state
|
|
699
|
-
|
|
700
|
-
oldest open gap
|
|
701
|
-
|
|
879
|
+
**2. Watch for gaps that don't close.** An open gap is easy to miss, because
|
|
880
|
+
the pending edit doesn't show up in the document until its dependency arrives.
|
|
881
|
+
Usually the gap closes by itself. The sender retransmits the missing update
|
|
882
|
+
until it's acked, and on every join or reconnect handshake the client sends
|
|
883
|
+
everything the server hasn't integrated, so any client holding the dependency
|
|
884
|
+
supplies it by connecting. The gap worth alerting on is one that no live client
|
|
885
|
+
can supply, and that's what the `on_gap` hook is for. It fires with the
|
|
886
|
+
document key whenever a document is loaded to serve state while a gap is open.
|
|
887
|
+
Use it to emit a metric, such as a pending-document count or the age of the
|
|
888
|
+
oldest open gap, so you can see a gap that isn't closing. yrby also logs gaps
|
|
889
|
+
at `info`. Errors raised in the hook are swallowed, so a broken metrics call
|
|
890
|
+
can't break
|
|
702
891
|
frame handling.
|
|
703
892
|
|
|
704
893
|
```ruby
|
|
@@ -711,28 +900,32 @@ end
|
|
|
711
900
|
|
|
712
901
|
#### Multi-process deployments
|
|
713
902
|
|
|
714
|
-
Most Rails apps run several processes, and any of them might
|
|
715
|
-
Two
|
|
903
|
+
Most Rails apps run several processes, and any of them might end up serving a
|
|
904
|
+
given document. Two things keep them consistent.
|
|
716
905
|
|
|
717
|
-
Broadcasts
|
|
718
|
-
|
|
906
|
+
Broadcasts go between processes through the Action Cable adapter, so use
|
|
907
|
+
`redis`, `solid_cable`, or another cross-process adapter, since `async` only
|
|
908
|
+
works inside one process. With one of those adapters in place, a change
|
|
719
909
|
on one process reaches clients on all of them.
|
|
720
910
|
|
|
721
911
|
Every process rebuilds document state from the durable store through `on_load`.
|
|
722
|
-
|
|
723
|
-
|
|
724
|
-
|
|
912
|
+
Changes are still recorded before they're broadcast, so whichever process
|
|
913
|
+
receives a change writes it to the shared store before any client on any
|
|
914
|
+
process sees it.
|
|
725
915
|
|
|
726
|
-
`bun multiprocess.mjs`
|
|
727
|
-
|
|
916
|
+
In the demo, `bun multiprocess.mjs` runs clients across two processes and
|
|
917
|
+
checks that the documents converge, that fresh reads work on both processes,
|
|
918
|
+
that presence reaches clients on the other process, and that both processes
|
|
919
|
+
write to one shared log.
|
|
728
920
|
|
|
729
921
|
##### AnyCable
|
|
730
922
|
|
|
731
|
-
`yrby`
|
|
923
|
+
`yrby` supports AnyCable end to end.
|
|
732
924
|
|
|
733
|
-
The demo
|
|
734
|
-
|
|
735
|
-
yrby client provider, cross-process reads, and concurrent
|
|
925
|
+
The demo tests it against a real anycable-go server and RPC server in
|
|
926
|
+
`frontend/anycable_probe.mjs` and `anycable_concurrent.mjs`, covering liveness,
|
|
927
|
+
the yrby client provider, cross-process reads, and convergence under concurrent
|
|
928
|
+
editing.
|
|
736
929
|
|
|
737
930
|
##### Demo
|
|
738
931
|
|
|
@@ -741,8 +934,8 @@ app using the yrby provider, with end-to-end tests.
|
|
|
741
934
|
|
|
742
935
|
#### Record Before Distribute
|
|
743
936
|
|
|
744
|
-
Every document change
|
|
745
|
-
|
|
937
|
+
Every document change goes through your `on_change` handler before it's
|
|
938
|
+
broadcast, and the handler is where you record it durably:
|
|
746
939
|
|
|
747
940
|
```ruby
|
|
748
941
|
class DocumentChannel < ApplicationCable::Channel
|
|
@@ -759,21 +952,21 @@ class DocumentChannel < ApplicationCable::Channel
|
|
|
759
952
|
end
|
|
760
953
|
```
|
|
761
954
|
|
|
762
|
-
If the
|
|
763
|
-
applied
|
|
764
|
-
of every change.
|
|
765
|
-
document can both record (at
|
|
766
|
-
duplicate record replays to the same document.
|
|
955
|
+
If the handler raises (say the store is down), the change is rejected, so it
|
|
956
|
+
isn't applied or sent to anyone. The cost is a synchronous durable write on the
|
|
957
|
+
path of every change. The gem doesn't take a per-document lock, so two
|
|
958
|
+
concurrent writes to one document can both record (at least once). Because CRDT
|
|
959
|
+
apply is idempotent, a duplicate record replays to the same document.
|
|
767
960
|
|
|
768
|
-
The demo wires `on_change` to a durable Postgres-backed log by default, and
|
|
769
|
-
end to end that the log alone rebuilds the document.
|
|
961
|
+
The demo wires `on_change` to a durable Postgres-backed log by default, and
|
|
962
|
+
checks end to end that the log alone rebuilds the document.
|
|
770
963
|
|
|
771
964
|
#### Ephemeral documents (no database)
|
|
772
965
|
|
|
773
|
-
`on_load` and `on_change` are plain blocks, and
|
|
774
|
-
|
|
775
|
-
|
|
776
|
-
|
|
966
|
+
`on_load` and `on_change` are plain blocks, and they don't have to touch a
|
|
967
|
+
database. Some documents only need to last for a session, such as a scratchpad,
|
|
968
|
+
live form state, or a draft you save on submit. For those, the store can be
|
|
969
|
+
connection state that's sent with each request:
|
|
777
970
|
|
|
778
971
|
```ruby
|
|
779
972
|
class ScratchpadChannel < ApplicationCable::Channel
|
|
@@ -790,6 +983,11 @@ class ScratchpadChannel < ApplicationCable::Channel
|
|
|
790
983
|
|
|
791
984
|
def subscribed = sync_subscribed(params[:id])
|
|
792
985
|
def receive(data) = sync_receive(data, params[:id])
|
|
986
|
+
|
|
987
|
+
private
|
|
988
|
+
|
|
989
|
+
# Each subscriber gets its own scratchpad on this connection.
|
|
990
|
+
def authorized?(_key) = true
|
|
793
991
|
end
|
|
794
992
|
```
|
|
795
993
|
|
|
@@ -815,65 +1013,69 @@ class ScratchpadChannel < ApplicationCable::Channel
|
|
|
815
1013
|
|
|
816
1014
|
def subscribed = sync_subscribed(params[:id])
|
|
817
1015
|
def receive(data) = sync_receive(data, params[:id])
|
|
1016
|
+
|
|
1017
|
+
private
|
|
1018
|
+
|
|
1019
|
+
def authorized?(_key) = true # per-connection scratchpad, see above
|
|
818
1020
|
end
|
|
819
1021
|
```
|
|
820
1022
|
|
|
821
|
-
Both hooks run in the channel instance
|
|
822
|
-
anything the channel can
|
|
823
|
-
|
|
824
|
-
|
|
825
|
-
|
|
826
|
-
|
|
827
|
-
|
|
828
|
-
|
|
829
|
-
the full delivery contract
|
|
830
|
-
|
|
831
|
-
has never seen
|
|
832
|
-
that client (which always
|
|
833
|
-
|
|
834
|
-
|
|
835
|
-
|
|
836
|
-
|
|
1023
|
+
Both hooks run in the channel instance through `instance_exec`, so they can use
|
|
1024
|
+
anything the channel can. `sync_receive` rebuilds the document from `on_load`
|
|
1025
|
+
on every update, which is why the store can sit on the connection. On Action
|
|
1026
|
+
Cable the channel instance lasts as long as the connection, so an instance
|
|
1027
|
+
variable is all the store you need. Merging with `compacted_state_update` keeps
|
|
1028
|
+
the store at one blob so it doesn't grow into an update log.
|
|
1029
|
+
|
|
1030
|
+
Because the store is per connection, this pattern has limits. A single writer
|
|
1031
|
+
gets the full delivery contract without any database. When several people edit
|
|
1032
|
+
at the same time, one client's update can depend on edits its own connection
|
|
1033
|
+
has never seen. That update is recorded as pending, and the next handshake with
|
|
1034
|
+
that client (which always has the full document) supplies the missing state and
|
|
1035
|
+
closes the gap. The document still converges, but heavy concurrent editing
|
|
1036
|
+
leaves more pending between handshakes than a shared store would. On AnyCable,
|
|
1037
|
+
watch the payload size as well. The blob is sent with every message, so this
|
|
1038
|
+
only makes sense for small documents.
|
|
837
1039
|
|
|
838
1040
|
Durability is the connection plus the browsers. A reconnecting client re-seeds
|
|
839
|
-
an empty server through the ordinary sync handshake, so the document survives
|
|
840
|
-
server
|
|
841
|
-
shared across clients on a single-process deployment, the
|
|
842
|
-
a class-level `Concurrent::Map
|
|
843
|
-
coherent
|
|
1041
|
+
an empty server through the ordinary sync handshake, so the document survives a
|
|
1042
|
+
server restart as long as some client still has it. If an ephemeral document
|
|
1043
|
+
needs to be shared across clients on a single-process deployment, point the
|
|
1044
|
+
same two hooks at a class-level `Concurrent::Map`. That version is no longer
|
|
1045
|
+
coherent once you run more than one process.
|
|
844
1046
|
|
|
845
1047
|
#### Reliable delivery (acks)
|
|
846
1048
|
|
|
847
1049
|
yrby document delivery is ack-tracked. Browser document updates carry an
|
|
848
|
-
`"id"`, and the server replies `{ "ack": <id> }`
|
|
849
|
-
|
|
850
|
-
|
|
1050
|
+
`"id"`, and the server replies `{ "ack": <id> }` after `on_change` succeeds.
|
|
1051
|
+
The server records and acks every decodable document update, including one that
|
|
1052
|
+
arrives out of order.
|
|
851
1053
|
|
|
852
1054
|
```
|
|
853
1055
|
client -> server { "update": "<base64 update>", "id": 42 }
|
|
854
1056
|
server -> client { "ack": 42 } # update accepted; safe to forget
|
|
855
1057
|
```
|
|
856
1058
|
|
|
857
|
-
`yrby-client`'s `ActionCableProvider` handles this
|
|
858
|
-
|
|
859
|
-
|
|
860
|
-
one `{ ack: id }`
|
|
861
|
-
|
|
862
|
-
|
|
1059
|
+
`yrby-client`'s `ActionCableProvider` handles this for you. It queues the
|
|
1060
|
+
unacknowledged tail of local document updates and sends it merged into one
|
|
1061
|
+
causally complete delta, using the highest sequence in the batch as the id, so
|
|
1062
|
+
one `{ ack: id }` confirms everything up to it. Because CRDT apply is
|
|
1063
|
+
idempotent, resending an update the server already has does no harm, and the
|
|
1064
|
+
server acks it again. Awareness is ephemeral and isn't acked.
|
|
863
1065
|
|
|
864
|
-
|
|
865
|
-
|
|
866
|
-
See `yrby-client` for the client-side awareness API.
|
|
1066
|
+
The browser clients manage presence (cursors, selections). The server doesn't
|
|
1067
|
+
set or hold presence state, and it relays awareness frames without reading
|
|
1068
|
+
them. See `yrby-client` for the client-side awareness API.
|
|
867
1069
|
|
|
868
1070
|
## Thread Safety
|
|
869
1071
|
|
|
870
|
-
|
|
871
|
-
|
|
872
|
-
locking.
|
|
1072
|
+
You can share a `Doc` across Ruby threads. Puma workers, ActionCable connection
|
|
1073
|
+
threads, and background jobs can all use the same one concurrently, with no
|
|
1074
|
+
locking in your code.
|
|
873
1075
|
|
|
874
1076
|
`test/thread_safety_test.rb` runs shared docs, the full sync handshake, and
|
|
875
|
-
fan-in sync across 8 threads
|
|
876
|
-
convergence.
|
|
1077
|
+
fan-in sync across 8 concurrent threads and checks that the interleaving
|
|
1078
|
+
doesn't change convergence.
|
|
877
1079
|
|
|
878
1080
|
### Parallelism (GVL release)
|
|
879
1081
|
|
|
@@ -881,21 +1083,21 @@ Every method that does real CRDT work (applying updates, encoding state,
|
|
|
881
1083
|
handling sync messages) releases Ruby's Global VM Lock
|
|
882
1084
|
(`rb_thread_call_without_gvl`) while the native code runs. That buys two things.
|
|
883
1085
|
|
|
884
|
-
CRDT work runs in parallel across Ruby threads on MRI,
|
|
885
|
-
JRuby
|
|
886
|
-
speedup applying a
|
|
887
|
-
couldn't beat serial time.
|
|
888
|
-
|
|
889
|
-
|
|
890
|
-
the doc's write lock
|
|
891
|
-
|
|
892
|
-
|
|
893
|
-
Each
|
|
894
|
-
|
|
895
|
-
|
|
896
|
-
|
|
897
|
-
order can't deadlock.
|
|
898
|
-
|
|
1086
|
+
First, CRDT work runs in parallel across Ruby threads on MRI, with no need for
|
|
1087
|
+
JRuby or TruffleRuby. `bench/parallelism_bench.rb` measures more than a 2x
|
|
1088
|
+
wall-clock speedup when applying a roughly 900 KB update concurrently. Native
|
|
1089
|
+
code that held the GVL couldn't beat serial time.
|
|
1090
|
+
|
|
1091
|
+
Second, a slow operation can't stall the VM. A thread applying a large update
|
|
1092
|
+
holds the doc's write lock but not the GVL, so other Ruby threads keep running
|
|
1093
|
+
while it works.
|
|
1094
|
+
|
|
1095
|
+
Each of those methods copies the Ruby byte strings, releases the GVL, does the
|
|
1096
|
+
yrs work (taking and releasing the native locks inside that closure), takes the
|
|
1097
|
+
GVL back, and then builds the Ruby objects. Ruby APIs are only called while
|
|
1098
|
+
holding the GVL, and no native lock is held while reacquiring it, so the lock
|
|
1099
|
+
order can't deadlock. A panic in native code is caught and re-raised as a Ruby
|
|
1100
|
+
exception.
|
|
899
1101
|
|
|
900
1102
|
## Message Type Constants
|
|
901
1103
|
|