yrby 0.6.0-aarch64-linux → 0.7.0-aarch64-linux
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +73 -28
- data/README.md +352 -103
- data/lib/generators/yrby/install/install_generator.rb +44 -0
- data/lib/generators/yrby/install/templates/document_channel.rb +36 -0
- data/lib/generators/yrby/tables/tables_generator.rb +41 -0
- data/lib/generators/yrby/tables/templates/create_y_tables.rb +24 -0
- data/lib/y/3.4/yrby.so +0 -0
- data/lib/y/4.0/yrby.so +0 -0
- data/lib/y/decoder.rb +64 -0
- data/lib/y/lexxy.rb +25 -4
- data/lib/y/version.rb +1 -1
- data/lib/y.rb +1 -0
- metadata +8 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 7b56482181dafbce1bac53dedf886c50be3e59440275bded5598dae9ce0879fa
|
|
4
|
+
data.tar.gz: 2c107975f0a63abe25fd26a04acab155a908aa2a8795ef4c44c0ad9bacee20cf
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: ad63b30fc03e05f50321a925cb8714852922ebffb13847d65a36e96db5165694c66e599bff27a610b1813788ecb5c7cb729c78ee1375a63cb584512e46a6b250
|
|
7
|
+
data.tar.gz: '039e1cda7052a5688470361c574ff8621989c55c9a9a9a4b4d12a9168a72a2498712996b75e61383465b3df830d467aef41bf734d20a4029f393afa8125a27bf'
|
data/CHANGELOG.md
CHANGED
|
@@ -4,21 +4,66 @@ All notable changes to this project are documented here. The format is based on
|
|
|
4
4
|
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project aims
|
|
5
5
|
to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
6
|
|
|
7
|
+
## [0.7.0] - 2026-08-11
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- `Y::Decoder` ships in the core gem and loads with `require "y"`. It was
|
|
12
|
+
scaffolded as a separate `yrby-decoder` gem, but it is 66 lines of pure
|
|
13
|
+
Ruby over `Doc#read_text` / `read_xml`, requires the native core either
|
|
14
|
+
way, and the separate gem was never published, so the split left the
|
|
15
|
+
module in no gem at all. The `yrby-decoder` name is retired unused.
|
|
16
|
+
|
|
17
|
+
### Changed
|
|
18
|
+
|
|
19
|
+
- `Doc#handle_sync_message` answers a SyncStep1 with the doc's full state,
|
|
20
|
+
pending included, matching Y.js's `encodeStateAsUpdate`. It previously
|
|
21
|
+
served integrated-only state. A peer parks a served pending struct the
|
|
22
|
+
same way the doc did and heals it when the missing dependency arrives
|
|
23
|
+
from its sender's ack-driven retransmit. `compacted_state_update` still
|
|
24
|
+
excludes pending, so compaction cannot freeze a gap into a snapshot.
|
|
25
|
+
|
|
26
|
+
### Fixed
|
|
27
|
+
|
|
28
|
+
- **`Doc#update_advances?` no longer misreads a gappy merged update carrying
|
|
29
|
+
novel content as a no-op.** A crafted frame can hide an internal gap behind
|
|
30
|
+
a Skip block while its post-gap blocks still integrate (yrs plants a Skip
|
|
31
|
+
hole in the store). That moves neither the doc's public state vector nor
|
|
32
|
+
pending, so the probe comparison reported genuinely novel content as
|
|
33
|
+
"doesn't advance"; `update_ready?` accepted the frame and the doc applied
|
|
34
|
+
it, but it was never recorded to the durable log or broadcast. Any
|
|
35
|
+
insertion past the update's own (Skip-capped) `state_vector()` now
|
|
36
|
+
conservatively reports as advancing. Not reachable through standard Yjs
|
|
37
|
+
providers (a client's own updates and diffs are gap-free), so this closes
|
|
38
|
+
a hostile-input hole, not a real-world regression.
|
|
39
|
+
|
|
40
|
+
## [0.6.1] - 2026-08-04
|
|
41
|
+
|
|
42
|
+
### Fixed
|
|
43
|
+
|
|
44
|
+
- `Y::Lexxy` emits the attachment tag the node was created with instead of
|
|
45
|
+
a hardcoded `<action-text-attachment>`. Lexxy makes the tag configurable
|
|
46
|
+
(`Lexxy.configure`'s `attachmentTagName`, paired with
|
|
47
|
+
`ActionText::Attachment.tag_name` in Rails), and each attachment node
|
|
48
|
+
stores its tag, so a custom-tag app's rendered HTML now matches its
|
|
49
|
+
editor. A stored value that doesn't look like a tag name falls back to
|
|
50
|
+
the default; documents from before the tag was stored render unchanged.
|
|
51
|
+
|
|
7
52
|
## [0.6.0] - 2026-07-11
|
|
8
53
|
|
|
9
54
|
### Added
|
|
10
55
|
|
|
11
56
|
- **Custom render rules for `Y::Lexical` and `Y::ProseMirror`.** Both
|
|
12
|
-
renderers now take a block registering rules per node type
|
|
57
|
+
renderers now take a block registering rules per node type:
|
|
13
58
|
`rules.node "callout", tag: "aside"` for markup-as-data, a Ruby block for
|
|
14
|
-
logic
|
|
59
|
+
logic, plus `nodes:`/`marks:` keywords as the equivalent data form, to
|
|
15
60
|
render node types the pinned schemas don't know or to override how a
|
|
16
61
|
built-in renders.
|
|
17
62
|
Declarative rules (`tag`/`attrs`/`text`/`contains`, with templates mixing
|
|
18
63
|
literals and attribute references) render natively at full speed. Callback
|
|
19
64
|
rules run a Ruby block per node, receiving its type, attributes,
|
|
20
65
|
already-rendered children, and `child_types` (its element/block children by
|
|
21
|
-
type
|
|
66
|
+
type, the structural facts behind gallery counts and nested-list classes).
|
|
22
67
|
The block runs after the document read has finished, never while the doc is
|
|
23
68
|
locked, so it can safely read or write the same doc. Blocks are proven
|
|
24
69
|
sufficient for whole schemas: the gem's own editor schemas (`Y::Lexxy`,
|
|
@@ -26,13 +71,13 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
|
26
71
|
output byte-identical to a live editor's. With no callback rules the
|
|
27
72
|
render path is unchanged, byte for byte. See "Custom nodes and marks" in
|
|
28
73
|
the README.
|
|
29
|
-
- **`Y::Lexical#node_types` / `Y::ProseMirror#node_types
|
|
74
|
+
- **`Y::Lexical#node_types` / `Y::ProseMirror#node_types`: schema
|
|
30
75
|
discovery.** Ask a real document which node types it holds and what they
|
|
31
76
|
look like: counts, attribute names as stored, child types, whether text
|
|
32
77
|
runs appear, and whether a builtin or one of your rules already handles
|
|
33
78
|
each ("handled" nil marks what still needs a rule). Editors store names
|
|
34
79
|
you'd never guess; this is how you find them.
|
|
35
|
-
- `Y::RenderRules.escape_text` / `escape_attr
|
|
80
|
+
- `Y::RenderRules.escape_text` / `escape_attr`: the exact escaping the
|
|
36
81
|
native renderers use, for blocks that build markup from stored values
|
|
37
82
|
(ERB's `html_escape` also rewrites apostrophes, which breaks byte parity
|
|
38
83
|
with editor output).
|
|
@@ -42,15 +87,15 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
|
42
87
|
- **Lexical rendering is now two classes: `Y::Lexical` (core Lexical) and
|
|
43
88
|
`Y::Lexxy` (core plus the Lexxy schema as render rules).** Stock Lexical
|
|
44
89
|
has no canonical serializer, so the editor-specific class carries the
|
|
45
|
-
editor's name
|
|
90
|
+
editor's name; `Y::Lexxy.new(doc).to_html` is the byte-parity call for
|
|
46
91
|
Lexxy/Rails apps, and `Y::Lexical` is the base any other Lexical editor
|
|
47
92
|
extends with its own rules. The
|
|
48
|
-
native side renders core structure
|
|
93
|
+
native side renders core structure (paragraphs, headings, quotes, code,
|
|
49
94
|
lists, tables, links, the full text-format model. Lexxy's own node types
|
|
50
95
|
(attachments, galleries, `early_escape_code`, `horizontal_divider`) and its
|
|
51
96
|
decorations of core nodes (the table figure wrapper, header-cell styling,
|
|
52
97
|
the nested-list-item class) are rules applied beneath the app's, on the
|
|
53
|
-
same extension API
|
|
98
|
+
same extension API; the gem's Lexxy support is the API's first consumer.
|
|
54
99
|
Output is unchanged: the fixture tests still hold `to_html` byte-identical
|
|
55
100
|
to a live editor's serialized value, now through the extension path. An
|
|
56
101
|
unknown Lexical container also degrades better: its block children render
|
|
@@ -59,8 +104,8 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
|
59
104
|
ProseMirror) and `Y::Tiptap` (core plus Tiptap's extension nodes as render
|
|
60
105
|
rules).** `Y::Tiptap.new(doc).to_html` is the byte-parity call for Tiptap
|
|
61
106
|
apps. The native side renders prosemirror-schema-basic plus the
|
|
62
|
-
prosemirror-tables family; Tiptap's extension nodes
|
|
63
|
-
the details family
|
|
107
|
+
prosemirror-tables family; Tiptap's extension nodes (task lists, mentions,
|
|
108
|
+
the details family) are `Y::Tiptap::NODES` rules. Marks stay native in the
|
|
64
109
|
base class: mark rendering (nesting order, `textStyle` CSS, `code`
|
|
65
110
|
exclusivity) runs through text-run machinery node rules don't reach, so
|
|
66
111
|
`Y::ProseMirror` still renders Tiptap's full mark set and `rules.mark`
|
|
@@ -71,7 +116,7 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
|
71
116
|
|
|
72
117
|
### Added
|
|
73
118
|
|
|
74
|
-
- **`Y::Lexical
|
|
119
|
+
- **`Y::Lexical`: render Lexical/Lexxy documents to HTML.**
|
|
75
120
|
`Y::Lexical.new(doc).to_html` turns a Lexxy document into HTML on the server,
|
|
76
121
|
with no Node process or headless editor. The output is identical to the HTML
|
|
77
122
|
a `lexxy-editor` submits to Rails; the tests check it byte-for-byte against a
|
|
@@ -89,12 +134,12 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
|
89
134
|
|
|
90
135
|
### Added
|
|
91
136
|
|
|
92
|
-
- **`Y::ProseMirror
|
|
137
|
+
- **`Y::ProseMirror`: render ProseMirror/Tiptap documents to HTML.**
|
|
93
138
|
`Y::ProseMirror.new(doc).to_html` turns a Tiptap document into HTML on the
|
|
94
139
|
server, with no Node process or headless editor. The output matches Tiptap's
|
|
95
140
|
own `getHTML()`; the tests check it byte-for-byte against a document captured
|
|
96
141
|
from a real editor. It follows `ueberdosis/tiptap-php` and reads both name
|
|
97
|
-
styles editors use
|
|
142
|
+
styles editors use: Tiptap's `bulletList`/`bold` and prosemirror-schema-basic's
|
|
98
143
|
`bullet_list`/`strong`. Covers paragraphs, headings, blockquotes,
|
|
99
144
|
bullet/ordered/task lists, code blocks, links, images, mentions, details,
|
|
100
145
|
hard breaks, horizontal rules, tables, text styles (color, font family), and
|
|
@@ -110,8 +155,8 @@ Fixes from a full source review.
|
|
|
110
155
|
|
|
111
156
|
- **`Doc#update_ready?` is now exact.** It previously checked only the
|
|
112
157
|
per-client clock lower bound, but yrs's real integration gate also requires
|
|
113
|
-
every block referenced by an item's origin / right-origin / parent
|
|
114
|
-
routinely belong to *other* clients
|
|
158
|
+
every block referenced by an item's origin / right-origin / parent (which
|
|
159
|
+
routinely belong to *other* clients) and post-Skip blocks in a merged update
|
|
115
160
|
sit above the lower bound. An update could pass the clock check yet park as
|
|
116
161
|
pending; downstream, `update_advances?` then misread the parked update as an
|
|
117
162
|
already-applied retry (pending doesn't move a state vector) and the sync
|
|
@@ -135,7 +180,7 @@ Fixes from a full source review.
|
|
|
135
180
|
### Changed
|
|
136
181
|
|
|
137
182
|
- `update_advances?` skips its full-document probe when the update carries
|
|
138
|
-
blocks beyond the doc's state vector (a novel update trivially advances)
|
|
183
|
+
blocks beyond the doc's state vector (a novel update trivially advances);
|
|
139
184
|
the common case no longer pays O(doc) per frame.
|
|
140
185
|
- The gem no longer packages the `yrby-decoder` gem's files (they ship in that
|
|
141
186
|
gem; the duplicate copy could shadow a newer standalone release), and now
|
|
@@ -146,7 +191,7 @@ Fixes from a full source review.
|
|
|
146
191
|
### Fixed
|
|
147
192
|
|
|
148
193
|
- **Sync no longer serves un-integrable pending structs.** When a doc holds a
|
|
149
|
-
*pending* struct (a gappy update whose causally-prior update is missing
|
|
194
|
+
*pending* struct (a gappy update whose causally-prior update is missing, e.g.
|
|
150
195
|
legacy data recorded before the `update_ready?` gate existed), its integrated
|
|
151
196
|
state vector is empty but `encode_state_as_update` merges the pending bytes back
|
|
152
197
|
in. Answering a peer's `SyncStep1` with that state handed the peer content it
|
|
@@ -154,18 +199,18 @@ Fixes from a full source review.
|
|
|
154
199
|
non-empty-content mismatch drove endless resync traffic (observed as a browser
|
|
155
200
|
re-sending frames several times a second). `handle_sync_message` now answers
|
|
156
201
|
`SyncStep1` with **integrated-only** state, so a server never serves a struct it
|
|
157
|
-
can't integrate itself. Neutralizes existing poisoned server state on deploy
|
|
202
|
+
can't integrate itself. Neutralizes existing poisoned server state on deploy;
|
|
158
203
|
no migration needed. The server's own pending is untouched and still heals if
|
|
159
204
|
the missing dependency later arrives (only then does the content become
|
|
160
205
|
visible in sync). Live delta relay (`Update` frames) is unchanged.
|
|
161
206
|
|
|
162
207
|
### Added
|
|
163
208
|
|
|
164
|
-
- `Doc#pending
|
|
209
|
+
- `Doc#pending?`: true if the doc holds un-integrable pending structs or a
|
|
165
210
|
pending delete set (content waiting on a missing causally-prior update).
|
|
166
|
-
- `Doc#compacted_state_update
|
|
211
|
+
- `Doc#compacted_state_update`: like `encode_state_as_update` (full state) but
|
|
167
212
|
**gap-free**: excludes pending structs/delete set. Use it when persisting or
|
|
168
|
-
serving state other peers will apply. Non-destructive
|
|
213
|
+
serving state other peers will apply. Non-destructive; the doc keeps its
|
|
169
214
|
pending (so it can still heal), and `encode_state_as_update` stays lossless for
|
|
170
215
|
raw-update recovery.
|
|
171
216
|
|
|
@@ -183,13 +228,13 @@ Fixes from a full source review.
|
|
|
183
228
|
before vs. after a trial apply on an isolated probe: a genuinely new deletion
|
|
184
229
|
changes it (`true`); an already-applied retry re-encodes identically (`false`).
|
|
185
230
|
Insert/format-only updates keep the cheaper state-vector path, so only
|
|
186
|
-
delete-bearing frames
|
|
231
|
+
delete-bearing frames (a minority) pay for the exact comparison. The exactly-
|
|
187
232
|
once guarantee is unchanged in the safe direction: a real deletion is never
|
|
188
233
|
dropped.
|
|
189
234
|
|
|
190
235
|
This lets `yrby-actioncable` (and any caller gating `on_change` on
|
|
191
|
-
`update_advances?`) settle a duplicate pure-delete frame as `:applied`
|
|
192
|
-
but not stored or relayed
|
|
236
|
+
`update_advances?`) settle a duplicate pure-delete frame as `:applied` (acked,
|
|
237
|
+
but not stored or relayed), so apps no longer need an app-level
|
|
193
238
|
encode-and-compare guard around their durable writes.
|
|
194
239
|
|
|
195
240
|
## [0.2.2] - 2026-06-30
|
|
@@ -200,7 +245,7 @@ Fixes from a full source review.
|
|
|
200
245
|
embeds child blocks (list items, table cells, nested lists) as `Y.XmlText`
|
|
201
246
|
embeds that `get_string` silently drops, so lists and tables previously came
|
|
202
247
|
back empty. `read_xml` now walks the embeds: text runs build a line, inline
|
|
203
|
-
children (links) join it, and nested block children flush and recurse
|
|
248
|
+
children (links) join it, and nested block children flush and recurse, so a
|
|
204
249
|
document with headings, formatted text, links, bullet/numbered/check/nested
|
|
205
250
|
lists, blockquotes, code blocks and tables extracts every piece of text.
|
|
206
251
|
Lexical decorator elements (horizontal rule, image) are skipped instead of
|
|
@@ -211,17 +256,17 @@ Fixes from a full source review.
|
|
|
211
256
|
|
|
212
257
|
### Changed
|
|
213
258
|
- **Internal:** renamed the native extension crate `y_ruby` → `yrby` (now loads
|
|
214
|
-
from `lib/y/yrby.bundle`). No public API change
|
|
259
|
+
from `lib/y/yrby.bundle`). No public API change: `require "y"` and `Y::Doc`
|
|
215
260
|
are unchanged.
|
|
216
261
|
|
|
217
262
|
## [0.2.0] - 2026-06-28
|
|
218
263
|
|
|
219
264
|
First release. The public Ruby interface is the top-level module **`Y`**
|
|
220
|
-
(`Y::Doc`, `Y::Error`, `Y::VERSION`), loaded with `require "y"
|
|
265
|
+
(`Y::Doc`, `Y::Error`, `Y::VERSION`), loaded with `require "y"`, mirroring the
|
|
221
266
|
`y-rb` gem's `Y::Doc` interface.
|
|
222
267
|
|
|
223
268
|
### Added
|
|
224
|
-
- Native `Doc#read_text` and `Doc#read_map` readers
|
|
269
|
+
- Native `Doc#read_text` and `Doc#read_map` readers: reconstruct plain text and
|
|
225
270
|
a JSON map from the stored CRDT state in-process, server-side, with no Node or
|
|
226
271
|
subprocess.
|
|
227
272
|
|
data/README.md
CHANGED
|
@@ -2,48 +2,85 @@
|
|
|
2
2
|
|
|
3
3
|
[](https://github.com/jpcamara/yrby/actions/workflows/ci.yml)
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
(the Rust
|
|
7
|
-
|
|
8
|
-
|
|
5
|
+
yrby (pronounced "yer-bee") makes Rails a real Yjs backend. It binds
|
|
6
|
+
[y-crdt](https://github.com/y-crdt/y-crdt), the Rust engine behind Y.js, into
|
|
7
|
+
Ruby, and builds the rest of the stack around it: a sync server for Action
|
|
8
|
+
Cable and AnyCable, a browser provider, and server-side reading and rendering
|
|
9
|
+
of the documents. Real-time collaboration in a Rails app with no Node process
|
|
10
|
+
anywhere in the path.
|
|
11
|
+
|
|
12
|
+

|
|
13
|
+
|
|
14
|
+
On the server, `yrby-rails` implements the full y-websocket protocol
|
|
15
|
+
(document sync plus presence) as a channel concern. Its delivery contract is
|
|
16
|
+
stricter than the usual Yjs servers: every update is ack-tracked and durably
|
|
17
|
+
recorded before it is acknowledged or broadcast to anyone. Replaying your
|
|
18
|
+
store always rebuilds the document, across any number of processes.
|
|
19
|
+
([Delivery guarantees](#delivery-guarantees))
|
|
20
|
+
|
|
21
|
+
In the browser, `yrby-client`'s `ActionCableProvider` connects anything that
|
|
22
|
+
speaks Yjs. The demo app runs four rich text editors, and CI drives each one
|
|
23
|
+
in real Chrome: Tiptap, [Lexxy](https://www.npmjs.com/package/lexxy-realtime),
|
|
24
|
+
Rhino Editor, and CodeMirror. The same channel also syncs Yjs shapes with no
|
|
25
|
+
editor at all: a whiteboard on a `Y.Map`, a kanban board on a `Y.Array`, a
|
|
26
|
+
co-filled form. ([Editors](#editors))
|
|
27
|
+
|
|
28
|
+
In Ruby, the documents are readable without a browser. `Doc#read_text` and
|
|
29
|
+
`Doc#read_map` reconstruct contents for search, validation, and exports.
|
|
30
|
+
`Y::Tiptap` and `Y::Lexxy` render a document to HTML byte-identical to the
|
|
31
|
+
editor's own serializer, take rules for your app's custom nodes, and drop
|
|
32
|
+
straight into ActionText. ([Rendering to HTML](#rendering-to-html))
|
|
33
|
+
|
|
34
|
+
Underneath, the core is built for a production Rails deployment. A `Doc` is
|
|
35
|
+
thread-safe across Puma and ActionCable threads. Native CRDT work runs with
|
|
36
|
+
the GVL released, so it parallelizes on MRI. Incoming frames are validated
|
|
37
|
+
before anything processes them, and multi-process and AnyCable setups are
|
|
38
|
+
tested end to end. ([Thread Safety](#thread-safety))
|
|
39
|
+
|
|
40
|
+
The whole server side of a collaborative document is one channel:
|
|
9
41
|
|
|
10
42
|
```ruby
|
|
11
43
|
class DocumentChannel < ApplicationCable::Channel
|
|
12
|
-
include Y::ActionCable
|
|
44
|
+
include Y::ActionCable
|
|
13
45
|
|
|
14
|
-
on_load { |key|
|
|
15
|
-
on_change { |key, update|
|
|
46
|
+
on_load { |key| Y::Document.load_state(key) }
|
|
47
|
+
on_change { |key, update| Y::Document.append(key, update) }
|
|
16
48
|
|
|
17
49
|
def subscribed = sync_subscribed(params[:id])
|
|
18
50
|
def receive(data) = sync_receive(data, params[:id])
|
|
19
51
|
end
|
|
20
52
|
```
|
|
21
53
|
|
|
22
|
-
|
|
23
|
-
[`yrby-client`](https://www.npmjs.com/package/yrby-client) npm package.
|
|
24
|
-
Integrates with any editor that includes Y.js support, such as Tiptap, ProseMirror
|
|
25
|
-
and [Lexxy](https://www.npmjs.com/package/lexxy-realtime).
|
|
26
|
-
|
|
27
|
-
## Usage
|
|
28
|
-
|
|
29
|
-
Install the gem and npm package:
|
|
54
|
+
Install the gem and the npm package:
|
|
30
55
|
|
|
31
56
|
```
|
|
32
|
-
gem install yrby-
|
|
57
|
+
gem install yrby-rails # depends on yrby
|
|
33
58
|
npm install yrby-client
|
|
34
59
|
```
|
|
35
60
|
|
|
36
|
-
##
|
|
37
|
-
|
|
38
|
-
-
|
|
39
|
-
|
|
40
|
-
-
|
|
41
|
-
|
|
42
|
-
-
|
|
43
|
-
|
|
44
|
-
-
|
|
45
|
-
|
|
46
|
-
|
|
61
|
+
## Contents
|
|
62
|
+
|
|
63
|
+
- [Scope](#scope)
|
|
64
|
+
- [Durability and delivery](#durability-and-delivery)
|
|
65
|
+
- [What about yrb?](#what-about-yrb)
|
|
66
|
+
- [Testing](#testing)
|
|
67
|
+
- [Install](#install)
|
|
68
|
+
- [Docs](#docs)
|
|
69
|
+
- [Editors](#editors)
|
|
70
|
+
- [Usage](#usage)
|
|
71
|
+
- [Doc (Low-Level Document Sync)](#doc-low-level-document-sync)
|
|
72
|
+
- [Reading document contents](#reading-document-contents)
|
|
73
|
+
- [Pending structs and gap-free state](#pending-structs-and-gap-free-state)
|
|
74
|
+
- [Rendering to HTML](#rendering-to-html)
|
|
75
|
+
- [Protocol codec (module functions)](#protocol-codec-module-functions)
|
|
76
|
+
- [ActionCable Integration](#actioncable-integration)
|
|
77
|
+
- [Thread Safety](#thread-safety)
|
|
78
|
+
- [Parallelism (GVL release)](#parallelism-gvl-release)
|
|
79
|
+
- [Message Type Constants](#message-type-constants)
|
|
80
|
+
- [Sync Flow](#sync-flow)
|
|
81
|
+
- [Development](#development)
|
|
82
|
+
- [License](#license)
|
|
83
|
+
- [Acknowledgments](#acknowledgments)
|
|
47
84
|
|
|
48
85
|
## Scope
|
|
49
86
|
|
|
@@ -59,19 +96,18 @@ and `Doc#read_map` reconstruct it server-side, in Ruby.
|
|
|
59
96
|
The surface is intentionally small, but the focus is durability, resiliency, delivery
|
|
60
97
|
guarantees, correctness, and thread safety.
|
|
61
98
|
|
|
62
|
-
Towards that goal, `yrby` adds
|
|
99
|
+
Towards that goal, `yrby` adds opinionated defaults on top of normal Yjs syncing:
|
|
63
100
|
|
|
64
|
-
- Built-in update acknowledgement: the `ActionCableProvider` in `yrby-client`
|
|
65
|
-
|
|
66
|
-
only
|
|
67
|
-
and because
|
|
68
|
-
- Gap
|
|
69
|
-
`yrby`
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
that may have been missed
|
|
101
|
+
- Built-in update acknowledgement: the `ActionCableProvider` in `yrby-client` keeps
|
|
102
|
+
sending an update until the server acks it, and [`yrby-rails`](https://rubygems.org/gems/yrby-rails)
|
|
103
|
+
only acks once the update is durably recorded. That gives you at-least-once
|
|
104
|
+
delivery, and because CRDT updates are idempotent a duplicate is a no-op.
|
|
105
|
+
- Gap awareness: an update can arrive before another update it depends on (a
|
|
106
|
+
"causal gap"). `yrby` records and acks it like any other, and the document
|
|
107
|
+
heals on its own once the missing update arrives; its sender keeps
|
|
108
|
+
retransmitting it until it is acked. `Doc#pending?` and the `on_gap` hook
|
|
109
|
+
tell you when a document is waiting on a missing update.
|
|
110
|
+
([Causal gaps](#causal-gaps))
|
|
75
111
|
|
|
76
112
|
## What about [yrb](https://github.com/y-crdt/yrb)?
|
|
77
113
|
|
|
@@ -101,8 +137,9 @@ Issues and PRs are welcome.
|
|
|
101
137
|
# Core CRDT + protocol primitives:
|
|
102
138
|
gem "yrby"
|
|
103
139
|
|
|
104
|
-
# For the Rails
|
|
105
|
-
|
|
140
|
+
# For the Rails side (the sync channel, document models, the generator).
|
|
141
|
+
# Formerly yrby-actioncable; that name stops at 0.3.1.
|
|
142
|
+
gem "yrby-rails"
|
|
106
143
|
```
|
|
107
144
|
|
|
108
145
|
Requires Ruby 3.4 or newer. The release workflow builds precompiled gems for
|
|
@@ -144,8 +181,8 @@ editor's own serializer. Each page is a working integration to copy from:
|
|
|
144
181
|
| [Rhino Editor](https://github.com/KonnorRogers/rhino-editor) (Tiptap 3) | `@tiptap/extension-collaboration` + `-caret` | [`rhino.js`](examples/actioncable-demo/frontend/src/rhino.js) |
|
|
145
182
|
| [CodeMirror 6](https://codemirror.net) | `y-codemirror.next` | [`codemirror.js`](examples/actioncable-demo/frontend/src/codemirror.js) |
|
|
146
183
|
|
|
147
|
-
The demo also syncs plain Yjs shapes with no editor at all
|
|
148
|
-
on a `Y.Map`, a kanban board on a `Y.Array`, a co-filled form
|
|
184
|
+
The demo also syncs plain Yjs shapes with no editor at all (a whiteboard
|
|
185
|
+
on a `Y.Map`, a kanban board on a `Y.Array`, a co-filled form) over the
|
|
149
186
|
same channel. The demo README's "Using this in your own app" section has
|
|
150
187
|
the integration recipe, and its `NoteMaterializer` shows how to render a
|
|
151
188
|
document to ActionText server-side with `Y::Tiptap` or `Y::Lexxy`.
|
|
@@ -170,17 +207,19 @@ doc.compacted_state_update # => full update, gap-free (excludes pending)
|
|
|
170
207
|
# Applying updates
|
|
171
208
|
doc.apply_update(update_bytes) # apply raw V1 update
|
|
172
209
|
doc.pending? # => true if holding un-integrable pending structs
|
|
210
|
+
doc.update_ready?(update) # => true if update would integrate cleanly (no gap)
|
|
211
|
+
doc.update_advances?(update) # => true if update moves integrated state forward
|
|
173
212
|
|
|
174
213
|
# Sync protocol
|
|
175
214
|
doc.sync_step1 # => SyncStep1 message (this doc's state vector)
|
|
176
215
|
doc.handle_sync_message(data) # => [msg_type, sync_type, response]; answers a
|
|
177
|
-
# peer's SyncStep1 with
|
|
178
|
-
#
|
|
216
|
+
# peer's SyncStep1 with full state (lossless,
|
|
217
|
+
# pending included, like Y.js)
|
|
179
218
|
```
|
|
180
219
|
|
|
181
220
|
### Reading document contents
|
|
182
221
|
|
|
183
|
-
Reconstruct a document server-side
|
|
222
|
+
Reconstruct a document server-side (search, exports, emails, SSR) with no
|
|
184
223
|
Node process:
|
|
185
224
|
|
|
186
225
|
```ruby
|
|
@@ -196,16 +235,18 @@ update), yrs parks it as a **pending** struct: the integrated state vector stays
|
|
|
196
235
|
empty, but the pending block is held as a recovery buffer and heals if the
|
|
197
236
|
missing dependency later arrives. `Doc#pending?` reports this.
|
|
198
237
|
|
|
199
|
-
Pending structs
|
|
200
|
-
|
|
201
|
-
|
|
238
|
+
Pending structs travel like any other state. `handle_sync_message` answers
|
|
239
|
+
`SyncStep1` with the doc's full state, pending included, just like Y.js's
|
|
240
|
+
`encodeStateAsUpdate`: a peer parks the pending struct the same way this doc
|
|
241
|
+
did and heals it the same way. The one place pending must not go is a
|
|
242
|
+
compacted snapshot:
|
|
202
243
|
|
|
203
|
-
- `
|
|
204
|
-
|
|
205
|
-
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
244
|
+
- `Doc#compacted_state_update` returns a gap-free full-state update for
|
|
245
|
+
compaction. Folding a log into one blob would otherwise freeze an
|
|
246
|
+
un-integrable struct into the base state forever. It's non-destructive: the
|
|
247
|
+
doc keeps its pending.
|
|
248
|
+
- `encode_state_as_update` stays lossless, so persistence and serving keep
|
|
249
|
+
the raw pending bytes and the gap can still heal.
|
|
209
250
|
|
|
210
251
|
### Rendering to HTML
|
|
211
252
|
|
|
@@ -228,7 +269,7 @@ tiptap.to_html("content") # or another XML root
|
|
|
228
269
|
The output matches Tiptap's own `getHTML()`, checked byte-for-byte in the tests
|
|
229
270
|
against a document captured from a real editor. It follows
|
|
230
271
|
[`tiptap-php`](https://github.com/ueberdosis/tiptap-php) and reads both name
|
|
231
|
-
styles editors use
|
|
272
|
+
styles editors use: Tiptap's `bulletList`/`bold` and prosemirror-schema-basic's
|
|
232
273
|
`bullet_list`/`strong`.
|
|
233
274
|
|
|
234
275
|
It covers paragraphs, headings, blockquotes, bullet/ordered/task lists, code
|
|
@@ -238,8 +279,8 @@ as semantic `<table><tbody>`, without the column-width styling Tiptap's editor
|
|
|
238
279
|
view adds.
|
|
239
280
|
|
|
240
281
|
The support is layered like the Lexical side: `Y::ProseMirror` covers core
|
|
241
|
-
ProseMirror natively
|
|
242
|
-
family
|
|
282
|
+
ProseMirror natively (prosemirror-schema-basic plus the prosemirror-tables
|
|
283
|
+
family) and Tiptap's extension nodes (task lists, mentions, the details
|
|
243
284
|
family) are `Y::Tiptap`'s rule set (`Y::Tiptap::NODES`), built on the
|
|
244
285
|
extension API below. Marks stay in the base: mark rendering (nesting order,
|
|
245
286
|
`textStyle` CSS, `code` exclusivity) runs through native text-run machinery
|
|
@@ -256,8 +297,8 @@ lexxy.to_html("notepad") # or another XML root
|
|
|
256
297
|
|
|
257
298
|
The HTML is identical to what a `lexxy-editor` submits to Rails (its `value`).
|
|
258
299
|
The tests check this byte-for-byte against a document captured from a real
|
|
259
|
-
editor. Stock Lexical has no canonical serializer
|
|
260
|
-
its own
|
|
300
|
+
editor. Stock Lexical has no canonical serializer (every editor configures
|
|
301
|
+
its own), so the editor-specific class carries the editor's name, and
|
|
261
302
|
`Y::Lexical` is the core-Lexical base: paragraphs, headings, quotes, code,
|
|
262
303
|
lists, tables, links, and the full text-format model, for any other Lexical
|
|
263
304
|
editor to extend with rules.
|
|
@@ -270,14 +311,14 @@ mentions both emit `<action-text-attachment>` elements that ActionText can
|
|
|
270
311
|
re-render).
|
|
271
312
|
|
|
272
313
|
Internally that support is layered: `Y::Lexical` covers core Lexical
|
|
273
|
-
structure natively, and everything Lexxy adds
|
|
314
|
+
structure natively, and everything Lexxy adds, its node types (attachments,
|
|
274
315
|
galleries) and its decorations of core nodes (the table wrapper, header-cell
|
|
275
|
-
styling, nested-list classes)
|
|
316
|
+
styling, nested-list classes), is `Y::Lexxy`'s rule set
|
|
276
317
|
(`Y::Lexxy::NODES`), built on the extension API below. The gem's own Lexxy
|
|
277
318
|
support is the API's first consumer: an app rule for one of those types
|
|
278
319
|
simply replaces it.
|
|
279
320
|
|
|
280
|
-
In both renderers an unknown node keeps its content
|
|
321
|
+
In both renderers an unknown node keeps its content: text and nested blocks
|
|
281
322
|
fall back to readable markup rather than disappearing.
|
|
282
323
|
|
|
283
324
|
#### Custom nodes and marks
|
|
@@ -287,7 +328,7 @@ their own node types. Both renderers take rules for them. A rule is checked
|
|
|
287
328
|
before the built-in schema, so it can add a node type or replace how a
|
|
288
329
|
built-in renders.
|
|
289
330
|
|
|
290
|
-
Rules register in a block
|
|
331
|
+
Rules register in a block, one `rules.node` call per type. A declarative
|
|
291
332
|
rule is markup as data, rendered natively:
|
|
292
333
|
|
|
293
334
|
```ruby
|
|
@@ -301,14 +342,14 @@ end
|
|
|
301
342
|
`tag` names the element. `attrs` values are templates: a string is a literal,
|
|
302
343
|
a symbol reads that attribute off the node, an array concatenates both kinds;
|
|
303
344
|
an attribute that resolves empty is left out. `text` (same template form)
|
|
304
|
-
emits literal text content. `contains` declares what lives inside the node
|
|
305
|
-
the default), `:blocks` (child block nodes
|
|
345
|
+
emits literal text content. `contains` declares what lives inside the node: `:inline` (formatted text,
|
|
346
|
+
the default), `:blocks` (child block nodes, a container), or `:none` (a
|
|
306
347
|
leaf). `void: true` skips the closing tag.
|
|
307
348
|
|
|
308
349
|
You don't have to guess any of those names or shapes. Editors store types
|
|
309
350
|
and attributes under names you'd never predict (Rhino's strike mark is
|
|
310
351
|
`rhino-strike`; Lexical prefixes its own props `__`), so ask a real
|
|
311
|
-
document instead
|
|
352
|
+
document instead: make one in your editor using your custom node, then:
|
|
312
353
|
|
|
313
354
|
```ruby
|
|
314
355
|
Y::Tiptap.new(doc).node_types
|
|
@@ -333,25 +374,25 @@ lexical = Y::Lexical.new(doc) do |rules|
|
|
|
333
374
|
end
|
|
334
375
|
```
|
|
335
376
|
|
|
336
|
-
The block gets the node's type, its stored attributes, `node.content`
|
|
337
|
-
children, already rendered to HTML
|
|
377
|
+
The block gets the node's type, its stored attributes, `node.content` (the
|
|
378
|
+
children, already rendered to HTML), and `node.child_types`, the node's
|
|
338
379
|
element/block children by type, in document order. `child_types` answers the
|
|
339
380
|
structural questions attributes can't: how many images a gallery holds, or
|
|
340
381
|
whether a list item carries a nested list. Whatever the block returns is
|
|
341
382
|
spliced into the output as-is: it's trusted HTML, so escape any values you
|
|
342
|
-
interpolate. To set the content mode for a callback, give the node both
|
|
383
|
+
interpolate. To set the content mode for a callback, give the node both:
|
|
343
384
|
`rules.node "embed", contains: :blocks do |node| ... end`.
|
|
344
385
|
|
|
345
386
|
Callbacks never run while the document is locked. The render finishes first
|
|
346
387
|
(inside one read transaction, GVL released), then the blocks run and their
|
|
347
|
-
output is spliced in
|
|
388
|
+
output is spliced in, so a callback can safely read or even write the same
|
|
348
389
|
doc. With no callback rules, `to_html` skips the splicing entirely.
|
|
349
390
|
|
|
350
391
|
Blocks are the escape hatch for everything the declarative form can't say,
|
|
351
392
|
and they're proven sufficient: `Y::Lexxy` and `Y::Tiptap` are themselves
|
|
352
|
-
built on this API (`lib/y/lexxy.rb`, `lib/y/tiptap.rb`)
|
|
393
|
+
built on this API (`lib/y/lexxy.rb`, `lib/y/tiptap.rb`): simple nodes as
|
|
353
394
|
declarative hashes, everything with logic as plain methods mapped by node
|
|
354
|
-
type (a `Method` responds to `call` like any lambda)
|
|
395
|
+
type (a `Method` responds to `call` like any lambda), and the fixture tests
|
|
355
396
|
hold their output byte-identical to a live editor's.
|
|
356
397
|
|
|
357
398
|
The ProseMirror side also takes custom marks:
|
|
@@ -368,7 +409,7 @@ for a built-in mark name (`"bold"`) replaces its built-in tag.
|
|
|
368
409
|
|
|
369
410
|
##### Worked examples
|
|
370
411
|
|
|
371
|
-
A video-embed node from an app's Tiptap extension
|
|
412
|
+
A video-embed node from an app's Tiptap extension, a type the pinned schema
|
|
372
413
|
has never heard of:
|
|
373
414
|
|
|
374
415
|
```ruby
|
|
@@ -395,7 +436,7 @@ tiptap = Y::Tiptap.new(doc) do |rules|
|
|
|
395
436
|
end
|
|
396
437
|
```
|
|
397
438
|
|
|
398
|
-
Overriding a shipped rule
|
|
439
|
+
Overriding a shipped rule: rendering Lexxy uploads as real image markup
|
|
399
440
|
instead of the `<action-text-attachment>` elements ActionText re-renders:
|
|
400
441
|
|
|
401
442
|
```ruby
|
|
@@ -411,7 +452,7 @@ lexxy = Y::Lexxy.new(doc) do |rules|
|
|
|
411
452
|
end
|
|
412
453
|
```
|
|
413
454
|
|
|
414
|
-
Markup that depends on structure
|
|
455
|
+
Markup that depends on structure: `node.child_types` lists the node's
|
|
415
456
|
element/block children in document order, so a layout container can size
|
|
416
457
|
itself by its column count while the columns themselves stay declarative:
|
|
417
458
|
|
|
@@ -424,7 +465,7 @@ tiptap = Y::Tiptap.new(doc) do |rules|
|
|
|
424
465
|
end
|
|
425
466
|
```
|
|
426
467
|
|
|
427
|
-
Content-aware overrides
|
|
468
|
+
Content-aware overrides: dropping the empty paragraphs an editor keeps
|
|
428
469
|
around the cursor, since `node.content` arrives already rendered:
|
|
429
470
|
|
|
430
471
|
```ruby
|
|
@@ -435,10 +476,10 @@ lexical = Y::Lexical.new(doc) do |rules|
|
|
|
435
476
|
end
|
|
436
477
|
```
|
|
437
478
|
|
|
438
|
-
For a larger reference, the gem's own editor schemas ship this way
|
|
479
|
+
For a larger reference, the gem's own editor schemas ship this way; see
|
|
439
480
|
`Y::Lexxy::NODES` in `lib/y/lexxy.rb` (declarative hashes for the simple
|
|
440
|
-
nodes, a plain method per node that needs logic
|
|
441
|
-
header cells, both attachment types
|
|
481
|
+
nodes, a plain method per node that needs logic (galleries, list items,
|
|
482
|
+
header cells, both attachment types) mapped with `method(:name)`) and
|
|
442
483
|
`Y::Tiptap::NODES` in `lib/y/tiptap.rb` (task lists, mentions, the details
|
|
443
484
|
family).
|
|
444
485
|
|
|
@@ -446,7 +487,7 @@ family).
|
|
|
446
487
|
|
|
447
488
|
Classifying and unwrapping wire frames is stateless, so it's exposed as
|
|
448
489
|
`Y` module functions rather than a class. The server never holds presence
|
|
449
|
-
or document state to route a frame
|
|
490
|
+
or document state to route a frame; presence lives in the browser clients, and
|
|
450
491
|
the server only relays awareness frames opaquely.
|
|
451
492
|
|
|
452
493
|
```ruby
|
|
@@ -457,31 +498,84 @@ Y.wrap_update(update_bytes) # => wrap a raw doc update as a sync Update frame
|
|
|
457
498
|
|
|
458
499
|
### ActionCable Integration
|
|
459
500
|
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
501
|
+
In a Rails app, one generator creates the channel and the migration:
|
|
502
|
+
|
|
503
|
+
```bash
|
|
504
|
+
bin/rails generate yrby:install
|
|
505
|
+
bin/rails db:migrate
|
|
506
|
+
```
|
|
507
|
+
|
|
508
|
+
The models ship in the gem, the way Action Text owns
|
|
509
|
+
`ActionText::RichText`:
|
|
510
|
+
|
|
511
|
+
- **`Y::Document`**: one row per document, addressed two ways: by `key`
|
|
512
|
+
(what a channel addresses; one opaque, unique string, sometimes
|
|
513
|
+
app-supplied, never parsed) and, optionally, by polymorphic `record` +
|
|
514
|
+
`name` (which model attribute it backs; `name` is the attribute name,
|
|
515
|
+
`"body"`; one document per attribute per record, the
|
|
516
|
+
ActionText::RichText scheme). Key-only documents leave the binding nil.
|
|
517
|
+
Either side can arrive first: `Y::Document.for(record, name)` finds or
|
|
518
|
+
creates the binding, derives a readable key (`post/1/body`), and adopts
|
|
519
|
+
a key-only row already holding that key, so a channel writing first and
|
|
520
|
+
a binding created later converge on one document. The row also holds
|
|
521
|
+
the merged `state` snapshot, CRDT state only; derived data (rendered
|
|
522
|
+
HTML, search text) is the application's job, typically in the channel's
|
|
523
|
+
on_change. `.load_state(key)` / `.append(key, update)` are the store
|
|
524
|
+
calls the generated channel uses.
|
|
525
|
+
- **`Y::DocumentUpdate`**: the uncompacted tail, one delta per row,
|
|
526
|
+
compacted into `state` and deleted once the tail reaches `compact_every`
|
|
527
|
+
(default 64). Loading reads the snapshot plus the current tail; an
|
|
528
|
+
empty tail returns `state` directly. Compaction serializes on a
|
|
529
|
+
per-document row lock and skips causally-gapped rows; they're
|
|
530
|
+
quarantined until they heal rather than compacted into state or
|
|
531
|
+
deleted. Destroying a document deletes its updates with it.
|
|
532
|
+
|
|
533
|
+
Encrypted storage: `Y::EncryptedDocument` stores `state` and update
|
|
534
|
+
payloads through Active Record encryption on the same tables, the way
|
|
535
|
+
`ActionText::EncryptedRichText` does. Point the channel's
|
|
536
|
+
`on_load`/`on_change` at it instead and configure your app's encryption
|
|
537
|
+
keys. Use one access path per document: rows written encrypted read back
|
|
538
|
+
as ciphertext through the plain classes.
|
|
539
|
+
|
|
540
|
+
The migration creates `y_documents` and `y_document_updates`. To rename
|
|
541
|
+
them, edit the generated migration and point `Y::Document.table_name` /
|
|
542
|
+
`Y::DocumentUpdate.table_name` at the new names in an initializer.
|
|
543
|
+
|
|
544
|
+
Storage is swappable: the channel only needs `on_load` and `on_change`
|
|
545
|
+
answered, and they can point at anything.
|
|
546
|
+
|
|
547
|
+
`include Y::ActionCable` (from the `yrby-rails` gem) is the channel
|
|
548
|
+
integration: the y-websocket protocol (document sync +
|
|
549
|
+
awareness/presence) over ActionCable.
|
|
463
550
|
|
|
464
551
|
```ruby
|
|
465
552
|
# app/channels/document_channel.rb
|
|
466
553
|
class DocumentChannel < ApplicationCable::Channel
|
|
467
|
-
include Y::ActionCable
|
|
554
|
+
include Y::ActionCable
|
|
468
555
|
|
|
469
|
-
on_load { |key|
|
|
470
|
-
on_change { |key, update|
|
|
556
|
+
on_load { |key| Y::Document.load_state(key) } # rebuild from storage
|
|
557
|
+
on_change { |key, update| Y::Document.append(key, update) } # record, then broadcast
|
|
471
558
|
|
|
472
559
|
def subscribed
|
|
560
|
+
return reject unless authorized?(params[:id])
|
|
561
|
+
|
|
473
562
|
sync_subscribed params[:id]
|
|
474
563
|
end
|
|
475
564
|
|
|
476
565
|
def receive(data)
|
|
477
566
|
sync_receive(data, params[:id])
|
|
478
567
|
end
|
|
568
|
+
|
|
569
|
+
private
|
|
570
|
+
|
|
571
|
+
# Everyone is denied until you wire this to your app's auth.
|
|
572
|
+
def authorized?(_document_key) = false
|
|
479
573
|
end
|
|
480
574
|
```
|
|
481
575
|
|
|
482
576
|
The concern is store-backed. A handshake is answered from `on_load`; document
|
|
483
|
-
changes are
|
|
484
|
-
|
|
577
|
+
changes are recorded through `on_change`, then broadcast. Nothing
|
|
578
|
+
authoritative is kept in ActionCable process memory, so
|
|
485
579
|
AnyCable RPC workers, Puma workers, and separate dynos can all handle messages
|
|
486
580
|
for the same document as long as they share the same store and cable adapter.
|
|
487
581
|
|
|
@@ -499,16 +593,18 @@ no single client can relay garbage that breaks the others in a room.
|
|
|
499
593
|
|
|
500
594
|
#### Delivery guarantees
|
|
501
595
|
|
|
502
|
-
The contract is the same at every scale
|
|
596
|
+
The contract is the same at every scale: one process, or hundreds across many
|
|
503
597
|
servers:
|
|
504
598
|
|
|
505
599
|
- **The document always converges.** CRDT updates are commutative and
|
|
506
600
|
idempotent, so out-of-order, duplicate, or concurrent delivery all converge to
|
|
507
601
|
the same correct document. This needs no coordination and holds everywhere.
|
|
508
|
-
- **
|
|
509
|
-
|
|
510
|
-
|
|
511
|
-
|
|
602
|
+
- **An acked update is durable, even one that arrived out of order.** An
|
|
603
|
+
update with a missing dependency is recorded and acked like any other, and
|
|
604
|
+
parks as pending in the document. That missing dependency is an update some
|
|
605
|
+
client still holds unacked, so that client keeps retransmitting it until
|
|
606
|
+
the server records it, and the gap closes. The ack loop is the guarantee.
|
|
607
|
+
See [Causal gaps](#causal-gaps).
|
|
512
608
|
- **`on_change` is at-least-once, and the durable guarantee is that replaying the
|
|
513
609
|
log reconstructs the document.** Every update triggers `on_change` before it's acked or
|
|
514
610
|
broadcast (record-before-distribute). If exactly-once updates matter for you, **you
|
|
@@ -518,13 +614,13 @@ servers:
|
|
|
518
614
|
There is no negative-ack: the client simply never receives the ack, keeps the
|
|
519
615
|
update pending, and retransmits on its timer/reconnect. This is built for
|
|
520
616
|
*transient* failures (the store is briefly down → a retry lands). A block that
|
|
521
|
-
raises *deterministically*
|
|
617
|
+
raises *deterministically* (a validation that always fails for this edit)
|
|
522
618
|
will be retried forever, since nothing tells the client to stop. Enforce hard
|
|
523
619
|
rejections before the edit reaches `on_change` (channel authorization in
|
|
524
620
|
`subscribed`), not by raising inside it.
|
|
525
621
|
- **An over-cap frame is dropped the same silent way.** A frame larger than
|
|
526
|
-
`max_frame_bytes` (default 8 MiB) is dropped before decoding
|
|
527
|
-
broadcast
|
|
622
|
+
`max_frame_bytes` (default 8 MiB) is dropped before decoding (no ack, no
|
|
623
|
+
broadcast) to bound the work a client can force. For a genuine document
|
|
528
624
|
update that means the same implicit rejection as above: unacked, retransmitted
|
|
529
625
|
forever. Normal typing never approaches the cap, but a large paste, an embedded
|
|
530
626
|
image, or a big initial `SyncStep2` can. The drop is logged (`warn` for
|
|
@@ -534,6 +630,85 @@ servers:
|
|
|
534
630
|
genuinely-too-big content upstream rather than relying on the cap to reject it
|
|
535
631
|
gracefully.
|
|
536
632
|
|
|
633
|
+
#### Causal gaps
|
|
634
|
+
|
|
635
|
+
Yjs updates can arrive out of order: an update can reach the server before
|
|
636
|
+
another update it depends on. yrby treats that as normal. The update is
|
|
637
|
+
recorded and acked like any other, parks as a pending struct in the document,
|
|
638
|
+
and integrates on its own the moment the missing dependency lands. The write
|
|
639
|
+
path never rebuilds the document; it appends, relays, and acks, so a gapped
|
|
640
|
+
update costs the same as any other.
|
|
641
|
+
|
|
642
|
+
Serving is lossless too, like any Yjs server. `handle_sync_message` serves
|
|
643
|
+
full state, pending included, so a peer parks the same pending struct and
|
|
644
|
+
heals it the same way. Healing needs no special machinery: the missing
|
|
645
|
+
dependency is an update its sender still holds unacked, and at-least-once
|
|
646
|
+
retransmission delivers it. Only compaction excludes pending
|
|
647
|
+
(`compacted_state_update`), because folding a log must not freeze an
|
|
648
|
+
un-integrable struct into the base state.
|
|
649
|
+
|
|
650
|
+
The bundled `Y::Document` store handles all of this. If you write your own
|
|
651
|
+
store, keep two things in mind:
|
|
652
|
+
|
|
653
|
+
**1. Load losslessly, and tolerate duplicates.** `on_load` should return
|
|
654
|
+
state that preserves pending: `encode_state_as_update`, or a replay of the
|
|
655
|
+
raw append log. Don't compact with `compacted_state_update` while
|
|
656
|
+
`doc.pending?`; that strips the pending struct and the acked edit inside it.
|
|
657
|
+
(`Y::Document` quarantines pending rows for exactly this reason.) A lost ack
|
|
658
|
+
also means a client resends an update the store already has. Replay converges
|
|
659
|
+
anyway, because CRDT apply is idempotent, so deduping is optional. If log
|
|
660
|
+
size matters, dedup by content hash:
|
|
661
|
+
|
|
662
|
+
```ruby
|
|
663
|
+
class DocumentStore
|
|
664
|
+
# append tolerates duplicates: a re-delivered update upserts to a no-op.
|
|
665
|
+
def append(key, update)
|
|
666
|
+
Revision.upsert({ doc_key: key, update_hash: Digest::SHA256.hexdigest(update), update: update },
|
|
667
|
+
unique_by: %i[doc_key update_hash])
|
|
668
|
+
end
|
|
669
|
+
|
|
670
|
+
# load is lossless: replay the raw log so a pending struct is preserved and
|
|
671
|
+
# heals when its dependency arrives.
|
|
672
|
+
def load(key)
|
|
673
|
+
updates = Revision.where(doc_key: key).order(:id).pluck(:update)
|
|
674
|
+
return nil if updates.empty?
|
|
675
|
+
|
|
676
|
+
doc = Y::Doc.new
|
|
677
|
+
updates.each { |u| doc.apply_update(u) }
|
|
678
|
+
doc.encode_state_as_update # lossless: keeps pending
|
|
679
|
+
end
|
|
680
|
+
|
|
681
|
+
# optional compaction: only when there is no open gap, or you would drop it.
|
|
682
|
+
def compact(key)
|
|
683
|
+
doc = Y::Doc.new
|
|
684
|
+
Revision.where(doc_key: key).order(:id).pluck(:update).each { |u| doc.apply_update(u) }
|
|
685
|
+
return if doc.pending? # a gap is open; compacting now would drop it
|
|
686
|
+
# ... replace the log with a single revision holding doc.compacted_state_update ...
|
|
687
|
+
end
|
|
688
|
+
end
|
|
689
|
+
```
|
|
690
|
+
|
|
691
|
+
**2. Watch for gaps that never heal.** An open gap is quiet: the edit sits
|
|
692
|
+
as pending, invisible in the document, until its dependency arrives. Normally
|
|
693
|
+
that resolves itself. The sender retransmits the missing update until it is
|
|
694
|
+
acked, and every join or reconnect handshake has the client send everything
|
|
695
|
+
the server hasn't integrated, so any client holding the dependency supplies
|
|
696
|
+
it just by connecting. The gap worth alerting on is one no live client can
|
|
697
|
+
supply, and that is what the `on_gap` hook surfaces. It fires with the
|
|
698
|
+
document key whenever a document is loaded to serve state and a gap is still
|
|
699
|
+
open. Use it to emit a metric (a pending-document count, or the age of the
|
|
700
|
+
oldest open gap) so a stuck gap is visible. Gaps are also logged at `info`,
|
|
701
|
+
and errors raised in the hook are swallowed so observability can never break
|
|
702
|
+
frame handling.
|
|
703
|
+
|
|
704
|
+
```ruby
|
|
705
|
+
class DocumentChannel < ApplicationCable::Channel
|
|
706
|
+
include Y::ActionCable
|
|
707
|
+
|
|
708
|
+
on_gap { |key| StatsD.increment("yrby.gap", tags: ["doc:#{key}"]) }
|
|
709
|
+
end
|
|
710
|
+
```
|
|
711
|
+
|
|
537
712
|
#### Multi-process deployments
|
|
538
713
|
|
|
539
714
|
Most Rails apps run several processes, and any of them might serve a given document.
|
|
@@ -571,7 +746,7 @@ It is up to you to durably record it:
|
|
|
571
746
|
|
|
572
747
|
```ruby
|
|
573
748
|
class DocumentChannel < ApplicationCable::Channel
|
|
574
|
-
include Y::ActionCable
|
|
749
|
+
include Y::ActionCable
|
|
575
750
|
|
|
576
751
|
# ...
|
|
577
752
|
|
|
@@ -593,12 +768,86 @@ duplicate record replays to the same document.
|
|
|
593
768
|
The demo wires `on_change` to a durable Postgres-backed log by default, and checks
|
|
594
769
|
end to end that the log alone rebuilds the document.
|
|
595
770
|
|
|
771
|
+
#### Ephemeral documents (no database)
|
|
772
|
+
|
|
773
|
+
`on_load` and `on_change` are plain blocks, and nothing requires them to touch
|
|
774
|
+
a database. For documents that don't need to outlive their session (a
|
|
775
|
+
scratchpad, live form state, a draft you only persist on submit) the store
|
|
776
|
+
can be connection state that travels with each request:
|
|
777
|
+
|
|
778
|
+
```ruby
|
|
779
|
+
class ScratchpadChannel < ApplicationCable::Channel
|
|
780
|
+
include Y::ActionCable
|
|
781
|
+
|
|
782
|
+
on_load { |key| @doc_state }
|
|
783
|
+
|
|
784
|
+
on_change do |key, update|
|
|
785
|
+
doc = Y::Doc.new
|
|
786
|
+
doc.apply_update(@doc_state) if @doc_state
|
|
787
|
+
doc.apply_update(update)
|
|
788
|
+
@doc_state = doc.compacted_state_update
|
|
789
|
+
end
|
|
790
|
+
|
|
791
|
+
def subscribed = sync_subscribed(params[:id])
|
|
792
|
+
def receive(data) = sync_receive(data, params[:id])
|
|
793
|
+
end
|
|
794
|
+
```
|
|
795
|
+
|
|
796
|
+
On AnyCable the channel object doesn't survive between messages, so an
|
|
797
|
+
instance variable won't hold. Declare the store as channel state instead
|
|
798
|
+
(`state_attr_accessor` comes from anycable-rails) and Base64 it, because that
|
|
799
|
+
state is serialized as JSON into each RPC exchange with `anycable-go`:
|
|
800
|
+
|
|
801
|
+
```ruby
|
|
802
|
+
class ScratchpadChannel < ApplicationCable::Channel
|
|
803
|
+
include Y::ActionCable
|
|
804
|
+
|
|
805
|
+
state_attr_accessor :doc_state
|
|
806
|
+
|
|
807
|
+
on_load { |key| doc_state && Base64.strict_decode64(doc_state) }
|
|
808
|
+
|
|
809
|
+
on_change do |key, update|
|
|
810
|
+
doc = Y::Doc.new
|
|
811
|
+
doc.apply_update(Base64.strict_decode64(doc_state)) if doc_state
|
|
812
|
+
doc.apply_update(update)
|
|
813
|
+
self.doc_state = Base64.strict_encode64(doc.compacted_state_update)
|
|
814
|
+
end
|
|
815
|
+
|
|
816
|
+
def subscribed = sync_subscribed(params[:id])
|
|
817
|
+
def receive(data) = sync_receive(data, params[:id])
|
|
818
|
+
end
|
|
819
|
+
```
|
|
820
|
+
|
|
821
|
+
Both hooks run in the channel instance (`instance_exec`), so they can use
|
|
822
|
+
anything the channel can, and `sync_receive` rebuilds the document from
|
|
823
|
+
`on_load` on every update, which is what lets the store live on the
|
|
824
|
+
connection. On Action Cable the channel instance lasts as long as the
|
|
825
|
+
connection, so an instance variable is the whole store. Merging into
|
|
826
|
+
`compacted_state_update` keeps it one blob instead of a growing update log.
|
|
827
|
+
|
|
828
|
+
The store is per connection, which shapes what this fits. A single writer gets
|
|
829
|
+
the full delivery contract with no database anywhere. With several people
|
|
830
|
+
editing at once, one client's update can depend on edits its own connection
|
|
831
|
+
has never seen; that update records as pending, and the next handshake with
|
|
832
|
+
that client (which always holds the full document) supplies the missing state
|
|
833
|
+
and heals it. The document still converges; heavy concurrent editing just
|
|
834
|
+
parks more pending between handshakes than a shared store would. On
|
|
835
|
+
AnyCable, keep the payload in mind too: the blob travels with every message,
|
|
836
|
+
so that variant suits small documents, not long manuscripts.
|
|
837
|
+
|
|
838
|
+
Durability is the connection plus the browsers. A reconnecting client re-seeds
|
|
839
|
+
an empty server through the ordinary sync handshake, so the document survives
|
|
840
|
+
server restarts as long as some client still has it. For ephemeral documents
|
|
841
|
+
shared across clients on a single-process deployment, the same two hooks over
|
|
842
|
+
a class-level `Concurrent::Map` work instead; that version stops being
|
|
843
|
+
coherent the moment you scale past one process.
|
|
844
|
+
|
|
596
845
|
#### Reliable delivery (acks)
|
|
597
846
|
|
|
598
847
|
yrby document delivery is ack-tracked. Browser document updates carry an
|
|
599
|
-
`"id"`, and the server replies `{ "ack": <id> }` once `on_change` has
|
|
600
|
-
|
|
601
|
-
|
|
848
|
+
`"id"`, and the server replies `{ "ack": <id> }` once `on_change` has
|
|
849
|
+
successfully fired. Every decodable document update is recorded and acked,
|
|
850
|
+
including one that arrives out of order.
|
|
602
851
|
|
|
603
852
|
```
|
|
604
853
|
client -> server { "update": "<base64 update>", "id": 42 }
|
|
@@ -612,13 +861,13 @@ one `{ ack: id }` cumulatively confirms everything up to it. Because CRDT apply
|
|
|
612
861
|
is idempotent, a resend that already landed is a harmless no-op that just
|
|
613
862
|
re-acks. Awareness stays ephemeral and is not acked.
|
|
614
863
|
|
|
615
|
-
Presence (cursors, selections) is owned by the browser clients
|
|
864
|
+
Presence (cursors, selections) is owned by the browser clients; the server
|
|
616
865
|
never sets or holds presence state, it only relays awareness frames opaquely.
|
|
617
866
|
See `yrby-client` for the client-side awareness API.
|
|
618
867
|
|
|
619
868
|
## Thread Safety
|
|
620
869
|
|
|
621
|
-
A `Doc` is safe to share across Ruby threads
|
|
870
|
+
A `Doc` is safe to share across Ruby threads, used concurrently from Puma
|
|
622
871
|
workers, ActionCable connection threads, or background jobs without external
|
|
623
872
|
locking.
|
|
624
873
|
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "rails/generators"
|
|
4
|
+
require "generators/yrby/tables/tables_generator"
|
|
5
|
+
|
|
6
|
+
module Yrby
|
|
7
|
+
module Generators
|
|
8
|
+
# `bin/rails generate yrby:install` — a DocumentChannel speaking the
|
|
9
|
+
# y-websocket protocol over the gem's document storage, plus the storage
|
|
10
|
+
# migration (via yrby:tables). The models ship in the gem; only the
|
|
11
|
+
# migration lands in the app.
|
|
12
|
+
class InstallGenerator < ::Rails::Generators::Base
|
|
13
|
+
source_root File.expand_path("templates", __dir__)
|
|
14
|
+
|
|
15
|
+
def create_channel
|
|
16
|
+
template "document_channel.rb", "app/channels/document_channel.rb"
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
def create_tables
|
|
20
|
+
invoke "yrby:tables"
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
def show_next_steps
|
|
24
|
+
say <<~NEXT
|
|
25
|
+
|
|
26
|
+
Next steps:
|
|
27
|
+
|
|
28
|
+
1. Authorize document access: implement `authorized?` in
|
|
29
|
+
app/channels/document_channel.rb (it denies everyone until you do).
|
|
30
|
+
2. bin/rails db:migrate
|
|
31
|
+
3. Install the yrby-client npm package and connect an editor:
|
|
32
|
+
|
|
33
|
+
import { ActionCableProvider } from "yrby-client"
|
|
34
|
+
const provider = new ActionCableProvider(doc, consumer,
|
|
35
|
+
"DocumentChannel", { id: documentId })
|
|
36
|
+
provider.connect()
|
|
37
|
+
|
|
38
|
+
The README's Editors section links working integrations for
|
|
39
|
+
Tiptap, Lexxy, Rhino Editor, and CodeMirror.
|
|
40
|
+
NEXT
|
|
41
|
+
end
|
|
42
|
+
end
|
|
43
|
+
end
|
|
44
|
+
end
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# Collaborative documents over Action Cable: one channel speaking the
|
|
4
|
+
# y-websocket protocol (sync plus presence). Storage is Y::Document +
|
|
5
|
+
# Y::DocumentUpdate; a document is created on its first change and its
|
|
6
|
+
# history goes with it when it is destroyed. Point on_load/on_change
|
|
7
|
+
# elsewhere to swap storage.
|
|
8
|
+
class DocumentChannel < ApplicationCable::Channel
|
|
9
|
+
include Y::ActionCable
|
|
10
|
+
|
|
11
|
+
# Rebuild a document from durable storage (nil means a brand-new document).
|
|
12
|
+
on_load { |key| Y::Document.load_state(key) }
|
|
13
|
+
|
|
14
|
+
# Record each CRDT delta durably. Runs before the change is acknowledged
|
|
15
|
+
# or broadcast; if this raises, the change is neither acked nor relayed,
|
|
16
|
+
# and yrby-client retries it.
|
|
17
|
+
on_change { |key, update| Y::Document.append(key, update) }
|
|
18
|
+
|
|
19
|
+
def subscribed
|
|
20
|
+
return reject unless authorized?(params[:id])
|
|
21
|
+
|
|
22
|
+
sync_subscribed(params[:id])
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
def receive(data) = sync_receive(data, params[:id])
|
|
26
|
+
|
|
27
|
+
private
|
|
28
|
+
|
|
29
|
+
# Everyone is denied until you fill this in. Wire it to your app's auth:
|
|
30
|
+
# identify current_user on the cable connection, then check they may read
|
|
31
|
+
# and write this document. Don't lean on on_change raising for access
|
|
32
|
+
# control; that path exists for store failures.
|
|
33
|
+
def authorized?(_document_key)
|
|
34
|
+
false
|
|
35
|
+
end
|
|
36
|
+
end
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "rails/generators"
|
|
4
|
+
require "rails/generators/active_record"
|
|
5
|
+
|
|
6
|
+
module Yrby
|
|
7
|
+
module Generators
|
|
8
|
+
# `bin/rails generate yrby:tables`: the migration for the gem-owned
|
|
9
|
+
# document models (Y::Document + Y::DocumentUpdate). Invoked by
|
|
10
|
+
# yrby:install, and by other gems building on the same storage.
|
|
11
|
+
#
|
|
12
|
+
# Template notes (kept here, not in the emitted migration): state is
|
|
13
|
+
# 4.gigabytes - 1 (longblob on MySQL; a compacted snapshot is the whole
|
|
14
|
+
# document; a 16 MB cap would break compaction) and payload is
|
|
15
|
+
# 16.megabytes - 1 (one update can carry a big paste or a client's
|
|
16
|
+
# accumulated offline edits; the 64 KB default blob is too small).
|
|
17
|
+
# The partial unique index's WHERE only keeps
|
|
18
|
+
# key-only rows out of the index: uniqueness holds without it, since
|
|
19
|
+
# unique indexes treat NULLs as distinct on every supported database,
|
|
20
|
+
# and MySQL drops the predicate harmlessly. y_document_updates indexes
|
|
21
|
+
# (document_id, pending) instead of bare document_id: the prefix
|
|
22
|
+
# serves the tail and foreign-key lookups, and the pair serves the
|
|
23
|
+
# clean-row count every append runs.
|
|
24
|
+
class TablesGenerator < ::Rails::Generators::Base
|
|
25
|
+
include ActiveRecord::Generators::Migration
|
|
26
|
+
|
|
27
|
+
source_root File.expand_path("templates", __dir__)
|
|
28
|
+
|
|
29
|
+
def create_migration_file
|
|
30
|
+
migration_template "create_y_tables.rb",
|
|
31
|
+
File.join(db_migrate_path, "create_y_tables.rb")
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
private
|
|
35
|
+
|
|
36
|
+
def migration_version
|
|
37
|
+
"[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
|
|
38
|
+
end
|
|
39
|
+
end
|
|
40
|
+
end
|
|
41
|
+
end
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
class CreateYTables < ActiveRecord::Migration<%= migration_version %>
|
|
4
|
+
def change
|
|
5
|
+
create_table :y_documents do |t|
|
|
6
|
+
t.string :key, null: false, index: { unique: true }
|
|
7
|
+
t.references :record, polymorphic: true, null: true, index: false
|
|
8
|
+
t.string :name
|
|
9
|
+
t.binary :state, limit: 4.gigabytes - 1
|
|
10
|
+
t.timestamps
|
|
11
|
+
t.index %i[record_type record_id name], unique: true,
|
|
12
|
+
where: "record_type IS NOT NULL",
|
|
13
|
+
name: "index_y_documents_on_record_and_name"
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
create_table :y_document_updates do |t|
|
|
17
|
+
t.references :document, null: false, foreign_key: { to_table: :y_documents }, index: false
|
|
18
|
+
t.binary :payload, null: false, limit: 16.megabytes - 1
|
|
19
|
+
t.boolean :pending, null: false, default: false
|
|
20
|
+
t.datetime :created_at, null: false
|
|
21
|
+
t.index %i[document_id pending]
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
end
|
data/lib/y/3.4/yrby.so
CHANGED
|
Binary file
|
data/lib/y/4.0/yrby.so
CHANGED
|
Binary file
|
data/lib/y/decoder.rb
ADDED
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "y"
|
|
4
|
+
|
|
5
|
+
module Y
|
|
6
|
+
# Plain-text reconstruction of a stored Yjs document, for search indexing
|
|
7
|
+
# and previews. It reads the text out of the shared type the editor uses
|
|
8
|
+
# (Lexical's `Y.XmlText`, plain `Y.Text`, or ProseMirror's `Y.XmlFragment`)
|
|
9
|
+
# in-process, on the same native extension as `Doc`: no Node, no
|
|
10
|
+
# subprocess.
|
|
11
|
+
#
|
|
12
|
+
# state = doc.encode_state_as_update # opaque CRDT bytes from the store
|
|
13
|
+
# Y::Decoder.text(state) # => "hello world"
|
|
14
|
+
# Y::Decoder.preview(state, 280) # => "hello world…"
|
|
15
|
+
#
|
|
16
|
+
# For full-fidelity HTML, `Y::Lexxy` and `Y::Tiptap` render the document
|
|
17
|
+
# with the editor's own semantics; this module is the cheap plain-text
|
|
18
|
+
# path.
|
|
19
|
+
module Decoder
|
|
20
|
+
class Error < Y::Error; end
|
|
21
|
+
|
|
22
|
+
module_function
|
|
23
|
+
|
|
24
|
+
# Plain text of the document. `field` pins the root key (Lexical: the editor
|
|
25
|
+
# id; ProseMirror: "default"); omit it to use the document's sole root.
|
|
26
|
+
def text(state, field: nil)
|
|
27
|
+
field ||= Y::Doc.new.tap { |d| d.apply_update(state) }.root_names.first
|
|
28
|
+
return "" unless field
|
|
29
|
+
|
|
30
|
+
# A plain `Y.Text` root (a simple shared-text editor) reads straight out.
|
|
31
|
+
# (A yrs root's type is fixed by its first typed access, so each reader
|
|
32
|
+
# gets a fresh doc to try a different shared type against the same state.)
|
|
33
|
+
direct = load(state).read_text(field)
|
|
34
|
+
return normalize(direct) if direct && !direct.strip.empty?
|
|
35
|
+
|
|
36
|
+
# Lexical (each block a sibling `Y.XmlText`) and ProseMirror (blocks are
|
|
37
|
+
# `Y.XmlElement`s) both come back from read_xml as block-per-line markup;
|
|
38
|
+
# strip any element tags to plain text.
|
|
39
|
+
markup = load(state).read_xml(field)
|
|
40
|
+
markup ? normalize(strip_tags(markup)) : ""
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
# A compact, single-line preview for list UIs.
|
|
44
|
+
def preview(state, limit: 280, field: nil)
|
|
45
|
+
body = text(state, field: field).gsub(/\s+/, " ").strip
|
|
46
|
+
body.length > limit ? "#{body[0, limit].rstrip}…" : body
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
def load(state)
|
|
50
|
+
Y::Doc.new.tap { |doc| doc.apply_update(state) }
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
def strip_tags(markup)
|
|
54
|
+
markup.gsub(/<[^>]*>/, " ")
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
def normalize(text)
|
|
58
|
+
text.gsub(/[ \t]+/, " ") # collapse runs of spaces/tabs
|
|
59
|
+
.gsub(/ *\n */, "\n") # trim spaces left around block separators
|
|
60
|
+
.gsub(/\n{3,}/, "\n\n") # cap blank-line runs
|
|
61
|
+
.strip
|
|
62
|
+
end
|
|
63
|
+
end
|
|
64
|
+
end
|
data/lib/y/lexxy.rb
CHANGED
|
@@ -52,24 +52,44 @@ module Y
|
|
|
52
52
|
# and presence mirror Lexxy's exportDOM (nulls omitted, `previewable`
|
|
53
53
|
# only when true, `presentation="gallery"` always).
|
|
54
54
|
def self.upload(node)
|
|
55
|
-
|
|
55
|
+
tag = attachment_tag(node)
|
|
56
|
+
out = "<#{tag}"
|
|
56
57
|
out << attachment_attr(node, "sgid", "sgid")
|
|
57
58
|
out << %( previewable="true") if node.attrs["previewable"] == true
|
|
58
59
|
[%w[url src], %w[alt altText], %w[caption caption],
|
|
59
60
|
%w[content-type contentType], %w[filename fileName],
|
|
60
61
|
%w[filesize fileSize], %w[width width], %w[height height]]
|
|
61
62
|
.each { |html_name, stored| out << attachment_attr(node, html_name, stored) }
|
|
62
|
-
%(#{out} presentation="gallery"
|
|
63
|
+
%(#{out} presentation="gallery"></#{tag}>)
|
|
63
64
|
end
|
|
64
65
|
|
|
65
66
|
# A content attachment (mention, embed): `content` carries the escaped
|
|
66
67
|
# inner HTML; `plainText` is not exported.
|
|
67
68
|
def self.mention(node)
|
|
68
|
-
|
|
69
|
+
tag = attachment_tag(node)
|
|
70
|
+
out = "<#{tag}"
|
|
69
71
|
out << attachment_attr(node, "sgid", "sgid")
|
|
70
72
|
out << attachment_attr(node, "content", "innerHtml")
|
|
71
73
|
out << attachment_attr(node, "content-type", "contentType")
|
|
72
|
-
"#{out}
|
|
74
|
+
"#{out}></#{tag}>"
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
# Lexxy's attachment tag is configurable (`Lexxy.configure`'s
|
|
78
|
+
# attachmentTagName, paired with ActionText::Attachment.tag_name on the
|
|
79
|
+
# Rails side), and each attachment node stores the tag it was created
|
|
80
|
+
# with. Emit the stored tag. The value is stored document data, not
|
|
81
|
+
# markup, so anything that doesn't look like a tag name falls back to
|
|
82
|
+
# ActionText's default.
|
|
83
|
+
def self.attachment_tag(node)
|
|
84
|
+
tag = node.attrs["tagName"].to_s
|
|
85
|
+
tag.match?(/\A[a-zA-Z][a-zA-Z0-9-]*\z/) ? tag : "action-text-attachment"
|
|
86
|
+
end
|
|
87
|
+
|
|
88
|
+
# An in-flight upload placeholder. Only the uploader's browser can
|
|
89
|
+
# complete it, so it is never finished content -- materialized output
|
|
90
|
+
# (Action Text bodies, search text) must not carry it.
|
|
91
|
+
def self.pending_upload(_node)
|
|
92
|
+
""
|
|
73
93
|
end
|
|
74
94
|
|
|
75
95
|
# A stored nil (unset) is skipped; a stored empty string still emits.
|
|
@@ -90,6 +110,7 @@ module Y
|
|
|
90
110
|
"tablecell" => { contains: :blocks, render: method(:table_cell) },
|
|
91
111
|
"listitem" => { contains: :blocks, render: method(:list_item) },
|
|
92
112
|
"action_text_attachment" => method(:upload),
|
|
113
|
+
"action_text_attachment_upload" => method(:pending_upload),
|
|
93
114
|
"custom_action_text_attachment" => method(:mention)
|
|
94
115
|
}.freeze
|
|
95
116
|
|
data/lib/y/version.rb
CHANGED
data/lib/y.rb
CHANGED
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: yrby
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.7.0
|
|
5
5
|
platform: aarch64-linux
|
|
6
6
|
authors:
|
|
7
7
|
- JP Camara
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-
|
|
11
|
+
date: 2026-08-12 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: minitest
|
|
@@ -55,7 +55,7 @@ dependencies:
|
|
|
55
55
|
description: 'yrby is a thread-safe Ruby binding over the Rust y-crdt (yrs) library:
|
|
56
56
|
CRDT documents, awareness/presence, and the y-websocket sync protocol primitives,
|
|
57
57
|
with the GVL released during native work so documents sync in parallel. The ActionCable/Rails
|
|
58
|
-
integration lives in the companion yrby-
|
|
58
|
+
integration lives in the companion yrby-rails gem.'
|
|
59
59
|
email:
|
|
60
60
|
- johnpcamara@gmail.com
|
|
61
61
|
executables: []
|
|
@@ -65,9 +65,14 @@ files:
|
|
|
65
65
|
- CHANGELOG.md
|
|
66
66
|
- LICENSE
|
|
67
67
|
- README.md
|
|
68
|
+
- lib/generators/yrby/install/install_generator.rb
|
|
69
|
+
- lib/generators/yrby/install/templates/document_channel.rb
|
|
70
|
+
- lib/generators/yrby/tables/tables_generator.rb
|
|
71
|
+
- lib/generators/yrby/tables/templates/create_y_tables.rb
|
|
68
72
|
- lib/y.rb
|
|
69
73
|
- lib/y/3.4/yrby.so
|
|
70
74
|
- lib/y/4.0/yrby.so
|
|
75
|
+
- lib/y/decoder.rb
|
|
71
76
|
- lib/y/lexxy.rb
|
|
72
77
|
- lib/y/rendering.rb
|
|
73
78
|
- lib/y/tiptap.rb
|