yrb-lite 0.1.0.beta7 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +0 -132
- data/README.md +92 -82
- data/ext/yrb_lite/src/lib.rs +64 -373
- data/ext/yrb_lite/src/protocol.rs +22 -21
- data/lib/yrb-lite.rb +1 -1
- data/lib/yrb_lite/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 44da78141b84de4cf438a8b0315b4fdf92bf10290787f569e1c9f63087f2abf8
|
|
4
|
+
data.tar.gz: dce6b3a9227637b8e8828c8d23726b178107c19dd8670bd580da6d71d848bb6a
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: d4b3df640c6e502acd8545e18e019998c8d4200f3e7c3a6e70fc9f4616dc468fd57253b55d149713d314d4b706a029716dc42a09bbbe461e22c9ec3247241689
|
|
7
|
+
data.tar.gz: 14673427e3ebd711859572c5b589cdccfb0e8727d7e767daac214e9c5cef8d35fa12c3e7192ed415a4e18c3cedae6e4bf03b08ac1070fe4bec9e160a6aede1a7
|
data/CHANGELOG.md
CHANGED
|
@@ -5,135 +5,3 @@ All notable changes to this project are documented here. The format is based on
|
|
|
5
5
|
to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
|
-
|
|
9
|
-
## [0.1.0.beta6] - 2026-06-22
|
|
10
|
-
|
|
11
|
-
(yrb-lite core gem. The `yrb-lite-client` npm package ships these client changes as 0.1.2.)
|
|
12
|
-
|
|
13
|
-
### Added
|
|
14
|
-
|
|
15
|
-
- `yrb-lite-client`, the TypeScript client package for the yrb-lite
|
|
16
|
-
ActionCable/AnyCable protocol. It provides `ActionCableProvider`,
|
|
17
|
-
`YProtocolSession`, and the standalone `ReliableSync` delivery core.
|
|
18
|
-
|
|
19
|
-
### Changed
|
|
20
|
-
|
|
21
|
-
- Document delivery is ack-tracked by default in `yrb-lite-client`: document
|
|
22
|
-
frames use `{ update, id }`, acknowledgements use `{ ack }`, and pending
|
|
23
|
-
document updates stay queued until acked.
|
|
24
|
-
- The ActionCable protocol surface uses a single canonical document envelope:
|
|
25
|
-
`{ "update" => "<base64 frame>" }`.
|
|
26
|
-
- AnyCable awareness/presence uses an awareness-only whisper envelope,
|
|
27
|
-
`{ awareness: "<base64 awareness frame>" }`, while document frames stay on
|
|
28
|
-
the server persistence/ack path.
|
|
29
|
-
|
|
30
|
-
### Fixed
|
|
31
|
-
|
|
32
|
-
- Incoming protocol frames are validated before mutating documents or awareness
|
|
33
|
-
state, including trailing-byte rejection on the TypeScript client.
|
|
34
|
-
- Native/Rust protocol entry points reject wire client IDs that are unsafe for
|
|
35
|
-
JavaScript clients.
|
|
36
|
-
- `lib0` is declared as a direct runtime dependency of `yrb-lite-client`.
|
|
37
|
-
|
|
38
|
-
## [0.1.0.beta5] - 2026-06-18
|
|
39
|
-
|
|
40
|
-
### Changed
|
|
41
|
-
|
|
42
|
-
- **Breaking:** the ActionCable integration has been extracted into a separate
|
|
43
|
-
gem, [`yrb-lite-actioncable`](https://rubygems.org/gems/yrb-lite-actioncable).
|
|
44
|
-
`yrb-lite` is now a standalone y-crdt wrapper: CRDT documents, awareness, and
|
|
45
|
-
the y-websocket sync protocol primitives, with no Rails/ActionCable coupling
|
|
46
|
-
(mirrors the `y-rb` / `yrb-actioncable` split). The `base64` runtime
|
|
47
|
-
dependency moved with it.
|
|
48
|
-
|
|
49
|
-
### Migration
|
|
50
|
-
|
|
51
|
-
- Using `YrbLite::Sync`? Add `gem "yrb-lite-actioncable"` and change
|
|
52
|
-
`include YrbLite::Sync` to `include YrbLite::ActionCable::Sync`. The concern's
|
|
53
|
-
API is otherwise unchanged. If you only use `YrbLite::Doc`/`YrbLite::Awareness`,
|
|
54
|
-
nothing changes.
|
|
55
|
-
|
|
56
|
-
## [0.1.0.beta4] - 2026-06-18
|
|
57
|
-
|
|
58
|
-
### Changed
|
|
59
|
-
|
|
60
|
-
- `on_change` block recorders now run in the **channel instance's context**
|
|
61
|
-
(via `instance_exec`), so a recorder can call the channel's own methods --
|
|
62
|
-
`current_user`, `params`, request/connection-scoped accessors -- directly,
|
|
63
|
-
instead of plumbing them in through a thread-local. A non-Proc callable (an
|
|
64
|
-
object responding to `#call`) is still invoked with `#call` and its own
|
|
65
|
-
context. Existing block recorders that use
|
|
66
|
-
only the `(key, update)` arguments and lexically-scoped constants are
|
|
67
|
-
unaffected; the only behavioral change is `self` inside the block.
|
|
68
|
-
|
|
69
|
-
## [0.1.0.beta3] - 2026-06-18
|
|
70
|
-
|
|
71
|
-
### Changed
|
|
72
|
-
|
|
73
|
-
- Upgraded the bundled `yrs` (y-crdt) from 0.21 to 0.27.2. No change to the
|
|
74
|
-
`YrbLite::Doc`, `YrbLite::Awareness`, or `YrbLite::Sync` public API; existing
|
|
75
|
-
code and the wire protocol are unaffected.
|
|
76
|
-
- Thread-safety is preserved across the upgrade. yrs 0.27 dropped `Awareness`'s
|
|
77
|
-
internal locking (its mutating methods now take `&mut self`, and `Awareness`
|
|
78
|
-
is no longer `Sync`), so `YrbLite::Awareness` now serializes access through an
|
|
79
|
-
internal `Mutex`. The lock is taken only while the GVL is released and is
|
|
80
|
-
never held across the GVL boundary, so concurrent access from multiple Ruby
|
|
81
|
-
threads stays safe and deadlock-free, and document reads still run in parallel
|
|
82
|
-
(they operate on a cheaply-cloned, `Arc`-backed `Doc` handle, not under the
|
|
83
|
-
presence lock).
|
|
84
|
-
|
|
85
|
-
### Build
|
|
86
|
-
|
|
87
|
-
- Building the gem from source now requires **Rust 1.94 or newer** (yrs 0.27.2
|
|
88
|
-
uses `let`-chains). The precompiled platform gems are unaffected -- they need
|
|
89
|
-
no Rust toolchain to install.
|
|
90
|
-
|
|
91
|
-
## [0.1.0.beta2] - 2026-06-16
|
|
92
|
-
|
|
93
|
-
### Added
|
|
94
|
-
|
|
95
|
-
- Reliable delivery (opt-in, client-driven). A client may tag a document update
|
|
96
|
-
with an `"id"`; the server replies `{ "ack": <id> }` once the update has been
|
|
97
|
-
durably recorded. This lets an
|
|
98
|
-
ack-aware client retain and retransmit an update until delivery is confirmed,
|
|
99
|
-
so an edit can't be silently lost on a flaky connection. Clients that omit
|
|
100
|
-
`"id"` are still accepted, but their delivery is not ack-tracked.
|
|
101
|
-
- Demo coverage for reliable delivery with "sync-since-last-ack" framing (the
|
|
102
|
-
unacknowledged tail is sent as one merged, causally-complete delta), plus a
|
|
103
|
-
minimal reference client and an intensive message-loss stress test.
|
|
104
|
-
|
|
105
|
-
### Fixed
|
|
106
|
-
|
|
107
|
-
- Causal-gap protection. The authoritative, fast, and store paths now reject a
|
|
108
|
-
document update that isn't causally ready -- one whose dependencies are
|
|
109
|
-
missing because an earlier update was lost in transit or its durable record
|
|
110
|
-
failed -- and ask the client to resync, instead of recording or relaying an
|
|
111
|
-
un-integrable update that would leave the log permanently pending. Adds native
|
|
112
|
-
`Doc#update_ready?`/`#pending?` (cheap, read-only checks) used to gate the
|
|
113
|
-
record-before-distribute path.
|
|
114
|
-
|
|
115
|
-
## [0.1.0.beta1]
|
|
116
|
-
|
|
117
|
-
### Added
|
|
118
|
-
|
|
119
|
-
- Thread-safe `YrbLite::Doc` and `YrbLite::Awareness` over `yrs` (magnus/rb-sys
|
|
120
|
-
native extension). The GVL is released during CRDT work so docs can run in
|
|
121
|
-
parallel on MRI.
|
|
122
|
-
- `YrbLite::Sync` ActionCable channel concern implementing the y-websocket
|
|
123
|
-
protocol (document sync plus awareness/presence).
|
|
124
|
-
- A "record-before-distribute" mode via an `on_change` hook, so every change is
|
|
125
|
-
recorded durably before it's applied or relayed.
|
|
126
|
-
- Presence cleanup on disconnect, and idle-document eviction.
|
|
127
|
-
- Store-backed ActionCable delivery for AnyCable and multi-process use.
|
|
128
|
-
- Hardening against bad input: malformed or multi-message frames are dropped
|
|
129
|
-
before processing or relay, and native panics are contained at the FFI
|
|
130
|
-
boundary.
|
|
131
|
-
- Precompiled native gems for common platforms (no Rust toolchain needed to
|
|
132
|
-
install) via the cross-gem workflow.
|
|
133
|
-
|
|
134
|
-
[Unreleased]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta5...main
|
|
135
|
-
[0.1.0.beta5]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta4...v0.1.0.beta5
|
|
136
|
-
[0.1.0.beta4]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta3...v0.1.0.beta4
|
|
137
|
-
[0.1.0.beta3]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta2...v0.1.0.beta3
|
|
138
|
-
[0.1.0.beta2]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta1...v0.1.0.beta2
|
|
139
|
-
[0.1.0.beta1]: https://github.com/jpcamara/yrb-lite/releases/tag/v0.1.0.beta1
|
data/README.md
CHANGED
|
@@ -11,20 +11,19 @@ documents.
|
|
|
11
11
|
class DocumentChannel < ApplicationCable::Channel
|
|
12
12
|
include YrbLite::ActionCable::Sync
|
|
13
13
|
|
|
14
|
-
def subscribed =
|
|
14
|
+
def subscribed = sync_subscribed(params[:id])
|
|
15
15
|
def receive(data) = sync_receive(data)
|
|
16
|
-
def unsubscribed = sync_unsubscribed(params[:id])
|
|
17
16
|
end
|
|
18
17
|
```
|
|
19
18
|
|
|
20
19
|
On the browser, use the `yrb-lite-client` `ActionCableProvider`. Tiptap,
|
|
21
20
|
ProseMirror, and BlockNote all sync through the `Y.Doc` you pass in and the
|
|
22
|
-
provider's Awareness instance
|
|
21
|
+
provider's Awareness instance.
|
|
23
22
|
|
|
24
23
|
## What you get
|
|
25
24
|
|
|
26
|
-
-
|
|
27
|
-
|
|
25
|
+
- A thread-safe Ruby `Doc` you can share across Puma threads; native CRDT work
|
|
26
|
+
runs with the GVL released.
|
|
28
27
|
- The y-websocket protocol (document sync plus awareness/presence) as a
|
|
29
28
|
one-include ActionCable concern.
|
|
30
29
|
- Store-backed ActionCable/AnyCable delivery for multi-process deployments.
|
|
@@ -35,6 +34,22 @@ What it doesn't do: auth, read-only connections, rate limiting, webhooks,
|
|
|
35
34
|
metrics. Hocuspocus ships extensions for those; here you'd build them with
|
|
36
35
|
Rails.
|
|
37
36
|
|
|
37
|
+
## Why "lite"
|
|
38
|
+
|
|
39
|
+
The "lite" is the size of the surface. yrb-lite binds just the part of y-crdt you
|
|
40
|
+
need to *sync and persist* collaborative documents — a `Doc`, awareness, and the
|
|
41
|
+
y-websocket protocol primitives. The Ruby side treats a document as opaque CRDT
|
|
42
|
+
state: it applies updates, answers sync handshakes, and records deltas, but never
|
|
43
|
+
reaches in to read or edit the contents. The browser editor owns the document's
|
|
44
|
+
shape; Rails owns durability and delivery.
|
|
45
|
+
|
|
46
|
+
A full y-crdt Ruby binding like `y-rb` gives you the whole type system — shared
|
|
47
|
+
text, arrays, maps, XML — to build and query documents in Ruby. yrb-lite leaves
|
|
48
|
+
that out on purpose. What's left is a sync engine plus a one-include ActionCable
|
|
49
|
+
concern, with the server concerns it skips (auth, rate limiting, metrics — see
|
|
50
|
+
above) built from the Rails you already run, and no Node process hosting the
|
|
51
|
+
documents.
|
|
52
|
+
|
|
38
53
|
## Testing
|
|
39
54
|
|
|
40
55
|
Ruby and Rust unit tests cover the core. CI also runs the npm client tests and a
|
|
@@ -86,11 +101,7 @@ require "yrb_lite"
|
|
|
86
101
|
|
|
87
102
|
# Create docs
|
|
88
103
|
doc = YrbLite::Doc.new # random client ID
|
|
89
|
-
doc = YrbLite::Doc.new(12345) # specific client ID
|
|
90
|
-
|
|
91
|
-
# Get document info
|
|
92
|
-
doc.client_id # => unique client identifier
|
|
93
|
-
doc.guid # => document GUID
|
|
104
|
+
doc = YrbLite::Doc.new(12345) # specific client ID (used for CRDT identity)
|
|
94
105
|
|
|
95
106
|
# Encoding
|
|
96
107
|
doc.encode_state_vector # => current state vector
|
|
@@ -100,36 +111,23 @@ doc.encode_state_as_update(sv) # => update diff against state vector
|
|
|
100
111
|
# Applying updates
|
|
101
112
|
doc.apply_update(update_bytes) # apply raw V1 update
|
|
102
113
|
|
|
103
|
-
# Sync protocol
|
|
104
|
-
doc.sync_step1 # => SyncStep1 message (
|
|
105
|
-
doc.
|
|
106
|
-
|
|
107
|
-
doc.encode_update_message(update) # => wrap update as sync Update message
|
|
114
|
+
# Sync protocol
|
|
115
|
+
doc.sync_step1 # => SyncStep1 message (this doc's state vector)
|
|
116
|
+
doc.handle_sync_message(data) # => [msg_type, sync_type, response]; answers a
|
|
117
|
+
# peer's SyncStep1 with a SyncStep2
|
|
108
118
|
```
|
|
109
119
|
|
|
110
|
-
###
|
|
120
|
+
### Protocol codec (module functions)
|
|
111
121
|
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
# Get document info
|
|
118
|
-
awareness.client_id # => unique client identifier
|
|
119
|
-
awareness.guid # => document GUID
|
|
120
|
-
```
|
|
121
|
-
|
|
122
|
-
### Handling Sync Messages
|
|
122
|
+
Classifying and unwrapping wire frames is stateless, so it's exposed as
|
|
123
|
+
`YrbLite` module functions rather than a class. The server never holds presence
|
|
124
|
+
or document state to route a frame — presence lives in the browser clients, and
|
|
125
|
+
the server only relays awareness frames opaquely.
|
|
123
126
|
|
|
124
127
|
```ruby
|
|
125
|
-
#
|
|
126
|
-
|
|
127
|
-
#
|
|
128
|
-
|
|
129
|
-
# When receiving messages from peer
|
|
130
|
-
response = awareness.handle(incoming_data)
|
|
131
|
-
# Send response back to peer if not empty
|
|
132
|
-
send_to_peer(response) unless response.empty?
|
|
128
|
+
YrbLite.message_kind(frame) # => 0 drop / 1 step1 / 2 update / 3 awareness / 4 query
|
|
129
|
+
YrbLite.update_from_message(frame) # => the document delta carried by a frame, or nil
|
|
130
|
+
YrbLite.wrap_update(update_bytes) # => wrap a raw doc update as a sync Update frame
|
|
133
131
|
```
|
|
134
132
|
|
|
135
133
|
### ActionCable Integration
|
|
@@ -147,16 +145,12 @@ class DocumentChannel < ApplicationCable::Channel
|
|
|
147
145
|
on_change { |key, update| MyStore.append(key, update) } # durable record
|
|
148
146
|
|
|
149
147
|
def subscribed
|
|
150
|
-
|
|
148
|
+
sync_subscribed params[:id]
|
|
151
149
|
end
|
|
152
150
|
|
|
153
151
|
def receive(data)
|
|
154
152
|
sync_receive(data, params[:id])
|
|
155
153
|
end
|
|
156
|
-
|
|
157
|
-
def unsubscribed
|
|
158
|
-
sync_unsubscribed(params[:id])
|
|
159
|
-
end
|
|
160
154
|
end
|
|
161
155
|
```
|
|
162
156
|
|
|
@@ -178,6 +172,54 @@ oversized, or unknown frames are dropped. A bad frame can't crash the process: a
|
|
|
178
172
|
Rust panic is caught at the FFI boundary and re-raised as a Ruby exception. And
|
|
179
173
|
no single client can relay garbage that breaks the others in a room.
|
|
180
174
|
|
|
175
|
+
#### Delivery guarantees
|
|
176
|
+
|
|
177
|
+
The contract is the same at every scale — one process, or hundreds across many
|
|
178
|
+
servers:
|
|
179
|
+
|
|
180
|
+
- **The document always converges.** CRDT updates are commutative and
|
|
181
|
+
idempotent, so out-of-order, duplicate, or concurrent delivery all converge to
|
|
182
|
+
the same correct document. This needs no coordination and holds everywhere.
|
|
183
|
+
- **The durable log never goes gappy.** An update is recorded only once its
|
|
184
|
+
causal dependencies are already in the store (checked against `on_load`); a
|
|
185
|
+
causally-incomplete update triggers a resync instead, so the log always
|
|
186
|
+
rebuilds cleanly.
|
|
187
|
+
- **`on_change` is at-least-once, and the durable guarantee is that replaying the
|
|
188
|
+
log reconstructs the document.** Every change is recorded before it's acked or
|
|
189
|
+
broadcast (record-before-distribute). Entry count is not 1:1 with edits: a
|
|
190
|
+
best-effort check skips most lost-ack retries but isn't cross-process exact (a
|
|
191
|
+
retry on another process can record the same update twice), and a resync can
|
|
192
|
+
coalesce a client's un-acked tail into a single record. So **make `on_change`
|
|
193
|
+
idempotent** if duplicate side effects would matter (a webhook, a counter) — a
|
|
194
|
+
raw append-only delta log is naturally fine, since it replays to the same
|
|
195
|
+
document either way.
|
|
196
|
+
- **A raising `on_change` rejects the update implicitly.** If the block raises,
|
|
197
|
+
the update is neither acked nor broadcast (record-before-distribute stops both).
|
|
198
|
+
There is no negative-ack: the client simply never receives the ack, keeps the
|
|
199
|
+
update pending, and retransmits on its timer/reconnect. This is built for
|
|
200
|
+
*transient* failures (the store is briefly down → a retry lands). A block that
|
|
201
|
+
raises *deterministically* — a validation that always fails for this edit —
|
|
202
|
+
will be retried forever, since nothing tells the client to stop. Enforce hard
|
|
203
|
+
rejections before the edit reaches `on_change` (channel authorization in
|
|
204
|
+
`subscribed`), not by raising inside it.
|
|
205
|
+
- **An over-cap frame is dropped the same silent way.** A frame larger than
|
|
206
|
+
`max_frame_bytes` (default 8 MiB) is dropped before decoding — no ack, no
|
|
207
|
+
broadcast — to bound the work a client can force. For a genuine document
|
|
208
|
+
update that means the same implicit rejection as above: unacked, retransmitted
|
|
209
|
+
forever. Normal typing never approaches the cap, but a large paste, an embedded
|
|
210
|
+
image, or a big initial `SyncStep2` can. The drop is logged (`warn` for
|
|
211
|
+
over-cap, `debug` for undecodable) with the document key and update id so it's
|
|
212
|
+
findable; override `sync_log_context` on the channel to add a user/connection
|
|
213
|
+
id. Size the cap for your largest expected payload, and reject
|
|
214
|
+
genuinely-too-big content upstream rather than relying on the cap to reject it
|
|
215
|
+
gracefully.
|
|
216
|
+
|
|
217
|
+
There is deliberately no in-gem cross-process lock. One that only spanned a
|
|
218
|
+
single process would give exactly-once at small scale and silently degrade as
|
|
219
|
+
you scale out, so the guarantee is uniform instead. If you need exactly-once
|
|
220
|
+
*side effects*, enforce it in your store (a unique key on the update) or with
|
|
221
|
+
your own distributed lock — the gem stays storage-agnostic and assumes neither.
|
|
222
|
+
|
|
181
223
|
#### Multi-process deployments
|
|
182
224
|
|
|
183
225
|
Most Rails apps run several processes (Puma workers, multiple dynos), and any of
|
|
@@ -204,9 +246,8 @@ class DocumentChannel < ApplicationCable::Channel
|
|
|
204
246
|
on_load { |key| MyStore.load(key) } # required: source of truth
|
|
205
247
|
on_change { |key, update| MyStore.append(key, update) } # required: record
|
|
206
248
|
|
|
207
|
-
def subscribed =
|
|
249
|
+
def subscribed = sync_subscribed(params[:id])
|
|
208
250
|
def receive(data) = sync_receive(data, params[:id]) # pass the key each call
|
|
209
|
-
def unsubscribed = sync_unsubscribed(params[:id])
|
|
210
251
|
end
|
|
211
252
|
```
|
|
212
253
|
|
|
@@ -218,8 +259,8 @@ end
|
|
|
218
259
|
separate awareness stream with AnyCable `whisper: true`, so cursor traffic can
|
|
219
260
|
take the low-latency client-to-client path without bypassing document
|
|
220
261
|
durability.
|
|
221
|
-
- Pass `params[:id]` into `sync_receive
|
|
222
|
-
|
|
262
|
+
- Pass `params[:id]` into `sync_receive` so the document key survives AnyCable's
|
|
263
|
+
per-command instances.
|
|
223
264
|
- The sender gets its own updates echoed back (no Ruby callback to filter them).
|
|
224
265
|
That's a no-op, since applying an update twice does nothing.
|
|
225
266
|
|
|
@@ -256,9 +297,8 @@ class DocumentChannel < ApplicationCable::Channel
|
|
|
256
297
|
AuditLog.append!(key, update) # raise to REJECT the change
|
|
257
298
|
end
|
|
258
299
|
|
|
259
|
-
def subscribed =
|
|
300
|
+
def subscribed = sync_subscribed(params[:id])
|
|
260
301
|
def receive(data) = sync_receive(data, params[:id])
|
|
261
|
-
def unsubscribed = sync_unsubscribed(params[:id])
|
|
262
302
|
end
|
|
263
303
|
```
|
|
264
304
|
|
|
@@ -295,43 +335,15 @@ one `{ ack: id }` cumulatively confirms everything up to it. Because CRDT apply
|
|
|
295
335
|
is idempotent, a resend that already landed is a harmless no-op that just
|
|
296
336
|
re-acks. Awareness stays ephemeral and is not acked.
|
|
297
337
|
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
# Set local user state (cursor position, name, etc.)
|
|
302
|
-
awareness.set_local_state('{"user": {"name": "Alice", "color": "#ff0000"}}')
|
|
303
|
-
|
|
304
|
-
# Get local state
|
|
305
|
-
awareness.local_state # => '{"user": {"name": "Alice", "color": "#ff0000"}}'
|
|
306
|
-
|
|
307
|
-
# Clear local state (e.g., when disconnecting)
|
|
308
|
-
awareness.clear_local_state
|
|
309
|
-
|
|
310
|
-
# Encode awareness update for broadcasting
|
|
311
|
-
update = awareness.encode_awareness_update
|
|
312
|
-
```
|
|
313
|
-
|
|
314
|
-
### Low-Level Access
|
|
315
|
-
|
|
316
|
-
```ruby
|
|
317
|
-
# Get state vector for manual sync
|
|
318
|
-
sv = awareness.encode_state_vector
|
|
319
|
-
|
|
320
|
-
# Get update diffed against a state vector
|
|
321
|
-
update = awareness.encode_state_as_update(remote_state_vector)
|
|
322
|
-
|
|
323
|
-
# Apply raw update to the document
|
|
324
|
-
awareness.apply_update(update_bytes)
|
|
325
|
-
|
|
326
|
-
# Wrap raw update data in a sync message
|
|
327
|
-
message = awareness.encode_update(update_bytes)
|
|
328
|
-
```
|
|
338
|
+
Presence (cursors, selections) is owned by the browser clients — the server
|
|
339
|
+
never sets or holds presence state, it only relays awareness frames opaquely.
|
|
340
|
+
See `yrb-lite-client` for the client-side awareness API.
|
|
329
341
|
|
|
330
342
|
## Thread Safety
|
|
331
343
|
|
|
332
|
-
`Doc`
|
|
333
|
-
|
|
334
|
-
|
|
344
|
+
A `Doc` is safe to share across Ruby threads — used concurrently from Puma
|
|
345
|
+
workers, ActionCable connection threads, or background jobs without external
|
|
346
|
+
locking.
|
|
335
347
|
|
|
336
348
|
That comes from how the underlying types work, not from locking on top:
|
|
337
349
|
|
|
@@ -379,8 +391,6 @@ exceptions.
|
|
|
379
391
|
```ruby
|
|
380
392
|
YrbLite::MSG_SYNC # 0 - Document sync messages
|
|
381
393
|
YrbLite::MSG_AWARENESS # 1 - User presence data
|
|
382
|
-
YrbLite::MSG_AUTH # 2 - Authentication
|
|
383
|
-
YrbLite::MSG_QUERY_AWARENESS # 3 - Request awareness state
|
|
384
394
|
|
|
385
395
|
YrbLite::MSG_SYNC_STEP1 # 0 - State vector request
|
|
386
396
|
YrbLite::MSG_SYNC_STEP2 # 1 - Update response
|
data/ext/yrb_lite/src/lib.rs
CHANGED
|
@@ -1,16 +1,13 @@
|
|
|
1
1
|
use magnus::{
|
|
2
2
|
function, method, prelude::*, Error, ExceptionClass, RString, Ruby, TryConvert, Value,
|
|
3
3
|
};
|
|
4
|
-
use
|
|
5
|
-
use yrs::sync::{Awareness, DefaultProtocol, Message, Protocol, SyncMessage};
|
|
4
|
+
use yrs::sync::{Message, SyncMessage};
|
|
6
5
|
use yrs::updates::decoder::Decode;
|
|
7
|
-
use yrs::updates::encoder::
|
|
6
|
+
use yrs::updates::encoder::Encode;
|
|
8
7
|
use yrs::{Doc, ReadTxn, Transact};
|
|
9
8
|
|
|
10
9
|
mod protocol;
|
|
11
|
-
use protocol::{
|
|
12
|
-
classify_message, doc_has_pending, merged_doc_update, update_advances_doc, update_is_ready,
|
|
13
|
-
};
|
|
10
|
+
use protocol::{classify_message, merged_doc_update, update_advances_doc, update_is_ready};
|
|
14
11
|
|
|
15
12
|
/// Wrapper around yrs Doc.
|
|
16
13
|
///
|
|
@@ -22,37 +19,13 @@ use protocol::{
|
|
|
22
19
|
#[magnus::wrap(class = "YrbLite::Doc", free_immediately, size)]
|
|
23
20
|
struct RbDoc(Doc);
|
|
24
21
|
|
|
25
|
-
///
|
|
26
|
-
///
|
|
27
|
-
///
|
|
28
|
-
/// its mutating methods (`handle`, `set_local_state`, `clean_local_state`,
|
|
29
|
-
/// `remove_state`, `update_with_clients`) take `&mut self`. It is `Send` but no
|
|
30
|
-
/// longer `Sync`, so we serialize all access through a `Mutex`.
|
|
31
|
-
///
|
|
32
|
-
/// CRITICAL: the `Mutex` is ALWAYS locked inside the `nogvl` closure (never with
|
|
33
|
-
/// the GVL held) and the guard is dropped before the closure returns. This obeys
|
|
34
|
-
/// the same rule as the doc's RwLock (see `nogvl`): a thread never waits on this
|
|
35
|
-
/// lock while holding the GVL, and never reacquires the GVL while holding this
|
|
36
|
-
/// lock, so the GVL and this `Mutex` can't deadlock on lock order. Locking with
|
|
37
|
-
/// the GVL held (outside `nogvl`) reintroduces that deadlock -- don't.
|
|
38
|
-
///
|
|
39
|
-
/// For doc-only reads we clone the (Arc-backed) `Doc` out under the brief lock
|
|
40
|
-
/// and operate on the owned clone, so a long encode holds only the doc's own
|
|
41
|
-
/// RwLock, not this `Mutex`, and never blocks presence updates on another
|
|
42
|
-
/// thread. Lock order is always Mutex-then-doc-RwLock (or doc-RwLock alone),
|
|
43
|
-
/// never the reverse.
|
|
44
|
-
#[magnus::wrap(class = "YrbLite::Awareness", free_immediately, size)]
|
|
45
|
-
struct RbAwareness(Mutex<Awareness>);
|
|
46
|
-
|
|
47
|
-
/// Compile-time proof that the wrapped types are thread-safe. If a future
|
|
48
|
-
/// yrs upgrade makes Doc lose Send/Sync, or Awareness lose Send, this fails the
|
|
49
|
-
/// build instead of silently shipping a thread-unsafe gem. (Awareness is no
|
|
50
|
-
/// longer `Sync` as of yrs 0.27, hence the `Mutex` wrapper, which restores it.)
|
|
22
|
+
/// Compile-time proof that the wrapped Doc is thread-safe. If a future yrs
|
|
23
|
+
/// upgrade makes Doc lose Send/Sync, this fails the build instead of silently
|
|
24
|
+
/// shipping a thread-unsafe gem.
|
|
51
25
|
#[allow(dead_code)]
|
|
52
26
|
fn assert_thread_safe() {
|
|
53
27
|
fn is_send_sync<T: Send + Sync>() {}
|
|
54
28
|
is_send_sync::<Doc>();
|
|
55
|
-
is_send_sync::<Mutex<Awareness>>();
|
|
56
29
|
}
|
|
57
30
|
|
|
58
31
|
/// Run `f` with the GVL (Global VM Lock) released, so other Ruby threads,
|
|
@@ -62,14 +35,17 @@ fn assert_thread_safe() {
|
|
|
62
35
|
/// - It must not touch any Ruby object or call any Ruby API. Inputs are copied
|
|
63
36
|
/// out of Ruby strings before entering, and results are converted to Ruby
|
|
64
37
|
/// objects after returning.
|
|
65
|
-
/// - It must be `Send` (it runs while other threads own the GVL). `&Doc`
|
|
66
|
-
///
|
|
67
|
-
/// -
|
|
68
|
-
///
|
|
69
|
-
///
|
|
70
|
-
///
|
|
71
|
-
///
|
|
72
|
-
///
|
|
38
|
+
/// - It must be `Send` (it runs while other threads own the GVL). `&Doc` is
|
|
39
|
+
/// fine: it's `Sync` (asserted above).
|
|
40
|
+
/// - Lock discipline: any native lock it takes (the doc's internal RwLock) must
|
|
41
|
+
/// be acquired and released inside this closure, with the GVL already dropped.
|
|
42
|
+
/// Never lock with the GVL held (e.g. before calling `nogvl`), or a thread
|
|
43
|
+
/// waiting on the lock while holding the GVL can deadlock against the GVL
|
|
44
|
+
/// reacquire. Same reason we never hold a lock across the GVL boundary.
|
|
45
|
+
///
|
|
46
|
+
/// The closure runs with no unblock function, so it is not interruptible: a
|
|
47
|
+
/// Thread#kill, timeout, or signal can't preempt it mid-run. That's fine for the
|
|
48
|
+
/// bounded CRDT work it does; never call anything blocking or unbounded inside it.
|
|
73
49
|
///
|
|
74
50
|
/// Panics inside the closure are caught and re-raised (resumed) after the GVL
|
|
75
51
|
/// is reacquired, where magnus converts them to Ruby exceptions.
|
|
@@ -140,11 +116,6 @@ fn yrb_error(msg: String) -> Error {
|
|
|
140
116
|
Error::new(class, msg)
|
|
141
117
|
}
|
|
142
118
|
|
|
143
|
-
// CLIENT IDs ARE NOT VALIDATED -- whoever supplies the id (the app via
|
|
144
|
-
// `Doc.new(id)` / `Awareness.new(id)`, or a remote peer over the wire) is
|
|
145
|
-
// responsible for keeping it JS-safe (<= 2^53 - 1). See the protocol.rs header
|
|
146
|
-
// for why (and `ClientID::try_new`, proposed upstream, for strict rejection).
|
|
147
|
-
|
|
148
119
|
// ============================================================================
|
|
149
120
|
// Doc Implementation
|
|
150
121
|
// ============================================================================
|
|
@@ -161,14 +132,6 @@ impl RbDoc {
|
|
|
161
132
|
Ok(RbDoc(doc))
|
|
162
133
|
}
|
|
163
134
|
|
|
164
|
-
fn client_id(&self) -> u64 {
|
|
165
|
-
self.0.client_id().get()
|
|
166
|
-
}
|
|
167
|
-
|
|
168
|
-
fn guid(&self) -> String {
|
|
169
|
-
self.0.guid().to_string()
|
|
170
|
-
}
|
|
171
|
-
|
|
172
135
|
fn encode_state_vector(&self) -> RString {
|
|
173
136
|
let doc = &self.0;
|
|
174
137
|
let sv = nogvl(move || {
|
|
@@ -211,7 +174,7 @@ impl RbDoc {
|
|
|
211
174
|
}
|
|
212
175
|
|
|
213
176
|
/// True if applying `update` would integrate cleanly (its dependencies are
|
|
214
|
-
/// all present). False means it would leave a pending struct
|
|
177
|
+
/// all present). False means it would leave a pending struct, i.e. an earlier
|
|
215
178
|
/// update is missing. Pure read; does not mutate.
|
|
216
179
|
fn update_ready(&self, update: RString) -> Result<bool, Error> {
|
|
217
180
|
let update_bytes = copy_bytes(update);
|
|
@@ -228,13 +191,6 @@ impl RbDoc {
|
|
|
228
191
|
nogvl(move || update_advances_doc(doc, &update_bytes)).map_err(yrb_error)
|
|
229
192
|
}
|
|
230
193
|
|
|
231
|
-
/// True if the document holds pending (un-integrable) structs waiting on a
|
|
232
|
-
/// missing dependency.
|
|
233
|
-
fn pending(&self) -> bool {
|
|
234
|
-
let doc = &self.0;
|
|
235
|
-
nogvl(move || doc_has_pending(doc))
|
|
236
|
-
}
|
|
237
|
-
|
|
238
194
|
/// Sync step 1: Create a sync message with our state vector
|
|
239
195
|
fn sync_step1(&self) -> RString {
|
|
240
196
|
let doc = &self.0;
|
|
@@ -246,23 +202,10 @@ impl RbDoc {
|
|
|
246
202
|
binary_string(&encoded)
|
|
247
203
|
}
|
|
248
204
|
|
|
249
|
-
///
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
let encoded = nogvl(move || -> Result<Vec<u8>, String> {
|
|
254
|
-
let sv = yrs::StateVector::decode_v1(&sv_data).map_err(|e| e.to_string())?;
|
|
255
|
-
let txn = doc.transact();
|
|
256
|
-
let update = txn.encode_state_as_update_v1(&sv);
|
|
257
|
-
Ok(Message::Sync(SyncMessage::SyncStep2(update)).encode_v1())
|
|
258
|
-
})
|
|
259
|
-
.map_err(yrb_error)?;
|
|
260
|
-
Ok(binary_string(&encoded))
|
|
261
|
-
}
|
|
262
|
-
|
|
263
|
-
/// Handle a sync message and return response (if any)
|
|
264
|
-
/// Returns [message_type, sync_type, response_bytes] or nil
|
|
265
|
-
fn handle_sync_message(&self, data: RString) -> Result<Option<(u8, u8, RString)>, Error> {
|
|
205
|
+
/// Handle a Sync or Awareness message, returning
|
|
206
|
+
/// [message_type, sync_type, response_bytes]. Only Sync (step1/step2/update)
|
|
207
|
+
/// and Awareness are handled; any other frame type is rejected.
|
|
208
|
+
fn handle_sync_message(&self, data: RString) -> Result<(u8, u8, RString), Error> {
|
|
266
209
|
let data_bytes = copy_bytes(data);
|
|
267
210
|
let doc = &self.0;
|
|
268
211
|
|
|
@@ -297,264 +240,55 @@ impl RbDoc {
|
|
|
297
240
|
}
|
|
298
241
|
},
|
|
299
242
|
Message::Awareness(_) => Ok((1, 0, Vec::new())),
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
243
|
+
// Auth, awareness-query, and custom frames aren't part of this
|
|
244
|
+
// protocol; reject rather than pretend to handle them.
|
|
245
|
+
_ => Err("unsupported message type".to_string()),
|
|
303
246
|
}
|
|
304
247
|
})
|
|
305
248
|
.map_err(yrb_error)?;
|
|
306
249
|
|
|
307
|
-
Ok(
|
|
308
|
-
}
|
|
309
|
-
|
|
310
|
-
/// Encode raw update bytes as a sync Update message
|
|
311
|
-
fn encode_update_message(&self, update: RString) -> RString {
|
|
312
|
-
let update_bytes = copy_bytes(update);
|
|
313
|
-
let msg = Message::Sync(SyncMessage::Update(update_bytes));
|
|
314
|
-
binary_string(&msg.encode_v1())
|
|
250
|
+
Ok((msg_type, sync_type, binary_string(&response)))
|
|
315
251
|
}
|
|
316
252
|
}
|
|
317
253
|
|
|
318
254
|
// ============================================================================
|
|
319
|
-
//
|
|
255
|
+
// Protocol codec (stateless), exposed as `YrbLite` module functions
|
|
320
256
|
// ============================================================================
|
|
257
|
+
//
|
|
258
|
+
// The server never holds presence or document state to classify a frame; these
|
|
259
|
+
// are pure functions of their bytes. (Presence lives in the browser clients; the
|
|
260
|
+
// server only relays awareness frames opaquely.)
|
|
261
|
+
|
|
262
|
+
/// Wrap a raw document update in a sync Update message frame, ready to relay.
|
|
263
|
+
fn wrap_update(update: RString) -> RString {
|
|
264
|
+
let update_bytes = copy_bytes(update);
|
|
265
|
+
let msg = Message::Sync(SyncMessage::Update(update_bytes));
|
|
266
|
+
binary_string(&msg.encode_v1())
|
|
267
|
+
}
|
|
321
268
|
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
|
|
334
|
-
|
|
335
|
-
|
|
336
|
-
nogvl(move || awareness.lock().unwrap().doc().client_id().get())
|
|
337
|
-
}
|
|
338
|
-
|
|
339
|
-
fn guid(&self) -> String {
|
|
340
|
-
let awareness = &self.0;
|
|
341
|
-
nogvl(move || awareness.lock().unwrap().doc().guid().to_string())
|
|
342
|
-
}
|
|
343
|
-
|
|
344
|
-
/// A standalone SyncStep1 message (the server's state vector). Sent as its
|
|
345
|
-
/// own frame in the opening handshake so providers that parse one message
|
|
346
|
-
/// per frame (e.g. @y-rb/actioncable) handle it correctly.
|
|
347
|
-
fn sync_step1(&self) -> RString {
|
|
348
|
-
let awareness = &self.0;
|
|
349
|
-
let encoded = nogvl(move || {
|
|
350
|
-
let doc = awareness.lock().unwrap().doc().clone();
|
|
351
|
-
let txn = doc.transact();
|
|
352
|
-
let sv = txn.state_vector();
|
|
353
|
-
Message::Sync(SyncMessage::SyncStep1(sv)).encode_v1()
|
|
354
|
-
});
|
|
355
|
-
binary_string(&encoded)
|
|
356
|
-
}
|
|
357
|
-
|
|
358
|
-
/// Create initial sync messages to send when connection opens.
|
|
359
|
-
/// Returns binary data containing SyncStep1 + Awareness update.
|
|
360
|
-
fn start(&self) -> Result<RString, Error> {
|
|
361
|
-
let awareness = &self.0;
|
|
362
|
-
let encoded = nogvl(move || -> Result<Vec<u8>, String> {
|
|
363
|
-
let awareness = awareness.lock().unwrap();
|
|
364
|
-
let protocol = DefaultProtocol;
|
|
365
|
-
let mut encoder = EncoderV1::new();
|
|
366
|
-
protocol
|
|
367
|
-
.start(&awareness, &mut encoder)
|
|
368
|
-
.map_err(|e| e.to_string())?;
|
|
369
|
-
Ok(encoder.to_vec())
|
|
370
|
-
})
|
|
371
|
-
.map_err(yrb_error)?;
|
|
372
|
-
Ok(binary_string(&encoded))
|
|
373
|
-
}
|
|
374
|
-
|
|
375
|
-
/// Handle incoming message and return response messages (if any).
|
|
376
|
-
/// Returns binary data containing response messages, or empty if no response needed.
|
|
377
|
-
fn handle(&self, data: RString) -> Result<RString, Error> {
|
|
378
|
-
let data_bytes = copy_bytes(data);
|
|
379
|
-
let awareness = &self.0;
|
|
380
|
-
|
|
381
|
-
let encoded = nogvl(move || -> Result<Vec<u8>, String> {
|
|
382
|
-
let mut awareness = awareness.lock().unwrap();
|
|
383
|
-
let protocol = DefaultProtocol;
|
|
384
|
-
let responses = protocol
|
|
385
|
-
.handle(&mut awareness, &data_bytes)
|
|
386
|
-
.map_err(|e| e.to_string())?;
|
|
387
|
-
|
|
388
|
-
if responses.is_empty() {
|
|
389
|
-
return Ok(Vec::new());
|
|
390
|
-
}
|
|
391
|
-
|
|
392
|
-
let mut encoder = EncoderV1::new();
|
|
393
|
-
for msg in responses {
|
|
394
|
-
msg.encode(&mut encoder);
|
|
395
|
-
}
|
|
396
|
-
Ok(encoder.to_vec())
|
|
397
|
-
})
|
|
398
|
-
.map_err(yrb_error)?;
|
|
399
|
-
Ok(binary_string(&encoded))
|
|
400
|
-
}
|
|
401
|
-
|
|
402
|
-
/// Encode an update message for broadcasting changes to peers.
|
|
403
|
-
fn encode_update(&self, update: RString) -> RString {
|
|
404
|
-
let update_bytes = copy_bytes(update);
|
|
405
|
-
let msg = Message::Sync(SyncMessage::Update(update_bytes));
|
|
406
|
-
binary_string(&msg.encode_v1())
|
|
407
|
-
}
|
|
408
|
-
|
|
409
|
-
fn encode_state_vector(&self) -> RString {
|
|
410
|
-
let awareness = &self.0;
|
|
411
|
-
let sv = nogvl(move || {
|
|
412
|
-
let doc = awareness.lock().unwrap().doc().clone();
|
|
413
|
-
let txn = doc.transact();
|
|
414
|
-
txn.state_vector().encode_v1()
|
|
415
|
-
});
|
|
416
|
-
binary_string(&sv)
|
|
417
|
-
}
|
|
418
|
-
|
|
419
|
-
/// Encode state as update (optionally diffed against a state vector)
|
|
420
|
-
fn encode_state_as_update(&self, args: &[Value]) -> Result<RString, Error> {
|
|
421
|
-
let sv_bytes: Option<Vec<u8>> = if args.is_empty() {
|
|
422
|
-
None
|
|
423
|
-
} else {
|
|
424
|
-
let sv_string: RString = TryConvert::try_convert(args[0])?;
|
|
425
|
-
Some(copy_bytes(sv_string))
|
|
426
|
-
};
|
|
427
|
-
let awareness = &self.0;
|
|
428
|
-
let update = nogvl(move || -> Result<Vec<u8>, String> {
|
|
429
|
-
let sv = match &sv_bytes {
|
|
430
|
-
None => yrs::StateVector::default(),
|
|
431
|
-
Some(bytes) => yrs::StateVector::decode_v1(bytes).map_err(|e| e.to_string())?,
|
|
432
|
-
};
|
|
433
|
-
let doc = awareness.lock().unwrap().doc().clone();
|
|
434
|
-
let txn = doc.transact();
|
|
435
|
-
Ok(txn.encode_state_as_update_v1(&sv))
|
|
436
|
-
})
|
|
437
|
-
.map_err(yrb_error)?;
|
|
438
|
-
Ok(binary_string(&update))
|
|
439
|
-
}
|
|
440
|
-
|
|
441
|
-
/// Set local awareness state (JSON string)
|
|
442
|
-
fn set_local_state(&self, json: String) -> Result<(), Error> {
|
|
443
|
-
let value: serde_json::Value =
|
|
444
|
-
serde_json::from_str(&json).map_err(|e| yrb_error(e.to_string()))?;
|
|
445
|
-
let awareness = &self.0;
|
|
446
|
-
nogvl(move || -> Result<(), String> {
|
|
447
|
-
awareness
|
|
448
|
-
.lock()
|
|
449
|
-
.unwrap()
|
|
450
|
-
.set_local_state(value)
|
|
451
|
-
.map_err(|e| e.to_string())
|
|
452
|
-
})
|
|
453
|
-
.map_err(yrb_error)
|
|
454
|
-
}
|
|
455
|
-
|
|
456
|
-
/// Get local awareness state as JSON string (or nil if not set)
|
|
457
|
-
fn local_state(&self) -> Option<String> {
|
|
458
|
-
let awareness = &self.0;
|
|
459
|
-
nogvl(move || {
|
|
460
|
-
awareness
|
|
461
|
-
.lock()
|
|
462
|
-
.unwrap()
|
|
463
|
-
.local_state::<serde_json::Value>()
|
|
464
|
-
.map(|v| v.to_string())
|
|
465
|
-
})
|
|
466
|
-
}
|
|
467
|
-
|
|
468
|
-
/// Clear local awareness state
|
|
469
|
-
fn clear_local_state(&self) {
|
|
470
|
-
let awareness = &self.0;
|
|
471
|
-
nogvl(move || awareness.lock().unwrap().clean_local_state());
|
|
472
|
-
}
|
|
473
|
-
|
|
474
|
-
/// Get awareness update for broadcasting to peers
|
|
475
|
-
fn encode_awareness_update(&self) -> Result<RString, Error> {
|
|
476
|
-
let awareness = &self.0;
|
|
477
|
-
let encoded = nogvl(move || -> Result<Vec<u8>, String> {
|
|
478
|
-
let awareness = awareness.lock().unwrap();
|
|
479
|
-
let update = awareness.update().map_err(|e| e.to_string())?;
|
|
480
|
-
Ok(Message::Awareness(update).encode_v1())
|
|
481
|
-
})
|
|
482
|
-
.map_err(yrb_error)?;
|
|
483
|
-
Ok(binary_string(&encoded))
|
|
484
|
-
}
|
|
485
|
-
|
|
486
|
-
fn apply_update(&self, update: RString) -> Result<(), Error> {
|
|
487
|
-
let update_bytes = copy_bytes(update);
|
|
488
|
-
let awareness = &self.0;
|
|
489
|
-
nogvl(move || -> Result<(), String> {
|
|
490
|
-
let update = yrs::Update::decode_v1(&update_bytes).map_err(|e| e.to_string())?;
|
|
491
|
-
let doc = awareness.lock().unwrap().doc().clone();
|
|
492
|
-
let mut txn = doc.transact_mut();
|
|
493
|
-
txn.apply_update(update).map_err(|e| e.to_string())
|
|
494
|
-
})
|
|
495
|
-
.map_err(yrb_error)
|
|
496
|
-
}
|
|
497
|
-
|
|
498
|
-
/// True if applying `update` would integrate cleanly (its dependencies are
|
|
499
|
-
/// all present). False means it depends on a missing, causally-prior update.
|
|
500
|
-
/// Pure read; does not mutate.
|
|
501
|
-
fn update_ready(&self, update: RString) -> Result<bool, Error> {
|
|
502
|
-
let update_bytes = copy_bytes(update);
|
|
503
|
-
let awareness = &self.0;
|
|
504
|
-
nogvl(move || {
|
|
505
|
-
let doc = awareness.lock().unwrap().doc().clone();
|
|
506
|
-
update_is_ready(&doc, &update_bytes)
|
|
507
|
-
})
|
|
508
|
-
.map_err(yrb_error)
|
|
509
|
-
}
|
|
510
|
-
|
|
511
|
-
/// True if applying `update` would change the document, false if it's an
|
|
512
|
-
/// already-applied retry. See `update_advances_doc`. Pure read.
|
|
513
|
-
fn update_advances(&self, update: RString) -> Result<bool, Error> {
|
|
514
|
-
let update_bytes = copy_bytes(update);
|
|
515
|
-
let awareness = &self.0;
|
|
516
|
-
nogvl(move || {
|
|
517
|
-
let doc = awareness.lock().unwrap().doc().clone();
|
|
518
|
-
update_advances_doc(&doc, &update_bytes)
|
|
519
|
-
})
|
|
520
|
-
.map_err(yrb_error)
|
|
521
|
-
}
|
|
522
|
-
|
|
523
|
-
/// True if the document holds pending (un-integrable) structs waiting on a
|
|
524
|
-
/// missing dependency.
|
|
525
|
-
fn pending(&self) -> bool {
|
|
526
|
-
let awareness = &self.0;
|
|
527
|
-
nogvl(move || {
|
|
528
|
-
let doc = awareness.lock().unwrap().doc().clone();
|
|
529
|
-
doc_has_pending(&doc)
|
|
530
|
-
})
|
|
531
|
-
}
|
|
532
|
-
|
|
533
|
-
/// Classify a frame for safe routing and relay. Returns a code only when
|
|
534
|
-
/// the frame is exactly one well-formed message that consumes the whole
|
|
535
|
-
/// buffer, so a malformed, truncated, multi-message, or trailing-garbage
|
|
536
|
-
/// frame (which a malicious client could craft to disrupt others if
|
|
537
|
-
/// relayed) is rejected up front:
|
|
538
|
-
/// 0 = drop (malformed, multiple, unknown, or empty)
|
|
539
|
-
/// 1 = sync step1 (a request: respond, do not relay)
|
|
540
|
-
/// 2 = sync step2/update (a document change: record/apply/relay)
|
|
541
|
-
/// 3 = awareness (presence: relay)
|
|
542
|
-
/// 4 = awareness query (a request: respond, do not relay)
|
|
543
|
-
fn message_kind(&self, data: RString) -> u8 {
|
|
544
|
-
let data_bytes = copy_bytes(data);
|
|
545
|
-
nogvl(move || classify_message(&data_bytes))
|
|
546
|
-
}
|
|
269
|
+
/// Classify a frame for safe routing and relay. Returns a code only when the
|
|
270
|
+
/// frame is exactly one well-formed message that consumes the whole buffer, so
|
|
271
|
+
/// a malformed, truncated, multi-message, or trailing-garbage frame (which a
|
|
272
|
+
/// malicious client could craft to disrupt others if relayed) is rejected up
|
|
273
|
+
/// front:
|
|
274
|
+
/// 0 = drop (malformed, multiple, unknown, or empty)
|
|
275
|
+
/// 1 = sync step1 (a request: respond, do not relay)
|
|
276
|
+
/// 2 = sync step2/update (a document change: record/apply/relay)
|
|
277
|
+
/// 3 = awareness (presence: relay)
|
|
278
|
+
/// 4 = awareness query (a request: respond, do not relay)
|
|
279
|
+
fn message_kind(data: RString) -> u8 {
|
|
280
|
+
let data_bytes = copy_bytes(data);
|
|
281
|
+
nogvl(move || classify_message(&data_bytes))
|
|
282
|
+
}
|
|
547
283
|
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
|
|
551
|
-
|
|
552
|
-
|
|
553
|
-
|
|
554
|
-
|
|
555
|
-
|
|
556
|
-
Ok(merged.map(|b| binary_string(&b)))
|
|
557
|
-
}
|
|
284
|
+
/// Extract the document-update delta carried by a protocol message: the payloads
|
|
285
|
+
/// of any Update or SyncStep2 sub-messages, merged into a single update. Returns
|
|
286
|
+
/// nil if the message carries no document change (a SyncStep1 request or an
|
|
287
|
+
/// awareness update). The store-backed path records this exact delta before relay.
|
|
288
|
+
fn update_from_message(data: RString) -> Result<Option<RString>, Error> {
|
|
289
|
+
let data_bytes = copy_bytes(data);
|
|
290
|
+
let merged = nogvl(move || merged_doc_update(&data_bytes)).map_err(yrb_error)?;
|
|
291
|
+
Ok(merged.map(|b| binary_string(&b)))
|
|
558
292
|
}
|
|
559
293
|
|
|
560
294
|
// ============================================================================
|
|
@@ -572,8 +306,6 @@ fn init(ruby: &Ruby) -> Result<(), Error> {
|
|
|
572
306
|
// Define Doc class
|
|
573
307
|
let doc_class = module.define_class("Doc", ruby.class_object())?;
|
|
574
308
|
doc_class.define_singleton_method("new", function!(RbDoc::new, -1))?;
|
|
575
|
-
doc_class.define_method("client_id", method!(RbDoc::client_id, 0))?;
|
|
576
|
-
doc_class.define_method("guid", method!(RbDoc::guid, 0))?;
|
|
577
309
|
doc_class.define_method(
|
|
578
310
|
"encode_state_vector",
|
|
579
311
|
method!(RbDoc::encode_state_vector, 0),
|
|
@@ -585,60 +317,19 @@ fn init(ruby: &Ruby) -> Result<(), Error> {
|
|
|
585
317
|
doc_class.define_method("apply_update", method!(RbDoc::apply_update, 1))?;
|
|
586
318
|
doc_class.define_method("update_ready?", method!(RbDoc::update_ready, 1))?;
|
|
587
319
|
doc_class.define_method("update_advances?", method!(RbDoc::update_advances, 1))?;
|
|
588
|
-
doc_class.define_method("pending?", method!(RbDoc::pending, 0))?;
|
|
589
320
|
doc_class.define_method("sync_step1", method!(RbDoc::sync_step1, 0))?;
|
|
590
|
-
doc_class.define_method("sync_step2", method!(RbDoc::sync_step2, 1))?;
|
|
591
321
|
doc_class.define_method(
|
|
592
322
|
"handle_sync_message",
|
|
593
323
|
method!(RbDoc::handle_sync_message, 1),
|
|
594
324
|
)?;
|
|
595
|
-
|
|
596
|
-
|
|
597
|
-
|
|
598
|
-
)?;
|
|
599
|
-
|
|
600
|
-
// Define Awareness class
|
|
601
|
-
let awareness_class = module.define_class("Awareness", ruby.class_object())?;
|
|
602
|
-
awareness_class.define_singleton_method("new", function!(RbAwareness::new, -1))?;
|
|
603
|
-
awareness_class.define_method("client_id", method!(RbAwareness::client_id, 0))?;
|
|
604
|
-
awareness_class.define_method("guid", method!(RbAwareness::guid, 0))?;
|
|
605
|
-
awareness_class.define_method("start", method!(RbAwareness::start, 0))?;
|
|
606
|
-
awareness_class.define_method("sync_step1", method!(RbAwareness::sync_step1, 0))?;
|
|
607
|
-
awareness_class.define_method("handle", method!(RbAwareness::handle, 1))?;
|
|
608
|
-
awareness_class.define_method("encode_update", method!(RbAwareness::encode_update, 1))?;
|
|
609
|
-
awareness_class.define_method(
|
|
610
|
-
"encode_state_vector",
|
|
611
|
-
method!(RbAwareness::encode_state_vector, 0),
|
|
612
|
-
)?;
|
|
613
|
-
awareness_class.define_method(
|
|
614
|
-
"encode_state_as_update",
|
|
615
|
-
method!(RbAwareness::encode_state_as_update, -1),
|
|
616
|
-
)?;
|
|
617
|
-
awareness_class.define_method("apply_update", method!(RbAwareness::apply_update, 1))?;
|
|
618
|
-
awareness_class.define_method("update_ready?", method!(RbAwareness::update_ready, 1))?;
|
|
619
|
-
awareness_class.define_method("update_advances?", method!(RbAwareness::update_advances, 1))?;
|
|
620
|
-
awareness_class.define_method("pending?", method!(RbAwareness::pending, 0))?;
|
|
621
|
-
awareness_class.define_method("set_local_state", method!(RbAwareness::set_local_state, 1))?;
|
|
622
|
-
awareness_class.define_method("local_state", method!(RbAwareness::local_state, 0))?;
|
|
623
|
-
awareness_class.define_method(
|
|
624
|
-
"clear_local_state",
|
|
625
|
-
method!(RbAwareness::clear_local_state, 0),
|
|
626
|
-
)?;
|
|
627
|
-
awareness_class.define_method(
|
|
628
|
-
"encode_awareness_update",
|
|
629
|
-
method!(RbAwareness::encode_awareness_update, 0),
|
|
630
|
-
)?;
|
|
631
|
-
awareness_class.define_method(
|
|
632
|
-
"update_from_message",
|
|
633
|
-
method!(RbAwareness::update_from_message, 1),
|
|
634
|
-
)?;
|
|
635
|
-
awareness_class.define_method("message_kind", method!(RbAwareness::message_kind, 1))?;
|
|
325
|
+
// Stateless protocol codec, as YrbLite module functions.
|
|
326
|
+
module.define_module_function("wrap_update", function!(wrap_update, 1))?;
|
|
327
|
+
module.define_module_function("message_kind", function!(message_kind, 1))?;
|
|
328
|
+
module.define_module_function("update_from_message", function!(update_from_message, 1))?;
|
|
636
329
|
|
|
637
330
|
// Define message type constants
|
|
638
331
|
module.const_set("MSG_SYNC", 0u8)?;
|
|
639
332
|
module.const_set("MSG_AWARENESS", 1u8)?;
|
|
640
|
-
module.const_set("MSG_AUTH", 2u8)?;
|
|
641
|
-
module.const_set("MSG_QUERY_AWARENESS", 3u8)?;
|
|
642
333
|
module.const_set("MSG_SYNC_STEP1", 0u8)?;
|
|
643
334
|
module.const_set("MSG_SYNC_STEP2", 1u8)?;
|
|
644
335
|
module.const_set("MSG_SYNC_UPDATE", 2u8)?;
|
|
@@ -1,9 +1,9 @@
|
|
|
1
|
-
// Pure
|
|
1
|
+
// Pure Rust protocol helpers (no Ruby interop).
|
|
2
2
|
//
|
|
3
|
-
//
|
|
4
|
-
//
|
|
5
|
-
//
|
|
6
|
-
//
|
|
3
|
+
// Client ids are not validated here, on purpose: every legitimate peer (browser
|
|
4
|
+
// Yjs, and yrs's own `ClientID::random`) already emits 53-bit ids, so it's the
|
|
5
|
+
// client's responsibility not to send a bad one, and we don't want to own that
|
|
6
|
+
// logic.
|
|
7
7
|
use yrs::encoding::read::{Cursor, Read};
|
|
8
8
|
use yrs::sync::protocol::MessageReader;
|
|
9
9
|
use yrs::sync::{Message, SyncMessage};
|
|
@@ -11,7 +11,7 @@ use yrs::updates::decoder::{Decode, DecoderV1};
|
|
|
11
11
|
use yrs::{Doc, ReadTxn, Transact};
|
|
12
12
|
|
|
13
13
|
/// Classify a frame: a non-zero code only for exactly one well-formed message
|
|
14
|
-
/// that consumes the whole buffer (
|
|
14
|
+
/// that consumes the whole buffer (the codes are the match arms below).
|
|
15
15
|
pub(crate) fn classify_message(bytes: &[u8]) -> u8 {
|
|
16
16
|
let mut decoder = DecoderV1::new(Cursor::new(bytes));
|
|
17
17
|
let msg = match Message::decode(&mut decoder) {
|
|
@@ -53,7 +53,7 @@ pub(crate) fn merged_doc_update(bytes: &[u8]) -> Result<Option<Vec<u8>>, String>
|
|
|
53
53
|
let update = yrs::Update::decode_v1(&merged).map_err(|e| e.to_string())?;
|
|
54
54
|
// A genuine no-op (e.g. the empty SyncStep2 in an opening handshake) carries
|
|
55
55
|
// no structs, no deletes, and no dependencies. We must NOT treat a causally-
|
|
56
|
-
// pending update as a no-op:
|
|
56
|
+
// pending update as a no-op: such an update reports an empty
|
|
57
57
|
// state_vector (its structs can't integrate yet), but it still carries
|
|
58
58
|
// content and a non-empty lower bound (the deps it's waiting on). Dropping it
|
|
59
59
|
// here would silently swallow a gappy update instead of rejecting + resyncing.
|
|
@@ -69,29 +69,29 @@ pub(crate) fn merged_doc_update(bytes: &[u8]) -> Result<Option<Vec<u8>>, String>
|
|
|
69
69
|
/// True if applying `update_bytes` to `doc` would integrate cleanly: every
|
|
70
70
|
/// dependency the update references is already present (the doc's state vector
|
|
71
71
|
/// covers the update's lower bound). A pure read; does not mutate the doc.
|
|
72
|
-
/// When false, applying it would park a pending struct
|
|
72
|
+
/// When false, applying it would park a pending struct, the signal that an
|
|
73
73
|
/// earlier, causally-prior update is missing.
|
|
74
74
|
pub(crate) fn update_is_ready(doc: &Doc, update_bytes: &[u8]) -> Result<bool, String> {
|
|
75
75
|
let update = yrs::Update::decode_v1(update_bytes).map_err(|e| e.to_string())?;
|
|
76
76
|
Ok(doc.transact().state_vector() >= update.state_vector_lower())
|
|
77
77
|
}
|
|
78
78
|
|
|
79
|
-
/// True if applying `update_bytes` would actually change `doc
|
|
80
|
-
/// content the doc doesn't already have.
|
|
79
|
+
/// True if applying `update_bytes` would actually change `doc`, i.e. it carries
|
|
80
|
+
/// content the doc doesn't already have. This lets the server make durable side
|
|
81
81
|
/// effects exactly-once: a lost-ack retry re-sends an update the server already
|
|
82
82
|
/// applied; that retry is causally ready (so `update_is_ready` is true) but must
|
|
83
|
-
///
|
|
83
|
+
/// not re-run `on_change`.
|
|
84
84
|
///
|
|
85
85
|
/// We can't read the update's own state vector to decide this: yrs reports an
|
|
86
|
-
///
|
|
86
|
+
/// empty state_vector() for a causally-pending diff (e.g. a resync delta whose
|
|
87
87
|
/// structs depend on updates the doc has but the standalone update doesn't),
|
|
88
88
|
/// which would look identical to a no-op. So measure the real effect: seed an
|
|
89
89
|
/// independent probe with the doc's current state, apply the update there, and
|
|
90
90
|
/// see whether the state vector grew. Deletes don't move the state vector, so we
|
|
91
|
-
/// can't cheaply prove a delete-bearing update is a duplicate
|
|
92
|
-
///
|
|
93
|
-
///
|
|
94
|
-
///
|
|
91
|
+
/// can't cheaply prove a delete-bearing update is a duplicate; we conservatively
|
|
92
|
+
/// report it as advancing (record it). That can still double-record a pure-delete
|
|
93
|
+
/// retry, but it never drops a real deletion, which is the safe direction.
|
|
94
|
+
/// Assumes the update is already causally ready.
|
|
95
95
|
pub(crate) fn update_advances_doc(doc: &Doc, update_bytes: &[u8]) -> Result<bool, String> {
|
|
96
96
|
let update = yrs::Update::decode_v1(update_bytes).map_err(|e| e.to_string())?;
|
|
97
97
|
if !update.delete_set().is_empty() {
|
|
@@ -114,9 +114,10 @@ pub(crate) fn update_advances_doc(doc: &Doc, update_bytes: &[u8]) -> Result<bool
|
|
|
114
114
|
Ok(after != before)
|
|
115
115
|
}
|
|
116
116
|
|
|
117
|
-
/// True if the doc holds pending structs or a pending delete set
|
|
118
|
-
/// couldn't integrate because a dependency is missing.
|
|
119
|
-
///
|
|
117
|
+
/// True if the doc holds pending structs or a pending delete set: blocks that
|
|
118
|
+
/// couldn't integrate because a dependency is missing. Test-only: asserts the
|
|
119
|
+
/// causal-chain parking behavior in the unit tests below.
|
|
120
|
+
#[cfg(test)]
|
|
120
121
|
pub(crate) fn doc_has_pending(doc: &Doc) -> bool {
|
|
121
122
|
let txn = doc.transact();
|
|
122
123
|
txn.store().pending_update().is_some() || txn.store().pending_ds().is_some()
|
|
@@ -212,8 +213,8 @@ mod tests {
|
|
|
212
213
|
#[test]
|
|
213
214
|
fn update_advances_handles_a_dependent_diff_update() {
|
|
214
215
|
// A causally-pending diff (its structs depend on content the doc already
|
|
215
|
-
// has) reports an
|
|
216
|
-
// misread
|
|
216
|
+
// has) reports an empty state_vector() in isolation, which a naive check
|
|
217
|
+
// would misread as a no-op. Verify the trial-apply gets it right.
|
|
217
218
|
let doc = Doc::new();
|
|
218
219
|
let text = doc.get_or_insert_text("content");
|
|
219
220
|
text.insert(&mut doc.transact_mut(), 0, "a");
|
data/lib/yrb-lite.rb
CHANGED
data/lib/yrb_lite/version.rb
CHANGED