yrb-lite 0.1.0.beta4-aarch64-linux → 0.1.0.beta6-aarch64-linux
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +58 -17
- data/README.md +100 -146
- data/lib/yrb_lite/3.4/yrb_lite.so +0 -0
- data/lib/yrb_lite/4.0/yrb_lite.so +0 -0
- data/lib/yrb_lite/version.rb +1 -1
- data/lib/yrb_lite.rb +4 -4
- metadata +8 -23
- data/lib/yrb_lite/sync.rb +0 -580
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 2730b0d6a2bc31a36de1f842d7345659788d4c45968a895dfbbf7855ee6be21d
|
|
4
|
+
data.tar.gz: 9731717390fc3e45c983f20dd51429c214a6aaca85cb75b0e375a34390a785e0
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: '0913343e81dd6564a2ce23f058ec63cadb23917a105a86e430d859df77856054a8c0e4c02051a28a0dc8903c916f8c7ea60dd051ae8cbc022064d28539610680'
|
|
7
|
+
data.tar.gz: fb98a8a3e81ec173f812618dce6075a84f6516e4c7991c650c35835958465312b5b0380176a1e5628fdf302de8faa0ec05d3df64b1f78533c96820825ec4d2bc
|
data/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,53 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [0.1.0.beta6] - 2026-06-22
|
|
10
|
+
|
|
11
|
+
(yrb-lite core gem. The `yrb-lite-client` npm package ships these client changes as 0.1.2.)
|
|
12
|
+
|
|
13
|
+
### Added
|
|
14
|
+
|
|
15
|
+
- `yrb-lite-client`, the TypeScript client package for the yrb-lite
|
|
16
|
+
ActionCable/AnyCable protocol. It provides `ActionCableProvider`,
|
|
17
|
+
`YProtocolSession`, and the standalone `ReliableSync` delivery core.
|
|
18
|
+
|
|
19
|
+
### Changed
|
|
20
|
+
|
|
21
|
+
- Document delivery is ack-tracked by default in `yrb-lite-client`: document
|
|
22
|
+
frames use `{ update, id }`, acknowledgements use `{ ack }`, and pending
|
|
23
|
+
document updates stay queued until acked.
|
|
24
|
+
- The ActionCable protocol surface uses a single canonical document envelope:
|
|
25
|
+
`{ "update" => "<base64 frame>" }`.
|
|
26
|
+
- AnyCable awareness/presence uses an awareness-only whisper envelope,
|
|
27
|
+
`{ awareness: "<base64 awareness frame>" }`, while document frames stay on
|
|
28
|
+
the server persistence/ack path.
|
|
29
|
+
|
|
30
|
+
### Fixed
|
|
31
|
+
|
|
32
|
+
- Incoming protocol frames are validated before mutating documents or awareness
|
|
33
|
+
state, including trailing-byte rejection on the TypeScript client.
|
|
34
|
+
- Native/Rust protocol entry points reject wire client IDs that are unsafe for
|
|
35
|
+
JavaScript clients.
|
|
36
|
+
- `lib0` is declared as a direct runtime dependency of `yrb-lite-client`.
|
|
37
|
+
|
|
38
|
+
## [0.1.0.beta5] - 2026-06-18
|
|
39
|
+
|
|
40
|
+
### Changed
|
|
41
|
+
|
|
42
|
+
- **Breaking:** the ActionCable integration has been extracted into a separate
|
|
43
|
+
gem, [`yrb-lite-actioncable`](https://rubygems.org/gems/yrb-lite-actioncable).
|
|
44
|
+
`yrb-lite` is now a standalone y-crdt wrapper: CRDT documents, awareness, and
|
|
45
|
+
the y-websocket sync protocol primitives, with no Rails/ActionCable coupling
|
|
46
|
+
(mirrors the `y-rb` / `yrb-actioncable` split). The `base64` runtime
|
|
47
|
+
dependency moved with it.
|
|
48
|
+
|
|
49
|
+
### Migration
|
|
50
|
+
|
|
51
|
+
- Using `YrbLite::Sync`? Add `gem "yrb-lite-actioncable"` and change
|
|
52
|
+
`include YrbLite::Sync` to `include YrbLite::ActionCable::Sync`. The concern's
|
|
53
|
+
API is otherwise unchanged. If you only use `YrbLite::Doc`/`YrbLite::Awareness`,
|
|
54
|
+
nothing changes.
|
|
55
|
+
|
|
9
56
|
## [0.1.0.beta4] - 2026-06-18
|
|
10
57
|
|
|
11
58
|
### Changed
|
|
@@ -15,9 +62,7 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
|
15
62
|
`current_user`, `params`, request/connection-scoped accessors -- directly,
|
|
16
63
|
instead of plumbing them in through a thread-local. A non-Proc callable (an
|
|
17
64
|
object responding to `#call`) is still invoked with `#call` and its own
|
|
18
|
-
context.
|
|
19
|
-
document registry during a cold load or eviction, where no connection
|
|
20
|
-
instance exists, so they remain key-only. Existing block recorders that use
|
|
65
|
+
context. Existing block recorders that use
|
|
21
66
|
only the `(key, update)` arguments and lexically-scoped constants are
|
|
22
67
|
unaffected; the only behavioral change is `self` inside the block.
|
|
23
68
|
|
|
@@ -49,15 +94,13 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
|
49
94
|
|
|
50
95
|
- Reliable delivery (opt-in, client-driven). A client may tag a document update
|
|
51
96
|
with an `"id"`; the server replies `{ "ack": <id> }` once the update has been
|
|
52
|
-
|
|
97
|
+
durably recorded. This lets an
|
|
53
98
|
ack-aware client retain and retransmit an update until delivery is confirmed,
|
|
54
|
-
so an edit can't be silently lost on a flaky connection.
|
|
55
|
-
`"id"
|
|
56
|
-
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
causally-complete delta), plus a minimal reference client and an intensive
|
|
60
|
-
message-loss stress test.
|
|
99
|
+
so an edit can't be silently lost on a flaky connection. Clients that omit
|
|
100
|
+
`"id"` are still accepted, but their delivery is not ack-tracked.
|
|
101
|
+
- Demo coverage for reliable delivery with "sync-since-last-ack" framing (the
|
|
102
|
+
unacknowledged tail is sent as one merged, causally-complete delta), plus a
|
|
103
|
+
minimal reference client and an intensive message-loss stress test.
|
|
61
104
|
|
|
62
105
|
### Fixed
|
|
63
106
|
|
|
@@ -77,21 +120,19 @@ to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
|
77
120
|
native extension). The GVL is released during CRDT work so docs can run in
|
|
78
121
|
parallel on MRI.
|
|
79
122
|
- `YrbLite::Sync` ActionCable channel concern implementing the y-websocket
|
|
80
|
-
protocol (document sync plus awareness/presence).
|
|
81
|
-
the [`@y-rb/actioncable`](https://www.npmjs.com/package/@y-rb/actioncable)
|
|
82
|
-
browser provider, and accepts its `{ update: ... }` envelope and `{ m: ... }`.
|
|
123
|
+
protocol (document sync plus awareness/presence).
|
|
83
124
|
- A "record-before-distribute" mode via an `on_change` hook, so every change is
|
|
84
125
|
recorded durably before it's applied or relayed.
|
|
85
126
|
- Presence cleanup on disconnect, and idle-document eviction.
|
|
86
|
-
-
|
|
87
|
-
`sync_backend :store` (stateless, AnyCable-ready, multi-process).
|
|
127
|
+
- Store-backed ActionCable delivery for AnyCable and multi-process use.
|
|
88
128
|
- Hardening against bad input: malformed or multi-message frames are dropped
|
|
89
129
|
before processing or relay, and native panics are contained at the FFI
|
|
90
130
|
boundary.
|
|
91
131
|
- Precompiled native gems for common platforms (no Rust toolchain needed to
|
|
92
132
|
install) via the cross-gem workflow.
|
|
93
133
|
|
|
94
|
-
[Unreleased]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.
|
|
134
|
+
[Unreleased]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta5...main
|
|
135
|
+
[0.1.0.beta5]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta4...v0.1.0.beta5
|
|
95
136
|
[0.1.0.beta4]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta3...v0.1.0.beta4
|
|
96
137
|
[0.1.0.beta3]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta2...v0.1.0.beta3
|
|
97
138
|
[0.1.0.beta2]: https://github.com/jpcamara/yrb-lite/compare/v0.1.0.beta1...v0.1.0.beta2
|
data/README.md
CHANGED
|
@@ -9,26 +9,27 @@ documents.
|
|
|
9
9
|
|
|
10
10
|
```ruby
|
|
11
11
|
class DocumentChannel < ApplicationCable::Channel
|
|
12
|
-
include YrbLite::Sync
|
|
12
|
+
include YrbLite::ActionCable::Sync
|
|
13
13
|
|
|
14
14
|
def subscribed = sync_for(params[:id])
|
|
15
15
|
def receive(data) = sync_receive(data)
|
|
16
|
-
def unsubscribed =
|
|
16
|
+
def unsubscribed = sync_unsubscribed(params[:id])
|
|
17
17
|
end
|
|
18
18
|
```
|
|
19
19
|
|
|
20
|
-
On the browser, use the
|
|
21
|
-
|
|
20
|
+
On the browser, use the `yrb-lite-client` `ActionCableProvider`. Tiptap,
|
|
21
|
+
ProseMirror, and BlockNote all sync through the `Y.Doc` you pass in and the
|
|
22
|
+
provider's Awareness instance, unless you supply your own.
|
|
22
23
|
|
|
23
24
|
## What you get
|
|
24
25
|
|
|
25
|
-
- Thread-safe `Doc` and `Awareness`. You can share them
|
|
26
|
-
|
|
26
|
+
- Thread-safe Ruby wrappers for `Doc` and `Awareness`. You can share them
|
|
27
|
+
across Puma threads; native CRDT work runs with the GVL released.
|
|
27
28
|
- The y-websocket protocol (document sync plus awareness/presence) as a
|
|
28
29
|
one-include ActionCable concern.
|
|
29
|
-
-
|
|
30
|
-
-
|
|
31
|
-
goes out to anyone.
|
|
30
|
+
- Store-backed ActionCable/AnyCable delivery for multi-process deployments.
|
|
31
|
+
- Authoritative record-before-distribute semantics: each document change is
|
|
32
|
+
recorded durably before it goes out to anyone.
|
|
32
33
|
|
|
33
34
|
What it doesn't do: auth, read-only connections, rate limiting, webhooks,
|
|
34
35
|
metrics. Hocuspocus ships extensions for those; here you'd build them with
|
|
@@ -36,21 +37,26 @@ Rails.
|
|
|
36
37
|
|
|
37
38
|
## Testing
|
|
38
39
|
|
|
39
|
-
Ruby and Rust unit tests cover the core
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
The benchmark
|
|
43
|
-
welcome.
|
|
40
|
+
Ruby and Rust unit tests cover the core. CI also runs the npm client tests and a
|
|
41
|
+
Rails demo smoke slice against the real ActionCable stack. The demo includes
|
|
42
|
+
heavier local suites for hostile input, crash recovery, multi-browser editing,
|
|
43
|
+
AnyCable, and load testing. The benchmark note below is from a single laptop.
|
|
44
|
+
Issues and PRs are welcome.
|
|
44
45
|
|
|
45
46
|
## Install
|
|
46
47
|
|
|
47
48
|
```ruby
|
|
49
|
+
# Core CRDT + protocol primitives:
|
|
48
50
|
gem "yrb-lite"
|
|
51
|
+
|
|
52
|
+
# For the Rails/ActionCable server concern (YrbLite::ActionCable::Sync):
|
|
53
|
+
gem "yrb-lite-actioncable"
|
|
49
54
|
```
|
|
50
55
|
|
|
51
|
-
Requires Ruby 3.4 or newer.
|
|
52
|
-
3.4 and 4.0
|
|
53
|
-
|
|
56
|
+
Requires Ruby 3.4 or newer. The release workflow builds precompiled gems for
|
|
57
|
+
Ruby 3.4 and 4.0 across the supported Ruby platforms, with native smoke tests
|
|
58
|
+
on Linux x86_64 and macOS arm64. Installing from a matching platform gem needs
|
|
59
|
+
no Rust; a source build needs [Rust](https://rustup.rs).
|
|
54
60
|
|
|
55
61
|
To work on the gem itself:
|
|
56
62
|
|
|
@@ -128,44 +134,43 @@ send_to_peer(response) unless response.empty?
|
|
|
128
134
|
|
|
129
135
|
### ActionCable Integration
|
|
130
136
|
|
|
131
|
-
`YrbLite::Sync`
|
|
132
|
-
protocol (document sync +
|
|
137
|
+
`YrbLite::ActionCable::Sync` (from the `yrb-lite-actioncable` gem) is a channel
|
|
138
|
+
concern that implements the full y-websocket protocol (document sync +
|
|
139
|
+
awareness/presence) over ActionCable:
|
|
133
140
|
|
|
134
141
|
```ruby
|
|
135
142
|
# app/channels/document_channel.rb
|
|
136
143
|
class DocumentChannel < ApplicationCable::Channel
|
|
137
|
-
include YrbLite::Sync
|
|
144
|
+
include YrbLite::ActionCable::Sync
|
|
138
145
|
|
|
139
|
-
#
|
|
140
|
-
|
|
141
|
-
# on_save { |key, update| Document.find_by(key: key)&.update!(content: update) }
|
|
146
|
+
on_load { |key| MyStore.load(key) } # source of truth
|
|
147
|
+
on_change { |key, update| MyStore.append(key, update) } # durable record
|
|
142
148
|
|
|
143
149
|
def subscribed
|
|
144
150
|
sync_for params[:id]
|
|
145
151
|
end
|
|
146
152
|
|
|
147
153
|
def receive(data)
|
|
148
|
-
sync_receive(data)
|
|
154
|
+
sync_receive(data, params[:id])
|
|
149
155
|
end
|
|
150
156
|
|
|
151
157
|
def unsubscribed
|
|
152
|
-
|
|
158
|
+
sync_unsubscribed(params[:id])
|
|
153
159
|
end
|
|
154
160
|
end
|
|
155
161
|
```
|
|
156
162
|
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
163
|
+
The concern is store-backed. A handshake is answered from `on_load`; document
|
|
164
|
+
changes are checked against that durable state, recorded through `on_change`,
|
|
165
|
+
then broadcast. Nothing authoritative is kept in ActionCable process memory, so
|
|
166
|
+
AnyCable RPC workers, Puma workers, and separate dynos can all handle messages
|
|
167
|
+
for the same document as long as they share the same store and cable adapter.
|
|
162
168
|
|
|
163
|
-
`
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
it, the in-memory copy is the only one and stays put.
|
|
169
|
+
`on_load` and `on_change` are required. If either is missing, the channel fails
|
|
170
|
+
closed before it can acknowledge or broadcast edits. Presence is ephemeral:
|
|
171
|
+
awareness frames are relayed, and `yrb-lite-client` sends a best-effort
|
|
172
|
+
presence-removal frame on disconnect/pagehide, with the client-side awareness
|
|
173
|
+
timeout as the fallback for abrupt disconnects.
|
|
169
174
|
|
|
170
175
|
Incoming frames are validated as a single well-formed protocol message before
|
|
171
176
|
anything processes or relays them. Malformed, truncated, multi-message,
|
|
@@ -182,37 +187,19 @@ Broadcasts cross processes through the Action Cable adapter, so it needs to be a
|
|
|
182
187
|
real one (`redis` or `solid_cable`, not `async`). With that in place, a change
|
|
183
188
|
on one process reaches clients on all of them.
|
|
184
189
|
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
tells a process to skip its own.
|
|
190
|
-
|
|
191
|
-
A cold process (no copy yet) rebuilds from the durable store through `on_load`.
|
|
192
|
-
In authoritative mode the store is always current, since changes are recorded
|
|
193
|
-
before they go out. Record-before-distribute therefore holds across processes:
|
|
194
|
-
whichever process receives a change records it to the shared store before
|
|
195
|
-
anyone, anywhere, sees it.
|
|
190
|
+
Every process rebuilds document state from the durable store through `on_load`.
|
|
191
|
+
Because changes are recorded before broadcast, record-before-distribute holds
|
|
192
|
+
across processes: whichever process receives a change records it to the shared
|
|
193
|
+
store before anyone, anywhere, sees it.
|
|
196
194
|
|
|
197
195
|
`bun multiprocess.mjs` in the demo runs clients across two processes and checks
|
|
198
|
-
|
|
199
|
-
shared log.
|
|
200
|
-
|
|
201
|
-
##### AnyCable (`sync_backend :store`)
|
|
196
|
+
convergence, fresh reads on both, presence across processes, and one shared log.
|
|
202
197
|
|
|
203
|
-
|
|
204
|
-
block running in Ruby for each broadcast. AnyCable breaks both assumptions.
|
|
205
|
-
anycable-go delivers broadcasts outside Ruby, so the block never runs. Each RPC
|
|
206
|
-
gets a fresh channel instance, which means ivars set in `subscribed` are gone by
|
|
207
|
-
`receive`. And there's no fixed worker-to-document mapping to lean on.
|
|
208
|
-
|
|
209
|
-
`sync_backend :store` is the path for that: stateless per message, no warm
|
|
210
|
-
copy.
|
|
198
|
+
##### AnyCable
|
|
211
199
|
|
|
212
200
|
```ruby
|
|
213
201
|
class DocumentChannel < ApplicationCable::Channel
|
|
214
|
-
include YrbLite::Sync
|
|
215
|
-
sync_backend :store
|
|
202
|
+
include YrbLite::ActionCable::Sync
|
|
216
203
|
|
|
217
204
|
on_load { |key| MyStore.load(key) } # required: source of truth
|
|
218
205
|
on_change { |key, update| MyStore.append(key, update) } # required: record
|
|
@@ -227,6 +214,10 @@ end
|
|
|
227
214
|
- A handshake (SyncStep1) is answered from the store. Changes are recorded, then
|
|
228
215
|
broadcast. Nothing is held in Ruby between calls, so any worker can handle any
|
|
229
216
|
message.
|
|
217
|
+
- Document frames use the normal server path. Awareness/presence uses a
|
|
218
|
+
separate awareness stream with AnyCable `whisper: true`, so cursor traffic can
|
|
219
|
+
take the low-latency client-to-client path without bypassing document
|
|
220
|
+
durability.
|
|
230
221
|
- Pass `params[:id]` into `sync_receive`/`sync_unsubscribed` so the document key
|
|
231
222
|
survives AnyCable's per-command instances.
|
|
232
223
|
- The sender gets its own updates echoed back (no Ruby callback to filter them).
|
|
@@ -234,34 +225,31 @@ end
|
|
|
234
225
|
|
|
235
226
|
The demo checks this against a real anycable-go + RPC server
|
|
236
227
|
(`frontend/anycable_probe.mjs`, `anycable_concurrent.mjs`): liveness, the
|
|
237
|
-
|
|
228
|
+
yrb-lite client provider, cross-process reads, and concurrent convergence.
|
|
238
229
|
|
|
239
230
|
The wire format is the standard y-protocols binary messages, base64-encoded in
|
|
240
|
-
the ActionCable envelope.
|
|
241
|
-
`{ "update" => ... }
|
|
242
|
-
message per frame, so the off-the-shelf provider works with no custom client
|
|
243
|
-
code:
|
|
231
|
+
the ActionCable envelope. yrb-lite uses one canonical document envelope,
|
|
232
|
+
`{ "update" => ... }`, and sends one message per frame.
|
|
244
233
|
|
|
245
234
|
```js
|
|
246
|
-
import { createConsumer } from "@
|
|
247
|
-
import {
|
|
235
|
+
import { createConsumer } from "@anycable/web"
|
|
236
|
+
import { ActionCableProvider } from "yrb-lite-client"
|
|
248
237
|
|
|
249
|
-
const provider = new
|
|
238
|
+
const provider = new ActionCableProvider(ydoc, createConsumer(), "DocumentChannel", { id: docId })
|
|
239
|
+
provider.connect()
|
|
250
240
|
```
|
|
251
241
|
|
|
252
242
|
[`examples/actioncable-demo`](examples/actioncable-demo) is a full Rails + Tiptap
|
|
253
|
-
app using
|
|
243
|
+
app using the yrb-lite provider, with end-to-end tests.
|
|
254
244
|
|
|
255
|
-
####
|
|
245
|
+
#### Record Before Distribute
|
|
256
246
|
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
auditing or to guarantee nothing is distributed until it's stored, register an
|
|
260
|
-
`on_change` recorder:
|
|
247
|
+
Every document change is durably recorded before anyone else sees it. Register
|
|
248
|
+
an `on_change` recorder:
|
|
261
249
|
|
|
262
250
|
```ruby
|
|
263
251
|
class DocumentChannel < ApplicationCable::Channel
|
|
264
|
-
include YrbLite::Sync
|
|
252
|
+
include YrbLite::ActionCable::Sync
|
|
265
253
|
|
|
266
254
|
on_change do |key, update|
|
|
267
255
|
# Synchronous, durable write. `update` is the exact CRDT delta.
|
|
@@ -269,78 +257,43 @@ class DocumentChannel < ApplicationCable::Channel
|
|
|
269
257
|
end
|
|
270
258
|
|
|
271
259
|
def subscribed = sync_for(params[:id])
|
|
272
|
-
def receive(data) = sync_receive(data)
|
|
273
|
-
def unsubscribed =
|
|
260
|
+
def receive(data) = sync_receive(data, params[:id])
|
|
261
|
+
def unsubscribed = sync_unsubscribed(params[:id])
|
|
274
262
|
end
|
|
275
263
|
```
|
|
276
264
|
|
|
277
265
|
With `on_change` registered, a change is recorded before it goes anywhere. The
|
|
278
266
|
recorder writes the raw CRDT delta synchronously; only then is the change
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
it's applied. That's what makes the log authoritative. Replay the deltas onto a
|
|
282
|
-
fresh `Y.Doc` and you get the document back exactly.
|
|
267
|
+
broadcast. Replay the deltas onto a fresh `Y.Doc` and you get the document back
|
|
268
|
+
exactly.
|
|
283
269
|
|
|
284
270
|
If the recorder raises (say the store is down), the change is rejected: not
|
|
285
271
|
applied, not sent to anyone. The cost is a synchronous durable write per change,
|
|
286
272
|
which serializes that document's writes. Other documents use other locks and run
|
|
287
273
|
in parallel.
|
|
288
274
|
|
|
289
|
-
`on_change`
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
`on_change` to an fsync'd append-only log and checks, end to end, that the log
|
|
293
|
-
alone rebuilds the document.
|
|
275
|
+
The demo wires `on_change` to a durable Postgres-backed log by default, with an
|
|
276
|
+
fsync'd file log available via `STORE_KIND=file`, and checks end to end that the
|
|
277
|
+
log alone rebuilds the document.
|
|
294
278
|
|
|
295
279
|
#### Reliable delivery (acks)
|
|
296
280
|
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
state everyone *has*; they don't recover an update that never arrived.
|
|
302
|
-
|
|
303
|
-
yrb-lite closes that gap with an opt-in, client-driven acknowledgement. If an
|
|
304
|
-
incoming frame carries an `"id"`, the server replies `{ "ack": <id> }` once the
|
|
305
|
-
update has been **accepted** -- recorded in audit mode, applied in fast mode. A
|
|
306
|
-
causally-gapped update is not acked (it gets a resync instead), so the client
|
|
307
|
-
knows it hasn't landed yet.
|
|
281
|
+
yrb-lite document delivery is ack-tracked. Browser document updates carry an
|
|
282
|
+
`"id"`, and the server replies `{ "ack": <id> }` once the update has been
|
|
283
|
+
**durably recorded**. A causally-gapped update is not acked; the server sends a
|
|
284
|
+
resync request, and the client keeps the update queued until it lands.
|
|
308
285
|
|
|
309
286
|
```
|
|
310
|
-
client -> server { "
|
|
287
|
+
client -> server { "update": "<base64 update>", "id": 42 }
|
|
311
288
|
server -> client { "ack": 42 } # update accepted; safe to forget
|
|
312
289
|
```
|
|
313
290
|
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
This is entirely **self-gating**: stock clients send no `"id"`, so they never get
|
|
322
|
-
acks and behave exactly as before. Only a client that opts in by tagging its
|
|
323
|
-
frames participates.
|
|
324
|
-
|
|
325
|
-
Two client examples ship in the demo:
|
|
326
|
-
|
|
327
|
-
- [`frontend/reliable.mjs`](examples/actioncable-demo/frontend/reliable.mjs) — a
|
|
328
|
-
minimal reference client showing the raw mechanism (tag with an id, retain,
|
|
329
|
-
retransmit on a timer, drain on ack), with an end-to-end test that loses an
|
|
330
|
-
update mid-flight and recovers it purely by retransmit.
|
|
331
|
-
- [`frontend/provider/reliable_actioncable_provider.mjs`](examples/actioncable-demo/frontend/provider/reliable_actioncable_provider.mjs)
|
|
332
|
-
— the standard `@y-rb/actioncable` `WebsocketProvider`, vendored and augmented
|
|
333
|
-
for production use. It's a drop-in replacement that speaks the same protocol
|
|
334
|
-
and envelope, and adds reliability with **sync-since-last-ack** framing: rather
|
|
335
|
-
than retransmitting updates one by one, it keeps the unacknowledged local
|
|
336
|
-
updates in a queue and sends their *merge* as a single causally-complete delta,
|
|
337
|
-
with the id being the highest sequence in the batch (so one `{ ack: id }`
|
|
338
|
-
cumulatively confirms everything up to it). Because the whole unacked tail goes
|
|
339
|
-
as one self-contained delta, the server never sees an internal gap and never
|
|
340
|
-
has to round-trip a resync for a lost middle update — the next edit, or the
|
|
341
|
-
next timer tick, carries it. Awareness stays fire-and-forget; against a server
|
|
342
|
-
that doesn't implement acks it warns once and falls back to plain delivery; and
|
|
343
|
-
`reliable: false` opts out entirely. The demo's editor uses this provider.
|
|
291
|
+
`yrb-lite-client`'s `ActionCableProvider` handles this automatically. It keeps
|
|
292
|
+
the unacknowledged local document tail in a queue and sends the merged tail as a
|
|
293
|
+
single causally-complete delta. The id is the highest sequence in the batch, so
|
|
294
|
+
one `{ ack: id }` cumulatively confirms everything up to it. Because CRDT apply
|
|
295
|
+
is idempotent, a resend that already landed is a harmless no-op that just
|
|
296
|
+
re-acks. Awareness stays ephemeral and is not acked.
|
|
344
297
|
|
|
345
298
|
### User Awareness/Presence
|
|
346
299
|
|
|
@@ -376,24 +329,24 @@ message = awareness.encode_update(update_bytes)
|
|
|
376
329
|
|
|
377
330
|
## Thread Safety
|
|
378
331
|
|
|
379
|
-
|
|
380
|
-
`
|
|
381
|
-
|
|
332
|
+
`Doc` and `Awareness` are safe to share across Ruby threads. A `Doc` or
|
|
333
|
+
`Awareness` can be used concurrently from Puma workers, ActionCable connection
|
|
334
|
+
threads, or background jobs without external locking.
|
|
382
335
|
|
|
383
336
|
That comes from how the underlying types work, not from locking on top:
|
|
384
337
|
|
|
385
338
|
- `yrs::Doc` is `Send + Sync`. Every operation takes the document's internal
|
|
386
339
|
RwLock with blocking semantics (`read_blocking`/`write_blocking`), so
|
|
387
340
|
concurrent access serializes instead of erroring or corrupting state.
|
|
388
|
-
- `yrs::sync::Awareness` is
|
|
389
|
-
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
Each native method opens and closes its transaction
|
|
393
|
-
|
|
394
|
-
-
|
|
395
|
-
a yrs upgrade regressed
|
|
396
|
-
turning thread-unsafe.
|
|
341
|
+
- `yrs::sync::Awareness` is `Send` but not `Sync` in the current yrs version,
|
|
342
|
+
so the Ruby wrapper stores it in a `Mutex`. The mutex is always acquired
|
|
343
|
+
inside the no-GVL native section and released before Ruby runs again.
|
|
344
|
+
- The extension uses no `RefCell`-style runtime borrows that could panic under
|
|
345
|
+
re-entrancy. Each native method opens and closes its transaction or mutex
|
|
346
|
+
guard inside one call.
|
|
347
|
+
- Static assertions in `lib.rs` prove `Doc` and `Mutex<Awareness>` are
|
|
348
|
+
`Send + Sync`. If a yrs upgrade regressed either wrapper's thread-safety, the
|
|
349
|
+
gem would fail to compile instead of quietly turning thread-unsafe.
|
|
397
350
|
|
|
398
351
|
`test/thread_safety_test.rb` runs shared docs, the full sync handshake, fan-in
|
|
399
352
|
sync, and awareness state across 8 threads at once, and checks the interleaving
|
|
@@ -414,11 +367,12 @@ A slow operation also can't stall the VM. A thread applying a large update holds
|
|
|
414
367
|
the doc's write lock without holding the GVL, so other Ruby threads keep running
|
|
415
368
|
instead of queuing behind it.
|
|
416
369
|
|
|
417
|
-
Each method has the same shape: copy
|
|
418
|
-
yrs work
|
|
419
|
-
the GVL back, then build Ruby objects. No Ruby API is touched
|
|
420
|
-
|
|
421
|
-
deadlock. Panics in native code are caught and re-raised as Ruby
|
|
370
|
+
Each method has the same shape: copy Ruby byte strings first, drop the GVL, do
|
|
371
|
+
the yrs work while taking and releasing native locks entirely inside the
|
|
372
|
+
closure, take the GVL back, then build Ruby objects. No Ruby API is touched
|
|
373
|
+
without the GVL, and no native lock is held while reacquiring it, so the lock
|
|
374
|
+
order can't deadlock. Panics in native code are caught and re-raised as Ruby
|
|
375
|
+
exceptions.
|
|
422
376
|
|
|
423
377
|
## Message Type Constants
|
|
424
378
|
|
|
Binary file
|
|
Binary file
|
data/lib/yrb_lite/version.rb
CHANGED
data/lib/yrb_lite.rb
CHANGED
|
@@ -13,8 +13,8 @@ rescue LoadError
|
|
|
13
13
|
end
|
|
14
14
|
|
|
15
15
|
module YrbLite
|
|
16
|
-
# Error class is defined in Rust extension
|
|
17
|
-
|
|
18
|
-
#
|
|
19
|
-
|
|
16
|
+
# Error class is defined in the Rust extension.
|
|
17
|
+
#
|
|
18
|
+
# The ActionCable integration (YrbLite::ActionCable::Sync) lives in the
|
|
19
|
+
# separate `yrb-lite-actioncable` gem; require "yrb_lite/action_cable".
|
|
20
20
|
end
|
metadata
CHANGED
|
@@ -1,29 +1,15 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: yrb-lite
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.0.
|
|
4
|
+
version: 0.1.0.beta6
|
|
5
5
|
platform: aarch64-linux
|
|
6
6
|
authors:
|
|
7
7
|
- JP Camara
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-06-
|
|
11
|
+
date: 2026-06-22 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
|
-
- !ruby/object:Gem::Dependency
|
|
14
|
-
name: base64
|
|
15
|
-
requirement: !ruby/object:Gem::Requirement
|
|
16
|
-
requirements:
|
|
17
|
-
- - "~>"
|
|
18
|
-
- !ruby/object:Gem::Version
|
|
19
|
-
version: '0.2'
|
|
20
|
-
type: :runtime
|
|
21
|
-
prerelease: false
|
|
22
|
-
version_requirements: !ruby/object:Gem::Requirement
|
|
23
|
-
requirements:
|
|
24
|
-
- - "~>"
|
|
25
|
-
- !ruby/object:Gem::Version
|
|
26
|
-
version: '0.2'
|
|
27
13
|
- !ruby/object:Gem::Dependency
|
|
28
14
|
name: minitest
|
|
29
15
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -66,10 +52,10 @@ dependencies:
|
|
|
66
52
|
- - "~>"
|
|
67
53
|
- !ruby/object:Gem::Version
|
|
68
54
|
version: '1.2'
|
|
69
|
-
description: yrb-lite is a thread-safe Ruby binding over the Rust y-crdt (yrs) library
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
55
|
+
description: 'yrb-lite is a thread-safe Ruby binding over the Rust y-crdt (yrs) library:
|
|
56
|
+
CRDT documents, awareness/presence, and the y-websocket sync protocol primitives,
|
|
57
|
+
with the GVL released during native work so documents sync in parallel. The ActionCable/Rails
|
|
58
|
+
integration lives in the companion yrb-lite-actioncable gem.'
|
|
73
59
|
email:
|
|
74
60
|
- johnpcamara@gmail.com
|
|
75
61
|
executables: []
|
|
@@ -83,7 +69,6 @@ files:
|
|
|
83
69
|
- lib/yrb_lite.rb
|
|
84
70
|
- lib/yrb_lite/3.4/yrb_lite.so
|
|
85
71
|
- lib/yrb_lite/4.0/yrb_lite.so
|
|
86
|
-
- lib/yrb_lite/sync.rb
|
|
87
72
|
- lib/yrb_lite/version.rb
|
|
88
73
|
homepage: https://github.com/jpcamara/yrb-lite
|
|
89
74
|
licenses:
|
|
@@ -114,6 +99,6 @@ requirements: []
|
|
|
114
99
|
rubygems_version: 3.5.23
|
|
115
100
|
signing_key:
|
|
116
101
|
specification_version: 4
|
|
117
|
-
summary: Thread-safe Ruby bindings for y-crdt (Y.js)
|
|
118
|
-
|
|
102
|
+
summary: 'Thread-safe Ruby bindings for y-crdt (Y.js): documents, awareness, and the
|
|
103
|
+
y-websocket sync protocol'
|
|
119
104
|
test_files: []
|
data/lib/yrb_lite/sync.rb
DELETED
|
@@ -1,580 +0,0 @@
|
|
|
1
|
-
# frozen_string_literal: true
|
|
2
|
-
|
|
3
|
-
require "base64"
|
|
4
|
-
require "securerandom"
|
|
5
|
-
|
|
6
|
-
module YrbLite
|
|
7
|
-
# y-websocket protocol over ActionCable.
|
|
8
|
-
#
|
|
9
|
-
# Include this module in an ActionCable channel to sync Y.js documents
|
|
10
|
-
# (and awareness/presence) with browser clients. Messages are the standard
|
|
11
|
-
# y-protocols binary messages, base64-encoded in a JSON envelope:
|
|
12
|
-
#
|
|
13
|
-
# { "m" => "<base64 bytes>" } # client -> server
|
|
14
|
-
# { "m" => "...", "origin" => "<id>" } # server -> subscribers
|
|
15
|
-
#
|
|
16
|
-
# Example:
|
|
17
|
-
# class DocumentChannel < ApplicationCable::Channel
|
|
18
|
-
# include YrbLite::Sync
|
|
19
|
-
#
|
|
20
|
-
# on_load { |key| Document.find_by(key: key)&.content }
|
|
21
|
-
# on_save { |key, update| Document.find_by(key: key)&.update!(content: update) }
|
|
22
|
-
#
|
|
23
|
-
# # on_change blocks run in the channel instance's context, so instance
|
|
24
|
-
# # methods (current_user, params, ...) are available without plumbing:
|
|
25
|
-
# on_change { |key, update| Document.record!(key, update, by: current_user) }
|
|
26
|
-
#
|
|
27
|
-
# def subscribed
|
|
28
|
-
# sync_for params[:id]
|
|
29
|
-
# end
|
|
30
|
-
#
|
|
31
|
-
# def receive(data)
|
|
32
|
-
# sync_receive(data)
|
|
33
|
-
# end
|
|
34
|
-
#
|
|
35
|
-
# def unsubscribed
|
|
36
|
-
# sync_clear_presence
|
|
37
|
-
# end
|
|
38
|
-
# end
|
|
39
|
-
#
|
|
40
|
-
# The shared YrbLite::Awareness instances are safe to use from ActionCable's
|
|
41
|
-
# worker thread pool: the native types are Send + Sync and every operation
|
|
42
|
-
# releases the GVL, so concurrent clients sync in parallel.
|
|
43
|
-
module Sync
|
|
44
|
-
# Validated frame kinds from Awareness#message_kind. A frame only gets a
|
|
45
|
-
# non-DROP kind if it is exactly one well-formed message; anything
|
|
46
|
-
# malformed, truncated, multi-message, or unknown is dropped before it can
|
|
47
|
-
# be processed or relayed.
|
|
48
|
-
MSG_KIND_DROP = 0
|
|
49
|
-
MSG_KIND_SYNC_STEP1 = 1
|
|
50
|
-
MSG_KIND_UPDATE = 2
|
|
51
|
-
MSG_KIND_AWARENESS = 3
|
|
52
|
-
MSG_KIND_AWARENESS_QUERY = 4
|
|
53
|
-
|
|
54
|
-
def self.included(base)
|
|
55
|
-
base.extend(ClassMethods)
|
|
56
|
-
end
|
|
57
|
-
|
|
58
|
-
module ClassMethods
|
|
59
|
-
# Load persisted document state. Called once per key with (key);
|
|
60
|
-
# return a binary Y.js update (or nil for a fresh document).
|
|
61
|
-
def on_load(callable = nil, &block)
|
|
62
|
-
@on_load = callable || block if callable || block
|
|
63
|
-
@on_load
|
|
64
|
-
end
|
|
65
|
-
|
|
66
|
-
# Persist document state. Called with (key, update) after every
|
|
67
|
-
# message that modified the document.
|
|
68
|
-
def on_save(callable = nil, &block)
|
|
69
|
-
@on_save = callable || block if callable || block
|
|
70
|
-
@on_save
|
|
71
|
-
end
|
|
72
|
-
|
|
73
|
-
# Record every document change durably before it is applied or
|
|
74
|
-
# distributed (authoritative audit mode). Called synchronously with
|
|
75
|
-
# (key, update), where update is the exact CRDT delta, serialized per
|
|
76
|
-
# document so the recorded order is the apply order. If the block raises,
|
|
77
|
-
# the change is rejected: neither applied to the shared document nor
|
|
78
|
-
# broadcast to other subscribers.
|
|
79
|
-
#
|
|
80
|
-
# A block recorder runs in the *channel instance's* context, so it can
|
|
81
|
-
# call the channel's own methods (current_user, params, a per-connection
|
|
82
|
-
# Current.* accessor) directly, with no thread-local plumbing. (A non-Proc
|
|
83
|
-
# callable is invoked with #call instead, since it carries its own
|
|
84
|
-
# context.) on_change always fires from within sync_receive, unlike
|
|
85
|
-
# on_load/on_save, which can run context-free in the shared registry.
|
|
86
|
-
#
|
|
87
|
-
# Registering an on_change switches that channel onto the strict path
|
|
88
|
-
# (record, apply, broadcast). Without it, the default fast path applies
|
|
89
|
-
# and broadcasts, with an optional on_save snapshot.
|
|
90
|
-
def on_change(callable = nil, &block)
|
|
91
|
-
@on_change = callable || block if callable || block
|
|
92
|
-
@on_change
|
|
93
|
-
end
|
|
94
|
-
|
|
95
|
-
# Select the document backend:
|
|
96
|
-
# :memory (default): keep a warm in-memory replica per process and keep
|
|
97
|
-
# it current via a custom stream_from callback. Fast, but it assumes
|
|
98
|
-
# classic ActionCable (the callback runs in Ruby) and
|
|
99
|
-
# process<->document affinity.
|
|
100
|
-
# :store: stateless per message, with no warm replica and no custom
|
|
101
|
-
# stream callback. Handshakes and reads are served from the durable
|
|
102
|
-
# store (`on_load`); changes are recorded (`on_change`) and relayed.
|
|
103
|
-
# Works under AnyCable (broadcasts handled outside Ruby, no worker
|
|
104
|
-
# affinity) and across processes. Requires `on_load` and `on_change`.
|
|
105
|
-
def sync_backend(mode = nil)
|
|
106
|
-
@sync_backend = mode if mode
|
|
107
|
-
@sync_backend || :memory
|
|
108
|
-
end
|
|
109
|
-
end
|
|
110
|
-
|
|
111
|
-
# Call from `subscribed`. Streams broadcasts for this document and
|
|
112
|
-
# transmits the server's opening handshake (SyncStep1 + awareness).
|
|
113
|
-
def sync_for(key)
|
|
114
|
-
@sync_key = key.to_s
|
|
115
|
-
@sync_origin = SecureRandom.hex(8)
|
|
116
|
-
@sync_clients = [] # awareness client IDs seen on this connection
|
|
117
|
-
|
|
118
|
-
return sync_for_store_backed if self.class.sync_backend == :store
|
|
119
|
-
|
|
120
|
-
Sync.subscribe(@sync_key)
|
|
121
|
-
awareness = sync_awareness
|
|
122
|
-
|
|
123
|
-
stream_from sync_stream_name, coder: ActiveSupport::JSON do |payload|
|
|
124
|
-
sync_on_broadcast(payload)
|
|
125
|
-
end
|
|
126
|
-
|
|
127
|
-
# Opening handshake: SyncStep1 then the current awareness, each as its
|
|
128
|
-
# own single-message frame, so providers that parse one message per frame
|
|
129
|
-
# (e.g. @y-rb/actioncable) handle both. The client replies SyncStep2 to
|
|
130
|
-
# the SyncStep1, delivering its state to the server.
|
|
131
|
-
sync_transmit(awareness.sync_step1)
|
|
132
|
-
sync_transmit(awareness.encode_awareness_update)
|
|
133
|
-
end
|
|
134
|
-
|
|
135
|
-
# Call from `receive`. Applies the client's message, replies directly
|
|
136
|
-
# when the protocol calls for it, and relays document/awareness changes
|
|
137
|
-
# to the other subscribers.
|
|
138
|
-
#
|
|
139
|
-
# If an `on_change` recorder is registered, document changes take the
|
|
140
|
-
# strict authoritative path (record -> apply -> broadcast, serialized per
|
|
141
|
-
# document); otherwise the fast path is used.
|
|
142
|
-
#
|
|
143
|
-
# Reliable delivery (opt-in, client-driven): if the frame carries an "id",
|
|
144
|
-
# the server replies `{ "ack" => id }` once the update has been accepted
|
|
145
|
-
# (recorded in audit mode, applied in fast mode). A causally-gapped update
|
|
146
|
-
# is not acked -- it gets a resync instead -- so an ack-aware client knows
|
|
147
|
-
# to retransmit until the update lands. Stock clients send no "id", never
|
|
148
|
-
# get acks, and are completely unaffected.
|
|
149
|
-
def sync_receive(data, key = nil)
|
|
150
|
-
# Pass `key` (params[:id]) when your transport doesn't keep the channel
|
|
151
|
-
# instance alive across actions. Under AnyCable each RPC command gets a
|
|
152
|
-
# fresh channel, so instance variables set in `subscribed` are gone here.
|
|
153
|
-
@sync_key = key.to_s if key
|
|
154
|
-
|
|
155
|
-
# Accept both envelope keys: "m" (yrb-lite's own clients) and "update"
|
|
156
|
-
# (the @y-rb/actioncable browser provider).
|
|
157
|
-
m = data.is_a?(Hash) ? (data["m"] || data["update"]) : nil
|
|
158
|
-
return unless m.is_a?(String)
|
|
159
|
-
|
|
160
|
-
# Optional client-supplied id for reliable delivery (see sync_send_ack).
|
|
161
|
-
id = data.is_a?(Hash) ? data["id"] : nil
|
|
162
|
-
|
|
163
|
-
begin
|
|
164
|
-
bytes = Base64.strict_decode64(m)
|
|
165
|
-
rescue ArgumentError
|
|
166
|
-
return # not valid base64; ignore the frame and keep the connection
|
|
167
|
-
end
|
|
168
|
-
|
|
169
|
-
sync_send_ack(id, sync_dispatch(m, bytes))
|
|
170
|
-
end
|
|
171
|
-
|
|
172
|
-
# Route a decoded frame to the backend/path that handles it and return the
|
|
173
|
-
# outcome symbol (:recorded/:applied/:gap/:noop) used by the reliable-
|
|
174
|
-
# delivery ack. A dropped frame returns nil (never acked).
|
|
175
|
-
def sync_dispatch(encoded, bytes)
|
|
176
|
-
return sync_receive_store_backed(encoded, bytes) if self.class.sync_backend == :store
|
|
177
|
-
|
|
178
|
-
awareness = sync_awareness
|
|
179
|
-
kind = awareness.message_kind(bytes)
|
|
180
|
-
# Malformed / truncated / multi-message / unknown frames are dropped
|
|
181
|
-
# before they can be processed or relayed to other clients.
|
|
182
|
-
return if kind == MSG_KIND_DROP
|
|
183
|
-
|
|
184
|
-
sync_track_clients(awareness, bytes) if kind == MSG_KIND_AWARENESS
|
|
185
|
-
|
|
186
|
-
if kind == MSG_KIND_UPDATE && self.class.on_change
|
|
187
|
-
sync_apply_authoritative(awareness, encoded, bytes)
|
|
188
|
-
else
|
|
189
|
-
sync_apply_fast(awareness, encoded, bytes, kind)
|
|
190
|
-
end
|
|
191
|
-
end
|
|
192
|
-
|
|
193
|
-
# Call from `unsubscribed`. Clears the presence states this connection
|
|
194
|
-
# introduced and tells the other subscribers to drop those cursors, so a
|
|
195
|
-
# closed tab or dropped socket doesn't leave a ghost cursor behind until
|
|
196
|
-
# the client-side timeout reaps it.
|
|
197
|
-
def sync_clear_presence
|
|
198
|
-
return if @sync_clients.nil? || @sync_clients.empty?
|
|
199
|
-
|
|
200
|
-
removal = sync_awareness.remove_clients(@sync_clients)
|
|
201
|
-
@sync_clients = []
|
|
202
|
-
return if removal.empty?
|
|
203
|
-
|
|
204
|
-
sync_distribute(Base64.strict_encode64(removal))
|
|
205
|
-
end
|
|
206
|
-
|
|
207
|
-
# Call from `unsubscribed`. Clears this connection's presence and, when the
|
|
208
|
-
# last subscriber for the document leaves, persists and unloads it from
|
|
209
|
-
# memory (only when an `on_load` is configured to bring it back; otherwise
|
|
210
|
-
# the in-memory document is the only copy and is kept). Prevents a
|
|
211
|
-
# long-running server from accumulating every document it has ever served.
|
|
212
|
-
def sync_unsubscribed(key = nil)
|
|
213
|
-
@sync_key = key.to_s if key
|
|
214
|
-
return if self.class.sync_backend == :store # nothing cached per process
|
|
215
|
-
|
|
216
|
-
sync_clear_presence
|
|
217
|
-
saver = self.class.on_save
|
|
218
|
-
Sync.release(@sync_key, evictable: !self.class.on_load.nil?) do |awareness|
|
|
219
|
-
saver&.call(@sync_key, awareness.encode_state_as_update)
|
|
220
|
-
end
|
|
221
|
-
end
|
|
222
|
-
|
|
223
|
-
# The shared Awareness (document + presence) for this channel's key.
|
|
224
|
-
# Also useful for server-side reads, e.g.:
|
|
225
|
-
# sync_awareness.encode_state_as_update
|
|
226
|
-
def sync_awareness
|
|
227
|
-
Sync.awareness_for(@sync_key, self.class.on_load)
|
|
228
|
-
end
|
|
229
|
-
|
|
230
|
-
private
|
|
231
|
-
|
|
232
|
-
# Default path: apply the message, answer direct requests, relay
|
|
233
|
-
# state-changing messages to the other subscribers. Routing comes from the
|
|
234
|
-
# native `kind` (from Awareness#message_kind) rather than peeking at bytes.
|
|
235
|
-
# Document changes (SyncStep2, Update) and awareness get relayed; requests
|
|
236
|
-
# (SyncStep1, awareness-query) are answered above and not relayed. An
|
|
237
|
-
# optional on_save snapshot is taken after a document change.
|
|
238
|
-
#
|
|
239
|
-
# Returns an outcome symbol for the reliable-delivery ack: :applied when a
|
|
240
|
-
# document update was integrated and relayed, :gap when it was rejected for
|
|
241
|
-
# a resync, :noop for everything else (requests, awareness, empty updates).
|
|
242
|
-
def sync_apply_fast(awareness, encoded, bytes, kind)
|
|
243
|
-
# A document update that isn't causally ready (an earlier one was lost in
|
|
244
|
-
# transit) would relay an un-integrable change to peers and stall the
|
|
245
|
-
# replica. Drop it and ask the client to resync instead, which re-delivers
|
|
246
|
-
# the missing piece. See sync_apply_authoritative for the durable variant.
|
|
247
|
-
if kind == MSG_KIND_UPDATE
|
|
248
|
-
update = awareness.update_from_message(bytes)
|
|
249
|
-
# A no-op message (e.g. the empty SyncStep2 in an opening handshake)
|
|
250
|
-
# carries no change, so there's nothing to relay, persist, or ack.
|
|
251
|
-
return :noop unless update
|
|
252
|
-
|
|
253
|
-
unless awareness.update_ready?(update)
|
|
254
|
-
sync_request_resync(awareness)
|
|
255
|
-
return :gap
|
|
256
|
-
end
|
|
257
|
-
end
|
|
258
|
-
|
|
259
|
-
response = awareness.handle(bytes)
|
|
260
|
-
sync_transmit(response) unless response.empty?
|
|
261
|
-
|
|
262
|
-
return :noop unless [MSG_KIND_UPDATE, MSG_KIND_AWARENESS].include?(kind)
|
|
263
|
-
|
|
264
|
-
sync_distribute(encoded)
|
|
265
|
-
return :noop unless kind == MSG_KIND_UPDATE
|
|
266
|
-
|
|
267
|
-
sync_persist
|
|
268
|
-
:applied
|
|
269
|
-
end
|
|
270
|
-
|
|
271
|
-
# Authoritative path: record the change durably, then apply it to the
|
|
272
|
-
# shared document, then distribute it. The sequence runs under a
|
|
273
|
-
# per-document lock so changes are recorded in a single total order that
|
|
274
|
-
# matches the order they're applied, and nothing is distributed (or applied)
|
|
275
|
-
# before it has been recorded. If the recorder raises, the change is
|
|
276
|
-
# rejected (not applied, not broadcast) and the exception propagates, so the
|
|
277
|
-
# channel can surface it and the client can resync.
|
|
278
|
-
#
|
|
279
|
-
# Before recording, the update must be causally ready: every dependency it
|
|
280
|
-
# references must already be in the doc. If an earlier update was lost in
|
|
281
|
-
# transit, or its record failed, a later update arrives with a gap. Recording
|
|
282
|
-
# it would write a permanently-pending entry to the log -- one that can never
|
|
283
|
-
# be replayed until the missing update shows up. Such an update is rejected
|
|
284
|
-
# (not recorded, not applied, not relayed) and the client is asked to resync,
|
|
285
|
-
# which re-delivers the missing range as one causally-complete delta.
|
|
286
|
-
def sync_apply_authoritative(awareness, encoded, bytes)
|
|
287
|
-
recorder = self.class.on_change
|
|
288
|
-
|
|
289
|
-
outcome = Sync.lock_for(@sync_key).synchronize do
|
|
290
|
-
update = awareness.update_from_message(bytes)
|
|
291
|
-
# A no-op message (e.g. the empty SyncStep2 in a client's opening
|
|
292
|
-
# handshake) carries no change, so there's nothing to record or relay.
|
|
293
|
-
next :noop unless update
|
|
294
|
-
next :gap unless awareness.update_ready?(update)
|
|
295
|
-
|
|
296
|
-
sync_record_change(recorder, update) # durable write; raise to reject
|
|
297
|
-
awareness.apply_update(update) # only recorded changes reach the doc
|
|
298
|
-
sync_distribute(encoded) # ...and only then the wire
|
|
299
|
-
:recorded
|
|
300
|
-
end
|
|
301
|
-
|
|
302
|
-
case outcome
|
|
303
|
-
when :recorded then sync_persist
|
|
304
|
-
when :gap then sync_request_resync(awareness)
|
|
305
|
-
end
|
|
306
|
-
|
|
307
|
-
# Surface the outcome for the reliable-delivery ack: :recorded means the
|
|
308
|
-
# update is durably written (and will be acked); :gap triggered a resync
|
|
309
|
-
# (no ack); :noop carried no change.
|
|
310
|
-
outcome
|
|
311
|
-
end
|
|
312
|
-
|
|
313
|
-
# Ask this connection's client to resync: re-send SyncStep1 carrying the
|
|
314
|
-
# server's current (gap-free) state vector. The client replies SyncStep2
|
|
315
|
-
# with everything the server is missing, delivered as one causally-complete
|
|
316
|
-
# delta -- which heals the gap that triggered the resync.
|
|
317
|
-
def sync_request_resync(awareness)
|
|
318
|
-
sync_transmit(awareness.sync_step1)
|
|
319
|
-
end
|
|
320
|
-
|
|
321
|
-
# Reliable delivery: acknowledge an accepted update back to the sending
|
|
322
|
-
# connection. An ack-aware client tags each outgoing update with an "id"
|
|
323
|
-
# and retains it until the matching `{ "ack" => id }` returns, retransmitting
|
|
324
|
-
# on a timer or reconnect; idempotent CRDT apply makes resends free. We ack
|
|
325
|
-
# only when the client supplied an id (so stock clients are unaffected) and
|
|
326
|
-
# the update was actually accepted -- recorded in audit mode, applied in fast
|
|
327
|
-
# mode. A gapped update gets no ack (it got a resync), so the client keeps
|
|
328
|
-
# retransmitting until the missing range lands and the update can integrate.
|
|
329
|
-
def sync_send_ack(id, outcome)
|
|
330
|
-
return if id.nil?
|
|
331
|
-
return unless %i[recorded applied].include?(outcome)
|
|
332
|
-
|
|
333
|
-
# Braces are load-bearing: a bare hash would bind to transmit's `via:`
|
|
334
|
-
# keyword instead of its positional data argument.
|
|
335
|
-
transmit({ "ack" => id })
|
|
336
|
-
end
|
|
337
|
-
|
|
338
|
-
# Single broadcast point for both paths (and presence removal), so the
|
|
339
|
-
# relay semantics live in one place and tests can observe distribution.
|
|
340
|
-
# `origin` identifies the sending connection (don't echo to it); `pid`
|
|
341
|
-
# identifies the sending process (other processes apply it to their own
|
|
342
|
-
# replica; see sync_on_broadcast).
|
|
343
|
-
def sync_distribute(encoded)
|
|
344
|
-
ActionCable.server.broadcast(
|
|
345
|
-
sync_stream_name,
|
|
346
|
-
sync_envelope(encoded, "origin" => @sync_origin, "pid" => Sync.process_id)
|
|
347
|
-
)
|
|
348
|
-
end
|
|
349
|
-
|
|
350
|
-
# Transmit raw protocol bytes to this connection (base64, dual-key).
|
|
351
|
-
def sync_transmit(bytes)
|
|
352
|
-
transmit(sync_envelope(Base64.strict_encode64(bytes)))
|
|
353
|
-
end
|
|
354
|
-
|
|
355
|
-
# Build an outgoing envelope. We send the payload under both keys: "m"
|
|
356
|
-
# (yrb-lite's own clients) and "update" (the @y-rb/actioncable provider),
|
|
357
|
-
# so either client works against the same server.
|
|
358
|
-
def sync_envelope(encoded, extra = {})
|
|
359
|
-
{ "m" => encoded, "update" => encoded }.merge(extra)
|
|
360
|
-
end
|
|
361
|
-
|
|
362
|
-
# Handle a broadcast delivered by the cable adapter. With a multi-process
|
|
363
|
-
# adapter (Redis, solid_cable), it may have come from another server
|
|
364
|
-
# process. Keep this process's in-memory replica current with changes that
|
|
365
|
-
# originated elsewhere, then relay to this connection's browser.
|
|
366
|
-
def sync_on_broadcast(payload)
|
|
367
|
-
sync_apply_remote(payload["m"]) if payload["pid"] != Sync.process_id
|
|
368
|
-
transmit(payload) unless payload["origin"] == @sync_origin
|
|
369
|
-
end
|
|
370
|
-
|
|
371
|
-
# Apply a change that originated on another process to this process's
|
|
372
|
-
# replica, without re-recording it (the origin process already recorded it
|
|
373
|
-
# before broadcasting). The CRDT merge is idempotent and commutative, so a
|
|
374
|
-
# cold replica converges regardless of ordering, and applying from several
|
|
375
|
-
# local connections is harmless.
|
|
376
|
-
def sync_apply_remote(encoded)
|
|
377
|
-
return unless encoded.is_a?(String)
|
|
378
|
-
|
|
379
|
-
begin
|
|
380
|
-
bytes = Base64.strict_decode64(encoded)
|
|
381
|
-
rescue ArgumentError
|
|
382
|
-
return
|
|
383
|
-
end
|
|
384
|
-
|
|
385
|
-
awareness = sync_awareness
|
|
386
|
-
case awareness.message_kind(bytes)
|
|
387
|
-
when MSG_KIND_UPDATE
|
|
388
|
-
update = awareness.update_from_message(bytes)
|
|
389
|
-
awareness.apply_update(update) if update
|
|
390
|
-
when MSG_KIND_AWARENESS
|
|
391
|
-
awareness.handle(bytes)
|
|
392
|
-
end
|
|
393
|
-
end
|
|
394
|
-
|
|
395
|
-
# -- Store-backed (AnyCable-native) path --------------------------------
|
|
396
|
-
|
|
397
|
-
# Subscribe without a custom block, so AnyCable (which delivers broadcasts
|
|
398
|
-
# outside Ruby) relays them directly. Send the opening SyncStep1 built from
|
|
399
|
-
# the durable store. No warm replica is kept.
|
|
400
|
-
def sync_for_store_backed
|
|
401
|
-
stream_from sync_stream_name
|
|
402
|
-
sync_transmit(sync_load_doc.sync_step1)
|
|
403
|
-
end
|
|
404
|
-
|
|
405
|
-
# Stateless per message: no warm replica, no assumptions about which process
|
|
406
|
-
# owns a document. A client's SyncStep1 is answered from the store, document
|
|
407
|
-
# changes are recorded durably before relay and then broadcast, and
|
|
408
|
-
# awareness is relayed best-effort. Echoing back to the sender is harmless,
|
|
409
|
-
# since the CRDT apply is idempotent.
|
|
410
|
-
#
|
|
411
|
-
# Returns an outcome symbol for the reliable-delivery ack: :recorded when a
|
|
412
|
-
# document update was durably recorded and relayed, :gap when it was
|
|
413
|
-
# rejected for a resync, :noop for everything else.
|
|
414
|
-
def sync_receive_store_backed(encoded, bytes)
|
|
415
|
-
case Sync.codec.message_kind(bytes)
|
|
416
|
-
when MSG_KIND_SYNC_STEP1
|
|
417
|
-
result = sync_load_doc.handle_sync_message(bytes)
|
|
418
|
-
sync_transmit(result[2]) if result
|
|
419
|
-
:noop
|
|
420
|
-
when MSG_KIND_UPDATE
|
|
421
|
-
update = Sync.codec.update_from_message(bytes)
|
|
422
|
-
return :noop unless update
|
|
423
|
-
|
|
424
|
-
# Store mode keeps no warm replica, so to tell whether this update is
|
|
425
|
-
# causally ready we rebuild the doc from the store and check against it.
|
|
426
|
-
# That's an O(history) load per update (mitigated by snapshotting the
|
|
427
|
-
# store on the load path). A gappy update -- an earlier one was lost or
|
|
428
|
-
# its record failed -- is rejected and the client asked to resync,
|
|
429
|
-
# rather than written to the log as a permanently-pending entry.
|
|
430
|
-
doc = sync_load_doc
|
|
431
|
-
unless doc.update_ready?(update)
|
|
432
|
-
sync_transmit(doc.sync_step1)
|
|
433
|
-
return :gap
|
|
434
|
-
end
|
|
435
|
-
|
|
436
|
-
if (recorder = self.class.on_change)
|
|
437
|
-
sync_record_change(recorder, update) # record before relay
|
|
438
|
-
end
|
|
439
|
-
sync_distribute(encoded)
|
|
440
|
-
:recorded
|
|
441
|
-
when MSG_KIND_AWARENESS
|
|
442
|
-
sync_distribute(encoded)
|
|
443
|
-
:noop
|
|
444
|
-
else
|
|
445
|
-
:noop
|
|
446
|
-
end
|
|
447
|
-
end
|
|
448
|
-
|
|
449
|
-
# Build a fresh document from the durable store (on_load).
|
|
450
|
-
def sync_load_doc
|
|
451
|
-
doc = YrbLite::Doc.new
|
|
452
|
-
state = self.class.on_load&.call(@sync_key)
|
|
453
|
-
doc.apply_update(state) if state
|
|
454
|
-
doc
|
|
455
|
-
end
|
|
456
|
-
|
|
457
|
-
# Record the awareness client IDs carried by an incoming message (already
|
|
458
|
-
# known to be an awareness frame) so we can clear them when this connection
|
|
459
|
-
# closes.
|
|
460
|
-
def sync_track_clients(awareness, bytes)
|
|
461
|
-
awareness.awareness_client_ids(bytes).each do |id|
|
|
462
|
-
@sync_clients << id unless @sync_clients.include?(id)
|
|
463
|
-
end
|
|
464
|
-
end
|
|
465
|
-
|
|
466
|
-
def sync_stream_name
|
|
467
|
-
"yrb_lite:#{@sync_key}"
|
|
468
|
-
end
|
|
469
|
-
|
|
470
|
-
def sync_persist
|
|
471
|
-
return unless (saver = self.class.on_save)
|
|
472
|
-
|
|
473
|
-
saver.call(@sync_key, sync_awareness.encode_state_as_update)
|
|
474
|
-
end
|
|
475
|
-
|
|
476
|
-
# Invoke the on_change recorder. A block/proc runs in this channel instance's
|
|
477
|
-
# context (instance_exec) so it can reach the channel's own methods; a
|
|
478
|
-
# non-Proc callable is invoked with #call, since it carries its own context.
|
|
479
|
-
def sync_record_change(recorder, update)
|
|
480
|
-
args = [@sync_key, update]
|
|
481
|
-
recorder.is_a?(Proc) ? instance_exec(*args, &recorder) : recorder.call(*args)
|
|
482
|
-
end
|
|
483
|
-
|
|
484
|
-
# -- Shared document registry ------------------------------------------
|
|
485
|
-
|
|
486
|
-
@registry = {}
|
|
487
|
-
@locks = {}
|
|
488
|
-
@subscribers = Hash.new(0)
|
|
489
|
-
@registry_mutex = Mutex.new
|
|
490
|
-
|
|
491
|
-
class << self
|
|
492
|
-
# A stable id for this server process, stamped on every broadcast so
|
|
493
|
-
# other processes know to apply it to their replica and this process
|
|
494
|
-
# knows to skip its own. Survives for the life of the process.
|
|
495
|
-
def process_id
|
|
496
|
-
@process_id ||= SecureRandom.hex(8)
|
|
497
|
-
end
|
|
498
|
-
|
|
499
|
-
# A shared, stateless decoder for the store-backed path. message_kind and
|
|
500
|
-
# update_from_message only read their argument (they don't touch the
|
|
501
|
-
# instance's document), so one shared instance is safe across threads.
|
|
502
|
-
def codec
|
|
503
|
-
@codec ||= YrbLite::Awareness.new
|
|
504
|
-
end
|
|
505
|
-
|
|
506
|
-
# Get or create the shared Awareness for a key. Creation (including
|
|
507
|
-
# the on_load callback) is serialized under a mutex so concurrent
|
|
508
|
-
# subscribers can never observe two documents for one key; all
|
|
509
|
-
# subsequent operations run lock-free on the thread-safe native types.
|
|
510
|
-
def awareness_for(key, loader = nil)
|
|
511
|
-
@registry_mutex.synchronize do
|
|
512
|
-
@registry[key] ||= begin
|
|
513
|
-
awareness = YrbLite::Awareness.new
|
|
514
|
-
if loader && (state = loader.call(key))
|
|
515
|
-
awareness.apply_update(state)
|
|
516
|
-
end
|
|
517
|
-
awareness
|
|
518
|
-
end
|
|
519
|
-
end
|
|
520
|
-
end
|
|
521
|
-
|
|
522
|
-
# Per-document mutex serializing the authoritative record -> apply ->
|
|
523
|
-
# broadcast section, so a document's audit log is a single total order.
|
|
524
|
-
# Only briefly holds the registry mutex to fetch/create the lock; the
|
|
525
|
-
# durable write itself runs while holding only this per-key lock.
|
|
526
|
-
def lock_for(key)
|
|
527
|
-
@registry_mutex.synchronize { @locks[key] ||= Mutex.new }
|
|
528
|
-
end
|
|
529
|
-
|
|
530
|
-
# Count a new subscriber for a document.
|
|
531
|
-
def subscribe(key)
|
|
532
|
-
@registry_mutex.synchronize { @subscribers[key] += 1 }
|
|
533
|
-
end
|
|
534
|
-
|
|
535
|
-
# Drop a subscriber. When the last one leaves and the document is
|
|
536
|
-
# evictable (there's an on_load to bring it back, so unloading can't lose
|
|
537
|
-
# data), persist it via the given block and unload it from memory, so a
|
|
538
|
-
# long-running server doesn't accumulate every document and lock it has
|
|
539
|
-
# ever seen. Returns true if the document was evicted.
|
|
540
|
-
#
|
|
541
|
-
# The persist runs outside the registry lock (it may do I/O), and we
|
|
542
|
-
# re-check the subscriber count afterward: if someone reconnected while
|
|
543
|
-
# we were saving, eviction is aborted and the warm document is kept.
|
|
544
|
-
def release(key, evictable:)
|
|
545
|
-
awareness = @registry_mutex.synchronize do
|
|
546
|
-
@subscribers[key] -= 1 if @subscribers[key].positive?
|
|
547
|
-
next nil unless @subscribers[key].zero?
|
|
548
|
-
|
|
549
|
-
@subscribers.delete(key)
|
|
550
|
-
evictable ? @registry[key] : nil
|
|
551
|
-
end
|
|
552
|
-
return false unless awareness
|
|
553
|
-
|
|
554
|
-
yield awareness if block_given?
|
|
555
|
-
|
|
556
|
-
@registry_mutex.synchronize do
|
|
557
|
-
# A subscriber may have returned during the persist above.
|
|
558
|
-
next false unless @subscribers[key].zero?
|
|
559
|
-
|
|
560
|
-
@subscribers.delete(key)
|
|
561
|
-
@locks.delete(key)
|
|
562
|
-
!@registry.delete(key).nil?
|
|
563
|
-
end
|
|
564
|
-
end
|
|
565
|
-
|
|
566
|
-
def registry
|
|
567
|
-
@registry_mutex.synchronize { @registry.dup }
|
|
568
|
-
end
|
|
569
|
-
|
|
570
|
-
# Clear all documents (useful for testing).
|
|
571
|
-
def reset!
|
|
572
|
-
@registry_mutex.synchronize do
|
|
573
|
-
@registry = {}
|
|
574
|
-
@locks = {}
|
|
575
|
-
@subscribers = Hash.new(0)
|
|
576
|
-
end
|
|
577
|
-
end
|
|
578
|
-
end
|
|
579
|
-
end
|
|
580
|
-
end
|